My lerna-ing

This article offers an overview about Lerna.

- Lerna
  - Commands
    - bootstrap
    - link
    - link convert
    - filter flags
  - Modes: fixed vs independent
  - Common mistakes

- Tips For Reducing The Bootstrap Time

- Appendix: NPM and the `require()` method

Conclusion

Lerna

Lerna is a tool that helps you manage monorepositories.

A monorepository is a repository containing multiple packages.

We rely on the definition of what a package is given by npm:

A package is a file or directory that is described by a package.json.

In short, a monorepository: One repository. Multiple packages.

From now on, I refer to lerna package to name those packages forming your monorepository.

// Monorepository with two lerna packages
packages
├── package-a
└── package-b

package-a and package-b are lerna packages.

In the following sections I will present an overview of what Lerna can do for you. I will also show you the monorepository tree before executing the lerna command and after it. In this way, you see the changes that it did to your node_modules folder.

Let’s start.

Commands

Lerna exposes a series of commands for helping you to work with a monorepository. We will review the most useful ones. If you want to see the full list of commands visit the lerna repository.

bootstrap

The bootstrap command is the most used one. It does multiple things:

  • npm install for each lerna package
  • symlink together lerna packages that are declared as dependencies
  • npm run prepublish for each lerna package
  • npm run prepare for each lerna package

It installs the packages declared in your package.json or package-lock.json for each repository.

The symlink happens for those lerna packages whose version matches the versions declared in package.json or package-lock.json.

Let’s see an example to understand this better.

I have two packages: lerna-ing-package-a and lerna-ing-package-b.

package-b/package.json
{
  "name": "lerna-ing-package-b",
  "version": "2.0.0",
  ...
}
package-a/package.json
{
  "name": "lerna-ing-package-a",
  "version": "2.0.0",
  "dependencies": {
    "lerna-ing-package-b": "2.0.0"
  }
}

package-a/package.json is using lerna-ing-package-b@2.0.0. The version we have in the monorepository for lerna-ing-package-b package is also version 2.0.0.

Let’s run the bootstrap command :

npx lerna bootstrap

lerna-ing-package-a has a dependency to "lerna-ing-package-b": "2.0.0".

The current working version of the lerna package lerna-ing-package-b is version": "2.0.0.

Since the dependency version matches the local version of the lerna package lerna-ing-package-b, the symlink takes place.

packages
├── package-a
│   ├── README.md
│   ├── __tests__
│   │   └── package-a.test.js
│   ├── lib
│   │   └── package-a.js
│   ├── node_modules
│   │   └── lerna-ing-package-b -> ../../package-b    // <----- Symlinked
│   ├── package-lock.json                             //       Uses lerna-ing-package-b
│   └── package.json                                  //       version 2.0.0
└── package-b
    ├── README.md
    ├── __tests__
    │   └── package-b.test.js
    ├── lib
    │   └── package-b.js
    ├── package-lock.json
    └── package.json    // <--- version: 2.0.0

Now, if we update the package-a/package.json to use an older version of lerna-ing-package-b

package-a/package.json
{
  "name": "lerna-ing-package-a",
  "version": "2.0.0",
  "dependencies": {
    "lerna-ing-package-b": "1.0.0"   // <---- Older version
  }
}

the symlink will not take place.

Let me run the bootstrap command for you:

npx lerna bootstrap

You end-up with the following project structure:

packages
├── package-a
│   ├── README.md
│   ├── __tests__
│   │   └── package-a.test.js
│   ├── lib
│   │   └── package-a.js
│   ├── node_modules
│   │   └── lerna-ing-package-b          // <------ No symlink
│   │       ├── README.md                //         Uses version 1.0.0
│   │       ├── lib                      //         of lerna-ing-package-b
│   │       │   └── package-b.js
│   │       └── package.json
│   ├── package-lock.json
│   └── package.json
└── package-b
    ├── README.md
    ├── __tests__
    │   └── package-b.test.js
    ├── lib
    │   └── package-b.js
    ├── package-lock.json
    └── package.json    // <--- version: 2.0.0

The bootstrap command offers the --force-local flag to symlink to the current lerna packages , ignoring the versions declared in the package.json or package-lock.json.

For instance, even if we refer to an older lerna package version as below:

package-a/package.json
{
  "name": "lerna-ing-package-a",
  "version": "2.0.0",
  "dependencies": {
    "lerna-ing-package-b": "1.0.0"   // <----- Try to use older version
  }
}

when we run the bootstrap command with the --force-local flag

npx lerna bootstrap --force-local

we end up with the following structure:

packages
├── package-a
│   ├── README.md
│   ├── __tests__
│   │   └── package-a.test.js
│   ├── lib
│   │   └── package-a.js
│   ├── node_modules
│   │   └── lerna-ing-package-b -> ../../package-b    // <----- Symlinked
│   ├── package-lock.json                             //       Uses local version
│   └── package.json                                  //       of lerna-ing-package-b
└── package-b                                         //       instead of
    ├── README.md                                     //       version 1.0.0
    ├── __tests__
    │   └── package-b.test.js
    ├── lib
    │   └── package-b.js
    ├── package-lock.json
    └── package.json    // <--- version: 2.0.0

As you see, the symlink took place, ignoring the version declared for the lerna package.

This is because we run the bootstrap command using the --force-local flag.

lerna link command symlinks together lerna packages that are dependencies of each other in your monorepository.

While lerna bootstrap does the symlinking , it also runs npm install, the prepublish hook and the prepare npm script command for each lerna repository .

If you are just interested in symlinking the lerna packages , lerna offers the lerna link command.

npx lerna link

To work with local versions of all your lerna packages then you can run:

npx lerna link --force-local

Be aware that it will ignore the specified versions declared in package.json or package-lock.json.

lerna link convert command edits the devDependencies for every lerna package.

It extracts the devDependencies declared in each lerna package and places them into the package.json devDependencies of the monorepository.

It additionally replaces lerna packages dependencies to relative file: specifiers.

This is the state of our monorepository before running lerna link convert command:

package.json
{
  "name": "lerning",
  "version": "1.0.0",
  "description": "package.json for the monorepository",
  "devDependencies": {
    "lerna": "^3.20.2"
  }
}
package-a/package.json
{
  "name": "lerna-ing-package-a",
  "version": "2.0.0",
  "scripts": {
    "gulp-version": "gulp --version",
    "test": "echo \"Package A\" && exit 0"
  },
  "dependencies": {
    "lerna-ing-package-b": "2.0.0"   // <---- version specifier
  },
  "devDependencies": {
    "gulp": "^4.0.2"
  }
}

Now, we run the lerna command :

lerna link convert

The monorepository package.json gets the devDependencies from the lerna packages.

packages
├── package-a
│   └── package.json  // <---  devDependencies are removed, changed to use file specifier
├── package-b
│   └── package.json  // <---  devDependencies are removed, changed to use file specifier
│
└── package.json      // <---  devDependencies from package-a
                      //       and package-b are moved here
package.json
{
  "name": "lerning",
  "version": "1.0.0",
  "description": "Repository for demo how lerna works",
  "dependencies": {                                       // <--- Create dependencies hash
    "lerna-ing-package-a": "file:packages/package-a",
    "lerna-ing-package-b": "file:packages/package-b"
  }
  "devDependencies": {                                    // <--- Move devDependencies
    "gulp": "^4.0.2",                                     //      from lerna packages
    "lerna": "^3.20.2"                                    //      to here
  },

}
{
  "name": "lerna-ing-package-a",
  "version": "2.0.0",
  "scripts": {
    "gulp-version": "gulp --version",
    "test": "echo \"Package A\" && exit 0"
  },                                            // <---- No devDependencies definition anymore
  "dependencies": {
    "lerna-ing-package-b": "file:../package-b"  // <---- Changed from specific version
  }                                             //       to `file:` specifier
}

This command might be useful when running in development environment and your lerna packages are using the same devDependencies. It can save time and space while bootstraping the monorepository.

Be aware that, since lerna packages changed their references from specific versioning to local paths, you must be careful to not publish the packages.

As mentioned in npmjs.com documentation when talking about using local paths :

This feature is helpful for local offline development and creating tests that require npm installing where you don’t want to hit an external server, but should not be used when publishing packages to the public registry.

filter flags

Lerna provides filter flags for having a more granular control of the lerna repositories where you apply the lerna commands.

An appropiate usage of the filters flags can reduce significantly bootstraping time.

Let’s see the most common ones.

scope flag

It applies a command to any lerna package name matching the given glob.

lerna bootstrap --scope lerna-ing-package-a

include-dependencies flag

It applies a command to a specific scope and its dependencies.

Suppose that you have a monorepo with package A, package B, package C and package D. Package A depends on package B. Package C depends on package D.

You are working on package A.

If you execute npx lerna bootstrap you would bootstrap all the packages.

But you dont want that. You are working on a feature for package A. There is no need to bootstrap all the packages.

To fasten the bootstraping time when working on package A, you execute:

lerna bootstrap --scope package-a --include-dependencies

This command bootstraps package A and package B but not package C and neither package D.

include-dependents flag

It applies a command to the specified scope and any lerna packages depending on it.

Suppose that you have a monorepo with package A, package B, package C and package D. Package A depends on package B. Package C depends on package D.

You need to do sensitive changes in package B. After you are done, you want to check that you did not break packages relying on package B.

If you execute lerna run test , you would run the npm script test command to package A, package B, package C and package D. Despite package C and package D do not depend on package B, you are still running the tests for them.

You want fast iteration. You want to see the green light in your tests as fast as possible.

lerna run test --scope package-b --include-dependents

This command runs the npm script test command to package B but also to package A. This is because package A depends on package B. You would not run the command for package C neither for package D since they are not dependent of package B.

Modes: fixed vs independent

This section is related to the publishing of your lerna packages .

Lerna offers two modes:

  • fixed: All lerna packages keep the same version.

  • independent: Every lerna package has its own version.

Common mistakes

In this section I will list common mistakes I see people do when using lerna.

Usage of npm install

The point that lerna allows you to run npm scripts on lerna packages does not mean that it is the solution to everything.

When something goes wrong with lerna , people tend to write the following:

npx lerna exec -- npm install

This will not symlink your lerna packages (which most of the times you want to).

Lerna offers commands for handling the bootstraping phase. Moreover, it offers also filter flags for a more granular control.

Tips For Reducing The Bootstrap Time

Here are some tips that might help you reducing the bootstraping time in development environments.

  • Use appropiate filter flags for reducing the repositories to bootstrap. Review ´–include-dependencies´ and ´–ignore´ flags.
  • Use ´lerna link convert´ for hoisting devDependencies.
  • Use --hoist flag for hoisting dependencies

Appendix: NPM and the require() method

npm install installs the dependencies and devDependencies set in your package-lock.json or package.json.

The dependencies are installed in the node_modules folder. This folder contains more packages than the specified in your package.json. This is due to an updated introduced in npm version 3. Some secondary dependencies (meaning the dependencies of the dependencies) might get installed in the directory of the primary dependency. This makes flatter node_modules folders.

When require() a module (not core module neither one starting with ‘/‘, ‘./‘ nor ‘../‘) a lookup will start at the parent directory. If it fails it will continue traversing up the directory looking for a node_modules containing the required dependency.

It will stop when the root of the file system is reached.

This is the reason why lerna link convert and the --hoist flag work. They extract packages to put them in the parent package. When you application refers to those packages, it will find those in the parent package instead of in the lerna packages.

This is fine for development environment but do not use when running your tests in the test environment. You might get a false negative.