If you're like me, every once in a while your CI process crosses a magic threshold and you decide you have to carve it back down to size. I ran just over 1000 TeamCity CI buildson a less-than-beefy VM with a basic React/TypeScript/ASP.Net Core project, trying different combinations of npm calls, flags, and alternatives.
Here's the options that performed the best.
Suggested Options
If you do not clean the workspace on every build:
- Best: Use
yarn installorpnpm install- 88% + 80% faster thannpm install - 2nd Best: Use
npm install --prefer-offline --no-audit- 15% faster thannpm install - Do not: Do not use
npm ci, see note below
npm install at 20 seconds, vs yarn and npm at under 4 seconds
If you clean the workspace on every build (or use a build service that doesn't cache environments):
- Best: Use
yarn installorpnpm install- 77% + 63% faster thannpm install - 2nd Best: Use
npm ci --prefer-offline --no-audit- 53% faster thannpm install
vanilla npm install at 270 seconds vs vs yarn and npm under 1/2 of that
The biggest gain on the npm calls is due to --prefer-offline, which tells npm to use locally cached packages when available, only calling the registry if it isn't already available.
What if I use multiple agents? Update package versions very frequently?
You will get poorer performance from the machine cache, try using a private registryor caching proxy. This way, once a developer or alternate build agent has downloaded thepackage once, it will be available closer to your other systems.
Example: Verdaccio (Note: I haven't personally tested this system yet)
Can I get the faster build times if I have to clean my build every time?
Use yarn or pnpm (kidding, not kidding)
Another option I've seen has been to archive node_modules to a common directory at theend of your CI build, then add a first step to restore it if the archive exists. This movesyou from the "clean" timings to the "dirty" timings, which is a 10x increase (then switch to yarn for another 8x).
The test data was produced using the following versions:
- node.js: 10.15.1
- npm: 6.4.1
- yarn: 1.13.0
- pnpm: 2.25.6
- TeamCity: 2018.2.2 (build 61245)
And the following project:
- github/BlogExample.Web/ClientApp: React 16.2 with TypeScript 3.3.3, Redux, Thunk, etc
One result that really stood out during my tests was how poorly npm ci performedon non-clean builds. TeamCity can nuke and recreate the workspace in seconds, whilerunning npm ci on a dirty folder was adding 50% (1 minute on this system) overrunning it on an empty folder (clean build).
Clean `npm ci` at 128 seconds, dirty `npm install` at 185 seconds?
One other interesting item was that the release notes for npm ci came with someincredible performance results over yarn and pnpm. I don't know whether thoseresults were a fluke, ci has gotten slower, everything else has gotten faster, or what. npm ci was visibly slower than both of those tools under the same circ*mstances andlatest versions.
Here are the details on the options I used:
npm --prefer-offline option
By default, verifies packages against the registry after a minimum cache timeto see if the package is still good. The release notes say this is just a 304 check,but this still includes the network latency and lookup time in the registry.That minimum cache time is 10 seconds. 🙄
prefer-offline tells npm to ignore the cache minimum time and just go ahead and usethe locally cached package if it's already been downloaded, without verifying it against the registry.
The official docs are light on this command now, but some details available here as well as in the release notes here
npm --no-audit option
The new feature to check packages for known vulnerabilities is great! However, the CIrun may not be the best place to run that (is anyone looking at those results?). Skippingthe audit step doesn't cut a lot off, but it does help.
I legitimately like the audit feature and think it should be checked regularly, more coming in a future post
npm --progress=false option
Older versions of npm produced a progress bar as it installed packages, slowing down some systems (Windows) with the console interaction.
I did not see a statisticallysignificant impact from this flag in up to date npm and assume I have one of theenvironment variables set on the build server that turns this off OR that theperformance has become less of a blocker.
npm install
Installs all packages specified in the package-lock.json file (or if that does not exist, package.json). By default it does not clean out existing packages in node_modules and performs a number of operations sequentially (like building the dependency tree of your millions of packages).
npm ci
"Continuous Integration" or "Clean Install", we may never know (both are used in the docs)
npm ci performs a clean install from your package-lock.json, with the goal of sort-ofreproducing a deterministic result. It will install exactly the versions you specify inyour package-lock, but the dependencies of those packages may be updated.
yarn
Yarn is an alternative to npm. It takes better advantage ofcaching, parallelizes operations (npm does a number of things sequentially), and producesmore repeatable, deterministic results than npm install.
pnpm
pnpm aims to be a faster, more efficient package management client. The main advantagesare that it links files from it's cache rather than copying them and does not use the flatstructure that npm has moved to in recent years.
pnpm supports the same commands as npm. Operations not directly implemented by pnpmare passed through to npm to execute.
From those options, I ran a minimum of 20 builds for each test case on "clean" and "dirty" working folders. "Clean" folders were cleared by TeamCity before the build started, "Dirty"folders came after at least one prior untracked run with the same package command.
24 Test Cases of npm install, npm ci, yarn, and pnpm
Test Case Labels:
- po: --prefer-offline
- na: --no-audit
- pf: --progress=false
Tests were run over the course of a week and a half. The build server was stable andrunning on a VM on a dedicated server, with no other active VMs or workload. Network performancewas consistent over this time and some cases (especially that long ci) were run over morethan one interval or for much longer sampling times. Time was measured both for the total run time as well as specifically just the npm step. In total, 1120 TeamCity builds were run.
Keep in mind, these won't be exactly the same on your environment, but they should be directionallycorrect enough to give you a solid start on your own environment.
