Use JOBS=max to speed up native node modules

JOBS.md

When npm installs native node modules it uses node-gyp to compile code. This is the seam node uses for targeting different operating systems, e.g. OS X, linux, Windows etc.

By default node-gyp compiles using one core and if you have more than one you probably want to utilize that power to speed up compile time. The way node-gyp handles this is by using the JOBS environment variable, which sets the jobs variable here. This piece of code then checks the value of jobs to determine how many cores to use.

Note that if the value of JOBS is max then all cores will be used. So lets try this on leveldown. First lets check that JOBS isn't set yet:

lms@ux301|01:34|~/src/leveldb-repos/leveldown (master) $ echo $JOBS

Ok lets install leveldown and time it. Everything has been pre-installed so we aren't measuring the time for downloading the modules:

lms@ux301|01:34|~/src/leveldb-repos/leveldown (master) $ time npm i
> leveldown@1.0.2 install /home/lms/src/leveldb-repos/leveldown
> node-gyp rebuild

make: Entering directory '/home/lms/src/leveldb-repos/leveldown/build'
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.17.0/db/builder.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.17.0/db/db_impl.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.17.0/db/db_iter.o
..
..
make: Leaving directory '/home/lms/src/leveldb-repos/leveldown/build'

real	0m21.028s
user	0m19.299s
sys	0m1.926s

Now lets set JOBS=max:

lms@ux301|01:35|~/src/leveldb-repos/leveldown (master) $ export JOBS=max
lms@ux301|01:37|~/src/leveldb-repos/leveldown (master) $ echo $JOBS
max

And lets time it again:

lms@ux301|01:37|~/src/leveldb-repos/leveldown (master) $ time npm i

> leveldown@1.0.2 install /home/lms/src/leveldb-repos/leveldown
> node-gyp rebuild

make: Entering directory '/home/lms/src/leveldb-repos/leveldown/build'
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.17.0/db/builder.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.17.0/db/db_impl.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.17.0/db/db_iter.o
..
..
make: Leaving directory '/home/lms/src/leveldb-repos/leveldown/build'

real	0m10.630s
user	0m32.511s
sys	0m2.808s

I'm using four cores so I expected something slightly better than a 50% improvement, but I'm just happy I can speed it up somewhat.

So, if you're on *nix, make sure to add export JOBS=max to your ~/.bashrc or ~/.profile or whatever you use :)

Magnus

permalink: https://github.com/nodejs/node-gyp/blob/8c4b0ffa503aa385f4186e028542dcedca2efeee/lib/build.js#L34

Just a small note, this isn't always a good idea. If you're building a native module like OpenCV on a low spec board like the Raspberry Pi, you will often run into "Segmentation Fault" errors.