First, please read On Maintaining a Native Node Module
Problem: There is no standard way to build + distribute prebuilt binaries for node modules with native addons (e.g. C++ bindings).
There's a thing called NODE_MODULE_VERSION
which is used to know if a native module was compiled for the correct version + flavor of node. If you are trying to load something compiled for e.g. iojs
from February 2015 into node
from March 2014 you will get an error something like Error: Module version mismatch. Expected 11, got 42.
when you try and run your program. node
uses numbers from 10 going up, and iojs
uses numbers from 40 going up. The NODE_MODULE_VERSION
gets bumped usually when V8 is upgraded, but sometimes for other reasons. It also gets even more complicated when you start talking about runtimes like Electron or Node-Webkit, but I won't get into that here. To my knowledge there is no list of all the NODE_MODULE_VERSION
s out there (EDIT here's one).
For maintainers, this results in a version matrix like this (duplicate this chart for every release they do of their native module):
11 | 12 | 43 | 44 | |
---|---|---|---|---|
linux-x64 | ||||
linux-x86 | ||||
darwin-x64 | ||||
win32-x64 | ||||
win32-ia32 | ||||
linux-armv7l | ||||
linux-armv6l |
In my opinion all native modules should offer prebuilt binaries. An example of a module that switched to this recently is leveldown
.
Previously, when you did npm install leveldown
, it required you to have a C++ build chain installed so it could compile leveldb. This means doing e.g. apt-get install build-essential
on Linux, downloading the multi-GB XCode from the Mac App Store on Mac, or downloading Visual Studio Free Edition on Windows. It's a big pain in the butt, especially for newbies (I have lots of empirical data on this from the nodeschool community).
Now, thanks to use of node-pre-gyp, npm looks at the leveldown releases for a prebuilt version matching the platform + architecture of the computer that is installing leveldown. If it finds a match, it just downloads the trusted prebuilt release. If it doesn't find a match, it falls back to the old method of compiling the sources.
To revisit our problem from above: Maintainers need to build every version in the above chart, for every version of their native addon!
Some people have hodge-podged together hacks that take advantage of Travis-CI (linux) or Appveyor (windows) to compile their modules and even upload them to GitHub Releases or S3. I'm proposing that someone in the community should take on the project of creating a dedicated, streamlined service that essentially does this, but focusing on making it easy specifically for native modules, because the tools that exist for this today aren't good enough, especially when you want to support building for platforms like Mac or ARM.
Feedback? Solutions? Use the comments below.
I shared some ideas in this io.js issue:
I suggest
node-gyp
,nan
, etc. should be automated, not a responsibility of module maintainers. Even if this hypothetical (yet possible) build tool existed, it also needs to be incorporated in node-core and / or npm.We should be able to determine whether or not a module is compatible on our system prior to install:
During the install process, the correct versions of
nan
,node-gyp
, or aprebuilt binary
would be installed independent the original maintainer'spackage.json
npm install
processnode-gyp
,node-gyp-prebuilt
, andnan
should be pulled into the node foundation as indispensable assetsSometimes, source code needs modification depending on the situation, and I don't know a good solution for that problem. Sometimes, the author just wrote a bad
package.json
file, and that would be trivial to modify. But IMO the weirdest scenario would be if an older / newer module works on your system, should npm inform us and let us choose the compatible alternative?There may be other solutions, but I believe all of them will require the cooperation of companies like npm and the node-foundation, along with contributions from the open source community (even people like myself who've never programmed a line of C but still know how to use
nan
andnode-gyp
)