Skip to content

Instantly share code, notes, and snippets.

@calvinmetcalf
Last active August 29, 2015 13:57
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save calvinmetcalf/9838383 to your computer and use it in GitHub Desktop.
Save calvinmetcalf/9838383 to your computer and use it in GitHub Desktop.

NPM as a build system

The reasons that you often don't need grunt have been done to death, so I don't feel any need to lend my voice except to agree. I've also seen Make thrown around as another alternative. Make is a fine program on *nix systems but for node.js it's missing a copule things,

  1. it is global, you have to specify ./node_modules/uglify-js/bin/uglifyjs instead of just uglifyjs.
  2. it is a pain to use on windows.

Now before we turn this into a windows/*nix flame war let me point out, I do not develop on windows, I despise developing on windows, the only time I ever voluntarily touch windows machines is when I'm helping my fiance fix her laptop. But that being said I work with people who use windows and I'd like to continue working with them as I have no interest in doing the heroic work they do with our legacy .NET apps.

Many people starting out with node are also on windows. While we may later be able to seduce them over to the *nix side, for now they are learning on the machine they have.

So back to your build step needing to work on windows, keep in mind not every node project on windows is simply going to get deployed to a linux box so simply using vagrant is not the answer. I've used IIS node to replace pieces of old asp apps, and I've used node to compile static assets that are then pushed somewhere.

And yes it is theoretically possible to get make installed on windows, but not easy, most of the suggestions I've seen either contain sourceforge links or references to Cygwin. These are not things that sane people want to do.

What instead?

For the 99% of projects that are a build step and some tests, you can use NPM as a build system, just put them in your package.json, when you run the script that way, npm makes sure your path has your node_module .bin folder is in the paths so if you use mocha to test you can just add

"test": "mocha -R nyan test/test.js"

to the scripts object in your package.json, and it will run tests on all your files that start with test. and are in the test folder, if you like tape you can do

"test": "node test/test.js | tspec"

jshinting can be done with

"lint": "jshint lib/*",

or you can combine it with your testing setup

"test": "jshint lib/* && node test/test.js | tspec"

both of these will look to .jshintrc for the rules, no need to specify them in a config.

karma can be run this way easily and testling can as well with browserify

"test": "browserify test/tests.js | testling"

For building browserify excels at using a package.json script, but if you must use [require.js] instead of putting the overly verbose and totally unnecessary 30 lines of config options to do something simple in with the build stuff, you can put it in a build.js file and then go

"build": "r.js -o build.js"

but if you don't like having to specify:

({
   baseUrl: '../src',
   mainConfigFile: '../src/script.js',
   wrap: false,
   name: '../node_modules/almond/almond',
   include:'script',
   insertRequire:['script'],
   out: '../dist/script.js',
   optimize: 'uglify2'
})

you can just have a package.json with:

{
  "main": "./src/script.js",
  "scripts": {
    "test": "something!!",
    "build": "browserify . | uglifyjs -mc > dist/script.js"
  }
}

(I'll get to the syntax in a minute) though to be fair since I often have a couple steps here I usually just separate it out into a couple steps:

{
  "main": "./src/script.js",
  "scripts": {
    "test": "something!!",
    "build-js": "browserify . > dist/script.js",
    "min": "uglifyjs -mc dist/script.js > dist/script.min.js",
    "build": "npm run build-js && npm run min"
  }
}

with browserify you can set specify other things in the package.json, for instance the "browser" key defines a map where you can exclude or substitute things from the build, when you specify the substitutions here instead of in the grunt file then other people browserifing a library that uses your lib as a dep also get the substitution applied, in other words, you need to be doing this anyway, why do it twice?

browserify-shim is a very helpful library in this respect (so helpful, grunt-browserify includes it but fails to mention it causing a confusing blending of the options in the docs).

If things get too busy we can just launch a batch script with

"build": "./build.sh"

over in build.sh same rules apply, it will work in windows if you stick to node commands (and git) you work cross platform, you can also call node scripts aka

"build": "node build.js"

in the some the examples I used | and && to separate commands on a single line and used > to out put a file. In shell commands && means the same thing as in JavaScript, it's a lazy and operator so command1 && command2 means run command1 and if it doesn't fail, run command2.

The other commands related to pipes/stdin/stdout. Commands can read from stdin and write to stdout and in node those are both streams, available at process.stdin and process.stdout, though another thing that outputs to stdout in node is console.log. By default unless it is piped anywhere your console will print stdout and thats why programs like browserify will print your bundle on the screen if you forget the -o option.

The | command (pipe) takes the output from one commands stdin and pipes it to the next commands stdin, so when you want to browserify a file and give it to testling, you can do

browserify . | testling

which takes the output of browserifing the current directory (the dot) and pipes it to testling.

or if you want to take the ugly tap output from tape and make it look nice, use

node test/test.js | tspec

which pipes it to tap-spec, or you could use

node test/test.js | tnyan

to pipe it to tap-nyan, because testing should be fun.

the > command means write stdout to a file, so you'll sometimes see people write

browserify . > bundle.js

which means write the bundle to bundle.js (yes I know you could also write -o)

then can be combined with pipes as well so to browserify and minify you can write

browserify . | uglifyjs -mc > dist/script.js

which browserifies it and then pipes the output to uglifyjs to minify it with the mangle and compress option.

N.B. uglifyjs is the command line program, but it's in NPM as uglify-js so you need to do npm install --save-dev uglify-js

The < operator reads the specified file as stdin so the following, while stupid, would work

browserify . > temp.js && uglifyjs -mc < temp.js > dist/script.js

in which we write our browserify output to a tempfile and then read it into stdin for uglify, a non contrived example would involve something like istanbul which always writes code coverage to a file and prints test results to stdout, so if you wanted to run coverage on a mocha script and then pipe it to coveralls then

istanbul cover ./node_modules/mocha/bin/_mocha --report lcovonly -- tests/*.mocha.js -R spec -t 5000 && coveralls < ./coverage/lcov.info

would do it

N.B. ./node_modules/mocha/bin/_mocha is the path to the real mocha binary, the mocha in your path (that we would get if we ran mocha) is a command line parser that will prevent istanbul from working right.

You will sometimes see people writing the above as

istanbul ... spec -t 5000 && cat ./coverage/lcov.info || coveralls

those people are wrong, cat is for concatenating files, not for reading them into stdout for you because cat isn't going to be available on windows.

WINDOWS!

There is no rm -rf so instead use rimraf which will remove a directory and all the stuff in it recursively so from our previous example

rimraf ./foo

will remove ./foo, and ./foo/bar, and ./foo/bar/baz and anything that might be in them.

N.B. Don't be an idiot with rimraf, if your careless you can delete everything, especially on windows.

To do batch copies ala grunt-copy you can use copyfiles which I wrote expressly as a replacement for grunt-copy, so to move all the JavaScript files from the vendor and src folder to the dist

copyfiles ./vendor/*.js ./src/*.js dist

but if you want them with out the outer vendor and src folder

copyfiles -f ./vendor/*.js ./src/*.js dist

if if you want to do a globstar

copyfiles -f './vendor/**/*.js' './src/**/*.js' dist

for operating system consistency you need the quotes around globstars, and for once that's not windows that is the problem, it's OSX with its out of date bash.

Conclusion

You probably don't need to use any build system for many node projects because npm can act as a build system and for the love of god don't use make because that's practically holding up a sign telling .NET programers to continue writing .NET programs and we want them publishing all the things to NPM.

Other articles on the subject

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment