The idea behind grunt pipeline is that instead of hardcoding the inputs and outputs, a pipeline configuration will figure out which files are out of date, and only recompile the ones that are needed.
Let's look at this from a high-level, assuming input files like this:
app/
router.coffee
config.coffee
app.js
models/
post.js
comment.js
controllers/
posts.js
comments.js
routes/
posts.js
comments.js
Assume that we want to transpile the CoffeeScript files into JS, then transpile the JS files via ES6 module transpiler, and then concatenate all of the remaining JS files into a single app.js
.
Note: In practice, we wouldn't use tmp-X
. Instead, we would namespace by a SHA of whatever configuration inputs were used to build the pipeline, which could include bower/npm versions of packages, the SHA any .json
configuration, etc.
We will have to emulate the environment that multitasks are expecting, but it should be easy.
The best implementation strategy for the steps below is to build a full tree of all inputs and outputs before doing any work, and then pruning branches that don't need to be followed (see "Second Compile" below).
▼ <output>/app.js (contrib-concat)
▼ tmp-2/router.js (es6-module-transpiler)
▼ tmp-1/router.js (coffee)
- <input>/router.coffee
▼ tmp-2/config.js (es6-module-transpiler)
▼ tmp-1/config.js (coffee)
- <input>/config.coffee
▼ tmp-2/models/post.js (es6-module-transpiler)
- <input>/models/post.js
▼ tmp-2/models/comment.js (es6-module-transpiler)
- <input>/models/comment.js
▼ tmp-2/controllers/posts.js (es6-module-transpiler)
- <input>/controllers/posts.js
▼ tmp-2/controllers/comments.js (es6-module-transpiler)
- <input>/controllers/comments.js
▼ tmp-2/routes/posts.js (es6-module-transpiler)
- <input>/routers/posts.js
▼ tmp-2/routes/comments.js (es6-module-transpiler)
- <input>/routes/comments.js
Recursively walk the tree.
At every level, make a list of the direct children, and compare their actual SHAs with the SHAs in the manifest. If the SHAs match, prune the current node.
Create a manifest, which will be flushed to manifest.json
when we're done.
In each of the following steps, calculate the SHA of the input files and stick them in the manifest.
Input files: **/*.coffee
Output files: tmp-1/**/*.js
Multi-Task: grunt-coffee
Input files: {INPUT,tmp-1}/**/*.js
Output files: tmp-2/**/*.js
Multi-Task: grunt-es6-module-transpiler
Input files: tmp-2/**/*.js
Output files: OUTPUT/app.js
Multi-task: grunt-contrib-concat
Load the manifest.
For each of the above steps, remove input files whose SHAs haven't changed.
Again, the easiest implementation strategy would be to build up a tree of all inputs and outputs before doing anything else, and then pruning branches whose SHAs haven't changed.
pipeline:foo
task:target that has["coffee:bar", "transpile:bar", "concat:bar"]
defined.coffee:bar
,transpile:bar
,concat:bar
are expanded, analyzed, and all non-modified files are filtered out.__bar
target is dynamically written into the coffee config usinggrunt.config.set("coffee.__bar", {...})
that contains all the options fromcoffee.bar
but a rewrittenfiles
list.["coffee:__bar", "transpile:__bar", "concat:__bar"]
are run.__bar
targets are deleted.Also, if these tasks are to be run in a sub-shell, a
__Gruntfile.js
could be written out with just the necessary config and the tasks could be run from that.