Thoughts on how to write faster broccoli plugins

faster_broccoli_plugins.md

1. Ensure your build method returns a promise, and any file system or network operations are done in an asynchronous fashion when possible.
2. Do not share memory between instances of your broccoli plugin. This prevents it from being used in parallel. All data should be in local or instance variables or in the caches or the tmp directory.
3. Cache the output of your plugin, if the input is the same and your plugin is n:1 from inputs to output, make sure you aren't rerunning your code.
4. Note there are two caching modes, build and rebuild. Build is when you are starting from persistent caches only, while rebuild is when files that you are watching have changed. Both cases need to be optimized for.
5. Caching is very important. Learn all about hashes and cache keys here:

• https://en.wikipedia.org/wiki/Hash_function
• https://en.wikipedia.org/wiki/Cache_(computing)

6. Please remember there is no need for your hash functions to be cryptographically secure. Choose fast hashing functions, like trivial, CRC32, MD5.
7. If you are thinking of extending from:

• broccoli-writer, consider broccoli-caching-writer
• broccoli-filter, consider broccoli-persistent-filter

8. Do not just use a directory name as the input tree to a broccoli plugin, use UnwatchedDir or WatchedDir from broccoli-source.
9. Need to recursively traverse all files in a node?

• Walk and create a list first, then process second, do not walk and process as you go.
• Consider using node-walk or node-walk-sync instead of fs.walk or glob.sync.

10. Update to the latest versions of any code you are using, especially other broccoli plugins.
11. A single poorly performing broccoli plugin can greatly slow down an entire build. Make sure your plugin isn't the weak link in the speedy pipeline.
12. Check out heimdall visualizer to see where your build is slowing down: https://github.com/rwjblue/heimdalljs-visualizer https://rwjblue.github.io/heimdalljs-visualizer/ (instructions?)

1. ensure your build method returns a promise, and any file system operations are done in an asynchronous fashion. Some things will have to be sync, for example, shelling out to a CLI process for example, so saying everything must be async is a little misleading.
2. good!
3. agreed, links to how to do this would be great. We need to improve this API, and I actually think making plugins opt-out (default is opted in) would be a good API change.
5. nice
6. nice
7. perhaps explain the difference between the caching modes (build vs rebuild)
8. perhaps an example might be useful here, when would someone do this?
9. this is cool, something that broccoli should really provide OOTB
10. yup
11. perhaps some links to how to enable the heimdall visualizer output https://github.com/rwjblue/heimdalljs-visualizer https://rwjblue.github.io/heimdalljs-visualizer/ would be good here

This is a good list, thanks for writing up.

(Edited to match numbers up - Gaurav)

Thank you @oligriffiths, will edit.

8. I remember broccoli-unwatched-tree and broccoli-watched-tree are deprecated? Is https://github.com/broccolijs/broccoli-source a better choice?

@nightire. Thanks. Updated.