Skip to content

Instantly share code, notes, and snippets.

@Gaurav0

Gaurav0/faster_broccoli_plugins.md

Last active June 30, 2018 11:03
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Gaurav0/c1eb3a00670eed28e57c2cf92d3f7668 to your computer and use it in GitHub Desktop.
Save Gaurav0/c1eb3a00670eed28e57c2cf92d3f7668 to your computer and use it in GitHub Desktop.
Download ZIP
Thoughts on how to write faster broccoli plugins
Raw
faster_broccoli_plugins.md
  1. Ensure your build method returns a promise, and any file system or network operations are done in an asynchronous fashion when possible.
  2. Do not share memory between instances of your broccoli plugin. This prevents it from being used in parallel. All data should be in local or instance variables or in the caches or the tmp directory.
  3. Cache the output of your plugin, if the input is the same and your plugin is n:1 from inputs to output, make sure you aren't rerunning your code.
  4. Note there are two caching modes, build and rebuild. Build is when you are starting from persistent caches only, while rebuild is when files that you are watching have changed. Both cases need to be optimized for.
  5. Caching is very important. Learn all about hashes and cache keys here:
  1. Please remember there is no need for your hash functions to be cryptographically secure. Choose fast hashing functions, like trivial, CRC32, MD5.
  2. If you are thinking of extending from:
  • broccoli-writer, consider broccoli-caching-writer
  • broccoli-filter, consider broccoli-persistent-filter
  1. Do not just use a directory name as the input tree to a broccoli plugin, use UnwatchedDir or WatchedDir from broccoli-source.
  2. Need to recursively traverse all files in a node?
  • Walk and create a list first, then process second, do not walk and process as you go.
  • Consider using node-walk or node-walk-sync instead of fs.walk or glob.sync.
  1. Update to the latest versions of any code you are using, especially other broccoli plugins.
  2. A single poorly performing broccoli plugin can greatly slow down an entire build. Make sure your plugin isn't the weak link in the speedy pipeline.
  3. Check out heimdall visualizer to see where your build is slowing down: https://github.com/rwjblue/heimdalljs-visualizer https://rwjblue.github.io/heimdalljs-visualizer/ (instructions?)
@Gaurav0
Copy link
Author

Gaurav0 commented Jun 29, 2018

Thank you @oligriffiths, will edit.

@nightire
Copy link

  1. I remember broccoli-unwatched-tree and broccoli-watched-tree are deprecated? Is https://github.com/broccolijs/broccoli-source a better choice?

@Gaurav0
Copy link
Author

Gaurav0 commented Jun 30, 2018

@nightire. Thanks. Updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment