Parallel execution with find and xargs

build.sh

#! /usr/bin/env bash

find ./src/bookmarklets -name "*.js"  -print0 | xargs -I FILE -0 -P "$(nproc)" npm run rollup -- --input "FILE" --file "output/FILE"

# EDIT: Per my latest comment - I now prefer this:
for bookmarklet in ./src/bookmarklets/**/*.js
do
  npm run rollup -- --input "$bookmarklet" --file "output/$bookmarklet" &
done
wait

I ran into a behavior of rollup that I didn't want. Specifically, it was trying to require other modules instead of inlining them.

After a few minutes of searching, I had not found a way to force inlining all code other than to disable multi-file input/output and process each file on its' own.

My first solution was to use find/--exec on each file I wanted to process.

It was taking 3+ seconds per file, which isn't a lot but also isn't super quick. So, I was able to process all files in parallel with find/xargs -P N.

Since my command consumes the filename in multiple spots, I needed to use the placeholder -I FILE.

https://serverfault.com/questions/193319/a-better-unix-find-with-parallel-processing
https://remysharp.com/2016/12/16/tricks-with-xargs
https://www.cyberciti.biz/faq/check-how-many-cpus-are-there-in-linux-system/

After some time to stew on this, and reading some alternatives, I feel that using a loop with background processes is more readable.

for thing in glob
do
  npm run doThingWithArg -- --file=thing & # & sends proc to bg
done
wait # waits for bg procs to complete