Skip to content

Instantly share code, notes, and snippets.

@gibson042
Last active October 12, 2023 17:37
Show Gist options
  • Save gibson042/6ffa7e61e4b7acea02e5f47f33babd20 to your computer and use it in GitHub Desktop.
Save gibson042/6ffa7e61e4b7acea02e5f47f33babd20 to your computer and use it in GitHub Desktop.
ad hoc multi-engine JavaScript benchmarking
#!/bin/sh
# TODO start versioning
usage () { awk -v CMD="$0" '{ gsub(/\$0/, CMD); print }' <<'END_USAGE'
Usage: $0 [OPTION]... [--] SETUP VARIATIONS
Measures the performance of code variations in multiple ECMAScript implementations.
Options:
--number COUNT, -n COUNT
The count of timed iterations of each variation (defaults to 1000).
--repetitions COUNT
The number of times to repeat each variation _within_ a timed block for
purposes of ensuring a measurable duration, which should rarely need
configuration (defaults to 500).
--eshost-option OPTION, -E OPTION
Set a single option for use by eshost. May be specified multiple times.
Example: --eshost-option '-h V8,*XS*'
--init CODE, --init-file PATH
Code to execute once, before anything else (including setup, which runs once
per iteration). Each appearance is appended to results from previous ones.
Example: --init 'import "/path/to/module.js";'
Example: --init-file \
<(npx rollup -p @rollup/plugin-node-resolve -i /path/to/module.js -f iife)
--tmp
Write all code to a temporary file and run that rather than a command-line
argument.
--verbose, -v
--no-verbose, +v
Print (default: do not print) details such as the eshost command.
Exit Status:
0 Results
64 Bad command-line invocation
127 Dependent command not found
other values propagated from failed dependent commands (e.g., eshost)
Example:
$0 --eshost-option '-h V8,*XS*' 'const str=String.fromCharCode(0x40 + (i % 0x20)).repeat(1000)' '{
"str[0]": `const ch = str[0]; result = "@" <= ch && ch <= "A"`,
"str.charAt(0)": `const ch = str.charAt(0); result = "@" <= ch && ch <= "A"`,
"str.charCodeAt(0)": `const cu = str.charCodeAt(0); result = 0x40 <= cu && cu <= 0x41`,
}'
END_USAGE
exit ${1:-64} # EX_USAGE from BSD sysexits
}
fail () {
printf '%s\n\n' "$1" >&2
usage $2 >&2
}
# Global setup.
set -e # Exit if a command fails
# Process the command line.
tmp=
count=1000
repetitions=500
eshost_options=
init=
verbose=0
[ "$#" -eq 0 ] && usage
while [ "$#" -gt 0 ]; do
case "$1" in
--)
shift; break ;;
--number|-n*)
o="$1"; shift; [ ${#o} -gt 2 -a "${o#--}" = "$o" ] && set -- "${o#??}" "$@"
[ "$#" -gt 0 ] || fail "Missing argument: $o"
[ "$1" -gt 0 ] || fail "$o requires a positive count"
count="$1"
shift ;;
--repetitions)
o="$1"; shift
[ "$#" -gt 0 ] || fail "Missing argument: $o"
[ "$1" -gt 0 ] || fail "$o requires a positive count"
repetitions="$1"
shift ;;
--eshost-option|-E)
o="$1"; shift; [ ${#o} -gt 2 -a "${o#--}" = "$o" ] && set -- "${o#??}" "$@"
[ "$#" -gt 0 ] || fail "Missing argument: $o"
eshost_options="$eshost_options$1 "; shift ;;
--init)
o="$1"; shift
[ "$#" -gt 0 ] || fail "Missing argument: $o"
if [ ":$tmp" = : ]; then
init="$(printf '%s\n%s;' "$init" "$1")"
else
printf '%s;\n' "$1" >> "$tmp"
fi
shift ;;
--init-file)
o="$1"; shift
[ "$#" -gt 0 ] || fail "Missing argument: $o"
if [ ":$tmp" = : ]; then
init="$(printf '%s\n%s;' "$init" "$(cat "$1")")"
else
cat "$1" >> "$tmp"
printf ';\n' >> "$tmp"
fi
shift ;;
--tmp)
shift
if [ ":$tmp" = : ]; then
command -v mktemp >/dev/null 2>&1 || fail "Command not found: mktemp" 127
tmp="$(mktemp)"
trap 'if [ "$verbose" -gt 0 ]; then rm -fv "$tmp"; else rm -f "$tmp"; fi' 0
[ ":$init" = : ] || printf '%s\n' "$init" >> "$tmp";
init=
fi ;;
--verbose|-v* | --no-verbose|+v*)
o="$1"; shift; [ ${#o} -gt 2 -a "${o#--}" = "$o" ] && set -- "`printf %c "$o"`${o#??}" "$@"
[ "`printf %.5s "$o"`" = --no- -o `printf %c "$o"` = + ] && verbose=0 ||
verbose=`expr $verbose + 1` ;;
--help|-help|help|-?)
usage ;;
[+-]*)
fail "Unknown option: $1" ;;
*)
break ;;
esac
done
[ "$#" -eq 2 ] || fail 'Wrong argument count'
setup="$1"
variations="$2"
# Verify dependencies.
for dep in eshost; do
command -v $dep >/dev/null 2>&1 || fail "Command not found: $dep" 127
done
# Execute!
script="$(printf '
const run = (print => {
return (N, R, setup, variations) => {
if (!(0 < +N && +N < Infinity && N %% 1 === 0)) throw `invalid iteration count: ${N}`;
if (!(0 < +R && +R < Infinity && R %% 1 === 0)) throw `invalid repetition count: ${R}`;
const now = Date.now, splice = Function.prototype.call.bind(Array.prototype.splice);
const results = Object.entries(variations).map(([label, src]) => {
const result = Function(/* now, splice, */ `
const __now = arguments[arguments.length - 1](arguments, 0)[0];
let result, __T = 0;
for (let i = 0; i < ${N}; i++) {
const __addT = (calls => t => { if (calls++) throw "__addT called twice"; __T += t; })(0);
{ let __T;
${setup};
const __then = __now();
for (let j = 0; j < ${R}; j++) { let i, j, __then, __now;
{ ${src}; }
}
__addT(__now() - __then);
}
}
return [__T, result];
`)(now, splice);
const T = result[0];
if (result[1] === result) throw result; // references consequences of `src`
return `${label}: ${(N / T).toFixed(2)} ops/ms`;
});
let output = "";
for (let i = 0; i < results.length; i++) output += (i ? "\\n" : "") + results[i];
print(output);
};
})(print);
%s
run(`%s`, `%s`, `%s`, %s);\n' \
"$init" \
"$(printf '%s' "$count" | sed 's/\\/\\\\/g; s/`/\\`/g; s/[$]/\\u0024/g')" \
"$(printf '%s' "$repetitions" | sed 's/\\/\\\\/g; s/`/\\`/g; s/[$]/\\u0024/g')" \
"$(printf '%s' "$setup" | sed 's/\\/\\\\/g; s/`/\\`/g; s/[$]/\\u0024/g')" \
"$variations"
)"
if [ ":$tmp" = : ]; then
[ $verbose -ge 1 ] && set -x
eshost $eshost_options -x "$script"
else
printf '%s\n' "$script" >> "$tmp";
[ $verbose -ge 1 ] && set -x
eshost $eshost_options "$tmp"
fi

A shell script that wraps eshost for cross-implementation ECMA-262 benchmarking. Signature is subject to change, particularly for how to express variations (e.g., a future change will probably use distinct arguments rather than a single composite argument).

Usage: esbench [OPTION]... [--] SETUP VARIATIONS

Measures the performance of code variations in multiple ECMAScript implementations.

Options:
  --number COUNT, -n COUNT
    The count of timed iterations of each variation (defaults to 1000).

  --repetitions COUNT
    The number of times to repeat each variation _within_ a timed block for
    purposes of ensuring a measurable duration, which should rarely need
    configuration (defaults to 500).

  --eshost-option OPTION, -E OPTION
    Set a single option for use by eshost. May be specified multiple times.
    Example: --eshost-option '-h V8,*XS*'

  --init CODE, --init-file PATH
    Code to execute once, before anything else (including setup, which runs once
    per iteration). Each appearance is appended to results from previous ones.
    Example: --init 'import "/path/to/module.js";'
    Example: --init-file \
      <(npx rollup -p @rollup/plugin-node-resolve -i /path/to/module.js -f iife)

  --tmp
    Write all code to a temporary file and run that rather than a command-line
    argument.

  --verbose, -v
  --no-verbose, +v
    Print (default: do not print) details such as the eshost command.

Exit Status:
    0 Results
   64 Bad command-line invocation
  127 Dependent command not found
  other values propagated from failed dependent commands (e.g., eshost)

Example:
  esbench --eshost-option '-h V8,*XS*' 'const str=String.fromCharCode(0x40 + (i % 0x20)).repeat(1000)' '{
    "str[0]": `const ch = str[0]; result = "@" <= ch && ch <= "A"`,
    "str.charAt(0)": `const ch = str.charAt(0); result = "@" <= ch && ch <= "A"`,
    "str.charCodeAt(0)": `const cu = str.charCodeAt(0); result = 0x40 <= cu && cu <= 0x41`,
  }'

For checking that a string starts with a character from some range, XS is fastest with numeric comparison against charCodeAt(0) while V8 is pretty much equally fast regardless of approach.

$ esbench -E '-h V8,*XS*' 'const str = String.fromCharCode(0x40 + (i % 0x20)).repeat(1000);' '{
  "str[0]": `const ch = str[0]; result = "@" <= ch && ch <= "A"`,
  "str.charAt(0)": `const ch = str.charAt(0); result = "@" <= ch && ch <= "A"`,
  "str.charCodeAt(0)": `const cu = str.charCodeAt(0); result = 0x40 <= cu && cu <= 0x41`,
}'
#### Moddable XS
str[0]: 4.12 ops/ms
str.charAt(0): 6.33 ops/ms
str.charCodeAt(0): 7.94 ops/ms

#### V8
str[0]: 250.00 ops/ms
str.charAt(0): 250.00 ops/ms
str.charCodeAt(0): 250.00 ops/ms
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment