Skip to content

Instantly share code, notes, and snippets.

@lencioni
Forked from trotzig/find-dead-js-modules.sh
Last active June 26, 2017 18:51
Show Gist options
  • Save lencioni/c375119a1015bf1550214b82b58ec2c6 to your computer and use it in GitHub Desktop.
Save lencioni/c375119a1015bf1550214b82b58ec2c6 to your computer and use it in GitHub Desktop.
This script will find javascript modules that aren't currently used in your application.
#!/bin/bash
# Make the script fail on the first error encountered.
set -euo pipefail
# Create a temp folder that we can use to store files in.
if [ "$(uname -s)" = "Darwin" ]; then
tmp_dir=$(mktemp -d -t find-dead-modules.XXXXXXXX)
else
tmp_dir=$(mktemp -d --tmpdir find-dead-modules.XXXXXXXX)
fi
# Make sure that the temp folder is removed on exit.
trap 'rm -rf "$tmp_dir"' EXIT INT QUIT TERM
touch "${tmp_dir}/all_modules.txt"
touch "${tmp_dir}/all_used_modules.txt"
# Find all javascript modules.
#
# We do this by listing all files, stripping out stuff that isn't part of the
# module "name" ("/index.js*", file suffix, path, etc). We also exclude a few
# utility modules that would otherwise be incorrectly flagged as dead.
echo "Finding all JavaScript files..."
find app/assets -name "**.js" -o -name "**.jsx" \
| sed 's/\/index\.js$//' \
| sed 's/\/index\.jsx$//' \
| xargs -I % basename % .js \
| xargs -I % basename % .jsx \
| sort -u \
>> "${tmp_dir}/all_modules.txt"
# Find all modules matching:
#
# //= require ...
# require('...')
# import ... from '...'
# import '...'
# javascriptPaths['...']
#
# We exclude certain folders and files from the grep. The most important ones to
# ignore are specs, otherwise we would treat modules that are required only by
# their tests as "alive".
echo "Finding uses of JavaScript files in JavaScript..."
pcregrep -r -M --no-filename --only-matching \
--buffer-size=1048576 \
--exclude-dir=".git" \
--exclude-dir="coverage" \
--exclude-dir="node_modules" \
--exclude-dir="public" \
--exclude-dir="spec" \
--exclude-dir="tmp" \
--include="\.jsx?$" \
"(//=\s*require\s+|require\(\s*'|import\s+'|from\s+'|javascriptPaths\[')\S+" . \
| cut -d\' -f2 \
| xargs -I % basename % .js \
| xargs -I % basename % .jsx \
>> "${tmp_dir}/all_used_modules.txt"
# Find all modules currently being used in Ruby via:
#
# add_js_package
# add_js_file
# add_hypernova_package
# render_react_component
# render_react_component_or_mock_payload
# javascript_path
# javascript_include_tag
# File.read
echo "Finding uses of JavaScript files in Ruby..."
pcregrep -r -M --no-filename --only-matching \
--buffer-size=1048576 \
--exclude-dir=".git" \
--exclude-dir="coverage" \
--exclude-dir="node_modules" \
--exclude-dir="public" \
--exclude-dir="spec" \
--exclude-dir="tmp" \
--include="\.e?rb$" \
"(add_js_package|add_js_file|add_hypernova_package|render_react_component|render_react_component_or_mock_payload|javascript_path|javascript_include_tag|File\.read)\b\s*[:\(]?['\"]?\S+" . \
| cut -d\' -f2 \
| cut -d\" -f2 \
| cut -d: -f2 \
| xargs -I % basename % .js \
| xargs -I % basename % .jsx \
>> "${tmp_dir}/all_used_modules.txt"
# Sorting needs to happen to make the `comm` call below work the way we want it
# to.
echo "Sorting list..."
sort -u "${tmp_dir}/all_used_modules.txt" \
--output="${tmp_dir}/all_used_modules.txt"
# Use `comm` to give us a list of all modules that aren't used.
echo "Finding unused files..."
UNUSED_MODULES=$(comm -23 \
"${tmp_dir}/all_modules.txt" \
"${tmp_dir}/all_used_modules.txt")
if [ -z "$UNUSED_MODULES" ]; then
exit 0
else
echo "These modules may not be used:"
echo ""
echo "$UNUSED_MODULES"
exit 1
fi
@trotzig
Copy link

trotzig commented Jun 21, 2016

Neat! I think you could dry up all the --exclude-dirs with a variable. Out of curiousity, how long does it take to run on your codebase?

@lencioni
Copy link
Author

lencioni commented Jun 21, 2016

I think you could dry up all the --exclude-dirs with a variable.

Yeah, I thought about doing this but decided to be lazy instead. If I need to make more modifications, I'll do it.

Out of curiousity, how long does it take to run on your codebase?

./find-dead-modules.sh  32.29s user 92.08s system 187% cpu 1:06.21 total

@trotzig
Copy link

trotzig commented Jun 21, 2016

Ah, that's annoyingly slow. It takes about two seconds on our codebase, so we have it wired up as an overcommit check.

@lencioni
Copy link
Author

Oh nice! I might be able to make it faster by combining greps and excluding more directories, but it is fast enough for my purpose right now. Also, too many false positives to set it up as a blocker to anything.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment