Reviewing front-end tooling changes to Wagtail

README.md

Reviewing front-end tooling changes to Wagtail

Here are the steps I take when reviewing tooling changes to Wagtail. Those steps help us assess whether the tooling changes are likely to result in UI changes for end users, thereby informing how much testing might be needed.

• Those commands assume some familiarity with command-line tools.
• I use those commands with zsh. Glob patterns might be slightly different for bash.
• Always test your work manually to some extent, even if the commands above seem to indicate no significant changes.

View all input and output files

It can be useful to have an overview of "front-end" files before and after compilation.

For input files:

tree wagtail/{*,contrib/*}/static_src/
# For the full picture – also do,
tree client/

For output files:

tree wagtail/{*,contrib/*}/static/

Compile from master

Whenever testing tooling changes, the first step is always to first get a "canonical" build from master.

# Make sure you’re up-to-date.
git pull upstream master
# Use the correct Node (and npm) version.
nvm use
# Now using node v10.15.2 (npm v6.4.1)
# Reset your node_modules if relevant.
rm -rf node_modules
npm i
# Remove existing output static files.
rm -rf wagtail/*/static/*
# Also do this for contrib apps, but not `table_block`.
rm -rf wagtail/contrib/wagtailmodeladmin/static/*
rm -rf wagtail/contrib/wagtailsettings/static/*
rm -rf wagtail/contrib/wagtailstyleguide/static/*
# Run the front-end build.
npm run dist
## Make a copy of the "master build" for future reference.
mkdir ../upstream-master-wagtail
cp -R wagtail/ ../upstream-master-wagtail

Compile from the branch to review

Checkout the branch with the tooling changes to review, and run the same process:

# Reset your node_modules if relevant.
rm -rf node_modules
npm i
# Remove existing output static files.
rm -rf wagtail/**/static/*
# Run the front-end build.
npm run dist

Compare files before/after

Now you can review the impact of the front-end tooling changes by comparing the build’s output from master and the one from the branch.

# List all static files in the "master" build
cd ../upstream-master-wagtail
ls wagtail/{*,contrib/*}/static/**/*.* > ../upstream-master-wagtail-list.txt
cd -
ls wagtail/{*,contrib/*}/static/**/*.* > ../wagtail-list.txt
diff ../upstream-master-wagtail-list.txt ../wagtail-list.txt

This can also be made into a one-liner without intermediary files if you have good sed or awk-fu, and know process substitution for your shell.

If the above command shows any differences, it’s for you to decide whether they are expected based on the tooling changes to review, and whether they require further testing.

Compare file-by-file

Based on the above, I generally proceed to creating a "diff script" to review individual files’ changes. Here are the rough steps:

touch diff.sh
chmod +x diff.sh
ls ../upstream-master-wagtail/wagtail/{*,contrib/*}/static/**/*.* > diff.sh

With this baseline, I then use my editor’s multiline cursors to have a script with all lines consisting of a diff like:

diff ../upstream-master-wagtail/wagtail/admin/static/wagtailadmin/css/core.css wagtail/admin/static/wagtailadmin/css/core.css

I generally proceeding to removing lines for sourcemaps (.map files), as well as font files and images if they aren’t relevant.

You can then run:

./diff.sh

If there is no output, congratulations, it looks like you don’t need to do any manual testing of the Wagtail admin! Have a cursory look around basic features of the admin just to be sure, in all supported browsers.

Reviewing changes

If the above command shows some changes, I generally try to narrow down which files have changed by manually editing diff.sh. It also helps to run CSS and JS code through Prettier so the changes are more human-readable.

cp -R ../upstream-master-wagtail ../upstream-master-wagtail-save
prettier --write '../upstream-master-wagtail-/**/*.{css,js}'
prettier --write 'wagtail/**/*.{css,js}'

You can then run the individual commands inside diff.sh, with a potentially more readable output.

Testing small changes

If the above highlights a few changes, it’s for you to decide whether it’s worthwhile to manually review those code changes and determine what parts of Wagtail to test, or test significant portions of Wagtail as a default. In any case, make sure to at least test basic functionality in all supported browsers.

Going further

• Consider using https://github.com/thibaudcolas/wagtail-tooling
• Always do manual testing in multiple browsers, following Wagtail’s testing guidelines.