Process goes something like this:
bundle update webpacker
rails webpacker:binstubs
yarn upgrade @rails/webpacker@4.0.0-pre.2 # or 'add' instead of upgrade
yarn upgrade webpack-dev-server@"'3.1.4' # or 'add' instead of upgrade
yarn add webpack-cli
bundle exec rails webpacker:install # don't do this! the scrpit seems to overwrite the @rails/webpacker version to 3.5.3