Codemods & ASTs talk outline

README.md

Codemods & Underlying tech powering frontend today

What is a codemod?

• Definition
  • A tool for large-scale codebase refactors. In this case, it's a script that is run against the files you wish to change
  • When regex/find and replace isn't enough. Needing the context of the surrounding code and its meaning
• How? Abstract Syntax Trees!
  • Source-to-source compilation (transpilation), transforming ASTs.
  • Basic AST
• Codemod Example
  • React PropTypes to proptypes astexplorer
    • Covers the many different syntaxes for importing. Also adds prop-type import line, which find and replace couldn't do
  • My Codemod
    • Result after feeding this transformed file into create-element-to-jsx diff
    • PR explaining process
• Benefits of Codemods
  • Large refactors to your codebase become easy and reliable. String/regex replacement can accidentally alter unrelated code, whereas codemods target pieces of code with the awareness of the function of that code
  • Liberates library developers to make large-scale renaming changes, API changes, or import location changes since users can run a command to update.

Other AST uses

• ESLint.
  • no-console
• Prettier
  • ui-kit diff
  • Live demo (js and markdown)
• Babel
  • Adding your own language features
    • For functionality
    • For dev experience
  • Selectively choose your language and what it can do. Would avoid the whole Python 2/3 debacle.
• Uglify for minifying JS, complete with sourcemaps.

Not necessarily limited to JS

• 2to3 from Python also uses an AST to update from Python 2 to Python 3.
• yapf and gofmt use ASTs to act like Prettier for Python and Go
• However, these tools tie in the actual transformations with the code parsing, AST transformation API, and AST printing to code. So it's not extensible to create your own tool that acts on these ASTs.

State of the technology

• Lots of different AST parsers for the many flavors of JS (different language features, Flow, Typescript, etc)
• Relatively standard AST representation for JS, but the tools for working on the AST all have slightly different APIs
• Individual tools are self-contained. Each parsing file to AST, acting on the AST, and returning a new file. Multiple tools do this in a row (In our case, Prettier -> ESLint -> Babel). If we could pass a parsed AST from one tool to the next things could speed up
  • The new Parcel build tool does this
• Running a codemod takes a few more steps than expected. Need to clone repo with the transform, then run something like jscodeshift -t path/to/transform files/to/change/**/*.js.

Questions & Answers

• What codemods did I run?
  • For React 16 upgrade, I used the following facebook codemods: React-PropTypes-to-prop-types, react-to-react-dom, React-DOM-to-react-dom-factories react-codemod source
  • For the ui-kit factory to jsx, I used my own codemod followed by react codemod's create-element-to-jsx
• What did I have to do manually?
  • Cleanup of some syntax oddities in our own code. The more uniform the code you're transforming, the better.
• How common is it to write these custom codemods or babel plugins?
  • Writing your own codemods can be common when doing mass refactors on. Writing one is also common for libraries/tools introducing breaking changes.
  • Writing your own babel plugin is much more niche. In general, babel stage presets cover your needs for modern language features.