Learnings from 'ts-blank-space`

ts-blank-space-doc.md

Learnings from ts-blank-space

tags: TypeScript, type erasure, type stripping

ts-blank-space

As part of my work on the JavaScript Tooling team at Bloomberg I have implemented an experimental (not yet used in production) package to transform TypeScript into JavaScript using a somewhat novel approach.

This is a description of what I learned from implementing the idea. The source code will be open sourced soon - it just needs some regular IP approval.

The observation

The majority of TypeScript's type annotations can be erased by replacing them with whitespace.

export const mySet = new Set<string>();

can become:

export const mySet = new Set        ();

Why this is interesting

This is interesting because it means all remaining JavaScript characters are in their original positions.

• The transform does not need to generate a source-map describing the changes it made.
  • Specifically, it does not need to calculate or record "mappings".
  • The only thing needed in the sourcemap is a small O(1) filename mapping from the *.js to the *.ts
• The emitted code is debuggable even without sourcemaps
  • Crash stack coordinates retain their original accuracy
• The implementation of such a transform is small and easy to maintain
  • ts-blank-space is less than 700 lines of TypeScript (no code golf).
  • 7kb minified
  • Note: does not contain a parser and instead works off the TypeScript AST
    • The only dependency is TypeScript in order to reuse the parser
    • A port of ts-blank-space with a differet parser is possible, e.g. using babel-parser.
• The above results in best-in-class performance compared to other pure-JavaScript TypeScript transformations.
  • A native implementation should also be able to utilize the relative simplicity of this approach to get even higher performance
• The transformation can be trivially validated by asserting that all the tokens in the output match the same substring in the input.
• It enforces our "TypeScript = JavaScript + erasable types" approach directly in the tooling
  • Deciding on the supported type syntax has an easy-to-understand rule: 'can it be erased?'

The exceptions to the rule

There are two exceptions where 'ts-blank-space' changes the JavaScript.

New line in the return type of => arrow functions

The ) end of the arguments in an arrow function may need to be moved later to preserve semantics.

Given the following input:

let f = (): Array<
   string
> => [""];

'ts-blank-space' returns:

let f = (

) => [""];

This is because it is not valid JavaScript for there to be a newline between the end of the arguments and the =>.

ASI

To guard against ASI issues in the output ts-blank-space will sometimes add ; to the end of type-only statements.

Example input:

statementWithNoSemiColon
type Erased = true
("not calling above statement")

becomes:

statementWithNoSemiColon
;
("not calling above statement");

but

type NoSemi = true;//eol
"use strict";

is left as:

                   //eol    
"use strict";

New lines

New lines within erased types are preserved. To ensure the line position of the remaining JavaScript does not change

console.log("start");
interface Abc {



}
console.log("end");

'ts-blank-space' returns:

console.log("start");





console.log("end");

Unsupported syntax

The following is the TypeScript syntax which ts-blank-space does not transform:

• TSX
• enum
  • unless declare enum E { … }
• namespace, module
  • unless declare namespace N { … }
• class constructor parameter properties
  • e.g constructor(public x) { … }
• TypeScript's CommonJS syntax
  • export = …
  • import n = …
• Legacy type assertions
  • While they can sometimes be erased safely this is not always the case
  • e.g. () => <Type>{}
  • Code will need to switch to () => ({} as Type)

Recommended tsconfig:

• useDefineForClassFields: true to match the ECMAScript specification
• verbatimModuleSyntax: true to ensure type-only imports and exports are explicitly annotated with type

Fuller example

Input:

class C /**/< T >/*︎*/ extends Array/**/<T> /*︎*/implements I,J/*︎*/ {
//          ^^^^^                      ^^^     ^^^^^^^^^^^^^^
    readonly field/**/: string/**/ = "";
//  ^^^^^^^^          ^^^^^^^^
    static accessor f1;
    private f2/**/!/**/: string/*︎*/;
//  ^^^^^^^       ^    ^^^^^^^^

    method/**/<T>/*︎*/(/*︎*/this: T,/**/ a? /*︎*/: string/**/)/*︎*/: void/*︎*/ {
//            ^^^         ^^^^^^^^      ^     ^^^^^^^^         ^^^^^^
    }
}

Output:

class C /**/     /*︎*/ extends Array/**/    /*︎*/              /*︎*/ {
//          ^^^^^                      ^^^     ^^^^^^^^^^^^^^
             field/**/        /**/ = "";
//  ^^^^^^^^          ^^^^^^^^
    static accessor f1;
            f2/**/ /**/        /*︎*/;
//  ^^^^^^^       ^    ^^^^^^^^

    method/**/   /*︎*/(/*︎*/        /**/ a  /*︎*/        /**/)/*︎*/      /*︎*/ {
//            ^^^         ^^^^^^^^      ^     ^^^^^^^^         ^^^^^^
    }
}

Performance

Because all the JavaScript we need already exists exactly as needed in the original source string it can be re-used directly. The emit in ts-blank-space is a linear loop alternating between:

• Add JS
  • outStr += input.slice(endOfLastType + 1, startOfNextType)
• Add space
  • outStr += toSpace(input.slice(startOfNextType, endOfNextType+1))

A node --cpu-prof of ts-blank-space shows that the timeof calling tsBlankSpace(input) is dominated by creating the AST.

• 80% of the time is in ts.createSourceFile
• 18.5% is walking the AST collecting the positions of the type annotations
• 1.5% is spent concatenating the new string together

(the cpu profile also reports that the process spends 17% of it's time in the garbage collector)

A tool that can either generate the AST faster, or skip creating the AST and instead only collect the type-annotation positions while parsing should be able to go even faster.

Benchmark

Transforming checker.ts 10 times. Using Apple M2, Node.js v20.11.1. Comparing to Sucrase (3.34.0) and TypeScript (5.5.2).

hyperfine --warmup 3 \
   'node ./ts-blank-space.js checker.txt 10'\
   'node ./sucrase.js checker.txt 10'\
   'node ./ts.js checker.txt 10'

Benchmark 1: node ./ts-blank-space.js checker.txt 10
  Time (mean ± σ):      1.375 s ±  0.009 s    [User: 2.232 s, System: 0.126 s]
  Range (min … max):    1.361 s …  1.388 s    10 runs
 
Benchmark 2: node ./sucrase.js checker.txt 10
  Time (mean ± σ):      1.630 s ±  0.032 s    [User: 2.300 s, System: 0.223 s]
  Range (min … max):    1.570 s …  1.668 s    10 runs
 
Benchmark 3: node ./ts.js checker.txt 10
  Time (mean ± σ):      6.472 s ±  0.062 s    [User: 9.856 s, System: 0.297 s]
  Range (min … max):    6.379 s …  6.568 s    10 runs
 
Summary
  node ./ts-blank-space.js checker.txt 10 ran
    1.19 ± 0.02 times faster than node ./sucrase.js checker.txt 10
    4.71 ± 0.05 times faster than node ./ts.js checker.txt 10

code

import * as fs from "node:fs";
import blankSpace from "../index.js"

const input = fs.readFileSync(process.argv[2], "utf-8");
const count = Number(process.argv[3]) || 100;

for (let i = 0; i < count; i++) {
    const output = blankSpace(input);
}

import sucrase from "sucrase";
import * as fs from "node:fs";

const input = fs.readFileSync(process.argv[2], "utf-8");
const count = Number(process.argv[3]) || 100;

const options = {
    transforms: ["typescript"],
    disableESTransforms: true,
    preserveDynamicImport: true,
    filePath: "file.ts",
    sourceMapOptions: {
        compiledFilename: "file.ts"
   },
    production: true,
    keepUnusedImports: true,
};

for (let i = 0; i < count; i++) {
    const output = sucrase.transform(input, options);
}

import * as fs from "node:fs";
import ts from "typescript"

const input = fs.readFileSync(process.argv[2], "utf-8");
const count = Number(process.argv[3]) || 100;

const options = {
    fileName: "input.ts",
    compilerOptions: {
        target: ts.ScriptTarget.ESNext,
        module: ts.ModuleKind.ESNext,
        sourceMap: true,
    },
};

for (let i = 0; i < count; i++) {
    const output = ts.transpileModule(input, options).outputText;
}

Hello! This is a great idea.

By the way, what do you think about Function.prototype.toString()? Function.prototype.toString() outputs with whitespace preserved. Therefore, ts-blank-space and other tools have different runtime behavior.

Source (TypeScript code)

function fn() {
  const mySet = new Set<string>();
}
console.log(fn.toString());

The output of executing transformed code by ts-blank-space:

function fn() {
  const mySet = new Set        ();
}

The output of executing transformed code by other tools:

function fn() {
  const mySet = new Set();
}

This seems to mean that the user can observe how type annotations are removed via Function.prototype.toString().

Hello! This is a great idea.

Thanks!

This seems to mean that the user can observe how type annotations are removed via Function.prototype.toString().

I think that applies to many JavaScript tools. The return value of someFunction.toString() could change depending on the version of tsc/babel/rollup/terser.

Oh, you're right. I guess I was worrying too much :)