Skip to content

Instantly share code, notes, and snippets.

@rbuckton
Last active January 31, 2020 20:04
Show Gist options
  • Save rbuckton/090b2b5d27079dec5639e79c072a289e to your computer and use it in GitHub Desktop.
Save rbuckton/090b2b5d27079dec5639e79c072a289e to your computer and use it in GitHub Desktop.
TypeScript Compiler API

Requirements/Goals:

  • Plugins must be loaded via tsc.
  • Plugins must be loaded in language service.
  • Plugins must be loaded explicitly (i.e. from tsconfig.json), and not automatically from node_modules.
  • Plugins may need custom configuration.
  • Plugins may need an activation event to support initialization.
  • Plugins may need a deactivation event to support cleanup.
  • Minifier plugins must to be able to perform a pre-emit walk of program and return custom transformations for emit.

Open Questions:

  • How should plugin configuration be applied?
    • Should we follow Babel's model for configuration?
      "plugins": ["plugin-1", ["plugin-2", { }]]
      
    • Or should we use a simple hash map?
      "plugins": { "plugin-1": true, "plugin-2": { } }
      
  • How should we enforce a minimum typescript-api version for plugins?
  • How much context do we need about a plugin to report useful diagnostics (i.e. % time spent in plugin, etc.)?
  • Should we be maintain a plugin blacklist for known problematic plugins (poor performance, unstable, etc.)?
  • Should we follow a naming scheme similar to Babel plugins (i.e. typescript-plugin-x or ts-plugin-x)?
// actually will be just the .d.ts from compiler, plus the following declarations:
export interface CompilerPluginContext {
readonly compilerHost: CompilerHost;
readonly compilerOptions: CompilerOptions;
readonly options: MapLike<any>; // plugin config options
}
export type CompilerPluginHookResult<T> = PromiseLike<T> | T;
export interface CompilerPluginModule {
activate?(context: CompilerPluginContext, args: CompilerPluginActivationArgs):
CompilerPluginHookResult<CompilerPluginActivationResult | void>;
preParse?(context: CompilerPluginContext, args: CompilerPluginPreParseArgs):
CompilerPluginHookResult<CompilerPluginPreParseResult | void>;
preEmit?(context: CompilerPluginContext, args: CompilerPluginPreEmitArgs):
CompilerPluginHookResult<CompilerPluginPreEmitResult | void>;
deactivate?(context: CompilerPluginContext): CompilerPluginHookResult<void>;
}
export interface CompilerPluginResult {
diagnostics?: ReadonlyArray<Diagnostic>;
}
export interface CompilerPluginActivationArgs {
}
/**
* An optional result that can be returned from the `CompilerPluginModule.activate` hook.
*/
export interface CompilerPluginActivationResult extends CompilerPluginResult {
}
export interface CompilerPluginPreParseArgs {
readonly rootNames: readonly string[];
readonly projectReferences: readonly ProjectReference[] | undefined;
}
export interface CompilerPluginPreParseResult extends CompilerPluginResult {
rootNames?: readonly string[];
projectReferences?: readonly ProjectReference[];
preprocessors?: readonly TransformerFactory<SourceFile>[];
}
export interface CompilerPluginPreEmitArgs {
/** The current `Program`. */
readonly program: BaseProgram;
/** The `SourceFile` that is about to be emitted, or `undefined` when emitting all outputs. */
readonly targetSourceFile: SourceFile | undefined;
/** A `CancellationToken` that can be used to abort an operation when running in the language service. */
readonly cancellationToken: CancellationToken | undefined;
}
/**
* An optional result that can be returned from the `CompilerPluginModule.preEmit` hook.
*/
export interface CompilerPluginPreEmitResult extends CompilerPluginResult {
customTransformers?: CustomTransformers;
}
export interface CompilerPluginDeactivationResult extends CompilerPluginResult {
}
// NOTE: All factory methods originally on `ts` have been moved to `NodeFactory`, which is accessible
// from the `factory` property of a transformation context. Existing factory methods exist for
// backwards compatibility when using the full TypeScript Language Service or TS Server API, but
// will not be accessible for Compiler Plugins.
export interface NodeFactory {
createNodeArray<T extends Node>(elements?: readonly T[], hasTrailingComma?: boolean): NodeArray<T>;
createNumericLiteral(value: string | number, numericLiteralFlags?: TokenFlags): NumericLiteral;
...
}
export interface TransformationContext {
// The `NodeFactory` you should use to create nodes.
// NOTE: Preprocessors (returned from the `preParse` hook) and Emit Transformers (returned
// from the `preEmit` hook) will be provided *different* `NodeFactory` instances and should not be conflated.
readonly factory: NodeFactory;
...
}
export interface BaseProgram {
// essentially the same as `Program`, except the `emit` method is not provided.
...
}
// NOTE: Import from "typescript" as normal, which is redirected to the active compiler.
import * as ts from "typescript";
// triggered when the extension is loaded
// - opportunity to inspect/mutate compiler options
// - opportunity to hook compiler host functionality
// - opportunity to return user-defined diagnostics
// - may be asynchronous (i.e. return a Promise)
export function activate(context: ts.CompilerPluginContext) {
// return { diagnostics };
}
// triggered immediately before parse begins
// - opportunity to customize the root names or project references
// - opportunity to inject custom preprocessors
// - opportunity to return user-defined diagnostics
// - may be asynchronous (i.e. return a Promise)
// NOTE: Custom preprocessors should only be used to augment the AST with additional type information
// and should not be used to change runtime semantics. This restriction is not currently enforced,
// but may be in the future.
export function preParse(context: ts.CompilerPluginContext,
{ rootNames, projectReferences }: ts.CompilerPluginPreParseArgs) {
// return { diagnostics, rootNames, projectReferences, preprocessors };
}
// triggered immediately before emit begins
// - opportunity to inject custom emit transformers
// - opportunity to return user-defined diagnostics
// - may be asynchronous (i.e. return a Promise)
export function preEmit(context: ts.CompilerPluginContext,
{ program, targetSourceFile, cancellationToken }: ts.CompilerPluginPreEmitArgs) {
// return { diagnostics, customTransformers };
}
// triggered when the extension is unloaded (i.e. in between compilations in watch mode)
// - opportunity to perform cleanup
// - may be asynchronous (i.e. return a Promise)
export function deactivate(context: ts.CompilerPluginContext) {
// return;
}
{
"name": "typescript-plugin-my-plugin",
"version": "1.0.0",
"main": "index.js",
"typescriptPlugin": {
"activationEvents": ["preEmit"],
"pluginDependencies": ["other-plugin"],
},
"dependencies": {
"typescript-api": "^3.4.0",
"other-plugin": "^1.0.0"
}
}
{
"compilerOptions": {
},
"plugins": {
"my-plugin": true,
"other-plugin": { "options": "here" }
}
}
@mprobst
Copy link

mprobst commented Mar 22, 2019

@alxhub I think you can program.getCompilerOptions().

I've prototyped putting tsickle on this. This all looks fine, though I have two questions:

With this API, it's unclear to me when a plugin can emit diagnostics into the array that preEmit returns. We could walk the AST once to produce diagnostics eagerly in the preEmit hook, and then a second time in the returned transformers without producing diagnostics (but carefully still avoiding the error conditions). That seems expensive though, and would also require somewhat awkward code that implements the same walking and error detection logic twice.

Another question is how a plugin would be notified of the actual emit happening for a (1) an individual source file and (2) the compilation unit as a whole. We need to emit extra outputs, some per file, some per compilation unit (i.e. once for a run of tsc over a tsconfig.json). With the API as is, I can see how we can write files through the compiler host, but I wouldn't know when is the right moment to do so. On preEmit seems wrong, within some transformer seems worse :-)

@rbuckton
Copy link
Author

rbuckton commented Nov 21, 2019

I've updated my compilerPluginModel branch with the latest changes for the compiler plugin prototype. Notable changes:

  • Added a preParse hook to allow plugins to change root names and project references, and add "preprocessor" transforms (intended to be used to inject type annotations and casts into existing source code).
  • All hooks can now be asynchronous (i.e. return a Promise).
  • Accessing the TypeScript API from a plugin is achieved using import * as ts from "typescript" syntax.
  • All factory functions normally used to create/update ASTs have been moved to a special NodeFactory object. You can access a NodeFactory from within a preprocessor/transformer by accessing the factory property of the provided TransformationContext:
    import * as ts from "typescript";
    function myTransformer(context: ts.TransformationContext) {
      // old: createIdentifier("foo")
      // new:
      const { factory } = context;
      factory.createIdentifier("foo")
    }
  • The parser now also uses a NodeFactory to create AST nodes.
  • Exceptions thrown by a plugin are wrapped in user-code diagnostics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment