Choco Curry

Choco Curry.md

Context

• Choco is using AWS Lambda fully, we don't have any bare metal server or virtual machine.
• AWS AppSync + AWS Lambda + AWS DynamoDB are our main tech stack.
• AWS CloudFormation is the only tool to manage all AWS resources.

Pain Point

There are not perfect tech, especially for the new one. In our daily dev life, we have two main issues.

Groundhog day

Here is a small sample of our code.

/**
 *
 * @lambda getBook
 * @gqlType Query
 */
export const handler = async ({
  id,
}: {
  id?: string;
}): Promise<Book> => {
  return {
    name: 'Fox',
    id,
    author: {
      id: 'author-id',
      name: 'book name',
    },
    tags: ['sci-fi'],
  };
};

But in order to let our code alive, we have to copy and paste those boilerplate code.

AWSTemplateFormatVersion: 2010-09-09
Transform: AWS::Serverless-2016-10-31
Resources:
  GetBookFunction:
    Type: AWS::Serverless::Function
    Properties:
      Runtime: nodejs14.x
      CodeUri: GetBook
      Handler: index.handler
  GetBookResolver:
    Type: AWS::AppSync::Resolver
    Properties:
      TypeName: Query
      FieldName: GetBook
      ApiId: !GetAtt GraphQLAPI.ApiId
      DataSourceName: !GetAtt GetBookDataSource.Name
      RequestMappingTemplateS3Location: ../resolver/Resolvers.generic.request
      ResponseMappingTemplateS3Location: ../resolver/Resolvers.generic.response
  GetBookDataSource:
    Type: AWS::AppSync::DataSource
    Properties:
      Type: AWS_LAMBDA
      ServiceRoleArn: !GetAtt GraphQLRole.Arn
      ApiId: !GetAtt GraphQLAPI.ApiId
      Name: GetBookHandler
      LambdaConfig:
        LambdaFunctionArn: !GetAtt GetBookFunction.Arn

In most of case, we just need to change the Resouce name, the code location and adjust a bit permission policies. Every lambda is looks like the same. But this not only one problem. Another one is this.

Multiple Source of truth

In order to expose this Lamdba to AWS AppSync, we need to register our Lambda to API gateway. So we have to define the GraphL Schema as follows:

type Author {
  id: String!
  name: String!
}

type Book {
  id: String!
  name: String!
  author: Author
  tags: [String!]!
}

type Query {
  GetBook(id: String): Book!
}

As you know, we are using TypeScript. In TypeScript, we need to define a type to tell compiler to do the type check. Regrading of the Book entity, we have two definitions(truth), one is defined in the Lambda(TypeScript), another is the GraphQL. Who the ultimate truth? If the tags is mandatory in GraphqQL schema, but in our TypeScript is null-able field. Which one is correct?

Let the code generate the code

"When you copy the same code 3 times, it's time to refactor it". Seems we copied the same code more than hundred times. We have to find a way to save our time. So the first idea come to our mind: let the code to generate the code.

In order to let the code to genrate the code, we have understand the how the code works. Obviously, we have to ask our old friend: compiler: https://github.com/Microsoft/TypeScript/wiki/Using-the-Compiler-API

In the TS complier API, they provide a vistor pattern to allow us to access the AST when the compiler is parsing the code file. Also they provide a type checker to help us to know what's the type. Here as example from TS Complier API doc:

import * as ts from "typescript";
import * as fs from "fs";

let program = ts.createProgram(fileNames, options);
// Get the checker, we will use it to find more about classes
let checker = program.getTypeChecker();
// Visit every sourceFile in the program
for (const sourceFile of program.getSourceFiles()) {
  if (!sourceFile.isDeclarationFile) {
    // Walk the tree to search for classes
    ts.forEachChild(sourceFile, visit);
  }
}
/** The vistor to access all the node */
function visit(node: ts.Node) {
  if (ts.isClassDeclaration(node) && node.name) {
    // This is a top level class, get its symbol
    let symbol = checker.getSymbolAtLocation(node.name);
  } else if (ts.isModuleDeclaration(node)) {
    // This is a namespace, visit its children
    ts.forEachChild(node, visit);
  }
}

Util now, the problem becomes a "easy" one. Just like "How to put a elephant into a fridge". To archive it, we need to solve this sub goals.

Locale the entry point (Open the door of fridge)

As normal application, Lambda also has "main" function as the entry point. Instead of use main function, AWS Lambda allow user to choose the entry point. So any function can become the entry point. It becomes a problem.

Our solution is using JS comment with annotation. TS complier doesn't only analysis our code and but also our comment, essentially the JS comment. We are using this comment to mark the function as the enrty point.

```nodejs
/**
 * @lambda getBook
 */
export const handler = async () => {}

In the Compiler Side we can use this to find the lambda:

function vist(node: TS.Node) {
  const tags = ts.getJSDocTags(node);
  if (
      tags.filter((tag: ts.JSDocTag) => tag.tagName.escapedText === 'lambda')
        .length > 0
    )
  {
     // we found the function
  }
}

Find the type definition (Put the elephant into fridge)

After we located the lambda, we just need to pass the node into another function to check the type info. So First thing, the node must be a function type, it must have a return type (we assume it's a GraphlQL query lambda).

if (!ts.isArrowFunction(node)) return;
const nodeReturnType = node.type;
if (nodeReturnType == null || !ts.isTypeReferenceNode(nodeReturnType)) {
  throw Error(`Handler must have a return type; at handler ${name}`);
}
// typeText is the elephant
const typeText = nodeReturnType.getText(); // typeText = Book

Generate Code (Close the door)

We can the lambda and name, we can just replace the boilerplate with the value.

  ${Name}Function:
    Type: AWS::Serverless::Function
    Properties:
      Runtime: nodejs14.x
      CodeUri: ${Name}
      Handler: index.handler

But for the GraphQL Schema, we need to a "little" extra works, we need to mapping TypeScript type into GraphQL type. For the primitive type, such as String, we can have a hard code mapping, something like this.

{
  'string?': graphql.GraphQLString,
  'bolean?': graphql.GraphQLBoolean,
   string: graphql.GraphQLNonNull(graphql.GraphQLString),
}

For other parts, we can use the same tricky in the second phase. Here is a demo:

interface Book extends ChocoGraphQLType {
  id?: string;
  name: string;
  author?: Author;
  tags: string[];
}

In here, we use "Interface" to mark the type is a GraphQLType, so we using Complier to find the node implement "ChocoGraphQLType" and get all the defined properties. Then we can build a mapping like this.

{
  "Book": {
     type: Book
     // checker.typeToString(checker.getTypeFromTypeNode(node))
     args: {
       id?: graphql.GraphQLString,
       ...
     }
  }
}

Those mapping object is GraphQL Schema ready definition, we just need to clean a bit, then we can dump the definition to Schema file.