Skip to content

Instantly share code, notes, and snippets.

@conartist6

conartist6/output.md

Last active September 15, 2022 22:21
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save conartist6/503bd36f1028ae61349183001d3537ba to your computer and use it in GitHub Desktop.
Save conartist6/503bd36f1028ae61349183001d3537ba to your computer and use it in GitHub Desktop.
Download ZIP
cst-tokens alpha output
Raw
output.md

Input:

if (foo) {
  // comment
}

Traversal log

  cst-tokens  -> Fragment (at sourceText[0]) +0ms
  cst-tokens  -> Program (at sourceText[0]) +1ms
  cst-tokens  -> IfStatement (at sourceText[0]) +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take Whitespace +0ms
  cst-tokens:stt reject +1ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take BlockCommentStart +1ms
  cst-tokens:stt reject +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take LineCommentStart +0ms
  cst-tokens:stt reject +0ms
  cst-tokens:cmd take Keyword +0ms
  cst-tokens:stt branch +1ms
  cst-tokens:cmd   take Whitespace +1ms
  cst-tokens:stt accept +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take BlockCommentStart +0ms
  cst-tokens:stt reject +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take LineCommentStart +0ms
  cst-tokens:stt reject +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take Whitespace +1ms
  cst-tokens:stt reject +1ms
  cst-tokens:cmd take LeftPunctuator +0ms
  cst-tokens  -> Identifier (at sourceText[4]) +3ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take Whitespace +0ms
  cst-tokens:stt reject +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take BlockCommentStart +0ms
  cst-tokens:stt reject +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take LineCommentStart +0ms
  cst-tokens:stt reject +0ms
  cst-tokens:cmd take Identifier +0ms
  cst-tokens:cmd emit Identifier +1ms
  cst-tokens:stt accept +1ms
  cst-tokens  <- IfStatement (at sourceText[7]) +1ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take Whitespace +0ms
  cst-tokens:stt reject +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take BlockCommentStart +0ms
  cst-tokens:stt reject +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take LineCommentStart +0ms
  cst-tokens:stt reject +0ms
  cst-tokens:cmd take RightPunctuator +0ms
  cst-tokens  -> BlockStatement (at sourceText[8]) +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take Whitespace +0ms
  cst-tokens:stt accept +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take BlockCommentStart +0ms
  cst-tokens:stt reject +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take LineCommentStart +0ms
  cst-tokens:stt reject +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take Whitespace +0ms
  cst-tokens:stt reject +0ms
  cst-tokens:cmd take LeftPunctuator +0ms
  cst-tokens:cmd emit Whitespace LeftPunctuator +1ms
  cst-tokens:cmd emit +0ms
  cst-tokens:stt branch +1ms
  cst-tokens:cmd   take Whitespace +0ms
  cst-tokens:stt accept +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take BlockCommentStart +0ms
  cst-tokens:stt reject +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take LineCommentStart +0ms
  cst-tokens:stt accept +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take Comment +0ms
  cst-tokens:stt   branch +0ms
  cst-tokens:stt   reject +0ms
  cst-tokens:stt   branch +0ms
  cst-tokens:stt   reject +0ms
  cst-tokens:stt   branch +0ms
  cst-tokens:stt   reject +0ms
  cst-tokens:stt   branch +0ms
  cst-tokens:stt   reject +0ms
  cst-tokens:stt   branch +0ms
  cst-tokens:stt   reject +0ms
  cst-tokens:stt   branch +0ms
  cst-tokens:stt   reject +0ms
  cst-tokens:stt   branch +0ms
  cst-tokens:stt   reject +0ms
  cst-tokens:stt   branch +0ms
  cst-tokens:stt   reject +1ms
  cst-tokens:stt   branch +0ms
  cst-tokens:stt   reject +0ms
  cst-tokens:stt accept +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take Whitespace +1ms
  cst-tokens:stt accept +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take Whitespace +0ms
  cst-tokens:stt reject +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take BlockCommentStart +0ms
  cst-tokens:stt reject +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take LineCommentStart +0ms
  cst-tokens:stt reject +0ms
  cst-tokens:cmd take RightPunctuator +0ms
  cst-tokens:cmd emit Whitespace LineCommentStart Comment Whitespace RightPunctuator +0ms
  cst-tokens:stt accept +0ms
  cst-tokens  <- IfStatement (at sourceText[25]) +2ms
  cst-tokens:cmd emit Keyword Whitespace LeftPunctuator Reference RightPunctuator Whitespace Reference +0ms
  cst-tokens:stt accept +0ms
  cst-tokens  <- Program (at sourceText[25]) +0ms
  cst-tokens:cmd emit Reference +0ms
  cst-tokens:stt accept +0ms
  cst-tokens  <- Fragment (at sourceText[25]) +0ms
  cst-tokens:cmd emit Reference +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take Whitespace +0ms
  cst-tokens:stt accept +1ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take BlockCommentStart +1ms
  cst-tokens:stt reject +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take LineCommentStart +0ms
  cst-tokens:stt reject +0ms
  cst-tokens:stt branch +0ms
  cst-tokens:cmd   take Whitespace +0ms
  cst-tokens:stt reject +0ms
  cst-tokens:cmd emit Whitespace +0ms
  cst-tokens done +1ms

CST

{
  "type": "Program",
  "sourceType": "module",
  "body": [
    {
      "type": "IfStatement",
      "test": {
        "type": "Identifier",
        "name": "foo",
        "cstTokens": [
          {
            "type": "Identifier",
            "value": "foo"
          }
        ]
      },
      "consequent": {
        "type": "BlockStatement",
        "body": [],
        "cstTokens": [
          {
            "type": "LeftPunctuator",
            "value": "{"
          },
          {
            "type": "Whitespace",
            "value": "\n  "
          },
          {
            "type": "LineCommentStart",
            "value": "//"
          },
          {
            "type": "Comment",
            "value": " comment"
          },
          {
            "type": "Whitespace",
            "value": "\n"
          },
          {
            "type": "RightPunctuator",
            "value": "}"
          }
        ]
      },
      "alternate": null,
      "cstTokens": [
        {
          "type": "Keyword",
          "value": "if"
        },
        {
          "type": "Whitespace",
          "value": " "
        },
        {
          "type": "LeftPunctuator",
          "value": "("
        },
        {
          "type": "Reference",
          "value": "test"
        },
        {
          "type": "RightPunctuator",
          "value": ")"
        },
        {
          "type": "Whitespace",
          "value": " "
        },
        {
          "type": "Reference",
          "value": "consequent"
        }
      ]
    }
  ],
  "cstTokens": [
    {
      "type": "Reference",
      "value": "body"
    },
    {
      "type": "Whitespace",
      "value": "\n"
    }
  ]
}

Output by simple tree traversal (same as input):

if (foo) {
  // comment
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment