Skip to content

Instantly share code, notes, and snippets.

@conartist6
Last active Jan 22, 2023
Embed
What would you like to do?
https://github.com/js-cst-tokens/cst-tokens comment attachment example
function /*1*/ foo /*2*/ (/*3*/) /*4*/ {/*5*/} /*6*/
{
"type": "CSTFragment",
"cstTokens": [
{ "type": "Reference", "value": "fragment" },
{ "type": "Literal", "value": " " },
{ "type": "CommentStart", "value": "/*" },
{ "type": "Literal", "value": "6" },
{ "type": "CommentEnd", "value": "*/" }
],
"fragment": {
"type": "Program",
"cstTokens": [
{ "type": "Reference", "value": "body" }
],
"sourceType": "module",
"body": [
{
"type": "FunctionDeclaration",
"cstTokens": [
{ "type": "Keyword", "value": "function" },
{ "type": "Literal", "value": " " },
{ "type": "CommentStart", "value": "/*" },
{ "type": "Literal", "value": "1" },
{ "type": "CommentEnd", "value": "*/" },
{ "type": "Literal", "value": " " },
{ "type": "Reference", "value": "id" },
{ "type": "Literal", "value": " " },
{ "type": "CommentStart", "value": "/*" },
{ "type": "Literal", "value": "2" },
{ "type": "CommentEnd", "value": "*/" },
{ "type": "Literal", "value": " " },
{ "type": "LeftPunctuator", "value": "(" },
{ "type": "CommentStart", "value": "/*" },
{ "type": "Literal", "value": "3" },
{ "type": "CommentEnd", "value": "*/" },
{ "type": "RightPunctuator", "value": ")" },
{ "type": "CommentStart", "value": "/*" },
{ "type": "Literal", "value": "4" },
{ "type": "CommentEnd", "value": "*/" },
{ "type": "Literal", "value": " " },
{ "type": "Reference", "value": "body" }
],
"id": {
"type": "Identifier",
"cstTokens": [
{ "type": "SymbolDefinition", "value": "foo" }
],
"name": "foo"
},
"params": [],
"body": {
"type": "BlockStatement",
"cstTokens": [
{ "type": "LeftPunctuator", "value": "{" },
{ "type": "CommentStart", "value": "/*" },
{ "type": "Literal", "value": "5" },
{ "type": "CommentEnd", "value": "*/" },
{ "type": "RightPunctuator", "value": "}" }
],
"body": []
},
"async": false,
"generator": false
}
]
}
}
# Don't look too closely -- this is out of date!
-> CSTFragment
-> Program
-> FunctionDeclaration
--> eatGrammar WithSeparator
--> eatMatchGrammar Separator
? [ ] Whitespace
? [ ] LineBreak
--> eatMatchGrammar Comment
? [ ] CommentStart
? [ ] LineCommentStart
<--
<--
startNode 1 Program
startNode 2
[+] Keyword
<--
--> eatGrammar WithWhitespace_All
-> Identifier
--> eatGrammar WithSeparator
--> eatMatchGrammar Separator
? [+] Whitespace
? [ ] LineBreak
--> eatMatchGrammar Comment
? [+] CommentStart
? [+] Comment
[+] CommentEnd
<--
? [+] Whitespace
? [ ] LineBreak
--> eatMatchGrammar Comment
? [ ] CommentStart
? [ ] LineCommentStart
<--
? [ ] Whitespace
<--
startNode 3
[+] SymbolDefinition
<--
endNode 3
<- Identifier
--> eatGrammar WithSeparator
--> eatMatchGrammar Separator
? [+] Whitespace
? [ ] LineBreak
--> eatMatchGrammar Comment
? [+] CommentStart
? [+] Comment
[+] CommentEnd
<--
? [+] Whitespace
? [ ] LineBreak
--> eatMatchGrammar Comment
? [ ] CommentStart
? [ ] LineCommentStart
<--
? [ ] Whitespace
<--
[+] LeftPunctuator
<--
<--
--> eatGrammar WithWhitespace_All
--> eatGrammar WithSeparator
--> eatMatchGrammar Separator
? [ ] Whitespace
? [ ] LineBreak
--> eatMatchGrammar Comment
? [+] CommentStart
? [+] Comment
[+] CommentEnd
<--
? [ ] Whitespace
? [ ] LineBreak
--> eatMatchGrammar Comment
? [ ] CommentStart
? [ ] LineCommentStart
<--
<--
[+] RightPunctuator
<--
-> BlockStatement
--> eatGrammar WithSeparator
--> eatMatchGrammar Separator
? [+] Whitespace
? [ ] LineBreak
--> eatMatchGrammar Comment
? [+] CommentStart
? [+] Comment
[+] CommentEnd
<--
? [+] Whitespace
? [ ] LineBreak
--> eatMatchGrammar Comment
? [ ] CommentStart
? [ ] LineCommentStart
<--
? [ ] Whitespace
<--
startNode 3
[+] LeftPunctuator
<--
--> eatGrammar WithSeparator
--> eatMatchGrammar Separator
? [ ] Whitespace
? [ ] LineBreak
--> eatMatchGrammar Comment
? [+] CommentStart
? [+] Comment
[+] CommentEnd
<--
? [ ] Whitespace
? [ ] LineBreak
--> eatMatchGrammar Comment
? [ ] CommentStart
? [ ] LineCommentStart
<--
<--
[+] RightPunctuator
<--
endNode 3
<- BlockStatement
<--
--> eatMatchGrammar WithSeparator
--> eatMatchGrammar Separator
? [+] Whitespace
? [ ] LineBreak
--> eatMatchGrammar Comment
? [+] CommentStart
? [+] Comment
[+] CommentEnd
<--
<--
[x] Punctuator
<--
endNode 2
<- FunctionDeclaration
endNode 1
<- Program
--> eatMatchGrammar Separator
? [+] Whitespace
? [ ] LineBreak
--> eatMatchGrammar Comment
? [+] CommentStart
? [+] Comment
[+] CommentEnd
<--
<--
<- CSTFragment
function /*1*/ foo /*2*/ (/*3*/) /*4*/ {/*5*/} /*6*/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment