Skip to content

Instantly share code, notes, and snippets.

@vsajip
Last active June 25, 2021 22:39
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save vsajip/0c7b0ed6bdfa87472f254e77be4c2ca3 to your computer and use it in GitHub Desktop.
Save vsajip/0c7b0ed6bdfa87472f254e77be4c2ca3 to your computer and use it in GitHub Desktop.
Python code generation template and result - how to pass indentation across multiple levels of nested directives?
[#macro BuildCodeSequence expansion indent]
[#var is = ""?right_pad(indent)]
${is}# DBG > BuildCodeSequence ${indent}
[#list expansion.units as subexp]
[@BuildCode subexp indent /]
[/#list]
${is}# DBG < BuildCodeSequence ${indent}
[/#macro]
[#macro BuildExpansionCode expansion indent]
[#var is=""?right_pad(indent)]
[#var classname=expansion.simpleName]
${is}# DBG > BuildExpansionCode ${indent} ${classname}
[#var prevLexicalStateVar = "previousLexicalState"]
[#if classname = "ExpansionWithParentheses"]
[@BuildExpansionCode expansion.nestedExpansion indent /]
[#elseif classname = "CodeBlock"]
${is}# CodeBlock expansion start
${is}${expansion}
${is}# CodeBlock expansion end
[#elseif classname = "Failure"]
[@BuildCodeFailure expansion/]
[#elseif classname = "TokenTypeActivation"]
[@BuildCodeTokenTypeActivation expansion indent /]
[#elseif classname = "ExpansionSequence"]
[@BuildCodeSequence expansion indent /]
[#elseif classname = "NonTerminal"]
[@BuildCodeNonTerminal expansion indent /]
[#elseif expansion.isRegexp]
[@BuildCodeRegexp expansion indent /]
[#elseif classname = "TryBlock"]
[@BuildCodeTryBlock expansion/]
[#elseif classname = "AttemptBlock"]
[@BuildCodeAttemptBlock expansion /]
[#elseif classname = "ZeroOrOne"]
[@BuildCodeZeroOrOne expansion indent /]
[#elseif classname = "ZeroOrMore"]
[@BuildCodeZeroOrMore expansion indent /]
[#elseif classname = "OneOrMore"]
[@BuildCodeOneOrMore expansion /]
[#elseif classname = "ExpansionChoice"]
[@BuildCodeChoice expansion indent /]
[#elseif classname = "Assertion"]
[@BuildAssertionCode expansion /]
[/#if]
${is}# DBG < BuildExpansionCode ${indent} ${classname}
[/#macro]
[#macro TreeBuildingAndRecovery expansion indent]
[#-- This macro handles both tree building AND recovery. It doesn't seem right.
It should probably be two macros. Also, it is too darned big. --]
[#var is=""?right_pad(indent)]
${is}# DBG > TreeBuildingAndRecovery ${indent}
[#var nodeVarName,
production,
treeNodeBehavior,
buildTreeNode=false,
closeCondition = "True",
javaCodePrologue = "",
parseExceptionVar = "parseException1",
callStackSizeVar = "callStackSize1",
canRecover = false
]
[#-- set treeNodeBehavior = expansion.treeNodeBehavior --]
[#-- if expansion.parent.simpleName = "BNFProduction"]
[#set production = expansion.parent]
[#set javaCodePrologue = production.javaCode!]
[/#if --]
[#-- if grammar.treeBuildingEnabled]
[#set buildTreeNode = (treeNodeBehavior?is_null && production?? && !grammar.nodeDefaultVoid)
|| (treeNodeBehavior?? && !treeNodeBehavior.neverInstantiated)]
[/#if --]
[#if false && !buildTreeNode && !canRecover]
${javaCodePrologue}
[#nested]
[#else]
[#if buildTreeNode]
[#set nodeNumbering = nodeNumbering +1]
[#set nodeVarName = currentProduction.name + nodeNumbering]
${grammar.utils.pushNodeVariableName(nodeVarName)!}
[#if !treeNodeBehavior?? && !production?is_null]
[#if grammar.smartNodeCreation]
[#set treeNodeBehavior = {"name" : production.name, "condition" : "1", "gtNode" : true, "void" :false}]
[#else]
[#set treeNodeBehavior = {"name" : production.name, "condition" : null, "gtNode" : false, "void" : false}]
[/#if]
[/#if]
[#if treeNodeBehavior.condition?has_content]
[#set closeCondition = treeNodeBehavior.condition]
[#if treeNodeBehavior.gtNode]
[#set closeCondition = "nodeArity() > " + closeCondition]
[/#if]
[/#if]
[@createNode treeNodeBehavior nodeVarName false indent /]
[/#if]
[#-- I put this here for the hypertechnical reason
that I want the initial code block to be able to
reference CURRENT_NODE. --]
${is}${javaCodePrologue}
${is}${parseExceptionVar} = None
${is}${callStackSizeVar} = len(self.parsing_stack)
${is}try:
${is} pass # if False: raise ParseException('Never happens!')
${is} # nested code starts, passing indent of ${indent + 4}
[#nested indent + 4]
${is} # nested code ends
${is}except ParseException as e:
${is} ${parseExceptionVar} = e
[#if !canRecover]
[#if false && grammar.faultTolerant]
${is} if self.is_tolerant: self.pending_recovery = True
[/#if]
${is} raise
[#else]
${is} if not self.is_tolerant: raise
${is} self.pending_recovery = True
${expansion.customErrorRecoveryBlock!}
[#if !production?is_null && production.returnType != "void"]
[#var rt = production.returnType]
[#-- We need a return statement here or the code won't compile! --]
[#if rt = "int" || rt="char" || rt=="byte" || rt="short" || rt="long" || rt="float"|| rt="double"]
${is} return 0
[#else]
${is} return None
[/#if]
[/#if]
[/#if]
${is}finally:
${is} self.restore_call_stack(${callStackSizeVar})
[#if buildTreeNode]
${is} if ${nodeVarName}:
${is} if ${parseExceptionVar} is None:
${is} self.close_node_scope(${nodeVarName}, ${closeCondition})
[#list grammar.closeNodeHooksByClass[nodeClassName(treeNodeBehavior)]! as hook]
${is} ${hook}(${nodeVarName})
[/#list]
${is} else:
${is} if self.trace_enabled:
${is} logger.warning('ParseException: %s', ${parseExceptionVar})
[#if grammar.faultTolerant]
${is} self.close_node_scope(${nodeVarName}, True)
${is} ${nodeVarName}.dirty = True
[#else]
${is} self.clear_node_scope()
[/#if]
${grammar.utils.popNodeVariableName()!}
[/#if]
${is} self.currently_parsed_production = self.prev_production
[/#if]
${is}# DBG < TreeBuildingAndRecovery ${indent}
[/#macro]
[#macro HandleLexicalStateChange expansion inLookahead indent=0]
[#var is=""?right_pad(indent)]
${is}# DBG > HandleLexicalStateChange ${indent} ${expansion.simpleName}
[#if expansion.specifiedLexicalState?is_null]
[#nested indent]
[#else]
[#var resetToken = inLookahead?string("currentLookaheadToken", "lastConsumedToken")]
[#var prevLexicalStateVar = newVarName("previousLexicalState")]
${is}${prevLexicalStateVar} = self.token_source.lexical_state
${is}if self.token_source.lexical_state != LexicalState.${expansion.specifiedLexicalState}:
${is} self.token_source.reset(${resetToken}, LexicalState.${expansion.specifiedLexicalState})
${is} try:
[#nested indent + 4 /]
${is} finally:
${is} if ${prevLexicalStateVar} != LexicalState.${expansion.specifiedLexicalState}:
[#if !grammar.hugeFileSupport && !grammar.userDefinedLexer]
${is} self.token_source.reset(${resetToken}, ${prevLexicalStateVar})
[#else]
${is} self.token_source.switch_to(${prevLexicalStateVar})
[/#if]
[/#if]
${is}# DBG < HandleLexicalStateChange ${indent} ${expansion.simpleName}
[/#macro]
[#macro BuildCode expansion indent]
[#var is=""?right_pad(indent)]
${is}# DBG > BuildCode ${indent} ${expansion.simpleName}
[#if expansion.simpleName != "ExpansionSequence" && expansion.simpleName != "ExpansionWithParentheses"]
${is}# Code for ${expansion.simpleName} specified at:
${is}# ${expansion.location}
[/#if]
[@HandleLexicalStateChange expansion false indent]
[#-- if grammar.faultTolerant && expansion.requiresRecoverMethod && !expansion.possiblyEmpty]
if self.pending_recovery:
if self.debug_fault_tolerant:
logger.info('Re-synching to expansion at: ${expansion.location?j_string}')
${expansion.recoverMethodName}()
}
[/#if --]
[@TreeBuildingAndRecovery expansion indent]
[@BuildExpansionCode expansion indent/]
[/@TreeBuildingAndRecovery]
[/@HandleLexicalStateChange]
${is}# DBG < BuildCode ${indent} ${expansion.simpleName}
[/#macro]
[#macro ParserProduction production]
def parse_${production.name}(self):
if self.trace_enabled:
logger.info('Entering production ${production.name?j_string}')
if self.cancelled:
raise CancellationException('operation cancelled')
prev_production = self.currently_parsed_production
self.currently_parsed_production = '${production.name}'
[@BuildCode production.expansion 8 /]
# end of parse_${production.name}
[/#macro]
[#assign model = {
"name": "PRODNAME",
"expansion": {
"simpleName": "ExpansionSequence",
"location": "LOCATION",
"specifiedLexicalState": null,
"units": [
]
}
}]
[@ParserProduction model /]
def parse_PRODNAME(self):
if self.trace_enabled:
logger.info('Entering production PRODNAME')
if self.cancelled:
raise CancellationException('operation cancelled')
prev_production = self.currently_parsed_production
self.currently_parsed_production = 'PRODNAME'
# DBG > BuildCode 8 ExpansionSequence
# DBG > HandleLexicalStateChange 8 ExpansionSequence
# DBG > TreeBuildingAndRecovery 8
parseException1 = None
callStackSize1 = len(self.parsing_stack)
try:
pass # if False: raise ParseException('Never happens!')
# nested code starts, passing indent of 12
# DBG > BuildExpansionCode 8 ExpansionSequence
# DBG > BuildCodeSequence 8
# DBG < BuildCodeSequence 8
# DBG < BuildExpansionCode 8 ExpansionSequence
# nested code ends
except ParseException as e:
parseException1 = e
raise
finally:
self.restore_call_stack(callStackSize1)
self.currently_parsed_production = self.prev_production
# DBG < TreeBuildingAndRecovery 8
# DBG < HandleLexicalStateChange 8 ExpansionSequence
# DBG < BuildCode 8 ExpansionSequence
# end of parse_PRODNAME
@vsajip
Copy link
Author

vsajip commented Jun 25, 2021

Notice that in the output file test.py, line 16 is generated by line 102 in the template. This is followed by a [#nested indent + 4] directive, which should pass a value of 12 to the nested template. However, line 17 of the result, from the nested content, shows that the indent is 8 there, not 12 as it should be. It's not clear why that's happening.

My FM is no doubt very rusty. Is there a better way of passing the indentation level correctly across multiple levels of nested directives, as in this example?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment