Skip to content

Instantly share code, notes, and snippets.

@chapmajs
Created July 26, 2017 02:44
Show Gist options
  • Save chapmajs/ea78713bf457c47984bf1be70101696d to your computer and use it in GitHub Desktop.
Save chapmajs/ea78713bf457c47984bf1be70101696d to your computer and use it in GitHub Desktop.
Fixing squiggly heredoc on Pygments
*** pygments/lexers/ruby.old 2017-07-25 22:03:14.847653103 -0400
--- pygments/lexers/ruby.py 2017-07-25 22:38:27.475907303 -0400
***************
*** 44,49 ****
--- 44,52 ----
def heredoc_callback(self, match, ctx):
# okay, this is the hardest part of parsing Ruby...
# match: 1 = <<-?, 2 = quote? 3 = name 4 = quote? 5 = rest of line
+ #
+ # Ruby >=2.3 has squiggly heredoc
+ # match: 1 = <<~?, 2 = quote? 3 = name 4 = quote? 5 = rest of line
start = match.start(1)
yield start, Operator, match.group(1) # <<-?
***************
*** 53,59 ****
heredocstack = ctx.__dict__.setdefault('heredocstack', [])
outermost = not bool(heredocstack)
! heredocstack.append((match.group(1) == '<<-', match.group(3)))
ctx.pos = match.start(5)
ctx.end = match.end(5)
--- 56,62 ----
heredocstack = ctx.__dict__.setdefault('heredocstack', [])
outermost = not bool(heredocstack)
! heredocstack.append(((match.group(1) == '<<-' or match.group(1) == '<<~'), match.group(3)))
ctx.pos = match.start(5)
ctx.end = match.end(5)
***************
*** 251,256 ****
--- 254,264 ----
heredoc_callback),
# empty string heredocs
(r'(<<-?)("|\')()(\2)(.*?\n)', heredoc_callback),
+ # squiggly heredocs, Ruby >=2.3
+ (r'(?<!\w)(<<~?)(["`\']?)([a-zA-Z_]\w*)(\2)(.*?\n)',
+ heredoc_callback),
+ # empty string squiggly heredocs, Ruby >2.3
+ (r'(<<~?)("|\')()(\2)(.*?\n)', heredoc_callback),
(r'__END__', Comment.Preproc, 'end-part'),
# multiline regex (after keywords or assignments)
(r'(?:^|(?<=[=<>~!:])|'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment