Skip to content

Instantly share code, notes, and snippets.

@lsegal
Created March 25, 2012 22:20
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save lsegal/2200383 to your computer and use it in GitHub Desktop.
Save lsegal/2200383 to your computer and use it in GitHub Desktop.
Encoding bug in ripper with heredoc strings
require 'ripper'
require 'pp'
sexp = Ripper.sexp(DATA.read)
pp sexp
sexp = sexp.flatten
heredoc = sexp[sexp.flatten.index(:@tstring_content) + 1]
string = sexp[sexp.flatten.rindex(:@tstring_content) + 1]
puts
puts "String: #{string.inspect} Encoding: #{string.encoding.inspect}"
puts "Heredoc: #{heredoc.inspect} Encoding: #{heredoc.encoding.inspect}"
__END__
# encoding: utf-8
<<-eos
ƒøø
eos
"ƒøø"
$ ruby -v ripper_bug.rb
ruby 1.9.3p125 (2012-02-16 revision 34643) [x86_64-darwin11.0.0]
[:program,
[[:string_literal,
[:string_content,
[:@tstring_content, "\xC6\x92\xC3\xB8\xC3\xB8\n", [4, 0]]]],
[:string_literal, [:string_content, [:@tstring_content, "ƒøø", [6, 1]]]]]]
String: "ƒøø" Encoding: #<Encoding:UTF-8>
Heredoc: "\xC6\x92\xC3\xB8\xC3\xB8\n" Encoding: #<Encoding:ASCII-8BIT>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment