Skip to content

Instantly share code, notes, and snippets.

@tamtam180
Created June 3, 2015 14:03
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save tamtam180/29028c011abdd36ae0ef to your computer and use it in GitHub Desktop.
Save tamtam180/29028c011abdd36ae0ef to your computer and use it in GitHub Desktop.
# coding: utf-8
require 'json'
require 'yajl'
x = eval('{"logType"=>"ads", "abc"=>"\xE3\x82\xA4\xE3\x83\xB3\xE3\x82\xB9\xE3\x83\x88\xE3\x83\xBC\xE3\x83\xAB", "time"=>"2015-05-17T03:00:00Z", "tag"=>"aaa"}')
puts x["abc"].force_encoding("ASCII-8BIT")
puts x["abc"].encoding
puts x
puts Yajl.dump(x)
puts JSON.dump(x)
インストール
ASCII-8BIT
{"logType"=>"ads", "abc"=>"\xE3\x82\xA4\xE3\x83\xB3\xE3\x82\xB9\xE3\x83\x88\xE3\x83\xBC\xE3\x83\xAB", "time"=>"2015-05-17T03:00:00Z", "tag"=>"aaa"}
{"logType":"ads","abc":"インストール","time":"2015-05-17T03:00:00Z","tag":"aaa"}
/Library/Ruby/Gems/2.0.0/gems/json-1.8.1/lib/json/common.rb:223:in `encode': "\xE3" from ASCII-8BIT to UTF-8 (Encoding::UndefinedConversionError)
from /Library/Ruby/Gems/2.0.0/gems/json-1.8.1/lib/json/common.rb:223:in `generate'
from /Library/Ruby/Gems/2.0.0/gems/json-1.8.1/lib/json/common.rb:223:in `generate'
from /Library/Ruby/Gems/2.0.0/gems/json-1.8.1/lib/json/common.rb:394:in `dump'
from hoge.rb:12:in `<main>'
@tamtam180
Copy link
Author

もともとはfluentdの in-tail -> kinesis の流れで発生したやつで、
awslabs/aws-fluent-plugin-kinesis#13 (comment)

状況を再現すると hoge.rb な感じです。

@nurse
Copy link

nurse commented Jun 3, 2015

本質的な流れは以下のような感じですかね。in_tail側が悪いような気もしますが、どうしたものかな

require "yajl"
require "json"
Encoding.default_internal=Encoding::ASCII_8BIT
text = %'{"logType":"ads", "abc":"\xE3\x82\xA4\xE3\x83\xB3\xE3\x82\xB9\xE3\x83\x88\xE3\x83\xBC\xE3\x83\xAB", "time":"2015-05-17T03:00:00Z", "tag":"aaa"}'.b
h = Yajl.load(text) # in_tail
p h["abc"].encoding #=> #<Encoding:ASCII-8BIT>
puts JSON.dump(h) #=> error

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment