Skip to content

Instantly share code, notes, and snippets.

@joakimk
Last active Nov 24, 2015
Embed
What would you like to do?
Seeing "Encoding::CompatibilityError: incompatible character encodings: UTF-8 and ASCII-8BIT" after upgrading from Rails 3 to Rails 4?

For us this turned out to be that we had cached html generated by the rails 3 app that had the ASCII-8BIT encoding, clearing all html caches fixed this.

To arrive at this, we first added this debug code:

# if you reuse this: ensure it matches the method in your version of rails
# as the method is overwritten by this monkeypatch

# config/initializers/debug_encoding_errors.rb
ActiveSupport::SafeBuffer
class ActiveSupport::SafeBuffer
  def safe_concat(value)
    raise SafeConcatError unless html_safe?
    string_was = self.dup
    encoding_was = { bad: value.encoding, current: string_was.encoding }

    # This error is caused by something like this:
    #"foä".encode("iso-8859-1").concat("bär")

    original_concat(value)
  rescue Encoding::CompatibilityError => ex
    File.open("#{Rails.root}/log/encoding_error_debug_info.log", "ab") do |f|
      f.puts("Current string: #{string_was}")
      f.puts
      f.puts("Bad string: #{value}")
      f.puts
      f.puts("Encodings: #{encoding_was.inspect}")
      f.puts
      f.puts("Bad string characters: #{value.unpack("C*").map { |code| "#{code.chr.inspect} #{code}" }.join(", ")}")
      f.puts
      f.puts "Error:"
      f.puts ex.message
      f.puts ex.backtrace.inspect
      10.times { f.puts }
    end

    raise
  end
end

We could not reproduce the error in the browser, but using curl it worked, probably due to not having locale cookies, etc.

As this error was caused by caches in production it was not reproducable in development or staging.

We made a separate copy of the code in production (to get as close to the problem as possible) and got the error there as well. Then we simply traced though the call stack and printed values until we figured out that the cache returned bad data.

Hope this helps you debug similar issues, please tweet @joakimk or comment here if you find this useful.

@benjaminwood

This comment has been minimized.

Copy link

@benjaminwood benjaminwood commented Nov 24, 2015

@joakimk you da real MVP. I was starting to loose my mind on this. Thanks for the tip!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment