Skip to content

Instantly share code, notes, and snippets.

@pbyrne
Forked from rchampourlier/google_bot_aware.rb
Created November 13, 2012 16:50
Show Gist options
  • Save pbyrne/4066926 to your computer and use it in GitHub Desktop.
Save pbyrne/4066926 to your computer and use it in GitHub Desktop.
Rack middleware to make Rails deal correctly with GoogleBot's '*/*;q=0.6' Accept header
Gem::Specification.new do |s|
s.name = 'google_bot_aware'
s.version = '0.1.0'
s.platform = Gem::Platform::RUBY
s.author = 'Romain Champourlier'
s.email = 'romain@softr.li'
s.summary = 'Google Bot Aware'
s.description = "Strip extra characters from Google Bot's Accept header so that Rails knows to use default MIME type in response."
s.files = ['google_bot_aware.rb']
s.test_file = 'google_bot_aware_spec.rb'
s.require_path = '.'
s.add_development_dependency('rspec', ["~> 2.0"])
end
# This Rack middleware helps solving the issue with some Rails versions which do not accept
# a '*/*;q=0.6' and their variants 'Accept' request header. This header is particularly used
# by Google Bot, and if Rails doesn't like it, it will return a 500 or 406 error to Google Bot,
# which is not the best way to get your pages indexed.
#
# References:
# - http://stackoverflow.com/questions/8881756/googlebot-receiving-missing-template-error-for-an-existing-template
# - https://github.com/rails/rails/issues/4127
#
class GoogleBotAware
def initialize(app)
@app = app
end
def call(env)
# If the request 'Content Accept' header indicates a '*/*' format,
# we set the format to :html.
# This is necessary for GoogleBot which requests / with '*/*;q=0.6' for example.
if env["HTTP_ACCEPT"] =~ %r%\*\/\*;q=\d\.\d%
env["HTTP_ACCEPT"] = '*/*'
end
@app.call(env)
end
end
# spec/integration/google_bot_aware_spec.rb
#
# This integration test is intended to check if your application is correctly managing
# GoogleBot's requests which may have an Accept header looking like this: '*/*;q=0.6'.
#
# This should be the case if GoogleBotAware Rack middleware is correctly loaded, and is
# correctly behaving.
#
describe "GoogleBot aware" do
['/', 'add-your-own-paths'].each do |path|
it "should be successful for a Google-Bot-style request on '#{path}'" do
get path, nil, {"HTTP_ACCEPT" => "*/*;q=0.6"}
response.should be_successful
end
end
end
# config/initializers/rack_middlewares.rb
#
# This initializer configures Rack middlewares and loads custom middlewares.
require File.join(Rails.root, 'lib', 'google_bot_aware')
App::Application.configure do
config.middleware.use("GoogleBotAware")
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment