URL Mangling...

gistfile1.txt

Given this url:

http://example.com/?select=a&select=b

In a real rails app, request in controller is ActionController::Request, and #url returns: http://example.com/?select=a&select=b

In Rspec "request" specs:

get "http://example.com/?select=a&select=b"

request in the spec and in the controller code is AcitonDispatch::Request, and #url (request.url in the controller) now returns "http://example.com/?select=b"

The ActionDispatch inspect yields a bogus @full_path, but REQUET_URI is in tact.

Near as I can figure, Rack::Test, or Dispatch, or something is parsing the query string as parse_nested_query, which is fine, but in the process, #url is not returning the same thing in ActionDispatch::Request that id does in a real world rails app request in ActionController::Request.


I'm stumped after looking at the source. Ideas?


-----
#<ActionDispatch::Request:0x11bb01910 @fullpath="/api/members?_signature=5ZwgwR45YruA3fBnVysWbemHPkrt12VpigT7%2F6%2FLHMg%3D&select=token", @method="GET", @filtered_parameters={"action"=>"index", "_signature"=>"5ZwgwR45YruA3fBnVysWbemHPkrt12VpigT7/6/LHMg=", "controller"=>"api/members", "select"=>"token"}, @env={"action_dispatch.request.formats"=>[#<Mime::Type:0x104df21b8 @string="application/json", @synonyms=["text/x-json", "application/jsonrequest"], @symbol=:json>], "action_dispatch.request.parameters"=>{"action"=>"index", "_signature"=>"5ZwgwR45YruA3fBnVysWbemHPkrt12VpigT7/6/LHMg=", "select"=>"token", "controller"=>"api/members"}, "rack.session"=>{}, "rack.test"=>true, "HTTP_ACCEPT"=>"application/json", "HTTP_HOST"=>"www.example.com", "SERVER_NAME"=>"www.example.com", "rack.request.cookie_hash"=>{}, "action_dispatch.remote_ip"=>#<ActionDispatch::RemoteIp::RemoteIpGetter:0x11bc4de40 @trusted_proxies=/(^127\.0\.0\.1$|^(10|172\.(1[6-9]|2[0-9]|30|31)|192\.168)\.)/i, @env={...}, @check_ip_spoofing=true>, "CONTENT_LENGTH"=>"0", "rack.url_scheme"=>"http", "action_dispatch.request.query_parameters"=>{"_signature"=>"5ZwgwR45YruA3fBnVysWbemHPkrt12VpigT7/6/LHMg=", "select"=>"token"}, "action_dispatch.request.unsigned_session_cookie"=>{}, "CONTENT_TYPE"=>"application/x-www-form-urlencoded", "HTTPS"=>"off", "rack.errors"=>#<StringIO:0x11bc612b0>, "action_dispatch.request.accepts"=>[#<Mime::Type:0x104df21b8 @string="application/json", @synonyms=["text/x-json", "application/jsonrequest"], @symbol=:json>], "action_dispatch.secret_token"=>"aa29852b13417d43f4a1b48343dcbc40f065f6e87ec1985cb038e331881a165b4b52140feac36964e32016505c02fca1e06115a7f888b2e169f3fd6e5dfebbf5", "REMOTE_ADDR"=>"127.0.0.1", "PATH_INFO"=>"/api/members", "rack.version"=>[1, 1], "rack.run_once"=>false, "action_dispatch.request.path_parameters"=>{:controller=>"api/members", :action=>"index"}, "rack.request.cookie_string"=>"", "SCRIPT_NAME"=>"", "action_dispatch.parameter_filter"=>[:password, :password, :password], "action_dispatch.show_exceptions"=>true, "HTTP_COOKIE"=>"", "rack.multithread"=>false, "action_dispatch.request.request_parameters"=>{}, "rack.request.form_vars"=>"", "action_dispatch.cookies"=>{}, "REQUEST_URI"=>"/api/members?select=id&select=guid&select=primary_email_address&select=token&_signature=5ZwgwR45YruA3fBnVysWbemHPkrt12VpigT7%2F6%2FLHMg%3D", "rack.multiprocess"=>true, "rack.request.query_hash"=>{"_signature"=>"5ZwgwR45YruA3fBnVysWbemHPkrt12VpigT7/6/LHMg=", "select"=>"token"},

And here's a github issue on Rack::Test that led me to a "solution".

rack/rack-test#26

Rack::Test chooses to not send REQUEST_URI, but it also sends a QUERY_STRING that is parsed, which means it might not be parsed the way you want it parsed. (Think Rack parse_query vs parse_nested_query). Then apparently, ActionDispatch or Rails uses the bogus QUERY_STRING to rebuild what ends up being @full_path, exposed to request.url in Rails controllers.

In other words, somewhere between Rack::Test or how RSpec interacts with it in "request" specs, controllers could end up with a request.url that is not what users entered into their GET request. As I mentioned above, this doesn't happen with RSpec "controller" specs.

Now for the fun part.

In my failing spec, I change this line:

get "http://www.example.com/?select=1&select=2"

to this line:

get "http://www.example.com/?select=1&select=2", {}, {"QUERY_STRING" => "select=1&select=2"}

request.url in the controller code is now the full, correct url with the query string as it was entered.

Curiously, this does NOT work:

get "http://www.example.com/?select=1&select=2", {}, {"REQUEST_URI" => "http://www.example.com/?select=1&select=2"}

I'm not sure what the correct solution is, but falling into this rabbit hole sucked, and I'd like to help others not suffer the same.

Hi Christopher -- Based on your understanding, is Rack-Test sending the wrong QUERY_STRING?

This is why I CCed you and dhh.dchelinky on teh Twitters. I'm not sure if it's RSpecs use of Rack::Test, or Rack::Test itself.

Based on the inspect dump above, the requests QUERT_STRING env variable (and eventually @full_pathin Dispatch) is being sent with a parsed query string, and not simply everything after the ? of the original uri given to it. Base on the nature of what is sent, it's almost certainly someone doing a parse_nested_query on it first, since that's the destructive rails-like parse. The normal Rack Utils parse_query simply packs like named values into an Array.

I started going through the Rack:Test source last night, but still had a hard time figuring out how to replicate the issue, which is about when I stumbled upon rack/rack-test#26

@claco I'm almost 100% certain this is not rspec, since the get method in a request spec is defined in actionpack-3.1.3/lib/action_dispatch/testing/integration.rb and rspec does not override it in any way. A quick debugging session showed that by the time we get to actionpack-3.1.3/lib/action_dispatch/http/request.rb:39, env["QUERY_STRING"] is "select=b". I'm guessing that the query string is built by converting it to hash and then back, thus overwriting the "select" key with the value "b", but I'm not sure where that happens.

Thanks for looking into this. I wasn't sure if Rspec was making it's own ActionDispatch or letting Rack::Test do it, since controller specs seems to work just fine.

Thanks again!