em-proxy-http.rb

require 'em-proxy'
require 'http/parser'
require 'uuid'

# > ruby em-proxy-http.rb
# > curl --proxy localhost:9889 www.google.com

host = "0.0.0.0"
port = 9889
puts "listening on #{host}:#{port}..."

Proxy.start(:host => host, :port => port) do |conn|

  @p = Http::Parser.new
  @p.on_headers_complete = proc do |h|
    session = UUID.generate
    puts "New session: #{session} (#{h.inspect})"

    host, port = h['Host'].split(':')
    conn.server session, :host => host, :port => (port || 80)
    conn.relay_to_servers @buffer

    @buffer.clear
  end

  @buffer = ''

  conn.on_connect do |data,b|
    puts [:on_connect, data, b].inspect
  end

  conn.on_data do |data|
    @buffer << data
    @p << data

    data
  end

  conn.on_response do |backend, resp|
    puts [:on_response, backend, resp].inspect
    resp
  end

  conn.on_finish do |backend, name|
    puts [:on_finish, name].inspect
  end
end

It definitely works for a single request - but try using it as the proxy for Firefox. First request will pull the website down and usually get all the associated assets... but subsequent requests will definitely not render 100%... i'm not sure it might be an encoding issue...

Oh, interesting.. Is FF keeping a persistent connection to the proxy? Because if so, then if it tries to request some assets which are associated with a different IP, then we've got a problem.

This is pure speculation on my part -- need to actually play with it.

It is actually keeping a persistent - I'm sure that is the issue.

Hmm, so if its interfacing with it as an "HTTP proxy", then in theory it should send the absolute URI as part of the initial header on each request. The trick is to detect the request boundaries and start/close new backends on per request basis. In other words, this is getting more complicated..

Might actually be easier to instead of opening a TCP backend to actually open an http request (em-http) and pipe data back through the callbacks.

any updates on this ?

Nothing yet from me - been pulled into other items, will hopefully get back to this soon...

hey, thanks anyway :D

The proxy above do not handle HTTP/1.1 persistence connection.

To workaround this, i disabled the persistence connection on both client and server side (by adding Connection: close). Not a good example for implementing proxy, but it work fine as a proxy that work on modern browsers and sites. Check my fork here: https://gist.github.com/1560703

If you want an http proxy with persistent connections, etc, take a look at goliath + mongo logging example in there. What we're doing here is basically reimplementing all of the connection management that Goliath already provides. :-)

@igrigorik thanks! this seems what i'm looking for.