Skip to content

Instantly share code, notes, and snippets.

@rkh
Last active December 11, 2015 18:49
Show Gist options
  • Save rkh/4644647 to your computer and use it in GitHub Desktop.
Save rkh/4644647 to your computer and use it in GitHub Desktop.

The Hunt for the right HTTP library

Use case: Travis CLI. One command invokes one or more sequential HTTP requests.

Note that startup time and the first request are very significant for a command line tool like this.

Clone the repo if you want to follow along.

Note: numbers might be due to how libraries are used (via faraday)

net-http

Every request is slow. This is basically unusable.

[~/Workspace/travis]$ bin/travis show --debug --adapter net-http >/dev/null
** GET "/repos/travis-ci/travis"
**   took 4.1 seconds
** GET "/builds/" {:number=>"108", :repository_id=>409371}
**   took 5.7 seconds
** GET "/jobs/4395174"
**   took 5.7 seconds
** GET "/jobs/4395175"
**   took 5.6 seconds
** GET "/jobs/4395176"
**   took 5.6 seconds
** GET "/jobs/4395177"
**   took 6 seconds
** GET "/jobs/4395178"
**   took 5.8 seconds
** GET "/jobs/4395179"
**   took 5.7 seconds

excon

All requests are slow.

[~/Workspace/travis]$ bin/travis show --debug --adapter excon >/dev/null
** GET "/repos/travis-ci/travis"
**   took 3.1 seconds
** GET "/builds/" {:number=>"108", :repository_id=>409371}
**   took 5.7 seconds
** GET "/jobs/4395174"
**   took 8.8 seconds
** GET "/jobs/4395175"
**   took 6 seconds
** GET "/jobs/4395176"
**   took 5.5 seconds
** GET "/jobs/4395177"
**   took 5.9 seconds
** GET "/jobs/4395178"
**   took 5.7 seconds
** GET "/jobs/4395179"
**   took 5.6 seconds

net-http-persistent

Only the first request is slow, any subsequent request is fast. Problem is: A lot of commands are only one request.

[~/Workspace/travis]$ bin/travis show --debug --adapter net-http-persistent >/dev/null
** GET "/repos/travis-ci/travis"
**   took 3.9 seconds
** GET "/builds/" {:number=>"108", :repository_id=>409371}
**   took 0.13 seconds
** GET "/jobs/4395174"
**   took 0.09 seconds
** GET "/jobs/4395175"
**   took 0.10 seconds
** GET "/jobs/4395176"
**   took 0.14 seconds
** GET "/jobs/4395177"
**   took 0.07 seconds
** GET "/jobs/4395178"
**   took 0.06 seconds
** GET "/jobs/4395179"
**   took 0.11 seconds

typhoeus

Only the first request is slow, any subsequent request is fast. Problem is: A lot of commands are only one request. Might also not work flawlessly on Windows.

[~/Workspace/travis]$ bin/travis show --debug --adapter typhoeus >/dev/null
** GET "/repos/travis-ci/travis"
**   took 3.1 seconds
** GET "/builds/" {:number=>"108", :repository_id=>409371}
**   took 0.16 seconds
** GET "/jobs/4395174"
**   took 0.2 seconds
** GET "/jobs/4395175"
**   took 0.15 seconds
** GET "/jobs/4395176"
**   took 0.16 seconds
** GET "/jobs/4395177"
**   took 0.18 seconds
** GET "/jobs/4395178"
**   took 0.14 seconds
** GET "/jobs/4395179"
**   took 0.14 seconds

patron

Times out.

[~/Workspace/travis]$ bin/travis show --debug --adapter patron >/dev/null
** GET "/repos/travis-ci/travis"
Connection time-out

em-http

Every request is fast, not as fast as subsequent requests with net-http-persistent, though. But I guess it won't work well on Windows.

[~/Workspace/travis]$ bin/travis show --debug --adapter em-http >/dev/null
** GET "/repos/travis-ci/travis"
**   took 0.59 seconds
** GET "/builds/" {:number=>"108", :repository_id=>409371}
**   took 0.76 seconds
** GET "/jobs/4395174"
**   took 0.56 seconds
** GET "/jobs/4395175"
**   took 0.52 seconds
** GET "/jobs/4395176"
**   took 0.57 seconds
** GET "/jobs/4395177"
**   took 0.52 seconds
** GET "/jobs/4395178"
**   took 0.62 seconds
** GET "/jobs/4395179"
**   took 0.52 seconds

em-synchrony

Every request is fast, not as fast as subsequent requests with net-http-persistent, though. But I guess it won't work well on Windows.

[~/Workspace/travis]$ bin/travis show --debug --adapter em-synchrony >/dev/null
** GET "/repos/travis-ci/travis"
**   took 0.63 seconds
** GET "/builds/" {:number=>"108", :repository_id=>409371}
**   took 0.62 seconds
** GET "/jobs/4395174"
**   took 0.6 seconds
** GET "/jobs/4395175"
**   took 0.58 seconds
** GET "/jobs/4395176"
**   took 1 seconds
** GET "/jobs/4395177"
**   took 0.84 seconds
** GET "/jobs/4395178"
**   took 0.83 seconds
** GET "/jobs/4395179"
**   took 0.69 seconds
@raggi
Copy link

raggi commented Jan 26, 2013

https://gist.github.com/fd3b2570e41a2557e07f for reference. probably will get deleted sometime..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment