Skip to content

Instantly share code, notes, and snippets.

@hyone
Created August 2, 2012 16:05
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save hyone/3238199 to your computer and use it in GitHub Desktop.
Save hyone/3238199 to your computer and use it in GitHub Desktop.
Multiple async HTTP requests by using http.async.client callback API
;; (defproject async-test2 "0.1.0-SNAPSHOT"
;; :description "FIXME: write description"
;; :url "http://example.com/FIXME"
;; :license {:name "Eclipse Public License"
;; :url "http://www.eclipse.org/legal/epl-v10.html"}
;; :dependencies [[org.clojure/clojure "1.4.0"]
;; [http.async.client "0.4.5"]])
(ns async-test2.core
(:require [http.async.client :as client]
[http.async.client.request :refer (prepare-request execute-request)]))
(defn gen-url [keyword]
(format "http://www.google.com/search?q=%s" keyword))
(def urls (concat (map gen-url ["Clojure" "Haskell" "OCaml" "Scala" "F#"])
["http://localhost:3000/"]
(map gen-url ["Ruby" "Python" "Perl" "PHP" "JavaScript"])))
(defn async-fetch [cl urls]
(let [start (System/nanoTime)]
(doall
(for [[url i] (map vector urls (drop 1 (range)))]
(execute-request cl (prepare-request :get url)
:completed (fn [response]
(println (format "%8.3f: %2d: %s => %d"
(/ (- (System/nanoTime) start) 1000000.0)
i url (:code @(:status response)))))
:error (fn [response exception]
(println "Error: " exception)))))))
(defn -main [& args]
(with-open [cl (client/create-client)]
(let [responses (async-fetch cl urls)]
(doseq [response responses] (client/await response))))
(println "Finish.")
(System/exit 0))
;; To check async process, prepare a long time request:
;; $ plackup -p 3000 -e 'sub { sleep 5; return [200, ["Content-Type" => "text/plain"], ["Hello World."]] }'
;;
;; $ lein run -m async-test2.core
;; SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
;; SLF4J: Defaulting to no-operation (NOP) logger implementation
;; SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
;; 422.961: 5: http://www.google.com/search?q=F# => 200
;; 444.097: 11: http://www.google.com/search?q=JavaScript => 200
;; 516.303: 7: http://www.google.com/search?q=Ruby => 200
;; 526.895: 2: http://www.google.com/search?q=Haskell => 200
;; 544.540: 9: http://www.google.com/search?q=Perl => 200
;; 546.011: 1: http://www.google.com/search?q=Clojure => 200
;; 547.384: 3: http://www.google.com/search?q=OCaml => 200
;; 577.401: 10: http://www.google.com/search?q=PHP => 200
;; 583.873: 4: http://www.google.com/search?q=Scala => 200
;; 602.113: 8: http://www.google.com/search?q=Python => 200
;; 5229.313: 6: http://localhost:3000/ => 200
;; Finish.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment