Last active
May 27, 2016 22:57
-
-
Save niquola/d12ccb2f313d2f146a3d to your computer and use it in GitHub Desktop.
naive kafka + http-kit
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ab -k -n 10000 -c 1000 localhost:3000/ | |
| This is ApacheBench, Version 2.3 <$Revision: 1604373 $> | |
| Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ | |
| Licensed to The Apache Software Foundation, http://www.apache.org/ | |
| Benchmarking localhost (be patient) | |
| Completed 1000 requests | |
| Completed 2000 requests | |
| Completed 3000 requests | |
| Completed 4000 requests | |
| Completed 5000 requests | |
| Completed 6000 requests | |
| Completed 7000 requests | |
| Completed 8000 requests | |
| Completed 9000 requests | |
| Completed 10000 requests | |
| Finished 10000 requests | |
| Server Software: http-kit | |
| Server Hostname: localhost | |
| Server Port: 3000 | |
| Document Path: / | |
| Document Length: 9 bytes | |
| Concurrency Level: 1000 | |
| Time taken for tests: 0.678 seconds | |
| Complete requests: 10000 | |
| Failed requests: 0 | |
| Keep-Alive requests: 10000 | |
| Total transferred: 1610000 bytes | |
| HTML transferred: 90000 bytes | |
| Requests per second: 14749.20 [#/sec] (mean) | |
| Time per request: 67.800 [ms] (mean) | |
| Time per request: 0.068 [ms] (mean, across all concurrent requests) | |
| Transfer rate: 2318.97 [Kbytes/sec] received | |
| Connection Times (ms) | |
| min mean[+/-sd] median max | |
| Connect: 0 1 4.1 0 21 | |
| Processing: 8 62 10.1 65 88 | |
| Waiting: 8 62 10.1 65 88 | |
| Total: 28 64 7.2 65 88 | |
| Percentage of the requests served within a certain time (ms) | |
| 50% 65 | |
| 66% 66 | |
| 75% 67 | |
| 80% 69 | |
| 90% 71 | |
| 95% 72 | |
| 98% 75 | |
| 99% 78 | |
| 100% 88 (longest request) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ab -k -n 300 -c 10 localhost:3000/ | |
| This is ApacheBench, Version 2.3 <$Revision: 1604373 $> | |
| Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ | |
| Licensed to The Apache Software Foundation, http://www.apache.org/ | |
| Benchmarking localhost (be patient) | |
| Completed 100 requests | |
| Completed 200 requests | |
| Completed 300 requests | |
| Finished 300 requests | |
| Server Software: http-kit | |
| Server Hostname: localhost | |
| Server Port: 3000 | |
| Document Path: / | |
| Document Length: 8 bytes | |
| Concurrency Level: 10 | |
| Time taken for tests: 0.084 seconds | |
| Complete requests: 300 | |
| Failed requests: 0 | |
| Keep-Alive requests: 300 | |
| Total transferred: 48000 bytes | |
| HTML transferred: 2400 bytes | |
| Requests per second: 3556.06 [#/sec] (mean) | |
| Time per request: 2.812 [ms] (mean) | |
| Time per request: 0.281 [ms] (mean, across all concurrent requests) | |
| Transfer rate: 555.63 [Kbytes/sec] received | |
| Connection Times (ms) | |
| min mean[+/-sd] median max | |
| Connect: 0 0 0.1 0 0 | |
| Processing: 1 3 1.9 2 10 | |
| Waiting: 1 3 1.9 2 10 | |
| Total: 1 3 2.0 2 10 | |
| Percentage of the requests served within a certain time (ms) | |
| 50% 2 | |
| 66% 2 | |
| 75% 3 | |
| 80% 3 | |
| 90% 6 | |
| 95% 8 | |
| 98% 9 | |
| 99% 9 | |
| 100% 10 (longest request) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| (use 'clj-kafka.consumer.zk) | |
| (use 'clj-kafka.core) | |
| (require '[clj-kafka.producer :as kp]) | |
| (require '[org.httpkit.server :refer :all]) | |
| (def config {"zookeeper.connect" "127.0.0.1:2181" | |
| "group.id" "clj-kafka.consumer" | |
| "auto.offset.reset" "smallest" | |
| "auto.commit.enable" "false"}) | |
| (def p (kp/producer {"metadata.broker.list" "127.0.0.1:9092" | |
| "serializer.class" "kafka.serializer.DefaultEncoder" | |
| "partitioner.class" "kafka.producer.DefaultPartitioner"})) | |
| (def channel-hub (atom {})) | |
| (def channel-name "request-7") | |
| (defn handler [request] | |
| (with-channel request channel | |
| (let [id (str (gensym))] | |
| (kp/send-message p (kp/message channel-name (.getBytes id))) | |
| (swap! channel-hub assoc id channel) | |
| (on-close channel (fn [status] (swap! channel-hub dissoc id)))))) | |
| (defn value [msg] (java.lang.String. (:value msg))) | |
| (def cnt (atom 0)) | |
| (defn consume [] | |
| (with-resource [c (consumer config)] | |
| shutdown | |
| (doseq [x (messages c channel-name)] | |
| (let [id (value x)] | |
| (try | |
| (when-let [ch (get @channel-hub id)] | |
| (swap! cnt inc) | |
| (println @cnt) | |
| (send! ch {:status 200 | |
| :headers {"Content-Type" "text; charset=utf-8"} | |
| :body id}) | |
| (close ch))))))) | |
| (defn start [] | |
| (run-server #'handler {:port 3000}) | |
| (future (consume)) | |
| (future (consume)) | |
| (future (consume)) | |
| (future (consume))) | |
| (comment | |
| (def stop (start))) | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| (use 'clj-kafka.core) | |
| (require '[clj-kafka.producer :as kp]) | |
| (require '[org.httpkit.server :refer :all]) | |
| (def config {"zookeeper.connect" "127.0.0.1:2181" | |
| "group.id" "clj-kafka.consumer" | |
| "auto.offset.reset" "smallest" | |
| "auto.commit.enable" "false"}) | |
| (def p (kp/producer {"metadata.broker.list" "127.0.0.1:9092" | |
| "serializer.class" "kafka.serializer.DefaultEncoder" | |
| "partitioner.class" "kafka.producer.DefaultPartitioner"})) | |
| (def channel-hub (atom {})) | |
| (def channel-name "request-6") | |
| (defn handler [request] | |
| (with-channel request channel | |
| (let [id (str (gensym))] | |
| (kp/send-message p (kp/message channel-name (.getBytes id))) | |
| (swap! channel-hub assoc id channel) | |
| (on-close channel (fn [status] (swap! channel-hub dissoc id)))))) | |
| (defn value [msg] (java.lang.String. (:value msg))) | |
| (def cnt (atom 0)) | |
| (defn start [] | |
| (run-server #'handler {:port 3000}) | |
| (with-resource [c (consumer config)] | |
| shutdown | |
| (doseq [x (messages c channel-name)] | |
| (let [id (value x)] | |
| (try | |
| (when-let [ch (get @channel-hub id)] | |
| (swap! cnt inc) | |
| (println @cnt) | |
| (send! ch {:status 200 | |
| :headers {"Content-Type" "text; charset=utf-8"} | |
| :body id}) | |
| (close ch))))))) | |
| (comment | |
| (def stop (start))) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Но периодически залипает (все получается в один поток) - можно пул запустить