Using memorize to cache CSV data

01-command.md

$ ab -n 1000 -c 10 -l http://localhost:3000/nhs-trusts

• Server Software: Jetty(7.6.13.v20130916)
• Server Hostname: localhost
• Server Port: 3000
• Document Path: /nhs-trusts
• Document Length: Variable
• Concurrency Level: 10

02-before memorize

Time taken for tests:   4.397 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      104052000 bytes
HTML transferred:       103912000 bytes
Requests per second:    227.45 [#/sec] (mean)
Time per request:       43.966 [ms] (mean)
Time per request:       4.397 [ms] (mean, across all concurrent requests)
Transfer rate:          23111.60 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    25   44  12.1     40     110
Waiting:       25   42  11.4     38      95
Total:         25   44  12.1     40     110

03-after memorize

Time taken for tests:   1.804 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      104052000 bytes
HTML transferred:       103912000 bytes
Requests per second:    554.27 [#/sec] (mean)
Time per request:       18.042 [ms] (mean)
Time per request:       1.804 [ms] (mean, across all concurrent requests)
Transfer rate:          56320.78 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     8   18   7.1     15      60
Waiting:        8   17   6.5     14      59
Total:          8   18   7.1     15      60

04-the-code.clj

;; converts a CSV file + header columns to a collection of maps
(defn csv-to-map [csv-file columns]
  (keywordize-keys (remove-empty-keys (map #(zipmap columns %1) (parse-csv (slurp csv-file)))) ))

(def cached-csv-to-map (memoize csv-to-map))