Clojure vs GoLang benchmark - 500 HTTP GET requests

$Clojure_vs_GoLang_benchmark_http_get.md

Clojure vs GoLang benchmark - Making 500 HTTP GET requests concurently

Context

In this Gist I wanted to test how quickly Clojure performs 500 HTTP GET requests compared to GO. Surprisingly, using the http-kit Clojure library, a high-performance HTTP client/server implemented in Clojure(and some Java), in turned out that Clojure was faster at fetching http://clojure.org/ 500 times concurently.

I chose http-kit because it is a popular Clojure http client/server, and I did not want to test using purely existing Java based http client/server, like the http client in JDK 11+ which are more painful to use from Clojure due to their API being designed for Java usage.

In both the Clojure and GO solutions, I am doing asynchronous Get calls, to maximize the performance.

I've used Clojure's most common benchmark lib called criterium, which runs the form multiple times to make the JIT hot, and runs until it starts seeing conssistent results on timing to report the proper execution mean time.

I've used Go's testing lib, availaible in its standard library to run a Benchmark, where it too, will run multiple time until it starts getting confident timings. As per something I read online, Go can aggressively remove logic that it believes is unused, so based on advice online, I had results be stored in a package level var to avoid such optimiation and have Go simple not actually run getClojure.

I've used GO processes to concurently make the 500 requests in GO.

Disclaimer

I am much more knowledgeable of Clojure, even though I did not try to optimize the Clojure version, I wrote the simplest implementation I knew. With GO, I am less knowledgeable, but similarly used the most straighforward way I knew how to implement the problem.

Winner

Clojure is the winner!

I was honestly surprised, I thought Go was going to be faster and was curious to see by how much, but Clojure turned out to be faster than GO.

• Clojure execution time : 745.907356 ms
• Go execution time : 1.269s

The benchmarks themselves run multiple times each, but I ran the benchmarks 10 times as well to see in case connection speed played a role, and everytime Clojure beat Go, and I was seeing similar-ish timings at +/- 200ms

get_clojure.clj

(ns get-clojure
  (:require [org.httpkit.client :as http]
            [org.httpkit.sni-client :as sni-client]
            [criterium.core :as cr]))

;; For JDK > 8 compatibility
(alter-var-root #'org.httpkit.client/*default-client* (fn [_] sni-client/default-client))

(cr/quick-bench
 (let [requests (mapv http/get (repeat 500 "http://clojure.org/"))
       responses (mapv deref requests)]
   (assert (every? realized? requests))
   (:body (peek responses))))

;; On my laptop and ran through a Cider nREPL started with tools.build
;; 11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz
;; 32GB Memory
;; 512GB SSD
;;
;; Evaluation count : 6 in 6 samples of 1 calls.
;;              Execution time mean : 745.907356 ms
;;     Execution time std-deviation : 319.366855 ms
;;    Execution time lower quantile : 432.409292 ms ( 2.5%)
;;    Execution time upper quantile : 1.073080 sec (97.5%)
;;                    Overhead used : 7.892455 ns

getclojure.go

package main

import (
  "fmt"
  "io/ioutil"
  "log"
  "net/http"
  "sync"
)

// Storing in package level var to avoid compiler optimizing away what isn't used
var content string

func main() {
  getClojure()
  // Printing last fetched to check it gets the same exact body as the Clojure  version
  fmt.Println(content)
}

func getClojure() int {

  const n = 500

  var urls [n]string

  for i := 0 ; i < n ; i++ {
    urls[i] = "http://clojure.org/"
  }

  var wg sync.WaitGroup

  for _, url := range urls {
    
    wg.Add(1)
    go func(url string) {
    
      defer wg.Done()
    
      content = doReq(url)

    }(url)
  }

  wg.Wait()

  return 1
}

func doReq(url string) (content string) {

  resp, err := http.Get(url)

  if err != nil {
    log.Fatal(err)
  }

  defer resp.Body.Close()

  body, err := ioutil.ReadAll(resp.Body)

  if err != nil {
    log.Fatal(err)
  }

  return string(body)
}

getclojure_test.go

package main

import (
  "testing"
)

// Storing in package level var to avoid compiler optimizing away what isn't used
var result int

func BenchmarkGetClojure(b *testing.B) {
  var r int

  // run the getClojure function b.N times
  for n := 0; n < b.N; n++ {
    r = getClojure()
  }

  result = r
}

// On my laptop and ran using: go test -bench=.
// 11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz
// 32GB Memory
// 512GB SSD
//
// goos: linux
// goarch: amd64
// pkg: bench/goclojure
// cpu: 11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz
// BenchmarkGetClojure-16                 1        1261584711 ns/op
// PASS
// ok      bench/goclojure 1.269s

mentirada