Skip to content

Instantly share code, notes, and snippets.



Last active Feb 25, 2021
What would you like to do?
Racket performance improvement over 17 years

Arseny Kapoulkine (h/t John Regehr) wrote a nice exploration of how LLVM from 2010 compares to the performance we get today. I decided to try out an old version of Racket, and compare performance to a modern version.

The versions I compared were:

  • MzScheme 206p1, released January 2004
  • Racket, compiled February 2021 (using the Chez Scheme backend)

I measured exactly one benchmark, a mandelbrot set computation from the Computer Language Benchmarks Game, using the implementation in the Racket repository. Of course, that code won't run on v206p1, so I modified it to avoid new features of Racket. The modified version is below.

Of course, many of the features that have been added make the program faster, so it's important to compare with the performance of the modern code. There's even a parallel version that we can test.

Here are the results, measured with hyperfine on a 6-core i7-6700 Linux machine:

Benchmark MzS 206p1 Racket 18.570 s 1.184 s
mandelbrot.rkt 0.255 s
mandelbrot-futures.rkt 0.125 s

On this benchmark, 17 years of compiler improvement produced 15.7x speedup. That's a doubling time of about 4.3 years, rather than the 18 of Proebsting's Law. Further language improvements get us to 148.6x speedup (or 72x if we don't count parallelism), for a doubling time of 2.4 years (or 2.7). That's not quite Moore's law, but it's getting close.

This emphasizes John's point that compiler work doesn't mostly go into making code from 20 years ago run faster, but enabling new code to be fast, and often faster than the old code could be.

The other lesson is that big performance improvements require big changes. Racket today is implemented totally differently from Racket of 2004, and the speedups seen here would not be possible without those changes.

(module mandelbrot-old mzscheme
(define (write-byte b o) (write-char (integer->char b) o))
(define (->fl x) (if (inexact? x) (fx->fl x) x))
(define O (current-output-port))
(define fx->fl exact->inexact)
(define LIMIT-SQR 4.0)
(define ITERATIONS 50)
(define N (string->number (vector-ref (current-command-line-arguments) 0)))
(define N.0 (fx->fl N))
(define 2/N (/ 2.0 N.0))
(define Crs
(let ([v (make-vector N)])
(let loop ([x 0])
(when (< x N)
(vector-set! v x (- (/ (fx->fl (* 2 x)) N.0) 1.5))
(loop (+ x 1))))
(define-syntax mandelbrot
(syntax-rules ()
[(mandelbrot Cr Ci)
(let loop ([i 0] [Zr 0.0] [Zi 0.0])
(cond [(> (+ (* Zr Zr) (* Zi Zi)) LIMIT-SQR) 0]
[(= i ITERATIONS) 1]
[else (let ([Zr (+ (- (* Zr Zr) (* Zi Zi)) Cr)]
[Zi (+ (* 2.0 (* Zr Zi)) Ci)])
(loop (+ i 1) Zr Zi))]))]))
(fprintf O "P4\n~a ~a\n" N N)
(let loop-y ([y N])
(let ([Ci (- (* 2/N (->fl y)) 1.0)])
(let loop-x ([x 0] [bitnum 0] [byteacc 0])
(if (< x N)
(let* ([Cr (vector-ref Crs x)]
[bitnum (+ bitnum 1)]
[byteacc (+ (arithmetic-shift byteacc 1) (mandelbrot Cr Ci))])
(cond [(= bitnum 8)
(write-byte byteacc O)
(loop-x (+ x 1) 0 0)]
[else (loop-x (+ x 1) bitnum byteacc)]))
(begin (when (> bitnum 0)
(write-byte (arithmetic-shift byteacc (- 8 (and N #x7))) O))
(when (> y 1) (loop-y (- y 1))))))))

This comment has been minimized.

Copy link

@austinwk austinwk commented Feb 2, 2021

Concise, well-written, and informative. Excellent work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment