Jepsen notes

jepsen.md

Setup, development

• can start a cluster of Docker containers locally for development/testing
  • see up.sh in jepsen/docker
• use these from Leiningen running locally on same machine as Docker host: https://stackoverflow.com/a/45071285/1479173
  • ie resolve Docker containers from host
  • add node host names to a file (e.g. nodes.txt) with a trailing dot and do lein run test --nodes-file ./nodes.txt

n1.
n2.
n3.
n4.
n5.

Writing tests

• Jepsen does everything including installing the software under test
• Source of installation package is usually an archive on the internet: can share a local volume to give nodes access to files directly

volumes: 
  - ./db:/opt/jepsen_db:ro

(c/su
  (let [url (str "file:///opt/jepsen_db/etcd-" version "-linux-amd64.tar.gz")]
    (cu/install-archive! url dir))

• the system under test is called the "DB"
• DB installation happens in parallel

Checking

• a model describes how the system should behave.
• a checker verifies aspects of the system, depending on the model to provide specifics. E.g. linearizability, performance.
• https://github.com/jepsen-io/jepsen/blob/master/doc/tutorial/04-checker.md
• check out the Knossos readme as well

What to check?

• you need to decide which part of the system under test to model and check
• usually no need to model the entire API
• focus on parts that are easily verifiable, e.g. sets (should never contain duplicates)

Misc

• the etcd test is probably the simplest one to copy for a new use case!

REPL

Construct a new test from the last one:

jepsen.etcdemo> (require '[jepsen.core :as jepsen]
                         '[jepsen.repl :as repl])
jepsen.etcdemo> (jepsen/run! (-> (repl/last-test "etcd") etcd-test))