Not knowing that docker rm would just skip running containers I came up with the following:
docker ps -a | grep -v "Up" | cut -f 1 -d ' ' | tail -n +2 | xargs docker rm
but the docker docs say:
docker rm $(docker ps -a -q)
which is probably better as it doesn't depend on inverse grepping for "Up".