benbjohnson/race.txt

## race.txt
$ go test -v -race
=== RUN TestFileSnapshotStoreImpl
--- PASS: TestFileSnapshotStoreImpl (0.00 seconds)
=== RUN TestFileSnapshotSinkImpl
--- PASS: TestFileSnapshotSinkImpl (0.00 seconds)
=== RUN TestFileSS_CreateSnapshot
2014/04/29 16:03:28 [INFO] snapshot: Creating new snapshot at /var/folders/p3/nqt2bgp17fn7_t9hr6tr_1m00000gn/T/raft166453656/snapshots/3-10-2014-04-29T16:03:28.194235901-06:00.tmp
2014/04/29 16:03:28 [WARN] snapshot: Found temporary snapshot: 3-10-2014-04-29T16:03:28.194235901-06:00.tmp
--- PASS: TestFileSS_CreateSnapshot (0.00 seconds)
=== RUN TestFileSS_CancelSnapshot
2014/04/29 16:03:28 [INFO] snapshot: Creating new snapshot at /var/folders/p3/nqt2bgp17fn7_t9hr6tr_1m00000gn/T/raft614393879/snapshots/3-10-2014-04-29T16:03:28.197887586-06:00.tmp
--- PASS: TestFileSS_CancelSnapshot (0.00 seconds)
=== RUN TestFileSS_Retention
2014/04/29 16:03:28 [INFO] snapshot: Creating new snapshot at /var/folders/p3/nqt2bgp17fn7_t9hr6tr_1m00000gn/T/raft991497354/snapshots/3-10-2014-04-29T16:03:28.199131737-06:00.tmp
2014/04/29 16:03:28 [INFO] snapshot: Creating new snapshot at /var/folders/p3/nqt2bgp17fn7_t9hr6tr_1m00000gn/T/raft991497354/snapshots/3-11-2014-04-29T16:03:28.200176006-06:00.tmp
2014/04/29 16:03:28 [INFO] snapshot: Creating new snapshot at /var/folders/p3/nqt2bgp17fn7_t9hr6tr_1m00000gn/T/raft991497354/snapshots/3-12-2014-04-29T16:03:28.201376038-06:00.tmp
2014/04/29 16:03:28 [INFO] snapshot: reaping snapshot /var/folders/p3/nqt2bgp17fn7_t9hr6tr_1m00000gn/T/raft991497354/snapshots/3-10-2014-04-29T16:03:28.199131737-06:00
2014/04/29 16:03:28 [INFO] snapshot: Creating new snapshot at /var/folders/p3/nqt2bgp17fn7_t9hr6tr_1m00000gn/T/raft991497354/snapshots/3-13-2014-04-29T16:03:28.203346163-06:00.tmp
2014/04/29 16:03:28 [INFO] snapshot: reaping snapshot /var/folders/p3/nqt2bgp17fn7_t9hr6tr_1m00000gn/T/raft991497354/snapshots/3-11-2014-04-29T16:03:28.200176006-06:00
2014/04/29 16:03:28 [INFO] snapshot: Creating new snapshot at /var/folders/p3/nqt2bgp17fn7_t9hr6tr_1m00000gn/T/raft991497354/snapshots/3-14-2014-04-29T16:03:28.204967537-06:00.tmp
2014/04/29 16:03:28 [INFO] snapshot: reaping snapshot /var/folders/p3/nqt2bgp17fn7_t9hr6tr_1m00000gn/T/raft991497354/snapshots/3-12-2014-04-29T16:03:28.201376038-06:00
--- PASS: TestFileSS_Retention (0.01 seconds)
=== RUN TestFileSS_BadPerm
--- PASS: TestFileSS_BadPerm (0.00 seconds)
=== RUN TestFileSS_Names
--- PASS: TestFileSS_Names (0.00 seconds)
=== RUN TestFileSS_Ordering
2014/04/29 16:03:28 [INFO] snapshot: Creating new snapshot at /var/folders/p3/nqt2bgp17fn7_t9hr6tr_1m00000gn/T/raft107346529/snapshots/5-130350-2014-04-29T16:03:28.208340748-06:00.tmp
2014/04/29 16:03:28 [INFO] snapshot: Creating new snapshot at /var/folders/p3/nqt2bgp17fn7_t9hr6tr_1m00000gn/T/raft107346529/snapshots/36-204917-2014-04-29T16:03:28.209223391-06:00.tmp
--- PASS: TestFileSS_Ordering (0.00 seconds)
=== RUN TestInflight_StartCommit
--- PASS: TestInflight_StartCommit (0.00 seconds)
=== RUN TestInflight_Cancel
--- PASS: TestInflight_Cancel (0.00 seconds)
=== RUN TestInflight_CommitRange
--- PASS: TestInflight_CommitRange (0.00 seconds)
=== RUN TestInflight_NonContiguous
--- PASS: TestInflight_NonContiguous (0.00 seconds)
=== RUN TestInmemAddrImpl
--- PASS: TestInmemAddrImpl (0.00 seconds)
=== RUN TestInmemAddr
--- PASS: TestInmemAddr (0.00 seconds)
=== RUN TestInmemTransportImpl
--- PASS: TestInmemTransportImpl (0.00 seconds)
=== RUN TestRaft_Integ
--- SKIP: TestRaft_Integ (0.00 seconds)
=== RUN TestNetworkTransport_StartStop
==================
WARNING: DATA RACE
Read by goroutine 22:
  github.com/benbjohnson/raft.(*NetworkTransport).listen()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport.go:277 +0xaa

Previous write by goroutine 21:
  github.com/benbjohnson/raft.(*NetworkTransport).Close()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport.go:118 +0x106
  github.com/benbjohnson/raft.TestNetworkTransport_StartStop()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport_test.go:16 +0x18c
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Goroutine 22 (running) created at:
  github.com/benbjohnson/raft.NewNetworkTransport()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport.go:106 +0x2e2
  github.com/benbjohnson/raft.NewTCPTransport()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/tcp_transport.go:44 +0x2c0
  github.com/benbjohnson/raft.TestNetworkTransport_StartStop()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport_test.go:12 +0x76
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Goroutine 21 (finished) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc
==================
--- PASS: TestNetworkTransport_StartStop (0.00 seconds)
=== RUN TestNetworkTransport_AppendEntries
2014/04/29 16:03:28 [DEBUG] raft-net: 127.0.0.1:57714 accepted connection from: 127.0.0.1:57716
==================
WARNING: DATA RACE
Read by goroutine 26:
  github.com/benbjohnson/raft.(*NetworkTransport).listen()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport.go:277 +0xaa

Previous write by goroutine 23:
  github.com/benbjohnson/raft.(*NetworkTransport).Close()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport.go:118 +0x106
  github.com/benbjohnson/raft.TestNetworkTransport_AppendEntries()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport_test.go:82 +0xc33
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Goroutine 26 (running) created at:
  github.com/benbjohnson/raft.NewNetworkTransport()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport.go:106 +0x2e2
  github.com/benbjohnson/raft.NewTCPTransport()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/tcp_transport.go:44 +0x2c0
  github.com/benbjohnson/raft.TestNetworkTransport_AppendEntries()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport_test.go:67 +0x6db
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Goroutine 23 (finished) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc
==================
--- PASS: TestNetworkTransport_AppendEntries (0.00 seconds)
=== RUN TestNetworkTransport_RequestVote
2014/04/29 16:03:28 [DEBUG] raft-net: 127.0.0.1:57717 accepted connection from: 127.0.0.1:57719
==================
WARNING: DATA RACE
Read by goroutine 32:
  github.com/benbjohnson/raft.(*NetworkTransport).listen()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport.go:277 +0xaa

Previous write by goroutine 29:
  github.com/benbjohnson/raft.(*NetworkTransport).Close()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport.go:118 +0x106
  github.com/benbjohnson/raft.TestNetworkTransport_RequestVote()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport_test.go:139 +0xb93
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Goroutine 32 (running) created at:
  github.com/benbjohnson/raft.NewNetworkTransport()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport.go:106 +0x2e2
  github.com/benbjohnson/raft.NewTCPTransport()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/tcp_transport.go:44 +0x2c0
  github.com/benbjohnson/raft.TestNetworkTransport_RequestVote()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport_test.go:124 +0x62e
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Goroutine 29 (finished) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc
==================
--- PASS: TestNetworkTransport_RequestVote (0.00 seconds)
=== RUN TestNetworkTransport_InstallSnapshot
2014/04/29 16:03:28 [DEBUG] raft-net: 127.0.0.1:57720 accepted connection from: 127.0.0.1:57722
==================
WARNING: DATA RACE
Read by goroutine 38:
  github.com/benbjohnson/raft.(*NetworkTransport).listen()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport.go:277 +0xaa

Previous write by goroutine 35:
  github.com/benbjohnson/raft.(*NetworkTransport).Close()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport.go:118 +0x106
  github.com/benbjohnson/raft.TestNetworkTransport_InstallSnapshot()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport_test.go:209 +0xde7
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Goroutine 38 (running) created at:
  github.com/benbjohnson/raft.NewNetworkTransport()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport.go:106 +0x2e2
  github.com/benbjohnson/raft.NewTCPTransport()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/tcp_transport.go:44 +0x2c0
  github.com/benbjohnson/raft.TestNetworkTransport_InstallSnapshot()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport_test.go:191 +0x664
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Goroutine 35 (finished) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc
==================
--- PASS: TestNetworkTransport_InstallSnapshot (0.00 seconds)
=== RUN TestNetworkTransport_EncodeDecode
==================
WARNING: DATA RACE
Read by goroutine 41:
  github.com/benbjohnson/raft.(*NetworkTransport).listen()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport.go:277 +0xaa

Previous write by goroutine 40:
  github.com/benbjohnson/raft.(*NetworkTransport).Close()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport.go:118 +0x106
  github.com/benbjohnson/raft.TestNetworkTransport_EncodeDecode()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport_test.go:226 +0x2a9
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Goroutine 41 (running) created at:
  github.com/benbjohnson/raft.NewNetworkTransport()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport.go:106 +0x2e2
  github.com/benbjohnson/raft.NewTCPTransport()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/tcp_transport.go:44 +0x2c0
  github.com/benbjohnson/raft.TestNetworkTransport_EncodeDecode()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport_test.go:213 +0x76
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Goroutine 40 (finished) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc
==================
--- PASS: TestNetworkTransport_EncodeDecode (0.00 seconds)
=== RUN TestNetworkTransport_PooledConn
2014/04/29 16:03:28 [DEBUG] raft-net: 127.0.0.1:57724 accepted connection from: 127.0.0.1:57726
2014/04/29 16:03:28 [DEBUG] raft-net: 127.0.0.1:57724 accepted connection from: 127.0.0.1:57727
2014/04/29 16:03:28 [DEBUG] raft-net: 127.0.0.1:57724 accepted connection from: 127.0.0.1:57728
2014/04/29 16:03:28 [DEBUG] raft-net: 127.0.0.1:57724 accepted connection from: 127.0.0.1:57729
2014/04/29 16:03:28 [DEBUG] raft-net: 127.0.0.1:57724 accepted connection from: 127.0.0.1:57730
==================
WARNING: DATA RACE
Read by goroutine 45:
  github.com/benbjohnson/raft.(*NetworkTransport).listen()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport.go:277 +0xaa

Previous write by goroutine 42:
  github.com/benbjohnson/raft.(*NetworkTransport).Close()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport.go:118 +0x106
  github.com/benbjohnson/raft.TestNetworkTransport_PooledConn()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport_test.go:313 +0xbff
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Goroutine 45 (running) created at:
  github.com/benbjohnson/raft.NewNetworkTransport()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport.go:106 +0x2e2
  github.com/benbjohnson/raft.NewTCPTransport()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/tcp_transport.go:44 +0x2c0
  github.com/benbjohnson/raft.TestNetworkTransport_PooledConn()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/net_transport_test.go:277 +0x74b
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Goroutine 42 (finished) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc
==================
--- PASS: TestNetworkTransport_PooledConn (0.00 seconds)
=== RUN TestJSONPeers
--- PASS: TestJSONPeers (0.00 seconds)
=== RUN TestRaft_StartStop
2014/04/29 16:03:28 [WARN] Fully Connecting
--- PASS: TestRaft_StartStop (0.01 seconds)
=== RUN TestRaft_AfterShutdown
2014/04/29 16:03:28 [WARN] Fully Connecting
--- PASS: TestRaft_AfterShutdown (0.01 seconds)
=== RUN TestRaft_SingleNode
2014/04/29 16:03:28 [WARN] Fully Connecting
2014/04/29 16:03:28 [INFO] raft: Node at 90a72a7a-630c-3976-95f4-7605e776f2a6 [Follower] entering Follower state
2014/04/29 16:03:28 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:28 [INFO] raft: Node at 90a72a7a-630c-3976-95f4-7605e776f2a6 [Candidate] entering Candidate state
2014/04/29 16:03:28 [DEBUG] raft: Votes needed: 1
2014/04/29 16:03:28 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:28 [INFO] raft: Election won. Tally: 1
2014/04/29 16:03:28 [INFO] raft: Node at 90a72a7a-630c-3976-95f4-7605e776f2a6 [Leader] entering Leader state
--- PASS: TestRaft_SingleNode (0.02 seconds)
=== RUN TestRaft_TripleNode
2014/04/29 16:03:28 [WARN] Fully Connecting
2014/04/29 16:03:28 [INFO] raft: Node at 986cdd0b-5841-a3f7-0014-2a4709b0c438 [Follower] entering Follower state
2014/04/29 16:03:28 [INFO] raft: Node at 6ed23dd1-cbf6-6791-6bab-e03b559cb9d7 [Follower] entering Follower state
2014/04/29 16:03:28 [INFO] raft: Node at 062d42cf-8960-ff18-7e13-b3ea70af6187 [Follower] entering Follower state
2014/04/29 16:03:28 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:28 [INFO] raft: Node at 062d42cf-8960-ff18-7e13-b3ea70af6187 [Candidate] entering Candidate state
2014/04/29 16:03:28 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:28 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:28 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:28 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:28 [INFO] raft: Node at 062d42cf-8960-ff18-7e13-b3ea70af6187 [Leader] entering Leader state
==================
WARNING: DATA RACE
Read by goroutine 69:
  github.com/benbjohnson/raft.(*cluster).EnsureLeader()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:164 +0x113
  github.com/benbjohnson/raft.TestRaft_TripleNode()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:386 +0xdf
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 70:
  github.com/benbjohnson/raft.(*Raft).appendEntries()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:1062 +0x3ee
  github.com/benbjohnson/raft.(*Raft).processRPC()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:1023 +0x1b3
  github.com/benbjohnson/raft.(*Raft).runFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:516 +0x1f8
  github.com/benbjohnson/raft.(*Raft).run()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:499 +0xd2
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.run)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 69 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 70 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0xd8b
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_TripleNode()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:381 +0x54
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
==================
WARNING: DATA RACE
Read by goroutine 69:
  github.com/benbjohnson/raft.(*InmemAddr).String()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/inmem_transport.go:27 +0x35
  github.com/benbjohnson/raft.(*cluster).EnsureLeader()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:174 +0x2eb
  github.com/benbjohnson/raft.TestRaft_TripleNode()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:386 +0xdf
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 70:
  github.com/benbjohnson/raft.(*InmemTransport).DecodePeer()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/inmem_transport.go:134 +0x47
  github.com/benbjohnson/raft.(*Raft).appendEntries()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:1062 +0x3ba
  github.com/benbjohnson/raft.(*Raft).processRPC()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:1023 +0x1b3
  github.com/benbjohnson/raft.(*Raft).runFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:516 +0x1f8
  github.com/benbjohnson/raft.(*Raft).run()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:499 +0xd2
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.run)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 69 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 70 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0xd8b
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_TripleNode()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:381 +0x54
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
==================
WARNING: DATA RACE
Write by goroutine 82:
  github.com/benbjohnson/raft.(*Raft).replicateTo()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/replication.go:158 +0x11cb
  github.com/benbjohnson/raft.(*Raft).replicate()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/replication.go:70 +0x42b
  github.com/benbjohnson/raft.func·003()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:687 +0x6d
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Previous write by goroutine 84:
  github.com/benbjohnson/raft.(*Raft).heartbeat()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/replication.go:311 +0x55b
  github.com/benbjohnson/raft.func·006()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/replication.go:56 +0x91
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 82 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.(*Raft).startReplication()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:687 +0x430
  github.com/benbjohnson/raft.(*Raft).runLeader()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:661 +0x479
  github.com/benbjohnson/raft.(*Raft).run()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:503 +0x106
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.run)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 84 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.(*Raft).replicate()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/replication.go:56 +0x188
  github.com/benbjohnson/raft.func·003()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:687 +0x6d
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b
==================
==================
WARNING: DATA RACE
Read by goroutine 69:
  github.com/benbjohnson/raft.TestRaft_TripleNode()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:399 +0x370
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 71:
  github.com/benbjohnson/raft.(*MockFSM).Apply()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:30 +0x175
  github.com/benbjohnson/raft.(*Raft).runFSM()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:465 +0xe73
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.runFSM)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 69 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 71 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0xdf9
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_TripleNode()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:381 +0x54
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
--- PASS: TestRaft_TripleNode (0.03 seconds)
=== RUN TestRaft_LeaderFail
2014/04/29 16:03:28 [WARN] Fully Connecting
2014/04/29 16:03:28 [INFO] raft: Node at 7e4c8936-42e8-a1c2-089e-06cc8e65dbb9 [Follower] entering Follower state
2014/04/29 16:03:28 [INFO] raft: Node at 4c3b38ae-7136-a4bb-60e7-9b44fc0bc912 [Follower] entering Follower state
2014/04/29 16:03:28 [INFO] raft: Node at 17430b0b-6111-a2ec-a578-34b9fdbefc0d [Follower] entering Follower state
2014/04/29 16:03:28 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:28 [INFO] raft: Node at 17430b0b-6111-a2ec-a578-34b9fdbefc0d [Candidate] entering Candidate state
2014/04/29 16:03:28 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:28 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:28 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:28 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:28 [INFO] raft: Node at 17430b0b-6111-a2ec-a578-34b9fdbefc0d [Leader] entering Leader state
2014/04/29 16:03:28 [INFO] Disconnecting Node at 17430b0b-6111-a2ec-a578-34b9fdbefc0d [Leader]
2014/04/29 16:03:28 [WARN] Disconnecting 17430b0b-6111-a2ec-a578-34b9fdbefc0d
2014/04/29 16:03:28 [ERR] raft: Failed to heartbeat to 7e4c8936-42e8-a1c2-089e-06cc8e65dbb9: Failed to connect to peer: 7e4c8936-42e8-a1c2-089e-06cc8e65dbb9
2014/04/29 16:03:28 [ERR] raft: Failed to AppendEntries to 7e4c8936-42e8-a1c2-089e-06cc8e65dbb9: Failed to connect to peer: 7e4c8936-42e8-a1c2-089e-06cc8e65dbb9
2014/04/29 16:03:28 [ERR] raft: Failed to heartbeat to 4c3b38ae-7136-a4bb-60e7-9b44fc0bc912: Failed to connect to peer: 4c3b38ae-7136-a4bb-60e7-9b44fc0bc912
2014/04/29 16:03:28 [ERR] raft: Failed to AppendEntries to 4c3b38ae-7136-a4bb-60e7-9b44fc0bc912: Failed to connect to peer: 4c3b38ae-7136-a4bb-60e7-9b44fc0bc912
2014/04/29 16:03:28 [ERR] raft: Failed to heartbeat to 7e4c8936-42e8-a1c2-089e-06cc8e65dbb9: Failed to connect to peer: 7e4c8936-42e8-a1c2-089e-06cc8e65dbb9
2014/04/29 16:03:28 [ERR] raft: Failed to heartbeat to 4c3b38ae-7136-a4bb-60e7-9b44fc0bc912: Failed to connect to peer: 4c3b38ae-7136-a4bb-60e7-9b44fc0bc912
2014/04/29 16:03:28 [ERR] raft: Failed to AppendEntries to 7e4c8936-42e8-a1c2-089e-06cc8e65dbb9: Failed to connect to peer: 7e4c8936-42e8-a1c2-089e-06cc8e65dbb9
2014/04/29 16:03:28 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:28 [INFO] raft: Node at 4c3b38ae-7136-a4bb-60e7-9b44fc0bc912 [Candidate] entering Candidate state
2014/04/29 16:03:28 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:28 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:28 [ERR] raft: Failed to AppendEntries to 4c3b38ae-7136-a4bb-60e7-9b44fc0bc912: Failed to connect to peer: 4c3b38ae-7136-a4bb-60e7-9b44fc0bc912
2014/04/29 16:03:28 [ERR] raft: Failed to make RequestVote RPC to 17430b0b-6111-a2ec-a578-34b9fdbefc0d: Failed to connect to peer: 17430b0b-6111-a2ec-a578-34b9fdbefc0d
2014/04/29 16:03:28 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:28 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:28 [INFO] raft: Node at 4c3b38ae-7136-a4bb-60e7-9b44fc0bc912 [Leader] entering Leader state
2014/04/29 16:03:28 [ERR] raft: Failed to AppendEntries to 17430b0b-6111-a2ec-a578-34b9fdbefc0d: Failed to connect to peer: 17430b0b-6111-a2ec-a578-34b9fdbefc0d
2014/04/29 16:03:28 [ERR] raft: Failed to heartbeat to 17430b0b-6111-a2ec-a578-34b9fdbefc0d: Failed to connect to peer: 17430b0b-6111-a2ec-a578-34b9fdbefc0d
2014/04/29 16:03:28 [INFO] Reconnecting Node at 17430b0b-6111-a2ec-a578-34b9fdbefc0d [Leader]
2014/04/29 16:03:28 [WARN] Fully Connecting
2014/04/29 16:03:28 [ERR] raft: peer 7e4c8936-42e8-a1c2-089e-06cc8e65dbb9 has newer term, stopping replication
2014/04/29 16:03:28 [ERR] raft: peer 4c3b38ae-7136-a4bb-60e7-9b44fc0bc912 has newer term, stopping replication
2014/04/29 16:03:28 [WARN] raft: Clearing log suffix from 3 to 3
2014/04/29 16:03:28 [INFO] raft: Node at 17430b0b-6111-a2ec-a578-34b9fdbefc0d [Follower] entering Follower state
==================
WARNING: DATA RACE
Read by goroutine 4:
  github.com/benbjohnson/raft.TestRaft_LeaderFail()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:475 +0xc5c
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 6:
  github.com/benbjohnson/raft.(*MockFSM).Apply()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:30 +0x175
  github.com/benbjohnson/raft.(*Raft).runFSM()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:465 +0xe73
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.runFSM)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 4 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 6 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0xdf9
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_LeaderFail()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:407 +0x57
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
==================
WARNING: DATA RACE
Read by goroutine 4:
  github.com/benbjohnson/raft.TestRaft_LeaderFail()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:481 +0x105e
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 6:
  github.com/benbjohnson/raft.(*MockFSM).Apply()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:30 +0x139
  github.com/benbjohnson/raft.(*Raft).runFSM()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:465 +0xe73
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.runFSM)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 4 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 6 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0xdf9
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_LeaderFail()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:407 +0x57
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
--- PASS: TestRaft_LeaderFail (0.07 seconds)
=== RUN TestRaft_BehindFollower
2014/04/29 16:03:28 [WARN] Fully Connecting
2014/04/29 16:03:28 [INFO] raft: Node at 22972c68-0250-da53-37f5-1ddc9237fde2 [Follower] entering Follower state
2014/04/29 16:03:28 [INFO] raft: Node at a1f43684-bd37-e22c-515c-875f3cc63d07 [Follower] entering Follower state
2014/04/29 16:03:28 [INFO] raft: Node at 10e58e6d-37aa-4fce-95c4-f58f4212c128 [Follower] entering Follower state
2014/04/29 16:03:28 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:28 [INFO] raft: Node at 22972c68-0250-da53-37f5-1ddc9237fde2 [Candidate] entering Candidate state
2014/04/29 16:03:28 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:28 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:28 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:28 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:28 [INFO] raft: Node at 22972c68-0250-da53-37f5-1ddc9237fde2 [Leader] entering Leader state
2014/04/29 16:03:28 [WARN] Disconnecting a1f43684-bd37-e22c-515c-875f3cc63d07
2014/04/29 16:03:28 [ERR] raft: Failed to AppendEntries to a1f43684-bd37-e22c-515c-875f3cc63d07: Failed to connect to peer: a1f43684-bd37-e22c-515c-875f3cc63d07
2014/04/29 16:03:28 [ERR] raft: Failed to heartbeat to a1f43684-bd37-e22c-515c-875f3cc63d07: Failed to connect to peer: a1f43684-bd37-e22c-515c-875f3cc63d07
==================
WARNING: DATA RACE
Write by goroutine 36:
  github.com/benbjohnson/raft.(*Raft).replicateTo()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/replication.go:158 +0x11cb
  github.com/benbjohnson/raft.(*Raft).replicate()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/replication.go:68 +0x328
  github.com/benbjohnson/raft.func·003()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:687 +0x6d
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Previous write by goroutine 41:
  github.com/benbjohnson/raft.(*Raft).heartbeat()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/replication.go:311 +0x55b
  github.com/benbjohnson/raft.func·006()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/replication.go:56 +0x91
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 36 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.(*Raft).startReplication()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:687 +0x430
  github.com/benbjohnson/raft.(*Raft).runLeader()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:661 +0x479
  github.com/benbjohnson/raft.(*Raft).run()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:503 +0x106
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.run)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 41 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.(*Raft).replicate()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/replication.go:56 +0x188
  github.com/benbjohnson/raft.func·003()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:687 +0x6d
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b
==================
2014/04/29 16:03:28 [INFO] Finished apply without behind follower
2014/04/29 16:03:28 [WARN] Fully Connecting
==================
WARNING: DATA RACE
Write by goroutine 28:
  github.com/benbjohnson/raft.(*MockFSM).Apply()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:30 +0x175
  github.com/benbjohnson/raft.(*Raft).runFSM()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:465 +0xe73
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.runFSM)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Previous read by goroutine 22:
  github.com/benbjohnson/raft.(*cluster).EnsureSame()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:200 +0x1a1
  github.com/benbjohnson/raft.TestRaft_BehindFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:520 +0x45b
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Goroutine 28 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0xdf9
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_BehindFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:489 +0x54
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Goroutine 22 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc
==================
==================
WARNING: DATA RACE
Read by goroutine 22:
  github.com/benbjohnson/raft.(*cluster).EnsureSame()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:210 +0x3e6
  github.com/benbjohnson/raft.TestRaft_BehindFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:520 +0x45b
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 28:
  github.com/benbjohnson/raft.(*MockFSM).Apply()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:30 +0x139
  github.com/benbjohnson/raft.(*Raft).runFSM()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:465 +0xe73
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.runFSM)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 22 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 28 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0xdf9
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_BehindFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:489 +0x54
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
==================
WARNING: DATA RACE
Read by goroutine 22:
  github.com/benbjohnson/raft.(*InmemAddr).String()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/inmem_transport.go:27 +0x35
  github.com/benbjohnson/raft.(*cluster).EnsureLeader()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:174 +0x2eb
  github.com/benbjohnson/raft.TestRaft_BehindFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:524 +0x4bd
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 24:
  github.com/benbjohnson/raft.(*InmemTransport).DecodePeer()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/inmem_transport.go:134 +0x47
  github.com/benbjohnson/raft.(*Raft).appendEntries()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:1062 +0x3ba
  github.com/benbjohnson/raft.(*Raft).processRPC()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:1023 +0x1b3
  github.com/benbjohnson/raft.(*Raft).runFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:516 +0x1f8
  github.com/benbjohnson/raft.(*Raft).run()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:499 +0xd2
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.run)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 22 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 24 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0xd8b
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_BehindFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:489 +0x54
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
2014/04/29 16:03:28 [ERR] raft: Failed to heartbeat to a1f43684-bd37-e22c-515c-875f3cc63d07: command timed out
--- PASS: TestRaft_BehindFollower (0.10 seconds)
=== RUN TestRaft_ApplyNonLeader
2014/04/29 16:03:28 [WARN] Fully Connecting
2014/04/29 16:03:28 [INFO] raft: Node at 2527c501-ba53-0262-00c9-e37861b1585c [Follower] entering Follower state
2014/04/29 16:03:28 [INFO] raft: Node at a3b73532-1806-40dd-cc80-f8333f353947 [Follower] entering Follower state
2014/04/29 16:03:28 [INFO] raft: Node at acf7f4f7-f6cc-4055-8389-389c2e5cceb6 [Follower] entering Follower state
2014/04/29 16:03:28 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:28 [INFO] raft: Node at acf7f4f7-f6cc-4055-8389-389c2e5cceb6 [Candidate] entering Candidate state
2014/04/29 16:03:28 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:28 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:28 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:28 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:28 [INFO] raft: Node at acf7f4f7-f6cc-4055-8389-389c2e5cceb6 [Leader] entering Leader state
--- PASS: TestRaft_ApplyNonLeader (0.03 seconds)
=== RUN TestRaft_ApplyConcurrent
2014/04/29 16:03:28 [WARN] Fully Connecting
2014/04/29 16:03:28 [INFO] raft: Node at 1068b880-07ad-dcbb-26d5-efb77758bfb1 [Follower] entering Follower state
2014/04/29 16:03:28 [INFO] raft: Node at 7998bdd3-849a-0f2d-90f5-430396fba624 [Follower] entering Follower state
2014/04/29 16:03:28 [INFO] raft: Node at 12583754-04b2-f5fd-543e-bd4142527c27 [Follower] entering Follower state
2014/04/29 16:03:28 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:28 [INFO] raft: Node at 12583754-04b2-f5fd-543e-bd4142527c27 [Candidate] entering Candidate state
2014/04/29 16:03:28 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:28 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:28 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:28 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:28 [INFO] raft: Node at 12583754-04b2-f5fd-543e-bd4142527c27 [Leader] entering Leader state
==================
WARNING: DATA RACE
Read by goroutine 62:
  github.com/benbjohnson/raft.(*cluster).EnsureSame()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:200 +0x193
  github.com/benbjohnson/raft.TestRaft_ApplyConcurrent()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:597 +0x369
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 64:
  github.com/benbjohnson/raft.(*MockFSM).Apply()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:30 +0x175
  github.com/benbjohnson/raft.(*Raft).runFSM()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:465 +0xe73
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.runFSM)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 62 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 64 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0xdf9
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_ApplyConcurrent()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:561 +0xc3
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
==================
WARNING: DATA RACE
Read by goroutine 62:
  github.com/benbjohnson/raft.(*cluster).EnsureSame()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:200 +0x1a1
  github.com/benbjohnson/raft.TestRaft_ApplyConcurrent()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:597 +0x369
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 66:
  github.com/benbjohnson/raft.(*MockFSM).Apply()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:30 +0x175
  github.com/benbjohnson/raft.(*Raft).runFSM()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:465 +0xe73
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.runFSM)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 62 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 66 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0xdf9
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_ApplyConcurrent()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:561 +0xc3
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
==================
WARNING: DATA RACE
Read by goroutine 62:
  github.com/benbjohnson/raft.(*cluster).EnsureSame()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:210 +0x3a7
  github.com/benbjohnson/raft.TestRaft_ApplyConcurrent()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:597 +0x369
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 64:
  github.com/benbjohnson/raft.(*MockFSM).Apply()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:30 +0x139
  github.com/benbjohnson/raft.(*Raft).runFSM()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:465 +0xe73
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.runFSM)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 62 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 64 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0xdf9
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_ApplyConcurrent()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:561 +0xc3
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
==================
WARNING: DATA RACE
Read by goroutine 62:
  github.com/benbjohnson/raft.(*cluster).EnsureSame()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:210 +0x3e6
  github.com/benbjohnson/raft.TestRaft_ApplyConcurrent()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:597 +0x369
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 66:
  github.com/benbjohnson/raft.(*MockFSM).Apply()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:30 +0x139
  github.com/benbjohnson/raft.(*Raft).runFSM()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:465 +0xe73
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.runFSM)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 62 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 66 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0xdf9
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_ApplyConcurrent()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:561 +0xc3
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
--- PASS: TestRaft_ApplyConcurrent (0.18 seconds)
=== RUN TestRaft_ApplyConcurrent_Timeout
2014/04/29 16:03:28 [WARN] Fully Connecting
2014/04/29 16:03:28 [INFO] raft: Node at e4c9d7ce-8616-948d-6f05-2d9cd2fdce84 [Follower] entering Follower state
2014/04/29 16:03:28 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:28 [INFO] raft: Node at e4c9d7ce-8616-948d-6f05-2d9cd2fdce84 [Candidate] entering Candidate state
2014/04/29 16:03:28 [DEBUG] raft: Votes needed: 1
2014/04/29 16:03:28 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:28 [INFO] raft: Election won. Tally: 1
2014/04/29 16:03:28 [INFO] raft: Node at e4c9d7ce-8616-948d-6f05-2d9cd2fdce84 [Leader] entering Leader state
==================
WARNING: DATA RACE
Write by goroutine 14:
  github.com/benbjohnson/raft.func·018()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:617 +0x20b

Previous write by goroutine 69:
  github.com/benbjohnson/raft.func·018()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:617 +0x20b

Goroutine 14 (running) created at:
  github.com/benbjohnson/raft.TestRaft_ApplyConcurrent_Timeout()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:619 +0x1a2
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Goroutine 69 (finished) created at:
  github.com/benbjohnson/raft.TestRaft_ApplyConcurrent_Timeout()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:619 +0x1a2
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
--- PASS: TestRaft_ApplyConcurrent_Timeout (0.22 seconds)
=== RUN TestRaft_JoinNode
2014/04/29 16:03:28 [WARN] Fully Connecting
2014/04/29 16:03:28 [INFO] raft: Node at 9598edb6-2caf-d436-233a-c8a69e29e2e8 [Follower] entering Follower state
2014/04/29 16:03:28 [INFO] raft: Node at 6cf9867a-44d2-b7ae-850a-490b7bee94b4 [Follower] entering Follower state
2014/04/29 16:03:28 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:28 [INFO] raft: Node at 9598edb6-2caf-d436-233a-c8a69e29e2e8 [Candidate] entering Candidate state
2014/04/29 16:03:28 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:28 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:28 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:28 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:28 [INFO] raft: Node at 9598edb6-2caf-d436-233a-c8a69e29e2e8 [Leader] entering Leader state
2014/04/29 16:03:28 [INFO] Applied log
2014/04/29 16:03:28 [WARN] Fully Connecting
2014/04/29 16:03:28 [WARN] Fully Connecting
2014/04/29 16:03:28 [INFO] raft: Node at 2fe60d00-a6e8-0987-ad2e-964168c0eea3 [Follower] entering Follower state
2014/04/29 16:03:28 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:28 [INFO] raft: Node at 2fe60d00-a6e8-0987-ad2e-964168c0eea3 [Candidate] entering Candidate state
2014/04/29 16:03:28 [DEBUG] raft: Votes needed: 1
2014/04/29 16:03:28 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:28 [INFO] raft: Election won. Tally: 1
2014/04/29 16:03:28 [INFO] raft: Node at 2fe60d00-a6e8-0987-ad2e-964168c0eea3 [Leader] entering Leader state
2014/04/29 16:03:28 [DEBUG] raft: Node 9598edb6-2caf-d436-233a-c8a69e29e2e8 updated peer set (add): [2fe60d00-a6e8-0987-ad2e-964168c0eea3 9598edb6-2caf-d436-233a-c8a69e29e2e8 6cf9867a-44d2-b7ae-850a-490b7bee94b4]
2014/04/29 16:03:28 [INFO] raft: Added peer 2fe60d00-a6e8-0987-ad2e-964168c0eea3, starting replication
2014/04/29 16:03:28 [WARN] raft: Failed to get previous log: 3 log not found (last: 1)
2014/04/29 16:03:28 [INFO] raft: Node at 2fe60d00-a6e8-0987-ad2e-964168c0eea3 [Follower] entering Follower state
2014/04/29 16:03:28 [WARN] raft: AppendEntries to 2fe60d00-a6e8-0987-ad2e-964168c0eea3 rejected, sending older logs (next: 2)
2014/04/29 16:03:28 [DEBUG] raft: Node 9598edb6-2caf-d436-233a-c8a69e29e2e8 updated peer set (add): [2fe60d00-a6e8-0987-ad2e-964168c0eea3 9598edb6-2caf-d436-233a-c8a69e29e2e8 6cf9867a-44d2-b7ae-850a-490b7bee94b4]
2014/04/29 16:03:28 [DEBUG] raft: Node 6cf9867a-44d2-b7ae-850a-490b7bee94b4 updated peer set (add): [2fe60d00-a6e8-0987-ad2e-964168c0eea3 9598edb6-2caf-d436-233a-c8a69e29e2e8 6cf9867a-44d2-b7ae-850a-490b7bee94b4]
==================
WARNING: DATA RACE
Read by goroutine 135:
  github.com/benbjohnson/raft.(*cluster).EnsureSame()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:200 +0x1a1
  github.com/benbjohnson/raft.TestRaft_JoinNode()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:682 +0x982
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 140:
  github.com/benbjohnson/raft.(*MockFSM).Apply()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:30 +0x175
  github.com/benbjohnson/raft.(*Raft).runFSM()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:465 +0xe73
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.runFSM)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 135 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 140 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0xdf9
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_JoinNode()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:633 +0x57
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
==================
WARNING: DATA RACE
Read by goroutine 135:
  github.com/benbjohnson/raft.(*cluster).EnsureSame()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:210 +0x3e6
  github.com/benbjohnson/raft.TestRaft_JoinNode()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:682 +0x982
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 140:
  github.com/benbjohnson/raft.(*MockFSM).Apply()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:30 +0x139
  github.com/benbjohnson/raft.(*Raft).runFSM()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:465 +0xe73
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.runFSM)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 135 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 140 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0xdf9
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_JoinNode()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:633 +0x57
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
2014/04/29 16:03:28 [DEBUG] raft: Node 2fe60d00-a6e8-0987-ad2e-964168c0eea3 updated peer set (add): [2fe60d00-a6e8-0987-ad2e-964168c0eea3 9598edb6-2caf-d436-233a-c8a69e29e2e8 6cf9867a-44d2-b7ae-850a-490b7bee94b4]
==================
WARNING: DATA RACE
Read by goroutine 135:
  github.com/benbjohnson/raft.raftToPeerSet()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:229 +0xd2
  github.com/benbjohnson/raft.(*cluster).EnsureSamePeers()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:245 +0x1b0
  github.com/benbjohnson/raft.TestRaft_JoinNode()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:685 +0x99d
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 139:
  github.com/benbjohnson/raft.(*Raft).processLog()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:952 +0x6a2
  github.com/benbjohnson/raft.(*Raft).processLogs()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:918 +0x428
  github.com/benbjohnson/raft.(*Raft).appendEntries()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:1119 +0xd95
  github.com/benbjohnson/raft.(*Raft).processRPC()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:1023 +0x1b3
  github.com/benbjohnson/raft.(*Raft).runFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:516 +0x1f8
  github.com/benbjohnson/raft.(*Raft).run()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:499 +0xd2
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.run)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 135 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 139 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0xd8b
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_JoinNode()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:633 +0x57
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
==================
WARNING: DATA RACE
Read by goroutine 135:
  github.com/benbjohnson/raft.raftToPeerSet()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:229 +0x11a
  github.com/benbjohnson/raft.(*cluster).EnsureSamePeers()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:245 +0x1b0
  github.com/benbjohnson/raft.TestRaft_JoinNode()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:685 +0x99d
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 139:
  github.com/benbjohnson/raft.ExcludePeer()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/util.go:85 +0x27b
  github.com/benbjohnson/raft.(*Raft).processLog()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:952 +0x70a
  github.com/benbjohnson/raft.(*Raft).processLogs()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:918 +0x428
  github.com/benbjohnson/raft.(*Raft).appendEntries()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:1119 +0xd95
  github.com/benbjohnson/raft.(*Raft).processRPC()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:1023 +0x1b3
  github.com/benbjohnson/raft.(*Raft).runFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:516 +0x1f8
  github.com/benbjohnson/raft.(*Raft).run()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:499 +0xd2
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.run)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 135 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 139 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0xd8b
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_JoinNode()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:633 +0x57
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
==================
WARNING: DATA RACE
Read by goroutine 135:
  github.com/benbjohnson/raft.(*InmemAddr).String()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/inmem_transport.go:27 +0x35
  github.com/benbjohnson/raft.raftToPeerSet()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:230 +0x138
  github.com/benbjohnson/raft.(*cluster).EnsureSamePeers()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:245 +0x1b0
  github.com/benbjohnson/raft.TestRaft_JoinNode()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:685 +0x99d
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 139:
  github.com/benbjohnson/raft.(*InmemTransport).DecodePeer()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/inmem_transport.go:134 +0x47
  github.com/benbjohnson/raft.decodePeers()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/util.go:139 +0x272
  github.com/benbjohnson/raft.(*Raft).processLog()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:948 +0x4d4
  github.com/benbjohnson/raft.(*Raft).processLogs()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:918 +0x428
  github.com/benbjohnson/raft.(*Raft).appendEntries()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:1119 +0xd95
  github.com/benbjohnson/raft.(*Raft).processRPC()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:1023 +0x1b3
  github.com/benbjohnson/raft.(*Raft).runFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:516 +0x1f8
  github.com/benbjohnson/raft.(*Raft).run()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:499 +0xd2
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.run)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 135 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 139 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0xd8b
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_JoinNode()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:633 +0x57
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
==================
WARNING: DATA RACE
Read by goroutine 135:
  github.com/benbjohnson/raft.(*InmemAddr).String()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/inmem_transport.go:27 +0x35
  github.com/benbjohnson/raft.(*cluster).EnsureLeader()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:174 +0x2eb
  github.com/benbjohnson/raft.TestRaft_JoinNode()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:689 +0x9ff
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 139:
  github.com/benbjohnson/raft.(*InmemTransport).DecodePeer()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/inmem_transport.go:134 +0x47
  github.com/benbjohnson/raft.(*Raft).appendEntries()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:1062 +0x3ba
  github.com/benbjohnson/raft.(*Raft).processRPC()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:1023 +0x1b3
  github.com/benbjohnson/raft.(*Raft).runFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:516 +0x1f8
  github.com/benbjohnson/raft.(*Raft).run()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:499 +0xd2
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.run)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 135 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 139 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0xd8b
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_JoinNode()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:633 +0x57
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
--- PASS: TestRaft_JoinNode (0.08 seconds)
=== RUN TestRaft_RemoveFollower
2014/04/29 16:03:28 [WARN] Fully Connecting
2014/04/29 16:03:28 [INFO] raft: Node at 5cb71f1b-06b9-012e-7f04-72bf7c2e562e [Follower] entering Follower state
2014/04/29 16:03:28 [INFO] raft: Node at 21c7b566-cea7-fc67-c605-0ece48684d7b [Follower] entering Follower state
2014/04/29 16:03:28 [INFO] raft: Node at 961de145-a6d8-84ef-b209-139c5e6d4773 [Follower] entering Follower state
2014/04/29 16:03:28 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:28 [INFO] raft: Node at 961de145-a6d8-84ef-b209-139c5e6d4773 [Candidate] entering Candidate state
2014/04/29 16:03:28 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:28 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:28 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:28 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:28 [INFO] raft: Node at 961de145-a6d8-84ef-b209-139c5e6d4773 [Leader] entering Leader state
2014/04/29 16:03:28 [DEBUG] raft: Node 961de145-a6d8-84ef-b209-139c5e6d4773 updated peer set (remove): [961de145-a6d8-84ef-b209-139c5e6d4773 21c7b566-cea7-fc67-c605-0ece48684d7b]
2014/04/29 16:03:28 [INFO] raft: Removed peer 5cb71f1b-06b9-012e-7f04-72bf7c2e562e, stopping replication
2014/04/29 16:03:28 [DEBUG] raft: Node 961de145-a6d8-84ef-b209-139c5e6d4773 updated peer set (remove): [961de145-a6d8-84ef-b209-139c5e6d4773 21c7b566-cea7-fc67-c605-0ece48684d7b]
2014/04/29 16:03:28 [DEBUG] raft: Node 21c7b566-cea7-fc67-c605-0ece48684d7b updated peer set (remove): [961de145-a6d8-84ef-b209-139c5e6d4773 21c7b566-cea7-fc67-c605-0ece48684d7b]
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at 5cb71f1b-06b9-012e-7f04-72bf7c2e562e [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [INFO] raft: Node at 961de145-a6d8-84ef-b209-139c5e6d4773 [Follower] entering Follower state
2014/04/29 16:03:29 [WARN] raft: Remote peer 21c7b566-cea7-fc67-c605-0ece48684d7b does not have local node 5cb71f1b-06b9-012e-7f04-72bf7c2e562e as a peer
2014/04/29 16:03:29 [WARN] raft: Remote peer 961de145-a6d8-84ef-b209-139c5e6d4773 does not have local node 5cb71f1b-06b9-012e-7f04-72bf7c2e562e as a peer
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Node at 5cb71f1b-06b9-012e-7f04-72bf7c2e562e [Leader] entering Leader state
2014/04/29 16:03:29 [DEBUG] raft: Node 5cb71f1b-06b9-012e-7f04-72bf7c2e562e updated peer set (remove): [961de145-a6d8-84ef-b209-139c5e6d4773 21c7b566-cea7-fc67-c605-0ece48684d7b]
2014/04/29 16:03:29 [INFO] raft: Removed peer 21c7b566-cea7-fc67-c605-0ece48684d7b, stopping replication
2014/04/29 16:03:29 [INFO] raft: Removed peer 961de145-a6d8-84ef-b209-139c5e6d4773, stopping replication
2014/04/29 16:03:29 [INFO] raft: Removed ourself, shutting down
==================
WARNING: DATA RACE
Read by goroutine 150:
  github.com/benbjohnson/raft.TestRaft_RemoveFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:725 +0x513
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 154:
  github.com/benbjohnson/raft.(*Raft).processLog()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:975 +0x1540
  github.com/benbjohnson/raft.(*Raft).processLogs()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:918 +0x428
  github.com/benbjohnson/raft.(*Raft).appendEntries()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:1119 +0xd95
  github.com/benbjohnson/raft.(*Raft).processRPC()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:1023 +0x1b3
  github.com/benbjohnson/raft.(*Raft).runFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:516 +0x1f8
  github.com/benbjohnson/raft.(*Raft).run()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:499 +0xd2
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.run)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 150 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 154 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0xd8b
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_RemoveFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:694 +0x54
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
--- PASS: TestRaft_RemoveFollower (0.05 seconds)
=== RUN TestRaft_RemoveLeader
2014/04/29 16:03:29 [WARN] Fully Connecting
2014/04/29 16:03:29 [INFO] raft: Node at 16e96470-8b63-7e31-2ddf-904af2af5031 [Follower] entering Follower state
2014/04/29 16:03:29 [INFO] raft: Node at d8c045bf-2fe8-fe0a-735a-12310d94d92e [Follower] entering Follower state
2014/04/29 16:03:29 [INFO] raft: Node at 1f806495-6977-4715-52d3-870b68855694 [Follower] entering Follower state
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at 16e96470-8b63-7e31-2ddf-904af2af5031 [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Node at 16e96470-8b63-7e31-2ddf-904af2af5031 [Leader] entering Leader state
2014/04/29 16:03:29 [DEBUG] raft: Node 16e96470-8b63-7e31-2ddf-904af2af5031 updated peer set (remove): [d8c045bf-2fe8-fe0a-735a-12310d94d92e 1f806495-6977-4715-52d3-870b68855694]
2014/04/29 16:03:29 [INFO] raft: Removed peer d8c045bf-2fe8-fe0a-735a-12310d94d92e, stopping replication
2014/04/29 16:03:29 [INFO] raft: Removed peer 1f806495-6977-4715-52d3-870b68855694, stopping replication
2014/04/29 16:03:29 [DEBUG] raft: Node 16e96470-8b63-7e31-2ddf-904af2af5031 updated peer set (remove): [d8c045bf-2fe8-fe0a-735a-12310d94d92e 1f806495-6977-4715-52d3-870b68855694]
2014/04/29 16:03:29 [INFO] raft: Removed ourself, shutting down
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at d8c045bf-2fe8-fe0a-735a-12310d94d92e [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Node at d8c045bf-2fe8-fe0a-735a-12310d94d92e [Leader] entering Leader state
2014/04/29 16:03:29 [DEBUG] raft: Node d8c045bf-2fe8-fe0a-735a-12310d94d92e updated peer set (remove): [d8c045bf-2fe8-fe0a-735a-12310d94d92e 1f806495-6977-4715-52d3-870b68855694]
2014/04/29 16:03:29 [INFO] raft: Removed peer 16e96470-8b63-7e31-2ddf-904af2af5031, stopping replication
2014/04/29 16:03:29 [DEBUG] raft: Node 1f806495-6977-4715-52d3-870b68855694 updated peer set (remove): [d8c045bf-2fe8-fe0a-735a-12310d94d92e 1f806495-6977-4715-52d3-870b68855694]
==================
WARNING: DATA RACE
Read by goroutine 172:
  github.com/benbjohnson/raft.TestRaft_RemoveLeader()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:762 +0x334
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 176:
  github.com/benbjohnson/raft.(*Raft).processLog()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:975 +0x1540
  github.com/benbjohnson/raft.(*Raft).processLogs()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:918 +0x428
  github.com/benbjohnson/raft.(*Raft).leaderLoop()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:707 +0x344
  github.com/benbjohnson/raft.(*Raft).runLeader()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:669 +0x517
  github.com/benbjohnson/raft.(*Raft).run()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:503 +0x106
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.run)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 172 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 176 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0xd8b
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_RemoveLeader()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:732 +0x54
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
2014/04/29 16:03:29 [ERR] raft: Failed to make RequestVote RPC to 16e96470-8b63-7e31-2ddf-904af2af5031: command timed out
2014/04/29 16:03:29 [ERR] raft: Failed to AppendEntries to 16e96470-8b63-7e31-2ddf-904af2af5031: command timed out
2014/04/29 16:03:29 [ERR] raft: Failed to heartbeat to 16e96470-8b63-7e31-2ddf-904af2af5031: command timed out
2014/04/29 16:03:29 [ERR] raft: Failed to AppendEntries to 16e96470-8b63-7e31-2ddf-904af2af5031: command timed out
2014/04/29 16:03:29 [ERR] raft: Failed to heartbeat to 16e96470-8b63-7e31-2ddf-904af2af5031: command timed out
--- PASS: TestRaft_RemoveLeader (0.16 seconds)
=== RUN TestRaft_RemoveLeader_SplitCluster
2014/04/29 16:03:29 [WARN] Fully Connecting
2014/04/29 16:03:29 [INFO] raft: Node at 384f4d97-73d8-6e64-dd8e-2745333698bb [Follower] entering Follower state
2014/04/29 16:03:29 [INFO] raft: Node at bdfc583f-cf9b-6066-a311-e1f80c609c64 [Follower] entering Follower state
2014/04/29 16:03:29 [INFO] raft: Node at aff757ec-6eab-315e-3df7-35656fc2fd81 [Follower] entering Follower state
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at bdfc583f-cf9b-6066-a311-e1f80c609c64 [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Node at bdfc583f-cf9b-6066-a311-e1f80c609c64 [Leader] entering Leader state
2014/04/29 16:03:29 [DEBUG] raft: Node bdfc583f-cf9b-6066-a311-e1f80c609c64 updated peer set (remove): [384f4d97-73d8-6e64-dd8e-2745333698bb aff757ec-6eab-315e-3df7-35656fc2fd81]
2014/04/29 16:03:29 [INFO] raft: Removed peer 384f4d97-73d8-6e64-dd8e-2745333698bb, stopping replication
2014/04/29 16:03:29 [INFO] raft: Removed peer aff757ec-6eab-315e-3df7-35656fc2fd81, stopping replication
2014/04/29 16:03:29 [DEBUG] raft: Node bdfc583f-cf9b-6066-a311-e1f80c609c64 updated peer set (remove): [384f4d97-73d8-6e64-dd8e-2745333698bb aff757ec-6eab-315e-3df7-35656fc2fd81]
2014/04/29 16:03:29 [INFO] raft: Removed ourself, transitioning to follower
2014/04/29 16:03:29 [INFO] raft: Node at bdfc583f-cf9b-6066-a311-e1f80c609c64 [Follower] entering Follower state
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at bdfc583f-cf9b-6066-a311-e1f80c609c64 [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 1
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [INFO] raft: Election won. Tally: 1
2014/04/29 16:03:29 [INFO] raft: Node at bdfc583f-cf9b-6066-a311-e1f80c609c64 [Leader] entering Leader state
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at aff757ec-6eab-315e-3df7-35656fc2fd81 [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [INFO] raft: Duplicate RequestVote for same term: 2
2014/04/29 16:03:29 [WARN] raft: Remote peer bdfc583f-cf9b-6066-a311-e1f80c609c64 does not have local node aff757ec-6eab-315e-3df7-35656fc2fd81 as a peer
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Node at aff757ec-6eab-315e-3df7-35656fc2fd81 [Leader] entering Leader state
2014/04/29 16:03:29 [WARN] raft: Clearing log suffix from 3 to 3
2014/04/29 16:03:29 [INFO] raft: Node at bdfc583f-cf9b-6066-a311-e1f80c609c64 [Follower] entering Follower state
2014/04/29 16:03:29 [DEBUG] raft: Node aff757ec-6eab-315e-3df7-35656fc2fd81 updated peer set (remove): [384f4d97-73d8-6e64-dd8e-2745333698bb aff757ec-6eab-315e-3df7-35656fc2fd81]
2014/04/29 16:03:29 [INFO] raft: Removed peer bdfc583f-cf9b-6066-a311-e1f80c609c64, stopping replication
2014/04/29 16:03:29 [DEBUG] raft: Node 384f4d97-73d8-6e64-dd8e-2745333698bb updated peer set (remove): [384f4d97-73d8-6e64-dd8e-2745333698bb aff757ec-6eab-315e-3df7-35656fc2fd81]
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at bdfc583f-cf9b-6066-a311-e1f80c609c64 [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 1
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [INFO] raft: Election won. Tally: 1
2014/04/29 16:03:29 [INFO] raft: Node at bdfc583f-cf9b-6066-a311-e1f80c609c64 [Leader] entering Leader state
--- PASS: TestRaft_RemoveLeader_SplitCluster (0.06 seconds)
=== RUN TestRaft_AddKnownPeer
2014/04/29 16:03:29 [WARN] Fully Connecting
2014/04/29 16:03:29 [INFO] raft: Node at b1f0b4cb-c619-7024-ea79-470af08c0d41 [Follower] entering Follower state
2014/04/29 16:03:29 [INFO] raft: Node at 47b18433-41ed-b65a-eeb8-51cd7e49786c [Follower] entering Follower state
2014/04/29 16:03:29 [INFO] raft: Node at b412e497-d11f-48d7-5ce4-50f602a49c8c [Follower] entering Follower state
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at b412e497-d11f-48d7-5ce4-50f602a49c8c [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Node at b412e497-d11f-48d7-5ce4-50f602a49c8c [Leader] entering Leader state
--- PASS: TestRaft_AddKnownPeer (0.02 seconds)
=== RUN TestRaft_RemoveUnknownPeer
2014/04/29 16:03:29 [WARN] Fully Connecting
2014/04/29 16:03:29 [INFO] raft: Node at 42ffa9fd-bb8a-4c84-1b4f-50f44c2d7da6 [Follower] entering Follower state
2014/04/29 16:03:29 [INFO] raft: Node at 1dba7870-1e87-eead-4f4e-c262fdb2177d [Follower] entering Follower state
2014/04/29 16:03:29 [INFO] raft: Node at 8065ce86-a3d4-6d67-6b36-d102436f41b3 [Follower] entering Follower state
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at 1dba7870-1e87-eead-4f4e-c262fdb2177d [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Node at 1dba7870-1e87-eead-4f4e-c262fdb2177d [Leader] entering Leader state
--- PASS: TestRaft_RemoveUnknownPeer (0.02 seconds)
=== RUN TestRaft_SnapshotRestore
2014/04/29 16:03:29 [WARN] Fully Connecting
2014/04/29 16:03:29 [INFO] raft: Node at 357e5000-6129-7bd4-0134-beecc59664e4 [Follower] entering Follower state
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at 357e5000-6129-7bd4-0134-beecc59664e4 [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 1
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [INFO] raft: Election won. Tally: 1
2014/04/29 16:03:29 [INFO] raft: Node at 357e5000-6129-7bd4-0134-beecc59664e4 [Leader] entering Leader state
2014/04/29 16:03:29 [INFO] raft: Starting snapshot up to 101
2014/04/29 16:03:29 [INFO] snapshot: Creating new snapshot at /var/folders/p3/nqt2bgp17fn7_t9hr6tr_1m00000gn/T/raft767593768/snapshots/1-101-2014-04-29T16:03:29.30552547-06:00.tmp
2014/04/29 16:03:29 [INFO] raft: Compacting logs from 1 to 91
2014/04/29 16:03:29 [INFO] raft: Snapshot to 101 complete
2014/04/29 16:03:29 [INFO] raft: Restored from snapshot 1-101-2014-04-29T16:03:29.30552547-06:00
--- PASS: TestRaft_SnapshotRestore (0.04 seconds)
=== RUN TestRaft_AutoSnapshot
2014/04/29 16:03:29 [WARN] Fully Connecting
2014/04/29 16:03:29 [INFO] raft: Node at 20886764-90aa-89fa-d687-5da38fe9c5c7 [Follower] entering Follower state
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at 20886764-90aa-89fa-d687-5da38fe9c5c7 [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 1
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [INFO] raft: Election won. Tally: 1
2014/04/29 16:03:29 [INFO] raft: Node at 20886764-90aa-89fa-d687-5da38fe9c5c7 [Leader] entering Leader state
==================
WARNING: DATA RACE
Read by goroutine 74:
  github.com/benbjohnson/raft.(*InmemStore).LastIndex()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/inmem_store.go:35 +0x3a
  github.com/benbjohnson/raft.(*Raft).shouldSnapshot()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:1425 +0x8b
  github.com/benbjohnson/raft.(*Raft).runSnapshots()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:1395 +0xd1
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.runSnapshots)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:204 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Previous write by goroutine 70:
  github.com/benbjohnson/raft.(*InmemStore).StoreLogs()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/inmem_store.go:62 +0x1c3
  github.com/benbjohnson/raft.(*InmemStore).StoreLog()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/inmem_store.go:50 +0xa4
  github.com/benbjohnson/raft.(*Raft).dispatchLog()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:870 +0x2e0
  github.com/benbjohnson/raft.(*Raft).runLeader()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:666 +0x506
  github.com/benbjohnson/raft.(*Raft).run()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:503 +0x106
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.run)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 74 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:204 +0xe63
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_AutoSnapshot()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:907 +0xaf
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Goroutine 70 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0xd8b
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_AutoSnapshot()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:907 +0xaf
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
2014/04/29 16:03:29 [INFO] raft: Starting snapshot up to 57
2014/04/29 16:03:29 [INFO] snapshot: Creating new snapshot at /var/folders/p3/nqt2bgp17fn7_t9hr6tr_1m00000gn/T/raft054581402/snapshots/1-57-2014-04-29T16:03:29.338303383-06:00.tmp
2014/04/29 16:03:29 [INFO] raft: Compacting logs from 1 to 49
2014/04/29 16:03:29 [INFO] raft: Snapshot to 57 complete
--- PASS: TestRaft_AutoSnapshot (0.08 seconds)
=== RUN TestRaft_SendSnapshotFollower
2014/04/29 16:03:29 [WARN] Fully Connecting
2014/04/29 16:03:29 [WARN] Disconnecting 05bd295a-6927-0ecf-7f71-49625d650f50
2014/04/29 16:03:29 [INFO] raft: Node at 05bd295a-6927-0ecf-7f71-49625d650f50 [Follower] entering Follower state
2014/04/29 16:03:29 [INFO] raft: Node at ebc6f769-107a-d29a-9996-500b09acf8e8 [Follower] entering Follower state
2014/04/29 16:03:29 [INFO] raft: Node at 97bc070c-8088-d7f7-1364-d99a75afab6e [Follower] entering Follower state
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at ebc6f769-107a-d29a-9996-500b09acf8e8 [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [ERR] raft: Failed to make RequestVote RPC to 05bd295a-6927-0ecf-7f71-49625d650f50: Failed to connect to peer: 05bd295a-6927-0ecf-7f71-49625d650f50
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Node at ebc6f769-107a-d29a-9996-500b09acf8e8 [Leader] entering Leader state
2014/04/29 16:03:29 [ERR] raft: Failed to AppendEntries to 05bd295a-6927-0ecf-7f71-49625d650f50: Failed to connect to peer: 05bd295a-6927-0ecf-7f71-49625d650f50
2014/04/29 16:03:29 [ERR] raft: Failed to heartbeat to 05bd295a-6927-0ecf-7f71-49625d650f50: Failed to connect to peer: 05bd295a-6927-0ecf-7f71-49625d650f50
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at 05bd295a-6927-0ecf-7f71-49625d650f50 [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [ERR] raft: Failed to make RequestVote RPC to ebc6f769-107a-d29a-9996-500b09acf8e8: Failed to connect to peer: ebc6f769-107a-d29a-9996-500b09acf8e8
2014/04/29 16:03:29 [ERR] raft: Failed to make RequestVote RPC to 97bc070c-8088-d7f7-1364-d99a75afab6e: Failed to connect to peer: 97bc070c-8088-d7f7-1364-d99a75afab6e
2014/04/29 16:03:29 [ERR] raft: Failed to AppendEntries to 05bd295a-6927-0ecf-7f71-49625d650f50: Failed to connect to peer: 05bd295a-6927-0ecf-7f71-49625d650f50
2014/04/29 16:03:29 [INFO] Finished apply without behind follower
2014/04/29 16:03:29 [INFO] raft: Starting snapshot up to 0
2014/04/29 16:03:29 [INFO] snapshot: Creating new snapshot at /var/folders/p3/nqt2bgp17fn7_t9hr6tr_1m00000gn/T/raft177891548/snapshots/0-0-2014-04-29T16:03:29.425225495-06:00.tmp
2014/04/29 16:03:29 [INFO] raft: Compacting logs from 0 to 0
2014/04/29 16:03:29 [INFO] raft: Snapshot to 0 complete
2014/04/29 16:03:29 [INFO] raft: Starting snapshot up to 101
2014/04/29 16:03:29 [INFO] snapshot: Creating new snapshot at /var/folders/p3/nqt2bgp17fn7_t9hr6tr_1m00000gn/T/raft731920494/snapshots/1-101-2014-04-29T16:03:29.427763228-06:00.tmp
2014/04/29 16:03:29 [INFO] raft: Compacting logs from 1 to 91
2014/04/29 16:03:29 [INFO] raft: Snapshot to 101 complete
2014/04/29 16:03:29 [INFO] raft: Starting snapshot up to 98
2014/04/29 16:03:29 [INFO] snapshot: Creating new snapshot at /var/folders/p3/nqt2bgp17fn7_t9hr6tr_1m00000gn/T/raft905684432/snapshots/1-98-2014-04-29T16:03:29.42973687-06:00.tmp
2014/04/29 16:03:29 [INFO] raft: Compacting logs from 1 to 91
2014/04/29 16:03:29 [INFO] raft: Snapshot to 98 complete
2014/04/29 16:03:29 [WARN] Fully Connecting
2014/04/29 16:03:29 [INFO] raft: Node at 05bd295a-6927-0ecf-7f71-49625d650f50 [Follower] entering Follower state
2014/04/29 16:03:29 [INFO] snapshot: Creating new snapshot at /var/folders/p3/nqt2bgp17fn7_t9hr6tr_1m00000gn/T/raft177891548/snapshots/1-101-2014-04-29T16:03:29.435607883-06:00.tmp
2014/04/29 16:03:29 [INFO] raft: Copied 693 bytes to local snapshot
==================
WARNING: DATA RACE
Write by goroutine 89:
  github.com/benbjohnson/raft.(*MockFSM).Restore()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:43 +0x163
  github.com/benbjohnson/raft.(*Raft).runFSM()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:427 +0x405
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.runFSM)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Previous read by goroutine 73:
  github.com/benbjohnson/raft.(*cluster).EnsureSame()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:200 +0x193
  github.com/benbjohnson/raft.TestRaft_SendSnapshotFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:969 +0x5a9
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Goroutine 89 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0xdf9
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_SendSnapshotFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:935 +0x79
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Goroutine 73 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc
==================
2014/04/29 16:03:29 [INFO] raft: Compacting logs from 1 to 101
2014/04/29 16:03:29 [INFO] raft: Installed remote snapshot
==================
WARNING: DATA RACE
Write by goroutine 101:
  github.com/benbjohnson/raft.(*Raft).heartbeat()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/replication.go:311 +0x55b
  github.com/benbjohnson/raft.func·006()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/replication.go:56 +0x91
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Previous write by goroutine 99:
  github.com/benbjohnson/raft.(*Raft).sendLatestSnapshot()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/replication.go:260 +0xd7a
  github.com/benbjohnson/raft.(*Raft).replicateTo()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/replication.go:191 +0x5a3
  github.com/benbjohnson/raft.(*Raft).replicate()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/replication.go:68 +0x328
  github.com/benbjohnson/raft.func·003()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:687 +0x6d
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 101 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.(*Raft).replicate()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/replication.go:56 +0x188
  github.com/benbjohnson/raft.func·003()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:687 +0x6d
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 99 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.(*Raft).startReplication()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:687 +0x430
  github.com/benbjohnson/raft.(*Raft).runLeader()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:661 +0x479
  github.com/benbjohnson/raft.(*Raft).run()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:503 +0x106
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.run)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b
==================
2014/04/29 16:03:29 [WARN] raft: Skipping application of old log: 101
==================
WARNING: DATA RACE
Read by goroutine 73:
  github.com/benbjohnson/raft.(*cluster).EnsureSame()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:210 +0x3a7
  github.com/benbjohnson/raft.TestRaft_SendSnapshotFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:969 +0x5a9
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 89:
  reflect.Value.SetBytes()
      /usr/local/go/src/pkg/reflect/value.go:1409 +0xcd
  github.com/ugorji/go/codec.(*decFnInfo).kSlice()
      /Users/benbjohnson/go/src/github.com/ugorji/go/codec/decode.go:423 +0xb97
  github.com/ugorji/go/codec.(*Decoder).decodeValue()
      /Users/benbjohnson/go/src/github.com/ugorji/go/codec/decode.go:825 +0x905
  github.com/ugorji/go/codec.(*decFnInfo).kSlice()
      /Users/benbjohnson/go/src/github.com/ugorji/go/codec/decode.go:472 +0xa34
  github.com/ugorji/go/codec.(*Decoder).decodeValue()
      /Users/benbjohnson/go/src/github.com/ugorji/go/codec/decode.go:825 +0x905
  github.com/ugorji/go/codec.(*Decoder).decode()
      /Users/benbjohnson/go/src/github.com/ugorji/go/codec/decode.go:693 +0x3ac
  github.com/ugorji/go/codec.(*Decoder).Decode()
      /Users/benbjohnson/go/src/github.com/ugorji/go/codec/decode.go:624 +0x76
  github.com/benbjohnson/raft.(*MockFSM).Restore()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:44 +0x199
  github.com/benbjohnson/raft.(*Raft).runFSM()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:427 +0x405
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.runFSM)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 73 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 89 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0xdf9
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_SendSnapshotFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:935 +0x79
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
==================
WARNING: DATA RACE
Read by goroutine 73:
  github.com/benbjohnson/raft.(*cluster).EnsureSame()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:210 +0x3e6
  github.com/benbjohnson/raft.TestRaft_SendSnapshotFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:969 +0x5a9
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 95:
  github.com/benbjohnson/raft.(*MockFSM).Apply()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:30 +0x139
  github.com/benbjohnson/raft.(*Raft).runFSM()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:465 +0xe73
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.runFSM)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 73 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 95 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0xdf9
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_SendSnapshotFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:935 +0x79
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
2014/04/29 16:03:29 [ERR] raft: Failed to heartbeat to 97bc070c-8088-d7f7-1364-d99a75afab6e: command timed out
2014/04/29 16:03:29 [ERR] raft: Failed to AppendEntries to 97bc070c-8088-d7f7-1364-d99a75afab6e: command timed out
--- PASS: TestRaft_SendSnapshotFollower (0.12 seconds)
=== RUN TestRaft_ReJoinFollower
2014/04/29 16:03:29 [WARN] Fully Connecting
2014/04/29 16:03:29 [INFO] raft: Node at ef69a1c0-f12e-6bed-8dc5-f1528674d220 [Follower] entering Follower state
2014/04/29 16:03:29 [INFO] raft: Node at b7abd8b7-b511-6b5f-6f37-289b72cceb0c [Follower] entering Follower state
2014/04/29 16:03:29 [INFO] raft: Node at 30bd734d-5c54-1ec3-0263-d9fc40ef25d4 [Follower] entering Follower state
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at ef69a1c0-f12e-6bed-8dc5-f1528674d220 [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Node at ef69a1c0-f12e-6bed-8dc5-f1528674d220 [Leader] entering Leader state
2014/04/29 16:03:29 [DEBUG] raft: Node ef69a1c0-f12e-6bed-8dc5-f1528674d220 updated peer set (remove): [ef69a1c0-f12e-6bed-8dc5-f1528674d220 30bd734d-5c54-1ec3-0263-d9fc40ef25d4]
2014/04/29 16:03:29 [INFO] raft: Removed peer b7abd8b7-b511-6b5f-6f37-289b72cceb0c, stopping replication
2014/04/29 16:03:29 [DEBUG] raft: Node ef69a1c0-f12e-6bed-8dc5-f1528674d220 updated peer set (remove): [ef69a1c0-f12e-6bed-8dc5-f1528674d220 30bd734d-5c54-1ec3-0263-d9fc40ef25d4]
2014/04/29 16:03:29 [DEBUG] raft: Node 30bd734d-5c54-1ec3-0263-d9fc40ef25d4 updated peer set (remove): [ef69a1c0-f12e-6bed-8dc5-f1528674d220 30bd734d-5c54-1ec3-0263-d9fc40ef25d4]
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at b7abd8b7-b511-6b5f-6f37-289b72cceb0c [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [INFO] raft: Node at ef69a1c0-f12e-6bed-8dc5-f1528674d220 [Follower] entering Follower state
2014/04/29 16:03:29 [WARN] raft: Remote peer ef69a1c0-f12e-6bed-8dc5-f1528674d220 does not have local node b7abd8b7-b511-6b5f-6f37-289b72cceb0c as a peer
2014/04/29 16:03:29 [WARN] raft: Remote peer 30bd734d-5c54-1ec3-0263-d9fc40ef25d4 does not have local node b7abd8b7-b511-6b5f-6f37-289b72cceb0c as a peer
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Node at b7abd8b7-b511-6b5f-6f37-289b72cceb0c [Leader] entering Leader state
2014/04/29 16:03:29 [DEBUG] raft: Node b7abd8b7-b511-6b5f-6f37-289b72cceb0c updated peer set (remove): [ef69a1c0-f12e-6bed-8dc5-f1528674d220 30bd734d-5c54-1ec3-0263-d9fc40ef25d4]
2014/04/29 16:03:29 [INFO] raft: Removed peer ef69a1c0-f12e-6bed-8dc5-f1528674d220, stopping replication
2014/04/29 16:03:29 [INFO] raft: Removed peer 30bd734d-5c54-1ec3-0263-d9fc40ef25d4, stopping replication
2014/04/29 16:03:29 [INFO] raft: Removed ourself, transitioning to follower
2014/04/29 16:03:29 [INFO] raft: Node at b7abd8b7-b511-6b5f-6f37-289b72cceb0c [Follower] entering Follower state
==================
WARNING: DATA RACE
Read by goroutine 105:
  github.com/benbjohnson/raft.TestRaft_ReJoinFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:1009 +0x57f
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 112:
  github.com/benbjohnson/raft.(*Raft).processLog()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:975 +0x1540
  github.com/benbjohnson/raft.(*Raft).processLogs()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:918 +0x428
  github.com/benbjohnson/raft.(*Raft).appendEntries()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:1119 +0xd95
  github.com/benbjohnson/raft.(*Raft).processRPC()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:1023 +0x1b3
  github.com/benbjohnson/raft.(*Raft).runFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:516 +0x1f8
  github.com/benbjohnson/raft.(*Raft).run()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:499 +0xd2
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.run)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 105 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 112 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0xd8b
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_ReJoinFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:978 +0x81
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
==================
WARNING: DATA RACE
Read by goroutine 105:
  github.com/benbjohnson/raft.(*StaticPeers).Peers()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/peer.go:37 +0x35
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:159 +0x54f
  github.com/benbjohnson/raft.TestRaft_ReJoinFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:1020 +0x723
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 109:
  github.com/benbjohnson/raft.(*StaticPeers).SetPeers()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/peer.go:41 +0x35
  github.com/benbjohnson/raft.(*Raft).processLog()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:973 +0xe84
  github.com/benbjohnson/raft.(*Raft).processLogs()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:918 +0x428
  github.com/benbjohnson/raft.(*Raft).leaderLoop()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:707 +0x344
  github.com/benbjohnson/raft.(*Raft).runLeader()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:669 +0x517
  github.com/benbjohnson/raft.(*Raft).run()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:503 +0x106
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.run)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 105 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 109 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0xd8b
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_ReJoinFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:978 +0x81
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
==================
WARNING: DATA RACE
Read by goroutine 105:
  github.com/benbjohnson/raft.ExcludePeer()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/util.go:83 +0xdd
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:163 +0x6bb
  github.com/benbjohnson/raft.TestRaft_ReJoinFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:1020 +0x723
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 109:
  github.com/benbjohnson/raft.(*Raft).processLog()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:973 +0xdec
  github.com/benbjohnson/raft.(*Raft).processLogs()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:918 +0x428
  github.com/benbjohnson/raft.(*Raft).leaderLoop()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:707 +0x344
  github.com/benbjohnson/raft.(*Raft).runLeader()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:669 +0x517
  github.com/benbjohnson/raft.(*Raft).run()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:503 +0x106
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.run)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 105 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 109 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:202 +0xd8b
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_ReJoinFollower()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:978 +0x81
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
2014/04/29 16:03:29 [INFO] raft: Node at b7abd8b7-b511-6b5f-6f37-289b72cceb0c [Follower] entering Follower state
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at 30bd734d-5c54-1ec3-0263-d9fc40ef25d4 [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Node at 30bd734d-5c54-1ec3-0263-d9fc40ef25d4 [Leader] entering Leader state
2014/04/29 16:03:29 [WARN] raft: EnableSingleNode disabled, and no known peers. Aborting election.
2014/04/29 16:03:29 [WARN] raft: EnableSingleNode disabled, and no known peers. Aborting election.
2014/04/29 16:03:29 [DEBUG] raft: Node 30bd734d-5c54-1ec3-0263-d9fc40ef25d4 updated peer set (add): [b7abd8b7-b511-6b5f-6f37-289b72cceb0c 30bd734d-5c54-1ec3-0263-d9fc40ef25d4 ef69a1c0-f12e-6bed-8dc5-f1528674d220]
2014/04/29 16:03:29 [INFO] raft: Added peer b7abd8b7-b511-6b5f-6f37-289b72cceb0c, starting replication
2014/04/29 16:03:29 [WARN] raft: Failed to get previous log: 5 log not found (last: 3)
2014/04/29 16:03:29 [WARN] raft: AppendEntries to b7abd8b7-b511-6b5f-6f37-289b72cceb0c rejected, sending older logs (next: 4)
2014/04/29 16:03:29 [DEBUG] raft: Node 30bd734d-5c54-1ec3-0263-d9fc40ef25d4 updated peer set (add): [b7abd8b7-b511-6b5f-6f37-289b72cceb0c 30bd734d-5c54-1ec3-0263-d9fc40ef25d4 ef69a1c0-f12e-6bed-8dc5-f1528674d220]
2014/04/29 16:03:29 [DEBUG] raft: Node ef69a1c0-f12e-6bed-8dc5-f1528674d220 updated peer set (add): [b7abd8b7-b511-6b5f-6f37-289b72cceb0c 30bd734d-5c54-1ec3-0263-d9fc40ef25d4 ef69a1c0-f12e-6bed-8dc5-f1528674d220]
2014/04/29 16:03:29 [DEBUG] raft: Node b7abd8b7-b511-6b5f-6f37-289b72cceb0c updated peer set (add): [b7abd8b7-b511-6b5f-6f37-289b72cceb0c 30bd734d-5c54-1ec3-0263-d9fc40ef25d4 ef69a1c0-f12e-6bed-8dc5-f1528674d220]
--- PASS: TestRaft_ReJoinFollower (0.09 seconds)
=== RUN TestRaft_LeaderLeaseExpire
2014/04/29 16:03:29 [WARN] Fully Connecting
2014/04/29 16:03:29 [INFO] raft: Node at 72bbb540-1259-f642-7370-d02a54646151 [Follower] entering Follower state
2014/04/29 16:03:29 [INFO] raft: Node at 8f998608-3526-543c-e247-278c72135f9e [Follower] entering Follower state
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at 8f998608-3526-543c-e247-278c72135f9e [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Node at 8f998608-3526-543c-e247-278c72135f9e [Leader] entering Leader state
2014/04/29 16:03:29 [INFO] Disconnecting Node at 72bbb540-1259-f642-7370-d02a54646151 [Follower]
2014/04/29 16:03:29 [WARN] Disconnecting 72bbb540-1259-f642-7370-d02a54646151
2014/04/29 16:03:29 [ERR] raft: Failed to heartbeat to 72bbb540-1259-f642-7370-d02a54646151: Failed to connect to peer: 72bbb540-1259-f642-7370-d02a54646151
2014/04/29 16:03:29 [ERR] raft: Failed to AppendEntries to 72bbb540-1259-f642-7370-d02a54646151: Failed to connect to peer: 72bbb540-1259-f642-7370-d02a54646151
2014/04/29 16:03:29 [ERR] raft: Failed to heartbeat to 72bbb540-1259-f642-7370-d02a54646151: Failed to connect to peer: 72bbb540-1259-f642-7370-d02a54646151
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at 72bbb540-1259-f642-7370-d02a54646151 [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [ERR] raft: Failed to make RequestVote RPC to 8f998608-3526-543c-e247-278c72135f9e: Failed to connect to peer: 8f998608-3526-543c-e247-278c72135f9e
2014/04/29 16:03:29 [ERR] raft: Failed to AppendEntries to 72bbb540-1259-f642-7370-d02a54646151: Failed to connect to peer: 72bbb540-1259-f642-7370-d02a54646151
2014/04/29 16:03:29 [ERR] raft: Failed to heartbeat to 72bbb540-1259-f642-7370-d02a54646151: Failed to connect to peer: 72bbb540-1259-f642-7370-d02a54646151
2014/04/29 16:03:29 [ERR] raft: Failed to AppendEntries to 72bbb540-1259-f642-7370-d02a54646151: Failed to connect to peer: 72bbb540-1259-f642-7370-d02a54646151
2014/04/29 16:03:29 [WARN] raft: Election timeout reached, restarting election
2014/04/29 16:03:29 [INFO] raft: Node at 72bbb540-1259-f642-7370-d02a54646151 [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [ERR] raft: Failed to make RequestVote RPC to 8f998608-3526-543c-e247-278c72135f9e: Failed to connect to peer: 8f998608-3526-543c-e247-278c72135f9e
2014/04/29 16:03:29 [WARN] raft: Failed to contact 72bbb540-1259-f642-7370-d02a54646151 in 40.739553ms
2014/04/29 16:03:29 [WARN] raft: Failed to contact quorum of nodes, stepping down
2014/04/29 16:03:29 [INFO] raft: Node at 8f998608-3526-543c-e247-278c72135f9e [Follower] entering Follower state
2014/04/29 16:03:29 [WARN] raft: Election timeout reached, restarting election
2014/04/29 16:03:29 [INFO] raft: Node at 72bbb540-1259-f642-7370-d02a54646151 [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [ERR] raft: Failed to make RequestVote RPC to 8f998608-3526-543c-e247-278c72135f9e: Failed to connect to peer: 8f998608-3526-543c-e247-278c72135f9e
2014/04/29 16:03:29 [ERR] raft: Failed to heartbeat to 72bbb540-1259-f642-7370-d02a54646151: Failed to connect to peer: 72bbb540-1259-f642-7370-d02a54646151
2014/04/29 16:03:29 [ERR] raft: Failed to AppendEntries to 72bbb540-1259-f642-7370-d02a54646151: Failed to connect to peer: 72bbb540-1259-f642-7370-d02a54646151
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at 8f998608-3526-543c-e247-278c72135f9e [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [ERR] raft: Failed to make RequestVote RPC to 72bbb540-1259-f642-7370-d02a54646151: Failed to connect to peer: 72bbb540-1259-f642-7370-d02a54646151
2014/04/29 16:03:29 [WARN] raft: Election timeout reached, restarting election
2014/04/29 16:03:29 [INFO] raft: Node at 72bbb540-1259-f642-7370-d02a54646151 [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [ERR] raft: Failed to make RequestVote RPC to 8f998608-3526-543c-e247-278c72135f9e: Failed to connect to peer: 8f998608-3526-543c-e247-278c72135f9e
2014/04/29 16:03:29 [WARN] raft: Election timeout reached, restarting election
2014/04/29 16:03:29 [INFO] raft: Node at 8f998608-3526-543c-e247-278c72135f9e [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [ERR] raft: Failed to make RequestVote RPC to 72bbb540-1259-f642-7370-d02a54646151: Failed to connect to peer: 72bbb540-1259-f642-7370-d02a54646151
2014/04/29 16:03:29 [WARN] raft: Election timeout reached, restarting election
2014/04/29 16:03:29 [INFO] raft: Node at 72bbb540-1259-f642-7370-d02a54646151 [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [ERR] raft: Failed to make RequestVote RPC to 8f998608-3526-543c-e247-278c72135f9e: Failed to connect to peer: 8f998608-3526-543c-e247-278c72135f9e
2014/04/29 16:03:29 [WARN] raft: Election timeout reached, restarting election
2014/04/29 16:03:29 [INFO] raft: Node at 8f998608-3526-543c-e247-278c72135f9e [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [ERR] raft: Failed to make RequestVote RPC to 72bbb540-1259-f642-7370-d02a54646151: Failed to connect to peer: 72bbb540-1259-f642-7370-d02a54646151
--- PASS: TestRaft_LeaderLeaseExpire (0.12 seconds)
=== RUN TestRaft_Barrier
2014/04/29 16:03:29 [WARN] Fully Connecting
2014/04/29 16:03:29 [INFO] raft: Node at 542f445a-594e-6ae4-0073-1f2a3360e698 [Follower] entering Follower state
2014/04/29 16:03:29 [INFO] raft: Node at cb9ebbf9-04f2-3b83-2c0d-45b9661f2773 [Follower] entering Follower state
2014/04/29 16:03:29 [INFO] raft: Node at 43b9b409-26db-b8e0-9d99-fe190ce183af [Follower] entering Follower state
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at 43b9b409-26db-b8e0-9d99-fe190ce183af [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Node at 43b9b409-26db-b8e0-9d99-fe190ce183af [Leader] entering Leader state
==================
WARNING: DATA RACE
Read by goroutine 204:
  github.com/benbjohnson/raft.(*cluster).EnsureSame()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:200 +0x193
  github.com/benbjohnson/raft.TestRaft_Barrier()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:1126 +0x2fa
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 206:
  github.com/benbjohnson/raft.(*MockFSM).Apply()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:30 +0x175
  github.com/benbjohnson/raft.(*Raft).runFSM()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:465 +0xe73
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.runFSM)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 204 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 206 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0xdf9
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_Barrier()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:1106 +0x54
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
==================
WARNING: DATA RACE
Read by goroutine 204:
  github.com/benbjohnson/raft.(*cluster).EnsureSame()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:200 +0x1a1
  github.com/benbjohnson/raft.TestRaft_Barrier()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:1126 +0x2fa
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 209:
  github.com/benbjohnson/raft.(*MockFSM).Apply()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:30 +0x175
  github.com/benbjohnson/raft.(*Raft).runFSM()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:465 +0xe73
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.runFSM)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 204 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 209 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0xdf9
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_Barrier()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:1106 +0x54
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
==================
WARNING: DATA RACE
Read by goroutine 204:
  github.com/benbjohnson/raft.(*cluster).EnsureSame()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:210 +0x3a7
  github.com/benbjohnson/raft.TestRaft_Barrier()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:1126 +0x2fa
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 206:
  github.com/benbjohnson/raft.(*MockFSM).Apply()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:30 +0x139
  github.com/benbjohnson/raft.(*Raft).runFSM()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:465 +0xe73
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.runFSM)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 204 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 206 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0xdf9
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_Barrier()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:1106 +0x54
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
==================
WARNING: DATA RACE
Read by goroutine 204:
  github.com/benbjohnson/raft.(*cluster).EnsureSame()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:210 +0x3e6
  github.com/benbjohnson/raft.TestRaft_Barrier()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:1126 +0x2fa
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f

Previous write by goroutine 209:
  github.com/benbjohnson/raft.(*MockFSM).Apply()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:30 +0x139
  github.com/benbjohnson/raft.(*Raft).runFSM()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:465 +0xe73
  github.com/benbjohnson/raft.*Raft.(github.com/benbjohnson/raft.runFSM)·fm()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0x33
  github.com/benbjohnson/raft.func·007()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:143 +0x7b

Goroutine 204 (running) created at:
  testing.RunTests()
      /usr/local/go/src/pkg/testing/testing.go:471 +0xb3c
  testing.Main()
      /usr/local/go/src/pkg/testing/testing.go:403 +0xa2
  main.main()
      github.com/benbjohnson/raft/_test/_testmain.go:167 +0xdc

Goroutine 209 (running) created at:
  github.com/benbjohnson/raft.(*raftState).goFunc()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/state.go:144 +0xf1
  github.com/benbjohnson/raft.NewRaft()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft.go:203 +0xdf9
  github.com/benbjohnson/raft.MakeCluster()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:303 +0x119b
  github.com/benbjohnson/raft.TestRaft_Barrier()
      /Users/benbjohnson/go/src/github.com/benbjohnson/raft/raft_test.go:1106 +0x54
  testing.tRunner()
      /usr/local/go/src/pkg/testing/testing.go:391 +0x10f
==================
--- PASS: TestRaft_Barrier (0.07 seconds)
=== RUN TestRaft_VerifyLeader
2014/04/29 16:03:29 [WARN] Fully Connecting
2014/04/29 16:03:29 [INFO] raft: Node at eb56bc1c-13fd-d45e-63e1-901cbb9a258b [Follower] entering Follower state
2014/04/29 16:03:29 [INFO] raft: Node at db517165-6d73-e035-3244-42371136cb04 [Follower] entering Follower state
2014/04/29 16:03:29 [INFO] raft: Node at 5f17b68d-b56b-42e1-e0a9-f2ee1206b540 [Follower] entering Follower state
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at 5f17b68d-b56b-42e1-e0a9-f2ee1206b540 [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Node at 5f17b68d-b56b-42e1-e0a9-f2ee1206b540 [Leader] entering Leader state
--- PASS: TestRaft_VerifyLeader (0.02 seconds)
=== RUN TestRaft_VerifyLeader_Single
2014/04/29 16:03:29 [WARN] Fully Connecting
2014/04/29 16:03:29 [INFO] raft: Node at 591ae712-3e1b-2afe-4f4b-987009355644 [Follower] entering Follower state
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at 591ae712-3e1b-2afe-4f4b-987009355644 [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 1
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [INFO] raft: Election won. Tally: 1
2014/04/29 16:03:29 [INFO] raft: Node at 591ae712-3e1b-2afe-4f4b-987009355644 [Leader] entering Leader state
--- PASS: TestRaft_VerifyLeader_Single (0.02 seconds)
=== RUN TestRaft_VerifyLeader_Fail
2014/04/29 16:03:29 [WARN] Fully Connecting
2014/04/29 16:03:29 [INFO] raft: Node at 02a0e22b-5fbd-19e1-ee52-0116a0bd62cd [Follower] entering Follower state
2014/04/29 16:03:29 [INFO] raft: Node at b0805341-7ca0-82da-9fe2-754de4f22f72 [Follower] entering Follower state
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at 02a0e22b-5fbd-19e1-ee52-0116a0bd62cd [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Node at 02a0e22b-5fbd-19e1-ee52-0116a0bd62cd [Leader] entering Leader state
2014/04/29 16:03:29 [WARN] raft: New leader elected, stepping down
2014/04/29 16:03:29 [INFO] raft: Node at 02a0e22b-5fbd-19e1-ee52-0116a0bd62cd [Follower] entering Follower state
--- PASS: TestRaft_VerifyLeader_Fail (0.03 seconds)
=== RUN TestRaft_VerifyLeader_ParitalConnect
2014/04/29 16:03:29 [WARN] Fully Connecting
2014/04/29 16:03:29 [INFO] raft: Node at 8e164987-95c9-b8d1-2497-a5d56d2d9e8e [Follower] entering Follower state
2014/04/29 16:03:29 [INFO] raft: Node at 80496076-661e-370d-1cef-8634e62c4673 [Follower] entering Follower state
2014/04/29 16:03:29 [INFO] raft: Node at 17ab3269-0f8a-76ec-2ed1-3675f86434a9 [Follower] entering Follower state
2014/04/29 16:03:29 [WARN] raft: Heartbeat timeout reached, starting election
2014/04/29 16:03:29 [INFO] raft: Node at 17ab3269-0f8a-76ec-2ed1-3675f86434a9 [Candidate] entering Candidate state
2014/04/29 16:03:29 [DEBUG] raft: Votes needed: 2
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 1
2014/04/29 16:03:29 [DEBUG] raft: Vote granted. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Election won. Tally: 2
2014/04/29 16:03:29 [INFO] raft: Node at 17ab3269-0f8a-76ec-2ed1-3675f86434a9 [Leader] entering Leader state
2014/04/29 16:03:29 [INFO] Disconnecting Node at 8e164987-95c9-b8d1-2497-a5d56d2d9e8e [Follower]
2014/04/29 16:03:29 [WARN] Disconnecting 8e164987-95c9-b8d1-2497-a5d56d2d9e8e
2014/04/29 16:03:29 [ERR] raft: Failed to heartbeat to 8e164987-95c9-b8d1-2497-a5d56d2d9e8e: Failed to connect to peer: 8e164987-95c9-b8d1-2497-a5d56d2d9e8e
2014/04/29 16:03:29 [ERR] raft: Failed to AppendEntries to 80496076-661e-370d-1cef-8634e62c4673: command timed out
2014/04/29 16:03:29 [ERR] raft: Failed to heartbeat to 80496076-661e-370d-1cef-8634e62c4673: command timed out
--- PASS: TestRaft_VerifyLeader_ParitalConnect (0.08 seconds)
=== RUN TestTCPTransport_BadAddr
--- PASS: TestTCPTransport_BadAddr (0.00 seconds)
=== RUN TestTCPTransport_WithAdvertise
--- PASS: TestTCPTransport_WithAdvertise (0.00 seconds)
=== RUN TestRandomTimeout
--- PASS: TestRandomTimeout (0.00 seconds)
=== RUN TestRandomTimeout_NoTime
--- PASS: TestRandomTimeout_NoTime (0.00 seconds)
=== RUN TestMin
--- PASS: TestMin (0.00 seconds)
=== RUN TestMax
--- PASS: TestMax (0.00 seconds)
=== RUN TestGenerateUUID
--- PASS: TestGenerateUUID (0.04 seconds)
=== RUN TestAsyncNotify
--- PASS: TestAsyncNotify (0.00 seconds)
=== RUN TestExcludePeer
--- PASS: TestExcludePeer (0.00 seconds)
=== RUN TestPeerContained
--- PASS: TestPeerContained (0.00 seconds)
=== RUN TestAddUniquePeer
--- PASS: TestAddUniquePeer (0.00 seconds)
=== RUN TestEncodeDecodePeers
--- PASS: TestEncodeDecodePeers (0.00 seconds)
=== RUN TestBackoff
--- PASS: TestBackoff (0.00 seconds)
PASS
Found 41 data race(s)
exit status 66
FAIL	github.com/benbjohnson/raft	2.864s