CAS (atomic compare-and-swap) support (https://issues.apache.org/jira/browse/CASSANDRA-5062) is now finished in Cassandra for 2.0, along with the earlier corresponding CQL support (https://issues.apache.org/jira/browse/CASSANDRA-5443). Specifically this support was implemented in the context of high latency, much like C* in general.
In their implementation, they chose to do a Paxos implementation; Zookeeper is not used at all.
CAS is a basic building block, much like the ZNode ops in ZK. In particular, here's a first attempt at a recipe to do leader election:
- Potential leaders each generate a UUID
- Attempt to store the UUID; if the following returns true, this means it's the leader:
UPDATE elections SET leader = $UUID WHERE group = $MY_GROUP IF leader = $OLD_LEADER_UUID;
For heartbeats, simply update a heartbeat status, with a TTL set appropriately on the column. Followers will simply re-read this column on a regular basis. When they detect deletion (using QUORUM consistency for multiple DC; LOCAL_QUORUM would allow this to work for single DC), they will attempt to become leaders, as above.
Note that eventual consistency for when this heartbeat failure is detected is OK, since we don't need to care about guaranteeing the fairness of the election, just that some leader is elected.
The heartbeat status should be keyed to the current leader UUID to ensure that an immediate leader failure can be detected.