Skip to content

Instantly share code, notes, and snippets.

@jeffjirsa

jeffjirsa/Change from hub spoke to full mesh

Last active November 22, 2015 08:06
Show Gist options
  • Save jeffjirsa/e8d66a203619e3687439 to your computer and use it in GitHub Desktop.
Save jeffjirsa/e8d66a203619e3687439 to your computer and use it in GitHub Desktop.
Download ZIP
7306 WIP & experiments
Raw
add-data-7306.py
#!/usr/bin/python
import uuid
import cql
import time
schema_host = "127.0.0.1"
host = "127.0.0.3" # Writes data to node in dc3, will replicate to dc1 until dc2 is joined, then dc1+dc2
port = 9160
schema_con = cql.connect(schema_host, port, cql_version='3.0.0')
query_con = cql.connect(host, port, cql_version='3.0.0')
cursor = query_con.cursor()
query = """CREATE KEYSPACE IF NOT EXISTS test WITH replication = {'class': 'NetworkTopologyStrategy', 'datacenter1':1, 'datacenter2':1, 'datacenter3':1};"""
print schema_con.cursor().execute(query)
query = """USE test; """
print cursor.execute(query)
print schema_con.cursor().execute(query)
query = """CREATE TABLE IF NOT EXISTS dtcs (id int, value int, primary key (id)); """
print schema_con.cursor().execute(query)
for i in range(0, 10000000):
query = """INSERT INTO dtcs (id, value) VALUES(:i, :inc) USING TTL 3600; """
params = {'i': i, 'inc': i}
r = cursor.execute(query, params, consistency_level="ALL")
if r is False or i % 10 == 0:
print(i)
if i % 100 == 1:
time.sleep(1/10)
Raw
cassandra-7306
ccm remove test;
ccm create test --install-dir=/Users/jeff/Desktop/Dev/cassandra/ -n 3 ;
sed -i'' -e 's/SimpleSnitch/GossipingPropertyFileSnitch/g' ~/.ccm/test/node1/conf/cassandra.yaml;
sed -i'' -e 's/SimpleSnitch/GossipingPropertyFileSnitch/g' ~/.ccm/test/node2/conf/cassandra.yaml;
sed -i'' -e 's/SimpleSnitch/GossipingPropertyFileSnitch/g' ~/.ccm/test/node3/conf/cassandra.yaml ;
sed -i'' -e 's/MeshDatacenter/WhitelistingPropertyFileDatacenter/g' ~/.ccm/test/node1/conf/cassandra.yaml ;
sed -i'' -e 's/MeshDatacenter/WhitelistingPropertyFileDatacenter/g' ~/.ccm/test/node2/conf/cassandra.yaml ;
sed -i'' -e 's/MeshDatacenter/WhitelistingPropertyFileDatacenter/g' ~/.ccm/test/node3/conf/cassandra.yaml ;
sed -i'' -e 's/seeds: 127.0.0.1,127.0.0.2,127.0.0.3/seeds: 127.0.0.1,127.0.0.3/g' ~/.ccm/test/node3/conf/cassandra.yaml;
echo 'include_datacenters=datacenter1,datacenter2,datacenter3' > ~/.ccm/test/node1/conf/cassandra-dctopology.properties ;
echo 'include_datacenters=datacenter1,datacenter2' > ~/.ccm/test/node2/conf/cassandra-dctopology.properties ;
echo 'include_datacenters=datacenter1,datacenter3' > ~/.ccm/test/node3/conf/cassandra-dctopology.properties ;
sed -i'' -e 's/dc1/datacenter1/g' ~/.ccm/test/node1/conf/cassandra-rackdc.properties ;
sed -i'' -e 's/dc1/datacenter2/g' ~/.ccm/test/node2/conf/cassandra-rackdc.properties ;
sed -i'' -e 's/dc1/datacenter3/g' ~/.ccm/test/node3/conf/cassandra-rackdc.properties ;
ccm node1 start ;
ccm node1 nodetool enablethrift ;
ccm node2 start ;
ccm node3 start ;
ccm node2 nodetool enablethrift ;
ccm node3 nodetool enablethrift
while true; do ccm node1 flush; ccm node2 flush; ccm node3 flush ; sleep 5; done
Raw
Change from hub spoke to full mesh
echo 'include_datacenters=datacenter1,datacenter2,datacenter3' > ~/.ccm/test/node3/conf/cassandra-dctopology.properties
echo 'include_datacenters=datacenter1,datacenter2,datacenter3' > ~/.ccm/test/node2/conf/cassandra-dctopology.properties
Raw
Isolated, with add-data running
# Data only goes to DCs 1 and 3
bash-3.2# ccm node1 nodetool ring
objc[38011]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home//bin/java and /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/libinstrument.dylib. One of the two will be used. Which one is undefined.
Datacenter: datacenter1
==========
Address Rack Status State Load Owns Token
127.0.0.1 rack1 Up Normal 927.18 KB ? -9223372036854775808
Datacenter: datacenter2
==========
Address Rack Status State Load Owns Token
127.0.0.2 rack1 Up Normal 107.37 KB ? -3074457345618258603
Datacenter: datacenter3
==========
Address Rack Status State Load Owns Token
127.0.0.3 rack1 Up Normal 78.96 KB ? 3074457345618258602
Note: Non-system keyspaces don't have the same replication settings, effective ownership information is meaningless
bash-3.2# ccm node2 nodetool ring
objc[38068]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home//bin/java and /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/libinstrument.dylib. One of the two will be used. Which one is undefined.
Datacenter: datacenter1
==========
Address Rack Status State Load Owns Token
127.0.0.1 rack1 Up Normal 107.27 KB ? -9223372036854775808
Datacenter: datacenter2
==========
Address Rack Status State Load Owns Token
127.0.0.2 rack1 Up Normal 74.03 KB ? -3074457345618258603
Datacenter: datacenter3 [Disconnected]
==========
Address Rack Status State Load Owns Token
127.0.0.3 rack1 Up Normal 78.96 KB ? 3074457345618258602
Note: Non-system keyspaces don't have the same replication settings, effective ownership information is meaningless
bash-3.2# ccm node3 nodetool ring
objc[38152]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home//bin/java and /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/libinstrument.dylib. One of the two will be used. Which one is undefined.
Datacenter: datacenter1
==========
Address Rack Status State Load Owns Token
127.0.0.1 rack1 Up Normal 107.27 KB ? -9223372036854775808
Datacenter: datacenter2 [Disconnected]
==========
Address Rack Status State Load Owns Token
127.0.0.2 rack1 Up Normal 107.37 KB ? -3074457345618258603
Datacenter: datacenter3
==========
Address Rack Status State Load Owns Token
127.0.0.3 rack1 Up Normal 1.76 MB ? 3074457345618258602
Note: Non-system keyspaces don't have the same replication settings, effective ownership information is meaningless
Raw
Joined, with add-data running
#Data is now flowing into dc2
bash-3.2# ccm node1 nodetool ring
objc[39203]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home//bin/java and /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/libinstrument.dylib. One of the two will be used. Which one is undefined.
Datacenter: datacenter1
==========
Address Rack Status State Load Owns Token
127.0.0.1 rack1 Up Normal 7.39 MB ? -9223372036854775808
Datacenter: datacenter2
==========
Address Rack Status State Load Owns Token
127.0.0.2 rack1 Up Normal 74.03 KB ? -3074457345618258603
Datacenter: datacenter3
==========
Address Rack Status State Load Owns Token
127.0.0.3 rack1 Up Normal 7.1 MB ? 3074457345618258602
Note: Non-system keyspaces don't have the same replication settings, effective ownership information is meaningless
bash-3.2# ccm node2 nodetool ring
objc[39261]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home//bin/java and /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/libinstrument.dylib. One of the two will be used. Which one is undefined.
Datacenter: datacenter1
==========
Address Rack Status State Load Owns Token
127.0.0.1 rack1 Up Normal 6.1 MB ? -9223372036854775808
Datacenter: datacenter2
==========
Address Rack Status State Load Owns Token
127.0.0.2 rack1 Up Normal 458.92 KB ? -3074457345618258603
Datacenter: datacenter3
==========
Address Rack Status State Load Owns Token
127.0.0.3 rack1 Up Normal 7.1 MB ? 3074457345618258602
Note: Non-system keyspaces don't have the same replication settings, effective ownership information is meaningless
bash-3.2# ccm node3 nodetool ring
objc[39344]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home//bin/java and /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/libinstrument.dylib. One of the two will be used. Which one is undefined.
Datacenter: datacenter1
==========
Address Rack Status State Load Owns Token
127.0.0.1 rack1 Up Normal 6.1 MB ? -9223372036854775808
Datacenter: datacenter2
==========
Address Rack Status State Load Owns Token
127.0.0.2 rack1 Up Normal 74.03 KB ? -3074457345618258603
Datacenter: datacenter3
==========
Address Rack Status State Load Owns Token
127.0.0.3 rack1 Up Normal 8.23 MB ? 3074457345618258602
Note: Non-system keyspaces don't have the same replication settings, effective ownership information is meaningless
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment