Last active
August 29, 2015 14:20
-
-
Save JensRantil/18aab0579d59f91224a3 to your computer and use it in GitHub Desktop.
Benchmark of how vnodes impact Shark/Spark execution performance for DSE 4.6. See `output.txt` for execution.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
m3-large |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import uuid | |
import sys | |
n = int(sys.argv[1]) | |
print >> sys.stderr, "Populating", n, "items." | |
for i in range(n): | |
print "INSERT INTO emp (empID, deptID, first_name, last_name) VALUES ({0},{1}, '{2}', '{3}');".format(i, i*2, str(uuid.uuid4()), str(uuid.uuid4())) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
root@ip-10-121-138-247:/home/ubuntu# # Installing Datastax Enterprise 4.6 using apt. | |
root@ip-10-121-138-247:/home/ubuntu# # # Enabling vnodes in /etc/dse/cassandra/cassandra.yaml. | |
root@ip-10-121-138-247:/home/ubuntu# time ./sparkbench_all.sh > with_vnodes_result.txt 2>&1 | |
real 1411m25.761s | |
user 979m25.149s | |
sys 68m24.381s | |
root@ip-10-121-138-247:/home/ubuntu# # # Disabling vnodes in /etc/dse/cassandra/cassandra.yaml. | |
root@ip-10-121-138-247:/home/ubuntu# time ./sparkbench_all.sh > without_vnodes_result.txt 2>&1 | |
real 1382m2.011s | |
user 966m52.110s | |
sys 68m21.212s |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
set -e | |
echo Running benchmark for $1 items. | |
service dse stop | |
cd /var/lib/cassandra | |
rm -fr * | |
service dse start | |
# Wait until started | |
sleep 100 | |
cd ~ubuntu | |
cqlsh < sparkbench_cqlsh.txt | |
python gen.py $1 | cqlsh -k testkeyspace | |
dse shark < sparkbench_spark.txt | |
echo Done with benchmark. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
./sparkbench.sh 2 | |
./sparkbench.sh 10 | |
./sparkbench.sh 100 | |
./sparkbench.sh 1000 | |
./sparkbench.sh 10000 | |
./sparkbench.sh 100000 | |
./sparkbench.sh 1000000 | |
./sparkbench.sh 10000000 | |
./sparkbench.sh 100000000 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CREATE KEYSPACE testkeyspace WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '1'}; | |
USE testkeyspace; | |
CREATE TABLE emp ( empID int, deptID int, first_name varchar, last_name varchar, PRIMARY KEY (empID, deptID) ); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
USE testkeyspace; | |
SELECT COUNT(*) FROM emp; | |
SELECT COUNT(*) FROM emp; | |
SELECT COUNT(*) FROM emp; | |
SELECT COUNT(*) FROM emp; | |
SELECT COUNT(*) FROM emp; | |
SELECT COUNT(*) FROM emp; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Running benchmark for 2 items. | |
* Stopping DSE daemon dse | |
...done. | |
* Starting DSE daemon dse | |
DSE daemon starting with Spark enabled (edit /etc/default/dse to disable) | |
DSE daemon starting with CFS enabled (edit /etc/default/dse to disable) | |
...done. | |
Populating 2 items. | |
Starting the Shark Command Line Client | |
Logging initialized using configuration in /etc/dse/shark/log4j.properties | |
Reloading cached RDDs from previous Shark sessions... (use -skipRddReload flag to skip reloading) | |
shark> USE testkeyspace; | |
OK | |
Time taken: 0.42 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
2 | |
Time taken: 38.908 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
2 | |
Time taken: 23.307 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
2 | |
Time taken: 20.544 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
2 | |
Time taken: 19.629 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
2 | |
Time taken: 18.264 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
2 | |
Time taken: 18.402 seconds | |
shark> Done with benchmark. | |
Running benchmark for 10 items. | |
* Stopping DSE daemon dse | |
...done. | |
* Starting DSE daemon dse | |
DSE daemon starting with Spark enabled (edit /etc/default/dse to disable) | |
DSE daemon starting with CFS enabled (edit /etc/default/dse to disable) | |
...done. | |
Populating 10 items. | |
Starting the Shark Command Line Client | |
Logging initialized using configuration in /etc/dse/shark/log4j.properties | |
Reloading cached RDDs from previous Shark sessions... (use -skipRddReload flag to skip reloading) | |
shark> USE testkeyspace; | |
OK | |
Time taken: 0.42 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10 | |
Time taken: 39.205 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10 | |
Time taken: 23.442 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10 | |
Time taken: 20.537 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10 | |
Time taken: 19.264 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10 | |
Time taken: 18.668 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10 | |
Time taken: 18.271 seconds | |
shark> 15/05/01 17:37:12 ERROR network.ConnectionManager: Corresponding SendingConnection to ConnectionManagerId(localhost,60047) not found | |
Done with benchmark. | |
Running benchmark for 100 items. | |
* Stopping DSE daemon dse | |
...done. | |
* Starting DSE daemon dse | |
DSE daemon starting with Spark enabled (edit /etc/default/dse to disable) | |
DSE daemon starting with CFS enabled (edit /etc/default/dse to disable) | |
...done. | |
Populating 100 items. | |
Starting the Shark Command Line Client | |
Logging initialized using configuration in /etc/dse/shark/log4j.properties | |
Reloading cached RDDs from previous Shark sessions... (use -skipRddReload flag to skip reloading) | |
shark> USE testkeyspace; | |
OK | |
Time taken: 0.422 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100 | |
Time taken: 38.668 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100 | |
Time taken: 23.751 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100 | |
Time taken: 20.571 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100 | |
Time taken: 19.273 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100 | |
Time taken: 18.555 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100 | |
Time taken: 18.254 seconds | |
shark> Done with benchmark. | |
Running benchmark for 1000 items. | |
* Stopping DSE daemon dse | |
...done. | |
* Starting DSE daemon dse | |
DSE daemon starting with Spark enabled (edit /etc/default/dse to disable) | |
DSE daemon starting with CFS enabled (edit /etc/default/dse to disable) | |
...done. | |
Populating 1000 items. | |
Starting the Shark Command Line Client | |
Logging initialized using configuration in /etc/dse/shark/log4j.properties | |
Reloading cached RDDs from previous Shark sessions... (use -skipRddReload flag to skip reloading) | |
shark> USE testkeyspace; | |
OK | |
Time taken: 0.424 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000 | |
Time taken: 39.335 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000 | |
Time taken: 23.683 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000 | |
Time taken: 21.079 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000 | |
Time taken: 19.698 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000 | |
Time taken: 18.666 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000 | |
Time taken: 18.778 seconds | |
shark> Done with benchmark. | |
Running benchmark for 10000 items. | |
* Stopping DSE daemon dse | |
...done. | |
* Starting DSE daemon dse | |
DSE daemon starting with Spark enabled (edit /etc/default/dse to disable) | |
DSE daemon starting with CFS enabled (edit /etc/default/dse to disable) | |
...done. | |
Populating 10000 items. | |
Starting the Shark Command Line Client | |
Logging initialized using configuration in /etc/dse/shark/log4j.properties | |
Reloading cached RDDs from previous Shark sessions... (use -skipRddReload flag to skip reloading) | |
shark> USE testkeyspace; | |
OK | |
Time taken: 0.418 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000 | |
Time taken: 39.366 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000 | |
Time taken: 23.92 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000 | |
Time taken: 20.984 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000 | |
Time taken: 19.482 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000 | |
Time taken: 18.826 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000 | |
Time taken: 18.853 seconds | |
shark> 15/05/01 17:50:34 ERROR network.ConnectionManager: Corresponding SendingConnection to ConnectionManagerId(localhost,40091) not found | |
15/05/01 17:50:34 ERROR network.ConnectionManager: Corresponding SendingConnection to ConnectionManagerId(localhost,40091) not found | |
15/05/01 17:50:34 WARN network.ConnectionManager: All connections not cleaned up | |
Done with benchmark. | |
Running benchmark for 100000 items. | |
* Stopping DSE daemon dse | |
...done. | |
* Starting DSE daemon dse | |
DSE daemon starting with Spark enabled (edit /etc/default/dse to disable) | |
DSE daemon starting with CFS enabled (edit /etc/default/dse to disable) | |
...done. | |
Populating 100000 items. | |
Starting the Shark Command Line Client | |
Logging initialized using configuration in /etc/dse/shark/log4j.properties | |
Reloading cached RDDs from previous Shark sessions... (use -skipRddReload flag to skip reloading) | |
shark> USE testkeyspace; | |
OK | |
Time taken: 0.416 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000 | |
Time taken: 41.032 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000 | |
Time taken: 25.189 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000 | |
Time taken: 22.138 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000 | |
Time taken: 20.478 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000 | |
Time taken: 19.607 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000 | |
Time taken: 19.827 seconds | |
shark> Done with benchmark. | |
Running benchmark for 1000000 items. | |
* Stopping DSE daemon dse | |
...done. | |
* Starting DSE daemon dse | |
DSE daemon starting with Spark enabled (edit /etc/default/dse to disable) | |
DSE daemon starting with CFS enabled (edit /etc/default/dse to disable) | |
...done. | |
Populating 1000000 items. | |
Starting the Shark Command Line Client | |
Logging initialized using configuration in /etc/dse/shark/log4j.properties | |
Reloading cached RDDs from previous Shark sessions... (use -skipRddReload flag to skip reloading) | |
shark> USE testkeyspace; | |
OK | |
Time taken: 0.421 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000000 | |
Time taken: 51.348 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000000 | |
Time taken: 34.197 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000000 | |
Time taken: 31.564 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000000 | |
Time taken: 30.569 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000000 | |
Time taken: 29.664 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000000 | |
Time taken: 29.094 seconds | |
shark> Done with benchmark. | |
Running benchmark for 10000000 items. | |
* Stopping DSE daemon dse | |
...done. | |
* Starting DSE daemon dse | |
DSE daemon starting with Spark enabled (edit /etc/default/dse to disable) | |
DSE daemon starting with CFS enabled (edit /etc/default/dse to disable) | |
...done. | |
Populating 10000000 items. | |
Starting the Shark Command Line Client | |
Logging initialized using configuration in /etc/dse/shark/log4j.properties | |
Reloading cached RDDs from previous Shark sessions... (use -skipRddReload flag to skip reloading) | |
shark> USE testkeyspace; | |
OK | |
Time taken: 0.519 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000000 | |
Time taken: 155.338 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000000 | |
Time taken: 135.418 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000000 | |
Time taken: 131.599 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000000 | |
Time taken: 131.105 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000000 | |
Time taken: 129.764 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000000 | |
Time taken: 129.513 seconds | |
shark> 15/05/01 20:18:02 WARN network.ConnectionManager: All connections not cleaned up | |
Done with benchmark. | |
Running benchmark for 100000000 items. | |
* Stopping DSE daemon dse | |
...done. | |
* Starting DSE daemon dse | |
DSE daemon starting with Spark enabled (edit /etc/default/dse to disable) | |
DSE daemon starting with CFS enabled (edit /etc/default/dse to disable) | |
...done. | |
Populating 100000000 items. | |
Starting the Shark Command Line Client | |
Logging initialized using configuration in /etc/dse/shark/log4j.properties | |
Reloading cached RDDs from previous Shark sessions... (use -skipRddReload flag to skip reloading) | |
shark> USE testkeyspace; | |
OK | |
Time taken: 0.466 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000000 | |
Time taken: 1259.254 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000000 | |
Time taken: 1238.126 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000000 | |
Time taken: 1233.475 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000000 | |
Time taken: 1234.755 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000000 | |
Time taken: 1232.848 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000000 | |
Time taken: 1236.304 seconds | |
shark> Done with benchmark. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Running benchmark for 2 items. | |
* Stopping DSE daemon dse | |
...done. | |
* Starting DSE daemon dse | |
DSE daemon starting with Spark enabled (edit /etc/default/dse to disable) | |
DSE daemon starting with CFS enabled (edit /etc/default/dse to disable) | |
...done. | |
Populating 2 items. | |
Starting the Shark Command Line Client | |
Logging initialized using configuration in /etc/dse/shark/log4j.properties | |
Reloading cached RDDs from previous Shark sessions... (use -skipRddReload flag to skip reloading) | |
shark> USE testkeyspace; | |
OK | |
Time taken: 0.413 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
2 | |
Time taken: 9.051 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
2 | |
Time taken: 1.931 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
2 | |
Time taken: 1.332 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
2 | |
Time taken: 1.868 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
2 | |
Time taken: 1.449 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
2 | |
Time taken: 1.152 seconds | |
shark> Done with benchmark. | |
Running benchmark for 10 items. | |
* Stopping DSE daemon dse | |
...done. | |
* Starting DSE daemon dse | |
DSE daemon starting with Spark enabled (edit /etc/default/dse to disable) | |
DSE daemon starting with CFS enabled (edit /etc/default/dse to disable) | |
...done. | |
Populating 10 items. | |
Starting the Shark Command Line Client | |
Logging initialized using configuration in /etc/dse/shark/log4j.properties | |
Reloading cached RDDs from previous Shark sessions... (use -skipRddReload flag to skip reloading) | |
shark> USE testkeyspace; | |
OK | |
Time taken: 0.409 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10 | |
Time taken: 9.034 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10 | |
Time taken: 1.872 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10 | |
Time taken: 1.368 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10 | |
Time taken: 1.869 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10 | |
Time taken: 1.443 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10 | |
Time taken: 1.239 seconds | |
shark> Done with benchmark. | |
Running benchmark for 100 items. | |
* Stopping DSE daemon dse | |
...done. | |
* Starting DSE daemon dse | |
DSE daemon starting with Spark enabled (edit /etc/default/dse to disable) | |
DSE daemon starting with CFS enabled (edit /etc/default/dse to disable) | |
...done. | |
Populating 100 items. | |
Starting the Shark Command Line Client | |
Logging initialized using configuration in /etc/dse/shark/log4j.properties | |
Reloading cached RDDs from previous Shark sessions... (use -skipRddReload flag to skip reloading) | |
shark> USE testkeyspace; | |
OK | |
Time taken: 0.412 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100 | |
Time taken: 9.246 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100 | |
Time taken: 1.92 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100 | |
Time taken: 1.43 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100 | |
Time taken: 1.914 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100 | |
Time taken: 1.493 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100 | |
Time taken: 1.147 seconds | |
shark> 15/05/02 19:09:22 ERROR network.ConnectionManager: Corresponding SendingConnection to ConnectionManagerId(localhost,44842) not found | |
Done with benchmark. | |
Running benchmark for 1000 items. | |
* Stopping DSE daemon dse | |
...done. | |
* Starting DSE daemon dse | |
DSE daemon starting with Spark enabled (edit /etc/default/dse to disable) | |
DSE daemon starting with CFS enabled (edit /etc/default/dse to disable) | |
...done. | |
Populating 1000 items. | |
Starting the Shark Command Line Client | |
Logging initialized using configuration in /etc/dse/shark/log4j.properties | |
Reloading cached RDDs from previous Shark sessions... (use -skipRddReload flag to skip reloading) | |
shark> USE testkeyspace; | |
OK | |
Time taken: 0.43 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000 | |
Time taken: 9.264 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000 | |
Time taken: 2.073 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000 | |
Time taken: 1.447 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000 | |
Time taken: 1.74 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000 | |
Time taken: 1.532 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000 | |
Time taken: 1.247 seconds | |
shark> 15/05/02 19:11:44 ERROR network.ConnectionManager: Corresponding SendingConnection to ConnectionManagerId(localhost,52733) not found | |
15/05/02 19:11:45 WARN network.ConnectionManager: All connections not cleaned up | |
Done with benchmark. | |
Running benchmark for 10000 items. | |
* Stopping DSE daemon dse | |
...done. | |
* Starting DSE daemon dse | |
DSE daemon starting with Spark enabled (edit /etc/default/dse to disable) | |
DSE daemon starting with CFS enabled (edit /etc/default/dse to disable) | |
...done. | |
Populating 10000 items. | |
Starting the Shark Command Line Client | |
Logging initialized using configuration in /etc/dse/shark/log4j.properties | |
Reloading cached RDDs from previous Shark sessions... (use -skipRddReload flag to skip reloading) | |
shark> USE testkeyspace; | |
OK | |
Time taken: 0.412 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000 | |
Time taken: 11.031 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000 | |
Time taken: 1.952 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000 | |
Time taken: 1.496 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000 | |
Time taken: 1.867 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000 | |
Time taken: 1.476 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000 | |
Time taken: 1.53 seconds | |
shark> Done with benchmark. | |
Running benchmark for 100000 items. | |
* Stopping DSE daemon dse | |
...done. | |
* Starting DSE daemon dse | |
DSE daemon starting with Spark enabled (edit /etc/default/dse to disable) | |
DSE daemon starting with CFS enabled (edit /etc/default/dse to disable) | |
...done. | |
Populating 100000 items. | |
Starting the Shark Command Line Client | |
Logging initialized using configuration in /etc/dse/shark/log4j.properties | |
Reloading cached RDDs from previous Shark sessions... (use -skipRddReload flag to skip reloading) | |
shark> USE testkeyspace; | |
OK | |
Time taken: 0.408 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000 | |
Time taken: 12.852 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000 | |
Time taken: 3.38 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000 | |
Time taken: 2.446 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000 | |
Time taken: 2.808 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000 | |
Time taken: 2.521 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000 | |
Time taken: 2.626 seconds | |
shark> Done with benchmark. | |
Running benchmark for 1000000 items. | |
* Stopping DSE daemon dse | |
...done. | |
* Starting DSE daemon dse | |
DSE daemon starting with Spark enabled (edit /etc/default/dse to disable) | |
DSE daemon starting with CFS enabled (edit /etc/default/dse to disable) | |
...done. | |
Populating 1000000 items. | |
Starting the Shark Command Line Client | |
Logging initialized using configuration in /etc/dse/shark/log4j.properties | |
Reloading cached RDDs from previous Shark sessions... (use -skipRddReload flag to skip reloading) | |
shark> USE testkeyspace; | |
OK | |
Time taken: 0.411 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000000 | |
Time taken: 25.325 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000000 | |
Time taken: 14.077 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000000 | |
Time taken: 13.889 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000000 | |
Time taken: 13.624 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000000 | |
Time taken: 13.332 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
1000000 | |
Time taken: 13.613 seconds | |
shark> 15/05/02 19:32:11 ERROR network.ConnectionManager: Corresponding SendingConnection to ConnectionManagerId(localhost,34444) not found | |
Done with benchmark. | |
Running benchmark for 10000000 items. | |
* Stopping DSE daemon dse | |
...done. | |
* Starting DSE daemon dse | |
DSE daemon starting with Spark enabled (edit /etc/default/dse to disable) | |
DSE daemon starting with CFS enabled (edit /etc/default/dse to disable) | |
...done. | |
Populating 10000000 items. | |
Starting the Shark Command Line Client | |
Logging initialized using configuration in /etc/dse/shark/log4j.properties | |
Reloading cached RDDs from previous Shark sessions... (use -skipRddReload flag to skip reloading) | |
shark> USE testkeyspace; | |
OK | |
Time taken: 0.406 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000000 | |
Time taken: 141.819 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000000 | |
Time taken: 124.746 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000000 | |
Time taken: 121.652 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000000 | |
Time taken: 121.609 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000000 | |
Time taken: 120.836 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
10000000 | |
Time taken: 120.774 seconds | |
shark> 15/05/02 21:37:11 ERROR network.ConnectionManager: Corresponding SendingConnection to ConnectionManagerId(localhost,36817) not found | |
Done with benchmark. | |
Running benchmark for 100000000 items. | |
* Stopping DSE daemon dse | |
...done. | |
* Starting DSE daemon dse | |
DSE daemon starting with Spark enabled (edit /etc/default/dse to disable) | |
DSE daemon starting with CFS enabled (edit /etc/default/dse to disable) | |
...done. | |
Populating 100000000 items. | |
Starting the Shark Command Line Client | |
Logging initialized using configuration in /etc/dse/shark/log4j.properties | |
Reloading cached RDDs from previous Shark sessions... (use -skipRddReload flag to skip reloading) | |
shark> USE testkeyspace; | |
OK | |
Time taken: 0.445 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000000 | |
Time taken: 1225.747 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000000 | |
Time taken: 1206.83 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000000 | |
Time taken: 1207.403 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000000 | |
Time taken: 1204.192 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000000 | |
Time taken: 1206.776 seconds | |
shark> SELECT COUNT(*) FROM emp; | |
OK | |
100000000 | |
Time taken: 1208.103 seconds | |
shark> 15/05/03 18:04:23 ERROR network.ConnectionManager: Corresponding SendingConnection to ConnectionManagerId(localhost,48652) not found | |
Done with benchmark. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment