Random cassandra benchmark results, comparing versions (looking for a specific regression noted by an IRC user)

gistfile1.txt

Test hardware = 3x blades, 2xE5630 @ 2.53GHz (cores, 16 HT), 24GB RAM , RAID0, -Xms6025M -Xmx6025M -Xmn1506M

1.2.19, 2 tokens per node, cql3
./tools/bin/cassandra-stress --operation INSERT --num-keys 1000000  --columns 20   --replication-factor 2 --enable-cql3 --column-size 128 --nodesfile=nodes

Averages from the middle 80% of values:
interval_op_rate          : 14815
interval_key_rate         : 14815
latency median            : 1.4
latency 95th percentile   : 3.6
latency 99.9th percentile : 553.0
Total operation time      : 00:01:10
END

1.2.19, 2 tokens per node, thrift
./tools/bin/cassandra-stress --operation INSERT --num-keys 1000000  --columns 20   --replication-factor 2 --column-size 128 --nodesfile=nodes

Averages from the middle 80% of values:
interval_op_rate          : 14542
interval_key_rate         : 14542
latency median            : 0.9
latency 95th percentile   : 2.6
latency 99.9th percentile : 433.8
Total operation time      : 00:01:11
END

1.2.19, 1 token per node, thrift 
# ./tools/bin/cassandra-stress --operation INSERT --num-keys 1000000  --columns 20   --replication-factor 2 --column-size 128 --nodesfile=nodes

Averages from the middle 80% of values:
interval_op_rate          : 14071
interval_key_rate         : 14071
latency median            : 0.9
latency 95th percentile   : 3.1
latency 99.9th percentile : 1354.0
Total operation time      : 00:01:12
END

1.2.19, cql3, 1 token per node, SMALL WRITES (3 columns)
# ./tools/bin/cassandra-stress --operation INSERT --num-keys 1000000  --columns 3   --replication-factor 2 --enable-cql3 --nodesfile=nodes

Averages from the middle 80% of values:
interval_op_rate          : 44372
interval_key_rate         : 44372
latency median            : 0.6
latency 95th percentile   : 2.6
latency 99.9th percentile : 72.9
Total operation time      : 00:00:19
END

1.2.19, cql3, 2 tokens per node, SMALL WRITES (3 columns)
./tools/bin/cassandra-stress --operation INSERT --num-keys 1000000  --columns 3   --replication-factor 2 --enable-cql3 --nodesfile=nodes

Averages from the middle 80% of values:
interval_op_rate          : 39787
interval_key_rate         : 39787
latency median            : 0.6
latency 95th percentile   : 2.6
latency 99.9th percentile : 12.3
Total operation time      : 00:00:21
END
2.0.10

ubuntu@cass01:~$ cassandra-stress --operation INSERT --num-keys 1000000  --columns 20   --replication-factor 2 --enable-cql3 --column-size 128 --nodesfile=nodes
Averages from the middle 80% of values:
interval_op_rate          : 11577
interval_key_rate         : 11577
latency median            : 1.3
latency 95th percentile   : 3.1
latency 99.9th percentile : 1066.0
Total operation time      : 00:01:33
END


ubuntu@cass01:~$ cassandra-stress --operation INSERT --num-keys 10000000  --columns 20   --replication-factor 2 --enable-cql3 --column-size 128 --nodesfile=nodes
Averages from the middle 80% of values:
interval_op_rate          : 7425
interval_key_rate         : 7425
latency median            : 1.3
latency 95th percentile   : 3.3
latency 99.9th percentile : 2404.4
Total operation time      : 00:22:14


2.0.13
ubuntu@cass01:~$ cassandra-stress --operation INSERT --num-keys 1000000  --columns 20   --replication-factor 2 --enable-cql3 --column-size 128 --nodesfile=nodes
Averages from the middle 80% of values:
interval_op_rate          : 11035
interval_key_rate         : 11035
latency median            : 1.4
latency 95th percentile   : 3.8
latency 99.9th percentile : 1412.3
Total operation time      : 00:01:30
END

2.0.13, 2 tokens per node, SMALL WRITES (3 columns): 
cassandra-stress --operation INSERT --num-keys 1000000  --columns 3   --replication-factor 2 --enable-cql3 --nodesfile=nodes

Averages from the middle 80% of values:
interval_op_rate          : 44655
interval_key_rate         : 44655
latency median            : 0.6
latency 95th percentile   : 2.0
latency 99.9th percentile : 25.7
Total operation time      : 00:00:20
END

2.0.13, 2 tokens per node, SMALL WRITES (3 columns), java 8
cassandra-stress --operation INSERT --num-keys 1000000  --columns 3   --replication-factor 2 --enable-cql3 --nodesfile=nodes

Averages from the middle 80% of values:
interval_op_rate          : 50050
interval_key_rate         : 50050
latency median            : 0.5
latency 95th percentile   : 1.3
latency 99.9th percentile : 15.7
Total operation time      : 00:00:18
END

2.0.13, 2 tokens per node, thrift, SMALL WRITES
cassandra-stress --operation INSERT --num-keys 1000000  --columns 3   --replication-factor 2  --nodesfile=nodes

Averages from the middle 80% of values:
interval_op_rate          : 55577
interval_key_rate         : 55577
latency median            : 0.5
latency 95th percentile   : 1.2
latency 99.9th percentile : 173.2
Total operation time      : 00:00:18
END



2.0.13, 2 tokens per node, lots of small columns, thrift:

cassandra-stress --operation INSERT --num-keys 1000000  --columns 66 --column-size=10   --replication-factor 2  --nodesfile=nodes
Averages from the middle 80% of values:
interval_op_rate          : 10720
interval_key_rate         : 10720
latency median            : 1.2
latency 95th percentile   : 3.8
latency 99.9th percentile : 294.7
Total operation time      : 00:01:39
END

2.0.13, 2 tokens per node, 20 columns, 10 bytes per column, thrift:
cassandra-stress --operation INSERT --num-keys 1000000  --columns 20 --column-size=10   --replication-factor 2  --nodesfile=nodes

Averages from the middle 80% of values:
interval_op_rate          : 28667
interval_key_rate         : 28667
latency median            : 0.7
latency 95th percentile   : 2.0
latency 99.9th percentile : 220.5
Total operation time      : 00:00:38
END

2.0.13, 2 tokens per node, 2  large columns, thrift:
cassandra-stress --operation INSERT --num-keys 1000000  --columns 2 --column-size=1024   --replication-factor 2  --nodesfile=nodes

Averages from the middle 80% of values:
interval_op_rate          : 23489
interval_key_rate         : 23489
latency median            : 0.6
latency 95th percentile   : 1.2
latency 99.9th percentile : 1596.8
Total operation time      : 00:00:42
END

2.0.13, 2 tokens per node, 2 large columns, cql3:
cassandra-stress --operation INSERT --num-keys 1000000  --columns 2 --column-size=1024   --replication-factor 2  --nodesfile=nodes --enable-cql3

Averages from the middle 80% of values:
interval_op_rate          : 24540
interval_key_rate         : 24540
latency median            : 0.9
latency 95th percentile   : 1.7
latency 99.9th percentile : 81.1
Total operation time      : 00:00:45
END

2.0.13, 2 large columns, far more keys: 
cassandra-stress --operation INSERT --num-keys 10000000  --columns 2 --column-size=1024   --replication-factor 2  --nodesfile=nodes --enable-cql3 

Averages from the middle 80% of values:
interval_op_rate          : 16334
interval_key_rate         : 16334
latency median            : 1.1
latency 95th percentile   : 2.2
latency 99.9th percentile : 619.8
Total operation time      : 00:10:27
END

2.0.13, 66 small columns, far more keys:
cassandra-stress --operation INSERT --num-keys 10000000  --columns 66 --column-size=10   --replication-factor 2  --nodesfile=nodes --enable-cql3 
Averages from the middle 80% of values:
interval_op_rate          : 11929
interval_key_rate         : 11929
latency median            : 1.5
latency 95th percentile   : 4.7
latency 99.9th percentile : 628.6
Total operation time      : 00:15:00
END

Cassandra 2.0.12.274 | DSE 4.6.2
ubuntu@cass01:~$ cassandra-stress --operation INSERT --num-keys 1000000  --columns 20   --replication-factor 2 --enable-cql3 --column-size 128 --nodesfile=nodes


Averages from the middle 80% of values:
interval_op_rate          : 11558
interval_key_rate         : 11558
latency median            : 1.7
latency 95th percentile   : 3.7
latency 99.9th percentile : 617.1
Total operation time      : 00:01:31
END

cassandra 2.1.3, 2 tokesn per node, small writes, thrift: 
cassandra-stress --operation INSERT --num-keys 1000000  --columns 3   --replication-factor 2  --nodesfile=nodes
Averages from the middle 80% of values:
interval_op_rate          : 72698
interval_key_rate         : 72698
latency median            : 0.4
latency 95th percentile   : 1.2
latency 99.9th percentile : 11.4
Total operation time      : 00:00:13
END

cassandra 2.1.3, 2 tokens per node, 20 columns, thrift:
cassandra-stress --operation INSERT --num-keys 10000000  --columns 20 --column-size 128   --replication-factor 2  --nodesfile=nodes

Averages from the middle 80% of values:
interval_op_rate          : 19330
interval_key_rate         : 19330
latency median            : 0.8
latency 95th percentile   : 2.1
latency 99.9th percentile : 1583.3
Total operation time      : 00:08:58
END

cassandra 2.1.3, 2 tokens per node, 20 columns, cql3:
cassandra-stress --operation INSERT --num-keys 10000000  --columns 20 --column-size 128   --replication-factor 2  --nodesfile=nodes --enable-cql3
Averages from the middle 80% of values:
interval_op_rate          : 25667
interval_key_rate         : 25667
latency median            : 1.4
latency 95th percentile   : 2.7
latency 99.9th percentile : 92.5
Total operation time      : 00:07:06
END

3.0.0-SNAPSHOT (from roughly Mar 28 2015), 2 tokens per node, 20 columns, cql3:
cassandra-stress --operation INSERT --num-keys 10000000  --columns 20 --column-size 128   --replication-factor 2  --nodesfile=nodes --enable-cql3 

Averages from the middle 80% of values:
interval_op_rate          : 24392
interval_key_rate         : 24392
latency median            : 1.2
latency 95th percentile   : 2.8
latency 99.9th percentile : 118.5
Total operation time      : 00:07:14
END

3.0.0-SNAPSHOT with 8099 engine:
cassandra-stress --operation INSERT --num-keys 10000000  --columns 20 --column-size 128   --replication-factor 2  --nodesfile=nodes --enable-cql3

Averages from the middle 80% of values:
interval_op_rate          : 24171
interval_key_rate         : 24171
latency median            : 1.1
latency 95th percentile   : 2.9
latency 99.9th percentile : 99.1
Total operation time      : 00:07:14
END

3.0.0-SNAPSHOT with 8099, small writes:
cassandra-stress --operation INSERT --num-keys 10000000  --columns 3  --replication-factor 2  --nodesfile=nodes --enable-cql3
Averages from the middle 80% of values:
interval_op_rate          : 60348
interval_key_rate         : 60348
latency median            : 0.4
latency 95th percentile   : 2.0
latency 99.9th percentile : 173.5
Total operation time      : 00:03:05