Last active
June 26, 2019 09:42
-
-
Save rayhassan/ebfa5d17e6e8ffeb36bc48124cfed858 to your computer and use it in GitHub Desktop.
Cassandra sizing notes
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
refrences: | |
https://dzone.com/articles/cassandra-design-best-practices | |
https://www.datastax.com/wp-content/uploads/2014/04/WP-DataStax-Enterprise-Best-Practices.pdf | |
https://docs.datastax.com/en/dse/5.1/dse-admin/datastax_enterprise/config/configRecommendedSettings.html | |
As regards tips/advice then bear in mind most of this will apply to the Guest OS running Cassandra | |
run noop scheduler | |
set OS limits | |
* soft nofile 32768 | |
* hard nofile 32768 | |
root soft nofile 32768 | |
root hard nofile 32768 | |
* soft memlock unlimited | |
* hard memlock unlimited | |
root soft memlock unlimited | |
root hard memlock unlimited | |
* soft as unlimited | |
* hard as unlimited | |
root soft as unlimited | |
root hard as unlimited | |
Set the nproc limits to 32768 in the /etc/security/limits.d/90-nproc.conf configuration file: | |
cassandra_user - nproc 32768 | |
Tarball installation /etc/security/limits.conf | |
Package installation /etc/security/limits.d/cassandra.conf | |
Configure the following settings for the <cassandra_user> in the configuration file: | |
<cassandra_user> - memlock unlimited | |
<cassandra_user> - nofile 1048576 | |
<cassandra_user> - nproc 32768 | |
<cassandra_user> - as unlimited | |
net.ipv4.tcp_keepalive_time=60 | |
net.ipv4.tcp_keepalive_probes=3 | |
net.ipv4.tcp_keepalive_intvl=10 | |
net.core.rmem_max=16777216 | |
net.core.wmem_max=16777216 | |
net.core.rmem_default=16777216 | |
net.core.wmem_default=16777216 | |
net.core.optmem_max=40960 | |
net.ipv4.tcp_rmem=4096 87380 16777216 | |
net.ipv4.tcp_wmem=4096 65536 16777216 | |
sysctl -w vm.max_map_count=1048575 To make this setting permanent, | |
vm.max_map_count = 1048575 Should be added to: /etc/sysctl.conf | |
o disable freq scaling / set frequency governor | |
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/power_management_guide/cpufreq_setup#enabling_a_cpufreq_governor | |
o disable numa and/or zone_reclaim | |
https://www.thegeekdiary.com/how-to-disable-numa-in-centos-rhel-67/ | |
To disable NUMA, add numa=off to the kernel line in grub.conf file | |
echo 0 > /proc/sys/vm/zone_reclaim_mode | |
So that the Java Virtual Machine (JVM) does not endlessly swap, all paging spaces should be | |
deactivated like so: sudo swapoff –all or as root swapoff –all This change can be made | |
permanent by removing swap file entries from /etc/fstab | |
Tune the JVM Heap size on each node of the cluster, depending on the amount of memory on that | |
specific node. Too big a Heap size can impair Cassandra’s efficiency. Recommended settings are: | |
RAM Heap Size | |
< 2GB ½ of RAM | |
2GB to 4GB 1GB | |
> 4GB ¼ of RAM, but not more than 8GB | |
The database automatically calculates the maximum heap size (MAX_HEAP_SIZE) based on this formula: | |
max(min(1/2 ram, 1024 megabytes), min(1/4 ram, 32765 megabytes)) <<<<< number taken from blog (1) ! | |
(1) http://java-performance.info/over-32g-heap-java | |
(2) http://blog.ragozin.info/2012/03/secret-hotspot-option-improving-gc.html | |
Writes are done as updates and then the original data gets tombstoned for later deletion after compaction. | |
Often, you will need double the amount of space in relation to the data to be deleted. They may have a handle | |
on that already. They will know often they overwrite/update data I hope. Otherwise check incremental backups perhaps? | |
one other thing to consider - which relates the cassandra rf3 - if they need n VMs to host the | |
entire working set in RAM then the usual caveat is they need 3n VMs for the replica set to provide | |
redundancy as per casssandra guidelines |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
DataStax Enterprise (DSE) 5.1 uses the garbage first collector (G1) by default. G1 is recommended for the following reasons:
Heap sizes from 16 GB to 64 GB.
G1 performs better than CMS (concurrent-mark-sweep) for larger heaps because it scans the regions of the heap containing the most garbage objects first, and compacts the heap on-the-go, while CMS stops the application when performing garbage collection.
The workload is variable, that is, the cluster is performing the different processes all the time.
For future proofing, as CMS will be deprecated in Java 9.
G1 is easier to configure.
G1 is self tuning.
You only need to set MAX_HEAP_SIZE.
However, G1 incurs some latency due to profiling.