Skip to content

Instantly share code, notes, and snippets.

@anhtranbk
Last active September 29, 2017 04:43
Show Gist options
  • Save anhtranbk/9ec3df3a238a62ad1aaf6a159498d914 to your computer and use it in GitHub Desktop.
Save anhtranbk/9ec3df3a238a62ad1aaf6a159498d914 to your computer and use it in GitHub Desktop.
My production configuration for Cassandra, Kafka, Presto
cluster_name: 'DataCollection Cluster'
listen_address: 192.168.23.64
rpc_address: 192.168.23.64
native_transport_port: 9042
storage_port: 7000
seed_provider:
- class_name: org.apache.cassandra.locator.SimpleSeedProvider
parameters:
# seeds is actually a comma-delimited list of addresses.
# Ex: "<ip1>,<ip2>,<ip3>"
- seeds: "192.168.23.64,192.168.23.65,192.168.23.66"
concurrent_reads: 256
concurrent_writes: 400
concurrent_counter_writes: 32
# Use native os cache to store memtable
memtable_allocation_type: offheap_objects
# The number of memtable flush writer threads. These threads are blocked
# by disk I/O, and each one holds a memtable in memory while blocked.
# If your data directories are backed by SSDs, increase this setting to the number of cores.
memtable_flush_writers: 8
# Optimize strategy for disk: spinning or ssd
disk_optimization_strategy: spinning
# Total memory to use for SSTable-reading buffers.
file_cache_size_in_mb: 8192
# Maximum size of the key cache in memory
key_cache_size_in_mb: 8192
# one or more directories where data files are located.
data_file_directories: /data/cassandra/data
# the directory where commitlog files are located.
commitlog_directory: /storage/cassandra/cassandra-data/commitlog
# : the directory where saved caches are located.
saved_caches_directory: /storage/cassandra/cassandra-data/saved_caches
# the directory where hints are located.
hints_directory: /storage/cassandra/cassandra-data/hints
zookeeper.connect=192.168.23.64:2181,192.168.23.65:2181
broker.id=65
listeners=PLAINTEXT://192.168.23.65:9092
auto.create.topics.enable=true
min.insync.replicas=2
delete.topic.enable=true
# Log configuration
num.partitions=3
default.replication.factor=2
log.dirs=/storage/kafka/kafka-data,/data/kafka-data
# The minimum age of a log file to be eligible for deletion due to age
log.retention.hours=720
# Log retention window in minutes for offsets topic
offsets.retention.minutes=14400
# Increase max message in bytes
replica.fetch.max.bytes = 10485760
message.max.bytes = 10485760
max.message.bytes = 10485760
# The number of threads handling network requests
num.network.threads=3
# The number of threads doing disk I/O
num.io.threads=8
# The send buffer (SO_SNDBUF) used by the socket server
socket.send.buffer.bytes=1024000
# The receive buffer (SO_RCVBUF) used by the socket server
socket.receive.buffer.bytes=1024000
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment