Skip to content

Instantly share code, notes, and snippets.

@mdcallag
Created April 3, 2022 14:31
Show Gist options
  • Save mdcallag/a1f1db9d5d6b03607034cd8e97e3f1c6 to your computer and use it in GitHub Desktop.
Save mdcallag/a1f1db9d5d6b03607034cd8e97e3f1c6 to your computer and use it in GitHub Desktop.
# When some levels are compresseion (db_bench --min_level_to_compress=2 --compression_type=lz4)
# then compaction IO stats are easier to explain with dynamic leveled disabled by
# --level_compaction_dynamic_level_bytes=false. First, the level names displayed in the stats
# are L0,L1,L2 and L3 vs L0,L4,L5 and L6 when dynamic leveled was enabled. Second, for each
# level either Write(GB)=0 or Moved(GB)=0. With dynamic leveled they could both be non-zero.
M1=$(( 1024 * 1024 )); ./db_bench --benchmarks=fillseq,stats \
--db_write_buffer_size=$(( $M1 * 8 )) \
--target_file_size_base=$(( $M1 * 8 )) \
--max_bytes_for_level_base=$(( $M1 * 32 )) \
--level0_file_num_compaction_trigger=4 \
--level0_slowdown_writes_trigger=20 \
--level0_stop_writes_trigger=30 \
--level_compaction_dynamic_level_bytes=false \
--num=$(( $M1 * 16 )) \
--min_level_to_compress=2 \
--compression_type=lz4
Set seed to 1648995927893027 because --seed was 0
Initializing RocksDB Options from the specified file
Initializing RocksDB Options from command-line flags
RocksDB: version 7.1
Date: Sun Apr 3 07:25:28 2022
CPU: 32 * Intel Xeon Processor (Skylake)
CPUCache: 16384 KB
Keys: 16 bytes each (+ 0 bytes user-defined timestamp)
Values: 100 bytes each (50 bytes after compression)
Entries: 16777216
Prefix: 0 bytes
Keys per prefix: 0
RawSize: 1856.0 MB (estimated)
FileSize: 1056.0 MB (estimated)
Write rate: 0 bytes/second
Read rate: 0 ops/second
Compression: LZ4
Compression sampling rate: 0
Memtablerep: SkipListFactory
Perf Level: 1
------------------------------------------------
Initializing RocksDB Options from the specified file
Initializing RocksDB Options from command-line flags
DB path: [/tmp/rocksdbtest-2260/dbbench]
fillseq : 4.125 micros/op 242405 ops/sec; 26.8 MB/s
** Compaction Stats [default] **
Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
L0 3/0 15.13 MB 0.8 0.0 0.0 0.0 1.9 1.9 0.0 1.0 0.0 141.4 13.77 6.64 386 0.036 0 0 0.0 0.0
L1 6/0 30.25 MB 0.9 0.0 0.0 0.0 0.0 0.0 1.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0
L2 121/0 319.09 MB 1.0 1.9 1.9 0.0 1.0 1.0 0.0 0.5 138.1 72.2 13.77 12.85 377 0.037 16M 0 0.0 0.0
L3 256/0 675.08 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0
Sum 386/0 1.02 GB 0.0 1.9 1.9 0.0 2.9 2.9 2.5 1.5 69.0 106.8 27.53 19.49 763 0.036 16M 0 0.0 0.0
Int 0/0 0.00 KB 0.0 1.8 1.8 0.0 2.8 2.8 2.5 1.5 69.7 106.2 26.76 18.97 740 0.036 16M 0 0.0 0.0
** Compaction Stats [default] **
Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB)
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Low 0/0 0.00 KB 0.0 1.9 1.9 0.0 1.0 1.0 0.0 0.0 138.1 72.2 13.77 12.85 377 0.037 16M 0 0.0 0.0
High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.9 1.9 0.0 0.0 0.0 141.4 13.77 6.64 386 0.036 0 0 0.0 0.0
Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0
Uptime(secs): 69.2 total, 66.2 interval
Flush(GB): cumulative 1.901, interval 1.822
AddFile(GB): cumulative 0.000, interval 0.000
AddFile(Total Files): cumulative 0, interval 0
AddFile(L0 Files): cumulative 0, interval 0
AddFile(Keys): cumulative 0, interval 0
Cumulative compaction: 2.87 GB write, 42.48 MB/s write, 1.86 GB read, 27.46 MB/s read, 27.5 seconds
Interval compaction: 2.77 GB write, 42.91 MB/s write, 1.82 GB read, 28.18 MB/s read, 26.8 seconds
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Block cache LRUCache@0x7f610e629290#2403666 capacity: 8.00 MB collections: 1 last_copies: 1 last_secs: 0.000113 secs_since: 69
Block cache entry stats(count,size,portion): WriteBuffer(1,256.00 KB,3.125%) Misc(1,0.00 KB,0%)
** File Read Latency Histogram By Level [default] **
** DB Stats **
Uptime(secs): 69.2 total, 66.2 interval
Cumulative writes: 16M writes, 16M keys, 16M commit groups, 1.0 writes per commit group, ingest: 2.05 GB, 30.28 MB/s
Cumulative WAL: 16M writes, 0 syncs, 16777216.00 writes per sync, written: 2.05 GB, 30.28 MB/s
Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent
Interval writes: 16M writes, 16M keys, 16M
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment