Skip to content

Instantly share code, notes, and snippets.

@mdcallag
Created April 3, 2022 14:31
Show Gist options
  • Save mdcallag/02937c7f2fc3b6368b3d99b18a244fb8 to your computer and use it in GitHub Desktop.
Save mdcallag/02937c7f2fc3b6368b3d99b18a244fb8 to your computer and use it in GitHub Desktop.
Basic fillseq with small write buffer and small L1 to get more levels with less data. Dynamic leveled compaction enabled.
# This is a best case for trivial move
# * writes into L0 (memtable flush) can't use trivial move because you have to write once
# * compaction for all levels (Ln->Ln+1 where n >= 0) is trivial move
# In this case dynamic leveled compaction is enabled
# Above I use logical names for the levels while compaction IO stats below uses the physical
# names. This is a source of confusion because of the way that dynamic leveled compaction adds
# levels as the LSM tree grows. It starts with L0, then adds Lmax, then adds Lmax-1, ... and
# below there is data in L0, L4, L5, L6 where L6=Lmax.
M1=$(( 1024 * 1024 )); ./db_bench --benchmarks=fillseq,stats \
--db_write_buffer_size=$(( $M1 * 8 )) \
--target_file_size_base=$(( $M1 * 8 )) \
--max_bytes_for_level_base=$(( $M1 * 32 )) \
--level0_file_num_compaction_trigger=4 \
--level0_slowdown_writes_trigger=20 \
--level0_stop_writes_trigger=30 \
--level_compaction_dynamic_level_bytes=true \
--num=$(( $M1 * 16 ))
Set seed to 1648994460964062 because --seed was 0
Initializing RocksDB Options from the specified file
Initializing RocksDB Options from command-line flags
RocksDB: version 7.1
Date: Sun Apr 3 07:01:01 2022
CPU: 32 * Intel Xeon Processor (Skylake)
CPUCache: 16384 KB
Keys: 16 bytes each (+ 0 bytes user-defined timestamp)
Values: 100 bytes each (50 bytes after compression)
Entries: 16777216
Prefix: 0 bytes
Keys per prefix: 0
RawSize: 1856.0 MB (estimated)
FileSize: 1056.0 MB (estimated)
Write rate: 0 bytes/second
Read rate: 0 ops/second
Compression: Snappy
Compression sampling rate: 0
Memtablerep: SkipListFactory
Perf Level: 1
------------------------------------------------
Initializing RocksDB Options from the specified file
Initializing RocksDB Options from command-line flags
DB path: [/tmp/rocksdbtest-2260/dbbench]
fillseq : 4.105 micros/op 243602 ops/sec; 26.9 MB/s
** Compaction Stats [default] **
Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
L0 3/0 8.37 MB 0.8 0.0 0.0 0.0 1.1 1.1 0.0 1.0 0.0 66.5 16.23 9.68 387 0.042 0 0 0.0 0.0
L4 11/0 30.72 MB 1.0 0.0 0.0 0.0 0.0 0.0 0.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0
L5 33/0 92.15 MB 1.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0
L6 340/0 948.06 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0
Sum 387/0 1.05 GB 0.0 0.0 0.0 0.0 1.1 1.1 2.6 1.0 0.0 66.5 16.23 9.68 387 0.042 0 0 0.0 0.0
Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.0 1.0 2.6 1.0 0.0 66.9 15.46 9.30 371 0.042 0 0 0.0 0.0
** Compaction Stats [default] **
Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB)
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.1 1.1 0.0 0.0 0.0 66.5 16.23 9.68 387 0.042 0 0 0.0 0.0
Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0
Uptime(secs): 68.9 total, 65.9 interval
Flush(GB): cumulative 1.054, interval 1.010
AddFile(GB): cumulative 0.000, interval 0.000
AddFile(Total Files): cumulative 0, interval 0
AddFile(L0 Files): cumulative 0, interval 0
AddFile(Keys): cumulative 0, interval 0
Cumulative compaction: 1.05 GB write, 15.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 16.2 seconds
Interval compaction: 1.01 GB write, 15.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 15.5 seconds
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 1 memtable_compaction, 0 memtable_slowdown, interval 1 total count
Block cache LRUCache@0x7ff12aa29290#2307414 capacity: 8.00 MB collections: 1 last_copies: 1 last_secs: 0.000101 secs_since: 68
Block cache entry stats(count,size,portion): WriteBuffer(1,256.00 KB,3.125%) Misc(1,0.00 KB,0%)
** File Read Latency Histogram By Level [default] **
** DB Stats **
Uptime(secs): 68.9 total, 65.9 interval
Cumulative writes: 16M writes, 16M keys, 16M commit groups, 1.0 writes per commit group, ingest: 2.05 GB, 30.43 MB/s
Cumulative WAL: 16M writes, 0 syncs, 16777216.00 writes per sync, written: 2.05 GB, 30.43 MB/s
Cumulative stall: 00:00:0.014 H:M:S, 0.0 percent
Interval writes: 16M writes, 16M keys, 16M commit groups, 1.0 writes per commit group, ingest: 2003.63 MB, 30.42 MB/s
Interval WAL: 16M writes, 0 syncs, 16037814.00 writes per sync, written: 1.96 GB, 30.42 MB/s
Interval stall: 00:00:0.014 H:M:S, 0.0 percent
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment