Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
# Compression changes from L1 (no compression) to L2 (lz4). So trivial move is not done from
# L1 to L2. But it starts again for L2 to L3 because both use lz4. However this is more complex
# because the tree starts with data in L0, then grows and uses L0,L1 then grows and uses L0,L1,L2.
# Describing that with physical names for the levels it starts with L0, then grows into L0,L6 and
# then grows into L0,L5,L6 and finally reaches L0,L4,L5,L6.
# Note that I used --min_level_compress=2. So when there is data in L0,L5,L6 then L6 (physical)
# is L2 (logical) and so L6 is compressed while L5 & L0 are not. At that point compaction from
# L0 to L5 can use trivial move (both levels have same compression type) but compacton from L5 to L6
# cannot (different compression types).
# This is why there is a mix of trivial move and proper compaction for L6.
# Above I use logical names for the levels while compaction IO stats below uses the physical
# names. This is a source of confusion because of the way that dynamic leveled compaction adds
# levels as the LSM tree grows. It starts with L0, then adds Lmax, then adds Lmax-1, ... and
# below there is data in L0, L4, L5, L6 where L6=Lmax.
M1=$(( 1024 * 1024 )); ./db_bench --benchmarks=fillseq,stats \
--db_write_buffer_size=$(( $M1 * 8 )) \
--target_file_size_base=$(( $M1 * 8 )) \
--max_bytes_for_level_base=$(( $M1 * 32 )) \
--level0_file_num_compaction_trigger=4 \
--level0_slowdown_writes_trigger=20 \
--level0_stop_writes_trigger=30 \
--level_compaction_dynamic_level_bytes=true \
--num=$(( $M1 * 16 )) \
--min_level_to_compress=2 \
--compression_type=lz4
Set seed to 1648995247827258 because --seed was 0
Initializing RocksDB Options from the specified file
Initializing RocksDB Options from command-line flags
RocksDB: version 7.1
Date: Sun Apr 3 07:14:07 2022
CPU: 32 * Intel Xeon Processor (Skylake)
CPUCache: 16384 KB
Keys: 16 bytes each (+ 0 bytes user-defined timestamp)
Values: 100 bytes each (50 bytes after compression)
Entries: 16777216
Prefix: 0 bytes
Keys per prefix: 0
RawSize: 1856.0 MB (estimated)
FileSize: 1056.0 MB (estimated)
Write rate: 0 bytes/second
Read rate: 0 ops/second
Compression: LZ4
Compression sampling rate: 0
Memtablerep: SkipListFactory
Perf Level: 1
------------------------------------------------
Initializing RocksDB Options from the specified file
Initializing RocksDB Options from command-line flags
DB path: [/tmp/rocksdbtest-2260/dbbench]
fillseq : 4.018 micros/op 248882 ops/sec; 27.5 MB/s
** Compaction Stats [default] **
Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
L0 3/0 15.13 MB 0.8 0.0 0.0 0.0 1.9 1.9 0.0 1.0 0.0 152.0 12.80 6.59 386 0.033 0 0 0.0 0.0
L4 6/0 30.25 MB 0.9 0.0 0.0 0.0 0.0 0.0 1.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0
L5 35/0 92.30 MB 1.0 1.3 1.3 0.0 0.7 0.7 0.6 0.5 139.4 72.9 9.26 8.63 256 0.036 11M 0 0.0 0.0
L6 342/0 933.15 MB 0.0 0.5 0.5 0.0 0.3 0.3 0.6 0.5 137.8 72.1 3.95 3.66 108 0.037 4684K 0 0.0 0.0
Sum 386/0 1.05 GB 0.0 1.8 1.8 0.0 2.8 2.8 2.5 1.5 70.6 111.7 26.01 18.89 750 0.035 15M 0 0.0 0.0
Int 0/0 0.00 KB 0.0 1.8 1.8 0.0 2.8 2.8 2.4 1.5 72.2 111.1 25.42 18.60 734 0.035 15M 0 0.0 0.0
** Compaction Stats [default] **
Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB)
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Low 0/0 0.00 KB 0.0 1.8 1.8 0.0 0.9 0.9 0.0 0.0 139.0 72.7 13.21 12.30 364 0.036 15M 0 0.0 0.0
High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.9 1.9 0.0 0.0 0.0 152.0 12.80 6.59 386 0.033 0 0 0.0 0.0
Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0
Uptime(secs): 67.4 total, 64.4 interval
Flush(GB): cumulative 1.901, interval 1.822
AddFile(GB): cumulative 0.000, interval 0.000
AddFile(Total Files): cumulative 0, interval 0
AddFile(L0 Files): cumulative 0, interval 0
AddFile(Keys): cumulative 0, interval 0
Cumulative compaction: 2.84 GB write, 43.11 MB/s write, 1.79 GB read, 27.22 MB/s read, 26.0 seconds
Interval compaction: 2.76 GB write, 43.87 MB/s write, 1.79 GB read, 28.50 MB/s read, 25.4 seconds
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Block cache LRUCache@0x7f8fdba29290#2356776 capacity: 8.00 MB collections: 1 last_copies: 1 last_secs: 0.000101 secs_since: 67
Block cache entry stats(count,size,portion): WriteBuffer(1,256.00 KB,3.125%) Misc(1,0.00 KB,0%)
** File Read Latency Histogram By Level [default] **
** DB Stats **
Uptime(secs): 67.4 total, 64.4 interval
Cumulative writes: 16M writes, 16M keys, 16M commit groups, 1.0 writes per commit group, ingest: 2.05 GB, 31.09 MB/s
Cumulative WAL: 16M writes, 0 syncs, 16777216.00 writes per sync, written: 2.05 GB, 31.09 MB/s
Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent
Interval writes: 16M writes, 16M keys, 16M commit groups, 1.0 writes per commit group, ingest: 2003.85 MB, 31.11 MB/s
Interval WAL: 16M writes, 0 syncs, 16039640.00 writes per sync, written: 1.96 GB, 31.11 MB/s
Interval stall: 00:00:0.000 H:M:S, 0.0 percent
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment