Created
April 3, 2022 14:31
-
-
Save mdcallag/a32beec6b69f20ef5e3932b1a64e4f80 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Compression changes from L1 (no compression) to L2 (lz4). So trivial move is not done from | |
# L1 to L2. But it starts again for L2 to L3 because both use lz4. However this is more complex | |
# because the tree starts with data in L0, then grows and uses L0,L1 then grows and uses L0,L1,L2. | |
# Describing that with physical names for the levels it starts with L0, then grows into L0,L6 and | |
# then grows into L0,L5,L6 and finally reaches L0,L4,L5,L6. | |
# Note that I used --min_level_compress=2. So when there is data in L0,L5,L6 then L6 (physical) | |
# is L2 (logical) and so L6 is compressed while L5 & L0 are not. At that point compaction from | |
# L0 to L5 can use trivial move (both levels have same compression type) but compacton from L5 to L6 | |
# cannot (different compression types). | |
# This is why there is a mix of trivial move and proper compaction for L6. | |
# Above I use logical names for the levels while compaction IO stats below uses the physical | |
# names. This is a source of confusion because of the way that dynamic leveled compaction adds | |
# levels as the LSM tree grows. It starts with L0, then adds Lmax, then adds Lmax-1, ... and | |
# below there is data in L0, L4, L5, L6 where L6=Lmax. | |
M1=$(( 1024 * 1024 )); ./db_bench --benchmarks=fillseq,stats \ | |
--db_write_buffer_size=$(( $M1 * 8 )) \ | |
--target_file_size_base=$(( $M1 * 8 )) \ | |
--max_bytes_for_level_base=$(( $M1 * 32 )) \ | |
--level0_file_num_compaction_trigger=4 \ | |
--level0_slowdown_writes_trigger=20 \ | |
--level0_stop_writes_trigger=30 \ | |
--level_compaction_dynamic_level_bytes=true \ | |
--num=$(( $M1 * 16 )) \ | |
--min_level_to_compress=2 \ | |
--compression_type=lz4 | |
Set seed to 1648995247827258 because --seed was 0 | |
Initializing RocksDB Options from the specified file | |
Initializing RocksDB Options from command-line flags | |
RocksDB: version 7.1 | |
Date: Sun Apr 3 07:14:07 2022 | |
CPU: 32 * Intel Xeon Processor (Skylake) | |
CPUCache: 16384 KB | |
Keys: 16 bytes each (+ 0 bytes user-defined timestamp) | |
Values: 100 bytes each (50 bytes after compression) | |
Entries: 16777216 | |
Prefix: 0 bytes | |
Keys per prefix: 0 | |
RawSize: 1856.0 MB (estimated) | |
FileSize: 1056.0 MB (estimated) | |
Write rate: 0 bytes/second | |
Read rate: 0 ops/second | |
Compression: LZ4 | |
Compression sampling rate: 0 | |
Memtablerep: SkipListFactory | |
Perf Level: 1 | |
------------------------------------------------ | |
Initializing RocksDB Options from the specified file | |
Initializing RocksDB Options from command-line flags | |
DB path: [/tmp/rocksdbtest-2260/dbbench] | |
fillseq : 4.018 micros/op 248882 ops/sec; 27.5 MB/s | |
** Compaction Stats [default] ** | |
Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) | |
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | |
L0 3/0 15.13 MB 0.8 0.0 0.0 0.0 1.9 1.9 0.0 1.0 0.0 152.0 12.80 6.59 386 0.033 0 0 0.0 0.0 | |
L4 6/0 30.25 MB 0.9 0.0 0.0 0.0 0.0 0.0 1.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 | |
L5 35/0 92.30 MB 1.0 1.3 1.3 0.0 0.7 0.7 0.6 0.5 139.4 72.9 9.26 8.63 256 0.036 11M 0 0.0 0.0 | |
L6 342/0 933.15 MB 0.0 0.5 0.5 0.0 0.3 0.3 0.6 0.5 137.8 72.1 3.95 3.66 108 0.037 4684K 0 0.0 0.0 | |
Sum 386/0 1.05 GB 0.0 1.8 1.8 0.0 2.8 2.8 2.5 1.5 70.6 111.7 26.01 18.89 750 0.035 15M 0 0.0 0.0 | |
Int 0/0 0.00 KB 0.0 1.8 1.8 0.0 2.8 2.8 2.4 1.5 72.2 111.1 25.42 18.60 734 0.035 15M 0 0.0 0.0 | |
** Compaction Stats [default] ** | |
Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) | |
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
Low 0/0 0.00 KB 0.0 1.8 1.8 0.0 0.9 0.9 0.0 0.0 139.0 72.7 13.21 12.30 364 0.036 15M 0 0.0 0.0 | |
High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.9 1.9 0.0 0.0 0.0 152.0 12.80 6.59 386 0.033 0 0 0.0 0.0 | |
Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 | |
Uptime(secs): 67.4 total, 64.4 interval | |
Flush(GB): cumulative 1.901, interval 1.822 | |
AddFile(GB): cumulative 0.000, interval 0.000 | |
AddFile(Total Files): cumulative 0, interval 0 | |
AddFile(L0 Files): cumulative 0, interval 0 | |
AddFile(Keys): cumulative 0, interval 0 | |
Cumulative compaction: 2.84 GB write, 43.11 MB/s write, 1.79 GB read, 27.22 MB/s read, 26.0 seconds | |
Interval compaction: 2.76 GB write, 43.87 MB/s write, 1.79 GB read, 28.50 MB/s read, 25.4 seconds | |
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count | |
Block cache LRUCache@0x7f8fdba29290#2356776 capacity: 8.00 MB collections: 1 last_copies: 1 last_secs: 0.000101 secs_since: 67 | |
Block cache entry stats(count,size,portion): WriteBuffer(1,256.00 KB,3.125%) Misc(1,0.00 KB,0%) | |
** File Read Latency Histogram By Level [default] ** | |
** DB Stats ** | |
Uptime(secs): 67.4 total, 64.4 interval | |
Cumulative writes: 16M writes, 16M keys, 16M commit groups, 1.0 writes per commit group, ingest: 2.05 GB, 31.09 MB/s | |
Cumulative WAL: 16M writes, 0 syncs, 16777216.00 writes per sync, written: 2.05 GB, 31.09 MB/s | |
Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent | |
Interval writes: 16M writes, 16M keys, 16M commit groups, 1.0 writes per commit group, ingest: 2003.85 MB, 31.11 MB/s | |
Interval WAL: 16M writes, 0 syncs, 16039640.00 writes per sync, written: 1.96 GB, 31.11 MB/s | |
Interval stall: 00:00:0.000 H:M:S, 0.0 percent |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment