Skip to content

Instantly share code, notes, and snippets.

💭
for advice ask https://www.percona.com/

Mark Callaghan mdcallag

💭
for advice ask https://www.percona.com/
Block or report user

Report or block mdcallag

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
View gist:ae8c23a64b038026f9634a123a221104
fpr = c(seq(0.00001, 0.00009, 0.00001), seq(0.0001, 0.0009, 0.0001), seq(0.001, 0.009, 0.001), seq(0.01, 0.09, 0.01))
bf <- function(e) { ceiling(log2(1/e) * 1.44) }
bf2 = sapply(fpr, bf)
cf <- function(e) { ceiling(log2(1/e) + 3) / 0.95 }
cf2 = sapply(fpr, cf)
bf2:
View gist:4005e27018655a818bece429b07cfaf4
cached test
server use discard (SSD TRIM)
measured from iostat
dyn=1; cpri=3; for j in 1 2 4 8 12 16 24 32 40 48 64 ; do echo j $j, pri $cpri, dyn $dyn; head -3 hw.j$j.pri$cpri.dyn$dyn | tail -1 ; done; done; done | grep -v pri | awk '{ printf "%.0f\t%.0f\t%.0f\t%.0f\n", $3, $5, $6, $7 }'
rMB/s wMB/s rGB wGB
0 196 0 1374
0 260 0 1376
0 624 0 1308
View gist:c6e6c886ccef6e59bbf5063e786c9a38
cached test
server use discard (SSD TRIM)
measured from iostat
for dyn in 0 1; do for cpri in 0 3; do for j in 1 2 4 8 12 16 24 32 40 48 64 ; do echo j $j, pri $cpri, dyn $dyn; head -3 hw.j$j.pri$cpri.dyn$dyn | tail -1 ; done; done; done | grep -v pri | awk '{ printf "%.0f\t%.0f\t%.0f\t%.0f\n", $3, $5, $6, $7 }'
rMB/s wMB/s rGB wGB
0 210 0 2075
0 258 0 2083
0 647 0 1773
View gist:91c9d7f4bf8f1e22c5eeb3dcfb34bcde
cached test
server does not use discard (no SSD TRIM)
measured from iostat
for dyn in 0 1; do for cpri in 0 3; do for j in 1 2 4 8 12 ; do echo j $j, pri $cpri, dyn $dyn; head -3 hw.j$j.pri$cpri.dyn$dyn | tail -1 ; done; done; done | grep -v pri | awk '{ printf "%.0f\t%.0f\t%.0f\t%.0f\n", $3, $5, $6, $7 }'
rMB/s wMB/s rGB wGB
0 98 0 955
0 118 0 1000
0 302 0 827
View gist:eb5b2a0354efa07efa0d50dd313eece1
IO-bound test
server does not use discard (no SSD TRIM)
measured from iostat
for dyn in 0 1; do for cpri in 0 3; do for j in 1 2 4 8 12 16 24 32 40 48 64 ; do echo j $j, pri $cpri, dyn $dyn; head -3 hw.j$j.pri$cpri.dyn$dyn | tail -1 ; done; done; done | grep -v pri | awk '{ printf "%.0f\t%.0f\t%.0f\t%.0f\n", $3, $5, $6, $7 }'
rMB/s wMB/s rGB wGB
58 130 1514 3389
62 141 1400 3188
157 385 1205 2950
View gist:de06277af1bc4e1648c984e15f597ce7
nohup bash cmu2.sh 250000000 3600 md2 1000 &
for dyn in 0 1; do for cpri in 0 3; do for j in 1 2 4 8 12 16 24 32 40 48 64 ; do echo j $j, pri $cpri, dyn $dyn; head -3 hw.j$j.pri$cpri.dyn$dyn | tail -1 ; done; done; done | grep -v pri | awk '{ printf "%d\t%.0f\t%.0f\t%.0f\t%.0f\t%.0f\n", $2, $3, $4, $5, $6, $7 }'
dyn=0, pri=0
r/s rMB/s w/s wMB/s rGB wGB
485 58 1053 130 1514 3389
515 62 1136 141 1400 3188
1292 157 3104 385 1205 2950
2848 343 5388 667 1483 2882
3898 470 6661 825 1285 2255
View gist:900ee63a2f480d69351f388b05f8e050
nohup bash cmu2.sh 250000000 3600 md2 400 &
for dyn in 0 1; do for cpri in 0 3; do for j in 1 2 4 8 12 16 24 32 40 48 64 ; do echo j $j, pri $cpri, dyn $dyn; head -3 hw.j$j.pri$cpri.dyn$dyn | tail -1 ; done; done; done | grep -v pri | awk '{ printf "%d\t%.0f\t%.0f\n", $4, $5, $7 }'
dyn=0, cpri=0
w/s wMB/s wGB
786 210 2075
949 258 2083
2371 647 1773
3995 1091 1716
4629 1264 1781
View gist:6da742bb8ad0c7c30101ce2dd969a005
nr=$1
maxrsecs=$2
dname=$3
vsz=$4
for j in 1 2 4 8 12 16 24 32 40 48 64; do
for cpri in 0 3; do
for dyn in 0 1 ; do
# reduce time to run read-only when there are more compaction threads
View gist:c12f0634cd882e3e06fee0d9231483d6
j=$1
nr=$2
cpri=$3
dyn=$4
secs=$5
dname=$6
vsz=$7
dbdir="./rdb"
View gist:56253c6809b1de3e02904ef8282e97bd
Throughput is better with larger values for max_background_jobs (j 8, j 12) because there are fewer stalls.
25.474 micros/op 39256 ops/sec; 15.7 MB/s j 1, pri 0, dyn 0
27.338 micros/op 36579 ops/sec; 14.7 MB/s j 2, pri 0, dyn 0
7.603 micros/op 131532 ops/sec; 52.7 MB/s j 4, pri 0, dyn 0
4.352 micros/op 229792 ops/sec; 92.0 MB/s j 8, pri 0, dyn 0
3.947 micros/op 253381 ops/sec; 101.5 MB/s j 12, pri 0, dyn 0
-
26.050 micros/op 38388 ops/sec; 15.4 MB/s j 1, pri 0, dyn 1
26.322 micros/op 37991 ops/sec; 15.2 MB/s j 2, pri 0, dyn 1
You can’t perform that action at this time.