Skip to content

Instantly share code, notes, and snippets.

@mdcallag
Created July 10, 2023 18:44
Show Gist options
  • Save mdcallag/9bc88f6f571016b0e608e36ac8830f42 to your computer and use it in GitHub Desktop.
Save mdcallag/9bc88f6f571016b0e608e36ac8830f42 to your computer and use it in GitHub Desktop.
Columns are:
1 - job number
2 - compaction time (wall clock) in seconds
3 - compaction output size in MB
--- for 4GB block cache, largest 10 intra-L0 by compaction time
# grep "output_level\": 0" 480m.fbmy8028_ef5b9b101_jun23.cy9c_4g_u/l.i0/LOG | grep compaction_finished | awk '{ print $11, $15, $23 }' | tr ',' ' ' | awk '{ printf "%s\t%.1f\t%.1f\n", $1, $2 / 1000000.0, $3/(1024*1024) }' | sort -nk 2,2 | tail -10
51894 35.8 1381.3
26955 35.9 1387.2
55562 36.1 1386.8
84256 36.3 1386.9
86715 36.4 1384.8
13826 36.6 1383.9
93331 36.6 1385.1
47761 36.8 1388.5
77601 40.1 1467.9
70429 43.7 1171.6
--- for 4GB block cache, largest 10 intra-L0 by compaction output size
# grep "output_level\": 0" 480m.fbmy8028_ef5b9b101_jun23.cy9c_4g_u/l.i0/LOG | grep compaction_finished | awk '{ print $11, $15, $23 }' | tr ',' ' ' | awk '{ printf "%s\t%.1f\t%.1f\n", $1, $2 / 1000000.0, $3/(1024*1024) }' | sort -nk 3,3 | tail -10
86715 36.4 1384.8
98428 34.4 1384.9
93331 36.6 1385.1
23487 35.8 1385.8
55562 36.1 1386.8
84256 36.3 1386.9
26955 35.9 1387.2
59183 34.5 1387.8
47761 36.8 1388.5
77601 40.1 1467.9
--- for 8GB block cache, largest 10 intra-L0 by compaction time
# grep "output_level\": 0" 480m.fbmy8028_ef5b9b101_jun23.cy9c_8g_u/l.i0/LOG | grep compaction_finished | awk '{ print $11, $15, $23 }' | tr ',' ' ' | awk '{ printf "%s\t%.1f\t%.1f\n", $1, $2 / 1000000.0, $3/(1024*1024) }' | sort -nk 2,2 | tail -10
63604 36.6 1385.8
24629 36.7 1387.7
10660 36.8 1385.8
91429 37.5 1402.6
84958 38.2 1468.3
90390 38.4 1462.0
25089 39.4 1470.6
62104 39.9 1468.0
87603 40.3 1484.8
75143 40.4 1468.7
--- for 8GB block cache, largest 10 intra-L0 by compaction output size
# grep "output_level\": 0" 480m.fbmy8028_ef5b9b101_jun23.cy9c_8g_u/l.i0/LOG | grep compaction_finished | awk '{ print $11, $15, $23 }' | tr ',' ' ' | awk '{ printf "%s\t%.1f\t%.1f\n", $1, $2 / 1000000.0, $3/(1024*1024) }' | sort -nk 3,3 | tail -10
24629 36.7 1387.7
28119 34.5 1387.7
26494 35.7 1387.9
91429 37.5 1402.6
90390 38.4 1462.0
62104 39.9 1468.0
84958 38.2 1468.3
75143 40.4 1468.7
25089 39.4 1470.6
87603 40.3 1484.8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment