Skip to content

Instantly share code, notes, and snippets.

@84adam

84adam/progress-compaction.sh

Last active November 16, 2022 17:20
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save 84adam/5b0f5bff958f0c4df73a045b038bb5b3 to your computer and use it in GitHub Desktop.
Save 84adam/5b0f5bff958f0c4df73a045b038bb5b3 to your computer and use it in GitHub Desktop.
Download ZIP
Monitor Esplora/Electrs compaction process
Raw
progress-compaction.sh
#!/bin/bash
MOUNTPOINT="/mnt/ext"
DISK="/dev/sda1"
echo ""
echo "-------------------------------------------------------------------------------- DISK SPACE USED/REMAINING"
df -m | head -n 1 ; df -m | grep "$DISK"
df -H | head -n 1 ; df -H | grep "$DISK"
echo "-------------------------------------------------------------------------------- DISK I/O"
echo ""
iostat $DISK | grep "Device" -A 1
echo ""
echo "-------------------------------------------------------------------------------- FREE MEMORY"
free -m
echo ""
echo "-------------------------------------------------------------------------------- logs/electrs/current"
tail -n 5 $MOUNTPOINT/esplora/logs/electrs/current
echo ""
echo "-------------------------------------------------------------------------------- electrs_bitcoin_db/mainnet/mainnet/newindex/history/LOG"
tail -n 30 $MOUNTPOINT/esplora/electrs_bitcoin_db/mainnet/mainnet/newindex/history/LOG
echo ""
@84adam
Copy link
Author

84adam commented Sep 22, 2022 •
edited

Example Output

-------------------------------------------------------------------------------- DISK SPACE USED/REMAINING
Filesystem     1M-blocks   Used Available Use% Mounted on
/dev/sda         1218914 989943    169949  86% /mnt/ext
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda        1.3T  1.1T  179G  86% /mnt/ext
-------------------------------------------------------------------------------- DISK I/O

Device             tps    kB_read/s    kB_wrtn/s    kB_dscd/s    kB_read    kB_wrtn    kB_dscd
sda              17.26      1096.29       591.09       765.12 1418734169  764946556  990164612

-------------------------------------------------------------------------------- FREE MEMORY
              total        used        free      shared  buff/cache   available
Mem:          16009        9430         170           1        6408        6241
Swap:             0           0           0

-------------------------------------------------------------------------------- logs/electrs/current
2-2022-11-16T17:08:31.875+00:00 - INFO - handle GET /blocks
2-2022-11-16T17:08:32.096+00:00 - INFO - handle GET /mempool/recent
2-2022-11-16T17:08:34.100+00:00 - INFO - handle GET /blocks/tip/height
2-2022-11-16T17:08:39.094+00:00 - INFO - handle GET /mempool/recent
2-2022-11-16T17:08:45.003+00:00 - INFO - hash=000000000000000000027c4914acfea77110f930f4f3e9938a673a63cbf5be8d height=763451 @ 2022-11-16T17:08:10Z (1 left to index)

-------------------------------------------------------------------------------- electrs_bitcoin_db/mainnet/mainnet/newindex/history/LOG
Level    Files   Size     Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  L0      0/0    0.00 KB   0.0      0.0     0.0      0.0       2.0      2.0       0.0   1.0      0.0     91.2     22.97             15.02        14    1.641       0      0       0.0       0.0
  L1      0/0    0.00 KB   0.0      2.2     2.2      0.0       2.1      2.1       0.0   1.0     76.5     72.8     29.44             19.83         7    4.205     33M   366K       0.0       0.0
  L2      2/0    1.68 GB   0.7     13.8     2.1     11.7      13.5      1.8       0.0   6.4     77.2     75.6    182.71            135.25         7   26.101    233M  1477K       0.0       0.0
  L3      1/0    1.01 GB   0.0      0.0     0.0      0.0       0.0      0.0       1.0   0.0      0.0      0.0      0.00              0.00         0    0.000       0      0       0.0       0.0
  L5    354/0   352.32 GB   0.1      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0      0.00              0.00         0    0.000       0      0       0.0       0.0
 Sum    357/0   355.01 GB   0.0     16.0     4.3     11.7      17.6      6.0       1.0   8.6     69.5     76.8    235.12            170.10        28    8.397    266M  1843K       0.0       0.0
 Int      0/0    0.00 KB   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0      0.00              0.00         0    0.000       0      0       0.0       0.0

** Compaction Stats [default] **
Priority    Files   Size     Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB)
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Low      0/0    0.00 KB   0.0     16.0     4.3     11.7      15.6      3.9       0.0   0.0     77.1     75.2    212.14            155.08        14   15.153    266M  1843K       0.0       0.0
High      0/0    0.00 KB   0.0      0.0     0.0      0.0       2.0      2.0       0.0   0.0      0.0     91.6     22.42             15.02        13    1.725       0      0       0.0       0.0
User      0/0    0.00 KB   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0     74.1      0.55              0.00         1    0.552       0      0       0.0       0.0

Blob file count: 0, total size: 0.0 GB

Uptime(secs): 829805.9 total, 0.0 interval
Flush(GB): cumulative 2.046, interval 0.000
AddFile(GB): cumulative 0.000, interval 0.000
AddFile(Total Files): cumulative 0, interval 0
AddFile(L0 Files): cumulative 0, interval 0
AddFile(Keys): cumulative 0, interval 0
Cumulative compaction: 17.63 GB write, 0.02 MB/s write, 15.97 GB read, 0.02 MB/s read, 235.1 seconds
Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

** File Read Latency Histogram By Level [default] **

TIP: Try with watch command. E.g. watch -n 2 ./progress-compaction.sh

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment