Skip to content

Instantly share code, notes, and snippets.

@darkmuggle

darkmuggle/LUKS Secret

Created December 10, 2019 18:54
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save darkmuggle/34ff7f19d6e7fe261ed8988526dc8ead to your computer and use it in GitHub Desktop.
Save darkmuggle/34ff7f19d6e7fe261ed8988526dc8ead to your computer and use it in GitHub Desktop.
Download ZIP
Raw
LUKS
# open null-cipher via "cryptsetup luksOpen /dev/nbd5p4 LUKS"
# mounted /dev/mapper/LUKS /mnt/test
# mkdir /mnt/test/ostree/test
# cd /mnt/test/ostree/test
[root@porpentina mnt]# blkid /dev/nbd0p4
/dev/nbd0p4: UUID="00000000-0000-4000-a000-000000000002" LABEL="crypt_rootfs" TYPE="crypto_LUKS" PARTLABEL
="luks_root" PARTUUID="c91e8850-531b-4133-bdc8-6deb7bc0e337"
[root@porpentina mnt]# blkid /dev/mapper/LUKS
/dev/mapper/LUKS: LABEL="root" UUID="18f7c0dd-9623-4148-ae2b-f00102101106" TYPE="xfs"
[root@porpentina test]# fio --rw=write --ioengine=sync --fdatasync=1 --directory=. --size=22m --bs=2300 --name=mytest
mytest: (g=0): rw=write, bs=(R) 2300B-2300B, (W) 2300B-2300B, (T) 2300B-2300B, ioengine=sync, iodepth=1
fio-3.14
Starting 1 process
mytest: Laying out IO file (1 file / 22MiB)
Jobs: 1 (f=1): [W(1)][100.0%][w=427KiB/s][w=190 IOPS][eta 00m:00s]
mytest: (groupid=0, jobs=1): err= 0: pid=4117: Tue Dec 10 11:49:07 2019
write: IOPS=188, BW=422KiB/s (433kB/s)(21.0MiB/53329msec); 0 zone resets
clat (usec): min=7, max=133, avg=38.48, stdev= 9.61
lat (usec): min=7, max=135, avg=39.70, stdev= 9.79
clat percentiles (usec):
| 1.00th=[ 13], 5.00th=[ 29], 10.00th=[ 31], 20.00th=[ 32],
| 30.00th=[ 33], 40.00th=[ 36], 50.00th=[ 39], 60.00th=[ 41],
| 70.00th=[ 42], 80.00th=[ 43], 90.00th=[ 49], 95.00th=[ 56],
| 99.00th=[ 71], 99.50th=[ 77], 99.90th=[ 96], 99.95th=[ 100],
| 99.99th=[ 120]
bw ( KiB/s): min= 399, max= 476, per=99.94%, avg=421.76, stdev=12.91, samples=106
iops : min= 177, max= 212, avg=187.95, stdev= 5.75, samples=106
lat (usec) : 10=0.26%, 20=1.93%, 50=90.17%, 100=7.60%, 250=0.04%
fsync/fdatasync/sync_file_range:
sync (usec): min=1057, max=19817, avg=5268.03, stdev=2737.32
sync percentiles (usec):
| 1.00th=[ 1827], 5.00th=[ 2180], 10.00th=[ 2245], 20.00th=[ 2343],
| 30.00th=[ 2442], 40.00th=[ 2573], 50.00th=[ 6849], 60.00th=[ 7046],
| 70.00th=[ 7177], 80.00th=[ 7373], 90.00th=[ 7635], 95.00th=[ 9634],
| 99.00th=[12387], 99.50th=[13304], 99.90th=[15401], 99.95th=[16319],
| 99.99th=[19006]
cpu : usr=0.58%, sys=3.43%, ctx=31871, majf=0, minf=10
IO depths : 1=200.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,10029,0,0 short=10029,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=422KiB/s (433kB/s), 422KiB/s-422KiB/s (433kB/s-433kB/s), io=21.0MiB (23.1MB), run=53329-53329msec
Disk stats (read/write):
dm-4: ios=0/31219, merge=0/0, ticks=0/78660, in_queue=78660, util=38.01%, aggrios=0/26112, aggrmerge=0/5215, aggrticks=0/50895, aggrin_queue=37884, aggrutil=41.19%
nbd0: ios=0/26112, merge=0/5215, ticks=0/50895, in_queue=37884, util=41.19%
Raw
NO LUKS
# mounted /
# mkdir /mnt/test/ostree/test
# cd /mnt/test/ostree/test
[root@porpentina mnt]# blkid /dev/nbd5
/dev/nbd5: PTUUID="00000000-0000-4000-a000-000000000001" PTTYPE="gpt"
[root@porpentina mnt]# blkid /dev/nbd5p4
/dev/nbd5p4: LABEL="root" UUID="18f7c0dd-9623-4148-ae2b-f00102101106" TYPE="xfs" PARTLABEL="luks_root" PARTUUID="c91e8850-531b-4133-bdc8-6deb7bc0e337"
[root@porpentina mnt]# mount /dev/nbd5p4 testing
[root@porpentina ostree]# fio --rw=write --ioengine=sync --fdatasync=1 --directory=. --size=22m --bs=2300 --name=mytest
mytest: (g=0): rw=write, bs=(R) 2300B-2300B, (W) 2300B-2300B, (T) 2300B-2300B, ioengine=sync, iodepth=1
fio-3.14
Starting 1 process
mytest: Laying out IO file (1 file / 22MiB)
Jobs: 1 (f=1): [W(1)][100.0%][w=427KiB/s][w=190 IOPS][eta 00m:00s]
mytest: (groupid=0, jobs=1): err= 0: pid=4303: Tue Dec 10 11:53:35 2019
write: IOPS=192, BW=432KiB/s (442kB/s)(21.0MiB/52183msec); 0 zone resets
clat (usec): min=5, max=145, avg=36.55, stdev=12.01
lat (usec): min=6, max=147, avg=37.72, stdev=12.31
clat percentiles (usec):
| 1.00th=[ 9], 5.00th=[ 12], 10.00th=[ 18], 20.00th=[ 31],
| 30.00th=[ 32], 40.00th=[ 34], 50.00th=[ 39], 60.00th=[ 41],
| 70.00th=[ 42], 80.00th=[ 43], 90.00th=[ 49], 95.00th=[ 55],
| 99.00th=[ 68], 99.50th=[ 73], 99.90th=[ 92], 99.95th=[ 95],
| 99.99th=[ 101]
bw ( KiB/s): min= 381, max= 503, per=100.00%, avg=431.26, stdev=25.76, samples=104
iops : min= 170, max= 224, avg=192.20, stdev=11.46, samples=104
lat (usec) : 10=2.50%, 20=8.04%, 50=80.81%, 100=8.62%, 250=0.03%
fsync/fdatasync/sync_file_range:
sync (usec): min=1013, max=49174, avg=5155.86, stdev=2934.82
sync percentiles (usec):
| 1.00th=[ 1369], 5.00th=[ 1844], 10.00th=[ 2040], 20.00th=[ 2212],
| 30.00th=[ 2311], 40.00th=[ 2409], 50.00th=[ 6390], 60.00th=[ 6980],
| 70.00th=[ 7177], 80.00th=[ 7373], 90.00th=[ 7898], 95.00th=[ 9241],
| 99.00th=[13698], 99.50th=[14091], 99.90th=[16581], 99.95th=[19006],
| 99.99th=[22676]
cpu : usr=0.54%, sys=3.65%, ctx=40463, majf=0, minf=11
IO depths : 1=200.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,10029,0,0 short=10029,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=432KiB/s (442kB/s), 432KiB/s-432KiB/s (442kB/s-442kB/s), io=21.0MiB (23.1MB), run=52183-52183m
sec
Disk stats (read/write):
nbd5: ios=0/25656, merge=0/23, ticks=0/73849, in_queue=61323, util=40.73%
@darkmuggle
Copy link
Author

darkmuggle commented Dec 10, 2019 •
edited

NO LUKS test using the preferred method

[root@porpentina test]# cat etcd.fio 
[global]
name=etcd-io
filename=/mount/etcd-io-delete-me
rw=write
bs=2300
fdatasync=1

[file1]
size=22M
ioengine=sync

[root@porpentina test]#  podman run --volume "$PWD:/mount:Z" docker.io/ljishen/fio  /mount/etcd.fio > result
[root@porpentina test]#  cat result 
file1: (g=0): rw=write, bs=(R) 2300B-2300B, (W) 2300B-2300B, (T) 2300B-2300B, ioengine=sync, iodepth=1
fio-3.6
Starting 1 process
file1: Laying out IO file (1 file / 22MiB)
file1: (groupid=0, jobs=1): err= 0: pid=11: Tue Dec 10 19:25:40 2019
  write: IOPS=311, BW=700KiB/s (717kB/s)(21.0MiB/32167msec)
    clat (usec): min=5, max=417, avg=34.13, stdev=18.66
     lat (usec): min=5, max=420, avg=35.07, stdev=19.06
    clat percentiles (usec):
     |  1.00th=[    8],  5.00th=[   10], 10.00th=[   12], 20.00th=[   17],
     | 30.00th=[   29], 40.00th=[   32], 50.00th=[   35], 60.00th=[   40],
     | 70.00th=[   41], 80.00th=[   43], 90.00th=[   51], 95.00th=[   63],
     | 99.00th=[   92], 99.50th=[  105], 99.90th=[  198], 99.95th=[  210],
     | 99.99th=[  322]
   bw (  KiB/s): min=  570, max= 1046, per=99.99%, avg=699.91, stdev=101.03, samples=64
   iops        : min=  254, max=  466, avg=311.87, stdev=44.96, samples=64
  lat (usec)   : 10=6.46%, 20=16.83%, 50=66.13%, 100=9.97%, 250=0.56%
  lat (usec)   : 500=0.05%
  fsync/fdatasync/sync_file_range:
    sync (usec): min=968, max=51041, avg=3163.54, stdev=1499.68
    sync percentiles (usec):
     |  1.00th=[ 1188],  5.00th=[ 1352], 10.00th=[ 1565], 20.00th=[ 2057],
     | 30.00th=[ 2212], 40.00th=[ 2376], 50.00th=[ 2802], 60.00th=[ 3621],
     | 70.00th=[ 4293], 80.00th=[ 4424], 90.00th=[ 4621], 95.00th=[ 4817],
     | 99.00th=[ 6325], 99.50th=[ 7439], 99.90th=[10159], 99.95th=[15008],
     | 99.99th=[46400]
  cpu          : usr=0.90%, sys=5.38%, ctx=38594, majf=0, minf=11
  IO depths    : 1=200.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,10029,0,0 short=10029,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: bw=700KiB/s (717kB/s), 700KiB/s-700KiB/s (717kB/s-717kB/s), io=21.0MiB (23.1MB), run=32167-32167msec

Disk stats (read/write):
  nbd5: ios=0/25461, merge=0/211, ticks=0/39513, in_queue=28165, util=62.63%

@darkmuggle
Copy link
Author

LUKS

root@porpentina test]# df .                                                                         [9/9]
Filesystem       1K-blocks    Used Available Use% Mounted on                                              
/dev/mapper/LUKS   2594816 1928568    666248  75% /var/mnt/testing                                        
[root@porpentina test]# dmsetup info /dev/mapper/LUKS                                                     
Name:              LUKS                                                                                   
State:             ACTIVE                                                                                 
Read Ahead:        256                                                                                    
Tables present:    LIVE                                                                                   
Open count:        1                                                                                      
Event number:      0                                                                                      
Major, minor:      253, 4
Number of targets: 1
UUID: CRYPT-LUKS2-0000000000004000a000000000000002-LUKS

[root@porpentina test]# cat etcd.fio 
[global]
name=etcd-io
filename=/mount/etcd-io-delete-me
rw=write
bs=2300
fdatasync=1

[file1]
size=22M
ioengine=sync
[root@porpentina test]# podman run --volume "$PWD:/mount:Z" docker.io/ljishen/fio  /mount/etcd.fio > resul
t
[root@porpentina test]# cat result 
file1: (g=0): rw=write, bs=(R) 2300B-2300B, (W) 2300B-2300B, (T) 2300B-2300B, ioengine=sync, iodepth=1
fio-3.6
Starting 1 process
file1: Laying out IO file (1 file / 22MiB)

file1: (groupid=0, jobs=1): err= 0: pid=11: Tue Dec 10 19:30:44 2019
  write: IOPS=278, BW=627KiB/s (642kB/s)(21.0MiB/35953msec)
    clat (usec): min=5, max=124, avg=37.05, stdev=10.41
     lat (usec): min=5, max=127, avg=38.04, stdev=10.59
    clat percentiles (usec):
     |  1.00th=[   10],  5.00th=[   20], 10.00th=[   30], 20.00th=[   31],
     | 30.00th=[   32], 40.00th=[   34], 50.00th=[   40], 60.00th=[   41],
     | 70.00th=[   41], 80.00th=[   42], 90.00th=[   46], 95.00th=[   55],
     | 99.00th=[   72], 99.50th=[   79], 99.90th=[   97], 99.95th=[  100],
     | 99.99th=[  115]
   bw (  KiB/s): min=  588, max=  750, per=100.00%, avg=626.11, stdev=32.29, samples=71
   iops        : min=  262, max=  334, avg=279.03, stdev=14.36, samples=71
  lat (usec)   : 10=1.51%, 20=3.59%, 50=88.22%, 100=6.62%, 250=0.06%
  fsync/fdatasync/sync_file_range:
    sync (usec): min=1032, max=14477, avg=3537.58, stdev=1282.09
    sync percentiles (usec):
     |  1.00th=[ 1467],  5.00th=[ 2057], 10.00th=[ 2180], 20.00th=[ 2311],
     | 30.00th=[ 2409], 40.00th=[ 2507], 50.00th=[ 4047], 60.00th=[ 4359],
     | 70.00th=[ 4490], 80.00th=[ 4621], 90.00th=[ 4686], 95.00th=[ 4817],
     | 99.00th=[ 7439], 99.50th=[ 7898], 99.90th=[11207], 99.95th=[13042],
     | 99.99th=[14091]
  cpu          : usr=0.86%, sys=5.70%, ctx=31501, majf=0, minf=11
  IO depths    : 1=200.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,10029,0,0 short=10029,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1
     
Run status group 0 (all jobs):
  WRITE: bw=627KiB/s (642kB/s), 627KiB/s-627KiB/s (642kB/s-642kB/s), io=21.0MiB (23.1MB), run=35953-35953m
sec

Disk stats (read/write):
    dm-4: ios=0/31273, merge=0/0, ticks=0/44791, in_queue=44791, util=56.55%, aggrios=0/26099, aggrmerge=0
/5224, aggrticks=0/32401, aggrin_queue=19325, aggrutil=60.59%
  nbd0: ios=0/26099, merge=0/5224, ticks=0/32401, in_queue=19325, util=60.59%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment