Skip to content

Instantly share code, notes, and snippets.

@salrashid123

salrashid123/cos_gcp.md

Last active January 3, 2021 13:48
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save salrashid123/c9a8e59b86e41329dd8f2052a38915f5 to your computer and use it in GitHub Desktop.
Save salrashid123/c9a8e59b86e41329dd8f2052a38915f5 to your computer and use it in GitHub Desktop.
Download ZIP
Create LUKS mount on GCP COS image (loopback)
Raw
cos_gcp.md

Procedure to mount a LUKS loopback device on attached GCP Persistent Disk into a Container Optimized OS

  • Create a disk and mount to a plain VM
  • Create loopback device on the disk and mount as LUKS
  • Write file to LUKS mount path
  • Detach disk from VM
  • Create COS VM and attach disk (in this case, as /dev/sdb)
  • Start container with cloud-init and map device /usr/bin/docker run --rm -u 0 --privileged --device /dev/sdb:/dev/sdb --name=mycloudservice docker.io/
  • Mount disk
  • Create loopback device and mount as LUKS
  • Access file

Create plain VM, Disk

### Create VM
$ gcloud compute  instances create instance-1 \
  --zone=us-central1-a  --no-service-account \
  --no-scopes --image=debian-10-buster-v20201014 \
  --image-project=debian-cloud --boot-disk-size=10GB \
  --boot-disk-type=pd-standard --boot-disk-device-name=instance-1 

### Create and attach disk
$ gcloud beta compute disks create luks-lo  --type=pd-standard --size=10GB --zone=us-central1-a

$ gcloud compute instances attach-disk instance-1  --disk luks-lo

Create filesystem

### SSH to VM
$ gcloud compute ssh instance-1

$ apt-get update && apt-get install -y cryptsetup

### Check block devices
$ lsblk -a
    NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
    sda       8:0    0   10G  0 disk 
    ├─sda1    8:1    0  9.9G  0 part /
    ├─sda14   8:14   0    3M  0 part 
    └─sda15   8:15   0  124M  0 part /boot/efi
    sdb       8:16   0   10G  0 disk            <<<<<<<<< PD

### Make filesystem on /dev/sdb

$ mkfs.ext4 -m 0 -E lazy_itable_init=0,lazy_journal_init=0,discard /dev/sdb

### Mount Disk

$ mkdir -p /mnt/disks/test-losetup
$ mount -o discard,defaults /dev/sdb /mnt/disks/test-losetup

Create Loopback

### Create losetup

$ dd if=/dev/zero of=/mnt/disks/test-losetup/loopbackfile.img bs=100M count=10
$ losetup -fP /mnt/disks/test-losetup/loopbackfile.img


### Verify loopback

$ losetup -a
   /dev/loop0: [2096]:12 (/mnt/disks/test-losetup/loopbackfile.img)

$ lsblk -a
        NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
        loop0     7:0    0 1000M  0 loop               <<< Loopback
        loop1     7:1    0        0 loop 
        loop2     7:2    0        0 loop 
        loop3     7:3    0        0 loop 
        loop4     7:4    0        0 loop 
        loop5     7:5    0        0 loop 
        loop6     7:6    0        0 loop 
        loop7     7:7    0        0 loop 
        sda       8:0    0   10G  0 disk 
        ├─sda1    8:1    0  9.9G  0 part /
        ├─sda14   8:14   0    3M  0 part 
        └─sda15   8:15   0  124M  0 part /boot/efi
        sdb       8:16   0   10G  0 disk /mnt/disks/test-losetup  << Disk

Create LUKS

### Make LUKS

$ export luks_key=`openssl rand 16 | xxd -p`
$ echo $luks_key
# Static
$ export luks_key=e8e580f16de5d34ba73f66151bd4363a


$ echo  -n $luks_key | cryptsetup luksFormat --type luks1 /dev/loop0 -
$ echo  -n $luks_key | cryptsetup luksOpen /dev/loop0 my_encrypted_volume -


## Mount LUKS

$ mkdir -p /media/encnfs
$ mkfs.ext4 /dev/mapper/my_encrypted_volume
$ mount /dev/mapper/my_encrypted_volume /media/encnfs

### Verify
$ lsblk -a
        NAME                  MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
        loop0                   7:0    0 1000M  0 loop                        <<<< Loopback
        └─my_encrypted_volume 254:0    0  984M  0 crypt /media/encnfs         <<<< LUKS 
        loop1                   7:1    0        0 loop  
        loop2                   7:2    0        0 loop  
        loop3                   7:3    0        0 loop  
        loop4                   7:4    0        0 loop  
        loop5                   7:5    0        0 loop  
        loop6                   7:6    0        0 loop  
        loop7                   7:7    0        0 loop  
        sda                     8:0    0   10G  0 disk  
        ├─sda1                  8:1    0  9.9G  0 part  /
        ├─sda14                 8:14   0    3M  0 part  
        └─sda15                 8:15   0  124M  0 part  /boot/efi
        sdb                     8:16   0   10G  0 disk  /mnt/disks/test-losetup  <<< Disk

Create test file onLUKS

### Write file
$ echo fooobar > /media/encnfs/a.txt

Unmount, unwind

$ umount /media/encnfs
$ cryptsetup luksClose   /dev/mapper/my_encrypted_volume 
$ losetup -d /dev/loop0
$ umount /mnt/disks/test-losetup

$ lsblk -a
        NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
        loop0     7:0    0        0 loop 
        loop1     7:1    0        0 loop 
        loop2     7:2    0        0 loop 
        loop3     7:3    0        0 loop 
        loop4     7:4    0        0 loop 
        loop5     7:5    0        0 loop 
        loop6     7:6    0        0 loop 
        loop7     7:7    0        0 loop 
        sda       8:0    0   10G  0 disk 
        ├─sda1    8:1    0  9.9G  0 part /
        ├─sda14   8:14   0    3M  0 part 
        └─sda15   8:15   0  124M  0 part /boot/efi
        sdb       8:16   0   10G  0 disk

Detach the disk

gcloud compute instances detach-disk instance-1  --disk luks-lo

COS Image

Create COS image with Disk attached

Note image ExecStart mounts device directly

gcloud compute instances create cos-1  \
 --image cos-stable-81-12871-130-0 \
 --image-project cos-cloud \
 --disk=auto-delete=no,boot=no,device-name=sdb,mode=rw,name=luks-lo  
 --zone us-central1-a --machine-type n1-standard-1  \
 --metadata-from-file user-data=cloud-init.yaml

COS initscript runs container to mount disk and LUKS

Normally the container image would perform the steps contained within /home/cloudservice/luksmount/mount.sh

  • cos-init.yaml:
#cloud-config

users:
- name: cloudservice
  uid: 2000

write_files:
- path: /home/cloudservice/luksmount/mount.sh
  permissions: 0644
  owner: root
  content: |
    export DEBIAN_FRONTEND=noninteractive                                                 
    apt-get update && apt-get install -yq cryptsetup

    mkdir -p /mnt/disks/test-losetup
    mount  /dev/sdb /mnt/disks/test-losetup

    losetup -fP /mnt/disks/test-losetup/loopbackfile.img

    export luks_key=e8e580f16de5d34ba73f66151bd4363a
    echo  -n $luks_key | cryptsetup luksOpen /dev/loop0 my_encrypted_volume -

    mkdir -p /media/encnfs
    mount /dev/mapper/my_encrypted_volume /media/encnfs    

    sleep 10000  
  
- path: /etc/systemd/system/cloudservice.service
  permissions: 0644
  owner: root
  content: |
    [Unit]
    Description=Start a simple docker container
    Wants=gcr-online.target
    After=gcr-online.target

    [Service]
    Environment="HOME=/home/cloudservice"
    ExecStartPre=/usr/bin/docker-credential-gcr configure-docker
    ExecStart=/usr/bin/docker run --rm -u 0 --privileged --device /dev/sdb:/dev/sdb -v /home/cloudservice/luksmount/:/luksmount --name=mycloudservice docker.io/debian:buster /bin/bash /luksmount/mount.sh
    ExecStop=/usr/bin/docker stop mycloudservice
    ExecStopPost=/usr/bin/docker rm mycloudservice

# bootcmd:
# - iptables -D INPUT -p tcp -m tcp --dport 22 -j ACCEPT
# - systemctl mask --now serial-getty@ttyS0.service

runcmd:
- systemctl daemon-reload
- systemctl start cloudservice.service

Verify

$ gcloud compute ssh cos-1

cos-1 /home/cloudservice # lsblk -a
    NAME                  MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
    loop0                   7:0    0 1000M  0 loop                  < Loopback
    `-my_encrypted_volume 253:1    0  998M  0 crypt /media/encnfs   < LUKS
    loop1                   7:1    0        0 loop  
    loop2                   7:2    0        0 loop  
    loop3                   7:3    0        0 loop  
    loop4                   7:4    0        0 loop  
    loop5                   7:5    0        0 loop  
    loop6                   7:6    0        0 loop  
    loop7                   7:7    0        0 loop  
    sda                     8:0    0   10G  0 disk  
    |-sda1                  8:1    0  5.9G  0 part  /etc/hosts
    |-sda2                  8:2    0   16M  0 part  
    |-sda3                  8:3    0    2G  0 part  
    | `-vroot             253:0    0    2G  1 dm    
    |-sda4                  8:4    0   16M  0 part  
    |-sda5                  8:5    0    2G  0 part  
    |-sda6                  8:6    0  512B  0 part  
    |-sda7                  8:7    0  512B  0 part  
    |-sda8                  8:8    0   16M  0 part  
    |-sda9                  8:9    0  512B  0 part  
    |-sda10                 8:10   0  512B  0 part  
    |-sda11                 8:11   0    8M  0 part  
    `-sda12                 8:12   0   32M  0 part  
    sdb                     8:16   0   10G  0 disk  /mnt/disks/test-losetup   < Disk
    md0                     9:0    0        0 md    


cos-1 /home/cloudservice $ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
034932e81a56        debian:buster       "/bin/bash /luksmoun…"   28 seconds ago      Up 26 seconds                           mycloudservice

cos-1 /home/cloudservice $ docker exec -ti 034932e81a56 /bin/bash

root@034932e81a56:/$ df -kh
Filesystem                       Size  Used Avail Use% Mounted on
overlay                          5.7G  262M  5.5G   5% /
tmpfs                             64M     0   64M   0% /dev
tmpfs                            1.9G     0  1.9G   0% /sys/fs/cgroup
shm                               64M     0   64M   0% /dev/shm
/dev/sda1                        5.7G  262M  5.5G   5% /luksmount                 < mapped volume
/dev/sdb                         9.8G  1.1G  8.8G  11% /mnt/disks/test-losetup    < disk
/dev/mapper/my_encrypted_volume  967M  2.5M  898M   1% /media/encnfs              < LUKS

root@034932e81a56:/$ cd /media/encnfs
root@034932e81a56:/media/encnfs$ ls
a.txt  lost+found

root@034932e81a56:/media/encnfs$ more a.txt 
fooobar
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment