virtualdata-cheatsheet.md

VirtualData cheatsheet

Pre-requisites

Before starting, each user needs to open an account on the cloud: https://registration.lal.in2p3.fr/register. In addition, you would need Python 3.5+ installed on your computer.

Install the component clients

Openstack is a cloud computing platform, containing many components that offer different services. Let's cite Nova for the compute, Cinder for the block storage, Keystone to manage identity, or Glance for image. All components can be found at https://en.wikipedia.org/wiki/OpenStack#Components. For our purpose, we need to install a few of them

# Compute, Identity, Image, Network, Object Store and Block Storage APIs together
pip3 install python-openstackclient

# We add the Glance one as well
pip3 install python-glanceclient

More information at https://github.com/openstack

Configuration

The IJCLab infrastructure uses TERENA certificates which is an intermediate authority. This poses a concern with OpenStack which does not know how to send back the certification chains. By default, the OpenStack CLI therefore cannot verify server certificates. Hence you must use the following workaround:

# create a new folder on your $HOME
mkdir $HOME/.certs

# Download additional auth and move it into the previous folder
wget https://openstack.lal.in2p3.fr/files/2016/02/terena.pem
mv terena.pem $HOME/.certs

Then connect to the web interface (https://keystone.lal.in2p3.fr), download your credential (RC file) and move it under $HOME/.certs. You can check you correctly register by running:

source $HOME/.certs/<name of the file>.rc

openstack token issue
# you should see a table with long ID numbers

Finally, you can register your ssh key to ease the creation of resources later. If you already have a public key available:

# You can give a better name than `ma-clef-ssh`
openstack keypair create --public-key $HOME/.ssh/id_rsa.pub ma-clef-ssh

Otherwise, you will need to create one

# Check before that you have no key!
if [[ ! -f $HOME/.ssh/id_rsa ]]; then
  mkdir -p $HOME/.ssh/ && \
  openstack keypair create ma-clef-ssh > $HOME/.ssh/id_rsa && \
  chmod 600 $HOME/.ssh/id_rsa
fi

Complete information can be found here.

Create your first machine (CLI)

First you need to activate your permissions:

source $HOME/.certs/<name of the file>.rc

Instantiate a new VM

Now you are ready! Let's inspect which images are available to you, that is the Operating System (or OS):

glance image-list --visibility public

You can also check the available flavors, that is the amount of vCPU, amount of memory (RAM), size of system disk. You will note that the system disk size must be greater or equal to the boot image size.

nova flavor-list

Finally as they are several networks, you need to specify one. To list all:

openstack network list

In our case, choose the ID corresponding to public-2. For example, let's instantiate a VM running on centos7, with 2 vCPU and 2GB RAM:

# `my-ssh-key` should be the name of the key you set above
# `mysupervm` can be anything (choose a great name!)
nova boot --nic net-id=<ID public-2> --flavor vd.2 --image CentOS-7-x86_64-GenericCloud-2009 --key-name my-ssh-key mysupervm

It will take a few seconds to be active. You can check the status:

nova list
+--------------------------------------+-----------------+--------+------------+-------------+-----------------------+
| ID                                   | Name            | Status | Task State | Power State | Networks              |
+--------------------------------------+-----------------+--------+------------+-------------+-----------------------+
| 99f42bd2-8372-4228-9c74-47e21e26f5a1 | masupervm       | ACTIVE | -          | Running     | public=xx.xx.xx.xx    |
+--------------------------------------+-----------------+--------+------------+-------------+-----------------------+

Once active, the public IP will be available on the status, and you can connect:

# In case you do not know the user login, try root@...
# an error message will tell you which user to use.
ssh centos@xx.xx.xx.xx

Here you are!

Volume management

By default, you have a small local disk that comes with your VM (typically 20 GB). You might need more, and Cinder is here for you! On your computer (not in the VM!):

# The last number is the size in GB
cinder create --name my-super-volume-name 5

When you create your volume, there is a table summarizing properties, and one of the row is the id. You can attach your new volume to your previous VM:

nova volume-attach masupervm id-of-my-super-volume-name auto

You can check in the VM that the volume has been attached:

ssh centos@xx.xx.xx.xx
sudo su -
fdisk -l /dev/vdb

There should be something not comprehensible by normal human. You are almost there - you need to format your volume now to use it, that is creating a filesystem and mounting it. Let's use standard xfs filesystem, and mount it under /data:

# you must be sudo again
parted /dev/vdb mklabel gpt
parted --align none /dev/vdb -- mkpart primary xfs 0 -1
mkfs.xfs /dev/vdb1
mkdir /data
mount /dev/vdb1 /data

Finally, you need to persist it, otherwise it will be destroyed at the next reboot of the VM:

# you must be sudo again
data=$(blkid|grep /dev/vdb1|cut -d " " -f 2)
cat >> /etc/fstab << EOF
$data        /data   xfs    defaults,nofail        0       0
EOF

Et voilà! Now you can use your VM, with an additional volume on it :-)

Checking resources

On your laptop, you can quickly view VM and storage usage:

# List VMs
nova list

# List storages
cinder list

Releasing the resources

Keep the resources when you need them, but do not forget to release them when they are not needed anymore. From your local computer:

# Detaching a volume
nova volume-detach masupervm id-of-my-super-volume-name

# Destroying a volume
cinder delete id-of-my-super-volume-name

# Destroying a VM
nova delete masupervm

Web interface

You can also instantiate VM and manage volumes directly from the web interface: https://keystone.lal.in2p3.fr