These notes were compiled from a fresh installation on Ubuntu 20.04. The resulting Openstack installation is really just good for local development.
The network was on Google Wifi pucks. There is some additional notes about how to handle accessing the Openstack network in the routes section
My network network details are as follows.
- Gateway
192.168.86.1 - Subnet
192.168.86.0/24 - Host IP
192.168.86.152 -
- This was through DHCP and will cause problems if the computer ever loses it's lease.
The ip_forward setting seems to be necessary. The rest is optimizations.
sysctl -w net.ipv4.ip_forward=1 | sudo tee -a /etc/sysctl.conf
sysctl -w fs.inotify.max_queued_events=1048576 | sudo tee -a /etc/sysctl.conf
sysctl -w fs.inotify.max_user_instances=1048576 | sudo tee -a /etc/sysctl.conf
sysctl -w fs.inotify.max_user_watches=1048576 | sudo tee -a /etc/sysctl.conf
sysctl -w vm.max_map_count=262144 | sudo tee -a /etc/sysctl.conf
sysctl -w vm.swappiness=1 | sudo tee -a /etc/sysctl.conf
This will speed up boot time.
sed -i -e 's:^GRUB_TIMEOUT=*.:GRUB_TIMEOUT=1:g' /etc/default/grub
update-grub
The installation is done via snap and we set aliases for the openstack and ovs-vsctl commands.
snap install microstack --devmode --beta
snap alias microstack.openstack openstack
snap alias microstack.ovs-vsctl ovs-vsctl
This will initialize a control node without prompting you as well as setup a 50G cinder volume via .img file mounted over loopback.
microstack init --auto --control --setup-loop-based-cinder-lvm-backend --loop-device-file-size 50
This will increase some of the default quotas. For other options, see openstack quota set --help
openstack quota set --secgroups -1 --cores 128 --instances 100 --ram 52000 admin --volumes 1000 --routers 50 --networks 50 --secgroup-rules 500
Running these commands will add a security group called allow that provides open access to instances it is attached to.
openstack security group delete $(openstack security group list --project admin -c ID -f value)
openstack security group create --project admin allow
openstack security group rule list allow -f value -c ID | xargs -n1 -I{} openstack security group rule delete {}
openstack security group rule create allow --project admin --ethertype IPv4 --ingress
openstack security group rule create allow --project admin --ethertype IPv4 --egress
This will delete the default microstack ssh key if it exists.
openstack keypair delete microstack
Instead of deleting the existing test networks, this will rename them so they can just be consumed. The idea is that you will create static routes to reach 10.20.20.0/24. More on that later though.
openstack subnet set --name internal-subnet test-subnet # 192.168.222.0/24
openstack network set --name internal-net test # 192.168.222.0/24
openstack subnet set --name openstack-subnet external-subnet # 10.20.20.0/24
openstack network set --name openstack-net external # 10.20.20.0/24
openstack router set --name openstack-router test-router # 10.20.20.0/24
Sharing the network between projects is recommended so that lesser privileged users can have reduced access to specific projects, but still assign networks.
openstack network set --share openstack-net
openstack network set --share internal-net
Setup DNS with these commands.
cat <<EOF > /etc/systemd/resolved.conf
[Resolve]
DNS=8.8.8.8 8.8.4.4
EOF
systemctl restart systemd-resolved.service
This is completely optional and takes a few moments to complete. The following commands will delete the existing flavors and generate an array of options between vCPU, RAM, and Disk sizing. It is convenient when you need the options.
openstack flavor list -c Name -f value | xargs -n1 -I{} openstack flavor delete {}
for cpu in 1 2 4 8; do for ram in 1024 2048 4096 8192; do for disk in 5 10 20; do openstack flavor create "$cpu.$ram.$disk" --vcpus $cpu --ram $ram --disk $disk; done; done; done
This is necessary to fix volumes.
cat <<EOF > /var/snap/microstack/common/etc/cinder/cinder.conf.d/glance.conf
[DEFAULT]
glance_ca_certificates_file = /var/snap/microstack/common/etc/ssl/certs/cacert.pem
EOF
snap restart microstack.cinder-{uwsgi,scheduler,volume}
You will need to modify this to fit your system.
This script will ensure device enp4s0 is added to br-ex, purge both enp4s0 & wlp5s0, add IP 192.168.86.152 to br-ex, bring up br-ex, and finally add a default route for 192.168.86.1 using br-ex.
cat <<EOF > /usr/local/bin/microstack-br-workaround
#!/bin/bash
which ovs-vsctl > /dev/null || (echo ovs-vsctl not found && exit 0)
ovs-vsctl add-port br-ex enp4s0 || :
ip addr flush dev enp4s0 || :
ip addr flush dev wlp5s0 || :
ip address add 192.168.86.152/24 dev br-ex || :
ip link set br-ex up || :
ip route add default via 192.168.86.1 dev br-ex || :
EOF
chmod +x /usr/local/bin/microstack-br-workaround
/usr/local/bin/microstack-br-workaround
This will run the workaround on boot.
cat <<EOF > /etc/systemd/system/microstack-br-workaround.service
[Unit]
Description=Service for adding physical ip to microstack bridge
Requires=snap.microstack.external-bridge.service
After=snap.microstack.external-bridge.service
[Service]
ExecStart=/usr/local/bin/microstack-br-workaround
SyslogIdentifier=microstack-br-workaround
Restart=no
WorkingDirectory=/usr/local/bin
TimeoutStopSec=30
Type=oneshot
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable microstack-br-workaround.service
At this point it should be possible to persist a reboot. Check out the Cirros Image section and confirm networking works. Reboot and double check it.
Previously when setting up the networking, attempts at assinging anything from 192.168.86.0/24 was avoided. So that leaves the question of how you would connect to the "external" IP's that are being assigned.
You setup static routes! Ideally, you can set the static route on your router. However if you are using something as braindead as Google Wifi pucks, you can just add static routes to your devices that need to reach the network.
route -n add -net 10.20.20.0/24 192.168.86.152
route -n delete -net 10.20.20.0/24 192.168.86.152
Persisting this will vary. You can always do something similar to the networking workaround.
ip route add 10.20.20.0/24 via 192.168.86.152
Setting up the network inside a VPN that is isolated inside of the server, but can be connected to via whatever IP happened to be DHCP'ed on the nic could provide a consistent access method.
/var/snap/microstack/common/log- Most Openstack logs can be found in here./var/log/syslog- For things like Cinder, relevant logging will likely end up here.
Connectivity problems tend to boil down to routes if everything else in this guide was followed.
You should only see a default route for the 192 subnet on br-ex like so.
# ip r
default via 192.168.86.1 dev br-ex
10.20.20.0/24 dev br-ex proto kernel scope link src 10.20.20.1
192.168.86.0/24 dev br-ex proto kernel scope link src 192.168.86.152
Using the cirros, some simple testing can be done. The default password is gocubsgo.
openstack server create --flavor m1.tiny --image cirros --nic net-id=test --security-group allow test --wait
ssh cirros@10.20.20.208
...
curl google.com
ping 1.1.1.1
This is an example of getting Ubuntu 20.04. More images are discussed and discoverable here: https://docs.openstack.org/image-guide/obtain-images.html
curl -O https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.img
openstack image create ubuntu-20.04 --public --disk-format=qcow2 --container-format=bare --property os_distro='ubuntu' --file=focal-server-cloudimg-amd64.img
rm -rf focal-server-cloudimg-amd64.img
The admin password can be retrieved with this command. The dashboard will be at https://${IP-OF-HOST}. The username is admin.
snap get microstack config.credentials.keystone-password
echo 'user ALL=(ALL) NOPASSWD: ALL' > /etc/sudoers.d/user
This will increate the root partition by 500G.
lvresize /dev/mapper/ubuntu--vg-ubuntu--lv -L+500G --resizefs
Increase cinder-lvm.img by 500G
dd if=/dev/zero bs=1G of=/var/snap/microstack/common/cinder-lvm.img conv=notrunc oflag=append count=500
Identify the loopback device. In the example below, it is /dev/loop4.
# pvs -S vg_name=cinder-volumes
PV VG Fmt Attr PSize PFree
/dev/loop4 cinder-volumes lvm2 a-- <538.28g 0
Force the loop driver to reread the size of the file associated with the specified loop device.
losetup -c /dev/loop4
Resize the physical and logical volumes.
pvresize /dev/loop4
lvextend -r -l +100%FREE /dev/cinder-volumes/cinder-volumes-pool
When a volume is added, this is how to setup the file system.
pvcreate /dev/vdb
vgcreate asdf-vg /dev/vdb
lvcreate -l100%FREE --name asdf-vol asdf-vg
mkfs.xfs /dev/asdf-vg/asdf-vol
mkdir /mnt/asdf-vol
mount /dev/asdf-vg/asdf-vol /mnt/asdf-vol
This is a pretty simple way to persist the mountpoint between reboots.
# mount | tail -n1
/dev/mapper/asdf--vg-asdf--vol on /mnt/asdf-vol type xfs (rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota)
# mount | tail -n1 >> /etc/fstab
Open screen if you are connected remotely. This way if you get disconnected, the commands will still complete. After the reboot networking should be reverted to normal.
screen
Copy and past the entire block.
rm -rf /etc/systemd/system/microstack-br-workaround.service; \
systemctl daemon-reload; \
snap remove --purge microstack; \
shutdown -r now