Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Ultimate OpenStack Grizzly Guide - `Quantum Single Flat', no GRE, no L3...
# Guide to OpenStack Grizzly on top of Ubuntu 12.04.2
# Covers: Ubuntu (hostnames, LVM), OpenvSwitch, MySQL, Keystone, Glance, Quantum, Nova, Cinder and
# Dashboard.
# This is a "step-by-step", a "cut-and-paste" guide...
# This guide was inspired by: http://openstack-folsom-install-guide.readthedocs.org/en/latest/
# Limitations:
# 1- No Metadata, no GRE, no L3.
# 2- Only 1 ethernet for each physical server.
# 3- The gateway (physical / instances) is located outside of the cloud.
# Features:
# 1- No NAT, no `Floating IPs', no multihost=true!
# My idea is to move on and forget about IPv4 and NAT tables, so, with IPv6, we don't need
# `Floating IPs' or NAT on compute nodes, for example with multi=true.
# NOTE:
# The contents between `---' and `---' are supposed to be added to the respective
# files, it is not a entire config file replacement. Keep the rest of the
# original files intact when possible (i.e. when not duplicating the entries).
# TODO List:
# Setup: Metadata, Spice, Swift and Ceilometer.
# External to the Cloud Computing
---- Gateway (Ubuntu 12.04.2 recommended) ----
eth0 - public IPv4 and/or IPv6 from from SixxS, TunnelBroker.net or native.
eth1 - 10.32.14.1/24, 10.33.14.1/24 and your IPv6 /64 (or /48) Block from SixxS, TunnelBroker.net or native.
# Example of its /etc/network/interfaces file:
---
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface connected to your ISP's WAN
auto eth0
iface eth0 inet static
address 200.10.1.2
netmask 28
gateway 200.10.1.1
# dns-* options are implemented by the resolvconf package, if installed
dns-search yourdomain.com
# Google Public DNS
dns-nameservers 8.8.8.8 8.8.4.4
# OpenDNS
# dns-nameservers 208.67.222.222 208.67.220.220 208.67.222.220 208.67.220.222
# OpenNIC
# dns-nameservers 66.244.95.20 74.207.247.4 216.87.84.211
# OpenStack Components Network (API, physical servers (compute) and/or generic hypervisors, gateway itself)
auto eth1
iface eth1 inet static
address 10.32.14.1
netmask 24
# OpenStack Instances Network
auto eth1:0
iface eth1:0 inet static
address 10.33.14.1
netmask 24
---
# Enable IPv4 package forwarding
sed -i 's/#net.ipv4.ip_forward=1/net.ipv4.ip_forward=1/' /etc/sysctl.conf
sysctl -p
# NAT rule
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
NOTE: There is only 1 NAT table on this environment, which resides on this gateway, to do IPv4
SNAT/DNAT to/from the "old" Internet infrastructure. There is no IPv4 NAT within this OpenStack
environment (no Floating IPs, "no multihost=true"). Also, there is no NAT when enjoying the
new `Internet Powered by IPv6'!
NOTE 2: If your have more IPv4 public blocks available, your Instances can have public IPs on it!
OPTIONAL: Install and enable the `radvd' on your eth1, with your IPv6 Block from
SixxS (NOT OpenNIC-friendly) or TunnelBroker.net (OpenNIC-friendly), that way, your
physical servers will get its IPv6 automatically, and you can start kissing goodbye to IPv4.
# Inside the Cloud!
---- Ubuntu 12.04.2 (controller.youdomain.com) ----
# Requirements: 1 Virtual Machine (KVM/Xen) with 2G of RAM - 2 Virtual HDs about 100G each - 1 eth
# 64 bits O.S. Recommended
# hostname: controller.yourdomain.com
# IPv4: 10.32.14.232/24
# Gateway: 10.32.14.1 (outside of the Cloud)
# Install Ubuntu 12.04.2 on the first disk, can be the `Minimum Virtual Machine' flavor, using
# `Guided LVM Paritioning', leave the second disk untouched for now.
# Login as root
aptitude update
aptitude install vim iptables ubuntu-cloud-keyring
# Add the Ubuntu Cloud Archive repository to your APT sources.list:
vi /etc/apt/sources.list.d/uca-grizzly.list
---
# The primary updates archive that users should be using
deb http://ubuntu-cloud.archive.canonical.com/ubuntu precise-updates/grizzly main
# Public -proposed archive mimicking the SRU process for extended testing.
# Packages should bake here for at least 7 days.
#
# deb http://ubuntu-cloud.archive.canonical.com/ubuntu precise-proposed/grizzly main
---
aptitude update
aptitude safe-upgrade -y
reboot
vi /etc/hosts
---
127.0.0.1 localhost.localdomain localhost
10.32.14.232 controller.yourdomain.com controller
10.32.14.234 compute1.yourdomain.com compute1
10.32.14.236 compute2.yourdomain.com compute2
---
aptitude install openvswitch-switch
vi /etc/network/interfaces
---
# The primary network interface
auto eth0
iface eth0 inet manual
up ip address add 0/0 dev $IFACE
up ip link set $IFACE up
#       up ip link set $IFACE promisc on
#       down ip link set $IFACE promisc off
down ip link set $IFACE down
auto br-eth0
iface br-eth0 inet static
address 10.32.14.232
netmask 255.255.255.0
network 10.32.14.0
broadcast 10.32.14.255
gateway 10.32.14.1
# dns-* options are implemented by the resolvconf package, if installed
dns-search yourdomain.com
# Google Public DNS
dns-nameservers 8.8.8.8 8.8.4.4
# OpenDNS
# dns-nameservers 208.67.222.222 208.67.220.220 208.67.222.220 208.67.220.222
# OpenNIC
# dns-nameservers 66.244.95.20 74.207.247.4 216.87.84.211
iface br-eth0 inet6 auto
# Google Public DNS
# dns-nameservers 2001:4860:4860::8888 2001:4860:4860::8844
# OpenNIC
# dns-nameservers 2001:530::216:3cff:fe8d:e704 2600:3c00::f03c:91ff:fe96:a6ad 2600:3c00::f03c:91ff:fe96:a6ad
# OpenDNS Public Name Servers:
# dns-nameservers 2620:0:ccc::2 2620:0:ccd::2
---
ovs-vsctl add-br br-int
ovs-vsctl add-br br-eth0
ovs-vsctl add-port br-eth0 eth0 && reboot # restart take ~2 min (see SLEEP at /etc/init/failsafe.conf)
ovs-vsctl show
aptitude update
aptitude install mysql-server python-mysqldb ntp curl openssl rabbitmq-server python-keyring
sed -i 's/server ntp.ubuntu.com/server ntp.ubuntu.com\nserver 127.127.1.0\nfudge 127.127.1.0 stratum 10/g' /etc/ntp.conf
service ntp restart
sed -i 's/127.0.0.1/0.0.0.0/g' /etc/mysql/my.cnf
service mysql restart
mysql -u root -p
---
CREATE DATABASE keystone;
GRANT ALL ON keystone.* TO 'keystoneUser'@'%' IDENTIFIED BY 'keystonePass';
CREATE DATABASE glance;
GRANT ALL ON glance.* TO 'glanceUser'@'%' IDENTIFIED BY 'glancePass';
CREATE DATABASE nova;
GRANT ALL ON nova.* TO 'novaUser'@'%' IDENTIFIED BY 'novaPass';
CREATE DATABASE cinder;
GRANT ALL ON cinder.* TO 'cinderUser'@'%' IDENTIFIED BY 'cinderPass';
CREATE DATABASE quantum;
GRANT ALL ON quantum.* TO 'quantumUser'@'%' IDENTIFIED BY 'quantumPass';
quit;
---
vi ~/.novarc
---
export OS_NO_CACHE=1
export SERVICE_TOKEN=ADMIN
export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=admin_pass
export OS_AUTH_URL="http://controller.yourdomain.com:5000/v2.0/"
export SERVICE_ENDPOINT="http://controller.yourdomain.com:35357/v2.0/"
---
vi ~/.bashrc
---
if [ -f ~/.novarc ]; then
. ~/.novarc
fi
---
source ~/.bashrc
---- Keystone ----
aptitude install keystone
vi /etc/keystone/keystone.conf
---
[DEFAULT]
admin_token = ADMIN
connection = mysql://keystoneUser:keystonePass@controller.yourdomain.com/keystone
---
keystone-manage db_sync
service keystone restart
cd ~
wget https://gist.github.com/tmartinx/5453358/raw/f132d27eeab0c3c25d5b3e65bfec6704503e84b6/keystone_basic.sh
wget https://gist.github.com/tmartinx/5453336/raw/eded917b78213123c46b62be18f55f3c7aac558e/keystone_endpoints_basic.sh
vi keystone_basic.sh
---
HOST_IP=controller.yourdomain.com
---
vi keystone_endpoints_basic.sh
---
HOST_IP=controller.yourdomain.com
EXT_HOST_IP=controller.yourdomain.com
---
chmod +x keystone_basic.sh
chmod +x keystone_endpoints_basic.sh
./keystone_basic.sh
./keystone_endpoints_basic.sh
keystone tenant-list
curl http://controller.yourdomain.com:35357/v2.0/endpoints -H 'x-auth-token: ADMIN' | python -m json.tool
---- Glance ----
aptitude install glance
vi /etc/glance/glance-api-paste.ini
---
[filter:authtoken]
paste.filter_factory = keystoneclient.middleware.auth_token:filter_factory
auth_host = controller.yourdomain.com
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = glance
admin_password = service_pass
---
vi /etc/glance/glance-registry-paste.ini
---
[filter:authtoken]
paste.filter_factory = keystoneclient.middleware.auth_token:filter_factory
auth_host = controller.yourdomain.com
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = glance
admin_password = service_pass
---
vi /etc/glance/glance-api.conf
---
sql_connection = mysql://glanceUser:glancePass@controller.yourdomain.com/glance
[paste_deploy]
flavor = keystone
---
vi /etc/glance/glance-registry.conf
---
sql_connection = mysql://glanceUser:glancePass@controller.yourdomain.com/glance
[paste_deploy]
flavor = keystone
---
glance-manage db_sync
service glance-api restart; service glance-registry restart
cd ~
wget http://download.cirros-cloud.net/0.3.1/cirros-0.3.1-i386-disk.img
wget http://download.cirros-cloud.net/0.3.1/cirros-0.3.1-x86_64-disk.img
glance image-create --name "CirrOS Minimalist - 32 Bits - Cloud Based Image" --is-public true --container-format bare --disk-format qcow2 < cirros-0.3.1-i386-disk.img
glance image-create --name "CirrOS Minimalist - 64 Bits - Cloud Based Image" --is-public true --container-format bare --disk-format qcow2 < cirros-0.3.1-x86_64-disk.img
glance image-create --location http://uec-images.ubuntu.com/releases/12.04/release/ubuntu-12.04-server-cloudimg-i386-disk1.img --is-public true --disk-format qcow2 --container-format bare --name "Ubuntu 12.04.3 LTS - Precise Pangolin - 32 Bits - Cloud Based Image"
glance image-create --location http://uec-images.ubuntu.com/releases/12.04/release/ubuntu-12.04-server-cloudimg-amd64-disk1.img --is-public true --disk-format qcow2 --container-format bare --name "Ubuntu 12.04.2 LTS - Precise Pangolin - 64 Bits - Cloud Based Image"
glance image-list
---- Quantum (controller.yourdomain.com) ----
aptitude install quantum-server quantum-plugin-openvswitch quantum-plugin-openvswitch-agent quantum-dhcp-agent
vi /etc/quantum/quantum.conf
---
[DEFAULT]
debug = True
verbose = True
allow_overlapping_ips = True
rabbit_host = controller.yourdomain.com
[keystone_authtoken]
auth_host = controller.yourdomain.com
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = quantum
admin_password = service_pass
signing_dir = /var/lib/quantum/keystone-signing
---
vi /etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini
---
[DATABASE]
sql_connection = mysql://quantumUser:quantumPass@controller.yourdomain.com/quantum
[OVS]
network_vlan_ranges = physnet1
bridge_mappings = physnet1:br-eth0
---
vi /etc/quantum/metadata_agent.ini
---
auth_url = http://controller.yourdomain.com:35357/v2.0
auth_region = RegionOne
admin_tenant_name = service
admin_user = quantum
admin_password = service_pass
nova_metadata_ip = 127.0.0.1
nova_metadata_port = 8775
metadata_proxy_shared_secret = metasecret13
---
vi /etc/quantum/dhcp_agent.ini
---
use_namespaces = True
---
cd /etc/init.d/; for i in $(ls quantum-*); do sudo service $i restart; done
# Workaround a BUG
visudo
----
quantum ALL=NOPASSWD: ALL
----
keystone tenant-list # To note the admin tenant id.
quantum net-create --tenant-id $ADMIN_TENTANT_ID sharednet1 --shared --provider:network_type flat --provider:physical_network physnet1
quantum subnet-create --ip-version 4 --tenant-id $ADMIN_TENANT_ID --gateway 10.33.14.1 sharednet1 10.33.14.0/24 --dns_nameservers list=true 8.8.8.8 8.8.4.4
--- Nova ---
aptitude install nova-api nova-cert nova-consoleauth nova-scheduler nova-novncproxy novnc
vi /etc/nova/api-paste.ini
---
[filter:authtoken]
paste.filter_factory = keystoneclient.middleware.auth_token:filter_factory
auth_host = controller.yourdomain.com
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = nova
admin_password = service_pass
signing_dir = /tmp/keystone-signing-nova
# Workaround for https://bugs.launchpad.net/nova/+bug/1154809
auth_version = v2.0
---
mv /etc/nova/nova.conf /etc/nova/nova.conf_Ubuntu
wget https://gist.github.com/tmartinx/6214746/raw/0a999bb2f6bb462ef0f687b8585f31b62884b9ec/nova.conf
chown nova: /etc/nova/nova.conf
chmod 640 /etc/nova/nova.conf
nova-manage db sync
cd /etc/init.d/; for i in $(ls nova-*); do sudo service $i restart; done
---- Cinder ----
# Use the extra Virtual HD of you controller (about 100G).
# If don't have, add one:
# halt -> virt-manager -> Add hardware -> VirtIO Disk / 100G / RAW
cfdisk /dev/vdb
pvcreate /dev/vdb1
vgcreate cinder-volumes /dev/vdb1
aptitude install cinder-api cinder-scheduler cinder-volume
vi /etc/cinder/api-paste.ini
---
[filter:authtoken]
paste.filter_factory = keystoneclient.middleware.auth_token:filter_factory
service_protocol = http
service_host = controller.yourdomain.com
service_port = 5000
auth_host = controller.yourdomain.com
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = cinder
admin_password = service_pass
signing_dir = /var/lib/cinder
---
echo "sql_connection = mysql://cinderUser:cinderPass@controller.yourdomain.com/cinder" >> /etc/cinder/cinder.conf
cinder-manage db sync
cd /etc/init.d/; for i in $(ls cinder-*); do sudo service $i restart; done
---- Dashboard ----
aptitude install openstack-dashboard memcached
vi /etc/openstack-dashboard/local_settings.py
---
COMPRESS_OFFLINE = False
---
---- Ubuntu 12.04.2 (compute1.yourdomain.com) ----
# Requirements: 1 Physical Server with Virtualization support on CPU, 1 ethernet
# IPv4: 10.32.14.234/24
# Gateway 10.32.14.1
# Install Ubuntu 12.04.2, can be the `Minimum Installation' flavor, using `Manual Paritioning',
# make the following partitions:
#
# /dev/sda1 on /boot (~256M - /dev/md0 if raid1[0], bootable)
# /dev/sda2 on LVM VG vg01 (~50G - /dev/md1 if raid1[0]) - lv_root (25G), lv_swap (XG) of compute1
# /dev/sda3 on LVM VG nova-local (~450G - /dev/md2 if raid1[0]) - Instances
aptitude update
 
aptitude install vim iptables ubuntu-cloud-keyring sysfsutils
 
# Add the Ubuntu Cloud Archive repository to your APT sources.list:
vi /etc/apt/sources.list.d/uca-grizzly.list
---
# The primary updates archive that users should be using
deb http://ubuntu-cloud.archive.canonical.com/ubuntu precise-updates/grizzly main
# Public -proposed archive mimicking the SRU process for extended testing.
# Packages should bake here for at least 7 days.
#
# deb http://ubuntu-cloud.archive.canonical.com/ubuntu precise-proposed/grizzly main
---
aptitude update
aptitude safe-upgrade -y
reboot
vi /etc/hosts
---
127.0.0.1 localhost.localdomain localhost
10.32.14.232 controller.yourdomain.com controller
10.32.14.234 compute1.yourdomain.com compute1
10.32.14.236 compute2.yourdomain.com compute2
---
vi /etc/network/interfaces
---
# The primary network interface
auto eth0
iface eth0 inet manual
up ip address add 0/0 dev $IFACE
up ip link set $IFACE up
#       up ip link set $IFACE promisc on
#       down ip link set $IFACE promisc off
down ip link set $IFACE down
auto br-eth0
iface br-eth0 inet static
address 10.32.14.234
netmask 255.255.255.0
network 10.32.14.0
broadcast 10.32.14.255
gateway 10.32.14.1
# dns-* options are implemented by the resolvconf package, if installed
dns-search yourdomain.com
# Google Public DNS
dns-nameservers 8.8.8.8 8.8.4.4
# OpenDNS
# dns-nameservers 208.67.222.222 208.67.220.220 208.67.222.220 208.67.220.222
# OpenNIC
# dns-nameservers 66.244.95.20 74.207.247.4 216.87.84.211
iface br-eth0 inet6 auto
# Google Public DNS
# dns-nameservers 2001:4860:4860::8888 2001:4860:4860::8844
# OpenNIC
# dns-nameservers 2001:530::216:3cff:fe8d:e704 2600:3c00::f03c:91ff:fe96:a6ad 2600:3c00::f03c:91ff:fe96:a6ad
# OpenDNS Public Name Servers:
# dns-nameservers 2620:0:ccc::2 2620:0:ccd::2
---
vi /etc/default/grub
---
GRUB_CMDLINE_LINUX="elevator=deadline"
---
update-grub
echo vhost_net >> /etc/modules
aptitude install openvswitch-switch
ovs-vsctl add-br br-int
ovs-vsctl add-br br-eth0
ovs-vsctl add-port br-eth0 eth0 && reboot
aptitude update
aptitude install qemu-kvm-spice kvm libvirt-bin pm-utils nova-compute-kvm quantum-plugin-openvswitch-agent nova-conductor
virsh net-destroy default
virsh net-undefine default
vi /etc/libvirt/libvirtd.conf
---
listen_tls = 0
listen_tcp = 1
auth_tcp = "none"
---
vi /etc/init/libvirt-bin.conf
---
env libvirtd_opts="-d -l"
---
vi /etc/default/libvirt-bin
---
libvirtd_opts="-d -l"
---
service libvirt-bin restart
vi /etc/nova/api-paste.ini
---
[filter:authtoken]
paste.filter_factory = keystoneclient.middleware.auth_token:filter_factory
auth_host = controller.yourdomain.com
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = nova
admin_password = service_pass
signing_dir = /tmp/keystone-signing-nova
# Workaround for https://bugs.launchpad.net/nova/+bug/1154809
auth_version = v2.0
---
mv /etc/nova/nova.conf /etc/nova/nova.conf_Ubuntu
vi /etc/nova/nova.conf
wget https://gist.github.com/tmartinx/6214770/raw/e9d55c0dd093d885ba407edd8812c4435e6c9519/nova.conf
chown nova: /etc/nova/nova.conf
chmod 640 /etc/nova/nova.conf
cd /etc/init.d/; for i in $(ls nova-*); do sudo service $i restart; done
--- Quantum (still on compute1.yourdomain.com) ---
vi /etc/quantum/quantum.conf
---
debug = True
verbose = True
allow_overlapping_ips = True
rabbit_host = controller.yourdomain.com
[keystone_authtoken]
auth_host = controller.yourdomain.com
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = quantum
admin_password = service_pass
signing_dir = /var/lib/quantum/keystone-signing
---
vi /etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini
---
[DATABASE]
sql_connection = mysql://quantumUser:quantumPass@controller.yourdomain.com/quantum
[OVS]
network_vlan_ranges = physnet1
bridge_mappings = physnet1:br-eth0
---
service quantum-plugin-openvswitch-agent restart
# Done!
Point mycloud.yourdomain.com to 10.32.14.232 and open the Dashboard at:
http://mycloud.yourdomain.com/horizon - user admin, pass admin_pass
Of course, if you have IPv6 enabled on controller.yourdomain.com eth0,
point mycloud.yourdomain.com to your IPv6 address and enjoy it!
Congrats!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.