Skip to content

Instantly share code, notes, and snippets.

@yuanying

yuanying/create-dev-env-magnum.md

Last active March 11, 2019 09:16
Show Gist options
  • Save yuanying/f27cef22738acf8b1db5 to your computer and use it in GitHub Desktop.
Save yuanying/f27cef22738acf8b1db5 to your computer and use it in GitHub Desktop.
Download ZIP
Develop Magnum with Devstack
Raw
create-dev-env-magnum.md

Develop Magnum with Devstack

Vagrant

I'm using vagrant (parallels on Mac) to boot a devstack. Vagrantfile is below. Devstack IP Address is 192.168.11.197.

Vagrant.configure('2') do |config|
  config.vm.box = "trusty64"

  config.vm.define :devstack do |devstack|
    devstack.vm.hostname = "devstack"
    devstack.vm.network :private_network, ip: "192.168.123.10"
    devstack.vm.network :public_network, dev: 'br0', mode: 'bridge', ip: "192.168.11.197"

    devstack.vm.synced_folder ".", "/vagrant", type: "nfs"
    #devstack.vm.synced_folder "/home/yuanying/Projects", "/home/yuanying/Projects", type: "nfs"

    devstack.vm.provider :libvirt do |libvirt, override|
      libvirt.memory = 8192
      libvirt.nested = true
    end

    devstack.vm.provision "shell", path: "./install.sh"
  end

end

and vagrant up.

Inside Devstack VM

Install requirement packages

$ sudo apt-get update
$ sudo apt-get install libffi-dev libssl-dev git vim \
                       libxml2-dev libsqlite3-dev libxslt1-dev -y

Network settings

sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

Install DevStack

$ cd ~
$ sudo mkdir - p /etc/neutron
$ sudo chown -R $USER /etc/neutron
$ echo "dhcp-option-force=26,1400" >> /etc/neutron/dnsmasq.conf
$ git clone https://git.openstack.org/openstack-dev/devstack
$ cd devstack
$ ./stack.sh

local.conf is below.

[[local|localrc]]
HOST_IP=192.168.11.197
#SERVICE_HOST=192.168.202.4
#HEAT_API_HOST=${SERVICE_HOST}
#HEAT_API_CFN_HOST=${SERVICE_HOST}
#HEAT_ENGINE_HOST=${SERVICE_HOST}
#HEAT_API_CW_HOST=${SERVICE_HOST}

FLOATING_RANGE=172.16.12.0/24
Q_FLOATING_ALLOCATION_POOL="start=172.16.12.10,end=172.16.12.200"
PUBLIC_NETWORK_GATEWAY=172.16.12.1

Q_USE_SECGROUP=True
ENABLE_TENANT_VLANS=True
TENANT_VLAN_RANGE=1000:1999
PHYSICAL_NETWORK=default
OVS_PHYSICAL_BRIDGE=br-ex

NETWORK_GATEWAY=10.11.12.1
FIXED_RANGE=10.11.12.0/24
FIXED_NETWORK_SIZE=256

ADMIN_PASSWORD=openstack
MYSQL_PASSWORD=stackdb
RABBIT_PASSWORD=stackqueue
SERVICE_PASSWORD=$ADMIN_PASSWORD
SERVICE_TOKEN=tokentoken

disable_service n-net
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service q-lbaas
enable_service neutron

enable_service h-eng
enable_service h-api
enable_service h-api-cfn
enable_service h-api-cw

enable_plugin barbican https://git.openstack.org/openstack/barbican

#LOGFILE=$DEST/logs/devstack.log
DEST=/opt/stack
#SCREEN_LOGDIR=$DEST/logs/screen

[[post-config|/etc/neutron/dhcp_agent.ini]]
[DEFAULT]
dnsmasq_config_file = /etc/neutron/dnsmasq.conf

Magnum

Install

Magnum is outside of devstack. In this case, magnum will install to 192.168.11.132 host.

Magnum Server

$ cd ~
$ git clone https://github.com/openstack/magnum.git
$ cd magnum
$ tox -evenv -- echo 'done'

Magnum Client

$ cd ~
$ git clone https://github.com/openstack/python-magnumclient.git
$ cd python-magnumclient
$ tox -evenv -- echo 'done'

Configuration

Setup trust

TRUSTEE_DOMAIN_ID=$(
    openstack domain create magnum \
        --description "Owns users and projects created by magnum" \
        -f value -c id
)
TRUSTEE_DOMAIN_ADMIN_ID=$(
    openstack user create trustee_domain_admin \
        --password "password" \
        --domain=${TRUSTEE_DOMAIN_ID} \
        --or-show \
        -f value -c id
)
openstack --os-identity-api-version 3 role add \
          --user $TRUSTEE_DOMAIN_ADMIN_ID --domain $TRUSTEE_DOMAIN_ID \
          admin

Create config

$ sudo mkdir -p /etc/magnum
$ cd /etc/magnum
$ sudo vim magnum.conf

magnum.conf has below content. Change 192.168.11.197 to your devstack IP address.

[DEFAULT]
debug = True
verbose = True

rabbit_userid=stackrabbit
rabbit_password = stackqueue
rabbit_hosts = 192.168.11.197
rpc_backend = rabbit

[database]
connection = mysql://root:stackdb@192.168.11.197/magnum

[keystone_authtoken]
admin_password = openstack
admin_user = nova
admin_tenant_name = service
identity_uri = http://192.168.11.197:35357
#user_domain_id = default
#project_domain_id = default

auth_uri=http://192.168.11.197:5000/v3

[api]

host = 0.0.0.0

[trust]
#trustee_domain_id = magnum
#trustee_domain_admin_id = trustee_domain_admin
trustee_domain_admin_password = password

Update trust config

# set trustee domain id
sudo sed -i "s/#trustee_domain_id\s*=.*/trustee_domain_id=${TRUSTEE_DOMAIN_ID}/" \
         /etc/magnum/magnum.conf

# set trustee domain admin id
sudo sed -i "s/#trustee_domain_admin_id\s*=.*/trustee_domain_admin_id=${TRUSTEE_DOMAIN_ADMIN_ID}/" \
         /etc/magnum/magnum.conf

# set trustee domain admin password
sudo sed -i "s/#trustee_domain_admin_password\s*=.*/trustee_domain_admin_password=password/" \
         /etc/magnum/magnum.conf

# set correct region name to clients
sudo sed -i "s/#region_name\s*=.*/region_name=RegionOne/" \
         /etc/magnum/magnum.conf

register magnum service to keystone

$ source ~/devstack/openrc admin admin
$ openstack service create --name=magnum \
                           --description="Magnum Container Service" \
                           container
$ openstack endpoint create --region=RegionOne \
                            magnum public http://192.168.11.132:9511/v1
$ openstack endpoint create --region=RegionOne \
                            magnum internal http://192.168.11.132:9511/v1
$ openstack endpoint create --region=RegionOne \
                            magnum admin http://192.168.11.132:9511/v1

Register Image to glance

$ curl -O https://fedorapeople.org/groups/magnum/fedora-21-atomic-5.qcow2
$ source ~/devstack/openrc admin admin
$ glance image-create --name fedora-21-atomic-5 \
                    --visibility public \
                    --disk-format qcow2 \
                    --os-distro fedora-atomic \
                    --container-format bare < fedora-21-atomic-5.qcow2

Add default keypair to demo user

$ ssh-keygen
$ source ~/devstack/openrc demo demo
$ nova keypair-add --pub-key ~/.ssh/id_rsa.pub default

Database

$ mysql -h 192.168.11.197 -u root -pstackdb mysql <<EOF
CREATE DATABASE IF NOT EXISTS magnum DEFAULT CHARACTER SET utf8;
GRANT ALL PRIVILEGES ON magnum.* TO
    'root'@'%' IDENTIFIED BY 'stackdb'
EOF

and create tables.

$ cd ~/magnum
$ source .tox/venv/bin/activate
$ pip install mysql-python
$ magnum-db-manage upgrade

Start Magnum

magnum-api

$ cd ~/magnum
$ source .tox/venv/bin/activate
$ magnum-api

magnum-conductor

$ cd ~/magnum
$ source .tox/venv/bin/activate
$ magnum-conductor

python-magnumclient

$ cd ~/python-magnumclient
$ source .tox/venv/bin/activate
$ magnum bay-list

Test magnum

Try to create bay

$ magnum baymodel-create --name kubernetes --keypair-id default \
                         --external-network-id public \
                         --image-id fedora-21-atomic-5 \
                         --flavor-id m1.small \
                         --docker-volume-size 1 \
                         --network-driver flannel \
                         --coe kubernetes

$ magnum bay-create --name k8s_bay --baymodel kubernetes

$ magnum baymodel-create --name swarm \
                         --image-id fedora-21-atomic-5 \
                         --keypair-id default \
                         --external-network-id public \
                         --flavor-id m1.small \
                         --docker-volume-size 1 \
                         --coe swarm

Try to create pod

$ magnum pod-create --bay-id 99cab72f-16a7-4564-8d73-d4497f51f557 \
    --pod-file redis-master.json

After reload

$ sudo ip addr add 10.0.0.1/24 dev br-ex
$ sudo ip addr add 172.16.12.1/24 dev br-ex
$ sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
$ sudo losetup /dev/loop0 /opt/stack/data/stack-volumes-default-backing-file ;
$ sudo losetup /dev/loop1 /opt/stack/data/stack-volumes-lvmdriver-1-backing-file ;
Raw
demo.md

Demo command list

nova list
neutron net-list

magnum baymodel-list

nova keypair-list

ssh-keygen

nova keypair-add --pub-key ~/.ssh/id_rsa.pub default

magnum baymodel-create \
  --name kubernetes \
  --keypair-id default \
  --external-network-id public \
  --image-id fedora-21-atomic-5 \
  --flavor-id m1.small \
  --docker-volume-size 1 \
  --network-driver flannel \
  --coe kubernetes

magnum bay-create --name k8sbay --baymodel kubernetes

heat stack-list

nova list

heat resource-list k8sbay-ap76ggow3rpc

https://github.com/openstack/magnum/blob/master/doc/source/dev/dev-tls.rst

openssl genrsa -out client.key 4096

cat > client.conf << END
[req]
distinguished_name = req_distinguished_name
req_extensions     = req_ext
prompt = no
[req_distinguished_name]
CN = yuanying@fraction.jp
[req_ext]
extendedKeyUsage = clientAuth
END

openssl req -new -days 365 \
    -config client.conf \
    -key client.key \
    -out client.csr

openssl req -text -in client.csr

magnum ca-sign --bay k8sbay --csr client.csr > client.crt

openssl x509 -text -in client.crt

magnum ca-show --bay k8sbay > ca.crt


KUBERNETES_URL=$(magnum bay-show k8sbay |
                 awk '/ api_address /{print $4}')

kubectl version --certificate-authority=ca.crt \
               --client-key=client.key \
               --client-certificate=client.crt -s $KUBERNETES_URL

kubectl config set-cluster secure-k8sbay --server=${KUBERNETES_URL} \
   --certificate-authority=${PWD}/ca.crt
kubectl config set-credentials client --certificate-authority=${PWD}/ca.crt \
   --client-key=${PWD}/client.key --client-certificate=${PWD}/client.crt
kubectl config set-context secure-k8sbay --cluster=secure-k8sbay --user=client
kubectl config use-context secure-k8sbay


cat > nginx.yml << END
apiVersion: v1
kind: Pod
metadata:
  name: nginx

spec:
  containers:
  - name: nginx
    image: nginx
    ports:
      - containerPort: 80
END

cat > nginx-service.yml << END
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  ports:
    - port: 80
  selector:
    app: nginx
END

kubectl create -f nginx.yml
@shinobu-x
Copy link

@yuanying,

Thank you for your great guide.
One question from my side.
Why we need to force mtu to 1400?

$ echo "dhcp-option-force=26,1400" >> /etc/neutron/dnsmasq.conf

I'm just curious about that.

Rgds,
Shinobu

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment