Juju is having problems creating trusty containers on wily.
Here's a status output from after an openstack installer run (not a deployer bundle):
ubuntu@openstack-single-ubuntu:~$ juju --version
1.25.0-wily-amd64
ubuntu@openstack-single-ubuntu:~$ juju status --format=tabular
[Services]
NAME STATUS EXPOSED CHARM
glance unknown false cs:trusty/glance-24
glance-simplestreams-sync unknown false cs:trusty/glance-simplestreams-sync-3
keystone unknown false cs:trusty/keystone-28
mysql unknown false cs:trusty/mysql-28
neutron-api unknown false cs:trusty/neutron-api-17
neutron-gateway unknown false cs:trusty/neutron-gateway-5
neutron-openvswitch false cs:trusty/neutron-openvswitch-9
nova-cloud-controller unknown false cs:trusty/nova-cloud-controller-60
nova-compute unknown false cs:trusty/nova-compute-26
ntp false cs:trusty/ntp-11
openstack-dashboard unknown false cs:trusty/openstack-dashboard-16
rabbitmq-server unknown false cs:trusty/rabbitmq-server-34
[Units]
ID WORKLOAD-STATE AGENT-STATE VERSION MACHINE PORTS PUBLIC-ADDRESS MESSAGE
glance-simplestreams-sync/0 unknown allocating 1/lxc/6 Waiting for agent initialization to finish
glance/0 unknown allocating 1/lxc/7 Waiting for agent initialization to finish
keystone/0 unknown allocating 1/lxc/1 Waiting for agent initialization to finish
mysql/0 unknown allocating 1/lxc/0 Waiting for agent initialization to finish
neutron-api/0 unknown allocating 1/lxc/5 Waiting for agent initialization to finish
neutron-gateway/0 unknown idle 1.25.0.1 3 10.0.6.54
nova-cloud-controller/0 unknown allocating 1/lxc/3 Waiting for agent initialization to finish
nova-compute/0 unknown idle 1.25.0.1 2 10.0.6.29
openstack-dashboard/0 unknown allocating 1/lxc/4 Waiting for agent initialization to finish
rabbitmq-server/0 unknown allocating 1/lxc/2 Waiting for agent initialization to finish
[Machines]
ID STATE VERSION DNS INS-ID SERIES HARDWARE
0 started 1.25.0.1 localhost localhost wily
1 started 1.25.0.1 10.0.6.216 ubuntu-local-machine-1 trusty arch=amd64 cpu-cores=2 mem=6144M root-disk=20480M
2 started 1.25.0.1 10.0.6.29 ubuntu-local-machine-2 trusty arch=amd64 cpu-cores=1 mem=4096M root-disk=40960M
3 started 1.25.0.1 10.0.6.54 ubuntu-local-machine-3 trusty arch=amd64 cpu-cores=1 mem=2048M root-disk=20480M
Note all the containers on machine 1 are stalled.
Here I poke around and look at groups and versions:
ubuntu@openstack-single-ubuntu:~$ juju ssh 1
ubuntu@ubuntu-local-machine-1:~$ groups
ubuntu adm dialout cdrom floppy sudo audio dip video plugdev netdev
ubuntu@ubuntu-local-machine-1:~$ sudo apt-cache policy lxc
lxc:
Installed: 1.0.7-0ubuntu0.9
Candidate: 1.0.7-0ubuntu0.9
Version table:
*** 1.0.7-0ubuntu0.9 0
500 http://archive.ubuntu.com/ubuntu/ trusty-updates/main amd64 Packages
100 /var/lib/dpkg/status
1.0.3-0ubuntu3 0
500 http://archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages
ubuntu@ubuntu-local-machine-1:~$ sudo apt-cache policy libvirt-bin
libvirt-bin:
Installed: (none)
Candidate: 1.2.2-0ubuntu13.1.14
Version table:
1.2.2-0ubuntu13.1.14 0
500 http://archive.ubuntu.com/ubuntu/ trusty-updates/main amd64 Packages
1.2.2-0ubuntu13 0
500 http://archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages
And check networking out from the VM:
ubuntu@ubuntu-local-machine-1:~$ ping google.com
PING google.com (173.194.204.102) 56(84) bytes of data.
64 bytes from qb-in-f102.1e100.net (173.194.204.102): icmp_seq=1 ttl=38 time=38.6 ms
64 bytes from qb-in-f102.1e100.net (173.194.204.102): icmp_seq=2 ttl=38 time=38.0 ms
64 bytes from qb-in-f102.1e100.net (173.194.204.102): icmp_seq=3 ttl=38 time=38.5 ms
^C
--- google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 38.010/38.407/38.617/0.323 ms
ubuntu@ubuntu-local-machine-1:~$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.3 LTS"
We see no containers have been created:
ubuntu@ubuntu-local-machine-1:~$ sudo lxc-ls -f
NAME STATE IPV4 IPV6 AUTOSTART
----------------------------------
Looking at the log:
ubuntu@ubuntu-local-machine-1:~$ sudo less /var/log/juju/machine-1.log
...
2015-11-02 19:19:33 INFO juju.container lock.go:50 acquire lock "juju-trusty-lxc-template", ensure clone exists
2015-11-02 19:19:33 INFO juju.container.lxc clonetemplate.go:77 template does not exist, creating
2015-11-02 19:19:33 DEBUG juju.service discovery.go:65 discovered init system "upstart" from series "trusty"
2015-11-02 19:19:34 DEBUG juju.container image.go:88 lxc image for trusty (amd64) is https://cloud-images.ubuntu.com/server/releases/trusty/release-20151019/ubuntu-14.04-server-cloudimg-amd64.tar.gz
2015-11-02 19:19:34 DEBUG juju.container.lxc lxc.go:425 creating lxc container "juju-trusty-lxc-template"
2015-11-02 19:19:34 DEBUG juju.container.lxc lxc.go:426 lxc-create template params: [--debug --userdata /var/lib/juju/containers/juju-trusty-lxc-template/cloud-init --hostid juju-trusty-lxc-template -r trusty -T https://10.0.3.241:17070/environment/eaca592e-186f-4a4d-85ab-a342663ffffd/images/lxc/trusty/amd64/ubuntu-14.04-server-cloudimg-amd64-root.tar.gz]
2015-11-02 19:19:34 ERROR juju.container.lxc clonetemplate.go:129 lxc template container creation failed: lxc container creation failed: juju-trusty-lxc-template
2015-11-02 19:19:34 INFO juju.container lock.go:66 release lock "juju-trusty-lxc-template"
...
OK, the template container create failed. So I retried the lxc-create with the same parameters as above, to see if I could get better debug info:
ubuntu@ubuntu-local-machine-1:~$ sudo lxc-create -t ubuntu-cloud -n mikestest -- --debug --userdata /var/lib/juju/containers/juju-trusty-lxc-template/cloud-init --hostid juju-trusty-lxc-template -r trusty -T https://10.0.3.241:17070/environment/eaca592e-186f-4a4d-85ab-a342663ffffd/images/lxc/trusty/amd64/ubuntu-14.04-server-cloudimg-amd64-root.tar.gz
+ '[' amd64 = i686 ']'
+ '[' 0 = 0 ']'
+ case "$hostarch:$arch" in
+ :
+ '[' tryreleased '!=' daily -a tryreleased '!=' released -a tryreleased '!=' tryreleased ']'
+ '[' -z /var/lib/lxc/mikestest ']'
++ id -u
+ '[' 0 '!=' 0 ']'
+ config=/var/lib/lxc/mikestest/config
+ '[' -z /var/lib/lxc/mikestest/rootfs ']'
+ type ubuntu-cloudimg-query
ubuntu-cloudimg-query is /usr/bin/ubuntu-cloudimg-query
+ type wget
wget is /usr/bin/wget
+ cache=/var/cache/lxc/cloud-trusty
+ '[' 0 -eq 1 ']'
+ mkdir -p /var/cache/lxc/cloud-trusty
+ '[' tryreleased = tryreleased ']'
+ stream=released
+ ubuntu-cloudimg-query trusty released amd64
+ '[' -n https://10.0.3.241:17070/environment/eaca592e-186f-4a4d-85ab-a342663ffffd/images/lxc/trusty/amd64/ubuntu-14.04-server-cloudimg-amd64-root.tar.gz ']'
+ url2=https://10.0.3.241:17070/environment/eaca592e-186f-4a4d-85ab-a342663ffffd/images/lxc/trusty/amd64/ubuntu-14.04-server-cloudimg-amd64-root.tar.gz
++ basename https://10.0.3.241:17070/environment/eaca592e-186f-4a4d-85ab-a342663ffffd/images/lxc/trusty/amd64/ubuntu-14.04-server-cloudimg-amd64-root.tar.gz
+ filename=ubuntu-14.04-server-cloudimg-amd64-root.tar.gz
+ '[' -n https://10.0.3.241:17070/environment/eaca592e-186f-4a4d-85ab-a342663ffffd/images/lxc/trusty/amd64/ubuntu-14.04-server-cloudimg-amd64-root.tar.gz ']'
+ do_extract_rootfs
+ cd /var/cache/lxc/cloud-trusty
+ '[' 0 -eq 1 ']'
+ trap wgetcleanup EXIT SIGHUP SIGINT SIGTERM
+ '[' '!' -f ubuntu-14.04-server-cloudimg-amd64-root.tar.gz ']'
+ wget https://10.0.3.241:17070/environment/eaca592e-186f-4a4d-85ab-a342663ffffd/images/lxc/trusty/amd64/ubuntu-14.04-server-cloudimg-amd64-root.tar.gz
--2015-11-02 20:06:02-- https://10.0.3.241:17070/environment/eaca592e-186f-4a4d-85ab-a342663ffffd/images/lxc/trusty/amd64/ubuntu-14.04-server-cloudimg-amd64-root.tar.gz
Connecting to 10.0.3.241:17070... connected.
ERROR: cannot verify 10.0.3.241's certificate, issued by ‘/O=juju/CN=juju-generated CA for environment "local"’:
Unable to locally verify the issuer's authority.
ERROR: no certificate subject alternative name matches
requested host name ‘10.0.3.241’.
To connect to 10.0.3.241 insecurely, use `--no-check-certificate'
... spurious cascading errors from the ubuntu-cloud lxc template are snipped here.
lxc_container: lxccontainer.c: create_run_template: 1125 container creation template for mikestest failed
lxc_container: lxc_create.c: main: 271 Error creating container mikestest
So, what's going on with the certificates for the state server?
Looks like the properly wrapped wget is also having cert issues: