Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Ubuntu on VMware with LXD containers

Paraphrased/edited/slightly updated form of: http://mike.teczno.com/notes/disposable-virtualbox-lxc-environments.html

VM Setup

Download an Ubuntu Server ISO (http://www.ubuntu.com/download/server)

Create a new VMWare virtual machine, select boot from the ISO with 2 NICs configured as "Share with My Mac" & "Private to my Mac" respectively. During the creation of the Ubuntu VM there will be a page to select installed software, select "OpenSSH server".

VM Networking

Log in to the newly created VM and install bridge-utils:

apt install bridge-utils

Edit /etc/network/interfaces with a bridge each for the 2 network cards, both DHCP (ip link show will give you the correct names for these interfaces).

## The primary network interface
#auto ens33
#iface ens33 inet dhcp

auto br0
iface br0 inet dhcp
        bridge_ports ens33
        bridge_fd 0
        bridge_maxwait 0
        dns-search home

auto br1
iface br1 inet dhcp
        bridge_ports ens38
        bridge_fd 0
        bridge_maxwait 0
        dns-search local

Restart sudo reboot, and you should now be able to ping the outside world.

$ ping 8.8.8.8

PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_req=1 ttl=63 time=340 ms
…

Install open-vm-tools:

$ apt install open-vm-tools

Avahi on VM

Next, set up Avahi to broadcast host names so we don’t need to remember DHCP-assigned IP addresses. On the Ubuntu host, install avahi-daemon:

$ apt install avahi-daemon

In the configuration file /etc/avahi/avahi-daemon.conf, change these lines to clarify that our host names need only work on the second, host-only network adapter:

allow-interfaces=br1,ens38
deny-interfaces=br0,ens33,lxdbr0

Then restart Avahi:

$ sudo service avahi-daemon restart

Now, you should be able to ping and ssh to $hostname from within the virtual machine and your Mac command line.

LXD server (Ubuntu VM)

sudo add-apt-repository ppa:ubuntu-lxc/lxd-stable
sudo apt update
sudo apt upgrade

See: https://linuxcontainers.org/lxd/getting-started-cli/

$ sudo apt install zfsutils-linux
$ sudo lxd init

Generate a new LXD profile from the script here: https://gist.github.com/earnubs/7dffc5bb5fe613d02ef9fc924cc583ee

Test out lxc launch:

$ lxc launch ubuntu:16.10 --ephemeral -p $USER
$ lxc list
+--------------+---------+--------------------------------+------+-----------+-----------+
|     NAME     |  STATE  |              IPV4              | IPV6 |   TYPE    | SNAPSHOTS |
+--------------+---------+--------------------------------+------+-----------+-----------+
| enabling-ape | RUNNING | 192.168.234.179 (eth0)         |      | EPHEMERAL | 0         |
|              |         | 172.16.183.150 (eth1)          |      |           |           |
+--------------+---------+--------------------------------+------+-----------+-----------+

NB. It may take a few minutes for cloud-init to configure and restart eth1.

Allow remote operations on the LXD server (from macOS)

lxc config set core.https_address "[::]"
lxc config set core.trust_password some-password

LXD client (macOS)

With a working Go setup:

$ go get github.com/lxc/lxd
$ cd $GOPATH/src/github.com/lxc/lxd
$ go install -v ./lxc
$ lxc remote add <name> UBUNTU_VM_HOST.local

Finally, on macOS

$ ssh enabling-ape.local -A
@earnubs

This comment has been minimized.

Copy link
Owner Author

earnubs commented Jan 11, 2017

ufw complaining about ip6tables:

ERROR: initcaps
[Errno 2] modprobe: ERROR: ../libkmod/libkmod.c:556 kmod_search_moddep() could not open moddep file '/lib/modules/4.4.0-59-generic/modules.dep.bin'
ip6tables v1.4.21: can't initialize ip6tables table `filter': Table does not exist (do you need to insmod?)
Perhaps ip6tables or your kernel needs to be upgraded.

Either http://askubuntu.com/questions/664668/ufw-not-working-in-an-lxc-container or, in your host:

sudo modprobe ip6_tables
@vilagithub

This comment has been minimized.

Copy link

vilagithub commented Jan 11, 2017

sudo modprobe ip_tables
sudo modprobe ip6_tables
@earnubs

This comment has been minimized.

Copy link
Owner Author

earnubs commented Jan 11, 2017

ufw complaining about iptables:

ERROR: initcaps
[Errno 2] modprobe: ERROR: ../libkmod/libkmod.c:556 kmod_search_moddep() could not open moddep file '/lib/modules/4.4.0-59-generic/modules.dep.bin'
iptables v1.4.21: can't initialize iptables table `filter': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.

On the host:

sudo modprobe ip_tables
@earnubs

This comment has been minimized.

Copy link
Owner Author

earnubs commented Jan 17, 2017

From https://www.stgraber.org/2017/01/13/kubernetes-inside-lxd/:

lxc launch ubuntu:16.04 kubernetes -c security.privileged=true -c security.nesting=true -c linux.kernel_modules=ip_tables,ip6_tables,netlink_diag,nf_nat,overlay -c raw.lxc=lxc.aa_profile=unconfined
@vilagithub

This comment has been minimized.

Copy link

vilagithub commented Jan 17, 2017

also, lxd -c linux.kernel_modules=ip_tables,ip6_tables may be a simpler (and more robust) alternative

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.