Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Vagrant file for setting up a single-node Kubernetes cluster that I can access from my desktop. Read more: https://medium.com/@lizrice/kubernetes-in-vagrant-with-kubeadm-21979ded6c63
# -*- mode: ruby -*-
# vi: set ft=ruby :
# This script to install Kubernetes will get executed after we have provisioned the box
$script = <<-SCRIPT
# Install kubernetes
apt-get update && apt-get install -y apt-transport-https
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF
apt-get update
apt-get install -y kubelet kubeadm kubectl
# kubelet requires swap off
swapoff -a
# keep swap off after reboot
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
# Get the IP address that VirtualBox has given this VM
IPADDR=`ip -4 address show dev eth1 | grep inet | awk '{print $2}' | cut -f1 -d/`
echo This VM has IP address $IPADDR
# Writing the IP address to a file in the shared folder
echo $IPADDR > /vagrant/ip-address.txt
# Set up Kubernetes
NODENAME=$(hostname -s)
kubeadm init --apiserver-cert-extra-sans=$IPADDR --node-name $NODENAME
# Set up admin creds for the vagrant user
echo Copying credentials to /home/vagrant...
sudo --user=vagrant mkdir -p /home/vagrant/.kube
cp -i /etc/kubernetes/admin.conf /home/vagrant/.kube/config
chown $(id -u vagrant):$(id -g vagrant) /home/vagrant/.kube/config
SCRIPT
Vagrant.configure("2") do |config|
config.vm.provider "virtualbox" do |v|
v.memory = 16384
v.cpus = 2
end
# Specify your hostname if you like
# config.vm.hostname = "name"
config.vm.box = "bento/ubuntu-20.04"
config.vm.network "private_network", type: "dhcp"
config.vm.provision "docker"
# Specify the shared folder mounted from the host if you like
# By default you get "." synced as "/vagrant"
# config.vm.synced_folder ".", "/folder"
config.vm.provision "shell", inline: $script
end
@lizrice

This comment has been minimized.

Copy link
Owner Author

@lizrice lizrice commented Apr 2, 2018

  • Copy out the admin credentials from /etc/kubernetes/admin.conf to your Mac so you can run kubectl from there. You will need to modify the server IP address to the one output during the Vagrant installation process. Make your kubectl commands refer to this config file with export KUBECONFIG=./admin.conf
  • Add pod networking add-on with e.g. kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
  • Allow pods to run on this master node: kubectl taint nodes --all node-role.kubernetes.io/master-
@a7xce

This comment has been minimized.

Copy link

@a7xce a7xce commented Jun 14, 2018

I am getting errors on this for the sed commands . . . can you provide the requirements and versioning ?

@lizrice

This comment has been minimized.

Copy link
Owner Author

@lizrice lizrice commented Jul 31, 2018

I just upgraded vagrant to 2.1.2 and virtualbox to v5.2.16. I haven't investigated why, but it looks like the relevant network interface is now showing as eth1 so the line that gets IPADDR needs to be

IPADDR=`ifconfig eth1 | grep Mask | awk '{print $2}'| cut -f2 -d:`
@uMtMu

This comment has been minimized.

Copy link

@uMtMu uMtMu commented Sep 14, 2018

Hi Liz,

I gues i have a problem about vagrant/virtualbox. I set master ip with

sudo kubeadm init --pod-network-cidr=172.168.20.0/24 --apiserver-advertise-address=172.168.20.101

like this my worker nodes can join the cluster but the send that 10.0.2.15 that i guess because of default gateway setting. Are there any parameter or setting to set worker's kubernetes interface?

@niya3

This comment has been minimized.

Copy link

@niya3 niya3 commented Oct 30, 2018

I am getting errors on this for the sed commands . . . can you provide the requirements and versioning ?

Try to add slash to escape new-line symbol in the end of sed command:

sed -i '0,/ExecStart=/s//Environment="KUBELET_EXTRA_ARGS=--cgroup-driver=cgroupfs"\n&/'
->
sed -i '0,/ExecStart=/s//Environment="KUBELET_EXTRA_ARGS=--cgroup-driver=cgroupfs"\\n&/'
@ekandreas

This comment has been minimized.

Copy link

@ekandreas ekandreas commented Dec 16, 2018

The new version of preflight requires two cpu's. Add this to the config section:

  config.vm.provider "virtualbox" do |v|
    v.cpus = 2
  end
@smoyer64

This comment has been minimized.

Copy link

@smoyer64 smoyer64 commented Feb 25, 2019

@lizrice Thanks so much for the great article and this Gist ... I've played around with this in the past but never got all the steps right.

One small issue I had was that my terminal buffer was cleared when I exited after the vagrant ssh and I lost the IP address I needed to change in admin.conf. It would be great to also echo the IP address out to a file by following line 26 with something like:

echo $IPADDR > ip-address.txt
@lizrice

This comment has been minimized.

Copy link
Owner Author

@lizrice lizrice commented Mar 25, 2019

echo $IPADDR > ip-address.txt

@smoyer64 that's a good idea, I added it. Thanks!

@lizrice

This comment has been minimized.

Copy link
Owner Author

@lizrice lizrice commented May 22, 2020

Now using ip instead of ifconfig, and using systemd rather than cgroupfs driver

@kodefoundry

This comment has been minimized.

Copy link

@kodefoundry kodefoundry commented Dec 9, 2020

Hi @lizrice,
I am seeing one issue,
Steps,
I do a vagrant halt, to shut down the VM
then do a 'vagrant up' again

After this 'kubectl' command does not work. I see the below error
The connection to the server <IP noted from $IPADDR>:6443 was refused - did you specify the right host or port?

Any clue??

@lizrice

This comment has been minimized.

Copy link
Owner Author

@lizrice lizrice commented Dec 11, 2020

@kodefoundry I have seen a few times that the swapoff doesn't "stick" between reboots (I don't know why, I'm afraid). Try sudo swapoff -a on the virtual machine. Wait a few seconds and then ps -eaf | grep kube on the virtual machine should show the running kubernetes components.

@saschagrunert

This comment has been minimized.

Copy link

@saschagrunert saschagrunert commented Jul 13, 2021

A good way to retrieve the IP independently of the interface could be: ip route get 1.2.3.4 | cut -d ' ' -f7 | tr -d '[:space:]'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment