Skip to content

Instantly share code, notes, and snippets.

@noonien
Last active September 16, 2015 11:49
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save noonien/86287f19b35205290cbd to your computer and use it in GitHub Desktop.
Save noonien/86287f19b35205290cbd to your computer and use it in GitHub Desktop.
k8s v0.15.0 on CoreOS
#cloud-config
hostname: node-01
ssh_authorized_keys:
- ssh-rsa <redacted>
coreos:
etcd2:
discovery: https://discovery.etcd.io/<redacted>
advertise-client-urls: http://node-01.mux.lan:2379
initial-advertise-peer-urls: http://node-01.mux.lan:2380
listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001
listen-peer-urls: http://node-01.mux.lan:2380,http://node-01.mux.lan:7001
fleet:
metadata: "name=node-01,role=kubernetes"
flannel:
interface: 10.0.0.2
units:
- name: 00-enp1s0.network
runtime: true
content: |
[Match]
Name=enp1s0
[Network]
Address=10.0.0.2/16
Gateway=10.0.0.1
DNS=10.0.0.1
Domains=mux.lan
- name: etcd2.service
command: start
- name: fleet.service
command: start
- name: flanneld.service
command: start
drop-ins:
- name: 50-network-config.conf
content: |
[Unit]
Requires=etcd2.service
[Service]
ExecStartPre=/usr/bin/etcdctl set /coreos.com/network/config '{"Network":"10.10.0.0/16", "Backend": {"Type": "host-gw"}}'
- name: setup-network-environment.service
command: start
content: |
[Unit]
Description=Setup Network Environment
Documentation=https://github.com/kelseyhightower/setup-network-environment
Requires=network-online.target
After=network-online.target
[Service]
ExecStartPre=-/usr/bin/mkdir -p /opt/bin
ExecStartPre=/usr/bin/wget -N -P /opt/bin https://github.com/kelseyhightower/setup-network-environment/releases/download/v1.0.0/setup-network-environment
ExecStartPre=/usr/bin/chmod +x /opt/bin/setup-network-environment
ExecStart=/opt/bin/setup-network-environment
RemainAfterExit=yes
Type=oneshot
update:
group: alpha
reboot-strategy: off
#cloud-config
hostname: node-02
ssh_authorized_keys:
- ssh-rsa <redacted>
coreos:
etcd2:
discovery: https://discovery.etcd.io/<redacted>
advertise-client-urls: http://node-02.mux.lan:2379
initial-advertise-peer-urls: http://node-02.mux.lan:2380
listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001
listen-peer-urls: http://node-02.mux.lan:2380,http://node-02.mux.lan:7001
fleet:
metadata: "name=node-02,role=kubernetes"
flannel:
interface: 10.0.0.3
units:
- name: 00-enp1s0.network
runtime: true
content: |
[Match]
Name=enp1s0
[Network]
Address=10.0.0.3/16
Gateway=10.0.0.1
DNS=10.0.0.1
Domains=mux.lan
- name: etcd2.service
command: start
- name: fleet.service
command: start
- name: flanneld.service
command: start
drop-ins:
- name: 50-network-config.conf
content: |
[Unit]
Requires=etcd2.service
[Service]
ExecStartPre=/usr/bin/etcdctl set /coreos.com/network/config '{"Network":"10.10.0.0/16", "Backend": {"Type": "host-gw"}}'
- name: setup-network-environment.service
command: start
content: |
[Unit]
Description=Setup Network Environment
Documentation=https://github.com/kelseyhightower/setup-network-environment
Requires=network-online.target
After=network-online.target
[Service]
ExecStartPre=-/usr/bin/mkdir -p /opt/bin
ExecStartPre=/usr/bin/wget -N -P /opt/bin https://github.com/kelseyhightower/setup-network-environment/releases/download/v1.0.0/setup-network-environment
ExecStartPre=/usr/bin/chmod +x /opt/bin/setup-network-environment
ExecStart=/opt/bin/setup-network-environment
RemainAfterExit=yes
Type=oneshot
update:
group: alpha
reboot-strategy: off
# fleetctl list-machines
MACHINE IP METADATA
5845c818... 10.0.0.2 name=node-01
71e7fd75... 10.0.0.3 name=node-02
# fleetctl list-units
UNIT MACHINE ACTIVE SUB
# fleetctl start *.service
Triggered global unit kube-kubelet.service start
2015/04/26 15:32:55 WARN fleetctl.go:758: Error retrieving Unit(kube-controller-manager.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-controller-manager.service?alt=json: forwarding request denied
2015/04/26 15:32:55 WARN fleetctl.go:758: Error retrieving Unit(kube-proxy.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-proxy.service?alt=json: forwarding request denied
2015/04/26 15:32:55 WARN fleetctl.go:758: Error retrieving Unit(kube-scheduler.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-scheduler.service?alt=json: forwarding request denied
2015/04/26 15:32:56 WARN fleetctl.go:758: Error retrieving Unit(kube-proxy.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-proxy.service?alt=json: forwarding request denied
2015/04/26 15:32:56 WARN fleetctl.go:758: Error retrieving Unit(kube-scheduler.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-scheduler.service?alt=json: forwarding request denied
2015/04/26 15:32:56 WARN fleetctl.go:758: Error retrieving Unit(kube-apiserver.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-apiserver.service?alt=json: forwarding request denied
2015/04/26 15:32:56 WARN fleetctl.go:758: Error retrieving Unit(kube-scheduler.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-scheduler.service?alt=json: forwarding request denied
2015/04/26 15:32:56 WARN fleetctl.go:758: Error retrieving Unit(kube-apiserver.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-apiserver.service?alt=json: forwarding request denied
2015/04/26 15:32:56 WARN fleetctl.go:758: Error retrieving Unit(kube-controller-manager.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-controller-manager.service?alt=json: forwarding request denied
2015/04/26 15:32:57 WARN fleetctl.go:758: Error retrieving Unit(kube-apiserver.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-apiserver.service?alt=json: ssh: rejected: administratively prohibited (open failed)
2015/04/26 15:32:57 WARN fleetctl.go:758: Error retrieving Unit(kube-proxy.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-proxy.service?alt=json: ssh: rejected: administratively prohibited (open failed)
2015/04/26 15:32:57 WARN fleetctl.go:758: Error retrieving Unit(kube-scheduler.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-scheduler.service?alt=json: ssh: rejected: administratively prohibited (open failed)
2015/04/26 15:32:57 WARN fleetctl.go:758: Error retrieving Unit(kube-proxy.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-proxy.service?alt=json: ssh: rejected: administratively prohibited (open failed)
2015/04/26 15:32:57 WARN fleetctl.go:758: Error retrieving Unit(kube-controller-manager.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-controller-manager.service?alt=json: ssh: rejected: administratively prohibited (open failed)
2015/04/26 15:32:58 WARN fleetctl.go:758: Error retrieving Unit(kube-proxy.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-proxy.service?alt=json: ssh: rejected: administratively prohibited (open failed)
2015/04/26 15:32:58 WARN fleetctl.go:758: Error retrieving Unit(kube-controller-manager.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-controller-manager.service?alt=json: ssh: rejected: administratively prohibited (open failed)
2015/04/26 15:32:58 WARN fleetctl.go:758: Error retrieving Unit(kube-apiserver.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-apiserver.service?alt=json: ssh: rejected: administratively prohibited (open failed)
2015/04/26 15:32:58 WARN fleetctl.go:758: Error retrieving Unit(kube-controller-manager.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-controller-manager.service?alt=json: ssh: rejected: administratively prohibited (open failed)
2015/04/26 15:32:58 WARN fleetctl.go:758: Error retrieving Unit(kube-apiserver.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-apiserver.service?alt=json: ssh: rejected: administratively prohibited (open failed)
2015/04/26 15:32:58 WARN fleetctl.go:758: Error retrieving Unit(kube-scheduler.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-scheduler.service?alt=json: ssh: rejected: administratively prohibited (open failed)
Unit kube-proxy.service launched on 5845c818.../10.0.0.2
2015/04/26 15:32:59 WARN fleetctl.go:758: Error retrieving Unit(kube-apiserver.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-apiserver.service?alt=json: ssh: rejected: administratively prohibited (open failed)
2015/04/26 15:32:59 WARN fleetctl.go:758: Error retrieving Unit(kube-scheduler.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-scheduler.service?alt=json: ssh: rejected: administratively prohibited (open failed)
2015/04/26 15:32:59 WARN fleetctl.go:758: Error retrieving Unit(kube-scheduler.service) from Registry: Get http://domain-sock/fleet/v1/units/kube-scheduler.service?alt=json: ssh: rejected: administratively prohibited (open failed)
Unit kube-apiserver.service launched on 5845c818.../10.0.0.2
Unit kube-controller-manager.service launched on 5845c818.../10.0.0.2
Unit kube-scheduler.service launched on 5845c818.../10.0.0.2
# fleetctl list-units
UNIT MACHINE ACTIVE SUB
kube-apiserver.service 5845c818.../10.0.0.2 active running
kube-controller-manager.service 5845c818.../10.0.0.2 active running
kube-kubelet.service 5845c818.../10.0.0.2 active running
kube-kubelet.service 71e7fd75.../10.0.0.3 active running
kube-proxy.service 5845c818.../10.0.0.2 active running
kube-scheduler.service 5845c818.../10.0.0.2 active running
# kubectl -s http://node-01.mux.lan:18080 version
Client Version: version.Info{Major:"0", Minor:"15", GitVersion:"v0.15.0", GitCommit:"831f3e60d7cd64c61a775d6c78acce1673dd8aa9", GitTreeState:"clean"}
Server Version: version.Info{Major:"0", Minor:"15", GitVersion:"v0.15.0", GitCommit:"831f3e60d7cd64c61a775d6c78acce1673dd8aa9", GitTreeState:"clean"}
# kubectl -s http://node-01.mux.lan:18080 get nodes
NAME LABELS STATUS
# fleetctl ssh 584 ping -c 1 node-01
PING node-01.mux.lan (10.0.0.2) 56(84) bytes of data.
64 bytes from node-01.mux.lan (10.0.0.2): icmp_seq=1 ttl=64 time=0.091 ms
# fleetctl ssh 584 ping -c 1 node-02
PING node-02.mux.lan (10.0.0.3) 56(84) bytes of data.
64 bytes from node-02.mux.lan (10.0.0.3): icmp_seq=1 ttl=64 time=0.228 ms
# fleetctl ssh 71e ping -c 1 node-01
PING node-01.mux.lan (10.0.0.2) 56(84) bytes of data.
64 bytes from node-01.mux.lan (10.0.0.2): icmp_seq=1 ttl=64 time=0.198 ms
# fleetctl ssh 71e ping -c 1 node-02
PING node-02.mux.lan (10.0.0.3) 56(84) bytes of data.
64 bytes from node-02.mux.lan (10.0.0.3): icmp_seq=1 ttl=64 time=0.043 ms
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
Requires=etcd2.service
After=etcd2.service
[Service]
ExecStartPre=/usr/bin/mkdir -p /opt/bin
ExecStartPre=/usr/bin/wget -N -P /opt/bin https://storage.googleapis.com/kubernetes-release/release/v0.15.0/bin/linux/amd64/kube-apiserver
ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-apiserver
ExecStart=/opt/bin/kube-apiserver \
--v=2 \
--address=0.0.0.0 \
--port=18080 \
--portal_net=10.10.0.0/16 \
--etcd_servers=http://127.0.0.1:4001 \
--logtostderr=true
Restart=always
RestartSec=10
[X-Fleet]
MachineMetadata=name=node-01
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
Requires=kube-apiserver.service
After=kube-apiserver.service
[Service]
ExecStartPre=/usr/bin/mkdir -p /opt/bin
ExecStartPre=/usr/bin/wget -N -P /opt/bin https://storage.googleapis.com/kubernetes-release/release/v0.15.0/bin/linux/amd64/kube-controller-manager
ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-controller-manager
ExecStart=/opt/bin/kube-controller-manager \
--v=2 \
--master=http://127.0.0.1:18080 \
--logtostderr=true
Restart=always
RestartSec=10
[X-Fleet]
MachineOf=kube-apiserver.service
[Unit]
Description=Kubernetes Kubelet
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
Requires=etcd2.service
After=etcd2.service
[Service]
EnvironmentFile=/etc/network-environment
ExecStartPre=/usr/bin/mkdir -p /opt/bin
ExecStartPre=/usr/bin/wget -N -P /opt/bin https://storage.googleapis.com/kubernetes-release/release/v0.15.0/bin/linux/amd64/kubelet
ExecStartPre=/usr/bin/chmod +x /opt/bin/kubelet
ExecStart=/opt/bin/kubelet \
--v=2 \
--address=0.0.0.0 \
--port=10250 \
--api_servers=http://node-01.mux.lan:18080 \
--hostname_override=${DEFAULT_IPV4} \
--healthz_bind_address=0.0.0.0 \
--healthz_port=10248 \
--logtostderr=true
Restart=always
RestartSec=10
[X-Fleet]
Global=true
MachineMetadata=role=kubernetes
[Unit]
Description=Kubernetes Proxy
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
Requires=etcd2.service
After=etcd2.service
[Service]
ExecStartPre=/usr/bin/mkdir -p /opt/bin
ExecStartPre=/usr/bin/wget -N -P /opt/bin https://storage.googleapis.com/kubernetes-release/release/v0.15.0/bin/linux/amd64/kube-proxy
ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-proxy
ExecStart=/opt/bin/kube-proxy \
--v=2 \
--master=http://node-01.mux.lan:18080 \
--logtostderr=true
Restart=always
RestartSec=10
[X-Fleet]
Global=true
MachineMetadata=role=kubernetes
[Unit]
Description=Kubernetes Registration Service
Documentation=https://github.com/kelseyhightower/kube-register
[Service]
ExecStartPre=/usr/bin/mkdir -p /opt/bin
ExecStartPre=/usr/bin/wget -N -O /opt/bin/kube-register https://github.com/kelseyhightower/kube-register/releases/download/v0.0.3/kube-register-0.0.3-linux-amd64
ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-register
ExecStart=/opt/bin/kube-register \
--metadata=role=kubernetes \
--fleet-endpoint=unix:///var/run/fleet.sock \
--healthz-port=10248 \
--api-endpoint=http://node-01.mux.lan:18080
Restart=always
RestartSec=10
[X-Fleet]
MachineOf=kube-apiserver.service
[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
Requires=kube-apiserver.service
After=kube-apiserver.service
[Service]
ExecStartPre=/usr/bin/mkdir -p /opt/bin
ExecStartPre=/usr/bin/wget -N -P /opt/bin https://storage.googleapis.com/kubernetes-release/release/v0.11.0/bin/linux/amd64/kube-scheduler
ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-scheduler
ExecStart=/opt/bin/kube-scheduler --v=2 --master=http://127.0.0.1:18080
Restart=always
RestartSec=10
[X-Fleet]
MachineOf=kube-apiserver.service
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment