coreos installation & usage

coreos_installation.md

##install coreos to the baremetal server

Prepare the cloud init configuration file like below:

#cloud-config

# include one or more SSH public keys
ssh_authorized_keys:
  - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDi9rg+WeNW2WkhBniXViHoMOJhEIK/JoIxMt6ig2Kksoot4V5oJwRs7odCVejHWaUYMhq8QsuwTejpt5172rtyQLLyd4QbwCct21+XHZul7lgjJR8EAhRBZDPHxg+c4d9gJYCQnxSxdBLYBpg2K8UQJY7jvHwpi2jjpAZQnQzJ5YYzpcyz+SjbONu8FZaqXxt7/s+O8U0OqIeCMJIeinu69xKwtCaW+ROe/kMPoMSrjcr66sRkcWVyrcPCjMI3nLoE6Ix+DB2UHK+mxnR8u5j0S3oX2AMPGZgQE/emd4Epa8KS3/Vh6I35De+RcT5+3HzJviYhxJLY7kLP3FyVsxQX ochinchina@ubuntu-1504-1

Then call the coreos-install script to install the coreos to disk

$ sudo coreos-install -d /dev/sda -c your-cloud-config.yml

The coreos-install script will download the image from the coreos website during the installation. If you want to install multiple coreos or your machine is behind a proxy, you can download the image from the coreos website and then setup a http server to install the coreos.

For example, if you want to install stable coreos 1010.5.0, you can download the file coreos_production_image.bin.bz2 and coreos_production_image.bin.bz2.sig from website https://stable.release.core-os.net/amd64-usr/1010.5.0/ to directory like ~/coreos/1010.5.0.

$ mkdir ~/coreos/1010.5.0
$ cd ~/coreos/1010.5.0
$ wget https://stable.release.core-os.net/amd64-usr/1010.5.0/coreos_production_image.bin.bz2
$ wget https://stable.release.core-os.net/amd64-usr/1010.5.0/coreos_production_image.bin.bz2.sig

Then start a simple static HTTP server like below:

$ cd ~
$ python -m SimpleHttpServer 8888

After the HTTP server is started, you can install the coreos like below:

$ sudo coreos-install -d /dev/sda -c your-cloud-config.yml -b http://your_http_server:8888/coreos

##set a etcd cluster

suppose if there are three node: 10.245.1.101,10.245.1.102,10.245.1.103.

In the node 10.245.1.101, edit file /usr/lib/coreos-install/user_data and put following conents to the file:

#cloud-config

coreos:
  etcd2:
    name: "infra0"
    initial-advertise-peer-urls: "http://10.245.1.101:2380"
    listen-peer-urls: "http://10.245.1.101:2380"
    listen-client-urls: "http://10.245.1.101:2379,http://127.0.0.1:2379"
    advertise-client-urls: "http://10.245.1.101:2379"
    initial-cluster-token: "etcd-cluster-1"
    initial-cluster: "infra0=http://10.245.1.101:2380,infra1=http://10.245.1.102:2380,infra2=http://10.245.1.103:2380"
    initial-cluster-state: "new"

in the node 10.245.1.102, the etcd2 configure in file /usr/lib/coreos-install/user_data file should be:

#cloud-config

coreos:
  etcd2:
    name: "infra1"
    initial-advertise-peer-urls: "http://10.245.1.102:2380"
    listen-peer-urls: "http://10.245.1.102:2380"
    listen-client-urls: "http://10.245.1.102:2379,http://127.0.0.1:2379"
    advertise-client-urls: "http://10.245.1.102:2379"
    initial-cluster-token: "etcd-cluster-1"
    initial-cluster: "infra0=http://10.245.1.101:2380,infra1=http://10.245.1.102:2380,infra2=http://10.245.1.103:2380"
    initial-cluster-state: "new"

in the node 10.245.1.103, the etcd2 configureation in file /usr/lib/coreos-install/user_data should be:

#cloud-config

coreos:
  etcd2:
    name: "infra2"
    initial-advertise-peer-urls: "http://10.245.1.103:2380"
    listen-peer-urls: "http://10.245.1.103:2380"
    listen-client-urls: "http://10.245.1.103:2379,http://127.0.0.1:2379"
    advertise-client-urls: "http://10.245.1.103:2379"
    initial-cluster-token: "etcd-cluster-1"
    initial-cluster: "infra0=http://10.245.1.101:2380,infra1=http://10.245.1.102:2380,infra2=http://10.245.1.103:2380"
    initial-cluster-state: "new"

##set the flannel options

set during installation

add the flannel options like below to the /var/lib/coreos-install/user_data file:

coreos:
  flannel:
    etcd_prefix: "/coreos.com/network"
    interface: enp0s8

###add the option settings in the file /var/lib/coreos-install/user_data

if missing set the flannel options during the installation, the above setting can be copied to the file /var/lib/coreos-install/user_data

##add hosts to /etc/hosts

$ cat hosts.yml
#cloud-config
write_files:
  - path: /etc/hosts
    content: |
       10.245.1.101 coreos-1
       10.245.1.102 coreos-2
       10.245.1.103 coreos-3
$ sudo coreos-cloudinit --from-file hosts.yml

##set the http proxy for docker Two methods ###1)through configure file

$ mkdir /etc/systemd/system/docker.service.d
$ vim /etc/systemd/system/docker.service.d
$ sudo systemctl daemon-reload
$ sudo systemctl restart docker

the contents of the /etc/systemd/system/docker.service.d looks like:

[Service]
Environment="HTTP_PROXY=http://proxy.example.com:8080"

###2)through the cloudinit

$ cat cloud-config.yml
#cloud-config

coreos:
  units:
    - name: docker.service
      drop-ins:
        - name: 20-http-proxy.conf
          content: |
            [Service]
            Environment="HTTP_PROXY=http://proxy.example.com:8080"
      command: restart
$ sudo coreos-cloudinit --from-file cloud-config.yml

##start flannel

set the flannel options in /run/flannel/options.env

$ sudo cat /run/flannel/options.env
FLANNELD_ETCD_PREFIX=/coreos.com/network2
FLANNELD_IFACE=enp0s8
$ sudo systemctl restart flanneld
$ sudo systemctl restart docker

###etcd data directory

the data directory of etcd is in /var/lib/etcd2/member/

###access early-docker

The flanneld is running in the early-docker and it listens on "unix:///var/run/early-docker.sock". To list what images are downloaded in this docker:

$sudo docker -H unix:///var/run/early-docker.sock images

##start kubernets

###start master

$  sudo ./kube-apiserver --insecure-bind-address=10.245.1.101 --insecure-port=8080  --logtostderr=true --service-cluster-ip-range=10.1.0.0/16 --etcd-servers=http://10.245.1.101:2379 --admission_control=NamespaceLifecycle,NamespaceExists,LimitRanger,ResourceQuota --service_account_lookup=false --advertise-address=10.245.1.101

###deploy a pod

$ kubectl run redis --image=redis

the k8s will always try to pull the image from the docker registry server because the image is the latest even if the redis:latest image is pulled

but if deploy a pod like this:

$ kubectl run redis --image=redis:3.2.0

the k8s will not pull the image from the docker registry server if the image redis:3.2.0 is pulled.