expose services on k8s via nginx-ingress-controller, traefik, ambassador

k8s.md

httpbin, whoami, qotm (quote of the moment) 服务特别适合用来作为测试

kubectl create namespace httpbin
kubectl create deploy httpbin --image=citizenstig/httpbin --port=8000 -n httpbin
kubectl expose deployment httpbin --name=httpbin -n httpbin --port=80 --target-port=8000

kubectl create ns whoami
kubectl create deploy whoami --image=containous/whoami --port=80 -n whoami --replicas=3
kubectl expose deployment whoami --name=whoami -n whoami --port=80 --target-port=80

kubectl create ns qotm
kubectl create deploy qotm --image=datawire/qotm:1.7 --port=5000 -n qotm --replicas=3
kubectl expose deployment qotm --name=qotm -n qotm --port=80 --target-port=5000

// https://github.com/kubernetes-up-and-running/kuard
kubectl run --restart=Never --image=gcr.io/kuar-demo/kuard-amd64:blue kuard
kubectl port-forward kuard 8080:8080

tips:

1. docker for mac stable channel 比较新的版本支持kubernetes single node cluster了。也可试试edge channel.

2. 较新版本的docker for mac 或 minikube支持 type=LoadBalancer类型 的service了(通过vpnkit), minikube tunnel也支持. 因此mac中不需要 MetalLB的方案，除非线上vps的microk8s

3. mac 中 k3d 可以创建multi-nodes kubernetes cluster. 非常方便本地开发测试

k3d version
k3d version v1.7.0
k3s version v1.17.3-k3s1

tips: k3s 1.17 里自带的traefik是 1.xx版本的 非2.0的 

# 会自动安装traefik 1.x
k3d create --name="demo" --workers="2" --publish "80:80" --publish "443:443" --api-port=16443

# --server-arg "--no-deploy=traefik" 禁止自动安装traefik 1.x, 然后通过helm 3 安装 traefik 2.x
k3d create --name k3s-traefik-v2 --workers 2 --publish "80:80" --publish "443:443" --server-arg "--no-deploy=traefik"
helm install traefik traefik/traefik --set dashboard.ingressRoute=true

4. 需要添加相关的记录到 /etc/hosts

use nginx ingress controller

helm repo update
helm install nginx-ingress-controller stable/nginx-ingress

kubectl apply -f - << EOF
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
  name: httpbin
  namespace: httpbin
spec:
  rules:
    - host: example.io
      http:
        paths:
          - backend:
              serviceName: httpbin
              servicePort: 8000
            path: /
EOF

注意：nginx ingress 中的serviceName 不支持 .: 所以namespace 得和所指向的service相同

servicePort 填该service 的port 或 name 或者 targetPort 都可以

完整的snippet传送门 https://pastebin.com/raw/bm3dmxYr

use Traefik 1.x as ingress controller and enabled dashboard UI

helm install my-release --namespace kube-system   --set dashboard.enabled=true,dashboard.domain=traefik.example.io stable/traefik --version 1.86.2

helm install my-release --namespace kube-system \
  --set dashboard.enabled=true,dashboard.domain=traefik.example.io stable/traefik 

kubectl apply -f - << EOF
kind: Ingress
apiVersion: networking.k8s.io/v1beta1
metadata:
  name: nginx
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: web
spec:
  rules:
    - host: nginx.example.io
      http:
        paths:
          - path: /
            backend:
              serviceName: nginx
              servicePort: 80
---
kind: Ingress
apiVersion: networking.k8s.io/v1beta1
metadata:
  name: httpbin
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: web
  namespace: httpbin
spec:
  rules:
    - host: httpbin.example.io
      http:
        paths:
          - path: /
            backend:
              serviceName: httpbin
              servicePort: 8000
EOF

可以用 kubectl apply -f - << EOF创建 example 里的Whoami和Ingress

然后在dashboard http://traefik.example.io 观察下效果

注意 traefik 的ingress apiVersion， annotations 中的 traefik.ingress.kubernetes.io/router.entrypoints: web 不加也可以识别

/etc/hosts 需要加一些记录
127.0.0.1 traefik.example.io
127.0.0.1 httpbin.example.io
127.0.0.1 whoami.example.io
127.0.0.1 qotm.example.io

传送门： https://pastebin.com/raw/Z599Eg1F

use Traefik 2.x as ingress controller and enabled dashboard UI

安装时可以set的字段参考 https://github.com/containous/traefik-helm-chart/blob/master/traefik/values.yaml

helm repo add traefik https://containous.github.io/traefik-helm-chart
helm repo update
helm install traefik traefik/traefik --set ingressRoute.dashboard.enabled=true -n traefik

再创建一条IngressRoute, 官方文档 https://github.com/containous/traefik-helm-chart/tree/master/traefik 中有

# dashboard.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: dashboard
spec:
  entryPoints:
    - web
  routes:
    - match: Host(`traefik.localhost`) && (PathPrefix(`/dashboard`) || PathPrefix(`/api`))
      kind: Rule
      services:
        - name: api@internal
          kind: TraefikService

# ingressRoute.yaml

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroute
  namespace: whoami
spec:
  entryPoints:
    - web
  routes:
  - match: Host(`whoami.example.io`)
    kind: Rule
    services:
    - name: whoami
      port: 80

---

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroute
  namespace: httpbin
spec:
  entryPoints:
    - web
  routes:
  - match: Host(`httpbin.example.io`)
    kind: Rule
    services:
    - name: httpbin
      port: 8000

---

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroute
  namespace: qotm
spec:
  entryPoints:
    - web
  routes:
  - match: Host(`qotm.example.io`)
    kind: Rule
    services:
    - name: qotm
      port: 5000

Use ambassador edge stack

https://www.getambassador.io/docs/latest/topics/install/helm/

---
apiVersion: getambassador.io/v2
kind: Mapping
metadata:
  name: quote-backend
  namespace: ambassador
spec:
  prefix: /qotm/
  service: qotm.qotm:5000

---

apiVersion: getambassador.io/v2
kind: Mapping
metadata:
    name: whoami
    namespace: whoami
spec:
    prefix: /whoami/
    service: whoami.whoami:80

---
apiVersion: getambassador.io/v2
kind: Mapping
metadata:
  name: httpbin
  namespace: httpbin
spec:
  prefix: /httpbin/
  service: httpbin.httpbin:8000

note:

这些地方容易踩坑，这是实践出来的
namespace 可以是任意存在的namespace名称
service 的格式是 <NAMESPACE>.<SERVICE>:<PORT>
如果和指向的服务在一个namespace，可以省略掉
如果是服务是80端口，可以省略掉
如果prefix指定了第二个斜杠 那么在浏览器访问 /prefix/ 的时候就不能省略掉第二个/

tips:
端口可以查看 kubectl get svc,mapping -A, 或者 kubectl expose deployment 的时候通过 --port --target-port来指定

istio 1.5.4 通过 Ingress Gateway 和 VirtualService 可以实现同样的效果

文档链接
https://istio.io/docs/tasks/traffic-management/ingress/ingress-control/#accessing-ingress-services-using-a-browser

istioctl manifest apply --set profile=demo

kubectl apply -f samples/httpbin/httpbin.yaml

kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: httpbin-gateway
spec:
  selector:
    istio: ingressgateway # use Istio default gateway implementation
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: httpbin
spec:
  hosts:
  - "*"
  gateways:
  - httpbin-gateway
  http:
  - match:
    - uri:
        prefix: "/headers"
    route:
    - destination:
        port:
          number: 8000
        host: httpbin
EOF

注解：
httpbin-gateway 的host 要设为 *，because you can’t pass the Host header to a browser like you did with curl

httpbin virtualservice 的host 设为 *， 表示不限定， localhost/headers 或 httpbin.example.io/headers 都可以打开

设置了match，那么只能打开 /headers 这个页面。 如果去掉 match 部分，就可以打开 httpbin.exmaple.io 的其他页面了

试试这个

kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: httpbin
spec:
  hosts:
  - "httpbin.example.io"
  gateways:
  - httpbin-gateway
  http:
  - route:
     - destination:
        port:
          number: 8000
        host: httpbin
EOF

Contour as ingress controller

https://blog.heptio.com/making-it-easy-to-use-envoy-as-a-kubernetes-load-balancer-dde82959f171

kubectl apply -f - << EOF
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: httpbin
  namespace: httpbin
spec:
  rules:
    - host: httpbin.example.io
      http:
        paths:
          - backend:
              serviceName: httpbin
              servicePort: 8000
            path: /
EOF

HTTPProxy

Note: IngressRoute is deprecated and will be removed after Contour 1.0 ships in November.

IngressRoute renamed to HTTPProxy

kubectl apply -f - << EOF
apiVersion: projectcontour.io/v1
kind: HTTPProxy
metadata:
  name: httpbin
  namespace: httpbin
spec:
  virtualhost:
    fqdn: httpbin.example.io
  routes:
    - conditions:
      - prefix: /
      services:
        - name: httpbin
          port: 8000
EOF