kubernetes-ingress websockets with nodejs

frontend-ws-connection.ts

export const ws = webSocket<WebsocketMessage>(`wss://${location.hostname}:${location.protocol === 'https:' ? 443 : 80}/ws/`);

export const wsObserver = ws
  .pipe(
    retryWhen(errors =>
      errors.pipe(
        delay(1000)
      )
    )
  );

wsObserver.subscribe(console.log);

ingress-service.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-service
  namespace: <YOUR_NAMESPACE>
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/proxy-read-timeout: 3600
    nginx.ingress.kubernetes.io/proxy-send-timeout: 3600
spec:
  rules:
    - http:
        paths:
          - path: /
            backend:
              serviceName: client-cluster-ip-service
              servicePort: 3000
          # Below is the important part!
          - path: /ws/
            backend:
              serviceName: server-cluster-ip-service
              servicePort: 40510

server-cluster-ip-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: server-cluster-ip-service
  namespace: <YOUR_NAMESPACE>
spec:
  type: ClusterIP
  selector:
    component: server
  ports:
    - port: 40510
      targetPort: 40510
      # The below line isn't required.
      protocol: TCP

server-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: server-deployment
  namespace: <YOUR_NAMESPACE>
spec:
  replicas: 1
  selector:
    matchLabels:
      component: server
  template:
    metadata:
      labels:
        component: server
    spec:
      containers:
        - name: server
          image: <YOUR_DOCKER_IMAGE>
          ports:
            - containerPort: 40510

server.js

import { Server as WebSocketServer } from 'ws';

// IMPORTANT: not a secure connection
const wss = new WebSocketServer({
    path: '/ws/',
    port: 40510,
});

wss.on('connection', function (ws) {
    console.log('connection!');
});

wss.on('close', function close() {
    console.log('ws disconnected');
});

It took me roughly a total of 28 hours to get my frontend to be able to access the node.js websocket server with Kubernetes in MiniKube.

Mainly due to my lack of knowledge/ plain stupidity in the realms of networking. I tried many debugging techniques. For people who run in to similar errors, here is a list of pitfalls mixed with a checklist of tips to debug/ solve this problem:

* [ ]  **There is a confusing difference between [kubernetes-ingress](https://github.com/nginxinc/kubernetes-ingress) and [ingress-nginx](https://github.com/kubernetes/ingress-nginx/)**. You probably want [ingress-nginx](https://github.com/kubernetes/ingress-nginx/). When looking at GitHub issues/ docs, make sure you're reading from the correct project.

* [ ]  **Ensure the path of the websocket is correct and consistent across files**. For example,
  
  * in the ingress-service.yaml you have `- path: /ws/` NOT  `- path: /ws`
  * in your server you have `path: '/ws/'`
  * in your websocket client you have `wss://${location.hostname}:${location.protocol === 'https:' ? 443 : 80}/ws/`

* [ ]  **Support for websockets is provided by NGINX out of the box**. No special configuration required. The only requirement to avoid the close of connections is the increase of the values of proxy-read-timeout and proxy-send-timeout.[**¹**](https://kubernetes.github.io/ingress-nginx/user-guide/miscellaneous/#websockets) ([as shown above](https://gist.github.com/jsdevtom/7045c03c021ce46b08cb3f41db0d76da#file-ingress-service-yaml))

* [ ]  **[Kubernetes tab completion](https://kubernetes.io/docs/tasks/tools/install-kubectl/#optional-kubectl-configurations) will save you hours on finding pod names** make sure you have it

* [ ]  **Ensure your server's websocket is 'insecure'** (using `ws` and NOT `wss`). This is because the ingress will most likely be managing the certificate for you. Trying to use wss inside your server will result in mysterious 502 Bad Gateway errors that don't show up in the in ingress' pod's logs (which can be found using `kubectl exec -n ingress-nginx <name of ingress controller> cat nginx.conf`)

* [ ]  **[Kubernetes-ingress' troubleshooting page](https://kubernetes.github.io/ingress-nginx/troubleshooting/#troubleshooting) is very helpful** but the [debug logging](https://kubernetes.github.io/ingress-nginx/troubleshooting/#debug-logging) didn't do anything for me

Criticism and suggestions are extremely welcome!