kubernetes-ingress websockets with nodejs

frontend-ws-connection.ts

export const ws = webSocket<WebsocketMessage>(`wss://${location.hostname}:${location.protocol === 'https:' ? 443 : 80}/ws/`);

export const wsObserver = ws
  .pipe(
    retryWhen(errors =>
      errors.pipe(
        delay(1000)
      )
    )
  );

wsObserver.subscribe(console.log);

ingress-service.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-service
  namespace: <YOUR_NAMESPACE>
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/proxy-read-timeout: 3600
    nginx.ingress.kubernetes.io/proxy-send-timeout: 3600
spec:
  rules:
    - http:
        paths:
          - path: /
            backend:
              serviceName: client-cluster-ip-service
              servicePort: 3000
          # Below is the important part!
          - path: /ws/
            backend:
              serviceName: server-cluster-ip-service
              servicePort: 40510

server-cluster-ip-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: server-cluster-ip-service
  namespace: <YOUR_NAMESPACE>
spec:
  type: ClusterIP
  selector:
    component: server
  ports:
    - port: 40510
      targetPort: 40510
      # The below line isn't required.
      protocol: TCP

server-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: server-deployment
  namespace: <YOUR_NAMESPACE>
spec:
  replicas: 1
  selector:
    matchLabels:
      component: server
  template:
    metadata:
      labels:
        component: server
    spec:
      containers:
        - name: server
          image: <YOUR_DOCKER_IMAGE>
          ports:
            - containerPort: 40510

server.js

import { Server as WebSocketServer } from 'ws';

// IMPORTANT: not a secure connection
const wss = new WebSocketServer({
    path: '/ws/',
    port: 40510,
});

wss.on('connection', function (ws) {
    console.log('connection!');
});

wss.on('close', function close() {
    console.log('ws disconnected');
});

Does anyone know any Guide for configuring haproxy-ingress to support WebSockets?

Fantastic post by the way has really helped me.

So following the OP instructions, I have the ingress using SSL and using wss: from the client to connect to the ingress, but then the back-end is just HTTP.

I am getting this error: failed: Error in connection establishment: net::ERR_SSL_PROTOCOL_ERROR

So my follow up question:

Do I need to configure the back-end service with an SSL cert or is there some ingress magic configuration that would allow it to up stream the request as ws: ?

thx

If you are using kubernetes you should probably instlal https://github.com/jetstack/cert-manager and update your ingress to handle SSL certificate.

If you are using kubernetes you should probably instlal https://github.com/jetstack/cert-manager and update your ingress to handle SSL certificate.

I have SSL terminating at the ingress controller, with a certificate configured. Do I also need to configure an additional certificate on the backend service?

Thanks @jsdevtom!!!
I just had the issue with the websocket connection close after 30 seconds and therefore I find your gist.

I'm using GKE with the L7 GLB. For those, who are using the Google cloud infrastructure it is pretty simple to get websocket connections working with google managed ssl certificates. It looks a little bit different:

apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
  name: service-backend-config
spec:
  timeoutSec: 3600

---

apiVersion: v1
kind: Service
metadata:
  labels:
    app: service
  name: service
  annotations:
    cloud.google.com/backend-config: '{"ports": {"8080":"service-backend-config"}}'
spec:
  ports:
    - name: http
      port: 8080
      protocol: TCP
      targetPort: 8080
  selector:
    app: service
  type: NodePort

---

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: public-ingress
  annotations:
    kubernetes.io/ingress.global-static-ip-name: "your-ip"
    networking.gke.io/managed-certificates: "your-certificates"
spec:
  rules: 
  - host: "your-domain.com"
    http:
      paths:
      - backend:
          serviceName: service
          servicePort: 8080
        path: /*

in k8s 1.18.8 the annotations need a string not a number:
nginx.ingress.kubernetes.io/proxy-read-timeout: 3600
nginx.ingress.kubernetes.io/proxy-send-timeout: 3600
==>
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"

still not working though

I'm sure that missing trailing forward slash on /ws/ would have cost me 10 hours too. So, thanks!

I cant get this to work with socket io and typescript. Any suggestion will be highly appreciated

After some hours trying to fix it, i found the proper config for my ingress ( k8s azure )

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-service
  namespace: development
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
spec:
  rules:
    - host: svc-host-domain.com
      http:
        paths:
        - backend:
            service:
              name: my-service
              port:
                number: 8084
          path: /publish
          pathType: Prefix
        - backend:
            service:
              name: my-service
              port:
                number: 8080
          path: /ws/
          pathType: Prefix

Thank you for the great snippet

( i'm still struggling to have the keep-alive working well, it disconnects my websocket clients suddenly )

I'm sure that missing trailing forward slash on /ws/ would have cost me 10 hours too. So, thanks!

What should happen if the trailing slash is missing? I'm trying to debug an error related to my k8s websocket setup, and I'm using /ws right now, but I'm not exactly sure it's failing...

A working e2e complete setup of this scenario is available here: https://github.com/martencassel/nodejs-websocket-nginx-ingress-setup

A working e2e complete setup of this scenario is available here: https://github.com/martencassel/nodejs-websocket-nginx-ingress-setup

🔥

Thank you you save me a lot of time. I have been struggling with this issue, at the time I didn't know that I have to configure my ingress for websocket. Thank you again.