Create a gist now

Instantly share code, notes, and snippets.

Embed
Configuring an AWS Elastic Load Balancer for a Node.js application using WebSockets on EC2

AWS ELB with Node.js and WebSockets

This assumes that:

  • You are using Nginx.
  • You want to accept incoming connections on port 80.
  • Your Node.js app is listening on port 3000.
  • You want to be able to connect to your Node.js instance directly as well as via the load balancer.

####1. Create load balancer

Create a load balancer in the relevant AWS region with these settings:

  • Load Balancer Protocol: TCP
  • Load Balancer Port: 80
  • Instance Protocol: TCP
  • Instance Port: 8080

For HTTPS/WSS support, add a listener with these settings:

  • Load Balancer Protocol: SSL
  • Load Balancer Port: 443
  • Instance Protocol: TCP
  • Instance Port: 8080

####2. Enable Proxy Protocol support

If the Node.js app needs to be able to see the client's real IP address for incoming WebSocket connections, you need to enable Proxy Protocol support on the load balancer. Without doing this all requests will appear to come from the load balancer.

Pour yourself a strong drink then follow the instructions here:

http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/enable-proxy-protocol.html

####2. Configure Nginx

Configure Nginx to get the client's IP address from the load balancer using Proxy Protocol. The requests from the load balancer need to be sent to a different port from ordinary connections, so that Nginx can intercept and remove the Proxy Protocol headers.

Your Nginx config should look similar to this:

# The Node.js application
upstream node_upstream {
  server localhost:3000 fail_timeout=10;
}

# Accept connections directly from clients
server {
  listen 80;
  charset utf-8;
  location / {
      proxy_pass http://node_upstream;
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "upgrade";
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
  }
}

# Accept connections via the load balancer
server {
  listen 8080 proxy_protocol;
  set_real_ip_from 10.0.0.0/8;
  real_ip_header proxy_protocol;
  charset utf-8;
  location / {
      proxy_pass http://node_upstream;
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "upgrade";
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
  }
}

####3. Create EC2 security groups

  • Assign a new security group to the load balancer which accepts incoming connections on port 80 (and 443 if you need SSL).
  • Create a security group for the EC2 instance hosting the Node.js app that:
    1. Accepts incoming connections on port 80 from all addresses.
    2. Accepts incoming connections on port 8080 only from the load balancer security group.
@Javlopez

This comment has been minimized.

Show comment
Hide comment
@Javlopez

Javlopez Oct 11, 2015

Hi, this guide is very usefull, but I have some errors (502 [upstream prematurely closed connection while reading response header from upstream]), do you have some idea?.

I hope can you help me

Hi, this guide is very usefull, but I have some errors (502 [upstream prematurely closed connection while reading response header from upstream]), do you have some idea?.

I hope can you help me

@lukem512

This comment has been minimized.

Show comment
Hide comment
@lukem512

lukem512 Jan 4, 2016

Thank you for posting this!

lukem512 commented Jan 4, 2016

Thank you for posting this!

@singh1469

This comment has been minimized.

Show comment
Hide comment
@singh1469

singh1469 Apr 12, 2016

Hi, how do you switch KeepAlive on together with a timeout to play nicely with the load balancer's idle timeout?

Hi, how do you switch KeepAlive on together with a timeout to play nicely with the load balancer's idle timeout?

@rturk

This comment has been minimized.

Show comment
Hide comment
@rturk

rturk Oct 9, 2016

Is this is also valid for Application Load Balancer?

rturk commented Oct 9, 2016

Is this is also valid for Application Load Balancer?

@GitHubUserToo

This comment has been minimized.

Show comment
Hide comment
@GitHubUserToo

GitHubUserToo Feb 19, 2017

Also interested if this works with Application Load Balancer, like rturk. Anyone? Thanks.

Also interested if this works with Application Load Balancer, like rturk. Anyone? Thanks.

@chiragpurohit71085

This comment has been minimized.

Show comment
Hide comment
@chiragpurohit71085

chiragpurohit71085 Mar 3, 2017

Hi what I need to do if I have apache as a server?

Hi what I need to do if I have apache as a server?

@InstanceOfMichael

This comment has been minimized.

Show comment
Hide comment
@InstanceOfMichael

InstanceOfMichael Mar 31, 2017

I'm currently having this probem with a setup like EC2 (Nginx) -> ELB -> EC2 (Nginx -> Internal NodeJS)

This relationship EC2 (Nginx) -> ELB makes this error:

2017/03/31 00:15:09 [error] 30674#30674: *1 upstream prematurely closed connection while reading response header from upstream, client: 10.218.65.120, server: testserver.com, request: "GET /api/services/v1/hello-world HTTP/1.1", upstream: "http://10.x.x.x:80/api/services/v1/hello-world", host: "testserver.com"

The other side of the ELB isn't reporting any issues.

I'm currently having this probem with a setup like EC2 (Nginx) -> ELB -> EC2 (Nginx -> Internal NodeJS)

This relationship EC2 (Nginx) -> ELB makes this error:

2017/03/31 00:15:09 [error] 30674#30674: *1 upstream prematurely closed connection while reading response header from upstream, client: 10.218.65.120, server: testserver.com, request: "GET /api/services/v1/hello-world HTTP/1.1", upstream: "http://10.x.x.x:80/api/services/v1/hello-world", host: "testserver.com"

The other side of the ELB isn't reporting any issues.

@kodeshpa

This comment has been minimized.

Show comment
Hide comment
@kodeshpa

kodeshpa Apr 21, 2017

Followed these but now getting this error " while reading PROXY protocol,

Followed these but now getting this error " while reading PROXY protocol,

@abalad

This comment has been minimized.

Show comment
Hide comment
@abalad

abalad Sep 12, 2017

By following this tutorial Elastic Bean .. returns me the following message:

Following services are not running: proxy.

abalad commented Sep 12, 2017

By following this tutorial Elastic Bean .. returns me the following message:

Following services are not running: proxy.

@appleboy

This comment has been minimized.

Show comment
Hide comment
@appleboy

appleboy Oct 27, 2017

@GitHubUserToo You don't need the Nginx server if you use the Application Load Balancer (ALB) since ALB support industry-standard protocols (WebSocket).

See: https://aws.amazon.com/blogs/aws/new-aws-application-load-balancer/

@GitHubUserToo You don't need the Nginx server if you use the Application Load Balancer (ALB) since ALB support industry-standard protocols (WebSocket).

See: https://aws.amazon.com/blogs/aws/new-aws-application-load-balancer/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment