Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Configuring NGINX for Maximum Throughput Under High Concurrency
user web;
# One worker process per CPU core.
worker_processes 8;
# Also set
# /etc/security/limits.conf
# web soft nofile 65535
# web hard nofile 65535
# /etc/default/nginx
# ULIMIT="-n 65535"
worker_rlimit_nofile 65535;
pid /run/nginx.pid;
events {
#
# Determines how many clients will be served by each worker process.
# (Max clients = worker_connections * worker_processes)
# Should be equal to `ulimit -n`
#
worker_connections 65535;
#
# Let each process accept multiple connections.
# Accept as many connections as possible, after nginx gets notification
# about a new connection.
# May flood worker_connections, if that option is set too low.
#
multi_accept on;
#
# Preferred connection method for newer linux versions.
# Essential for linux, optmized to serve many clients with each thread.
#
use epoll;
}
http {
##
# Basic Settings
##
#
# Override some buffer limitations, will prevent DDOS too.
#
client_body_buffer_size 10K;
client_header_buffer_size 1k;
client_max_body_size 8m;
large_client_header_buffers 2 1k;
#
# Timeouts
# The client_body_timeout and client_header_timeout directives are
# responsible for the time a server will wait for a client body or
# client header to be sent after request. If neither a body or header
# is sent, the server will issue a 408 error or Request time out.
#
# The keepalive_timeout assigns the timeout for keep-alive connections
# with the client. Simply put, Nginx will close connections with the
# client after this period of time.
#
# Finally, the send_timeout is a timeout for transmitting a response
# to the client. If the client does not receive anything within this
# time, then the connection will be closed.
#
#
# send the client a "request timed out" if the body is not loaded
# by this time. Default 60.
#
client_body_timeout 32;
client_header_timeout 32;
#
# Every 60 seconds server broadcasts Sync packets, so 90 is
# a conservative upper bound.
#
keepalive_timeout 90; # default 65
send_timeout 120; # default 60
#
# Allow the server to close the connection after a client stops
# responding.
# Frees up socket-associated memory.
#
reset_timedout_connection on;
#
# Open file descriptors.
# Caches information about open FDs, freqently accessed files.
#
open_file_cache max=200000 inactive=20s;
open_file_cache_valid 30s;
open_file_cache_min_uses 2;
open_file_cache_errors on;
#
# Sendfile copies data between one FD and other from within the kernel.
# More efficient than read() + write(), since the requires transferring
# data to and from the user space.
#
sendfile on;
# Tcp_nopush causes nginx to attempt to send its HTTP response head in one
# packet, instead of using partial frames. This is useful for prepending
# headers before calling sendfile, or for throughput optimization.
tcp_nopush on;
#
# don't buffer data-sends (disable Nagle algorithm). Good for sending
# frequent small bursts of data in real time.
#
tcp_nodelay on;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# Logging Settings
##
#
# Use analytics to track stuff instead of using precious file IO resources.
# Disabling logging speeds up IO.
#
access_log off;
error_log /root/PROJECTS/logs/error.log crit;
##
# Gzip Settings
##
gzip on;
gzip_disable "MSIE [1-6]\.";
# Only allow proxy request with these headers to be gzipped.
gzip_proxied expired no-cache no-store private auth;
# Default is 6 (1<n<9), but 2 -- even 1 -- is enough. The higher it is, the
# more CPU cycles will be wasted.
gzip_comp_level 9;
gzip_min_length 500; # Default 20
gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
##
# Virtual Host Configs
##
include /etc/nginx/conf.d/*.conf;
}
@scrnjakovic

This comment has been minimized.

Copy link

commented Jan 29, 2018

Thanks for the gist! Was wondering if you can explain why 65535 for worker_rlimit_nofile and worker_connection? Does that value depend on anything, CPU cores etc? I'm trying to figure out... Regardless of my VPS size, output of ulimit -n was always 1024...

@zyf0330

This comment has been minimized.

Copy link

commented Mar 1, 2018

My three servers use nginx to serve HTTP/1.0 connections, and I set worker_connections to 32768. My app process is killed causing by memory usage too high. Then I change it to 10240, no problem happens anymore. My servers have 2 cpu cores and 4G memory.
Who knows why in this situation? Now I just know that worker_connections cannot be set too high always.

@ashishdungdung

This comment has been minimized.

Copy link

commented Mar 17, 2018

Thanks for this please keep this updated over time.

By the way do you have any suggestions for a server with 16 cores and 32GB RAM ?

@gdewey

This comment has been minimized.

Copy link

commented Jun 6, 2018

tks for sharing

@Spholt

This comment has been minimized.

Copy link

commented Jul 2, 2018

Thanks for sharing this, great comments

@v0lkan

This comment has been minimized.

Copy link
Owner Author

commented Dec 1, 2018

Here is a variant of it that I used to reverse proxy an HTTPS service

# One worker process per vCPU.
worker_processes  4;

events {
    #
    # Determines how many clients will be served by each worker process.
    # (Max clients = worker_connections * worker_processes)
    # Should be equal to `ulimit -n`
    #
    worker_connections 1024;

    #
    # Let each process accept multiple connections.
    # Accept as many connections as possible, after nginx gets notification
    # about a new connection.
    # May flood worker_connections, if that option is set too low.
    #
    multi_accept on;

    #
    # Preferred connection method for newer linux versions.
    # Essential for linux, optmized to serve many clients with each thread.
    #
    # Didn’t woork on Mac. — try on prod to see if it works.
    # use epoll;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    # error_log /Users/volkan/Desktop/error.log;
    # access_log /Users/volkan/Desktop/access.log;

    error_log off;
    access_log off;

    #
    # Override some buffer limitations, will prevent DDOS too.
    #
    client_body_buffer_size 10K;
    client_header_buffer_size 1k;
    client_max_body_size 8m;
    large_client_header_buffers 2 1k;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;

    # M$IE closes keepalive connections in 60secs anyway.
    # 90sec is a conservative upper bound.
    # This number should ideally be the MAX number of
    # seconds everything is donwlaoded.
    # So if "time to interactive" on a web app is 20secs,
    # Choosing 40secs (to be conservative) is a good ballpark
    # guesstimate.
    keepalive_timeout  90;

    # To be conservative: Default is 60.
    send_timeout 120;

    #gzip  on;


    #
    # Allow the server to close the connection after a client stops
    # responding.
    # Frees up socket-associated memory.
    #
    reset_timedout_connection on;

    #
    # don't buffer data-sends (disable Nagle algorithm). Good for sending
    # frequent small bursts of data in real time.
    #
    tcp_nodelay on;

    types_hash_max_size 2048;

    server {
        listen       8080;
        server_name  localhost;

        location / {
            # this also works:
            # proxy_pass https://--REDACTED-API-SERVER--$request_uri;
            # this works:
            proxy_pass https://--REDACTED-API-SERVER--;
            proxy_http_version 1.1;

            # these are optional:
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Host $host;
            proxy_cache_bypass $http_upgrade;

        }

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }

    include servers/*;
}
@v0lkan

This comment has been minimized.

Copy link
Owner Author

commented Dec 4, 2018

@scrnjakovic a couple of years (!) late to respond.

All my github notifications are filtered, so I tend to miss them unless someone mentions that something needs to taken care of :)

Certain VPSs will disable you modifying the ulimit settings (no matter if you are root or not) for security reasons.

If that’s the case: One solution is to use clustering and load balancing your traffic to more than one server/container/box .

@jstowey

This comment has been minimized.

Copy link

commented Dec 13, 2018

@v0lkan Thanks for this, is it safe to set the open file limit to the same as the output from ulimit -Hn? Or is it recommended to set it a bit lower to avoid potential issues?

My ulimit -n also always returns 1024

@v0lkan

This comment has been minimized.

Copy link
Owner Author

commented Jan 2, 2019

@jstowey For some reason I don’t get gist notifications in my inbox, or they are filtered out as “social” I dunno :)

Anyways, you can configure your system (depending on what your system is) to change your ulimit -n outcome. You can google “how to increase ulimit“

Also if you have 1.5hours, I have a talk about it (scaling your Node.js App like A Boss)
https://www.youtube.com/watch?v=Ogjb60Fg10A (part one)
https://www.youtube.com/watch?v=f5phsX4VUOU (part two)

If after updating your environment, ulimit still stays the same, then it’s probably managed and fixed by your hosting provider (AWS typically does that)

In that case I’d suggest you to horizontally scale (instead of a big fat machine that has a big fat ulimit, have smaller minions with ulimit is fixed at 1024)

It should be "in general" safe to set it at ulimit -n, it is the maximum number of handles NGINX will try to open, it does not mean it will necessarily exhaust all your file handles. — but to be safe, feel free to keep it a few 100s below the max if you want.

Also my assumption is there is no other stuff that consumes excessive file handles (like a database or redis or some CDN file service etc)

That said, your system might behave differently: so make sure to test and monitor first.

Hope that helps.

@electropolis

This comment has been minimized.

Copy link

commented Feb 8, 2019

You made small mistake in line 20. Max clients is for worker_rlimit not for worker_connections so ulimit -n should be equal for worker_rlimit and worker_connections should be one of the multiplier value. So consider also change line 23 value 65535 to: 65535/8 = 8192

@rocketspacer

This comment has been minimized.

Copy link

commented Mar 1, 2019

http://nginx.org/en/docs/ngx_core_module.html#worker_connections
worker_connections is per each worker. So you should set it to ulimit -n / worker_processes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.