Configuring NGINX for Maximum Throughput Under High Concurrency

nginx.conf

user web;

# One worker process per CPU core.
worker_processes 8;

# Also set
# /etc/security/limits.conf
#   web soft nofile 65535
#   web hard nofile 65535
# /etc/default/nginx
#       ULIMIT="-n 65535"
worker_rlimit_nofile 65535;

pid /run/nginx.pid;

events {

    #
    # Determines how many clients will be served by each worker process.
    # (Max clients = worker_connections * worker_processes)
    # Should be equal to `ulimit -n / worker_processes`
    #
    worker_connections 65535;

    #
    # Let each process accept multiple connections.
    # Accept as many connections as possible, after nginx gets notification
    # about a new connection.
    # May flood worker_connections, if that option is set too low.
    #
    multi_accept on;

    #
    # Preferred connection method for newer linux versions.
    # Essential for linux, optmized to serve many clients with each thread.
    #
    use epoll;
}

http {

    ##
    # Basic Settings
    ##

    #
    # Override some buffer limitations, will prevent DDOS too.
    #
    client_body_buffer_size 10K;
    client_header_buffer_size 1k;
    client_max_body_size 8m;
    large_client_header_buffers 2 1k;

    #
    # Timeouts
    # The client_body_timeout and client_header_timeout directives are
    # responsible for the time a server will wait for a client body or
    # client header to be sent after request. If neither a body or header
    # is sent, the server will issue a 408 error or Request time out.
    #
    # The keepalive_timeout assigns the timeout for keep-alive connections
    # with the client. Simply put, Nginx will close connections with the
    # client after this period of time.
    #
    # Finally, the send_timeout is a timeout for transmitting a response
    # to the client. If the client does not receive anything within this
    # time, then the connection will be closed.
    #

    #
    # send the client a "request timed out" if the body is not loaded
    # by this time. Default 60.
    #
    client_body_timeout   32;
    client_header_timeout 32;

    #
    # Every 60 seconds server broadcasts Sync packets, so 90 is
    # a conservative upper bound.
    #
    keepalive_timeout 90; # default 65
    send_timeout 120;     # default 60

    #
    # Allow the server to close the connection after a client stops
    # responding.
    # Frees up socket-associated memory.
    #
    reset_timedout_connection on;

    #
    # Open file descriptors.
    # Caches information about open FDs, freqently accessed files.
    #
    open_file_cache max=200000 inactive=20s;
    open_file_cache_valid 30s;
    open_file_cache_min_uses 2;
    open_file_cache_errors on;

    #
    # Sendfile copies data between one FD and other from within the kernel.
    # More efficient than read() + write(), since the requires transferring
    # data to and from the user space.
    #
    sendfile on;

    # Tcp_nopush causes nginx to attempt to send its HTTP response head in one
    # packet, instead of using partial frames. This is useful for prepending
    # headers before calling sendfile, or for throughput optimization.
    tcp_nopush on;

    #
    # don't buffer data-sends (disable Nagle algorithm). Good for sending
    # frequent small bursts of data in real time.
    #
    tcp_nodelay on;


    types_hash_max_size 2048;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    ##
    # Logging Settings
    ##

    #
    # Use analytics to track stuff instead of using precious file IO resources.
    # Disabling logging speeds up IO.
    #
    access_log off;
    error_log /root/PROJECTS/logs/error.log crit;

    ##
    # Gzip Settings
    ##

    gzip on;
    gzip_disable "MSIE [1-6]\.";

    # Only allow proxy request with these headers to be gzipped.
    gzip_proxied expired no-cache no-store private auth;

    # Default is 6 (1<n<9), but 2 -- even 1 -- is enough. The higher it is, the
    # more CPU cycles will be wasted.
    gzip_comp_level 9;
    gzip_min_length 500; # Default 20

    gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

    ##
    # Virtual Host Configs
    ##

    include /etc/nginx/conf.d/*.conf;
}

Let me explain more why I asked the above question -

In Kubernetes, you can pass the CPU request to the container e.g. we decided to pass the request as 1000m which is 1 CPU whereas the nodes where we have created the cluster are 8 core 16 GB memory. In such a case what should we set for worker_processes ? Has anyone done any configuration to optimize resource utilization as such? Thanks, Rajib

@arnav13081994

keepalive_timeout 15;
send_timeout 10;

You are right. I lower the keepalive_timeout to 15 than the CPU usage is dropped. Almost 100% when I set it 150 seconds.

Tried this on my nginx for django, but i keep getting this weird error when i access admin panel returns 500 bad request with an error : too many open files.

@Altroo some cloud vendors might not allow you modify file handler limits; that might be the issue.

Nobody talks about this, but you MUST increase the file max descriptors per process, as for very historical reasons, Linux limits to 1024 even in modern distros...

sudo nano /etc/security/limits.conf
# add:
* - nofile 65536
root - nofile 65536

sudo nano /etc/systemd/system.conf
# add:
DefaultLimitNOFILE=65536

sudo nano /etc/sysctl.conf
# add:
fs.file-max = 65536

nano /etc/default/nginx
# add:
ULIMIT="-n 65536"

Then reboot.

@v0lkan @jjxtra Why are you setting the soft limit on open files equal to the hard limit on open files? Will the worker processes not up their limit by themselves until they hit the hard limit?

Edit:

https://www.freedesktop.org/software/systemd/man/systemd.exec.html says regarding LimitNOFILE:

Nowadays, the hard limit defaults to 524288, a very high value compared to historical defaults. Typically applications should increase their soft limit to the hard limit on their own, if they are OK with working with file descriptors above 1023, i.e. do not use select(2).

But I'm still not sure if that means that nginx should do that on it's own or if we have to set that explicitly via worker_rlimit_nofile. From my tests it seems that worker_rlimit_nofile sets both the soft limit and the hard limit to that value.

---

For what it's worth on a Ubuntu Server 22.04 machine the values I get are:

$ cat /proc/sys/fs/file-max
9223372036854775807
$ ulimit -Hn
1048576

And I've read online a couple of times now that worker_rlimit_nofile should be double the worker_connections if you are using nginx as an reverse proxy because a revers proxy connection needs 2 file handles, is not not true?

I ended up using 1000000 for most of these values and it was fine. For nginx you can then set the worker limit to 2000000.

@v0lkan @jjxtra Why are you setting the soft limit on open files equal to the hard limit on open files?

That is an arbitrary choice.

You can make hard limit larger than the soft limit.

As long as they are large enough (and you have enough resources) things should work fine.

--

@ everyone else

I am typically terribly busy, and only have fraction of a time to look at and maintain this gist.

And it has been a long time since I last updated it; please cross-check this one with the latest NGINX documentation.

I don't think much have changed since then, since these are pretty low-level config; but still, the only time I will likely maintain this document will be when I need to scale NGINX to its limits again :) — which may be five or ten years from now; or maybe never :).

Nobody talks about this, but you MUST increase the file max descriptors per process, as for very historical reasons, Linux limits to 1024 even in modern distros...

Yes, I forgot to mention that. I assume that the reader would know about that.

In addition certain cloud vendors do not allow fiddling with the limits; so always verify that you have the limits as expected after you set them.

Hi @v0lkan,

Unlike for the access_log directive, error_log off; does not disable the error logs.
error_log off; would create a file named "off" for logging the errors within the default NGINX config files directory.

This is also mentioned in the NGINX blog post, in which it recommends specifying error_log /dev/null emerg; in cases where "disabling" error logging is preferred :)

Thanks! :)