Configuring NGINX for Maximum Throughput Under High Concurrency

nginx.conf

user web;

# One worker process per CPU core.
worker_processes 8;

# Also set
# /etc/security/limits.conf
#   web soft nofile 65535
#   web hard nofile 65535
# /etc/default/nginx
#       ULIMIT="-n 65535"
worker_rlimit_nofile 65535;

pid /run/nginx.pid;

events {

    #
    # Determines how many clients will be served by each worker process.
    # (Max clients = worker_connections * worker_processes)
    # Should be equal to `ulimit -n / worker_processes`
    #
    worker_connections 65535;

    #
    # Let each process accept multiple connections.
    # Accept as many connections as possible, after nginx gets notification
    # about a new connection.
    # May flood worker_connections, if that option is set too low.
    #
    multi_accept on;

    #
    # Preferred connection method for newer linux versions.
    # Essential for linux, optmized to serve many clients with each thread.
    #
    use epoll;
}

http {

    ##
    # Basic Settings
    ##

    #
    # Override some buffer limitations, will prevent DDOS too.
    #
    client_body_buffer_size 10K;
    client_header_buffer_size 1k;
    client_max_body_size 8m;
    large_client_header_buffers 2 1k;

    #
    # Timeouts
    # The client_body_timeout and client_header_timeout directives are
    # responsible for the time a server will wait for a client body or
    # client header to be sent after request. If neither a body or header
    # is sent, the server will issue a 408 error or Request time out.
    #
    # The keepalive_timeout assigns the timeout for keep-alive connections
    # with the client. Simply put, Nginx will close connections with the
    # client after this period of time.
    #
    # Finally, the send_timeout is a timeout for transmitting a response
    # to the client. If the client does not receive anything within this
    # time, then the connection will be closed.
    #

    #
    # send the client a "request timed out" if the body is not loaded
    # by this time. Default 60.
    #
    client_body_timeout   32;
    client_header_timeout 32;

    #
    # Every 60 seconds server broadcasts Sync packets, so 90 is
    # a conservative upper bound.
    #
    keepalive_timeout 90; # default 65
    send_timeout 120;     # default 60

    #
    # Allow the server to close the connection after a client stops
    # responding.
    # Frees up socket-associated memory.
    #
    reset_timedout_connection on;

    #
    # Open file descriptors.
    # Caches information about open FDs, freqently accessed files.
    #
    open_file_cache max=200000 inactive=20s;
    open_file_cache_valid 30s;
    open_file_cache_min_uses 2;
    open_file_cache_errors on;

    #
    # Sendfile copies data between one FD and other from within the kernel.
    # More efficient than read() + write(), since the requires transferring
    # data to and from the user space.
    #
    sendfile on;

    # Tcp_nopush causes nginx to attempt to send its HTTP response head in one
    # packet, instead of using partial frames. This is useful for prepending
    # headers before calling sendfile, or for throughput optimization.
    tcp_nopush on;

    #
    # don't buffer data-sends (disable Nagle algorithm). Good for sending
    # frequent small bursts of data in real time.
    #
    tcp_nodelay on;


    types_hash_max_size 2048;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    ##
    # Logging Settings
    ##

    #
    # Use analytics to track stuff instead of using precious file IO resources.
    # Disabling logging speeds up IO.
    #
    access_log off;
    error_log /root/PROJECTS/logs/error.log crit;

    ##
    # Gzip Settings
    ##

    gzip on;
    gzip_disable "MSIE [1-6]\.";

    # Only allow proxy request with these headers to be gzipped.
    gzip_proxied expired no-cache no-store private auth;

    # Default is 6 (1<n<9), but 2 -- even 1 -- is enough. The higher it is, the
    # more CPU cycles will be wasted.
    gzip_comp_level 9;
    gzip_min_length 500; # Default 20

    gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

    ##
    # Virtual Host Configs
    ##

    include /etc/nginx/conf.d/*.conf;
}

Let me explain more why I asked the above question -

In Kubernetes, you can pass the CPU request to the container e.g. we decided to pass the request as 1000m which is 1 CPU whereas the nodes where we have created the cluster are 8 core 16 GB memory. In such a case what should we set for worker_processes ? Has anyone done any configuration to optimize resource utilization as such? Thanks, Rajib

@arnav13081994

keepalive_timeout 15;
send_timeout 10;

You are right. I lower the keepalive_timeout to 15 than the CPU usage is dropped. Almost 100% when I set it 150 seconds.

Tried this on my nginx for django, but i keep getting this weird error when i access admin panel returns 500 bad request with an error : too many open files.

@Altroo some cloud vendors might not allow you modify file handler limits; that might be the issue.

Nobody talks about this, but you MUST increase the file max descriptors per process, as for very historical reasons, Linux limits to 1024 even in modern distros...

sudo nano /etc/security/limits.conf
# add:
* - nofile 65536
root - nofile 65536

sudo nano /etc/systemd/system.conf
# add:
DefaultLimitNOFILE=65536

sudo nano /etc/sysctl.conf
# add:
fs.file-max = 65536

nano /etc/default/nginx
# add:
ULIMIT="-n 65536"

Then reboot.