Adjusting child processes for PHP-FPM (Nginx)

php-pools.md

Adjusting child processes for PHP-FPM (Nginx)

When setting these options consider the following:

• How long is your average request?
• What is the maximum number of simultaneous visitors the site(s) get?
• How much memory on average does each child process consume?

Determine if the max_children limit has been reached.

• sudo grep max_children /var/log/php?.?-fpm.log.1 /var/log/php?.?-fpm.log

Determine system RAM and average pool size memory.

• free -h
• All fpm processes: ps -ylC php-fpm7.0 --sort:rss
• Average memory: ps --no-headers -o "rss,cmd" -C php-fpm7.0 | awk '{ sum+=$1 } END { printf ("%d%s\n", sum/NR/1024,"M") }'
• All fpm processes memory: ps -eo size,pid,user,command --sort -size | awk '{ hr=$1/1024 ; printf("%13.2f Mb ",hr) } { for ( x=4 ; x<=NF ; x++ ) { printf("%s ",$x) } print "" }' | grep php-fpm

Calculate max_children

Based on RAM

• pm.max_children = Total RAM dedicated to the web server / Max child process size

• System RAM: 2GB

• Average Pool size: 85Mb

• pm.max_children = 1500MB / 85MB = 17

Based on average script execution time

• max_children = (average PHP script execution time) * (PHP requests per second)
• visitors = max_children * (seconds between page views) / (avg. execution time)

Configure

sudo vim /etc/php/7.0/fpm/pool.d/www.conf

pm.max_children = 17
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 3
pm.max_request = 1000

; Choose how the process manager will control the number of child processes.
; Possible Values:
;   static  - a fixed number (pm.max_children) of child processes;
;   dynamic - the number of child processes are set dynamically based on the
;             following directives:
;             pm.max_children      - the maximum number of children that can
;                                    be alive at the same time.
;
;             pm.start_servers     - the number of children created on startup.
;                                    this value must not be less than min_spare_servers 
;                                    and not greater than max_spare_servers.
;
;             pm.min_spare_servers - the minimum number of children in 'idle'
;                                    state (waiting to process). If the number
;                                    of 'idle' processes is less than this
;                                    number then some children will be created.
;
;             pm.max_spare_servers - the maximum number of children in 'idle'
;                                    state (waiting to process). If the number
;                                    of 'idle' processes is greater than this
;                                    number then some children will be killed.
; Note: This value is mandatory.

@endrureza neither is better than the other. Each one can have a better fit for a particular use-case, and it's up to the developer of that system to consider the trade-offs.

Static

• Each process sits in memory at all times, how much memory is consumed will vary for each server.
• CPU experiences less load spikes when traffic spikes; since there is no overhead in spawning new processes.

Dynamic

• Spawns new processes when traffic load increases.
• Spawning processes will put some load on the CPU.
• Frees up memory that can be used by other services when traffic is low.

i have a dedicated server which have 128 gb of ram and xeon processor with 1gbps bandwidth. I have installed centminmod on my server but the issue I am facing is that when my traffic raise to 3000 realtime the server stop responding.

What I am looking for what will be the configuration of php-fpm so it can handle 10000 realtime users
My configuration is this

`global]
; Log level
; Possible Values: alert, error, warning, notice, debug
; Default Value: notice
log_level = warning
pid = /var/run/php-fpm/php-fpm.pid
error_log = /var/log/php-fpm/www-error.log
emergency_restart_threshold = 10
emergency_restart_interval = 1m
process_control_timeout = 10s
include=/usr/local/nginx/conf/phpfpmd/*.conf

[www]
user = nginx
group = nginx

listen = 127.0.0.1:9000
listen.allowed_clients = 127.0.0.1
listen.backlog = 65535

;listen = /tmp/php5-fpm.sock
listen.owner = nginx
listen.group = nginx
listen.mode = 0660

pm = static
pm.max_children = 3072
; Default Value: min_spare_servers + (max_spare_servers - min_spare_servers) / 2
pm.start_servers = 100
pm.min_spare_servers = 10000
pm.max_spare_servers = 15000
pm.max_requests = 10000

; PHP 5.3.9 setting
; The number of seconds after which an idle process will be killed.
; Note: Used only when pm is set to 'ondemand'
; Default Value: 10s
pm.process_idle_timeout = 10s;

rlimit_files = 65536
rlimit_core = 0

; The timeout for serving a single request after which the worker process will
; be killed. This option should be used when the 'max_execution_time' ini option
; does not stop script execution for some reason. A value of '0' means 'off'.
; Available units: s(econds)(default), m(inutes), h(ours), or d(ays)
; Default Value: 0
;request_terminate_timeout = 0
; Default Value: 0
;request_slowlog_timeout = 0
slowlog = /var/log/php-fpm/www-slow.log

pm.status_path = /phpstatus
ping.path = /phpping
ping.response = pong

; Limits the extensions of the main script FPM will allow to parse. This can
; prevent configuration mistakes on the web server side. You should only limit
; FPM to .php extensions to prevent malicious users to use other extensions to
; exectute php code.
; Note: set an empty value to allow all extensions.
; Default Value: .php
security.limit_extensions = .php

; catch_workers_output = yes
php_admin_value[error_log] = /var/log/php-fpm/www-php.error.log
php_admin_value[disable_functions] = shell_exec.
`

i have a dedicated server which have 128 gb of ram and xeon processor with 1gbps bandwidth. I have installed centminmod on my server but the issue I am facing is that when my traffic raise to 3000 realtime the server stop responding.

What I am looking for what will be the configuration of php-fpm so it can handle 10000 realtime users
My configuration is this

`global]
; Log level
; Possible Values: alert, error, warning, notice, debug
; Default Value: notice
log_level = warning
pid = /var/run/php-fpm/php-fpm.pid
error_log = /var/log/php-fpm/www-error.log
emergency_restart_threshold = 10
emergency_restart_interval = 1m
process_control_timeout = 10s
include=/usr/local/nginx/conf/phpfpmd/*.conf

[www]
user = nginx
group = nginx

listen = 127.0.0.1:9000
listen.allowed_clients = 127.0.0.1
listen.backlog = 65535

;listen = /tmp/php5-fpm.sock
listen.owner = nginx
listen.group = nginx
listen.mode = 0660

pm = static
pm.max_children = 3072
; Default Value: min_spare_servers + (max_spare_servers - min_spare_servers) / 2
pm.start_servers = 100
pm.min_spare_servers = 10000
pm.max_spare_servers = 15000
pm.max_requests = 10000

; PHP 5.3.9 setting
; The number of seconds after which an idle process will be killed.
; Note: Used only when pm is set to 'ondemand'
; Default Value: 10s
pm.process_idle_timeout = 10s;

rlimit_files = 65536
rlimit_core = 0

; The timeout for serving a single request after which the worker process will
; be killed. This option should be used when the 'max_execution_time' ini option
; does not stop script execution for some reason. A value of '0' means 'off'.
; Available units: s(econds)(default), m(inutes), h(ours), or d(ays)
; Default Value: 0
;request_terminate_timeout = 0
; Default Value: 0
;request_slowlog_timeout = 0
slowlog = /var/log/php-fpm/www-slow.log

pm.status_path = /phpstatus
ping.path = /phpping
ping.response = pong

; Limits the extensions of the main script FPM will allow to parse. This can
; prevent configuration mistakes on the web server side. You should only limit
; FPM to .php extensions to prevent malicious users to use other extensions to
; exectute php code.
; Note: set an empty value to allow all extensions.
; Default Value: .php
security.limit_extensions = .php

; catch_workers_output = yes
php_admin_value[error_log] = /var/log/php-fpm/www-php.error.log
php_admin_value[disable_functions] = shell_exec.
`

your ram is quite high, what about your cpu ? as for my problem, it has nothing to do with ram, it's the cpu. and i kinda solve this one not only from setting the php setting, but also using horizontal scaling (ex: load balancer)

I am looking for best server configuration which can handle around 10000 real time traffic or any recommendation.

I am looking for best server configuration which can handle around 10000 real time traffic or any recommendation.

yeah, as for me i wouldn't tweak this setting too far in order to reach high traffic. i would consider using load balancer

Hi, I have 8GB RAM and 2vCPU (m5.large AWS) any idea to maximize server performance? Avg pool size is : 32MB
I have use these settings:
pm.max_children = 192
pm.start_servers = 100
pm.min_spare_servers = 100
pm.max_spare_servers = 175
pm.max_request = 2000
is this ok?

Guys! you must make everything auto/dynamic if you are running on top of Kubernetes cluster. When CPU or Memory reaches the thresholds, k8s HPA will do the work.
So now the question: how to make everything dynamic and open ?
Even nginx, I want to configure it like this :

worker_processes auto;
events {
  worker_connections  ??;
  multi_accept on;
  use epoll;
}

Any reason you put php-fpm7.0 in process name instead of php-fpm for the Determine system RAM and average pool size memory.
part ?

@bonyod whats your final config?

@bonyod whats your final config?

pm.max_children = 50
pm.start_servers = 12
pm.min_spare_servers = 5
pm.max_spare_servers = 25
pm.max_request = 0

But still not at best performance.

@bonyod the prop is called pm.max_requests not pm.max_request 👀

When setting it to zero you're going to run into memory issues if garbage collection of PHP or some framework isn't working properly. You can set the number quite high to 20 - 50k requests if your app is performing well with memory but zero isn't recommended.

@pultho what should be the ideal config for m5.large?

Hi @holmberd,

How did you came up with the values for the following?

pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 3

Is it safe to calculate it this way?

pm.start_servers = No. of cpu cores x 4
pm.min_spare_servers = No. of cpu cores x 2
pm.max_spare_servers = No. of cpu cores x 4

I have a t3.small server which has 2 cpus and 2 gb ram, is it okay to use the setting below with 'dynamic'?

Average Process = 70mb
RAM = 1900mb

pm.max_children = 27
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 3
pm.max_request = 1000

Hi, i did sudo nano on /etc/php/7.4/fpm/pool.d/www.conf

But the problem is I still get the same error log after increasing children to 17 i still get : WARNING: [pool www] server reached pm.max_children setting (5), consider raising it

Help!!

@thanksmia I suggest you use a more appropriate channel to ask for help, e.g. https://stackoverflow.com

@thanksmia this might be because your server is creating individual files for each virtual server / domain that you add. You need to check other files in the same folder of pool.d named as random numbers, and find the one that is for your server (it has the name of your server account on top of each file) and edit it as explained here: https://socalledhacker.com/index.php/2022/02/14/error-fpm-initialization-failed-solved/

@abdennour I'm not sure you'd really want to set anything to "auto", since Kubernetes doesn't "replace" the presented values for available CPU cores or RAM size for each pod, it only applies its limits and that's it; if you run free -h or check for the number of CPU cores inside a pod, you'll see that the reported numbers are for the actual node this pod is running inside of. Setting everything to "auto" will only lead to a scenario where your HPA scales everything, but your ingress doesn't know that those pods are already over their limit, leading to a snowball effect inside each pod, leading to OOM kills and broken pipes for that pod while PHP-FPM restarts or tries to respawn dead workers that have no mCPUs or RAM available to work with (and the HPA starts going crazy, scaling to the max since from its POV, the scaling isn't solving the excessive resource consumption for some of the pods)