Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Adjusting child processes for PHP-FPM (Nginx)

Adjusting child processes for PHP-FPM (Nginx)

When setting these options consider the following:

  • How long is your average request?
  • What is the maximum number of simultaneous visitors the site(s) get?
  • How much memory on average does each child process consume?

Determine if the max_children limit has been reached.

  • sudo grep max_children /var/log/php?.?-fpm.log.1 /var/log/php?.?-fpm.log

Determine system RAM and average pool size memory.

  • free -h
  • All fpm processes: ps -ylC php-fpm7.0 --sort:rss
  • Average memory: ps --no-headers -o "rss,cmd" -C php-fpm7.0 | awk '{ sum+=$1 } END { printf ("%d%s\n", sum/NR/1024,"M") }'
  • All fpm processes memory: ps -eo size,pid,user,command --sort -size | awk '{ hr=$1/1024 ; printf("%13.2f Mb ",hr) } { for ( x=4 ; x<=NF ; x++ ) { printf("%s ",$x) } print "" }' | grep php-fpm

Calculate max_children

Based on RAM

  • pm.max_children = Total RAM dedicated to the web server / Max child process size

  • System RAM: 2GB

  • Average Pool size: 85Mb

  • pm.max_children = 1500MB / 85MB = 17

Based on average script execution time

  • max_children = (average PHP script execution time) * (PHP requests per second)
  • visitors = max_children * (seconds between page views) / (avg. execution time)

Configure

sudo vim /etc/php/7.0/fpm/pool.d/www.conf

pm.max_children = 17
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 3
pm.max_request = 1000
; Choose how the process manager will control the number of child processes.
; Possible Values:
;   static  - a fixed number (pm.max_children) of child processes;
;   dynamic - the number of child processes are set dynamically based on the
;             following directives:
;             pm.max_children      - the maximum number of children that can
;                                    be alive at the same time.
;
;             pm.start_servers     - the number of children created on startup.
;                                    this value must not be less than min_spare_servers 
;                                    and not greater than max_spare_servers.
;
;             pm.min_spare_servers - the minimum number of children in 'idle'
;                                    state (waiting to process). If the number
;                                    of 'idle' processes is less than this
;                                    number then some children will be created.
;
;             pm.max_spare_servers - the maximum number of children in 'idle'
;                                    state (waiting to process). If the number
;                                    of 'idle' processes is greater than this
;                                    number then some children will be killed.
; Note: This value is mandatory.
@Yv-o-rL

This comment has been minimized.

Copy link

@Yv-o-rL Yv-o-rL commented Jan 25, 2019

Hi!

I'm trying to get more information on how PHP children work. I can't find any documentation, blog or anything which would give general answers. I was wondering if you could help me out.

Does one PHP worker equal to one CPU core? I mean having 64 workers on a 32 core VM is unnecessary since they can only use a maximum of 32 cores?

What does a request include (pm.max_requests)? An Nginx request like "index.php"?

When I check PHP status and see "active processes", what's that? Is that one request? One PHP function? A call to a MySQL DB? A PHP worker?

How does PHP handle the Nginx requests coming in regarding workers? I mean on a high level. Is there a queue? Does it depend on the functions, extensions in the PHP file? For example, having 2 PHP children and opening one "index.php" every second when the processing time is .9 seconds. What happens?

When does FPM log lines like the one below? When all the workers have stuff to do or when there's a request which can't be fulfilled since all workers are busy? As you see my questions above, I don't know how the requests(one dynamic file)->manual functions (calculate_cart_items_shipping_cost())->PHP functions(round(3.14159)) work in this regard.

WARNING: [pool www] server reached pm.max_children setting (5), consider raising it.

Thank you!

@holmberd

This comment has been minimized.

Copy link
Owner Author

@holmberd holmberd commented Apr 12, 2019

@Yv-o-rL one php-fpm child process handles one request.

WARNING: [pool www] server reached pm.max_children setting (5), consider raising it.

The warning means that you are hitting the upper limit of how many request you can handle with your current configuration. You have more request coming in than you have workers available to handle the requests. The request will be queued until a worker becomes available to handle the request. Reason for this is that your pm.max_children is set too low, i.e. you have more requests than the number of workers can handle.

@endrureza

This comment has been minimized.

Copy link

@endrureza endrureza commented Jun 10, 2019

is it not better to set pm as static for high traffic web ? i mean, use dynamic will be getting confused to the setting. Cause of the adjustment of pm.max_children , pm.start_servers, pm.min_spare_servers and pm.max_spare_servers.

@holmberd

This comment has been minimized.

Copy link
Owner Author

@holmberd holmberd commented Jun 10, 2019

@endrureza neither is better than the other. Each one can have a better fit for a particular use-case, and it's up to the developer of that system to consider the trade-offs.

Static

  • Each process sits in memory at all times, how much memory is consumed will vary for each server.
  • CPU experiences less load spikes when traffic spikes; since there is no overhead in spawning new processes.

Dynamic

  • Spawns new processes when traffic load increases.
  • Spawning processes will put some load on the CPU.
  • Frees up memory that can be used by other services when traffic is low.
@synetalsolutions

This comment has been minimized.

Copy link

@synetalsolutions synetalsolutions commented Jan 18, 2020

i have a dedicated server which have 128 gb of ram and xeon processor with 1gbps bandwidth. I have installed centminmod on my server but the issue I am facing is that when my traffic raise to 3000 realtime the server stop responding.

What I am looking for what will be the configuration of php-fpm so it can handle 10000 realtime users
My configuration is this

`global]
; Log level
; Possible Values: alert, error, warning, notice, debug
; Default Value: notice
log_level = warning
pid = /var/run/php-fpm/php-fpm.pid
error_log = /var/log/php-fpm/www-error.log
emergency_restart_threshold = 10
emergency_restart_interval = 1m
process_control_timeout = 10s
include=/usr/local/nginx/conf/phpfpmd/*.conf

[www]
user = nginx
group = nginx

listen = 127.0.0.1:9000
listen.allowed_clients = 127.0.0.1
listen.backlog = 65535

;listen = /tmp/php5-fpm.sock
listen.owner = nginx
listen.group = nginx
listen.mode = 0660

pm = static
pm.max_children = 3072
; Default Value: min_spare_servers + (max_spare_servers - min_spare_servers) / 2
pm.start_servers = 100
pm.min_spare_servers = 10000
pm.max_spare_servers = 15000
pm.max_requests = 10000

; PHP 5.3.9 setting
; The number of seconds after which an idle process will be killed.
; Note: Used only when pm is set to 'ondemand'
; Default Value: 10s
pm.process_idle_timeout = 10s;

rlimit_files = 65536
rlimit_core = 0

; The timeout for serving a single request after which the worker process will
; be killed. This option should be used when the 'max_execution_time' ini option
; does not stop script execution for some reason. A value of '0' means 'off'.
; Available units: s(econds)(default), m(inutes), h(ours), or d(ays)
; Default Value: 0
;request_terminate_timeout = 0
; Default Value: 0
;request_slowlog_timeout = 0
slowlog = /var/log/php-fpm/www-slow.log

pm.status_path = /phpstatus
ping.path = /phpping
ping.response = pong

; Limits the extensions of the main script FPM will allow to parse. This can
; prevent configuration mistakes on the web server side. You should only limit
; FPM to .php extensions to prevent malicious users to use other extensions to
; exectute php code.
; Note: set an empty value to allow all extensions.
; Default Value: .php
security.limit_extensions = .php

; catch_workers_output = yes
php_admin_value[error_log] = /var/log/php-fpm/www-php.error.log
php_admin_value[disable_functions] = shell_exec.
`

@endrureza

This comment has been minimized.

Copy link

@endrureza endrureza commented Jan 19, 2020

i have a dedicated server which have 128 gb of ram and xeon processor with 1gbps bandwidth. I have installed centminmod on my server but the issue I am facing is that when my traffic raise to 3000 realtime the server stop responding.

What I am looking for what will be the configuration of php-fpm so it can handle 10000 realtime users
My configuration is this

`global]
; Log level
; Possible Values: alert, error, warning, notice, debug
; Default Value: notice
log_level = warning
pid = /var/run/php-fpm/php-fpm.pid
error_log = /var/log/php-fpm/www-error.log
emergency_restart_threshold = 10
emergency_restart_interval = 1m
process_control_timeout = 10s
include=/usr/local/nginx/conf/phpfpmd/*.conf

[www]
user = nginx
group = nginx

listen = 127.0.0.1:9000
listen.allowed_clients = 127.0.0.1
listen.backlog = 65535

;listen = /tmp/php5-fpm.sock
listen.owner = nginx
listen.group = nginx
listen.mode = 0660

pm = static
pm.max_children = 3072
; Default Value: min_spare_servers + (max_spare_servers - min_spare_servers) / 2
pm.start_servers = 100
pm.min_spare_servers = 10000
pm.max_spare_servers = 15000
pm.max_requests = 10000

; PHP 5.3.9 setting
; The number of seconds after which an idle process will be killed.
; Note: Used only when pm is set to 'ondemand'
; Default Value: 10s
pm.process_idle_timeout = 10s;

rlimit_files = 65536
rlimit_core = 0

; The timeout for serving a single request after which the worker process will
; be killed. This option should be used when the 'max_execution_time' ini option
; does not stop script execution for some reason. A value of '0' means 'off'.
; Available units: s(econds)(default), m(inutes), h(ours), or d(ays)
; Default Value: 0
;request_terminate_timeout = 0
; Default Value: 0
;request_slowlog_timeout = 0
slowlog = /var/log/php-fpm/www-slow.log

pm.status_path = /phpstatus
ping.path = /phpping
ping.response = pong

; Limits the extensions of the main script FPM will allow to parse. This can
; prevent configuration mistakes on the web server side. You should only limit
; FPM to .php extensions to prevent malicious users to use other extensions to
; exectute php code.
; Note: set an empty value to allow all extensions.
; Default Value: .php
security.limit_extensions = .php

; catch_workers_output = yes
php_admin_value[error_log] = /var/log/php-fpm/www-php.error.log
php_admin_value[disable_functions] = shell_exec.
`

your ram is quite high, what about your cpu ? as for my problem, it has nothing to do with ram, it's the cpu. and i kinda solve this one not only from setting the php setting, but also using horizontal scaling (ex: load balancer)

@synetalsolutions

This comment has been minimized.

Copy link

@synetalsolutions synetalsolutions commented Jan 19, 2020

I am looking for best server configuration which can handle around 10000 real time traffic or any recommendation.

@endrureza

This comment has been minimized.

Copy link

@endrureza endrureza commented Jan 20, 2020

I am looking for best server configuration which can handle around 10000 real time traffic or any recommendation.

yeah, as for me i wouldn't tweak this setting too far in order to reach high traffic. i would consider using load balancer

@bonyod

This comment has been minimized.

Copy link

@bonyod bonyod commented Feb 14, 2020

Hi, I have 8GB RAM and 2vCPU (m5.large AWS) any idea to maximize server performance? Avg pool size is : 32MB
I have use these settings:
pm.max_children = 192
pm.start_servers = 100
pm.min_spare_servers = 100
pm.max_spare_servers = 175
pm.max_request = 2000
is this ok?

@abdennour

This comment has been minimized.

Copy link

@abdennour abdennour commented Jul 6, 2020

Guys! you must make everything auto/dynamic if you are running on top of Kubernetes cluster. When CPU or Memory reaches the thresholds, k8s HPA will do the work.
So now the question: how to make everything dynamic and open ?
Even nginx, I want to configure it like this :

worker_processes auto;
events {
  worker_connections  ??;
  multi_accept on;
  use epoll;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.