A couple of month ago we ran into an issue with hanging PHP processes during/after HTTP calls.
Today another friend of mine ran into that issue, so here some notes in the hopes that this might be helpful to someone :)
strace the hanging process and see if it hangs in rt_sigaction
sudo strace -ttfp $pid
11:25:04.663911 rt_sigaction(SIGPIPE, NULL, {sa_handler=SIG_IGN, sa_mask=[PIPE], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7f763229e090}, 8) = 0
11:25:04.663974 rt_sigaction(SIGPIPE, {sa_handler=SIG_IGN, sa_mask=[PIPE], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7f763229e090}, NULL, 8) = 0
11:25:04.664026 rt_sigaction(SIGPIPE, {sa_handler=SIG_IGN, sa_mask=[PIPE], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7f763229e090}, NULL, 8) = 0
And/or confirm via gdb
If you don't have debug symbols installed, this will look differently:
sudo gdb -p $pid bt
(gdb) bt
#0 0x00007fbfef531166 in __GI___libc_sigaction (sig=sig@entry=13, act=act@entry=0x7ffd98161290, oact=oact@entry=0x0) at ../sysdeps/unix/sysv/linux/sigaction.c:58
#1 0x00007fbfef531299 in __GI___sigaction (sig=sig@entry=13, act=act@entry=0x7ffd98161290, oact=oact@entry=0x0) at ../nptl/sigaction.c:30
#2 0x00007fbfece84f2a in sigpipe_ignore (ig=ig@entry=0x7ffd98161340, data=<optimized out>) at sigpipe.h:56
#3 0x00007fbfece85a42 in sigpipe_ignore (ig=0x7ffd98161340, data=<optimized out>) at sigpipe.h:48
#4 Curl_conncache_close_all_connections (connc=connc@entry=0x55863df2b750) at conncache.c:564
#5 0x00007fbfece70a5e in curl_share_cleanup (share=0x55863df2b730) at share.c:198
#6 0x00007fbfed42e1f1 in curl_share_free_obj (object=0x7fbfe91c5ba8) at ./ext/curl/share.c:160
#7 0x000055863c548833 in zend_objects_store_free_object_storage (objects=objects@entry=0x55863c737988 <executor_globals+840>, fast_shutdown=fast_shutdown@entry=true) at ./Zend/zend_objects_API.c:109
#8 0x000055863c4a7b1a in zend_shutdown_executor_values (fast_shutdown=fast_shutdown@entry=true) at ./Zend/zend_execute_API.c:386
#9 0x000055863c4a7f93 in shutdown_executor () at ./Zend/zend_execute_API.c:403
#10 0x000055863c4b7eb3 in zend_deactivate () at ./Zend/zend.c:1271
#11 0x000055863c453c56 in php_request_shutdown (dummy=<optimized out>) at ./main/main.c:1847
#12 0x000055863c59fd80 in do_cli (argc=10, argv=0x55863db55150) at ./sapi/cli/php_cli.c:1135
#13 0x000055863c2f6f68 in main (argc=10, argv=0x55863db55150) at ./sapi/cli/php_cli.c:1367
A bit of digging led me to the following:
curl/curl@c069027#diff-daf91468c288f8c97776ec9805e160a8a3bcb52e0001103d35b2079b7e842dd4R543 & curl/curl#4915 GCP issue on the topic: googleapis/google-cloud-cpp#3860
Links from along the way:
- We're in curl_share_free_obj suggest that CURLOPT_SHARE is used
- Curl share init is called here: https://github.com/symfony/http-client/blob/6.1/Internal/CurlClientState.php#L94
- CURLOPT_SHARE is set here: https://github.com/symfony/http-client/blob/6.1/CurlHttpClient.php#L294
- Related curl https://github.com/curl/curl/blob/7632c0d25ad797a8f17ff071f3c4e9fe8a73a398/lib/share.c#L193-L194 & https://github.com/curl/curl/blob/c616259bd1a94fd6b0a39ace9f68a3616f77383c/lib/conncache.c#L540
- https://github.com/curl/curl/blob/6d90308a39dd5ca699cb9db6d73f687b7639a340/lib/conncache.c#L539-L548
- This is the call we see in the
stracehttps://github.com/curl/curl/blob/curl-7_68_0/lib/conncache.c#L564 - https://github.com/symfony/http-client/commit/653dc41a26bd34f06bae25ab89532df938e8c9c5
Updating to a new libcurl version solved the issue.
To test this for only one PHP process.
Build curl:
wget https://github.com/curl/curl/releases/download/curl-7_87_0/curl-7.87.0.tar.gz
tar -zxf curl-7_87_0.tar.gz
cd curl-7.87.0/
./buildconf
./configure --with-openssl
make
From there, you can run PHP with the new Curl version:
LD_LIBRARY_PATH="/home/edo/curl/lib/.libs/" php -r 'var_dump(curl_version()["version"]);'
string(10) "7.87.0-DEV"
For us, the problems went away with this updated version so we could roll it out on our systems.