Skip to content

Instantly share code, notes, and snippets.

@nicksantamaria
Created October 20, 2016 22:35
Show Gist options
  • Save nicksantamaria/21dce5ff2a6640cdff76ce7bc57d2981 to your computer and use it in GitHub Desktop.
Save nicksantamaria/21dce5ff2a6640cdff76ce7bc57d2981 to your computer and use it in GitHub Desktop.
Example: Parallel processing in PHP using pcntl_fork()
<?php
/**
* @file
* Basic demonstration of how to do parallel threads in PHP.
*/
// This array of "tasks" could be anything. For demonstration purposes
// these are just strings, but they could be a callback, class or
// include file (hell, even code-as-a-string to pass to eval()).
$tasks = [
"fetch_remote_data",
"post_async_updates",
"clear_caches",
"notify_admin",
];
// This loop creates a new fork for each of the items in $tasks.
foreach ($tasks as $task) {
$pid = pcntl_fork();
if ($pid == -1) {
exit("Error forking...\n");
}
else if ($pid == 0) {
execute_task($task);
exit();
}
}
// This while loop holds the parent process until all the child threads
// are complete - at which point the script continues to execute.
while(pcntl_waitpid(0, $status) != -1);
// You could have more code here.
echo "Do stuff after all parallel execution is complete.\n";
/**
* Helper method to execute a task.
*/
function execute_task($task_id) {
echo "Starting task: ${task_id}\n";
// Simulate doing actual work with sleep().
$execution_time = rand(5, 10);
sleep($execution_time);
echo "Completed task: ${task_id}. Took ${execution_time} seconds.\n";
}
@ahmadmarafa
Copy link

Well explained , thank you

@fenriz07
Copy link

Nice

@harkalygergo
Copy link

First: thank you!
Is this stops after 99. on a 500+ task list just me?

@filmo
Copy link

filmo commented May 22, 2018

Question. If a mysql resource is created in the scope of the parent process, can it be used in the child worker process? For example, if above line 17, you had created a $mysqli = new mysqli(...); would you be able to access that db handle inside the execute_task function either by 'global $mysqli' or by passing it in as a parameter to the execute_task function?

Great tutorial! Thanks.

@silasmagare
Copy link

fantastic

@SergioAraya
Copy link

muchas gracias muy útil el código

@th30c0der
Copy link

My problem when i for loop the pcntl_fork its repeat the $i
like
$i 0
it duplicated
how i can pass this problem

@f14a2cd
Copy link

f14a2cd commented Aug 13, 2020

Question. If a mysql resource is created in the scope of the parent process, can it be used in the child worker process? For example, if above line 17, you had created a $mysqli = new mysqli(...); would you be able to access that db handle inside the execute_task function either by 'global $mysqli' or by passing it in as a parameter to the execute_task function?

Great tutorial! Thanks.

Yes, you can.

@v0ff4k
Copy link

v0ff4k commented Mar 6, 2021

not work for me in the method of class. cause it try to execute deconstructors in every loop, so i got alot errors abous sql had gone away
and
Uncaught PDOException: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away

thats all folks !
p.s. use Amphp, its better way !

@alexalouit
Copy link

not work for me in the method of class. cause it try to execute deconstructors in every loop, so i got alot errors abous sql had gone away
and
Uncaught PDOException: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away

thats all folks !
p.s. use Amphp, its better way !

you must connect to database again in child process, this one lost connection during the fork.

@chung362
Copy link

Great tutorial , thank you

@williamdes
Copy link

Here is my improved version
Also posted as a note on https://www.php.net/manual/en/function.pcntl-fork.php

<?php

declare(strict_types = 1);

/**
 * Helper method to execute a task
 */
function execute_task(int $task_id): void
{
    echo 'Starting task: ' . $task_id . PHP_EOL;

    // Simulate doing actual work with sleep()
    $execution_time = rand(5, 10);
    sleep($execution_time);

    echo "Completed task: ${task_id}. Took ${execution_time} seconds.\n";
}

/**
 * Builds a list of tasks
 */
function generator(): Generator
{
    $item_count = 50;
    for ($i = 1; $i <= $item_count; $i++) {
        yield $i;
    }
}

/**
 * Starts the work
 */
function launch(): void
{
    $processCount = 0;
    $status       = null;
    echo 'Running as pid: ' . getmypid() . PHP_EOL;

    $taskList = generator();
    do {
        echo 'Still tasks to do' . PHP_EOL;

        if ($processCount >= 5) {
            echo 'Waiting, currently running: ' . $processCount . PHP_EOL;
            pcntl_wait($status);
            $processCount--;
            continue;
        }

        echo 'Tasks running: ' . $processCount . PHP_EOL;
        $task = $taskList->current();
        $taskList->next();
        $processCount++;

        $pid = pcntl_fork();

        if ($pid === -1) {
            exit('Error forking...' . PHP_EOL);
        }

        if ($pid === 0) {
            $processList[] = getmypid();
            echo 'Running task as pid: ' . getmypid() . PHP_EOL;
            execute_task($task);
            exit();
        }
    } while ($taskList->valid());

    $waitForChildrenToFinish = true;
    while ($waitForChildrenToFinish) {
        // This while loop holds the parent process until all the child threads
        // are complete - at which point the script continues to execute.
        if (pcntl_waitpid(0, $status) === -1) {
            echo 'parallel execution is complete' . PHP_EOL;
            $waitForChildrenToFinish = false;
        }
    }

    // Code to run after all tasks are processed
}

launch();

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment