Example: Parallel processing in PHP using pcntl_fork()

fork-example.php

<?php

/**
 * @file
 * Basic demonstration of how to do parallel threads in PHP.
 */

// This array of "tasks" could be anything. For demonstration purposes
// these are just strings, but they could be a callback, class or
// include file (hell, even code-as-a-string to pass to eval()).
$tasks = [
  "fetch_remote_data",
  "post_async_updates",
  "clear_caches",
  "notify_admin",
];

// This loop creates a new fork for each of the items in $tasks.
foreach ($tasks as $task) {
  $pid = pcntl_fork();

  if ($pid == -1) {
    exit("Error forking...\n");
  }
  else if ($pid == 0) {
    execute_task($task);
    exit();
  }
}

// This while loop holds the parent process until all the child threads
// are complete - at which point the script continues to execute.
while(pcntl_waitpid(0, $status) != -1);

// You could have more code here.
echo "Do stuff after all parallel execution is complete.\n";

/**
 * Helper method to execute a task.
 */
function execute_task($task_id) {
  echo "Starting task: ${task_id}\n";

  // Simulate doing actual work with sleep().
  $execution_time = rand(5, 10);
  sleep($execution_time);

  echo "Completed task: ${task_id}. Took ${execution_time} seconds.\n";
}

Well explained , thank you

Nice

First: thank you!
Is this stops after 99. on a 500+ task list just me?

Question. If a mysql resource is created in the scope of the parent process, can it be used in the child worker process? For example, if above line 17, you had created a $mysqli = new mysqli(...); would you be able to access that db handle inside the execute_task function either by 'global $mysqli' or by passing it in as a parameter to the execute_task function?

Great tutorial! Thanks.

fantastic

muchas gracias muy útil el código

My problem when i for loop the pcntl_fork its repeat the $i
like
$i 0
it duplicated
how i can pass this problem

Question. If a mysql resource is created in the scope of the parent process, can it be used in the child worker process? For example, if above line 17, you had created a $mysqli = new mysqli(...); would you be able to access that db handle inside the execute_task function either by 'global $mysqli' or by passing it in as a parameter to the execute_task function?

Great tutorial! Thanks.

Yes, you can.

not work for me in the method of class. cause it try to execute deconstructors in every loop, so i got alot errors abous sql had gone away
and
Uncaught PDOException: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away

thats all folks !
p.s. use Amphp, its better way !

not work for me in the method of class. cause it try to execute deconstructors in every loop, so i got alot errors abous sql had gone away
and
Uncaught PDOException: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away

thats all folks !
p.s. use Amphp, its better way !

you must connect to database again in child process, this one lost connection during the fork.

Great tutorial , thank you

Here is my improved version
Also posted as a note on https://www.php.net/manual/en/function.pcntl-fork.php

<?php

declare(strict_types = 1);

/**
 * Helper method to execute a task
 */
function execute_task(int $task_id): void
{
    echo 'Starting task: ' . $task_id . PHP_EOL;

    // Simulate doing actual work with sleep()
    $execution_time = rand(5, 10);
    sleep($execution_time);

    echo "Completed task: ${task_id}. Took ${execution_time} seconds.\n";
}

/**
 * Builds a list of tasks
 */
function generator(): Generator
{
    $item_count = 50;
    for ($i = 1; $i <= $item_count; $i++) {
        yield $i;
    }
}

/**
 * Starts the work
 */
function launch(): void
{
    $processCount = 0;
    $status       = null;
    echo 'Running as pid: ' . getmypid() . PHP_EOL;

    $taskList = generator();
    do {
        echo 'Still tasks to do' . PHP_EOL;

        if ($processCount >= 5) {
            echo 'Waiting, currently running: ' . $processCount . PHP_EOL;
            pcntl_wait($status);
            $processCount--;
            continue;
        }

        echo 'Tasks running: ' . $processCount . PHP_EOL;
        $task = $taskList->current();
        $taskList->next();
        $processCount++;

        $pid = pcntl_fork();

        if ($pid === -1) {
            exit('Error forking...' . PHP_EOL);
        }

        if ($pid === 0) {
            $processList[] = getmypid();
            echo 'Running task as pid: ' . getmypid() . PHP_EOL;
            execute_task($task);
            exit();
        }
    } while ($taskList->valid());

    $waitForChildrenToFinish = true;
    while ($waitForChildrenToFinish) {
        // This while loop holds the parent process until all the child threads
        // are complete - at which point the script continues to execute.
        if (pcntl_waitpid(0, $status) === -1) {
            echo 'parallel execution is complete' . PHP_EOL;
            $waitForChildrenToFinish = false;
        }
    }

    // Code to run after all tasks are processed
}

launch();