Fetch WP "posts" in chunks.

functions.php

<?php
/**
 * Fetch posts in chunks.
 *
 * This function fetches posts in chunks of a specified size, which can be useful for large datasets.
 *
 * @param string $post_type The type of post to fetch. Default is 'post'.
 * @param int $posts_per_page The number of posts to fetch per page. Default is 100.
 * @param array $custom_args Custom arguments to pass to WP_Query. Default is an empty array.
 *
 * @return array An array of all fetched posts.
 */
function tprt_fetch_posts_in_chunks( $post_type = 'post', $posts_per_page = 100, $custom_args = [] ) {
  global $post;

  // Initialize the page number and the array to hold all posts.
  $paged = 1;
  $all_posts = array();

  // Keep fetching posts until there are no more posts to fetch.
  while ( true ) :
    // Merge the default arguments with the custom arguments.
    $args = array_merge(
      array(
        'post_type'      => $post_type,
        'posts_per_page' => $posts_per_page,
        'paged'          => $paged,
      ),
      $custom_args
    );

    // Create a new WP_Query with the merged arguments.
    $query = new WP_Query( $args );

    // If there are no posts, break the loop.
    if ( ! $query->have_posts() ) :
      break;
    endif;

    // Add each post to the all_posts array.
    while ( $query->have_posts() ) : $query->the_post();
      $all_posts[] = $post;
    endwhile;

    // Reset post data to ensure that the global $post variable has the correct data in the next iteration of the loop.
    wp_reset_postdata();

    // Increment the page number for the next iteration of the loop.
    $paged++;
  endwhile;

  // Return the array of all fetched posts.
  return $all_posts;
}

It is considered to be bad practice to use the following parameter:

'posts_per_page' => -1

This should more efficiently get all the "posts".

One should consider storing the results in a transient as well.

Adding an example for folks who would like to see how to use this function.

$custom_args = array(
    'orderby' => 'title',
    'order'   => 'ASC',
);
$all_posts = tprt_fetch_posts_in_chunks('post', 50, $custom_args);

$all_posts is an array of post objects