Sqlalchemy result batching for inserts

batched_results.py

def batch_results(results, batch_size=10000):
  batch = results.fetchmany(batch_size)
  while batch:
    yield batch
    batch = results.fetchmany(batch_size)
    
    
for batch in batch_results(conn.execute(query)):
  new_rows = [dict(row) for row in batch]
  # Modify new_rows here, if needed
  conn.execute(table.insert(), new_rows)

I came up with this solution that is shorter and delegates more work to Python. This uses of the two-argument, sentinel usage of iter and the fact that fetchmany returns an empty list when all rows have been exhausted:

def chunk_results(results, size):
    """
    Yields chunks of a SQL select result.
    Arguments:
        results (sqlalchemy.ResultProxy): Execution result
        size (int): Chunk size
    Yields:
        List[sqlalchemy.RowProxy]: List of size results.
    """
    yield from iter(lambda: results.fetchmany(size=size), [])