Skip to content

Instantly share code, notes, and snippets.

@IHosseini083
Last active May 13, 2023 13:19
Show Gist options
  • Save IHosseini083/6bb7fd1cbdf208254165e58c2a74c442 to your computer and use it in GitHub Desktop.
Save IHosseini083/6bb7fd1cbdf208254165e58c2a74c442 to your computer and use it in GitHub Desktop.
Iterate over an iterable in chunks of n elements using generator function.
import itertools
from typing import Iterable, Iterator, Tuple, TypeVar
_T = TypeVar("_T")
def chunks(iterable: Iterable[_T], size: int) -> Iterator[Tuple[_T, ...]]:
"""Iterate over an iterable in chunks of n elements.
Parameters:
iterable (`Iterable`): The iterable to iterate over.
size (`int`): The size of each chunk.
Yields:
`Tuple`: A tuple of n elements from the iterable.
"""
if not isinstance(size, int) or size <= 0:
raise ValueError("size must be an integer greater than 0")
it = iter(iterable)
# itertools.islice is lazy so we used tuple() to force the evaluation:
while chunk := tuple(itertools.islice(it, size)):
# `chunk` is an empty tuple if the iterator is exhausted, therefore the `while`
# condition is not met at the end of the iteration.
yield chunk # Yield the next chunk here.
if __name__ == "__main__":
for chunk in chunks(["a", "b", "c", "d"], 2):
print(chunk)
# ('a', 'b')
# ('c', 'd')
@mahdihaghverdi
Copy link

Hello I'd like to suggest a code change:

it = iter(iterable)
while chunk := tuple(itertools.islice(it, size)):
    # itertools.islice is lazy so we need to use tuple.
    # `chunk` is an empty tuple if the iterable is exhausted! 
    yield chunk  # Yield the next chunk here.

I like this one more :))

@IHosseini083
Copy link
Author

Hello I'd like to suggest a code change:

it = iter(iterable)
while chunk := tuple(itertools.islice(it, size)):
    # itertools.islice is lazy so we need to use tuple.
    # `chunk` is an empty tuple if the iterable is exhausted! 
    yield chunk  # Yield the next chunk here.

I like this one more :))

Hey there! thanks for the review bro, I also like this one better :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment