Skip to content

Instantly share code, notes, and snippets.

@MattFaus
MattFaus / SortedGcsCsvShardFileMergeReader.py
Last active February 23, 2022 11:58
Merge-reads several sorted .csv files stored on Google Cloud Storage.
class SortedGcsCsvShardFileMergeReader(object):
"""Merges several sorted .csv files stored on GCS.
This class is both an iterator and a context manager.
Let's say there are 2 .csv files stored on GCS, with contents like:
/bucket/file_1.csv:
[0, "Matt"],
[0, "Sam"],