Skip to content

Instantly share code, notes, and snippets.

@williballenthin

williballenthin/mmap-readlines.py

Created January 28, 2021 19:41
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save williballenthin/ca629ec9530bfe3264d7a06bd1de3cec to your computer and use it in GitHub Desktop.
Save williballenthin/ca629ec9530bfe3264d7a06bd1de3cec to your computer and use it in GitHub Desktop.
Download ZIP
enumerate the lines of a (utf-8) file incrementally via mmap
Raw
mmap-readlines.py
import mmap
def lines(m):
line = m.readline()
while line:
yield line.decode("utf-8").rstrip("\n")
line = m.readline()
def filelines(path):
with open(path, "rb") as f:
return lines(mmap.mmap(f.fileno(), 0, prot=mmap.PROT_READ))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment