Created
February 4, 2020 23:35
-
-
Save canimus/1be38665594014a089df2371c2137c67 to your computer and use it in GitHub Desktop.
Custom delimiter file reader
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def DataStoreFileReader(file, chunk_size=512, lineterminator=172, delimiter=171): | |
| chunk = "" | |
| while True: | |
| curr = file.read(chunk_size) | |
| chunk += curr | |
| if not curr: | |
| break | |
| if chr(lineterminator) in chunk: | |
| lines = chunk.split(chr(lineterminator)) | |
| for line in lines[0:-1]: | |
| yield line | |
| chunk = lines[-1] |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment