Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Python script for decoding NCSBE voter data
#!/usr/bin/env python3
"""
streamncv.py
============
Translates from latin-1 tab-separated-values to UTF-8 comma-separated values
on the fly, without trying to allocate the entire file.
It also prints progress updates.
"""
import csv
import io
import sys
latin1stdin = io.TextIOWrapper(sys.stdin.buffer, 'latin1')
reader = csv.reader(latin1stdin, dialect='excel-tab')
writer = csv.writer(sys.stdout, dialect='excel')
n = 0
for row in reader:
writer.writerow([c.strip() for c in row])
n += 1
if n % 1000 == 0:
sys.stderr.write("\r{}".format(n))
sys.stderr.flush()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.