Skip to content

Instantly share code, notes, and snippets.

@scruss-elmwood
Created June 30, 2022 19:02
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save scruss-elmwood/f2329b217772816e3a82df292ec17db4 to your computer and use it in GitHub Desktop.
Save scruss-elmwood/f2329b217772816e3a82df292ec17db4 to your computer and use it in GitHub Desktop.
csvthing - clean up horrid csv so it will work with q and SQLite
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# csvthing - clean up horrid csv so it will work with q and SQLite
# scruss, 2022-06
import sys
import csv
if len(sys.argv) < 2:
print("need an input and output file argument")
exit()
with open(sys.argv[1], newline='') as csvfile:
reader = csv.reader(csvfile, dialect='excel')
with open(sys.argv[2], "w", newline='') as outfile:
writer = csv.writer(outfile, dialect='excel')
for (i, row) in enumerate(reader):
if i == 0:
# do something with first line to make it SQLite friendly
newrow = []
for (j, item) in enumerate(row):
# remove iffy chars from header field
newitem = item.replace(" ", "_").replace(",", "")
# make sure every field has a non-blank name
if newitem == "":
newitem = "Field"+str(j)
newrow.append(newitem)
writer.writerow(newrow)
else:
writer.writerow(row)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment