Skip to content

Instantly share code, notes, and snippets.

@companje
Created March 29, 2022 16:44
Show Gist options
  • Save companje/72ddf8f4ddba271580af2a55f62bcfad to your computer and use it in GitHub Desktop.
Save companje/72ddf8f4ddba271580af2a55f62bcfad to your computer and use it in GitHub Desktop.
Create text-files with unique values per column in a CSV
#!/usr/bin/env python3
import csv,sys,json
from sys import argv
from collections import defaultdict
if len(argv)!=3:
sys.exit("Usage: "+argv[0]+" input.csv output_folder")
input_filename = argv[1]
output_folder = argv[2]
# create a dict with unique values per column
unique_values_per_column = defaultdict(dict)
for row in csv.DictReader(open(input_filename,encoding='utf-8'), delimiter=";"):
cols = dict(row)
for key,val in cols.items():
unique_values_per_column[key][val] = "x"
# write unique values to a file per column_name
for col_name,values in unique_values_per_column.items():
with open(output_folder + "/" + col_name + ".txt", "w") as f:
for val in sorted(list(values.keys())):
print(val,file=f)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment