Quick and dirty script to parse a Supreme Court transcript and output a JSON file of [speaker, words]

parse_scotus.py

import re
import json

ws_re = re.compile("\s+")
line_num_re = re.compile("\s\d+\s{2,}", re.M)

# first, pdftotext -layout <pdf> <text>
with open("12-307_jnt1.txt", "r") as f:
    data = f.read()


exclude = (
    "Alderson Reporting Company",
    "Official - Subject to Final Review",
)

data = re.sub(line_num_re, "", data)

for xc in exclude:
    data = data.replace(xc, "")

data = re.sub(ws_re, " ", data)

data_split = re.split('([A-Z+.]{3,} [A-Z ]+):', data)
del data_split[0]
pairs = zip(data_split[0::2], data_split[1::2])

js = json.dumps(pairs)

with open("sc-doma.json", "w") as f:
    f.write(js)

After you dump the text with pdftotext, you'll want to clear away anything that's not direct conversation (the top of the doc, the bottom of the doc, and a few ALL CAP PHRASES in the middle of the text). Then run the script and voila.