Skip to content

Instantly share code, notes, and snippets.

@brentajones
Created October 23, 2020 16:37
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save brentajones/2768a998503ecd3c0732d30fd3fa72a2 to your computer and use it in GitHub Desktop.
Save brentajones/2768a998503ecd3c0732d30fd3fa72a2 to your computer and use it in GitHub Desktop.
Working with Missouri Department of Corrections data

Working with Missouri DOC data:

This is for working with the Missouri Department of Corrections offender database.

The ZIP file is some binary format that needs to be converted before using:

strings ~/Downloads/SunshineLawFak930.txt > doc-data.txt

It's a fixed-width file. Here's the schema as of 22 October 2020:

column,start,length
doc_id,1,8
last_name,9,18
first_name,27,12
middle_name,39,12
suffix_name,51,3
race,54,30
sex,84,30
birth_date,114,8
offender_assigned_place,122,8
doc_loc_func_flag,130,1
cause_no,131,20
offense_county,151,4
sentence_county,155,4
ncic_code,159,4
missouri_charge,163,8
offense_description,171,74
completed_flag,245,1
cc_cs_ind,246,2
sentence_i_date,248,8
sentence_max_release_date,256,8
sentence_i_min_release_date,264,8
sentence_i_length_years,268,4
sentence_i_length_months,270,2
sentence_i_length_days,272,2
sentence_f_probation_date,280,8
probation_type,288,3
probation_term_years,291,4
probation_term_months,295,2
probation_term_days,297,2

If you save that as doc-schema.txt, you can use CSVkit's in2csv to create a CSV file:

in2csv -s doc-schema.txt -e 'utf-8' doc-data.txt > 2020-10-22-doc-data.csv

Example code:

Filter for sentence dates in 2020:

csvgrep -c 19 -r "^2020" 2020-10-22-doc-data.csv

column,start,length
doc_id,1,8
last_name,9,18
first_name,27,12
middle_name,39,12
suffix_name,51,3
race,54,30
sex,84,30
birth_date,114,8
offender_assigned_place,122,8
doc_loc_func_flag,130,1
cause_no,131,20
offense_county,151,4
sentence_county,155,4
ncic_code,159,4
missouri_charge,163,8
offense_description,171,74
completed_flag,245,1
cc_cs_ind,246,2
sentence_i_date,248,8
sentence_max_release_date,256,8
sentence_i_min_release_date,264,8
sentence_i_length_years,268,4
sentence_i_length_months,270,2
sentence_i_length_days,272,2
sentence_f_probation_date,280,8
probation_type,288,3
probation_term_years,291,4
probation_term_months,295,2
probation_term_days,297,2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment