Converting CSV into Patron Records for III Millennium

csv2patron-marc.py

#!/usr/bin/env python

# note final, more robust version at https://github.com/cca/millennium_patron_import

import sys
import csv # https://docs.python.org/2/library/csv.html
# PCODE mappings, etc.
from mapping import *
import datetime
import re

# usage info
def usage():
    print 'Usage: python csv2patron-marc.py semester <csv> [<output>]'
    print '\twhere "semester" is one of F, sp, or Su'
    print '\t<csv> is the csv file you want to convert'
    print '\t& <output> is the (optional) destination file.'
    print '\tIf you don\'t specify an output file, the default is "import".'
    exit(0)

try:
    if sys.argv[1] == 'help' or sys.argv[1] == '-h' or sys.argv[1] == '--help':
        usage()
except IndexError:
    usage()

# used later to construct notes
yr = str(datetime.date.today().year)[2:]
semester = sys.argv[1]
if not re.match('^((F)|(sp)|(Su))$', semester):
    print 'Invalid semester: use F, sp, or Su'
    exit(1)

# files
csv = csv.DictReader( open(sys.argv[2], 'r') )
try:
    output = open(sys.argv[3], 'w')
except IndexError:
    output = open('import', 'w')


for row in csv:
    # "zero field"
    output.write('0')
    # These fields are all defined in mapping.py
    # P TYPE
    output.write(ptype[row['Academic Level']])
    # PCODE 1
    output.write(pcode1[row['Programs']])
    # PCODE 2
    output.write(pcode2)
    # PCODE 3
    output.write(pcode3)

    # Home Library (5 chars), Message Code (1), & Block Code (1) are all left blank
    output.write('       ')

    # Expiration Date, end of fixed-length fields
    output.write(expiration + '\n')

    output.write('n' + row['Family Name'] + ', ' + row['Given Name'] + '\n')
    output.write('t' + row['Home Phone'] + '\n')
    output.write('u' + row['ID'] + '\n') # student ID
    output.write('z' + row['CCA Email'] + '\n') # email
    # note - we put in what semester the record was created
    output.write('x' + semester + yr + '\n')

# wrapping up
output.close()

example.csv

mapping.py

# See Millennium Circulation > Admin > Parameters > Circulation > Patron Type
# 3-digit, 000 to 255
ptype = { 'UG': '001', 'GR': '002' }

# For Pcode 1 & 2, see
# Millennium Circulation > Admin > Parameters > General > Fixed-length Codes
pcode1 = {
    'ANIMA.BFA': 'a',
    'ARCHT.BARC': 'b' # etc…, not putting all our programs in here
} 
# leave blank for now as these aren't set up
pcode2 = ' '

# See Millennium Circulation > Admin > Parameters > Circulation > Pcode 3
# We only use this for faculty, leave blank
pcode3 = '    ' # 3-digit, 000 to 255

# expiration is always the final day of the semester for us
# format is 'MM-DD-YY'
expiration = '12-12-14'

FYI full version of this here: https://github.com/cca/millennium_patron_import

The basic outline of adding new patrons before the semester is:

• Run an Informer report to get new student info in CSV format
• Use "csv2patron-marc.py" to convert the CSV into a MARC-like format
• Load, prep, & finally import the resulting file in Millennium

Details

Run a report and set the Start Term to the upcoming semester. In the CSV export, ensure that Columns Headers is checked. Then, on the command line, navigate to the directory with the "csv2patron-marc.py" script on it and run:

python csv2patron-marc.py semester "export.csv" "output"

where "semester" is the type of semester, "export.csv" is the name of the exported CSV file, and "output" is the name of the file you want the converted patron data to go into (if you leave this filename off, it'll go into "import" by default). If your file names have spaces in them, you can wrap them in quotation marks. "Semester" must be one of the following: "F", "sp", or "Su" (for Fall, Spring, & Summer respectively).

Inside Millennium Circulation, select the Data Exchange section and then Select Process "Load MARC Patron records from tape or FTS (pta)". Click Get PC to browse your hard drive for the import file and choose extension "ptfs" if asked.

Highlight the newly loaded file and choose Tools > Prep > "PREPROCESS TEXT Patron records loaded via FTS", then click Start. A brief message should appear as Millennium processes the records and when you close the dialog there will be a ".pat" file listed. Select the .pat, then Tools > Load > Load a MARC file (or just click Load in the upper right). Here you can first Test the MARC file, which can provide warnings about things like improper PCODEs. Then, if the test is successful, hit Load to import the patron records.

Links & Documentation

Relevant manual pages:
106003 "Preprocessing Files for Import in Data Exchange"
106004 "Loading Records in Data Exchange"
107503 "Importing Patron Data"
http://gsm.iii.com/data2_6.html
http://gsm.iii.com/data3_1.html