Skip to content

Instantly share code, notes, and snippets.

@krdyke
Forked from saverkamp/CatcherUploadExample.csv
Created July 17, 2017 18:21
Show Gist options
  • Save krdyke/b4933b1d68b0b8670a381552aa508d92 to your computer and use it in GitHub Desktop.
Save krdyke/b4933b1d68b0b8670a381552aa508d92 to your computer and use it in GitHub Desktop.
Sample CONTENTdm Catcher upload script. Uses Catcher class from: https://gist.github.com/saverkamp/9197945
Alias CDM_page_id CDM_field Value
byington 3160 transc Thursday October 25th 1900 Did two churnings this morning and finished putting the tomato pickle away. I put in all the afternoon doing mending Will busy about the place and Mrs Evans helped Leonard with the corn the new man went back to town tonight. Friday October 26th 1900 Will took the butter to town. I was busy with the work till noon I did a lot of baking. I cut out shirts and sewed all afternoon. Saturday October 27th 1900 I sewed a little and got the dinner be a little after eleven. Will and I went to town in the afternoon to have my teeth finished. It has been a warm week, today was like summer. I had a letter from Mother. Sunday October 28th 1900 I was busy about the house most of the forenoon Leonard and his wife were away all day. It rained some in the forenoon. We were up to Stevens in the afternoon. I spent the evening reading. Monday October 29th 1900 It rained in the morning so Leonard could not husk corn. Will was busy about the place. I sewed some and went up to Nelsons in the afternoon. A hard rain at supper time.
byington 3161 transc Tuesday October 30th 1900 I put in all the forenoon making some jelly and cleaning up the summer kitchen. One of our fat hogs got down at Johns and they killed it so the boys dressed it in the afternoon. I went to town in the afternoon to see if they could use it at the meat shops It was an awful trip through the mud. Wednesday October 31st 1900 It rained the entire day. The meat of that hog was not fit to use Will cut it up for soap grease. I did a little sewing and helped with the work. wrote to Mother Thursday November 1st 1900 We churned and washed in the forenoon. Mrs Evans did not feel very well in the afternoon. I did our share of the ironing and other work. getting the butter ready to go town tomorrow. A letter from Mother. Friday November 2nd 1900 I fried out that meat before going to town with Will We delivered the butter and I did some shopping before going over the river. I found Hattie busy house cleaning I helped her what little I could but we came home early. It was a very pleasant day.
byington 3196 transc Sunday April 7th 1901 "Easter Sunday" It cleared today so the sun shone part of the time. Ida & Ernest went home. After dinner Will and I took a walk out over the place I found our deed today we thought was lost. A year ago today We went up to see how Lee was. I did not think then he would be with us as long as he was. Monday April 8th 1901 We had a very large washing to do today and churning besides. I put some things back in the pantry and we did some work in the yard. quite a pleasant day. Tuesday April 9th 1901 The boys began to put in oats and Russell worked a great part of the day fixing the hen house I was out in the yard a great part of the day. and finished fixing the pantry we got the ironing done today too. Wednesday April 10th 1901 I set some hens this morning and did some work out of doors and did the mending and put away the clothes. Thursday April 11th 1901 We cleaned yard most of the forenoon day when not busy with the regular work did the churning and got the butter ready to go down tomorrow The boys are still busy with the oats. Ida went to town.
cookbooks 9653 fula (9) How to make mackrounds Take 3 pound of almonds after they are blancht put them into water. then take them out again into a cloath let them drayne but do not dry them. Then beat them first in a brass-morter and then in a marble one until they be very fine - paist then have ready [bott?] and put thro your wire sieve 3 pound of fine powder sugar- put it to your almonds and mix it well with your hand. Then have ready broken in a pot the whites of 12 eggs and put into it. then stir it well with your wooden stirer. Then lay them out have your wisk round with wafers double- you must wet the edges with a little water and let them [touch] one to another so drop them on to the wafers and when you have got all laid out, set them into the oven and just air them then take them out and have some double refined sugar dried and Pott in your little Tiffany Toms and dust them well over so set them into the oven again and bake them but let not your oven be too hot [illegible] must be up- they bake very well with savoy biskotts. if you would have any of them in little drops you must make your paist much thiner with the whites of eggs this will make 6 pounds of mackrounds.
pioneers 137 fula March, Monday 1. 1880. Went with Belle to Dr McClure's then shopping Handk'fs 1.70 Socks for Frank .30 Belle [Mirspials?] Paint. .50 Tuesday 2. Morning went with Mrs. Howard to see Mrs. Griffith about poor children. also went to see Supervisors - P. M. Some house work. Wednesday 3. May and I went to Mrs Green's to Club
iwai 5056 hidden Ogden Sept. 1 - 94 Rec'd of treasurer of Missionary Society (Mrs. G. Miller) the sum of ten Dollars ($10) J. T. Bentley Also $15 to apply in Benevolences, Aug. 19. J. Bentley Treasurer of W.T.M.S. Mrs. E.K. Stanley 1102 High St Des Moines No. Perry Ia Dec 6 1893 Received of Mr. Miller T Maddox Six Dollars In full [lawp?] $6 J.L. Morgan
iwai 7191 hidden Wed. Febr 6-1923 the Executive Board of the I.L.of W. Voters met at the call of the President Mrs Daniels in regular business session with the following officers present: Mrs Clark Daniels, President, Des Moines. Mrs Max Mayer, 2nd Vice ",". Mrs Clarence Knutson, Sec. residing in Des Moines for the present. Mrs Robert Jackson, Treas. , Des Moines. The Treasurer gave her report, the amount on hand February 1 - $399.50 This report was from Jan 1 1924 to Febr. 1 - 1924. Motion prevailed to accept this report. Bills allowed were $68.68 Mrs Mayer moved that $400. be sent to the National. Motion seconded and carried. Four $100. pledges were reported as follows to the National Mrs N.M. Willchinske $100. Mr Morris Sidney Mandelbaum $100. Mrs Florence Koch $100. L.L.W. Voters $100. No further business appearing the Board adjourned. Hazel Wood Knutson, Sec.
iwai 7192 hidden Febr 11, 1924 the Board of Directors of the I.L.W.Voters met in regular session at the call of the President one o'clock at the Iowa Headquarters 507 Shopps Bldg. with the following officers present. President - Mrs Clark Daniels, Des Moines. Treas - Mrs Robert Jackson,". Chr of International Co-op to Prevent War - Mrs Fred Hunter ". Chr of Uniform Laws - Miss Leffert. " . Organizer - Mrs I.H. Tomlinson. Social Hygiene - Miss Anna M. Drake. Secretary - Mrs Clarence Knutson. The Secretary reported copies of the Jan 15 - 1924 Board meeting had been sent out to 15 Board members who were not present at that meeting. The Treasurer reported $156-69 on hand. Mrs Tomlinson moved that the report be accepted, seconed by Mrs Hunter. Motion carried. A letter was read from Mrs Sam Wolf of Charles City chairman of Finance relative to the raising of the money pledged to the National L of W. Voters. Mrs Tomlinson moved that the amounts of $156.24 in the Commercial Savings Bank remaining for the Finance drive be sent to Mrs Wolf. Motion was seconded by Mrs Hunter and carried. Bills presented $4.00 Mrs Tomlinson moved that the bill of $4.00 be allowed. Motion seconded by Miss Drake and carried.
iwai 7193 hidden Mrs I.H. Tomlinson spoke of her work as Organizer and gave the following report. Trip to Gilman via Marshalltown ..... Total expense $10.75 Talk Friday P.M. 20th Century Club,"Present Status of Women." forty minutes, home of Mrs Perry, fifteen present, all sold to League of Women Voters. Sat P.M. Methodist Church 2:30 History and Purpose of League, 45 minutes. Rep of P.T.A., F.B.W.C. Auxiliary Legion, and 20th Century Club. Present about forty. Unanimous vote to organize. Half dozen very favorable affirmative talks on the motion to organize. No opposition. Officers chosen, splendid young married women, college training. Present, Mrs Fannie Tone, wife of Supt. of Schools, former teacher, W.C. 1st Vice " , Mrs Gertrude Darrah, wife of Hardware merchant, former librarian, Am. Legion and 20th Century. 2nd Vice President, Mrs Grace, wife of farmer, former Grad. nurse F.B. & P.T.A.Secretary & Treasurer Mrs Paul East, wife of bank cashier, now teaching, 20th Century and P.T.A. Fine list of prospective committee chairmen. Date and place of next meeting, 2nd Monday eve, School House. Executive Committee next Wednesday at President's home. Speaker, Judge Cummins, Marshalltown on "Property and Personal Rights of Women". Chairman of this committee, Mrs Berthelson, Norweigan and rare character. Suggest the name of the H.P. Mead, 107 W. Church Street, Marshalltown, Iowa possible District Director of Fifth District. Mrs R.J. Schillingar assisted in organizing.
iwai 7194 hidden Mrs Hunter moved that Mrs Tomlinson's expense of $10.75 be allowed. Miss Leffert seconded the motion. Motion carried. Mrs Hunter announced that she had spoken at Gilmore City [twice?]. No further business appearing, the Board of Directors adjourned to meet Febr 19 - 1924. (Mrs) Hazel Wood Knutson, Sec'y. appr.
iwai 7195 hidden February 19 - 1924. The Board of Directors of the I.L.of W. Voters met in regular session at the call of the President at Headquarters 507 Shopps Building with the following members present - President - Mrs Clark Daniels, Des Moines. 2nd Vice President - Mrs Max Mayer, Des Moines. Secretary - Mrs Clarence Knutson, Clear Lake. State Chr. of International Co-operation To Prevent War - Mrs Fred Hunter, Des Moines.. Organizer - Mrs I.H. Tomlinson, Des Moines. Parliamentarian - Mrs Blanchard. The minutes of February 11- 1924 were read and approved. The report of the Treasurer Mrs Jackson was read and approved. Balance on hand ... $357.64 Per capita dues paid - - Nov 14 - - Keota League $1.00 Dec. Grinnell League (Officers not elected) $1.50 Jan. 25 Mason City League by Edith Ride ... $7.00 Quota - - Nov 1 --- Jefferson County League by Mrs Bock to Mrs Tomlinson $25.00 Feb. 19 --- Mahaska County League by Mrs Orvis $25.00
iwai 7197 hidden Mrs Mayer moved that we send $200. for 2 sustaining memberships to the National League. Motion was seconded by Mrs Hunter. Motion carried. Bills for current expenses amounting to $29.00 were read. Mrs Hunter moved that these bills be allowed and an order drawn for same. Motion was seconded by Mrs Mayer. Motion carried. Mrs I.H. Tomlinson, State Organizer, reported as follows: Tentative dates for 4th District, March 11, Mitchell County. Osage, afternoon sponsored by Delphians and clubs. Purpose, organization. Mrs A.A. Kugler (lawyer's wife) Nashua, Chickasaw County New Hampton," " Calmar, Winnishiek County. Above are other prospects that could be made at the time of trip to Osage with change of future organization. Probable expense not to exceed, $25.00. Please allow this if you wish me to go. I find it almost impossible to absolutely sell the idea and tie up a a completed organization in one day or session. Follow-up work is needed, hence the desirability of having various teams of workers. Feb. 21 --- Bloomfield Township League Luncheon. Talk "Immigration", forty minutes. About thirty-five present. Many questions asked on pending legislation, faxes, etc. Contributed fifty cents to League "quilt fund". Car fare, $.14. Total expenses $.64. Report Expenses State Organizer Feb. 25, 1924 ----- 7:20 P.M. To Forest City, fare...$5.68 Berth, Lower 6 --Car #3...3.00 Taxi, Des Moines... .65 Taxi, Forest City... .50 Tip, Porter Pullman... .25 Tip," Notel... .25 Feb. 26, 1924 Breakfst & tip... .50 No luncheon Telehpones, four calls... 1.70 Dinner... .60 Feb 27, 1924 Breakfast... .10 Hotel, 2 nights ($1.75)... 3.50 Tip, maid... .25 Fare to Garner, self & Blake... .88 Mrs Pritchard, hostess, no meals or taxi. Fare from Mason City, to Des Moines... 4.40 Taxi, Des Moines... .65 Total $22.91 Feb. 26. Afternoon meeting, F.C. Womens Club. About twenty minute talk on League. About 50 present. Evening meeting Court Room F.C. about forty-five minutes, forty present. Feb. 27 Motion to organize League. Afternoon, Garner, Sager Hall, 40 present, 30 minutes, 15 minutes, Blake. Strong desire for organization
iwai 7198 hidden Mrs Fred Hunter, chairman of the committee on International Co-operation to Prevent War reported on the work done in regard to the World Court Petitions. Mrs A.B. Pugh a member of the Polk County League of Women Voters took charge of sending out 250 of the new World Court petitions to interested person as prepared by the League of nations Non-Partison Association. All petitions sent [own?] by the various organizations as W.C.J.V., Federated Clubs etc some 10000 in all when signed will be returned to the I.L.W.V Hdq. and sent by them to the National headquarters. This final work to be done by Mrs Jay Tone. Mrs Max Mayer, chairman of the committee on American citizenship reported that she had asked Dr. Moorehouse, President of Drake University to put in to the summer session, a summer Institution on Citizenship training and that Mrs Mayer in conjunction with Mrs Ida B. Wise Smith, President of the State W.C.J.V will suggest a course of study on citizenship. Also a request will be made of every college in the state having a summer session to put in a similar Institute. A letter from Mrs Shaley of on boat to South America was read Last Bulletin ordered sent to her. The President asked Mrs Blanchard to talk of the call for a special Convention to amend the Constitution. Mrs Mayer moved that the words "and a director from each Congressional District", in Art. 1 Sec 4 be left in the Constitution, Mrs Tomlinson seconded the motion. Motion carried. Mrs Mayer moved that in Art. 6, Section 4 - after the word "appoint", insert the words,"a state organizer and." Mrs Hunter seconded the motion. Motion carried. Mrs Hunter moved that in Art 6, Section 5 -- insert after the word "committee" the
iwai 7199 hidden words "Finance Chairman." Mrs Mayer moved that in Art. 7, Section 1- Strike out the words "Oct or Nov." and insert the words "March or April." Mrs Tomlinson seconded the motion. Motion carried. Mrs Tomlinson moved that in Art 8 Sec 1. Insert the words "Finance Chairman" after the word "Committees." Mrs Hunter seconded the motion. Motion carried. Mrs Hunter moved that in Art 11 Section 1 to strike out the words "at least" and strike out the words "at such time and place as the president may designate" and insert the words "one immediately following the annual Convention, one in October, and one immediately preceeding the annual Convention. Mrs Mayer seconded the motion. Motion carried. Mrs Mayer moved to strike out the words "state headquarters" in Art 12 Sec 2 and insert the words "finance chairman." Mrs Tomlinson seconded the motion. Motion carried. Mrs Tomlinson moved to strike out the words "Board of Directors" in Art 12 Sec 2 and insert the words "Executive Committtee". Mrs Hunter seconded the motion. Motion carried. A splendid letter was read from the third Vice President, Mrs Passig. No further business the Board adjourned. Hazel Wood Knutson, Recording Sec'y.
#!/usr/bin/python
# sampleCatcher.py
# Updates existing metadata records with CONTENTdm Catcher protocol. (Documentation at:
# http://contentdm.org/help6/addons/catcher.asp).
# ***requires SUDS, a third-party SOAP python client: https://fedorahosted.org/suds/***
#
# Given a CSV file with a minimum of four fields: Alias, CDM_id, CDM_field, and Value, this script updates
# one value per row and returns a csv log of transcactions. For file input, currently set to iterate
# through the current directory and process any csv files beginning with "CatcherUpload". Input and
# transaction files are output to "Completed" directory with timestamp.
#
# Author: Shawn Averkamp
# Last modified: 2014-02-24
from suds.client import Client
import datetime
import csv
import os
import fnmatch
# set variables for SOAP connection--requires config.ini file
user = 'yourcdmusernamehere'
password = 'yourcdmpasswordhere'
license = 'yourcdmlicensehere'
base = 'yourbaseurl'
port = 'yourportnumberhere'
url = base + ":" + port
class Catcher(object):
"""A CONTENTdm Catcher session."""
def __init__(self, url=url, user=user, password=password, license=license):
self.transactions = []
self.client = Client('https://worldcat.org/webservices/contentdm/catcher/6.0/CatcherService.wsdl')
self.url = url
self.user = user
self.password = password
self.license = license
def processCONTENTdm(self, action, user, password, license, alias, metadata):
# function to connect to CatcherServices and process metadata updates
transaction = self.client.service.processCONTENTdm(action, url, user, password, license, alias, metadata)
self.transactions.append(transaction)
def edit(self, alias, recordid, field, value):
#function to edit metadata--call packageMetadata and processCONTENTdm
metadata = self.packageMetadata('edit', recordid, field, value)
self.processCONTENTdm('edit', self.user, self.password, self.license, alias, metadata)
def packageMetadata(self, action, recordid, field, value):
#function to package metadata in metadata wrapper
action = action
if action == 'edit':
metadata = self.client.factory.create('metadataWrapper')
metadata.metadataList = self.client.factory.create('metadataWrapper.metadataList')
metadata1 = self.client.factory.create('metadata')
metadata1.field = 'dmrecord'
metadata1.value = recordid
metadata2 = self.client.factory.create('metadata')
metadata2.field = field
metadata2.value = value
metadata.metadataList.metadata = [metadata1, metadata2]
return metadata
def UnicodeDictReader(str_data, encoding, **kwargs):
csv_reader = csv.DictReader(str_data, **kwargs)
# Decode the keys once
keymap = dict((k, k.decode(encoding)) for k in csv_reader.fieldnames)
for row in csv_reader:
yield dict((keymap[k], v.decode(encoding)) for k, v in row.iteritems())
#iterate through current directory to find any files starting with 'CatcherUpload' and process
for file in os.listdir('.'):
if fnmatch.fnmatch(file, 'CatcherUpload*.csv'):
csvfile = open(file, 'rb')
c = UnicodeDictReader(csvfile, encoding='utf-8')
# create directory for completed files and transaction logs if it doesn't exist
newdir = 'Completed'
if not os.path.isdir(newdir):
os.mkdir(newdir)
# create csv file for transactions
today = datetime.datetime.now().strftime('%Y-%m-%d--%H-%M')
fname = 'Completed/Transactions_' + file[0:-4] + '_' + today + '.csv'
f = open(fname, 'wb')
fwtr = csv.writer(f)
header = ['Alias', 'CDM_id', 'CDM_field', 'Transaction']
fwtr.writerow(header)
#initialize Catcher session
session = Catcher(password=password)
#iterate through rows in csv and process through Catcher
for row in c:
# get values from row and package metadata for edits
cdmid = row['CDM_page_id']
alias = row['Alias']
field = row['CDM_field']
value = row['Value']
# package metadata for Catcher and upload if value is not empty
if value != '':
session.edit(alias, cdmid, field, value)
# write transaction message to file
fRow = [alias, cdmid, field, session.transactions[-1]]
else:
fRow = [alias, cdmid, field, 'No content--not uploaded']
fwtr.writerow(fRow)
f.close()
csvfile.close()
# append timestamp to upload csv filename and move to Completed directory
newcsv = 'Completed/' + file[0:-4] + '_' + today + '.csv'
oldfile = file
os.rename(oldfile, newcsv)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment