peaeater
peaeater
/ date-fix.ps1
Last active
August 29, 2015 13:56
Transforms a variety of kooky date strings into y-m-d. This example assumes Record Created in an Inmagic Tagged Format dump.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <# | |
| Fix ambiguous and non-strict dates in dbtext dump data. | |
| #> | |
| param( | |
| [string]$in = ".", | |
| [string]$out = $in | |
| ) | |
peaeater
/ jp22jpg.ps1
Created
March 19, 2014 16:57
Produces a JPG per JP2, given an input directory. Output size defaults to 1000px width, and output name mirrors source JP2s. Requires imagemagick.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # convert .jp2 to .jpg | |
| # requires imagemagick | |
| Param( | |
| [int]$size = 1000, | |
| [string]$indir = ".", | |
| [string]$outdir = ".\jpg" | |
| ) | |
| if (!(test-path $outdir)) { |
peaeater
/ meta2manifest.ps1
Created
March 19, 2014 20:44
Converts Internet Archive XML metadata about a digitized publication to an XML manifest prepped for Solr ingest. Elements are mapped to Andi fields, and sometimes need transformation (e.g. dates to decades).
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # convert IA metadata XML to Solr-ready manifest XML | |
| <# | |
| metadata.imagecount - 2 => pagecount | |
| metadata.identifier => WebSafe($1) => id | |
| metadata.title => title, freetext | |
| metadata.date => toDecade($1) => date, date_free, freetext | |
| metadata.creator => name, name_free, freetext | |
| metadata.publisher => name, name_free, freetext | |
| metadata.year => date_free, freetext |
peaeater
/ raw-ia.ps1
Last active
August 29, 2015 13:57
Processes Internet Archive packages, producing 1 txt, djvu xml, jpg per page of a digitized publication, plus an XML manifest. The output is intended for ingest by Solr through Andi's DIH handler. Jobs are broken into subscript dependencies. Requires imagemagick and djvulibre.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # processes Internet Archive packages, producing per page: 1 txt, 1 ocrxml, 1 jpg | |
| # requires djvulibre, imagemagick | |
| param( | |
| [string]$indir = ".", | |
| [string]$outbase = $indir | |
| ) | |
| [Reflection.Assembly]::LoadWithPartialName("System.IO.Compression.FileSystem") |
peaeater
/ solr-dih-ingest-with-polling.ps1
Last active
August 29, 2015 14:02
Triggers Solr DIH update and monitors its status. Writes exceptions or final success message to Windows Application Event Log.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <# | |
| Trigger Solr update and poll for status. | |
| - Writes events to Application Event Log; log source must already have been added | |
| Peter Tyrrell | |
| #> | |
| param( | |
| [Parameter(Mandatory=$false,Position=0)] |
peaeater
/ add-eventlog-source.ps1
Created
June 19, 2014 00:30
Adds supplied source value to the Windows Application Event Log. Requires admin privileges and will warn the user if elevation is required.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <# | |
| Add log source to Application Event Log if not already there - REQUIRES ADMIN privileges | |
| Peter Tyrrell | |
| #> | |
| param( | |
| [Parameter(Mandatory=$false,ValueFromPipeline=$true,Position=0)] | |
| [string]$logsrc = "Andi Solr Update" |
peaeater
/ delete-eventlog-source.ps1
Created
June 19, 2014 00:32
Removes a source value from Windows Application Event Log (careful!). Requires admin privileges and warns user if elevation is required.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <# | |
| Remove log source from Application Event Log - requires ADMIN PRIVILEGES | |
| Peter Tyrrell | |
| #> | |
| param( | |
| [Parameter(Mandatory=$true,ValueFromPipeline=$true,Position=0)] | |
| [string]$logsrc |
peaeater
/ extractomatic.ps1
Last active
August 29, 2015 14:03
Sample Andi extractomatic script that creates its own ODBC connection string and logs to Application event log.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <# | |
| Extracts data from the named textbase as files like {tn}-{0}.xml in output folder. | |
| Logs to Application event log - source must already have been added. | |
| #> | |
| param( | |
| [Parameter(Mandatory=$false,Position=0)] | |
| [string]$logsrc = "Andi Solr Update" | |
| ) |
peaeater
/ update.ps1
Created
June 25, 2014 15:44
Sample master update script that coordinates extractomatic, remote sync, and Solr ingest.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <# | |
| Manages component scripts to perform textbase-to-solr update. | |
| Script halts if any component reports failure. (Components must return exit code 1 on failure, 0 on success.) | |
| Peter Tyrrell | |
| #> | |
| $logsrc = "Andi Solr Update" | |
| & '.\mytextbase.ps1' $logsrc |
peaeater
/ Default (Windows).sublime-keymap
Last active
August 29, 2015 14:03
— forked from LeVM/Default (OSX).sublime-keymap
Insert accented characters in Sublime Text 3
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Show hidden characters
| [ | |
| { "keys": ["ctrl+\\","a","`"], "command": "insert", "args": {"characters": "à"} }, | |
| { "keys": ["ctrl+\\","a","^"], "command": "insert", "args": {"characters": "â"} }, | |
| { "keys": ["ctrl+\\","e","'"], "command": "insert", "args": {"characters": "é"} }, | |
| { "keys": ["ctrl+\\","e","`"], "command": "insert", "args": {"characters": "è"} }, | |
| { "keys": ["ctrl+\\","e","^"], "command": "insert", "args": {"characters": "ê"} }, | |
| { "keys": ["ctrl+\\","e",":"], "command": "insert", "args": {"characters": "ë"} }, | |
| { "keys": ["ctrl+\\","i",":"], "command": "insert", "args": {"characters": "ï"} }, | |
| { "keys": ["ctrl+\\","i","^"], "command": "insert", "args": {"characters": "î"} }, | |
| { "keys": ["ctrl+\\","o","^"], "command": "insert", "args": {"characters": "ô"} }, |
OlderNewer