Skip to content

Instantly share code, notes, and snippets.

View mbklein's full-sized avatar

Michael B. Klein mbklein

  • Northwestern University
  • Evanston, IL
View GitHub Profile
arch avalon donut glaze
cantaloupe X X
db O O O
elasticproxy X
elasticsearch X X X X
fedora X X X
hls X
kibana X
matterhorn X
# Use n-triples for indexing to prevent Fedora from blocking while assembling the RDF response
ActiveFedora::Fedora.class_eval do
def ntriples_connection
authorized_connection.tap { |conn| conn.headers['Accept'] = 'application/n-triples' }
end
def build_ntriples_connection
ActiveFedora::InitializingConnection.new(ActiveFedora::CachingConnection.new(ntriples_connection, omit_ldpr_interaction_model: true), root_resource_path)
end
end
@mbklein
mbklein / championships_between_non_champions.rb
Last active June 8, 2018 13:01
Find championship meetings between two teams, neither of which has won a prior championship
# Assuming data is a tab-delimited list with columns
# date, winner, loser
def champs(data)
winners = {}
losers = {}
data.split(/\n/).sort.each do |line|
date, winner, loser = line.split(/\t/)
winners[winner] = date unless winners.key?(winner)
@mbklein
mbklein / delete_children_of_resource.sh
Last active June 7, 2018 17:25
Delete all of a Fedora container's contained resources without blocking
#!/bin/bash
RESOURCE=http://fcrepo.mbk.vpc.devops-test.library.northwestern.edu/rest/nuf
curl -s -N -H accept:application/n-triples $RESOURCE \
| grep 'ldp#contains' \
| sed 's/^.\+> <\(.\+\)> \.$/\1/' \
| xargs -I % sh -c 'echo -n "%..." && curl -X DELETE % && echo -n "fcr:tombstone..." && curl -X DELETE %/fcr:tombstone && echo "done."'
@mbklein
mbklein / production.log
Created May 18, 2018 18:07
Hyrax::Operation not found
I, [2018-05-18T17:36:02.740337 #2152] INFO -- : ActiveFedora: loading fedora config from FEDORA_URL
I, [2018-05-18T17:36:02.740468 #2152] INFO -- : ActiveFedora: loading solr config from SOLR_URL
W, [2018-05-18T17:36:04.642655 #2152] WARN -- : Same predicate (http://purl.org/dc/elements/1.1/contributor) used for properties contributor and contributor
W, [2018-05-18T17:36:04.645264 #2152] WARN -- : Same predicate (http://purl.org/dc/elements/1.1/creator) used for properties creator and creator
W, [2018-05-18T17:36:04.649874 #2152] WARN -- : Same predicate (http://purl.org/dc/elements/1.1/language) used for properties language and language
I, [2018-05-18T17:36:06.440326 #2152] INFO -- : Hyrax::Engine.after_initialize - persisting registered roles!
I, [2018-05-18T17:36:48.644242 #29] INFO -- : [bbebce07-b878-4c4b-bf62-e1c89fb228a0] Started POST "/" for 172.17.0.1 at 2018-05-18 17:36:48 +0000
I, [2018-05-18T17:36:48.645039 #29] INFO -- : [bbebce07-b878-4c4b-bf62-e1c89fb228a0] [ActiveJob] [S3ImportJob] [49
@mbklein
mbklein / aws_etag_md5.rb
Created May 3, 2018 16:13
AWS Fixity Example
[1] pry(main)> require 'aws-sdk-s3'
=> true
[2] pry(main)> require 'digest/md5'
=> true
[3] pry(main)>
[4] pry(main)> obj = Aws::S3::Object.new(bucket_name: 'avr-masterfiles-staging-rd56wk8', key: 'preservation/b2/77/3w/17/b2773w177/1_Sec_audio.ogg')
=> #<Aws::S3::Object:0x00007fe902b147f0
@bucket_name="avr-masterfiles-staging-rd56wk8",
@client=#<Aws::S3::Client>,
@data=nil,
require 'active_fedora/base'
require 'singleton'
class CallCounter
include Singleton
attr_reader :result
def initialize
@result = {}
@mutex = Mutex.new
@mbklein
mbklein / proxy-toggle.sh
Created April 16, 2018 23:04 — forked from dgoguerra/proxy-toggle.sh
Script to toggle Mac OS X SOCKS proxy, and setup a SOCKS server with SSH
#!/bin/sh
PROXY_INTERFACE="Wi-Fi"
PROXY_HOST=127.0.0.1
PROXY_PORT=1080
# A host where ssh can login without interaction, with a key-based
# authentication.
SOCKS_PROC_USER="username"
SOCKS_PROC_HOST="example.com"
@mbklein
mbklein / download_to_s3.py
Created April 5, 2018 18:59
Lambda for pushing URL to S3
import boto3
import botocore.vendored.requests.packages.urllib3 as urllib3
def lambda_handler(event, context):
url=event.url
bucket = event.s3_bucket_name
key = event.s3_key
s3=boto3.client('s3')
class GemBacktracer
class << self
def cleanup(backtrace)
backtrace.collect do |line|
path = gem_path_for(line)
if path.nil?
line
else
"(#{bundle_paths[path]}):#{Pathname(line).relative_path_from(Pathname(path)).to_s}"