Skip to content

Instantly share code, notes, and snippets.

View mbklein's full-sized avatar

Michael B. Klein mbklein

64 followers · 6 following
  • Northwestern University
  • Evanston, IL
View GitHub Profile
@mbklein
mbklein / overview.ipynb
Last active May 9, 2024 15:10
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@mbklein
mbklein / Elixir Deep Enum Sort
Created December 21, 2023 01:47
defmodule DeepSort do
def sort(x) do
case Enumerable.impl_for(x) do
nil ->
x
_ ->
x
|> Enum.map(fn
{k, v} -> {k, sort(v)}
@mbklein
mbklein / Untitled-2
Created November 2, 2023 15:50
AVR.Restore.restore(bucket, prefix, limit: 200)
@mbklein
mbklein / tiff_metadata_benchmarks.js
Created March 24, 2023 16:55
sharp.metadata() vs probe-image-size benchmarks
#!/usr/bin/env node
const AWS = require('aws-sdk');
const Benchmark = require('benchmark');
const fs = require('fs');
const probe = require('probe-image-size');
const sharp = require('sharp');
function * urlGenerator () {
const ring = fs.readFileSync('./s3Urls.txt').toString().split(/\n/);
@mbklein
mbklein / install_libvips_amazon_linux.sh
Last active February 22, 2022 18:09
#!/bin/sh
echo "Installing libvips dependencies via yum"
yum groupinstall -y "Development Tools"
yum install -y curl jq \
gtk-doc libxml2-devel libjpeg-turbo-devel libpng-devel libtiff-devel libexif-devel libgsf-devel lcms-devel ImageMagick-devel gobject-introspection-devel libwebp-devel \
libpng-devel libxml2-devel
vips_release=$(curl -s https://api.github.com/repos/libvips/libvips/releases | jq -r '.[0]')
vips_release_version=$(jq -r '.name' <<< $vips_release)
@mbklein
mbklein / migration_snippets.ex
Last active January 6, 2022 20:42
AVR Migration Snippets
# Copy file set files for 10 works
(AVR.Migration.list_avr_works()
|> Enum.group_by(& &1.work_type.id)
|> then(fn map ->
~w(AUDIO VIDEO)
|> Enum.flat_map(fn type -> Map.get(map, type) |> Enum.take(10) end)
|> Repo.preload(:file_sets)
|> Enum.flat_map(& &1.file_sets)
|> Task.async_stream(&AVR.Migration.FileMover.process_file_set_files/1, max_concurrency: , timeout: :infinity)
|> Stream.run()
@mbklein
mbklein / DOI.rb
Last active July 6, 2021 15:54
NU's IR DOI Minting Service
class DOI
class Error < StandardError
attr_reader :response
def initialize(response)
@response = response
end
def to_s
"#{response.status} #{response.reason_phrase}\n\t#{response.body}"
@mbklein
mbklein / rerun_failed_file_sets.exs
Created March 17, 2021 13:45
Re-Run Failed FileSets
from(as in ActionState, join: fs in FileSet, on: fs.id == as.object_id,
where: as.outcome == "error", distinct: true, select: fs)
|> Repo.all()
|> Enum.map(fn file_set ->
Meadow.Pipeline.kickoff(file_set, %{overwrite: "false", role: file_set.role.id})
end)
@mbklein
mbklein / index.md
Last active January 21, 2021 06:01
Celebrating America
Time Description
0:13:04 Land of Hope and Dreams (Bruce Springsteen)
0:19:43 Excerpts from Previous Inaugural Addresses:, Lincoln's Second (Kareem Abdul-Jabbar), FDR's First (Dolores Huerta), JFK (Brayden Harrington), Reagan's First (Kim Ng)
0:22:22 Inauguration Recap (Eva Longoria)
@mbklein
mbklein / reindex_common.rb
Created September 28, 2020 16:03
def recommon
query = {"size" => 1000, "query"=>{"bool"=>{"must"=>[{"match"=>{"model.name"=>"FileSet"}}], "must_not"=>{"exists"=>{"field"=>"simple_title"}}}}}
result = CommonIndexer.client.search(body: query)
while result['hits']['hits'].length > 0
result['hits']['hits'].each do |hit|
fs = FileSet.find(hit['_id'])
fs.update_common_index
$stderr.print('.')
end