Skip to content

Instantly share code, notes, and snippets.

View atomotic's full-sized avatar

raffaele messuti atomotic

75 followers · 157 following
View GitHub Profile
@atomotic
atomotic / README.md
Last active February 10, 2024 13:43
load xml files into SQLite and transform to json

Install sqlpkg

Install extensions

sqlpkg install sqlite/fileio
sqlpkg install jakethaw/xmltojson

Start

@atomotic
atomotic / full.md
Last active February 1, 2024 14:23
opac sbn api

full.json

http://opac.sbn.it/opacmobilegw/full.json?bid={BID}

--

➜  ~  http "http://opac.sbn.it/opacmobilegw/full.json?bid=IT%5CICCU%5CRAV%5C2002745"
HTTP/1.1 200 OK
Connection: close
Content-Type: application/json;charset=UTF-8
@atomotic
atomotic / gist:90a7e8c30faeeba37c7cc6459c5d947b
Created December 21, 2023 15:54
➜ file 89595bd2-8076-4da0-8880-518c291e7904
89595bd2-8076-4da0-8880-518c291e7904: EPUB document
➜ tika -m -j 89595bd2-8076-4da0-8880-518c291e7904
Exception in thread "main" org.apache.tika.exception.TikaException: TIKA-237: Illegal SAXException from org.apache.tika.parser.epub.EpubParser@3a320ade
at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:310)
at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:298)
at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:203)
at org.apache.tika.cli.TikaCLI$OutputType.process(TikaCLI.java:1071)
at org.apache.tika.cli.TikaCLI.process(TikaCLI.java:493)
@atomotic
atomotic / wget-lua.sh
Created May 3, 2016 09:28
install wget-lua on osx
brew install lua5.1
curl http://warriorhq.archiveteam.org/downloads/wget-lua/wget-1.14.lua.LATEST.tar.bz2 | tar -xj --strip-components=1
sed -i -e s=usr/include/lua5.1=usr/local/include/lua5.1=g configure.ac
./configure
make
cp src/wget /usr/local/bin/wget-lua
wget-lua --help|grep lua
GNU Wget 1.14.lua.20130523-9a5c, a non-interactive network retriever.
Usage: wget-lua [OPTION]... [URL]...
@atomotic
atomotic / mastodon-followers.sh
Created August 31, 2018 08:26
get the list of followers of a mastodon user. output in ntriples
#!/usr/bin/env bash
instance="https://digipres.club"
user="raffaele"
json=$(curl -s -H "Accept: application/activity+json" $instance/users/$user/followers?page=1)
echo "$json" | jq -r .orderedItems[] | xargs -I% echo "<%> <follows> <$instance/user/$user> ."
next=$(echo "$json" | jq -r .next)
while true; do
@atomotic
atomotic / Readme.md
Last active September 9, 2022 09:39
Internet Archive Save Page Now
@atomotic
atomotic / gist:e96982253136dcd253f8376ded2fd0e5
Created March 18, 2022 07:27
zotero annotations
sqlite> .schema itemAnnotations
CREATE TABLE IF NOT EXISTS "itemAnnotations" (
itemID INTEGER PRIMARY KEY,
parentItemID INT NOT NULL,
type INTEGER NOT NULL,
authorName TEXT,
text TEXT,
comment TEXT,
color TEXT,
pageLabel TEXT,
@atomotic
atomotic / himalayandatabase.md
Last active December 5, 2021 13:31
himalayandatabase - from dbf to json api

himalayandatabase

from a Visual FoxPro GUI to json api with datasette

http://himalayandatabase.com

The Himalayan Database is a compilation of records for all expeditions that have climbed in the Nepalese Himalaya. The database is based on the expedition archives of Elizabeth Hawley, a longtime journalist based in Kathmandu, and it is supplemented by information gathered from books, alpine journals and correspondence with Himalayan climbers.

The Himalayan Database is a Microsoft Visual Foxpro 9 program.

@atomotic
atomotic / epub-search.md
Created November 13, 2021 12:11
indexing epub content into solr

indexing epub content into solr

solr schema

  • 1 document per chapter, then collapse
  • multivalued fields: chapter_title and chapter_text, keeping order.

text extraction

how to extract structured text from epub

@atomotic
atomotic / docker-compose.yml
Created November 7, 2021 12:52
version: "3"
node-exporter:
image: prom/node-exporter
volumes:
- /proc:/host/proc:ro
- /sys:/host/sys:ro
- /:/rootfs:ro
command:
- "--path.procfs=/host/proc"