Skip to content

Instantly share code, notes, and snippets.

@lucaswerkmeister

lucaswerkmeister/script.sh Secret

Created November 3, 2017 13:34
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save lucaswerkmeister/351ad0ffac1191658e39975063b8c19a to your computer and use it in GitHub Desktop.
Save lucaswerkmeister/351ad0ffac1191658e39975063b8c19a to your computer and use it in GitHub Desktop.
Download ZIP
running rdf2hdt
Raw
script.sh
# build the program
apt install libserd-dev
git clone https://github.com/rdfhdt/hdt-cpp
cd hdt-cpp
./autogen.sh
./configure
make
# you can probably do this on a non-stats system,
# in which case I assume you’d need to install libserd-0-0 on the stats system
# convert
pv path/to/full/dump.gz | gunzip | tools/rdf2hdt -i -f ttl -B 'http://wikiba.se/ontology-beta#Dump' /dev/stdin wikidata.hdt
# if you have systemd v235, consider wrapping tools/rdf2hdt in something like
# systemd-run --pipe --unit=rdf2hdt -p PrivateUser=yes -p ReadWritePaths="$PWD" -p IOSchedulingClass=idle -p Nice=19 etc.
# otherwise, perhaps run the whole pipeline inside systemd-run with sh -c '…' as the command
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment