Skip to content

Instantly share code, notes, and snippets.

@mariushoch

mariushoch/check-lexeme-json-dumps

Last active February 12, 2022 21:02
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mariushoch/b44bf04146d507e3f8e2881872e01e9a to your computer and use it in GitHub Desktop.
Save mariushoch/b44bf04146d507e3f8e2881872e01e9a to your computer and use it in GitHub Desktop.
Download ZIP
Check the Wikidata Lexeme JSON dumps each Thursday
Raw
check-lexeme-json-dumps
#!/bin/bash
set -e
cd "$HOME/wikibase-validate-dump"
export https_proxy=http://webproxy:8080
git fetch
if [[ ! "$(git log HEAD..origin/master)" == "" ]]; then
echo "New commits, please rebase:"
git log --format=oneline HEAD..origin/master
fi
echo
echo
dumpDate="$(date --date 'last Wednesday' +"%Y%m%d")"
php validateJsonDump.php compress.zlib:///mnt/data/xmldatadumps/public/other/wikibase/wikidatawiki/"$dumpDate"/wikidata-"$dumpDate"-lexemes.json.gz
Raw
crontab
MAILTO=foo@bar.invalid
30 10 * * 4 nice -n19 bash $HOME/bin/check-lexeme-json-dumps
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment