Skip to content

Instantly share code, notes, and snippets.

@ohac

ohac/monacard_backup.sh

Last active Mar 24, 2021
Embed
What would you like to do?
#!/bin/bash
API=https://card.mona.jp/api
DB=monacard_db
IMG=img
TMPD=tmp.d
BACKUPD=backup.d
mkdir -p $DB
mkdir -p $IMG
mkdir -p $TMPD
mkdir -p $BACKUPD
getnameext() {
FILENAME=$(basename "$1")
NAME=$(echo $FILENAME | cut -d. -f1)
EXT=$(echo $FILENAME | cut -d. -f2)
EXT=${EXT:0:3}
case $EXT in
'png')
;;
'PNG')
EXT='png'
;;
'gif')
;;
'jpg')
;;
esac
}
if [ -a $TMPD/card_list_update.txt ]; then
csv=$(ls -l --time-style +%s $TMPD/card_list_update.txt | tr -s ' ' ,)
mtime=$(echo $csv | cut -d, -f6)
else
mtime=$(date +%s)
fi
updt=$((${mtime}-60*60*1))
curl -s "$API/card_detail?update_time=$updt" > $TMPD/card_list_update.json
jsonerr=$(cat $TMPD/card_list_update.json | jq -M '.error')
if [ "$jsonerr" == 'null' ]; then
cat $TMPD/card_list_update.json \
| jq -M '.details[].asset_common_name' \
| sed 's/"//g' \
| busybox sort > $TMPD/card_list_update.txt
cardlist=$(cat $TMPD/card_list_update.txt)
else
cardlist=
fi
for key in $cardlist; do
if ! [ -a $DB/$key.json ]; then
continue
fi
URL=$(jq -M '.details[0].imgur_url' < $DB/$key.json | sed 's/"//g')
getnameext "$URL"
FILENAME=$IMG/${NAME}.${EXT}
HASH=$(sha256sum $DB/$key.json | cut -d ' ' -f1)
mv -f $DB/$key.json $BACKUPD/${HASH}.${key}.json
if ! [ -a $FILENAME ]; then
continue
fi
HASH=$(sha256sum $FILENAME | cut -d ' ' -f1)
mv -f $FILENAME $BACKUPD/${HASH}.${NAME}.${EXT}
done
if ! [ -a $DB/card_list.txt ]; then
curl -s $API/card_list > $DB/card_list.json
jq -M '.list[]' < $DB/card_list.json | sed 's/"//g' \
| busybox sort > $DB/card_list.txt
fi
cardlist=$(cat $DB/card_list.txt)
for key in $cardlist; do
if ! [ -a $DB/$key.json ]; then
echo $key
curl -s "$API/card_detail?assets=$key" > $DB/$key.json
echo sleep 2
sleep 2
fi
URL=$(jq -M '.details[0].imgur_url' < $DB/$key.json | sed 's/"//g')
getnameext "$URL"
FILENAME=$IMG/${NAME}.${EXT}
if ! [ -a $FILENAME ]; then
echo $FILENAME
# TODO save image on img.monaffy to another dir
# TODO handle very big .png (convert) animation?
URL=$(echo $URL | sed 's/\.PNG/.png/')
curl -s -L $URL > $FILENAME
echo sleep 1
sleep 1
fi
done
ls -Sl $IMG|tail -20
ipfs add -r -Q $DB
ipfs add -r -Q $IMG
@ohac

This comment has been minimized.

Copy link
Owner Author

@ohac ohac commented Mar 22, 2021

あるファイルのタイムスタンプを起点にカードの更新分のJSONから名称を取り出すスニペット。

csv=$(ls -l --time-style +%s $TMPD/card_list_update.txt | tr -s ' ' ,)
mtime=$(echo $csv | cut -d, -f6)
# or mtime=$(date +%s)
updt=$((${mtime}-60*60*24*1))
curl -s "$API/card_detail?update_time=$updt" \
    | jq -M '.details[].asset_common_name' \
    | sed 's/"//g' \
    | busybox sort > $TMPD/card_list_update.txt
@ohac

This comment has been minimized.

Copy link
Owner Author

@ohac ohac commented Mar 22, 2021

FOOBARというカードが更新された場合にすべきこと

  • $DB/FOOBAR.jsonがあるか調べる
  • あれば関連付けされた画像を$OLDへmvする
  • FOOBAR.jsonを削除する
  • あとは通常の処理で再度取得される流れに
@ohac

This comment has been minimized.

Copy link
Owner Author

@ohac ohac commented Mar 23, 2021

apngファイルの判別例

ls img/*.png \
  | xargs -i bash -c 'ffprobe -hide_banner {} 2>&1' \
  > /tmp/ffprobelog.log

grep ', apng' /tmp/ffprobelog.log | cut -d \' -f2
@ohac

This comment has been minimized.

Copy link
Owner Author

@ohac ohac commented Mar 24, 2021

拡張子が合っていないものを表示する例

ls img/*.png | xargs -i file {} | grep -v PNG
ls img/*.gif | xargs -i file {} | grep -v GIF
ls img/*.jpg | xargs -i file {} | grep -v JPEG
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment