Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
#!/bin/bash
ALL_TEXT=all.txt
URL="https://pointcloud.pref.shizuoka.jp/lasmap/ankenmapsrc?request=MarkerSet&Xmax=139.493408203125&Xmin=137.27279663085938&Ymax=35.52775582793653&Ymin=34.42163771567672"
BASE_URL="https://pointcloud.pref.shizuoka.jp/lasmap/ankendetail?ankenno="
RESULT=result.txt
wget -O $ALL_TEXT $URL
records=(`cat $ALL_TEXT | tr -s '?' ' '`)
for record in "${records[@]}"; do
array=(`echo $record | tr -s ':' ' '`)
target="${array[0]}"
if [[ ${target} =~ ^[0-9A-Za-z]+$ ]]; then
if [[ ${target} = MMS ]]; then
echo "wrong"
else
url=${BASE_URL}${target}
mkdir -p "mirror/${target}"
cd "mirror/${target}"
wget --wait=10 --random-wait -m -p -E -k $url
cd ../..
echo "fetched ${url}" >> $RESULT
fi
fi
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment