Skip to content

Instantly share code, notes, and snippets.

@radianttap
Created June 6, 2019 11:23
Show Gist options
  • Save radianttap/19a88bb56973b3571247939bdcc8841d to your computer and use it in GitHub Desktop.
Save radianttap/19a88bb56973b3571247939bdcc8841d to your computer and use it in GitHub Desktop.
Fetch all WWDC 2019 session videos and PDFs
#!/bin/bash
#Setup the environment
mkdir tmp_download
cd tmp_download
#Extract IDs
echo "Downloading the index"
wget -q https://developer.apple.com/videos/wwdc2019/ -O index.html
# find parts of the document where data-released=true, all the way to the first H4 header where title of that talk is
# then find lines containing "videos/play/wwdc2019", then remove all chars except session number, then clean duplicated lines
cat index.html | sed -n '/data-released=\"true\"/,/\<h4/p' | grep videos/play/wwdc2019 | sed -e 's/.*wwdc2019\///' -e 's/\/\"\>//' | sed '$!N; /^\(.*\)\n\1$/!P; D' > ../downloadData
rm index.html
#Iterate through the talk IDs
while read -r line
do
echo "Trying $line"
#Download the page with the real download URL and the talk name
wget -q "https://developer.apple.com/videos/play/wwdc2019/$line/" -O webpage
#We grab the title of the page then clean it up
talkName=$(cat webpage | grep "<title" | sed -e "s/.*\<title\>//" -e "s/ \- WWDC 2019.*//")
#We grep "_hd_" which bring up the download URL, then some cleanup
#If we were to want SD video, all we would have to do is replace _hd_ by _sd_
dlURL=$(cat webpage | grep _hd_ | sed -e "s/.*href\=//" -e "s/\>.*//" -e "s/\"//g")
pdfURL=$(cat webpage | grep .pdf | grep devstreaming | sed -e "s/.*href\=//" -e "s/\>.*//" -e "s/\"//g" -e "s/ .*$//g")
rm webpage
#Is there a video URL?
if [ -z "$dlURL" ]; then
echo
else
echo "Video $line ($talkName)"
echo " url: $dlURL"
#Great, we download the file
wget -c "$dlURL" -O "../$line - $talkName.mp4"
fi
#Is there a PDF URL?
if [ -z "$pdfURL" ]; then
echo
else
echo "PDF $line ($talkName)"
echo " url: $pdfURL"
#Great, we download the file
wget -c "$pdfURL" -O "../$line - $talkName.pdf"
fi
done < "../downloadData"
#cleanup
cd ..
rm -rf tmp_download
rm downloadData
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment