Download entire iCloud shared albums

icloud-album-download.sh

#!/bin/bash

# requires jq
# arg 1: iCloud web album URL
# arg 2: folder to download into (optional)

function curl_post_json {
	curl -sH "Content-Type: application/json" -X POST -d "@-" "$@"
}

BASE_API_URL="https://p23-sharedstreams.icloud.com/$(echo $1 | cut -d# -f2)/sharedstreams"

pushd $2 > /dev/null
STREAM=$(echo '{"streamCtag":null}' | curl_post_json "$BASE_API_URL/webstream")
CHECKSUMS=$(echo $STREAM | jq -r '.photos[] | [(.derivatives[] | {size: .fileSize | tonumber, value: .checksum})] | max_by(.size | tonumber).value')

echo $STREAM \
| jq -c "{photoGuids: [.photos[].photoGuid]}" \
| curl_post_json "$BASE_API_URL/webasseturls" \
| jq -r '.items[] | "https://" + .url_location + .url_path' \
| while read URL; do
	for CHECKSUM in $CHECKSUMS; do
		if echo $URL | grep $CHECKSUM > /dev/null; then
			curl -sOJ $URL &
			break
		fi
	done
done

popd > /dev/null
wait

i made some slight changes to be able to download also mp4, as well as eventual files with same name. Also hides the pushd/popd warnings when no specific path is set:

#!/bin/bash

# requires jq
# arg 1: iCloud web album URL
# arg 2: folder to download into (optional)

clear

function curl_post_json {
	curl -sH "Content-Type: application/json" -X POST -d "@-" "$@"
}

printf "Getting iCloud Stream\n"
BASE_API_URL="https://p23-sharedstreams.icloud.com/$(echo $1 | cut -d# -f2)/sharedstreams"

pushd $2 2> /dev/null
STREAM=$(echo '{"streamCtag":null}' | curl_post_json "$BASE_API_URL/webstream")
HOST=$(echo $STREAM | jq '.["X-Apple-MMe-Host"]' | cut -c 2- | rev | cut -c 2- | rev)

if [ "$HOST" ]; then
    BASE_API_URL="https://$(echo $HOST)/$(echo $1 | cut -d# -f2)/sharedstreams"
    STREAM=$(echo '{"streamCtag":null}' | curl_post_json "$BASE_API_URL/webstream")
fi

printf "Grabbing Large File Checksums\n"
CHECKSUMS=$(echo $STREAM | jq -r '.photos[] | [(.derivatives[] | {size: .fileSize | tonumber, value: .checksum})] | max_by(.size | tonumber).value')

printf "Adding Checksums to Array\n"
for CHECKSUM in $CHECKSUMS; do
    arrCHKSUM+=($CHECKSUM)
done
printf "Total Downloads: ${#arrCHKSUM[@]}\n"

# Dedup checksum to only include unique ids.
arrCHKSUM=($(printf "%s\n" "${arrCHKSUM[@]}" | sort -u))
printf "Unique Downloads: ${#arrCHKSUM[@]}\n"

printf "Streaming All Assets\n"
echo $STREAM \
| jq -c "{photoGuids: [.photos[].photoGuid]}" \
| curl_post_json "$BASE_API_URL/webasseturls" \
| jq -r '.items | to_entries[] | "https://" + .value.url_location + .value.url_path + "&" + .key' \
| while read URL; do

	# Get this URL's checksum value, not all URL's will be downloaded as there are both the fill size AND the thumbnail link in the Assets stream.
	LOCAL_CHECKSUM=$(echo "${URL##*&}")

	# If the url's checksum exists in the large checksum array then proceed with the download steps.
	if [[ " ${arrCHKSUM[*]} " =~ " ${LOCAL_CHECKSUM} " ]]; then

			# Get the filename from the URL, first we delimit on the forward slashes grabbing index 6 where the filename starts.
			# then we must delimit again on ? to remove all the URL parameters after the filename.
			# Example: https://www.example.com/4/5/IMG_0828.JPG?o=param1&v=param2&z=param3....
			FILE=$(echo $URL|cut -d "/" -f6 | cut -d "?" -f1)

			# Don't download movies
			if [[ "$FILE" == *.mp4* ]]; then
				echo "Downloading movie"
					curl -OJ $URL
			else

				# Don't download files that already exist
				if [[ -f "$FILE" ]]; then
					printf "File $FILE already present. Renaming..\n"
					TIMESTAMP=$(date +%s%N)
					curl $URL -o "${TIMESTAMP}_${FILE}"

				else
					# Original curl -sOJ $URL -> s = silent : O = download to file : J = Save using uploaded filename -- this also skips files that already exist.
					curl -OJ $URL
				fi

			fi

	else
		echo "Skipping Thumbnail"
	fi

done

popd 2> /dev/null
wait

i made some slight changes to be able to download also mp4, as well as eventual files with same name. Also hides the pushd/popd warnings when no specific path is set:

#!/bin/bash

# requires jq
# arg 1: iCloud web album URL
# arg 2: folder to download into (optional)

clear

function curl_post_json {
	curl -sH "Content-Type: application/json" -X POST -d "@-" "$@"
}

printf "Getting iCloud Stream\n"
BASE_API_URL="https://p23-sharedstreams.icloud.com/$(echo $1 | cut -d# -f2)/sharedstreams"

pushd $2 2> /dev/null
STREAM=$(echo '{"streamCtag":null}' | curl_post_json "$BASE_API_URL/webstream")
HOST=$(echo $STREAM | jq '.["X-Apple-MMe-Host"]' | cut -c 2- | rev | cut -c 2- | rev)

if [ "$HOST" ]; then
    BASE_API_URL="https://$(echo $HOST)/$(echo $1 | cut -d# -f2)/sharedstreams"
    STREAM=$(echo '{"streamCtag":null}' | curl_post_json "$BASE_API_URL/webstream")
fi

printf "Grabbing Large File Checksums\n"
CHECKSUMS=$(echo $STREAM | jq -r '.photos[] | [(.derivatives[] | {size: .fileSize | tonumber, value: .checksum})] | max_by(.size | tonumber).value')

printf "Adding Checksums to Array\n"
for CHECKSUM in $CHECKSUMS; do
    arrCHKSUM+=($CHECKSUM)
done
printf "Total Downloads: ${#arrCHKSUM[@]}\n"

# Dedup checksum to only include unique ids.
arrCHKSUM=($(printf "%s\n" "${arrCHKSUM[@]}" | sort -u))
printf "Unique Downloads: ${#arrCHKSUM[@]}\n"

printf "Streaming All Assets\n"
echo $STREAM \
| jq -c "{photoGuids: [.photos[].photoGuid]}" \
| curl_post_json "$BASE_API_URL/webasseturls" \
| jq -r '.items | to_entries[] | "https://" + .value.url_location + .value.url_path + "&" + .key' \
| while read URL; do

	# Get this URL's checksum value, not all URL's will be downloaded as there are both the fill size AND the thumbnail link in the Assets stream.
	LOCAL_CHECKSUM=$(echo "${URL##*&}")

	# If the url's checksum exists in the large checksum array then proceed with the download steps.
	if [[ " ${arrCHKSUM[*]} " =~ " ${LOCAL_CHECKSUM} " ]]; then

			# Get the filename from the URL, first we delimit on the forward slashes grabbing index 6 where the filename starts.
			# then we must delimit again on ? to remove all the URL parameters after the filename.
			# Example: https://www.example.com/4/5/IMG_0828.JPG?o=param1&v=param2&z=param3....
			FILE=$(echo $URL|cut -d "/" -f6 | cut -d "?" -f1)

			# Don't download movies
			if [[ "$FILE" == *.mp4* ]]; then
				echo "Downloading movie"
					curl -OJ $URL
			else

				# Don't download files that already exist
				if [[ -f "$FILE" ]]; then
					printf "File $FILE already present. Renaming..\n"
					TIMESTAMP=$(date +%s%N)
					curl $URL -o "${TIMESTAMP}_${FILE}"

				else
					# Original curl -sOJ $URL -> s = silent : O = download to file : J = Save using uploaded filename -- this also skips files that already exist.
					curl -OJ $URL
				fi

			fi

	else
		echo "Skipping Thumbnail"
	fi

done

popd 2> /dev/null
wait

works 100% - thank you

works great, thanks
any chance to modify for google photos shared albums?

Thanks @fusionneur now October 2023 and still working

i made some slight changes to be able to download also mp4, as well as eventual files with same name. Also hides the pushd/popd warnings when no specific path is set:

#!/bin/bash

# requires jq
# arg 1: iCloud web album URL
# arg 2: folder to download into (optional)

clear

function curl_post_json {
	curl -sH "Content-Type: application/json" -X POST -d "@-" "$@"
}

printf "Getting iCloud Stream\n"
BASE_API_URL="https://p23-sharedstreams.icloud.com/$(echo $1 | cut -d# -f2)/sharedstreams"

pushd $2 2> /dev/null
STREAM=$(echo '{"streamCtag":null}' | curl_post_json "$BASE_API_URL/webstream")
HOST=$(echo $STREAM | jq '.["X-Apple-MMe-Host"]' | cut -c 2- | rev | cut -c 2- | rev)

if [ "$HOST" ]; then
    BASE_API_URL="https://$(echo $HOST)/$(echo $1 | cut -d# -f2)/sharedstreams"
    STREAM=$(echo '{"streamCtag":null}' | curl_post_json "$BASE_API_URL/webstream")
fi

printf "Grabbing Large File Checksums\n"
CHECKSUMS=$(echo $STREAM | jq -r '.photos[] | [(.derivatives[] | {size: .fileSize | tonumber, value: .checksum})] | max_by(.size | tonumber).value')

printf "Adding Checksums to Array\n"
for CHECKSUM in $CHECKSUMS; do
    arrCHKSUM+=($CHECKSUM)
done
printf "Total Downloads: ${#arrCHKSUM[@]}\n"

# Dedup checksum to only include unique ids.
arrCHKSUM=($(printf "%s\n" "${arrCHKSUM[@]}" | sort -u))
printf "Unique Downloads: ${#arrCHKSUM[@]}\n"

printf "Streaming All Assets\n"
echo $STREAM \
| jq -c "{photoGuids: [.photos[].photoGuid]}" \
| curl_post_json "$BASE_API_URL/webasseturls" \
| jq -r '.items | to_entries[] | "https://" + .value.url_location + .value.url_path + "&" + .key' \
| while read URL; do

	# Get this URL's checksum value, not all URL's will be downloaded as there are both the fill size AND the thumbnail link in the Assets stream.
	LOCAL_CHECKSUM=$(echo "${URL##*&}")

	# If the url's checksum exists in the large checksum array then proceed with the download steps.
	if [[ " ${arrCHKSUM[*]} " =~ " ${LOCAL_CHECKSUM} " ]]; then

			# Get the filename from the URL, first we delimit on the forward slashes grabbing index 6 where the filename starts.
			# then we must delimit again on ? to remove all the URL parameters after the filename.
			# Example: https://www.example.com/4/5/IMG_0828.JPG?o=param1&v=param2&z=param3....
			FILE=$(echo $URL|cut -d "/" -f6 | cut -d "?" -f1)

			# Don't download movies
			if [[ "$FILE" == *.mp4* ]]; then
				echo "Downloading movie"
					curl -OJ $URL
			else

				# Don't download files that already exist
				if [[ -f "$FILE" ]]; then
					printf "File $FILE already present. Renaming..\n"
					TIMESTAMP=$(date +%s%N)
					curl $URL -o "${TIMESTAMP}_${FILE}"

				else
					# Original curl -sOJ $URL -> s = silent : O = download to file : J = Save using uploaded filename -- this also skips files that already exist.
					curl -OJ $URL
				fi

			fi

	else
		echo "Skipping Thumbnail"
	fi

done

popd 2> /dev/null
wait

Feb 9, 2024. Still working! Thank you very much guys!

Hi,
Can anyone explain how exactly to run this script?
I'm not a coder (never learned how to code) so I don't even know where to start to run this. Like, where do I paste the shared album link in the script. Am I supposed to edit the script before running it in Terminal?

p.s. I downloaded jq and python and Git because a tutorial told me to, but I still, don't know where to start with getting this script running.

Hi, Can anyone explain how exactly to run this script? I'm not a coder (never learned how to code) so I don't even know where to start to run this. Like, where do I paste the shared album link in the script. Am I supposed to edit the script before running it in Terminal?

p.s. I downloaded jq and python and Git because a tutorial told me to, but I still, don't know where to start with getting this script running.

hi. you don't need python, the code is a shell script (runs on linux (baremetal host, virtual machine, container or WSL) or linux related apps such as cygwin or git bash.

are you mainly using windows or linux?

a TLDR guide

1. First, you need jq, and also you need to be able to run it from anywhere, by properly installing or adding the portable version to environment variables. Assuming you downloaded it and properly installed it
   https://ioflood.com/blog/install-jq-command-linux/

2.1
if you are using windows, assuming you already have git downloaded from https://git-scm.com/download/win, you should have git bash in start menu, so you can run it and browser to the folder where you placed this script using cd
UPDATE 1 MAY 2024: git bash doesn't recognize rev command. Just use cygwin instead
eg.

cd /c/Users/<your_user>/Desktop

2.2
if you are using linux already, then just run the following in the terminal:

cd ~/Desktop

3
then make the script executable

chmod +x ./icloud-album-download.sh

4
then run it from current directory, by passing the URL as argument as described in the comments in the script (first lines)

./icloud-album-download.sh <URL>

# arg 1: iCloud web album URL
# arg 2: folder to download into (optional)

Would someone be able to help me out... I've used this app in the past, but having issues today when trying to run it again. Using Windows 11 with Git installed (jq as well)

./icloud-album-download.sh: line 18: rev: command not found
./icloud-album-download.sh: line 18: rev: command not found
Grabbing Large File Checksums
jq: error (at :1): Cannot iterate over null (null)
Adding Checksums to Array
Total Downloads: 0
Unique Downloads: 0
Streaming All Assets
jq: error (at :1): Cannot iterate over null (null)

Would someone be able to help me out... I've used this app in the past, but having issues today when trying to run it again. Using Windows 11 with Git installed (jq as well)

./icloud-album-download.sh: line 18: rev: command not found
./icloud-album-download.sh: line 18: rev: command not found
Grabbing Large File Checksums
jq: error (at :1): Cannot iterate over null (null)
Adding Checksums to Array
Total Downloads: 0
Unique Downloads: 0
Streaming All Assets
jq: error (at :1): Cannot iterate over null (null)

i updated the comment above. apparently rev command is not recognized by git bash. Just use cygwin instead. It's pretty easy to setup. Download the setup from https://www.cygwin.com/.

Thanks.

For anyone else who's technologically challenged, I used cd /cygdrive/<the path to your file>, ie. cd /cygdrive/c/Users/<your_user>/Desktop to navigate to where the file is located