Download entire iCloud shared albums

icloud-album-download.sh

#!/bin/bash

# requires jq
# arg 1: iCloud web album URL
# arg 2: folder to download into (optional)

function curl_post_json {
	curl -sH "Content-Type: application/json" -X POST -d "@-" "$@"
}

BASE_API_URL="https://p23-sharedstreams.icloud.com/$(echo $1 | cut -d# -f2)/sharedstreams"

pushd $2 > /dev/null
STREAM=$(echo '{"streamCtag":null}' | curl_post_json "$BASE_API_URL/webstream")
CHECKSUMS=$(echo $STREAM | jq -r '.photos[] | [(.derivatives[] | {size: .fileSize | tonumber, value: .checksum})] | max_by(.size | tonumber).value')

echo $STREAM \
| jq -c "{photoGuids: [.photos[].photoGuid]}" \
| curl_post_json "$BASE_API_URL/webasseturls" \
| jq -r '.items[] | "https://" + .url_location + .url_path' \
| while read URL; do
	for CHECKSUM in $CHECKSUMS; do
		if echo $URL | grep $CHECKSUM > /dev/null; then
			curl -sOJ $URL &
			break
		fi
	done
done

popd > /dev/null
wait

This didn't work for me for an album that had >300 items. I came across other similar script written in python (https://github.com/VMannello/iCloud-PS-Download/blob/master/iCloudBD.py) that was doing batches of 20 items requests to /webassetsurls endpoint. That seemed to do the trick.

That said I don't really know how I would implement batching with jq so just leaving a comment here in case someone faces this issue and wonders wtf.

When running in Terminal OSX I get following errors
/Users/myname/Downloads/icloud-album-download.sh: line 15: jq: command not found
/Users/myname/Downloads/icloud-album-download.sh: line 18: jq: command not found
/Users/myname/Downloads/icloud-album-download.sh: line 20: jq: command not found

No longer seems to work. :-/ You probably need to replace "p23" by "p43", but even then the call to "webasserturls" doesn't work.

I make few changes.

1. Check right hostname
2. Add checksum to url

Works for me. Lets try

#!/bin/bash

# requires jq
# arg 1: iCloud web album URL
# arg 2: folder to download into (optional)

function curl_post_json {
	curl -sH "Content-Type: application/json" -X POST -d "@-" "$@"
}

BASE_API_URL="https://p23-sharedstreams.icloud.com/$(echo $1 | cut -d# -f2)/sharedstreams"

pushd $2 > /dev/null
STREAM=$(echo '{"streamCtag":null}' | curl_post_json "$BASE_API_URL/webstream")
HOST=$(echo $STREAM | jq '.["X-Apple-MMe-Host"]' | cut -c 2- | rev | cut -c 2- | rev)

if [ "$HOST" ]; then
    BASE_API_URL="https://$(echo $HOST)/$(echo $1 | cut -d# -f2)/sharedstreams"
    STREAM=$(echo '{"streamCtag":null}' | curl_post_json "$BASE_API_URL/webstream")
fi

CHECKSUMS=$(echo $STREAM | jq -r '.photos[] | [(.derivatives[] | {size: .fileSize | tonumber, value: .checksum})] | max_by(.size | tonumber).value')

echo $STREAM \
| jq -c "{photoGuids: [.photos[].photoGuid]}" \
| curl_post_json "$BASE_API_URL/webasseturls" \
| jq -r '.items | to_entries[] | "https://" + .value.url_location + .value.url_path + "&" + .key' \
| while read URL; do
	for CHECKSUM in $CHECKSUMS; do
		if echo $URL | grep $CHECKSUM > /dev/null; then
			curl -sOJ $URL &
			break
		fi
	done
done

popd > /dev/null
wait

Where do you put the URL?

Last version working for me thanks !

@jeffuu in terminal, you have to enter :
./icloud-album-download.sh https://www.icloud.com/path/to/album path/to/local/folder

I make few changes.

1. Check right hostname
2. Add checksum to url

Works for me. Lets try

#!/bin/bash

# requires jq
# arg 1: iCloud web album URL
# arg 2: folder to download into (optional)

function curl_post_json {
	curl -sH "Content-Type: application/json" -X POST -d "@-" "$@"
}

BASE_API_URL="https://p23-sharedstreams.icloud.com/$(echo $1 | cut -d# -f2)/sharedstreams"

pushd $2 > /dev/null
STREAM=$(echo '{"streamCtag":null}' | curl_post_json "$BASE_API_URL/webstream")
HOST=$(echo $STREAM | jq '.["X-Apple-MMe-Host"]' | cut -c 2- | rev | cut -c 2- | rev)

if [ "$HOST" ]; then
    BASE_API_URL="https://$(echo $HOST)/$(echo $1 | cut -d# -f2)/sharedstreams"
    STREAM=$(echo '{"streamCtag":null}' | curl_post_json "$BASE_API_URL/webstream")
fi

CHECKSUMS=$(echo $STREAM | jq -r '.photos[] | [(.derivatives[] | {size: .fileSize | tonumber, value: .checksum})] | max_by(.size | tonumber).value')

echo $STREAM \
| jq -c "{photoGuids: [.photos[].photoGuid]}" \
| curl_post_json "$BASE_API_URL/webasseturls" \
| jq -r '.items | to_entries[] | "https://" + .value.url_location + .value.url_path + "&" + .key' \
| while read URL; do
	for CHECKSUM in $CHECKSUMS; do
		if echo $URL | grep $CHECKSUM > /dev/null; then
			curl -sOJ $URL &
			break
		fi
	done
done

popd > /dev/null
wait

Thank you very much, working for me !
Cheers

This is really cool. Do you know if there is a way to get the comments?

@vchatela:

1. I downloaded this script and run it, but didn't know why it's not working for me:
   https://www.icloud.com/photos/ -- is the URL of my browser logged into icloud photo;
   ./ -- this is the local directory in my MacBook:
   $ ./icloud-album-download.sh https://www.icloud.com/photos/ ./
   $ (return nothing back here, nothing downloaded, no error message)

2. tried it in shell debug mode: still the same, even no debugging info showed:
   $ sh +x icloud-album-download.sh https://www.icloud.com/photos/ ./
   $ (again, return nothing back here, nothing downloaded, no error message)

Do you know what's was wrong here?

Hi @gwu888
Did you try the original gist or the one I commented ? Only the second was working for me.
I used this script for a friend, but I think you need public sharing for icloud links. Your session cookie isn't shared and so it cannot works with this url for sure

some of the error occur because jq is missing.
You will found jq

jq is a lightweight and flexible command-line JSON processor

https://stedolan.github.io/jq/download/