Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Download entire iCloud shared albums
#!/bin/bash
# requires jq
# arg 1: iCloud web album URL
# arg 2: folder to download into (optional)
function curl_post_json {
curl -sH "Content-Type: application/json" -X POST -d "@-" "$@"
}
BASE_API_URL="https://p23-sharedstreams.icloud.com/$(echo $1 | cut -d# -f2)/sharedstreams"
pushd $2 > /dev/null
STREAM=$(echo '{"streamCtag":null}' | curl_post_json "$BASE_API_URL/webstream")
CHECKSUMS=$(echo $STREAM | jq -r '.photos[] | [(.derivatives[] | {size: .fileSize | tonumber, value: .checksum})] | max_by(.size | tonumber).value')
echo $STREAM \
| jq -c "{photoGuids: [.photos[].photoGuid]}" \
| curl_post_json "$BASE_API_URL/webasseturls" \
| jq -r '.items[] | "https://" + .url_location + .url_path' \
| while read URL; do
for CHECKSUM in $CHECKSUMS; do
if echo $URL | grep $CHECKSUM > /dev/null; then
curl -sOJ $URL &
break
fi
done
done
popd > /dev/null
wait
@vishna

This comment has been minimized.

Copy link

@vishna vishna commented Sep 30, 2017

This didn't work for me for an album that had >300 items. I came across other similar script written in python (https://github.com/VMannello/iCloud-PS-Download/blob/master/iCloudBD.py) that was doing batches of 20 items requests to /webassetsurls endpoint. That seemed to do the trick.

That said I don't really know how I would implement batching with jq so just leaving a comment here in case someone faces this issue and wonders wtf.

@DunhamGitHub

This comment has been minimized.

Copy link

@DunhamGitHub DunhamGitHub commented Sep 26, 2018

When running in Terminal OSX I get following errors
/Users/myname/Downloads/icloud-album-download.sh: line 15: jq: command not found
/Users/myname/Downloads/icloud-album-download.sh: line 18: jq: command not found
/Users/myname/Downloads/icloud-album-download.sh: line 20: jq: command not found

@a3nm

This comment has been minimized.

Copy link

@a3nm a3nm commented Dec 15, 2018

No longer seems to work. :-/ You probably need to replace "p23" by "p43", but even then the call to "webasserturls" doesn't work.

@WildDIC

This comment has been minimized.

Copy link

@WildDIC WildDIC commented Jan 22, 2019

I make few changes.

  1. Check right hostname
  2. Add checksum to url

Works for me. Lets try

#!/bin/bash

# requires jq
# arg 1: iCloud web album URL
# arg 2: folder to download into (optional)

function curl_post_json {
	curl -sH "Content-Type: application/json" -X POST -d "@-" "$@"
}

BASE_API_URL="https://p23-sharedstreams.icloud.com/$(echo $1 | cut -d# -f2)/sharedstreams"

pushd $2 > /dev/null
STREAM=$(echo '{"streamCtag":null}' | curl_post_json "$BASE_API_URL/webstream")
HOST=$(echo $STREAM | jq '.["X-Apple-MMe-Host"]' | cut -c 2- | rev | cut -c 2- | rev)

if [ "$HOST" ]; then
    BASE_API_URL="https://$(echo $HOST)/$(echo $1 | cut -d# -f2)/sharedstreams"
    STREAM=$(echo '{"streamCtag":null}' | curl_post_json "$BASE_API_URL/webstream")
fi

CHECKSUMS=$(echo $STREAM | jq -r '.photos[] | [(.derivatives[] | {size: .fileSize | tonumber, value: .checksum})] | max_by(.size | tonumber).value')

echo $STREAM \
| jq -c "{photoGuids: [.photos[].photoGuid]}" \
| curl_post_json "$BASE_API_URL/webasseturls" \
| jq -r '.items | to_entries[] | "https://" + .value.url_location + .value.url_path + "&" + .key' \
| while read URL; do
	for CHECKSUM in $CHECKSUMS; do
		if echo $URL | grep $CHECKSUM > /dev/null; then
			curl -sOJ $URL &
			break
		fi
	done
done

popd > /dev/null
wait
@jeffuu

This comment has been minimized.

Copy link

@jeffuu jeffuu commented Apr 12, 2019

Where do you put the URL?

@bricepepin

This comment has been minimized.

Copy link

@bricepepin bricepepin commented Aug 6, 2019

Last version working for me thanks !

@jeffuu in terminal, you have to enter :
./icloud-album-download.sh https://www.icloud.com/path/to/album path/to/local/folder

@vchatela

This comment has been minimized.

Copy link

@vchatela vchatela commented Sep 1, 2019

I make few changes.

  1. Check right hostname
  2. Add checksum to url

Works for me. Lets try

#!/bin/bash

# requires jq
# arg 1: iCloud web album URL
# arg 2: folder to download into (optional)

function curl_post_json {
	curl -sH "Content-Type: application/json" -X POST -d "@-" "$@"
}

BASE_API_URL="https://p23-sharedstreams.icloud.com/$(echo $1 | cut -d# -f2)/sharedstreams"

pushd $2 > /dev/null
STREAM=$(echo '{"streamCtag":null}' | curl_post_json "$BASE_API_URL/webstream")
HOST=$(echo $STREAM | jq '.["X-Apple-MMe-Host"]' | cut -c 2- | rev | cut -c 2- | rev)

if [ "$HOST" ]; then
    BASE_API_URL="https://$(echo $HOST)/$(echo $1 | cut -d# -f2)/sharedstreams"
    STREAM=$(echo '{"streamCtag":null}' | curl_post_json "$BASE_API_URL/webstream")
fi

CHECKSUMS=$(echo $STREAM | jq -r '.photos[] | [(.derivatives[] | {size: .fileSize | tonumber, value: .checksum})] | max_by(.size | tonumber).value')

echo $STREAM \
| jq -c "{photoGuids: [.photos[].photoGuid]}" \
| curl_post_json "$BASE_API_URL/webasseturls" \
| jq -r '.items | to_entries[] | "https://" + .value.url_location + .value.url_path + "&" + .key' \
| while read URL; do
	for CHECKSUM in $CHECKSUMS; do
		if echo $URL | grep $CHECKSUM > /dev/null; then
			curl -sOJ $URL &
			break
		fi
	done
done

popd > /dev/null
wait

Thank you very much, working for me !
Cheers

@Jwink3101

This comment has been minimized.

Copy link

@Jwink3101 Jwink3101 commented Feb 15, 2020

This is really cool. Do you know if there is a way to get the comments?

@gwu888

This comment has been minimized.

Copy link

@gwu888 gwu888 commented Jul 12, 2020

@vchatela:

  1. I downloaded this script and run it, but didn't know why it's not working for me:
    https://www.icloud.com/photos/ -- is the URL of my browser logged into icloud photo;
    ./ -- this is the local directory in my MacBook:
    $ ./icloud-album-download.sh https://www.icloud.com/photos/ ./
    $ (return nothing back here, nothing downloaded, no error message)

  2. tried it in shell debug mode: still the same, even no debugging info showed:
    $ sh +x icloud-album-download.sh https://www.icloud.com/photos/ ./
    $ (again, return nothing back here, nothing downloaded, no error message)

Do you know what's was wrong here?

@vchatela

This comment has been minimized.

Copy link

@vchatela vchatela commented Jul 14, 2020

Hi @gwu888
Did you try the original gist or the one I commented ? Only the second was working for me.
I used this script for a friend, but I think you need public sharing for icloud links. Your session cookie isn't shared and so it cannot works with this url for sure

@skauss

This comment has been minimized.

Copy link

@skauss skauss commented Oct 25, 2020

some of the error occur because jq is missing.
You will found jq

jq is a lightweight and flexible command-line JSON processor

https://stedolan.github.io/jq/download/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment