Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Download assets from private Github releases
#!/usr/bin/env bash
#
# gh-dl-release! It works!
#
# This script downloads an asset from latest or specific Github release of a
# private repo. Feel free to extract more of the variables into command line
# parameters.
#
# PREREQUISITES
#
# curl, wget, jq
#
# USAGE
#
# Set all the variables inside the script, make sure you chmod +x it, then
# to download specific version to my_app.tar.gz:
#
# gh-dl-release 2.1.1 my_app.tar.gz
#
# to download latest version:
#
# gh-dl-release latest latest.tar.gz
#
# If your version/tag doesn't match, the script will exit with error.
TOKEN="<github_access_token>"
REPO="<user_or_org>/<repo_name>"
FILE="<name_of_asset_file>" # the name of your release asset file, e.g. build.tar.gz
VERSION=$1 # tag name or the word "latest"
GITHUB="https://api.github.com"
alias errcho='>&2 echo'
function gh_curl() {
curl -H "Authorization: token $TOKEN" \
-H "Accept: application/vnd.github.v3.raw" \
$@
}
if [ "$VERSION" = "latest" ]; then
# Github should return the latest release first.
parser=".[0].assets | map(select(.name == \"$FILE\"))[0].id"
else
parser=". | map(select(.tag_name == \"$VERSION\"))[0].assets | map(select(.name == \"$FILE\"))[0].id"
fi;
asset_id=`gh_curl -s $GITHUB/repos/$REPO/releases | jq "$parser"`
if [ "$asset_id" = "null" ]; then
errcho "ERROR: version not found $VERSION"
exit 1
fi;
wget -q --auth-no-challenge --header='Accept:application/octet-stream' \
https://$TOKEN:@api.github.com/repos/$REPO/releases/assets/$asset_id \
-O $2
@maxenglander

This comment has been minimized.

Copy link

@maxenglander maxenglander commented Aug 13, 2016

Exactly what I needed - TY!

@nneves

This comment has been minimized.

Copy link

@nneves nneves commented Jan 16, 2017

Thanks, this worked! Cheers!

@nneves

This comment has been minimized.

Copy link

@nneves nneves commented Jan 18, 2017

sorry, line 47 should be:

asset_id=`gh_curl -L -s $GITHUB/repos/$REPO/releases | jq "$parser"`
@nneves

This comment has been minimized.

Copy link

@nneves nneves commented Jan 18, 2017

Please ignore that, someone in my organization moved the REPO around and broke the script! All good, nice work!

@leoforney

This comment has been minimized.

Copy link

@leoforney leoforney commented Feb 5, 2017

Can you give an example usage of this?

@saaguero

This comment has been minimized.

Copy link

@saaguero saaguero commented Jun 28, 2017

This is a great script, thanks!

There is one problem though that I've been facing. If the release artifact is not in the first page (ie: you have to paginate until you get the desired release) this won't work.

So I ended up with a modified version to directly consume a particular tag, adapt it to your needs following the github documentation.

TOKEN="<github_access_token>"
REPO="<user_or_org>/<repo_name>"
FILE="<name_of_asset_file>"      # the name of your release asset file, e.g. build.tar.gz
VERSION=$1                       # tag name or the word "latest"
GITHUB="https://api.github.com"

alias errcho='>&2 echo'

function gh_curl() {
  curl -H "Authorization: token $TOKEN" \
       -H "Accept: application/vnd.github.v3.raw" \
       $@
}

parser=".assets[0].id"
asset_id=$(gh_curl -s $GITHUB/api/v3/repos/$REPO/releases/tags/$VERSION | jq $parser)

if [ "$asset_id" = "null" ]; then
  errcho "ERROR: version not found $VERSION"
  exit 1
fi;

wget -q --auth-no-challenge --header='Accept:application/octet-stream' \
  https://$TOKEN:@api.github.com/repos/$REPO/releases/assets/$asset_id \
  -O $2
@dankhan-adl

This comment has been minimized.

Copy link

@dankhan-adl dankhan-adl commented Jul 16, 2017

I am getting

parse error: Invalid numeric literal at line 1, column 9
@dedelala

This comment has been minimized.

Copy link

@dedelala dedelala commented Jun 4, 2018

This can be done without using a token in the URL, but the token cannot be in the header sent to the redirect link.

redir=$(curl -H "Authorization: token $TOKEN" -H "Accept: application/octet-stream" \
  -w "%{redirect_url}" -o "$dest" "$url")

# redir will be a non-empty string if the response was a 302, and dest will be an empty file
[[ -n $redir ]] && curl -o "$dest" "$url"
@deevus

This comment has been minimized.

Copy link

@deevus deevus commented Jan 15, 2019

Update using getopts instead of hard coded variables https://gist.github.com/deevus/af29b590e53a8249df5a852a7f7c5b14

@alexivkin

This comment has been minimized.

Copy link

@alexivkin alexivkin commented Feb 6, 2019

wget can be replaced with
curl -sL --header "Authorization: token $TOKEN" --header 'Accept: application/octet-stream' https://api.github....

@wyozi

This comment has been minimized.

Copy link

@wyozi wyozi commented Mar 6, 2019

I made a Github action partially based on this: https://github.com/wyozi/download-gh-release-asset

@jessp01

This comment has been minimized.

Copy link

@jessp01 jessp01 commented Aug 5, 2019

Thanks for this. Quite useful. Here's a somewhat more minimal version that doesn't require wget and accepts the needed params as CLI args:

if [ $# -lt 4 ] ;then
    echo "Usage: <github token> <org/repo> <filename> <version or 'latest'>"
    exit 1
fi

TOKEN="$1"
REPO="$2"
FILE="$3"      # the name of your release asset file, e.g. build.tar.gz
VERSION=$4                       # tag name or the word "latest"
GITHUB_API_ENDPOINT="api.github.com"

alias errcho='>&2 echo'

function gh_curl() {
  curl -sL -H "Authorization: token $TOKEN" \
       -H "Accept: application/vnd.github.v3.raw" \
       $@
}

if [ "$VERSION" = "latest" ]; then
  # Github should return the latest release first.
  PARSER=".[0].assets | map(select(.name == \"$FILE\"))[0].id"
else
  PARSER=". | map(select(.tag_name == \"$VERSION\"))[0].assets | map(select(.name == \"$FILE\"))[0].id"
fi

ASSET_ID=`gh_curl https://$GITHUB_API_ENDPOINT/repos/$REPO/releases | jq "$PARSER"`
if [ "$ASSET_ID" = "null" ]; then
  errcho "ERROR: version not found $VERSION"
  exit 1
fi

curl -sL --header "Authorization: token $TOKEN" --header 'Accept: application/octet-stream' https://$TOKEN:@$GITHUB_API_ENDPOINT/repos/$REPO/releases/assets/$ASSET_ID > $FILE
@germanattanasio

This comment has been minimized.

Copy link

@germanattanasio germanattanasio commented Aug 5, 2019

@jessp01 to run this I had to remove the gh_curl and do it in inline

@jessp01

This comment has been minimized.

Copy link

@jessp01 jessp01 commented Aug 5, 2019

Hi @germanattanasio,

I'm not sure I understand what you mean. Both the call to gh_curl():

ASSET_ID=`gh_curl https://$GITHUB_API_ENDPOINT/repos/$REPO/releases | jq "$PARSER"`

and:

curl -sL --header "Authorization: token $TOKEN" --header 'Accept: application/octet-stream' https://$TOKEN:@$GITHUB_API_ENDPOINT/repos/$REPO/releases/assets/$ASSET_ID > $FILE

Are needed. One fetches the asset ID using the API and the other fetches the actual asset/file.
The reason why both requests cannot be made by calling gh_curl() as it is now is because it sets the Accept: application/vnd.github.v3.raw headers, which is needed for the first request but hinders the latter. You could refactor the function so that it sets either header based on the request (fetching the ASSET_ID or downloading the actual asset). To me, it felt like too much trouble for this sort of script :)

@germanattanasio

This comment has been minimized.

Copy link

@germanattanasio germanattanasio commented Aug 6, 2019

I was trying to say that when I run the script with the gh_curl function in Travis I get an error. I ended up doing:

ASSET_ID=`curl -sL -H "Authorization: token $TOKEN" -H "Accept: application/vnd.github.v3.raw" https://$GITHUB_API_ENDPOINT/repos/$REPO/releases | jq "$PARSER"`
@kavinda1995

This comment has been minimized.

Copy link

@kavinda1995 kavinda1995 commented Aug 15, 2019

Maaaan. You save my day and me ❤️ 😍
Trying to find a solution to fix 404 error in whole day

@hoomaan-kh

This comment has been minimized.

Copy link

@hoomaan-kh hoomaan-kh commented Nov 27, 2019

Thanks it was awesome !

@fabriciobastian

This comment has been minimized.

Copy link

@fabriciobastian fabriciobastian commented Dec 28, 2019

Thanks! Awesome! I created this action to download assets from releases based on this code

@jairov4

This comment has been minimized.

Copy link

@jairov4 jairov4 commented Apr 21, 2020

Nice I made my version from this

@jonatasteixeira

This comment has been minimized.

Copy link

@jonatasteixeira jonatasteixeira commented May 8, 2020

When I tried to download file using curl, I get the response:

curl -sL --header "Authorization: token $TOKEN" --header 'Accept: application/octet-stream' https://$TOKEN:@$GITHUB_API_ENDPOINT/repos/$REPO/releases/assets/$ASSET_ID > $FILE

{
  "message": "Unsupported 'Accept' header: [#<Sinatra::Request::AcceptEntry:0x00007feb2a6c3e88 @entry=\"application/octet-stream\", @type=\"application/octet-stream\", @params={}, @q=1.0>]. Must accept 'application/json'.",
  "documentation_url": "https://developer.github.com/v3/media"
}
@jonatasteixeira

This comment has been minimized.

Copy link

@jonatasteixeira jonatasteixeira commented May 8, 2020

When I tried to download file using curl, I get the response:

curl -sL --header "Authorization: token $TOKEN" --header 'Accept: application/octet-stream' https://$TOKEN:@$GITHUB_API_ENDPOINT/repos/$REPO/releases/assets/$ASSET_ID > $FILE

{
  "message": "Unsupported 'Accept' header: [#<Sinatra::Request::AcceptEntry:0x00007feb2a6c3e88 @entry=\"application/octet-stream\", @type=\"application/octet-stream\", @params={}, @q=1.0>]. Must accept 'application/json'.",
  "documentation_url": "https://developer.github.com/v3/media"
}

According to: https://curl.haxx.se/docs/CVE-2018-1000007.html is necessary updarte curl to version >= 7.58.0

@gaieges

This comment has been minimized.

Copy link

@gaieges gaieges commented May 8, 2020

Just discovered the above for myself. If you get the following error on the call to the s3 redirected url, it's because you're using an older version of curl and require > 7.58.0. The reason is that the authorization header gets passed along to s3, and not only is that insecure, it also breaks the s3 request.

* The requested URL returned error: 400 Bad Request

@jacksonporter

This comment has been minimized.

Copy link

@jacksonporter jacksonporter commented Jun 25, 2020

Thank you! This is fantastic!

@EduardoSantanaSeverino

This comment has been minimized.

Copy link

@EduardoSantanaSeverino EduardoSantanaSeverino commented Jun 27, 2020

This is my version of download the latest asset file of a release for a private repository. In case you want to use this script with Jenkins you would need to sent the token while executing the script.

Steps:

  1. Create a file with: nano download.sh and fill the file with the repository name and owner.
  2. Give permissions to file: chmod +x download.sh
  3. Execute the file: ./download.sh "--GITHUB TOKEN HERE--"

Example of the file:

#!/bin/bash

# This script downloads the first asset from the latest Github release of a
# private repo. 
#
# PREREQUISITES
#
# curl, jq
#
# USAGE
#
# Set owner and repo variables inside the script, make sure you chmod +x it.
#
#     ./download.sh "--GITHUB TOKEN HERE--"
#

# Define variables
echo "---------------------------------------------------------------------"
echo "Define variables"
echo "---------------------------------------------------------------------"

owner="--Repository Owner--"
repo="--Repository Name--"
GITHUB_API_TOKEN=$1
GH_API="https://api.github.com"
GH_REPO="$GH_API/repos/$owner/$repo"
GH_LATEST="$GH_REPO/releases/latest"
AUTH="Authorization: token $GITHUB_API_TOKEN"

# Read asset name and id
echo "---------------------------------------------------------------------"
echo "Read asset name and id"
echo "---------------------------------------------------------------------"

response=$(curl -sH "$AUTH" $GH_LATEST)
id=`echo "$response" | jq '.assets[0] .id' |  tr -d '"'`
name=`echo "$response" | jq '.assets[0] .name' |  tr -d '"'`
GH_ASSET="$GH_REPO/releases/assets/$id"

# Print Details
echo "---------------------------------------------------------------------"
echo "Print Details"
echo "Assets Id: $id"
echo "Name: $name"
echo "Assets URL: $GH_ASSET"
echo "---------------------------------------------------------------------"

# Downloading asset file
echo "---------------------------------------------------------------------"
echo "Downloading asset file"
echo "---------------------------------------------------------------------"
curl -v -L -o "$name" -H "$AUTH" -H 'Accept: application/octet-stream' "$GH_ASSET"

@jboyd01

This comment has been minimized.

Copy link

@jboyd01 jboyd01 commented Jul 23, 2020

+1, good stuff. @jessp01 thanks for posting your refinements.

@a0s

This comment has been minimized.

Copy link

@a0s a0s commented Aug 16, 2020

Dont' forget to enable repo permission (yes, exactly root permission, enabling all nested permissions does not work) for the token

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.