Skip to content

Instantly share code, notes, and snippets.

@josh-padnick
Last active July 21, 2023 11:45
Show Gist options
  • Star 21 You must be signed in to star a gist
  • Fork 6 You must be signed in to fork a gist
  • Save josh-padnick/fdae42c07e648c798fc27dec2367da21 to your computer and use it in GitHub Desktop.
Save josh-padnick/fdae42c07e648c798fc27dec2367da21 to your computer and use it in GitHub Desktop.
Download a private binary release file from GitHub in bash
#!/usr/bin/env bash
#
# This is an adaptation of code I wrote to download a private binary from GitHub. Such...pain.
# Why can't GitHub just offer a standardized URL you can download a release binary from and attach
# your Github Personal Access Token as a header?
#
# Since this code is an adaptation it hasn't been directly tested, but the code it was adapted from works
# and hopefully you can get the missing piece you're after by looking here.
#
set -e
# Parse CLI args
readonly github_oauth_token="$1"
readonly git_tag="$2"
readonly github_repo_owner="$3"
readonly github_repo_name="$4"
readonly release_asset_filename="$5"
readonly output_path="$6"
# Get the "github tag id" of this release
github_tag_id=$(curl --silent --show-error \
--header "Authorization: token $github_oauth_token" \
--request GET \
"https://api.github.com/repos/$github_repo_owner/$github_repo_name/releases" \
| jq --raw-output ".[] | select(.tag_name==\"$git_tag\").id")
# Get the download URL of our desired asset
download_url=$(curl --silent --show-error \
--header "Authorization: token $github_oauth_token" \
--header "Accept: application/vnd.github.v3.raw" \
--location \
--request GET \
"https://api.github.com/repos/$github_repo_owner/$github_repo_name/releases/$github_tag_id" \
| jq --raw-output ".assets[] | select(.name==\"$release_asset_filename\").url")
# Get GitHub's S3 redirect URL
# Why not just curl's built-in "--location" option to auto-redirect? Because curl then wants to include all the original
# headers we added for the GitHub request, which makes AWS complain that we're trying strange things to authenticate.
redirect_url=$(curl --silent --show-error \
--header "Authorization: token $github_oauth_token" \
--header "Accept: application/octet-stream" \
--request GET \
--write-out "%{redirect_url}" \
"$download_url")
# Finally download the actual binary
sudo curl --silent --show-error \
--header "Accept: application/octet-stream" \
--output "$output_path" \
--request GET \
"$redirect_url"
@josh-padnick
Copy link
Author

Thanks! Fixed that one, too. :)

@pnathan
Copy link

pnathan commented Oct 23, 2019

Probably don't need sudo on line 48. :)

@jonasgraudums
Copy link

I was starting pulling hairs from my head getting binaries from GitHub. Just removed the "sudo" from line 48. Thanks!

@mathew-fleisch
Copy link

The first curl isn't necessary if you change the second curl to get the download url by tag
Change:
"https://api.github.com/repos/$github_repo_owner/$github_repo_name/releases/$github_tag_id" \
to:
"https://api.github.com/repos/$github_repo_owner/$github_repo_name/releases/tags/$git_tag" \

@mathew-fleisch
Copy link

mathew-fleisch commented Oct 28, 2020

also, the first curl is paged by 30 releases per page. so if the $git_tag isn't found in the first page, that curl will return a jq error like this jq: error (at <stdin>:4): Cannot index string with string "tag_name" jq: error (at <stdin>:4): Cannot iterate over null (null) curl: (3) <url> malformed curl: (3) <url> malformed

@mathew-fleisch
Copy link

mathew-fleisch commented Oct 29, 2020

ok. sorry. I have found another optimization. If you pass the filename via xargs, you can bypass that redirect problem. Other than defining the variable names, it becomes a one-liner and it'd look something like this:

curl -sL -H "Authorization: token $github_oauth_token" "https://api.github.com/repos/$github_repo_owner/$github_repo_name/releases/tags/$git_tag" | jq -r '.assets[] | select(.name == "'$release_asset_filename'").url' | xargs -I {} curl -sL -H "Authorization: token $github_oauth_token" -H 'Accept:application/octet-stream' -o $output_path {}

I made a new gist using this method here:
https://gist.github.com/mathew-fleisch/f2e0308eefe974701370f569027a7cfe

@zero88
Copy link

zero88 commented Jan 4, 2021

If anyone interest, give a try: https://github.com/zero88/gh-release-downloader

@josh-padnick
Copy link
Author

Actually, I can't believe I forgot to share here, but we wrote a first-class CLI tool in Go at https://github.com/gruntwork-io/fetch to do exactly what this script does and much more. It's worked well for us so far and seems to have a small, but active community of users.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment