Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Download a single file from a private GitHub repo. You'll need an access token as described in this GitHub Help article: https://help.github.com/articles/creating-an-access-token-for-command-line-use
curl --header 'Authorization: token INSERTACCESSTOKENHERE' \
--header 'Accept: application/vnd.github.v3.raw' \
--remote-name \
--location https://api.github.com/repos/owner/repo/contents/path
# Example...
TOKEN="INSERTACCESSTOKENHERE"
OWNER="BBC-News"
REPO="responsive-news"
PATH="scripts/build/tabloid.sh"
FILE="https://api.github.com/repos/$OWNER/$REPO/contents/$PATH"
curl --header 'Authorization: token $TOKEN' \
--header 'Accept: application/vnd.github.v3.raw' \
--remote-name \
--location $FILE
  • -H --header Extra header to use when getting a web page. You may specify any number of extra headers

  • -O --remote-name Write output to a local file named like the remote file we get (only the file part of the remote file is used, the path is cut off)

  • -L --location If the server reports that the requested page has moved to a different location (indicated with a Location: header and a 3XX response code), this option will make curl redo the request on the new place

@ohadperry
Copy link

ohadperry commented Oct 27, 2015

👍

@Kif11
Copy link

Kif11 commented Dec 11, 2015

Is there a way to do the same thing without authorization?

@hayesmaker
Copy link

hayesmaker commented Mar 8, 2016

this only works on files <1mb unfortunately.. but if that's not an issue for you it's useful

@jmcshane
Copy link

jmcshane commented Jun 14, 2016

Just as a note, the remote-name parameter can append the ref parameter if present. I haven't found a good workaround to this

@brikis98
Copy link

brikis98 commented Jun 20, 2016

Great script!

If you download files from private GitHub repos often, you can also check out fetch, an open source, cross-platform tool which makes it easy to download source files and release assets from a git tag, commit, or branch of public and private GitHub repos.

For example, to download the file baz from version 0.1.3 of a private GitHub repo to /tmp, you would do the following:

GITHUB_OAUTH_TOKEN="your token"
fetch --repo="https://github.com/foo/bar" --tag="0.1.3" --source-path="/baz" /tmp

@haemi
Copy link

haemi commented Jul 20, 2016

does this also work for github enterprise?

@girishkg
Copy link

girishkg commented Aug 8, 2016

fetch doesn't work for the URL with subdomain like "github.example.com"
gruntwork-io/fetch#14

@mdtareque
Copy link

mdtareque commented Aug 16, 2016

Any idea why I am not able to run this? I am getting below error

$ ./getGithubFile.sh
curl: (77) error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
./getGithubFile.sh: line 16: curl: command not found

@zevarito
Copy link

zevarito commented Aug 18, 2016

@mdtareque did you override your PATH env variable perhaps?

@chusiang
Copy link

chusiang commented Oct 17, 2016

Hi, @mdtareque:

The PATH="scripts/build/tabloid.sh" will override system $PATH, maype you can rename it.

@neiledelsten
Copy link

neiledelsten commented Feb 6, 2017

Ref rem7's comment about the ref parameter for downloading from a specific branch, I'm struggling to work out how to integrate this into the curl.sh shell script. Can anyone provide an example?

@brooksa321
Copy link

brooksa321 commented Apr 10, 2017

@mkarthik415
Copy link

mkarthik415 commented Jun 4, 2017

I am getting below message when trying to access a private repo.

curl -H 'Authorization: token TOKEN' -H 'Accept: application/vnd.github.v3.raw' -O \ -L https://api.github.com/mkarthik415/repo/blob/master/AppManager/AppManager/maven-metadata-local.xml
curl: Remote file name has no length!
curl: try 'curl --help' or 'curl --manual' for more information
{
"message": "Not Found",
"documentation_url": "https://developer.github.com/v3"
}

@vsmori
Copy link

vsmori commented Aug 9, 2017

Just one comment: rename PATH to another variable name, since PATH is a special variable in Linux Console. You may get "command not found" after setting PATH used in the example.

@tianqig
Copy link

tianqig commented Aug 23, 2017

I would like to download a file from a private repo which I am a member, is it possible I can use the download link to download the file from a browser? I tried it without an error message, but failed to download the file.

BTW, I already logged into my Github account from the browser.

Thanks

@vambo
Copy link

vambo commented Oct 25, 2017

+1 to renaming PATH

@vonas
Copy link

vonas commented Nov 5, 2017

Is there a way that doesn't utilize the GitHub API?
I usually download a file via raw.githubusercontent.com but that doesn't seem to be available for private repositories.

@CodeMan99
Copy link

CodeMan99 commented Feb 1, 2018

Github enforces usage of a valid User-Agent in the request headers.

If you are making the same request multiple times, include a If-Modified-Since header and respect the 304 Not Modified response (using modification time via --time-cond).

curl --header "Authorization: token $TOKEN" \
     --header "Accept: application/vnd.github.v3.raw" \
     --header "User-Agent: ${OWNER}/${REPO} (curl v7.47.0)" \
     --time-cond "$CACHE" \
     --remote-name \
     --location "$FILE"

@suraj19
Copy link

suraj19 commented Sep 10, 2018

How can I download the entire repository from GitHub any idea, please??

@VincentCATILLON
Copy link

VincentCATILLON commented Sep 11, 2018

Using fetch ($TOKEN -> TOKEN, $FILE -> FILE):

fetch('https://api.github.com/repos/owner/repo/contents/path', {
  headers: {
    'Authorization': 'token ' + TOKEN,
    'Accept': 'application/vnd.github.v3.raw',
  },
})
  .then(response => response.buffer())
  .then(content => {
    console.log('[FILE]', 'Writing ' + FILE);
    fs.writeFileSync(FILE, content);
  })
  .catch(() => {
    throw new Error('File unreachable');
  });

@vnermolaev
Copy link

vnermolaev commented Oct 3, 2018

+1 Rename $PATH

@oganm
Copy link

oganm commented Oct 12, 2018

Note that this endpoint is limited to 1 MB files. any larger will fail. working on a workaround

@oganm
Copy link

oganm commented Oct 16, 2018

I have found a workaround for large files (up to 100 MB but as far as I know it is not possible to drop larger files to github so this should work with all files)

the contents endpoint used above cannot get the file if larger than 1 MB so trim the $PATH to direct to the folder where your file lies in the initial request using the contents endpoint

In the response, find your file listed, get its SHA,

use the /repos/:username/:reponame/git/blobs/:sha endpoint to get your content in 64 bit encoded form. Decode it using your favorite decoder.

For reference, see the R code below

getGithubFile = function(githubPath,branch = 'master', downloadPath = NULL,token = NULL){
    if(is.null(downloadPath)){
        downloadPath = tempfile()
    }
    
    path = strsplit(githubPath,'/')[[1]]
    file = paste(path[3:length(path)], collapse = '/')
    contents = gh::gh('GET /repos/:username/:reponame/contents/:dir?ref=:branch',
                  username = path[1],
                  reponame = path[2],
                  branch = branch,
                  dir = dirname(file),
                  .token = token)
    
    names(contents) = contents %>% purrr::map_chr('name')
    
    fileInfo = contents[contents %>% purrr::map_chr('name') %>% {.%in%basename(file)}][[1]]
    
    blob = gh::gh('GET /repos/:username/:reponame/git/blobs/:sha',
                  username = path[1],
                  reponame = path[2],
                  sha = fileInfo$sha,
                  .token = token)
    
    decodeContent = openssl::base64_decode(blob$content)
    writeBin(decodeContent,downloadPath)
    
    return(invisible(downloadPath))
}

@interaminense
Copy link

interaminense commented Nov 3, 2018

Using API V4 with Graphql

query {
   repository(name: "repo-name", owner: "owner") {
      object(expression: "branch:file/to/path.extension") {
         ... on Blob {
            text
         }
      }
   }
}

@eliranmal
Copy link

eliranmal commented May 30, 2019

@Kif11 - it's possible with basic auth (without the user having to explicitly create a token first).

it can be boiled down to a one-liner (see the original gist for details):

curl -u "$USER" -k https://${GHE_DOMAIN}/raw/${REPO_OWNER}/${REPO_NAME}/${REF}/${FILE} > ${FILE}

@jean
Copy link

jean commented Nov 27, 2019

I don't know if things changed since May, but neither token nor basic auth works for me. I'm getting

{
  "message": "Not Found",
  "documentation_url": "https://developer.github.com/v3"
}

@almajoy
Copy link

almajoy commented Dec 6, 2019

curl --header 'Authorization: token INSERTACCESSTOKENHERE'
--header 'Accept: application/vnd.github.v3.raw'
--remote-name
--location https://api.github.com/repos/owner/repo/contents/path

Hey,

it code can work using the is oauth app token any idea please share me?

@zelfick
Copy link

zelfick commented May 24, 2020

use the solution in the first part of the article it works without problem

@frederico-klein
Copy link

frederico-klein commented Jan 24, 2021

I don't know if things changed since May, but neither token nor basic auth works for me. I'm getting

{
  "message": "Not Found",
  "documentation_url": "https://developer.github.com/v3"
}

I thought this was no longer working, however what I did wrong is not adding the correct permissions to the token I was generating.

I marked

"repo

Full control of private repositories "

to get it working. There is maybe a less permissive alternative that still allows you to just read.

@mrmattipants
Copy link

mrmattipants commented May 2, 2021

Fashionably Late, as always!

Originally, I was having trouble with the Token, itself.
However, after creating a New Token, with Everything Checked, my issues seemed to Clear-up.

From Command Line, the following worked, on my end.

curl -v -H "Authorization: token ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" -H "Accept: application/vnd.github.v4.raw" -O -L "https://api.github.com/repos/<UserName>/<RepoName>/contents/<FolderPath>/<FileName>.ps1"

With that being said, I would imagine that something like the following, should do the trick.

curl --header 'Authorization: token ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
--header 'Accept: application/vnd.github.v3.raw'
--location 'https://api.github.com/repos/<UserName>/<RepoName>/contents/<FolderPath>/<FileName>.ps1'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment