Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Download a single file from a private GitHub repo. You'll need an access token as described in this GitHub Help article: https://help.github.com/articles/creating-an-access-token-for-command-line-use
curl --header 'Authorization: token INSERTACCESSTOKENHERE' \
--header 'Accept: application/vnd.github.v3.raw' \
--remote-name \
--location https://api.github.com/repos/owner/repo/contents/path
# Example...
TOKEN="INSERTACCESSTOKENHERE"
OWNER="BBC-News"
REPO="responsive-news"
PATH="scripts/build/tabloid.sh"
FILE="https://api.github.com/repos/$OWNER/$REPO/contents/$PATH"
curl --header 'Authorization: token $TOKEN' \
--header 'Accept: application/vnd.github.v3.raw' \
--remote-name \
--location $FILE
  • -H --header Extra header to use when getting a web page. You may specify any number of extra headers

  • -O --remote-name Write output to a local file named like the remote file we get (only the file part of the remote file is used, the path is cut off)

  • -L --location If the server reports that the requested page has moved to a different location (indicated with a Location: header and a 3XX response code), this option will make curl redo the request on the new place

@rem7

This comment has been minimized.

Copy link

commented Feb 11, 2015

Hey! This is pretty cool. Do you know if there is a way to download a file from a specific branch?

@rem7

This comment has been minimized.

Copy link

commented Feb 11, 2015

nevermind, just found the docs https://developer.github.com/v3/repos/contents/
Just have to pass the ref parameter for the branch.

@Vrakfall

This comment has been minimized.

Copy link

commented Apr 7, 2015

You should use another name for the PATH variable because it would obviously overwrite the system PATH variable and even make the curl command not to be found.

@antonbormotov

This comment has been minimized.

Copy link

commented Sep 17, 2015

Instead of single quote marks,
line 14: curl --header 'Authorization: token $TOKEN' \
should be double quote marks in order shell to substitute variable $TOKEN.

@ohadperry

This comment has been minimized.

Copy link

commented Oct 27, 2015

👍

@Kif11

This comment has been minimized.

Copy link

commented Dec 11, 2015

Is there a way to do the same thing without authorization?

@hayesmaker

This comment has been minimized.

Copy link

commented Mar 8, 2016

this only works on files <1mb unfortunately.. but if that's not an issue for you it's useful

@jmcshane

This comment has been minimized.

Copy link

commented Jun 14, 2016

Just as a note, the remote-name parameter can append the ref parameter if present. I haven't found a good workaround to this

@brikis98

This comment has been minimized.

Copy link

commented Jun 20, 2016

Great script!

If you download files from private GitHub repos often, you can also check out fetch, an open source, cross-platform tool which makes it easy to download source files and release assets from a git tag, commit, or branch of public and private GitHub repos.

For example, to download the file baz from version 0.1.3 of a private GitHub repo to /tmp, you would do the following:

GITHUB_OAUTH_TOKEN="your token"
fetch --repo="https://github.com/foo/bar" --tag="0.1.3" --source-path="/baz" /tmp
@haemi

This comment has been minimized.

Copy link

commented Jul 20, 2016

does this also work for github enterprise?

@girishkg

This comment has been minimized.

Copy link

commented Aug 8, 2016

fetch doesn't work for the URL with subdomain like "github.example.com"
gruntwork-io/fetch#14

@mdtareque

This comment has been minimized.

Copy link

commented Aug 16, 2016

Any idea why I am not able to run this? I am getting below error

$ ./getGithubFile.sh
curl: (77) error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
./getGithubFile.sh: line 16: curl: command not found
@zevarito

This comment has been minimized.

Copy link

commented Aug 18, 2016

@mdtareque did you override your PATH env variable perhaps?

@chusiang

This comment has been minimized.

Copy link

commented Oct 17, 2016

Hi, @mdtareque:

The PATH="scripts/build/tabloid.sh" will override system $PATH, maype you can rename it.

@neiledelsten

This comment has been minimized.

Copy link

commented Feb 6, 2017

Ref rem7's comment about the ref parameter for downloading from a specific branch, I'm struggling to work out how to integrate this into the curl.sh shell script. Can anyone provide an example?

@brooksa321

This comment has been minimized.

Copy link

commented Apr 10, 2017

@mkarthik415

This comment has been minimized.

Copy link

commented Jun 4, 2017

I am getting below message when trying to access a private repo.

curl -H 'Authorization: token TOKEN' -H 'Accept: application/vnd.github.v3.raw' -O \ -L https://api.github.com/mkarthik415/repo/blob/master/AppManager/AppManager/maven-metadata-local.xml
curl: Remote file name has no length!
curl: try 'curl --help' or 'curl --manual' for more information
{
"message": "Not Found",
"documentation_url": "https://developer.github.com/v3"
}

@vsmori

This comment has been minimized.

Copy link

commented Aug 9, 2017

Just one comment: rename PATH to another variable name, since PATH is a special variable in Linux Console. You may get "command not found" after setting PATH used in the example.

@tianqig

This comment has been minimized.

Copy link

commented Aug 23, 2017

I would like to download a file from a private repo which I am a member, is it possible I can use the download link to download the file from a browser? I tried it without an error message, but failed to download the file.

BTW, I already logged into my Github account from the browser.

Thanks

@vambo

This comment has been minimized.

Copy link

commented Oct 25, 2017

+1 to renaming PATH

@snrk0

This comment has been minimized.

Copy link

commented Nov 5, 2017

Is there a way that doesn't utilize the GitHub API?
I usually download a file via raw.githubusercontent.com but that doesn't seem to be available for private repositories.

@CodeMan99

This comment has been minimized.

Copy link

commented Feb 1, 2018

Github enforces usage of a valid User-Agent in the request headers.

If you are making the same request multiple times, include a If-Modified-Since header and respect the 304 Not Modified response (using modification time via --time-cond).

curl --header "Authorization: token $TOKEN" \
     --header "Accept: application/vnd.github.v3.raw" \
     --header "User-Agent: ${OWNER}/${REPO} (curl v7.47.0)" \
     --time-cond "$CACHE" \
     --remote-name \
     --location "$FILE"
@suraj19

This comment has been minimized.

Copy link

commented Sep 10, 2018

How can I download the entire repository from GitHub any idea, please??

@VincentCATILLON

This comment has been minimized.

Copy link

commented Sep 11, 2018

Using fetch ($TOKEN -> TOKEN, $FILE -> FILE):

fetch('https://api.github.com/repos/owner/repo/contents/path', {
  headers: {
    'Authorization': 'token ' + TOKEN,
    'Accept': 'application/vnd.github.v3.raw',
  },
})
  .then(response => response.buffer())
  .then(content => {
    console.log('[FILE]', 'Writing ' + FILE);
    fs.writeFileSync(FILE, content);
  })
  .catch(() => {
    throw new Error('File unreachable');
  });
@vnermolaev

This comment has been minimized.

Copy link

commented Oct 3, 2018

+1 Rename $PATH

@oganm

This comment has been minimized.

Copy link

commented Oct 12, 2018

Note that this endpoint is limited to 1 MB files. any larger will fail. working on a workaround

@oganm

This comment has been minimized.

Copy link

commented Oct 16, 2018

I have found a workaround for large files (up to 100 MB but as far as I know it is not possible to drop larger files to github so this should work with all files)

the contents endpoint used above cannot get the file if larger than 1 MB so trim the $PATH to direct to the folder where your file lies in the initial request using the contents endpoint

In the response, find your file listed, get its SHA,

use the /repos/:username/:reponame/git/blobs/:sha endpoint to get your content in 64 bit encoded form. Decode it using your favorite decoder.

For reference, see the R code below

getGithubFile = function(githubPath,branch = 'master', downloadPath = NULL,token = NULL){
    if(is.null(downloadPath)){
        downloadPath = tempfile()
    }
    
    path = strsplit(githubPath,'/')[[1]]
    file = paste(path[3:length(path)], collapse = '/')
    contents = gh::gh('GET /repos/:username/:reponame/contents/:dir?ref=:branch',
                  username = path[1],
                  reponame = path[2],
                  branch = branch,
                  dir = dirname(file),
                  .token = token)
    
    names(contents) = contents %>% purrr::map_chr('name')
    
    fileInfo = contents[contents %>% purrr::map_chr('name') %>% {.%in%basename(file)}][[1]]
    
    blob = gh::gh('GET /repos/:username/:reponame/git/blobs/:sha',
                  username = path[1],
                  reponame = path[2],
                  sha = fileInfo$sha,
                  .token = token)
    
    decodeContent = openssl::base64_decode(blob$content)
    writeBin(decodeContent,downloadPath)
    
    return(invisible(downloadPath))
}
@interaminense

This comment has been minimized.

Copy link

commented Nov 3, 2018

Using API V4 with Graphql

query {
   repository(name: "repo-name", owner: "owner") {
      object(expression: "branch:file/to/path.extension") {
         ... on Blob {
            text
         }
      }
   }
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.