Skip to content

Instantly share code, notes, and snippets.

@Integralist
Forked from madrobby/gist:9476733
Last active September 7, 2023 03:53
Show Gist options
  • Save Integralist/9482061 to your computer and use it in GitHub Desktop.
Save Integralist/9482061 to your computer and use it in GitHub Desktop.
Download a single file from a private GitHub repo. You'll need an access token as described in this GitHub Help article: https://help.github.com/articles/creating-an-access-token-for-command-line-use
curl --header 'Authorization: token INSERTACCESSTOKENHERE' \
--header 'Accept: application/vnd.github.v3.raw' \
--remote-name \
--location https://api.github.com/repos/owner/repo/contents/path
# Example...
TOKEN="INSERTACCESSTOKENHERE"
OWNER="BBC-News"
REPO="responsive-news"
PATH="scripts/build/tabloid.sh"
FILE="https://api.github.com/repos/$OWNER/$REPO/contents/$PATH"
curl --header 'Authorization: token $TOKEN' \
--header 'Accept: application/vnd.github.v3.raw' \
--remote-name \
--location $FILE
  • -H --header Extra header to use when getting a web page. You may specify any number of extra headers

  • -O --remote-name Write output to a local file named like the remote file we get (only the file part of the remote file is used, the path is cut off)

  • -L --location If the server reports that the requested page has moved to a different location (indicated with a Location: header and a 3XX response code), this option will make curl redo the request on the new place

@brooksa321
Copy link

brooksa321 commented Apr 10, 2017

@mkarthik415
Copy link

I am getting below message when trying to access a private repo.

curl -H 'Authorization: token TOKEN' -H 'Accept: application/vnd.github.v3.raw' -O \ -L https://api.github.com/mkarthik415/repo/blob/master/AppManager/AppManager/maven-metadata-local.xml
curl: Remote file name has no length!
curl: try 'curl --help' or 'curl --manual' for more information
{
"message": "Not Found",
"documentation_url": "https://developer.github.com/v3"
}

@vsmori
Copy link

vsmori commented Aug 9, 2017

Just one comment: rename PATH to another variable name, since PATH is a special variable in Linux Console. You may get "command not found" after setting PATH used in the example.

@tianqig
Copy link

tianqig commented Aug 23, 2017

I would like to download a file from a private repo which I am a member, is it possible I can use the download link to download the file from a browser? I tried it without an error message, but failed to download the file.

BTW, I already logged into my Github account from the browser.

Thanks

@vambo
Copy link

vambo commented Oct 25, 2017

+1 to renaming PATH

@jonasberge
Copy link

Is there a way that doesn't utilize the GitHub API?
I usually download a file via raw.githubusercontent.com but that doesn't seem to be available for private repositories.

@CodeMan99
Copy link

CodeMan99 commented Feb 1, 2018

Github enforces usage of a valid User-Agent in the request headers.

If you are making the same request multiple times, include a If-Modified-Since header and respect the 304 Not Modified response (using modification time via --time-cond).

curl --header "Authorization: token $TOKEN" \
     --header "Accept: application/vnd.github.v3.raw" \
     --header "User-Agent: ${OWNER}/${REPO} (curl v7.47.0)" \
     --time-cond "$CACHE" \
     --remote-name \
     --location "$FILE"

@suraj19
Copy link

suraj19 commented Sep 10, 2018

How can I download the entire repository from GitHub any idea, please??

@VincentCATILLON
Copy link

VincentCATILLON commented Sep 11, 2018

Using fetch ($TOKEN -> TOKEN, $FILE -> FILE):

fetch('https://api.github.com/repos/owner/repo/contents/path', {
  headers: {
    'Authorization': 'token ' + TOKEN,
    'Accept': 'application/vnd.github.v3.raw',
  },
})
  .then(response => response.buffer())
  .then(content => {
    console.log('[FILE]', 'Writing ' + FILE);
    fs.writeFileSync(FILE, content);
  })
  .catch(() => {
    throw new Error('File unreachable');
  });

@vnermolaev
Copy link

+1 Rename $PATH

@oganm
Copy link

oganm commented Oct 12, 2018

Note that this endpoint is limited to 1 MB files. any larger will fail. working on a workaround

@oganm
Copy link

oganm commented Oct 16, 2018

I have found a workaround for large files (up to 100 MB but as far as I know it is not possible to drop larger files to github so this should work with all files)

the contents endpoint used above cannot get the file if larger than 1 MB so trim the $PATH to direct to the folder where your file lies in the initial request using the contents endpoint

In the response, find your file listed, get its SHA,

use the /repos/:username/:reponame/git/blobs/:sha endpoint to get your content in 64 bit encoded form. Decode it using your favorite decoder.

For reference, see the R code below

getGithubFile = function(githubPath,branch = 'master', downloadPath = NULL,token = NULL){
    if(is.null(downloadPath)){
        downloadPath = tempfile()
    }
    
    path = strsplit(githubPath,'/')[[1]]
    file = paste(path[3:length(path)], collapse = '/')
    contents = gh::gh('GET /repos/:username/:reponame/contents/:dir?ref=:branch',
                  username = path[1],
                  reponame = path[2],
                  branch = branch,
                  dir = dirname(file),
                  .token = token)
    
    names(contents) = contents %>% purrr::map_chr('name')
    
    fileInfo = contents[contents %>% purrr::map_chr('name') %>% {.%in%basename(file)}][[1]]
    
    blob = gh::gh('GET /repos/:username/:reponame/git/blobs/:sha',
                  username = path[1],
                  reponame = path[2],
                  sha = fileInfo$sha,
                  .token = token)
    
    decodeContent = openssl::base64_decode(blob$content)
    writeBin(decodeContent,downloadPath)
    
    return(invisible(downloadPath))
}

@interaminense
Copy link

interaminense commented Nov 3, 2018

Using API V4 with Graphql

query {
   repository(name: "repo-name", owner: "owner") {
      object(expression: "branch:file/to/path.extension") {
         ... on Blob {
            text
         }
      }
   }
}

@eliranmal
Copy link

@Kif11 - it's possible with basic auth (without the user having to explicitly create a token first).

it can be boiled down to a one-liner (see the original gist for details):

curl -u "$USER" -k https://${GHE_DOMAIN}/raw/${REPO_OWNER}/${REPO_NAME}/${REF}/${FILE} > ${FILE}

@jean
Copy link

jean commented Nov 27, 2019

I don't know if things changed since May, but neither token nor basic auth works for me. I'm getting

{
  "message": "Not Found",
  "documentation_url": "https://developer.github.com/v3"
}

@almajoy
Copy link

almajoy commented Dec 6, 2019

curl --header 'Authorization: token INSERTACCESSTOKENHERE'
--header 'Accept: application/vnd.github.v3.raw'
--remote-name
--location https://api.github.com/repos/owner/repo/contents/path

Hey,

it code can work using the is oauth app token any idea please share me?

@zelfick
Copy link

zelfick commented May 24, 2020

use the solution in the first part of the article it works without problem

@frederico-klein
Copy link

frederico-klein commented Jan 24, 2021

I don't know if things changed since May, but neither token nor basic auth works for me. I'm getting

{
  "message": "Not Found",
  "documentation_url": "https://developer.github.com/v3"
}

I thought this was no longer working, however what I did wrong is not adding the correct permissions to the token I was generating.

I marked

"repo

Full control of private repositories "

to get it working. There is maybe a less permissive alternative that still allows you to just read.

@mrmattipants
Copy link

mrmattipants commented May 2, 2021

Fashionably Late, as always!

Originally, I was having trouble with the Token, itself.
However, after creating a New Token, with Everything Checked, my issues seemed to Clear-up.

From Command Line, the following worked, on my end.

curl -v -H "Authorization: token ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" -H "Accept: application/vnd.github.v4.raw" -O -L "https://api.github.com/repos/<UserName>/<RepoName>/contents/<FolderPath>/<FileName>.ps1"

With that being said, I would imagine that something like the following, should do the trick.

curl --header 'Authorization: token ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
--header 'Accept: application/vnd.github.v3.raw'
--location 'https://api.github.com/repos/<UserName>/<RepoName>/contents/<FolderPath>/<FileName>.ps1'

@renzedj
Copy link

renzedj commented Sep 9, 2022

What are the minimum permissions for a Personal Access required to do this (specifically for Enterprise, if there's a difference).

Thx.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment