Instantly share code, notes, and snippets.

@caspyin /curl.md
Last active Dec 12, 2018

Embed
What would you like to do?
A curl tutorial using GitHub's API

Introduction

An introduction to curl using GitHub's API.

The Basics

Makes a basic GET request to the specifed URI

curl https://api.github.com/users/caspyin

Includes HTTP-Header information in the output

curl --include https://api.github.com/users/caspyin

Pass user credential to basic auth to access protected resources like a users starred gists, or private info associated with their profile

curl --user "caspyin:PASSWD" https://api.github.com/gists/starred
curl --user "caspyin:PASSWD" https://api.github.com/users/caspyin

Passing just the username without the colon (:) will cause you to be prompted for your account password. This avoids having your password in your command line history

curl --user "caspyin" https://api.github.com/users/caspyin

POST

Use the --request (-X) flag along with --data (-d) to POST data

curl --user "caspyin" --request POST --data '{"description":"Created via API","public":"true","files":{"file1.txt":{"content":"Demo"}}' https://api.github.com/gists

curl --user "caspyin" -X POST --data '{"description":"Created via API","public":"true","files":{"file1.txt":{"content":"Demo"}}' https://api.github.com/gists

Of course --data implies POST so you don't have to also specify the --request flag

curl --user "caspyin" --data '{"description":"Created via API","public":"true","files":{"file1.txt":{"content":"Demo"}}' https://api.github.com/gists

Here is an example that uses the old GitHub API (v2). You can use multiple --data flags

curl --data "login=caspyin" --data "token=TOKEN" https://github.com/api/v2/json/user/show/caspyin

The post data gets combined into one so you can also just combine them yourself into a single --data flag

curl --data "login=caspyin&token=TOKEN" https://github.com/api/v2/json/user/show/caspyin

You can tell curl to read from a file (@) to POST data

curl --user "caspyin" --data @data.txt https://api.github.com/gists 

Or it can read from STDIN (@-)

curl --user "caspyin" --data @- https://api.github.com/gists
{
  "description":"Test",
  "public":false,
  "files": {
    "file1.txt": {
      "content":"Demo"
    }
  }
}
end with ctrl+d

Headers

Often when POSTing data you'll need to add headers for things like auth tokens or setting the content type. You can set a header using -H.

curl -H "Content-Type: application/json" -H "authToken: 349ab29a-xtab-423b-a5hc-5623bc39b8c8" --data '{}' https://api.example.com/endpoint

Dealing with HTTPS

If an API doens't have an SSL cert but is using HTTPS you can tell curl to ignore the security by using --insecure. Be warned this is a very "insecure" thing to do and is only listed here for "educational purposes".

curl --insecure https://api.example.com/endpoint

For my own reference mostly, here is where I first learned about using --insecure https://github.com/wayneeseguin/rvm/issues/1684

OAuth

The first thing to know is that your API Token (found in https://github.com/settings/admin) is not the same token used by OAuth. They are different tokens and you will need to generate an OAuth token to be authorized.

Follow the API's instructions at http://developer.github.com/v3/oauth/ under the sections "Non-Web Application Flow" and "Create a new authorization" to become authorized.

Note: Use Basic Auth once to create an OAuth2 token http://developer.github.com/v3/oauth/#oauth-authorizations-api

curl https://api.github.com/authorizations \
--user "caspyin" \
--data '{"scopes":["gist"],"note":"Demo"}'

This will prompt you for your GitHub password and return your OAuth token in the response. It will also create a new Authorized application in your account settings https://github.com/settings/applications

Now that you have the OAuth token there are two ways to use the token to make requests that require authentication (replace "OAUTH-TOKEN" with your actual token)

curl https://api.github.com/gists/starred?access_token=OAUTH-TOKEN
curl -H "Authorization: token OAUTH-TOKEN" https://api.github.com/gists/starred

List the authorizations you already have

curl --user "caspyin" https://api.github.com/authorizations

Resources

@jeremyschulman

This comment has been minimized.

jeremyschulman commented Jan 20, 2013

Thank you for this post. Is it possible to send the gist content as encoded-64? I've got a file with lots of characters that seem to cause json parse issues when I attempt to post the data. I can encode64 the string; just don't know how to tell github that it's encoded ... I just see the encoded string show up in my gist.

I appreciate any help & guidance you can offer!

@caspyin

This comment has been minimized.

Owner

caspyin commented Jan 31, 2013

@jeremyschulman Not sure. It looks like encoding the string should work (http://stackoverflow.com/q/2122105/1139787). Can you give me an example of what you're trying?

@KrzysztofCiba

This comment has been minimized.

KrzysztofCiba commented Apr 6, 2013

Nice. Thanks for the info.

@atif93

This comment has been minimized.

atif93 commented May 27, 2013

I want my followers to stop receiving notifications. For that I read somewhere that the command is
DELETE /followers/:id
and I should be logged in and authenticated with basic auth or OAuth with the user:follow scope.
Can you please tell me how to do this.

@luisibanez

This comment has been minimized.

luisibanez commented Sep 29, 2013

Very nice, clean and clear.

Thanks for sharing !

@caspyin

This comment has been minimized.

Owner

caspyin commented Oct 1, 2013

Added info on using -H (headers) and the --insecure options

@victorluissantos

This comment has been minimized.

victorluissantos commented Jun 23, 2014

Thamks

@carlmart

This comment has been minimized.

carlmart commented Dec 15, 2014

Can you create a new repository? tried the following. , saw one that did create a new repo
curl -u "$carlmart :$ch4yrj685432eecb289484848483736363636363" https://api.github.com/user/repos -d '{"CarlMart":"'$test1'"}

@JaredCowan

This comment has been minimized.

JaredCowan commented Dec 21, 2014

@carlmart
You have two easy ways to create a repo via terminal

This first one you just need to put in your username between the single quotes and then the name of the repo you want in-between the double quotes. This will then prompt you to type in your password. P.S. You wont see your password as your type. It's hidden for security.

curl -u 'YOUR-USER-NAME' https://api.github.com/user/repos -d '{"name": "PUT-YOUR-REPO-NAME-HERE"}'

Option two is the best (IMO) Just go to your account and create an authtoken and paste it where it says below. Then just give your repo a name. This will not ask your for a password.

curl -H "Authorization: token PUT-YOUR-AUTH-TOKEN-HERE" https://api.github.com/user/repos -d '{"name": "PUT-YOUR-REPO-NAME-HERE"}'

@jfraboni

This comment has been minimized.

jfraboni commented Apr 15, 2015

Thanks, great info, plain and simple!

@i4h

This comment has been minimized.

i4h commented May 5, 2015

Thank you for the helpful and clear tutorial!
I have been trying to write a curl request to send an .md-file from my computer to the github markdown api, but i couldn't figure out how to send data (with unescaped quotes) from a file in a named jason variable. I am using httpie now, but that solution bugs me because it is not as available. Could you maybe add this use case?

@kenorb

This comment has been minimized.

@rohansharma1992

This comment has been minimized.

rohansharma1992 commented Aug 31, 2015

Can i retrieve the number of open issues from a public repository using this?

@becky7

This comment has been minimized.

becky7 commented Dec 5, 2015

HI IS BECKY

@TylerNakamura

This comment has been minimized.

TylerNakamura commented Jan 27, 2016

Great, thanks for the information.

@saintplay

This comment has been minimized.

saintplay commented Feb 12, 2016

Pretty cool. Hurl.it rocks

@Alex1990

This comment has been minimized.

Alex1990 commented Feb 14, 2016

In POST section, a close curly is missing in the first example.

@carc1n0gen

This comment has been minimized.

carc1n0gen commented May 26, 2016

Hello I know this is old, but I'm having issues sending the content of a file that has one or more new lines.

I get:

{
  "message": "Problems parsing JSON",
  "documentation_url": "https://developer.github.com/enterprise/2.4/v3"
}

But if the file content I send has a single line it works perfectly fine

@arthas353

This comment has been minimized.

arthas353 commented Jul 7, 2016

Hello,
I want to update a specific file in my git master repo; How do I do that using curl ?

@zh-wowtv

This comment has been minimized.

zh-wowtv commented Jul 14, 2016

Is it possible to upload image file through API?

@konradkonrad

This comment has been minimized.

konradkonrad commented Nov 30, 2016

Not sure if this is the right place, but: have you ever figured out a way to trigger obtaining an OTP (e.g. via sms) by cURL against the github API?

curl -u <myusername> https://github.com/user

reponds with

{
  "message": "Must specify two-factor authentication OTP code.",
  "documentation_url": "https://developer.github.com/v3/auth#working-with-two-factor-authentication"
}

however, it doesn't trigger the creation of a OTP. Signing in through the website does send me an sms, so there doesn't seem to be a setup problem.

@larshp

This comment has been minimized.

larshp commented Feb 5, 2017

had the same issue as @konradkonrad describes above, following curl generates a SMS code when I try,

curl -i -u your_username -d '{"scopes": ["repo", "user"], "note": "getting-started"}' https://api.github.com/authorizations

@caspyin

This comment has been minimized.

Owner

caspyin commented Feb 23, 2017

It seems this has been useful over the years to various people. If you would like to contribute any changes to this gist, please fork and then ping me here in the comments to review.

@pwsouth

This comment has been minimized.

@suntong

This comment has been minimized.

suntong commented Aug 14, 2017

@caspyin, thanks for the great post. How can you send base64 encoded content?

I tried,

curl -H 'Accept: application/vnd.github.VERSION.base64' --data @data.txt https://api.github.com/gists

But the content is still base64 encoded:
https://gist.github.com/anonymous/c79cbfcf466af02eb08dc9ad6f26b587

Here is my test data.txt:

{
  "description": "the description for this gist",
  "public": true,
  "files": {
    "file1.txt": {
      "content": "aG9zdEBtYWlsLmNvbTpwYXNzd29yZAo="
    }
  }
}

Thx.

@suntong

This comment has been minimized.

suntong commented Aug 14, 2017

I figured it out -- base64 encoding is only supported for download, not upload.

So to post any arbitrary string or file, check out

jsonfiddle

Usage
https://github.com/go-jsonfile/jsonfiddle#usage-1

jsonfiddle esc will escape any arbitrary string so as to embed it as content of json string. This seems useless at first, but it actually allows you to embed any arbitrary file into GitHub Gists JSON API, so as to post any arbitrary file onto GitHub Gist:

echo '{"description":"SmartyStreets API Demo","public":true,"files":{"SmartyStreets.json":{"content":'"`jsonfiddle fmt -i test/SmartyStreets.json | jsonfiddle esc -i`"'}}}' | curl --data @- https://api.github.com/gists

This will give you
https://gist.github.com/anonymous/1423d4768dd9b88262ca513626e68d8e

By "arbitrary file" I do mean arbitrary file. Check this out:
https://gist.github.com/anonymous/a51798ce99ff59d8d4ba536cbf4b6996

This is why jsonfiddle esc is a command on its own, instead of being part of functionalities of jsonfiddle fmt or jsonfiddle sort.

@aman0446

This comment has been minimized.

aman0446 commented Aug 22, 2017

I'm trying to access my company's github repository(github enterprise). command i'm using is
curl --user "name:pwd" "https://github."company name".com/{owner}/repo" , but i'm getting "Not found error" .
When i login through the browser with the same credentials, it is going through.
i was reading through " OAuth and PAT(Personal Access tokens). Couldn't get to working. Can someone walk me through the steps if you have already done it?

@gmilos

This comment has been minimized.

gmilos commented Sep 1, 2017

@aman0446 you need something like this:

curl -H "Authorization: token PAT-TOKEN" https://github.COMPANY_NAME.com/api/v3/users/USER/repos
@rpbaptist

This comment has been minimized.

rpbaptist commented Dec 12, 2017

For OTP (2FA) you will have to send this header along:

curl -H "X-GitHub-OTP: 123456"
@ttpro1995

This comment has been minimized.

ttpro1995 commented Jun 12, 2018

In

curl --user "ttpro1995:MY_OWN_PASSWORD" --data '{"description":"Created via API","public":"false","files":{"file1.txt":{"content":"Demo"}}' https://api.github.com/gists
How can I replace password with something else (API key )?

@testqazqwe

This comment has been minimized.

testqazqwe commented Oct 31, 2018

create a github repo webhook with curl:

curl -v --header "Authorization: token 123_generated_token" -X POST -d '{"active": true, "events": ["push"], "config": {"url": "https://service_url", "content_type": "json"}}' https://api.github.com/repos/OWNER/REPONAME/hooks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment