Create a gist now

Instantly share code, notes, and snippets.

@caspyin /curl.md
Last active Jun 22, 2018

Embed
A curl tutorial using GitHub's API

Introduction

An introduction to curl using GitHub's API.

The Basics

Makes a basic GET request to the specifed URI

curl https://api.github.com/users/caspyin

Includes HTTP-Header information in the output

curl --include https://api.github.com/users/caspyin

Pass user credential to basic auth to access protected resources like a users starred gists, or private info associated with their profile

curl --user "caspyin:PASSWD" https://api.github.com/gists/starred
curl --user "caspyin:PASSWD" https://api.github.com/users/caspyin

Passing just the username without the colon (:) will cause you to be prompted for your account password. This avoids having your password in your command line history

curl --user "caspyin" https://api.github.com/users/caspyin

POST

Use the --request (-X) flag along with --data (-d) to POST data

curl --user "caspyin" --request POST --data '{"description":"Created via API","public":"true","files":{"file1.txt":{"content":"Demo"}}' https://api.github.com/gists

curl --user "caspyin" -X POST --data '{"description":"Created via API","public":"true","files":{"file1.txt":{"content":"Demo"}}' https://api.github.com/gists

Of course --data implies POST so you don't have to also specify the --request flag

curl --user "caspyin" --data '{"description":"Created via API","public":"true","files":{"file1.txt":{"content":"Demo"}}' https://api.github.com/gists

Here is an example that uses the old GitHub API (v2). You can use multiple --data flags

curl --data "login=caspyin" --data "token=TOKEN" https://github.com/api/v2/json/user/show/caspyin

The post data gets combined into one so you can also just combine them yourself into a single --data flag

curl --data "login=caspyin&token=TOKEN" https://github.com/api/v2/json/user/show/caspyin

You can tell curl to read from a file (@) to POST data

curl --user "caspyin" --data @data.txt https://api.github.com/gists 

Or it can read from STDIN (@-)

curl --user "caspyin" --data @- https://api.github.com/gists
{
  "description":"Test",
  "public":false,
  "files": {
    "file1.txt": {
      "content":"Demo"
    }
  }
}
end with ctrl+d

Headers

Often when POSTing data you'll need to add headers for things like auth tokens or setting the content type. You can set a header using -H.

curl -H "Content-Type: application/json" -H "authToken: 349ab29a-xtab-423b-a5hc-5623bc39b8c8" --data '{}' https://api.example.com/endpoint

Dealing with HTTPS

If an API doens't have an SSL cert but is using HTTPS you can tell curl to ignore the security by using --insecure. Be warned this is a very "insecure" thing to do and is only listed here for "educational purposes".

curl --insecure https://api.example.com/endpoint

For my own reference mostly, here is where I first learned about using --insecure https://github.com/wayneeseguin/rvm/issues/1684

OAuth

The first thing to know is that your API Token (found in https://github.com/settings/admin) is not the same token used by OAuth. They are different tokens and you will need to generate an OAuth token to be authorized.

Follow the API's instructions at http://developer.github.com/v3/oauth/ under the sections "Non-Web Application Flow" and "Create a new authorization" to become authorized.

Note: Use Basic Auth once to create an OAuth2 token http://developer.github.com/v3/oauth/#oauth-authorizations-api

curl https://api.github.com/authorizations \
--user "caspyin" \
--data '{"scopes":["gist"],"note":"Demo"}'

This will prompt you for your GitHub password and return your OAuth token in the response. It will also create a new Authorized application in your account settings https://github.com/settings/applications

Now that you have the OAuth token there are two ways to use the token to make requests that require authentication (replace "OAUTH-TOKEN" with your actual token)

curl https://api.github.com/gists/starred?access_token=OAUTH-TOKEN
curl -H "Authorization: token OAUTH-TOKEN" https://api.github.com/gists/starred

List the authorizations you already have

curl --user "caspyin" https://api.github.com/authorizations

Resources

@jeremyschulman

This comment has been minimized.

Show comment
Hide comment
@jeremyschulman

jeremyschulman Jan 20, 2013

Thank you for this post. Is it possible to send the gist content as encoded-64? I've got a file with lots of characters that seem to cause json parse issues when I attempt to post the data. I can encode64 the string; just don't know how to tell github that it's encoded ... I just see the encoded string show up in my gist.

I appreciate any help & guidance you can offer!

Thank you for this post. Is it possible to send the gist content as encoded-64? I've got a file with lots of characters that seem to cause json parse issues when I attempt to post the data. I can encode64 the string; just don't know how to tell github that it's encoded ... I just see the encoded string show up in my gist.

I appreciate any help & guidance you can offer!

@caspyin

This comment has been minimized.

Show comment
Hide comment
@caspyin

caspyin Jan 31, 2013

@jeremyschulman Not sure. It looks like encoding the string should work (http://stackoverflow.com/q/2122105/1139787). Can you give me an example of what you're trying?

Owner

caspyin commented Jan 31, 2013

@jeremyschulman Not sure. It looks like encoding the string should work (http://stackoverflow.com/q/2122105/1139787). Can you give me an example of what you're trying?

@KrzysztofCiba

This comment has been minimized.

Show comment
Hide comment
@KrzysztofCiba

KrzysztofCiba Apr 6, 2013

Nice. Thanks for the info.

Nice. Thanks for the info.

@atif93

This comment has been minimized.

Show comment
Hide comment
@atif93

atif93 May 27, 2013

I want my followers to stop receiving notifications. For that I read somewhere that the command is
DELETE /followers/:id
and I should be logged in and authenticated with basic auth or OAuth with the user:follow scope.
Can you please tell me how to do this.

atif93 commented May 27, 2013

I want my followers to stop receiving notifications. For that I read somewhere that the command is
DELETE /followers/:id
and I should be logged in and authenticated with basic auth or OAuth with the user:follow scope.
Can you please tell me how to do this.

@luisibanez

This comment has been minimized.

Show comment
Hide comment
@luisibanez

luisibanez Sep 29, 2013

Very nice, clean and clear.

Thanks for sharing !

Very nice, clean and clear.

Thanks for sharing !

@caspyin

This comment has been minimized.

Show comment
Hide comment
@caspyin

caspyin Oct 1, 2013

Added info on using -H (headers) and the --insecure options

Owner

caspyin commented Oct 1, 2013

Added info on using -H (headers) and the --insecure options

@victorluissantos

This comment has been minimized.

Show comment
Hide comment

Thamks

@carlmart

This comment has been minimized.

Show comment
Hide comment
@carlmart

carlmart Dec 15, 2014

Can you create a new repository? tried the following. , saw one that did create a new repo
curl -u "$carlmart :$ch4yrj685432eecb289484848483736363636363" https://api.github.com/user/repos -d '{"CarlMart":"'$test1'"}

Can you create a new repository? tried the following. , saw one that did create a new repo
curl -u "$carlmart :$ch4yrj685432eecb289484848483736363636363" https://api.github.com/user/repos -d '{"CarlMart":"'$test1'"}

@JaredCowan

This comment has been minimized.

Show comment
Hide comment
@JaredCowan

JaredCowan Dec 21, 2014

@carlmart
You have two easy ways to create a repo via terminal

This first one you just need to put in your username between the single quotes and then the name of the repo you want in-between the double quotes. This will then prompt you to type in your password. P.S. You wont see your password as your type. It's hidden for security.

curl -u 'YOUR-USER-NAME' https://api.github.com/user/repos -d '{"name": "PUT-YOUR-REPO-NAME-HERE"}'

Option two is the best (IMO) Just go to your account and create an authtoken and paste it where it says below. Then just give your repo a name. This will not ask your for a password.

curl -H "Authorization: token PUT-YOUR-AUTH-TOKEN-HERE" https://api.github.com/user/repos -d '{"name": "PUT-YOUR-REPO-NAME-HERE"}'

@carlmart
You have two easy ways to create a repo via terminal

This first one you just need to put in your username between the single quotes and then the name of the repo you want in-between the double quotes. This will then prompt you to type in your password. P.S. You wont see your password as your type. It's hidden for security.

curl -u 'YOUR-USER-NAME' https://api.github.com/user/repos -d '{"name": "PUT-YOUR-REPO-NAME-HERE"}'

Option two is the best (IMO) Just go to your account and create an authtoken and paste it where it says below. Then just give your repo a name. This will not ask your for a password.

curl -H "Authorization: token PUT-YOUR-AUTH-TOKEN-HERE" https://api.github.com/user/repos -d '{"name": "PUT-YOUR-REPO-NAME-HERE"}'

@jfraboni

This comment has been minimized.

Show comment
Hide comment
@jfraboni

jfraboni Apr 15, 2015

Thanks, great info, plain and simple!

Thanks, great info, plain and simple!

@i4h

This comment has been minimized.

Show comment
Hide comment
@i4h

i4h May 5, 2015

Thank you for the helpful and clear tutorial!
I have been trying to write a curl request to send an .md-file from my computer to the github markdown api, but i couldn't figure out how to send data (with unescaped quotes) from a file in a named jason variable. I am using httpie now, but that solution bugs me because it is not as available. Could you maybe add this use case?

i4h commented May 5, 2015

Thank you for the helpful and clear tutorial!
I have been trying to write a curl request to send an .md-file from my computer to the github markdown api, but i couldn't figure out how to send data (with unescaped quotes) from a file in a named jason variable. I am using httpie now, but that solution bugs me because it is not as available. Could you maybe add this use case?

@rohansharma1992

This comment has been minimized.

Show comment
Hide comment
@rohansharma1992

rohansharma1992 Aug 31, 2015

Can i retrieve the number of open issues from a public repository using this?

Can i retrieve the number of open issues from a public repository using this?

@becky7

This comment has been minimized.

Show comment
Hide comment
@becky7

becky7 Dec 5, 2015

HI IS BECKY

becky7 commented Dec 5, 2015

HI IS BECKY

@TylerNakamura

This comment has been minimized.

Show comment
Hide comment
@TylerNakamura

TylerNakamura Jan 27, 2016

Great, thanks for the information.

Great, thanks for the information.

@saintplay

This comment has been minimized.

Show comment
Hide comment
@saintplay

saintplay Feb 12, 2016

Pretty cool. Hurl.it rocks

Pretty cool. Hurl.it rocks

@Alex1990

This comment has been minimized.

Show comment
Hide comment
@Alex1990

Alex1990 Feb 14, 2016

In POST section, a close curly is missing in the first example.

In POST section, a close curly is missing in the first example.

@carc1n0gen

This comment has been minimized.

Show comment
Hide comment
@carc1n0gen

carc1n0gen May 26, 2016

Hello I know this is old, but I'm having issues sending the content of a file that has one or more new lines.

I get:

{
  "message": "Problems parsing JSON",
  "documentation_url": "https://developer.github.com/enterprise/2.4/v3"
}

But if the file content I send has a single line it works perfectly fine

carc1n0gen commented May 26, 2016

Hello I know this is old, but I'm having issues sending the content of a file that has one or more new lines.

I get:

{
  "message": "Problems parsing JSON",
  "documentation_url": "https://developer.github.com/enterprise/2.4/v3"
}

But if the file content I send has a single line it works perfectly fine

@arthas353

This comment has been minimized.

Show comment
Hide comment
@arthas353

arthas353 Jul 7, 2016

Hello,
I want to update a specific file in my git master repo; How do I do that using curl ?

Hello,
I want to update a specific file in my git master repo; How do I do that using curl ?

@zh-wowtv

This comment has been minimized.

Show comment
Hide comment
@zh-wowtv

zh-wowtv Jul 14, 2016

Is it possible to upload image file through API?

Is it possible to upload image file through API?

@konradkonrad

This comment has been minimized.

Show comment
Hide comment
@konradkonrad

konradkonrad Nov 30, 2016

Not sure if this is the right place, but: have you ever figured out a way to trigger obtaining an OTP (e.g. via sms) by cURL against the github API?

curl -u <myusername> https://github.com/user

reponds with

{
  "message": "Must specify two-factor authentication OTP code.",
  "documentation_url": "https://developer.github.com/v3/auth#working-with-two-factor-authentication"
}

however, it doesn't trigger the creation of a OTP. Signing in through the website does send me an sms, so there doesn't seem to be a setup problem.

Not sure if this is the right place, but: have you ever figured out a way to trigger obtaining an OTP (e.g. via sms) by cURL against the github API?

curl -u <myusername> https://github.com/user

reponds with

{
  "message": "Must specify two-factor authentication OTP code.",
  "documentation_url": "https://developer.github.com/v3/auth#working-with-two-factor-authentication"
}

however, it doesn't trigger the creation of a OTP. Signing in through the website does send me an sms, so there doesn't seem to be a setup problem.

@larshp

This comment has been minimized.

Show comment
Hide comment
@larshp

larshp Feb 5, 2017

had the same issue as @konradkonrad describes above, following curl generates a SMS code when I try,

curl -i -u your_username -d '{"scopes": ["repo", "user"], "note": "getting-started"}' https://api.github.com/authorizations

larshp commented Feb 5, 2017

had the same issue as @konradkonrad describes above, following curl generates a SMS code when I try,

curl -i -u your_username -d '{"scopes": ["repo", "user"], "note": "getting-started"}' https://api.github.com/authorizations

@caspyin

This comment has been minimized.

Show comment
Hide comment
@caspyin

caspyin Feb 23, 2017

It seems this has been useful over the years to various people. If you would like to contribute any changes to this gist, please fork and then ping me here in the comments to review.

Owner

caspyin commented Feb 23, 2017

It seems this has been useful over the years to various people. If you would like to contribute any changes to this gist, please fork and then ping me here in the comments to review.

@suntong

This comment has been minimized.

Show comment
Hide comment
@suntong

suntong Aug 14, 2017

@caspyin, thanks for the great post. How can you send base64 encoded content?

I tried,

curl -H 'Accept: application/vnd.github.VERSION.base64' --data @data.txt https://api.github.com/gists

But the content is still base64 encoded:
https://gist.github.com/anonymous/c79cbfcf466af02eb08dc9ad6f26b587

Here is my test data.txt:

{
  "description": "the description for this gist",
  "public": true,
  "files": {
    "file1.txt": {
      "content": "aG9zdEBtYWlsLmNvbTpwYXNzd29yZAo="
    }
  }
}

Thx.

suntong commented Aug 14, 2017

@caspyin, thanks for the great post. How can you send base64 encoded content?

I tried,

curl -H 'Accept: application/vnd.github.VERSION.base64' --data @data.txt https://api.github.com/gists

But the content is still base64 encoded:
https://gist.github.com/anonymous/c79cbfcf466af02eb08dc9ad6f26b587

Here is my test data.txt:

{
  "description": "the description for this gist",
  "public": true,
  "files": {
    "file1.txt": {
      "content": "aG9zdEBtYWlsLmNvbTpwYXNzd29yZAo="
    }
  }
}

Thx.

@suntong

This comment has been minimized.

Show comment
Hide comment
@suntong

suntong Aug 14, 2017

I figured it out -- base64 encoding is only supported for download, not upload.

So to post any arbitrary string or file, check out

jsonfiddle

Usage
https://github.com/go-jsonfile/jsonfiddle#usage-1

jsonfiddle esc will escape any arbitrary string so as to embed it as content of json string. This seems useless at first, but it actually allows you to embed any arbitrary file into GitHub Gists JSON API, so as to post any arbitrary file onto GitHub Gist:

echo '{"description":"SmartyStreets API Demo","public":true,"files":{"SmartyStreets.json":{"content":'"`jsonfiddle fmt -i test/SmartyStreets.json | jsonfiddle esc -i`"'}}}' | curl --data @- https://api.github.com/gists

This will give you
https://gist.github.com/anonymous/1423d4768dd9b88262ca513626e68d8e

By "arbitrary file" I do mean arbitrary file. Check this out:
https://gist.github.com/anonymous/a51798ce99ff59d8d4ba536cbf4b6996

This is why jsonfiddle esc is a command on its own, instead of being part of functionalities of jsonfiddle fmt or jsonfiddle sort.

suntong commented Aug 14, 2017

I figured it out -- base64 encoding is only supported for download, not upload.

So to post any arbitrary string or file, check out

jsonfiddle

Usage
https://github.com/go-jsonfile/jsonfiddle#usage-1

jsonfiddle esc will escape any arbitrary string so as to embed it as content of json string. This seems useless at first, but it actually allows you to embed any arbitrary file into GitHub Gists JSON API, so as to post any arbitrary file onto GitHub Gist:

echo '{"description":"SmartyStreets API Demo","public":true,"files":{"SmartyStreets.json":{"content":'"`jsonfiddle fmt -i test/SmartyStreets.json | jsonfiddle esc -i`"'}}}' | curl --data @- https://api.github.com/gists

This will give you
https://gist.github.com/anonymous/1423d4768dd9b88262ca513626e68d8e

By "arbitrary file" I do mean arbitrary file. Check this out:
https://gist.github.com/anonymous/a51798ce99ff59d8d4ba536cbf4b6996

This is why jsonfiddle esc is a command on its own, instead of being part of functionalities of jsonfiddle fmt or jsonfiddle sort.

@aman0446

This comment has been minimized.

Show comment
Hide comment
@aman0446

aman0446 Aug 22, 2017

I'm trying to access my company's github repository(github enterprise). command i'm using is
curl --user "name:pwd" "https://github."company name".com/{owner}/repo" , but i'm getting "Not found error" .
When i login through the browser with the same credentials, it is going through.
i was reading through " OAuth and PAT(Personal Access tokens). Couldn't get to working. Can someone walk me through the steps if you have already done it?

I'm trying to access my company's github repository(github enterprise). command i'm using is
curl --user "name:pwd" "https://github."company name".com/{owner}/repo" , but i'm getting "Not found error" .
When i login through the browser with the same credentials, it is going through.
i was reading through " OAuth and PAT(Personal Access tokens). Couldn't get to working. Can someone walk me through the steps if you have already done it?

@gmilos

This comment has been minimized.

Show comment
Hide comment
@gmilos

gmilos Sep 1, 2017

@aman0446 you need something like this:

curl -H "Authorization: token PAT-TOKEN" https://github.COMPANY_NAME.com/api/v3/users/USER/repos

gmilos commented Sep 1, 2017

@aman0446 you need something like this:

curl -H "Authorization: token PAT-TOKEN" https://github.COMPANY_NAME.com/api/v3/users/USER/repos
@rpbaptist

This comment has been minimized.

Show comment
Hide comment
@rpbaptist

rpbaptist Dec 12, 2017

For OTP (2FA) you will have to send this header along:

curl -H "X-GitHub-OTP: 123456"

rpbaptist commented Dec 12, 2017

For OTP (2FA) you will have to send this header along:

curl -H "X-GitHub-OTP: 123456"
@ttpro1995

This comment has been minimized.

Show comment
Hide comment
@ttpro1995

ttpro1995 Jun 12, 2018

In

curl --user "ttpro1995:MY_OWN_PASSWORD" --data '{"description":"Created via API","public":"false","files":{"file1.txt":{"content":"Demo"}}' https://api.github.com/gists
How can I replace password with something else (API key )?

In

curl --user "ttpro1995:MY_OWN_PASSWORD" --data '{"description":"Created via API","public":"false","files":{"file1.txt":{"content":"Demo"}}' https://api.github.com/gists
How can I replace password with something else (API key )?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment