Let's have some command-line fun with curl, jq, and the new GitHub Search API.
Today we're looking for:
Let's have some command-line fun with curl, jq, and the new GitHub Search API.
Today we're looking for:
Keep an eye on the latest trending repos.
# We'll use the `date` command to get the date for "7 days ago"
$ date -v-7d '+%Y-%m-%d'
# => 2013-07-15
$ curl -G https://api.github.com/search/repositories \
--data-urlencode "q=created:>`date -v-7d '+%Y-%m-%d'`" \
--data-urlencode "sort=stars" \
--data-urlencode "order=desc" \
-H "Accept: application/vnd.github.preview" \
| jq ".items[0,1,2] | {name, description, language, watchers_count, html_url}"
{
"html_url": "https://github.com/mame/quine-relay",
"watchers_count": 2492,
"language": "Ruby",
"description": "An uroboros program with 50 programming languages",
"name": "quine-relay"
}
{
"html_url": "https://github.com/jehna/VerbalExpressions",
"watchers_count": 483,
"language": "JavaScript",
"description": "JavaScript Regular expressions made easy",
"name": "VerbalExpressions"
}
{
"html_url": "https://github.com/applidium/ADTransitionController",
"watchers_count": 340,
"language": "Objective-C",
"description": "UINavigationController with custom transitions",
"name": "ADTransitionController"
}
Check out the Repository Search API docs for more details.
Go follow these poor souls.
$ curl -G https://api.github.com/search/users \
--data-urlencode 'q=followers:0' \
--data-urlencode 'sort=joined' \
--data-urlencode 'order=asc' \
-H 'Accept: application/vnd.github.preview' \
| jq '.items[0,1,2] | {html_url, login, id}'
{
"id": 30,
"login": "fanvsfan",
"html_url": "https://github.com/fanvsfan"
}
{
"id": 32,
"login": "railsjitsu",
"html_url": "https://github.com/railsjitsu"
}
{
"id": 44,
"login": "errfree",
"html_url": "https://github.com/errfree"
}
Check out the User Search API docs for more details.
Track issue trends over time.
for i in {1..7}
do
created_on=`date -v-"${i}"d '+%Y-%m-%d'`
issue_count=$( \
curl -G https://api.github.com/search/issues \
--data-urlencode "q=language:ruby created:$created_on" \
-H "Authorization: token REDACTED" \
-H "Accept: application/vnd.github.preview" | jq ".total_count" \
)
echo "$created_on: $issue_count"
done
2013-07-22: 1174
2013-07-21: 716
2013-07-20: 687
2013-07-19: 1336
2013-07-18: 1348
2013-07-17: 1471
2013-07-16: 1386
Check out the Issue Search API docs for more details.
Rebels! I bet these dudes don't like hammocks either.
$ curl -G https://api.github.com/search/code \
--data-urlencode 'q=MIT License path:project.clj' \
--data-urlencode 'sort=indexed' \
--data-urlencode 'order=desc' \
-H 'Accept: application/vnd.github.preview' \
| jq '.items[0,1,2] | {description: (.repository.description), name: (.repository.full_name), html_url}'
{
"html_url": "https://github.com/royvandewater/optparse/blob/e4bb8558405ffd7ba6a14718b89e7cd418b5565e/project.clj",
"name": "royvandewater/optparse",
"description": "Option parser for clojure"
}
{
"html_url": "https://github.com/SnootyMonkey/coming-soon/blob/108decfd74338ba5428580c8cd156f684484d3b2/project.clj",
"name": "SnootyMonkey/coming-soon",
"description": "coming-soon is a simple Clojure/ClojureScript/Redis 'landing page' application that takes just a few minute to setup"
}
{
"html_url": "https://github.com/rreas/ring-test/blob/5fd78404eec4f81033ae073c5a42ea8a55ccc75a/project.clj",
"name": "rreas/ring-test",
"description": "An integration test framework for ring web applications."
}
Check out the Code Search API docs for more details.
$ curl -G https://api.github.com/search/repositories \
--data-urlencode 'q=@jeresig -language:javascript' \
-H 'Accept: application/vnd.github.preview' \
| jq '.items[] | {html_url, watchers_count, language, name}'
{
"name": "processing-js",
"language": "Java",
"watchers_count": 1455,
"html_url": "https://github.com/jeresig/processing-js"
}
{
"name": "selectortest",
"language": null,
"watchers_count": 12,
"html_url": "https://github.com/jeresig/selectortest"
}
{
"name": "wtpa-bot",
"language": "Perl",
"watchers_count": 9,
"html_url": "https://github.com/jeresig/wtpa-bot"
}
{
"name": "jeresig.github.com",
"language": null,
"watchers_count": 6,
"html_url": "https://github.com/jeresig/jeresig.github.com"
}
{
"name": "apples2artworks",
"language": "Python",
"watchers_count": 1,
"html_url": "https://github.com/jeresig/apples2artworks"
}
{
"name": "datacook",
"language": "Perl",
"watchers_count": 0,
"html_url": "https://github.com/jeresig/datacook"
}
@jeresig has a repo with zero people watching it? This changes my whole worldview.
Check out the Repository Search API docs for more details.
OK. OK. You were only expecting 5 examples. But since you read this far, here's a bonus one.
# We'll use the `date` command to get the date for "1 month ago"
$ date -v-1m '+%Y-%m-%d'
# => 2013-06-23
$ curl -G https://api.github.com/search/issues \
--data-urlencode "q=created:>`date -v-7d '+%Y-%m-%d'`" \
--data-urlencode 'sort=comments' \
--data-urlencode 'order=desc' \
-H 'Accept: application/vnd.github.preview' \
| jq '.items[0,1,2] | {html_url, title, comments}'
I ran this a couple months ago, when we were first sketching out the search API. What did I find? A Pull Request to remove all press representatives from bitcoin.org. Followed by another Pull Request to counteract the first one.
Interesting times.
{
"comments": 158,
"title": "Remove press representatives",
"html_url": "https://github.com/bitcoin/bitcoin.org/issues/152"
}
{
"comments": 151,
"title": "Add several independent voices to the Press Center page",
"html_url": "https://github.com/bitcoin/bitcoin.org/issues/162"
}
{
"comments": 150,
"title": "Select and implement a template engine (or The issue to discuss everything Framework refactoring)",
"html_url": "https://github.com/joomla/jissues/issues/86"
}
Check out the Issue Search API docs for more details.
Try @rochefort's git-trend
. It's a command line utility that shows you trending github repos. Another is my github-trending
gem that fetches daily, weekly and monthly trending repositories on github.
Does any of the current github python api does advanced search like this?
Its probably only entertaining on MacOS date: invalid option -- 'v'
created_on=`date --date="${i} days ago" '+%Y-%m-%d'
and add --silent to curl argument
How about Find the hottest repositories created in the last week, just like https://github.com/trending
Your 04-rebel-clojure-projects.md
query doesn't work. I wonder if they added this requirement after you posted this.
"message": "Validation Failed",
"errors": [
{
"message": "Must include at least one user, organization, or repository",
"resource": "Search",
"field": "q",
"code": "invalid"
}
],
"documentation_url": "https://developer.github.com/v3/search/"
}
@nealmcb This has to do with authentication. Quoting from the "Considerations for code search" section of the article on searching code:
Logged in users can search all public repositories while anonymous searches must include a limit on
org:
,user:
, orrepo:
.
If you add authentication to your calls, things should work as described...
Jason, basically I want to find the pull requests of a repository which are older than 7 days of the present date so my query for getting the pull requests is as follows:::
https://api.github.com/repos/octokit/octokit.rb/pulls
the response is as follows
[
{
"url": "https://api.github.com/repos/octokit/octokit.rb/pulls/951",
"id": 147991930,
"html_url": "https://github.com/octokit/octokit.rb/pull/951",
"diff_url": "https://github.com/octokit/octokit.rb/pull/951.diff",
"patch_url": "https://github.com/octokit/octokit.rb/pull/951.patch",
"issue_url": "https://api.github.com/repos/octokit/octokit.rb/issues/951",
"number": 951,
"state": "open",
"locked": false,
"title": "CI against Ruby 2.2.8, 2.3.5, 2.4.2",
"user": {
"login": "yatmsu",
"id": 436515,
"avatar_url": "https://avatars1.githubusercontent.com/u/436515?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/yatmsu",
"html_url": "https://github.com/yatmsu",
"followers_url": "https://api.github.com/users/yatmsu/followers",
"following_url": "https://api.github.com/users/yatmsu/following{/other_user}",
"gists_url": "https://api.github.com/users/yatmsu/gists{/gist_id}",
"starred_url": "https://api.github.com/users/yatmsu/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/yatmsu/subscriptions",
"organizations_url": "https://api.github.com/users/yatmsu/orgs",
"repos_url": "https://api.github.com/users/yatmsu/repos",
"events_url": "https://api.github.com/users/yatmsu/events{/privacy}",
"received_events_url": "https://api.github.com/users/yatmsu/received_events",
"type": "User",
"site_admin": false
},
"body": "https://www.ruby-lang.org/en/news/2017/09/14/ruby-2-2-8-released/\r\nhttps://www.ruby-lang.org/en/news/2017/09/14/ruby-2-3-5-released/\r\nhttps://www.ruby-lang.org/en/news/2017/09/14/ruby-2-4-2-released/",
"created_at": "2017-10-22T12:51:14Z",
"updated_at": "2017-10-22T13:36:43Z",
"closed_at": null,
"merged_at": null,
"merge_commit_sha": "d0ea0ed9a32999b62c154515c4ebb351486243ef",
"assignee": null,
"assignees": [],
"requested_reviewers": [],
"milestone": null,
"commits_url": "https://api.github.com/repos/octokit/octokit.rb/pulls/951/commits",
"review_comments_url": "https://api.github.com/repos/octokit/octokit.rb/pulls/951/comments",
"review_comment_url": "https://api.github.com/repos/octokit/octokit.rb/pulls/comments{/number}",
"comments_url": "https://api.github.com/repos/octokit/octokit.rb/issues/951/comments",
"statuses_url": "https://api.github.com/repos/octokit/octokit.rb/statuses/d1dc7dd2e8e3c21a24f4cfe2a7d8d785557f0b54",
"head": {
so what parameters should I add after pulls to get the pull requests based on date (older than 7 days),
can you give me the exact query, (I can see the created at parameter in the response but I don't know how to use it)
I want to use this get get call in my python script, so I think we can't use --data--urlencode.
can you give me the exact query so that it works in postman and my python script.
thanks
Find all github.io repositories that are actually online?
Is Code search API useful for searching, filename:x across all repositories with "xyz in the file content? As of now, it throws me error saying that it, need User/repo/Organization
The above requests seem not to be working anymore?
The date command on Ubuntu Linux does not have the '-v' flag.