Skip to content

Instantly share code, notes, and snippets.

@kellyrob99
Created April 23, 2012 01:25
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save kellyrob99/2468052 to your computer and use it in GitHub Desktop.
Save kellyrob99/2468052 to your computer and use it in GitHub Desktop.
Access the Github V3 api and find all watchers for a specified user's repositories. Generates output suitable for processing by https://gist.github.com/2475460
@Grab('org.codehaus.groovy.modules.http-builder:http-builder:0.5.2')
import groovy.json.JsonBuilder
import groovyx.net.http.HTTPBuilder
import static groovyx.net.http.ContentType.JSON
final rootUser = args[0]
final String RATE_LIMIT_HEADER = 'X-RateLimit-Remaining'
final String rootUrl = 'https://api.github.com'
final Closure<Boolean> hasWatchers = {it.watchers > 1}
final Closure findReposForUser = { HTTPBuilder http, username ->
http.get(path: "/users/$username/repos", contentType: JSON) { resp, json ->
return [resp.headers[RATE_LIMIT_HEADER].value as int, json.toList()]
}
}
final Closure findWatchers = { HTTPBuilder http, username, repo ->
http.get(path: "/repos/$username/$repo/watchers", contentType: JSON) { resp, json ->
return [resp.headers[RATE_LIMIT_HEADER].value as int, json.toList()*.login.flatten().unique() - username]
}
}
LinkedList nodes = [rootUser] as LinkedList
Map<String, List> usersToRepos = [:]
Map<String, List<String>> watcherMap = [:]
boolean hasRemainingCalls = true
final HTTPBuilder builder = new HTTPBuilder(rootUrl)
//add a default error handler
builder.handler.failure = { resp ->
"Unexpected failure: ${resp.statusLine}"
}
while(!nodes.isEmpty() && hasRemainingCalls)
{
String username = nodes.remove()
println "processing $username"
println "remaining nodes = ${nodes.size()}"
def remainingApiCalls, repos, watchers
(remainingApiCalls, repos) = findReposForUser(builder, username)
usersToRepos[username] = repos
hasRemainingCalls = remainingApiCalls > 300
repos.findAll(hasWatchers).each{ repo ->
(remainingApiCalls, watchers) = findWatchers(builder, username, repo.name)
def oldValue = watcherMap.get(username, [] as LinkedHashSet)
oldValue.addAll(watchers)
watcherMap[username] = oldValue
nodes.addAll(watchers)
nodes.removeAll(watcherMap.keySet())
hasRemainingCalls = remainingApiCalls > 300
}
if(!hasRemainingCalls)
{
println "Stopped with $remainingApiCalls api calls left."
println "Still have not processed ${nodes.size()} users."
}
}
new File('reposOutput.json').withWriter {writer ->
writer << new JsonBuilder(watcherMap).toPrettyString()
}
@victorbordo
Copy link

As per the Github REST API v3 Watching docs:

In August 2012, we changed the way watching works on GitHub. At the time of that change, many API clients were already using the existing "watcher" endpoints to access starring data. To avoid breaking those applications, the legacy "watcher" endpoints continue to provide starring data.

This script is now consuming the starring data instead of watchers on the repo. Add the subscribers endpoint here: path: "/repos/$username/$repo/subscribers" to get users watching the repo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment