Skip to content

Instantly share code, notes, and snippets.

@rjurney
Created June 3, 2013 18:12
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save rjurney/5700101 to your computer and use it in GitHub Desktop.
Save rjurney/5700101 to your computer and use it in GitHub Desktop.
Loading the github data, as implied recommendations...
/* Watch events happen whenever a user 'watches' a github project */
watch_events = LOAD '/tmp/WatchEvent'; /* 's3://github-explorer/WatchEvent' */
watch_ratings = FOREACH watch_events GENERATE (chararray)$0#'actor'#'login' AS follower:chararray,
(chararray)$0#'repo'#'name' AS repo:chararray,
1.0 AS rating;
/* Download events, whenever a user downloads a tarball of a repo */
download_events = LOAD '/tmp/DownloadEvent' as (json: map[]); /* 's3://github-explorer/DownloadEvent' */
download_ratings = FOREACH download_events GENERATE (chararray)$0#'actor_attributes'#'login' AS follower:chararray,
StringConcat((chararray)$0#'repository'#'owner', '/', $0#'repository'#'name') AS repo:chararray,
1.0 AS rating;
/* Create issues events - implies a user has already downloaded/forked and tried the software */
issues_events = LOAD '/tmp/IssuesEvent' AS (json: map[]); /* 's3://github-explorer/IssuesEvent' */
issues_ratings = FOREACH issues_events GENERATE (chararray)$0#'actor_attributes'#'login' AS follower:chararray,
StringConcat((chararray)$0#'repository'#'owner', '/', $0#'repository'#'name') AS repo:chararray,
2.0 AS rating;
/* Fork events happen whenever a github project is 'forked' */
fork_events = LOAD '/tmp/ForkEvent'; /* 's3://github-explorer/ForkEvent' */
fork_ratings = FOREACH fork_events GENERATE (chararray)$0#'actor'#'login' AS follower:chararray,
(chararray)$0#'repo'#'name' as repo:chararray,
3.0 AS rating;
/* Create repository event - strongest association with a repo possible */
create_events = LOAD '/tmp/CreateEvent' as (json: map[]); /* 's3://github-explorer/CreateEvent' */
create_ratings = FOREACH create_events GENERATE (chararray)$0#'actor_attributes'#'login' AS follower:chararray,
StringConcat((chararray)$0#'repository'#'owner', '/', $0#'repository'#'name') AS repo:chararray,
4.0 AS rating;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment