scheduled_go_packages.py contains the packages which were scheduled by Jobs API for 200 go packages. This list is obtained by parsing the jobs server logs.
go_ecosystem_graph.json contains the actual packages contained in the graph after go ingestion finished
With some Python scripting we have two sets scheduled_set and graph_set as folllows:
In [39]: scheduled_set
Out[39]:
{'github.com/BurntSushi/toml',
'github.com/ChimeraCoder/anaconda',
'github.com/GeertJohan/go.rice',
'github.com/PuerkitoBio/goquery',
'github.com/Shopify/sarama',
'github.com/Sirupsen/logrus',
'github.com/Unknwon/goconfig',
'github.com/asaskevich/govalidator',
'github.com/astaxie/beego',
'github.com/atotto/clipboard',
'github.com/bitly/go-nsq',
'github.com/bitly/go-simplejson',
'github.com/blang/semver',
'github.com/bmizerany/pat',
'github.com/bmizerany/pq',
'github.com/boltdb/bolt',
'github.com/cenkalti/backoff',
'github.com/cheggaaa/pb',
'github.com/cihub/seelog',
'github.com/codegangsta/cli',
'github.com/codegangsta/martini',
'github.com/codegangsta/negroni',
'github.com/coopernurse/gorp',
'github.com/dancannon/gorethink',
'github.com/dchest/uniuri',
'github.com/deckarep/golang-set',
'github.com/dgrijalva/jwt-go',
'github.com/disintegration/imaging',
'github.com/docopt/docopt-go',
'github.com/dustin/go-humanize',
'github.com/elazarl/go-bindata-assetfs',
'github.com/emicklei/go-restful',
'github.com/fatih/color',
'github.com/fatih/structs',
'github.com/fsnotify/fsnotify',
'github.com/fsouza/go-dockerclient',
'github.com/ghodss/yaml',
'github.com/gin-gonic/gin',
'github.com/gizak/termui',
'github.com/go-fsnotify/fsnotify',
'github.com/go-martini/martini',
'github.com/go-sql-driver/mysql',
'github.com/go-xorm/xorm',
'github.com/gocql/gocql',
'github.com/golang/glog',
'github.com/google/gopacket',
'github.com/googollee/go-socket.io',
'github.com/gorilla/context',
'github.com/gorilla/handlers',
'github.com/gorilla/mux',
'github.com/gorilla/schema',
'github.com/gorilla/securecookie',
'github.com/gorilla/sessions',
'github.com/gorilla/websocket',
'github.com/hashicorp/golang-lru',
'github.com/hoisie/web',
'github.com/howeyc/fsnotify',
'github.com/howeyc/gopass',
'github.com/jessevdk/go-flags',
'github.com/jinzhu/gorm',
'github.com/jmoiron/sqlx',
'github.com/joho/godotenv',
'github.com/julienschmidt/httprouter',
'github.com/justinas/alice',
'github.com/kardianos/osext',
'github.com/kelseyhightower/envconfig',
'github.com/kr/pretty',
'github.com/kr/pty',
'github.com/labstack/echo',
'github.com/lib/pq',
'github.com/libgit2/git2go',
'github.com/martini-contrib/binding',
'github.com/martini-contrib/render',
'github.com/martini-contrib/sessions',
'github.com/mattn/go-colorable',
'github.com/mattn/go-sqlite3',
'github.com/mgutz/ansi',
'github.com/microcosm-cc/bluemonday',
'github.com/mitchellh/cli',
'github.com/mitchellh/go-homedir',
'github.com/mitchellh/mapstructure',
'github.com/mrjones/oauth',
'github.com/nfnt/resize',
'github.com/nlopes/slack',
'github.com/nsf/termbox-go',
'github.com/nu7hatch/gouuid',
'github.com/ogier/pflag',
'github.com/olekukonko/tablewriter',
'github.com/op/go-logging',
'github.com/parnurzeal/gorequest',
'github.com/pborman/uuid',
'github.com/pebbe/zmq4',
'github.com/pkg/errors',
'github.com/rcrowley/go-metrics',
'github.com/revel/revel',
'github.com/robertkrimen/otto',
'github.com/robfig/cron',
'github.com/robfig/revel',
'github.com/rs/cors',
'github.com/russross/blackfriday',
'github.com/satori/go.uuid',
'github.com/spf13/cobra',
'github.com/spf13/pflag',
'github.com/spf13/viper',
'github.com/streadway/amqp',
'github.com/tarm/serial',
'github.com/tealeg/xlsx',
'github.com/thoj/go-ircevent',
'github.com/twinj/uuid',
'github.com/unrolled/render',
'github.com/urfave/cli',
'github.com/valyala/fasthttp',
'github.com/zenazn/goji'}
In [40]: graph_set
Out[40]:
{'github.com/ActiveState/tail',
'github.com/BurntSushi/toml',
'github.com/ChimeraCoder/anaconda',
'github.com/ChimeraCoder/tokenbucket',
'github.com/GeertJohan/go.incremental',
'github.com/GeertJohan/go.rice',
'github.com/Go-SQL-Driver/MySQL',
'github.com/Knetic/govaluate',
'github.com/Pallinder/go-randomdata',
'github.com/PuerkitoBio/goquery',
'github.com/PuerkitoBio/purell',
'github.com/PuerkitoBio/urlesc',
'github.com/Shopify/sarama',
'github.com/Sirupsen/logrus',
'github.com/Unknwon/com',
'github.com/Unknwon/goconfig',
'github.com/abbot/go-http-auth',
'github.com/agtorre/gocolorize',
'github.com/ajstarks/svgo',
'github.com/akavel/rsrc',
'github.com/alecthomas/gozmq',
'github.com/alecthomas/kingpin',
'github.com/andlabs/ui',
'github.com/andybalholm/cascadia',
'github.com/antonholmquist/jason',
'github.com/armon/go-radix',
'github.com/asaskevich/govalidator',
'github.com/astaxie/beego',
'github.com/atotto/clipboard',
'github.com/avelino/awesome-go',
'github.com/azr/backoff',
'github.com/beego/x2j',
'github.com/belogik/goes',
'github.com/bgentry/speakeasy',
'github.com/bitly/go-nsq',
'github.com/bitly/go-simplejson',
'github.com/blang/semver',
'github.com/blevesearch/bleve',
'github.com/bmizerany/pat',
'github.com/bmizerany/pq',
'github.com/boj/redistore',
'github.com/boltdb/bolt',
'github.com/bradfitz/gomemcache',
'github.com/braintree/manners',
'github.com/casbin/casbin',
'github.com/cenkalti/backoff',
'github.com/cheggaaa/pb',
'github.com/chzyer/readline',
'github.com/cihub/seelog',
'github.com/clbanning/x2j',
'github.com/cloudflare/golz4',
'github.com/cloudfoundry/gosigar',
'github.com/codegangsta/cli',
'github.com/codegangsta/inject',
'github.com/codegangsta/martini',
'github.com/codegangsta/negroni',
'github.com/coopernurse/gorp',
'github.com/couchbase/cbauth',
'github.com/couchbase/clog',
'github.com/couchbase/go-couchbase',
'github.com/couchbase/go_n1ql',
'github.com/couchbase/gomemcached',
'github.com/couchbase/goutils',
'github.com/couchbaselabs/go_n1ql',
'github.com/cupcake/rdb',
'github.com/daaku/go.zipexe',
'github.com/dancannon/gorethink',
'github.com/davecgh/go-spew',
'github.com/daviddengcn/go-colortext',
'github.com/dchest/uniuri',
'github.com/deckarep/golang-set',
'github.com/denisenkom/go-mssqldb',
'github.com/dgrijalva/jwt-go',
'github.com/digitalocean/godo',
'github.com/disintegration/imaging',
'github.com/djimenez/iconv-go',
'github.com/docopt/docopt-go',
'github.com/drone/routes',
'github.com/dustin/go-humanize',
'github.com/dustin/go-jsonpointer',
'github.com/dustin/gojson',
'github.com/eapache/go-resiliency',
'github.com/eapache/go-xerial-snappy',
'github.com/eapache/queue',
'github.com/edsrzf/mmap-go',
'github.com/elazarl/go-bindata-assetfs',
'github.com/elazarl/goproxy',
'github.com/emicklei/go-restful',
'github.com/emicklei/go-restful-openapi',
'github.com/emicklei/go-restful-swagger12',
'github.com/ernesto-jimenez/gogen',
'github.com/fatih/color',
'github.com/fatih/pool',
'github.com/fatih/structs',
'github.com/flosch/pongo2',
'github.com/franela/goreq',
'github.com/fsnotify/fsnotify',
'github.com/fsouza/go-dockerclient',
'github.com/garyburd/go-oauth',
'github.com/garyburd/redigo',
'github.com/getsentry/raven-go',
'github.com/ghodss/yaml',
'github.com/gin-gonic/gin',
'github.com/gizak/termui',
'github.com/glendc/gopher-json',
'github.com/go-errors/errors',
'github.com/go-fsnotify/fsnotify',
'github.com/go-gorp/gorp',
'github.com/go-inf/inf',
'github.com/go-ini/ini',
'github.com/go-martini/martini',
'github.com/go-openapi/jsonpointer',
'github.com/go-openapi/jsonreference',
'github.com/go-openapi/spec',
'github.com/go-openapi/swag',
'github.com/go-sql-driver/mysql',
'github.com/go-xorm/xorm',
'github.com/go-yaml/yaml',
'github.com/gocql/gocql',
'github.com/gocraft/web',
'github.com/godbus/dbus',
'github.com/gogits/gogs',
'github.com/goji/param',
'github.com/golang/appengine',
'github.com/golang/crypto',
'github.com/golang/freetype',
'github.com/golang/glog',
'github.com/golang/groupcache',
'github.com/golang/image',
'github.com/golang/mock',
'github.com/golang/net',
'github.com/golang/oauth2',
'github.com/golang/protobuf',
'github.com/golang/snappy',
'github.com/golang/sys',
'github.com/golang/text',
'github.com/golang/tools',
'github.com/gonum/plot',
'github.com/google/go-genproto',
'github.com/google/google-api-go-client',
'github.com/google/gopacket',
'github.com/google/gxui',
'github.com/googollee/go-engine.io',
'github.com/googollee/go-socket.io',
'github.com/gorilla/context',
'github.com/gorilla/feeds',
'github.com/gorilla/handlers',
'github.com/gorilla/mux',
'github.com/gorilla/pat',
'github.com/gorilla/rpc',
'github.com/gorilla/schema',
'github.com/gorilla/securecookie',
'github.com/gorilla/sessions',
'github.com/gorilla/websocket',
'github.com/grpc/grpc-go',
'github.com/hailocab/go-hostpool',
'github.com/hashicorp/go-multierror',
'github.com/hashicorp/go-version',
'github.com/hashicorp/golang-lru',
'github.com/hashicorp/hcl',
'github.com/hoisie/mustache',
'github.com/hoisie/web',
'github.com/howeyc/fsnotify',
'github.com/howeyc/gopass',
'github.com/hpcloud/tail',
'github.com/hybridgroup/gobot',
'github.com/imdario/mergo',
'github.com/inconshreveable/log15',
'github.com/jessevdk/go-flags',
'github.com/jinzhu/gorm',
'github.com/jinzhu/inflection',
'github.com/jmhodges/levigo',
'github.com/jmoiron/sqlx',
'github.com/joho/godotenv',
'github.com/jroimartin/gocui',
'github.com/jteeuwen/go-pkg-rss',
'github.com/juju/errors',
'github.com/juju/ratelimit',
'github.com/julienschmidt/httprouter',
'github.com/justinas/alice',
'github.com/justinas/nosurf',
'github.com/kardianos/osext',
'github.com/kataras/iris',
'github.com/kelseyhightower/envconfig',
'github.com/kennygrant/sanitize',
'github.com/klauspost/compress',
'github.com/klauspost/cpuid',
'github.com/kr/pretty',
'github.com/kr/pty',
'github.com/kr/text',
'github.com/kurrik/oauth1a',
'github.com/labstack/echo',
'github.com/labstack/gommon',
'github.com/lib/pq',
'github.com/libgit2/git2go',
'github.com/lucasb-eyer/go-colorful',
'github.com/mailgun/mailgun-go',
'github.com/mailru/easyjson',
'github.com/martini-contrib/binding',
'github.com/martini-contrib/cors',
'github.com/martini-contrib/render',
'github.com/martini-contrib/sessions',
'github.com/maruel/panicparse',
'github.com/mattn/go-colorable',
'github.com/mattn/go-isatty',
'github.com/mattn/go-runewidth',
'github.com/mattn/go-shellwords',
'github.com/mattn/go-sqlite3',
'github.com/mgutz/ansi',
'github.com/mholt/caddy',
'github.com/microcosm-cc/bluemonday',
'github.com/mitchellh/cli',
'github.com/mitchellh/colorstring',
'github.com/mitchellh/go-homedir',
'github.com/mitchellh/go-wordwrap',
'github.com/mitchellh/mapstructure',
'github.com/mjibson/appstats',
'github.com/moovweb/gokogiri',
'github.com/moul/http2curl',
'github.com/mrjones/oauth',
'github.com/naoina/toml',
'github.com/nats-io/nats',
'github.com/nfnt/resize',
'github.com/nlopes/slack',
'github.com/nsf/termbox-go',
'github.com/nsqio/go-nsq',
'github.com/nu7hatch/gouuid',
'github.com/ogier/pflag',
'github.com/olekukonko/tablewriter',
'github.com/onsi/ginkgo',
'github.com/onsi/gomega',
'github.com/op/go-logging',
'github.com/oschwald/geoip2-golang',
'github.com/oxtoacart/bpool',
'github.com/parnurzeal/gorequest',
'github.com/patrickmn/go-cache',
'github.com/pborman/uuid',
'github.com/pebbe/zmq4',
'github.com/pelletier/go-toml',
'github.com/peterh/liner',
'github.com/pierrec/lz4',
'github.com/pierrec/xxHash',
'github.com/pivotal-golang/lager',
'github.com/pkg/errors',
'github.com/pkg/sftp',
'github.com/pmezard/go-difflib',
'github.com/pmylund/go-cache',
'github.com/posener/complete',
'github.com/pquerna/ffjson',
'github.com/rackspace/gophercloud',
'github.com/rcrowley/go-metrics',
'github.com/revel/revel',
'github.com/robertkrimen/otto',
'github.com/robfig/config',
'github.com/robfig/cron',
'github.com/robfig/go-cache',
'github.com/robfig/gomemcache',
'github.com/robfig/pathtree',
'github.com/robfig/revel',
'github.com/rs/cors',
'github.com/russross/blackfriday',
'github.com/samalba/dockerclient',
'github.com/satori/go.uuid',
'github.com/sendgrid/sendgrid-go',
'github.com/siddontang/go',
'github.com/siddontang/goredis',
'github.com/siddontang/rdb',
'github.com/sirupsen/logrus',
'github.com/spf13/cobra',
'github.com/spf13/hugo',
'github.com/spf13/pflag',
'github.com/spf13/viper',
'github.com/stathat/go',
'github.com/streadway/amqp',
'github.com/streadway/simpleuuid',
'github.com/stretchr/objx',
'github.com/stretchr/testify',
'github.com/syndtr/goleveldb',
'github.com/tarm/goserial',
'github.com/tarm/serial',
'github.com/tealeg/xlsx',
'github.com/thoj/go-ircevent',
'github.com/twinj/uuid',
'github.com/tylerb/graceful',
'github.com/ugorji/go',
'github.com/unrolled/render',
'github.com/urfave/cli',
'github.com/urfave/negroni',
'github.com/valyala/bytebufferpool',
'github.com/valyala/fasthttp',
'github.com/valyala/fasttemplate',
'github.com/valyala/tcplisten',
'github.com/vaughan0/go-ini',
'github.com/vishvananda/netlink',
'github.com/wendal/errors',
'github.com/xeipuuv/gojsonschema',
'github.com/yuin/gopher-lua',
'github.com/yvasiyarov/gorelic',
'github.com/zenazn/goji'}
In [41]:
And no package which was successfully scheduled, failed to reach the graph storage:
In [41]: scheduled_set - graph_set
Out[41]: set()