Skip to content

Instantly share code, notes, and snippets.

@mgagliardo91

mgagliardo91/GitHub_Ingester.md

Last active May 29, 2019 14:53
Show Gist options
  • Save mgagliardo91/6809bb73eac960329e0ff3229630af19 to your computer and use it in GitHub Desktop.
Save mgagliardo91/6809bb73eac960329e0ff3229630af19 to your computer and use it in GitHub Desktop.
Download ZIP
GitHub Ingester
Raw
GitHub_Ingester.md

Completed Work:

  • SDM-141: Create empty GitHub service that is responsible for pulling data in from github
  • SDM-142: GitHub service can be configured to connect to a github organisation
  • SDM-242: Create a "github app" for the SDM
  • SDM-144: GitHub service pushes the list of repos into the System of Record

Work Remaining:

  • SDM-???: [General] CI/CD for Flink Topologies
  • SDM-???: [Repositories] Handle deletions of repositories
  • SDM-???: [Repositories] Handle updates/duplicates (upsert)
  • SDM-???: [Repositories] Add logic to manage rate limits
  • SDM-???: [General] Fetch more details about a repository
  • SDM-???: [Repositories] Reconcilliation (fetch and compare)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment