Skip to content

Instantly share code, notes, and snippets.

@mgagliardo91
mgagliardo91 / GitHub_Ingester.md
Last active May 29, 2019 14:53
GitHub Ingester

Completed Work:

  • SDM-141: Create empty GitHub service that is responsible for pulling data in from github
  • SDM-142: GitHub service can be configured to connect to a github organisation
  • SDM-242: Create a "github app" for the SDM
  • SDM-144: GitHub service pushes the list of repos into the System of Record

Work Remaining:

  • SDM-???: [General] CI/CD for Flink Topologies
  • SDM-???: [Repositories] Handle deletions of repositories
@mgagliardo91
mgagliardo91 / ComparingMetrics.md
Last active August 29, 2018 13:26
Metrics v1 to v2 Client Migration

With the migration to Metrics v2, there are some minor updates required by the client to make it compatible with both versions (deprecation will come later).

Metrics v1

To start, for v1, when you request "Give me metrics for Gate XYZ", you'll get something like this:

[
  {
      "id": {
@mgagliardo91
mgagliardo91 / Analysis.md
Last active June 27, 2018 14:35
ANALYTICS-1932: Frequent EOFExceptions

ANALYTICS-1932

There seems to be two different EOFExceptions being thrown: java.io.EOFException and org.eclipse.jetty.io.EofException. The main file in play here (used by both client/server side) is CommitSpool.java

org.eclipse.jetty.io.EofException

This exception is being thrown to indicate an early, unexpected, EOF. The exception is most commonly thrown at the start of an ATH test against staging, but it can be found in prod with clients other than cloudbees. Following the stack trace, it seems like, while reading the input stream, a read is requested with a valid size determined from what is remaining to be read but the read() call throws an exception because the stream returned at -1 here.

I think that it would be safe to just handle