Metrics v1 to v2 Client Migration

ComparingMetrics.md

With the migration to Metrics v2, there are some minor updates required by the client to make it compatible with both versions (deprecation will come later).

Metrics v1

To start, for v1, when you request "Give me metrics for Gate XYZ", you'll get something like this:

[
  {
      "id": {
          "type": "DeploymentFrequencyMetric", // The Metric Name
          "metricScope": "GATE", // The "depth" of the metric
          "valueStreamId": "806853778--DevOptics-Deliver-Value-Stream", // ValueStreamId (always present)
          "phaseId": "HylprF40Z7", // PhaseId (only if metric is a "PHASE" or "GATE" metric
          "gateId": "H16rFNCbm", // GateId (only if a metric is a "GATE" metric
          "version": 1
      },
      "timestamp": 1535483948985, // When the metric was calculated
      "devOpticsId": "cloudbees",
      "deploymentFrequency": 3.0 // The value,
      "errors": [ "valueStreamDeploymentFrequencyNoDataInRange", ... ] // Errors within the metric 
  },
  ... // more Metrics for the same scope
]

The result is a returned list of metrics with the similar IDs, a timestamp value, a specific metric-based value, namely deploymentFrequency above, and then a errors array containing a list of string errors that map directly to keys in the resource file in devoptics-ui.

Metrics v2

For Metrics v2, the same request will result in a single object, called a MetricsRollupBucket which will collapse a lot of the shared information across the above metrics. This object will contain a list of metrics, within the data property:

{
    "startTime": 1534338349617, // Data points used in calc had to occur here or after
    "endTime": 1535547949617, // Data points used in calc had to occur here or before
    "scope": { // Represents where the calc was compiled
        "devOpticsId": "cloudbees",
        "valueStreamId": "219174495--Acme-Online-SDLC",
        "phaseId": "build", // Not included in a VS-scoped metric
        "gateId": "checkout" // Not included in a Phase- or VS-scoped metric
    },
    "data": [{ // The metrics
        "@c": ".DeploymentFrequencyMetric", // ignore @c - used by the backend
        "value": 0.0, // The metric value
        "name": "deploymentFrequency" // the metric name
    }, {
        "@c": ".MeanTimeToRecoveryMetric",
        "errors": ["meanTimeToRecoveryNoTtrsAvailable"], // Errors look the same
        "name": "meanTimeToRecovery"
    }, {
        "@c": ".ChangeFailureRateMetric",
        "errors": ["changeFailureRateNoDataInRange"],
        "name": "changeFailureRate"
    }]
}

A MetricsRollupBucket will have a scope parameter outlining the scope that the metric was calculated for. A startTime and endTime representing the date range that the calculation was compiled across, and a data property containing a list of Metrics.

Each Metric will have a value property representing its individual value, an errors array following the same format as v1 (list of strings that map to resource keys), and a name property to identify the metric.

TheChanges.md

Background

Metrics v2 is currently behind a global disable flag (going away soon) as well as two feature flags - METRICS_V2_CALC and METRICS_V2_REST. The CALC flag is used to enable Metrics v2 processing for a given account. MetricRollupBuckets will be created but never seen by the client until REST is enabled, which then flips the switch and returns these buckets instead of the array of v1 metrics.

Changes

So there are two main changes that need to take place in the UI:

1. Update the fetch requests for metrics (See #fetchGateMetrics() and #fetchValueStreamMetrics() in Application.js to include the interval argument.
2. Using the feature flag METRICS_V2_REST, handle the result accordingly.

1. Interval

The current mechanism of requesting metrics is by hitting one of the following endpoints:

• /metrics/value-streams : Returns ALL ValueStream-scoped metrics in an object keyed by valueStreamId
• /metrics/value-streams/{valueStreamId} : Returns the ValueStream-scoped metric for the provided valueStreamId
• /metrics/value-streams/{valueStreamId}/phases/{phaseId}/gates/{gateId} : Returns the Gate-scoped metric for the parameters

(There is also a phase-scoped endpoint in between the last two, but its not used at this point)

For v1, you supplied the query parameters with the request:

• startTime
• endTime

These times were calculated client side based on the selected interval: 24 hour, 48 hour, 7 days, etc.

For v2, it only expects a single query parameter int with a value containing one of the predefined ranges:

• 1d
• 2d
• 7d
• 14d
• 30d
• 90d

Until we remove v1, all three of these parameters need to be supplied with the request. So whever the logic is to calculate the start and end time based on the current interval, we just need the interval also passed into the request with the correct value.

2. Handling the result

This is exactly what it is - using the feature flag as a condition, change the logic to parse the result based on the new format so that it can be used the same as v1. I think almost all of this work is in Application.java in the method #compileMetrics() which currently tries to normalize the v1 format to get it into a format like:

{
    "timestamp": 1535475759330,
    "errors": ["valueStreamMeanTimeToRecoveryInProgress", "valueStreamMeanTimeToRecoveryNoTtrsAvailable"], // not really needed
    // each metric keyed by the "old" naming style, with their value either being the value of the metric or the errors (if exists)
    "valueStreamMeanTimeToRecovery": ["valueStreamMeanTimeToRecoveryInProgress", "valueStreamMeanTimeToRecoveryNoTtrsAvailable"],
    "valueStreamChangeFailureRate": 0.5222664489257851,
    "valueStreamDeploymentFrequency": 7.315030616100197,
    "valueStreamMeanLeadTime": 8058364
}

I wouldn't try too hard to replicate what this method was doing, just attempt to get the v2 result into the above format, which shouldn't be difficult. For now, we can continue to use these old naming schemes to limit changes, but when we deprecate v1, it would be nice to clean all of that up.