Disclaimer: This is very much a work in progress and I haven't worked through the details or fretted over naming. The goal is to get feedback on the basics and iterate. Perhaps we'll find it is fatally flawed, but we got to start somewhere :)
I think there are likely two different schools of thought for how people will want to use these APIs:
- Ad-hoc usage - These are folks who want to add some metrics to their code with minimal effort.
- Localized code changes only, ideally within a single class in a single file
- Add no new types
- Minimize LoC required, simple to follow best practice
- Easy to iterate and change in the future
- Easy to learn, not much abstraction
- Easily unit testable
- Not bothered by mixing telemetry schema/metadata definition and usage in their code
- De-coupled usage - These are folks who want a strong boundary between defining the instrumentation schema/metadata and its usage.
- Strongly typed API that defines available instrumentation
- Configuration of names/descriptions/units/versions/tags centralized and separated from usage
- Instrumentation can be easily registered and shared via DI
- Easily unit testable
- Accept a modest increase in abstractions/LoC/files touched in order to achieve the decoupling
My hope is to have a single API surface that can reasonably satisfy both use-cases depending on the pattern chosen, and that it shouldn't be too hard to switch between the patterns or mix and match within the same project as needs change.
public class FruitStoreController
{
Counter<int> _fruitSold;
ObservableGauge<int> _ordersOutstanding;
public FruitStoreController(IMeterFactory meterFactory, IOrderService orders)
{
Meter m = meterFactory.CreateMeter<FruitStoreController>(); // alternately you could specify a string name
_fruitSold = m.CreateCounter<int>("fruit-sold", "Amount of fruit sold at our store");
_ordersOutstanding = m.CreateObservableCounter<int>("orders-outstanding", orders.GetOutstanding, "Orders created and not yet shipped");
}
public IActionResult PlaceOrder(int fruitCount)
{
_fruitSold.Add(fruitCount);
...
}
}
This one consists of three parts: defining the instrumentation, registering the instrumentation, and using the instrumentation
class FruitStoreInstrumentation
{
public FruitStoreInstrumentation(IMeterFactory factory, IOrderService orders)
{
Meter m = meterFactory.CreateMeter("FruitCo.FruitStore"); // alternately you could specify a type name
// but if this instrumentation is shared across several
// components then there may not be a single type name which makes sense
FruitSold = m.CreateCounter<int>("fruit-sold", "Amount of fruit sold at our store");
OutstandingOrders = m.CreateObservableCounter<int>(orders-outstanding", orders.GetOutstanding, "Orders created and not yet shipped");
}
public Counter<int> FruitSold { get; init }
public ObservableCounter<int> OutstandingOrders { get; init; }
}
services.AddSingleton<FruitStoreInstrumentation>();
public class FruitStoreController
{
FruitStoreInstrumentation _instrumentation;
public FruitStoreController(FruitStoreInstrumentation instrumentation)
{
_instrumentation = instrumentation;
}
public IActionResult PlaceOrder(int fruitCount)
{
_instrumentation.FruitSold.Add(fruitCount);
...
}
}
[Fact]
public void OrderIncreasesFruitSold()
{
// arrange
using meterFactory = new MeterFactory();
IOrderService orderService = new FakeOrderService();
FruitStoreController controller = new FruitStoreController(meterFactory, orderService);
using InstrumentRecorder<int> recorder = new InstrumentRecorder(meterFactory, typeof(FruitStoreController), "fruits-sold");
// act
controller.PlaceOrder(4);
controller.PlaceOrder(18);
// assert
Assert.Equal(recorder.Measurements.Count, 2);
Assert.Equal(recorder.Measurements[0], 4);
Assert.Equal(recorder.Measurements[1], 18);
}
[Fact]
public void OutstandingOrdersReportsOrderServiceValue()
{
// arrange
using meterFactory = new MeterFactory();
IOrderService orderService = new FakeOrderService();
FruitStoreController controller = new FruitStoreController(meterFactory, orderService);
using InstrumentRecorder<int> recorder = new InstrumentRecorder(meterFactory, typeof(FruitStoreController), "fruits-sold");
// act
// this takes a snapshot of the current value
// we could do things that change the value and take multiple snapshots if necessary
recorder.SnapshotObservableInstrument();
// assert
Assert.Equal(recorder.Measurements.Count, 1);
Assert.Equal(recorder.Measurements[0], 49);
}
class FakeOrderService : IOrderService
{
public int GetOutstandingOrders() => 49;
}
Different overloads of the InstrumentRecorder constructor are possible depending on what information is easily available:
- factory + Meter name + instrument name
- Meter reference + instrument name
- Instrument reference
In the decoupled case where we directly expose the instrument as part of the API the instrument references are probably easier to use. With the APIs from .NET 6 it is possible to implement (2) and (3), but we'd need something new as part of this feature if we want to identify a Meter by factory+name.
- Create MeterFactory and IMeterFactory. The factory would need to cache any Meters it creates and Dispose them when the factory is disposed. It also needs to return pre-existing Meters when given the same name+version.
- We probably want an extension method on ServiceCollection to add the factory and call it by default from hosts, similar to logging.
- All the Meter.CreateXXX APIs need to start returning cached instruments when given the same argument values
- We need some support for a MeterListener to receive publish notifications for Meters from a certain factory rather than from all Meters
- The InstrumentRecorder either needs to be implemented in some assembly or it could be a small code snippet we document and people paste it into their tests.
This design does not add any interfaces for Meters or Instruments on the premise that they aren't needed to accomplish what devs want to do.
- For creating test mocks - the unit test above provides an easy alternative that requires no mocks.
- For capturing and transmitting metric measurements during production - use MeterListener to receive the data.
- For changing any other aspect of Meter/Instrument API behavior - I've never heard anyone ask and its probably by-design that you can't, but if anyone thinks there is an important scenario here being missed lmk.
Strongly typed instruments (dotnet/runtime#77516)
Either of the patterns could use more strongly typed instruments in place of the current ones.
Meter could be injected directly rather than MeterFactory if a good name can be automatically inferred. This eliminates needing to use a MeterFactory abstraction as part of the Meter/Instrument definition process, saves 1 LoC, and may improve naming consistency. However I am not confident that the enclosing type name will always be a good name so I expect the documentation will still need to show users how to use the MeterFactory to get a different name when needed. Subjectively I think this represents a modest improvement to the overall scenario but others felt this difference represented a more substantial improvement.
I have no opposition to either of these DI features but nor do the patterns above feel in critical need of them.
Today if you want to view the telemetry from outside the process then a FruitStore meter from DI container A looks identical to one in container B. Two approaches that might useful:
- Similar to the InstrumentRecorder that was bound to a particular factory during testing, any telemetry library could scope itself to the Meters that came from one factory rather than all Meters in the process. This lets folks set up distinct pipelines per-container and similar to what OTel does for logging.
- Customizing the IMeterFactory could allow tagging Meters with some container specific data that allows them to disambiguated later. This would also rely on Meters themselves gaining some functionality to store tags. Today tags are only defined on individual measurements, rather than the broader instrument or meter scopes.
I deliberately didn't name the FruitStoreInstrumentation
using the word Metrics. I think it would be perfectly reasonable for developers
to put other instrumentation objects such as ILogger, ActivitySource, DiagnosticSource, etc in the same containing type. In the ad-hoc
case I expect other instrumentation objects would be stored as fields in the controller (or whatever class is producing the data).
From all your back and forth, I see the discussion appears to be assuming that having some sort of "instrumentation" container/wrapper is going to be the way to use instrumentation... which seems odd to me: that's not what I'm doing at all, for example.
You see, instead of:
I'd rather have:
The additional indirection of having an "instrumentation" container class that holds the instruments for me just adds nothing semantically. It seems that, once again, you are introducing boilerplate and additional abstractions and indirections to solve the container-based problem that "there is no mechanism to inject different raw
Counter
types into certain specific services". I have to once again go back to the wayHttpClient
works here. You don't see this type of thing:And the main reason for not seeing this is that the indirection is unwanted... the controller (or any other consumer) shouldn't have to be forced to known about a certain indirection just to get to what it needs: an already configured
HttpClient
. Which is why we do this instead:IMHO, the exact same needs to happen with instrumentation. I want the class that's going to use the instruments to:
Your decoupled version solves 2, but violates 1. My proposal below solves both.
Now... what is the main difference between
HttpClient
andCounter/ActivitySource/Histogram/Observable...
usage? Usually, a class only depends on a singleHttpClient
, while it is fairly normal for a service to need to handle multiple instruments at once. This, of course, makes it that much harder to create a similar API as the one forHttpClient
, because it only accepts a single client per service.And this is why I've been mentioning the container itself so much: if the container provided a away to associate arbitrary configurable types to underlying services, you could configure multiple instruments and associate them with any given type similar to what
AddHttpClient
does today.I believe an API that promotes configuration at the aggregation root level (like
AddHttpClient
) is vastly superior to something where I suddenly have to inject the "configurator" into my class and manually configure the instruments in the constructor: we don't do that forHttpClient
, and we shouldn't have to do it for any other type of dependency.My ideal API would probably look something like:
Which, again, would be a generalized but extensible version of what
AddHttpClient
does:WithCounter
andWithActivitySource
above would be extensions over some raw interface in the container that allows arbitrary association, such as:Which would tie back to the APIs I mentioned from Autofac, Structuremap, Unity, etc.