Skip to content

Instantly share code, notes, and snippets.

View lmolkova's full-sized avatar

Liudmila Molkova lmolkova

View GitHub Profile
@lmolkova
lmolkova / Program.cs
Last active May 23, 2018 01:01
Demo/prototype for preventing certain dependency calls based on the scope
using System;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.ApplicationInsights.Channel;
using Microsoft.ApplicationInsights.DependencyCollector;
using Microsoft.ApplicationInsights.Extensibility;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Queue;
namespace filtering
@lmolkova
lmolkova / proto_mappings.md
Last active October 29, 2018 22:02
Exceptions

Java Exception

Throwable property name Proto property Language-specific description
message message message that describes the current exception
cause inner_errors linked list of instances that caused the current exception
stackTrace stack_frames immediate frames on the call stack
suppressed ??? TODO exceptions that were suppressed in order to deliver this exception
throwable.getClass() error_info.error_type type of Throwable

C# Exception

@lmolkova
lmolkova / operation.bond
Last active May 7, 2019 20:00
OperationTelemetry proposal
import "Domain.bond"
namespace AI
[Description("An instance of operation represents completion of an logical operation.")]
struct OperationData
: Domain
{
[Description("Schema version")]
10: required int32 ver = 2;
@lmolkova
lmolkova / WorkerScope.cs
Last active July 19, 2019 18:52
Worker scopes
using System;
using System.Diagnostics;
using Microsoft.Extensions.Logging;
namespace Microsoft.Extensions.Hosting
{
public static class LoggerExtensions
{
public static IDisposable BeginWorkerScope(this ILogger logger)
{
@lmolkova
lmolkova / minimal_instrumentation.md
Last active August 15, 2023 20:41
EventHub and ServiceBus instrumentation

Short version

Context propagation protocol

application SDK MUST propagate trace parent part of the context (version, traceId, parentId and traceFlags) in the message in Diagnostic-Id application property following W3C trace-context format for traceparent encoding.

Also, propagate W3C trace context using W3C Trace Context Propagator from OpenTelemetry. It should populate traceparent and tracestate application properties. traceparent must match Diagnostic-Id value.

When extracting the context, first get W3C trace-context and then (if missing) read Diagnostic-Id.

@lmolkova
lmolkova / 0_Tracing_API.md
Last active April 3, 2024 07:08
OpenTelemetry Tracing APIs vs .NET Activity/DiagnosticSource

Tracing API Comparison

A distributed trace is a set of events, triggered as a result of a single logical operation, consolidated across various components of an application. A distributed trace contains events that cross process, network and security boundaries. A distributed trace may be initiated when someone presses a button to start an action on a website - in this example, the trace will represent calls made between the downstream services that handled the chain of requests initiated by this button being pressed.

Contract difference

OpenTelemetry Tracing API is a very strict contract that enables tracing signal (not debugging or profiling). This contract is the same for all kinds of libraries and tracing backends and includes several related concepts:

  • span creation, required and optional properties
  • sampling
  • exporting
  • noop behavior in absence of tracing implementaiton
  • extra information certain types spans should include (e.g. spans for http calls).
public class ClientIpHeaderTelemetryInitializerCopy : TelemetryInitializerBase
{
private const string HeaderNameDefault = "X-Forwarded-For";
private readonly char[] headerValuesSeparatorDefault = { ',' };
private char[] headerValueSeparators;
/// <summary>
/// Initializes a new instance of the <see cref="ClientIpHeaderTelemetryInitializer" /> class.
/// </summary>
@lmolkova
lmolkova / resourceidprocessor.cs
Last active September 18, 2020 21:49
resourceidprocessor
public class ResourceIdProcessor : ActivityProcessor
{
private readonly AsyncLocal<string> resourceId = new AsyncLocal<string>();
private readonly IHttpContextAccessor httpContextAccessor;
public ResourceIdProcessor(IHttpContextAccessor httpContextAccessor)
{
resourceId = null;
}

Problem

Same operation may be instrumented multiple times (manual + auto or library-native + auto) because of multiple reasons (below). This usually affects protocol instrumentation (HTTP/gRCP/etc) as they are auto-instrumented already and quite popular. It manifests as duplicated spans that fights for context injection on RPC calls, double performance impact and increase telemetry bills. Here are some cases when it happens:

  1. specialized instrumentation: native library instrumentaion can provide more rich data, better quality/performance + auto-instrumentation thats always on
  2. new library behavior (user manually instrumented and then new version brings auto-instrumentation)
  3. configuration error

While p1 is valid case, and p2/p3 are not, but we'd still would rather communicate the need to remove extra instrumentation instead of duplicating data.

Assumptions