lmolkova/distro_arch_board.md

## distro_arch_board.md

      
    Raw
  

              distro_arch_board.md
            
          
Scope
Hero scenarios

Python

Distro

Configuration

Enable AAD
Configure sampling
Tune OpenTelemetry configuration
Enable 3rd party instrumentations or report custom traces and metrics


Exporter

Traces, metrics, and logs exporters


.NET

Distro

Configuration

Enable AAD
Configure sampling
Tune OpenTelemetry instrumentation libraries
Enable 3rd party instrumentations or report custom traces and metrics


Exporter

Traces, metrics, and logs exporters


JS

Distro

Configuration
Enable AAD
Configure sampling
Tune OpenTelemetry configuration
Enable 3rd party instrumentations or report custom traces and metrics


Exporter

Traces and metrics exporters


Java

Distro
Exporter


API views

Scope

GA review for

Azure Monitor OpenTelemetry disto in .NET, Python, and JS.

desired GA date Aug 18


Azure Monitor OpenTelemetry Exporters in .NET, Python, JS and Java

desired GA date: .NET - Aug 18, Java - by the end of 2023, Python, JS - TBD


Hero scenarios

There are two different ways to collect and send OpenTelemetry data to Azure Monitor:

distribution (aka distro) - a customized version of OTel with predefined set of instrumentation and configurations. Minimal configuration is required from the user side.
exporter - user picks instrumentations, configures OpenTelemetry directly, and enables specific exporter

We'll take a look at both and see how to apply basic configuration to them.
See also Application Insights learn,microsoft.com articles for OpenTelemetry.
Python

Distro

pip install azure-monitor-opentelemetry --pre
Configure auto-collection for traces, metrics, and logs.
from azure.monitor.opentelemetry import configure_azure_monitor

configure_azure_monitor()
This configuration enables collection of the following data:

HTTP server-side: Django, Flask, FastAPI (traces, metrics)
HTTP client side: requests, urllib, urllib3 (traces, metrics)
DB: psycopg2 (traces, metrics)
Azure SDKs (traces)

By default, connection string is obtained from APPLICATIONINSIGHTS_CONNECTION_STRING environment variable.
Configuration

Users can configure the distro using environment variables and/or configuration options.
Enable AAD

from azure.monitor.opentelemetry import configure_azure_monitor
configure_azure_monitor(
    credential=ManagedIdentityCredential(),
)
Configure sampling

export OTEL_TRACES_SAMPLER_ARG=0.1
from azure.monitor.opentelemetry import configure_azure_monitor
configure_azure_monitor()
Can also be done with OTEL_TRACES_SAMPLER_ARG environment variable.
Tune OpenTelemetry configuration

pip install opentelemetry-exporter-otlp
from azure.monitor.opentelemetry import configure_azure_monitor
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace.export import BatchSpanProcessor

from azure.monitor.opentelemetry import configure_azure_monitor

configure_azure_monitor()
otlp_exporter = OTLPSpanExporter(endpoint="http://localhost:4317")
span_processor = BatchSpanProcessor(otlp_exporter)
trace.get_tracer_provider().add_span_processor(span_processor)
Enable 3rd party instrumentations or report custom traces and metrics

pip install opentelemetry-instrumentation-sqlalchemy --pre
from azure.monitor.opentelemetry import configure_azure_monitor
from opentelemetry.instrumentation.sqlalchemy import SQLAlchemyInstrumentor
from logging import getLogger

configure_azure_monitor()
SQLAlchemyInstrumentor().instrument(...)

tracer = trace.get_tracer(__name__)
logger = getLogger(__name__)
with tracer.start_as_current_span("test"):
    logger.info("Correlated info log")
Logging is supported through logging handler.
See also distro README
Following configuration options are supported through environment variables (defined by OpenTelemetry) and/or kwargs:

Connection String
Cloud Role Name
Cloud Role Instance
Sample rate
AAD Auth
Offline Storage
Tracing export interval
Logging export interval
Disabling instrumentations

Exporter

Users who want to have full control over telemetry collection configuration, can configure OpenTelemetry and just export their telemetry to Azure Monitor.
pip install azure-monitor-opentelemetry-exporter --pre
pip install opentelemetry-instrumentation-requests --pre
from opentelemetry import trace
from opentelemetry.instrumentation.requests import RequestInstrumentor
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from azure.monitor.opentelemetry.exporter import (
    ApplicationInsightsSampler,
    AzureMonitorTraceExporter,
)

sampler = ApplicationInsightsSampler(0.75)
trace.set_tracer_provider(TracerProvider(sampler=sampler))

exporter = AzureMonitorTraceExporter(
    connection_string=connection-string
)
span_processor = BatchSpanProcessor(exporter)
trace.get_tracer_provider().add_span_processor(span_processor)

RequestInstrumentor.instrument()
Traces, metrics, and logs exporters

from azure.monitor.opentelemetry.exporter import (
	AzureMonitorTraceExporter,
	AzureMonitorMetricExporter,
	AzureMonitorLogExporter,
)

trace_exporter = AzureMonitorTraceExporter()
metric_exporter = AzureMonitorMetricExporter()
log_exporter = AzureMonitorLogExporter()
See also exporter README
.NET

Distro

dotnet add package Azure.Monitor.OpenTelemetry.AspNetCore --prerelease
Configure auto-collection for traces, metrics, and logs:
using Azure.Monitor.OpenTelemetry.AspNetCore;

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddOpenTelemetry().UseAzureMonitor();
//...
This configuration enables collection of the following data:

HTTP client and ASP.NET Core (all signals)
DB: SQL client (traces)
Azure SDK (traces, still requires user to opt in with feature flag)

By default, connection string is obtained from APPLICATIONINSIGHTS_CONNECTION_STRING environment variable - staying consistent with Track 1 SDK.
Configuration

Users can configure the distro explicitly in-code, via Microsoft.Extensions.Configuration, and with a few environment variables.
Enable AAD

builder.Services.AddOpenTelemetry().UseAzureMonitor(o =>
{
    o.ConnectionString = "InstrumentationKey=00000000-0000-0000-0000-000000000000";
    o.Credential = new DefaultAzureCredential();
});
Configure sampling

builder.Services.AddOpenTelemetry().UseAzureMonitor(o =>
{
    o.SamplingRatio = 0.5F;
});
Tune OpenTelemetry instrumentation libraries

builder.Services.AddOpenTelemetry().UseAzureMonitor();
builder.Services.Configure<AspNetCoreInstrumentationOptions>(options =>
{
    options.RecordException = true;
    options.Filter = (httpContext) =>
    {
        // only collect telemetry about HTTP GET requests
        return HttpMethods.IsGet(httpContext.Request.Method);
    };
});
Note that AspNetCoreInstrumentationOptions is defined in OpenTelemetry.Instrumentation.AspNetCore package.
Enable 3rd party instrumentations or report custom traces and metrics

dotnet add package OpenTelemetry.Instrumentation.GrpcNetClient --prerelease
builder.Services.AddOpenTelemetry().UseAzureMonitor();
builder.Services.ConfigureOpenTelemetryTracerProvider((sp, builder) =>
    builder.AddGrpcClientInstrumentation()
           .AddSource("MyCompany.MyProduct.MyLibrary"));
builder.Services.ConfigureOpenTelemetryMeterProvider((sp, builder) => builder.AddMeter("MyCompany.MyProduct.MyLibrary"));
Logging is supported through Microsoft.Extensions.Logging, so all the configuration is done with corresponding options and outside of the distro.
See also distro README
Exporter

Users who don't use ASP.NET Core or who want to have full control over telemetry collection configuration, can configure OpenTelemetry and just export their telemetry to Azure Monitor.
dotnet add package Azure.Monitor.OpenTelemetry.Exporter --prerelease
dotnet add package OpenTelemetry.Instrumentation.Http  --prerelease
dotnet add package OpenTelemetry.Exporter.Console
var resourceAttributes = new Dictionary<string, object>
{
    { "service.name", "my-service" },
    { "service.namespace", "my-namespace" },
    { "service.instance.id", "my-instance" },
};

var resourceBuilder = ResourceBuilder.CreateDefault().AddAttributes(resourceAttributes);

_tracerProvider = Sdk.CreateTracerProviderBuilder()
                .SetResourceBuilder(resourceBuilder)
                .AddSource("my source")
                .AddHttpClientInstrumentation()
                .AddProcessor(new ActivityFilteringProcessor())
                .AddConsoleExporter()
                .AddAzureMonitorTraceExporter(o => { o.ConnectionString = connectionString; o.SamplingRatio = 1.0F; }, credential)
                .Build();
Only AddAzureMonitorTraceExporter and it's parameters are part of Azure Monitor Exporter, the rest of this code is vanilla OpenTelemetry configuration.
Traces, metrics, and logs exporters

Sdk.CreateTracerProviderBuilder()
    .AddAzureMonitorTraceExporter()
    .Build();

Sdk.CreateMeterProviderBuilder()
    .AddAzureMonitorMetricExporter()
    .Build();

LoggerFactory.Create(builder =>
{
    builder.AddOpenTelemetry(options =>
    {
        options.AddAzureMonitorLogExporter();
    });
});
See also exporter README
JS

Distro

npm install @azure/monitor-opentelemetry
const { AzureMonitorOpenTelemetryClient, AzureMonitorOpenTelemetryOptions } = require("@azure/monitor-opentelemetry");

const options: AzureMonitorOpenTelemetryOptions = {
  azureMonitorExporterConfig: {},
}
const azureMonitorClient = new AzureMonitorOpenTelemetryClient(options);
This configuration enables collection of the following data:

HTTP: Node.js http/https (traces, metrics)
DB: mongodb, mysql, redis, postgres  (traces)
Azure SDKs (traces)
standard cpu/mem usage counters

By default, connection string is obtained from APPLICATIONINSIGHTS_CONNECTION_STRING environment variable.
Configuration

Users can configure distro through explicit configuration options or
configuration file.
Enable AAD

const { ApplicationInsightsClient, ApplicationInsightsConfig } = require("applicationinsights");
const { ManagedIdentityCredential } = require("@azure/identity");

const credential = new ManagedIdentityCredential();

const config = new ApplicationInsightsConfig();
config.azureMonitorExporterConfig.aadTokenCredential = credential;
const appInsights = new ApplicationInsightsClient(config);
Configure sampling

const { ApplicationInsightsClient, ApplicationInsightsConfig } = require("applicationinsights");
const config = new ApplicationInsightsConfig();
config.samplingRatio = 0.75;
const appInsights = new ApplicationInsightsClient(config);
Tune OpenTelemetry configuration

const { AzureMonitorOpenTelemetryClient } = require("@azure/monitor-opentelemetry");
const { ReadableSpan, SpanProcessor } = require("@opentelemetry/sdk-trace-base");

const azureMonitorClient = new AzureMonitorOpenTelemetryClient();

class SpanEnrichingProcessor implements SpanProcessor{
    //...
    onEnd(span: ReadableSpan){
        span.attributes["CustomDimension1"] = "value1";
        span.attributes["CustomDimension2"] = "value2";
    }
}

azureMonitorClient.getTracerProvider().addSpanProcessor(new SpanEnrichingProcessor());
Enable 3rd party instrumentations or report custom traces and metrics

npm install @opentelemetry/instrumentation-express
const { AzureMonitorOpenTelemetryClient } = require("@azure/monitor-opentelemetry");
const { registerInstrumentations } = require("@opentelemetry/instrumentation");
const { ExpressInstrumentation } = require('@opentelemetry/instrumentation-express');

const azureMonitorClient = new AzureMonitorOpenTelemetryClient();
const instrumentations = [
    new ExpressInstrumentation(),
];
registerInstrumentations({
    tracerProvider: azureMonitorClient.getTracerProvider(),
    meterProvider: azureMonitorClient.getMeterProvider(),
    instrumentations: instrumentations,
});
See also distro README
Exporter

Users who want to have full control over telemetry collection configuration, can configure OpenTelemetry and just export their telemetry to Azure Monitor.
npm install @azure/monitor-opentelemetry
npm install @opentelemetry/instrumentation-http
const { BatchSpanProcessor } = require("@opentelemetry/sdk-trace-base");
const { NodeTracerProvider } = require("@opentelemetry/sdk-trace-node");
const { registerInstrumentations } = require('@opentelemetry/instrumentation');
const { HttpInstrumentation } = require('@opentelemetry/instrumentation-http');
const { ApplicationInsightsSampler, AzureMonitorTraceExporter} = require("@azure/monitor-opentelemetry-exporter");
const { Resource } = require("@opentelemetry/resources");

const aiSampler = new ApplicationInsightsSampler(0.75);
const provider = new NodeTracerProvider({
  sampler: aiSampler,
  resource: new Resource({
    ["service.name"]: "basic-service",
  }),
});

registerInstrumentations({
  instrumentations: [
    new HttpInstrumentation()
  ],
});

const trace_exporter = new AzureMonitorTraceExporter();
trace_provider.addSpanProcessor(
  new BatchSpanProcessor(exporter, {
    bufferTimeout: 15000,
    bufferSize: 1000
  })
);

provider.register();
Traces and metrics exporters

const { AzureMonitorMetricExporter, AzureMonitorTraceExporter} = require("@azure/monitor-opentelemetry-exporter");

const trace_exporter = new AzureMonitorTraceExporter();
const metric_exporter = new AzureMonitorMetricExporter();
See also exporter README
Java

Distro

Not in the scope of this review.
java -javaagent:applicationinsights-agent-3.4.15.jar -jar ....
Supports configuration with OTEL_* and APPLICATIONINSIGHT_* environment variables and configuration file.
AAD is supported with `AZURE_*`` identity env vars.
Auto-collects a variety of signals from Java libraries: HTTP, GRPC, databases, messaging systems, Azure SDKs, common logging libraries, common metrics, etc.
Supports custom telemetry collection - see distro docs.
Exporter

<dependency>
  <groupId>com.azure</groupId>
  <artifactId>azure-monitor-opentelemetry-exporter</artifactId>
</dependency>
import io.opentelemetry.sdk.OpenTelemetrySdk;
import io.opentelemetry.sdk.metrics.export.MetricExporter;
import io.opentelemetry.sdk.metrics.export.PeriodicMetricReader;
import io.opentelemetry.sdk.metrics.SdkMeterProvider;
import io.opentelemetry.sdk.trace.SdkTracerProvider;
import io.opentelemetry.sdk.trace.data.SpanData;
import io.opentelemetry.sdk.trace.export.SimpleSpanProcessor;
import io.opentelemetry.sdk.trace.export.SpanExporter;

import com.azure.monitor.opentelemetry.exporter.*;

SpanExporter traceExporter = new AzureMonitorExporterBuilder()
    .buildTraceExporter();

MetricExporter metricExporter = new AzureMonitorExporterBuilder()
    .buildMetricExporter();

// otel config
SdkTracerProvider tracerProvider = SdkTracerProvider.builder()
    .addSpanProcessor(SimpleSpanProcessor.create(exporter))
    .build();
PeriodicMetricReader periodicMetricReader = PeriodicMetricReader
    .builder(exporter)
    .build();
SdkMeterProvider sdkMeterProvider = SdkMeterProvider.builder()
    .registerMetricReader(periodicMetricReader)
    .build();

OpenTelemetry openTelemetry = OpenTelemetrySdk.builder()
    .setTracerProvider(tracerProvider)
    .setMeterProvider(sdkMeterProvider)
    .buildAndRegisterGlobal();
See also exporter README
API views


.NET

Distro
Exporter


Python

Distro
Exporter


Javascript

Distro
Exporter


Java

Exporter