Example Custom OpenTelemetry Collector Build

.gitignore

build

README.md

Custom OpenTelemetry Collector

Overview

If the OpenTelemetry project-provided builds of the core collector or the contrib collector are not suitable or complete for use in your environment then you can easily build a custom collector tailored to your needs.

This example uses the ocb utility to build a custom OpenTelemetry Collector.

builder-config

The ocb utility requires a configuration file. This configuration file acts as the manifest and defines the receivers, processors, extensions, and exporters that you want included in your custom Collector.

Anything not in the core Collector will need to be added to the builder configuration, and with recent changes you may be surprised how minimal the core has become.

An example builder config file is provided with builder-config.yaml. Detailed documentation on this file is available here.

Building

The ocb must be installed on the machine running the build. The latest version (0.52.0) of the utility can be obtained from here.

The provided Makefile's default target will build a custom Collector as described by the builder configuration manifest.

Docker image

The provided Makefile can build a minimal Docker image with the docker-build target. This target does not require that the ocb utility is installed on the host as the first stage of the Dockerfile obtains the version needed.

builder-config.yaml

---
dist:
  module: github.com/open-telemetry/opentelemetry-collector
  description: "My Custom OpenTelemetry Collector"
  include_core: true
  otelcol_version: "0.50.0"
  output_path: build
receivers:
  - import: go.opentelemetry.io/collector/receiver/otlpreceiver
    gomod: go.opentelemetry.io/collector v0.50.0
  - gomod: "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver v0.50.0"
  - gomod: "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/kubeletstatsreceiver v0.50.0"
  - gomod: "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver v0.50.0"
  - gomod: "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/statsdreceiver v0.50.0"
extensions:
  - import: go.opentelemetry.io/collector/extension/zpagesextension
    gomod: go.opentelemetry.io/collector v0.50.0
  - import: go.opentelemetry.io/collector/extension/ballastextension
    gomod: go.opentelemetry.io/collector v0.50.0
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/extension/healthcheckextension v0.50.0
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/extension/pprofextension v0.50.0
exporters:
  - import: go.opentelemetry.io/collector/exporter/loggingexporter
    gomod: go.opentelemetry.io/collector v0.50.0
  - import: go.opentelemetry.io/collector/exporter/otlpexporter
    gomod: go.opentelemetry.io/collector v0.50.0
  - import: go.opentelemetry.io/collector/exporter/otlphttpexporter
    gomod: go.opentelemetry.io/collector v0.50.0
processors:
  - import: go.opentelemetry.io/collector/processor/batchprocessor
    gomod: go.opentelemetry.io/collector v0.50.0
  - import: go.opentelemetry.io/collector/processor/memorylimiterprocessor
    gomod: go.opentelemetry.io/collector v0.50.0
  - gomod: "github.com/open-telemetry/opentelemetry-collector-contrib/processor/filterprocessor v0.50.0"
  - gomod: "github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor v0.50.0"
  - gomod: "github.com/open-telemetry/opentelemetry-collector-contrib/processor/metricstransformprocessor v0.50.0"
  - gomod: "github.com/honeycombio/opentelemetry-collector-configs/timestampprocessor v1.4.0"

collector-config.yaml

---
receivers:
  hostmetrics:
    collection_interval: "$OTEL_METRICS_INTERVAL"
    scrapers:
      cpu:
      disk:
      filesystem:
      load:
      memory:
      network:
      paging:
  otlp:
    protocols:
      grpc:
      http:

processors:
  # See: https://github.com/honeycombio/opentelemetry-collector-configs/README.md
  timestamp:
    round_to_nearest: 30s
  batch:
  # See: https://github.com/open-telemetry/opentelemetry-collector/blob/main/processor/memorylimiter/README.md
  memory_limiter:
    check_interval: 5s
    limit_percentage: 75
    spike_limit_percentage: 25

exporters:
  otlphttp/honeycomb-traces:
    endpoint: "https://api.honeycomb.io"
    headers:
      "x-honeycomb-team": "$HONEYCOMB_API_KEY"
      "x-honeycomb-dataset": "$HONEYCOMB_DATASET"
  otlp/honeycomb-metrics:
    endpoint: "api.honeycomb.io:443"
    headers:
      "x-honeycomb-team": "$HONEYCOMB_API_KEY"
      "x-honeycomb-dataset": "$HONEYCOMB_METRICS_DATASET"
  logging:
    loglevel: warn

extensions:
  memory_ballast:
    size_in_percentage: 35
  health_check:
    endpoint: :13133

service:
  extensions: [health_check]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlphttp/honeycomb-traces, logging]
    metrics:
      receivers: [otlp, hostmetrics]
      processors: [memory_limiter, timestamp, batch]
      exporters: [otlp/honeycomb-metrics, logging]

Dockerfile

FROM golang:1.17-alpine3.14 as base

ENV OTC_BUILDER_VERSION=0.50.0
ARG OTC_BUILD_VERSION=0.50.0+custom

RUN apk add ca-certificates

# download the ocb utility
RUN export GOOS="$(go env GOOS)" GOARCH="$(go env GOARCH)" && \
  wget https://github.com/open-telemetry/opentelemetry-collector/releases/download/v${OTC_BUILDER_VERSION}/ocb_${OTC_BUILDER_VERSION}_${GOOS}_${GOARCH} \
  -O /usr/local/bin/ocb && \
  chmod 755 /usr/local/bin/ocb

COPY builder-config.yaml /

# build our custom collector using our provided builder-config
RUN export CGO_ENABLED=0 && /usr/local/bin/ocb \
  --config /builder-config.yaml \
  --name /otelcol-custom \
  --version ${OTC_BUILD_VERSION}

FROM scratch

ARG USER_UID=10001
USER ${USER_UID}

COPY --from=base /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
COPY --from=base /otelcol-custom /
COPY collector-config.yaml /etc/otel/config.yaml
ENTRYPOINT ["/otelcol-custom"]
CMD ["--config", "/etc/otel/config.yaml"]
EXPOSE 4317 4318 13333

Makefile

VERSION?=1.0.0

.PHONY: all build
all: build

build:
	ocb \
		--config ./builder-config.yaml \
		--name otelcol-custom \
		--version $(VERSION)

docker-build:
	docker build . -t otelcol-custom:$(VERSION) --build-arg OTC_BUILD_VERSION=$(VERSION)

clean:
	rm -f build/*