WIP: How to use experimental OpenTelemetry Logs for Ruby

README.md

WIP: How to use experimental OpenTelemetry logs for Ruby

Overview

This gist demonstrates how to augment a Ruby application to use an experimental implementation of OpenTelemetry Logs for Ruby.

PRs with source code for the experimental gems can be found at:

• kaylareopelle/opentelemetry-ruby
• kaylareopelle/opentelemetry-ruby-contrib

The branches used in the example Gemfile are experimental and subject to change.

Though there's a working solution in these branches, we want to get the code reviewed, approved, and merged to the opentelemetry-ruby and opentelemetry-ruby-contrib repositories. Once we're at that stage, opentelemetry-ruby can release experimental versions of opentelemetry-logs-api, opentelemetry-logs-sdk, and opentelemetry-exporter-otlp-logs.

I try to keep the branches up to date with the latest code on main, so you shouldn't miss any new features or fixes. In addition, I'll do my best to keep them functioning. If something's broken or you have some feedback, please comment on the PRs linked above.

You can find a demo using a Rails 7.0 application here: https://github.com/kaylareopelle/otel_ruby_logs_demo

Setup

To start using experimental OpenTelemetry logs for Ruby in your application, you'll need to update two files in your application:

Gemfile

Replace your current installation of opentelemetry-sdk, opentelemetry-instrumentation-all, and opentelemetry-exporter-otlp with the gems in the example Gemfile. Even if you're not explicitly installing opentelemetry-api and some of the other gems in this Gemfile, you'll need to do so in order to get all the pieces to work. If you're not using opentelemetry-instrumentation-all, just add opentelemetry-instrumentation-logger to your Gemfile.

opentelemetry.rb or whatever you call your OpenTelemetry initializer/configuration file

Require the gems needed for logs to work and adjust the exporter settings to suit your needs. You can export via OTLP or via the console. The example is set up to export to the console. ENV['OTEL_LOGS_EXPORTER'] can be set to 'otlp', 'console', or 'otlp,console'. Your settings for logs export should not conflict with settings for trace exports.

What you should see

The example opentelemetry.rb file will export logs to the console. You should see something like this:

#<struct OpenTelemetry::SDK::Logs::LogRecordData
 timestamp=2024-04-19 15:06:50.118496 -0700,
 observed_timestamp=2024-04-19 15:06:50.118496 -0700,
 trace_id="\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00",
 span_id="\x00\x00\x00\x00\x00\x00\x00\x00",
 trace_flags=#<OpenTelemetry::Trace::TraceFlags:0x0000000106a73580 @flags=0>,
 severity_text="INFO",
 severity_number=9,
 body="Instrumentation: OpenTelemetry::Instrumentation::Logger was successfully installed with the following options {}",
 resource=
  #<OpenTelemetry::SDK::Resources::Resource:0x0000000107b9c3c0
   @attributes=
    {"service.name"=>"Logs Demo - OTel Ruby Agent",
     "process.pid"=>91454,
     "process.command"=>"bin/rails",
     "process.runtime.name"=>"ruby",
     "process.runtime.version"=>"3.2.2",
     "process.runtime.description"=>"ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-darwin22]",
     "telemetry.sdk.name"=>"opentelemetry",
     "telemetry.sdk.language"=>"ruby",
     "telemetry.sdk.version"=>"1.4.1"}>,
 instrumentation_scope=#<struct OpenTelemetry::SDK::InstrumentationScope name="opentelemetry-instrumentation-logger", version="0.0.0">,
 attributes=nil,
 total_recorded_attributes=0>

The Ruby Logger bridge/instrumentation should automatically bridge any logs that pass through Ruby's built-in Logger library. The Logger library is leveraged by many Ruby gems, including Rails. The instrumentation is not currently compatible with Rails 7.1+. This example does not provide a bridge for lograge, semantic_logger, logstash, or other logging gems.

The example opentelemetry.rb file also instantiates an app-specific logger provider, APP_LOGGER. This can be used to emit your own OTel logs without the bridge.

Gemfile

source 'https://rubygems.org'

# Main OpenTelemetry gems with logging-related enhancements
gem 'opentelemetry-api',
  git: 'https://github.com/kaylareopelle/opentelemetry-ruby',
  branch: 'log-record-processor3',
  glob: 'api/*.gemspec'

gem 'opentelemetry-sdk',
  git: 'https://github.com/kaylareopelle/opentelemetry-ruby',
  branch: 'log-record-processor3',
  glob: 'sdk/*.gemspec'

# Unreleased, logs-specific gems
gem 'opentelemetry-logs-api',
  git: 'https://github.com/kaylareopelle/opentelemetry-ruby',
  branch: 'log-record-processor3',
  glob: 'logs_api/*.gemspec'

gem 'opentelemetry-logs-sdk',
  git: 'https://github.com/kaylareopelle/opentelemetry-ruby',
  branch: 'log-record-processor3',
  glob: 'logs_sdk/*.gemspec'

# OTLP Exporters with logging enhancements
gem 'opentelemetry-exporter-otlp',
  git: 'https://github.com/kaylareopelle/opentelemetry-ruby',
  branch: 'log-record-processor3',
  glob: 'exporter/otlp/*.gemspec'

gem 'opentelemetry-exporter-otlp-logs',
  git: 'https://github.com/kaylareopelle/opentelemetry-ruby',
  branch: 'log-record-processor3',
  glob: 'exporter/otlp-logs/*.gemspec'

# Instrumentation and Ruby Logger Bridge
gem 'opentelemetry-instrumentation-all',
  git: 'https://github.com/kaylareopelle/opentelemetry-ruby-contrib',
  branch: 'logger-instrumentation',
  glob: 'instrumentation/all/*.gemspec'

gem 'opentelemetry-instrumentation-logger',
  git: 'https://github.com/kaylareopelle/opentelemetry-ruby-contrib',
  branch: 'logger-instrumentation',
  glob: 'instrumentation/logger/*.gemspec'

opentelemetry.rb

# frozen_string_literal: true

require 'opentelemetry/sdk'
require 'opentelemetry/exporter/otlp'
require 'opentelemetry/exporter/otlp_logs'
require 'opentelemetry/instrumentation/all'
# if you're not using instrumentation-all, uncomment the require below to get access to the Ruby logger bridge/instrumentation
# require 'opentelemetry/instrumentation/logger' 

ENV['OTEL_LOGS_EXPORTER'] = 'console' # logs can also be exported via otlp

OpenTelemetry::SDK.configure do |c|
  # c.use_all # enables all instrumentation!
  c.use('OpenTelemetry::Instrumentation::Logger') # alternatively, use just the logger
end

# Instantiate an app logger if you'd like to manually emit logs without using a logs bridge/instrumentation.
APP_LOGGER = OpenTelemetry.logger_provider.logger(name: 'APP_LOGGER')
# Usage example: 
# APP_LOGGER.on_emit(
#   body: 'Counselor Deanna Troi, personal log, stardate 44805.3. My mother is on board.', 
#   severity_text: 'DEBUG', 
#   severity_number: 0, 
#   attributes: { 'hi' => 'mom' }
# )