How to build a aws-otel-lambda/opentelemetry-lambda python lambda layer

Build-aws-otel-lambda.md

How to build a aws-otel-lambda / opentelemetry-lambda python lambda layer

Setup a local repo of the aws-otel-lambda

1. Fork or clone aws-observability/aws-otel-lambda
2. Cd into your local copy of that repo
3. Do the git submodule thing to link in the associate branch of open-telemetry/opentelemetry-lambda

git submodule update --init --recursive

Build the layer

• You will need to be setup to build docker containers in your local dev machine.

The following will:

• Build the opentelemetry-lambda/collector extension
• Build the open-telemetry/opentelemetry-lambda/python/src
• Build the sample-apps in open-telemetry/opentelemetry-lambda
• Add in the opentelemetry-sdk-extension-aws to the open-telemetry/opentelemetry-lambda
• Inject its own otel-instrument layer startup script in front of the standard otel-instrument layer startup script
• Zip the startup script, collector extension and the open-telemetry/opentelemetry-lambda/python build into a layer.zip in the same format as aws-otel-python-<amd64|arm64>-ver-1-11-1 for being a lambda layer.

Changing the versions of the opentelemetry libraries used in the layer

• For the collector

  • opentelemetry-lambda/collector/go.mod
    • I didn't try changing this

• For everything else (except the aws-xray stuff)

  • opentelemetry-lambda/python/src/otel/otel_sdk/requirements.txt
  • opentelemetry-lambda/python/src/otel/otel_sdk/requirements-nodeps.txt
  • I bumped these up to v1.12.0rc2 / 0.32b0

You might want to first try it without changing anything and make sure it works end to end.

Assuming you are at the top level directory of aws-otel-lambda

cd python
./build.sh

After a successful run, all the build results will be in aws-otel-lambda/open-telemetry/opentelemetry-lambda/python/src/build (where aws-otel-lambda is the top level of the repo directory).

It should contain the following file / directories

• collector-config
  • The default collector config
• extensions
  • The collector extension / executable
• otel-instrument
  • The AWS lambda startup script override
• otel-instrument-upstream-lambda
  • The original lambda startup script which gets called by the override script
• python
  • The build results of aws-otel-lambda/opentelemetry-lambda/python/src/build.sh
• layer.zip
  • All of the above zipped up ready to be made into a lambda layer

Deploy the layer

Using the AWS CLI:

aws lambda publish-layer-version --layer-name my-aws-otel-layer-python-arm64-ver-1-11-1 --zip-file fileb://layer.zip  --compatible-runtimes nodejs12.x nodejs10.x java11 python3.8 python3.9 --query 'LayerVersionArn' --output text

Note, by default the architecture (arm64 or amd64) will be determined by how you did the build. By default it will be the architecture of the machine / docker environment you did the build steps with.

The instructions above were done on an M1 Mac running the vanilla docker install. So the image was arm64.

Using the Layer with your lambda

The final output of aws lambda publish-layer-version command will show you the ARN of the new layer. You should capture that ARN and use that in the same way as the standard aws-otel-lambda layer