Skip to content

Instantly share code, notes, and snippets.

@kubek2k
Created December 1, 2018 22:50
Show Gist options
  • Save kubek2k/42528a93d5aa6a5a861bcfdb18aa2e28 to your computer and use it in GitHub Desktop.
Save kubek2k/42528a93d5aa6a5a861bcfdb18aa2e28 to your computer and use it in GitHub Desktop.
Short tutorial on how to run any statically linked binary on lambda

Preamble

The introduction of Custom Runtimes has opened up new possibilities when it comes to the spectrum of languages that can be used in Amazon Lambda. This short document will show how to run virtually any statically linked binary on it. The example will cover a stack built Haskell project.

Preparations

  • Create an Amazon Linux instance compatible with Lambda, you can use AMI link provided on the page (this is the one current for the time when this document was written, but make sure you have a look at the page if something goes not the way you expected).

    • In theory it should be also possible to install Amazon Linux locally
  • SSH to the instance as you would usually do

  • Upgrade pip and aws-cli on the instance

  • sudo pip install --upgrade pip

  • sudo /usr/local/bin/pip install --upgrade awscli

  • Create an IAM user for lambda operations with AWSLambdaFullAccess policy attached

    • Enable programmatic access and export appropriate env vars:
    export AWS_ACCESS_KEY_ID=..
    export AWS_SECRET_ACCESS_KEY=..
    export AWS_DEFAULT_REGION=..
    
  • Create IAM role called basic-execution-role with AWSLambdaBasicExecutionRole policy attached to it

    • note down role arn

Install bootstrap file as a new lambda layer

  • Save this file to file named bootstrap
#!/bin/sh

set -euo pipefail

while true
do
 HEADERS="$(mktemp)"

 # Pass the event straight to binary
 RESPONSE=$(curl -sS -LD "$HEADERS" -X GET "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/next" | ./${_HANDLER})
 REQUEST_ID=$(grep -Fi Lambda-Runtime-Aws-Request-Id "$HEADERS" | tr -d '[:space:]' | cut -d: -f2)

 # Send the response
 curl -X POST "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/$REQUEST_ID/response"  -d "$RESPONSE"
done
  • zip bootstrap.zip boostrap
  • aws lambda public-layer-version --layer-name binary-layer --zip-file fileb://bootstrap.zip
  • note down the LayerVersionArn

Create lambda binary

  • Create Haskell binary to run on the instance
    • Install stack
      • curl -sSL https://get.haskellstack.org/ | sh
      • Install required software
        • sudo yum install perl make automake gcc gmp-devel libffi zlib xz tar git gnupg
    • Create a simple project
      • stack new lambda
      • cd lambda
      • stack build
      • stack exec lambda-exe (to make sure it build and works properly)
  • Package the binary
    • cd .stack-work/install/x86_64-linux/lts-12.20/8.4.4/bin/ && zip ~/lambda.zip lambda-exe && cd - (beware of lts directory which may vary)

Upload lambda code

  • cd ~ && aws lambda create-function --function-name my-lambda --layers <layer_version_arn_from_above> --handler lambda --runtime provided --role <execution_role_arn> --zip-file fileb://lambda.zip

Test the lambda

  • aws lambda invoke --function-name my-lambda response.txt
  • if everything goes well you should see success - make sure by looking into output file response.txt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment