calvinlfer/README.md

## README.md

      
    Raw
  

              README.md
            
          
    A build process for GraalVM native images that can be deployed on AWS Lambda as Docker container images

Use zio-lambda to write your own Lambda function based on Scala and ZIO

Checkout the example project for inspiration
Make sure you enable the GraalVMNativeImagePlugin like the example project does and configure it with the appropriate flags to build the native image
You may most likely need to add some reflect configuration and this post from Inner Product shows a good way to include the configuration in the project so Graal native-image will find it

If you have graal native already installed with the correct libraries in place for linking then great, otherwise use one of the following Docker images tagged with graalvm
NOTE: If you want to use the latest version of GraalVM, you can adapt the following Dockerfile
FROM ghcr.io/graalvm/graalvm-ce:ol8-java17

ENV SBT_VERSION ${SBT_VERSION:-1.6.2}
ENV SCALA_VERSION ${SCALA_VERSION:-2.13.8}
ENV JAVA_OPTS -XX:+UseG1GC

ENV USER_ID ${USER_ID:-1001}
ENV GROUP_ID ${GROUP_ID:-1001}

# Install sbt
RUN \
  curl -fsL "https://github.com/sbt/sbt/releases/download/v$SBT_VERSION/sbt-$SBT_VERSION.tgz" | tar xfz - -C /usr/share && \
  chown -R root:root /usr/share/sbt && \
  chmod -R 755 /usr/share/sbt && \
  ln -s /usr/share/sbt/bin/sbt /usr/local/bin/sbt

# Install Scala
## Piping curl directly in tar
RUN \
  case $SCALA_VERSION in \
    "3"*) URL=https://github.com/lampepfl/dotty/releases/download/$SCALA_VERSION/scala3-$SCALA_VERSION.tar.gz SCALA_DIR=/usr/share/scala3-$SCALA_VERSION ;; \
    *) URL=https://downloads.typesafe.com/scala/$SCALA_VERSION/scala-$SCALA_VERSION.tgz SCALA_DIR=/usr/share/scala-$SCALA_VERSION ;; \
  esac && \
  curl -fsL $URL | tar xfz - -C /usr/share && \
  mv $SCALA_DIR /usr/share/scala && \
  chown -R root:root /usr/share/scala && \
  chmod -R 755 /usr/share/scala && \
  ln -s /usr/share/scala/bin/* /usr/local/bin && \
  case $SCALA_VERSION in \
    "3"*) echo "@main def main = println(util.Properties.versionMsg)" > test.scala ;; \
    *) echo "println(util.Properties.versionMsg)" > test.scala ;; \
  esac && \
  scala -nocompdaemon test.scala && rm test.scala

# Add and use user sbtuser
RUN groupadd --gid $GROUP_ID sbtuser && useradd --gid $GROUP_ID --uid $USER_ID sbtuser --shell /bin/bash
USER sbtuser

# Switch working directory
WORKDIR /home/sbtuser

# Prepare sbt (warm cache)
RUN \
  sbt sbtVersion && \
  mkdir -p project && \
  echo "scalaVersion := \"${SCALA_VERSION}\"" > build.sbt && \
  echo "sbt.version=${SBT_VERSION}" > project/build.properties && \
  echo "// force sbt compiler-bridge download" > project/Dependencies.scala && \
  echo "case object Temp" > Temp.scala && \
  sbt compile && \
  rm -r project && rm build.sbt && rm Temp.scala && rm -r target

# Link everything into root as well
# This allows users of this container to choose, whether they want to run the container as sbtuser (non-root) or as root
USER root
RUN \
  ln -s /home/sbtuser/.cache /root/.cache && \
  ln -s /home/sbtuser/.sbt /root/.sbt && \
  if [ -d "/home/sbtuser/.ivy2" ]; then ln -s /home/sbtuser/.ivy2 /root/.ivy2; fi

# Switch working directory back to root
## Users wanting to use this container as non-root should combine the two following arguments
## -u sbtuser
## -w /home/sbtuser
WORKDIR /root

CMD sbt
Tag it as a builder:
docker build -t scala-graal-builder .
Package your runtime as a Docker image

Use this builder image to build the application for Graal native binary via a multi-stage build. As an example, we will build the example project found in zio-lambda:
FROM scala-graal-builder:latest as builder
COPY . /build
WORKDIR /build

RUN gu install native-image && sbt graalvm-native-image:packageBin

FROM gcr.io/distroless/base
COPY --from=builder /build/lambda-example/target/graalvm-native-image /app
CMD ["/app/zio-lambda-example"]
docker build -t native-image-binary .
Upload this native-binary-image image to the AWS ECR container registry so that AWS Lambda can access it.
Simplification: Let SBT Native package do most of the work for you

Alternatively, you can greatly simplify the process and simply add the following setting for native package to do it all for you in build.sbt:
  .settings(
    publish / skip := true,
    name := "zio-lambda-example",
    stdSettings("zio-lambda-example"),
    assembly / assemblyJarName := "zio-lambda-example.jar",
    GraalVMNativeImage / mainClass := Some("zio.lambda.example.SimpleHandler"),
    // add the following
    GraalVMNativeImage / containerBuildImage := GraalVMNativeImagePlugin
      .generateContainerBuildImage(
        "hseeberger/scala-sbt:graalvm-ce-21.3.0-java17_1.6.2_3.1.1"
      )
      .value,
    ...
Now you can just run sbt graalvm-native-image:packageBin on your host machine and SBT native package will take care of the rest and automatically build the image inside of Docker.
You would still need to follow the steps from earlier if you want to package your native image binary into a container image that would need to be uploaded to ECR. However, we found that you can directly use this native binary directly since all GLIBC dependencies are linked properly and you can follow one of the options on the zio-lambda readme.
Deploying the native binary directly

Here's how we create our AWS Lambda

Once we run sbt graalvm-native-image:packageBin, we'll find the binary present under the graalvm-native-image folder:

We'll just create the following bootstap file (which calls out to the binary) and place it in the same directory alongside the binary
#!/usr/bin/env bash

set -euo pipefail

./zio-lambda-example

Now we can zip both these files up:
> pwd
/home/cal/IdeaProjects/zio-lambda/lambda-example/target/graalvm-native-image                                                                                                                                
> zip upload.zip bootstrap zio-lambda-example

Now take upload.zip and upload it to AWS Lambda and test your function


Deploying the native binary via a Docker container

Following the steps from Package your runtime as a Docker image
You should now have a Docker image that contains the native image and invokes it. Take this image and push it to AWS ECR
pass=$(aws ecr get-login-password --region us-east-1) 
docker login --username AWS --password $pass <your_AWS_ECR_REPO>   
docker tag native-image-binary <your-particular-ecr-image-repository>:latest
docker push <your-particular-ecr-image-repository>:latest
Here is an example:

Create a Lambda function and choose container image:


Please note that because you incur the overhead of your native binary residing within a Docker container, there is more overhead than the other approach of deploying the binary straight to AWS Lambda