This project uses a GithubActions workflow to deploy the lambda function to AWS.
The function is deployed to different regions (eu-central-1 or us-west-2) as either a production or a stage (suffixes: -prod or -stage) version depending on which branch the code was pushed to.
Branch options: eu-prod, eu-stage, us-prod and us-stage.
The respective regions that are used are eu-central-1 and us-west-2.
- Create an AWS User for deployment or use an existing on e.g.
fp-lambda-sqs-worker. For necessary permissions see AWS Deployment User. - Add the secrets to github repo - for a list see Github Secrets.
- ECR Repository
- create a new repository named according to package.json
- add permissions to the repository for lambda to be able to pull the images (see: ECR Repository)
- Role for lambda function
- Create a role for the lambda function or use an existing one e.g.
Lambda-SQS-DB-Worker. For necessary permissions see Lambda Role. - Enter arn of the lambda role in workflow (specific instructions see Lambda Role)
- Create a role for the lambda function or use an existing one e.g.
- Push to github on one of the supported branches.
- The workflow will run and...
- build the docker image
- push to ecr
- create or update the lambda function
- After the deployment finished you can create a trigger for the lambda (this is only necessary the first time after a function is created. If a function is updated, the triggers will persist)
The ts source code can be found in ./src.
It is compiled into ./dist.
The source can be compiled manually with npm run build.
For development it can be helpful to compile in watch mode with npm run watch.
The lambda function is built into a docker image during the deployment.
A docker image can be build with npm run docker:build. This will compile the code and build an image named like the npm package name. This produces theoretically the exact same image as the build process on github actions.
Environment variables for the Lambda Function, among other things, are provided through a docker.env file. Create one in the root of the project.
MSSQL_PASSWORD=the_MSSQL-password
PG_PASSWORD=the_PG-passwordAfter the docker image is built it can be run with npm run docker:start. The container then exposes the runtime interface client on port 9000.
A development version can be run with npm run dev.
This runs the base image of our Dockerfile and mounts the project root folder (incl. node_modules) into the lambda task root.
The entrypoint is overwritten and set to node which then executes watch.js, which is a custom script, that watches for file changes in the dist directory and then restarts the runtime interface environment.
It is advised to run npm run watch simultaneously in another terminal, to watch for changes of the js sources and automatically compile into dist, which will then trigger watch.js.
To test your function you can invoke it like described in Invoke Lambda Function.
Running the image with the node_modules mounted in the container like "the normal dev mode" can lead to problems if the project uses packages that are compiled for a specific architecture.
Running the image with npm run dev:installOnDocker does not mount the whole project folder to the lambda task root and instead mounts specific files and directories. The import difference is that the node_modules folder is not mounted. Instead npm install is executed after the container starts and before launching watch.js. To do this the entrypoint is set to sh and the cmd to devSetup.sh.
This method differs from the "full"/poduction Image because all packages are installed (incl. dev dependencies)
Mounted files and directories:
A function in a running container that exposes the runtime interface client on port 9000 (see: run locally) can be invoked in the following ways:
# invoke with json object
npm run test -- '{"test":"me"}'
# invoke with file - the @ symbol indicates that a filename follows
npm run test -- @event.example.jsonExamples and explanations regarding the different resources.
A user with programmatic access is needed to deploy the function to lambda. It is used in the github workflow to create and update resources using the aws cli. The policies json of an example user can be found below. The access and secret key are provided to the workflow through Github Secrets.
Example IAM policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "GetAuthorizationToken",
"Effect": "Allow",
"Action": [
"ecr:GetAuthorizationToken"
],
"Resource": "*"
},
{
"Sid": "AllowPush",
"Effect": "Allow",
"Action": [
"ecr:PutImage",
"ecr:InitiateLayerUpload",
"ecr:UploadLayerPart",
"ecr:CompleteLayerUpload",
"ecr:GetDownloadUrlForLayer",
"ecr:BatchCheckLayerAvailability",
"ecr:DescribeRepositories"
],
"Resource": "arn:aws:ecr:eu-central-1:151879656186:repository/*"
},
{
"Sid": "Lambda",
"Effect": "Allow",
"Action": [
"lambda:AddPermission",
"lambda:CreateFunction",
"lambda:DeleteFunction",
"lambda:GetFunction",
"lambda:GetFunctionConfiguration",
"lambda:ListTags",
"lambda:RemovePermission",
"lambda:TagResource",
"lambda:UntagResource",
"lambda:UpdateFunctionCode",
"lambda:UpdateFunctionConfiguration"
],
"Resource": [
"arn:aws:lambda:*:*:function:*"
]
},
{
"Sid": "Lambdalistfunctions",
"Effect": "Allow",
"Action": [
"lambda:ListFunctions"
],
"Resource": [
"*"
]
},
{
"Sid": "IAM",
"Effect": "Allow",
"Action": [
"iam:AttachRolePolicy",
"iam:DetachRolePolicy",
"iam:GetRole",
"iam:TagRole"
],
"Resource": [
"arn:aws:iam::*:role/*"
]
},
{
"Sid": "IAMPassRole",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "*",
"Condition": {
"StringEquals": {
"iam:PassedToService": "lambda.amazonaws.com"
}
}
}
]
}The Lambda function needs an role attached to it. The role needs to have at least some basic permissions. This is an example of a basic role, that grants permission to create CloudWatch logs, poll from queues (i.e. get triggered by messages) and access a VPC to connect to a database.
AWS managed policies attached to role:
- AWSLambdaBasicExecutionRole
- AWSLambdaSQSQueueExecutionRole
- AWSLambdaVPCAccessExecutionRole
Replace the < LAMBDA ROLE ARN > placeholder shown in the section below in the workflow file
- name: create function
if: ${{ steps.check-func.outputs.function != steps.package-name.outputs.name-suff}}
run: |
aws lambda create-function \
--region ${{steps.region.outputs.region}} \
--function-name ${{steps.package-name.outputs.name-suff}} \
--package-type Image \
--code ImageUri=${{steps.build-image.outputs.image}} \
--role < LAMBDA ROLE ARN > \
--environment '{
"Variables": {
"PG_PASSWORD": "${{steps.env-vars.outputs.pgpass}}",
"MSSQL_PASSWORD": "${{steps.env-vars.outputs.mssqlpass}}"
}
}' \| name | default |
|---|---|
| AWS_ACCESS_KEY_ID | null |
| AWS_SECRET_ACCESS_KEY | null |
| EU_PROD_PG_PASSWORD | null |
| EU_PROD_MSSQL_PASSWORD | null |
| EU_STAGE_PG_PASSWORD | null |
| EU_STAGE_MSSQL_PASSWORD | null |
| US_PROD_PG_PASSWORD | null |
| US_PROD_MSSQL_PASSWORD | null |
| US_STAGE_PG_PASSWORD | null |
| US_STAGE_MSSQL_PASSWORD | null |
NOTE: AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are required.
For deploying to eu-stage only to EU_STAGE_* secrets are necessary. Same is true for the other branches respectively.
If they are not set the function will still be deployed and the env vars will be set to an empty string.
The env vars for the function will always be available as PG_PASSWORD and MSSQL_PASSWORD, no matter the branch that has been pushed to.
Repository permission statement:
{
"Sid": "LambdaECRImageRetrievalPolicy",
"Effect": "Allow",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Action": [
"ecr:BatchGetImage",
"ecr:GetDownloadUrlForLayer"
]
}This allows all lambda to pull images from this repository. For help on how to set a repository policy see this AWS User Guide.