Skip to content

Instantly share code, notes, and snippets.

@stevecaldwell77
Last active October 2, 2015 17:42
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save stevecaldwell77/48056d72e3e480e19842 to your computer and use it in GitHub Desktop.
Save stevecaldwell77/48056d72e3e480e19842 to your computer and use it in GitHub Desktop.

lambda-deploy.js

Description

A standalone executable that helps in deploying AWS lambda functions.

Synopsis

lambda-deploy.js --function-name FUNCTION_NAME --region REGION --action (create|update|delete) --path /path/to/your/project [ARGS]

Arguments

--function-name
Name of lambda function to deploy. REQUIRED.

--region
AWS region in which to perform actions. REQUIRED.

--action
Which deployment action to take. Valid values are "create", "update", or "delete". REQUIRED.

--path
Path to project files. REQUIRED, unless running with "--action delete". NOTE: value given will be referred to as [PROJECT_PATH] in the rest of the documentation.

--handler
Name of function in your source code that will be called by Lambda. REQUIRED, unless running with "--action delete".

--function-src-dir
Path that will be zipped up and deployed as the lambda function's code. Defaults to [PROJECT_PATH]/task.

--execution-policy-file
File that holds IAM policies for this function. Defaults to [PROJECT_PATH]/lambda-execution-policy.json.

--timeout
Value for lambda function timeout. If not specified, AWS default will be used (currently 3).

--memory-size
Amount of memory to allocate to lambda function. If not specified, AWS default will be used (currently 128).

--kinesis-stream
JSON string defining how to map this function to a Kinesis stream. Can be specified multiple times. See Kinesis and DynamoDb stream definitions.

--dynamodb-stream
JSON string defining how to map this function to a DynamoDb stream. Can be specified multiple times. See Kinesis and DynamoDb stream definitions.

--s3-notification
JSON string defining how to subscribe this function to an S3 bucket notification. Can be specified multiple times. See S3 bucket notification definitions.

--sns-topic-arn
Subscribe this function to the SNS topic with this ARN. Can be specified multiple times.

--conf-file
Path to a file that can provide defaults for any of these arguments. Defaults to [PROJECT_PATH]/lambda-deploy.json. See Configuration file.

--silent
Quiet down informational output. See Outputs.

--debug
Output debugging information. See Outputs.

Kinesis and DynamoDb stream definitions

The JSON describing a kinesis or dynamodb stream mapping looks like:

{
    arn: "arn:aws:dynamodb:us-east-1:00000:table/foo/stream/2015-01-01T00:00:00.000",
    batch_size: 50,
    starting_position: "TRIM_HORIZON"
}

Explanation:

S3 bucket notification definitions

The JSON describing an S3 bucket notification looks like:

{
    bucket_name: "my_bucket_name"
    events : [
        "s3:ObjectCreated:*"
    ],
    filter_rules: [
        {
            name: 'prefix',
            value: 'lookforme'
        }
    ]
}

Explanation:

  • bucket_name: REQUIRED. Name of bucket to listen for notifications.
  • events: REQUIRED. The event types that should trigger our lambda function.
  • filter_rules: Optional. Object key name filters to apply.

Configuration file

This command should look for a configuration file that exists at the path specified by --conf-file, or its default if not given. The JSON found in that file should provide defaults for any of the arguments not specified on the command line. The JSON will use snake case'd version of the argument names. For arguments that take in a JSON string, the file should contain actual JSON (i.e. not stringified).

As an example, running this command

deploy-lambda.js --quiet

With this file at [PROJECT_PATH]/lambda-deploy.json:

{
    function_name : "hello_world",
    handler: "index.helloWorld",
    timeout: 45,
    sns_topic_arn: "some-arn",
    kinesis_stream: {
        arn: "some-other-arn",
        batch_size: 50
    }
}

Is equivalent to running:

deploy-lambda.js --quiet --function-name hello_world --handler "index.helloWorld" --timeout 45 --sns-topic-arn "some-arn" --kinesis-stream "{\"arn\":\"some-other-arn\",\"batch_size\":50}"

Action: create

If this command is called with "--action create", it will perform the following tasks in order:

  1. Create an IAM role.
  2. Create a zip file.
  3. Create a lambda function.
  4. Subscribe function to invocants.
  5. Cleanup.

Action: update

If this command is called with "--action update", it will perform the following tasks in order:

  1. Update IAM role.
  2. Create a zip file.
  3. Update lambda function.
  4. Update kinesis and dynamodb streams.
  5. Update SNS subscriptions.
  6. Update S3 notifications.
  7. Cleanup.

Action: delete

If this command is called with "--action delete", it will perform the following tasks in order:

  1. Delete kinesis and dynamodb streams.
  2. Delete SNS subscriptions.
  3. Delete S3 notifications.
  4. Delete lambda function and IAM role.

Document Conventions

In the below Task sections, the following conventions are used:

  • lambdaFunctionName: the value passed to --function-name
  • randomId: a string of 10 hexadecimal characters, e.g. "54c89a9dd2"

Task: Create an IAM role

To create a new IAM role, first create the role via IAM.createRole.

RoleName should be in the format:

'lambda-' + lambdaFunctionName + '-' + randomId

AssumeRolePolicyDocument should be:

{"Version":"2012-10-17","Statement":[{"Sid":"","Effect":"Allow","Principal":{"Service":"lambda.amazonaws.com"},"Action":"sts:AssumeRole"}]}

Once the role is created we need to set its policy, by following Set IAM role policy.

Task: Update IAM role

To update the IAM role for an existing function, first deterimine the name of the function's IAM role via Lammbda.getFunction

Then follow Set IAM role policy.

Task: Set IAM role policy

To set the permissions policy for an IAM role, call IAM.putRolePolicy

RoleName is the role you're updating.

PolicyName should be lambdaFunctionName

PolicyDocument should be the contents of --execution-policy-file.

Task: Create a zip file

Take the contents of the directory at --function-src-dir and create a temporary zip file.

Task: Create a lambda function

To create a new lambda function, call Lambda.createFunction

FunctionName should be lambdaFunctionName.

Runtime should be "nodejs".

Role should be the ARN of the role created for this function.

Handler should be set to the value of --handler.

Timeout should be set to the value of --timeout if specified, otherwise it should be excluded.

MemorySize should be set to the value of --memory-size if specified, otherwise it should be excluded.

Code should use the temporary zip file created for ZipFile.

Task: Update lambda function

To update a lambda function, get the current state of the function via Lambda.getFunction

FunctionName should be lambdaFunctionName.

Then, update its code via Lambda.updateFunctionCode

FunctionName should be lambdaFunctionName.

ZipFile should use the temporary zip file.

Then, if any of the current function properties do not match our command arguments, update the function via Lambda.updateFunctionConfiguration

FunctionName should be lambdaFunctionName.

Handler maps to --handler.

Timeout maps to --timeout.

MemorySize maps to --memory-size.

Task: Delete lambda function and IAM role

First, find the name of the lambda function's IAM role via Lambda.getFunction.

Then, delete the function via Lambda.deleteFunction.

Then, delete the role via IAM.deleteRole

Task: Subscribe function to invocants

For every Kinesis and/or DynamoDb stream in our command arguments, create a mapping for that stream.

For every SNS topic ARN specified in our command arguments, create a subscription.

For every S3 notification specified in our command arguments, create a notification.

Task: Create mapping for Kinesis or DynamoDb stream

Make a call to Lambda.createEventSourceMapping:

FunctionName should be lambdaFunctionName.

EventSourceArn is the ARN of our stream.

StartingPosition should be taken from "starting_position" in our call's stream definition.

BatchSize should be taken from "batch_size" in our call's stream definition, if specified.

Task: Update Kinesis and Dynamodb streams

First, fetch our function's current streams, via Lambda.listEventSourceMappings

FunctionName should be lambdaFunctionName.

If the stream is currently mapped to our function, but was not specified in our command arguments, we need to delete the mapping

If we have a stream mapping defined in our command arguments that does not currently exist, we need to create a new mapping

Otherwise, we should compare the batch_size setting of the existing stream with that specified in our command arguments. If they don't match, we should update the batch size via Lambda.updateEventSourceMapping.

Task: Delete mapping for Kinesis or DynamoDb stream

Make a call to Lambda.deleteEventSourceMapping:

UUID can be found in the previous call to listEventSourceMappings.

Task: Delete Kinesis and Dynamodb streams

Follow Update kinesis and dynamodb streams, but delete all existing streams and don't create any new ones.

Task: Create SNS subscription

First, grant the SNS topic permission to invoke our lambda function, via Lambda.addPermission.

Action should be "lambda:InvokeFunction".

FunctionName should be lambdaFunctionName.

Principal should be "sns.amazonaws.com".

StatementId should be a new UUID string.

SourceArn should be the ARN of our SNS topic.

Then, call SNS.subscribe.

Protocol should be "lambda".

TopicArn should be the ARN of our SNS topic.

Endpoint should be the ARN of our lambda function.

Task: Update SNS subscriptions

Fetch our current SNS subscriptions using SNS.listSubscriptions. Make sure to handle NextToken correctly when there are more than 100 results.

Filter the list to include entries where the Endpoint matches our function's ARN.

If a SNS subscription currently exists for our function, but was not specified in our command arguments, we need to delete the SNS subscription.

If we have a SNS subscription defined in our command arguments that does not exist, we need to create a subscription.

Otherwise, no further action needs to be taken.

Task: Delete SNS Subscription

First, unsubscribe our lambda function from the SNS topic, via SNS.unsubscribe, using the SubscriptionArn retrieved by listSubscriptions.

Then, retrieve the current access policy for our lambda function, using Lambda.getPolicy.

In the response, find the policy statement that matches our SNS topic. Then, using that policy's Sid, make a call to Lambda.removePermission to delete this access permission.

Task: Delete SNS Subscriptions

Follow Update SNS subscriptions, but delete all existing subscriptions and don't create any new ones.

Task: Create S3 notification

First, grant the S3 bucket permission to invoke our lambda function, via Lambda.addPermission.

Action should be "lambda:InvokeFunction".

FunctionName should be lambdaFunctionName.

Principal should be "s3.amazonaws.com".

StatementId should be a new UUID string.

SourceArn should be the ARN of our S3 bucket.

We then need to add our lambda function to this bucket's notification configuration. We do this by first fetching the existing notification via S3.getBucketNotificationConfiguration.

We then add an entry to the bucket's LambdaFunctionConfigurations array for our lambda function using S3.putBucketNotificationConfiguration. The entry should look like:

{
    LambdaFunctionArn: arn,
    Events: events,
    Filter: {
        Key: {
            FilterRules: filterRules
    }
}

Where "arn" is our lambda function's ARN, "events" is the events specified in our S3 notifiction definition, and "filterRules" is the filter_rules array in our S3 notification definition. If our definition has no filter_rules, the entire "Filter" section above should be left off.

Task: Update S3 notifications

Fetch our current S3 notifications by first fetching all buckets using S3.listBuckets, then calling S3.getBucketNotificationConfiguration for each bucket.

Filter the list to include LambdaFunctionConfigurations's that have our lambda function's ARN.

If an S3 notification currently exists for our function, but was not specified in our command arguments, we need to delete the s3 notification.

If we have an S3 notification subscription defined in our command arguments that does not exist, we need to create a new notification.

If an existing S3 notification has events or filter rules that do not match the definition in our command arguments, we should create a new notification, then delete the old one.

Task: Delete S3 Notification

First fetch the existing bucket notification via S3.getBucketNotificationConfiguration.

Then remove our lambda function from the LambdaFunctionConfigurations array, and update the bucket configuration using S3.putBucketNotificationConfiguration.

Then, retrieve the current access policy for our lambda function, using Lambda.getPolicy.

In the response, find the policy statement that matches our S3 bucket. Then, using that policy's Sid, make a call to Lambda.removePermission to delete this access permission.

Task: Delete S3 Notifications

Follow Update S3 notifications, but delete all existing notifications and don't create any new ones.

Task: Cleanup

To cleanup, delete the temporary zip file.

Outputs

Normal mode: The command should output a line to STDOUT at the beginning of performing each task, then a summary line indented below when the task is finished, including relevant object ids, etc.. E.g.:

creating IAM role...
  created role lambda-my_function_name-54c89a9dd2
creating zip file...
  /tmp/a3ds3c.zip created
...

Debug mode: If the command is run with --debug, in addition to normal output the command should also output details about AWS calls, e.g.

creating IAM role...
DEBUG: calling IAM.createRole
DEBUG: params: { RoleName : "lambda-my_function_name-54c89a9dd2", ...
  created role lambda-my_function_name-54c89a9dd2

Silent mode: If the command is run with --silent, only error output should occurl

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment