apig-lambda-dynamo integration.md

Use case: hit counter

At every path, just returns the number of times that path has been requested. Each stage should get its own table to store the counters, rather than sharing a table. Ideally, the linking between the table and the Lambda function happens at deployment time (i.e., the stages share the same code object in S3, not each having separate but identical objects) and not at build time (with separate, different code objects) or run time (that requires making extra API calls inside the Lambda function).

The crux of it is the following:

• A plugin architecture for creating resources that are not built in
• The ability for these plugins to both receive and return key-value pairs
• The ability to inject the outputs of these resource-creating plugins into a Lambda function at deployment time
• A dependency resolve to order the plugins by the connections of input and output

The following is how I would implement this using CloudFormation:

Consists of:

• API Gateway with a single resource /{name}
• DynamoDB table one document per {name} with field "hits"
• Lambda function

Lambda function should look like:

def handler(event, context):
    name = event['name']

    env = get_env()
    table_name = env['table']
    table = boto3.service('dynamodb').Table(table_name)

    response = table.update_item(
        Key={"name": name},
        UpdateExpression="ADD counter 1",
        ReturnValues="UPDATED_NEW")
    return response['Attributes']['counter']

def get_env():
    with open('.env') as fp:
        return dict(line.split('=', 1) for line in fp)

In CloudFormation, the resources would look something like this:

{
"table": {
  "Type": "AWS::DynamoDB"::"Table",
  "Properties": {
    "KeySchema": [ { "AttributeName": "name", KeyType: "HASH" } ],
    "AttributeDefinitions": [
      { "AttributeName": "name", "AttributeType": "S" },
      { "AttributeName": "counter", "AttributeType: "N" }
    ]
  }
},
"gateway": {
  "Type": "Custom::APIGatewayResource",
  "Properties": {
    "Path": "{name}",
    "Method": "GET",
    /* etc. etc. */
    "IntegrationRequest": {
      "Type": "Lambda",
      "FunctionArn": { "GetAtt": [ "function", "Arn" ] }
    }
  }
},
"function": {
  "Type": "Custom::LambdaFunction",
  "Properties": {
    "Code": { ... },
    "Handler": "index.handler",
    "Role": { "Ref": "role" },
    "Env": {
      "table": { "Ref": "table" }
    }
  }
},
"role": {
  "Type": "AWS::IAM::Role",
  "Properties": {
    /* assume role permission for Lambda */
    /* DynamoDB access policy */
  }
}

This requires a Lambda function for each of the custom CloudFormation resource types. The one for API Gateway is a straightforward replication of the API Gateway API calls (speaking from experience, this is quite simple). The other requires wrapping a call to Create/Update/DeleteFunction. The meat of that function:

# event is Lambda input
physical_resource_id = generate_unique_id(event['StackId'], event['LogicalResourceId']) # like CF does
env_from_input = event['Properties']['Env']
old_zip = get_zip_from_s3(event['Code'])
new_zip = new_zip_file()
for filename, data in old_zip:
  if filename == '.env':
    env_to_update = get_env_from_file(data)
    env = {}
    env.update(env_to_update)
    env.update(env_from_input)
    data = '\n'.join('%=%' % (key, value) for key, value in env)
  new_zip.write(filename, data)
lambda_service.CreateFunction(Name=physical_resource_id, Code=new_zip)

Making these custom Lambdas is relatively straightforward, the heavy lifting can be done by a wrapper that processes the CF request and handles exceptions and communicating back to CF.

See https://github.com/benkehoe/cfnlambda/blob/master/cfnlambda_obj.py for an example in Python, which just requires the developer to implement the create, update, and delete functions and raise an exception on failure.