API Gateway Approach

APIGatewayApproach.MD

Overview

We had several goals in mind as we evaluated the various solutions:

1. use API Gateway
2. use Lambda
3. have a way to test locally
4. have a way to deploy to prod while managing everything as code
5. integrate with existing Terraform infra.

Many of these items have point solutions, but there isn't a comprehensive solution that doesn't require a lot of copy/paste and extra verbosity. For example, you can achieve #1, #2 and #3 with Serverless Application Model (SAM), but you have to maintain two sets of "mappings", one in swagger and one in SAM template language. And creating an API Gateway by hand in Terraform is exremely verbose, often requiring 5 or 6 stanzas to define one simple GET /foo endpoint.

Having performed many experiements, we believe the following solution would be the best approach to implementing API Gateway while acheiving the best balance of DRY definitions along with local testability:

Step 1: You as the developer write normal Swagger

swagger: '2.0'
info:
  description: This is a simple example API
  version: 0.0.1
  title: Gruntwork Simple Inventory example API
  contact:
    email: matt@gruntwork.io
paths:
  /inventory:
    get:
      summary: searches inventory
      operationId: searchInventory
      description: |
        By passing in the appropriate options, you can search for
        available inventory in the system
      produces:
      - application/json
      responses:
        200:
          description: search results matching criteria
          schema:
            type: array
        400:
          description: bad input parameter
schemes:
 - https

Step 2: You test locally by adding some simple gruntwork extension markup

swagger: '2.0'
info:
  description: This is a simple example API
  version: 0.0.1
  title: Gruntwork Simple Inventory example API
  contact:
    email: matt@gruntwork.io
paths:
  /inventory:
    get:
      summary: searches inventory
      operationId: searchInventory
      description: |
        By passing in the appropriate options, you can search for
        available inventory in the system
      produces:
      - application/json
      responses:
        200:
          description: search results matching criteria
          schema:
            type: array
        400:
          description: bad input parameter
      x-gruntwork-apigateway-integration:      <<<<<<<<<<<<<<<<<<<<<<<<<Gruntwork Extention
        type: lambda
        sourcePath: ./js
        handler: index.get
        runtime: nodejs6.10
        
schemes:
 - https

Step 3: Run this gruntwork command to test your code locally

$ gruntwork aws api-gateway local --swagger-source ./swagger.yml
$ curl http://localhost:3000/inventory

Step 4: Generate Terraform API Gateway code from swagger this other gruntwork command (which also knows to process the custom extensions)

$ gruntwork aws api-gateway generate --swagger-source ./swagger.yml --output /code/infrastructure-modules/api-gateway-foo

which generates:

resource "aws_api_gateway_rest_api" "foo" {
  name        = "my-api-gateway"
  description = "This is a simple example API"
}

resource "aws_api_gateway_resource" "inventory" {
  rest_api_id = "${aws_api_gateway_rest_api.foo.id}"
  parent_id   = "${aws_api_gateway_rest_api.foo.root_resource_id}"
  path_part   = "inventory"
}

resource "aws_api_gateway_method" "inventory_get" {
  rest_api_id   = "${aws_api_gateway_rest_api.foo.id}"
  resource_id   = "${aws_api_gateway_resource.inventory.id}"
  http_method   = "GET"
  authorization = "NONE"
}

resource "aws_api_gateway_integration" "integration" {
  rest_api_id             = "${aws_api_gateway_rest_api.foo.id}"
  resource_id             = "${aws_api_gateway_resource.inventory.id}"
  http_method             = "${aws_api_gateway_method.inventory_get.http_method}"
  integration_http_method = "POST"
  type                    = "AWS_PROXY"
  uri                     = "${var.inventory_get_invoke_arn}"
}

Step 5: Use the generated API gateway code along with your normal Terraform Lambda code to deploy everything

infrastructure-modules/api-foo/main.tf:

module "lambda_inventory_get" {
  source = "git::git@github.com:gruntwork-io/package-lambda.git//modules/lambda?ref=v0.1.1"

  name = "${var.name}"
  description = "Lambda to get a list of inventory items"

  source_path = "${path.module}/js"
  runtime = "nodejs6.10"
  handler = "index.get"
  
  environment_variables = {
    DB_URL = "${data.terraform_remote_state.db.url}"
  }

  timeout = 30
  memory_size = 128
}

module "api_gateway" {
  source = "../api-gateway-foo"

  inventory_get_invoke_arn = "${module.lambda_inventory_get.invoke_arn}"  
  vpc_id = "${data.terraform_remote_state.vpc.id}"
}

data "terraform_remote_state" "db" {
  backend = "s3"
  config {
    bucket = "${var.bucket}"
    key = "${var.db_key}"
  }
}

data "terraform_remote_state" "vpc" {
  backend = "s3"
  config {
    bucket = "${var.bucket}"
    key = "${var.vpc_key}"
  }
}

infrastructure-live/stage/api-foo/terraform.tfvars:

terragrunt = {
  terraform {
    source = "git:.../infra-modules.git//modules/api-foo?ref=v0.0.4"
  }
}

name = "foo-stage"
bucket = "my-stage-terraform-bucket"
vpc_key = "vpc/terraform.tfstate"
db_key = "mysql/terraform.tfstate"

infra-live/prod/api-foo/terraform.tfvars:

terragrunt = {
  terraform {
    source = "git:.../infra-modules.git//modules/api-foo?ref=v0.0.4"
  }
}

name = "foo-prod"
bucket = "my-prod-terraform-bucket"
vpc_key = "vpc/terraform.tfstate"
db_key = "mysql/terraform.tfstate"