Serverless framework directory structure and KMS integration for Multi-function case.

serverless.md

Serverless with multiple functions

This gist describes multi-function-in-one-project case of serverless framework for AWS lambda.

Features

• Multi-function-in-one-project AWS lambda function management, via Serverless and WebPack
• Solid-type development with Typescript

Usage

• First, install all your npm packages via npm install command.
  Additionally, typescript, tslint, ts-node, serverless, webpack, serverless-webpack, ts-loader should be install globally. Install them by npm install -g typescript tslint ts-node serverless webpack serverless-webpack ts-loader
• Second, setup your AWS credentials. AWS credentials are AWS IAM access key and access key secret pair. They can be found at AWS IAM console -> Users -> (user) -> Security credentials.
  After creating AWS credential pair, setup via

serverless config credentials --provider aws --key AKIAIOSFODNN7EXAMPLE --secret wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY</code>

or set AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variable.

• Third, write your own code.
  Each lambda function code should be in separate directory, following below structure.

(root)
|-hello
|--src
|---hello.ts
|---webpack.config.js
|---serverless.yml
|
|-world
|--src
|---world.ts
|---webpack.config.js
|---serverless.yml
|
|-package.json
|-package.lock.json
|-tsconfig.json
|-tslint.json

• Fourth, write each function's webpack.js file and serverless.yml file.
  Examples are below:

// webpack.js
var path = require('path');
var slsw = require('serverless-webpack'); // For your convenient.

module.exports = {  
  entry: {
    "hello": "./hello.ts",
  },
  target: 'node',
  module: {
    loaders: [
      { test: /\.ts(x?)$/, loader: 'ts-loader' }
    ]
  },
  resolve: {
    extensions: ['.ts', '.js', '.tsx', '.jsx']
  },
  output: {
    libraryTarget: 'commonjs2',
    path: path.join(__dirname, "dist"),
    filename: "[name].js"
  },
};

# serverless.yml

service: hello # Service name. Individual for each AWS lambda functions.

provider:
  name: aws # This serverless targets AWS lambda function.
  runtime: nodejs6.10 # This serverless code targets node.js version 6.10
  region: ap-northeast-2 # Region for hosting serverless function.
  stage: production # Development stage.
  profile: default # Default AWS credential profile.
  memorySize: 128 # Memory size may not exceed minimum value.
  timeout: 60 # Timeout
  role: arn:aws:iam::xxxxxxxxxxxx:role/lambda_basic_execution # Role for AWS lambda function.
  versionFunctions: false


plugins:
  - serverless-webpack # For packaging individual functions locally.

functions:
  hello:
    handler: hello.index # Handler for lambda function.
    name: hello # Lambda function name.
    description: Hello world lambda function. # Lambda function description.
    environment: # Lambda function environment variables.
      variable-key-1: variable-value-1
      variable-key-2: variable-value-2
    events:
      - schedule: cron(10 0 * * ? *) # Cron scheduler.

• Fifth, all done. Use function.ts with npm commands:

"webpack": "ts-node function.ts webpack",
"package": "ts-node function.ts package",
"deploy": "ts-node function.ts deploy",
"invoke": "ts-node function.ts invoke",
"invoke-local": "ts-node function.ts invoke-local"

Available commands are below.

# For serverless arguments '--function', it should be omitted since we will take care of.

# webpack. If [--function name] part is ommited, all available functions will be webpacked.
npm run webpack [-- [--function name] [serverless args]]

# package. If [--function name] part is ommited, all available functions will be packaged.
npm run package [-- [--function name] [serverless args]]

# deploy. If [--function name] part is ommited, all available functions will be deployed.
npm run deploy [-- [--function name] [serverless args]]

# invoke. If [--function name] part is ommited, error will be thrown.
npm run invoke -- --function name [serverless args]

# invoke local. If [--function name] part is ommited, error will be thrown.
npm run invoke-local -- --function name [serverless args]

Tips

Securing private or sensitive data.

• For secure use of private or sensitive information, use of AWS KMS(Key Management Service) is encouraged.
• Private or sensitive information can be:
  • Database connection string with user name and password
  • API secret
  • Private key
  • ...
• Follow these steps to more secure application.
  1. Go to AWS IAM console and create master key. AWS KMS master key cannot reside outside of AWS server, i.e. cannot be exported by any means. So we can only encrypt plain text or decrypt cipher text via API calls.
  2. Via AWS CLI or any other library installed locally, encrypt your private, sensitive data. Store them in json/yml file. These files MUST not be commited.
  3. For serverless.yml file, Serverless framework provides fetching values from external json/yml file. Add encrypted data to environment variable. Use ${file(path_to_your_file):variable_name} syntax.
  4. At your lambda function, decrypt by calling KMS api. Example of javascript(node.js) is (assume cipher text is stored base64 encoded):

  // Import AWS sdk library.
  const aws = require("aws-sdk");
  
  // Decrypt callback.
  const decryptCallback = (value) => {
      if (value.$response.error || value.Plaintext === undefined) {
          throw new Error("Data decryption failure.");
      } else {
          let res = "";
          try {
              res = (value.Plaintext).toString();
          } catch (ex) {
              global.console.log("Data decryption failure.");
          } finally {
              return res;
          }
      }
  };
  
  // Assume environment variable to decrypt is ENCRYPTED_DATA.
  let decryptedData = "";
  console.log("Decrypting data...")
  decryptedData = await kms.decrypt({
      CiphertextBlob: Buffer.from(process.env.ENCRYPTED_DATA, "base64")
  }).promise().then(decryptCallback);
  
  // Now decryptedData variable holds securly decrypted data.

• By above approach, private or sensitive data does not appears in anywhere of your code or development environment. Plus, even cipher text only appears in environment variable for lambda function.
• For best practice, do not commit cipher text files and serverless.yml files.