Lambda Cloudwatch Alarm demo

.CLOUDWATCH_ALARM_DEMO.md

hello

.gitignore

env
*.pyc
__pycache__
tmp
setenv-priv.sh

app.props

AppName=cloudwatch-alarm-demo

delete_stack.sh

#!/usr/bin/env bash

. app.props

aws cloudformation delete-stack --stack-name $AppName

deploy_stack.sh

#!/usr/bin/env bash

. app.props

aws cloudformation deploy --stack-name $AppName --template-file stack.yaml --capabilities CAPABILITY_NAMED_IAM

dump_logs.py

from botocore.exceptions import ClientError

import boto3, os, re, sys, time

def fetch_log_events(logs, kwargs):
    events, token = [], None
    while True:
        if token:
            kwargs["nextToken"]=token
        resp=logs.filter_log_events(**kwargs)
        events+=resp["events"]
        if "nextToken" in resp:
            token=resp["nextToken"]
        else:
            break
    return sorted(events,
                  key=lambda x: x["timestamp"])

if __name__=="__main__":
    try:
        if len(sys.argv) < 3:
            raise RuntimeError("please enter lambda name, window")
        lambdaname, window = sys.argv[1:3]
        if not re.search("^\\d+$", window):
            raise RuntimeError("window is invalid")
        window=int(window)
        logs=boto3.client("logs")
        starttime=int(1000*(time.time()-window))
        loggroupname="/aws/lambda/%s" % lambdaname
        kwargs={"logGroupName": loggroupname,
                "startTime": starttime,
                "interleaved": True}
        events=fetch_log_events(logs, kwargs)
        for event in events:
            msg=re.sub("\\r|\\n", "", event["message"])
            print (msg)
    except RuntimeError as error:
        print ("Error: %s" % str(error))
    except ClientError as error:
        print ("Error: %s" % str(error))

list_events.sh

#!/usr/bin/env bash

. app.props

aws cloudformation describe-stack-events --stack-name $AppName --query "StackEvents[].{\"1.Timestamp\":Timestamp,\"2.Id\":LogicalResourceId,\"3.Type\":ResourceType,\"4.Status\":ResourceStatus,\"5.Reason\":ResourceStatusReason}"

list_outputs.sh

#!/usr/bin/env bash

. app.props

aws cloudformation describe-stacks --stack-name $AppName --query 'Stacks[0].Outputs' --output table

list_resources.sh

#!/usr/bin/env bash

. app.props

aws cloudformation describe-stack-resources --stack-name $AppName --query "StackResources[].{\"1.Timestamp\":Timestamp,\"2.LogicalId\":LogicalResourceId,\"3.PhysicalId\":PhysicalResourceId,\"4.Type\":ResourceType,\"5.Status\":ResourceStatus}"

list_stacks.sh

#!/usr/bin/env bash

aws cloudformation describe-stacks --query "Stacks[].{\"1.Name\":StackName,\"2.Status\":StackStatus}"

NOTES.md

pareto implementation 03/04/24

• AWS::SNS::Topic, AWS::SNS::Subscription
• AWS::CloudWatch::Alarm
• alarm mixin (like Slack) defining topic, subscription, function (logger.warning), role and policy
• not sure this function needs a subscription filter
• then there is a hook() function which adds the cloudwatch alarm, connecting the target function to the topic via the alarm

ping_target.py

from botocore.exceptions import ClientError

import boto3, json, os, re, time

def hungarorise(text):
    return "".join([tok.capitalize()
                    for tok in re.split("\\-|\\_", text)])

def fetch_outputs(cf, stackname):
    outputs={}
    for stack in cf.describe_stacks()["Stacks"]:
        if (stack["StackName"].startswith(stackname) and
            "Outputs" in stack):
            for output in stack["Outputs"]:
                outputs[output["OutputKey"]]=output["OutputValue"]
    return outputs

if __name__=="__main__":
    try:
        props=dict([tuple(row.split("="))
                    for row in open("app.props").read().split("\n")
                    if row!=''])
        stackname=props["AppName"]
        cf=boto3.client("cloudformation")
        outputs=fetch_outputs(cf, stackname)
        targetkey=hungarorise("target-lambda-function-name")
        if targetkey not in outputs:
            raise RuntimeError("target not found")
        targetname=outputs[targetkey]
        L=boto3.client("lambda")
        for i in range(10):
            print (L.invoke(FunctionName=targetname,
                            InvocationType='RequestResponse',
                            Payload=json.dumps({"hello": "world"})))
            time.sleep(1)
    except RuntimeError as error:
        print ("Error: %s" % str(error))
    except ClientError as error:
        print ("Error: %s" % str(error))

requirements.txt

awscli
boto3
botocore
pyyaml

setenv.sh

#!/usr/bin/env bash

export AWS_DEFAULT_OUTPUT=table
export AWS_PROFILE=#{your-aws-profile-here}
export AWS_REGION=#{your-aws-region-here}

stack.yaml

AWSTemplateFormatVersion: '2010-09-09'
Resources:
  TargetLambdaFunction:
    Type: AWS::Lambda::Function
    Properties:
      Code:
        ZipFile: |
          def handler(event, context):
              print("Target Lambda called")
              return {
                  'statusCode': 200,
                  'body': 'Hello from Lambda!'
              }
      Handler: index.handler
      Role: !GetAtt GenericLambdaRole.Arn      
      Runtime: python3.10
  AlarmTopic:
    Type: AWS::SNS::Topic
  CloudWatchAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmDescription: "Invoke Alarm when target Lambda is called more than 5 times in 1 minute"
      Namespace: "AWS/Lambda"
      MetricName: "Invocations"
      Dimensions:
        - Name: "FunctionName"
          Value: !Ref TargetLambdaFunction
      Statistic: "Sum"
      Period: 60
      EvaluationPeriods: 1
      Threshold: 5
      ComparisonOperator: "GreaterThanThreshold"
      AlarmActions:
        - Ref: AlarmTopic
  GenericLambdaRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: sts:AssumeRole
      Policies:
        - PolicyName: AlarmLambdaPolicy
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - logs:CreateLogGroup
                  - logs:CreateLogStream
                  - logs:PutLogEvents
                Resource: arn:aws:logs:*:*:*
              - Effect: Allow
                Action: "lambda:InvokeFunction"
                Resource: "*"
  AlarmLambdaFunction:
    Type: AWS::Lambda::Function
    Properties:
      Code:
        ZipFile: |
          def handler(event, context):
              print (event)
      Handler: index.handler
      Role: !GetAtt GenericLambdaRole.Arn
      Runtime: python3.10
  TopicSubscriptionForAlarm:
    Type: AWS::SNS::Subscription
    Properties:
      TopicArn: !Ref AlarmTopic
      Protocol: lambda
      Endpoint: !GetAtt AlarmLambdaFunction.Arn
  PermissionForAlarmLambda:
    Type: AWS::Lambda::Permission
    Properties:
      FunctionName: !GetAtt AlarmLambdaFunction.Arn
      Action: "lambda:InvokeFunction"
      Principal: "sns.amazonaws.com"
      SourceArn: !Ref AlarmTopic
Outputs:
  TargetLambdaFunctionName:
    Value: !Ref TargetLambdaFunction
  AlarmLambdaFunctionName:
    Value: !Ref AlarmLambdaFunction

TODO.md

short

thoughts

• test stack deployment
• script to ping target lambda
• slack logging stuff?
  • surely dump_logs.py will suffice for now