Skip to content

Instantly share code, notes, and snippets.

@alexcasalboni
Last active October 3, 2020 06:36
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save alexcasalboni/9ce2cef56a7d052d4f5e798b37083525 to your computer and use it in GitHub Desktop.
Save alexcasalboni/9ce2cef56a7d052d4f5e798b37083525 to your computer and use it in GitHub Desktop.
AWS Lambda Power Tuning - Demo Setup
# config
BUCKET_NAME=your-bucket-name
STACK_NAME=lambda-power-tuning-demo
# package
sam package --s3-bucket $BUCKET_NAME --template-file template.yml --output-template-file packaged.yml
# deploy
sam deploy --template-file packaged.yml --stack-name $STACK_NAME --capabilities CAPABILITY_AUTO_EXPAND CAPABILITY_IAM
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
# This is the Power Tuning app
# SAR will deploy it as a nested stack using the default params
PowerTuningApp:
Type: AWS::Serverless::Application
Properties:
Location:
ApplicationId: arn:aws:serverlessrepo:us-east-1:451282441545:applications/aws-lambda-power-tuning
SemanticVersion: 3.2.4
# Below you find a few functions that you can use to experiment with Lambda Power Tuning
# This function doesn't do much, just print and return
NoOpFunction:
Type: AWS::Serverless::Function
Properties:
Handler: index.lambda_handler
Runtime: python3.7
MemorySize: 1024
InlineCode: |
import json
def lambda_handler(event, context):
print("NOOP")
return {
'OK': 'OK',
}
Timeout: 2
Policies:
- AWSLambdaExecute
# This function computes all the prime numbers up to 1M (1000 times per execution)
PrimeNumbersFunction:
Type: AWS::Serverless::Function
Properties:
Handler: index.lambda_handler
Runtime: python3.7
MemorySize: 1024
InlineCode: |
import numpy as np
def lambda_handler(event, context):
for i in range(1000):
primes = compute_primes_up_to(1000000)
return {'OK': 'OK'}
def compute_primes_up_to(n):
# https://stackoverflow.com/questions/2068372/fastest-way-to-list-all-primes-below-n-in-python/3035188#3035188
sieve = np.ones(int(n/3) + (n%6==2), dtype=np.bool)
sieve[0] = False
for i in range(int(int(n**0.5)/3+1)):
if sieve[i]:
k=3*i+1|1
sieve[ int((k*k)/3) ::2*k] = False
sieve[int((k*k+4*k-2*k*(i&1))/3)::2*k] = False
return np.r_[2,3,((3*np.nonzero(sieve)[0]+1)|1)]
Layers:
- arn:aws:lambda:eu-west-1:399891621064:layer:AWSLambda-Python37-SciPy1x:2
Timeout: 60
Policies:
- AWSLambdaExecute
# This function creates a random 1500x1500 matrix and then inverts it (it's a lot of multiplications)
LinearAlgebraMatrixFunction:
Type: AWS::Serverless::Function
Properties:
Handler: index.lambda_handler
Runtime: python3.7
MemorySize: 1024
InlineCode: |
import numpy as np
np.random.seed(10)
def lambda_handler(event, context):
# create 1500x1500 matrix randomly
matrix = np.random.rand(1500, 1500)
# invert it
inverted_matrix = np.linalg.inv(matrix)
print(inverted_matrix)
return {'OK': 'OK'}
Layers:
- arn:aws:lambda:eu-west-1:399891621064:layer:AWSLambda-Python37-SciPy1x:2
Timeout: 60
Policies:
- AWSLambdaExecute
# This function downloads 150MB from S3 using 10 threads
S3DownloadFunction:
Type: AWS::Serverless::Function
Properties:
Handler: index.lambda_handler
Runtime: python3.7
MemorySize: 512 # min value to work
InlineCode: |
import os
import boto3
s3 = boto3.client('s3')
def lambda_handler(event, context):
# download 150MB from S3 with 10 threads
transfer_config = boto3.s3.transfer.TransferConfig(max_concurrency=10)
s3.download_file('amazon-reviews-pds', 'tsv/amazon_reviews_us_Watches_v1_00.tsv.gz', '/tmp/test.gz', Config=transfer_config)
bytes = os.stat('/tmp/test.gz').st_size
total = bytes / 1024 / 1024
unit = 'MB'
if total > 1024:
total = total / 1024
unit = 'GB'
print("Downloaded %s%s" % (round(total, 2), unit))
return {'OK': 'OK'}
Timeout: 60
Policies:
- AWSLambdaExecute
# This function calls a third-party API (cat-fact)
ExternalHTTPAPIFunction:
Type: AWS::Serverless::Function
Properties:
Handler: index.lambda_handler
Runtime: python3.7
MemorySize: 1024
InlineCode: |
import json
import urllib.request
URL = 'https://cat-fact.herokuapp.com/facts'
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64)'}
def lambda_handler(event, context):
req = urllib.request.Request(URL, headers=headers)
with urllib.request.urlopen(req) as response:
json_response = response.read()
return {
'response': json.loads(json_response),
}
Timeout: 10
Policies:
- AWSLambdaExecute
{
"lambdaARN": "your-lambda-function-arn",
"powerValues": [128, 256, 512, 1024, 2048, 3008],
"num": 50,
"payload": {},
"parallelInvocation": true,
"strategy": "cost"
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment