Skip to content

Instantly share code, notes, and snippets.

@refayathaque
Last active February 22, 2018 22:12
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save refayathaque/c4d68fc285c9e73faa401012a9f755f2 to your computer and use it in GitHub Desktop.
Save refayathaque/c4d68fc285c9e73faa401012a9f755f2 to your computer and use it in GitHub Desktop.
Dependency Worry-Free Python Lambda Deployments
In the past, our team has been plagued with python dependency issues when updating our Lambda deployment packages. With the
help of senior developers we were able to update the python modules by 'building from source'. Essentially, our senior
developers would spin up an Amazon Linux machines and build out the python package there by running `pip install` for all
our modules. What resulted from these processess were certain `.so` files which we then had to include in our Lambda
deployment packages before we zipped them up and uploaded them to the Lambda Management Console. However, now, with the help
of virtual environments we may be able to introduce updated versions of existing python modules in our Lambda functions
without the need for these esoteric `.so` files. Here is what we advice you run on your terminal:
# Installing the python virtual environment
pip install virtualenv
# Creating our folder
mkdir lambda_dep_pkg_venv
# Going into our newly created folder
cd lambda_dep_pkg_venv
# Setting up the virtual environment in the 'lambda_dep_pkg_venv' folder
python -m virtualenv venv
# Activating our virtual environment
source venv/bin/activate
# Creating another sub-folder which we will use as our Lambda deployment package
mkdir lambda_dep_pkg_pre_zip
# Going into our new sub-folder
cd lambda_dep_pkg_pre_zip
# Installing whatever python module we want, NOT globally, but only within this sub-folder
pip install pshtt -t ./
# Creating the obligatory __init__.py file, this is required to make python treat directories as containing packages
touch __init__.py
# Creating our lambda_handler file, which will be read by Lambda
# Convention is to call the handler function 'lambda_handler', IE `def lambda_handler(event, context):`
touch lambda_handler.py
# Copying over all virtual environment related files into the lambda_dep_pkg_pre_zip sub-folder
# https://docs.aws.amazon.com/lambda/latest/dg/lambda-dg.pdf (Page 102)
cd ../venv/lib/python2.7/site-packages
cp -r . ../../../../lambda_dep_pkg_pre_zip
# Going back to our lambda_dep_pkg_pre_zip sub-folder
cd ../../../../lambda_dep_pkg_pre_zip
# Zipping the sub-folder in preparation for uploading through Lambda Management Console
zip -r ../lambda_dep_pkg.zip .
Now go to your Lambda Management Console and upload the .zip file. When configuring the test case keep in mind that by
default the event variable is a python dictionary, so you'll have something like {'key1':'value'}. Therefore, to get to
`value` your handler function needs to have `event['key1']`, not `event.key1`. This is not JavaScript, don't make the same
mistake we made. :P
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment