Last active
February 22, 2018 22:12
-
-
Save refayathaque/c4d68fc285c9e73faa401012a9f755f2 to your computer and use it in GitHub Desktop.
Dependency Worry-Free Python Lambda Deployments
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In the past, our team has been plagued with python dependency issues when updating our Lambda deployment packages. With the | |
help of senior developers we were able to update the python modules by 'building from source'. Essentially, our senior | |
developers would spin up an Amazon Linux machines and build out the python package there by running `pip install` for all | |
our modules. What resulted from these processess were certain `.so` files which we then had to include in our Lambda | |
deployment packages before we zipped them up and uploaded them to the Lambda Management Console. However, now, with the help | |
of virtual environments we may be able to introduce updated versions of existing python modules in our Lambda functions | |
without the need for these esoteric `.so` files. Here is what we advice you run on your terminal: | |
# Installing the python virtual environment | |
pip install virtualenv | |
# Creating our folder | |
mkdir lambda_dep_pkg_venv | |
# Going into our newly created folder | |
cd lambda_dep_pkg_venv | |
# Setting up the virtual environment in the 'lambda_dep_pkg_venv' folder | |
python -m virtualenv venv | |
# Activating our virtual environment | |
source venv/bin/activate | |
# Creating another sub-folder which we will use as our Lambda deployment package | |
mkdir lambda_dep_pkg_pre_zip | |
# Going into our new sub-folder | |
cd lambda_dep_pkg_pre_zip | |
# Installing whatever python module we want, NOT globally, but only within this sub-folder | |
pip install pshtt -t ./ | |
# Creating the obligatory __init__.py file, this is required to make python treat directories as containing packages | |
touch __init__.py | |
# Creating our lambda_handler file, which will be read by Lambda | |
# Convention is to call the handler function 'lambda_handler', IE `def lambda_handler(event, context):` | |
touch lambda_handler.py | |
# Copying over all virtual environment related files into the lambda_dep_pkg_pre_zip sub-folder | |
# https://docs.aws.amazon.com/lambda/latest/dg/lambda-dg.pdf (Page 102) | |
cd ../venv/lib/python2.7/site-packages | |
cp -r . ../../../../lambda_dep_pkg_pre_zip | |
# Going back to our lambda_dep_pkg_pre_zip sub-folder | |
cd ../../../../lambda_dep_pkg_pre_zip | |
# Zipping the sub-folder in preparation for uploading through Lambda Management Console | |
zip -r ../lambda_dep_pkg.zip . | |
Now go to your Lambda Management Console and upload the .zip file. When configuring the test case keep in mind that by | |
default the event variable is a python dictionary, so you'll have something like {'key1':'value'}. Therefore, to get to | |
`value` your handler function needs to have `event['key1']`, not `event.key1`. This is not JavaScript, don't make the same | |
mistake we made. :P |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment