sudo yum install update
sudo yum groupinstall -y "Development Tools"
sudo yum install -y python-devel libpng-devel freetype-devel gcc72-c++
If you want to use docker, you can use the following command:
docker run -it lambci/lambda:build-python3.7 bash
Otherwise, launch an EC2 instance and ssh in. From there, we can use virtual environments to package our code.
virtualenv v-env
source v-env/bin/activate
pip install lxml
deactivate
mkdir python
cp -a ./v-env/lib/python3.7/site-packages/. ./python
cp -a ./v-env/lib64/python3.7/site-packages/. ./python
zip -r9 ../layer37.zip python
Which will give us a zipped folder containing a structure like
./python/
lxml/
...
Where each of the underlying .so files correctly references the existing system libraries. We can use this structure as a layer. To download it, if you used docker, open another terminal window and run:
docker ps (note the Container ID!)
docker cp {container id}:/var/layer37.zip .
Or from the EC2 instance:
scp -i {key location} ec2-user@{instance dns address}:layer37.zip ./layer.zip
We can add it to any existing Lambda functions which need lxml and don't currently have the improperly packaged lxml. To create the layer, navigate to the Lambda Layers console and follow the prompts to create a new layer, uploading the layer37.zip
file. Be careful to select the compatible runtime environments. Now, navigate to your Lambda function and add the layer! You'll be able to successfully import lxml.
START RequestId: 066db9dd-a447-446d-b6bc-85b885121dd7 Version: $LATEST
(4, 3, 4, 0)
completed all processes
END RequestId: 066db9dd-a447-446d-b6bc-85b885121dd7
REPORT RequestId: 066db9dd-a447-446d-b6bc-85b885121dd7 Duration: 1.31 ms Billed Duration: 100 ms Memory Size: 3008 MB Max Memory Used: 65 MB
If you want to do this using a deployment package, simply navigate to the inside of your site packages folder, zip those files, and download that artifact to your local machine or build server. From there run zip -g {zip_name} {files/folders to include} where you might include for instance function.py
.
https://stackoverflow.com/questions/56818579/unable-to-import-lxml-etree-on-aws-lambda/67267811#67267811