Skip to content

Instantly share code, notes, and snippets.

What would you like to do?
Get the SciPy stack for deployment to AWS Lambda
#! /usr/bin/env bash
# Install the SciPy stack on Amazon Linux and prepare it for AWS Lambda
yum -y update
yum -y groupinstall "Development Tools"
yum -y install blas --enablerepo=epel
yum -y install lapack --enablerepo=epel
yum -y install atlas-sse3-devel --enablerepo=epel
yum -y install Cython --enablerepo=epel
yum -y install python27
yum -y install python27-numpy.x86_64
yum -y install python27-numpy-f2py.x86_64
yum -y install python27-scipy.x86_64
yum -y install python27-matplotlib.x86_64
cd /home/ec2-user
pip install pandas -t /home/ec2-user/pandas
pip install sympy -t /home/ec2-user/sympy
cp -R /usr/lib64/python2.7/dist-packages/numpy /home/ec2-user/numpy
cp -R /usr/lib64/python2.7/dist-packages/scipy /home/ec2-user/scipy
cp -R /usr/lib64/python2.7/dist-packages/matplotlib /home/ec2-user/matplotlib
# need the supporting libraries in place on the desintation host
tar -czvf scipy-stack.tgz /home/ec2-user/*
# could easily edit to copy to S3 or push to a repo
Copy link

ankona commented May 10, 2017

Does this actually work for you? I'm fairly well versed in lambda but have no luck getting numpy+pandas+matplotlib to work in lambda. Not because of a technical issue, but because the amount of code to upload comes up against the lambda zip size limits (50mb).

I've been trying fruitlessly to deploy a zip that excludes various /test/* directories but no matter which ones I remove, the resulting lambda complains that it can no longer import numpy. As soon as I remove matplotlib from the dependencies, I go back under the size limit & all the non-plotting code works perfectly.

Update: After reading some more, it looks like maybe it's a matplotlib problem. Once I get the size down, I was assuming that I removed something essential because it immediately started giving an error "no module named TKinter." Now I think that it's something to do with lambda not having a UI lib?


Do you have any additional deployment steps you use?

Update 2 I got it. Using this link and this link, I was able to get it to run without tkinter using:
import matplotlib matplotlib.use('PS')

Copy link

jessalfredsen commented Jan 29, 2018

@THEHighlander Your Update 2 might have just prevented a suicide. I was going mad over this. Thank you so much.

Copy link

RajeshThallam commented Feb 6, 2018

@THEHighlander, thanks so much!!! Your Update 2 kept me going.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment