Skip to content

Instantly share code, notes, and snippets.

@haranjackson
Created August 15, 2019 14:33
Show Gist options
  • Save haranjackson/50748c9f1e40940336e730a1783f0dd8 to your computer and use it in GitHub Desktop.
Save haranjackson/50748c9f1e40940336e730a1783f0dd8 to your computer and use it in GitHub Desktop.
Deploys Python Spacy library to an AWS Lambda layer. You can specify the region, library version, language, language model, and runtime.
REGION=eu-west-1
VER=2.1.8
LANG=en
MODEL=en_core_web_sm-2.1.0
RUNTIME=python3.7
MODEL_URL=https://github.com/explosion/spacy-models/releases/download/$MODEL/$MODEL.tar.gz
docker run -v $(pwd):/out -it lambci/lambda:build-$RUNTIME \
pip install spacy==$VER $MODEL_URL -t /out/build/spacy/python
# remove all languages except chosen LANG
pushd build/spacy/python/spacy/lang
find . ! -name $LANG -maxdepth 1 -type d -exec rm -rf {} +
popd
pushd build/spacy
zip -r ../../spacy.zip python/
popd
aws lambda publish-layer-version \
--layer-name Spacy \
--region $REGION \
--description "$VER $LANG $MODEL" \
--zip-file fileb://spacy.zip \
--compatible-runtimes $RUNTIME
rm -rf build *.zip
@sk-t3ch
Copy link

sk-t3ch commented Dec 30, 2020

Very nice

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment