Skip to content

Instantly share code, notes, and snippets.

@haranjackson
Last active January 25, 2024 00:31
Show Gist options
  • Star 13 You must be signed in to star a gist
  • Fork 5 You must be signed in to fork a gist
  • Save haranjackson/77e9665289a0ca781bd30e60b52ec361 to your computer and use it in GitHub Desktop.
Save haranjackson/77e9665289a0ca781bd30e60b52ec361 to your computer and use it in GitHub Desktop.
Deploys the Python Selenium library and Chrome Headless to an AWS Lambda layer. You can specify the region, library version, and runtime. An example Lambda function is given.
from selenium.webdriver import Chrome
from selenium.webdriver.chrome.options import Options
options = Options()
options.binary_location = '/opt/headless-chromium'
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--start-maximized')
options.add_argument('--start-fullscreen')
options.add_argument('--single-process')
options.add_argument('--disable-dev-shm-usage')
driver = Chrome('/opt/chromedriver', options=options)
REGION=eu-west-1
RUNTIME=python3.7
BUCKET=
SELENIUM_VER=3.141.0
CHROME_BINARY_VER=v1.0.0-55 # based on Chromium 69.0.3497.81
CHROMEDRIVER_VER=2.43 # supports Chrome v69-71
OUT_DIR=/out/build/chrome_headless/python/lib/$RUNTIME/site-packages
docker run -v $(pwd):/out -it lambci/lambda:build-$RUNTIME \
pip install selenium==$SELENIUM_VER -t $OUT_DIR
cp chrome_headless.py build/chrome_headless/python/chrome_headless.py
pushd build/chrome_headless
DRIVER_URL=https://chromedriver.storage.googleapis.com/$CHROMEDRIVER_VER/chromedriver_linux64.zip
curl -SL $DRIVER_URL >chromedriver.zip
unzip chromedriver.zip
rm chromedriver.zip
# download chrome binary
CHROME_URL=https://github.com/adieuadieu/serverless-chrome/releases/download/$CHROME_BINARY_VER/stable-headless-chromium-amazonlinux-2017-03.zip
curl -SL $CHROME_URL >headless-chromium.zip
unzip headless-chromium.zip
rm headless-chromium.zip
zip -r ../../chrome_headless.zip *
popd
aws s3 cp chrome_headless.zip s3://$BUCKET/chrome_headless.zip
aws lambda publish-layer-version \
--layer-name ChromeHeadless \
--region $REGION \
--content S3Bucket=$BUCKET,S3Key=chrome_headless.zip \
--compatible-runtimes $RUNTIME
rm -rf build *.zip
from chrome_headless import driver
def lambda_handler(event, context):
driver.get(event['url'])
return driver.page_source
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment