Skip to content

Instantly share code, notes, and snippets.

@marceloleiva
Forked from haranjackson/chrome_headless.py
Created January 25, 2024 00:31
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save marceloleiva/e5deed44c9c6cc6cebf903384a95e863 to your computer and use it in GitHub Desktop.
Save marceloleiva/e5deed44c9c6cc6cebf903384a95e863 to your computer and use it in GitHub Desktop.
Deploys the Python Selenium library and Chrome Headless to an AWS Lambda layer. You can specify the region, library version, and runtime. An example Lambda function is given.
from selenium.webdriver import Chrome
from selenium.webdriver.chrome.options import Options
options = Options()
options.binary_location = '/opt/headless-chromium'
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--start-maximized')
options.add_argument('--start-fullscreen')
options.add_argument('--single-process')
options.add_argument('--disable-dev-shm-usage')
driver = Chrome('/opt/chromedriver', options=options)
REGION=eu-west-1
RUNTIME=python3.7
BUCKET=
SELENIUM_VER=3.141.0
CHROME_BINARY_VER=v1.0.0-55 # based on Chromium 69.0.3497.81
CHROMEDRIVER_VER=2.43 # supports Chrome v69-71
OUT_DIR=/out/build/chrome_headless/python/lib/$RUNTIME/site-packages
docker run -v $(pwd):/out -it lambci/lambda:build-$RUNTIME \
pip install selenium==$SELENIUM_VER -t $OUT_DIR
cp chrome_headless.py build/chrome_headless/python/chrome_headless.py
pushd build/chrome_headless
DRIVER_URL=https://chromedriver.storage.googleapis.com/$CHROMEDRIVER_VER/chromedriver_linux64.zip
curl -SL $DRIVER_URL >chromedriver.zip
unzip chromedriver.zip
rm chromedriver.zip
# download chrome binary
CHROME_URL=https://github.com/adieuadieu/serverless-chrome/releases/download/$CHROME_BINARY_VER/stable-headless-chromium-amazonlinux-2017-03.zip
curl -SL $CHROME_URL >headless-chromium.zip
unzip headless-chromium.zip
rm headless-chromium.zip
zip -r ../../chrome_headless.zip *
popd
aws s3 cp chrome_headless.zip s3://$BUCKET/chrome_headless.zip
aws lambda publish-layer-version \
--layer-name ChromeHeadless \
--region $REGION \
--content S3Bucket=$BUCKET,S3Key=chrome_headless.zip \
--compatible-runtimes $RUNTIME
rm -rf build *.zip
from chrome_headless import driver
def lambda_handler(event, context):
driver.get(event['url'])
return driver.page_source
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment