haranjackson
/ python_function_name_duplicates.sh
Created
May 26, 2020 23:20
Shell script for finding Python function name duplicates
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| find . -not -path '*/\.*' -type f -name "*.py" | # find non-hidden python files | |
| xargs egrep -o 'def [A-Za-z0-9_]{1,}' | # find function def lines | |
| egrep -v 'def __[a-z]{1,}__' | # ignore __init__, __repr__,... | |
| awk -F "def " '{print $NF}' | # take function names | |
| sort | # sort list of function names | |
| uniq -c | # count occurrences of each | |
| sort # sort again by count |
haranjackson
/ authAtEdge.yml
Last active
April 5, 2020 00:33
CloudFormation template for quickly creating a password-protected website. Put your website's source in folder "src" next to authAtEdge.yml and deploy.sh. Based on: github.com/aws-samples/cloudfront-authorization-at-edge.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| AWSTemplateFormatVersion: 2010-09-09 | |
| Transform: AWS::Serverless-2016-10-31 | |
| Parameters: | |
| DomainName: | |
| Type: String | |
| Description: Domain name of the website | |
| HostedZone: | |
| Type: String |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| STACK= # give the stack a name | |
| REGION= # choose your region | |
| DOMAIN= # a Route53 domain - wiki will be hosted at wiki.[DOMAIN] | |
| aws cloudformation deploy \ | |
| --template-file wiki.yaml \ | |
| --stack-name $STACK \ | |
| --region $REGION \ | |
| --parameter-overrides Domain=$DOMAIN |
haranjackson
/ sumy_lambda_layer.sh
Created
November 29, 2019 00:20
Deploys Python sumy library to an AWS Lambda layer. You can specify the region, library version, and runtime. You must specify a temporary S3 bucket to use.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # must set environment variable NLTK_DATA=/opt/python in the lambda function | |
| REGION=eu-west-1 | |
| RUNTIME=python3.7 | |
| BUCKET= | |
| VER=0.8.1 | |
| NUMPY_VER=1.17.4 | |
| OUT_DIR=/out/build/sumy/python |
haranjackson
/ pypi_deployment_workflow.yml
Last active
November 26, 2019 02:52
GitHub workflow for deploying a Python package to PyPI. The package can contain a CMake build step. Wheels are built for Linux, MacOS, and Windows, for Python versions 3.6-3.8. You must first create a GitHub secret called 'pypi_password' containing your PyPI API key.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: CI | |
| on: [push] | |
| jobs: | |
| nonlinux-build: | |
| strategy: | |
| fail-fast: false | |
| matrix: | |
| os: [macOS-latest, windows-latest] |
haranjackson
/ pdfminer_lambda_layer.sh
Created
August 30, 2019 12:18
Deploys Python PDFMiner library to an AWS Lambda layer (removing unnecessary test files to reduce size). You can specify the region, library version, and runtime.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| REGION=eu-west-1 | |
| VER=20181108 | |
| RUNTIME=python3.7 | |
| docker run -v $(pwd):/out -it lambci/lambda:build-$RUNTIME \ | |
| pip install pdfminer.six==$VER -t /out/build/pdfminer/python | |
| pushd build/pdfminer | |
| rm -rf python/Crypto/SelfTest/ | |
| zip -r ../../pdfminer.zip python/ |
haranjackson
/ chrome_headless.py
Last active
January 25, 2024 00:31
Deploys the Python Selenium library and Chrome Headless to an AWS Lambda layer. You can specify the region, library version, and runtime. An example Lambda function is given.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from selenium.webdriver import Chrome | |
| from selenium.webdriver.chrome.options import Options | |
| options = Options() | |
| options.binary_location = '/opt/headless-chromium' | |
| options.add_argument('--headless') | |
| options.add_argument('--no-sandbox') | |
| options.add_argument('--start-maximized') | |
| options.add_argument('--start-fullscreen') |
haranjackson
/ spacy_lambda_layer.sh
Created
August 15, 2019 14:33
Deploys Python Spacy library to an AWS Lambda layer. You can specify the region, library version, language, language model, and runtime.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| REGION=eu-west-1 | |
| VER=2.1.8 | |
| LANG=en | |
| MODEL=en_core_web_sm-2.1.0 | |
| RUNTIME=python3.7 | |
| MODEL_URL=https://github.com/explosion/spacy-models/releases/download/$MODEL/$MODEL.tar.gz | |
| docker run -v $(pwd):/out -it lambci/lambda:build-$RUNTIME \ | |
| pip install spacy==$VER $MODEL_URL -t /out/build/spacy/python |
haranjackson
/ scrapy_lambda_layer.sh
Last active
May 7, 2022 09:24
Deploys Python Scrapy library to an AWS Lambda layer. You can specify the region, library version, and runtime.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| REGION=eu-west-1 | |
| VER=1.7.3 | |
| RUNTIME=python3.7 | |
| docker run -v $(pwd):/out -it lambci/lambda:build-$RUNTIME \ | |
| pip install scrapy==$VER -t /out/build/scrapy/python | |
| cd build/scrapy | |
| zip -r ../../scrapy.zip python/ | |
| cd ../.. |
haranjackson
/ api_deployment.yaml
Created
August 11, 2019 16:48
An AWS CloudFormation template for a 2-stage (prod/dev) deployment on an API using API Gateway, with optional API key
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| AWSTemplateFormatVersion: 2010-09-09 | |
| Parameters: | |
| ApiId: | |
| Type: String | |
| UsagePlanName: | |
| Type: String | |
| Description: >- |
NewerOlder