haranjackson
/ revolut_statement.py
Created
April 10, 2018 10:15
Convert a Revolut HTML statement into a Pandas dataframe
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from pandas import read_csv | |
| # Enter path to file here | |
| fname = '' | |
| def convert_to_floats(col): | |
| return col.apply(lambda x: float(x.replace(',', '')) if x != '' else 0) |
haranjackson
/ apigateway_with_ec2.yaml
Last active
May 22, 2019 15:38
An AWS CloudFormation template for creating an API using API Gateway, with an EC2 backend. As an example, API Gateway's /api_endpoint points to the EC2's /ec2_endpoint.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| AWSTemplateFormatVersion: 2010-09-09 | |
| Transform: AWS::Serverless-2016-10-31 | |
| Parameters: | |
| VpcId: | |
| Type: String | |
| Description: The ID of the VPC containing the EC2 instance | |
| InstanceAZ: |
haranjackson
/ https_s3_website.sh
Last active
April 4, 2024 13:38
An AWS CloudFormation template for a static website hosted on S3, served over HTTPS with CloudFront. "www." redirects to the naked domain.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| DOMAIN= # insert your domain here (e.g. example.com) | |
| STACK= # choose a name for your stack | |
| REGION=us-east-1 # the ACM certificate must be in us-east-1 | |
| aws cloudformation deploy --template-file https_s3_website.yaml \ | |
| --stack-name $STACK \ | |
| --region $REGION \ | |
| --parameter-overrides DomainName=$DOMAIN | |
| # push the website source to the s3 bucket - assuming it is contained in src/ |
haranjackson
/ nltk_punkt_lambda_layer.sh
Last active
June 18, 2019 13:31
A bash script for creating a layer on AWS Lambda (Python 3.7) that includes NLTK and the Punkt sentence tokenizer
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| LAYER_NAME=NltkPunkt | |
| REGION=us-east-1 | |
| mkdir -p build/nltk_punkt/python | |
| docker run -v $(pwd):/out lambci/lambda:build-python3.7 /bin/bash -c \ | |
| "pip install nltk -t /out/build/nltk_punkt/python; | |
| cd /out/build/nltk_punkt/python; | |
| python -c \"import nltk; nltk.download('punkt', '/out/build/nltk_punkt/python')\"; | |
| rm tokenizers/punkt.zip; |
haranjackson
/ transfer_ddb.py
Created
July 5, 2019 22:34
Transfers the all the entries from one DynamoDB table to another
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from boto3 import resource | |
| OLD_TABLE = '' | |
| NEW_TABLE = '' | |
| OLD_REGION = '' | |
| NEW_REGION = '' | |
| def put_new_items(response, newTable): |
haranjackson
/ aws_migrate_elasticsearch.py
Last active
September 10, 2019 23:54
Migrates all indices (except .kibana) from an old AWS-hosted ES domain to a new one. Set up an S3 bucket and IAM role beforehand, as described here: https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-managedomains-snapshots.html#es-managedomains-snapshot-prerequisites
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import requests | |
| from time import sleep, time | |
| from boto3 import Session | |
| from requests_aws4auth import AWS4Auth | |
| OLD_ES_ENDPOINT = '' | |
| NEW_ES_ENDPOINT = '' | |
| INDEX = '' # e.g. 'index1', or 'index1,index2,index3', or '*' |
haranjackson
/ api_custom_domain.yaml
Created
August 11, 2019 16:39
An AWS CloudFormation template for adding a custom domain to an API deployed with API Gateway
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| AWSTemplateFormatVersion: 2010-09-09 | |
| Parameters: | |
| ApiId: | |
| Type: String | |
| Description: The ID of the API for which to create the custom domain | |
| ApiStage: | |
| Type: String |
haranjackson
/ api_endpoint.yaml
Created
August 11, 2019 16:44
An AWS CloudFormation template for an API Gateway GET/POST endpoint, backed by a Lambda function
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| AWSTemplateFormatVersion: 2010-09-09 | |
| Parameters: | |
| ApiId: | |
| Type: String | |
| Description: The ID of API to which this endpoint should be added | |
| FunctionArn: | |
| Type: String |
haranjackson
/ api_deployment.yaml
Created
August 11, 2019 16:48
An AWS CloudFormation template for a 2-stage (prod/dev) deployment on an API using API Gateway, with optional API key
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| AWSTemplateFormatVersion: 2010-09-09 | |
| Parameters: | |
| ApiId: | |
| Type: String | |
| UsagePlanName: | |
| Type: String | |
| Description: >- |
haranjackson
/ scrapy_lambda_layer.sh
Last active
May 7, 2022 09:24
Deploys Python Scrapy library to an AWS Lambda layer. You can specify the region, library version, and runtime.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| REGION=eu-west-1 | |
| VER=1.7.3 | |
| RUNTIME=python3.7 | |
| docker run -v $(pwd):/out -it lambci/lambda:build-$RUNTIME \ | |
| pip install scrapy==$VER -t /out/build/scrapy/python | |
| cd build/scrapy | |
| zip -r ../../scrapy.zip python/ | |
| cd ../.. |
OlderNewer