haranjackson
/ revolut_statement.py
Created
April 10, 2018 10:15
Convert a Revolut HTML statement into a Pandas dataframe
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from pandas import read_csv | |
| # Enter path to file here | |
| fname = '' | |
| def convert_to_floats(col): | |
| return col.apply(lambda x: float(x.replace(',', '')) if x != '' else 0) |
haranjackson
/ apigateway_with_ec2.yaml
Last active
May 22, 2019 15:38
An AWS CloudFormation template for creating an API using API Gateway, with an EC2 backend. As an example, API Gateway's /api_endpoint points to the EC2's /ec2_endpoint.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| AWSTemplateFormatVersion: 2010-09-09 | |
| Transform: AWS::Serverless-2016-10-31 | |
| Parameters: | |
| VpcId: | |
| Type: String | |
| Description: The ID of the VPC containing the EC2 instance | |
| InstanceAZ: |
haranjackson
/ nltk_punkt_lambda_layer.sh
Last active
June 18, 2019 13:31
A bash script for creating a layer on AWS Lambda (Python 3.7) that includes NLTK and the Punkt sentence tokenizer
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| LAYER_NAME=NltkPunkt | |
| REGION=us-east-1 | |
| mkdir -p build/nltk_punkt/python | |
| docker run -v $(pwd):/out lambci/lambda:build-python3.7 /bin/bash -c \ | |
| "pip install nltk -t /out/build/nltk_punkt/python; | |
| cd /out/build/nltk_punkt/python; | |
| python -c \"import nltk; nltk.download('punkt', '/out/build/nltk_punkt/python')\"; | |
| rm tokenizers/punkt.zip; |
haranjackson
/ transfer_ddb.py
Created
July 5, 2019 22:34
Transfers the all the entries from one DynamoDB table to another
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from boto3 import resource | |
| OLD_TABLE = '' | |
| NEW_TABLE = '' | |
| OLD_REGION = '' | |
| NEW_REGION = '' | |
| def put_new_items(response, newTable): |
haranjackson
/ api_custom_domain.yaml
Created
August 11, 2019 16:39
An AWS CloudFormation template for adding a custom domain to an API deployed with API Gateway
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| AWSTemplateFormatVersion: 2010-09-09 | |
| Parameters: | |
| ApiId: | |
| Type: String | |
| Description: The ID of the API for which to create the custom domain | |
| ApiStage: | |
| Type: String |
haranjackson
/ api_endpoint.yaml
Created
August 11, 2019 16:44
An AWS CloudFormation template for an API Gateway GET/POST endpoint, backed by a Lambda function
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| AWSTemplateFormatVersion: 2010-09-09 | |
| Parameters: | |
| ApiId: | |
| Type: String | |
| Description: The ID of API to which this endpoint should be added | |
| FunctionArn: | |
| Type: String |
haranjackson
/ api_deployment.yaml
Created
August 11, 2019 16:48
An AWS CloudFormation template for a 2-stage (prod/dev) deployment on an API using API Gateway, with optional API key
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| AWSTemplateFormatVersion: 2010-09-09 | |
| Parameters: | |
| ApiId: | |
| Type: String | |
| UsagePlanName: | |
| Type: String | |
| Description: >- |
haranjackson
/ aws_migrate_elasticsearch.py
Last active
September 10, 2019 23:54
Migrates all indices (except .kibana) from an old AWS-hosted ES domain to a new one. Set up an S3 bucket and IAM role beforehand, as described here: https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-managedomains-snapshots.html#es-managedomains-snapshot-prerequisites
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import requests | |
| from time import sleep, time | |
| from boto3 import Session | |
| from requests_aws4auth import AWS4Auth | |
| OLD_ES_ENDPOINT = '' | |
| NEW_ES_ENDPOINT = '' | |
| INDEX = '' # e.g. 'index1', or 'index1,index2,index3', or '*' |
haranjackson
/ pypi_deployment_workflow.yml
Last active
November 26, 2019 02:52
GitHub workflow for deploying a Python package to PyPI. The package can contain a CMake build step. Wheels are built for Linux, MacOS, and Windows, for Python versions 3.6-3.8. You must first create a GitHub secret called 'pypi_password' containing your PyPI API key.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: CI | |
| on: [push] | |
| jobs: | |
| nonlinux-build: | |
| strategy: | |
| fail-fast: false | |
| matrix: | |
| os: [macOS-latest, windows-latest] |
haranjackson
/ sumy_lambda_layer.sh
Created
November 29, 2019 00:20
Deploys Python sumy library to an AWS Lambda layer. You can specify the region, library version, and runtime. You must specify a temporary S3 bucket to use.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # must set environment variable NLTK_DATA=/opt/python in the lambda function | |
| REGION=eu-west-1 | |
| RUNTIME=python3.7 | |
| BUCKET= | |
| VER=0.8.1 | |
| NUMPY_VER=1.17.4 | |
| OUT_DIR=/out/build/sumy/python |
OlderNewer