Adrian Drummond drumadrian
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from cmreslogging.handlers import CMRESHandler | |
| import logging | |
| import os | |
| import sys | |
| import time | |
| import boto3 | |
| from imdb import IMDb | |
| from botocore.credentials import RefreshableCredentials |
drumadrian
/ cleanup_amazon_s3_bucket.sh
Created
December 12, 2020 06:57
cleanup and delete amazon s3 bucket using ec2 user data for multiple spot instances
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| yum update -y | |
| aws s3 rb s3://yourbucket --force | |
| poweroff |
drumadrian
/ custom-cloudformation-bucket-cleanup.yaml
Created
April 2, 2019 21:10
How to Delete S3 Bucket Contents in CloudFormation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| AWSTemplateFormatVersion: "2010-09-09" | |
| Description: > | |
| This template builds a bucket and Custom Cloudformation resource that cleans up the bucket on delete | |
| Project site: https://github.com/drumadrian/custom-cloudformation-bucket-cleanup | |
| Parameters: | |
| ArtifactStoreBucketName: |
drumadrian
/ AWS S3 CLI speedup settings
Created
January 11, 2019 23:08
copy and paste the lines below into your PowerShell or Bash prompt for faster AWS S3 sync OR multi-part uploads.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # copy and paste the lines below into your PowerShell or Bash prompt for faster S3 sync OR multi-part uploads | |
| aws configure set default.s3.max_concurrent_requests 100 | |
| aws configure set default.s3.max_queue_size 20000 | |
| aws configure set default.s3.multipart_threshold 64MB | |
| aws configure set default.s3.multipart_chunksize 64MB | |
| aws configure set default.s3.max_bandwidth 50MB/s |
drumadrian
/ install_logstash_on_amazon_linux.bash
Last active
March 15, 2024 17:22
Sample logstash.conf file for S3 Input plugin
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # References: | |
| # https://www.elastic.co/guide/en/logstash/current/plugins-inputs-s3.html | |
| # https://www.elastic.co/blog/logstash-lines-inproved-resilience-in-S3-input | |
| # https://www.elastic.co/guide/en/logstash/6.3/installing-logstash.html | |
| # https://www.elastic.co/guide/en/logstash/current/working-with-plugins.html | |
| # https://www.garron.me/en/bits/curl-delete-request.html | |
| sudo yum update -y | |
| sudo yum install -y java-1.8.0-openjdk | |
| java -version |
drumadrian
/ ec2-host-from-tag-to-env-vars.sh
Created
September 10, 2018 09:58
— forked from marcellodesales/ec2-host-from-tag-to-env-vars.sh
Create Environment Variables in EC2 Hosts from EC2 Host Tags, just like Beanstalk or Heroku does!
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ###### | |
| # Author: Marcello de Sales (marcello.desales@gmail.com) | |
| # Description: Create Create Environment Variables in EC2 Hosts from EC2 Host Tags | |
| # | |
| ### Requirements: | |
| # * Install jq library (sudo apt-get install -y jq) | |
| # * Install the EC2 Instance Metadata Query Tool (http://aws.amazon.com/code/1825) | |
| # | |
| ### Installation: | |
| # * Add the Policy EC2:DescribeTags to a User |
drumadrian
/ Install Jenkins on AWS EC2 Instance
Created
December 7, 2017 10:47
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Install Jenkins on AWS EC2 instance | |
| # https://pkg.jenkins.io/redhat/ | |
| # https://d1.awsstatic.com/Projects/P5505030/aws-project_Jenkins-build-server.pdf | |
| # https://d1.awsstatic.com/Projects/P5505030/aws-project_Jenkins-build-server.pdf | |
| # https://gist.github.com/diegopacheco/6d69e0cfaf13d4351cfa700bb4af8172 | |
| # https://www.youtube.com/watch?v=uu5XcU4EPzQ | |
| sudo yum update |
drumadrian
/ setup_nvm_as_root
Created
July 17, 2017 05:28
Install NVM (globally) on Amazon Linux as root user
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| wget https://raw.githubusercontent.com/creationix/nvm/v0.33.2/install.sh /home/ec2-user/install.sh | |
| NVM_DIR=/usr/bin | |
| bash /home/ec2-user/install.sh | |
| mv /usr/bin/nvm.sh /usr/bin/nvm | |
| nvm install 4.3 |
drumadrian
/ LambdaStorm.py
Created
June 9, 2017 07:12
AWS Lambda and AWS API Gateway circular request Storm
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ################################################################################################## | |
| # Purpose: This function is triggerd to start a circular Lambda API Gateway storm | |
| # | |
| # Author: Adrian Drummond & Ben Harloe | |
| # | |
| # Date: June 8, 2017 (initial creation) | |
| # ? (updated) | |
| # | |
| # Triggered by: A manual Lambda function execution |