Skip to content

Instantly share code, notes, and snippets.

Amal G Jose amalgjose

Block or report user

Report or block amalgjose

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@amalgjose
amalgjose / background_function.py
Last active Jul 18, 2018
An example of execution a function in the background. The main execution will continue in the same flow without waiting for the background function to complete and the function set for background will continue its execution in the background
View background_function.py
import time
import threading
def background(f):
'''
a threading decorator
use @background above the function you want to run in the background
'''
def backgrnd_func(*a, **kw):
threading.Thread(target=f, args=a, kwargs=kw).start()
@amalgjose
amalgjose / AirflowDocker
Created Feb 22, 2018
Snippet for launching Airflow docker
View AirflowDocker
sudo docker run --name airflow-db -e MYSQL_ROOT_PASSWORD=airflow-db -d -v /opt/airflow/airflow-db:/var/lib/mysql mysql
sudo docker run --name airflow --link ailens-airflow-db:mysql -v /opt/airflow/dags:/airflow/dags -v /opt/airflow/logs:/airflow/logs -p 24004:8080 -p 24005:5555 -d drunkar/airflow
@amalgjose
amalgjose / generate_signed_url.py
Last active Jan 2, 2018
This code snippet generates a time-bound signed URL for an S3 object.
View generate_signed_url.py
import boto3
from urlparse import urlparse
__aws_access_key__ = "XXXXXXXXXXXXXXX"
__aws_secret_key__ = "XXXXXXXXXXXXX"
def s3_client():
"""
This method establishes the connection to S3.
@amalgjose
amalgjose / get_account_id.py
Last active Jan 1, 2018
Code snippet to find the account id.
View get_account_id.py
import boto3
client = boto3.client("sts", aws_access_key_id="XXXXXXXX", aws_secret_access_key="XXXXXXXX")
account_id = client.get_caller_identity()["Account"]
print account_id
@amalgjose
amalgjose / GetEC2Instances.py
Created Mar 8, 2017
Simple python code snippet to get the list of all the running ec2 instances across regions. The code will dynamically pick the list of available regions. So this will work perfectly without a code change even if a new region gets added.
View GetEC2Instances.py
import boto3
access_key = "XXXXXXXXXXXXXXXXXX"
secret_key = "XXXXXXXXXXXXXXXXXX"
client = boto3.client('ec2', aws_access_key_id=access_key, aws_secret_access_key=secret_key,
region_name='us-east-1')
ec2_regions = [region['RegionName'] for region in client.describe_regions()['Regions']]
@amalgjose
amalgjose / EC2Reboot.py
Created Feb 27, 2016
Python program to reboot multiple ec2 instances together. This can be used for performing some scheduled restart of EC2 machines in an Amazon account. I used python boto library for performing this operation.
View EC2Reboot.py
__author__ = 'Amal G Jose'
import boto.ec2
class EC2Reboot(object):
def __init__(self):
self.instance_id_list = ["i-xxxxxx", "i-xxxxxx", "i-xxxxx", "i-xxxx", "i-xxxxx", "i-xxxxx", "i-xxxxx"]
self.aws_access_key = "XXXXXXXXXXX"
self.aws_secret_key = "XXXXXXXXXXX"
@amalgjose
amalgjose / EtagExample.py
Created Jan 3, 2016
Python program to find the checksum and Etag of a file. Etag is a property of a file in S3 which is similar to checksum
View EtagExample.py
import os
import boto
import traceback
from boto.s3.connection import S3Connection
class ETagExample(object):
def __init__(self):
self.aws_access_key = "XXXXXXXXX"
self.aws_secret_key = "XXXXXXXXX"
@amalgjose
amalgjose / GetRedShiftDetails.py
Created Oct 11, 2015
Program to list all the redshift clusters across all regions.
View GetRedShiftDetails.py
__author__ = 'Amal G Jose'
import boto
class GetRedShiftDetails(object):
def __init__(self):
self.aws_access_key = "XXXXXXXX"
self.aws_secret_key = "XXXXXXXX"
@amalgjose
amalgjose / GetAWSSummary.py
Last active Jan 19, 2016
Program to find get the summary of EMR clusters and EC2 instances running across all regions in an AWS account. This program can be executed via cron to get summary alerts in periodic intervals. An HTML email will be generated and send to the list of recipients mentioned in program. In this program, EMR and EC2 service summary are only considere…
View GetAWSSummary.py
__author__ = 'Amal G Jose'
import sys
import smtplib
import time
import boto
import boto.ses
from email.mime.text import MIMEText
from email.mime.application import MIMEApplication
@amalgjose
amalgjose / TornadoSSL.py
Last active Sep 1, 2015
Sample service using python tornado with SSL.
View TornadoSSL.py
__author__ = 'Amal G Jose'
import os
import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web
class HelloWorldHandler(tornado.web.RequestHandler):
You can’t perform that action at this time.