Skip to content

Instantly share code, notes, and snippets.

View iMilnb's full-sized avatar

iMil iMilnb

View GitHub Profile
@iMilnb
iMilnb / README.md
Last active January 18, 2024 08:08
AWS Terraform configuration: Stream CloudWatch Logs to ElasticSearch

Rationale

This snippet is a sample showing how to implement CloudWatch Logs streaming to ElasticSearch using terraform. I wrote this gist because I didn't found a clear, end-to-end example on how to achieve this task. In particular, I understood the resource "aws_lambda_permission" "cloudwatch_allow" part by reading a couple of bug reports plus this stackoverflow post.

The js file is actually the Lambda function automatically created by AWS when creating this pipeline through the web console. I only added a endpoint variable handling so it is configurable from terraform.

@iMilnb
iMilnb / snsread.py
Created August 2, 2015 20:20
Basic Flask snippet to handle AWS SNS messages and subscription
from flask import Flask, request
import requests
import json
app = Flask(__name__)
def msg_process(msg, tstamp):
js = json.loads(msg)
msg = 'Region: {0} / Alarm: {1}'.format(
js['Region'], js['AlarmName']
@iMilnb
iMilnb / gist:7604f04223d44a3e13af
Last active June 17, 2023 16:14
Keep track of Twitter "unfollowers"
#!/bin/sh
# track unfollows on twitter using "t" https://github.com/sferik/t
# usage:
#
# to check if any of your followers has stopped following you since the last
# time the script wall called, use:
# $ this_script followers
#
# to check if YOU stopped following someone (it appears many of us witnessed
@iMilnb
iMilnb / boto3_hands_on.md
Last active October 19, 2022 09:15
Programmatically manipulate AWS resources with boto3 - a quick hands on

boto3 quick hands-on

This documentation aims at being a quick-straight-to-the-point-hands-on AWS resources manipulation with [boto3][0].

First of all, you'll need to install [boto3][0]. Installing it along with [awscli][1] is probably a good idea as

  • [awscli][1] is boto-based
  • [awscli][1] usage is really close to boto's
@iMilnb
iMilnb / getmonit.py
Last active February 11, 2022 08:24
Fetch monit XML status URL content, transform it to JSON and display a status report
#!/usr/bin/env python
import requests
import xmltodict
import json
import os
import sys
with open('{0}/.getmonitrc'.format(os.path.expanduser('~'))) as f:
cf = json.loads(f.read())
@iMilnb
iMilnb / ec2.py
Created May 27, 2015 12:29
AWS EC2 simple manipulation script using python and boto3
#!/usr/bin/env python
# Simple [boto3](https://github.com/boto/boto3) based EC2 manipulation tool
#
# To start an instance, create a yaml file with the following format:
#
# frankfurt:
# - subnet-azb:
# - type: t2.micro
# image: image-tagname
@iMilnb
iMilnb / meta2dict.py
Last active February 9, 2020 02:49
Imports AWS EC2 instance metadata (http://169.254.169.254/latest) into a python dict
#!/usr/bin/env python
import requests
import json
# Converts AWS EC2 instance metadata to a dictionary
def load():
metaurl = 'http://169.254.169.254/latest'
# those 3 top subdirectories are not exposed with a final '/'
@iMilnb
iMilnb / tf_debug.log
Created December 6, 2018 14:09
terraform debug for aws_ses_receipt_rule bug report
terraform apply tfplan
2018/12/06 14:44:36 [INFO] Terraform version: 0.11.10 17850e9a55d33c43d7c31fd6ac122ba97a51d899
2018/12/06 14:44:36 [INFO] Go runtime version: go1.11.1
2018/12/06 14:44:36 [INFO] CLI args: []string{"/home/imil/pkg/bin/terraform", "apply", "tfplan"}
2018/12/06 14:44:36 [DEBUG] Attempting to open CLI config file: /home/imil/.terraformrc
2018/12/06 14:44:36 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2018/12/06 14:44:36 [INFO] CLI command args: []string{"apply", "tfplan"}
2018/12/06 14:44:36 [INFO] command: initializing local backend from plan (not set)
2018/12/06 14:44:36 [DEBUG] checking for provider in "."
2018/12/06 14:44:36 [DEBUG] checking for provider in "/home/imil/pkg/bin"
@iMilnb
iMilnb / collectd.conf
Last active October 10, 2018 21:33
collectd-python plugin to read data from ethermine.org
<Plugin python>
ModulePath "/home/imil/collectd"
Import "ethermine"
<Module ethermine>
wallet "0xf00f00f00"
interval "300"
url "https://ethermine.org/api/miner_new"
</Module>
</Plugin>
@iMilnb
iMilnb / aws_sandbox.md
Last active September 26, 2018 16:30
Custom AWS sandbox using packer, Troposphere / CloudFormation and Ansible

packer template

{
  "variables": {
    "aws_access_key": "",
    "aws_secret_key": ""
  },
  "builders": [{
    "type": "amazon-ebs",