Skip to content

Instantly share code, notes, and snippets.

View edthrn's full-sized avatar

ed edthrn

View GitHub Profile
@edthrn
edthrn / Makefile
Created July 26, 2018 20:44 — forked from mpneuried/Makefile
Simple Makefile to build, run, tag and publish a docker containier to AWS-ECR
# import config.
# You can change the default config with `make cnf="config_special.env" build`
cnf ?= config.env
include $(cnf)
export $(shell sed 's/=.*//' $(cnf))
# import deploy config
# You can change the default deploy config with `make cnf="deploy_special.env" release`
dpl ?= deploy.env
include $(dpl)
@edthrn
edthrn / cognito-auth.js
Last active August 15, 2018 17:11
A sample use case of sending a message to an API protected by AWS Cognito.
function ajax(method, url, data, callback) {
var req = new XMLHttpRequest();
req.open(method, url);
req.addEventListener("load", function () {
if (req.status >= 200 && req.status < 400) {
callback();
} else {
console.error(req.status + " " + req.statusText + " " + url);
}
@edthrn
edthrn / 1-Unit-testing with PyGithub.md
Last active October 19, 2023 05:09
Unit-testing with PyGithub

Intro

Lately, I've been building a Python platform that relies heavily on interactions with Github. Instead of re-inventing the wheel, I decided to go with PyGithub as a wrapper around Github v3 API.

The problem

This library implements the full API, which allows me to be very productive when I need to add a workflow in my platform that involves Github. However, it quickly became a real PITA to write unit tests. Even if the package comes with its own testing Framework, it is not documented yet and I didn't manage to crack it up in a decent amount of time.

I decided to hack the testing a little bit, using another very cool package, httpretty. Httpretty allows you to monkey patch the socket module during testing, so you can respond anything you want to any kind of network requests. Here's what I came up with, do not hesitate to give any feedback.


@edthrn
edthrn / reduce_pdf_size.sh
Created April 3, 2019 16:48
Reduce PDF size from Ubuntu command-line
#!/bin/sh
# From https://askubuntu.com/a/626301
gs \
-sDEVICE=pdfwrite \
-dCompatibilityLever=1.4 \
-dPDFSETTINGS=/default \
-dNOPAUSE \
-dQUIET \
-dBATCH \
@edthrn
edthrn / install-hadoop.sh
Last active April 10, 2019 00:09
A Bash script to automate Hadoop installation.
#!/bin/bash
# MIT License
# Copyright (c) 2019 nibble.ai
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
@edthrn
edthrn / TODO.md
Last active May 13, 2019 09:48
TODO

Guidelines

Implement a Vector class. It should take an undefenite amount of parameters and behave like this:

>>> v1 = Vector(5, 9, -8, 2, 13, -23, 0, 0, 12)
>>> v2 = Vector(6, 8)
>>> v3 = Vector(-1, 1, 1, -1)
&gt;&gt;&gt; v4 = Vector(0, 0, 0)
@edthrn
edthrn / execute_shell_ec2.py
Last active June 23, 2019 16:57
Execute shell commands in EC2 with SSM
"""
https://aws.amazon.com/blogs/aws/manage-instances-at-scale-without-ssh-access-using-ec2-run-command/
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ssm.html
https://stackoverflow.com/questions/34028219/how-to-execute-commands-on-aws-instance-using-boto3
"""
import boto3
# Need AWS credentials in ~/.aws...
ssm = boto3.client('ssm')
@edthrn
edthrn / sync-postgres-db.md
Last active July 23, 2019 00:45
Sync content from one Postgres database to another

Say we have a database A hosted at my.first.database.com loaded with data, and we want to replicate this data over database B, located at my.second.database.com. We suppose database A and B have identical schemas, and both databases are accessible via port 5432.

Dump content from database A

 pg_dump --data-only \
 -h my.first.database.com \
 -U {database_user} \
 -f /tmp/dump.sql \
 -T={table pattern to exclude [optional]} \
@edthrn
edthrn / execute.py
Last active January 26, 2023 18:08
Execute Shell command on EC2 Linux instance with Python and Boto3
# Following https://stackoverflow.com/questions/34028219/how-to-execute-commands-on-aws-instance-using-boto3
import boto3
ssm = boto3.client('ssm')
response = ssm.send_command(
InstanceIds=['i-abcde12345...'],
DocumentName='AWS-RunShellScript',
Parameters={'commands': ['echo "This command ran on $(date)"']
)
@edthrn
edthrn / humansorted.py
Last active August 16, 2019 15:57
Sorting for humans with Python
# Inspired by https://blog.codinghorror.com/sorting-for-humans-natural-sort-order/
# Adding a `key` argument so we can expect the same behaviour as the builtin `sorted`
import re
def humansorted(seq, key=None):
"""Return the sequence in the order that a human being expects.
If `key` is provided, as per the built-in `sorted` function, it must be a
callable that accepts a single argument.