Skip to content

Instantly share code, notes, and snippets.

View edthrn's full-sized avatar

ed edthrn

View GitHub Profile
@edthrn
edthrn / 1-Unit-testing with PyGithub.md
Last active October 19, 2023 05:09
Unit-testing with PyGithub

Intro

Lately, I've been building a Python platform that relies heavily on interactions with Github. Instead of re-inventing the wheel, I decided to go with PyGithub as a wrapper around Github v3 API.

The problem

This library implements the full API, which allows me to be very productive when I need to add a workflow in my platform that involves Github. However, it quickly became a real PITA to write unit tests. Even if the package comes with its own testing Framework, it is not documented yet and I didn't manage to crack it up in a decent amount of time.

I decided to hack the testing a little bit, using another very cool package, httpretty. Httpretty allows you to monkey patch the socket module during testing, so you can respond anything you want to any kind of network requests. Here's what I came up with, do not hesitate to give any feedback.


@edthrn
edthrn / execute.py
Last active January 26, 2023 18:08
Execute Shell command on EC2 Linux instance with Python and Boto3
# Following https://stackoverflow.com/questions/34028219/how-to-execute-commands-on-aws-instance-using-boto3
import boto3
ssm = boto3.client('ssm')
response = ssm.send_command(
InstanceIds=['i-abcde12345...'],
DocumentName='AWS-RunShellScript',
Parameters={'commands': ['echo "This command ran on $(date)"']
)
@edthrn
edthrn / remove-from-git.sh
Created February 10, 2021 20:15
Remove old data from Git repo and Git history
# Taken from https://docs.github.com/en/github/authenticating-to-github/removing-sensitive-data-from-a-repository
# Warning:
# --------
# All stashes may be lost!
PATH_TO_REMOVE=big-folder/commited/by/error
# 1. Force Git to process, but not check out, the entire history of every branch and tag
# 2. Remove the specified file, as well as any empty commits generated as a result
@edthrn
edthrn / redis.clj
Created February 20, 2021 16:27 — forked from ragnard/redis.clj
Using Redis for persistent memoization of Clojure functions
(ns util.redis
(:refer-clojure :exclude [memoize])
(:require [taoensso.carmine :as car]))
;; boilerplate stuff that is not in Carmine
(def ^:dynamic ^:private *pool*)
(def ^:dynamic ^:private *spec*)
(defmacro with-redis
@edthrn
edthrn / pw.sh
Last active November 20, 2019 20:31
Put a password stored in `1password` to clipboard
# Usage: ./pw.sh Github.com
op get item $1 | jq -r '.details.fields[] | select(.designation=="password").value' | xclip -selection c
@edthrn
edthrn / humansorted.py
Last active August 16, 2019 15:57
Sorting for humans with Python
# Inspired by https://blog.codinghorror.com/sorting-for-humans-natural-sort-order/
# Adding a `key` argument so we can expect the same behaviour as the builtin `sorted`
import re
def humansorted(seq, key=None):
"""Return the sequence in the order that a human being expects.
If `key` is provided, as per the built-in `sorted` function, it must be a
callable that accepts a single argument.
@edthrn
edthrn / sync-postgres-db.md
Last active July 23, 2019 00:45
Sync content from one Postgres database to another

Say we have a database A hosted at my.first.database.com loaded with data, and we want to replicate this data over database B, located at my.second.database.com. We suppose database A and B have identical schemas, and both databases are accessible via port 5432.

Dump content from database A

 pg_dump --data-only \
 -h my.first.database.com \
 -U {database_user} \
 -f /tmp/dump.sql \
 -T={table pattern to exclude [optional]} \
@edthrn
edthrn / execute_shell_ec2.py
Last active June 23, 2019 16:57
Execute shell commands in EC2 with SSM
"""
https://aws.amazon.com/blogs/aws/manage-instances-at-scale-without-ssh-access-using-ec2-run-command/
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ssm.html
https://stackoverflow.com/questions/34028219/how-to-execute-commands-on-aws-instance-using-boto3
"""
import boto3
# Need AWS credentials in ~/.aws...
ssm = boto3.client('ssm')
@edthrn
edthrn / TODO.md
Last active May 13, 2019 09:48
TODO

Guidelines

Implement a Vector class. It should take an undefenite amount of parameters and behave like this:

>>> v1 = Vector(5, 9, -8, 2, 13, -23, 0, 0, 12)
>>> v2 = Vector(6, 8)
>>> v3 = Vector(-1, 1, 1, -1)
>>> v4 = Vector(0, 0, 0)
@edthrn
edthrn / install-hadoop.sh
Last active April 10, 2019 00:09
A Bash script to automate Hadoop installation.
#!/bin/bash
# MIT License
# Copyright (c) 2019 nibble.ai
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is