Skip to content

Instantly share code, notes, and snippets.

View LouisAmon's full-sized avatar

Louis Amon LouisAmon

View GitHub Profile
@LouisAmon
LouisAmon / shovel.py
Created July 5, 2017 05:05 — forked from jdmaturen/shovel.py
SQS + Boto + Eventlet, re http://gist.github.com/434053
(jd@XXX) /home/jd/shovel> time ./shovel.py jd_test
got 901 messages
real 0m1.617s
user 0m0.558s
sys 0m0.044s
@LouisAmon
LouisAmon / serialize-numpy-array.py
Last active November 16, 2017 10:03 — forked from alexland/serialize-numpy-array.py
serialize, persist, retrieve, and de-serialize a NumPy array as a binary string (any dimension, any dtype); exemplary use case: a web app calculates some result--eg, from a Machine Learning algorithm, using NumPy and the result is a NumPy array; it is efficient to just return that result to rather than persist the array then retrieve it via query
import time
import numpy
import redis
# a 2D array to serialize
A = 10 * numpy.random.randn(10000).reshape(1000, 10)
# flatten the 2D NumPy array and save it as a binary string
@LouisAmon
LouisAmon / fanout.js
Created March 16, 2018 08:46
Fanout AWS Lambda
'use strict';
const aws = require('aws-sdk');
const lambda = new aws.Lambda({region: 'eu-west-1'});
const async = require('async');
exports.handler = (event, context, callback) => {
/*
Invokes a given Lambda function `event.function` in parallel for each given
argument `event.events`
@LouisAmon
LouisAmon / build.sh
Created April 21, 2018 11:52
Building an AWS Lambda package that includes `numpy`, `pandas` & `scipy`
# System dependencies
yum update -y
yum install -y \
binutils \
findutils \
python36
# Create directory for our lambda
mkdir build
pip install -t build numpy pandas scipy
# Check uncompressed file size
def trigo():
product = 1.0
for counter in range(1, 1000, 1):
for dex in list(range(1, 360, 1)):
angle = radians(dex)
product *= sin(angle)**2 + cos(angle)**2
return product
def sha():
h = hashlib.sha1('secret')
@LouisAmon
LouisAmon / lambda_handler.py
Created October 15, 2018 08:16
Run a system command in a Lambda
import subprocess
import logging
logger = logging.getLogger()
logger.setLevel(logging.INFO)
def lambda_handler(event, context):
"""
Runs a shell command and returns the output code
STDOUT & STDERR are logged into Cloudwatch
@LouisAmon
LouisAmon / s3_download_file_progress_bar.py
Created October 19, 2018 10:57 — forked from wy193777/s3_download_file_progress_bar.py
Show AWS s3 download_file Progress using tqdm
#python3
def hook(t):
def inner(bytes_amount):
t.update(bytes_amount)
return inner
BUCKET_NAME = 'your_s3_bucket_name'
FILE_NAME = 'your_s3_file_name'
s3 = boto3.resource('s3')
@LouisAmon
LouisAmon / filebase64sha256.py
Last active January 25, 2020 09:17
Terraform's `filebase64sha256` function, in Python (cf. https://www.terraform.io/docs/providers/aws/r/lambda_function.html)
import base64
import hashlib
def sha256sum(filename):
"""
Helper function that calculates the hash of a file
using the SHA256 algorithm
Inspiration:
https://stackoverflow.com/a/44873382
@LouisAmon
LouisAmon / example_provider.py
Created February 26, 2020 07:15
Pulumi Dynamic Provider (Python)
# This simple example illustrates how Nuage recommends you should build
# a Dynamic Provider in Pulumi
#
# The example is based on the official documentation:
# https://www.pulumi.com/docs/intro/concepts/programming-model/#example-github-labels-rest-api
#
# It defines a Github Label programmatically via Pulumi.
#
# The recommended directory structure is the following:
#
@LouisAmon
LouisAmon / mod10.py
Created May 18, 2021 14:35 — forked from emesik/mod10.py
python mod10 checker
def mod10(number):
digits = []
even = False
for digit in reversed(number):
digit = ord(digit) - ord('0')
if even:
digit = digit * 2
if digit >= 10:
digit = digit % 10 + digit / 10
digits.append(digit)