Skip to content

Instantly share code, notes, and snippets.

Avatar

Kurt Griffiths kgriffs

View GitHub Profile
@kgriffs
kgriffs / async-does-not-mix.py
Last active November 18, 2022 21:22
Example demonstrating how a synchronous control flow never yields to tasks on the event loop, starving them.
View async-does-not-mix.py
import asyncio
import httpx
import time
async def do_work_async():
http = httpx.AsyncClient()
while True:
resp = await http.get('https://httpbin.org/status/202')
@kgriffs
kgriffs / example_asyncio_signal_shutdown.py
Last active November 2, 2022 01:52
Example showing how to trap a signal with a Python asyncio coroutine function and exit cleanly without logging an error.
View example_asyncio_signal_shutdown.py
import asyncio
import functools
import signal
import sys
import time
async def worker(shutdown_event):
try:
while True:
@kgriffs
kgriffs / aiobotocore-config-example.py
Last active November 7, 2022 16:39
aiobotocore config example (vs. boto3) for setting connect_timeout and usign the standard retries mode
View aiobotocore-config-example.py
import asyncio
import aiobotocore.session
import aiobotocore.config
async def example():
# Equivalent to boto3.client('s3', config=botocore.client.Config(**kwargs))
# See also for available options:
# * https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html
@kgriffs
kgriffs / boto3async.py
Last active November 1, 2022 17:43
asyncio boto3 wrapper to run boto3 client methods in a thread pool executor as an alternative to aiobotocore
View boto3async.py
# =============================================================================
# Copyright 2022 by Rafid Al-Humaimidi. All rights reserved.
# Licensed via Apache 2.0 (https://www.apache.org/licenses/LICENSE-2.0)
#
# Forked from: https://github.com/rafidka/boto3async
# =============================================================================
"""Adds simple async wrappers around boto3 client methods.
This module adds async methods to the stock boto3 clients. The async versions of
@kgriffs
kgriffs / example.sh
Last active August 24, 2022 03:18
Start redis server without disk persistence; memory only for testing (avoids having to clean up a dump.rdb file)
View example.sh
redis-server --save "" --appendonly no
@kgriffs
kgriffs / listening.sh
Created July 29, 2022 22:01
macOS port listening bash
View listening.sh
# https://stackoverflow.com/a/30029855/21784
listening() {
if [ $# -eq 0 ]; then
sudo lsof -iTCP -sTCP:LISTEN -n -P
elif [ $# -eq 1 ]; then
sudo lsof -iTCP -sTCP:LISTEN -n -P | grep -i --color $1
else
echo "Usage: listening [pattern]"
fi
}
@kgriffs
kgriffs / recipe.py
Created July 28, 2022 18:36
Falcon recipe: Log an error whenever an error status code is returned
View recipe.py
import logging
import falcon
import falcon.asgi
import falcon.util
class HelloResource:
async def on_get(self):
pass
@kgriffs
kgriffs / httpx_multiprocessing.py
Last active July 25, 2022 18:39
Using httpx with multiprocessing
View httpx_multiprocessing.py
import asyncio
import multiprocessing
import httpx
class Publisher:
def __init__(self):
# NOTE(kgriffs): Explicitly manage the client so we can use the
# connection pool. This class could be made into a singleton
@kgriffs
kgriffs / lru_cache_ttl.py
Created July 18, 2022 20:29
Grant Jenks' LRU Cache with TTL for Python
View lru_cache_ttl.py
# Grant Jenks' LRU Cache with TTL for Python
#
# https://stackoverflow.com/questions/31771286/python-in-memory-cache-with-time-to-live/71634221#71634221
from functools import lru_cache, wraps
from time import monotonic
def lru_cache_with_ttl(maxsize=128, typed=False, ttl=60):
"""Least-recently used cache with time-to-live (ttl) limit."""
@kgriffs
kgriffs / compressed-multipart-python-requests.sh
Last active July 8, 2022 20:30
Compressed multipart form (file) upload using Python requests.
View compressed-multipart-python-requests.sh
In [16]: data = {'some': 'data'}
In [17]: data_compressed = gzip.compress(json.dumps(data, ensure_ascii=False).encode())
In [18]: requests.post("https://httpbin.org/anything", files={"msg": ('message.json.gz', body_compressed)}).json()
Out[18]:
{'args': {},
'data': '',
'files': {'msg': 'data:application/octet-stream;base64,H4sIAGeLyGIC/6tWKs7PTVWyUlBKSSxJVKoFABmPLCMQAAAA'},