Skip to content

Instantly share code, notes, and snippets.

View wshayes's full-sized avatar

William Hayes wshayes

View GitHub Profile
wshayes /
Created October 29, 2020 20:10
Export (via webscraping) Intercom Article Help documents
#!/usr/bin/env python
# -*- coding: utf-8-*-
Webscrape Intercom Help Articles and export them into markdown and html as a JSON data file
This captures Collection info but not Sections. Images are downloaded into an images folder.
The images are renamed with the collection directory name (see the dir_map dict below) in numerical order.
You will need to dedupe duplicate images using another tool. Renaming the images avoids bad initial names
and duplicate image filenames (quick hack and not ideal - feel free to improve as desired.)
wshayes / poetry_dockerfile
Last active July 10, 2023 23:29
Poetry docker file that can support private git repository packages
# syntax=docker/dockerfile:experimental
FROM python:3.7-slim AS base
# ENV LANG=C.UTF-8 # Sets utf-8 encoding for Python et al
# ENV PYTHONDONTWRITEBYTECODE=1 # Turns off writing .pyc files; superfluous on an ephemeral container.
# ENV PYTHONUNBUFFERED=1 # Seems to speed things up
wshayes / secrets.yaml
Created April 14, 2020 18:37
[Github Actions Secrets Mgmt] Update github secrets for all repos in a github org
token: <Github Personal Access token with github actions/secrets scopes>
username: <github username>
- name: <secret_name>
value: <value>
description: <description>
wshayes / docker-compose.yaml
Created April 10, 2020 15:19
[Traefik SSL setup] #traefik #SSL
image: traefik:1.7
container_name: traefik
- 80:80
- 443:443
- --api
- --debug=false
- --logLevel=ERROR
wshayes /
Last active December 2, 2021 18:12
[Safe repr] safe recursive repr function #python
def safe_repr(obj, maxdepth=5):
if not maxdepth:
return '...'
obj_type = type(obj)
obj_type_type = type(obj_type)
newdepth = maxdepth - 1
# only represent exact builtins
wshayes /
Created January 19, 2020 17:06
[Websockets with fastapi] #fastapi #websockets
# You can probably use an async queue so your MQTT client will push messages to the queue and the WS server will get from the queue and send them to the WS client.
from asyncio import Queue
queue: Queue = None
async def websocket_endpoint(websocket: WebSocket):
await websocket.accept()
wshayes /
Created December 24, 2019 18:46
[python multiprocessing example] writing to file from a queue #python #multiprocessing
import multiprocessing as mp
import time
fn = 'c:/temp/temp.txt'
def worker(arg, q):
'''stupidly simulates long running process'''
start = time.clock()
wshayes /
Last active December 10, 2019 20:15
[python snippets] Python snippets that are useful #python

Tips and tricks


Skipping Begining of Iterable

Sometimes you have to work with files which you know that start with variable number of unwanted lines such as comments. itertools again provides easy solution to that:

wshayes /
Created October 28, 2019 14:43
[Route dependencies] #fastapi
from fastapi import FastAPI, Depends
from starlette.testclient import TestClient
app = FastAPI()
def capture_exception(exc: Exception) -> None:
wshayes /
Created August 2, 2019 11:07
[Pydantic subclassing] camelcasing aliases and json dumps #fastapi #pydantic
# @Rehket for what it's worth, I recommend subclassing BaseModel and using that as the root for all of your API models (to reduce code repetition). For example, I use a base class that mostly looks like this:
import re
from functools import partial
from typing import Any, Dict
from fastapi.encoders import jsonable_encoder
from pydantic import BaseConfig, BaseModel