Skip to content

Instantly share code, notes, and snippets.

@mihow
mihow / example_matching_rows.sql
Created January 25, 2024 01:24
5 example matching rows for each category in a joined table (PostgreSQL)
SELECT c.category_name, i.image_id FROM categories c
CROSS JOIN LATERAL (
SELECT image_id
FROM image_hashes v
WHERE v.category_id = c.category_id
LIMIT 5
) i;
@mihow
mihow / image_from_url.py
Created September 11, 2023 23:10
Load image into PIL object from URL with streaming request, one line
import requests
from PIL import Image
image = Image.open(requests.get(url, stream=True).raw)
@mihow
mihow / send_get_as_post.py
Created March 14, 2023 01:16
Send a GET request as a POST to overcome 414 Request-URI Too Large
# Submit a "GET" request via POST so we can send
# more data than fits in a URL
resp = requests.post(
url,
headers = {'X-HTTP-Method-Override': 'GET'},
data=params)
@mihow
mihow / parallel_download.sh
Last active February 4, 2023 00:08
Bash Parallel File Download
#!/bin/bash
# USAGE
# bash ./parallel_download.sh url_list.txt
set -o nounset
set -o errexit
FILELIST=$1
@mihow
mihow / get_table_sizes.sql
Created January 28, 2023 00:11
PostgreSQL - Get size of all tables
SELECT *, pg_size_pretty(total_bytes) AS total
, pg_size_pretty(index_bytes) AS index
, pg_size_pretty(toast_bytes) AS toast
, pg_size_pretty(table_bytes) AS table
FROM (
SELECT *, total_bytes-index_bytes-coalesce(toast_bytes,0) AS table_bytes FROM (
SELECT c.oid,nspname AS table_schema, relname AS table_name
, c.reltuples AS row_estimate
, pg_total_relation_size(c.oid) AS total_bytes
, pg_indexes_size(c.oid) AS index_bytes
@mihow
mihow / python_logging.py
Last active August 23, 2022 00:06
Example python logging configs
"""
This works when running code as script, module and with doctests.
Also sets the log level for imported packages like requests.
python -m mymodule.myfile
python mymodule/myfile.py
python -m unittest mymodule/myfile.py
"""
import logging
@mihow
mihow / install_docker.sh
Last active July 8, 2022 00:07
Install Docker on Ubuntu (Tested on 20.04 LTS Focal)
#! /bin/bash
set -o errexit
set -o nounset
set -o xtrace
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list
sudo apt update
@mihow
mihow / ssh-agent-example.sh
Last active January 26, 2022 03:53
Using SSH agent forwarding & bastion server / jump box
# Start SSH agent
eval $(ssh-agent)
# Add specific key
ssh-add ~/.ssh/butterfly.pem
# Add default key (if needed for GitHub, etc)
ssh-add
# SSH config entries
@mihow
mihow / pytorch_utils.py
Last active January 23, 2022 03:45
PyTorch model to JIT / TorchScript
import os
import time
import pathlib
import tarfile
import tempfile
import torch
PROJECT_PATH = pathlib.Path(os.environ.get("PROJECT_PATH", "."))
@mihow
mihow / nginx.conf
Created May 3, 2020 01:45
Multi RTMP proxy
worker_processes auto;
rtmp_auto_push on;
events {}
rtmp {
server {
listen 1935;
listen [::]:1935 ipv6only=on;
application live {
live on;