Skip to content

Instantly share code, notes, and snippets.

@mihow
mihow / load_dotenv.sh
Last active May 4, 2024 12:32
Load environment variables from dotenv / .env file in Bash
if [ ! -f .env ]
then
export $(cat .env | xargs)
fi
@mihow
mihow / example_matching_rows.sql
Created January 25, 2024 01:24
5 example matching rows for each category in a joined table (PostgreSQL)
SELECT c.category_name, i.image_id FROM categories c
CROSS JOIN LATERAL (
SELECT image_id
FROM image_hashes v
WHERE v.category_id = c.category_id
LIMIT 5
) i;
@mihow
mihow / image_from_url.py
Created September 11, 2023 23:10
Load image into PIL object from URL with streaming request, one line
import requests
from PIL import Image
image = Image.open(requests.get(url, stream=True).raw)
@mihow
mihow / import_key.py
Last active May 31, 2023 00:45
Import existing PEM file as EC2 Key Pair with boto3 & paramiko
import os
import base64
import struct
import boto3
import paramiko
from paramiko.util import deflate_long
PEM_FILEPATH = '~/.ssh/test-key.pem'
@mihow
mihow / send_get_as_post.py
Created March 14, 2023 01:16
Send a GET request as a POST to overcome 414 Request-URI Too Large
# Submit a "GET" request via POST so we can send
# more data than fits in a URL
resp = requests.post(
url,
headers = {'X-HTTP-Method-Override': 'GET'},
data=params)
@mihow
mihow / export_env_file_vars.sh
Created December 19, 2018 00:05
Set environment vars from a file
# Ignores commented out lines and handles spaces
# https://stackoverflow.com/questions/19331497/set-environment-variables-from-file-of-key-pair-values
export $(grep -v '^#' .env | xargs -d '\n')
@mihow
mihow / bash_prompt.sh
Created June 4, 2012 19:14 — forked from insin/bash_prompt.sh
Set color bash prompt according to active virtualenv, git branch and return status of last command.
#!/bin/bash
#
# DESCRIPTION:
#
# Set the bash prompt according to:
# * the active virtualenv
# * the branch/status of the current git repository
# * the return value of the previous command
# * the fact you just came from Windows and are used to having newlines in
# your prompts.
@mihow
mihow / parallel_download.sh
Last active February 4, 2023 00:08
Bash Parallel File Download
#!/bin/bash
# USAGE
# bash ./parallel_download.sh url_list.txt
set -o nounset
set -o errexit
FILELIST=$1
@mihow
mihow / get_table_sizes.sql
Created January 28, 2023 00:11
PostgreSQL - Get size of all tables
SELECT *, pg_size_pretty(total_bytes) AS total
, pg_size_pretty(index_bytes) AS index
, pg_size_pretty(toast_bytes) AS toast
, pg_size_pretty(table_bytes) AS table
FROM (
SELECT *, total_bytes-index_bytes-coalesce(toast_bytes,0) AS table_bytes FROM (
SELECT c.oid,nspname AS table_schema, relname AS table_name
, c.reltuples AS row_estimate
, pg_total_relation_size(c.oid) AS total_bytes
, pg_indexes_size(c.oid) AS index_bytes
@mihow
mihow / python_logging.py
Last active August 23, 2022 00:06
Example python logging configs
"""
This works when running code as script, module and with doctests.
Also sets the log level for imported packages like requests.
python -m mymodule.myfile
python mymodule/myfile.py
python -m unittest mymodule/myfile.py
"""
import logging