Skip to content

Instantly share code, notes, and snippets.

@hartleybrody
hartleybrody / python-version-setup.md
Last active February 13, 2024 15:30
modern python versions + virtualenvs

I do this infrequently enought that I always forget how to do it. Note that I've finally decided to move away from virtualenvwrapper (workon, mkvirtualenv, etc) since it didn't play nicely with the python versions set by pyenv.

Install the tools (pyenv & pyenv-virtualenv)

Only necessary if this is a new laptop

brew install pyenv
brew install pyenv-virtualenv

kotlin cheatsheet

string interpolation

val foo = 1
var bar = "one is $foo"
var baz = "two is ${foo + 1}"

Unit is a placeholder return value for functions that don't actually return a meaninfgul value. It is the default return type for any function if one is not specified explicitly. Nothing is a placeholder return value for functions that should never actually return (ie they throw an exception or have infinite loop). Good SO discussion on differences.

@hartleybrody
hartleybrody / click.py
Last active February 8, 2023 14:38
setting up multiple click commands within a single file
"""
Example of setting up multiple click commands within one file.
Run these commands with:
python /path/to/file.py do-foo --foo 'hello world'
Note that click converts '_' to '-' in function name
"""
@hartleybrody
hartleybrody / notes.md
Last active December 5, 2023 17:36
pagination

Pagination Details

Everyone loves to say "you're probably doing pagination wrong" -- in terms of writing efficient SQL queries.

Here is the tl;dr:

  • make sure you are using ORDER BY in your query, otherwise subsequent queries with a LIMIT and OFFSET could either skip rows or duplicate rows across pages
  • when going to next page, don't use LIMIT .. OFFSET since the database still has to fetch all rows, then order them, then move to the correct page
  • instead using something like WHERE id > {max_id_from_prev_page} and (assuming there's an index on that column) the database can jump directly to the correct spot to read just the number of rows you're displaying on the next page
  • this helps to keep the query constant time as the size of the data grows and/or as you page deeper into the data
@hartleybrody
hartleybrody / psql-csv.md
Last active October 4, 2023 01:11
use CSV to bulk export/import data to postgres tables

Intro

For when you need to insert or update millions of rows and don't want to mess around with scripting to batch individual SQL commands, it's worth using postgres's built in, super-fast support for importing and exporting data from the CSV file format.

Note: that we're using the \COPY command and not the COPY command, since it allows us to work with files locally available on our machine and not worry about transferring files to the database server's file system.

Via the psql manual on \COPY:

Performs a frontend (client) copy. This is an operation that runs an SQL COPY command, but instead of the server reading or writing the specified file, psql reads or writes the file and routes the data between the server and the local file system. This means that file accessibility and privileges are those of the local user, not the server, and no SQL superuser privileges are required.

Export data from database t

@hartleybrody
hartleybrody / backup-analyze-redis.sh
Last active August 23, 2022 16:31
backup a production redis URL to your local redis server (and analyze key usage)
# 1. download a copy of prod db to localhost
# connect to remote redis database and
# download a snapshot of the entire database
# to a local file on your filesystem
# via https://stackoverflow.com/a/61365048/625840
# docs https://redis.io/docs/manual/cli/#remote-backups-of-rdb-files
redis-cli -u $PROD_REDIS_URL --rdb dump.rdb
@hartleybrody
hartleybrody / upgrading-postgresql-database.md
Last active December 1, 2021 13:30
upgrading postgres

Postgres is usually run automatically on my laptop. I also have a weekly cronjob that does brew update and brew upgrade which sometimes bumps the version of portgres that I'm running. Newer versions of the postgres server software aren't necessarily compatible with the older version's data directory structure, and so you need to "upgrade" your database to see all of your old data in the upgraded postgres server.

Trying to start the database manually with

pg_ctl -D /usr/local/var/postgres -l /usr/local/var/postgres/server.log start

Which gives an error message and tells me where to look (note that I passed in the log directory when I ran the command)

waiting for server to start.... stopped waiting

pg_ctl: could not start server

@hartleybrody
hartleybrody / import-db.sh
Last active April 10, 2024 16:09
Copy data from Heroku Postgres into local database
# copy/import data from heroku postgres to localhost pg database
# useful for copying admin's work on live site into local database to reproduce errors
# https://devcenter.heroku.com/articles/heroku-postgres-import-export
# take heroku pg snapshot and download
heroku pg:backups:capture
heroku pg:backups:download
# load the dump into local postgres database, assuming $DATABASE_URL set locally
import json
import simplejson # pip install simplejson
import pickle
import cPickle
import marshal
import yaml # pip install pyyaml
import ujson # pip install ujson
from cStringIO import StringIO
import bson # pip install bson
@hartleybrody
hartleybrody / 0_reuse_code.js
Created April 2, 2014 15:50
Here are some things you can do with Gists in GistBox.
// Use Gists to store code you would like to remember later on
console.log(window); // log the "window" object to the console