hartleybrody

# copy/import data from heroku postgres to localhost pg database

# useful for copying admin's work on live site into local database to reproduce errors
# https://devcenter.heroku.com/articles/heroku-postgres-import-export

# take heroku pg snapshot and download
heroku pg:backups:capture
heroku pg:backups:download

# load the dump into local postgres database, assuming $DATABASE_URL set locally

hartleybrody

I do this infrequently enought that I always forget how to do it. Note that I've finally decided to move away from virtualenvwrapper (workon, mkvirtualenv, etc) since it didn't play nicely with the python versions set by pyenv.

Install the tools (pyenv & pyenv-virtualenv)

Only necessary if this is a new laptop

brew install pyenv
brew install pyenv-virtualenv

hartleybrody

Pagination Details

Everyone loves to say "you're probably doing pagination wrong" -- in terms of writing efficient SQL queries.

Here is the tl;dr:

• make sure you are using ORDER BY in your query, otherwise subsequent queries with a LIMIT and OFFSET could either skip rows or duplicate rows across pages
• when going to next page, don't use LIMIT .. OFFSET since the database still has to fetch all rows, then order them, then move to the correct page
• instead using something like WHERE id > {max_id_from_prev_page} and (assuming there's an index on that column) the database can jump directly to the correct spot to read just the number of rows you're displaying on the next page
• this helps to keep the query constant time as the size of the data grows and/or as you page deeper into the data

hartleybrody

Intro

For when you need to insert or update millions of rows and don't want to mess around with scripting to batch individual SQL commands, it's worth using postgres's built in, super-fast support for importing and exporting data from the CSV file format.

Note: that we're using the \COPY command and not the COPY command, since it allows us to work with files locally available on our machine and not worry about transferring files to the database server's file system.

Via the psql manual on \COPY:

Performs a frontend (client) copy. This is an operation that runs an SQL COPY command, but instead of the server reading or writing the specified file, psql reads or writes the file and routes the data between the server and the local file system. This means that file accessibility and privileges are those of the local user, not the server, and no SQL superuser privileges are required.

Export data from database t

hartleybrody

kotlin cheatsheet

string interpolation

val foo = 1
var bar = "one is $foo"
var baz = "two is ${foo + 1}"

Unit is a placeholder return value for functions that don't actually return a meaninfgul value. It is the default return type for any function if one is not specified explicitly. Nothing is a placeholder return value for functions that should never actually return (ie they throw an exception or have infinite loop). Good SO discussion on differences.

hartleybrody

"""
    Example of setting up multiple click commands within one file.
    Run these commands with:
    
        python /path/to/file.py do-foo --foo 'hello world'
        
    Note that click converts '_' to '-' in function name

"""

hartleybrody

# 1. download a copy of prod db to localhost

# connect to remote redis database and
# download a snapshot of the entire database
# to a local file on your filesystem

# via https://stackoverflow.com/a/61365048/625840
# docs https://redis.io/docs/manual/cli/#remote-backups-of-rdb-files

redis-cli -u $PROD_REDIS_URL --rdb dump.rdb

hartleybrody

Postgres is usually run automatically on my laptop. I also have a weekly cronjob that does brew update and brew upgrade which sometimes bumps the version of portgres that I'm running. Newer versions of the postgres server software aren't necessarily compatible with the older version's data directory structure, and so you need to "upgrade" your database to see all of your old data in the upgraded postgres server.

Trying to start the database manually with

pg_ctl -D /usr/local/var/postgres -l /usr/local/var/postgres/server.log start

Which gives an error message and tells me where to look (note that I passed in the log directory when I ran the command)

waiting for server to start.... stopped waiting

pg_ctl: could not start server

hartleybrody

Javascript has two kinds of scope:

1. Local (inside a function)
2. Global (outside a function)

The var Keyword

• Using the var keywords creates the variable in the current scope
  • If the current scope is global, then var is unnecessary (see below)
  • If the current scope is local, then you’re creating a local variable in the current scope
• If you don’t use var, then Javascript goes up the “scope chain” to see if it’s already been declared

hartleybrody

import json
import simplejson               # pip install simplejson
import pickle
import cPickle
import marshal
import yaml                     # pip install pyyaml
import ujson                    # pip install ujson
from cStringIO import StringIO
import bson                     # pip install bson