Skip to content

Instantly share code, notes, and snippets.

Avatar

Luke Rodgers lukeasrodgers

View GitHub Profile
@lukeasrodgers
lukeasrodgers / pg_tuning_notes.md
Last active Jul 29, 2020
postgres conf - tuning notes
View pg_tuning_notes.md

269 tunable settings, these 20 are most helpful

max_connections

  • set to useful peak
  • above 200, look at conn pooler
  • consider superuser_reserved_connections

shared_buffers - shared RAM

  • amt of memory pg takes for itself
  • generally, avail ram ÷ 4
@lukeasrodgers
lukeasrodgers / json_post.lua
Created Oct 26, 2018
sample wrk lua script for json post
View json_post.lua
wrk.method = "POST"
wrk.body = "{\"email\":{\"value\":\"admin@example.com\"}}"
wrk.headers["Content-Type"] = "application/json"
wrk.headers["Authorization"] = "Basic eHh4Onl5eQ=="
View space_key.applescript
activate application "iTerm"
repeat 5009 times
tell application "System Events" to key code 36
delay (random number from 0.5 to 5)
end repeat
@lukeasrodgers
lukeasrodgers / timings.md
Last active Jun 11, 2020
time do do things
View timings.md

databases

pg

  • dropping not null constraint on table with 800k rows, approx 35 columns: 32ms (8 core / 16 G RAM mbp)
  • counting table with 5.1M records: 42s (on heroku)
  • adding index on column, all values NULL: 42s (on heroku)
  • adding index concurrently on updated_at column, about 17M rows, aptible postgres ~800 IOPS: 455.8086s
  • adding index concurrently on 3 cols, item_type, event_type, updated_at, ~70M rows: 1.5hours
  • removing index concurrently, one index, table about 20M rows: 80s
  • adding index concurrently on 2 cols, banking_system_id (string, non-fk), platform (string), ~30M rows, about 3k IOPS: ~35 mins
View dtruss.txt
# TYPE /a in vim
select(0x1, 0x7FFF5DD1C850, 0x7FFF5DD1C7D0, 0x7FFF5DD1C750, 0x7FFF5DD1C8D8) = 0 0
lseek(0x3, 0x2000, 0x0) = 8192 0
dtrace: error on enabled probe ID 2132 (ID 320: syscall::write:return): invalid kernel access in action #12 at DIF offset 92
select(0x1, 0x7FFF5DD1C720, 0x7FFF5DD1C6A0, 0x7FFF5DD1C620, 0x7FFF5DD1C7A8) = 0 0
fsync(0x3, 0x7FFF5DD1C720, 0x7FFF5DD1C6A0) = 0 0
select(0x1, 0x7FFF5DD1C770, 0x7FFF5DD1C6F0, 0x7FFF5DD1C670, 0x7FFF5DD1C7F8) = 0 0
select(0x1, 0x7FFF5DD1C850, 0x7FFF5DD1C7D0, 0x7FFF5DD1C750, 0x0) = 1 0
@lukeasrodgers
lukeasrodgers / junk.r
Last active Feb 13, 2017
messing with R
View junk.r
# install CRAN package from R repl
install.packages("mypkg", lib="/Users/luke/.r/pkg/\/R-packages/")
#random numbers from normal distribution
set.seed(42) #for reproducible numbers
x = rnorm(5000) #generate random numbers from normal dist
hist(x,breaks=50, main="Normal distribution, N=5000") #plot
shapiro.test(x) #SW test
>W = 0.9997, p-value = 0.744
@lukeasrodgers
lukeasrodgers / mysql_dummydata.sql
Created Sep 12, 2016
some misc procedures for creating test/dummy data in mysql
View mysql_dummydata.sql
/* quickly/hackily create randomish string data with md5 */
DELIMITER $$
DROP PROCEDURE IF EXISTS user_promos;
CREATE PROCEDURE user_promos()
BEGIN
DECLARE i INT DEFAULT 1;
WHILE i < 5000 DO
View keybase.md

Keybase proof

I hereby claim:

  • I am lukeasrodgers on github.
  • I am lukeasrodgers (https://keybase.io/lukeasrodgers) on keybase.
  • I have a public key whose fingerprint is 9E9C 2A25 69EA CF84 DF8D 6B8A 0CA0 92B0 E2F9 8F01

To claim this, I am signing this object:

View generate postgres data
http://bonesmoses.org/2016/07/15/pg-phriday-a-postgres-persepctive-on-mongodb/
CREATE TABLE sensor_log (
id SERIAL NOT NULL PRIMARY KEY,
location VARCHAR NOT NULL,
reading BIGINT NOT NULL,
reading_date TIMESTAMP NOT NULL
);
\timing
You can’t perform that action at this time.