Skip to content

Instantly share code, notes, and snippets.

🕳
[ o o ]

Brian Abelson abelsonlive

🕳
[ o o ]
Block or report user

Report or block abelsonlive

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@abelsonlive
abelsonlive / wait_browser.py
Last active Jun 18, 2019
Waiting For responses in selenium
View wait_browser.py
from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
class WaitBrowser(object):
def __init__(self, **kw):
self.browser = webdriver.PhantomJS()
def readystate_complete(self):
@abelsonlive
abelsonlive / dbt_backfill.py
Created Apr 11, 2019
Script for backfilling DBT models which use partitions
View dbt_backfill.py
#!/usr/bin/env python
import json
import copy
import sys
import argparse
from datetime import datetime, timedelta
from subprocess import Popen, PIPE
# dictionary of partiton names to variable generating functions
@abelsonlive
abelsonlive / mongo_import.sh
Created Nov 17, 2012
import csv into mongodb from command line
View mongo_import.sh
#!/usr/bin/bash
## include --upsert if adding to a prexistitng collection
mongoimport -d db_name -c coll_name --type csv --file file.csv --headerline
@abelsonlive
abelsonlive / cosine_similarity.R
Last active Aug 20, 2018
Cosine Distance Recommendation / Collaborative Filtering Example
View cosine_similarity.R
# lets make some dummy data
n_rows <- 1000
n_cols <- 100
mat <- matrix(0, nrow=n_rows, ncol=n_cols)
mat <- apply(mat, 2, function(x) { return(rbinom(n_rows, size=1, prob=0.1))})
colnames(mat) <- paste0("event", 1:n_cols)
rownames(mat) <- paste0("pol", 1:n_rows)
# lets take a look at it before we do some math
head(mat)
@abelsonlive
abelsonlive / keybase.md
Created May 28, 2018
keybase verification
View keybase.md

Keybase proof

I hereby claim:

  • I am abelsonlive on github.
  • I am abelsonlive (https://keybase.io/abelsonlive) on keybase.
  • I have a public key ASAQinczA9eMApbqyXq8pZ8Le9PtWFenaQFwkCjHkuIhqAo

To claim this, I am signing this object:

@abelsonlive
abelsonlive / srapeshell.R
Created Sep 23, 2012
# best practices for web scraping in R // ldply
View srapeshell.R
# best practices for web scraping in R #
# function should be used with ldply
# eg:
ldply(urls, scrape)
# add a try to ignore broken links/ unresponsive pages
# eg:
View dot-cat.sh
#!/bin/sh
curl http://cat.www.$1.com.meowbify.com/ > index.html
s3cmd put index.html s3://$1.cat/
View base_models.yml
profile_dir: '~/.dbt' # the location of dbt configurations
target: prod # the dbt profile to use for connection
source_schema: fivetran_app # the tables' source schema (where fivetran imports your data to)
materialized: view # how to materialize these views
target_schema: app # the schema to materialize the base models under (where users will access the data from)
table_prefix: app # a prefix to prepend to each base model/view
directory: models/base/ # the directory under which to save the base models
empty_as_null: true # whether or not to apply EMPTYASNULL logic to all text fields
incl_fivetran_deleted: false # whether or not to include records marked as "deleted" by fivetran
excl_fivetran_synced: true # whether or not to include the time fivetran synced each record
@abelsonlive
abelsonlive / phase-say.py
Created Apr 14, 2018
generative phase music with say
View phase-say.py
import os
import sys
import time
import random
VOICES = [
"Agnes",
"Alex",
"Alice",
"Alva",
@abelsonlive
abelsonlive / config.yml
Last active Feb 28, 2018
Dynamically add partitions to a spectrum table
View config.yml
aws_access_key_id:
aws_secret_access_key:
rs_user:
rs_server:
rs_db:
rs_port:
rs_password:
You can’t perform that action at this time.