Skip to content

Instantly share code, notes, and snippets.

Avatar

Russell Jurney rjurney

View GitHub Profile
View keybase.md

Keybase proof

I hereby claim:

  • I am rjurney on github.
  • I am rjurney (https://keybase.io/rjurney) on keybase.
  • I have a public key ASCimxw2-8kZzIKOFyFXSNVK-d0lnokOpHRQaJLdMciuTgo

To claim this, I am signing this object:

@rjurney
rjurney / pandas_dataframe_parallel.py
Created Apr 13, 2020
How to run a method on a field of a pandas DataFrame and set the result to another field
View pandas_dataframe_parallel.py
def process_split(df: pd.DataFrame, f: types.FunctionType, in_key: str, out_key: str):
"""Process each chunk of a DataFrame, apply a funtion on an in_key and store it in an out_key"""
rows = []
for index, row in df.iterrows():
result = f(row[in_key])
row[out_key] = result
rows.append(row)
df_out = pd.DataFrame(rows)
@rjurney
rjurney / pacoid1.py
Last active Apr 12, 2020
Dear @pacoid, what is the right way to make PyTextRank in spaCy work in parallel?
View pacoid1.py
from multiprocessing import cpu_count
# Use all (96?) cores
n_cores = cpu_count()
print(f'Total cores: {n_cores}')
summaries = []
for doc in nlp.pipe(
df['readme'].astype('unicode').values,
@rjurney
rjurney / bzip_validate.sh
Last active Mar 29, 2020
How to occupy 64 cores with bzip2 validating files if you're a MORON :)
View bzip_validate.sh
#!/bin/bash
for year in 2017 2018 2019 2020
do
for months in 01_02 03_04 05_06 07_08 09_10 11_12
do
month1=$(echo "${months}" | cut -d '_' -f1)
month2=$(echo "${months}" | cut -d '_' -f2)
# 30 days x 2 is around 64 pids
@rjurney
rjurney / support.txt
Created Mar 13, 2020
Template for NextDoor Isolation Support Groups
View support.txt
IF YOU’RE SICK AND NEED SUPPLIES, POST YOUR NEEDS HERE.
I wanted to create a thread where people in quarantine can ask for help for supplies when they’re in need. This supports their immune system with potentially life saving food, beverage and medicine and can be done in a SAFE manner by dropping supplies outside the door.
Hopefully this won’t happen but based on what is going on in Italy (which has a first rate healthcare system) our healthcare system may be overloaded and there may be no room in a hospital, so very sick people will be at home and in need of help.
My neighbor brought me orange juice because I’m sick. My doctor thinks I have the flu and prescribed Tamiflu. I’m glad I know my neighbors.
@rjurney
rjurney / forms.js
Created Feb 27, 2020
How to make a Google Form from a Google Sheet using Google Apps Script that nobody will be able to load to label Amazon open source projects because Forms doesn't scale to 50 questions...
View forms.js
//
// Google Apps Script that reads a Google Sheet based on the Github data and creates a form to classify projects
//
function createForm() {
// Create a new form, then add a checkbox question, a multiple choice question,
// a page break, then a date question and a grid of questions.
var form = FormApp.create('Amazon Open Source Project Classification');
form.setDescription("This form is for labeling open source projects using the project url and description with the following 4 categories:\n\n1) \
API - this is an API library that can only be used to access an Amazon Web Services (AWS) API\n2) EDUCATION - this is an educational project or example \
@rjurney
rjurney / show_error_labels.py
Last active Feb 21, 2020
Code to show the true/predicted labels for gold labeled data in a Snorkel LabelModel
View show_error_labels.py
from snorkel.analysis import get_label_buckets
# Trim the fields for figuring out problems
df_viz = df_test[['full_name', 'description', 'label']]
# Display all errors for debugging purposes
pd.set_option('display.max_rows', len(df_viz.index))
@rjurney
rjurney / snorkel.py
Last active Feb 13, 2020
Utilities for making Snorkel display all LabelFunction errors
View snorkel.py
from snorkel.analysis import get_label_buckets
ABSTAIN = -1
GENERAL = 0
API = 1
EDUCATION = 2
DATASET = 3
names = ['GENERAL', 'API', 'EDUCATION', 'DATASET']
@rjurney
rjurney / README.md
Last active Feb 13, 2020
Pot is physically addictive and kills young adults by causing arrhythmia and strokes
View README.md

Pot is not harmless. It is physically addictive and causes strokes in young people more than any other risk factor including smoking cigarettes.

  • Pot is physically addictive. This isn’t reefer madness and has been known for 20 years. Cannabinoid withdrawal work just like any other drug withdrawal: cannabinoid receptors down regulate when heavily and repeatedly activated and then you feel like shit when they are under stimulated. Every adult reading this has seen someone in withdrawal freaking out back when pot was illegal. This is why.
  • Pot is a depressant until you exceed a certain dose, then it causes severe anxiety, racing heart, etc. Every adult reading this has watched this happen to someone.

For 18-49 year olds, according to the American Heart Association:

  • Smoking cigarettes and pot raises your stroke risk 300%
  • Smoking pot 10 or more times a month without smoking cigarettes raises your stroke risk 250%
View keybase.md

Keybase proof

I hereby claim:

To claim this, I am signing this object:

You can’t perform that action at this time.