Skip to content

Instantly share code, notes, and snippets.

Using virtualenv: /private/tmp/example-deleteme/.venv
Updating dependencies
Resolving dependencies...
1: fact: example-deleteme is 0.1.0
1: derived: example-deleteme
1: fact: example-deleteme depends on dxpy (^0.306.0)
1: fact: example-deleteme depends on Sphinx (^3.5.4)
1: selecting example-deleteme (0.1.0)
1: derived: Sphinx (>=3.5.4,<4.0.0)
1: derived: dxpy (>=0.306.0,<0.307.0)
@jtratner
jtratner / follow_workflow.sh
Created July 11, 2019 16:13
Follow DNAnexus workflow
#!/bin/bash
set -o pipefail
ANALYSIS_ID=$1
dx describe $ANALYSIS_ID
project=$(dx describe $ANALYSIS_ID --json | jq -r .project)
function get_jobs_for_state {
dx find jobs --root $ANALYSIS_ID --state $1 --brief --project ${project} --all-jobs
}
until ! (cat <(get_jobs_for_state runnable) <(get_jobs_for_state running) <(get_jobs_for_state waiting_on_input) <(get_jobs_for_state idle) | grep job | head -n 1); do
FIRST_JOB=$(get_jobs_for_state running | head -n 1 | grep job)
@jtratner
jtratner / compare_pipfiles.py
Created June 24, 2019 00:04
Compare Pipfiles
import json
import sys
import csv
def pipfile_to_pkg_version(path):
with open(path, "r") as fp:
new_pip = json.load(fp)
by_env = {}
@jtratner
jtratner / athena_client.py
Created April 10, 2018 00:35
athena client type stubs
from mypy_extensions import TypedDict
QueryExecutionContext = TypedDict('QueryExecutionContext', {
'Database': str,
})
EncryptionConfiguration = TypedDict('EncryptionConfiguration', {
'EncryptionOption': str,
'KmsKey': str,
}, total=False)
ResultConfiguration = TypedDict('ResultConfiguration', {
@jtratner
jtratner / generate_typed_dicts.py
Created April 10, 2018 00:32
Example of auto generating type stubs
from datetime import datetime
STRING_DEFAULT = 'string'
DATETIME_DEFAULT = datetime(2015, 1, 1)
INT_DEFAULT = 123
def generate_typed_dicts(name, nested_dict):
full_context = []
final_dict = [f"class {name}(TypedDict):"]
@jtratner
jtratner / mode-perf-comparison.md
Last active December 26, 2015 22:49
Series.mode() perf tests.

Results from doing all of value_counts() first

---- Basics - int64 ----
100 length
10000 loops, best of 3: 100 µs per loop
1000 length
10000 loops, best of 3: 166 µs per loop
10000 length
1000 loops, best of 3: 367 µs per loop
100000 length
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jtratner
jtratner / Comparing cost of function calls.ipynb
Last active December 21, 2015 18:29
Comparing cost of function calls [ipython-notebook]
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jtratner
jtratner / testing_out_string_comparisons_for_inf_like_values.ipynb
Last active December 19, 2015 15:59
Perf-tests on Cython string comparisons
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.