Skip to content

Instantly share code, notes, and snippets.

@EthanRosenthal
EthanRosenthal / swe4ds.md
Created April 19, 2021 16:43
Notes on Software Engineering skills for Data Scientists

SWE for DS

  • Tests
    • Basic how to with pytest
      • Directory and naming structure
      • Run from CLI
      • How to run specific tests
      • Test coverage
      • Gotchas around package structure and pycache
  • Some intro test principles
@EthanRosenthal
EthanRosenthal / index_slice_ordering.ipynb
Last active May 6, 2019 19:29
Slicing a MultiIndex does not preserve ordering
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@EthanRosenthal
EthanRosenthal / numpy_dot_dot_dot.ipynb
Last active March 27, 2019 00:41
TIL: ellipsis indexing
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@EthanRosenthal
EthanRosenthal / gifify
Created April 9, 2018 19:57
Turn series of images into a gif
# Run like
# gifify image_names_* awesome_movie.gif
# where you glob the list of images as the first argument
# then the last argument is the name of the output gif
alias gifify='python -c "import imageio;import sys;ims=[]; *in_, out=sys.argv[1:];[ims.append(imageio.imread(i)) for i in sorted(in_)];imageio.mimsave(out,ims)"'
@EthanRosenthal
EthanRosenthal / stacking_matrices.py
Last active February 7, 2018 15:11
Lazily stack matrices
import numpy as np
"""Stacking as we loop"""
%%timeit -n 10
res = None
for i in range(100):
batch = np.random.random((100, 100))
if res is None:
res = batch
@EthanRosenthal
EthanRosenthal / explicit_logistic_speedtest.py
Last active October 24, 2017 03:26
Speed test for explicit logistic losses
from time import time
import pandas as pd
import torch
from torch import nn
from torch.nn import functional as F
from torch.autograd import Variable
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.