Skip to content

Instantly share code, notes, and snippets.

Dan Blanchard dan-blanchard

View GitHub Profile
@dan-blanchard
dan-blanchard / sadface.pyx
Created Dec 29, 2017
parallel-ssh 1.2.1.post1 wheel compilation failures
View sadface.pyx
----------------------------------------
Failed building wheel for parallel-ssh
Running setup.py clean for parallel-ssh
Failed to build parallel-ssh
Installing collected packages: parallel-ssh, setuptools, certifi, urllib3, chardet, ply
Found existing installation: parallel-ssh 1.2.0
Uninstalling parallel-ssh-1.2.0:
Successfully uninstalled parallel-ssh-1.2.0
Running setup.py install for parallel-ssh ... error
Complete output from command /Users/dan/.pyenv/versions/miniconda3-latest/envs/streamparse3/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/h6/389yk8291lj2h9q9020105p00000gn/T/pip-build-eu8bdpha/parallel-ssh/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/h6/389yk8291lj2h9q9020105p00000gn/T/pip-31jq3nnx-record/install-record.txt --single-version-externally-managed --compile:
View keybase.md

Keybase proof

I hereby claim:

  • I am dan-blanchard on github.
  • I am danblanchard (https://keybase.io/danblanchard) on keybase.
  • I have a public key ASBzkQX9EzbCix_coIlz0AeZQGKe7UNiGbSBXW6sMI37Ago

To claim this, I am signing this object:

@dan-blanchard
dan-blanchard / .1.miniconda.md
Last active May 11, 2018
Quicker Travis builds that rely on numpy and scipy using Miniconda
View .1.miniconda.md

For ETS's SKLL project, we found out the hard way that Travis-CI's support for numpy and scipy is pretty abysmal. There are pre-installed versions of numpy for some versions of Python, but those are seriously out of date, and scipy is not there are at all. The two most popular approaches for working around this are to (1) build everything from scratch, or (2) use apt-get to install more recent (but still out of date) versions of numpy and scipy. Both of these approaches lead to longer build times, and with the second approach, you still don't have the most recent versions of anything. To circumvent these issues, we've switched to using Miniconda (Anaconda's lightweight cousin) to install everything.

A template for installing a simple Python package that relies on numpy and scipy using Miniconda is provided below. Since it's a common s

@dan-blanchard
dan-blanchard / check_gridsearch.py
Created Sep 30, 2013
Simple script to check if you run into ValueErrors when you do grid search with more jobs/folds than you have cores.
View check_gridsearch.py
#!/usr/bin/env python
'''
Simple test script to see if we get ValueErrors with joblib and the current
version of Python
'''
import logging
import sys
from multiprocessing import cpu_count
import sklearn.datasets
@dan-blanchard
dan-blanchard / fix_timestamps.py
Created Sep 29, 2013
Script to fix timestamps of files based on filenames. Default filename regular expression looks for filenames named by CameraSync. I developed this to fix an annoying issue where timestamps were wrong for files uploaded from my iPhone, which made Aperture/iPhoto sort them incorrectly (because videos don't have EXIF data).
View fix_timestamps.py
#!/usr/bin/env python
'''
Little script to set timestamps based on filenames for files uploaded to
Dropbox by CameraSync.
:author: Dan Blanchard
:date: September, 2013
'''
from __future__ import print_function, unicode_literals
@dan-blanchard
dan-blanchard / create_single_machine_sge.md
Last active Jun 3, 2018
How to setup a single-machine (Sun) Grid Engine installation for unit tests on Travis-CI
View create_single_machine_sge.md

I recently needed a way to run unit tests on Travis for a project that uses Sun Grid Engine, Grid Map. Unfortunately, it seemed like no one had figured out how to set that up on Travis before (or simply create a single-machine installation without any user interaction). After hours of trial-and-error, I now know the secrets to making a single-machine installation of SGE that runs on Travis, and I'm sharing my script to prevent other people from going through the same frustrating experience.

To use the install_sge.sh script below, you just need to copy all of the files in this gist to a travis sub-directory directly under the root of your GitHub project, and add the following lines to your .travis.yml

before_install:
  - travis/install_sge.sh
  - export SGE_ROOT=/var/lib/gridengine
  - export SGE_CELL=default
  - export DRMAA_LIBRARY_PATH=/usr/lib/libdrmaa.so.1.0
You can’t perform that action at this time.