Skip to content

Instantly share code, notes, and snippets.

View nehalecky's full-sized avatar

Nicholaus (Nico) Halecky nehalecky

View GitHub Profile
#!/usr/bin/env bash
###
# NB: You probably don't want this gist any more.
# Instead, use this version from `fastsetup`:
# https://github.com/fastai/fastsetup/blob/master/setup-conda.sh
###
set -e
cd
@pshapiro
pshapiro / Auto301Redirects.ipynb
Created June 19, 2019 16:50
Automatic 301 Redirects with SpaCy
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jbielick
jbielick / git_sync.py
Last active October 11, 2023 09:02
an Apache Airflow DAG to sync a git repository to the google cloud storage bucket for your Composer environment
from datetime import datetime, timedelta
from airflow.models import Variable
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
private_key = Variable.get("git_deploy_private_key_secret")
repo_url = Variable.get("git_remote_url")
default_args = {
@webframp
webframp / keybase.md
Created July 25, 2017 18:14
Signing git commits on github using keybase.io gpg key

Probably one of the easiest things you'll ever do with gpg

Install Keybase: https://keybase.io/download and Ensure the keybase cli is in your PATH

First get the public key

keybase pgp export | gpg --import

Next get the private key

@audreyfeldroy
audreyfeldroy / pypi-release-checklist2.md
Last active March 9, 2020 19:26
My PyPI Release Checklist 2 (now with bumpversion)
  • Update HISTORY.rst
  • Commit the changes:
git add HISTORY.rst
git commit -m "Changelog for upcoming release 0.1.1."
  • Update version number (can also be patch or major)
bumpversion minor
@pbugnion
pbugnion / ipython_notebook_in_git.md
Last active October 22, 2023 12:25
Keeping IPython notebooks under Git version control

This gist lets you keep IPython notebooks in git repositories. It tells git to ignore prompt numbers and program outputs when checking that a file has changed.

To use the script, follow the instructions given in the script's docstring.

For further details, read this blogpost.

The procedure outlined here is inspired by this answer on Stack Overflow.

@eteq
eteq / local_build.py
Last active July 3, 2018 14:49
Authorea script to build locally with LaTeX NOTE: updates at https://github.com/eteq/authorea-scripts
#!/usr/bin/env python
from __future__ import division, print_function
"""
This script generates a file to use for building authorea papers, and then runs
latex on them.
Requires python >= 2.6 (3.x should work, too)
The key assumptions are:
@dan-blanchard
dan-blanchard / .1.miniconda.md
Last active December 11, 2019 22:38
Quicker Travis builds that rely on numpy and scipy using Miniconda

For ETS's SKLL project, we found out the hard way that Travis-CI's support for numpy and scipy is pretty abysmal. There are pre-installed versions of numpy for some versions of Python, but those are seriously out of date, and scipy is not there are at all. The two most popular approaches for working around this are to (1) build everything from scratch, or (2) use apt-get to install more recent (but still out of date) versions of numpy and scipy. Both of these approaches lead to longer build times, and with the second approach, you still don't have the most recent versions of anything. To circumvent these issues, we've switched to using Miniconda (Anaconda's lightweight cousin) to install everything.

A template for installing a simple Python package that relies on numpy and scipy using Miniconda is provided below. Since it's a common s

@wpm
wpm / spark_parallel_boost.py
Last active December 3, 2018 02:56
A simple example of how to integrate the Spark parallel computing framework and the scikit-learn machine learning toolkit. This script randomly generates test and train data sets, trains an ensemble of decision trees using boosting, and applies the ensemble to the test set. The ensemble training is done in parallel.
from pyspark import SparkContext
import numpy as np
from sklearn.cross_validation import train_test_split, Bootstrap
from sklearn.datasets import make_classification
from sklearn.metrics import accuracy_score
from sklearn.tree import DecisionTreeClassifier
def run(sc):
@dln
dln / Vagrantfile
Created August 26, 2013 11:38
Vagrant setup for a Mesos cluster
# -*- mode: ruby -*-
Vagrant.configure("2") do |config|
config.vm.box = "precise64"
config.vm.box_url = "http://files.vagrantup.com/precise64.box"
config.vm.network :public_network, :bridge => "eth0"
config.vm.synced_folder "data", "/data"
config.vm.synced_folder "mesos", "/mesos"
config.vm.synced_folder "/home/dln/src/mesos-docker/target/scala-2.10", "/mesos/mesos-docker"
config.vm.synced_folder "salt", "/srv/salt"