Skip to content

Instantly share code, notes, and snippets.

@nathairtras
nathairtras / fixvirtualenvs
Created November 9, 2019 20:09
Script to fix all virtualenvwrapper virtual environments after Brew python upgrade
#!/bin/bash
# Relies on having fixvirtualenv : https://gist.github.com/nathairtras/7b2df168d9a919b4e644b1d7da0aa05d
# Ensure virtualenv script exists
if [ -z $VIRTUALENVWRAPPER_SCRIPT ]
then
echo "VirtualEnvWrapper Script Environment Variable Not Found. Please run `pip3 install virtualenvwrapper` to continue."
exit 1
fi
# Ensure WORKON_HOME set
@nathairtras
nathairtras / fixvirtualenv
Last active November 9, 2019 19:49
Script for fixing broken Brew VirtualEnvWrapper environments
#!/bin/bash
# @@TODO: Script assumes Python3. If you are wanting to fix a Python2 Virtual Environment,
# consider adding a command line argument to specify which python to use and adjust the `mkvirtualenv` line.
# Reminder: Python2 is EOL Jan1 2020, so script assumes you wouldn't want to do that...
# Ensure virtualenv script exists
if [ -z $VIRTUALENVWRAPPER_SCRIPT ]
then
echo "VirtualEnvWrapper Script Environment Variable Not Found. Please run `pip3 install virtualenvwrapper` to continue."
exit 1
@nathairtras
nathairtras / pytds-tvp-demo.py
Created May 19, 2019 02:24
Example of using PyTDS with a Table Valued Parameter
import os
import pytds
# Grabbing the user and password from environment variables
# Load these however you would like
server = os.environ["MSSQL_SERVER"]
user = os.environ["MSSQL_USER"]
password = os.environ["MSSQL_PASS"]
# Fake some rows, these could be from CSV
@nathairtras
nathairtras / non_ntlk_phrase_counts.py
Created May 18, 2019 03:17
Extracting words and known phrases without NTLK
import re
# List of phrases
phrases = ["computer science","lots of fun"]
# Text to parse
paper_text = """
This is a sentence that includes the phrase computer science. Computer science is fun.
Writing code is lots of fun.
"""
body { background: #222; color: #e6e6e6; }
a { color: #949494; }
a:link, a:visited { color: #949494; }
a:hover, a:active, a:focus { color: #c7c7c7; }
hr { border-bottom: 1px solid #424242; border-top: 1px solid #222; }
@nathairtras
nathairtras / create_connection.py
Last active September 27, 2017 00:44
creating airflow connections via python
from airflow import settings
from airflow.hooks import BaseHook
from airflow.models import Connection
# Start a session
session = settings.Session()
# Create a new connection - these are not unique by name!
cnx_kwargs = {
"conn_id": "mssql_named",
@nathairtras
nathairtras / callback_retry_clear_subdag.py
Last active May 28, 2021 15:47
Callback to clear Airflow SubDag on retry
import logging
from airflow.models import DagBag
def callback_subdag_clear(context):
"""Clears a subdag's tasks on retry."""
dag_id = "{}.{}".format(
context['dag'].dag_id,
context['ti'].task_id,
)
execution_date = context['execution_date']
@nathairtras
nathairtras / tmux_local_install.sh
Created May 21, 2016 02:25 — forked from ryin/tmux_local_install.sh
bash script for installing tmux without root access
#!/bin/bash
# Script for installing tmux on systems where you don't have root access.
# tmux will be installed in $HOME/local/bin.
# It's assumed that wget and a C/C++ compiler are installed.
# exit on error
set -e
TMUX_VERSION=1.8
@nathairtras
nathairtras / aws_policy_read_bucket_write_prefix.json
Created April 15, 2016 13:53
AWS IAM policy template for granting read access to a full bucket and write access to a key prefix in that bucket.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:Get*",
"s3:List*"
],
"Resource": [