Skip to content

Instantly share code, notes, and snippets.

View beugley's full-sized avatar

Brian Eugley beugley

View GitHub Profile
@beugley
beugley / httplib_proxy.py
Last active February 22, 2023 12:21
Python: use httplib with proxy authentication
import urlparse
import httplib
import base64
proxy_uri = "http://user:password@proxy_host:proxy_port"
host = 'www.google.com'
port = 443
url = urlparse.urlparse(proxy_uri)
conn = httplib.HTTPSConnection(url.hostname, url.port)
@beugley
beugley / sar_to_json.py
Last active March 14, 2022 00:13
Python: convert sar (sadf) to json format
#!/usr/bin/env python
###############################################################################
## sar_to_json.py
##
## Reformats output from sadf to json. sadf must be invoked with the -D
## switch. Timestamps are displayed in ISO6801 format (YYYY-MM-DDThh:mm:ssZ).
## Example: sadf -D -- -A | sar_to_json.py
###############################################################################
import sys
@beugley
beugley / stop_thread.py
Last active March 14, 2022 00:13
Python: implement a stoppable thread
#!/usr/bin/env python
import sys
import logging
import threading
from time import sleep
class StoppableThread(threading.Thread):
"""
Implements a thread that can be stopped.
"""
@beugley
beugley / start_if_not_running.ksh
Created October 1, 2015 13:35
Shell: script that will start a process if it's not already running
#!/bin/ksh
###############################################################################
## Shell script that will start a process if it's not already running.
## Usage: start_if_not_running.ksh -n name
## where "name" is the name of the process. name must exist relative
## to the current directory, or contain a full path.
###############################################################################
function Print
{
@beugley
beugley / start_if_not_running.py
Created October 1, 2015 16:26
Python: start a process if it's not running, or if the process's timestamp is newer than the start time
'''This script will start a process if it's not already running, or if the
timestamp of the process file is newer than the start time.
'''
import sys
import os
import pwd
import subprocess
import re
import time
@beugley
beugley / runsas.ksh
Created October 1, 2015 16:36
Shell: runsas.ksh is a wrapper around sasgsub (grid enabled sas) that acts like command line sas
#!/bin/ksh
##############################################################################
## This script is a wrapper around sasgsub. It will:
## 1) run your SAS script and create a combined log/lst file in the
## directory where your script is located.
## 2) Return an error code if your script has an error.
##
## Usage: runsas.ksh script.sas [-b] [sas command-line options]
## Specify the -b switch when invoking from a batch script. That will
## cause the log/lst to be written to stdout. Multiple invocations of this
@beugley
beugley / ExecAsynch.sas
Last active November 7, 2022 15:23
SAS macro to improve performance/effiiciency via parallel processing
Run time performance of your SAS process can be greatly improved with parallel execution.
This gist describes an approach where your input data set(s) are divided into N equal-sized subsets and your code is
executed in parallel against each subset. For information on other methods of parallel processing, please see this page
from SAS Support: http://support.sas.com/rnd/scalability/tricks/connect.html
Step 1
Divide your input data set(s) into N subsets that are approximately equal in size. The following macro shows one way to
do this.
%macro DIVIDE_INPUT_DATA(N);
@beugley
beugley / json_to_csv.py
Last active March 14, 2022 00:13
Python snippet to convert a json file to csv
import os
import sys
import json
import csv
fin = sys.argv[1]
fout = os.path.splitext(fin)[0]+'.csv'
with open(fin) as f:
j = [json.loads(i) for i in f.readlines()]
@beugley
beugley / human_readable_to_bytes.py
Created August 24, 2016 17:51
Python function to convert a human-readable byte string (e.g. 2G, 10GB, 30MB, 20KB) to a number of bytes
def human_readable_to_bytes(size):
"""Given a human-readable byte string (e.g. 2G, 10GB, 30MB, 20KB),
return the number of bytes. Will return 0 if the argument has
unexpected form.
"""
if (size[-1] == 'B'):
size = size[:-1]
if (size.isdigit()):
bytes = int(size)
else:
@beugley
beugley / add_to_path.sh
Created October 7, 2016 15:18
bash/ksh function to add an entry to the head or tail of a path. Existing duplicate entries are removed.
function add_to_path
{
# Adds an entry to the path (adds to the head by default).
# If the entry already exists, it is removed from its current location
# and added to the tail or head, as specified.
# Duplicate entries are removed.
# Examples: PATH=$(add_to_path $PATH /path/to/add head)
# LD_LIBRARY_PATH=$(add_to_path $LD_LIBRARY_PATH /path/to/add)
NEW=$(echo $1 | /bin/awk -F':' -vADD=$2 -vLOC=$3 'BEGIN {new=""}
{for (i=1;i<=NF;i++) {if ($i!=ADD)