beugley

#!/bin/sh
#
# To prevent multiple instances of a script from executing concurrently,
# include this function, and a statement to invoke it, at the top of any
# ksh/bash script.  This function will check for other instances of the
# same script that are executing by the same user.  If any are found, then
# this instance terminates immediately.
#
# When scripts are invoked in a repetitive manner (cron or other scheduler),
# it's possible that an instance can still be running when the time arrives

beugley

function add_to_path
{
   # Adds an entry to the path (adds to the head by default).
   # If the entry already exists, it is removed from its current location
   # and added to the tail or head, as specified.
   # Duplicate entries are removed.
   # Examples: PATH=$(add_to_path $PATH /path/to/add head)
   #           LD_LIBRARY_PATH=$(add_to_path $LD_LIBRARY_PATH /path/to/add)
   NEW=$(echo $1 | /bin/awk -F':' -vADD=$2 -vLOC=$3 'BEGIN {new=""}
          {for (i=1;i<=NF;i++) {if ($i!=ADD)

beugley

def human_readable_to_bytes(size):
   """Given a human-readable byte string (e.g. 2G, 10GB, 30MB, 20KB),
      return the number of bytes.  Will return 0 if the argument has
      unexpected form.
   """
   if (size[-1] == 'B'):
      size = size[:-1]
   if (size.isdigit()):
      bytes = int(size)
   else:

beugley

import os
import sys
import json
import csv

fin = sys.argv[1]
fout = os.path.splitext(fin)[0]+'.csv'

with open(fin) as f:
   j = [json.loads(i) for i in f.readlines()]

beugley

Run time performance of your SAS process can be greatly improved with parallel execution.
This gist describes an approach where your input data set(s) are divided into N equal-sized subsets and your code is
executed in parallel against each subset.  For information on other methods of parallel processing, please see this page
from SAS Support: http://support.sas.com/rnd/scalability/tricks/connect.html


Step 1
Divide your input data set(s) into N subsets that are approximately equal in size.  The following macro shows one way to
do this.
  %macro DIVIDE_INPUT_DATA(N);

beugley

#!/bin/ksh
##############################################################################
## This script is a wrapper around sasgsub.  It will:
##    1) run your SAS script and create a combined log/lst file in the
##       directory where your script is located.
##    2) Return an error code if your script has an error.
##
## Usage: runsas.ksh script.sas [-b] [sas command-line options]
##    Specify the -b switch when invoking from a batch script.  That will
##    cause the log/lst to be written to stdout.  Multiple invocations of this

beugley

'''This script will start a process if it's not already running, or if the
   timestamp of the process file is newer than the start time.
'''

import sys
import os
import pwd
import subprocess
import re
import time

beugley

#!/bin/ksh
###############################################################################
## Shell script that will start a process if it's not already running.
## Usage: start_if_not_running.ksh -n name
##        where "name" is the name of the process.  name must exist relative
##        to the current directory, or contain a full path.
###############################################################################

function Print
{

beugley

#!/usr/bin/env python
import sys
import logging
import threading
from time import sleep

class StoppableThread(threading.Thread):
   """
   Implements a thread that can be stopped.
   """

beugley

#!/usr/bin/env python
###############################################################################
## sar_to_json.py
##
## Reformats output from sadf to json.  sadf must be invoked with the -D
## switch.  Timestamps are displayed in ISO6801 format (YYYY-MM-DDThh:mm:ssZ).
## Example:    sadf -D -- -A | sar_to_json.py
###############################################################################

import sys