Skip to content

Instantly share code, notes, and snippets.

@ronaldtse
Last active December 13, 2018 02:40
Show Gist options
  • Save ronaldtse/63da1c550a5c137512d310ee7940b293 to your computer and use it in GitHub Desktop.
Save ronaldtse/63da1c550a5c137512d310ee7940b293 to your computer and use it in GitHub Desktop.

monitor.sh: reduce command output and protect against inactivity

Purpose

This script is handy for situations that require output but is unable to handle too much output.

A catch-22 situation like on Travis CI:

  • Travis terminates a job on over-logging: when output exceeds 4MB

  • Travis terminates a job on inactivity: when there has been no output for 10 minutes

While this is reasonable, in certain situations, such as when processing a lot of objects, the command used (e.g. aws) do not provide any viable alternative that can satisfy these two conditions at the same time.

So this script is useful in those situations.

Usage

./monitor.sh <my-command> [--with-arguments <values>]

For example, this is how a site with over 40,000 objects gets uploaded to S3 (over 15 minutes) without getting terminated by Travis for inactivity or over-logging.

./monitor.sh aws s3 sync _site s3://$S3_BUCKET_NAME \
  --region=$AWS_REGION --delete --no-progress

Mechanism

This script does the following:

  1. Spawns your specified command in the background.

  2. Stores its PID and checks its existence every 1 second.

  3. On every 10 seconds, it prints out a message indicating how long the command has been run.

  4. When the command exits, it will print out a message indicating so.

  5. Then the script exits.

This code is licensed under 2-clause BSD.

Copyright 2018 Ribose.

#!/bin/bash -e
logfile=monitor.log
echo "[monitor] Executing command '$@' with redirection to $logfile."
end=$((SECONDS))
$@ 2>&1 > $logfile &
pid=$!
while kill -0 $pid 2> /dev/null; do
if ! (( $SECONDS % 10 )) ; then
echo "[monitor] Process $pid running ($SECONDS seconds elapsed)"
fi
sleep 1
done
echo "[monitor] Process $pid complete in $SECONDS seconds."
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment