Skip to content

Instantly share code, notes, and snippets.

@reddragon
Created March 6, 2012 06:41
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save reddragon/1984365 to your computer and use it in GitHub Desktop.
Save reddragon/1984365 to your computer and use it in GitHub Desktop.
Use it for finding the total CPU time taken by tasks on Hadoop, after making the change.
#!/bin/bash
LOGS_PATH="logs/userlogs/"
declare JOB=`ls $LOGS_PATH`;
JOB_PATH="$LOGS_PATH$JOB";
COMMAND="ls -1 --hide=job* $JOB_PATH/";
declare RET=`$COMMAND`;
`rm cputime.out`;
`touch cputime.out`;
for TASK in $RET
do
FILE="$JOB_PATH/$TASK/stderr";
`cat $FILE >> cputime.out`;
done;
declare TIME=`cat cputime.out | grep -o -E "(\+|-)?([0-9]+\.?[0-9]*|\.[0-9]+)([eE](\+|-)?[0-9]+)?user"`;
EXPR="0.0";
for T in $TIME
do
EXPR="$EXPR+${T/user/}";
done;
declare RES=`echo "$EXPR" | bc`;
echo "CPU Time taken by Job $JOB: $RES";
echo "Please do 'rm -rf logs/userlogs' before you run another job";
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment