public
Last active

memusg -- Measure memory usage of processes

  • Download Gist
memusg
Shell
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46
#!/usr/bin/env bash
# memusg -- Measure memory usage of processes
# Usage: memusg COMMAND [ARGS]...
#
# Author: Jaeho Shin <netj@sparcs.org>
# Created: 2010-08-16
set -um
 
# check input
[ $# -gt 0 ] || { sed -n '2,/^#$/ s/^# //p' <"$0"; exit 1; }
 
# TODO support more options: peak, footprint, sampling rate, etc.
 
pgid=`ps -o pgid= $$`
# make sure we're in a separate process group
if [ $pgid = $(ps -o pgid= $(ps -o ppid= $$)) ]; then
cmd=
set -- "$0" "$@"
for a; do cmd+="'${a//"'"/"'\\''"}' "; done
exec bash -i -c "$cmd"
fi
 
# detect operating system and prepare measurement
case `uname` in
Darwin|*BSD) sizes() { /bin/ps -o rss= -g $1; } ;;
Linux) sizes() { /bin/ps -o rss= -$1; } ;;
*) echo "`uname`: unsupported operating system" >&2; exit 2 ;;
esac
 
# monitor the memory usage in the background.
(
peak=0
while sizes=`sizes $pgid`
do
set -- $sizes
sample=$((${@/#/+}))
let peak="sample > peak ? sample : peak"
sleep 0.1
done
echo "memusg: peak=$peak" >&2
) &
monpid=$!
 
 
# run the given command
exec "$@"

Hi Jaeho Shin,

Thanks for your easy to use tool!
I would like to use your script for a series of benchmarks and I would like to get the output for each benchmark separately. But somehow all peaks get printed after the whole suite is finished. Can I do something about this?

Cheers
Bernd

Hi brassel,
I'm so glad to meet a fan of my tool :)

I think you just pointed out a caveat of my script.
This script monitors the rss peak of all processes in the same process group which shares a PGID.
It seems you're running all the benchmarks with this tool within a single shell script, right?
Since processes launched from a non-interactive shell usually uses the same PGID, this tool won't work correctly.
I only considered using this from an interactive shell, but I should fix this limitation as I'll encounter a similar case soon.

For the moment, you can launch each benchmark prefixing it with setsid:
setsid memusg ./benchmark-XYZ ...
Or, if setsid isn't available on your system, then using bash -i (makes a little clutter):
bash -i -c 'memusg ./benchmark-XYZ ...'

Thanks for your feedback!
~Jaeho

Thank you very much for the fast answer!
Using "setsid" worked like a charm.

Thankful Greetings,
Bernd

Bernd, I actually updated the script an hour ago so we can now use it without the workaround. :)

As I have read your comments in the script

# TODO support more options: peak, footprint, sampling rate, etc.

I thought you might be interested that for my purposes I have changed by hand

  • the sampling rate
  • the redirection of the resulting output to &2

Everything else was just as needed.
Bernd

One thing I notice is that I do not seem to be able to stop processes with ^C while using memusg.
What might be the reason for this?

Hello!
I have a problem using your tool with a multi-threaded program...

This last program when started launches itself a couple of times.
(Different processes with different PIDs but with the same name appear on "top".)

I am using a shell script to launch "memusg". Something like:
[script.sh] memusg.sh command

And I run it through "nohup":
nohup script.sh &

But by watching "top", the command itself isn't executed!
(Or it shows up and quickly disappears constantly!)

And the output file shows multiple times:
[nohup.out] bash: no job control in this shell

The only processes that I can see running are:
/bin/bash script.sh
bash memusg.sh command

But if I kill the process (the one initiated by "nohup"):
/bin/bash script.sh
the command gets executed!

Do you know what is the problem and how can I fix it?
Thank you very much!

Sorry for being dumb... what is the output? kb? Cheers,
Dan.

@brassel and @XICO2KX I know it's very late, but I'll look into this problem. It seems there're quite a few folks who want to use this in various ways.

+1 Just used this for some quick profiling. Thanks.

Nice work! I used this script to do some quick profiling of some PHP scripts.

Just a heads up...
User Jonathan Clark has a reviewed and improved version of this memusg script here:
https://github.com/jhclark/memusg
It seems to have fixed all the problems with child processes, threads and nohup.
The only disadvantage is that it measures the Virtual Size instead.
But you only need to change the expression 'vsize=' to 'rssize=' to get the Peak (High Water Mark) RSS Memory Usage.

Just FYI, the version linked by @XICO2KX does not work on OSX/BSD (due to differences in the ps binary).

Please sign in to comment on this gist.

Something went wrong with that request. Please try again.