Monit example for restarting a service on excessive memory consumption

monitex

# Relevant example from our monitrc file.
check process couchdb with pidfile /usr/local/var/run/couchdb/couchdb.pid
    start program = "/etc/init.d/couchdb start"
    stop program  = "/etc/init.d/couchdb stop"
    if cpu > 60% for 2 cycles then alert
    if cpu > 80% for 5 cycles then restart
    if memory usage > 70% MB for 5 cycles then restart

Nice, but what if I wanted to check the system memory usage of the machine itself?

Something like

check system
If memory usage > 70% then exec killall node


You have to specify the system you're checking.

check system localhost
if memory usage > 70% then exec killall node

"check system localhost
if memory usage > 70% then alert"
works =)

But when i was using monit to monitor memory and set to restart tomcat, it is showing this error

Apr 12 18:33:41 la1inv-incentivessvc05.int.dealer.com systemd[1]: Started Pro-active monitoring utility for unix systems.
Apr 12 18:33:41 la1inv-incentivessvc05.int.dealer.com systemd[1]: Starting Pro-active monitoring utility for unix systems...
Apr 12 18:33:41 la1inv-incentivessvc05.int.dealer.com monit[27177]: /etc/monit.d/tomcat:6: syntax error 'memory'
Apr 12 18:33:41 la1inv-incentivessvc05.int.dealer.com systemd[1]: monit.service: main process exited, code=exited, status=1/FAILURE
Apr 12 18:33:41 la1inv-incentivessvc05.int.dealer.com monit[27179]: /etc/monit.d/tomcat:6: syntax error 'memory'
Apr 12 18:33:41 la1inv-incentivessvc05.int.dealer.com systemd[1]: monit.service: control process exited, code=exited status=1
Apr 12 18:33:41 la1inv-incentivessvc05.int.dealer.com systemd[1]: Unit monit.service entered failed state.
Apr 12 18:33:41 la1inv-incentivessvc05.int.dealer.com systemd[1]: monit.service failed.

monit cannot able to read word 'memory', Don't know why.

class platforms::monit::incentives_svc {
include ::monit
monit::check { 'tomcat':
content => 'check host tomcat with address localhost
stop program = "/etc/init.d/tomcat stop"
start program = "/etc/init.d/tomcat restart"
if failed port 9620 then restart
if failed port 37984 then restart
if memory usage > 70% then restart
',
}
}

Try totalmem. Example httpd rule (Amazon Linux/CentOS/RHEL):

check process httpd with pidfile /var/run/httpd/httpd.pid
    start program = "/usr/sbin/httpd -k start"
    stop program = "/usr/sbin/httpd -k stop"
    if totalmem > 100 MB for 2 cycles then alert
    if totalmem > 1000 MB for 2 cycles then restart

In the original example, if memory usage > 70% MB for 5 cycles then restart seems ambiguous. It looks like it should either be % or MB in the documentation, not both. I would guess that in this case monit would use the % and ignore the MB (or maybe throw an error?) but I'm not sure and it doesn't seem like a good idea to have the ambiguity!