Skip to content

Instantly share code, notes, and snippets.

@nvie
Last active May 15, 2018 06:41
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save nvie/4619344 to your computer and use it in GitHub Desktop.
Save nvie/4619344 to your computer and use it in GitHub Desktop.
I'm stuck with this problem for a long time now and could really use the help of a gevent expert. Anybody? Please? Please read the comment on top of the file for a description. This is a minimal example that showcases my problem.
"""
Theres a gevent pool of 4 greenlets. Jobs are spawned continuously, blocking
until a free greenlet is available (default gevent.pool behaviour). Say,
a unit of work takes rougly a second. Then, at any moment, there are likely
4 jobs mid-execution.
The Goal
--------
I want to catch both SIGINT/KeyboardInterrupt _and_ SIGTERM in
the main loop, and in a nondestructive way. By nondestructive, I mean (1)
without aborting any of the running 1-sec jobs, and (2) without spawning any
new greenlets after that.
In other words: I want to gracefully terminate the loop as soon as the current
work has finished.
Progress
--------
This example has the correct behaviour on Ctrl+C when you run it. I also want
to support SIGTERM as a graceful terminator, and that's where the shit
happens.
Assigning the default int signal handler (i.e. "raise KeyboardInterrupt") to
the SIGTERM signal doesn't work. For some reason, normal KeyboardInterrupt
due to SIGINT get delivered to the "main greenlet", so we can catch them
nicely in the while loop. If we setup the default int handler on SIGTERM,
however, when we `kill -SIGTERM` the process, it seems the KeyboardInterrupt
gets raised in any of the spawned greenlets.
This is a problem, since it (1) aborts that particular job and (2) doesn't
stop the spawning loop.
Any ideas, anyone?
"""
from gevent import monkey
monkey.patch_all()
import time
import signal
import gevent.pool
from gevent import getcurrent
def do_something():
print 'ohai from', id(getcurrent())
time.sleep(1.0)
print 'kthxbye from', id(getcurrent())
# Have SIGTERM act like SIGINT
signal.signal(signal.SIGTERM, signal.default_int_handler)
pool = gevent.pool.Pool(4)
while True:
try:
pool.spawn(do_something)
except KeyboardInterrupt:
break
print 'Waiting for jobs to finish...'
pool.join()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment