Last active
May 15, 2018 06:41
-
-
Save nvie/4619344 to your computer and use it in GitHub Desktop.
I'm stuck with this problem for a long time now and could really use the help of a gevent expert. Anybody? Please? Please read the comment on top of the file for a description. This is a minimal example that showcases my problem.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
Theres a gevent pool of 4 greenlets. Jobs are spawned continuously, blocking | |
until a free greenlet is available (default gevent.pool behaviour). Say, | |
a unit of work takes rougly a second. Then, at any moment, there are likely | |
4 jobs mid-execution. | |
The Goal | |
-------- | |
I want to catch both SIGINT/KeyboardInterrupt _and_ SIGTERM in | |
the main loop, and in a nondestructive way. By nondestructive, I mean (1) | |
without aborting any of the running 1-sec jobs, and (2) without spawning any | |
new greenlets after that. | |
In other words: I want to gracefully terminate the loop as soon as the current | |
work has finished. | |
Progress | |
-------- | |
This example has the correct behaviour on Ctrl+C when you run it. I also want | |
to support SIGTERM as a graceful terminator, and that's where the shit | |
happens. | |
Assigning the default int signal handler (i.e. "raise KeyboardInterrupt") to | |
the SIGTERM signal doesn't work. For some reason, normal KeyboardInterrupt | |
due to SIGINT get delivered to the "main greenlet", so we can catch them | |
nicely in the while loop. If we setup the default int handler on SIGTERM, | |
however, when we `kill -SIGTERM` the process, it seems the KeyboardInterrupt | |
gets raised in any of the spawned greenlets. | |
This is a problem, since it (1) aborts that particular job and (2) doesn't | |
stop the spawning loop. | |
Any ideas, anyone? | |
""" | |
from gevent import monkey | |
monkey.patch_all() | |
import time | |
import signal | |
import gevent.pool | |
from gevent import getcurrent | |
def do_something(): | |
print 'ohai from', id(getcurrent()) | |
time.sleep(1.0) | |
print 'kthxbye from', id(getcurrent()) | |
# Have SIGTERM act like SIGINT | |
signal.signal(signal.SIGTERM, signal.default_int_handler) | |
pool = gevent.pool.Pool(4) | |
while True: | |
try: | |
pool.spawn(do_something) | |
except KeyboardInterrupt: | |
break | |
print 'Waiting for jobs to finish...' | |
pool.join() |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment