Created
June 11, 2019 20:00
-
-
Save zzzeek/91fd34c4a9245cfbd703b685c1f1d5c0 to your computer and use it in GitHub Desktop.
PyMySQL + eventlet authenticaiton timeout
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import random | |
import time as blocking_time | |
import eventlet | |
import pymysql | |
# monkeypatch Python network libraries so that they are non blocking. all | |
# Python code that uses the pure Python network IO functions will be non | |
# blocking, where all IO waiting will defer back to eventlet where it will | |
# pass off work to another greenlet. | |
eventlet.monkey_patch(socket=True) | |
def connect(): | |
"""Connect to a local MySQL database. | |
No hostname is given, so this uses a UNIX socket. Any kind of socket | |
will do. | |
""" | |
return pymysql.connect() | |
def do_work(greenlet, cpu_blocking): | |
while True: | |
print( | |
"greenlet %s CPU blocking for %.4f sec" % (greenlet, cpu_blocking) | |
) | |
blocking_time.sleep(cpu_blocking) | |
conn = connect() | |
print("greenlet %s connected to the database" % greenlet) | |
def retreive_connection_timeout(): | |
"""Get the configured connect_timeout from the server. | |
This is usually ten seconds. If it's much higher, then this script | |
will take a lot longer to illustrate the effect. | |
""" | |
conn = connect() | |
cursor = conn.cursor() | |
cursor.execute("show variables like 'connect_timeout'") | |
connect_timeout = int(cursor.fetchone()[1]) | |
cursor.close() | |
conn.close() | |
return connect_timeout | |
connect_timeout = retreive_connection_timeout() | |
pool = eventlet.GreenPool() | |
print( | |
"Creating ten continuous greenlets with a tiny amount of CPU work in " | |
"between database connections" | |
) | |
for i in range(10): | |
pool.spawn_n(do_work, i, random.choice([0.001, 0.002, 0.005])) | |
eventlet.sleep(1) | |
print( | |
"Now adding a CPU expensive worker that will wait " | |
"longer than %d seconds to finish its CPU work" % connect_timeout | |
) | |
pool.spawn_n(do_work, i, connect_timeout + 2) | |
pool.waitall() |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The script creates ten greenlets, then an 11th one that is going to hog the CPU longer than MySQL's connect_timeout. The connection attempts begin to fail as MySQL is rejecting the connections that have waited longer than ten seconds to respond to a security challenge. The stack trace generated by PyMySQL is very distinctive and is a 100% match for this exact situation. You can see the "2006 MySQL server has gone away", the fact that it occurred right in pymysql _process_auth(), and the greenio monkeypatching of PyMySQL which prevents it from working correctly.