For several weeks our production deployment (Express, Bookshelf/Knex -> MySQL) was randomly having queries fail resulting in 500 errors and server crashes.
The first issue is that the Knex library was using Pool2 slightly incorrectly, and when a ETIMEDOUT error occurred, it tried to release the pooled connection twice. This would create an exception which would crash express (unless you have a top level error handler defined in your express setup)
In this issue (myndzi/pool2#12 (comment)) filed on pool2 by the author of knex (tgriesser) the author of pool2 (myndzi) points out the error handling issue and created a fork of knex with the fix git+https://github.com/myndzi/knex#double-release
After installing the fix on my server, whenever ETIMEDOUT occured an error would be generated in the expected part of the code so that I could decide to retry the query or fail gracefully. Removing the ETIMEDOUT entirely would be