Skip to content

Instantly share code, notes, and snippets.

@jczaplew
Last active July 19, 2021 09:34
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jczaplew/5799404 to your computer and use it in GitHub Desktop.
Save jczaplew/5799404 to your computer and use it in GitHub Desktop.
502 Proxy Error with Apache, Node, and MySQL
Setup:
Node running on :8080
Apache running on :80
Using Apache mod_proxy to forward application requests to :8080
Symptoms
On client:
1. Proxy Error
The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request GET /route.
Reason: Error reading from remote server
(with 502 header)
2. Hanging page load (request sends, no response received)
3. Routes without complex/any queries work fine after restarting Node
In Apache error_log:
1. [error] [client xx.xx.xxx.10] proxy: Error reading from remote server returned by /route
2. [error] [client xx.xx.xxx.10] (70014)End of file found: proxy: error reading status line from remote server localhost:8080
In Node error log:
1. Nothing
On server:
1. Simple SELECT queries on database work fine
2. Complex queries hang indefinitely
3. Running "SHOW PROCESSLIST" produces a list of queries with the status "Waiting for table-level lock"
Temporary fix
1. In MySQL, run "SHOW PROCESSLIST"
2. For each process, run "KILL [unique id]"
3. Restart Apache
4. Restart Node
After doing this, CPU usage dropped from ~12% to ~0.5% and performance went back to normal
A slightly better fix
1. Bash script to kill all at once per this answer: http://serverfault.com/questions/11357/bulk-or-mass-killing-misbehaving-mysql-queries
Ideal fix
1. Figure out what is causing certain queries to obtain the status of "Waiting for table-level lock". Could have something to do with the database storage engine being InnoDB instead of MyISAM (http://stackoverflow.com/questions/6543624/what-is-waiting-for-table-level-lock)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment