Skip to content

Instantly share code, notes, and snippets.

@datashaman
Last active November 9, 2023 01:05
Show Gist options
  • Save datashaman/a517da0ebfe7939c6b83 to your computer and use it in GitHub Desktop.
Save datashaman/a517da0ebfe7939c6b83 to your computer and use it in GitHub Desktop.
More succinct resilient session
from requests import Session
import time
class ResilientSession(Session):
"""
This class is supposed to retry requests that return temporary errors.
At this moment it supports: 502, 503, 504
"""
def request(self, method, url, **kwargs):
counter = 0
while True:
counter += 1
r = super(ResilientSession, self).request(method, url, **kwargs)
if r.status_code in [ 502, 503, 504 ]:
delay = 10 * counter
logging.warn("Got recoverable error [%s] from %s %s, retry #%s in %ss" % (r.status_code, method, url, counter, delay))
time.sleep(delay)
continue
return r
if __name__ == '__main__':
s = ResilientSession()
s.get('http://httpstat.us/502')
@datashaman
Copy link
Author

In the Session class, all HTTP methods are channelled through the request method, so adding the retry there makes it work for all HTTP methods.

I have changed the behaviour slightly. Exceptions are not caught, so the controlling process chooses how to deal with them. Also the log message is slightly different (no HTTP in front of status_code).

@sladkovm
Copy link

sladkovm commented Jun 3, 2017

I think you need to add the exit clause for the while loop... at least I had to do it to get it working:

...
                elif r.status_code==200:
                    break
                else:
                    raise ConnectionError

The last else is to cover for all other status codes

@datashaman
Copy link
Author

datashaman commented Aug 14, 2017

Argh, I have the return r in the wrong place. Thanks @sladkovm!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment