kevinburke/hierarchy.md

## hierarchy.md

      
    Raw
  

              hierarchy.md
            
          
    An Ideal HTTP Exception Hierarchy

Here is how I'd like to group exceptions in urllib3/requests. I would argue that if anyone is catching/handling different types of errors in an HTTP request, they want to know:

was my request bad?
did my client screw up?
is the remote server slow?
did the remote server screw up?
is it safe to try this request again?

I would argue that these line up with the point in the request where the request failed. So telling someone an error was an SSLError is not too useful, as it could have happened during certificate negotiation (safe to retry), because the SSL module was not available (safe, and well, useless, to retry), or because the server timed out (probably not safe to retry). If errors are grouped by the point in the request/response cycle where the request failed, we get this information.
Here's how I would group all errors that can be raised during a request. I've covered everything that httplib raises and (I think) most socket-level errors that could be raised.
Invalid Request errors

These should be raised before any request is attempted

missing/invalid scheme
type errors
no url specified
host is something weird like an integer
HTTPS request, SSL module unavailable
SSL cert not found (would require moving cert_verify earlier in the request chain)

Connection Errors

These are errors which are raised while establishing a connection to a remote server.

DNS lookup failures
Most, but not all, socket errors (see the "Errors" section here)
Connection timeouts
CannotSendRequest, CannotSendHeader, NotConnected, InvalidURL in httplib (previous request made on socket, socket to be closed)

Request Errors

These are errors which occur while data was being sent to the server

anything that can be raised by socket.sendall (EINPROGRESS, ECONNRESET, ECONNABORTED, EPIPE, other errors)
errors establishing a SSL connection (CertificateError)
errors connecting to a proxy server

Read Timeout

Sort of its own category since it can happen at any point after the request has been sent.

EWOULDBLOCK, EAGAIN, or ETIMEDOUT
ECONNRESET can also be thrown on the read, more here

Response Error

These errors occur while trying to read/parse the response from the server.

httplib errors (ResponseNotReady, LineTooLong, BadStatusLine, UnknownProtocol, IncompleteRead)
followed too many redirects
chunked encoding errors
content decoding errors
response code errors (raise_for_status etc)


This lends itself to something like the following
class InvalidRequestException:
    """The request data was invalid and it was not even attempted."""

    # MissingSchema, InvalidURL, URLRequired, InvalidSchema, ImportError (SSL module not available), IOError(SSL certificate file not found)

class ConnectionError:
    """An error occurred while trying to connect to the remote server. This error is safe to retry"""
    
    # DNSLookupError (wraps socket.gaierror), ConnectionTimeoutError, SocketConnectionError are children

class RequestError:
    """An error occurred while sending your request to the remote server. The server may have 
    begun processing your request, though it is unlikely, since the request was not fully sent to the server ."""

    # CertificateError, SSLError, SocketWriteError, ProxyError are children

class ReadTimeoutError:
    """The server did not send any data in the allotted amount of time."""

class ResponseError:
    """An error occurred while attempting to parse the response from the server."""

    # HTTPError, ChunkedEncodingError, ContentDecodingError, TooManyRedirects, InvalidResponse(for the httplib errors described above)
What would have to change


ConnectTimeout and ReadTimeout currently subclass Timeout, this should be broken out so ConnectTimeout subclasses ConnectionError. While these are both timeouts, they have different implications (server down or not listening, vs slow to send a response)


ConnectionError currently handles ResponseNotReady, LineTooLong, BadStatusLine, UnknownProtocol, IncompleteRead, all of which imply the request made it to the server. These would have to be handled and moved to the ResponseError.


SocketError encompasses errors which occur while trying to open/connect a socket (see error codes above), while trying to write data to a socket, and while trying to read data to a socket. Confusingly some socket errors can be thrown during the connect and the read (ETIMEDOUT), or during the write and the read. There's not a great answer here besides try/catch around the relevant socket calls (thankfully separate in urllib3) and raise them as different errors. I propose SocketConnectError, SocketWriteError and SocketReadError.


SSLError includes the SSL module not being found, the certificate not found on disk, timeout errors, and certificate negotiation errors. I would split this out per the above - make the module not being found an ImportError, make an unavailable file an IOError.


In the requests library, TooManyRedirects, ChunkedEncodingError, and ContentDecodingError are all subclasses of RequestError. These would need to move to ResponseError.