Project: QUIC Support in mitmproxy
Contributor: Manuel Meitinger
Organization: The Honeynet Project
Mentor: Maximilian Hils
The goal of this Google Summer of Code project was adding QUIC support to mitmproxy. QUIC is a new "UDP-based multiplexed and secure transport" protocol1, and also the foundation for the next version of HTTP: HTTP/32. Before this project, mitmproxy, a "free and open source interactive HTTPS proxy"3, supported only HTTP/1 and HTTP/2, and had no UDP support.
The following section details what has changed since then.
-
Add DNS support. (#5232)
The first step was to introduce UDP in mitmproxy. Since having a transport protocol without a top-layer isn't really all that useful, a DNS server/proxy was implemented as well. (QUIC was still far down the line at this point and not readily available.) -
Add support for raw UDP. (#5414)
After UDP support had been added, other people started implementing new features on top of it, one of which is DTLS. This made it necessary to also show raw, transmitted UDP data in the UI. (Previously only DNS flows had been made visible.) Since QUIC should offer similar functionality, implementing a raw UDP layer was given priority. -
Add QUIC support. (#5435)
The draft PR landed on Jun 27th. The first runnable version, which featured raw QUIC and HTTP/3 support, was #d94345b. After that, changes were made to how mitmproxy handles multiple modes, which meant that mitmproxy would no longer listen on just one socket per protocol. This, together with concerns about alterations to core layer events4, required a rewrite of QUIC's roaming code and layer decision logic, and led to the next couple of PRs. -
Include server information in bind/listen errors. (#5495)
With multiple modes listening on different IP/ports, the information that mitmproxy provided in case of an error was insufficient for users to deduce which mode is failing. The PR rectified that, also changing the error description to include command line samples on how to fix the issue. -
Fix running hook issue and signal proxyserver updates. (#5512)
Having multiple modes implies having multiple server instances, which can fail independently of the main proxyserver addon. This could have led to a state, where mitmproxy reported 'running', only to crash immediately afterwards. The PR fixed that, while also introducing a mechanism to listen for start/stop events. -
Unify proxy modes, introduce UDP protocol detection. (#5556)
In preparation for QUIC, and to consolidate all added protocols (UDP, DTLS, DNS, etc.) and modes (regular, reverse, etc.), the command line syntax and source was changed to reduce the amount of different arguments, as well as making it easier for new protocols using existing modes to be introduced.
Some additional (but minor) PRs have been submitted and merged as well. For a complete list see https://github.com/mitmproxy/mitmproxy/pulls?q=is:pr+author:meitinger.
The aforementioned rewrite of QUIC's roaming code and layer decision logic is still ongoing. Also the unification of proxy modes started in #5556 will continue once all UDP protocols are deemed stable enough.
Another thing - not directly related to QUIC but rather concerning all UDP protocols - is better support for transparent mode, which had actually been implemented in #5232. Since it only supported Linux and wasn't very user-friendly to setup, it got removed but will be reintroduced later.
The project helped mitmproxy reach the second half of IP-based protocols, i.e. those based on UDP. It also showed that QUIC and HTTP/3 can (with the possible exception of roaming) be implemented in fairly straight forward and isolated way, regarding complexity and dependencies.
The missing pieces, together with a thorough evaluation of performance and compatibility, will be completed as part of a master's thesis. (Link will be provided once it's done.)
Lastly, I'd like to thank Maximilian Hils for all his invaluable feedback and Google for organizing GSoC and their tremendous open source efforts.