The chunked responses were originally suggested and discussed in the first network spec PR: ethereum/consensus-specs#1158
Here are the relavant bits from a possible spec:
Response -- After receiving a request, the responding party MUST reply on the same stream within 10 seconds with a response code, followed by the contents of the response.
Certain RPCs MAY allow for multiple responses to be generated for each request. The responder MUST transmit the Success response code before each serialized message written to the stream. The responder MUST close the stream when no more messages are expected. If an error is encountered after some messages have already been written, the responder MUST transmit the ServerError response code, followed by a serialized mesage using the ServerErrorMsg schema.
The request initiator MAY close the streat at any time.
Reduced latency
Chunked responses add the ability to stream large responses such as list of blocks asynchronously as the data required to serve them is gradually fetched from some slower I/O device. You don't need to wait for the entire response to be ready before you can start sending data.
Reduced memory footprint
Without chunked responses, we must use SSZ lists which require a list of correct offset values to be written before the actual data forcing the clients to buffer the entire response in memory before sending it.
A new design tool for future protocols - peer-based pub/sub schemes
The chunked streams can be used to implement a pub/scheme between two peers. The requesting peers opens a stream by making a request and the responding peer writes chunked messages to the stream when certain events happen. This extra ability can be taken into consideration in the design of future protocols.