Skip to content

Instantly share code, notes, and snippets.

@ZenGround0
Last active August 8, 2023 04:55
Show Gist options
  • Save ZenGround0/af448f56882c16aaf10f39db86b4991e to your computer and use it in GitHub Desktop.
Save ZenGround0/af448f56882c16aaf10f39db86b4991e to your computer and use it in GitHub Desktop.
golang http auto-streaming with transfer-encoding: chunked
package main
import (
"net/http"
"net/url"
"os"
"fmt"
)
func main() {
tr := http.DefaultTransport
client := &http.Client{
Transport: tr,
Timeout: 0,
}
r := os.Stdin
req := &http.Request{
Method: "POST",
URL: &url.URL{
Scheme: "http",
Host: "localhost:9094",
Path: "/",
},
ProtoMajor: 1,
ProtoMinor: 1,
ContentLength: -1,
Body: r,
}
fmt.Printf("Doing request\n")
_, err := client.Do(req)
fmt.Printf("Done request. Err: %v\n", err)
}
package main
import (
"net/http"
"net"
"fmt"
"io"
)
func handle(w http.ResponseWriter, req *http.Request) {
buf := make([]byte, 256)
var n int
for {
n, err := req.Body.Read(buf)
if err == io.EOF {
break
}
fmt.Printf(string(buf[:n]))
}
fmt.Printf(string(buf[:n]))
fmt.Printf("TRANSMISSION COMPLETE")
}
func main() {
/* Net listener */
n := "tcp"
addr := "127.0.0.1:9094"
l, err := net.Listen(n, addr)
if err != nil {
panic("AAAAH")
}
/* HTTP server */
server := http.Server{
Handler: http.HandlerFunc(handle),
}
server.Serve(l)
}
@ZenGround0
Copy link
Author

I made this simple proof of concept to test out go's support for http streaming via the transfer-encoding: chunked header. It is seamlessly part of the standard http clients and servers.

The client will send out chunked http requests provided the request is of version HTTP/1.1 and the content-length is set to a non-positive value. The client reads from the input stream (here os.Stdin, but it could be any reader) and immediately sends chunks to the server. As more data arrives on the input stream more chunks are passed to the server.

The server receives the request and accesses a stream of data taken from HTTP chunk messages via the request body reader. The server consumes data immediately as it arrives by reading the body's data into a buffer. Here the data is simply printed but it could be streamed elsewhere for further processing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment