Skip to content

Instantly share code, notes, and snippets.

@ifels
Last active February 1, 2023 19:19
Show Gist options
  • Save ifels/10392762 to your computer and use it in GitHub Desktop.
Save ifels/10392762 to your computer and use it in GitHub Desktop.
golang pipe the http response
package main
import (
"io"
"net/http"
"os/exec"
)
var (
BUF_LEN = 1024
)
func handler(w http.ResponseWriter, r *http.Request) {
cmd := exec.Command("./build.sh")
pipeReader, pipeWriter := io.Pipe()
cmd.Stdout = pipeWriter
cmd.Stderr = pipeWriter
go writeCmdOutput(w, pipeReader)
cmd.Run()
pipeWriter.Close()
}
func writeCmdOutput(res http.ResponseWriter, pipeReader *io.PipeReader) {
buffer := make([]byte, BUF_LEN)
for {
n, err := pipeReader.Read(buffer)
if err != nil {
pipeReader.Close()
break
}
data := buffer[0:n]
res.Write(data)
if f, ok := res.(http.Flusher); ok {
f.Flush()
}
//reset buffer
for i := 0; i < n; i++ {
buffer[i] = 0
}
}
}
func main() {
http.HandleFunc("/", handler)
http.ListenAndServe(":8080", nil)
}
@lox
Copy link

lox commented Sep 11, 2017

This was a life saver, but I'd love to know a bit more about why this is needed. I was previously using io.Copy and was just seeing output hanging.

@linkerlin
Copy link

pipeWriter.Close() is important

@luoqeng
Copy link

luoqeng commented Sep 20, 2018

@XiaYinchang
Copy link

XiaYinchang commented Nov 18, 2019

This was a life saver, but I'd love to know a bit more about why this is needed. I was previously using io.Copy and was just seeing output hanging.

I think io.Copy is fine but need to do that in another go routine.
Just something like this:

package main

import (
	"io"
	"net/http"
	"os/exec"
)

func handler(w http.ResponseWriter, r *http.Request) {
	cmd := exec.Command("ls")
	pipeReader, pipeWriter := io.Pipe()
	cmd.Stdout = pipeWriter
	cmd.Stderr = pipeWriter
	go io.Copy(w, pipeReader)
	cmd.Run()
	pipeWriter.Close()
}

func main() {
	http.HandleFunc("/", handler)
	http.ListenAndServe(":8080", nil)
}

@ahmetozer
Copy link

Some times while processing f.Flush() i get a SIGSEGV err

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x52ddbc]

prevent this defer recovery function before f.Flush()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment