A pretty small Go application compressing from stdin to stdout as gzip.

gzipstream.go

package main

import (
	"compress/gzip"
	"fmt"
	"io"
	"os"
	"strconv"
	"time"
)

func main() {
	level := gzip.DefaultCompression
	if len(os.Args) >= 2 {
		l, err := strconv.ParseInt(os.Args[1], 10, 0)
		if err != nil {
			panic(err)
		}
		level = int(l)
	}
	out, err := gzip.NewWriterLevel(os.Stdout, level)
	if err != nil {
		panic(err)
	}
	defer out.Close()
	before := time.Now().UnixNano()
	written, err := io.Copy(out, os.Stdin)
	duration := float64(time.Now().UnixNano() - before)
	if err != nil && err != io.EOF {
		panic(err)
	}
	fmt.Fprintf(os.Stderr, "Compressed %d bytes in %f ms\n", written, duration/(1000*1000))
}

I built this to compare compression speeds of Go, gzip and pigz.

It accepts one (optional) parameter: the compression level.

• default: -1
• none: 0
• fastest: 1
• smallest: 9
  See http://golang.org/pkg/compress/flate/#pkg-constants

Compressing the binary created with go build gzipstream.go with time COMMAND < gzipstream > gzipstream.gz takes Go twice as long as gzip with the best compression (9). On my i7, pigz wins hands down (about 1/4 real time compared to gzip).

time ./gzipstream 9 < gzipstream > test-go.gz
time gzip -c -9 < gzipstream > test-gzip.gz
time pigz -c -9 < gzipstream > test-pigz.gz