Skip to content

Instantly share code, notes, and snippets.

@d4l3k
Last active February 16, 2023 10:56
Show Gist options
  • Star 31 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save d4l3k/fc32c65049c88a509a90 to your computer and use it in GitHub Desktop.
Save d4l3k/fc32c65049c88a509a90 to your computer and use it in GitHub Desktop.
Golang Performance Tricks

This is a gist with a set of helpful performance tricks and best practices that I've found on the internet.

Fasthttp best practices

  • Do not allocate objects and []byte buffers - just reuse them as much as possible. Fasthttp API design encourages this.
  • sync.Pool is your best friend.
  • Profile your program in production. go tool pprof --alloc_objects your-program mem.pprof usually gives better insights for optimization opportunities than go tool pprof your-program cpu.pprof.
  • Write tests and benchmarks for hot paths.
  • Avoid conversion between []byte and string, since this may result in memory allocation+copy. Fasthttp API provides functions for both []byte and string - use these functions instead of converting manually between []byte and string.
  • Verify your tests and production code under race detector on a regular basis.

Tricks with []byte buffers

The following tricks are used by fasthttp. Use them in your code too.

  • Standard Go functions accept nil buffers
var (
	// both buffers are uninitialized
	dst []byte
	src []byte
)
dst = append(dst, src...)  // is legal if dst is nil and/or src is nil
copy(dst, src)  // is legal if dst is nil and/or src is nil
(string(src) == "")  // is true if src is nil
(len(src) == 0)  // is true if src is nil
src = src[:0]  // works like a charm with nil src

// this for loop doesn't panic if src is nil
for i, ch := range src {
	doSomething(i, ch)
}

So throw away nil checks for []byte buffers from you code. For example,

srcLen := 0
if src != nil {
	srcLen = len(src)
}

becomes

srcLen := len(src)
  • String may be appended to []byte buffer with append
dst = append(dst, "foobar"...)
  • []byte buffer may be extended to its' capacity.
buf := make([]byte, 100)
a := buf[:10]  // len(a) == 10, cap(a) == 100.
b := a[:100]  // is valid, since cap(a) == 100.
  • All fasthttp functions accept nil []byte buffer
statusCode, body, err := fasthttp.Get(nil, "http://google.com/")
uintBuf := fasthttp.AppendUint(nil, 1234)
@dio
Copy link

dio commented Dec 17, 2017

Re: Cockroach DB article, on the array-slice treatment.

https://twitter.com/CockroachDB/status/940250460052709376

The blog post is a bit old. That piece of code was removed in cockroachdb/cockroach#3337 and we made further improvements to allocations in this area in cockroachdb/cockroach#6175 Hope that helps! =)

https://twitter.com/CockroachDB/status/940265165983019008

It's not exactly the same thing, but we still do something similar with putBuffer.tmpbuf: https://t.co/NHJSWpBEfn

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment