Last active
December 10, 2023 16:26
-
-
Save Ozoniuss/87e78f8e35d2ad147d3b81149e438308 to your computer and use it in GitHub Desktop.
bufio.Scanner benchmark
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package main | |
import ( | |
"bufio" | |
"os" | |
"testing" | |
) | |
var ret = "" | |
func BenchmarkBufio(b *testing.B) { | |
f, _ := os.Open("moby.txt") | |
defer f.Close() | |
var line string | |
for n := 0; n < b.N; n++ { | |
f.Seek(0, 0) | |
s := bufio.NewScanner(f) | |
for s.Scan() { | |
line = s.Text() | |
} | |
} | |
ret = line | |
} | |
// $ go test -bench=. -benchmem main_test.go | |
// goos: windows | |
// goarch: amd64 | |
// cpu: AMD Ryzen 7 5800H with Radeon Graphics | |
// BenchmarkBufio-16 848 1394192 ns/op 1412961 B/op 19231 allocs/op | |
// PASS | |
// ok command-line-arguments 1.439s | |
// | |
// mem profile shows that s.Text() allocates | |
// | |
// Note that it is possible to use s.Bytes() to avoid the allocations. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
(Apologies if you saw a version of the below with a mistake in it—this is now corrected.)
If you call
string
, it'd allocate. It's identical tos.Text()
.If you used the same unsafe trick that (*strings.Builder).String uses, namely
then you'd avoid the allocation, but it'd be unsafe. The next scan may overwrite the underlying data in the string, which breaks the guarantee that strings are immutable. For example,
prints out
for me.
The interfaces of the bytes and strings packages are almost isomorphic so if you're only wanting to handle the string 'immediately' as a reader then you can avoid the allocation by handling it as a byte slice and doing the necessary work within the scanning loop.
(Note that there are some differences between the packages, e.g. bytes.Runes exists but strings.Runes doesn't.)