Skip to content

Instantly share code, notes, and snippets.

@scbizu
Created March 7, 2017 12:49
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save scbizu/23a7d3f3436a35ab3b79c3cce91ca8fe to your computer and use it in GitHub Desktop.
Save scbizu/23a7d3f3436a35ab3b79c3cce91ca8fe to your computer and use it in GitHub Desktop.
How Golang solve the invalid UTF-8 character?
package main
import (
"fmt"
"unicode/utf8"
)
func main() {
s := "a\xc5z"
fmt.Printf("%q\n", s)
if !utf8.ValidString(s) {
v := make([]rune, 0, len(s))
for i, r := range s {
if r == utf8.RuneError {
_, size := utf8.DecodeRuneInString(s[i:])
if size == 1 {
continue
}
}
v = append(v, r)
}
s = string(v)
}
fmt.Printf("%q\n", s)
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment