Skip to content

Instantly share code, notes, and snippets.

@t-sin
Last active February 7, 2017 15:41
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save t-sin/b46c9171d7d184687812f7bc03f96050 to your computer and use it in GitHub Desktop.
Save t-sin/b46c9171d7d184687812f7bc03f96050 to your computer and use it in GitHub Desktop.
UTF-8 character boundaries mis-recognized with flexi-streams, on SBCL
(ql:quickload '(:flexi-streams :circular-streams))
(setf s "こんにちは")
(setf o (flex:string-to-octets s :external-format :utf-8))
(with-input-from-string (in s)
(format t "read from stream: ~s~%" (read-char in)))
;; => read from stream: #\HIRAGANA_LETTER_KO
(flex:with-input-from-sequence (in o)
(format t "read from circular-stream: ~s~%"
(trivial-gray-streams:stream-read-char
(flex:make-flexi-stream in))))
;; => read from circular-stream: #\LATIN_SMALL_LETTER_A_WITH_TILDE
(print (code-char (aref o 0)))
;; => #\LATIN_SMALL_LETTER_A_WITH_TILDE
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment