Skip to content

Instantly share code, notes, and snippets.

@KillyMXI
Created November 18, 2017 19:21
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save KillyMXI/e19905ecf123f097cabcff6c1c999188 to your computer and use it in GitHub Desktop.
Save KillyMXI/e19905ecf123f097cabcff6c1c999188 to your computer and use it in GitHub Desktop.
Haskell Unicode normalization
import Data.Text.Normalize
import qualified Data.Text as T
import GHC.IO.Encoding
import System.Win32.Console
procString :: (T.Text -> T.Text) -> String -> String
procString f = T.unpack . f . T.pack
testStr = "😀éé"
main :: IO ()
main = do
setLocaleEncoding utf8
setConsoleOutputCP 65001 -- Windows specific
print $ length $ procString (normalize NFD) $ testStr
print $ length $ procString (normalize NFKD) $ testStr
print $ length $ procString (normalize NFC) $ testStr
print $ length $ procString (normalize NFKC) $ testStr
putStrLn $ procString (T.reverse . (normalize NFC)) $ testStr
-- Output:
-- 5
-- 5
-- 3
-- 3
-- éé😀
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment