Skip to content

Instantly share code, notes, and snippets.

@shoooe

shoooe/ngrams.hs

Created April 16, 2015 23:02
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save shoooe/2fb6a538e1c9893ecd7a to your computer and use it in GitHub Desktop.
Save shoooe/2fb6a538e1c9893ecd7a to your computer and use it in GitHub Desktop.
Download ZIP
N-grams!
Raw
ngrams.hs
groupEvery :: Int -> [a] -> [[a]]
groupEvery n lst
| length lst < n = []
| otherwise = take n lst : groupEvery n (tail lst)
ngrams :: Int -> String -> [String]
ngrams n = concatMap (groupEvery n . fillWord '_') . words
where fillWord f w = f : (w ++ replicate (n - 1) f)
main :: IO ()
main = print $ ngrams 3 "This is simple and a example"
-- ["_Th","Thi","his","is_","s__","_is","is_","s__","_si","sim","imp"
-- ,"mpl","ple","le_","e__","_an","and","nd_","d__","_a_","a__","_ex"
-- ,"exa","xam","amp","mpl","ple","le_","e__"]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment