Created
June 2, 2013 17:46
-
-
Save dpk/5694265 to your computer and use it in GitHub Desktop.
Python: iterate over the graphemes in a string.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import unicodedata as u | |
def itergraphemes(str): | |
def modifierp(char): return u.category(char)[0] == 'M' | |
start = 0 | |
for end, char in enumerate(str): | |
if not modifierp(char) and not start == end: | |
yield str[start:end] | |
start = end | |
yield str[start:] |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
(This is broken: the definition of a 'grapheme' in Unicode is more complex than I thought. See http://unicode.org/reports/tr29/#Grapheme_Cluster_Boundaries for the actual definition — Python's unicodedata does not expose enough character data to make this work.)