bcse/remove_utf16.py

Created June 19, 2013 19:57

Star () You must be signed in to star a gist
Fork () You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/bcse/5817497.js"></script>
Save bcse/5817497 to your computer and use it in GitHub Desktop.

Download ZIP

Remove all UTF-16 characters by regular expression

Raw

remove_utf16.py

	txt = u'\U0001f600'
	r = re.compile(u'[\uD800-\uDBFF][\uDC00-\uDFFF]')
	r.sub('', txt)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment