Skip to content

Instantly share code, notes, and snippets.

@yashh
Created December 14, 2010 02:55
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save yashh/739942 to your computer and use it in GitHub Desktop.
Save yashh/739942 to your computer and use it in GitHub Desktop.
pulled from django
def force_unicode(s, encoding='utf-8', errors='ignore'):
"""
Returns a unicode object representing 's'. Treats bytestrings using the
'encoding' codec.
"""
import codecs
if s is None:
return ''
try:
if not isinstance(s, basestring,):
if hasattr(s, '__unicode__'):
s = unicode(s)
else:
try:
s = unicode(str(s), encoding, errors)
except UnicodeEncodeError:
if not isinstance(s, Exception):
raise
# If we get to here, the caller has passed in an Exception
# subclass populated with non-ASCII data without special
# handling to display as a string. We need to handle this
# without raising a further exception. We do an
# approximation to what the Exception's standard str()
# output should be.
s = ' '.join([force_unicode(arg, encoding, errors) for arg in s])
elif not isinstance(s, unicode):
# Note: We use .decode() here, instead of unicode(s, encoding,
# errors), so that if s is a SafeString, it ends up being a
# SafeUnicode at the end.
s = s.decode(encoding, errors)
except UnicodeDecodeError, e:
if not isinstance(s, Exception):
raise UnicodeDecodeError (s, *e.args)
else:
# If we get to here, the caller has passed in an Exception
# subclass populated with non-ASCII bytestring data without a
# working unicode method. Try to handle this without raising a
# further exception by individually forcing the exception args
# to unicode.
s = ' '.join([force_unicode(arg, encoding, errors) for arg in s])
return s
# from http://www.codigomanso.com/en/2010/05/una-de-python-force_unicode/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment