Skip to content

Instantly share code, notes, and snippets.

@jonathanrobie
Created June 27, 2018 16:54
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jonathanrobie/cea26d9318a7b8b0413103329479807a to your computer and use it in GitHub Desktop.
Save jonathanrobie/cea26d9318a7b8b0413103329479807a to your computer and use it in GitHub Desktop.
Convert files to Unicode (NFKC)
#!/usr/bin/env python3
import unicodedata
import fnmatch
import sys
import os
for file in os.listdir('.'):
for arg in sys.argv:
if fnmatch.fnmatch(file, arg):
inf = file + '.bak'
os.rename(file, inf)
with open(inf) as inf:
with open(file,'w+') as outf:
for line in inf:
outf.write(unicodedata.normalize("NFKC", line))
@jonathanrobie
Copy link
Author

Converts all files in the current directory to Unicode, using NFKC normalization. Creates .bak files just in case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment