Skip to content

Instantly share code, notes, and snippets.

@guitarrapc
Created February 6, 2018 07:29
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save guitarrapc/91ecadc664021a8a994dc17fb15cffa4 to your computer and use it in GitHub Desktop.
Save guitarrapc/91ecadc664021a8a994dc17fb15cffa4 to your computer and use it in GitHub Desktop.
file encoding detector based on Mozilla Universal Charset Detector by https://github.com/errepi/ude
void Main()
{
DetectEncodings(Directory.EnumerateFiles(@"PATH to your project", "*.cs", SearchOption.AllDirectories)
.Where(x => !x.Contains(@"\bin\") && !x.Contains(@"\obj\")))
.Where(x => x.Charset != "UTF-8" && x.Charset != "ASCII" && x.Charset != null)
.Dump();
}
IEnumerable<(string Path, string Charset)> DetectEncodings(IEnumerable<string> paths)
{
var detector = new Ude.CharsetDetector();
foreach (var path in paths)
{
using (var stream = File.OpenRead(path))
{
detector.Feed(stream);
detector.DataEnd();
yield return (path, detector.Charset);
detector.Reset();
}
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment