Skip to content

Instantly share code, notes, and snippets.

@jonjensen
Created March 10, 2010 00:33
Show Gist options
  • Save jonjensen/327332 to your computer and use it in GitHub Desktop.
Save jonjensen/327332 to your computer and use it in GitHub Desktop.
#!/usr/local/bin/perl
use strict;
use warnings;
use Encode;
use IsUTF8 'isUTF8';
binmode(STDIN);
binmode(STDOUT);
binmode(STDERR);
no warnings 'uninitialized';
my $fallback = 'windows-1252';
if (@ARGV) {
# testing by only showing questionable output
while (<STDIN>) {
my $result = isUTF8($_);
print STDERR $_ if $result;
print encode_utf8(decode($fallback, $_)) if not defined $result;
}
}
else {
# the real thing
while (<STDIN>) {
my $result = isUTF8($_);
print, next if defined $result;
print encode_utf8(decode($fallback, $_)) if not defined $result;
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment