Created
November 25, 2020 16:26
-
-
Save andreoss/99b9d3f10c6bec16d45b23a431759e7b to your computer and use it in GitHub Desktop.
Remove >3 bytes characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env perl | |
use strict; | |
use warnings; | |
use open qw( :encoding(UTF-8) :std ); | |
use English qw( -no_match_vars ); | |
while ( my $line = <> ) { | |
my $count = $line =~ s{ | |
[\N{U+07FF}-\N{U+7FFFFFF}] | |
} [?]xg; | |
if ($count) { | |
print {*STDERR} "Bad $count symbols in file $ARGV, line $INPUT_LINE_NUMBER\n"; | |
} | |
print $line; | |
} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment