Skip to content

Instantly share code, notes, and snippets.

@andreoss
Created November 25, 2020 16:26
Show Gist options
  • Save andreoss/99b9d3f10c6bec16d45b23a431759e7b to your computer and use it in GitHub Desktop.
Save andreoss/99b9d3f10c6bec16d45b23a431759e7b to your computer and use it in GitHub Desktop.
Remove >3 bytes characters
#!/usr/bin/env perl
use strict;
use warnings;
use open qw( :encoding(UTF-8) :std );
use English qw( -no_match_vars );
while ( my $line = <> ) {
my $count = $line =~ s{
[\N{U+07FF}-\N{U+7FFFFFF}]
} [?]xg;
if ($count) {
print {*STDERR} "Bad $count symbols in file $ARGV, line $INPUT_LINE_NUMBER\n";
}
print $line;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment