Skip to content

Instantly share code, notes, and snippets.

@jacoby
Created February 13, 2015 15:29
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jacoby/77a9439cfb27fa5aa8fb to your computer and use it in GitHub Desktop.
Save jacoby/77a9439cfb27fa5aa8fb to your computer and use it in GitHub Desktop.
Working through UTF8 issues with MySQL
jacoby@oz:~$ my_oz -e 'select * from utf8_test '
+----+-----------------------------------------------------------------------------------------------------------------------+
| id | string |
+----+-----------------------------------------------------------------------------------------------------------------------+
| 1 | Begin |
| 2 | ♈ ♉ ♊ ♋ ♌ ♍ ♎ ♏ ♐ ♑ ♒ ♓ ⛎ |
| 3 | End |
| 4 | 0 — 1 — 10 — 2 — 3 — 4 — 5 — 6 — 7 — 8 — 9 — ♈ — ♉ — ♊ — ♋ — ♌ |
| 5 | 0 ???? 1 ???? 10 ???? 2 ???? 3 ???? 4 ???? 5 ???? 6 ???? 7 ???? 8 ???? 9 ???? ♈ ???? ♉ ???? ♊ ???? ♋ ???? ♌ |
+----+-----------------------------------------------------------------------------------------------------------------------+
# Tried this in Perl 5.18.4
use feature qw'say state' ;
use strict ;
use utf8 ;
use warnings ;
use Data::Dumper ;
use lib '/home/jacoby/lib' ;
use DB ; # wrapper for MySQL interaction
binmode STDOUT, ':utf8' ;
for my $wide ( map { " $_ " } "\x{2014}", "\x{1F4A9}" ) {
# 2014 == em-dash
# 1F4A9 == pile-of-poo
# join an array of mixed ASCII and UTF8 chars with either
# dash or poo
my $string = join $wide, sort 0 .. 10, qw{♈ ♉ ♊ ♋ ♌} ;
my $insert = 'INSERT INTO utf8_test ( string )VALUES( ? )' ;
db_do( $insert, $string ) ; # wrapper for easy data import
}
DROP TABLE IF EXISTS utf8_test ;
CREATE TABLE utf8_test (
id int( 10 ) NOT NULL PRIMARY KEY auto_increment ,
string char( 255 ) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL
)
ENGINE="InnoDB"
CHARACTER SET utf8
;
INSERT INTO utf8_test ( string ) VALUES
( 'Begin' ) ,
( '♈ ♉ ♊ ♋ ♌ ♍ ♎ ♏ ♐ ♑ ♒ ♓ ⛎' ) ,
( 'End' )
;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment