Can anybody tell me what's going on here? Everything I've read says to use "\N" for both STRINGs and non-STRINGs, but Hive is treating them differently. WTF?
Created
August 8, 2014 15:49
-
-
Save ryanbriones/e30166f202ce43460022 to your computer and use it in GitHub Desktop.
WTF NULL HIVE?
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
\\N\\N |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hive> SELECT * FROM default.null_test; | |
OK | |
NULL \N |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CREATE EXTERNAL TABLE default.null_test ( | |
int_test INT, | |
string_test STRING) | |
ROW FORMAT DELIMITED | |
FIELDS TERMINATED BY '\001' ESCAPED BY '\\' | |
STORED AS TEXTFILE | |
LOCATION '/user/rbriones/test_null'; |
This issue appears to be a bad combination of --escaped-by '\\'
and --null-string '\\N'
. Works great when removing --escaped-by
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Seems like the data in the file should actually be literal
\N
and not\\N
. It's getting in the data file that way because it (seems) impossible to specify\N
as an parameter in an Oozie Sqoop job.Also, non-STRING types seems to work fine since the value doesn't parse to those types and the underlying Java probably just returns null on parse errors...