Skip to content

Instantly share code, notes, and snippets.

@alotau
Created March 28, 2015 00:20
Show Gist options
  • Save alotau/8c47b923d326afebd9b3 to your computer and use it in GitHub Desktop.
Save alotau/8c47b923d326afebd9b3 to your computer and use it in GitHub Desktop.
A quick note about text elements going into sqlite databases wherein some of the test is in double quotes, but not all of it. Specifically, when the text in double quotes to start the text, you care going to have issues.
Just a note on double quotes in sqlite.
I have a workflow where I parse some data and generate an input file for a sqlite DB. Within sqlite I do the following:
sqlite> .separator |
sqlite> .import my_data_file.txt my_table
I was getting errors for some lines like the following:
my_data_file.txt:2389: unescaped " character
my_data_file.txt:2416: unescaped " character
my_data_file.txt:2416: unescaped " character
my_data_file.txt:3299: unescaped " character
my_data_file.txt:3299: unescaped " character
my_data_file.txt:4410: unescaped " character
my_data_file.txt:4410: unescaped " character
... many more, then the last one:
my_data_file.txt:2389: unterminated "-quoted field
Now, sure enough, each of those lines had data with a " in it. Normally, these are just warnings and the data are actually in the DB. However, notice the last error/warning has a line number that matched the first error/warning. Turns out that line had a data field that BEGAN with a double quote. sqlite took this to mean that the entire field should be enclosed in double quotes, whereas my data was actually something more like:
"In quotes" then not in quotes
So sqlite was wondering where the end quote was. It was expecting something like:
"In quotes then not in quotes"
The way around this was to just enclose the original data field in double quotes:
""In quotes" then not in quotes"
Now a SELECT for that row would correctly return:
"In quotes" then not in quotes
Hooray!
Now for all of those other pairs of warnings:
my_data_file.txt:3299: unescaped " character
my_data_file.txt:3299: unescaped " character
If you don't start the data element with quotes, those lines end up in the DB as expected and these warnings can be safely ignored.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment