Skip to content

Instantly share code, notes, and snippets.

@chricke

chricke/create_records.sh

Last active March 28, 2019 19:01
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save chricke/fe5c873abda6107b1ac8ceb58b83fb56 to your computer and use it in GitHub Desktop.
Save chricke/fe5c873abda6107b1ac8ceb58b83fb56 to your computer and use it in GitHub Desktop.
Download ZIP
create TFRecords
Raw
create_records.sh
lumi dataset transform --type csv --data-dir data/ --output-dir tfdata/ --split train --split val --only-classes=table
@jyoti59n
Copy link

Hi,

I'm unable to run the above luminoth script - trying to replicate the run on the same set of train/val images that you have used. I get the following message:

'Columns missing from CSV: {}'.format(missing_keys)
luminoth.tools.dataset.readers.base_reader.InvalidDataDirectory: Columns missing from CSV: {'xmin', 'ymax', 'xmax', 'image_id', 'label', 'ymin'}

Also notice that this error throws up for a random order of the csv columns for different runs. e.g. in an earlier run the message was
luminoth.tools.dataset.readers.base_reader.InvalidDataDirectory: Columns missing from CSV: {'xmax', 'ymin', 'image_id', 'xmin', 'ymax', 'label'}

Would you be able to suggest what I'm doing wrong here? Thanks.

@goodhamgupta
Copy link

goodhamgupta commented Jan 16, 2019 •
edited

Hi @jyoti59n,

There is a problem with the given train.csv and val.csv files. The header rows are missing because of which you get the missing columns error(Check the _check_csv function of the csv_reader in luminoth here).

Add the following to the first line of both the flies:

image_id,xmin,ymin,xmax,ymax,label

It should work after this change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment