ImageNet validation set fix:
- The training set is organized in directories, with each directory matching a class, e.g. "n01751748" matching "sea snake." However, the valset is a flat dir of JPEGs.
The ImageNet labels provided in the devkit for the validation set (ILSVRC2012_validation_ground_truth.txt) are not consistent with the ordering used by PyTorch/TF/Keras/MXNet/Caffe, etc. for pre-trained models.
For example, in the the above ground truth label file, "sea snake" is 490, but in PyTorch/TF, it's 65.
Proof:- ImageNet validation labels: https://gist.githubusercontent.com/aaronpolhamus/964a4411c0906315deb9f4a3723aac57/raw/aa66dd9dbf6b56649fa3fab83659b2acbf3cbfd1/map_clsloc.txt
- DL framework labels: https://gist.githubusercontent.com/aaronpolhamus/964a4411c0906315deb9f4a3723aac57/raw/aa66dd9dbf6b56649fa3fab83659b2acbf3cbfd1/map_clsloc.txt
- Untar the valset file, you will get a flat dir of JPEGs.
- Pull in the unflattening script into the directory where the val images were unpacked:
https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh - Run the valprep.sh script - it will re-create the "n01440764" etc. directory structure familiar from the training set.
- DO NOT use the original ILSVRC 2012 ground truth validation labels to match JPEGs to classes. Instead, use the DL framework directory-class mapping (where "sea snake" is 0-indexed 65, NOT 490) to generate labels for directory-assigned images, with directories generated using valprep.sh. This will match pre-created TFRecord/lmdb/RecordIO etc. and will match the predictions of pre-trained models, e.g. from TorchVision.