Skip to content

Instantly share code, notes, and snippets.

@szrxiao

szrxiao/brand1k_perf

Last active October 25, 2017 22:02
Show Gist options
  • Save szrxiao/8de604b5c9cff0f6e9fcbc22e4bb5fbe to your computer and use it in GitHub Desktop.
Save szrxiao/8de604b5c9cff0f6e9fcbc22e4bb5fbe to your computer and use it in GitHub Desktop.
Download ZIP
Faster RCNN on brand1k dataset
Raw
brand1k_perf
Dataset 1: 822 categories, from VIG Brand1500, Flickr32 Logo, Flickr27 Logo, Belga Logo dataset,
Processing strategy:
1. keep images containing at least one medium/large size logo (>=32*32 pixels)
2. remove categories with containing less than 40 logos after step 1.
Results on overall objects (11044)
| IOU | MAP | Th@0.8 | Prec@0.8 | Rec@0.8 | Th@0.9 | Prec@0.9 | Rec@0.9 | Th@0.95 | Prec@0.95 | Rec@0.95 |
| :-- | :----: | :----: | :------: | :-----: | :----: | :------: | :-----: | :-----: | :-------: | :------: |
| 0.3 | 0.6249 | 0.7626 | 0.8 | 0.5872 | 0.9321 | 0.9001 | 0.4772 | 0.9891 | 0.9502 | 0.3022 |
| 0.4 | 0.6073 | 0.7864 | 0.8 | 0.5666 | 0.9551 | 0.9001 | 0.4341 | 0.9963 | 0.9503 | 0.1868 |
| 0.5 | 0.5721 | 0.8386 | 0.8 | 0.5235 | 0.9832 | 0.9001 | 0.3305 | 0.9995 | 0.9507 | 0.0367 |
Dataset 2: 583 categories, from _Dataset 1:
Processing strategy:
1. Remove images which aspect ratio >3 or <0.333
2. remove categories with containing less than 40 logos after step 1.
Results on overall objects (7286)
| IOU | MAP | Th@0.8 | Prec@0.8 | Rec@0.8 | Th@0.9 | Prec@0.9 | Rec@0.9 | Th@0.95 | Prec@0.95 | Rec@0.95 |
| :-- | :----: | :----: | :------: | :-----: | :----: | :------: | :-----: | :-----: | :-------: | :------: |
| 0.3 | 0.6743 | 0.7628 | 0.8 | 0.6584 | 0.9519 | 0.9001 | 0.5478 | 0.9961 | 0.95 | 0.3132 |
| 0.4 | 0.6612 | 0.7811 | 0.8001 | 0.6464 | 0.9684 | 0.9001 | 0.5082 | 0.9972 | 0.9501 | 0.2744 |
| 0.5 | 0.628 | 0.8371 | 0.8001 | 0.6087 | 0.9862 | 0.9001 | 0.4281 | 0.999 | 0.9501 | 0.1543 |
Dataset 3: 1048 categories,
1. keep images containing at least one medium/large size logo (>=32*32 pixels)
2. Remove images which aspect ratio >4 or <0.25
3. remove categories with containing less than 20 logos after step 2.
Results on overall objects (11545)
| IOU | MAP | Th@0.8 | Prec@0.8 | Rec@0.8 | Th@0.9 | Prec@0.9 | Rec@0.9 | Th@0.95 | Prec@0.95 | Rec@0.95 |
| :-- | :----: | :----: | :------: | :-----: | :----: | :------: | :-----: | :-----: | :-------: | :------: |
| 0.3 | 0.6483 | 0.7949 | 0.8 | 0.6081 | 0.9684 | 0.9 | 0.4696 | 0.9983 | 0.9503 | 0.1804 |
| 0.4 | 0.6336 | 0.8139 | 0.8 | 0.5936 | 0.9806 | 0.9 | 0.4249 | 0.999 | 0.9504 | 0.126 |
| 0.5 | 0.6038 | 0.8628 | 0.8001 | 0.5575 | 0.9907 | 0.9 | 0.3494 | 0.9993 | 0.95 | 0.0955 |
## YoloV2 darknet19 128 epochs Train on 416x416
| IOU | MAP | Th@0.8 | Prec@0.8 | Rec@0.8 | Th@0.9 | Prec@0.9 | Rec@0.9 | Th@0.95 | Prec@0.95 | Rec@0.95 |
| :-- | :----: | :----: | :------: | :-----: | :----: | :------: | :-----: | :-----: | :-------: | :------: |
| 0.3 | 0.0887 | 0.9387 | 1.0 | 0.0 | 0.9387 | 1.0 | 0.0 | 0.9387 | 1.0 | 0.0 |
| 0.4 | 0.0786 | 0.9387 | 1.0 | 0.0 | 0.9387 | 1.0 | 0.0 | 0.9387 | 1.0 | 0.0 |
| 0.5 | 0.0663 | 0.9387 | 1.0 | 0.0 | 0.9387 | 1.0 | 0.0 | 0.9387 | 1.0 | 0.0 |
Results on large objects (5444)
| IOU | MAP | Th@0.8 | Prec@0.8 | Rec@0.8 | Th@0.9 | Prec@0.9 | Rec@0.9 | Th@0.95 | Prec@0.95 | Rec@0.95 |
| :-- | :----: | :----: | :------: | :-----: | :----: | :------: | :-----: | :-----: | :-------: | :------: |
| 0.3 | 0.64 | 0.5423 | 0.8001 | 0.6352 | 0.7141 | 0.9001 | 0.4914 | 0.8101 | 0.9504 | 0.2851 |
| 0.4 | 0.6247 | 0.5726 | 0.8 | 0.61 | 0.7319 | 0.9001 | 0.4587 | 0.8229 | 0.9501 | 0.2485 |
| 0.5 | 0.5929 | 0.6258 | 0.8002 | 0.5628 | 0.7661 | 0.9002 | 0.3927 | 0.8282 | 0.9501 | 0.2309 |
Results on medium objects (5200)
| IOU | MAP | Th@0.8 | Prec@0.8 | Rec@0.8 | Th@0.9 | Prec@0.9 | Rec@0.9 | Th@0.95 | Prec@0.95 | Rec@0.95 |
| :-- | :----: | :----: | :------: | :-----: | :----: | :------: | :-----: | :-----: | :-------: | :------: |
| 0.3 | 0.5101 | 0.669 | 0.8001 | 0.3179 | 0.9445 | 1.0 | 0.0004 | 0.9445 | 1.0 | 0.0004 |
| 0.4 | 0.4893 | 0.7057 | 0.8002 | 0.2688 | 0.9445 | 1.0 | 0.0004 | 0.9445 | 1.0 | 0.0004 |
| 0.5 | 0.4497 | 0.7827 | 0.8004 | 0.1419 | 0.9445 | 1.0 | 0.0004 | 0.9445 | 1.0 | 0.0004 |
Results on overall objects (11545)
| IOU | MAP | Th@0.8 | Prec@0.8 | Rec@0.8 | Th@0.9 | Prec@0.9 | Rec@0.9 | Th@0.95 | Prec@0.95 | Rec@0.95 |
| :-- | :----: | :----: | :------: | :-----: | :----: | :------: | :-----: | :-----: | :-------: | :------: |
| 0.3 | 0.6404 | 0.4054 | 0.8 | 0.5718 | 0.6275 | 0.9001 | 0.4443 | 0.7743 | 0.9502 | 0.2528 |
| 0.4 | 0.6203 | 0.4326 | 0.8001 | 0.5505 | 0.6536 | 0.9001 | 0.4161 | 0.7941 | 0.9502 | 0.21 |
| 0.5 | 0.5801 | 0.4941 | 0.8001 | 0.5036 | 0.7003 | 0.9 | 0.358 | 0.8132 | 0.95 | 0.1664 |
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment