gistfile1.txt

(fair_env) matthijs@devfair0144:~/src/low-shot-with-diffusion$ python diffusion.py --mode test --nlabeled 2 --seed 1 --nbg 1000000 --niter 3
========================== run on Test
load train + test set
nb of eval classes 311
/public/apps/anaconda2/5.0.1/envs/fair_env/lib/python2.7/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
open  /checkpoint/matthijs/low-shot/features/val.hdf5
read  /checkpoint/matthijs/low-shot/features/val_features.npy
   features loaded in 3.415 s
nb of eval classes 311
open  /checkpoint/matthijs/low-shot/features/train.hdf5
read  /checkpoint/matthijs/low-shot/features/train_features.npy
   features loaded in 87.034 s
selecting images, seed=1
load /checkpoint/matthijs/low-shot/features/PCAR256.vt
dataset sizes: Xtr (250860, 256) (250860 labeled), Xte (25350, 256), 507 classes (eval on 25350 examples, 507 classes, 311 novel classes)
memmapping  /checkpoint/matthijs/low-shot/features//f100m/concatenated_PCAR256.raw
[0.000 s, 0.36 GiB] make knn graph for 250860+1000000 k=30 d=256
  [0.000 s, 0.36 GiB] make distractor graph for ndis=1000000 k=30
  [0.000 s, 0.36 GiB] fname_base= /checkpoint/matthijs/low-shot/knngraph/ndis1000000
  [0.001 s, 0.36 GiB] load /checkpoint/matthijs/low-shot/knngraph/ndis1000000.index
   moving to 2 GPUs
      done in 6.657 s
  [7.566 s, 3.07 GiB] mmap /checkpoint/matthijs/low-shot/knngraph/ndis1000000_k32_D_11.float32 /checkpoint/matthijs/low-shot/knngraph/ndis1000000_k32_I_11.int32
  [0.006 s, 3.07 GiB] distractor graph ready
[7.573 s, 3.07 GiB] spherifying all descriptors
[0.337 s, 3.08 GiB] search labelled (250860, 256)
    search 245760:250860 / 250860
[7.780 s, 3.08 GiB] search test (25350, 256)
[1.132 s, 2.58 GiB] make Flat index for labelled
   moving to 2 GPUs
      done in 0.328 s
[0.495 s, 3.08 GiB] alloc output
[0.000 s, 3.08 GiB] search labelled
[2.215 s, 3.08 GiB] search distractors
    search 999424:1000000 / 1000000
[9.513 s, 4.08 GiB] search test
[0.361 s, 3.58 GiB] merge results
[0.080 s, 3.55 GiB] graph done
[3.55 GiB] begin diffusion
   knngraph_to_CSRMatrix in: 3.55 GiB
      A: 3.55 GiB
      B: 4.32 GiB
      C: 4.79 GiB
      D: 4.79 GiB, matrix nnz=64074274
[1.233 s, 4.79 GiB] preproc done
  [0.000 s, 4.79 GiB] Classes 0:507
  [20.490 s, 7.20 GiB] start iter
  [25.131 s, 7.25 GiB] iter 0 val nnz 0.042
  [28.888 s, 7.29 GiB] iter 1 val nnz 0.213
  [33.233 s, 7.39 GiB] iter 2 val nnz 0.499

   intial state nnz 2.71 top-5 accuracy: 0.412 (0.093 on novel)
   iter 0/3 nnz 21.15 top-5 accuracy: 0.587 (0.370 on novel)
   iter 1/3 nnz 108.02 top-5 accuracy: 0.672 (0.531 on novel)
   iter 2/3 nnz 252.89 top-5 accuracy: 0.678 (0.544 on novel)