embedding_images.md

Before you start

Make sure you have python, OpenFace and dlib installed. You can either install them manually or use a preconfigured docker image that has everying already installed:

docker pull bamos/openface
docker run -p 9000:9000 -p 8000:8000 -t -i bamos/openface /bin/bash
cd /root/openface

Step 1

Make a folder called /training-images/ somewhere on your computer.

Step 2

Make a subfolder for each person you want to recognize. For example:

• /training-images/will-ferrell/
• /training-images/chad-smith/
• /training-images/jimmy-fallon/

Step 3

Copy all your images of each person into the correct sub-folders

Step 4

Run the openface scripts from inside the openface root directory:

First, do pose detection and alignment:

./util/align-dlib.py ./training-images/ align outerEyesAndNose ./aligned-images/ --size 96

Second, generate the representations from the aligned images:

./batch-represent/main.lua -outDir ./generated-embeddings/ -data ./aligned-images/

When you are done, the ./generated-embeddings/ folder will contain a csv file with the embeddings for each image.

Can you post the complete commands you run and the full output that you get?

How perfect do the aligned-images have to be?

I installed everything manually but when I run ./batch-represent/main.lua -outDir ./generated-embeddings/ -data ./aligned-images/ I get the following output:

{
  data : "./aligned-images/"
  imgDim : 96
  model : "/Users/benton/openface/models/openface/nn4.small2.v1.t7"
  device : 1
  outDir : "./generated-embeddings/"
  cache : false
  cuda : false
  batchSize : 50
}
./aligned-images/
cache lotation:         /Users/benton/openface/aligned-images/cache.t7
Creating metadata for cache.
{
  sampleSize : 
    {
      1 : 3
      2 : 96
      3 : 96
    }
  split : 0
  verbose : true
  paths : 
    {
      1 : "./aligned-images/"
    }
  samplingMode : "balanced"
  loadSize : 
    {
      1 : 3
      2 : 96
      3 : 96
    }
}
running "find" on each class directory, and concatenate all those filenames into a single file containing all image paths for a given class
/tmp/lua_oUtbbz: line 1: gfind: command not found
/tmp/lua_oUtbbz: line 2: gfind: command not found
/tmp/lua_oUtbbz: line 3: gfind: command not found
/tmp/lua_oUtbbz: line 4: gfind: command not found
/tmp/lua_oUtbbz: line 5: gfind: command not found
now combine all the files to a single large file
load the large concatenated list of sample paths to self.imagePath
sh: gwc: command not found
/Users/benton/torch/install/bin/luajit: /Users/benton/openface/batch-represent/dataset.lua:189: attempt to perform arithmetic on a nil value
stack traceback:
        /Users/benton/openface/batch-represent/dataset.lua:189: in function '__init'
        /Users/benton/torch/install/share/lua/5.1/torch/init.lua:91: in function </Users/benton/torch/install/share/lua/5.1/torch/init.lua:87>
        [C]: in function 'dataLoader'
        /Users/benton/openface/batch-represent/batch-represent.lua:19: in function 'batchRepresent'
        ./batch-represent/main.lua:42: in main chunk
        [C]: in function 'dofile'
        ...nton/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
        [C]: at 0x0101ff11c0

P.S. I was also prompted to install the following luarocks modules csvigo and dpnn, so not sure if you wanted to add that to the docs.

Actually figured it out. Apparently you need to install findutils and coreutils using homebrew.

I did ./util/align-dlib.py ./training-images/ align outerEyesAndNose ./aligned-images/ --size 96 but my training images are not being saved in the aligned images folder. It does not show any errors, but just does not work. ( I presume the dots in the path are to be replaced by the actual path.)

RudradeepGuha I had the same problem, In case the problem was that image type file, openface just use "jpg" and "png". Be careful.

@RudradeepGuha, notice that when you run "docker run -p 9000:9000 -p 8000:8000 -t -i bamos/openface /bin/bash" the host name get changed. So, download images in current directory(/root/openface/). Then run the script. It worked for me.
Use docker cp ...

What's the problem here?!
That's the output I've got :

{
  device : 1
  cache : false
  model : "/home/soma/openface/models/openface/nn4.small2.v1.t7"
  data : "/home/soma/PycharmProjects/Face_Rec/aligned-images/"
  batchSize : 50
  cuda : false
  imgDim : 96
  outDir : "/home/soma/PycharmProjects/Face_Rec/generated-embeddings/"
}
/home/soma/PycharmProjects/Face_Rec/aligned-images/     
cache lotation:         /home/soma/PycharmProjects/Face_Rec/aligned-images/cache.t7     
Creating metadata for cache.    
{
  loadSize : 
    {
      1 : 3
      2 : 96
      3 : 96
    }
  split : 0
  verbose : true
  sampleSize : 
    {
      1 : 3
      2 : 96
      3 : 96
    }
  paths : 
    {
      1 : "/home/soma/PycharmProjects/Face_Rec/aligned-images/"
    }
  samplingMode : "balanced"
}
running "find" on each class directory, and concatenate all those filenames into a single file containing all image paths for a given class     
now combine all the files to a single large file        
load the large concatenated list of sample paths to self.imagePath      
4962 samples found........................ 0/4962 ......................................]  ETA: 0ms | Step: 0ms         
Updating classList and imageClass appropriately 
 [======================================== 4/4 ========================================>]  Tot: 18ms | Step: 4ms        
Cleaning up temporary files     
Splitting training and test sets to a ratio of 0/100    
/home/soma/torch/install/bin/lua: cannot open </home/soma/PycharmProjects/Face_Rec/aligned-images/cache.t7> in mode  w at /home/soma/torch/pkg/torch/lib/TH/THDiskFile.c:670
stack traceback:
        [C]: in ?
        [C]: in function 'DiskFile'
        /home/soma/torch/install/share/lua/5.2/torch/File.lua:385: in function 'save'
        /home/soma/openface/batch-represent/batch-represent.lua:26: in function 'batchRepresent'
        ./batch-represent/main.lua:42: in main chunk
        [C]: in function 'dofile'
        ...soma/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
        [C]: in ?

thilakraj@thilakraj-Aspire-V5-471PG:/openface$ ./util/align-dlib.py ./training-images/ align outerEyesAndNose ./aligned-images/ --size 96
thilakraj@thilakraj-Aspire-V5-471PG:/openface$ ./batch-represent/main.lua -outDir ./generated-embeddings/ -data ./aligned-images/
{
data : "./aligned-images/"
imgDim : 96
model : "/home/thilakraj/openface/models/openface/nn4.small2.v1.t7"
device : 1
outDir : "./generated-embeddings/"
cache : false
cuda : false
batchSize : 50
}
./aligned-images/
cache lotation: /home/thilakraj/openface/aligned-images/cache.t7
Creating metadata for cache.
{
sampleSize :
{
1 : 3
2 : 96
3 : 96
}
split : 0
verbose : true
paths :
{
1 : "./aligned-images/"
}
samplingMode : "balanced"
loadSize :
{
1 : 3
2 : 96
3 : 96
}
}
running "find" on each class directory, and concatenate all those filenames into a single file containing all image paths for a given class
now combine all the files to a single large file
load the large concatenated list of sample paths to self.imagePath
/home/thilakraj/torch/install/bin/luajit: /home/thilakraj/openface/batch-represent/dataset.lua:193: Could not find any image file in the given input paths
stack traceback:
[C]: in function 'assert'
/home/thilakraj/openface/batch-represent/dataset.lua:193: in function '__init'
/home/thilakraj/torch/install/share/lua/5.1/torch/init.lua:91: in function </home/thilakraj/torch/install/share/lua/5.1/torch/init.lua:87>
[C]: in function 'dataLoader'
...e/thilakraj/openface/batch-represent/batch-represent.lua:19: in function 'batchRepresent'
./batch-represent/main.lua:42: in main chunk
[C]: in function 'dofile'
...kraj/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x00406670
thilakraj@thilakraj-Aspire-V5-471PG:~/openface$

Can someone tell me Why I got like this?

@arceda thanks for mentioning about .jpg and .png

vinay@pe:~/torch/openface$ ./util/align-dlib.py ./training-images/ align outerEyesAndNose ./aligned-images/ --size 96

------> after this step i can see aligned images in aligned-images folder
------>when i run the next step i am getting this error

vinay@pe:~/torch/openface$ ./batch-represent/main.lua -outDir ./generated-embeddings/ -data ./aligned-images/
{
data : "./aligned-images/"
imgDim : 96
model : "/home/vinay/torch/openface/models/openface/nn4.small2.v1.t7"
device : 1
outDir : "./generated-embeddings/"
cache : false
cuda : false
batchSize : 50
}
/home/vinay/torch/install/bin/luajit: cannot open </home/vinay/torch/openface/models/openface/nn4.small2.v1.t7> in mode r at /home/vinay/torch/pkg/torch/lib/TH/THDiskFile.c:670
stack traceback:
[C]: at 0x7f6fbabd6090
[C]: in function 'DiskFile'
/home/vinay/torch/install/share/lua/5.1/torch/File.lua:405: in function 'load'
./batch-represent/main.lua:33: in main chunk
[C]: in function 'dofile'
...inay/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x00405d50

could someone please help me out !!
I installed everything manually .

guys i am getting this error help me out

home/pi/torch/install/bin/luajit: cannot open </home/pi/openface/models/openface/nn4.small2.v1.t7> in mode r at /home/pi/torch/pkg/torch/lib/TH/THDiskFile.c:670
stack traceback:
[C]: at 0x76a23230
[C]: in function 'DiskFile'
/home/pi/torch/install/share/lua/5.1/torch/File.lua:405: in function 'load'
./batch-represent/main.lua:33: in main chunk
[C]: in function 'dofile'
...e/pi/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x00014fa8

File "main.lua", line 3
require 'torch'
^
SyntaxError: invalid syntax
I am getting this error after running the last line I have installed torch but still it is giving the same error

@alimehdi242 Install pytorch and then try the same code.After installing pytorch you may also require dpnn and csvigo...which you can install bt "luarocks install dpnn"

When I run the code

./util/align-dlib.py /home/abhishek/ml/Image_processing/Faces/training-images align outerEyesAndNose /home/abhishek/ml/Image_processing/Faces/aligned-images/ --size 96

I do not get any error but couldn't see any such aligned-images directory
and when in the next line I run
./batch-represent/main.lua -outDir /home/abhishek/ml/Image_processing/Faces/generated-embeddings/ -data /home/abhishek/ml/Image_processing/Faces/aligned-images/

I get the following error in the output :

running "find" on each class directory, and concatenate all those filenames into a single file containing all image paths for a given class
now combine all the files to a single large file
load the large concatenated list of sample paths to self.imagePath
/root/torch/install/bin/luajit: /root/openface/batch-represent/dataset.lua:193: Could not find any image file in the given input paths
stack traceback:
[C]: in function 'assert'
/root/openface/batch-represent/dataset.lua:193: in function '__init'
/root/torch/install/share/lua/5.1/torch/init.lua:91: in function </root/torch/install/share/lua/5.1/torch/init.lua:87>
[C]: in function 'dataLoader'
/root/openface/batch-represent/batch-represent.lua:19: in function 'batchRepresent'
./batch-represent/main.lua:42: in main chunk
[C]: in function 'dofile'
/root/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
[C]: at 0x00406670

What could be the error and why aligned-image directory is not formed ? I have all the images in the provided ./training-images/ directory

Finally I solved my issue. I am writing for those who might face the same problem.
What's the actual problem is that when we install the openface , it gets installed in the root directory and the line
/util/align-dlib.py ./training-images/ align outerEyesAndNose ./aligned-images/ --size 96
doesn't give any error but doesn't form any /aligned-images in the given path.
I copied the openface in my user directory and just altered few lines of the "/util/align-dlib.py"

in the line
29 modelDir = os.path.join(fileDir, '..', 'models')
changed it to
29 modelDir = os.path.join(fileDir, 'models')
after this is done, copy the "training-images" folder into this openface folder
Now from the dlib directory copy the "shape_predictor_68_face_landmarks.dat" into the folder "openface"
Now run
/util/align-dlib.py ./training-images/ align outerEyesAndNose ./aligned-images/ --size 96
and now you will find a folder /aligned-images in openface folder
Now if you run the 2nd line
./batch-represent/main.lua -outDir ./generated-embeddings/ -data ./aligned-images/

you will get an error that you don't have torch installed in your system , so first install torch
to install torch type->
git clone https://github.com/torch/distro.git ~/torch --recursive
cd ~/torch; bash install-deps;
./install.sh

after torch is installed, if you run
./batch-represent/main.lua -outDir ./generated-embeddings/ -data ./aligned-images/
again you will encounter the error like

no field package.preload['csvigo']
no file '/root/.luarocks/share/lua/5.1/csvigo.lua'
no file '/root/.luarocks/share/lua/5.1/csvigo/init.lua'
no file '/root/torch/install/share/lua/5.1/csvigo.lua'
no file '/root/torch/install/share/lua/5.1/csvigo/init.lua'
no file './csvigo.lua'
no file '/root/torch/install/share/luajit-2.1.0-beta1/csvigo.lua'
no file '/usr/local/share/lua/5.1/csvigo.lua'
no file '/usr/local/share/lua/5.1/csvigo/init.lua'
no file '/root/.luarocks/lib/lua/5.1/csvigo.so'
no file '/root/torch/install/lib/lua/5.1/csvigo.so'
no file '/root/torch/install/lib/csvigo.so'
no file './csvigo.so'
no file '/usr/local/lib/lua/5.1/csvigo.so'
no file '/usr/local/lib/lua/5.1/loadall.so'

to remove this type
luarocks install cutorch
luarocks install csvigo
luarocks install dnn
and you are done

Hello how can I run the script on windows?

How do I access the root directory? and what does the './' in "./util/align-dlib.py" mean?

how to proceed with docker on mac?
In particular how to do step 3?

Running "sudo apt install torch-trepl "helped me resolved my issue with " /usr/bin/env: ‘th’: No such file or directory ".
I also installed:
luarocks install csvigo
luarocks install dnn

I tried to run this code on windows environment .I have set up everything dlib and pytorch using anaconda torch,csvigo,dnn using luarorks
and the first part of aligning the images was a success, I used "python util/align-dlib.py training-images/ align outerEyesAndNose aligned-images/ --size 96" command in a conda environment and got the aligned images in "aligned-images" folder but the problem is I was unable to get the embeddings in the "generated-embeddings" folder i used the command "lua batch-represent/main.lua -outDir generated-embeddings/ -data aligned-images/" at first i got lots of errors like requred csvigo,dnn,nn etc.but after installing all those packages the command executed successfully but the folder "generated-embeddings" was not created ,I am not even getting any errors.can some one help me how to run this code successfully on a windows environment.

-Thanks

Im sorry but after I run this it says
/root/torch/install/bin/luajit: /root/openface/batch-represent/dataset.lua:193: could not find any image file in the given input paths

I am wondering why this happens, and also, if I have the embeddings, how can I use this to test the videos or other images on the Internet?

May be you followed the exact steps( specially step2) The path given ( /training-images/will-ferrell/) is relative to root ("/") but in steps further , author assumed in current directory.( i.e. ./training-images/will-ferrell/, note the dot ahead).

C:\Program Files (x86)\Lua\5.1\lua.exe: C:\Program Files (x86)\Lua\5.1\lua\paths\init.lua:1: module 'libpaths' not found:
no field package.preload['libpaths']
no file '.\libpaths.lua'
no file 'C:\Program Files (x86)\Lua\5.1\lua\libpaths.lua'
no file 'C:\Program Files (x86)\Lua\5.1\lua\libpaths\init.lua'
no file 'C:\Program Files (x86)\Lua\5.1\libpaths.lua'
no file 'C:\Program Files (x86)\Lua\5.1\libpaths\init.lua'
no file 'C:\Program Files (x86)\Lua\5.1\lua\libpaths.luac'
no file '.\libpaths.dll'
no file '.\libpaths51.dll'
no file 'C:\Program Files (x86)\Lua\5.1\libpaths.dll'
no file 'C:\Program Files (x86)\Lua\5.1\libpaths51.dll'
no file 'C:\Program Files (x86)\Lua\5.1\clibs\libpaths.dll'
no file 'C:\Program Files (x86)\Lua\5.1\clibs\libpaths51.dll'
no file 'C:\Program Files (x86)\Lua\5.1\loadall.dll'
no file 'C:\Program Files (x86)\Lua\5.1\clibs\loadall.dll'
stack traceback:
[C]: in function 'require'
C:\Program Files (x86)\Lua\5.1\lua\paths\init.lua:1: in main chunk
[C]: in function 'require'
C:\Program Files (x86)\Lua\5.1\lua\torch\init.lua:12: in main chunk
[C]: in function 'require'
...p\TCC\face_estudos\openface\batch-represent\main.lua:3: in main chunk
[C]: ?
what should i do on windows?

Is there any model to get face embedding features?

When executing lua batch-represent/main.lua -outDir generated-embeddings -data aligned-images I got :
table: 0x56399ed5d410
aligned-images
cache lotation: /mnt/c/Users/aleksandra/Desktop/Diplomski/openface/aligned-images/cache.t7
Creating metadata for cache.
table: 0x56399ee143d0
running "find" on each class directory, and concatenate all those filenames into a single file containing all image paths for a given class
now combine all the files to a single large file
load the large concatenated list of sample paths to self.imagePath
Segmentation fault
Did someone have this problem? Please, help