Install flownet2-pytorch locally in a conda environment - Ubuntu 22.04

flownet2-pytorch installation.md

DETAILS

Since I ran into some difficulties installing flownet2-pytorch using conda, I am saving the steps I undertook for posterity.

NOTE:

• This aids in the installation of this fork, which contains a few tweaks for ease of installation. It has not been tested on the original repository. This fork is of the latest version of flownet2-pytorch, as of the time of publication of this gist. The fork does not change any of the source files.
• I will share the complete set of dependencies, as well as my system details in later files in this gist. You may try skipping step 2 below by directly installing this set of dependencies using conda env create -f env.yml, and it should work if you have a similar system.
• The steps assume that the conda-forge channel is a part of your conda config. If this is not the case, you can add it as below (because it should be anyway):

conda config --add channels conda-forge

---

STEPS

1. clone the repository

git clone https://github.com/VedanshM/flownet2-pytorch/tree/60119989782151c15157661ca8cf0a77e414368c

2. The smallest set of reproducible steps I could plan to successfully install the dependencies, are as below:
   • Create the environment, and activate it.

   conda create -n flownet2 -y
   conda activate flownet2

   • First, install the python dependencies mentioned in the official repository.

   conda install pytorch==0.4.1 -y
   conda install colorama numpy scikit-image setproctitle tensorboardx tqdm -y

   • Install cudatoolkit-dev=11.3 for ensured compatibility with flownet2. My local nvcc version was (is) 11.5, which was causing some conflicts. 11.3 seemed to work based on trial and error.

   conda install cudatoolkit-dev=11.3 -y

   • The gcc version cannot be more than 10, to ensure compatibility with CUDA.

   conda install 'gxx<10' -y

   • Install pytorch locally in accordance with our cuda version (11.3), as recommended in the official site. (I know that pytorch was installed before as well, but the dependencies don't seem to work if I immediately install 1.12.1. Thus, either directly install from the env file below, or follow these steps in this order).

   conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch -y

   • Finally, some gcc activation scripts are required to run the install scripts in flownet that are not installed above. Thus, run the following:

   conda install gxx_linux-64 -y

3. The installation scripts require crypt.h, which is not installed by conda along with python as mentioned here. Thus, copy the file from your local include, as:

cp /usr/include/crypt.h ~/miniconda3/envs/flownet2/include/python3.7m

if you're not using miniconda, use the script in the linked discussion instead. You may have to change your env name, and python version as required.

4. Now, attempt to run bash install.sh. If it complains about some missing modules, then proceed, otherwise you're done!

cd flownet2-pytorch
bash install.sh

5. If the above step failed, you may need to build the resample2d_package module by running the setup.py script in the corresponding directory. Navigate to the resample2d_package directory and run the setup scripts:

cd flownet2-pytorch/networks/resample2d_package
python setup.py build
python setup.py install

6. Finally, navigate back to the root of the clone, and run bash install.sh again.

You should have flownet2-pytorch installed succesfully! Good luck!

pytorch-flownet-envs.yml

name: flownet2
channels:
  - pytorch
  - conda-forge
  - defaults
dependencies:
  - _libgcc_mutex=0.1=conda_forge
  - _openmp_mutex=4.5=2_gnu
  - binutils_impl_linux-64=2.39=he00db2b_1
  - binutils_linux-64=2.39=h5fc0e48_13
  - blas=1.0=mkl
  - blosc=1.21.5=h0f2a231_0
  - bottleneck=1.3.5=py37hda87dfa_0
  - brotli=1.0.9=h166bdaf_9
  - brotli-bin=1.0.9=h166bdaf_9
  - brotli-python=1.0.9=py37hd23a5d3_7
  - brunsli=0.1=h9c3ff4c_0
  - bzip2=1.0.8=hd590300_5
  - c-ares=1.28.1=hd590300_0
  - c-blosc2=2.12.0=hb4ffafa_0
  - ca-certificates=2024.6.2=hbcca054_0
  - certifi=2024.6.2=pyhd8ed1ab_0
  - cffi=1.15.1=py37h43b0acd_1
  - cfitsio=4.0.0=h9a35b8e_0
  - charls=2.3.4=h9c3ff4c_0
  - charset-normalizer=3.3.2=pyhd8ed1ab_0
  - cloudpickle=2.2.1=pyhd8ed1ab_0
  - colorama=0.4.6=pyhd8ed1ab_0
  - cuda-version=11.3=hbc958af_3
  - cudatoolkit=11.3.1=hb98b00a_13
  - cudatoolkit-dev=11.3.1=py37h5e8e339_0
  - cudnn=8.9.7.29=hbc23b4c_3
  - cycler=0.11.0=pyhd8ed1ab_0
  - cytoolz=0.12.0=py37h540881e_0
  - dask-core=2022.2.0=pyhd8ed1ab_0
  - ffmpeg=4.3=hf484d3e_0
  - fonttools=4.38.0=py37h540881e_0
  - freetype=2.12.1=h267a509_2
  - fsspec=2023.1.0=pyhd8ed1ab_0
  - gcc=9.5.0=h1fea6ba_13
  - gcc_impl_linux-64=9.5.0=h99780fb_19
  - gcc_linux-64=9.5.0=h4258300_13
  - giflib=5.2.2=hd590300_0
  - gmp=6.3.0=h59595ed_1
  - gnutls=3.6.13=h85f3911_1
  - gxx=9.5.0=h1fea6ba_13
  - gxx_impl_linux-64=9.5.0=h99780fb_19
  - gxx_linux-64=9.5.0=h43f449f_13
  - idna=3.7=pyhd8ed1ab_0
  - imagecodecs=2021.11.20=py37h119f88a_2
  - imageio=2.34.1=pyh4b66e23_0
  - intel-openmp=2023.1.0=hdb19cb5_46306
  - jpeg=9e=h0b41bf4_3
  - jxrlib=1.1=hd590300_3
  - kernel-headers_linux-64=2.6.32=he073ed8_17
  - keyutils=1.6.1=h166bdaf_0
  - kiwisolver=1.4.4=py37h7cecad7_0
  - krb5=1.21.2=h659d440_0
  - lame=3.100=h166bdaf_1003
  - lcms2=2.14=h6ed2654_0
  - ld_impl_linux-64=2.39=hcc3a1bd_1
  - lerc=3.0=h9c3ff4c_0
  - libaec=1.1.3=h59595ed_0
  - libblas=3.9.0=1_h86c2bf4_netlib
  - libbrotlicommon=1.0.9=h166bdaf_9
  - libbrotlidec=1.0.9=h166bdaf_9
  - libbrotlienc=1.0.9=h166bdaf_9
  - libcblas=3.9.0=5_h92ddd45_netlib
  - libcurl=8.8.0=hca28451_0
  - libdeflate=1.10=h7f98852_0
  - libedit=3.1.20191231=he28a2e2_2
  - libev=4.33=hd590300_2
  - libffi=3.4.2=h7f98852_5
  - libgcc-devel_linux-64=9.5.0=h0a57e50_19
  - libgcc-ng=13.2.0=h77fa898_10
  - libgfortran-ng=13.2.0=h69a702a_10
  - libgfortran5=13.2.0=h3d2ce59_10
  - libgomp=13.2.0=h77fa898_10
  - libiconv=1.17=hd590300_2
  - liblapack=3.9.0=5_h92ddd45_netlib
  - libnghttp2=1.58.0=h47da74e_1
  - libnsl=2.0.1=hd590300_0
  - libpng=1.6.43=h2797004_0
  - libprotobuf=3.20.1=h6239696_4
  - libsanitizer=9.5.0=h2f262e1_19
  - libsqlite=3.46.0=hde9e2c9_0
  - libssh2=1.11.0=h0841786_0
  - libstdcxx-devel_linux-64=9.5.0=h0a57e50_19
  - libstdcxx-ng=13.2.0=hc0a3c3a_10
  - libtiff=4.4.0=h0fcbabc_0
  - libwebp-base=1.4.0=hd590300_0
  - libxcb=1.13=h7f98852_1004
  - libzlib=1.2.13=h4ab18f5_6
  - libzopfli=1.0.3=h9c3ff4c_0
  - locket=1.0.0=pyhd8ed1ab_0
  - lz4-c=1.9.3=h9c3ff4c_1
  - matplotlib-base=3.5.3=py37hf395dca_2
  - mkl=2018.0.3=1
  - munkres=1.1.4=pyh9f0ad1d_0
  - nccl=2.21.5.1=h0800d71_0
  - ncurses=6.5=h59595ed_0
  - nettle=3.6=he412f7d_0
  - networkx=2.6.3=pyhd8ed1ab_1
  - ninja=1.12.1=h297d8ca_0
  - numexpr=2.7.3=py37he8f5f7f_1
  - numpy=1.21.6=py37h976b520_0
  - openh264=2.1.1=h780b84a_0
  - openjpeg=2.5.0=h7d73246_1
  - openssl=3.3.1=h4ab18f5_0
  - packaging=23.2=pyhd8ed1ab_0
  - pandas=1.3.5=py37h8c16a72_0
  - partd=1.4.1=pyhd8ed1ab_0
  - pillow=9.2.0=py37h850a105_2
  - pip=24.0=pyhd8ed1ab_0
  - protobuf=3.20.1=py37hd23a5d3_0
  - pthread-stubs=0.4=h36c2ea0_1001
  - pycparser=2.21=pyhd8ed1ab_0
  - pyparsing=3.1.2=pyhd8ed1ab_0
  - pysocks=1.7.1=py37h89c1867_5
  - python=3.7.12=hf930737_100_cpython
  - python-dateutil=2.9.0=pyhd8ed1ab_0
  - python_abi=3.7=4_cp37m
  - pytorch=1.12.1=py3.7_cuda11.3_cudnn8.3.2_0
  - pytorch-mutex=1.0=cuda
  - pytz=2024.1=pyhd8ed1ab_0
  - pywavelets=1.3.0=py37hda87dfa_1
  - pyyaml=6.0=py37h540881e_4
  - readline=8.2=h8228510_1
  - requests=2.32.2=pyhd8ed1ab_0
  - scikit-image=0.19.3=py37hfb7772e_1
  - scipy=1.7.3=py37hf2a6cf1_0
  - setproctitle=1.3.2=py37h540881e_0
  - setuptools=69.0.3=pyhd8ed1ab_0
  - six=1.16.0=pyh6c4a22f_0
  - snappy=1.1.10=hdb0a2a9_1
  - sqlite=3.46.0=h6d4b2fc_0
  - sysroot_linux-64=2.12=he073ed8_17
  - tensorboardx=2.5.1=pyhd8ed1ab_0
  - tifffile=2021.11.2=pyhd8ed1ab_0
  - tk=8.6.13=noxft_h4845f30_101
  - toolz=0.12.1=pyhd8ed1ab_0
  - torchaudio=0.12.1=py37_cu113
  - torchvision=0.13.1=py37_cu113
  - tqdm=4.66.4=pyhd8ed1ab_0
  - typing-extensions=4.7.1=hd8ed1ab_0
  - typing_extensions=4.7.1=pyha770c72_0
  - unicodedata2=14.0.0=py37h540881e_1
  - urllib3=2.2.1=pyhd8ed1ab_0
  - wheel=0.42.0=pyhd8ed1ab_0
  - xorg-libxau=1.0.11=hd590300_0
  - xorg-libxdmcp=1.1.3=h7f98852_0
  - xz=5.2.6=h166bdaf_0
  - yaml=0.2.5=h7f98852_2
  - zfp=0.5.5=h9c3ff4c_8
  - zlib=1.2.13=h4ab18f5_6
  - zlib-ng=2.0.7=h0b41bf4_0
  - zstd=1.5.6=ha6fb4c9_0
prefix: /home/varun/miniconda3/envs/flownet2

system-details.md

• Operating System: Ubuntu 22.04
• Conda version: 24.4.0
• Graphics Information

(base) varun@varun-Inspiron-16-7620-2-in-1:~$ nvidia-smi
Mon Jun 17 06:24:06 2024       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.171.04             Driver Version: 535.171.04   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce MX550           Off | 00000000:02:00.0 Off |                  N/A |
| N/A   66C    P0               9W /  30W |    552MiB /  2048MiB |     13%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A      2237      G   /usr/lib/xorg/Xorg                          199MiB |
|    0   N/A  N/A      2919      G   /usr/bin/picom                                1MiB |
|    0   N/A  N/A      3310      G   /usr/lib/insync/insync                        1MiB |
|    0   N/A  N/A      4261      G   ...irefox/4424/usr/lib/firefox/firefox      263MiB |
|    0   N/A  N/A     35662      G   ...erProcess --variations-seed-version       47MiB |
+---------------------------------------------------------------------------------------+

(local nvcc, not in env):

(base) varun@varun-Inspiron-16-7620-2-in-1:~$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Thu_Nov_18_09:45:30_PST_2021
Cuda compilation tools, release 11.5, V11.5.119
Build cuda_11.5.r11.5/compiler.30672275_0