SharkWipf/Nerfstudio-WSL2.md

## Nerfstudio-WSL2.md

      
    Raw
  

              Nerfstudio-WSL2.md
            
          
    Nerfstudio (with COLMAP* + HLOC* + FFMPEG*) on Windows via WSL2

* Via Pixi
Old instructions stopped working on Gsplat 1.0.0+. Updated and simplified on 2024-07-09. Now no longer deviates from official installation instructions.

Tested working on Windows 11 Enterprise on 2024-05-31 (should work on all mainstream up-to-date Windows 10/11 versions).

Warning: Deviates from official installation instructions (Both Windows and Linux) in several areas. YMMV.
For native Linux start-to-finish installation instructions, see this document instead.
For context: WSL2 is a Windows built-in way to run Linux applications on Windows, in a tightly integrated virtual machine.

This, in theory, lets you run Linux applications without the complexity and trouble of actually having to run Linux.

In practice, there are still a few caveats, most notably the complex custom CUDA implementation Windows uses to make this work.

So why would we want to do this? Easy. Because the default Windows installation process of Nerfstudio is awful, due to all the unavoidable tools and dependencies that are required (like Visual Studio).

WSL2 allows us to (mostly) follow the much better tested and mature Linux instructions and codebase.

These instructions should also work for other CUDA-based projects.

Note: WSL2 reserves, by default 50%, of your memory when running. This amount should automatically expand and shrink as needed, however as this is a new feature, it might not always work correctly. Worst case a reboot will fix it.

First stage: Install WSL2.


You need to be running at least a fully updated Windows 10. WSL1 will not work.
Fully update both your Nvidia drivers and your Windows installation first.
If you already had a WSL2 environment you don't use, you can wipe it using wsl.exe --unregister ubuntu for a clean start. If you get stuck with a "dirty" installation, consider doing this.
In the highly unlikely event you are using a VFIO/GPU passthrough setup (you know if you are, if not, don't worry about it), enabling WSL2 can cause performance issues or even prevent booting your VM.
Anyone else needn't worry.
You can copy by highlighting text and right clicking. You can paste by pressing shift+insert or right clicking with nothing selected.

1. Open a terminal (cmd.exe, powershell, whichever you want, doesn't matter).

2. If you don't have WSL2 yet, install it (This will be followed by a reboot):

wsl.exe --install
At the time of writing, this will install the default Ubuntu 22.04.3 LTS. Complete the initial configuration steps if prompted. If it immediately drops you into the wsl2 installation afterwards (signified by the green prompt), you can enter exit to get out if you need to for the next step.
3. Make sure it's updated to the latest version:

wsl.exe --update
4. Start your new WSL2 environment:

wsl.exe
You should be greeted with a nice green prompt now.
Second stage: Install CUDA/-toolkit


Normally on Linux, when managing installations via Conda, you can switch between cuda-toolkit versions on a per-application basis.
On WSL2, because it has a lot of custom tweaks to make CUDA work on a GPU shared with Windows, I haven't found a way to make this work yet.
Whatever CUDA-toolkit version we install in this step is what we're stuck with for all applications in this WSL2 environment.
While Nerfstudio and its dependencies (particularly tinycudann) currently support up to cuda 12.1, we will be sticking to cuda 11.8 due to wider support and compatibility with conda-forge's prebuilt pytorch (which we need to install colmap).
Reference: https://docs.nvidia.com/cuda/wsl-user-guide/index.html#cuda-support-for-wsl-2

1. First we follow the instructions on this page.

You can use any of the 3 listed installation methods, but we will be using the deb (network) method.
Important: The last step is different! Do NOT run sudo apt-get -y install cuda!

wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-keyring_1.0-1_all.deb

sudo dpkg -i cuda-keyring_1.0-1_all.deb

sudo apt-get update

# Update Ubuntu while we're at it.
sudo apt-get -y upgrade

# Important! We deviate here! Do not install the `cuda` package, regardless of what installation method you choose!
sudo apt-get -y install cuda-toolkit-11-8 build-essential
If, by accident, you did install the cuda package, purge all cuda-related packages via sudo apt-get purge cuda\*; sudo apt-get autoremove and restart from this step.
2. Once (finally) done, we'll have to add some WSL2-specific environment variables to our system to make everything play nice with it. Open ~/.bashrc in a text editor of your choosing:

nano ~/.bashrc

Scroll to the very bottom, and on a new line, paste (shift+insert or right click), in order, the following lines:

export CUDA_PATH="/usr/local/cuda-11.8"
export CUDA_HOME="$CUDA_PATH"
export PATH="$CUDA_PATH/bin:$PATH"
export LD_LIBRARY_PATH="/usr/local/cuda/lib64/stubs:$CUDA_PATH/lib64:/usr/lib/wsl/lib/:$LD_LIBRARY_PATH"
export LIBRARY_PATH="$LD_LIBRARY_PATH:$LIBRARY_PATH"


Save the file with ctrl+x y (if you used nano).


These changes won't apply until we restart the shell. Exit out of your WSL2 environment by typing exit, then restart WSL2 again by simply typing wsl.exe again. This step is critical.


Confirm our exports from before are working before proceeding to the next step:


echo -e "CUDA_PATH: $CUDA_PATH \n\nCUDA_HOME: $CUDA_HOME \n\nPATH: $PATH \n\nLD_LIBRARY_PATH: $LD_LIBRARY_PATH \n\nLIBRARY_PATH: $LIBRARY_PATH\n"
Every line should be filled with one or more paths.
Third stage (new version): Installing Nerfstudio


(For the old version (won't work anymore), see the previous revisions of this gist)
At this point, your WSL2 environment should be almost completely the same as a "real" Ubuntu Linux installation, and, some CUDA quirks aside, you should be able to treat it as an Ubuntu installation as far as guides are concerned.


Important: Do not git clone Nerfstudio to a C-drive folder, you will get errors. Pick a folder outside of /mnt/c/.

For instance: mkdir ~/src; cd ~/src; git clone https://github.com/nerfstudio-project/nerfstudio.git to fetch Nerfstudio to the ~/src/nerfstudio folder.
From there on out you need to cd ~/src/nerfstudio before running pixi shell (if using pixi).


Since we're essentially regular Ubuntu now, you can simply follow the official installation instructions from here on out.
I would recommend following the Pixi instructions as they, generally, are most stable and straightforward, and include Colmap, HLOC and ffmpeg.

Note: As of 2024-07-27, some people report Pixi (in general) is not working in WSL2 anymore. I haven't had time to look into this yet. Conda/Mamba still works.
If you go the conda (or (micro)mamba) route, you can pull in colmap and optionally ffmpeg via conda-forge, though I couldn't get hloc working that way.

Some general (not-WSL2-specific) tips:


At the time of writing, there are some issues with numpy>=2 (warnings and errors mentioning numpy 2.0) and setuptools>=70 ("could not import packaging" error). You can workaround these by specifying 'numpy<2' 'setuptools<70' at every pip (or conda) command, for example: pip install -e . 'numpy<2' 'setuptools<70'.
If your projects want to up- or downgrade pytorch, try tacking a $(pip freeze | grep -e ^torch) onto your pip command, i.e. pip install -e . $(pip freeze | grep -e ^torch). This will freeze all pytorch packages so your installed torch versions don't get replaced. Particularly useful when you installed a specific Torch version via conda (might not work for torch via pip).
You can access your WSL2-internal folders from Windows Explorer by entering \\wsl$\ in your Explorer task bar. You can also add it as a permanent network mount.

Congratulations, you're done. You now (hopefully) have a fully functional nerfstudio environment in WSL2, complete with COLMAP and even HLOC.
Any time you want to use it, you simply pop open a terminal, punch in wsl.exe, go into the folder you installed to (if any), then pixi shell (or conda activate nerfstudio), and you're good to go.
You can now continue with the regular Nerfstudio instructions (or if you want to hit the ground running: ns-process-data video --data some/video.mp4 --output-dir video_colmapped, then ns-train nerfacto --data video_colmapped --output-dir trained. Tip: Each sub-method has its own --help output and flags are positional).