Evaluate Instant NGP on a dataset created with BlenderNeRF

TestBlenderNeRF.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0ht4AB0Nh107",
    "jp-MarkdownHeadingCollapsed": true,
    "tags": []
   },
   "source": [
    "# Install Instant NGP"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OPBw1NJwYn1W"
   },
   "source": [
    "Check your assigned NVIDIA GPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "QJC-0GitoJEh"
   },
   "outputs": [],
   "source": [
    "!nvidia-smi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "M69lSJhLYsgi"
   },
   "source": [
    "Download and install packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "yKFsL8wSdQ8Q"
   },
   "outputs": [],
   "source": [
    "!apt update && apt install build-essential git python3-dev python3-pip libopenexr-dev libxi-dev libglfw3-dev libglew-dev libomp-dev libxinerama-dev libxcursor-dev\n",
    "!pip install --upgrade cmake"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fLgrKJHOZIKW"
   },
   "source": [
    "Clone the Instant NGP repository"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ijgdl-TUcxnd"
   },
   "outputs": [],
   "source": [
    "!git clone --recursive https://github.com/nvlabs/instant-ngp ngp\n",
    "%cd /content/ngp/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ENC6fi_tZgOb"
   },
   "source": [
    "Build the project with cmake (and disable GUI, as not supported in python notebooks) **~ 10 min**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "cpuhlQmJcDPJ"
   },
   "outputs": [],
   "source": [
    "!cmake . -B build -DNGP_BUILD_WITH_GUI=OFF\n",
    "!cmake --build build --config RelWithDebInfo -j `nproc`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3GVrYFfgZ_ol"
   },
   "source": [
    "Install package requirements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "TRfrshaCdpY8"
   },
   "outputs": [],
   "source": [
    "!pip3 install -r requirements.txt\n",
    "%cd /content/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7JTtHxkkdBj0"
   },
   "source": [
    "Replace code chunk incompatible with `transforms_test.json`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "YZbjDzQFcL0i"
   },
   "outputs": [],
   "source": [
    "ngpfile = '/content/ngp/scripts/run.py'\n",
    "\n",
    "textin = 'f.get(\"transform_matrix\", f[\"transform_matrix_start\"])'\n",
    "textout = 'f[\"transform_matrix\"]'\n",
    "\n",
    "readf = open(ngpfile)\n",
    "content = readf.read()\n",
    "content = content.replace(textin, textout)\n",
    "\n",
    "writef = open(ngpfile, 'w')\n",
    "writef.write(content)\n",
    "\n",
    "# incompatible with https://github.com/NVlabs/instant-ngp/tree/b9b7753ba5a0fa280c837a7c4f088f9c637ad575\n",
    "# currently master branch (15.08.2023)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Z8f0ywQSYMNH",
    "jp-MarkdownHeadingCollapsed": true,
    "tags": []
   },
   "source": [
    "# Upload and Prepare Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "078tmHBm3qDE"
   },
   "source": [
    "You can upload and access your **BlenderNeRF** dataset either via Google Drive (fast uploading) or directly by dragging your zip file to the *Files* panel on the left\n",
    "\n",
    "WARNING : the dataset is expected to contain a `train` folder with training images, alongside `transforms_train.json` and `transforms_test.json` files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "wBAOGXqCwYFb"
   },
   "outputs": [],
   "source": [
    "# @title Select your uploading type\n",
    "\n",
    "# @markdown Upload your zipped data and edit `<datasetPath.zip>` in either **gdrivePath** or **colabPath** to match the relative path to the dataset\n",
    "\n",
    "uploadType = 'google_drive' # @param ['google_drive', 'colab_drive'] {allow-input: false}\n",
    "\n",
    "gdrivePath = '/content/drive/MyDrive/\\u003CdatasetPath.zip>' # @param {type: 'string'}\n",
    "\n",
    "colabPath = '/content/<datasetPath.zip>' # @param {type: 'string'}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rEFw3ngHe8ig"
   },
   "source": [
    "Extract the archived dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "l7Avmsq5okwl"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "datasetPath = gdrivePath if uploadType == 'google_drive' else colabPath\n",
    "gdrive = False\n",
    "\n",
    "if uploadType == 'google_drive' and not gdrive:\n",
    "    from google.colab import drive\n",
    "    drive.mount('/content/drive/')\n",
    "    gdrive = True\n",
    "\n",
    "if not datasetPath.endswith('.zip'):\n",
    "    raise Warning('Please provide a valid dataset path ending in .zip')\n",
    "\n",
    "datasetDir = '/content/' + os.path.basename(datasetPath.split('.zip')[0]) # /content/<datasetName>/\n",
    "\n",
    "!unzip $datasetPath -d $datasetDir"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0t8_qI4mWlIX"
   },
   "source": [
    "Split data into `traindata` and `testdata` folders, and set the parameters as defined with **BlenderNeRF**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "R2ogM8hWU1_l"
   },
   "outputs": [],
   "source": [
    "import json\n",
    "%cd $datasetDir\n",
    "\n",
    "!mkdir -p traindata testdata\n",
    "!mv train traindata\n",
    "!mv transforms_train.json traindata\n",
    "!mv transforms_test.json testdata\n",
    "\n",
    "f = open('testdata/transforms_test.json')\n",
    "testdata = json.loads(f.read())\n",
    "testWidth, testHeight = int(testdata['w']), int(testdata['h'])\n",
    "testDir = os.path.dirname(testdata['frames'][0]['file_path'])\n",
    "\n",
    "trainPath = os.path.join(datasetDir, 'traindata')\n",
    "testPath = os.path.join(datasetDir, 'testdata', testDir)\n",
    "testJson = os.path.join(datasetDir, 'testdata/transforms_test.json')\n",
    "\n",
    "%cd /content/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "7a6YbsOJbr8R"
   },
   "outputs": [],
   "source": [
    "# @title Set parameters\n",
    "\n",
    "# @markdown Keep the width and height at 0 to use the resolution defined in **BlenderNeRF**\n",
    "\n",
    "# @markdown **spp** = samples per pixel (larger means less noise, but slower rendering)\n",
    "\n",
    "width = 0 # @param {type:\"integer\"}\n",
    "height = 0 # @param {type:\"integer\"}\n",
    "\n",
    "iterations = 2000 # @param {type:\"slider\", min:100, max:20000, step:100}\n",
    "\n",
    "spp = 16 # @param {type:\"slider\", min:1, max:64, step:1}\n",
    "\n",
    "# resolution\n",
    "width = width if width else testWidth\n",
    "height = height if height else testHeight\n",
    "\n",
    "# warnings\n",
    "if width < 0 or height < 0:\n",
    "    raise Warning('Width and height cannot be negative')\n",
    "if iterations <= 0:\n",
    "    raise Warning('The number of training iterations must be strictly positive')\n",
    "if spp <= 0 or spp > 64:\n",
    "    raise Warning('The number of samples per pixel must be > 0 and ≤ 64')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MgnWAW1biLZG",
    "jp-MarkdownHeadingCollapsed": true,
    "tags": []
   },
   "source": [
    "# Run NeRF"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5PuG2e2dfJk5"
   },
   "source": [
    "Training might take from seconds to a few minutes depending on your assigned GPU. Rendered test images are stored under `<datasetPath>/testdata/`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "5ayIkZzsmuDF"
   },
   "outputs": [],
   "source": [
    "%cd /content/ngp/\n",
    "\n",
    "!python scripts/run.py --scene $trainPath \\\n",
    "                       --save_snapshot ./snapshot.ingp \\\n",
    "                       --screenshot_transforms $testJson \\\n",
    "                       --screenshot_dir $testPath \\\n",
    "                       --screenshot_spp $spp \\\n",
    "                       --width $width \\\n",
    "                       --height $height \\\n",
    "                       --n_steps $iterations\n",
    "\n",
    "%cd /content/\n",
    "\n",
    "# load snapshot with the below flag\n",
    "# --load_snapshot ./snapshot.ingp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZiLs3TqNB_v0",
    "jp-MarkdownHeadingCollapsed": true,
    "tags": []
   },
   "source": [
    "# Save Output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nt5_6L2nCHmd"
   },
   "source": [
    "Encode the rendered test frames into a video, stored under `<datasetPath>/testdata/`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "A5rf8teNEzcw"
   },
   "outputs": [],
   "source": [
    "# @title Encode the rendered test frames into a video\n",
    "\n",
    "# @markdown **fps** = frames per second\n",
    "\n",
    "# @markdown **quality** = quality constant rate factor (lower is better quality)\n",
    "\n",
    "fps = \"24\" # @param [1, 2, 4, 8, 12, 24, 25, 30, 50, 60, 120, 240]\n",
    "fps = int(fps)\n",
    "\n",
    "quality = 20 # @param {type:\"slider\", min:1, max:50}\n",
    "\n",
    "output = 'video.mp4' # @param {type:\"string\"}\n",
    "\n",
    "# warning\n",
    "if quality < 0 or quality > 51:\n",
    "    raise Warning('The quality level must be >= 0 and ≤ 30')\n",
    "if fps <= 0:\n",
    "    raise Warning('The frame rate (fps) cannot be negative')\n",
    "if not output.endswith('.mp4'):\n",
    "    raise Warning ('The output filename must end in .mp4')\n",
    "    raise Warning('Please provide a valid dataset path ending in .zip')\n",
    "\n",
    "ext = os.listdir(testPath)[0].split('.')[-1]\n",
    "digits = len(os.listdir(testPath)[0].split('.')[0])\n",
    "input = os.path.join(testPath, f'%0{digits}d.{ext}')\n",
    "output = os.path.join(testPath, output)\n",
    "\n",
    "# encode video\n",
    "!ffmpeg -framerate $fps -i $input -crf $quality -pix_fmt yuv420p -y $output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "d_MZzNb32fKs"
   },
   "source": [
    "Compress the test data and copy the archived file to your gdrive. The file can otherwise be downloaded from the left *Files* window"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "MPYimXmx2gxy"
   },
   "outputs": [],
   "source": [
    "# @title Compress and save the test data to Google Drive\n",
    "\n",
    "# @markdown Set a valid **filename** for the test data. If **save** is ticked, the archived file will be stored to **savePath** (by default your gdrive home path)\n",
    "\n",
    "filename = 'output.zip' # @param {type: 'string'}\n",
    "\n",
    "savePath = '/content/drive/MyDrive/' # @param {type: 'string'}\n",
    "\n",
    "save = True # @param {type:\"boolean\"}\n",
    "\n",
    "# compress\n",
    "folder = filename.replace('.zip', '')\n",
    "!mkdir -p $folder\n",
    "!cp -r $testPath/* $folder\n",
    "!zip -r $filename $folder\n",
    "!rm -rf $folder\n",
    "\n",
    "# save to drive\n",
    "if save and not gdrive:\n",
    "    from google.colab import drive\n",
    "    drive.mount('/content/drive/')\n",
    "    gdrive = True\n",
    "\n",
    "if save and not os.path.isdir(savePath):\n",
    "    raise Warning('savePath must be a valid folder location on your gdrive')\n",
    "\n",
    "if save:\n",
    "    !cp $filename $savePath"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [
    "0ht4AB0Nh107",
    "Z8f0ywQSYMNH",
    "MgnWAW1biLZG",
    "ZiLs3TqNB_v0"
   ],
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}