Skip to content

Instantly share code, notes, and snippets.

@xtanion

xtanion/Final-Report-GSoC-2022-Shivam.md

Last active October 10, 2022 15:20
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save xtanion/00272777d9f067a937aad6c4d7b427e8 to your computer and use it in GitHub Desktop.
Save xtanion/00272777d9f067a937aad6c4d7b427e8 to your computer and use it in GitHub Desktop.
Download ZIP
Final-Report-GSoC-2022-Shivam.md
Raw
Final-Report-GSoC-2022-Shivam.md
gsoc fury

Google Summer of Code Final Work Product

Proposed Objectives

  • Ability to load glTF models
    • Should be able to load static glTF models
    • Should be able to add model to the scene.
  • Exporting scene data as glTF file
  • Materials & Textures
    • Textures
    • PBRs
  • Animations
    • Simple Actor animations
    • Skinning
    • Morphing
  • Stretch Goals
    • Ability to load .glb files

Objectives Completed

  • Loading Static glTF models

    A glTF file is a JSON like file format containing required data for 3D scenes. VTK has two built-in glTF loaders. However, they lack ability to animate and apply materials. Added methods to load binary data and create actors from them. These actors can be added directly to the scene. The glTF class reads texture data from either base64 encoded string or from the image file, and maps the texture to the actor using the given UV data. It is capable of doing the following:

    • Loading both gltf and glb files. Get actors from the model.
    • Applying textures and colors from materials.
    • Setting cameras if the model contains multiple cameras.
    • Apply normals (for a smoother surface).

    image

    Pull Requests:

  • Exporting Scene as a glTF

    The fury scene can contain multiple objects such as actors, cameras, textures, etc. We need to get the primitive information (such as Vertices, Triangles, UVs, Normals, etc.) from these objects and store them into a .bin file. Added methods that export these information to a .gltf or .glb file format.

    Pull Requests:

  • Simple Actor Animations

    Added simple actor animations (translation, rotation & scale of actors) support. The animation data (transformation and timestamp) is stored in buffers. It converts the binary data to ndarrays and creates a timleline for each animation in glTF animations. This timeline containsactors an can be added to the scene. We can animate the scene by updating timline inside a timer callback.

    interpolation_test_working_right.mp4

    Pull Requests:

  • Morphing in glTF

    glTF allows us to animate meshes using morph targets. A morph target stores displacements or differences for certain mesh attributes. At runtime, these differences may be added to the original mesh, with different weights, to animate parts of the mesh. Added methods to extract this information, update the timeline and apply morphing to each actor in the scene.

    2022-10-05.00-12-14.mp4

    Pull Requests:

  • Skeletal Animations (Skining)

    Another way of animating a glTF is by skinning. It allows the geometry (vertices) of a mesh to be deformed based on the pose of a skeleton. This is essential in order to give animated geometry. It combines every parameter of a glTF file. While working with skinning, we need to keep track of the parent-child hierarchy of transformations. Vertex Skinning takes full advantage of newly implemented Timeline & Animation modules to track hierarchical transformation order. Though the current version of the skinning implementation works with most of the glTF sample modes, It struggles with models that have multiple actors (e.g. BrainStem). It can be fixed by using the vertex shader to update the vertices. The current implementation of skinning supports the following:

    • Multiple animation support
    • Multiple node and multiple actor animation with textures
    • Show or hide bones/skeleton of the model.
    2022-09-15.03-26-27.mp4

    Pull Requests:

Objectives in Progress

  • PBR and emission materials in glTF

    The glTF format supports Physically based rendering also. PBR allow renderers to display objects with a realistic appearance under different lighting conditions, the shading model has to take the physical properties of the object surface into account. There are different representations of these physical material properties. One that is frequently used is the metallic-roughness-model. We have various material properties already in FURY, we need to apply it to glTF models as well.

  • Skinning for models with no indices

    The glTF format supports non-indexed geometry (e.g., the Fox model). We currently do not know how to render the model without indices. I tried estimating it in this branch. However, It fails to render in skinning.

    Branch URL:

Other Objectives

  • Fetcher for importing glTF files from Khronos-glTF-Samples

    The KhronosGroup/gltf-samples contain multiple glTF sample models to test a glTF viewer for free. Implemented new methods in fetcher that can load all of these models by (using name and type) asynchronously. The glTF fetcher is capable of the following:

    • Downloading multiple models asynchronously.
    • Get the path to the downloaded model using it's name.
    • Download any model using the URL of the model.

    Pull Requests:

  • Other Pull Requests

  • GSoC weekly blogs

Timeline

Date Description Blog Link
Week 0
(24-05-2022)		 My Journey to GSoC 2022 FURY-Python
Week 1
(20-06-2022)		 Week 1 - A Basic glTF Importer FURY-Python
Week 2
(29-06-2022)		 Week 2 - Improving Fetcher and Exporting glTF FURY-Python
Week 3
(04-07-2022)		 Week 3 - Fixing fetcher, adding tests and docs FURY-Python
Week 4
(12-07-2022)		 Week 4 - Finalizing glTF loader FURY-Python
Week 5
(19-07-2022)		 Week 5 - Creating PR for glTF exporter and fixing the loader FURY-Python
Week 6
(25-07-2022)		 Week 6 - Extracting the animation data FURY-Python
Week 7
(01-08-2022)		 Week 7 - Fixing bugs in animations FURY-Python
Week 8
(09-08-2022)		 Week 8 - Fixing animation bugs FURY-Python
Week 9
(17-08-2022)		 Week 9 - First working skeletal animation prototype FURY-Python
Week 10
(25-08-2022)		 Week 10 - Multi-node skinning support FURY-Python
Week 11
(31-08-2022)		 Week 11 - Multiple transformations support and adding tests FURY-Python
Week 12
(08-09-2022)		 Week 12 - Adding skeleton as actors and fix global transformation FURY-Python
Week 13
(15-09-2022)		 Week 13 - Multi bone skeletal animations FURY-Python
Week 14
(28-09-2022)		 Week 14 - Morphing is here ! FURY-Python
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment