A Beginner's Guide to 3D Gaussian Splatting (3DGS)

00_Start_Here.md

3D Gaussian Splatting Quick-Start Guide

A practical guide to generating, training, and cleaning 3D Gaussian Splats (3DGS). Navigate to specific workflows below.

📋 Contents & Navigation

• Phase 0: Capture Fundamentals — Essential rules for capturing photos or videos before training.
• Use Case: Video Game Capture — Capture and reconstruct virtual environments using in-game photo modes or camera mods.
• Option A: Local Desktop Training — High-quality local training using LichtFeld Studio and Postshot.
• Option B: Synthetic Splatting in Blender — Skip physical capture and generate perfect 3DGS training datasets from 3D models.
• Option C: Cloud-Hosted Web Services — Upload videos and let external servers do the heavy lifting (Polycam, KIRI Engine, FreeGaussian).
• Option D: Local Mobile App — Run everything on your phone with zero setup using Scaniverse.
• Phase E: Editing & Cleanup — Post-processing and cleaning up your splats in SuperSplat.

---

What is Gaussian Splatting?

Instead of representing 3D scenes with standard polygonal meshes and textures, 3D Gaussian Splatting (3DGS) represents space as millions of tiny, semi-transparent 3D ellipsoids (splats). This technique captures complex features like reflections, glass, thin structures, and fuzzy boundaries (like leaves or hair) with photorealism.

01_Capture_Fundamentals.md

Phase 0: Capture Fundamentals

Your output is directly limited by the quality of your inputs. Reconstruction algorithms need sharp, consistent images to compute camera paths and splat densities.

Camera Movement & Coverage

• Move Slowly: Keep camera panning slow and deliberate. Motion blur will ruin tracking.
• The Orbit Technique: Walk in a circle (360 degrees) around your subject.
  • Do this at three different heights: looking slightly down, looking straight ahead, and looking slightly up.
• Maintain Overlap: Ensure each photo overlaps the last one by at least 70–80%. If capturing a video, do not swing the camera around too quickly.

Lighting & Environment

• Avoid Harsh Shadows: Direct sunlight creates sharp shadows and glare that change depending on camera angle. Diffuse, even lighting (like an overcast day or a well-lit indoor room) is ideal.
• Static Scenes Only: Any movement in the scene (moving cars, wind blowing leaves, shifting light) will break the alignment phase. Ensure the scene remains perfectly static during capture.

Material Restrictions

• Highly Reflective Materials: Mirrors, polished metal, and glass confuse camera solvers because the reflections shift as you move.
• Featureless Surfaces: Pure white walls or completely smooth plastic are difficult for alignment software to lock onto. Textures provide the points the software uses to track positions.

02_Video_Game_Capture.md

Use Case: Video Game Capture

You can generate photorealistic Gaussian Splats of virtual characters, weapons, or environments directly from video games by capturing screenshots using built-in photo modes or community-made camera mods.

---

Step 1: Getting a Free Camera

To capture the 360-degree coverage needed for training, you must be able to move the camera independently of the character.

• Official Photo Modes: Games like Marvel's Spider-Man, God of War (2016 & Ragnarok), Space Marine 2, and Cyberpunk 2077 have built-in photo modes. Pause the gameplay and activate the free-camera mode.
• Community Mods: For games without native free-cameras (e.g., Elden Ring, Dark Souls series, or The Witcher 3), you can use custom injection tools or camera mods (such as Otis_inf's camera tools or cheat engine tables) to freeze gameplay and unlock the camera.

---

Step 2: The Capture Process

Once the game action is frozen:

1. Disable HUD & UI: Ensure all menu overlays, grids, and UI buttons are hidden.
2. Turn Off Depth of Field (DoF): Set DoF to 0. Background blur will corrupt the training dataset. The entire frame needs to be as sharp as possible.
3. Lock the Lighting: Do not change filters, brightness, contrast, or the game's time-of-day clock during the orbit, as the solver expects lighting to remain constant.
4. Capture your Orbit:
   • Rotate the free camera around your subject, snapping screenshots at regular intervals (aim for 80–150 images).
   • Take shots from multiple heights (looking down, looking straight, looking up) keeping consistent overlap between frames.
5. Export: Move the screenshots into a single folder on your PC.

---

Step 3: Reconstruction

Because screenshots are digital and contain no physical noise, they reconstruct beautifully.

• Option A: Drag the folder directly into Postshot or LichtFeld Studio to align camera poses and train the model locally.
• Option B: Upload the zipped screenshots folder to a cloud builder like Polycam or KIRI Engine.
• Option C: Process the screenshots folder in COLMAP to generate camera tracking data, which can then be imported into desktop training programs.

03_Local_Desktop_LichtFeld_Postshot.md

Option A: Local Desktop Training

Training locally gives you total control over privacy, resolution, and output files, but it requires a Windows PC with an NVIDIA GPU (minimum 6GB VRAM, 8GB+ recommended) and CUDA drivers installed.

---

Step 1: Camera Alignment (SfM)

Before training a splat, software needs to calculate the exact 3D position and angle of the camera for each frame. This is called Structure-from-Motion (SfM).

• How to do it: Use RealityScan (mobile app that syncs to Epic Games' cloud) or a local photogrammetry solver like COLMAP.
• The Output: You need to export the aligned camera data in COLMAP format (a directory containing a sparse point cloud folder and camera metadata).

---

Step 2: Training with Postshot (GUI-focused)

Postshot is one of the fastest and easiest local GUI programs for training splats.

1. Import: Drag and drop your folder of photos or video file directly into the Postshot GUI.
2. Align Cameras: Postshot has a built-in tracker. Click Import and let it calculate camera positions. You'll see a sparse cloud of points representing camera locations.
3. Train: Click Train. The 3D viewport will show the splat refining in real-time, sharpening from a cloudy haze into a clear object.
4. Crop & Export: Use the GUI bounding box tool to crop out background noise. Once satisfied with the quality (usually 5–15 minutes), click Export to save as a .ply file.
5. Note: Advanced users can automate and batch-process jobs in Postshot via its command line interface (CLI).

---

Step 3: Training with LichtFeld Studio (GUI-focused)

LichtFeld Studio is a powerful, open-source workstation for training, editing, and managing Gaussian Splats.

1. Import Aligned Project: Drag and drop your pre-aligned COLMAP project folder (generated in Step 1) into the LichtFeld Studio GUI.
2. Configure Settings:
   • Strategy: Select the MRNF training strategy (highly recommended for balanced quality).
   • Max Gaussians: Set based on scene size (typically 3,000,000 to 12,000,000).
   • SH Degree: Usually set to 1, 2, or 3 (lower values use less VRAM).
3. Train: Click Start Training. Watch the model converge live in the viewport.
4. Interactive Cleanup: LichtFeld Studio features powerful selection tools. Use the brush or lasso tools directly in the viewport to highlight and delete unwanted "floaters" or background structures.
5. Export: Export the cleaned splat as .ply, .sog, .spz, or as a standalone web viewer.
6. Note: While the GUI is the primary way to interact with LichtFeld Studio, you can also control and script it via CLI arguments for automated pipeline integrations.

04_Synthetic_Splatting_Blender.md

Option B: Synthetic Splatting in Blender

If you want to turn a digital 3D model (mesh) into a Gaussian Splat instead of a physical object, you can use synthetic splatting.

Normally, camera alignment (SfM) is the most time-consuming and error-prone phase of Gaussian Splatting because software has to "guess" camera angles from photos. With a digital model in Blender, we know the exact camera coordinates mathematically. By rendering the model from multiple angles and exporting the exact camera matrices, we can skip tracking altogether and train the splat immediately.

---

The Automation Scripts

By running two Python scripts inside Blender, you can automate this entire dataset generation process.

Script 1: Camera Sphere Setup

This script creates a collection of cameras uniformly distributed in a sphere pointing directly at your target object, using the Fibonacci Sphere algorithm for even coverage.

import bpy
import math

# PARAMETERS
DISTANCE = 50.0     # Distance of cameras from target object
COUNT = 300         # Total cameras to generate
TARGET_NAME = ""    # Put target object name here, or leave empty for (0,0,0)

def setup_camera_sphere(radius, count, target_obj_name=""):
    collection_name = "Splat_Cameras"
    if collection_name in bpy.data.collections:
        col = bpy.data.collections[collection_name]
        for obj in col.objects:
            bpy.data.objects.remove(obj, do_unlink=True)
    else:
        col = bpy.data.collections.new(collection_name)
        bpy.context.scene.collection.children.link(col)

    target_loc = (0, 0, 0)
    if target_obj_name and target_obj_name in bpy.data.objects:
        target_loc = bpy.data.objects[target_obj_name].location
    
    phi = math.pi * (3. - math.sqrt(5.))  # Golden angle in radians

    for i in range(count):
        y = 1 - (i / float(count - 1)) * 2
        rad = math.sqrt(1 - y * y)
        theta = phi * i

        x = math.cos(theta) * rad
        z = math.sin(theta) * rad

        pos = (x * radius + target_loc[0], 
               y * radius + target_loc[1], 
               z * radius + target_loc[2])

        cam_data = bpy.data.cameras.new(name=f"SplatCam_{i}")
        cam_obj = bpy.data.objects.new(f"SplatCam_{i}", cam_data)
        col.objects.link(cam_obj)
        cam_obj.location = pos

        constraint = cam_obj.constraints.new(type='TRACK_TO')
        
        if not bpy.data.objects.get("Splat_Target"):
            empty = bpy.data.objects.new("Splat_Target", None)
            empty.location = target_loc
            bpy.context.scene.collection.objects.link(empty)
        
        constraint.target = bpy.data.objects["Splat_Target"]
        constraint.track_axis = 'TRACK_NEGATIVE_Z'
        constraint.up_axis = 'UP_Y'

    print(f"Successfully created {count} cameras in collection '{collection_name}'")

setup_camera_sphere(DISTANCE, COUNT, TARGET_NAME)

---

Script 2: Rendering & Metadata Export

This script renders the target mesh from every camera generated by Script 1, extracts high-contrast alpha masks (which tell the training algorithm to ignore the background), and exports the camera parameters into standard COLMAP text files (cameras.txt, images.txt, points3D.txt).

import bpy
import os
import re
from mathutils import Matrix

# CONFIGURATION
BASE_PATH = "C:/path/to/output/folder/" # Adjust this to your target project folder
CAMERA_COLLECTION = "Splat_Cameras"

def sanitize_folder_name(name):
    name = re.sub(r"\.\d+$", "", name)
    name = re.sub(r'[<>:"/\\|?*]', "_", name)
    return name.strip().strip(".") or "mesh"

def mesh_automator():
    selected = [obj for obj in bpy.context.selected_objects if obj.type == 'MESH']
    if not selected:
        print("No mesh selected!")
        return

    scene = bpy.context.scene
    w = int(scene.render.resolution_x * (scene.render.resolution_percentage / 100))
    h = int(scene.render.resolution_y * (scene.render.resolution_percentage / 100))

    if CAMERA_COLLECTION not in bpy.data.collections:
        print(f"Camera collection '{CAMERA_COLLECTION}' not found.")
        return

    cameras = [obj for obj in bpy.data.collections[CAMERA_COLLECTION].objects if obj.type == 'CAMERA']
    if not cameras:
        print("No cameras found.")
        return

    for target_obj in selected:
        folder_name = sanitize_folder_name(target_obj.name)
        project_path = os.path.join(BASE_PATH, folder_name)
        sparse_path = os.path.join(project_path, "sparse", "0")
        img_path = os.path.join(project_path, "images")
        mask_path = os.path.join(project_path, "masks")

        for p in [sparse_path, img_path, mask_path]:
            if not os.path.exists(p):
                os.makedirs(p)

        # Hide other meshes
        for obj in bpy.data.objects:
            if obj.type == 'MESH':
                obj.hide_viewport = True
                obj.hide_render = True
        target_obj.hide_viewport = False
        target_obj.hide_render = False

        setup_compositor(scene, mask_path)
        for cam in cameras:
            scene.camera = cam
            scene.render.filepath = os.path.join(img_path, f"{cam.name}.png")
            file_output = scene.node_tree.nodes.get('File Output')
            file_output.file_slots[0].path = f"{cam.name}_mask"

            bpy.ops.render.render(write_still=True)

            old_mask = os.path.join(mask_path, f"{cam.name}_mask{scene.frame_current:04d}.png")
            new_mask = os.path.join(mask_path, f"{cam.name}.png")
            if os.path.exists(old_mask):
                if os.path.exists(new_mask):
                    os.remove(new_mask)
                os.rename(old_mask, new_mask)

        export_points_txt(target_obj, sparse_path)
        export_metadata_txt(cameras, sparse_path, w, h)

def setup_compositor(scene, mask_path):
    scene.use_nodes = True
    nodes = scene.node_tree.nodes
    links = scene.node_tree.links
    nodes.clear()
    rl = nodes.new('CompositorNodeRLayers')
    comp = nodes.new('CompositorNodeComposite')
    fo = nodes.new('CompositorNodeOutputFile')
    fo.name = 'File Output'
    fo.base_path = mask_path
    fo.format.file_format = 'PNG'
    fo.format.color_mode = 'BW'
    links.new(rl.outputs['Image'], comp.inputs[0])
    links.new(rl.outputs['Alpha'], fo.inputs[0])

def export_points_txt(obj, path):
    depsgraph = bpy.context.evaluated_depsgraph_get()
    eval_obj = obj.evaluated_get(depsgraph)
    mesh = eval_obj.to_mesh()
    mat = obj.matrix_world
    verts = [mat @ v.co for v in mesh.vertices]

    with open(os.path.join(path, "points3D.txt"), 'w') as f:
        f.write("# POINT3D_ID, X, Y, Z, R, G, B, ERROR, TRACK[]\n")
        for i, v in enumerate(verts):
            f.write(f"{i+1} {v.x} {v.y} {v.z} 200 200 200 0.0\n")
    eval_obj.to_mesh_clear()

def export_metadata_txt(cameras, path, w, h):
    fl_x = (cameras[0].data.lens * w) / cameras[0].data.sensor_width
    with open(os.path.join(path, "cameras.txt"), 'w') as f:
        f.write(f"1 PINHOLE {w} {h} {fl_x} {fl_x} {w/2} {h/2}\n")

    with open(os.path.join(path, "images.txt"), 'w') as f:
        for i, cam in enumerate(cameras):
            flip = Matrix(((1, 0, 0, 0), (0, -1, 0, 0), (0, 0, -1, 0), (0, 0, 0, 1)))
            w2c = (cam.matrix_world @ flip).inverted()
            q, t = w2c.to_quaternion(), w2c.to_translation()
            f.write(f"{i+1} {q.w} {q.x} {q.y} {q.z} {t.x} {t.y} {t.z} 1 {cam.name}.png\n\n")

mesh_automator()

for obj in bpy.data.objects:
    if obj.type == 'MESH':
        obj.hide_viewport = False
        obj.hide_render = False

---

How to use this workflow:

1. Open Blender and import your 3D model. Center it at (0, 0, 0).
2. Go to the Scripting tab. Create a new script, paste Script 1, and run it. You will see a sphere of cameras appear around your model.
3. Create another script, paste Script 2.
   • Edit BASE_PATH (line 9) to point to an empty directory on your drive.
4. Select the target mesh in Blender, then run Script 2.
   • Blender will render the mesh from each camera, generate corresponding black-and-white alpha masks, and export cameras.txt, images.txt, and points3D.txt directly to the sparse/0 subfolder.
5. Launch LichtFeld Studio or Postshot and drag-and-drop the generated folder directly into the GUI.
6. Since the camera matrices are already fully defined in the files, the software skips alignment and starts training your Gaussian Splat instantly!

05_Cloud_Hosted_Services.md

Option C: Cloud-Hosted Web Services

Cloud services process your models on remote servers. This is ideal if you have a recorded video file (MP4/MOV) and want high-quality processing without using a powerful desktop GPU.

Polycam (Web Interface)

1. Go to Polycam's website and log in.
2. Select Upload and choose Gaussian Splatting.
3. Upload your video file or a ZIP archive containing individual photo frames.
4. Let the cloud servers process the tracking and training (usually 10–20 minutes).
5. Open the web viewer to view and download your .ply or .splat file.

KIRI Engine (Web Dashboard)

1. Open the KIRI Engine Web App.
2. Click New Project and select Gaussian Splatting.
3. Upload your photos or video. KIRI Engine offers specific project settings based on whether you are scanning an object or a full scene.
4. Once processed on the cloud, you can edit the model using KIRI's web-based tools and export to standard formats.

FreeGaussian.ai

1. Visit FreeGaussian.ai.
2. Drag and drop your MP4 video file or a sequence of images.
3. The platform processes the reconstruction for free in the cloud.
4. Download the trained scene files directly when completed.

06_Local_Mobile_Scaniverse.md

Option D: Local Mobile Capture (Scaniverse)

Scaniverse processes Gaussian Splats completely locally on modern iOS and Android mobile devices. No cloud uploads, subscriptions, or desktop hardware required.

Setup & Capture

1. Download Scaniverse from the App Store or Google Play.
2. Tap New Scan and select Splat mode.
3. Move slowly around your target, aiming to fill in the colored overlay shown in the viewport. Make sure to capture it from multiple vertical angles.

Processing & Exporting

1. Tap Stop once coverage is complete.
2. Choose Standard (fast) or High (slower, more detail) processing.
3. The app trains the splat locally using your phone's processor.
4. Once completed, inspect the 3D model.
5. Tap Share or Export to export as a .ply file to your computer for further editing.

07_Cleanup_SuperSplat.md

Phase E: Editing & Cleanup

Trained splats often contain unwanted background noise, ground planes, or floating artifacts ("floaters") in areas the camera couldn't resolve clearly. You can clean these up using a visual editor.

Editing in SuperSplat

SuperSplat is a free, web-based editor designed for editing .ply files.

1. Open SuperSplat in your browser.
2. Drag and drop your .ply file directly into the window.
3. Selection Tools: Use the brush, lasso, or bounding box tools in the toolbar to select the messy areas or floaters you want to remove.
4. Delete: Hit Delete or use the trash icon to clear the selected Gaussians.
5. Alignment: Use the transform gizmo (W/E/R keys) to rotate, scale, or reposition the entire splat to align it with the grid.
6. Export: Export the cleaned and optimized file.

---

Critical: Export Formats & File Sizes

When exporting your clean splat, you will have several file format choices. If you plan to load your splat into the Splatter app on Reddit, pay attention to the format:

• .ply (Standard): Large, uncompressed file size. Great for archiving or editing in other software, but too heavy for web applications.
• .sog (Splats on Grid - Recommended): This is the smallest, most optimized format your splat can easily get. It compresses the data onto a grid structure, significantly reducing the memory footprint and download size without visible quality loss.
• Why .sog is required: Reddit imposes strict upload file size limitations on media and application assets. The Splatter app is designed to load .sog files to ensure instant load times and conform to these Reddit platform constraints.