Skip to content

Instantly share code, notes, and snippets.

@Aeva

Aeva/tangerine_roadmap.md Secret

Last active June 21, 2022 01:23
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Aeva/ab8c18201eb801dd75d912299cf8aaa5 to your computer and use it in GitHub Desktop.
Save Aeva/ab8c18201eb801dd75d912299cf8aaa5 to your computer and use it in GitHub Desktop.
Download ZIP
Tangerine Roadmap
Raw
tangerine_roadmap.md

Tangerine Development Outline

This is a loose outline of my goals for the project. As I'm working on my own on this in my spare time, there are no concrete milestone dates planned, and I may not ever get to everything. The goals listed below are loosely organized in order of my current priorities, which will naturally change over time, especially if anyone else starts using the project.

My intention is to build Tangerine into a small but interesting art program first, then expand it to be somewhere between an interactive art program and a game engine, and then finally (if it still seems like a good idea) broaden it into a full featured game engine and expand the platforms supported.

near term goals

  • Rlease build blockers

    • Basic frustum culling.
    • Option to only re-render when there's an input event.
    • Easy to move camera controller.
    • Correct license information for full dependency graph.
      • Waiting for Racket 8.6

  • Finish implementing occlusion culling

    • Current implementation is promising, but buggy. A complete implementation is expected to cut the rendering time in half or better for most objects.
    • I believe the HZB generation is correct.
    • I believe the VS behavior is producing the wrong UVs, and thus is reading from the HZB with a slight offset.
    • Need better debugging tools and some kind of feedback mechanism for perf metrics.
    • Secondary stage missing.
    • Culling pass + indirect draw? Mesh shaders?

  • Embedded languages

    • Lua and/or Luau
    • Wren
    • Python?
    • Should these all be optional plugins?
      • Should there be a plugin interface for other things?
    • Should any always be embedded?
      • Lua is the obvious candidate, though Wren is a solid contender.
    • API changes?
      • Should there be a common high level AST like the "CSGST" expressions the Racket frontend currently builds, or should the backend SDF nodes be constructed directly and held with handles?
      • How should octree compilation be exposed, and when should it be explicit?
    • Interop between scripts, so models and behaviors written in different languages can be mixed and matched.
      • For example, script requests Tangerine load another script and extract a model. The return would be a handle that the requester could then used to draw the model or modify it further.
      • Move stuff like color name resolution out of the Racket frontend API into a common C API.

  • Precompile the interpreter shader for all stack sizes up to maybe 10 on application start.

  • Async shader compilation with SPIRV

    • First implement synchronous compiling w/ SPIRV
    • Then make it threaded and use the interpreter while compiling.
    • Occlusion feedback should eventually guide compiling.

  • Elimination of empty octree nodes

    • Possible to solve if a volume contains an edge for most of the primitives.
    • Possible to determine these states for all voxels:
      • Interior only
      • Exterior only
      • Surface
      • Maybe Surface
    • This strategy won't catch everything, but could be refined upon later.

  • Revise paint semantics for diff nodes so subtractions can optionally transfer materials.

  • Expand named color definitions to support all CSS colors.

    • It is critical that there be a color named "tangerine" and that it is aesthetically pleasing.

  • More primitive shapes nodes.

  • Move material painting into own pass.

  • Gradient materials?

  • Better backgrounds

    • Solid image?
    • Sky boxes?
    • Configurable gradients?
    • Atmospheric scattering simulation?

  • Better tools for rapid modeling and procgen

    • Utilities for repetition. These would generate CSG nodes, and not be represented by any GPU operations.
      • Repeat along spline
      • Rotate around point
      • Repeat along line
      • Mirror
      • These probably will probably not be implemented in the HLLs to avoid redundant boilerplating, however it would be nice if a thunk could modify the generated nodes.
    • Heightmaps? Reading image data?
    • Better ray casting for projecting CSG trees onto other trees.
      • Make it possible to just return a null value, with semantics for deleting operators to get a valid tree.
      • Align projection to normal/gradient (think mounting spikes on a curved surface).
    • Option for vector types instead of split coordinates for all relevant HLL APIs.
    • Sampling point generation along isosurfaces?
    • Markov algorithm primitives?

long term goals

  • Touch screen audio experiment

    • This would mostly just be a JS export thing.
    • I'm unfamiliar with audio synthesis APIs.
    • This is intended to be an accessibility feature so that the user can "feel" the depth buffer like a relief.
    • It would also be nice if color was represented somehow.
    • The results should be aesthetically pleasing.
      • It is critical that colors associated with skin tones all sound equally nice.
    • Use as an instrument is a reasonable secondary use case. To that end, it might make sense to support this in the main application w/ midi out.

  • Mastodon bot

    • This should probably be luau based only.
    • Some easy way to link touch the touch screen audio stuff w/o requiring additional uploads? How would depth be encoded?

  • Basic Interactivity APIs

    • Making a simple game like minesweeper to help guide this?
    • High level lang (HLL) can register low level events for input and interactivity.
      • eg mouse over, mouse down, mouse click, mouse up, various keyboard events
      • Also for loading events
    • HLL queries for event updates. Some might also be reasonable to support callbacks for.
    • HLL can make ray casting queries and overlap queries.
    • HLL should be able to construct CSG graphs and place them in the scene as model instances.
    • HLL can freely update a global transform for all placed model instances.
    • HLL should be able to show and hide model instances, to front load compiling costs.
    • Loading screen transitions, pause screens?

  • Tangerine as a library

    • I'd like to have the "press button recieve game" model be optional.
    • Some languages are better served by this, like Racket and Python.
      • In these cases the parent language should not also be embedded, because that is silly.
      • Python could have a specialized C API to take advantage of specialized calling conventions.
      • Racket might need a generic C API, since to my knowledge there is no stable ABI.
    • Lua or Wren always embedded for interop with planned free asset library. They're smol anyway so whatever.
    • Can remove the silly "Miniscule" / "Majescule" split. Features like the renderer will just be platform specific until properly ported.

  • Distributing Tangerine games

    • This is the "press button recieve game" model.
    • I'd like this to be easier than making modifications to the main system and recompiling to get a new exe.
    • No specific ideas at this time.
    • Honestly I also like the idea of being able to distribute your game as a pile of files and then play it by just opening the starting level in the viewer program. Especially cool if you could export 3D models as "screen shots".

  • Gamepad input events

    • And vibration?

  • Audio support?

    • I'm unfamiliar with audio APIs. Maybe some or all of this could be done through SDL?
    • looping or non-looping sounds
    • support common audio formats
      • mp3, wav, ogg?
      • midi files too, might require providing a soft synth
    • constant volume or scale based on proximity to point or spline
    • controlled by HLL

  • Advanced Interactivity APIs

    • Ability to change projection would be nice. Eg fov, or switch to orthographic.
      • The renderer currently does not support this at all.
      • Ray generation currently assumes a single ray origin.
      • Screen space archoring would be useful for some cases.
    • Ability to control camera behavior directly or hook up several useful hard coded systems.
    • Multiple world layers w/ different projections and camera setups.
      • Worlds can be active and inactive.
      • Compositing rules and draw order.
      • Rendering to a texture.
      • Background layers eg sky boxes.
    • Animation drivers kind of like M.GRL, but they're just requested by the HLL and implemented either in the backend or by shaders. Eg, easing, path finding, etc.
    • Dynamic octree construction. Eg, using diffs to add damage.
      • I don't want to have a rigid distinction between a script and a model. For example, it would be nice for models to be composed of other models and define simple behavioral handles, eg vehicles.
      • Live parameterization for composite models, like changing the type of tires on a vehicle.
      • Live changes within composite models, like mounting spikes onto your car and pasting decals.

  • Sprites and text

    • Easy enough to do in-world.

  • Dynamic empty octree node elimination

    • Requires occlusion feedback, might be impractical.
    • If a voxel is drawn completely without occlusion, and generates no rayhits, then it should never be drawn again.

  • Dedicated UI APIs

    • Probably better to use a good middleware than to reinvent this.

  • Antialiasing???

  • Decals

  • Boneless animation

    • Key frames control model instance placement and orientation
    • Maybe also benefit from easing rules and paths.
    • Likely generates either an octree per frame, or a set of octrees that are shown / hidden / moved.
    • Container "format" (script export convention) and playback machinery needed.
    • Should be friendly for parameterization, eg sockets for connecting other models at runtime.
    • Would benefit from a wysiwyg editor.

  • Ligting improvements

    • Directional lights
    • Spot lights
    • Point lights
    • Shadows

  • Improve minspec

    • Profiling on different SKUs to figure out current realistic minspecs for 15 / 30 / 60 fps are.
    • Ideally should be able to make interesting things on XBox One equivalent hardware.
    • I'd like to support much older hardware if possible.
    • Reexplore mixing meshes and ray marching?
      • The main problem is preventing seams.
      • Maybe seams are acceptable for ultra-low quality modes?
      • Rayless mode that just has solid voxels w/ some simple deformation for blending regions?
    • Option to replace blend operators w/ equivalent non-blending ops.

  • Linux support

    • Mostly blocked on being unsure how to embed racket on Linux.
    • Current architecture is not friendly enough for volunteer packaging.
    • Binary distribution on Linux is weird.

very long term goals

  • Octree optimizing

    • Eliminating nodes that do not affect the final render. Eg, the inner sphere in a union of two concentric spheres.

  • Better mesh generation for export.

    • Watertight meshes suitable for 3D printing w/ requiring meshlab.

  • Replace OpenGL with Vulkan?

    • Somewhat blocked on the SPIRV conversion.
    • Needs to be built modularly so that other backends are possible.
      • Maybe keep GL as an optional one for a while?
    • Tedious.
    • NVRHI might be a good fit for this project?
    • Likewise for AMD's allocator libraries etc.

  • More ligting improvements

    • Optional material models, eg PBR materials?
    • Area lights
    • Transmission
    • Transparency?

  • Impove "max spec"

    • Sexy DXR stuff?

  • Graphical model editor

    • Wysiwyg
    • Visual scripting interface for graph connectivity?
    • Click on things to translate and rotate them and change properties.

  • Integrated text editor

    • Formatting, errors, and easy reload for the supported high level languages would be nice.
    • Basic text editing.
    • Mixed editing w/ the gui would be nice, but I don't know what that would look like.

  • Physics simulation?

    • Should this be SDF based, or use a middleware?

  • Some kind of posable character rig system.

    • Blending constraints (eg inserting a sphere area where a limb segment is blended with something else)
    • Heirarchical transforms, should support dynamic changes.
    • Parameterization friendly, sockets for attaching other models.

  • Skeletal animation

    • Unsure how best to do this. This definitely needs a wysiwyg. Might be better to have some interop with blender for this?
    • Use character rig system.
    • IK possible?

  • Style guide and asset library

    • Lua, luau, or wren?

  • Console builds?

    • Xbox needs a UWP D3D12 renderer
    • Other platforms TBD
    • Demonstration games
    • Perf needs to be good enough to run w/o compiling, or there needs to be a way to record and precompile shaders.

  • OSX support

    • Blocked on like everything lol
      • I don't have any apple hardware.
      • I'm Unfamiliar development ecosystem.
      • I'd probably need to write a Metal backend.
      • I'm very nonplussed about the app store and requirements for code signing.
    • This is most likely not going to happen unless someone with the right skillset volunteers or I hire someone to do it.
    • I'd also have to acquire the appropriate hardware anyway to do my own regression testing and pushing new releases, and I'd also have to learn whatever cert workflow Apple requires, so this is basically just another game console.

  • IOS Support

    • Same problems as OSX support, possibly worse if none of the embedded languages are allowed.
    • Might be better to provide as a library for swift or something, should any enterprising app devolpers be interested in this for some reason.
    • Concerns about perf, and rendering quality.

  • More embedded languages?

    • Ruby?
    • Mono?

  • Midi input?

    • It wouldn't be hard to add, but I don't have a specific use case in mind.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment