Skip to content

Instantly share code, notes, and snippets.

@Aeva
Last active Jul 4, 2022
Embed
What would you like to do?
Revised Tangerine Roadmap and Priorities

Revised Tangerine Development Outline

This is a loose outline of my goals for the project. As I'm working on my own on this in my spare time, there are no concrete milestone dates planned, and I may not ever get to everything. The goals listed below are loosely organized in order of my current priorities, which will naturally change over time, especially if anyone else starts using the project.

My intention is to build Tangerine into a small but interesting art program first, then expand it to be somewhere between an interactive art program and a game engine, and then finally (if it still seems like a good idea) broaden it into a full featured game engine and expand the platforms supported.

Other notes:

  • I have release builds now.
  • Linux builds deferred to volunteers, looks like vivi has finished most of it.

Basic Interactivity

These are tasks that will make it possible to at least make rudimentry interactive games w/ Tangerine. My plan is to focus on Lua for building these APIs and also exposing these via a generic C API. This will make wrapping Tangerine with other languages easier. Interactivity APIs will probably not be backported to Racket right away, but I would like to keep Racket models functioning while working on this. More thoughts on this.

  • Finish embedding Lua

    • Wrap the existing shape construction APIs
    • Move remaining racket functionality into the backend
    • Revise API naming
    • Should the API symbols just go directly into the global scope?
    • Rework paint semantics to merge PaintNode into BrushNode
    • Double check shape APIs
    • Ray casting API
  • API for registering active keyboard events

    • C API for polling for new events
    • Lua API for registering callbacks?
  • Instancing multiple models together in one scene.

  • API for updating global transforms on model instances.

  • API for showing and hiding models.

  • Mouse events w/ picking.

  • Sigmar's Garden

Improved Interactivity

These are things that I expect to be useful for building ordinary games, but are gated on building out the basic interactivity API first.

  • Expose loading state as registerable events

  • API for showing/hiding a loading screens

  • Ability to change projection. Eg fov, or switch to orthographic.

    • The renderer currently does not support this at all.
    • Ray generation currently assumes a single ray origin.
    • Screen space archoring would be useful for some cases.
  • Ability to control camera behavior directly or hook up several useful hard coded systems.

  • Multiple world layers w/ different projections and camera setups.

    • Worlds can be active and inactive.
    • Compositing rules and draw order.
    • Rendering to a texture.
    • Background layers eg sky boxes.
  • Animation drivers kind of like M.GRL, but they're just requested by the high level API and implemented either in the backend or by shaders. Eg, easing, path finding, etc.

  • Dynamic octree construction. Eg, using diffs to add damage.

    • I don't want to have a rigid distinction between a script and a model. For example, it would be nice for models to be composed of other models and define simple behavioral handles, eg vehicles.
    • Live parameterization for composite models, like changing the type of tires on a vehicle.
    • Live changes within composite models, like mounting spikes onto your car and pasting decals.
  • Audio APIs

    • I'm unfamiliar with audio APIs. Maybe some or all of this could be done through SDL?
    • looping or non-looping sounds
    • support common audio formats
      • mp3, wav, ogg?
      • midi files too, might require providing a soft synth
    • constant volume or scale based on proximity to point or spline
  • Gamepad input events

    • And vibration?
  • Improve minspec

    • Profiling on different SKUs to figure out current realistic minspecs for 15 / 30 / 60 fps are.
    • Ideally should be able to make interesting things on XBox One equivalent hardware.
    • I'd like to support much older hardware if possible.
    • Reexplore mixing meshes and ray marching?
      • The main problem is preventing seams.
      • Maybe seams are acceptable for ultra-low quality modes?
      • Rayless mode that just has solid voxels w/ some simple deformation for blending regions?
    • Option to replace blend operators w/ equivalent non-blending ops.
  • Make more elaborate games.

Misc Low Hanging Fruit

These are things that can be implemented at any time, and are well suited to evening and low energy weekend projects.

  • Option to only re-render when there's an input event. implemented!

  • Easy to move camera controller.

  • Revise paint semantics for diff nodes so subtractions can optionally transfer materials.

  • Elimination of empty octree nodes

    • Possible to solve if a volume contains an edge for most of the primitives.
    • Possible to determine these states for all voxels:
      • Interior only
      • Exterior only
      • Surface
      • Maybe Surface
    • This strategy won't catch everything, but could be refined upon later.
  • Expand named color definitions to support all CSS colors.

  • Update Racket boilerplate to use the new color names API.

  • More primitive shapes nodes.

  • Move material painting into own pass.

  • Better backgrounds

    • Solid image?
    • Sky boxes?
    • Configurable gradients?
    • Atmospheric scattering simulation?
  • Utilities for repetition. These would generate CSG nodes, and not be represented by any GPU operations.

    • Repeat along spline
    • Rotate around point
    • Repeat along line
    • Mirror
    • These probably will probably not be implemented in the HLLs to avoid redundant boilerplating, however it would be nice if a thunk could modify the generated nodes.
  • Heightmaps? Reading image data?

  • Better ray casting for projecting CSG trees onto other trees.

    • Make it possible to just return a null value, with semantics for deleting operators to get a valid tree.
    • Align projection to normal/gradient (think mounting spikes on a curved surface).
  • Volume overlap queries.

  • Option for vector types instead of split coordinates for all relevant HLL APIs.

  • Precompile the interpreter shader for all stack sizes up to maybe 10 on application start.

  • Dedicated UI APIs

    • Probably better to use a good middleware than to reinvent this.

Mañana

These are larger items that I would like to do sooner than later, but either need more research or require a higher functioning weekend or three each to finish.

  • Release builds to check for updates.

  • Learning resources

  • Stencil operators. These would be to for using SDF shapes as material masks.

  • Move embedded Racket into optional plugin

  • Backport interactivity APIs to Racket

  • Frustum culling

    • Option 1: CPU culling
      • ModelSubtree would need to be updated to add bounds info.
      • Expected to scale poorly, would have to test hundreds of AABBs with many models right now.
    • Option 2: GPU culling
      • Would need a large buffer of parameters, and a large buffer for indirect dispatch.
      • In theory the buffers would only need to be updated when the scene composition changes.
      • Compute shader cranks through all of the tests and disables out of bound draws.
      • More natural place to do HZB culling as well than VS.
  • Finish implementing occlusion culling

    • Current implementation is promising, but buggy. A complete implementation is expected to cut the rendering time in half or better for most objects.
    • I believe the HZB generation is correct.
    • I believe the VS behavior is producing the wrong UVs, and thus is reading from the HZB with a slight offset.
    • Need better debugging tools and some kind of feedback mechanism for perf metrics.
    • Secondary stage missing.
    • Culling pass + indirect draw? Mesh shaders?
  • Async shader compilation with SPIRV

    • First implement synchronous compiling w/ SPIRV
    • Then make it threaded and use the interpreter while compiling.
    • Occlusion feedback should eventually guide compiling.
  • Tangerine as a library

    • I'd like to have the "press button recieve game" model be optional.
    • Some languages are better served by this, like Racket.
      • In these cases the parent language should not also be embedded, because that is silly.
      • Generic C API for wrapping against.
    • Lua or Wren always embedded for interop with planned free asset library. They're smol anyway so whatever.
    • Can remove the silly "Miniscule" / "Majescule" split. Features like the renderer will just be platform specific until properly ported.
  • Decals

Very Mañana

These are items that I think are important, but are not necessary for my near term goals. Many of these are ideas that need more research and experimentation, or just need to sit in the thought cabinent for a while. That doesn't mean I'll do these last, just don't bet money on it.

  • Antialiasing???

  • Distributing Tangerine games

    • This is the "press button recieve game" model.
    • I'd like this to be easier than making modifications to the main system and recompiling to get a new exe.
    • No specific ideas at this time.
    • Honestly I also like the idea of being able to distribute your game as a pile of files and then play it by just opening the starting level in the viewer program. Especially cool if you could export 3D models as "screen shots".
  • Sampling point generation along isosurfaces?

    • Markov algorithm primitives?
  • Gradient materials?

  • world space sprites and text

  • Touch screen audio experiment

    • This would mostly just be a JS export thing.
    • I'm unfamiliar with audio synthesis APIs.
    • This is intended to be an accessibility feature so that the user can "feel" the depth buffer like a relief.
    • It would also be nice if color was represented somehow.
    • The results should be aesthetically pleasing.
      • It is critical that colors associated with skin tones all sound equally nice.
    • Use as an instrument is a reasonable secondary use case. To that end, it might make sense to support this in the main application w/ midi out.
  • Mastodon bot

    • Requires decent sandboxing.
    • Some easy way to link touch the touch screen audio stuff w/o requiring additional uploads? How would depth be encoded?
  • Dynamic empty octree node elimination

    • Requires occlusion feedback, might be impractical.
    • If a voxel is drawn completely without occlusion, and generates no rayhits, then it should never be drawn again.
  • Boneless animation

    • Key frames control model instance placement and orientation
    • Maybe also benefit from easing rules and paths.
    • Likely generates either an octree per frame, or a set of octrees that are shown / hidden / moved.
    • Container "format" (script export convention) and playback machinery needed.
    • Should be friendly for parameterization, eg sockets for connecting other models at runtime.
    • Would benefit from a wysiwyg editor.
  • Ligting improvements

    • Directional lights
    • Spot lights
    • Point lights
    • Shadows
  • Octree optimizing

    • Eliminating nodes that do not affect the final render. Eg, the inner sphere in a union of two concentric spheres.
  • Better mesh generation for export.

    • Watertight meshes suitable for 3D printing w/ requiring meshlab.
  • Replace OpenGL with Vulkan?

    • Somewhat blocked on the SPIRV conversion.
    • Needs to be built modularly so that other backends are possible.
      • Maybe keep GL as an optional one for a while?
    • Tedious.
    • NVRHI might be a good fit for this project?
    • Likewise for AMD's allocator libraries etc.
  • More ligting improvements

    • Optional material models, eg PBR materials?
    • Area lights
    • Transmission
    • Transparency?
  • Polygon mesh import

    • Mix triangle rendering, only switch to CSG where ops are applied?
    • Montecarlo mode to approximate w/ shapes instead of triangle brushes.
  • Impove "max spec"

    • Sexy DXR stuff?
  • Graphical model editor

    • Wysiwyg
    • Visual scripting interface for graph connectivity?
    • Click on things to translate and rotate them and change properties.
  • Integrated text editor

    • Formatting, errors, and easy reload for the supported high level languages would be nice.
    • Basic text editing.
    • Mixed editing w/ the gui would be nice, but I don't know what that would look like.
  • Physics simulation?

    • Should this be SDF based, or use a middleware?
  • Some kind of posable character rig system.

    • Blending constraints (eg inserting a sphere area where a limb segment is blended with something else)
    • Heirarchical transforms, should support dynamic changes.
    • Parameterization friendly, sockets for attaching other models.
  • Skeletal animation

    • Unsure how best to do this. This definitely needs a wysiwyg. Might be better to have some interop with blender for this?
    • Use character rig system.
    • IK possible?
  • Style guide and asset library

    • Character design first probably, to establish scale
    • Ideally I'd hire an artist to make the models, but I'll probably have better luck hiring a concept artist to help me build the sytle guide and examples, and then implement the models myself. Hard to say what the future holds though.
  • Embed or wrap other languages / runtimes

    • Wren
    • Python?
    • Ruby?
    • Mono?
  • Console builds?

    • Xbox needs a UWP D3D12 renderer
    • Other platforms TBD
    • Demonstration games
    • Perf needs to be good enough to run w/o compiling, or there needs to be a way to record and precompile shaders.
  • OSX support

    • Blocked on like everything lol
      • I don't have any apple hardware.
      • I'm Unfamiliar development ecosystem.
      • I'd probably need to write a Metal backend.
      • I'm very nonplussed about the app store and requirements for code signing.
    • This is most likely not going to happen unless someone with the right skillset volunteers or I hire someone to do it.
    • I'd also have to acquire the appropriate hardware anyway to do my own regression testing and pushing new releases, and I'd also have to learn whatever cert workflow Apple requires, so this is basically just another game console.
  • IOS Support?

    • Same problems as OSX support, possibly worse if none of the embedded languages are allowed.
    • Might be better to provide as a library for swift or something, should any enterprising app devolpers be interested in this for some reason.
    • Concerns about perf, and rendering quality.
  • Android Support????

  • Midi input?

    • It wouldn't be hard to add, but I don't have a specific use case in mind.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment