I would like to thank my mentors Hein-Pieter van Braam and Thomas Herzog, as well as the Godot contributors, for supporting me, and reviewing my ideas and code through out the project.
Godot is an MIT licensed Game Engine (Yes, that means it's free to use any way you want, and to modify.) With ability to make games for Windows, MacOS, Linux, Android, iOS, HTML, and XBox One (Through UWP)
I have been contributing to Godot for almost 6 months now. Solving smaller issues and now, contributing on a larger piece!
In particular, my Google Summer of Code project was to create interfaces in the game engine to support video decoders so that the in-game VideoPlayer could play more than just Theora and Webm.
But why stop there? I also made a prototype video decoder GDNative plugin using FFmpeg that allows the user to play mp4 files in Godot.
GDNative's API allows users to get access to a lot of godot's internal 'helper' functions. In turn, the user can write a dynamic library that contains functions that can be called from classes inside the engine.
In this case, the VideoPlayer class in Godot uses a VideoStream to get video and audio frames from, in order to play the video. Before my project, two concrete implementations of VideoStream existed - VideoStreamTheora and VideoStreamWebm, both consisting decoders for their respective formats.
I added a VideoStreamGDNative class that is used as the VideoStream for the player, but instead of containing decoding logic inside, it makes calls to the plugin through the exposed interface.
The plugins here, are what contains the decoding logic, and can be added, swapped, removed by the user without any need to re-build the engine.
A static instance of VideoDecoderServer (As opposed to singleton to keep it from being global unnecessarily) registers interfaces for any plugins added, and makes their supported extensions available to the resource loader, so as to appropriately support loading the right resource, as well as allow the editor to recognize any formats as formats to be sent to VideoStreamGDNative.
The interface helps the VideoStreamGDNative class to get decoded audio and video frames.
On the other side, the video decoder plugin is responsible for all the decoding and similar heavy lifting. It can use any system that it needs in order to decode the videos and merely has to fulfill the requirement of the engine.
This abstraction allows for an easier development for plugin developers as they don't need to worry about Godot's internal systems.
So far the interface is complete (with precautions for possible additions), and Godot can use any interface simply. It can recognize the required decoder by the supported extensions and loads the appropriate one.
The plugin right now can decode videos for any video format you configure the FFmpeg libraries for. However, the build system is not configured to build FFmpeg as per need, nor does the library support the ability to move around the libraries.
Making movable and redistributable versions of FFmpeg's libraries has to be worked out (maybe requiring an installation on the platform)
There is no support for threading, which needs to be added, for possible performance improvements.
The repo for the plugin can be found here. You will also need, FFmpeg source and the supporting version of Godot.
Simply configure the build system to your liking, and build into the test/addons/bin/osx (replace osx with win64 for windows, and x11 for linux, depending on your system) subfolder of the git repository. Then, using SCons, build the project with scons test=yes platform=osx. (Replace osx with windows or linux)
Building the source is a bit of a pain right now, since it is build for faster testing at the moment. This will be the case until the videodecoder is ready for final release. (Requires a good amount of cleanup and testing.)