Skip to content

Instantly share code, notes, and snippets.

@meryacine
Last active Nov 25, 2021
Embed
What would you like to do?
Shaka Streamer Enhancements Work Product Report - GSoC 2021

GSoC-2021 With Shaka Streamer

Project Information:

Project Name: Shaka Streamer Enhancements

Mentor: Joey Parrish
Student: Omer Yacine

Project Description:

Shaka Streamer is a tool that simplifies media transcoding and packaging by utilizing FFmpeg and Shaka Packager and managing the command line arguments that should be fed to them using simple and reusable configuration files.

The two features concerning this project:

  • Adding a PeriodConcat node that will make Shaka Streamer capable of producing multi-period DASH manifests using the single-period manifests outputted by Shaka Packager(which yet doesn't support multi-period DASH). A workaround can be done for HLS since it doesn't support periods, by using #EXT-X-DISCONTINUITY tag as a separator between different periods.
  • Bundling hermetic copies of FFmpeg and Shaka Packager and allowing the user to specify whether Shaka Streamer should use the hermetic versions or the system-wide versions instead.

Work Done:

  • Multi-period for DASH (Merged)
    This commit manipulated the structure of the input configuration to let it accept the new multi-period option. The field multiperiod_inputs_list was added to the input configuration. This commit also implemented the DASH period concatenation, in which, the streamer would manage multiple PackagerNodes and concatenate their output when they finish using python's native xml module.
  • Multi-period for HLS (Merged)
    In this PR, a minimal HLS parser was implemented. It searches only for the tags we care about and keep the rest of the playlist untouched. After all the master and media playlists are parsed and represented as python objects, all the media playlists are then sorted out into the appropriate type(video-audio-subtitles) and get fed to one of the four concatenation methods. These methods will make the best concatenation choices based on a playlist's attributes: codec, resolution, channel layout, and language.
  • Bringing Windows Support (Merged)
    This commit has introduced a new Pipe object that would abstract POSIX pipes(which are made using the os.mkfifo() system call), Windows pipes(implemented as two named pipes between two processes and a server thread that moves the data between the two pipes), and file paths. With all the platform-dependent code being isolated away in a new file(pipe.py), the rest of the project could then use the Pipe object without needing to know the platform it operates on.
  • Offering hermetic platform-specific binaries (Merged)
    This PR implements the integration between Shaka Streamer and the new PyPi package shaka-streamer-binaries. It also has a build script to automate building shaka-streamer-binaries for all our supported platforms, and a download script to download and sort out the latest versions of the binaries we need for the building process.

Other relating changes:

  • Adding channel_layouts field to the pipeline config (Merged)
    This commit helped simplifying the multi-period for HLS task. Prior to this, the implementation of the audio concatenation for HLS was more complex and would sometimes produce multiple media playlists with the same attributes but different period characteristics in some periods, which an HLS player could not differentiate between. In this commit, the field channel_count was removed and replaced with channel_layouts which offered more audio encoding flexibility.

All my PRs to Shaka Streamer can be found here.

TODO:

  • Writing documentation about the new package shaka-streamer-binaries in Shaka Streamer's docs.
  • Writing documentation on how to use multi-period concatenation and the prerequisites to carry out a successful concatenation.
  • Integrating the Streamer's multi-period support with Shaka Packager's HTTP PUT support.
  • Use posixpath module instead of os.path for period concatenation as these manifests will be used on the web on windows.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment