Skip to content

Instantly share code, notes, and snippets.

@alexspeller
Created January 10, 2017 01:08
Show Gist options
  • Save alexspeller/aefdd5a6d7100d28d0bbc4838527f797 to your computer and use it in GitHub Desktop.
Save alexspeller/aefdd5a6d7100d28d0bbc4838527f797 to your computer and use it in GitHub Desktop.

I have multiple mp4 video files and I want to composite them into a single video. Each stream is an mp4 video. They are of different lengths, and each file also has audio.

The tricky thing is, I want the layout to change depending on how many streams are currently visible.

As a concrete example, say I have 3 video files:

File Duration Start End
a.mp4 30s 0s 30s
b.mp4 10s 10s 20s
c.mp4 15s 15s 30s

So at t=0 seconds, I want the video to look like this:

   +-------------------------+
   |                         |
   |                         |
   |                         |
   |                         |
   |          a.mp4          |
   |                         |
   |                         |
   |                         |
   |                         |
   |                         |
   +-------------------------+


At t=10s, I want the video to look like this:

+------------------------------+--------+
|                              |        |
|                              |        |
|                              | a.mp4  |
|                              |        |
|                              +--------+
|         b.mp4                |
|                              |
|                              |
|                              |
|                              |
|                              |
+------------------------------+

At t=15s, I want the video to look like this:

+------------------------------+--------+
|                              |        |
|                              |        |
|                              | a.mp4  |
|                              |        |
|                              +--------+
|         b.mp4                |        |
|                              |        |
|                              | c.mp4  |
|                              |        |
|                              +--------+
|                              |
+------------------------------+

And at t=20s until the end, I want the video to look like this:

+------------------------------+--------+
|                              |        |
|                              |        |
|                              | a.mp4  |
|                              |        |
|                              +--------+
|         c.mp4                |
|                              |
|                              |
|                              |
|                              |
|                              |
+------------------------------+

Ideally there would be some animated transitions between the states, but that's not essential.

I have found two possible approaches that might work, but I'm not sure what the best one is. The first is using filters to acheive the result, but I'm not sure if it will cope well with (a) the changing layouts and (b) keeping the audio without any artefacts when the layout changes.

The other approach I thought of would be exporting all frames to images, building new frames with imagemagick, and then layering the new frames on top of the audio like in this blog post.

Any suggestions on if either of these approaches is better, or any alternatives? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment