audio waveform rendering

# audio waveformss

## situation

- we use audiosurfer.js, which renders the waveform on the client
- currently, v1 does not support media fragment URI queries like `?t=10,15` and we must fall back to `#t=10,15`
- this means that the data for the entire audio file must be transferred to and processed on the client

## SOLUTIONS

### Enable media query in v1 servers

- Server could support `?t=10,15` URI query and return only the requested fraction of the data
- pro: will save the system tons of bandwidth in general
- con: still slow when drawing waveforms on the client when fragments are large, e.g 30 minutes, 1 hour etc
- con: still no support for Internet Explorer
- efforts: low for client, unclear for v1 side
* https://www.w3.org/TR/media-frags/#URIquery-URIfragment

### implement a waveform rendering service

- A nodejs server that renders waveforms and saves PNG images or even just textfiles
- pro: most efficient solution
- pro: no change on v1 side
- pro: works in all browsers
- massive boost in client performance and user experience 
- efforts: medium to high, for both client and server sides

nodejs:
* https://developers.soundcloud.com/blog/waveforms-let-s-talk-about-them
* https://github.com/katspaugh/wavesurfer.js/issues/2
other:
* http://www.bbc.co.uk/rd/blog/2013/10/audio-waveforms


## WORKAROUNDS

Biggest pro for all workarounds is the reltaively low effort needed to implement them

### 1) never render waveforms, use v1 keyframe images

The client displays a single keyframe image generated by v1 at a time, and updates it every 30 seconds

- effort: low for client, none for server
- con: the keyframe images are small, they will either be scaled up and look blurry or stay small and centered
- con: no playhead position relative to the waveform, they waveform does not correspond with the timeline at all
- con: not exactly what was promised

### 2) use v1 keyframe images and stitch them together

The client loads all the keyframe images from v1 for a given range, then places then next to each other and scales them
so that the composite image corresponds with the timeline

- effort: medium for client, none for server
- pro: the waveform corresponds with the timeline
- pro: only stretched vertically, doesn't look as blurry

### 3) do not render waveforms initially, simple

Client displays a static image or just nothing but a play-button initially.
Once playback is requested, downloading begins
Once the entire file is loaded, the playback starts and the waveform is rendered

- effort: low for client side, none for server side
- con: no playback of fragment before entire file is laoded
- con: no way to show preview waveforms
- con: still no support in Internet Explorer
- con: potential bandwidth waste for small fragments

### 4) do not display waveforms intially, advanced

Client performs two requests in parallel:
one request or quick playbacks start using just a fragment like `#t=10,15`
one request for waveform rendering, loading the entire file
Playback needs to either switch from the simple audio element to the one with the entire file,
or both audio elements are playing, but the one with the waveform is set to volume=0 and is only used for showing the waveform

- effort medium to high for client, none for server
- pro: better UX than the simple workaround, playback starts earlier
- con: potential glitch when switching playback between the two audio elements
- con: complicated solution
- all other cons from #3) remain valid

### 5) A combination of initially displaying v1 keyframes and later rendered waveforms

This is a quite complicatied solution, combining multiple workarounds and possibly ending up
with same amount of required efforts as a proper solution.

It would initially display either a single v1 keyframe and update every 30s (easy), or the stitched-together keyframes (harder),
use the simple media fragment player for playback, but also load the entire file in the background and then
render the waveform once possible, switching players or doing the muted playback trick and just replace the displayed images by the waveform

- effort: medium to high for client, none for server
- con: complicated and potentially buggy