HTTP PUT file uploads from Shaka Packager to Nginx

README.rst

Introduction

This is about the new HTTP PUT file upload feature of Shaka Packager, see also shaka-project/shaka-packager#149. The code is currently living in the http-upload branch.

Nginx configuration

The receiver is based on the native Nginx module "ngx_http_dav_module", it should well handle HTTP PUT requests with chunked transfer encoding like emitted by Shaka Packager.

The configuration is very simple:

server {
    listen 6767 default_server;

    access_log  /dev/stdout combined;
    error_log   /dev/stdout info;

    root /var/spool;
    location ~ ^/hls-live/(.+)$ {

        dav_methods PUT;
        create_full_put_path on;

        proxy_buffering off;
        client_max_body_size 20m;

    }

}

Running Shaka Packager

HTTP upload will get enabled by using an URL as the output path:

export BASE_URL=put+http://localhost:6767/hls-live

Then, run Shaka Packager as usual:

packager \
    "input=${PIPE},stream=audio,segment_template=${BASE_URL}/media/bigbuckbunny-audio-aac-\$Number%04d\$.aac,playlist_name=bigbuckbunny-audio.m3u8,hls_group_id=audio" \
    "input=${PIPE},stream=video,segment_template=${BASE_URL}/media/bigbuckbunny-video-h264-450-\$Number%04d\$.ts,playlist_name=bigbuckbunny-video-450.m3u8" \
    --io_block_size 65536 --fragment_duration 2 --segment_duration 2 \
    --time_shift_buffer_depth 3600 --preserved_segments_outside_live_window 7200 \
    --hls_master_playlist_output "${BASE_URL}/meta/bigbuckbunny.m3u8" \
    --hls_playlist_type LIVE \
    --vmodule=http_file=2,buffer_writer=1,ts_writer=2,packed_audio_writer=2,curl=3,libcurl=3

tracing-howto.rst

How to trace what's going on?

Watch the Nginx logs

nginx -p `pwd` -c nginx.conf -g "daemon off;"

127.0.0.1 - - [08/Nov/2018:20:01:23 +0000] "PUT /hls-live/media/bigbuckbunny-audio-aac-0001.aac HTTP/1.1" 204 0 "-" "shaka-packager-uploader/0.1"
127.0.0.1 - - [08/Nov/2018:20:01:23 +0000] "PUT /hls-live/meta/bigbuckbunny-video-450.m3u8 HTTP/1.1" 204 0 "-" "shaka-packager-uploader/0.1"
127.0.0.1 - - [08/Nov/2018:20:01:23 +0000] "PUT /hls-live/meta/bigbuckbunny-audio.m3u8 HTTP/1.1" 204 0 "-" "shaka-packager-uploader/0.1"
127.0.0.1 - - [08/Nov/2018:20:01:23 +0000] "PUT /hls-live/meta/bigbuckbunny.m3u8 HTTP/1.1" 204 0 "-" "shaka-packager-uploader/0.1"
127.0.0.1 - - [08/Nov/2018:20:01:24 +0000] "PUT /hls-live/media/bigbuckbunny-audio-aac-0002.aac HTTP/1.1" 204 0 "-" "shaka-packager-uploader/0.1"
127.0.0.1 - - [08/Nov/2018:20:01:24 +0000] "PUT /hls-live/meta/bigbuckbunny-video-450.m3u8 HTTP/1.1" 204 0 "-" "shaka-packager-uploader/0.1"
127.0.0.1 - - [08/Nov/2018:20:01:24 +0000] "PUT /hls-live/meta/bigbuckbunny-audio.m3u8 HTTP/1.1" 204 0 "-" "shaka-packager-uploader/0.1"
127.0.0.1 - - [08/Nov/2018:20:01:32 +0000] "PUT /hls-live/media/bigbuckbunny-video-h264-450-0001.ts HTTP/1.1" 204 0 "-" "shaka-packager-uploader/0.1"

Watch the network

ngrep -Wbyline -dlo port 6767