Skip to content

Instantly share code, notes, and snippets.

@mrbar42
Last active November 16, 2024 18:32
Show Gist options
  • Save mrbar42/ae111731906f958b396f30906004b3fa to your computer and use it in GitHub Desktop.
Save mrbar42/ae111731906f958b396f30906004b3fa to your computer and use it in GitHub Desktop.
bash scripts to create VOD HLS stream with ffmpeg almighty (tested on Linux and OS X)

running:

bash create-vod-hls.sh beach.mkv

will produce:

    beach/
      |- playlist.m3u8
      |- 360p.m3u8
      |- 360p_001.ts
      |- 360p_002.ts
      |- 480p.m3u8
      |- 480p_001.ts
      |- 480p_002.ts
      |- 720p.m3u8
      |- 720p_001.ts
      |- 720p_002.ts
      |- 1080p.m3u8
      |- 1080p_001.ts
      |- 1080p_002.ts  

origin: http://docs.peer5.com/guides/production-ready-hls-vod/

#!/usr/bin/env bash
set -e
# Usage create-vod-hls.sh SOURCE_FILE [OUTPUT_NAME]
[[ ! "${1}" ]] && echo "Usage: create-vod-hls.sh SOURCE_FILE [OUTPUT_NAME]" && exit 1
# comment/add lines here to control which renditions would be created
renditions=(
# resolution bitrate audio-rate
# "426x240 400k 64k"
"640x360 800k 96k"
"842x480 1400k 128k"
"1280x720 2800k 128k"
"1920x1080 5000k 192k"
)
segment_target_duration=4 # try to create a new segment every X seconds
max_bitrate_ratio=1.07 # maximum accepted bitrate fluctuations
rate_monitor_buffer_ratio=1.5 # maximum buffer size between bitrate conformance checks
#########################################################################
source="${1}"
target="${2}"
if [[ ! "${target}" ]]; then
target="${source##*/}" # leave only last component of path
target="${target%.*}" # strip extension
fi
mkdir -p ${target}
key_frames_interval="$(echo `ffprobe ${source} 2>&1 | grep -oE '[[:digit:]]+(.[[:digit:]]+)? fps' | grep -oE '[[:digit:]]+(.[[:digit:]]+)?'`*2 | bc || echo '')"
key_frames_interval=${key_frames_interval:-50}
key_frames_interval=$(echo `printf "%.1f\n" $(bc -l <<<"$key_frames_interval/10")`*10 | bc) # round
key_frames_interval=${key_frames_interval%.*} # truncate to integer
# static parameters that are similar for all renditions
static_params="-c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0"
static_params+=" -g ${key_frames_interval} -keyint_min ${key_frames_interval} -hls_time ${segment_target_duration}"
static_params+=" -hls_playlist_type vod"
# misc params
misc_params="-hide_banner -y"
master_playlist="#EXTM3U
#EXT-X-VERSION:3
"
cmd=""
for rendition in "${renditions[@]}"; do
# drop extraneous spaces
rendition="${rendition/[[:space:]]+/ }"
# rendition fields
resolution="$(echo ${rendition} | cut -d ' ' -f 1)"
bitrate="$(echo ${rendition} | cut -d ' ' -f 2)"
audiorate="$(echo ${rendition} | cut -d ' ' -f 3)"
# calculated fields
width="$(echo ${resolution} | grep -oE '^[[:digit:]]+')"
height="$(echo ${resolution} | grep -oE '[[:digit:]]+$')"
maxrate="$(echo "`echo ${bitrate} | grep -oE '[[:digit:]]+'`*${max_bitrate_ratio}" | bc)"
bufsize="$(echo "`echo ${bitrate} | grep -oE '[[:digit:]]+'`*${rate_monitor_buffer_ratio}" | bc)"
bandwidth="$(echo ${bitrate} | grep -oE '[[:digit:]]+')000"
name="${height}p"
cmd+=" ${static_params} -vf scale=w=${width}:h=${height}:force_original_aspect_ratio=decrease"
cmd+=" -b:v ${bitrate} -maxrate ${maxrate%.*}k -bufsize ${bufsize%.*}k -b:a ${audiorate}"
cmd+=" -hls_segment_filename ${target}/${name}_%03d.ts ${target}/${name}.m3u8"
# add rendition entry in the master playlist
master_playlist+="#EXT-X-STREAM-INF:BANDWIDTH=${bandwidth},RESOLUTION=${resolution}\n${name}.m3u8\n"
done
# start conversion
echo -e "Executing command:\nffmpeg ${misc_params} -i ${source} ${cmd}"
ffmpeg ${misc_params} -i ${source} ${cmd}
# create master playlist file
echo -e "${master_playlist}" > ${target}/playlist.m3u8
echo "Done - encoded HLS is at ${target}/"
@Feujo
Copy link

Feujo commented Jul 22, 2020

Thanks for the script. But when I run this, here's my error message.

./create-vod-hls.sh 022505043294641312470435.mp4

[libx264 @ 0x55a878bcb800] [Eval @ 0x7fff9d53be00] Undefined constant or missing '(' in 'keyint_min'
[libx264 @ 0x55a878bcb800] Unable to parse option value "-keyint_min"
[libx264 @ 0x55a878bcb800] Error setting option g to value -keyint_min.
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[aac @ 0x55a878bca840] Qavg: 119.779
[aac @ 0x55a878bca840] 2 frames left in the queue on closing
[aac @ 0x55a878b9c080] Qavg: 119.779
[aac @ 0x55a878b9c080] 2 frames left in the queue on closing
[aac @ 0x55a878bcf800] Qavg: 119.779
[aac @ 0x55a878bcf800] 2 frames left in the queue on closing
[aac @ 0x55a878ba8e40] Qavg: 119.779
[aac @ 0x55a878ba8e40] 2 frames left in the queue on closing

@seenickcode
Copy link

It seems this script only outputs the filename on the next line after each #EXT-X-STREAM-INF line and this seems to 'break' any m3u8 playlist file because the entire URL to the file has to exist, no? (reference from Apple https://developer.apple.com/documentation/http_live_streaming/example_playlists_for_http_live_streaming/creating_a_master_playlist)

On other words, I cannot get this to work with this example:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:BANDWIDTH=800000,RESOLUTION=640x360
360p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2800000,RESOLUTION=1280x720
720p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=5000000,RESOLUTION=1920x1080
1080p.m3u8

It seems to only work if I cite the URLs to the files:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:BANDWIDTH=800000,RESOLUTION=640x360
http://mydomain.com/360p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2800000,RESOLUTION=1280x720
http://mydomain.com/720p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=5000000,RESOLUTION=1920x1080
http://mydomain.com/1080p.m3u8

@kashkashio
Copy link

kashkashio commented Aug 19, 2020

This is my extended version of this script that supports auto generate bitrates based on video source resolution, and handle cases when video after scaling has dimensions are not divisible by 2 that will cause error at run time.

Link: https://gist.github.com/maitrungduc1410/9c640c61a7871390843af00ae1d8758e

You are a life saver ! Works great !
Tho thanks to OP for first answer !

Tho i wonder for exampe how dose sfari determines which resultion to take from the playlist?
By screen size? by network speed? latency etc... ?

P.S
How can i force it to split to even more chunks ? it dose like 3 chunks for 22 seconds video
Can i force it into more ?

@mindjek07
Copy link

mindjek07 commented Aug 31, 2020

Hi, thanks for you script. I need help with other problem. I have one video with multiple quality and multiple audios. The audio have the same quality. How i can modify script for this.
Example
video1 --> 1080 | 720 | 480
audio --> english | spanish (same quality)
It possible?
Thanks!

Replace Line Number 39 :

static_params="-c:a copy -map 0:a -map 0:v -c:v h264 -profile:v main -crf 19 -sc_threshold 0

-c:a copy >> copy audio without convert
-c:v copy >> copy video without convert
-c:av copy >> copy audio and video without convert
-map 0:a >> all audio
-map 0:v >> all videos
-map 0:s >> all subtitles

@kutanoor
Copy link

kutanoor commented Sep 24, 2020

when i try to convert the video it gives me error
bash create-vod-hls.sh vid.mp4 Executing command: ffmpeg -hide_banner -y -i vid.mp4 -c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 60 -keyint_min 60 -hls_time 4 -hls_playlist_type vod -vf scale=w=640:h=360:force_original_aspect_ratio=decrease -b:v 800k -maxrate 856k -bufsize 1200k -b:a 96k -hls_segment_filename vid/360p_%03d.ts vid/360p.m3u8 -c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 60 -keyint_min 60 -hls_time 4 -hls_playlist_type vod -vf scale=w=842:h=480:force_original_aspect_ratio=decrease -b:v 1400k -maxrate 1498k -bufsize 2100k -b:a 128k -hls_segment_filename vid/480p_%03d.ts vid/480p.m3u8 -c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 60 -keyint_min 60 -hls_time 4 -hls_playlist_type vod -vf scale=w=1280:h=720:force_original_aspect_ratio=decrease -b:v 2800k -maxrate 2996k -bufsize 4200k -b:a 128k -hls_segment_filename vid/720p_%03d.ts vid/720p.m3u8 -c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 60 -keyint_min 60 -hls_time 4 -hls_playlist_type vod -vf scale=w=1920:h=1080:force_original_aspect_ratio=decrease -b:v 5000k -maxrate 5350k -bufsize 7500k -b:a 192k -hls_segment_filename vid/1080p_%03d.ts vid/1080p.m3u8 Unrecognized option 'hls_playlist_type'. Error splitting the argument list: Option not found

@throne1986
Copy link

Thanks for the script. But when I run this, here's my error message.

./create-vod-hls.sh 022505043294641312470435.mp4

[libx264 @ 0x55a878bcb800] [Eval @ 0x7fff9d53be00] Undefined constant or missing '(' in 'keyint_min'
[libx264 @ 0x55a878bcb800] Unable to parse option value "-keyint_min"
[libx264 @ 0x55a878bcb800] Error setting option g to value -keyint_min.
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[aac @ 0x55a878bca840] Qavg: 119.779
[aac @ 0x55a878bca840] 2 frames left in the queue on closing
[aac @ 0x55a878b9c080] Qavg: 119.779
[aac @ 0x55a878b9c080] 2 frames left in the queue on closing
[aac @ 0x55a878bcf800] Qavg: 119.779
[aac @ 0x55a878bcf800] 2 frames left in the queue on closing
[aac @ 0x55a878ba8e40] Qavg: 119.779
[aac @ 0x55a878ba8e40] 2 frames left in the queue on closing

did you solve this? I have the same error

@Aniket-Singla
Copy link

Aniket-Singla commented Dec 29, 2020

hi, is there any way to include codec info as well in manifest?
Currently it is like:

#EXT-X-STREAM-INF:BANDWIDTH=5000000,RESOLUTION=1920x1080```

What i want is :

#EXT-X-STREAM-INF:BANDWIDTH=767744,AVERAGE-BANDWIDTH=720351,CODECS="avc1.4d401f,mp4a.40.5",RESOLUTION=640x480,FRAME-RATE=29.970

Is there a way to get CODECS in the manifest? If I use the following command:

ffmpeg -y -i ~/work/sample-videos/flowers.mp4  \
  -preset slow -g 48 -sc_threshold 0 \
  -map 0:0 -map 0:1 -map 0:0 -map 0:1 \
  -s:v:0 640x360 -c:v:0 libx264 -b:v:0 365k \
  -s:v:1 960x540 -c:v:1 libx264 -b:v:1 2000k  \
  -c:a copy \
  -var_stream_map "v:0,a:0 v:1,a:1" \
  -master_pl_name master.m3u8 \
  -f hls -hls_time 6 -hls_list_size 0 \
  -hls_segment_filename "v%v/fileSequence%d.ts" \
  v%v/prog_index.m3u8

CODECS are auto included for h264(for above command) but the above command does not support resolution based naming of segments.

Thanks

@DravitLochan
Copy link

DravitLochan commented Jan 28, 2021

I tried to get all the segments for 10 seconds except for the first one which will be of 5 seconds by passing-hls_init_time 5 -hls_time 10 -hls_playlist_type VOD but seems like hls_time is being ignored all together, i.e. all the segments are of 5 seconds instead of 10. Am I doing something wrong? Can someone help?

@deman777
Copy link

deman777 commented Feb 18, 2021

Hi @mrbar42. Thank you for great script! Why resolution is 842x480 but by standard it is 854x480? from here https://support.google.com/youtube/answer/6375112?co=GENIE.Platform%3DDesktop&hl=en

@Kick4U2
Copy link

Kick4U2 commented May 29, 2021

Hi @mrbar42. Thank you for great script! Why resolution is 842x480 but by standard it is 854x480? from here https://support.google.com/youtube/answer/6375112?co=GENIE.Platform%3DDesktop&hl=en

There are many iterations of 480p; use the one relevant to your most prevalent client device.
https://en.wikipedia.org/wiki/480p#Resolutions

@cheldernunes
Copy link

Thanks for the script. But when I run this, here's my error message.
./create-vod-hls.sh 022505043294641312470435.mp4
[libx264 @ 0x55a878bcb800] [Eval @ 0x7fff9d53be00] Undefined constant or missing '(' in 'keyint_min'
[libx264 @ 0x55a878bcb800] Unable to parse option value "-keyint_min"
[libx264 @ 0x55a878bcb800] Error setting option g to value -keyint_min.
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[aac @ 0x55a878bca840] Qavg: 119.779
[aac @ 0x55a878bca840] 2 frames left in the queue on closing
[aac @ 0x55a878b9c080] Qavg: 119.779
[aac @ 0x55a878b9c080] 2 frames left in the queue on closing
[aac @ 0x55a878bcf800] Qavg: 119.779
[aac @ 0x55a878bcf800] 2 frames left in the queue on closing
[aac @ 0x55a878ba8e40] Qavg: 119.779
[aac @ 0x55a878ba8e40] 2 frames left in the queue on closing

did you solve this? I have the same error

I change L:35 to awk

key_frames_interval="$(awk  "BEGIN { rounded = sprintf(\"%.0f\", ${key_frames_interval}); print rounded }")"

@masterofthesith
Copy link

check this single comment multi output => https://github.com/masterofthesith/ffmpeg

@shreyesh0610
Copy link

How about having differntial segments accoding to the duration of the original video. For example, having shorter segments for the first 10% duration of video and then longer segments for the rest 90%?

@ciniexpress
Copy link

ciniexpress commented Feb 10, 2022

I was looking for a code to convert video files in a directory where it has multiple subfolders for each video file. I want to run a batch code so it can convert the video files and save them where the original file was there.

Example:

: tree movies/
movies/
├── subfolder1
│ └── movie1.mp4
├── subfolder2
│ └── movie2.mp4
└── subfolder3
└── movie3.mp4

And so on.

I want the output to be the same as my main structure but in .m3u8 format.

I have used a code where it can convert each movie manually but for some reason, I can use the .m3u8 file or link only in that server I can't make it work all over the internet.

for i in .mp4; do
ffmpeg -i "$i" -c copy -map 0 -hls_time 3 -hls_list_size 0 -f hls "${i%.
}.hls"
done

Thanks

@lnxct
Copy link

lnxct commented Jul 1, 2022

@DravitLochan i am also thinking like you to initial hls video size size is small like first few are divided in 2 sec chunk and then after 10 sec we make it to 5 sec chunk.
so have you achieved this solution ?
so something we can do 5 chunk are in increasing order like 1s,2s,3s,4s,5s and then all remain are 5s/10sec chunk

@appumistri
Copy link

appumistri commented Oct 31, 2022

Isn't it easier to create a master playlist file using FFMPEG itself?
Using -master_pl_name param

Ref: https://stackoverflow.com/a/71985380/6513289

@sgtcoder
Copy link

sgtcoder commented Jul 8, 2023

Great script to get started, doesn't play in Safari though, might be the codec this script is using.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment