Skip to content

Instantly share code, notes, and snippets.

@maitrungduc1410
Last active October 26, 2023 11:05
  • Star 55 You must be signed in to star a gist
  • Fork 25 You must be signed in to fork a gist
Star You must be signed in to star a gist
Embed
What would you like to do?
Bash scripts to create VOD HLS stream with ffmpeg (Extended version)
#!/usr/bin/env bash
START_TIME=$SECONDS
set -e
echo "-----START GENERATING HLS STREAM-----"
# Usage create-vod-hls.sh SOURCE_FILE [OUTPUT_NAME]
[[ ! "${1}" ]] && echo "Usage: create-vod-hls.sh SOURCE_FILE [OUTPUT_NAME]" && exit 1
# comment/add lines here to control which renditions would be created
renditions=(
# resolution bitrate audio-rate
"426x240 400k 128k"
"640x360 800k 128k"
"842x480 1400k 192k"
"1280x720 2800k 192k"
"1920x1080 5000k 256k"
)
segment_target_duration=10 # try to create a new segment every 10 seconds
max_bitrate_ratio=1.07 # maximum accepted bitrate fluctuations
rate_monitor_buffer_ratio=1.5 # maximum buffer size between bitrate conformance checks
#########################################################################
source="${1}"
target="${2}"
if [[ ! "${target}" ]]; then
target="${source##*/}" # leave only last component of path
target="${target%.*}" # strip extension
fi
mkdir -p ${target}
# ----CUSTOM----
sourceResolution="$(ffprobe -v error -select_streams v:0 -show_entries stream=width,height -of csv=s=x:p=0 ${source})"
# echo ${sourceResolution}
arrIN=(${sourceResolution//x/ })
sourceWidth="${arrIN[0]}"
sourceHeight="${arrIN[1]}"
echo ${sourceWidth}
echo ${sourceHeight}
sourceAudioBitRate="$(ffprobe -v error -select_streams a:0 -show_entries stream=bit_rate -of csv=s=x:p=0 ${source})"
sourceAudioBitRateFormatted=$((sourceAudioBitRate / 1000))
# ----END CUSTOM----
key_frames_interval="$(echo `ffprobe ${source} 2>&1 | grep -oE '[[:digit:]]+(.[[:digit:]]+)? fps' | grep -oE '[[:digit:]]+(.[[:digit:]]+)?'`*2 | bc || echo '')"
key_frames_interval=${key_frames_interval:-50}
key_frames_interval=$(echo `printf "%.1f\n" $(bc -l <<<"$key_frames_interval/10")`*10 | bc) # round
key_frames_interval=${key_frames_interval%.*} # truncate to integer
# static parameters that are similar for all renditions
static_params="-c:a aac -ar 48000 -c:v h264 -profile:v main -crf 19 -sc_threshold 0"
static_params+=" -g ${key_frames_interval} -keyint_min ${key_frames_interval} -hls_time ${segment_target_duration}"
static_params+=" -hls_playlist_type vod"
# misc params
misc_params="-hide_banner -y"
master_playlist="#EXTM3U
#EXT-X-VERSION:3
"
cmd=""
resolutionValid=0
prevHeight=0
for rendition in "${renditions[@]}"; do
# drop extraneous spaces
rendition="${rendition/[[:space:]]+/ }"
# rendition fields
resolution="$(echo ${rendition} | cut -d ' ' -f 1)"
bitrate="$(echo ${rendition} | cut -d ' ' -f 2)"
audiorate="$(echo ${rendition} | cut -d ' ' -f 3)"
audioBitRateFormatted=${audiorate%?} # remove "k" at the last index
# take highest possible audio bit rate
if [ $audioBitRateFormatted -gt $sourceAudioBitRateFormatted ]; then
audiorate=${sourceAudioBitRateFormatted}k
fi
# calculated fields
width="$(echo ${resolution} | grep -oE '^[[:digit:]]+')"
height="$(echo ${resolution} | grep -oE '[[:digit:]]+$')"
maxrate="$(echo "`echo ${bitrate} | grep -oE '[[:digit:]]+'`*${max_bitrate_ratio}" | bc)"
bufsize="$(echo "`echo ${bitrate} | grep -oE '[[:digit:]]+'`*${rate_monitor_buffer_ratio}" | bc)"
bandwidth="$(echo ${bitrate} | grep -oE '[[:digit:]]+')000"
name="${height}p"
if [ $sourceHeight -le $prevHeight ]; then
echo "video source has height smaller than output height (${height})"
break
fi
widthParam=0
heightParam=0
if [ $(((width / sourceWidth) * sourceHeight)) -gt $height ]; then
widthParam=-2
heightParam=$height
else
widthParam=$width
heightParam=-2
fi
cmd+=" ${static_params} -vf scale=w=${widthParam}:h=${heightParam}"
cmd+=" -b:v ${bitrate} -maxrate ${maxrate%.*}k -bufsize ${bufsize%.*}k -b:a ${audiorate}"
cmd+=" -hls_segment_filename ${target}/${name}_%03d.ts ${target}/${name}.m3u8"
# add rendition entry in the master playlist
master_playlist+="#EXT-X-STREAM-INF:BANDWIDTH=${bandwidth},RESOLUTION=${resolution}\n${name}.m3u8\n"
resolutionValid=1
prevHeight=${height}
done
if [ $resolutionValid -eq 1 ]; then
# start conversion
echo -e "Executing command:\nffmpeg ${misc_params} -i ${source} ${cmd}\n"
ffmpeg ${misc_params} -i ${source} ${cmd}
# create master playlist file
echo -e "${master_playlist}" > ${target}/playlist.m3u8
echo "Done - encoded HLS is at ${target}/"
else
echo "Video source is too small"
exit 1
fi
ELAPSED_TIME=$(($SECONDS - $START_TIME))
echo "Elapsed time: ${ELAPSED_TIME}"
echo "-----FINISH GENERATING HLS STREAM-----"
@fivethreeo
Copy link

Had a issue with this script because I use a charset with comma separator.

Fixed it like this:

key_frames_interval="$(echo `ffprobe ${source} 2>&1 | grep -oE '[[:digit:]]+(.[[:digit:]]+)? fps' | grep -oE '[[:digit:]]+(.[[:digit:]]+)?'`*2 | bc || echo '')"
key_frames_interval=${key_frames_interval:-50}
key_frames_interval=${key_frames_interval/,/\.}
key_frames_interval=$(bc -l <<<"$key_frames_interval/10")
key_frames_interval=${key_frames_interval/\./,}
key_frames_interval=$(echo `printf "%.1f\n" ${key_frames_interval}`)
key_frames_interval=${key_frames_interval/,/\.}
key_frames_interval=$(echo ${key_frames_interval}*10| bc) # round)
key_frames_interval=${key_frames_interval%.*} # truncate to integer

@dontfreakout
Copy link

Had a issue with this script because I use a charset with comma separator.

Fixed it like this:

key_frames_interval="$(echo `ffprobe ${source} 2>&1 | grep -oE '[[:digit:]]+(.[[:digit:]]+)? fps' | grep -oE '[[:digit:]]+(.[[:digit:]]+)?'`*2 | bc || echo '')"
key_frames_interval=${key_frames_interval:-50}
key_frames_interval=${key_frames_interval/,/\.}
key_frames_interval=$(bc -l <<<"$key_frames_interval/10")
key_frames_interval=${key_frames_interval/\./,}
key_frames_interval=$(echo `printf "%.1f\n" ${key_frames_interval}`)
key_frames_interval=${key_frames_interval/,/\.}
key_frames_interval=$(echo ${key_frames_interval}*10| bc) # round)
key_frames_interval=${key_frames_interval%.*} # truncate to integer

@fivethreeo there is actually an easier fix, see my comment

@dlobjoie
Copy link

Thanks for this script, it does a really good job.
I'm working on a project where, besides creating the HLS stream, I also want to encrypt it.
I already managed to do it with another ffmpeg script I made, but with this one, I don't see where I could insert the -hls_key_file keyinfo.drm parameter (where keyinfo.drm is a text file that contains my AES128 encryption key). Could you guide me on this ?

@xubmuajkub
Copy link

@dlobjoie This is what I've done for my script. Hope this might help you.

cmd+=" ${static_params} -vf scale=w=${widthParam}:h=${heightParam}"
cmd+=" -b:v ${bitrate} -maxrate ${maxrate%.*}k -bufsize ${bufsize%.*}k -b:a ${audiorate}"
# Add this line for encryption
cmd+=" -hls_key_info_file KEY_INFO_FILE"
cmd+=" -hls_segment_filename ${target}/${name}_%03d.html ${target}/${name}.m3u8"

@dlobjoie
Copy link

@xubmuajkub Thank you very much. I had been looking for a long time.
Did you also automate the addition of the path to the key in the creation of the playlist?
If yes how did you do it?

@xubmuajkub
Copy link

@dlobjoie
Copy link

@dlobjoie Just follow the step they do here https://hlsbook.net/how-to-encrypt-hls-video-with-ffmpeg/

Thanks @xubmuajkub that's where I started my HLS adventure a few months ago :)

@xubmuajkub
Copy link

@dlobjoie in my case, I just want a simple encryption so I only use 1 file for every video.

@AhmedAbouelkher
Copy link

Hi @maitrungduc1410
Thanks for your amazing script.

Is there a way to add FRAME-RATE, CODECS, and AUDIO for every resolution?

@mkammes
Copy link

mkammes commented Oct 23, 2022

In the case where we want multiple renditions at the same frame size - but at different bitrates - the script doesn't account for it, and the m3u8 points to the same manifest file, e.g.
`#EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:BANDWIDTH=400000,RESOLUTION=384x216
216p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=700000,RESOLUTION=512x288
288p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1100000,RESOLUTION=720x404
404p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1750000,RESOLUTION=720x404
404p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2750000,RESOLUTION=1280x720
720p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=3500000,RESOLUTION=1280x720
720p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=4300000,RESOLUTION=1920x1080
1080p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=5800000,RESOLUTION=1920x1080
1080p.m3u8

`

Is there a way to solve this; perhaps outputting each rendition and playlist in a separate folder?

Thanks!

@maitrungduc1410
Copy link
Author

Hi @maitrungduc1410 Thanks for your amazing script.

Is there a way to add FRAME-RATE, CODECS, and AUDIO for every resolution?

FRAME-RATE: not sure
CODECS: now supported: https://trac.ffmpeg.org/ticket/8904
AUDIO: seems possible: https://stackoverflow.com/questions/60017730/create-hls-streamable-audio-file-from-mp3

@maitrungduc1410
Copy link
Author

In the case where we want multiple renditions at the same frame size - but at different bitrates - the script doesn't account for it, and the m3u8 points to the same manifest file, e.g. `#EXTM3U #EXT-X-VERSION:3 #EXT-X-STREAM-INF:BANDWIDTH=400000,RESOLUTION=384x216 216p.m3u8 #EXT-X-STREAM-INF:BANDWIDTH=700000,RESOLUTION=512x288 288p.m3u8 #EXT-X-STREAM-INF:BANDWIDTH=1100000,RESOLUTION=720x404 404p.m3u8 #EXT-X-STREAM-INF:BANDWIDTH=1750000,RESOLUTION=720x404 404p.m3u8 #EXT-X-STREAM-INF:BANDWIDTH=2750000,RESOLUTION=1280x720 720p.m3u8 #EXT-X-STREAM-INF:BANDWIDTH=3500000,RESOLUTION=1280x720 720p.m3u8 #EXT-X-STREAM-INF:BANDWIDTH=4300000,RESOLUTION=1920x1080 1080p.m3u8 #EXT-X-STREAM-INF:BANDWIDTH=5800000,RESOLUTION=1920x1080 1080p.m3u8

`

Is there a way to solve this; perhaps outputting each rendition and playlist in a separate folder?

Thanks!

my script is only for improving and fixing bugs based on the original script.

for your purpose you may need to write your own logics

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment