Skip to content

Instantly share code, notes, and snippets.

@maitrungduc1410
Last active September 2, 2023 08:54
Show Gist options
  • Star 18 You must be signed in to star a gist
  • Fork 6 You must be signed in to fork a gist
  • Save maitrungduc1410/1768ce3b369bf04a229ab043201c7197 to your computer and use it in GitHub Desktop.
Save maitrungduc1410/1768ce3b369bf04a229ab043201c7197 to your computer and use it in GitHub Desktop.
Bash scripts to create VOD HLS stream with ffmpeg using GPU
#!/usr/bin/env bash
START_TIME=$SECONDS
set -e
echo "-----START GENERATING HLS STREAM-----"
# Usage create-vod-hls-gpu.sh SOURCE_FILE [OUTPUT_NAME]
[[ ! "${1}" ]] && echo "Usage: create-vod-hls.sh SOURCE_FILE [OUTPUT_NAME]" && exit 1
# comment/add lines here to control which renditions would be created
renditions=(
# resolution bitrate audio-rate
"426x240 400k 128k"
"640x360 800k 192k"
"842x480 1400k 256k"
"1280x720 2800k 256k"
"1920x1080 5000k 320k"
)
segment_target_duration=10 # try to create a new segment every X seconds
max_bitrate_ratio=1.07 # maximum accepted bitrate fluctuations
rate_monitor_buffer_ratio=1.5 # maximum buffer size between bitrate conformance checks
#########################################################################
source="${1}"
target="${2}"
if [[ ! "${target}" ]]; then
target="${source##*/}" # leave only last component of path
target="${target%.*}" # strip extension
fi
mkdir -p ${target}
# ----CUSTOM----
sourceResolution="$(ffprobe -v error -select_streams v:0 -show_entries stream=width,height -of csv=s=x:p=0 ${source})"
# echo ${sourceResolution}
arrIN=(${sourceResolution//x/ })
sourceWidth="${arrIN[0]}"
sourceHeight="${arrIN[1]}"
echo ${sourceWidth}
echo ${sourceHeight}
sourceAudioBitRate="$(ffprobe -v error -select_streams a:0 -show_entries stream=bit_rate -of csv=s=x:p=0 ${source})"
sourceAudioBitRateFormatted=$((sourceAudioBitRate / 1000))
# ----END CUSTOM----
key_frames_interval="$(echo `ffprobe ${source} 2>&1 | grep -oE '[[:digit:]]+(.[[:digit:]]+)? fps' | grep -oE '[[:digit:]]+(.[[:digit:]]+)?'`*2 | bc || echo '')"
key_frames_interval=${key_frames_interval:-50}
key_frames_interval=$(echo `printf "%.1f\n" $(bc -l <<<"$key_frames_interval/10")`*10 | bc) # round
key_frames_interval=${key_frames_interval%.*} # truncate to integer
# static parameters that are similar for all renditions
static_params="-c:a aac -ar 48000 -c:v h264_nvenc -rc:v vbr_hq -cq:v 19 -profile:v main -sc_threshold 0"
static_params+=" -g ${key_frames_interval} -keyint_min ${key_frames_interval} -hls_time ${segment_target_duration}"
static_params+=" -hls_playlist_type vod"
# misc params
misc_params="-hide_banner -y -hwaccel cuvid -vsync 0"
master_playlist="#EXTM3U
#EXT-X-VERSION:3
"
cmd=""
resolutionValid=0
prevHeight=0
for rendition in "${renditions[@]}"; do
# drop extraneous spaces
rendition="${rendition/[[:space:]]+/ }"
# rendition fields
resolution="$(echo ${rendition} | cut -d ' ' -f 1)"
bitrate="$(echo ${rendition} | cut -d ' ' -f 2)"
audiorate="$(echo ${rendition} | cut -d ' ' -f 3)"
audioBitRateFormatted=${audiorate%?} # remove "k" at the last index
# take highest possible audio bit rate
if [ $audioBitRateFormatted -gt $sourceAudioBitRateFormatted ]; then
audiorate=${sourceAudioBitRateFormatted}k
fi
# calculated fields
width="$(echo ${resolution} | grep -oE '^[[:digit:]]+')"
height="$(echo ${resolution} | grep -oE '[[:digit:]]+$')"
maxrate="$(echo "`echo ${bitrate} | grep -oE '[[:digit:]]+'`*${max_bitrate_ratio}" | bc)"
bufsize="$(echo "`echo ${bitrate} | grep -oE '[[:digit:]]+'`*${rate_monitor_buffer_ratio}" | bc)"
bandwidth="$(echo ${bitrate} | grep -oE '[[:digit:]]+')000"
name="${height}p"
if [ $sourceHeight -le $prevHeight ]; then
echo "video source has height smaller than output height (${height})"
break
fi
widthParam=0
heightParam=0
if [ $(((width / sourceWidth) * sourceHeight)) -gt $height ]; then
widthParam=-2
heightParam=$height
else
widthParam=$width
heightParam=-2
fi
cmd+=" ${static_params} -vf hwupload_cuda,scale_npp=w=${widthParam}:h=${heightParam}"
cmd+=" -b:v ${bitrate} -maxrate ${maxrate%.*}k -bufsize ${bufsize%.*}k -b:a ${audiorate}"
cmd+=" -hls_segment_filename ${target}/${name}_%03d.ts ${target}/${name}.m3u8"
# add rendition entry in the master playlist
master_playlist+="#EXT-X-STREAM-INF:BANDWIDTH=${bandwidth},RESOLUTION=${resolution}\n${name}.m3u8\n"
resolutionValid=1
prevHeight=${height}
done
if [ $resolutionValid -eq 1 ]; then
# start conversion
echo -e "Executing command:\n/content/ffmpeg ${misc_params} -i ${source} ${cmd}\n"
/content/ffmpeg ${misc_params} -i ${source} ${cmd}
# create master playlist file
echo -e "${master_playlist}" > ${target}/playlist.m3u8
echo "Done - encoded HLS is at ${target}/"
else
echo "Video source is too small"
exit 1
fi
ELAPSED_TIME=$(($SECONDS - $START_TIME))
echo "Elapsed time: ${ELAPSED_TIME}"
echo "-----FINISH GENERATING HLS STREAM-----"
@maitrungduc1410
Copy link
Author

maitrungduc1410 commented Jul 24, 2020

Hi @xubmuajkub,

For support with AMD GPU, you may refer to this thread, basically the solution is updating my script with the following:

  • change hwaccel to auto
  • chang -c:v to h264_amf

I haven't tested yet ('cuz I don't have any PC that has AMD 😅😅), but searching for awhile and most them have same solution as described above.

@xubmuajkub
Copy link

xubmuajkub commented Sep 24, 2020

Do you have anyway to fix this issue?

[AVHWDeviceContext @ 0x7ffff3582f40] Could not initialize the CUDA driver API
[AVFilterGraph @ 0x7ffff340b4e0] Error initializing filter 'hwupload_cuda'
Error reinitializing filters!
Failed to inject frame into filter network: Unknown error occurred

Additional Information:
I use GTX 1060 on WSL on Windows 10

@xubmuajkub
Copy link

It would be good if we can have Windows .bat file

@maitrungduc1410
Copy link
Author

maitrungduc1410 commented Sep 25, 2020

Do you have anyway to fix this issue?

[AVHWDeviceContext @ 0x7ffff3582f40] Could not initialize the CUDA driver API
[AVFilterGraph @ 0x7ffff340b4e0] Error initializing filter 'hwupload_cuda'
Error reinitializing filters!
Failed to inject frame into filter network: Unknown error occurred

Additional Information:
I use GTX 1060 on WSL on Windows 10

  • Make sure you have installed nv-codec-headers (see the beginning part of my instructions)
  • Note that latest version of ffmpeg may not work for all graphics card, for me I use version 3.4, you may want to choose another version (I think from 3.0-3.4 is good)

This issue is described here.

I don't have time at the moment to convert my script to .bat, you're free to create your own.

@marcfon
Copy link

marcfon commented Oct 8, 2020

I've got this one almost running correctly on OSX with an Radeon Pro 560X.

I made two tweaks as suggested here.
static_params="-c:a aac -ar 48000 -c:v h264_videotoolbox -profile:v main -crf 19 -sc_threshold 0"
misc_params="-hide_banner -y -hwaccel auto -vsync 0"

Now it only breaks breaks on this parameter '-vf scale=w=${widthParam}:h=${heightParam}'.

When I remove it from the command it starts rendering but of course the resulting video is in the wrong scale.

@maitrungduc1410
Copy link
Author

I've got this one almost running correctly on OSX with an Radeon Pro 560X.

I made two tweaks as suggested here.
static_params="-c:a aac -ar 48000 -c:v h264_videotoolbox -profile:v main -crf 19 -sc_threshold 0"
misc_params="-hide_banner -y -hwaccel auto -vsync 0"

Now it only breaks breaks on this parameter '-vf scale=w=${widthParam}:h=${heightParam}'.

When I remove it from the command it starts rendering but of course the resulting video is in the wrong scale.

I only run this script on Ubuntu so I'm not sure it works same in other platforms.

Thanks for your report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment