Skip to content

Instantly share code, notes, and snippets.

@maitrungduc1410
Last active September 2, 2023 08:54
  • Star 18 You must be signed in to star a gist
  • Fork 6 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save maitrungduc1410/1768ce3b369bf04a229ab043201c7197 to your computer and use it in GitHub Desktop.
Bash scripts to create VOD HLS stream with ffmpeg using GPU
#!/usr/bin/env bash
START_TIME=$SECONDS
set -e
echo "-----START GENERATING HLS STREAM-----"
# Usage create-vod-hls-gpu.sh SOURCE_FILE [OUTPUT_NAME]
[[ ! "${1}" ]] && echo "Usage: create-vod-hls.sh SOURCE_FILE [OUTPUT_NAME]" && exit 1
# comment/add lines here to control which renditions would be created
renditions=(
# resolution bitrate audio-rate
"426x240 400k 128k"
"640x360 800k 192k"
"842x480 1400k 256k"
"1280x720 2800k 256k"
"1920x1080 5000k 320k"
)
segment_target_duration=10 # try to create a new segment every X seconds
max_bitrate_ratio=1.07 # maximum accepted bitrate fluctuations
rate_monitor_buffer_ratio=1.5 # maximum buffer size between bitrate conformance checks
#########################################################################
source="${1}"
target="${2}"
if [[ ! "${target}" ]]; then
target="${source##*/}" # leave only last component of path
target="${target%.*}" # strip extension
fi
mkdir -p ${target}
# ----CUSTOM----
sourceResolution="$(ffprobe -v error -select_streams v:0 -show_entries stream=width,height -of csv=s=x:p=0 ${source})"
# echo ${sourceResolution}
arrIN=(${sourceResolution//x/ })
sourceWidth="${arrIN[0]}"
sourceHeight="${arrIN[1]}"
echo ${sourceWidth}
echo ${sourceHeight}
sourceAudioBitRate="$(ffprobe -v error -select_streams a:0 -show_entries stream=bit_rate -of csv=s=x:p=0 ${source})"
sourceAudioBitRateFormatted=$((sourceAudioBitRate / 1000))
# ----END CUSTOM----
key_frames_interval="$(echo `ffprobe ${source} 2>&1 | grep -oE '[[:digit:]]+(.[[:digit:]]+)? fps' | grep -oE '[[:digit:]]+(.[[:digit:]]+)?'`*2 | bc || echo '')"
key_frames_interval=${key_frames_interval:-50}
key_frames_interval=$(echo `printf "%.1f\n" $(bc -l <<<"$key_frames_interval/10")`*10 | bc) # round
key_frames_interval=${key_frames_interval%.*} # truncate to integer
# static parameters that are similar for all renditions
static_params="-c:a aac -ar 48000 -c:v h264_nvenc -rc:v vbr_hq -cq:v 19 -profile:v main -sc_threshold 0"
static_params+=" -g ${key_frames_interval} -keyint_min ${key_frames_interval} -hls_time ${segment_target_duration}"
static_params+=" -hls_playlist_type vod"
# misc params
misc_params="-hide_banner -y -hwaccel cuvid -vsync 0"
master_playlist="#EXTM3U
#EXT-X-VERSION:3
"
cmd=""
resolutionValid=0
prevHeight=0
for rendition in "${renditions[@]}"; do
# drop extraneous spaces
rendition="${rendition/[[:space:]]+/ }"
# rendition fields
resolution="$(echo ${rendition} | cut -d ' ' -f 1)"
bitrate="$(echo ${rendition} | cut -d ' ' -f 2)"
audiorate="$(echo ${rendition} | cut -d ' ' -f 3)"
audioBitRateFormatted=${audiorate%?} # remove "k" at the last index
# take highest possible audio bit rate
if [ $audioBitRateFormatted -gt $sourceAudioBitRateFormatted ]; then
audiorate=${sourceAudioBitRateFormatted}k
fi
# calculated fields
width="$(echo ${resolution} | grep -oE '^[[:digit:]]+')"
height="$(echo ${resolution} | grep -oE '[[:digit:]]+$')"
maxrate="$(echo "`echo ${bitrate} | grep -oE '[[:digit:]]+'`*${max_bitrate_ratio}" | bc)"
bufsize="$(echo "`echo ${bitrate} | grep -oE '[[:digit:]]+'`*${rate_monitor_buffer_ratio}" | bc)"
bandwidth="$(echo ${bitrate} | grep -oE '[[:digit:]]+')000"
name="${height}p"
if [ $sourceHeight -le $prevHeight ]; then
echo "video source has height smaller than output height (${height})"
break
fi
widthParam=0
heightParam=0
if [ $(((width / sourceWidth) * sourceHeight)) -gt $height ]; then
widthParam=-2
heightParam=$height
else
widthParam=$width
heightParam=-2
fi
cmd+=" ${static_params} -vf hwupload_cuda,scale_npp=w=${widthParam}:h=${heightParam}"
cmd+=" -b:v ${bitrate} -maxrate ${maxrate%.*}k -bufsize ${bufsize%.*}k -b:a ${audiorate}"
cmd+=" -hls_segment_filename ${target}/${name}_%03d.ts ${target}/${name}.m3u8"
# add rendition entry in the master playlist
master_playlist+="#EXT-X-STREAM-INF:BANDWIDTH=${bandwidth},RESOLUTION=${resolution}\n${name}.m3u8\n"
resolutionValid=1
prevHeight=${height}
done
if [ $resolutionValid -eq 1 ]; then
# start conversion
echo -e "Executing command:\n/content/ffmpeg ${misc_params} -i ${source} ${cmd}\n"
/content/ffmpeg ${misc_params} -i ${source} ${cmd}
# create master playlist file
echo -e "${master_playlist}" > ${target}/playlist.m3u8
echo "Done - encoded HLS is at ${target}/"
else
echo "Video source is too small"
exit 1
fi
ELAPSED_TIME=$(($SECONDS - $START_TIME))
echo "Elapsed time: ${ELAPSED_TIME}"
echo "-----FINISH GENERATING HLS STREAM-----"
@maitrungduc1410
Copy link
Author

maitrungduc1410 commented Jul 20, 2020

This is GPU-version that has same features with my previous script (non-GPU)

What's in this version?

This script run ffmpeg that uses NVIDIA GPU to reduce significant time to generate HLS stream.

Prerequisite

  • Make sure you're having NVIDIA GPU card and driver installed (run nvidia-smito check)
  • FFMPEG build from source that supports GPU (see my instructions on how to build below)
  • sudo privilege

Build FFmpeg from source with GPU support

This instructions is for Ubuntu, other Linux distributions should be same

First we need to install NVIDIA API Header (required before building ffmpeg).

Change to some directory before proceed

git clone https://git.videolan.org/git/ffmpeg/nv-codec-headers.git
cd nv-codec-headers && sudo make install

Next install required dependencies:

#install dependencies
sudo apt-get update -qq && sudo apt-get -y install \
  autoconf \
  automake \
  build-essential \
  cmake \
  git-core \
  libass-dev \
  libfreetype6-dev \
  libgnutls28-dev \
  libtool \
  libvorbis-dev \
  pkg-config \
  texinfo \
  wget \
  yasm \
  zlib1g-dev \
  bc

sudo apt-get install nasm libx264-dev libx265-dev libnuma-dev libvpx-dev libfdk-aac-dev libmp3lame-dev libopus-dev

Finally clone and build FFmpeg from source. Change to some directory before proceed:

# clone ffmpeg
git clone https://git.ffmpeg.org/ffmpeg.git

# checkout to version 3.4. DO NOT use latest version at the moment (4.0) because it's incompatible with many NVIDIA drivers
cd ffmpeg && git checkout 567c20f78109588620c57696ab39ff314fe12d6c

./configure \
  --enable-gpl \
  --enable-gnutls \
  --enable-libass \
  --enable-libfdk-aac \
  --enable-libfreetype \
  --enable-libmp3lame \
  --enable-libopus \
  --enable-libvorbis \
  --enable-libvpx \
  --enable-libx264 \
  --enable-libx265 \
  --enable-cuda-sdk \
  --enable-cuvid \
  --enable-nvenc \
  --enable-nonfree \
  --enable-libnpp \
  --extra-cflags=-I/usr/local/cuda/include \
  --extra-ldflags=-L/usr/local/cuda/lib64 \
  --nvccflags="-gencode arch=compute_75,code=sm_75 -O2"

make -j 10

After finishing, you'll see an ffmpeg binary generated in your current working directory (NOTICE this). Now you're ready to go with FFMPEG with GPU support.

How to run

Look back my script on line 120:

# Change this line
/content/ffmpeg ${misc_params} -i ${source} ${cmd}

# To this
<path_to_the_ffmpeg_binary> ${misc_params} -i ${source} ${cmd}

Save it and now run this command:

bash create-vod-hls-gpu.sh video.mp4

How fast is it?

Tested in environment:

  • Ubuntu 18.04
  • 16GB RAM + 2.2GHz CPU
  • One Tesla K80 graphics card (11.5GB Memory-----NVIDIA-SMI 450.51.05-----Driver Version: 418.67-----CUDA Version: 10.1 )

Test video:

  • Duration: 120 mins
  • Resolution: 1080p (1920x800)

Total time to process reduced from 14 hours down to just 1,5 hours 😄😄

Working example

https://colab.research.google.com/drive/1nME0I9AHvJu-tRnT3c1SOa40wgs9RIqD?usp=sharing

@xubmuajkub
Copy link

Can you add AMD GPU support?

@maitrungduc1410
Copy link
Author

maitrungduc1410 commented Jul 24, 2020

Hi @xubmuajkub,

For support with AMD GPU, you may refer to this thread, basically the solution is updating my script with the following:

  • change hwaccel to auto
  • chang -c:v to h264_amf

I haven't tested yet ('cuz I don't have any PC that has AMD 😅😅), but searching for awhile and most them have same solution as described above.

@xubmuajkub
Copy link

xubmuajkub commented Sep 24, 2020

Do you have anyway to fix this issue?

[AVHWDeviceContext @ 0x7ffff3582f40] Could not initialize the CUDA driver API
[AVFilterGraph @ 0x7ffff340b4e0] Error initializing filter 'hwupload_cuda'
Error reinitializing filters!
Failed to inject frame into filter network: Unknown error occurred

Additional Information:
I use GTX 1060 on WSL on Windows 10

@xubmuajkub
Copy link

It would be good if we can have Windows .bat file

@maitrungduc1410
Copy link
Author

maitrungduc1410 commented Sep 25, 2020

Do you have anyway to fix this issue?

[AVHWDeviceContext @ 0x7ffff3582f40] Could not initialize the CUDA driver API
[AVFilterGraph @ 0x7ffff340b4e0] Error initializing filter 'hwupload_cuda'
Error reinitializing filters!
Failed to inject frame into filter network: Unknown error occurred

Additional Information:
I use GTX 1060 on WSL on Windows 10

  • Make sure you have installed nv-codec-headers (see the beginning part of my instructions)
  • Note that latest version of ffmpeg may not work for all graphics card, for me I use version 3.4, you may want to choose another version (I think from 3.0-3.4 is good)

This issue is described here.

I don't have time at the moment to convert my script to .bat, you're free to create your own.

@marcfon
Copy link

marcfon commented Oct 8, 2020

I've got this one almost running correctly on OSX with an Radeon Pro 560X.

I made two tweaks as suggested here.
static_params="-c:a aac -ar 48000 -c:v h264_videotoolbox -profile:v main -crf 19 -sc_threshold 0"
misc_params="-hide_banner -y -hwaccel auto -vsync 0"

Now it only breaks breaks on this parameter '-vf scale=w=${widthParam}:h=${heightParam}'.

When I remove it from the command it starts rendering but of course the resulting video is in the wrong scale.

@maitrungduc1410
Copy link
Author

I've got this one almost running correctly on OSX with an Radeon Pro 560X.

I made two tweaks as suggested here.
static_params="-c:a aac -ar 48000 -c:v h264_videotoolbox -profile:v main -crf 19 -sc_threshold 0"
misc_params="-hide_banner -y -hwaccel auto -vsync 0"

Now it only breaks breaks on this parameter '-vf scale=w=${widthParam}:h=${heightParam}'.

When I remove it from the command it starts rendering but of course the resulting video is in the wrong scale.

I only run this script on Ubuntu so I'm not sure it works same in other platforms.

Thanks for your report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment