A script to download multiple videos in parallel

readme.md

The purpose of this gist is to provide a script for downloading videos off the internet in parallel.

There are also single-video implementations at the bottom of the python file, this is useful if you just want API examples, or if you can't modify the path variable to allow the yt-dlp cli.

Setup

You will need to install yt-dlp, it's command line, and have it on your path for the script to work. You should be able to do all of this with:

pip install yt-dlp

or

sudo pip3 install yt-dlp

or

python -m pip install yt-dlp

Usage

To run the script modify download.py and include your videos to download in the videos variable. From there just run the script using python download.py or python3 download.py

Other sites besides youtube

Lots of other sites are supported by yt-dlp. If you are watching a stream and unable to get the URL to work, try using plugins to determine the location of the stream file like:

• https://addons.mozilla.org/en-CA/firefox/addon/hls-stream-detector/

download.py

# A script to download many videos in parallel
import subprocess                   # Used to instantiate yt-dlp processes to download
from typing import List
from multiprocessing import Process # Used to parrallel process download subprocesses

# Confirm dependencies are installed
try:
    import yt_dlp # Needed for suprocessing
except ImportError:
    print("Youtube DLP not installed, please install with:\n\tpip install yt-dlp\n\t\tor\n\tsudo pip3 install yt-dlp")
    exit()

videos = [
        # Format is
        # [URL, title]
        ["https://www.youtube.com/watch?v=aqz-KE-bpKQ","Big Brick Bunny.mp4"],
]

def download_video(title:str, url:str):
    """A function to download a video

    Parameters
    ----------
    title : str
        The filename to export the video to

    url : str
        The url to download from
    """
    if not title.endswith(".mp4"):
        title = f"{title}.mp4"
    with subprocess.Popen(["yt-dlp", "-o", title, url]) as p:
        print(p)

def run_downloads_in_parallel(args:List[List[str]]):
    """The primary entrypoint into the script that actually runs the download processes

    Notes
    -----
    - From: https://stackoverflow.com/a/56138825/11602400

    Parameters
    ----------
    args : List[List[str]]
        The arguments to pass to download_video() in format [[title, url],[title, url]]
    """
    running_tasks = [Process(target=download_video, args=arg) for arg in args]
    for running_task in running_tasks:
        running_task.start()
    for running_task in running_tasks:
        running_task.join()

if __name__ == '__main__':
    run_downloads_in_parallel([(title, url) for url, title in videos])

#################### Pytube (single download) ####################
# try:
#     from pytube import YouTube
# except ImportError:
#     print("Pytube not installed, please install with:\n\tpip install pytube3\n\t\tor\n\tsudo pip3 install pytube3")
#     exit()
# for link in ["https://www.youtube.com/watch?v=4BZoqVoxHys",
#              "https://www.youtube.com/watch?v=jOoy3UfKDgM"]:
#     YouTube(link).streams.first().download()
#     # yt = YouTube(link)
#     # yt.streams.filter(progressive=True, file_extension='mp4').order_by('resolution').desc().first().download()

#################### YoutubeDL (single download) ####################
# try:
#     from yt_dlp  import YoutubeDL
# except ImportError:
#     print("Youtube DLP not installed, please install with:\n\tpip install yt-dlp\n\t\tor\n\tsudo pip3 install yt-dlp")
#     exit()
# with YoutubeDL() as ydl:
#     ydl.download(videos)