Skip to content

Instantly share code, notes, and snippets.

@selfawaresoup
Last active May 23, 2022 21:51
Show Gist options
  • Save selfawaresoup/0dbb330b5ab5dbdb059fd46b81880ff2 to your computer and use it in GitHub Desktop.
Save selfawaresoup/0dbb330b5ab5dbdb059fd46b81880ff2 to your computer and use it in GitHub Desktop.
Parallel youtube-dl with tmux

to get a list of URLs from a video listing page:

Array.from(document.querySelectorAll('#items a#video-title')).map(a => a.href).join('\n')

put that into a urls file, or gather the URLs some other way

then follow the instructions at https://github.com/dandv/convert-chrome-cookies-to-netscape-format for getting youtube cookies in netscape format in netscape-cookies.txt.

install tmux

run ./download.sh

Check the tmux session for errors (sometimes youtube-dl fails for strange reasons), if needed, you can manually restart the failed tasks

/**
* @file Convert cookies copy/pasted from Chrome's Application -> Storage -> Cookies -> [domain] table,
* into the Netscape cookies format used by tools like `curl` or `youtube-dl`.
*
* from: https://github.com/dandv/convert-chrome-cookies-to-netscape-format
*/
const fs = require('fs');
const filename = process.argv[2];
if (!filename) {
console.error(`Usage: node convert-cookies.js <file-with-cookies-copy-pasted-from-Chrome.txt> > netscape-cookies.txt`);
console.error();
console.error(`Make sure to replace <file-with-cookies-copy-pasted-from-Chrome.txt> with the name of the\nfile in which you copy/pasted the cookies from Chrome's Application -> Storage -> Cookies.`);
console.error(`\nThen, pass the 'netscape-cookies.txt' file to 'curl' or 'youtube-dl' or any other tool\nthat reads cookies in the Netscape cookies format.`);
process.exit(1);
}
const content = fs.readFileSync(filename, 'utf8');
const cookies = content.split('\n');
console.log('# Netscape HTTP Cookie File');
for (const cookie of cookies) {
let [name, value, domain, path, expiration, /* size */, httpOnly] = cookie.split('\t');
if (!name)
continue;
if (domain.charAt(0) !== '.')
domain = '.' + domain;
httpOnly = httpOnly === '✓' ? 'TRUE' : 'FALSE'
if (expiration === 'Session')
expiration = new Date(Date.now() + 86400 * 1000);
expiration = Math.trunc(new Date(expiration).getTime() / 1000);
console.log([domain, 'TRUE', path, httpOnly, expiration, name, value].join('\t'));
}
#!/usr/bin/env bash
FORMAT="137"
NAME_TEMPLATE="%(upload_date)s - %(title)s.%(ext)s"
COOKIE=$(cat cookie.txt)
SESSION="youtube-dl"
mkdir -p output
tmux new-session -d -s "$SESSION"
I=0
for URL in $(cat urls); do
if ((I > 0)); then
tmux split-window -t $SESSION
fi
tmux select-layout tiled
COMMAND="youtube-dl -f \"$FORMAT\" -o \"output/$NAME_TEMPLATE\" --cookies netscape-cookies.txt \"$URL\"; wait"
tmux send-keys -t $SESSION "$COMMAND" Enter
((I++))
done
tmux attach-session -t $SESSION
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment