Skip to content

Instantly share code, notes, and snippets.

@Kudusch
Last active November 22, 2023 11:09
Show Gist options
  • Star 39 You must be signed in to star a gist
  • Fork 15 You must be signed in to fork a gist
  • Save Kudusch/8175167 to your computer and use it in GitHub Desktop.
Save Kudusch/8175167 to your computer and use it in GitHub Desktop.
<?php
/*
Made by Kudusch (blog.kudusch.de, kudusch.de, @Kudusch)
---------
DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
Version 2, December 2004
Copyright (C) 2004 Sam Hocevar <sam@hocevar.net>
Everyone is permitted to copy and distribute verbatim or modified
copies of this license document, and changing it is allowed as long
as the name is changed.
DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. You just DO WHAT THE FUCK YOU WANT TO.
---------
# How to use
- go to http://mediathek.daserste.de, open with iPad User Agent
- navigate to the desired program, get link to the master.m3u8 file from source (in the <video>-tag)
- run the script with url to the master.m3u8 file as argument (e.g. php get_tshls 'http://http://hls.daserste.de/i/videoportal/Film/c_380000/386059/ios.smil/master.m3u8'
- wait for the script to download and merge all media files; every 10 second part is about 3 MB (when it's done, it will output the runtime)
- if necessary, convert the *.ts file with a media converter eg. handbrake to a *.mp4 file
- enjoy
*/
//runtime
$startTime = microtime(get_as_float);
//get url from input
$url = $argv[1];
//get stream with highest bandwith
$streamUrl = getHighBandwidthStream($url);
//get array of all links to *.ts files
$list = getHlsFiles($streamUrl);
//make new directory
if (!is_dir('files')) {
mkdir('files');
}
//download all files from array, name with 3 leading zeros
//if file is longer than 166.5 minutes, adjust str_pad params
$n = 1;
foreach ($list as $key) {
$number = str_pad($n, 3, "0", STR_PAD_LEFT);
print_r($n." ");
file_put_contents("files/part.".$number.".ts", fopen($key, 'r'));
$n++;
}
//merge files and delte parts
sleep(10);
mergeFiles('files');
//echo part numbers and runtime for debugging
echo("\nRun in ".(microtime(get_as_float)-$startTime)." seconds.");
//input: string, output: string
function getHighBandwidthStream($masterUrl) {
//get content of master.m3u8
$ch = curl_init($masterUrl);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
$result = curl_exec($ch);
curl_close($ch);
//return link to second last stream (https://developer.apple.com/library/ios/documentation/networkinginternet/conceptual/streamingmediaguide/FrequentlyAskedQuestions/FrequentlyAskedQuestions.html#//apple_ref/doc/uid/TP40008332-CH103-SW1)
$result = split("#", $result);
for ($i = 0; $i < 2; $i++) {
array_shift($result);
}
$length = count($result);
$result = split("\n", $result[$length-2]);
return $result[1];
}
//input: string, output: array
function getHlsFiles($streamUrl) {
//get content of *.m3u8 file
$ch = curl_init($streamUrl);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
$raw = curl_exec($ch);
curl_close($ch);
//remove comments and unnecessary data
$list_raw = split("\n", $raw);
for ($i = 0; $i < 5; $i++) {
array_shift($list_raw);
}
for ($i = 0; $i < 2; $i++) {
array_pop($list_raw);
}
//extract file links
$list = array();
$i = 1;
foreach ($list_raw as $key) {
if($i%2 == 0) {
array_push($list, $key);
}
$i++;
}
//return array
return $list;
}
function mergeFiles($dirName) {
//get all *.ts files in directory
if ($handle = opendir($dirName)) {
while (false !== ($file = readdir($handle))) {
if (strpos($file, ".ts") !== false) {
$fileList = $fileList." files/".$file;
}
}
closedir($handle);
}
//join and remove parts
$shellScript = "cat ".substr($fileList, 1)." >> movie.ts";
shell_exec($shellScript);
shell_exec("rm -r files");
}
?>
@solidv2
Copy link

solidv2 commented Nov 11, 2015

when I run the script it says :
'cat is not recognized as an internal or external command, operable program or batch file.'
'rm is not recognized as an internal or external command, operable program or batch file.'

@ryancwalsh
Copy link

I found that using FFMpeg (for free) on the Mac Terminal command line worked perfectly to consolidate the tiny .ts files into one big mp4 video file. Example: ffmpeg -i http://embed.wistia.com/deliveries/blahBlahYadaYada.m3u8 -c copy -bsf:a aac_adtstoasc output.mp4 See: http://stackoverflow.com/questions/32528595/ffmpeg-mp4-from-http-live-streaming-m3u8-file

@jimdays2788
Copy link

I am trying to download the video from:
https://thevideobee.to/rhcdmrzk4zm7.html
Which has 248 one MB segments.
The url of these 248 segments is (number near end increments from 1 to 248):
https://fsc.thevideobee.to:444/hls/exfplmti7sfr4navd5wkli7sqa4j65suzuyad4q5pgjqd6cryu2pqnyb4nlq/seg-1-v1-a1.ts
https://fsc.thevideobee.to:444/hls/exfplmti7sfr4navd5wkli7sqa4j65suzuyad4q5pgjqd6cryu2pqnyb4nlq/seg-248-v1-a1.ts
Question: how can I download all 248 segments and join into one file?

@JSzaszvari
Copy link

@ryancwalsh - That snippet is amazing, You have saved me countless hours of trying to figure this out.

Tried a bunch of scripts that did nothing except spew a bunch of ts files out that didnt work. Your command downloaded and joined the whole video in less than 10 seconds.

You are a true legend

@jewuwu
Copy link

jewuwu commented Apr 19, 2021

Trying to get this working in Linux Mint 20.0 Cinnamon
(Linux 5.8.0-50-generic #56~20.04.1-Ubuntu x86_64 GNU/Linux)

I installed php7.4
(these packages)
php-common
php7.4-cli
php7.4-common
php7.4-json
php7.4-opcache
php7.4-readline

Ran your script as recommended.

Got these errors while running the script:

PHP Warning: Use of undefined constant get_as_float - assumed 'get_as_float' (this will throw an Error in a future version of PHP) in /home/hulk/test/get_tshls on line 34
PHP Fatal error: Uncaught Error: Call to undefined function curl_init() in /home/hulk/test/get_tshls:68
Stack trace:
#0 /home/hulk/test/get_tshls(39): getHighBandwidthStream()
#1 {main}
thrown in /home/hulk/test/get_tshls on line 68

I commented out the runtime measurement but still get stuck with the "curl_init()" function issue.

Any ideas how to fix this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment