SCTE-35 PID presence on HLS TS Segments

scte35-pid-on-ts-segment.py

import datetime
import os
from concurrent.futures import ThreadPoolExecutor
from requests import get
import m3u8
from threefive import decode
import time


PLAYLIST = ""

downloader = ThreadPoolExecutor(max_workers=10)

def download(uri, output_dir):
	date = datetime.datetime.now().strftime("%Y_%m_%d_%H_%M_%S")
	file_path = os.path.join(output_dir, date + "_" + uri.split('/')[-1])
	with open(file_path, "wb") as f:
		response = get(uri)
		f.write(response.content)
	print("decoding ", file_path)
	decode(file_path)

if __name__ == "__main__":
	while True:
		master_manifest = m3u8.load(PLAYLIST)
		for manifest in master_manifest.playlists:
			rendition_manifest = m3u8.load(manifest.absolute_uri)
			for segment in rendition_manifest.segments:
				downloader.submit(download, segment.absolute_uri, "/tmp/")
		time.sleep(6)

Example of SCTE-35 present on .ts file:

decoding  /tmp/2021_01_28_10_46_36_pluto-ads-automation-hls-clean-720p_2_20210128T154618_2_00094.ts
{
  "info_section": {
    "table_id": "0xfc",
    "section_syntax_indicator": false,
    "private": false,
    "reserved": "0x3",
    "section_length": 37,
    "protocol_version": 0,
    "encrypted_packet": false,
    "encryption_algorithm": 0,
    "pts_adjustment": 2.3,
    "cw_index": "0x0",
    "tier": "0xfff",
    "splice_command_length": 20,
    "splice_command_type": 5,
    "descriptor_loop_length": 0,
    "crc": "0xe174bb06"
  },
  "command": {
    "command_length": 20,
    "name": "Splice Insert",
    "time_specified_flag": true,
    "pts_time": 44822.946222,
    "break_auto_return": true,
    "break_duration": 119.986533,
    "splice_event_id": 1611848779,
    "splice_event_cancel_indicator": false,
    "out_of_network_indicator": true,
    "program_splice_flag": true,
    "duration_flag": true,
    "splice_immediate_flag": false,
    "unique_program_id": 1,
    "avail_num": 1,
    "avail_expected": 1
  },
  "descriptors": [],
  "pid": 500,
  "program": 1,
  "pts": 44824.204444
}

Nice.

You made me realize that threefive.decode() should parse http and https urls directly.
As of threefive v.2.2.69, this works.

from threefive import decode

decode("https://futzu.com/xaa.ts")

that's awesome @futzu, thanks for adding this.

Check this out. AES segment decode with pyaes.
https://github.com/futzu/threefive/blob/master/examples/hls/hasp.py

nice stuff @futzu, the Stanza class would be simplified by a lot if the m3u8 lib was used for the parsing

Hey man you doing well?

That was my original plan, it would have been a lot easier and cleaner to use the m3u8 lib.

I submitted SCTE-35 patch a few months ago for m3u8 and I never got any reply.
This particular feed uses #EXT-X-SCTE35:CUE, which I dont think is supported by m3u8.

I thought you might want to try pyaes.
since m3u8 already has all the aes info parsed.

here is the bare bones version

import pyaes

with open('the_key_file','rb') as quay:
    key = (quay.read())
    iv= bytes.fromhex('5245E5D4EACB1B5C522064043A299565')
    mode = pyaes.AESModeOfOperationCBC(key,iv=iv)
    with open('segment.ts','rb') as infile:
        with open('decoded_segment.ts','wb') as outfile:
            pyaes.decrypt_stream(mode,infile,outfile)```

I did use m3u8 in this example/.
https://github.com/futzu/threefive/blob/master/examples/hls/threefive-m3u8.py

where is the PR you submitted? I can try and get people to look at it

globocom/m3u8#248

did you open the pull request or only opened an issue?

I only did the issue. .I know my issue and how to solve it, but I just don't know the m3u8 code well enough.

Ill do a pull request, give me a minute,

hey man. m3u8 needs a little love. I forked it. If you want, I'll add you the repo. Let's collaborate.

hey @futzu, m3u8 is actually pretty active, with a lot of people using it.. I think we can suggest things there and implement some PRs. What do you think? Also, if you want a more dynamic conversation, join http://video-dev.org, I'll be there!