5shekel/big_d_ripper.md

## big_d_ripper.md

      
    Raw
  

              big_d_ripper.md
            
          
    User - postmortem (TLDR)
ok we are done, this is the powershell script.

written and tested on tami ubuntu machine.

powershell is a modern, cross platfrom shell
# adapt input directory and output directory, fit your operating system
# all movie scenes will be saved in output dir in a new folder (media name)

$indir='/mnt/Video/Au Service De La France - season 2/'
$outdir="au/"

Get-ChildItem $indir -Filter *.* | ForEach-Object {
    #next 5 lines are housekeeping, getting the filenames in order. 
    $input = $_.FullName
    $baseName = $_.BaseName.Replace(' ', '_')
    $outputFolder = $outdir + $baseName
    New-Item -ItemType Directory -Force -Path $outputFolder 
    $output = $outputFolder + '/' + $baseName

    #using scene detection
    # & ffmpeg -hide_banner -i "$input" -vf "scdet" -sn -map 0 -f segment -segment_format mp4 "${output}__%07d_detect.mp4"
    
    #using source keyframe data
    & ffmpeg -hide_banner -i "$input" -c copy -map 0 -segment_time 0.01 -f segment "${output}__%07d_key.mp4"

}
note the suggested source keyframe extraction method is much faster.
compare scenedetection using -vf "scdet"
Measure-Command { & ffmpeg -i "$infile" -vf "scdet" -map 0 -sn -f segment -segment_format mp4 "/tmp/$($input.BaseName)%03d.mp4"}
TotalSeconds      : ???

i didnt bother to wait for the  detector to finish parsing the 40 minutes clip.

it is processor heavy and might force re-encoding
keyframe did it in a few seconds.
Measure-Command { & ffmpeg -i "$infile" -c copy -map 0 -segment_time 0.01 -f segment "/tmp/$($input.BaseName)%03d.mp4"}
TotalSeconds      : 2.384013

bellow is the complete conversation with the gpt

User
time ./ffmpeg -f lavfi -i "$input,scdet=s=1:t=14" -vf "scale=160:-1,tile=6x85" -frames:v 1 -qscale:v 3 preview.jpg

ChatGPT
It seems like you're trying to use FFmpeg, a software suite to handle multimedia data, to create a preview image from a video file using scene detection. However, your command contains some issues that might prevent it from working as expected. Here's a revised version of your command:
time ffmpeg -i "$input" -vf "select='gt(scene,0.4)',scale=160:-1,tile=6x85" -frames:v 1 -q:v 3 preview.jpg
Here's what this command does:

time: This is a command that measures the total time the following command takes to execute.
ffmpeg: This is the main command for the FFmpeg program.
\-i "$input": This option specifies the input file. Replace "$input" with the path to your video file.
\-vf "select='gt(scene,0.4)',scale=160:\-1,tile=6x85": This option applies video filters to the input. In this case, it selects frames where the scene change is greater than 0.4 (you may want to adjust this value based on your specific video), scales the frames to a width of 160 pixels (maintaining the aspect ratio), and arranges the frames into a grid of 6 columns and 85 rows.
\-frames:v 1: This option specifies that only one video frame should be output. In this case, it means that only one image (which consists of a grid of frames) should be output.
\-q:v 3: This option sets the quality of the output image. The value ranges from 1 (highest quality) to 31 (lowest quality). Here, it's set to 3, which means the image should have quite high quality.
preview.jpg: This is the output file. The command will create a JPEG image with this name.

Please ensure that you replace "$input" with your actual video file path. Also, please adjust the scene change value, the

User
use the scdat info to split the input file

ChatGPT
If you want to split your video into segments at the points where scene changes occur, you can use the scdet filter with the segment muxer in FFmpeg.
Here's a basic example command:
ffmpeg -i "$input" -vf "scdet" -f segment -segment_time 5 -c copy "$output%03d.mp4"
In this command:

\-i "$input" specifies the input file. Replace "$input" with the path to your video file.
\-vf "scdet" applies the scene detection filter.
\-f segment specifies that the output should be segmented, meaning that it should be split into multiple files.
\-segment\_time 5 specifies the maximum segment duration in seconds. In this case, it's set to 5 seconds, but you can adjust this to your needs. Note: this option is not based on detected scene changes, it just splits the video every N seconds.
\-c copy specifies that the codec of the input file should be copied to the output files, meaning that no re-encoding should be performed. This can significantly speed up the process, but it may not always give the best results, especially if the input file uses a codec that doesn't support splitting.
"$output%03d.mp4" specifies the names of the output files. The %03d part will be replaced with the index of the segment, starting from 0 and formatted as a three-digit number. Replace "$output" with the path where you want the output files to be stored.

Please note that scene detection in FFmpeg is a complex process and the above command is a simplification. You may need to adjust the parameters or use additional filters to get the best results for your specific video. Also, FFmpeg doesn't currently support splitting videos exactly at scene changes out of the box, you might need to write a custom script to do that.

User
[vost#0:0/copy @ 0x55ce1be46240] Filtergraph 'scdet' was defined, but codec copy was selected.
Filtering and streamcopy cannot be used together.


ChatGPT
Yes, you're correct. The \-c copy option can't be used with filters because it tells FFmpeg to copy the input streams as is without any processing. When you're using filters like scdet, you need to re-encode the video.
Here is a simple example of how you can split the video into new scenes using the scdet filter:
ffmpeg -i "$input" -vf "scdet" -map 0 -f segment -segment_format mp4 "$output%03d.mp4"

In this command:

\-i "$input" specifies the input file. Replace "$input" with the path to your video file.
\-vf "scdet" applies the scene detection filter.
\-map 0 includes all streams from the input file in the output files.
\-f segment specifies that the output should be segmented, meaning that it should be split into multiple files.
\-segment\_format mp4 specifies the format of the segments. In this case, it's set to MP4, but you can adjust this to your needs.
"$output%03d.mp4" specifies the names of the output files. The %03d part will be replaced with the index of the segment, starting from 0 and formatted as a three-digit number. Replace "$output" with the path where you want the output files to be stored.

Please note that FFmpeg doesn't currently support splitting videos exactly at scene changes out of the box, you might need to write a custom script to process the output of the scdet filter and split the video accordingly.

User
run it on all files in folder,. output filename is input filename

ChatGPT
Assuming you're using a Unix-like shell such as bash, you can use a simple loop to process all files in a directory. Here's an example script:
#!/bin/bash

for input in /path/to/your/videos/*; do
    base=$(basename "$input")
    output="/path/to/your/output/${base%.*}"
    ffmpeg -i "$input" -vf "scdet" -map 0 -f segment -segment_format mp4 "${output}%03d.mp4"
done

In this script:

for input in /path/to/your/videos/\*; do starts a loop that iterates over all files in the directory /path/to/your/videos/.
base=$(basename "$input") extracts the filename from the path stored in the variable $input.
output="/path/to/your/output/${base%.\*}" specifies the output directory and filename (without the extension).
ffmpeg \-i "$input" \-vf "scdet" \-map 0 \-f segment \-segment\_format mp4 "${output}%03d.mp4" is the FFmpeg command that processes the current video file. It uses the scdet filter to detect scene changes and splits the video into multiple segments.
done ends the loop.

Remember to replace /path/to/your/videos/ and /path/to/your/output/ with the actual paths to your video files and the output directory, respectively.
Please note that FFmpeg doesn't currently support splitting videos exactly at scene changes out of the box, you might need to write a custom script to process the output of the scdet filter and split the video accordingly.

User
in powershell

ChatGPT
Here's how you can do this in PowerShell:
Get-ChildItem 'C:\path\to\your\videos\' -Filter *.mp4 | ForEach-Object {
    $input = $_.FullName
    $output = 'C:\path\to\your\output\' + $_.BaseName
    ffmpeg -i "$input" -vf "scdet" -map 0 -f segment -segment_format mp4 "${output}%03d.mp4"
}
This script does the following:

Get\-ChildItem 'C:\\path\\to\\your\\videos\\' \-Filter \*.mp4 gets all MP4 files in the specified directory.
| ForEach\-Object { ... } performs the following operations for each file:


$input = $\_.FullName sets the $input variable to the full path of the current file.
$output = 'C:\\path\\to\\your\\output\\' + $\_.BaseName sets the $output variable to the base name of the current file, prepended with the output directory path.
ffmpeg \-i "$input" \-vf "scdet" \-map 0 \-f segment \-segment\_format mp4 "${output}%03d.mp4" runs the FFmpeg command to split the video into segments at the detected scene changes.

Remember to replace 'C:\\path\\to\\your\\videos\\' and 'C:\\path\\to\\your\\output\\' with the actual paths to your videos and the output directory, respectively.
Please note that FFmpeg doesn't currently support splitting videos exactly at scene changes out of the box, you might need to write a custom script to process the output of the scdet filter and split the video accordingly.

User
make output files in new folder named after the avi basename

ChatGPT
Certainly! Here's how you can modify the script:
Get-ChildItem 'C:\path\to\your\videos\' -Filter *.avi | ForEach-Object {
    $input = $_.FullName
    $outputFolder = 'C:\path\to\your\output\' + $_.BaseName
    New-Item -ItemType Directory -Force -Path $outputFolder
    $output = $outputFolder + '\' + $_.BaseName
    ffmpeg -i "$input" -vf "scdet" -map 0 -f segment -segment_format mp4 "${output}%03d.mp4"
}
This script does the following:

Get\-ChildItem 'C:\\path\\to\\your\\videos\\' \-Filter \*.avi gets all AVI files in the specified directory.
| ForEach\-Object { ... } performs the following operations for each file:


$input = $\_.FullName sets the $input variable to the full path of the current file.
$outputFolder = 'C:\\path\\to\\your\\output\\' + $\_.BaseName sets the $outputFolder variable to a new folder named after the base name of the current file, prepended with the output directory path.
New\-Item \-ItemType Directory \-Force \-Path $outputFolder creates the new output folder, if it doesn't exist.
$output = $outputFolder + '\\' + $\_.BaseName sets the $output variable to the base name of the current file, prepended with the new output folder path.
ffmpeg \-i "$input" \-vf "scdet" \-map 0 \-f segment \-segment\_format mp4 "${output}%03d.mp4" runs the FFmpeg command to split the video into segments at the detected scene changes.

Remember to replace 'C:\\path\\to\\your\\videos\\' and 'C:\\path\\to\\your\\output\\' with the actual paths to your videos and the output directory, respectively.
Please note that FFmpeg doesn't currently support splitting videos exactly at scene changes out of the box, you might need to write a custom script to process the output of the scdet filter and split the video accordingly.

User
powershell replace spaces with underscore

ChatGPT
If you need to replace spaces with underscores in the filename in PowerShell, you can use the Replace() method of the string object. Here is how you can incorporate it into the script:
Get-ChildItem 'C:\path\to\your\videos\' -Filter *.avi | ForEach-Object {
    $input = $_.FullName
    $baseName = $_.BaseName.Replace(' ', '_')
    $outputFolder = 'C:\path\to\your\output\' + $baseName
    New-Item -ItemType Directory -Force -Path $outputFolder
    $output = $outputFolder + '\' + $baseName
    ffmpeg -i "$input" -vf "scdet" -map 0 -f segment -segment_format mp4 "${output}%03d.mp4"
}
Here, $baseName = $\_.BaseName.Replace(' ', '\_') replaces spaces with underscores in the base name of the file.

User
[sost#0:2 @ 0x558327946b00] Automatic encoder selection failed Default encoder for format segment (codec none) is probably disabled. Please choose an encoder manually.
[sost#0:2 @ 0x558327946b00] Error selecting an encoder


ChatGPT
The error message suggests that an encoder for the second stream (indexed at 0:2) couldn't be automatically selected. This typically occurs when the output format (in this case, segment) doesn't have a default encoder for the input stream's codec.
The video and audio codecs can be specified using \-c:v and \-c:a respectively. However, if your input file has additional streams (like subtitles, data, etc.), you might encounter this issue.
To handle this, you can specify encoders for these streams as well or choose to ignore them using the \-vn (no video), \-an (no audio), \-sn (no subtitle) or \-dn (no data) options. For example, if the problematic stream is a subtitle stream, you can choose to ignore it with \-sn.
Here's an example:
Get-ChildItem 'C:\path\to\your\videos\' -Filter *.avi | ForEach-Object {
    $input = $_.FullName
    $baseName = $_.BaseName.Replace(' ', '_')
    $outputFolder = 'C:\path\to\your\output\' + $baseName
    New-Item -ItemType Directory -Force -Path $outputFolder
    $output = $outputFolder + '\' + $baseName
    & ffmpeg -i "$input" -vf "scdet" -c:v libx264 -c:a aac -sn -map 0 -f segment -segment_format mp4 "${output}%03d.mp4"
}

In this script, \-c:v libx264 specifies the H.264 encoder for video, \-c:a aac specifies the AAC encoder for audio, and \-sn ignores subtitle streams. If the problematic stream is not a subtitle stream, you may need to adjust the command accordingly.

User
list streams in file

ChatGPT
You can use FFmpeg to list the streams in a video file using the \-hide\_banner option to suppress unnecessary information and the \-i option to specify the input file. Here's an example:
ffmpeg -hide_banner -i input.mp4

Replace input.mp4 with the path to your video file.
This command will output information about the input file, including a list of its streams. Each stream is listed with its index number (e.g., 0:0, 0:1, etc.), its type (video, audio, subtitle, etc.), and its codec, among other details.
In PowerShell, you can list the streams for all video files in a directory using a loop, like so:
Get-ChildItem 'C:\path\to\your\videos\' -Filter *.avi | ForEach-Object {
    $input = $_.FullName
    Write-Output "Processing $input"
    & ffmpeg -hide_banner -i "$input"
}
This script will print the name of each file before listing its streams. Replace 'C:\\path\\to\\your\\videos\\' with the actual path to your videos.

User
ok, that is good
$indir='/mnt/Video/Au Service De La France - season 2/'
$outdir="au/"

Get-ChildItem $indir -Filter *.* | ForEach-Object {
    $input = $_.FullName
    $baseName = $_.BaseName.Replace(' ', '_')
    $outputFolder = $outdir + $baseName
    New-Item -ItemType Directory -Force -Path $outputFolder 
    $output = $outputFolder + '/' + $baseName
    # $input 
    $output
    & ffmpeg -hide_banner -i "$input" -vf "scdet" -sn -map 0 -f segment -segment_format mp4 "${output}%03d.mp4"
}

ChatGPT
Great! Your script looks correct and should work as expected. It loops over every file in the $indir directory, creates a new directory for each file in the $outdir directory, and then uses FFmpeg to split each video file into segments at scene changes.
Here's a quick overview of what your script does:

Get\-ChildItem $indir \-Filter \*.\* | ForEach\-Object { ... } gets all files in the directory specified by $indir and performs the operations in the { ... } block for each file.
$input = $\_.FullName sets the $input variable to the full path of the current file.
$baseName = $\_.BaseName.Replace(' ', '\_') sets the $baseName variable to the base name of the current file, with spaces replaced by underscores.
$outputFolder = $outdir + $baseName sets the $outputFolder variable to a new folder named after $baseName in the $outdir directory.
New\-Item \-ItemType Directory \-Force \-Path $outputFolder creates the new output folder, if it doesn't exist already.
$output = $outputFolder + '/' + $baseName sets the $output variable to the base name of the current file, prepended with the new output folder path.
& ffmpeg \-hide\_banner \-i "$input" \-vf "scdet" \-sn \-map 0 \-f segment \-segment\_format mp4 "${output}%03d.mp4" runs the FFmpeg command to split the video into segments at the detected scene changes.

Please note that FFmpeg doesn't currently support splitting videos exactly at scene changes out of the box, you might need to write a custom script to process the output of the scdet filter and split the video accordingly. Also, make sure you have the necessary permissions to create directories and write files in the $outdir directory.

User
split a file using the input file keyframe information

ChatGPT
To split a video file into segments based on the keyframe information, you can use FFmpeg's segment muxer with the \-g option. The \-g option sets the group of picture (GOP) size, which is the distance between keyframes.
Here's an example of how you could use these options to split a video file into segments at every keyframe:
ffmpeg -i input.mp4 -c copy -map 0 -segment_time 0.01 -f segment output%03d.mp4
In this command:

\-i input.mp4 specifies the input file.
\-c copy \-map 0 tells FFmpeg to copy all streams from the input file to the output files.
\-segment\_time 0.01 sets the maximum duration of each segment to 0.01 seconds. Since the actual duration of each segment is the timestamp of the first keyframe that exceeds this value, this effectively causes each segment to end at a keyframe.
\-f segment specifies the segment muxer, which splits the input into multiple output files.
output%03d.mp4 is the filename pattern for the output files. The %03d is replaced with the segment number.

Please note that this command doesn't actually create a keyframe at scene changes. Creating a keyframe at scene changes would require re-encoding the video, which can take a long time and may reduce video quality. If you need to create a keyframe at every scene change, you should use the scdet filter as described in my previous messages.

_{chat2gist v0.1}