Last active
June 26, 2018 05:36
-
-
Save albertony/7727aa860e18e154f480275f98f4fd1d to your computer and use it in GitHub Desktop.
Shotcut-SourceFileFixer.ps1
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <# | |
| .SYNOPSIS | |
| Fix source file references in Shotcut video editor project files (.mlt). | |
| .DESCRIPTION | |
| Parses a specified Shotcut project file (in the XML-based MLT format), | |
| finds all source file resources (producers), checks if the referenced files | |
| exists and matches the found files with the hash values stored for them in | |
| the project file. | |
| If the stored file hash does not match the resolved file, the project | |
| file will be updated with the correct hash value. Specify argument | |
| -Confirm to be prompted what to do before any changes are made permanent | |
| by saving back the project file, or use argument -WhatIf to never do any | |
| permanent modifications but just see if anything would have been done. | |
| Currently ConfirmImpact of this script is set to 'High', which means you | |
| will be prompted by default, without having to specify -Confirm, and | |
| instead you have to specify -Confirm:$false to save changes without being | |
| prompted. | |
| Shotcut manages file paths relative to the directory of the project file, | |
| so this script does the same by default. Can optionally resolve paths | |
| against a different location specified in argument -SourceDirectory, but | |
| make sure you understand the implications first. | |
| When specifying alternative source path with -SourceDirectory, the default | |
| behavior is to only use this for checking existance and hash, updating | |
| hash if necessary, but by also specifying -UpdatePaths the file references | |
| in the project file will be updated accordingly. | |
| This functionality can be used to check and fix basic file reference issues | |
| in the project file, but you can also use it to implement a proxy editing | |
| work process: To increase responsiveness during the editing process you | |
| create low resolution versions of the source files which you use for | |
| creating the Shotcut project. Then just before exporting your finished | |
| project you replace the low resolution files with the original full | |
| resolution files. The Shotcut project file includes a generated hash value | |
| for each source file, so when simply replacing the source files used by an | |
| existing project the hash values will no longer be correct. This script | |
| fixes that. It is not critical to do so, the Shotcut project will still be | |
| fully usable, but some minor features may be affected. According to the | |
| Shotcut developer the hash is used for two things only: When a file is | |
| missing to determine if the replacement file offered is the same (gives | |
| yellow warning symbol in the dialog when not), and as a key for looking up | |
| cached thumbnails and audio levels | |
| (https://forum.shotcut.org/t/jerky-video-in-timeline/5992/8). Before going | |
| into the proxy editing process, try disabling the option | |
| "Realtime (frame dropping)" from the settings menu in the Shotcut user | |
| interface. This makes the timeline playback smoother because it stops | |
| trying to ensure playback is in realtime, but also it enables additional | |
| parallel processing for effects. | |
| .NOTES | |
| When saving a modified file the existing whitespace and formatting | |
| are mostly kept, but the following minor changes were impossible to avoid: | |
| - Adding encoding to the XML declaration (changing from | |
| <?xml version="1.0"?> to <?xml version="1.0" encoding="utf-8"?>). | |
| - Removing unnecessary whitespace before "/>" (e.g. changing from | |
| <track producer="background" /> to <track producer="background"/>). | |
| Version history: | |
| 22.06.2018: Initial version | |
| .LINK | |
| https://www.shotcut.org/ | |
| .PARAMETER ProjectFile | |
| The Shotcut project file (.mlt) to process, with absolute or relative path. | |
| .PARAMETER SourceDirectory | |
| Alternative source directory for resolving relative path to source files | |
| in the Shotcut project file (.mlt). If also -UpdatePaths the project file | |
| will be updated with new paths into the alternative source directory, | |
| without it is only used for checking existance and hash values. | |
| Default is the directory of the project file, this is what Shotcut is using | |
| so be careful if you are saving changes to the project file using a | |
| different source directory. | |
| .PARAMETER UpdatePaths | |
| When specifying -SourceDirectory, then update source file paths in the | |
| project file according to the specified path. | |
| .EXAMPLE | |
| Shotcut-SourceFileFixer.ps1 myproject.mlt -WhatIf | |
| Verify that the stored hash value for source files in the project file | |
| myproject.mlt in current directory is correct. Report any issues, but do | |
| not perform any permanent changes. | |
| .EXAMPLE | |
| Shotcut-SourceFileFixer.ps1 .\Projects\myproject.mlt | |
| Verify file hashes in the project file specified with relative path, update | |
| all that are not matching. Depending on the ConfirmPreference setting it | |
| may or may not prompt for confirmation before saving any changes back to | |
| the project file (the default is to always prompt). | |
| .EXAMPLE | |
| Shotcut-SourceFileFixer.ps1 .\Projects\myproject.mlt -Confirm | |
| Prompt for confirmation before saving file, regardless of current | |
| ConfirmPreference setting. | |
| .EXAMPLE | |
| Shotcut-SourceFileFixer.ps1 C:\Shotcut\Projects\myproject.mlt -Confirm:$false | |
| Save any changes back to the project file without confirmation. | |
| .EXAMPLE | |
| Shotcut-SourceFileFixer.ps1 myproject.mlt -SourceDirectory source\lowres | |
| Verify file hashes in the project file, but resolve relative file | |
| references according to specified directory path source\lowres instead | |
| of the project file directory. Do not change paths in the project file, | |
| only hash values. | |
| .EXAMPLE | |
| Shotcut-SourceFileFixer.ps1 myproject.mlt -SourceDirectory source\lowres -UpdatePaths | |
| Update all source file references in the project file to make them relative | |
| to the specified directory source directory source\lowres instead of the | |
| project file directory. Verify file hashes and update if needed. | |
| #> | |
| [CmdletBinding(SupportsShouldProcess=$true, ConfirmImpact='High')] | |
| param | |
| ( | |
| [Parameter(Mandatory=$true,Position=0, ParameterSetName='OriginalSource')] | |
| [Parameter(Mandatory=$true,Position=0, ParameterSetName='DifferentSource')] | |
| [string] $ProjectFile, | |
| [Parameter(Mandatory=$true, ParameterSetName='DifferentSource')] | |
| [string] $SourceDirectory, | |
| [Parameter(Mandatory=$false, ParameterSetName='DifferentSource')] | |
| [switch] $UpdatePaths | |
| ) | |
| $projectFileObject = Get-Item $ProjectFile -ErrorAction Ignore | |
| if ($projectFileObject -eq $null) | |
| { | |
| throw "File `"$ProjectFile`" does not exist!" | |
| } | |
| if (-not $SourceDirectory) | |
| { | |
| $SourceDirectory = $projectFileObject.Directory.FullName | |
| } | |
| $xml = New-Object System.Xml.XmlDocument | |
| $xml.PreserveWhitespace = $true | |
| $xml.Load($projectFileObject.FullName) | |
| $hasher = [System.Security.Cryptography.HashAlgorithm]::Create("MD5") | |
| $numberOfFoundFiles = 0 | |
| $numberOfMissingFiles = 0 | |
| $numberOfChangedHashes = 0 | |
| $numberOfChangedPaths = 0 | |
| foreach ($producer in $xml.mlt.SelectNodes("producer")) | |
| { | |
| $type = $producer.SelectSingleNode("property[@name='mlt_service']").'#text' | |
| #if ($type -in 'avformat-novalidate', 'qimage', 'pixbuf') # Only consider producers with file resource (silently skip things like color, count etc) | |
| if ($type -notin 'color', 'count', 'luma') # Only consider producers with file resource (silently skip things like color, count etc), see shotcut\src\mltxmlchecker.cpp | |
| { | |
| $resourceProperty = $producer.SelectSingleNode("property[@name='resource']") # File paths are stored here | |
| $fileDetailProperty = $producer.SelectSingleNode("property[@name='shotcut:detail']") # Sometimes the file path is also here, but sounds like just additional info | |
| $fileRelativePath = $resourceProperty.'#text' | |
| if ($fileRelativePath) | |
| { | |
| $fileFullPath = Join-Path $SourceDirectory $fileRelativePath # This also converts path separators from / to \ | |
| if (Test-Path $fileFullPath -ErrorAction Ignore) | |
| { | |
| if ($UpdatePaths) | |
| { | |
| $commonRootLevels = 0 | |
| $commonRoot = $projectFileObject.Directory | |
| while ($commonRoot -and -not $fileFullPath.StartsWith($commonRoot.FullName)) | |
| { | |
| $commonRoot = $commonRoot.Parent | |
| $commonRootLevels++ | |
| } | |
| if ($commonRoot) | |
| { | |
| $newFileRelativePath = $fileFullPath.Substring($commonRoot.FullName.Length + 1) | |
| $newFileRelativePath = "..\"*$commonRootLevels + $newFileRelativePath | |
| } | |
| else | |
| { | |
| $newFileRelativePath = $fileFullPath | |
| } | |
| $newFileRelativePath = $newFileRelativePath.Replace('\','/') | |
| if ($resourceProperty.'#text' -ne $newFileRelativePath) | |
| { | |
| $resourceProperty.'#text' = "$newFileRelativePath" | |
| Write-Host "Changed file path in `"$($producer.id)`" to `"${newFileRelativePath}`"" | |
| if ($fileDetailProperty) | |
| { | |
| if ($fileDetailProperty.'#text' -eq $fileRelativePath) # Update additional property only if existing value is the existing path | |
| { | |
| Write-Verbose "Detail property in `"$($producer.id)`" was set to previous path, updating to new path" | |
| $fileDetailProperty.'#text' = "$newFileRelativePath" | |
| } | |
| else | |
| { | |
| Write-Verbose "Keeping existing value of detail property in `"$($producer.id)`": `"$($fileDetailProperty.'#text')`"" | |
| } | |
| } | |
| $fileRelativePath = $newFileRelativePath # Keep new path for further references below | |
| $numberOfChangedPaths++ | |
| } | |
| else | |
| { | |
| Write-Verbose "Existing path is correct for `"$($producer.id)`": ${fileRelativePath}" | |
| } | |
| } | |
| $hashProperty = $producer.SelectSingleNode("property[@name='shotcut:hash']") | |
| $prefixSize = $suffixSize = 1000000 | |
| $inputStream = New-Object IO.FileStream($fileFullPath, [IO.FileMode]::Open, [IO.FileAccess]::Read, [IO.FileShare]::Read) | |
| $fileSize = $inputStream.Length | |
| if ($fileSize -gt $prefixSize+$suffixSize) | |
| { | |
| $fileBytes = New-Object byte[] ($prefixSize+$suffixSize) | |
| $prefixReadSize = $inputStream.Read($fileBytes, 0, $prefixSize) | |
| if ($prefixReadSize -lt $prefixSize) | |
| { | |
| Write-Warning "Read ${prefixReadSize} of expected ${prefixSize} prefix bytes from file ${fileRelativePath}" | |
| } | |
| $inputStream.Position = $inputStream.Length - $suffixSize | |
| $suffixReadSize = $inputStream.Read($fileBytes, $prefixSize, $suffixSize) | |
| if ($suffixReadSize -lt $suffixSize) | |
| { | |
| Write-Warning "Read ${suffixReadSize} of expected ${suffixSize} suffix bytes from file ${fileRelativePath}" | |
| } | |
| $readSize = $prefixReadSize + $suffixReadSize | |
| } | |
| else | |
| { | |
| $fileBytes = New-Object byte[] ($fileSize) | |
| $readSize = $inputStream.Read($fileBytes, 0, $fileSize) | |
| if ($readSize -lt $fileSize) | |
| { | |
| Write-Warning "Read ${readSize} of expected ${fileSize} bytes from file ${fileRelativePath}" | |
| } | |
| } | |
| $hashBytes = $hasher.ComputeHash($fileBytes, 0, $readSize) | |
| $inputStream.Close() | |
| $builder = New-Object System.Text.StringBuilder | |
| $hashBytes | Foreach-Object { [void] $builder.Append($_.ToString("X2")) } | |
| $newHash = $builder.ToString().ToLower() | |
| if ($hashProperty.'#text' -ne $newHash) | |
| { | |
| $hashProperty.'#text' = "$newHash" | |
| Write-Host "Changed hash in `"$($producer.id)`" for file `"${fileRelativePath}`"" | |
| $numberOfChangedHashes++ | |
| } | |
| else | |
| { | |
| Write-Verbose "Existing hash in `"$($producer.id)`" for file `"${fileRelativePath}`" is valid" | |
| } | |
| $numberOfFoundFiles++ | |
| } | |
| else | |
| { | |
| Write-Warning "Resource of `"$($producer.id)`" does not point to an existing file: ${fileFullPath}" | |
| $numberOfMissingFiles++ | |
| } | |
| } | |
| else | |
| { | |
| Write-Verbose "Skipping `"$($producer.id)`": No resource" | |
| } | |
| } | |
| else | |
| { | |
| Write-Verbose "Skipping `"$($producer.id)`": Type `"${type}`" is not known to have file resource" | |
| } | |
| } | |
| Write-Host "Processed $($numberOfFoundFiles+$numberOfMissingFiles) producer source file resources" | |
| if ($numberOfChangedPaths -or $numberOfChangedHashes -or $numberOfMissingFiles) | |
| { | |
| if ($numberOfChangedPaths) | |
| { | |
| Write-Host "${numberOfChangedPaths} file path$(if($numberOfChangedPaths -gt 1){'s'}) changed" | |
| } | |
| if ($numberOfChangedHashes) | |
| { | |
| Write-Host "${numberOfChangedHashes} file hash$(if($numberOfChangedHashes -gt 1){'es'}) changed" | |
| } | |
| if ($numberOfMissingFiles) | |
| { | |
| Write-Warning "Skipped ${numberOfMissingFiles} missing file$(if($numberOfMissingFiles -gt 1){'s'})" | |
| } | |
| if ($numberOfChangedPaths -or $numberOfChangedHashes) | |
| { | |
| $reason = [System.Management.Automation.ShouldProcessReason]::None | |
| if ($PSCmdlet.ShouldProcess("Save changes to file `"${ProjectFile}`"", "Are you sure you want to save changes to file `"${ProjectFile}`"?", "Confirm", [ref]$reason)) | |
| { | |
| $writer = New-Object System.Xml.XmlTextWriter $projectFileObject.FullName, $null | |
| $writer.Formatting = "None" | |
| $xml.Save($writer) | |
| $writer.Close() | |
| Write-Host "File `"${ProjectFile}`" saved" | |
| } | |
| else | |
| { | |
| if ($reason -ne [System.Management.Automation.ShouldProcessReason]::WhatIf) | |
| { | |
| Write-Host "All changes discarded (file not saved)" | |
| } | |
| } | |
| } | |
| } | |
| else | |
| { | |
| Write-Host "All OK" | |
| } |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment