Skip to content

Instantly share code, notes, and snippets.

@bhollis
Last active February 25, 2021 21:07
Show Gist options
  • Star 12 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save bhollis/6682c8b6b6357e2fedc5 to your computer and use it in GitHub Desktop.
Save bhollis/6682c8b6b6357e2fedc5 to your computer and use it in GitHub Desktop.
When you import movies into iMovie 10 libraries, the file is always copied, wasting space and hindering editability. This script replaces the copy with a hardlink, reclaiming disk space.
#!/usr/bin/env ruby
# Usage: dedup-imovie-library LIBRARY ORIGINALS
#
# Goes through an iMovie 10 library and replaces all the "Original Media" with
# hardlinks to the actual original media, in order to conserve disk space. Note
# that because they're hardlinks, if you copy the originals and the iMovie event
# to a new disk, you'll end up with two copies again and will have to re-run this
# tool.
#
# This assumes you've already imported the files into iMovie and waited for them
# all to be copied.
#
# This also assumes movie files in LIBRARY have unique matches in ORIGINALS with
# the same filename!
require 'fileutils'
library = ARGV.shift
originals = ARGV.shift
fail "Library #{library} does not exist" unless library && File.exist?(library)
fail "Originals folder #{originals} does not exist" unless originals && File.exist?(originals)
# For each original file in the imovie library
Dir.glob(File.join(library, '**', 'Original Media', '*')) do |library_file|
next unless File.file? library_file
# Skip it if we've already replaced it with a hardlink
next if File.stat(library_file).nlink > 1
original = Dir.glob(File.join(originals, '**', File.basename(library_file))).first
next unless original
puts "Linking #{library_file} => #{original}"
FileUtils.rm_f library_file, verbose: true
FileUtils.ln original, library_file, verbose: true
end
@cclavijo
Copy link

Great script thanks. For me the problem was that i dont have it the original files in one place. so i should to copy firstable the file by adding these line
FileUtils.cp library_file,File.join(originals, File.basename(library_file)) unless original

and voila it works great!

Thank you very much!

@porg
Copy link

porg commented Feb 22, 2021

Hi guys,
I did not read into the details of this script, but I think it has become obsolete meanwhile.
As on a contemporary iMovie on macOS 11 Big Sur with APFS my tests/checks have shown that:
iMovie avoids unnecessary data duplication by facilitating the copy-on-write feature of APFS. 🙂

@bmaupin
Copy link

bmaupin commented Feb 24, 2021

Doesn't Time Machine back up hard links as separate files? 😬

https://superuser.com/a/836655/93066

This would be a no-go in that case, especially if symlinks work just as well.

@porg
Copy link

porg commented Feb 25, 2021

I can not give definitive answers, but approximated answers at least:

  1. Hardlinks on a HFS+ source volume end up as separate files in a HFS+ TimeMachine backup.
  • Have never verified this myself, or if, do not recall it anymore.
  • But this source seems creditable, as it's a long discussion with a clarification followed by a final answer and the people quote Pondini.org which at this time (2013) was the most widely respected knowledge source on Time Machine. Its author James Pond has passed away in the meantime, but there's a mirror available.
  • Meanwhile my most trusted public knowledge base on macOS TimeMachine and filesystem topics is Eclecticlight.co (Howard Oakley).
  1. When the source is APFS and the destination is an APFS volume with the "Backup" role, then I do not know how they behave in detail. Just collect a few related facts plus assumptions:

APFS doesn’t support directory hard links, so can’t use the same mechanism when storing Time Machine backups. Instead, what appears to function as a form of virtual file system is created using new features in APFS. The volume assigned the role of Backup appears to be a regular APFS volume, and is protected from normal access, even by root.

  • Copy-on-write files are saved as separate files as soon as the travel to another volume.
  • No information how hard links on an APFS source volume end up on an APFS TimeMachine backup volume.
    • Still as in HFS+ ending up as duplicates or meanwhile cleverly preserving them?
    • Expert answer appreciated!

@bmaupin
Copy link

bmaupin commented Feb 25, 2021

What about an existing tool like rdfind? e.g.

rdfind -dryrun true -minsize 1048576 -makesymlinks true ~/Pictures/ ~/Movies/

(remove -dryrun true when you're ready to make the changes)

It will also let you use hard links if you're so inclined ;)

https://apple.stackexchange.com/a/414495/22772

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment