Skip to content

Instantly share code, notes, and snippets.

Last active February 25, 2021 21:07
Show Gist options
  • Save bhollis/6682c8b6b6357e2fedc5 to your computer and use it in GitHub Desktop.
Save bhollis/6682c8b6b6357e2fedc5 to your computer and use it in GitHub Desktop.
When you import movies into iMovie 10 libraries, the file is always copied, wasting space and hindering editability. This script replaces the copy with a hardlink, reclaiming disk space.
#!/usr/bin/env ruby
# Usage: dedup-imovie-library LIBRARY ORIGINALS
# Goes through an iMovie 10 library and replaces all the "Original Media" with
# hardlinks to the actual original media, in order to conserve disk space. Note
# that because they're hardlinks, if you copy the originals and the iMovie event
# to a new disk, you'll end up with two copies again and will have to re-run this
# tool.
# This assumes you've already imported the files into iMovie and waited for them
# all to be copied.
# This also assumes movie files in LIBRARY have unique matches in ORIGINALS with
# the same filename!
require 'fileutils'
library = ARGV.shift
originals = ARGV.shift
fail "Library #{library} does not exist" unless library && File.exist?(library)
fail "Originals folder #{originals} does not exist" unless originals && File.exist?(originals)
# For each original file in the imovie library
Dir.glob(File.join(library, '**', 'Original Media', '*')) do |library_file|
next unless File.file? library_file
# Skip it if we've already replaced it with a hardlink
next if File.stat(library_file).nlink > 1
original = Dir.glob(File.join(originals, '**', File.basename(library_file))).first
next unless original
puts "Linking #{library_file} => #{original}"
FileUtils.rm_f library_file, verbose: true
FileUtils.ln original, library_file, verbose: true
Copy link

PromInc commented May 9, 2015

This is a great looking concept, but I am struggling to get it to work for me.

Based on your post about this script:
I got the impression the script should be saved to the ~/Movies directory. I saved the script as "dedup-imovie-library" with no file extension using TextEdit.

When I run the terminal command however, I get the following message:
-bash: dedup-imovie-library: command not found

Can you please help me to understand where I'm going wrong here?

Thanks so much for offering a great script to fix yet another one of Apple's poorly designed architectures.

Copy link

bhollis commented May 9, 2015

You have to run ./dedup-imovie-library.

Copy link


I had the same problem above but fixed it by using chmod 777 dedup-imovie-library then using it as suggested, ./dedup-imovie-library.

I have a folder with all my GoPro files imported by GoPro Studio Importer, this puts them into subfolders within the GoPro folder.

My question is, can I use this code to scan through all the subfolders? If not is there an easy change to the code to allow this. Sorry, I don't know Ruby at all.



Copy link

Thanks for the code, I'm very happy to use it.

I made 3 modifications:

  1. Replace the last line's command from "FileUtils.ln" to "FileUtils.ln_s" so it becomes a symlink(=softlink), which is working very well, also on other drives (not only which the iMovie library is on)

  2. To not to replace symlinks, change the line
    next if File.stat(library_file).nlink > 1
    next if File.stat(library_file).nlink > 1 || File.lstat(library_file).symlink?

    or if you don't want audio files to be processed too:
    next if File.stat(library_file).nlink > 1 || File.lstat(library_file).symlink? || File.extname(library_file) == ".wav" || File.extname(library_file) == ".mp3" || File.extname(library_file) == ".m4a"

  3. Add "File::FNM_CASEFOLD" to the original file search command, so it searches case insensitive. (My original files are .MTS and iMovies modifies them during copy to lower case (.mts). So that's the modified command:
    original = Dir.glob(File.join(originals, '**', File.basename(library_file)), File::FNM_CASEFOLD).first

Copy link

Great script! I'm a fairly new Mac user and was blown away when I discovered that iMovie copies files into its library. I'm currently on a one year trip and hard disk space is at an absolute premium. With tons of GoPro footage sitting duplicated on my drive, I was not a happy camper. I wanted to add a safety feature to the script though. You can check it out here:

All I did was add a check that the file contents are the same. This way, you won't accidentally delete/link a file that happens to have the same name as another. It will slow the entire process down, obviously, so maybe you'd want to add a switch or something... but for me I'd rather it be as safe as possible and I don't mind the hit to the speed.

Great work on this script. A tip of the hat to you, and a wag of the finger to Apple for wasting half of my hard disk with duplicated videos!

Copy link

swrobel commented May 4, 2016

Thanks for this! I noticed that iMovie saves its files as .mp4 whereas my GoPro saves them as .MP4 and the comparison was failing because of the mis-matched case. I updated this gist (after incorporating @kabadisha's fork)

Copy link

I am running MacOS 10.14.4 and I am getting the following error: 22:in '<main>': Library does not exist (RuntimeError)

I am thinking it is because I do not have 'fileutils' installed. Can someone offer some help on how on how to install 'fileutils' or in the alternative advise if I am off-base and the problem is something else? Thanks.


Copy link

I adapted it to Python here. I also used symlinks, assumed the iMovie library and the originals folder is organized by event name, and included projects to skip in case you are working on them.

Copy link

@miguelmorin do you encounter these issues described in his original post? He discuss the use of hard link versus symbolic link.

Another option besides hardlinking the files together is to use a symbolic link instead. Symbolic links are simply a pointer to the original file, and as far as I can tell iMovie 2013 has no problem with using them. Symbolic links would solve the problems of not being able to link across disks, and would not take up double space in backups. However, iMovie ‘11 had a lot of weird behavior when using symbolically linked files (such as not being able to “favorite” sections of a clip) and I haven’t had a chance to test this with iMovie 2013. I’m also concerned about what would happen if I dragged an event into another library where the videos were stored as symbolic links. I’ll try out the alternative at some point and write another article if it works well.

Copy link

emonigma commented Nov 4, 2020

@dingzeyuli No, I haven't, although to be sure I run the deduplication only after I am done with a project to save space. Maybe just make a backup and try it?

Copy link

Great script thanks. For me the problem was that i dont have it the original files in one place. so i should to copy firstable the file by adding these line
FileUtils.cp library_file,File.join(originals, File.basename(library_file)) unless original

and voila it works great!

Thank you very much!

Copy link

porg commented Feb 22, 2021

Hi guys,
I did not read into the details of this script, but I think it has become obsolete meanwhile.
As on a contemporary iMovie on macOS 11 Big Sur with APFS my tests/checks have shown that:
iMovie avoids unnecessary data duplication by facilitating the copy-on-write feature of APFS. 🙂

Copy link

bmaupin commented Feb 24, 2021

Doesn't Time Machine back up hard links as separate files? 😬

This would be a no-go in that case, especially if symlinks work just as well.

Copy link

porg commented Feb 25, 2021

I can not give definitive answers, but approximated answers at least:

  1. Hardlinks on a HFS+ source volume end up as separate files in a HFS+ TimeMachine backup.
  • Have never verified this myself, or if, do not recall it anymore.
  • But this source seems creditable, as it's a long discussion with a clarification followed by a final answer and the people quote which at this time (2013) was the most widely respected knowledge source on Time Machine. Its author James Pond has passed away in the meantime, but there's a mirror available.
  • Meanwhile my most trusted public knowledge base on macOS TimeMachine and filesystem topics is (Howard Oakley).
  1. When the source is APFS and the destination is an APFS volume with the "Backup" role, then I do not know how they behave in detail. Just collect a few related facts plus assumptions:

APFS doesn’t support directory hard links, so can’t use the same mechanism when storing Time Machine backups. Instead, what appears to function as a form of virtual file system is created using new features in APFS. The volume assigned the role of Backup appears to be a regular APFS volume, and is protected from normal access, even by root.

  • Copy-on-write files are saved as separate files as soon as the travel to another volume.
  • No information how hard links on an APFS source volume end up on an APFS TimeMachine backup volume.
    • Still as in HFS+ ending up as duplicates or meanwhile cleverly preserving them?
    • Expert answer appreciated!

Copy link

bmaupin commented Feb 25, 2021

What about an existing tool like rdfind? e.g.

rdfind -dryrun true -minsize 1048576 -makesymlinks true ~/Pictures/ ~/Movies/

(remove -dryrun true when you're ready to make the changes)

It will also let you use hard links if you're so inclined ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment