When you import movies into iMovie 10 libraries, the file is always copied, wasting space and hindering editability. This script replaces the copy with a hardlink, reclaiming disk space.

dedup-imovie-library

#!/usr/bin/env ruby

# Usage: dedup-imovie-library LIBRARY ORIGINALS
#
# Goes through an iMovie 10 library and replaces all the "Original Media" with
# hardlinks to the actual original media, in order to conserve disk space. Note
# that because they're hardlinks, if you copy the originals and the iMovie event
# to a new disk, you'll end up with two copies again and will have to re-run this
# tool.
#
# This assumes you've already imported the files into iMovie and waited for them
# all to be copied.
#
# This also assumes movie files in LIBRARY have unique matches in ORIGINALS with
# the same filename!

require 'fileutils'

library = ARGV.shift
originals = ARGV.shift

fail "Library #{library} does not exist" unless library && File.exist?(library)
fail "Originals folder #{originals} does not exist" unless originals && File.exist?(originals)

# For each original file in the imovie library
Dir.glob(File.join(library, '**', 'Original Media', '*')) do |library_file|
  next unless File.file? library_file

  # Skip it if we've already replaced it with a hardlink
  next if File.stat(library_file).nlink > 1

  original = Dir.glob(File.join(originals, '**', File.basename(library_file))).first

  next unless original

  puts "Linking #{library_file} => #{original}"
  FileUtils.rm_f library_file, verbose: true
  FileUtils.ln original, library_file, verbose: true
end

You have to run ./dedup-imovie-library.

Hi,

I had the same problem above but fixed it by using chmod 777 dedup-imovie-library then using it as suggested, ./dedup-imovie-library.

I have a folder with all my GoPro files imported by GoPro Studio Importer, this puts them into subfolders within the GoPro folder.

My question is, can I use this code to scan through all the subfolders? If not is there an easy change to the code to allow this. Sorry, I don't know Ruby at all.

Thanks,

Stu.

Thanks for the code, I'm very happy to use it.

I made 3 modifications:

1. Replace the last line's command from "FileUtils.ln" to "FileUtils.ln_s" so it becomes a symlink(=softlink), which is working very well, also on other drives (not only which the iMovie library is on)

2. To not to replace symlinks, change the line
   next if File.stat(library_file).nlink > 1
   to
   next if File.stat(library_file).nlink > 1 || File.lstat(library_file).symlink?

   or if you don't want audio files to be processed too:
   next if File.stat(library_file).nlink > 1 || File.lstat(library_file).symlink? || File.extname(library_file) == ".wav" || File.extname(library_file) == ".mp3" || File.extname(library_file) == ".m4a"

3. Add "File::FNM_CASEFOLD" to the original file search command, so it searches case insensitive. (My original files are .MTS and iMovies modifies them during copy to lower case (.mts). So that's the modified command:
   original = Dir.glob(File.join(originals, '**', File.basename(library_file)), File::FNM_CASEFOLD).first

Great script! I'm a fairly new Mac user and was blown away when I discovered that iMovie copies files into its library. I'm currently on a one year trip and hard disk space is at an absolute premium. With tons of GoPro footage sitting duplicated on my drive, I was not a happy camper. I wanted to add a safety feature to the script though. You can check it out here:

https://gist.github.com/aardvarkk/9206a4eb5d5e78e5dbc0#file-dedup-imovie-library

All I did was add a check that the file contents are the same. This way, you won't accidentally delete/link a file that happens to have the same name as another. It will slow the entire process down, obviously, so maybe you'd want to add a switch or something... but for me I'd rather it be as safe as possible and I don't mind the hit to the speed.

Great work on this script. A tip of the hat to you, and a wag of the finger to Apple for wasting half of my hard disk with duplicated videos!

Thanks for this! I noticed that iMovie saves its files as .mp4 whereas my GoPro saves them as .MP4 and the comparison was failing because of the mis-matched case. I updated this gist (after incorporating @kabadisha's fork)

https://gist.github.com/swrobel/1fa71ea870a4d0feac2a6ca93b5e36eb

I am running MacOS 10.14.4 and I am getting the following error: 22:in '<main>': Library does not exist (RuntimeError)

I am thinking it is because I do not have 'fileutils' installed. Can someone offer some help on how on how to install 'fileutils' or in the alternative advise if I am off-base and the problem is something else? Thanks.

CG

I adapted it to Python here. I also used symlinks, assumed the iMovie library and the originals folder is organized by event name, and included projects to skip in case you are working on them.

@miguelmorin do you encounter these issues described in his original post? He discuss the use of hard link versus symbolic link.

Another option besides hardlinking the files together is to use a symbolic link instead. Symbolic links are simply a pointer to the original file, and as far as I can tell iMovie 2013 has no problem with using them. Symbolic links would solve the problems of not being able to link across disks, and would not take up double space in backups. However, iMovie ‘11 had a lot of weird behavior when using symbolically linked files (such as not being able to “favorite” sections of a clip) and I haven’t had a chance to test this with iMovie 2013. I’m also concerned about what would happen if I dragged an event into another library where the videos were stored as symbolic links. I’ll try out the alternative at some point and write another article if it works well.

https://benhollis.net/blog/2014/06/28/hard-linking-videos-to-save-space-in-your-imovie-library/

@dingzeyuli No, I haven't, although to be sure I run the deduplication only after I am done with a project to save space. Maybe just make a backup and try it?

Great script thanks. For me the problem was that i dont have it the original files in one place. so i should to copy firstable the file by adding these line
FileUtils.cp library_file,File.join(originals, File.basename(library_file)) unless original

and voila it works great!

Thank you very much!

Hi guys,
I did not read into the details of this script, but I think it has become obsolete meanwhile.
As on a contemporary iMovie on macOS 11 Big Sur with APFS my tests/checks have shown that:
iMovie avoids unnecessary data duplication by facilitating the copy-on-write feature of APFS. 🙂

Doesn't Time Machine back up hard links as separate files? 😬

https://superuser.com/a/836655/93066

This would be a no-go in that case, especially if symlinks work just as well.

I can not give definitive answers, but approximated answers at least:

1. Hardlinks on a HFS+ source volume end up as separate files in a HFS+ TimeMachine backup.

• Have never verified this myself, or if, do not recall it anymore.
• But this source seems creditable, as it's a long discussion with a clarification followed by a final answer and the people quote Pondini.org which at this time (2013) was the most widely respected knowledge source on Time Machine. Its author James Pond has passed away in the meantime, but there's a mirror available.
• Meanwhile my most trusted public knowledge base on macOS TimeMachine and filesystem topics is Eclecticlight.co (Howard Oakley).

2. When the source is APFS and the destination is an APFS volume with the "Backup" role, then I do not know how they behave in detail. Just collect a few related facts plus assumptions:

• https://eclecticlight.co/2020/06/29/apfs-changes-in-big-sur-how-time-machine-backs-up-to-apfs-and-more/ states:

APFS doesn’t support directory hard links, so can’t use the same mechanism when storing Time Machine backups. Instead, what appears to function as a form of virtual file system is created using new features in APFS. The volume assigned the role of Backup appears to be a regular APFS volume, and is protected from normal access, even by root.

• Copy-on-write files are saved as separate files as soon as the travel to another volume.
  • That's shown in a macOS High Sierra video by Mike Bombich of Bombich.com (Carbon Copy Cloner).
• No information how hard links on an APFS source volume end up on an APFS TimeMachine backup volume.
  • Still as in HFS+ ending up as duplicates or meanwhile cleverly preserving them?
  • Expert answer appreciated!

What about an existing tool like rdfind? e.g.

rdfind -dryrun true -minsize 1048576 -makesymlinks true ~/Pictures/ ~/Movies/

(remove -dryrun true when you're ready to make the changes)

It will also let you use hard links if you're so inclined ;)

https://apple.stackexchange.com/a/414495/22772