Skip to content

Instantly share code, notes, and snippets.

@pgib
Created May 21, 2020 20:23
Show Gist options
  • Save pgib/38b0f8d05431fb354b8473955664fd69 to your computer and use it in GitHub Desktop.
Save pgib/38b0f8d05431fb354b8473955664fd69 to your computer and use it in GitHub Desktop.
Clean up the bloated Backblaze bzfileids.dat

Place in /Library/Backblaze.bzpkg/bzdata/bzbackup, and run with:

ruby purge_nonexistent.rb

It will process bzfileids.dat placing any file that exists in bzfileids.dat-found and any missing file in bzfileids.dat-missing. You can then back up your original file and replace it with bzfileids.dat-found.

My original file was almost 1.4GB. It had grown so large that the backup would cause all my fans spin and it never seemed to complete. Backblaze support suggested I delete my entire backup with them and start over. After running this script, my new bzfileids.dat file is 301MB. Still huge, but about 1/5th the size. The backup seemed to go much more smoothly.

#!/usr/bin/env ruby
# vim: ft=ruby
#
File.open('./bzfileids.dat').each_with_index do |line, index|
split = line.match(/^([a-z0-9_-]+)\s+(.+)$/)
if split
file_id = split[1]
path = split[2]
if !File.exists?(path)
File.open('./bzfileids.dat-missing', 'a+b') do |f|
f.puts line
f.close
end
else
File.open('./bzfileids.dat-found', 'a+b') do |f|
f.puts line
f.close
end
end
else
puts "#{index} Invalid line found: #{line}"
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment