Skip to content

Instantly share code, notes, and snippets.

@chabala
Last active July 13, 2024 14:44
Show Gist options
  • Save chabala/22ed01d7acf9ee0de9e3d867133f83fb to your computer and use it in GitHub Desktop.
Save chabala/22ed01d7acf9ee0de9e3d867133f83fb to your computer and use it in GitHub Desktop.
Merge and extract tgz files from Google Takeout

Recently found some clowny gist was the top result for 'google takeout multiple tgz', where it was using two bash scripts to extract all the tgz files and then merge them together. Don't do that. Use brace expansion, cat the TGZs, and extract:

$ cat takeout-20201023T123551Z-{001..011}.tgz | tar xzivf -

You don't even need to use brace expansion. Globbing will order the files numerically:

$ cat takeout-20201023T123551Z-*.tgz | tar xzivf -

tar has been around forever, they didn't design it to need custom scripts to deal with multipart archives. Since it's extracting the combined archive, there's no 'mess of partial directories' to be merged. It just works, as intended.

An additional tip, courtesy of Dmitriy Otstavnov (@bvc3at): if you have pv available, you can track the progress of the extraction:

> pv takeout-* | tar xzif -
 190GiB 2:37:54 [18.9MiB/s] [==============>                                   ] 30% ETA 5:03:49
@dturner
Copy link

dturner commented Apr 25, 2024

@paradite Worked perfectly for zip files on MacOS - thank you!

@Thinkscape
Copy link

The gnu-tar doesn't seem to support -i shorthand, needs the full --ignore-zeros to work.

pv takeout-* | tar --ignore-zeros -xzf -

@chabala
Copy link
Author

chabala commented May 24, 2024

gnu-tar doesn't seem to support -i shorthand, needs the full --ignore-zeros to work.

@Thinkscape are you sure you're using GNU tar? Because that's the same complaint made earlier about bsdtar, where installing GNU tar was the solution.

@Thinkscape
Copy link

gnu-tar doesn't seem to support -i shorthand, needs the full --ignore-zeros to work.

@Thinkscape are you sure you're using GNU tar? Because that's the same complaint made earlier about bsdtar, where installing GNU tar was the solution.

My bad. homebrew linking mess got me this time ...

@JohnGemstone
Copy link

Thanks @paradite unzip \*.zip is all we need for zip archives.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment