Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save fulldeck/e24e4c6156999ac8f57c3ba5b2aecb91 to your computer and use it in GitHub Desktop.
Save fulldeck/e24e4c6156999ac8f57c3ba5b2aecb91 to your computer and use it in GitHub Desktop.
/bin/tar: Argument list too long
@url https://major.io/2007/07/05/bintar-argument-list-too-long/
If you find yourself stuck with over 30,000 files in a directory (text files in this example), packing them into a tar file can be tricky.
Largest number of files completed 507,000 files of 355bytes each
NOTE: This will tap out all your servers resources and find the weak link; so be careful. CPU or Disk I/O?
You can get around it with this:
find . -name '*.txt' -print >/tmp/test.manifest
tar -cvzf textfiles.tar.gz --files-from /tmp/test.manifest
find . -name '*.txt' | xargs rm -v
# remove files all in one go ( have not tested which is faster)
find . -name '*.txt' -print >/tmp/test.manifest
tar -cvzf textfiles.tar.gz --files-from /tmp/test.manifest --remove-files
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment