Skip to content

Instantly share code, notes, and snippets.

@fried

fried/tarfilecopybuf

Created June 3, 2016 00:40
Show Gist options
  • Save fried/30cae14f10efe204b3fa877de1019e82 to your computer and use it in GitHub Desktop.
Save fried/30cae14f10efe204b3fa877de1019e82 to your computer and use it in GitHub Desktop.
Download ZIP
Benchmarch of Tarfile where you can set the copyfileobj bufsize beyond the default of 16k, using two random tar files of 1024 files each 10MB in size
Raw
tarfilecopybuf
Generating Random Test Tar Files................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Testing Tar Impls
Time Delta for TarFile: 146.3240258693695
Time Delta for FastTarFile 4MB copybufsize: 102.76440262794495
Time Diff: 43.55962324142456 0.2976928975444698
@fried
Copy link
Author

fried commented Jun 3, 2016

Copybuffer and notruncate 35% improvement in linux

Time Delta for TarFile: 139.1847152709961
Time Delta for FastTarFile 4MB copybufsize: 90.22910213470459
Time Diff: 48.955613136291504 0.3517312446339651

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment