Skip to content

Instantly share code, notes, and snippets.

@cransom
Last active March 11, 2018 04:34
Show Gist options
  • Save cransom/e09212bc070420db27d82c417f36693b to your computer and use it in GitHub Desktop.
Save cransom/e09212bc070420db27d82c417f36693b to your computer and use it in GitHub Desktop.
pipe speeds
143yoga:~% timeout 1 cat /dev/zero | mbuffer -q -l cat.txt -o /dev/null
zsh: exit 124 timeout 1 cat /dev/zero |
zsh: mbuffer -q -l cat.txt -o /dev/null
143yoga:~% timeout 1 mbuffer -q -l native.txt -i /dev/zero -o /dev/null
124yoga:~% tail -n 1 cat.txt native.txt
==> cat.txt <==
mbuffer: in @ 1140 MiB/s, out @ 1140 MiB/s, 799 MiB total, buffer 0% full
==> native.txt <==
mbuffer: in @ 1995 MiB/s, out @ 1995 MiB/s, 1398 MiB total, buffer 0% full
Granted that we are dealing with the optimal, super fast speed of `/dev/zero`, I
didn't realize that passing data from cat could be around half as fast as just
letting a command ingest it's own file. It would be less significant on smaller
datasets, read from disk, but it would all add up over time.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment