Skip to content

Instantly share code, notes, and snippets.

@dmalikov
Created August 12, 2013 20:17
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save dmalikov/6214731 to your computer and use it in GitHub Desktop.
Save dmalikov/6214731 to your computer and use it in GitHub Desktop.
unsorted stream intersection with grep, -xF magician
What is going on here is creating 2 streams with numbers from 1 to 100000 sorting randomly and finding with grep all lines that each of them contains. All of them, actually. -c flag is a shortcut for `| wc -l`, nothng more.
$> n=100000; time grep -f <(seq 1 $n | sort -R) <(seq 1 $n | sort -R) -c
100000
grep -f <(seq 1 $n | sort -R) <(seq 1 $n | sort -R) -c 148.65s user 0.28s system 98% cpu 2:31.89 total
$> n=100000; time grep -xF -f <(seq 1 $n | sort -R) <(seq 1 $n | sort -R) -c
100000
grep -xF -f <(seq 1 $n | sort -R) <(seq 1 $n | sort -R) -c 0.34s user 0.01s system 26% cpu 1.315 total
Hope it will never be useful for me =(
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment