Skip to content

Instantly share code, notes, and snippets.

@peijiehu

peijiehu/data_compression.md

Created June 23, 2017 05:08
Show Gist options
  • Save peijiehu/93514e01de542a89519bff78feca7376 to your computer and use it in GitHub Desktop.
Save peijiehu/93514e01de542a89519bff78feca7376 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
data_compression.md

Compression is a good option when you have a lot of CPU to throw around and limited IO.

Data transferred between map and reduce servers is compressed. The idea is that because servers aren't CPU bound it makes sense to spend on data compression and decompression in order to save on bandwidth and I/O.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment