Skip to content

Instantly share code, notes, and snippets.

@yadudoc

yadudoc/SwiftReduce.md

Last active January 2, 2016 15:29
Show Gist options
  • Save yadudoc/8324194 to your computer and use it in GitHub Desktop.
Save yadudoc/8324194 to your computer and use it in GitHub Desktop.
Download ZIP
Swift Reduce - Hadoop
Raw
SwiftReduce.md

SwiftReduce - Hadoop Comparison

#Hadoop

  • Hadoop streaming uses STDIN to feed data to the mapper and STDOUT picks up data
  • No concept of files, instead each line is treated as a record.

#SwiftReduce

  • Currently mappers take filename as input via STDIN
  • TODO - Mappers to take filepointers in addition to filenames.
  • Reducers currently are expected to send output to STDOUT, this limits output streams to 1.
  • TODO - Reducers to be able to returns FILES ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment