Skip to content

Instantly share code, notes, and snippets.

@vsimko
Forked from timvw/RemoteIteratorWrapper.scala
Created August 30, 2017 23:06
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save vsimko/04fd6d289bac4b7c66a7e870c2ffc718 to your computer and use it in GitHub Desktop.
Save vsimko/04fd6d289bac4b7c66a7e870c2ffc718 to your computer and use it in GitHub Desktop.
scala wrapper for hadoop remote iterator
import org.apache.hadoop.fs.RemoteIterator
/**
* Converts RemoteIterator from Hadoop to Scala Iterator that provides all the familiar functions such as map,
* filter, foreach, etc.
*
* @param underlying The RemoteIterator that needs to be wrapped
* @tparam T Items inside the iterator
* @return Standard Scala Iterator
*/
implicit def convertToScalaIterator[T](underlying: RemoteIterator[T]): Iterator[T] = {
case class wrapper(underlying: RemoteIterator[T]) extends Iterator[T] {
override def hasNext = underlying.hasNext
override def next = underlying.next
}
wrapper(underlying)
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment