Skip to content

Instantly share code, notes, and snippets.

@johnynek
Last active July 23, 2020 12:42
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save johnynek/7b99ca860220845d976c70ded5bb0276 to your computer and use it in GitHub Desktop.
Save johnynek/7b99ca860220845d976c70ded5bb0276 to your computer and use it in GitHub Desktop.
and example of using fs2 (and some private CSV row typeclass code) to do wordcount.
package dev.posco.hiona.jobs
import cats.effect.{Blocker, IOApp, IO, ExitCode}
import dev.posco.hiona._
import java.nio.file.Paths
object WordCountSimple extends IOApp {
case class Record(key: String, value: Long)
def run(args: List[String]): IO[ExitCode] =
Blocker[IO].use { blocker =>
val path = Paths.get(args(0))
val stream = Row.csvToStream[IO, Record](path, skipHeader = true, blocker = blocker)
stream.fold(Map.empty[String, Long]) { case (m, Record(k, v)) =>
m.get(k) match {
case Some(v0) => m.updated(k, v0 + v)
case None => m.updated(k, v)
}
}
.map(_.toString)
.lines(System.out)
.compile
.drain
.as(ExitCode.Success)
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment