Skip to content

Instantly share code, notes, and snippets.

View debasishg's full-sized avatar
🏠
Working from home

Debasish Ghosh debasishg

🏠
Working from home
View GitHub Profile
@danclien
danclien / scalaz_console_io_free_monad.scala
Last active August 29, 2015 13:57
Attempt to write a monad for console IO using scalaz's Free monad
:paste
import scalaz._, Scalaz._, scalaz.Free.{Suspend, Return}
// Console grammar
sealed trait ConsoleF[+A]
object Console {
case class WriteLine[A](msg: String, o: A) extends ConsoleF[A]
case class ReadLine[A](o: String => A) extends ConsoleF[A]
}
@jboner
jboner / how-akka-maps-to-eai-patterns.txt
Last active October 9, 2022 21:57
How Akka maps to EAI Patterns
# How Akka maps to EAI Patterns
Might be up for debate or just plain wrong. Just some notes I scribbled down some time ago.
-----------------------------------------------------------------------------------------------------------------
EAI PATTERN AKKA PATTERN REFERENCE
-----------------------------------------------------------------------------------------------------------------
Point to Point Channel Regular Actor Communication http://www.eaipatterns.com/PointToPointChannel.html
Event-Driven Consumer Regular Actor Receive http://www.eaipatterns.com/EventDrivenConsumer.html
Message Selector Actor with Stash http://www.eaipatterns.com/MessageSelector.html

Recent versions of Cloudera's Impala added NDV, a "number of distinct values" aggregate function that uses the HyperLogLog algorithm to estimate this number, in parallel, in a fixed amount of space.

This can make a really, really big difference: in a large table I tested this on, which had roughly 100M unique values of mycolumn, using NDV(mycolumn) got me an approximate answer in 27 seconds, whereas the exact answer using count(distinct mycolumn) took ... well, I don't know how long, because I got tired of waiting for it after 45 minutes.

It's fun to note, though, that because of another recent addition to Impala's dialect of SQL, the fnv_hash function, you don't actually need to use NDV; instead, you can build HyperLogLog yourself from mathematical primitives.

HyperLogLog hashes each value it sees, and then assigns them to a bucket based on the low order bits of the hash. It's common to use 1024 buckets, so we can get the bucket by using a bitwise & with 1023:

select
@tonymorris
tonymorris / scalaz.scala
Created January 5, 2014 01:16
Questions about \/ and Validation
// Questions about \/ and Validation
object scalaz {
/*
This explanation might help. This is a compilable source file with revisions
available at https://gist.github.com/tonymorris/8263051
Fact: All monads are applicative functors. As has been seen we can witness the
`Applicative` that arises from the `Monad` primitives. Let's illustrate this:
@arosien
arosien / runar-io-free.scala
Last active September 10, 2016 07:17
Translation of Runar's ScalaIO 2013 presentation on IO and Free monads (http://blog.higher-order.com/assets/scalaio.pdf) to scalaz.
import scalaz._
import Scalaz._
import Free._
/** "Pure" interactions with a console. */
sealed trait Console[+A]
case class GetLine[A](k: String => A) extends Console[A]
case class PutLine[A](s: String, a: A) extends Console[A]
object Console {
@seanparsons
seanparsons / gist:7005543
Created October 16, 2013 10:08
Stacking ReaderT and OptionT.
import scalaz._,Scalaz._,scalaz.effect._,OptionT._,Kleisli._
type OptionTIO[+X] = OptionT[IO, X]
val reader1: ReaderT[Option, Int, String] = kleisli[Option, Int, String](n => n.toString.some)
val reader2: ReaderT[IO, Int, String] = kleisli[IO, Int, String](n => IO((n * 2).toString))
val convertedReader1: ReaderT[OptionTIO, Int, String] = reader1.mapK[OptionTIO, String](opt => optionT(IO(opt)))
val convertedReader2: ReaderT[OptionTIO, Int, String] = reader2.mapK[OptionTIO, String](io => io.liftM[OptionT])
@YoEight
YoEight / Conversion.scala
Last active December 19, 2015 18:48
Scalaz-stream Process to Play Enumerator
package deiko
import scala.concurrent.Future
import scala.concurrent.duration.Duration
import scalaz.concurrent.Task
import scalaz.stream._
import play.api.libs.iteratee._
import Process._
object Conversion {
object ArgonautPlay {
implicit val writeableJson: Writeable[Json] = {
// TODO this should explicitly be a UTF-8 encoder
val stringWriter = implicitly[Writeable[String]].transform
Writeable((a: Json) => stringWriter(a.spaces2), Some("application/json; charset=utf-8"))
}
}
@coreyhaines
coreyhaines / Current shopping cart
Last active December 18, 2015 00:08
Damn you MITPress and your 50% off weekend!
Computing
Paul E. Ceruzzi
ISBN: 9780262517676
Modeling Business Processes
Wil van der Aalst and Christian Stahl
ISBN: 9780262015387
The Reasoned Schemer
Daniel P. Friedman, William E. Byrd, and Oleg Kiselyov
@retronym
retronym / instruction.sh
Last active December 17, 2015 00:29
scala-2.10
sudo gem install keydown # optional, only needed if you update the markdown.
git clone git@github.com:retronym/talks.git
git clone -b topic/play-keydown git@github.com:retronym/replhtml.git && cd replhtml
sbt -Dkeydown.root=$PWD/../talks/scala-2.10 run
open http://localhost:8080