Skip to content

Instantly share code, notes, and snippets.


Raymond Tay raymondtay

View GitHub Profile

Understanding Comparative Benchmarks

I'm going to do something that I don't normally do, which is to say I'm going to talk about comparative benchmarks. In general, I try to confine performance discussion to absolute metrics as much as possible, or comparisons to other well-defined neutral reference points. This is precisely why Cats Effect's readme mentions a comparison to a fixed thread pool, rather doing comparisons with other asynchronous runtimes like Akka or ZIO. Comparisons in general devolve very quickly into emotional marketing.

But, just once, today we're going to talk about the emotional marketing. In particular, we're going to look at Cats Effect 3 and ZIO 2. Now, for context, as of this writing ZIO 2 has released their first milestone; they have not released a final 2.0 version. This implies straight off the bat that we're comparing apples to oranges a bit, since Cats Effect 3 has been out and in production for months. However, there has been a post going around which cites various compar

View Origami.hs
module Origami where
-- Origami is the japanese art of folding and unfolding
import Data.Bifunctor
-- Origami Programming refers to a style of generic programming that focuses on leveraging core patterns
-- of recursion: map, fold and unfold.
View GADTs.hs
import Numeric
data Expr a where
I :: Int -> Expr Int
B :: Bool -> Expr Bool
Add :: Expr Int -> Expr Int -> Expr Int
Mul :: Expr Int -> Expr Int -> Expr Int
Eq :: Eq a => Expr a -> Expr a -> Expr Bool
View Closet.hs
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE TypeFamilies #-}
import Control.Concurrent.STM
import Control.Concurrent.MVar
import Data.Foldable (forM_)
import Data.IORef
-- class Closet closet where
-- newClosetIO :: a -> IO (closet a)
View IO.scala
object IO {
def cancelable[A](k: (Either[Throwable, A] => Unit) => CancelToken[IO]): IO[A]
View transducer.scala
// The type of the transducer function happens to resemble the folding function
// you would normally pass to `foldLeft` or `foldRight`. Here's an example
// using the following:
// {{{
// val xs = List("1", "2", "3")
// scala> xs.foldLeft
// override def foldLeft[B](z: B)(op: (B, String) => B): B
// }}}
// The type of `op` is exactly that of the type `RF` - what a coincidence !
View LocalFsManager.scala
package com.databricks
import{File, FileNotFoundException}
import com.databricks.backend.daemon.dbutils.{FileInfo, MountInfo}
import com.databricks.dbutils_v1.DbfsUtils
import org.apache.hadoop.fs.FileSystem
trait LocalFileSystem extends DbfsUtils {
View Common.scala
import org.apache.spark.sql.DataFrameReader
def configureReaderWithElasticSearch(options: Map[String,String]) : DataFrameReader = {
val fr ="es")
def go(fr: DataFrameReader, optMap: Map[String, String]) =
optMap.foldLeft(fr)((reader, c) => reader.option(c._1, c._2))
go(fr, options)
def loadDataFromElasticSearch(index: String)(reader: DataFrameReader) : DataFrame =
raymondtay / RateLimitingStream.hs
Last active Nov 4, 2018
Rate Limiting Producer using Par Monad Direct Scheduler
View RateLimitingStream.hs
module RateLimitingStream (
) where
import System.Environment
import Control.Monad.IO.Class
import Control.Monad.Par
import Control.DeepSeq

Explaining Miles's Magic

Miles Sabin recently opened a pull request fixing the infamous SI-2712. First off, this is remarkable and, if merged, will make everyone's life enormously easier. This is a bug that a lot of people hit often without even realizing it, and they just assume that either they did something wrong or the compiler is broken in some weird way. It is especially common for users of scalaz or cats.

But that's not what I wanted to write about. What I want to write about is the exact semantics of Miles's fix, because it does impose some very specific assumptions about the way that type constructors work, and understanding those assumptions is the key to getting the most of it his fix.

For starters, here is the sort of thing that SI-2712 affects:

def foo[F[_], A](fa: F[A]): String = fa.toString