Skip to content

Instantly share code, notes, and snippets.

View calvinlfer's full-sized avatar

Calvin Lee Fernandes calvinlfer

View GitHub Profile
calvinlfer / Avro4sDeserializer.scala
Created June 8, 2024 17:37
FS2 Kafka Avro4S support (Schema Registry aware)
import com.sksamuel.avro4s.{Decoder, SchemaFor}
import fs2.kafka.{Deserializer, ValueDeserializer, KeyDeserializer}
import fs2.kafka.vulcan.AvroSettings
import cats.effect.{Sync, Resource}
import java.nio.ByteBuffer
import io.confluent.kafka.schemaregistry.avro.AvroSchema
final class Avro4sDeserializer[A >: Null](
private val decoder: Decoder[A],
private val schemaFor: SchemaFor[A]
package io.kaizensolutions.virgil.nextlevel
import io.kaizensolutions.virgil.nextlevel.nextlevel.peopleTable
object nextlevel {
sealed trait Table {
type TableTag
object Table {
calvinlfer /
Created May 8, 2023 13:06
ZIO HTTP (Graal Native Image & Scala CLI)
scala-cli --power package --native-image -f *.scala -o ziohttp.bin -- --no-fallback --enable-url-protocols=http,https -Djdk.http.auth.tunneling.disabledSchemes= --install-exit-handlers --enable-http --initialize-at-run-time=io.netty.handler.codec.compression.ZstdOptions --initialize-at-run-time=io.netty.incub
calvinlfer / ZioHttpServerTracer.scala
Created February 17, 2023 19:42
getting tracing to work with zio-http 0.0.4's new Handler abstraction
package io.kaizensolutions.trace4cats.zio.extras.ziohttp.server
import trace4cats.ErrorHandler
import trace4cats.model.AttributeValue.{LongValue, StringValue}
import trace4cats.model.SemanticAttributeKeys.*
import trace4cats.model.{AttributeValue, SpanKind, SpanStatus}
import io.kaizensolutions.trace4cats.zio.extras.ZTracer
import io.kaizensolutions.trace4cats.zio.extras.ziohttp.{extractTraceHeaders, toSpanStatus}
import zio.*
import zio.http.*
calvinlfer / OneToManyJoin.scala
Created November 25, 2022 17:49
In memory one to many join
def joinLManyR[Err, LeftElem, LeftKey, MiddleElem, MiddleKey, RightElem, RightKey, RightProjection](left: Iterable[LeftKey])(
middle: Iterable[LeftKey] => Stream[Err, MiddleElem]
)(middleKey: MiddleElem => MiddleKey)(middleToLeftKey: MiddleElem => LeftKey)(middleToRightKey: MiddleElem => RightKey)(
right: Iterable[RightKey] => Stream[Err, RightElem]
)(rightKey: RightElem => RightKey)(rightProjection: RightElem => RightProjection): IO[Err, Map[LeftKey, Set[RightProjection]]] =
val extractMiddle: IO[Err, Chunk[MiddleElem]] = middle(left).runCollect
def rightElems(in: Iterable[RightKey]): IO[Err, Chunk[RightElem]] =
calvinlfer / ZonedDateTimeRange.scala
Last active October 31, 2022 20:16
Preliminary support for a specific range in Skunk
import skunk.Codec
import zio.ZIOAppDefault
import java.time.{OffsetDateTime, ZonedDateTime}
import java.time.format.{DateTimeFormatter, DateTimeFormatterBuilder, DateTimeParseException}
import java.time.temporal.ChronoField.*
import java.util.Locale
// TODO: account for closed and open intervals so like [x,y), (x,y), etc.
calvinlfer / docker-compose.yaml
Created May 7, 2022 20:21
Single node Kafka docker-compose compatible with Docker on M1 Macs
version: "3.9"
restart: unless-stopped
image: ubuntu/zookeeper:latest
- "2181:2181"
calvinlfer / pipeline.scala
Created May 3, 2022 14:12
Understanding ZChannels (zio-streams 2.x)
object Experiment extends ZIOAppDefault {
Collect elements and emit them in single chunks of 100
val aggregate: ZPipeline[Any, Nothing, Int, Long] = {
def go[Err](acc: Long): ZChannel[Any, Err, Chunk[Int], Any, Err, Chunk[Long], Any] =
ZChannel.readWith[Any, Err, Chunk[Int], Any, Err, Chunk[Long], Any](
in = { inChunk =>
val next = acc + inChunk.sum
if (next > 100L) {
calvinlfer /
Last active April 20, 2022 09:10
How to build a GraalVM native image packaged as a Docker image that can be deployed on AWS Lambda

A build process for GraalVM native images that can be deployed on AWS Lambda as Docker container images

Use zio-lambda to write your own Lambda function based on Scala and ZIO

  • Checkout the example project for inspiration
  • Make sure you enable the GraalVMNativeImagePlugin like the example project does and configure it with the appropriate flags to build the native image
  • You may most likely need to add some reflect configuration and this post from Inner Product shows a good way to include the configuration in the project so Graal native-image will find it

If you have graal native already installed with the correct libraries in place for linking then great, otherwise use one of the following Docker images tagged with graalvm


calvinlfer / CStream.scala
Created December 17, 2021 20:47
Zymposium part 1
import{BufferedReader, FileReader}
final case class CStream[-R, +E, +A](
private val process: ZManaged[R, Nothing, ZIO[R, Option[E], Chunk[A]]]
) { self =>
def take(n: Int): CStream[R, E, A] = {
def go(rem: Ref[Int], pull: ZIO[R, Option[E], Chunk[A]]): ZIO[R, Option[E], Chunk[A]] =
rem.get.flatMap {
case n if n <= 0 =>