Skip to content

Instantly share code, notes, and snippets.

View vasnake's full-sized avatar
🏠
Working from home

Valentin vasnake

🏠
Working from home
View GitHub Profile
@vasnake
vasnake / spark.udf.fizzbuzz.scala
Last active August 25, 2023 15:28
Spark generic UDF, spark.sql.api.java
package org.apache.spark.sql.fizzbuzz.udf
import org.apache.spark.sql.api.java.UDF1
import scala.util.Try
import org.scalatest._
import flatspec._
class FizzBuzz extends UDF1[Any, String] {
def dbg(s: => String): Unit = println(s)

Explaining Miles's Magic

Miles Sabin recently opened a pull request fixing the infamous SI-2712. First off, this is remarkable and, if merged, will make everyone's life enormously easier. This is a bug that a lot of people hit often without even realizing it, and they just assume that either they did something wrong or the compiler is broken in some weird way. It is especially common for users of scalaz or cats.

But that's not what I wanted to write about. What I want to write about is the exact semantics of Miles's fix, because it does impose some very specific assumptions about the way that type constructors work, and understanding those assumptions is the key to getting the most of it his fix.

For starters, here is the sort of thing that SI-2712 affects:

def foo[F[_], A](fa: F[A]): String = fa.toString

Explaining Miles's Magic

Miles Sabin recently opened a pull request fixing the infamous SI-2712. First off, this is remarkable and, if merged, will make everyone's life enormously easier. This is a bug that a lot of people hit often without even realizing it, and they just assume that either they did something wrong or the compiler is broken in some weird way. It is especially common for users of scalaz or cats.

But that's not what I wanted to write about. What I want to write about is the exact semantics of Miles's fix, because it does impose some very specific assumptions about the way that type constructors work, and understanding those assumptions is the key to getting the most of it his fix.

For starters, here is the sort of thing that SI-2712 affects:

def foo[F[_], A](fa: F[A]): String = fa.toString
@djspiewak
djspiewak / 0introduction.md
Last active November 28, 2023 15:03
Scala Collections Proposal

Collections Redesign Proposal

I'm going to start off by motivating what I'm doing here. And I want to be clear that I'm not "dissing" the existing collections implementation or anything as unproductively negative as that. It was a really good experiment, it was a huge step forward given what we knew back in 2.8, but now it's time to learn from that experiment and do better. This proposal uses what I believe are the lessons we can learn about what worked, what didn't work, and what is and isn't important about collections in Scala.

This is going to start out sounding really negative and pervasively dismissive, but bear with me! There's a point to all my ranting. I want to be really clear about my motivations for the proposal being the way that it is.

Problems

Generic Interfaces

@Vestride
Vestride / encoding-video.md
Last active April 24, 2024 09:59
Encoding video for the web

Encoding Video

Installing

Install FFmpeg with homebrew. You'll need to install it with a couple flags for webm and the AAC audio codec.

brew install ffmpeg --with-libvpx --with-libvorbis --with-fdk-aac --with-opus
@sloria
sloria / bobp-python.md
Last active May 1, 2024 08:37
A "Best of the Best Practices" (BOBP) guide to developing in Python.

The Best of the Best Practices (BOBP) Guide for Python

A "Best of the Best Practices" (BOBP) guide to developing in Python.

In General

Values

  • "Build tools for others that you want to be built for you." - Kenneth Reitz
  • "Simplicity is alway better than functionality." - Pieter Hintjens

Moved

Now located at https://github.com/JeffPaine/beautiful_idiomatic_python.

Why it was moved

Github gists don't support Pull Requests or any notifications, which made it impossible for me to maintain this (surprisingly popular) gist with fixes, respond to comments and so on. In the interest of maintaining the quality of this resource for others, I've moved it to a proper repo. Cheers!