Skip to content

Instantly share code, notes, and snippets.

@p4bl0-
p4bl0- / 00_readme.md
Last active October 12, 2023 09:09
A complete compiler for a simple language (in less than 150 LoC)

This project is a tiny compiler for a very simple language consisting of boolean expression.

The language has two constants: 1 for true and 0 for false, and 4 logic gates: ! (not), & (and), | (or), and ^ (xor).

It can also use parentheses to manage priorities.

Here is its grammar in BNF format:

expr ::= "0" | "1"

@rxwei
rxwei / ad-manifesto.md
Last active November 9, 2023 09:58
First-Class Automatic Differentiation in Swift: A Manifesto
In order to work the Bose QC35 Headset Bluetooth do the following:
- sudo pacman -Syu pulseaudio-alsa pulseaudio-bluetooth bluez bluez-libs bluez-utils
- turn off bluetooth from your computer
- sudo btmgmt ssp of
- gpasswd -a YOUR_USER lp
@marwei
marwei / how_to_reset_kafka_consumer_group_offset.md
Created November 9, 2017 23:39
How to Reset Kafka Consumer Group Offset

Kafka 0.11.0.0 (Confluent 3.3.0) added support to manipulate offsets for a consumer group via cli kafka-consumer-groups command.

  1. List the topics to which the group is subscribed
kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --describe

Note the values under "CURRENT-OFFSET" and "LOG-END-OFFSET". "CURRENT-OFFSET" is the offset where this consumer group is currently at in each of the partitions.

  1. Reset the consumer offset for a topic (preview)

Thread Pools

Thread pools on the JVM should usually be divided into the following three categories:

  1. CPU-bound
  2. Blocking IO
  3. Non-blocking IO polling

Each of these categories has a different optimal configuration and usage pattern.

@juanjux
juanjux / xrandr_multihead_multidpi.sh
Last active February 12, 2021 22:35
how to set xrandr for multihead X systems mixing normal and high DPI screens
#!/bin/sh
# Note: this works well on Gnome3. KDE 5.7 seems to have issues with the
# framebuffer (toolbars are over the place). 5.9 works well except for
# the desktop background image on the scaled monitor (0 fucks given) but
# for KDE you have to add the xrandr command in another config file:
# http://askubuntu.com/questions/299241/how-to-reset-kde-display-settings-after-a-move-to-a-new-machine
#
# tldr: upscale the lowdpi screen to match the resolution of the hdpi one
# and create a virtual framebuffer resolution that is the size of the combined
# resolutions
@olafurpg
olafurpg / AutomateScalafmtPlugin.scala
Last active September 5, 2018 07:58
Experiment to add reformatOnCompile settings to scalafmt sbt >v0.5
// Works for scalafmt 0.5.5
// From https://gist.github.com/hseeberger/03677ef75bfadb7663c3b41bb58c702b
// Thank you @hseeberger!
import org.scalafmt.bootstrap.ScalafmtBootstrap
import org.scalafmt.sbt.ScalafmtPlugin
import sbt._
import sbt.Keys._
import sbt.inc.Analysis

Generating Flame Graphs for Apache Spark

Flame graphs are a nifty debugging tool to determine where CPU time is being spent. Using the Java Flight recorder, you can do this for Java processes without adding significant runtime overhead.

When are flame graphs useful?

Shivaram Venkataraman and I have found these flame recordings to be useful for diagnosing coarse-grained performance problems. We started using them at the suggestion of Josh Rosen, who quickly made one for the Spark scheduler when we were talking to him about why the scheduler caps out at a throughput of a few thousand tasks per second. Josh generated a graph similar to the one below, which illustrates that a significant amount of time is spent in serialization (if you click in the top right hand corner and search for "serialize", you can see that 78.6% of the sampled CPU time was spent in serialization). We used this insight to spee

@jkpl
jkpl / Main.scala
Last active February 5, 2024 08:29
Ways to pattern match generic types in Scala
object Main extends App {
AvoidLosingGenericType.run()
AvoidMatchingOnGenericTypeParams.run()
TypeableExample.run()
TypeTagExample.run()
}
class Funky[A, B](val foo: A, val bar: B) {
override def toString: String = s"Funky($foo, $bar)"
}
@JoshRosen
JoshRosen / scala-lambda-serialization-with-lifted-local-defs.md
Last active June 12, 2021 16:35
Serialization of Scala closures that contain local defs

Serialization of Scala closures that contain local defs

Several Apache Spark APIs rely on the ability to serialize Scala closures. Closures may reference non-Serializable objects, preventing them from being serialized. In some cases (SI-1419 and others), however, these references are unnecessary and can be nulled out, allowing otherwise-unserializable closures to be serialized (in Spark, this nulling is performed by the ClosureCleaner).

Scala 2.12's use of Java 8 lambdas for implementing closures appears to have broken our ability to serialize closures which contain local defs. If we cannot resolve this problem, Spark will be unable to support Scala 2.12 and will be stuck on 2.10 and 2.11 forever.

As an example which illustrates this problem, the following closure has a nested localDef and is defined inside of a non-serializable class:

``