Skip to content

Instantly share code, notes, and snippets.

View manuzhang's full-sized avatar
🧒
Working from home

Manu Zhang manuzhang

🧒
Working from home
View GitHub Profile
@dcsobral
dcsobral / README.md
Created April 15, 2020 23:40
Scala Configurable Warnings

Pull request #8373 implementing issue 333 (but please note differences).

Pull request description follows:

This PR adds a -Wconf compiler flag that allows filtering and configuring compiler warnings (silence them, or turn them into errors).

It also integrates the fantastic silencer compiler plugin by @ghik into the compiler, which allows suppressing warnings locally using the @nowarn annotation.

An interesting question

This morning, one of my colleagues asked me the following interesting question:

So, I have been wondering about this question (which is not necessarily applicable to Kudu): when certain software says they use a single hot replica for failover, how does it handle confusion caused by transient network failure which breaks the communication between the primary and second ? In other words, how can such configuration guarantee that both nodes won't think of itself as the primary? Do they usually fall back to some third party as an arbitrator ? However, that third party may itself suffer random network partition with one or either of the nodes

This led to my brain-dumping for 20 minutes on Slack, which I then figured I'd copy-paste into a "blog post" gist.

My brain-dump

@johnynek
johnynek / TreeList.scala
Created December 14, 2018 19:45
Implementation of "Purely Functional Random Access Lists" by Chris Okasaki in scala. This gives O(1) cons and uncons, and 2 log_2 N lookup.
package org.bykn.list
import cats.Applicative
import cats.implicits._
/**
* Implementation of "Purely Functional Random Access Lists" by Chris Okasaki.
* This gives O(1) cons and uncons, and 2 log_2 N lookup.
*/
@TeemuKoivisto
TeemuKoivisto / HADOOP-ON-K8S.md
Last active December 1, 2022 12:21
How to install Hadoop to your local Kubernetes cluster

How to install Hadoop on your local Kubernetes cluster

Okey this is not the easiest way of running Hadoop on your local computer and probably you should instead just install it locally.

However if you really insist doing this here's how:

  1. Install kubectl, minikube and Docker if you don't already have it. I recommend using package-manager like Chocolatey. Minikube should install with VirtualBox as default driver which I recommend. When starting minikube we should increase its memory limit since our Hadoop node's pods need at least 2GB: minikube --memory 4096 --cpus 2 start (minikube's default is 1GB). NOTE: actually the Hadoop cluster by default uses about 10GB in memory limits and about 3GB running memory. From what I looked my k8s will overprovision to 300% of its capacity limits but use far less.
  2. Install helm. Then run helm init.
  3. Now you
<project>
...
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
@non
non / seeds.md
Last active April 12, 2024 09:18
Simple example of using seeds with ScalaCheck for deterministic property-based testing.

introduction

ScalaCheck 1.14.0 was just released with support for deterministic testing using seeds. Some folks have asked for examples, so I wanted to produce a Gist to help people use this feature.

simple example

These examples will assume the following imports:

@eed3si9n
eed3si9n / Main.scala
Last active August 5, 2017 13:33 — forked from xuwei-k/Build.scala
sbt plugin star ranking (using Gigahorse)
package sbt_plugin_ranking
import gigahorse._, support.okhttp.Gigahorse
import gigahorse.github.Github
import scala.concurrent._, duration._
object Main extends App{
val repo = """https:\/\/github.com\/([a-zA-Z0-9_\-]+)\/([a-zA-Z0-9_\-]+)""".r
val url = "http://www.scala-sbt.org/release/docs/Community-Plugins.html"
val html = io.Source.fromURL(url).mkString
@joyrexus
joyrexus / README.md
Last active February 24, 2024 15:16
collapsible markdown

collapsible markdown?

CLICK ME

yes, even hidden code blocks!

print("hello world!")
package hello
import java.io.ByteArrayOutputStream
import chisel3._
import chisel3.iotesters.{Driver, PeekPokeTester}
sealed trait Op
object Op{
case class GotoIfZero(dest: Byte, valueAddr: Byte) extends Op
package demo
import scala.tools.nsc.io.AbstractFile
import scala.tools.nsc.{Global, Phase}
import scala.tools.nsc.plugins.{Plugin, PluginComponent}
class DemoPlugin(val global: Global) extends Plugin {
import global._
override def init(options: List[String], error: String => Unit): Boolean = true