Skip to content

Instantly share code, notes, and snippets.

Jeff Wartes randomstatistic

View GitHub Profile
@randomstatistic
randomstatistic / ConfigMask.scala
Created Jun 21, 2019
Mask sensitive TypeSafe-Config values for rendering
View ConfigMask.scala
import java.util.Map.Entry
import com.typesafe.config.{ Config, ConfigRenderOptions, ConfigValue, ConfigValueFactory }
import org.mindrot.jbcrypt.BCrypt
import scala.io.{ Codec, Source }
import scala.collection.JavaConverters._
import scala.util.Try
import scala.util.matching.Regex
@randomstatistic
randomstatistic / MGet.scala
Last active Oct 9, 2018
Get 200 random keys from a redis cluster
View MGet.scala
import io.lettuce.core.RedisURI
import io.lettuce.core.cluster.{ ClusterClientOptions, ClusterTopologyRefreshOptions, RedisClusterClient }
import scala.util.Random
object MGet {
def main(args: Array[String]): Unit = {
val host = args(0)
val port = args(1).toInt
@randomstatistic
randomstatistic / TryWithResources.scala
Created Jul 13, 2018
Scala try-with-resources equivelent
View TryWithResources.scala
import java.io.Closable
import scala.util.control.NonFatal
import scala.util.{ Success, Try }
// This should really be scala-standard-lib.
object TryWithResources {
def withClose[T <: Closeable, V](closable: T)(f: T => V): V = {
(Try(f(closable)), Try(closable.close())) match {
case (Success(v), Success(_)) => v
case (a, b) => throw preferFirstException(a, b).get
View ClasspathReader.scala
import java.io.FileNotFoundException
import java.util.zip.GZIPInputStream
import scala.io.Codec
object ClasspathReader {
def getResourceAsStream(filename: String) = {
// why doesn't the more scala-like getClass.getResourceAsStream work?
val inputStream = Thread.currentThread().getContextClassLoader().getResourceAsStream(filename)
if (inputStream == null) throw new FileNotFoundException(s"Couldn't find $filename on the classpath")
@randomstatistic
randomstatistic / AsyncFunctionAction.scala
Created Dec 9, 2016
Gatling classes for testing an arbitrary async function
View AsyncFunctionAction.scala
import io.gatling.commons.stats.{KO, OK}
import io.gatling.commons.util.TimeHelper._
import io.gatling.core.Predef._
import io.gatling.core.action.{Action, ExitableAction}
import io.gatling.core.action.builder.ActionBuilder
import io.gatling.core.stats.StatsEngine
import io.gatling.core.stats.message.ResponseTimings
import io.gatling.core.structure.ScenarioContext
import io.gatling.core.util.NameGen
@randomstatistic
randomstatistic / LeakyBucket.scala
Created Jun 15, 2016
Leaky bucket implementation in scala
View LeakyBucket.scala
import java.util.concurrent.locks.ReentrantLock
import scala.concurrent.{ExecutionContext, Future, Promise}
import scala.concurrent.duration._
class LeakyBucket(dripEvery: FiniteDuration, maxSize: Int) {
require(maxSize > 0, "A bucket must have a size > 0")
private val dripEveryNanos = dripEvery.toNanos
private val lock = new ReentrantLock()
View FixedBitSetPool.java
public class FixedBitSetPool {
public static final int poolSize = 10;
private static ArrayBlockingQueue<FixedBitSet> pool = new ArrayBlockingQueue<FixedBitSet>(poolSize);
// Ask for a FBS
public static FixedBitSet request(int size) {
FixedBitSet next = pool.poll();
if (next == null || next.length() < size) {
// if the size doesn't match, throw it away and return a new one of the requested size
View KafkaUtil.scala
import org.apache.zookeeper.server.{NIOServerCnxn, ZooKeeperServer}
import java.net.{ServerSocket, InetSocketAddress}
import kafka.server.{KafkaServer, KafkaConfig}
import kafka.producer.{ProducerConfig, Producer}
import java.util.Properties
import kafka.serializer.{DefaultEncoder, StringEncoder}
import java.io.File
import scala.util.Random
import kafka.admin.AdminUtils
View Performance
Java:
http://java-performance.com/
Linux:
perf (https://perf.wiki.kernel.org/index.php/Main_Page)
perf can be a bit finicky to set up though, One of the most interesting payoffs for the effort is getting cool flame graphs of cpu usage, like this example:
http://www.brendangregg.com/FlameGraphs/cpu-mixedmode-flamegraph-java.svg (from the blog post http://techblog.netflix.com/2015/07/java-in-flames.html)
Install perf:
@randomstatistic
randomstatistic / solr_or_match.txt
Last active May 24, 2016
Which OR'ed field matched a given solr doc?
View solr_or_match.txt
The idea is to add calculated fields to the “fl” of your query that match the components of your query.
Solr 4.0+ allows calculated functions in your field list using a <field name>:<function> syntax, and one of the available functions is “query”.
This seems like a vaguely unpleasant way to determine which clauses matched, but I could see it working. I think it would go something like:
q=colA:foo OR colB:bar&fl=*,score,matchedColA:query({!standard q=“colA:foo”},0),matchedColB:query({!standard q=“colB:bar”},0)
Presumably the field matchedColA would be non-zero if colA:foo matched on that document, and matchedColB would be non-zero
if colB:bar matched.
(I’m actually not sure if “standard” works as the name of the default query parser, but whatever, the idea is that it needs to match the relevant bit of your query.)
You can’t perform that action at this time.