Skip to content

Instantly share code, notes, and snippets.

View randomstatistic's full-sized avatar

Jeff Wartes randomstatistic

View GitHub Profile
@randomstatistic
randomstatistic / AsyncFunctionAction.scala
Created December 9, 2016 18:09
Gatling classes for testing an arbitrary async function
import io.gatling.commons.stats.{KO, OK}
import io.gatling.commons.util.TimeHelper._
import io.gatling.core.Predef._
import io.gatling.core.action.{Action, ExitableAction}
import io.gatling.core.action.builder.ActionBuilder
import io.gatling.core.stats.StatsEngine
import io.gatling.core.stats.message.ResponseTimings
import io.gatling.core.structure.ScenarioContext
import io.gatling.core.util.NameGen
@randomstatistic
randomstatistic / Performance
Last active October 7, 2019 21:44
Tool notes
Java:
http://java-performance.com/
Linux:
perf (https://perf.wiki.kernel.org/index.php/Main_Page)
perf can be a bit finicky to set up though, One of the most interesting payoffs for the effort is getting cool flame graphs of cpu usage, like this example:
http://www.brendangregg.com/FlameGraphs/cpu-mixedmode-flamegraph-java.svg (from the blog post http://techblog.netflix.com/2015/07/java-in-flames.html)
Install perf:
@randomstatistic
randomstatistic / ConfigMask.scala
Created June 21, 2019 18:54
Mask sensitive TypeSafe-Config values for rendering
import java.util.Map.Entry
import com.typesafe.config.{ Config, ConfigRenderOptions, ConfigValue, ConfigValueFactory }
import org.mindrot.jbcrypt.BCrypt
import scala.io.{ Codec, Source }
import scala.collection.JavaConverters._
import scala.util.Try
import scala.util.matching.Regex
@randomstatistic
randomstatistic / MGet.scala
Last active October 9, 2018 18:10
Get 200 random keys from a redis cluster
import io.lettuce.core.RedisURI
import io.lettuce.core.cluster.{ ClusterClientOptions, ClusterTopologyRefreshOptions, RedisClusterClient }
import scala.util.Random
object MGet {
def main(args: Array[String]): Unit = {
val host = args(0)
val port = args(1).toInt
@randomstatistic
randomstatistic / TryWithResources.scala
Created July 13, 2018 18:21
Scala try-with-resources equivelent
import java.io.Closable
import scala.util.control.NonFatal
import scala.util.{ Success, Try }
// This should really be scala-standard-lib.
object TryWithResources {
def withClose[T <: Closeable, V](closable: T)(f: T => V): V = {
(Try(f(closable)), Try(closable.close())) match {
case (Success(v), Success(_)) => v
case (a, b) => throw preferFirstException(a, b).get
@randomstatistic
randomstatistic / cached_by_extension.sh
Last active January 23, 2018 17:13
Investigate linux filesystem cache usage
#!/bin/bash
dir=$1
if [ "$1" == "" ]; then
echo "Must provide a directory as an argument"
exit 1
fi
ftypes=$(find $1 -type f -size +10c | grep -E ".*\.[a-zA-Z0-9]*$" | sed -e 's/.*\(\.[a-zA-Z0-9]*\)$/\1/' | sort | uniq)
import org.apache.zookeeper.server.{NIOServerCnxn, ZooKeeperServer}
import java.net.{ServerSocket, InetSocketAddress}
import kafka.server.{KafkaServer, KafkaConfig}
import kafka.producer.{ProducerConfig, Producer}
import java.util.Properties
import kafka.serializer.{DefaultEncoder, StringEncoder}
import java.io.File
import scala.util.Random
import kafka.admin.AdminUtils
@randomstatistic
randomstatistic / ClasspathReader.scala
Created April 24, 2017 16:57
Resource file reader
import java.io.FileNotFoundException
import java.util.zip.GZIPInputStream
import scala.io.Codec
object ClasspathReader {
def getResourceAsStream(filename: String) = {
// why doesn't the more scala-like getClass.getResourceAsStream work?
val inputStream = Thread.currentThread().getContextClassLoader().getResourceAsStream(filename)
if (inputStream == null) throw new FileNotFoundException(s"Couldn't find $filename on the classpath")
@randomstatistic
randomstatistic / LeakyBucket.scala
Created June 15, 2016 17:38
Leaky bucket implementation in scala
import java.util.concurrent.locks.ReentrantLock
import scala.concurrent.{ExecutionContext, Future, Promise}
import scala.concurrent.duration._
class LeakyBucket(dripEvery: FiniteDuration, maxSize: Int) {
require(maxSize > 0, "A bucket must have a size > 0")
private val dripEveryNanos = dripEvery.toNanos
private val lock = new ReentrantLock()
@randomstatistic
randomstatistic / solr_or_match.txt
Last active May 24, 2016 16:39
Which OR'ed field matched a given solr doc?
The idea is to add calculated fields to the “fl” of your query that match the components of your query.
Solr 4.0+ allows calculated functions in your field list using a <field name>:<function> syntax, and one of the available functions is “query”.
This seems like a vaguely unpleasant way to determine which clauses matched, but I could see it working. I think it would go something like:
q=colA:foo OR colB:bar&fl=*,score,matchedColA:query({!standard q=“colA:foo”},0),matchedColB:query({!standard q=“colB:bar”},0)
Presumably the field matchedColA would be non-zero if colA:foo matched on that document, and matchedColB would be non-zero
if colB:bar matched.
(I’m actually not sure if “standard” works as the name of the default query parser, but whatever, the idea is that it needs to match the relevant bit of your query.)