djspiewak

Quick Tips for Fast Code on the JVM

I was talking to a coworker recently about general techniques that almost always form the core of any effort to write very fast, down-to-the-metal hot path code on the JVM, and they pointed out that there really isn't a particularly good place to go for this information. It occurred to me that, really, I had more or less picked up all of it by word of mouth and experience, and there just aren't any good reference sources on the topic. So… here's my word of mouth.

This is by no means a comprehensive gist. It's also important to understand that the techniques that I outline in here are not 100% absolute either. Performance on the JVM is an incredibly complicated subject, and while there are rules that almost always hold true, the "almost" remains very salient. Also, for many or even most applications, there will be other techniques that I'm not mentioning which will have a greater impact. JMH, Java Flight Recorder, and a good profiler are your very best friend! Mea

djspiewak

Schedulers

As Cats Effect is a runtime system, it ultimately must deal with the problem of how best to execute the programs which are defined using its concrete implementation (IO). Fibers are an incredibly powerful model, but they don't map 1:1 or even 1:n with any JVM or JavaScript construct, which means that some interpretation is required. The fashion in which this is achieved has a profound impact on the performance and elasticity of programs written using IO.

This is true across both the JVM and JavaScript, and while it seems intuitive that JavaScript scheduling would be a simpler problem (due to its single-threaded nature), there are still some significant subtleties which become relevant in real-world applications.

JVM

IO programs and fibers are ultimately executed on JVM threads, which are themselves mapped directly to kernel threads and, ultimately (when scheduled), to processors. Determining the optimal method of mapping a real-world, concurrent application down to kernel-level thr

djspiewak

A Study in Multi-Point Deadlocks

Deadlocks are extremely difficult to reason about sometimes. We are used to thinking about them in terms of contention over shared resources, with the pair of exclusive locks being a good and relatively canonical example of this phenomenon. However, sometimes you can find yourself in deadlock scenarios which are caused not so much by an improper sequencing of exclusivity, but rather by insufficient buffer capacity!

These kinds of scenarios are a lot rarer and much more difficult to diagnose and describe, which is why I found this particular puzzle so incredibly fascinating. The following is a screenshot from Cities Skylines (I added textual markers and arrows to make things easier to follow). All vehicles pictured are stationary and unable to move, indefinitely:

Do you see the deadlock? It took me a bit to understand it, but this situation can and does arise in software resource contention where it is dramatically harder to conc

djspiewak

Thread Pools

Thread pools on the JVM should usually be divided into the following three categories:

1. CPU-bound
2. Blocking IO
3. Non-blocking IO polling

Each of these categories has a different optimal configuration and usage pattern.

djspiewak

Explaining Miles's Magic

Miles Sabin recently opened a pull request fixing the infamous SI-2712. First off, this is remarkable and, if merged, will make everyone's life enormously easier. This is a bug that a lot of people hit often without even realizing it, and they just assume that either they did something wrong or the compiler is broken in some weird way. It is especially common for users of scalaz or cats.

But that's not what I wanted to write about. What I want to write about is the exact semantics of Miles's fix, because it does impose some very specific assumptions about the way that type constructors work, and understanding those assumptions is the key to getting the most of it his fix.

For starters, here is the sort of thing that SI-2712 affects:

def foo[F[_], A](fa: F[A]): String = fa.toString

djspiewak

Getting Started in Scala

This is my attempt to give Scala newcomers a quick-and-easy rundown to the prerequisite steps they need to a) try Scala, and b) get a standard project up and running on their machine. I'm not going to talk about the language at all; there are plenty of better resources a google search away. This is just focused on the prerequisite tooling and machine setup. I will not be assuming you have any background in JVM languages. So if you're coming from Python, Ruby, JavaScript, Haskell, or anywhere…  I hope to present the information you need without assuming anything.

Disclaimer It has been over a decade since I was new to Scala, and when I was new to Scala, I was coming from a Java and Ruby background. This has probably caused me to unknowingly make some assumptions. Please feel free to call me out in comments/tweets!

One assumption I'm knowingly making is that you're on a Unix-like platform. Sorry, Windows users.

Getting the JVM

djspiewak

[error] (testsJS / Test / fastLinkJS) org.scalajs.linker.frontend.optimizer.OptimizerCore$OptimizeException: The Scala.js optimizer crashed while optimizing constructor cats.effect.unsafe.SchedulerSuite.<init>;V: java.lang.AssertionError: assertion failed: Found Assign in expression position
[error] Methods attempted to inline:
[error] * constructor munit.FunSuite.<init>;V
[error] * munit.internal.MacroCompat$CompileErrorMacro.$init$;V
[error] * munit.FunFixtures.$init$;V
[error] * munit.TestOptionsConversions.$init$;V
[error] * munit.TestTransforms.$init$;V
[error] * munit.SuiteTransforms.$init$;V
[error] * munit.ValueTransforms.$init$;V
[error] * munit.BaseFunSuite.$init$;V

djspiewak

Git DMZ Flow

I've been asked a few times over the last few months to put together a full write-up of the Git workflow we use at RichRelevance (and at Precog before), since I have referenced it in passing quite a few times in tweets and in person. The workflow is appreciably different from GitFlow and its derivatives, and thus it brings with it a different set of tradeoffs and optimizations. To that end, it would probably be helpful to go over exactly what workflow benefits I find to be beneficial or even necessary.

• Two developers working on independent features must never be blocked by each other
  • No code freeze! Ever! For any reason!
• A developer must be able to base derivative work on another developer's work, without waiting for any third party
• Two developers working on inter-dependent features (or even the same feature) must be able to do so without interference from (or interfering with) any other parties
• Developers must be able to work on multiple features simultaneously, or at lea

djspiewak

Where does Cats Effect cancelation become practically unavoidable? I feel like I never really need to think about it.

Timeouts are the easiest practical example to think about. If you contemplate that whole chain, from request handling in the server layer through the fiber which processes that request, making new requests to upstream services, waiting on those results, etc… there's a lot of timeouts involved in that. Any time you have a timeout, you need to recursively cancel everything which rolls up under it, and you need that cancelation to have a few properties:

• It must be irreversible (if you have a timeout, you can't un-timeout; this also implies a degree of determinism)
• It must be respected promptly
• It must backpressure control flow (ensuring that the continuation is not yielded until all finalizers have completed, implying constituent resources are released)
• It must not create invalid states (use after free) or leaks (related to backpressure)

Some of these are actually in mutual confl

djspiewak

org.scalajs.linker.runtime.UndefinedBehaviorError: java.lang.NullPointerException
  at $throwNullPointerException(/Users/daniel/Development/Scala/cats-effect/series-3.6.x/tests/js/target/scala-2.13/cats-effect-tests-test-fastopt/main.js:67:9)
  at $n(/Users/daniel/Development/Scala/cats-effect/series-3.6.x/tests/js/target/scala-2.13/cats-effect-tests-test-fastopt/main.js:71:5)
  at cats.effect.std.MutexSpec.<init>(/Users/daniel/Development/Scala/cats-effect/series-3.6.x/tests/js/target/scala-2.13/cats-effect-tests-test-fastopt/file:/Users/daniel/Development/Scala/cats-effect/series-3.6.x/tests/shared/src/test/scala/cats/effect/std/MutexSpec.scala:30:37)
  at {anonymous}()(/Users/daniel/Development/Scala/cats-effect/series-3.6.x/tests/js/target/scala-2.13/cats-effect-tests-test-fastopt/file:/Users/daniel/Development/Scala/cats-effect/series-3.6.x/tests/shared/src/test/scala/cats/effect/std/MutexSpec.scala:28:13)
  at scala.scalajs.reflect.InvokableConstructor.newInstance(/Users/daniel/Development/Scala/cats-ef