djspiewak

Schedulers

As Cats Effect is a runtime system, it ultimately must deal with the problem of how best to execute the programs which are defined using its concrete implementation (IO). Fibers are an incredibly powerful model, but they don't map 1:1 or even 1:n with any JVM or JavaScript construct, which means that some interpretation is required. The fashion in which this is achieved has a profound impact on the performance and elasticity of programs written using IO.

This is true across both the JVM and JavaScript, and while it seems intuitive that JavaScript scheduling would be a simpler problem (due to its single-threaded nature), there are still some significant subtleties which become relevant in real-world applications.

JVM

IO programs and fibers are ultimately executed on JVM threads, which are themselves mapped directly to kernel threads and, ultimately (when scheduled), to processors. Determining the optimal method of mapping a real-world, concurrent application down to kernel-level thr

djspiewak

Greasing the Skids: Building Remote Teams

In the wake of the virus that-must-not-be-named (which most people misname anyway), it seems like everyone and their cat has posted some sort of opinion or how-to on making remote work, work. This is a good thing! Working remotely, particularly full-time, is hard! I've done it for my entire career (aside from an odd 14 month office period in the middle that we shall not speak of), but more relevantly, for the past two years I've been responsible for building, managing, and enabling an entirely remote team, distributed across nine timezones. Remote teams don't just happen by installing Slack and telling everyone to work on their couch: they require a lot of effort to do correctly and efficiently. But, done right, it can be a massive multiplier on your team efficiency and flexibility.

Here's how we do it. I'm going to attempt to structure this post more towards management than engineering, and so I apologize in advance if I assume terminology or knowledge which

djspiewak

Quick Tips for Fast Code on the JVM

I was talking to a coworker recently about general techniques that almost always form the core of any effort to write very fast, down-to-the-metal hot path code on the JVM, and they pointed out that there really isn't a particularly good place to go for this information. It occurred to me that, really, I had more or less picked up all of it by word of mouth and experience, and there just aren't any good reference sources on the topic. So… here's my word of mouth.

This is by no means a comprehensive gist. It's also important to understand that the techniques that I outline in here are not 100% absolute either. Performance on the JVM is an incredibly complicated subject, and while there are rules that almost always hold true, the "almost" remains very salient. Also, for many or even most applications, there will be other techniques that I'm not mentioning which will have a greater impact. JMH, Java Flight Recorder, and a good profiler are your very best friend! Mea

djspiewak

Thread Pools

Thread pools on the JVM should usually be divided into the following three categories:

1. CPU-bound
2. Blocking IO
3. Non-blocking IO polling

Each of these categories has a different optimal configuration and usage pattern.

djspiewak

Getting Started in Scala

This is my attempt to give Scala newcomers a quick-and-easy rundown to the prerequisite steps they need to a) try Scala, and b) get a standard project up and running on their machine. I'm not going to talk about the language at all; there are plenty of better resources a google search away. This is just focused on the prerequisite tooling and machine setup. I will not be assuming you have any background in JVM languages. So if you're coming from Python, Ruby, JavaScript, Haskell, or anywhere…  I hope to present the information you need without assuming anything.

Disclaimer It has been over a decade since I was new to Scala, and when I was new to Scala, I was coming from a Java and Ruby background. This has probably caused me to unknowingly make some assumptions. Please feel free to call me out in comments/tweets!

One assumption I'm knowingly making is that you're on a Unix-like platform. Sorry, Windows users.

Getting the JVM

djspiewak

Git DMZ Flow

I've been asked a few times over the last few months to put together a full write-up of the Git workflow we use at RichRelevance (and at Precog before), since I have referenced it in passing quite a few times in tweets and in person. The workflow is appreciably different from GitFlow and its derivatives, and thus it brings with it a different set of tradeoffs and optimizations. To that end, it would probably be helpful to go over exactly what workflow benefits I find to be beneficial or even necessary.

• Two developers working on independent features must never be blocked by each other
  • No code freeze! Ever! For any reason!
• A developer must be able to base derivative work on another developer's work, without waiting for any third party
• Two developers working on inter-dependent features (or even the same feature) must be able to do so without interference from (or interfering with) any other parties
• Developers must be able to work on multiple features simultaneously, or at lea

kimus

First of all, it just seems like doing anything with Oracle is obnoxiously painful for no good reason. It's the nature of the beast I suppose. cx_oracle is a python module that allows you to connect to an Oracle Database and issue queries, inserts, updates..usual jazz.

Linux

Step 1:

sudo apt-get install build-essential unzip python-dev libaio-dev

Step 2. Click here to download the appropriate zip files required for this. You'll need:

myusuf3

# Next time you need to install something with python setup.py -- which should be never but things happen. 

python setup.py install --record files.txt

# This will cause all the installed files to be printed to that directory.
# Then when you want to uninstall it simply run; be careful with the 'sudo'

cat files.txt | xargs sudo rm -rf