Skip to content

Instantly share code, notes, and snippets.

View kachayev's full-sized avatar
Fighting for freedom

Oleksii Kachaiev kachayev

Fighting for freedom
View GitHub Profile
kachayev /
Last active January 25, 2022 17:27
Converting RL loop (PPO, DQN, etc) into a selfplay setting by changing 2 lines
Imagine we have a training loop for an agent. E.g. PPO, or DQN, or whatever.
What is the easiest way to convert this into a selfplay?
To make this happen we want to run 2 identical loop: 1 loop for each agent.
But. There are 2 sync points: `env.reset()`, and each `env.step()`.
Is there a way to avoid duplicating code for setting/loading agent model, creating/updating
buffers, running training loops etc... Yes, it is. By utilizing generators protocol: `yield` and `send`.
kachayev /
Created May 2, 2021 02:40
Efficient Divide-and-Conquer using compression over little endian
from functools import reduce
from itertools import compress
from operator import mul
def iterate(f, initial):
"""Keeps applying function `f` to accumulated state, yielding each intermediate result.
E.g. iterate(lambda k: k*k, 2) -> 2,4,16,64,256,65536,..."""
state = initial
while True:
kachayev /
Last active May 3, 2021 21:06
Simplified Encoder implementation for "NLP From Scratch: Translation with a Sequence to Sequence Network and Attention" tutorial
# The following is the simplification of EncoderRNN code from
# "NLP From Scratch: Translation with a Sequence to Sequence Network and Attention"
# PyTorch tutorial (link:
# In fact, `nn.GRU` module can execute loop mechanics over the given input sequence when provided
# with such. In the tutorial, the sentence is presented as a tensor of 1-word sequences hence the
# loop is handled in `train` and `evaluation` "manually". Given the fact the `nn.Embedding` handles
# set of indecies and `nn.GRU` executes loops when given (seq_len, batch_size, elem_dim) shaped
# input, encoder could be constructed as the following:

Same data structure that HPACK is using right now:

Benchmark                                                      Mode  Cnt         Score        Error  Units
QpackStaticTableBenchmark.findFieldNameAndValueMatch          thrpt    5   1829339.334 ± 121376.837  ops/s
QpackStaticTableBenchmark.findFieldNameRefForEmptyField       thrpt    5   2341547.888 ±  27061.256  ops/s
QpackStaticTableBenchmark.findFieldNameRefForMultipleMatches  thrpt    5   3637243.304 ±  74366.646  ops/s
QpackStaticTableBenchmark.findFieldNameRefForSingleMatch      thrpt    5   6284579.831 ± 247083.622  ops/s
QpackStaticTableBenchmark.findFieldNotFound                   thrpt    5  67002791.898 ± 372832.308  ops/s

Aleph, Part I

A lot has happened these last few weeks. The key focus was on implementing new features, so let's enumerate all of them.


  • #422 "Handle WebSocket handshake timeouts".

Initially implemented by Denis Shilov. Thanks a lot! I've reimplemented the functionality partially to avoid usage of Manifold's timer. Mostly becase of performance considerations. I've also opened the same issue for Netty as this functionality is not covered by built-in websocket protocol handlers. And it was already implemented here by Qin Shicheng.

(require '[manifold.deferred :as d])
(defn long-boring-computation []
(let [result (d/deferred)]
(dotimes [_ 100]
(when-not (d/realized? result)
;; block the thread is still a bad idea :(
(Thread/sleep 1e3)))
(d/success! result "I'm done with this"))
kachayev /
Last active December 12, 2022 16:28
A few thoughts on Aleph development

Aleph, Async, HTTP, Clojure

I've been working with Aleph rougly for last 5 years, actively contributing to the library for last 2 (or so). I also put some effort into spreading the word about it, including educational tech talks, like "Deep HTTP Dive Throught Aleph & Netty". But the more I talk to people the more confusion I find, mostly about how Aleph works and what can you expect when adding it to your stack. Clojurists Together has recently announced Aleph to get Q1 funding, I think it's a good time to share my priorities and thoughts on development plans that were mentioned in the blog post. Hope the community would find it interesting and helpful.

Aleph describes itself as "asynchronous communication for Clojure" library. And you should probably pay a good portion of your attention to the first word: "asynchronous".

kachayev /
Last active July 31, 2022 12:26
Lazy developer's approach to do exercises on Propositional Logic
# -*- coding: utf-8 -*-
from tabulate import tabulate
class Var(object):
def __init__(self, name): = name
self.value = None
def bind(self, value):
(defn fetch-user
(fetch-item id))
(defn fetch-post
(fetch-item id))
(defn timeline-ids
kachayev /
Last active May 31, 2024 09:34
Channels Are Not Enough or Why Pipelining Is Not That Easy