kod3r

Turning PostgreSQL into a queue serving 10,000 jobs per second

RDBMS-based job queues have been criticized recently for being unable to handle heavy loads. And they deserve it, to some extent, because the queries used to safely lock a job have been pretty hairy. SELECT FOR UPDATE followed by an UPDATE works fine at first, but then you add more workers, and each is trying to SELECT FOR UPDATE the same row (and maybe throwing NOWAIT in there, then catching the errors and retrying), and things slow down.

On top of that, they have to actually update the row to mark it as locked, so the rest of your workers are sitting there waiting while one of them propagates its lock to disk (and the disks of however many servers you're replicating to). QueueClassic got some mileage out of the novel idea of randomly picking a row near the front of the queue to lock, but I can't still seem to get more than an an extra few hundred jobs per second out of it under heavy load.

So, many developers have started going straight t

kod3r

Idiomatic Data Fetching using React Hooks

This post has been written in collaboration with @klervicn

Virtually all web apps and websites need to pull data from a server, usually through a JSON-returning API. When it comes to integrating data fetching in React component, the "impedence mismatch" between the React components, which are declarative and synchronous, and the HTTP requests, which are imperative and asynchronous, is often problematic.

Many apps use third-party libraries such as Redux or Apollo Client to abstract it away. This requires extra dependencies, and couple your app with a specific library to perform data fetching. In most cases, what we want is a direct way to integrate plain HTTP requests (e.g. using native fetch) for usage in React components.

Here we will discuss how we can use React Hooks to do this in an elegant, scalable manner.

kod3r

# A syllable dictionary I built with Python. See https://zacs.site/blog/building-a-syllable-dictionary-with-python.html for the full writeup.
# 
# This work is licensed under a Creative Commons Attribution 4.0 International License
# See https://zacs.site/disclaimers.html for more information.
# © 2012-2019 Zachary Szewczyk.

a,1
a,1
aa,1
aal,-1

kod3r

High-Performance Matrix Multiplication

This is a short post that explains how to write a high-performance matrix multiplication program on modern processors. In this tutorial I will use a single core of the Skylake-client CPU with AVX2, but the principles in this post also apply to other processors with different instruction sets (such as AVX512).

Intro

Matrix multiplication is a mathematical operation that defines the product of

kod3r

dlib and OpenCV in Android app

• Create project and get OpenCV working

Follow the instructions on the following link. NOTICE: When creating the new project, on the C++ Support tab, select C++11 from the dropdown. Link: https://github.com/leadrien/opencv_native_androidstudio#how-to-create-the-native-opencv-project-from-scratch

• Download correct version of dlib

kod3r

Idiomatic React Testing Patterns

Testing React components seems simple at first. Then you need to test something that isn't a pure interaction and things seem to break down. These 4 patterns should help you use a pattern that is repeatable and readable for the type of test you need.

Setup

I recommend doing all setup in the most functional way possible. If you can avoid it, don't set variables in a beforeEach. This will help ensure tests are isolated and make things a bit easier to reason about. I use a pattern

kod3r

When setting up (that's immediately when user enables it in settings) Opera VPN sends few API requests to https://api.surfeasy.com to obtain credentials and proxy IPs, see below, also see The Oprah Proxy.

The browser then talks to a proxy de0.opera-proxy.net (when VPN location is set to Germany), it's IP address can only be resolved from within Opera when VPN is on, it's 185.108.219.42 (or similar, see below). It's an HTTP/S proxy which requires auth.

When loading a page with Opera VPN enabled, the browser sends a lot of requests to de0.opera-proxy.net with Proxy-Authorization request header.

The Proxy-Authorization header decoded: CC68FE24C34B5B2414FB1DC116342EADA7D5C46B:9B9BE3FAE674A33D1820315F4CC94372926C8210B6AEC0B662EC7CAD611D86A3 (that's sha1(device_id):device_password, where device_id and device_password come from the POST /v2/register_device API call, please note that this decoded header is from another Opera installation and thus contains

kod3r

Using Google Cloud Vision API's OCR to extract text from photos and scanned documents

Just a quickie test in Python 3 (using Requests) to see if Google Cloud Vision can be used to effectively OCR a scanned data table and preserve its structure, in the way that products such as ABBYY FineReader can OCR an image and provide Excel-ready output.

The short answer: No. While Cloud Vision provides bounding polygon coordinates in its output, it doesn't provide it at the word or region level, which would be needed to then calculate the data delimiters.

On the other hand, the OCR quality is pretty good, if you just need to identify text anywhere in an image, without regards to its physical coordinates. I've included two examples:

####### 1. A low-resolution photo of road signs

kod3r

Efficient Estimation of Word Representations in Vector Space

Introduction

• Introduces techniques to learn word vectors from large text datasets.
• Can be used to find similar words (semantically, syntactically, etc).
• Link to the paper
• Link to open source implementation

Model Architecture

kod3r

Installing CartoDB on Ubuntu Server 12.04

Get CartoDB Rails application source

sudo apt-get update

sudo apt-get install git

cd ~

git clone --recursive https://github.com/CartoDB/cartodb20.git