This document is now outdated, thanks for the feedback everyone. This is basically what I will be trying out next:
-
Right now we have the need to compute histogram data on large files with cache support. This can work great by having a reducer process agregating results from mappers. this works because we have one type of reducer and one type of mapper with a minimum map/reduce ratio among the workers.
-
What happens if in the future we need something like A <- B <- C. This will require at least three processes with two of them waiting. We need to make
The purpose of this document is to explain the design decisions being taken for the multiprocessing and IPC architecture of binglide v2.
The main idea that this document ends up advocating is to have N workers for each type of service. In this scenario each worker knows how to do only one task. Those workers receive tasks from the broker who is responsable for splitting jobs into smaller tasks accoring to the cache. This endsup being hard
#!/usr/bin/env python3 | |
# -*- coding: utf-8 -*- | |
import collections | |
import zmq | |
import ipc | |
# name: wapiflapi | |
function _git_branch_name | |
command git symbolic-ref HEAD ^/dev/null | sed -e 's|^refs/heads/||' | |
end | |
function _git_repo_name | |
basename (git rev-parse --show-toplevel ^/dev/null) ^/dev/null | |
end |
#!/bin/sh | |
source ./venv/bin/activate | |
"$@" |
{ | |
repositoryOwner(login: "github") { | |
login | |
avatarUrl | |
repositories(first: 10, isFork: false, orderBy: {field: CREATED_AT, direction: DESC}) { | |
edges { | |
node { | |
description | |
createdAt | |
stargazers { |
from graphql.execution import ExecutionResult | |
from .schema import schema | |
import datetime | |
import dateutil.parser | |
import dateutil.tz | |
import monotonic |
#!/bin/bash | |
mkdir -p /tmp/tftp | |
wget https://boot.netboot.xyz/ipxe/netboot.xyz.efi -O /tmp/tftp/netboot.xyz.efi | |
# /!\ Make sure the router is OK. | |
ROUTER=$(ip r | grep default | cut -d ' ' -f3) | |
# Make sure interface is the right one, | |
# and dhcp-range should be a subset of the real range. |
<!DOCTYPE html> | |
<html lang="en"> | |
<head><title>Self Share Screen</title></head> | |
<body><video id="video" autoplay></video></body> | |
<script> | |
navigator.mediaDevices.getDisplayMedia({ | |
video: { cursor: "never" }, | |
audio: false | |
}).then(function (stream) { | |
document.getElementById("video").srcObject = stream; |