Skip to content

Instantly share code, notes, and snippets.

View WJWH's full-sized avatar

Wander Hillen WJWH

View GitHub Profile
@WJWH
WJWH / c10k.ru
Created October 15, 2018 09:12
The double hijack trick, serving 65k connections from a single ruby process
# This server demo does a socket hijack in Rack and then saves the socket to a global variable
# to prevent it from being GCed when the Puma thread ends. It will then write "BEEP" to each
# socket every ten seconds to prevent the connection timing out. During testing, it easily
# handled up to 65523 connections, after which it ran into the `ulimit` for open file descriptors.
# The bit with the waiting area is there because a normal `Set` is not thread safe and it would
# drop socket due to race conditions. The `Queue` is thread safe and will make sure all sockets
# are preserved.
# run with `rackup -q -p 8000 -o 0.0.0.0 c10k.ru`
# testing: install `ab` and then run `ab -c 20000 -n 20000 <ip adress of server>:8000/
@WJWH
WJWH / redisparsing.hs
Created December 31, 2018 14:55
Parsing the redis replication stream with AttoParsec
{-# LANGUAGE OverloadedStrings #-}
module RedisParsing where
import Control.Exception
import qualified Data.ByteString as B
import Data.Attoparsec.ByteString as AB
import Data.Attoparsec.ByteString.Char8
import Data.List
import Network.Socket hiding (recv)
import Network.Socket.ByteString (recv, sendAll)
# omg this is such a long comment, good thing I can spread it out over multiple lines like this so it will be clear for my colleagues what this code will do
@WJWH
WJWH / gist:2d4cb80f2799256e8703d2e904d0b707
Last active August 25, 2019 08:24
Generic dockerfile for statically compiled Crystal applications
# See https://wjwh.eu/posts/2019-07-21-static-crystal-images.html for the reasoning behind each line
# Build stage
FROM durosoft/crystal-alpine:0.28.0 as build
COPY shard.yml shard.yml
RUN shards install --production
COPY . ./
RUN crystal build src/my_app.cr --release --static
{-# LANGUAGE OverloadedStrings #-}
-- Echo server program
module Main (main) where
import Control.Concurrent
import Control.Concurrent.MVar
import qualified Control.Exception as E
import Control.Monad (unless, forever, void)
import qualified Data.ByteString as S
import qualified Data.ByteString.Char8 as S8
(gdb) bt
#0 LOOKS_LIKE_CLOSURE_PTR (p=0x318)
at includes/rts/storage/ClosureMacros.h:255
#1 0x0000000000a044e6 in evacuate (p=0x420193ce48) at rts/sm/Evac.c:591
#2 0x0000000000a14ddb in scavenge_small_bitmap (p=0x420193ce48, size=1,
bitmap=0) at rts/sm/Scav.c:289
#3 0x0000000000a17692 in scavenge_stack (p=0x420193ce48,
stack_end=0x420193cf80) at rts/sm/Scav.c:1923
#4 0x0000000000a16e15 in scavenge_one (p=0x420193cbf8) at rts/sm/Scav.c:1484
#5 0x0000000000a17218 in scavenge_mutable_list (bd=0x4201500880, gen=0xcb88e0)
(gdb) x/80a 0x42009ffe28-80
0x42009ffdd8: 0x2d68 0x718018 <base_GHCziEventziPSQ_zdwunsafeInsertNew_info+2408>
0x42009ffde8: 0x42003216b1 0x8
0x42009ffdf8: 0x4200613ed9 0x7899c0 <ghczmbignum_GHCziNumziInteger_integerToWordzh_info+48>
0x42009ffe08: 0x769528 <base_GHCziEventziIoUring_zdwconstructUserdata_info+816> 0x0
0x42009ffe18: 0x1 0x825fd0 <stg_ret_p_info>
0x42009ffe28: 0x7ffff45e0940 0x75e7c8 <base_SystemziLinuxziIOziURingziRing_zdwpushSqe_info+264>
0x42009ffe38: 0x7ffff45dc040 0x7ffff45dc100
0x42009ffe48: 0x7ffff45e0940 0xa19
0x42009ffe58: 0x1ff 0xa19
{-# LANGUAGE OverloadedStrings #-}
module Main where
import Foreign.C.Types
import System.Posix.IO
import System.Posix.Types
import System.Linux.IO.URing
import System.Linux.IO.URing.PollEvent
@WJWH
WJWH / gist:7867b0726fa6d667bb9c456c52bc303c
Created July 20, 2020 18:56
Not segfaulting program
{-# LANGUAGE OverloadedStrings, BangPatterns, GeneralizedNewtypeDeriving, CPP #-}
module Main where
#include <sys/epoll.h>
import Foreign.C.Error (eNOENT, getErrno, throwErrno,
throwErrnoIfMinus1, throwErrnoIfMinus1_)
import Foreign.C.Types
import Foreign.Marshal hiding (void, newArray)
-- To benchmark, make sure to expose the GHC.Event.* in /libraries/base/base.cabal,
-- otherwise GHC will complain that they are hidden modules.
import GHC.Event.Manager hiding (closeFd)
import GHC.Event.Internal
import GHC.Event.IoUring as IoUring
import GHC.Event.EPoll as Epoll
import Control.Monad
import Data.Time
import System.Posix.IO