Skip to content

Instantly share code, notes, and snippets.

View test-rgw-object-tags-lua.md
  • start ceph cluster with an RGW using vstart:
MON=1 OSD=1 MDS=0 MGR=1 RGW=1 ../src/vstart.sh -d -n
  • use the following lua script postrequest.lua:
RGWDebugLog("Post Request Request.RGWOp = " .. Request.RGWOp)

RGWDebugLog("number of http metadata entries is: " .. #Request.HTTP.Metadata)
for k, v in pairs(Request.HTTP.Metadata) do
View build-wireshark-fedora36.md

prerequisites

sudo dnf install glib2-devel
sudo dnf install libgcrypt-devel
sudo dnf install qt-devel
sudo dnf install qt6-qtbase-devel
sudo dnf install qt6-linguist-devel
sudo dnf install qt6-qttools-devel
sudo dnf install qt6-qt5compat-devel
View build-apache-kafka.md

Instructions for fedora 36

Java

install java. curently gradle does not work with jdk higher than 11, so we would need to:

sudo dnf install java-11-openjdk-devel.x86_64

if other version is already installed, use:

sudo alternatives --config java
View mqtt-converter-todo.md
View delayed_http_server.py
from http.server import ThreadingHTTPServer, BaseHTTPRequestHandler
import threading
import time
import sys
class RequestHandler(BaseHTTPRequestHandler):
def do_GET(self):
print("active threads: ", str(threading.active_count()))
self.send_response(200)
View rgw_notification_extensions.go
// extensions to: https://docs.aws.amazon.com/sdk-for-go/api/service/s3/
type RGWNotificationConfigurationFilter struct {
// A container for object key name prefix and suffix filtering rules.
Key *KeyFilter `locationName:"S3Key" type:"structure"`
Metadata *RGWMapFilter `locationName:"RGWMetadata" type:"structure"`
Tag *RGWMapFilter `locationName:"RGWTag" type:"structure"`
// contains filtered or unexported fields
}
View ransomware.lua
-- calculate entropy of an object
function object_entropy(full_name)
local byte_hist = {}
local byte_hist_size = 256
for i = 1,byte_hist_size do
byte_hist[i] = 0
end
local total = 0
View measure-rgw-latency.py
# assuming we are running a vstart cluster
import subprocess
import time
import sys
def system(cmd):
output = subprocess.check_output(cmd, shell=True).decode(sys.stdout.encoding).strip()
return output
prev_rgw_sum = 0
View manual-rgw-scaling.md

Goal

The goal of this setup is to overload a single RGW so that adding another one would increase the throughput without overloaifng the OSDs.

Setup

  • machine with multiple nvme drives and enough CPU/RAM to run both Ceph and the clients. e.g.
$ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda           8:0    0 893.8G  0 disk 
└─sda1        8:1    0 893.8G  0 part /
View notifications_test_http.md

start cluster:

MON=1 OSD=1 MDS=0 MGR=0 RGW=1 ../src/vstart.sh -n -d 

start HTTP endpoint:

wget https://gist.githubusercontent.com/mdonkers/63e115cc0c79b4f6b8b3a6b797e485c7/raw/a6a1d090ac8549dac8f2bd607bd64925de997d40/server.py
python server.py 10900