this test assumes ceph cluster with RGW is deployed via
vstart
- create the "log' bucket:
aws --endpoint-url http://localhost:8000 s3 mb s3://all-logs
- create a bucket for standard logging:
radosgw-admin bucket logging get
install:
sudo dnf -y install epel-release
curl -s https://packagecloud.io/install/repositories/rabbitmq/erlang/script.rpm.sh | sudo bash
sudo dnf -y install erlang
curl -s https://packagecloud.io/install/repositories/rabbitmq/rabbitmq-server/script.rpm.sh | sudo bash
sudo dnf -y install rabbitmq-server
add conf to allow guest when server is not on localhost:
start a single zone cluster with a realm:
OSD=1 MON=1 MDS=0 MGR=0 ../src/test/rgw/test-rgw-multisite.sh 1
create a persistent topic with a bucket and notifications:
aws --region=zg1 --endpoint-url http://localhost:8001 sns create-topic --name=fishtopic --attributes='{"push-endpoint": "kafka://localhost", "persistent": "true"}'
aws --region=zg1 --endpoint-url http://localhost:8001 s3 mb s3://fish
aws --region=zg1 --endpoint-url http://localhost:8001 s3api put-bucket-notification-configuration --bucket fish --notification-configuration='{"TopicConfigurations": [{"Id": "notif1", "TopicArn": "arn:aws:sns:default::fishtopic", "Events": []}]}'
based on the data from here:
sudo dnf install http://rpmfind.net/linux/fedora/linux/releases/40/Everything/x86_64/os/Packages/j/jericho-html-3.3-30.fc40.noarch.rpm
sudo dnf install http://rpmfind.net/linux/fedora/linux/releases/40/Everything/x86_64/os/Packages/b/batik-util-1.14-13.fc40.noarch.rpm
sudo dnf install http://rpmfind.net/linux/fedora/linux/releases/40/Everything/x86_64/os/Packages/x/xmlgraphics-commons-2.9-3.fc40.noarch.rpm
sudo dnf install http://rpmfind.net/linux/fedora/linux/releases/40/Everything/x86_64/os/Packages/b/batik-css-1.14-13.fc40.noarch.rpm
sudo dnf install http://rpmfind.net/linux/fedora/linux/releases/40/Everything/x86_64/os/Packages/b/batik-1.14-13.fc40.noarch.rpm
sudo dnf install http://rpmfind.net/linux/fedora/linux/updates/testing/40/Everything/x86_64/Packages/d/ditaa-0.10-24.fc40.noarch.rpm
# bucket notification parser according to the "pacific" formatting: | |
# https://github.com/ceph/ceph/blob/pacific/src/rgw/rgw_pubsub.h | |
# Note: JSON formatting does not match the one define at: | |
# https://docs.aws.amazon.com/AmazonS3/latest/dev/notification-content-structure.html | |
# but includes all data except: metadata, tags and opaque data | |
import struct | |
import sys | |
def prase_string(buff, offset): |
backtrace:
#0 0x0000561fd85bf9a4 in rgw::notify::Manager::tokens_waiter::token::~token (this=0x7f625e4e8bf8, __in_chrg=<optimized out>) at /home/yuvalif/ceph3/src/rgw/driver/rados/rgw_notify.cc:199
#1 0x0000561fd85c9d75 in rgw::notify::Manager::process_queue(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, spawn::basic_yield_context<boost::asio::executor_binder<void (*)(), boost::asio::any_io_
executor> >)::{lambda(spawn::basic_yield_context<boost::asio::executor_binder<void (*)(), boost::asio::any_io_executor> >)#7}::operator()(spawn::basic_yield_context<boost::asio::executor_binder<void (*)(), boost::asio::any_io_executor> >)
const (__closure=__closure@entry=0x561fdf2abc18, yield=...) at /home/yuvalif/ceph3/src/rgw/driver/rados/rgw_notify.cc:457
#2 0x0000561fd85ca050 in spawn::detail::spawn_helper<boost::asio::executor_binder<void (*)(), boost::as
from http.server import ThreadingHTTPServer, BaseHTTPRequestHandler | |
import threading | |
from multiprocessing import Process | |
import random | |
import json | |
class HTTPPostHandler(BaseHTTPRequestHandler): | |
"""HTTP POST hanler class storing the received events in its http server""" | |
def do_POST(self): | |
"""implementation of POST handler""" |