yuvalif

backtrace:

#0  0x0000561fd85bf9a4 in rgw::notify::Manager::tokens_waiter::token::~token (this=0x7f625e4e8bf8, __in_chrg=<optimized out>) at /home/yuvalif/ceph3/src/rgw/driver/rados/rgw_notify.cc:199
#1  0x0000561fd85c9d75 in rgw::notify::Manager::process_queue(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, spawn::basic_yield_context<boost::asio::executor_binder<void (*)(), boost::asio::any_io_
executor> >)::{lambda(spawn::basic_yield_context<boost::asio::executor_binder<void (*)(), boost::asio::any_io_executor> >)#7}::operator()(spawn::basic_yield_context<boost::asio::executor_binder<void (*)(), boost::asio::any_io_executor> >)
 const (__closure=__closure@entry=0x561fdf2abc18, yield=...) at /home/yuvalif/ceph3/src/rgw/driver/rados/rgw_notify.cc:457                                                                                                                    
#2  0x0000561fd85ca050 in spawn::detail::spawn_helper<boost::asio::executor_binder<void (*)(), boost::as

yuvalif

# on Fedora28 as root

if [ "$#" -ne 1 ]; then
    echo "Usage: $0 <master IP>"
    exit 1
fi

set -ex

MASTER_IP=$1

yuvalif

from http.server import ThreadingHTTPServer, BaseHTTPRequestHandler
import threading
from multiprocessing import Process
import random
import json
                        
class HTTPPostHandler(BaseHTTPRequestHandler):
    """HTTP POST hanler class storing the received events in its http server"""
    def do_POST(self):
        """implementation of POST handler"""

yuvalif

• run vstart cluster
• create the parts:

$ head -c 12M </dev/urandom > myfile1
$ head -c 12M </dev/urandom > myfile2

• create the bucket:

$ aws s3api create-bucket --bucket aws-bkt5 --endpoint-url http://localhost:8000

yuvalif

From RADOS to REDIS

Background

Bucket notifications are important building block in the object storage ecosystem. And persistent bucket notifications in particular, as they let the system overcome broker outages. However, since the persistent notifications are backed with a RADOS queue, they have a cost. Both in the extra load on the RADOS cluster, and with the inability to operate in environemnts where there is no RADOS backend. In this project, we would like to implement persistent bucket notifications in the RADOS Gateway using a Redis Queue. Combined with the "zipper" project we would be able to enjoy bucket notifications with backends like posix, dbstore, daos etc.

Note that on top of using RADOS for the notification queue, our code is depended with RADOS with its implementation of a distributed lock (to make sure that one and only one

yuvalif

generic setup

git config --global user.name "My Name"
git config --global user.email "my@email"
git config --global credential.helper cache
git config --global credential.helper "cache --timeout=86400"
git config --global diff.tool vimdiff
git config --global difftool.prompt false
git config --global merge.tool vimdiff
git config --global mergetool.prompt false

yuvalif

Tidy Up Song

Background

Static analysis is a powerful tool when searching for code issues. It does, however, have its costs. Careful tuning is required so that the developers are not overwhelmed with "false positive" detections, while on the other hand - critical errors are not missed. It also requires simple integration into the development flow, so that new issues are continuously detected and addressed.

In this project we use clang-tidy for automated static analysis

yuvalif

• vstart
• create bucket:

~/go/bin/hsbench -a 0555b35654ad1656d804 -s h7GhxuBLTrlhVUyxSPUKUV8r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q== \
  -u http://localhost:8000 -bp bk -b 1 -r default -m i

• create kafka topic:

aws --region=default --endpoint-url http://localhost:8000 sns create-topic --name=fishtopic \
  --attributes='{"push-endpoint": "kafka://localhost"}'

yuvalif

#!/usr/bin/env python3
"""
License: MIT License
Copyright (c) 2023 Miel Donkers

Very simple HTTP server in python for logging requests
Usage::
    ./server.py [<port>]
"""
from http.server import BaseHTTPRequestHandler, HTTPServer

yuvalif

locking a machine

first, use:

teuthology-lock --brief

to see if you already have a machine locked. if not, use:

teuthology-lock --lock-many 1 --machine-type smithi

to lock a machine.