test_rook_notifications.md

Minikube

install minikube

curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-latest.x86_64.rpm
rpm -Uvh minikube-latest.x86_64.rpm

install and run libvirt

dnf install @virtualization
systemctl start libvirtd
systemctl enable libvirtd

minikube cleanup

minikube delete
rm -rf ~/.minikube

prepare minikube for Rook

• we use the kvm2 dirver
• make sure that the user (non root) is part the libvirt group:

$ sudo usermod -aG libvirt $USER && newgrp libvirt

• run minikube (with 8 CPUs):

$ minikube start --cpus 8 --vm-driver=kvm2

• create an sdb partition in minikube

$ UUID=$(uuidgen)
$ IMAGE=/var/lib/libvirt/images/minikube-$UUID
$ sudo qemu-img create -f raw $IMAGE 30G
$ sudo virsh attach-disk minikube $IMAGE vdb --cache none --persistent

• restart minikube to use that partition:

$ minikube stop
$ minikube start

• verify it is there:

$ minikube ssh lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vda    253:0    0 19.5G  0 disk 
`-vda1 253:1    0 19.5G  0 part /mnt/vda1
vdb    253:16   0   30G  0 disk 

Rook

• get the Rook code:

git clone https://github.com/rook/rook.git

• enter the ceph yamls directory:

cd rook/cluster/examples/kubernetes/ceph

• install basic rook operator

kubectl apply -f crds.yaml -f common.yaml -f operator.yaml

since bucket notification support is still work-in-progress, replace image in: operator.yaml with: quay.io/ylifshit/rook-ceph

• install the ceph cluster

kubectl apply -f cluster-test.yaml

in order to workaround an issue with rabbitmq, replace the ceph image in: cluster-test.yaml with quay.io/ceph-ci/ceph:wip-yuval-fix-50611

• install the object store

kubectl apply -f object-test.yaml

and wait for the RGW to run:

kubectl -n rook-ceph get pod -l app=rook-ceph-rgw

to workaround the rabbitmq issue, change the following conf parameter in the RGW:

ceph -c ceph.conf config set client.rgw.8000 rgw_allow_secrets_in_cleartext true

now we can set a storage class, topics, notifications, and OBCs. Documentation is here

Logs

operator logs:

kubectl logs -l app=rook-ceph-operator -n rook-ceph -f 

RGW logs:

kubectl logs -l app=rook-ceph-rgw -n rook-ceph -f 

RabbitMQ

install the rabbitmq operator

kubectl apply -f "https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml"