Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?

Minikube

install minikube

curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-latest.x86_64.rpm
rpm -Uvh minikube-latest.x86_64.rpm

install and run libvirt

dnf install @virtualization
systemctl start libvirtd
systemctl enable libvirtd

minikube cleanup

minikube delete
rm -rf ~/.minikube

prepare minikube for Rook

  • we use the kvm2 dirver
  • make sure that the user (non root) is part the libvirt group:
$ sudo usermod -aG libvirt $USER && newgrp libvirt
  • run minikube (with 8 CPUs):
$ minikube start --cpus 8 --vm-driver=kvm2
  • create an sdb partition in minikube
$ UUID=$(uuidgen)
$ IMAGE=/var/lib/libvirt/images/minikube-$UUID
$ sudo qemu-img create -f raw $IMAGE 30G
$ sudo virsh attach-disk minikube $IMAGE vdb --cache none --persistent
  • restart minikube to use that partition:
$ minikube stop
$ minikube start
  • verify it is there:
$ minikube ssh lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vda    253:0    0 19.5G  0 disk 
`-vda1 253:1    0 19.5G  0 part /mnt/vda1
vdb    253:16   0   30G  0 disk 

Rook

  • get the Rook code:
git clone https://github.com/rook/rook.git
  • enter the ceph yamls directory:
cd rook/cluster/examples/kubernetes/ceph
  • install basic rook operator
kubectl apply -f crds.yaml -f common.yaml -f operator.yaml

since bucket notification support is still work-in-progress, replace image in: operator.yaml with: quay.io/ylifshit/rook-ceph

  • install the ceph cluster
kubectl apply -f cluster-test.yaml

in order to workaround an issue with rabbitmq, replace the ceph image in: cluster-test.yaml with quay.io/ceph-ci/ceph:wip-yuval-fix-50611

  • install the object store
kubectl apply -f object-test.yaml

and wait for the RGW to run:

kubectl -n rook-ceph get pod -l app=rook-ceph-rgw

to workaround the rabbitmq issue, change the following conf parameter in the RGW:

ceph -c ceph.conf config set client.rgw.8000 rgw_allow_secrets_in_cleartext true

now we can set a storage class, topics, notifications, and OBCs. Documentation is here

Logs

operator logs:

kubectl logs -l app=rook-ceph-operator -n rook-ceph -f 

RGW logs:

kubectl logs -l app=rook-ceph-rgw -n rook-ceph -f 

RabbitMQ

install the rabbitmq operator

kubectl apply -f "https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment