test-topic-migration.md

Basic Case

• start cluser with realm (single zone):

OSD=1 MON=1 MDS=0 MGR=0 ../src/test/rgw/test-rgw-multisite.sh 1

• disable v2 topics:

bin/radosgw-admin -c run/c1/ceph.conf zonegroup modify --disable-feature=notification_v2
bin/radosgw-admin -c run/c1/ceph.conf period update
bin/radosgw-admin -c run/c1/ceph.conf period commit

• create a topic a bucket and a notification:

export AWS_SECRET_ACCESS_KEY=pencil
export AWS_ACCESS_KEY_ID=1234567890
aws --region=zg1 --endpoint-url http://localhost:8101 sns create-topic --name=fishtopic \
  --attributes='{"push-endpoint": "kafka://localhost"}'
aws --region=zg1 --endpoint-url http://localhost:8101 s3 mb s3://fish
aws --region=zg1 --endpoint-url http://localhost:8101 s3api put-bucket-notification-configuration \
  --bucket fish --notification-configuration='{"TopicConfigurations": [{"Id": "notif1", "TopicArn": "arn:aws:sns:default::fishtopic", "Events": []}]}'

• make sure notifications are working:

head -c 512 </dev/urandom > myfile
aws --endpoint-url http://localhost:8101 s3 cp myfile s3://fish

• change to v2:

bin/radosgw-admin -c run/c1/ceph.conf zonegroup modify --enable-feature=notification_v2
bin/radosgw-admin -c run/c1/ceph.conf period update
bin/radosgw-admin -c run/c1/ceph.conf period commit

• make sure notifications are still working:

aws --endpoint-url http://localhost:8101 s3 cp myfile s3://fish

• make sure REST topic modification operations are failing (with HTTP status 500):

aws --region=zg1 --endpoint-url http://localhost:8101 sns create-topic --name=fishtopic1 \
  --attributes='{"push-endpoint": "kafka://localhost"}'
aws --region=zg1 --endpoint-url http://localhost:8101 sns delete-topic --topic-arn="arn:aws:sns:default::fishtopic"
aws --region=zg1 --endpoint-url http://localhost:8101 sns set-topic-attributes --topic-arn="arn:aws:sns:default::fishtopic" \
  --attribute-name=OpaqueData

• and read operations returh the v1 topics/notifications:

aws --region=zg1 --endpoint-url http://localhost:8101 sns get-topic-attributes --topic-arn="arn:aws:sns:default::fishtopic"
aws --region=zg1 --endpoint-url http://localhost:8101 s3api get-bucket-notification-configuration --bucket fish

• make sure REST notification modification operations are failing (with HTTP status 500):

aws --region=zg1 --endpoint-url http://localhost:8101 s3api put-bucket-notification-configuration \
  --bucket fish --notification-configuration='{"TopicConfigurations": [{"Id": "notif1", "TopicArn": "arn:aws:sns:default::fishtopic", "Events": []}]}'
aws --region=zg1 --endpoint-url http://localhost:8101 s3api delete-bucket-notification-configuration \
  --bucket fish

• make sure radosgw-admin related erite operations are also failing and read operations returning the v1 topics/notifications:

bin/radosgw-admin -c run/c1/ceph.conf topic list
bin/radosgw-admin -c run/c1/ceph.conf topic get --topic=fishtopic
bin/radosgw-admin -c run/c1/ceph.conf topic rm --topic=fishtopic
bin/radosgw-admin -c run/c1/ceph.conf notification list --bucket=fish
bin/radosgw-admin -c run/c1/ceph.conf notification get --bucket=fish --notification-id=notif1
bin/radosgw-admin -c run/c1/ceph.conf notification rm --bucket=fish --notification-id=notif1

multisite case (when the 2ndary site does not have any topic/notifications)

• create the cluster with 2 zones:

OSD=1 MON=1 MDS=0 MGR=0 ../src/test/rgw/test-rgw-multisite.sh 2

• follow all the steps from above
• perform write oprations for topics and notifications - all should fail with HTTP status 500:

aws --region=zg2 --endpoint-url http://localhost:8201 sns create-topic --name=fishtopic \
  --attributes='{"push-endpoint": "kafka://localhost"}'
aws --region=zg2 --endpoint-url http://localhost:8201 sns delete-topic --topic-arn="arn:aws:sns:default::fishtopic"
aws --region=zg2 --endpoint-url http://localhost:8201 sns set-topic-attributes --topic-arn="arn:aws:sns:default::fishtopic" \
  --attribute-name=OpaqueData
aws --region=zg2 --endpoint-url http://localhost:8201 s3api put-bucket-notification-configuration \
  --bucket fish --notification-configuration='{"TopicConfigurations": [{"Id": "notif1", "TopicArn": "arn:aws:sns:default::fishtopic", "Events": []}]}'
aws --region=zg2 --endpoint-url http://localhost:8201 s3api delete-bucket-notification-configuration \
  --bucket fish

• perform read operations for topics and notifications - should fail with HTTP status 404 or return empty results:

aws --region=zg2 --endpoint-url http://localhost:8201 sns get-topic-attributes --topic-arn="arn:aws:sns:default::fishtopic"
aws --region=zg2 --endpoint-url http://localhost:8201 s3api get-bucket-notification-configuration --bucket fish

Persistent Notifications Case

• similar as the basic case, but with persistent topic:

aws --region=zg1 --endpoint-url http://localhost:8101 sns create-topic --name=fishtopic \
  --attributes='{"push-endpoint": "kafka://localhost"}'

• also, verify that the persistent queue is not deleted:

bin/radosgw-admin -c run/c1/ceph.conf topic stats --topic=fishtopic