Ceph RGW multisite tests suite

rgw_multisite_tests.md

Running Tests Locally

• you run the tests locally from your "build" directory in your test environment
• then you need to add a "test_multi.conf" file:

[DEFAULT]
num_zonegroups=1
num_zones=1
num_ps_zones=0
gateways_per_zone=1
no_bootstrap=false
log_level=5

Note that you would need "num_ps_zones=1" to run the pubsub tests. But in your case you can keep it at zero. The First time you run tests, you would need to have "no_bootstrap=false", but in any consequent run, you would need to set it to "no_bootstrap=true", so that the cluster is not re-created. If you want to change the setup (e.g. number of gateways per zone), you would need to re-create the cluster. I do that by running:

$ /path/to/ceph/src/mstop.sh c1 && /path/to/ceph/src/mstop.sh c2

(assuming I have 2 gateways). Setting "no_bootstrap=false" and invoking the test again.

To invoke the tests (after the conf file is ready), I use:

$ MON=1 OSD=1 MGR=0 MSD=0 RGW_MULTI_TEST_CONF=./test_multi.conf nosetests -s /path/to/ceph/src/test/rgw/test_multi.py

To run only notification and pubsub related tests, I use the -m flag for nosetest:

$ MON=1 OSD=1 MGR=0 MSD=0 RGW_MULTI_TEST_CONF=./test_multi.conf nosetests -s /path/to/ceph/src/test/rgw/test_multi.py -m "test_ps_*"

To run a specific test (e.g. "test_ps_s3_versioned_deletion"), use the following format:

$ MON=1 OSD=1 MGR=0 MSD=0 RGW_MULTI_TEST_CONF=./test_multi.conf nosetests -s /path/to/ceph/src/test/rgw/test_multi.py:test_ps_s3_versioned_deletion

The multisite setup is used in teuthology as well, however, in teuthology it cannot run tests that push notification to HTTP, AMQP, etc. This is why there is a variable called: "skip_push_tests" which is set to "True" in the file. If you run the tests locally, you can enable the "push" tests, by setting it to "False".

AMQP

• In order to run the amqp tests you need to install rabbitmq:

$ sudo dnf install rabbitmq-server

dont need to do anything else, since the test is running the rabbitmq broker by itself.

kafka

• In order to run the kafka tests you need to install kafka. this is little more complex than rabbitmq.
• install kafka: download and unzip [1]
• the tests are running zookeeper and kafka binaries (see the init_kafka() function), so you don't need to run them yourself
• unless you want to tests security, there is no need to change anything with the kafka configuration file
• even if you don't test security, you would still need to create one file on the kafka config directory, called "kafka_client_jaas.conf":

KafkaClient {
          org.apache.kafka.common.security.plain.PlainLoginModule required
          username="alice"
          password="alice-secret";
      };

when you run kafka tests you need to set the KAFKA_DIR env variable, e.g.:

$ MON=1 OSD=1 MGR=0 MSD=0 KAFKA_DIR=/path/to/kafka RGW_MULTI_TEST_CONF=./test_multi.conf nosetests -s /path/to/ceph/src/test/rgw/test_multi.py

[1] https://www.apache.org/dyn/closer.cgi?path=/kafka/2.6.0/kafka-2.6.0-src.tgz