Skip to content

Instantly share code, notes, and snippets.

@jmikola

jmikola/README.md Secret

Last active October 20, 2016 17:07
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jmikola/111994d5867058a2bdce28bbe83950de to your computer and use it in GitHub Desktop.
Save jmikola/111994d5867058a2bdce28bbe83950de to your computer and use it in GitHub Desktop.
Download ZIP
Replica set availability test
Raw
README.md

Install PHP driver 1.1.8:

pecl install -f mongodb-1.1.8

Download server.pem from:

https://github.com/mongodb/mongo-php-driver/tree/master/scripts/ssl

Create necessary database paths:

mkdir -p /tmp/cluster/a
mkdir -p /tmp/cluster/b
mkdir -p /tmp/cluster/c

Create a 3-node replica set with SSL:

mongod --replSet foo --sslMode requireSSL --sslPEMKeyFile /path/to/server.pem --port 3001 --dbpath /tmp/cluster/a
mongod --replSet foo --sslMode requireSSL --sslPEMKeyFile /path/to/server.pem --port 3002 --dbpath /tmp/cluster/b
mongod --replSet foo --sslMode requireSSL --sslPEMKeyFile /path/to/server.pem --port 3003 --dbpath /tmp/cluster/c

Connect to port 3001 and configure the replica set:

mongo --port 3001 --ssl --sslAllowInvalidCertificates

Configure the replica set:

rs.initiate({
    _id: "foo",
    members: [
        { _id: 1, host: "127.0.0.1:3001" },
        { _id: 2, host: "127.0.0.1:3002" },
        { _id: 3, host: "127.0.0.1:3003" }
    ]
});

Run this script. It should report the selected primary server and then dump the replica set topology:

Selected primary server: 127.0.0.1:3001

Dumping server topology:
 - 127.0.0.1:3001 is primary
 - 127.0.0.1:3002 is secondary
 - 127.0.0.1:3003 is secondary

Use iptables to drop traffic to port 3003, which appears last in the URI:

sudo iptables -A INPUT -p tcp --destination-port 3003 -j DROP

Wait for a new primary to be elected. Run the script again the driver will fail to connect to all servers and report an empty topology:

No suitable servers found (`serverselectiontryonce` set):
[connection error calling ismaster on '127.0.0.1:3001']
[connection error calling ismaster on '127.0.0.1:3002']
[Failed connecting to '127.0.0.1:3003': Connection timed out]

Dumping server topology:

Move port 3003 first in the URI. Run the script again and the driver will connect to the remaining hosts and select the primary:

Selected primary server: 127.0.0.1:3001

Dumping server topology:
 - 127.0.0.1:3001 is primary
 - 127.0.0.1:3002 is secondary

Move port 3003 to the middle of the seed list and the primary to the start of the seed list. Run the script again and the driver will fail to select the primary; however, the primary will be listed in the topology as "unknown":

No suitable servers found (`serverselectiontryonce` set):
[connection error calling ismaster on '127.0.0.1:3001']
[Failed connecting to '127.0.0.1:3003': Connection timed out]

Dumping server topology:
 - 127.0.0.1:3001 is unknown
 - 127.0.0.1:3002 is secondary

Leave port 3003 to the middle of the seed list and swap the primary and secondary ports so that the primary appears last in the seed list. Run the script again and the driver will select the primary but list no other nodes in the topology:

Selected primary server: 127.0.0.1:3001

Dumping server topology:
 - 127.0.0.1:3001 is primary

Clean up the iptables rule to restore traffic:

sudo iptables -D INPUT -p tcp --destination-port 3003 -j DROP
Raw
results.md

Below are results for running this test script on both the 1.1.8 and 1.2.0alpha3 PHP driver versions. In these tests, we have a three-node replica set running with SSL and have iptables configured to drop all traffic to 127.0.0.1:3003, which is one of the secondaries.

I then recreated the replica set without SSL and repeated the test for both driver versions. The results were the same, so I won't share those logs; however, it was good to rule out SSL.

I should note that regardless of whether an exception was raised on 1.1.8, libmongoc invoked the phongo_stream_failed() callback for the streams corresponding to nodes before 127.0.0.1:3003 in the seed list, which closes the socket. I believe the logic here is that even though libmongoc has successfully selected a server, it considers some nodes to have failed due to the timeout bug described in CDRIVER-1571.

tl;dr: the problem does not present itself in 1.2.0alpha3, which has removed the PHP streams API in favor of libmongoc's native socket handling and TLS. The issue presents itself in 1.1.8 irrespective of whether SSL is being used.

PHP driver 1.1.8 (PHP streams)

For all permutations of the URI seed list, server selection blocks for one second (i.e. {{connectTimeoutMS}}) before returning the selected server or raising an exception. Exceptions are raised only when the unavailable host appears after the primary in the seed list.

URI: mongodb://127.0.0.1:3001,127.0.0.1:3002,127.0.0.1:3003/

No suitable servers found (`serverselectiontryonce` set):
[connection error calling ismaster on '127.0.0.1:3001']
[connection error calling ismaster on '127.0.0.1:3002']
[Failed connecting to '127.0.0.1:3003': Connection timed out]

Dumping server topology:

------------------------------------------------------------
URI: mongodb://127.0.0.1:3001,127.0.0.1:3003,127.0.0.1:3002/

No suitable servers found (`serverselectiontryonce` set):
[connection error calling ismaster on '127.0.0.1:3001']
[Failed connecting to '127.0.0.1:3003': Connection timed out]

Dumping server topology:
 - 127.0.0.1:3001 is unknown
 - 127.0.0.1:3002 is secondary

------------------------------------------------------------
URI: mongodb://127.0.0.1:3002,127.0.0.1:3001,127.0.0.1:3003/

No suitable servers found (`serverselectiontryonce` set):
[connection error calling ismaster on '127.0.0.1:3002']
[connection error calling ismaster on '127.0.0.1:3001']
[Failed connecting to '127.0.0.1:3003': Connection timed out]

Dumping server topology:

------------------------------------------------------------
URI: mongodb://127.0.0.1:3002,127.0.0.1:3003,127.0.0.1:3001/

Selected primary server: 127.0.0.1:3001

Dumping server topology:
 - 127.0.0.1:3001 is primary

------------------------------------------------------------
URI: mongodb://127.0.0.1:3003,127.0.0.1:3001,127.0.0.1:3002/

Selected primary server: 127.0.0.1:3001

Dumping server topology:
 - 127.0.0.1:3001 is primary
 - 127.0.0.1:3002 is secondary

------------------------------------------------------------
URI: mongodb://127.0.0.1:3003,127.0.0.1:3002,127.0.0.1:3001/

Selected primary server: 127.0.0.1:3001

Dumping server topology:
 - 127.0.0.1:3002 is secondary
 - 127.0.0.1:3001 is primary

PHP driver 1.2.0alpha3 (libmongoc socket handling)

For all permutations of the URI seed list, server selection blocks for one second (i.e. connectTimeoutMS) before returning the selected server. No exceptions are raised, since the primary was available.

URI: mongodb://127.0.0.1:3001,127.0.0.1:3002,127.0.0.1:3003/

Selected primary server: 127.0.0.1:3001

Dumping server topology:
 - 127.0.0.1:3001 is primary
 - 127.0.0.1:3002 is secondary

------------------------------------------------------------
URI: mongodb://127.0.0.1:3001,127.0.0.1:3003,127.0.0.1:3002/

Selected primary server: 127.0.0.1:3001

Dumping server topology:
 - 127.0.0.1:3001 is primary
 - 127.0.0.1:3002 is secondary

------------------------------------------------------------
URI: mongodb://127.0.0.1:3002,127.0.0.1:3001,127.0.0.1:3003/

Selected primary server: 127.0.0.1:3001

Dumping server topology:
 - 127.0.0.1:3002 is secondary
 - 127.0.0.1:3001 is primary

------------------------------------------------------------
URI: mongodb://127.0.0.1:3002,127.0.0.1:3003,127.0.0.1:3001/

Selected primary server: 127.0.0.1:3001

Dumping server topology:
 - 127.0.0.1:3002 is secondary
 - 127.0.0.1:3001 is primary

------------------------------------------------------------
URI: mongodb://127.0.0.1:3003,127.0.0.1:3001,127.0.0.1:3002/

Selected primary server: 127.0.0.1:3001

Dumping server topology:
 - 127.0.0.1:3001 is primary
 - 127.0.0.1:3002 is secondary

------------------------------------------------------------
URI: mongodb://127.0.0.1:3003,127.0.0.1:3002,127.0.0.1:3001/

Selected primary server: 127.0.0.1:3001

Dumping server topology:
 - 127.0.0.1:3002 is secondary
 - 127.0.0.1:3001 is primary
Raw
test.php
<?php
$uri = 'mongodb://127.0.0.1:3001,127.0.0.1:3002,127.0.0.1:3003/';
$uriOptions = [
'replicaSet' => 'foo',
'ssl' => true,
'connectTimeoutMS' => 1000,
];
$driverOptions = [
// Options for ext-mongodb >= 1.2.0alpha2
'allow_invalid_hostname' => true,
'weak_cert_validation' => true,
// Options for ext-mongodb <= 1.2.0alpha1
'verify_peer' => false,
'verify_peer_name' => false,
];
$manager = new MongoDB\Driver\Manager($uri, $uriOptions, $driverOptions);
try {
$server = $manager->selectServer(new MongoDB\Driver\ReadPreference(MongoDB\Driver\ReadPreference::RP_PRIMARY));
printf("Selected primary server: %s:%s\n", $server->getHost(), $server->getPort());
} catch (MongoDB\Driver\Exception\Exception $e) {
echo $e->getMessage(), "\n";
}
echo "\nDumping server topology:\n";
foreach ($manager->getServers() as $server) {
if ($server->isPrimary()) {
$role = 'primary';
} elseif ($server->isSecondary()) {
$role = 'secondary';
} elseif ($server->isArbiter()) {
$role = 'arbiter';
} elseif ($server->isHidden()) {
$role = 'hidden';
} elseif ($server->isPassive()) {
$role = 'passive';
} else {
$role = 'unknown';
}
printf(" - %s:%s is %s\n", $server->getHost(), $server->getPort(), $role);
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment