Host this php on php-fpm + nginx, each page hit creates TWO Pairs of connections to replicaset, FOUR Authentication done (primary + secondary) * 2

gistfile1.php

<?
// simple mongoclient dumpper
$a=getMongo();
$b=getMongo();

var_dump($a->getConnectionss());
print("===================================SECOND MONGO CONN REQUEST ===================================\n");
var_dump($b->getConnectionss());

function getMongo(){//{{{
            $mongo  = new MongoClient(MG_HOST, array(
                    'replicaSet' => MG_REPLICASET,
                    'password'=>MG_PWD,
                    'username'=>MG_USER,
                    'db'=>MG_DB,
                    'journal' => true,
                    "readPreference"=> MongoClient::RP_SECONDARY_PREFERRED
                )
            );

            $mgdb = $mongo->{MG_DB};
        return $mongo;
}
?>

When test under ngnix (1 worker).
TCP socket level is persistent:
tcp 0 0 ip-10-114-149-254.ec2:37426 unknown.servercentral:10044 ESTABLISHED 31867/php-fpm
tcp 0 0 ip-10-114-149-254.ec2:56806 ge0-41.mongo101.iad1.:10044 ESTABLISHED 31867/php-fpm

but sent 4 auth cmd to mongo (3 master 2 secondary)

Output:

array(2) {
  [0]=>
  array(3) {
    ["hash"]=>
    string(129) "xxx"
    ["server"]=>
    array(7) {
      ["host"]=>
      string(30) "xx"
      ["port"]=>
      int(10044)
      ["repl_set_name"]=>
      string(28) "xx"
      ["database"]=>
      string(16) "mixerbox_new_ssd"
      ["username"]=>
      string(7) "xx"
      ["auth_hash"]=>
      string(32) "xx"
      ["pid"]=>
      int(32492)
    }
    ["connection"]=>
    array(8) {
      ["last_ping"]=>
      int(1375308556)
      ["last_ismaster"]=>
      int(1375308557)
      ["ping_ms"]=>
      int(1)
      ["connection_type"]=>
      int(2)
      ["connection_type_desc"]=>
      string(7) "PRIMARY"
      ["max_bson_size"]=>
      int(16777216)
      ["tag_count"]=>
      int(0)
      ["tags"]=>
      array(0) {
      }
    }
  }
  [1]=>
  array(3) {
    ["hash"]=>
    string(129) "xx"
    ["server"]=>
    array(7) {
      ["host"]=>
      string(30) "xx"
      ["port"]=>
      int(10044)
      ["repl_set_name"]=>
      string(28) "xx"
      ["database"]=>
      string(16) "xx"
      ["username"]=>
      string(7) "xx"
      ["auth_hash"]=>
      string(32) "xx"
      ["pid"]=>
      int(32492)
    }
    ["connection"]=>
    array(8) {
      ["last_ping"]=>
      int(1375308556)
      ["last_ismaster"]=>
      int(1375308557)
      ["ping_ms"]=>
      int(3)
      ["connection_type"]=>
      int(4)
      ["connection_type_desc"]=>
      string(9) "SECONDARY"
      ["max_bson_size"]=>
      int(16777216)
      ["tag_count"]=>
      int(0)
      ["tags"]=>
      array(0) {
      }
    }
  }
}
===================================SECOND MONGO CONN REQUEST ===================================
array(2) {
  [0]=>
  array(3) {
    ["hash"]=>
    string(129) "xx"
    ["server"]=>
    array(7) {
      ["host"]=>
      string(30) "xx"
      ["port"]=>
      int(10044)
      ["repl_set_name"]=>
      string(28) "xx"
      ["database"]=>
      string(16) "xx"
      ["username"]=>
      string(7) "ilivapp"
      ["auth_hash"]=>
      string(32) "xx"
      ["pid"]=>
      int(32492)
    }
    ["connection"]=>
    array(8) {
      ["last_ping"]=>
      int(1375308556)
      ["last_ismaster"]=>
      int(1375308557)
      ["ping_ms"]=>
      int(1)
      ["connection_type"]=>
      int(2)
      ["connection_type_desc"]=>
      string(7) "PRIMARY"
      ["max_bson_size"]=>
      int(16777216)
      ["tag_count"]=>
      int(0)
      ["tags"]=>
      array(0) {
      }
    }
  }
  [1]=>
  array(3) {
    ["hash"]=>
    string(129) "xxx"
    ["server"]=>
    array(7) {
      ["host"]=>
      string(30) "xx"
      ["port"]=>
      int(10044)
      ["repl_set_name"]=>
      string(28) "set-xx"
      ["database"]=>
      string(16) "xx"
      ["username"]=>
      string(7) "xx"
      ["auth_hash"]=>
      string(32) "xx"
      ["pid"]=>
      int(32492)
    }
    ["connection"]=>
    array(8) {
      ["last_ping"]=>
      int(1375308556)
      ["last_ismaster"]=>
      int(1375308557)
      ["ping_ms"]=>
      int(3)
      ["connection_type"]=>
      int(4)
      ["connection_type_desc"]=>
      string(9) "SECONDARY"
      ["max_bson_size"]=>
      int(16777216)
      ["tag_count"]=>
      int(0)
      ["tags"]=>
      array(0) {
      }
    }
  }
}

Each connection has Two Servers, 1 to Primary and 1 To Secondary.
ALl connections have ["pid"]=> int(32492) ( I guess it is process Id)

It seems, alone with 2 connections (one to primary and one to secondary), there are TWO topology connections built, one for primary and one to secondary) - seems additional overheads for me.

Particularly this worries me:
WARN: discover_topology: ismaster worked, but we need to remove the seed host's connection ....

[31-Jul-2013 22:32:07 UTC] PHP Notice: CON INFO: ismaster: last ran at 1375309927 in /var/app/current/testmongo.php on line 47
[31-Jul-2013 22:32:07 UTC] PHP Notice: CON WARN: discover_topology: ismaster worked, but we need to remove the seed host's connection in /var/app/current/testmongo.php on line 47
[31-Jul-2013 22:32:07 UTC] PHP Notice: CON FINE: mongo_connection_destroy: Destroying connection object for $MASKED$;3367 in /var/app/current/testmongo.php on line 47
[31-Jul-2013 22:32:07 UTC] PHP Notice: CON FINE: mongo_connection_destroy: Closing socket for $MASKED$;3367. in /var/app/current/testmongo.php on line 47
[31-Jul-2013 22:32:07 UTC] PHP Notice: CON FINE: discover_topology: ismaster worked in /var/app/current/testmongo.php on line 47

@leozc: Please format the var_dump output with monospace formatting (three ` characters on a line before and after will do it). It's a bit hard to read in the current state.

Re:jmlkola.

@leozc: The error you're seeing is because the connection string hostname doesn't match what the servers themselves identify as. Take a look at mongo_connection_ismaster and look for the retval = 3 case.

Very nice hint! It ends up it is a domain name alias problem, let me investigate further...

Take a look at mongo_connection_ismaster and look for the retval = 3 case. helps