This document explains how the library handles TCP connections internally. This will cover both HTTP and Memcached connections
During initial bootstrap, the library effectively loops over the list of hosts twice, the first loop will attempt bootstrapping via CCCP, while the second iteration will attempt bootstrapping via HTTP.
In cases where the bucket/server supports CCCP (i.e. any Couchbase bucket on a >= 2.5 cluster), this will result in the library successfuly connecting to the first node and bootstrapping successfully via CCCP.
Sockets opened as part of the CCCP bootstrap process are placed inside a connection pool once they are no longer required. These sockets are then re-used for later connections for actual key-value operations.
There is a known issue regarding connection re-use and using hostnames different than what the cluster map provides:
Connection "Identity" in the library is defined on a per-hostname rather than per-IP basis; thus even if two hostnames point to the same IP, the library is unable to determine this. As a result, if the cluster sends us a list of hostnames different from what was provided in the initial bootstrap list, the library will fail to re-use these existing connections and will initiate new ones.
This issue becomes furthermore difficult to diagnose and reproduce due to the fact
that the CCCP-based configuration replaces the hostname of the currently-connected
connection with $HOST -- and triggering this issue only when the config is
received from another node (possibly as a result of a not-my-vbucket).
Once the client has bootstrapped (i.e. received an initial configuration), the client is not actively connected to any node: Any existing connections created during bootstrap are placed in a connection pool (which expires idle sockets after 10 seconds).
The client will gradually "create" (this means use an existing connection from the socket pool, if present, or establish a new TCP connection) connections to nodes on demand.
This means that even if there are 10 nodes in the cluster, the client will only open all the 10 connections if operations demand it. Conversely, this means that a "short-lived" client instance whose only job is to connect to the cluster and operate on a single key will end up creating only one or two connections (one for boostrap, and another one if the node for the given key is not the same as the bootstrap node).
During topology changes, the client will check to see which nodes are still in the cluster, and maintain any open connections; thus in the case of a rebalance there is typically no new creation of connections (unless of course a node was addded).
View connections are pooled, where the library will pool up to numnodes sockets. The lifetime of these sockets is 10 seconds when idle.
Note that the number of pooled view sockets may also be adjusted via a library setting.