Preliminary servers_per_port benchmark results

64K_get_base1.scenario

{
  "name": "64K_get",
  "sizes": [{
    "name": "64KiB",
    "size_min": 65536,
    "size_max": 65536
  }],
  "initial_files": {
    "64KiB": 10000000
  },
  "container_base": "base1_64K",
  "run_seconds": 300,
  "container_count": 200,
  "crud_profile": [0, 1, 0, 0],
  "user_count": 1
}

container-server.conf

[DEFAULT]
backlog = 4096
workers = 8
mount_check = true

[app:container-server]
node_timeout = 4
conn_timeout = 10.0
allow_versions = True
db_preallocation = False
replication_server = False

[container-updater]
interval = 300
concurrency = 4
node_timeout = 4
conn_timeout = 10.0
slowdown = 0.001
account_suppression_time = 60
db_preallocation = False

[container-auditor]
interval = 1800
containers_per_second = 200
db_preallocation = False

In separate config for replication-only container-server:
[container-replicator]
per_diff = 1000
concurrency = 8
run_pause = 30
node_timeout = 10
conn_timeout = 10.0
reclaim_age = 7200
db_preallocation = False

description.txt

Hardware:
proxy1: 12x E5-2650 0 @ 2.00GHz (24 HT cores); 32 GiB RAM; 10 Gbps NIC
proxy2: 12x E5-2650 0 @ 2.00GHz (24 HT cores); 32 GiB RAM; 10 Gbps NIC
stor1:  12x E5-2620 0 @ 2.00GHz (24 HT cores); 64 GiB RAM; 10 Gbps NIC; 32x 3T obj disks; 2x 120 GB acct/cont SSDs
stor2:  12x E5-2620 0 @ 2.00GHz (24 HT cores); 64 GiB RAM; 10 Gbps NIC; 29x 3T obj disks; 2x 120 GB acct/cont SSDs

Operating Systems:
proxy1, stor1, stor2: Ubuntu fully-patched 12.04.5 running 3.13.0-53-generic kernel
proxy2: Ubuntu fully-patched 14.04.2 running 3.13.0-53-generic kernel

Software:
Swift "normal" & "threads_per_disk": my patch just past upstream e9a032f
Swift "severs_per_disk=X": Patch Set 8 https://review.openstack.org/#/c/184189/8/
ssbench 0.3.5

I ran ssbench-master on proxy1, with 4 ssbench-workers per proxy1 and proxy2; each worker talked only to its local proxy-server processes (127.0.0.1).  Workers looked like:
```
ssbench-worker --zmq-host 192.168.201.51 --concurrency 256 --batch-size 256 0
```

The master was invoked as:
```
ssbench-master run-scenario -A http://127.0.0.1/auth/v1.0 -S http://127.0.0.1/v1/AUTH_bench1 -f 64K_get_base1.scenario -u 2048 -k -r 3600 --batch-size 256
```

The run itself was 100% GET of 64 KiB objects, of which 10 million initial objects were used, spread across 200 containers (same 10M initial files used between all the runs, each run HEADs all 10M objects before the start of the actual benchmark run).  Each run of GETs lasted 1 hour.

The Swift account used had 2800 containers, 11,139,089 objects, consuming 3,236,629,069,824 bytes; that was about it for the whole cluster--there was one other account with a tiny amount of data.

object-server.conf

[DEFAULT]
backlog = 4096
# workers was 29 for "normal" and 6 for threads_per_disk
workers = 6
mount_check = true
servers_per_port = 4
node_timeout = 20
conn_timeout = 10.0
network_chunk_size = 65536
disk_chunk_size = 65536

[app:object-server]
slow = 0
mb_per_sync = 512
log_requests = True
threads_per_disk = 0
# We always use a separate object-server for replication:
replication_server = False
disk_chunk_size = 65536

[object-updater]
interval = 300
concurrency = 2
node_timeout = 20
conn_timeout = 10.0
slowdown = 0.01

[object-auditor]
disk_chunk_size = 1048576
files_per_second = 10
bytes_per_second = 1000000
zero_byte_files_per_second = 20

In a separate config for replication-only object-server:

[object-replicator]
run_pause = 60
concurrency = 1
stats_interval = 60
rsync_timeout = 900
rsync_io_timeout = 30
rsync_bwlimit = 0
http_timeout = 60
node_timeout = 10
lockup_timeout = 1800
reclaim_age = 7200
ring_check_interval = 15
disk_chunk_size = 1048576

proxy-server.conf

(condensed)

[DEFAULT]
backlog = 4096
workers = 16
client_timeout = 60

[pipeline:main]
pipeline = catch_errors healthcheck proxy-logging cache formpost tempurl swc swiftstack_authen swiftstack_authz dlo slo proxy-logging proxy-server

[app:proxy-server]
use = egg:swift#proxy
account_autocreate = True
allow_account_management = true
recheck_account_existence = 60
recheck_container_existence = 60
object_chunk_size = 65536
client_chunk_size = 65536
node_timeout = 10
conn_timeout = 10.0
post_quorum_timeout = 0.01
error_suppression_interval = 60
error_suppression_limit = 10
object_post_as_copy = False
put_queue_depth = 10
sorting_method = timing
# If the timing sorting_method is used, the timings will only be valid for
# the number of seconds configured by timing_expiry.
timing_expiry = 300

[filter:cache]
memcache_servers = 192.168.3.51:11211,192.168.3.52:11211
max_connections = 10
connect_timeout = 0.5
pool_timeout = 2.0
tries = 3
io_timeout = 4.0

tabular_results.md

64 KiB GETs; 1 hr| min| max| avg| stddev| 95th %ile| req/s ---|---:|---:|---:|---:|---:|---:|---: normal; 29 workers| 0.006| 19.439| 0.344| 0.565| 1.073| 2129.5 threads_per_disk=1| 0.006| 19.256| 1.118| 1.342| 3.831| 903.5 servers_per_port=1| 0.006| 20.079| 0.371| 1.287| 0.822| 2133.9 servers_per_port=2| 0.007| 3.140| 0.237| 0.158| 0.549| 2523.9 servers_per_port=3| 0.007| 2.212| 0.171| 0.103| 0.369| 3281.5 servers_per_port=4| 0.007| 2.377| 0.161| 0.096| 0.345| 3366.9

Values are for time-to-first-byte, though there was <= 0.001 difference between time-to-first-byte and total time for these small objects. Units for min, max, avg, stddev, and 95th-percentile are seconds and are statistical descriptions of the request latencies for all requests during the run. The "req/s" is the average requests per second for the full hour-long run (i.e. total requests divided by run duration).