Helios Scale Test Results - pass 1

elios.HorizontalScaling.Tests.Performance.Channels.TcpServerSocketChannelHorizontalScaleSpec+TcpServerSocketChannel_horizontal_scale_stress_test.md

Helios.HorizontalScaling.Tests.Performance.Channels.TcpServerSocketChannelHorizontalScaleSpec+TcpServerSocketChannel_horizontal_scale_stress_test

Measures how quickly and with how much GC overhead a TcpSocketChannel --> TcpServerSocketChannel connection can decode / encode realistic messages 5/3/2016 7:30:02 PM

System Info

NBench=NBench, Version=0.2.2.0, Culture=neutral, PublicKeyToken=null
OS=Microsoft Windows NT 6.2.9200.0
ProcessorCount=2
CLR=4.0.30319.42000,IsMono=False,MaxGcGeneration=2
WorkerThreads=32767, IOThreads=2

NBench Settings

RunMode=Iterations, TestMode=Measurement
NumberOfIterations=1, MaximumRunTime=00:00:01

Data

---

Totals

      Metric |           Units |             Max |         Average |             Min |          StdDev |

---------------- |---------------- |---------------- |---------------- |---------------- |---------------- | TotalCollections [Gen0] | collections | 4,472.00 | 4,472.00 | 4,472.00 | 0.00 | TotalCollections [Gen1] | collections | 263.00 | 263.00 | 263.00 | 0.00 | TotalCollections [Gen2] | collections | 44.00 | 44.00 | 44.00 | 0.00 | TotalBytesAllocated | bytes | 39,427,760.00 | 39,427,760.00 | 39,427,760.00 | 0.00 | [Counter] inbound ops | operations | 6,244,623.00 | 6,244,623.00 | 6,244,623.00 | 0.00 | [Counter] outbound ops | operations | 6,244,632.00 | 6,244,632.00 | 6,244,632.00 | 0.00 | [Counter] connected clients | operations | 340.00 | 340.00 | 340.00 | 0.00 | [Counter] exceptions caught | operations | 0.00 | 0.00 | 0.00 | 0.00 |

Per-second Totals

      Metric |       Units / s |         Max / s |     Average / s |         Min / s |      StdDev / s |

---------------- |---------------- |---------------- |---------------- |---------------- |---------------- | TotalCollections [Gen0] | collections | 31.41 | 31.41 | 31.41 | 0.00 | TotalCollections [Gen1] | collections | 1.85 | 1.85 | 1.85 | 0.00 | TotalCollections [Gen2] | collections | 0.31 | 0.31 | 0.31 | 0.00 | TotalBytesAllocated | bytes | 276,930.88 | 276,930.88 | 276,930.88 | 0.00 | [Counter] inbound ops | operations | 43,860.70 | 43,860.70 | 43,860.70 | 0.00 | [Counter] outbound ops | operations | 43,860.76 | 43,860.76 | 43,860.76 | 0.00 | [Counter] connected clients | operations | 2.39 | 2.39 | 2.39 | 0.00 | [Counter] exceptions caught | operations | 0.00 | 0.00 | 0.00 | 0.00 |

Raw Data

TotalCollections [Gen0]

       Run # |     collections | collections / s |ns / collections |

---------------- |---------------- |---------------- |---------------- | 1 | 4,472.00 | 31.41 | 31,836,763.98 |

TotalCollections [Gen1]

       Run # |     collections | collections / s |ns / collections |

---------------- |---------------- |---------------- |---------------- | 1 | 263.00 | 1.85 | 541,346,039.92 |

TotalCollections [Gen2]

       Run # |     collections | collections / s |ns / collections |

---------------- |---------------- |---------------- |---------------- | 1 | 44.00 | 0.31 |3,235,772,920.45 |

TotalBytesAllocated

       Run # |           bytes |       bytes / s |      ns / bytes |

---------------- |---------------- |---------------- |---------------- | 1 | 39,427,760.00 | 276,930.88 | 3,611.01 |

[Counter] inbound ops

       Run # |      operations |  operations / s | ns / operations |

---------------- |---------------- |---------------- |---------------- | 1 | 6,244,623.00 | 43,860.70 | 22,799.46 |

[Counter] outbound ops

       Run # |      operations |  operations / s | ns / operations |

---------------- |---------------- |---------------- |---------------- | 1 | 6,244,632.00 | 43,860.76 | 22,799.42 |

[Counter] connected clients

       Run # |      operations |  operations / s | ns / operations |

---------------- |---------------- |---------------- |---------------- | 1 | 340.00 | 2.39 | 418,747,083.82 |

[Counter] exceptions caught

       Run # |      operations |  operations / s | ns / operations |

---------------- |---------------- |---------------- |---------------- | 1 | 0.00 | 0.00 |142,374,008,500.00 |

Helios-Duplex-throughput.md

Helios.Tests.Performance.Channels.TcpChannelPerfSpecs+TcpChannel_Duplex_Throughput

Measures how quickly and with how much GC overhead a TcpSocketChannel --> TcpServerSocketChannel connection can decode / encode realistic messages 5/4/2016 12:58:06 AM

System Info

NBench=NBench, Version=0.2.2.0, Culture=neutral, PublicKeyToken=null
OS=Microsoft Windows NT 6.2.9200.0
ProcessorCount=2
CLR=4.0.30319.42000,IsMono=False,MaxGcGeneration=2
WorkerThreads=32767, IOThreads=2

NBench Settings

RunMode=Iterations, TestMode=Measurement
NumberOfIterations=5, MaximumRunTime=00:00:01

Data

---

Totals

      Metric |           Units |             Max |         Average |             Min |          StdDev |

---------------- |---------------- |---------------- |---------------- |---------------- |---------------- | TotalCollections [Gen0] | collections | 768.00 | 735.00 | 701.00 | 24.63 | TotalCollections [Gen1] | collections | 14.00 | 13.20 | 13.00 | 0.45 | TotalCollections [Gen2] | collections | 2.00 | 2.00 | 2.00 | 0.00 | TotalBytesAllocated | bytes | 222,807,656.00 | 210,474,356.80 | 186,500,784.00 | 14,625,653.42 | [Counter] inbound ops | operations | 1,000,000.00 | 1,000,000.00 | 1,000,000.00 | 0.00 | [Counter] outbound ops | operations | 1,000,000.00 | 1,000,000.00 | 1,000,000.00 | 0.00 |

Per-second Totals

      Metric |       Units / s |         Max / s |     Average / s |         Min / s |      StdDev / s |

---------------- |---------------- |---------------- |---------------- |---------------- |---------------- | TotalCollections [Gen0] | collections | 53.59 | 51.00 | 47.87 | 2.23 | TotalCollections [Gen1] | collections | 0.96 | 0.92 | 0.85 | 0.04 | TotalCollections [Gen2] | collections | 0.15 | 0.14 | 0.13 | 0.01 | TotalBytesAllocated | bytes | 16,479,404.37 | 14,665,487.05 | 12,069,578.18 | 1,815,978.34 | [Counter] inbound ops | operations | 73,962.47 | 69,476.78 | 64,715.96 | 4,253.90 | [Counter] outbound ops | operations | 73,962.47 | 69,476.78 | 64,715.96 | 4,253.90 |

Raw Data

TotalCollections [Gen0]

       Run # |     collections | collections / s |ns / collections |

---------------- |---------------- |---------------- |---------------- | 1 | 768.00 | 49.70 | 20,119,970.96 | 2 | 735.00 | 47.87 | 20,889,106.67 | 3 | 726.00 | 52.01 | 19,227,167.36 | 4 | 745.00 | 53.59 | 18,659,662.68 | 5 | 701.00 | 51.85 | 19,287,262.05 |

TotalCollections [Gen1]

       Run # |     collections | collections / s |ns / collections |

---------------- |---------------- |---------------- |---------------- | 1 | 14.00 | 0.91 |1,103,724,121.43 | 2 | 13.00 | 0.85 |1,181,037,953.85 | 3 | 13.00 | 0.93 |1,073,763,346.15 | 4 | 13.00 | 0.94 |1,069,342,207.69 | 5 | 13.00 | 0.96 |1,040,028,515.38 |

TotalCollections [Gen2]

       Run # |     collections | collections / s |ns / collections |

---------------- |---------------- |---------------- |---------------- | 1 | 2.00 | 0.13 |7,726,068,850.00 | 2 | 2.00 | 0.13 |7,676,746,700.00 | 3 | 2.00 | 0.14 |6,979,461,750.00 | 4 | 2.00 | 0.14 |6,950,724,350.00 | 5 | 2.00 | 0.15 |6,760,185,350.00 |

TotalBytesAllocated

       Run # |           bytes |       bytes / s |      ns / bytes |

---------------- |---------------- |---------------- |---------------- | 1 | 186,500,784.00 | 12,069,578.18 | 82.85 | 2 | 208,285,160.00 | 13,565,978.41 | 73.71 | 3 | 213,622,344.00 | 15,303,640.28 | 65.34 | 4 | 221,155,840.00 | 15,908,834.02 | 62.86 | 5 | 222,807,656.00 | 16,479,404.37 | 60.68 |

[Counter] inbound ops

       Run # |      operations |  operations / s | ns / operations |

---------------- |---------------- |---------------- |---------------- | 1 | 1,000,000.00 | 64,715.96 | 15,452.14 | 2 | 1,000,000.00 | 65,131.76 | 15,353.49 | 3 | 1,000,000.00 | 71,638.76 | 13,958.92 | 4 | 1,000,000.00 | 71,934.95 | 13,901.45 | 5 | 1,000,000.00 | 73,962.47 | 13,520.37 |

[Counter] outbound ops

       Run # |      operations |  operations / s | ns / operations |

---------------- |---------------- |---------------- |---------------- | 1 | 1,000,000.00 | 64,715.96 | 15,452.14 | 2 | 1,000,000.00 | 65,131.76 | 15,353.49 | 3 | 1,000,000.00 | 71,638.76 | 13,958.92 | 4 | 1,000,000.00 | 71,934.95 | 13,901.45 | 5 | 1,000,000.00 | 73,962.47 | 13,520.37 |

Helios-inbound-only.md

Helios.Tests.Performance.Channels.TcpChannelInboundOnlyPerfSpec+TcpChannel_Duplex_Throughput

Measures how quickly and with how much GC overhead a TcpSocketChannel --> TcpServerSocketChannel connection can decode / encode realistic messages 5/4/2016 12:52:04 AM

System Info

NBench=NBench, Version=0.2.2.0, Culture=neutral, PublicKeyToken=null
OS=Microsoft Windows NT 6.2.9200.0
ProcessorCount=2
CLR=4.0.30319.42000,IsMono=False,MaxGcGeneration=2
WorkerThreads=32767, IOThreads=2

NBench Settings

RunMode=Iterations, TestMode=Measurement
NumberOfIterations=5, MaximumRunTime=00:00:01

Data

---

Totals

      Metric |           Units |             Max |         Average |             Min |          StdDev |

---------------- |---------------- |---------------- |---------------- |---------------- |---------------- | TotalCollections [Gen0] | collections | 208.00 | 175.40 | 152.00 | 23.89 | TotalCollections [Gen1] | collections | 2.00 | 0.60 | 0.00 | 0.89 | TotalCollections [Gen2] | collections | 0.00 | 0.00 | 0.00 | 0.00 | TotalBytesAllocated | bytes | 3,428,976.00 | 2,746,652.80 | 1,890,864.00 | 684,563.46 | [Counter] inbound ops | operations | 1,000,000.00 | 1,000,000.00 | 1,000,000.00 | 0.00 | [Counter] outbound ops | operations | 0.00 | 0.00 | 0.00 | 0.00 |

Per-second Totals

      Metric |       Units / s |         Max / s |     Average / s |         Min / s |      StdDev / s |

---------------- |---------------- |---------------- |---------------- |---------------- |---------------- | TotalCollections [Gen0] | collections | 47.72 | 41.87 | 35.76 | 5.23 | TotalCollections [Gen1] | collections | 0.47 | 0.14 | 0.00 | 0.21 | TotalCollections [Gen2] | collections | 0.00 | 0.00 | 0.00 | 0.00 | TotalBytesAllocated | bytes | 841,377.24 | 658,515.23 | 433,779.86 | 173,159.99 | [Counter] inbound ops | operations | 245,372.74 | 239,004.28 | 229,408.28 | 6,556.04 | [Counter] outbound ops | operations | 0.00 | 0.00 | 0.00 | 0.00 |

Raw Data

TotalCollections [Gen0]

       Run # |     collections | collections / s |ns / collections |

---------------- |---------------- |---------------- |---------------- | 1 | 186.00 | 44.96 | 22,240,239.25 | 2 | 152.00 | 36.97 | 27,048,190.13 | 3 | 208.00 | 47.72 | 20,956,925.96 | 4 | 152.00 | 35.76 | 27,963,359.87 | 5 | 179.00 | 43.92 | 22,767,778.21 |

TotalCollections [Gen1]

       Run # |     collections | collections / s |ns / collections |

---------------- |---------------- |---------------- |---------------- | 1 | 1.00 | 0.24 |4,136,684,500.00 | 2 | 0.00 | 0.00 |4,111,324,900.00 | 3 | 0.00 | 0.00 |4,359,040,600.00 | 4 | 2.00 | 0.47 |2,125,215,350.00 | 5 | 0.00 | 0.00 |4,075,432,300.00 |

TotalCollections [Gen2]

       Run # |     collections | collections / s |ns / collections |

---------------- |---------------- |---------------- |---------------- | 1 | 0.00 | 0.00 |4,136,684,500.00 | 2 | 0.00 | 0.00 |4,111,324,900.00 | 3 | 0.00 | 0.00 |4,359,040,600.00 | 4 | 0.00 | 0.00 |4,250,430,700.00 | 5 | 0.00 | 0.00 |4,075,432,300.00 |

TotalBytesAllocated

       Run # |           bytes |       bytes / s |      ns / bytes |

---------------- |---------------- |---------------- |---------------- | 1 | 2,149,264.00 | 519,561.98 | 1,924.70 | 2 | 3,025,808.00 | 735,969.08 | 1,358.75 | 3 | 1,890,864.00 | 433,779.86 | 2,305.32 | 4 | 3,238,352.00 | 761,887.97 | 1,312.53 | 5 | 3,428,976.00 | 841,377.24 | 1,188.53 |

[Counter] inbound ops

       Run # |      operations |  operations / s | ns / operations |

---------------- |---------------- |---------------- |---------------- | 1 | 1,000,000.00 | 241,739.49 | 4,136.68 | 2 | 1,000,000.00 | 243,230.59 | 4,111.32 | 3 | 1,000,000.00 | 229,408.28 | 4,359.04 | 4 | 1,000,000.00 | 235,270.28 | 4,250.43 | 5 | 1,000,000.00 | 245,372.74 | 4,075.43 |

[Counter] outbound ops

       Run # |      operations |  operations / s | ns / operations |

---------------- |---------------- |---------------- |---------------- | 1 | 0.00 | 0.00 |4,136,684,500.00 | 2 | 0.00 | 0.00 |4,111,324,900.00 | 3 | 0.00 | 0.00 |4,359,040,600.00 | 4 | 0.00 | 0.00 |4,250,430,700.00 | 5 | 0.00 | 0.00 |4,075,432,300.00 |

Specs were computed in a single run on a Windows Azure D2V2 box - 2 cores and 7 GB of RAM.

Future versions of NBench will produce more detailed hardware specifications, such as clock speeds and caches, but for now this is the coarse grained data I have :p