gistfile1.txt

Status:

Publishing gets completed in under a second: 
info: [{memory,303357},
       {owner,<0.2888.0>},
       {heir,none},
       {name,subs},
       {size,20000},
       {node,'janus@domU-12-31-39-02-B5-46'},
       {named_table,false},
       {type,set},
       {keypos,1},
       {protection,public}]
time: 614.971


Max client latency edges right around 2s:


=INFO REPORT==== 16-Jul-2009::03:53:40 ===
setup: 85402.31ms, good: 20000, bad: 0, run: 14193.93ms
   15.9900ms | min   
  500.0000ms | 3042   -  15.21%
 1000.0000ms | 5379   -  26.89%
 1500.0000ms | 7310   -  36.55%
 2000.0000ms | 4267   -  21.34%
 2500.0000ms | 2      -   0.01%
 2097.4250ms | max   
ok

I think this is mostly a problem with scheduling and the fact that the clients are all running on a single EC2 small instance which has only 1 virtual cpu.  Unless the final production environment has 20k clients per box, we should be allowed to either split up the number of clients between boxes or swap up to a larger instance for the clients.

This version also has protocol changes, but they don't effect performance and can be backed out of.