NGINX + COMET problems ...

gistfile1.txt

Hi all.
We are using nginx-push-stream-module.

And we get next errors:
-----
2011/12/11 18:52:23 [crit] 24244#0: ngx_slab_alloc() failed: no memory
2011/12/11 18:52:23 [error] 24244#0: *73289555 push stream module: unable to allocate worker subscriber queue marker in shared memory, client: 213.85.187.19, server: 172.17.0.4, request: "GET /xxx/v1/sub/106q2chk9s1ld1fd3m1hx9vuib"
2011/12/11 18:52:23 [crit] 24244#0: ngx_slab_alloc() failed: no memory
2011/12/11 18:52:23 [error] 24244#0: *73289563 push stream module: unable to allocate worker subscriber queue marker in shared memory, client: 213.85.187.19, server: 172.17.0.4, request: "GET /xxx/v1/sub/34lgiys1st5ip0t0qkw81gcr?rnd=0.02243080991320312"
----


In view of the memory leaks in the push-module nginx in production, I spent a load testing comet.
The test is written in Java. It creates a thread pool of 500 threads. Each of them is connected to the comet-channel.
each time to the unique. The test worked through the night.
 
Now the results.

~ ENVIRONMENT:
Platform: x86-64.
uname-a: Linux server1 2.6.18-194.32.1.el5 # 1 SMP Wed Jan 5 17:52:25 EST 2011 x86_64 x86_64 x86_64 GNU / Linux
4 processor with two cores in each = 8 nuclei.
Version of nginx: 1.1.7.
Version push_module: 0.3.1.

~ MEMORY CONSUMPTION
No-load each nginx worker consumes ~ 42 248 KB
c 500 compounds attached worker nginx-a consumption: 45 080 KB
Hence, at 500 connections took 45 080 - 42 248 = 3560 KB
Therefore, one compound is spent, empirically, 3560/500 ~ 7 kb.
That is, on 50000 compounds have only ~ 350 mb.

Questions:
  Exactly how much memory is consumed on one channel?
  Exactly how much memory is consumed on a single connection?
  Exactly how much system memory is necessary for 50,000 connections?

The test worked through the night. Timeout on nginx stood at 1 minute.
That is, flows constantly reconnect to random channels.
By morning no leaks were found. all workers consume memory> 50 MB.
 
Hence comes the idea that the problem could be in production at the level of TCP / IP.
We are writing middleware for IPTV. Clients use SetTopBoxes to connect.
Maybe the operating system at STBs disconnects from nginx don't correct?

Question:
  What Linux kernel configuration recommened for use push_module?
  We are all time-outs are reduced to a minimum. TIMEWAITS connections should not be much.

I want to get tcpdump from our clients and publish its here.

I hope the above information will help to eliminate memory leaks.

P.S.
cat /proc/net/sockstat
sockets: used 829
TCP: inuse 658 orphan 0 tw 0 alloc 662 mem 504
UDP: inuse 12 mem 0
RAW: inuse 0
FRAG: inuse 0 memory 0