sphinx statistics.textile

Sphinx Statistics

I’ve been working on a Thinking Sphinx add-on for Heroku. It’s not launched yet, and I’m ironing out the possible plans I’ll offer. One key piece of information I’d love to have is a rough rule of how much RAM a searchd process uses in relation to how big the index files are.

If you’re using Sphinx on an app, and are willing to share this information, it’d be wonderful if you could get in touch with me, via email (pat at freelancing-gods dot com), or via twitter.

I’m after real (ie: production) data – development environment stats aren’t quite so useful :)

Here’s how I check the stats on my own sites:

RAM Usage

I’m interested in both the real (RSS) and virtual (VSZ) memory usage of the searchd process.

$ ps x -o rss -o vsz -o comm | grep searchd

The first value is the real memory, the second is the virtual – both in kilobytes.

Disk Usage

This one’s pretty easy as well – cd through to the directory where all your index files are located, then run the following command:

$ du -s .

There should only be one number in the output – and that’s the total size of the directory in kilobytes.

If you can make five minutes to find out this information and send it my way, that’d be brilliant. I’m more than happy to put together a chart of all the values if people are truly interested in that ;)

ps x -o rss -o vsz -o comm | grep searchd

284  29104 searchd

du -s .

14828   .

Not huge in terms of what we're indexing :)

$ ps ax -o rss -o vsz -o comm | grep searchd
932  33604 searchd

$du -s
9924

There you go:

$ ps x -o rss -o vsz -o comm | grep searchd

904 38784 searchd

$ du -s .

127112

Two systems:
1900 10316 searchd
11836 .

2008 10824 searchd
1076 .

$ ps x -o rss -o vsz -o comm | grep searchd

1320 190116 searchd

$ du -s db/sphinx/production/

5359592 db/sphinx/production/

Here's another one:

$ ps x -o rss -o vsz -o comm | grep searchd

704 19360 searchd

$ du -s db/sphinx/production/

27380 db/sphinx/production/

Thanks for all the responses! It's been a great help, but if anyone has more, all the better :)

$ ps x -o rss -o vsz -o comm | grep searchd
1640 235184 searchd

$ du -s .
450264 .