Skip to content

Instantly share code, notes, and snippets.

What would you like to do?

Long story short, Celluloid versions 0.17+ have a memory leak.

The Reason behind this is that completed Celluloid threads are never cleaned up.

We have discovered that our Sidekiq process is leaking memory when we have a lot of tasks that were failed because of exceptions. Unfortunately, having a lot of failed tasks is specific for our application — we do have a lot of small queued jobs to work with social network APIs and other external services.

You can reproduce the problem with this:

Dead jobs   rss      alive/total
1:          15MB     (Fibers: 4/4, Threads: 3/3)
1000:       52MB     (Fibers: 1003/2002, Threads: 3/1002)

Initial issue is described here: celluloid/celluloid#670

And the fix is here:

The fix is is included in Celluloid 0.17.2 that was released on September 30th, 2015.

Кратко - потому что версии 0.17+ текут.

Суть: мертвые потоки не чистятся

Заметили утечку памяти при большом количестве упавших тасок (для приложения специфично падение тасок, так как много мелких тасок работающих с соц сетями и другими внешними сервисами).

Воспроизвести можно следующим образом:

Умерших задач  rss  живые/всего
1:             15MB (Fibers: 4/4, Threads: 3/3)
1000:          52MB (Fibers: 1003/2002, Threads: 3/1002)

Проблема: celluloid/celluloid#670


Включен в релиз 0.17.2 от 30-го сентября

Copy link

gazay commented Oct 1, 2015

Many thanks to @brainopia for helping me to find this out and make reproduce steps

Copy link

sponomarev commented Oct 1, 2015


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment