gistfile1.txt

17:16 bbrowning: enebo: So I'm taking another crack at tracking down a memory leak on Windows that appears related to FFI
17:17 bbrowning: All the paths to GC roots show the refmap of org.jruby.ext.ffi.AbstractInvoker as the only thing keeping RubyObjects alive that are in turn keeping Ruby runtimes alive
17:17 bbrowning: refmap is a WeakHashMap<DynamicMethod, AbstractInvoker>
17:18 bbrowning: the value of the entries is what's keeping the runtimes alive
17:18 bbrowning: so, does that mean that somewhere else something is hanging on to the DynamicMethod key objects and thus they don't get GC'd causing the values to never go away?
17:19 enebo: bbrowning: really?
17:19 enebo: I don't hack on ffi but I am surprised to hear about weak hashmap
17:19 bbrowning: yeah TorqueBox integ runs on windows don't really unload any classes during the entire integ run and we get Ruby runtimes piling up in memory
17:19 enebo: This is not a ruby objects method table?  It is an ffi thing right?
17:20 bbrowning: WeakHashMap seems like the right thing here I guess, but does my logic sound right? If the WeakHashMap entries are sticking around then something is still holding a strong reference to the key?
17:20 bbrowning: enebo: yeah - refmap is "Reference map to keep libraries open for as long as there is a method mapped to that library."
17:20 stephenjudkins: bbrowning: are you sure they stick around even after a full GC is performed?
17:20 bbrowning: stephenjudkins: I am, yes
17:21 enebo: bbrowning: not possible it is doing an 'attach' is it?
17:21 bbrowning: enebo: I'm not sure I understand what you mean :D
17:21 enebo: doh…of course it is
17:21 bbrowning: enebo: the FFI code in question does get called during the "attach" function
17:21 bbrowning: but it needs to somehow unattach ;)
17:21 enebo: FFI will attach each generated method to a map when it maps a library
17:22 bbrowning: and so I dispose of a Ruby runtime and those methods need to unattach, right?
17:22 bbrowning: or is there any reason they should stick around longer than a Ruby?
17:22 enebo: So only unloading the library should actually clean up which presumably happens on runtime shutdown
17:22 enebo: oh geezus
17:22 stephenjudkins: bbrowning: have you found the GC roots of the *keys* in the refmap?
17:22 enebo: it is static
17:22 bbrowning: enebo: right
17:23 bbrowning: stephenjudkins: I have not yet, no - that was going to be my next task
17:23 stephenjudkins: bbrowning: that will be the source of your leak
17:23 bbrowning: enebo: Before I dug too far into the GC, I wondered if maybe the refmap should just be stored somewhere in a Ruby vs a static thing
17:23 enebo: ok so it must be static to not have n copies of bound methods….but the method is pinning the runtime
17:23 enebo: so that is not right :)
17:24 enebo: bbrowning: It should just be a special place on an instance per runtime
17:25 enebo: bbrowning: The thing which I don't know is can you bind n copies of the same DLL/.so in one JVM
17:25 enebo: but even if so using DynamicMethod is wrong since it pins our runtimes to a static map
17:25 bbrowning: enebo: yeah I don't really understand all the FFI internals enough to feel comfortable mucking around too much, other than knowing that it's leaking ;)
17:26 enebo: bbrowning: I will send wmeissner an email about it
17:26 bbrowning: enebo: great thanks
17:26 enebo: bbrowning: This seems obviously wrong but there might be a good reason why it is static
17:26 enebo: what is stored in it (DynamicMethod) is wrong though
17:27 bbrowning: k
17:27 enebo: bbrowning: This is happening on more than just Windows though right?
17:27 bbrowning: enebo: Well I only really notice the issue on Windows. We don't explicitly use FFI in any of the TorqueBox integration tests.
17:27 bbrowning: enebo: Does something internal to JRuby switch to an FFI implementation on Windows?
17:28 enebo: bbrowning: yeah looks generic to me…we may use ffi more on windows
17:28 enebo: bbrowning: it wouldn't surprise me anyways
17:28 bbrowning: yeah I'd guess this is more than just a windows issue