Skip to content

Instantly share code, notes, and snippets.

@DiveInto
Created November 12, 2012 05:19
Show Gist options
  • Save DiveInto/4057644 to your computer and use it in GitHub Desktop.
Save DiveInto/4057644 to your computer and use it in GitHub Desktop.
try leak memory according to http://stackoverflow.com/a/6471947/404145
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
public class MLeaks {
static byte[] m = new byte[1024 * 1024];
static ThreadLocal<MLeaks> tLocal = new ThreadLocal<MLeaks>(){
@Override
protected MLeaks initialValue()
{
return new MLeaks();
}
};
public static void main(String[] args){
while(true){
Thread thread = new Thread(){
public void run() {
try {
URLClassLoader cl = new URLClassLoader(new URL[]{new URL("file:/path-to/MLeaks/bin/")});
System.out.println(cl.toString());
Class<MLeaks> tmpClass = (Class<MLeaks>) cl.loadClass(MLeaks.class.getName());
tmpClass = null;
} catch (MalformedURLException e1) {
e1.printStackTrace();
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
};
thread.start();
}
}
}
@DiveInto
Copy link
Author

GC logs


...
{Heap before GC invocations=151 (full 0):
 par new generation   total 19136K, used 17028K [7f3000000, 7f44c0000, 7f44c0000)
  eden space 17024K, 100% used [7f3000000, 7f40a0000, 7f40a0000)
  from space 2112K,   0% used [7f42b0000, 7f42b1000, 7f44c0000)
  to   space 2112K,   0% used [7f40a0000, 7f40a0000, 7f42b0000)
 concurrent mark-sweep generation total 63872K, used 1352K [7f44c0000, 7f8320000, 7fae00000)
 concurrent-mark-sweep perm gen total 21248K, used 4671K [7fae00000, 7fc2c0000, 800000000)
2012-11-12T15:17:33.024+0800: 1.563: [GC 1.563: [ParNew
Desired survivor size 1081344 bytes, new threshold 4 (max 4)
- age   1:        656 bytes,        656 total
: 17028K->6K(19136K), 0.0005397 secs] 18380K->1358K(83008K), 0.0005714 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 
Heap after GC invocations=152 (full 0):
 par new generation   total 19136K, used 6K [7f3000000, 7f44c0000, 7f44c0000)
  eden space 17024K,   0% used [7f3000000, 7f3000000, 7f40a0000)
  from space 2112K,   0% used [7f40a0000, 7f40a1800, 7f42b0000)
  to   space 2112K,   0% used [7f42b0000, 7f42b0000, 7f44c0000)
 concurrent mark-sweep generation total 63872K, used 1352K [7f44c0000, 7f8320000, 7fae00000)
 concurrent-mark-sweep perm gen total 21248K, used 4671K [7fae00000, 7fc2c0000, 800000000)
}
{Heap before GC invocations=152 (full 0):
 par new generation   total 19136K, used 17030K [7f3000000, 7f44c0000, 7f44c0000)
  eden space 17024K, 100% used [7f3000000, 7f40a0000, 7f40a0000)
  from space 2112K,   0% used [7f40a0000, 7f40a1800, 7f42b0000)
  to   space 2112K,   0% used [7f42b0000, 7f42b0000, 7f44c0000)
 concurrent mark-sweep generation total 63872K, used 1352K [7f44c0000, 7f8320000, 7fae00000)
 concurrent-mark-sweep perm gen total 21248K, used 4671K [7fae00000, 7fc2c0000, 800000000)
2012-11-12T15:17:33.045+0800: 1.584: [GC 1.584: [ParNew
Desired survivor size 1081344 bytes, new threshold 4 (max 4)
- age   1:        856 bytes,        856 total
: 17030K->4K(19136K), 0.0003783 secs] 18382K->1356K(83008K), 0.0004115 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 
Heap after GC invocations=153 (full 0):
 par new generation   total 19136K, used 4K [7f3000000, 7f44c0000, 7f44c0000)
  eden space 17024K,   0% used [7f3000000, 7f3000000, 7f40a0000)
  from space 2112K,   0% used [7f42b0000, 7f42b1000, 7f44c0000)
  to   space 2112K,   0% used [7f40a0000, 7f40a0000, 7f42b0000)
 concurrent mark-sweep generation total 63872K, used 1352K [7f44c0000, 7f8320000, 7fae00000)
 concurrent-mark-sweep perm gen total 21248K, used 4671K [7fae00000, 7fc2c0000, 800000000)
}
{Heap before GC invocations=153 (full 0):
 par new generation   total 19136K, used 17028K [7f3000000, 7f44c0000, 7f44c0000)
  eden space 17024K, 100% used [7f3000000, 7f40a0000, 7f40a0000)
  from space 2112K,   0% used [7f42b0000, 7f42b1000, 7f44c0000)
  to   space 2112K,   0% used [7f40a0000, 7f40a0000, 7f42b0000)
 concurrent mark-sweep generation total 63872K, used 1352K [7f44c0000, 7f8320000, 7fae00000)
 concurrent-mark-sweep perm gen total 21248K, used 4671K [7fae00000, 7fc2c0000, 800000000)
2012-11-12T15:17:33.052+0800: 1.592: [GC 1.592: [ParNew
Desired survivor size 1081344 bytes, new threshold 4 (max 4)
- age   1:        656 bytes,        656 total
: 17028K->4K(19136K), 0.0005990 secs] 18380K->1356K(83008K), 0.0006341 secs] [Times: user=0.00 sys=0.01, real=0.00 secs] 
Heap after GC invocations=154 (full 0):
 par new generation   total 19136K, used 4K [7f3000000, 7f44c0000, 7f44c0000)
  eden space 17024K,   0% used [7f3000000, 7f3000000, 7f40a0000)
  from space 2112K,   0% used [7f40a0000, 7f40a1000, 7f42b0000)
  to   space 2112K,   0% used [7f42b0000, 7f42b0000, 7f44c0000)
 concurrent mark-sweep generation total 63872K, used 1352K [7f44c0000, 7f8320000, 7fae00000)
 concurrent-mark-sweep perm gen total 21248K, used 4671K [7fae00000, 7fc2c0000, 800000000)
}
...

@dpryden
Copy link

dpryden commented Nov 12, 2012

Because you load MLeaks with the system classloader (your main() is inside MLeaks, after all), then any URLClassLoader you create has no effect. The default behavior of URLClassLoader is to first check its parent ClassLoader to see if the class has already been loaded, and if so it doesn't load it again.

To make this a proper test, isolate your main() into a different class from your MLeaks, and don't have any reference to MLeaks from your main(). If you're doing it correctly, each Class instance returned by loadClass() should be distinct.

@DiveInto
Copy link
Author

@dpryden thanks for your reply, now I change the code into this:

package com.demo.app;

import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;

public class LeakApp {

  public static void main(String[] args){
      while(true){
          Thread thread = new Thread(){
              public void run() {
                  try {
                      URLClassLoader cl = new URLClassLoader(new URL[]{new URL("file:/path-to/MLeaks/bin/")});

                      //System.out.println(cl.toString());
                      Class<?> tmpClass = cl.loadClass("com.demo.learn.MLeaks");
                      //System.out.println(">>>>" + tmpClass.hashCode());

                      tmpClass = null;
                      Thread.sleep(1000);
                  } catch (Exception e1) {
                      e1.printStackTrace();
                  } 
              }
          };

          thread.start();
      }
  }
}

and the above app load class below:

package com.demo.learn;

public class MLeaks {

  static byte[] m = new byte[1024 * 1024];

  static ThreadLocal<MLeaks> tLocal = new ThreadLocal<MLeaks>(){
      @Override
      protected MLeaks initialValue()
      {
          return new MLeaks();
      }
  };
}

Is that right now?

According to your answer at stackoverflow, this cause memory leak cause the ThreadLocal refers to the instance of MLeaks which refers to the MLeaks class which refers to the URLClassLoader,so even though the thread loading the class cannot access the MLeaks class again, the memeory occupied by MLeaks class can not be GCed.

while according to the javadoc of ThreadLocal

This method(ThreadLocal's initialValue method) will be invoked the first time a thread accesses the variable with the get() method

so I think simply load the class won't call the initialValue method, so no instance of MLeaks will be stored into ThreadLocal, and that makes the rest reasoning invalid.

Correct me if I make mistake here, Thanks again.

@dpryden
Copy link

dpryden commented Nov 16, 2012

It's true, if you only subclass ThreadLocal and never actually assign a value into the the thread local storage, then you're not leaking. In my experience, for these kinds of use cases (a thread local in a framework), the ThreadLocal object is almost always instantiated directly instead of being subclassed.

As it turns out, a lot of existing code follows this kind of pattern. For example, most Java servlet containers create a bunch of long running threads which are used to handle requests, then load each application in its own ClassLoader. When you update the application (e.g. by replacing the WAR file), the old ClassLoader is discarded and a new one is created. All of this works fine, unless you also have a framework in place that uses ThreadLocals. As it turns out, lots of popular frameworks use thread local storage (including many Java web application frameworks), and many frameworks compound the problem by creating their own ClassLoaders to do runtime bytecode manipulation (Hibernate and many AOP frameworks do this). A large application can easily pull in a dozen or so different large libraries and frameworks, and it can be very difficult to determine exactly where you have the one dangling reference that is keeping your entire application in memory.

The common way that this problem manifests is on a development/debugging server. For example, on a previous project, I would develop our web application (which used Stripes, Guice, Hibernate, and a handful of other big libraries) on a local Tomcat server. Throughout the day I would frequently redeploy the application to my local server, and I was guaranteed to to have to restart my local server at least once every few hours because the PermGen would fill up and the server would crash.

The point of my answer over at StackOverflow was that this is a true memory leak: once the ClassLoader is discarded, there's no way to retrieve the ThreadLocal instance anymore, and so short of killing the Thread (and maybe not even then) there is no way to release the memory.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment