Skip to content

Instantly share code, notes, and snippets.

@sandys
Created August 17, 2012 09:19
Show Gist options
  • Save sandys/3377335 to your computer and use it in GitHub Desktop.
Save sandys/3377335 to your computer and use it in GitHub Desktop.
Java debugging howtos

locate /usr/java/tomcat/temp/hsperfdata_tomcat/

to get processid of tomcat jps -J-Djava.io.tmpdir=/usr/java/tomcat/temp

/usr/java/jdk1.6.0_24/bin/jmap -J-Djava.io.tmpdir=/usr/local/tomcat-instance/traduscms/temp -heap 23420

sudo -u root /usr/java/jdk1.6.0_24/bin/jstack -J-Djava.io.tmpdir=/usr/local/tomcat-instance/tradusgv/temp -J-d64 10960

jstat -J-Djava.io.tmpdir=/usr/java/tomcat/temp -gc -t 3s

FGC - # of full GC

FGCT - time of full GC

check EC (total eden) vs EU (current eden) vs changes in OU (current old gen)

Configuring the heap

What can we configure, now that we know the occupation and sizes of the spaces? While the sizes offer a lot of configuration functionality, they are usually only touched when looking at garbage collection. All heap areas are equally good for storing data, but for garbage collection you might want to change some.

Without configuration, a client JVM uses this calculation for the spaces: Heap = Tenured + Young Tenured = 2 x Young Young = Survivor x 2 + Eden

A common change is increasing the maximum Permanent generation size using -XX:MaxPermSize=128m. The initial size can be set with -XX:PermSize=64m. Depending on the application, the New size can also be adjusted. This can be done as ratio -XX:NewRatio=2 (which is recommended, as it adjusts automatically), or as fixed size -XX:NewSize=128m or -Xmn128m (which is less flexible but easier to understand). All ratios are configured as “one of me – N of the other”, where N is the value provided to the flag. For example, -XX:NewRatio=2 means 33% of the heap are for New and the remaining 66% are for Old.

Young space can be configured by using -XX:SurvivorRatio to control the size of the survivor spaces, but this is often not very effective. A typical configuration for a web application could look like this:

-Xms2g -Xmx2g -XX:NewRatio=4 -XX:MaxPermSize=512m -XX:SurvivorRatio=6

The total memory used by Java would be slightly over 2.5GB. Old would be 1.6GB, Eden 300MB, and each Survivor Space 50MB.

Why does OutOfMemory occur with free memory available?

The Old generation is full. Survivor Spaces are empty, and Eden has only 30%. Which means that we get an OutOfMemoryError even when we have still almost 30MB free memory in our Xmx limit of 128MB. So why do we get the error at all? While we do not know exactly what’s going on, the code creates bigger and bigger arrays to hold the Long objects created in the loop. A new Array is about to be created right here: at java.util.Arrays.copyOf(Arrays.java:2760). Now suppose that at some point there is not enough space in Eden for doing this (which means that we can assume that the array is larger than 22MB). A garbage collection should kick in and make some space available. However, there is no space left in Old, and the existing Eden objects cannot be moved anywhere. That’s when OutOfMemoryError is thrown. To conclude, when you see an OutOfMemoryError it doesn’t mean that your whole heap memory has been exhausted. But you can be sure that at least one of the spaces is filled, preventing creation or movement of objects there.

One pattern of GC to watch out for is the case when you don't have enough survivor/eden space to handle all of the transient data you need to do a big chunk of work, and not enough old capacity to take it all, but just enough freeable state to keep slowly moving forward with your computation. The result will be neverending full-gcs while your process moves slightly forward, does a full GC, moves forward again, does a full GC, and moves forward again.This looks in jstat like an ever-increasing series of FGC where each one finishes but only causes a small amount of space to be freed in OU (current old gen), while EU and the survivor spaces are constantly near-full.

S0C Displays the current size of Survivor0 area in KB S1C Displays the current size of Survivor1 area in KB S0U Displays the current usage of Survivor0 area in KB

S1U Displays the current usage of Survivor1 area in KB EC Displays the current size of Eden area in KB EU Displays the current usage of Eden area in KB OC Displays the current size of old area in KB OU Displays the current usage of old area in KB

PC Displays the current size of permanent area in KB PU Displays the current usage of permanent area in KB YGC The number of GC event occurred in young area YGCT The accumulated time for GC operations for Yong area

FGC The number of full GC event occurred FGCT The accumulated time for full GC operations GCT The total accumulated time for GC operations NGCMN The minimum size of new area in KB NGCMX The maximum size of max area in KB

NGC The current size of new area in KB OGCMN The minimum size of old area in KB OGCMX The maximum size of old area in KB OGC The current size of old area in KB PGCMN The minimum size of permanent area in KB

PGCMX The maximum size of permanent area in KB PGC The current size of permanent generation area in KB PC The current size of permanent area in KB PU The current usage of permanent area in KB LGCC The cause for the last GC GCC The cause for the current GC

TT Tenuring threshold. If copied this amount of times in young area (S0 ->S1, S1->S0), they are then moved to old area. MTT Maximum Tenuring threshold. If copied this amount of times inside young arae, then they are moved to old area.

DSS Adequate size of survivor in KB

From and To space: http://help.sap.com/saphelp_nwpi71/helpdata/en/f0/cec51dabd1461b87e4db9e3958710e/content.htm

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/java/jdk1.6.0_24/jre/lib/amd64/jli

valgrind --smc-check=all --leak-check=yes --tool=memcheck /usr/java/jdk1.6.0_24//bin/java -Djava.util.logging.config.file=/usr/local/tomcat-instance/tradusoffering/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/usr/local/apache-tomcat-6.0.33/endorsed -classpath /usr/local/tomcat-instance/tradusoffering/bin/tomcat-juli.jar:/usr/local/apache-tomcat-6.0.33/bin/bootstrap.jar -Dcatalina.base=/usr/local/tomcat-instance/tradusoffering -Dcatalina.home=/usr/local/apache-tomcat-6.0.33 -Djava.io.tmpdir=/tmp -XX:ErrorFile=/tmp/error.log -Xms1024M -Xmx1024M -server -XX:+AggressiveOpts -XX:PermSize=256m -XX:MaxPermSize=256m -XX:NewSize=512m -XX:MaxNewSize=512m -XX:SurvivorRatio=8 -XX:+UseParallelGC -verbose:memory -XX:+PrintGCDetails -XX:+HeapDumpOnOutOfMemoryError -Xcheck:jni -XX:+AlwaysPreTouch -Xrs -Xprof -Xint org.apache.catalina.startup.Bootstrap start

Jruby

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment