Profiling Memory Usage (JVM)

ProfilingMemoryUsageJVM.md

Profiling Memory Usage of a Method (JVM)

Prerequisites:

• YourKit

Steps:

1. Add Thread.sleep(60000) to the START and END of the method you want to profile.

   If you're profiling an async method (e.g. Future/Task/ConnectionIO), put the END in a final flatMap to ensure the results from all previous continuations remain in memory.

2. Add a println(obj) after the last Thread.sleep(60000) for each object you want to measure. This ensures the object is not GC'd before you take the snapshot. This can occur (possibly as a scalac optimisation) when a local variable x is in-scope, but is not used by any subsequent calls. This behaviour has been observed to happen inside for-comprehensions (e.g. for { a <- fooA(); b <- fooB(a); c <- fooC(); } yield() -- in this example if you paused on c <- fooC() then a and b may be GC'd). So it's important you use the objects AFTER the Thread.sleep() where you're taking the snapshot.

3. Run your application, and attach YourKit.

4. Navigate to the Memory section.

5. Wait for 1st Thread.sleep (start of method).

6. Click Advance Object Generation Number in YourKit.

7. Wait for 2nd Thread.sleep (end of method).

8. Click Capture Memory Snapshot in YourKit.

9. Open the snapshot.

10. Navigate to Objects by Category > Reachability > [Green] Objects reachable ... via strong references > Right-click > Selected Objects

11. Then navigate to: Objects by Category > Generation > Last Item in List > Right-click > Selected Objects.

    This will now display all non-GCd objects created while this method was running.

12. Keep clicking 'Calculate Retained Size' until the objects sizes have been calculated.

13. Then either:

    • For the largest object(s): Navigate to Objects by Category > Biggest Objects and expand the top-level items.

      This reveals the largest objects in a non-overlapping way. I.e. each object is only retained by one object in this view (probably the object that was first to ever reference it), so if an object is shared among many other objects, you won't see that object's size bloating the 'retained size' of many objects, like you would in the Class view.

    • For a specific type: Navigate to Objects by Category > Class and find your class.

14. Select class > Right-click > Selected Objects

15. This will show you the individual instances of that type (and their heirarchical memory usage),

Notes

• When you click Advance Object Generation Number YourKit marks all objects currently in the heap as being generation N. If you click the button again, all new objects created since then will be marked as generation N+1, and so on. It's a way of dividing the heap into time slices, so that when you're analysing the heap, you can look only at the objects created during the execution of a method, rather than all the objects on the heap (most of which will have been present before your method was called).

• When you click Capture Memory Snapshot YourKit captures all the objects in the heap at that instant in time (i.e. it does not capture some cumulative set of all objects created from the start of the application).

• It's IMPORTANT to click Capture Memory Snapshot while your method's stack frame still exists. If you click after the method completes, YourKit won't capture sufficient information to correctly calculate the "retained sizes" of the objects that were created while the method was executing, as some/all of those objects may be GC'd.

• Snapshots record both "used" and "unused" objects, where the latter are ready for garbage-collection.

  You can manually force a GC before performing the snapshot, which will remove the majority of the "noise" from the snapshot.

  To avoid having to do this, however, YourKit allows you to filter-our GC-able objects via their "reachability" filter (used above).