Asked  7 Months ago    Answers:  5   Viewed   77 times

How do you find a memory leak in Java (using, for example, JHat)? I have tried to load the heap dump up in JHat to take a basic look. However, I do not understand how I am supposed to be able to find the root reference (ref) or whatever it is called. Basically, I can tell that there are several hundred megabytes of hash table entries ([java.util.HashMap$Entry or something like that), but maps are used all over the place... Is there some way to search for large maps, or perhaps find general roots of large object trees?

[Edit] Ok, I've read the answers so far but let's just say I am a cheap bastard (meaning I am more interested in learning how to use JHat than to pay for JProfiler). Also, JHat is always available since it is part of the JDK. Unless of course there is no way with JHat but brute force, but I can't believe that can be the case.

Also, I do not think I will be able to actually modify (adding logging of all map sizes) and run it for long enough for me to notice the leak.

 Answers

65

I use following approach to finding memory leaks in Java. I've used jProfiler with great success, but I believe that any specialized tool with graphing capabilities (diffs are easier to analyze in graphical form) will work.

  1. Start the application and wait until it get to "stable" state, when all the initialization is complete and the application is idle.
  2. Run the operation suspected of producing a memory leak several times to allow any cache, DB-related initialization to take place.
  3. Run GC and take memory snapshot.
  4. Run the operation again. Depending on the complexity of operation and sizes of data that is processed operation may need to be run several to many times.
  5. Run GC and take memory snapshot.
  6. Run a diff for 2 snapshots and analyze it.

Basically analysis should start from greatest positive diff by, say, object types and find what causes those extra objects to stick in memory.

For web applications that process requests in several threads analysis gets more complicated, but nevertheless general approach still applies.

I did quite a number of projects specifically aimed at reducing memory footprint of the applications and this general approach with some application specific tweaks and trick always worked well.

Tuesday, June 1, 2021
 
mnagel
answered 7 Months ago
23

Use

jps -v

for finding your java process. Sample Output:

3825 RemoteMavenServer -Djava.awt.headless=true -Xmx512m -Dfile.encoding=MacRoman
6172 AppMain -Didea.launcher.port=7533 -Didea.launcher.bin.path=/Applications/IntelliJ IDEA 10.app/bin -Dfile.encoding=UTF-8
6175 Jps -Dapplication.home=/Library/Java/JavaVirtualMachines/1.6.0_31-b04-411.jdk/Contents/Home -Xms8m

Then use

jstack 6172

(6172 is id of your process) to get stack of threads inside jvm. Thread priority could be found from it. Sample output:

.....
"main" **prio=5** tid=7ff255800800 nid=0x104bec000 waiting on condition [104beb000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
    at java.lang.Thread.sleep(Native Method)
    at au.com.byr.Sample.main(Sample.java:11)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)

..... 

Enjoy!

EDIT: If application running under different user than yourself (typical case on production and other non-local environments) then jps/jstack should be run via sudo. Examples:

sudo jps -v

sudo jstack 6172
Wednesday, June 30, 2021
 
erotsppa
answered 6 Months ago
25

second the valgrind... and I'll add electric fence.

Tuesday, July 27, 2021
 
edsk
answered 5 Months ago
50

What you are seeing is most likely not a memory leak at all. Operating systems and malloc/new heaps both do very complex accounting of memory these days. This is, in general, a very good thing. Chances are any attempt on your part to force the OS to free the memory will only hurt both your application performance and overall system performance.

To illustrate:

  1. The Heap reserves several areas of virtual memory for use. None of it is actually committed (backed by physical memory) until malloc'd.

  2. You allocate memory. The Heap grows accordingly. You see this in task manager.

  3. You allocate more memory on the Heap. It grows more.

  4. You free memory allocated in Step 2. The Heap cannot shrink, however, because the memory in #3 is still allocated, and Heaps are unable to compact memory (it would invalidate your pointers).

  5. You malloc/new more stuff. This may get tacked on after memory allocated in step #3, because it cannot fit in the area left open by free'ing #2, or because it would be inefficient for the Heap manager to scour the heap for the block left open by #2. (depends on the Heap implementation and the chunk size of memory being allocated/free'd)

So is that memory at step #2 now dead to the world? Not necessarily. For one thing, it will probably get reused eventually, once it becomes efficient to do so. In cases where it isn't reused, the Operating System itself may be able to use the CPU's Virtual Memory features (the TLB) to "remap" the unused memory right out from under your application, and assign it to another application -- on the fly. The Heap is aware of this and usually manages things in a way to help improve the OS's ability to remap pages.

These are valuable memory management techniques that have the unmitigated side effect of rendering fine-grained memory-leak detection via Process Explorer mostly useless. If you want to detect small memory leaks in the heap, then you'll need to use runtime heap leak-detection tools. Since you mentioned that you're able to build on Windows as well, I will note that Microsoft's CRT has adequate leak-checking tools built-in. Instructions for use found here:

http://msdn.microsoft.com/en-us/library/974tc9t1(v=vs.100).aspx

There are also open-source replacements for malloc available for use with GCC/Clang toolchains, though I have no direct experience with them. I think on Linux Valgrind is the preferred and more reliable method for leak-detection anyway. (and in my experience easier to use than MSVCRT Debug).

Friday, August 13, 2021
 
jakubos
answered 4 Months ago
39

One way is to insert file name and line number strings (via pointer) of the module allocating memory into the allocated block of data. The file and line number is handled by using the C++ standard "__FILE__" and "__LINE__" macros. When the memory is de-allocated, that information is removed.

One of our systems has this feature and we call it a "memory hog report". So anytime from our CLI we can print out all the allocated memory along with a big list of information of who has allocated memory. This list is sorted by which code module has the most memory allocated. Many times we'll monitor memory usage this way over time, and eventually the memory hog (leak) will bubble up to the top of the list.

Monday, September 13, 2021
 
Manuel Selva
answered 3 Months ago
Only authorized users can answer the question. Please sign in first, or register a free account.
Not the answer you're looking for? Browse other questions tagged :  
Share