Analyzing & Troubleshooting StackOverflowError Using JFR 

Java Flight Recorder (JFR) provides continuous, low-overhead profiling by capturing runtime execution samples directly within the JVM. In this blog, we will examine StackOverflowError, a performance issue commonly caused by unbounded or excessive recursive method calls that exceed the thread’s allocated stack size. We will simulate the problem, capture the relevant JFR data, and analyze the recording to identify the underlying issue. Let’s take a closer look.

What is StackOverflowError?

Fig: Progressive stack frame accumulation leading to StackOverflowError

Each thread in the JVM has its own stack, which holds a frame for every method call that hasn’t yet completed. A frame stores that method’s local variables, references to any objects it created, and the point it should return to once finished. Frames are pushed onto the stack when a method is called and popped off once it completes. The stack itself has a fixed size limit, and if it grows beyond that limit, the JVM throws java.lang.StackOverflowError.

What causes ‘StackOverflowError’?

Let’s look at the list of causes for this StackOverflowError issue: 

1. Recursive Method Calls Without a Base Case: A method that calls itself with no terminating condition will keep pushing new frames onto the stack indefinitely, since none of the earlier calls ever return. Eventually, the stack runs out of room.
2. Cyclic Relationships Between Classes: If constructing an object of one class triggers the construction of another class, which in turn constructs the first class again, this creates the same unbounded growth as direct recursion, just spread across two or more classes instead of a single method.
3. Too Many Local Variables Held Across a Long-Running Call: Less common, but if a method creates a large number of variables and then calls another long-running method before returning, those variables stay on the stack for the entire duration, which can compound the problem, especially across many concurrent threads.

Simulating StackOverflowError Performance Issue

To understand how StackOverflowError appears in JFR data, let’s reproduce the issue using a sample Java application. The following program deliberately recurses without a terminating base case, so each call adds another frame to the stack until it runs out of space.

public class StackOverflowDemo {
    private static boolean flag = true;
    public static void setFlag(boolean newValue) {
        flag = newValue;
    }
    public int counter = 0;
    public void start() throws InterruptedException {
        ++counter;
        if (counter % 1000 == 0) {
            System.out.println("Looped " + counter + " times");
        }
        if (flag) {
            Thread.sleep(20);
            start();
        }
    }
    public static void stop() {
        System.out.println("Stack overflow problem terminated!");
    }
}

In this program, start() increments an instance-level counter on every call, then, as long as the static flag remains true, sleeps for 20 milliseconds and calls start() again, recursively. Since nothing in the normal execution path ever calls setFlag(false), this recursion has no exit condition: every call to start() adds a new stack frame on top of the last, without any of them ever returning. As the recursion continues, the thread’s call stack grows deeper and deeper until it exceeds the JVM’s allocated stack size for that thread, at which point the JVM throws java.lang.StackOverflowError. 

Capturing JFR Data for Troubleshooting StackOverflowError

To capture JFR data for troubleshooting the StackOverflowError issue, we recommend that you follow the steps below:

If you are interested in learning about the other methods available to capture JFR recordings, we recommend that you read ‘How to Capture Java Flight Recorder (JFR)?’ blog.

Step 1: Start a JFR recording against the running JVM:

jcmd {PID} JFR.start \
    name=loadTestCapture \
    settings=profile \
    filename=/tmp/tomcat.jfr

Note: Replace <PID> with the Process ID (PID) of your Java application running inside the container. If you don’t know how to find it, refer to our guide on finding the Java application Process ID (PID) for step-by-step instructions.

Step 2: Let it run while the StackOverflowError is occurring, then stop it manually: 

jcmd {PID} JFR.stop \
    name=loadTestCapture

Step 3: Alternatively, for Fixed-Duration Capture,  you can start a recording that automatically exits after a predefined duration (for example, 15 minutes) in a single step:

jcmd {PID} JFR.start \
    name=loadTestCapture \
    settings=profile \
    duration=15m \
    filename=/tmp/tomcat.jfr

Analyzing StackOverflowError Using the JFR Data

You can analyze JFR recording using yCrash JFRPlayer by following the steps mentioned below.  

Step 1: Install yCrash JFRPlayer, which is available in both cloud and on-premises versions. Use one of the options below to get started:

• Cloud service: Register and upload your JFR file online.
• On-Premises: Install and run yCrash JFRPlayer on your local machine or within your organization’s environment.

Step 2: Upload the jfr file to your yCrash JFRPlayer. Once JFR file is uploaded, JFRPlayer parses the JFR file and generates an incident report instantly.

Fig: Uploading a standalone JFR file to yCrash JFRPlayer 

Step 3: I’d recommend you review the AI overview section first, as it gives you an executive summary of the issue in plain language, along with the Root Cause Analysis (RCA). It identifies a severe recursive loop in the main thread, tracing it directly to StackOverflowDemo.start(), repeated across dozens of stack frames. 

Fig: yCrash JFRPlayer AI Overview identifying a recursive loop in StackOverflowDemo.start()

Step 4: Click into the main thread’s stack trace to confirm the recursion: dozens of repeated frames at StackOverflowDemo.start(StackOverflowDemo.java:32), each one calling the next, matching the source code precisely.

Fig: Stack trace confirming repeated recursive calls to StackOverflowDemo.start() 

Simple, right? Now that we have analyzed the data, we are equipped with all the information to fix the problem.

If you face any challenges while analyzing a JFR file using yCrash JFRPlayer, check out our FAQ for answers to common questions and troubleshooting guidance.

How to fix StackOverflowError

The following are potential solutions to fix this issue:

1. Add a Termination Condition: Give the recursion an actual exit point, for example, a counter limit or a condition that flips flag to false once the intended work is done, rather than relying on it staying true indefinitely.
2. Convert Recursion to Iteration: Where possible, rewrite the recursive calls as a loop instead. A while loop performs the same repeated work without adding a new stack frame on every pass.
3. Review the Stack Trace for the Repeating Pattern: As seen in this report, a StackOverflowError‘s stack trace will show the same method and line number repeating many times. Spotting that pattern quickly points to the exact recursive call responsible.
4. Increase Thread Stack Size as a Last Resort: If the recursion is legitimate and deep by design, rather than a bug, increasing the stack size with the -Xss JVM argument can help, though fixing the underlying recursion should always be tried first.

Conclusion

Diagnosing and identifying the root cause of a StackOverflowError can be challenging, especially in complex production environments where multiple symptoms often overlap. In this example, the JFR recording was analyzed using yCrash JFRPlayer, which automatically identified the key performance bottlenecks and correlated JVM events to pinpoint the underlying problem. The analysis traced the issue to the main thread recursively calling StackOverflowDemo.start() with no termination condition. By examining the relevant JFR insights, including execution samples and repeated stack frame patterns, we were able to identify the root cause and determine the appropriate fix.