Java Production Issue Troubleshooting

1. 100% Memory Usage / Memory Problems

1.1. Find the Java Process ID

Run top, then press M to see which Java process is using the most memory.

1.2. Analyze Whether OOM Has Occurred

OOM Example

2. 100% CPU Usage / Thread Problems

2.1. Find the Java Process ID and Thread ID

1. Run top, press P to see which Java process has high load, and record its PID.
2. Run top -Hp <PID from step 1> to view the running state of all threads in that process.
3. Record the high-load thread ID and run printf "%x\n" <thread ID> to convert it to hexadecimal.

2.2. Analyze the Thread Dump File

4. Run jstack <PID from step 1> > <file> to generate a jstack file containing stack states.
5. In the jstack file from step 4, find the thread whose nid equals the hexadecimal thread ID from step 3.
6. Use the stack trace to understand what task the thread is executing, then combine that with the business context and code to locate the problem.

3. Thread Deadlock

3.1. Find the Java Process ID

There are two methods:

• Use top to find the PID whose STATE is sleeping and COMMAND is java.
• Use jps.

3.2. Analyze the Thread Dump File

jstack -l PID

Analyze whether there is a cyclic locking situation.

4. JVM Tuning

Tuning Approach

5. References

• jstack Thread Analysis - Jianshu
• Summary of Troubleshooting Java Memory Leaks - CSDN
• Using Memory Analyzer Tool (MAT) to Analyze Java Project Memory Leaks - CSDN
• JVM Optimization Essentials — Monitoring Tools - Juejin