To forecast production performance problems accurately, we need to validate the right set of Micro-Metrics in the performance labs. In the earlier post, I shared which Micro-Metrics we should track: ‘Micro-Metrics Every Performance Engineer Should Validate Before Sign-Off‘. In this post, let’s discuss what are the best practices that we should follow when capturing those Micro-Metrics.
1. Enable Garbage Collection Log
Garbage Collection Log is a vital artifact to uncover various performance problems such as: Memory Leaks, OutOfMemoryError, Long GC Pauses, Frequent GC Pauses, Memory Over Allocation, Memory Under Allocation… So, make sure to enable GC Log on your application, if it’s not already enabled. You can enable GC Logging on your application by passing these JVM arguments.
2. Trigger ‘yc-360 Script’ in the middle of the Test
‘yc-360 Script’ should be triggered midway through the performance test, when the application is handling peak traffic. Say you are running the performance test for 60 minutes, then ‘yc-360 Script’ should be triggered in the middle of the test at the 30th minute. This helps us to identify any bottlenecks which often surface during peak load.
3. Trigger ‘yc-360 Script’ at the end of the Test
Also run the ‘yc-360 Script’ at the end of the test, once traffic has subsided. This helps you verify whether all resources (threads, memory, connections, …) are being properly released after peak usage.
4. For Endurance Test, run ‘yc-360 Script’ in M3 Mode
If you are running a long-duration endurance test (like 24 hours), run the ‘yc-360 Script’ in a self-trigger m3-mode. In this mode, the script will continuously monitor the application. If any anomalies are detected or any thresholds are breached, automatically script will capture the 360° artifacts and report the issue on your yCrash dashboard.
5. Compare with Baseline Incident Reports
Once Micro-metrics incident reports are captured in the new release code, compare them with the previous release reports. yCrash supports comparison of reports to see trends like increased pause times, memory usage drift, or GC frequency changes over time.
Final Thoughts
Capturing & Analyzing Micro-Metrics facilitates you to catch the silent degradations before they hit production. Following the above best practices will ensure you leave fewer blind spots in your performance certification process.