Long-running processes in Boomi can significantly impact overall system performance and expected response times. These processes often consume a disproportionate amount of resources, leading to bottlenecks that can slow down other operations within the integration environment. In a typical Boomi setup, integration processes are expected to execute quickly and efficiently, facilitating real-time or near-real-time data exchanges. When processes run for extended periods, they can monopolize CPU cycles and memory, preventing other, more critical tasks from executing in a timely manner.

Memory management plays a crucial role in the performance of long-running Boomi processes. Each integration process runs in the Java Virtual Machine (JVM), which has specific memory allocation settings that govern how resources are distributed. When a process runs for too long, it may not release memory properly, leading to memory leaks. Over time, these leaks can consume the available heap space, causing the JVM to slow down as it struggles to manage the dwindling resources. Eventually, if the JVM runs out of memory, it can trigger garbage collection cycles that momentarily halt all processes, further degrading performance and response times.

Moreover, the JVM is designed to handle multiple threads and processes concurrently, but long-running integrations can disrupt this balance. If a significant number of threads are tied up in lengthy operations, it can lead to thread contention, where processes must wait for resources to become available. This contention not only delays the completion of the long-running processes but also hampers the overall throughput of the integration platform, resulting in slower response times for all users.

Finally, the architecture of Boomi and its reliance on cloud-based or hybrid deployment models can exacerbate these issues. Network latency and cloud resource availability are critical factors in performance. Long-running processes may lead to timeouts or errors in network communication, impacting the reliability of integrations. As businesses increasingly rely on real-time data for decision-making, the detrimental effects of long-running processes become even more pronounced, highlighting the need for optimized, efficient integration workflows that minimize processing time and resource consumption.

Simulating Long Running Boomi Processes

Simulating a long-running process in Boomi can be useful for testing performance, resource consumption, and error handling. Ensure that the process can handle simultaneous executions. Here’s a step-by-step approach to create a simple long-running process:

1. Create a New Process

• In the Boomi platform, start by creating a new process. Give it a meaningful name that reflects its purpose (e.g., “Simulate Long Running Process”).

2. Add a Start Shape

• Use a “Start” shape to initiate the process. You can configure it to trigger via a scheduled start, HTTP request, or another suitable method based on your testing needs.

3. Add a Data Shape

• Introduce a “Data” shape to define any incoming data. For a simulation, you can either use a dummy payload or generate random data.

4. Introduce a Delay

• To simulate the long-running aspect, use a “Set Properties” shape to define a delay. For example, you can set a variable that specifies how long the process should “sleep.”
• Then, include a “Custom Script” or a “Process Call” shape that includes a delay. In the script, you could use a command like Thread.sleep(milliseconds) where the duration is long enough to simulate a lengthy operation (e.g., 300,000 milliseconds for 5 minutes).

5. Add Processing Logic

• After the delay, add any necessary processing logic. This could be data transformation or integration with another system.

6. Add an End Shape

• Finally, conclude the process with an “End” shape to signify completion.

Fig: Boomi Process Builder Screen – Long Running Process

Observing Boomi Process Execution

In order to simulate multiple concurrent long running processes executing at once. I executed 7 processes with a 5 minute delay all at once. 

Fig: Boomi Process Reporting – Multiple Concurrent Long Running Processes

Notably, there are no performance concerns and all processes have executed successfully. When checking the container logs, there is no mention of a performance issue or system failure.

yCrash Installation and Configuration

The latest version of the yCrash Agent needs to be installed. Please visit the document hub for yCrash here for further installation instructions. Below is a sample config.yaml that covers the configuration for the agent with the Boomi module enabled:

version: '1'
options:
   k: <your server key>
   s: http://localhost:8080 #(or other address for the yCrash Receiver)
   j: /opt/homebrew/opt/openjdk/ #jdk path
   p: 67730 #port for process that is being monitored
   boomi: true #enable the boomi module true / false
   boomiUrl:  https://api.boomi.com/api/rest/v1/<account id>/ExecutionRecord/query
   boomiUser: <username to https://platform.boomi.com>
   boomiPassword: <password to https://platform.boomi.com with above username combo>
   boomiAcctId: <account id>
   period: 2 # 1 hour, 2 hours etc - lookback period for job information
   atomId: <guid for atom id found under atom management, atom information>

To run the agent with your newly created config YAML file and enable m3 collection:

mac/yc -c config.yaml -m3

yCrash Diagnostics with Boomi Module

To identify long running jobs and evaluate them against JVM runtime metrics, I decided to use the new yCrash Boomi module to assist my diagnostic troubleshooting for these long running jobs. Inside the generated incident summary dashboard there is a new Boomi icon to utilize.

Fig: yCrash Incident Dashboard

The screenshot below captures the Boomi Report which is rendered after I clicked on the Boomi icon inside the dashboard.

Fig: yCrash Incident Boomi Report  – Top View

Conclusion

Diagnosing and triaging long-running Boomi processes is crucial for maintaining the efficiency and reliability of integration workflows. Such processes can lead to bottlenecks that affect overall system performance, delay data processing, and disrupt business operations. Identifying the root causes of these delays—whether due to inefficient configurations, unexpected data volumes, or external system issues—enables organizations to optimize their workflows and ensure timely data integration. Regular monitoring and analysis of these processes can help prevent cascading failures, enhance system reliability, and ultimately improve user satisfaction.

yCrash serves as a valuable diagnostic tool in this context, providing in-depth insights into Boomi processes and their performance. By capturing detailed error reports and process execution metrics, yCrash enables teams to quickly identify performance anomalies and bottlenecks. Its ability to analyze process executions in real-time allows for prompt triage, helping teams to focus on the most pressing issues affecting long-running processes. By leveraging yCrash, organizations can enhance their diagnostic capabilities, streamline troubleshooting efforts, and make informed decisions that lead to more efficient integration processes and better resource management.