FusionReactor CPU Alerting and Crash Protection
In this blog, we’re going to show you a great feature of FusionReactor, which is how to configure CPU Alerting for Crash Protection. This Alert is based on the “Instance” CPU, rather than the Server CPU.
What Is CPU Alerting?
CPU alerting tells you if you have a request using a large amount of the CPU or any background processes using vast amounts of resources or a memory tuning issue. For example, if your garbage collector is using too much CPU then your alert will tell you as it happens.
The CPU protection works in a very similar way to memory protection. You configure the threshold, the minimum duration, and then set your alerting strategy. You have a few options, such as generating an email, or you could queue requests in FusionReactor until the usage is under the threshold. Or you can reject requests in FusionReactor until the CPU is under the threshold. A combination of these can also be used.
So as an example, I’ll set my CPU usage to 50% for a duration of 10 seconds and then set the alert to email notification only. If I save these settings and then run a request that runs Fibonacci on a number of threads, this will give us the CPU usage.
While we wait for the alert to trigger, we can take a look at the email settings. To configure emails, you can do this in the FusionReactor Settings area. The email credentials are added in the first tab, such as the email address, username, and password. Remember to check the box in the “Email Settings” option, to ensure the box is set to “Enabled” for Notification emails.
Back in the Web Metrics, if we look at the last minute, we’ve had constant CPU usage of just above 50%. If I check my email, I can now see that I have my crash protection email notification from FusionReactor. So if I look at my Alert, I can see in the information section the threshold, the actual values, and the trigger time. Under the “Load”, I can see the number of requests running as well as the CPU usage at the time, which was 56%. So it was above the 50% needed to trigger, and then the memory usage.
One thing to note is that even though the Fibonacci request was using the CPU, it’s not always going to be the triggering request. Because the alert is based on the CPU usage “at the time”, it’s not always possible to route the triggering request back to the “Instance CPU”.
This means in our case, “execute query”, was the triggering request. “Fibonacci” which was actually in the “Running Requests”, was the one that caused the issue. So you can’t see the exact request, but you can see all that was running at the time the alert fired. This is the same with the memory protection as well as the Web Requests quantity protection.