Detecting Anomalies with Basil

You're using the Basil Application Performance Management Tool to watch your web application and page you when pages slow down or throughput spikes. Basil uses historical data to build a baseline for expected performance and pages you when it detects an anomaly based on the following parameters:

• The number of standard deviations \(\sigma\) from the baseline that represent a violation.
• The number of minutes of consecutive violations (lead time) required to flag an anomaly and send you a page.

How do you tune these parameters so you get notified when things are bad but don't get woken up for a non-emergency?

Knowing how to set the parameters will be tricky so I've built an application that let's you see what alerts would be generated on a sample dataset for any given parameters.

The tool will show you a plot of real time page response time averages over a six hour period along with the baseline used to determine when response times are excessively slow.

Based on the settings you will see areas highlighted in red where an alert would be triggered given the parameters you set.

With a lead time of 6 minutes using 3 \(\sigma\), you can see in the sample data the gray area at \(baseline+/-3\sigma\) where alerts will get triggered. With these parameters you get poor selectivity--there are too many alerts.

With a lead time of 1 minutes using 6 \(\sigma\), you get fewer alerts but they also clear too quickly--they are too narrow.

The ideal parameters depend on the underlying data and your prefererred tradeoffs for sensitivity and selectivity. Using the Basil Parameter Tuning application you can adjust parameters and examine the alerts that would be triggered until you are satisfied the algorithm meets your needs.