Monitors on charts¶
A monitor is two things: a number it watches, and a limit it watches for. The charts draw both, so you can see how close a metric is running to its limit without waiting for the alert that says it crossed.
This page is about reading that. It applies to every MPM chart that can carry a monitor, but most of it matters on the distribution chart, which draws seven metrics at once and so has the most ways to surprise you.
The line and the band¶
A monitor's threshold appears as a dashed line, with a faint wash over the side that would alert.
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ ← the threshold
░░░░░░░░░░░░░░░░░░░░░░░░░ ← the alerting side, shaded
▁▂▃▂▁▁▂▃▄▃▂▁▁▂▃▂▁▁▂▃▄▅▄▃▂▁▁▂▃ ← the metric
Which side is shaded comes from the monitor's rule, not from the number. The same threshold of 0.2 means "shade above" for a drift monitor and "shade below" for one watching AUC, so the chart reads it off the comparison you chose: is greater than shades upward, is less than shades downward.
The wash reaches past the data deliberately, so a metric sitting comfortably inside its limit still shows a shaded region rather than a sliver. It is drawn faintly and beneath the data, because it is context for reading the chart rather than the subject of it.
One band is drawn per monitor, not per line. Splitting a chart by predicate draws several lines, but the monitor behind them is the same one — shading once per line would stack the same wash over itself and read as more urgent than it is.
A distribution chart draws seven metrics, not one¶
This is the part worth knowing before it puzzles you.
The distribution chart for a numerical column does not plot the column. It plots seven separate metrics computed from it:
| series | metric |
|---|---|
min | minimum |
p10, p25, p50, p75, p90 | percentiles — p50 is the median |
max | maximum |
A monitor watches exactly one of them. "Alert when the 90th percentile of amount exceeds 4000" is a monitor on p90, and it says nothing about the median.
So a band on one series and none on another is correct, not a missing band:
4000 ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ ← monitor on p90
░░░░░░░░░░░░░░░░░░░░░
▁▁▂▂▃▃▂▂▁▁▁▂▂▃▃▂▂▁▁▁▁ ← p90
▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ← p50, unmonitored, no band
The seven series share a column, so the column name cannot tell them apart. The metric is what separates them, and it is what the chart matches on.
Note
A threshold is only meaningful against the series it was set on. A limit on the 90th percentile drawn across the median would claim the median is being judged against a number nothing is judging it against — which is why the chart will not do it, even though both lines come from the same column.
When a monitor is named instead of drawn¶
Sometimes a monitor exists, applies to the column, and still has no band. The chart says so above it rather than leaving you to wonder, because a missing band otherwise reads as "nothing is watching this column".
There are two reasons.
The series it watches is not on screen¶
The chart draws three of its seven series at a time, chosen by the Percentiles control:
| setting | series drawn |
|---|---|
| Minimum - Maximum | min, p50, max |
| 10th Percentile - 90th Percentile | p10, p50, p90 |
| 25th Percentile - 75th Percentile | p25, p50, p75 |
| No Shadow | p50 only |
A monitor on p90 has nothing to attach to while you are looking at Minimum - Maximum. Rather than drawing its line against a series that is not there, the chart names it and tells you where to look:
1 monitor on p90 p90 ceiling — switch to "10th Percentile - 90th Percentile" to see it
Change the setting and the band appears.
On dashboard panels the notice names the series but does not tell you to switch, because those panels have no Percentiles control to switch with. Set the percentile range in the panel's own configuration instead.
It is scoped to a different model version¶
A monitor can be bound to one model version. A chart showing all versions plots an aggregate across them — a number that monitor never evaluated — so its limit is not a limit on what you are looking at, and the chart does not draw it:
1 monitor scoped to 1.0.0 p25 ceiling (p25)
This one names the version rather than telling you to switch, because the version selector is not part of the chart.
Note
This cuts both ways, and it is worth knowing when you create a monitor. A monitor saved against a specific version will not appear on the all-versions chart, and a monitor saved with no version will not appear on a chart filtered to one. If you want a monitor visible where you normally look, create it with the same version scope you normally view.
Alerts that have fired¶
When a monitor fires, the moment it fired is marked on the chart, on the series that monitor watches — the same series its band is on. A marker on the median means a monitor on the median fired, not a monitor somewhere else on the same column.
Markers and bands are selected together, so they cannot disagree about which monitors belong to a chart: if you see a marker, that monitor's band is on the same line.
Read the absence of a marker more carefully than its presence, though. Only alerts raised inside the time range you are looking at are drawn, so widening the range can reveal markers that were always there.
The other charts¶
Number of predictions, missing values, drift, custom metrics and the classification curves all draw thresholds the same way — a dashed line and a shaded alerting side.
Custom metrics are the one other place where a chart has several series. A custom metric returning more than one column draws one line per column, and a monitor watches one of them, matched by column name. As with the distribution chart, a band on one line and not another is the monitor being specific, not the chart being wrong.
Learn more¶
- Drift — the drift algorithms and choosing a threshold for them
- Classification performance — monitoring ROC AUC, KS and Gini
- Custom metrics — defining a metric to monitor
- MPM UI — where these charts live