Alerts & hysteresis

Alerts fire when a metric breaches a threshold. To keep them reliable rather than noisy, the monitor has several mechanisms.

Warning and critical

Each rule can have a warning and a critical threshold, so an alert escalates in stages instead of jumping straight to critical.

Hysteresis – against flapping

Without hysteresis, a metric hovering around the threshold produces a flood of open/close alerts. Two fields prevent that:

• Resolve threshold (ResolveThreshold) – the alert only counts as resolved once the metric drops below this (lower) value, not the moment it dips under the trigger threshold.
• Consecutive good samples (ResolveConsecutiveSamples) – several good readings in a row are required before resolving.

Example: trigger at CPU > 90 %, resolve only below 70 % after three good samples in a row. A brief spike to 91 % and back no longer creates an alert storm.

Silences – maintenance windows

During planned maintenance (backups, deployments), silences (Settings → Silences) suppress alerts for a period, optionally per server or per rule.

Severity routing

Alerts are routed to channels by severity – e.g. warnings by email, critical also to Microsoft Teams or PagerDuty. Channels are configured, encrypted, in the database (Settings → Notifications).

Server-offline correlation

If a server itself is unreachable, every dependent metric would alarm at once. The monitor detects this and suppresses the dependent alerts, reporting only that the server is offline.

Tuning

The Reports → Alert Stats page shows firing counts and false-positive ratio per rule – the basis for sharpening thresholds over time.