feat(notifications): add a failed-check threshold for node.down alerts (#6768)

One lost heartbeat on a lossy link flipped a node offline and sent node.down,
and the next probe sent node.up, so a flaky node flooded Telegram, email and
Discord with alert pairs.

A new nodeDownThreshold setting (default 1, today's behaviour) works like
outboundDownThreshold: node.down fires once the failed-check streak reaches it,
once per outage, and node.up only after a node.down was sent. The announcement
is persisted in nodes.down_notified, so a restart inside the threshold window
neither drops the outage's node.down nor re-announces nodes already reported
offline; nodes offline at upgrade are seeded as announced.
This commit is contained in:
Grigoriy
2026-10-24 05:26:13 +03:00
committed by GitHub
parent c3037fc716
commit 2f62dadd25
34 changed files with 379 additions and 35 deletions
+1
View File
@@ -80,6 +80,7 @@ type AllSetting struct {
DiscordEnabledEvents string `json:"discordEnabledEvents" form:"discordEnabledEvents"`
OutboundDownThreshold int `json:"outboundDownThreshold" form:"outboundDownThreshold" validate:"gte=1,lte=100"`
NodeDownThreshold int `json:"nodeDownThreshold" form:"nodeDownThreshold" validate:"gte=1,lte=100"`
TimeLocation string `json:"timeLocation" form:"timeLocation"`
TwoFactorEnable bool `json:"twoFactorEnable" form:"twoFactorEnable"`