perf(ip-sync): avoid redundant client IP reads and writes (#6930)

The access-log scanner and the cross-node merge stored the same IP list in
opposite orders, so neither saw a row as unchanged, and the merge re-read
every preloaded row and re-upserted every attribution row each sync cycle.
All writers now share one canonical encoder (timestamp, then IP ascending),
the first CAS is seeded from the preload, and identical attribution rows are
skipped. A create race between the scanner and node sync now merges into the
winning row instead of dropping the loser's IPs, which on PostgreSQL also
aborted the scanner's whole scan transaction.

Closes #6927
This commit is contained in:
Hamed
2026-10-25 16:43:18 +03:30
committed by GitHub
parent 952c1c751a
commit 8eb58a4a14
22 changed files with 1273 additions and 247 deletions
+25 -4
View File
@@ -1,14 +1,35 @@
package model
// ClientIpEntry is the wire/JSON shape of a single observed client IP with the
// last time it was seen (unix seconds). It mirrors job.IPWithTimestamp and the
// service-internal clientIpEntry so the per-node attribution blob round-trips
// with the existing inbound_client_ips storage.
import (
"cmp"
"encoding/json"
"slices"
)
// ClientIpEntry records an observed client IP and its last-seen Unix timestamp.
type ClientIpEntry struct {
IP string `json:"ip"`
Timestamp int64 `json:"timestamp"`
}
func SortClientIpEntries(entries []ClientIpEntry) {
slices.SortFunc(entries, func(a, b ClientIpEntry) int {
if order := cmp.Compare(a.Timestamp, b.Timestamp); order != 0 {
return order
}
return cmp.Compare(a.IP, b.IP)
})
}
func MarshalClientIpEntries(entries []ClientIpEntry) (string, error) {
if len(entries) == 0 {
return "[]", nil
}
SortClientIpEntries(entries)
encoded, err := json.Marshal(entries)
return string(encoded), err
}
// NodeClientIp records which panel (identified by its stable panelGuid) observed
// a client's IPs on its own Xray. Unlike InboundClientIps (a flattened,
// cluster-wide union used for IP-limit counting and that is pushed back to every