The access-log scanner and the cross-node merge stored the same IP list in
opposite orders, so neither saw a row as unchanged, and the merge re-read
every preloaded row and re-upserted every attribution row each sync cycle.
All writers now share one canonical encoder (timestamp, then IP ascending),
the first CAS is seeded from the preload, and identical attribution rows are
skipped. A create race between the scanner and node sync now merges into the
winning row instead of dropping the loser's IPs, which on PostgreSQL also
aborted the scanner's whole scan transaction.
Closes#6927
Most tests opened a throwaway panel DB with database.InitDB, which runs the
full AutoMigrate + seed on an empty file every time: ~230ms, and ~850ms under
-race because GORM's reflection-heavy migration is what the detector slows
most. internal/web/service does this in ~550 of its 830 tests, so the CI race
job spent ~10 of its ~14.6 minutes re-migrating empty databases.
internal/database/dbtest.InitDB migrates once per test process, then hands
each test its own copy of that file (~130ms under -race) and registers the
CloseDB cleanup. The copy then goes through InitDB like a panel restart, so
every test still starts from the state a fresh install has. Tests that reopen
an existing file, migrate a hand-built legacy DB or target Postgres keep
calling database.InitDB.
Locally under -race: internal/web/service 626s (last CI run) -> 114s,
internal/sub 246s -> 35s.
Replace strings.Split loops with strings.SplitSeq iterators in the CSV
parsers (reality_scan and the scale-test helpers) and swap a manual map
copy for maps.Copy in the MTProto traffic collector. No behavior change;
these are the fixes the modernize analyzer reports.
The paths that run continuously in production had no scale coverage: the
5s traffic poll (AddTraffic with its auto-renew and depleted scans), the
websocket snapshot the job broadcasts while a browser is connected, the
10s ip-limit job (hasLimitIp LIKE scan + per-email settings parse), a
subscription fetch inside a huge inbound, and the full Xray config build.
New benchmarks reuse the XUI_SCALE_TEST / XUI_DB_TYPE gating and stay
log-only. Sizes default to 10k/100k; XUI_SCALE_SIZES=500000 raises the
ladder without editing code. seedScaleDataset writes inbounds, clients,
client_inbounds and client_traffics directly in one transaction instead
of SyncInbound, so a 500k seed takes seconds. XUI_SCALE_DB_PATH persists
the seeded SQLite file for manual smoke runs against a live panel.
Hamed
MHSanaei