Most tests opened a throwaway panel DB with database.InitDB, which runs the
full AutoMigrate + seed on an empty file every time: ~230ms, and ~850ms under
-race because GORM's reflection-heavy migration is what the detector slows
most. internal/web/service does this in ~550 of its 830 tests, so the CI race
job spent ~10 of its ~14.6 minutes re-migrating empty databases.
internal/database/dbtest.InitDB migrates once per test process, then hands
each test its own copy of that file (~130ms under -race) and registers the
CloseDB cleanup. The copy then goes through InitDB like a panel restart, so
every test still starts from the state a fresh install has. Tests that reopen
an existing file, migrate a hand-built legacy DB or target Postgres keep
calling database.InitDB.
Locally under -race: internal/web/service 626s (last CI run) -> 114s,
internal/sub 246s -> 35s.
A master's per-node sync must scope what it sends to the clients that node
serves, so its cost tracks the node and not the fleet. The global-usage
push already did (node_client_traffics by node_id); the 10s client-IP push
sent GetAllInboundClientIps, the whole table, to every node.
Each node's MergeInboundClientIps then created a row for every foreign
email, and its next GET clientIps echoed the whole fleet back. Its IP-limit
job only ever reads rows for its own clients, so none of it was used. With
150 nodes x 150 clients, one IP tick pushed 299 MB and pulled 264 MB, every
node held 22,500 rows instead of 150, and sync ticks grew 3.8s -> 10.2s
even at 1ms latency; the cost grows with the square of the fleet.
Both pushes now share nodeHostedEmails. After the change the same fleet
moves 2.0 MB / 1.8 MB per tick and ticks stay near 3.2s. Nodes upgraded
with foreign rows shed them within 30 minutes via pruneStaleIpRows.
MHSanaei