fix(ci): bound the forgotten-sibling report so an advisory step stops failing the job (#13889)

"Fast Quality Gates" is red on every open PR against release/v3.8.51. The failing
step is `forgotten-sibling-tests`, which is explicitly advisory — its own output
says "Report-only calibration: these findings do not fail the job" — yet it exits 1.

When a PR diff touches a hub module (`open-sse/config/providerRegistry.ts` in the
current reds), the analysis walks every import edge in the repo and multiplies each
consumer by its candidate tests. The result reaches millions of rows, and
`lines.join("\n")` then exceeds V8's maximum string length. The throw lands in
main()'s catch, which exits 1 — so an advisory report takes the whole job down.

Measured with a synthetic hub cross-product, before the change:

  3,000,000 findings -> a 435 MB report string (no throw, but absurd)
  4,500,000 findings -> Invalid string length   (the CI failure, verbatim)

After: the same 4,500,000 findings render as 27 KB.

The fix bounds only the ENUMERATION. The header keeps the exact totals, so the
signal ("this diff has N unreviewed sibling tests") is unchanged; at most 200 rows
per section are listed, followed by a line naming how many were withheld. The JSON
artifact gets the same treatment (5,000 items per array) plus an explicit `totals`
object, since `JSON.stringify` would throw on the same input for the same reason.

`markdown()` is exported so the bound is testable without a CI-sized diff.
This commit is contained in:
Diego Rodrigues de Sa e Souza
2026-09-16 11:45:40 -03:00
committed by GitHub
parent 5faf44f975
commit 1cb4c82071
2 changed files with 47 additions and 10 deletions

View File

@@ -0,0 +1 @@
- **fix(ci):** the advisory `forgotten-sibling-tests` step no longer fails "Fast Quality Gates" when a PR touches a hub module — the cross-product of consumers × candidate tests reached millions of rows and rendering them exceeded V8's maximum string length, so the throw hit `main()`'s catch and exited 1. The report now lists at most 200 rows per section (and 5 000 per array in the JSON artifact) while the header keeps the exact totals

View File

@@ -205,7 +205,26 @@ function changedSymbols(root, base, entries) {
return result;
}
function markdown(result, base) {
// A changed hub module (providerRegistry.ts, providers.ts, …) is imported by thousands of
// consumers, and every consumer multiplies by its candidate tests, so the cross-product reaches
// millions of rows. Rendering all of them made `lines.join("\n")` exceed V8's maximum string
// length; the throw landed in main()'s catch, which exits 1 — so an ADVISORY step turned
// "Fast Quality Gates" red on every PR whose diff touched a hub (#13866 follow-up). The header
// keeps the exact totals; only the enumeration is bounded.
const RENDER_LIMIT = 200;
const JSON_ITEM_LIMIT = 5000;
/** First `limit` items plus a one-line note naming how many were withheld. */
function renderBounded(lines, items, format, limit = RENDER_LIMIT) {
for (const item of items.slice(0, limit)) lines.push(format(item));
if (items.length > limit) {
lines.push(
`- _… and ${items.length - limit} more not listed (report bounded at ${limit} rows per section; the counts above are exact)._`
);
}
}
export function markdown(result, base) {
const lines = [
"## Forgotten sibling tests (advisory)",
"",
@@ -218,12 +237,10 @@ function markdown(result, base) {
];
if (result.findings.length) {
lines.push("### Candidate tests absent from this diff", "");
for (const item of result.findings) {
renderBounded(lines, result.findings, (item) => {
const symbol = item.changedSymbols.length ? ` (${item.changedSymbols.join(", ")})` : "";
lines.push(
`- \`${item.changedModule}\`${symbol} -> \`${item.consumer}\` -> \`${item.candidateTest}\``
);
}
return `- \`${item.changedModule}\`${symbol} -> \`${item.consumer}\` -> \`${item.candidateTest}\``;
});
lines.push("", "> Report-only calibration: these findings do not fail the job.", "");
}
for (const [heading, items] of [
@@ -232,10 +249,12 @@ function markdown(result, base) {
]) {
if (!items.length) continue;
lines.push(`### ${heading}`, "");
for (const item of items)
lines.push(
renderBounded(
lines,
items,
(item) =>
`- \`${item.changedModule}\` -> \`${item.consumer}\`${item.candidateTest ? ` -> \`${item.candidateTest}\`` : ""}: ${item.reason || item.message}`
);
);
lines.push("");
}
return `${lines.join("\n")}\n`;
@@ -271,9 +290,26 @@ function main() {
});
const report = markdown(result, base);
process.stdout.write(report);
// The JSON artifact is bounded for the same reason the markdown is: a hub-module diff
// produces millions of rows and `JSON.stringify` would throw the same "Invalid string
// length". `totals` keeps every count exact, so tooling can still see the real numbers.
const jsonResult = {
...result,
totals: {
findings: result.findings.length,
diagnostics: result.diagnostics.length,
suppressed: result.suppressed.length,
maskingRisks: result.maskingRisks.length,
},
itemLimit: JSON_ITEM_LIMIT,
findings: result.findings.slice(0, JSON_ITEM_LIMIT),
diagnostics: result.diagnostics.slice(0, JSON_ITEM_LIMIT),
suppressed: result.suppressed.slice(0, JSON_ITEM_LIMIT),
maskingRisks: result.maskingRisks.slice(0, JSON_ITEM_LIMIT),
};
for (const [target, contents] of [
[summaryPath, report],
[jsonPath, `${JSON.stringify(result, null, 2)}\n`],
[jsonPath, `${JSON.stringify(jsonResult, null, 2)}\n`],
]) {
if (!target) continue;
fs.mkdirSync(path.dirname(target), { recursive: true });