Compare commits

...

3 Commits

Author SHA1 Message Date
Bob.Hou
02c663cdd0 ci/main-green: parse ci.yml with the declared js-yaml package (#13955)
validate-release-green imported the yaml package, which is not a
direct dependency on main. The nightly job then crashed before the
verdict fence, so the tracker comment was empty. js-yaml is already
declared and load() is the same parse for this workflow YAML.

Signed-off-by: Minxi Hou <houminxi@gmail.com>
2026-09-17 17:06:54 -03:00
JasonBroderick
a3ca33fa64 fix(sse): hoist leading text system messages on the Claude mid-conversation-system passthrough (#13072)
On the Claude Code semantic passthrough with a 1M-context model (system + tools
present), system-role messages are deliberately kept inside messages[] and only
directive-only messages (content: [] + output_config) are relocated off
messages[0]. Anthropic also rejects a text-bearing system message at messages[0]:

    messages.0: use the top-level 'system' parameter for the initial system
    prompt; the directive-only form (content: [] with output_config) is
    accepted at any position

That is exactly where the Output Styles injection lands
(open-sse/services/compression/outputStyles/apply.ts unshifts a system message),
so every Claude Code turn with an output style active on such a model fails
with 400.

Add hoistLeadingTextSystemMessages(): move only the leading run of text-bearing
system-role messages (everything before the first user/assistant turn) into the
top-level system parameter, and call it before relocateDirectiveOnlyMessages()
on that path. Genuine mid-conversation system turns keep their position and
cache prefix; directive-only messages in the run are left for the existing
relocation. Unit tests cover the injected-style case, the string top-level
system case, mixed directive/text runs, and the no-op case.

Repro: POST /v1/messages with Claude Code client headers (user-agent
claude-cli/..., x-app: cli), model claude-opus-5, a top-level system, one tool,
and messages[0] = {role: "system", content: "[OmniRoute Output Styles] ..."}.
Before: 400 from Anthropic. After: 200.

Co-authored-by: ai-stack <ops@ai-stack.local>
2026-09-09 20:59:05 -03:00
Diego Rodrigues de Sa e Souza
c0b2253f21 fix(ci): port the release-green ESLint gate fix to main (base-red #12363) (#12618)
Porta para `main` o fix do gate de ESLint que só havia entrado na branch de release — o padrão de PR-companheiro que `_shared/merge-gates.md` §8 prescreve.

As 12 falhas de CI foram discriminadas como o **outro** base-red do main, não deste diff. Todas descendem de um único ponto: `Package Artifact` falha e os 9 shards de E2E mais os 2 Electron Package Smoke consomem esse artefato. A própria issue #12363 lista os dois separadamente:

- ` ESLint: could not parse eslint json` — que é justamente o que este PR conserta;
- ` Package artifact (npm pack policy): gate exceeded its 1200s ceiling` — a raiz da cascata.

O PR toca apenas `scripts/quality/validate-release-green.mjs` e seu teste, então não tem caminho para afetar o build do pacote. Teste portado primeiro e falhando no script atual do main (TDD).
2026-09-03 20:50:04 -03:00
6 changed files with 236 additions and 37 deletions

View File

@@ -0,0 +1 @@
- **fix(ci):** `validate-release-green` parses `ci.yml` with the already-declared `js-yaml` dependency instead of the undeclared `yaml` package, so a main-green nightly crash no longer posts an empty verdict on the tracker.

View File

@@ -9,10 +9,12 @@ import { buildFailureUsageRecord } from "./chatCore/failureUsage.ts";
import { estimateFinalInputTokens } from "./chatCore/contextEstimation.ts"; import { estimateFinalInputTokens } from "./chatCore/contextEstimation.ts";
import { import {
extractSystemRoleMessages, extractSystemRoleMessages,
hoistLeadingTextSystemMessages,
relocateDirectiveOnlyMessages, relocateDirectiveOnlyMessages,
} from "./chatCore/claudeSystemRole.ts"; } from "./chatCore/claudeSystemRole.ts";
export { export {
extractSystemRoleMessages, extractSystemRoleMessages,
hoistLeadingTextSystemMessages,
relocateDirectiveOnlyMessages, relocateDirectiveOnlyMessages,
} from "./chatCore/claudeSystemRole.ts"; } from "./chatCore/claudeSystemRole.ts";
import { checkIdempotencyCache } from "./chatCore/idempotency.ts"; import { checkIdempotencyCache } from "./chatCore/idempotency.ts";
@@ -2312,6 +2314,10 @@ export async function handleChatCore({
// messages[], but a directive-only message (content: [] + // messages[], but a directive-only message (content: [] +
// output_config) at messages[0] is rejected by Anthropic. Move it past // output_config) at messages[0] is rejected by Anthropic. Move it past
// the first real turn; Anthropic accepts the form at any other position. // the first real turn; Anthropic accepts the form at any other position.
// A text-bearing system message at messages[0] (e.g. the Output Styles
// injection) is rejected there too: hoist the leading run into the
// top-level `system` parameter first.
hoistLeadingTextSystemMessages(translatedBody);
relocateDirectiveOnlyMessages(translatedBody); relocateDirectiveOnlyMessages(translatedBody);
} }
if (Array.isArray(translatedBody.messages)) { if (Array.isArray(translatedBody.messages)) {

View File

@@ -164,6 +164,61 @@ export function extractSystemRoleMessages(payload: Record<string, unknown>): voi
payload.messages = messages.filter((m) => !isSystemRole(m.role)); payload.messages = messages.filter((m) => !isSystemRole(m.role));
} }
/**
* Hoists the leading run of text-bearing system-role messages (everything
* before the first real user/assistant turn) into the top-level `system`
* parameter. Anthropic treats `messages[0]` as the initial system prompt
* position and rejects any non-directive system-role message there ("use the
* top-level 'system' parameter for the initial system prompt"), which is
* exactly where the Output Styles injection lands on the mid-conversation
* system passthrough (provider `claude` + 1M-context models). Only the leading
* run is hoisted so genuine mid-conversation system turns keep their position
* and cache prefix; empty (directive-only) messages in the run are left in
* place for relocateDirectiveOnlyMessages to handle.
*/
export function hoistLeadingTextSystemMessages(payload: Record<string, unknown>): void {
if (!Array.isArray(payload.messages) || payload.messages.length === 0) return;
const messages = payload.messages as Array<Record<string, unknown>>;
const isSystemRole = (role: unknown): boolean =>
typeof role === "string" &&
(role.toLowerCase() === "system" || role.toLowerCase() === "developer");
const blocks: Array<Record<string, unknown>> = [];
const kept: Array<Record<string, unknown>> = [];
let i = 0;
for (; i < messages.length; i++) {
const m = messages[i];
if (m == null || typeof m !== "object" || !isSystemRole(m.role)) break;
if (typeof m.content === "string") {
if (m.content.length > 0) blocks.push({ type: "text", text: m.content });
continue;
}
if (Array.isArray(m.content) && m.content.length > 0) {
let hoisted = false;
for (const block of m.content as Array<Record<string, unknown>>) {
if (block?.type === "text" && typeof block.text === "string" && block.text.length > 0) {
blocks.push({ type: "text", text: block.text });
hoisted = true;
}
}
if (!hoisted) kept.push(m);
continue;
}
kept.push(m);
}
if (blocks.length === 0) return;
const existing = payload.system;
if (typeof existing === "string" && existing.length > 0) {
payload.system = [{ type: "text", text: existing }, ...blocks];
} else if (Array.isArray(existing)) {
payload.system = [...(existing as Array<Record<string, unknown>>), ...blocks];
} else {
payload.system = blocks;
}
payload.messages = [...kept, ...messages.slice(i)];
}
/** /**
* Moves a directive-only system message (empty content array + message-level * Moves a directive-only system message (empty content array + message-level
* `output_config`, the shape Claude Code clients emit) off `messages[0]`. * `output_config`, the shape Claude Code clients emit) off `messages[0]`.

View File

@@ -55,11 +55,12 @@ import { promisify } from "node:util";
import { mkdirSync, readFileSync, writeFileSync } from "node:fs"; import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
import { dirname, join } from "node:path"; import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url"; import { fileURLToPath } from "node:url";
import { parse as parseYaml } from "yaml"; import { load as parseYaml } from "js-yaml";
const __dirname = dirname(fileURLToPath(import.meta.url)); const __dirname = dirname(fileURLToPath(import.meta.url));
const ROOT = join(__dirname, "..", ".."); const ROOT = join(__dirname, "..", "..");
const npmCmd = process.platform === "win32" ? "npm.cmd" : "npm"; const npmCmd = process.platform === "win32" ? "npm.cmd" : "npm";
export const ESLINT_TIMEOUT_MS = 60 * 60 * 1000;
// Per-gate captured output. execFileSync buffers everything and the report only // Per-gate captured output. execFileSync buffers everything and the report only
// shows a one-line summary, so without these files every red requires RE-RUNNING // shows a one-line summary, so without these files every red requires RE-RUNNING
@@ -179,6 +180,56 @@ export function parseEslintJson(out) {
return null; return null;
} }
/**
* Turn one ESLint process result into release-green records.
*
* Keep process failures distinct from report parsing failures. In particular, a timed-out
* ESLint process has no JSON report by definition; collapsing its code-124 diagnostic into
* "could not parse eslint json" hides the actionable cause and sends maintainers debugging
* the parser instead of the gate ceiling.
*/
export function evaluateEslintRun({ code, out }, warningBaseline) {
const parsed = parseEslintJson(out);
if (!parsed) {
return [
{
id: "lint",
label: "ESLint",
kind: "hard",
ok: false,
detail:
code === 0
? "ESLint exited successfully but produced no valid JSON report"
: firstFailureLine(out),
},
];
}
const { errors, warnings } = eslintCounts(parsed);
const warningDrift = isDrift(warnings, warningBaseline);
return [
{
id: "lint-errors",
label: "ESLint errors",
kind: "hard",
ok: errors === 0,
detail: `${errors} error(s)`,
},
{
id: "eslint-warnings",
label: "ESLint warnings (ratchet)",
kind: "drift",
ok: !warningDrift,
detail:
warningBaseline == null
? `${warnings} (no baseline)`
: `${warnings} vs baseline ${warningBaseline}${
warningDrift ? ` (+${warnings - warningBaseline} drift → rebaseline at release)` : ""
}`,
},
];
}
/** Pull the cognitive-complexity violation count from the gate's output. */ /** Pull the cognitive-complexity violation count from the gate's output. */
export function parseCognitiveCount(out) { export function parseCognitiveCount(out) {
const s = String(out || ""); const s = String(out || "");
@@ -451,16 +502,19 @@ async function main() {
// ESLint: ONE pass → errors (hard) + warnings (drift) // ESLint: ONE pass → errors (hard) + warnings (drift)
{ {
announce("ESLint (errors + warnings — ~5-15min)"); announce("ESLint (errors + warnings — ~15-45min)");
// Suppressions-aware, matching `npm run lint` (Pacote 4 no-new-warnings): the frozen // Suppressions-aware, matching `npm run lint` (Pacote 4 no-new-warnings): the frozen
// pre-existing debt in config/quality/eslint-suppressions.json must not count as // pre-existing debt in config/quality/eslint-suppressions.json must not count as
// errors here — only NET-NEW violations are release reds. Timeout raised: a full // errors here — only NET-NEW violations are release reds. The cold release runner can
// repo pass takes ~14min alone and this pre-flight often runs alongside test suites. // exceed 30 minutes as the repository grows, and this pre-flight often runs under load.
const { out } = run( const lintRun = run(
"npx", "npx",
[ [
"eslint", "eslint",
".", ".",
"--cache",
"--cache-location",
".eslintcache",
"--format", "--format",
"json", "json",
"--suppressions-location", "--suppressions-location",
@@ -471,39 +525,16 @@ async function main() {
// reason alone, which used to mask the real `--format json` report (#7837). // reason alone, which used to mask the real `--format json` report (#7837).
"--pass-on-unpruned-suppressions", "--pass-on-unpruned-suppressions",
], ],
{ timeout: 30 * 60 * 1000 } // The cold release runner crossed the old 30-minute ceiling as the repository grew,
// then the timeout text was misreported as invalid JSON. Keep a real upper bound, but
// leave enough headroom for the same full-tree walk that completes immediately after it
// under the complexity config on that runner.
{ timeout: ESLINT_TIMEOUT_MS }
); );
const { out } = lintRun;
saveGateLog("lint", out); saveGateLog("lint", out);
const parsed = parseEslintJson(out); for (const result of evaluateEslintRun(lintRun, baselineValue("eslintWarnings"))) {
if (!parsed) { record(result);
record({
id: "lint",
label: "ESLint",
kind: "hard",
ok: false,
detail: "could not parse eslint json",
});
} else {
const { errors, warnings } = eslintCounts(parsed);
record({
id: "lint-errors",
label: "ESLint errors",
kind: "hard",
ok: errors === 0,
detail: `${errors} error(s)`,
});
const base = baselineValue("eslintWarnings");
const over = isDrift(warnings, base);
record({
id: "eslint-warnings",
label: "ESLint warnings (ratchet)",
kind: "drift",
ok: !over,
detail:
base == null
? `${warnings} (no baseline)`
: `${warnings} vs baseline ${base}${over ? ` (+${warnings - base} drift → rebaseline at release)` : ""}`,
});
} }
} }
@@ -638,7 +669,8 @@ async function main() {
// forever) into a visible failure — survives at 100min. // forever) into a visible failure — survives at 100min.
// Measured on idle .113: unavailable (checkout not found). Tightened to 80min from 100min as a conservative step. TODO: re-measure on idle .113 and tighten to ~1.8× measured. // Measured on idle .113: unavailable (checkout not found). Tightened to 80min from 100min as a conservative step. TODO: re-measure on idle .113 and tighten to ~1.8× measured.
id: "unit", id: "unit",
label: "Unit tests (full suite, CI concurrency — ~30-50min idle, up to ~80min under load (awaiting idle .113 measurement, #9532))", label:
"Unit tests (full suite, CI concurrency — ~30-50min idle, up to ~80min under load (awaiting idle .113 measurement, #9532))",
args: ["run", "test:unit:ci"], args: ["run", "test:unit:ci"],
timeout: 80 * 60 * 1000, timeout: 80 * 60 * 1000,
}, },

View File

@@ -0,0 +1,77 @@
import assert from "node:assert/strict";
import { test } from "node:test";
import {
hoistLeadingTextSystemMessages,
relocateDirectiveOnlyMessages,
} from "../../open-sse/handlers/chatCore/claudeSystemRole.ts";
// Reproduction of the production 400 (2026-09-03/04): Output Styles injects a
// text system message at messages[0]; on the mid-conversation-system passthrough
// Anthropic rejects it ("use the top-level 'system' parameter for the initial
// system prompt").
test("hoistLeadingTextSystemMessages moves a text system messages[0] into top-level system", () => {
const payload: Record<string, unknown> = {
system: [{ type: "text", text: "You are Claude." }],
output_config: { effort: "medium" },
messages: [
{ role: "system", content: "[OmniRoute Output Styles]\nRespond terse." },
{ role: "user", content: "Reply exactly: MIDCONV_TOPLEVEL_OK" },
],
};
hoistLeadingTextSystemMessages(payload);
assert.deepEqual(payload.system, [
{ type: "text", text: "You are Claude." },
{ type: "text", text: "[OmniRoute Output Styles]\nRespond terse." },
]);
assert.equal((payload.messages as Array<{ role: string }>)[0].role, "user");
assert.equal((payload.messages as unknown[]).length, 1);
});
test("hoistLeadingTextSystemMessages converts a string top-level system and keeps mid-conversation system turns", () => {
const payload: Record<string, unknown> = {
system: "base",
messages: [
{ role: "system", content: [{ type: "text", text: "style" }] },
{ role: "user", content: "hello" },
{ role: "system", content: "mid-conversation context" },
{ role: "assistant", content: "hi" },
],
};
hoistLeadingTextSystemMessages(payload);
assert.deepEqual(payload.system, [
{ type: "text", text: "base" },
{ type: "text", text: "style" },
]);
const roles = (payload.messages as Array<{ role: string }>).map((m) => m.role);
assert.deepEqual(roles, ["user", "system", "assistant"]);
});
test("hoistLeadingTextSystemMessages leaves directive-only messages for relocateDirectiveOnlyMessages", () => {
const payload: Record<string, unknown> = {
messages: [
{ role: "system", content: [], output_config: { effort: "high" } },
{ role: "system", content: "style" },
{ role: "user", content: "hello" },
{ role: "assistant", content: "hi" },
],
};
hoistLeadingTextSystemMessages(payload);
assert.deepEqual(payload.system, [{ type: "text", text: "style" }]);
relocateDirectiveOnlyMessages(payload);
const msgs = payload.messages as Array<Record<string, unknown>>;
assert.equal(msgs[0].role, "user");
assert.equal(msgs[1].role, "system");
assert.deepEqual(msgs[1].output_config, { effort: "high" });
assert.equal(msgs[2].role, "assistant");
});
test("hoistLeadingTextSystemMessages is a no-op for a normal user first message", () => {
const payload: Record<string, unknown> = {
system: "base",
messages: [{ role: "user", content: "hello" }],
};
hoistLeadingTextSystemMessages(payload);
assert.equal(payload.system, "base");
assert.equal((payload.messages as unknown[]).length, 1);
});

View File

@@ -7,6 +7,7 @@ const mod = await import("../../scripts/quality/validate-release-green.mjs");
const { const {
firstFailureLine, firstFailureLine,
eslintCounts, eslintCounts,
evaluateEslintRun,
parseEslintJson, parseEslintJson,
parseCognitiveCount, parseCognitiveCount,
isDrift, isDrift,
@@ -17,6 +18,7 @@ const {
fullCiTimeoutFor, fullCiTimeoutFor,
curatedEquivalentId, curatedEquivalentId,
fullCiKindFor, fullCiKindFor,
ESLINT_TIMEOUT_MS,
} = mod; } = mod;
const extract = extractCiGates as ( const extract = extractCiGates as (
@@ -50,6 +52,22 @@ test("parseEslintJson tolerates ESLint's trailing unpruned-suppressions stderr s
]); ]);
}); });
test("evaluateEslintRun preserves an ESLint timeout instead of misreporting invalid JSON", () => {
const timedOut = classifyRunError({ killed: true, code: "ETIMEDOUT" }, 30 * 60 * 1000);
assert.deepEqual(evaluateEslintRun(timedOut, 0), [
{
id: "lint",
label: "ESLint",
kind: "hard",
ok: false,
detail:
"gate exceeded its 1800s ceiling and was killed — treat as a hung/failed gate (e.g. an unreleased DB handle in the unit suite); does NOT pass",
},
]);
assert.equal(ESLINT_TIMEOUT_MS, 60 * 60 * 1000, "cold release lint needs >30m headroom");
});
test("parseCognitiveCount reads the gate's count (en + pt)", () => { test("parseCognitiveCount reads the gate's count (en + pt)", () => {
assert.equal( assert.equal(
parseCognitiveCount("[cognitive-complexity] 797 function(s) exceed the threshold (15)."), parseCognitiveCount("[cognitive-complexity] 797 function(s) exceed the threshold (15)."),
@@ -364,6 +382,16 @@ test("extractCiGates: the REAL ci.yml yields the base-reds that leaked in v3.8.4
assert.ok(ids.size >= 20, "the real gate set is substantial (>= 20 static gates)"); assert.ok(ids.size >= 20, "the real gate set is substantial (>= 20 static gates)");
}); });
test("validate-release-green parses workflow YAML via the declared js-yaml dependency", async () => {
const fs = await import("node:fs");
const src = fs.readFileSync(
new URL("../../scripts/quality/validate-release-green.mjs", import.meta.url),
"utf8"
);
assert.match(src, /from ["']js-yaml["']/);
assert.doesNotMatch(src, /from ["']yaml["']/);
});
// ─── Verdict accuracy (review of the #9985 release-green verdict) ──────────── // ─── Verdict accuracy (review of the #9985 release-green verdict) ────────────
test("firstFailureLine never blames a PASSING line whose test FILE NAME contains 'fail' (#9985)", () => { test("firstFailureLine never blames a PASSING line whose test FILE NAME contains 'fail' (#9985)", () => {