Files
OmniRoute/open-sse/handlers/chatCore/attemptLogging.ts
Markus Hartung 0f402a84a4 feat(responses): virtualize previous_response_id continuation regardless of upstream support (#10262)
* feat(responses): virtualize previous_response_id continuation regardless of upstream support

OmniRoute now exposes OpenAI-compatible previous_response_id/store
continuation to clients unconditionally, even when the selected upstream
provider has no native Responses-API state support. Reconstruction happens
server-side in handleChatImplementation, before any downstream validation
or provider translation: OmniRoute resolves the response id back to the
full input/output it previously produced, prepends it to the client's
delta, and forwards the full reconstructed history upstream exactly as it
does today. Client<->OmniRoute traffic shrinks to the new delta only;
OmniRoute<->provider traffic is unchanged.

Storage reuses the existing call-log pipeline artifact (already gated by
call_log_pipeline_enabled, already retained/cleaned up by the existing
call-log lifecycle) instead of duplicating conversation content into a
second store -- only a lightweight call_logs.response_id index is new.
Every lookup is scoped by api_key_id so one client can never resolve
another client's stored conversation, and any unresolvable/missing/
size-limit-omitted state fails closed with OpenAI's own
previous_response_not_found contract.

Stacked on feat/openai-responses-store-toggle (#10121).

* fix(db): re-export responsesContinuationStore from the localDb barrel

check-db-rules requires every db/ module to be re-exported (or explicitly
allowlisted as intentionally-internal) for discoverability. Missed this
when the module was first added.

* fix(db): renumber previous_response_id index migration to 154

The migration was numbered 153, but release/v3.8.50 already carries
153_radar_local_model_state.sql. The emngrating runner's collision guard
throws on two live .sql files sharing a numeric prefix, so the refreshed
merge would fail DB startup. Renumber to the next free slot (154).

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>

* docs(db): sync migration count to 149 across llm.txt mirrors

The responses-continuation store adds one migration, so the docs'
migration count is now 149 (was 148). Update README/AGENTS/llm.txt and
regenerate the i18n llm.txt mirrors to keep check:docs-all green.

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>

* fix(responses-continuation): respect preserve mode, drop dead export

- Un-export ResponsesContinuationState: it's never imported outside
  responsesContinuationStore.ts, its own defining file. Fixes the
  check:dead-code regression (410 > baseline 409).
- Scope the previous_response_id virtualization interception in chat.ts to
  skip entirely when responsesPreviousResponseIdMode=preserve. The
  interception ran unconditionally before target/connection selection,
  ahead of applyResponsesPreviousResponseIdPolicy (chatCore.ts) -- the
  existing per-target enforcement point for this setting -- so "preserve"
  (the explicit, connection-independent contract for "let the upstream
  resolve previous_response_id natively") was silently unreachable: the
  field was already deleted and replaced with locally-reconstructed input
  by the time that policy ran. This also broke Codex's own executor, which
  relies on an untouched previous_response_id to delegate history
  resolution upstream (see stripOrphanedCodexFunctionCallOutputs in
  codex.ts). "auto" and "strip" modes are unaffected -- virtualization is
  a strict improvement over their old "drop the field, hope the client
  resent everything" behavior.
- Add a regression test exercising the actual chat.ts handler (not just
  the policy helper in isolation): confirms mode=preserve now proceeds to
  normal routing instead of the virtualization's previous_response_not_found
  rejection, and that default/auto mode's existing virtualization behavior
  is unchanged. Verified the test fails for the right reason against
  pre-fix chat.ts.

Addresses PR review feedback.

---------

Co-authored-by: adevwithpurpose <adevwithpurpose@users.noreply.github.com>
Co-authored-by: hartmark <hartmark@users.noreply.github.com>
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
2026-08-17 08:22:17 -03:00

320 lines
11 KiB
TypeScript

/**
* chatCore per-attempt logging persistence (Quality Gate v2 / Fase 9 — chatCore god-file
* decomposition, #3501).
*
* Extracted from handleChatCore: persists one attempt's call log. Emits a provider.warning audit
* event when the provider response carries warnings, fills the detailed pipeline payloads (when
* detailed logging is on), and writes the bounded/truncated call-log row (request/response bodies
* with the Claude prompt-cache meta attached). Best-effort: the saveCallLog write swallows its own
* errors. The per-request context (provider/model/ids/combo/etc.) is threaded via `ctx` so the 16
* call sites in the handler stay byte-identical; behaviour is unchanged.
*/
import { extractProviderWarnings } from "@/lib/compliance/providerAudit";
import { logAuditEvent } from "@/lib/compliance";
import { emit } from "@/lib/events/eventBus";
import type { RequestCompletedPayload, RequestFailedPayload } from "@/lib/events/types";
import { saveCallLog } from "@/lib/usageDb";
import { FORMATS } from "../../translator/formats.ts";
import { cloneBoundedChatLogPayload, truncateForLog } from "./logTruncation.ts";
import { attachLogMeta } from "./cacheUsageMeta.ts";
/**
* Extract the OpenAI Responses API response id this attempt produced, so it
* can be indexed for OmniRoute-native `previous_response_id` continuation
* (see src/lib/db/responsesContinuationStore.ts). Only meaningful when the
* client actually used the Responses endpoint -- a Chat Completions
* `chatcmpl-*` id must never be mistaken for a Responses response id.
*/
function extractResponsesId(sourceFormat: unknown, clientResponse: unknown): string | null {
if (sourceFormat !== FORMATS.OPENAI_RESPONSES) return null;
if (!clientResponse || typeof clientResponse !== "object") return null;
const id = (clientResponse as { id?: unknown }).id;
return typeof id === "string" && id.length > 0 ? id : null;
}
export type PersistAttemptLogsArgs = {
status: number;
tokens?: unknown;
responseBody?: unknown;
error?: string | null;
providerRequest?: unknown;
providerResponse?: unknown;
clientResponse?: unknown;
claudeCacheMeta?: Record<string, unknown>;
claudeCacheUsageMeta?: Record<string, unknown>;
cacheSource?: "upstream" | "semantic";
};
export type PersistAttemptLogsContext = {
/** Per-attempt trace id — MUST match the id emitted in `request.started` so the live
* dashboard can pair the terminal event and clear the topology node's active pulse. */
traceId: string;
provider: string | null | undefined;
connectionId: string | null | undefined;
model: string | null | undefined;
skillRequestId: string;
detailedLoggingEnabled: boolean;
reqLogger: { getPipelinePayloads?: () => Record<string, unknown> | undefined } | null | undefined;
pendingRequestId: unknown;
clientRawRequest: { endpoint?: string } | null | undefined;
requestedModel: unknown;
credentials: { connectionId?: string } | null | undefined;
startTime: number;
body: unknown;
sourceFormat: unknown;
targetFormat: unknown;
comboName: unknown;
comboStepId: unknown;
comboExecutionKey: unknown;
tokensCompressed: unknown;
apiKeyInfo: { id?: string | null; name?: string | null } | null | undefined;
noLogEnabled: unknown;
correlationId?: string | null;
modelPinned?: boolean;
/** #8249: caller-supplied X-OmniRoute-Session-Id header, only set when the header was
* explicitly present (never synthesized from skillRequestId) — persisted as call_logs.session_tag
* for per-session cost attribution. */
sessionTag?: string | null;
};
function toConnectionId(value: unknown): string | null {
return typeof value === "string" && value.trim().length > 0 ? value.trim() : null;
}
function buildAccountRotationMeta(
provider: string | null | undefined,
initialConnectionId: string | null,
finalConnectionId: string | null
) {
if (provider !== "codex" || !initialConnectionId || !finalConnectionId) return null;
if (initialConnectionId === finalConnectionId) return null;
return {
codexAccountRotation: {
initialConnectionId,
finalConnectionId,
},
};
}
/**
* Pure resolver for the terminal request-lifecycle dashboard event. Extracted so the
* "stuck green" latch fix (emitting request.completed/failed to clear the live topology
* node) is unit-testable without the DB write in persistAttemptLogs. A 2xx/3xx status
* with no error is a completion; everything else (including a missing/odd status) is a
* failure. `id` mirrors the `traceId` used by the paired `request.started`.
*/
export function resolveRequestLifecycleEvent(input: {
traceId: string;
status: number;
error?: string | null;
model?: string | null;
provider?: string | null;
comboName?: unknown;
tokens?: unknown;
latencyMs: number;
}):
| { name: "request.completed"; payload: RequestCompletedPayload }
| { name: "request.failed"; payload: RequestFailedPayload } {
const { traceId, status, error, model, provider, comboName, tokens, latencyMs } = input;
const succeeded = typeof status === "number" && status >= 200 && status < 400 && !error;
const resolvedComboName = typeof comboName === "string" && comboName ? comboName : undefined;
if (succeeded) {
const tokenBag = (tokens && typeof tokens === "object" ? tokens : {}) as Record<
string,
unknown
>;
const num = (v: unknown) => (typeof v === "number" && Number.isFinite(v) ? v : 0);
return {
name: "request.completed",
payload: {
id: traceId,
status: "success",
model: model || "unknown",
provider: provider || "unknown",
tokensInput: num(tokenBag.input ?? tokenBag.prompt_tokens ?? tokenBag.inputTokens),
tokensOutput: num(tokenBag.output ?? tokenBag.completion_tokens ?? tokenBag.outputTokens),
latencyMs,
comboName: resolvedComboName,
},
};
}
return {
name: "request.failed",
payload: {
id: traceId,
error: error || `HTTP ${status}`,
statusCode: typeof status === "number" ? status : undefined,
latencyMs,
model: model || undefined,
provider: provider || undefined,
},
};
}
export function persistAttemptLogs(args: PersistAttemptLogsArgs, ctx: PersistAttemptLogsContext) {
const {
status,
tokens,
responseBody,
error,
providerRequest,
providerResponse,
clientResponse,
claudeCacheMeta,
claudeCacheUsageMeta,
cacheSource,
} = args;
const {
traceId,
provider,
connectionId,
model,
skillRequestId,
detailedLoggingEnabled,
reqLogger,
pendingRequestId,
clientRawRequest,
requestedModel,
credentials,
startTime,
body,
sourceFormat,
targetFormat,
comboName,
comboStepId,
comboExecutionKey,
tokensCompressed,
apiKeyInfo,
noLogEnabled,
correlationId,
modelPinned,
sessionTag,
} = ctx;
const initialConnectionId = toConnectionId(connectionId);
const finalConnectionId = toConnectionId(credentials?.connectionId) || initialConnectionId;
const accountRotationMeta = buildAccountRotationMeta(
provider,
initialConnectionId,
finalConnectionId
);
const providerWarnings = extractProviderWarnings(providerResponse, clientResponse, responseBody);
if (providerWarnings.length > 0) {
logAuditEvent({
action: "provider.warning",
actor: "system",
target: [provider, finalConnectionId].filter(Boolean).join(":") || provider || model,
resourceType: "provider_warning",
status: "warning",
requestId: skillRequestId,
details: {
provider,
model,
connectionId: finalConnectionId,
httpStatus: status,
warnings: providerWarnings,
},
});
}
const capturedPipeline = reqLogger?.getPipelinePayloads?.() ?? null;
const pipelinePayloads = detailedLoggingEnabled
? (capturedPipeline ?? {})
: capturedPipeline?.routeDecision
? { routeDecision: capturedPipeline.routeDecision }
: null;
if (pipelinePayloads) {
if (providerRequest !== undefined && !pipelinePayloads.providerRequest) {
pipelinePayloads.providerRequest = providerRequest as Record<string, unknown>;
}
if (providerResponse !== undefined && !pipelinePayloads.providerResponse) {
pipelinePayloads.providerResponse = providerResponse as Record<string, unknown>;
}
if (clientResponse !== undefined) {
pipelinePayloads.clientResponse = clientResponse as Record<string, unknown>;
}
if (error) {
pipelinePayloads.error = {
...(typeof pipelinePayloads.error === "object" && pipelinePayloads.error
? (pipelinePayloads.error as Record<string, unknown>)
: {}),
message: error,
};
}
}
saveCallLog({
id: pendingRequestId,
method: "POST",
path: clientRawRequest?.endpoint || "/v1/chat/completions",
status,
model,
requestedModel,
provider,
connectionId: finalConnectionId || undefined,
duration: Date.now() - startTime,
tokens: tokens || {},
requestBody: cloneBoundedChatLogPayload(
attachLogMeta(truncateForLog(body as Record<string, unknown>), {
...accountRotationMeta,
claudePromptCache: claudeCacheMeta,
})
),
responseBody: cloneBoundedChatLogPayload(
attachLogMeta(truncateForLog(responseBody as Record<string, unknown>), {
...accountRotationMeta,
claudePromptCache: claudeCacheMeta
? {
applied: claudeCacheMeta.applied,
totalBreakpoints: claudeCacheMeta.totalBreakpoints,
anthropicBeta: claudeCacheMeta.anthropicBeta,
}
: null,
claudePromptCacheUsage: claudeCacheUsageMeta,
})
),
error: error || null,
sourceFormat,
targetFormat,
comboName,
comboStepId,
comboExecutionKey,
tokensCompressed,
cacheSource: cacheSource === "semantic" ? "semantic" : "upstream",
apiKeyId: apiKeyInfo?.id || null,
apiKeyName: apiKeyInfo?.name || null,
noLog: noLogEnabled,
pipelinePayloads,
correlationId,
modelPinned: modelPinned || false,
sessionTag: sessionTag || null,
responseId: extractResponsesId(sourceFormat, clientResponse),
}).catch(() => {});
// Emit the terminal request-lifecycle event to the live dashboard bus. `request.started`
// is emitted in chatCore with this same `traceId`; without a matching completed/failed the
// client's active-request map never drains, so the topology node stays green forever (the
// "stuck green" latch — request.completed/failed were declared + consumed but never emitted).
// Deferred via setImmediate to keep it off the response hot path, mirroring request.started.
setImmediate(() => {
const lifecycle = resolveRequestLifecycleEvent({
traceId,
status,
error,
model,
provider,
comboName,
tokens,
latencyMs: Date.now() - startTime,
});
if (lifecycle.name === "request.completed") {
emit("request.completed", lifecycle.payload);
} else {
emit("request.failed", lifecycle.payload);
}
});
}