Files
OmniRoute/src/lib/db/responsesContinuationStore.ts
Markus Hartung 0f402a84a4 feat(responses): virtualize previous_response_id continuation regardless of upstream support (#10262)
* feat(responses): virtualize previous_response_id continuation regardless of upstream support

OmniRoute now exposes OpenAI-compatible previous_response_id/store
continuation to clients unconditionally, even when the selected upstream
provider has no native Responses-API state support. Reconstruction happens
server-side in handleChatImplementation, before any downstream validation
or provider translation: OmniRoute resolves the response id back to the
full input/output it previously produced, prepends it to the client's
delta, and forwards the full reconstructed history upstream exactly as it
does today. Client<->OmniRoute traffic shrinks to the new delta only;
OmniRoute<->provider traffic is unchanged.

Storage reuses the existing call-log pipeline artifact (already gated by
call_log_pipeline_enabled, already retained/cleaned up by the existing
call-log lifecycle) instead of duplicating conversation content into a
second store -- only a lightweight call_logs.response_id index is new.
Every lookup is scoped by api_key_id so one client can never resolve
another client's stored conversation, and any unresolvable/missing/
size-limit-omitted state fails closed with OpenAI's own
previous_response_not_found contract.

Stacked on feat/openai-responses-store-toggle (#10121).

* fix(db): re-export responsesContinuationStore from the localDb barrel

check-db-rules requires every db/ module to be re-exported (or explicitly
allowlisted as intentionally-internal) for discoverability. Missed this
when the module was first added.

* fix(db): renumber previous_response_id index migration to 154

The migration was numbered 153, but release/v3.8.50 already carries
153_radar_local_model_state.sql. The emngrating runner's collision guard
throws on two live .sql files sharing a numeric prefix, so the refreshed
merge would fail DB startup. Renumber to the next free slot (154).

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>

* docs(db): sync migration count to 149 across llm.txt mirrors

The responses-continuation store adds one migration, so the docs'
migration count is now 149 (was 148). Update README/AGENTS/llm.txt and
regenerate the i18n llm.txt mirrors to keep check:docs-all green.

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>

* fix(responses-continuation): respect preserve mode, drop dead export

- Un-export ResponsesContinuationState: it's never imported outside
  responsesContinuationStore.ts, its own defining file. Fixes the
  check:dead-code regression (410 > baseline 409).
- Scope the previous_response_id virtualization interception in chat.ts to
  skip entirely when responsesPreviousResponseIdMode=preserve. The
  interception ran unconditionally before target/connection selection,
  ahead of applyResponsesPreviousResponseIdPolicy (chatCore.ts) -- the
  existing per-target enforcement point for this setting -- so "preserve"
  (the explicit, connection-independent contract for "let the upstream
  resolve previous_response_id natively") was silently unreachable: the
  field was already deleted and replaced with locally-reconstructed input
  by the time that policy ran. This also broke Codex's own executor, which
  relies on an untouched previous_response_id to delegate history
  resolution upstream (see stripOrphanedCodexFunctionCallOutputs in
  codex.ts). "auto" and "strip" modes are unaffected -- virtualization is
  a strict improvement over their old "drop the field, hope the client
  resent everything" behavior.
- Add a regression test exercising the actual chat.ts handler (not just
  the policy helper in isolation): confirms mode=preserve now proceeds to
  normal routing instead of the virtualization's previous_response_not_found
  rejection, and that default/auto mode's existing virtualization behavior
  is unchanged. Verified the test fails for the right reason against
  pre-fix chat.ts.

Addresses PR review feedback.

---------

Co-authored-by: adevwithpurpose <adevwithpurpose@users.noreply.github.com>
Co-authored-by: hartmark <hartmark@users.noreply.github.com>
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
2026-08-17 08:22:17 -03:00

76 lines
3.3 KiB
TypeScript

/**
* responsesContinuationStore.ts — OmniRoute-native `previous_response_id`
* virtualization for the OpenAI Responses API.
*
* Exposes `previous_response_id` continuation to clients unconditionally,
* regardless of whether the actual upstream provider for a connection
* supports Responses-API state at all: OmniRoute resolves the response id
* back to the full input/output it produced and reconstructs the full
* request server-side before forwarding upstream (full history, exactly as
* today) -- the client only ever has to resend the new delta.
*
* Storage: reuses the existing call-log pipeline artifact (full, untruncated
* request/response payloads, already gated by `call_log_pipeline_enabled`
* and already retained/cleaned up by the existing call-log lifecycle)
* instead of duplicating conversation content into a second store. Only a
* lightweight `call_logs.response_id` index (154_call_logs_response_id.sql)
* is new. Every lookup is scoped by `api_key_id` -- one client can never
* resolve another client's stored conversation.
*/
import { getDbInstance } from "./core";
import { readCallArtifact } from "../usage/callLogArtifacts";
type ResponsesContinuationState = {
input: unknown[];
output: unknown[];
};
function isPlainRecord(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null && !Array.isArray(value);
}
/**
* Resolve the full input + output a prior Responses API call produced, so
* the caller can reconstruct `full_input = stored.input + stored.output +
* new_delta`. Returns null on any lookup/read/shape failure (unknown id,
* wrong tenant, artifact missing, or an artifact whose pipeline payload was
* size-limit-omitted -- see MAX_CALL_LOG_ARTIFACT_BYTES in
* callLogArtifacts.ts) so the caller can fail closed and ask the client to
* resend full history, exactly like a real `previous_response_not_found`
* from OpenAI itself.
*/
export function resolvePreviousResponseState(
responseId: string,
apiKeyId: string | null | undefined
): ResponsesContinuationState | null {
if (!responseId) return null;
const db = getDbInstance();
const row = db
.prepare(
`SELECT artifact_relpath, api_key_id FROM call_logs
WHERE response_id = ? AND detail_state = 'ready'
ORDER BY timestamp DESC LIMIT 1`
)
.get(responseId) as { artifact_relpath: string | null; api_key_id: string | null } | undefined;
if (!row || !row.artifact_relpath) return null;
// Tenant isolation: a response id is only ever handed back to the API key
// that created it. A stored row with no api_key_id at all (no-log/legacy)
// can never be resolved by any key -- fail closed rather than guess.
if (!apiKeyId || row.api_key_id !== apiKeyId) return null;
const { artifact, state } = readCallArtifact(row.artifact_relpath);
if (state !== "ready" || !artifact?.pipeline) return null;
const providerRequest = artifact.pipeline.providerRequest as { body?: unknown } | undefined;
const clientResponse = artifact.pipeline.clientResponse as { output?: unknown } | undefined;
const input = isPlainRecord(providerRequest?.body) ? providerRequest.body.input : undefined;
const output = clientResponse?.output;
if (!Array.isArray(input) || !Array.isArray(output)) return null;
return { input, output };
}