mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-26 09:52:11 +03:00
50896699d3
* fix(sse): per-model 403 on passthrough providers locks the model, not the connection (#3027)
A per-model subscription 403 from a passthrough / per-model-quota provider
(e.g. ollama-cloud "this model requires a subscription, upgrade for access" on
deepseek-v4-pro) cooled down the ENTIRE connection instead of locking out only
the paid model, knocking out the free models on the same key and escalating an
exponential connection-wide backoff on repeats.
markAccountUnavailable's per-model lockout gate only covered 404/429/>=500, and
the only 403 -> model-lockout special case was hard-coded to grok-web. Generalize
it: a model-scoped 403 on an isPerModelQuotaProvider becomes a model lockout
(connection stays active). Terminal whole-key 403s (permanent/banned, account
deactivated, credits exhausted, project-route) keep their connection-level path.
Test-first: reproduction (paid model locked, connection active, free model still
eligible), regression guard (deactivated key still terminal -> banned, not
downgraded), and backoff guard (repeated 403s do not escalate connection backoff).
* feat(sse): persistent session + rolling-window memory for deepseek-web (#2942)
The DeepSeek web API takes only a single `prompt` string (no messages array) and
the executor created a fresh chat session per request, deleting it afterward — so
agentic multi-turn clients got per-turn amnesia.
Add two opt-in, per-connection settings (providerSpecificData), both defaulting to
the legacy behavior so plain-chat users are unaffected:
- historyWindow (number, default 0): when > 0, messagesToPrompt stitches the last N
non-system turns into a role-tagged transcript ("User:"/"Assistant:") so context
carries across turns within the single prompt string.
- persistSession (bool, default false): reuse one upstream chat session per userToken
(cached in the existing sessionCache) instead of creating/deleting one per request.
A reused session that fails is treated as stale (deleted in the DeepSeek UI): the
cache entry is dropped, a fresh session is created, and the completion is retried
once. Error paths invalidate the cached session so the next turn self-heals.
Test-first: pure prompt-builder window semantics (legacy/window/cap/empty) and
execute()-level session behavior via mocked fetch (fresh-per-request default, reuse,
stale-session fresh retry, history threaded into the prompt). Full existing
deepseek-web suite (35 tests) still green.
Note: chat.deepseek.com is an unofficial reverse-engineered surface; the live
round-trip can't be exercised in CI (no userToken/session). Treated as best-effort
per the issue; the prompt/session logic is unit-tested in isolation.
* feat(sse): tool-call translation for deepseek-web (#2820)
deepseek-web previously hard-failed any request carrying tools[] with a 400 (#2848),
so agentic clients could not use it for function calling at all. Add a bidirectional
translation layer (new open-sse/translator/webTools.ts, reusable by the other
web-cookie executors):
- Request: serializeToolsToPrompt() turns the OpenAI tools[] into a <tool>{...}</tool>
prompt contract, injected as a leading system message.
- Response: parseToolCallsFromText() extracts the model's <tool> blocks into OpenAI
tool_calls (arguments as a JSON string), strips them from content, and the executor
emits finish_reason "tool_calls" — for both non-stream and stream clients (the reply
is buffered since a tool block must be parsed whole).
A plain reply (no <tool> block) still streams normally with finish_reason "stop".
The superseded #2848 400-contract test is updated to assert the new translation.
Test-first: pure serializer/parser units + execute() round-trip via mocked fetch
(no-400, prompt serialization, non-stream tool_calls, stream tool_calls, plain reply).
Note: chat.deepseek.com is an unofficial reverse-engineered surface and the exact
<tool> emission depends on the model following the injected contract; the live
round-trip can't be exercised in CI. Best-effort per the issue; translation logic is
unit-tested in isolation.
* fix(sse): drop duplicate per-model 403 block — already in release via #3096
The release branch already scopes per-model 403 to model lockout (PR #3096,
commit 7042d562c) with the canonical !terminalStatus guard + exactCooldownMs
upstream hint. This PR's separate isTerminalOrRoute403 block shadowed it and
omitted the retry hint. Keep only the deepseek-web (#2942) + webTools (#2820)
changes here; the #3027 regression test is retained as coverage for #3096.
62 lines
2.7 KiB
TypeScript
62 lines
2.7 KiB
TypeScript
// #2942 — rolling-window prompt memory for deepseek-web. The web API takes only a single
|
|
// `prompt` string, so multi-turn context must be stitched into that prompt. With the
|
|
// window disabled (default) the legacy behavior (system + last user only) is preserved;
|
|
// with a window > 0, the last N turns are stitched into a role-tagged transcript.
|
|
import test from "node:test";
|
|
import assert from "node:assert/strict";
|
|
|
|
const { messagesToPrompt } = await import("../../open-sse/executors/deepseek-web.ts");
|
|
|
|
const CONVO = [
|
|
{ role: "system", content: "You are helpful." },
|
|
{ role: "user", content: "first question" },
|
|
{ role: "assistant", content: "first answer" },
|
|
{ role: "user", content: "second question" },
|
|
];
|
|
|
|
test("window 0 (default) keeps legacy behavior: system + last user only", () => {
|
|
const prompt = messagesToPrompt(CONVO, 0);
|
|
assert.ok(prompt.includes("You are helpful."), "system prompt present");
|
|
assert.ok(prompt.includes("second question"), "last user message present");
|
|
assert.ok(!prompt.includes("first question"), "earlier user turn must be dropped");
|
|
assert.ok(!prompt.includes("first answer"), "assistant turn must be dropped");
|
|
});
|
|
|
|
test("default call (no window arg) behaves like window 0", () => {
|
|
const prompt = messagesToPrompt(CONVO);
|
|
assert.ok(prompt.includes("second question"));
|
|
assert.ok(!prompt.includes("first answer"));
|
|
});
|
|
|
|
test("window > 0 stitches recent turns into a role-tagged transcript", () => {
|
|
const prompt = messagesToPrompt(CONVO, 10);
|
|
assert.ok(prompt.includes("You are helpful."), "system prompt still leads");
|
|
assert.ok(prompt.includes("first question"), "earlier user turn carried");
|
|
assert.ok(prompt.includes("first answer"), "assistant turn carried");
|
|
assert.ok(prompt.includes("second question"), "latest user turn carried");
|
|
assert.ok(/User:\s*first question/.test(prompt), "user turns role-tagged");
|
|
assert.ok(/Assistant:\s*first answer/.test(prompt), "assistant turns role-tagged");
|
|
});
|
|
|
|
test("window caps to the last N non-system turns", () => {
|
|
const long = [
|
|
{ role: "system", content: "sys" },
|
|
{ role: "user", content: "u1" },
|
|
{ role: "assistant", content: "a1" },
|
|
{ role: "user", content: "u2" },
|
|
{ role: "assistant", content: "a2" },
|
|
{ role: "user", content: "u3" },
|
|
];
|
|
const prompt = messagesToPrompt(long, 2);
|
|
// last 2 non-system turns are a2 + u3
|
|
assert.ok(prompt.includes("a2"), "second-to-last turn present");
|
|
assert.ok(prompt.includes("u3"), "last turn present");
|
|
assert.ok(!prompt.includes("u1"), "older turn dropped");
|
|
assert.ok(!prompt.includes("a1"), "older turn dropped");
|
|
assert.ok(prompt.includes("sys"), "system prompt always present");
|
|
});
|
|
|
|
test("empty messages -> empty prompt", () => {
|
|
assert.equal(messagesToPrompt([], 10), "");
|
|
});
|