feat(providers): complete Jina + Gemini Embedding 2 multimodal via OmniRoute (#10581)

* feat(providers): complete Jina AI via OmniRoute including Omni multimodal

Dashboard and env keys share one Jina credential pool, native v5 Omni
{text}/{image}/{content} docs pass through /v1/embeddings intact, and
classify/segment/search are proxied without a third unused Jina card.

* chore(changelog): name Jina complete-provider fragment for #10581

* feat(providers): make Gemini Embedding 2 multimodal work via OmniRoute

Route gemini-embedding-2 through embedContent/batchEmbedContents so N
OpenAI input items become N vectors, pass through native multimodal
parts, and use dashboard Gemini keys (GEMINI_API_KEY only as fallback).

* fix(providers): resolve rebase fallout for Jina/Gemini embeddings

- narrow the two new no-explicit-any violations introduced by this PR
  (validateJinaFoundationProvider's params + catch, search.ts's
  normalizeJinaSearchResponse data param)
- cast credentials to Record<string, unknown> at the two quota-preflight
  call sites in src/sse/services/auth.ts so the new JinaEnvCredentials /
  GeminiEnvCredentials union members type-check without loosening the
  allRateLimited narrowing used elsewhere in the same function

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: Ravi Tharuma <RaviTharuma@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
This commit is contained in:
Ravi Tharuma
2026-08-18 15:52:43 +02:00
committed by GitHub
parent 228ef6fba9
commit 3d0ffb49a4
41 changed files with 2416 additions and 91 deletions

View File

@@ -1275,6 +1275,14 @@ CURSOR_USER_AGENT="Cursor/3.4"
# hatches that are referenced in code today.
# DEEPSEEK_API_KEY=
# NVIDIA_API_KEY=
# Jina Foundation API + Reader fallback when no dashboard jina-ai / jina-reader
# connection exists. Dashboard keys always win (fill-first).
# JINA_AI_API_KEY=
# JINA_API_KEY=
# Gemini / Google AI Studio embeddings fallback when no dashboard gemini
# connection exists. Dashboard keys always win (fill-first).
# GEMINI_API_KEY=
# GOOGLE_API_KEY=
# Windsurf / Devin CLI direct API key.
# Used by: open-sse/executors/devin-cli.ts — bypasses OAuth when set.

View File

@@ -0,0 +1 @@
- **feat(providers):** complete Jina AI as one credential pool — dashboard `jina-ai` / `jina-reader` share a token, `JINA_AI_API_KEY` is a real fallback, Test probes `GET https://api.jina.ai/v1/models` (embeddings fallback hits `jina-embeddings-v5-omni-small`), embed/rerank logs keep `connection_id`, catalog adds `jina-reranker-v3.5`, Omni v5 multimodal `{text}`/`{image}`/`{content}` docs pass through intact, and OmniRoute proxies classify / segment / `jina-search` (`s.jina.ai`). Reader stays a separate `r.jina.ai` card with an explicit label. Gemini Embedding 2 (`gemini/gemini-embedding-2`, alias `google/gemini-embedding-2`) uses dashboard `gemini` keys (or `GEMINI_API_KEY` / `GOOGLE_API_KEY` only when none exist), forwards native multimodal parts, and maps N OpenAI `input` items to N `:batchEmbedContents` vectors instead of one aggregated `:embedContent`. ([#10581](https://github.com/diegosouzapw/OmniRoute/pull/10581))

View File

@@ -129,18 +129,43 @@ Content-Type: application/json
}
```
Available providers: Nebius, OpenAI, Mistral, Together AI, Fireworks, NVIDIA, **OpenRouter**.
Available providers: Nebius, OpenAI, Mistral, Together AI, Fireworks, NVIDIA, **OpenRouter**, Jina AI.
Catalog ids are `provider/model` (example: `jina-ai/jina-embeddings-v5-omni-small`). Bare Jina model ids that appear in the registry (for example `jina-embeddings-v5-text-small`, `jina-reranker-v3.5`) also resolve. Jina embed/rerank/classify/segment use dashboard `jina-ai` credentials first; `JINA_AI_API_KEY` is a fallback only when no dashboard key exists. The `jina-reader` card is Reader / `r.jina.ai` only (`POST /v1/web/fetch`) and never serves embeddings or rerank.
Registry models that advertise multimodal support also accept up to 32 provider-neutral structured
items. Media item types are `text`, `image`, `audio`, `video`, and `document`. Their media `source`
is either `{"type":"url","url":"https://..."}` or
`{"type":"base64","data":"...","media_type":"..."}`.
Jina v5 Omni (`jina-ai/jina-embeddings-v5-omni-small`, `jina-ai/jina-embeddings-v5-omni-nano`,
and the family alias `jina-ai/jina-embeddings-v5-omni` → omni-small) also accepts Jina's native
EmbeddingsV5Request docs and **forwards them intact** to `https://api.jina.ai/v1/embeddings`:
```json
{
"model": "jina-ai/jina-embeddings-v5-omni-small",
"task": "retrieval.query",
"normalized": true,
"input": [
{ "text": "a red bicycle" },
{ "image": "https://example.com/bike.png" },
{ "content": [{ "text": "caption" }, { "image": "data:image/png;base64,..." }] }
]
}
```
Native `{ image | audio | video | pdf }` values may be a public HTTPS URL, a `data:` URI, or raw
base64. OmniRoute does not stringify those objects or fetch native image URLs — Jina retrieves
public media itself. Extra Jina fields (`task`, `normalized`, `truncate`, `embedding_type`) are
forwarded. Text-only Jina SKUs still reject non-text docs.
Security and transport bounds:
- Remote media URLs must be public HTTPS. OmniRoute fetches them server-side with redirect
revalidation, timeout, decoded size limits, public DNS checks, and connection pinning to a
validated answer before the provider call. Providers never receive the original remote URL.
- Remote media URLs must be public HTTPS. Canonical `{type,source:url}` items are fetched
server-side (redirect revalidation, timeout, size limits, public DNS, connection pinning) and
inlined before the provider call. Jina-native `{image:"https://..."}` items are forwarded as-is
after the same public-HTTPS check; Jina fetches the URL.
- Inline base64 media is limited to 8 MiB decoded per item and 16 MiB decoded across the request.
Provider translation (canonical items are never forwarded unchanged):
@@ -338,6 +363,8 @@ Use this endpoint when a sidecar runs out-of-process and cannot import
| POST | `/v1/audio/transcriptions` | OpenAI Audio (STT) |
| POST | `/v1/audio/speech` | OpenAI TTS (returns audio body) |
| POST | `/v1/rerank` | Cohere/Voyage-style rerank |
| POST | `/v1/classify` | Jina classify (`api.jina.ai`) |
| POST | `/v1/segment` | Jina segmenter (`segment.jina.ai`) |
| POST | `/v1/moderations` | OpenAI Moderations |
| GET | `/v1/models` | OpenAI |
| POST | `/v1/messages/count_tokens` | Anthropic |
@@ -357,7 +384,16 @@ For clients that cannot attach `Authorization: Bearer ...`, OmniRoute also accep
```bash
# Rerank
POST /v1/rerank { "model": "cohere/rerank-3", "query": "...", "documents": ["..."] }
POST /v1/rerank { "model": "jina-ai/jina-reranker-v3.5", "query": "...", "documents": ["..."] }
# Jina classify (Foundation API credentials)
POST /v1/classify { "model": "jina-embeddings-v5-text-small", "input": ["..."], "labels": ["a", "b"] }
# Jina segmenter
POST /v1/segment { "content": "...", "return_chunks": true }
# Jina search (s.jina.ai; provider aliases: jina-search, jina-ai, jina)
POST /v1/search { "query": "...", "provider": "jina-search" }
# Moderations
POST /v1/moderations { "model": "omni-moderation-latest", "input": "..." }

View File

@@ -656,12 +656,20 @@ Recognized pattern: `{PROVIDER_ID}_API_KEY`
| ------------------ | ---------- |
| `DEEPSEEK_API_KEY` | DeepSeek |
| `NVIDIA_API_KEY` | NVIDIA NIM |
| `JINA_AI_API_KEY` | Jina AI (Foundation API + Reader fallback) |
| `JINA_API_KEY` | Jina AI (alias for `JINA_AI_API_KEY`) |
| `GEMINI_API_KEY` | Gemini (Google AI Studio) embeddings + chat fallback |
| `GOOGLE_API_KEY` | Gemini (alias for `GEMINI_API_KEY`) |
> [!NOTE]
> Static `${PROVIDER}_API_KEY` entries for Groq, xAI, Mistral, Perplexity, Together AI, Fireworks, Cerebras, Cohere, Nebius, and Qianfan were removed in v3.8.0 because the runtime no longer reads them — those providers rely exclusively on Dashboard / `data/provider-credentials.json` / the encrypted DB. See the _Audit: Removed / Dead Variables_ section at the bottom of this document for the migration path.
> [!TIP]
> Keys set via the Dashboard are stored encrypted in SQLite and take precedence over environment variables.
>
> **Jina:** `jina-ai/…` embeddings, rerank, classify, segment, and `jina-search` do **not** bill a cluster env key when a dashboard `jina-ai` (or shared `jina-reader`) connection exists — `getProviderCredentials` is fill-first. `JINA_AI_API_KEY` / `JINA_API_KEY` are used only when no usable dashboard key exists. Call logs attribute the env fallback as `connection_id=env:JINA_AI_API_KEY`. The Reader card (`jina-reader`, `r.jina.ai`) never serves `/v1/embeddings` or `/v1/rerank`.
>
> **Gemini:** `gemini/gemini-embedding-2` (alias `google/gemini-embedding-2`) uses the dashboard `gemini` connection first. `GEMINI_API_KEY` / `GOOGLE_API_KEY` are used only when no usable dashboard key exists. Call logs attribute the env fallback as `connection_id=env:GEMINI_API_KEY`. Native multimodal traffic uses `x-goog-api-key` against `:embedContent` / `:batchEmbedContents` — N OpenAI `input` items become N vectors.
---

View File

@@ -224,8 +224,8 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
| `inception` | `inception` | Inception | API key | [link](https://docs.inceptionlabs.ai) | 10M free tokens on signup, no credit card required. |
| `inference-net` | `inet` | Inference.net | API key | [link](https://inference.net) | $25 free credits on signup plus research grants available |
| `internlm` | `internlm` | InternLM (Intern-S1) | API key | [link](https://internlm.intern-ai.org.cn/) | Free monthly quota ~1M input / 3M output tokens (~10 RPM) |
| `jina-ai` | `jina` | Jina AI | API key, embed/rerank | [link](https://jina.ai) | Bearer API key for the Jina AI rerank API. |
| `jina-reader` | `jr` | Jina Reader | API key | [link](https://jina.ai/reader) | |
| `jina-ai` | `jina` | Jina AI (Foundation API) | API key, embed/rerank | [link](https://jina.ai) | Bearer API key for api.jina.ai — embeddings, rerank, classify, segment, and search. Dashboard keys take precedence over JINA_AI_API_KEY. This is not the Reader / r.jina.ai card and does not fetch URLs. |
| `jina-reader` | `jr` | Jina Reader (r.jina.ai) | API key | [link](https://jina.ai/reader) | Bearer API key for r.jina.ai URL-to-markdown (/v1/web/fetch only). Does not serve /v1/embeddings or /v1/rerank. The same Jina token as Foundation API works; OmniRoute reuses a jina-ai dashboard key or JINA_AI_API_KEY when this card is empty. |
| `kenari` | `kenari` | Kenari | API key | [link](https://kenari.id) | Use your Kenari API key (kn-...) in Authorization: Bearer <key>. Fully OpenAI-compatible. API base URL: https://kenari.id/v1. |
| `kie` | `kie` | KIE.AI | API key | [link](https://kie.ai) | — |
| `kilo-gateway` | `kg` | Kilo Gateway | API key, aggregator | [link](https://kilo.ai) | — |

View File

@@ -264,13 +264,13 @@ export const EMBEDDING_PROVIDERS: Record<string, EmbeddingProvider> = {
{
id: "gemini-embedding-2",
name: "Gemini Embedding 2",
dimensions: 768,
dimensions: 3072,
modalities: ["text", "image", "audio", "video", "document"],
},
{
id: "gemini-embedding-2-preview",
name: "Gemini Embedding 2 Preview",
dimensions: 768,
dimensions: 3072,
modalities: ["text", "image", "audio", "video", "document"],
},
{ id: "gemini-embedding-001", name: "Gemini Embedding 001", dimensions: 768 },
@@ -415,6 +415,28 @@ const EMBEDDING_PROVIDER_ALIASES: Record<string, string> = {
voyage: "voyage-ai",
};
/** Family name used by clients; Jina's public SKU is omni-small. */
const EMBEDDING_MODEL_ALIASES: Record<string, string> = {
"jina-embeddings-v5-omni": "jina-embeddings-v5-omni-small",
// Live native catalog is gemini/gemini-embedding-2. Clients that send the
// OpenRouter-style google/ prefix still resolve to the Gemini provider —
// do not steal a custom provider_node whose prefix is `google`.
"google/gemini-embedding-2": "gemini/gemini-embedding-2",
"google/gemini-embedding-2-preview": "gemini/gemini-embedding-2-preview",
};
function applyEmbeddingModelAliases(modelStr: string): string {
for (const [alias, canonical] of Object.entries(EMBEDDING_MODEL_ALIASES)) {
if (modelStr === alias) return canonical;
// Slash-containing aliases are exact-match only so
// openrouter/google/gemini-embedding-2 stays on OpenRouter.
if (!alias.includes("/") && modelStr.endsWith(`/${alias}`)) {
return `${modelStr.slice(0, -alias.length)}${canonical}`;
}
}
return modelStr;
}
function resolveEmbeddingProviderId(providerId: string): string {
return EMBEDDING_PROVIDER_ALIASES[providerId] || providerId;
}
@@ -452,6 +474,7 @@ export function parseEmbeddingModel(
dynamicProviders?: EmbeddingProvider[]
): { provider: string | null; model: string | null } {
if (!modelStr) return { provider: null, model: null };
modelStr = applyEmbeddingModelAliases(modelStr);
// Check for "provider/model" format
const slashIdx = modelStr.indexOf("/");

View File

@@ -71,8 +71,10 @@ export const RERANK_PROVIDERS = {
authType: "apikey",
authHeader: "bearer",
models: [
{ id: "jina-reranker-v3.5", name: "Jina Reranker v3.5" },
{ id: "jina-reranker-v3", name: "Jina Reranker v3" },
{ id: "jina-reranker-m0", name: "Jina Reranker m0" },
{ id: "jina-reranker-v2-base-multilingual", name: "Jina Reranker v2 Base Multilingual" },
],
},

View File

@@ -243,6 +243,24 @@ export const SEARCH_PROVIDERS: Record<string, SearchProviderConfig> = {
cacheTTLMs: 5 * 60 * 1000,
},
// Jina Search (s.jina.ai). No extra dashboard card — credentials reuse
// jina-ai / jina-reader / JINA_AI_API_KEY via SEARCH_CREDENTIAL_FALLBACKS.
"jina-search": {
id: "jina-search",
name: "Jina Search (s.jina.ai)",
baseUrl: "https://s.jina.ai",
method: "POST",
authType: "apikey",
authHeader: "bearer",
costPerQuery: 0.002,
freeMonthlyQuota: 1000,
searchTypes: ["web"],
defaultMaxResults: 5,
maxMaxResults: 50,
timeoutMs: 15_000,
cacheTTLMs: 5 * 60 * 1000,
},
// Free, no-API-key DuckDuckGo lite scraping (free-claude-code port). Last-resort
// only (fallbackOnly): never auto-selected over a configured provider; served by
// the dedicated HTML path in open-sse/handlers/search.ts (not the generic JSON one).
@@ -272,21 +290,45 @@ export const SEARCH_CREDENTIAL_FALLBACKS: Record<string, string> = {
"perplexity-search": "perplexity",
"ollama-search": "ollama-cloud",
"zai-search": "zai",
"jina-search": "jina-ai",
};
/**
* Get search provider config by ID
* Request-only aliases for POST /v1/search.
*
* Do not apply these in getSearchProvider(). jina-ai is the Foundation
* embed/rerank/classify provider; remapping it here made the models
* catalog treat jina-ai as a search-only card (searchTypes → "web").
*/
export const SEARCH_PROVIDER_ALIASES: Record<string, string> = {
"jina-ai": "jina-search",
jina: "jina-search",
};
export function resolveSearchProviderId(providerId: string): string {
return SEARCH_PROVIDER_ALIASES[providerId] || providerId;
}
/**
* Exact catalog lookup. Used by model listing / static catalogs.
* Request routing should use resolveSearchProvider() so aliases work
* without colliding with the Foundation jina-ai provider id.
*/
export function getSearchProvider(providerId: string): SearchProviderConfig | null {
return SEARCH_PROVIDERS[providerId] || null;
}
/** Resolve a /v1/search provider id, including Foundation aliases. */
export function resolveSearchProvider(providerId: string): SearchProviderConfig | null {
return SEARCH_PROVIDERS[resolveSearchProviderId(providerId)] || null;
}
export function supportsSearchType(
providerOrId: SearchProviderConfig | string | null | undefined,
searchType: string
): boolean {
const provider =
typeof providerOrId === "string" ? getSearchProvider(providerOrId) : providerOrId || null;
typeof providerOrId === "string" ? resolveSearchProvider(providerOrId) : providerOrId || null;
if (!provider) return false;
return provider.searchTypes.includes(searchType);
}
@@ -316,7 +358,7 @@ export function selectProvider(
searchType?: string
): SearchProviderConfig | null {
if (explicitProvider) {
const provider = SEARCH_PROVIDERS[explicitProvider] || null;
const provider = resolveSearchProvider(explicitProvider);
if (!provider) return null;
if (searchType && !supportsSearchType(provider, searchType)) return null;
return provider;

View File

@@ -1,6 +1,18 @@
import { MAX_EMBEDDING_INLINE_TOTAL_BYTES } from "@/shared/validation/schemas/apiV1";
import type { EmbeddingMultimodalItem } from "@/shared/validation/schemas/apiV1";
import type { EmbeddingProvider } from "../config/embeddingRegistry.ts";
import {
isCanonicalEmbeddingItem,
isJinaMergedContentGroup,
isJinaNativeDoc,
isJinaNativeEmbeddingItem,
isPlainObject,
} from "@/shared/validation/jinaNativeEmbeddingInput";
import {
isGeminiNativeContent,
isGeminiNativeEmbedRequest,
isGeminiNativePart,
} from "@/shared/validation/geminiNativeEmbeddingInput";
const AGGREGATE_SIZE_ERROR = "decoded inline media must not exceed 16 MiB per request";
@@ -101,12 +113,165 @@ async function prepareJinaInput(
});
}
/**
* Mixed batches: keep Jina-native docs / strings intact and only translate
* OmniRoute canonical `{ type, source }` items into Jina ImageDoc/TextDoc.
*/
export async function prepareJinaMixedEmbeddingInput(
input: unknown[],
fetchMedia: StructuredEmbeddingFetchOptions["fetchMedia"]
): Promise<unknown[]> {
const out: unknown[] = [];
for (const item of input) {
if (typeof item === "string" || isJinaNativeEmbeddingItem(item)) {
out.push(item);
continue;
}
if (isCanonicalEmbeddingItem(item)) {
const [translated] = await prepareJinaInput(
[item as EmbeddingMultimodalItem],
fetchMedia
);
out.push(translated);
continue;
}
out.push(item);
}
return out;
}
function mapGeminiTaskType(value: unknown): unknown {
if (value === "retrieval.query") return "RETRIEVAL_QUERY";
if (value === "retrieval.passage") return "RETRIEVAL_DOCUMENT";
return value;
}
function geminiNativeUrl(model: string, method: "embedContent" | "batchEmbedContents"): string {
return `https://generativelanguage.googleapis.com/v1beta/models/${encodeURIComponent(model)}:${method}`;
}
function geminiRequestExtras(body: Record<string, unknown>): Record<string, unknown> {
const extras: Record<string, unknown> = {};
if (body.dimensions !== undefined) extras.output_dimensionality = body.dimensions;
if (body.task !== undefined) extras.task_type = mapGeminiTaskType(body.task);
return extras;
}
function embeddingValues(entry: unknown): unknown[] {
if (!entry || typeof entry !== "object") return [];
const values = (entry as { values?: unknown }).values;
return Array.isArray(values) ? values : [];
}
function normalizeGeminiEmbedContentResponse(data: Record<string, unknown>): Record<string, unknown> {
return {
object: "list",
data: [{ object: "embedding", embedding: embeddingValues(data.embedding), index: 0 }],
usage: { prompt_tokens: 0, total_tokens: 0 },
};
}
function normalizeGeminiBatchResponse(data: Record<string, unknown>): Record<string, unknown> {
const embeddings = Array.isArray(data.embeddings) ? data.embeddings : [];
return {
object: "list",
data: embeddings.map((entry, index) => ({
object: "embedding",
embedding: embeddingValues(entry),
index,
})),
usage: { prompt_tokens: 0, total_tokens: 0 },
};
}
function dataUriToInlineData(value: string): { mime_type: string; data: string } | null {
const match = /^data:([^;,]+);base64,(.+)$/i.exec(value.trim());
if (!match) return null;
return { mime_type: match[1], data: match[2] };
}
async function mediaStringToGeminiPart(
raw: string,
fallbackMime: string,
fetchMedia: StructuredEmbeddingFetchOptions["fetchMedia"]
): Promise<Record<string, unknown>> {
const trimmed = raw.trim();
const fromDataUri = dataUriToInlineData(trimmed);
if (fromDataUri) return { inline_data: fromDataUri };
if (/^https:\/\//i.test(trimmed)) {
const fetched = await fetchMedia(trimmed);
if (!fetched.contentType) {
throw new Error("Remote embedding media must include a Content-Type header");
}
return {
inline_data: {
mime_type: fetched.contentType,
data: fetched.buffer.toString("base64"),
},
};
}
return { inline_data: { mime_type: fallbackMime, data: trimmed } };
}
async function jinaDocToGeminiPart(
item: Record<string, unknown>,
fetchMedia: StructuredEmbeddingFetchOptions["fetchMedia"]
): Promise<Record<string, unknown>> {
if (typeof item.text === "string") return { text: item.text };
if (typeof item.image === "string") {
return mediaStringToGeminiPart(item.image, "image/png", fetchMedia);
}
if (typeof item.audio === "string") {
return mediaStringToGeminiPart(item.audio, "audio/mpeg", fetchMedia);
}
if (typeof item.video === "string") {
return mediaStringToGeminiPart(item.video, "video/mp4", fetchMedia);
}
if (typeof item.pdf === "string") {
return mediaStringToGeminiPart(item.pdf, "application/pdf", fetchMedia);
}
throw new Error("Unsupported Jina-native embedding item for Gemini");
}
/**
* Map one OpenAI-compat input element to one Gemini Content.
* A fused multimodal item (native parts / Jina content group / one canonical
* object) stays one Content. Do not dump sibling array elements into parts.
*/
async function itemToGeminiContent(
item: unknown,
fetchMedia: StructuredEmbeddingFetchOptions["fetchMedia"]
): Promise<Record<string, unknown>> {
if (typeof item === "string") return { parts: [{ text: item }] };
if (isGeminiNativeEmbedRequest(item)) {
return (item as { content: Record<string, unknown> }).content;
}
if (isGeminiNativeContent(item)) {
return item as Record<string, unknown>;
}
if (isGeminiNativePart(item)) {
return { parts: [item as Record<string, unknown>] };
}
if (isJinaMergedContentGroup(item)) {
const parts: Record<string, unknown>[] = [];
for (const chunk of (item as { content: unknown[] }).content) {
if (isPlainObject(chunk)) parts.push(await jinaDocToGeminiPart(chunk, fetchMedia));
}
return { parts };
}
if (isJinaNativeDoc(item) && isPlainObject(item)) {
return { parts: [await jinaDocToGeminiPart(item, fetchMedia)] };
}
if (isCanonicalEmbeddingItem(item)) {
const [part] = await prepareGeminiParts(
[item as EmbeddingMultimodalItem],
fetchMedia
);
return { parts: [part] };
}
throw new Error("Unsupported Gemini embedding input item");
}
async function prepareGeminiParts(
items: EmbeddingMultimodalItem[],
fetchMedia: StructuredEmbeddingFetchOptions["fetchMedia"]
@@ -118,19 +283,17 @@ async function prepareGeminiParts(
});
}
function normalizeGeminiResponse(data: Record<string, unknown>): Record<string, unknown> {
const embedding = data.embedding as { values?: unknown } | undefined;
return {
object: "list",
data: [{ object: "embedding", embedding: embedding?.values ?? [], index: 0 }],
usage: { prompt_tokens: 0, total_tokens: 0 },
};
function normalizeEmbeddingInputItems(input: unknown): unknown[] {
if (Array.isArray(input)) return input;
if (input === undefined || input === null) return [];
return [input];
}
/**
* Translate OmniRoute's provider-neutral structured input into a documented
* provider-native transport. Each top-level canonical array is one logical
* multimodal item for Gemini and one vector-per-item batch for Jina.
* provider-native transport. Each top-level input array element is one
* embedding. Gemini Embedding 2 fuses multiple parts inside one Content;
* N OpenAI `input` items must become N vectors via batchEmbedContents.
*/
export async function prepareStructuredEmbeddingRequest(
provider: EmbeddingProvider,
@@ -139,25 +302,46 @@ export async function prepareStructuredEmbeddingRequest(
token: string,
options: StructuredEmbeddingFetchOptions
): Promise<PreparedEmbeddingRequest> {
const items = body.input as EmbeddingMultimodalItem[];
const items = normalizeEmbeddingInputItems(body.input);
if (provider.structuredInputProtocol === "jina-v1") {
return {
url: provider.baseUrl,
body: { ...body, model, input: await prepareJinaInput(items, options.fetchMedia) },
body: {
...body,
model,
input: await prepareJinaInput(items as EmbeddingMultimodalItem[], options.fetchMedia),
},
};
}
if (provider.structuredInputProtocol === "gemini-embed-content") {
const parts = await prepareGeminiParts(items, options.fetchMedia);
const request: Record<string, unknown> = {
content: { parts },
};
if (body.dimensions !== undefined) request.output_dimensionality = body.dimensions;
if (body.task !== undefined) request.task_type = mapGeminiTaskType(body.task);
const contents: Record<string, unknown>[] = [];
for (const item of items) {
contents.push(await itemToGeminiContent(item, options.fetchMedia));
}
if (contents.length === 0) {
throw new Error("Gemini embedding input must contain at least one item");
}
const extras = geminiRequestExtras(body);
const authHeader = { name: "x-goog-api-key", value: token };
if (contents.length === 1) {
return {
url: geminiNativeUrl(model, "embedContent"),
body: { content: contents[0], ...extras },
authHeader,
normalizeResponse: normalizeGeminiEmbedContentResponse,
};
}
return {
url: `https://generativelanguage.googleapis.com/v1beta/models/${encodeURIComponent(model)}:embedContent`,
body: request,
authHeader: { name: "x-goog-api-key", value: token },
normalizeResponse: normalizeGeminiResponse,
url: geminiNativeUrl(model, "batchEmbedContents"),
body: {
requests: contents.map((content) => ({
model: `models/${model}`,
content,
...extras,
})),
},
authHeader,
normalizeResponse: normalizeGeminiBatchResponse,
};
}
throw new Error(`Provider ${provider.id} has no structured embedding input translator`);

View File

@@ -32,10 +32,20 @@ import { stripTrailingSlashes } from "../utils/urlSanitize.ts";
import { fetchRemoteImage } from "@/shared/network/remoteImageFetch";
import {
hasStructuredEmbeddingInput,
prepareJinaMixedEmbeddingInput,
prepareStructuredEmbeddingRequest,
} from "./embeddingStructuredInput.ts";
import { MAX_EMBEDDING_INLINE_ITEM_BYTES } from "@/shared/validation/schemas/apiV1";
import { markAccountUnavailable } from "../../src/sse/services/auth.ts";
import {
collectJinaNativeModalities,
isJinaNativeEmbeddingInput,
} from "@/shared/validation/jinaNativeEmbeddingInput";
import {
collectGeminiNativeModalities,
isGeminiEmbedding2Family,
isGeminiNativeEmbeddingInput,
} from "@/shared/validation/geminiNativeEmbeddingInput";
interface ClientRawRequest {
endpoint: string;
@@ -171,7 +181,15 @@ export async function handleEmbedding({
typeof item === "object" && item !== null && "type" in item
)
: [];
if (structuredItems.length > 0) {
const nativeModalities = [
...(isJinaNativeEmbeddingInput(body.input)
? collectJinaNativeModalities(body.input)
: []),
...(isGeminiNativeEmbeddingInput(body.input)
? collectGeminiNativeModalities(body.input)
: []),
].filter((modality) => modality !== "text");
if (structuredItems.length > 0 || nativeModalities.length > 0) {
const supportedModalities = getEmbeddingModelModalities(providerConfig, model);
if (!supportedModalities) {
return {
@@ -180,12 +198,24 @@ export async function handleEmbedding({
error: `Embedding model ${body.model} does not advertise structured embedding input support`,
};
}
const unsupported = structuredItems.find((item) => !supportedModalities.includes(item.type));
if (unsupported) {
const unsupportedCanonical = structuredItems.find(
(item) => !supportedModalities.includes(item.type)
);
if (unsupportedCanonical) {
return {
success: false,
status: 400,
error: `Embedding model ${body.model} does not support ${unsupported.type} input`,
error: `Embedding model ${body.model} does not support ${unsupportedCanonical.type} input`,
};
}
const unsupportedNative = nativeModalities.find(
(modality) => !supportedModalities.includes(modality)
);
if (unsupportedNative) {
return {
success: false,
status: 400,
error: `Embedding model ${body.model} does not support ${unsupportedNative} input`,
};
}
}
@@ -278,7 +308,39 @@ export async function handleEmbedding({
};
}
if (hasStructuredEmbeddingInput(body.input)) {
// Jina v5 Omni native docs ({ text }, { image: url|base64 }, { content: [...] })
// must reach api.jina.ai unchanged. Do not fetch those image URLs or collapse
// to string[]. Canonical { type, source } items still go through the translator.
const jinaNative = isJinaNativeEmbeddingInput(body.input);
const geminiNative = isGeminiNativeEmbeddingInput(body.input);
const canonicalStructured = hasStructuredEmbeddingInput(body.input);
const passThroughJinaNative =
providerConfig.structuredInputProtocol === "jina-v1" && jinaNative && !canonicalStructured;
// gemini-embedding-2 aggregates a string[] on Google's OpenAI shim into one
// vector. Always use embedContent / batchEmbedContents so N input items
// become N embeddings. Native multimodal parts take the same path.
const useGeminiNativeTransport =
providerConfig.structuredInputProtocol === "gemini-embed-content" &&
(isGeminiEmbedding2Family(model) ||
canonicalStructured ||
geminiNative ||
jinaNative);
if (providerConfig.structuredInputProtocol === "jina-v1" && jinaNative && canonicalStructured) {
try {
const mixed = Array.isArray(body.input) ? body.input : [body.input];
upstreamBody.input = await prepareJinaMixedEmbeddingInput(mixed, async (url) => {
const result = await fetchRemoteImage(url, {
guard: "public-only",
maxBytes: MAX_EMBEDDING_INLINE_ITEM_BYTES,
pinDns: true,
});
return { buffer: result.buffer, contentType: result.contentType || null };
});
} catch (error) {
return { success: false, status: 400, error: sanitizeErrorMessage(error) };
}
} else if (useGeminiNativeTransport || (!passThroughJinaNative && canonicalStructured)) {
if (!model) {
return {
success: false,

View File

@@ -0,0 +1,101 @@
/**
* Jina Foundation API proxy.
*
* Forwards classify / segment (and similar JSON POSTs) to Jina using the same
* dashboard-or-env credentials as embeddings and rerank.
*/
import { CORS_HEADERS } from "../utils/cors.ts";
import { errorResponse } from "../utils/error.ts";
import { attachOmniRouteMetaHeaders } from "@/domain/omnirouteResponseMeta";
import { generateRequestId } from "@/shared/utils/requestId";
import { saveCallLog } from "@/lib/usageDb";
export interface JinaFoundationCredentials {
apiKey?: string | null;
accessToken?: string | null;
connectionId?: string | null;
}
export interface JinaFoundationProxyOptions {
path: string;
upstreamUrl: string;
body: Record<string, unknown>;
credentials: JinaFoundationCredentials | null;
provider?: string;
model?: string | null;
}
export async function handleJinaFoundationProxy(
options: JinaFoundationProxyOptions
): Promise<Response> {
const startTime = Date.now();
const provider = options.provider || "jina-ai";
const token = options.credentials?.apiKey || options.credentials?.accessToken;
const connectionId = options.credentials?.connectionId || null;
if (!token) {
return errorResponse(401, `No credentials for Jina provider: ${provider}`);
}
try {
const res = await fetch(options.upstreamUrl, {
method: "POST",
headers: {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: `Bearer ${token}`,
},
body: JSON.stringify(options.body),
});
const text = await res.text();
let parsed: unknown = null;
try {
parsed = text ? JSON.parse(text) : null;
} catch {
parsed = { error: text.slice(0, 500) };
}
saveCallLog({
method: "POST",
path: options.path,
status: res.status,
model: options.model || `${provider}${options.path}`,
provider,
duration: Date.now() - startTime,
tokens: { prompt_tokens: 0, completion_tokens: 0 },
connectionId,
...(res.ok
? {}
: {
error:
(parsed as { message?: string; error?: { message?: string } } | null)?.message ||
(parsed as { error?: { message?: string } } | null)?.error?.message ||
text.slice(0, 500),
}),
}).catch(() => {});
if (!res.ok) {
const err = parsed as { message?: string; error?: { message?: string } | string } | null;
const message =
err?.message ||
(typeof err?.error === "string" ? err.error : err?.error?.message) ||
`Provider returned HTTP ${res.status}`;
return errorResponse(res.status, message);
}
const headers = new Headers({ ...CORS_HEADERS, "Content-Type": "application/json" });
attachOmniRouteMetaHeaders(headers, {
provider,
model: options.model || provider,
costUsd: 0,
latencyMs: Date.now() - startTime,
requestId: generateRequestId(),
});
return new Response(JSON.stringify(parsed), { status: 200, headers });
} catch (err) {
const message = err instanceof Error ? err.message : String(err);
return errorResponse(500, `Jina request failed: ${message}`);
}
}

View File

@@ -292,6 +292,7 @@ export async function handleRerank({
duration: Date.now() - startTime,
tokens: { prompt_tokens: 0, completion_tokens: 0 },
responseBody: { results_count: Array.isArray(result?.results) ? result.results.length : 0 },
connectionId,
}).catch(() => {});
const headers = new Headers({ ...CORS_HEADERS, "Content-Type": "application/json" });

View File

@@ -6,7 +6,8 @@ import { randomUUID } from "crypto";
* Routes to search providers with automatic failover:
* serper-search, brave-search, perplexity-search, exa-search, tavily-search,
* firecrawl, google-pse-search, linkup-search, searchapi-search,
* youcom-search, searxng-search, ollama-search, zai-search, duckduckgo-free
* youcom-search, searxng-search, ollama-search, zai-search, jina-search,
* duckduckgo-free
*
* Request format:
* {
@@ -21,6 +22,7 @@ import { getSearchProvider, type SearchProviderConfig } from "../config/searchRe
import { buildPerplexityRequest, parsePerplexitySearchOptions } from "./search/perplexitySearch.ts";
import * as fcSearch from "./search/firecrawlSearch.ts";
import { type FirecrawlSearchEnvelope } from "./search/firecrawlSearch.ts";
import { buildJinaSearchRequest, extractJinaSearchItems } from "./search/jinaSearch.ts";
import { freeWebSearch } from "../services/freeWebSearch.ts";
import { saveCallLog } from "@/lib/usageDb";
import { safeOutboundFetch } from "@/shared/network/safeOutboundFetch";
@@ -625,6 +627,7 @@ const requestBuilders: Record<string, SearchRequestBuilder> = {
"youcom-search": buildYouComRequest,
"searxng-search": buildSearxngRequest,
"ollama-search": buildOllamaRequest,
"jina-search": buildJinaSearchRequest,
};
function buildRequest(
@@ -1202,6 +1205,7 @@ const responseNormalizers: Record<string, SearchResponseNormalizer> = {
"youcom-search": normalizeYouComResponse,
"searxng-search": normalizeSearxngResponse,
"ollama-search": normalizeOllamaResponse,
"jina-search": normalizeJinaSearchResponse,
};
function normalizeResponse(
@@ -1216,6 +1220,30 @@ function normalizeResponse(
return { results: [], totalResults: null };
}
function normalizeJinaSearchResponse(
data: unknown,
_query: string,
_searchType: string
): { results: SearchResult[]; totalResults: number | null } {
const now = new Date().toISOString();
const items = extractJinaSearchItems(data);
const results = items.map((item, idx) =>
makeResult(
"jina-search",
{
title: item.title,
url: item.url,
snippet: item.description || item.snippet || "",
full_text: item.content || item.text,
text_format: "markdown",
},
idx,
now
)
);
return { results, totalResults: results.length };
}
export async function handleSearch(options: SearchHandlerOptions): Promise<SearchHandlerResult> {
const {
query,

View File

@@ -0,0 +1,69 @@
/**
* Jina Search (s.jina.ai) request builder + response normalizer.
*
* Uses the same Bearer token as the Jina Foundation API. OmniRoute does not
* add a third dashboard card — credentials come from jina-ai / jina-reader /
* JINA_AI_API_KEY.
*/
import type { SearchProviderConfig } from "../../config/searchRegistry.ts";
export interface JinaSearchRequestParams {
query: string;
maxResults: number;
token?: string | null;
country?: string;
language?: string;
offset?: number;
}
export interface JinaSearchNormalizeItem {
title?: string;
url?: string;
description?: string;
snippet?: string;
content?: string;
text?: string;
}
export function buildJinaSearchRequest(
config: SearchProviderConfig,
params: JinaSearchRequestParams
): { url: string; init: RequestInit } {
const headers: Record<string, string> = {
"Content-Type": "application/json",
Accept: "application/json",
};
if (params.token) {
headers.Authorization = `Bearer ${params.token}`;
}
const body: Record<string, unknown> = {
q: params.query,
num: params.maxResults,
};
if (params.country) body.gl = params.country;
if (params.language) body.hl = params.language;
if (typeof params.offset === "number" && params.offset > 0) {
body.page = params.offset;
}
return {
url: config.baseUrl.endsWith("/") ? config.baseUrl : `${config.baseUrl}/`,
init: {
method: "POST",
headers,
body: JSON.stringify(body),
},
};
}
export function extractJinaSearchItems(data: unknown): JinaSearchNormalizeItem[] {
if (Array.isArray(data)) return data as JinaSearchNormalizeItem[];
if (data && typeof data === "object") {
const record = data as { data?: unknown; results?: unknown };
if (Array.isArray(record.data)) return record.data as JinaSearchNormalizeItem[];
if (Array.isArray(record.results)) return record.results as JinaSearchNormalizeItem[];
}
return [];
}

View File

@@ -35,7 +35,7 @@ const FETCH_PROVIDERS: FetchProviderDef[] = [
},
{
id: "jina-reader",
name: "Jina Reader",
name: "Jina Reader (r.jina.ai)",
costPerQuery: 0.0005,
freeMonthlyQuota: 1000,
fetchFormats: ["markdown", "text"],

View File

@@ -0,0 +1,79 @@
import { handleJinaFoundationProxy } from "@omniroute/open-sse/handlers/jinaFoundation.ts";
import {
getProviderCredentialsWithQuotaPreflight,
clearRecoveredProviderState,
} from "@/sse/services/auth";
import { withInjectionGuard } from "@/middleware/promptInjectionGuard";
import { errorResponse } from "@omniroute/open-sse/utils/error.ts";
import { HTTP_STATUS } from "@omniroute/open-sse/config/constants.ts";
import { enforceApiKeyPolicy } from "@/shared/utils/apiKeyPolicy";
import { v1ClassifySchema } from "@/shared/validation/schemas";
import { isValidationFailure, validateBody } from "@/shared/validation/helpers";
import {
isAllRateLimitedCredentials,
rateLimitedProviderResponse,
} from "@/app/api/v1/_shared/rateLimit";
import { JINA_FOUNDATION_BASE_URL, JINA_FOUNDATION_PROVIDER_ID } from "@/lib/providers/jina";
/**
* Handle CORS preflight
*/
export async function OPTIONS() {
return new Response(null, {
headers: {
"Access-Control-Allow-Methods": "POST, OPTIONS",
"Access-Control-Allow-Headers": "*",
},
});
}
/**
* POST /v1/classify — Jina zero/few-shot classification.
*
* Proxies to https://api.jina.ai/v1/classify using jina-ai dashboard
* credentials (or JINA_AI_API_KEY when no dashboard key exists).
*/
async function postHandler(request: Request) {
let rawBody: unknown;
try {
rawBody = await request.json();
} catch {
return errorResponse(HTTP_STATUS.BAD_REQUEST, "Invalid JSON body");
}
const validation = validateBody(v1ClassifySchema, rawBody);
if (isValidationFailure(validation)) {
return errorResponse(HTTP_STATUS.BAD_REQUEST, validation.error.message);
}
const body = validation.data;
const model = typeof body.model === "string" ? body.model : undefined;
const policy = await enforceApiKeyPolicy(request, model || "jina-ai/classify");
if (policy.rejection) return policy.rejection;
const credentials = await getProviderCredentialsWithQuotaPreflight(JINA_FOUNDATION_PROVIDER_ID);
if (!credentials) {
return errorResponse(
HTTP_STATUS.BAD_REQUEST,
`No credentials for provider: ${JINA_FOUNDATION_PROVIDER_ID}`
);
}
if (isAllRateLimitedCredentials(credentials)) {
return rateLimitedProviderResponse(JINA_FOUNDATION_PROVIDER_ID, credentials);
}
const response = await handleJinaFoundationProxy({
path: "/v1/classify",
upstreamUrl: `${JINA_FOUNDATION_BASE_URL}/v1/classify`,
body,
credentials,
provider: JINA_FOUNDATION_PROVIDER_ID,
model: model || null,
});
if (response?.ok) {
await clearRecoveredProviderState(credentials);
}
return response;
}
export const POST = withInjectionGuard(postHandler);

View File

@@ -7,6 +7,7 @@ import {
import {
getAllSearchProviders,
getSearchProvider,
resolveSearchProvider,
selectProvider,
supportsSearchType,
SEARCH_PROVIDERS,
@@ -129,7 +130,7 @@ async function postHandler(request: Request, context: unknown) {
// Resolve provider and credentials
if (body.provider) {
const explicitProvider = getSearchProvider(body.provider);
const explicitProvider = resolveSearchProvider(body.provider);
if (!explicitProvider) {
return errorResponse(HTTP_STATUS.BAD_REQUEST, `Unknown search provider: ${body.provider}`);
}

View File

@@ -0,0 +1,79 @@
import { handleJinaFoundationProxy } from "@omniroute/open-sse/handlers/jinaFoundation.ts";
import {
getProviderCredentialsWithQuotaPreflight,
clearRecoveredProviderState,
} from "@/sse/services/auth";
import { withInjectionGuard } from "@/middleware/promptInjectionGuard";
import { errorResponse } from "@omniroute/open-sse/utils/error.ts";
import { HTTP_STATUS } from "@omniroute/open-sse/config/constants.ts";
import { enforceApiKeyPolicy } from "@/shared/utils/apiKeyPolicy";
import { v1SegmentSchema } from "@/shared/validation/schemas";
import { isValidationFailure, validateBody } from "@/shared/validation/helpers";
import {
isAllRateLimitedCredentials,
rateLimitedProviderResponse,
} from "@/app/api/v1/_shared/rateLimit";
import { JINA_FOUNDATION_PROVIDER_ID, JINA_SEGMENT_BASE_URL } from "@/lib/providers/jina";
/**
* Handle CORS preflight
*/
export async function OPTIONS() {
return new Response(null, {
headers: {
"Access-Control-Allow-Methods": "POST, OPTIONS",
"Access-Control-Allow-Headers": "*",
},
});
}
/**
* POST /v1/segment — Jina segmenter (tokenize / chunk).
*
* Proxies to https://segment.jina.ai/ using the same jina-ai credentials as
* embeddings and classify. Segment lives on a dedicated host; the UI card is
* still Foundation API, not Reader.
*/
async function postHandler(request: Request) {
let rawBody: unknown;
try {
rawBody = await request.json();
} catch {
return errorResponse(HTTP_STATUS.BAD_REQUEST, "Invalid JSON body");
}
const validation = validateBody(v1SegmentSchema, rawBody);
if (isValidationFailure(validation)) {
return errorResponse(HTTP_STATUS.BAD_REQUEST, validation.error.message);
}
const body = validation.data;
const policy = await enforceApiKeyPolicy(request, "jina-ai/segment");
if (policy.rejection) return policy.rejection;
const credentials = await getProviderCredentialsWithQuotaPreflight(JINA_FOUNDATION_PROVIDER_ID);
if (!credentials) {
return errorResponse(
HTTP_STATUS.BAD_REQUEST,
`No credentials for provider: ${JINA_FOUNDATION_PROVIDER_ID}`
);
}
if (isAllRateLimitedCredentials(credentials)) {
return rateLimitedProviderResponse(JINA_FOUNDATION_PROVIDER_ID, credentials);
}
const response = await handleJinaFoundationProxy({
path: "/v1/segment",
upstreamUrl: `${JINA_SEGMENT_BASE_URL}/`,
body,
credentials,
provider: JINA_FOUNDATION_PROVIDER_ID,
model: "segment",
});
if (response?.ok) {
await clearRecoveredProviderState(credentials);
}
return response;
}
export const POST = withInjectionGuard(postHandler);

View File

@@ -6094,8 +6094,8 @@
"inception": "Inception Labs is OpenAI-compatible at https://api.inceptionlabs.ai/v1. mercury-2 is the first diffusion LLM (dLLM) in the catalog — 5-10x faster generation than comparable autoregressive models, with tool calling, json_mode, and structured outputs.",
"inference-net": "$25 free credits on signup plus research grants available",
"internlm": "Free monthly quota ~1M input / 3M output tokens (~10 RPM)",
"jina-ai": "Bearer API key for the Jina AI rerank API.",
"jina-reader": "Connect Jina Reader with an API key.",
"jina-ai": "Bearer API key for api.jina.ai (embeddings, rerank, classify, segment, search). Not the Reader / r.jina.ai card. Dashboard keys take precedence over JINA_AI_API_KEY.",
"jina-reader": "Bearer API key for r.jina.ai URL-to-markdown only. Does not serve /v1/embeddings or /v1/rerank. The same Jina token as Foundation API works.",
"kenari": "Kenari exposes an OpenAI-compatible chat completions endpoint at https://kenari.id/v1/chat/completions, plus a live /v1/models catalog covering Claude, GPT, DeepSeek, GLM, Kimi and more. OmniRoute uses the OpenAI protocol and lists models via passthrough.",
"kie": "Connect KIE.AI with an API key.",
"kilo-gateway": "Connect Kilo Gateway with an API key.",

View File

@@ -311,6 +311,7 @@ export async function createEmbeddingResponse(
connectionId:
((credentials as { connectionId?: string } | null)?.connectionId) ||
options.connectionId ||
connectionIdForProxy ||
null,
});

View File

@@ -0,0 +1,86 @@
/**
* Shared Gemini (Google AI Studio) integration helpers.
*
* Dashboard `gemini` connections stay preferred. GEMINI_API_KEY /
* GOOGLE_API_KEY are a headless fallback when no usable dashboard key
* exists — the same class of bug as unused JINA_AI_API_KEY. Do not treat
* the env as billed if a dashboard connection is selected (fill-first).
*/
export const GEMINI_PROVIDER_ID = "gemini";
/** Call-log / credential sentinel when the request used the process env key. */
export const GEMINI_ENV_CONNECTION_ID = "env:GEMINI_API_KEY";
export const GEMINI_ENV_API_KEY_NAMES = ["GEMINI_API_KEY", "GOOGLE_API_KEY"] as const;
export interface GeminiEnvCredentials {
apiKey: string;
accessToken: null;
connectionId: typeof GEMINI_ENV_CONNECTION_ID;
id: typeof GEMINI_ENV_CONNECTION_ID;
provider: string;
authType: "apikey";
defaultModel: null;
}
export function isGeminiCredentialProvider(providerId: string | null | undefined): boolean {
return providerId === GEMINI_PROVIDER_ID;
}
/**
* Read the first non-empty Gemini env key. Dashboard connections always win
* when getProviderCredentials finds one.
*/
export function readGeminiEnvApiKey(): string | null {
for (const name of GEMINI_ENV_API_KEY_NAMES) {
const value = process.env[name]?.trim();
if (value) return value;
}
return null;
}
/**
* Synthetic credentials for headless / Docker operators who inject
* GEMINI_API_KEY (or GOOGLE_API_KEY) instead of adding a dashboard connection.
*/
export function buildGeminiEnvCredentials(
providerId: string,
options: {
forcedConnectionId?: string | null;
allowedConnections?: string[] | null;
excludedConnectionIds?: Iterable<string> | null;
} = {}
): GeminiEnvCredentials | null {
if (!isGeminiCredentialProvider(providerId)) return null;
const forced =
typeof options.forcedConnectionId === "string" && options.forcedConnectionId.trim().length > 0
? options.forcedConnectionId.trim()
: null;
if (forced && forced !== GEMINI_ENV_CONNECTION_ID) return null;
const allowed = options.allowedConnections;
if (Array.isArray(allowed) && allowed.length > 0 && !allowed.includes(GEMINI_ENV_CONNECTION_ID)) {
return null;
}
if (options.excludedConnectionIds) {
for (const excluded of options.excludedConnectionIds) {
if (excluded === GEMINI_ENV_CONNECTION_ID) return null;
}
}
const apiKey = readGeminiEnvApiKey();
if (!apiKey) return null;
return {
apiKey,
accessToken: null,
connectionId: GEMINI_ENV_CONNECTION_ID,
id: GEMINI_ENV_CONNECTION_ID,
provider: providerId,
authType: "apikey",
defaultModel: null,
};
}

103
src/lib/providers/jina.ts Normal file
View File

@@ -0,0 +1,103 @@
/**
* Shared Jina AI integration helpers.
*
* OmniRoute keeps two dashboard cards because the hosts differ:
* - jina-ai Foundation API https://api.jina.ai
* - jina-reader Reader https://r.jina.ai
*
* One Jina token works on both hosts. Dashboard connections stay preferred;
* JINA_AI_API_KEY / JINA_API_KEY are a headless fallback when no usable
* dashboard key exists. Do not treat the env as billed if a dashboard
* connection is selected (fill-first, priority ascending).
*/
export const JINA_FOUNDATION_PROVIDER_ID = "jina-ai";
export const JINA_READER_PROVIDER_ID = "jina-reader";
export const JINA_SEARCH_PROVIDER_ID = "jina-search";
export const JINA_FOUNDATION_BASE_URL = "https://api.jina.ai";
export const JINA_READER_BASE_URL = "https://r.jina.ai";
export const JINA_SEARCH_BASE_URL = "https://s.jina.ai";
export const JINA_SEGMENT_BASE_URL = "https://segment.jina.ai";
/** Call-log / credential sentinel when the request used the process env key. */
export const JINA_ENV_CONNECTION_ID = "env:JINA_AI_API_KEY";
export const JINA_ENV_API_KEY_NAMES = ["JINA_AI_API_KEY", "JINA_API_KEY"] as const;
const JINA_CREDENTIAL_PROVIDERS = new Set<string>([
JINA_FOUNDATION_PROVIDER_ID,
JINA_READER_PROVIDER_ID,
JINA_SEARCH_PROVIDER_ID,
]);
export interface JinaEnvCredentials {
apiKey: string;
accessToken: null;
connectionId: typeof JINA_ENV_CONNECTION_ID;
id: typeof JINA_ENV_CONNECTION_ID;
provider: string;
authType: "apikey";
defaultModel: null;
}
export function isJinaCredentialProvider(providerId: string | null | undefined): boolean {
return typeof providerId === "string" && JINA_CREDENTIAL_PROVIDERS.has(providerId);
}
/**
* Read the first non-empty Jina env key. Dashboard connections always win
* when getProviderCredentials finds one.
*/
export function readJinaEnvApiKey(): string | null {
for (const name of JINA_ENV_API_KEY_NAMES) {
const value = process.env[name]?.trim();
if (value) return value;
}
return null;
}
/**
* Synthetic credentials for headless / Docker operators who inject
* JINA_AI_API_KEY instead of adding a dashboard connection.
*/
export function buildJinaEnvCredentials(
providerId: string,
options: {
forcedConnectionId?: string | null;
allowedConnections?: string[] | null;
excludedConnectionIds?: Iterable<string> | null;
} = {}
): JinaEnvCredentials | null {
if (!isJinaCredentialProvider(providerId)) return null;
const forced =
typeof options.forcedConnectionId === "string" && options.forcedConnectionId.trim().length > 0
? options.forcedConnectionId.trim()
: null;
if (forced && forced !== JINA_ENV_CONNECTION_ID) return null;
const allowed = options.allowedConnections;
if (Array.isArray(allowed) && allowed.length > 0 && !allowed.includes(JINA_ENV_CONNECTION_ID)) {
return null;
}
if (options.excludedConnectionIds) {
for (const excluded of options.excludedConnectionIds) {
if (excluded === JINA_ENV_CONNECTION_ID) return null;
}
}
const apiKey = readJinaEnvApiKey();
if (!apiKey) return null;
return {
apiKey,
accessToken: null,
connectionId: JINA_ENV_CONNECTION_ID,
id: JINA_ENV_CONNECTION_ID,
provider: providerId,
authType: "apikey",
defaultModel: null,
};
}

View File

@@ -1,5 +1,4 @@
import { getEmbeddingProvider } from "@omniroute/open-sse/config/embeddingRegistry.ts";
import { getRerankProvider } from "@omniroute/open-sse/config/rerankRegistry.ts";
import { getRegistryEntry } from "@omniroute/open-sse/config/providerRegistry.ts";
import {
isClaudeCodeCompatibleProvider,
@@ -86,6 +85,7 @@ import { validateSearchProvider, SEARCH_VALIDATOR_CONFIGS } from "./validation/s
import {
validateClarifaiProvider,
validateEmbeddingApiProvider,
validateJinaFoundationProvider,
validateRerankApiProvider,
} from "./validation/embeddingProviders";
import {
@@ -281,15 +281,8 @@ export async function validateProviderApiKey({ provider, apiKey, providerSpecifi
modelId: embeddingProvider?.models?.[0]?.id || "voyage-4-lite",
});
},
"jina-ai": ({ apiKey, providerSpecificData }: any) => {
const rerankProvider = getRerankProvider("jina-ai");
return validateRerankApiProvider({
apiKey,
providerSpecificData,
url: rerankProvider?.baseUrl,
modelId: rerankProvider?.models?.[0]?.id || "jina-reranker-v3",
});
},
"jina-ai": ({ apiKey, providerSpecificData }: any) =>
validateJinaFoundationProvider({ apiKey, providerSpecificData }),
gitlab: ({ apiKey, providerSpecificData }: any) =>
validateGitlabProvider({ apiKey, providerSpecificData, isLocal }),
vertex: validateVertexProvider,

View File

@@ -101,6 +101,107 @@ export async function validateEmbeddingApiProvider({
}
}
/**
* Jina Foundation API key probe.
*
* Dashboard Test used to POST rerank with jina-reranker-v3, which can 200
* while production Omni embed / rerank-v3.5 403. Prefer GET /v1/models
* (key validity). Embeddings fallback hits jina-embeddings-v5-omni-small
* so Test exercises the Omni SKU, not a text-only stand-in. Always report
* the endpoint and model that were hit.
*/
export async function validateJinaFoundationProvider({
apiKey,
providerSpecificData = {},
}: {
apiKey: string;
providerSpecificData?: { validationModelId?: string; [key: string]: unknown };
}) {
const modelsUrl = "https://api.jina.ai/v1/models";
const embeddingsUrl = "https://api.jina.ai/v1/embeddings";
const embeddingsModel =
providerSpecificData?.validationModelId || "jina-embeddings-v5-omni-small";
try {
const modelsRes = await validationRead(modelsUrl, {
method: "GET",
headers: buildBearerHeaders(apiKey, providerSpecificData),
});
if (modelsRes.ok) {
return {
valid: true,
error: null,
method: "jina_models",
testedEndpoint: "GET https://api.jina.ai/v1/models",
};
}
if (modelsRes.status === 401 || modelsRes.status === 403) {
return {
valid: false,
error: `Invalid API key (GET https://api.jina.ai/v1/models)`,
method: "jina_models",
testedEndpoint: "GET https://api.jina.ai/v1/models",
};
}
const embedRes = await validationWrite(embeddingsUrl, {
method: "POST",
headers: buildBearerHeaders(apiKey, providerSpecificData),
body: JSON.stringify({
model: embeddingsModel,
input: ["test"],
}),
});
if (embedRes.status === 401 || embedRes.status === 403) {
return {
valid: false,
error: `Invalid API key (POST https://api.jina.ai/v1/embeddings model=${embeddingsModel})`,
method: "jina_embeddings",
testedEndpoint: "POST https://api.jina.ai/v1/embeddings",
testedModel: embeddingsModel,
};
}
if (
embedRes.ok ||
embedRes.status === 400 ||
embedRes.status === 422 ||
embedRes.status === 429
) {
return {
valid: true,
error: null,
method: "jina_embeddings",
testedEndpoint: "POST https://api.jina.ai/v1/embeddings",
testedModel: embeddingsModel,
};
}
if (embedRes.status >= 500) {
return {
valid: false,
error: `Provider unavailable (${embedRes.status}) at POST https://api.jina.ai/v1/embeddings model=${embeddingsModel}`,
method: "jina_embeddings",
testedEndpoint: "POST https://api.jina.ai/v1/embeddings",
testedModel: embeddingsModel,
};
}
return {
valid: false,
error: `Validation failed: ${embedRes.status} (POST https://api.jina.ai/v1/embeddings model=${embeddingsModel})`,
method: "jina_embeddings",
testedEndpoint: "POST https://api.jina.ai/v1/embeddings",
testedModel: embeddingsModel,
};
} catch (error: unknown) {
return toValidationErrorResult(error);
}
}
export async function validateRerankApiProvider({ apiKey, providerSpecificData = {}, url, modelId }: any) {
if (!url) {
return { valid: false, error: "Missing rerank endpoint" };

View File

@@ -4,6 +4,7 @@ import * as defaultLog from "@/sse/utils/logger";
import {
getAllSearchProviders,
getSearchProvider,
resolveSearchProvider,
selectProvider,
supportsSearchType,
SEARCH_CREDENTIAL_FALLBACKS,
@@ -121,7 +122,7 @@ export async function executeWebSearch(
const searchType = input.search_type || "web";
if (input.provider) {
const explicitProvider = getSearchProvider(input.provider);
const explicitProvider = resolveSearchProvider(input.provider);
if (!explicitProvider) {
throw new WebSearchExecutionError(`Unknown search provider: ${input.provider}`, 400);
}

View File

@@ -152,12 +152,13 @@ export const APIKEY_PROVIDERS_SPECIALTY = {
"jina-ai": {
id: "jina-ai",
alias: "jina",
name: "Jina AI",
name: "Jina AI (Foundation API)",
icon: "sort",
color: "#2563EB",
textIcon: "JA",
website: "https://jina.ai",
authHint: "Bearer API key for the Jina AI rerank API.",
authHint:
"Bearer API key for api.jina.ai — embeddings, rerank, classify, segment, and search. Dashboard keys take precedence over JINA_AI_API_KEY. This is not the Reader / r.jina.ai card and does not fetch URLs.",
hasFree: true,
freeNote: "10M free tokens on signup (non-commercial), no credit card required",
},
@@ -262,14 +263,16 @@ export const APIKEY_PROVIDERS_SPECIALTY = {
"jina-reader": {
id: "jina-reader",
alias: "jr",
name: "Jina Reader",
name: "Jina Reader (r.jina.ai)",
icon: "menu_book",
color: "#0EA5E9",
textIcon: "JR",
website: "https://jina.ai/reader",
authHint:
"Bearer API key for r.jina.ai URL-to-markdown (/v1/web/fetch only). Does not serve /v1/embeddings or /v1/rerank. The same Jina token as Foundation API works; OmniRoute reuses a jina-ai dashboard key or JINA_AI_API_KEY when this card is empty.",
hasFree: true,
notice: {
text: "Free tier: 1M fetches/month.",
text: "Reader / r.jina.ai only — not embeddings or rerank. Free tier: 1M fetches/month.",
apiKeyUrl: "https://jina.ai/api-dashboard",
},
serviceKinds: ["webFetch"],

View File

@@ -0,0 +1,126 @@
/**
* Gemini Embedding 2 native items (Google AI Studio embedContent / batchEmbedContents).
*
* Official 2026 contract (ai.google.dev/gemini-api/docs/embeddings):
* - Model id: gemini-embedding-2 (GA April 2026). Legacy text-only: gemini-embedding-001.
* - One Content (parts[]) → one embedding. Multiple parts in one Content fuse.
* - N Content objects / N batchEmbedContents requests → N embeddings.
* - Parts: { text }, { inline_data: { mime_type, data } }, { file_data: { mime_type, file_uri } }.
* CamelCase SDK spellings (inlineData / fileData) are accepted and forwarded.
*
* These are not OmniRoute's canonical `{ type, source }` items. For gemini
* they must reach generativelanguage.googleapis.com as Content parts — do
* not collapse the OpenAI `input` array to string[].
*/
import { isCanonicalEmbeddingItem, isPlainObject } from "./jinaNativeEmbeddingInput";
export type GeminiEmbeddingModality = "text" | "image" | "audio" | "video" | "document";
const GEMINI_EMBEDDING_2_IDS = new Set(["gemini-embedding-2", "gemini-embedding-2-preview"]);
export function isGeminiEmbedding2Family(modelId: string | null | undefined): boolean {
return typeof modelId === "string" && GEMINI_EMBEDDING_2_IDS.has(modelId);
}
function asRecord(value: unknown): Record<string, unknown> | null {
return isPlainObject(value) ? value : null;
}
function mimeFromInline(value: Record<string, unknown>): string | null {
const snake = asRecord(value.inline_data);
if (typeof snake?.mime_type === "string") return snake.mime_type;
const camel = asRecord(value.inlineData);
if (typeof camel?.mimeType === "string") return camel.mimeType;
return null;
}
function mimeFromFile(value: Record<string, unknown>): string | null {
const snake = asRecord(value.file_data);
if (typeof snake?.mime_type === "string") return snake.mime_type;
const camel = asRecord(value.fileData);
if (typeof camel?.mimeType === "string") return camel.mimeType;
return null;
}
export function modalityFromGeminiMime(mimeType: string): GeminiEmbeddingModality {
const mime = mimeType.trim().toLowerCase();
if (mime.startsWith("image/")) return "image";
if (mime.startsWith("audio/")) return "audio";
if (mime.startsWith("video/")) return "video";
if (mime === "application/pdf" || mime.startsWith("application/pdf")) return "document";
return "document";
}
export function isGeminiNativePart(value: unknown): boolean {
const record = asRecord(value);
if (!record || isCanonicalEmbeddingItem(record)) return false;
if (typeof record.text === "string" && record.text.trim().length > 0) {
return !("image" in record) && !("audio" in record) && !("video" in record) && !("pdf" in record);
}
if (asRecord(record.inline_data)?.data || asRecord(record.inlineData)?.data) return true;
if (asRecord(record.file_data)?.file_uri || asRecord(record.fileData)?.fileUri) return true;
return false;
}
export function isGeminiNativeContent(value: unknown): boolean {
const record = asRecord(value);
if (!record || isCanonicalEmbeddingItem(record)) return false;
if (!Array.isArray(record.parts) || record.parts.length === 0) return false;
return record.parts.every((part) => isGeminiNativePart(part));
}
export function isGeminiNativeEmbedRequest(value: unknown): boolean {
const record = asRecord(value);
if (!record || isCanonicalEmbeddingItem(record)) return false;
const content = record.content;
if (Array.isArray(content)) return false;
return isGeminiNativeContent(content);
}
export function isGeminiNativeEmbeddingItem(value: unknown): boolean {
return isGeminiNativePart(value) || isGeminiNativeContent(value) || isGeminiNativeEmbedRequest(value);
}
/**
* True when the request already uses Gemini's documented multimodal contract
* (a part, a Content with parts, or an EmbedContentRequest).
*/
export function isGeminiNativeEmbeddingInput(input: unknown): boolean {
if (isGeminiNativeEmbeddingItem(input)) return true;
if (!Array.isArray(input)) return false;
return input.some((item) => isGeminiNativeEmbeddingItem(item));
}
export function collectGeminiNativeModalities(input: unknown): GeminiEmbeddingModality[] {
const found = new Set<GeminiEmbeddingModality>();
const visitPart = (value: unknown) => {
const record = asRecord(value);
if (!record) return;
if (typeof record.text === "string" && record.text.trim().length > 0) found.add("text");
const inlineMime = mimeFromInline(record);
if (inlineMime) found.add(modalityFromGeminiMime(inlineMime));
const fileMime = mimeFromFile(record);
if (fileMime) found.add(modalityFromGeminiMime(fileMime));
};
const visit = (value: unknown) => {
if (isGeminiNativeEmbedRequest(value)) {
visit((value as { content: unknown }).content);
return;
}
if (isGeminiNativeContent(value)) {
for (const part of (value as { parts: unknown[] }).parts) visitPart(part);
return;
}
if (isGeminiNativePart(value)) visitPart(value);
};
if (Array.isArray(input)) {
for (const item of input) visit(item);
} else {
visit(input);
}
return [...found];
}

View File

@@ -0,0 +1,87 @@
/**
* Jina Search Foundation native embedding items (api.jina.ai EmbeddingsV5Request).
*
* Official 2026 input shapes (OpenAPI 2026.07.27):
* TextDoc { text }
* ImageDoc { image } URL or base64 / data URI
* AudioDoc { audio }
* VideoDoc { video }
* PDFDoc { pdf } single input only upstream; we still accept it in a list
* MergedContentGroup { content: [TextDoc|ImageDoc|AudioDoc|VideoDoc, ...] }
*
* These are not OmniRoute's canonical `{ type, source }` items. For jina-ai
* they must be forwarded intact — do not stringify, do not fetch image URLs
* into data URIs. Jina fetches public media itself.
*/
export const JINA_NATIVE_MEDIA_KEYS = ["text", "image", "audio", "video", "pdf"] as const;
export type JinaNativeMediaKey = (typeof JINA_NATIVE_MEDIA_KEYS)[number];
const NATIVE_KEY_TO_MODALITY: Record<JinaNativeMediaKey, "text" | "image" | "audio" | "video" | "document"> =
{
text: "text",
image: "image",
audio: "audio",
video: "video",
pdf: "document",
};
export function isPlainObject(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null && !Array.isArray(value);
}
/** OmniRoute canonical structured item — leave those on the translator path. */
export function isCanonicalEmbeddingItem(value: unknown): boolean {
return isPlainObject(value) && "type" in value && typeof value.type === "string";
}
export function isJinaNativeDoc(value: unknown): boolean {
if (!isPlainObject(value) || isCanonicalEmbeddingItem(value)) return false;
if ("content" in value && Array.isArray(value.content)) return false;
const present = JINA_NATIVE_MEDIA_KEYS.filter((key) => key in value);
if (present.length !== 1) return false;
return typeof value[present[0]] === "string" && String(value[present[0]]).trim().length > 0;
}
export function isJinaMergedContentGroup(value: unknown): boolean {
if (!isPlainObject(value) || isCanonicalEmbeddingItem(value)) return false;
if (!Array.isArray(value.content) || value.content.length === 0) return false;
return value.content.every((item) => isJinaNativeDoc(item) && !("pdf" in (item as object)));
}
export function isJinaNativeEmbeddingItem(value: unknown): boolean {
return isJinaNativeDoc(value) || isJinaMergedContentGroup(value);
}
/**
* True when the request already uses Jina's documented multimodal contract
* (single doc, mixed string+doc batch, or fused content groups).
*/
export function isJinaNativeEmbeddingInput(input: unknown): boolean {
if (isJinaNativeEmbeddingItem(input)) return true;
if (!Array.isArray(input)) return false;
return input.some((item) => isJinaNativeEmbeddingItem(item));
}
export function collectJinaNativeModalities(
input: unknown
): Array<"text" | "image" | "audio" | "video" | "document"> {
const found = new Set<"text" | "image" | "audio" | "video" | "document">();
const visit = (value: unknown) => {
if (isJinaMergedContentGroup(value)) {
for (const item of (value as { content: unknown[] }).content) visit(item);
return;
}
if (!isJinaNativeDoc(value)) return;
const key = JINA_NATIVE_MEDIA_KEYS.find((mediaKey) => mediaKey in (value as object));
if (key) found.add(NATIVE_KEY_TO_MODALITY[key]);
};
if (Array.isArray(input)) {
for (const item of input) visit(item);
} else {
visit(input);
}
return [...found];
}

View File

@@ -20,6 +20,11 @@ import {
} from "@/shared/reasoning/effortStandardization";
import { modelIdSchema, nonEmptyStringSchema } from "./misc.ts";
import {
isCanonicalEmbeddingItem,
JINA_NATIVE_MEDIA_KEYS,
} from "../jinaNativeEmbeddingInput.ts";
import { isGeminiNativeEmbeddingItem } from "../geminiNativeEmbeddingInput.ts";
export const embeddingTokenArraySchema = z
.array(z.number().int().min(0))
@@ -110,15 +115,260 @@ export const embeddingMultimodalItemSchema = z.discriminatedUnion("type", [
),
]);
function decodedInlineBytesFromEmbeddingItem(item: unknown): number {
if (!item || typeof item !== "object") return 0;
const record = item as Record<string, unknown>;
if (
"type" in record &&
record.type !== "text" &&
record.source &&
typeof record.source === "object"
) {
const source = record.source as { type?: string; data?: string };
if (source.type === "base64" && typeof source.data === "string") {
return decodedBase64Bytes(source.data);
}
}
for (const key of JINA_NATIVE_MEDIA_KEYS) {
if (key === "text" || typeof record[key] !== "string") continue;
const value = String(record[key]);
const dataUri = /^data:([^;,]+);base64,(.+)$/i.exec(value);
if (dataUri) return decodedBase64Bytes(dataUri[2]);
if (/^https:\/\//i.test(value)) return 0;
return decodedBase64Bytes(value);
}
if (Array.isArray(record.content)) {
return record.content.reduce(
(total, chunk) => total + decodedInlineBytesFromEmbeddingItem(chunk),
0
);
}
if (record.content && typeof record.content === "object" && !Array.isArray(record.content)) {
return decodedInlineBytesFromEmbeddingItem(record.content);
}
if (Array.isArray(record.parts)) {
return record.parts.reduce(
(total, chunk) => total + decodedInlineBytesFromEmbeddingItem(chunk),
0
);
}
const inline = record.inline_data ?? record.inlineData;
if (inline && typeof inline === "object") {
const data = (inline as { data?: unknown }).data;
if (typeof data === "string") return decodedBase64Bytes(data);
}
return 0;
}
const embeddingMultimodalInputSchema = z
.array(embeddingMultimodalItemSchema)
.min(1, "input must contain at least one item")
.max(MAX_EMBEDDING_INPUT_ITEMS, `input must contain at most ${MAX_EMBEDDING_INPUT_ITEMS} items`)
.superRefine((items, context) => {
const totalBytes = items.reduce((total, item) => {
if (item.type === "text" || item.source.type !== "base64") return total;
return total + decodedBase64Bytes(item.source.data);
}, 0);
const totalBytes = items.reduce(
(total, item) => total + decodedInlineBytesFromEmbeddingItem(item),
0
);
if (totalBytes > MAX_EMBEDDING_INLINE_TOTAL_BYTES) {
context.addIssue({
code: "custom",
message: "decoded inline media must not exceed 16 MiB per request",
});
}
});
function refineJinaMediaString(value: string, context: z.RefinementCtx) {
const trimmed = value.trim();
if (/^https:\/\//i.test(trimmed)) {
if (trimmed.length > MAX_EMBEDDING_URL_LENGTH) {
context.addIssue({ code: "custom", message: "media URL is too long" });
return;
}
try {
const url = parseAndValidatePublicUrl(trimmed);
if (url.protocol !== "https:") {
context.addIssue({ code: "custom", message: "media URLs must use HTTPS" });
}
} catch {
context.addIssue({ code: "custom", message: "media URL must be a safe public HTTPS URL" });
}
return;
}
if (/^(https?:|file:|data:text\/html)/i.test(trimmed) && !trimmed.startsWith("data:")) {
context.addIssue({ code: "custom", message: "media URL must be a safe public HTTPS URL" });
return;
}
const dataUri = /^data:([^;,]+);base64,(.+)$/i.exec(trimmed);
const payload = dataUri ? dataUri[2] : trimmed;
if (
payload.length > MAX_EMBEDDING_INLINE_ITEM_BASE64_LENGTH ||
decodedBase64Bytes(payload) > MAX_EMBEDDING_INLINE_ITEM_BYTES
) {
context.addIssue({
code: "custom",
message: "decoded inline media must not exceed 8 MiB",
});
}
}
const jinaNativeMediaStringSchema = z.string().trim().min(1).superRefine(refineJinaMediaString);
function exactlyOneJinaMediaKey(value: Record<string, unknown>, key: string): boolean {
if (isCanonicalEmbeddingItem(value)) return false;
return JINA_NATIVE_MEDIA_KEYS.filter((mediaKey) => mediaKey in value).length === 1 && key in value;
}
const jinaTextDocSchema = z
.object({ text: z.string().trim().min(1).max(MAX_EMBEDDING_TEXT_LENGTH) })
.passthrough()
.refine((value) => exactlyOneJinaMediaKey(value, "text"), {
message: "Jina TextDoc must be { text }",
});
const jinaImageDocSchema = z
.object({ image: jinaNativeMediaStringSchema })
.passthrough()
.refine((value) => exactlyOneJinaMediaKey(value, "image"), {
message: "Jina ImageDoc must be { image }",
});
const jinaAudioDocSchema = z
.object({ audio: jinaNativeMediaStringSchema })
.passthrough()
.refine((value) => exactlyOneJinaMediaKey(value, "audio"), {
message: "Jina AudioDoc must be { audio }",
});
const jinaVideoDocSchema = z
.object({ video: jinaNativeMediaStringSchema })
.passthrough()
.refine((value) => exactlyOneJinaMediaKey(value, "video"), {
message: "Jina VideoDoc must be { video }",
});
const jinaPdfDocSchema = z
.object({ pdf: jinaNativeMediaStringSchema })
.passthrough()
.refine((value) => exactlyOneJinaMediaKey(value, "pdf"), {
message: "Jina PDFDoc must be { pdf }",
});
export const jinaNativeDocSchema = z.union([
jinaTextDocSchema,
jinaImageDocSchema,
jinaAudioDocSchema,
jinaVideoDocSchema,
jinaPdfDocSchema,
]);
export const jinaMergedContentGroupSchema = z
.object({
content: z
.array(z.union([jinaTextDocSchema, jinaImageDocSchema, jinaAudioDocSchema, jinaVideoDocSchema]))
.min(1, "content must contain at least one chunk"),
})
.passthrough();
const geminiInlineBlobSchema = z
.object({
mime_type: z.string().trim().min(1).max(MAX_MEDIA_TYPE_LENGTH).optional(),
mimeType: z.string().trim().min(1).max(MAX_MEDIA_TYPE_LENGTH).optional(),
data: z.string().min(1),
})
.passthrough()
.superRefine((value, context) => {
if (!value.mime_type && !value.mimeType) {
context.addIssue({ code: "custom", message: "Gemini inline_data requires mime_type" });
}
const data = value.data;
if (
data.length > MAX_EMBEDDING_INLINE_ITEM_BASE64_LENGTH ||
decodedBase64Bytes(data) > MAX_EMBEDDING_INLINE_ITEM_BYTES
) {
context.addIssue({
code: "custom",
message: "decoded inline media must not exceed 8 MiB",
});
}
});
const geminiFileUriSchema = z
.string()
.trim()
.min(1)
.max(MAX_EMBEDDING_URL_LENGTH)
.superRefine((value, context) => {
if (value.startsWith("files/")) return;
try {
const url = parseAndValidatePublicUrl(value);
if (url.protocol !== "https:") {
context.addIssue({ code: "custom", message: "media URLs must use HTTPS" });
}
} catch {
context.addIssue({ code: "custom", message: "media URL must be a safe public HTTPS URL" });
}
});
const geminiFileDataSchema = z
.object({
mime_type: z.string().trim().min(1).max(MAX_MEDIA_TYPE_LENGTH).optional(),
mimeType: z.string().trim().min(1).max(MAX_MEDIA_TYPE_LENGTH).optional(),
file_uri: geminiFileUriSchema.optional(),
fileUri: geminiFileUriSchema.optional(),
})
.passthrough()
.refine((value) => Boolean(value.file_uri || value.fileUri), {
message: "Gemini file_data requires file_uri",
});
export const geminiNativePartSchema = z
.object({
text: z.string().trim().min(1).max(MAX_EMBEDDING_TEXT_LENGTH).optional(),
inline_data: geminiInlineBlobSchema.optional(),
inlineData: geminiInlineBlobSchema.optional(),
file_data: geminiFileDataSchema.optional(),
fileData: geminiFileDataSchema.optional(),
})
.passthrough()
.refine((value) => isGeminiNativeEmbeddingItem(value) && !("parts" in value) && !("content" in value), {
message: "Gemini part must be { text }, { inline_data }, or { file_data }",
});
export const geminiNativeContentSchema = z
.object({
parts: z.array(geminiNativePartSchema).min(1, "parts must contain at least one part"),
})
.passthrough();
export const geminiNativeEmbedRequestSchema = z
.object({
content: geminiNativeContentSchema,
})
.passthrough();
export const geminiNativeItemSchema = z.union([
geminiNativePartSchema,
geminiNativeContentSchema,
geminiNativeEmbedRequestSchema,
]);
const jinaNativeOrCanonicalArraySchema = z
.array(
z.union([
nonEmptyStringSchema,
embeddingMultimodalItemSchema,
jinaNativeDocSchema,
jinaMergedContentGroupSchema,
geminiNativeItemSchema,
])
)
.min(1, "input must contain at least one item")
.max(MAX_EMBEDDING_INPUT_ITEMS, `input must contain at most ${MAX_EMBEDDING_INPUT_ITEMS} items`)
.superRefine((items, context) => {
const totalBytes = items.reduce(
(total, item) => total + decodedInlineBytesFromEmbeddingItem(item),
0
);
if (totalBytes > MAX_EMBEDDING_INLINE_TOTAL_BYTES) {
context.addIssue({
code: "custom",
@@ -133,6 +383,10 @@ export const embeddingInputSchema = z.union([
embeddingTokenArraySchema,
z.array(embeddingTokenArraySchema).min(1, "input must contain at least one item"),
embeddingMultimodalInputSchema,
jinaNativeDocSchema,
jinaMergedContentGroupSchema,
geminiNativeItemSchema,
jinaNativeOrCanonicalArraySchema,
]);
export type EmbeddingMultimodalItem = z.infer<typeof embeddingMultimodalItemSchema>;
@@ -244,6 +498,30 @@ export const v1RerankSchema = z
})
.catchall(z.unknown());
// POST /v1/classify — Jina zero/few-shot classification (api.jina.ai).
export const v1ClassifySchema = z
.object({
model: modelIdSchema.optional(),
classifier_id: z.string().trim().min(1).optional(),
input: z.union([
nonEmptyStringSchema,
z.array(z.unknown()).min(1, "input must contain at least one item"),
]),
labels: z.array(z.string().trim().min(1)).min(1).optional(),
})
.catchall(z.unknown());
// POST /v1/segment — Jina segmenter (segment.jina.ai).
export const v1SegmentSchema = z
.object({
content: nonEmptyStringSchema,
tokenizer: z.string().trim().min(1).optional(),
return_tokens: z.boolean().optional(),
return_chunks: z.boolean().optional(),
max_chunk_length: z.coerce.number().positive().optional(),
})
.catchall(z.unknown());
export const providerChatCompletionSchema = z
.object({
model: modelIdSchema,
@@ -305,6 +583,9 @@ export const v1SearchSchema = z
"youcom-search",
"searxng-search",
"zai-search",
"jina-search",
"jina-ai",
"jina",
"duckduckgo-free",
])
.optional(),

View File

@@ -15,6 +15,8 @@ import {
} from "@/lib/db/providers";
import { validateApiKey } from "@/lib/db/apiKeys";
import { getSettings } from "@/lib/db/settings";
import { buildJinaEnvCredentials } from "@/lib/providers/jina";
import { buildGeminiEnvCredentials } from "@/lib/providers/gemini";
import { toNumber } from "@/shared/utils/numeric";
import {
createLazyConnectionView,
@@ -969,6 +971,12 @@ const PROVIDER_SEARCH_PAIRS: string[][] = [
// The model layer canonicalizes `agy/` to `antigravity`, but the Antigravity
// CLI card stores its connection under `agy`. Same account, either id serves.
["antigravity", "agy"],
// One Jina token works on api.jina.ai, r.jina.ai, and s.jina.ai.
// Requested id stays first so embed/rerank do not silently pick a
// Reader-only row when both cards are filled. jina-search has no
// dashboard card — it must still see jina-ai / jina-reader keys
// before falling through to JINA_AI_API_KEY.
["jina-ai", "jina-reader", "jina-search"],
];
/**
* Resolve provider aliases (e.g., nvidia -> nvidia_nim) for DB lookup
@@ -977,8 +985,8 @@ async function getProviderSearchPool(provider: string): Promise<string[]> {
const canonicalProvider = resolveProviderId(provider);
const canonicalAlias = getProviderAlias(canonicalProvider);
const pair = PROVIDER_SEARCH_PAIRS.find((aliases) => aliases.includes(provider));
if (pair) return pair[0] === provider ? pair : [pair[1], pair[0]];
const group = PROVIDER_SEARCH_PAIRS.find((aliases) => aliases.includes(provider));
if (group) return [provider, ...group.filter((id) => id !== provider)];
const searchPool = new Set([provider, canonicalProvider, canonicalAlias].filter(Boolean));
@@ -1287,6 +1295,24 @@ export async function getProviderCredentials(
allowedConnections
);
if (syntheticFallback) return syntheticFallback;
const jinaEnvCredentials = buildJinaEnvCredentials(resolvedId, {
forcedConnectionId,
allowedConnections,
excludedConnectionIds,
});
if (jinaEnvCredentials) {
log.info("AUTH", `${provider} | using ${jinaEnvCredentials.connectionId} env fallback`);
return jinaEnvCredentials;
}
const geminiEnvCredentials = buildGeminiEnvCredentials(resolvedId, {
forcedConnectionId,
allowedConnections,
excludedConnectionIds,
});
if (geminiEnvCredentials) {
log.info("AUTH", `${provider} | using ${geminiEnvCredentials.connectionId} env fallback`);
return geminiEnvCredentials;
}
log.warn("AUTH", `No credentials for ${provider}`);
return null;
}
@@ -1993,7 +2019,7 @@ export async function getProviderCredentialsWithQuotaPreflight(
if (legacyForceDisable) return credentials;
const hasConnectionOverrides = Object.keys(perConnectionWindowOverrides).length > 0;
const legacyForceEnable = isQuotaPreflightEnabled(credentials);
const legacyForceEnable = isQuotaPreflightEnabled(credentials as Record<string, unknown>);
if (
!hasConnectionOverrides &&
!providerHasDefaults &&
@@ -2029,10 +2055,15 @@ export async function getProviderCredentialsWithQuotaPreflight(
requestedModel && modelAwarePreflight ? { ...credentials, requestedModel } : credentials;
let preflight;
try {
preflight = await preflightQuota(provider, connectionId, preflightCredentials, {
resolveMinRemainingPercent,
resolveWarnRemainingPercent: () => warnThresholdPercent,
});
preflight = await preflightQuota(
provider,
connectionId,
preflightCredentials as Record<string, unknown>,
{
resolveMinRemainingPercent,
resolveWarnRemainingPercent: () => warnThresholdPercent,
}
);
} catch (error) {
selectedCredentials.releaseOAuthSession?.();
throw error;

View File

@@ -2,7 +2,7 @@
* Integration tests for GET /api/search/providers — extended catalog (F4).
*
* Tests:
* - Returns 18 items total (14 search + 4 fetch providers).
* - Returns 19 items total (15 search + 4 fetch providers).
* - Each item carries the correct `kind` field.
* - Status reflects actual DB credential state:
* - "configured" when an active, non-rate-limited connection exists.
@@ -48,10 +48,10 @@ const route = await import("../../src/app/api/search/providers/route.ts");
// Constants
// ---------------------------------------------------------------------------
// 14 search-kind providers: serper, brave, perplexity, exa, tavily, firecrawl,
// google-pse, linkup, searchapi, youcom, searxng, ollama, zai + duckduckgo-free
// (registry open-sse/config/searchRegistry.ts).
const EXPECTED_SEARCH_COUNT = 14;
// 15 search-kind providers: serper, brave, perplexity, exa, tavily, firecrawl,
// google-pse, linkup, searchapi, youcom, searxng, ollama, zai, jina-search +
// duckduckgo-free (registry open-sse/config/searchRegistry.ts).
const EXPECTED_SEARCH_COUNT = 15;
const EXPECTED_FETCH_COUNT = 4;
const EXPECTED_TOTAL = EXPECTED_SEARCH_COUNT + EXPECTED_FETCH_COUNT;
@@ -307,7 +307,7 @@ test("search-providers-catalog: fetch providers have correct metadata", async ()
);
const jina = fetchProviders.find((p: { id: string }) => p.id === "jina-reader");
assert.equal(jina.name, "Jina Reader");
assert.equal(jina.name, "Jina Reader (r.jina.ai)");
assert.equal(jina.costPerQuery, 0.0005);
assert.ok(jina.fetchFormats.includes("text"), "jina fetchFormats must include text");

View File

@@ -20,6 +20,8 @@ test("getEmbeddingDimension resolves known dimensions from the registry", () =>
assert.equal(getEmbeddingDimension("openai/text-embedding-3-large"), 3072);
assert.equal(getEmbeddingDimension("nebius/Qwen/Qwen3-Embedding-8B"), 4096);
assert.equal(getEmbeddingDimension("gemini/gemini-embedding-001"), 768);
assert.equal(getEmbeddingDimension("gemini/gemini-embedding-2"), 3072);
assert.equal(getEmbeddingDimension("google/gemini-embedding-2"), 3072);
// OpenRouter re-exports OpenAI ids under its own prefix at the same dimension.
assert.equal(getEmbeddingDimension("openrouter/openai/text-embedding-3-small"), 1536);
});

View File

@@ -64,6 +64,7 @@ test("voyage-ai and jina-ai rerank registries expose supported models", () => {
assert.ok(jina);
assert.equal(jina.baseUrl, "https://api.jina.ai/v1/rerank");
assert.ok(jina.models.some((model) => model.id === "jina-reranker-v3.5"));
assert.ok(jina.models.some((model) => model.id === "jina-reranker-v3"));
assert.ok(jina.models.some((model) => model.id === "jina-reranker-m0"));

View File

@@ -149,7 +149,7 @@ test("translates canonical items to Jina's modality-keyed request contract", asy
});
});
test("translates one canonical array to Gemini native embedContent parts", async () => {
test("translates N canonical items to Gemini batchEmbedContents (N vectors)", async () => {
const { prepareStructuredEmbeddingRequest } =
await import("../../open-sse/handlers/embeddingStructuredInput.ts");
const provider = {
@@ -186,17 +186,99 @@ test("translates one canonical array to Gemini native embedContent parts", async
);
assert.equal(
prepared.url,
"https://generativelanguage.googleapis.com/v1beta/models/gemini-embedding-2:embedContent"
"https://generativelanguage.googleapis.com/v1beta/models/gemini-embedding-2:batchEmbedContents"
);
assert.deepEqual(prepared.authHeader, { name: "x-goog-api-key", value: "gemini-key" });
assert.deepEqual(prepared.body, {
requests: [
{
model: "models/gemini-embedding-2",
content: { parts: [{ text: "caption" }] },
output_dimensionality: 1536,
task_type: "RETRIEVAL_QUERY",
},
{
model: "models/gemini-embedding-2",
content: { parts: [{ inline_data: { mime_type: "image/png", data: "aQ==" } }] },
output_dimensionality: 1536,
task_type: "RETRIEVAL_QUERY",
},
{
model: "models/gemini-embedding-2",
content: { parts: [{ inline_data: { mime_type: "audio/mpeg", data: "YQ==" } }] },
output_dimensionality: 1536,
task_type: "RETRIEVAL_QUERY",
},
{
model: "models/gemini-embedding-2",
content: { parts: [{ inline_data: { mime_type: "video/mp4", data: "dg==" } }] },
output_dimensionality: 1536,
task_type: "RETRIEVAL_QUERY",
},
{
model: "models/gemini-embedding-2",
content: { parts: [{ inline_data: { mime_type: "application/pdf", data: "cA==" } }] },
output_dimensionality: 1536,
task_type: "RETRIEVAL_QUERY",
},
],
});
assert.deepEqual(
prepared.normalizeResponse?.({
embeddings: [{ values: [0.1] }, { values: [0.2] }, { values: [0.3] }],
}),
{
object: "list",
data: [
{ object: "embedding", embedding: [0.1], index: 0 },
{ object: "embedding", embedding: [0.2], index: 1 },
{ object: "embedding", embedding: [0.3], index: 2 },
],
usage: { prompt_tokens: 0, total_tokens: 0 },
}
);
});
test("translates one fused Gemini Content to embedContent (one vector)", async () => {
const { prepareStructuredEmbeddingRequest } =
await import("../../open-sse/handlers/embeddingStructuredInput.ts");
const provider = {
id: "gemini",
baseUrl: "https://generativelanguage.googleapis.com/v1beta/openai/embeddings",
authType: "apikey",
authHeader: "bearer",
structuredInputProtocol: "gemini-embed-content" as const,
models: [],
};
const prepared = await prepareStructuredEmbeddingRequest(
provider,
"gemini-embedding-2",
{
input: {
parts: [
{ text: "caption" },
{ inline_data: { mime_type: "image/png", data: "aQ==" } },
],
},
dimensions: 1536,
task: "retrieval.query",
},
"gemini-key",
{
fetchMedia: async () => {
throw new Error("unexpected URL");
},
}
);
assert.equal(
prepared.url,
"https://generativelanguage.googleapis.com/v1beta/models/gemini-embedding-2:embedContent"
);
assert.deepEqual(prepared.body, {
content: {
parts: [
{ text: "caption" },
{ inline_data: { mime_type: "image/png", data: "aQ==" } },
{ inline_data: { mime_type: "audio/mpeg", data: "YQ==" } },
{ inline_data: { mime_type: "video/mp4", data: "dg==" } },
{ inline_data: { mime_type: "application/pdf", data: "cA==" } },
],
},
output_dimensionality: 1536,

View File

@@ -0,0 +1,310 @@
import test from "node:test";
import assert from "node:assert/strict";
import { mkdtempSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
process.env.DATA_DIR = mkdtempSync(join(tmpdir(), "omniroute-gemini-embed2-"));
import {
GEMINI_ENV_CONNECTION_ID,
buildGeminiEnvCredentials,
isGeminiCredentialProvider,
readGeminiEnvApiKey,
} from "../../src/lib/providers/gemini.ts";
import { parseEmbeddingModel, getEmbeddingDimension } from "../../open-sse/config/embeddingRegistry.ts";
import { v1EmbeddingsSchema } from "../../src/shared/validation/schemas/apiV1.ts";
import { handleEmbedding } from "../../open-sse/handlers/embeddings.ts";
const ENV_KEYS = ["GEMINI_API_KEY", "GOOGLE_API_KEY"] as const;
const savedEnv = Object.fromEntries(ENV_KEYS.map((key) => [key, process.env[key]]));
function restoreEnv() {
for (const key of ENV_KEYS) {
if (savedEnv[key] === undefined) delete process.env[key];
else process.env[key] = savedEnv[key];
}
}
test.afterEach(restoreEnv);
const PNG_B64 =
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg==";
const IMAGE_URL = "https://example.com/bike.png";
function batchEmbeddingResponse(count: number) {
return new Response(
JSON.stringify({
embeddings: Array.from({ length: count }, (_, index) => ({
values: [0.1 * (index + 1), 0.2],
})),
}),
{ status: 200, headers: { "content-type": "application/json" } }
);
}
function singleEmbeddingResponse() {
return new Response(JSON.stringify({ embedding: { values: [0.1, 0.2] } }), {
status: 200,
headers: { "content-type": "application/json" },
});
}
test("Gemini env helper prefers GEMINI_API_KEY over GOOGLE_API_KEY", () => {
delete process.env.GEMINI_API_KEY;
delete process.env.GOOGLE_API_KEY;
process.env.GOOGLE_API_KEY = "alias-key";
assert.equal(readGeminiEnvApiKey(), "alias-key");
process.env.GEMINI_API_KEY = "primary-key";
assert.equal(readGeminiEnvApiKey(), "primary-key");
});
test("Gemini env credentials are scoped to gemini and honor filters", () => {
process.env.GEMINI_API_KEY = "env-gemini-key";
assert.equal(isGeminiCredentialProvider("gemini"), true);
assert.equal(isGeminiCredentialProvider("google"), false);
assert.equal(isGeminiCredentialProvider("jina-ai"), false);
assert.equal(buildGeminiEnvCredentials("openai"), null);
const creds = buildGeminiEnvCredentials("gemini");
assert.ok(creds);
assert.equal(creds.apiKey, "env-gemini-key");
assert.equal(creds.connectionId, GEMINI_ENV_CONNECTION_ID);
assert.equal(buildGeminiEnvCredentials("gemini", { forcedConnectionId: "dashboard-row" }), null);
assert.ok(buildGeminiEnvCredentials("gemini", { forcedConnectionId: GEMINI_ENV_CONNECTION_ID }));
assert.equal(buildGeminiEnvCredentials("gemini", { allowedConnections: ["other-id"] }), null);
assert.equal(
buildGeminiEnvCredentials("gemini", { excludedConnectionIds: [GEMINI_ENV_CONNECTION_ID] }),
null
);
});
test("catalog id is gemini/gemini-embedding-2; google/ is an alias", () => {
const native = parseEmbeddingModel("gemini/gemini-embedding-2");
assert.equal(native.provider, "gemini");
assert.equal(native.model, "gemini-embedding-2");
assert.equal(getEmbeddingDimension("gemini/gemini-embedding-2"), 3072);
const aliased = parseEmbeddingModel("google/gemini-embedding-2");
assert.equal(aliased.provider, "gemini");
assert.equal(aliased.model, "gemini-embedding-2");
const preview = parseEmbeddingModel("google/gemini-embedding-2-preview");
assert.equal(preview.provider, "gemini");
assert.equal(preview.model, "gemini-embedding-2-preview");
// Custom provider_node prefix `google` plus embedding-001 must stay unaliased.
const custom = parseEmbeddingModel("google/gemini-embedding-001");
assert.equal(custom.provider, "google");
assert.equal(custom.model, "gemini-embedding-001");
});
test("schema accepts Gemini native text + inline_data mixed batches", () => {
const parsed = v1EmbeddingsSchema.safeParse({
model: "gemini/gemini-embedding-2",
task: "retrieval.query",
input: [
{ text: "a red bicycle" },
{ inline_data: { mime_type: "image/png", data: PNG_B64 } },
],
});
assert.equal(parsed.success, true);
if (parsed.success) {
assert.deepEqual(parsed.data.input, [
{ text: "a red bicycle" },
{ inline_data: { mime_type: "image/png", data: PNG_B64 } },
]);
}
});
test("schema accepts fused Gemini Content and rejects unsafe file URIs", () => {
assert.equal(
v1EmbeddingsSchema.safeParse({
model: "gemini/gemini-embedding-2",
input: {
parts: [{ text: "caption" }, { inline_data: { mime_type: "image/png", data: PNG_B64 } }],
},
}).success,
true
);
for (const file_uri of [
"http://example.com/bike.png",
"https://127.0.0.1/bike.png",
"https://169.254.169.254/latest/meta-data/",
"file:///etc/passwd",
]) {
const parsed = v1EmbeddingsSchema.safeParse({
model: "gemini/gemini-embedding-2",
input: [{ file_data: { mime_type: "image/png", file_uri } }],
});
assert.equal(parsed.success, false, `expected reject: ${file_uri}`);
}
});
test("handleEmbedding sends N Gemini Embedding 2 inputs as N batch requests", async () => {
const originalFetch = globalThis.fetch;
const seen: Array<{ url: string; headers: Record<string, string>; body: Record<string, unknown> }> =
[];
globalThis.fetch = async (url, init = {}) => {
const headers = (init.headers || {}) as Record<string, string>;
seen.push({
url: String(url),
headers,
body: JSON.parse(String(init.body || "{}")) as Record<string, unknown>,
});
return batchEmbeddingResponse(3);
};
try {
const result = await handleEmbedding({
body: {
model: "gemini/gemini-embedding-2",
input: ["alpha", "beta", "gamma"],
dimensions: 768,
},
credentials: { apiKey: "test-gemini-token", connectionId: "conn-gemini-embed" },
log: null,
});
assert.equal(result.success, true, result.error);
assert.equal(seen.length, 1);
assert.equal(
seen[0].url,
"https://generativelanguage.googleapis.com/v1beta/models/gemini-embedding-2:batchEmbedContents"
);
assert.equal(seen[0].headers["x-goog-api-key"], "test-gemini-token");
assert.equal(seen[0].headers.Authorization, undefined);
const requests = seen[0].body.requests as Array<{ content: { parts: unknown[] } }>;
assert.equal(requests.length, 3);
assert.deepEqual(
requests.map((request) => request.content.parts),
[[{ text: "alpha" }], [{ text: "beta" }], [{ text: "gamma" }]]
);
const data = (result.data as { data: Array<{ embedding: number[]; index: number }> }).data;
assert.equal(data.length, 3);
assert.deepEqual(
data.map((row) => row.index),
[0, 1, 2]
);
} finally {
globalThis.fetch = originalFetch;
}
});
test("handleEmbedding forwards Gemini native text+image parts and does not strip to string[]", async () => {
const originalFetch = globalThis.fetch;
const seen: Array<{ url: string; body: Record<string, unknown> }> = [];
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target === IMAGE_URL || target.includes("bike.png")) {
throw new Error("Gemini-native inline_data must not trigger a media fetch");
}
seen.push({
url: target,
body: JSON.parse(String(init.body || "{}")) as Record<string, unknown>,
});
return batchEmbeddingResponse(2);
};
try {
const result = await handleEmbedding({
body: {
model: "google/gemini-embedding-2",
input: [
{ text: "a red bicycle" },
{ inline_data: { mime_type: "image/png", data: PNG_B64 } },
],
},
credentials: { apiKey: "test-gemini-token" },
log: null,
});
assert.equal(result.success, true, result.error);
assert.equal(seen.length, 1);
assert.equal(
seen[0].url,
"https://generativelanguage.googleapis.com/v1beta/models/gemini-embedding-2:batchEmbedContents"
);
const requests = seen[0].body.requests as Array<{ content: { parts: unknown[] } }>;
assert.equal(requests.length, 2);
assert.deepEqual(requests[0].content.parts, [{ text: "a red bicycle" }]);
assert.deepEqual(requests[1].content.parts, [
{ inline_data: { mime_type: "image/png", data: PNG_B64 } },
]);
assert.equal(typeof seen[0].body.input, "undefined");
const data = (result.data as { data: unknown[] }).data;
assert.equal(data.length, 2);
} finally {
globalThis.fetch = originalFetch;
}
});
test("handleEmbedding fuses one Gemini Content with multiple parts into one vector", async () => {
const originalFetch = globalThis.fetch;
let seenBody: Record<string, unknown> | null = null;
let seenUrl = "";
globalThis.fetch = async (url, init = {}) => {
seenUrl = String(url);
seenBody = JSON.parse(String(init.body || "{}")) as Record<string, unknown>;
return singleEmbeddingResponse();
};
try {
const result = await handleEmbedding({
body: {
model: "gemini/gemini-embedding-2",
input: {
parts: [{ text: "caption" }, { inline_data: { mime_type: "image/png", data: PNG_B64 } }],
},
},
credentials: { apiKey: "test-gemini-token" },
log: null,
});
assert.equal(result.success, true, result.error);
assert.equal(
seenUrl,
"https://generativelanguage.googleapis.com/v1beta/models/gemini-embedding-2:embedContent"
);
assert.deepEqual(seenBody?.content, {
parts: [{ text: "caption" }, { inline_data: { mime_type: "image/png", data: PNG_B64 } }],
});
const data = (result.data as { data: unknown[] }).data;
assert.equal(data.length, 1);
} finally {
globalThis.fetch = originalFetch;
}
});
test("handleEmbedding keeps gemini-embedding-001 text batches on the OpenAI shim", async () => {
const originalFetch = globalThis.fetch;
let seenUrl = "";
let seenBody: Record<string, unknown> | null = null;
globalThis.fetch = async (url, init = {}) => {
seenUrl = String(url);
seenBody = JSON.parse(String(init.body || "{}")) as Record<string, unknown>;
return new Response(
JSON.stringify({
data: [
{ object: "embedding", embedding: [0.1], index: 0 },
{ object: "embedding", embedding: [0.2], index: 1 },
],
usage: { prompt_tokens: 2, total_tokens: 2 },
}),
{ status: 200, headers: { "content-type": "application/json" } }
);
};
try {
const result = await handleEmbedding({
body: {
model: "gemini/gemini-embedding-001",
input: ["alpha", "beta"],
},
credentials: { apiKey: "test-gemini-token" },
log: null,
});
assert.equal(result.success, true, result.error);
assert.equal(seenUrl, "https://generativelanguage.googleapis.com/v1beta/openai/embeddings");
assert.deepEqual(seenBody?.input, ["alpha", "beta"]);
} finally {
globalThis.fetch = originalFetch;
}
});

View File

@@ -0,0 +1,229 @@
import test from "node:test";
import assert from "node:assert/strict";
import {
JINA_ENV_CONNECTION_ID,
buildJinaEnvCredentials,
isJinaCredentialProvider,
readJinaEnvApiKey,
} from "../../src/lib/providers/jina.ts";
import {
buildJinaSearchRequest,
extractJinaSearchItems,
} from "../../open-sse/handlers/search/jinaSearch.ts";
import { parseRerankModel, getRerankProvider } from "../../open-sse/config/rerankRegistry.ts";
import { parseEmbeddingModel } from "../../open-sse/config/embeddingRegistry.ts";
import {
getSearchProvider,
resolveSearchProvider,
selectProvider,
SEARCH_CREDENTIAL_FALLBACKS,
SEARCH_PROVIDERS,
} from "../../open-sse/config/searchRegistry.ts";
import { getStaticModelsForProvider } from "../../src/lib/providers/staticModels.ts";
import { v1ClassifySchema, v1SegmentSchema, v1SearchSchema } from "../../src/shared/validation/schemas.ts";
import { APIKEY_PROVIDERS } from "../../src/shared/constants/providers.ts";
const ENV_KEYS = ["JINA_AI_API_KEY", "JINA_API_KEY"] as const;
const savedEnv = Object.fromEntries(ENV_KEYS.map((key) => [key, process.env[key]]));
function restoreEnv() {
for (const key of ENV_KEYS) {
if (savedEnv[key] === undefined) delete process.env[key];
else process.env[key] = savedEnv[key];
}
}
test.afterEach(restoreEnv);
test("Jina env helper prefers JINA_AI_API_KEY over JINA_API_KEY", () => {
delete process.env.JINA_AI_API_KEY;
delete process.env.JINA_API_KEY;
process.env.JINA_API_KEY = "alias-key";
assert.equal(readJinaEnvApiKey(), "alias-key");
process.env.JINA_AI_API_KEY = "primary-key";
assert.equal(readJinaEnvApiKey(), "primary-key");
});
test("Jina env credentials are scoped to Jina provider ids", () => {
process.env.JINA_AI_API_KEY = "env-jina-key";
assert.equal(isJinaCredentialProvider("jina-ai"), true);
assert.equal(isJinaCredentialProvider("jina-reader"), true);
assert.equal(isJinaCredentialProvider("jina-search"), true);
assert.equal(isJinaCredentialProvider("openai"), false);
assert.equal(buildJinaEnvCredentials("openai"), null);
const creds = buildJinaEnvCredentials("jina-ai");
assert.ok(creds);
assert.equal(creds.apiKey, "env-jina-key");
assert.equal(creds.connectionId, JINA_ENV_CONNECTION_ID);
});
test("Jina env credentials honor forced / allowed / excluded connection filters", () => {
process.env.JINA_AI_API_KEY = "env-jina-key";
assert.equal(
buildJinaEnvCredentials("jina-ai", { forcedConnectionId: "dashboard-row" }),
null
);
assert.ok(
buildJinaEnvCredentials("jina-reader", { forcedConnectionId: JINA_ENV_CONNECTION_ID })
);
assert.equal(
buildJinaEnvCredentials("jina-search", { allowedConnections: ["other-id"] }),
null
);
assert.equal(
buildJinaEnvCredentials("jina-ai", { excludedConnectionIds: [JINA_ENV_CONNECTION_ID] }),
null
);
});
test("Jina catalog aliases resolve bare embed and rerank ids", () => {
const embed = parseEmbeddingModel("jina-embeddings-v5-omni-small");
assert.equal(embed.provider, "jina-ai");
assert.equal(embed.model, "jina-embeddings-v5-omni-small");
const family = parseEmbeddingModel("jina-ai/jina-embeddings-v5-omni");
assert.equal(family.provider, "jina-ai");
assert.equal(family.model, "jina-embeddings-v5-omni-small");
const nano = parseEmbeddingModel("jina-embeddings-v5-omni-nano");
assert.equal(nano.provider, "jina-ai");
assert.equal(nano.model, "jina-embeddings-v5-omni-nano");
const rerank = parseRerankModel("jina-reranker-v3.5");
assert.equal(rerank.provider, "jina-ai");
assert.equal(rerank.model, "jina-reranker-v3.5");
const prefixed = parseRerankModel("jina-ai/jina-reranker-v3.5");
assert.equal(prefixed.provider, "jina-ai");
assert.equal(prefixed.model, "jina-reranker-v3.5");
const jina = getRerankProvider("jina-ai");
assert.ok(jina?.models.some((model) => model.id === "jina-reranker-v3.5"));
});
test("Jina dashboard labels distinguish Foundation API from Reader", () => {
assert.equal(APIKEY_PROVIDERS["jina-ai"].name, "Jina AI (Foundation API)");
assert.equal(APIKEY_PROVIDERS["jina-reader"].name, "Jina Reader (r.jina.ai)");
assert.match(APIKEY_PROVIDERS["jina-ai"].authHint || "", /api\.jina\.ai/);
assert.match(APIKEY_PROVIDERS["jina-reader"].authHint || "", /r\.jina\.ai/);
assert.match(APIKEY_PROVIDERS["jina-reader"].authHint || "", /Does not serve/);
});
test("jina-search reuses Foundation credentials and accepts jina-ai alias", () => {
assert.ok(SEARCH_PROVIDERS["jina-search"]);
assert.equal(SEARCH_PROVIDERS["jina-search"].baseUrl, "https://s.jina.ai");
assert.equal(SEARCH_CREDENTIAL_FALLBACKS["jina-search"], "jina-ai");
assert.equal(getSearchProvider("jina-ai"), null);
assert.equal(resolveSearchProvider("jina-ai")?.id, "jina-search");
assert.equal(selectProvider("jina")?.id, "jina-search");
assert.equal(selectProvider("jina-ai")?.id, "jina-search");
assert.equal(selectProvider("jina-search")?.id, "jina-search");
});
test("jina-ai static catalog stays embed/rerank, not searchTypes web", () => {
const models = getStaticModelsForProvider("jina-ai") || [];
assert.ok(
models.some(
(model) =>
model.id === "jina-embeddings-v5-text-small" && model.apiFormat === "embeddings"
)
);
assert.ok(models.some((model) => model.id === "jina-reranker-v3.5" && model.apiFormat === "rerank"));
assert.equal(
models.some((model) => model.id === "web"),
false
);
});
test("v1SearchSchema accepts Jina search aliases", () => {
for (const provider of ["jina-search", "jina-ai", "jina"] as const) {
const result = v1SearchSchema.safeParse({ query: "jina embeddings", provider });
assert.equal(result.success, true, `${provider} should be accepted`);
}
});
test("classify and segment schemas accept Jina-shaped bodies", () => {
const classify = v1ClassifySchema.safeParse({
model: "jina-embeddings-v5-text-small",
input: ["hello"],
labels: ["greeting", "other"],
});
assert.equal(classify.success, true);
const segment = v1SegmentSchema.safeParse({
content: "Split this text into chunks.",
return_chunks: true,
});
assert.equal(segment.success, true);
const missing = v1SegmentSchema.safeParse({ tokenizer: "cl100k_base" });
assert.equal(missing.success, false);
});
test("Jina search builder posts q/num to s.jina.ai with bearer auth", () => {
const built = buildJinaSearchRequest(SEARCH_PROVIDERS["jina-search"], {
query: "jina rerank",
maxResults: 3,
token: "test-jina-token",
country: "US",
});
assert.equal(built.url, "https://s.jina.ai/");
assert.equal(built.init.method, "POST");
const headers = built.init.headers as Record<string, string>;
assert.equal(headers.Authorization, "Bearer test-jina-token");
const body = JSON.parse(String(built.init.body));
assert.equal(body.q, "jina rerank");
assert.equal(body.num, 3);
assert.equal(body.gl, "US");
});
test("Jina search normalizer reads data[] items", () => {
const items = extractJinaSearchItems({
data: [{ title: "Jina", url: "https://jina.ai", description: "Search foundation", content: "# Hi" }],
});
assert.equal(items.length, 1);
assert.equal(items[0].url, "https://jina.ai");
});
test("Jina foundation proxy logs connection_id and forwards JSON", async () => {
const { handleJinaFoundationProxy } = await import(
"../../open-sse/handlers/jinaFoundation.ts"
);
const originalFetch = globalThis.fetch;
globalThis.fetch = async () =>
new Response(JSON.stringify({ data: [{ label: "ok" }] }), {
status: 200,
headers: { "Content-Type": "application/json" },
});
try {
const response = await handleJinaFoundationProxy({
path: "/v1/classify",
upstreamUrl: "https://api.jina.ai/v1/classify",
body: { model: "jina-embeddings-v5-text-small", input: ["hi"], labels: ["a"] },
credentials: { apiKey: "test-jina-token", connectionId: "conn-jina-1" },
provider: "jina-ai",
model: "jina-embeddings-v5-text-small",
});
assert.equal(response.status, 200);
const json = (await response.json()) as { data: Array<{ label: string }> };
assert.equal(json.data[0].label, "ok");
} finally {
globalThis.fetch = originalFetch;
}
});
test("Jina foundation proxy 401s without a key", async () => {
const { handleJinaFoundationProxy } = await import(
"../../open-sse/handlers/jinaFoundation.ts"
);
const response = await handleJinaFoundationProxy({
path: "/v1/segment",
upstreamUrl: "https://segment.jina.ai/",
body: { content: "hello" },
credentials: {},
provider: "jina-ai",
});
assert.equal(response.status, 401);
});

View File

@@ -0,0 +1,148 @@
import test from "node:test";
import assert from "node:assert/strict";
import { mkdtempSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
process.env.DATA_DIR = mkdtempSync(join(tmpdir(), "omniroute-jina-omni-"));
const { v1EmbeddingsSchema } = await import("../../src/shared/validation/schemas/apiV1.ts");
const { handleEmbedding } = await import("../../open-sse/handlers/embeddings.ts");
const PNG_B64 =
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg==";
const DATA_URL = `data:image/png;base64,${PNG_B64}`;
const IMAGE_URL = "https://example.com/bike.png";
const vectorResponse = () =>
new Response(
JSON.stringify({
data: [{ object: "embedding", embedding: [0.1, 0.2], index: 0 }],
usage: { prompt_tokens: 3, total_tokens: 3 },
}),
{ status: 200, headers: { "content-type": "application/json" } }
);
test("schema accepts Jina native text + image URL mixed batches", () => {
const parsed = v1EmbeddingsSchema.safeParse({
model: "jina-ai/jina-embeddings-v5-omni-small",
task: "retrieval.query",
normalized: true,
input: [{ text: "a red bicycle" }, { image: IMAGE_URL }],
});
assert.equal(parsed.success, true);
if (parsed.success) {
assert.deepEqual(parsed.data.input, [{ text: "a red bicycle" }, { image: IMAGE_URL }]);
assert.equal(parsed.data.task, "retrieval.query");
assert.equal(parsed.data.normalized, true);
}
});
test("schema accepts Jina native single ImageDoc, data URI, and fused content groups", () => {
assert.equal(
v1EmbeddingsSchema.safeParse({
model: "jina-ai/jina-embeddings-v5-omni-nano",
input: { image: DATA_URL },
}).success,
true
);
assert.equal(
v1EmbeddingsSchema.safeParse({
model: "jina-ai/jina-embeddings-v5-omni-small",
input: {
content: [{ text: "caption" }, { image: DATA_URL }],
},
}).success,
true
);
});
test("schema still rejects unsafe native image URLs", () => {
for (const image of [
"http://example.com/bike.png",
"https://127.0.0.1/bike.png",
"https://169.254.169.254/latest/meta-data/",
"file:///etc/passwd",
]) {
const parsed = v1EmbeddingsSchema.safeParse({
model: "jina-ai/jina-embeddings-v5-omni-small",
input: [{ image }],
});
assert.equal(parsed.success, false, `expected reject: ${image}`);
}
});
test("handleEmbedding forwards Jina Omni native text+image URL intact and does not fetch the image", async () => {
const originalFetch = globalThis.fetch;
const seen: Array<{ url: string; body: Record<string, unknown> }> = [];
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target === IMAGE_URL || target.includes("bike.png")) {
throw new Error("OmniRoute must not fetch Jina-native image URLs");
}
seen.push({
url: target,
body: JSON.parse(String(init.body || "{}")) as Record<string, unknown>,
});
return vectorResponse();
};
try {
const result = await handleEmbedding({
body: {
model: "jina-ai/jina-embeddings-v5-omni-small",
task: "retrieval.query",
normalized: true,
input: [{ text: "a red bicycle" }, { image: IMAGE_URL }],
},
credentials: { apiKey: "test-jina-token", connectionId: "conn-jina-omni" },
log: null,
});
assert.equal(result.success, true, result.error);
assert.equal(seen.length, 1);
assert.equal(seen[0].url, "https://api.jina.ai/v1/embeddings");
assert.deepEqual(seen[0].body.input, [{ text: "a red bicycle" }, { image: IMAGE_URL }]);
assert.equal(seen[0].body.model, "jina-embeddings-v5-omni-small");
assert.equal(seen[0].body.task, "retrieval.query");
assert.equal(seen[0].body.normalized, true);
} finally {
globalThis.fetch = originalFetch;
}
});
test("handleEmbedding family alias jina-embeddings-v5-omni sends omni-small upstream", async () => {
const originalFetch = globalThis.fetch;
let upstreamModel = "";
globalThis.fetch = async (_url, init = {}) => {
upstreamModel = JSON.parse(String(init.body || "{}")).model;
return vectorResponse();
};
try {
const result = await handleEmbedding({
body: {
model: "jina-ai/jina-embeddings-v5-omni",
input: [{ text: "hello" }, { image: DATA_URL }],
},
credentials: { apiKey: "test-jina-token" },
log: null,
});
assert.equal(result.success, true, result.error);
assert.equal(upstreamModel, "jina-embeddings-v5-omni-small");
} finally {
globalThis.fetch = originalFetch;
}
});
test("handleEmbedding rejects native image docs on text-only Jina SKUs", async () => {
const result = await handleEmbedding({
body: {
model: "jina-ai/jina-embeddings-v5-text-small",
input: [{ image: IMAGE_URL }],
},
credentials: { apiKey: "test-jina-token" },
log: null,
});
assert.equal(result.success, false);
assert.equal(result.status, 400);
assert.match(result.error, /does not advertise structured embedding input/i);
});

View File

@@ -288,13 +288,9 @@ test("embedding and rerank specialty validators cover Voyage AI and Jina AI", as
return new Response(JSON.stringify({ data: [{ embedding: [0.1, 0.2] }] }), { status: 200 });
}
if (target === "https://api.jina.ai/v1/rerank") {
if (target === "https://api.jina.ai/v1/models") {
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer jina-key");
const body = JSON.parse(String(init.body));
assert.equal(body.model, "jina-reranker-v3");
return new Response(JSON.stringify({ results: [{ index: 0, relevance_score: 0.99 }] }), {
status: 200,
});
return new Response(JSON.stringify({ data: [] }), { status: 200 });
}
throw new Error(`unexpected fetch: ${target}`);
@@ -352,7 +348,7 @@ test("embedding and rerank specialty validators surface auth failures for Voyage
if (target === "https://api.voyageai.com/v1/embeddings") {
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
}
if (target === "https://api.jina.ai/v1/rerank") {
if (target === "https://api.jina.ai/v1/models") {
return new Response(JSON.stringify({ error: "forbidden" }), { status: 403 });
}
throw new Error(`unexpected fetch: ${target}`);
@@ -362,7 +358,7 @@ test("embedding and rerank specialty validators surface auth failures for Voyage
const jina = await validateProviderApiKey({ provider: "jina-ai", apiKey: "jina-key" });
assert.equal(voyage.error, "Invalid API key");
assert.equal(jina.error, "Invalid API key");
assert.equal(jina.error, "Invalid API key (GET https://api.jina.ai/v1/models)");
});
test("v0-vercel specialty validator checks the Platform API chats endpoint", async () => {

View File

@@ -33,8 +33,9 @@ test("SEARCH_PROVIDERS has all registered providers", () => {
assert.ok(SEARCH_PROVIDERS["searxng-search"], "searxng should exist");
assert.ok(SEARCH_PROVIDERS["ollama-search"], "ollama-search should exist");
assert.ok(SEARCH_PROVIDERS["zai-search"], "zai should exist");
assert.ok(SEARCH_PROVIDERS["jina-search"], "jina-search should exist");
assert.ok(SEARCH_PROVIDERS["duckduckgo-free"], "duckduckgo-free should exist");
assert.equal(Object.keys(SEARCH_PROVIDERS).length, 14);
assert.equal(Object.keys(SEARCH_PROVIDERS).length, 15);
});
test("duckduckgo-free config is a no-key, fallback-only provider", () => {
@@ -96,6 +97,8 @@ test("getSearchProvider returns config for valid ID", () => {
test("getSearchProvider returns null for unknown ID", () => {
assert.equal(getSearchProvider("unknown"), null);
// jina-ai is the Foundation embed/rerank card, not a search catalog id.
assert.equal(getSearchProvider("jina-ai"), null);
});
test("tavily config is correct", () => {
@@ -166,8 +169,9 @@ test("zai-search config is correct", () => {
test("getAllSearchProviders returns flat list", () => {
const all = getAllSearchProviders();
assert.equal(all.length, 14);
assert.equal(all.length, 15);
assert.ok(all.some((p) => p.id === "duckduckgo-free"));
assert.ok(all.some((p) => p.id === "jina-search"));
assert.ok(all.some((p) => p.id === "serper-search"));
assert.ok(all.some((p) => p.id === "brave-search"));
assert.ok(all.some((p) => p.id === "perplexity-search"));

View File

@@ -52,7 +52,7 @@ test("v1 search GET lists all search providers", async () => {
assert.equal(response.status, 200);
assert.equal(body.object, "list");
assert.equal(body.data.length, 14);
assert.equal(body.data.length, 15);
assert.deepEqual(ids, [
"serper-search",
"brave-search",
@@ -67,6 +67,7 @@ test("v1 search GET lists all search providers", async () => {
"searxng-search",
"ollama-search",
"zai-search",
"jina-search",
"duckduckgo-free",
]);
});

View File

@@ -1159,6 +1159,21 @@ test("getProviderCredentials resolves the antigravity / agy alias pool", async (
assert.equal(selected.connectionId, connection.id);
});
test("getProviderCredentials shares one Jina token across foundation, reader, and search", async () => {
const connection = await seedConnection("jina-ai", {
name: "jina-foundation-key",
apiKey: "jina-dashboard-key",
});
const viaSearch = await auth.getProviderCredentials("jina-search");
const viaReader = await auth.getProviderCredentials("jina-reader");
assert.ok(viaSearch && !("allExpired" in viaSearch));
assert.ok(viaReader && !("allExpired" in viaReader));
assert.equal(viaSearch.connectionId, connection.id);
assert.equal(viaReader.connectionId, connection.id);
});
test("getProviderCredentials exposes copilotToken when present in providerSpecificData", async () => {
const connection = await seedConnection("codex", {
authType: "oauth",