pjandro/OmniRoute
1
0
Fork 0
mirror of https://github.com/diegosouzapw/OmniRoute.git synced 2026-08-18 05:02:15 +03:00
Code Issues Packages Projects Releases Wiki Activity
4,640 Commits 102 Branches 313 Tags
b6f2bd85feafcfca4aff708b07f8be0e2f3fafef
Commit Graph

829 Commits

Author SHA1 Message Date
diegosouzapw
8f6651d053 feat(memory): rewire retrieval/store/settings/summarization + reindex (plan 21 F5) 
- retrieval.ts — semantic/hybrid usa vectorStore quando disponível; degrada para FTS5 transparente
- retrieval.ts — adiciona retrievePreview() (dry-run para Playground) e engineStatus()
- retrieval.ts — rerank opcional via provider configurado (D13)
- store.ts — createMemory/updateMemory geram vetor best-effort; deleteMemory sincroniza vec + Qdrant (D15)
- settings.ts — 7 campos novos (embeddingSource, embeddingProviderModel, transformersEnabled, staticEnabled, rerankEnabled, rerankProviderModel, vectorStore) com defaults
- summarization.ts — summarizeMemoriesOlderThan exposta para uso manual (D19)
- reindex.ts (novo) — runReindexBatch processa fila lazy de backfill (D21)
- 9 testes unitários adicionados; testes F1-F4 sem regressão
 2026-05-28 08:27:28 -03:00
diegosouzapw
bdaa045d5d merge(F4): vector store (sqlite-vec + hybrid RRF) 2026-05-28 01:26:16 -03:00
diegosouzapw
c16ce8a9e1 feat(api): traffic-inspector hosts + capture-modes + tls routes (F6) 2026-05-28 01:24:20 -03:00
diegosouzapw
5508dc4e3c feat(memory): add sqlite-vec vector store with hybrid RRF search (plan 21 F4) 
Implements VectorStore interface contract from master plan 21 §3.4:
- sqlite-vec v0.1.9 extension loaded via createRequire (ESM compat)
- vec0 virtual table with FLOAT[N] dimensions driven by EmbeddingResolution
- Upsert via DELETE+INSERT (vec0 does not support INSERT OR REPLACE)
- BigInt rowids required by vec0 v0.1.9 for primary key insertion
- Hybrid RRF (k=60) fusing FTS5 + vector KNN via UNION ALL + GROUP BY
- FTS join on m.memory_id = fts.rowid (migration 023 bridge column)
- VECTOR_STORE_DISABLE_VEC=true test seam for null-extension path
- sanitizeErrorMessage in 3 error paths (Hard Rule #12)
- Raw SQL exception documented in header comment (Hard Rule #5 §D5)
- 27 unit tests across 5 files; all lint/typecheck/cycles checks pass
 2026-05-28 01:22:38 -03:00
diegosouzapw
02bc079ef9 feat(memory): add embedding layer — remote/static/transformers/cache (plan 21 F3) 
Implements the multi-source embedding layer for the Memory Engine Redesign (plan 21).
Adds 5 production modules under src/lib/memory/embedding/:
- cache.ts: LRU+TTL in-memory cache (max=1000, TTL=5min, sha256 keyed)
- remote.ts: delegates to createEmbeddingResponse(), maps HTTP 401/403→no_key, 429→rate_limited, AbortError→timeout; all errors via sanitizeErrorMessage()
- staticPotion.ts: download-once potion-base-8M (JS-only WordPiece tokenizer + mean pooling, no WASM)
- transformersLocal.ts: lazy await import('@huggingface/transformers') singleton pipeline (Xenova/all-MiniLM-L6-v2, q8)
- index.ts: resolveEmbeddingSource (pure, sync), embed (cached dispatch), listEmbeddingProviders, invalidateEmbeddingCache

Also adds @huggingface/transformers and sqlite-vec to dependencies, and registers
@huggingface/transformers in next.config.mjs serverExternalPackages (D8/D25).

6 unit test files: cache (9), resolve (14), remote (10), static-potion (13), transformers (6), list-providers (8) — all 60 tests green.
 2026-05-28 00:40:56 -03:00
diegosouzapw
2f92399e23 feat(playground): add streamMetrics pure function 2026-05-27 23:59:23 -03:00
diegosouzapw
bb312062ea merge(F2): db migrations + memoryVec + localDb re-export 2026-05-27 23:26:22 -03:00
diegosouzapw
b9f93a5c07 feat(memory): add migration 073, memoryVec CRUD module, and localDb re-export (plan 21 F2) 
- 073_memory_vec.sql: creates memory_vec_meta singleton table (active_dim,
  embedding_signature, last_reset_at, vec_loaded) and adds needs_reindex column
  to memories table with a partial index; idempotent via CREATE IF NOT EXISTS +
  INSERT OR IGNORE + migration runner's duplicate-column-name guard
- src/lib/db/memoryVec.ts: implements 6 CRUD functions per §3.8 contract
  (getMemoryVecMeta, setMemoryVecMeta, markMemoryNeedsReindex,
  markAllMemoriesNeedReindex, getMemoryReindexQueue, countMemoryReindexPending)
- src/lib/localDb.ts: adds re-export block for the 6 functions (Hard Rule #2)
- .env.example: documents 7 new MEMORY_* env vars per §3.9
- tests/unit/memory-vec-meta.test.ts: 7 tests (meta get/set, migration idempotency)
- tests/unit/memory-needs-reindex.test.ts: 12 tests (mark/unmark, markAll, queue)
 2026-05-27 23:16:05 -03:00
diegosouzapw
5dd75be1b9 test(batch): add integration tests + sanitization asserts + coverage gap fillers (F9) 
- Add tests/unit/batches-f9-helpers.test.ts (19 tests, top-level for c8 coverage gate)
  covering uncovered branches: alias-match pricing, blank CSV rows, body.input/prompt paths,
  non-object JSON lines, invalid Anthropic params, body-is-array validation
- Add tests/unit/dashboard/batch/concept-cards.test.tsx (16 tests)
  covering BatchConceptCard + FilesConceptCard: render, toggle, localStorage hydration, sanitization
- Add tests/unit/dashboard/batch/list-regression.test.tsx (15 tests)
  covering BatchListTab + FilesListTab: render N items, Remove-completed flow,
  status/purpose filter, loading/empty states, sanitization
- Add tests/unit/dashboard/batch/sanitization.test.tsx (8 tests)
  covering NewBatchWizard + UploadFileModal + useBatchActions: each error path
  asserts zero stack-trace/path leakage into the UI (D14 / Hard Rule #12)
- Fix bug in validateJsonl.ts: body=array was not caught as invalid
  (typeof array === "object" is true — add Array.isArray guard, 1-line fix)

Local src/lib/batches/ coverage: 100% stmts / 93.7% branches / 100% funcs / 100% lines.
Global coverage gate: 75.96% stmts / 71.97% branches / 75.52% funcs (all above 75/75/75/70).
 2026-05-27 23:14:17 -03:00
diegosouzapw
6236b604ec feat(memory): add shared foundation types, Zod schemas, and roundtrip tests (plan 21 F1) 
- src/lib/memory/embedding/types.ts — EmbeddingSource, EmbeddingProviderListing, EmbeddingResolution, EmbeddingResult, EmbeddingError (verbatim §3.1)
- src/shared/schemas/memory.ts — 7 Zod schemas (MemorySettingsExtended, MemoryUpdatePut, RetrievePreview, MemoryReindex, MemorySummarize, EmbeddingProviderListing, MemoryEngineStatus, RetrievePreviewResult) + z.infer types (verbatim §3.2)
- src/shared/schemas/qdrant.ts — 4 Zod schemas (QdrantSettings, QdrantSettingsUpdate, QdrantSearch, QdrantHealthResult) + z.infer types (verbatim §3.3)
- tests/unit/memory-schemas-roundtrip.test.ts — 34 assertions (≥22 required); all pass
 2026-05-27 22:54:18 -03:00
diegosouzapw
f211fcf509 merge: F7 enforce wiring (chatCore PRE/POST + combo soft penalty + spendRecorder) 2026-05-27 22:23:33 -03:00
diegosouzapw
0e357a9cc7 merge(F6): integrate W3 frente F6 into base 2026-05-27 22:17:24 -03:00
diegosouzapw
0181348cee feat(quota): add spendRecorder fire-and-forget wrapper (B/F7) 
scheduleRecordConsumption() wraps recordConsumption() in setImmediate so it
never adds latency to the client response path. Errors are caught and logged
via pino warn but NEVER propagated to the caller (B29 fail-open contract).
 2026-05-27 22:17:16 -03:00
diegosouzapw
d80e1b63eb feat(quota): add enforce.ts (enforceQuotaShare + recordConsumption) (B/F7) 
Implements the quota share enforcement gate and consumption recorder:
- enforceQuotaShare(): PRE-request check that returns allow/block/deprioritize
  based on fair-share algorithm, saturation signals, and pool allocations.
- recordConsumption(): POST-response tracker that increments per-key counters
  for each active plan dimension.
Both functions fail-open per B16/B29: any infra error → allow + warn log.
 2026-05-27 22:17:10 -03:00
diegosouzapw
6a19882646 feat(compliance): support level=high filter in audit-log API (B/F4) 2026-05-27 22:00:56 -03:00
diegosouzapw
fb54bcd994 feat(audit): add timeline helpers (groupByDay + relativeTime) (B/F4) 2026-05-27 22:00:51 -03:00
diegosouzapw
f86e905efb feat(agent-skills): add generator with idempotent skill md generation + prune 
Implements src/lib/agentSkills/generator.ts (§3.4 contract):
- generateAgentSkills(opts): idempotent, dryRun:true default, prune:false default
- buildSkillMarkdown(skillId, sources): generates frontmatter + API/CLI body
- API body: Visão geral + Autenticação + Endpoints (with curl) + Payloads
- CLI body: Visão geral + Instalação rápida + Subcomandos (with flags + examples)
- Prune: detects orphans in skills/{id}/ not in catalog; deletes in apply mode
- Marker preservation: <!-- skill:custom-start --> ... <!-- skill:custom-end -->
- Generated comment: per D24 spec
 2026-05-27 21:54:42 -03:00
diegosouzapw
c91decf543 feat(a2a): register list-capabilities in A2A_SKILL_HANDLERS + Agent Card 
Adds "list-capabilities" entry to A2A_SKILL_HANDLERS in taskExecution.ts
(dynamic import pattern, consistent with the 5 existing skills) and adds
the 6th skill entry to /.well-known/agent.json with tags [discovery, capabilities]
and example questions for agent discovery.
 2026-05-27 21:48:57 -03:00
diegosouzapw
6bad368dcb feat(a2a): add list-capabilities skill with markdown table of 42 skills 
Implements executeListCapabilities() which calls getCatalog() + computeCoverage()
from the agentSkills catalog (F1/F2) and returns a markdown table covering all
42 skills (22 API + 20 CLI) with ID, name, category, area, endpoints/commands,
and raw SKILL.md URL, matching the §3.7 result contract.
 2026-05-27 21:48:50 -03:00
diegosouzapw
c5f697dbc6 feat(inspector): add harExport (F4) 2026-05-27 21:44:46 -03:00
diegosouzapw
c1952db4cb feat(cli-tools): add /api/cli-tools/all-statuses batch endpoint + mtime cache + DRY checkToolConfigStatus (plan 14 F2) 
- Extract checkToolConfigStatus() from /api/cli-tools/status/route.ts → src/lib/cliTools/checkToolConfigStatus.ts (DRY, sentinel comment, optional configPathOverride for tests)
- Create batchStatusCache.ts singleton in-memory Map<toolId,{mtimeMs,result}> (getCached/setCached/invalidate/clearCache)
- Create /api/cli-tools/all-statuses GET route: auth via requireCliToolsAuth, iterates CLI_TOOLS, Promise.allSettled per tool, timeout 5s, mtime-based cache, endpoint extraction, lastConfiguredAt merge, buildErrorBody on error path
- Update /api/cli-tools/status/route.ts to import from new module (no behavior change)
- 27 unit tests (batch-status-cache + check-tool-config-status) + 8 integration tests (all-statuses-route) — all passing
 2026-05-27 21:43:44 -03:00
diegosouzapw
3c3a02ed42 feat(playground): add types.ts with static provider pricing table 
Re-exports all playground types and adds static MODEL_PRICING_TABLE with
8-10 popular models labeled (estimated) for client-side cost estimation (D13).
Exports getModelPricing and getProviderPricing helpers.
 2026-05-27 21:28:57 -03:00
diegosouzapw
bda03ce5dc feat(playground): add promptImprover.ts meta-prompt helpers 
Adds META_SYSTEM_PROMPT, ImprovePromptRequestSchema, buildImproveChatBody,
and parseImprovedContent for the Prompt Improver feature (D8). Handles
system-only, prompt-only, and both-present scenarios with <<SYSTEM>>/<<PROMPT>>
markers.
 2026-05-27 21:28:52 -03:00
diegosouzapw
25613e6176 feat(playground): add codeExport.ts generator (curl/python/typescript) 
Implements the shared codeExport.ts foundation for the Playground Studio.
Generates curl/python/typescript snippets for all 10 endpoints (chat.completions,
completions, embeddings, images, audio.transcriptions, audio.speech, moderations,
rerank, search, web.fetch). Always uses $OMNIROUTE_API_KEY placeholder (D11).
 2026-05-27 21:28:47 -03:00
diegosouzapw
4d825ad482 feat(quota): add saturationSignals reader with 30s cache and fail-open (B/F6) 2026-05-27 20:38:41 -03:00
diegosouzapw
23f5b6f8b8 feat(quota): add planResolver (DB override > known catalog > empty) (B/F6) 2026-05-27 20:38:33 -03:00
diegosouzapw
c3c0817c3b feat(quota): add burnRate EMA estimator and time-to-exhaustion (B/F6) 2026-05-27 20:38:25 -03:00
diegosouzapw
9905d92244 feat(quota): add fairShare work-conserving algorithm (multi-dimension, generous/strict modes, cap-absolute) (B/F6) 2026-05-27 20:38:18 -03:00
diegosouzapw
67548034be feat(quota): add storeFactory with setting/env-driven driver selection (B/F6) 2026-05-27 20:38:11 -03:00
diegosouzapw
4bb44b10d3 feat(quota): add redisQuotaStore (optional driver, gated by ioredis availability) (B/F6) 2026-05-27 20:38:02 -03:00
diegosouzapw
ca85652bab feat(quota): add sqliteQuotaStore with sliding window counter and per-key mutex (B/F6) 2026-05-27 20:37:51 -03:00
diegosouzapw
8a10b3ecd8 feat(quota): add QuotaStore facade and types re-export (B/F6) 2026-05-27 20:37:44 -03:00
diegosouzapw
0a95372746 feat(agent-skills): add openapiParser and cliRegistryParser 
openapiParser.ts:
- parseOpenapi(): reads docs/reference/openapi.yaml via js-yaml (already a dep)
  and returns { paths: Map<METHOD+path, OpenapiPath>, areas: Map<SkillArea, ops[]> }
- PATH_AREA_MAP maps 30+ path prefixes to SkillArea values
- getEndpointsForArea(area): convenience helper returning 'METHOD /path' strings

cliRegistryParser.ts:
- parseCliRegistry(): reads all bin/cli/commands/*.mjs via fs.readdirSync
  and regex-parses .command(), .description(), .option() calls
- FILE_FAMILY_MAP maps 40+ file basenames to CLI SkillArea families
- getCommandsForFamily(family): convenience helper for catalog consumers
- Does NOT import Commander.js modules to avoid side-effects (D15)
 2026-05-27 20:31:55 -03:00
diegosouzapw
a0cc22be73 feat(agent-skills): add catalog.ts with getCatalog/filter/coverage/fetch helpers 
Implements the catalog.ts public API defined in §3.3 of the master plan:
- getCatalog(): AgentSkill[] — returns 42 entries, lazy-cached in module scope
- getSkillById(id): AgentSkill | null — lookup by canonical ID
- filterCatalog(opts): AgentSkill[] — filter by category and/or area
- computeCoverage(): SkillCoverage — reads skills/ dir and counts SKILL.md present
- refreshCatalog(): void — invalidates cache (used by tests + generator)
- fetchSkillMarkdown(id): Promise<SkillMarkdown> — reads local fs first,
  falls back to GitHub raw fetch with 1h Next.js cache (for F4 /raw route)

API_SKILL_IDS and CLI_SKILL_IDS exported as readonly string arrays (D28 order).
Single source of truth for all consumers (REST routes, MCP, A2A).
 2026-05-27 20:31:44 -03:00
diegosouzapw
269fce6f0b feat(batches): add pure helpers — csvToJsonl, validateJsonl, costEstimator, retryFailed (F2) 2026-05-27 19:42:51 -03:00
diegosouzapw
ddf6b0ef63 merge(F2): DB migrations + modules into Group A parent 2026-05-27 19:41:11 -03:00
diegosouzapw
411a6d85d1 feat(inspector): add types, contextKey, kindDetector (F1) 
- InterceptedRequest/LlmMetadata/WsEvent types + InterceptedRequestSchema
- extractSystemPrompt() supports OpenAI/Anthropic/Gemini formats
- computeContextKey() returns 12-hex SHA-256 of system prompt
- detectKind() classifies traffic via 18 host patterns + path + body + UA
- src/lib/inspector/secretMask.ts re-exports maskSecret (plano 12 bridge)
 2026-05-27 19:39:26 -03:00
diegosouzapw
3c50ebce8f merge: F2 DB migrations + modules (pools, consumption, plans) 2026-05-27 19:25:12 -03:00
diegosouzapw
47c0dce062 chore(env): document AgentBridge + Inspector env vars and re-exports (F2) 2026-05-27 19:24:26 -03:00
diegosouzapw
9fcfc2bd0b feat(db): add inspector custom hosts + sessions CRUD modules (F2) 2026-05-27 19:24:22 -03:00
diegosouzapw
45f602606b feat(db): add agentBridge state/mappings/bypass CRUD modules (F2) 2026-05-27 19:24:19 -03:00
diegosouzapw
89d3304a93 feat(db): add migrations 073/074/075 agent_bridge + inspector (F2) 2026-05-27 19:24:14 -03:00
diegosouzapw
051ce5e786 feat(agent-skills): add foundation types + Zod schemas 2026-05-27 19:23:20 -03:00
diegosouzapw
44e49f635b chore(db): re-export quota modules in localDb (B/F2) 
Adds re-export blocks for quotaPools (7 functions), quotaConsumption
(4 functions with gcQuotaConsumption alias), and providerPlans (4
functions with getProviderPlan/listProviderPlans/etc. aliases).
Zero logic added to localDb.ts — Hard Rule #2 maintained.
 2026-05-27 19:14:38 -03:00
diegosouzapw
1721cf7b32 feat(db): add providerPlans module with CRUD for per-connection quota plans (B/F2) 
Implements getPlan, listPlans, upsertPlan (idempotent ON CONFLICT DO
UPDATE), and deletePlan. Serializes QuotaDimension[] as JSON into
dimensions_json column and parses back on read. Malformed JSON returns
empty dimensions rather than throwing.
 2026-05-27 19:14:32 -03:00
diegosouzapw
83790b6c6a feat(db): add quotaConsumption sliding-window counter storage (B/F2) 
Implements getBucket, incrementBucket (atomic UPSERT), getPair (curr+prev
for sliding window formula), and gcOlderThan (stale bucket cleanup).
Atomic increment uses INSERT ... ON CONFLICT DO UPDATE — no separate
read-modify-write cycle needed.
 2026-05-27 19:14:27 -03:00
diegosouzapw
07d6a17643 feat(db): add quotaPools module with CRUD and allocation management (B/F2) 
Implements listPools, getPool, createPool, updatePool, deletePool,
upsertAllocations (replace strategy via transaction), and
listAllocationsForApiKey. All SQL uses prepared statements. Local type
shapes aligned with src/lib/quota/dimensions.ts contract (B13).
 2026-05-27 19:14:21 -03:00
diegosouzapw
4f149fb5a3 feat(db): add quota_pools and quota_consumption migrations (B/F2) 
Creates migrations 073 (quota_pools + quota_allocations) and 074
(quota_consumption sliding-window counter). Both are idempotent via
CREATE TABLE IF NOT EXISTS and CREATE INDEX IF NOT EXISTS. FK ON DELETE
CASCADE from quota_allocations to quota_pools. Fixes B2 IDs.
 2026-05-27 19:14:15 -03:00
diegosouzapw
2c4f26d726 chore(db): re-export playgroundPresets from localDb 
Adds one re-export block at the end of localDb.ts per Hard Rule #2
(re-export only, zero logic, zero function/const/class additions).
 2026-05-27 19:05:26 -03:00
diegosouzapw
617d761948 feat(db): add playgroundPresets CRUD module 
Implements listPlaygroundPresets, getPlaygroundPreset, createPlaygroundPreset,
updatePlaygroundPreset, and deletePlaygroundPreset using db.prepare() (never
raw db.exec or string interpolation). randomUUID() from node:crypto for IDs;
params serialized via JSON.stringify/JSON.parse with fallback to {}.
 2026-05-27 19:05:21 -03:00