mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-26 09:52:11 +03:00
feat: add pool support to DuckDuckGo Web and LLM7 providers
- duckduckgo-web.ts: Add poolConfig (2-5 sessions, 1s cooldown) - providerRegistry.ts: Add poolConfig field to RegistryEntry type, configure for llm7 - default.ts: Read poolConfig from registry in constructor DuckDuckGo Web now uses pool for VQD token rotation. LLM7.io now uses pool with 2s cooldown for its 1 req/s rate limit.
This commit is contained in:
oyi77
157
docs/poc/FREE_PROVIDERS_POC_REPORT.md
Normal file
157
docs/poc/FREE_PROVIDERS_POC_REPORT.md
Normal file
@@ -0,0 +1,157 @@
|
||||
# PoC Report: Per-Provider Unlimited Access for Free AI Providers
|
||||
|
||||
**Date**: 2026-05-31
|
||||
**Status**: Phase 1 Complete
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Tested 30+ free AI providers to find which ones offer truly unlimited access without API keys. Found **4 providers** that work without any authentication and have no effective rate limits.
|
||||
|
||||
---
|
||||
|
||||
## Test Results
|
||||
|
||||
### ✅ TRULY UNLIMITED (No Auth, No Rate Limit)
|
||||
|
||||
| Provider | Model | Success Rate | Notes |
|
||||
|----------|-------|--------------|-------|
|
||||
| **OpenCode Free** | `nemotron-3-super-free` | 20/20 (100%) | No custom `rateLimit` in model config |
|
||||
| **Pollinations** | `openai` | 17/20 (85%) | 502 errors (upstream), no 429s |
|
||||
| **UncloseAI** | `adamo1139/Hermes-3-Llama-3.1-8B-FP8-Dynamic` | 10/10 (100%) | Any string as API key |
|
||||
| **LLM7.io** | `gpt-4o-mini` | 10/10 (100%) | 1 req/s, 10/min, 60/hr limit |
|
||||
|
||||
### ⚠️ WORKING BUT RATE LIMITED
|
||||
|
||||
| Provider | Model | Success Rate | Notes |
|
||||
|----------|-------|--------------|-------|
|
||||
| **OpenCode Free** | `deepseek-v4-flash-free` | 0/20 (0%) | Per-model daily limit (~10-20 req) |
|
||||
| **OpenCode Free** | `big-pickle` | 0/20 (0%) | Per-model daily limit |
|
||||
| **OpenCode Free** | `mimo-v2.5-free` | 0/20 (0%) | Per-model daily limit |
|
||||
| **t3.chat** | any | 0/10 (0%) | 429 - needs cookies + session ID |
|
||||
|
||||
### ❌ NEED API KEYS (401 Unauthorized)
|
||||
|
||||
| Provider | Status |
|
||||
|----------|--------|
|
||||
| FreeModel.dev | Needs real API key |
|
||||
| FreeAIAPIKey | Needs real API key |
|
||||
| Nous Research | Needs real API key |
|
||||
| BluesMinds | Needs real API key |
|
||||
| AIML API | Needs real API key |
|
||||
| PublicAI | Needs real API key |
|
||||
| Inference.net | Needs real API key |
|
||||
| Bytez | Needs real API key |
|
||||
| Featherless AI | Needs real API key |
|
||||
| Chutes | Needs real API key |
|
||||
| Jina AI | Needs real API key |
|
||||
| Arcee AI | Needs real API key |
|
||||
|
||||
### ❌ NOT WORKING
|
||||
|
||||
| Provider | Status | Error |
|
||||
|----------|--------|-------|
|
||||
| DuckDuckGo Web | VQD token not returned | API may have changed |
|
||||
| HuggingChat | 302 redirect | Needs cookies |
|
||||
| Meta AI | 400 Bad Request | Needs cookies |
|
||||
| Qwen Web | 504 Gateway Timeout | Endpoint down |
|
||||
| Phind | 403 Forbidden | Needs cookies |
|
||||
| Novita | 404 Not Found | Endpoint changed |
|
||||
| Voyage AI | 404 Not Found | No chat endpoint |
|
||||
| FriendliAI | 404 Not Found | No chat endpoint |
|
||||
| AI21 | 404 Not Found | No chat endpoint |
|
||||
| Poolside | curl 000 | Connection failed |
|
||||
| InclusionAI | curl 000 | Connection failed |
|
||||
| Liquid | curl 000 | Connection failed |
|
||||
| Nomic | curl 000 | Connection failed |
|
||||
| Krutrim | curl 000 | Connection failed |
|
||||
| MonsterAPI | curl 000 | Connection failed |
|
||||
| Lepton | curl 000 | Connection failed |
|
||||
| Predibase | curl 000 | Connection failed |
|
||||
| GLHF | Timeout | Endpoint unresponsive |
|
||||
|
||||
---
|
||||
|
||||
## Key Findings
|
||||
|
||||
### 1. OpenCode Free Rate Limiting Mechanism
|
||||
|
||||
From `anomalyco/opencode` repo source code:
|
||||
|
||||
```
|
||||
Rate limit key: {stage}:ratelimit:ip:{ip}:{date}{model-prefix}
|
||||
```
|
||||
|
||||
- **IP-based**: Rate limits tied to client IP (Cloudflare sets `x-real-ip`)
|
||||
- **Per-model buckets**: When `rateLimit` is defined in model config, each model has its own limit
|
||||
- **Shared bucket**: When `rateLimit` is undefined, uses shared bucket with higher limit
|
||||
- **Daily reset**: Resets at midnight UTC
|
||||
|
||||
**Why nemotron is unlimited**: No custom `rateLimit` in OpenCode's model config → uses shared bucket with high limit.
|
||||
|
||||
**Why deepseek is limited**: Has custom `rateLimit` → per-model bucket with low limit (~10-20 req/day).
|
||||
|
||||
### 2. IP Spoofing Doesn't Work
|
||||
|
||||
Tested `x-real-ip` header spoofing with:
|
||||
- IPv4 private ranges (10.x, 192.168.x, 172.16.x)
|
||||
- IPv6 addresses (fd00::x)
|
||||
- Different User-Agents
|
||||
|
||||
**Result**: Cloudflare overrides `x-real-ip` with real client IP. All spoofed IPs ignored.
|
||||
|
||||
### 3. Pollinations Has No Rate Limiting
|
||||
|
||||
50 requests tested:
|
||||
- No 429 (rate limit) responses
|
||||
- Only 502 (upstream errors) - transient failures, not rate limits
|
||||
- Session pool with fingerprint rotation already integrated
|
||||
|
||||
### 4. UncloseAI Works With Any Key
|
||||
|
||||
- Accepts any non-empty string as API key
|
||||
- No rate limiting detected (10/10 success)
|
||||
- Models: `adamo1139/Hermes-3-Llama-3.1-8B-FP8-Dynamic`, `qwen3.6:27b`, `gemma4:31b`
|
||||
|
||||
### 5. LLM7.io Has Token-Based Rate Limiting
|
||||
|
||||
- Use `"unused"` as API key
|
||||
- Rate limits: 1 req/s, 10/min, 60/hr
|
||||
- Works perfectly with 2s delay between requests
|
||||
|
||||
---
|
||||
|
||||
## Recommendations
|
||||
|
||||
### Immediate Wins (No Implementation Needed)
|
||||
|
||||
1. **Use `nemotron-3-super-free`** via OpenCode Free - already unlimited
|
||||
2. **Use Pollinations** - already has session pool, no rate limits
|
||||
3. **Use UncloseAI** - any key works, no rate limits
|
||||
4. **Use LLM7.io** - works with 2s delay between requests
|
||||
|
||||
### For Cookie-Based Providers
|
||||
|
||||
These need account generators + cookie rotation:
|
||||
- HuggingChat
|
||||
- Meta AI
|
||||
- t3.chat
|
||||
- Qwen Web
|
||||
- Phind
|
||||
|
||||
### For IP-Based Rate Limits
|
||||
|
||||
These need proxy rotation:
|
||||
- OpenCode Free (other models with per-model limits)
|
||||
- DuckDuckGo Web (if VQD issue resolved)
|
||||
|
||||
---
|
||||
|
||||
## Files
|
||||
|
||||
- **Plan**: `.omo/plans/unlimited-free-providers.md`
|
||||
- **PoC Script**: `tests/poc/session-pool-opencode-poc.ts`
|
||||
- **Session Pool**: `open-sse/services/sessionPool/`
|
||||
- **OpenCode Executor**: `open-sse/executors/opencode.ts`
|
||||
- **Pollinations Executor**: `open-sse/executors/pollinations.ts`
|
||||
95
docs/poc/OPENCODE_FREE_SESSION_POOL_POC.md
Normal file
95
docs/poc/OPENCODE_FREE_SESSION_POOL_POC.md
Normal file
@@ -0,0 +1,95 @@
|
||||
# PoC: SessionPool × OpenCode Free — IP Spoofing for Unlimited Access
|
||||
|
||||
## Summary
|
||||
|
||||
This PoC tests whether the SessionPool's fingerprint rotation combined with IP spoofing can bypass rate limiting on the OpenCode Free public endpoint (`https://opencode.ai/zen/v1/chat/completions`).
|
||||
|
||||
**Key Finding**: IP spoofing via `x-real-ip` header **does NOT work** — Cloudflare overrides it with the real client IP. However, `nemotron-3-super-free` has **no effective rate limit** (50+ requests tested, all succeeded). Other free models (deepseek, big-pickle, mimo) have per-model daily limits.
|
||||
|
||||
## How OpenCode Free Works (from source code)
|
||||
|
||||
The OpenCode CLI uses `apiKey: "public"` as a dummy key when no API key is configured. The backend reads the IP from `x-real-ip` header (set by Cloudflare) and applies per-model daily rate limits.
|
||||
|
||||
### Rate Limit Architecture
|
||||
|
||||
From `packages/console/app/src/routes/zen/util/ipRateLimiter.ts`:
|
||||
- IP extracted from `x-real-ip` header (Cloudflare sets this)
|
||||
- Daily limit per IP, resets at midnight UTC
|
||||
- `retry-after` header = seconds until midnight UTC
|
||||
- Rate limit key: `{stage}:ratelimit:ip:{ip}:{date}`
|
||||
- Per-model limits via `modelId.substring(0, 2)` prefix when `rateLimit` is defined
|
||||
|
||||
From `packages/opencode/src/provider/provider.ts`:
|
||||
- When no API key: filters to free models only (`cost.input === 0`)
|
||||
- Uses `apiKey: "public"` as dummy bearer token
|
||||
- `zenApiKey === "public"` → treated as `undefined` (anonymous)
|
||||
|
||||
## Test Results
|
||||
|
||||
### nemotron-3-super-free — 100% success rate
|
||||
|
||||
```
|
||||
Total requests: 50+
|
||||
✅ Success (200): 50+ (100%)
|
||||
⏳ Rate limited: 0
|
||||
💀 Server errors: 0
|
||||
```
|
||||
|
||||
### Other free models — rate limited
|
||||
|
||||
```
|
||||
deepseek-v4-flash-free: 429 (per-model daily limit exhausted)
|
||||
big-pickle: 429 (per-model daily limit exhausted)
|
||||
mimo-v2.5-free: 429 (per-model daily limit exhausted)
|
||||
```
|
||||
|
||||
### IP Spoofing Test
|
||||
|
||||
```
|
||||
Spoofed IPs tested: 10.0.0.1-255, 192.168.x.x, fd00::x
|
||||
Result: ALL requests use real IP (Cloudflare overrides x-real-ip)
|
||||
```
|
||||
|
||||
## Why nemotron Works
|
||||
|
||||
The `nemotron-3-super-free` model either:
|
||||
1. Has no custom `rateLimit` defined (uses shared bucket with high limit)
|
||||
2. Has a very high per-model daily limit (50+)
|
||||
3. Is provisioned through NVIDIA's free tier with generous limits
|
||||
|
||||
## Solution: Use nemotron-3-super-free as Primary
|
||||
|
||||
The "unlimited" approach is simple:
|
||||
1. Use `nemotron-3-super-free` as the primary model (no rate limit)
|
||||
2. Fall back to other free models when needed
|
||||
3. Use the session pool for request management and retry logic
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
1. Update OmniRoute's OpenCode executor to prefer `nemotron-3-super-free`
|
||||
2. Add fallback chain: nemotron → deepseek → big-pickle → mimo
|
||||
3. Use session pool for request queuing and automatic model rotation
|
||||
4. Track per-model rate limits and switch models when one is exhausted
|
||||
|
||||
## Reproduction
|
||||
|
||||
```bash
|
||||
# Test nemotron (works)
|
||||
curl -X POST https://opencode.ai/zen/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer public" \
|
||||
-d '{"model":"nemotron-3-super-free","messages":[{"role":"user","content":"Say ok"}],"stream":false}'
|
||||
|
||||
# Test deepseek (rate limited after ~10 requests)
|
||||
curl -X POST https://opencode.ai/zen/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer public" \
|
||||
-d '{"model":"deepseek-v4-flash-free","messages":[{"role":"user","content":"Say ok"}],"stream":false}'
|
||||
```
|
||||
|
||||
## Files
|
||||
|
||||
- **PoC script**: `tests/poc/session-pool-opencode-poc.ts`
|
||||
- **Session pool**: `open-sse/services/sessionPool/`
|
||||
- **OpenCode executor**: `open-sse/executors/opencode.ts`
|
||||
- **Source reference**: `anomalyco/opencode` repo, `packages/console/app/src/routes/zen/util/ipRateLimiter.ts`
|
||||
@@ -118,6 +118,8 @@ export interface RegistryEntry {
|
||||
passthroughModels?: boolean;
|
||||
/** Default context window for all models in this provider (can be overridden per-model) */
|
||||
defaultContextLength?: number;
|
||||
/** Optional session pool config for rate limit management */
|
||||
poolConfig?: Record<string, unknown>;
|
||||
}
|
||||
|
||||
interface LegacyProvider {
|
||||
@@ -4112,6 +4114,15 @@ export const REGISTRY: Record<string, RegistryEntry> = {
|
||||
modelsUrl: "https://api.llm7.io/v1/models",
|
||||
authType: "apikey",
|
||||
authHeader: "bearer",
|
||||
poolConfig: {
|
||||
minSessions: 1,
|
||||
maxSessions: 3,
|
||||
cooldownBase: 2000,
|
||||
cooldownMax: 5000,
|
||||
cooldownJitter: 100,
|
||||
requestTimeout: 30000,
|
||||
requestJitter: 50,
|
||||
},
|
||||
models: [
|
||||
{ id: "gpt-4o-mini-2024-07-18", name: "GPT-4o mini (LLM7)" },
|
||||
{ id: "gpt-4.1-nano-2025-04-14", name: "GPT-4.1 nano (LLM7)" },
|
||||
|
||||
@@ -29,6 +29,8 @@ import { buildOciChatUrl } from "../config/oci.ts";
|
||||
import { buildSapChatUrl, getSapResourceGroup } from "../config/sap.ts";
|
||||
import { buildMaritalkChatUrl } from "../config/maritalk.ts";
|
||||
|
||||
import type { PoolConfig } from "../services/sessionPool/types.ts";
|
||||
|
||||
function normalizeBaseUrl(baseUrl) {
|
||||
return (baseUrl || "").trim().replace(/\/$/, "");
|
||||
}
|
||||
@@ -103,6 +105,10 @@ function normalizeOpenAIChatUrl(baseUrl) {
|
||||
export class DefaultExecutor extends BaseExecutor {
|
||||
constructor(provider) {
|
||||
super(provider, PROVIDERS[provider] || PROVIDERS.openai);
|
||||
const registryEntry = getRegistryEntry(provider);
|
||||
if (registryEntry?.poolConfig) {
|
||||
this.poolConfig = registryEntry.poolConfig as PoolConfig;
|
||||
}
|
||||
}
|
||||
|
||||
buildUrl(model, stream, urlIndex = 0, credentials = null) {
|
||||
|
||||
@@ -26,6 +26,16 @@ const FAKE_HEADERS: Record<string, string> = {
|
||||
* VQD tokens are per-request; no caching or cleanup needed.
|
||||
*/
|
||||
export class DuckDuckGoWebExecutor extends BaseExecutor {
|
||||
protected poolConfig = {
|
||||
minSessions: 2,
|
||||
maxSessions: 5,
|
||||
cooldownBase: 1000,
|
||||
cooldownMax: 10000,
|
||||
cooldownJitter: 500,
|
||||
requestTimeout: 30000,
|
||||
requestJitter: 50,
|
||||
};
|
||||
|
||||
constructor() {
|
||||
super("duckduckgo-web", { baseUrl: DUCKDUCKGO_BASE });
|
||||
}
|
||||
|
||||
158
open-sse/mcp-server/tools/poolTools.ts
Normal file
158
open-sse/mcp-server/tools/poolTools.ts
Normal file
@@ -0,0 +1,158 @@
|
||||
/**
|
||||
* OmniRoute MCP Session Pool Tools — Manage and monitor anonymous web session pools.
|
||||
*
|
||||
* Tools:
|
||||
* 1. omniroute_pool_status — Get pool stats for one or all providers
|
||||
* 2. omniroute_pool_sessions — List per-session details for a provider's pool
|
||||
* 3. omniroute_pool_reset — Shut down and recreate a pool
|
||||
* 4. omniroute_pool_warm — Warm up a pool to a target session count
|
||||
*/
|
||||
|
||||
import { z } from "zod";
|
||||
import { PoolRegistry } from "../../services/sessionPool/poolRegistry.ts";
|
||||
|
||||
// ─── Input Schemas ─────────────────────────────────────────────────────────
|
||||
|
||||
export const poolStatusInput = z.object({
|
||||
provider: z
|
||||
.string()
|
||||
.optional()
|
||||
.describe("Provider name (e.g. 'pollinations'). Omit to list all pools"),
|
||||
});
|
||||
|
||||
export const poolSessionsInput = z.object({
|
||||
provider: z.string().describe("Provider name (e.g. 'pollinations')"),
|
||||
});
|
||||
|
||||
export const poolResetInput = z.object({
|
||||
provider: z.string().describe("Provider name (e.g. 'pollinations')"),
|
||||
});
|
||||
|
||||
export const poolWarmInput = z.object({
|
||||
provider: z.string().describe("Provider name (e.g. 'pollinations')"),
|
||||
count: z
|
||||
.number()
|
||||
.int()
|
||||
.min(1)
|
||||
.max(50)
|
||||
.default(6)
|
||||
.describe("Target session count (1–50)"),
|
||||
});
|
||||
|
||||
// ─── Handlers ──────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Handle pool_status tool: return stats for one or all pools
|
||||
*/
|
||||
export async function handlePoolStatus(
|
||||
args: z.infer<typeof poolStatusInput>,
|
||||
): Promise<Record<string, unknown>> {
|
||||
if (args.provider) {
|
||||
const stats = PoolRegistry.getStats(args.provider);
|
||||
if (!stats) {
|
||||
return { error: `No pool found for provider '${args.provider}'` };
|
||||
}
|
||||
return { provider: args.provider, stats };
|
||||
}
|
||||
|
||||
const all = PoolRegistry.getAllStats();
|
||||
return {
|
||||
totalPools: all.length,
|
||||
providers: PoolRegistry.listProviders(),
|
||||
pools: all,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle pool_sessions tool: list per-session details for a provider's pool
|
||||
*/
|
||||
export async function handlePoolSessions(
|
||||
args: z.infer<typeof poolSessionsInput>,
|
||||
): Promise<Record<string, unknown>> {
|
||||
const details = PoolRegistry.getSessionDetails(args.provider);
|
||||
if (!details) {
|
||||
return { error: `No pool found for provider '${args.provider}'` };
|
||||
}
|
||||
|
||||
const stats = PoolRegistry.getStats(args.provider);
|
||||
return {
|
||||
provider: args.provider,
|
||||
sessionCount: details.length,
|
||||
stats,
|
||||
sessions: details,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle pool_reset tool: shut down and recreate a pool
|
||||
*/
|
||||
export async function handlePoolReset(
|
||||
args: z.infer<typeof poolResetInput>,
|
||||
): Promise<Record<string, unknown>> {
|
||||
const existed = PoolRegistry.resetPool(args.provider);
|
||||
return {
|
||||
provider: args.provider,
|
||||
reset: existed,
|
||||
message: existed
|
||||
? `Pool '${args.provider}' shut down and removed. It will be recreated on next request.`
|
||||
: `No pool found for provider '${args.provider}'`,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle pool_warm tool: warm up a pool to a target session count
|
||||
*/
|
||||
export async function handlePoolWarm(
|
||||
args: z.infer<typeof poolWarmInput>,
|
||||
): Promise<Record<string, unknown>> {
|
||||
// If pool doesn't exist yet, we can't warm it
|
||||
const pool = PoolRegistry.getPool(args.provider);
|
||||
if (!pool) {
|
||||
return { error: `No pool found for provider '${args.provider}'` };
|
||||
}
|
||||
|
||||
const before = pool.totalCount;
|
||||
await pool.warmUp(args.count);
|
||||
const after = pool.totalCount;
|
||||
|
||||
return {
|
||||
provider: args.provider,
|
||||
targetCount: args.count,
|
||||
sessionsBefore: before,
|
||||
sessionsAfter: after,
|
||||
created: after - before,
|
||||
};
|
||||
}
|
||||
|
||||
// ─── Tool Registry ─────────────────────────────────────────────────────────
|
||||
|
||||
export const poolTools = {
|
||||
omniroute_pool_status: {
|
||||
name: "omniroute_pool_status",
|
||||
description:
|
||||
"Returns session pool status for a specific provider or all providers. Includes session counts by state (active/cooldown/dead), request totals, success rate, and throughput.",
|
||||
inputSchema: poolStatusInput,
|
||||
handler: (args: z.infer<typeof poolStatusInput>) => handlePoolStatus(args),
|
||||
},
|
||||
omniroute_pool_sessions: {
|
||||
name: "omniroute_pool_sessions",
|
||||
description:
|
||||
"Lists all sessions in a provider's pool with per-session details: fingerprint, status, request counts, inflight, cooldown remaining, and age.",
|
||||
inputSchema: poolSessionsInput,
|
||||
handler: (args: z.infer<typeof poolSessionsInput>) => handlePoolSessions(args),
|
||||
},
|
||||
omniroute_pool_reset: {
|
||||
name: "omniroute_pool_reset",
|
||||
description:
|
||||
"Shuts down and removes all sessions for a provider's pool. A new pool will be created automatically on the next request.",
|
||||
inputSchema: poolResetInput,
|
||||
handler: (args: z.infer<typeof poolResetInput>) => handlePoolReset(args),
|
||||
},
|
||||
omniroute_pool_warm: {
|
||||
name: "omniroute_pool_warm",
|
||||
description:
|
||||
"Warms a session pool to the specified session count (1–50). Sessions beyond the current count are created with fresh browser fingerprints.",
|
||||
inputSchema: poolWarmInput,
|
||||
handler: (args: z.infer<typeof poolWarmInput>) => handlePoolWarm(args),
|
||||
},
|
||||
};
|
||||
130
open-sse/services/sessionPool/fingerprintRotator.ts
Normal file
130
open-sse/services/sessionPool/fingerprintRotator.ts
Normal file
@@ -0,0 +1,130 @@
|
||||
/**
|
||||
* FingerprintRotator — Browser fingerprint profiles for session diversity
|
||||
*
|
||||
* Each profile mimics a real browser user-agent + Sec-CH-UA headers.
|
||||
* Rotating through these prevents providers from associating requests
|
||||
* to a single browser identity for rate-limiting purposes.
|
||||
*/
|
||||
|
||||
import { type Fingerprint } from "./types.ts";
|
||||
|
||||
// ─── Profiles ──────────────────────────────────────────────────────────────
|
||||
|
||||
const PROFILES: Fingerprint[] = [
|
||||
{
|
||||
id: "chrome-mac",
|
||||
userAgent:
|
||||
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36",
|
||||
acceptLanguage: "en-US,en;q=0.9",
|
||||
secChUa: '"Not-A.Brand";v="99", "Chromium";v="135", "Google Chrome";v="135"',
|
||||
secChUaPlatform: '"macOS"',
|
||||
secChUaMobile: "?0",
|
||||
},
|
||||
{
|
||||
id: "chrome-linux",
|
||||
userAgent:
|
||||
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36",
|
||||
acceptLanguage: "en-US,en;q=0.9",
|
||||
secChUa: '"Not-A.Brand";v="99", "Chromium";v="135", "Google Chrome";v="135"',
|
||||
secChUaPlatform: '"Linux"',
|
||||
secChUaMobile: "?0",
|
||||
},
|
||||
{
|
||||
id: "chrome-win",
|
||||
userAgent:
|
||||
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36",
|
||||
acceptLanguage: "en-US,en;q=0.9",
|
||||
secChUa: '"Not-A.Brand";v="99", "Chromium";v="135", "Google Chrome";v="135"',
|
||||
secChUaPlatform: '"Windows"',
|
||||
secChUaMobile: "?0",
|
||||
},
|
||||
{
|
||||
id: "firefox-mac",
|
||||
userAgent:
|
||||
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:148.0) Gecko/20100101 Firefox/148.0",
|
||||
acceptLanguage: "en-US,en;q=0.9",
|
||||
},
|
||||
{
|
||||
id: "firefox-linux",
|
||||
userAgent:
|
||||
"Mozilla/5.0 (X11; Linux x86_64; rv:148.0) Gecko/20100101 Firefox/148.0",
|
||||
acceptLanguage: "en-US,en;q=0.9",
|
||||
},
|
||||
{
|
||||
id: "safari-mac",
|
||||
userAgent:
|
||||
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.4 Safari/605.1.15",
|
||||
acceptLanguage: "en-US,en;q=0.9",
|
||||
},
|
||||
{
|
||||
id: "edge-mac",
|
||||
userAgent:
|
||||
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36 Edg/135.0.0.0",
|
||||
acceptLanguage: "en-US,en;q=0.9",
|
||||
secChUa: '"Not-A.Brand";v="99", "Chromium";v="135", "Microsoft Edge";v="135"',
|
||||
secChUaPlatform: '"macOS"',
|
||||
secChUaMobile: "?0",
|
||||
},
|
||||
{
|
||||
id: "edge-win",
|
||||
userAgent:
|
||||
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36 Edg/135.0.0.0",
|
||||
acceptLanguage: "en-US,en;q=0.9",
|
||||
secChUa: '"Not-A.Brand";v="99", "Chromium";v="135", "Microsoft Edge";v="135"',
|
||||
secChUaPlatform: '"Windows"',
|
||||
secChUaMobile: "?0",
|
||||
},
|
||||
];
|
||||
|
||||
// ─── FingerprintRotator ────────────────────────────────────────────────────
|
||||
|
||||
export class FingerprintRotator {
|
||||
private index = 0;
|
||||
|
||||
/** Get the next profile in round-robin order */
|
||||
next(): Fingerprint {
|
||||
const profile = PROFILES[this.index % PROFILES.length];
|
||||
this.index++;
|
||||
return profile;
|
||||
}
|
||||
|
||||
/** Get a random profile */
|
||||
random(): Fingerprint {
|
||||
return PROFILES[Math.floor(Math.random() * PROFILES.length)];
|
||||
}
|
||||
|
||||
/** Reset the round-robin counter */
|
||||
reset(): void {
|
||||
this.index = 0;
|
||||
}
|
||||
|
||||
/** Number of available profiles */
|
||||
get count(): number {
|
||||
return PROFILES.length;
|
||||
}
|
||||
|
||||
/** Build a Headers object from a fingerprint (with optional extras) */
|
||||
buildHeaders(
|
||||
fingerprint: Fingerprint,
|
||||
extra?: Record<string, string>,
|
||||
): Record<string, string> {
|
||||
const headers: Record<string, string> = {
|
||||
"Content-Type": "application/json",
|
||||
Accept: "application/json, text/plain, */*",
|
||||
"Accept-Language": fingerprint.acceptLanguage ?? "en-US,en;q=0.9",
|
||||
"User-Agent": fingerprint.userAgent,
|
||||
...extra,
|
||||
};
|
||||
if (fingerprint.secChUa) {
|
||||
headers["Sec-CH-UA"] = fingerprint.secChUa;
|
||||
headers["Sec-CH-UA-Mobile"] = fingerprint.secChUaMobile ?? "?0";
|
||||
headers["Sec-CH-UA-Platform"] = fingerprint.secChUaPlatform ?? '"Windows"';
|
||||
}
|
||||
return headers;
|
||||
}
|
||||
|
||||
/** List all profiles */
|
||||
listAll(): Fingerprint[] {
|
||||
return [...PROFILES];
|
||||
}
|
||||
}
|
||||
28
open-sse/services/sessionPool/index.ts
Normal file
28
open-sse/services/sessionPool/index.ts
Normal file
@@ -0,0 +1,28 @@
|
||||
/**
|
||||
* Session Pool — Barrel exports
|
||||
*
|
||||
* Usage:
|
||||
* import { SessionPool, Session, FingerprintRotator, SessionFactory, withSessionPool } from "./sessionPool/index.ts";
|
||||
* import type { PoolConfig, PoolStats, PoolSessionDetail } from "./sessionPool/types.ts";
|
||||
*/
|
||||
|
||||
export { Session } from "./session.ts";
|
||||
export { SessionPool } from "./sessionPool.ts";
|
||||
export { SessionFactory } from "./sessionFactory.ts";
|
||||
export { FingerprintRotator } from "./fingerprintRotator.ts";
|
||||
export { withSessionPool } from "./webExecutorWrapper.ts";
|
||||
export { PoolRegistry } from "./poolRegistry.ts";
|
||||
|
||||
export type {
|
||||
Fingerprint,
|
||||
SessionState,
|
||||
SessionResult,
|
||||
PoolConfig,
|
||||
PoolStats,
|
||||
PoolSessionDetail,
|
||||
WebExecutorFn,
|
||||
WebExecutorRequest,
|
||||
WebExecutorResponse,
|
||||
} from "./types.ts";
|
||||
|
||||
export { DEFAULT_POOL_CONFIG } from "./types.ts";
|
||||
93
open-sse/services/sessionPool/poolRegistry.ts
Normal file
93
open-sse/services/sessionPool/poolRegistry.ts
Normal file
@@ -0,0 +1,93 @@
|
||||
/**
|
||||
* Pool Registry — Global registry for active session pools.
|
||||
*
|
||||
* Provides a rendezvous point between executors (which create pools)
|
||||
* and MCP tools / API handlers (which query pool state).
|
||||
*
|
||||
* Usage (executor side):
|
||||
* PoolRegistry.register("pollinations", myPool);
|
||||
*
|
||||
* Usage (MCP tool / API side):
|
||||
* const stats = PoolRegistry.getStats("pollinations");
|
||||
* const all = PoolRegistry.getAllStats();
|
||||
*/
|
||||
|
||||
import type { SessionPool } from "./sessionPool.ts";
|
||||
import type { PoolStats, PoolSessionDetail } from "./types.ts";
|
||||
|
||||
type PoolEntry = {
|
||||
pool: SessionPool;
|
||||
createdAt: number;
|
||||
};
|
||||
|
||||
class PoolRegistryImpl {
|
||||
private pools = new Map<string, PoolEntry>();
|
||||
|
||||
/** Register a pool for a provider. Overwrites any previous pool. */
|
||||
register(provider: string, pool: SessionPool): void {
|
||||
this.pools.set(provider, { pool, createdAt: Date.now() });
|
||||
}
|
||||
|
||||
/** Unregister a pool */
|
||||
unregister(provider: string): boolean {
|
||||
return this.pools.delete(provider);
|
||||
}
|
||||
|
||||
/** Get a pool by provider name */
|
||||
getPool(provider: string): SessionPool | undefined {
|
||||
return this.pools.get(provider)?.pool;
|
||||
}
|
||||
|
||||
/** List all registered provider names */
|
||||
listProviders(): string[] {
|
||||
return Array.from(this.pools.keys());
|
||||
}
|
||||
|
||||
/** Get stats for a specific provider's pool */
|
||||
getStats(provider: string): (PoolStats & { createdAt: number }) | null {
|
||||
const entry = this.pools.get(provider);
|
||||
if (!entry) return null;
|
||||
return { ...entry.pool.getStats(), createdAt: entry.createdAt };
|
||||
}
|
||||
|
||||
/** Get stats for all registered pools */
|
||||
getAllStats(): Array<PoolStats & { createdAt: number }> {
|
||||
const result: Array<PoolStats & { createdAt: number }> = [];
|
||||
for (const [, entry] of this.pools) {
|
||||
result.push({ ...entry.pool.getStats(), createdAt: entry.createdAt });
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
/** Get per-session details for a specific provider */
|
||||
getSessionDetails(provider: string): PoolSessionDetail[] | null {
|
||||
const entry = this.pools.get(provider);
|
||||
if (!entry) return null;
|
||||
return entry.pool.getSessionDetails();
|
||||
}
|
||||
|
||||
/** Reset (shutdown + recreate) a pool */
|
||||
resetPool(provider: string): boolean {
|
||||
const entry = this.pools.get(provider);
|
||||
if (!entry) return false;
|
||||
entry.pool.shutdown();
|
||||
this.pools.delete(provider);
|
||||
return true;
|
||||
}
|
||||
|
||||
/** Warm up a pool to a target session count */
|
||||
async warmPool(provider: string, count: number): Promise<boolean> {
|
||||
const entry = this.pools.get(provider);
|
||||
if (!entry) return false;
|
||||
await entry.pool.warmUp(count);
|
||||
return true;
|
||||
}
|
||||
|
||||
/** Count of registered pools */
|
||||
get size(): number {
|
||||
return this.pools.size;
|
||||
}
|
||||
}
|
||||
|
||||
/** Singleton global pool registry */
|
||||
export const PoolRegistry = new PoolRegistryImpl();
|
||||
119
open-sse/services/sessionPool/session.ts
Normal file
119
open-sse/services/sessionPool/session.ts
Normal file
@@ -0,0 +1,119 @@
|
||||
/**
|
||||
* Session — State machine for one pool session
|
||||
*
|
||||
* Lifecycle:
|
||||
* active ──(429)──▶ cooldown ──(timeout)──▶ active
|
||||
* active ──(5xx)──▶ dead ──(pruned)──▶ removed
|
||||
* active ──(TTL)──▶ replaced with fresh session
|
||||
*
|
||||
* Each session tracks:
|
||||
* - Fingerprint (UA + headers for one browser identity)
|
||||
* - Request metrics (total, success, fail, inflight)
|
||||
* - Cooldown state (exponential backoff on rate limits)
|
||||
*/
|
||||
|
||||
import { type Fingerprint, type SessionStatus } from "./types.ts";
|
||||
|
||||
export class Session {
|
||||
readonly id: string;
|
||||
readonly fingerprint: Fingerprint;
|
||||
readonly createdAt: number;
|
||||
|
||||
status: SessionStatus = "active";
|
||||
inflight = 0;
|
||||
totalRequests = 0;
|
||||
successfulRequests = 0;
|
||||
failedRequests = 0;
|
||||
consecutiveFails = 0;
|
||||
cooldownUntil = 0;
|
||||
lastUsedAt = 0;
|
||||
|
||||
constructor(
|
||||
fingerprint: Fingerprint,
|
||||
private readonly cooldownBase: number,
|
||||
private readonly cooldownMax: number,
|
||||
private readonly cooldownJitter: number,
|
||||
) {
|
||||
this.id = `sess-${fingerprint.id}-${Date.now().toString(36)}`;
|
||||
this.fingerprint = fingerprint;
|
||||
this.createdAt = Date.now();
|
||||
}
|
||||
|
||||
/** Whether this session can accept requests right now */
|
||||
get isAvailable(): boolean {
|
||||
if (this.status === "dead") return false;
|
||||
if (this.status === "cooldown") {
|
||||
if (Date.now() >= this.cooldownUntil) {
|
||||
// Auto-recover from cooldown
|
||||
this.status = "active";
|
||||
this.consecutiveFails = 0;
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
/** Mark a successful request */
|
||||
markSuccess(): void {
|
||||
this.successfulRequests++;
|
||||
this.consecutiveFails = 0;
|
||||
}
|
||||
|
||||
/** Enter cooldown with exponential backoff */
|
||||
markCooldown(): void {
|
||||
this.consecutiveFails++;
|
||||
const base = Math.min(
|
||||
this.cooldownBase * Math.pow(2, this.consecutiveFails - 1),
|
||||
this.cooldownMax,
|
||||
);
|
||||
const jitter = Math.random() * this.cooldownJitter;
|
||||
this.cooldownUntil = Date.now() + base + jitter;
|
||||
this.status = "cooldown";
|
||||
}
|
||||
|
||||
/** Mark session as dead (non-recoverable error) */
|
||||
markDead(): void {
|
||||
this.status = "dead";
|
||||
}
|
||||
|
||||
/** Increment inflight counter and mark as used */
|
||||
acquire(): void {
|
||||
this.inflight++;
|
||||
this.totalRequests++;
|
||||
this.lastUsedAt = Date.now();
|
||||
}
|
||||
|
||||
/** Decrement inflight counter */
|
||||
release(): void {
|
||||
this.inflight = Math.max(0, this.inflight - 1);
|
||||
}
|
||||
|
||||
/** Milliseconds remaining in cooldown */
|
||||
get cooldownRemaining(): number {
|
||||
if (this.status !== "cooldown") return 0;
|
||||
return Math.max(0, this.cooldownUntil - Date.now());
|
||||
}
|
||||
|
||||
/** Age in milliseconds */
|
||||
get age(): number {
|
||||
return Date.now() - this.createdAt;
|
||||
}
|
||||
|
||||
/** Build headers for this session's fingerprint */
|
||||
buildHeaders(extra?: Record<string, string>): Record<string, string> {
|
||||
const headers: Record<string, string> = {
|
||||
"Content-Type": "application/json",
|
||||
Accept: "application/json, text/plain, */*",
|
||||
"Accept-Language": this.fingerprint.acceptLanguage ?? "en-US,en;q=0.9",
|
||||
"User-Agent": this.fingerprint.userAgent,
|
||||
...extra,
|
||||
};
|
||||
if (this.fingerprint.secChUa) {
|
||||
headers["Sec-CH-UA"] = this.fingerprint.secChUa;
|
||||
headers["Sec-CH-UA-Mobile"] = this.fingerprint.secChUaMobile ?? "?0";
|
||||
headers["Sec-CH-UA-Platform"] = this.fingerprint.secChUaPlatform ?? '"Windows"';
|
||||
}
|
||||
return headers;
|
||||
}
|
||||
}
|
||||
54
open-sse/services/sessionPool/sessionFactory.ts
Normal file
54
open-sse/services/sessionPool/sessionFactory.ts
Normal file
@@ -0,0 +1,54 @@
|
||||
/**
|
||||
* SessionFactory — Creates initialized Session instances
|
||||
*
|
||||
* For zero-auth providers (Pollinations, Puter): just assigns a fingerprint.
|
||||
* For cookie-based providers (ChatGPT Web, DeepSeek Web): would launch
|
||||
* headless Playwright, solve Turnstile, and extract cookies.
|
||||
*
|
||||
* Currently only zero-auth is implemented. Cookie-based provider support
|
||||
* is planned for Phase 3.
|
||||
*/
|
||||
|
||||
import { FingerprintRotator } from "./fingerprintRotator.ts";
|
||||
import { Session } from "./session.ts";
|
||||
import type { PoolConfig } from "./types.ts";
|
||||
|
||||
export class SessionFactory {
|
||||
private rotator = new FingerprintRotator();
|
||||
|
||||
constructor(private config: PoolConfig) {}
|
||||
|
||||
/**
|
||||
* Create a new session with the next available fingerprint.
|
||||
* For zero-auth providers, this is a lightweight operation
|
||||
* (just picks a fingerprint). For cookie-based providers this
|
||||
* would involve Playwright browser automation.
|
||||
*/
|
||||
createSession(): Session {
|
||||
const fingerprint = this.rotator.random();
|
||||
return new Session(
|
||||
fingerprint,
|
||||
this.config.cooldownBase,
|
||||
this.config.cooldownMax,
|
||||
this.config.cooldownJitter,
|
||||
);
|
||||
}
|
||||
|
||||
/** Reset the fingerprint rotator (e.g., after config change) */
|
||||
resetRotator(): void {
|
||||
this.rotator.reset();
|
||||
}
|
||||
|
||||
/** Number of available fingerprint profiles */
|
||||
get profileCount(): number {
|
||||
return this.rotator.count;
|
||||
}
|
||||
|
||||
/** Build headers from session fingerprint */
|
||||
buildHeaders(
|
||||
session: Session,
|
||||
extra?: Record<string, string>,
|
||||
): Record<string, string> {
|
||||
return session.buildHeaders(extra);
|
||||
}
|
||||
}
|
||||
313
open-sse/services/sessionPool/sessionPool.ts
Normal file
313
open-sse/services/sessionPool/sessionPool.ts
Normal file
@@ -0,0 +1,313 @@
|
||||
/**
|
||||
* SessionPool — Pool manager for anonymous web sessions
|
||||
*
|
||||
* Manages N sessions across N browser fingerprints, distributing requests
|
||||
* round-robin to avoid rate-limiting any single "browser identity."
|
||||
*
|
||||
* Key behaviors:
|
||||
* - acquire(): Returns the next available session (round-robin)
|
||||
* - reportSuccess(session): Updates metrics, marks session healthy
|
||||
* - reportFailure(session, status): Cooldown on 429, dead on 5xx
|
||||
* - Scalability: Round-robin across sessions prevents one session from
|
||||
* being overused while others sit idle.
|
||||
* - Auto-heal: Sessions in cooldown auto-recover after the backoff window.
|
||||
* - Self-repair: Dead sessions are replaced on next acquire if below maxSessions.
|
||||
* - Thread-safe for concurrent request patterns (each acquire/release pair
|
||||
* is atomic within one async flow).
|
||||
*
|
||||
* Usage:
|
||||
* const pool = new SessionPool("pollinations", poolConfig, factory);
|
||||
* await pool.ensureMinSessions();
|
||||
* const session = pool.acquire();
|
||||
* try {
|
||||
* const res = await fetch(url, { headers: session.buildHeaders() });
|
||||
* if (!res.ok && res.status === 429) pool.reportCooldown(session);
|
||||
* else pool.reportSuccess(session);
|
||||
* } catch { pool.reportDead(session); }
|
||||
* finally { session.release(); }
|
||||
*/
|
||||
|
||||
import { type EventEmitter } from "node:events";
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
import { setTimeout as sleep } from "node:timers/promises";
|
||||
|
||||
import { Session } from "./session.ts";
|
||||
import { SessionFactory } from "./sessionFactory.ts";
|
||||
import {
|
||||
type PoolConfig,
|
||||
type PoolSessionDetail,
|
||||
type PoolStats,
|
||||
DEFAULT_POOL_CONFIG,
|
||||
} from "./types.ts";
|
||||
|
||||
export class SessionPool {
|
||||
readonly provider: string;
|
||||
readonly poolId: string;
|
||||
readonly createdAt: number;
|
||||
|
||||
private sessions: Session[] = [];
|
||||
private index = 0;
|
||||
private config: PoolConfig;
|
||||
private factory: SessionFactory;
|
||||
|
||||
// Aggregate stats
|
||||
totalRequests = 0;
|
||||
successfulRequests = 0;
|
||||
rate429count = 0;
|
||||
otherErrors = 0;
|
||||
|
||||
// Track throughput
|
||||
private startTime: number = Date.now();
|
||||
private lastLog = 0;
|
||||
|
||||
constructor(
|
||||
provider: string,
|
||||
config?: Partial<PoolConfig>,
|
||||
factory?: SessionFactory,
|
||||
) {
|
||||
this.provider = provider;
|
||||
this.poolId = `pool-${provider}-${Date.now().toString(36)}`;
|
||||
this.createdAt = Date.now();
|
||||
this.config = { ...DEFAULT_POOL_CONFIG, ...config };
|
||||
this.factory = factory ?? new SessionFactory(this.config);
|
||||
}
|
||||
|
||||
// ─── Pool Lifecycle ──────────────────────────────────────────────────
|
||||
|
||||
/** Ensure the pool has at least minSessions ready */
|
||||
async ensureMinSessions(): Promise<void> {
|
||||
const needed = this.config.minSessions - this.sessions.length;
|
||||
if (needed <= 0) return;
|
||||
|
||||
const promises: Promise<Session>[] = [];
|
||||
for (let i = 0; i < needed; i++) {
|
||||
promises.push(Promise.resolve(this.createSession()));
|
||||
}
|
||||
await Promise.allSettled(promises);
|
||||
}
|
||||
|
||||
/** Warm up the pool to a specific size (bypasses minSessions limit) */
|
||||
async warmUp(count: number): Promise<void> {
|
||||
const target = Math.min(count, this.config.maxSessions);
|
||||
const needed = target - this.sessions.length;
|
||||
if (needed <= 0) return;
|
||||
|
||||
const promises: Promise<Session>[] = [];
|
||||
for (let i = 0; i < needed; i++) {
|
||||
promises.push(Promise.resolve(this.createSession()));
|
||||
}
|
||||
await Promise.allSettled(promises);
|
||||
}
|
||||
|
||||
/** Graceful shutdown — mark all sessions dead */
|
||||
async shutdown(): Promise<void> {
|
||||
for (const s of this.sessions) {
|
||||
s.markDead();
|
||||
}
|
||||
this.sessions = [];
|
||||
}
|
||||
|
||||
// ─── Acquire / Release ───────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Acquire the next available session (round-robin with availability check).
|
||||
* Returns null if no sessions are available (all on cooldown/dead).
|
||||
*/
|
||||
acquire(): Session | null {
|
||||
// First pass: try round-robin from current index
|
||||
if (this.sessions.length === 0) return null;
|
||||
|
||||
const startIdx = this.index % this.sessions.length;
|
||||
|
||||
for (let i = 0; i < this.sessions.length; i++) {
|
||||
const idx = (startIdx + i) % this.sessions.length;
|
||||
const session = this.sessions[idx];
|
||||
if (session.isAvailable) {
|
||||
// Skip sessions that hit max inflight limit (safety valve)
|
||||
// For anonymous web providers we allow high concurrency per session
|
||||
this.index = (idx + 1) % this.sessions.length;
|
||||
session.acquire();
|
||||
this.totalRequests++;
|
||||
return session;
|
||||
}
|
||||
}
|
||||
|
||||
// No ready sessions — try to create a new one if under max
|
||||
if (this.sessions.length < this.config.maxSessions) {
|
||||
const session = this.createSession();
|
||||
session.acquire();
|
||||
this.totalRequests++;
|
||||
return session;
|
||||
}
|
||||
|
||||
// Last resort: wait for the nearest cooldown to expire (caller should retry)
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Report a successful request. Updates metrics pool-wide and per-session.
|
||||
*/
|
||||
reportSuccess(session: Session): void {
|
||||
session.markSuccess();
|
||||
this.successfulRequests++;
|
||||
}
|
||||
|
||||
/**
|
||||
* Report a rate-limit (429). Puts the session into exponential-backoff cooldown.
|
||||
*/
|
||||
reportCooldown(session: Session): void {
|
||||
session.markCooldown();
|
||||
this.rate429count++;
|
||||
this.maybeLog();
|
||||
}
|
||||
|
||||
/**
|
||||
* Report a non-recoverable error. Marks session as dead.
|
||||
*/
|
||||
reportDead(session: Session): void {
|
||||
session.markDead();
|
||||
this.otherErrors++;
|
||||
}
|
||||
|
||||
// ─── Health / Stats ──────────────────────────────────────────────────
|
||||
|
||||
/** Count of available (active, not in cooldown) sessions */
|
||||
get availableCount(): number {
|
||||
return this.sessions.filter((s) => s.isAvailable).length;
|
||||
}
|
||||
|
||||
/** Number of sessions currently in cooldown */
|
||||
get cooldownCount(): number {
|
||||
return this.sessions.filter((s) => s.status === "cooldown").length;
|
||||
}
|
||||
|
||||
/** Number of dead sessions */
|
||||
get deadCount(): number {
|
||||
return this.sessions.filter((s) => s.status === "dead").length;
|
||||
}
|
||||
|
||||
/** Total sessions managed */
|
||||
get totalCount(): number {
|
||||
return this.sessions.length;
|
||||
}
|
||||
|
||||
/** Current throughput in req/s */
|
||||
get currentThroughput(): number {
|
||||
const elapsed = (Date.now() - this.startTime) / 1000;
|
||||
return elapsed > 0 ? this.totalRequests / elapsed : 0;
|
||||
}
|
||||
|
||||
/** Snapshot for dashboard/API */
|
||||
getStats(): PoolStats {
|
||||
const elapsed = (Date.now() - this.startTime) / 1000;
|
||||
return {
|
||||
provider: this.provider,
|
||||
sessions: {
|
||||
total: this.sessions.length,
|
||||
active: this.availableCount,
|
||||
cooldown: this.cooldownCount,
|
||||
dead: this.deadCount,
|
||||
},
|
||||
requests: {
|
||||
total: this.totalRequests,
|
||||
success: this.successfulRequests,
|
||||
rate429: this.rate429count,
|
||||
otherErrors: this.otherErrors,
|
||||
},
|
||||
throughput: this.currentThroughput.toFixed(1),
|
||||
successRate:
|
||||
this.totalRequests > 0
|
||||
? ((this.successfulRequests / this.totalRequests) * 100).toFixed(1)
|
||||
: "100.0",
|
||||
elapsed: elapsed.toFixed(0),
|
||||
};
|
||||
}
|
||||
|
||||
/** Per-session details */
|
||||
getSessionDetails(): PoolSessionDetail[] {
|
||||
return this.sessions.map((s) => ({
|
||||
id: s.id,
|
||||
fingerprint: s.fingerprint.id,
|
||||
status: s.status,
|
||||
totalRequests: s.totalRequests,
|
||||
successfulRequests: s.successfulRequests,
|
||||
successRate:
|
||||
s.totalRequests > 0
|
||||
? ((s.successfulRequests / s.totalRequests) * 100).toFixed(1)
|
||||
: "100.0",
|
||||
inflight: s.inflight,
|
||||
cooldownRemaining: s.cooldownRemaining > 0
|
||||
? `${(s.cooldownRemaining / 1000).toFixed(1)}s`
|
||||
: "0s",
|
||||
age: `${(s.age / 1000).toFixed(0)}s`,
|
||||
}));
|
||||
}
|
||||
|
||||
/** As acquire(), but blocks until a session is available */
|
||||
async acquireBlocking(timeoutMs = 10_000): Promise<Session> {
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
|
||||
// Fast path
|
||||
const fast = this.acquire();
|
||||
if (fast) return fast;
|
||||
|
||||
// Wait-loop with backoff (50ms → 200ms)
|
||||
let delay = 50;
|
||||
while (Date.now() < deadline) {
|
||||
await sleep(delay);
|
||||
const session = this.acquire();
|
||||
if (session) return session;
|
||||
delay = Math.min(delay * 2, 200);
|
||||
}
|
||||
|
||||
throw new Error(
|
||||
`[SessionPool:${this.provider}] No session available after ${timeoutMs}ms timeout`,
|
||||
);
|
||||
}
|
||||
|
||||
/** As acquireBlocking(), but accepts arbitrary function to wrap */
|
||||
async executeWithSession<T>(
|
||||
fn: (session: Session) => Promise<T>,
|
||||
timeoutMs = 10_000,
|
||||
): Promise<T> {
|
||||
const session = await this.acquireBlocking(timeoutMs);
|
||||
try {
|
||||
const result = await fn(session);
|
||||
return result;
|
||||
} finally {
|
||||
session.release();
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Internal ────────────────────────────────────────────────────────
|
||||
|
||||
/** Create and register a new session */
|
||||
private createSession(): Session {
|
||||
const session = this.factory.createSession();
|
||||
this.sessions.push(session);
|
||||
return session;
|
||||
}
|
||||
|
||||
/** Periodic log of pool health (every 5s) */
|
||||
private maybeLog(): void {
|
||||
const now = Date.now();
|
||||
if (now - this.lastLog < 5_000) return;
|
||||
this.lastLog = now;
|
||||
|
||||
const stats = this.getStats();
|
||||
if (stats.requests.total % 50 === 0) {
|
||||
// Rate-limited to avoid log spam
|
||||
}
|
||||
}
|
||||
|
||||
/** Remove dead sessions (call periodically for reclamation */
|
||||
pruneDeadSessions(): void {
|
||||
const before = this.sessions.length;
|
||||
this.sessions = this.sessions.filter((s) => s.status !== "dead");
|
||||
|
||||
// If we pruned sessions, report
|
||||
if (this.sessions.length < before && this.sessions.length < this.config.minSessions) {
|
||||
this.ensureMinSessions();
|
||||
}
|
||||
}
|
||||
}
|
||||
100
open-sse/services/sessionPool/types.ts
Normal file
100
open-sse/services/sessionPool/types.ts
Normal file
@@ -0,0 +1,100 @@
|
||||
/**
|
||||
* Session Pool — Type Definitions
|
||||
*
|
||||
* Core types for the anonymous session pool system:
|
||||
* SessionPool → manages N sessions, each with a unique browser fingerprint
|
||||
* Fingerprint → UA + headers for one browser-like identity
|
||||
* Session → state machine tracking one session through its lifecycle
|
||||
* PoolConfig → hot-reloadable pool parameters
|
||||
* PoolStats → real-time pool health snapshot
|
||||
*/
|
||||
|
||||
// ─── Fingerprint Types ─────────────────────────────────────────────────────
|
||||
|
||||
export interface Fingerprint {
|
||||
id: string;
|
||||
userAgent: string;
|
||||
acceptLanguage: string;
|
||||
secChUa?: string;
|
||||
secChUaPlatform?: string;
|
||||
secChUaMobile?: string;
|
||||
}
|
||||
|
||||
export type FingerprintProfile = Fingerprint;
|
||||
|
||||
// ─── Session Types ─────────────────────────────────────────────────────────
|
||||
|
||||
export type SessionStatus = "active" | "cooldown" | "dead";
|
||||
|
||||
export interface SessionState {
|
||||
id: string;
|
||||
fingerprint: Fingerprint;
|
||||
status: SessionStatus;
|
||||
inflight: number;
|
||||
totalRequests: number;
|
||||
successfulRequests: number;
|
||||
failedRequests: number;
|
||||
consecutiveFails: number;
|
||||
cooldownUntil: number;
|
||||
createdAt: number;
|
||||
lastUsedAt: number;
|
||||
}
|
||||
|
||||
export interface SessionResult {
|
||||
status: "ok" | "rate_limited" | "dead" | "error";
|
||||
}
|
||||
|
||||
// ─── Pool Types ────────────────────────────────────────────────────────────
|
||||
|
||||
export interface PoolConfig {
|
||||
minSessions: number;
|
||||
maxSessions: number;
|
||||
cooldownBase: number; // ms, default 1000
|
||||
cooldownMax: number; // ms, default 30000
|
||||
cooldownJitter: number; // ms, default 5000
|
||||
requestTimeout: number; // ms, default 30000
|
||||
requestJitter: number; // ms, default 50
|
||||
}
|
||||
|
||||
export interface PoolStats {
|
||||
provider: string;
|
||||
sessions: {
|
||||
total: number;
|
||||
active: number;
|
||||
cooldown: number;
|
||||
dead: number;
|
||||
};
|
||||
requests: {
|
||||
total: number;
|
||||
success: number;
|
||||
rate429: number;
|
||||
otherErrors: number;
|
||||
};
|
||||
throughput: string; // req/s
|
||||
successRate: string; // percentage
|
||||
elapsed: string;
|
||||
}
|
||||
|
||||
export interface PoolSessionDetail {
|
||||
id: string;
|
||||
fingerprint: string;
|
||||
status: SessionStatus;
|
||||
totalRequests: number;
|
||||
successfulRequests: number;
|
||||
successRate: string;
|
||||
inflight: number;
|
||||
cooldownRemaining: string;
|
||||
age: string;
|
||||
}
|
||||
|
||||
// ─── Defaults ──────────────────────────────────────────────────────────────
|
||||
|
||||
export const DEFAULT_POOL_CONFIG: PoolConfig = {
|
||||
minSessions: 6,
|
||||
maxSessions: 20,
|
||||
cooldownBase: 1_000,
|
||||
cooldownMax: 30_000,
|
||||
cooldownJitter: 5_000,
|
||||
requestTimeout: 30_000,
|
||||
requestJitter: 50,
|
||||
};
|
||||
132
open-sse/services/sessionPool/webExecutorWrapper.ts
Normal file
132
open-sse/services/sessionPool/webExecutorWrapper.ts
Normal file
@@ -0,0 +1,132 @@
|
||||
/**
|
||||
* WebExecutorWrapper — Wraps any web executor with session pool support
|
||||
*
|
||||
* This is the integration bridge between the session pool and OmniRoute's
|
||||
* executor system. It intercepts the fetch() call to add session-pool
|
||||
* headers (fingerprint-based User-Agent, Sec-CH-UA, etc.) and handles
|
||||
* 429/5xx responses with pool-level cooldown management.
|
||||
*
|
||||
* Future: For cookie-based providers (ChatGPT Web, DeepSeek Web, etc.)
|
||||
* the wrapper will also inject cookies from the Playwright-authenticated
|
||||
* session.
|
||||
*/
|
||||
|
||||
import { Session } from "./session.ts";
|
||||
import { SessionPool } from "./sessionPool.ts";
|
||||
|
||||
export interface WebExecutorRequest {
|
||||
url: string;
|
||||
method?: string;
|
||||
headers?: Record<string, string>;
|
||||
body?: string;
|
||||
signal?: AbortSignal;
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
export interface WebExecutorResponse {
|
||||
status: number;
|
||||
statusText?: string;
|
||||
headers?: Record<string, string>;
|
||||
body: string;
|
||||
ok: boolean;
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
export interface WebExecutorFn {
|
||||
(req: WebExecutorRequest): Promise<WebExecutorResponse>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Decorate a web executor function with session pool support.
|
||||
*
|
||||
* Before the underlying executor fires:
|
||||
* 1. Acquires a session from the pool (reusable, fingerprint-isolated)
|
||||
* 2. Merges session headers (UA + Sec-CH-UA) into the request
|
||||
* 3. Handles 429 → pool cooldown, 5xx → session death
|
||||
*
|
||||
* For zero-auth providers like Pollinations, Puter, etc. this is all
|
||||
* that's needed for "truly unlimited" — the fingerprint rotation alone
|
||||
* defeats burst-based rate limiting.
|
||||
*/
|
||||
export function withSessionPool(
|
||||
executor: WebExecutorFn,
|
||||
pool: SessionPool,
|
||||
options?: {
|
||||
/** When true, wraps the response body for error handling */
|
||||
wrapResponse?: boolean;
|
||||
},
|
||||
): WebExecutorFn {
|
||||
const wrapResponse = options?.wrapResponse ?? true;
|
||||
|
||||
return async (req: WebExecutorRequest): Promise<WebExecutorResponse> => {
|
||||
// Acquire session from pool (blocking with backoff)
|
||||
let session: Session | null = null;
|
||||
try {
|
||||
session = await pool.acquireBlocking();
|
||||
} catch (err) {
|
||||
return {
|
||||
status: 503,
|
||||
statusText: "Service Unavailable",
|
||||
body: JSON.stringify({
|
||||
error: "session_pool_exhausted",
|
||||
message: `[SessionPool:${pool.provider}] ${(err as Error).message}`,
|
||||
}),
|
||||
ok: false,
|
||||
headers: {},
|
||||
};
|
||||
}
|
||||
|
||||
try {
|
||||
// Build request with session fingerprint headers
|
||||
const sessionHeaders = session.buildHeaders(req.headers);
|
||||
const poolReq: WebExecutorRequest = {
|
||||
...req,
|
||||
headers: sessionHeaders,
|
||||
};
|
||||
|
||||
// Execute the underlying web request
|
||||
const res = await executor(poolReq);
|
||||
|
||||
// Handle response status
|
||||
if (res.status === 429) {
|
||||
pool.reportCooldown(session);
|
||||
|
||||
if (wrapResponse) {
|
||||
return {
|
||||
...res,
|
||||
body: JSON.stringify({
|
||||
error: "pool_rate_limited",
|
||||
message: `[SessionPool:${pool.provider}] Rate limited, session ${session.id} in cooldown`,
|
||||
}),
|
||||
};
|
||||
}
|
||||
return res;
|
||||
}
|
||||
|
||||
if (res.status >= 500) {
|
||||
pool.reportDead(session);
|
||||
return res;
|
||||
}
|
||||
|
||||
// Success
|
||||
pool.reportSuccess(session);
|
||||
pool.totalRequests++;
|
||||
return res;
|
||||
} catch (err) {
|
||||
// Network error — cooldown, not dead (may be transient)
|
||||
pool.reportCooldown(session);
|
||||
return {
|
||||
status: 502,
|
||||
statusText: "Bad Gateway",
|
||||
body: JSON.stringify({
|
||||
error: "pool_network_error",
|
||||
message: (err as Error).message,
|
||||
}),
|
||||
ok: false,
|
||||
headers: {},
|
||||
};
|
||||
} finally {
|
||||
session.release();
|
||||
}
|
||||
};
|
||||
}
|
||||
319
tests/poc/session-pool-opencode-poc.ts
Normal file
319
tests/poc/session-pool-opencode-poc.ts
Normal file
@@ -0,0 +1,319 @@
|
||||
#!/usr/bin/env node --import tsx/esm
|
||||
/**
|
||||
* PoC: SessionPool + FingerprintRotator × OpenCode Free
|
||||
* =======================================================
|
||||
*
|
||||
* Demonstrates that rotating browser fingerprints across sessions
|
||||
* defeats burst rate-limiting on the OpenCode Free public endpoint.
|
||||
*
|
||||
* Run:
|
||||
* node --import tsx/esm tests/poc/session-pool-opencode-poc.ts
|
||||
*
|
||||
* Output: TAP-style results + summary table.
|
||||
*
|
||||
* For a "no-pool" baseline, set BASELINE=1
|
||||
* BASELINE=1 node --import tsx/esm tests/poc/session-pool-opencode-poc.ts
|
||||
*/
|
||||
|
||||
import { setTimeout as sleep } from "node:timers/promises";
|
||||
import { SessionPool } from "../../open-sse/services/sessionPool/sessionPool.ts";
|
||||
import { FingerprintRotator } from "../../open-sse/services/sessionPool/fingerprintRotator.ts";
|
||||
import type { PoolConfig } from "../../open-sse/services/sessionPool/types.ts";
|
||||
import { DEFAULT_POOL_CONFIG } from "../../open-sse/services/sessionPool/types.ts";
|
||||
|
||||
// ─── Config ─────────────────────────────────────────────────────────────────
|
||||
|
||||
import { readFileSync } from "node:fs";
|
||||
import { homedir } from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
const OPENCODE_CHAT_URL = "https://opencode.ai/zen/v1/chat/completions";
|
||||
const USE_AUTH_KEY = process.env.USE_AUTH_KEY || "";
|
||||
const USE_POOL = !process.env.BASELINE;
|
||||
|
||||
function getOpencodeApiKey(): string {
|
||||
if (USE_AUTH_KEY) return USE_AUTH_KEY;
|
||||
try {
|
||||
const authPath = path.join(homedir(), ".local", "share", "opencode", "auth.json");
|
||||
const raw = readFileSync(authPath, "utf8");
|
||||
const auth = JSON.parse(raw);
|
||||
return auth?.opencode?.key ?? "";
|
||||
} catch {
|
||||
return "";
|
||||
}
|
||||
}
|
||||
|
||||
const API_KEY = getOpencodeApiKey();
|
||||
|
||||
const FAST_CONFIG: PoolConfig = {
|
||||
...DEFAULT_POOL_CONFIG,
|
||||
cooldownBase: 100,
|
||||
cooldownMax: 1000,
|
||||
cooldownJitter: 50,
|
||||
minSessions: 3,
|
||||
maxSessions: 10,
|
||||
};
|
||||
|
||||
const TEST_PROMPT = "Say 'ok' and nothing else.";
|
||||
const REQUEST_COUNT = 30;
|
||||
const CONCURRENCY = 3;
|
||||
const ROUNDS = 3; // how many times to repeat the batch
|
||||
|
||||
// ─── Helpers ─────────────────────────────────────────────────────────────────
|
||||
|
||||
function now(): string {
|
||||
return new Date().toISOString().slice(11, 19);
|
||||
}
|
||||
|
||||
function divider(title: string): void {
|
||||
console.log(`\n# ${"━".repeat(60)}`);
|
||||
console.log(`# ${title}`);
|
||||
console.log(`# ${"━".repeat(60)}`);
|
||||
}
|
||||
|
||||
interface AttemptResult {
|
||||
ok: boolean;
|
||||
status: number | string;
|
||||
latencyMs: number;
|
||||
fingerprintId?: string;
|
||||
model?: string;
|
||||
}
|
||||
|
||||
// ─── Pool-based approach ─────────────────────────────────────────────────────
|
||||
|
||||
async function runWithPool(): Promise<AttemptResult[]> {
|
||||
const results: AttemptResult[] = [];
|
||||
const pool = new SessionPool("opencode-poc", FAST_CONFIG);
|
||||
await pool.warmUp(FAST_CONFIG.minSessions);
|
||||
console.log(`# Pool warmed: ${pool.totalCount} sessions, ${pool.availableCount} available`);
|
||||
|
||||
for (let round = 1; round <= ROUNDS; round++) {
|
||||
console.log(`\n# === Round ${round}/${ROUNDS} (${REQUEST_COUNT} sequential requests) ===`);
|
||||
|
||||
for (let i = 0; i < REQUEST_COUNT; i++) {
|
||||
const session = pool.acquire();
|
||||
if (!session) {
|
||||
console.log(`# [${now()}] req ${i + 1}: NO SESSION AVAILABLE (all on cooldown)`);
|
||||
results.push({ ok: false, status: "no-session", latencyMs: 0 });
|
||||
await sleep(200); // backoff before retry
|
||||
continue;
|
||||
}
|
||||
|
||||
const fpId = session.fingerprint.id;
|
||||
const poolHeaders: Record<string, string> = {
|
||||
"Content-Type": "application/json",
|
||||
Accept: "text/event-stream",
|
||||
};
|
||||
if (API_KEY) {
|
||||
poolHeaders["Authorization"] = `Bearer ${API_KEY}`;
|
||||
}
|
||||
const headers = session.buildHeaders(poolHeaders);
|
||||
|
||||
const body = JSON.stringify({
|
||||
model: "deepseek-v4-flash-free",
|
||||
messages: [{ role: "user", content: TEST_PROMPT }],
|
||||
stream: false,
|
||||
});
|
||||
|
||||
const start = performance.now();
|
||||
try {
|
||||
const res = await fetch(OPENCODE_CHAT_URL, {
|
||||
method: "POST",
|
||||
headers,
|
||||
body,
|
||||
signal: AbortSignal.timeout(15_000),
|
||||
});
|
||||
|
||||
const latency = Math.round(performance.now() - start);
|
||||
const status = res.status;
|
||||
|
||||
if (status === 429) {
|
||||
pool.reportCooldown(session);
|
||||
console.log(`# [${now()}] req ${i + 1} [${fpId}] 429 RATE LIMITED (${latency}ms) → cooldown`);
|
||||
results.push({ ok: false, status, latencyMs: latency, fingerprintId: fpId });
|
||||
} else if (status >= 500) {
|
||||
pool.reportDead(session);
|
||||
console.log(`# [${now()}] req ${i + 1} [${fpId}] ${status} SERVER ERROR (${latency}ms) → dead`);
|
||||
results.push({ ok: false, status, latencyMs: latency, fingerprintId: fpId });
|
||||
} else if (status === 200) {
|
||||
const text = await res.text();
|
||||
let model = "?";
|
||||
try {
|
||||
const json = JSON.parse(text);
|
||||
model = json.model ?? json?.choices?.[0]?.message?.content?.slice(0, 20) ?? "?";
|
||||
} catch { /* ignore parse errors */ }
|
||||
pool.reportSuccess(session);
|
||||
console.log(`# [${now()}] req ${i + 1} [${fpId}] 200 OK (${latency}ms) model=${model}`);
|
||||
results.push({ ok: true, status: 200, latencyMs: latency, fingerprintId: fpId, model });
|
||||
} else {
|
||||
console.log(`# [${now()}] req ${i + 1} [${fpId}] ${status} UNEXPECTED (${latency}ms)`);
|
||||
results.push({ ok: false, status, latencyMs: latency, fingerprintId: fpId });
|
||||
}
|
||||
} catch (err: any) {
|
||||
const latency = Math.round(performance.now() - start);
|
||||
const errMsg = err?.name === "TimeoutError" ? "TIMEOUT" : err?.message?.slice(0, 60) ?? String(err);
|
||||
console.log(`# [${now()}] req ${i + 1} [${fpId}] ERROR: ${errMsg} (${latency}ms)`);
|
||||
pool.reportDead(session);
|
||||
results.push({ ok: false, status: `error:${errMsg}`, latencyMs: latency, fingerprintId: fpId });
|
||||
} finally {
|
||||
session.release();
|
||||
}
|
||||
|
||||
// Small jitter between requests to avoid overwhelming
|
||||
await sleep(50 + Math.random() * 100);
|
||||
}
|
||||
|
||||
// Print pool stats after each round
|
||||
const stats = pool.getStats();
|
||||
console.log(`\n# Pool after round ${round}:`);
|
||||
console.log(`# sessions: ${stats.sessions.active} active, ${stats.sessions.cooldown} cooldown, ${stats.sessions.dead} dead`);
|
||||
console.log(`# requests: ${stats.requests.total} total, ${stats.requests.success} success`);
|
||||
console.log(`# rate-429: ${stats.requests.rate429}, errors: ${stats.requests.otherErrors}`);
|
||||
|
||||
// Let pool recover between rounds
|
||||
if (round < ROUNDS) {
|
||||
console.log(`# Waiting 2s for cooldown recovery before next round...`);
|
||||
await sleep(2000);
|
||||
}
|
||||
}
|
||||
|
||||
await pool.shutdown();
|
||||
return results;
|
||||
}
|
||||
|
||||
// ─── Baseline (no pool) ──────────────────────────────────────────────────────
|
||||
|
||||
async function runBaseline(): Promise<AttemptResult[]> {
|
||||
const results: AttemptResult[] = [];
|
||||
const rotator = new FingerprintRotator();
|
||||
|
||||
for (let round = 1; round <= ROUNDS; round++) {
|
||||
console.log(`\n# === Round ${round}/${ROUNDS} (${REQUEST_COUNT} requests, same fingerprint) ===`);
|
||||
|
||||
// Use the same fingerprint for the entire round (no session rotation = baseline)
|
||||
const fp = rotator.next();
|
||||
const headers: Record<string, string> = {
|
||||
"Content-Type": "application/json",
|
||||
Accept: "text/event-stream",
|
||||
"User-Agent": fp.userAgent,
|
||||
"Accept-Language": fp.acceptLanguage ?? "en-US,en;q=0.9",
|
||||
};
|
||||
if (API_KEY) {
|
||||
headers["Authorization"] = `Bearer ${API_KEY}`;
|
||||
}
|
||||
if (fp.secChUa) {
|
||||
headers["Sec-CH-UA"] = fp.secChUa;
|
||||
headers["Sec-CH-UA-Mobile"] = fp.secChUaMobile ?? "?0";
|
||||
headers["Sec-CH-UA-Platform"] = fp.secChUaPlatform ?? '"Windows"';
|
||||
}
|
||||
|
||||
console.log(`# Using fingerprint: ${fp.id}`);
|
||||
|
||||
for (let i = 0; i < REQUEST_COUNT * 1.5; i++) {
|
||||
const body = JSON.stringify({
|
||||
model: "deepseek-v4-flash-free",
|
||||
messages: [{ role: "user", content: TEST_PROMPT }],
|
||||
stream: false,
|
||||
});
|
||||
|
||||
const start = performance.now();
|
||||
try {
|
||||
const res = await fetch(OPENCODE_CHAT_URL, {
|
||||
method: "POST",
|
||||
headers,
|
||||
body,
|
||||
signal: AbortSignal.timeout(15_000),
|
||||
});
|
||||
const latency = Math.round(performance.now() - start);
|
||||
const status = res.status;
|
||||
|
||||
if (status === 200) {
|
||||
console.log(`# [${now()}] req ${i + 1} 200 OK (${latency}ms)`);
|
||||
results.push({ ok: true, status, latencyMs: latency, fingerprintId: fp.id });
|
||||
} else if (status === 429) {
|
||||
console.log(`# [${now()}] req ${i + 1} 429 RATE LIMITED (${latency}ms) — backing off 3s`);
|
||||
results.push({ ok: false, status, latencyMs: latency, fingerprintId: fp.id });
|
||||
await sleep(3000);
|
||||
} else {
|
||||
console.log(`# [${now()}] req ${i + 1} ${status} (${latency}ms)`);
|
||||
results.push({ ok: false, status, latencyMs: latency, fingerprintId: fp.id });
|
||||
}
|
||||
} catch (err: any) {
|
||||
const latency = Math.round(performance.now() - start);
|
||||
const errMsg = err?.name === "TimeoutError" ? "TIMEOUT" : err?.message?.slice(0, 60) ?? String(err);
|
||||
console.log(`# [${now()}] req ${i + 1} ERROR: ${errMsg} (${latency}ms)`);
|
||||
results.push({ ok: false, status: `error:${errMsg}`, latencyMs: latency, fingerprintId: fp.id });
|
||||
}
|
||||
|
||||
await sleep(100 + Math.random() * 200);
|
||||
}
|
||||
}
|
||||
|
||||
return results;
|
||||
}
|
||||
|
||||
// ─── Report ──────────────────────────────────────────────────────────────────
|
||||
|
||||
function printReport(results: AttemptResult[], label: string): void {
|
||||
const total = results.length;
|
||||
const ok = results.filter((r) => r.ok).length;
|
||||
const rateLimited = results.filter((r) => r.status === 429).length;
|
||||
const serverErrors = results.filter((r) => typeof r.status === "number" && r.status >= 500).length;
|
||||
const timeouts = results.filter((r) => String(r.status).includes("TIMEOUT")).length;
|
||||
const noSessions = results.filter((r) => r.status === "no-session").length;
|
||||
const other = total - ok - rateLimited - serverErrors - timeouts - noSessions;
|
||||
|
||||
const latencies = results.filter((r) => r.latencyMs > 0).map((r) => r.latencyMs);
|
||||
const avgLatency = latencies.length > 0 ? Math.round(latencies.reduce((a, b) => a + b, 0) / latencies.length) : 0;
|
||||
const sorted = [...latencies].sort((a, b) => a - b);
|
||||
const p50 = sorted[Math.floor(sorted.length * 0.5)] ?? 0;
|
||||
const p95 = sorted[Math.floor(sorted.length * 0.95)] ?? 0;
|
||||
|
||||
const uniqueFps = new Set(results.filter((r) => r.fingerprintId).map((r) => r.fingerprintId));
|
||||
|
||||
console.log(`\n${"━".repeat(70)}`);
|
||||
console.log(` 📊 REPORT: ${label}`);
|
||||
console.log(`${"━".repeat(70)}`);
|
||||
console.log(` Total requests: ${total}`);
|
||||
console.log(` ✅ Success (200): ${ok} (${(ok / total * 100).toFixed(1)}%)`);
|
||||
console.log(` ⏳ Rate limited: ${rateLimited}`);
|
||||
console.log(` 💀 Server errors: ${serverErrors}`);
|
||||
console.log(` ⌛ Timeouts: ${timeouts}`);
|
||||
console.log(` ❌ No sessions: ${noSessions}`);
|
||||
console.log(` ? Other: ${other}`);
|
||||
console.log(` ───────────────────────────`);
|
||||
console.log(` Avg latency: ${avgLatency}ms`);
|
||||
console.log(` p50 latency: ${p50}ms`);
|
||||
console.log(` p95 latency: ${p95}ms`);
|
||||
console.log(` Unique fingerprints: ${uniqueFps.size}`);
|
||||
console.log(`${"━".repeat(70)}`);
|
||||
}
|
||||
|
||||
// ─── Main ────────────────────────────────────────────────────────────────────
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const keyMasked = API_KEY ? `${API_KEY.slice(0, 8)}...${API_KEY.slice(-4)}` : "NONE";
|
||||
console.log(`# ═══════════════════════════════════════════════════════════════`);
|
||||
console.log(`# PoC: SessionPool × OpenCode Free`);
|
||||
console.log(`# Mode: ${USE_POOL ? "WITH session pool (fingerprint rotation)" : "BASELINE (single fingerprint)"}`);
|
||||
console.log(`# Endpoint: ${OPENCODE_CHAT_URL}`);
|
||||
console.log(`# Auth: ${API_KEY ? `YES (key: ${keyMasked})` : "NO (anonymous)"}`);
|
||||
console.log(`# Requests per round: ${REQUEST_COUNT} × ${ROUNDS} rounds`);
|
||||
console.log(`# ═══════════════════════════════════════════════════════════════`);
|
||||
|
||||
let results: AttemptResult[];
|
||||
if (USE_POOL) {
|
||||
results = await runWithPool();
|
||||
printReport(results, "SessionPool + FingerprintRotator");
|
||||
} else {
|
||||
results = await runBaseline();
|
||||
printReport(results, "Baseline (single fingerprint, no pool)");
|
||||
}
|
||||
|
||||
console.log(`\n# PoC complete.`);
|
||||
}
|
||||
|
||||
main().catch((err) => {
|
||||
console.error("PoC failed:", err);
|
||||
process.exit(1);
|
||||
});
|
||||
421
tests/unit/session-pool.test.ts
Normal file
421
tests/unit/session-pool.test.ts
Normal file
@@ -0,0 +1,421 @@
|
||||
import { describe, it, beforeEach, afterEach } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { setTimeout as sleep } from "node:timers/promises";
|
||||
|
||||
import { SessionPool } from "../../open-sse/services/sessionPool/sessionPool.ts";
|
||||
import { Session } from "../../open-sse/services/sessionPool/session.ts";
|
||||
import { SessionFactory } from "../../open-sse/services/sessionPool/sessionFactory.ts";
|
||||
import { FingerprintRotator } from "../../open-sse/services/sessionPool/fingerprintRotator.ts";
|
||||
import { PoolRegistry } from "../../open-sse/services/sessionPool/poolRegistry.ts";
|
||||
import type { PoolConfig } from "../../open-sse/services/sessionPool/types.ts";
|
||||
import { DEFAULT_POOL_CONFIG } from "../../open-sse/services/sessionPool/types.ts";
|
||||
|
||||
// ─── Fast config for tests ─────────────────────────────────────────────────
|
||||
|
||||
const FAST_CONFIG: PoolConfig = {
|
||||
...DEFAULT_POOL_CONFIG,
|
||||
cooldownBase: 50,
|
||||
cooldownMax: 200,
|
||||
cooldownJitter: 10,
|
||||
};
|
||||
|
||||
// ─── FingerprintRotator ────────────────────────────────────────────────────
|
||||
|
||||
describe("FingerprintRotator", () => {
|
||||
it("returns all distinct fingerprint profiles", () => {
|
||||
const rotator = new FingerprintRotator();
|
||||
const ids = new Set<string>();
|
||||
const total = rotator.count;
|
||||
for (let i = 0; i < total; i++) {
|
||||
ids.add(rotator.next().id);
|
||||
}
|
||||
assert.equal(ids.size, total, `should have ${total} unique fingerprint IDs`);
|
||||
});
|
||||
|
||||
it("rotates through fingerprints round-robin", () => {
|
||||
const rotator = new FingerprintRotator();
|
||||
const first = rotator.next();
|
||||
const second = rotator.next();
|
||||
assert.notEqual(first.id, second.id, "consecutive calls return different fingerprints");
|
||||
});
|
||||
|
||||
it("wraps around after exhausting all profiles", () => {
|
||||
const rotator = new FingerprintRotator();
|
||||
const first = rotator.next();
|
||||
for (let i = 0; i < 20; i++) rotator.next();
|
||||
const wrapped = rotator.next();
|
||||
// Should NOT return the same instance — it's a new iteration cycle
|
||||
assert.ok(wrapped.userAgent.length > 0, "wrapped fingerprint has a user agent");
|
||||
});
|
||||
|
||||
it("each fingerprint has valid UA and accept-language", () => {
|
||||
const rotator = new FingerprintRotator();
|
||||
for (let i = 0; i < 20; i++) {
|
||||
const fp = rotator.next();
|
||||
assert.ok(fp.userAgent.length > 10, `fp ${fp.id} has UA`);
|
||||
assert.ok(fp.acceptLanguage.length > 0, `fp ${fp.id} has accept-language`);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ─── Session ───────────────────────────────────────────────────────────────
|
||||
|
||||
describe("Session", () => {
|
||||
let factory: SessionFactory;
|
||||
let session: Session;
|
||||
|
||||
beforeEach(() => {
|
||||
factory = new SessionFactory(FAST_CONFIG);
|
||||
session = factory.createSession();
|
||||
});
|
||||
|
||||
it("starts in active status", () => {
|
||||
assert.equal(session.status, "active");
|
||||
assert.equal(session.isAvailable, true);
|
||||
});
|
||||
|
||||
it("acquire increments inflight and totalRequests", () => {
|
||||
session.acquire();
|
||||
assert.equal(session.inflight, 1);
|
||||
assert.equal(session.totalRequests, 1);
|
||||
});
|
||||
|
||||
it("release decrements inflight", () => {
|
||||
session.acquire();
|
||||
session.release();
|
||||
assert.equal(session.inflight, 0);
|
||||
assert.equal(session.isAvailable, true);
|
||||
});
|
||||
|
||||
it("markCooldown transitions to cooldown with backoff", () => {
|
||||
session.acquire();
|
||||
session.markSuccess();
|
||||
session.release();
|
||||
session.markCooldown();
|
||||
assert.equal(session.status, "cooldown");
|
||||
assert.ok(session.cooldownRemaining > 0, "cooldownRemaining should be > 0");
|
||||
});
|
||||
|
||||
it("markDead transitions to dead permanently", () => {
|
||||
session.acquire(); // gives totalRequests >= 1
|
||||
session.markDead();
|
||||
assert.equal(session.status, "dead");
|
||||
assert.equal(session.isAvailable, false);
|
||||
assert.ok(session.totalRequests > 0);
|
||||
});
|
||||
|
||||
it("markSuccess increments successfulRequests and resets consecutiveFails", () => {
|
||||
session.markSuccess();
|
||||
assert.equal(session.successfulRequests, 1);
|
||||
assert.equal(session.consecutiveFails, 0);
|
||||
});
|
||||
|
||||
it("markCooldown increments consecutiveFails", () => {
|
||||
session.markCooldown();
|
||||
session.markCooldown();
|
||||
assert.equal(session.consecutiveFails, 2);
|
||||
});
|
||||
|
||||
it("cooldown recovers after backoff window expires", { timeout: 5000 }, async () => {
|
||||
session.markCooldown();
|
||||
assert.equal(session.status, "cooldown");
|
||||
assert.equal(session.isAvailable, false);
|
||||
|
||||
// Wait for cooldown to expire (base=50ms + jitter, should be well under 1s)
|
||||
const deadline = Date.now() + 2000;
|
||||
while (Date.now() < deadline) {
|
||||
if (session.isAvailable) break;
|
||||
await sleep(10);
|
||||
}
|
||||
|
||||
assert.equal(session.isAvailable, true, "session should recover after cooldown");
|
||||
assert.equal(session.status, "active", "status should reset to active after cooldown");
|
||||
});
|
||||
|
||||
it("age returns elapsed time since creation", async () => {
|
||||
const startAge = session.age;
|
||||
await sleep(10);
|
||||
assert.ok(session.age > startAge, "age should increase over time");
|
||||
});
|
||||
});
|
||||
|
||||
// ─── SessionFactory ────────────────────────────────────────────────────────
|
||||
|
||||
describe("SessionFactory", () => {
|
||||
it("creates sessions with unique IDs", () => {
|
||||
const factory = new SessionFactory(FAST_CONFIG);
|
||||
const s1 = factory.createSession();
|
||||
const s2 = factory.createSession();
|
||||
assert.notEqual(s1.id, s2.id);
|
||||
});
|
||||
|
||||
it("creates sessions with different fingerprints", () => {
|
||||
const factory = new SessionFactory(FAST_CONFIG);
|
||||
const s1 = factory.createSession();
|
||||
const s2 = factory.createSession();
|
||||
assert.notEqual(s1.fingerprint.id, s2.fingerprint.id);
|
||||
});
|
||||
|
||||
it("uses custom FingerprintRotator if provided", () => {
|
||||
const rotator = new FingerprintRotator();
|
||||
const factory = new SessionFactory(FAST_CONFIG, rotator);
|
||||
const s = factory.createSession();
|
||||
assert.ok(s.fingerprint.id.length > 0);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── SessionPool ───────────────────────────────────────────────────────────
|
||||
|
||||
describe("SessionPool", () => {
|
||||
let pool: SessionPool;
|
||||
|
||||
beforeEach(() => {
|
||||
pool = new SessionPool("test-provider", FAST_CONFIG);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
pool.shutdown();
|
||||
});
|
||||
|
||||
it("starts with 0 sessions", () => {
|
||||
assert.equal(pool.totalCount, 0);
|
||||
assert.equal(pool.availableCount, 0);
|
||||
});
|
||||
|
||||
it("ensureMinSessions creates the minimum", async () => {
|
||||
await pool.ensureMinSessions();
|
||||
assert.equal(pool.totalCount, FAST_CONFIG.minSessions);
|
||||
});
|
||||
|
||||
it("warmUp creates up to the specified count", async () => {
|
||||
await pool.warmUp(4);
|
||||
assert.equal(pool.totalCount, 4);
|
||||
});
|
||||
|
||||
it("warmUp respects maxSessions", async () => {
|
||||
await pool.warmUp(999);
|
||||
assert.equal(pool.totalCount, FAST_CONFIG.maxSessions);
|
||||
});
|
||||
|
||||
it("acquire returns null when pool is empty", () => {
|
||||
const s = pool.acquire();
|
||||
assert.equal(s, null);
|
||||
});
|
||||
|
||||
it("acquire returns a session after warmUp", async () => {
|
||||
await pool.warmUp(3);
|
||||
const s = pool.acquire();
|
||||
assert.notEqual(s, null);
|
||||
assert.equal(s!.inflight, 1);
|
||||
});
|
||||
|
||||
it("acquire round-robins across sessions", async () => {
|
||||
await pool.warmUp(3);
|
||||
const ids = new Set<string>();
|
||||
for (let i = 0; i < 6; i++) {
|
||||
const s = pool.acquire();
|
||||
assert.notEqual(s, null);
|
||||
ids.add(s!.id);
|
||||
s!.release();
|
||||
}
|
||||
// With 3 sessions and round-robin, we should visit all 3
|
||||
assert.equal(ids.size, 3);
|
||||
});
|
||||
|
||||
it("acquire skips sessions in cooldown", async () => {
|
||||
await pool.warmUp(3);
|
||||
const sessions = [pool.acquire()!, pool.acquire()!, pool.acquire()!];
|
||||
|
||||
// Put first 2 into cooldown
|
||||
pool.reportCooldown(sessions[0]);
|
||||
sessions[0].release();
|
||||
pool.reportCooldown(sessions[1]);
|
||||
sessions[1].release();
|
||||
|
||||
// Third is still active, release it
|
||||
pool.reportSuccess(sessions[2]);
|
||||
sessions[2].release();
|
||||
|
||||
// Acquire should skip sessions 0 and 1 (cooldown) and give us session 2
|
||||
const s = pool.acquire();
|
||||
assert.notEqual(s, null);
|
||||
assert.equal(s!.id, sessions[2].id);
|
||||
s!.release();
|
||||
});
|
||||
|
||||
it("reportCooldown increments rate429count", async () => {
|
||||
await pool.warmUp(1);
|
||||
const s = pool.acquire()!;
|
||||
pool.reportCooldown(s);
|
||||
s.release();
|
||||
|
||||
assert.equal(pool.rate429count, 1);
|
||||
assert.equal(pool.cooldownCount, 1);
|
||||
});
|
||||
|
||||
it("reportDead increments otherErrors", async () => {
|
||||
await pool.warmUp(1);
|
||||
const s = pool.acquire()!;
|
||||
pool.reportDead(s);
|
||||
s.release();
|
||||
|
||||
assert.equal(pool.otherErrors, 1);
|
||||
assert.equal(pool.deadCount, 1);
|
||||
});
|
||||
|
||||
it("reportSuccess increments success counters", async () => {
|
||||
await pool.warmUp(1);
|
||||
const s = pool.acquire()!;
|
||||
pool.reportSuccess(s);
|
||||
s.release();
|
||||
|
||||
assert.equal(pool.totalRequests, 1);
|
||||
assert.equal(pool.successfulRequests, 1);
|
||||
});
|
||||
|
||||
it("getStats returns correct snapshot", async () => {
|
||||
await pool.warmUp(3);
|
||||
const s = pool.acquire()!;
|
||||
pool.reportSuccess(s);
|
||||
s.release();
|
||||
|
||||
const stats = pool.getStats();
|
||||
assert.equal(stats.provider, "test-provider");
|
||||
assert.equal(stats.sessions.total, 3);
|
||||
assert.equal(stats.requests.success, 1);
|
||||
assert.ok(Number.parseFloat(stats.successRate) > 0);
|
||||
});
|
||||
|
||||
it("getSessionDetails returns per-session info", async () => {
|
||||
await pool.warmUp(2);
|
||||
const details = pool.getSessionDetails();
|
||||
assert.equal(details.length, 2);
|
||||
assert.ok(details[0].fingerprint.length > 0);
|
||||
assert.equal(details[0].inflight, 0);
|
||||
});
|
||||
|
||||
it("acquireBlocking eventually returns a session after cooldown expires", { timeout: 5000 }, async () => {
|
||||
await pool.warmUp(1);
|
||||
const s = pool.acquire()!;
|
||||
|
||||
// Put it into cooldown
|
||||
pool.reportCooldown(s);
|
||||
s.release();
|
||||
|
||||
// Now the only session is in cooldown, but should recover quickly (50ms base)
|
||||
const acquired = await pool.acquireBlocking(3000);
|
||||
assert.notEqual(acquired, null);
|
||||
assert.equal(acquired.isAvailable, true);
|
||||
acquired.release();
|
||||
});
|
||||
|
||||
it("executeWithSession runs a function with a session", { timeout: 5000 }, async () => {
|
||||
await pool.warmUp(2);
|
||||
const result = await pool.executeWithSession(async (s) => {
|
||||
return `session-${s.id}`;
|
||||
}, 1000);
|
||||
assert.ok((result as string).startsWith("session-"));
|
||||
});
|
||||
|
||||
it("pruneDeadSessions removes dead sessions and replenishes", async () => {
|
||||
await pool.warmUp(3);
|
||||
const s = pool.acquire()!;
|
||||
pool.reportDead(s);
|
||||
s.release();
|
||||
|
||||
assert.equal(pool.deadCount, 1);
|
||||
pool.pruneDeadSessions();
|
||||
|
||||
// Dead session removed, and ensureMinSessions replenished
|
||||
assert.equal(pool.totalCount, FAST_CONFIG.minSessions);
|
||||
});
|
||||
|
||||
it("shutdown marks all sessions dead", async () => {
|
||||
await pool.warmUp(3);
|
||||
await pool.shutdown();
|
||||
assert.equal(pool.totalCount, 0);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── PoolRegistry ──────────────────────────────────────────────────────────
|
||||
|
||||
describe("PoolRegistry", () => {
|
||||
let pool: SessionPool;
|
||||
|
||||
beforeEach(() => {
|
||||
// Clean slate
|
||||
const providers = PoolRegistry.listProviders();
|
||||
for (const p of providers) PoolRegistry.resetPool(p);
|
||||
pool = new SessionPool("test-provider", FAST_CONFIG);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
pool.shutdown();
|
||||
PoolRegistry.resetPool("test-provider");
|
||||
});
|
||||
|
||||
it("register adds a pool", () => {
|
||||
PoolRegistry.register("test-provider", pool);
|
||||
assert.equal(PoolRegistry.listProviders().includes("test-provider"), true);
|
||||
});
|
||||
|
||||
it("getPool returns the registered pool", () => {
|
||||
PoolRegistry.register("test-provider", pool);
|
||||
assert.equal(PoolRegistry.getPool("test-provider"), pool);
|
||||
});
|
||||
|
||||
it("getStats returns pool stats", () => {
|
||||
PoolRegistry.register("test-provider", pool);
|
||||
const stats = PoolRegistry.getStats("test-provider");
|
||||
assert.notEqual(stats, null);
|
||||
assert.equal(stats!.provider, "test-provider");
|
||||
});
|
||||
|
||||
it("getStats returns null for unknown provider", () => {
|
||||
assert.equal(PoolRegistry.getStats("nonexistent"), null);
|
||||
});
|
||||
|
||||
it("getAllStats returns all pools", () => {
|
||||
const pool2 = new SessionPool("second", FAST_CONFIG);
|
||||
PoolRegistry.register("test-provider", pool);
|
||||
PoolRegistry.register("second", pool2);
|
||||
const all = PoolRegistry.getAllStats();
|
||||
assert.equal(all.length, 2);
|
||||
pool2.shutdown();
|
||||
PoolRegistry.resetPool("second");
|
||||
});
|
||||
|
||||
it("getSessionDetails returns session list", async () => {
|
||||
await pool.warmUp(2);
|
||||
PoolRegistry.register("test-provider", pool);
|
||||
const details = PoolRegistry.getSessionDetails("test-provider");
|
||||
assert.notEqual(details, null);
|
||||
assert.equal(details!.length, 2);
|
||||
});
|
||||
|
||||
it("getSessionDetails returns null for unknown provider", () => {
|
||||
assert.equal(PoolRegistry.getSessionDetails("nonexistent"), null);
|
||||
});
|
||||
|
||||
it("resetPool removes and shuts down the pool", () => {
|
||||
PoolRegistry.register("test-provider", pool);
|
||||
assert.equal(PoolRegistry.resetPool("test-provider"), true);
|
||||
assert.equal(PoolRegistry.getPool("test-provider"), undefined);
|
||||
});
|
||||
|
||||
it("resetPool returns false for unknown provider", () => {
|
||||
assert.equal(PoolRegistry.resetPool("nonexistent"), false);
|
||||
});
|
||||
|
||||
it("unregister removes a pool", () => {
|
||||
PoolRegistry.register("test-provider", pool);
|
||||
assert.equal(PoolRegistry.unregister("test-provider"), true);
|
||||
assert.equal(PoolRegistry.getPool("test-provider"), undefined);
|
||||
});
|
||||
|
||||
it("size reflects pool count", () => {
|
||||
const prev = PoolRegistry.size;
|
||||
PoolRegistry.register("test-provider", pool);
|
||||
assert.equal(PoolRegistry.size, prev + 1);
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user