My Role
Systems / Platform
In Development
Orchestration Lab
Control plane for AI work: tune local models, expose them as our own provider inference, and route across hosted providers using cost, latency, quality, and health metrics.
Last updated: Jul 2026
Project signals
Tech Stack
AI Orchestrator SDK, Custom provider adapters, Local inference, Queues, Observability, Human-in-the-loop
Key Metric / Proof
Unified generate() across local + cloud · strategies: cost / latency / quality · health-checked custom endpoints
Scope
In scope: local model serving as custom provider, multi-provider adapters, routing strategies, metrics & fallbacks, durable workflow spine. Research notes cover enrichment tuning that feeds the local endpoint.
Next / Status Note
In Development — local custom provider + hosted adapters under one orchestrator. AI Orchestrator SDK is the published product output.
Stack
Local models + hosted providers under one control plane
We tune local models for enrichment and product jobs, expose them as our own provider inference, and let the orchestrator route across that endpoint and third-party APIs using cost, latency, and quality metrics.
Lab product output
AI Orchestrator SDK
The open Node.js SDK this lab ships — unified generate / stream / embed across local custom providers and hosted adapters, with strategy routing and automatic fallbacks.
Install
Click to copy
01
Our inference
Custom provider
Local / self-hosted models
Tuned for enrichment, classification, extraction, and private workloads. Connected via baseUrl + health checks — same generate() surface as cloud models.
- Fine-tune / eval
- Quantization
- Health endpoint
02
Hosted providers
Adapters
OpenAI · Google · Anthropic · …
First-class adapters for public APIs. Key rotation, streaming, embeddings, and automatic fallback when a provider fails or rate-limits.
- BYOK keys
- Fallbacks
- Streaming
03
Orchestrator
AI Orchestrator SDK
Route · fallback · measure
Unified generate / stream / embed. Strategies pick the next model; metrics decide when to stay local vs go hosted.
- Cost Prefer cheapest viable model
- Latency Prefer fastest path
- Quality Prefer highest tier
- Health Skip unhealthy local endpoints
04
Products & spine
Consumers
Orchestration Lab · Virabase · PromptQT
Durable jobs, queues, and human checkpoints sit above the SDK — products call one API whether the answer came from a local tuned model or a cloud provider.
Narrative
How local tuning, custom inference, hosted providers, and metrics fit together as one control plane.
Spine
Orchestration Lab is the spine for Recash AI products — not a feature buried inside PromptQT or Virabase. It decides which model answers a job: a local model we tuned ourselves, or a hosted provider, based on measurable signals.
-
Product output
AI Orchestrator SDK
The open Node.js package this lab ships — unified generate / stream / embed, key rotation, streaming, and automatic fallback across local custom providers and hosted adapters.
generate()generateStream()embedContent()
InstallClick to copy -
Our inference
Local model tuning → custom provider
We fine-tune and evaluate local models for enrichment and product-side tasks — classification, extraction, summarization, private workloads. They register as a custom provider in the orchestrator pool: baseUrl, optional health-check, auth headers, and request/response templates — the same surface as cloud adapters.
Product code calls one API. Whether the answer came from a machine we run or from OpenAI / Google / Anthropic is a routing decision, not a rewrite.
baseUrl- Health check
- Auth headers
- Templates
-
Hosted adapters
Supporting other inference providers
First-class adapters cover major hosted APIs. Strategies pick the next model; metrics decide when to stay local vs go hosted — with fallbacks when a provider fails or rate-limits.
- OpenAI
- Anthropic
- Custom / local
-
Decision layer
Metrics that drive the route
Strategies can be composed — for example cost then quality — so ties break cleanly. Observability on the spine (queues, retries, human checkpoints) sits above this layer so PromptQT Studio and Virabase agentic briefs stay auditable.
- Cost Prefer the cheapest viable model — often local for high-volume enrichment.
- Latency Prefer the fastest path when UX is interactive.
- Quality Prefer the highest-tier model when the job demands it.
- Health Skip unhealthy local endpoints; fall through to the next provider.
Example strategy
['cost', 'quality'] -
Open foundation
Publishable control plane
Related research on enrichment tuning is linked below as artifacts under this project. The SDK is what those patterns become in the open.
PackageClick to copy
Artifacts
Linked research under this project — evidence trails, not the product itself.
See Orchestration Lab in the browser
A Full HD product view inside a laptop stage. Preview only — open the live site to interact.
Open Orchestration Lab → davoodmood.github.io
3D stage
Open live site