Appearance
Multi-Provider Model Routing
GhostAI's multi-provider model routing gives your organization access to the full landscape of large language models through a single, unified interface. Instead of managing separate accounts, credentials, and tooling for each AI provider, GhostAI aggregates 12+ providers into one intelligent catalog — so your teams can pick the best model for every task without leaving the platform.
Unified Model Catalog
GhostAI connects to 12+ direct LLM providers simultaneously, including OpenAI, Anthropic, Google Gemini, AWS Bedrock, Mistral, xAI, DeepSeek, Groq, Together, Perplexity, and Cohere. In addition, OpenRouter integration provides access to 200+ models through a single aggregator, and local inference backends enable fully private, on-premises model hosting.
All available models are presented in a single Model Browser with two views:
Smart Picks (Default)
For users who want a recommendation rather than a catalog. Select an intent (Coding, Creative, Analysis, General Chat, or Private) and receive curated recommendations: Top Pick, Fast Pick, Budget Pick, and Private Pick. Each card shows three key signals (speed, locality, context length) and a single "Use this model" button. A favorites strip provides one-click access to pinned models.
Catalog (Power User)
A dense, sortable table with a collapsible filter sidebar for full control:
- Provider Filters — Checkboxes with provider logos
- Speed Filters — Fast, Standard, Reasoning
- Location Filters — Private / Local, Cloud
- Feature Filters — Tools, Vision, and other capabilities
- Sortable Columns — Model, Provider, Speed, Context, Features
- Inline Detail Panel — Click any row to expand brand, parameters, origin, routes, capabilities, and use-case tags
Shared Features (Both Views)
- Universal Search — Type model names, providers, or smart tokens like "fast", "private", "vision", or "tools" to filter instantly
- Favorites — Pin models you use most often
- Source Status Bar — Click any provider in the footer to filter to that source
- Route Selector — Choose between direct provider access and OpenRouter when both are available
- Compare Mode — Select 2-4 models and compare side by side
Models available through multiple providers are automatically merged into a single catalog entry, so you see one "GPT-4o" with the option to route through OpenAI directly or via OpenRouter — not two confusing duplicates.
Provider Flexibility
Every connected provider is queried in parallel, so the catalog loads quickly even when individual providers are slow or temporarily unreachable. Unreachable providers are gracefully excluded without affecting the rest of the catalog.
If your preferred route encounters a rate limit or transient error, GhostAI can automatically fall back to an alternative route for the same model — for example, retrying through OpenRouter when a direct provider returns an error.
Supported Providers
| Provider | Type |
|---|---|
| OpenAI | Direct |
| Anthropic | Direct |
| Google Gemini | Direct |
| AWS Bedrock | Direct |
| Mistral | Direct |
| xAI (Grok) | Direct |
| DeepSeek | Direct |
| Groq | Direct |
| Together AI | Direct |
| Perplexity | Direct |
| Cohere | Direct |
| OpenRouter | Aggregator (200+ models) |
| Local backends | Private inference |
Bring Your Own Key (BYOK)
GhostAI supports a flexible key management model that puts organizations in control of how they access each provider:
- Platform-managed keys — The GhostAI platform provides default access to supported providers out of the box.
- Organization keys — Tenant administrators can supply their own API keys for any provider, routing all usage through their own accounts for billing and compliance purposes.
- Personal keys — Individual users can add their own keys to unlock providers or models not available at the organization level.
Key resolution follows a priority chain: personal keys take precedence over organization keys, which take precedence over platform defaults. Administrators can lock a provider to enforce a specific key (preventing user overrides) or disable a provider entirely to remove it from the catalog.
Admin-Controlled Model Access
Administrators have fine-grained control over which models and providers are available to their organization. Access policies follow a layered inheritance model where each level can only restrict — never expand — what the level above permits.
What Admins Can Control
- Allowed / blocked providers — Permit or deny entire providers (e.g., allow only OpenAI and Anthropic).
- Allowed / blocked sources — Control at the source level (e.g., block OpenRouter to force direct-provider usage only, or restrict to local models).
- Model-level overrides — Allow or block specific models within an otherwise permitted provider.
- Geographic filtering — Restrict available models based on the country of origin of the model provider (see below).
Policies are configured through the Admin Dashboard or the management API and apply across the organization instantly.
Geographic Filtering
For organizations with data sovereignty, compliance, or risk management requirements, GhostAI provides geographic filtering that classifies model providers by country of origin and risk tier.
How It Works
- Model providers are grouped into geographic risk tiers ranging from minimal risk (e.g., Five Eyes nations) to restricted or sanctioned jurisdictions.
- Administrators set which tiers are permitted at the organization level.
- Users can further restrict their own allowed tiers but cannot expand beyond what the organization permits.
- Models with no geographic data are not blocked by default, ensuring broad compatibility.
Configuration
- Admin Dashboard — Model Access → Country Tiers tab
- User Settings — Model Preferences → Countries tab
- API —
GET /api/chat/country-tiersreturns available tier definitions - API —
PUT /api/settings/model-config/organizationor/userwith theallowed_country_tiersfield
The Model Browser displays tier badges and country flags alongside each model, and a dedicated tier filter lets users narrow the catalog by geographic classification.
Private & Local Model Support
GhostAI supports running models entirely on your own infrastructure — no data leaves your network. Private models appear in the Private Section of the Model Browser with a "Local" badge and support full conversational chat with streaming.
Key Benefits
- Zero data egress — Prompts and responses stay on your hardware.
- Full chat support — Private models support the same streaming chat experience as cloud models.
- Capability detection — GhostAI automatically detects each local model's capabilities (vision, tool calling, context length, parameter count) so they integrate seamlessly with the Model Browser's filters.
- Multiple backend options — Connect popular local inference engines to serve models in the format that fits your infrastructure.
Private models are governed by the same access policies as cloud models, so administrators retain full control over which local models are available to users.
Multi-Model Comparison
GhostAI enables users to send the same prompt to multiple models simultaneously and compare responses side by side. Combined with the unified catalog and route selection, this makes it easy to evaluate model quality, cost, and speed for any given use case — all within a single interface.
Cost Optimization
With visibility into multiple providers and routes for the same model, organizations can optimize AI spend:
- Route selection — Choose the most cost-effective route when a model is available through both a direct provider and OpenRouter.
- Provider diversity — Take advantage of competitive pricing across providers for equivalent models.
- Local offloading — Route appropriate workloads to private models at zero per-token cost.
- Granular controls — Restrict expensive models or providers to specific users or teams.
API Endpoints
The model routing system exposes the following endpoints:
| Endpoint | Description |
|---|---|
GET /api/chat/models | Returns the full model catalog for the current user |
GET /api/chat/models/detailed | Returns detailed model metadata including capabilities and routes |
GET /api/chat/country-tiers | Returns geographic tier definitions |
PUT /api/settings/model-config/organization | Update organization-level model access policy |
PUT /api/settings/model-config/user | Update user-level model preferences |
Model Routing Protocol (MRP)
Under the hood, GhostAI's routing is powered by the Model Routing Protocol (MRP) — a BGP/OSPF-inspired framework that makes intelligent, real-time decisions about which provider and model should handle every request.
How MRP Works
MRP maintains a Model Routing Information Base (MRIB) — a live table of every available route to every model across all configured providers. Each route carries attributes like health score, latency, cost, error rate, in-flight request count, and compliance metadata.
When a request arrives, MRP runs a 15-step best-path algorithm that evaluates candidates across reachability, policy compliance, geographic restrictions, health, latency, cost, throughput, rate limits, and provider diversity — then applies load balancing across equivalent paths.
Key MRP Features
- Circuit breakers — Automatically isolate unhealthy providers after consecutive failures and restore them when probes confirm recovery.
- Route dampening — Suppress flapping routes (providers that alternate between healthy and unhealthy) using exponential penalty decay, inspired by RFC 2439.
- Health probing — Two-tier probing: lightweight adjacency probes every 30s and canary completion probes every 5m measure actual TTFB and tokens-per-second.
- Budget enforcement — Per-tenant cost ceilings shift traffic to cheaper providers as spend approaches the limit.
- ECMP load balancing — When multiple equivalent routes exist, traffic is distributed using weighted round-robin, least-outstanding, cost-weighted random, or sticky session strategies.
- QoS class mapping — Request types are automatically mapped to routing profiles (realtime, batch, critical, best-effort) that reorder scoring priorities.
- Shadow mode — Run MRP in parallel with legacy routing to compare decisions without affecting production traffic.
- Simulation — Test routing decisions against hypothetical scenarios without changing live state.
Model ID Normalization
GhostAI automatically normalizes model IDs before routing resolution. This means you can use any common form of a model ID and it will resolve correctly:
| Input | Canonical Form | Why |
|---|---|---|
openai/gpt-4.1 | gpt-4.1 | Vendor prefix stripped |
anthropic/claude-sonnet-4.6 | claude-sonnet-4-6 | Prefix stripped + dot-to-hyphen alias |
google/gemini-2.5-flash | gemini-2.5-flash | Vendor prefix stripped |
us.anthropic.claude-sonnet-4-20250514-v1:0 | anthropic.claude-sonnet-4-20250514-v1:0 | Bedrock regional prefix stripped |
foundry/gpt-4.1 | foundry/gpt-4.1 | Foundry/Vertex prefixes preserved |
Normalization is applied automatically in MRP resolve, failover, simulate, counterfactual, token estimation, cost tracking, credit weights, and model allowlist checks.
MRP Admin API
Platform administrators can monitor and control MRP through the admin API:
API Path Prefix
All MRP endpoints are prefixed with /api/platform-admin/routing in the actual HTTP path. For example, POST /mrp/simulate is accessed at POST /api/platform-admin/routing/mrp/simulate.
| Endpoint | Description |
|---|---|
GET /mrp/overview | System status: enabled, route counts, provider health |
GET /mrp/routes | Full route table with filtering |
GET /mrp/health-matrix | Per-provider health, latency, error rates |
POST /mrp/maintenance/{provider} | Enter/exit maintenance mode |
POST /mrp/simulate | Simulate routing decisions with decision waterfall |
POST /mrp/simulate/counterfactual | What-if analysis against modified MRIB state |
POST /mrp/simulate-all | Dry-run route resolution for every model in the MRIB |
POST /mrp/probe-all | Send "Say OK" to every reachable model (sync) |
GET /mrp/probe-all/stream | SSE stream of per-model probe results |
POST /mrp/audit-context | Compare hardcoded context windows against live provider data |
GET /mrp/events | Recent MRP events (circuit breaks, failovers, etc.) |
GET /mrp/events/stream | SSE real-time event stream |
GET /mrp/decisions | Decision history with MRP metadata |
GET /mrp/decisions/{id}/trace | Full decision waterfall for a request |
GET /mrp/snapshot | Capture MRIB state for counterfactual analysis |
GET /mrp/anomalies | Provider anomaly detection status |
GET /mrp/quality | Per-provider/model quality scores |
GET/POST /mrp/sla/* | SLA definitions and compliance tracking |
GET/POST /mrp/ab/experiments | A/B routing experiments with CUSUM regression |
POST /mrp/chaos/inject | Inject controlled failures for resilience testing |
POST /mrp/pipeline/plan | Route multi-step AI pipelines |
GET/POST /mrp/policy/* | Declarative YAML routing policies |
POST /mrp/policy/canary | Start canary policy deployment at X% traffic |
GET /mrp/policy/canary | Canary deployment status and error rates |
POST /mrp/policy/canary/promote | Promote canary to current policy |
POST /mrp/policy/canary/rollback | Roll back canary deployment |
Advanced Capabilities
- Decision waterfall — Every routing decision produces a structured trace showing how candidates are filtered at each of the 16 best-path steps, visualized as an interactive waterfall chart in the dashboard.
- Counterfactual simulation — Test "what if provider X goes down?" scenarios against a snapshot of the live routing table without affecting production.
- A/B testing — Create experiments that split traffic between routing strategies, with automatic CUSUM change-point detection that aborts experiments when quality regresses.
- Anomaly detection — EWMA + z-score statistical anomaly detection on per-provider latency and error rate; automatically reduces local preference for anomalous providers and restores it when anomalies resolve. Includes predicted time-to-circuit-break extrapolation.
- Trace-derived health — Production request traces provide the primary health signal; synthetic probe frequency is reduced when trace volume is sufficient.
- SLA tracking — Per-tenant SLA compliance monitoring with warning events at 90% of thresholds and breach events when violated.
- Chaos engineering — Inject controlled provider outages, latency spikes, rate limit storms, and cascade failures to test system resilience, with full state restoration on experiment stop.
- Canary policy deployments — Apply new routing policies to a configurable percentage of traffic, compare error rates, and promote or roll back via API.
- Pipeline routing — Route multi-step AI pipelines (summarize → translate → classify) with cumulative budget/latency optimization and input-size awareness.
- OpenTelemetry tracing — MRP resolution, provider calls, and failovers produce structured OTel spans for distributed trace correlation.
- Live weathermap — Real-time provider health grid and SSE event ticker in the dashboard for at-a-glance operational awareness.
Summary
GhostAI's multi-provider model routing delivers:
- One catalog, every model — 12+ direct providers, 200+ via OpenRouter, and local private models in a single browser.
- Intelligent routing — 15-step best-path algorithm with circuit breakers, dampening, ECMP load balancing, and automatic failover.
- Full admin control — Layered access policies, provider locking, geographic filtering, and declarative YAML routing policies.
- BYOK flexibility — Use platform keys, organization keys, or personal keys for any provider.
- Private-first option — Run models on your own infrastructure with zero data egress.
- Cost visibility — Per-tenant budget enforcement, A/B testing, and pipeline cost optimization.
- Observability — Decision waterfall, anomaly detection, SLA tracking, live weathermap, and OTel trace integration.
Whether your priority is compliance, cost, capability, or all three, GhostAI's model routing ensures your teams always have access to the right model through the right path.