Appearance
Microsoft Foundry (Azure AI)
Route GhostAI chat, PII referee, and embedding traffic through Microsoft Foundry on Azure.
Prerequisites
| Requirement | Details |
|---|---|
| Azure subscription | Active subscription with Foundry resource provisioned |
| Foundry resource | Created via Azure Portal → Azure AI Foundry |
| Model deployments | At least one model deployed (e.g. gpt-4.1, gpt-5) |
| Network access | Backend must reach https://<resource>.services.ai.azure.com |
Environment Variables
Add these to your .env (or set via config-manager):
bash
# Foundry endpoint — the base URL of your Azure AI Foundry resource
FOUNDRY_ENDPOINT=https://my-foundry.services.ai.azure.com
# API key (required when FOUNDRY_AUTH_MODE=api_key)
FOUNDRY_API_KEY=your-foundry-api-key
# Auth mode: "api_key" or "entra_id"
FOUNDRY_AUTH_MODE=api_keyAll three variables are also wired through Docker Compose (both dev and production).
Authentication Modes
API Key (default)
Set FOUNDRY_AUTH_MODE=api_key and provide FOUNDRY_API_KEY. The key is found in the Azure Portal under your Foundry resource → Keys and Endpoint.
bash
FOUNDRY_AUTH_MODE=api_key
FOUNDRY_API_KEY=abc123...Entra ID (Managed Identity)
Set FOUNDRY_AUTH_MODE=entra_id and leave FOUNDRY_API_KEY empty. The backend uses DefaultAzureCredential from the Azure Identity SDK, which automatically picks up:
- Managed Identity when running on Azure (App Service, AKS, Container Apps)
- Azure CLI credentials during local development
- Environment variables (
AZURE_CLIENT_ID,AZURE_TENANT_ID,AZURE_CLIENT_SECRET) for service principal auth
bash
FOUNDRY_AUTH_MODE=entra_id
FOUNDRY_API_KEY=Ensure the identity has the Cognitive Services OpenAI User role on the Foundry resource.
Model Naming Conventions
GhostAI maps Foundry model names using the deployment name you configure in Azure:
| Azure Deployment Name | GhostAI Model ID |
|---|---|
gpt-4.1 | gpt-4.1 |
gpt-4.1-mini | gpt-4.1-mini |
gpt-4.1-nano | gpt-4.1-nano |
gpt-5 | gpt-5 |
gpt-5-mini | gpt-5-mini |
o4-mini | o4-mini |
Use the exact deployment name as the model ID when selecting models in the GhostAI UI. If your deployment name differs from the base model (e.g. my-gpt5-deployment), use that custom name.
PII Referee Configuration
To use a Foundry-hosted model as the PII referee instead of a local model:
bash
PII_REFEREE_RUNTIME=foundry
PII_REFEREE_FOUNDRY_MODEL=gpt-4.1-miniThe referee reuses FOUNDRY_ENDPOINT and FOUNDRY_API_KEY / Entra ID credentials from the main Foundry configuration. Token usage and cost are tracked via dedicated Prometheus metrics (ghostai_referee_foundry_tokens_total, ghostai_referee_foundry_cost_dollars).
Embedding Backend Configuration
To use Foundry for RAG and memory embeddings instead of local Sentence Transformers:
bash
EMBEDDING_BACKEND=foundry
FOUNDRY_EMBEDDING_MODEL=text-embedding-3-largeThe embedding client shares the same FOUNDRY_ENDPOINT and authentication settings. Embedding dimensions default to the model's native output (3072 for text-embedding-3-large).
Cost Tracking
Foundry model costs are tracked automatically using static per-million-token pricing:
| Model | Input (per 1M) | Output (per 1M) |
|---|---|---|
| gpt-4.1 | $2.00 | $8.00 |
| gpt-4.1-mini | $0.40 | $1.60 |
| gpt-4.1-nano | $0.10 | $0.40 |
| gpt-5 | $5.00 | $15.00 |
| gpt-5-mini | $1.25 | $5.00 |
| o4-mini | $1.10 | $4.40 |
These are billed through your Azure subscription, not through GhostAI. The cost tracking in GhostAI provides visibility into usage patterns for budgeting and chargeback.
Troubleshooting
"401 Unauthorized" or "403 Forbidden"
- API key mode: verify
FOUNDRY_API_KEYis correct and not expired. Regenerate from the Azure Portal if needed. - Entra ID mode: confirm the managed identity or service principal has the Cognitive Services OpenAI User role assigned at the Foundry resource scope (not just the resource group).
"404 Model Not Found"
- The model deployment name in Azure must match the model ID you're requesting. Check Azure AI Foundry → Deployments for the exact name.
- Ensure the model is fully deployed (status: Succeeded) and not in a provisioning state.
Timeout Errors
- Default LLM timeout is 120s. For large context windows or reasoning models (
o4-mini), consider increasing:bashLLM_TIMEOUT=180 - Check Azure Portal → Foundry resource → Metrics for throttling (HTTP 429). You may need to increase your TPM (tokens-per-minute) quota.
Entra ID Credential Chain Failures
- Locally: ensure
az loginis current (az account get-access-token --resource https://cognitiveservices.azure.com) - On Azure: confirm the system-assigned or user-assigned managed identity is enabled on the compute resource
- Check that
azure-identityis installed:pip install azure-identity
Prometheus Metrics Not Appearing
Foundry referee metrics (ghostai_referee_foundry_tokens_total, ghostai_referee_foundry_cost_dollars) only appear after the first Foundry referee call. Verify the referee is configured with PII_REFEREE_RUNTIME=foundry and trigger a test message.