Skip to content

Microsoft Foundry (Azure AI) ​

Route GhostAI chat, PII referee, and embedding traffic through Microsoft Foundry on Azure.

Prerequisites ​

RequirementDetails
Azure subscriptionActive subscription with Foundry resource provisioned
Foundry resourceCreated via Azure Portal → Azure AI Foundry
Model deploymentsAt least one model deployed (e.g. gpt-4.1, gpt-5)
Network accessBackend must reach https://<resource>.services.ai.azure.com

Environment Variables ​

Add these to your .env (or set via config-manager):

bash
# Foundry endpoint — the base URL of your Azure AI Foundry resource
FOUNDRY_ENDPOINT=https://my-foundry.services.ai.azure.com

# API key (required when FOUNDRY_AUTH_MODE=api_key)
FOUNDRY_API_KEY=your-foundry-api-key

# Auth mode: "api_key" or "entra_id"
FOUNDRY_AUTH_MODE=api_key

All three variables are also wired through Docker Compose (both dev and production).

Authentication Modes ​

API Key (default) ​

Set FOUNDRY_AUTH_MODE=api_key and provide FOUNDRY_API_KEY. The key is found in the Azure Portal under your Foundry resource → Keys and Endpoint.

bash
FOUNDRY_AUTH_MODE=api_key
FOUNDRY_API_KEY=abc123...

Entra ID (Managed Identity) ​

Set FOUNDRY_AUTH_MODE=entra_id and leave FOUNDRY_API_KEY empty. The backend uses DefaultAzureCredential from the Azure Identity SDK, which automatically picks up:

  • Managed Identity when running on Azure (App Service, AKS, Container Apps)
  • Azure CLI credentials during local development
  • Environment variables (AZURE_CLIENT_ID, AZURE_TENANT_ID, AZURE_CLIENT_SECRET) for service principal auth
bash
FOUNDRY_AUTH_MODE=entra_id
FOUNDRY_API_KEY=

Ensure the identity has the Cognitive Services OpenAI User role on the Foundry resource.

Model Naming Conventions ​

GhostAI maps Foundry model names using the deployment name you configure in Azure:

Azure Deployment NameGhostAI Model ID
gpt-4.1gpt-4.1
gpt-4.1-minigpt-4.1-mini
gpt-4.1-nanogpt-4.1-nano
gpt-5gpt-5
gpt-5-minigpt-5-mini
o4-minio4-mini

Use the exact deployment name as the model ID when selecting models in the GhostAI UI. If your deployment name differs from the base model (e.g. my-gpt5-deployment), use that custom name.

PII Referee Configuration ​

To use a Foundry-hosted model as the PII referee instead of a local model:

bash
PII_REFEREE_RUNTIME=foundry
PII_REFEREE_FOUNDRY_MODEL=gpt-4.1-mini

The referee reuses FOUNDRY_ENDPOINT and FOUNDRY_API_KEY / Entra ID credentials from the main Foundry configuration. Token usage and cost are tracked via dedicated Prometheus metrics (ghostai_referee_foundry_tokens_total, ghostai_referee_foundry_cost_dollars).

Embedding Backend Configuration ​

To use Foundry for RAG and memory embeddings instead of local Sentence Transformers:

bash
EMBEDDING_BACKEND=foundry
FOUNDRY_EMBEDDING_MODEL=text-embedding-3-large

The embedding client shares the same FOUNDRY_ENDPOINT and authentication settings. Embedding dimensions default to the model's native output (3072 for text-embedding-3-large).

Cost Tracking ​

Foundry model costs are tracked automatically using static per-million-token pricing:

ModelInput (per 1M)Output (per 1M)
gpt-4.1$2.00$8.00
gpt-4.1-mini$0.40$1.60
gpt-4.1-nano$0.10$0.40
gpt-5$5.00$15.00
gpt-5-mini$1.25$5.00
o4-mini$1.10$4.40

These are billed through your Azure subscription, not through GhostAI. The cost tracking in GhostAI provides visibility into usage patterns for budgeting and chargeback.

Troubleshooting ​

"401 Unauthorized" or "403 Forbidden" ​

  • API key mode: verify FOUNDRY_API_KEY is correct and not expired. Regenerate from the Azure Portal if needed.
  • Entra ID mode: confirm the managed identity or service principal has the Cognitive Services OpenAI User role assigned at the Foundry resource scope (not just the resource group).

"404 Model Not Found" ​

  • The model deployment name in Azure must match the model ID you're requesting. Check Azure AI Foundry → Deployments for the exact name.
  • Ensure the model is fully deployed (status: Succeeded) and not in a provisioning state.

Timeout Errors ​

  • Default LLM timeout is 120s. For large context windows or reasoning models (o4-mini), consider increasing:
    bash
    LLM_TIMEOUT=180
  • Check Azure Portal → Foundry resource → Metrics for throttling (HTTP 429). You may need to increase your TPM (tokens-per-minute) quota.

Entra ID Credential Chain Failures ​

  • Locally: ensure az login is current (az account get-access-token --resource https://cognitiveservices.azure.com)
  • On Azure: confirm the system-assigned or user-assigned managed identity is enabled on the compute resource
  • Check that azure-identity is installed: pip install azure-identity

Prometheus Metrics Not Appearing ​

Foundry referee metrics (ghostai_referee_foundry_tokens_total, ghostai_referee_foundry_cost_dollars) only appear after the first Foundry referee call. Verify the referee is configured with PII_REFEREE_RUNTIME=foundry and trigger a test message.

SOC 2 Type II Certified | BAA Available | HIPAA-Ready