Appearance
Chat Endpoints
Send messages to AI with automatic PII protection.
Send Message
Send a chat message with PII detection and protection.
POST /api/chatRequest
json
{
"message": "My SSN is 123-45-6789. Can you help?",
"session_id": "sess_abc123",
"model": "gpt-4",
"settings": {
"threshold": 0.5,
"tokenize": true,
"stream": false
}
}| Field | Type | Required | Description |
|---|---|---|---|
message | string | Yes | User message |
session_id | string | No | Existing session ID |
model | string | No | LLM model to use |
route | string | No | Preferred route source (e.g. openai-direct, openrouter, ollama). Falls back to primary provider if route unavailable. |
settings.threshold | number | No | PII detection threshold |
settings.tokenize | boolean | No | Enable tokenization |
settings.stream | boolean | No | Enable streaming |
custom_system_prompt | string | No | Per-session custom system instructions (max 50,000 chars). PII-tokenized before injection. |
assistant_id | string | No | Load instructions/config from a saved AI Assistant |
max_tokens | integer | No | Override the model's max output tokens (1–65536) |
exclude_entity_spans | array | No | Per-message override. Spans to exclude from PII redaction for this request only. Each item: { "start": number, "end": number, "entity_type": string, "source"?: string }. source is "message" for main text or "attachment:filename" for attachment content. High-risk types (SSN, credit card, etc.) are never excluded. Requires org policy pii_allow_per_message_override. |
add_to_safe_terms_and_send | string[] | No | Per-message override. Terms to add to the user's Safe Terms and send unredacted this message. Deduplicated and normalized. Requires org policy pii_allow_per_message_override. |
Response
json
{
"response": "Based on your SSN 123-45-6789, here's my advice...",
"session_id": "sess_abc123",
"message_id": "msg_xyz789",
"pii_detected": 1,
"entities": [
{
"type": "US_SSN",
"original": "123-45-6789",
"token": "[US_SSN_a1b2c3]",
"confidence": 0.99,
"start": 10,
"end": 21
}
],
"usage": {
"prompt_tokens": 25,
"completion_tokens": 150,
"total_tokens": 175
}
}When the organization disables per-message overrides but the client sends exclude_entity_spans or add_to_safe_terms_and_send, the response includes pii_override_warning with a message directing users to add safe terms in Settings.
The actual route used for the request is logged in response metadata and diagnostic traces.
Chat UI rendering: Assistant responses are rendered as markdown in the chat UI. Fenced code blocks (e.g. ```python) are syntax-highlighted client-side (Prism, One Dark theme) with copy, download, and language label. Generated PDFs use server-side Pygments for code highlighting.
Example
bash
curl -X POST https://api.ghostai.com/v1/chat \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"message": "My name is John Smith and my email is john@example.com",
"settings": {
"threshold": 0.5
}
}'Enhance Prompt
Enhance a rough prompt with detail, structure, and industry-specific context.
POST /api/chat/enhanceRequest
json
{
"message": "explain this lab result",
"session_id": "sess_abc123"
}| Field | Type | Required | Description |
|---|---|---|---|
message | string | Yes | Raw user prompt to enhance (1-10000 chars) |
session_id | string | No | Current session ID for conversation context |
Response
json
{
"enhanced": "As a clinical professional, I need you to interpret the following lab result. Explain each biomarker's significance, flag any values outside normal ranges, suggest potential differential diagnoses, and recommend follow-up tests. Present in a structured clinical note format.",
"original": "explain this lab result",
"model": "gpt-4o-mini"
}| Field | Type | Description |
|---|---|---|
enhanced | string | The context-aware enhanced prompt |
original | string | The original input (for undo) |
model | string | Model used for enhancement |
Context Used
The enhancement gathers context automatically from the authenticated user's session:
- Industry vertical from organization configuration
- Custom instructions (about_user, response_style)
- Matter context (legal vertical: matter name, practice area)
- Recent messages from the current conversation (last 6 messages)
Example
bash
curl -X POST https://api.ghostai.com/api/chat/enhance \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"message": "analyze these transactions",
"session_id": "sess_abc123"
}'Streaming
Enable streaming for real-time responses.
POST /api/chat/streamRequest
json
{
"message": "Tell me a story",
"session_id": "sess_abc123",
"stream": true,
"route": "openai-direct"
}| Field | Type | Required | Description |
|---|---|---|---|
message | string | Yes | User message |
session_id | string | No | Existing session ID |
stream | boolean | No | Enable streaming |
route | string | No | Preferred route source (e.g. openai-direct, openrouter, ollama). Falls back to primary provider if route unavailable. |
exclude_entity_spans | array | No | Per-message override: spans to exclude from PII redaction for this request only. Same shape as non-streaming. |
add_to_safe_terms_and_send | string[] | No | Per-message override: terms to add to Safe Terms and send unredacted. Same as non-streaming. |
Response (Server-Sent Events)
data: {"type": "start", "session_id": "sess_abc123"}
data: {"type": "token", "content": "Once"}
data: {"type": "token", "content": " upon"}
data: {"type": "token", "content": " a"}
data: {"type": "token", "content": " time"}
data: {"type": "pii", "entity": {"type": "PERSON", "token": "[PERSON_x1]"}}
data: {"type": "tool_start", "tool_name": "web_search", "tool_id": "call_abc123"}
data: {"type": "tool_result", "tool_name": "web_search", "tool_id": "call_abc123", "result_preview": "Found 5 results..."}
data: {"type": "end", "usage": {"total_tokens": 50}, "time_to_first_token_ms": 250, "llm_total_time_ms": 3200, "performance": {"pii_detection_ms": 12.3, "tokenization_ms": 0.8, "prompt_build_ms": 1.2, "llm_ttft_ms": 250.0, "llm_streaming_ms": 2950.0, "tool_execution_ms": 0.0, "detokenization_ms": 0.5, "db_save_ms": 3.1, "total_ms": 3218.0}}Public Figure Reconciliation in start Event
When public figure reconciliation is enabled, the start event includes an optional reconciled_public_figures array. Each entry describes a PERSON entity that was identified as a public figure and excluded from PII masking. Note: entities blocked by safety filters (co-occurrence filter, multi-detector consensus gate) will NOT appear here — they remain masked as PII.
data: {"type": "start", "session_id": "sess_abc123", "reconciled_public_figures": [{"entity_type": "PERSON", "text": "Albert Einstein", "start": 7, "end": 22, "reconciled_as": "Albert Einstein", "reconciled_source": "builtin", "reconciled_category": "scientist", "reconciled_confidence": 0.99}]}| Field | Type | Description |
|---|---|---|
entity_type | string | Always "PERSON" |
text | string | Original text span from the user message |
start | number | Start character offset |
end | number | End character offset |
reconciled_as | string | Canonical name of the matched public figure |
reconciled_source | string | Resolution tier: "builtin", "wikidata", or "llm" |
reconciled_category | string | Category (e.g. "scientist", "politician") |
reconciled_confidence | number | Confidence score (0-1) |
Example: A message like "Who is Albert Einstein?" with public figure reconciliation enabled produces a start event containing:
json
{
"type": "start",
"session_id": "sess_abc123",
"reconciled_public_figures": [
{
"entity_type": "PERSON",
"text": "Albert Einstein",
"start": 7,
"end": 22,
"reconciled_as": "Albert Einstein",
"reconciled_source": "builtin",
"reconciled_category": "scientist",
"reconciled_confidence": 0.99
}
]
}The name "Albert Einstein" is not masked or tokenized — it passes through to the LLM as-is. See Detection Endpoints for full field descriptions.
PII Override Warning in start Event
When the organization disables per-message overrides (pii_allow_per_message_override = false) but the client sends exclude_entity_spans or add_to_safe_terms_and_send, the start event includes pii_override_warning (string). The client should display this message and not apply overrides for that request.
Public Knowledge Reconciliation in start Event
When public knowledge reconciliation is enabled, start can also include reconciled_public_knowledge entries for entities like LOCATION, ORGANIZATION, and EVENT.
json
{
"type": "start",
"session_id": "sess_abc123",
"reconciled_public_knowledge": [
{
"entity_type": "LOCATION",
"text": "Utah",
"start": 25,
"end": 30,
"reconciled_as": "public_knowledge",
"reconciled_source": "builtin",
"reconciled_category": "us_state_or_territory",
"reconciled_confidence": 1.0
}
]
}Performance Breakdown (Latency Waterfall)
The done event includes a performance object with granular timing for each phase of request processing. This powers the user-facing Performance Waterfall in the chat UI.
| Field | Description | Category |
|---|---|---|
pii_detection_ms | Time to detect PII entities in the user message | GhostAI |
tokenization_ms | Time to tokenize (replace) detected PII | GhostAI |
prompt_build_ms | Time to build system prompt, custom instructions, tool awareness | GhostAI |
llm_ttft_ms | Time from LLM request to first token received | External LLM |
llm_streaming_ms | Time spent streaming tokens (total LLM time minus TTFT) | External LLM |
tool_execution_ms | Total time spent executing tools (web search, image gen) | External |
detokenization_ms | Time to restore PII tokens in the response | GhostAI |
db_save_ms | Time to persist messages to the database | GhostAI |
total_ms | End-to-end request time from the server perspective | Total |
Tool Execution Events
When the AI invokes tools (web search, image generation), additional SSE events are emitted:
| Event Type | Description |
|---|---|
tool_start | Tool execution has begun, includes tool_name and tool_id |
tool_result | Tool finished, includes result_preview with a summary |
Supported tools:
web_search— Real-time web search (Tavily, Brave, or Serper)generate_image— Image generation via DALL-E 3generate_pdf— PDF generation from markdown contentcreate_canvas— Generate interactive HTML canvas artifacts (CSP-sandboxed, PII-scanned)update_canvas— Update existing canvas artifacts with version tracking
generate_pdf
Generate a professional PDF document from rich markdown content. Supports world-class rendering of mermaid diagrams, LaTeX math, styled tables, syntax-highlighted code, embedded images, blockquotes, and clickable links — all with graceful fallback if any element fails to render. Heavy assets (mermaid, math, images, code highlighting) are rendered in parallel for improved response time on asset-heavy documents.
Parameters:
| Parameter | Type | Required | Description |
|---|---|---|---|
title | string | Yes | Document title |
content | string | Yes | Rich markdown content. Supports: mermaid diagrams (```mermaid blocks rendered via mermaid.ink API), LaTeX math (block $$...$$ and inline $...$ rendered via matplotlib), pipe-delimited tables (with styled branded headers), syntax-highlighted code blocks (```lang rendered via Pygments), image embedding (URLs including GhostAI-generated images), blockquotes (with left border styling), and clickable links ([text](url)). All rich elements fall back gracefully if rendering fails. |
template | string | No | Template: briefing, report, memo, summary (default: report) |
page_size | string | No | Page size: letter, a4 (default: letter) |
ocr_text | boolean | No | Add searchable text layer (default: false) |
SSE Response:
json
{
"type": "tool_result",
"tool_name": "generate_pdf",
"pdf_url": "/api/generated-pdfs/Report_a1b2c3d4.pdf",
"filename": "Report_a1b2c3d4.pdf",
"pages": 3,
"contains_pii": false,
"result_preview": "PDF generated: Report_a1b2c3d4.pdf"
}When the PDF contains detokenized PII (zero-trust secure mode), contains_pii is true.
PDFs are served as static files at /api/generated-pdfs/.
Example (JavaScript)
javascript
const response = await fetch('https://api.ghostai.com/v1/chat/stream', {
method: 'POST',
headers: {
'Authorization': 'Bearer YOUR_TOKEN',
'Content-Type': 'application/json'
},
body: JSON.stringify({
message: 'Tell me a story',
stream: true
})
});
const reader = response.body.getReader();
const decoder = new TextDecoder();
while (true) {
const { done, value } = await reader.read();
if (done) break;
const chunk = decoder.decode(value);
const lines = chunk.split('\n');
for (const line of lines) {
if (line.startsWith('data: ')) {
const data = JSON.parse(line.slice(6));
console.log(data);
}
}
}Sessions
List Sessions
GET /api/sessionsQuery parameters:
page- Page number (default: 1)limit- Results per page (default: 20)
Response:
json
{
"sessions": [
{
"id": "sess_abc123",
"title": "Tax Questions",
"created_at": "2024-01-15T10:30:00Z",
"last_accessed": "2024-01-15T11:00:00Z",
"message_count": 5,
"pii_count": 3
}
],
"pagination": {
"page": 1,
"limit": 20,
"total": 45
}
}Get Session
GET /api/sessions/{session_id}Response:
json
{
"id": "sess_abc123",
"title": "Tax Questions",
"created_at": "2024-01-15T10:30:00Z",
"messages": [
{
"id": "msg_1",
"role": "user",
"content": "My SSN is [US_SSN_a1b2]...",
"timestamp": "2024-01-15T10:30:00Z",
"pii_entities": [...]
},
{
"id": "msg_2",
"role": "assistant",
"content": "Based on your information...",
"timestamp": "2024-01-15T10:30:05Z"
}
],
"token_count": 3
}Delete Session
DELETE /api/sessions/{session_id}Response:
json
{
"deleted": true,
"tokens_cleared": 5
}Get Session Tokens
View token mappings for a session.
GET /api/sessions/{session_id}/tokensResponse:
json
{
"session_id": "sess_abc123",
"tokens": [
{
"token": "[US_SSN_a1b2]",
"type": "US_SSN",
"created_at": "2024-01-15T10:30:00Z"
},
{
"token": "[PERSON_c3d4]",
"type": "PERSON",
"created_at": "2024-01-15T10:30:00Z"
}
],
"count": 2
}WARNING
Original values are not returned for security. Use detokenize endpoint with session context.
Conversation Management
Rename Conversation
PUT /api/conversations/{session_id}Request:
json
{
"title": "My Custom Title"
}Generate Smart Title
AI-generated title based on conversation content.
POST /api/conversations/{conversation_id}/generate-titleRequest:
json
{
"current_title": "My first message text..."
}Response:
json
{
"title": "Tax Planning Questions"
}Custom Instructions
Get Custom Instructions
GET /api/settings/custom-instructionsResponse:
json
{
"about_user": "I'm a healthcare compliance officer...",
"response_style": "Be concise and use bullet points",
"enabled": true
}Update Custom Instructions
PUT /api/settings/custom-instructionsRequest:
json
{
"about_user": "I'm a healthcare compliance officer at a 500-bed hospital",
"response_style": "Be concise, use bullet points, cite regulations",
"enabled": true
}Custom instructions are injected into the system prompt for all subsequent conversations when enabled.
Chat Flow Debugger (Admin)
Diagnostic endpoints for platform admins to inspect the full lifecycle of chat requests. Requires is_platform_admin=true.
List Diagnostic Traces
GET /api/admin/chat-debug/tracesQuery parameters:
| Parameter | Type | Default | Description |
|---|---|---|---|
limit | int | 50 | Max traces to return (1-200) |
session_id | string | - | Filter by session ID |
user_id | string | - | Filter by user ID |
has_error | bool | - | Filter to traces with/without errors |
has_tool_calls | bool | - | Filter to traces with/without tool calls |
Response:
json
{
"traces": [
{
"trace_id": "abc-123",
"session_id": "sess-456",
"user_id": "user-789",
"started_at": 1738900000.0,
"model": "openai/gpt-4o-mini",
"provider": "openrouter",
"message_preview": "What is the latest AI news?",
"duration_ms": 3200.5,
"event_count": 10,
"has_tool_calls": true,
"has_fallback": false,
"has_error": false,
"status": "success",
"tools_available": ["web_search", "generate_image", "create_canvas", "update_canvas"],
"tools_enabled": ["web_search"],
"finished": true
}
],
"count": 1
}Get Trace Detail
GET /api/admin/chat-debug/traces/{trace_id}Returns the full trace including all diagnostic events:
json
{
"trace_id": "abc-123",
"session_id": "sess-456",
"events": [
{
"timestamp_ms": 0.0,
"wall_time": 1738900000.0,
"event_type": "pii_detect",
"detail": { "entity_count": 3, "detection_time_ms": 12.5 }
},
{
"timestamp_ms": 12.5,
"wall_time": 1738900000.012,
"event_type": "pii_tokenize",
"detail": { "token_count": 3 }
},
{
"timestamp_ms": 13.0,
"wall_time": 1738900000.013,
"event_type": "tools_resolved",
"detail": { "available": ["web_search"], "final": ["web_search"], "tool_count": 1 }
},
{
"timestamp_ms": 14.0,
"wall_time": 1738900000.014,
"event_type": "llm_request",
"detail": { "model": "openai/gpt-4o-mini", "provider": "openrouter", "tool_count": 1 }
},
{
"timestamp_ms": 850.0,
"wall_time": 1738900000.85,
"event_type": "llm_first_token",
"detail": { "ttft_ms": 836.0, "tool_round": 0 }
}
]
}Event Types
| Event | Description |
|---|---|
pii_detect | PII detection completed (entity count, timing) |
pii_tokenize | PII tokenization completed (token count) |
tools_resolved | Tool availability resolved after user filtering |
llm_request | LLM API call initiated (model, provider, tools, message count) |
llm_first_token | First streaming token received (TTFT) |
tool_call | LLM requested a tool call (tool name, arguments) |
tool_result | Tool execution completed (result preview, timing) |
fallback_no_tools | Provider rejected tools, retried without (error, stripped tools) |
detokenize | PII detokenization completed (response length) |
llm_done | Stream completed (total time, response length) |
error | An error occurred (message, type) |
Clear All Traces
DELETE /api/admin/chat-debug/tracesResponse:
json
{
"cleared": 42,
"message": "Cleared 42 traces"
}File Pipeline Diagnostics
Diagnostic endpoint for monitoring worker health and file processing configuration. No authentication required (designed for health checks and monitoring).
Get Pipeline Status
GET /health/file-pipelineResponse:
json
{
"worker_pid": 12345,
"peak_rss_mb": 842.3,
"pii_ready": true,
"uvicorn_workers": "4",
"pii_max_concurrent": 4,
"rag_threshold_mb": 1.0,
"rag_embedding_model": "all-MiniLM-L6-v2",
"rag_embedding_device": "cpu",
"rag_hybrid_search_enabled": true,
"rag_chunk_size_chars": 2000,
"rag_chunk_overlap_chars": 200,
"rag_tabular_chunk_rows": 50,
"rag_ann_threshold": 100000
}| Field | Type | Description |
|---|---|---|
worker_pid | int | OS process ID of the responding worker |
peak_rss_mb | float | Peak resident set size (memory high-water mark) in MB |
pii_ready | bool | Whether PII detection models are loaded and warm |
uvicorn_workers | string | Number of configured Gunicorn/Uvicorn workers |
pii_max_concurrent | int | Max concurrent PII detection operations |
rag_threshold_mb | float | File size threshold (MB) for RAG indexing |
rag_embedding_model | string | Sentence-transformer model used for embeddings |
rag_embedding_device | string | Device for embedding inference (cpu, cuda, mps) |
rag_hybrid_search_enabled | bool | Whether BM25 + semantic fusion is active |
rag_chunk_size_chars | int | Characters per text chunk |
rag_chunk_overlap_chars | int | Overlap between consecutive chunks |
rag_tabular_chunk_rows | int | Rows per chunk for CSV/spreadsheet files |
rag_ann_threshold | int | Switch to HNSW approximate search above this chunk count |
Use this endpoint to:
- Monitor worker memory pressure before and after file processing
- Verify PII models are loaded after deployment
- Confirm RAG configuration and embedding device match expectations
- Check worker configuration matches expectations
bash
# Check worker memory
curl -s http://localhost:8000/health/file-pipeline | python3 -m json.toolOllama Model Management (Platform Admin)
Manage Ollama models from the API. All endpoints require is_platform_admin=true.
List Installed Ollama Models
GET /api/admin/ollama/modelsResponse:
json
{
"models": [
{
"name": "llama3.1:8b",
"size": 4661224676,
"parameter_size": "8.0B",
"quantization_level": "Q4_0",
"modified_at": "2026-02-20T10:30:00Z",
"digest": "a1b2c3d4..."
}
]
}Pull an Ollama Model
Pull a model from the Ollama library. Streams NDJSON progress events.
POST /api/admin/ollama/pullRequest:
json
{
"model": "deepseek-r1:14b"
}| Field | Type | Required | Description |
|---|---|---|---|
model | string | Yes | Model name to pull (e.g. llama3.1:8b, deepseek-r1:14b) |
Response (NDJSON stream):
{"status":"pulling manifest"}
{"status":"downloading","digest":"sha256:abc...","total":4661224676,"completed":1024000}
{"status":"downloading","digest":"sha256:abc...","total":4661224676,"completed":2048000}
{"status":"verifying sha256 digest"}
{"status":"writing manifest"}
{"status":"success"}Delete an Ollama Model
DELETE /api/admin/ollama/modelsRequest:
json
{
"model": "llama3.1:8b"
}| Field | Type | Required | Description |
|---|---|---|---|
model | string | Yes | Name of the installed model to delete |
Response:
json
{
"status": "success",
"deleted": "llama3.1:8b"
}Returns 404 if the model is not installed. Returns 403 if the caller is not a platform admin.
Message Feedback
Submit and retrieve per-message feedback (thumbs up/down). Feedback is persisted in the message_feedback database table.
Submit Feedback
POST /api/chat/messages/{message_id}/feedbackRequest:
json
{
"vote": "up"
}| Field | Type | Required | Description |
|---|---|---|---|
vote | string | Yes | "up" or "down" |
Response:
json
{
"message_id": "msg_xyz789",
"vote": "up",
"created_at": "2025-02-07T15:30:00Z"
}Get Feedback
GET /api/chat/messages/{message_id}/feedbackResponse:
json
{
"message_id": "msg_xyz789",
"vote": "up",
"created_at": "2025-02-07T15:30:00Z"
}Returns 404 if no feedback has been submitted for the message.
Example
bash
# Submit thumbs-up feedback
curl -X POST https://api.ghostai.com/api/chat/messages/msg_xyz789/feedback \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{"vote": "up"}'
# Retrieve feedback
curl https://api.ghostai.com/api/chat/messages/msg_xyz789/feedback \
-H "Authorization: Bearer YOUR_TOKEN"Combined Answer (Multi-Model Deliberation)
Run a 3-stage deliberation pipeline that produces one clean answer from multiple models.
Combine Endpoint
POST /api/chat/combineTriggers the full 3-stage pipeline (Initial → Deliberation → Blend) and streams the result via SSE.
Request
json
{
"message": "Explain the pros and cons of microservices",
"session_id": "sess_abc123",
"models": ["openai/gpt-4o", "anthropic/claude-3.5-sonnet", "google/gemini-pro"],
"blend_model": "openai/gpt-4o",
"settings": {
"threshold": 0.5,
"tokenize": true
}
}| Field | Type | Required | Description |
|---|---|---|---|
message | string | Yes | User prompt |
session_id | string | No | Existing session ID |
models | string[] | Yes | 2-4 model IDs to use for initial responses and deliberation |
blend_model | string | No | Model to use for the final blend stage (defaults to first model) |
settings.threshold | number | No | PII detection threshold |
settings.tokenize | boolean | No | Enable tokenization |
Response (Server-Sent Events)
data: {"type": "stage", "stage": "initial", "model": "openai/gpt-4o", "status": "streaming"}
data: {"type": "token", "stage": "initial", "model": "openai/gpt-4o", "content": "Micro"}
data: {"type": "stage", "stage": "initial", "model": "openai/gpt-4o", "status": "done"}
data: {"type": "stage", "stage": "deliberation", "model": "openai/gpt-4o", "status": "streaming"}
data: {"type": "token", "stage": "deliberation", "model": "openai/gpt-4o", "content": "After"}
data: {"type": "stage", "stage": "deliberation", "model": "openai/gpt-4o", "status": "done"}
data: {"type": "stage", "stage": "blend", "status": "streaming"}
data: {"type": "token", "stage": "blend", "content": "When"}
data: {"type": "done", "combined_text": "When considering microservices...", "usage": {"prompt_tokens": 850, "completion_tokens": 600, "total_tokens": 1450}, "stages_completed": ["initial", "deliberation", "blend"]}SSE Event Types
| Event Type | Stage | Description |
|---|---|---|
stage | all | Stage lifecycle event — includes stage name, optional model, and status (streaming or done) |
token | all | Streaming content token — includes stage, optional model, and content |
pii | all | PII entity detected during any stage |
done | — | Pipeline complete — includes combined_text, usage, and stages_completed |
error | any | Error during pipeline — includes message, stage, and optional model |
Combined Mode on Streaming Endpoint
The existing streaming endpoint also supports combined mode via additional fields.
POST /api/chat/streamAdditional Request Fields
json
{
"message": "Explain the pros and cons of microservices",
"session_id": "sess_abc123",
"stream": true,
"combined_mode": true,
"combined_models": ["openai/gpt-4o", "anthropic/claude-3.5-sonnet", "google/gemini-pro"]
}| Field | Type | Required | Description |
|---|---|---|---|
combined_mode | boolean | No | Enable combined answer mode (3-stage pipeline). Default: false |
combined_models | string[] | No | 2-4 model IDs for the pipeline. Required when combined_mode is true |
When combined_mode is true, the response SSE stream includes the same stage and token events as the /api/chat/combine endpoint. The final done event includes combined_text with the blended answer.
Example
bash
# Full combined answer pipeline
curl -N -X POST https://api.ghostai.com/api/chat/combine \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"message": "Explain the pros and cons of microservices",
"models": ["openai/gpt-4o", "anthropic/claude-3.5-sonnet"],
"blend_model": "openai/gpt-4o"
}'bash
# Combined mode via streaming endpoint
curl -N -X POST https://api.ghostai.com/api/chat/stream \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"message": "Explain the pros and cons of microservices",
"stream": true,
"combined_mode": true,
"combined_models": ["openai/gpt-4o", "anthropic/claude-3.5-sonnet"]
}'Analytics (Admin)
Admin endpoints for viewing aggregated feedback and performance metrics. Requires admin privileges.
Vote Summary
Aggregated thumbs up/down counts across all messages.
GET /api/admin/analytics/votesQuery parameters:
| Parameter | Type | Default | Description |
|---|---|---|---|
start_date | string | - | ISO 8601 start date filter |
end_date | string | - | ISO 8601 end date filter |
Response:
json
{
"total_votes": 1250,
"thumbs_up": 980,
"thumbs_down": 270,
"approval_rate": 0.784
}Feedback Details
Detailed feedback records with filtering support.
GET /api/admin/analytics/feedbackQuery parameters:
| Parameter | Type | Default | Description |
|---|---|---|---|
start_date | string | - | ISO 8601 start date filter |
end_date | string | - | ISO 8601 end date filter |
vote | string | - | Filter by vote type ("up" or "down") |
user_id | string | - | Filter by user ID |
limit | int | 50 | Max records to return |
offset | int | 0 | Pagination offset |
Response:
json
{
"feedback": [
{
"message_id": "msg_xyz789",
"user_id": "user_123",
"vote": "up",
"model": "gpt-4o",
"created_at": "2025-02-07T15:30:00Z"
}
],
"total": 1250,
"limit": 50,
"offset": 0
}Latency Analytics
Historical performance metrics from the performance_metrics database table.
GET /api/admin/analytics/latencyQuery parameters:
| Parameter | Type | Default | Description |
|---|---|---|---|
start_date | string | - | ISO 8601 start date filter |
end_date | string | - | ISO 8601 end date filter |
model | string | - | Filter by model name |
granularity | string | day | Aggregation granularity (hour, day, week) |
Response:
json
{
"metrics": [
{
"period": "2025-02-07",
"request_count": 450,
"avg_total_ms": 3200.5,
"p50_total_ms": 2800.0,
"p95_total_ms": 5400.0,
"avg_llm_ttft_ms": 850.0,
"avg_pii_detection_ms": 12.3,
"avg_ghostai_overhead_ms": 18.5
}
],
"summary": {
"total_requests": 450,
"avg_total_ms": 3200.5,
"avg_ghostai_overhead_pct": 0.58
}
}File Attachments & RAG Retrieval
When a chat message includes file attachments (attachment_ids), the chat pipeline automatically integrates RAG retrieval for indexed files.
How RAG Works in Chat
Single indexed file: The user's message is used as a semantic query. Relevant chunks are retrieved via hybrid BM25+semantic search (with RRF merge) and injected into the LLM context, replacing the full file text.
Multiple indexed files: Cross-file retrieval activates automatically. Chunks from all attached files are scored together against the query, with a shared token budget. Results include source attribution per file.
PII protection: Retrieved chunks pass through the full PII detection and tokenization pipeline before reaching the LLM. Chunk-level PII caching avoids redundant detection when the same chunks are retrieved across queries.
Token-budget-aware: The number of chunks injected adapts to the model's context window — more for Gemini (1M tokens), fewer for GPT-4 (8K tokens). The system allocates 50% of the context window for file content.
Request Example
bash
curl -X POST https://api.ghostai.com/api/chat \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"message": "What are the top revenue customers?",
"session_id": "sess_abc123",
"attachment_ids": ["file_xyz789"],
"model": "gpt-4o"
}'The response will be based on the most relevant chunks from the attached file, with all PII tokenized.
Compare Models
Compare responses from multiple LLMs side-by-side with the same PII-protected prompt.
POST /api/chat/compareRequest
| Field | Type | Required | Description |
|---|---|---|---|
message | string | Yes | User message |
models | string[] | Yes | 1-4 model IDs to compare |
session_id | string | No | Existing session ID |
attachments | string[] | No | File attachment IDs |
enabled_tools | string[] | No | Tools to enable (e.g. web_search, execute_python) |
tool_mode | string | No | per_model (default) or shared |
custom_system_prompt | string | No | Per-session system instructions (max 50K chars) |
assistant_id | string | No | Load config from a saved AI Assistant |
max_tokens | integer | No | Override max output tokens per model |
max_tool_rounds | integer | No | Override max tool execution rounds (1-15) |
memory_enabled | boolean | No | Enable/disable memory recall |
SSE Event Types
| Event | Description |
|---|---|
compare_processing | Connection keepalive emitted immediately and every ~5s during PII detection. Fields: session_id, models, status (scanning, preparing_attachments). Prevents proxy timeouts on large documents. |
compare_start | Session initialized with PII detection results |
compare_chunk | Streaming content chunk from a model |
compare_tool_start | A model initiated a tool call |
compare_tool_result | Tool execution completed |
compare_done | Model finished with performance metrics |
compare_all_done | All models complete |
Large Document Handling
Comparison mode includes optimizations for large file attachments:
- Lower RAG threshold: Files >= 10KB are automatically indexed for RAG retrieval in comparison mode (vs 1MB in regular chat), so only relevant chunks are sent to each model instead of the full document.
- Async PII with heartbeats: PII detection runs as a background task with
compare_processingheartbeat events emitted every 5 seconds, keeping the SSE connection alive through proxies. - PII scan caching: If a file was previously scanned via
POST /api/files/{file_id}/scan, cached PII results are reused during chat, eliminating redundant detection.
Code Interpreter
The execute_python tool runs Python code in an isolated sandbox with no network access. Uploaded files are available at /workspace/. Output is PII-scanned before reaching the LLM.
The sandbox supports two pluggable backends:
- Docker — For local dev and docker-compose deployments. Hardened with read-only filesystem, tmpfs mounts,
pids_limit=100,no-new-privileges, restricted builtins, and import allowlist. - Lambda — For AWS ECS Fargate and Kubernetes deployments. Uses AWS Lambda with container images, S3 for file transfer, minimal IAM, and VPC isolation.
Security is enforced at multiple layers: tool_validator.py blocks eval/exec/compile/importlib at pre-execution validation; the sandbox entrypoint restricts builtins and only allows whitelisted imports. The feature is gated behind billing (Team plan+), has per-user rate limiting (default 20/min), and supports gradual rollout.
Sandbox Environment
- Libraries: pandas, numpy, matplotlib, seaborn, scipy, nltk, textblob, scikit-learn, beautifulsoup4, lxml
- Limits: 2 CPU cores, 4GB RAM, 120s timeout
- Output: stdout/stderr captured, artifacts from
/workspace/output/collected
Configuration
| Variable | Default | Description |
|---|---|---|
SANDBOX_ENABLED | true | Master toggle |
SANDBOX_BACKEND | docker | Backend: docker (local dev) or lambda (AWS/Kubernetes) |
SANDBOX_IMAGE | ghostai-sandbox:latest | Docker image (Docker backend) |
SANDBOX_LAMBDA_FUNCTION | — | Lambda function name (Lambda backend) |
SANDBOX_S3_BUCKET | — | S3 bucket for file transfer (Lambda backend) |
SANDBOX_RATE_LIMIT | 20 | Per-user executions per minute |
SANDBOX_TIMEOUT | 120 | Execution timeout (seconds) |
SANDBOX_CPU_LIMIT | 2.0 | CPU cores per container (Docker backend) |
SANDBOX_MEMORY_LIMIT | 4g | RAM per container (Docker backend) |
SANDBOX_MAX_CONCURRENT | 5 | Max parallel containers (Docker backend) |
Prometheus metrics are emitted with a backend label; CloudWatch alarms and Grafana dashboards are available for Lambda deployments.
AI Assistants
Reusable AI assistant configurations with custom instructions, knowledge files, and default settings.
GET /api/assistants # List all (own + shared)
POST /api/assistants # Create new assistant
GET /api/assistants/:id # Get by ID
PUT /api/assistants/:id # Update
DELETE /api/assistants/:id # Delete
POST /api/assistants/:id/duplicate # CloneAssistant Schema
| Field | Type | Description |
|---|---|---|
name | string | Display name (required) |
description | string | Brief description |
instructions | string | System instructions (max 50K chars) |
knowledge_file_ids | string[] | Attached knowledge file IDs |
starter_message | string | Pre-filled message (max 10K chars) |
max_tokens | integer | Output token limit override |
max_tool_rounds | integer | Tool round limit override (1-15) |
default_models | string[] | Default model selection |
is_shared | boolean | Share with organization |
icon | string | Display icon/emoji |