Appearance
Context Window Optimization
GhostAI automatically manages the AI's context window — the working memory that holds your conversation, files, tools, and instructions — so the AI stays focused on what matters.
How It Works
Every AI model has a limited context window (measured in tokens). When conversations get long or involve many files and tools, the context can fill up with redundant information — verbose tool output, repeated file headers, guidance for tools you aren't using. GhostAI applies intelligent filtering to compress this noise while preserving the information the AI needs to reason well.
What Gets Optimized
- System prompt — Detailed guidance for canvas, code interpreter, search, and other tools is included only when those tools are active and relevant to your message.
- Tool output — When the AI uses tools (search, code execution, data queries), the raw output is compressed before being added to context. Test results show only failures and statistics; data tables show structure rather than every row.
- Conversation history — In long tool-heavy sessions, older tool call details are summarized while keeping recent context intact.
- File wrappers — Documents are injected with compact headers instead of verbose per-file instructions.
Fail-Safe Design
If any optimization encounters an error, the original unmodified content passes through. The AI never receives less information than it would without optimization — it just receives better-organized information.
Context Composition Bar
Team tier and above users see a segmented bar showing how their context window is allocated:
| Color | Category | What It Includes |
|---|---|---|
| Blue | System | System prompt and tool guidance |
| Purple | Memory | Recalled AI memories |
| Amber | Attachments | Files, RAG chunks, Content Library items |
| Gray | History | Prior conversation messages |
| Teal | Tools | Tool schemas and recent tool output |
The bar updates in real time as you add files, use tools, and continue the conversation. Hover over any segment to see its token count and percentage.
Savings Indicator
After each AI response, the ThinkingBlock performance panel shows:
- Efficiency — what percentage of the context window contains useful content
- Tokens Saved — how many tokens were removed by optimization, with a per-strategy breakdown
Why Long Conversations May See Content Trimmed
When the context window fills up, GhostAI trims from the middle of the conversation (preserving the system prompt and recent messages). With context optimization enabled, this threshold is reached much later because noise is removed proactively.
In tool-heavy sessions, older tool call arguments are summarized first, and only if that isn't enough are entire older messages removed.
Overflow Recovery
When a conversation's context exceeds the model's limit — due to long history, many attached files, active canvas, and Content Library items all at once — GhostAI recovers automatically instead of showing an error. Recovery happens in layers, each progressively more aggressive:
Layer 1: Smart Trimming
GhostAI drops the oldest conversation messages first (keeping recent ones intact), then trims Content Library and attachment text. This handles most overflow cases without any visible impact.
Layer 2: History Compression
When trimming alone is not enough, GhostAI summarizes older conversation messages into a compact paragraph using a fast model. The summary preserves key facts, decisions, and entity references so the AI retains full conversational context. Recent messages remain verbatim. This is automatic and transparent — you will not notice it happening.
Layer 3: Split and Converge
For very large payloads (many files, active canvas, and Content Library all competing for space), GhostAI splits the request into two parallel threads using the same model:
- Thread A receives the conversation history and your question.
- Thread B receives the reference documents, files, and canvas content along with your question.
Both threads run simultaneously, and GhostAI then blends their responses into a single coherent answer. The activity indicator shows "Processing parallel threads..." followed by "Converging responses..." so you know what is happening.
Layer 4: Switch to a Larger Model
Alternatively, GhostAI can retry the request with a model that has a larger context window. For example, if you are using GPT-5.3 (400K tokens), GhostAI can switch to GPT-4.1 (1M tokens) to accommodate the full payload.
Your Choice
A setting in Settings → AI Preferences → Context Overflow Strategy lets you control how overflows are handled:
| Setting | Behavior |
|---|---|
| Split Context (default) | Automatically split into parallel threads and converge. No interruption. |
| Switch Model | Automatically retry with a larger-context model. No interruption. |
| Ask Me | Show an inline card with both options so you can choose per-occurrence. |
When set to "Ask Me", a card appears below your message with a "Split & Continue" button and a "Switch Model" button that opens a picker showing available larger-context models with their window sizes.
No Data Loss
All PII protection, memory recall, and tool access remain fully intact regardless of which recovery strategy is used. The split-and-converge path runs the same PII pipeline on both threads.
How Content Library, RAG, and Tools Interact
- Content Library items are injected into context alongside file attachments, with PII scanning applied.
- RAG chunks from file uploads are inserted with compact separators showing chunk number and source filename.
- Tool schemas are always included for enabled tools; tool output from prior rounds is compressed after 2 rounds.
All three compete for the same context budget. The pre-flight budget check ensures the total stays within the model's limits.
Admin Visibility
Organization admins can view aggregated context optimization metrics in the Admin Dashboard → Context Health tab:
- Total tokens saved and estimated cost savings
- Average context efficiency
- Savings breakdown by strategy
- Daily optimization trend
These metrics also appear in the Observer Dashboard and Board Report (Page 5: AI Quality Optimization).