Appearance
Performance
This guide covers optimizing PII detection speed, managing memory usage, and tuning the detection pipeline for your workload.
Detection Latency
Typical Latency by Configuration
| Configuration | Typical Latency | Notes |
|---|---|---|
| Presidio only | 5-15 ms | Pattern matching, fastest option |
| Presidio + NER | 15-40 ms | Adds Dymium NER inference |
| Presidio + NER + AI Detector | 200-800 ms | Adds LLM round-trip |
| Presidio + NER + AI Verify (suspicious) | 50-250 ms | Verifies only low-confidence entities |
| Presidio + NER + AI Verify (all) | 100-500 ms | Verifies all entities |
| Full pipeline (all detectors) | 300-1000 ms | Maximum accuracy, highest latency |
Where Time Is Spent
| Phase | Typical Time | Optimization |
|---|---|---|
| Presidio pattern matching | 5-15 ms | Cache helps with repeated text |
| Dymium NER inference | 10-25 ms | GPU acceleration if available |
| Code & Secrets detection | 3-10 ms | Regex-based, very fast |
| AI Enhancement (LLM call) | 200-500 ms | Use dedicated model (gpt-4o-mini) |
| AI Verification (LLM call) | 50-300 ms | suspicious mode is faster than all |
| Anti-exfiltration decode | 2-5 ms | Minimal overhead |
Execution Modes
The execution mode controls how detectors run relative to each other:
Parallel Mode (Fastest)
All enabled detectors run simultaneously. Total latency is the maximum of any single detector.
Presidio ──────────┐
NER ───────────────┤
Code & Secrets ────┤── Merge results
AI Detector ───────┘Best for: Real-time chat, high-volume APIs, latency-sensitive workloads.
Series Mode (Most Accurate)
Detectors run sequentially. Each detector can use results from previous detectors for context.
Presidio → NER → Code & Secrets → AI Detector → MergeBest for: Maximum accuracy, offline batch processing, compliance audits.
Grouped Mode (Balanced)
Similar detectors run in parallel groups, then groups run in sequence.
Group 1 (Pattern): Presidio + Code & Secrets + Anti-Exfiltration (parallel)
→ Group 2 (ML): NER (parallel)
→ Group 3 (AI): AI DetectorBest for: General-purpose use (default recommendation).
To change the execution mode: Settings → Detection → Execution Mode.
Custom Pipeline Configuration
For fine-grained control over detector ordering and grouping, open Settings → Pipeline. You can:
- Set detector priority — higher values run first in series mode
- Set detector groups — detectors with the same group number run in parallel in grouped mode
- Enable/disable individual detectors — toggle each detector on/off
Custom priority and group values are persisted to your user profile and override the defaults. The execution mode (parallel, series, grouped) still determines the overall pattern, but your custom ordering is respected within that pattern.
Language-Specific Performance
When multiple languages are enabled (via Settings → Languages), Presidio runs detection for each selected language. NER (Dymium NER) only supports English — for non-English text, only Presidio pattern matching is available.
Each additional language adds minimal overhead to Presidio but does not affect NER or AI detector latency.
Memory Management
Model Memory Requirements
| Model | Memory | When Loaded |
|---|---|---|
| Dymium NER | ~800 MB | Always (primary NER model) |
| Clinical NER | ~500 MB | When Healthcare vertical is active |
| CLIP (images) | ~400 MB | When image intelligence is used |
| Presidio engine | ~200 MB | Always |
Total memory with all models: ~2 GB (base) to ~4 GB (with all vertical models)
Recommendations
| Deployment | RAM | Models |
|---|---|---|
| Minimal (Presidio + NER) | 2 GB | Dymium NER only |
| Standard (+ AI detector) | 4 GB | Dymium NER + LLM client |
| Healthcare vertical | 4-6 GB | Dymium NER + Clinical NER |
| Full (all models + images) | 6-8 GB | All models loaded |
Reducing Memory Usage
- Disable unused detectors -- If you don't need the AI detector, turn it off in Settings
- Skip vertical models -- Use the
generalvertical if you don't need industry-specific detection - Lazy loading -- Models load on first use; unused models consume no memory
- Set container limits -- Configure Docker memory limits to prevent OOM kills
Detection Cache
The detection cache avoids redundant processing when the same text is scanned multiple times (e.g., message retries, page reloads).
Cache Configuration
| Variable | Default | Description |
|---|---|---|
PII_CACHE_SIZE | 2000 | Maximum cached entries |
PII_CACHE_TTL | 300 | TTL in seconds (5 minutes) |
PII_MAX_CONCURRENT | 8 | Max parallel detection threads |
Tuning the Cache
High-volume deployments (100+ users):
bash
PII_CACHE_SIZE=10000
PII_CACHE_TTL=600
PII_MAX_CONCURRENT=16Memory-constrained deployments:
bash
PII_CACHE_SIZE=500
PII_CACHE_TTL=120
PII_MAX_CONCURRENT=4Monitoring Cache Performance
Prometheus metrics:
ghostai_pii_cache_hits_total-- Cache hits (higher is better)ghostai_pii_cache_misses_total-- Cache misses- Hit rate = hits / (hits + misses)
A healthy cache hit rate is 30-50% for typical chat workloads. If the hit rate is below 10%, the cache may be too small or the TTL too short.
The /api/pii/cache endpoint returns live cache statistics.
Concurrent Detection
The PII_MAX_CONCURRENT setting limits how many detection requests run in parallel. This prevents CPU/memory spikes under load.
| Setting | Effect |
|---|---|
| 4 | Conservative -- good for 1-2 CPU cores |
| 8 | Default -- good for 4+ CPU cores |
| 16 | Aggressive -- for high-CPU machines with 8+ cores |
If detection requests queue up (visible as increased latency under load), increase this value. If the system runs out of memory under load, decrease it.
AI Verification Performance
AI Verification adds an LLM call to the detection pipeline. The cost depends on the mode:
Suspicious Mode
- Only entities with confidence below 0.7 or from FP-prone types are verified
- Typically 0-5 entities per message need verification
- Adds ~50-200 ms total
All Mode
- Every detected entity is sent to the LLM
- Can be 5-20+ entities per message
- Adds ~100-500 ms total
Optimization Tips
- Use
suspiciousmode unless you needall-- it covers the most common false positives with much less overhead - Use a fast model --
gpt-4o-minior similar small/fast models work well for verification - Set a dedicated model -- Configure
PII_AI_DEDICATED_MODELto use a separate model for detection/verification, keeping the chat model free for responses - Custom endpoint -- If you have a fine-tuned model on a local inference server, set a custom endpoint via Settings → AI Detector tab. Local endpoints can dramatically reduce latency compared to cloud API calls
Latency Budgets
GhostAI supports configurable latency budgets that trigger intelligent fallback:
bash
PII_LATENCY_BUDGET_MS=500When the detection pipeline approaches the budget:
- Low-priority detectors are skipped
- AI verification is downgraded from
alltosuspicious - AI enhancement is disabled
- Presidio + NER results are returned as-is
This ensures chat responsiveness even under heavy detection load.
Prometheus Metrics
Key metrics for monitoring detection performance:
| Metric | Type | Description |
|---|---|---|
ghostai_pii_detection_duration_seconds | Histogram | End-to-end detection time |
ghostai_pii_cache_hits_total | Counter | Cache hit count |
ghostai_pii_cache_misses_total | Counter | Cache miss count |
ghostai_ai_audit_call_latency_seconds | Histogram | AI detection/verification LLM latency |
ghostai_ai_audit_calls_total | Counter | Total AI detection/verification calls |
View these in the Grafana dashboard or query directly via the /metrics endpoint.
RAG File Intelligence Performance
GPU-Accelerated Embedding
The embedding device significantly affects indexing speed for large files:
| Device | 40,000 Chunks (100 MB CSV) | 100,000 Chunks |
|---|---|---|
| CPU (4 cores) | ~20 sec | ~50 sec |
| CUDA (RTX 4090) | ~4 sec | ~10 sec |
| MPS (M3 Pro) | ~7 sec | ~17 sec |
Configure via Admin Dashboard → RAG / File Intelligence → Embedding Device or rag_embedding_device in config. The auto setting probes CUDA > MPS > CPU at startup.
HNSW for Large Files
Files with >100,000 chunks automatically use HNSW approximate nearest neighbor search instead of brute-force cosine similarity:
| Chunks | Brute-Force | HNSW | Speedup |
|---|---|---|---|
| 100,000 | ~50 ms | ~1.2 ms | 42x |
| 500,000 | ~250 ms | ~1.8 ms | 139x |
HNSW indices are built lazily on first query and cached in memory. Disable via rag_ann_enabled = false if memory is constrained.
Streaming Ingestion
Files over 50 MB (configurable via rag_streaming_threshold) are streamed into the index without loading the full text into memory:
| Mode | Peak Memory (100 MB CSV) |
|---|---|
| Standard | ~500 MB |
| Streaming | ~50 MB |
Streaming applies to CSV, TSV, TXT, LOG, JSONL, NDJSON, and MD files. Binary formats always use the standard path.
Chunk-Level PII Caching
The PII cache eliminates redundant detection when the same chunks are retrieved across multiple queries:
| Queries | Without Cache | With Cache |
|---|---|---|
| 10 queries, same file | ~450 ms total | ~63 ms total (86% reduction) |
Monitor effectiveness via ghostai_rag_pii_cache_hits_total and ghostai_rag_pii_cache_misses_total.
Related Documentation
- PII Detection -- Full detection feature documentation
- File Intelligence -- RAG indexing and retrieval details
- AI Verification -- False positive filtering configuration
- Common Issues -- General troubleshooting