Skip to content

Performance ​

This guide covers optimizing PII detection speed, managing memory usage, and tuning the detection pipeline for your workload.

Detection Latency ​

Typical Latency by Configuration ​

ConfigurationTypical LatencyNotes
Presidio only5-15 msPattern matching, fastest option
Presidio + NER15-40 msAdds Dymium NER inference
Presidio + NER + AI Detector200-800 msAdds LLM round-trip
Presidio + NER + AI Verify (suspicious)50-250 msVerifies only low-confidence entities
Presidio + NER + AI Verify (all)100-500 msVerifies all entities
Full pipeline (all detectors)300-1000 msMaximum accuracy, highest latency

Where Time Is Spent ​

PhaseTypical TimeOptimization
Presidio pattern matching5-15 msCache helps with repeated text
Dymium NER inference10-25 msGPU acceleration if available
Code & Secrets detection3-10 msRegex-based, very fast
AI Enhancement (LLM call)200-500 msUse dedicated model (gpt-4o-mini)
AI Verification (LLM call)50-300 mssuspicious mode is faster than all
Anti-exfiltration decode2-5 msMinimal overhead

Execution Modes ​

The execution mode controls how detectors run relative to each other:

Parallel Mode (Fastest) ​

All enabled detectors run simultaneously. Total latency is the maximum of any single detector.

Presidio ──────────┐
NER ───────────────┤
Code & Secrets ────┤── Merge results
AI Detector ───────┘

Best for: Real-time chat, high-volume APIs, latency-sensitive workloads.

Series Mode (Most Accurate) ​

Detectors run sequentially. Each detector can use results from previous detectors for context.

Presidio → NER → Code & Secrets → AI Detector → Merge

Best for: Maximum accuracy, offline batch processing, compliance audits.

Grouped Mode (Balanced) ​

Similar detectors run in parallel groups, then groups run in sequence.

Group 1 (Pattern): Presidio + Code & Secrets + Anti-Exfiltration (parallel)
    → Group 2 (ML): NER (parallel)
        → Group 3 (AI): AI Detector

Best for: General-purpose use (default recommendation).

To change the execution mode: Settings → Detection → Execution Mode.

Custom Pipeline Configuration ​

For fine-grained control over detector ordering and grouping, open Settings → Pipeline. You can:

  • Set detector priority — higher values run first in series mode
  • Set detector groups — detectors with the same group number run in parallel in grouped mode
  • Enable/disable individual detectors — toggle each detector on/off

Custom priority and group values are persisted to your user profile and override the defaults. The execution mode (parallel, series, grouped) still determines the overall pattern, but your custom ordering is respected within that pattern.

Language-Specific Performance ​

When multiple languages are enabled (via Settings → Languages), Presidio runs detection for each selected language. NER (Dymium NER) only supports English — for non-English text, only Presidio pattern matching is available.

Each additional language adds minimal overhead to Presidio but does not affect NER or AI detector latency.

Memory Management ​

Model Memory Requirements ​

ModelMemoryWhen Loaded
Dymium NER~800 MBAlways (primary NER model)
Clinical NER~500 MBWhen Healthcare vertical is active
CLIP (images)~400 MBWhen image intelligence is used
Presidio engine~200 MBAlways

Total memory with all models: ~2 GB (base) to ~4 GB (with all vertical models)

Recommendations ​

DeploymentRAMModels
Minimal (Presidio + NER)2 GBDymium NER only
Standard (+ AI detector)4 GBDymium NER + LLM client
Healthcare vertical4-6 GBDymium NER + Clinical NER
Full (all models + images)6-8 GBAll models loaded

Reducing Memory Usage ​

  1. Disable unused detectors -- If you don't need the AI detector, turn it off in Settings
  2. Skip vertical models -- Use the general vertical if you don't need industry-specific detection
  3. Lazy loading -- Models load on first use; unused models consume no memory
  4. Set container limits -- Configure Docker memory limits to prevent OOM kills

Detection Cache ​

The detection cache avoids redundant processing when the same text is scanned multiple times (e.g., message retries, page reloads).

Cache Configuration ​

VariableDefaultDescription
PII_CACHE_SIZE2000Maximum cached entries
PII_CACHE_TTL300TTL in seconds (5 minutes)
PII_MAX_CONCURRENT8Max parallel detection threads

Tuning the Cache ​

High-volume deployments (100+ users):

bash
PII_CACHE_SIZE=10000
PII_CACHE_TTL=600
PII_MAX_CONCURRENT=16

Memory-constrained deployments:

bash
PII_CACHE_SIZE=500
PII_CACHE_TTL=120
PII_MAX_CONCURRENT=4

Monitoring Cache Performance ​

Prometheus metrics:

  • ghostai_pii_cache_hits_total -- Cache hits (higher is better)
  • ghostai_pii_cache_misses_total -- Cache misses
  • Hit rate = hits / (hits + misses)

A healthy cache hit rate is 30-50% for typical chat workloads. If the hit rate is below 10%, the cache may be too small or the TTL too short.

The /api/pii/cache endpoint returns live cache statistics.

Concurrent Detection ​

The PII_MAX_CONCURRENT setting limits how many detection requests run in parallel. This prevents CPU/memory spikes under load.

SettingEffect
4Conservative -- good for 1-2 CPU cores
8Default -- good for 4+ CPU cores
16Aggressive -- for high-CPU machines with 8+ cores

If detection requests queue up (visible as increased latency under load), increase this value. If the system runs out of memory under load, decrease it.

AI Verification Performance ​

AI Verification adds an LLM call to the detection pipeline. The cost depends on the mode:

Suspicious Mode ​

  • Only entities with confidence below 0.7 or from FP-prone types are verified
  • Typically 0-5 entities per message need verification
  • Adds ~50-200 ms total

All Mode ​

  • Every detected entity is sent to the LLM
  • Can be 5-20+ entities per message
  • Adds ~100-500 ms total

Optimization Tips ​

  1. Use suspicious mode unless you need all -- it covers the most common false positives with much less overhead
  2. Use a fast model -- gpt-4o-mini or similar small/fast models work well for verification
  3. Set a dedicated model -- Configure PII_AI_DEDICATED_MODEL to use a separate model for detection/verification, keeping the chat model free for responses
  4. Custom endpoint -- If you have a fine-tuned model on a local inference server, set a custom endpoint via Settings → AI Detector tab. Local endpoints can dramatically reduce latency compared to cloud API calls

Latency Budgets ​

GhostAI supports configurable latency budgets that trigger intelligent fallback:

bash
PII_LATENCY_BUDGET_MS=500

When the detection pipeline approaches the budget:

  1. Low-priority detectors are skipped
  2. AI verification is downgraded from all to suspicious
  3. AI enhancement is disabled
  4. Presidio + NER results are returned as-is

This ensures chat responsiveness even under heavy detection load.

Prometheus Metrics ​

Key metrics for monitoring detection performance:

MetricTypeDescription
ghostai_pii_detection_duration_secondsHistogramEnd-to-end detection time
ghostai_pii_cache_hits_totalCounterCache hit count
ghostai_pii_cache_misses_totalCounterCache miss count
ghostai_ai_audit_call_latency_secondsHistogramAI detection/verification LLM latency
ghostai_ai_audit_calls_totalCounterTotal AI detection/verification calls

View these in the Grafana dashboard or query directly via the /metrics endpoint.

RAG File Intelligence Performance ​

GPU-Accelerated Embedding ​

The embedding device significantly affects indexing speed for large files:

Device40,000 Chunks (100 MB CSV)100,000 Chunks
CPU (4 cores)~20 sec~50 sec
CUDA (RTX 4090)~4 sec~10 sec
MPS (M3 Pro)~7 sec~17 sec

Configure via Admin Dashboard → RAG / File Intelligence → Embedding Device or rag_embedding_device in config. The auto setting probes CUDA > MPS > CPU at startup.

HNSW for Large Files ​

Files with >100,000 chunks automatically use HNSW approximate nearest neighbor search instead of brute-force cosine similarity:

ChunksBrute-ForceHNSWSpeedup
100,000~50 ms~1.2 ms42x
500,000~250 ms~1.8 ms139x

HNSW indices are built lazily on first query and cached in memory. Disable via rag_ann_enabled = false if memory is constrained.

Streaming Ingestion ​

Files over 50 MB (configurable via rag_streaming_threshold) are streamed into the index without loading the full text into memory:

ModePeak Memory (100 MB CSV)
Standard~500 MB
Streaming~50 MB

Streaming applies to CSV, TSV, TXT, LOG, JSONL, NDJSON, and MD files. Binary formats always use the standard path.

Chunk-Level PII Caching ​

The PII cache eliminates redundant detection when the same chunks are retrieved across multiple queries:

QueriesWithout CacheWith Cache
10 queries, same file~450 ms total~63 ms total (86% reduction)

Monitor effectiveness via ghostai_rag_pii_cache_hits_total and ghostai_rag_pii_cache_misses_total.

SOC 2 Type II Certified | BAA Available | HIPAA-Ready