Appearance
False Positives
A false positive occurs when GhostAI flags text as PII when it is not actually sensitive data. This guide covers common false positive patterns, how to diagnose them, and multiple strategies to reduce them.
Reduce False Positives with Intelligence Profiles
If you're experiencing persistent false positives, consider setting up your organization's intelligence profile:
Auto-Research (quickest start): Click Auto-Research in Admin Dashboard → Company Profile to automatically discover your company's public products, terms, and industry context. Ghost AI will suggest both sensitive terms to protect and safe terms to allowlist — review and apply in one step. See Auto-Research for details.
Safe Terms Allowlist: Add terms that are frequently false-positived to your organization's safe terms list. Go to Admin Dashboard → Company Profile → Terms Registry and add them as "Safe" terms.
Company Context: Describe your organization so the AI detection system understands your business context. For example, if your company name is "Atlas Technologies" and "Atlas" keeps getting flagged, adding it as a safe term will suppress those detections. See Company Intelligence Profile for setup details.
Personal Safe Terms: Individual users can add their own safe terms in Settings → My Profile → My Terms. See Personal Intelligence Profile for details.
Common False Positive Patterns
DATE_TIME on Age References
Phrases describing age are sometimes flagged as dates:
| Input | Incorrectly Flagged As | Why |
|---|---|---|
| "20 year old" | DATE_TIME | The number + time word triggers date patterns |
| "in the last 3 months" | DATE_TIME | Time duration resembles a date reference |
| "5 years of experience" | DATE_TIME | Number + "years" pattern match |
Fix: Enable AI Verification in suspicious mode. The LLM understands that "20 year old" describes age, not a date.
STREET_ADDRESS on Standalone Numbers
Numbers that happen to match address patterns:
| Input | Incorrectly Flagged As | Why |
|---|---|---|
| "529 plan" | STREET_ADDRESS | "529" looks like a street number |
| "401(k) account" | STREET_ADDRESS | "401" matches address patterns |
| "Section 8 housing" | STREET_ADDRESS | Short number + context word |
Fix: Increase the STREET_ADDRESS threshold to 0.7 or higher, or enable AI Verification.
PERSON on Common Words
Words that coincidentally match name patterns:
| Input | Incorrectly Flagged As | Why |
|---|---|---|
| "she have to do" | PROVIDER_NAME | Word sequence matches name-like pattern |
| "Will do that" | PERSON | "Will" is a common first name |
| "May I ask" | PERSON | "May" is a common first name |
Fix: AI Verification in suspicious mode handles these reliably. The LLM understands the grammatical context.
Quick fix before send: If a single message has a false positive, expand the PII preview list (it shows the original detected text for each item so you know what you're overriding), then click the flagged item. Hover or focus an item to highlight it in the prompt or the associated file. Choose Don't redact for this message (send once without redacting) or Add to safe terms and send (add to your Safe Terms and send). See PII Detection — Real-Time Detection.
PERSON on English Contractions
English contractions at the start of a sentence can be flagged as PERSON because the NER model sees a capitalised word fragment:
| Input | Incorrectly Flagged As | Score | Why |
|---|---|---|---|
| "What's the best route?" | PERSON | 0.53 | "What's" parsed as capitalised word |
| "Who's coming to lunch?" | PERSON | 0.48 | "Who's" resembles a name |
| "Let's review the report" | PERSON | 0.51 | "Let's" matches name-like pattern |
| "Where's the nearest store?" | PERSON | 0.45 | "Where's" parsed as capitalised word |
Fix: This is now handled automatically by three layers of protection:
- Contraction regex filter -- A pattern matching
word'(s|t|re|ll|ve|d|m)rejects all English contractions at the NER level. - Common word allowlist -- Question words (what, who, where, when, how, which) and pronouns (it, he, she, they, etc.) are in the NER false-positive set.
- Sentence-initial guard -- Words at position 0 that are common English words (only capitalised because they start a sentence) are rejected as PERSON candidates.
No manual configuration needed.
ORGANIZATION on Product Names and Common Nouns
Product names, common nouns, and famous last names can be flagged as ORGANIZATION by the NER model:
| Input | Incorrectly Flagged As | Score | Why |
|---|---|---|---|
| "use Excel for this" | ORGANIZATION | 0.58 | "Excel" is associated with Microsoft |
| "search on Google" | ORGANIZATION | 0.65 | Brand name pattern |
| "Einstein was brilliant" | ORGANIZATION | 0.59 | Famous surname without first name context |
| "study physics today" | ORGANIZATION | 0.58 | Academic discipline misclassified |
Fix: The global threshold post-filter (added in PR #82) now catches these. The NER model runs at a lower internal threshold (NER_MAX_THRESHOLD = 0.45) to catch names, but all results must still meet the global PII_SCORE_THRESHOLD (default 0.7) after the high-risk entity boost. PERSON entities like "Sarah Johnson" receive a ~15% boost (0.61 → 0.70) and pass, while low-confidence ORGANIZATION entities like "Einstein" (0.59) or "physics" (0.58) are filtered out. No manual threshold adjustment needed.
Diagnostic Steps
1. Check the Detection Method
Click on any highlighted PII in the chat to see which detection layer flagged it:
- Presidio -- Pattern-based match. Adjust the per-entity threshold.
- NER Model -- Context-based match from Dymium NER (
dymium/Dymium-NER-v1). The NER model uses a lower internal threshold (0.45) to catch names, but results are post-filtered by the global threshold (0.7). If a low-confidence NER entity slips through, enable AI Verification. - Code & Secrets -- Language-specific pattern match. Review Code Protection settings.
- AI Detector -- LLM flagged it. This is unusual for false positives.
2. Check the Confidence Score
Low-confidence detections (below 0.7) are the most common source of false positives. You can:
- Raise the global threshold to 0.7 for fewer but higher-quality detections
- Raise per-entity thresholds for specific types that produce false positives
- Enable AI Verification in
suspiciousmode to automatically filter low-confidence hits
3. Check Your Vertical
Vertical presets adjust thresholds and enable additional detectors. If you're in the wrong vertical, you may see false positives from detectors that don't apply to your domain:
- A technology company in the
healthcarevertical will see false PHI detections - A healthcare org in the
generalvertical may miss PHI but get fewer false positives
Go to Settings → Detection Settings → Industry Vertical to verify.
Mitigation Strategies
Strategy 0: Per-Message Override (Immediate)
When a single message is blocked by a false positive and you need to send right away:
- In the PII preview above the input, expand the list (you'll see the original detected text for each item so you know what you're overriding). Hover or focus an item to highlight it in the prompt or the associated file. Click the flagged item
- In the explanation popup, choose:
- Don't redact for this message — Send this message with that span unredacted (one-time; high-risk types like SSN/credit card cannot be excluded)
- Add to safe terms and send — Add the text to your Safe Terms and send; future messages won't flag it
This is available when your organization allows it (Admin Dashboard → Policies → Allow PII overrides per message). If disabled, add the term in Settings → My Terms instead.
Strategy 1: AI Verification (Recommended)
The fastest path to reducing false positives without losing detection coverage.
- Go to Settings → Detection Settings → AI Verify Mode
- Select
suspicious(or set per-conversation via the "AI verify" dropdown in the chat header) - False positives from common patterns (dates, addresses, names) are automatically filtered
AI Verification adds ~50-200ms of latency per message in suspicious mode. See AI Verification for details.
Strategy 2: Per-Entity Threshold Tuning
For targeted control over specific entity types:
- Go to Settings → Thresholds
- Open the Confidence Thresholds tab
- Increase the threshold for the entity type producing false positives:
DATE_TIME: Raise from 0.5 to 0.7STREET_ADDRESS: Raise from 0.5 to 0.7PERSON: Raise from 0.5 to 0.7
- Click Save Settings — thresholds are persisted to your user profile
- Monitor the Quality Dashboard to ensure you're not missing real PII
Per-entity thresholds work as a post-filter: Presidio runs with the global minimum threshold, then results are filtered per-entity type. This gives you fine-grained control without reducing detection sensitivity for other entity types.
Strategy 3: Feedback Loop
Train the system to improve over time:
- Click on a false positive in the chat
- Select "Not PII" and optionally provide a reason
- After 5+ feedback entries, the system generates threshold adjustment recommendations
- Apply recommendations in Settings → Detection Accuracy
Strategy 4: Disable Specific Detectors
If an entire detection layer is producing false positives for your use case:
- Go to Settings → Pipeline
- Uncheck the problematic detector (e.g., disable the NER model if it's producing name false positives)
- Other detectors (Presidio patterns, AI detector) continue to operate
- You can also adjust detector priority and group to change execution order
WARNING
Disabling detectors reduces overall detection coverage. Use this as a last resort after trying AI Verification and threshold tuning.
Monitoring False Positive Rates
Quality Dashboard
The Quality Dashboard tracks PII Accuracy as a weighted metric (20% of overall quality score). It uses feedback data to calculate the real false positive rate per entity type.
AI Audit Logs
When AI Verification is enabled, the AI Audit Logs show rejected entities -- entities that the LLM determined were false positives. Filter by verification events to see:
- entities_in: Total entities sent for review
- entities_out: Entities confirmed as real PII
- rejected_count: False positives filtered out
A high rejected count indicates that upstream detectors (Presidio, NER) are generating many false positives, and AI Verification is catching them. Consider also adjusting thresholds to reduce the upstream noise.
Security Finding False Positives
AI threat analysis helps reduce security finding false positives by analyzing whether detected patterns (e.g., macros, embedded scripts, phishing URLs) are actually dangerous. The AI classifies findings as malicious, suspicious, or benign based on context and intent.
- User feedback: Thumbs up/down on AI verdicts helps improve accuracy over time.
- Admin policy locks: Admins can lock findings as malicious or benign to prevent AI from overriding organizational policy.
See Threat Analysis for configuration and Threat Analysis Policy for policy controls.
Related Documentation
- PII Detection -- Full detection feature documentation
- AI Audit Logs -- Monitor verification decisions
- Dashboards -- Quality score and accuracy tracking
- Public Knowledge Reconciliation -- Reduce over-redaction of public locations, organizations, and events
Over-Redaction of Public Knowledge
If harmless entities like states, public organizations, or famous events are being masked:
- Enable Public Knowledge Reconciliation in Settings
- Start with
builtintier only — organization names are reconciled via a curated list of well-known organizations (and optional Wikidata/LLM), not a registry of public companies; adding a company to the built-in list is a product/curation decision. - Add entity types (
LOCATION,ORGANIZATION,EVENT) as needed - Keep safety filters active and validate via audit logs