Skip to content

False Positives ​

A false positive occurs when GhostAI flags text as PII when it is not actually sensitive data. This guide covers common false positive patterns, how to diagnose them, and multiple strategies to reduce them.

Reduce False Positives with Intelligence Profiles ​

If you're experiencing persistent false positives, consider setting up your organization's intelligence profile:

  1. Auto-Research (quickest start): Click Auto-Research in Admin Dashboard → Company Profile to automatically discover your company's public products, terms, and industry context. Ghost AI will suggest both sensitive terms to protect and safe terms to allowlist — review and apply in one step. See Auto-Research for details.

  2. Safe Terms Allowlist: Add terms that are frequently false-positived to your organization's safe terms list. Go to Admin Dashboard → Company Profile → Terms Registry and add them as "Safe" terms.

  3. Company Context: Describe your organization so the AI detection system understands your business context. For example, if your company name is "Atlas Technologies" and "Atlas" keeps getting flagged, adding it as a safe term will suppress those detections. See Company Intelligence Profile for setup details.

  4. Personal Safe Terms: Individual users can add their own safe terms in Settings → My Profile → My Terms. See Personal Intelligence Profile for details.

Common False Positive Patterns ​

DATE_TIME on Age References ​

Phrases describing age are sometimes flagged as dates:

InputIncorrectly Flagged AsWhy
"20 year old"DATE_TIMEThe number + time word triggers date patterns
"in the last 3 months"DATE_TIMETime duration resembles a date reference
"5 years of experience"DATE_TIMENumber + "years" pattern match

Fix: Enable AI Verification in suspicious mode. The LLM understands that "20 year old" describes age, not a date.

STREET_ADDRESS on Standalone Numbers ​

Numbers that happen to match address patterns:

InputIncorrectly Flagged AsWhy
"529 plan"STREET_ADDRESS"529" looks like a street number
"401(k) account"STREET_ADDRESS"401" matches address patterns
"Section 8 housing"STREET_ADDRESSShort number + context word

Fix: Increase the STREET_ADDRESS threshold to 0.7 or higher, or enable AI Verification.

PERSON on Common Words ​

Words that coincidentally match name patterns:

InputIncorrectly Flagged AsWhy
"she have to do"PROVIDER_NAMEWord sequence matches name-like pattern
"Will do that"PERSON"Will" is a common first name
"May I ask"PERSON"May" is a common first name

Fix: AI Verification in suspicious mode handles these reliably. The LLM understands the grammatical context.

Quick fix before send: If a single message has a false positive, expand the PII preview list (it shows the original detected text for each item so you know what you're overriding), then click the flagged item. Hover or focus an item to highlight it in the prompt or the associated file. Choose Don't redact for this message (send once without redacting) or Add to safe terms and send (add to your Safe Terms and send). See PII Detection — Real-Time Detection.

PERSON on English Contractions ​

English contractions at the start of a sentence can be flagged as PERSON because the NER model sees a capitalised word fragment:

InputIncorrectly Flagged AsScoreWhy
"What's the best route?"PERSON0.53"What's" parsed as capitalised word
"Who's coming to lunch?"PERSON0.48"Who's" resembles a name
"Let's review the report"PERSON0.51"Let's" matches name-like pattern
"Where's the nearest store?"PERSON0.45"Where's" parsed as capitalised word

Fix: This is now handled automatically by three layers of protection:

  1. Contraction regex filter -- A pattern matching word'(s|t|re|ll|ve|d|m) rejects all English contractions at the NER level.
  2. Common word allowlist -- Question words (what, who, where, when, how, which) and pronouns (it, he, she, they, etc.) are in the NER false-positive set.
  3. Sentence-initial guard -- Words at position 0 that are common English words (only capitalised because they start a sentence) are rejected as PERSON candidates.

No manual configuration needed.

ORGANIZATION on Product Names and Common Nouns ​

Product names, common nouns, and famous last names can be flagged as ORGANIZATION by the NER model:

InputIncorrectly Flagged AsScoreWhy
"use Excel for this"ORGANIZATION0.58"Excel" is associated with Microsoft
"search on Google"ORGANIZATION0.65Brand name pattern
"Einstein was brilliant"ORGANIZATION0.59Famous surname without first name context
"study physics today"ORGANIZATION0.58Academic discipline misclassified

Fix: The global threshold post-filter (added in PR #82) now catches these. The NER model runs at a lower internal threshold (NER_MAX_THRESHOLD = 0.45) to catch names, but all results must still meet the global PII_SCORE_THRESHOLD (default 0.7) after the high-risk entity boost. PERSON entities like "Sarah Johnson" receive a ~15% boost (0.61 → 0.70) and pass, while low-confidence ORGANIZATION entities like "Einstein" (0.59) or "physics" (0.58) are filtered out. No manual threshold adjustment needed.

Diagnostic Steps ​

1. Check the Detection Method ​

Click on any highlighted PII in the chat to see which detection layer flagged it:

  • Presidio -- Pattern-based match. Adjust the per-entity threshold.
  • NER Model -- Context-based match from Dymium NER (dymium/Dymium-NER-v1). The NER model uses a lower internal threshold (0.45) to catch names, but results are post-filtered by the global threshold (0.7). If a low-confidence NER entity slips through, enable AI Verification.
  • Code & Secrets -- Language-specific pattern match. Review Code Protection settings.
  • AI Detector -- LLM flagged it. This is unusual for false positives.

2. Check the Confidence Score ​

Low-confidence detections (below 0.7) are the most common source of false positives. You can:

  • Raise the global threshold to 0.7 for fewer but higher-quality detections
  • Raise per-entity thresholds for specific types that produce false positives
  • Enable AI Verification in suspicious mode to automatically filter low-confidence hits

3. Check Your Vertical ​

Vertical presets adjust thresholds and enable additional detectors. If you're in the wrong vertical, you may see false positives from detectors that don't apply to your domain:

  • A technology company in the healthcare vertical will see false PHI detections
  • A healthcare org in the general vertical may miss PHI but get fewer false positives

Go to Settings → Detection Settings → Industry Vertical to verify.

Mitigation Strategies ​

Strategy 0: Per-Message Override (Immediate) ​

When a single message is blocked by a false positive and you need to send right away:

  1. In the PII preview above the input, expand the list (you'll see the original detected text for each item so you know what you're overriding). Hover or focus an item to highlight it in the prompt or the associated file. Click the flagged item
  2. In the explanation popup, choose:
    • Don't redact for this message — Send this message with that span unredacted (one-time; high-risk types like SSN/credit card cannot be excluded)
    • Add to safe terms and send — Add the text to your Safe Terms and send; future messages won't flag it

This is available when your organization allows it (Admin Dashboard → Policies → Allow PII overrides per message). If disabled, add the term in Settings → My Terms instead.

The fastest path to reducing false positives without losing detection coverage.

  1. Go to Settings → Detection Settings → AI Verify Mode
  2. Select suspicious (or set per-conversation via the "AI verify" dropdown in the chat header)
  3. False positives from common patterns (dates, addresses, names) are automatically filtered

AI Verification adds ~50-200ms of latency per message in suspicious mode. See AI Verification for details.

Strategy 2: Per-Entity Threshold Tuning ​

For targeted control over specific entity types:

  1. Go to Settings → Thresholds
  2. Open the Confidence Thresholds tab
  3. Increase the threshold for the entity type producing false positives:
    • DATE_TIME: Raise from 0.5 to 0.7
    • STREET_ADDRESS: Raise from 0.5 to 0.7
    • PERSON: Raise from 0.5 to 0.7
  4. Click Save Settings — thresholds are persisted to your user profile
  5. Monitor the Quality Dashboard to ensure you're not missing real PII

Per-entity thresholds work as a post-filter: Presidio runs with the global minimum threshold, then results are filtered per-entity type. This gives you fine-grained control without reducing detection sensitivity for other entity types.

Strategy 3: Feedback Loop ​

Train the system to improve over time:

  1. Click on a false positive in the chat
  2. Select "Not PII" and optionally provide a reason
  3. After 5+ feedback entries, the system generates threshold adjustment recommendations
  4. Apply recommendations in Settings → Detection Accuracy

Strategy 4: Disable Specific Detectors ​

If an entire detection layer is producing false positives for your use case:

  1. Go to Settings → Pipeline
  2. Uncheck the problematic detector (e.g., disable the NER model if it's producing name false positives)
  3. Other detectors (Presidio patterns, AI detector) continue to operate
  4. You can also adjust detector priority and group to change execution order

WARNING

Disabling detectors reduces overall detection coverage. Use this as a last resort after trying AI Verification and threshold tuning.

Monitoring False Positive Rates ​

Quality Dashboard ​

The Quality Dashboard tracks PII Accuracy as a weighted metric (20% of overall quality score). It uses feedback data to calculate the real false positive rate per entity type.

AI Audit Logs ​

When AI Verification is enabled, the AI Audit Logs show rejected entities -- entities that the LLM determined were false positives. Filter by verification events to see:

  • entities_in: Total entities sent for review
  • entities_out: Entities confirmed as real PII
  • rejected_count: False positives filtered out

A high rejected count indicates that upstream detectors (Presidio, NER) are generating many false positives, and AI Verification is catching them. Consider also adjusting thresholds to reduce the upstream noise.

Security Finding False Positives ​

AI threat analysis helps reduce security finding false positives by analyzing whether detected patterns (e.g., macros, embedded scripts, phishing URLs) are actually dangerous. The AI classifies findings as malicious, suspicious, or benign based on context and intent.

  • User feedback: Thumbs up/down on AI verdicts helps improve accuracy over time.
  • Admin policy locks: Admins can lock findings as malicious or benign to prevent AI from overriding organizational policy.

See Threat Analysis for configuration and Threat Analysis Policy for policy controls.

Over-Redaction of Public Knowledge ​

If harmless entities like states, public organizations, or famous events are being masked:

  1. Enable Public Knowledge Reconciliation in Settings
  2. Start with builtin tier only — organization names are reconciled via a curated list of well-known organizations (and optional Wikidata/LLM), not a registry of public companies; adding a company to the built-in list is a product/curation decision.
  3. Add entity types (LOCATION, ORGANIZATION, EVENT) as needed
  4. Keep safety filters active and validate via audit logs

SOC 2 Type II Certified | BAA Available | HIPAA-Ready