Appearance
PII Per-Message Override: Keeping Users in the Product When Detection Gets It Wrong
Executive Summary
GhostAI’s multi-layer PII detection protects 64+ entity types before data reaches AI models. When the system flags something that is not actually sensitive—a product name, a public figure, or a benign phrase—users have historically had no way to “fix” that single message. The result: they leave the product and paste the same text into raw LLM interfaces, bypassing all protection. PII Per-Message Override lets users correct for false positives in two controlled ways—exclude from redaction for this message only or add to Safe Terms and send—so they stay inside the protected workflow while retaining full auditability and org-level policy control.
The Problem: False Positives Drive Users Away
What Happens Today Without Override
- User types a message that triggers a detection (e.g., “Discuss the 529 plan with John”).
- The system highlights “529” as
STREET_ADDRESSand “John” asPERSON. - User knows these are false positives but has only two choices:
- Send anyway — The message is tokenized; the AI sees
[STREET_ADDRESS_1]and[PERSON_1], and the answer may be useless. - Leave — Copy the text into ChatGPT, Claude, or another interface with no PII protection.
- Send anyway — The message is tokenized; the AI sees
In both cases the organization loses: either poor AI outcomes or total loss of visibility and control.
Why “Just Add a Global Safe Term” Isn’t Enough
Safe Terms (allowlists) are the right long-term fix for known false positives like product names or internal codes. But they require a trip to Settings or Admin → Company Profile. For a one-off (“this one message is about a public report title”) or for users who don’t yet know about Safe Terms, friction is high. Per-message override provides an immediate path that still feeds the same governance model: one action is “add to Safe Terms and send,” so the next time that term appears it is no longer flagged.
Solution Overview
PII Per-Message Override adds two user actions that apply only when the organization allows them (org policy pii_allow_per_message_override):
| Action | Effect | Scope |
|---|---|---|
| Don’t redact for this message | The selected span(s) are excluded from tokenization for this request only. The AI receives the original text. | Single message |
| Add to safe terms and send | The selected text is added to the user’s Safe Terms (or org Safe Terms, depending on configuration). The client also sends an exclude span for that text so this message is sent with it unredacted; future messages benefit from the persisted Safe Term. | This message + all future |
Both actions are offered in the Explainability popup when the user clicks a flagged item in the PII preview (the bar above the input that appears before send). They are not available for high-risk entity types (SSN, credit card, bank account, passport, driver’s license, medical record number, etc.), which can never be excluded from redaction.
How It Works (User Flow)
User composes a message.
As they type, GhostAI runs detection and shows a PII preview: “N sensitive item(s) in message will be protected.”User expands the preview and sees one badge per entity. Each badge shows the entity type, the detected text (truncated if long), and the token (e.g., PERSON | Sarah Johnson → [PERSON_abc123]). For attachments, the same pattern appears per file.
User hovers or focuses a badge (keyboard/tab or tap) to locate it in context: the corresponding span is highlighted in the prompt (message PII), or the associated file chip is highlighted and scrolled into view (attachment PII).
User clicks a badge (e.g., a false-positive product name).
The Explainability popup opens with:- Why it was flagged (detector, confidence, pattern)
- Don’t redact for this message — “Send this span as-is for this message only.”
- Add to safe terms and send — “Add this text to Safe Terms and send; future messages won’t flag it.”
User chooses an action.
- If “Don’t redact”: the client adds that span to a list (
exclude_entity_spans) for the next send. The UI may show “1 item won’t be redacted for this message.” - If “Add to safe terms”: the client adds the text to
add_to_safe_terms_and_sendfor the next send; the backend persists it to Safe Terms and sends the message with that term unredacted.
- If “Don’t redact”: the client adds that span to a list (
User sends the message.
The request includesexclude_entity_spansand/oradd_to_safe_terms_and_send. The server:- Checks org policy. If override is disabled, it ignores the override fields and returns
pii_override_warningso the client can show: “Your organization does not allow per-message overrides. Add terms in Settings.” - If override is enabled, it filters the detected-entity list (excluding only non–high-risk spans that match the request), tokenizes the rest, optionally adds terms to Safe Terms, and logs an audit event.
- Checks org policy. If override is disabled, it ignores the override fields and returns
Attachments.
Override applies to attachment content as well. Spans inexclude_entity_spanscan carrysource: "attachment:filename". The server applies the same filtering when preparing attachment text for the LLM, so a user can exclude a false positive inside an attached document for that message only.
Technical Architecture
Span-Based Exclusion
Override is expressed as spans, not raw strings. Each exclusion is:
start,end— character offsets in the message (or attachment content)entity_type— must match the detected entity type (e.g.PERSON,DATE_TIME)source— optional;"message"for the main input text,"attachment:filename"for a specific file
The server filters its own detection result: it does not trust the client to decide what is PII. The client only says “do not redact the span at (start, end) of type X.” The server:
- Runs detection as usual.
- Removes from the entity list any non–high-risk entity that overlaps a requested exclude span and matches type (and source, if provided).
- Tokenizes the message using the filtered entity list.
So the server never sends unmasked data unless it has explicitly decided that the excluded span is allowed for this request (and only for non–high-risk types).
High-Risk Entity Types (Never Excludable)
The following types cannot be excluded via per-message override; they are always tokenized:
US_SSNCREDIT_CARDUS_BANK_NUMBERUS_ITINMEDICAL_RECORD_NUMBERHEALTH_PLAN_IDIBAN_CODEUS_PASSPORTUS_DRIVER_LICENSE
This list is fixed in the chat router. Override applies only to other types (e.g. PERSON, EMAIL_ADDRESS, DATE_TIME, STREET_ADDRESS, ORGANIZATION, LOCATION).
Add to Safe Terms and Send
When the request includes add_to_safe_terms_and_send (array of strings):
- The server normalizes and deduplicates (e.g. by lowercased value).
- For each term, it calls the existing user-term (or org-term) API to add a “safe” term.
- The client also sends a matching entry in
exclude_entity_spansfor the same text so that this message is sent with that span unredacted. The server applies span exclusion as described above; the persisted Safe Term ensures future messages do not flag that text.
So “Add to safe terms and send” both persists the term for the future and ensures the current message is sent without redacting that text.
Organization Policy
pii_allow_per_message_override(default:true) is stored inorganization_policies.- When
false, the server:- Ignores
exclude_entity_spansandadd_to_safe_terms_and_send. - Returns
pii_override_warningin the chat response (non-streaming) or in the first SSEstartevent (streaming), so the client can show a clear message and hide or disable the override actions.
- Ignores
Admins control this in Admin Dashboard → Policies → Allow PII overrides per message.
Audit
Every time the server applies a per-message exclusion (i.e. removes one or more entities because of exclude_entity_spans), it logs an audit event:
- Action:
data.pii_per_message_override - Details: session id, list of excluded entity types (e.g.
["PERSON", "DATE_TIME"])
So compliance and security teams can see when users chose “don’t redact” and for which entity types, without storing the actual text in the audit log.
Attachments
For file attachments, detection runs per file and entities are tagged with source: "attachment:filename". When building the LLM payload:
- The server considers
exclude_entity_spanswhosesourcematches the current attachment (or is unset). - It runs the same filter (remove matching non–high-risk entities) on that attachment’s entity list before tokenizing the attachment text.
So override applies consistently to both the main message and attached files.
Compare Mode
In Compare mode (model comparison), the same PII preview behavior is available: users see the expandable bar, original text per entity, and hover/focus to highlight the span in the prompt. Per-message override (Don’t redact for this message / Add to safe terms and send) is not offered in Compare mode; the preview is informational only so users know what will be protected before sending to all models.
Security and Compliance
Principle: Server Authoritative
The client does not decide what is PII. It only submits span coordinates and types. The server:
- Runs detection itself.
- Applies override only by removing entities from its own result.
- Never excludes high-risk types.
So there is no “client says don’t redact SSN” path.
Principle: Scoped Override
- Don’t redact for this message does not persist. The next message is protected normally. So the blast radius of a mistake is one message.
- Add to safe terms and send persists a term; that is the same Safe Terms mechanism already used for allowlists, with the same governance (user vs org terms, admin review, etc.).
Principle: Visibility
- Override usage is audited (
data.pii_per_message_override). - Org policy can disable override entirely, and the client shows a warning when the user tries to use it.
Compliance Alignment
- HIPAA / PHI: High-risk types (e.g. MRN, health plan ID) are never excludable. Override applies to lower-risk contextual entities (e.g. names, dates) only when the org allows it, with audit.
- PCI-DSS: Credit card and bank account types are never excludable.
- GDPR / CCPA: Override is opt-in per message and auditable; orgs can turn it off.
API Contract
Request (POST /api/chat and POST /api/chat/stream)
Optional fields on the same JSON body as the message:
| Field | Type | Description |
|---|---|---|
exclude_entity_spans | array | Spans to exclude from redaction for this request only. Each element: { "start": number, "end": number, "entity_type": string, "source"?: string }. source is "message" or "attachment:filename". |
add_to_safe_terms_and_send | string[] | Terms to add to the user’s Safe Terms and to send unredacted in this message. Deduplicated/normalized by the server. |
If the organization has disabled per-message override, these fields are ignored.
Response
- Non-streaming (POST /api/chat): The JSON response may include
pii_override_warning(string) when the client sent override fields but org policy disallows them. The client should display this and not apply overrides. - Streaming (POST /api/chat/stream): The first SSE event (
type: "start") may includepii_override_warning(string) with the same meaning.
High-Risk Types (Reference)
Entity types that cannot appear in a successful exclusion (server will never remove them from the entity list):
US_SSN, CREDIT_CARD, US_BANK_NUMBER, US_ITIN, MEDICAL_RECORD_NUMBER, HEALTH_PLAN_ID, IBAN_CODE, US_PASSPORT, US_DRIVER_LICENSE.
Admin and Enterprise Controls
- Policies UI: Admin Dashboard → Policies → Allow PII overrides per message (checkbox). When unchecked, override actions are not applied and the client receives
pii_override_warning. - Audit: Override usage is logged as
data.pii_per_message_overridewith session id and excluded entity types. - Safe Terms: “Add to safe terms and send” uses the existing Safe Terms (user/org) pipeline, so existing allowlist and company-profile workflows apply.
Future phases can add override analytics, suggested safe terms from override patterns, and threshold recommendations—all feeding the same governance model.
Conclusion
PII Per-Message Override gives users a controlled way to correct false positives without leaving the product. By making override span-based and server-authoritative, never allowing high-risk types to be excluded, and tying both actions to org policy and audit, GhostAI keeps users inside the protected workflow while preserving security and compliance. “Don’t redact for this message” limits scope to a single send; “Add to safe terms and send” turns a one-time correction into a persistent allowlist entry. Together they reduce the incentive to paste sensitive-looking text into unprotected LLM interfaces and keep protection and visibility where they belong.
Related Documentation
- PII Detection — Detection layers, entity types, and redaction
- False Positives — Reducing false positives (AI verification, thresholds, and per-message override)
- Chat API — Request/response fields for
exclude_entity_spans,add_to_safe_terms_and_send, andpii_override_warning - Company Intelligence Profile — Safe Terms and sensitive terms at the organization level