Appearance
Preventing Data Leakage in AI-Assisted Knowledge Reconciliation
Executive Summary
Security-first AI systems often over-redact public knowledge, degrading response quality. Public Knowledge Reconciliation balances usability and security by allowing clearly public entities to remain unmasked while preserving strict controls around private context.
The Over-Redaction Problem
Over-redaction harms normal requests:
- "Find restaurants in Utah" -> state tokenized
- "What does NASA do?" -> organization tokenized
- "Explain World War II impacts" -> event tokenized
If users cannot complete basic tasks, they disable controls entirely, increasing risk.
Architecture Overview
The reconciliation pipeline applies to PERSON (public figures) and selected non-person entities (LOCATION, ORGANIZATION, EVENT, DATE_TIME) using three tiers:
- Built-in list — for ORGANIZATION, a curated set of well-known organizations; no registry of public companies (e.g. SEC or stock exchange) is used
- Wikidata + Wikipedia signal
- LLM fallback
Each tier is gated by safety filters and confidence thresholds.
Data Leakage Threat Model
Leakage risk exists only when external tiers are enabled:
- Tier 1 (built-in): no external transmission
- Tier 2 (Wikidata): candidate text transmitted for lookup
- Tier 3 (LLM): candidate text transmitted to model provider
Secondary leakage vectors:
- persistent storage of unresolved candidate text
- raw entity values in warning/error logs
Five-Layer Mitigation Architecture
1) Builtin-only default
Default tier configuration is ["builtin"]. External tiers require explicit opt-in.
2) Pre-filter before external calls
Candidates are blocked from external lookup when they look private:
- contains digits
- address-like structure
- URL/email markers
- excessively long candidate strings
3) Hash-first cache design
Cache lookup keys are SHA-256 hashes. For unresolved entities, raw normalized text is not persisted.
4) Reconciliation audit logging
Tier decisions can be logged at summary or full detail to support investigations and compliance evidence.
5) Log scrubbing
Runtime warnings include entity type and text length, not raw candidate text.
Safety Filter Design
Reconciliation is blocked in private context:
- location near address/financial identifiers
- organization near employment context
- dates in DOB/medical context
- multi-detector consensus requires higher authority (builtin-only override)
Compliance Alignment
- HIPAA: facility names and clinical date context remain protected
- GDPR: contextual controls prevent unnecessary disclosure
- PCI-DSS: financial identifiers are unaffected by reconciliation
- SOX/FCRA/FTC: employer and financial context remain redacted
- FERPA: educational private context remains guarded
Why Wikidata + Wikipedia
Wikidata provides structured type checks (P31) and Wikipedia sitelinks provide additional public-notability confidence. This enables deterministic, explainable tier-2 behavior.
Operational Guidance
- Start with builtin-only
- Enable Wikidata for broader coverage where policy allows
- Enable LLM fallback only with trusted model governance
- Monitor reconciliation audit events and tune thresholds
Conclusion
Public Knowledge Reconciliation is a controlled exception mechanism, not a relaxation of protection. With layered mitigations and context-aware safety filters, organizations can reduce over-redaction without increasing leakage risk.