Skip to content

Preventing Data Leakage in AI-Assisted Knowledge Reconciliation ​

Executive Summary ​

Security-first AI systems often over-redact public knowledge, degrading response quality. Public Knowledge Reconciliation balances usability and security by allowing clearly public entities to remain unmasked while preserving strict controls around private context.

The Over-Redaction Problem ​

Over-redaction harms normal requests:

  • "Find restaurants in Utah" -> state tokenized
  • "What does NASA do?" -> organization tokenized
  • "Explain World War II impacts" -> event tokenized

If users cannot complete basic tasks, they disable controls entirely, increasing risk.

Architecture Overview ​

The reconciliation pipeline applies to PERSON (public figures) and selected non-person entities (LOCATION, ORGANIZATION, EVENT, DATE_TIME) using three tiers:

  1. Built-in list — for ORGANIZATION, a curated set of well-known organizations; no registry of public companies (e.g. SEC or stock exchange) is used
  2. Wikidata + Wikipedia signal
  3. LLM fallback

Each tier is gated by safety filters and confidence thresholds.

Data Leakage Threat Model ​

Leakage risk exists only when external tiers are enabled:

  • Tier 1 (built-in): no external transmission
  • Tier 2 (Wikidata): candidate text transmitted for lookup
  • Tier 3 (LLM): candidate text transmitted to model provider

Secondary leakage vectors:

  • persistent storage of unresolved candidate text
  • raw entity values in warning/error logs

Five-Layer Mitigation Architecture ​

1) Builtin-only default ​

Default tier configuration is ["builtin"]. External tiers require explicit opt-in.

2) Pre-filter before external calls ​

Candidates are blocked from external lookup when they look private:

  • contains digits
  • address-like structure
  • URL/email markers
  • excessively long candidate strings

3) Hash-first cache design ​

Cache lookup keys are SHA-256 hashes. For unresolved entities, raw normalized text is not persisted.

4) Reconciliation audit logging ​

Tier decisions can be logged at summary or full detail to support investigations and compliance evidence.

5) Log scrubbing ​

Runtime warnings include entity type and text length, not raw candidate text.

Safety Filter Design ​

Reconciliation is blocked in private context:

  • location near address/financial identifiers
  • organization near employment context
  • dates in DOB/medical context
  • multi-detector consensus requires higher authority (builtin-only override)

Compliance Alignment ​

  • HIPAA: facility names and clinical date context remain protected
  • GDPR: contextual controls prevent unnecessary disclosure
  • PCI-DSS: financial identifiers are unaffected by reconciliation
  • SOX/FCRA/FTC: employer and financial context remain redacted
  • FERPA: educational private context remains guarded

Why Wikidata + Wikipedia ​

Wikidata provides structured type checks (P31) and Wikipedia sitelinks provide additional public-notability confidence. This enables deterministic, explainable tier-2 behavior.

Operational Guidance ​

  1. Start with builtin-only
  2. Enable Wikidata for broader coverage where policy allows
  3. Enable LLM fallback only with trusted model governance
  4. Monitor reconciliation audit events and tune thresholds

Conclusion ​

Public Knowledge Reconciliation is a controlled exception mechanism, not a relaxation of protection. With layered mitigations and context-aware safety filters, organizations can reduce over-redaction without increasing leakage risk.

SOC 2 Type II Certified | BAA Available | HIPAA-Ready