Skip to content

PHI Detection ​

Detailed documentation on GhostAI's Protected Health Information (PHI) detection capabilities.

PHI Detection Architecture ​

Input Text → Pre-processing → Multi-Layer Detection → PHI Classification
                                     │
              ┌──────────────────────┼──────────────────────┐
              ▼                      ▼                      ▼
        ┌──────────┐          ┌──────────┐          ┌──────────┐
        │ Presidio │          │   NER    │          │ Medical  │
        │ Patterns │          │  Model   │          │ Specific │
        └──────────┘          └──────────┘          └──────────┘
              │                      │                      │
              └──────────────────────┼──────────────────────┘
                                     ▼
                              PHI Classification
                              & HIPAA Tagging

Detection Categories ​

Patient Identifiers ​

PHI TypeDetection MethodExamples
Patient NameNER + PresidioJohn Smith, Jane Doe
MRNPattern matchingMRN12345678, #123456
Account NumberPattern + contextAcct: 987654321
Insurance IDPattern matchingBC12345678

Demographic Information ​

PHI TypeDetection MethodExamples
Date of BirthDate parser01/15/1980, Jan 15, 1980
Age (>89)Date calculation94 years old
AddressNER + pattern123 Main St, Apt 4B
ZIP CodePattern12345, 12345-6789
PhonePattern(555) 123-4567
EmailPatternpatient@email.com

Medical Identifiers ​

PHI TypeDetection MethodExamples
NPIPattern (10-digit)1234567890
DEA NumberPatternAB1234567
License NumberPattern + contextRN123456
Device SerialPattern + contextSN-ABC123456

Clinical Indicators ​

PHI TypeDetection MethodExamples
DiagnosisMedical NERType 2 diabetes, CHF
MedicationMedical NERMetformin 500mg
ProcedureMedical NERColonoscopy, CABG
Lab ValuesPattern + contextA1C 7.2%, Cr 1.2

Detection Thresholds ​

Entity TypeDefaultHealthcare Recommended
US_SSN0.50.3 (more sensitive)
PERSON0.50.4 (catch more names)
DATE_TIME0.50.4 (important for PHI)
PHONE_NUMBER0.50.5 (keep default)
EMAIL_ADDRESS0.50.5 (keep default)
LOCATION0.50.3 (catch addresses)
MEDICAL_LICENSE0.50.4 (catch MRNs, NPIs)

Adjusting Thresholds ​

Via UI:

  1. Open Settings → Thresholds
  2. Go to the Confidence Thresholds tab
  3. Adjust the slider for each entity type to the recommended healthcare values
  4. Click Save Settings — thresholds are persisted per-user

Via API:

bash
curl -X PUT http://localhost:8000/api/settings/detection \
  -H "Authorization: Bearer TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "entity_thresholds": {
      "US_SSN": 0.3,
      "PERSON": 0.4,
      "LOCATION": 0.3,
      "DATE_TIME": 0.4,
      "MEDICAL_LICENSE": 0.4
    }
  }'

Per-entity thresholds are applied as a post-filter after Presidio detection. Presidio runs with the global minimum threshold, then results are filtered against each entity type's configured threshold. This ensures maximum sensitivity for critical PHI types while reducing false positives for less sensitive types.

Medical-Specific Detection ​

Clinical Terminology ​

GhostAI uses medical NLP to detect:

  • Diagnoses: ICD-10 codes, condition names
  • Procedures: CPT codes, procedure names
  • Medications: Drug names, dosages
  • Lab tests: Test names, values, units

MRN Pattern Matching ​

Common MRN formats detected:

FormatPatternExample
Numeric\d{6,10}12345678
PrefixMRN\d{6,10}MRN12345678
Hash prefix#\d{6,10}#12345678
Dash format\d{3}-\d{3}-\d{4}123-456-7890

Insurance ID Detection ​

Patterns for major payers:

PayerPatternExample
Medicare\d{9}[A-Z]{1}123456789A
MedicaidVaries by stateMCD123456789
BCBS[A-Z]{3}\d{9}ABC123456789
UHC\d{9}123456789
AetnaW\d{8}W12345678

Clinical NER Model ​

When the healthcare vertical is active, GhostAI loads the obi/deid_roberta_i2b2 clinical NER model trained on de-identification data from i2b2 clinical notes. This model provides enhanced detection of:

  • Patient names in clinical narratives
  • Provider names and credentials
  • Dates of service and admission/discharge dates
  • Medical record numbers in free text
  • Facility names and locations

Dymium NER (dymium/Dymium-NER-v1) also detects cross-domain PII entities, including Medicare Beneficiary Identifiers (US_MBI).

DICOM Metadata Detection ​

Detected Tags ​

TagNameExample
0010,0010Patient NameSMITH^JOHN
0010,0020Patient ID12345678
0010,0030Patient DOB19800115
0010,0040Patient SexM
0008,0080InstitutionGeneral Hospital
0008,0090Referring PhysicianDr. Jane Doe

Processing Flow ​

  1. Upload: DICOM file uploaded
  2. Extract: Metadata tags extracted
  3. Detect: PHI fields identified
  4. Tokenize: PHI replaced with tokens
  5. Clean: Option to strip metadata

HL7 Message Detection ​

Supported Segments ​

SegmentPurposePHI Fields
PIDPatient IdentificationName, DOB, SSN, Address
NK1Next of KinContact name, relationship
PV1Patient VisitAttending, location
OBXObservationsValues with patient context
ORCOrdersOrdering provider

Example HL7 Processing ​

Input:

MSH|^~\&|LAB|HOSP|EMR|HOSP|202401151030||ORU^R01|123|P|2.5
PID|||12345678^^^HOSP^MR||SMITH^JOHN^A||19800115|M
OBX|1|NM|GLU^Glucose||95|mg/dL|70-100|N|||F

Detected PHI:

  • MRN: 12345678
  • Name: SMITH^JOHN^A
  • DOB: 19800115

Contextual Detection ​

Healthcare Context Awareness ​

GhostAI uses context to improve detection:

Example 1: Name in clinical context

"Patient John Smith presents with chest pain"
→ "John Smith" detected as PERSON (patient context)
→ Higher confidence due to "Patient" keyword

Example 2: Date in medical context

"Admission date: 01/15/2024"
→ Date detected with HIPAA significance
→ Compliance tag: HIPAA PHI

False Positive Reduction ​

Context helps reduce false positives:

Example: Generic date

"The policy was enacted on 01/15/2024"
→ Date detected but NOT flagged as PHI
→ No patient context present

API Examples ​

Detect PHI in Text ​

bash
curl -X POST http://localhost:8000/api/detect \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Patient John Smith (MRN: 12345678) DOB: 01/15/1980",
    "vertical": "healthcare"
  }'

Response:

json
{
  "entities": [
    {
      "type": "PERSON",
      "text": "John Smith",
      "start": 8,
      "end": 18,
      "confidence": 0.95,
      "compliance": ["HIPAA"],
      "phi_category": "patient_name"
    },
    {
      "type": "MEDICAL_ID",
      "text": "12345678",
      "start": 25,
      "end": 33,
      "confidence": 0.92,
      "compliance": ["HIPAA"],
      "phi_category": "mrn"
    },
    {
      "type": "DATE_TIME",
      "text": "01/15/1980",
      "start": 40,
      "end": 50,
      "confidence": 0.98,
      "compliance": ["HIPAA"],
      "phi_category": "dob"
    }
  ],
  "hipaa_phi_count": 3
}

Process DICOM File ​

bash
curl -X POST http://localhost:8000/api/files/upload \
  -H "Authorization: Bearer TOKEN" \
  -F "file=@scan.dcm" \
  -F "vertical=healthcare" \
  -F "strip_metadata=true"

Best Practices ​

Maximizing Detection ​

  1. Use healthcare vertical: Enables medical-specific patterns
  2. Lower thresholds: Catch more PHI, review false positives
  3. Enable all detectors: Multi-layer for best coverage
  4. Use feedback: Report missed PHI to improve

Reducing False Positives ​

  1. Review patterns: Some MRN formats may conflict with others
  2. Adjust per-entity: Fine-tune specific types
  3. Use allowlists: Exclude known safe patterns
  4. Provide feedback: Mark false positives

Compliance Documentation ​

  1. Export audit logs: Monthly compliance reports
  2. Track PHI counts: Monitor trends
  3. Document feedback: Record tuning decisions
  4. Regular review: Quarterly accuracy assessment

Troubleshooting ​

PHI Not Detected ​

  1. Check vertical is set to healthcare
  2. Verify entity type threshold
  3. Check format matches expected patterns
  4. Review if detector is enabled

Too Many False Positives ​

  1. Increase threshold for problematic entity
  2. Add pattern to allowlist
  3. Provide feedback on false positives
  4. Consider context-specific tuning

Medical Terms Detected as PHI ​

Medical terminology should NOT be detected as PHI unless it contains patient identifiers. If you're seeing:

  • Drug names flagged → Check if patient context is present
  • Diagnosis codes flagged → Verify no embedded identifiers
  • Procedure names flagged → Review for unique identifiers

SOC 2 Type II Certified | BAA Available | HIPAA-Ready