Appearance
PHI Detection
Detailed documentation on GhostAI's Protected Health Information (PHI) detection capabilities.
PHI Detection Architecture
Input Text → Pre-processing → Multi-Layer Detection → PHI Classification
│
┌──────────────────────┼──────────────────────┐
▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ Presidio │ │ NER │ │ Medical │
│ Patterns │ │ Model │ │ Specific │
└──────────┘ └──────────┘ └──────────┘
│ │ │
└──────────────────────┼──────────────────────┘
▼
PHI Classification
& HIPAA TaggingDetection Categories
Patient Identifiers
| PHI Type | Detection Method | Examples |
|---|---|---|
| Patient Name | NER + Presidio | John Smith, Jane Doe |
| MRN | Pattern matching | MRN12345678, #123456 |
| Account Number | Pattern + context | Acct: 987654321 |
| Insurance ID | Pattern matching | BC12345678 |
Demographic Information
| PHI Type | Detection Method | Examples |
|---|---|---|
| Date of Birth | Date parser | 01/15/1980, Jan 15, 1980 |
| Age (>89) | Date calculation | 94 years old |
| Address | NER + pattern | 123 Main St, Apt 4B |
| ZIP Code | Pattern | 12345, 12345-6789 |
| Phone | Pattern | (555) 123-4567 |
| Pattern | patient@email.com |
Medical Identifiers
| PHI Type | Detection Method | Examples |
|---|---|---|
| NPI | Pattern (10-digit) | 1234567890 |
| DEA Number | Pattern | AB1234567 |
| License Number | Pattern + context | RN123456 |
| Device Serial | Pattern + context | SN-ABC123456 |
Clinical Indicators
| PHI Type | Detection Method | Examples |
|---|---|---|
| Diagnosis | Medical NER | Type 2 diabetes, CHF |
| Medication | Medical NER | Metformin 500mg |
| Procedure | Medical NER | Colonoscopy, CABG |
| Lab Values | Pattern + context | A1C 7.2%, Cr 1.2 |
Detection Thresholds
Recommended Healthcare Settings
| Entity Type | Default | Healthcare Recommended |
|---|---|---|
| US_SSN | 0.5 | 0.3 (more sensitive) |
| PERSON | 0.5 | 0.4 (catch more names) |
| DATE_TIME | 0.5 | 0.4 (important for PHI) |
| PHONE_NUMBER | 0.5 | 0.5 (keep default) |
| EMAIL_ADDRESS | 0.5 | 0.5 (keep default) |
| LOCATION | 0.5 | 0.3 (catch addresses) |
| MEDICAL_LICENSE | 0.5 | 0.4 (catch MRNs, NPIs) |
Adjusting Thresholds
Via UI:
- Open Settings → Thresholds
- Go to the Confidence Thresholds tab
- Adjust the slider for each entity type to the recommended healthcare values
- Click Save Settings — thresholds are persisted per-user
Via API:
bash
curl -X PUT http://localhost:8000/api/settings/detection \
-H "Authorization: Bearer TOKEN" \
-H "Content-Type: application/json" \
-d '{
"entity_thresholds": {
"US_SSN": 0.3,
"PERSON": 0.4,
"LOCATION": 0.3,
"DATE_TIME": 0.4,
"MEDICAL_LICENSE": 0.4
}
}'Per-entity thresholds are applied as a post-filter after Presidio detection. Presidio runs with the global minimum threshold, then results are filtered against each entity type's configured threshold. This ensures maximum sensitivity for critical PHI types while reducing false positives for less sensitive types.
Medical-Specific Detection
Clinical Terminology
GhostAI uses medical NLP to detect:
- Diagnoses: ICD-10 codes, condition names
- Procedures: CPT codes, procedure names
- Medications: Drug names, dosages
- Lab tests: Test names, values, units
MRN Pattern Matching
Common MRN formats detected:
| Format | Pattern | Example |
|---|---|---|
| Numeric | \d{6,10} | 12345678 |
| Prefix | MRN\d{6,10} | MRN12345678 |
| Hash prefix | #\d{6,10} | #12345678 |
| Dash format | \d{3}-\d{3}-\d{4} | 123-456-7890 |
Insurance ID Detection
Patterns for major payers:
| Payer | Pattern | Example |
|---|---|---|
| Medicare | \d{9}[A-Z]{1} | 123456789A |
| Medicaid | Varies by state | MCD123456789 |
| BCBS | [A-Z]{3}\d{9} | ABC123456789 |
| UHC | \d{9} | 123456789 |
| Aetna | W\d{8} | W12345678 |
Clinical NER Model
When the healthcare vertical is active, GhostAI loads the obi/deid_roberta_i2b2 clinical NER model trained on de-identification data from i2b2 clinical notes. This model provides enhanced detection of:
- Patient names in clinical narratives
- Provider names and credentials
- Dates of service and admission/discharge dates
- Medical record numbers in free text
- Facility names and locations
Dymium NER (dymium/Dymium-NER-v1) also detects cross-domain PII entities, including Medicare Beneficiary Identifiers (US_MBI).
DICOM Metadata Detection
Detected Tags
| Tag | Name | Example |
|---|---|---|
| 0010,0010 | Patient Name | SMITH^JOHN |
| 0010,0020 | Patient ID | 12345678 |
| 0010,0030 | Patient DOB | 19800115 |
| 0010,0040 | Patient Sex | M |
| 0008,0080 | Institution | General Hospital |
| 0008,0090 | Referring Physician | Dr. Jane Doe |
Processing Flow
- Upload: DICOM file uploaded
- Extract: Metadata tags extracted
- Detect: PHI fields identified
- Tokenize: PHI replaced with tokens
- Clean: Option to strip metadata
HL7 Message Detection
Supported Segments
| Segment | Purpose | PHI Fields |
|---|---|---|
| PID | Patient Identification | Name, DOB, SSN, Address |
| NK1 | Next of Kin | Contact name, relationship |
| PV1 | Patient Visit | Attending, location |
| OBX | Observations | Values with patient context |
| ORC | Orders | Ordering provider |
Example HL7 Processing
Input:
MSH|^~\&|LAB|HOSP|EMR|HOSP|202401151030||ORU^R01|123|P|2.5
PID|||12345678^^^HOSP^MR||SMITH^JOHN^A||19800115|M
OBX|1|NM|GLU^Glucose||95|mg/dL|70-100|N|||FDetected PHI:
- MRN: 12345678
- Name: SMITH^JOHN^A
- DOB: 19800115
Contextual Detection
Healthcare Context Awareness
GhostAI uses context to improve detection:
Example 1: Name in clinical context
"Patient John Smith presents with chest pain"
→ "John Smith" detected as PERSON (patient context)
→ Higher confidence due to "Patient" keywordExample 2: Date in medical context
"Admission date: 01/15/2024"
→ Date detected with HIPAA significance
→ Compliance tag: HIPAA PHIFalse Positive Reduction
Context helps reduce false positives:
Example: Generic date
"The policy was enacted on 01/15/2024"
→ Date detected but NOT flagged as PHI
→ No patient context presentAPI Examples
Detect PHI in Text
bash
curl -X POST http://localhost:8000/api/detect \
-H "Content-Type: application/json" \
-d '{
"text": "Patient John Smith (MRN: 12345678) DOB: 01/15/1980",
"vertical": "healthcare"
}'Response:
json
{
"entities": [
{
"type": "PERSON",
"text": "John Smith",
"start": 8,
"end": 18,
"confidence": 0.95,
"compliance": ["HIPAA"],
"phi_category": "patient_name"
},
{
"type": "MEDICAL_ID",
"text": "12345678",
"start": 25,
"end": 33,
"confidence": 0.92,
"compliance": ["HIPAA"],
"phi_category": "mrn"
},
{
"type": "DATE_TIME",
"text": "01/15/1980",
"start": 40,
"end": 50,
"confidence": 0.98,
"compliance": ["HIPAA"],
"phi_category": "dob"
}
],
"hipaa_phi_count": 3
}Process DICOM File
bash
curl -X POST http://localhost:8000/api/files/upload \
-H "Authorization: Bearer TOKEN" \
-F "file=@scan.dcm" \
-F "vertical=healthcare" \
-F "strip_metadata=true"Best Practices
Maximizing Detection
- Use healthcare vertical: Enables medical-specific patterns
- Lower thresholds: Catch more PHI, review false positives
- Enable all detectors: Multi-layer for best coverage
- Use feedback: Report missed PHI to improve
Reducing False Positives
- Review patterns: Some MRN formats may conflict with others
- Adjust per-entity: Fine-tune specific types
- Use allowlists: Exclude known safe patterns
- Provide feedback: Mark false positives
Compliance Documentation
- Export audit logs: Monthly compliance reports
- Track PHI counts: Monitor trends
- Document feedback: Record tuning decisions
- Regular review: Quarterly accuracy assessment
Troubleshooting
PHI Not Detected
- Check vertical is set to healthcare
- Verify entity type threshold
- Check format matches expected patterns
- Review if detector is enabled
Too Many False Positives
- Increase threshold for problematic entity
- Add pattern to allowlist
- Provide feedback on false positives
- Consider context-specific tuning
Medical Terms Detected as PHI
Medical terminology should NOT be detected as PHI unless it contains patient identifiers. If you're seeing:
- Drug names flagged → Check if patient context is present
- Diagnosis codes flagged → Verify no embedded identifiers
- Procedure names flagged → Review for unique identifiers