Appearance
Detection Endpoints
Detect and tokenize PII without sending to AI.
Detect PII
Scan text for sensitive data.
POST /api/detectRequest
json
{
"text": "John Smith, SSN 123-45-6789, email john@example.com",
"threshold": 0.5,
"entity_types": ["US_SSN", "EMAIL_ADDRESS", "PERSON"]
}| Field | Type | Required | Description |
|---|---|---|---|
text | string | Yes | Text to scan |
threshold | number | No | Min confidence (0-1) |
entity_types | array | No | Filter to specific types |
public_figure_enabled | boolean|null | No | Enable public figure reconciliation. When true, PERSON entities matching known public figures are excluded from PII results (not masked). Defaults to organization setting when null. Safety filters (co-occurrence, multi-detector gate) are always active. |
public_figure_tiers | string[] | No | Resolution tiers to use: "builtin" (local list — highest authority, can override multi-detector consensus), "wikidata" (Wikidata lookup — blocked when 2+ detectors agree), "llm" (LLM-based resolution — blocked when 2+ detectors agree). Defaults to ["builtin"]. |
public_knowledge_enabled | boolean|null | No | Enable reconciliation for non-person public entities (LOCATION, ORGANIZATION, EVENT, contextual DATE_TIME). |
public_knowledge_tiers | string[] | No | Tiers for non-person reconciliation: "builtin", "wikidata", "llm". Defaults to ["builtin"]. |
public_knowledge_entity_types | string[] | No | Entity types to reconcile. Typical values: ["LOCATION","ORGANIZATION","EVENT","DATE_TIME"]. |
ai_verify_mode | string | No | AI verification mode: all, suspicious, off |
ai_audit_level | string | No | AI audit level: none, summary, full. Clamped to org ceiling/floor. |
vendor_debug_access | boolean | No | Enable vendor debug access for audit data |
entity_thresholds | object | No | Per-entity confidence thresholds, e.g. {"US_SSN": 0.3, "PERSON": 0.7} |
languages | string[] | No | Detection languages, e.g. ["en", "es"]. NER is English-only; Presidio supports multiple. |
custom_model_endpoint | string | No | Custom OpenAI-compatible endpoint URL for AI detection |
referee_mode | string | No | PII referee mode: local (on-device LLM), external (cloud LLM), off. Overrides PII_AI_MODEL for this request. |
Response
json
{
"entities": [
{
"type": "PERSON",
"text": "John Smith",
"start": 0,
"end": 10,
"confidence": 0.95,
"compliance": ["GDPR", "CCPA"]
},
{
"type": "US_SSN",
"text": "123-45-6789",
"start": 16,
"end": 27,
"confidence": 0.99,
"compliance": ["HIPAA", "PCI-DSS"]
},
{
"type": "EMAIL_ADDRESS",
"text": "john@example.com",
"start": 35,
"end": 51,
"confidence": 0.98,
"compliance": ["GDPR", "CCPA"]
}
],
"pii_count": 3,
"compliance_tags": ["HIPAA", "PCI-DSS", "GDPR", "CCPA"],
"detection_time_ms": 45
}Public Figure Reconciliation
When public_figure_enabled is true, PERSON entities that match known public figures are excluded from the entities array — they are not masked or tokenized. Instead, they appear in the reconciled_public_figures array (see below). This prevents false-positive masking of names like "Albert Einstein" or "Barack Obama."
Three safety filters run before reconciliation:
- Co-occurrence filter — blocks reconciliation if correlated PII (email, phone, URL) appears near the name
- Multi-detector consensus gate — when 2+ detectors flagged the name, only the
"builtin"tier can override - Confidence threshold — each tier has a minimum confidence score
Reconciled Public Figures
When public figure reconciliation is enabled, the response includes a reconciled_public_figures array in the SSE start event (streaming) or alongside the entities array (non-streaming). Each entry describes a PERSON entity that was identified as a public figure and therefore excluded from PII masking.
json
{
"reconciled_public_figures": [
{
"entity_type": "PERSON",
"text": "Albert Einstein",
"start": 7,
"end": 22,
"reconciled_as": "Albert Einstein",
"reconciled_source": "builtin",
"reconciled_category": "scientist",
"reconciled_confidence": 0.99
}
]
}| Field | Type | Description |
|---|---|---|
entity_type | string | Always "PERSON" for reconciled public figures |
text | string | Original text span from the input |
start | number | Start character offset in the input |
end | number | End character offset in the input |
reconciled_as | string | Canonical name of the matched public figure |
reconciled_source | string | Resolution tier that matched: "builtin", "wikidata", or "llm" |
reconciled_category | string | Category of the public figure (e.g. "politician", "scientist", "athlete") |
reconciled_confidence | number | Confidence score of the reconciliation (0-1) |
Reconciled Public Knowledge
When public knowledge reconciliation is enabled, responses can include reconciled_public_knowledge.
json
{
"reconciled_public_knowledge": [
{
"entity_type": "LOCATION",
"text": "Utah",
"start": 22,
"end": 27,
"reconciled_as": "public_knowledge",
"reconciled_source": "builtin",
"reconciled_category": "us_state_or_territory",
"reconciled_confidence": 1.0
}
]
}Example
bash
curl -X POST https://api.ghostai.com/v1/detect \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"text": "Contact me at (555) 123-4567 or visit 123 Main St",
"threshold": 0.7
}'Tokenize
Replace PII with tokens.
POST /api/tokenizeRequest
json
{
"text": "My SSN is 123-45-6789",
"session_id": "sess_abc123"
}Response
json
{
"tokenized_text": "My SSN is [US_SSN_a1b2c3]",
"session_id": "sess_abc123",
"tokens": [
{
"token": "[US_SSN_a1b2c3]",
"type": "US_SSN",
"start": 10,
"end": 21
}
],
"token_count": 1
}Example
bash
curl -X POST https://api.ghostai.com/v1/tokenize \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"text": "Patient John Doe, MRN 12345678",
"session_id": "medical_session_1"
}'Detokenize
Restore original values from tokens.
POST /api/detokenizeRequest
json
{
"text": "Your SSN [US_SSN_a1b2c3] has been verified",
"session_id": "sess_abc123"
}Response
json
{
"detokenized_text": "Your SSN 123-45-6789 has been verified",
"tokens_resolved": 1,
"unresolved_tokens": []
}Example
bash
curl -X POST https://api.ghostai.com/v1/detokenize \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"text": "Hello [PERSON_x1y2], your email [EMAIL_a1b2] is confirmed",
"session_id": "sess_abc123"
}'Security Detection Fields
The scan response includes security threat findings alongside PII detection:
json
{
"security": {
"macros": {
"detected": true,
"type": "VBA (OOXML)",
"auto_exec": true,
"suspicious_keywords": ["Shell", "CreateObject"],
"risk_level": "critical"
},
"steganography": {
"detected": false,
"confidence": 0.12,
"method": "",
"risk_level": "none"
},
"phishing_urls": {
"suspicious_count": 2,
"urls": [
{"url": "https://g00gle.com/login", "risk": "high", "reasons": ["Possible typosquatting of google.com"]}
]
},
"encryption": {
"is_encrypted": false,
"type": ""
},
"malware": {
"scanned": true,
"is_clean": true,
"malware_name": ""
},
"injection": {
"detected": false,
"patterns": []
}
}
}| Field | Type | Description |
|---|---|---|
security.macros | object | VBA/XLM macro detection results |
security.steganography | object | Image steganography analysis |
security.phishing_urls | object | Phishing URL detection |
security.encryption | object | Encryption/password detection |
security.malware | object | ClamAV scan results |
security.injection | object | Prompt injection pattern detection |
Detect Code
Scan for source code and secrets.
POST /api/detect/codeRequest
json
{
"text": "const apiKey = 'sk-1234567890abcdef';"
}Response
json
{
"has_code": true,
"language": "javascript",
"confidence": 0.95,
"secrets": [
{
"type": "API_KEY",
"provider": "openai",
"masked": "sk-****cdef",
"start": 16,
"end": 36
}
],
"code_blocks": [
{
"start": 0,
"end": 37,
"language": "javascript"
}
]
}Detect Exfiltration
Check for obfuscated PII.
POST /api/detect/exfiltrationRequest
json
{
"text": "My ID is MTIzLTQ1LTY3ODk="
}Response
json
{
"obfuscation_detected": true,
"methods_found": ["base64"],
"decoded_entities": [
{
"original": "MTIzLTQ1LTY3ODk=",
"decoded": "123-45-6789",
"method": "base64",
"entity_type": "US_SSN",
"confidence": 0.99
}
]
}Batch Detection
Scan multiple texts at once.
POST /api/detect/batchRequest
json
{
"items": [
{"id": "1", "text": "Email: john@example.com"},
{"id": "2", "text": "SSN: 123-45-6789"},
{"id": "3", "text": "No PII here"}
]
}Response
json
{
"results": [
{
"id": "1",
"pii_count": 1,
"entities": [...]
},
{
"id": "2",
"pii_count": 1,
"entities": [...]
},
{
"id": "3",
"pii_count": 0,
"entities": []
}
],
"total_pii": 2,
"processing_time_ms": 120
}Detection Settings
Get and update user detection preferences, resolved through the 5-tier configuration hierarchy.
Get Detection Settings
GET /api/settings/detectionReturns the resolved detection settings for the authenticated user.
Response
json
{
"execution_mode": "grouped",
"threshold": 0.5,
"presidio_enabled": true,
"ner_enabled": true,
"code_enabled": true,
"ai_enabled": false,
"ai_model": "user",
"entity_thresholds": {},
"detector_priorities": {},
"detector_groups": {},
"enabled_entities": ["US_SSN", "EMAIL_ADDRESS", "PERSON"],
"languages": ["en"],
"custom_model_endpoint": null,
"redaction_style": "tokenize",
"ai_verify_mode": "off",
"ai_audit_level": "summary",
"vendor_debug_access": false,
"local_llm_available": true,
"local_llm_status": {
"model": "llama-3.1-8b",
"runtime": "llama-cpp",
"platform": "metal",
"loaded": true
}
}| Field | Type | Description |
|---|---|---|
local_llm_available | boolean | Whether the local LLM referee is available on this server (model downloaded and runtime configured) |
local_llm_status | object|null | Status of the local LLM referee when available. null when local_llm_available is false. |
local_llm_status.model | string | Active local model key (e.g. llama-3.1-8b, phi-3.5-mini) |
local_llm_status.runtime | string | Active runtime backend (llama-cpp, ollama, vllm) |
local_llm_status.platform | string | Detected hardware platform (metal, cuda, cpu) |
local_llm_status.loaded | boolean | Whether the model is currently loaded in memory |
pii_gliner_model_source | string | null | Deprecated. Formerly indicated the GLiNER model source. Omitted in new deployments. |
pii_gliner_model_path | string | null | Deprecated. Formerly indicated the GLiNER model path. Omitted in new deployments. |
All other fields match the Update Detection Settings request schema described below.
Update Detection Settings
PUT /api/settings/detectionRequest
json
{
"execution_mode": "grouped",
"threshold": 0.5,
"presidio_enabled": true,
"ner_enabled": true,
"code_enabled": true,
"ai_enabled": false,
"ai_model": "user",
"entity_thresholds": {
"US_SSN": 0.3,
"PERSON": 0.7,
"DATE_TIME": 0.5
},
"detector_priorities": {
"code": 100,
"presidio": 90,
"ner": 80,
"ai": 70
},
"detector_groups": {
"code": 0,
"presidio": 1,
"ner": 1,
"ai": 2
},
"enabled_entities": ["US_SSN", "EMAIL_ADDRESS", "PERSON"],
"languages": ["en", "es"],
"custom_model_endpoint": null,
"redaction_style": "tokenize",
"ai_verify_mode": "suspicious",
"ai_audit_level": "summary",
"vendor_debug_access": false
}| Field | Type | Description |
|---|---|---|
execution_mode | string | parallel, series, or grouped |
threshold | number | Global confidence threshold (0-1) |
presidio_enabled | boolean | Enable Presidio pattern detection |
ner_enabled | boolean | Enable NER model detection |
code_enabled | boolean | Enable code & secrets detection |
ai_enabled | boolean | Enable AI-powered detection |
ai_model | string | dedicated (Private Model — default), user (Cloud API — your configured LLM), local (on-device local LLM), or custom (custom endpoint). Selectable in Settings → AI Detector tab. |
entity_thresholds | object | Per-entity type confidence thresholds |
detector_priorities | object | Per-detector priority values (higher = runs first) |
detector_groups | object | Per-detector group numbers (same group = parallel) |
enabled_entities | list[str] | Entity types enabled for detection. Only these types are detected; disabled types produce no results. Available on both GET and PUT /api/settings/detection. |
languages | string[] | Detection languages (NER is English-only) |
custom_model_endpoint | string|null | Custom OpenAI-compatible endpoint URL |
redaction_style | string | tokenize, mask, hash, encrypt, remove, partial |
ai_verify_mode | string | off, suspicious, all |
verify_model_mode | string | Model used for AI verification: dedicated (Private Model — default), same (use ai_model), user (Cloud API), local (on-device), or custom (custom endpoint). Selectable in Settings → AI Detector tab. |
ai_audit_level | string | none, summary, full |
vendor_debug_access | boolean | Allow vendor debug access to audit data |
All fields are optional on update. Only provided (non-null) values are persisted.
Per-Conversation AI Model Override
The ai_model setting can be overridden per conversation via the ConversationConfigRequest:
PUT /api/conversations/{id}/configRequest
json
{
"ai_model": "local"
}| Field | Type | Required | Description |
|---|---|---|---|
ai_model | string | No | Override the detection AI model for this conversation: user (Cloud API) or local (Private Model). When set, overrides the user-level ai_model setting for this conversation only. Follows the 5-tier config hierarchy (Cloud → Tenant → User → Conversation → Subject). |
AI Protection admin policy
Organization admins can get and update the org-wide AI Protection policy (detection defaults and custom PII model path). See AI Protection for the UI.
Get AI Protection policy
GET /api/admin/ai-protection/policyReturns the organization's AI Protection policy. Requires org admin.
Response
json
{
"default_ai_model": "private",
"lock_ai_model": false,
"default_verify_mode": "suspicious",
"lock_verify_mode": false,
"pii_ner_model": "dymium/Dymium-NER-v1"
}| Field | Type | Description |
|---|---|---|
default_ai_model | string | cloud, private, or custom |
lock_ai_model | boolean | When true, users cannot override the org default detection model |
default_verify_mode | string | all, suspicious, or off |
lock_verify_mode | boolean | When true, users cannot override the org default verification mode |
pii_gliner_model_path | string | null | Deprecated. Formerly the per-org custom GLiNER checkpoint path. NER detection now uses dymium/Dymium-NER-v1. Field retained for backward compatibility; ignored in new deployments. |
Update AI Protection policy
PUT /api/admin/ai-protection/policyRequest
Same schema as the GET response. All fields are optional; only provided fields are updated. The pii_gliner_model_path field is deprecated and ignored; NER detection uses dymium/Dymium-NER-v1.
PII Model Health
Check the status of the local PII LLM model.
GET /health/pii-modelResponse
json
{
"status": "ready",
"model": "llama-3.1-8b",
"runtime": "llama-cpp",
"loaded": true,
"platform": "cuda",
"auto_selected_model": "llama-3.1-8b"
}| Field | Type | Description |
|---|---|---|
status | string | ready, not configured, or error message |
model | string | Active model key (or auto) |
runtime | string | Active runtime backend |
loaded | boolean | Whether the model is loaded in memory |
platform | string | Detected platform (metal, cuda, cpu) |
auto_selected_model | string | Model auto-selected for the platform (e.g. llama-3.1-8b for GPU, phi-3.5-mini for CPU; only when model=auto) |
Detect Subject
Auto-detect conversation subjects/topics for context-appropriate detection settings.
POST /api/detect-subjectRequest
json
{
"text": "Please forward the patient's lab results to Dr. Johnson at the clinic."
}| Field | Type | Required | Description |
|---|---|---|---|
text | string | Yes | Text to analyze for subject detection |
Response
json
{
"subjects": [
{
"subject_type": "medical",
"confidence": 0.92,
"keywords_matched": ["patient", "lab results", "clinic"]
}
]
}| Field | Type | Description |
|---|---|---|
subject_type | string | Matched subject preset (see table below) |
confidence | number | Detection confidence (0-1) |
keywords_matched | string[] | Keywords or patterns that triggered the match |
Built-in Subject Presets
| Subject Type | Description | Example Triggers |
|---|---|---|
medical | Healthcare and clinical content | "patient", "diagnosis", "prescription" |
financial | Banking and financial content | "account number", "routing number", "credit card" |
legal | Legal proceedings and contracts | "plaintiff", "defendant", "attorney" |
code | Source code and technical content | Code block patterns, API key patterns |
personal_info | General PII patterns | Name + contact keywords, email/phone/SSN patterns, title + name patterns ("Dr. Smith") |
The personal_info preset uses both keyword matching and regex patterns to detect general PII-related content, triggering the subject indicator above the chat input when matched.
CLI Reference
pii-model
Manage local PII LLM models.
bash
# List available models with platform recommendation
python -m app.cli.pii_model list
# Download the auto-recommended model
python -m app.cli.pii_model download
# Download a specific model via HuggingFace
python -m app.cli.pii_model download --model llama-3.1-8b
# Download via Ollama
python -m app.cli.pii_model download --model llama-3.1-8b --runtime ollama
# Show current platform and model status
python -m app.cli.pii_model status
# Verify SHA-256 integrity of downloaded models
python -m app.cli.pii_model verify
# Verify a specific model
python -m app.cli.pii_model verify --model llama-3.1-8b| Subcommand | Description |
|---|---|
list | List all models in the registry with platform fit and download status |
download | Download a model GGUF from HuggingFace or pull via Ollama |
status | Show detected platform, recommended model, and downloaded models |
verify | Verify SHA-256 checksum integrity of downloaded model files |