Skip to content

Detection Endpoints ​

Detect and tokenize PII without sending to AI.

Detect PII ​

Scan text for sensitive data.

POST /api/detect

Request ​

json
{
  "text": "John Smith, SSN 123-45-6789, email john@example.com",
  "threshold": 0.5,
  "entity_types": ["US_SSN", "EMAIL_ADDRESS", "PERSON"]
}
FieldTypeRequiredDescription
textstringYesText to scan
thresholdnumberNoMin confidence (0-1)
entity_typesarrayNoFilter to specific types
public_figure_enabledboolean|nullNoEnable public figure reconciliation. When true, PERSON entities matching known public figures are excluded from PII results (not masked). Defaults to organization setting when null. Safety filters (co-occurrence, multi-detector gate) are always active.
public_figure_tiersstring[]NoResolution tiers to use: "builtin" (local list — highest authority, can override multi-detector consensus), "wikidata" (Wikidata lookup — blocked when 2+ detectors agree), "llm" (LLM-based resolution — blocked when 2+ detectors agree). Defaults to ["builtin"].
public_knowledge_enabledboolean|nullNoEnable reconciliation for non-person public entities (LOCATION, ORGANIZATION, EVENT, contextual DATE_TIME).
public_knowledge_tiersstring[]NoTiers for non-person reconciliation: "builtin", "wikidata", "llm". Defaults to ["builtin"].
public_knowledge_entity_typesstring[]NoEntity types to reconcile. Typical values: ["LOCATION","ORGANIZATION","EVENT","DATE_TIME"].
ai_verify_modestringNoAI verification mode: all, suspicious, off
ai_audit_levelstringNoAI audit level: none, summary, full. Clamped to org ceiling/floor.
vendor_debug_accessbooleanNoEnable vendor debug access for audit data
entity_thresholdsobjectNoPer-entity confidence thresholds, e.g. {"US_SSN": 0.3, "PERSON": 0.7}
languagesstring[]NoDetection languages, e.g. ["en", "es"]. NER is English-only; Presidio supports multiple.
custom_model_endpointstringNoCustom OpenAI-compatible endpoint URL for AI detection
referee_modestringNoPII referee mode: local (on-device LLM), external (cloud LLM), off. Overrides PII_AI_MODEL for this request.

Response ​

json
{
  "entities": [
    {
      "type": "PERSON",
      "text": "John Smith",
      "start": 0,
      "end": 10,
      "confidence": 0.95,
      "compliance": ["GDPR", "CCPA"]
    },
    {
      "type": "US_SSN",
      "text": "123-45-6789",
      "start": 16,
      "end": 27,
      "confidence": 0.99,
      "compliance": ["HIPAA", "PCI-DSS"]
    },
    {
      "type": "EMAIL_ADDRESS",
      "text": "john@example.com",
      "start": 35,
      "end": 51,
      "confidence": 0.98,
      "compliance": ["GDPR", "CCPA"]
    }
  ],
  "pii_count": 3,
  "compliance_tags": ["HIPAA", "PCI-DSS", "GDPR", "CCPA"],
  "detection_time_ms": 45
}

Public Figure Reconciliation

When public_figure_enabled is true, PERSON entities that match known public figures are excluded from the entities array — they are not masked or tokenized. Instead, they appear in the reconciled_public_figures array (see below). This prevents false-positive masking of names like "Albert Einstein" or "Barack Obama."

Three safety filters run before reconciliation:

  1. Co-occurrence filter — blocks reconciliation if correlated PII (email, phone, URL) appears near the name
  2. Multi-detector consensus gate — when 2+ detectors flagged the name, only the "builtin" tier can override
  3. Confidence threshold — each tier has a minimum confidence score

Reconciled Public Figures ​

When public figure reconciliation is enabled, the response includes a reconciled_public_figures array in the SSE start event (streaming) or alongside the entities array (non-streaming). Each entry describes a PERSON entity that was identified as a public figure and therefore excluded from PII masking.

json
{
  "reconciled_public_figures": [
    {
      "entity_type": "PERSON",
      "text": "Albert Einstein",
      "start": 7,
      "end": 22,
      "reconciled_as": "Albert Einstein",
      "reconciled_source": "builtin",
      "reconciled_category": "scientist",
      "reconciled_confidence": 0.99
    }
  ]
}
FieldTypeDescription
entity_typestringAlways "PERSON" for reconciled public figures
textstringOriginal text span from the input
startnumberStart character offset in the input
endnumberEnd character offset in the input
reconciled_asstringCanonical name of the matched public figure
reconciled_sourcestringResolution tier that matched: "builtin", "wikidata", or "llm"
reconciled_categorystringCategory of the public figure (e.g. "politician", "scientist", "athlete")
reconciled_confidencenumberConfidence score of the reconciliation (0-1)

Reconciled Public Knowledge ​

When public knowledge reconciliation is enabled, responses can include reconciled_public_knowledge.

json
{
  "reconciled_public_knowledge": [
    {
      "entity_type": "LOCATION",
      "text": "Utah",
      "start": 22,
      "end": 27,
      "reconciled_as": "public_knowledge",
      "reconciled_source": "builtin",
      "reconciled_category": "us_state_or_territory",
      "reconciled_confidence": 1.0
    }
  ]
}

Example ​

bash
curl -X POST https://api.ghostai.com/v1/detect \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Contact me at (555) 123-4567 or visit 123 Main St",
    "threshold": 0.7
  }'

Tokenize ​

Replace PII with tokens.

POST /api/tokenize

Request ​

json
{
  "text": "My SSN is 123-45-6789",
  "session_id": "sess_abc123"
}

Response ​

json
{
  "tokenized_text": "My SSN is [US_SSN_a1b2c3]",
  "session_id": "sess_abc123",
  "tokens": [
    {
      "token": "[US_SSN_a1b2c3]",
      "type": "US_SSN",
      "start": 10,
      "end": 21
    }
  ],
  "token_count": 1
}

Example ​

bash
curl -X POST https://api.ghostai.com/v1/tokenize \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Patient John Doe, MRN 12345678",
    "session_id": "medical_session_1"
  }'

Detokenize ​

Restore original values from tokens.

POST /api/detokenize

Request ​

json
{
  "text": "Your SSN [US_SSN_a1b2c3] has been verified",
  "session_id": "sess_abc123"
}

Response ​

json
{
  "detokenized_text": "Your SSN 123-45-6789 has been verified",
  "tokens_resolved": 1,
  "unresolved_tokens": []
}

Example ​

bash
curl -X POST https://api.ghostai.com/v1/detokenize \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Hello [PERSON_x1y2], your email [EMAIL_a1b2] is confirmed",
    "session_id": "sess_abc123"
  }'

Security Detection Fields ​

The scan response includes security threat findings alongside PII detection:

json
{
  "security": {
    "macros": {
      "detected": true,
      "type": "VBA (OOXML)",
      "auto_exec": true,
      "suspicious_keywords": ["Shell", "CreateObject"],
      "risk_level": "critical"
    },
    "steganography": {
      "detected": false,
      "confidence": 0.12,
      "method": "",
      "risk_level": "none"
    },
    "phishing_urls": {
      "suspicious_count": 2,
      "urls": [
        {"url": "https://g00gle.com/login", "risk": "high", "reasons": ["Possible typosquatting of google.com"]}
      ]
    },
    "encryption": {
      "is_encrypted": false,
      "type": ""
    },
    "malware": {
      "scanned": true,
      "is_clean": true,
      "malware_name": ""
    },
    "injection": {
      "detected": false,
      "patterns": []
    }
  }
}
FieldTypeDescription
security.macrosobjectVBA/XLM macro detection results
security.steganographyobjectImage steganography analysis
security.phishing_urlsobjectPhishing URL detection
security.encryptionobjectEncryption/password detection
security.malwareobjectClamAV scan results
security.injectionobjectPrompt injection pattern detection

Detect Code ​

Scan for source code and secrets.

POST /api/detect/code

Request ​

json
{
  "text": "const apiKey = 'sk-1234567890abcdef';"
}

Response ​

json
{
  "has_code": true,
  "language": "javascript",
  "confidence": 0.95,
  "secrets": [
    {
      "type": "API_KEY",
      "provider": "openai",
      "masked": "sk-****cdef",
      "start": 16,
      "end": 36
    }
  ],
  "code_blocks": [
    {
      "start": 0,
      "end": 37,
      "language": "javascript"
    }
  ]
}

Detect Exfiltration ​

Check for obfuscated PII.

POST /api/detect/exfiltration

Request ​

json
{
  "text": "My ID is MTIzLTQ1LTY3ODk="
}

Response ​

json
{
  "obfuscation_detected": true,
  "methods_found": ["base64"],
  "decoded_entities": [
    {
      "original": "MTIzLTQ1LTY3ODk=",
      "decoded": "123-45-6789",
      "method": "base64",
      "entity_type": "US_SSN",
      "confidence": 0.99
    }
  ]
}

Batch Detection ​

Scan multiple texts at once.

POST /api/detect/batch

Request ​

json
{
  "items": [
    {"id": "1", "text": "Email: john@example.com"},
    {"id": "2", "text": "SSN: 123-45-6789"},
    {"id": "3", "text": "No PII here"}
  ]
}

Response ​

json
{
  "results": [
    {
      "id": "1",
      "pii_count": 1,
      "entities": [...]
    },
    {
      "id": "2",
      "pii_count": 1,
      "entities": [...]
    },
    {
      "id": "3",
      "pii_count": 0,
      "entities": []
    }
  ],
  "total_pii": 2,
  "processing_time_ms": 120
}

Detection Settings ​

Get and update user detection preferences, resolved through the 5-tier configuration hierarchy.

Get Detection Settings ​

GET /api/settings/detection

Returns the resolved detection settings for the authenticated user.

Response ​

json
{
  "execution_mode": "grouped",
  "threshold": 0.5,
  "presidio_enabled": true,
  "ner_enabled": true,
  "code_enabled": true,
  "ai_enabled": false,
  "ai_model": "user",
  "entity_thresholds": {},
  "detector_priorities": {},
  "detector_groups": {},
  "enabled_entities": ["US_SSN", "EMAIL_ADDRESS", "PERSON"],
  "languages": ["en"],
  "custom_model_endpoint": null,
  "redaction_style": "tokenize",
  "ai_verify_mode": "off",
  "ai_audit_level": "summary",
  "vendor_debug_access": false,
  "local_llm_available": true,
  "local_llm_status": {
    "model": "llama-3.1-8b",
    "runtime": "llama-cpp",
    "platform": "metal",
    "loaded": true
  }
}
FieldTypeDescription
local_llm_availablebooleanWhether the local LLM referee is available on this server (model downloaded and runtime configured)
local_llm_statusobject|nullStatus of the local LLM referee when available. null when local_llm_available is false.
local_llm_status.modelstringActive local model key (e.g. llama-3.1-8b, phi-3.5-mini)
local_llm_status.runtimestringActive runtime backend (llama-cpp, ollama, vllm)
local_llm_status.platformstringDetected hardware platform (metal, cuda, cpu)
local_llm_status.loadedbooleanWhether the model is currently loaded in memory
pii_gliner_model_sourcestring | nullDeprecated. Formerly indicated the GLiNER model source. Omitted in new deployments.
pii_gliner_model_pathstring | nullDeprecated. Formerly indicated the GLiNER model path. Omitted in new deployments.

All other fields match the Update Detection Settings request schema described below.

Update Detection Settings ​

PUT /api/settings/detection

Request ​

json
{
  "execution_mode": "grouped",
  "threshold": 0.5,
  "presidio_enabled": true,
  "ner_enabled": true,
  "code_enabled": true,
  "ai_enabled": false,
  "ai_model": "user",
  "entity_thresholds": {
    "US_SSN": 0.3,
    "PERSON": 0.7,
    "DATE_TIME": 0.5
  },
  "detector_priorities": {
    "code": 100,
    "presidio": 90,
    "ner": 80,
    "ai": 70
  },
  "detector_groups": {
    "code": 0,
    "presidio": 1,
    "ner": 1,
    "ai": 2
  },
  "enabled_entities": ["US_SSN", "EMAIL_ADDRESS", "PERSON"],
  "languages": ["en", "es"],
  "custom_model_endpoint": null,
  "redaction_style": "tokenize",
  "ai_verify_mode": "suspicious",
  "ai_audit_level": "summary",
  "vendor_debug_access": false
}
FieldTypeDescription
execution_modestringparallel, series, or grouped
thresholdnumberGlobal confidence threshold (0-1)
presidio_enabledbooleanEnable Presidio pattern detection
ner_enabledbooleanEnable NER model detection
code_enabledbooleanEnable code & secrets detection
ai_enabledbooleanEnable AI-powered detection
ai_modelstringdedicated (Private Model — default), user (Cloud API — your configured LLM), local (on-device local LLM), or custom (custom endpoint). Selectable in Settings → AI Detector tab.
entity_thresholdsobjectPer-entity type confidence thresholds
detector_prioritiesobjectPer-detector priority values (higher = runs first)
detector_groupsobjectPer-detector group numbers (same group = parallel)
enabled_entitieslist[str]Entity types enabled for detection. Only these types are detected; disabled types produce no results. Available on both GET and PUT /api/settings/detection.
languagesstring[]Detection languages (NER is English-only)
custom_model_endpointstring|nullCustom OpenAI-compatible endpoint URL
redaction_stylestringtokenize, mask, hash, encrypt, remove, partial
ai_verify_modestringoff, suspicious, all
verify_model_modestringModel used for AI verification: dedicated (Private Model — default), same (use ai_model), user (Cloud API), local (on-device), or custom (custom endpoint). Selectable in Settings → AI Detector tab.
ai_audit_levelstringnone, summary, full
vendor_debug_accessbooleanAllow vendor debug access to audit data

All fields are optional on update. Only provided (non-null) values are persisted.

Per-Conversation AI Model Override ​

The ai_model setting can be overridden per conversation via the ConversationConfigRequest:

PUT /api/conversations/{id}/config

Request ​

json
{
  "ai_model": "local"
}
FieldTypeRequiredDescription
ai_modelstringNoOverride the detection AI model for this conversation: user (Cloud API) or local (Private Model). When set, overrides the user-level ai_model setting for this conversation only. Follows the 5-tier config hierarchy (Cloud → Tenant → User → Conversation → Subject).

AI Protection admin policy ​

Organization admins can get and update the org-wide AI Protection policy (detection defaults and custom PII model path). See AI Protection for the UI.

Get AI Protection policy ​

GET /api/admin/ai-protection/policy

Returns the organization's AI Protection policy. Requires org admin.

Response ​

json
{
  "default_ai_model": "private",
  "lock_ai_model": false,
  "default_verify_mode": "suspicious",
  "lock_verify_mode": false,
  "pii_ner_model": "dymium/Dymium-NER-v1"
}
FieldTypeDescription
default_ai_modelstringcloud, private, or custom
lock_ai_modelbooleanWhen true, users cannot override the org default detection model
default_verify_modestringall, suspicious, or off
lock_verify_modebooleanWhen true, users cannot override the org default verification mode
pii_gliner_model_pathstring | nullDeprecated. Formerly the per-org custom GLiNER checkpoint path. NER detection now uses dymium/Dymium-NER-v1. Field retained for backward compatibility; ignored in new deployments.

Update AI Protection policy ​

PUT /api/admin/ai-protection/policy

Request ​

Same schema as the GET response. All fields are optional; only provided fields are updated. The pii_gliner_model_path field is deprecated and ignored; NER detection uses dymium/Dymium-NER-v1.

PII Model Health ​

Check the status of the local PII LLM model.

GET /health/pii-model

Response ​

json
{
  "status": "ready",
  "model": "llama-3.1-8b",
  "runtime": "llama-cpp",
  "loaded": true,
  "platform": "cuda",
  "auto_selected_model": "llama-3.1-8b"
}
FieldTypeDescription
statusstringready, not configured, or error message
modelstringActive model key (or auto)
runtimestringActive runtime backend
loadedbooleanWhether the model is loaded in memory
platformstringDetected platform (metal, cuda, cpu)
auto_selected_modelstringModel auto-selected for the platform (e.g. llama-3.1-8b for GPU, phi-3.5-mini for CPU; only when model=auto)

Detect Subject ​

Auto-detect conversation subjects/topics for context-appropriate detection settings.

POST /api/detect-subject

Request ​

json
{
  "text": "Please forward the patient's lab results to Dr. Johnson at the clinic."
}
FieldTypeRequiredDescription
textstringYesText to analyze for subject detection

Response ​

json
{
  "subjects": [
    {
      "subject_type": "medical",
      "confidence": 0.92,
      "keywords_matched": ["patient", "lab results", "clinic"]
    }
  ]
}
FieldTypeDescription
subject_typestringMatched subject preset (see table below)
confidencenumberDetection confidence (0-1)
keywords_matchedstring[]Keywords or patterns that triggered the match

Built-in Subject Presets ​

Subject TypeDescriptionExample Triggers
medicalHealthcare and clinical content"patient", "diagnosis", "prescription"
financialBanking and financial content"account number", "routing number", "credit card"
legalLegal proceedings and contracts"plaintiff", "defendant", "attorney"
codeSource code and technical contentCode block patterns, API key patterns
personal_infoGeneral PII patternsName + contact keywords, email/phone/SSN patterns, title + name patterns ("Dr. Smith")

The personal_info preset uses both keyword matching and regex patterns to detect general PII-related content, triggering the subject indicator above the chat input when matched.

CLI Reference ​

pii-model ​

Manage local PII LLM models.

bash
# List available models with platform recommendation
python -m app.cli.pii_model list

# Download the auto-recommended model
python -m app.cli.pii_model download

# Download a specific model via HuggingFace
python -m app.cli.pii_model download --model llama-3.1-8b

# Download via Ollama
python -m app.cli.pii_model download --model llama-3.1-8b --runtime ollama

# Show current platform and model status
python -m app.cli.pii_model status

# Verify SHA-256 integrity of downloaded models
python -m app.cli.pii_model verify

# Verify a specific model
python -m app.cli.pii_model verify --model llama-3.1-8b
SubcommandDescription
listList all models in the registry with platform fit and download status
downloadDownload a model GGUF from HuggingFace or pull via Ollama
statusShow detected platform, recommended model, and downloaded models
verifyVerify SHA-256 checksum integrity of downloaded model files

SOC 2 Type II Certified | BAA Available | HIPAA-Ready