Appearance
AI Threat Analysis: Reducing Alert Fatigue with Intelligent Classification
Executive Summary
Traditional pattern-matching security scanners produce high volumes of alerts, many of which are false positives. A PDF with a simple print-dialog script receives the same HIGH severity as a document containing obfuscated shellcode. Every /JS tag triggers a block. Security teams suffer alert fatigue—tuning out real threats amid noise—while users are frustrated by legitimate files being blocked.
GhostAI AI Threat Analysis addresses this by classifying each finding as malicious, suspicious, or benign using a multi-layer pipeline that combines deterministic heuristics with large language model (LLM) analysis. The system reduces false positives for benign content while ensuring known-bad patterns are never downgraded. This whitepaper describes the architecture, adversarial resistance, per-detector behavior, and regulatory alignment.
The Problem: Alert Fatigue
Symptom
- Benign PDF JavaScript (e.g.,
app.alert(),this.print(), form validation) is flagged at the same risk level as obfuscated code that executes shell commands or exfiltrates data. - Every
/JSor/JavaScripttag in a PDF triggers a block, regardless of intent. - Macro-enabled Office documents, suspicious URLs, and formula-injection candidates are treated uniformly—no context, no nuance.
- Security teams either disable detectors (increasing risk) or ignore alerts (missing real threats).
Root Cause
Pattern-based detectors cannot distinguish intent. They match structure, not semantics. A regex for eval( catches both malicious payloads and rare but legitimate uses. Without context, the scanner must assume the worst.
Architecture
The AI Threat Analysis pipeline processes security findings through six stages:
Detectors → PII Redaction → Heuristic Overrides → Multi-Finding Correlation
→ LLM Analysis → Confidence Gate → Admin Policy → PersistStage 1: Detectors
Upstream file scanners (PDF, Office, image, CSV, SVG, etc.) produce raw SecurityFinding objects with type, risk_level, detail, and optional extracted content.
Stage 2: PII Redaction
Before any content is sent to the LLM, detected PII entities (names, SSNs, emails, etc.) are replaced with type placeholders (e.g., [PERSON], [EMAIL]). This protects sensitive data from exposure to the AI provider while preserving structural context for analysis.
Stage 3: Heuristic Overrides
Content is checked against known-bad patterns: obfuscation (e.g., eval(), String.fromCharCode, base64 blobs), exfiltration (e.g., XMLHttpRequest, Invoke-WebRequest), shell execution (e.g., cmd.exe, powershell, certutil -decode), and exploit indicators (NOP sleds, shellcode). Findings matching these patterns bypass the LLM and are immediately classified as malicious with override_source: heuristic. They are never downgraded.
Stage 4: Multi-Finding Correlation
When a file triggers multiple detectors (e.g., PDF JavaScript + phishing URLs, or macro + prompt injection), a correlation context is built and injected into the LLM prompt. Compound signals (e.g., "JavaScript combined with suspicious URLs is a strong indicator of a multi-stage attack") improve classification accuracy.
Stage 5: LLM Analysis
Per-finding prompts are sent to the LLM with:
- Detector-specific instructions (e.g., "Determine whether this JavaScript is benign or malicious")
- Extracted content wrapped in
<file_content>delimiters - Correlation context when applicable
The LLM returns structured JSON: verdict (malicious | suspicious | benign), confidence (0.0–1.0), and explanation.
Stage 6: Confidence Gate
If confidence < 0.7, the verdict is treated as inconclusive. The original risk level is retained; no downgrade occurs. This prevents low-confidence "benign" verdicts from weakening security.
Stage 7: Admin Policy
Organization-level policy can:
- Lock specific finding types to a minimum risk level (e.g., always treat
pdf_jsas high) - Disable AI threat analysis for the org
- Override adjusted risk when a type is locked (
override_source: admin_policy)
Stage 8: Persist
Enriched findings (with ai_status, ai_verdict, ai_confidence, adjusted_risk_level, override_source) are stored. A content-hash cache avoids re-analyzing identical content; cache keys include PROMPT_VERSION so prompt changes invalidate stale entries.
Adversarial Resistance
Heuristic Overrides
Patterns in the heuristic set are associated with known malicious techniques. Findings matching these patterns never receive a downgrade. The LLM is not consulted for them. This prevents:
- Adversarial content crafted to elicit "benign" verdicts from the LLM
- Obfuscation, eval, shell, or exfiltration patterns being misclassified
Prompt Hardening
- Content delimiters: All user-controlled content is wrapped in
<file_content>tags. The system prompt instructs the LLM to treat content inside these tags as data to analyze, not instructions to follow. - Structured output: The LLM is required to respond with JSON only. No free-form instructions or markdown. Parsing extracts only the JSON object; extraneous text is ignored.
- Truncation: Extracted content is limited to
MAX_CONTENT_CHARS(e.g., 2000 characters) to reduce prompt-injection surface and control token usage.
Confidence Floor
A CONFIDENCE_FLOOR (e.g., 0.7) ensures that low-confidence "benign" verdicts do not downgrade risk. If the LLM is uncertain, the original detector risk level is preserved.
Per-Detector Analysis
| Detector Type | Context Extracted | What AI Can Determine |
|---|---|---|
| pdf_js | JavaScript from PDF xref objects, launch actions | Benign (print, form validation, zoom) vs. malicious (obfuscation, shell exec, exfiltration) |
| macro | Macro detection details, extracted macro context | Benign (spreadsheet automation, formatting) vs. malicious (auto-exec, payload download, registry mod) |
| phishing | URLs found, lines containing http | Benign (shortened links, docs) vs. malicious (typosquatting, credential harvesting, IDN homograph) |
| stego | Detection details, filename, content-type | False positive (JPEG artifacts, AI-generated LSB patterns) vs. genuine hidden data |
| csv_formula | Lines starting with =, +, -, @ | Benign (HYPERLINK, SUM, VLOOKUP) vs. malicious (CMD/EXEC, DDE, IMPORTXML exfiltration) |
| prompt_injection | Matched patterns, text context (~1000 chars) | Benign (educational content, security research) vs. malicious (actual injection, jailbreak attempts) |
| svg_threat | SVG content (decoded, truncated) | Benign (D3.js charts, animations) vs. malicious (XSS, credential stealing, redirects) |
| encrypted_archive, zip_bomb | — | Skipped; no LLM analysis (structural risk only) |
Regulatory Mapping
| Framework | How AI Threat Analysis Supports Compliance |
|---|---|
| HIPAA (Transmission Security) | Reduces false positives that block legitimate PHI-bearing documents; ensures malicious files are correctly identified and blocked before transmission or storage. |
| SOC 2 (Security Monitoring) | Demonstrates continuous monitoring and intelligent triage of security findings; audit trail of AI verdicts and overrides supports evidence of effective controls. |
| NIST AI RMF (AI Risk Identification) | Addresses AI system risks (e.g., adversarial manipulation) through heuristic overrides, prompt hardening, and confidence gates; supports governance of AI-assisted security decisions. |
| PCI-DSS (Cardholder Data Protection) | Helps prevent malicious files (e.g., keyloggers, exfiltration scripts) from entering cardholder data environments; reduces operational burden of manual review. |
Conclusion
AI Threat Analysis in GhostAI reduces alert fatigue by classifying security findings as malicious, suspicious, or benign using a multi-layer pipeline. Heuristic overrides ensure known-bad patterns are never downgraded. Prompt hardening, content truncation, and a confidence floor protect against adversarial manipulation of the LLM. Per-detector prompts and multi-finding correlation improve accuracy. The design supports HIPAA, SOC 2, NIST AI RMF, and PCI-DSS by improving security monitoring while reducing false positives that impede legitimate workflows.
References
- Threat Analysis Policy -- Admin configuration
- File Processing -- File processing pipeline