Skip to content

AI Threat Analysis: Reducing Alert Fatigue with Intelligent Classification ​

Executive Summary ​

Traditional pattern-matching security scanners produce high volumes of alerts, many of which are false positives. A PDF with a simple print-dialog script receives the same HIGH severity as a document containing obfuscated shellcode. Every /JS tag triggers a block. Security teams suffer alert fatigue—tuning out real threats amid noise—while users are frustrated by legitimate files being blocked.

GhostAI AI Threat Analysis addresses this by classifying each finding as malicious, suspicious, or benign using a multi-layer pipeline that combines deterministic heuristics with large language model (LLM) analysis. The system reduces false positives for benign content while ensuring known-bad patterns are never downgraded. This whitepaper describes the architecture, adversarial resistance, per-detector behavior, and regulatory alignment.


The Problem: Alert Fatigue ​

Symptom ​

  • Benign PDF JavaScript (e.g., app.alert(), this.print(), form validation) is flagged at the same risk level as obfuscated code that executes shell commands or exfiltrates data.
  • Every /JS or /JavaScript tag in a PDF triggers a block, regardless of intent.
  • Macro-enabled Office documents, suspicious URLs, and formula-injection candidates are treated uniformly—no context, no nuance.
  • Security teams either disable detectors (increasing risk) or ignore alerts (missing real threats).

Root Cause ​

Pattern-based detectors cannot distinguish intent. They match structure, not semantics. A regex for eval( catches both malicious payloads and rare but legitimate uses. Without context, the scanner must assume the worst.


Architecture ​

The AI Threat Analysis pipeline processes security findings through six stages:

Detectors → PII Redaction → Heuristic Overrides → Multi-Finding Correlation
    → LLM Analysis → Confidence Gate → Admin Policy → Persist

Stage 1: Detectors ​

Upstream file scanners (PDF, Office, image, CSV, SVG, etc.) produce raw SecurityFinding objects with type, risk_level, detail, and optional extracted content.

Stage 2: PII Redaction ​

Before any content is sent to the LLM, detected PII entities (names, SSNs, emails, etc.) are replaced with type placeholders (e.g., [PERSON], [EMAIL]). This protects sensitive data from exposure to the AI provider while preserving structural context for analysis.

Stage 3: Heuristic Overrides ​

Content is checked against known-bad patterns: obfuscation (e.g., eval(), String.fromCharCode, base64 blobs), exfiltration (e.g., XMLHttpRequest, Invoke-WebRequest), shell execution (e.g., cmd.exe, powershell, certutil -decode), and exploit indicators (NOP sleds, shellcode). Findings matching these patterns bypass the LLM and are immediately classified as malicious with override_source: heuristic. They are never downgraded.

Stage 4: Multi-Finding Correlation ​

When a file triggers multiple detectors (e.g., PDF JavaScript + phishing URLs, or macro + prompt injection), a correlation context is built and injected into the LLM prompt. Compound signals (e.g., "JavaScript combined with suspicious URLs is a strong indicator of a multi-stage attack") improve classification accuracy.

Stage 5: LLM Analysis ​

Per-finding prompts are sent to the LLM with:

  • Detector-specific instructions (e.g., "Determine whether this JavaScript is benign or malicious")
  • Extracted content wrapped in <file_content> delimiters
  • Correlation context when applicable

The LLM returns structured JSON: verdict (malicious | suspicious | benign), confidence (0.0–1.0), and explanation.

Stage 6: Confidence Gate ​

If confidence < 0.7, the verdict is treated as inconclusive. The original risk level is retained; no downgrade occurs. This prevents low-confidence "benign" verdicts from weakening security.

Stage 7: Admin Policy ​

Organization-level policy can:

  • Lock specific finding types to a minimum risk level (e.g., always treat pdf_js as high)
  • Disable AI threat analysis for the org
  • Override adjusted risk when a type is locked (override_source: admin_policy)

Stage 8: Persist ​

Enriched findings (with ai_status, ai_verdict, ai_confidence, adjusted_risk_level, override_source) are stored. A content-hash cache avoids re-analyzing identical content; cache keys include PROMPT_VERSION so prompt changes invalidate stale entries.


Adversarial Resistance ​

Heuristic Overrides ​

Patterns in the heuristic set are associated with known malicious techniques. Findings matching these patterns never receive a downgrade. The LLM is not consulted for them. This prevents:

  • Adversarial content crafted to elicit "benign" verdicts from the LLM
  • Obfuscation, eval, shell, or exfiltration patterns being misclassified

Prompt Hardening ​

  • Content delimiters: All user-controlled content is wrapped in <file_content> tags. The system prompt instructs the LLM to treat content inside these tags as data to analyze, not instructions to follow.
  • Structured output: The LLM is required to respond with JSON only. No free-form instructions or markdown. Parsing extracts only the JSON object; extraneous text is ignored.
  • Truncation: Extracted content is limited to MAX_CONTENT_CHARS (e.g., 2000 characters) to reduce prompt-injection surface and control token usage.

Confidence Floor ​

A CONFIDENCE_FLOOR (e.g., 0.7) ensures that low-confidence "benign" verdicts do not downgrade risk. If the LLM is uncertain, the original detector risk level is preserved.


Per-Detector Analysis ​

Detector TypeContext ExtractedWhat AI Can Determine
pdf_jsJavaScript from PDF xref objects, launch actionsBenign (print, form validation, zoom) vs. malicious (obfuscation, shell exec, exfiltration)
macroMacro detection details, extracted macro contextBenign (spreadsheet automation, formatting) vs. malicious (auto-exec, payload download, registry mod)
phishingURLs found, lines containing httpBenign (shortened links, docs) vs. malicious (typosquatting, credential harvesting, IDN homograph)
stegoDetection details, filename, content-typeFalse positive (JPEG artifacts, AI-generated LSB patterns) vs. genuine hidden data
csv_formulaLines starting with =, +, -, @Benign (HYPERLINK, SUM, VLOOKUP) vs. malicious (CMD/EXEC, DDE, IMPORTXML exfiltration)
prompt_injectionMatched patterns, text context (~1000 chars)Benign (educational content, security research) vs. malicious (actual injection, jailbreak attempts)
svg_threatSVG content (decoded, truncated)Benign (D3.js charts, animations) vs. malicious (XSS, credential stealing, redirects)
encrypted_archive, zip_bomb—Skipped; no LLM analysis (structural risk only)

Regulatory Mapping ​

FrameworkHow AI Threat Analysis Supports Compliance
HIPAA (Transmission Security)Reduces false positives that block legitimate PHI-bearing documents; ensures malicious files are correctly identified and blocked before transmission or storage.
SOC 2 (Security Monitoring)Demonstrates continuous monitoring and intelligent triage of security findings; audit trail of AI verdicts and overrides supports evidence of effective controls.
NIST AI RMF (AI Risk Identification)Addresses AI system risks (e.g., adversarial manipulation) through heuristic overrides, prompt hardening, and confidence gates; supports governance of AI-assisted security decisions.
PCI-DSS (Cardholder Data Protection)Helps prevent malicious files (e.g., keyloggers, exfiltration scripts) from entering cardholder data environments; reduces operational burden of manual review.

Conclusion ​

AI Threat Analysis in GhostAI reduces alert fatigue by classifying security findings as malicious, suspicious, or benign using a multi-layer pipeline. Heuristic overrides ensure known-bad patterns are never downgraded. Prompt hardening, content truncation, and a confidence floor protect against adversarial manipulation of the LLM. Per-detector prompts and multi-finding correlation improve accuracy. The design supports HIPAA, SOC 2, NIST AI RMF, and PCI-DSS by improving security monitoring while reducing false positives that impede legitimate workflows.


References ​

SOC 2 Type II Certified | BAA Available | HIPAA-Ready