Appearance
Tool Security Center Full Lab
Ten steps. Fifty minutes. Catalog to compliance export — all through the Tooling API.
This lab exercises the Tool Security Center using HTTP calls against /api/tooling/* endpoints. You will browse the tool catalog and marketplace, inspect organization posture, create and simulate policies, process approvals, author chain rules, analyze supply chain and shadow tools, watch enforcement events, and generate a multi-framework compliance report with CSV export.
Scenario
You are a security engineer at Northwind Health. Your tenant runs chat tools, MCP servers, and vendor function calling in one GhostAI deployment. Leadership asked for: (1) authoritative tool inventory, (2) enforced PII policies, (3) approval gates, (4) chain-based controls, (5) supply chain visibility, and (6) auditor-ready exports. You will validate each control via the API.
Prerequisites
- [ ] GhostAI backend reachable (e.g.,
http://localhost:8000or your staging URL) - [ ] Admin or security admin API key with Tool Security entitlements:
GHOSTAI_API_KEYexported - [ ]
curlandpython3(orjq) available for JSON formatting - [ ] Feature flags enabled for your tenant: Tool Security Center, policy write, approvals, chain rules, supply chain, compliance export (confirm with platform admin — flag names vary by deployment tier)
- [ ] Optional: second, non-admin API key to submit approval requests (more realistic approval queue)
Set environment variables
bash
export GHOSTAI_BASE_URL="${GHOSTAI_BASE_URL:-http://localhost:8000}"
export GHOSTAI_API_KEY="gst_your_admin_key"Verify authentication
bash
curl -sS -H "Authorization: Bearer $GHOSTAI_API_KEY" \
"$GHOSTAI_BASE_URL/api/tooling/catalog?limit=1"You should receive HTTP 200 and JSON (not 401/403). If you receive 403, your key lacks Tool Security permissions or the feature is disabled for the tenant.
Step 1: Access the Tool Security Center (UI)
Duration: 3 minutes
Goal: Confirm the admin surface matches API entitlements before driving everything from curl.
1.1 Open Admin Dashboard
- Sign in as an admin user in the browser.
- Navigate to Admin Dashboard → Tool Security (or Tooling → Security).
1.2 Confirm panels
You should see navigation or cards for Catalog, Posture, Policies, Approvals, Chain rules, Supply chain, Enforcement, and Compliance (exact labels may vary).
Expected outcome
- UI loads without entitlement errors.
- Catalog shows at least one row (if empty, complete Step 2 first, then refresh).
Step 2: Browse the Tool Catalog
Duration: 4 minutes
Goal: Retrieve the authoritative tool list and optional marketplace extensions.
2.1 List catalog entries
bash
curl -sS -H "Authorization: Bearer $GHOSTAI_API_KEY" \
"$GHOSTAI_BASE_URL/api/tooling/catalog?limit=50" | python3 -m json.toolInspect each item for:
tool_id— stable identifier used in policies and approvalscategory— one of:chat,mcp_native,mcp_upstream,openai_function,anthropic_tool,a2a,langchain,integration,infrastructurerisk_level—low,medium,high,criticaldata_classification— sensitivity labels used in posture and compliance mapping
2.2 Browse marketplace / extended catalog
bash
curl -sS -H "Authorization: Bearer $GHOSTAI_API_KEY" \
"$GHOSTAI_BASE_URL/api/tooling/catalog/browse?limit=50" | python3 -m json.toolExpected outcome
GET /api/tooling/catalogreturns org-visible, approved (or registered) tools.GET /api/tooling/catalog/browsereturns installable or discoverable templates (response shape may includesource,publisher,risk_level).
2.3 Pick a lab tool
Export the first catalog tool_id returned by the API (adjust the Python if your response nests items differently):
bash
export LAB_TOOL_ID="$(curl -sS -H "Authorization: Bearer $GHOSTAI_API_KEY" \
"$GHOSTAI_BASE_URL/api/tooling/catalog?limit=5" | python3 -c "
import json, sys
data = json.load(sys.stdin)
items = data.get('items') or data.get('tools') or data
if isinstance(items, dict):
items = items.get('items') or items.get('results') or []
if not items:
sys.stderr.write('error: empty catalog\n')
sys.exit(1)
print(items[0]['tool_id'])
")"
echo "LAB_TOOL_ID=$LAB_TOOL_ID"Step 3: View Security Posture
Duration: 4 minutes
Goal: Pull aggregate posture for the last 24 hours and relate fields to enforcement.
3.1 Posture summary
bash
curl -sS -H "Authorization: Bearer $GHOSTAI_API_KEY" \
"$GHOSTAI_BASE_URL/api/tooling/posture?period=24h" | python3 -m json.toolReview typical fields (names may vary slightly by version):
posture_score— composite 0–100 or similarrisk_distribution— counts perlow/medium/high/criticalpii_stats_by_tool— detections, tokenizations, blocks pertool_idopen_approvals— count of pending governance itemschain_rule_matches— count of sequence violations or escalations
3.2 Compare with UI
Open Tool Security → Posture and confirm the API numbers align with charts (same period).
Expected outcome
GET /api/tooling/posture?period=24hreturns 200 with numeric summaries.- Per-tool PII stats correlate with tools you will touch in Step 4 and Step 10.
Step 4: Set a Tool Policy
Duration: 6 minutes
Goal: Update enforcement for LAB_TOOL_ID using structured policy fields.
4.1 Build policy JSON
Create policy_payload.json (adjust values to your org's standards):
json
{
"scan_depth": "deep",
"entity_types": ["EMAIL", "PHONE", "US_SSN", "CREDIT_CARD", "IP_ADDRESS"],
"blocked_types": ["CREDIT_CARD", "US_SSN"],
"action_on_detect": "tokenize",
"scan_direction": "both",
"rate_limit_per_user_hour": 120,
"data_residency": "us"
}Allowed values (conceptual):
scan_depth: typicallyshallow,standard,deepaction_on_detect:tokenize,block,warn,log_onlyscan_direction:inbound,outbound,bothdata_residency: region code your deployment supports (e.g.,us,eu) orinherit
4.2 Apply policy
bash
curl -sS -X PUT \
-H "Authorization: Bearer $GHOSTAI_API_KEY" \
-H "Content-Type: application/json" \
-d @policy_payload.json \
"$GHOSTAI_BASE_URL/api/tooling/policies/$LAB_TOOL_ID" | python3 -m json.toolExpected outcome
- HTTP 200 with updated policy object including
versionorupdated_at. - Audit log entry (visible in admin audit UI if available) for policy change.
4.3 Optional — AI generate-policy
If your tenant has assistant endpoints wired to Tool Security:
bash
curl -sS -X POST \
-H "Authorization: Bearer $GHOSTAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{"prompt":"Policy for MCP repo tool: block SSN and cards, tokenize email, warn on phone, both directions, 60 calls per user per hour"}' \
"$GHOSTAI_BASE_URL/api/tooling/policies/generate-draft" | python3 -m json.tool(If this route returns 404, skip — not all tiers expose generate-policy.)
Step 5: Test Policy Simulation
Duration: 4 minutes
Goal: Validate a proposed policy without committing it.
5.1 Simulation request
Build the JSON body with your LAB_TOOL_ID inline (avoids editing a file for the tool id):
bash
curl -sS -X POST \
-H "Authorization: Bearer $GHOSTAI_API_KEY" \
-H "Content-Type: application/json" \
-d "$(python3 -c "
import json, os
print(json.dumps({
\"tool_id\": os.environ[\"LAB_TOOL_ID\"],
\"proposed_policy\": {
\"scan_depth\": \"deep\",
\"entity_types\": [\"EMAIL\", \"US_SSN\"],
\"blocked_types\": [\"US_SSN\"],
\"action_on_detect\": \"block\",
\"scan_direction\": \"inbound\",
\"rate_limit_per_user_hour\": 30,
\"data_residency\": \"us\"
},
\"sample_payloads\": [
{\"direction\": \"inbound\", \"content\": \"Patient MRN 12345 — follow up at patient@example.com\"},
{\"direction\": \"outbound\", \"content\": \"SSN: 078-05-1120 for verification\"}
]
}))
")" \
"$GHOSTAI_BASE_URL/api/tooling/simulate" | python3 -m json.toolExpected outcome
- Response lists per sample outcomes: detections, actions (
block,tokenize, etc.), and entity hits. - Inbound-only
scan_directionshould not inspect outbound samples (or should reportskipped/not_scannedfor outbound — behavior depends on version; note what you observe for your runbook).
Step 6: Approval Workflow
Duration: 6 minutes
Goal: List pending approvals and resolve one approval record.
6.1 List pending approvals and capture tool IDs
bash
curl -sS -H "Authorization: Bearer $GHOSTAI_API_KEY" \
"$GHOSTAI_BASE_URL/api/tooling/approvals/pending" | python3 -m json.toolCapture the first pending tool_id for approval, and the second for denial, in one fetch (deny step is skipped automatically if only one row exists):
bash
eval "$(curl -sS -H "Authorization: Bearer $GHOSTAI_API_KEY" \
"$GHOSTAI_BASE_URL/api/tooling/approvals/pending" | python3 -c "
import json, sys, shlex
data = json.load(sys.stdin)
items = data.get('items') or data.get('approvals') or data
if isinstance(items, dict):
items = items.get('pending') or items.get('results') or []
if not items:
sys.stderr.write('No pending approvals: register tools as a non-admin or lower auto-approval.\n')
sys.exit(1)
print('export APPROVAL_TOOL_ID=' + shlex.quote(items[0]['tool_id']))
if len(items) > 1:
print('export DENY_TOOL_ID=' + shlex.quote(items[1]['tool_id']))
else:
print('unset DENY_TOOL_ID')
")"
echo "APPROVAL_TOOL_ID=$APPROVAL_TOOL_ID"
echo "DENY_TOOL_ID=${DENY_TOOL_ID:-}"6.2 Approve with expiry
bash
curl -sS -X POST \
-H "Authorization: Bearer $GHOSTAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"decision": "approved",
"expires_at": "2026-06-30T23:59:59Z",
"notes": "Lab approval — MCP staging server for engineering"
}' \
"$GHOSTAI_BASE_URL/api/tooling/approvals/$APPROVAL_TOOL_ID" | python3 -m json.tool6.3 Deny with reason (second tool)
Run only when DENY_TOOL_ID was set in §6.1:
bash
if [ -n "${DENY_TOOL_ID:-}" ]; then
curl -sS -X POST \
-H "Authorization: Bearer $GHOSTAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"decision": "denied",
"reason": "Upstream vendor cannot meet BAA / PHI routing requirements"
}' \
"$GHOSTAI_BASE_URL/api/tooling/approvals/$DENY_TOOL_ID" | python3 -m json.tool
else
echo "Only one pending item: submit a second tool request, re-run §6.1, then deny the remaining pending tool."
fiExpected outcome
- Approved tool shows
status: approvedandexpires_atin catalog or approvals detail. - Denied tool shows
status: deniedwithreasonpreserved for audit.
Step 7: Chain Rules
Duration: 7 minutes
Goal: Author a sequence rule and evaluate it against a session.
7.1 Create or update chain rules
Create chain_rules_payload.json:
json
{
"rules": [
{
"id": "lab-block-read-then-exfil",
"name": "Block read-class then external publish pattern",
"trigger_sequence": ["read_*", "*slack*", "external_*"],
"action": "block",
"enabled": true
},
{
"id": "lab-escalate-high-risk-chain",
"name": "Escalate DB access followed by email send",
"trigger_sequence": ["*database*", "*email*"],
"action": "escalate",
"enabled": true
}
]
}bash
curl -sS -X PUT \
-H "Authorization: Bearer $GHOSTAI_API_KEY" \
-H "Content-Type: application/json" \
-d @chain_rules_payload.json \
"$GHOSTAI_BASE_URL/api/tooling/chain-rules" | python3 -m json.toolSupported actions (conceptual): block, escalate, deep_scan, warn, log.
7.2 Evaluate against a session
Resolve a session_id from enforcement events (falls back to a short note if none exist yet). If this prints the “No session_id” message, complete Step 9 to generate traffic, then run this subsection again:
bash
export SESSION_ID="$(curl -sS -H "Authorization: Bearer $GHOSTAI_API_KEY" \
"$GHOSTAI_BASE_URL/api/tooling/enforcement/events?limit=20" | python3 -c "
import json, sys
data = json.load(sys.stdin)
rows = data.get('items') or data.get('events') or data
if isinstance(rows, dict):
rows = rows.get('results') or []
for row in rows:
sid = row.get('session_id')
if sid:
print(sid)
break
")"
if [ -z "$SESSION_ID" ]; then
echo "No session_id in recent events: run a short chat with tool calls, then re-run this block."
else
curl -sS -H "Authorization: Bearer $GHOSTAI_API_KEY" \
"$GHOSTAI_BASE_URL/api/tooling/chain-rules/evaluate/$SESSION_ID" | python3 -m json.tool
fiExpected outcome
PUT /api/tooling/chain-rulesreturns the stored ruleset version.GET /api/tooling/chain-rules/evaluate/{session_id}returns matched rules with the sequence that fired, or an emptymatchesarray if the session order does not satisfy anytrigger_sequence.
7.3 Optional — UI trace
In Tool Security → Sessions / Chain trace, open the same session_id and confirm the visual order matches API evaluation input.
Step 8: Supply Chain Analysis
Duration: 6 minutes
Goal: Inspect MCP and integration supply chain signals, blast radius, and shadow tools.
8.1 Supply chain report
bash
curl -sS -H "Authorization: Bearer $GHOSTAI_API_KEY" \
"$GHOSTAI_BASE_URL/api/tooling/supply-chain" | python3 -m json.toolInspect rows for:
- Transport: TLS version, cipher policy, pinning status
- Authentication: OAuth scopes, API key rotation hints, mTLS
- Schema drift:
fingerprint_currentvsfingerprint_last_approved - Behavioral flags: anomalous arguments, error rate spikes
8.2 Blast radius
bash
curl -sS -H "Authorization: Bearer $GHOSTAI_API_KEY" \
"$GHOSTAI_BASE_URL/api/tooling/supply-chain/blast-radius?tool_id=$LAB_TOOL_ID" | python3 -m json.toolConfirm the response enumerates dependent tools, affected data classifications, and approximate user reach (or equivalent metrics).
8.3 Shadow tools
bash
curl -sS -H "Authorization: Bearer $GHOSTAI_API_KEY" \
"$GHOSTAI_BASE_URL/api/tooling/supply-chain/shadow-tools" | python3 -m json.toolExpected outcome
- Supply chain endpoint returns one row per upstream MCP server or critical integration.
- Blast radius response is non-empty for well-connected tools.
- Shadow tools lists entries observed in traffic but not yet approved in catalog.
Step 9: Enforcement Feed
Duration: 5 minutes
Goal: Pull recent enforcement decisions and optional event stream metadata.
9.1 Enforcement summary or feed index
bash
curl -sS -H "Authorization: Bearer $GHOSTAI_API_KEY" \
"$GHOSTAI_BASE_URL/api/tooling/enforcement?limit=50" | python3 -m json.tool9.2 Enforcement events (detailed)
bash
curl -sS -H "Authorization: Bearer $GHOSTAI_API_KEY" \
"$GHOSTAI_BASE_URL/api/tooling/enforcement/events?limit=100" | python3 -m json.toolInspect events for:
tool_id,user_id,session_idaction—tokenize,block,warn,log_onlyrule_id— policy or chain rule identifier when applicabletimestamp— ordering for incident timelines
9.3 Generate traffic (if empty)
Invoke the lab tool in chat with synthetic PII that your policy should catch, then re-run the events query.
Expected outcome
- At least one event appears after controlled test traffic.
- Chain rule matches show
rule_idmatching Step 7 identifiers when sequences fire.
Step 10: Compliance Report and CSV Export
Duration: 5 minutes
Goal: Generate HIPAA, SOC 2, and PCI-DSS oriented evidence and download CSV.
10.1 Compliance report
bash
curl -sS -H "Authorization: Bearer $GHOSTAI_API_KEY" \
"$GHOSTAI_BASE_URL/api/tooling/compliance/report?frameworks=hipaa,soc2,pci-dss" | python3 -m json.toolReview sections for:
- Control mapping summaries
- Tool-level policy coverage
- Open approvals and exceptions
- Enforcement statistics for the reporting window
10.2 Export CSV
bash
curl -sS -o tool_security_compliance.csv \
-H "Authorization: Bearer $GHOSTAI_API_KEY" \
"$GHOSTAI_BASE_URL/api/tooling/compliance/report?frameworks=hipaa,soc2,pci-dss&format=csv"
head -n 20 tool_security_compliance.csvExpected outcome
- JSON report returns 200 with framework-keyed objects.
- CSV file contains headers such as
tool_id,category,risk_level,policy_version,approval_status,pii_detections,blocks,chain_hits,supply_chain_status(exact columns depend on release).
10.3 AI-assisted review (optional)
If enabled, call explain-anomaly or review endpoints your tenant exposes for Tool Security to narrate spikes in the CSV versus prior period. Skip if not available.
Troubleshooting
401 Unauthorized
Symptom: All curl calls return 401.
Fix: Export a valid GHOSTAI_API_KEY. Confirm the Authorization: Bearer header is present and the key is not expired.
403 Forbidden on /api/tooling/*
Symptom: 403 on catalog or policy PUT.
Fix: Use an admin or security-role key. Enable Tool Security feature flags for the tenant. Confirm subscription tier includes the endpoint family (chain rules and compliance are often tier-gated).
404 on specific routes
Symptom: simulate, generate-draft, or compliance/report returns 404.
Fix: Your deployment may run an older API revision. Compare with current API documentation or internal release notes. Fall back to UI-only steps for missing routes.
Empty approvals/pending
Symptom: Step 6 has nothing to approve.
Fix: Submit a tool registration as a non-admin, or lower auto-approval threshold in tenant settings. Some sandboxes ship with pre-seeded pending items — re-run tenant seed script if documented.
Chain evaluation always empty
Symptom: chain-rules/evaluate/{session_id} returns no matches.
Fix: Verify session_id has multiple tool calls in order. Wildcards in trigger_sequence must match your engine's pattern syntax. Try literal tool IDs first, then reintroduce wildcards.
Supply chain drift for all MCP servers
Symptom: Every server shows drift after upgrade.
Fix: Re-baseline fingerprints in the UI or via admin API if provided. Ensure each logical MCP endpoint maps to one backend schema.
CSV export empty or truncated
Symptom: CSV has headers only.
Fix: Widen the reporting window if supported (&period=30d). Ensure at least one tool has policy and enforcement history.
Rate limiting during lab
Symptom: HTTP 429 on tooling endpoints.
Fix: Space out curl calls or raise admin rate limits in development.
Lab Summary
| Step | Topic | Duration |
|---|---|---|
| 1 | Access Tool Security Center (UI) | 3 min |
| 2 | Browse catalog and marketplace | 4 min |
| 3 | View security posture (period=24h) | 4 min |
| 4 | Set tool policy (PUT .../policies/{tool_id}) | 6 min |
| 5 | Policy simulation (POST .../simulate) | 4 min |
| 6 | Approval workflow (pending + POST decision) | 6 min |
| 7 | Chain rules (PUT + GET .../evaluate/{session_id}) | 7 min |
| 8 | Supply chain, blast radius, shadow tools | 6 min |
| 9 | Enforcement feed and events | 5 min |
| 10 | Compliance report + CSV export | 5 min |
| Total | ~50 min |
For a customer-facing narrative and shorter UI path, see the Tool Security Demo Script.