Skip to content

Intelligence Profiles Full Lab ​

Nine steps. About forty minutes. You will drive every major Intelligence Profiles API: company CRUD, auto-research, organization terms (single and bulk), personal profile and terms, merged effective lists, live detection preview, and verification of org-side audit records.


Prerequisites ​

  • [ ] GhostAI backend reachable (examples use http://localhost:8000)
  • [ ] Bearer token for a user in an organization: export GHOSTAI_API_KEY='gst_...' (or paste JWT from the app)
  • [ ] Organization ID for that user: export GHOSTAI_ORG_ID='<your-organization-id>' (from Admin UI or GET user/org metadata)
  • [ ] Account has Admin or Owner for org write operations
  • [ ] curl and python3 -m json.tool (optional, for pretty printing)
  • [ ] For Step 9: access to the deployment database or SIEM sink that receives settings.update events (optional but recommended)

Health check ​

bash
curl -s http://localhost:8000/health | python3 -m json.tool

Step 1: Get and update the company Intelligence Profile ​

Duration: 4 minutes
Goal: Read and write company_description, industry_details, products_services, data_classification_tiers, and sensitivity_floor.

1.1 GET company profile ​

bash
curl -s -X GET \
  -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "http://localhost:8000/api/intelligence-profiles/company/${GHOSTAI_ORG_ID}" \
  | python3 -m json.tool

Expected shape includes: organization_id, company_description, industry_details, products_services, data_classification_tiers, sensitivity_floor (relaxed | balanced | paranoid), profile_completeness, timestamps.

1.2 PUT company profile ​

bash
curl -s -X PUT \
  -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  "http://localhost:8000/api/intelligence-profiles/company/${GHOSTAI_ORG_ID}" \
  -d '{
    "company_description": "Northwind Analytics provides regulated healthcare analytics to hospital networks. Customer payloads routinely contain PHI and contractually protected operational metrics.",
    "industry_details": {
      "sector": "Healthcare technology",
      "regulated": true,
      "frameworks": ["HIPAA", "SOC 2"]
    },
    "products_services": [
      {"name": "Clinical Insights Suite", "description": "Cohort analytics and outcomes reporting"},
      {"name": "Revenue Integrity Module", "description": "Claims-adjacent analytics; handles payer identifiers"}
    ],
    "data_classification_tiers": [
      {"label": "Public", "examples": ["marketing site copy"]},
      {"label": "Internal", "examples": ["roadmaps", "slack summaries"]},
      {"label": "Restricted", "examples": ["PHI-adjacent metrics", "customer identities"]}
    ],
    "sensitivity_floor": "balanced"
  }' | python3 -m json.tool

1.3 Verify ​

Confirm the response echoes your fields and profile_completeness is greater than zero.


Step 2: Run auto-research (suggestions only) ​

Duration: 5 minutes
Goal: Call auto-research with company_name and domain; inspect suggested profile fields and terms. Nothing is persisted until you apply Step 1 / Step 3 yourself.

bash
curl -s -X POST \
  -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  "http://localhost:8000/api/intelligence-profiles/company/${GHOSTAI_ORG_ID}/auto-research" \
  -d '{
    "company_name": "Northwind Analytics",
    "domain": "northwind-analytics.example",
    "website_url": "https://northwind-analytics.example"
  }' | python3 -m json.tool

Inspect:

  • company_description, industry_details, products_services, data_classification_tiers
  • suggested_sensitivity_floor
  • suggested_sensitive_terms / suggested_safe_terms (each with term, term_type, category, sensitivity_action, etc.)
  • sources, confidence, field_confidence, warnings

Optional exercise: Copy one paragraph and two terms you trust into Step 1 and Step 3.

Note: The service rate-limits repeat research per key; if you receive a throttle message, wait two minutes or change the target company string.


Step 3: Manage organization terms (single term) ​

Duration: 4 minutes
Goal: Create sensitive and safe terms via POST /api/intelligence-profiles/company/{org_id}/terms.

3.1 Sensitive term (custom detection) ​

bash
curl -s -X POST \
  -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  "http://localhost:8000/api/intelligence-profiles/company/${GHOSTAI_ORG_ID}/terms" \
  -d '{
    "term": "NighthawkProtocol",
    "term_type": "sensitive",
    "category": "project",
    "sensitivity_action": "always_protect",
    "context_notes": "Internal pilot codename; must be tokenized in all customer-facing threads."
  }' | python3 -m json.tool

Save the returned id if you want to delete it later.

3.2 Safe term (suppression) ​

bash
curl -s -X POST \
  -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  "http://localhost:8000/api/intelligence-profiles/company/${GHOSTAI_ORG_ID}/terms" \
  -d '{
    "term": "Northwind Analytics",
    "term_type": "safe",
    "category": "brand",
    "sensitivity_action": "never_protect",
    "context_notes": "Public legal name; allow in clear text when detected as org-like span."
  }' | python3 -m json.tool

3.3 LIST terms ​

bash
# All terms
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "http://localhost:8000/api/intelligence-profiles/company/${GHOSTAI_ORG_ID}/terms" \
  | python3 -m json.tool

# Only sensitive
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "http://localhost:8000/api/intelligence-profiles/company/${GHOSTAI_ORG_ID}/terms?term_type=sensitive" \
  | python3 -m json.tool

# Only safe
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "http://localhost:8000/api/intelligence-profiles/company/${GHOSTAI_ORG_ID}/terms?term_type=safe" \
  | python3 -m json.tool

Step 4: Manage the personal Intelligence Profile ​

Duration: 3 minutes
Goal: GET / PUT personal fields: job_title, department, work_description, data_types_handled, sensitivity_level.

4.1 GET ​

bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  http://localhost:8000/api/intelligence-profiles/personal \
  | python3 -m json.tool

4.2 PUT ​

bash
curl -s -X PUT \
  -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  http://localhost:8000/api/intelligence-profiles/personal \
  -d '{
    "job_title": "Senior Compliance Analyst",
    "department": "GRC",
    "work_description": "Reviews AI-assisted workflows that reference patient cohorts and payer data.",
    "data_types_handled": ["PHI", "payer identifiers", "financial summaries"],
    "sensitivity_level": "paranoid"
  }' | python3 -m json.tool

Step 5: Add personal terms ​

Duration: 4 minutes
Goal: POST /api/intelligence-profiles/personal/terms and list/delete.

5.1 Personal sensitive term ​

bash
curl -s -X POST \
  -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  http://localhost:8000/api/intelligence-profiles/personal/terms \
  -d '{
    "term": "Dr. Elara Voss",
    "term_type": "sensitive",
    "category": "clinician",
    "sensitivity_action": "context_dependent",
    "context_notes": "Fictional clinician for lab narratives."
  }' | python3 -m json.tool

5.2 Conflict check (409) ​

Try adding a personal safe term that matches an org sensitive term (use the exact string from Step 3.1):

bash
curl -s -w "\nHTTP_STATUS:%{http_code}\n" -X POST \
  -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  http://localhost:8000/api/intelligence-profiles/personal/terms \
  -d '{
    "term": "NighthawkProtocol",
    "term_type": "safe",
    "category": "project",
    "sensitivity_action": "never_protect"
  }'

Expected: HTTP 409 with detail similar to:
Cannot mark this term as safe — your organization has it registered as sensitive.

5.3 List personal terms ​

bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  http://localhost:8000/api/intelligence-profiles/personal/terms \
  | python3 -m json.tool

5.4 Delete a personal term (optional) ​

bash
export PERSONAL_TERM_ID='<id-from-list-response>'

curl -s -o /dev/null -w "%{http_code}\n" -X DELETE \
  -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "http://localhost:8000/api/intelligence-profiles/personal/terms/${PERSONAL_TERM_ID}"

Expected: 204


Step 6: Test PII detection impact (preview) ​

Duration: 5 minutes
Goal: Use authenticated POST /api/detect so the user’s merged Intelligence Profile terms apply.

6.1 Custom sensitive term surfaces as CUSTOM_SENSITIVE ​

bash
curl -s -X POST \
  -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  http://localhost:8000/api/detect \
  -d '{"text": "Pilot gate: NighthawkProtocol cleared for west region only."}' \
  | python3 -m json.tool

In entities, locate an item with:

  • entity_type: CUSTOM_SENSITIVE
  • detector: intelligence_profile (same concept as detector_source in the pipeline)
  • score / confidence: 0.95

Matching uses word boundaries on the term (case-insensitive).

6.2 Safe term suppression ​

Use copy where a detector emits a span that exactly equals your safe string (case-insensitive). Example after Step 3.2:

bash
curl -s -X POST \
  -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  http://localhost:8000/api/detect \
  -d '{"text": "Vendor of record: Northwind Analytics"}' \
  | python3 -m json.tool

If an entity’s original_text would exactly match Northwind Analytics, it should be absent after safe-term suppression. If your model still splits the span differently, shorten the sentence so the detected span equals the safe term string exactly.


Step 7: View merged terms ​

Duration: 2 minutes
Goal: Effective sensitive and safe strings for the current user.

bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  http://localhost:8000/api/intelligence-profiles/merged-terms \
  | python3 -m json.tool

Expected:

json
{
  "sensitive": ["...", "NighthawkProtocol", "Dr. Elara Voss"],
  "safe": ["...", "Northwind Analytics"]
}

Arrays are deduplicated merged views of organization plus personal registries.


Step 8: Bulk import organization terms ​

Duration: 5 minutes
Goal: POST /api/intelligence-profiles/company/{org_id}/terms/bulk with 1–500 entries.

bash
curl -s -X POST \
  -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  "http://localhost:8000/api/intelligence-profiles/company/${GHOSTAI_ORG_ID}/terms/bulk" \
  -d '{
    "terms": [
      {
        "term": "NW-PI-2048",
        "term_type": "sensitive",
        "category": "internal_id",
        "sensitivity_action": "always_protect",
        "context_notes": "Lab-only internal project identifier"
      },
      {
        "term": "GhostAI",
        "term_type": "safe",
        "category": "vendor",
        "sensitivity_action": "never_protect",
        "context_notes": "Public vendor name"
      },
      {
        "term": "ChartStream API",
        "term_type": "sensitive",
        "category": "product",
        "sensitivity_action": "context_dependent",
        "context_notes": "Unreleased integration surface"
      }
    ]
  }' | python3 -m json.tool

Response:

  • inserted: count written
  • total: number submitted

Re-run Step 7 to see new strings in sensitive / safe.


Step 9: Verify the audit trail ​

Duration: 5 minutes
Goal: Confirm org-side operations logged with action settings.update and details.section in:

  • company_profile — company PUT
  • auto_research — auto-research POST (details include company_name, domain)
  • org_terms — single term POST or DELETE
  • org_terms_bulk — bulk POST (inserted, total)

Entries are stored in user_audit_logs (tenant-scoped) and emitted toward SIEM when configured.

9.1 SQLite (common local dev) ​

Point GHOSTAI_DB at your SQLite file (for example the path configured in .env for DATABASE_URL if it uses sqlite:///...).

bash
export GHOSTAI_DB="/absolute/path/to/your/ghostai.sqlite"

sqlite3 "$GHOSTAI_DB" "
SELECT timestamp, action, resource_type, resource_id, details
FROM user_audit_logs
WHERE organization_id = '${GHOSTAI_ORG_ID}'
  AND action = 'settings.update'
  AND (
    details LIKE '%\"section\": \"company_profile\"%'
    OR details LIKE '%\"section\": \"auto_research\"%'
    OR details LIKE '%\"section\": \"org_terms\"%'
    OR details LIKE '%\"section\": \"org_terms_bulk\"%'
  )
ORDER BY timestamp DESC
LIMIT 20;
"

9.2 PostgreSQL (adjust connection) ​

sql
SELECT timestamp, action, resource_type, resource_id, details
FROM user_audit_logs
WHERE organization_id = $1
  AND action = 'settings.update'
  AND (
    details::text LIKE '%"section": "company_profile"%'
    OR details::text LIKE '%"section": "auto_research"%'
    OR details::text LIKE '%"section": "org_terms"%'
    OR details::text LIKE '%"section": "org_terms_bulk"%'
  )
ORDER BY timestamp DESC
LIMIT 20;

9.3 What you should see ​

  • Rows after Step 1 (company_profile), Step 2 (auto_research), Steps 3 and 8 (org_terms / org_terms_bulk), and term deletes (org_terms with "action": "delete").

Personal profile PUT and personal term POST are not written through these same details.section values in the current implementation; call that out if auditors ask about user-level changes.


Appendix: Reference paths ​

MethodPath
GET/api/intelligence-profiles/company/{org_id}
PUT/api/intelligence-profiles/company/{org_id}
POST/api/intelligence-profiles/company/{org_id}/auto-research
GET/api/intelligence-profiles/company/{org_id}/terms
POST/api/intelligence-profiles/company/{org_id}/terms
POST/api/intelligence-profiles/company/{org_id}/terms/bulk
DELETE/api/intelligence-profiles/company/{org_id}/terms/{term_id}
GET/api/intelligence-profiles/personal
PUT/api/intelligence-profiles/personal
DELETE/api/intelligence-profiles/personal
POST/api/intelligence-profiles/personal/dismiss-onboarding
GET/api/intelligence-profiles/personal/terms
POST/api/intelligence-profiles/personal/terms
DELETE/api/intelligence-profiles/personal/terms/{term_id}
GET/api/intelligence-profiles/merged-terms
POST/api/detect (preview; uses merged profile when authenticated)

Completion checklist ​

  • [ ] Company profile updated and read back
  • [ ] Auto-research response inspected; optional manual apply
  • [ ] Org sensitive and safe terms created and listed
  • [ ] Personal profile updated
  • [ ] Personal term added; 409 conflict demonstrated against org sensitive term
  • [ ] /api/detect shows CUSTOM_SENSITIVE / intelligence_profile / 0.95; safe suppression observed where spans align
  • [ ] merged-terms reflects org + personal
  • [ ] Bulk import returned inserted / total
  • [ ] Audit query shows expected details.section values for org operations

SOC 2 Type II Certified | BAA Available | HIPAA-Ready