Skip to content

Memory + Vector Explorer Full Lab ​

Ten steps. Forty-five minutes. From empty config to embeddings on a map.

This lab exercises GhostAI memory end-to-end using the HTTP API. You will verify configuration, create and update memories, trigger auto-extraction from chat, run semantic recall, manage org knowledge, structure group/tag metadata, explore vector projections and similarity edges, import and export JSON, and tune recall behavior. All routes live under /api/memory and /api/memory/explorer.


Prerequisites ​

Before starting, ensure you have:

  • [ ] GhostAI deployment running (backend on port 8000, or your deployed URL)
  • [ ] API key with a user that has memory enabled on their tier (memory_limit > 0 when billing applies)
  • [ ] Feature flag memory enabled
  • [ ] curl and python3 -m json.tool (or httpie) for pretty-printed responses
  • [ ] For Steps 6–7: org admin access and flags memory_org_knowledge and memory_groups as needed
  • [ ] For Step 9: feature memory_api enabled for export/import
  • [ ] Optional: second user account to observe tenant-scoped recall as a non-admin

Set your key for all examples:

bash
export GHOSTAI_API_KEY="gst_your_key"
export BASE_URL="http://localhost:8000"

Verify prerequisites ​

bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "$BASE_URL/health" | python3 -m json.tool

Step 1: Verify Memory Is Enabled ​

Duration: 3 minutes
Goal: Confirm resolved configuration before writing data.

1.1 Fetch memory config ​

bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "$BASE_URL/api/memory/config" | python3 -m json.tool

1.2 Expected shape (MemoryConfigResponse) ​

You should see fields including:

FieldMeaning
memory_enabledMaster switch for recall/extraction features
auto_extractWhether chat triggers automatic memory extraction
max_memoriesTier cap
used_memoriesCurrent user-scope count
retention_daysPolicy retention window
allowed_categoriesCategories permitted for extraction/storage
memory_recall_thresholdMinimum semantic similarity for recall (0.0–1.0)
can_pinWhether pinning is allowed
org_lockedOrg policy override when true
tier_nameResolved subscription tier

Expected: "memory_enabled": true. If false, enable via Step 10 or org policy, then repeat.


Step 2: Create Memories Manually ​

Duration: 5 minutes
Goal: POST /api/memory with varied categories and optional scope fields.

2.1 Create a fact ​

bash
curl -s -X POST -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "content": "Primary data store for the analytics pilot is PostgreSQL 15 in eu-west-1.",
    "category": "fact",
    "is_pinned": false,
    "tags": ["pilot", "infra"]
  }' \
  "$BASE_URL/api/memory" | python3 -m json.tool

2.2 Create preference, instruction, context, and skill-style entries ​

Repeat POST /api/memory with bodies such as:

json
{
  "content": "Prefer short answers with a one-line summary first.",
  "category": "preference",
  "is_pinned": true,
  "tags": ["style"]
}
json
{
  "content": "When discussing Northwind, use the term Customer Logistics Portal for the external portal name.",
  "category": "instruction",
  "tags": ["terminology"]
}
json
{
  "content": "Q1 roadmap workshop is scheduled for April 14; focus on SSO and SCIM.",
  "category": "context",
  "tags": ["roadmap"]
}
json
{
  "content": "Runbook: for demo tenant reset, disable memory_api, wipe user memories, re-enable.",
  "category": "skill",
  "tags": ["runbook", "demo"]
}

category is a string; common user-facing values include fact, preference, instruction, context, and custom values such as skill.

You may also send scope and scope_id: cloud, tenant, group, user, agent, conversation, or subject, plus the identifier required for that scope (for example a group id when scope is group). Omitting both defaults to user scoped to the authenticated user.

2.3 Expected response (MemoryResponse) ​

Each successful POST returns JSON including:

  • id, scope, scope_id, content, category, source_agent, confidence, is_mandatory, is_pinned, created_at

The handler may return an empty tags array on create even when tags were sent; confirm assignments with GET /api/memory.

2.4 List memories ​

bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "$BASE_URL/api/memory?limit=20&offset=0" | python3 -m json.tool

Expected (MemoryListResponse): { "memories": [ ... ], "total": N, "offset": 0, "limit": 20 }

Optional query params: category, pinned_only=true.


Step 3: Auto-Extraction via Chat ​

Duration: 5 minutes
Goal: Confirm new memories appear after conversational messages.

3.1 Ensure auto-extract is on ​

bash
curl -s -X PUT -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"auto_extract": true}' \
  "$BASE_URL/api/memory/config" | python3 -m json.tool

3.2 Send chat messages ​

In the GhostAI UI, open a new thread and send:

I'm Alex; I work on compliance tooling. We never store raw national IDs in prompts—use placeholders. I want checklist-style answers for audit questions.

Wait for the assistant reply.

3.3 Verify extraction ​

bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "$BASE_URL/api/memory?limit=50&offset=0" | python3 -m json.tool

Expected: New memories with source_agent reflecting chat extraction (e.g. chat pipeline), categories allowed by allowed_categories, and sanitized content.

3.4 Optional: conversation-scoped listing ​

If you have a session id from the UI or API, list conversation memories:

bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "$BASE_URL/api/memory/session/{session_id}" | python3 -m json.tool

Expected: { "memories": [...], "total": N } for scope conversation tied to that session.


Step 4: Recall Across Scopes ​

Duration: 4 minutes
Goal: GET /api/memory/recall?q=... returns merged, ranked memories.

4.1 Run recall ​

bash
curl -s -G -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  --data-urlencode "q=EU postgres analytics pilot" \
  "$BASE_URL/api/memory/recall" | python3 -m json.tool

4.2 Expected response (MemoryRecallResponse) ​

json
{
  "memories": [ /* MemoryResponse items, ordered after ranking */ ],
  "scope_counts": { "user": 2, "tenant": 1 },
  "total_recalled": 3,
  "token_budget_used": 0
}

scope_counts keys are scope names; values are counts before final token trim. token_budget_used reflects how much of the recall budget was consumed.

4.3 Tuning breadth of recall ​

Match count and strictness are driven mainly by memory_recall_threshold in user config (Step 10) and the internal merge/token budget inside the service. Check /docs for the latest query parameters on this route.

4.4 Talking point ​

Recall merges cloud, tenant, group, user, agent, conversation, subject (and tag-associated pools where applicable), deduplicates, ranks by mandatory → scope priority → confidence, then trims to the internal token budget.


Step 5: Pin and Manage Memories ​

Duration: 4 minutes
Goal: Pin, update, and delete a specific memory by id.

Pick a MEMORY_ID from Step 2 or 3.

5.1 Pin ​

bash
curl -s -X PUT -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "$BASE_URL/api/memory/$MEMORY_ID/pin?pinned=true" | python3 -m json.tool

Expected: { "status": "pinned", "id": "<MEMORY_ID>" }

Unpin:

bash
curl -s -X PUT -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "$BASE_URL/api/memory/$MEMORY_ID/pin?pinned=false" | python3 -m json.tool

5.2 Update content and tags ​

bash
curl -s -X PUT -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "content": "Primary data store for the analytics pilot is PostgreSQL 16 in eu-west-1 (upgraded).",
    "category": "fact",
    "tags": ["pilot", "infra", "upgraded"]
  }' \
  "$BASE_URL/api/memory/$MEMORY_ID" | python3 -m json.tool

Expected: { "status": "updated", "id": "<MEMORY_ID>" }

5.3 Delete ​

bash
curl -s -X DELETE -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "$BASE_URL/api/memory/$MEMORY_ID" | python3 -m json.tool

Expected: { "status": "deleted", "id": "<MEMORY_ID>" }


Step 6: Org Knowledge Admin ​

Duration: 5 minutes
Goal: Tenant-scoped memories via admin routes and usage stats.

Requires org admin auth and memory_org_knowledge.

6.1 Create org knowledge ​

bash
curl -s -X POST -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "content": "Northwind public trademark is NORTHWIND in all caps in legal footers.",
    "category": "org_knowledge",
    "is_pinned": true,
    "tags": ["legal", "brand"]
  }' \
  "$BASE_URL/api/memory/admin" | python3 -m json.tool

Expected: MemoryResponse with "scope": "tenant" and scope_id equal to your organization id.

6.2 List org knowledge ​

bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "$BASE_URL/api/memory/admin?limit=20&offset=0" | python3 -m json.tool

Expected: MemoryListResponse shape (memories, total, offset, limit).

6.3 Org stats ​

bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "$BASE_URL/api/memory/admin/stats" | python3 -m json.tool

Expected (MemoryUsageStats): total_memories, by_scope, by_category, storage_bytes.

6.4 Delete org memory (optional) ​

bash
curl -s -X DELETE -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "$BASE_URL/api/memory/admin/{org_memory_id}" | python3 -m json.tool

Expected: { "status": "deleted", "id": "..." }


Step 7: Groups and Tags ​

Duration: 5 minutes
Goal: Create a group, add a group memory, create tags, assign tags to a memory.

Requires memory_groups and a resolved organization_id.

7.1 Create a group ​

bash
curl -s -X POST -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Lab — Solutions Engineering",
    "description": "Shared SE snippets and demo facts",
    "group_type": "team"
  }' \
  "$BASE_URL/api/memory/groups" | python3 -m json.tool

Expected (GroupResponse): id, organization_id, name, member_count, memory_count, created_at, etc. Save GROUP_ID.

7.2 Add a group memory ​

bash
curl -s -X POST -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "content": "Standard demo URL path for the lab tenant is /labs/ghostai-memory.",
    "category": "org_knowledge",
    "tags": ["demo"]
  }' \
  "$BASE_URL/api/memory/groups/$GROUP_ID/memories" | python3 -m json.tool

Expected: MemoryResponse with "scope": "group", scope_id = GROUP_ID.

7.3 Create tags ​

bash
curl -s -X POST -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "name": "lab-demo", "is_shared": true }' \
  "$BASE_URL/api/memory/tags" | python3 -m json.tool

Expected (TagResponse): id, name, is_shared, memory_count, created_at. Save TAG_ID.

7.4 Assign tag to a user memory ​

Use a USER_MEMORY_ID from Step 2:

bash
curl -s -X POST -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "$BASE_URL/api/memory/$USER_MEMORY_ID/tags/$TAG_ID" | python3 -m json.tool

Expected: { "status": "assigned" }

7.5 List tags ​

bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "$BASE_URL/api/memory/tags" | python3 -m json.tool

7.6 Remove tag (optional) ​

bash
curl -s -X DELETE -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "$BASE_URL/api/memory/$USER_MEMORY_ID/tags/$TAG_ID" | python3 -m json.tool

Expected: { "status": "removed" }


Step 8: Vector Explorer Projections ​

Duration: 6 minutes
Goal: 2D projections, neighbors, edges, and stats.

8.1 Projections ​

bash
curl -s -G -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  --data-urlencode "method=pca" \
  --data-urlencode "limit=500" \
  --data-urlencode "sources=memory,file_chunk,content_library" \
  "$BASE_URL/api/memory/explorer/projections" | python3 -m json.tool

Expected (ProjectionResponse):

  • points: array of { id, x, y, source, scope, category, content_preview, content, confidence, is_pinned, access_count, created_at, source_agent, tags }
  • total_embeddings, skipped, projection_method, computed_at, cached

Try method=tsne on a smaller limit if PCA succeeds but t-SNE is slow.

8.2 Neighbors ​

Pick a POINT_ID from points[].id:

bash
curl -s -G -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  --data-urlencode "k=8" \
  "$BASE_URL/api/memory/explorer/neighbors/$POINT_ID" | python3 -m json.tool

Expected (NeighborResponse): { "origin_id": "...", "neighbors": [ { "id", "source", "content_preview", "similarity", "scope", "category" }, ... ] }

Neighbors use cosine similarity on 384-dimensional embeddings (all-MiniLM-L6-v2).

8.3 Edges (graph overlay) ​

bash
curl -s -G -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  --data-urlencode "threshold=0.85" \
  --data-urlencode "max_edges=300" \
  "$BASE_URL/api/memory/explorer/edges" | python3 -m json.tool

Expected (EdgesResponse): { "edges": [ { "source", "target", "similarity" }, ... ], "threshold": 0.85, "total_pairs_checked": ... }

8.4 Stats ​

bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "$BASE_URL/api/memory/explorer/stats" | python3 -m json.tool

Expected (ExplorerStats): total_memories, memories_with_embeddings, by_scope, by_category, embedding_coverage_pct.

8.5 Optional query refinements ​

On projections, add scope=, category=, or refresh=true to bypass cache (see /docs).


Step 9: Export and Import ​

Duration: 4 minutes
Goal: Full user export JSON and bulk import with deduplication.

Requires memory_api.

9.1 Export ​

bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "$BASE_URL/api/memory/export" | python3 -m json.tool

Expected:

json
{
  "memories": [
    {
      "id": "...",
      "content": "...",
      "category": "fact",
      "created_at": "...",
      "is_pinned": false,
      "tags": ["a", "b"]
    }
  ],
  "count": 1,
  "exported_at": "2026-03-29T12:00:00+00:00"
}

9.2 Import ​

Save a file bulk.json:

json
{
  "memories": [
    {
      "content": "Imported onboarding fact: use JIRA project GHAI for security tickets.",
      "category": "fact",
      "is_pinned": false,
      "tags": ["onboarding"]
    },
    {
      "content": "Imported preference: weekly status summaries on Fridays.",
      "category": "preference",
      "is_pinned": true,
      "tags": ["ritual"]
    }
  ],
  "is_mandatory": false
}
bash
curl -s -X POST -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d @bulk.json \
  "$BASE_URL/api/memory/import" | python3 -m json.tool

Expected:

json
{
  "imported": 2,
  "skipped_duplicate": 0,
  "errors": []
}

Duplicate content (MD5 of normalized text) is skipped. Errors include { "index", "error" } entries (truncated in response if many).

9.3 Org bulk import (admin optional) ​

bash
curl -s -X POST -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d @bulk.json \
  "$BASE_URL/api/memory/admin/import" | python3 -m json.tool

Expected: { "status": "imported", "count": N } (org-scoped rows).


Step 10: Configure Memory Settings ​

Duration: 4 minutes
Goal: Tune PUT /api/memory/config for thresholds, categories, and toggles.

10.1 Update configuration ​

bash
curl -s -X PUT -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "memory_enabled": true,
    "auto_extract": true,
    "allowed_categories": ["fact", "preference", "instruction", "context", "skill"],
    "memory_recall_threshold": 0.55
  }' \
  "$BASE_URL/api/memory/config" | python3 -m json.tool

Expected: Full MemoryConfigResponse reflecting persisted settings.

10.2 Validate recall feel ​

Re-run:

bash
curl -s -G -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  --data-urlencode "q=demo tenant checklist" \
  "$BASE_URL/api/memory/recall" | python3 -m json.tool

Raise the threshold for stricter matches; lower it for broader surfacing.

10.3 UI cross-check ​

Open MemoryManager in the app: tabs My Memories, Org Knowledge, Groups, Tags, Explorer should reflect the same data you created here.


Troubleshooting ​

403 on any /api/memory route ​

Fix: Enable feature memory. Confirm API key maps to a user with memory allowed on plan.

POST /api/memory returns 429 with memory limit header ​

Fix: Delete old memories or upgrade tier; response may include X-Memory-Limit-Reached.

PUT /api/memory/config rejects threshold ​

Fix: memory_recall_threshold must be between 0 and 1.

Org admin routes forbidden ​

Fix: Use an org admin token. Enable memory_org_knowledge.

Groups/tags routes empty or 403 ​

Fix: Enable memory_groups. Ensure the user belongs to an organization.

Export/import 403 ​

Fix: Enable memory_api.

Explorer 503 / projection failed ​

Fix: Reduce limit on /api/memory/explorer/projections. Prefer method=pca for large sets. Ensure host has enough RAM.

t-SNE never returns ​

Fix: Use smaller limit or run PCA first; t-SNE is more expensive than PCA.

Recall returns empty memories ​

Fix: Lower memory_recall_threshold, add more diverse memories, or broaden the query string.


What's Next ​


Lab Summary ​

StepTopicDuration
1Verify memory (GET /api/memory/config)3 min
2Create memories manually (POST /api/memory)5 min
3Auto-extraction via chat5 min
4Recall across scopes (GET /api/memory/recall)4 min
5Pin, update, delete (PUT / DELETE /api/memory/...)4 min
6Org knowledge admin (POST/GET /api/memory/admin, stats)5 min
7Groups and tags5 min
8Vector Explorer (projections, neighbors, edges, stats)6 min
9Export / import4 min
10Configure settings (PUT /api/memory/config)4 min
Total~45 min

SOC 2 Type II Certified | BAA Available | HIPAA-Ready