Appearance
Data Pane Full Lab
Eight steps. About thirty-five minutes. From empty Dymium config to governed queries, insights, and a clean disconnect.
This lab exercises the Data Pane API in securechat/backend/app/routers/data.py. All routes use the prefix /api/data. Organization context is required for connection configuration and downstream calls: use an API key or session tied to a user who belongs to an organization.
PUT /api/data/connection/configrequires organization admin (require_admin).GET /api/data/connection/config, status, OAuth, sources, schema, query, insight, and disconnect use tenant context and optional user identity for per-user token caching.
Prerequisites
Before starting, ensure you have:
- [ ] GhostAI backend running (example:
http://localhost:8000) - [ ]
BASE_URLon the backend set correctly so OAuth redirect is{base_url}/api/data/connection/callback - [ ] A Dymium environment with OAuth client credentials and at least one data source
- [ ] Org admin API key for Step 1:
GHOSTAI_ADMIN_API_KEY(Bearer token for owner or admin role in the target organization) - [ ] The same or another user key for Steps 2–8 if you want to separate admin config from analyst usage:
GHOSTAI_API_KEY - [ ]
curlandpython3 -m json.tool(orjq)
Environment variables
bash
export GHOSTAI_BASE_URL="http://localhost:8000"
export GHOSTAI_ADMIN_API_KEY="gst_..." # org admin — Step 1
export GHOSTAI_API_KEY="gst_..." # analyst user — Steps 2–8 (can equal admin for a single-user lab)Verify you have organization context
If GET /api/data/connection/config returns "configured": false with empty platform_url after Step 1, confirm the token’s user is in an organization and you used the admin key for the PUT.
Step 1: Configure the Dymium connection (org admin)
Duration: 4 minutes
Goal: Persist platform_url, client_id, and client_secret for the organization. The secret is stored server-side and is never returned on GET.
Endpoint: PUT /api/data/connection/config
Replace the string values with your real Dymium tenant URL, OAuth client ID, and client secret. The example below uses a fictional lab tenant.
bash
curl -s -X PUT "$GHOSTAI_BASE_URL/api/data/connection/config" \
-H "Authorization: Bearer $GHOSTAI_ADMIN_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"platform_url": "https://dymium.lab.ghostai.example",
"client_id": "ghostai-data-pane-lab",
"client_secret": "dymium-oauth-secret-value-from-dymium-console"
}' | python3 -m json.toolExpected: 200 with DymiumConnectionConfigResponse:
platform_url— echoed from requestclient_id— echoed from requestconfigured— true when bothplatform_urlandclient_idare non-empty after save
Verify non-secret read:
bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
"$GHOSTAI_BASE_URL/api/data/connection/config" | python3 -m json.toolExpected: Same shape; no client_secret field.
Common errors
- 403 — not an org admin, or missing organization context on the token used for PUT.
Step 2: Check connection status
Duration: 3 minutes
Goal: Read ConnectionStatus before and after user OAuth.
Endpoint: GET /api/data/connection/status
bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
"$GHOSTAI_BASE_URL/api/data/connection/status" | python3 -m json.toolExpected fields
status—not_configured|disconnected|healthy|errormessage— human-readable detail (for example configuration hint whennot_configured)platform_version— when Dymium reports a version during health checkplatform_url— resolved base URL for the org’s Dymium integration
After Step 1, a user who has not completed OAuth typically sees disconnected or error depending on Dymium and token state—not not_configured.
Step 3: Authenticate with Dymium (OAuth flow)
Duration: 6 minutes
Goal: Obtain auth_url, complete browser authorization, post code and state to the callback endpoint.
3.1 Start OAuth
Endpoint: POST /api/data/connection/auth
bash
curl -s -X POST "$GHOSTAI_BASE_URL/api/data/connection/auth" \
-H "Authorization: Bearer $GHOSTAI_API_KEY" \
-H "Content-Type: application/json" | python3 -m json.toolExpected: 200 with:
auth_url— open this URL in a browser logged in as the analyst userstate— opaque CSRF token; must be sent back unchanged with the authorizationcode
400 if Dymium is not configured for the org: "Dymium platform is not configured..."
3.2 Complete authorization in the browser
Navigate to auth_url, sign in to Dymium, approve consent. Dymium redirects to:
{GHOSTAI_BASE_URL}/api/data/connection/callback?code=...&state=...
If the IdP shows an HTML page instead of posting JSON automatically, copy code and state from the query string.
3.3 Exchange the code (API callback)
Endpoint: POST /api/data/connection/callback
The redirect lands on /api/data/connection/callback with code and state query parameters. Assign those two strings to DYMIUM_OAUTH_CODE and DYMIUM_OAUTH_STATE in your shell (use single quotes if either value contains characters that would otherwise be expanded by the shell), then exchange them:
bash
curl -s -X POST "$GHOSTAI_BASE_URL/api/data/connection/callback" \
-H "Authorization: Bearer $GHOSTAI_API_KEY" \
-H "Content-Type: application/json" \
-d "{\"code\": \"$DYMIUM_OAUTH_CODE\", \"state\": \"$DYMIUM_OAUTH_STATE\"}" | python3 -m json.toolExpected: 200 with {"success": true, "message": "Connected to Dymium platform."}
GhostAI stores per-user OAuth tokens for subsequent /api/data/* calls.
400 on mismatch or expired code — repeat 3.1 to get a fresh auth_url and state.
3.4 Re-check status
bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
"$GHOSTAI_BASE_URL/api/data/connection/status" | python3 -m json.toolExpected: status healthy when Dymium is reachable and tokens are valid.
Step 4: Browse data sources and schema
Duration: 6 minutes
Goal: List accessible sources, then load full tables and columns (including governed flags).
4.1 List sources
Endpoint: GET /api/data/sources
bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
"$GHOSTAI_BASE_URL/api/data/sources" | python3 -m json.toolExpected: 200 — JSON array. Each element includes at minimum:
idnamesource_type(for examplepostgresql,mysql,snowflake)databasedescriptionpolicy_nametables(often empty in list response; subsection 4.2 loads full schema)
401 — "Not authenticated with Dymium. Please connect in Settings." — repeat Step 3.
502 — upstream Dymium error; inspect detail in the JSON error body.
Capture a source_id from the first object. The remainder of this lab uses a variable for clarity:
bash
export DYMIUM_SOURCE_ID="src_01_postgres_analytics"Replace the value with the id returned by your environment.
4.2 Schema for one source
Endpoint: GET /api/data/sources/{source_id}/schema
bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
"$GHOSTAI_BASE_URL/api/data/sources/${DYMIUM_SOURCE_ID}/schema" | python3 -m json.toolExpected: 200 — single DataSourceInfo object with populated tables. Each table includes:
name,schema_name,description, optionalrow_countcolumns:name,data_type,nullable,governed,description
403 — "Access denied by Dymium governance policy." — user lacks schema visibility for that source.
Step 5: Natural language query
Duration: 5 minutes
Goal: Run POST /api/data/query with natural_language: true and max_rows.
Endpoint: POST /api/data/query
bash
curl -s -X POST "$GHOSTAI_BASE_URL/api/data/query" \
-H "Authorization: Bearer $GHOSTAI_API_KEY" \
-H "Content-Type: application/json" \
-d "{
\"source_id\": \"${DYMIUM_SOURCE_ID}\",
\"query\": \"List the five most recent orders with customer name and total amount\",
\"natural_language\": true,
\"max_rows\": 25
}" | python3 -m json.toolExpected: 200 — DataQueryResponse:
columns,rows,row_count,truncated,query_ms,source_id,source_name- or
errorstring inside the body when Dymium returns a business error without throwing
Confirm truncated behavior when the result exceeds max_rows.
Inspect rows for cells that correspond to governed: true columns from Step 4; values should follow Dymium policy (masking, hashing, etc.).
Step 6: Direct SQL query
Duration: 4 minutes
Goal: Same endpoint with natural_language: false; compare enforcement with Step 5.
Endpoint: POST /api/data/query
Adjust table and column names to match your schema from Step 4.
bash
curl -s -X POST "$GHOSTAI_BASE_URL/api/data/query" \
-H "Authorization: Bearer $GHOSTAI_API_KEY" \
-H "Content-Type: application/json" \
-d "{
\"source_id\": \"${DYMIUM_SOURCE_ID}\",
\"query\": \"SELECT id, email, amount_cents, created_at FROM public.orders ORDER BY created_at DESC LIMIT 20\",
\"natural_language\": false,
\"max_rows\": 20
}" | python3 -m json.toolExpected: 200 with the same response shape as Step 5.
403 — policy denies table or column-level access.
Note: Omitting natural_language defaults it to true in the API model; for raw SQL you must send false explicitly.
Step 7: AI insight
Duration: 4 minutes
Goal: Request narrative analysis from the Dymium BI agent.
Endpoint: POST /api/data/insight
bash
curl -s -X POST "$GHOSTAI_BASE_URL/api/data/insight" \
-H "Authorization: Bearer $GHOSTAI_API_KEY" \
-H "Content-Type: application/json" \
-d "{
\"question\": \"Summarize week-over-week revenue trend and highlight any product category that moved more than ten percent.\",
\"source_id\": \"${DYMIUM_SOURCE_ID}\"
}" | python3 -m json.toolExpected: 200 — InsightResponse:
summary,detail, optionaldata,source_id,query_ms- or
errorin the body
Optional: omit source_id to let Dymium scope the question across sources available to the user (behavior depends on Dymium tenant configuration).
bash
curl -s -X POST "$GHOSTAI_BASE_URL/api/data/insight" \
-H "Authorization: Bearer $GHOSTAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"question": "Which single metric should leadership watch this month based on cross-system signals?"
}' | python3 -m json.toolStep 8: Disconnect
Duration: 2 minutes
Goal: Remove cached OAuth tokens for the current user.
Endpoint: POST /api/data/connection/disconnect
bash
curl -s -X POST "$GHOSTAI_BASE_URL/api/data/connection/disconnect" \
-H "Authorization: Bearer $GHOSTAI_API_KEY" \
-H "Content-Type: application/json" | python3 -m json.toolExpected: 200 — {"success": true, "message": "Disconnected from Dymium platform."}
Confirm tokens are cleared:
bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
"$GHOSTAI_BASE_URL/api/data/connection/status" | python3 -m json.toolExpected: disconnected (or error) until Step 3 is repeated. Org platform_url / client_id remain until an admin changes PUT /api/data/connection/config.
Reference: all Data Pane routes from data.py
| Method | Path |
|---|---|
| GET | /api/data/connection/config |
| PUT | /api/data/connection/config |
| GET | /api/data/connection/status |
| POST | /api/data/connection/auth |
| POST | /api/data/connection/callback |
| POST | /api/data/connection/disconnect |
| GET | /api/data/sources |
| GET | /api/data/sources/{source_id}/schema |
| POST | /api/data/query |
| POST | /api/data/insight |
Lab complete
You configured org-level Dymium credentials, checked status, completed OAuth, browsed sources and schema, ran natural-language and SQL queries with max_rows, requested a BI insight, and disconnected per-user tokens. Next steps on a real engagement: map policy_name to customer Dymium policies, validate governed columns with compliance stakeholders, and automate health checks against GET /api/data/connection/status.