Skip to content

Data Pane Full Lab ​

Eight steps. About thirty-five minutes. From empty Dymium config to governed queries, insights, and a clean disconnect.

This lab exercises the Data Pane API in securechat/backend/app/routers/data.py. All routes use the prefix /api/data. Organization context is required for connection configuration and downstream calls: use an API key or session tied to a user who belongs to an organization.

  • PUT /api/data/connection/config requires organization admin (require_admin).
  • GET /api/data/connection/config, status, OAuth, sources, schema, query, insight, and disconnect use tenant context and optional user identity for per-user token caching.

Prerequisites ​

Before starting, ensure you have:

  • [ ] GhostAI backend running (example: http://localhost:8000)
  • [ ] BASE_URL on the backend set correctly so OAuth redirect is {base_url}/api/data/connection/callback
  • [ ] A Dymium environment with OAuth client credentials and at least one data source
  • [ ] Org admin API key for Step 1: GHOSTAI_ADMIN_API_KEY (Bearer token for owner or admin role in the target organization)
  • [ ] The same or another user key for Steps 2–8 if you want to separate admin config from analyst usage: GHOSTAI_API_KEY
  • [ ] curl and python3 -m json.tool (or jq)

Environment variables ​

bash
export GHOSTAI_BASE_URL="http://localhost:8000"
export GHOSTAI_ADMIN_API_KEY="gst_..."    # org admin — Step 1
export GHOSTAI_API_KEY="gst_..."            # analyst user — Steps 2–8 (can equal admin for a single-user lab)

Verify you have organization context ​

If GET /api/data/connection/config returns "configured": false with empty platform_url after Step 1, confirm the token’s user is in an organization and you used the admin key for the PUT.


Step 1: Configure the Dymium connection (org admin) ​

Duration: 4 minutes
Goal: Persist platform_url, client_id, and client_secret for the organization. The secret is stored server-side and is never returned on GET.

Endpoint: PUT /api/data/connection/config

Replace the string values with your real Dymium tenant URL, OAuth client ID, and client secret. The example below uses a fictional lab tenant.

bash
curl -s -X PUT "$GHOSTAI_BASE_URL/api/data/connection/config" \
  -H "Authorization: Bearer $GHOSTAI_ADMIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "platform_url": "https://dymium.lab.ghostai.example",
    "client_id": "ghostai-data-pane-lab",
    "client_secret": "dymium-oauth-secret-value-from-dymium-console"
  }' | python3 -m json.tool

Expected: 200 with DymiumConnectionConfigResponse:

  • platform_url — echoed from request
  • client_id — echoed from request
  • configured — true when both platform_url and client_id are non-empty after save

Verify non-secret read:

bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "$GHOSTAI_BASE_URL/api/data/connection/config" | python3 -m json.tool

Expected: Same shape; no client_secret field.

Common errors

  • 403 — not an org admin, or missing organization context on the token used for PUT.

Step 2: Check connection status ​

Duration: 3 minutes
Goal: Read ConnectionStatus before and after user OAuth.

Endpoint: GET /api/data/connection/status

bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "$GHOSTAI_BASE_URL/api/data/connection/status" | python3 -m json.tool

Expected fields

  • status — not_configured | disconnected | healthy | error
  • message — human-readable detail (for example configuration hint when not_configured)
  • platform_version — when Dymium reports a version during health check
  • platform_url — resolved base URL for the org’s Dymium integration

After Step 1, a user who has not completed OAuth typically sees disconnected or error depending on Dymium and token state—not not_configured.


Step 3: Authenticate with Dymium (OAuth flow) ​

Duration: 6 minutes
Goal: Obtain auth_url, complete browser authorization, post code and state to the callback endpoint.

3.1 Start OAuth ​

Endpoint: POST /api/data/connection/auth

bash
curl -s -X POST "$GHOSTAI_BASE_URL/api/data/connection/auth" \
  -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" | python3 -m json.tool

Expected: 200 with:

  • auth_url — open this URL in a browser logged in as the analyst user
  • state — opaque CSRF token; must be sent back unchanged with the authorization code

400 if Dymium is not configured for the org: "Dymium platform is not configured..."

3.2 Complete authorization in the browser ​

Navigate to auth_url, sign in to Dymium, approve consent. Dymium redirects to:

{GHOSTAI_BASE_URL}/api/data/connection/callback?code=...&state=...

If the IdP shows an HTML page instead of posting JSON automatically, copy code and state from the query string.

3.3 Exchange the code (API callback) ​

Endpoint: POST /api/data/connection/callback

The redirect lands on /api/data/connection/callback with code and state query parameters. Assign those two strings to DYMIUM_OAUTH_CODE and DYMIUM_OAUTH_STATE in your shell (use single quotes if either value contains characters that would otherwise be expanded by the shell), then exchange them:

bash
curl -s -X POST "$GHOSTAI_BASE_URL/api/data/connection/callback" \
  -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"code\": \"$DYMIUM_OAUTH_CODE\", \"state\": \"$DYMIUM_OAUTH_STATE\"}" | python3 -m json.tool

Expected: 200 with {"success": true, "message": "Connected to Dymium platform."}

GhostAI stores per-user OAuth tokens for subsequent /api/data/* calls.

400 on mismatch or expired code — repeat 3.1 to get a fresh auth_url and state.

3.4 Re-check status ​

bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "$GHOSTAI_BASE_URL/api/data/connection/status" | python3 -m json.tool

Expected: status healthy when Dymium is reachable and tokens are valid.


Step 4: Browse data sources and schema ​

Duration: 6 minutes
Goal: List accessible sources, then load full tables and columns (including governed flags).

4.1 List sources ​

Endpoint: GET /api/data/sources

bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "$GHOSTAI_BASE_URL/api/data/sources" | python3 -m json.tool

Expected: 200 — JSON array. Each element includes at minimum:

  • id
  • name
  • source_type (for example postgresql, mysql, snowflake)
  • database
  • description
  • policy_name
  • tables (often empty in list response; subsection 4.2 loads full schema)

401 — "Not authenticated with Dymium. Please connect in Settings." — repeat Step 3.

502 — upstream Dymium error; inspect detail in the JSON error body.

Capture a source_id from the first object. The remainder of this lab uses a variable for clarity:

bash
export DYMIUM_SOURCE_ID="src_01_postgres_analytics"

Replace the value with the id returned by your environment.

4.2 Schema for one source ​

Endpoint: GET /api/data/sources/{source_id}/schema

bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "$GHOSTAI_BASE_URL/api/data/sources/${DYMIUM_SOURCE_ID}/schema" | python3 -m json.tool

Expected: 200 — single DataSourceInfo object with populated tables. Each table includes:

  • name, schema_name, description, optional row_count
  • columns: name, data_type, nullable, governed, description

403 — "Access denied by Dymium governance policy." — user lacks schema visibility for that source.


Step 5: Natural language query ​

Duration: 5 minutes
Goal: Run POST /api/data/query with natural_language: true and max_rows.

Endpoint: POST /api/data/query

bash
curl -s -X POST "$GHOSTAI_BASE_URL/api/data/query" \
  -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"source_id\": \"${DYMIUM_SOURCE_ID}\",
    \"query\": \"List the five most recent orders with customer name and total amount\",
    \"natural_language\": true,
    \"max_rows\": 25
  }" | python3 -m json.tool

Expected: 200 — DataQueryResponse:

  • columns, rows, row_count, truncated, query_ms, source_id, source_name
  • or error string inside the body when Dymium returns a business error without throwing

Confirm truncated behavior when the result exceeds max_rows.

Inspect rows for cells that correspond to governed: true columns from Step 4; values should follow Dymium policy (masking, hashing, etc.).


Step 6: Direct SQL query ​

Duration: 4 minutes
Goal: Same endpoint with natural_language: false; compare enforcement with Step 5.

Endpoint: POST /api/data/query

Adjust table and column names to match your schema from Step 4.

bash
curl -s -X POST "$GHOSTAI_BASE_URL/api/data/query" \
  -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"source_id\": \"${DYMIUM_SOURCE_ID}\",
    \"query\": \"SELECT id, email, amount_cents, created_at FROM public.orders ORDER BY created_at DESC LIMIT 20\",
    \"natural_language\": false,
    \"max_rows\": 20
  }" | python3 -m json.tool

Expected: 200 with the same response shape as Step 5.

403 — policy denies table or column-level access.

Note: Omitting natural_language defaults it to true in the API model; for raw SQL you must send false explicitly.


Step 7: AI insight ​

Duration: 4 minutes
Goal: Request narrative analysis from the Dymium BI agent.

Endpoint: POST /api/data/insight

bash
curl -s -X POST "$GHOSTAI_BASE_URL/api/data/insight" \
  -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"question\": \"Summarize week-over-week revenue trend and highlight any product category that moved more than ten percent.\",
    \"source_id\": \"${DYMIUM_SOURCE_ID}\"
  }" | python3 -m json.tool

Expected: 200 — InsightResponse:

  • summary, detail, optional data, source_id, query_ms
  • or error in the body

Optional: omit source_id to let Dymium scope the question across sources available to the user (behavior depends on Dymium tenant configuration).

bash
curl -s -X POST "$GHOSTAI_BASE_URL/api/data/insight" \
  -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "Which single metric should leadership watch this month based on cross-system signals?"
  }' | python3 -m json.tool

Step 8: Disconnect ​

Duration: 2 minutes
Goal: Remove cached OAuth tokens for the current user.

Endpoint: POST /api/data/connection/disconnect

bash
curl -s -X POST "$GHOSTAI_BASE_URL/api/data/connection/disconnect" \
  -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  -H "Content-Type: application/json" | python3 -m json.tool

Expected: 200 — {"success": true, "message": "Disconnected from Dymium platform."}

Confirm tokens are cleared:

bash
curl -s -H "Authorization: Bearer $GHOSTAI_API_KEY" \
  "$GHOSTAI_BASE_URL/api/data/connection/status" | python3 -m json.tool

Expected: disconnected (or error) until Step 3 is repeated. Org platform_url / client_id remain until an admin changes PUT /api/data/connection/config.


Reference: all Data Pane routes from data.py ​

MethodPath
GET/api/data/connection/config
PUT/api/data/connection/config
GET/api/data/connection/status
POST/api/data/connection/auth
POST/api/data/connection/callback
POST/api/data/connection/disconnect
GET/api/data/sources
GET/api/data/sources/{source_id}/schema
POST/api/data/query
POST/api/data/insight

Lab complete ​

You configured org-level Dymium credentials, checked status, completed OAuth, browsed sources and schema, ran natural-language and SQL queries with max_rows, requested a BI insight, and disconnected per-user tokens. Next steps on a real engagement: map policy_name to customer Dymium policies, validate governed columns with compliance stakeholders, and automate health checks against GET /api/data/connection/status.

SOC 2 Type II Certified | BAA Available | HIPAA-Ready