Appearance
GhostAI SaaS Integration Architecture
Enterprise Overview
Overview
GhostAI's SaaS Integration Architecture enables organizations to connect their existing SaaS tools — HR systems, CRM platforms, ticketing systems, communication tools — to GhostAI's AI memory system. The result: the AI understands your organization without your organization's sensitive data ever reaching the AI model.
The system implements a privacy-first ingestion pipeline: data is fetched from SaaS APIs via OAuth, normalized into a common record format, passed through PII field filtering that strips sensitive fields (SSN, salary, DOB, home address) before any data reaches the memory system, and stored with source attribution so the AI can cite where its knowledge comes from.
The architecture supports dual ingestion modes (scheduled polling and real-time webhooks), dual record types (structured data records and unstructured content), cross-connector entity resolution (matching people and companies across systems), and bidirectional actions (the AI can write back to connected tools with user confirmation).
The Context Problem
Large language models are stateless: they do not know who is on your engineering team, what your company's org chart looks like, or which customers your sales team is working with. Enterprise knowledge lives across dozens of SaaS tools — Rippling for HR, Salesforce for CRM, Jira for ticketing, Slack for communication — and none of it reaches the AI by default.
The current workarounds are inadequate:
- Manual copy-paste leaks PII. Users paste employee lists or customer data directly into chat, exposing SSNs, salaries, and personal contact information to the AI provider.
- Knowledge base curation requires ongoing human effort. Someone must manually enter and update organizational facts, which quickly becomes stale.
- Generic RAG connectors ingest everything indiscriminately, creating compliance risk when sensitive fields (compensation, health records, financial data) end up in vector databases accessible to the AI.
GhostAI's SaaS connector framework solves this by automating the ingestion of organizational context with PII protection built into the pipeline — not bolted on after the fact.
Architecture Overview
┌─────────────────────────────────────────────────────────────────────────────┐
│ SaaS CONNECTOR FRAMEWORK │
│ │
│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────────────┐ │
│ │ INGESTION LAYER │ │ PROCESSING LAYER│ │ MEMORY LAYER │ │
│ │ │ │ │ │ │ │
│ │ SyncEngine │ │ PII Field │ │ MemoryBridge │ │
│ │ (polling) │──│ Filter │──│ (records → memories) │ │
│ │ │ │ │ │ │ │
│ │ WebhookEngine │ │ Entity │ │ Source Attribution │ │
│ │ (push) │──│ Resolver │──│ (provenance metadata) │ │
│ │ │ │ │ │ │ │
│ └──────────────────┘ │ Content │ │ Visibility Scope │ │
│ │ Chunking │ │ (access control) │ │
│ ┌──────────────────┐ └──────────────────┘ └──────────────────────────┘ │
│ │ CONNECTOR LAYER │ │
│ │ │ ┌──────────────────┐ ┌──────────────────────────┐ │
│ │ BaseConnector │ │ STORAGE LAYER │ │ AI LAYER │ │
│ │ (ABC) │ │ │ │ │ │
│ │ │ │ saas_connections │ │ Memory Recall │ │
│ │ RipplingConn │ │ connector_records│ │ (with attribution) │ │
│ │ MergeHrisConn │ │ resolved_entities│ │ │ │
│ │ [Future...] │ │ sync_jobs │ │ System Prompt Addition │ │
│ │ │ │ webhook_configs │ │ (active connections) │ │
│ └──────────────────┘ └──────────────────┘ └──────────────────────────┘ │
│ │
│ ┌──────────────────────────────────────────────────────────────────────┐ │
│ │ SECURITY LAYER │ │
│ │ AES-256-GCM Token Encryption │ OAuth CSRF Protection │ Webhook │ │
│ │ Signature Verification │ Rate Limiting │ GDPR Consent │ Audit Log │ │
│ └──────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘Connector Abstraction
Every SaaS integration implements the BaseConnector abstract base class, which defines a consistent interface for OAuth, data fetching, webhook handling, PII filtering configuration, and bidirectional actions. The ConnectorRegistry auto-discovers connector implementations by scanning the providers/ directory.
Adding a new connector = creating one Python file. The file implements BaseConnector, defines its OAuth config, entity types, field allowlists, and optional webhook/action support. The registry picks it up automatically.
This one-file-per-connector model is what enables rapid expansion: Slack, Jira, Salesforce, Notion, and dozens more can be added by implementing the same interface without touching framework code.
Hybrid Architecture
The framework supports two connector strategies simultaneously:
- Direct connectors (e.g., RipplingConnector): Talk directly to the SaaS vendor's API. Full control over OAuth scopes, field mapping, and webhook integration. Best for strategic integrations where deep feature support matters.
- Unified API connectors (e.g., MergeHrisConnector): Delegate to a unified API platform (Merge.dev) that normalizes access to 50+ HRIS/CRM platforms through a single integration. Best for broad coverage of the long tail.
Both types implement the same BaseConnector interface — the SyncEngine and MemoryBridge do not care which strategy a connector uses.
Data Flow
Ingestion Pipeline
The complete data flow for a SaaS connector record:
SaaS Tool API
│
▼
OAuth Token Decrypt (AES-256-GCM)
│
▼
Paginated Fetch (with rate limit handling)
│
▼
Normalize to NormalizedRecord
│
▼
Upsert to connector_records (AES-256-GCM encrypted at rest)
│
▼
Entity Resolution (match across connectors)
│
▼
PII Field Filter (memory_field_allowlist)
│
├── Data Records → Structured Fact Memories
│
└── Content Records → Chunk → Embed → Document Memories
│
▼
Source Attribution Metadata
│
▼
Visibility Scope Assignment
│
▼
Memory Store (AI-accessible)Dual Record Types
The framework handles two kinds of external data:
| Type | Examples | Memory Path |
|---|---|---|
| Data records | Employees, contacts, tickets | Filtered through field allowlist, written as structured fact memories |
| Content records | Slack messages, Notion pages, Jira descriptions | Passed through PII detection pipeline, chunked, embedded, written as document memories |
Sample Data Flow: Rippling Employee
Raw data from Rippling API:
| Field | Value | In Memory? |
|---|---|---|
| name | Alice Smith | Yes |
| alice@acme.com | Yes | |
| department | Engineering | Yes |
| title | Senior Engineer | Yes |
| ssn | 123-45-6789 | No — PII filtered |
| salary | $185,000 | No — PII filtered |
| dateOfBirth | 1990-03-15 | No — PII filtered |
| homeAddress | 123 Main St, SF | No — PII filtered |
Memory text written to AI:
Alice Smith is a Senior Engineer in the Engineering department (Platform team),
based in San Francisco. Employment type: Full-time. Started: 2024-01-15.
Reports to: Bob Jones. Status: Active.
[Source: Rippling, synced 2h ago]The raw data (including SSN, salary, etc.) remains in connector_records encrypted with AES-256-GCM for admin use. Only the safe fields reach the AI.
Security Model
Token Encryption
OAuth tokens are encrypted at rest using AES-256-GCM with per-organization encryption keys:
- Key derivation uses the
tenant_encryption_keystable (per-org keys, already used for BYOK), falling back to ajwt_secret-derived key - Both
access_tokenandrefresh_tokenare encrypted before database write and decrypted on read - This replaces the XOR-based encryption used by the cloud drive integration surface, providing authenticated encryption with integrity verification
OAuth CSRF Protection
The OAuth state parameter is a cryptographic random token stored server-side with a 5-minute TTL. The callback endpoint validates that state matches before exchanging the authorization code. This prevents CSRF attacks where an attacker tricks an admin into linking a malicious account.
Webhook Security
Inbound webhooks are protected by three layers:
- Signature verification: Each connector implements
verify_webhook()to validate the provider's HMAC signature - Timestamp validation: Payloads older than 5 minutes are rejected (when the connector provides timestamps)
- Rate limiting: 100 requests/minute per webhook URL; excess returns 429
PII Guards
The memory_field_allowlist on each connector is the primary PII defense for data records. It operates as a default-deny filter: only explicitly listed fields pass through to the memory system. Content records take a different path through the existing PII detection pipeline.
Access Control
Each connection has a configurable visibility scope (organization-wide, group-restricted, or user-only) that determines which memory scope is used when writing. The MemoryBridge reads this scope to ensure data reaches only authorized users' AI sessions.
GDPR Consent Flow
Before OAuth authorization, a consent dialog explains what data will be accessed and what PII fields are filtered. The admin's consent timestamp and granted scopes are stored in saas_connections.consent_accepted_at and included in audit logs.
Entity Resolution
When multiple SaaS tools are connected, the same person or company appears in multiple systems. The EntityResolver matches entities across connectors to create unified profiles.
Matching Strategy (v1):
| Entity Type | Matching Field | Example |
|---|---|---|
| People | Email address | alice@acme.com in Rippling = alice@acme.com in Salesforce |
| Companies | Email domain | @acme.com across all connectors |
| Teams | Department/team name | "Engineering" within the same org |
When a match is found, the MemoryBridge writes a single unified memory that aggregates data from all sources, rather than creating fragmented per-connector memories. This is what makes "tell me everything about Alice" work across systems.
Bidirectional Actions
Connectors can expose write operations that the AI invokes as tools during conversation. The ActionExecutor registers connector actions with the existing tool_registry, so the AI discovers them naturally alongside other tools.
Example flow:
- User: "Create a Jira ticket for the login bug Alice reported"
- AI identifies the
create_jira_issueaction from the Jira connector - AI drafts the ticket: project=PLATFORM, summary="Fix login bug reported by Alice Smith", type=Bug
- Because
requires_confirmation=true, the AI asks: "I'll create this Jira ticket. Proceed?" - User confirms
- ActionExecutor loads the Jira connection's OAuth token and calls
connector.execute_action() - Result: ticket PLATFORM-1234 created, deep link returned
All action executions are logged via the audit system and gated by the connection's visibility scope.
Source Attribution
Every memory created from connector data carries provenance metadata:
source_type: "connector"source_connector_id: "rippling"source_connection_id: "conn_abc123"source_synced_at: "2026-03-29T10:00:00Z"source_external_url: deep link to the record in the SaaS tool (when available)
When the memory recall system includes connector-sourced memories in the AI's context window, it appends attribution: [Source: Rippling, synced 2h ago]. If the data is older than 24 hours, a staleness caveat is added.
When an organization has active SaaS connections, the system prompt includes: "You have access to data from the following connected systems: Rippling. When answering questions about people, teams, or organizational data, cite your source."
Webhook Real-Time Sync
Scheduled polling provides batch synchronization on a configurable interval (default: 60 minutes). For organizations that need real-time updates, the WebhookEngine processes inbound webhooks from SaaS tools.
Each connection gets a unique, unguessable webhook URL containing a cryptographic secret. The SaaS tool POSTs events to this URL; the WebhookEngine validates the signature, parses events into normalized records, upserts them, and triggers the MemoryBridge for immediate memory updates.
Polling vs. Webhooks:
| Aspect | Polling | Webhooks |
|---|---|---|
| Latency | Minutes (sync interval) | Seconds |
| Reliability | GhostAI controls timing | Depends on provider delivery |
| Completeness | Full scan catches everything | May miss events if webhook is down |
| Tier | All tiers | Business+ |
The recommended configuration is both: polling as the baseline with webhooks for real-time updates. Polling catches anything webhooks might miss.
Health Monitoring
The HealthMonitor runs on a schedule (default: every 15 minutes) and proactively detects connection problems:
- Token expiry: Warns 24 hours before expiry, marks as expired when token lapses
- Sync failures: Marks as error after 3 consecutive failures
- API reachability: Lightweight HEAD/status check against the SaaS API
Health status is surfaced in the UI with color-coded badges and communicated to org admins via the notification service when a connection degrades.
Feature Tiering
Five feature flags gate SaaS connector capabilities by subscription tier:
| Flag | Capability | Tier |
|---|---|---|
saas_connectors | Base connector framework, OAuth, polling sync, memory bridge | Personal+ |
saas_connector_webhooks | Real-time webhook ingestion | Business+ |
saas_connector_actions | AI-invoked write operations in connected tools | Business+ |
saas_connector_entities | Cross-connector entity resolution and unified profiles | Business+ |
saas_connector_unified_api | Merge.dev unified API connectors (50+ HRIS/CRM platforms) | Enterprise |
The base feature starts at Personal+ (any paying user) for broad adoption. Advanced capabilities (webhooks, actions, entity resolution) serve as Business+/Enterprise upsell levers. The unified API gate at Enterprise reflects the per-account cost of the Merge.dev platform.
Compliance Mapping
The SaaS connector framework addresses compliance requirements across major regulatory frameworks:
| Framework | Requirement | How Addressed |
|---|---|---|
| HIPAA | BAA coverage extends to third-party data | PII field filtering prevents PHI from reaching AI; encrypted storage; audit trail |
| SOC 2 | Access control, audit logging, encryption | Per-connection visibility scopes; full audit trail; AES-256-GCM token encryption |
| GDPR | Consent, right to deletion, data portability | Consent dialog with timestamp; cascade delete on disconnect; data export API |
| PCI-DSS | Cardholder data protection | Payment card fields excluded from memory_field_allowlist; encrypted raw storage |
| CCPA | Consumer data rights | Data export via existing export service; disconnect removes all synced data |
Deployment Considerations
Environment Variables
| Variable | Description | Required |
|---|---|---|
SAAS_CONNECTORS_ENABLED | Master kill switch | Yes |
RIPPLING_CLIENT_ID / RIPPLING_CLIENT_SECRET | Rippling OAuth credentials | For Rippling |
MERGE_API_KEY | Merge.dev API key | For Merge HRIS |
CONNECTOR_SYNC_INTERVAL_MINUTES | Polling interval | No (default: 60) |
CONNECTOR_HEALTH_CHECK_INTERVAL_MINUTES | Health check interval | No (default: 15) |
CONNECTOR_WEBHOOK_BASE_URL | Public URL for webhook callbacks | No (default: base_url) |
Egress Rules
The backend must be able to reach SaaS API endpoints:
api.rippling.com(Rippling)api.merge.dev(Merge.dev)- Provider-specific OAuth endpoints
Webhook Ingress
The webhook endpoint (POST /api/connectors/webhooks/...) must be reachable from the public internet. Configure firewall/ALB rules to allow inbound POST requests on this path.
Scheduled Workers
The SyncEngine and HealthMonitor are registered as periodic tasks in the existing scheduler.py service. No additional worker processes are needed.
Comparison to Alternatives
| Approach | Strengths | Weaknesses vs. GhostAI |
|---|---|---|
| Generic RAG connectors (LlamaIndex, Unstructured) | Broad document support | No PII filtering; no field-level control; no source attribution in AI responses |
| iPaaS tools (Workato, Zapier, n8n) | Workflow automation | Data transformation focus, not AI memory enrichment; no PII awareness; separate system to maintain |
| Unified API platforms (Merge, Finch) | Broad platform coverage | Data normalization only; no memory integration, PII filtering, or AI context management |
| Manual knowledge bases | Simple, controlled | Does not scale; stale within days; requires dedicated human effort |
| GhostAI SaaS Connectors | PII-filtered, source-attributed, memory-integrated, real-time, bidirectional | Requires connector implementation per SaaS tool (mitigated by unified API support) |
GhostAI's approach is unique in combining SaaS data ingestion with privacy-first PII filtering and AI memory enrichment in a single pipeline. The AI gets organizational context; sensitive data never reaches the model; and every answer cites its source.