Skip to content

GhostAI SaaS Integration Architecture ​

Enterprise Overview


Overview ​

GhostAI's SaaS Integration Architecture enables organizations to connect their existing SaaS tools — HR systems, CRM platforms, ticketing systems, communication tools — to GhostAI's AI memory system. The result: the AI understands your organization without your organization's sensitive data ever reaching the AI model.

The system implements a privacy-first ingestion pipeline: data is fetched from SaaS APIs via OAuth, normalized into a common record format, passed through PII field filtering that strips sensitive fields (SSN, salary, DOB, home address) before any data reaches the memory system, and stored with source attribution so the AI can cite where its knowledge comes from.

The architecture supports dual ingestion modes (scheduled polling and real-time webhooks), dual record types (structured data records and unstructured content), cross-connector entity resolution (matching people and companies across systems), and bidirectional actions (the AI can write back to connected tools with user confirmation).


The Context Problem ​

Large language models are stateless: they do not know who is on your engineering team, what your company's org chart looks like, or which customers your sales team is working with. Enterprise knowledge lives across dozens of SaaS tools — Rippling for HR, Salesforce for CRM, Jira for ticketing, Slack for communication — and none of it reaches the AI by default.

The current workarounds are inadequate:

  • Manual copy-paste leaks PII. Users paste employee lists or customer data directly into chat, exposing SSNs, salaries, and personal contact information to the AI provider.
  • Knowledge base curation requires ongoing human effort. Someone must manually enter and update organizational facts, which quickly becomes stale.
  • Generic RAG connectors ingest everything indiscriminately, creating compliance risk when sensitive fields (compensation, health records, financial data) end up in vector databases accessible to the AI.

GhostAI's SaaS connector framework solves this by automating the ingestion of organizational context with PII protection built into the pipeline — not bolted on after the fact.


Architecture Overview ​

┌─────────────────────────────────────────────────────────────────────────────┐
│                          SaaS CONNECTOR FRAMEWORK                           │
│                                                                             │
│  ┌──────────────────┐  ┌──────────────────┐  ┌──────────────────────────┐  │
│  │  INGESTION LAYER │  │  PROCESSING LAYER│  │     MEMORY LAYER         │  │
│  │                  │  │                  │  │                          │  │
│  │  SyncEngine      │  │  PII Field       │  │  MemoryBridge            │  │
│  │  (polling)       │──│  Filter          │──│  (records → memories)    │  │
│  │                  │  │                  │  │                          │  │
│  │  WebhookEngine   │  │  Entity          │  │  Source Attribution      │  │
│  │  (push)          │──│  Resolver        │──│  (provenance metadata)   │  │
│  │                  │  │                  │  │                          │  │
│  └──────────────────┘  │  Content         │  │  Visibility Scope        │  │
│                        │  Chunking        │  │  (access control)        │  │
│  ┌──────────────────┐  └──────────────────┘  └──────────────────────────┘  │
│  │  CONNECTOR LAYER │                                                      │
│  │                  │  ┌──────────────────┐  ┌──────────────────────────┐  │
│  │  BaseConnector   │  │  STORAGE LAYER   │  │     AI LAYER             │  │
│  │  (ABC)           │  │                  │  │                          │  │
│  │                  │  │  saas_connections │  │  Memory Recall           │  │
│  │  RipplingConn    │  │  connector_records│  │  (with attribution)      │  │
│  │  MergeHrisConn   │  │  resolved_entities│  │                          │  │
│  │  [Future...]     │  │  sync_jobs       │  │  System Prompt Addition  │  │
│  │                  │  │  webhook_configs │  │  (active connections)    │  │
│  └──────────────────┘  └──────────────────┘  └──────────────────────────┘  │
│                                                                             │
│  ┌──────────────────────────────────────────────────────────────────────┐   │
│  │  SECURITY LAYER                                                      │   │
│  │  AES-256-GCM Token Encryption │ OAuth CSRF Protection │ Webhook     │   │
│  │  Signature Verification │ Rate Limiting │ GDPR Consent │ Audit Log  │   │
│  └──────────────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────────┘

Connector Abstraction ​

Every SaaS integration implements the BaseConnector abstract base class, which defines a consistent interface for OAuth, data fetching, webhook handling, PII filtering configuration, and bidirectional actions. The ConnectorRegistry auto-discovers connector implementations by scanning the providers/ directory.

Adding a new connector = creating one Python file. The file implements BaseConnector, defines its OAuth config, entity types, field allowlists, and optional webhook/action support. The registry picks it up automatically.

This one-file-per-connector model is what enables rapid expansion: Slack, Jira, Salesforce, Notion, and dozens more can be added by implementing the same interface without touching framework code.

Hybrid Architecture ​

The framework supports two connector strategies simultaneously:

  • Direct connectors (e.g., RipplingConnector): Talk directly to the SaaS vendor's API. Full control over OAuth scopes, field mapping, and webhook integration. Best for strategic integrations where deep feature support matters.
  • Unified API connectors (e.g., MergeHrisConnector): Delegate to a unified API platform (Merge.dev) that normalizes access to 50+ HRIS/CRM platforms through a single integration. Best for broad coverage of the long tail.

Both types implement the same BaseConnector interface — the SyncEngine and MemoryBridge do not care which strategy a connector uses.


Data Flow ​

Ingestion Pipeline ​

The complete data flow for a SaaS connector record:

SaaS Tool API
    │
    ▼
OAuth Token Decrypt (AES-256-GCM)
    │
    ▼
Paginated Fetch (with rate limit handling)
    │
    ▼
Normalize to NormalizedRecord
    │
    ▼
Upsert to connector_records (AES-256-GCM encrypted at rest)
    │
    ▼
Entity Resolution (match across connectors)
    │
    ▼
PII Field Filter (memory_field_allowlist)
    │
    ├── Data Records → Structured Fact Memories
    │
    └── Content Records → Chunk → Embed → Document Memories
              │
              ▼
        Source Attribution Metadata
              │
              ▼
        Visibility Scope Assignment
              │
              ▼
        Memory Store (AI-accessible)

Dual Record Types ​

The framework handles two kinds of external data:

TypeExamplesMemory Path
Data recordsEmployees, contacts, ticketsFiltered through field allowlist, written as structured fact memories
Content recordsSlack messages, Notion pages, Jira descriptionsPassed through PII detection pipeline, chunked, embedded, written as document memories

Sample Data Flow: Rippling Employee ​

Raw data from Rippling API:

FieldValueIn Memory?
nameAlice SmithYes
emailalice@acme.comYes
departmentEngineeringYes
titleSenior EngineerYes
ssn123-45-6789No — PII filtered
salary$185,000No — PII filtered
dateOfBirth1990-03-15No — PII filtered
homeAddress123 Main St, SFNo — PII filtered

Memory text written to AI:

Alice Smith is a Senior Engineer in the Engineering department (Platform team),
based in San Francisco. Employment type: Full-time. Started: 2024-01-15.
Reports to: Bob Jones. Status: Active.
[Source: Rippling, synced 2h ago]

The raw data (including SSN, salary, etc.) remains in connector_records encrypted with AES-256-GCM for admin use. Only the safe fields reach the AI.


Security Model ​

Token Encryption ​

OAuth tokens are encrypted at rest using AES-256-GCM with per-organization encryption keys:

  • Key derivation uses the tenant_encryption_keys table (per-org keys, already used for BYOK), falling back to a jwt_secret-derived key
  • Both access_token and refresh_token are encrypted before database write and decrypted on read
  • This replaces the XOR-based encryption used by the cloud drive integration surface, providing authenticated encryption with integrity verification

OAuth CSRF Protection ​

The OAuth state parameter is a cryptographic random token stored server-side with a 5-minute TTL. The callback endpoint validates that state matches before exchanging the authorization code. This prevents CSRF attacks where an attacker tricks an admin into linking a malicious account.

Webhook Security ​

Inbound webhooks are protected by three layers:

  1. Signature verification: Each connector implements verify_webhook() to validate the provider's HMAC signature
  2. Timestamp validation: Payloads older than 5 minutes are rejected (when the connector provides timestamps)
  3. Rate limiting: 100 requests/minute per webhook URL; excess returns 429

PII Guards ​

The memory_field_allowlist on each connector is the primary PII defense for data records. It operates as a default-deny filter: only explicitly listed fields pass through to the memory system. Content records take a different path through the existing PII detection pipeline.

Access Control ​

Each connection has a configurable visibility scope (organization-wide, group-restricted, or user-only) that determines which memory scope is used when writing. The MemoryBridge reads this scope to ensure data reaches only authorized users' AI sessions.

Before OAuth authorization, a consent dialog explains what data will be accessed and what PII fields are filtered. The admin's consent timestamp and granted scopes are stored in saas_connections.consent_accepted_at and included in audit logs.


Entity Resolution ​

When multiple SaaS tools are connected, the same person or company appears in multiple systems. The EntityResolver matches entities across connectors to create unified profiles.

Matching Strategy (v1):

Entity TypeMatching FieldExample
PeopleEmail addressalice@acme.com in Rippling = alice@acme.com in Salesforce
CompaniesEmail domain@acme.com across all connectors
TeamsDepartment/team name"Engineering" within the same org

When a match is found, the MemoryBridge writes a single unified memory that aggregates data from all sources, rather than creating fragmented per-connector memories. This is what makes "tell me everything about Alice" work across systems.


Bidirectional Actions ​

Connectors can expose write operations that the AI invokes as tools during conversation. The ActionExecutor registers connector actions with the existing tool_registry, so the AI discovers them naturally alongside other tools.

Example flow:

  1. User: "Create a Jira ticket for the login bug Alice reported"
  2. AI identifies the create_jira_issue action from the Jira connector
  3. AI drafts the ticket: project=PLATFORM, summary="Fix login bug reported by Alice Smith", type=Bug
  4. Because requires_confirmation=true, the AI asks: "I'll create this Jira ticket. Proceed?"
  5. User confirms
  6. ActionExecutor loads the Jira connection's OAuth token and calls connector.execute_action()
  7. Result: ticket PLATFORM-1234 created, deep link returned

All action executions are logged via the audit system and gated by the connection's visibility scope.


Source Attribution ​

Every memory created from connector data carries provenance metadata:

  • source_type: "connector"
  • source_connector_id: "rippling"
  • source_connection_id: "conn_abc123"
  • source_synced_at: "2026-03-29T10:00:00Z"
  • source_external_url: deep link to the record in the SaaS tool (when available)

When the memory recall system includes connector-sourced memories in the AI's context window, it appends attribution: [Source: Rippling, synced 2h ago]. If the data is older than 24 hours, a staleness caveat is added.

When an organization has active SaaS connections, the system prompt includes: "You have access to data from the following connected systems: Rippling. When answering questions about people, teams, or organizational data, cite your source."


Webhook Real-Time Sync ​

Scheduled polling provides batch synchronization on a configurable interval (default: 60 minutes). For organizations that need real-time updates, the WebhookEngine processes inbound webhooks from SaaS tools.

Each connection gets a unique, unguessable webhook URL containing a cryptographic secret. The SaaS tool POSTs events to this URL; the WebhookEngine validates the signature, parses events into normalized records, upserts them, and triggers the MemoryBridge for immediate memory updates.

Polling vs. Webhooks:

AspectPollingWebhooks
LatencyMinutes (sync interval)Seconds
ReliabilityGhostAI controls timingDepends on provider delivery
CompletenessFull scan catches everythingMay miss events if webhook is down
TierAll tiersBusiness+

The recommended configuration is both: polling as the baseline with webhooks for real-time updates. Polling catches anything webhooks might miss.


Health Monitoring ​

The HealthMonitor runs on a schedule (default: every 15 minutes) and proactively detects connection problems:

  • Token expiry: Warns 24 hours before expiry, marks as expired when token lapses
  • Sync failures: Marks as error after 3 consecutive failures
  • API reachability: Lightweight HEAD/status check against the SaaS API

Health status is surfaced in the UI with color-coded badges and communicated to org admins via the notification service when a connection degrades.


Feature Tiering ​

Five feature flags gate SaaS connector capabilities by subscription tier:

FlagCapabilityTier
saas_connectorsBase connector framework, OAuth, polling sync, memory bridgePersonal+
saas_connector_webhooksReal-time webhook ingestionBusiness+
saas_connector_actionsAI-invoked write operations in connected toolsBusiness+
saas_connector_entitiesCross-connector entity resolution and unified profilesBusiness+
saas_connector_unified_apiMerge.dev unified API connectors (50+ HRIS/CRM platforms)Enterprise

The base feature starts at Personal+ (any paying user) for broad adoption. Advanced capabilities (webhooks, actions, entity resolution) serve as Business+/Enterprise upsell levers. The unified API gate at Enterprise reflects the per-account cost of the Merge.dev platform.


Compliance Mapping ​

The SaaS connector framework addresses compliance requirements across major regulatory frameworks:

FrameworkRequirementHow Addressed
HIPAABAA coverage extends to third-party dataPII field filtering prevents PHI from reaching AI; encrypted storage; audit trail
SOC 2Access control, audit logging, encryptionPer-connection visibility scopes; full audit trail; AES-256-GCM token encryption
GDPRConsent, right to deletion, data portabilityConsent dialog with timestamp; cascade delete on disconnect; data export API
PCI-DSSCardholder data protectionPayment card fields excluded from memory_field_allowlist; encrypted raw storage
CCPAConsumer data rightsData export via existing export service; disconnect removes all synced data

Deployment Considerations ​

Environment Variables ​

VariableDescriptionRequired
SAAS_CONNECTORS_ENABLEDMaster kill switchYes
RIPPLING_CLIENT_ID / RIPPLING_CLIENT_SECRETRippling OAuth credentialsFor Rippling
MERGE_API_KEYMerge.dev API keyFor Merge HRIS
CONNECTOR_SYNC_INTERVAL_MINUTESPolling intervalNo (default: 60)
CONNECTOR_HEALTH_CHECK_INTERVAL_MINUTESHealth check intervalNo (default: 15)
CONNECTOR_WEBHOOK_BASE_URLPublic URL for webhook callbacksNo (default: base_url)

Egress Rules ​

The backend must be able to reach SaaS API endpoints:

  • api.rippling.com (Rippling)
  • api.merge.dev (Merge.dev)
  • Provider-specific OAuth endpoints

Webhook Ingress ​

The webhook endpoint (POST /api/connectors/webhooks/...) must be reachable from the public internet. Configure firewall/ALB rules to allow inbound POST requests on this path.

Scheduled Workers ​

The SyncEngine and HealthMonitor are registered as periodic tasks in the existing scheduler.py service. No additional worker processes are needed.


Comparison to Alternatives ​

ApproachStrengthsWeaknesses vs. GhostAI
Generic RAG connectors (LlamaIndex, Unstructured)Broad document supportNo PII filtering; no field-level control; no source attribution in AI responses
iPaaS tools (Workato, Zapier, n8n)Workflow automationData transformation focus, not AI memory enrichment; no PII awareness; separate system to maintain
Unified API platforms (Merge, Finch)Broad platform coverageData normalization only; no memory integration, PII filtering, or AI context management
Manual knowledge basesSimple, controlledDoes not scale; stale within days; requires dedicated human effort
GhostAI SaaS ConnectorsPII-filtered, source-attributed, memory-integrated, real-time, bidirectionalRequires connector implementation per SaaS tool (mitigated by unified API support)

GhostAI's approach is unique in combining SaaS data ingestion with privacy-first PII filtering and AI memory enrichment in a single pipeline. The AI gets organizational context; sensitive data never reaches the model; and every answer cites its source.

SOC 2 Type II Certified | BAA Available | HIPAA-Ready