Skip to content

AWS Bedrock BYOK: Secure LLM Inference in Your AWS Account ​

Executive Summary ​

GhostAI supports Bring Your Own Key (BYOK) for Amazon Bedrock, allowing enterprises and regulated organizations to run LLM chat and tool use through their own AWS Bedrock account. Prompt and completion data are sent only to the customer’s Bedrock endpoint; inference runs entirely in the customer’s AWS region under their IAM and network controls. This whitepaper describes the architecture, security model, deployment options (SaaS vs. private), and compliance considerations for using AWS Bedrock with GhostAI.


Why AWS Bedrock BYOK ​

Enterprises, especially in regulated industries (financial services, healthcare, government), often require:

  • Data residency — LLM inference must stay within their cloud boundary and region.
  • Billing and control — Usage billed to their AWS account with no markup; they control quotas and limits.
  • Governance — Existing AWS security (IAM, VPC, CloudTrail, GuardDuty) applies to model access.

Bedrock BYOK lets customers connect GhostAI to their Bedrock account. They configure region and credentials (or IAM role when GhostAI runs in their environment); GhostAI uses that configuration only to invoke Bedrock. No prompt or completion content is processed or stored by GhostAI beyond what is necessary to route the request and stream the response.


Architecture Overview ​

  • Credential model: One Bedrock configuration per tier (Cloud, Tenant, User). Each configuration is either (a) BYOK: region + AWS Access Key ID + Secret Access Key, or (b) IAM-only: region only, with credentials from the default chain (e.g. ECS task role) when GhostAI runs in AWS.
  • Optional custom endpoint: Customers can supply an optional endpoint URL (e.g. VPC endpoint or PrivateLink) so traffic never crosses the public internet when using SaaS.
  • Request flow: User sends a chat request with a Bedrock model selected. GhostAI resolves the Bedrock key for that user/org (User → Tenant → Cloud), builds or reuses a Bedrock client for that region/credentials, and calls the Bedrock runtime OpenAI-compatible endpoint (/openai/v1/chat/completions). Responses stream back through GhostAI to the user. Tenant isolation is enforced: the resolved key is always from the request’s organization and user context.
  • Model discovery: When a Bedrock key is available for the current context, GhostAI lists models from that account/region (Bedrock control plane or runtime) and exposes them in the Model Browser with source bedrock-direct.

Security and Compliance ​

  • Encryption: Bedrock credentials (composite key: region + access key + secret) are encrypted at rest using the same tenant-level envelope encryption as other BYOK keys. Keys are decrypted only in memory at the moment of use and are never written to logs or error messages.
  • Key isolation: Each tenant’s Bedrock key is stored and resolved in isolation. GhostAI never uses one tenant’s key for another tenant’s request.
  • Audit: Every Bedrock invocation can be tied to identity (user, org), model, route, and timestamp. Key set/update/delete operations are audited without logging raw secrets. Optional export of “all Bedrock usage in last N days” supports compliance reporting.
  • Data flow: When using Bedrock BYOK, prompt and completion data are sent from the user to GhostAI and from GhostAI to the customer’s Bedrock endpoint. GhostAI does not process or store LLM content for inference beyond routing and streaming. For private deployment, when GhostAI runs in the customer’s VPC, traffic to Bedrock can stay entirely on the customer’s private network.
  • Compliance alignment: Bedrock BYOK supports data-residency and “your cloud, your keys” requirements common in HIPAA, PCI-DSS, SOC 2, and government contracts. A short “Bedrock BYOK security checklist” (data flow, key storage, isolation, audit) is available for security and compliance teams.

SaaS vs. Private Deployment ​

AspectSaaS (GhostAI hosted by vendor)Private (GhostAI in customer environment)
Network path to BedrockVendor backend → (internet or PrivateLink) → customer Bedrock. Customer may expose Bedrock via AWS PrivateLink or allowlist.GhostAI and Bedrock both in customer control; traffic stays private (VPC-internal or same-org).
CredentialsCustomer key stored encrypted in vendor DB (ciphertext only). Optional custom endpoint URL for PrivateLink.Customer can use IAM role only (task/pod role); no stored Bedrock keys.
Data residency“Inference runs in your AWS”; call is initiated by vendor backend. One-pager and optionally PrivateLink required for strict policies.“All inference and app traffic stays inside your environment.”
ComplianceCustomer relies on vendor isolation, encryption, and logging; contract/BAA and compliance docs apply.Customer controls infra, keys, and logs; easier for strict infosec.

For regulated banks and similar customers, private deployment offers the strongest story: no internet leg for LLM traffic and no stored Bedrock secrets when using IAM. SaaS with Bedrock BYOK still keeps inference in the customer’s AWS; optional PrivateLink and custom endpoint URL allow “no public internet” for the vendor-to-Bedrock leg when the customer requires it.


Operational Guidance ​

  1. Configure Bedrock: In GhostAI, go to Settings → Provider Keys (or Admin → Provider Keys). Add AWS Bedrock with region and, for BYOK, Access Key ID and Secret Access Key (or leave credentials empty for IAM-only when GhostAI runs in AWS). Optionally set a custom endpoint URL (e.g. VPC/PrivateLink).
  2. Test before save: Use “Test before save” to validate the key and region against Bedrock (list models or minimal completion).
  3. Use the Model Browser: Bedrock models appear under source bedrock-direct. Select a Bedrock model for chat; tool use is supported where the model and Bedrock endpoint allow it.
  4. Lock or disable at org/platform: Admins can lock Bedrock so users cannot override with personal keys, or disable Bedrock for an org or platform-wide.
  5. Monitor health: Key health monitoring runs on a schedule; Bedrock keys are validated like other providers. Fix or rotate keys when health shows unhealthy.
  6. Private deployment: When GhostAI runs in your AWS (e.g. ECS, EKS), prefer IAM role (region only, no stored keys). Attach a task/pod role with minimal Bedrock permissions (bedrock:InvokeModel, list) and set the Bedrock region in configuration.

Contract and SLA Considerations ​

  • Data processing: When Customer uses Bedrock BYOK, LLM inference is performed solely in Customer’s AWS account; Vendor does not process or store prompt or completion content for that inference. (Optional contract/order language.)
  • SLA: When the customer uses their own Bedrock, availability and rate limits are per the customer’s AWS SLA and quotas. Vendor SLA applies to GhostAI availability, not to Bedrock.

Bedrock PII Referee ​

In addition to chat, Bedrock can serve as the PII referee backend — the LLM that verifies ambiguous PII detections. This gives regulated deployments best-in-class accuracy (Claude Sonnet 4 or Opus) while keeping raw text within the customer's AWS account.

Referee Architecture ​

The referee path differs from the chat path: tokenized prompts go through chat (no PII leaves GhostAI), but the referee operates on raw text to verify whether a span is truly PII. When using Bedrock for referee, raw text goes to the customer's own Bedrock endpoint — never to a shared cloud.

Configuration ​

  • Set PII_REFEREE_RUNTIME=bedrock and configure PII_REFEREE_BEDROCK_REGION, PII_REFEREE_BEDROCK_MODEL.
  • Credentials follow the same resolution: explicit keys > ProviderKeyService BYOK > IAM chain.
  • Admin UI: Settings → AI Protection → Referee Configuration allows dynamic model selection.
  • Hot-swap: Changing referee config via admin UI rebuilds the backend without application restart.

Required IAM Permissions ​

PermissionUsage
bedrock:InvokeModelReferee inference, chat, embeddings, image generation
bedrock:ListFoundationModelsDynamic model discovery in admin UI (optional; falls back to static list)

Minimal IAM policy example:

json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "bedrock:InvokeModel",
        "bedrock:ListFoundationModels"
      ],
      "Resource": "*"
    }
  ]
}

Bedrock Embeddings ​

Set MEMORY_EMBEDDING_BACKEND=bedrock to use Amazon Titan Embeddings V2 for memory and RAG instead of local sentence-transformers. Shares Bedrock credentials. Important: Switching embedding backends requires re-embedding all existing vectors via POST /api/admin/embeddings/rebuild.

Bedrock Image Generation ​

Set IMAGE_GENERATION_PROVIDER=bedrock and BEDROCK_IMAGE_MODEL to use Amazon Titan Image Generator or Stability AI models within your AWS account.


Conclusion ​

AWS Bedrock BYOK in GhostAI gives enterprises and regulated organizations a path to use their own Bedrock account for secure, compliant LLM chat, PII referee verification, embeddings, and image generation. The same multi-tier key resolution, audit, and lock/disable controls used for other BYOK providers apply to Bedrock. For the strongest data-residency and “no secrets in the app” story, deploy GhostAI in your own environment and use IAM role-only access to Bedrock; for SaaS, use BYOK with optional PrivateLink or custom endpoint so that inference stays in your AWS and, if required, off the public internet.


References ​

SOC 2 Type II Certified | BAA Available | HIPAA-Ready