Skip to content

Native Deployment (Mac / Linux) ​

Run GhostAI directly on your machine. Uses Python (uvicorn), Node.js (Vite), and a local PostgreSQL container (started automatically by restart.sh).

Architecture ​

Prerequisites ​

RequirementVersionCheck
Python3.11+python3 --version
Node.js18+node --version
npm9+npm --version
Docker20.10+docker --version

Docker is required for the local PostgreSQL container. restart.sh starts it automatically.

Optional for local LLM:

RequirementPurposeInstall
OllamaLocal chat + PII modelsbrew install ollama (macOS)
psutilNative stress dashboardpip install psutil

Quick Start ​

1. Configure Environment ​

bash
cd securechat/backend
cp .env.example .env

Edit .env with your chosen LLM provider (see examples below). Provider API keys are automatically registered as platform defaults on first startup, making them visible in the admin UI.

2. Start Services ​

bash
# Using the deploy script
./infrastructure/scripts/targets/native.sh up

# Or manually:
cd securechat/backend
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
uvicorn app.main:app --host 0.0.0.0 --port 8000 &

cd ../frontend
npm install && npm run dev &

3. Verify ​

bash
# Check backend health
curl http://localhost:8000/health

# Check frontend
open http://localhost:5173

Example Configurations ​

Best balance: cloud-quality chat with local PII protection. Your sensitive text never leaves the machine.

bash
# --- Provider ---
LLM_PROVIDER=openai
OPENAI_API_KEY=sk-...
DEFAULT_MODEL=gpt-4o-mini

# --- PII (runs locally) ---
PII_ENABLE_AI_DETECTOR=true
PII_AI_MODEL=local
PII_REFEREE_RUNTIME=llama-cpp
PII_REFEREE_MODEL_NAME=auto           # Picks phi-3.5-mini on Apple Silicon
PII_REFEREE_MODEL_GPU_LAYERS=-1       # Metal acceleration
PII_REFEREE_MODEL_THREADS=0           # Auto CPU cores
PII_REFEREE_WARMUP=true               # Pre-load at startup

# --- Database ---
# PostgreSQL is started automatically by restart.sh (Docker container)
# DATABASE_URL=postgresql://ghostai:ghostai_dev_password@localhost:5432/ghostai

B) Fully Local with Ollama (Air-Gapped) ​

Zero cloud dependencies. Everything runs on your Mac. Works offline.

bash
# --- Provider (local) ---
LLM_PROVIDER=ollama
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_DEFAULT_MODEL=llama3.2

# --- PII (also local via Ollama) ---
PII_ENABLE_AI_DETECTOR=true
PII_AI_MODEL=local
PII_REFEREE_RUNTIME=ollama
PII_REFEREE_OLLAMA_MODEL=llama3.1:8b  # Or phi3.5 for smaller Macs
PII_REFEREE_MAX_CONCURRENT=4

# No API keys needed!

Setup Ollama first:

bash
brew install ollama
ollama serve &                  # Start daemon
ollama pull llama3.2            # Chat model (~2GB)
ollama pull llama3.1:8b         # PII referee model (~4.5GB)

C) OpenRouter Multi-Model + Local PII ​

Access Claude, Gemini, DeepSeek, and more through one API key. PII stays local.

bash
# --- Provider ---
LLM_PROVIDER=openrouter
OPENROUTER_API_KEY=sk-or-...
OPENROUTER_DEFAULT_MODEL=anthropic/claude-3.5-sonnet

# --- PII (runs locally) ---
PII_ENABLE_AI_DETECTOR=true
PII_AI_MODEL=local
PII_REFEREE_RUNTIME=llama-cpp
PII_REFEREE_MODEL_GPU_LAYERS=-1
PII_REFEREE_WARMUP=true

D) Pure CPU (No GPU, Budget Hardware) ​

For machines without Metal or CUDA. Uses the smallest models.

bash
LLM_PROVIDER=ollama
OLLAMA_DEFAULT_MODEL=phi3.5         # Small, fast model

PII_AI_MODEL=local
PII_REFEREE_RUNTIME=llama-cpp
PII_REFEREE_MODEL_NAME=distil-pii-1b  # 1B params, only 0.7GB RAM
PII_REFEREE_MODEL_GPU_LAYERS=0        # Force CPU only
PII_REFEREE_MODEL_THREADS=4           # Limit CPU threads

Process Management ​

The native deploy script manages processes via PID files in ~/.config/ghostai/pids/:

bash
# Start all services
./infrastructure/scripts/targets/native.sh up

# Check status
./infrastructure/scripts/targets/native.sh status

# View logs
./infrastructure/scripts/targets/native.sh logs

# Stop everything
./infrastructure/scripts/targets/native.sh down

# Full reset (wipe data + restart)
./infrastructure/scripts/targets/native.sh wipe

Monitoring ​

Use the native stress dashboard to monitor running processes:

bash
python infrastructure/scripts/stress-dashboard-native.py
# Opens http://localhost:9090 with live CPU/memory/PostgreSQL metrics

See Stress Test Dashboards for details.

Troubleshooting ​

Ollama connection refused ​

ConnectionError: Cannot connect to Ollama at localhost:11434

Ensure the daemon is running: ollama serve &. On macOS with Docker Desktop, the socket may be at ~/.docker/run/docker.sock -- but Ollama is a separate binary, not Docker.

llama-cpp-python installation fails ​

bash
# macOS (Metal support built-in on Apple Silicon)
pip install llama-cpp-python

# Linux with CUDA
CMAKE_ARGS="-DLLAMA_CUDA=on" pip install llama-cpp-python

# Linux CPU only
pip install llama-cpp-python

Port already in use ​

bash
# Find what's using the port
lsof -i :8000
# Kill it
kill <PID>

Model download stuck ​

GGUF models are downloaded to securechat/backend/models/ on first use. Large models (4-5GB) may take a few minutes. Check progress in the backend logs.

SOC 2 Type II Certified | BAA Available | HIPAA-Ready