Appearance
Native Deployment (Mac / Linux)
Run GhostAI directly on your machine. Uses Python (uvicorn), Node.js (Vite), and a local PostgreSQL container (started automatically by restart.sh).
Architecture
Prerequisites
| Requirement | Version | Check |
|---|---|---|
| Python | 3.11+ | python3 --version |
| Node.js | 18+ | node --version |
| npm | 9+ | npm --version |
| Docker | 20.10+ | docker --version |
Docker is required for the local PostgreSQL container. restart.sh starts it automatically.
Optional for local LLM:
| Requirement | Purpose | Install |
|---|---|---|
| Ollama | Local chat + PII models | brew install ollama (macOS) |
| psutil | Native stress dashboard | pip install psutil |
Quick Start
1. Configure Environment
bash
cd securechat/backend
cp .env.example .envEdit .env with your chosen LLM provider (see examples below). Provider API keys are automatically registered as platform defaults on first startup, making them visible in the admin UI.
2. Start Services
bash
# Using the deploy script
./infrastructure/scripts/targets/native.sh up
# Or manually:
cd securechat/backend
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
uvicorn app.main:app --host 0.0.0.0 --port 8000 &
cd ../frontend
npm install && npm run dev &3. Verify
bash
# Check backend health
curl http://localhost:8000/health
# Check frontend
open http://localhost:5173Example Configurations
A) Cloud Chat + Local PII (Recommended for Mac)
Best balance: cloud-quality chat with local PII protection. Your sensitive text never leaves the machine.
bash
# --- Provider ---
LLM_PROVIDER=openai
OPENAI_API_KEY=sk-...
DEFAULT_MODEL=gpt-4o-mini
# --- PII (runs locally) ---
PII_ENABLE_AI_DETECTOR=true
PII_AI_MODEL=local
PII_REFEREE_RUNTIME=llama-cpp
PII_REFEREE_MODEL_NAME=auto # Picks phi-3.5-mini on Apple Silicon
PII_REFEREE_MODEL_GPU_LAYERS=-1 # Metal acceleration
PII_REFEREE_MODEL_THREADS=0 # Auto CPU cores
PII_REFEREE_WARMUP=true # Pre-load at startup
# --- Database ---
# PostgreSQL is started automatically by restart.sh (Docker container)
# DATABASE_URL=postgresql://ghostai:ghostai_dev_password@localhost:5432/ghostaiB) Fully Local with Ollama (Air-Gapped)
Zero cloud dependencies. Everything runs on your Mac. Works offline.
bash
# --- Provider (local) ---
LLM_PROVIDER=ollama
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_DEFAULT_MODEL=llama3.2
# --- PII (also local via Ollama) ---
PII_ENABLE_AI_DETECTOR=true
PII_AI_MODEL=local
PII_REFEREE_RUNTIME=ollama
PII_REFEREE_OLLAMA_MODEL=llama3.1:8b # Or phi3.5 for smaller Macs
PII_REFEREE_MAX_CONCURRENT=4
# No API keys needed!Setup Ollama first:
bash
brew install ollama
ollama serve & # Start daemon
ollama pull llama3.2 # Chat model (~2GB)
ollama pull llama3.1:8b # PII referee model (~4.5GB)C) OpenRouter Multi-Model + Local PII
Access Claude, Gemini, DeepSeek, and more through one API key. PII stays local.
bash
# --- Provider ---
LLM_PROVIDER=openrouter
OPENROUTER_API_KEY=sk-or-...
OPENROUTER_DEFAULT_MODEL=anthropic/claude-3.5-sonnet
# --- PII (runs locally) ---
PII_ENABLE_AI_DETECTOR=true
PII_AI_MODEL=local
PII_REFEREE_RUNTIME=llama-cpp
PII_REFEREE_MODEL_GPU_LAYERS=-1
PII_REFEREE_WARMUP=trueD) Pure CPU (No GPU, Budget Hardware)
For machines without Metal or CUDA. Uses the smallest models.
bash
LLM_PROVIDER=ollama
OLLAMA_DEFAULT_MODEL=phi3.5 # Small, fast model
PII_AI_MODEL=local
PII_REFEREE_RUNTIME=llama-cpp
PII_REFEREE_MODEL_NAME=distil-pii-1b # 1B params, only 0.7GB RAM
PII_REFEREE_MODEL_GPU_LAYERS=0 # Force CPU only
PII_REFEREE_MODEL_THREADS=4 # Limit CPU threadsProcess Management
The native deploy script manages processes via PID files in ~/.config/ghostai/pids/:
bash
# Start all services
./infrastructure/scripts/targets/native.sh up
# Check status
./infrastructure/scripts/targets/native.sh status
# View logs
./infrastructure/scripts/targets/native.sh logs
# Stop everything
./infrastructure/scripts/targets/native.sh down
# Full reset (wipe data + restart)
./infrastructure/scripts/targets/native.sh wipeMonitoring
Use the native stress dashboard to monitor running processes:
bash
python infrastructure/scripts/stress-dashboard-native.py
# Opens http://localhost:9090 with live CPU/memory/PostgreSQL metricsSee Stress Test Dashboards for details.
Troubleshooting
Ollama connection refused
ConnectionError: Cannot connect to Ollama at localhost:11434Ensure the daemon is running: ollama serve &. On macOS with Docker Desktop, the socket may be at ~/.docker/run/docker.sock -- but Ollama is a separate binary, not Docker.
llama-cpp-python installation fails
bash
# macOS (Metal support built-in on Apple Silicon)
pip install llama-cpp-python
# Linux with CUDA
CMAKE_ARGS="-DLLAMA_CUDA=on" pip install llama-cpp-python
# Linux CPU only
pip install llama-cpp-pythonPort already in use
bash
# Find what's using the port
lsof -i :8000
# Kill it
kill <PID>Model download stuck
GGUF models are downloaded to securechat/backend/models/ on first use. Large models (4-5GB) may take a few minutes. Check progress in the backend logs.