Appearance
File Endpoints
Upload, scan, preview, and manage files with automatic PII detection and redaction.
Upload File
Upload a file for PII scanning and optional AI processing.
http
POST /api/files/upload
Content-Type: multipart/form-dataParameters
| Parameter | Type | Required | Description |
|---|---|---|---|
file | File | Yes | The file to upload |
scan | boolean | No | Run PII scan immediately (default: true) |
compressed | string | No | Set to "gzip" when sending a pre-compressed payload |
Max file size: 150MB.
Example
bash
curl -X POST http://localhost:8000/api/files/upload \
-H "Authorization: Bearer YOUR_TOKEN" \
-F "file=@report.pdf" \
-F "scan=true"Response
json
{
"file_id": "9c7124d6-4c4e-4943-a9fb-a684d9d93eba",
"filename": "report.pdf",
"content_type": "application/pdf",
"size": 245678,
"is_indexed": true,
"chunk_count": 12,
"scan_results": {
"pii_count": 14,
"entities": [
{"type": "PERSON", "count": 3, "confidence": 0.92},
{"type": "US_SSN", "count": 3, "confidence": 0.99},
{"type": "DATE_TIME", "count": 5, "confidence": 0.85}
],
"compliance_tags": ["HIPAA"],
"image_detections": [],
"has_sensitive_image_content": false,
"is_encrypted": false
}
}Get File Preview
Get file metadata, extracted text, and PII scan results for rendering a preview.
http
GET /api/files/{file_id}/previewResponse
json
{
"file_id": "9c7124d6-4c4e-4943-a9fb-a684d9d93eba",
"filename": "report.pdf",
"content_type": "application/pdf",
"text_content": "Patient John Smith, SSN 123-45-6789...",
"sanitized_text": "Patient [PERSON], SSN [US_SSN]...",
"pii_entities": [
{
"type": "PERSON",
"text": "John Smith",
"start": 8,
"end": 18,
"confidence": 0.92
}
],
"pii_redacted": true,
"has_sensitive_image_content": false,
"is_encrypted": false,
"code_detections": [],
"image_detections": []
}Get File (Download)
Download the file content. Returns the redacted version by default.
http
GET /api/files/{file_id}
GET /api/files/{file_id}?original=1Parameters
| Parameter | Type | Required | Description |
|---|---|---|---|
original | boolean | No | If 1, returns original unredacted file (admin only) |
Access Control
| Role | Redacted | Original |
|---|---|---|
| Member | Yes | No |
| Admin | Yes | Yes |
| Owner | Yes | Yes |
WARNING
Accessing original (unredacted) files requires admin or owner privileges. All access is audit-logged.
Example
bash
# Get redacted version (any user)
curl http://localhost:8000/api/files/{file_id} \
-H "Authorization: Bearer YOUR_TOKEN" \
-o redacted_report.pdf
# Get original version (admin only)
curl http://localhost:8000/api/files/{file_id}?original=1 \
-H "Authorization: Bearer YOUR_TOKEN" \
-o original_report.pdfConvert to PDF
Convert a file (PPTX, DOCX, etc.) to PDF format for preview rendering. Requires LibreOffice on the server.
http
GET /api/files/{file_id}/as-pdf
HEAD /api/files/{file_id}/as-pdfParameters
| Parameter | Type | Required | Description |
|---|---|---|---|
original | boolean | No | Convert original file (admin only) |
Status Codes
| Code | Meaning |
|---|---|
| 200 | Success — PDF content returned |
| 404 | File not found |
| 422 | File is password-protected/encrypted |
| 501 | LibreOffice not installed on server |
| 504 | Conversion timed out |
Example
bash
# Check if conversion is available
curl -I http://localhost:8000/api/files/{file_id}/as-pdf \
-H "Authorization: Bearer YOUR_TOKEN"
# Download converted PDF
curl http://localhost:8000/api/files/{file_id}/as-pdf \
-H "Authorization: Bearer YOUR_TOKEN" \
-o presentation.pdfDownload Redacted Copy
Download a redacted copy of the file with all PII removed.
http
GET /api/files/{file_id}/download-redactedResponse
Returns the file content with:
Content-Disposition: attachmentheaderX-PII-Redacted: trueheader- Redacted file content (PII replaced/blurred)
Scan File for PII
Trigger a PII scan on an already-uploaded file.
http
POST /api/files/{file_id}/scan
Content-Type: application/jsonParameters
| Parameter | Type | Required | Description |
|---|---|---|---|
sample_rows | number | No | Scan only first N rows for quick results |
Response
json
{
"file_id": "9c7124d6-4c4e-4943-a9fb-a684d9d93eba",
"scan_status": "complete",
"pii_count": 14,
"entities": [...],
"compliance_tags": ["HIPAA"],
"scan_time_ms": 1234,
"is_sample": false,
"sample_rows": null,
"total_rows": 1250
}Delete File
Delete an uploaded file and all associated data.
http
DELETE /api/files/{file_id}Response
json
{
"deleted": true,
"file_id": "9c7124d6-4c4e-4943-a9fb-a684d9d93eba"
}Get File Limits
Get current file upload limits and usage.
http
GET /api/files/info/limitsResponse
json
{
"max_file_size": 157286400,
"max_files_per_session": 10,
"allowed_types": ["application/pdf", "image/*", "text/*"],
"current_usage": {
"files_count": 3,
"total_size": 1234567
}
}RAG Configuration
RAG (Retrieval-Augmented Generation) settings control how files are indexed and retrieved for chat. Configure via environment variables or Admin Dashboard → RAG / File Intelligence.
| Setting | Type | Default | Description |
|---|---|---|---|
rag_embedding_device | string | auto | Embedding device: auto, cpu, cuda, mps |
rag_hybrid_search_enabled | boolean | true | Enable BM25 + semantic hybrid search |
rag_ann_enabled | boolean | false | Use HNSW approximate nearest neighbor |
rag_ann_threshold | number | 100000 | Chunk count threshold to enable ANN |
rag_ann_ef_search | number | 100 | HNSW ef_search parameter |
rag_pii_cache_enabled | boolean | true | Cache PII detection per chunk hash |
rag_streaming_enabled | boolean | true | Stream large text files instead of loading into memory |
rag_streaming_threshold | number | 52428800 | File size (bytes) above which streaming is used (default: 50 MB) |
Compare Mode RAG Threshold
In comparison mode, files are sent to 2-4 models simultaneously. To reduce payload duplication and latency, comparison mode uses a lower RAG indexing threshold:
| Constant | Value | Context |
|---|---|---|
RAG_THRESHOLD | 1 MB | Regular chat -- files below this are injected as full text |
RAG_THRESHOLD_COMPARE | 10 KB | Comparison chat -- files above this are indexed for RAG retrieval |
Files between 10KB and 1MB that are not already indexed are indexed on-the-fly when used in a comparison. This ensures only the most relevant chunks are sent to each model rather than the full document text repeated 4 times.
PII Scan Caching
When a file is scanned at upload time (POST /api/files/{file_id}/scan), the PII detection results are cached on the attachments row. During chat, the cached results are reused for non-RAG attachments, eliminating redundant Presidio detection. This is particularly impactful in comparison mode where the same file is attached to all model requests.
Streaming Ingestion
For CSV, TSV, and TXT files larger than rag_streaming_threshold (default 50 MB), GhostAI streams the file instead of loading it entirely into memory. This reduces memory usage and enables processing of very large datasets without timeout or OOM errors.
Response Headers
All file responses include these headers:
| Header | Description |
|---|---|
X-PII-Redacted | true if the file has been redacted |
Content-Disposition | inline or attachment depending on file type |
Cache-Control | no-cache for redacted, private, max-age=3600 for originals |
Get Threat Analysis
Poll for AI threat analysis results on a file's security findings.
http
GET /api/files/{file_id}/threat-analysisResponse includes findings array with AI fields and analysis_complete boolean. Poll every 2s until complete.
Response fields on each finding:
- ai_status: pending | analyzing | complete | skipped | unavailable | inconclusive | error
- ai_verdict: malicious | suspicious | benign | inconclusive
- ai_explanation: human-readable explanation
- ai_confidence: 0.0-1.0
- adjusted_risk_level: AI-adjusted severity
- override_source: heuristic | admin_policy | confidence_floor | null
Submit Threat Feedback
http
POST /api/files/{file_id}/threat-feedbackBody: { finding_type: string, feedback: "agree" | "disagree_malicious" | "disagree_benign" }
Trigger Batch Re-Analysis (Admin)
http
POST /api/files/admin/threat-reanalysisBody: { scope: "all" | "type:<type>" | "since:<date>" }