Compare commits

...
Author SHA1 Message Date
Nicolò Boschi da52a7e74d docs(skill): regenerate hindsight-docs skill mirror 2026-04-27 17:16:10 +02:00
Nicolò Boschi 2aff0f7a15 docs(installation): leave Docker Image Variants table alone, soften GPU note
- Revert the Idle RAM column on the Docker Image Variants table; the
  Hardware subsection already carries that detail.
- Reword the CPU/GPU line: CPU is fine for dev and basic workloads, but
  the local cross-encoder reranker typically benefits from a GPU under
  production traffic — or offload reranking to an external provider.
2026-04-27 17:04:23 +02:00
Nicolò Boschi 01f45fd069 docs(installation): document memory footprint and hardware requirements
Add a Hardware subsection under Prerequisites with per-component RAM
guidance (full vs slim image, control plane, worker, postgres) and
extend the Docker Image Variants table with an Idle RAM column so users
know what to provision before deploying.
2026-04-27 17:01:27 +02:00
2 changed files with 36 additions and 0 deletions
@@ -45,6 +45,24 @@ Configure which one to use with `HINDSIGHT_API_VECTOR_EXTENSION`. See [Configura
You need an LLM API key for fact extraction, entity resolution, and answer generation. See [Models](./models) for supported providers, model recommendations, and configuration.
### Hardware
Hindsight is designed to run on commodity hardware. The footprint depends mainly on whether the **full** image (which bundles local embedding and reranker models) or the **slim** image (which delegates those to external providers) is used.
| Component | Minimum RAM | Recommended RAM | Notes |
|-----------|-------------|-----------------|-------|
| **API — Full image** | 1.5 GB | 2 GB | Loads local BGE embedder (~130 MB) and MiniLM cross-encoder (~90 MB) into memory, plus PyTorch/ONNX runtime arenas. Idle RSS settles around 0.81.0 GB; expect 1.21.5 GB under load. |
| **API — Slim image** | 512 MB | 1 GB | No local models. Steady-state RSS is dominated by Python runtime and DB connections. Requires [external embedding and reranker providers](./configuration#embeddings) (e.g. TEI, OpenAI, Cohere). |
| **Control Plane (UI)** | 128 MB | 256 MB | Next.js process, lightweight. |
| **Worker** (if separated) | Same as API image variant | Same as API image variant | Workers load the same models as the API server. |
| **PostgreSQL** | 512 MB | 1 GB+ | Scales with the number of memories and indexes. |
:::tip Reducing the footprint
The bulk of the full image's memory comes from the bundled embedding and reranker models and their PyTorch/ONNX runtimes. To shrink the deployment to a few hundred MB of RAM, switch to the **slim** image and configure [external embedding and reranker providers](./configuration#embeddings).
:::
CPU vs GPU: 2 vCPUs on CPU-only is fine for development and basic workloads. For production traffic, the local reranker (cross-encoder) is the main bottleneck and typically benefits from a GPU to keep recall latency reasonable; alternatively, offload reranking to an [external reranker provider](./configuration#embeddings) (e.g. TEI, Cohere) on dedicated GPU hardware.
---
## Docker
@@ -45,6 +45,24 @@ Configure which one to use with `HINDSIGHT_API_VECTOR_EXTENSION`. See [Configura
You need an LLM API key for fact extraction, entity resolution, and answer generation. See [Models](./models) for supported providers, model recommendations, and configuration.
### Hardware
Hindsight is designed to run on commodity hardware. The footprint depends mainly on whether the **full** image (which bundles local embedding and reranker models) or the **slim** image (which delegates those to external providers) is used.
| Component | Minimum RAM | Recommended RAM | Notes |
|-----------|-------------|-----------------|-------|
| **API — Full image** | 1.5 GB | 2 GB | Loads local BGE embedder (~130 MB) and MiniLM cross-encoder (~90 MB) into memory, plus PyTorch/ONNX runtime arenas. Idle RSS settles around 0.81.0 GB; expect 1.21.5 GB under load. |
| **API — Slim image** | 512 MB | 1 GB | No local models. Steady-state RSS is dominated by Python runtime and DB connections. Requires [external embedding and reranker providers](./configuration#embeddings) (e.g. TEI, OpenAI, Cohere). |
| **Control Plane (UI)** | 128 MB | 256 MB | Next.js process, lightweight. |
| **Worker** (if separated) | Same as API image variant | Same as API image variant | Workers load the same models as the API server. |
| **PostgreSQL** | 512 MB | 1 GB+ | Scales with the number of memories and indexes. |
:::tip Reducing the footprint
The bulk of the full image's memory comes from the bundled embedding and reranker models and their PyTorch/ONNX runtimes. To shrink the deployment to a few hundred MB of RAM, switch to the **slim** image and configure [external embedding and reranker providers](./configuration#embeddings).
:::
CPU vs GPU: 2 vCPUs on CPU-only is fine for development and basic workloads. For production traffic, the local reranker (cross-encoder) is the main bottleneck and typically benefits from a GPU to keep recall latency reasonable; alternatively, offload reranking to an [external reranker provider](./configuration#embeddings) (e.g. TEI, Cohere) on dedicated GPU hardware.
---
## Docker