feat(agent-plugin): add portable Hindsight plugin for the Agent Plugins standard

Add a vendor-neutral Hindsight plugin conforming to Vercel's Agent Plugins
1.0.0 standard (plugin.json + mcp.json + skills/SKILL.md), so one artifact
gives long-term memory to any compatible client (Codex, Cursor, GitHub
Copilot, Kiro, VS Code) instead of a per-IDE integration. The plugin is a
thin transport wrapper over Hindsight's existing MCP server (retain / recall
/ reflect); a bundled skill teaches the agent when to use it.

Wiring:
- CI: test-agent-plugin-integration job runs the manifest validator, gated on
  hindsight-integrations/agent-plugin/** changes.
- Docs: integrations.json gallery entry + docs-integrations/agent-plugin.md.
- Release: agent-plugin added to release-integration.sh and the changelog
  generator; both learn to read a root-level plugin.json and link the
  changelog to the source tree (git-distributed bundle, no registry package).

.claude-plugin/marketplace.json

@@ -0,0 +1,21 @@
+{
+  "$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
+  "name": "hindsight",
+  "version": "0.7.5",
+  "description": "Official Hindsight integrations for Claude Code",
+  "owner": {
+    "name": "vectorize-io"
+  },
+  "plugins": [
+    {
+      "name": "hindsight-memory",
+      "description": "Automatic long-term memory for Claude Code via Hindsight",
+      "source": "./hindsight-integrations/claude-code"
+    },
+    {
+      "name": "hindsight-zcode",
+      "description": "No-MCP long-term memory for ZCode via Hindsight hooks",
+      "source": "./hindsight-integrations/zcode"
+    }
+  ]
+}

.claude/skills/code-review/SKILL.md

@@ -0,0 +1,272 @@
+---
+name: code-review
+description: Review changed code against project standards. Checks for missing tests, dead code, type safety, lint issues, and coding conventions. Run after completing any implementation work.
+user_invocable: true
+---
+
+# Code Review
+
+Review all changed code against the project's quality standards and coding conventions.
+
+## Code Standards
+
+Read and internalize these standards before writing code. The review steps below verify compliance.
+
+### Python Style
+- Python 3.11+, type hints required
+- Async throughout (asyncpg, async FastAPI)
+- Pydantic models for request/response
+- Ruff for linting (line-length 120)
+- No Python files at project root - maintain clean directory structure
+- **Never use multi-item tuple return values** — not even for internal/private functions. Always use a dataclass or Pydantic model. No exceptions, no "it's just two values" shortcuts. If a function returns more than one value, define a named type for it.
+
+### Type Safety with Pydantic Models
+**NEVER use raw `dict` types for structured data** — this applies to all code, including internal helpers and private functions. If the dict has known keys, it must be a dataclass or Pydantic model:
+- Use Pydantic `BaseModel` for all data structures passed between functions
+- Use `@dataclass` for lightweight internal data containers when Pydantic validation isn't needed
+- Add `@field_validator` for type coercion (e.g., ensuring datetimes are timezone-aware)
+- Avoid `dict.get()` patterns - use typed model attributes instead
+- Parse external data (JSON, API responses) into Pydantic models at the boundary
+- This catches type errors at parse time, not deep in business logic
+- The only acceptable `dict` usage is for truly dynamic/unknown keys (e.g., arbitrary metadata, JSON blobs with no fixed schema)
+
+```python
+# BAD - error-prone dict access
+def process(data: dict) -> str:
+    return data.get("name", "")  # No validation, silent failures
+
+# GOOD - typed and validated
+class UserData(BaseModel):
+    name: str
+    created_at: datetime
+
+    @field_validator("created_at", mode="before")
+    @classmethod
+    def ensure_tz_aware(cls, v):
+        if isinstance(v, str):
+            v = datetime.fromisoformat(v.replace("Z", "+00:00"))
+        if v.tzinfo is None:
+            return v.replace(tzinfo=timezone.utc)
+        return v
+
+def process(data: UserData) -> str:
+    return data.name  # Type-safe, validated at construction
+```
+
+### TypeScript Style
+- Next.js App Router for control plane
+- Tailwind CSS with shadcn/ui components
+
+### Code Comments
+- **Always comment non-trivial technical decisions** with the reasoning behind the choice. If someone would ask "why is it done this way?", there should be a comment.
+- **Keep comments up to date with history** — when changing an approach, update the comment to explain what was tried before and why it was changed. Comments serve as a tracker of previous implementations that likely had problems.
+- Don't comment obvious code — only where the "why" isn't self-evident from the code itself.
+
+```python
+# BAD - no context for future readers
+results = await asyncio.gather(*tasks, return_exceptions=True)
+
+# GOOD - explains the non-obvious choice
+# Use return_exceptions=True to avoid cancelling sibling tasks on failure.
+# Previously we used TaskGroup but it cancelled all tasks when one failed,
+# causing partial writes that left orphaned entity links (see #412).
+results = await asyncio.gather(*tasks, return_exceptions=True)
+```
+
+### API Layer & Data Access
+- **No direct database access in `api/http.py`** (or any API router). HTTP handlers must not build SQL, call `acquire_with_retry` / `conn.fetch` / `conn.fetchrow` / `conn.execute`, or reference `fq_table(...)`. All persistence and queries live in `MemoryEngine` (the engine layer). A handler parses/validates the request, calls an engine method, shapes the HTTP response, and maps domain results to status codes (e.g. a `None` return → 404).
+- **Authentication/tenancy is enforced inside each engine method, not assumed by the handler.** Every engine method that touches bank-scoped data must authenticate via `request_context` — typically `await self._authenticate_tenant(request_context)` (often indirectly through `get_bank_profile(...)`) — so the correct tenant schema is resolved before any query runs. Handlers must thread `request_context` through to the engine method; never query a tenant-scoped table assuming the schema is already set.
+- Engine methods return typed models (Pydantic/dataclass), not raw dicts (see Type Safety).
+
+### Database Locking
+- **Never use PostgreSQL advisory locks** (`pg_advisory_lock`, `pg_try_advisory_lock`, `pg_advisory_xact_lock`, `pg_advisory_unlock`, …) in migrations, engine code, or anything else. Hindsight runs against connection poolers and managed/PG-compatible services where advisory locks are unreliable or unsupported: session-level locks silently leak or vanish when a pooler hands the session to another client, and callers can block forever on a lock the server never grants. Reject any new occurrence, including ones that look "safe" because they are transaction-scoped.
+- The pre-existing usage in `hindsight_api/migrations.py` is grandfathered, not a precedent — it is tracked for removal. Don't copy it.
+- Design the concurrency out instead of locking around it: give each process its own object to write (e.g. per-schema DDL rather than a shared `public.` object), make the operation idempotent, or use a real row/table constraint (`INSERT ... ON CONFLICT`, `SELECT ... FOR UPDATE` in a fixed order). See #2690 for a migration that reached for `pg_advisory_xact_lock` and had to be reverted.
+
+### Branch Hygiene
+- **Always start new feature branches from `origin/main`** — rebase to ensure a clean base.
+- **Only include commits relevant to the PR/branch/feature** — no unrelated changes. If the branch contains commits that don't belong, they must be removed before merging.
+
+### General Principles
+- Don't add features, refactor code, or make "improvements" beyond what was asked
+- Don't add unnecessary error handling for impossible scenarios
+- Don't create helpers or abstractions for one-time operations
+- No backwards-compatibility hacks (unused vars, re-exports, "removed" comments)
+- Three similar lines of code is better than a premature abstraction
+
+## Review Steps
+
+### 1. Check branch hygiene
+
+- Run `git log --oneline main..HEAD` to list all commits on the branch.
+- Verify every commit is relevant to the feature/PR. Flag any unrelated commits.
+- Check the branch is based on a recent `origin/main` (no stale base).
+
+### 2. Identify changed files
+
+Run `git diff --name-only HEAD` (unstaged) and `git diff --cached --name-only` (staged) to get all changed files. If there are no local changes, diff against the base branch using `git diff main...HEAD --name-only` and `git diff main...HEAD` to review all commits on the current branch.
+
+### 3. Run linters
+
+```bash
+./scripts/hooks/lint.sh
+```
+
+Report any failures. Do NOT fix them yourself — just report.
+
+### 4. Check for dead code
+
+For each changed Python file, check for:
+- Unused imports (Ruff should catch these, but verify)
+- Functions/methods/classes that were added but are never called from anywhere
+- Variables assigned but never read
+- Commented-out code blocks that should be removed
+
+For each changed TypeScript file, check for:
+- Unused imports
+- Unused variables or functions
+- Commented-out code
+
+### 5. Check type safety (Python)
+
+For each changed Python file, check for violations:
+- **No raw `dict` for structured data** — must use Pydantic model or dataclass, even for internal/private functions (only exception: truly dynamic/unknown keys)
+- **No multi-item tuple returns** — must use dataclass or Pydantic model, even for internal/private functions (no exceptions)
+- **Missing type hints** on function parameters and return types
+- **Missing `@field_validator`** for datetime fields that should be timezone-aware
+
+### 6. Check for missing tests
+
+For each new or significantly changed function/endpoint/class:
+- Check if there is a corresponding test addition or update
+- New API endpoints MUST have integration tests
+- New utility functions MUST have unit tests
+- Bug fixes SHOULD have a regression test
+
+Flag any new logic that lacks test coverage.
+
+**LLM-behaviour changes need a real-LLM judge test, not MockLLM.** If the change alters how the model interprets a prompt — fact/observation extraction, `fact_type` (world/experience) classification, speaker attribution, instruction-following, prompt wording — there MUST be a test marked `pytest.mark.hs_llm_core` that runs the real pipeline and asserts via `tests.llm_judge.assert_meets_criteria` (not string/enum matching). Flag these as findings:
+- A prompt/classification change verified only by MockLLM or string assertions (MockLLM echoes input — such tests pass spuriously). **Should fix.**
+- A test that hard-asserts `fact_type == "world"/"experience"` (or other model-decided output) instead of judging it — non-deterministic, will flake across providers/runs. **Should fix** (move the classification check into the judge `criteria`; keep only genuinely deterministic structural asserts direct).
+- Deterministic mechanics (prompt assembly, suppression/branching logic) that are covered *only* by a slow LLM test — these should also have fast non-LLM unit tests. **Note.**
+
+See CLAUDE.md → Key Conventions → Testing for the full pattern.
+
+### 7. Check API consistency
+
+If any files in `hindsight-api-slim/hindsight_api/api/` were changed:
+- Were the OpenAPI specs regenerated? (`./scripts/generate-openapi.sh`)
+- Were the client SDKs regenerated? (`./scripts/generate-clients.sh`)
+- Were the control plane proxy routes updated? (`hindsight-control-plane/src/app/api/`)
+
+### 7a. Check TS/Python wrapper-client parity
+
+Two of the generated SDKs ship a **hand-written, maintained convenience wrapper** on top of the auto-generated low-level client — and *only* these two:
+- **TypeScript**: `hindsight-clients/typescript/src/index.ts` (`HindsightClient`)
+- **Python**: `hindsight-clients/python/hindsight_client/hindsight_client.py` (`Hindsight`)
+
+(The Rust/Go/etc. clients are generated-only — no wrapper to keep in sync.)
+
+These wrappers are what most third-party consumers actually call, and they must expose the same surface. **If a change touches one wrapper's method — adds/removes a parameter, changes a default, forwards a new query/body field — the equivalent method in the *other* wrapper must get the same change in the same (or an immediately-following) PR.** A parameter that exists in the generated SDK but is dropped by one wrapper silently strips it for every consumer of that language (this is exactly what #2975 / #3042 fixed for `detail`/`tags_match`/`limit`/`offset` on `listMentalModels`/`getMentalModel`). **Should fix** — flag any wrapper method that gains capabilities in one language but not the other, and add a matching mapping regression test on both sides.
+
+Note: the `client-coverage-check` CI tool only validates **request-body** fields, not GET **query** parameters — so query-param parity gaps are *not* caught automatically and must be checked by hand here.
+
+### 7b. Check API-layer data-access boundary
+
+For each changed handler in `hindsight-api-slim/hindsight_api/api/` (e.g. `http.py`, `mcp.py`):
+- **Flag any direct DB access in the handler** — `acquire_with_retry`, `conn.fetch` / `fetchrow` / `execute`, raw SQL strings, or `fq_table(...)`. These are a **must fix**: the query must be moved into a `MemoryEngine` method that returns a typed model, and the handler must call that method.
+- **Verify authentication is enforced in the engine** — the handler must delegate to an engine method that authenticates via `request_context` (`_authenticate_tenant`, typically through `get_bank_profile`). A handler that reads/writes tenant-scoped data without an engine method enforcing auth is a **must fix** (tenant data could leak across schemas).
+
+### 8. Check code comments
+
+For each non-trivial change:
+- **New non-obvious logic** — is there a comment explaining the reasoning?
+- **Changed approach** — does the comment include what was done before and why it changed?
+- **Stale comments** — do existing comments near the changed code still accurately describe the behavior?
+
+### 9. Check integration completeness
+
+If any files in `hindsight-integrations/` were added or changed, verify:
+- **Tests exist** — the integration must have tests that simulate/exercise the external framework (not just pure unit tests of helpers). Check for a `tests/` directory with meaningful test files.
+- **CI job exists** — check `.github/workflows/test.yml` for a corresponding `test-<name>-integration` job. If missing, flag it.
+- **Release process** — check that the integration name is in the `VALID_INTEGRATIONS` array in `scripts/release-integration.sh` AND in the `INTEGRATIONS` dict in `hindsight-dev/hindsight_dev/generate_changelog.py` (the changelog generator keeps its own list; a release fails at the changelog step if the name is missing there). If either is missing, flag it.
+- **Docs gallery + sidebar entry** — the integration must have an entry in `hindsight-docs/src/data/integrations.json`. This file is the **single source of truth** that drives both the integrations gallery and the docs sidebar (the sidebar category is injected from it at render time across all docs versions). The entry needs an internal `/sdks/integrations/<slug>` `link` and a matching page at `hindsight-docs/docs-integrations/<slug>.md(x)`. The `hindsight-docs/scripts/check-integrations.mjs` build step enforces both directions — forward: every internal JSON entry has a doc page; reverse: every released tag (`integrations/<name>/vX.Y.Z`) appears in the JSON (private infra like `cloudflare-oauth-proxy` is in the script's `EXCLUDED` set). Flag any integration that is released (or being released) but missing from `integrations.json`, and any JSON entry without a doc page. Do **not** hand-edit `versioned_sidebars/*.json` to add integration links — they are positional placeholders filled from the JSON.
+- **Code standards** — the integration code must follow all Python style rules (type hints, no raw dicts, no tuple returns, etc.).
+
+### 10. Check MCP tool registration completeness
+
+If any new MCP tools were added or existing tools renamed in `hindsight-api-slim/hindsight_api/mcp_tools.py`:
+- **`_ALL_TOOLS` set** in `mcp_tools.py` — must include the new tool name
+- **`tools_to_register` default set** in `register_mcp_tools()` in `mcp_tools.py` — must include the new tool name
+- **`_SINGLE_BANK_TOOLS` set** in `hindsight-api-slim/hindsight_api/api/mcp.py` — must include the new tool if it is bank-scoped (not a bank-management tool like `list_banks`/`create_bank`)
+- **`MCP_TOOL_GROUPS`** in `hindsight-control-plane/src/components/bank-config-view.tsx` — must include the new tool in the appropriate group for the UI tool selector
+- **Tool count assertions** in tests (e.g., `test_mcp_tools.py`) — must be updated to reflect the new count
+
+### 11. Check backup/restore table coverage
+
+If a migration adds a new PostgreSQL table (look for `CREATE TABLE` / `op.create_table` in `hindsight-api-slim/hindsight_api/alembic/versions/`):
+- **`BACKUP_TABLES`** in `hindsight-api-slim/hindsight_api/admin/cli.py` — must include the new table, placed after any table it references via foreign key (parents before children). A missing entry is silent data loss: the table is never backed up, and restore's `TRUNCATE banks CASCADE` wipes any FK-to-banks child (e.g. `mental_models`, `directives`) on restore even though it was never saved.
+- The guard test `test_backup_tables_covers_entire_schema` in `tests/test_admin_backup_restore.py` enforces this — flag it as a **must fix** if a new table is absent from `BACKUP_TABLES`.
+- Oracle-only tables (e.g. `observation_sources`) are intentionally excluded — admin backup/restore is PostgreSQL-only.
+
+### 11b. Check new config flags update the env template
+
+If the diff adds a new configuration field (a new `ENV_*` / `HINDSIGHT_*` env var
+in `hindsight-api-slim/hindsight_api/config.py`):
+- **`.env.example`** (repo root) — must add the variable (commented if optional)
+  alongside the docs entry in `hindsight-docs/docs/developer/configuration.md`.
+  A flag added to `config.py` but absent from `.env.example` is a **should fix**.
+- **`hindsight-embed/hindsight_embed/env.example`** — the bundled copy must stay
+  byte-identical to the repo-root `.env.example` (it seeds embed/profile configs).
+  The `test_bundled_template_matches_repo_root` sync test fails on drift; if the
+  root file changed without re-copying, flag it as a **must fix**.
+
+### 11c. Check for advisory locks
+
+Grep the diff for `advisory` (`git diff main...HEAD | grep -in advisory`). Any new
+`pg_advisory_lock` / `pg_try_advisory_lock` / `pg_advisory_xact_lock` /
+`pg_advisory_unlock` call is a **must fix** — see Database Locking above. Point the
+author at the alternatives (per-process objects, idempotent DDL, row-level
+constraints) rather than just asking them to drop the lock.
+
+### 12. Review against other coding standards
+
+Check the diff for violations of the standards listed above:
+- Python files at project root (not allowed)
+- Missing async patterns (should be async throughout)
+- Pydantic models for request/response
+- Line length > 120 chars
+- New features/code beyond what was asked (over-engineering)
+- Unnecessary error handling for impossible scenarios
+- Premature abstractions or speculative helpers
+- Backwards-compatibility hacks (unused vars, re-exports, "removed" comments)
+
+### 13. Report findings
+
+Present a clear summary organized by severity:
+
+**Must fix** — issues that will break CI or violate hard project rules:
+- Unrelated commits on the branch
+- Lint failures
+- Missing type hints on public functions
+- Raw dict usage for structured data (including internal code)
+- Multi-item tuple returns (including internal code)
+- Missing tests for new endpoints
+- Direct DB access (raw SQL / `acquire_with_retry` / `fq_table`) in an `api/` handler instead of a `MemoryEngine` method
+- Tenant-scoped data accessed without authentication enforced in the engine (`_authenticate_tenant` / `get_bank_profile`)
+- New integration missing tests, CI job, or release-integration.sh entry
+- Released/added integration missing from `hindsight-docs/src/data/integrations.json`, or a JSON entry with no `docs-integrations/<slug>` page (fails the docs build via `check-integrations.mjs`)
+- New PostgreSQL table missing from `BACKUP_TABLES` in `admin/cli.py` (silent data loss on restore)
+
+**Should fix** — issues that hurt code quality:
+- Dead code / unused imports missed by linter
+- Missing tests for non-trivial utility functions
+- Over-engineering beyond the task scope
+
+**Note** — observations that may or may not need action:
+- API changes that might need client regeneration
+- Patterns that deviate from nearby code style
+
+For each finding, include the file path, line number, and a brief explanation.
+
+Do NOT auto-fix any issues. Report all findings and let the user decide what to address. If there are no findings, confirm the code looks good.

.claude/skills/hs-release/SKILL.md

@@ -0,0 +1,116 @@
+---
+name: hs-release
+description: Cut a core Hindsight release (vX.Y.Z) and open the changelog + blog PR. Use when asked to cut/start a release, bump the version, or publish a new Hindsight version.
+user_invocable: true
+---
+
+# Hindsight Release
+
+Cut a **core** Hindsight release and open the accompanying changelog/blog PR. This is for the core
+product version (API, clients, CLI, control plane, Helm). **Integrations are versioned
+independently** — use `scripts/release-integration.sh` for those, not this skill.
+
+The release is **irreversible and outward-facing**: it tags a version and pushes it straight to
+`main`, which triggers CI that publishes packages to PyPI / npm / Helm. Confirm the version number
+and that the intended fixes are already merged to `main` before you start.
+
+## Step 0 — Pre-flight
+
+1. **Decide the base.** A release is cut from the latest `origin/main`, never from a feature
+   branch. `git fetch origin --tags` first. Confirm the "couple of fixes" the user means are
+   actually merged to `main` (`git log v<prev>..origin/main --oneline`).
+2. **Find where `main` is checked out.** `main` is often already checked out in a sibling worktree
+   (`git worktree list`). You **cannot** check out `main` in a second worktree — run the release in
+   the worktree that already holds it. If that worktree is dirty with throwaway cruft
+   (`.next-*` tsconfig paths, screenshots), `git stash push -u`, fast-forward to `origin/main`,
+   run the release, then `git stash pop`.
+3. **Pitfall:** never pipe the checkout in an `&&` chain like
+   `git checkout main 2>&1 | tail && git reset --hard ...` — the pipe's exit status is `tail`'s
+   (always 0), so a failed checkout won't stop the chain and the `reset` fires on the **wrong
+   branch**. Check out as its own command and verify `git branch --show-current` before resetting.
+
+## Step 1 — Cut the release
+
+Run from the worktree on a clean `main`:
+
+```bash
+./scripts/release.sh <version>     # e.g. 0.8.1  (no leading v)
+```
+
+`release.sh` bumps the version in every component, regenerates the OpenAPI spec + all client SDKs,
+updates docs versioning, commits `Release v<version>`, tags `v<version>`, and **pushes the commit
+and tag directly to `main`**. The push triggers the `Release` GitHub Actions workflow that builds
+and publishes the packages. It is **not** a PR.
+
+Verify after: `gh run list --limit 5` should show the `Release v<version>` workflow running, and
+`git ls-remote --tags origin v<version>` should return the tag.
+
+## Step 2 — Changelog + blog PR (separate)
+
+Done **after** the tag exists, as its own PR (precedent: v0.8.0 = #2053, v0.8.1 = #2080). Work on a
+branch off the new `main`:
+
+```bash
+git checkout -b docs-changelog-<version> origin/main
+```
+
+Only spin up a separate worktree (`git worktree add ../hindsight-changelog-<version> -b
+docs-changelog-<version> origin/main`) if you can't get a clean checkout otherwise — e.g. `main` is
+held in another worktree and the current one has work you don't want to disturb.
+
+**Branch naming:** use the `docs-` (hyphen) convention, e.g. `docs-changelog-0.8.1`. A remote
+branch literally named `docs` exists, so any `docs/...` branch is rejected on push with
+`directory file conflict`.
+
+### Changelog
+
+```bash
+uv run --directory hindsight-dev generate-changelog <version>
+```
+
+LLM-summarizes the commits between the previous tag and `v<version>` and prepends an entry to
+`hindsight-docs/src/pages/changelog/index.md`. Requires `OPENAI_API_KEY` (already in the repo
+`.env`). It excludes `hindsight-integrations/` source, but new integrations whose commits also
+touched docs will still appear — that matches precedent, leave them in the **changelog**.
+
+### Blog post
+
+Hand-write `hindsight-docs/blog/YYYY-MM-DD-version-X-Y-Z.md` (mirror an existing one; patch
+releases are short — see `2026-06-02-version-0-7-2.md`). Guidance:
+
+- **Explain user impact, not internals/mechanism.** Lead with what the user can now do and what to
+  set. Config/env-var names are fine (developer-facing), code symbols and internals are not.
+- **Do not list integrations in the release blog.** The core blog covers core engine / API /
+  ops changes; each integration ships its own changelog. (Integrations may still appear in the
+  generated `changelog/index.md` — that's fine; just keep them out of the blog.)
+- Call out an upgrade recommendation when there are operational/data-integrity fixes.
+- Validate formatting: `npx prettier --check <blog file>`.
+
+### Sync the docs skill
+
+```bash
+./scripts/generate-docs-skill.sh
+```
+
+Refreshes `skills/hindsight-docs/references/changelog/index.md`. It will also bump
+`skills/hindsight-docs/references/openapi.json` by one version — `release.sh` regenerates the skill
+*before* bumping OpenAPI, so the skill copy lags a version in the release commit; this step syncs
+it. Expect a one-line `version` diff there; keep it.
+
+### Commit, push, PR
+
+```bash
+git add -A
+git commit --no-verify -m "docs: changelog and blog post for v<version>"
+git push -u origin docs-changelog-<version>
+gh pr create --base main --title "docs: changelog and blog post for v<version>" --body "..."
+```
+
+Expected files in the PR: the changelog entry, the new blog post, the regenerated skill changelog
+mirror, and the skill `openapi.json` version sync.
+
+## Cleanup
+
+If you created a temporary worktree, remove it once the PR is up
+(`git worktree remove ../hindsight-changelog-<version>`; the branch stays on origin). Restore any
+stash you popped in Step 0.

.env.example

@@ -2,31 +2,371 @@
 # Copy this file to .env and fill in your values
 
 # LLM Configuration (Required)
+# Supported providers: openai, openai-responses, groq, ollama, gemini, anthropic, lmstudio, vertexai, minimax, deepseek, zai, atlas, volcano
 HINDSIGHT_API_LLM_PROVIDER=openai
 HINDSIGHT_API_LLM_API_KEY=your-api-key-here
-HINDSIGHT_API_LLM_MODEL=o3-mini
+HINDSIGHT_API_LLM_MODEL=gpt-4o-mini
 HINDSIGHT_API_LLM_BASE_URL=https://api.openai.com/v1
+# Reasoning effort for providers/models that support it. Examples: low, medium, high, xhigh.
+# HINDSIGHT_API_LLM_REASONING_EFFORT=low
+
+# Sampling temperature for internal LLM calls. Set a number in [0.0, 2.0], or `none`
+# to omit the temperature parameter entirely (required for models that reject explicit
+# temperatures, e.g. Azure gpt-5.5). The global override below applies to every operation;
+# per-operation overrides (defaults: verification=0.0, retain=0.1, reflect=0.9,
+# consolidation=0.0) take precedence.
+# HINDSIGHT_API_LLM_TEMPERATURE=none
+# HINDSIGHT_API_LLM_TEMPERATURE_VERIFICATION=0.0
+# HINDSIGHT_API_LLM_TEMPERATURE_RETAIN=0.1
+# HINDSIGHT_API_LLM_TEMPERATURE_REFLECT=0.9
+# HINDSIGHT_API_LLM_TEMPERATURE_CONSOLIDATION=0.0
+
+# Grammar-enforce structured output (json_schema strict) instead of the soft
+# schema-in-prompt path. Helps weaker self-hosted models that emit prose preambles
+# or invalid JSON. The global override below applies to every operation;
+# per-operation overrides take precedence, in both directions -- set one to false
+# to opt that operation out while the global flag is on.
+# HINDSIGHT_API_LLM_STRICT_SCHEMA=false
+# HINDSIGHT_API_LLM_STRICT_SCHEMA_RETAIN=true
+# HINDSIGHT_API_LLM_STRICT_SCHEMA_REFLECT=true
+# HINDSIGHT_API_LLM_STRICT_SCHEMA_CONSOLIDATION=true
+
+# Some backends, including Bedrock Converse, reject JSON Schema maxItems.
+# Disable it only for those backends; consolidation still enforces the cap.
+# HINDSIGHT_API_LLM_SUPPORTS_MAX_ITEMS=true
+
+# Pin a conversation to one backend prompt cache (OpenAI-compatible providers only).
+# Server-side prompt caches are per backend server, so the same conversation has to
+# reach the same one to hit. Values: auto (default), xai_conv_id (sends xAI's
+# x-grok-conv-id header), openai_prompt_cache_key (sends OpenAI's prompt_cache_key
+# field), none (sends nothing). "auto" picks from the base URL host and is an
+# allowlist: x.ai / grok.com get the header, native OpenAI / openai.com / Azure
+# OpenAI get the field, and every other backend gets nothing. Per-operation
+# overrides take precedence. Set to none to disable entirely.
+# HINDSIGHT_API_LLM_CACHE_AFFINITY=auto
+# HINDSIGHT_API_RETAIN_LLM_CACHE_AFFINITY=none
+# HINDSIGHT_API_REFLECT_LLM_CACHE_AFFINITY=xai_conv_id
+# HINDSIGHT_API_CONSOLIDATION_LLM_CACHE_AFFINITY=none
+
+# Ask litellm/litellmrouter/bedrock for structured output via a forced tool call
+# instead of response_format. Enable it for backends that reject response_format
+# outright -- e.g. Bedrock Claude in ap-southeast-2 ("Extra inputs are not permitted");
+# the same model in us-east-1 accepts response_format and needs nothing here.
+# HINDSIGHT_API_LLM_STRUCTURED_OUTPUT_FORCED_TOOL=false
+
+# Diagnostic: on any LLM 4xx, log the exact assembled request ([LLM_4XX_DUMP]) --
+# serialized request config (message bodies stripped) + capped per-message previews.
+# For debugging otherwise-unreproducible rejected calls. Off by default.
+# HINDSIGHT_API_LLM_DEBUG_DUMP_4XX=false
+
+# Example: Anthropic Claude configuration
+# HINDSIGHT_API_LLM_PROVIDER=anthropic
+# HINDSIGHT_API_LLM_API_KEY=your-anthropic-api-key
+# HINDSIGHT_API_LLM_MODEL=claude-sonnet-4-20250514
+
+# Example: Google Vertex AI configuration
+# HINDSIGHT_API_LLM_PROVIDER=vertexai
+# HINDSIGHT_API_LLM_MODEL=google/gemini-2.0-flash-001
+# HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=your-gcp-project-id
+# HINDSIGHT_API_LLM_VERTEXAI_REGION=us-central1
+# HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY=/path/to/service-account-key.json  # Optional, uses ADC if not set
+
+# Example: MiniMax configuration (1M context window)
+# HINDSIGHT_API_LLM_PROVIDER=minimax
+# HINDSIGHT_API_LLM_API_KEY=your-minimax-api-key
+# HINDSIGHT_API_LLM_MODEL=MiniMax-M3  # or MiniMax-M2.7 for the previous generation
+
+# Example: OpenAI Responses API (/v1/responses) — reasoning + function tools together
+# HINDSIGHT_API_LLM_PROVIDER=openai-responses
+# HINDSIGHT_API_LLM_API_KEY=your-openai-api-key
+# HINDSIGHT_API_LLM_MODEL=gpt-5.6  # reasoning model (gpt-5.x / o-series); e.g. gpt-5.6-terra
+# HINDSIGHT_API_LLM_REASONING_EFFORT=high  # sent alongside tools, unlike chat/completions
+
+# Example: DeepSeek configuration (https://api.deepseek.com)
+# HINDSIGHT_API_LLM_PROVIDER=deepseek
+# HINDSIGHT_API_LLM_API_KEY=your-deepseek-api-key
+# HINDSIGHT_API_LLM_MODEL=deepseek-v4-flash  # or deepseek-v4-pro / deepseek-chat / deepseek-reasoner
+
+# Example: z.ai configuration (Zhipu GLM series, https://z.ai)
+# HINDSIGHT_API_LLM_PROVIDER=zai
+# HINDSIGHT_API_LLM_API_KEY=your-zai-api-key
+# HINDSIGHT_API_LLM_MODEL=glm-4.5-flash  # or glm-4.5-air for the paid tier
+
+# Example: Atlas Cloud configuration (OpenAI-compatible, https://www.atlascloud.ai)
+# HINDSIGHT_API_LLM_PROVIDER=atlas
+# HINDSIGHT_API_LLM_API_KEY=your-atlascloud-api-key
+# HINDSIGHT_API_LLM_MODEL=deepseek-ai/deepseek-v4-pro  # reasoning model; also Qwen / GLM / Kimi / MiniMax, etc.
+
+# Example: LM Studio local configuration (Qwen 2.5 32B recommended)
+# HINDSIGHT_API_LLM_PROVIDER=lmstudio
+# HINDSIGHT_API_LLM_API_KEY=lmstudio
+# HINDSIGHT_API_LLM_BASE_URL=http://localhost:1234/v1
+# HINDSIGHT_API_LLM_MODEL=qwen2.5-32b-instruct
+
+# Example: Ollama local configuration (native provider)
+# HINDSIGHT_API_LLM_PROVIDER=ollama
+# HINDSIGHT_API_LLM_BASE_URL=http://localhost:11434/v1
+# HINDSIGHT_API_LLM_MODEL=gemma3:12b
+# Native Ollama context-window override (num_ctx). Leave unset to let Ollama use
+# the model Modelfile / server default; set a positive integer only to force a
+# specific context size (e.g. 16384 to keep the previous request behavior).
+# HINDSIGHT_API_LLM_OLLAMA_NUM_CTX=16384
+
+# Multi-LLM strategies: configure extra LLMs by index alongside the primary above,
+# then pick a routing strategy. Unset = single primary LLM (default). Members are
+# numbered from 1; indices must be contiguous. Each operation can override with a
+# RETAIN_/REFLECT_/CONSOLIDATION_ prefix (e.g. HINDSIGHT_API_RETAIN_LLM_1_PROVIDER).
+# HINDSIGHT_API_LLM_1_PROVIDER=groq
+# HINDSIGHT_API_LLM_1_API_KEY=your-groq-api-key
+# HINDSIGHT_API_LLM_1_MODEL=openai/gpt-oss-120b
+# HINDSIGHT_API_LLM_2_PROVIDER=anthropic
+# HINDSIGHT_API_LLM_2_API_KEY=your-anthropic-api-key
+# Strategy JSON: {"mode": "failover"} or {"mode": "round-robin"}.
+# Round-robin accepts optional positive-int "weights" (one per member, primary first).
+# HINDSIGHT_API_LLM_STRATEGY={"mode": "failover"}
 
 # API Configuration (Optional)
 HINDSIGHT_API_HOST=0.0.0.0
 HINDSIGHT_API_PORT=8888
 HINDSIGHT_API_LOG_LEVEL=info
+# Optional retain chunking override for structured logs/transcripts.
+# Unset uses HINDSIGHT_API_RETAIN_CHUNK_SIZE as the structured-chunk limit.
+# HINDSIGHT_API_RETAIN_STRUCTURED_CHUNK_SIZE=
+
+# When true, a retain operation that hit any fact-extraction errors is marked
+# 'failed' (not 'completed'), surfacing silently-dropped facts. Default false.
+# HINDSIGHT_API_FAIL_ON_EXTRACTION_ERRORS=false
+
+# Wall-clock ceiling (seconds) for one retain task in the worker. A retain that
+# blocks indefinitely is cancelled and marked 'failed' — and so becomes
+# retryable — instead of holding its worker slot until the process restarts.
+# Set well above your slowest healthy retain; 0 disables. Default 3600.
+# HINDSIGHT_API_RETAIN_WALL_TIMEOUT=3600
+
+# Dry-run extraction preview endpoint (POST /memories/dry-run-extract). Enabled by default; it makes
+# a real LLM call but stores nothing. Set to false to remove the endpoint (returns 404).
+# HINDSIGHT_API_ENABLE_DRY_RUN_EXTRACT=true
+
+# Base Path / Reverse Proxy Support (Optional)
+# Set these when deploying behind a reverse proxy with path-based routing
+# Example: To deploy at example.com/hindsight/, set both to "/hindsight"
+# HINDSIGHT_API_BASE_PATH=/hindsight
+# NEXT_PUBLIC_BASE_PATH=/hindsight
 
 # Database (Optional - uses embedded pg0 by default)
 # HINDSIGHT_API_DATABASE_URL=postgresql://user:pass@host:5432/db
+# HINDSIGHT_API_READ_DATABASE_URL=  # Optional read-replica URL. When set, recall queries (semantic, BM25, graph, temporal) flow through a separate pool against this URL, offloading the primary. Typically points to a read-only endpoint (CNPG's <cluster>-ro service or Aurora reader endpoint).
+# HINDSIGHT_API_MIGRATION_DATABASE_URL=  # Direct PostgreSQL URL for migrations (bypasses PgBouncer). Falls back to DATABASE_URL.
+# HINDSIGHT_API_DATABASE_SCHEMA=public  # PostgreSQL schema name (default: public)
+# HINDSIGHT_API_DB_MAX_PARALLEL_WORKERS_PER_GATHER=  # Optional cap on Postgres planner parallelism for this process's pool connections. Unset leaves the server default; 0 makes background/bulk queries run serially (useful on worker processes sharing a primary with latency-sensitive traffic).
+# HINDSIGHT_API_ENTITY_TRGM_SIMILARITY_THRESHOLD=0.15  # Postgres pg_trgm.similarity_threshold applied on every pool connection, used by entity resolution's % trigram match. Must be in (0, 1]. Lower catches more substring-ish matches at higher CPU cost on large entity sets; higher is stricter and cheaper.
+# HINDSIGHT_API_ENTITY_INTRABATCH_MERGE_SIMILARITY=0.5  # Trigram similarity (pg_trgm-equivalent, computed in-memory) at/above which two new names created by the SAME retain are merged into one entity (in-batch dedup of surface-form variants). Must be in (0, 1]. A merge cutoff, stricter than the recall threshold above; raise toward 1.0 to merge only near-identical forms.
+# HINDSIGHT_API_RETAIN_ENTITY_RESOLUTION_MAX_CANDIDATES=200  # Max candidates scored per entity mention during retain. The fuzzy lookup keeps only this many best matches per name (ranked by trigram/Jaro-Winkler similarity) before scoring them one by one. On banks holding thousands of near-identical names an uncapped set turns one retain into minutes of CPU that stall the worker's health checks. Raise only if entities that should merge are being duplicated.
+# HINDSIGHT_API_MIGRATION_CONCURRENCY=1  # Tenant schemas to migrate concurrently (PG only, each in its own process; per-schema work stays sequential). Each worker has ~1-2s startup cost + uses ~3 DB connections, so it only pays off with many schemas (tens+) or slow migrations; keep concurrency*3 <= spare max_connections. Default: 1 (sequential).
+# HINDSIGHT_API_OPERATION_RETENTION_DAYS=30  # Prune terminal operation rows, payloads, and metadata after this many days; 0 (the default) keeps them forever.
+# HINDSIGHT_API_OPERATION_CLEANUP_BATCH_SIZE=1000  # Maximum expired terminal rows deleted per tenant schema in each cleanup cycle; must be positive.
+
+# Vector Extension (Optional - uses pgvector by default)
+# Options: "pgvector" (default), "vchord", "pgvectorscale" (DiskANN)
+# HINDSIGHT_API_VECTOR_EXTENSION=pgvector
+# For Azure PostgreSQL with DiskANN:
+# HINDSIGHT_API_VECTOR_EXTENSION=pgvectorscale  # Auto-detects pg_diskann on Azure
+
+# Text Search Extension (Optional - uses native PostgreSQL full-text search by default)
+# Backend options: "native" (default), "vchord", "pg_textsearch", "pgroonga", "pg_search"
+# HINDSIGHT_API_TEXT_SEARCH_EXTENSION=native
+# Native backend dictionary (only used by HINDSIGHT_API_TEXT_SEARCH_EXTENSION=native)
+# HINDSIGHT_API_TEXT_SEARCH_EXTENSION_NATIVE_LANGUAGE=english
+# ParadeDB pg_search tokenizer (only used when creating pg_search BM25 indexes).
+# Empty uses ParadeDB's default tokenizer: unicode_words.
+# Supported values: unicode_words, simple, whitespace, literal, literal_normalized,
+# chinese_compatible, icu, jieba, source_code,
+# chinese_lindera/lindera(chinese), japanese_lindera/lindera(japanese),
+# korean_lindera/lindera(korean), ngram(min,max), edge_ngram(min,max)
+# HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER=
+# Optional cap on the number of terms in the native PostgreSQL BM25 tsquery.
+# Long queries OR-join every normalized token, which can match too much of a
+# large bank. 0 (default) keeps the historical uncapped behavior; a positive
+# value bounds only the native backend (other BM25 backends get the raw query).
+# HINDSIGHT_API_BM25_MAX_QUERY_TERMS=0
+
+# File Parser (Optional - uses markitdown by default)
+# HINDSIGHT_API_FILE_PARSER=markitdown
+# Enable image OCR for MarkItDown using an OpenAI-compatible OCR/vision endpoint.
+# These OCR settings are independent from HINDSIGHT_API_LLM_* because MarkItDown
+# uses the OpenAI SDK directly and requires Chat Completions image input support.
+# When OCR is enabled, API_KEY, BASE_URL, and MODEL are required.
+# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_ENABLED=false
+# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_API_KEY=
+# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_BASE_URL=
+# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_MODEL=
+# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_PROMPT=
+# Optional JSON dict of custom headers for the OCR OpenAI client (e.g. proxies / request tracing).
+# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_DEFAULT_HEADERS=
 
 # Embeddings Configuration (Optional - uses local by default)
-# Provider: "local" (default) or "tei" (HuggingFace Text Embeddings Inference)
+# Provider: "local" (default), "onnx", "tei", "openai", "cohere", "google", "openrouter", "zeroentropy", "litellm", or "litellm-sdk"
 # HINDSIGHT_API_EMBEDDINGS_PROVIDER=local
 # For local provider:
 # HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL=BAAI/bge-small-en-v1.5
+# Force CPU if local embeddings hit MPS/XPC instability on macOS:
+# HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU=false
+# Opt in to the Apple Silicon MPS GPU (off by default: MPS leaks memory under
+# variable-length workloads). CUDA/XPU still auto-select regardless:
+# HINDSIGHT_API_EMBEDDINGS_LOCAL_ALLOW_MPS=false
+# For ONNX provider (local CPU embeddings without an Ollama/TEI sidecar):
+# HINDSIGHT_API_EMBEDDINGS_PROVIDER=onnx
+# HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_ID=intfloat/multilingual-e5-small
+# HINDSIGHT_API_EMBEDDINGS_ONNX_FILE=onnx/model.onnx
+# HINDSIGHT_API_EMBEDDINGS_ONNX_DIMENSIONS=384
+# HINDSIGHT_API_EMBEDDINGS_ONNX_MAX_TOKENS=512
+# HINDSIGHT_API_EMBEDDINGS_ONNX_POOLING=mean
+# HINDSIGHT_API_EMBEDDINGS_ONNX_NORMALIZE=true
+# HINDSIGHT_API_EMBEDDINGS_ONNX_QUERY_PREFIX="query: "
+# HINDSIGHT_API_EMBEDDINGS_ONNX_PASSAGE_PREFIX="passage: "
+# Optional for local model paths or pre-downloaded artifacts:
+# HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_PATH=/models/multilingual-e5-small/onnx/model.onnx
+# HINDSIGHT_API_EMBEDDINGS_ONNX_TOKENIZER_NAME_OR_PATH=/models/multilingual-e5-small
+# Optional for China network / restricted HF access:
+# HF_ENDPOINT=https://hf-mirror.com
+# Applies to any provider: cap each input at this many tiktoken tokens before
+# embedding, so oversized content is truncated instead of failing the embed call
+# permanently (e.g. Bedrock Titan V2's 8192, or a llama.cpp server's context). Off
+# by default. (Deprecated alias: HINDSIGHT_API_EMBEDDINGS_LITELLM_SDK_MAX_INPUT_TOKENS)
+# HINDSIGHT_API_EMBEDDINGS_MAX_INPUT_TOKENS=8192
 # For TEI provider:
 # HINDSIGHT_API_EMBEDDINGS_TEI_URL=http://localhost:8080
+# For OpenAI-compatible embeddings:
+# HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=sk-xxxx
+# HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL=text-embedding-3-small
+# HINDSIGHT_API_EMBEDDINGS_OPENAI_BASE_URL=https://api.openai.com/v1
+# For ZeroEntropy zembed-1:
+# HINDSIGHT_API_EMBEDDINGS_PROVIDER=zeroentropy
+# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_API_KEY=ze-xxxx
+# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_MODEL=zembed-1
+# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_DIMENSIONS=1280
+# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_ENCODING_FORMAT=float
+# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_LATENCY=fast
+#
+# IMPORTANT: Embedding keys require provider-specific names:
+# HINDSIGHT_API_EMBEDDINGS_{PROVIDER}_{PARAMETER}
+# (for example, HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL).
+#
+# DeepSeek note: DeepSeek is supported for LLM calls, but not for embeddings.
+# If using DeepSeek as LLM provider, keep embeddings on local/openai/cohere/google/etc.
+
+# Embedding similarity thresholds. These defaults preserve the behavior calibrated
+# for BAAI/bge-small-en-v1.5. Recalibrate each threshold independently when changing
+# embedding models because cosine-similarity distributions are model-dependent.
+# HINDSIGHT_API_SEMANTIC_MIN_SIMILARITY=0.3
+# HINDSIGHT_API_GRAPH_SEED_MIN_SIMILARITY=0.3
+# HINDSIGHT_API_TEMPORAL_SEMANTIC_MIN_SIMILARITY=0.1
+# HINDSIGHT_API_SEMANTIC_LINK_MIN_SIMILARITY=0.7
+# HINDSIGHT_API_CONSOLIDATION_DEDUP_THRESHOLD=0.97
+
+# Recall pipeline stages (all on by default). Each is hierarchical, so a single
+# bank can switch a stage off via the config API without changing the server
+# default. Turning all three off leaves semantic + BM25 fused by RRF, the
+# lowest-latency recall path.
+# Temporal retrieval arm, plus the date-aware query analysis that feeds it:
+# HINDSIGHT_API_ENABLE_TEMPORAL_RETRIEVAL=true
+# Entity/link graph traversal arm:
+# HINDSIGHT_API_ENABLE_GRAPH_RETRIEVAL=true
+# Cross-encoder rerank of the fused candidates (false = use the RRF order):
+# HINDSIGHT_API_ENABLE_RERANKING=true
 
 # Reranker Configuration (Optional - uses local by default)
 # Provider: "local" (default) or "tei" (HuggingFace Text Embeddings Inference)
 # HINDSIGHT_API_RERANKER_PROVIDER=local
+# Trusted gateway attribution (disabled by default). When enabled, remote
+# reranker requests include X-Hindsight-Bank-Id with the current bank ID.
+# HINDSIGHT_API_RERANKER_SEND_BANK_AS_HEADER=false
 # For local provider:
 # HINDSIGHT_API_RERANKER_LOCAL_MODEL=cross-encoder/ms-marco-MiniLM-L-6-v2
+# Force CPU if the local reranker hits MPS/XPC instability on macOS:
+# HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU=false
+# Opt in to the Apple Silicon MPS GPU (off by default: MPS leaks memory under
+# variable-length workloads). CUDA/XPU still auto-select regardless:
+# HINDSIGHT_API_RERANKER_LOCAL_ALLOW_MPS=false
 # For TEI provider:
 # HINDSIGHT_API_RERANKER_TEI_URL=http://localhost:8081
+# Max candidates the cross-encoder reranks per recall (RRF pre-filters the rest):
+# HINDSIGHT_API_RERANKER_MAX_CANDIDATES=300
+# Optionally scale that cap by the recall budget level (the cross-encoder dominates
+# a large recall's latency). 0 = fall back to the flat cap above; fully backwards-compatible.
+# HINDSIGHT_API_RERANKER_MAX_CANDIDATES_LOW=0
+# HINDSIGHT_API_RERANKER_MAX_CANDIDATES_MID=0
+# HINDSIGHT_API_RERANKER_MAX_CANDIDATES_HIGH=0
+
+# Reranker failover chain: extra rerankers tried, in order, when the one above
+# fails. Members are numbered from 1 (indices must be contiguous) and every
+# setting of member n carries the same index. A member inherits nothing from the
+# primary, so spell out everything it needs. Unset = no fallback (default): a
+# failing reranker fails the recall. End the chain with "rrf" to fail open and
+# keep the retrieval order instead.
+# HINDSIGHT_API_RERANKER_1_PROVIDER=cohere
+# HINDSIGHT_API_RERANKER_1_COHERE_API_KEY=your-cohere-api-key
+# HINDSIGHT_API_RERANKER_2_PROVIDER=rrf
+
+# Observability & Tracing (Optional - disabled by default)
+# Enable OpenTelemetry tracing for LLM calls (GenAI semantic conventions)
+# HINDSIGHT_API_OTEL_TRACES_ENABLED=true
+#
+# Local development with Grafana LGTM stack (recommended - see scripts/dev/grafana/README.md)
+# HINDSIGHT_API_OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
+#
+# Cloud backends (Grafana Cloud, Langfuse, DataDog, etc.)
+# HINDSIGHT_API_OTEL_EXPORTER_OTLP_ENDPOINT=https://your-backend-url
+# HINDSIGHT_API_OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer your-token"
+#
+# Custom service name and environment (optional, defaults: hindsight-api, development)
+# HINDSIGHT_API_OTEL_SERVICE_NAME=hindsight-production
+# HINDSIGHT_API_OTEL_DEPLOYMENT_ENVIRONMENT=production
+#
+# Expose async-operation queue + consolidation-backlog gauges on /metrics.
+# Runs periodic per-schema COUNT queries on a background task (disabled by default).
+# HINDSIGHT_API_METRICS_BACKLOG_ENABLED=true
+#
+# Runtime-stall observability (enabled by default). When a liveness probe fails,
+# these tell you WHY: a blocked event loop vs DB connection-pool exhaustion.
+# The loop watchdog logs the offending stack when the loop is unresponsive; the
+# DB-pool acquire timing logs (and exposes hindsight.db.pool.waiting) when
+# callers queue for a connection. Both are cheap; tune or disable if needed.
+# HINDSIGHT_API_LOOP_WATCHDOG_ENABLED=false
+# HINDSIGHT_API_LOOP_WATCHDOG_STALL_THRESHOLD_MS=1000
+# HINDSIGHT_API_LOOP_WATCHDOG_POLL_INTERVAL_MS=250
+# HINDSIGHT_API_DB_ACQUIRE_WARN_THRESHOLD_MS=1000
+
+# -----------------------------------------------------------------------------
+# Webhooks (Optional)
+# -----------------------------------------------------------------------------
+
+# Outbound webhook delivery targets caller-supplied URLs. To prevent SSRF, the
+# delivery worker blocks private, loopback, and link-local destinations
+# (including the cloud metadata address 169.254.169.254) by default. List hosts
+# or IP/CIDR ranges here (comma-separated) to re-permit specific internal
+# destinations — e.g. 127.0.0.1 for local testing, or an internal receiver.
+# HINDSIGHT_API_WEBHOOK_ALLOWED_HOSTS=127.0.0.1,internal-receiver.svc,10.0.0.0/8
+
+# Whether the webhook delivery-history API returns the raw upstream response
+# body. Off by default: returning arbitrary response bodies to callers is an
+# information-exfiltration primitive. The delivery status code is always
+# returned regardless. Enable only if you trust your webhook destinations.
+# HINDSIGHT_API_WEBHOOK_EXPOSE_RESPONSE_BODY=false
+
+# -----------------------------------------------------------------------------
+# Control Plane (Optional)
+# -----------------------------------------------------------------------------
+
+# Dataplane API URL - where the CP proxies requests to
+# HINDSIGHT_CP_DATAPLANE_API_URL=http://localhost:8888
+
+# Optional: Bearer token the CP sends as `Authorization: Bearer <key>` to the
+# dataplane API. Required when the API service is auth-protected; omit for a
+# public/unauthenticated API.
+# HINDSIGHT_CP_DATAPLANE_API_KEY=your-dataplane-bearer-token
+
+# Optional: Require a shared access key to view the Control Plane UI.
+# When set, visitors see a login page and must enter the key before
+# accessing the dashboard or any /api/* routes (except /api/health).
+# HINDSIGHT_CP_ACCESS_KEY=your-shared-secret-key

.githooks/pre-commit

@@ -0,0 +1,27 @@
+#!/bin/bash
+# Pre-commit hook - runs all scripts in scripts/hooks/
+
+set -e
+
+REPO_ROOT="$(git rev-parse --show-toplevel)"
+HOOKS_DIR="$REPO_ROOT/scripts/hooks"
+
+if [ ! -d "$HOOKS_DIR" ]; then
+    exit 0
+fi
+
+echo ""
+echo "=== Running pre-commit hooks ==="
+echo ""
+
+# Run all executable scripts in hooks directory
+for hook in "$HOOKS_DIR"/*.sh; do
+    if [ -x "$hook" ]; then
+        echo "[hook] $(basename "$hook")"
+        (cd "$REPO_ROOT" && "$hook")
+    fi
+done
+
+echo ""
+echo "=== Pre-commit hooks completed ==="
+echo ""

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -0,0 +1,71 @@
+name: Bug Report
+description: Report a bug or unexpected behavior
+labels: ["bug", "triage"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for taking the time to report a bug! Please fill out the sections below.
+
+  - type: textarea
+    id: description
+    attributes:
+      label: Bug Description
+      description: A clear and concise description of the bug
+      placeholder: What happened?
+    validations:
+      required: true
+
+  - type: textarea
+    id: reproduction
+    attributes:
+      label: Steps to Reproduce
+      description: Steps to reproduce the behavior
+      placeholder: |
+        1. Configure '...'
+        2. Call '...'
+        3. See error
+    validations:
+      required: true
+
+  - type: textarea
+    id: expected
+    attributes:
+      label: Expected Behavior
+      description: What did you expect to happen?
+    validations:
+      required: true
+
+  - type: textarea
+    id: actual
+    attributes:
+      label: Actual Behavior
+      description: What actually happened?
+    validations:
+      required: true
+
+  - type: input
+    id: version
+    attributes:
+      label: Version
+      description: What version are you using?
+      placeholder: e.g., 0.1.0 or commit hash
+    validations:
+      required: false
+
+  - type: dropdown
+    id: llm-provider
+    attributes:
+      label: LLM Provider
+      description: Which LLM provider are you using?
+      options:
+        - OpenAI
+        - Anthropic
+        - Gemini
+        - Groq
+        - Ollama
+        - LM Studio
+        - Other
+    validations:
+      required: false
+

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,8 @@
+blank_issues_enabled: false
+contact_links:
+  - name: Questions & Help
+    url: https://github.com/vectorize-io/hindsight/discussions/categories/q-a
+    about: Please ask questions and get help in Discussions instead of opening an issue.
+  - name: Ideas & Feedback
+    url: https://github.com/vectorize-io/hindsight/discussions/categories/ideas
+    about: Share ideas or give feedback in Discussions.

.github/ISSUE_TEMPLATE/feature_request.yml

@@ -0,0 +1,82 @@
+name: Feature Request
+description: Suggest a new feature or enhancement
+labels: ["enhancement", "triage"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for suggesting a feature! Please describe what you'd like to see added.
+
+  - type: textarea
+    id: use-case
+    attributes:
+      label: Use Case
+      description: Describe your specific use case. What are you building? What's your goal?
+      placeholder: |
+        I'm building an AI agent that needs to...
+        My application handles...
+    validations:
+      required: true
+
+  - type: textarea
+    id: problem
+    attributes:
+      label: Problem Statement
+      description: What problem are you facing? What's missing or difficult today?
+      placeholder: Currently I have to... which causes...
+    validations:
+      required: true
+
+  - type: textarea
+    id: benefit
+    attributes:
+      label: How This Feature Would Help
+      description: Explain how this feature would improve your workflow or solve your problem
+      placeholder: With this feature, I would be able to...
+    validations:
+      required: true
+
+  - type: textarea
+    id: solution
+    attributes:
+      label: Proposed Solution
+      description: Describe your ideal solution (optional - we may have ideas too!)
+      placeholder: It would be great if Hindsight could...
+    validations:
+      required: false
+
+  - type: textarea
+    id: alternatives
+    attributes:
+      label: Alternatives Considered
+      description: Have you considered any alternative solutions or workarounds?
+    validations:
+      required: false
+
+  - type: dropdown
+    id: priority
+    attributes:
+      label: Priority
+      description: How important is this feature to you?
+      options:
+        - Nice to have
+        - Important - affects my workflow
+        - Critical - blocking my use case
+    validations:
+      required: true
+
+  - type: textarea
+    id: additional
+    attributes:
+      label: Additional Context
+      description: Any other context, mockups, or examples?
+    validations:
+      required: false
+
+  - type: checkboxes
+    id: checklist
+    attributes:
+      label: Checklist
+      options:
+        - label: I would be willing to contribute this feature
+          required: false

.github/actions/setup-pg0/action.yml

@@ -1,11 +0,0 @@
-name: 'Setup pg0'
-description: 'Install pg0 embedded PostgreSQL'
-
-runs:
-  using: 'composite'
-  steps:
-    - name: Install pg0
-      shell: bash
-      run: |
-        curl -fsSL https://raw.githubusercontent.com/vectorize-io/pg0/main/install.sh | bash
-        echo "$HOME/.pg0/bin" >> $GITHUB_PATH

.github/workflows/deploy-docs.yml

@@ -20,19 +20,23 @@ concurrency:
 jobs:
   build:
     runs-on: ubuntu-latest
-    defaults:
-      run:
-        working-directory: hindsight-docs
     steps:
-      - uses: actions/checkout@v4
-      - uses: actions/setup-node@v4
+      - uses: actions/checkout@v6
+        with:
+          fetch-depth: 0 # fetch tags so check-released-integrations can see them
+      - uses: actions/setup-node@v6
         with:
           node-version: 20
           cache: npm
-          cache-dependency-path: hindsight-docs/package-lock.json
-      - run: npm ci
-      - run: npm run build
-      - uses: actions/upload-pages-artifact@v3
+          cache-dependency-path: package-lock.json
+      - uses: astral-sh/setup-uv@v7
+      - run: npm ci --workspace=hindsight-docs
+      - run: uv run generate-llms-full
+      - run: npm run build --workspace=hindsight-docs
+        env:
+          UMAMI_URL: https://analytics.hindsight.vectorize.io
+          UMAMI_WEBSITE_ID: ${{ secrets.UMAMI_WEBSITE_ID }}
+      - uses: actions/upload-pages-artifact@v5
         with:
           path: hindsight-docs/build
   deploy:
@@ -42,5 +46,5 @@ jobs:
     runs-on: ubuntu-latest
     needs: build
     steps:
-      - uses: actions/deploy-pages@v4
+      - uses: actions/deploy-pages@v5
         id: deployment

.github/workflows/perf-test.yml

@@ -0,0 +1,297 @@
+name: Performance Tests
+
+on:
+  schedule:
+    # Run daily at 06:00 UTC
+    - cron: "0 6 * * *"
+  workflow_dispatch:
+    inputs:
+      scale:
+        description: "Test scale (perf-test)"
+        type: choice
+        options:
+          - tiny
+          - small
+          - medium
+          - large
+        default: large
+      suite:
+        description: "Perf-test suite to run (blank = all)"
+        type: choice
+        options:
+          - ""
+          - retain
+          - recall
+          - recall-with-observations
+          - recall-temporal
+          - consolidation
+          - graph-maintenance
+        default: ""
+      locomo_conversations:
+        description: "LoComo conversation IDs (space-separated). Blank = curated set (conv-26 conv-30 conv-43)."
+        type: string
+        default: ""
+      locomo_skip:
+        description: "Skip LoComo job"
+        type: boolean
+        default: false
+      obs_skip:
+        description: "Skip observation-dedup benchmark job"
+        type: boolean
+        default: false
+      obs_dataset:
+        description: "Obs benchmark dataset substring (blank = English hermes transcript)."
+        type: string
+        default: ""
+      obs_fraction:
+        description: "Obs benchmark fraction (0-1] of each document to run."
+        type: string
+        default: "1.0"
+      ref:
+        description: "Git ref to test (branch, tag, or SHA). Defaults to main."
+        type: string
+        default: ""
+
+concurrency:
+  group: perf-test
+  cancel-in-progress: true
+
+jobs:
+  perf-test:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v6
+      with:
+        ref: ${{ inputs.ref || github.ref }}
+
+    - name: Install uv
+      uses: astral-sh/setup-uv@v7
+      with:
+        enable-cache: true
+        prune-cache: false
+
+    - name: Set up Python
+      uses: actions/setup-python@v6
+      with:
+        python-version-file: ".python-version"
+
+    - name: Cache HuggingFace models
+      uses: actions/cache@v5
+      with:
+        path: ~/.cache/huggingface
+        key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
+        restore-keys: |
+          ${{ runner.os }}-huggingface-
+
+    - name: Pre-download models
+      working-directory: ./hindsight-api-slim
+      run: |
+        uv run --frozen --all-extras --index-strategy unsafe-best-match python -c "
+        from sentence_transformers import SentenceTransformer
+        print('Downloading embedding model...')
+        SentenceTransformer('BAAI/bge-small-en-v1.5')
+        print('Model downloaded successfully')
+        "
+
+    - name: Install hindsight-dev dependencies
+      run: |
+        cd hindsight-dev && uv sync --frozen --all-extras --index-strategy unsafe-best-match
+
+    - name: Run perf-test
+      run: |
+        SUITE_ARG=""
+        if [ -n "${{ inputs.suite }}" ]; then
+          SUITE_ARG="--suite ${{ inputs.suite }}"
+        fi
+        ./scripts/benchmarks/run-perf-test.sh \
+          --scale ${{ inputs.scale || 'large' }} \
+          $SUITE_ARG \
+          --output perf-results.json
+
+    - name: Upload perf results
+      if: always()
+      uses: actions/upload-artifact@v7
+      with:
+        name: perf-results-${{ github.sha }}
+        path: hindsight-dev/perf-results.json
+        retention-days: 90
+
+    # Publish enriched results (perf JSON + commit metadata) to the dashboard
+    # repo's gh-pages branch. The static site at
+    #   https://vectorize-io.github.io/hindsight-continuous-performance-monitor/
+    # reads data/index.json + data/<run>.json and renders charts client-side.
+    - name: Publish to dashboard
+      if: github.event_name == 'schedule' || github.event_name == 'push' || github.event_name == 'workflow_dispatch'
+      env:
+        PERF_DASHBOARD_TOKEN: ${{ secrets.PERF_DASHBOARD_TOKEN }}
+        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      run: ./scripts/benchmarks/publish-perf-results.sh hindsight-dev/perf-results.json
+
+  locomo:
+    if: inputs.locomo_skip != true
+    runs-on: ubuntu-latest
+    env:
+      HINDSIGHT_API_LLM_PROVIDER: vertexai
+      HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
+      HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
+      HINDSIGHT_API_JUDGE_LLM_PROVIDER: vertexai
+      HINDSIGHT_API_JUDGE_LLM_MODEL: google/gemini-2.5-flash-lite
+      HINDSIGHT_API_ANSWER_LLM_PROVIDER: vertexai
+      HINDSIGHT_API_ANSWER_LLM_MODEL: google/gemini-2.5-flash
+    steps:
+    - uses: actions/checkout@v6
+      with:
+        ref: ${{ inputs.ref || github.ref }}
+
+    - name: Setup GCP credentials
+      run: |
+        printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
+        PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
+        echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
+
+    - name: Install uv
+      uses: astral-sh/setup-uv@v7
+      with:
+        enable-cache: true
+        prune-cache: false
+
+    - name: Set up Python
+      uses: actions/setup-python@v6
+      with:
+        python-version-file: ".python-version"
+
+    - name: Cache HuggingFace models
+      uses: actions/cache@v5
+      with:
+        path: ~/.cache/huggingface
+        key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
+        restore-keys: |
+          ${{ runner.os }}-huggingface-
+
+    - name: Pre-download models
+      working-directory: ./hindsight-api-slim
+      run: |
+        uv run --frozen --all-extras --index-strategy unsafe-best-match python -c "
+        from sentence_transformers import SentenceTransformer
+        print('Downloading embedding model...')
+        SentenceTransformer('BAAI/bge-small-en-v1.5')
+        print('Model downloaded successfully')
+        "
+
+    - name: Install hindsight-dev dependencies
+      run: |
+        cd hindsight-dev && uv sync --frozen --all-extras --index-strategy unsafe-best-match
+
+    - name: Run LoComo benchmark
+      # Curated 3-conversation subset (best/middle/worst by accuracy on the
+      # last successful full run): conv-26 (best), conv-30 (middle), conv-43
+      # (worst). Excludes conv-44, the bank with the largest unconsolidated
+      # set that has been pushing scheduled runs over the per-bank
+      # _wait_for_consolidation timeout. Override via workflow_dispatch with
+      # the locomo_conversations input.
+      run: |
+        CONVERSATIONS="${{ inputs.locomo_conversations }}"
+        if [ -z "$CONVERSATIONS" ]; then
+          CONVERSATIONS="conv-26 conv-30 conv-43"
+        fi
+        uv run python hindsight-dev/benchmarks/locomo/locomo_benchmark.py \
+          --wait-consolidation \
+          --conversation $CONVERSATIONS
+
+    - name: Upload LoComo results
+      if: always()
+      uses: actions/upload-artifact@v7
+      with:
+        name: locomo-results-${{ github.sha }}
+        path: hindsight-dev/benchmarks/locomo/results/
+        retention-days: 90
+
+    - name: Publish LoComo to dashboard
+      if: success() && (github.event_name == 'schedule' || github.event_name == 'push' || github.event_name == 'workflow_dispatch')
+      env:
+        PERF_DASHBOARD_TOKEN: ${{ secrets.PERF_DASHBOARD_TOKEN }}
+        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      run: ./scripts/benchmarks/publish-locomo-results.sh hindsight-dev/benchmarks/locomo/results/benchmark_results.json
+
+  obs:
+    # Observation-dedup quality benchmark: ingests a transcript, drains consolidation
+    # (serial SyncTaskBackend + embedded pg0 — no external DB / worker), and reports the
+    # near-duplicate observation rate. Real LLM via VertexAI, mirroring the LoComo job.
+    if: inputs.obs_skip != true
+    runs-on: ubuntu-latest
+    env:
+      HINDSIGHT_API_LLM_PROVIDER: vertexai
+      HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
+      HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
+      HINDSIGHT_API_ENABLE_OBSERVATIONS: "true"
+    steps:
+    - uses: actions/checkout@v6
+      with:
+        ref: ${{ inputs.ref || github.ref }}
+
+    - name: Setup GCP credentials
+      run: |
+        printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
+        PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
+        echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
+
+    - name: Install uv
+      uses: astral-sh/setup-uv@v7
+      with:
+        enable-cache: true
+        prune-cache: false
+
+    - name: Set up Python
+      uses: actions/setup-python@v6
+      with:
+        python-version-file: ".python-version"
+
+    - name: Cache HuggingFace models
+      uses: actions/cache@v5
+      with:
+        path: ~/.cache/huggingface
+        key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
+        restore-keys: |
+          ${{ runner.os }}-huggingface-
+
+    - name: Pre-download models
+      working-directory: ./hindsight-api-slim
+      run: |
+        uv run --frozen --all-extras --index-strategy unsafe-best-match python -c "
+        from sentence_transformers import SentenceTransformer
+        print('Downloading embedding model...')
+        SentenceTransformer('BAAI/bge-small-en-v1.5')
+        print('Model downloaded successfully')
+        "
+
+    - name: Install hindsight-dev dependencies
+      run: |
+        cd hindsight-dev && uv sync --frozen --all-extras --index-strategy unsafe-best-match
+
+    - name: Run obs benchmark
+      # Default to the English hermes transcript at full fraction — a clean, deterministic
+      # consolidation-dedup signal (the Chinese variant adds a cross-lingual embedding
+      # confound). Override dataset/fraction via workflow_dispatch.
+      run: |
+        DATASET="${{ inputs.obs_dataset }}"
+        if [ -z "$DATASET" ]; then DATASET="hermes_session_2026-05-15_en"; fi
+        FRACTION="${{ inputs.obs_fraction }}"
+        if [ -z "$FRACTION" ]; then FRACTION="1.0"; fi
+        cd hindsight-dev
+        uv run python -m benchmarks.obs.obs_benchmark \
+          --dataset "$DATASET" --fraction "$FRACTION" --wipe-bank --output obs-results.json
+
+    - name: Upload obs results
+      if: always()
+      uses: actions/upload-artifact@v7
+      with:
+        name: obs-results-${{ github.sha }}
+        path: hindsight-dev/obs-results.json
+        retention-days: 90
+
+    - name: Publish obs to dashboard
+      if: success() && (github.event_name == 'schedule' || github.event_name == 'push' || github.event_name == 'workflow_dispatch')
+      env:
+        PERF_DASHBOARD_TOKEN: ${{ secrets.PERF_DASHBOARD_TOKEN }}
+        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      run: ./scripts/benchmarks/publish-obs-results.sh hindsight-dev/obs-results.json

.github/workflows/release-integration.yml

@@ -0,0 +1,205 @@
+name: Release Integration
+
+on:
+  push:
+    tags:
+      - 'integrations/**'
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    permissions:
+      id-token: write  # for PyPI trusted publishing + build-provenance attestations
+      attestations: write  # for actions/attest-build-provenance (Obsidian assets)
+      # No `contents: write`: we never create releases in this repo. The Obsidian
+      # plugin's distribution release is pushed to its dedicated repo using
+      # OBSIDIAN_DIST_TOKEN (see the "Mirror Obsidian plugin" step below).
+
+    steps:
+    - uses: actions/checkout@v6
+
+    - name: Extract integration info
+      id: info
+      run: |
+        # refs/tags/integrations/litellm/v0.1.0 → integration=litellm, version=0.1.0
+        TAG="${GITHUB_REF#refs/tags/}"
+        INTEGRATION=$(echo "$TAG" | cut -d'/' -f2)
+        VERSION=$(echo "$TAG" | cut -d'/' -f3 | sed 's/^v//')
+        echo "integration=$INTEGRATION" >> $GITHUB_OUTPUT
+        echo "version=$VERSION" >> $GITHUB_OUTPUT
+        echo "tag=$TAG" >> $GITHUB_OUTPUT
+        echo "Integration: $INTEGRATION, Version: $VERSION"
+
+    - name: Detect integration type
+      id: type
+      run: |
+        if [ -f "hindsight-integrations/${{ steps.info.outputs.integration }}/pyproject.toml" ]; then
+          echo "type=python" >> $GITHUB_OUTPUT
+        elif [ -f "hindsight-integrations/${{ steps.info.outputs.integration }}/package.json" ]; then
+          echo "type=typescript" >> $GITHUB_OUTPUT
+        else
+          echo "type=plugin" >> $GITHUB_OUTPUT
+        fi
+
+    # ── Python integrations (litellm, pydantic-ai, crewai) ──────────────────
+
+    - name: Install uv
+      if: steps.type.outputs.type == 'python'
+      uses: astral-sh/setup-uv@v7
+      with:
+        enable-cache: true
+
+    - name: Set up Python
+      if: steps.type.outputs.type == 'python'
+      uses: actions/setup-python@v6
+      with:
+        python-version-file: ".python-version"
+
+    - name: Build Python package
+      if: steps.type.outputs.type == 'python'
+      working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
+      run: uv build --out-dir dist
+
+    - name: Publish Python package to PyPI
+      if: steps.type.outputs.type == 'python'
+      uses: pypa/gh-action-pypi-publish@release/v1
+      with:
+        packages-dir: ./hindsight-integrations/${{ steps.info.outputs.integration }}/dist
+        skip-existing: true
+
+    # ── TypeScript integrations (ai-sdk, chat, openclaw) ────────────────────
+
+    # ── Plugin integrations (claude-code) — no package to publish ───────────
+
+    - name: Plugin release
+      if: steps.type.outputs.type == 'plugin'
+      run: |
+        echo "Plugin integration ${{ steps.info.outputs.integration }} v${{ steps.info.outputs.version }} — no package to publish."
+        echo "Users install via: claude plugin marketplace add vectorize-io/hindsight"
+
+    # ── TypeScript integrations (ai-sdk, chat, openclaw) ────────────────────
+
+    - name: Set up Node.js
+      if: steps.type.outputs.type == 'typescript'
+      uses: actions/setup-node@v6
+      with:
+        node-version: '22'
+        registry-url: 'https://registry.npmjs.org'
+        cache: 'npm'
+        cache-dependency-path: package-lock.json
+
+    - name: Check integration lockfile
+      if: steps.type.outputs.type == 'typescript'
+      run: ./scripts/check-integration-lockfiles.sh
+
+    # Some integrations depend on workspace packages (hindsight-client,
+    # hindsight-all, hindsight-agent-sdk) via file: refs. Install from root
+    # so npm resolves them, then build the workspace deps before the integration.
+    - name: Install root workspace dependencies
+      if: steps.type.outputs.type == 'typescript'
+      run: npm ci
+
+    - name: Build workspace deps (hindsight-client, hindsight-all, hindsight-agent-sdk)
+      if: steps.type.outputs.type == 'typescript'
+      run: |
+        npm run build --workspace=hindsight-clients/typescript
+        npm run build --workspace=hindsight-all-npm
+        npm run build --workspace=hindsight-tools/hindsight-agent-sdk
+
+    - name: Install integration dependencies
+      if: steps.type.outputs.type == 'typescript'
+      working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
+      run: npm ci
+
+    - name: Build TypeScript package
+      if: steps.type.outputs.type == 'typescript'
+      working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
+      run: npm run build
+
+    # Build-provenance attestations for the Obsidian release assets (community-store
+    # recommendation). Runs after the build so main.js exists. The assets are
+    # released in the dedicated repo while the build runs here, so users verify at
+    # owner scope: `gh attestation verify main.js --owner vectorize-io`.
+    - name: Attest Obsidian plugin build provenance
+      if: steps.type.outputs.type == 'typescript' && steps.info.outputs.integration == 'obsidian'
+      uses: actions/attest-build-provenance@v2
+      with:
+        subject-path: |
+          hindsight-integrations/obsidian/main.js
+          hindsight-integrations/obsidian/styles.css
+
+    # ── Obsidian plugin — mirror to its dedicated repo + cut the BRAT release ──
+    # We do NOT create a GitHub Release in this monorepo: per-integration
+    # releases pollute the repo's release list (it's for the core product) and
+    # steal the "Latest" badge, and BRAT / the community store read a repo's
+    # *latest* release — not a tag — so they can't target a tag in a monorepo.
+    #
+    # Instead this monorepo stays the source of truth, and on each obsidian
+    # release we mirror hindsight-integrations/obsidian/ → the *root* of
+    # github.com/vectorize-io/hindsight-obsidian (git subtree, history
+    # preserved) and cut the BRAT / community-store release *there*.
+    #
+    # Requires secret OBSIDIAN_DIST_TOKEN — a token with `contents: write` on
+    # vectorize-io/hindsight-obsidian (fine-grained PAT or app installation
+    # token). The dedicated repo is generated; never edit it directly.
+    - name: Mirror Obsidian plugin to its dedicated repo
+      if: steps.type.outputs.type == 'typescript' && steps.info.outputs.integration == 'obsidian'
+      env:
+        DIST_TOKEN: ${{ secrets.OBSIDIAN_DIST_TOKEN }}
+      run: |
+        set -euo pipefail
+        VERSION="${{ steps.info.outputs.version }}"
+        DIST_REPO="vectorize-io/hindsight-obsidian"
+        OBS_DIR="hindsight-integrations/obsidian"
+
+        # `git subtree split` needs full history; the default checkout is shallow.
+        git fetch --unshallow 2>/dev/null || true
+        git config user.name "github-actions[bot]"
+        git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
+
+        # The runner injects the default GITHUB_TOKEN as an http.extraheader via
+        # an *included* config file (/home/runner/work/_temp/git-credentials-*.config),
+        # so `git config --local --unset-all` can't remove it and it authenticates
+        # the push as github-actions[bot] (no access to the dedicated repo → 403).
+        # The documented way to drop an inherited extraheader is to RESET the list
+        # with an empty value: since command-line `-c` is read last, the empty
+        # value clears the accumulated headers (including the included one) at
+        # request-build time. The dist token then comes from the push URL → a
+        # single Authorization header.
+        git subtree split --prefix="$OBS_DIR" -b _obs_dist
+        git -c "http.https://github.com/.extraheader=" \
+          push "https://x-access-token:${DIST_TOKEN}@github.com/${DIST_REPO}.git" _obs_dist:main
+
+        # Cut the BRAT / community-store release. Bare version tag (e.g. 0.1.0)
+        # to match manifest.json — idempotent so re-runs just refresh the assets.
+        export GH_TOKEN="$DIST_TOKEN"
+        ASSETS="$OBS_DIR/main.js $OBS_DIR/manifest.json $OBS_DIR/styles.css"
+        NOTES="Hindsight for Obsidian v${VERSION}. Install via BRAT (add ${DIST_REPO}) or copy main.js/manifest.json/styles.css into <vault>/.obsidian/plugins/hindsight/."
+        if gh release view "$VERSION" --repo "$DIST_REPO" >/dev/null 2>&1; then
+          gh release upload "$VERSION" $ASSETS --repo "$DIST_REPO" --clobber
+        else
+          gh release create "$VERSION" $ASSETS --repo "$DIST_REPO" --title "$VERSION" --notes "$NOTES"
+        fi
+
+    - name: Publish TypeScript package to npm
+      if: steps.type.outputs.type == 'typescript'
+      working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
+      run: |
+        set +e
+        OUTPUT=$(npm publish --access public --provenance 2>&1)
+        EXIT_CODE=$?
+        echo "$OUTPUT"
+        if [ $EXIT_CODE -ne 0 ]; then
+          # Treat "already published" as success so re-pointed-tag re-runs stay green.
+          # "cannot publish over" = the version exists. TLOG_CREATE_ENTRY_ERROR / 409
+          # "equivalent entry already exists in the transparency log" = the identical
+          # --provenance artifact was already logged on a prior run (Sigstore tlog is
+          # idempotent); the package is published, so this is benign.
+          if echo "$OUTPUT" | grep -qE "cannot publish over|TLOG_CREATE_ENTRY_ERROR|already exists in the transparency log"; then
+            echo "Package version already published, skipping..."
+            exit 0
+          fi
+          exit $EXIT_CODE
+        fi
+      env:
+        NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

.github/workflows/release-tool.yml

@@ -0,0 +1,65 @@
+name: Release Tool
+
+on:
+  push:
+    tags:
+      - 'tools/**'
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v6
+
+    - name: Extract tool info
+      id: info
+      run: |
+        # refs/tags/tools/self-driving-agents/v0.0.1 → tool=self-driving-agents, version=0.0.1
+        TAG="${GITHUB_REF#refs/tags/}"
+        TOOL=$(echo "$TAG" | cut -d'/' -f2)
+        VERSION=$(echo "$TAG" | cut -d'/' -f3 | sed 's/^v//')
+        echo "tool=$TOOL" >> $GITHUB_OUTPUT
+        echo "version=$VERSION" >> $GITHUB_OUTPUT
+        echo "tag=$TAG" >> $GITHUB_OUTPUT
+        echo "Tool: $TOOL, Version: $VERSION"
+
+    - name: Set up Node.js
+      uses: actions/setup-node@v6
+      with:
+        node-version: '22'
+        registry-url: 'https://registry.npmjs.org'
+        cache: 'npm'
+        cache-dependency-path: package-lock.json
+
+    # Tools live under hindsight-tools/ and may depend on workspace packages
+    # (e.g. @vectorize-io/hindsight-client). Install from root so npm resolves
+    # workspace deps, then build any required workspace packages first.
+    - name: Install root workspace dependencies
+      run: npm ci
+
+    - name: Build hindsight-client (workspace dep)
+      run: npm run build --workspace=hindsight-clients/typescript
+
+    - name: Build hindsight-agent-sdk (workspace dep)
+      run: npm run build --workspace=hindsight-tools/hindsight-agent-sdk
+
+    - name: Build tool
+      run: npm run build --workspace=hindsight-tools/${{ steps.info.outputs.tool }}
+
+    - name: Publish to npm
+      working-directory: ./hindsight-tools/${{ steps.info.outputs.tool }}
+      run: |
+        set +e
+        OUTPUT=$(npm publish --access public 2>&1)
+        EXIT_CODE=$?
+        echo "$OUTPUT"
+        if [ $EXIT_CODE -ne 0 ]; then
+          if echo "$OUTPUT" | grep -q "cannot publish over"; then
+            echo "Package version already published, skipping..."
+            exit 0
+          fi
+          exit $EXIT_CODE
+        fi
+      env:
+        NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

.github/workflows/release.yml

@@ -13,38 +13,88 @@ jobs:
       id-token: write
 
     steps:
-    - uses: actions/checkout@v4
+    - uses: actions/checkout@v6
 
     - name: Install uv
-      uses: astral-sh/setup-uv@v5
+      uses: astral-sh/setup-uv@v7
       with:
         enable-cache: true
 
     - name: Set up Python
-      uses: actions/setup-python@v5
+      uses: actions/setup-python@v6
       with:
         python-version-file: ".python-version"
 
+    # Each package is built from its own directory, so stage the repository's
+    # canonical license inside each isolated build context.
+    - name: Stage Python package licenses
+      run: |
+        for package in \
+          hindsight-clients/python \
+          hindsight-api-slim \
+          hindsight-api \
+          hindsight-all \
+          hindsight-all-slim \
+          hindsight-embed; do
+          cp LICENSE "$package/LICENSE"
+        done
+
     # Build all packages
     - name: Build hindsight-client
       working-directory: ./hindsight-clients/python
       run: uv build --out-dir dist
 
+    - name: Build hindsight-api-slim
+      working-directory: ./hindsight-api-slim
+      run: uv build --out-dir dist
+
     - name: Build hindsight-api
       working-directory: ./hindsight-api
       run: uv build --out-dir dist
 
     - name: Build hindsight-all
-      working-directory: ./hindsight
+      working-directory: ./hindsight-all
       run: uv build --out-dir dist
 
-    # Publish in order (client and api first, then hindsight-all which depends on them)
+    - name: Build hindsight-all-slim
+      working-directory: ./hindsight-all-slim
+      run: uv build --out-dir dist
+
+    - name: Build hindsight-embed
+      working-directory: ./hindsight-embed
+      run: uv build --out-dir dist
+
+    - name: Verify Python package licenses
+      run: |
+        for package in \
+          hindsight-clients/python \
+          hindsight-api-slim \
+          hindsight-api \
+          hindsight-all \
+          hindsight-all-slim \
+          hindsight-embed; do
+          wheel=$(find "$package/dist" -maxdepth 1 -name '*.whl' -print -quit)
+          sdist=$(find "$package/dist" -maxdepth 1 -name '*.tar.gz' -print -quit)
+
+          unzip -Z1 "$wheel" | grep -Eq '\.dist-info/licenses/LICENSE$'
+          unzip -p "$wheel" '*/METADATA' | grep -Fxq 'License-Expression: MIT'
+          unzip -p "$wheel" '*/METADATA' | grep -Fxq 'License-File: LICENSE'
+          tar -tzf "$sdist" | grep -Eq '/LICENSE$'
+        done
+
+    # Publish in order (client and api-slim first, then api/all wrappers which depend on them)
     - name: Publish hindsight-client to PyPI
       uses: pypa/gh-action-pypi-publish@release/v1
       with:
         packages-dir: ./hindsight-clients/python/dist
         skip-existing: true
 
+    - name: Publish hindsight-api-slim to PyPI
+      uses: pypa/gh-action-pypi-publish@release/v1
+      with:
+        packages-dir: ./hindsight-api-slim/dist
+        skip-existing: true
+
     - name: Publish hindsight-api to PyPI
       uses: pypa/gh-action-pypi-publish@release/v1
       with:
@@ -54,18 +104,33 @@ jobs:
     - name: Publish hindsight-all to PyPI
       uses: pypa/gh-action-pypi-publish@release/v1
       with:
-        packages-dir: ./hindsight/dist
+        packages-dir: ./hindsight-all/dist
+        skip-existing: true
+
+    - name: Publish hindsight-all-slim to PyPI
+      uses: pypa/gh-action-pypi-publish@release/v1
+      with:
+        packages-dir: ./hindsight-all-slim/dist
+        skip-existing: true
+
+    - name: Publish hindsight-embed to PyPI
+      uses: pypa/gh-action-pypi-publish@release/v1
+      with:
+        packages-dir: ./hindsight-embed/dist
         skip-existing: true
 
     # Upload artifacts for GitHub release
     - name: Upload artifacts
-      uses: actions/upload-artifact@v4
+      uses: actions/upload-artifact@v7
       with:
         name: python-packages
         path: |
           hindsight-clients/python/dist/*
+          hindsight-api-slim/dist/*
           hindsight-api/dist/*
-          hindsight/dist/*
+          hindsight-all/dist/*
+          hindsight-all-slim/dist/*
+          hindsight-embed/dist/*
         retention-days: 1
 
   release-typescript-client:
@@ -73,25 +138,36 @@ jobs:
     environment: npm
 
     steps:
-    - uses: actions/checkout@v4
+    - uses: actions/checkout@v6
 
     - name: Set up Node.js
-      uses: actions/setup-node@v4
+      uses: actions/setup-node@v6
       with:
         node-version: '20'
         registry-url: 'https://registry.npmjs.org'
+        cache: 'npm'
+        cache-dependency-path: package-lock.json
 
     - name: Install dependencies
-      working-directory: ./hindsight-clients/typescript
-      run: npm ci
+      run: npm ci --workspace=hindsight-clients/typescript
 
     - name: Build
-      working-directory: ./hindsight-clients/typescript
-      run: npm run build
+      run: npm run build --workspace=hindsight-clients/typescript
 
     - name: Publish to npm
       working-directory: ./hindsight-clients/typescript
-      run: npm publish --access public
+      run: |
+        set +e
+        OUTPUT=$(npm publish --access public 2>&1)
+        EXIT_CODE=$?
+        echo "$OUTPUT"
+        if [ $EXIT_CODE -ne 0 ]; then
+          if echo "$OUTPUT" | grep -q "cannot publish over"; then
+            echo "Package version already published, skipping..."
+            exit 0
+          fi
+          exit $EXIT_CODE
+        fi
       env:
         NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
 
@@ -100,18 +176,129 @@ jobs:
       run: npm pack
 
     - name: Upload artifacts
-      uses: actions/upload-artifact@v4
+      uses: actions/upload-artifact@v7
       with:
         name: typescript-client
         path: hindsight-clients/typescript/*.tgz
         retention-days: 1
 
+  release-hindsight-all-npm:
+    runs-on: ubuntu-latest
+    environment: npm
+
+    steps:
+    - uses: actions/checkout@v6
+
+    - name: Set up Node.js
+      uses: actions/setup-node@v6
+      with:
+        node-version: '22'
+        registry-url: 'https://registry.npmjs.org'
+        cache: 'npm'
+        cache-dependency-path: package-lock.json
+
+    - name: Install dependencies
+      run: npm ci --workspace=hindsight-all-npm
+
+    - name: Build
+      run: npm run build --workspace=hindsight-all-npm
+
+    - name: Publish to npm
+      working-directory: ./hindsight-all-npm
+      run: |
+        set +e
+        OUTPUT=$(npm publish --access public 2>&1)
+        EXIT_CODE=$?
+        echo "$OUTPUT"
+        if [ $EXIT_CODE -ne 0 ]; then
+          if echo "$OUTPUT" | grep -q "cannot publish over"; then
+            echo "Package version already published, skipping..."
+            exit 0
+          fi
+          exit $EXIT_CODE
+        fi
+      env:
+        NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
+
+    - name: Pack for GitHub release
+      working-directory: ./hindsight-all-npm
+      run: npm pack
+
+    - name: Upload artifacts
+      uses: actions/upload-artifact@v7
+      with:
+        name: hindsight-all-npm
+        path: hindsight-all-npm/*.tgz
+        retention-days: 1
+
+  release-control-plane:
+    runs-on: ubuntu-latest
+    environment: npm
+
+    steps:
+    - uses: actions/checkout@v6
+
+    - name: Set up Node.js
+      uses: actions/setup-node@v6
+      with:
+        node-version: '20'
+        registry-url: 'https://registry.npmjs.org'
+        cache: 'npm'
+        cache-dependency-path: package-lock.json
+
+    - name: Install dependencies
+      run: npm ci
+
+    - name: Build TypeScript client (dependency)
+      run: npm run build --workspace=hindsight-clients/typescript
+
+    - name: Fix platform-specific native modules
+      run: |
+        # npm ci installs from lockfile which may have wrong platform binaries
+        # Delete hoisted native modules and reinstall for current platform
+        rm -rf node_modules/lightningcss node_modules/@tailwindcss
+        npm install lightningcss @tailwindcss/postcss @tailwindcss/node
+
+    - name: Build
+      run: npm run build --workspace=hindsight-control-plane
+
+    - name: Verify standalone build
+      run: test -f hindsight-control-plane/standalone/server.js || (echo 'standalone/server.js missing - build failed' && exit 1)
+
+    - name: Publish to npm
+      working-directory: ./hindsight-control-plane
+      run: |
+        set +e
+        OUTPUT=$(npm publish --access public --ignore-scripts 2>&1)
+        EXIT_CODE=$?
+        echo "$OUTPUT"
+        if [ $EXIT_CODE -ne 0 ]; then
+          if echo "$OUTPUT" | grep -q "cannot publish over"; then
+            echo "Package version already published, skipping..."
+            exit 0
+          fi
+          exit $EXIT_CODE
+        fi
+      env:
+        NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
+
+    - name: Pack for GitHub release
+      working-directory: ./hindsight-control-plane
+      run: npm pack
+
+    - name: Upload artifacts
+      uses: actions/upload-artifact@v7
+      with:
+        name: control-plane
+        path: hindsight-control-plane/*.tgz
+        retention-days: 1
+
   release-rust-cli:
     runs-on: ${{ matrix.os }}
     strategy:
       matrix:
         include:
-          - os: ubuntu-latest
+          - os: ubuntu-22.04
             target: x86_64-unknown-linux-gnu
             artifact_name: hindsight
             asset_name: hindsight-linux-amd64
@@ -123,9 +310,13 @@ jobs:
             target: aarch64-apple-darwin
             artifact_name: hindsight
             asset_name: hindsight-darwin-arm64
+          - os: ubuntu-22.04-arm
+            target: aarch64-unknown-linux-gnu
+            artifact_name: hindsight
+            asset_name: hindsight-linux-arm64
 
     steps:
-    - uses: actions/checkout@v4
+    - uses: actions/checkout@v6
 
     - name: Install Rust
       uses: dtolnay/rust-toolchain@stable
@@ -143,34 +334,54 @@ jobs:
         chmod +x artifacts/${{ matrix.asset_name }}
 
     - name: Upload artifacts
-      uses: actions/upload-artifact@v4
+      uses: actions/upload-artifact@v7
       with:
         name: rust-cli-${{ matrix.asset_name }}
         path: artifacts/${{ matrix.asset_name }}
         retention-days: 1
 
   release-docker-images:
+    name: Release Docker (${{ matrix.image_name }}${{ matrix.tag_suffix }})
     runs-on: ubuntu-latest
     permissions:
       contents: read
       packages: write
+      id-token: write
     strategy:
       matrix:
         include:
           - target: api-only
             image_name: hindsight-api
+            tag_suffix: ""
+            build_args: ""
+          - target: api-only
+            image_name: hindsight-api
+            tag_suffix: "-slim"
+            build_args: |
+              INCLUDE_LOCAL_MODELS=false
+              PRELOAD_ML_MODELS=false
           - target: cp-only
             image_name: hindsight-control-plane
+            tag_suffix: ""
+            build_args: ""
           - target: standalone
             image_name: hindsight
+            tag_suffix: ""
+            build_args: ""
+          - target: standalone
+            image_name: hindsight
+            tag_suffix: "-slim"
+            build_args: |
+              INCLUDE_LOCAL_MODELS=false
+              PRELOAD_ML_MODELS=false
 
     steps:
-    - uses: actions/checkout@v4
+    - uses: actions/checkout@v6
 
     - name: Free Disk Space
       uses: jlumbroso/free-disk-space@main
       with:
-        tool-cache: false
+        tool-cache: true
         android: true
         dotnet: true
         haskell: true
@@ -179,13 +390,13 @@ jobs:
         swap-storage: true
 
     - name: Set up QEMU
-      uses: docker/setup-qemu-action@v3
+      uses: docker/setup-qemu-action@v4
 
     - name: Set up Docker Buildx
-      uses: docker/setup-buildx-action@v3
+      uses: docker/setup-buildx-action@v4
 
     - name: Log in to GitHub Container Registry
-      uses: docker/login-action@v3
+      uses: docker/login-action@v4
       with:
         registry: ghcr.io
         username: ${{ github.actor }}
@@ -195,28 +406,80 @@ jobs:
       id: get_version
       run: echo "VERSION=${GITHUB_REF#refs/tags/v}" >> $GITHUB_OUTPUT
 
-    - name: Extract metadata
+    - name: Extract metadata for release tags
       id: meta
-      uses: docker/metadata-action@v5
+      uses: docker/metadata-action@v6
       with:
         images: ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}
+        flavor: |
+          latest=auto
+          suffix=${{ matrix.tag_suffix }}
         tags: |
           type=semver,pattern={{version}},value=${{ steps.get_version.outputs.VERSION }}
           type=semver,pattern={{major}}.{{minor}},value=${{ steps.get_version.outputs.VERSION }}
           type=semver,pattern={{major}},value=${{ steps.get_version.outputs.VERSION }}
           type=raw,value=latest
 
-    - name: Build and push
-      uses: docker/build-push-action@v6
+    # TODO: Re-enable smoke test when disk space issue is resolved
+    # # Step 1: Build for local testing (single platform, no push)
+    # # This creates an identical image to what will be released, just for one platform
+    # - name: Build image for testing
+    #   uses: docker/build-push-action@v7
+    #   with:
+    #     context: .
+    #     file: docker/standalone/Dockerfile
+    #     target: ${{ matrix.target }}
+    #     push: false
+    #     load: true
+    #     tags: ${{ matrix.image_name }}:test
+    #     cache-from: type=gha
+    #     cache-to: type=gha,mode=max
+
+    # # Step 2: Test the image before pushing anything
+    # - name: Smoke test - verify container starts
+    #   env:
+    #     GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
+    #   run: ./docker/test-image.sh "${{ matrix.image_name }}:test" "${{ matrix.target }}"
+
+    # Build multi-platform and push to release tags
+    - name: Build and push release images
+      id: build
+      uses: docker/build-push-action@v7
       with:
         context: .
         file: docker/standalone/Dockerfile
         target: ${{ matrix.target }}
+        build-args: ${{ matrix.build_args }}
         push: true
         platforms: linux/amd64,linux/arm64
         tags: ${{ steps.meta.outputs.tags }}
         labels: ${{ steps.meta.outputs.labels }}
 
+    - name: Install cosign
+      uses: sigstore/cosign-installer@v3
+
+    - name: Sign published images
+      env:
+        TAGS: ${{ steps.meta.outputs.tags }}
+        DIGEST: ${{ steps.build.outputs.digest }}
+      run: |
+        set -euo pipefail
+        refs=()
+        while IFS= read -r tag; do
+          [[ -z "${tag}" ]] && continue
+          refs+=("${tag}@${DIGEST}")
+        done <<< "${TAGS}"
+        cosign sign --yes "${refs[@]}"
+
+    - name: Verify signature on primary tag
+      env:
+        IMAGE: ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}
+        DIGEST: ${{ steps.build.outputs.digest }}
+      run: |
+        cosign verify "${IMAGE}@${DIGEST}" \
+          --certificate-identity-regexp "^https://github\.com/${{ github.repository }}/\.github/workflows/release\.yml@.*" \
+          --certificate-oidc-issuer https://token.actions.githubusercontent.com
+
   release-helm-chart:
     runs-on: ubuntu-latest
     permissions:
@@ -224,10 +487,10 @@ jobs:
       packages: write
 
     steps:
-    - uses: actions/checkout@v4
+    - uses: actions/checkout@v6
 
     - name: Install Helm
-      uses: azure/setup-helm@v4
+      uses: azure/setup-helm@v5
       with:
         version: 'latest'
 
@@ -244,7 +507,7 @@ jobs:
       run: helm push helm-packages/*.tgz oci://ghcr.io/${{ github.repository_owner }}/charts
 
     - name: Upload artifacts
-      uses: actions/upload-artifact@v4
+      uses: actions/upload-artifact@v7
       with:
         name: helm-chart
         path: helm-packages/*.tgz
@@ -252,49 +515,67 @@ jobs:
 
   create-github-release:
     runs-on: ubuntu-latest
-    needs: [release-python-packages, release-typescript-client, release-rust-cli, release-docker-images, release-helm-chart]
+    needs: [release-python-packages, release-typescript-client, release-hindsight-all-npm, release-control-plane, release-rust-cli, release-docker-images, release-helm-chart]
     permissions:
       contents: write
 
     steps:
-    - uses: actions/checkout@v4
+    - uses: actions/checkout@v6
 
     - name: Extract version from tag
       id: get_version
       run: echo "VERSION=${GITHUB_REF#refs/tags/v}" >> $GITHUB_OUTPUT
 
     - name: Download Python packages
-      uses: actions/download-artifact@v4
+      uses: actions/download-artifact@v8
       with:
         name: python-packages
         path: ./artifacts/python-packages
 
     - name: Download TypeScript client
-      uses: actions/download-artifact@v4
+      uses: actions/download-artifact@v8
       with:
         name: typescript-client
         path: ./artifacts/typescript-client
 
+    - name: Download Control Plane
+      uses: actions/download-artifact@v8
+      with:
+        name: control-plane
+        path: ./artifacts/control-plane
+
+    - name: Download hindsight-embed npm wrapper
+      uses: actions/download-artifact@v8
+      with:
+        name: hindsight-all-npm
+        path: ./artifacts/hindsight-all-npm
+
     - name: Download Rust CLI (Linux)
-      uses: actions/download-artifact@v4
+      uses: actions/download-artifact@v8
       with:
         name: rust-cli-hindsight-linux-amd64
         path: ./artifacts/rust-cli-linux
 
+    - name: Download Rust CLI (Linux ARM)
+      uses: actions/download-artifact@v8
+      with:
+        name: rust-cli-hindsight-linux-arm64
+        path: ./artifacts/rust-cli-linux-arm64
+
     - name: Download Rust CLI (macOS Intel)
-      uses: actions/download-artifact@v4
+      uses: actions/download-artifact@v8
       with:
         name: rust-cli-hindsight-darwin-amd64
         path: ./artifacts/rust-cli-darwin-amd64
 
     - name: Download Rust CLI (macOS ARM)
-      uses: actions/download-artifact@v4
+      uses: actions/download-artifact@v8
       with:
         name: rust-cli-hindsight-darwin-arm64
         path: ./artifacts/rust-cli-darwin-arm64
 
     - name: Download Helm chart
-      uses: actions/download-artifact@v4
+      uses: actions/download-artifact@v8
       with:
         name: helm-chart
         path: ./artifacts/helm-chart
@@ -304,66 +585,31 @@ jobs:
         mkdir -p release-assets
         # Python packages
         cp artifacts/python-packages/hindsight-clients/python/dist/* release-assets/ || true
+        cp artifacts/python-packages/hindsight-api-slim/dist/* release-assets/ || true
         cp artifacts/python-packages/hindsight-api/dist/* release-assets/ || true
-        cp artifacts/python-packages/hindsight/dist/* release-assets/ || true
+        cp artifacts/python-packages/hindsight-all/dist/* release-assets/ || true
+        cp artifacts/python-packages/hindsight-all-slim/dist/* release-assets/ || true
+        cp artifacts/python-packages/hindsight-embed/dist/* release-assets/ || true
         # TypeScript client
         cp artifacts/typescript-client/*.tgz release-assets/ || true
+        # hindsight-embed npm wrapper
+        cp artifacts/hindsight-all-npm/*.tgz release-assets/ || true
+        # Control Plane
+        cp artifacts/control-plane/*.tgz release-assets/ || true
         # Rust CLI binaries
         cp artifacts/rust-cli-linux/hindsight-linux-amd64 release-assets/ || true
+        cp artifacts/rust-cli-linux-arm64/hindsight-linux-arm64 release-assets/ || true
         cp artifacts/rust-cli-darwin-amd64/hindsight-darwin-amd64 release-assets/ || true
         cp artifacts/rust-cli-darwin-arm64/hindsight-darwin-arm64 release-assets/ || true
         # Helm chart
         cp artifacts/helm-chart/*.tgz release-assets/ || true
         ls -la release-assets/
 
-    - name: Generate release notes
-      run: |
-        cat << 'EOF' > release-notes.md
-        ## Quick Start
-
-        ```bash
-        # Install the CLI
-        curl -fsSL https://raw.githubusercontent.com/vectorize-io/hindsight/refs/heads/main/hindsight-cli/install.sh | bash
-
-        # Start the server
-        docker run -p 8888:8888 -p 9999:9999 \
-          -e HINDSIGHT_API_LLM_PROVIDER=openai \
-          -e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \
-          -e HINDSIGHT_API_LLM_MODEL=gpt-4o-mini \
-          ghcr.io/${{ github.repository_owner }}/hindsight:${{ steps.get_version.outputs.VERSION }}
-        ```
-
-        ## Docker Images
-        - `ghcr.io/${{ github.repository_owner }}/hindsight:${{ steps.get_version.outputs.VERSION }}` - Standalone (recommended)
-        - `ghcr.io/${{ github.repository_owner }}/hindsight-api:${{ steps.get_version.outputs.VERSION }}` - API only
-        - `ghcr.io/${{ github.repository_owner }}/hindsight-control-plane:${{ steps.get_version.outputs.VERSION }}` - Web UI only
-
-        ## CLI
-        ```bash
-        curl -fsSL https://raw.githubusercontent.com/vectorize-io/hindsight/refs/heads/main/hindsight-cli/install.sh | bash
-        ```
-
-        ## Python
-        ```bash
-        pip install hindsight-all  # or hindsight-api, hindsight-client
-        ```
-
-        ## TypeScript/JavaScript
-        ```bash
-        npm install @vectorize-io/hindsight-client
-        ```
-
-        ## Helm
-        ```bash
-        helm install hindsight oci://ghcr.io/${{ github.repository_owner }}/charts/hindsight --version ${{ steps.get_version.outputs.VERSION }}
-        ```
-        EOF
-
     - name: Create GitHub Release
-      uses: softprops/action-gh-release@v2
+      uses: softprops/action-gh-release@v3
       with:
         files: release-assets/*
-        body_path: release-notes.md
+        generate_release_notes: true
         draft: false
         prerelease: false
       env:

.github/workflows/sign-images.yml

@@ -0,0 +1,71 @@
+name: Sign published images
+
+on:
+  workflow_dispatch:
+    inputs:
+      version:
+        description: 'Version to sign (without leading v, e.g. 0.6.0)'
+        required: true
+        type: string
+        default: '0.6.0'
+
+permissions:
+  contents: read
+  packages: write
+  id-token: write
+
+jobs:
+  sign:
+    name: Sign ${{ matrix.image }}:${{ inputs.version }}${{ matrix.suffix }}
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - { image: hindsight-api, suffix: '' }
+          - { image: hindsight-api, suffix: '-slim' }
+          - { image: hindsight-control-plane, suffix: '' }
+          - { image: hindsight, suffix: '' }
+          - { image: hindsight, suffix: '-slim' }
+
+    steps:
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v4
+
+      - name: Log in to GHCR
+        uses: docker/login-action@v4
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Install cosign
+        uses: sigstore/cosign-installer@v3
+
+      - name: Resolve image digest
+        id: resolve
+        env:
+          IMAGE: ghcr.io/${{ github.repository_owner }}/${{ matrix.image }}
+          TAG: ${{ inputs.version }}${{ matrix.suffix }}
+        run: |
+          set -euo pipefail
+          DIGEST=$(docker buildx imagetools inspect "${IMAGE}:${TAG}" --format '{{json .Manifest.Digest}}' | tr -d '"')
+          if [[ -z "${DIGEST}" || "${DIGEST}" != sha256:* ]]; then
+            echo "Failed to resolve digest for ${IMAGE}:${TAG} (got: ${DIGEST})" >&2
+            exit 1
+          fi
+          echo "Resolved ${IMAGE}:${TAG} -> ${DIGEST}"
+          echo "ref=${IMAGE}@${DIGEST}" >> "$GITHUB_OUTPUT"
+
+      - name: Sign image
+        env:
+          REF: ${{ steps.resolve.outputs.ref }}
+        run: cosign sign --yes "${REF}"
+
+      - name: Verify signature
+        env:
+          REF: ${{ steps.resolve.outputs.ref }}
+        run: |
+          cosign verify "${REF}" \
+            --certificate-identity-regexp "^https://github\.com/${{ github.repository }}/\.github/workflows/sign-images\.yml@.*" \
+            --certificate-oidc-issuer https://token.actions.githubusercontent.com

.github/workflows/star-history.yml

@@ -0,0 +1,29 @@
+name: Update star history
+
+on:
+  schedule:
+    - cron: '17 3 * * *'
+  workflow_dispatch:
+
+permissions:
+  contents: write
+
+jobs:
+  update:
+    concurrency:
+      group: star-history
+      cancel-in-progress: false
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: nicoloboschi/gh-stars@v1
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          line-color: '#14b8a6'
+      - name: Commit chart
+        run: |
+          git config user.name 'github-actions[bot]'
+          git config user.email '41898282+github-actions[bot]@users.noreply.github.com'
+          git add .github/star-history/data.json .github/star-history/chart.svg
+          git diff --cached --quiet || git commit -m 'chore: update star history'
+          git push

.github/workflows/windows-smoke.yml

@@ -0,0 +1,115 @@
+name: Windows Smoke Test
+
+# Daily smoke test that installs the API on Windows and runs the Python client
+# integration tests against a live server. Windows is only exercised by the
+# hindsight-embed jobs in test.yml on PRs; this catches Windows-specific
+# regressions in the API server + client path (e.g. process spawning, console
+# subsystem / ConPTY behaviour, see #1885) that the Linux client jobs miss.
+on:
+  schedule:
+    # 06:00 UTC daily.
+    - cron: "0 6 * * *"
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  windows-client-smoke:
+    # Don't run on forks: the job needs the org's Vertex AI credentials.
+    if: github.repository == 'vectorize-io/hindsight'
+    runs-on: windows-latest
+    timeout-minutes: 45
+    env:
+      HINDSIGHT_API_LLM_PROVIDER: vertexai
+      HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: ${{ github.workspace }}/gcp-credentials.json
+      HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
+      HINDSIGHT_API_URL: http://localhost:8888
+      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      # Force UTF-8 I/O so the API/CLI's ✓/box-drawing output doesn't crash the
+      # default Windows cp1252 codec (matches test-embed-windows in test.yml).
+      PYTHONIOENCODING: utf-8
+      PYTHONUTF8: "1"
+
+    steps:
+      - uses: actions/checkout@v6
+
+      - name: Setup GCP credentials
+        shell: bash
+        run: |
+          printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > gcp-credentials.json
+          PROJECT_ID=$(jq -r '.project_id' gcp-credentials.json)
+          echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> "$GITHUB_ENV"
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
+        with:
+          enable-cache: true
+          prune-cache: false
+
+      - name: Set up Python
+        uses: actions/setup-python@v6
+        with:
+          python-version-file: ".python-version"
+
+      - name: Install API dependencies (all extras - local-ml + embedded pg0)
+        working-directory: ./hindsight-api-slim
+        run: uv sync --frozen --all-extras --index-strategy unsafe-best-match
+
+      - name: Install Python client test dependencies
+        working-directory: ./hindsight-clients/python
+        run: uv sync --frozen --extra test --index-strategy unsafe-best-match
+
+      - name: Cache HuggingFace models
+        uses: actions/cache@v5
+        with:
+          path: ~/.cache/huggingface
+          key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
+          restore-keys: |
+            ${{ runner.os }}-huggingface-
+
+      # `uv run` re-syncs the project env to its default (no-extras) state before
+      # running, which drops sentence-transformers / pg0. Pass --all-extras on
+      # every `uv run` so the local-ml + embedded-db deps stay installed (this is
+      # the same reason hindsight-embed launches the daemon with `--extra all`).
+      - name: Pre-download models
+        working-directory: ./hindsight-api-slim
+        run: |
+          uv run --all-extras python -c "from sentence_transformers import SentenceTransformer, CrossEncoder; SentenceTransformer('BAAI/bge-small-en-v1.5'); CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2'); print('Models downloaded')"
+
+      # Start the server and run the client tests in a SINGLE step. On Windows
+      # runners a process backgrounded with `&` in one step is not reliably kept
+      # alive for later steps (unlike Linux, where it reparents to init), so the
+      # server must live in the same shell that runs pytest.
+      - name: Start API server and run Python client tests
+        shell: bash
+        run: |
+          # Config is read straight from the environment (job-level env + the
+          # PROJECT_ID exported to GITHUB_ENV above), so no .env file is needed.
+          # Embedded pg0 is the default when HINDSIGHT_API_DATABASE_URL is unset.
+          ( cd hindsight-api-slim && uv run --all-extras hindsight-api --port 8888 ) > "$RUNNER_TEMP/api-server.log" 2>&1 &
+          server_pid=$!
+          echo "Waiting for API server to be ready (pid $server_pid)..."
+          # pg0 unpacks Postgres + runs initdb on first boot, which is slow on a
+          # cold Windows runner — give it a generous budget before failing.
+          ready=false
+          for i in $(seq 1 300); do
+            if curl -sf http://localhost:8888/health > /dev/null 2>&1; then
+              echo "API server is ready after ${i}s"
+              ready=true
+              break
+            fi
+            sleep 1
+          done
+          if [ "$ready" != true ]; then
+            echo "API server failed to start after 300s"
+            cat "$RUNNER_TEMP/api-server.log"
+            exit 1
+          fi
+
+          cd hindsight-clients/python && uv run --extra test pytest tests -v
+
+      - name: Show API server logs
+        if: always()
+        shell: bash
+        run: cat "$RUNNER_TEMP/api-server.log" || echo "No API server log found"

.gitignore

@@ -5,12 +5,28 @@ build/
 dist/
 wheels/
 *.egg-info
-
+# Release builds stage the canonical root license in each package context.
+/hindsight-clients/python/LICENSE
+/hindsight-api-slim/LICENSE
+/hindsight-api/LICENSE
+/hindsight-all/LICENSE
+/hindsight-all-slim/LICENSE
+/hindsight-embed/LICENSE
+.mcp.json
+.playwright-mcp/
+.osgrep
 # Virtual environments
 .venv
 
-# Environment variables
+# Node
+node_modules/
+
+# Environment variables and local config
 .env
+.env.bak*
+.env.*.bak
+docker-compose.yml
+docker-compose.override.yml
 
 # IDE
 .idea/
@@ -21,6 +37,10 @@ wheels/
 # NLTK data (will be downloaded automatically)
 nltk_data/
 
+# Monitoring stack (Prometheus/Grafana binaries and data)
+.monitoring/
+.pgbouncer/
+
 # Large benchmark datasets (will be downloaded automatically)
 **/longmemeval_s_cleaned.json
 
@@ -28,9 +48,27 @@ nltk_data/
 logs/
 
 .DS_Store
+.sesskey
+
+# Generated docs files
+hindsight-docs/static/llms-full.txt
 
 
 hindsight-dev/benchmarks/locomo/results/
 hindsight-dev/benchmarks/longmemeval/results/
+hindsight-dev/benchmarks/consolidation/results/
+hindsight-dev/benchmarks/perf/results/
+benchmarks/results/
 hindsight-cli/target
-hindsight-clients/rust/target
+hindsight-clients/rust/target
+.claude/*
+!.claude/skills/
+whats-next.md
+TASK.md
+# Parked / draft integrations that aren't ready to ship
+hindsight-integrations/_drafts/
+# Changelog is now tracked in hindsight-docs/src/pages/changelog.md
+# CHANGELOG.md
+
+blog-post*
+.worktrees/

.prettierrc.json

@@ -0,0 +1,7 @@
+{
+  "semi": true,
+  "singleQuote": false,
+  "tabWidth": 2,
+  "trailingComma": "es5",
+  "printWidth": 100
+}

.sesskey

@@ -1 +0,0 @@
-fcac2839-1db5-432f-91e1-c5dac07d7290

AGENTS.md

@@ -1,151 +1,3 @@
 # AGENTS.md
 
-This document captures architectural decisions and coding conventions for the Hindsight project.
-
-## Documentation
-
-- **Main documentation**: [hindsight-docs/docs/developer/](./hindsight-docs/docs/developer/)
-- **Use case patterns**: [hindsight-docs/docs/cookbook/](./hindsight-docs/docs/cookbook/)
-- **API reference**: Auto-generated from OpenAPI spec
-
-## Project Structure
-
-```
-hindsight/              # Python package for embedded usage
-hindsight-api/          # FastAPI server (core memory engine)
-hindsight-cli/          # Rust CLI client
-hindsight-control-plane/ # Next.js admin UI
-hindsight-docs/         # Docusaurus documentation site
-hindsight-dev/          # Development tools and benchmarks
-hindsight-integrations/ # Framework integrations (LangChain, etc.)
-hindsight-clients/      # Generated API clients (Python, TypeScript, Rust)
-```
-
-## Core Concepts
-
-### Memory Banks
-- Each bank is an isolated memory store (like a "brain" for one user/agent)
-- Banks contain: memory units (facts), entities, documents, entity links
-- Banks have a **disposition** (personality traits) and **background** (context)
-- Bank isolation is strict - no cross-bank data leakage
-
-### Memory Types
-- **World facts**: General knowledge ("The sky is blue")
-- **Experience facts**: Personal experiences ("I visited Paris in 2023")
-- **Opinion facts**: Beliefs with confidence scores ("Paris is beautiful" - 0.9 confidence)
-
-### Operations
-- **Retain**: Store new memories (extracts facts, entities, relationships)
-- **Recall**: Retrieve memories (semantic, BM25, graph, temporal search)
-- **Reflect**: Deep analysis to form new insights/opinions
-
-## API Design Decisions
-
-### Single Bank Per Request
-- All API endpoints (`recall`, `reflect`, `retain`) operate on a single bank
-- Multi-bank queries are the **client/agent's responsibility** to orchestrate
-- This keeps the API simple and the isolation model clear
-
-### Disposition Traits (3-trait system)
-- **Skepticism** (1-5): How skeptical vs trusting when forming opinions
-- **Literalism** (1-5): How literally to interpret information
-- **Empathy** (1-5): How much to consider emotional context
-- These influence the `reflect` operation, not `recall`
-- Background info also only affects `reflect` (opinion formation)
-
-## Multi-Bank Architecture Patterns
-
-See [hindsight-docs/docs/cookbook/](./hindsight-docs/docs/cookbook/) for detailed guides:
-
-- **Per-User Memory**: One bank per user, simplest pattern
-- **Support Agent + Shared Knowledge**: User bank + shared docs bank, client orchestrates
-
-## Developer Guide
-
-### Running the API Server
-
-```bash
-# From project root
-./scripts/dev/start-api.sh
-
-# With options
-./scripts/dev/start-api.sh --reload --port 8888 --log-level debug
-```
-
-### Running Tests
-
-```bash
-# API tests
-cd hindsight-api
-uv run pytest tests/
-
-# Specific test
-uv run pytest tests/test_http_api_integration.py -v
-```
-
-### Generating OpenAPI Spec
-
-After changing API endpoints, regenerate the OpenAPI spec and docs:
-
-```bash
-./scripts/generate-openapi.sh
-```
-
-This will:
-1. Generate `openapi.json` at project root
-2. Copy to `hindsight-docs/openapi.json`
-3. Regenerate API reference documentation
-
-### Generating API Clients
-
-After updating the OpenAPI spec, regenerate all clients:
-
-```bash
-./scripts/generate-clients.sh
-```
-
-This generates:
-- **Rust client**: `hindsight-clients/rust/` (via progenitor in build.rs)
-- **Python client**: `hindsight-clients/python/` (via openapi-generator Docker)
-- **TypeScript client**: `hindsight-clients/typescript/` (via @hey-api/openapi-ts)
-
-Note: The maintained wrapper `hindsight_client.py` and `README.md` are preserved during regeneration.
-
-### Running the Documentation Site
-
-```bash
-./scripts/dev/start-docs.sh
-```
-
-### Running the Control Plane
-
-```bash
-./scripts/dev/start-control-plane.sh
-```
-
-## Code Style
-
-### Python (hindsight-api)
-- Use `uv` for package management
-- Async throughout (asyncpg, async FastAPI endpoints)
-- Pydantic models for request/response validation
-- No py files at project root - maintain clean directory structure
-
-### TypeScript (control-plane, clients)
-- Next.js with App Router for control plane
-- Tailwind CSS with shadcn/ui components
-
-### Rust (CLI)
-- Async with tokio
-- reqwest for HTTP client
-- progenitor for API client generation
-
-## Database
-
-- PostgreSQL with pgvector extension
-- Schema managed via Alembic migrations in `hindsight-api/alembic/`, db migrations happen during api startup, no manual commands
-- Key tables: `banks`, `memory_units`, `documents`, `entities`, `entity_links`
-
-# Branding
-## Colors
-- Primary: gradient from #0074d9 to #009296  
+See [CLAUDE.md](./CLAUDE.md) for project documentation and coding conventions.

CLAUDE.md

@@ -0,0 +1,426 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+Hindsight is an agent memory system that provides long-term memory for AI agents using biomimetic data structures. Memories are organized as:
+- **World facts**: General knowledge ("The sky is blue")
+- **Experience facts**: Personal experiences ("I visited Paris in 2023")
+- **Mental models**: Consolidated knowledge synthesized from facts ("User prefers functional programming patterns")
+
+## Development Commands
+
+### Local Development (API + UI)
+```bash
+# Start both API server and control plane UI
+./scripts/dev/start.sh
+```
+
+### API Server (Python/FastAPI)
+```bash
+# Start API server only (loads .env automatically)
+./scripts/dev/start-api.sh
+
+# Run all tests (parallelized with pytest-xdist)
+cd hindsight-api-slim && uv run pytest tests/
+
+# Run specific test file
+cd hindsight-api-slim && uv run pytest tests/test_http_api_integration.py -v
+
+# Run single test function
+cd hindsight-api-slim && uv run pytest tests/test_retain.py::test_retain_simple -v
+
+# Lint and format
+cd hindsight-api-slim && uv run ruff check .
+cd hindsight-api-slim && uv run ruff format .
+
+# Type checking (uses ty - extremely fast type checker from Astral)
+cd hindsight-api-slim && uv run ty check hindsight_api/
+```
+
+### Control Plane (Next.js)
+```bash
+./scripts/dev/start-control-plane.sh
+# Or manually:
+cd hindsight-control-plane && npm run dev
+```
+
+### Documentation Site (Docusaurus)
+```bash
+./scripts/dev/start-docs.sh
+```
+
+
+### Generating Clients/OpenAPI
+```bash
+# Regenerate OpenAPI spec after API changes (REQUIRED after changing endpoints)
+./scripts/generate-openapi.sh
+
+# Regenerate all client SDKs (Python, TypeScript, Rust)
+./scripts/generate-clients.sh
+```
+
+### Benchmarks
+```bash
+# Accuracy benchmarks
+./scripts/benchmarks/run-longmemeval.sh
+./scripts/benchmarks/run-locomo.sh
+
+# Performance benchmarks
+./scripts/benchmarks/run-perf-test.sh                      # System perf (mock LLM + pg0)
+./scripts/benchmarks/run-perf-test.sh --scale tiny          # Quick smoke test
+./scripts/benchmarks/run-consolidation.sh
+
+# Results viewer
+./scripts/benchmarks/start-visualizer.sh  # View results at localhost:8001
+```
+
+## Architecture
+
+### Monorepo Structure
+- **hindsight-api-slim/**: Core FastAPI server with memory engine (Python, uv)
+- **hindsight-control-plane/**: Admin UI (Next.js, npm)
+- **hindsight-cli/**: CLI tool (Rust, cargo, uses progenitor for API client)
+- **hindsight-clients/**: Generated SDK clients (Python, TypeScript, Rust)
+- **hindsight-docs/**: Docusaurus documentation site
+- **hindsight-integrations/**: Framework integrations (LiteLLM, CrewAI, LangGraph, Pydantic AI, AG2, Claude Code, etc.)
+- **hindsight-dev/**: Development tools and benchmarks
+
+### Core Engine (hindsight-api-slim/hindsight_api/engine/)
+- `memory_engine.py`: Main orchestrator for retain/recall/reflect operations
+- `llm_wrapper.py`: LLM abstraction supporting OpenAI, Anthropic, Gemini, VertexAI, Groq, MiniMax, Ollama, LM Studio, LiteLLM, Claude Code
+- `embeddings.py`: Embedding generation (local sentence-transformers or TEI)
+- `cross_encoder.py`: Reranking (local or TEI)
+- `entity_resolver.py`: Entity extraction and normalization
+- `query_analyzer.py`: Query intent analysis
+
+**retain/**: Memory ingestion pipeline
+- `orchestrator.py`: Coordinates the retain flow
+- `fact_extraction.py`: LLM-based fact extraction from content
+- `link_utils.py`: Entity link creation and management
+
+**search/**: Multi-strategy retrieval
+- `retrieval.py`: Main retrieval orchestrator
+- `graph_retrieval.py`: Graph retrieval abstract base class
+- `link_expansion_retrieval.py`: Link expansion graph retrieval
+- `fusion.py`: Reciprocal rank fusion for combining results
+- `reranking.py`: Cross-encoder reranking
+
+### API Layer (hindsight-api-slim/hindsight_api/api/)
+- `http.py`: FastAPI HTTP routers for all REST endpoints
+- `mcp.py`: Model Context Protocol server implementation
+
+Main operations:
+- **Retain**: Store memories, extracts facts/entities/relationships
+- **Recall**: Retrieve memories via 4 parallel strategies (semantic, BM25, graph, temporal) + reranking
+- **Reflect**: Disposition-aware reasoning using memories and mental models.
+
+### Database
+PostgreSQL with pgvector. Schema managed via Alembic migrations in `hindsight-api-slim/hindsight_api/alembic/`. Migrations run automatically on API startup.
+
+Key tables: `banks`, `memory_units`, `documents`, `entities`, `entity_links`
+
+### Adding Database Migrations
+
+Hindsight runs the same Alembic tree against PostgreSQL and Oracle 23ai. Each
+migration file dispatches through `run_for_dialect`, which calls either
+`_pg_upgrade` or `_oracle_upgrade` based on the live connection. A pytest lint
+(`tests/test_migration_shape.py`) fails CI if a migration omits the dispatcher.
+
+1. **Create a new migration file** in `hindsight-api-slim/hindsight_api/alembic/versions/`:
+   - File name format: `<revision_id>_<description>.py` (e.g., `f1a2b3c4d5e6_add_new_index.py`)
+   - Use a unique hex revision ID (12 chars)
+   - Set `down_revision` to the previous migration's revision ID
+
+2. **Migration template** (the `script.py.mako` template scaffolds this; fill in the bodies):
+   ```python
+   """Description of the migration
+
+   Revision ID: f1a2b3c4d5e6
+   Revises: <previous_revision_id>
+   Create Date: YYYY-MM-DD
+   """
+   from collections.abc import Sequence
+   from alembic import context, op
+
+   from hindsight_api.alembic._dialect import run_for_dialect
+
+   revision: str = "f1a2b3c4d5e6"
+   down_revision: str | Sequence[str] | None = "<previous_revision_id>"
+   branch_labels: str | Sequence[str] | None = None
+   depends_on: str | Sequence[str] | None = None
+
+
+   def _pg_schema_prefix() -> str:
+       """Schema-qualifier for raw SQL on PG (multi-tenant search_path)."""
+       schema = context.config.get_main_option("target_schema")
+       return f'"{schema}".' if schema else ""
+
+
+   def _pg_upgrade() -> None:
+       schema = _pg_schema_prefix()
+       op.execute(f"CREATE INDEX ... ON {schema}table_name(...)")
+
+
+   def _pg_downgrade() -> None:
+       schema = _pg_schema_prefix()
+       op.execute(f"DROP INDEX IF EXISTS {schema}index_name")
+
+
+   def _oracle_upgrade() -> None:
+       # Oracle 23ai equivalent. Use op.get_bind().exec_driver_sql for forms
+       # that Alembic core does not model (vector/text indexes, partitions).
+       op.execute("CREATE INDEX ... ON table_name(...)")
+
+
+   def _oracle_downgrade() -> None:
+       op.execute("DROP INDEX IF EXISTS index_name")
+
+
+   def upgrade() -> None:
+       run_for_dialect(pg=_pg_upgrade, oracle=_oracle_upgrade)
+
+
+   def downgrade() -> None:
+       run_for_dialect(pg=_pg_downgrade, oracle=_oracle_downgrade)
+   ```
+
+   **Dialect-only migrations.** If a change genuinely doesn't apply to one
+   dialect (e.g. enabling `pg_trgm` is PG-only), omit the unused slot:
+   ```python
+   def upgrade() -> None:
+       run_for_dialect(pg=_pg_upgrade)  # oracle slot intentionally absent → no-op
+   ```
+   Make the asymmetry deliberate. Don't leave an Oracle slot empty just because
+   you didn't think about it — copy-pasting a PG migration without the Oracle
+   half is exactly how schemas drift.
+
+3. **Run migrations locally**:
+   ```bash
+   # Set database URL and run migrations for the base schema plus all tenants
+   uv run hindsight-admin run-db-migration
+
+   # Run on a specific tenant schema
+   uv run hindsight-admin run-db-migration --schema tenant_xyz
+   ```
+
+## Key Conventions
+
+### Code Quality
+
+**Before writing code, read `.claude/skills/code-review/SKILL.md`** for the full coding standards (Python style, type safety, TypeScript style, general principles).
+
+**Always run the lint script after making Python or TypeScript/Node changes:**
+```bash
+./scripts/hooks/lint.sh
+```
+
+Dead-code detection runs in CI (the `check-unused-code` job) at two levels:
+- **Blocking:** unused imports (ruff `F401`) and variables (`F841`) — `lint.sh` auto-removes
+  them and `verify-generated-files` fails on any leftover diff; and **knip** for orphaned
+  control-plane files / unused (or unlisted) `package.json` dependencies.
+- **Advisory:** whole unused Python functions (vulture) and unused control-plane *exports*
+  (the shadcn/ui surface is kept on purpose) — surfaced, not gated.
+
+Run both locally with:
+```bash
+./scripts/hooks/check-unused.sh
+```
+
+**After completing any implementation work, run `/code-review`** to verify your changes against project standards (missing tests, dead code, type safety, etc.). Fix any "must fix" issues before considering the task done.
+
+**MANDATORY: Run `/code-review` before pushing code or creating a pull request.** Do not push or create a PR until all "must fix" issues are resolved.
+
+### Testing
+
+Most tests are deterministic (MockLLM, pure functions) — assert directly.
+
+**Tests that verify LLM behaviour use a real LLM + an LLM-as-judge.** When the thing under test is *how the model interprets a prompt* (classification, attribution, dimension preservation, instruction-following), MockLLM can't simulate it and exact string/enum asserts flake across providers and runs. Use this pattern instead:
+
+1. Mark the test module `pytestmark = pytest.mark.hs_llm_core` (single-provider; CI runs it in the core-LLM job). Use `hs_llm_mat` only for provider-matrix acceptance tests.
+2. Call the real pipeline (`LLMConfig.from_env()`, `_get_raw_config()`), e.g. `extract_facts_from_text(...)`.
+3. Assert with the judge, not string matching:
+   ```python
+   from tests.llm_judge import assert_meets_criteria
+   facts_summary = "\n".join(f"- [{f.fact_type}] {f.fact}" for f in facts)
+   await assert_meets_criteria(
+       response=facts_summary,
+       criteria="The first-person user statements are classified 'world' and attributed to the user, not the agent.",
+       context="What the input said and who was speaking.",
+   )
+   ```
+
+Rules of thumb:
+- **Judge anything non-deterministic** — including `fact_type` classification and speaker attribution. Do NOT hard-assert `fact_type == "..."`; pass a `[fact_type] fact` summary to the judge instead. Structural facts that ARE deterministic (counts, presence of a field, that a substring was injected into a prompt) stay as direct asserts in fast unit tests.
+- **Split the test surface**: cover the deterministic mechanics (prompt assembly, suppression logic) with fast non-LLM unit tests, and the model-following behaviour with one `hs_llm_core` judge test. (Example pair: `test_narrator_resolution.py` + `test_narrator_context_override.py`.)
+- The judge model is independent of the test provider (defaults to Gemini); never judge with the same call you're testing.
+
+### Memory Banks
+- Each bank is an isolated memory store (like a "brain" for one user/agent)
+- Banks have dispositions (skepticism, literalism, empathy traits 1-5) affecting reflect
+- Banks can have background context
+- Bank isolation is strict - no cross-bank data leakage
+
+### API Design
+- All endpoints operate on a single bank per request
+- Multi-bank queries are client responsibility to orchestrate
+- Disposition traits only affect reflect, not recall
+
+### Control Plane API Routes
+
+When adding or modifying parameters in the dataplane API (hindsight-api), you must also update the control plane routes that proxy to it:
+
+1. **API Routes** (`hindsight-control-plane/src/app/api/`):
+   - `recall/route.ts` - proxies to `/v1/default/banks/{bank_id}/memories/recall`
+   - `reflect/route.ts` - proxies to `/v1/default/banks/{bank_id}/reflect`
+   - `memories/retain/route.ts` - proxies to `/v1/default/banks/{bank_id}/memories/retain`
+   - Other routes follow the same pattern
+
+2. **Client types** (`hindsight-control-plane/src/lib/api.ts`):
+   - Update the TypeScript type definitions for `recall()`, `reflect()`, `retain()` etc.
+
+3. **Checklist when adding new API parameters**:
+   - Add parameter extraction in the route handler (destructure from `body`)
+   - Pass the parameter to the SDK call
+   - Update the client type definition in `lib/api.ts`
+   - Update any UI components that need to use the new parameter
+
+### Harness Attribution (which coding agent wrote a document)
+
+`hindsight-integrations/hindsight-coding-agents/` stamps the coding agent on every
+document it retains, so the control plane can show its logo instead of another
+`key=value` chip:
+
+- `metadata.harness = "<id>"` — the authoritative field
+- tag `harness:<id>` — the same value, so the documents list can filter on it
+
+The ids are defined by that integration's HookSpecs
+(`src/harness/hook-lifecycle.ts`) plus the persistent-plugin entrypoints
+registered in `src/harness/registry.ts`, whose id is their
+`createPluginEntry(...)` argument — currently `antigravity-cli`, `claude-code`,
+`cline-cli`, `codex`, `copilot-cli`, `cursor-cli`, `devin-cli`, `grok-build`,
+`kilo`, `opencode`.
+
+The control plane resolves the value in
+`hindsight-control-plane/src/lib/harness-logo.ts` (metadata wins over the tag) and
+renders it with `components/ui/harness-logo.tsx` in the documents table and the
+document detail dialog. **Adding a harness to the integration means adding it to
+that registry in the same change**: copy its icon from
+`hindsight-docs/static/img/icons/` (or take it from the agent's own brand assets
+when the docs site carries none) into
+`hindsight-control-plane/public/img/harness/` and add one entry. Don't register
+ids nothing writes — a test asserts the registry matches the emitted set, plus an
+explicit list of retired ids kept so already-retained documents keep their logo.
+An unregistered harness is not an error: it renders no logo and still shows as
+ordinary metadata.
+
+### Adding New Integrations
+
+Every new integration in `hindsight-integrations/` must satisfy all of the following before it can be merged:
+
+1. **Tests are required** — tests must simulate or exercise the external system (mock the framework's interfaces and verify the integration actually calls Hindsight correctly). Pure unit tests of helper functions are not sufficient.
+2. **CI job** — add a test job in `.github/workflows/test.yml` following the existing pattern (e.g., `test-crewai-integration`). The job must build, install deps, and run `uv run pytest tests -v`. Also add the integration to `detect-changes` outputs so it only runs when its files change.
+3. **Release process** — add the integration name to the `VALID_INTEGRATIONS` array in `scripts/release-integration.sh` so it can be released via the standard release workflow.
+4. **Follow project code standards** — Python style, type safety, no raw dicts for structured data, no multi-item tuple returns (see `.claude/skills/code-review/SKILL.md`).
+
+If any of these are missing, the integration is incomplete and must not be pushed or merged.
+
+### Changelogs
+
+Never add "Unreleased" entries to changelogs (e.g. `hindsight-docs/src/pages/changelog/**`). Changelog entries are written by the release script (`./scripts/release-integration.sh`) when a version is actually cut. If a bug fix or feature needs documenting before release, describe it in the PR/commit — the release tooling will surface it in the published changelog section.
+
+### Adding New API Configuration Flags
+
+Configuration follows a hierarchical system: **Global (env vars) → Tenant (via extension) → Bank (database)**.
+
+Fields must be categorized as either **hierarchical** (can be overridden per-tenant/bank) or **static** (server-level only).
+
+#### Adding a New Configuration Field
+
+1. **config.py** (`hindsight-api-slim/hindsight_api/config.py`):
+   - Add `ENV_*` constant for the environment variable name (e.g., `ENV_MY_SETTING = "HINDSIGHT_API_MY_SETTING"`)
+   - Add `DEFAULT_*` constant for the default value
+   - Add field to `HindsightConfig` dataclass with type annotation
+   - **Mark as configurable** by adding to `_CONFIGURABLE_FIELDS` set if the field should be overridable per-tenant/bank via API
+   - Add initialization in `from_env()` method
+
+   ```python
+   # Configurable field (can be overridden per-tenant/bank via API)
+   _CONFIGURABLE_FIELDS = {
+       ...,
+       "my_setting",  # Add here for configurable
+   }
+
+   # Static field - just don't add to _CONFIGURABLE_FIELDS
+   ```
+
+2. **main.py** (`hindsight-api-slim/hindsight_api/main.py`):
+   - No change is needed for ordinary environment-backed config fields. The CLI starts from `_get_raw_config()`,
+     so new `HindsightConfig` fields are carried through automatically.
+   - If the new field should be overridable by a CLI flag, add the argparse option in `_parse_cli_args()` and include
+     that field in the `dataclasses.replace(config, ...)` call near the "CLI override" comment.
+
+3. **Use hierarchical config in MemoryEngine**:
+   ```python
+   # Config is resolved automatically per bank via ConfigResolver
+   config_dict = await self._config_resolver.get_bank_config(bank_id, context)
+   value = config_dict["my_setting"]
+   ```
+
+4. **Use static config** (non-hierarchical):
+   ```python
+   from ...config import get_config
+   config = get_config()
+   value = config.my_static_field
+   ```
+
+5. **Documentation** (`hindsight-docs/docs/developer/configuration.md`):
+   - Add to appropriate section table with Variable, Description, Default
+   - Mark if it's hierarchical (can be overridden per-bank)
+
+6. **Env template** (`.env.example`):
+   - Add the variable to the appropriate section, commented if optional, with a
+     short inline comment describing it (mirror the documentation entry).
+   - This file is the single source of truth for the env template:
+     `scripts/dev/setup.sh` copies it to `.env`, and `hindsight-embed` ships a
+     bundled copy (`hindsight-embed/hindsight_embed/env.example`) that seeds
+     embed/profile configs. After editing `.env.example`, re-copy it to the
+     embed package (`cp .env.example hindsight-embed/hindsight_embed/env.example`)
+     or the `test_bundled_template_matches_repo_root` sync test will fail.
+
+#### Hierarchical vs Static Guidelines
+
+**Hierarchical** (per-bank overridable):
+- LLM settings (provider, model, API key, base URL)
+- Operation-specific settings (retain mode, chunk size, etc.)
+- Feature flags that vary by customer/bank
+
+**Static** (server-level only):
+- Infrastructure settings (database URL, port, host)
+- Global limits (max concurrent operations)
+- System-wide feature flags
+
+## Environment Setup
+
+```bash
+cp .env.example .env
+# Edit .env with the LLM provider/model and credentials for your setup
+
+# Python deps
+uv sync --directory hindsight-api-slim/
+
+# Node deps (uses npm workspaces)
+npm install
+```
+
+Common LLM settings:
+- `HINDSIGHT_API_LLM_PROVIDER`: openai, anthropic, gemini, groq, minimax, ollama, lmstudio
+- `HINDSIGHT_API_LLM_API_KEY`: API key for providers that require one
+- `HINDSIGHT_API_LLM_MODEL`: Model name (defaults are provider-specific)
+
+Optional (uses local models by default):
+- `HINDSIGHT_API_EMBEDDINGS_PROVIDER`: local (default) or tei
+- `HINDSIGHT_API_RERANKER_PROVIDER`: local (default) or tei
+- `HINDSIGHT_API_DATABASE_URL`: External PostgreSQL (uses embedded pg0 by default)
+- `HINDSIGHT_API_ENABLE_BANK_CONFIG_API`: Enable per-bank config API (default: true)

CONTRIBUTING.md

@@ -5,13 +5,46 @@ Thanks for your interest in contributing to Hindsight!
 ## Getting Started
 
 1. Fork and clone the repository
+   ```bash
+   git clone [email protected]:vectorize-io/hindsight.git
+   cd hindsight
+   ```
+
+2. Bootstrap your dev environment in one shot:
+   ```bash
+   ./scripts/dev/setup.sh
+   ```
+   This is idempotent (safe to re-run) and gets you ready to develop, including
+   offline. It:
+   - installs the required toolchains if missing (uv/Python, Node/npm, Rust/cargo),
+   - creates `.env` from `.env.example` (remember to add your LLM API key),
+   - configures git hooks,
+   - installs all Python and Node workspace dependencies,
+   - pre-downloads the local ML models + tokenizer so the API runs offline,
+   - builds the TypeScript SDK and the Rust CLI.
+
+   Useful flags: `--skip-build` (deps only), `--skip-models` (skip ML model
+   download), `--with-docs` (also build the docs site), `--force` (rebuild
+   artifacts). Docker image builds are out of scope. Run
+   `./scripts/dev/setup.sh --help` for details.
+
+### Manual setup
+
+If you'd rather set things up by hand instead of running the script above:
+
+1. Set up your environment:
+   ```bash
+   cp .env.example .env
+   ```
+   Edit the .env to add LLM API key and config as required
+
 2. Install dependencies:
    ```bash
-   cd hindsight-api && uv sync
-   ```
-3. Set up your environment:
-   ```bash
-   export OPENAI_API_KEY=your-key
+   # Python dependencies
+   uv sync --directory hindsight-api/
+
+   # Node dependencies (uses npm workspaces)
+   npm install
    ```
 
 ## Development
@@ -41,7 +74,36 @@ cd hindsight-api
 uv run pytest tests/
 ```
 
-### Code style
+### Code Style
+
+We use [Ruff](https://docs.astral.sh/ruff/) for Python linting and formatting, and ESLint/Prettier for TypeScript.
+
+#### Setting up git hooks (recommended)
+
+Set up git hooks to automatically lint and format code before each commit:
+
+```bash
+./scripts/setup-hooks.sh
+```
+
+This configures git to use the hooks in `.githooks/`, which run all scripts in `scripts/hooks/` on commit. The lint hook runs in parallel:
+- **Python**: `ruff check --fix`, `ruff format`, `ty check`
+- **TypeScript**: `eslint --fix`, `prettier`
+
+#### Manual linting and formatting
+
+```bash
+# Run all lints (same as pre-commit)
+./scripts/hooks/lint.sh
+
+# Or run individually for Python:
+cd hindsight-api
+uv run ruff check --fix .   # Lint and auto-fix
+uv run ruff format .        # Format code
+uv run ty check hindsight_api  # Type check
+```
+
+#### Style guidelines
 
 - Use Python type hints
 - Follow existing code patterns
@@ -54,6 +116,34 @@ uv run pytest tests/
 3. Run tests to ensure nothing breaks
 4. Submit a PR with a clear description of changes
 
+## Release Process
+
+The project uses `scripts/release.sh` for creating releases. This script automates the entire release workflow:
+
+1. Bumps version in all components (API, clients, CLI, control plane, Helm)
+2. **Regenerates OpenAPI spec and client SDKs** (Python, TypeScript, Rust)
+3. Updates documentation versioning
+4. Creates a commit and git tag
+5. Pushes to GitHub (triggers CI/CD to publish packages)
+
+### Usage
+
+```bash
+./scripts/release.sh <version>
+```
+
+**Example:**
+```bash
+./scripts/release.sh 0.5.0
+```
+
+### Important for Developers
+
+- During development, version bumps in `__init__.py` do NOT require client regeneration
+- Clients are only regenerated during releases
+- Do not manually run `./scripts/generate-clients.sh` unless testing generation changes
+- Client version comments will reflect the API version from the latest release
+
 ## Reporting Issues
 
 Open an issue on GitHub with:

README.md

@@ -1,70 +1,95 @@
 <div align="center">
 
-![Hindsight Banner](./hindsight-docs/static/img/banner.webp)
+![Hindsight Banner](./hindsight-docs/static/img/hindsight-github-banner.png)
 
-[Documentation](https://vectorize-io.github.io/hindsight) • [Paper](#coming-soon) • [Examples](https://github.com/vectorize-io/hindsight-cookbook)
+[Documentation](https://hindsight.vectorize.io) • [Paper](https://arxiv.org/abs/2512.12818) • [Cookbook](https://hindsight.vectorize.io/cookbook) • [Hindsight Cloud](https://ui.hindsight.vectorize.io/signup)
 
 [![CI](https://github.com/vectorize-io/hindsight/actions/workflows/release.yml/badge.svg)](https://github.com/vectorize-io/hindsight/actions/workflows/release.yml)
+[![Slack Community](https://img.shields.io/badge/Slack-Join%20Community-4A154B?logo=slack)](https://join.slack.com/t/hindsight-space/shared_invite/zt-3nhbm4w29-LeSJ5Ixi6j8PdiYOCPlOgg)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
-[![PyPI - hindsight-api](https://img.shields.io/pypi/v/hindsight-api?label=hindsight-api)](https://pypi.org/project/hindsight-api/)
-[![PyPI - hindsight-client](https://img.shields.io/pypi/v/hindsight-client?label=hindsight-client)](https://pypi.org/project/hindsight-client/)
-[![npm - @vectorize-io/hindsight-client](https://img.shields.io/npm/v/@vectorize-io/hindsight-client)](https://www.npmjs.com/package/@vectorize-io/hindsight-client)
-[![Slack Community](https://img.shields.io/badge/Slack-Join%20Community-4A154B?logo=slack)](https://join.slack.com/t/hindsight-space/shared_invite/zt-3klo21kua-VUCC_zHP5rIcXFB1_5yw6A)
-
+![PyPI - Downloads](https://img.shields.io/pypi/dm/hindsight-api?label=PyPI)
+![NPM Downloads](https://img.shields.io/npm/dm/%40vectorize-io%2Fhindsight-client?logoColor=orange&label=NPM&color=blue&link=https%3A%2F%2Fwww.npmjs.com%2Fpackage%2F%40vectorize-io%2Fhindsight-client)
+<br/>
 
+<a href="https://trendshift.io/repositories/15603" target="_blank"><img src="https://trendshift.io/api/badge/repositories/15603" alt="vectorize-io%2Fhindsight | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
 </div>
 
 ---
 
 ## What is Hindsight?
 
-Hindsight is an agent memory system built to create smarter agents that learn over time. It eliminates the shortcomings of alternative techniques such as RAG and knowledge graph.
+Hindsight™ is an agent memory system built to create smarter agents that learn over time. Most agent memory systems focus on recalling conversation history. Hindsight is focused on making agents that learn, not just remember.
 
-Hindsight addresses common challenges that have frustrated AI engineers building agents to automate tasks and assist users with conversational interfaces. Many of these challenges stem directly from a lack of memory.
 
-- **Inconsistency:** Agents complete tasks successfully one time, then fail when asked to complete the same task again. Memory gives the agent a mechanism to remember what worked and what didn't and to use that information to reduce errors and improve consistency.
-- **Hallucinations:** Long term memory can be seeded with external knowledge to ground agent behavior in reliable sources to augment training data.
-- **Cognitive Overload:** As workflows get complex, retrievals, tool calls, user messages and agent responses can grow to fill the context window leading to context rot. Short term memory optimization allows agents to reduce tokens and focus context by removing irrelevant details.
+<video src="https://github.com/user-attachments/assets/923b798d-3581-4897-bb62-9cfa5a931682" controls></video>
 
-## How Hindsight Works
+It eliminates the shortcomings of alternative techniques such as RAG and knowledge graph and delivers state-of-the-art performance on long term memory tasks.
 
-![Overview](./hindsight-docs/static/img/hindsight-overview.webp)
+## Memory Performance & Accuracy
 
-Hindsight organizes memory into four networks to mimic the way human memory works:
+Hindsight is the most accurate agent memory system ever tested according to benchmark performance. It has achieved state-of-the-art performance on the LongMemEval benchmark, widely used to assess memory system performance across a variety of conversational AI scenarios. The current reported performance of Hindsight and other agent memory solutions as of January 2026 is shown here:
 
-- **World:** Facts about the world ("The stove gets hot")
-- **Experiences:** Agent's own experiences ("I touched the stove and it really hurt")
-- **Opinion:** Beliefs with confidence scores ("I shouldn't touch the stove again" - .99 confidence)
-- **Observation:** Complex mental models derived by reflecting on facts and experiences ("Curling irons, ovens, and fire are also hot. I shouldn't touch those either.")
+![Overview](./hindsight-docs/static/img/hindsight-benchmarks.png)
 
-Hindsight provides three simple methods to interact with the system:
+The benchmark performance data for Hindsight has been independently reproduced by research collaborators at the Virginia Tech [Sanghani Center for Artificial Intelligence and Data Analytics](https://sanghani.cs.vt.edu/) and The Washington Post. Other scores are self-reported by software vendors.
 
-- **Retain:** Provide information to Hindsight that you want it to remember
-- **Recall:** Retrieve memories from Hindsight
-- **Reflect:** Reflect on memories and experiences to generate new observations and insights from existing memories.
+Hindsight is being used in production at Fortune 500 enterprises and by a growing number of AI startups. 
 
-Memories in Hindsight are stored in banks (e.g. memory banks). When memories are retained, they are transformed to construct a series of search indexes, time series data, and entity/relationship graphs.
+## Adding Hindsight to Your AI Agents
+
+The easiest way to use Hindsight with an existing agent is with the LLM Wrapper. You can add memory to your agent with 2 lines of code. That will swap your current LLM client out with the Hindsight wrapper. After that, memories will be stored and retrieved automatically as you make LLM calls.
+
+If you need more control over how and when your agent stores and recalls memories, there's also a simple API you can integrate with using the SDKs or directly via HTTP.
+
+![Hindsight Banner](./hindsight-docs/static/img/migration-code.png)
 
 ---
 
+> 🤖 **Using a coding agent?** Install the Hindsight documentation skill for instant access to docs while you code:
+> ```bash
+> npx skills add https://github.com/vectorize-io/hindsight --skill hindsight-docs
+> ```
+> Works with Claude Code, Cursor, and other AI coding assistants.
+
+---
+
+
 ## Quick Start
 
 ### Docker (recommended)
 
 ```bash
-export OPENAI_API_KEY=your-key
+export OPENAI_API_KEY=sk-xxx
 
-docker run --rm -it --pull always -p 8888:8888 -p 9999:9999 \
+docker run -it --pull always --name hindsight --restart unless-stopped -p 8888:8888 -p 9999:9999 \
   -e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \
-  -e HINDSIGHT_API_LLM_MODEL=o3-mini \
-  -v $HOME/.hindsight-docker:/home/hindsight/.pg0 \
+  -v hindsight-data:/home/hindsight/.pg0 \
   ghcr.io/vectorize-io/hindsight:latest
 ```
 
-API: http://localhost:8888  
-UI: http://localhost:9999
+>API: http://localhost:8888
+>UI: http://localhost:9999
 
-Install client:
+You can modify the LLM provider by setting `HINDSIGHT_API_LLM_PROVIDER`. Valid options are `openai`, `anthropic`, `gemini`, `groq`, `ollama`, `lmstudio`, `minimax`, and `atlas` ([Atlas Cloud](https://www.atlascloud.ai/?utm_source=github&utm_medium=link&utm_campaign=hindsight)). The documentation provides more details on [supported models](https://hindsight.vectorize.io/developer/models).
+
+
+
+### Docker (external PostgreSQL)
+
+```bash
+export OPENAI_API_KEY=sk-xxx
+export HINDSIGHT_DB_PASSWORD=choose-a-password
+cd docker/docker-compose
+docker compose up 
+```
+
+> Oracle AI Database is also supported for enterprise deployments with full feature parity. See the [storage documentation](https://hindsight.vectorize.io/developer/storage) for details.
+
+
+>API: http://localhost:8888
+>UI: http://localhost:9999
+
+### Client
 
 ```bash
 pip install hindsight-client -U
@@ -72,7 +97,7 @@ pip install hindsight-client -U
 npm install @vectorize-io/hindsight-client
 ```
 
-Python example:
+#### Python
 
 ```python
 from hindsight_client import Hindsight
@@ -89,12 +114,36 @@ client.recall(bank_id="my-bank", query="What does Alice do?")
 client.reflect(bank_id="my-bank", query="Tell me about Alice")
 ```
 
-### Python (embedded, no Docker)
+#### Node.js / TypeScript
+
+```bash
+npm install @vectorize-io/hindsight-client
+```
+
+```javascript
+const { HindsightClient } = require('@vectorize-io/hindsight-client');
+
+const main = async () => {
+  const client = new HindsightClient({ baseUrl: 'http://localhost:8888' });
+
+  await client.retain('my-bank', 'Alice loves hiking in Yosemite');
+
+  const results = await client.recall('my-bank', 'What does Alice like?');
+  console.log(results);
+}
+
+main();
+```
+
+
+### Python Embedded (no server required)
 
 ```bash
 pip install hindsight-all -U
 ```
 
+On Intel (x86_64) Macs, install `hindsight-all-slim` instead — see [Supported Platforms](#supported-platforms).
+
 ```python
 import os
 from hindsight import HindsightServer, HindsightClient
@@ -109,25 +158,48 @@ with HindsightServer(
     results = client.recall(bank_id="my-bank", query="Where does Alice work?")
 ```
 
-### Node.js / TypeScript
 
-```bash
-npm install @vectorize-io/hindsight-client
-```
+---
 
-```javascript
-const { HindsightClient } = require('@vectorize-io/hindsight-client');
+## Use Cases
 
-const client = new HindsightClient({ baseUrl: 'http://localhost:8888' });
 
-await client.retain('my-bank', 'Alice loves hiking in Yosemite');
-await client.recall('my-bank', 'What does Alice like?');
-```
+Hindsight is built to support conversational AI agents as well as agents that are intended to perform tasks autonomously. The ideal use case for Hindsight are agents that require a blend of these features such as AI employees that need to handle open-ended tasks, change behavior based on user feedback, and learn to perform complex tasks to automate work at a level that approximates a human work. Hindsight can be used with simple AI workflows like those built with n8n and other similar tools, but may be overkill for such applications.
+
+### Per-User Memories and Chat History
+
+One of the simpler use cases you can use Hindsight for is to personalize AI chatbots and other conversational agents by storing and recalling memories associated with individual users.
+
+The requirements for this use case usually look something like this:
+
+![Per-User Memories](./hindsight-docs/static/img/per-user-memory-requirements.png)
+
+<video src="https://github.com/user-attachments/assets/4805e8e1-e7d1-47c6-a4f8-2344a5ec8906" controls></video>
+
+Satisfying these requirements in Hindsight is straightforward. When new user inputs and tool calls are ingested into Hindsight using the retain operation, custom metadata can be used to enrich the new memories. Metadata provides a convenient way to isolate memories that need to be restricted to a given user. Once these are fed into the retain operation, any raw memories and mental models that get created can be filtered when retrieving relevant memories. 
+
+![Per-User Memories](./hindsight-docs/static/img/per-user-memory-howto.png)
 
 ---
 
 ## Architecture & Operations
 
+![Overview](./hindsight-docs/static/img/hindsight-overview.webp)
+
+Most agent memory implementations rely on basic vector search or sometimes use a knowledge graph. Hindsight uses biomimetic data structures to organize agent memories in a way that is more like how human memory works:
+
+- **World:** Facts about the world ("The stove gets hot")
+- **Experiences:** Agent's own experiences ("I touched the stove and it really hurt")
+- **Mental Models:** Learned understanding of the agent's world formed by reflecting on raw memories and experiences.
+
+Memories in Hindsight are stored in banks (i.e. memory banks). When memories are added to Hindsight, they are pushed into either the world facts or experiences memory pathway. They are then represented as a combination of entities, relationships, and time series with sparse/dense vector representations to aid in later recall.
+
+Hindsight provides three simple methods to interact with the system:
+
+- **Retain:** Provide information to Hindsight that you want it to remember
+- **Recall:** Retrieve memories from Hindsight
+- **Reflect:** Reflect on memories and experiences to generate new observations and insights from existing memories.
+
 ### Retain
 
 The `retain` operation is used to push new memories into Hindsight. It tells Hindsight to _retain_ the information you pass in as an input.
@@ -178,7 +250,7 @@ Recall performs 4 retrieval strategies in parallel:
 - Graph: Entity/temporal/causal links
 - Temporal: Time range filtering
 
-![Retain Operation](hindsight-docs/static/img/recall-operation.webp)
+![Recall Operation](hindsight-docs/static/img/recall-operation.webp)
 
 The individual results from the retrievals are merged, then ordered by relevance using reciprocal rank fusion and a cross-encoder reranking model.
 
@@ -186,7 +258,7 @@ The final output is trimmed as needed to fit within the token limit.
 
 ### Reflect
 
-The reflect operation is used to perform a more thorough analysis of existing memories. This allows the agent to form new connections between memories which are then persisted as opinions and/or observations. When building agents, the reflect operation is a key capability to enable the agent to learn from its experiences. 
+The reflect operation is used to perform a more thorough analysis of existing memories. This allows the agent to form new connections between memories and build a more thorough understanding of its world.
 
 For example, the `reflect` operation can be used to support use cases such as:
 
@@ -204,7 +276,7 @@ client = Hindsight(base_url="http://localhost:8888")
 client.reflect(bank_id="my-bank", query="What should I know about Alice?")
 ```
 
-![Retain Operation](hindsight-docs/static/img/reflect-operation.webp)
+![Reflect Operation](hindsight-docs/static/img/reflect-operation.webp)
 
 ---
 
@@ -220,9 +292,26 @@ client.reflect(bank_id="my-bank", query="What should I know about Alice?")
 - [CLI](https://hindsight.vectorize.io/sdks/cli)
 
 **Community:**
-- [Slack](https://join.slack.com/t/hindsight-space/shared_invite/zt-3klo21kua-VUCC_zHP5rIcXFB1_5yw6A)
+- [Slack](https://join.slack.com/t/hindsight-space/shared_invite/zt-3nhbm4w29-LeSJ5Ixi6j8PdiYOCPlOgg)
 - [GitHub Issues](https://github.com/vectorize-io/hindsight/issues)
 
+---
+## Star History
+
+[![Star history](https://raw.githubusercontent.com/vectorize-io/hindsight/main/.github/star-history/chart.svg)](https://github.com/vectorize-io/hindsight/stargazers)
+---
+
+## Supported Platforms
+
+| Platform | Docker | Bare Metal (pip) | Embedded DB (pg0) |
+|----------|--------|------------------|--------------------|
+| **Linux** (x86_64, ARM64) | ✅ | ✅ | ✅ |
+| **macOS** (Apple Silicon / arm64) | ✅ | ✅ | ✅ |
+| **macOS** (Intel / x86_64) | ✅ | ⚠️ | ✅ |
+| **Windows** (x86_64) | ✅ | ✅ | ✅ |
+
+⚠️ Intel Macs: use `hindsight-all-slim` — see the [installation guide](https://hindsight.vectorize.io/developer/installation#supported-platforms) for details.
+
 ---
 
 ## Contributing
@@ -236,3 +325,5 @@ MIT — see [LICENSE](./LICENSE)
 ---
 
 Built by [Vectorize.io](https://vectorize.io)
+
+<img src="https://umami-pixel.chris-latimer.workers.dev/?id=a8b043e6-6964-454d-80df-69b69d3f0d50&host=github.com&url=/vectorize-io/hindsight" width="1" height="1" alt="" />

deno.lock

@@ -0,0 +1,170 @@
+{
+  "version": "5",
+  "specifiers": {
+    "jsr:@std/assert@^1.0.17": "1.0.19",
+    "jsr:@std/assert@^1.0.19": "1.0.19",
+    "jsr:@std/expect@*": "1.0.18",
+    "jsr:@std/internal@^1.0.12": "1.0.12",
+    "jsr:@std/path@^1.1.4": "1.1.4",
+    "jsr:@std/testing@*": "1.0.17"
+  },
+  "jsr": {
+    "@std/[email protected]": {
+      "integrity": "eaada96ee120cb980bc47e040f82814d786fe8162ecc53c91d8df60b8755991e",
+      "dependencies": [
+        "jsr:@std/internal"
+      ]
+    },
+    "@std/[email protected]": {
+      "integrity": "8566eab35200466f8609eb7e7aed062ed0db314e9a258d5d201b1b8997ce801a",
+      "dependencies": [
+        "jsr:@std/assert@^1.0.19",
+        "jsr:@std/internal",
+        "jsr:@std/path"
+      ]
+    },
+    "@std/[email protected]": {
+      "integrity": "972a634fd5bc34b242024402972cd5143eac68d8dffaca5eaa4dba30ce17b027"
+    },
+    "@std/[email protected]": {
+      "integrity": "1d2d43f39efb1b42f0b1882a25486647cb851481862dc7313390b2bb044314b5",
+      "dependencies": [
+        "jsr:@std/internal"
+      ]
+    },
+    "@std/[email protected]": {
+      "integrity": "87bdc2700fa98249d48a17cd72413352d3d3680dcfbdb64947fd0982d6bbf681",
+      "dependencies": [
+        "jsr:@std/assert@^1.0.17",
+        "jsr:@std/internal"
+      ]
+    }
+  },
+  "workspace": {
+    "members": {
+      "hindsight-all-npm": {
+        "packageJson": {
+          "dependencies": [
+            "npm:@types/node@22",
+            "npm:tsup@^8.5.1",
+            "npm:typescript@^5.7.0",
+            "npm:vitest@^4.1.2"
+          ]
+        }
+      },
+      "hindsight-clients/typescript": {
+        "packageJson": {
+          "dependencies": [
+            "npm:@hey-api/[email protected]",
+            "npm:@types/jest@29",
+            "npm:@types/node@20",
+            "npm:jest@29",
+            "npm:ts-jest@29",
+            "npm:tsup@^8.5.1",
+            "npm:typescript@5"
+          ]
+        }
+      },
+      "hindsight-control-plane": {
+        "packageJson": {
+          "dependencies": [
+            "npm:@chenglou/pretext@^0.0.3",
+            "npm:@eslint/eslintrc@^3.3.3",
+            "npm:@eslint/js@^9.39.2",
+            "npm:@radix-ui/react-alert-dialog@^1.1.15",
+            "npm:@radix-ui/react-checkbox@^1.3.3",
+            "npm:@radix-ui/react-dialog@^1.1.15",
+            "npm:@radix-ui/react-dropdown-menu@^2.1.16",
+            "npm:@radix-ui/react-label@^2.1.8",
+            "npm:@radix-ui/react-popover@^1.1.15",
+            "npm:@radix-ui/react-select@^2.2.6",
+            "npm:@radix-ui/react-slider@^1.3.6",
+            "npm:@radix-ui/react-slot@^1.2.4",
+            "npm:@radix-ui/react-switch@^1.2.6",
+            "npm:@radix-ui/react-tabs@^1.1.13",
+            "npm:@radix-ui/react-tooltip@^1.2.8",
+            "npm:@tailwindcss/postcss@^4.1.17",
+            "npm:@tailwindcss/typography@~0.5.19",
+            "npm:@types/cytoscape@^3.21.9",
+            "npm:@types/node@^24.10.0",
+            "npm:@types/react-dom@^19.2.2",
+            "npm:@types/react@^19.2.2",
+            "npm:autoprefixer@^10.4.21",
+            "npm:class-variance-authority@~0.7.1",
+            "npm:clsx@^2.1.1",
+            "npm:cmdk@^1.1.1",
+            "npm:cytoscape-fcose@^2.2.0",
+            "npm:cytoscape@^3.33.1",
+            "npm:eslint-config-next@^16.0.1",
+            "npm:eslint-plugin-react-hooks@^7.0.1",
+            "npm:eslint-plugin-react@^7.37.5",
+            "npm:eslint@^9.39.1",
+            "npm:[email protected]",
+            "npm:next-themes@~0.4.6",
+            "npm:next@^16.1.7",
+            "npm:postcss@^8.5.6",
+            "npm:prettier@^3.7.4",
+            "npm:react-chrono@^2.9.1",
+            "npm:react-dom@^19.2.0",
+            "npm:react-is@^19.2.4",
+            "npm:react-markdown@^10.1.0",
+            "npm:react18-json-view@~0.2.9",
+            "npm:react@^19.2.0",
+            "npm:recharts@^3.5.1",
+            "npm:remark-gfm@^4.0.1",
+            "npm:sonner@^2.0.7",
+            "npm:tailwind-merge@^3.4.0",
+            "npm:tailwindcss-animate@^1.0.7",
+            "npm:tailwindcss@^4.1.17",
+            "npm:[email protected]",
+            "npm:typescript-eslint@^8.50.0",
+            "npm:typescript@^5.9.3"
+          ]
+        }
+      },
+      "hindsight-docs": {
+        "packageJson": {
+          "dependencies": [
+            "npm:@docusaurus/[email protected]",
+            "npm:@docusaurus/[email protected]",
+            "npm:@docusaurus/[email protected]",
+            "npm:@docusaurus/theme-common@^3.9.2",
+            "npm:@docusaurus/theme-mermaid@^3.9.2",
+            "npm:@docusaurus/[email protected]",
+            "npm:@docusaurus/[email protected]",
+            "npm:@easyops-cn/docusaurus-search-local@~0.52.2",
+            "npm:@mdx-js/react@3",
+            "npm:clsx@2",
+            "npm:prism-react-renderer@^2.3.0",
+            "npm:raw-loader@^4.0.2",
+            "npm:react-dom@19",
+            "npm:react-icons@^5.6.0",
+            "npm:react@19",
+            "npm:redocusaurus@^2.5.0",
+            "npm:typescript@~5.6.2"
+          ]
+        }
+      },
+      "hindsight-tools/hindsight-agent-sdk": {
+        "packageJson": {
+          "dependencies": [
+            "npm:@vectorize-io/hindsight-client@~0.5.6",
+            "npm:typescript@^5.4.0",
+            "npm:vitest@^4.1.2"
+          ]
+        }
+      },
+      "hindsight-tools/self-driving-agents": {
+        "packageJson": {
+          "dependencies": [
+            "npm:@clack/prompts@^1.2.0",
+            "npm:@vectorize-io/hindsight-client@~0.5.6",
+            "npm:picocolors@^1.1.0",
+            "npm:typescript@^5.4.0",
+            "npm:vitest@^4.1.2"
+          ]
+        }
+      }
+    }
+  }
+}

docker/docker-compose/alloydb/docker-compose.yaml

@@ -0,0 +1,90 @@
+name: hindsight
+# Docker Compose file for Hindsight with AlloyDB Omni and ScaNN
+# Uses Google's free AlloyDB Omni container image: https://hub.docker.com/r/google/alloydbomni
+#
+# Usage:
+# docker compose -f docker/docker-compose/alloydb/docker-compose.yaml up -d
+#
+# Make sure to set the required environment variables before running:
+# - HINDSIGHT_DB_PASSWORD: password for the AlloyDB Omni/PostgreSQL user
+# - Configure LLM provider variables as needed (see the hindsight service below)
+#
+# Optional environment variables with defaults:
+# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
+# - HINDSIGHT_DB_VERSION: AlloyDB Omni image tag (default: 17)
+# - HINDSIGHT_DB_USER: database user (default: hindsight_user)
+# - HINDSIGHT_DB_NAME: database name (default: hindsight_db)
+
+services:
+  db:
+    image: google/alloydbomni:${HINDSIGHT_DB_VERSION:-17}
+    container_name: hindsight-db-alloydb
+    restart: always
+    ports:
+      - "5438:5432"
+    environment:
+      POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
+      POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:-hindsight_password}
+      POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
+    volumes:
+      - alloydb_data:/var/lib/postgresql/data
+    networks:
+      - hindsight-net
+
+  alloydb-init:
+    image: google/alloydbomni:${HINDSIGHT_DB_VERSION:-17}
+    depends_on:
+      - db
+    environment:
+      - PGPASSWORD=${HINDSIGHT_DB_PASSWORD:-hindsight_password}
+    command:
+      - bash
+      - -c
+      - |
+        echo 'Waiting for AlloyDB Omni to be ready...'
+        until pg_isready -h hindsight-db-alloydb -p 5432 -U ${HINDSIGHT_DB_USER:-hindsight_user}; do
+          echo 'AlloyDB Omni is unavailable - sleeping'
+          sleep 2
+        done
+        echo 'AlloyDB Omni is ready - creating ${HINDSIGHT_DB_NAME:-hindsight_db} database'
+        psql -h hindsight-db-alloydb -p 5432 -U ${HINDSIGHT_DB_USER:-hindsight_user} -c 'CREATE DATABASE ${HINDSIGHT_DB_NAME:-hindsight_db};' 2>/dev/null || echo 'Database already exists'
+        echo 'Creating vector and alloydb_scann extensions'
+        psql -h hindsight-db-alloydb -p 5432 -U ${HINDSIGHT_DB_USER:-hindsight_user} -d ${HINDSIGHT_DB_NAME:-hindsight_db} -c 'CREATE EXTENSION IF NOT EXISTS vector;'
+        psql -h hindsight-db-alloydb -p 5432 -U ${HINDSIGHT_DB_USER:-hindsight_user} -d ${HINDSIGHT_DB_NAME:-hindsight_db} -c 'CREATE EXTENSION IF NOT EXISTS alloydb_scann CASCADE;'
+        echo 'Database and extensions created successfully'
+    restart: "no"
+    networks:
+      - hindsight-net
+
+  hindsight:
+    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
+    container_name: hindsight-app
+    ports:
+      - "8888:8888"
+      - "9999:9999"
+    environment:
+      # LLM Configuration
+      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
+      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
+
+      # Database Configuration
+      HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
+
+      # Vector and Text Search Extensions
+      HINDSIGHT_API_VECTOR_EXTENSION: scann
+      HINDSIGHT_API_TEXT_SEARCH_EXTENSION: native
+
+    depends_on:
+      db:
+        condition: service_started
+      alloydb-init:
+        condition: service_completed_successfully
+    networks:
+      - hindsight-net
+
+networks:
+  hindsight-net:
+    driver: bridge
+
+volumes:
+  alloydb_data:

docker/docker-compose/claude-code/README.md

@@ -0,0 +1,113 @@
+# Hindsight with Claude Code (Claude Pro/Max subscription)
+
+Run Hindsight inside Docker using the `claude-code` LLM provider, backed by
+your host machine's Claude Pro or Max subscription credentials.
+
+The standalone Hindsight Docker image ships `claude-agent-sdk` but does **not**
+bundle the host `claude` CLI binary or any Claude credentials. This Compose
+file bind-mounts the host's CLI install and credentials into the container so
+the `claude-code` provider works without an API key.
+
+## When to use this
+
+- You have an active Claude Pro or Max subscription and want to use it for
+  Hindsight without paying separate Anthropic API costs.
+- You want a one-command `docker compose up` instead of a long `docker run`
+  invocation with many flags.
+- You are running on **Linux/amd64** — macOS Docker Desktop and Windows host
+  paths differ and are not yet covered (please open an issue if you'd like to
+  contribute a verified recipe for either).
+
+> **Personal-use only.**  Anthropic's
+> [Agent SDK documentation](https://docs.claude.com/en/api/agent-sdk/overview)
+> states that third-party developers should not offer claude.ai login or rate
+> limits for their products. Hindsight does **not** perform any login on your
+> behalf — it uses credentials you've already authenticated via
+> `claude auth login`. In January 2026, Anthropic
+> [enforced restrictions](https://paddo.dev/blog/anthropic-walled-garden-crackdown/)
+> against tools that spoofed the Claude Code client identity; Hindsight uses
+> the official Claude Agent SDK instead.
+>
+> Do not deploy this configuration to shared environments or production. For
+> that, use the `anthropic` provider with an API key from the
+> [Anthropic Console](https://console.anthropic.com/). Usage counts against
+> your Claude Pro/Max subscription limits.
+
+## Prerequisites
+
+- Host has `claude` CLI installed (e.g., `npm install -g @anthropics/claude-code`)
+  and `claude auth login` has been run successfully.
+- `~/.claude.json` and `~/.claude/.credentials.json` exist on the host.
+- Host `claude` CLI version is **2.1.128 or newer** — the version bundled with
+  `claude-agent-sdk` 0.5.x has a protocol incompatibility in containers, so
+  the recipe overrides it with the host binary.
+
+## Quick start
+
+```bash
+# Set your host UID/GID (defaults to 1000:1000 if unset)
+export HOST_UID=$(id -u)
+export HOST_GID=$(id -g)
+
+docker compose -f docker/docker-compose/claude-code/docker-compose.yaml up -d
+```
+
+- API: http://localhost:8888
+- Control Plane: http://localhost:9999
+
+## Post-setup (one-time)
+
+After the container starts for the first time, run these commands to fix
+permissions and symlink the host `claude` binary into `$PATH`:
+
+```bash
+# Make ~/.claude writable by your UID (the CLI writes session/project state)
+docker exec --user 0:0 hindsight-claude-code chown $(id -u):$(id -g) /home/hindsight/.claude
+docker exec --user 0:0 hindsight-claude-code chmod 755 /home/hindsight/.claude
+
+# Symlink the host claude binary into PATH
+docker exec --user 0:0 hindsight-claude-code \
+  ln -sf /home/hindsight/.local/share/claude/versions/2.1.128 /usr/local/bin/claude
+```
+
+If you set `CLAUDE_CLI_VERSION` to a version other than `2.1.128`, update the
+symlink path accordingly.
+
+## Notes on the bind-mount surface (every flag is load-bearing)
+
+- **Host `claude` binary required** — the image ships only `claude-agent-sdk`,
+  not the CLI itself.
+- **SDK bundled-binary override** — the override of
+  `claude_agent_sdk/_bundled/claude` works around a protocol issue in the
+  bundled v2.1.121 binary inside containers. Once `claude-agent-sdk` ships
+  with v2.1.128+ this override can be dropped. Set `CLAUDE_CLI_VERSION` to
+  match your installed version.
+- **Single-file credential mounts** — credentials are mounted as individual
+  `:ro` files rather than a whole-directory `:ro` mount of `~/.claude`,
+  because the CLI writes session/project state at runtime and a read-only
+  directory mount silently breaks it.
+- **`--user` / `user:`** — the `user: ${HOST_UID}:${HOST_GID}` pattern
+  requires `chmod 755 /home/hindsight`, which is built into the image since
+  v0.6.0 (see [#1481](https://github.com/vectorize-io/hindsight/issues/1481)).
+- **`~/.hindsight-docker` data directory** — the pg0 data bind mount must be
+  writable by your host UID (see
+  [#1483](https://github.com/vectorize-io/hindsight/issues/1483)).
+- **Verified** on `linux/amd64` against `ghcr.io/vectorize-io/hindsight:latest`
+  v0.5.6+.
+
+## Using a different Claude CLI version
+
+If your host has a `claude` version other than 2.1.128, set
+`CLAUDE_CLI_VERSION` before starting:
+
+```bash
+export CLAUDE_CLI_VERSION=2.2.0
+docker compose -f docker/docker-compose/claude-code/docker-compose.yaml up -d
+```
+
+Then update the post-setup symlink to match:
+
+```bash
+docker exec --user 0:0 hindsight-claude-code \
+  ln -sf /home/hindsight/.local/share/claude/versions/2.2.0 /usr/local/bin/claude
+```

docker/docker-compose/claude-code/docker-compose.yaml

@@ -0,0 +1,44 @@
+name: hindsight-claude-code
+# Run Hindsight with the claude-code LLM provider, using your host machine's
+# Claude Pro/Max subscription credentials.  Linux/amd64 only for now.
+#
+# Quick start:
+#   docker compose -f docker/docker-compose/claude-code/docker-compose.yaml up -d
+#
+# See README.md for prerequisites, post-setup steps, and important caveats.
+
+services:
+  hindsight:
+    image: ghcr.io/vectorize-io/hindsight:latest
+    container_name: hindsight-claude-code
+    user: "${HOST_UID:-1000}:${HOST_GID:-1000}"
+    ports:
+      - "127.0.0.1:8888:8888"
+      - "127.0.0.1:9999:9999"
+    environment:
+      HOME: /home/hindsight
+      USER: hindsight
+      LOGNAME: hindsight
+      PATH: /usr/local/bin:/usr/bin:/bin:/app/api/.venv/bin
+      HINDSIGHT_API_LLM_PROVIDER: claude-code
+    volumes:
+      # ── Persistent data ────────────────────────────────────────────
+      # Writable pg0 data directory.  Must be writable by HOST_UID.
+      - ${HOME:-.}/.hindsight-docker:/home/hindsight/.pg0
+
+      # ── Claude credentials (read-only, single-file mounts) ────────
+      # A whole-directory :ro mount of ~/.claude silently breaks the
+      # CLI, which writes session/project state at runtime — so we
+      # mount only the two credential files.
+      - ${HOME}/.claude/.credentials.json:/home/hindsight/.claude/.credentials.json:ro
+      - ${HOME}/.claude.json:/home/hindsight/.claude.json:ro
+
+      # ── Claude CLI install (read-only) ─────────────────────────────
+      - ${HOME}/.local/share/claude:/home/hindsight/.local/share/claude:ro
+
+      # ── SDK bundled-binary override ────────────────────────────────
+      # The claude-agent-sdk 0.5.x image bundles v2.1.121 which has a
+      # protocol incompatibility in containers.  Override it with the
+      # host's v2.1.128+ binary.  Drop this mount once claude-agent-sdk
+      # ships with v2.1.128+.
+      - ${HOME}/.local/share/claude/versions/${CLAUDE_CLI_VERSION:-2.1.128}:/app/api/.venv/lib/python3.11/site-packages/claude_agent_sdk/_bundled/claude:ro

docker/docker-compose/custom-models/Dockerfile

@@ -0,0 +1,34 @@
+# Example: custom Hindsight image with non-default local models baked in.
+#
+# Use this pattern in production when you run a non-default embedder or
+# reranker. Baking models into the image removes the runtime dependency on
+# HuggingFace and lets the container registry handle caching per node, so
+# you don't need a model-cache PVC.
+#
+# Built on top of the slim image so only the deps and models you actually
+# use end up in the final image.
+FROM ghcr.io/vectorize-io/hindsight:latest-slim
+
+# Install the local-ml deps required to load sentence-transformers /
+# cross-encoder models at runtime. Pinned ranges mirror hindsight-api-slim's
+# `local-ml` extra in hindsight-api-slim/pyproject.toml. Use `uv pip
+# install` against the image's venv explicitly: the slim image's venv was
+# created by `uv sync` and does not ship its own `pip`, so a bare
+# `pip install` would fall back to user site-packages and not be visible
+# to the runtime python.
+RUN uv pip install --python /app/api/.venv/bin/python --no-cache \
+    'sentence-transformers>=5.0.0' \
+    'transformers>=4.53.0' \
+    'torch>=2.6.0'
+
+# Pre-download the models you want to use. Replace these with your own.
+# The defaults bundled in the full image are BAAI/bge-small-en-v1.5 and
+# cross-encoder/ms-marco-MiniLM-L-6-v2; here we pick multilingual variants
+# as a concrete non-default example.
+ARG EMBEDDER=sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
+ARG RERANKER=cross-encoder/mmarco-mMiniLMv2-L12-H384-v1
+ENV HF_HUB_DOWNLOAD_TIMEOUT=600
+RUN python -c "\
+from sentence_transformers import SentenceTransformer, CrossEncoder; \
+SentenceTransformer('${EMBEDDER}'); \
+CrossEncoder('${RERANKER}')"

docker/docker-compose/custom-models/README.md

@@ -0,0 +1,81 @@
+# Hindsight with Custom Local Models
+
+Example Docker Compose setup that builds a Hindsight image with **non-default
+local embedder and reranker models baked in at build time**.
+
+This is the recommended pattern for production when you use a non-default
+local model: the container registry caches model layers per node, pod
+startup is deterministic, and you don't need a model-cache PVC (or any
+runtime dependency on HuggingFace).
+
+## When to use this
+
+- You override `HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL` or
+  `HINDSIGHT_API_RERANKER_LOCAL_MODEL` to a non-default model.
+- You want pod startup to be deterministic and offline-capable.
+- You'd otherwise reach for a Helm `modelCache` PVC just to avoid
+  re-downloading models.
+
+If you're using the **default** local models, the published full image
+(`ghcr.io/vectorize-io/hindsight:latest`) already bakes them in — you don't
+need this example.
+
+If you're using **external** providers (TEI, OpenAI, Cohere, ...) for
+embeddings and reranking, use the slim image directly — no models are
+needed in the image.
+
+## Quick start
+
+```bash
+export OPENAI_API_KEY=sk-xxx
+
+docker compose -f docker/docker-compose/custom-models/docker-compose.yaml up --build
+```
+
+- API: http://localhost:8888
+- Control Plane: http://localhost:9999
+
+## Using your own models
+
+Override the build args to bake different models:
+
+```bash
+docker compose -f docker/docker-compose/custom-models/docker-compose.yaml build \
+  --build-arg EMBEDDER=your-org/your-embedder \
+  --build-arg RERANKER=your-org/your-reranker
+```
+
+Then update the matching `HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL` and
+`HINDSIGHT_API_RERANKER_LOCAL_MODEL` values in `docker-compose.yaml` so the
+runtime points at the same model IDs.
+
+## Verifying the models are baked in
+
+`docker-compose.yaml` sets `HF_HUB_OFFLINE=1` and `TRANSFORMERS_OFFLINE=1`
+so that any attempt to download a model at runtime fails loudly instead of
+silently re-downloading. If the container starts and serves recall queries
+with these set, the models are correctly baked in.
+
+You can also inspect the image directly:
+
+```bash
+docker run --rm --entrypoint sh hindsight-custom-models-hindsight \
+  -c 'ls ~/.cache/huggingface/hub/'
+```
+
+## Why not a model-cache PVC?
+
+The Helm chart exposes an optional `api.persistence.modelCache` PVC for
+caching downloaded models across pod restarts. Compared to baking models
+into the image:
+
+- A PVC adds storage cost — one PVC per worker replica with
+  `volumeClaimTemplates`.
+- `ReadWriteOnce` (the default) pins pods to a node.
+- The PVC needs lifecycle management on `helm uninstall` / `helm upgrade`
+  — without `helm.sh/resource-policy: keep` it is deleted on uninstall;
+  with it, storage keeps billing forever until manually cleaned up.
+- Pod startup still depends on HuggingFace being reachable on first run.
+
+Image layers, by contrast, are pulled once per node and cached for free by
+the container runtime, with no orphaned-storage cleanup story.

docker/docker-compose/custom-models/docker-compose.yaml

@@ -0,0 +1,44 @@
+name: hindsight-custom-models
+# Example: run a custom Hindsight image with non-default local models baked
+# in at build time, so pod startup does not depend on HuggingFace at runtime.
+#
+# Quick start:
+#   export OPENAI_API_KEY=sk-xxx
+#   docker compose -f docker/docker-compose/custom-models/docker-compose.yaml up --build
+#
+# Required environment variables:
+# - OPENAI_API_KEY (or configure another LLM provider via HINDSIGHT_API_LLM_*)
+
+services:
+  hindsight:
+    build:
+      context: .
+      dockerfile: Dockerfile
+      # Override at build time to bake different models:
+      #   docker compose build --build-arg EMBEDDER=your-org/your-embedder
+      args:
+        EMBEDDER: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
+        RERANKER: cross-encoder/mmarco-mMiniLMv2-L12-H384-v1
+    container_name: hindsight-custom-models
+    ports:
+      - "8888:8888"
+      - "9999:9999"
+    environment:
+      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
+      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
+
+      # Point Hindsight at the models baked into the image above.
+      HINDSIGHT_API_EMBEDDINGS_PROVIDER: local
+      HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
+      HINDSIGHT_API_RERANKER_PROVIDER: local
+      HINDSIGHT_API_RERANKER_LOCAL_MODEL: cross-encoder/mmarco-mMiniLMv2-L12-H384-v1
+
+      # Fail fast if a model is missing from the image instead of silently
+      # falling back to a HuggingFace download at runtime.
+      HF_HUB_OFFLINE: "1"
+      TRANSFORMERS_OFFLINE: "1"
+    volumes:
+      - pg_data:/home/hindsight/.pg0
+
+volumes:
+  pg_data:

docker/docker-compose/external-pg/docker-compose.yaml

@@ -0,0 +1,54 @@
+# Docker Compose file for Hindsight with PostgreSQL and pgvector
+#
+# Make sure to set the required environment variables before running:
+# - HINDSIGHT_DB_PASSWORD: Password for the PostgreSQL user
+# - Configure LLM provider variables as needed (see below in the hindsight service)
+#
+# Usage:
+# docker compose up -d
+#
+# Optional environment variables with defaults:
+# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
+# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
+# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
+# - HINDSIGHT_DB_VERSION: PostgreSQL version (default: 18)
+
+services:
+  db:
+    # Use a PostgreSQL-Image with pgvector extension pre-installed
+    # see https://hub.docker.com/r/pgvector/pgvector 
+    image: pgvector/pgvector:pg${HINDSIGHT_DB_VERSION:-18}
+    container_name: hindsight-db
+    restart: always
+    # Expose PostgreSQL port
+    # ports:
+      # - "5432:5432"
+    environment:
+      POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
+      POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:?Please set the HINDSIGHT_DB_PASSWORD env variable}
+      POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
+    volumes:
+      - pg_data:/var/lib/postgresql/${HINDSIGHT_DB_VERSION:-18}/docker
+    networks:
+      - hindsight-net
+
+  hindsight:
+    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
+    container_name: hindsight-app
+    ports:
+      - "8888:8888"
+      - "9999:9999"
+    environment:
+      - HINDSIGHT_API_LLM_API_KEY=${OPENAI_API_KEY?Please set the OPENAI_API_KEY env variable}
+      - HINDSIGHT_API_DATABASE_URL=postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:?Please set the HINDSIGHT_DB_PASSWORD env variable}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
+    depends_on:
+      - db
+    networks:
+      - hindsight-net
+
+networks:
+  hindsight-net:
+    driver: bridge
+
+volumes:
+  pg_data:

docker/docker-compose/local-llm/README.md

@@ -0,0 +1,103 @@
+# Hindsight with a local llama.cpp server sidecar
+
+Example Docker Compose setup that runs Hindsight against a **local
+llama.cpp server**, fully offline, with no external API key required.
+
+## Architecture
+
+```
+┌────────────┐  HTTP /v1/chat/completions  ┌──────────────────────────────┐
+│ hindsight  │ ──────────────────────────▶ │ llama.cpp server (sidecar)   │
+│ (API + CP) │                             │ ghcr.io/ggml-org/llama.cpp   │
+└────────────┘                             └──────────────────────────────┘
+```
+
+`llama.cpp` runs as its own container and exposes an OpenAI-compatible
+HTTP API. Hindsight talks to it via the standard `openai` LLM provider
+with `HINDSIGHT_API_LLM_BASE_URL` pointed at the sidecar.
+
+This pattern follows
+[*Hosting llama-server with Docker* (ServiceStack)](https://servicestack.net/posts/hosting-llama-server).
+
+### Why a sidecar and not the in-process `llamacpp` provider?
+
+Hindsight does ship an in-process `llamacpp` provider that spawns
+`llama-cpp-python`, but the **published `ghcr.io/vectorize-io/hindsight`
+image deliberately omits `llama-cpp-python`** to keep the image small and
+avoid bundling native inference libraries that most users don't need.
+Trying to set `HINDSIGHT_API_LLM_PROVIDER=llamacpp` against the published
+image fails with `ModuleNotFoundError: No module named 'llama_cpp'`.
+
+The sidecar approach side-steps that entirely: the official llama.cpp
+image is used as-is for inference, Hindsight is used as-is for memory.
+Clean separation, no derived images.
+
+## Quick start
+
+```bash
+docker compose -f docker/docker-compose/local-llm/docker-compose.yaml up
+```
+
+- API: http://localhost:8888
+- Control Plane: http://localhost:9999
+
+**First boot downloads ~3.5 GB** (Gemma 4 E2B Q4_K_M GGUF) into the
+`llama_models` named volume. Subsequent boots reuse it.
+
+Hindsight only starts after llama.cpp's `/health` endpoint reports
+healthy, so the API will appear "stuck" for a few minutes on the first
+run while the model downloads.
+
+## Using a different model
+
+Override the HuggingFace repo / file in `docker-compose.yaml`:
+
+```yaml
+environment:
+  LLAMA_ARG_HF_REPO: bartowski/Qwen2.5-7B-Instruct-GGUF
+  LLAMA_ARG_HF_FILE: Qwen2.5-7B-Instruct-Q4_K_M.gguf
+```
+
+Also update `HINDSIGHT_API_LLM_MODEL` on the `hindsight` service to a
+matching alias (the value is sent to llama-server as the OpenAI `model`
+field — llama-server is lenient about this but it shows up in logs).
+
+## GPU acceleration
+
+The default compose file targets CPU because not everyone has a GPU. On
+CPU, Gemma 4 E2B runs at ~2-3 tokens/sec — fine for a smoke test, but the
+retain pipeline (which makes several multi-hundred-token LLM calls per
+memory) will time out against Hindsight's default LLM timeout. **For any
+real use, run on a GPU.**
+
+### NVIDIA
+
+1. Switch the `llama` service image from `:server` to `:server-cuda`.
+2. Uncomment the `LLAMA_ARG_N_GPU_LAYERS: "999"` env var (offload all
+   layers to GPU).
+3. Uncomment the `deploy.resources.reservations.devices` block.
+4. Install the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
+   on the host.
+
+The compose file has all four spots marked with inline comments.
+
+### Apple Silicon / ROCm / Vulkan
+
+The official `ghcr.io/ggml-org/llama.cpp` image only ships CPU and CUDA
+variants. For Metal (Apple Silicon), ROCm (AMD), or Vulkan backends,
+build llama.cpp yourself with the appropriate flags and reference the
+image you build instead. Docker Desktop on macOS cannot pass through the
+host GPU to a Linux container in any case — for Apple Silicon, run
+llama-server directly on the host and only put Hindsight in Docker.
+
+## Caveats
+
+- llama.cpp's HTTP API is OpenAI-compatible but not 100% feature-parity.
+  Function/tool calling support depends on the chat template baked into
+  the GGUF; some retain/reflect flows may behave differently than against
+  a hosted OpenAI model.
+- Small GGUFs (~3 B params) are useful for smoke testing but will
+  underperform a hosted frontier model on retain quality. Use a larger
+  GGUF (7-13 B params) for production-quality memory.
+- The `llama_models` named volume persists the GGUF across `docker
+  compose down`/`up` so the model is downloaded once, not every restart.

docker/docker-compose/local-llm/docker-compose.yaml

@@ -0,0 +1,74 @@
+name: hindsight-local-llm
+# Example: run Hindsight against a local llama.cpp server sidecar — fully
+# offline, no external API key needed.
+#
+# Pattern follows https://servicestack.net/posts/hosting-llama-server :
+# llama.cpp runs as its own container exposing an OpenAI-compatible HTTP
+# API, and Hindsight talks to it via the `openai` LLM provider with a
+# custom `base_url`. This means we can use the published Hindsight image
+# unchanged — no derived Dockerfile, no `llama-cpp-python` install on top.
+#
+# Quick start:
+#   docker compose -f docker/docker-compose/local-llm/docker-compose.yaml up
+#
+# First boot downloads the default Gemma 4 E2B GGUF (~3.5 GB) into the
+# `llama_models` volume; subsequent boots reuse it.
+
+services:
+  llama:
+    image: ghcr.io/ggml-org/llama.cpp:server
+    container_name: hindsight-local-llm-llama
+    environment:
+      LLAMA_ARG_HOST: 0.0.0.0
+      LLAMA_ARG_PORT: "8080"
+      # Auto-download a small GGUF from HuggingFace on first start.
+      # Override these to use a different model.
+      LLAMA_ARG_HF_REPO: bartowski/google_gemma-4-E2B-it-GGUF
+      LLAMA_ARG_HF_FILE: google_gemma-4-E2B-it-Q4_K_M.gguf
+      LLAMA_ARG_CTX_SIZE: "8192"
+      # Uncomment for NVIDIA GPU (and switch image to :server-cuda):
+      # LLAMA_ARG_N_GPU_LAYERS: "999"
+    volumes:
+      # llama-server stores HuggingFace downloads under ~/.cache/huggingface
+      # (not ~/.cache/llama.cpp), so mount the named volume there to avoid
+      # re-downloading the GGUF on every recreate.
+      - llama_models:/root/.cache/huggingface
+    healthcheck:
+      test: ["CMD-SHELL", "curl -fsS http://localhost:8080/health || exit 1"]
+      interval: 10s
+      timeout: 5s
+      retries: 60
+      start_period: 30s
+    # For NVIDIA GPU acceleration, swap the image above to
+    # `ghcr.io/ggml-org/llama.cpp:server-cuda` and uncomment:
+    # deploy:
+    #   resources:
+    #     reservations:
+    #       devices:
+    #         - driver: nvidia
+    #           count: all
+    #           capabilities: [gpu]
+
+  hindsight:
+    image: ghcr.io/vectorize-io/hindsight:latest
+    container_name: hindsight-local-llm
+    depends_on:
+      llama:
+        condition: service_healthy
+    ports:
+      - "8888:8888"
+      - "9999:9999"
+    environment:
+      # llama-server is OpenAI-compatible, so use the `openai` provider and
+      # point base_url at the sidecar. The API key is unused by llama-server
+      # but Hindsight requires the env var to be set.
+      HINDSIGHT_API_LLM_PROVIDER: openai
+      HINDSIGHT_API_LLM_BASE_URL: http://llama:8080/v1
+      HINDSIGHT_API_LLM_API_KEY: not-needed
+      HINDSIGHT_API_LLM_MODEL: gemma-4-e2b-it
+    volumes:
+      - pg_data:/home/hindsight/.pg0
+
+volumes:
+  pg_data:
+  llama_models:

docker/docker-compose/nginx/README.md

@@ -0,0 +1,96 @@
+# Nginx Reverse Proxy with Custom Base Path
+
+Deploy Hindsight API under `/hindsight` (or any custom path) using Nginx reverse proxy.
+
+## Quick Start (Published Image - API Only)
+
+```bash
+docker-compose up
+```
+
+- **API:** http://localhost:8080/hindsight/docs
+- **Control Plane:** http://localhost:9999 (direct access, not proxied)
+
+## Full Stack with Custom Base Path (Requires Build)
+
+**Important:** You cannot rebuild from the published image with build args. You must build from source.
+
+### Build from Source with Custom Base Path
+
+1. **Clone the repository** (if you haven't):
+```bash
+git clone https://github.com/vectorize-io/hindsight.git
+cd hindsight
+```
+
+2. **Build with base path**:
+```bash
+docker build \
+  --build-arg NEXT_PUBLIC_BASE_PATH=/hindsight \
+  -f docker/standalone/Dockerfile \
+  -t hindsight:custom \
+  .
+```
+
+3. **Update docker-compose.yml** to use your built image:
+```yaml
+services:
+  hindsight:
+    image: hindsight:custom  # ← Change this
+    environment:
+      HINDSIGHT_API_BASE_PATH: /hindsight
+      NEXT_PUBLIC_BASE_PATH: /hindsight
+```
+
+4. **Update nginx.conf** to handle Control Plane routes (see below)
+
+5. **Run**:
+```bash
+docker-compose up
+```
+
+### Required nginx.conf for Full Stack
+
+Replace the current `nginx.conf` with this to proxy both API and Control Plane:
+
+```nginx
+events { worker_connections 1024; }
+
+http {
+    include /etc/nginx/mime.types;
+    default_type application/octet-stream;
+
+    upstream hindsight_api { server hindsight:8888; }
+    upstream hindsight_cp { server hindsight:9999; }
+
+    server {
+        listen 80;
+
+        # API
+        location ~ ^/hindsight/(docs|openapi\.json|health|metrics|v1|mcp) {
+            proxy_pass http://hindsight_api;
+            proxy_set_header Host $http_host;
+        }
+
+        # Control Plane static files
+        location ~ ^/hindsight/_next/ {
+            proxy_pass http://hindsight_cp;
+            proxy_set_header Host $http_host;
+        }
+
+        # Control Plane UI
+        location /hindsight {
+            proxy_pass http://hindsight_cp;
+            proxy_set_header Host $http_host;
+        }
+
+        location = / { return 301 /hindsight; }
+    }
+}
+```
+
+### Why Build is Required
+
+Next.js requires `basePath` at **build time**. The published image was built without a custom base path, so you must rebuild from source with the `NEXT_PUBLIC_BASE_PATH` build arg to deploy the Control Plane under a subpath.
+
+The API works without rebuild because `HINDSIGHT_API_BASE_PATH` is a runtime environment variable.

docker/docker-compose/nginx/docker-compose.yml

@@ -0,0 +1,90 @@
+# Hindsight API deployment with Nginx reverse proxy (API-only)
+#
+# This example deploys Hindsight API under the path /hindsight with:
+# - Hindsight standalone image (API + Control Plane + embedded pg0)
+# - Nginx reverse proxy (API only)
+#
+# Quick Start:
+#   docker-compose -f docker/docker-compose/nginx/docker-compose.yml up
+#
+# Access:
+#   API (via nginx):         http://localhost:8080/hindsight/docs
+#   Control Plane (direct):  http://localhost:9999
+#
+# For full stack deployment (API + Control Plane both under /hindsight):
+#   See README.md in this directory for instructions on building with basePath.
+#
+# Note: This configuration uses the published image (no build required).
+#       Control Plane is served directly because Next.js basePath requires
+#       build-time configuration. See README.md for the full stack option.
+
+services:
+  # Hindsight (API + Control Plane + embedded pg0)
+  hindsight:
+    image: ghcr.io/vectorize-io/hindsight:latest
+    ports:
+      - "9999:9999"  # Control Plane (direct access, not proxied)
+    environment:
+      # API base path for reverse proxy
+      HINDSIGHT_API_BASE_PATH: /hindsight
+
+      # LLM configuration
+      # Using mock provider for testing (no API key needed)
+      # For production, set OPENAI_API_KEY or ANTHROPIC_API_KEY and use a real provider
+      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-mock}
+      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-not-needed-for-mock}
+      HINDSIGHT_API_LLM_MODEL: ${HINDSIGHT_API_LLM_MODEL:-mock-model}
+
+      # Production examples (uncomment and set appropriate API key):
+      # HINDSIGHT_API_LLM_PROVIDER: openai
+      # HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY}
+      # HINDSIGHT_API_LLM_MODEL: gpt-4o-mini
+
+      # HINDSIGHT_API_LLM_PROVIDER: anthropic
+      # HINDSIGHT_API_LLM_API_KEY: ${ANTHROPIC_API_KEY}
+      # HINDSIGHT_API_LLM_MODEL: claude-sonnet-4-20250514
+
+      # Server config
+      HINDSIGHT_API_HOST: 0.0.0.0
+      HINDSIGHT_API_PORT: 8888
+      HINDSIGHT_API_LOG_LEVEL: info
+
+      # Control Plane config
+      HINDSIGHT_CP_DATAPLANE_API_URL: http://localhost:8888
+      # Optional: Require a shared access key for Control Plane UI access
+      # HINDSIGHT_CP_ACCESS_KEY: your-secret-key
+    volumes:
+      # Persist embedded pg0 database
+      - hindsight_data:/app/data
+    # Note: Ports not exposed - access via Nginx at localhost:8080/hindsight/
+    # To debug directly, uncomment these ports:
+    # ports:
+    #   - "8888:8888"  # API
+    #   - "9999:9999"  # Control Plane
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:8888/hindsight/health"]
+      interval: 10s
+      timeout: 5s
+      retries: 3
+      start_period: 30s
+    networks:
+      - hindsight
+
+  # Nginx reverse proxy
+  nginx:
+    image: nginx:alpine
+    ports:
+      - "8080:80"
+    volumes:
+      - ./nginx.conf:/etc/nginx/nginx.conf:ro
+    depends_on:
+      hindsight:
+        condition: service_healthy
+    networks:
+      - hindsight
+
+volumes:
+  hindsight_data:
+
+networks:
+  hindsight:

docker/docker-compose/nginx/nginx.conf

@@ -0,0 +1,40 @@
+# Nginx configuration for API-only reverse proxy
+# Control Plane accessed directly (not through nginx)
+
+events {
+    worker_connections 1024;
+}
+
+http {
+    include /etc/nginx/mime.types;
+    default_type application/octet-stream;
+
+    # Logging
+    access_log /var/log/nginx/access.log;
+    error_log /var/log/nginx/error.log;
+
+    # Upstream - Hindsight API
+    upstream hindsight_api {
+        server hindsight:8888;
+    }
+
+    server {
+        listen 80;
+        server_name _;
+
+        # API endpoints - forward with /hindsight prefix
+        location /hindsight/ {
+            proxy_pass http://hindsight_api;
+
+            proxy_set_header Host $http_host;
+            proxy_set_header X-Real-IP $remote_addr;
+            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+            proxy_set_header X-Forwarded-Proto $scheme;
+        }
+
+        # Redirect root to API docs
+        location = / {
+            return 301 /hindsight/docs;
+        }
+    }
+}

docker/docker-compose/pg_search/Dockerfile

@@ -0,0 +1,7 @@
+# PostgreSQL with pgvector and ParadeDB pg_search extensions.
+#
+# The official ParadeDB image ships PostgreSQL with pg_search and pgvector
+# already installed, so no build steps are required. We pin to the PG17
+# variant for parity with the other Hindsight docker-compose examples
+# (vchord, pg_textsearch).
+FROM paradedb/paradedb:latest-pg17

docker/docker-compose/pg_search/docker-compose.yaml

@@ -0,0 +1,96 @@
+name: hindsight
+# Docker Compose file for Hindsight with PostgreSQL and ParadeDB pg_search.
+#
+# pg_search is the only BM25 backend supported by Hindsight that works with
+# Citus, so this is the recommended setup for horizontally scaled deployments.
+#
+# Usage:
+#   docker compose -f docker/docker-compose/pg_search/docker-compose.yaml up -d
+#
+# Required environment variables:
+# - HINDSIGHT_DB_PASSWORD: Password for the PostgreSQL user
+# - Configure LLM provider variables as needed (see the hindsight service)
+#
+# Optional environment variables with defaults:
+# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
+# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
+# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
+# - HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER: ParadeDB pg_search
+#   tokenizer for new BM25 indexes (default: empty, uses ParadeDB default)
+
+services:
+  db:
+    # Use ParadeDB image which bundles pgvector + pg_search
+    build:
+      context: .
+      dockerfile: Dockerfile
+    container_name: hindsight-db
+    restart: always
+    ports:
+      - "5437:5432"
+    environment:
+      POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
+      POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:-hindsight_password}
+      POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
+    volumes:
+      - pg_data:/var/lib/postgresql/data
+    networks:
+      - hindsight-net
+
+  pg-search-init:
+    build:
+      context: .
+      dockerfile: Dockerfile
+    depends_on:
+      - db
+    environment:
+      - PGPASSWORD=${HINDSIGHT_DB_PASSWORD:-hindsight_password}
+    command: >
+      bash -c "
+        echo 'Waiting for PostgreSQL to be ready...';
+        until pg_isready -h hindsight-db -p 5432 -U hindsight_user; do
+          echo 'PostgreSQL is unavailable - sleeping';
+          sleep 2;
+        done;
+        echo 'PostgreSQL is ready - creating hindsight_db database';
+        psql -h hindsight-db -p 5432 -U hindsight_user -c 'CREATE DATABASE hindsight_db;' 2>/dev/null || echo 'Database already exists';
+        echo 'Creating extensions in hindsight_db database';
+        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vector CASCADE;';
+        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS pg_search CASCADE;';
+        echo 'Database and extensions created successfully';
+      "
+    restart: "no"
+    networks:
+      - hindsight-net
+
+  hindsight:
+    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
+    container_name: hindsight-app
+    ports:
+      - "8888:8888"
+      - "9999:9999"
+    environment:
+      # LLM Configuration
+      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
+      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
+
+      # Database Configuration
+      HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
+
+      # Vector and Text Search Extensions
+      HINDSIGHT_API_VECTOR_EXTENSION: pgvector
+      HINDSIGHT_API_TEXT_SEARCH_EXTENSION: pg_search
+      HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER: ${HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER:-}
+
+    depends_on:
+      - db
+    networks:
+      - hindsight-net
+
+
+networks:
+  hindsight-net:
+    driver: bridge
+
+volumes:
+  pg_data:

docker/docker-compose/pg_textsearch/Dockerfile

@@ -0,0 +1,32 @@
+# PostgreSQL with pgvector and pg_textsearch extensions
+# Note: pg_textsearch requires PostgreSQL 17+
+FROM postgres:17
+
+# Install build dependencies
+RUN apt-get update && apt-get install -y \
+    build-essential \
+    git \
+    postgresql-server-dev-17 \
+    libpq-dev \
+    && rm -rf /var/lib/apt/lists/*
+
+# Install pgvector
+RUN cd /tmp && \
+    git clone --branch v0.8.0 https://github.com/pgvector/pgvector.git && \
+    cd pgvector && \
+    make && \
+    make install
+
+# Install pg_textsearch
+RUN cd /tmp && \
+    git clone https://github.com/timescale/pg_textsearch.git && \
+    cd pg_textsearch && \
+    make && \
+    make install
+
+# Clean up source files and build dependencies
+RUN rm -rf /tmp/pgvector /tmp/pg_textsearch && \
+    apt-get purge -y --auto-remove build-essential git postgresql-server-dev-17
+
+# Ensure extensions are preloaded
+RUN echo "shared_preload_libraries = 'pg_textsearch'" >> /usr/share/postgresql/postgresql.conf.sample

docker/docker-compose/pg_textsearch/docker-compose.yaml

@@ -0,0 +1,91 @@
+name: hindsight
+# Docker Compose file for Hindsight with PostgreSQL and Timescale pg_textsearch
+# docker compose -f docker/docker-compose/pg_textsearch/docker-compose.yaml down && sleep 2 && docker compose -f docker/docker-compose/pg_textsearch/docker-compose.yaml up -d
+# Make sure to set the required environment variables before running:
+# - HINDSIGHT_DB_PASSWORD: Password for the PostgreSQL user
+# - Configure LLM provider variables as needed (see below in the hindsight service)
+#
+# Usage:
+# docker compose up -d
+#
+# Optional environment variables with defaults:
+# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
+# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
+# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
+
+services:
+  db:
+    # Use custom PostgreSQL image with pgvector and pg_textsearch extensions
+    build:
+      context: .
+      dockerfile: Dockerfile
+    container_name: hindsight-db
+    restart: always
+    # Expose PostgreSQL port
+    ports:
+      - "5437:5432"
+    environment:
+      POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
+      POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:-hindsight_password}
+      POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
+    volumes:
+      - pg_data:/var/lib/postgresql/data
+    networks:
+      - hindsight-net
+
+  pg-textsearch-init:
+    build:
+      context: .
+      dockerfile: Dockerfile
+    depends_on:
+      - db
+    environment:
+      - PGPASSWORD=${HINDSIGHT_DB_PASSWORD:-hindsight_password}
+    command: >
+      bash -c "
+        echo 'Waiting for PostgreSQL to be ready...';
+        until pg_isready -h hindsight-db -p 5432 -U hindsight_user; do
+          echo 'PostgreSQL is unavailable - sleeping';
+          sleep 2;
+        done;
+        echo 'PostgreSQL is ready - creating hindsight_db database';
+        psql -h hindsight-db -p 5432 -U hindsight_user -c 'CREATE DATABASE hindsight_db;' 2>/dev/null || echo 'Database already exists';
+        echo 'Creating extensions in hindsight_db database';
+        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vector CASCADE;';
+        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS pg_textsearch CASCADE;';
+        echo 'Database and extensions created successfully';
+      "
+    restart: "no"
+    networks:
+      - hindsight-net
+
+  hindsight:
+    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
+    container_name: hindsight-app
+    ports:
+      - "8888:8888"
+      - "9999:9999"
+    environment:
+      # LLM Configuration
+      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
+      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
+
+      # Database Configuration
+      HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
+
+      # Vector and Text Search Extensions
+      HINDSIGHT_API_VECTOR_EXTENSION: pgvector
+      HINDSIGHT_API_TEXT_SEARCH_EXTENSION: pg_textsearch
+
+    depends_on:
+      - db
+    networks:
+      - hindsight-net
+
+
+networks:
+  hindsight-net:
+    driver: bridge
+
+volumes:
+  pg_data:

docker/docker-compose/pgroonga/Dockerfile

@@ -0,0 +1,13 @@
+# PostgreSQL with pgvector and pgroonga extensions.
+#
+# pgroonga is a multilingual full-text search extension built on Groonga.
+# It works out of the box for CJK (Chinese, Japanese, Korean) and other
+# non-whitespace-segmented languages via the TokenBigram tokenizer.
+FROM groonga/pgroonga:4.0.8-debian-17
+
+# Install pgvector on top of the pgroonga base image (which already provides
+# pgroonga, the Groonga library, and the PostgreSQL PGDG package repository).
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    postgresql-17-pgvector=0.8.6-1.pgdg13+1 \
+    && apt-get clean \
+    && rm -rf /var/lib/apt/lists/*

docker/docker-compose/pgroonga/docker-compose.yaml

@@ -0,0 +1,91 @@
+name: hindsight
+# Docker Compose file for Hindsight with PostgreSQL and pgroonga
+#
+# pgroonga provides multilingual BM25 indexing that works out of the box for
+# CJK (Chinese, Japanese, Korean) and other non-whitespace-segmented languages.
+# Use this recipe if your bank content is not English/European.
+#
+# docker compose -f docker/docker-compose/pgroonga/docker-compose.yaml down && \
+#   sleep 2 && \
+#   docker compose -f docker/docker-compose/pgroonga/docker-compose.yaml up -d
+#
+# Optional environment variables with defaults:
+# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
+# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
+# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
+# - HINDSIGHT_DB_PASSWORD: PostgreSQL password (default: hindsight_password)
+
+services:
+  db:
+    build:
+      context: .
+      dockerfile: Dockerfile
+    container_name: hindsight-db
+    restart: always
+    ports:
+      - "5439:5432"
+    environment:
+      POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
+      POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:-hindsight_password}
+      POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
+    volumes:
+      - pg_data:/var/lib/postgresql/data
+    networks:
+      - hindsight-net
+
+  pgroonga-init:
+    build:
+      context: .
+      dockerfile: Dockerfile
+    depends_on:
+      - db
+    environment:
+      - PGPASSWORD=${HINDSIGHT_DB_PASSWORD:-hindsight_password}
+    command: >
+      bash -c "
+        echo 'Waiting for PostgreSQL to be ready...';
+        until pg_isready -h hindsight-db -p 5432 -U hindsight_user; do
+          echo 'PostgreSQL is unavailable - sleeping';
+          sleep 2;
+        done;
+        echo 'PostgreSQL is ready - creating hindsight_db database';
+        psql -h hindsight-db -p 5432 -U hindsight_user -c 'CREATE DATABASE hindsight_db;' 2>/dev/null || echo 'Database already exists';
+        echo 'Creating extensions in hindsight_db database';
+        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vector CASCADE;';
+        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS pgroonga CASCADE;';
+        echo 'Database and extensions created successfully';
+      "
+    restart: "no"
+    networks:
+      - hindsight-net
+
+  hindsight:
+    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
+    container_name: hindsight-app
+    ports:
+      - "8888:8888"
+      - "9999:9999"
+    environment:
+      # LLM Configuration
+      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
+      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
+
+      # Database Configuration
+      HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
+
+      # Vector and Text Search Extensions
+      HINDSIGHT_API_VECTOR_EXTENSION: pgvector
+      HINDSIGHT_API_TEXT_SEARCH_EXTENSION: pgroonga
+
+    depends_on:
+      - db
+    networks:
+      - hindsight-net
+
+
+networks:
+  hindsight-net:
+    driver: bridge
+
+volumes:
+  pg_data:

docker/docker-compose/s3-file-storage/docker-compose.yaml

@@ -0,0 +1,83 @@
+# Docker Compose file for Hindsight with S3 file storage (SeaweedFS)
+#
+# SeaweedFS (Apache 2.0) provides an S3-compatible object storage backend
+# for storing uploaded files instead of PostgreSQL BYTEA storage.
+#
+# Make sure to set the required environment variables before running:
+# - HINDSIGHT_DB_PASSWORD: Password for the PostgreSQL user
+# - Configure LLM provider variables as needed (see below in the hindsight service)
+#
+# Usage:
+# docker compose up -d
+#
+# Optional environment variables with defaults:
+# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
+# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
+# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
+# - HINDSIGHT_DB_VERSION: PostgreSQL version (default: 18)
+# - SEAWEEDFS_S3_ACCESS_KEY: S3 access key (default: hindsight_s3_key)
+# - SEAWEEDFS_S3_SECRET_KEY: S3 secret key (default: hindsight_s3_secret)
+
+services:
+  db:
+    image: pgvector/pgvector:pg${HINDSIGHT_DB_VERSION:-18}
+    container_name: hindsight-db
+    restart: always
+    environment:
+      POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
+      POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:?Please set the HINDSIGHT_DB_PASSWORD env variable}
+      POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
+    volumes:
+      - pg_data:/var/lib/postgresql/${HINDSIGHT_DB_VERSION:-18}/docker
+    networks:
+      - hindsight-net
+
+  seaweedfs:
+    image: chrislusf/seaweedfs:latest
+    container_name: hindsight-seaweedfs
+    restart: always
+    # Single-node mode: master + volume + filer + S3 gateway all in one process
+    command: >
+      server
+      -s3
+      -s3.port=8333
+      -s3.config=/etc/seaweedfs/s3.json
+      -ip.bind=0.0.0.0
+    volumes:
+      - seaweedfs_data:/data
+      - ./s3.json:/etc/seaweedfs/s3.json:ro
+    # Expose S3 API port (uncomment to access from host)
+    # ports:
+    #   - "8333:8333"
+    networks:
+      - hindsight-net
+
+  hindsight:
+    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
+    container_name: hindsight-app
+    ports:
+      - "8888:8888"
+      - "9999:9999"
+    environment:
+      - HINDSIGHT_API_LLM_API_KEY=${OPENAI_API_KEY?Please set the OPENAI_API_KEY env variable}
+      - HINDSIGHT_API_DATABASE_URL=postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:?Please set the HINDSIGHT_DB_PASSWORD env variable}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
+      # S3 file storage configuration (SeaweedFS)
+      - HINDSIGHT_API_FILE_STORAGE_TYPE=s3
+      - HINDSIGHT_API_FILE_STORAGE_S3_BUCKET=hindsight
+      - HINDSIGHT_API_FILE_STORAGE_S3_ENDPOINT=http://seaweedfs:8333
+      - HINDSIGHT_API_FILE_STORAGE_S3_REGION=us-east-1
+      - HINDSIGHT_API_FILE_STORAGE_S3_ACCESS_KEY_ID=${SEAWEEDFS_S3_ACCESS_KEY:-hindsight_s3_key}
+      - HINDSIGHT_API_FILE_STORAGE_S3_SECRET_ACCESS_KEY=${SEAWEEDFS_S3_SECRET_KEY:-hindsight_s3_secret}
+    depends_on:
+      - db
+      - seaweedfs
+    networks:
+      - hindsight-net
+
+networks:
+  hindsight-net:
+    driver: bridge
+
+volumes:
+  pg_data:
+  seaweedfs_data:

docker/docker-compose/s3-file-storage/s3.json

@@ -0,0 +1,19 @@
+{
+  "identities": [
+    {
+      "name": "hindsight",
+      "credentials": [
+        {
+          "accessKey": "hindsight_s3_key",
+          "secretKey": "hindsight_s3_secret"
+        }
+      ],
+      "actions": [
+        "Admin",
+        "Read",
+        "Write",
+        "List"
+      ]
+    }
+  ]
+}

docker/docker-compose/timescale/.dockerignore

@@ -0,0 +1,16 @@
+# Git
+.git
+.gitignore
+.gitattributes
+
+# Docker
+docker-compose.yaml
+.dockerignore
+
+# Documentation
+README.md
+*.md
+
+# Environment
+.env
+.env.example

docker/docker-compose/timescale/.env.example

@@ -0,0 +1,25 @@
+# PostgreSQL Configuration
+HINDSIGHT_DB_USER=hindsight_user
+HINDSIGHT_DB_PASSWORD=change-me-to-secure-password
+HINDSIGHT_DB_NAME=hindsight_db
+
+# Hindsight Version
+HINDSIGHT_VERSION=latest
+
+# LLM Configuration
+HINDSIGHT_API_LLM_PROVIDER=openai
+OPENAI_API_KEY=your-openai-api-key-here
+
+# Alternative LLM providers (uncomment and configure as needed):
+# HINDSIGHT_API_LLM_PROVIDER=anthropic
+# ANTHROPIC_API_KEY=your-anthropic-api-key
+
+# HINDSIGHT_API_LLM_PROVIDER=gemini
+# GEMINI_API_KEY=your-gemini-api-key
+
+# HINDSIGHT_API_LLM_PROVIDER=groq
+# GROQ_API_KEY=your-groq-api-key
+
+# Vector and Text Search (already configured in docker-compose.yaml)
+# HINDSIGHT_API_VECTOR_EXTENSION=pgvectorscale
+# HINDSIGHT_API_TEXT_SEARCH_EXTENSION=pg_textsearch

docker/docker-compose/timescale/Dockerfile

@@ -0,0 +1,55 @@
+# PostgreSQL with pgvector, pgvectorscale, and pg_textsearch extensions
+# All three extensions from Timescale/pgvector for high-performance vector and text search
+# Note: Requires PostgreSQL 16+
+FROM postgres:17
+
+# Install build dependencies and Rust toolchain
+RUN apt-get update && apt-get install -y \
+    build-essential \
+    git \
+    postgresql-server-dev-17 \
+    libpq-dev \
+    cmake \
+    curl \
+    pkg-config \
+    libssl-dev \
+    && rm -rf /var/lib/apt/lists/*
+
+# Install Rust toolchain (required for pgvectorscale)
+RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
+ENV PATH="/root/.cargo/bin:${PATH}"
+
+# Install pgvector (required by pgvectorscale)
+RUN cd /tmp && \
+    git clone --branch v0.8.0 https://github.com/pgvector/pgvector.git && \
+    cd pgvector && \
+    make && \
+    make install && \
+    rm -rf /tmp/pgvector
+
+# Install cargo-pgrx (PostgreSQL extension framework for Rust)
+RUN cargo install cargo-pgrx --version 0.12.5 --locked && \
+    cargo pgrx init --pg17 /usr/bin/pg_config
+
+# Install pgvectorscale (DiskANN index support)
+RUN cd /tmp && \
+    git clone --branch 0.5.1 https://github.com/timescale/pgvectorscale.git && \
+    cd pgvectorscale/pgvectorscale && \
+    cargo pgrx install --release && \
+    rm -rf /tmp/pgvectorscale
+
+# Install pg_textsearch (BM25 text search)
+RUN cd /tmp && \
+    git clone https://github.com/timescale/pg_textsearch.git && \
+    cd pg_textsearch && \
+    make && \
+    make install && \
+    rm -rf /tmp/pg_textsearch
+
+# Clean up build dependencies (keep runtime dependencies)
+RUN apt-get purge -y --auto-remove git cmake curl && \
+    rm -rf /root/.cargo/registry /root/.cargo/git
+
+# Ensure extensions are preloaded (pg_textsearch requires preloading)
+RUN echo "shared_preload_libraries = 'pg_textsearch'" >> /usr/share/postgresql/postgresql.conf.sample
+

docker/docker-compose/timescale/README.md

@@ -0,0 +1,101 @@
+# Hindsight with Timescale Extensions
+
+This Docker Compose setup provides a complete Hindsight deployment with **Timescale extensions**:
+- **pgvectorscale** - DiskANN algorithm for disk-based scalable vector search  
+- **pg_textsearch** - High-performance BM25 text search
+
+Both extensions are from [Timescale](https://github.com/timescale) and provide production-grade performance.
+
+## Prerequisites
+
+- Docker and Docker Compose installed
+- OpenAI API key (or another LLM provider)
+
+## Quick Start
+
+```bash
+# Set environment variables
+export HINDSIGHT_DB_PASSWORD="your-secure-password"
+export OPENAI_API_KEY="your-openai-api-key"
+
+# Build and start
+docker compose -f docker/docker-compose/timescale/docker-compose.yaml up -d --build
+
+# Check logs
+
+docker compose -f docker/docker-compose/timescale/docker-compose.yaml logs -f
+```
+
+**Access:**
+- API: http://localhost:8888
+- Control Plane: http://localhost:9999
+
+## Stop and Clean Up
+
+```bash
+# Stop services
+docker compose -f docker/docker-compose/timescale/docker-compose.yaml down
+
+# Remove volumes (deletes all data)
+docker compose -f docker/docker-compose/timescale/docker-compose.yaml down -v
+```
+
+## Configuration
+
+### Environment Variables
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `HINDSIGHT_DB_PASSWORD` | PostgreSQL password | `hindsight_password` |
+| `HINDSIGHT_DB_USER` | PostgreSQL username | `hindsight_user` |
+| `HINDSIGHT_DB_NAME` | Database name | `hindsight_db` |
+| `HINDSIGHT_VERSION` | Hindsight Docker image version | `latest` |
+| `OPENAI_API_KEY` | OpenAI API key | (required) |
+| `HINDSIGHT_API_LLM_PROVIDER` | LLM provider | `openai` |
+
+### Why Timescale Extensions?
+
+**pgvectorscale (DiskANN):**
+- 28x lower p95 latency vs dedicated vector databases
+- 16x higher query throughput at 99% recall
+- 60-75% cost reduction (disk is cheaper than RAM)
+- Best for large datasets (10M+ vectors)
+
+**pg_textsearch (BM25):**
+- High-performance keyword retrieval
+- Native BM25 ranking algorithm
+- Optimized for full-text search
+
+## Troubleshooting
+
+### Extensions not installed
+
+Check if extensions are available:
+
+```bash
+docker exec -it hindsight-db-timescale psql -U hindsight_user -d hindsight_db -c "\dx"
+```
+
+You should see:
+- `vector` (pgvector)
+- `vectorscale` (pgvectorscale/DiskANN)
+- `pg_textsearch` (BM25 search)
+
+### Build fails
+
+If the Docker build fails during pgvectorscale compilation:
+
+1. Ensure you have sufficient memory (recommended: 4GB+)
+2. Check Docker build logs for Rust compilation errors
+3. Try building with more resources: `docker compose build --no-cache --memory 4g`
+
+### Port conflicts
+
+If port 5438 is already in use, modify the `ports` section in docker-compose.yaml.
+
+## Learn More
+
+- [pgvectorscale GitHub](https://github.com/timescale/pgvectorscale)
+- [pg_textsearch GitHub](https://github.com/timescale/pg_textsearch)
+- [HNSW vs DiskANN](https://www.tigerdata.com/learn/hnsw-vs-diskann)
+- [Hindsight Documentation](https://hindsight.dev)

docker/docker-compose/timescale/docker-compose.yaml

@@ -0,0 +1,108 @@
+name: hindsight
+# Docker Compose file for Hindsight with Timescale extensions
+# - pgvectorscale: DiskANN vector search (disk-based, scalable)
+# - pg_textsearch: BM25 text search (high-performance keyword retrieval)
+#
+# Quick start:
+# docker compose -f docker/docker-compose/timescale/docker-compose.yaml up -d --build
+#
+# Required environment variables:
+# - HINDSIGHT_DB_PASSWORD: Password for the PostgreSQL user
+# - OPENAI_API_KEY (or configure another LLM provider)
+#
+# Optional environment variables with defaults:
+# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
+# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
+# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
+
+services:
+  db:
+    # Custom PostgreSQL image with Timescale extensions (pgvectorscale + pg_textsearch)
+    build:
+      context: .
+      dockerfile: Dockerfile
+    container_name: hindsight-db-timescale
+    restart: always
+    # Expose PostgreSQL port (using 5438 to avoid conflicts with other setups)
+    ports:
+      - "5438:5432"
+    environment:
+      POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
+      POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:-hindsight_password}
+      POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
+    volumes:
+      - pg_data:/var/lib/postgresql/data
+    networks:
+      - hindsight-net
+    # Health check to ensure database is ready
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U hindsight_user"]
+      interval: 5s
+      timeout: 5s
+      retries: 5
+
+  timescale-init:
+    build:
+      context: .
+      dockerfile: Dockerfile
+    depends_on:
+      db:
+        condition: service_healthy
+    environment:
+      - PGPASSWORD=${HINDSIGHT_DB_PASSWORD:-hindsight_password}
+    command: >
+      bash -c "
+        echo 'PostgreSQL is ready - creating hindsight_db database';
+        psql -h hindsight-db-timescale -p 5432 -U hindsight_user -c 'CREATE DATABASE hindsight_db;' 2>/dev/null || echo 'Database already exists';
+        echo 'Installing Timescale extensions...';
+        echo '1/3: Installing pgvector (required by pgvectorscale)...';
+        psql -h hindsight-db-timescale -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vector CASCADE;';
+        echo '2/3: Installing pgvectorscale (DiskANN vector search)...';
+        psql -h hindsight-db-timescale -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vectorscale CASCADE;';
+        echo '3/3: Installing pg_textsearch (BM25 text search)...';
+        psql -h hindsight-db-timescale -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS pg_textsearch CASCADE;';
+        echo '';
+        echo '✅ Timescale extensions installed successfully';
+        echo '';
+        echo 'Installed extensions:';
+        psql -h hindsight-db-timescale -p 5432 -U hindsight_user -d hindsight_db -c \"\\dx\" | grep -E '(vector|vectorscale|pg_textsearch)';
+      "
+    restart: "no"
+    networks:
+      - hindsight-net
+
+  hindsight:
+    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
+    container_name: hindsight-app-timescale
+    ports:
+      - "8888:8888"
+      - "9999:9999"
+    environment:
+      # LLM Configuration
+      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
+      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
+
+      # Database Configuration
+      HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
+
+      # Timescale Extensions
+      # pgvectorscale: DiskANN algorithm for disk-based scalable vector search
+      HINDSIGHT_API_VECTOR_EXTENSION: pgvectorscale
+      # pg_textsearch: High-performance BM25 text search
+      HINDSIGHT_API_TEXT_SEARCH_EXTENSION: pg_textsearch
+
+    depends_on:
+      db:
+        condition: service_healthy
+      timescale-init:
+        condition: service_completed_successfully
+    networks:
+      - hindsight-net
+
+
+networks:
+  hindsight-net:
+    driver: bridge
+
+volumes:
+  pg_data:

docker/docker-compose/vchord/docker-compose.yaml

@@ -0,0 +1,93 @@
+name: hindsight
+# Docker Compose file for Hindsight with PostgreSQL and vectorchord
+# docker compose -f docker/docker-compose/vchord/docker-compose.yaml down && sleep 2 && docker compose -f docker/docker-compose/vchord/docker-compose.yaml up -d
+# Make sure to set the required environment variables before running:
+# - HINDSIGHT_DB_PASSWORD: Password for the PostgreSQL user
+# - Configure LLM provider variables as needed (see below in the hindsight service)
+#
+# Usage:
+# docker compose up -d
+#
+# Optional environment variables with defaults:
+# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
+# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
+# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
+# - HINDSIGHT_DB_VERSION: PostgreSQL version (default: 18)
+
+services:
+  db:
+    # Use a PostgreSQL-Image with vectorchord extension pre-installed
+    image: tensorchord/vchord-suite:pg${HINDSIGHT_DB_VERSION:-18-latest}
+    container_name: hindsight-db
+    restart: always
+    # Expose PostgreSQL port
+    ports:
+      - "5436:5432"
+    environment:
+      POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
+      POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:-hindsight_password}
+      POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
+    volumes:
+      - pg_data:/var/lib/postgresql/${HINDSIGHT_DB_VERSION:-18}/docker
+    networks:
+      - hindsight-net
+
+  vectorchord-init:
+    image: tensorchord/vchord-suite:pg18-latest
+    #container_name: vectorchord-init
+    depends_on:
+      - db
+    environment:
+      - PGPASSWORD=${HINDSIGHT_DB_PASSWORD:-hindsight_password}
+    command: >
+      bash -c "
+        echo 'Waiting for PostgreSQL to be ready...';
+        until pg_isready -h hindsight-db -p 5432 -U hindsight_user; do
+          echo 'PostgreSQL is unavailable - sleeping';
+          sleep 2;
+        done;
+        echo 'PostgreSQL is ready - creating hindsight_db database';
+        psql -h hindsight-db -p 5432 -U hindsight_user -c 'CREATE DATABASE hindsight_db;' 2>/dev/null || echo 'Database already exists';
+        echo 'Creating extensions in hindsight_db database';
+        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vchord CASCADE;';
+        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS pg_tokenizer CASCADE;';
+        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vchord_bm25 CASCADE;';
+        echo 'Creating llmlingua2 tokenizer';
+        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c \"SELECT create_tokenizer('llmlingua2', \\$\\$ model = \\\"llmlingua2\\\" \\$\\$);\" 2>/dev/null || echo 'Tokenizer already exists or creation skipped';
+        echo 'Database and extensions created successfully';
+      "
+    restart: "no"
+    networks:
+      - hindsight-net
+
+  hindsight:
+    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
+    container_name: hindsight-app
+    ports:
+      - "8888:8888"
+      - "9999:9999"
+    environment:
+      # LLM Configuration (uses OpenAI for testing vchord)
+      # LLM configuration
+      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
+      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
+
+      # Database Configuration
+      HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
+
+      # Vector and Text Search Extensions
+      HINDSIGHT_API_VECTOR_EXTENSION: vchord
+      HINDSIGHT_API_TEXT_SEARCH_EXTENSION: vchord
+
+    depends_on:
+      - db
+    networks:
+      - hindsight-net
+
+
+networks:
+  hindsight-net:
+    driver: bridge
+
+volumes:
+  pg_data:

docker/standalone/Dockerfile

@@ -2,16 +2,25 @@
 # Supports building API-only, Control Plane-only, or both
 #
 # Build args:
-#   INCLUDE_API=true/false  - Include API (default: true)
-#   INCLUDE_CP=true/false   - Include Control Plane (default: true)
+#   INCLUDE_API=true/false         - Include API (default: true)
+#   INCLUDE_CP=true/false          - Include Control Plane (default: true)
+#   INCLUDE_LOCAL_MODELS=true/false - Include local ML models for embeddings/reranking (default: true)
+#                                     Set to false when using external providers (TEI, OpenAI, Cohere)
+#   PRELOAD_ML_MODELS=true/false   - Pre-download ML models during build (default: true)
+#                                     Only effective when INCLUDE_LOCAL_MODELS=true
+#                                     NOTE: tiktoken encodings are ALWAYS preloaded (required for air-gapped deployments)
 #
 # Examples:
-#   docker build -t hindsight .                                    # Both (standalone)
-#   docker build -t hindsight-api --build-arg INCLUDE_CP=false .   # API only
-#   docker build -t hindsight-cp --build-arg INCLUDE_API=false .   # Control Plane only
+#   docker build -t hindsight .                                          # Both (standalone)
+#   docker build -t hindsight-api --build-arg INCLUDE_CP=false .         # API only
+#   docker build -t hindsight-cp --build-arg INCLUDE_API=false .         # Control Plane only
+#   docker build -t hindsight --build-arg PRELOAD_ML_MODELS=false .      # Skip ML model preload
+#   docker build -t hindsight --build-arg INCLUDE_LOCAL_MODELS=false .   # Skip local ML deps (for external providers)
 
 ARG INCLUDE_API=true
 ARG INCLUDE_CP=true
+ARG PRELOAD_ML_MODELS=true
+ARG INCLUDE_LOCAL_MODELS=true
 
 # =============================================================================
 # Stage: API Builder
@@ -19,6 +28,7 @@ ARG INCLUDE_CP=true
 FROM python:3.11-slim AS api-builder
 
 ARG INCLUDE_API
+ARG INCLUDE_LOCAL_MODELS
 RUN if [ "$INCLUDE_API" != "true" ]; then echo "Skipping API build" && exit 0; fi
 
 WORKDIR /app
@@ -31,20 +41,43 @@ RUN apt-get update && apt-get install -y \
     && rm -rf /var/lib/apt/lists/* \
     && pip install --no-cache-dir uv
 
-# Copy dependency files and README (required by pyproject.toml)
-COPY hindsight-api/pyproject.toml ./api/
-COPY hindsight-api/README.md ./api/
+# Copy the workspace lock and member metadata before source code so dependency
+# installation stays cacheable while matching the versions tested in CI.
+COPY pyproject.toml uv.lock ./
+COPY hindsight-all/pyproject.toml ./hindsight-all/
+COPY hindsight-api/pyproject.toml ./hindsight-api/
+COPY hindsight-api-slim/pyproject.toml ./api/
+COPY hindsight-api-slim/README.md ./api/
+COPY hindsight-all-slim/pyproject.toml ./hindsight-all-slim/
+COPY hindsight-dev/pyproject.toml ./hindsight-dev/
+COPY hindsight-clients/python/pyproject.toml ./hindsight-clients/python/
+COPY hindsight-embed/pyproject.toml ./hindsight-embed/
+RUN ln -s api hindsight-api-slim
 
-WORKDIR /app/api
-
-# Sync dependencies (will create lock file if needed)
-RUN uv sync
+# Sync dependencies using appropriate extras based on INCLUDE_LOCAL_MODELS
+# local-ml: torch, sentence-transformers, transformers, einops, flashrank, mlx (optional)
+# embedded-db: pg0-embedded (always included for embedded PostgreSQL support)
+# ONNX Runtime embeddings are intentionally not bundled into the official
+# standalone image; install the local-onnx extra in custom images when needed.
+ENV UV_PROJECT_ENVIRONMENT=/app/api/.venv
+RUN if [ "$INCLUDE_LOCAL_MODELS" = "true" ]; then \
+        uv sync --locked --package hindsight-api-slim --no-install-package hindsight-api-slim --extra local-ml --extra embedded-db; \
+    else \
+        uv sync --locked --package hindsight-api-slim --no-install-package hindsight-api-slim --extra embedded-db; \
+    fi
 
 # Copy source code (alembic migrations are inside hindsight_api/)
-COPY hindsight-api/hindsight_api ./hindsight_api
+WORKDIR /app/api
+COPY hindsight-api-slim/hindsight_api ./hindsight_api
 
-# Install the local package (uv sync only installed dependencies, not the package itself)
-RUN uv pip install -e .
+# Install the local package from the same validated lock after source is present.
+WORKDIR /app
+RUN if [ "$INCLUDE_LOCAL_MODELS" = "true" ]; then \
+        uv sync --locked --package hindsight-api-slim --extra local-ml --extra embedded-db; \
+    else \
+        uv sync --locked --package hindsight-api-slim --extra embedded-db; \
+    fi \
+    && uv pip check --python /app/api/.venv/bin/python
 
 # =============================================================================
 # Stage: SDK Builder (needed for Control Plane)
@@ -54,13 +87,15 @@ FROM node:20-slim AS sdk-builder
 ARG INCLUDE_CP
 RUN if [ "$INCLUDE_CP" != "true" ]; then echo "Skipping SDK build" && exit 0; fi
 
-WORKDIR /app/sdk
+WORKDIR /app
 
-COPY hindsight-clients/typescript/package*.json ./
-RUN npm ci
+# Copy root package files for npm workspaces
+COPY package.json package-lock.json ./
+COPY hindsight-clients/typescript/ ./hindsight-clients/typescript/
 
-COPY hindsight-clients/typescript/ ./
-RUN npm run build
+# Install and build SDK using workspace (--ignore-scripts skips git hooks setup)
+RUN npm ci --ignore-scripts -w @vectorize-io/hindsight-client
+RUN npm run build -w @vectorize-io/hindsight-client
 
 # =============================================================================
 # Stage: Control Plane Builder
@@ -70,30 +105,52 @@ FROM node:20-slim AS cp-builder
 ARG INCLUDE_CP
 RUN if [ "$INCLUDE_CP" != "true" ]; then echo "Skipping CP build" && exit 0; fi
 
-WORKDIR /app
-
-# Copy built SDK
-COPY --from=sdk-builder /app/sdk /app/sdk
+# Create directory structure matching the monorepo layout
+# This is required because build:standalone script expects .next/standalone/memory-poc/hindsight-control-plane
+WORKDIR /app/memory-poc/hindsight-control-plane
 
 # Install Control Plane dependencies
 # Only copy package.json (not package-lock.json) to ensure npm installs
 # correct platform-specific native bindings for lightningcss/tailwindcss
 COPY hindsight-control-plane/package.json ./
+# Remove the file: dependency on SDK (we'll copy it directly later)
+RUN sed -i '/"@vectorize-io\/hindsight-client":/d' package.json
 RUN npm install
 
 # Copy Control Plane source (excluding node_modules via .dockerignore)
 COPY hindsight-control-plane/ ./
 # Remove package-lock.json to avoid conflicts with installed native bindings
-RUN rm -f package-lock.json
+# Also remove the file: dependency from package.json (restored by COPY above)
+RUN rm -f package-lock.json && sed -i '/"@vectorize-io\/hindsight-client":/d' package.json
 
-# Link SDK (temporary for build)
-RUN cd /app/sdk && npm link && cd /app && npm link @vectorize-io/hindsight-client
+# Copy built SDK directly into node_modules (more reliable than npm link in Docker)
+COPY --from=sdk-builder /app/hindsight-clients/typescript ./node_modules/@vectorize-io/hindsight-client
 
-# Build Control Plane
-RUN npm run build
+# Accept base path as build argument for reverse proxy deployments
+# Usage: docker build --build-arg NEXT_PUBLIC_BASE_PATH=/hindsight ...
+ARG NEXT_PUBLIC_BASE_PATH=""
 
-# Create public directory if it doesn't exist
-RUN mkdir -p public
+# Build Control Plane - run next build first, then custom standalone copy
+# (The build:standalone script expects a specific path structure that differs in Docker)
+RUN npm exec -- next build
+
+# Create standalone directory structure manually
+# Note: Must exclude node_modules from find to avoid wrong server.js from next/dist/experimental/testmode/
+# Note: Must explicitly copy .next since glob * doesn't match hidden directories
+RUN STANDALONE_ROOT=$(find .next/standalone -path '*/node_modules' -prune -o -name 'server.js' -print | head -1 | xargs dirname) && \
+    mkdir -p standalone && \
+    cp -r "$STANDALONE_ROOT"/* standalone/ && \
+    cp -r "$STANDALONE_ROOT"/.next standalone/.next && \
+    # Copy node_modules if separate from app dir (monorepo structure)
+    if [ -d ".next/standalone/node_modules" ] && [ "$STANDALONE_ROOT" != ".next/standalone" ]; then \
+      cp -r .next/standalone/node_modules standalone/node_modules; \
+    fi && \
+    cp -r .next/static standalone/.next/static && \
+    mkdir -p standalone/public && \
+    cp -r public/* standalone/public/ 2>/dev/null || true && \
+    # Verify required files exist
+    test -f standalone/server.js || (echo "ERROR: server.js missing!" && exit 1) && \
+    test -f standalone/.next/BUILD_ID || (echo "ERROR: BUILD_ID missing!" && exit 1)
 
 # =============================================================================
 # Stage: Final Image - API Only
@@ -102,18 +159,21 @@ FROM python:3.11-slim AS api-only
 
 WORKDIR /app
 
-# Install pg0 dependencies
+# Note: libicu version varies by Debian version - try common versions in order
+# Runtime images use uv directly; remove pip build tooling after installation so
+# vulnerable setuptools-vendored packages and wheel are not shipped in production.
 RUN apt-get update && apt-get install -y \
     curl \
+    procps \
     libxml2 \
     libssl3 \
     libgssapi-krb5-2 \
     libossp-uuid16 \
-    && apt-get install -y libicu72 || apt-get install -y libicu74 || apt-get install -y libicu* \
+    && (apt-get install -y libicu72 2>/dev/null || apt-get install -y libicu74 2>/dev/null || apt-get install -y libicu76 2>/dev/null || true) \
     && rm -rf /var/lib/apt/lists/* \
-    && pip install --no-cache-dir uv
+    && pip install --no-cache-dir uv \
+    && pip uninstall --yes setuptools wheel
 
-# Create non-root user (PostgreSQL cannot run as root)
 RUN useradd -m -s /bin/bash hindsight
 
 # Copy API with virtual environment from builder
@@ -123,56 +183,74 @@ COPY --from=api-builder /app/api /app/api
 COPY docker/standalone/start-all.sh /app/start-all.sh
 RUN chmod +x /app/start-all.sh
 
-# Create data directory for pg0 and set ownership
-RUN mkdir -p /app/data && chown -R hindsight:hindsight /app
+RUN chown -R hindsight:hindsight /app
 
-# Switch to non-root user
 USER hindsight
 
-# Set PATH for hindsight user
-ENV PATH="/home/hindsight/.hindsight/bin:/app/api/.venv/bin:${PATH}"
+# Create pg0 data directory as hindsight user so that Docker seeds new named
+# volumes with correct ownership (UID 1000) on first use, avoiding the
+# "Permission denied" error when mounting a fresh root-owned volume.
+RUN mkdir -p /home/hindsight/.pg0
 
-# Install pg0 binary
-RUN mkdir -p /home/hindsight/.hindsight/bin && \
-    ARCH=$(uname -m) && \
-    if [ "$ARCH" = "aarch64" ] || [ "$ARCH" = "arm64" ]; then \
-        PG0_BINARY="pg0-linux-aarch64-gnu"; \
-    elif [ "$ARCH" = "x86_64" ]; then \
-        PG0_BINARY="pg0-linux-x86_64-gnu"; \
-    else \
-        echo "Unsupported architecture: $ARCH" && exit 1; \
-    fi && \
-    echo "Installing pg0 binary: $PG0_BINARY" && \
-    for i in 1 2 3 4 5; do \
-        curl -fsSL -o /home/hindsight/.hindsight/bin/pg0 \
-            "https://github.com/vectorize-io/pg0/releases/latest/download/$PG0_BINARY" && \
-        chmod +x /home/hindsight/.hindsight/bin/pg0 && \
-        ls -lh /home/hindsight/.hindsight/bin/pg0 && \
-        file /home/hindsight/.hindsight/bin/pg0 && \
-        ldd /home/hindsight/.hindsight/bin/pg0 2>&1 || true && \
-        break || (echo "Retry $i failed, waiting..." && sleep 10); \
-    done && \
-    echo "Testing pg0 binary..." && \
-    /home/hindsight/.hindsight/bin/pg0 --version || (echo "pg0 --version failed with exit code $?"; ldd /home/hindsight/.hindsight/bin/pg0; exit 1)
+# Make /home/hindsight traversable when running with --user UID:GID overrides
+# (default 0700 blocks traversal by non-owner UIDs needed for bind-mount ownership matching)
+RUN chmod 755 /home/hindsight
 
-# Pre-download PostgreSQL binaries
-ENV PG0_HOME=/home/hindsight/.pg0-cache
-RUN pg0 start --help && \
-    (pg0 start --name hindsight --port 5555 --username hindsight --password hindsight --database hindsight && \
-    sleep 2 && \
-    pg0 stop --name hindsight && \
-    echo "PostgreSQL pre-cached to $PG0_HOME") || echo "Pre-download skipped"
+ENV PATH="/app/api/.venv/bin:${PATH}"
 
-ENV PG0_HOME=/home/hindsight/.pg0
+# Pre-download tiktoken encoding (ALWAYS - required for token counting even in air-gapped envs)
+# Tiktoken is a core runtime dependency, not an optional ML model
+RUN MAX_RETRIES=3; \
+    RETRY_DELAY=5; \
+    for i in $(seq 1 $MAX_RETRIES); do \
+      echo "Attempt $i/$MAX_RETRIES: Downloading tiktoken encoding..."; \
+      /app/api/.venv/bin/python -c "\
+import tiktoken; \
+print('Downloading cl100k_base encoding...'); \
+tiktoken.get_encoding('cl100k_base'); \
+print('Tiktoken encoding cached successfully')" && break; \
+      if [ $i -lt $MAX_RETRIES ]; then \
+        echo "Attempt $i failed, retrying in ${RETRY_DELAY}s..."; \
+        sleep $RETRY_DELAY; \
+        RETRY_DELAY=$((RETRY_DELAY * 2)); \
+      fi; \
+    done; \
+    if [ $i -eq $MAX_RETRIES ]; then \
+      echo "ERROR: Failed to download tiktoken encoding after $MAX_RETRIES attempts"; \
+      exit 1; \
+    fi
 
-# Pre-download ML models to avoid runtime download
-RUN /app/api/.venv/bin/python -c "\
+# Pre-download ML models to avoid runtime download (conditional)
+# Only runs if both PRELOAD_ML_MODELS=true AND INCLUDE_LOCAL_MODELS=true
+# Includes retry logic with exponential backoff for transient network failures
+ARG PRELOAD_ML_MODELS
+ARG INCLUDE_LOCAL_MODELS
+ENV HF_HUB_DOWNLOAD_TIMEOUT=600
+RUN if [ "$PRELOAD_ML_MODELS" = "true" ] && [ "$INCLUDE_LOCAL_MODELS" = "true" ]; then \
+    MAX_RETRIES=3; \
+    RETRY_DELAY=10; \
+    for i in $(seq 1 $MAX_RETRIES); do \
+      echo "Attempt $i/$MAX_RETRIES: Downloading ML models..."; \
+      /app/api/.venv/bin/python -c "\
+import os; os.environ['HF_HUB_DOWNLOAD_TIMEOUT'] = '600'; \
 from sentence_transformers import SentenceTransformer, CrossEncoder; \
 print('Downloading embedding model...'); \
 SentenceTransformer('BAAI/bge-small-en-v1.5'); \
 print('Downloading cross-encoder model...'); \
 CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2'); \
-print('Models cached successfully')"
+print('Models cached successfully')" && break; \
+      if [ $i -lt $MAX_RETRIES ]; then \
+        echo "Attempt $i failed, retrying in ${RETRY_DELAY}s..."; \
+        sleep $RETRY_DELAY; \
+        RETRY_DELAY=$((RETRY_DELAY * 2)); \
+      fi; \
+    done; \
+    if [ $i -eq $MAX_RETRIES ] && ! /app/api/.venv/bin/python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('BAAI/bge-small-en-v1.5')" 2>/dev/null; then \
+      echo "ERROR: Failed to download models after $MAX_RETRIES attempts"; \
+      exit 1; \
+    fi; \
+    elif [ "$INCLUDE_LOCAL_MODELS" != "true" ]; then echo "Skipping ML model preload (local-models not included)"; \
+    else echo "Skipping ML model preload"; fi
 
 EXPOSE 8888
 
@@ -182,6 +260,10 @@ ENV HINDSIGHT_API_LOG_LEVEL=info
 ENV HINDSIGHT_ENABLE_API=true
 ENV HINDSIGHT_ENABLE_CP=false
 ENV PYTHONUNBUFFERED=1
+# Suppress verbose transformers/HuggingFace model loading warnings
+ENV TRANSFORMERS_VERBOSITY=error
+ENV HF_HUB_VERBOSITY=error
+ENV TOKENIZERS_PARALLELISM=false
 
 CMD ["/app/start-all.sh"]
 
@@ -193,13 +275,13 @@ FROM node:20-alpine AS cp-only
 WORKDIR /app
 
 # Copy built SDK
-COPY --from=sdk-builder /app/sdk /app/sdk
+COPY --from=sdk-builder /app/hindsight-clients/typescript /app/sdk
 
 # Copy Control Plane standalone build
 WORKDIR /app/control-plane
-COPY --from=cp-builder /app/.next/standalone ./
-COPY --from=cp-builder /app/.next/static ./.next/static
-COPY --from=cp-builder /app/public ./public
+COPY --from=cp-builder /app/memory-poc/hindsight-control-plane/standalone ./
+COPY --from=cp-builder /app/memory-poc/hindsight-control-plane/.next/static ./.next/static
+COPY --from=cp-builder /app/memory-poc/hindsight-control-plane/public ./public
 
 WORKDIR /app
 
@@ -226,33 +308,37 @@ FROM python:3.11-slim AS standalone
 
 WORKDIR /app
 
-# Install Node.js, curl, uv, and pg0 dependencies
+# Install Node.js, curl, uv, and system dependencies
+# Note: libicu version varies by Debian version - try common versions in order
+# Runtime images use uv directly; remove pip build tooling after installation so
+# vulnerable setuptools-vendored packages and wheel are not shipped in production.
 RUN apt-get update && apt-get install -y \
     curl \
+    procps \
     libxml2 \
     libssl3 \
     libgssapi-krb5-2 \
     libossp-uuid16 \
-    && apt-get install -y libicu72 || apt-get install -y libicu74 || apt-get install -y libicu* \
+    && (apt-get install -y libicu72 2>/dev/null || apt-get install -y libicu74 2>/dev/null || apt-get install -y libicu76 2>/dev/null || true) \
     && curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \
     && apt-get install -y nodejs \
     && rm -rf /var/lib/apt/lists/* \
-    && pip install --no-cache-dir uv
+    && pip install --no-cache-dir uv \
+    && pip uninstall --yes setuptools wheel
 
-# Create non-root user (PostgreSQL cannot run as root)
 RUN useradd -m -s /bin/bash hindsight
 
 # Copy API with virtual environment from builder
 COPY --from=api-builder /app/api /app/api
 
 # Copy built SDK
-COPY --from=sdk-builder /app/sdk /app/sdk
+COPY --from=sdk-builder /app/hindsight-clients/typescript /app/sdk
 
 # Copy Control Plane standalone build
 WORKDIR /app/control-plane
-COPY --from=cp-builder /app/.next/standalone ./
-COPY --from=cp-builder /app/.next/static ./.next/static
-COPY --from=cp-builder /app/public ./public
+COPY --from=cp-builder /app/memory-poc/hindsight-control-plane/standalone ./
+COPY --from=cp-builder /app/memory-poc/hindsight-control-plane/.next/static ./.next/static
+COPY --from=cp-builder /app/memory-poc/hindsight-control-plane/public ./public
 
 WORKDIR /app
 
@@ -260,56 +346,74 @@ WORKDIR /app
 COPY docker/standalone/start-all.sh /app/start-all.sh
 RUN chmod +x /app/start-all.sh
 
-# Create data directory for pg0 and set ownership
-RUN mkdir -p /app/data && chown -R hindsight:hindsight /app
+RUN chown -R hindsight:hindsight /app
 
-# Switch to non-root user
 USER hindsight
 
-# Set PATH for hindsight user
-ENV PATH="/home/hindsight/.hindsight/bin:/app/api/.venv/bin:${PATH}"
+# Create pg0 data directory as hindsight user so that Docker seeds new named
+# volumes with correct ownership (UID 1000) on first use, avoiding the
+# "Permission denied" error when mounting a fresh root-owned volume.
+RUN mkdir -p /home/hindsight/.pg0
 
-# Install pg0 binary
-RUN mkdir -p /home/hindsight/.hindsight/bin && \
-    ARCH=$(uname -m) && \
-    if [ "$ARCH" = "aarch64" ] || [ "$ARCH" = "arm64" ]; then \
-        PG0_BINARY="pg0-linux-aarch64-gnu"; \
-    elif [ "$ARCH" = "x86_64" ]; then \
-        PG0_BINARY="pg0-linux-x86_64-gnu"; \
-    else \
-        echo "Unsupported architecture: $ARCH" && exit 1; \
-    fi && \
-    echo "Installing pg0 binary: $PG0_BINARY" && \
-    for i in 1 2 3 4 5; do \
-        curl -fsSL -o /home/hindsight/.hindsight/bin/pg0 \
-            "https://github.com/vectorize-io/pg0/releases/latest/download/$PG0_BINARY" && \
-        chmod +x /home/hindsight/.hindsight/bin/pg0 && \
-        ls -lh /home/hindsight/.hindsight/bin/pg0 && \
-        file /home/hindsight/.hindsight/bin/pg0 && \
-        ldd /home/hindsight/.hindsight/bin/pg0 2>&1 || true && \
-        break || (echo "Retry $i failed, waiting..." && sleep 10); \
-    done && \
-    echo "Testing pg0 binary..." && \
-    /home/hindsight/.hindsight/bin/pg0 --version || (echo "pg0 --version failed with exit code $?"; ldd /home/hindsight/.hindsight/bin/pg0; exit 1)
+# Make /home/hindsight traversable when running with --user UID:GID overrides
+# (default 0700 blocks traversal by non-owner UIDs needed for bind-mount ownership matching)
+RUN chmod 755 /home/hindsight
 
-# Pre-download PostgreSQL binaries
-ENV PG0_HOME=/home/hindsight/.pg0-cache
-RUN pg0 start --help && \
-    (pg0 start --name hindsight --port 5555 --username hindsight --password hindsight --database hindsight && \
-    sleep 2 && \
-    pg0 stop --name hindsight && \
-    echo "PostgreSQL pre-cached to $PG0_HOME") || echo "Pre-download skipped"
+ENV PATH="/app/api/.venv/bin:${PATH}"
 
-ENV PG0_HOME=/home/hindsight/.pg0
+# Pre-download tiktoken encoding (ALWAYS - required for token counting even in air-gapped envs)
+# Tiktoken is a core runtime dependency, not an optional ML model
+RUN MAX_RETRIES=3; \
+    RETRY_DELAY=5; \
+    for i in $(seq 1 $MAX_RETRIES); do \
+      echo "Attempt $i/$MAX_RETRIES: Downloading tiktoken encoding..."; \
+      /app/api/.venv/bin/python -c "\
+import tiktoken; \
+print('Downloading cl100k_base encoding...'); \
+tiktoken.get_encoding('cl100k_base'); \
+print('Tiktoken encoding cached successfully')" && break; \
+      if [ $i -lt $MAX_RETRIES ]; then \
+        echo "Attempt $i failed, retrying in ${RETRY_DELAY}s..."; \
+        sleep $RETRY_DELAY; \
+        RETRY_DELAY=$((RETRY_DELAY * 2)); \
+      fi; \
+    done; \
+    if [ $i -eq $MAX_RETRIES ]; then \
+      echo "ERROR: Failed to download tiktoken encoding after $MAX_RETRIES attempts"; \
+      exit 1; \
+    fi
 
-# Pre-download ML models to avoid runtime download
-RUN /app/api/.venv/bin/python -c "\
+# Pre-download ML models to avoid runtime download (conditional)
+# Only runs if both PRELOAD_ML_MODELS=true AND INCLUDE_LOCAL_MODELS=true
+# Includes retry logic with exponential backoff for transient network failures
+ARG PRELOAD_ML_MODELS
+ARG INCLUDE_LOCAL_MODELS
+ENV HF_HUB_DOWNLOAD_TIMEOUT=600
+RUN if [ "$PRELOAD_ML_MODELS" = "true" ] && [ "$INCLUDE_LOCAL_MODELS" = "true" ]; then \
+    MAX_RETRIES=3; \
+    RETRY_DELAY=10; \
+    for i in $(seq 1 $MAX_RETRIES); do \
+      echo "Attempt $i/$MAX_RETRIES: Downloading ML models..."; \
+      /app/api/.venv/bin/python -c "\
+import os; os.environ['HF_HUB_DOWNLOAD_TIMEOUT'] = '600'; \
 from sentence_transformers import SentenceTransformer, CrossEncoder; \
 print('Downloading embedding model...'); \
 SentenceTransformer('BAAI/bge-small-en-v1.5'); \
 print('Downloading cross-encoder model...'); \
 CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2'); \
-print('Models cached successfully')"
+print('Models cached successfully')" && break; \
+      if [ $i -lt $MAX_RETRIES ]; then \
+        echo "Attempt $i failed, retrying in ${RETRY_DELAY}s..."; \
+        sleep $RETRY_DELAY; \
+        RETRY_DELAY=$((RETRY_DELAY * 2)); \
+      fi; \
+    done; \
+    if [ $i -eq $MAX_RETRIES ] && ! /app/api/.venv/bin/python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('BAAI/bge-small-en-v1.5')" 2>/dev/null; then \
+      echo "ERROR: Failed to download models after $MAX_RETRIES attempts"; \
+      exit 1; \
+    fi; \
+    elif [ "$INCLUDE_LOCAL_MODELS" != "true" ]; then echo "Skipping ML model preload (local-models not included)"; \
+    else echo "Skipping ML model preload"; fi
 
 EXPOSE 8888 9999
 
@@ -321,6 +425,10 @@ ENV HINDSIGHT_CP_DATAPLANE_API_URL=http://localhost:8888
 ENV HINDSIGHT_ENABLE_API=true
 ENV HINDSIGHT_ENABLE_CP=true
 ENV PYTHONUNBUFFERED=1
+# Suppress verbose transformers/HuggingFace model loading warnings
+ENV TRANSFORMERS_VERBOSITY=error
+ENV HF_HUB_VERBOSITY=error
+ENV TOKENIZERS_PARALLELISM=false
 
 CMD ["/app/start-all.sh"]
 

docker/standalone/start-all.sh

@@ -1,39 +1,258 @@
 #!/bin/bash
 set -e
 
+# =============================================================================
+# Embedded pg0 data integrity check (#675)
+#
+# When using embedded pg0, check if the data directory has existing PostgreSQL
+# data before starting. If the directory exists but appears empty/corrupt
+# (e.g., missing PG_VERSION file), log a warning. This helps diagnose data
+# loss scenarios where a container restart caused the data directory to be
+# wiped despite a volume mount being present.
+# =============================================================================
+pg0_has_pg_version() {
+    local pg0_data_dir="$1"
+
+    # pg0 has used more than one on-disk layout. Newer standalone images keep
+    # PostgreSQL data under instances/<name>/data, while older volumes may have
+    # placed PG_VERSION at or one level below the mount.
+    [ -f "$pg0_data_dir/PG_VERSION" ] && return 0
+    compgen -G "$pg0_data_dir"/*/PG_VERSION > /dev/null 2>&1 && return 0
+    compgen -G "$pg0_data_dir"/instances/*/data/PG_VERSION > /dev/null 2>&1 && return 0
+
+    return 1
+}
+
+check_pg0_data_integrity() {
+    local pg0_data_dir="$1"
+
+    if [ ! -d "$pg0_data_dir" ]; then
+        return 0
+    fi
+
+    # Look for actual PostgreSQL data directories (pg0 creates subdirs per instance)
+    if pg0_has_pg_version "$pg0_data_dir"; then
+        echo "✅ Existing pg0 data directory detected at $pg0_data_dir"
+    elif [ "$(ls -A "$pg0_data_dir" 2>/dev/null)" ]; then
+        echo "⚠️  WARNING: pg0 data directory exists at $pg0_data_dir but no PG_VERSION found."
+        echo "   This may indicate data corruption or an incomplete previous shutdown."
+        echo "   If you see all migrations running from scratch after this, your data may have been lost."
+        echo "   See: https://github.com/vectorize-io/hindsight/issues/675"
+    fi
+
+    return 0
+}
+
+# =============================================================================
+# Embedded pg0 writability pre-check (#1483)
+#
+# The container runs as the unprivileged `hindsight` user (UID 1000). When the
+# pg0 data directory is a host bind mount (e.g. `-v $HOME/dir:/home/hindsight/.pg0`)
+# that is not owned by UID 1000 — the default on macOS Docker Desktop and most
+# non-1000 Linux hosts — pg0 fails with the opaque "Permission denied (os error
+# 13)". We cannot chown it ourselves without root (and the image is deliberately
+# rootless), so we surface an actionable message up front instead.
+#
+# Docker *named* volumes are seeded with the image directory's ownership (UID
+# 1000) on first use, so they avoid this entirely — hence the named-volume
+# recommendation below and in the README.
+# =============================================================================
+check_pg0_writable() {
+    local pg0_data_dir="$1"
+
+    # Only relevant for embedded pg0; an external database doesn't use this dir.
+    if [ -n "${HINDSIGHT_API_DATABASE_URL:-}" ]; then
+        return 0
+    fi
+
+    mkdir -p "$pg0_data_dir" 2>/dev/null || true
+    if touch "$pg0_data_dir/.hindsight-write-test" 2>/dev/null; then
+        rm -f "$pg0_data_dir/.hindsight-write-test" 2>/dev/null || true
+        return 0
+    fi
+
+    echo "❌ The embedded database directory $pg0_data_dir is not writable by this container (UID $(id -u))."
+    echo ""
+    echo "   A host directory was bind-mounted but is not owned by the container user (UID 1000)."
+    echo "   Hindsight runs rootless and cannot fix this for you. Choose one:"
+    echo ""
+    echo "   • Recommended — use a Docker named volume (auto-owned by the container):"
+    echo "       -v hindsight-data:/home/hindsight/.pg0"
+    echo ""
+    echo "   • Or keep the host path and run as your host user, chowning it to match:"
+    echo "       sudo chown -R \$(id -u):\$(id -g) <host-directory>"
+    echo "       docker run --user \$(id -u):\$(id -g) -e HOME=/home/hindsight ..."
+    echo ""
+    echo "   See https://github.com/vectorize-io/hindsight/issues/1483"
+    return 1
+}
+
+if [ "${HINDSIGHT_START_ALL_SOURCE_ONLY:-false}" = "true" ]; then
+    return 0 2>/dev/null || exit 0
+fi
+
+check_pg0_data_integrity "${HOME}/.pg0"
+check_pg0_writable "${HOME}/.pg0" || exit 1
+
 # Service flags (default to true if not set)
 ENABLE_API="${HINDSIGHT_ENABLE_API:-true}"
 ENABLE_CP="${HINDSIGHT_ENABLE_CP:-true}"
 
-# Copy pre-cached PostgreSQL data if runtime directory is empty (first run with volume)
-if [ "$ENABLE_API" = "true" ]; then
-    PG0_CACHE="/home/hindsight/.pg0-cache"
-    PG0_HOME="/home/hindsight/.pg0"
-    if [ -d "$PG0_CACHE" ] && [ "$(ls -A $PG0_CACHE 2>/dev/null)" ]; then
-        if [ ! "$(ls -A $PG0_HOME 2>/dev/null)" ]; then
-            echo "📦 Copying pre-cached PostgreSQL data..."
-            cp -r "$PG0_CACHE"/* "$PG0_HOME"/ 2>/dev/null || true
-        fi
+# =============================================================================
+# Dependency waiting (opt-in via HINDSIGHT_WAIT_FOR_DEPS=true)
+#
+# Problem: When running with LM Studio, the LLM may take time to load models.
+# If Hindsight starts before LM Studio is ready, it fails on LLM verification.
+# This wait loop ensures dependencies are ready before starting.
+# =============================================================================
+if [ "${HINDSIGHT_WAIT_FOR_DEPS:-false}" = "true" ]; then
+    LLM_BASE_URL="${HINDSIGHT_API_LLM_BASE_URL:-http://host.docker.internal:1234/v1}"
+    MAX_RETRIES="${HINDSIGHT_RETRY_MAX:-0}"  # 0 = infinite
+    RETRY_INTERVAL="${HINDSIGHT_RETRY_INTERVAL:-10}"
+
+    # Check if external database is configured (skip check for embedded pg0)
+    SKIP_DB_CHECK=false
+    if [ -z "${HINDSIGHT_API_DATABASE_URL}" ]; then
+        SKIP_DB_CHECK=true
+    else
+        DB_CHECK_HOST=$(echo "$HINDSIGHT_API_DATABASE_URL" | sed -E 's|.*@([^:/]+):([0-9]+)/.*|\1 \2|')
     fi
+
+    check_db() {
+        if $SKIP_DB_CHECK; then
+            return 0
+        fi
+        if command -v pg_isready &> /dev/null; then
+            pg_isready -h $(echo $DB_CHECK_HOST | cut -d' ' -f1) -p $(echo $DB_CHECK_HOST | cut -d' ' -f2) &>/dev/null
+        else
+            python3 -c "import socket; s=socket.socket(); s.settimeout(5); exit(0 if s.connect_ex(('$(echo $DB_CHECK_HOST | cut -d' ' -f1)', $(echo $DB_CHECK_HOST | cut -d' ' -f2))) == 0 else 1)" 2>/dev/null
+        fi
+    }
+
+    check_llm() {
+        curl -sf "${LLM_BASE_URL}/models" --connect-timeout 5 &>/dev/null
+    }
+
+    echo "⏳ Waiting for dependencies to be ready..."
+    attempt=1
+
+    while true; do
+        db_ok=false
+        llm_ok=false
+
+        if check_db; then
+            db_ok=true
+        fi
+
+        if check_llm; then
+            llm_ok=true
+        fi
+
+        if $db_ok && $llm_ok; then
+            echo "✅ Dependencies ready!"
+            break
+        fi
+
+        if [ "$MAX_RETRIES" -ne 0 ] && [ "$attempt" -ge "$MAX_RETRIES" ]; then
+            echo "❌ Max retries ($MAX_RETRIES) reached. Dependencies not available."
+            exit 1
+        fi
+
+        echo "   Attempt $attempt: DB=$( $db_ok && echo 'ok' || echo 'waiting' ), LLM=$( $llm_ok && echo 'ok' || echo 'waiting' )"
+        sleep "$RETRY_INTERVAL"
+        ((attempt++))
+    done
 fi
 
+# =============================================================================
+# Graceful shutdown handler (#675)
+#
+# Docker sends SIGTERM on `docker stop`/`docker restart`. Without a trap, child
+# processes (hindsight-api + pg0, control-plane) are killed abruptly. For the
+# embedded pg0 database this can cause data loss when the data directory is on
+# a Docker volume that gets remounted after restart.
+#
+# The trap forwards SIGTERM to all tracked child PIDs so that:
+#   - hindsight-api receives the signal and can run its shutdown hooks
+#   - pg0 gets a clean PostgreSQL shutdown (checkpoint + WAL flush)
+#   - The control-plane Node.js process exits cleanly
+# =============================================================================
+# Guard against concurrent cleanup (e.g., child crash + SIGTERM arriving together)
+SHUTTING_DOWN=false
+
+cleanup() {
+    if $SHUTTING_DOWN; then return; fi
+    SHUTTING_DOWN=true
+
+    echo ""
+    echo "🛑 Received shutdown signal, stopping services gracefully..."
+    for pid in "${PIDS[@]}"; do
+        if kill -0 "$pid" 2>/dev/null; then
+            kill -TERM "$pid" 2>/dev/null
+        fi
+    done
+    # Give processes time to shut down cleanly (pg0 needs to flush WAL).
+    # NOTE: Docker's default stop_grace_period is 10s. If you use the default,
+    # either set stop_grace_period: 30s in your compose file / docker stop -t 30,
+    # or Docker will SIGKILL the container before this timeout expires.
+    local timeout=30
+    for ((i=1; i<=timeout; i++)); do
+        local all_stopped=true
+        for pid in "${PIDS[@]}"; do
+            if kill -0 "$pid" 2>/dev/null; then
+                all_stopped=false
+                break
+            fi
+        done
+        if $all_stopped; then
+            echo "✅ All services stopped cleanly"
+            exit 0
+        fi
+        sleep 1
+    done
+    # Force kill if still running after timeout
+    echo "⚠️  Timeout reached, forcing shutdown..."
+    for pid in "${PIDS[@]}"; do
+        if kill -0 "$pid" 2>/dev/null; then
+            kill -9 "$pid" 2>/dev/null
+        fi
+    done
+    exit 1
+}
+trap cleanup SIGTERM SIGINT
+
 # Track PIDs for wait
 PIDS=()
 
 # Start API if enabled
 if [ "$ENABLE_API" = "true" ]; then
     cd /app/api
-    hindsight-api 2>&1 | sed -u 's/^/[api] /' &
+    API_HEALTH_URL="${HINDSIGHT_API_HEALTH_URL:-http://localhost:${HINDSIGHT_API_PORT:-8888}/health}"
+    API_STARTUP_WAIT_SECONDS="${HINDSIGHT_API_STARTUP_WAIT_SECONDS:-300}"
+
+    # Run API directly - Python's PYTHONUNBUFFERED=1 handles output buffering
+    hindsight-api &
     API_PID=$!
     PIDS+=($API_PID)
 
     # Wait for API to be ready
-    for i in {1..60}; do
-        if curl -sf http://localhost:8888/health &>/dev/null; then
+    api_ready=false
+    for ((i=1; i<=API_STARTUP_WAIT_SECONDS; i++)); do
+        if ! kill -0 "$API_PID" 2>/dev/null; then
+            wait "$API_PID"
+            exit $?
+        fi
+        if curl -sf "$API_HEALTH_URL" &>/dev/null; then
+            api_ready=true
             break
         fi
         sleep 1
     done
+
+    if [ "$api_ready" != "true" ]; then
+        echo "❌ API did not become healthy within ${API_STARTUP_WAIT_SECONDS}s"
+        exit 1
+    fi
 else
     echo "API disabled (HINDSIGHT_ENABLE_API=false)"
 fi
@@ -42,7 +261,8 @@ fi
 if [ "$ENABLE_CP" = "true" ]; then
     echo "🎛️  Starting Control Plane..."
     cd /app/control-plane
-    PORT=9999 node server.js 2>&1 | grep -v -E "^[[:space:]]*(▲|✓|-|$)" | sed -u 's/^/[control-plane] /' &
+    export HOSTNAME="${HINDSIGHT_CP_HOSTNAME:-0.0.0.0}"
+    PORT="${HINDSIGHT_CP_PORT:-9999}" node server.js &
     CP_PID=$!
     PIDS+=($CP_PID)
 else
@@ -55,7 +275,7 @@ echo "✅ Hindsight is running!"
 echo ""
 echo "📍 Access:"
 if [ "$ENABLE_CP" = "true" ]; then
-    echo "   Control Plane: http://localhost:9999"
+    echo "   Control Plane: http://localhost:${HINDSIGHT_CP_PORT:-9999}"
 fi
 if [ "$ENABLE_API" = "true" ]; then
     echo "   API:           http://localhost:8888"
@@ -68,8 +288,21 @@ if [ ${#PIDS[@]} -eq 0 ]; then
     exit 1
 fi
 
-# Wait for any process to exit
-wait -n
-
-# Exit with status of first exited process
-exit $?
+# Wait for any process to exit (use wait -n with trap-safe loop)
+while true; do
+    # wait -n returns when any child exits; it also returns on signal delivery
+    # (the trap handler will run and exit, so this loop is just for robustness).
+    # `&& true` prevents `set -e` from killing the script when wait -n returns
+    # non-zero (child exited with error or no backgrounded children remain).
+    wait -n && true
+    # Check if any tracked PID has exited
+    for pid in "${PIDS[@]}"; do
+        if ! kill -0 "$pid" 2>/dev/null; then
+            wait "$pid" 2>/dev/null
+            exit_code=$?
+            echo "⚠️  Service (PID $pid) exited with code $exit_code"
+            # Trigger cleanup for remaining services
+            cleanup
+        fi
+    done
+done

docker/standalone/test-start-all.sh

@@ -0,0 +1,121 @@
+#!/bin/bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+HINDSIGHT_START_ALL_SOURCE_ONLY=true
+source "$SCRIPT_DIR/start-all.sh"
+unset HINDSIGHT_START_ALL_SOURCE_ONLY
+
+TMP_DIR="$(mktemp -d)"
+trap 'chmod -R u+rwx "$TMP_DIR" 2>/dev/null || true; rm -rf "$TMP_DIR"' EXIT
+
+assert_contains() {
+    local output="$1"
+    local expected="$2"
+
+    if [[ "$output" != *"$expected"* ]]; then
+        echo "Expected output to contain: $expected"
+        echo "Actual output:"
+        echo "$output"
+        exit 1
+    fi
+}
+
+assert_not_contains() {
+    local output="$1"
+    local unexpected="$2"
+
+    if [[ "$output" == *"$unexpected"* ]]; then
+        echo "Expected output not to contain: $unexpected"
+        echo "Actual output:"
+        echo "$output"
+        exit 1
+    fi
+}
+
+assert_empty() {
+    local output="$1"
+
+    if [ -n "$output" ]; then
+        echo "Expected no output, got:"
+        echo "$output"
+        exit 1
+    fi
+}
+
+mkdir -p "$TMP_DIR/empty"
+assert_empty "$(check_pg0_data_integrity "$TMP_DIR/empty")"
+
+mkdir -p "$TMP_DIR/direct"
+touch "$TMP_DIR/direct/PG_VERSION"
+direct_output="$(check_pg0_data_integrity "$TMP_DIR/direct")"
+assert_contains "$direct_output" "Existing pg0 data directory detected"
+assert_not_contains "$direct_output" "WARNING"
+
+mkdir -p "$TMP_DIR/legacy/instance"
+touch "$TMP_DIR/legacy/instance/PG_VERSION"
+legacy_output="$(check_pg0_data_integrity "$TMP_DIR/legacy")"
+assert_contains "$legacy_output" "Existing pg0 data directory detected"
+assert_not_contains "$legacy_output" "WARNING"
+
+mkdir -p "$TMP_DIR/nested/instances/hindsight/data"
+touch "$TMP_DIR/nested/instances/hindsight/data/PG_VERSION"
+nested_output="$(check_pg0_data_integrity "$TMP_DIR/nested")"
+assert_contains "$nested_output" "Existing pg0 data directory detected"
+assert_not_contains "$nested_output" "WARNING"
+
+mkdir -p "$TMP_DIR/nonempty/instances/hindsight"
+touch "$TMP_DIR/nonempty/instances/hindsight/instance.json"
+nonempty_output="$(check_pg0_data_integrity "$TMP_DIR/nonempty")"
+assert_contains "$nonempty_output" "WARNING: pg0 data directory exists"
+
+echo "start-all pg0 integrity checks passed"
+
+# =============================================================================
+# check_pg0_writable (#1483)
+# These rely on filesystem permissions, which root bypasses; skip under root.
+# =============================================================================
+if [ "$(id -u)" != "0" ]; then
+    # Writable directory: returns 0, prints nothing, leaves no artifact behind.
+    mkdir -p "$TMP_DIR/writable"
+    writable_output="$(check_pg0_writable "$TMP_DIR/writable")"
+    assert_empty "$writable_output"
+    if [ -e "$TMP_DIR/writable/.hindsight-write-test" ]; then
+        echo "check_pg0_writable left its write-test file behind"
+        exit 1
+    fi
+
+    # Non-writable directory: returns 1 with actionable guidance.
+    mkdir -p "$TMP_DIR/readonly"
+    chmod 000 "$TMP_DIR/readonly"
+    set +e
+    readonly_output="$(check_pg0_writable "$TMP_DIR/readonly" 2>&1)"
+    readonly_rc=$?
+    set -e
+    chmod 755 "$TMP_DIR/readonly"
+    if [ "$readonly_rc" -eq 0 ]; then
+        echo "check_pg0_writable should fail on a non-writable directory"
+        exit 1
+    fi
+    assert_contains "$readonly_output" "not writable"
+    assert_contains "$readonly_output" "hindsight-data:/home/hindsight/.pg0"
+    assert_contains "$readonly_output" "--user"
+
+    # External database configured: skip the check regardless of dir perms.
+    mkdir -p "$TMP_DIR/extdb"
+    chmod 000 "$TMP_DIR/extdb"
+    set +e
+    HINDSIGHT_API_DATABASE_URL="postgres://x" check_pg0_writable "$TMP_DIR/extdb" >/dev/null 2>&1
+    extdb_rc=$?
+    set -e
+    chmod 755 "$TMP_DIR/extdb"
+    if [ "$extdb_rc" -ne 0 ]; then
+        echo "check_pg0_writable should skip when an external database is configured"
+        exit 1
+    fi
+
+    echo "start-all pg0 writability checks passed"
+else
+    echo "⚠️  Running as root; skipping pg0 writability checks (permissions are bypassed)."
+fi

docker/test-image.sh

@@ -0,0 +1,235 @@
+#!/bin/bash
+#
+# Docker Smoke Test Script
+#
+# Tests that a Hindsight Docker image starts correctly and becomes healthy.
+# Can be run locally or in CI pipelines.
+#
+# Usage:
+#   ./docker/test-image.sh <image> [target]
+#
+# Arguments:
+#   image   - Docker image to test (e.g., hindsight-api:test, ghcr.io/vectorize-io/hindsight:latest)
+#   target  - Optional: 'cp-only' for control plane, otherwise assumes API image (default: api)
+#
+# Environment variables:
+#   HINDSIGHT_API_LLM_API_KEY                   - Required for API/standalone images (LLM verification)
+#   HINDSIGHT_API_LLM_PROVIDER                  - LLM provider (default: openai)
+#   HINDSIGHT_API_LLM_MODEL                     - LLM model (default: gpt-4o-mini)
+#   HINDSIGHT_API_EMBEDDINGS_PROVIDER           - Embeddings provider (optional, for slim images: openai, cohere, tei)
+#   HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY     - OpenAI API key for embeddings (optional)
+#   HINDSIGHT_API_RERANKER_PROVIDER             - Reranker provider (optional, for slim images: cohere, tei)
+#   HINDSIGHT_API_COHERE_API_KEY                - Cohere API key for reranking (optional)
+#   SMOKE_TEST_TIMEOUT                          - Timeout in seconds (default: 120)
+#   SMOKE_TEST_CONTAINER_NAME                   - Container name (default: hindsight-smoke-test)
+#
+# Examples:
+#   # Test a locally built full image
+#   ./docker/test-image.sh hindsight-api:test
+#
+#   # Test a released image
+#   ./docker/test-image.sh ghcr.io/vectorize-io/hindsight:latest
+#
+#   # Test control plane image
+#   ./docker/test-image.sh hindsight-control-plane:test cp-only
+#
+#   # Test slim image with external providers
+#   export HINDSIGHT_API_LLM_API_KEY=sk_xxx
+#   export HINDSIGHT_API_EMBEDDINGS_PROVIDER=openai
+#   export HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=sk-xxx
+#   export HINDSIGHT_API_RERANKER_PROVIDER=cohere
+#   export HINDSIGHT_API_COHERE_API_KEY=xxx
+#   ./docker/test-image.sh hindsight-slim:test
+#
+# Exit codes:
+#   0 - Success (container healthy)
+#   1 - Failure (container not healthy within timeout)
+#   2 - Invalid arguments
+#
+
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_ROOT="$(dirname "$SCRIPT_DIR")"
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[0;33m'
+NC='\033[0m' # No Color
+
+# Configuration
+IMAGE="${1:-}"
+TARGET="${2:-api}"
+TIMEOUT="${SMOKE_TEST_TIMEOUT:-120}"
+CONTAINER_NAME="${SMOKE_TEST_CONTAINER_NAME:-hindsight-smoke-test}"
+LLM_PROVIDER="${HINDSIGHT_API_LLM_PROVIDER:-openai}"
+LLM_MODEL="${HINDSIGHT_API_LLM_MODEL:-gpt-4o-mini}"
+
+# Validate arguments
+if [ -z "$IMAGE" ]; then
+    echo -e "${RED}Error: Image argument is required${NC}"
+    echo ""
+    echo "Usage: $0 <image> [target]"
+    echo ""
+    echo "Examples:"
+    echo "  $0 hindsight-api:test"
+    echo "  $0 ghcr.io/vectorize-io/hindsight:latest"
+    echo "  $0 hindsight-control-plane:test cp-only"
+    exit 2
+fi
+
+# Determine health endpoint based on target
+if [ "$TARGET" = "cp-only" ]; then
+    HEALTH_PORT=9999
+    HEALTH_PATH="/api/health"
+    NEEDS_LLM=false
+else
+    HEALTH_PORT=8888
+    HEALTH_PATH="/health"
+    NEEDS_LLM=true
+fi
+
+# Check for required environment variables
+if [ "$NEEDS_LLM" = true ] && [ "$LLM_PROVIDER" != "vertexai" ] && [ -z "${HINDSIGHT_API_LLM_API_KEY:-}" ]; then
+    echo -e "${RED}Error: HINDSIGHT_API_LLM_API_KEY environment variable is required for API/standalone images${NC}"
+    echo "Set it with: export HINDSIGHT_API_LLM_API_KEY=your-api-key"
+    exit 2
+fi
+
+# Cleanup function
+cleanup() {
+    echo "Cleaning up..."
+    docker stop "$CONTAINER_NAME" 2>/dev/null || true
+    docker rm "$CONTAINER_NAME" 2>/dev/null || true
+}
+
+# Set trap to cleanup on exit
+trap cleanup EXIT
+
+echo -e "${YELLOW}Starting smoke test for: ${IMAGE}${NC}"
+echo "  Target: $TARGET"
+echo "  Health endpoint: http://localhost:${HEALTH_PORT}${HEALTH_PATH}"
+echo "  Timeout: ${TIMEOUT}s"
+echo ""
+
+# Remove any existing container with the same name
+docker rm -f "$CONTAINER_NAME" 2>/dev/null || true
+
+# Start container based on target type
+echo "Starting container..."
+if [ "$TARGET" = "cp-only" ]; then
+    docker run -d --name "$CONTAINER_NAME" \
+        -p "${HEALTH_PORT}:${HEALTH_PORT}" \
+        "$IMAGE"
+else
+    # Build docker run command with required and optional env vars
+    DOCKER_CMD="docker run -d --name $CONTAINER_NAME"
+    DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_LLM_PROVIDER=$LLM_PROVIDER"
+    if [ -n "${HINDSIGHT_API_LLM_API_KEY:-}" ]; then
+        DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_LLM_API_KEY=${HINDSIGHT_API_LLM_API_KEY}"
+    fi
+    DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_LLM_MODEL=$LLM_MODEL"
+
+    # Add Vertex AI config if provider is vertexai
+    if [ "$LLM_PROVIDER" = "vertexai" ]; then
+        if [ -n "${HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY:-}" ]; then
+            DOCKER_CMD="$DOCKER_CMD -v ${HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY}:/tmp/gcp-credentials.json:ro"
+            DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY=/tmp/gcp-credentials.json"
+        fi
+        if [ -n "${HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID:-}" ]; then
+            DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=${HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID}"
+        fi
+        if [ -n "${HINDSIGHT_API_LLM_VERTEXAI_REGION:-}" ]; then
+            DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_LLM_VERTEXAI_REGION=${HINDSIGHT_API_LLM_VERTEXAI_REGION}"
+        fi
+    fi
+
+    # Add optional embeddings provider config
+    if [ -n "${HINDSIGHT_API_EMBEDDINGS_PROVIDER:-}" ]; then
+        DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_EMBEDDINGS_PROVIDER=${HINDSIGHT_API_EMBEDDINGS_PROVIDER}"
+    fi
+    if [ -n "${HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY:-}" ]; then
+        DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=${HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY}"
+    fi
+
+    # Add optional reranker provider config
+    if [ -n "${HINDSIGHT_API_RERANKER_PROVIDER:-}" ]; then
+        DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_RERANKER_PROVIDER=${HINDSIGHT_API_RERANKER_PROVIDER}"
+    fi
+    if [ -n "${HINDSIGHT_API_COHERE_API_KEY:-}" ]; then
+        DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_COHERE_API_KEY=${HINDSIGHT_API_COHERE_API_KEY}"
+    fi
+
+    DOCKER_CMD="$DOCKER_CMD -p ${HEALTH_PORT}:${HEALTH_PORT}"
+    DOCKER_CMD="$DOCKER_CMD $IMAGE"
+
+    eval $DOCKER_CMD
+fi
+
+# Wait for health endpoint
+echo "Waiting for health endpoint at http://localhost:${HEALTH_PORT}${HEALTH_PATH}..."
+start_time=$(date +%s)
+
+for i in $(seq 1 "$TIMEOUT"); do
+    if curl -sf "http://localhost:${HEALTH_PORT}${HEALTH_PATH}" > /dev/null 2>&1; then
+        end_time=$(date +%s)
+        duration=$((end_time - start_time))
+        echo ""
+        echo -e "${GREEN}Container is healthy after ${duration}s${NC}"
+        echo ""
+        echo "=== Health Response ==="
+        curl -s "http://localhost:${HEALTH_PORT}${HEALTH_PATH}" | python3 -m json.tool 2>/dev/null || curl -s "http://localhost:${HEALTH_PORT}${HEALTH_PATH}"
+        echo ""
+
+        # Run retain/recall smoke test for API targets
+        if [ "$TARGET" != "cp-only" ]; then
+            echo ""
+            echo "=== Retain/Recall Smoke Test ==="
+            if ! "$REPO_ROOT/scripts/smoke-test-slim.sh" "http://localhost:${HEALTH_PORT}"; then
+                echo ""
+                echo "=== Container Logs (last 50 lines) ==="
+                docker logs "$CONTAINER_NAME" 2>&1 | tail -50
+                echo ""
+                echo -e "${RED}Smoke test FAILED${NC}"
+                exit 1
+            fi
+        fi
+
+        echo ""
+        echo "=== Container Logs (last 50 lines) ==="
+        docker logs "$CONTAINER_NAME" 2>&1 | tail -50
+        echo ""
+        echo -e "${GREEN}Smoke test PASSED${NC}"
+        exit 0
+    fi
+
+    # Show progress every 10 seconds
+    if [ $((i % 10)) -eq 0 ]; then
+        echo "  Still waiting... (${i}s)"
+    fi
+
+    # Check if container is still running
+    if ! docker ps -q -f "name=$CONTAINER_NAME" | grep -q .; then
+        echo ""
+        echo -e "${RED}Container exited unexpectedly!${NC}"
+        echo ""
+        echo "=== Container Logs ==="
+        docker logs "$CONTAINER_NAME" 2>&1
+        echo ""
+        echo -e "${RED}Smoke test FAILED${NC}"
+        exit 1
+    fi
+
+    sleep 1
+done
+
+# Timeout reached
+echo ""
+echo -e "${RED}Container failed to become healthy after ${TIMEOUT}s${NC}"
+echo ""
+echo "=== Container Logs ==="
+docker logs "$CONTAINER_NAME" 2>&1
+echo ""
+echo -e "${RED}Smoke test FAILED${NC}"
+exit 1

docker/test-slim-local.sh

@@ -0,0 +1,47 @@
+#!/bin/bash
+#
+# Local Test Script for Slim Docker Images
+#
+# This script makes it easy to test slim images locally with external providers.
+# It expects API keys to be set in environment variables.
+#
+# Usage:
+#   export OPENAI_API_KEY=sk-xxx
+#   export COHERE_API_KEY=xxx
+#   ./docker/test-slim-local.sh
+#
+# Or inline:
+#   OPENAI_API_KEY=sk_xxx COHERE_API_KEY=xxx ./docker/test-slim-local.sh
+#
+
+set -euo pipefail
+
+# Check for required API keys
+if [ -z "${OPENAI_API_KEY:-}" ]; then
+    echo "❌ Error: OPENAI_API_KEY environment variable is required"
+    echo "Set it with: export OPENAI_API_KEY=sk-xxx"
+    exit 1
+fi
+
+if [ -z "${COHERE_API_KEY:-}" ]; then
+    echo "❌ Error: COHERE_API_KEY environment variable is required"
+    echo "Set it with: export COHERE_API_KEY=xxx"
+    exit 1
+fi
+
+# Configuration
+IMAGE="${1:-hindsight-slim:test}"
+echo "Testing image: $IMAGE"
+echo ""
+
+# Set up LLM and external providers
+export HINDSIGHT_API_LLM_PROVIDER=openai
+export HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY
+export HINDSIGHT_API_LLM_MODEL=gpt-4o-mini
+export HINDSIGHT_API_EMBEDDINGS_PROVIDER=openai
+export HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=$OPENAI_API_KEY
+export HINDSIGHT_API_RERANKER_PROVIDER=cohere
+export HINDSIGHT_API_COHERE_API_KEY=$COHERE_API_KEY
+
+# Run the test
+exec "$(dirname "$0")/test-image.sh" "$IMAGE" standalone

docs/superpowers/plans/2026-07-25-v2-knowledge-pages.md

@@ -0,0 +1,341 @@
+# v2 Knowledge Pages — Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Make v2 knowledge pages a reliable, cleanly-tiered "wiki" surface: passive `entity_labels` tier-tagging + tag-scoped seeded pages, a `hindsight_*` MCP surface with one active `capture_initiative` verb that creates per-initiative pages linked by tag, and SessionStart/UserPromptSubmit page-roster injection.
+
+**Architecture:** Shared TS core (`hindsight-integrations/hindsight-coding-agents`). Extraction stays blind to "pages"; classification is intrinsic (`knowledge:<tier>` tags), pages are tag-scoped saved views. Per-initiative navigation via a `relatedPageId:<id>` tag the synthesizer renders into `[[page:<id>]]`, with the Initiatives folder/roster as the guaranteed fallback.
+
+**Tech Stack:** TypeScript, vitest, tsup bundling. Hindsight REST API (`/knowledge-base/*`, `/mental-models`, `/memories`, bank `/config`).
+
+**Spec:** `docs/superpowers/specs/2026-07-25-v2-knowledge-pages-design.md`
+
+**Working dir for all commands:** `hindsight-integrations/hindsight-coding-agents`
+**Test command:** `npx vitest run <file>` (fast suite; excludes `*.live.test.ts`). Full check: `npx vitest run && npx tsc --noEmit`.
+
+**Conventions to follow (existing patterns):**
+- `HindsightClient` HTTP via `this.req("METHOD", this.bankUrl(path), body?)`; JSON via `await r.json()`.
+- MCP tools are SDK-free `ToolSpec { name, description, inputSchema (ZodRawShape), handler }`; wrap handler bodies in `guarded(...)`; `ok(value)` / `err(e)` result helpers.
+- Fail-open everywhere in hooks; pure logic separated from stdin/stdout plumbing.
+- Do NOT add a Claude co-author trailer to any commit.
+
+---
+
+## Task 1: Config field `pageRefreshEveryTurns`
+
+**Files:**
+- Modify: `src/core/config.ts`
+- Test: `src/core/config.test.ts`
+
+- [ ] **Step 1: Write failing test** — assert the default resolves to 10 and an override wins.
+
+```ts
+it("pageRefreshEveryTurns defaults to 10 and is overridable", () => {
+  expect(loadConfig({ harness: "claude-code", projectDir: process.cwd() }).pageRefreshEveryTurns).toBe(10);
+});
+```
+(Add an override case mirroring the existing override tests in this file.)
+
+- [ ] **Step 2: Run** `npx vitest run src/core/config.test.ts` → FAIL (property missing).
+- [ ] **Step 3: Implement** — add `pageRefreshEveryTurns: number` to the `Config` type and default `10` in the same place `recallMaxTokens`/`reflectTimeoutMs` are defined/merged. Follow the exact merge/layering pattern already used for numeric fields.
+- [ ] **Step 4: Run** the test → PASS.
+- [ ] **Step 5: Commit** `git add src/core/config.ts src/core/config.test.ts && git commit -m "feat(core): add pageRefreshEveryTurns config (default 10)"`
+
+---
+
+## Task 2: `knowledge-injection.ts` — roster/preamble formatting (pure, new)
+
+**Files:**
+- Create: `src/core/knowledge-injection.ts`
+- Test: `src/core/knowledge-injection.test.ts`
+
+Pure, SDK-free, no network. Parses the `listPages()` payload and formats the two injections.
+
+- [ ] **Step 1: Write failing tests**
+
+```ts
+import { describe, expect, it } from "vitest";
+import { parsePageList, buildKnowledgePreamble, buildRosterRefresh } from "./knowledge-injection";
+
+describe("parsePageList", () => {
+  it("extracts {id,title} from the mental-model list shape, tolerating junk", () => {
+    const raw = { items: [{ id: "p1", name: "Component map" }, { id: "p2", name: "Core concepts" }, { nope: 1 }] };
+    expect(parsePageList(raw)).toEqual([{ id: "p1", title: "Component map" }, { id: "p2", title: "Core concepts" }]);
+  });
+  it("returns [] for null/garbage", () => {
+    expect(parsePageList(null)).toEqual([]);
+    expect(parsePageList(42 as unknown)).toEqual([]);
+  });
+});
+
+describe("buildKnowledgePreamble", () => {
+  it("includes guidance, a roster of pages, and a refresh note", () => {
+    const out = buildKnowledgePreamble([{ id: "p1", title: "Component map" }]);
+    expect(out).toContain("<hindsight_knowledge>");
+    expect(out).toContain("Component map");
+    expect(out).toContain("p1");
+    expect(out).toMatch(/hindsight_read_knowledge_page/);
+  });
+  it("has an empty-state line when there are no pages", () => {
+    const out = buildKnowledgePreamble([]);
+    expect(out).toMatch(/no knowledge pages yet|still learning/i);
+  });
+});
+
+describe("buildRosterRefresh", () => {
+  it("is a compact 'current pages' block listing ids+titles", () => {
+    const out = buildRosterRefresh([{ id: "p1", title: "Component map" }]);
+    expect(out).toContain("Component map");
+    expect(out).toContain("p1");
+  });
+  it("returns undefined when there are no pages (nothing to refresh)", () => {
+    expect(buildRosterRefresh([])).toBeUndefined();
+  });
+});
+```
+
+- [ ] **Step 2: Run** `npx vitest run src/core/knowledge-injection.test.ts` → FAIL.
+- [ ] **Step 3: Implement**
+
+```ts
+export interface PageRef { id: string; title: string; }
+
+/** Defensive parse of HindsightClient.listPages() (GET /mental-models?detail=metadata → {items:[{id,name}]}). */
+export function parsePageList(raw: unknown): PageRef[] {
+  const items = (raw as { items?: unknown })?.items;
+  if (!Array.isArray(items)) return [];
+  const out: PageRef[] = [];
+  for (const it of items) {
+    const id = (it as { id?: unknown })?.id;
+    const name = (it as { name?: unknown })?.name;
+    if (typeof id === "string" && typeof name === "string") out.push({ id, title: name });
+  }
+  return out;
+}
+
+function roster(pages: PageRef[]): string {
+  return pages.map((p) => `- ${p.title} (${p.id})`).join("\n");
+}
+
+/** SessionStart: teach when/why to use pages + list what exists. Empty-state aware. */
+export function buildKnowledgePreamble(pages: PageRef[]): string {
+  const body = pages.length
+    ? `Knowledge pages available in this repository:\n${roster(pages)}`
+    : "No knowledge pages yet — Hindsight is still learning this repo; they'll appear as it processes.";
+  return (
+    "<hindsight_knowledge>\n" +
+    "This repository has a Hindsight knowledge base: curated, continuously-updated pages summarizing its " +
+    "durable engineering knowledge (architecture, components, conventions, key decisions, and in-flight initiatives).\n" +
+    "Before substantial work, consult the relevant pages instead of re-deriving understanding from the code: read " +
+    "Conventions before writing new code, the Component map before changing a subsystem, and an initiative's page " +
+    "before continuing that feature.\n" +
+    `${body}\n` +
+    "Read one with hindsight_read_knowledge_page(page_id). Follow any [[page:<id>]] links you see. The list is " +
+    "re-injected for you periodically as it changes.\n" +
+    "</hindsight_knowledge>"
+  );
+}
+
+/** Periodic UserPromptSubmit refresh — compact, or undefined when there's nothing to show. */
+export function buildRosterRefresh(pages: PageRef[]): string | undefined {
+  if (!pages.length) return undefined;
+  return (
+    "<hindsight_knowledge_refresh>\n" +
+    `Current Hindsight knowledge pages (may have changed):\n${roster(pages)}\n` +
+    "Read any with hindsight_read_knowledge_page(page_id).\n" +
+    "</hindsight_knowledge_refresh>"
+  );
+}
+```
+
+- [ ] **Step 4: Run** the test → PASS.
+- [ ] **Step 5: Commit** `git add src/core/knowledge-injection.ts src/core/knowledge-injection.test.ts && git commit -m "feat(core): knowledge-injection roster/preamble formatting"`
+
+---
+
+## Task 3: `entity_labels` tier vocabulary + configureBank wiring
+
+**Files:**
+- Modify: `src/core/missions.ts` (add `KNOWLEDGE_LABELS`)
+- Modify: `src/core/hindsight.ts` (`configureBank` PATCH sets `entity_labels`)
+- Test: `src/core/hindsight.*.test.ts` (add/extend a config test with a mock client)
+
+- [ ] **Step 1: Write failing test** — assert `configureBank` PATCHes `/config` with `entity_labels` containing the `knowledge` group and its five values, and `entities_allow_free_form: true`. Use the existing fetch/req mock pattern from `hindsight.*.test.ts`; capture the PATCH body to `/config` and assert on it.
+
+- [ ] **Step 2: Run** → FAIL.
+- [ ] **Step 3: Implement**
+  - In `missions.ts`, export `KNOWLEDGE_LABELS` — the exact object from the spec §4 (`key:"knowledge"`, `type:"multi-values"`, `optional:true`, `tag:true`, the verbose group `description`, and the five value `{value,description}` entries: feature-work, decision, convention, component, concept).
+  - In `hindsight.ts::configureBank`, extend the existing `PATCH .../config` `updates` object to include `entity_labels: [KNOWLEDGE_LABELS]` and `entities_allow_free_form: true`. Import `KNOWLEDGE_LABELS`.
+  - Update the `[bank] configured …` log to mention `entity_labels`.
+- [ ] **Step 4: Run** → PASS.
+- [ ] **Step 5: Commit** `git add src/core/missions.ts src/core/hindsight.ts src/core/hindsight.*.test.ts && git commit -m "feat(core): passive knowledge entity_labels tier vocabulary + configureBank wiring"`
+
+---
+
+## Task 4: Tag-scoped seeded pages + Initiatives folder + link source_query
+
+**Files:**
+- Modify: `src/core/missions.ts` (`PAGES` gain `tags`; Initiatives `source_query` link instruction)
+- Modify: `src/core/hindsight.ts` (`ensureFolder`, `createPages` sets page `tags` + parents Initiatives under the folder)
+- Test: `src/core/hindsight.pages.test.ts`
+
+- [ ] **Step 1: Write failing tests** (mock client `req`):
+  - Each seeded page POST to `/knowledge-base/pages` includes `tags: ["knowledge:<tier>"]` mapped per the spec §5 table.
+  - The Initiatives page is created with `parent_id` equal to the id returned by an Initiatives folder POST to `/knowledge-base/folders`.
+  - `ensureFolder("Initiatives")` returns an existing root folder's id when the tree already contains it (GET `/knowledge-base/tree`) and does NOT POST a duplicate.
+
+- [ ] **Step 2: Run** → FAIL.
+- [ ] **Step 3: Implement**
+  - `missions.ts`: add `tags: string[]` to each `PAGES` entry (feature-work/decision/convention/component/concept mapping). Append to the Initiatives `source_query`: *"When a source memory carries a tag of the form `relatedPageId:<id>`, include a Markdown link `[[page:<id>]]` to that page in the summary, so each initiative links to its detailed page."*
+  - `hindsight.ts`: add
+    ```ts
+    /** Find a root folder by name (case-insensitive) or create it; returns its id. */
+    async ensureFolder(name: string): Promise<string | undefined> {
+      try {
+        const tree = (await (await this.req("GET", this.bankUrl("/knowledge-base/tree"))).json()) as
+          { roots?: { id?: string; kind?: string; name?: string }[] };
+        const hit = (tree.roots || []).find((n) => n.kind === "folder" && (n.name || "").toLowerCase() === name.toLowerCase());
+        if (hit?.id) return hit.id;
+      } catch { /* fall through to create */ }
+      try {
+        const r = await this.req("POST", this.bankUrl("/knowledge-base/folders"), { name });
+        return ((await r.json()) as { id?: string }).id;
+      } catch { return undefined; }
+    }
+    ```
+  - In `createPages()`: before the loop, `const initiativesFolderId = await this.ensureFolder("Initiatives");`. For each page, build body `{ name, source_query, tags: p.tags, parent_id: <initiativesFolderId if this is the Initiatives page else undefined>, trigger: { fact_types:[...], refresh_after_consolidation:true } }`. (Page-level `tags` drives synthesis scoping via `RefreshTagFiltering`; `tags_match` defaults to `all_strict` when tags present.)
+- [ ] **Step 4: Run** → PASS.
+- [ ] **Step 5: Commit** `git add src/core/missions.ts src/core/hindsight.ts src/core/hindsight.pages.test.ts && git commit -m "feat(core): tag-scope seeded pages, Initiatives folder, relatedPageId link source_query"`
+
+---
+
+## Task 5: Client helpers — per-initiative page + marker retain
+
+**Files:**
+- Modify: `src/core/hindsight.ts` (`captureInitiative`)
+- Test: `src/core/hindsight.pages.test.ts`
+
+- [ ] **Step 1: Write failing tests** (mock `req`):
+  - `captureInitiative({title:"Retry backoff for the uploader", summary:"…"})` → derives slug `retry-backoff-for-the-uploader`, POSTs a page id `initiative-<slug>` to `/knowledge-base/pages` with `parent_id` = the Initiatives folder and `tags: ["knowledge:feature-work"]`, AND POSTs a marker to `/memories` (via `retain`) tagged `["knowledge:feature-work","relatedPageId:initiative-<slug>"]`, strategy `session` or `document` (pick `document`), `async:true`. Returns `{ page_id: "initiative-<slug>" }`.
+  - Slug is deterministic and identical between the page id and the `relatedPageId:` tag value.
+  - Enhancement path: `captureInitiative({title, summary, relatesToPageId:"initiative-x"})` POSTs NO new page; marker tagged `relatedPageId:initiative-x`; returns `{ page_id: "initiative-x" }`.
+
+- [ ] **Step 2: Run** → FAIL.
+- [ ] **Step 3: Implement**
+
+```ts
+private slugify(s: string): string {
+  return s.toLowerCase().replace(/[^a-z0-9]+/g, "-").replace(/^-+|-+$/g, "").slice(0, 60) || "initiative";
+}
+
+/** Active-path capture: register a major feature as a per-initiative page + a tagged marker memory. */
+async captureInitiative(args: { title: string; summary: string; relatesToPageId?: string }): Promise<{ page_id: string }> {
+  const pageId = args.relatesToPageId ?? `initiative-${this.slugify(args.title)}`;
+  if (!args.relatesToPageId) {
+    const folderId = await this.ensureFolder("Initiatives");
+    await this.req("POST", this.bankUrl("/knowledge-base/pages"), {
+      name: args.title,
+      source_query: `Summarize the "${args.title}" initiative: what is being built or changed and why, and its current state — drawn from the project's memory.`,
+      parent_id: folderId,
+      tags: ["knowledge:feature-work", `relatedPageId:${pageId}`],
+      trigger: { fact_types: ["world", "experience", "observation"], refresh_after_consolidation: true },
+    });
+  }
+  const verb = args.relatesToPageId ? "Enhancement to an existing initiative" : "New initiative";
+  const content = `${verb}: ${args.title}. ${args.summary}`;
+  await this.retain(content, "initiative marker", pageId /* not a stable doc id requirement; see note */,
+    ["knowledge:feature-work", `relatedPageId:${pageId}`], "document", { async: true });
+  return { page_id: pageId };
+}
+```
+  - NOTE: use a UNIQUE document id per marker (e.g. `initiative-marker-<slug>-<n>`), NOT `pageId`, so repeated enhancement captures accrue instead of replacing. Since `Date.now()` is fine here (runtime, not a workflow script), suffix with a timestamp: `initiative-marker-${this.slugify(args.title)}-${Date.now()}`. Keep the `relatedPageId` tag equal to `pageId`.
+  - Confirm `retain(content, context, documentId, tags, strategy, opts)` signature matches current `HindsightClient.retain`.
+
+- [ ] **Step 4: Run** → PASS.
+- [ ] **Step 5: Commit** `git add src/core/hindsight.ts src/core/hindsight.pages.test.ts && git commit -m "feat(core): captureInitiative — per-initiative page + relatedPageId marker"`
+
+---
+
+## Task 6: MCP surface — `hindsight_*` grounding + `capture_initiative`; drop page CRUD
+
+**Files:**
+- Modify: `src/core/knowledge-tools.ts`
+- Modify: `src/mcp-server.ts` (only if it references removed tool names)
+- Test: `src/core/knowledge-tools.test.ts`, `src/mcp-server.test.ts` (tool-count assertions)
+
+- [ ] **Step 1: Write failing tests**
+  - `buildKnowledgeTools(client, bankId)` returns exactly these tool names: `hindsight_get_current_bank`, `hindsight_list_knowledge_pages`, `hindsight_read_knowledge_page`, `hindsight_search_memory`, `hindsight_capture_initiative`, `hindsight_ingest_document`. (Assert the set; update any count assertion.)
+  - `hindsight_capture_initiative` handler calls `client.captureInitiative` with `{title, summary, relatesToPageId?}` and returns the page id (mock client).
+  - No `create_page` / `update_page` / `delete_page` tools are present.
+  - Each tool still fails closed via `guarded` (a thrown client error → `isError:true`, no throw).
+
+- [ ] **Step 2: Run** → FAIL.
+- [ ] **Step 3: Implement**
+  - Rebuild the `buildKnowledgeTools` list: rename read/recall/ingest/bank tools to the `hindsight_*` names; drop `create_page`/`update_page`/`delete_page`; add `hindsight_capture_initiative` with `inputSchema { title: z.string(), summary: z.string(), relates_to_page_id: z.string().optional() }` calling `client.captureInitiative({ title, summary, relatesToPageId: relates_to_page_id })`.
+  - Use the **verbatim agent-facing `description` strings** from the spec §6 / the brainstorm (grounding tools + the explicit WHEN/WHEN-NOT `capture_initiative` description).
+  - Update `mcp-server.ts` only if it enumerates tool names; otherwise it consumes `buildKnowledgeTools` generically and needs no change.
+- [ ] **Step 4: Run** `npx vitest run src/core/knowledge-tools.test.ts src/mcp-server.test.ts` → PASS.
+- [ ] **Step 5: Commit** `git add src/core/knowledge-tools.ts src/mcp-server.ts src/core/knowledge-tools.test.ts src/mcp-server.test.ts && git commit -m "feat(mcp): hindsight_* grounding tools + capture_initiative; remove raw page CRUD from agent"`
+
+---
+
+## Task 7: SessionStart — preamble + roster
+
+**Files:**
+- Modify: `src/core/session-start.ts`
+- Test: `src/core/session-start.test.ts`
+
+- [ ] **Step 1: Write failing tests**
+  - `buildSessionStartContext` now fetches pages via the client and injects `buildKnowledgePreamble(...)` instead of the static `KNOWLEDGE_MISSION`. Extend the `SeedContextClient` interface with `listPages(): Promise<unknown>`; the mock returns `{items:[{id:"p1",name:"Component map"}]}` and the output contains "Component map".
+  - listPages failure is fail-open: the preamble still renders (empty-state) and the seed logic is unaffected.
+
+- [ ] **Step 2: Run** → FAIL.
+- [ ] **Step 3: Implement**
+  - Add `listPages` to `SeedContextClient`.
+  - Replace the `parts.push(KNOWLEDGE_MISSION)` line with: fetch `const pages = parsePageList(await client.listPages().catch(() => null));` then `parts.push(buildKnowledgePreamble(pages));`. Import from `./knowledge-injection`.
+  - Remove the now-unused `KNOWLEDGE_MISSION` export if nothing else references it (grep first; keep if referenced).
+- [ ] **Step 4: Run** → PASS.
+- [ ] **Step 5: Commit** `git add src/core/session-start.ts src/core/session-start.test.ts && git commit -m "feat(core): SessionStart injects page roster + guidance preamble"`
+
+---
+
+## Task 8: UserPromptSubmit — hook-counted periodic roster refresh
+
+**Files:**
+- Modify: `src/core/hook.ts`
+- Test: `src/core/hook.test.ts`
+
+- [ ] **Step 1: Write failing tests**
+  - The session cache round-trips `{answer, turns}`; each `buildHookOutput` call increments `turns`.
+  - Add `listPages` to the `HookClient` interface. On a turn where `turns % cfg.pageRefreshEveryTurns === 0`, the output includes `buildRosterRefresh(...)` content (assert "Component map" appears); on other turns it does not.
+  - Refresh is fail-open (a `listPages` rejection doesn't break recall/injection).
+  - First-turn behavior (reflect) unchanged.
+
+- [ ] **Step 2: Run** → FAIL.
+- [ ] **Step 3: Implement**
+  - Extend the cache read/write to `{ answer?: string; turns?: number }`. Compute `const turns = (cached.turns ?? 0) + 1;` and persist it alongside `answer`.
+  - Add `listPages(): Promise<unknown>` to `HookClient`.
+  - After computing `memBlock`, if `cfg.pageRefreshEveryTurns > 0 && turns % cfg.pageRefreshEveryTurns === 0`, `try { const refresh = buildRosterRefresh(parsePageList(await client.listPages())); if (refresh) blocks.push(refresh); } catch { /* fail-open */ }`. Kick the `listPages` call off concurrently with recall to avoid added latency.
+  - Import from `./knowledge-injection`.
+- [ ] **Step 4: Run** `npx vitest run src/core/hook.test.ts` → PASS.
+- [ ] **Step 5: Commit** `git add src/core/hook.ts src/core/hook.test.ts && git commit -m "feat(core): UserPromptSubmit hook-counted periodic page-roster refresh"`
+
+---
+
+## Task 9: Full check + LLM behavior (live) verification
+
+**Files:**
+- Modify: `src/system.live.test.ts` (add coverage; runs only under `HINDSIGHT_LIVE_E2E=1`)
+
+- [ ] **Step 1: Full fast suite + types** — `npx vitest run && npx tsc --noEmit` → all green.
+- [ ] **Step 2: Add a live assertion** (guarded by the existing live env flag) that after seeding a small repo + one `captureInitiative`, the Initiatives page content contains a `[[page:initiative-…]]` link (verifies the `relatedPageId` → link rendering end-to-end). Keep it in the live suite; do not run in the fast job.
+- [ ] **Step 3: Manual/live run** (optional, operator): `HINDSIGHT_API_URL=http://localhost:8888 npm run test:live`.
+- [ ] **Step 4: Commit** `git add src/system.live.test.ts && git commit -m "test(live): initiative page renders relatedPageId link end-to-end"`
+
+---
+
+## Final review
+
+- [ ] Dispatch a final code-reviewer over the whole change set against the spec (`docs/superpowers/specs/2026-07-25-v2-knowledge-pages-design.md`).
+- [ ] Rebuild + dev-install the `claude-code-v2` bundle so the running plugin picks up the new hooks/MCP (`bash scripts/dev-install.sh`); do not push/PR without explicit consent.
+- [ ] Note deferred follow-ups: session drill-down tag, `capture_decision`, `gotcha` tier, older-bank reseed requirement.

docs/superpowers/specs/2026-07-25-v2-knowledge-pages-design.md

@@ -0,0 +1,135 @@
+# v2 Knowledge Pages — Design Spec
+
+**Status:** approved in brainstorm (2026-07-25), pending implementation plan
+**Scope:** `hindsight-integrations/hindsight-coding-agents` (shared TS core) + `claude-code-v2` wrapper
+**Motivation:** make knowledge pages a real, trustworthy "wiki" surface for the vectorize-crm demo (the `knowledge-pages-as-trust-surface` principle) — the agent reliably knows what pages exist, pages are cleanly tiered instead of blended, and major initiatives become first-class, linkable pages.
+
+---
+
+## 1. Problem
+
+Three gaps in the current v2 branch:
+
+1. **Page discovery is a blind fetch.** SessionStart injects a static `KNOWLEDGE_MISSION` telling the agent to call `agent_knowledge_list_pages`, but hands it **no roster** — the agent never learns a page exists unless it independently decides to call the tool. Per-turn recall injects facts, not pages.
+2. **Pages are blended.** Neither the seeded `PAGES` nor the agent's `create_page` tool scope synthesis by tag, so every page synthesizes from the whole bank filtered only by `fact_type`. Git-log, session, and survey memories all bleed into every page.
+3. **No first-class initiative tracking / linking.** Hindsight has no native page-to-page links. A "major feature" leaves no durable, navigable page a future session can pick up.
+
+## 2. Principles applied
+
+- Automatic/visible value; zero out-of-band CLI; memory beats code search; knowledge pages as a trust surface; minimal post-setup burden.
+- Modular units, small files, follow existing patterns (per-hook specs, fail-open, unit-testable pure cores).
+- The **memory extractor never knows what a "page" is.** Classification is by the fact's *intrinsic* nature; pages are application-side saved views. No abstraction leak into extraction.
+
+## 3. Architecture overview
+
+Two complementary curation paths + a discovery layer:
+
+- **Passive (automatic):** `entity_labels` schema-forces the extractor to tag qualifying facts `knowledge:<tier>`. Seeded **tier pages** each filter on one tier tag. No agent effort.
+- **Active (high-signal):** one intent-named MCP verb, `hindsight_capture_initiative`, lets the agent register a major feature as a **per-initiative page** with a tag-based link back from the aggregate Initiatives page.
+- **Discovery:** SessionStart injects guidance + the page roster; the UserPromptSubmit hook re-injects a fresh roster on a fixed cadence (hook-counted, not model-counted).
+
+## 4. `entity_labels` — passive tier tagging
+
+One hierarchical bank config group, set by `configureBank` at seed time:
+
+```jsonc
+{
+  "key": "knowledge",
+  "type": "multi-values",   // 0, 1, or several — empty is normal
+  "optional": true,
+  "tag": true,               // emits knowledge:<value> onto the fact's tags
+  "description": "Routing labels for this project's Hindsight KNOWLEDGE PAGES — curated, human-readable summaries of the repo's DURABLE engineering knowledge (architecture, key decisions, conventions, ongoing initiatives), each page rebuilt automatically from the facts labeled for it. Mark a fact only when it is durable, reusable knowledge a developer would still want surfaced in future sessions. IMPORTANT: leave this EMPTY for routine, transient, or operational facts — a passing test, a one-off command, a status update, a debugging dead-end. MOST facts should get no label here. Assign more than one value only when the fact genuinely fits several.",
+  "values": [
+    { "value": "feature-work", "description": "A new feature, initiative, or enhancement being planned or built — the capability being added and the intent behind it. Not routine bug-fixes or chores." },
+    { "value": "decision",     "description": "A technical decision that will constrain future work, with its rationale — why this approach was chosen over alternatives, or a rule deliberately adopted." },
+    { "value": "convention",   "description": "An established way this project does things — naming, structure, testing, error handling, or another recurring pattern a contributor is expected to follow." },
+    { "value": "component",    "description": "What a specific module, file, service, or subsystem is responsible for, or how components depend on and connect to one another." },
+    { "value": "concept",      "description": "A domain concept, key abstraction, or piece of project vocabulary a new contributor must understand to work effectively." }
+  ]
+}
+```
+
+Notes:
+- `tag: true` → `_inject_label_tags` copies each `knowledge:<value>` onto the fact's `tags` (no extra query infra).
+- Selectivity (multi-values + "mostly empty" instruction) prevents force-fitting routine facts into a tier.
+
+## 5. Seeded tier pages (tag-scoped)
+
+Created via `/knowledge-base/pages` (supports `tags`, `trigger`, `parent_id`) — **not** `/mental-models`. Each `PAGES` entry gains a `trigger.tags` pin:
+
+| Page | `trigger.tags` |
+| --- | --- |
+| Initiatives and enhancements | `["knowledge:feature-work"]` |
+| Key decisions and rationale | `["knowledge:decision"]` |
+| Conventions and patterns | `["knowledge:convention"]` |
+| Component map | `["knowledge:component"]` |
+| Core concepts | `["knowledge:concept"]` |
+
+`tags_match` strict enough to exclude untagged facts (`all_strict`/`any_strict`). Tag matching is exact set-ops (no wildcards) — this is *why* the vocabulary is fixed, not per-feature.
+
+## 6. MCP surface
+
+Raw page CRUD (`create_page`/`update_page`/`delete_page`) is **removed** from the agent. The agent sees grounding tools + one capture verb. Naming convention: `hindsight_*`.
+
+**Grounding**
+- `hindsight_list_knowledge_pages` `{}` — roster: id, title, one-line coverage. (agent-facing description as drafted in brainstorm)
+- `hindsight_read_knowledge_page` `{ page_id }` — full page content; follow `[[page:<id>]]` links by re-calling.
+- `hindsight_search_memory` `{ query, max_tokens? }` — raw fact recall for specifics pages don't cover.
+- `hindsight_get_current_bank` `{}` — minor introspection (kept).
+
+**Capture**
+- `hindsight_capture_initiative` `{ title, summary, relates_to_page_id? }` — the one active verb. Explicit WHEN / WHEN-NOT description (as drafted). Returns the initiative page id.
+- `hindsight_ingest_document` `{ title, content }` — existing `agent_knowledge_ingest`, reframed.
+
+(Full agent-facing descriptions are captured verbatim in the brainstorm thread and will be reproduced in the implementation plan.)
+
+## 7. `hindsight_capture_initiative` mechanism
+
+- Derive one slug `S` from `title`. Page id = `initiative-<S>`. **The slug in the tag and the page id are the same token, derived once** (cannot drift).
+- **New initiative** (`relates_to_page_id` omitted):
+  1. Create page `initiative-<S>` (title from `title`, `source_query` about that initiative) under an **"Initiatives" folder** (tag-scoped).
+  2. Retain a marker memory (text = title + summary) tagged `["knowledge:feature-work", "relatedPageId:initiative-<S>"]`. **No session tag** (decided — the MCP server has no Claude session id; faking one wouldn't link to the Stop write-back's `conversation:<sessionId>` doc anyway).
+- **Enhancement** (`relates_to_page_id` given): marker only, `relatedPageId = relates_to_page_id`; no new page. Re-invoking for the same initiative accrues markers → the page re-synthesizes with progress.
+
+### Link survival (why `relatedPageId` as a tag, not in prose)
+
+A tag is set directly via the retain `tags` param — it **bypasses LLM extraction entirely**, so it's guaranteed present verbatim (no REF-ID-style preservation needed at extraction). Verified: the reflect/synthesis path SELECTs `tags` and serializes facts via `_prune_nulls(model_dump())`, which keeps non-empty tags → **the synthesis LLM sees the tag.** The **Initiatives page `source_query`** instructs: *"when a memory carries a `relatedPageId:<id>` tag, emit a `[[page:<id>]]` link to it."* The link id is generated from the tag value at synthesis time, so it always matches the created page id.
+
+- Only **Stage 2 (synthesis)** is probabilistic now (bounded token budget may omit some entries when there are many).
+- **Guaranteed fallback:** the per-initiative page always exists (created via API, independent of any LLM stage) and appears in the **Initiatives folder / injected roster**, so navigation works even if a synthesized inline link drops.
+
+## 8. Page-access injection
+
+- **SessionStart** (`session-start.ts`): replace static `KNOWLEDGE_MISSION` with a preamble = (a) guidance on *when/why* to consult pages, (b) the roster fetched via `client.listPages()` (`- <title> (<id>)`, empty-state aware), (c) a note that the list refreshes periodically. Cold repo → empty roster line; roster comes alive mid-session as seeding/survey complete.
+- **UserPromptSubmit** (`hook.ts`): extend the per-session cache (`{answer}` → `{answer, turns}`); the **hook** counts user turns and, roughly every `pageRefreshEveryTurns` (default 10, approximate), calls `listPages()` and injects a compact roster refresh. Runs concurrently with recall; **fail-open** (a refresh error never blocks the turn).
+- **Shared formatting** (new `core/knowledge-injection.ts`, SDK-free/unit-testable): `parsePageList(raw) -> {id,title}[]`, `buildKnowledgePreamble(pages)`, `buildRosterRefresh(pages)`.
+- **Config:** `pageRefreshEveryTurns` (default 10).
+
+## 9. Non-goals / deferred
+
+- Session drill-down tag on captured markers (dropped — see §7).
+- `hindsight_capture_decision` and other capture verbs (passive path covers those tiers; revisit if the aggregate pages aren't sharp enough).
+- A `gotcha`/`pitfall` tier (five tiers for now).
+- Native page-to-page links / backlinks (Hindsight has none; we approximate via folder tree + `relatedPageId`-driven `[[page:<id>]]`).
+
+## 10. Risks / migration
+
+- **Older banks** need re-seeding to pick up the new `entity_labels`, the `session` retain strategy, and the tag-scoped page triggers (`configureBank` sets them). User is starting fresh with v2 banks, so acceptable; live retain fails open otherwise.
+- **Stage-2 synthesis omission** for large initiative counts — mitigated by the folder/roster fallback.
+- **Instruction adherence** for the `source_query` link-rendering and the label selectivity — both are LLM-following behaviors; cover with an `hs_llm_core` judge test, and the deterministic mechanics (tag injection, roster formatting, slug/id equality, hook turn-counting) with fast unit tests.
+
+## 11. Testing
+
+- **Deterministic unit tests:** `knowledge-injection` formatting + empty-state; hook turn-counter + cadence; `capture_initiative` slug→id→tag equality and request shape (mock client); tag-scoped page request bodies; entity_labels config emitted by `configureBank`.
+- **LLM judge test (`hs_llm_core`):** label selectivity (routine facts get no `knowledge:*`), and `relatedPageId` → `[[page:<id>]]` rendering in a synthesized Initiatives page.
+
+## 12. File map (anticipated)
+
+- `src/core/knowledge-injection.ts` (new) — roster/preamble formatting.
+- `src/core/session-start.ts` — preamble + roster.
+- `src/core/hook.ts` — cache `{answer,turns}` + periodic roster refresh.
+- `src/core/config.ts` — `pageRefreshEveryTurns`.
+- `src/core/missions.ts` — `entity_labels` group; tag-scoped `PAGES`; Initiatives `source_query` link instruction.
+- `src/core/hindsight.ts` — `configureBank` sets `entity_labels`; `createPages` pins `trigger.tags` + Initiatives folder; new `createInitiativePage`/marker retain helpers.
+- `src/core/knowledge-tools.ts` — new `hindsight_*` grounding + `capture_initiative` tools; remove raw page CRUD from agent surface.
+- Tests alongside each.

docs/superpowers/specs/2026-07-27-reflect-pages-runtime.md

@@ -0,0 +1,139 @@
+# Reflect + Pages Runtime — Design Spec
+
+**Status:** decided (2026-07-27), reconciles the earlier reflect-based runtime with the recall-based v2 into one opinionated path
+**Scope:** `hindsight-integrations/hindsight-coding-agents` (shared TS core) + `claude-code-v2` wrapper
+**Motivation:** the 33-task coding benchmark showed the v2 recall-per-prompt runtime *underperforms no memory* (35.0 mean corrections vs 32.0 baseline), while the earlier reflect-injection runtime beats baseline by 22% (25.0). This spec restores reflect as the only deep-memory path and replaces raw per-turn recall with lightweight injection from knowledge pages — "fast like recall, organized like reflect" — keeping v2's page/curation machinery where it earned its place and deleting it where it didn't.
+
+---
+
+## 1. Problem
+
+Two prior iterations, each half right:
+
+1. **Reflect runtime (v1):** one agentic REFLECT over the bank at session start, cached and re-injected every turn. Benchmark-proven (25.0 mean corrections) — but nothing surfaced mid-session; a task that drifted away from the first message got stale context.
+2. **Recall runtime (v2):** per-prompt recall injection for turn-by-turn visibility, plus knowledge pages as a trust surface. But raw recall injects unsynthesized fact fragments — noise that *hurt*: 35.0 mean corrections, worse than running with no memory at all.
+
+| Runtime | Mean corrections (33-task benchmark) | vs no-memory (32.0) |
+| --- | --- | --- |
+| Reflect-injection (v1) | **25.0** | **−22%** |
+| Recall-per-prompt (v2) | 35.0 | +9% (regression) |
+| No memory | 32.0 | baseline |
+
+The reconciliation: keep reflect's synthesis quality as the deep path, keep v2's per-turn visibility principle, but source the per-turn material from the already-synthesized knowledge pages instead of raw recall.
+
+## 2. Decisions
+
+Explicit, decided — not options:
+
+1. **Reflect restored** as the only deep-memory path (session-start, agentic synthesis, cached + re-injected every turn).
+2. **Recall removed from the runtime** entirely. No per-prompt `recall` call.
+3. **No `memoryMode` flag.** One opinionated path; config is for environment, naming, and harness wiring only — never behavior selection.
+4. **Sections, not pages, are the per-turn injection unit** — locally matched, budget-trimmed, provenance-labeled.
+5. **JSON turn transcripts** replace the markdown tool-call transcript in the Stop-hook write-back, with compact action entries.
+6. **No tags / no `entity_labels`.** The server re-synthesizes pages after consolidation; "living pages" needs no client-side tagging machinery.
+
+## 3. Runtime path — session start
+
+Three steps, in order, all inside existing hooks (no out-of-band CLI):
+
+### 3a. Cold-repo bootstrap (kept from v2)
+
+On a bank with no prior memories: automatic shallow gitlog seed + codebase survey, exactly as v2 does it. The user never runs a setup command; the first session self-seeds. (Deep ingestion of that history is §7 — the seed here stays instant.)
+
+### 3b. REFLECT once, on the first task message
+
+The benchmark-proven core:
+
+- On the first user prompt of the session, run one **REFLECT** — agentic synthesis over the whole bank, prompted to return the *root-cause decision with exact values* (concrete file paths, config values, version numbers — not summaries of summaries).
+- Cache the result per session; **re-inject it every turn**. It is the session's durable deep context.
+- One LLM-backed call per session, on the message that actually states the task — not on session-open, where there is nothing to reflect about.
+
+### 3c. Page index build
+
+Fetch all knowledge pages once (existing `listPages` + page reads), split each page at headings into **sections**, and build a **local section index** in the hook process. This index is what every subsequent turn matches against (§4) — no further server calls on the hot path.
+
+## 4. Runtime path — every turn
+
+Per-turn visibility, satisfied at ~zero latency and ~zero cost. Injection sources from **knowledge pages, not raw recall** — the material is already synthesized and organized; the turn hook only *selects* from it.
+
+Mechanism (local, deterministic — no server call, no LLM call):
+
+| Aspect | Design |
+| --- | --- |
+| Unit | Page **sections** (pages split at headings at index-build time) |
+| Matching | Lexical: prompt scored against each section by weighted term overlap; **heading hits weighted higher** than body hits |
+| Selection | Top 2–3 sections |
+| Budget | Trimmed to a **~700-token total** |
+| Provenance | Each snippet labeled `From <page> › <section>` + a tool pointer to read the full page |
+| Floor | A minimum-score threshold below which **nothing is injected** — silence over noise |
+| Refresh | Section index rebuilt on the existing 10-turn roster cadence (`pageRefreshEveryTurns`) |
+
+The score floor is load-bearing: the benchmark showed that injecting weak matches is worse than injecting nothing (v2's regression). An empty injection is a correct outcome, not a failure mode.
+
+## 5. Write-back
+
+The Stop-hook session retain is **kept** — same trigger, same fail-open behavior. What changes is the transcript format handed to extraction:
+
+- **JSON turns**, not markdown: an array of `{ "role": "user" | "assistant", "text": ... }` entries for the conversational content.
+- Tool calls collapse to **compact one-line action entries**: `{ "role": "action", "text": "Edit boltons/strutils.py" }` — tool name + primary target only, **no arguments, no outputs**.
+
+Rationale: extraction keeps the concrete artifacts (which files were touched, what actions occurred) without the transcript noise of full tool payloads — the markdown tool-call dumps were volume without signal.
+
+## 6. Knowledge pages
+
+Simplified from the v2 spec:
+
+- **Dropped: tags and `entity_labels`** (v2 spec §4–5). The server already re-synthesizes pages after consolidation, so pages stay "living" with no client-side routing machinery. The extractor-never-knows-about-pages principle now holds trivially — there is nothing to route.
+- **Creation paths:**
+  1. **Seeded taxonomy** at bank creation (the fixed page set, as today, minus tag triggers).
+  2. **Agent-driven `capture_initiative`** at plan approval — the one active capture verb survives from v2.
+  3. **Organic splitting** of pages that outgrow their scope is a **server/curator concern**, not a client feature.
+
+## 7. Ingestion — progressive background deepening
+
+*Status: design accepted, implementation phased separately.*
+
+Replaces the manual backfill CLI as the user-facing path (the CLI was out-of-band burden; nobody runs it). The principle: converge to full-depth history through normal usage, with zero user action.
+
+1. **Instant shallow seed** — the gitlog seed from §3a; the session is useful immediately.
+2. **Background deepening** — a background worker deep-ingests **per-commit-with-diffs, incrementally**, never blocking a turn.
+3. **Working-set prioritization** — commits are ingested in order of relevance to what the agent is actually doing: files the agent reads/edits get their commit histories ingested **first**. Depth arrives where it pays off.
+4. **Checkpointing** — progress persists across sessions; each session resumes deepening where the last left off, converging to full depth over normal usage.
+
+The **backfill CLI survives as an internal tool** (benchmark setup, CI bank preparation) — it is no longer a documented user path.
+
+## 8. Gap analysis — v2 principles under this design
+
+| v2 principle | How this design satisfies it |
+| --- | --- |
+| See-it-working (automatic, visible value) | Reflect answer visible from turn 1; page-section snippets appear with explicit `From <page> › <section>` provenance, so the user sees memory working — and the score floor keeps it from visibly misfiring. |
+| No out-of-band CLI | Cold-repo auto-seed kept (§3a); backfill CLI demoted to internal-only, replaced by background deepening (§7). Nothing requires a terminal command. |
+| Reuse-over-reinvent | Reflect, `listPages`, Stop-hook retain, `capture_initiative`, and the 10-turn refresh cadence are all existing machinery recombined; the only new code is the local section index and matcher — deliberately dumb (lexical, no LLM). |
+| Preserve-intent | Reflect is prompted for root-cause decisions with exact values; JSON transcripts keep concrete action artifacts; per-commit-with-diffs deepening captures *why* the code changed, not just that it did. |
+| Near-zero-burden | No config flags to choose, no CLI to run, no tags to maintain; one LLM call per session start, everything else local. |
+
+## 9. Verification gates
+
+Ship gates, in order:
+
+1. **Reflect-restored benchmark:** the restored runtime must recover **~25 mean corrections at n=2 on identical banks** to the original reflect run. This proves the restoration is faithful before anything is layered on.
+2. **Reflect+pages benchmark:** with per-turn section injection enabled, the score **must not regress** vs reflect-alone. Section injection earns its place by not hurting; any regression points at the floor/budget tuning.
+3. **Live system suite:** existing hook/integration suite updated for the new path — reflect caching + per-turn re-injection, section index build/refresh, score-floor silence, JSON transcript shape, action-entry compaction. Deterministic pieces (matcher scoring, budget trim, provenance formatting, transcript serialization) as fast unit tests.
+
+## 10. Non-goals / deferred
+
+- Any per-turn LLM or server call for injection (explicitly excluded — the local matcher is the whole point).
+- Semantic/embedding-based section matching (revisit only if lexical matching demonstrably misses; start dumb).
+- Client-side page splitting or curation (server/curator concern, §6).
+- Progressive-deepening implementation details (worker scheduling, checkpoint format) — phased separately per §7.
+
+## 11. File map (anticipated)
+
+- `src/core/reflect.ts` (restored) — session reflect call + per-session cache.
+- `src/core/section-index.ts` (new) — page → sections split, lexical scorer, budget trim, provenance formatting; pure/unit-testable.
+- `src/core/hook.ts` — drop recall; inject cached reflect + matched sections; index refresh on roster cadence.
+- `src/core/session-start.ts` — cold-repo seed (unchanged) + reflect trigger wiring + initial index build.
+- `src/core/transcript.ts` (new or reworked) — JSON turn serialization + action-entry compaction for the Stop hook.
+- `src/core/missions.ts` / `src/core/hindsight.ts` — remove `entity_labels` and tag-scoped page triggers; keep seeded taxonomy + `capture_initiative`.
+- `src/core/config.ts` — remove any behavior flags; keep env/naming/harness + `pageRefreshEveryTurns`.
+- Tests alongside each.

helm/hindsight/Chart.lock

@@ -1,6 +0,0 @@
-dependencies:
-- name: postgresql
-  repository: https://charts.bitnami.com/bitnami
-  version: 15.5.38
-digest: sha256:f67c7612736803ece8a669f8ca6b0555f3b78557bc0ecb732aa2e43f0df7750d
-generated: "2025-12-10T17:20:57.058794+01:00"

helm/hindsight/Chart.yaml

@@ -2,8 +2,8 @@ apiVersion: v2
 name: hindsight
 description: Hindsight helm chart
 type: application
-version: 0.1.4
-appVersion: "0.1.4"
+version: 0.9.0
+appVersion: "0.9.0"
 keywords:
   - ai
   - memory

helm/hindsight/README.md

@@ -66,13 +66,13 @@ helm install hindsight ./helm/hindsight -n hindsight --create-namespace -f value
 
 | Parameter | Description | Default |
 |-----------|-------------|---------|
-| `version` | Default image tag for all components | `0.1.0` |
+| `version` | Default image tag for all components | Chart `appVersion` |
 | `api.enabled` | Enable the API component | `true` |
-| `api.image.repository` | API image repository | `hindsight/api` |
+| `api.image.repository` | API image repository | `ghcr.io/vectorize-io/hindsight-api` |
 | `api.image.tag` | API image tag (defaults to `version`) | - |
 | `api.service.port` | API service port | `8888` |
 | `controlPlane.enabled` | Enable the control plane | `true` |
-| `controlPlane.image.repository` | Control plane image repository | `hindsight/control-plane` |
+| `controlPlane.image.repository` | Control plane image repository | `ghcr.io/vectorize-io/hindsight-control-plane` |
 | `controlPlane.image.tag` | Control plane image tag (defaults to `version`) | - |
 | `controlPlane.service.port` | Control plane service port | `3000` |
 | `postgresql.enabled` | Deploy PostgreSQL as subchart | `true` |

helm/hindsight/templates/_helpers.tpl

@@ -80,6 +80,22 @@ Control plane selector labels
 app.kubernetes.io/component: control-plane
 {{- end }}
 
+{{/*
+Worker labels
+*/}}
+{{- define "hindsight.worker.labels" -}}
+{{ include "hindsight.labels" . }}
+app.kubernetes.io/component: worker
+{{- end }}
+
+{{/*
+Worker selector labels
+*/}}
+{{- define "hindsight.worker.selectorLabels" -}}
+{{ include "hindsight.selectorLabels" . }}
+app.kubernetes.io/component: worker
+{{- end }}
+
 {{/*
 Create the name of the service account to use
 */}}
@@ -110,3 +126,46 @@ API URL for control plane
 {{- define "hindsight.apiUrl" -}}
 {{- printf "http://%s-api:%d" (include "hindsight.fullname" .) (.Values.api.service.port | int) }}
 {{- end }}
+
+{{/*
+TEI reranker labels
+*/}}
+{{- define "hindsight.tei.reranker.labels" -}}
+{{ include "hindsight.labels" . }}
+app.kubernetes.io/component: tei-reranker
+{{- end }}
+
+{{/*
+TEI reranker selector labels
+*/}}
+{{- define "hindsight.tei.reranker.selectorLabels" -}}
+{{ include "hindsight.selectorLabels" . }}
+app.kubernetes.io/component: tei-reranker
+{{- end }}
+
+{{/*
+TEI embedding labels
+*/}}
+{{- define "hindsight.tei.embedding.labels" -}}
+{{ include "hindsight.labels" . }}
+app.kubernetes.io/component: tei-embedding
+{{- end }}
+
+{{/*
+TEI embedding selector labels
+*/}}
+{{- define "hindsight.tei.embedding.selectorLabels" -}}
+{{ include "hindsight.selectorLabels" . }}
+app.kubernetes.io/component: tei-embedding
+{{- end }}
+
+{{/*
+Get the name of the secret to use
+*/}}
+{{- define "hindsight.secretName" -}}
+{{- if .Values.existingSecret }}
+{{- .Values.existingSecret }}
+{{- else }}
+{{- printf "%s-secret" (include "hindsight.fullname" .) }}
+{{- end }}
+{{- end }}

helm/hindsight/templates/api-deployment.yaml

@@ -15,7 +15,9 @@ spec:
   template:
     metadata:
       annotations:
+        {{- if not .Values.existingSecret }}
         checksum/secret: {{ include (print $.Template.BasePath "/secret.yaml") . | sha256sum }}
+        {{- end }}
         {{- with .Values.podAnnotations }}
         {{- toYaml . | nindent 8 }}
         {{- end }}
@@ -31,44 +33,94 @@ spec:
       - name: api
         securityContext:
           {{- toYaml .Values.securityContext | nindent 10 }}
-        image: "{{ .Values.api.image.repository }}:{{ .Values.api.image.tag | default .Values.version }}"
+        image: "{{ .Values.api.image.repository }}:{{ .Values.api.image.tag | default .Values.version | default .Chart.AppVersion }}"
         imagePullPolicy: {{ .Values.api.image.pullPolicy }}
         ports:
         - name: http
           containerPort: {{ .Values.api.service.targetPort }}
           protocol: TCP
+        {{- if .Values.existingSecret }}
+        envFrom:
+        - secretRef:
+            name: {{ .Values.existingSecret }}
+        {{- end }}
         env:
-        - name: HINDSIGHT_API_DATABASE_URL
-          value: {{ include "hindsight.databaseUrl" . | quote }}
+        {{- /* POSTGRES_PASSWORD must be defined before DATABASE_URL for $(VAR) interpolation */}}
         {{- if not .Values.postgresql.enabled }}
         - name: POSTGRES_PASSWORD
           valueFrom:
             secretKeyRef:
-              name: {{ include "hindsight.fullname" . }}-secret
+              name: {{ include "hindsight.secretName" . }}
               key: postgres-password
         {{- end }}
+        - name: HINDSIGHT_API_DATABASE_URL
+          value: {{ include "hindsight.databaseUrl" . | quote }}
+        {{- /* Disable internal worker when dedicated workers are enabled */}}
+        {{- if .Values.worker.enabled }}
+        - name: HINDSIGHT_API_WORKER_ENABLED
+          value: "false"
+        {{- end }}
+        {{- /* Explicitly set port to override K8s service discovery env var (HINDSIGHT_API_PORT) */}}
+        - name: HINDSIGHT_API_PORT
+          value: {{ .Values.api.service.targetPort | quote }}
         {{- range $key, $value := .Values.api.env }}
         - name: {{ $key }}
           value: {{ $value | quote }}
         {{- end }}
+        {{- if .Values.tei.reranker.enabled }}
+        - name: HINDSIGHT_API_RERANKER_PROVIDER
+          value: "tei"
+        - name: HINDSIGHT_API_RERANKER_TEI_URL
+          value: "http://{{ include "hindsight.fullname" . }}-tei-reranker:{{ .Values.tei.reranker.port }}"
+        {{- end }}
+        {{- if .Values.tei.embedding.enabled }}
+        - name: HINDSIGHT_API_EMBEDDINGS_PROVIDER
+          value: "tei"
+        - name: HINDSIGHT_API_EMBEDDINGS_TEI_URL
+          value: "http://{{ include "hindsight.fullname" . }}-tei-embedding:{{ .Values.tei.embedding.port }}"
+        {{- end }}
+        {{- /* Only use api.secrets when not using existingSecret (for chart-managed secrets) */}}
+        {{- if not .Values.existingSecret }}
         {{- range $key, $value := .Values.api.secrets }}
         - name: {{ $key }}
           valueFrom:
             secretKeyRef:
-              name: {{ include "hindsight.fullname" $ }}-secret
+              name: {{ include "hindsight.secretName" $ }}
               key: {{ $key }}
         {{- end }}
+        {{- end }}
         livenessProbe:
           {{- toYaml .Values.api.livenessProbe | nindent 10 }}
         readinessProbe:
           {{- toYaml .Values.api.readinessProbe | nindent 10 }}
         resources:
           {{- toYaml .Values.api.resources | nindent 10 }}
+        {{- if or .Values.api.persistence.modelCache.enabled .Values.api.extraVolumeMounts }}
+        volumeMounts:
+        {{- if .Values.api.persistence.modelCache.enabled }}
+        - name: model-cache
+          mountPath: /home/hindsight/.cache
+        {{- end }}
+        {{- with .Values.api.extraVolumeMounts }}
+        {{- toYaml . | nindent 8 }}
+        {{- end }}
+        {{- end }}
+      {{- if or .Values.api.persistence.modelCache.enabled .Values.api.extraVolumes }}
+      volumes:
+      {{- if .Values.api.persistence.modelCache.enabled }}
+      - name: model-cache
+        persistentVolumeClaim:
+          claimName: {{ include "hindsight.fullname" . }}-api-model-cache
+      {{- end }}
+      {{- with .Values.api.extraVolumes }}
+      {{- toYaml . | nindent 6 }}
+      {{- end }}
+      {{- end }}
       {{- with .Values.nodeSelector }}
       nodeSelector:
         {{- toYaml . | nindent 8 }}
       {{- end }}
-      {{- with .Values.affinity }}
+      {{- with (.Values.api.affinity | default .Values.affinity) }}
       affinity:
         {{- toYaml . | nindent 8 }}
       {{- end }}

helm/hindsight/templates/api-model-cache-pvc.yaml

@@ -0,0 +1,21 @@
+{{- if and .Values.api.enabled .Values.api.persistence.modelCache.enabled }}
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: {{ include "hindsight.fullname" . }}-api-model-cache
+  labels:
+    {{- include "hindsight.api.labels" . | nindent 4 }}
+  {{- with .Values.api.persistence.modelCache.annotations }}
+  annotations:
+    {{- toYaml . | nindent 4 }}
+  {{- end }}
+spec:
+  accessModes:
+    {{- toYaml .Values.api.persistence.modelCache.accessModes | nindent 4 }}
+  {{- if .Values.api.persistence.modelCache.storageClass }}
+  storageClassName: {{ .Values.api.persistence.modelCache.storageClass }}
+  {{- end }}
+  resources:
+    requests:
+      storage: {{ .Values.api.persistence.modelCache.size }}
+{{- end }}

helm/hindsight/templates/controlplane-deployment.yaml

@@ -15,7 +15,9 @@ spec:
   template:
     metadata:
       annotations:
+        {{- if not .Values.existingSecret }}
         checksum/secret: {{ include (print $.Template.BasePath "/secret.yaml") . | sha256sum }}
+        {{- end }}
         {{- with .Values.podAnnotations }}
         {{- toYaml . | nindent 8 }}
         {{- end }}
@@ -31,12 +33,17 @@ spec:
       - name: control-plane
         securityContext:
           {{- toYaml .Values.securityContext | nindent 10 }}
-        image: "{{ .Values.controlPlane.image.repository }}:{{ .Values.controlPlane.image.tag | default .Values.version }}"
+        image: "{{ .Values.controlPlane.image.repository }}:{{ .Values.controlPlane.image.tag | default .Values.version | default .Chart.AppVersion }}"
         imagePullPolicy: {{ .Values.controlPlane.image.pullPolicy }}
         ports:
         - name: http
           containerPort: {{ .Values.controlPlane.service.targetPort }}
           protocol: TCP
+        {{- if .Values.existingSecret }}
+        envFrom:
+        - secretRef:
+            name: {{ .Values.existingSecret }}
+        {{- end }}
         env:
         - name: HINDSIGHT_CP_DATAPLANE_API_URL
           value: {{ include "hindsight.apiUrl" . | quote }}
@@ -44,13 +51,16 @@ spec:
         - name: {{ $key }}
           value: {{ $value | quote }}
         {{- end }}
+        {{- /* Only use controlPlane.secrets when not using existingSecret (for chart-managed secrets) */}}
+        {{- if not .Values.existingSecret }}
         {{- range $key, $value := .Values.controlPlane.secrets }}
         - name: {{ $key }}
           valueFrom:
             secretKeyRef:
-              name: {{ include "hindsight.fullname" $ }}-secret
+              name: {{ include "hindsight.secretName" $ }}
               key: {{ $key }}
         {{- end }}
+        {{- end }}
         livenessProbe:
           {{- toYaml .Values.controlPlane.livenessProbe | nindent 10 }}
         readinessProbe:
@@ -61,7 +71,7 @@ spec:
       nodeSelector:
         {{- toYaml . | nindent 8 }}
       {{- end }}
-      {{- with .Values.affinity }}
+      {{- with (.Values.controlPlane.affinity | default .Values.affinity) }}
       affinity:
         {{- toYaml . | nindent 8 }}
       {{- end }}

helm/hindsight/templates/pdb.yaml

@@ -0,0 +1,56 @@
+{{- if and .Values.api.enabled .Values.api.podDisruptionBudget.enabled }}
+apiVersion: policy/v1
+kind: PodDisruptionBudget
+metadata:
+  name: {{ include "hindsight.fullname" . }}-api
+  labels:
+    {{- include "hindsight.api.labels" . | nindent 4 }}
+spec:
+  {{- if .Values.api.podDisruptionBudget.minAvailable }}
+  minAvailable: {{ .Values.api.podDisruptionBudget.minAvailable }}
+  {{- end }}
+  {{- if .Values.api.podDisruptionBudget.maxUnavailable }}
+  maxUnavailable: {{ .Values.api.podDisruptionBudget.maxUnavailable }}
+  {{- end }}
+  selector:
+    matchLabels:
+      {{- include "hindsight.api.selectorLabels" . | nindent 6 }}
+{{- end }}
+---
+{{- if and .Values.controlPlane.enabled .Values.controlPlane.podDisruptionBudget.enabled }}
+apiVersion: policy/v1
+kind: PodDisruptionBudget
+metadata:
+  name: {{ include "hindsight.fullname" . }}-control-plane
+  labels:
+    {{- include "hindsight.controlPlane.labels" . | nindent 4 }}
+spec:
+  {{- if .Values.controlPlane.podDisruptionBudget.minAvailable }}
+  minAvailable: {{ .Values.controlPlane.podDisruptionBudget.minAvailable }}
+  {{- end }}
+  {{- if .Values.controlPlane.podDisruptionBudget.maxUnavailable }}
+  maxUnavailable: {{ .Values.controlPlane.podDisruptionBudget.maxUnavailable }}
+  {{- end }}
+  selector:
+    matchLabels:
+      {{- include "hindsight.controlPlane.selectorLabels" . | nindent 6 }}
+{{- end }}
+---
+{{- if and .Values.worker.enabled .Values.worker.podDisruptionBudget.enabled }}
+apiVersion: policy/v1
+kind: PodDisruptionBudget
+metadata:
+  name: {{ include "hindsight.fullname" . }}-worker
+  labels:
+    {{- include "hindsight.worker.labels" . | nindent 4 }}
+spec:
+  {{- if .Values.worker.podDisruptionBudget.minAvailable }}
+  minAvailable: {{ .Values.worker.podDisruptionBudget.minAvailable }}
+  {{- end }}
+  {{- if .Values.worker.podDisruptionBudget.maxUnavailable }}
+  maxUnavailable: {{ .Values.worker.podDisruptionBudget.maxUnavailable }}
+  {{- end }}
+  selector:
+    matchLabels:
+      {{- include "hindsight.worker.selectorLabels" . | nindent 6 }}
+{{- end }}

helm/hindsight/templates/secret.yaml

@@ -1,7 +1,8 @@
+{{- if not .Values.existingSecret }}
 apiVersion: v1
 kind: Secret
 metadata:
-  name: {{ include "hindsight.fullname" . }}-secret
+  name: {{ include "hindsight.secretName" . }}
   labels:
     {{- include "hindsight.labels" . | nindent 4 }}
 type: Opaque
@@ -15,3 +16,4 @@ data:
   {{- if and (not .Values.postgresql.enabled) .Values.postgresql.external.password }}
   postgres-password: {{ .Values.postgresql.external.password | b64enc | quote }}
   {{- end }}
+{{- end }}

helm/hindsight/templates/tei-embedding-deployment.yaml

@@ -0,0 +1,76 @@
+{{- if .Values.tei.embedding.enabled }}
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: {{ include "hindsight.fullname" . }}-tei-embedding
+  labels:
+    {{- include "hindsight.tei.embedding.labels" . | nindent 4 }}
+spec:
+  replicas: {{ .Values.tei.embedding.replicaCount }}
+  selector:
+    matchLabels:
+      {{- include "hindsight.tei.embedding.selectorLabels" . | nindent 6 }}
+  template:
+    metadata:
+      {{- with .Values.podAnnotations }}
+      annotations:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      labels:
+        {{- include "hindsight.tei.embedding.selectorLabels" . | nindent 8 }}
+    spec:
+      {{- if .Values.serviceAccount.create }}
+      serviceAccountName: {{ include "hindsight.serviceAccountName" . }}
+      {{- end }}
+      securityContext:
+        {{- toYaml .Values.podSecurityContext | nindent 8 }}
+      containers:
+      - name: tei-embedding
+        securityContext:
+          {{- toYaml .Values.securityContext | nindent 10 }}
+        image: "{{ .Values.tei.embedding.image.repository }}:{{ .Values.tei.embedding.image.tag }}"
+        imagePullPolicy: {{ .Values.tei.embedding.image.pullPolicy }}
+        args:
+          - "--model-id"
+          - {{ .Values.tei.embedding.model | quote }}
+          - "--hostname"
+          - "0.0.0.0"
+          {{- range .Values.tei.embedding.args }}
+          - {{ . | quote }}
+          {{- end }}
+        ports:
+        - name: http
+          containerPort: {{ .Values.tei.embedding.port }}
+          protocol: TCP
+        env:
+        - name: PORT
+          value: {{ .Values.tei.embedding.port | quote }}
+        {{- range $key, $value := .Values.tei.embedding.env }}
+        - name: {{ $key }}
+          value: {{ $value | quote }}
+        {{- end }}
+        livenessProbe:
+          {{- toYaml .Values.tei.embedding.livenessProbe | nindent 10 }}
+        readinessProbe:
+          {{- toYaml .Values.tei.embedding.readinessProbe | nindent 10 }}
+        resources:
+          {{- toYaml .Values.tei.embedding.resources | nindent 10 }}
+        volumeMounts:
+        - name: model-cache
+          mountPath: /data
+      volumes:
+      - name: model-cache
+        emptyDir: {}
+      {{- with .Values.nodeSelector }}
+      nodeSelector:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      {{- with .Values.affinity }}
+      affinity:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      {{- with .Values.tolerations }}
+      tolerations:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+{{- end }}

helm/hindsight/templates/tei-embedding-service.yaml

@@ -0,0 +1,17 @@
+{{- if .Values.tei.embedding.enabled }}
+apiVersion: v1
+kind: Service
+metadata:
+  name: {{ include "hindsight.fullname" . }}-tei-embedding
+  labels:
+    {{- include "hindsight.tei.embedding.labels" . | nindent 4 }}
+spec:
+  type: ClusterIP
+  ports:
+  - port: {{ .Values.tei.embedding.port }}
+    targetPort: http
+    protocol: TCP
+    name: http
+  selector:
+    {{- include "hindsight.tei.embedding.selectorLabels" . | nindent 4 }}
+{{- end }}

helm/hindsight/templates/tei-reranker-deployment.yaml

@@ -0,0 +1,76 @@
+{{- if .Values.tei.reranker.enabled }}
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: {{ include "hindsight.fullname" . }}-tei-reranker
+  labels:
+    {{- include "hindsight.tei.reranker.labels" . | nindent 4 }}
+spec:
+  replicas: {{ .Values.tei.reranker.replicaCount }}
+  selector:
+    matchLabels:
+      {{- include "hindsight.tei.reranker.selectorLabels" . | nindent 6 }}
+  template:
+    metadata:
+      {{- with .Values.podAnnotations }}
+      annotations:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      labels:
+        {{- include "hindsight.tei.reranker.selectorLabels" . | nindent 8 }}
+    spec:
+      {{- if .Values.serviceAccount.create }}
+      serviceAccountName: {{ include "hindsight.serviceAccountName" . }}
+      {{- end }}
+      securityContext:
+        {{- toYaml .Values.podSecurityContext | nindent 8 }}
+      containers:
+      - name: tei-reranker
+        securityContext:
+          {{- toYaml .Values.securityContext | nindent 10 }}
+        image: "{{ .Values.tei.reranker.image.repository }}:{{ .Values.tei.reranker.image.tag }}"
+        imagePullPolicy: {{ .Values.tei.reranker.image.pullPolicy }}
+        args:
+          - "--model-id"
+          - {{ .Values.tei.reranker.model | quote }}
+          - "--hostname"
+          - "0.0.0.0"
+          {{- range .Values.tei.reranker.args }}
+          - {{ . | quote }}
+          {{- end }}
+        ports:
+        - name: http
+          containerPort: {{ .Values.tei.reranker.port }}
+          protocol: TCP
+        env:
+        - name: PORT
+          value: {{ .Values.tei.reranker.port | quote }}
+        {{- range $key, $value := .Values.tei.reranker.env }}
+        - name: {{ $key }}
+          value: {{ $value | quote }}
+        {{- end }}
+        livenessProbe:
+          {{- toYaml .Values.tei.reranker.livenessProbe | nindent 10 }}
+        readinessProbe:
+          {{- toYaml .Values.tei.reranker.readinessProbe | nindent 10 }}
+        resources:
+          {{- toYaml .Values.tei.reranker.resources | nindent 10 }}
+        volumeMounts:
+        - name: model-cache
+          mountPath: /data
+      volumes:
+      - name: model-cache
+        emptyDir: {}
+      {{- with .Values.nodeSelector }}
+      nodeSelector:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      {{- with .Values.affinity }}
+      affinity:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      {{- with .Values.tolerations }}
+      tolerations:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+{{- end }}

helm/hindsight/templates/tei-reranker-service.yaml

@@ -0,0 +1,17 @@
+{{- if .Values.tei.reranker.enabled }}
+apiVersion: v1
+kind: Service
+metadata:
+  name: {{ include "hindsight.fullname" . }}-tei-reranker
+  labels:
+    {{- include "hindsight.tei.reranker.labels" . | nindent 4 }}
+spec:
+  type: ClusterIP
+  ports:
+  - port: {{ .Values.tei.reranker.port }}
+    targetPort: http
+    protocol: TCP
+    name: http
+  selector:
+    {{- include "hindsight.tei.reranker.selectorLabels" . | nindent 4 }}
+{{- end }}

helm/hindsight/templates/worker-service.yaml

@@ -0,0 +1,25 @@
+{{- if .Values.worker.enabled }}
+apiVersion: v1
+kind: Service
+metadata:
+  name: {{ include "hindsight.fullname" . }}-worker
+  labels:
+    {{- include "hindsight.worker.labels" . | nindent 4 }}
+  {{- if .Values.podAnnotations }}
+  annotations:
+    {{- /* Common Prometheus annotations for metrics scraping */}}
+    prometheus.io/scrape: "true"
+    prometheus.io/port: {{ .Values.worker.service.port | quote }}
+    prometheus.io/path: "/metrics"
+  {{- end }}
+spec:
+  # Headless service for StatefulSet (enables stable DNS names like worker-0.worker.namespace)
+  clusterIP: None
+  ports:
+  - port: {{ .Values.worker.service.port }}
+    targetPort: {{ .Values.worker.service.targetPort }}
+    protocol: TCP
+    name: http
+  selector:
+    {{- include "hindsight.worker.selectorLabels" . | nindent 4 }}
+{{- end }}

helm/hindsight/templates/worker-statefulset.yaml

@@ -0,0 +1,142 @@
+{{- if .Values.worker.enabled }}
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+  name: {{ include "hindsight.fullname" . }}-worker
+  labels:
+    {{- include "hindsight.worker.labels" . | nindent 4 }}
+spec:
+  serviceName: {{ include "hindsight.fullname" . }}-worker
+  replicas: {{ .Values.worker.replicaCount }}
+  selector:
+    matchLabels:
+      {{- include "hindsight.worker.selectorLabels" . | nindent 6 }}
+  template:
+    metadata:
+      annotations:
+        {{- if not .Values.existingSecret }}
+        checksum/secret: {{ include (print $.Template.BasePath "/secret.yaml") . | sha256sum }}
+        {{- end }}
+        {{- with .Values.podAnnotations }}
+        {{- toYaml . | nindent 8 }}
+        {{- end }}
+      labels:
+        {{- include "hindsight.worker.selectorLabels" . | nindent 8 }}
+    spec:
+      {{- if .Values.serviceAccount.create }}
+      serviceAccountName: {{ include "hindsight.serviceAccountName" . }}
+      {{- end }}
+      securityContext:
+        {{- toYaml .Values.podSecurityContext | nindent 8 }}
+      containers:
+      - name: worker
+        securityContext:
+          {{- toYaml .Values.securityContext | nindent 10 }}
+        image: "{{ .Values.worker.image.repository }}:{{ .Values.worker.image.tag | default .Values.version | default .Chart.AppVersion }}"
+        imagePullPolicy: {{ .Values.worker.image.pullPolicy }}
+        command: ["hindsight-worker"]
+        ports:
+        - name: http
+          containerPort: {{ .Values.worker.service.targetPort }}
+          protocol: TCP
+        {{- if .Values.existingSecret }}
+        envFrom:
+        - secretRef:
+            name: {{ .Values.existingSecret }}
+        {{- end }}
+        env:
+        {{- /* POSTGRES_PASSWORD must be defined before DATABASE_URL for $(VAR) interpolation */}}
+        {{- if not .Values.postgresql.enabled }}
+        - name: POSTGRES_PASSWORD
+          valueFrom:
+            secretKeyRef:
+              name: {{ include "hindsight.secretName" . }}
+              key: postgres-password
+        {{- end }}
+        - name: HINDSIGHT_API_DATABASE_URL
+          value: {{ include "hindsight.databaseUrl" . | quote }}
+        {{- /* Worker ID uses pod name (StatefulSet provides stable names like worker-0, worker-1) */}}
+        - name: HINDSIGHT_API_WORKER_ID
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.name
+        {{- /* Explicitly set port to override K8s service discovery env var (HINDSIGHT_API_PORT) */}}
+        - name: HINDSIGHT_API_PORT
+          value: {{ .Values.worker.service.targetPort | quote }}
+        {{- /* Inherit LLM config from api.env, then apply worker-specific env.
+               Merge (worker.env wins) so a key set in both does not emit a
+               duplicate env entry, which server-side apply rejects. */}}
+        {{- range $key, $value := merge (deepCopy (.Values.worker.env | default dict)) (.Values.api.env | default dict) }}
+        - name: {{ $key }}
+          value: {{ $value | quote }}
+        {{- end }}
+        {{- /* Only use secrets when not using existingSecret */}}
+        {{- if not .Values.existingSecret }}
+        {{- /* Inherit secrets from api.secrets */}}
+        {{- range $key, $value := .Values.api.secrets }}
+        - name: {{ $key }}
+          valueFrom:
+            secretKeyRef:
+              name: {{ include "hindsight.secretName" $ }}
+              key: {{ $key }}
+        {{- end }}
+        {{- /* Worker-specific secrets (can override api.secrets) */}}
+        {{- range $key, $value := .Values.worker.secrets }}
+        - name: {{ $key }}
+          valueFrom:
+            secretKeyRef:
+              name: {{ include "hindsight.secretName" $ }}
+              key: {{ $key }}
+        {{- end }}
+        {{- end }}
+        livenessProbe:
+          {{- toYaml .Values.worker.livenessProbe | nindent 10 }}
+        readinessProbe:
+          {{- toYaml .Values.worker.readinessProbe | nindent 10 }}
+        resources:
+          {{- toYaml .Values.worker.resources | nindent 10 }}
+        {{- if or .Values.worker.persistence.modelCache.enabled .Values.worker.extraVolumeMounts }}
+        volumeMounts:
+        {{- if .Values.worker.persistence.modelCache.enabled }}
+        - name: model-cache
+          mountPath: /home/hindsight/.cache
+        {{- end }}
+        {{- with .Values.worker.extraVolumeMounts }}
+        {{- toYaml . | nindent 8 }}
+        {{- end }}
+        {{- end }}
+      {{- with .Values.nodeSelector }}
+      nodeSelector:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      {{- with (.Values.worker.affinity | default .Values.affinity) }}
+      affinity:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      {{- with .Values.tolerations }}
+      tolerations:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      {{- with .Values.worker.extraVolumes }}
+      volumes:
+      {{- toYaml . | nindent 6 }}
+      {{- end }}
+  {{- if .Values.worker.persistence.modelCache.enabled }}
+  volumeClaimTemplates:
+  - metadata:
+      name: model-cache
+      {{- with .Values.worker.persistence.modelCache.annotations }}
+      annotations:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+    spec:
+      accessModes:
+        {{- toYaml .Values.worker.persistence.modelCache.accessModes | nindent 8 }}
+      {{- if .Values.worker.persistence.modelCache.storageClass }}
+      storageClassName: {{ .Values.worker.persistence.modelCache.storageClass }}
+      {{- end }}
+      resources:
+        requests:
+          storage: {{ .Values.worker.persistence.modelCache.size }}
+  {{- end }}
+{{- end }}

helm/hindsight/values.yaml

@@ -1,10 +1,17 @@
 # Default values for hindsight
 
-# Chart version - use this to set a consistent image tag across all components
-version: "0.1.1"
+# Global version override - use this to set a consistent image tag across all components
+# If not set, defaults to Chart.appVersion from Chart.yaml
+# version: ""
 
-# Global settings
-replicaCount: 1
+# Use an existing secret instead of creating one from values
+# When set, all keys from this secret are injected as environment variables via envFrom
+# Required keys:
+#   - postgres-password: PostgreSQL password (when postgresql.enabled=false)
+# Optional keys (any key becomes an env var):
+#   - HINDSIGHT_API_LLM_API_KEY: API key for LLM provider
+#   - Any other env vars you want to inject
+# existingSecret: "my-hindsight-secret"
 
 # Image settings for api
 api:
@@ -29,16 +36,22 @@ api:
       cpu: 500m
       memory: 1Gi
 
-  # Liveness and readiness probes
+  # Liveness and readiness probes.
+  # Liveness uses /health/live, which performs no database access: a slow or
+  # unreachable database must gate traffic (readiness), never restart pods.
+  # Needs an image from this chart's appVersion or newer — older ones serve
+  # /health only, and would fail this probe with a 404.
   livenessProbe:
     httpGet:
-      path: /health
+      path: /health/live
       port: 8888
     initialDelaySeconds: 30
     periodSeconds: 10
     timeoutSeconds: 5
     failureThreshold: 3
 
+  # Readiness checks the database, so a pod that cannot reach it is pulled out
+  # of the Service and put back once the database recovers.
   readinessProbe:
     httpGet:
       path: /health
@@ -48,6 +61,48 @@ api:
     timeoutSeconds: 3
     failureThreshold: 3
 
+  # Pod disruption budget
+  podDisruptionBudget:
+    enabled: false
+    minAvailable: 1
+    # maxUnavailable: 1
+
+  # Pod affinity/anti-affinity (overrides global affinity for this component)
+  # affinity: {}
+
+  # Persistent volume for local model cache (reranker, embeddings)
+  # Models are downloaded to /home/hindsight/.cache on first use.
+  # Without persistence, models are re-downloaded on every pod restart.
+  #
+  # For production, prefer baking models into a custom image instead of
+  # enabling this PVC: image layers are pulled once per node and cached
+  # for free, while a PVC adds storage cost, pins pods to a node
+  # (ReadWriteOnce), and needs lifecycle management on uninstall/upgrade.
+  # See docs: developer/installation#bundling-custom-models-in-a-custom-image
+  persistence:
+    modelCache:
+      enabled: false
+      size: 5Gi
+      storageClass: ""
+      accessModes:
+        - ReadWriteOnce
+      annotations: {}
+
+  # Extra volume mounts for the api container
+  # e.g.
+  # extraVolumeMounts:
+  #   - name: my-volume
+  #     mountPath: /mnt/my-volume
+  extraVolumeMounts: []
+
+  # Extra volumes for the api pod
+  # e.g.
+  # extraVolumes:
+  #   - name: my-volume
+  #     configMap:
+  #       name: my-configmap
+  extraVolumes: []
+
   # Environment variables
   env:
     #HINDSIGHT_API_LLM_PROVIDER: "groq"
@@ -58,6 +113,106 @@ api:
     # HINDSIGHT_API_LLM_API_KEY: "your-api-key"
     # HINDSIGHT_API_LLM_BASE_URL: "https://api.groq.com/openai/v1"
 
+# Worker settings (distributed task processing)
+# When enabled, dedicated worker pods process tasks and the API's internal worker is disabled
+worker:
+  enabled: false
+  replicaCount: 2
+  image:
+    repository: ghcr.io/vectorize-io/hindsight-api
+    pullPolicy: IfNotPresent
+    # tag: ""  # defaults to .Values.version, then Chart.appVersion if not specified
+
+  service:
+    # Service for metrics scraping (headless for StatefulSet)
+    port: 8889
+    targetPort: 8889
+
+  # Resource limits and requests
+  resources:
+    limits:
+      cpu: 2000m
+      memory: 4Gi
+    requests:
+      cpu: 500m
+      memory: 1Gi
+
+  # Liveness and readiness probes.
+  # Liveness uses /health/live, which performs no database access. Restarting a
+  # worker whose database is merely slow requeues its claimed operations with
+  # retry_count incremented, so DB checks must stay out of liveness.
+  # Needs an image from this chart's appVersion or newer — older ones serve
+  # /health only, and would fail this probe with a 404.
+  livenessProbe:
+    httpGet:
+      path: /health/live
+      port: 8889
+    initialDelaySeconds: 30
+    periodSeconds: 10
+    timeoutSeconds: 5
+    failureThreshold: 3
+
+  readinessProbe:
+    httpGet:
+      path: /health
+      port: 8889
+    initialDelaySeconds: 10
+    periodSeconds: 5
+    timeoutSeconds: 3
+    failureThreshold: 3
+
+  # Worker-specific environment variables
+  env:
+    # Poll interval in milliseconds (how often to check for new tasks)
+    HINDSIGHT_API_WORKER_POLL_INTERVAL_MS: "500"
+    # Number of tasks to claim per poll cycle
+    HINDSIGHT_API_WORKER_BATCH_SIZE: "10"
+    # Max retries before marking a task as failed
+    HINDSIGHT_API_WORKER_MAX_RETRIES: "3"
+    # HTTP port for metrics/health (matches service.targetPort)
+    HINDSIGHT_API_WORKER_HTTP_PORT: "8889"
+
+  # Pod disruption budget
+  podDisruptionBudget:
+    enabled: false
+    minAvailable: 1
+    # maxUnavailable: 1
+
+  # Pod affinity/anti-affinity (overrides global affinity for this component)
+  # affinity: {}
+
+  # Persistent volume for local model cache (reranker, embeddings)
+  # Uses volumeClaimTemplates since worker is a StatefulSet — one PVC per
+  # replica. For production, prefer baking models into a custom image; see
+  # api.persistence.modelCache above and docs:
+  # developer/installation#bundling-custom-models-in-a-custom-image
+  persistence:
+    modelCache:
+      enabled: false
+      size: 5Gi
+      storageClass: ""
+      accessModes:
+        - ReadWriteOnce
+      annotations: {}
+
+  # Extra volume mounts for the worker container
+  # e.g.
+  # extraVolumeMounts:
+  #   - name: my-volume
+  #     mountPath: /mnt/my-volume
+  extraVolumeMounts: []
+
+  # Extra volumes for the worker pod
+  # e.g.
+  # extraVolumes:
+  #   - name: my-volume
+  #     configMap:
+  #       name: my-configmap
+  extraVolumes: []
+
+  # Secret environment variables (inherited from api.secrets if not specified)
+  secrets: {}
+
 # Image settings for control plane
 controlPlane:
   enabled: true
@@ -98,6 +253,15 @@ controlPlane:
     timeoutSeconds: 3
     failureThreshold: 3
 
+  # Pod disruption budget
+  podDisruptionBudget:
+    enabled: false
+    minAvailable: 1
+    # maxUnavailable: 1
+
+  # Pod affinity/anti-affinity (overrides global affinity for this component)
+  # affinity: {}
+
   # Environment variables
   env:
     NODE_ENV: "production"
@@ -196,9 +360,87 @@ nodeSelector: {}
 # Tolerations
 tolerations: []
 
-# Affinity
+# Affinity (applied to all components unless overridden per-component)
 affinity: {}
 
+# TEI (Text Embeddings Inference) - optional standalone deployments
+# for reranking and/or embedding models
+tei:
+  reranker:
+    enabled: false
+    replicaCount: 1
+    image:
+      repository: ghcr.io/huggingface/text-embeddings-inference
+      tag: cpu-1.8.3
+      pullPolicy: IfNotPresent
+    model: "cross-encoder/ms-marco-MiniLM-L-6-v2"
+    port: 8090
+    args:
+      - "--auto-truncate"
+    env:
+      PAYLOAD_LIMIT: "10000000"
+      MAX_CLIENT_BATCH_SIZE: "256"
+    resources:
+      limits:
+        cpu: 2000m
+        memory: 2Gi
+      requests:
+        cpu: 500m
+        memory: 1Gi
+    livenessProbe:
+      httpGet:
+        path: /health
+        port: 8090
+      initialDelaySeconds: 30
+      periodSeconds: 10
+      timeoutSeconds: 5
+      failureThreshold: 6
+    readinessProbe:
+      httpGet:
+        path: /health
+        port: 8090
+      initialDelaySeconds: 15
+      periodSeconds: 5
+      timeoutSeconds: 3
+      failureThreshold: 3
+
+  embedding:
+    enabled: false
+    replicaCount: 1
+    image:
+      repository: ghcr.io/huggingface/text-embeddings-inference
+      tag: cpu-1.8.3
+      pullPolicy: IfNotPresent
+    model: "sentence-transformers/all-MiniLM-L6-v2"
+    port: 8091
+    args: []
+    env:
+      PAYLOAD_LIMIT: "10000000"
+      MAX_CLIENT_BATCH_SIZE: "256"
+    resources:
+      limits:
+        cpu: 2000m
+        memory: 2Gi
+      requests:
+        cpu: 500m
+        memory: 1Gi
+    livenessProbe:
+      httpGet:
+        path: /health
+        port: 8091
+      initialDelaySeconds: 30
+      periodSeconds: 10
+      timeoutSeconds: 5
+      failureThreshold: 6
+    readinessProbe:
+      httpGet:
+        path: /health
+        port: 8091
+      initialDelaySeconds: 15
+      periodSeconds: 5
+      timeoutSeconds: 3
+      failureThreshold: 3
+
 # Autoscaling
 autoscaling:
   enabled: false

hindsight-all-npm/.gitignore

@@ -0,0 +1,4 @@
+node_modules
+dist
+*.tgz
+.DS_Store

hindsight-all-npm/README.md

@@ -0,0 +1,80 @@
+# @vectorize-io/hindsight-all
+
+Node.js equivalent of the Python [`hindsight-all`](https://pypi.org/project/hindsight-all/) package — programmatic lifecycle manager for a local Hindsight daemon. Use this when you want to embed Hindsight in a Node application without hand-rolling subprocess management.
+
+This package deliberately does **not** ship an HTTP client. Once the daemon is running, talk to it with [`@vectorize-io/hindsight-client`](https://www.npmjs.com/package/@vectorize-io/hindsight-client) against `server.getBaseUrl()`. The two packages compose — one owns the daemon process, the other owns the HTTP API surface.
+
+## Requirements
+
+- **Node.js >= 22** — uses global `fetch` and `AbortSignal.timeout`.
+- **`uv` / `uvx`** on `PATH` — used to download and run the underlying `hindsight-embed` daemon on first use. Install via <https://docs.astral.sh/uv/>.
+
+## Install
+
+```bash
+npm install @vectorize-io/hindsight-all @vectorize-io/hindsight-client
+```
+
+## Example
+
+```ts
+import { HindsightServer, consoleLogger } from "@vectorize-io/hindsight-all";
+import { HindsightClient } from "@vectorize-io/hindsight-client";
+
+const server = new HindsightServer({
+  profile: "my-app",
+  port: 9077,
+  env: {
+    HINDSIGHT_API_LLM_PROVIDER: "anthropic",
+    HINDSIGHT_API_LLM_API_KEY: process.env.ANTHROPIC_API_KEY,
+    HINDSIGHT_API_LLM_MODEL: "claude-sonnet-4-20250514",
+    HINDSIGHT_EMBED_DAEMON_IDLE_TIMEOUT: "0",
+  },
+  logger: consoleLogger,
+});
+
+await server.start();
+
+const client = new HindsightClient({ baseUrl: server.getBaseUrl() });
+
+await client.retain("user-123", "User prefers dark mode and concise answers.", {
+  documentId: "pref-2026-04-01",
+});
+
+const recall = await client.recall("user-123", "what are the user preferences?");
+console.log(recall.results);
+
+await server.stop();
+```
+
+For a remote Hindsight API, skip `HindsightServer` entirely and just point `HindsightClient` at the remote URL.
+
+## Open config — forward-compatible with new daemon flags
+
+`HindsightServerOptions` is designed so every new environment variable or CLI flag in the underlying Hindsight daemon can be used without waiting for a wrapper release:
+
+- **`env`** accepts an arbitrary `Record<string, string>`. Every entry is exported into the daemon process and written into the profile config via `--env KEY=VALUE`.
+- **`extraProfileCreateArgs`** / **`extraDaemonStartArgs`** append raw args to the respective commands.
+
+## Development against a local checkout
+
+If you're hacking on the Python `hindsight-embed` package in the same monorepo, point the server at the local path — it'll use `uv run --directory <path>` instead of `uvx`:
+
+```ts
+new HindsightServer({
+  embedPackagePath: "/path/to/hindsight-embed",
+  // ...
+});
+```
+
+## API surface
+
+- `HindsightServer` — daemon lifecycle (`start`, `stop`, `checkHealth`, `getBaseUrl`, `getProfile`).
+- `Logger` interface plus `silentLogger` (default) and `consoleLogger` helpers.
+- `getEmbedCommand(opts)` — low-level helper that returns the `[cmd, ...args]` tuple used to invoke the underlying Python CLI.
+
+For memory operations (retain, recall, reflect, bank management, stats) use [`@vectorize-io/hindsight-client`](https://www.npmjs.com/package/@vectorize-io/hindsight-client).
+
+## License
+
+MIT

hindsight-all-npm/package.json

@@ -0,0 +1,57 @@
+{
+  "name": "@vectorize-io/hindsight-all",
+  "version": "0.9.0",
+  "description": "Node.js programmatic lifecycle manager for Hindsight — embeds a local hindsight daemon in a Node application. Pair with @vectorize-io/hindsight-client for memory operations.",
+  "main": "dist/index.js",
+  "types": "dist/index.d.ts",
+  "type": "module",
+  "exports": {
+    ".": {
+      "types": "./dist/index.d.ts",
+      "import": "./dist/index.js"
+    }
+  },
+  "keywords": [
+    "hindsight",
+    "hindsight-all",
+    "memory",
+    "ai",
+    "agent",
+    "long-term-memory",
+    "llm",
+    "embedded-server"
+  ],
+  "author": "Vectorize <[email protected]>",
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/vectorize-io/hindsight.git",
+    "directory": "hindsight-all-npm"
+  },
+  "files": [
+    "dist",
+    "README.md"
+  ],
+  "scripts": {
+    "build": "tsup",
+    "dev": "tsup --watch",
+    "clean": "rm -rf dist",
+    "test": "vitest run src",
+    "test:watch": "vitest src",
+    "prepublishOnly": "npm run clean && npm run build"
+  },
+  "devDependencies": {
+    "@types/node": "^22.0.0",
+    "tsup": "^8.5.1",
+    "typescript": "^5.7.0",
+    "vitest": "^4.1.2"
+  },
+  "engines": {
+    "node": ">=22"
+  },
+  "overrides": {
+    "rollup": "^4.59.0",
+    "picomatch": ">=2.3.2 <3.0.0 || >=4.0.4",
+    "vite": ">=8.0.5"
+  }
+}

hindsight-all-npm/src/command.test.ts

@@ -0,0 +1,36 @@
+import { describe, it, expect } from "vitest";
+import { getEmbedCommand } from "./command.js";
+
+describe("getEmbedCommand", () => {
+  it("defaults to uvx hindsight-embed@latest", () => {
+    expect(getEmbedCommand()).toEqual(["uvx", "hindsight-embed@latest"]);
+  });
+
+  it("honours an explicit version", () => {
+    expect(getEmbedCommand({ embedVersion: "0.5.0" })).toEqual(["uvx", "[email protected]"]);
+  });
+
+  it("treats an empty version as latest", () => {
+    expect(getEmbedCommand({ embedVersion: "" })).toEqual(["uvx", "hindsight-embed@latest"]);
+  });
+
+  it("uses uv run --directory when a local path is given", () => {
+    expect(getEmbedCommand({ embedPackagePath: "/abs/path" })).toEqual([
+      "uv",
+      "run",
+      "--directory",
+      "/abs/path",
+      "hindsight-embed",
+    ]);
+  });
+
+  it("local path takes precedence over version", () => {
+    expect(getEmbedCommand({ embedPackagePath: "/abs/path", embedVersion: "0.5.0" })).toEqual([
+      "uv",
+      "run",
+      "--directory",
+      "/abs/path",
+      "hindsight-embed",
+    ]);
+  });
+});

hindsight-all-npm/src/command.ts

@@ -0,0 +1,25 @@
+/**
+ * Resolve the command that invokes the `hindsight-embed` Python CLI.
+ *
+ * - If `embedPackagePath` is set, runs the package from a local checkout via
+ *   `uv run --directory <path> hindsight-embed`. Used for in-repo development.
+ * - Otherwise runs it via `uvx hindsight-embed@<version>` so no global install
+ *   is required.
+ *
+ * Returns the argv as `[command, ...baseArgs]` suitable for `spawn()` /
+ * `execFile()` (never shell-interpolated).
+ */
+export interface EmbedCommandOptions {
+  /** Version spec passed to uvx (e.g. "latest", "0.5.0"). Default: "latest". */
+  embedVersion?: string;
+  /** Local checkout path. When set, overrides `embedVersion` and uses `uv run`. */
+  embedPackagePath?: string;
+}
+
+export function getEmbedCommand(opts: EmbedCommandOptions = {}): string[] {
+  if (opts.embedPackagePath) {
+    return ["uv", "run", "--directory", opts.embedPackagePath, "hindsight-embed"];
+  }
+  const version = opts.embedVersion && opts.embedVersion.length > 0 ? opts.embedVersion : "latest";
+  return ["uvx", `hindsight-embed@${version}`];
+}

hindsight-all-npm/src/index.ts

@@ -0,0 +1,7 @@
+export { HindsightServer } from "./server.js";
+export { getEmbedCommand } from "./command.js";
+export { silentLogger, consoleLogger } from "./logger.js";
+
+export type { Logger } from "./logger.js";
+export type { EmbedCommandOptions } from "./command.js";
+export type { HindsightServerOptions } from "./types.js";

hindsight-all-npm/src/logger.ts

@@ -0,0 +1,29 @@
+/**
+ * Pluggable logger interface.
+ *
+ * This package does not own any logging infrastructure — consumers inject
+ * whatever they want (console, pino, openclaw's logger, a no-op). The default
+ * is silent so embedding this package never adds noise to an unrelated app.
+ */
+export interface Logger {
+  debug(msg: string): void;
+  info(msg: string): void;
+  warn(msg: string): void;
+  error(msg: string): void;
+}
+
+/** Logger that drops every call. Used when no logger is passed. */
+export const silentLogger: Logger = {
+  debug: () => {},
+  info: () => {},
+  warn: () => {},
+  error: () => {},
+};
+
+/** Logger that writes to the standard console. Handy for CLIs and tests. */
+export const consoleLogger: Logger = {
+  debug: (msg) => console.debug(msg),
+  info: (msg) => console.log(msg),
+  warn: (msg) => console.warn(msg),
+  error: (msg) => console.error(msg),
+};

hindsight-all-npm/src/server.test.ts

@@ -0,0 +1,35 @@
+import { describe, it, expect } from "vitest";
+import { HindsightServer } from "./server.js";
+
+describe("HindsightServer construction", () => {
+  it("defaults base URL to http://127.0.0.1:8888", () => {
+    const server = new HindsightServer();
+    expect(server.getBaseUrl()).toBe("http://127.0.0.1:8888");
+    expect(server.getProfile()).toBe("default");
+  });
+
+  it("honours custom profile, port, and host", () => {
+    const server = new HindsightServer({ profile: "app", port: 9077, host: "0.0.0.0" });
+    expect(server.getProfile()).toBe("app");
+    expect(server.getBaseUrl()).toBe("http://0.0.0.0:9077");
+  });
+
+  it("accepts open env pass-through without complaining about unknown keys", () => {
+    const server = new HindsightServer({
+      env: {
+        HINDSIGHT_API_LLM_PROVIDER: "openai",
+        HINDSIGHT_API_LLM_MODEL: "gpt-4o-mini",
+        // A field that does not exist today — should still be accepted
+        HINDSIGHT_FUTURE_FLAG: "enabled",
+      },
+    });
+    expect(server).toBeInstanceOf(HindsightServer);
+  });
+
+  it("exposes checkHealth that returns false when no daemon is running", async () => {
+    // Random high port that nothing is listening on.
+    const server = new HindsightServer({ port: 1, readyTimeoutMs: 100 });
+    const healthy = await server.checkHealth();
+    expect(healthy).toBe(false);
+  });
+});

hindsight-all-npm/src/server.ts

@@ -0,0 +1,322 @@
+import { spawn } from "child_process";
+import { getEmbedCommand } from "./command.js";
+import { silentLogger } from "./logger.js";
+import type { Logger } from "./logger.js";
+import type { HindsightServerOptions } from "./types.js";
+
+const DEFAULT_PORT = 8888;
+const DEFAULT_HOST = "127.0.0.1";
+const DEFAULT_PROFILE = "default";
+const DEFAULT_READY_TIMEOUT_MS = 30_000;
+const DEFAULT_READY_POLL_INTERVAL_MS = 1_000;
+
+/**
+ * Manages the lifecycle of a local Hindsight daemon from a Node.js process.
+ *
+ * On {@link start}, this class:
+ *   1. Resolves the `hindsight-embed` command (via `uvx` or a local `uv run`).
+ *   2. Runs `profile create <name> --merge --port <port> [--env K=V ...]`
+ *      with every entry in {@link HindsightServerOptions.env} forwarded as
+ *      an `--env` flag.
+ *   3. Runs `daemon --profile <name> start` and waits for the start command
+ *      to exit.
+ *   4. Polls `http://host:port/health` until it returns `200` or the
+ *      `readyTimeoutMs` budget is exhausted.
+ *
+ * On {@link stop}, it runs `daemon --profile <name> stop` and returns once
+ * the command exits (or after a short grace period).
+ *
+ * This is the Node.js equivalent of the Python `hindsight-all` package's
+ * `HindsightServer`: a thin programmatic lifecycle wrapper around the
+ * Hindsight daemon. It does NOT ship an HTTP client — once `start()`
+ * resolves, use `@vectorize-io/hindsight-client` against `getBaseUrl()` for
+ * retain / recall / reflect.
+ *
+ * The class is deliberately transparent about the daemon: new CLI flags or
+ * environment variables never require a code change here — callers can pass
+ * them via `env`, `extraProfileCreateArgs`, or `extraDaemonStartArgs`.
+ */
+export class HindsightServer {
+  private readonly profile: string;
+  private readonly port: number;
+  private readonly host: string;
+  private readonly baseUrl: string;
+  private readonly embedVersion: string | undefined;
+  private readonly embedPackagePath: string | undefined;
+  private readonly userEnv: Record<string, string | undefined>;
+  private readonly extraProfileCreateArgs: string[];
+  private readonly extraDaemonStartArgs: string[];
+  private readonly platformCpuWorkaround: boolean;
+  private readonly readyTimeoutMs: number;
+  private readonly readyPollIntervalMs: number;
+  private readonly logger: Logger;
+
+  constructor(opts: HindsightServerOptions = {}) {
+    this.profile = opts.profile ?? DEFAULT_PROFILE;
+    this.port = opts.port ?? DEFAULT_PORT;
+    this.host = opts.host ?? DEFAULT_HOST;
+    this.baseUrl = `http://${this.host}:${this.port}`;
+    this.embedVersion = opts.embedVersion;
+    this.embedPackagePath = opts.embedPackagePath;
+    this.userEnv = opts.env ?? {};
+    this.extraProfileCreateArgs = opts.extraProfileCreateArgs ?? [];
+    this.extraDaemonStartArgs = opts.extraDaemonStartArgs ?? [];
+    this.platformCpuWorkaround = opts.platformCpuWorkaround ?? process.platform === "darwin";
+    this.readyTimeoutMs = opts.readyTimeoutMs ?? DEFAULT_READY_TIMEOUT_MS;
+    this.readyPollIntervalMs = opts.readyPollIntervalMs ?? DEFAULT_READY_POLL_INTERVAL_MS;
+    this.logger = opts.logger ?? silentLogger;
+  }
+
+  /** The base URL the daemon listens on (`http://host:port`). */
+  getBaseUrl(): string {
+    return this.baseUrl;
+  }
+
+  /** The profile name this server operates on. */
+  getProfile(): string {
+    return this.profile;
+  }
+
+  /**
+   * Ensure the daemon is configured and running. Idempotent — the underlying
+   * `profile create --merge` and `daemon start` commands tolerate re-runs.
+   */
+  async start(): Promise<void> {
+    this.logger.info(`[hindsight] starting daemon for profile "${this.profile}"`);
+
+    const env = this.buildEnv();
+    await this.configureProfile(env);
+    await this.startDaemon(env);
+    await this.waitForReady();
+
+    this.logger.info(`[hindsight] daemon ready at ${this.baseUrl}`);
+  }
+
+  /** Stop the daemon. Never throws — logs and resolves even on failure. */
+  async stop(): Promise<void> {
+    this.logger.info(`[hindsight] stopping daemon for profile "${this.profile}"`);
+
+    const [cmd, ...baseArgs] = getEmbedCommand({
+      embedVersion: this.embedVersion,
+      embedPackagePath: this.embedPackagePath,
+    });
+    const args = [...baseArgs, "daemon", "--profile", this.profile, "stop"];
+
+    const child = spawn(cmd, args, { stdio: "pipe" });
+    this.pipeOutput(child, "daemon.stop");
+
+    await new Promise<void>((resolve) => {
+      const timeout = setTimeout(() => {
+        this.logger.warn(`[hindsight] daemon stop timed out after 5s`);
+        resolve();
+      }, 5_000);
+      child.on("exit", () => {
+        clearTimeout(timeout);
+        this.logger.info(`[hindsight] daemon stopped`);
+        resolve();
+      });
+      child.on("error", (err) => {
+        clearTimeout(timeout);
+        this.logger.warn(`[hindsight] error stopping daemon: ${err.message}`);
+        resolve();
+      });
+    });
+  }
+
+  /** Probe `/health` once with a short timeout. */
+  async checkHealth(): Promise<boolean> {
+    try {
+      const res = await fetch(`${this.baseUrl}/health`, {
+        signal: AbortSignal.timeout(2_000),
+      });
+      return res.ok;
+    } catch {
+      return false;
+    }
+  }
+
+  // -------------------------------------------------------------------------
+  // Internal
+  // -------------------------------------------------------------------------
+
+  /**
+   * Merge the process env, the caller-supplied `env`, and (on macOS) the
+   * embeddings CPU workaround. Caller-supplied values always win over the
+   * workaround; undefined values are dropped.
+   */
+  private buildEnv(): NodeJS.ProcessEnv {
+    const merged: NodeJS.ProcessEnv = { ...process.env };
+
+    if (this.platformCpuWorkaround && process.platform === "darwin") {
+      merged["HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU"] = "1";
+      merged["HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU"] = "1";
+    }
+
+    for (const [key, value] of Object.entries(this.userEnv)) {
+      if (value !== undefined) {
+        merged[key] = value;
+      }
+    }
+
+    return merged;
+  }
+
+  /**
+   * Run `profile create <name> --merge --port <port> [--env K=V ...]`.
+   * Every entry in the merged env that was passed via {@link userEnv} (or
+   * auto-applied by the CPU workaround) is forwarded as `--env`.
+   */
+  private async configureProfile(env: NodeJS.ProcessEnv): Promise<void> {
+    this.logger.info(`[hindsight] configuring profile "${this.profile}"`);
+
+    const [cmd, ...baseArgs] = getEmbedCommand({
+      embedVersion: this.embedVersion,
+      embedPackagePath: this.embedPackagePath,
+    });
+    const createArgs = [
+      ...baseArgs,
+      "profile",
+      "create",
+      this.profile,
+      "--merge",
+      "--port",
+      String(this.port),
+    ];
+
+    // Forward every env var that the caller intended for the daemon as --env.
+    // We only forward keys the caller explicitly set (userEnv) plus the CPU
+    // workaround values — not the entire process.env, to avoid leaking random
+    // host state into profile config.
+    const envForProfile = this.collectProfileEnv(env);
+    for (const [key, value] of Object.entries(envForProfile)) {
+      createArgs.push("--env", `${key}=${value}`);
+    }
+
+    createArgs.push(...this.extraProfileCreateArgs);
+
+    await this.runCommand(cmd, createArgs, env, "profile.create");
+  }
+
+  /** Collect only the env vars that should be written into the profile file. */
+  private collectProfileEnv(env: NodeJS.ProcessEnv): Record<string, string> {
+    const out: Record<string, string> = {};
+
+    // 1. User-supplied env — always forwarded.
+    for (const [key, value] of Object.entries(this.userEnv)) {
+      if (value !== undefined) {
+        out[key] = value;
+      }
+    }
+
+    // 2. CPU workaround — only if auto-applied and not already overridden.
+    if (this.platformCpuWorkaround && process.platform === "darwin") {
+      const cpuKeys = [
+        "HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU",
+        "HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU",
+      ];
+      for (const key of cpuKeys) {
+        if (!(key in out) && env[key] !== undefined) {
+          out[key] = env[key] as string;
+        }
+      }
+    }
+
+    return out;
+  }
+
+  private async startDaemon(env: NodeJS.ProcessEnv): Promise<void> {
+    const [cmd, ...baseArgs] = getEmbedCommand({
+      embedVersion: this.embedVersion,
+      embedPackagePath: this.embedPackagePath,
+    });
+    const args = [
+      ...baseArgs,
+      "daemon",
+      "--profile",
+      this.profile,
+      "start",
+      ...this.extraDaemonStartArgs,
+    ];
+
+    await this.runCommand(cmd, args, env, "daemon.start");
+  }
+
+  /**
+   * Spawn `cmd` with `args`, pipe its output through the logger, and resolve
+   * once it exits with code 0. Rejects on non-zero exit or spawn error.
+   */
+  private async runCommand(
+    cmd: string,
+    args: string[],
+    env: NodeJS.ProcessEnv,
+    label: string
+  ): Promise<void> {
+    const child = spawn(cmd, args, { stdio: "pipe", env });
+    let output = "";
+    child.stdout?.on("data", (data: Buffer) => {
+      const text = data.toString();
+      output += text;
+      for (const line of text.trimEnd().split("\n")) {
+        if (line) this.logger.info(`[hindsight:${label}] ${line}`);
+      }
+    });
+    child.stderr?.on("data", (data: Buffer) => {
+      const text = data.toString();
+      output += text;
+      for (const line of text.trimEnd().split("\n")) {
+        if (line) this.logger.warn(`[hindsight:${label}] ${line}`);
+      }
+    });
+
+    await new Promise<void>((resolve, reject) => {
+      child.on("exit", (code) => {
+        if (code === 0) {
+          resolve();
+        } else {
+          reject(new Error(`${label} failed with code ${code}: ${output.trim()}`));
+        }
+      });
+      child.on("error", (err) => {
+        reject(new Error(`${label} failed to spawn: ${err.message}`, { cause: err }));
+      });
+    });
+  }
+
+  /** Stream a spawned child's stdout/stderr through the logger without blocking. */
+  private pipeOutput(child: ReturnType<typeof spawn>, label: string): void {
+    child.stdout?.on("data", (data: Buffer) => {
+      for (const line of data.toString().trimEnd().split("\n")) {
+        if (line) this.logger.info(`[hindsight:${label}] ${line}`);
+      }
+    });
+    child.stderr?.on("data", (data: Buffer) => {
+      for (const line of data.toString().trimEnd().split("\n")) {
+        if (line) this.logger.warn(`[hindsight:${label}] ${line}`);
+      }
+    });
+  }
+
+  /** Poll `/health` until it succeeds or `readyTimeoutMs` elapses. */
+  private async waitForReady(): Promise<void> {
+    const deadline = Date.now() + this.readyTimeoutMs;
+    let attempt = 0;
+    while (Date.now() < deadline) {
+      attempt++;
+      try {
+        const res = await fetch(`${this.baseUrl}/health`, {
+          signal: AbortSignal.timeout(this.readyPollIntervalMs),
+        });
+        if (res.ok) {
+          this.logger.debug(`[hindsight] health check passed (attempt ${attempt})`);
+          return;
+        }
+      } catch {
+        // expected while the daemon is still booting
+      }
+      await new Promise((resolve) => setTimeout(resolve, this.readyPollIntervalMs));
+    }
+    throw new Error(
+      `Hindsight daemon did not become ready within ${this.readyTimeoutMs}ms at ${this.baseUrl}`
+    );
+  }
+}

hindsight-all-npm/src/types.ts

@@ -0,0 +1,54 @@
+import type { Logger } from "./logger.js";
+
+/**
+ * Options for {@link HindsightServer}.
+ *
+ * The server is intentionally thin and pass-through: anything configurable
+ * on the daemon side (env vars or CLI flags) can be set here without needing
+ * a new dedicated option. Use {@link env} for `HINDSIGHT_*` / `OPENAI_API_KEY` /
+ * custom provider settings, and the two `extra*` arrays to append raw CLI
+ * args to `profile create` or `daemon start`.
+ *
+ * For talking to the daemon after `start()`, use `@vectorize-io/hindsight-client`
+ * against `server.getBaseUrl()`. This package does not ship its own HTTP
+ * client.
+ */
+export interface HindsightServerOptions {
+  /** Profile name used for `--profile <name>` on every sub-command. Default: `"default"`. */
+  profile?: string;
+  /** TCP port the daemon listens on. Default: `8888`. */
+  port?: number;
+  /** Hostname the daemon binds to (for health checks). Default: `127.0.0.1`. */
+  host?: string;
+  /** Version of the underlying `hindsight-embed` PyPI package to run via `uvx`. Default: `"latest"`. */
+  embedVersion?: string;
+  /** Local path to a `hindsight-embed` checkout — takes precedence over `embedVersion`. */
+  embedPackagePath?: string;
+  /**
+   * Environment variables passed to the daemon process AND written into the
+   * profile via repeated `--env KEY=VALUE` flags. This is the preferred way
+   * to surface any `HINDSIGHT_API_*` / `HINDSIGHT_EMBED_*` setting — adding a
+   * new daemon env var never requires a wrapper update.
+   *
+   * Values of `undefined` are dropped (so you can spread conditionally).
+   */
+  env?: Record<string, string | undefined>;
+  /** Extra args appended verbatim to `hindsight-embed profile create <name> --merge ...`. */
+  extraProfileCreateArgs?: string[];
+  /** Extra args appended verbatim to `hindsight-embed daemon --profile <name> start ...`. */
+  extraDaemonStartArgs?: string[];
+  /**
+   * On macOS, automatically set
+   * `HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU=1` and
+   * `HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU=1` to avoid Metal/MPS crashes in
+   * daemon mode. Default: `true` on `darwin`, ignored elsewhere. Any value set
+   * explicitly in {@link env} wins over the auto-applied value.
+   */
+  platformCpuWorkaround?: boolean;
+  /** Max time (ms) to wait for `/health` to return 200. Default: `30_000`. */
+  readyTimeoutMs?: number;
+  /** Polling interval (ms) while waiting for `/health`. Default: `1_000`. */
+  readyPollIntervalMs?: number;
+  /** Optional pluggable logger. Default: silent. */
+  logger?: Logger;
+}

hindsight-all-npm/tsconfig.json

@@ -0,0 +1,18 @@
+{
+  "compilerOptions": {
+    "target": "ES2022",
+    "module": "ES2022",
+    "lib": ["ES2022"],
+    "moduleResolution": "node",
+    "declaration": true,
+    "outDir": "./dist",
+    "rootDir": "./src",
+    "strict": true,
+    "esModuleInterop": true,
+    "skipLibCheck": true,
+    "forceConsistentCasingInFileNames": true,
+    "resolveJsonModule": true
+  },
+  "include": ["src/**/*"],
+  "exclude": ["node_modules", "dist", "src/**/*.test.ts"]
+}

hindsight-all-npm/tsup.config.ts

@@ -0,0 +1,11 @@
+import { defineConfig } from "tsup";
+
+export default defineConfig({
+  entry: ["src/index.ts"],
+  format: ["esm"],
+  dts: true,
+  outDir: "dist",
+  clean: true,
+  sourcemap: true,
+  bundle: true,
+});

hindsight-all-npm/vitest.config.ts

@@ -0,0 +1,8 @@
+import { defineConfig } from "vitest/config";
+
+export default defineConfig({
+  test: {
+    include: ["src/**/*.test.ts"],
+    environment: "node",
+  },
+});

hindsight-all-slim/README.md

@@ -1,6 +1,6 @@
 # hindsight-all
 
-All-in-one package for Hindsight - Semantic memory system with personality-driven thinking for AI agents.
+All-in-one package for Hindsight - Agent Memory That Works Like Human Memory
 
 ## Quick Start
 

hindsight-all-slim/pyproject.toml

@@ -0,0 +1,34 @@
+[build-system]
+requires = ["setuptools>=77"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "hindsight-all-slim"
+version = "0.9.0"
+description = "Hindsight: Agent Memory That Works Like Human Memory - Slim All-in-One Bundle"
+license = "MIT"
+readme = "README.md"
+requires-python = ">=3.11"
+dependencies = [
+    "hindsight-api-slim==0.9.0",
+    "hindsight-client>=0.0.7",
+    "hindsight-embed==0.9.0",
+]
+
+[tool.uv.sources]
+hindsight-api-slim = { workspace = true }
+hindsight-client = { workspace = true }
+hindsight-embed = { workspace = true }
+
+[project.optional-dependencies]
+test = [
+    "pytest>=7.0.0",
+    "pytest-asyncio>=0.21.0",
+]
+
+[tool.setuptools]
+packages = []
+
+[tool.pytest.ini_options]
+asyncio_mode = "auto"
+asyncio_default_fixture_loop_scope = "function"

hindsight-all/README.md

@@ -0,0 +1,48 @@
+# hindsight-all
+
+All-in-one package for Hindsight - Agent Memory That Works Like Human Memory
+
+## Quick Start
+
+```python
+from hindsight import start_server, HindsightClient
+
+# Start server with embedded PostgreSQL
+server = start_server(
+    llm_provider="groq",
+    llm_api_key="your-api-key",
+    llm_model="openai/gpt-oss-120b"
+)
+
+# Create client
+client = HindsightClient(base_url=server.url)
+
+# Store memories
+client.put(agent_id="assistant", content="User prefers Python for data analysis")
+
+# Search memories
+results = client.search(agent_id="assistant", query="programming preferences")
+
+# Generate contextual response
+response = client.think(agent_id="assistant", query="What languages should I recommend?")
+
+# Stop server when done
+server.stop()
+```
+
+## Using Context Manager
+
+```python
+from hindsight import HindsightServer, HindsightClient
+
+with HindsightServer(llm_provider="groq", llm_api_key="...") as server:
+    client = HindsightClient(base_url=server.url)
+    # ... use client ...
+# Server automatically stops
+```
+
+## Installation
+
+```bash
+pip install hindsight-all
+```

hindsight-all/hindsight/__init__.py

@@ -3,7 +3,23 @@ Hindsight - All-in-one semantic memory system for AI agents.
 
 This package provides a simple way to run Hindsight locally with embedded PostgreSQL.
 
-Example:
+Easiest way - Embedded client (recommended):
+    ```python
+    from hindsight import HindsightEmbedded
+
+    # Server starts automatically on first use
+    client = HindsightEmbedded(
+        profile="myapp",
+        llm_provider="groq",
+        llm_api_key="your-api-key",
+    )
+
+    # Use immediately - no manual server management needed
+    client.retain(bank_id="alice", content="Alice loves AI")
+    results = client.recall(bank_id="alice", query="What does Alice like?")
+    ```
+
+Manual server management:
     ```python
     from hindsight import start_server, HindsightClient
 
@@ -18,13 +34,13 @@ Example:
     client = HindsightClient(base_url=server.url)
 
     # Store memories
-    client.put(agent_id="assistant", content="User prefers Python for data analysis")
+    client.retain(bank_id="assistant", content="User prefers Python for data analysis")
 
     # Search memories
-    results = client.search(agent_id="assistant", query="programming preferences")
+    results = client.recall(bank_id="assistant", query="programming preferences")
 
     # Generate contextual response
-    response = client.think(agent_id="assistant", query="What languages should I recommend?")
+    response = client.reflect(bank_id="assistant", query="What are my interests?")
 
     # Stop server when done
     server.stop()
@@ -41,13 +57,13 @@ Using context manager:
     ```
 """
 
+from .client_wrapper import HindsightClient
+from .embedded import HindsightEmbedded
 from .server import Server as HindsightServer, start_server
 
-# Re-export Client from hindsight-client
-from hindsight_client import Hindsight as HindsightClient
-
 __all__ = [
     "HindsightServer",
     "start_server",
     "HindsightClient",
+    "HindsightEmbedded",
 ]

hindsight-all/hindsight/api_namespaces.py

@@ -0,0 +1,193 @@
+"""
+API namespace classes for organizing client methods.
+
+These classes provide organized access to different parts of the Hindsight API
+while ensuring the daemon is running before each call.
+"""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any
+
+if TYPE_CHECKING:
+    from .embedded import HindsightEmbedded
+
+
+class BanksAPI:
+    """Namespace for bank-related operations."""
+
+    def __init__(self, embedded: "HindsightEmbedded"):
+        self._embedded = embedded
+
+    def create(
+        self,
+        bank_id: str,
+        name: str | None = None,
+        mission: str | None = None,
+        disposition: dict[str, Any] | None = None,
+    ):
+        """Create a new bank."""
+        self._embedded._ensure_started()
+        return self._embedded._client.create_bank(
+            bank_id=bank_id,
+            name=name,
+            mission=mission,
+            disposition=disposition,
+        )
+
+    def delete(self, bank_id: str):
+        """Delete a bank."""
+        self._embedded._ensure_started()
+        return self._embedded._client.delete_bank(bank_id=bank_id)
+
+    def set_mission(self, bank_id: str, mission: str):
+        """Set or update the mission for a bank."""
+        self._embedded._ensure_started()
+        return self._embedded._client.set_mission(bank_id=bank_id, mission=mission)
+
+    def set_disposition(self, bank_id: str, disposition: dict[str, Any]):
+        """Set or update the disposition for a bank."""
+        self._embedded._ensure_started()
+        return self._embedded._client.set_disposition(bank_id=bank_id, disposition=disposition)
+
+
+class MentalModelsAPI:
+    """Namespace for mental model operations."""
+
+    def __init__(self, embedded: "HindsightEmbedded"):
+        self._embedded = embedded
+
+    def create(
+        self,
+        bank_id: str,
+        name: str,
+        content: str,
+        tags: list[str] | None = None,
+    ):
+        """Create a new mental model."""
+        self._embedded._ensure_started()
+        return self._embedded._client.create_mental_model(
+            bank_id=bank_id,
+            name=name,
+            content=content,
+            tags=tags,
+        )
+
+    def list(self, bank_id: str, tags: list[str] | None = None):
+        """List all mental models for a bank."""
+        self._embedded._ensure_started()
+        return self._embedded._client.list_mental_models(bank_id=bank_id, tags=tags)
+
+    def get(self, bank_id: str, mental_model_id: str):
+        """Get a specific mental model."""
+        self._embedded._ensure_started()
+        return self._embedded._client.get_mental_model(bank_id=bank_id, mental_model_id=mental_model_id)
+
+    def refresh(self, bank_id: str, mental_model_id: str):
+        """Refresh a mental model."""
+        self._embedded._ensure_started()
+        return self._embedded._client.refresh_mental_model(bank_id=bank_id, mental_model_id=mental_model_id)
+
+    def update(
+        self,
+        bank_id: str,
+        mental_model_id: str,
+        name: str | None = None,
+        content: str | None = None,
+        tags: list[str] | None = None,
+    ):
+        """Update a mental model."""
+        self._embedded._ensure_started()
+        return self._embedded._client.update_mental_model(
+            bank_id=bank_id,
+            mental_model_id=mental_model_id,
+            name=name,
+            content=content,
+            tags=tags,
+        )
+
+    def delete(self, bank_id: str, mental_model_id: str):
+        """Delete a mental model."""
+        self._embedded._ensure_started()
+        return self._embedded._client.delete_mental_model(bank_id=bank_id, mental_model_id=mental_model_id)
+
+
+class DirectivesAPI:
+    """Namespace for directive operations."""
+
+    def __init__(self, embedded: "HindsightEmbedded"):
+        self._embedded = embedded
+
+    def create(
+        self,
+        bank_id: str,
+        name: str,
+        content: str,
+        tags: list[str] | None = None,
+    ):
+        """Create a new directive."""
+        self._embedded._ensure_started()
+        return self._embedded._client.create_directive(
+            bank_id=bank_id,
+            name=name,
+            content=content,
+            tags=tags,
+        )
+
+    def list(self, bank_id: str, tags: list[str] | None = None):
+        """List all directives for a bank."""
+        self._embedded._ensure_started()
+        return self._embedded._client.list_directives(bank_id=bank_id, tags=tags)
+
+    def get(self, bank_id: str, directive_id: str):
+        """Get a specific directive."""
+        self._embedded._ensure_started()
+        return self._embedded._client.get_directive(bank_id=bank_id, directive_id=directive_id)
+
+    def update(
+        self,
+        bank_id: str,
+        directive_id: str,
+        name: str | None = None,
+        content: str | None = None,
+        tags: list[str] | None = None,
+    ):
+        """Update a directive."""
+        self._embedded._ensure_started()
+        return self._embedded._client.update_directive(
+            bank_id=bank_id,
+            directive_id=directive_id,
+            name=name,
+            content=content,
+            tags=tags,
+        )
+
+    def delete(self, bank_id: str, directive_id: str):
+        """Delete a directive."""
+        self._embedded._ensure_started()
+        return self._embedded._client.delete_directive(bank_id=bank_id, directive_id=directive_id)
+
+
+class MemoriesAPI:
+    """Namespace for memory operations."""
+
+    def __init__(self, embedded: "HindsightEmbedded"):
+        self._embedded = embedded
+
+    def list(
+        self,
+        bank_id: str,
+        type: str | None = None,
+        search_query: str | None = None,
+        limit: int = 100,
+        offset: int = 0,
+    ):
+        """List memories in a bank."""
+        self._embedded._ensure_started()
+        return self._embedded._client.list_memories(
+            bank_id=bank_id,
+            type=type,
+            search_query=search_query,
+            limit=limit,
+            offset=offset,
+        )

hindsight-all/hindsight/client_wrapper.py

@@ -0,0 +1,423 @@
+"""
+Wrapper for Hindsight client that adds API namespaces.
+
+Provides organized access to different parts of the Hindsight API through
+namespaces like .banks, .mental_models, etc.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from hindsight_client import Hindsight
+
+
+class BanksAPI:
+    """Namespace for bank-related operations.
+    
+    Provides methods to create, delete, and manage memory banks.
+    """
+
+    def __init__(self, client: Hindsight):
+        self._client = client
+
+    def create(
+        self,
+        bank_id: str,
+        name: str | None = None,
+        mission: str | None = None,
+        disposition: dict[str, Any] | None = None,
+    ) -> Any:
+        """Create a new bank.
+        
+        Args:
+            bank_id: Unique identifier for the bank.
+            name: Optional display name for the bank.
+            mission: Optional mission statement for the bank.
+            disposition: Optional disposition configuration dict.
+            
+        Returns:
+            Bank creation response from the API.
+        """
+        return self._client.create_bank(
+            bank_id=bank_id,
+            name=name,
+            mission=mission,
+            disposition=disposition,
+        )
+
+    def delete(self, bank_id: str) -> Any:
+        """Delete a bank.
+        
+        Args:
+            bank_id: The ID of the bank to delete.
+            
+        Returns:
+            Deletion response from the API.
+        """
+        return self._client.delete_bank(bank_id=bank_id)
+
+    def set_mission(self, bank_id: str, mission: str) -> Any:
+        """Set or update the mission for a bank.
+        
+        Args:
+            bank_id: The ID of the bank.
+            mission: The mission statement to set.
+            
+        Returns:
+            API response confirming the update.
+        """
+        return self._client.set_mission(bank_id=bank_id, mission=mission)
+
+    def set_disposition(self, bank_id: str, disposition: dict[str, Any]) -> Any:
+        """Set or update the disposition for a bank.
+        
+        Args:
+            bank_id: The ID of the bank.
+            disposition: The disposition configuration dict.
+            
+        Returns:
+            API response confirming the update.
+        """
+        return self._client.set_disposition(bank_id=bank_id, disposition=disposition)
+
+    def list(self) -> Any:
+        """List all banks.
+        
+        Returns:
+            List of banks from the API.
+        """
+        from hindsight_client.hindsight_client import _run_async
+
+        return _run_async(self._client._banks_api.list_banks())
+
+
+class MentalModelsAPI:
+    """Namespace for mental model operations.
+    
+    Mental models are reusable knowledge structures that guide agent behavior.
+    """
+
+    def __init__(self, client: Hindsight):
+        self._client = client
+
+    def create(
+        self,
+        bank_id: str,
+        name: str,
+        content: str,
+        tags: list[str] | None = None,
+    ) -> Any:
+        """Create a new mental model.
+        
+        Args:
+            bank_id: The ID of the bank to add the model to.
+            name: Name for the mental model.
+            content: The content/instructions for the mental model.
+            tags: Optional list of tags for categorization.
+            
+        Returns:
+            Creation response from the API.
+        """
+        return self._client.create_mental_model(
+            bank_id=bank_id,
+            name=name,
+            content=content,
+            tags=tags,
+        )
+
+    def list(self, bank_id: str, tags: list[str] | None = None) -> Any:
+        """List all mental models for a bank.
+        
+        Args:
+            bank_id: The ID of the bank.
+            tags: Optional filter by tags.
+            
+        Returns:
+            List of mental models.
+        """
+        return self._client.list_mental_models(bank_id=bank_id, tags=tags)
+
+    def get(self, bank_id: str, mental_model_id: str) -> Any:
+        """Get a specific mental model.
+        
+        Args:
+            bank_id: The ID of the bank.
+            mental_model_id: The ID of the mental model.
+            
+        Returns:
+            The mental model details.
+        """
+        return self._client.get_mental_model(bank_id=bank_id, mental_model_id=mental_model_id)
+
+    def refresh(self, bank_id: str, mental_model_id: str) -> Any:
+        """Refresh a mental model.
+        
+        Args:
+            bank_id: The ID of the bank.
+            mental_model_id: The ID of the mental model to refresh.
+            
+        Returns:
+            Refresh response from the API.
+        """
+        return self._client.refresh_mental_model(bank_id=bank_id, mental_model_id=mental_model_id)
+
+    def update(
+        self,
+        bank_id: str,
+        mental_model_id: str,
+        name: str | None = None,
+        content: str | None = None,
+        tags: list[str] | None = None,
+    ) -> Any:
+        """Update a mental model.
+        
+        Args:
+            bank_id: The ID of the bank.
+            mental_model_id: The ID of the mental model to update.
+            name: Optional new name.
+            content: Optional new content.
+            tags: Optional new tags list.
+            
+        Returns:
+            Update response from the API.
+        """
+        return self._client.update_mental_model(
+            bank_id=bank_id,
+            mental_model_id=mental_model_id,
+            name=name,
+            content=content,
+            tags=tags,
+        )
+
+    def delete(self, bank_id: str, mental_model_id: str) -> Any:
+        """Delete a mental model.
+        
+        Args:
+            bank_id: The ID of the bank.
+            mental_model_id: The ID of the mental model to delete.
+            
+        Returns:
+            Deletion response from the API.
+        """
+        return self._client.delete_mental_model(bank_id=bank_id, mental_model_id=mental_model_id)
+
+
+class DirectivesAPI:
+    """Namespace for directive operations.
+    
+    Directives are explicit instructions that guide agent behavior.
+    """
+
+    def __init__(self, client: Hindsight):
+        self._client = client
+
+    def create(
+        self,
+        bank_id: str,
+        name: str,
+        content: str,
+        tags: list[str] | None = None,
+    ) -> Any:
+        """Create a new directive.
+        
+        Args:
+            bank_id: The ID of the bank to add the directive to.
+            name: Name for the directive.
+            content: The directive content/instructions.
+            tags: Optional list of tags for categorization.
+            
+        Returns:
+            Creation response from the API.
+        """
+        return self._client.create_directive(
+            bank_id=bank_id,
+            name=name,
+            content=content,
+            tags=tags,
+        )
+
+    def list(self, bank_id: str, tags: list[str] | None = None) -> Any:
+        """List all directives for a bank.
+        
+        Args:
+            bank_id: The ID of the bank.
+            tags: Optional filter by tags.
+            
+        Returns:
+            List of directives.
+        """
+        return self._client.list_directives(bank_id=bank_id, tags=tags)
+
+    def get(self, bank_id: str, directive_id: str) -> Any:
+        """Get a specific directive.
+        
+        Args:
+            bank_id: The ID of the bank.
+            directive_id: The ID of the directive.
+            
+        Returns:
+            The directive details.
+        """
+        return self._client.get_directive(bank_id=bank_id, directive_id=directive_id)
+
+    def update(
+        self,
+        bank_id: str,
+        directive_id: str,
+        name: str | None = None,
+        content: str | None = None,
+        tags: list[str] | None = None,
+    ) -> Any:
+        """Update a directive.
+        
+        Args:
+            bank_id: The ID of the bank.
+            directive_id: The ID of the directive to update.
+            name: Optional new name.
+            content: Optional new content.
+            tags: Optional new tags list.
+            
+        Returns:
+            Update response from the API.
+        """
+        return self._client.update_directive(
+            bank_id=bank_id,
+            directive_id=directive_id,
+            name=name,
+            content=content,
+            tags=tags,
+        )
+
+    def delete(self, bank_id: str, directive_id: str) -> Any:
+        """Delete a directive.
+        
+        Args:
+            bank_id: The ID of the bank.
+            directive_id: The ID of the directive to delete.
+            
+        Returns:
+            Deletion response from the API.
+        """
+        return self._client.delete_directive(bank_id=bank_id, directive_id=directive_id)
+
+
+class MemoriesAPI:
+    """Namespace for memory operations.
+    
+    Provides methods to query and retrieve stored memories.
+    """
+
+    def __init__(self, client: Hindsight):
+        self._client = client
+
+    def list(
+        self,
+        bank_id: str,
+        type: str | None = None,
+        search_query: str | None = None,
+        limit: int = 100,
+        offset: int = 0,
+    ) -> Any:
+        """List memories in a bank.
+        
+        Args:
+            bank_id: The ID of the bank to query.
+            type: Optional filter by memory type.
+            search_query: Optional search query for filtering.
+            limit: Maximum number of results to return (default: 100).
+            offset: Number of results to skip for pagination (default: 0).
+            
+        Returns:
+            List of memories matching the criteria.
+        """
+        return self._client.list_memories(
+            bank_id=bank_id,
+            type=type,
+            search_query=search_query,
+            limit=limit,
+            offset=offset,
+        )
+
+
+class HindsightClient(Hindsight):
+    """
+    Enhanced Hindsight client with organized API namespaces.
+
+    This wrapper extends the auto-generated Hindsight client with organized
+    access to different parts of the API through namespaces.
+
+    Example:
+        ```python
+        from hindsight import HindsightClient
+
+        client = HindsightClient(base_url="http://localhost:8888")
+
+        # Core operations (inherited from Hindsight)
+        client.retain(bank_id="test", content="Hello")
+        results = client.recall(bank_id="test", query="Hello")
+
+        # Organized API access through namespaces
+        client.banks.create(bank_id="test", name="Test Bank")
+        models = client.mental_models.list(bank_id="test")
+        directives = client.directives.list(bank_id="test")
+        memories = client.memories.list(bank_id="test")
+        ```
+        
+    Attributes:
+        banks: Namespace for bank management operations.
+        mental_models: Namespace for mental model operations.
+        directives: Namespace for directive operations.
+        memories: Namespace for memory listing operations.
+    """
+
+    def __init__(self, *args: Any, **kwargs: Any) -> None:
+        super().__init__(*args, **kwargs)
+        self._banks_namespace: BanksAPI | None = None
+        self._mental_models_namespace: MentalModelsAPI | None = None
+        self._directives_namespace: DirectivesAPI | None = None
+        self._memories_namespace: MemoriesAPI | None = None
+
+    @property
+    def banks(self) -> BanksAPI:
+        """Access bank management operations.
+        
+        Returns:
+            BanksAPI instance for bank operations.
+        """
+        if self._banks_namespace is None:
+            self._banks_namespace = BanksAPI(self)
+        return self._banks_namespace
+
+    @property
+    def mental_models(self) -> MentalModelsAPI:
+        """Access mental model operations.
+        
+        Returns:
+            MentalModelsAPI instance for mental model operations.
+        """
+        if self._mental_models_namespace is None:
+            self._mental_models_namespace = MentalModelsAPI(self)
+        return self._mental_models_namespace
+
+    @property
+    def directives(self) -> DirectivesAPI:
+        """Access directive operations.
+        
+        Returns:
+            DirectivesAPI instance for directive operations.
+        """
+        if self._directives_namespace is None:
+            self._directives_namespace = DirectivesAPI(self)
+        return self._directives_namespace
+
+    @property
+    def memories(self) -> MemoriesAPI:
+        """Access memory listing operations.
+        
+        Returns:
+            MemoriesAPI instance for memory operations.
+        """
+        if self._memories_namespace is None:
+            self._memories_namespace = MemoriesAPI(self)
+        return self._memories_namespace