fix tests

.claude-plugin/marketplace.json

@@ -1,16 +0,0 @@
-{
-  "$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
-  "name": "hindsight",
-  "version": "0.7.2",
-  "description": "Official Hindsight integrations for Claude Code",
-  "owner": {
-    "name": "vectorize-io"
-  },
-  "plugins": [
-    {
-      "name": "hindsight-memory",
-      "description": "Automatic long-term memory for Claude Code via Hindsight",
-      "source": "./hindsight-integrations/claude-code"
-    }
-  ]
-}

.claude/skills/code-review/SKILL.md

@@ -1,247 +0,0 @@
----
-name: code-review
-description: Review changed code against project standards. Checks for missing tests, dead code, type safety, lint issues, and coding conventions. Run after completing any implementation work.
-user_invocable: true
----
-
-# Code Review
-
-Review all changed code against the project's quality standards and coding conventions.
-
-## Code Standards
-
-Read and internalize these standards before writing code. The review steps below verify compliance.
-
-### Python Style
-- Python 3.11+, type hints required
-- Async throughout (asyncpg, async FastAPI)
-- Pydantic models for request/response
-- Ruff for linting (line-length 120)
-- No Python files at project root - maintain clean directory structure
-- **Never use multi-item tuple return values** — not even for internal/private functions. Always use a dataclass or Pydantic model. No exceptions, no "it's just two values" shortcuts. If a function returns more than one value, define a named type for it.
-
-### Type Safety with Pydantic Models
-**NEVER use raw `dict` types for structured data** — this applies to all code, including internal helpers and private functions. If the dict has known keys, it must be a dataclass or Pydantic model:
-- Use Pydantic `BaseModel` for all data structures passed between functions
-- Use `@dataclass` for lightweight internal data containers when Pydantic validation isn't needed
-- Add `@field_validator` for type coercion (e.g., ensuring datetimes are timezone-aware)
-- Avoid `dict.get()` patterns - use typed model attributes instead
-- Parse external data (JSON, API responses) into Pydantic models at the boundary
-- This catches type errors at parse time, not deep in business logic
-- The only acceptable `dict` usage is for truly dynamic/unknown keys (e.g., arbitrary metadata, JSON blobs with no fixed schema)
-
-```python
-# BAD - error-prone dict access
-def process(data: dict) -> str:
-    return data.get("name", "")  # No validation, silent failures
-
-# GOOD - typed and validated
-class UserData(BaseModel):
-    name: str
-    created_at: datetime
-
-    @field_validator("created_at", mode="before")
-    @classmethod
-    def ensure_tz_aware(cls, v):
-        if isinstance(v, str):
-            v = datetime.fromisoformat(v.replace("Z", "+00:00"))
-        if v.tzinfo is None:
-            return v.replace(tzinfo=timezone.utc)
-        return v
-
-def process(data: UserData) -> str:
-    return data.name  # Type-safe, validated at construction
-```
-
-### TypeScript Style
-- Next.js App Router for control plane
-- Tailwind CSS with shadcn/ui components
-
-### Code Comments
-- **Always comment non-trivial technical decisions** with the reasoning behind the choice. If someone would ask "why is it done this way?", there should be a comment.
-- **Keep comments up to date with history** — when changing an approach, update the comment to explain what was tried before and why it was changed. Comments serve as a tracker of previous implementations that likely had problems.
-- Don't comment obvious code — only where the "why" isn't self-evident from the code itself.
-
-```python
-# BAD - no context for future readers
-results = await asyncio.gather(*tasks, return_exceptions=True)
-
-# GOOD - explains the non-obvious choice
-# Use return_exceptions=True to avoid cancelling sibling tasks on failure.
-# Previously we used TaskGroup but it cancelled all tasks when one failed,
-# causing partial writes that left orphaned entity links (see #412).
-results = await asyncio.gather(*tasks, return_exceptions=True)
-```
-
-### API Layer & Data Access
-- **No direct database access in `api/http.py`** (or any API router). HTTP handlers must not build SQL, call `acquire_with_retry` / `conn.fetch` / `conn.fetchrow` / `conn.execute`, or reference `fq_table(...)`. All persistence and queries live in `MemoryEngine` (the engine layer). A handler parses/validates the request, calls an engine method, shapes the HTTP response, and maps domain results to status codes (e.g. a `None` return → 404).
-- **Authentication/tenancy is enforced inside each engine method, not assumed by the handler.** Every engine method that touches bank-scoped data must authenticate via `request_context` — typically `await self._authenticate_tenant(request_context)` (often indirectly through `get_bank_profile(...)`) — so the correct tenant schema is resolved before any query runs. Handlers must thread `request_context` through to the engine method; never query a tenant-scoped table assuming the schema is already set.
-- Engine methods return typed models (Pydantic/dataclass), not raw dicts (see Type Safety).
-
-### Branch Hygiene
-- **Always start new feature branches from `origin/main`** — rebase to ensure a clean base.
-- **Only include commits relevant to the PR/branch/feature** — no unrelated changes. If the branch contains commits that don't belong, they must be removed before merging.
-
-### General Principles
-- Don't add features, refactor code, or make "improvements" beyond what was asked
-- Don't add unnecessary error handling for impossible scenarios
-- Don't create helpers or abstractions for one-time operations
-- No backwards-compatibility hacks (unused vars, re-exports, "removed" comments)
-- Three similar lines of code is better than a premature abstraction
-
-## Review Steps
-
-### 1. Check branch hygiene
-
-- Run `git log --oneline main..HEAD` to list all commits on the branch.
-- Verify every commit is relevant to the feature/PR. Flag any unrelated commits.
-- Check the branch is based on a recent `origin/main` (no stale base).
-
-### 2. Identify changed files
-
-Run `git diff --name-only HEAD` (unstaged) and `git diff --cached --name-only` (staged) to get all changed files. If there are no local changes, diff against the base branch using `git diff main...HEAD --name-only` and `git diff main...HEAD` to review all commits on the current branch.
-
-### 3. Run linters
-
-```bash
-./scripts/hooks/lint.sh
-```
-
-Report any failures. Do NOT fix them yourself — just report.
-
-### 4. Check for dead code
-
-For each changed Python file, check for:
-- Unused imports (Ruff should catch these, but verify)
-- Functions/methods/classes that were added but are never called from anywhere
-- Variables assigned but never read
-- Commented-out code blocks that should be removed
-
-For each changed TypeScript file, check for:
-- Unused imports
-- Unused variables or functions
-- Commented-out code
-
-### 5. Check type safety (Python)
-
-For each changed Python file, check for violations:
-- **No raw `dict` for structured data** — must use Pydantic model or dataclass, even for internal/private functions (only exception: truly dynamic/unknown keys)
-- **No multi-item tuple returns** — must use dataclass or Pydantic model, even for internal/private functions (no exceptions)
-- **Missing type hints** on function parameters and return types
-- **Missing `@field_validator`** for datetime fields that should be timezone-aware
-
-### 6. Check for missing tests
-
-For each new or significantly changed function/endpoint/class:
-- Check if there is a corresponding test addition or update
-- New API endpoints MUST have integration tests
-- New utility functions MUST have unit tests
-- Bug fixes SHOULD have a regression test
-
-Flag any new logic that lacks test coverage.
-
-**LLM-behaviour changes need a real-LLM judge test, not MockLLM.** If the change alters how the model interprets a prompt — fact/observation extraction, `fact_type` (world/experience) classification, speaker attribution, instruction-following, prompt wording — there MUST be a test marked `pytest.mark.hs_llm_core` that runs the real pipeline and asserts via `tests.llm_judge.assert_meets_criteria` (not string/enum matching). Flag these as findings:
-- A prompt/classification change verified only by MockLLM or string assertions (MockLLM echoes input — such tests pass spuriously). **Should fix.**
-- A test that hard-asserts `fact_type == "world"/"experience"` (or other model-decided output) instead of judging it — non-deterministic, will flake across providers/runs. **Should fix** (move the classification check into the judge `criteria`; keep only genuinely deterministic structural asserts direct).
-- Deterministic mechanics (prompt assembly, suppression/branching logic) that are covered *only* by a slow LLM test — these should also have fast non-LLM unit tests. **Note.**
-
-See CLAUDE.md → Key Conventions → Testing for the full pattern.
-
-### 7. Check API consistency
-
-If any files in `hindsight-api-slim/hindsight_api/api/` were changed:
-- Were the OpenAPI specs regenerated? (`./scripts/generate-openapi.sh`)
-- Were the client SDKs regenerated? (`./scripts/generate-clients.sh`)
-- Were the control plane proxy routes updated? (`hindsight-control-plane/src/app/api/`)
-
-### 7b. Check API-layer data-access boundary
-
-For each changed handler in `hindsight-api-slim/hindsight_api/api/` (e.g. `http.py`, `mcp.py`):
-- **Flag any direct DB access in the handler** — `acquire_with_retry`, `conn.fetch` / `fetchrow` / `execute`, raw SQL strings, or `fq_table(...)`. These are a **must fix**: the query must be moved into a `MemoryEngine` method that returns a typed model, and the handler must call that method.
-- **Verify authentication is enforced in the engine** — the handler must delegate to an engine method that authenticates via `request_context` (`_authenticate_tenant`, typically through `get_bank_profile`). A handler that reads/writes tenant-scoped data without an engine method enforcing auth is a **must fix** (tenant data could leak across schemas).
-
-### 8. Check code comments
-
-For each non-trivial change:
-- **New non-obvious logic** — is there a comment explaining the reasoning?
-- **Changed approach** — does the comment include what was done before and why it changed?
-- **Stale comments** — do existing comments near the changed code still accurately describe the behavior?
-
-### 9. Check integration completeness
-
-If any files in `hindsight-integrations/` were added or changed, verify:
-- **Tests exist** — the integration must have tests that simulate/exercise the external framework (not just pure unit tests of helpers). Check for a `tests/` directory with meaningful test files.
-- **CI job exists** — check `.github/workflows/test.yml` for a corresponding `test-<name>-integration` job. If missing, flag it.
-- **Release process** — check that the integration name is in the `VALID_INTEGRATIONS` array in `scripts/release-integration.sh` AND in the `INTEGRATIONS` dict in `hindsight-dev/hindsight_dev/generate_changelog.py` (the changelog generator keeps its own list; a release fails at the changelog step if the name is missing there). If either is missing, flag it.
-- **Docs gallery + sidebar entry** — the integration must have an entry in `hindsight-docs/src/data/integrations.json`. This file is the **single source of truth** that drives both the integrations gallery and the docs sidebar (the sidebar category is injected from it at render time across all docs versions). The entry needs an internal `/sdks/integrations/<slug>` `link` and a matching page at `hindsight-docs/docs-integrations/<slug>.md(x)`. The `hindsight-docs/scripts/check-integrations.mjs` build step enforces both directions — forward: every internal JSON entry has a doc page; reverse: every released tag (`integrations/<name>/vX.Y.Z`) appears in the JSON (private infra like `cloudflare-oauth-proxy` is in the script's `EXCLUDED` set). Flag any integration that is released (or being released) but missing from `integrations.json`, and any JSON entry without a doc page. Do **not** hand-edit `versioned_sidebars/*.json` to add integration links — they are positional placeholders filled from the JSON.
-- **Code standards** — the integration code must follow all Python style rules (type hints, no raw dicts, no tuple returns, etc.).
-
-### 10. Check MCP tool registration completeness
-
-If any new MCP tools were added or existing tools renamed in `hindsight-api-slim/hindsight_api/mcp_tools.py`:
-- **`_ALL_TOOLS` set** in `mcp_tools.py` — must include the new tool name
-- **`tools_to_register` default set** in `register_mcp_tools()` in `mcp_tools.py` — must include the new tool name
-- **`_SINGLE_BANK_TOOLS` set** in `hindsight-api-slim/hindsight_api/api/mcp.py` — must include the new tool if it is bank-scoped (not a bank-management tool like `list_banks`/`create_bank`)
-- **`MCP_TOOL_GROUPS`** in `hindsight-control-plane/src/components/bank-config-view.tsx` — must include the new tool in the appropriate group for the UI tool selector
-- **Tool count assertions** in tests (e.g., `test_mcp_tools.py`) — must be updated to reflect the new count
-
-### 11. Check backup/restore table coverage
-
-If a migration adds a new PostgreSQL table (look for `CREATE TABLE` / `op.create_table` in `hindsight-api-slim/hindsight_api/alembic/versions/`):
-- **`BACKUP_TABLES`** in `hindsight-api-slim/hindsight_api/admin/cli.py` — must include the new table, placed after any table it references via foreign key (parents before children). A missing entry is silent data loss: the table is never backed up, and restore's `TRUNCATE banks CASCADE` wipes any FK-to-banks child (e.g. `mental_models`, `directives`) on restore even though it was never saved.
-- The guard test `test_backup_tables_covers_entire_schema` in `tests/test_admin_backup_restore.py` enforces this — flag it as a **must fix** if a new table is absent from `BACKUP_TABLES`.
-- Oracle-only tables (e.g. `observation_sources`) are intentionally excluded — admin backup/restore is PostgreSQL-only.
-
-### 11b. Check new config flags update the env template
-
-If the diff adds a new configuration field (a new `ENV_*` / `HINDSIGHT_*` env var
-in `hindsight-api-slim/hindsight_api/config.py`):
-- **`.env.example`** (repo root) — must add the variable (commented if optional)
-  alongside the docs entry in `hindsight-docs/docs/developer/configuration.md`.
-  A flag added to `config.py` but absent from `.env.example` is a **should fix**.
-- **`hindsight-embed/hindsight_embed/env.example`** — the bundled copy must stay
-  byte-identical to the repo-root `.env.example` (it seeds embed/profile configs).
-  The `test_bundled_template_matches_repo_root` sync test fails on drift; if the
-  root file changed without re-copying, flag it as a **must fix**.
-
-### 12. Review against other coding standards
-
-Check the diff for violations of the standards listed above:
-- Python files at project root (not allowed)
-- Missing async patterns (should be async throughout)
-- Pydantic models for request/response
-- Line length > 120 chars
-- New features/code beyond what was asked (over-engineering)
-- Unnecessary error handling for impossible scenarios
-- Premature abstractions or speculative helpers
-- Backwards-compatibility hacks (unused vars, re-exports, "removed" comments)
-
-### 13. Report findings
-
-Present a clear summary organized by severity:
-
-**Must fix** — issues that will break CI or violate hard project rules:
-- Unrelated commits on the branch
-- Lint failures
-- Missing type hints on public functions
-- Raw dict usage for structured data (including internal code)
-- Multi-item tuple returns (including internal code)
-- Missing tests for new endpoints
-- Direct DB access (raw SQL / `acquire_with_retry` / `fq_table`) in an `api/` handler instead of a `MemoryEngine` method
-- Tenant-scoped data accessed without authentication enforced in the engine (`_authenticate_tenant` / `get_bank_profile`)
-- New integration missing tests, CI job, or release-integration.sh entry
-- Released/added integration missing from `hindsight-docs/src/data/integrations.json`, or a JSON entry with no `docs-integrations/<slug>` page (fails the docs build via `check-integrations.mjs`)
-- New PostgreSQL table missing from `BACKUP_TABLES` in `admin/cli.py` (silent data loss on restore)
-
-**Should fix** — issues that hurt code quality:
-- Dead code / unused imports missed by linter
-- Missing tests for non-trivial utility functions
-- Over-engineering beyond the task scope
-
-**Note** — observations that may or may not need action:
-- API changes that might need client regeneration
-- Patterns that deviate from nearby code style
-
-For each finding, include the file path, line number, and a brief explanation.
-
-Do NOT auto-fix any issues. Report all findings and let the user decide what to address. If there are no findings, confirm the code looks good.

.claude/skills/hs-release/SKILL.md

@@ -1,116 +0,0 @@
----
-name: hs-release
-description: Cut a core Hindsight release (vX.Y.Z) and open the changelog + blog PR. Use when asked to cut/start a release, bump the version, or publish a new Hindsight version.
-user_invocable: true
----
-
-# Hindsight Release
-
-Cut a **core** Hindsight release and open the accompanying changelog/blog PR. This is for the core
-product version (API, clients, CLI, control plane, Helm). **Integrations are versioned
-independently** — use `scripts/release-integration.sh` for those, not this skill.
-
-The release is **irreversible and outward-facing**: it tags a version and pushes it straight to
-`main`, which triggers CI that publishes packages to PyPI / npm / Helm. Confirm the version number
-and that the intended fixes are already merged to `main` before you start.
-
-## Step 0 — Pre-flight
-
-1. **Decide the base.** A release is cut from the latest `origin/main`, never from a feature
-   branch. `git fetch origin --tags` first. Confirm the "couple of fixes" the user means are
-   actually merged to `main` (`git log v<prev>..origin/main --oneline`).
-2. **Find where `main` is checked out.** `main` is often already checked out in a sibling worktree
-   (`git worktree list`). You **cannot** check out `main` in a second worktree — run the release in
-   the worktree that already holds it. If that worktree is dirty with throwaway cruft
-   (`.next-*` tsconfig paths, screenshots), `git stash push -u`, fast-forward to `origin/main`,
-   run the release, then `git stash pop`.
-3. **Pitfall:** never pipe the checkout in an `&&` chain like
-   `git checkout main 2>&1 | tail && git reset --hard ...` — the pipe's exit status is `tail`'s
-   (always 0), so a failed checkout won't stop the chain and the `reset` fires on the **wrong
-   branch**. Check out as its own command and verify `git branch --show-current` before resetting.
-
-## Step 1 — Cut the release
-
-Run from the worktree on a clean `main`:
-
-```bash
-./scripts/release.sh <version>     # e.g. 0.8.1  (no leading v)
-```
-
-`release.sh` bumps the version in every component, regenerates the OpenAPI spec + all client SDKs,
-updates docs versioning, commits `Release v<version>`, tags `v<version>`, and **pushes the commit
-and tag directly to `main`**. The push triggers the `Release` GitHub Actions workflow that builds
-and publishes the packages. It is **not** a PR.
-
-Verify after: `gh run list --limit 5` should show the `Release v<version>` workflow running, and
-`git ls-remote --tags origin v<version>` should return the tag.
-
-## Step 2 — Changelog + blog PR (separate)
-
-Done **after** the tag exists, as its own PR (precedent: v0.8.0 = #2053, v0.8.1 = #2080). Work on a
-branch off the new `main`:
-
-```bash
-git checkout -b docs-changelog-<version> origin/main
-```
-
-Only spin up a separate worktree (`git worktree add ../hindsight-changelog-<version> -b
-docs-changelog-<version> origin/main`) if you can't get a clean checkout otherwise — e.g. `main` is
-held in another worktree and the current one has work you don't want to disturb.
-
-**Branch naming:** use the `docs-` (hyphen) convention, e.g. `docs-changelog-0.8.1`. A remote
-branch literally named `docs` exists, so any `docs/...` branch is rejected on push with
-`directory file conflict`.
-
-### Changelog
-
-```bash
-uv run --directory hindsight-dev generate-changelog <version>
-```
-
-LLM-summarizes the commits between the previous tag and `v<version>` and prepends an entry to
-`hindsight-docs/src/pages/changelog/index.md`. Requires `OPENAI_API_KEY` (already in the repo
-`.env`). It excludes `hindsight-integrations/` source, but new integrations whose commits also
-touched docs will still appear — that matches precedent, leave them in the **changelog**.
-
-### Blog post
-
-Hand-write `hindsight-docs/blog/YYYY-MM-DD-version-X-Y-Z.md` (mirror an existing one; patch
-releases are short — see `2026-06-02-version-0-7-2.md`). Guidance:
-
-- **Explain user impact, not internals/mechanism.** Lead with what the user can now do and what to
-  set. Config/env-var names are fine (developer-facing), code symbols and internals are not.
-- **Do not list integrations in the release blog.** The core blog covers core engine / API /
-  ops changes; each integration ships its own changelog. (Integrations may still appear in the
-  generated `changelog/index.md` — that's fine; just keep them out of the blog.)
-- Call out an upgrade recommendation when there are operational/data-integrity fixes.
-- Validate formatting: `npx prettier --check <blog file>`.
-
-### Sync the docs skill
-
-```bash
-./scripts/generate-docs-skill.sh
-```
-
-Refreshes `skills/hindsight-docs/references/changelog/index.md`. It will also bump
-`skills/hindsight-docs/references/openapi.json` by one version — `release.sh` regenerates the skill
-*before* bumping OpenAPI, so the skill copy lags a version in the release commit; this step syncs
-it. Expect a one-line `version` diff there; keep it.
-
-### Commit, push, PR
-
-```bash
-git add -A
-git commit --no-verify -m "docs: changelog and blog post for v<version>"
-git push -u origin docs-changelog-<version>
-gh pr create --base main --title "docs: changelog and blog post for v<version>" --body "..."
-```
-
-Expected files in the PR: the changelog entry, the new blog post, the regenerated skill changelog
-mirror, and the skill `openapi.json` version sync.
-
-## Cleanup
-
-If you created a temporary worktree, remove it once the PR is up
-(`git worktree remove ../hindsight-changelog-<version>`; the branch stays on origin). Restore any
-stash you popped in Step 0.

.env.example

@@ -2,172 +2,38 @@
 # Copy this file to .env and fill in your values
 
 # LLM Configuration (Required)
-# Supported providers: openai, groq, ollama, gemini, anthropic, lmstudio, vertexai, minimax, deepseek, zai, atlas, volcano
+# Supported providers: openai, groq, ollama, gemini, anthropic, lmstudio
 HINDSIGHT_API_LLM_PROVIDER=openai
 HINDSIGHT_API_LLM_API_KEY=your-api-key-here
-HINDSIGHT_API_LLM_MODEL=gpt-4o-mini
+HINDSIGHT_API_LLM_MODEL=o3-mini
 HINDSIGHT_API_LLM_BASE_URL=https://api.openai.com/v1
-# Reasoning effort for providers/models that support it. Examples: low, medium, high, xhigh.
-# HINDSIGHT_API_LLM_REASONING_EFFORT=low
-
-# Sampling temperature for internal LLM calls. Set a number in [0.0, 2.0], or `none`
-# to omit the temperature parameter entirely (required for models that reject explicit
-# temperatures, e.g. Azure gpt-5.5). The global override below applies to every operation;
-# per-operation overrides (defaults: verification=0.0, retain=0.1, reflect=0.9,
-# consolidation=0.0) take precedence.
-# HINDSIGHT_API_LLM_TEMPERATURE=none
-# HINDSIGHT_API_LLM_TEMPERATURE_VERIFICATION=0.0
-# HINDSIGHT_API_LLM_TEMPERATURE_RETAIN=0.1
-# HINDSIGHT_API_LLM_TEMPERATURE_REFLECT=0.9
-# HINDSIGHT_API_LLM_TEMPERATURE_CONSOLIDATION=0.0
 
 # Example: Anthropic Claude configuration
 # HINDSIGHT_API_LLM_PROVIDER=anthropic
 # HINDSIGHT_API_LLM_API_KEY=your-anthropic-api-key
 # HINDSIGHT_API_LLM_MODEL=claude-sonnet-4-20250514
 
-# Example: Google Vertex AI configuration
-# HINDSIGHT_API_LLM_PROVIDER=vertexai
-# HINDSIGHT_API_LLM_MODEL=google/gemini-2.0-flash-001
-# HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=your-gcp-project-id
-# HINDSIGHT_API_LLM_VERTEXAI_REGION=us-central1
-# HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY=/path/to/service-account-key.json  # Optional, uses ADC if not set
-
-# Example: MiniMax configuration (1M context window)
-# HINDSIGHT_API_LLM_PROVIDER=minimax
-# HINDSIGHT_API_LLM_API_KEY=your-minimax-api-key
-# HINDSIGHT_API_LLM_MODEL=MiniMax-M3  # or MiniMax-M2.7 for the previous generation
-
-# Example: DeepSeek configuration (https://api.deepseek.com)
-# HINDSIGHT_API_LLM_PROVIDER=deepseek
-# HINDSIGHT_API_LLM_API_KEY=your-deepseek-api-key
-# HINDSIGHT_API_LLM_MODEL=deepseek-v4-flash  # or deepseek-v4-pro / deepseek-chat / deepseek-reasoner
-
-# Example: z.ai configuration (Zhipu GLM series, https://z.ai)
-# HINDSIGHT_API_LLM_PROVIDER=zai
-# HINDSIGHT_API_LLM_API_KEY=your-zai-api-key
-# HINDSIGHT_API_LLM_MODEL=glm-4.5-flash  # or glm-4.5-air for the paid tier
-
-# Example: Atlas Cloud configuration (OpenAI-compatible, https://www.atlascloud.ai)
-# HINDSIGHT_API_LLM_PROVIDER=atlas
-# HINDSIGHT_API_LLM_API_KEY=your-atlascloud-api-key
-# HINDSIGHT_API_LLM_MODEL=deepseek-ai/deepseek-v4-pro  # reasoning model; also Qwen / GLM / Kimi / MiniMax, etc.
-
 # Example: LM Studio local configuration (Qwen 2.5 32B recommended)
 # HINDSIGHT_API_LLM_PROVIDER=lmstudio
 # HINDSIGHT_API_LLM_API_KEY=lmstudio
 # HINDSIGHT_API_LLM_BASE_URL=http://localhost:1234/v1
 # HINDSIGHT_API_LLM_MODEL=qwen2.5-32b-instruct
 
-# Multi-LLM strategies: configure extra LLMs by index alongside the primary above,
-# then pick a routing strategy. Unset = single primary LLM (default). Members are
-# numbered from 1; indices must be contiguous. Each operation can override with a
-# RETAIN_/REFLECT_/CONSOLIDATION_ prefix (e.g. HINDSIGHT_API_RETAIN_LLM_1_PROVIDER).
-# HINDSIGHT_API_LLM_1_PROVIDER=groq
-# HINDSIGHT_API_LLM_1_API_KEY=your-groq-api-key
-# HINDSIGHT_API_LLM_1_MODEL=openai/gpt-oss-120b
-# HINDSIGHT_API_LLM_2_PROVIDER=anthropic
-# HINDSIGHT_API_LLM_2_API_KEY=your-anthropic-api-key
-# Strategy JSON: {"mode": "failover"} or {"mode": "round-robin"}.
-# Round-robin accepts optional positive-int "weights" (one per member, primary first).
-# HINDSIGHT_API_LLM_STRATEGY={"mode": "failover"}
-
 # API Configuration (Optional)
 HINDSIGHT_API_HOST=0.0.0.0
 HINDSIGHT_API_PORT=8888
 HINDSIGHT_API_LOG_LEVEL=info
-# Optional retain chunking override for structured logs/transcripts.
-# Unset uses HINDSIGHT_API_RETAIN_CHUNK_SIZE as the structured-chunk limit.
-# HINDSIGHT_API_RETAIN_STRUCTURED_CHUNK_SIZE=
-
-# Dry-run extraction preview endpoint (POST /memories/dry-run-extract). Enabled by default; it makes
-# a real LLM call but stores nothing. Set to false to remove the endpoint (returns 404).
-# HINDSIGHT_API_ENABLE_DRY_RUN_EXTRACT=true
-
-# Base Path / Reverse Proxy Support (Optional)
-# Set these when deploying behind a reverse proxy with path-based routing
-# Example: To deploy at example.com/hindsight/, set both to "/hindsight"
-# HINDSIGHT_API_BASE_PATH=/hindsight
-# NEXT_PUBLIC_BASE_PATH=/hindsight
 
 # Database (Optional - uses embedded pg0 by default)
 # HINDSIGHT_API_DATABASE_URL=postgresql://user:pass@host:5432/db
-# HINDSIGHT_API_READ_DATABASE_URL=  # Optional read-replica URL. When set, recall queries (semantic, BM25, graph, temporal) flow through a separate pool against this URL, offloading the primary. Typically points to a read-only endpoint (CNPG's <cluster>-ro service or Aurora reader endpoint).
-# HINDSIGHT_API_MIGRATION_DATABASE_URL=  # Direct PostgreSQL URL for migrations (bypasses PgBouncer). Falls back to DATABASE_URL.
-# HINDSIGHT_API_DATABASE_SCHEMA=public  # PostgreSQL schema name (default: public)
-# HINDSIGHT_API_MIGRATION_CONCURRENCY=1  # Tenant schemas to migrate concurrently (PG only, each in its own process; per-schema work stays sequential). Each worker has ~1-2s startup cost + uses ~3 DB connections, so it only pays off with many schemas (tens+) or slow migrations; keep concurrency*3 <= spare max_connections. Default: 1 (sequential).
-
-# Vector Extension (Optional - uses pgvector by default)
-# Options: "pgvector" (default), "vchord", "pgvectorscale" (DiskANN)
-# HINDSIGHT_API_VECTOR_EXTENSION=pgvector
-# For Azure PostgreSQL with DiskANN:
-# HINDSIGHT_API_VECTOR_EXTENSION=pgvectorscale  # Auto-detects pg_diskann on Azure
-
-# Text Search Extension (Optional - uses native PostgreSQL full-text search by default)
-# Backend options: "native" (default), "vchord", "pg_textsearch", "pgroonga", "pg_search"
-# HINDSIGHT_API_TEXT_SEARCH_EXTENSION=native
-# Native backend dictionary (only used by HINDSIGHT_API_TEXT_SEARCH_EXTENSION=native)
-# HINDSIGHT_API_TEXT_SEARCH_EXTENSION_NATIVE_LANGUAGE=english
-# ParadeDB pg_search tokenizer (only used when creating pg_search BM25 indexes).
-# Empty uses ParadeDB's default tokenizer: unicode_words.
-# Supported values: unicode_words, simple, whitespace, literal, literal_normalized,
-# chinese_compatible, icu, jieba, source_code,
-# chinese_lindera/lindera(chinese), japanese_lindera/lindera(japanese),
-# korean_lindera/lindera(korean), ngram(min,max), edge_ngram(min,max)
-# HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER=
-
-# File Parser (Optional - uses markitdown by default)
-# HINDSIGHT_API_FILE_PARSER=markitdown
-# Enable image OCR for MarkItDown using an OpenAI-compatible OCR/vision endpoint.
-# These OCR settings are independent from HINDSIGHT_API_LLM_* because MarkItDown
-# uses the OpenAI SDK directly and requires Chat Completions image input support.
-# When OCR is enabled, API_KEY, BASE_URL, and MODEL are required.
-# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_ENABLED=false
-# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_API_KEY=
-# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_BASE_URL=
-# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_MODEL=
-# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_PROMPT=
 
 # Embeddings Configuration (Optional - uses local by default)
-# Provider: "local" (default), "onnx", "tei", "openai", "cohere", "google", "openrouter", "zeroentropy", "litellm", or "litellm-sdk"
+# Provider: "local" (default) or "tei" (HuggingFace Text Embeddings Inference)
 # HINDSIGHT_API_EMBEDDINGS_PROVIDER=local
 # For local provider:
 # HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL=BAAI/bge-small-en-v1.5
-# For ONNX provider (local CPU embeddings without an Ollama/TEI sidecar):
-# HINDSIGHT_API_EMBEDDINGS_PROVIDER=onnx
-# HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_ID=intfloat/multilingual-e5-small
-# HINDSIGHT_API_EMBEDDINGS_ONNX_FILE=onnx/model.onnx
-# HINDSIGHT_API_EMBEDDINGS_ONNX_DIMENSIONS=384
-# HINDSIGHT_API_EMBEDDINGS_ONNX_MAX_TOKENS=512
-# HINDSIGHT_API_EMBEDDINGS_ONNX_POOLING=mean
-# HINDSIGHT_API_EMBEDDINGS_ONNX_NORMALIZE=true
-# HINDSIGHT_API_EMBEDDINGS_ONNX_QUERY_PREFIX="query: "
-# HINDSIGHT_API_EMBEDDINGS_ONNX_PASSAGE_PREFIX="passage: "
-# Optional for local model paths or pre-downloaded artifacts:
-# HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_PATH=/models/multilingual-e5-small/onnx/model.onnx
-# HINDSIGHT_API_EMBEDDINGS_ONNX_TOKENIZER_NAME_OR_PATH=/models/multilingual-e5-small
-# Optional for China network / restricted HF access:
-# HF_ENDPOINT=https://hf-mirror.com
 # For TEI provider:
 # HINDSIGHT_API_EMBEDDINGS_TEI_URL=http://localhost:8080
-# For OpenAI-compatible embeddings:
-# HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=sk-xxxx
-# HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL=text-embedding-3-small
-# HINDSIGHT_API_EMBEDDINGS_OPENAI_BASE_URL=https://api.openai.com/v1
-# For ZeroEntropy zembed-1:
-# HINDSIGHT_API_EMBEDDINGS_PROVIDER=zeroentropy
-# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_API_KEY=ze-xxxx
-# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_MODEL=zembed-1
-# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_DIMENSIONS=1280
-# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_ENCODING_FORMAT=float
-# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_LATENCY=fast
-#
-# IMPORTANT: Embedding keys require provider-specific names:
-# HINDSIGHT_API_EMBEDDINGS_{PROVIDER}_{PARAMETER}
-# (for example, HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL).
-#
-# DeepSeek note: DeepSeek is supported for LLM calls, but not for embeddings.
-# If using DeepSeek as LLM provider, keep embeddings on local/openai/cohere/google/etc.
 
 # Reranker Configuration (Optional - uses local by default)
 # Provider: "local" (default) or "tei" (HuggingFace Text Embeddings Inference)
@@ -176,39 +42,3 @@ HINDSIGHT_API_LOG_LEVEL=info
 # HINDSIGHT_API_RERANKER_LOCAL_MODEL=cross-encoder/ms-marco-MiniLM-L-6-v2
 # For TEI provider:
 # HINDSIGHT_API_RERANKER_TEI_URL=http://localhost:8081
-
-# Observability & Tracing (Optional - disabled by default)
-# Enable OpenTelemetry tracing for LLM calls (GenAI semantic conventions)
-# HINDSIGHT_API_OTEL_TRACES_ENABLED=true
-#
-# Local development with Grafana LGTM stack (recommended - see scripts/dev/grafana/README.md)
-# HINDSIGHT_API_OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
-#
-# Cloud backends (Grafana Cloud, Langfuse, DataDog, etc.)
-# HINDSIGHT_API_OTEL_EXPORTER_OTLP_ENDPOINT=https://your-backend-url
-# HINDSIGHT_API_OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer your-token"
-#
-# Custom service name and environment (optional, defaults: hindsight-api, development)
-# HINDSIGHT_API_OTEL_SERVICE_NAME=hindsight-production
-# HINDSIGHT_API_OTEL_DEPLOYMENT_ENVIRONMENT=production
-#
-# Expose async-operation queue + consolidation-backlog gauges on /metrics.
-# Runs periodic per-schema COUNT queries on a background task (disabled by default).
-# HINDSIGHT_API_METRICS_BACKLOG_ENABLED=true
-
-# -----------------------------------------------------------------------------
-# Control Plane (Optional)
-# -----------------------------------------------------------------------------
-
-# Dataplane API URL - where the CP proxies requests to
-# HINDSIGHT_CP_DATAPLANE_API_URL=http://localhost:8888
-
-# Optional: Bearer token the CP sends as `Authorization: Bearer <key>` to the
-# dataplane API. Required when the API service is auth-protected; omit for a
-# public/unauthenticated API.
-# HINDSIGHT_CP_DATAPLANE_API_KEY=your-dataplane-bearer-token
-
-# Optional: Require a shared access key to view the Control Plane UI.
-# When set, visitors see a login page and must enter the key before
-# accessing the dashboard or any /api/* routes (except /api/health).
-# HINDSIGHT_CP_ACCESS_KEY=your-shared-secret-key

.github/workflows/deploy-docs.yml

@@ -21,22 +21,17 @@ jobs:
   build:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v6
-        with:
-          fetch-depth: 0 # fetch tags so check-released-integrations can see them
-      - uses: actions/setup-node@v6
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
         with:
           node-version: 20
           cache: npm
           cache-dependency-path: package-lock.json
-      - uses: astral-sh/setup-uv@v7
+      - uses: astral-sh/setup-uv@v4
       - run: npm ci --workspace=hindsight-docs
       - run: uv run generate-llms-full
       - run: npm run build --workspace=hindsight-docs
-        env:
-          UMAMI_URL: https://analytics.hindsight.vectorize.io
-          UMAMI_WEBSITE_ID: ${{ secrets.UMAMI_WEBSITE_ID }}
-      - uses: actions/upload-pages-artifact@v5
+      - uses: actions/upload-pages-artifact@v3
         with:
           path: hindsight-docs/build
   deploy:
@@ -46,5 +41,5 @@ jobs:
     runs-on: ubuntu-latest
     needs: build
     steps:
-      - uses: actions/deploy-pages@v5
+      - uses: actions/deploy-pages@v4
         id: deployment

.github/workflows/perf-test.yml

@@ -1,297 +0,0 @@
-name: Performance Tests
-
-on:
-  schedule:
-    # Run daily at 06:00 UTC
-    - cron: "0 6 * * *"
-  workflow_dispatch:
-    inputs:
-      scale:
-        description: "Test scale (perf-test)"
-        type: choice
-        options:
-          - tiny
-          - small
-          - medium
-          - large
-        default: large
-      suite:
-        description: "Perf-test suite to run (blank = all)"
-        type: choice
-        options:
-          - ""
-          - retain
-          - recall
-          - recall-with-observations
-          - recall-temporal
-          - consolidation
-          - graph-maintenance
-        default: ""
-      locomo_conversations:
-        description: "LoComo conversation IDs (space-separated). Blank = curated set (conv-26 conv-30 conv-43)."
-        type: string
-        default: ""
-      locomo_skip:
-        description: "Skip LoComo job"
-        type: boolean
-        default: false
-      obs_skip:
-        description: "Skip observation-dedup benchmark job"
-        type: boolean
-        default: false
-      obs_dataset:
-        description: "Obs benchmark dataset substring (blank = English hermes transcript)."
-        type: string
-        default: ""
-      obs_fraction:
-        description: "Obs benchmark fraction (0-1] of each document to run."
-        type: string
-        default: "1.0"
-      ref:
-        description: "Git ref to test (branch, tag, or SHA). Defaults to main."
-        type: string
-        default: ""
-
-concurrency:
-  group: perf-test
-  cancel-in-progress: true
-
-jobs:
-  perf-test:
-    runs-on: ubuntu-latest
-    steps:
-    - uses: actions/checkout@v6
-      with:
-        ref: ${{ inputs.ref || github.ref }}
-
-    - name: Install uv
-      uses: astral-sh/setup-uv@v7
-      with:
-        enable-cache: true
-        prune-cache: false
-
-    - name: Set up Python
-      uses: actions/setup-python@v6
-      with:
-        python-version-file: ".python-version"
-
-    - name: Cache HuggingFace models
-      uses: actions/cache@v5
-      with:
-        path: ~/.cache/huggingface
-        key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
-        restore-keys: |
-          ${{ runner.os }}-huggingface-
-
-    - name: Pre-download models
-      working-directory: ./hindsight-api-slim
-      run: |
-        uv run --frozen --all-extras --index-strategy unsafe-best-match python -c "
-        from sentence_transformers import SentenceTransformer
-        print('Downloading embedding model...')
-        SentenceTransformer('BAAI/bge-small-en-v1.5')
-        print('Model downloaded successfully')
-        "
-
-    - name: Install hindsight-dev dependencies
-      run: |
-        cd hindsight-dev && uv sync --frozen --all-extras --index-strategy unsafe-best-match
-
-    - name: Run perf-test
-      run: |
-        SUITE_ARG=""
-        if [ -n "${{ inputs.suite }}" ]; then
-          SUITE_ARG="--suite ${{ inputs.suite }}"
-        fi
-        ./scripts/benchmarks/run-perf-test.sh \
-          --scale ${{ inputs.scale || 'large' }} \
-          $SUITE_ARG \
-          --output perf-results.json
-
-    - name: Upload perf results
-      if: always()
-      uses: actions/upload-artifact@v7
-      with:
-        name: perf-results-${{ github.sha }}
-        path: hindsight-dev/perf-results.json
-        retention-days: 90
-
-    # Publish enriched results (perf JSON + commit metadata) to the dashboard
-    # repo's gh-pages branch. The static site at
-    #   https://vectorize-io.github.io/hindsight-continuous-performance-monitor/
-    # reads data/index.json + data/<run>.json and renders charts client-side.
-    - name: Publish to dashboard
-      if: github.event_name == 'schedule' || github.event_name == 'push' || github.event_name == 'workflow_dispatch'
-      env:
-        PERF_DASHBOARD_TOKEN: ${{ secrets.PERF_DASHBOARD_TOKEN }}
-        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      run: ./scripts/benchmarks/publish-perf-results.sh hindsight-dev/perf-results.json
-
-  locomo:
-    if: inputs.locomo_skip != true
-    runs-on: ubuntu-latest
-    env:
-      HINDSIGHT_API_LLM_PROVIDER: vertexai
-      HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
-      HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
-      HINDSIGHT_API_JUDGE_LLM_PROVIDER: vertexai
-      HINDSIGHT_API_JUDGE_LLM_MODEL: google/gemini-2.5-flash-lite
-      HINDSIGHT_API_ANSWER_LLM_PROVIDER: vertexai
-      HINDSIGHT_API_ANSWER_LLM_MODEL: google/gemini-2.5-flash
-    steps:
-    - uses: actions/checkout@v6
-      with:
-        ref: ${{ inputs.ref || github.ref }}
-
-    - name: Setup GCP credentials
-      run: |
-        printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
-        PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
-        echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
-
-    - name: Install uv
-      uses: astral-sh/setup-uv@v7
-      with:
-        enable-cache: true
-        prune-cache: false
-
-    - name: Set up Python
-      uses: actions/setup-python@v6
-      with:
-        python-version-file: ".python-version"
-
-    - name: Cache HuggingFace models
-      uses: actions/cache@v5
-      with:
-        path: ~/.cache/huggingface
-        key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
-        restore-keys: |
-          ${{ runner.os }}-huggingface-
-
-    - name: Pre-download models
-      working-directory: ./hindsight-api-slim
-      run: |
-        uv run --frozen --all-extras --index-strategy unsafe-best-match python -c "
-        from sentence_transformers import SentenceTransformer
-        print('Downloading embedding model...')
-        SentenceTransformer('BAAI/bge-small-en-v1.5')
-        print('Model downloaded successfully')
-        "
-
-    - name: Install hindsight-dev dependencies
-      run: |
-        cd hindsight-dev && uv sync --frozen --all-extras --index-strategy unsafe-best-match
-
-    - name: Run LoComo benchmark
-      # Curated 3-conversation subset (best/middle/worst by accuracy on the
-      # last successful full run): conv-26 (best), conv-30 (middle), conv-43
-      # (worst). Excludes conv-44, the bank with the largest unconsolidated
-      # set that has been pushing scheduled runs over the per-bank
-      # _wait_for_consolidation timeout. Override via workflow_dispatch with
-      # the locomo_conversations input.
-      run: |
-        CONVERSATIONS="${{ inputs.locomo_conversations }}"
-        if [ -z "$CONVERSATIONS" ]; then
-          CONVERSATIONS="conv-26 conv-30 conv-43"
-        fi
-        uv run python hindsight-dev/benchmarks/locomo/locomo_benchmark.py \
-          --wait-consolidation \
-          --conversation $CONVERSATIONS
-
-    - name: Upload LoComo results
-      if: always()
-      uses: actions/upload-artifact@v7
-      with:
-        name: locomo-results-${{ github.sha }}
-        path: hindsight-dev/benchmarks/locomo/results/
-        retention-days: 90
-
-    - name: Publish LoComo to dashboard
-      if: success() && (github.event_name == 'schedule' || github.event_name == 'push' || github.event_name == 'workflow_dispatch')
-      env:
-        PERF_DASHBOARD_TOKEN: ${{ secrets.PERF_DASHBOARD_TOKEN }}
-        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      run: ./scripts/benchmarks/publish-locomo-results.sh hindsight-dev/benchmarks/locomo/results/benchmark_results.json
-
-  obs:
-    # Observation-dedup quality benchmark: ingests a transcript, drains consolidation
-    # (serial SyncTaskBackend + embedded pg0 — no external DB / worker), and reports the
-    # near-duplicate observation rate. Real LLM via VertexAI, mirroring the LoComo job.
-    if: inputs.obs_skip != true
-    runs-on: ubuntu-latest
-    env:
-      HINDSIGHT_API_LLM_PROVIDER: vertexai
-      HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
-      HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
-      HINDSIGHT_API_ENABLE_OBSERVATIONS: "true"
-    steps:
-    - uses: actions/checkout@v6
-      with:
-        ref: ${{ inputs.ref || github.ref }}
-
-    - name: Setup GCP credentials
-      run: |
-        printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
-        PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
-        echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
-
-    - name: Install uv
-      uses: astral-sh/setup-uv@v7
-      with:
-        enable-cache: true
-        prune-cache: false
-
-    - name: Set up Python
-      uses: actions/setup-python@v6
-      with:
-        python-version-file: ".python-version"
-
-    - name: Cache HuggingFace models
-      uses: actions/cache@v5
-      with:
-        path: ~/.cache/huggingface
-        key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
-        restore-keys: |
-          ${{ runner.os }}-huggingface-
-
-    - name: Pre-download models
-      working-directory: ./hindsight-api-slim
-      run: |
-        uv run --frozen --all-extras --index-strategy unsafe-best-match python -c "
-        from sentence_transformers import SentenceTransformer
-        print('Downloading embedding model...')
-        SentenceTransformer('BAAI/bge-small-en-v1.5')
-        print('Model downloaded successfully')
-        "
-
-    - name: Install hindsight-dev dependencies
-      run: |
-        cd hindsight-dev && uv sync --frozen --all-extras --index-strategy unsafe-best-match
-
-    - name: Run obs benchmark
-      # Default to the English hermes transcript at full fraction — a clean, deterministic
-      # consolidation-dedup signal (the Chinese variant adds a cross-lingual embedding
-      # confound). Override dataset/fraction via workflow_dispatch.
-      run: |
-        DATASET="${{ inputs.obs_dataset }}"
-        if [ -z "$DATASET" ]; then DATASET="hermes_session_2026-05-15_en"; fi
-        FRACTION="${{ inputs.obs_fraction }}"
-        if [ -z "$FRACTION" ]; then FRACTION="1.0"; fi
-        cd hindsight-dev
-        uv run python -m benchmarks.obs.obs_benchmark \
-          --dataset "$DATASET" --fraction "$FRACTION" --wipe-bank --output obs-results.json
-
-    - name: Upload obs results
-      if: always()
-      uses: actions/upload-artifact@v7
-      with:
-        name: obs-results-${{ github.sha }}
-        path: hindsight-dev/obs-results.json
-        retention-days: 90
-
-    - name: Publish obs to dashboard
-      if: success() && (github.event_name == 'schedule' || github.event_name == 'push' || github.event_name == 'workflow_dispatch')
-      env:
-        PERF_DASHBOARD_TOKEN: ${{ secrets.PERF_DASHBOARD_TOKEN }}
-        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      run: ./scripts/benchmarks/publish-obs-results.sh hindsight-dev/obs-results.json

.github/workflows/release-integration.yml

@@ -1,205 +0,0 @@
-name: Release Integration
-
-on:
-  push:
-    tags:
-      - 'integrations/**'
-
-jobs:
-  publish:
-    runs-on: ubuntu-latest
-    permissions:
-      id-token: write  # for PyPI trusted publishing + build-provenance attestations
-      attestations: write  # for actions/attest-build-provenance (Obsidian assets)
-      # No `contents: write`: we never create releases in this repo. The Obsidian
-      # plugin's distribution release is pushed to its dedicated repo using
-      # OBSIDIAN_DIST_TOKEN (see the "Mirror Obsidian plugin" step below).
-
-    steps:
-    - uses: actions/checkout@v6
-
-    - name: Extract integration info
-      id: info
-      run: |
-        # refs/tags/integrations/litellm/v0.1.0 → integration=litellm, version=0.1.0
-        TAG="${GITHUB_REF#refs/tags/}"
-        INTEGRATION=$(echo "$TAG" | cut -d'/' -f2)
-        VERSION=$(echo "$TAG" | cut -d'/' -f3 | sed 's/^v//')
-        echo "integration=$INTEGRATION" >> $GITHUB_OUTPUT
-        echo "version=$VERSION" >> $GITHUB_OUTPUT
-        echo "tag=$TAG" >> $GITHUB_OUTPUT
-        echo "Integration: $INTEGRATION, Version: $VERSION"
-
-    - name: Detect integration type
-      id: type
-      run: |
-        if [ -f "hindsight-integrations/${{ steps.info.outputs.integration }}/pyproject.toml" ]; then
-          echo "type=python" >> $GITHUB_OUTPUT
-        elif [ -f "hindsight-integrations/${{ steps.info.outputs.integration }}/package.json" ]; then
-          echo "type=typescript" >> $GITHUB_OUTPUT
-        else
-          echo "type=plugin" >> $GITHUB_OUTPUT
-        fi
-
-    # ── Python integrations (litellm, pydantic-ai, crewai) ──────────────────
-
-    - name: Install uv
-      if: steps.type.outputs.type == 'python'
-      uses: astral-sh/setup-uv@v7
-      with:
-        enable-cache: true
-
-    - name: Set up Python
-      if: steps.type.outputs.type == 'python'
-      uses: actions/setup-python@v6
-      with:
-        python-version-file: ".python-version"
-
-    - name: Build Python package
-      if: steps.type.outputs.type == 'python'
-      working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
-      run: uv build --out-dir dist
-
-    - name: Publish Python package to PyPI
-      if: steps.type.outputs.type == 'python'
-      uses: pypa/gh-action-pypi-publish@release/v1
-      with:
-        packages-dir: ./hindsight-integrations/${{ steps.info.outputs.integration }}/dist
-        skip-existing: true
-
-    # ── TypeScript integrations (ai-sdk, chat, openclaw) ────────────────────
-
-    # ── Plugin integrations (claude-code) — no package to publish ───────────
-
-    - name: Plugin release
-      if: steps.type.outputs.type == 'plugin'
-      run: |
-        echo "Plugin integration ${{ steps.info.outputs.integration }} v${{ steps.info.outputs.version }} — no package to publish."
-        echo "Users install via: claude plugin marketplace add vectorize-io/hindsight"
-
-    # ── TypeScript integrations (ai-sdk, chat, openclaw) ────────────────────
-
-    - name: Set up Node.js
-      if: steps.type.outputs.type == 'typescript'
-      uses: actions/setup-node@v6
-      with:
-        node-version: '22'
-        registry-url: 'https://registry.npmjs.org'
-        cache: 'npm'
-        cache-dependency-path: package-lock.json
-
-    - name: Check integration lockfile
-      if: steps.type.outputs.type == 'typescript'
-      run: ./scripts/check-integration-lockfiles.sh
-
-    # Some integrations depend on workspace packages (hindsight-client,
-    # hindsight-all, hindsight-agent-sdk) via file: refs. Install from root
-    # so npm resolves them, then build the workspace deps before the integration.
-    - name: Install root workspace dependencies
-      if: steps.type.outputs.type == 'typescript'
-      run: npm ci
-
-    - name: Build workspace deps (hindsight-client, hindsight-all, hindsight-agent-sdk)
-      if: steps.type.outputs.type == 'typescript'
-      run: |
-        npm run build --workspace=hindsight-clients/typescript
-        npm run build --workspace=hindsight-all-npm
-        npm run build --workspace=hindsight-tools/hindsight-agent-sdk
-
-    - name: Install integration dependencies
-      if: steps.type.outputs.type == 'typescript'
-      working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
-      run: npm ci
-
-    - name: Build TypeScript package
-      if: steps.type.outputs.type == 'typescript'
-      working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
-      run: npm run build
-
-    # Build-provenance attestations for the Obsidian release assets (community-store
-    # recommendation). Runs after the build so main.js exists. The assets are
-    # released in the dedicated repo while the build runs here, so users verify at
-    # owner scope: `gh attestation verify main.js --owner vectorize-io`.
-    - name: Attest Obsidian plugin build provenance
-      if: steps.type.outputs.type == 'typescript' && steps.info.outputs.integration == 'obsidian'
-      uses: actions/attest-build-provenance@v2
-      with:
-        subject-path: |
-          hindsight-integrations/obsidian/main.js
-          hindsight-integrations/obsidian/styles.css
-
-    # ── Obsidian plugin — mirror to its dedicated repo + cut the BRAT release ──
-    # We do NOT create a GitHub Release in this monorepo: per-integration
-    # releases pollute the repo's release list (it's for the core product) and
-    # steal the "Latest" badge, and BRAT / the community store read a repo's
-    # *latest* release — not a tag — so they can't target a tag in a monorepo.
-    #
-    # Instead this monorepo stays the source of truth, and on each obsidian
-    # release we mirror hindsight-integrations/obsidian/ → the *root* of
-    # github.com/vectorize-io/hindsight-obsidian (git subtree, history
-    # preserved) and cut the BRAT / community-store release *there*.
-    #
-    # Requires secret OBSIDIAN_DIST_TOKEN — a token with `contents: write` on
-    # vectorize-io/hindsight-obsidian (fine-grained PAT or app installation
-    # token). The dedicated repo is generated; never edit it directly.
-    - name: Mirror Obsidian plugin to its dedicated repo
-      if: steps.type.outputs.type == 'typescript' && steps.info.outputs.integration == 'obsidian'
-      env:
-        DIST_TOKEN: ${{ secrets.OBSIDIAN_DIST_TOKEN }}
-      run: |
-        set -euo pipefail
-        VERSION="${{ steps.info.outputs.version }}"
-        DIST_REPO="vectorize-io/hindsight-obsidian"
-        OBS_DIR="hindsight-integrations/obsidian"
-
-        # `git subtree split` needs full history; the default checkout is shallow.
-        git fetch --unshallow 2>/dev/null || true
-        git config user.name "github-actions[bot]"
-        git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
-
-        # The runner injects the default GITHUB_TOKEN as an http.extraheader via
-        # an *included* config file (/home/runner/work/_temp/git-credentials-*.config),
-        # so `git config --local --unset-all` can't remove it and it authenticates
-        # the push as github-actions[bot] (no access to the dedicated repo → 403).
-        # The documented way to drop an inherited extraheader is to RESET the list
-        # with an empty value: since command-line `-c` is read last, the empty
-        # value clears the accumulated headers (including the included one) at
-        # request-build time. The dist token then comes from the push URL → a
-        # single Authorization header.
-        git subtree split --prefix="$OBS_DIR" -b _obs_dist
-        git -c "http.https://github.com/.extraheader=" \
-          push "https://x-access-token:${DIST_TOKEN}@github.com/${DIST_REPO}.git" _obs_dist:main
-
-        # Cut the BRAT / community-store release. Bare version tag (e.g. 0.1.0)
-        # to match manifest.json — idempotent so re-runs just refresh the assets.
-        export GH_TOKEN="$DIST_TOKEN"
-        ASSETS="$OBS_DIR/main.js $OBS_DIR/manifest.json $OBS_DIR/styles.css"
-        NOTES="Hindsight for Obsidian v${VERSION}. Install via BRAT (add ${DIST_REPO}) or copy main.js/manifest.json/styles.css into <vault>/.obsidian/plugins/hindsight/."
-        if gh release view "$VERSION" --repo "$DIST_REPO" >/dev/null 2>&1; then
-          gh release upload "$VERSION" $ASSETS --repo "$DIST_REPO" --clobber
-        else
-          gh release create "$VERSION" $ASSETS --repo "$DIST_REPO" --title "$VERSION" --notes "$NOTES"
-        fi
-
-    - name: Publish TypeScript package to npm
-      if: steps.type.outputs.type == 'typescript'
-      working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
-      run: |
-        set +e
-        OUTPUT=$(npm publish --access public --provenance 2>&1)
-        EXIT_CODE=$?
-        echo "$OUTPUT"
-        if [ $EXIT_CODE -ne 0 ]; then
-          # Treat "already published" as success so re-pointed-tag re-runs stay green.
-          # "cannot publish over" = the version exists. TLOG_CREATE_ENTRY_ERROR / 409
-          # "equivalent entry already exists in the transparency log" = the identical
-          # --provenance artifact was already logged on a prior run (Sigstore tlog is
-          # idempotent); the package is published, so this is benign.
-          if echo "$OUTPUT" | grep -qE "cannot publish over|TLOG_CREATE_ENTRY_ERROR|already exists in the transparency log"; then
-            echo "Package version already published, skipping..."
-            exit 0
-          fi
-          exit $EXIT_CODE
-        fi
-      env:
-        NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

.github/workflows/release-tool.yml

@@ -1,65 +0,0 @@
-name: Release Tool
-
-on:
-  push:
-    tags:
-      - 'tools/**'
-
-jobs:
-  publish:
-    runs-on: ubuntu-latest
-
-    steps:
-    - uses: actions/checkout@v6
-
-    - name: Extract tool info
-      id: info
-      run: |
-        # refs/tags/tools/self-driving-agents/v0.0.1 → tool=self-driving-agents, version=0.0.1
-        TAG="${GITHUB_REF#refs/tags/}"
-        TOOL=$(echo "$TAG" | cut -d'/' -f2)
-        VERSION=$(echo "$TAG" | cut -d'/' -f3 | sed 's/^v//')
-        echo "tool=$TOOL" >> $GITHUB_OUTPUT
-        echo "version=$VERSION" >> $GITHUB_OUTPUT
-        echo "tag=$TAG" >> $GITHUB_OUTPUT
-        echo "Tool: $TOOL, Version: $VERSION"
-
-    - name: Set up Node.js
-      uses: actions/setup-node@v6
-      with:
-        node-version: '22'
-        registry-url: 'https://registry.npmjs.org'
-        cache: 'npm'
-        cache-dependency-path: package-lock.json
-
-    # Tools live under hindsight-tools/ and may depend on workspace packages
-    # (e.g. @vectorize-io/hindsight-client). Install from root so npm resolves
-    # workspace deps, then build any required workspace packages first.
-    - name: Install root workspace dependencies
-      run: npm ci
-
-    - name: Build hindsight-client (workspace dep)
-      run: npm run build --workspace=hindsight-clients/typescript
-
-    - name: Build hindsight-agent-sdk (workspace dep)
-      run: npm run build --workspace=hindsight-tools/hindsight-agent-sdk
-
-    - name: Build tool
-      run: npm run build --workspace=hindsight-tools/${{ steps.info.outputs.tool }}
-
-    - name: Publish to npm
-      working-directory: ./hindsight-tools/${{ steps.info.outputs.tool }}
-      run: |
-        set +e
-        OUTPUT=$(npm publish --access public 2>&1)
-        EXIT_CODE=$?
-        echo "$OUTPUT"
-        if [ $EXIT_CODE -ne 0 ]; then
-          if echo "$OUTPUT" | grep -q "cannot publish over"; then
-            echo "Package version already published, skipping..."
-            exit 0
-          fi
-          exit $EXIT_CODE
-        fi
-      env:
-        NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

.github/workflows/release.yml

@@ -13,15 +13,15 @@ jobs:
       id-token: write
 
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v4
 
     - name: Install uv
-      uses: astral-sh/setup-uv@v7
+      uses: astral-sh/setup-uv@v5
       with:
         enable-cache: true
 
     - name: Set up Python
-      uses: actions/setup-python@v6
+      uses: actions/setup-python@v5
       with:
         python-version-file: ".python-version"
 
@@ -30,39 +30,29 @@ jobs:
       working-directory: ./hindsight-clients/python
       run: uv build --out-dir dist
 
-    - name: Build hindsight-api-slim
-      working-directory: ./hindsight-api-slim
-      run: uv build --out-dir dist
-
     - name: Build hindsight-api
       working-directory: ./hindsight-api
       run: uv build --out-dir dist
 
     - name: Build hindsight-all
-      working-directory: ./hindsight-all
+      working-directory: ./hindsight
       run: uv build --out-dir dist
 
-    - name: Build hindsight-all-slim
-      working-directory: ./hindsight-all-slim
+    - name: Build hindsight-litellm
+      working-directory: ./hindsight-integrations/litellm
       run: uv build --out-dir dist
 
     - name: Build hindsight-embed
       working-directory: ./hindsight-embed
       run: uv build --out-dir dist
 
-    # Publish in order (client and api-slim first, then api/all wrappers which depend on them)
+    # Publish in order (client and api first, then hindsight-all which depends on them)
     - name: Publish hindsight-client to PyPI
       uses: pypa/gh-action-pypi-publish@release/v1
       with:
         packages-dir: ./hindsight-clients/python/dist
         skip-existing: true
 
-    - name: Publish hindsight-api-slim to PyPI
-      uses: pypa/gh-action-pypi-publish@release/v1
-      with:
-        packages-dir: ./hindsight-api-slim/dist
-        skip-existing: true
-
     - name: Publish hindsight-api to PyPI
       uses: pypa/gh-action-pypi-publish@release/v1
       with:
@@ -72,13 +62,13 @@ jobs:
     - name: Publish hindsight-all to PyPI
       uses: pypa/gh-action-pypi-publish@release/v1
       with:
-        packages-dir: ./hindsight-all/dist
+        packages-dir: ./hindsight/dist
         skip-existing: true
 
-    - name: Publish hindsight-all-slim to PyPI
+    - name: Publish hindsight-litellm to PyPI
       uses: pypa/gh-action-pypi-publish@release/v1
       with:
-        packages-dir: ./hindsight-all-slim/dist
+        packages-dir: ./hindsight-integrations/litellm/dist
         skip-existing: true
 
     - name: Publish hindsight-embed to PyPI
@@ -89,15 +79,14 @@ jobs:
 
     # Upload artifacts for GitHub release
     - name: Upload artifacts
-      uses: actions/upload-artifact@v7
+      uses: actions/upload-artifact@v4
       with:
         name: python-packages
         path: |
           hindsight-clients/python/dist/*
-          hindsight-api-slim/dist/*
           hindsight-api/dist/*
-          hindsight-all/dist/*
-          hindsight-all-slim/dist/*
+          hindsight/dist/*
+          hindsight-integrations/litellm/dist/*
           hindsight-embed/dist/*
         retention-days: 1
 
@@ -106,10 +95,10 @@ jobs:
     environment: npm
 
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v4
 
     - name: Set up Node.js
-      uses: actions/setup-node@v6
+      uses: actions/setup-node@v4
       with:
         node-version: '20'
         registry-url: 'https://registry.npmjs.org'
@@ -144,70 +133,21 @@ jobs:
       run: npm pack
 
     - name: Upload artifacts
-      uses: actions/upload-artifact@v7
+      uses: actions/upload-artifact@v4
       with:
         name: typescript-client
         path: hindsight-clients/typescript/*.tgz
         retention-days: 1
 
-  release-hindsight-all-npm:
-    runs-on: ubuntu-latest
-    environment: npm
-
-    steps:
-    - uses: actions/checkout@v6
-
-    - name: Set up Node.js
-      uses: actions/setup-node@v6
-      with:
-        node-version: '22'
-        registry-url: 'https://registry.npmjs.org'
-        cache: 'npm'
-        cache-dependency-path: package-lock.json
-
-    - name: Install dependencies
-      run: npm ci --workspace=hindsight-all-npm
-
-    - name: Build
-      run: npm run build --workspace=hindsight-all-npm
-
-    - name: Publish to npm
-      working-directory: ./hindsight-all-npm
-      run: |
-        set +e
-        OUTPUT=$(npm publish --access public 2>&1)
-        EXIT_CODE=$?
-        echo "$OUTPUT"
-        if [ $EXIT_CODE -ne 0 ]; then
-          if echo "$OUTPUT" | grep -q "cannot publish over"; then
-            echo "Package version already published, skipping..."
-            exit 0
-          fi
-          exit $EXIT_CODE
-        fi
-      env:
-        NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
-
-    - name: Pack for GitHub release
-      working-directory: ./hindsight-all-npm
-      run: npm pack
-
-    - name: Upload artifacts
-      uses: actions/upload-artifact@v7
-      with:
-        name: hindsight-all-npm
-        path: hindsight-all-npm/*.tgz
-        retention-days: 1
-
   release-control-plane:
     runs-on: ubuntu-latest
     environment: npm
 
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v4
 
     - name: Set up Node.js
-      uses: actions/setup-node@v6
+      uses: actions/setup-node@v4
       with:
         node-version: '20'
         registry-url: 'https://registry.npmjs.org'
@@ -230,14 +170,11 @@ jobs:
     - name: Build
       run: npm run build --workspace=hindsight-control-plane
 
-    - name: Verify standalone build
-      run: test -f hindsight-control-plane/standalone/server.js || (echo 'standalone/server.js missing - build failed' && exit 1)
-
     - name: Publish to npm
       working-directory: ./hindsight-control-plane
       run: |
         set +e
-        OUTPUT=$(npm publish --access public --ignore-scripts 2>&1)
+        OUTPUT=$(npm publish --access public 2>&1)
         EXIT_CODE=$?
         echo "$OUTPUT"
         if [ $EXIT_CODE -ne 0 ]; then
@@ -255,7 +192,7 @@ jobs:
       run: npm pack
 
     - name: Upload artifacts
-      uses: actions/upload-artifact@v7
+      uses: actions/upload-artifact@v4
       with:
         name: control-plane
         path: hindsight-control-plane/*.tgz
@@ -266,7 +203,7 @@ jobs:
     strategy:
       matrix:
         include:
-          - os: ubuntu-22.04
+          - os: ubuntu-latest
             target: x86_64-unknown-linux-gnu
             artifact_name: hindsight
             asset_name: hindsight-linux-amd64
@@ -278,13 +215,9 @@ jobs:
             target: aarch64-apple-darwin
             artifact_name: hindsight
             asset_name: hindsight-darwin-arm64
-          - os: ubuntu-22.04-arm
-            target: aarch64-unknown-linux-gnu
-            artifact_name: hindsight
-            asset_name: hindsight-linux-arm64
 
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v4
 
     - name: Install Rust
       uses: dtolnay/rust-toolchain@stable
@@ -302,49 +235,29 @@ jobs:
         chmod +x artifacts/${{ matrix.asset_name }}
 
     - name: Upload artifacts
-      uses: actions/upload-artifact@v7
+      uses: actions/upload-artifact@v4
       with:
         name: rust-cli-${{ matrix.asset_name }}
         path: artifacts/${{ matrix.asset_name }}
         retention-days: 1
 
   release-docker-images:
-    name: Release Docker (${{ matrix.image_name }}${{ matrix.tag_suffix }})
     runs-on: ubuntu-latest
     permissions:
       contents: read
       packages: write
-      id-token: write
     strategy:
       matrix:
         include:
           - target: api-only
             image_name: hindsight-api
-            tag_suffix: ""
-            build_args: ""
-          - target: api-only
-            image_name: hindsight-api
-            tag_suffix: "-slim"
-            build_args: |
-              INCLUDE_LOCAL_MODELS=false
-              PRELOAD_ML_MODELS=false
           - target: cp-only
             image_name: hindsight-control-plane
-            tag_suffix: ""
-            build_args: ""
           - target: standalone
             image_name: hindsight
-            tag_suffix: ""
-            build_args: ""
-          - target: standalone
-            image_name: hindsight
-            tag_suffix: "-slim"
-            build_args: |
-              INCLUDE_LOCAL_MODELS=false
-              PRELOAD_ML_MODELS=false
 
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v4
 
     - name: Free Disk Space
       uses: jlumbroso/free-disk-space@main
@@ -358,13 +271,13 @@ jobs:
         swap-storage: true
 
     - name: Set up QEMU
-      uses: docker/setup-qemu-action@v4
+      uses: docker/setup-qemu-action@v3
 
     - name: Set up Docker Buildx
-      uses: docker/setup-buildx-action@v4
+      uses: docker/setup-buildx-action@v3
 
     - name: Log in to GitHub Container Registry
-      uses: docker/login-action@v4
+      uses: docker/login-action@v3
       with:
         registry: ghcr.io
         username: ${{ github.actor }}
@@ -376,12 +289,9 @@ jobs:
 
     - name: Extract metadata for release tags
       id: meta
-      uses: docker/metadata-action@v6
+      uses: docker/metadata-action@v5
       with:
         images: ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}
-        flavor: |
-          latest=auto
-          suffix=${{ matrix.tag_suffix }}
         tags: |
           type=semver,pattern={{version}},value=${{ steps.get_version.outputs.VERSION }}
           type=semver,pattern={{major}}.{{minor}},value=${{ steps.get_version.outputs.VERSION }}
@@ -392,7 +302,7 @@ jobs:
     # # Step 1: Build for local testing (single platform, no push)
     # # This creates an identical image to what will be released, just for one platform
     # - name: Build image for testing
-    #   uses: docker/build-push-action@v7
+    #   uses: docker/build-push-action@v6
     #   with:
     #     context: .
     #     file: docker/standalone/Dockerfile
@@ -407,47 +317,20 @@ jobs:
     # - name: Smoke test - verify container starts
     #   env:
     #     GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
-    #   run: ./docker/test-image.sh "${{ matrix.image_name }}:test" "${{ matrix.target }}"
+    #   run: ./scripts/docker-smoke-test.sh "${{ matrix.image_name }}:test" "${{ matrix.target }}"
 
     # Build multi-platform and push to release tags
     - name: Build and push release images
-      id: build
-      uses: docker/build-push-action@v7
+      uses: docker/build-push-action@v6
       with:
         context: .
         file: docker/standalone/Dockerfile
         target: ${{ matrix.target }}
-        build-args: ${{ matrix.build_args }}
         push: true
         platforms: linux/amd64,linux/arm64
         tags: ${{ steps.meta.outputs.tags }}
         labels: ${{ steps.meta.outputs.labels }}
 
-    - name: Install cosign
-      uses: sigstore/cosign-installer@v3
-
-    - name: Sign published images
-      env:
-        TAGS: ${{ steps.meta.outputs.tags }}
-        DIGEST: ${{ steps.build.outputs.digest }}
-      run: |
-        set -euo pipefail
-        refs=()
-        while IFS= read -r tag; do
-          [[ -z "${tag}" ]] && continue
-          refs+=("${tag}@${DIGEST}")
-        done <<< "${TAGS}"
-        cosign sign --yes "${refs[@]}"
-
-    - name: Verify signature on primary tag
-      env:
-        IMAGE: ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}
-        DIGEST: ${{ steps.build.outputs.digest }}
-      run: |
-        cosign verify "${IMAGE}@${DIGEST}" \
-          --certificate-identity-regexp "^https://github\.com/${{ github.repository }}/\.github/workflows/release\.yml@.*" \
-          --certificate-oidc-issuer https://token.actions.githubusercontent.com
-
   release-helm-chart:
     runs-on: ubuntu-latest
     permissions:
@@ -455,10 +338,10 @@ jobs:
       packages: write
 
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v4
 
     - name: Install Helm
-      uses: azure/setup-helm@v5
+      uses: azure/setup-helm@v4
       with:
         version: 'latest'
 
@@ -475,7 +358,7 @@ jobs:
       run: helm push helm-packages/*.tgz oci://ghcr.io/${{ github.repository_owner }}/charts
 
     - name: Upload artifacts
-      uses: actions/upload-artifact@v7
+      uses: actions/upload-artifact@v4
       with:
         name: helm-chart
         path: helm-packages/*.tgz
@@ -483,67 +366,55 @@ jobs:
 
   create-github-release:
     runs-on: ubuntu-latest
-    needs: [release-python-packages, release-typescript-client, release-hindsight-all-npm, release-control-plane, release-rust-cli, release-docker-images, release-helm-chart]
+    needs: [release-python-packages, release-typescript-client, release-control-plane, release-rust-cli, release-docker-images, release-helm-chart]
     permissions:
       contents: write
 
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v4
 
     - name: Extract version from tag
       id: get_version
       run: echo "VERSION=${GITHUB_REF#refs/tags/v}" >> $GITHUB_OUTPUT
 
     - name: Download Python packages
-      uses: actions/download-artifact@v8
+      uses: actions/download-artifact@v4
       with:
         name: python-packages
         path: ./artifacts/python-packages
 
     - name: Download TypeScript client
-      uses: actions/download-artifact@v8
+      uses: actions/download-artifact@v4
       with:
         name: typescript-client
         path: ./artifacts/typescript-client
 
     - name: Download Control Plane
-      uses: actions/download-artifact@v8
+      uses: actions/download-artifact@v4
       with:
         name: control-plane
         path: ./artifacts/control-plane
 
-    - name: Download hindsight-embed npm wrapper
-      uses: actions/download-artifact@v8
-      with:
-        name: hindsight-all-npm
-        path: ./artifacts/hindsight-all-npm
-
     - name: Download Rust CLI (Linux)
-      uses: actions/download-artifact@v8
+      uses: actions/download-artifact@v4
       with:
         name: rust-cli-hindsight-linux-amd64
         path: ./artifacts/rust-cli-linux
 
-    - name: Download Rust CLI (Linux ARM)
-      uses: actions/download-artifact@v8
-      with:
-        name: rust-cli-hindsight-linux-arm64
-        path: ./artifacts/rust-cli-linux-arm64
-
     - name: Download Rust CLI (macOS Intel)
-      uses: actions/download-artifact@v8
+      uses: actions/download-artifact@v4
       with:
         name: rust-cli-hindsight-darwin-amd64
         path: ./artifacts/rust-cli-darwin-amd64
 
     - name: Download Rust CLI (macOS ARM)
-      uses: actions/download-artifact@v8
+      uses: actions/download-artifact@v4
       with:
         name: rust-cli-hindsight-darwin-arm64
         path: ./artifacts/rust-cli-darwin-arm64
 
     - name: Download Helm chart
-      uses: actions/download-artifact@v8
+      uses: actions/download-artifact@v4
       with:
         name: helm-chart
         path: ./artifacts/helm-chart
@@ -553,20 +424,16 @@ jobs:
         mkdir -p release-assets
         # Python packages
         cp artifacts/python-packages/hindsight-clients/python/dist/* release-assets/ || true
-        cp artifacts/python-packages/hindsight-api-slim/dist/* release-assets/ || true
         cp artifacts/python-packages/hindsight-api/dist/* release-assets/ || true
-        cp artifacts/python-packages/hindsight-all/dist/* release-assets/ || true
-        cp artifacts/python-packages/hindsight-all-slim/dist/* release-assets/ || true
+        cp artifacts/python-packages/hindsight/dist/* release-assets/ || true
+        cp artifacts/python-packages/hindsight-integrations/litellm/dist/* release-assets/ || true
         cp artifacts/python-packages/hindsight-embed/dist/* release-assets/ || true
         # TypeScript client
         cp artifacts/typescript-client/*.tgz release-assets/ || true
-        # hindsight-embed npm wrapper
-        cp artifacts/hindsight-all-npm/*.tgz release-assets/ || true
         # Control Plane
         cp artifacts/control-plane/*.tgz release-assets/ || true
         # Rust CLI binaries
         cp artifacts/rust-cli-linux/hindsight-linux-amd64 release-assets/ || true
-        cp artifacts/rust-cli-linux-arm64/hindsight-linux-arm64 release-assets/ || true
         cp artifacts/rust-cli-darwin-amd64/hindsight-darwin-amd64 release-assets/ || true
         cp artifacts/rust-cli-darwin-arm64/hindsight-darwin-arm64 release-assets/ || true
         # Helm chart
@@ -574,7 +441,7 @@ jobs:
         ls -la release-assets/
 
     - name: Create GitHub Release
-      uses: softprops/action-gh-release@v3
+      uses: softprops/action-gh-release@v2
       with:
         files: release-assets/*
         generate_release_notes: true

.github/workflows/sign-images.yml

@@ -1,71 +0,0 @@
-name: Sign published images
-
-on:
-  workflow_dispatch:
-    inputs:
-      version:
-        description: 'Version to sign (without leading v, e.g. 0.6.0)'
-        required: true
-        type: string
-        default: '0.6.0'
-
-permissions:
-  contents: read
-  packages: write
-  id-token: write
-
-jobs:
-  sign:
-    name: Sign ${{ matrix.image }}:${{ inputs.version }}${{ matrix.suffix }}
-    runs-on: ubuntu-latest
-    strategy:
-      fail-fast: false
-      matrix:
-        include:
-          - { image: hindsight-api, suffix: '' }
-          - { image: hindsight-api, suffix: '-slim' }
-          - { image: hindsight-control-plane, suffix: '' }
-          - { image: hindsight, suffix: '' }
-          - { image: hindsight, suffix: '-slim' }
-
-    steps:
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v4
-
-      - name: Log in to GHCR
-        uses: docker/login-action@v4
-        with:
-          registry: ghcr.io
-          username: ${{ github.actor }}
-          password: ${{ secrets.GITHUB_TOKEN }}
-
-      - name: Install cosign
-        uses: sigstore/cosign-installer@v3
-
-      - name: Resolve image digest
-        id: resolve
-        env:
-          IMAGE: ghcr.io/${{ github.repository_owner }}/${{ matrix.image }}
-          TAG: ${{ inputs.version }}${{ matrix.suffix }}
-        run: |
-          set -euo pipefail
-          DIGEST=$(docker buildx imagetools inspect "${IMAGE}:${TAG}" --format '{{json .Manifest.Digest}}' | tr -d '"')
-          if [[ -z "${DIGEST}" || "${DIGEST}" != sha256:* ]]; then
-            echo "Failed to resolve digest for ${IMAGE}:${TAG} (got: ${DIGEST})" >&2
-            exit 1
-          fi
-          echo "Resolved ${IMAGE}:${TAG} -> ${DIGEST}"
-          echo "ref=${IMAGE}@${DIGEST}" >> "$GITHUB_OUTPUT"
-
-      - name: Sign image
-        env:
-          REF: ${{ steps.resolve.outputs.ref }}
-        run: cosign sign --yes "${REF}"
-
-      - name: Verify signature
-        env:
-          REF: ${{ steps.resolve.outputs.ref }}
-        run: |
-          cosign verify "${REF}" \
-            --certificate-identity-regexp "^https://github\.com/${{ github.repository }}/\.github/workflows/sign-images\.yml@.*" \
-            --certificate-oidc-issuer https://token.actions.githubusercontent.com

.github/workflows/windows-smoke.yml

@@ -1,115 +0,0 @@
-name: Windows Smoke Test
-
-# Daily smoke test that installs the API on Windows and runs the Python client
-# integration tests against a live server. Windows is only exercised by the
-# hindsight-embed jobs in test.yml on PRs; this catches Windows-specific
-# regressions in the API server + client path (e.g. process spawning, console
-# subsystem / ConPTY behaviour, see #1885) that the Linux client jobs miss.
-on:
-  schedule:
-    # 06:00 UTC daily.
-    - cron: "0 6 * * *"
-  workflow_dispatch:
-
-permissions:
-  contents: read
-
-jobs:
-  windows-client-smoke:
-    # Don't run on forks: the job needs the org's Vertex AI credentials.
-    if: github.repository == 'vectorize-io/hindsight'
-    runs-on: windows-latest
-    timeout-minutes: 45
-    env:
-      HINDSIGHT_API_LLM_PROVIDER: vertexai
-      HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: ${{ github.workspace }}/gcp-credentials.json
-      HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
-      HINDSIGHT_API_URL: http://localhost:8888
-      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      # Force UTF-8 I/O so the API/CLI's ✓/box-drawing output doesn't crash the
-      # default Windows cp1252 codec (matches test-embed-windows in test.yml).
-      PYTHONIOENCODING: utf-8
-      PYTHONUTF8: "1"
-
-    steps:
-      - uses: actions/checkout@v6
-
-      - name: Setup GCP credentials
-        shell: bash
-        run: |
-          printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > gcp-credentials.json
-          PROJECT_ID=$(jq -r '.project_id' gcp-credentials.json)
-          echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> "$GITHUB_ENV"
-
-      - name: Install uv
-        uses: astral-sh/setup-uv@v7
-        with:
-          enable-cache: true
-          prune-cache: false
-
-      - name: Set up Python
-        uses: actions/setup-python@v6
-        with:
-          python-version-file: ".python-version"
-
-      - name: Install API dependencies (all extras - local-ml + embedded pg0)
-        working-directory: ./hindsight-api-slim
-        run: uv sync --frozen --all-extras --index-strategy unsafe-best-match
-
-      - name: Install Python client test dependencies
-        working-directory: ./hindsight-clients/python
-        run: uv sync --frozen --extra test --index-strategy unsafe-best-match
-
-      - name: Cache HuggingFace models
-        uses: actions/cache@v5
-        with:
-          path: ~/.cache/huggingface
-          key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
-          restore-keys: |
-            ${{ runner.os }}-huggingface-
-
-      # `uv run` re-syncs the project env to its default (no-extras) state before
-      # running, which drops sentence-transformers / pg0. Pass --all-extras on
-      # every `uv run` so the local-ml + embedded-db deps stay installed (this is
-      # the same reason hindsight-embed launches the daemon with `--extra all`).
-      - name: Pre-download models
-        working-directory: ./hindsight-api-slim
-        run: |
-          uv run --all-extras python -c "from sentence_transformers import SentenceTransformer, CrossEncoder; SentenceTransformer('BAAI/bge-small-en-v1.5'); CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2'); print('Models downloaded')"
-
-      # Start the server and run the client tests in a SINGLE step. On Windows
-      # runners a process backgrounded with `&` in one step is not reliably kept
-      # alive for later steps (unlike Linux, where it reparents to init), so the
-      # server must live in the same shell that runs pytest.
-      - name: Start API server and run Python client tests
-        shell: bash
-        run: |
-          # Config is read straight from the environment (job-level env + the
-          # PROJECT_ID exported to GITHUB_ENV above), so no .env file is needed.
-          # Embedded pg0 is the default when HINDSIGHT_API_DATABASE_URL is unset.
-          ( cd hindsight-api-slim && uv run --all-extras hindsight-api --port 8888 ) > "$RUNNER_TEMP/api-server.log" 2>&1 &
-          server_pid=$!
-          echo "Waiting for API server to be ready (pid $server_pid)..."
-          # pg0 unpacks Postgres + runs initdb on first boot, which is slow on a
-          # cold Windows runner — give it a generous budget before failing.
-          ready=false
-          for i in $(seq 1 300); do
-            if curl -sf http://localhost:8888/health > /dev/null 2>&1; then
-              echo "API server is ready after ${i}s"
-              ready=true
-              break
-            fi
-            sleep 1
-          done
-          if [ "$ready" != true ]; then
-            echo "API server failed to start after 300s"
-            cat "$RUNNER_TEMP/api-server.log"
-            exit 1
-          fi
-
-          cd hindsight-clients/python && uv run --extra test pytest tests -v
-
-      - name: Show API server logs
-        if: always()
-        shell: bash
-        run: cat "$RUNNER_TEMP/api-server.log" || echo "No API server log found"

.gitignore

@@ -6,7 +6,6 @@ dist/
 wheels/
 *.egg-info
 .mcp.json
-.playwright-mcp/
 .osgrep
 # Virtual environments
 .venv
@@ -16,8 +15,6 @@ node_modules/
 
 # Environment variables and local config
 .env
-.env.bak*
-.env.*.bak
 docker-compose.yml
 docker-compose.override.yml
 
@@ -30,10 +27,6 @@ docker-compose.override.yml
 # NLTK data (will be downloaded automatically)
 nltk_data/
 
-# Monitoring stack (Prometheus/Grafana binaries and data)
-.monitoring/
-.pgbouncer/
-
 # Large benchmark datasets (will be downloaded automatically)
 **/longmemeval_s_cleaned.json
 
@@ -48,19 +41,9 @@ hindsight-docs/static/llms-full.txt
 
 hindsight-dev/benchmarks/locomo/results/
 hindsight-dev/benchmarks/longmemeval/results/
-hindsight-dev/benchmarks/consolidation/results/
-hindsight-dev/benchmarks/perf/results/
-benchmarks/results/
 hindsight-cli/target
 hindsight-clients/rust/target
-.claude/*
-!.claude/skills/
+.claude
 whats-next.md
 TASK.md
-# Parked / draft integrations that aren't ready to ship
-hindsight-integrations/_drafts/
-# Changelog is now tracked in hindsight-docs/src/pages/changelog.md
-# CHANGELOG.md
-
-blog-post*
-.worktrees/
+CHANGELOG.md

.prettierrc.json

@@ -1,7 +0,0 @@
-{
-  "semi": true,
-  "singleQuote": false,
-  "tabWidth": 2,
-  "trailingComma": "es5",
-  "printWidth": 100
-}

AGENTS.md

@@ -1,3 +1,153 @@
 # AGENTS.md
 
-See [CLAUDE.md](./CLAUDE.md) for project documentation and coding conventions.
+This document captures architectural decisions and coding conventions for the Hindsight project.
+
+## Documentation
+
+- **Main documentation**: [hindsight-docs/docs/developer/](./hindsight-docs/docs/developer/)
+- **Use case patterns**: [hindsight-docs/docs/cookbook/](./hindsight-docs/docs/cookbook/)
+- **API reference**: Auto-generated from OpenAPI spec
+
+## Project Structure
+
+```
+hindsight/              # Python package for embedded usage
+hindsight-api/          # FastAPI server (core memory engine)
+hindsight-cli/          # Rust CLI client
+hindsight-embed/        # Embedded CLI (no server needed)
+hindsight-control-plane/ # Next.js admin UI
+hindsight-docs/         # Docusaurus documentation site
+hindsight-dev/          # Development tools and benchmarks
+hindsight-integrations/ # Framework integrations (LangChain, etc.)
+hindsight-clients/      # Generated API clients (Python, TypeScript, Rust)
+```
+
+## Core Concepts
+
+### Memory Banks
+- Each bank is an isolated memory store (like a "brain" for one user/agent)
+- Banks contain: memory units (facts), entities, documents, entity links
+- Banks have a **disposition** (personality traits) and **background** (context)
+- Bank isolation is strict - no cross-bank data leakage
+
+### Memory Types
+- **World facts**: General knowledge ("The sky is blue")
+- **Experience facts**: Personal experiences ("I visited Paris in 2023")
+- **Opinion facts**: Beliefs with confidence scores ("Paris is beautiful" - 0.9 confidence)
+
+### Operations
+- **Retain**: Store new memories (extracts facts, entities, relationships)
+- **Recall**: Retrieve memories (semantic, BM25, graph, temporal search)
+- **Reflect**: Deep analysis to form new insights/opinions
+
+## API Design Decisions
+
+### Single Bank Per Request
+- All API endpoints (`recall`, `reflect`, `retain`) operate on a single bank
+- Multi-bank queries are the **client/agent's responsibility** to orchestrate
+- This keeps the API simple and the isolation model clear
+
+### Disposition Traits (3-trait system)
+- **Skepticism** (1-5): How skeptical vs trusting when forming opinions
+- **Literalism** (1-5): How literally to interpret information
+- **Empathy** (1-5): How much to consider emotional context
+- These influence the `reflect` operation, not `recall`
+- Background info also only affects `reflect` (opinion formation)
+
+## Multi-Bank Architecture Patterns
+
+See [hindsight-docs/docs/cookbook/](./hindsight-docs/docs/cookbook/) for detailed guides:
+
+- **Per-User Memory**: One bank per user, simplest pattern
+- **Support Agent + Shared Knowledge**: User bank + shared docs bank, client orchestrates
+
+## Developer Guide
+
+### Running the API Server
+
+```bash
+# From project root
+./scripts/dev/start-api.sh
+
+# With options
+./scripts/dev/start-api.sh --reload --port 8888 --log-level debug
+```
+
+### Running Tests
+
+```bash
+# API tests
+cd hindsight-api
+uv run pytest tests/
+
+# Specific test
+uv run pytest tests/test_http_api_integration.py -v
+```
+
+### Generating OpenAPI Spec
+
+After changing API endpoints, regenerate the OpenAPI spec and docs:
+
+```bash
+./scripts/generate-openapi.sh
+```
+
+This will:
+1. Generate `openapi.json` at project root
+2. Copy to `hindsight-docs/openapi.json`
+3. Regenerate API reference documentation
+
+### Generating API Clients
+
+After updating the OpenAPI spec, regenerate all clients:
+
+```bash
+./scripts/generate-clients.sh
+```
+
+This generates:
+- **Rust client**: `hindsight-clients/rust/` (via progenitor in build.rs)
+- **Python client**: `hindsight-clients/python/` (via openapi-generator Docker)
+- **TypeScript client**: `hindsight-clients/typescript/` (via @hey-api/openapi-ts)
+
+Note: The maintained wrapper `hindsight_client.py` and `README.md` are preserved during regeneration.
+
+### Running the Documentation Site
+
+```bash
+./scripts/dev/start-docs.sh
+```
+
+### Running the Control Plane
+
+```bash
+./scripts/dev/start-control-plane.sh
+```
+
+## Code Style
+
+### Python (hindsight-api)
+- Use `uv` for package management
+- Async throughout (asyncpg, async FastAPI endpoints)
+- Pydantic models for request/response validation
+- No py files at project root - maintain clean directory structure
+
+### TypeScript (control-plane, clients)
+- Next.js with App Router for control plane
+- Tailwind CSS with shadcn/ui components
+
+### Rust (CLI)
+- Async with tokio
+- reqwest for HTTP client
+- progenitor for API client generation
+
+## Database
+
+- PostgreSQL with pgvector extension
+- Schema managed via Alembic migrations in `hindsight-api/alembic/`, db migrations happen during api startup, no manual commands
+- Key tables: `banks`, `memory_units`, `documents`, `entities`, `entity_links`
+
+# Branding
+## Colors
+- Primary: gradient from #0074d9 to #009296
+

CLAUDE.md

@@ -7,36 +7,31 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 Hindsight is an agent memory system that provides long-term memory for AI agents using biomimetic data structures. Memories are organized as:
 - **World facts**: General knowledge ("The sky is blue")
 - **Experience facts**: Personal experiences ("I visited Paris in 2023")
-- **Mental models**: Consolidated knowledge synthesized from facts ("User prefers functional programming patterns")
+- **Opinion facts**: Beliefs with confidence scores ("Paris is beautiful" - 0.9 confidence)
+- **Observations**: Complex mental models derived from reflection
 
 ## Development Commands
 
-### Local Development (API + UI)
-```bash
-# Start both API server and control plane UI
-./scripts/dev/start.sh
-```
-
 ### API Server (Python/FastAPI)
 ```bash
-# Start API server only (loads .env automatically)
+# Start API server (loads .env automatically)
 ./scripts/dev/start-api.sh
 
 # Run all tests (parallelized with pytest-xdist)
-cd hindsight-api-slim && uv run pytest tests/
+cd hindsight-api && uv run pytest tests/
 
 # Run specific test file
-cd hindsight-api-slim && uv run pytest tests/test_http_api_integration.py -v
+cd hindsight-api && uv run pytest tests/test_http_api_integration.py -v
 
 # Run single test function
-cd hindsight-api-slim && uv run pytest tests/test_retain.py::test_retain_simple -v
+cd hindsight-api && uv run pytest tests/test_retain.py::test_retain_simple -v
 
 # Lint and format
-cd hindsight-api-slim && uv run ruff check .
-cd hindsight-api-slim && uv run ruff format .
+cd hindsight-api && uv run ruff check .
+cd hindsight-api && uv run ruff format .
 
 # Type checking (uses ty - extremely fast type checker from Astral)
-cd hindsight-api-slim && uv run ty check hindsight_api/
+cd hindsight-api && uv run ty check hindsight_api/
 ```
 
 ### Control Plane (Next.js)
@@ -51,7 +46,6 @@ cd hindsight-control-plane && npm run dev
 ./scripts/dev/start-docs.sh
 ```
 
-
 ### Generating Clients/OpenAPI
 ```bash
 # Regenerate OpenAPI spec after API changes (REQUIRED after changing endpoints)
@@ -63,33 +57,26 @@ cd hindsight-control-plane && npm run dev
 
 ### Benchmarks
 ```bash
-# Accuracy benchmarks
 ./scripts/benchmarks/run-longmemeval.sh
 ./scripts/benchmarks/run-locomo.sh
-
-# Performance benchmarks
-./scripts/benchmarks/run-perf-test.sh                      # System perf (mock LLM + pg0)
-./scripts/benchmarks/run-perf-test.sh --scale tiny          # Quick smoke test
-./scripts/benchmarks/run-consolidation.sh
-
-# Results viewer
 ./scripts/benchmarks/start-visualizer.sh  # View results at localhost:8001
 ```
 
 ## Architecture
 
 ### Monorepo Structure
-- **hindsight-api-slim/**: Core FastAPI server with memory engine (Python, uv)
+- **hindsight-api/**: Core FastAPI server with memory engine (Python, uv)
+- **hindsight/**: Embedded Python bundle (hindsight-all package)
 - **hindsight-control-plane/**: Admin UI (Next.js, npm)
 - **hindsight-cli/**: CLI tool (Rust, cargo, uses progenitor for API client)
 - **hindsight-clients/**: Generated SDK clients (Python, TypeScript, Rust)
 - **hindsight-docs/**: Docusaurus documentation site
-- **hindsight-integrations/**: Framework integrations (LiteLLM, CrewAI, LangGraph, Pydantic AI, AG2, Claude Code, etc.)
+- **hindsight-integrations/**: Framework integrations (LiteLLM, OpenAI)
 - **hindsight-dev/**: Development tools and benchmarks
 
-### Core Engine (hindsight-api-slim/hindsight_api/engine/)
-- `memory_engine.py`: Main orchestrator for retain/recall/reflect operations
-- `llm_wrapper.py`: LLM abstraction supporting OpenAI, Anthropic, Gemini, VertexAI, Groq, MiniMax, Ollama, LM Studio, LiteLLM, Claude Code
+### Core Engine (hindsight-api/hindsight_api/engine/)
+- `memory_engine.py`: Main orchestrator (~170KB) for retain/recall/reflect operations
+- `llm_wrapper.py`: LLM abstraction supporting OpenAI, Anthropic, Gemini, Groq, Ollama, LM Studio
 - `embeddings.py`: Embedding generation (local sentence-transformers or TEI)
 - `cross_encoder.py`: Reranking (local or TEI)
 - `entity_resolver.py`: Entity extraction and normalization
@@ -102,159 +89,33 @@ cd hindsight-control-plane && npm run dev
 
 **search/**: Multi-strategy retrieval
 - `retrieval.py`: Main retrieval orchestrator
-- `graph_retrieval.py`: Graph retrieval abstract base class
-- `link_expansion_retrieval.py`: Link expansion graph retrieval
+- `graph_retrieval.py`: Entity/relationship graph traversal
+- `mpfp_retrieval.py`: Multi-Path Fact Propagation retrieval
 - `fusion.py`: Reciprocal rank fusion for combining results
 - `reranking.py`: Cross-encoder reranking
 
-### API Layer (hindsight-api-slim/hindsight_api/api/)
-- `http.py`: FastAPI HTTP routers for all REST endpoints
+### API Layer (hindsight-api/hindsight_api/api/)
+- `http.py`: FastAPI HTTP routers (~80KB) for all REST endpoints
 - `mcp.py`: Model Context Protocol server implementation
 
 Main operations:
 - **Retain**: Store memories, extracts facts/entities/relationships
 - **Recall**: Retrieve memories via 4 parallel strategies (semantic, BM25, graph, temporal) + reranking
-- **Reflect**: Disposition-aware reasoning using memories and mental models.
+- **Reflect**: Deep analysis forming new opinions/observations (disposition-aware)
 
 ### Database
-PostgreSQL with pgvector. Schema managed via Alembic migrations in `hindsight-api-slim/hindsight_api/alembic/`. Migrations run automatically on API startup.
+PostgreSQL with pgvector. Schema managed via Alembic migrations in `hindsight-api/hindsight_api/alembic/`. Migrations run automatically on API startup.
 
 Key tables: `banks`, `memory_units`, `documents`, `entities`, `entity_links`
 
-### Adding Database Migrations
-
-Hindsight runs the same Alembic tree against PostgreSQL and Oracle 23ai. Each
-migration file dispatches through `run_for_dialect`, which calls either
-`_pg_upgrade` or `_oracle_upgrade` based on the live connection. A pytest lint
-(`tests/test_migration_shape.py`) fails CI if a migration omits the dispatcher.
-
-1. **Create a new migration file** in `hindsight-api-slim/hindsight_api/alembic/versions/`:
-   - File name format: `<revision_id>_<description>.py` (e.g., `f1a2b3c4d5e6_add_new_index.py`)
-   - Use a unique hex revision ID (12 chars)
-   - Set `down_revision` to the previous migration's revision ID
-
-2. **Migration template** (the `script.py.mako` template scaffolds this; fill in the bodies):
-   ```python
-   """Description of the migration
-
-   Revision ID: f1a2b3c4d5e6
-   Revises: <previous_revision_id>
-   Create Date: YYYY-MM-DD
-   """
-   from collections.abc import Sequence
-   from alembic import context, op
-
-   from hindsight_api.alembic._dialect import run_for_dialect
-
-   revision: str = "f1a2b3c4d5e6"
-   down_revision: str | Sequence[str] | None = "<previous_revision_id>"
-   branch_labels: str | Sequence[str] | None = None
-   depends_on: str | Sequence[str] | None = None
-
-
-   def _pg_schema_prefix() -> str:
-       """Schema-qualifier for raw SQL on PG (multi-tenant search_path)."""
-       schema = context.config.get_main_option("target_schema")
-       return f'"{schema}".' if schema else ""
-
-
-   def _pg_upgrade() -> None:
-       schema = _pg_schema_prefix()
-       op.execute(f"CREATE INDEX ... ON {schema}table_name(...)")
-
-
-   def _pg_downgrade() -> None:
-       schema = _pg_schema_prefix()
-       op.execute(f"DROP INDEX IF EXISTS {schema}index_name")
-
-
-   def _oracle_upgrade() -> None:
-       # Oracle 23ai equivalent. Use op.get_bind().exec_driver_sql for forms
-       # that Alembic core does not model (vector/text indexes, partitions).
-       op.execute("CREATE INDEX ... ON table_name(...)")
-
-
-   def _oracle_downgrade() -> None:
-       op.execute("DROP INDEX IF EXISTS index_name")
-
-
-   def upgrade() -> None:
-       run_for_dialect(pg=_pg_upgrade, oracle=_oracle_upgrade)
-
-
-   def downgrade() -> None:
-       run_for_dialect(pg=_pg_downgrade, oracle=_oracle_downgrade)
-   ```
-
-   **Dialect-only migrations.** If a change genuinely doesn't apply to one
-   dialect (e.g. enabling `pg_trgm` is PG-only), omit the unused slot:
-   ```python
-   def upgrade() -> None:
-       run_for_dialect(pg=_pg_upgrade)  # oracle slot intentionally absent → no-op
-   ```
-   Make the asymmetry deliberate. Don't leave an Oracle slot empty just because
-   you didn't think about it — copy-pasting a PG migration without the Oracle
-   half is exactly how schemas drift.
-
-3. **Run migrations locally**:
-   ```bash
-   # Set database URL and run migrations for the base schema plus all tenants
-   uv run hindsight-admin run-db-migration
-
-   # Run on a specific tenant schema
-   uv run hindsight-admin run-db-migration --schema tenant_xyz
-   ```
-
 ## Key Conventions
 
 ### Code Quality
-
-**Before writing code, read `.claude/skills/code-review/SKILL.md`** for the full coding standards (Python style, type safety, TypeScript style, general principles).
-
 **Always run the lint script after making Python or TypeScript/Node changes:**
 ```bash
 ./scripts/hooks/lint.sh
 ```
-
-Dead-code detection runs in CI (the `check-unused-code` job) at two levels:
-- **Blocking:** unused imports (ruff `F401`) and variables (`F841`) — `lint.sh` auto-removes
-  them and `verify-generated-files` fails on any leftover diff; and **knip** for orphaned
-  control-plane files / unused (or unlisted) `package.json` dependencies.
-- **Advisory:** whole unused Python functions (vulture) and unused control-plane *exports*
-  (the shadcn/ui surface is kept on purpose) — surfaced, not gated.
-
-Run both locally with:
-```bash
-./scripts/hooks/check-unused.sh
-```
-
-**After completing any implementation work, run `/code-review`** to verify your changes against project standards (missing tests, dead code, type safety, etc.). Fix any "must fix" issues before considering the task done.
-
-**MANDATORY: Run `/code-review` before pushing code or creating a pull request.** Do not push or create a PR until all "must fix" issues are resolved.
-
-### Testing
-
-Most tests are deterministic (MockLLM, pure functions) — assert directly.
-
-**Tests that verify LLM behaviour use a real LLM + an LLM-as-judge.** When the thing under test is *how the model interprets a prompt* (classification, attribution, dimension preservation, instruction-following), MockLLM can't simulate it and exact string/enum asserts flake across providers and runs. Use this pattern instead:
-
-1. Mark the test module `pytestmark = pytest.mark.hs_llm_core` (single-provider; CI runs it in the core-LLM job). Use `hs_llm_mat` only for provider-matrix acceptance tests.
-2. Call the real pipeline (`LLMConfig.from_env()`, `_get_raw_config()`), e.g. `extract_facts_from_text(...)`.
-3. Assert with the judge, not string matching:
-   ```python
-   from tests.llm_judge import assert_meets_criteria
-   facts_summary = "\n".join(f"- [{f.fact_type}] {f.fact}" for f in facts)
-   await assert_meets_criteria(
-       response=facts_summary,
-       criteria="The first-person user statements are classified 'world' and attributed to the user, not the agent.",
-       context="What the input said and who was speaking.",
-   )
-   ```
-
-Rules of thumb:
-- **Judge anything non-deterministic** — including `fact_type` classification and speaker attribution. Do NOT hard-assert `fact_type == "..."`; pass a `[fact_type] fact` summary to the judge instead. Structural facts that ARE deterministic (counts, presence of a field, that a substring was injected into a prompt) stay as direct asserts in fast unit tests.
-- **Split the test surface**: cover the deterministic mechanics (prompt assembly, suppression logic) with fast non-LLM unit tests, and the model-following behaviour with one `hs_llm_core` judge test. (Example pair: `test_narrator_resolution.py` + `test_narrator_context_override.py`.)
-- The judge model is independent of the test provider (defaults to Gemini); never judge with the same call you're testing.
+This runs the same checks as the pre-commit hook (Ruff for Python, ESLint/Prettier for TypeScript).
 
 ### Memory Banks
 - Each bank is an isolated memory store (like a "brain" for one user/agent)
@@ -267,131 +128,59 @@ Rules of thumb:
 - Multi-bank queries are client responsibility to orchestrate
 - Disposition traits only affect reflect, not recall
 
-### Control Plane API Routes
+### Python Style
+- Python 3.11+, type hints required
+- Async throughout (asyncpg, async FastAPI)
+- Pydantic models for request/response
+- Ruff for linting (line-length 120)
+- No Python files at project root - maintain clean directory structure
 
-When adding or modifying parameters in the dataplane API (hindsight-api), you must also update the control plane routes that proxy to it:
-
-1. **API Routes** (`hindsight-control-plane/src/app/api/`):
-   - `recall/route.ts` - proxies to `/v1/default/banks/{bank_id}/memories/recall`
-   - `reflect/route.ts` - proxies to `/v1/default/banks/{bank_id}/reflect`
-   - `memories/retain/route.ts` - proxies to `/v1/default/banks/{bank_id}/memories/retain`
-   - Other routes follow the same pattern
-
-2. **Client types** (`hindsight-control-plane/src/lib/api.ts`):
-   - Update the TypeScript type definitions for `recall()`, `reflect()`, `retain()` etc.
-
-3. **Checklist when adding new API parameters**:
-   - Add parameter extraction in the route handler (destructure from `body`)
-   - Pass the parameter to the SDK call
-   - Update the client type definition in `lib/api.ts`
-   - Update any UI components that need to use the new parameter
-
-### Adding New Integrations
-
-Every new integration in `hindsight-integrations/` must satisfy all of the following before it can be merged:
-
-1. **Tests are required** — tests must simulate or exercise the external system (mock the framework's interfaces and verify the integration actually calls Hindsight correctly). Pure unit tests of helper functions are not sufficient.
-2. **CI job** — add a test job in `.github/workflows/test.yml` following the existing pattern (e.g., `test-crewai-integration`). The job must build, install deps, and run `uv run pytest tests -v`. Also add the integration to `detect-changes` outputs so it only runs when its files change.
-3. **Release process** — add the integration name to the `VALID_INTEGRATIONS` array in `scripts/release-integration.sh` so it can be released via the standard release workflow.
-4. **Follow project code standards** — Python style, type safety, no raw dicts for structured data, no multi-item tuple returns (see `.claude/skills/code-review/SKILL.md`).
-
-If any of these are missing, the integration is incomplete and must not be pushed or merged.
-
-### Changelogs
-
-Never add "Unreleased" entries to changelogs (e.g. `hindsight-docs/src/pages/changelog/**`). Changelog entries are written by the release script (`./scripts/release-integration.sh`) when a version is actually cut. If a bug fix or feature needs documenting before release, describe it in the PR/commit — the release tooling will surface it in the published changelog section.
+### TypeScript Style
+- Next.js App Router for control plane
+- Tailwind CSS with shadcn/ui components
 
 ### Adding New API Configuration Flags
 
-Configuration follows a hierarchical system: **Global (env vars) → Tenant (via extension) → Bank (database)**.
+When adding a new environment variable configuration:
 
-Fields must be categorized as either **hierarchical** (can be overridden per-tenant/bank) or **static** (server-level only).
-
-#### Adding a New Configuration Field
-
-1. **config.py** (`hindsight-api-slim/hindsight_api/config.py`):
-   - Add `ENV_*` constant for the environment variable name (e.g., `ENV_MY_SETTING = "HINDSIGHT_API_MY_SETTING"`)
+1. **config.py** (`hindsight-api/hindsight_api/config.py`):
+   - Add `ENV_*` constant for the environment variable name
    - Add `DEFAULT_*` constant for the default value
-   - Add field to `HindsightConfig` dataclass with type annotation
-   - **Mark as configurable** by adding to `_CONFIGURABLE_FIELDS` set if the field should be overridable per-tenant/bank via API
+   - Add field to `HindsightConfig` dataclass
    - Add initialization in `from_env()` method
 
-   ```python
-   # Configurable field (can be overridden per-tenant/bank via API)
-   _CONFIGURABLE_FIELDS = {
-       ...,
-       "my_setting",  # Add here for configurable
-   }
+2. **main.py** (`hindsight-api/hindsight_api/main.py`):
+   - Add field to the manual `HindsightConfig()` constructor call (search for "CLI override")
 
-   # Static field - just don't add to _CONFIGURABLE_FIELDS
-   ```
-
-2. **main.py** (`hindsight-api-slim/hindsight_api/main.py`):
-   - No change is needed for ordinary environment-backed config fields. The CLI starts from `_get_raw_config()`,
-     so new `HindsightConfig` fields are carried through automatically.
-   - If the new field should be overridable by a CLI flag, add the argparse option in `_parse_cli_args()` and include
-     that field in the `dataclasses.replace(config, ...)` call near the "CLI override" comment.
-
-3. **Use hierarchical config in MemoryEngine**:
-   ```python
-   # Config is resolved automatically per bank via ConfigResolver
-   config_dict = await self._config_resolver.get_bank_config(bank_id, context)
-   value = config_dict["my_setting"]
-   ```
-
-4. **Use static config** (non-hierarchical):
+3. **Use the config** in code:
    ```python
    from ...config import get_config
    config = get_config()
-   value = config.my_static_field
+   value = config.your_new_field
    ```
 
-5. **Documentation** (`hindsight-docs/docs/developer/configuration.md`):
+4. **Documentation** (`hindsight-docs/docs/developer/configuration.md`):
    - Add to appropriate section table with Variable, Description, Default
-   - Mark if it's hierarchical (can be overridden per-bank)
-
-6. **Env template** (`.env.example`):
-   - Add the variable to the appropriate section, commented if optional, with a
-     short inline comment describing it (mirror the documentation entry).
-   - This file is the single source of truth for the env template:
-     `scripts/dev/setup.sh` copies it to `.env`, and `hindsight-embed` ships a
-     bundled copy (`hindsight-embed/hindsight_embed/env.example`) that seeds
-     embed/profile configs. After editing `.env.example`, re-copy it to the
-     embed package (`cp .env.example hindsight-embed/hindsight_embed/env.example`)
-     or the `test_bundled_template_matches_repo_root` sync test will fail.
-
-#### Hierarchical vs Static Guidelines
-
-**Hierarchical** (per-bank overridable):
-- LLM settings (provider, model, API key, base URL)
-- Operation-specific settings (retain mode, chunk size, etc.)
-- Feature flags that vary by customer/bank
-
-**Static** (server-level only):
-- Infrastructure settings (database URL, port, host)
-- Global limits (max concurrent operations)
-- System-wide feature flags
 
 ## Environment Setup
 
 ```bash
 cp .env.example .env
-# Edit .env with the LLM provider/model and credentials for your setup
+# Edit .env with LLM API key
 
 # Python deps
-uv sync --directory hindsight-api-slim/
+uv sync --directory hindsight-api/
 
 # Node deps (uses npm workspaces)
 npm install
 ```
 
-Common LLM settings:
-- `HINDSIGHT_API_LLM_PROVIDER`: openai, anthropic, gemini, groq, minimax, ollama, lmstudio
-- `HINDSIGHT_API_LLM_API_KEY`: API key for providers that require one
-- `HINDSIGHT_API_LLM_MODEL`: Model name (defaults are provider-specific)
+Required env vars:
+- `HINDSIGHT_API_LLM_PROVIDER`: openai, anthropic, gemini, groq, ollama, lmstudio
+- `HINDSIGHT_API_LLM_API_KEY`: Your API key
+- `HINDSIGHT_API_LLM_MODEL`: Model name (e.g., o3-mini, claude-sonnet-4-20250514)
 
 Optional (uses local models by default):
 - `HINDSIGHT_API_EMBEDDINGS_PROVIDER`: local (default) or tei
 - `HINDSIGHT_API_RERANKER_PROVIDER`: local (default) or tei
 - `HINDSIGHT_API_DATABASE_URL`: External PostgreSQL (uses embedded pg0 by default)
-- `HINDSIGHT_API_ENABLE_BANK_CONFIG_API`: Enable per-bank config API (default: true)

CONTRIBUTING.md

@@ -9,36 +9,13 @@ Thanks for your interest in contributing to Hindsight!
    git clone [email protected]:vectorize-io/hindsight.git
    cd hindsight
    ```
-
-2. Bootstrap your dev environment in one shot:
-   ```bash
-   ./scripts/dev/setup.sh
-   ```
-   This is idempotent (safe to re-run) and gets you ready to develop, including
-   offline. It:
-   - installs the required toolchains if missing (uv/Python, Node/npm, Rust/cargo),
-   - creates `.env` from `.env.example` (remember to add your LLM API key),
-   - configures git hooks,
-   - installs all Python and Node workspace dependencies,
-   - pre-downloads the local ML models + tokenizer so the API runs offline,
-   - builds the TypeScript SDK and the Rust CLI.
-
-   Useful flags: `--skip-build` (deps only), `--skip-models` (skip ML model
-   download), `--with-docs` (also build the docs site), `--force` (rebuild
-   artifacts). Docker image builds are out of scope. Run
-   `./scripts/dev/setup.sh --help` for details.
-
-### Manual setup
-
-If you'd rather set things up by hand instead of running the script above:
-
-1. Set up your environment:
+2. Set up your environment:
    ```bash
    cp .env.example .env
    ```
    Edit the .env to add LLM API key and config as required
 
-2. Install dependencies:
+3. Install dependencies:
    ```bash
    # Python dependencies
    uv sync --directory hindsight-api/
@@ -116,34 +93,6 @@ uv run ty check hindsight_api  # Type check
 3. Run tests to ensure nothing breaks
 4. Submit a PR with a clear description of changes
 
-## Release Process
-
-The project uses `scripts/release.sh` for creating releases. This script automates the entire release workflow:
-
-1. Bumps version in all components (API, clients, CLI, control plane, Helm)
-2. **Regenerates OpenAPI spec and client SDKs** (Python, TypeScript, Rust)
-3. Updates documentation versioning
-4. Creates a commit and git tag
-5. Pushes to GitHub (triggers CI/CD to publish packages)
-
-### Usage
-
-```bash
-./scripts/release.sh <version>
-```
-
-**Example:**
-```bash
-./scripts/release.sh 0.5.0
-```
-
-### Important for Developers
-
-- During development, version bumps in `__init__.py` do NOT require client regeneration
-- Clients are only regenerated during releases
-- Do not manually run `./scripts/generate-clients.sh` unless testing generation changes
-- Client version comments will reflect the API version from the latest release
-
 ## Reporting Issues
 
 Open an issue on GitHub with:

README.md

@@ -1,95 +1,92 @@
 <div align="center">
 
-![Hindsight Banner](./hindsight-docs/static/img/hindsight-github-banner.png)
+![Hindsight Banner](./hindsight-docs/static/img/banner.svg)
 
-[Documentation](https://hindsight.vectorize.io) • [Paper](https://arxiv.org/abs/2512.12818) • [Cookbook](https://hindsight.vectorize.io/cookbook) • [Hindsight Cloud](https://ui.hindsight.vectorize.io/signup)
+[Documentation](https://hindsight.vectorize.io) • [Paper](https://arxiv.org/abs/2512.12818) • [Cookbook](https://hindsight.vectorize.io/cookbook) • [Hindsight Cloud](https://vectorize.io/hindsight/cloud)
 
 [![CI](https://github.com/vectorize-io/hindsight/actions/workflows/release.yml/badge.svg)](https://github.com/vectorize-io/hindsight/actions/workflows/release.yml)
-[![Slack Community](https://img.shields.io/badge/Slack-Join%20Community-4A154B?logo=slack)](https://join.slack.com/t/hindsight-space/shared_invite/zt-3nhbm4w29-LeSJ5Ixi6j8PdiYOCPlOgg)
+[![Slack Community](https://img.shields.io/badge/Slack-Join%20Community-4A154B?logo=slack)](https://join.slack.com/t/hindsight-space/shared_invite/zt-3klo21kua-VUCC_zHP5rIcXFB1_5yw6A)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 ![PyPI - Downloads](https://img.shields.io/pypi/dm/hindsight-api?label=PyPI)
 ![NPM Downloads](https://img.shields.io/npm/dm/%40vectorize-io%2Fhindsight-client?logoColor=orange&label=NPM&color=blue&link=https%3A%2F%2Fwww.npmjs.com%2Fpackage%2F%40vectorize-io%2Fhindsight-client)
-<br/>
 
-<a href="https://trendshift.io/repositories/15603" target="_blank"><img src="https://trendshift.io/api/badge/repositories/15603" alt="vectorize-io%2Fhindsight | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
+
 </div>
 
 ---
 
 ## What is Hindsight?
 
-Hindsight™ is an agent memory system built to create smarter agents that learn over time. Most agent memory systems focus on recalling conversation history. Hindsight is focused on making agents that learn, not just remember.
+Hindsight™ is an agent memory system built to create smarter agents that learn over time. It eliminates the shortcomings of alternative techniques such as RAG and knowledge graph and delivers state-of-the-art performance on long term memory tasks.
 
+Hindsight addresses common challenges that have frustrated AI engineers building agents to automate tasks and assist users with conversational interfaces. Many of these challenges stem directly from a lack of memory.
 
-<video src="https://github.com/user-attachments/assets/923b798d-3581-4897-bb62-9cfa5a931682" controls></video>
+- **Inconsistency:** Agents complete tasks successfully one time, then fail when asked to complete the same task again. Memory gives the agent a mechanism to remember what worked and what didn't and to use that information to reduce errors and improve consistency.
+- **Hallucinations:** Long term memory can be seeded with external knowledge to ground agent behavior in reliable sources to augment training data.
+- **Cognitive Overload:** As workflows get complex, retrievals, tool calls, user messages and agent responses can grow to fill the context window leading to context rot. Short term memory optimization allows agents to reduce tokens and focus context by removing irrelevant details.
 
-It eliminates the shortcomings of alternative techniques such as RAG and knowledge graph and delivers state-of-the-art performance on long term memory tasks.
+## How is Hindsight Different From Other Memory Systems?
+
+![Overview](./hindsight-docs/static/img/hindsight-overview.webp)
+
+Most agent memory implementation rely on basic vector search or sometimes use a knowledge graph. Hindsight uses biomimetic data structures to organize agent memories in a way that is more like how human memory works:
+
+- **World:** Facts about the world ("The stove gets hot")
+- **Experiences:** Agent's own experiences ("I touched the stove and it really hurt")
+- **Opinion:** Beliefs with confidence scores ("I shouldn't touch the stove again" - .99 confidence)
+- **Observation:** Complex mental models derived by reflecting on facts and experiences ("Curling irons, ovens, and fire are also hot. I shouldn't touch those either.")
+
+Memories in Hindsight are stored in banks (i.e. memory banks). When memories are added to Hindsight, they are pushed into either the world facts or experiences memory pathway. They are then represented as a combination of entities, relationships, and time series with sparse/dense vector representations to aid in later recall.
+
+Hindsight provides three simple methods to interact with the system:
+
+- **Retain:** Provide information to Hindsight that you want it to remember
+- **Recall:** Retrieve memories from Hindsight
+- **Reflect:** Reflect on memories and experiences to generate new observations and insights from existing memories.
+
+### Agent Memory That Learns
+
+A key goal of Hindsight is to build agent memory that enables agents to learn and improve over time. This is the role of the `reflect` operation which provides the agent to form broader opinions and observations over time.
+
+For example, imagine a product support agent that is helping a user troubleshoot a problem. It uses a `search-documentation` tool it found on an MCP server. Later in the conversation, the agent discovers that the documentation returned from the tool wasn't for the product the user was asking about. The agent now has an experience in its memory bank. And just like humans, we want that agent to learn from its experience.
+
+As the agent gains more experiences, `reflect` allows the agent to form observations about what worked, what didn't, and what to do differently the next time it encounters a similar task.
+
+---
 
 ## Memory Performance & Accuracy
 
-Hindsight is the most accurate agent memory system ever tested according to benchmark performance. It has achieved state-of-the-art performance on the LongMemEval benchmark, widely used to assess memory system performance across a variety of conversational AI scenarios. The current reported performance of Hindsight and other agent memory solutions as of January 2026 is shown here:
+Hindsight has achieved state-of-the-art performance on the LongMemEval benchmark, widely used to assess memory system performance across a variety of conversational
+AI scenarios. The current reported performance of Hindsight and other agent memory solutions as of December 2025 is shown here:
 
-![Overview](./hindsight-docs/static/img/hindsight-benchmarks.png)
+![Overview](./hindsight-docs/static/img/hindsight-bench.jpg)
 
-The benchmark performance data for Hindsight has been independently reproduced by research collaborators at the Virginia Tech [Sanghani Center for Artificial Intelligence and Data Analytics](https://sanghani.cs.vt.edu/) and The Washington Post. Other scores are self-reported by software vendors.
+The benchmark performance data for Hindsight and GPT-4o (full context) have been reproduced by research collaborators at the Virginia Tech [Sanghani Center for Artificial Intelligence and Data Analytics](https://sanghani.cs.vt.edu/) and The Washington Post. Other scores are self-reported by software vendors.
 
-Hindsight is being used in production at Fortune 500 enterprises and by a growing number of AI startups. 
-
-## Adding Hindsight to Your AI Agents
-
-The easiest way to use Hindsight with an existing agent is with the LLM Wrapper. You can add memory to your agent with 2 lines of code. That will swap your current LLM client out with the Hindsight wrapper. After that, memories will be stored and retrieved automatically as you make LLM calls.
-
-If you need more control over how and when your agent stores and recalls memories, there's also a simple API you can integrate with using the SDKs or directly via HTTP.
-
-![Hindsight Banner](./hindsight-docs/static/img/migration-code.png)
-
----
-
-> 🤖 **Using a coding agent?** Install the Hindsight documentation skill for instant access to docs while you code:
-> ```bash
-> npx skills add https://github.com/vectorize-io/hindsight --skill hindsight-docs
-> ```
-> Works with Claude Code, Cursor, and other AI coding assistants.
-
----
+A thorough examination of the techniques implemented in Hindsight and detailed breakdowns of benchmark performance are [available on arXiv](https://arxiv.org/abs/2512.12818). This research is currently being prepared for conference submission and the wider peer review process.
 
+The benchmark results from this research can be inspected in our [visual benchmark explorer](https://hindsight-benchmarks.vercel.app). As additional improvements are made to Hindsight, new benchmark data will be available for review using this same tool.
 
 ## Quick Start
 
 ### Docker (recommended)
 
 ```bash
-export OPENAI_API_KEY=sk-xxx
+export OPENAI_API_KEY=your-key
 
-docker run -it --pull always --name hindsight --restart unless-stopped -p 8888:8888 -p 9999:9999 \
+docker run --rm -it --pull always -p 8888:8888 -p 9999:9999 \
   -e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \
-  -v hindsight-data:/home/hindsight/.pg0 \
+  -e HINDSIGHT_API_LLM_MODEL=o3-mini \
+  -v $HOME/.hindsight-docker:/home/hindsight/.pg0 \
   ghcr.io/vectorize-io/hindsight:latest
 ```
 
->API: http://localhost:8888
->UI: http://localhost:9999
+You can modify the LLM provider by setting `HINDSIGHT_API_LLM_PROVIDER`. Valid options are `openai`, `anthropic`, `gemini`, `groq`, `ollama`, and `lmstudio`. The documentation provides more details on [supported models](https://hindsight.vectorize.io/developer/models).
 
-You can modify the LLM provider by setting `HINDSIGHT_API_LLM_PROVIDER`. Valid options are `openai`, `anthropic`, `gemini`, `groq`, `ollama`, `lmstudio`, `minimax`, and `atlas` ([Atlas Cloud](https://www.atlascloud.ai/?utm_source=github&utm_medium=link&utm_campaign=hindsight)). The documentation provides more details on [supported models](https://hindsight.vectorize.io/developer/models).
+API: http://localhost:8888  
+UI: http://localhost:9999
 
-
-
-### Docker (external PostgreSQL)
-
-```bash
-export OPENAI_API_KEY=sk-xxx
-export HINDSIGHT_DB_PASSWORD=choose-a-password
-cd docker/docker-compose
-docker compose up 
-```
-
-> Oracle AI Database is also supported for enterprise deployments with full feature parity. See the [storage documentation](https://hindsight.vectorize.io/developer/storage) for details.
-
-
->API: http://localhost:8888
->UI: http://localhost:9999
-
-### Client
+Install client:
 
 ```bash
 pip install hindsight-client -U
@@ -97,7 +94,7 @@ pip install hindsight-client -U
 npm install @vectorize-io/hindsight-client
 ```
 
-#### Python
+Python example:
 
 ```python
 from hindsight_client import Hindsight
@@ -114,36 +111,12 @@ client.recall(bank_id="my-bank", query="What does Alice do?")
 client.reflect(bank_id="my-bank", query="Tell me about Alice")
 ```
 
-#### Node.js / TypeScript
-
-```bash
-npm install @vectorize-io/hindsight-client
-```
-
-```javascript
-const { HindsightClient } = require('@vectorize-io/hindsight-client');
-
-const main = async () => {
-  const client = new HindsightClient({ baseUrl: 'http://localhost:8888' });
-
-  await client.retain('my-bank', 'Alice loves hiking in Yosemite');
-
-  const results = await client.recall('my-bank', 'What does Alice like?');
-  console.log(results);
-}
-
-main();
-```
-
-
-### Python Embedded (no server required)
+### Python (embedded, no Docker)
 
 ```bash
 pip install hindsight-all -U
 ```
 
-On Intel (x86_64) Macs, install `hindsight-all-slim` instead — see [Supported Platforms](#supported-platforms).
-
 ```python
 import os
 from hindsight import HindsightServer, HindsightClient
@@ -158,48 +131,25 @@ with HindsightServer(
     results = client.recall(bank_id="my-bank", query="Where does Alice work?")
 ```
 
+### Node.js / TypeScript
 
----
+```bash
+npm install @vectorize-io/hindsight-client
+```
 
-## Use Cases
+```javascript
+const { HindsightClient } = require('@vectorize-io/hindsight-client');
 
+const client = new HindsightClient({ baseUrl: 'http://localhost:8888' });
 
-Hindsight is built to support conversational AI agents as well as agents that are intended to perform tasks autonomously. The ideal use case for Hindsight are agents that require a blend of these features such as AI employees that need to handle open-ended tasks, change behavior based on user feedback, and learn to perform complex tasks to automate work at a level that approximates a human work. Hindsight can be used with simple AI workflows like those built with n8n and other similar tools, but may be overkill for such applications.
-
-### Per-User Memories and Chat History
-
-One of the simpler use cases you can use Hindsight for is to personalize AI chatbots and other conversational agents by storing and recalling memories associated with individual users.
-
-The requirements for this use case usually look something like this:
-
-![Per-User Memories](./hindsight-docs/static/img/per-user-memory-requirements.png)
-
-<video src="https://github.com/user-attachments/assets/4805e8e1-e7d1-47c6-a4f8-2344a5ec8906" controls></video>
-
-Satisfying these requirements in Hindsight is straightforward. When new user inputs and tool calls are ingested into Hindsight using the retain operation, custom metadata can be used to enrich the new memories. Metadata provides a convenient way to isolate memories that need to be restricted to a given user. Once these are fed into the retain operation, any raw memories and mental models that get created can be filtered when retrieving relevant memories. 
-
-![Per-User Memories](./hindsight-docs/static/img/per-user-memory-howto.png)
+await client.retain('my-bank', 'Alice loves hiking in Yosemite');
+await client.recall('my-bank', 'What does Alice like?');
+```
 
 ---
 
 ## Architecture & Operations
 
-![Overview](./hindsight-docs/static/img/hindsight-overview.webp)
-
-Most agent memory implementations rely on basic vector search or sometimes use a knowledge graph. Hindsight uses biomimetic data structures to organize agent memories in a way that is more like how human memory works:
-
-- **World:** Facts about the world ("The stove gets hot")
-- **Experiences:** Agent's own experiences ("I touched the stove and it really hurt")
-- **Mental Models:** Learned understanding of the agent's world formed by reflecting on raw memories and experiences.
-
-Memories in Hindsight are stored in banks (i.e. memory banks). When memories are added to Hindsight, they are pushed into either the world facts or experiences memory pathway. They are then represented as a combination of entities, relationships, and time series with sparse/dense vector representations to aid in later recall.
-
-Hindsight provides three simple methods to interact with the system:
-
-- **Retain:** Provide information to Hindsight that you want it to remember
-- **Recall:** Retrieve memories from Hindsight
-- **Reflect:** Reflect on memories and experiences to generate new observations and insights from existing memories.
-
 ### Retain
 
 The `retain` operation is used to push new memories into Hindsight. It tells Hindsight to _retain_ the information you pass in as an input.
@@ -250,7 +200,7 @@ Recall performs 4 retrieval strategies in parallel:
 - Graph: Entity/temporal/causal links
 - Temporal: Time range filtering
 
-![Recall Operation](hindsight-docs/static/img/recall-operation.webp)
+![Retain Operation](hindsight-docs/static/img/recall-operation.webp)
 
 The individual results from the retrievals are merged, then ordered by relevance using reciprocal rank fusion and a cross-encoder reranking model.
 
@@ -258,7 +208,7 @@ The final output is trimmed as needed to fit within the token limit.
 
 ### Reflect
 
-The reflect operation is used to perform a more thorough analysis of existing memories. This allows the agent to form new connections between memories and build a more thorough understanding of its world.
+The reflect operation is used to perform a more thorough analysis of existing memories. This allows the agent to form new connections between memories which are then persisted as opinions and/or observations. When building agents, the reflect operation is a key capability to enable the agent to learn from its experiences. 
 
 For example, the `reflect` operation can be used to support use cases such as:
 
@@ -276,7 +226,7 @@ client = Hindsight(base_url="http://localhost:8888")
 client.reflect(bank_id="my-bank", query="What should I know about Alice?")
 ```
 
-![Reflect Operation](hindsight-docs/static/img/reflect-operation.webp)
+![Retain Operation](hindsight-docs/static/img/reflect-operation.webp)
 
 ---
 
@@ -292,7 +242,7 @@ client.reflect(bank_id="my-bank", query="What should I know about Alice?")
 - [CLI](https://hindsight.vectorize.io/sdks/cli)
 
 **Community:**
-- [Slack](https://join.slack.com/t/hindsight-space/shared_invite/zt-3nhbm4w29-LeSJ5Ixi6j8PdiYOCPlOgg)
+- [Slack](https://join.slack.com/t/hindsight-space/shared_invite/zt-3klo21kua-VUCC_zHP5rIcXFB1_5yw6A)
 - [GitHub Issues](https://github.com/vectorize-io/hindsight/issues)
 
 ---
@@ -301,19 +251,6 @@ client.reflect(bank_id="my-bank", query="What should I know about Alice?")
 [![Star History Chart](https://api.star-history.com/svg?repos=vectorize-io/hindsight&type=date&legend=top-left)](https://www.star-history.com/#vectorize-io/hindsight&type=date&legend=top-left)
 ---
 
-## Supported Platforms
-
-| Platform | Docker | Bare Metal (pip) | Embedded DB (pg0) |
-|----------|--------|------------------|--------------------|
-| **Linux** (x86_64, ARM64) | ✅ | ✅ | ✅ |
-| **macOS** (Apple Silicon / arm64) | ✅ | ✅ | ✅ |
-| **macOS** (Intel / x86_64) | ✅ | ⚠️ | ✅ |
-| **Windows** (x86_64) | ✅ | ✅ | ✅ |
-
-⚠️ Intel Macs: use `hindsight-all-slim` — see the [installation guide](https://hindsight.vectorize.io/developer/installation#supported-platforms) for details.
-
----
-
 ## Contributing
 
 See [CONTRIBUTING.md](./CONTRIBUTING.md).
@@ -325,5 +262,3 @@ MIT — see [LICENSE](./LICENSE)
 ---
 
 Built by [Vectorize.io](https://vectorize.io)
-
-<img src="https://umami-pixel.chris-latimer.workers.dev/?id=a8b043e6-6964-454d-80df-69b69d3f0d50&host=github.com&url=/vectorize-io/hindsight" width="1" height="1" alt="" />

deno.lock

@@ -1,170 +0,0 @@
-{
-  "version": "5",
-  "specifiers": {
-    "jsr:@std/assert@^1.0.17": "1.0.19",
-    "jsr:@std/assert@^1.0.19": "1.0.19",
-    "jsr:@std/expect@*": "1.0.18",
-    "jsr:@std/internal@^1.0.12": "1.0.12",
-    "jsr:@std/path@^1.1.4": "1.1.4",
-    "jsr:@std/testing@*": "1.0.17"
-  },
-  "jsr": {
-    "@std/[email protected]": {
-      "integrity": "eaada96ee120cb980bc47e040f82814d786fe8162ecc53c91d8df60b8755991e",
-      "dependencies": [
-        "jsr:@std/internal"
-      ]
-    },
-    "@std/[email protected]": {
-      "integrity": "8566eab35200466f8609eb7e7aed062ed0db314e9a258d5d201b1b8997ce801a",
-      "dependencies": [
-        "jsr:@std/assert@^1.0.19",
-        "jsr:@std/internal",
-        "jsr:@std/path"
-      ]
-    },
-    "@std/[email protected]": {
-      "integrity": "972a634fd5bc34b242024402972cd5143eac68d8dffaca5eaa4dba30ce17b027"
-    },
-    "@std/[email protected]": {
-      "integrity": "1d2d43f39efb1b42f0b1882a25486647cb851481862dc7313390b2bb044314b5",
-      "dependencies": [
-        "jsr:@std/internal"
-      ]
-    },
-    "@std/[email protected]": {
-      "integrity": "87bdc2700fa98249d48a17cd72413352d3d3680dcfbdb64947fd0982d6bbf681",
-      "dependencies": [
-        "jsr:@std/assert@^1.0.17",
-        "jsr:@std/internal"
-      ]
-    }
-  },
-  "workspace": {
-    "members": {
-      "hindsight-all-npm": {
-        "packageJson": {
-          "dependencies": [
-            "npm:@types/node@22",
-            "npm:tsup@^8.5.1",
-            "npm:typescript@^5.7.0",
-            "npm:vitest@^4.1.2"
-          ]
-        }
-      },
-      "hindsight-clients/typescript": {
-        "packageJson": {
-          "dependencies": [
-            "npm:@hey-api/[email protected]",
-            "npm:@types/jest@29",
-            "npm:@types/node@20",
-            "npm:jest@29",
-            "npm:ts-jest@29",
-            "npm:tsup@^8.5.1",
-            "npm:typescript@5"
-          ]
-        }
-      },
-      "hindsight-control-plane": {
-        "packageJson": {
-          "dependencies": [
-            "npm:@chenglou/pretext@^0.0.3",
-            "npm:@eslint/eslintrc@^3.3.3",
-            "npm:@eslint/js@^9.39.2",
-            "npm:@radix-ui/react-alert-dialog@^1.1.15",
-            "npm:@radix-ui/react-checkbox@^1.3.3",
-            "npm:@radix-ui/react-dialog@^1.1.15",
-            "npm:@radix-ui/react-dropdown-menu@^2.1.16",
-            "npm:@radix-ui/react-label@^2.1.8",
-            "npm:@radix-ui/react-popover@^1.1.15",
-            "npm:@radix-ui/react-select@^2.2.6",
-            "npm:@radix-ui/react-slider@^1.3.6",
-            "npm:@radix-ui/react-slot@^1.2.4",
-            "npm:@radix-ui/react-switch@^1.2.6",
-            "npm:@radix-ui/react-tabs@^1.1.13",
-            "npm:@radix-ui/react-tooltip@^1.2.8",
-            "npm:@tailwindcss/postcss@^4.1.17",
-            "npm:@tailwindcss/typography@~0.5.19",
-            "npm:@types/cytoscape@^3.21.9",
-            "npm:@types/node@^24.10.0",
-            "npm:@types/react-dom@^19.2.2",
-            "npm:@types/react@^19.2.2",
-            "npm:autoprefixer@^10.4.21",
-            "npm:class-variance-authority@~0.7.1",
-            "npm:clsx@^2.1.1",
-            "npm:cmdk@^1.1.1",
-            "npm:cytoscape-fcose@^2.2.0",
-            "npm:cytoscape@^3.33.1",
-            "npm:eslint-config-next@^16.0.1",
-            "npm:eslint-plugin-react-hooks@^7.0.1",
-            "npm:eslint-plugin-react@^7.37.5",
-            "npm:eslint@^9.39.1",
-            "npm:[email protected]",
-            "npm:next-themes@~0.4.6",
-            "npm:next@^16.1.7",
-            "npm:postcss@^8.5.6",
-            "npm:prettier@^3.7.4",
-            "npm:react-chrono@^2.9.1",
-            "npm:react-dom@^19.2.0",
-            "npm:react-is@^19.2.4",
-            "npm:react-markdown@^10.1.0",
-            "npm:react18-json-view@~0.2.9",
-            "npm:react@^19.2.0",
-            "npm:recharts@^3.5.1",
-            "npm:remark-gfm@^4.0.1",
-            "npm:sonner@^2.0.7",
-            "npm:tailwind-merge@^3.4.0",
-            "npm:tailwindcss-animate@^1.0.7",
-            "npm:tailwindcss@^4.1.17",
-            "npm:[email protected]",
-            "npm:typescript-eslint@^8.50.0",
-            "npm:typescript@^5.9.3"
-          ]
-        }
-      },
-      "hindsight-docs": {
-        "packageJson": {
-          "dependencies": [
-            "npm:@docusaurus/[email protected]",
-            "npm:@docusaurus/[email protected]",
-            "npm:@docusaurus/[email protected]",
-            "npm:@docusaurus/theme-common@^3.9.2",
-            "npm:@docusaurus/theme-mermaid@^3.9.2",
-            "npm:@docusaurus/[email protected]",
-            "npm:@docusaurus/[email protected]",
-            "npm:@easyops-cn/docusaurus-search-local@~0.52.2",
-            "npm:@mdx-js/react@3",
-            "npm:clsx@2",
-            "npm:prism-react-renderer@^2.3.0",
-            "npm:raw-loader@^4.0.2",
-            "npm:react-dom@19",
-            "npm:react-icons@^5.6.0",
-            "npm:react@19",
-            "npm:redocusaurus@^2.5.0",
-            "npm:typescript@~5.6.2"
-          ]
-        }
-      },
-      "hindsight-tools/hindsight-agent-sdk": {
-        "packageJson": {
-          "dependencies": [
-            "npm:@vectorize-io/hindsight-client@~0.5.6",
-            "npm:typescript@^5.4.0",
-            "npm:vitest@^4.1.2"
-          ]
-        }
-      },
-      "hindsight-tools/self-driving-agents": {
-        "packageJson": {
-          "dependencies": [
-            "npm:@clack/prompts@^1.2.0",
-            "npm:@vectorize-io/hindsight-client@~0.5.6",
-            "npm:picocolors@^1.1.0",
-            "npm:typescript@^5.4.0",
-            "npm:vitest@^4.1.2"
-          ]
-        }
-      }
-    }
-  }
-}

docker/docker-compose/alloydb/docker-compose.yaml

@@ -1,90 +0,0 @@
-name: hindsight
-# Docker Compose file for Hindsight with AlloyDB Omni and ScaNN
-# Uses Google's free AlloyDB Omni container image: https://hub.docker.com/r/google/alloydbomni
-#
-# Usage:
-# docker compose -f docker/docker-compose/alloydb/docker-compose.yaml up -d
-#
-# Make sure to set the required environment variables before running:
-# - HINDSIGHT_DB_PASSWORD: password for the AlloyDB Omni/PostgreSQL user
-# - Configure LLM provider variables as needed (see the hindsight service below)
-#
-# Optional environment variables with defaults:
-# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
-# - HINDSIGHT_DB_VERSION: AlloyDB Omni image tag (default: 17)
-# - HINDSIGHT_DB_USER: database user (default: hindsight_user)
-# - HINDSIGHT_DB_NAME: database name (default: hindsight_db)
-
-services:
-  db:
-    image: google/alloydbomni:${HINDSIGHT_DB_VERSION:-17}
-    container_name: hindsight-db-alloydb
-    restart: always
-    ports:
-      - "5438:5432"
-    environment:
-      POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
-      POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-      POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
-    volumes:
-      - alloydb_data:/var/lib/postgresql/data
-    networks:
-      - hindsight-net
-
-  alloydb-init:
-    image: google/alloydbomni:${HINDSIGHT_DB_VERSION:-17}
-    depends_on:
-      - db
-    environment:
-      - PGPASSWORD=${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-    command:
-      - bash
-      - -c
-      - |
-        echo 'Waiting for AlloyDB Omni to be ready...'
-        until pg_isready -h hindsight-db-alloydb -p 5432 -U ${HINDSIGHT_DB_USER:-hindsight_user}; do
-          echo 'AlloyDB Omni is unavailable - sleeping'
-          sleep 2
-        done
-        echo 'AlloyDB Omni is ready - creating ${HINDSIGHT_DB_NAME:-hindsight_db} database'
-        psql -h hindsight-db-alloydb -p 5432 -U ${HINDSIGHT_DB_USER:-hindsight_user} -c 'CREATE DATABASE ${HINDSIGHT_DB_NAME:-hindsight_db};' 2>/dev/null || echo 'Database already exists'
-        echo 'Creating vector and alloydb_scann extensions'
-        psql -h hindsight-db-alloydb -p 5432 -U ${HINDSIGHT_DB_USER:-hindsight_user} -d ${HINDSIGHT_DB_NAME:-hindsight_db} -c 'CREATE EXTENSION IF NOT EXISTS vector;'
-        psql -h hindsight-db-alloydb -p 5432 -U ${HINDSIGHT_DB_USER:-hindsight_user} -d ${HINDSIGHT_DB_NAME:-hindsight_db} -c 'CREATE EXTENSION IF NOT EXISTS alloydb_scann CASCADE;'
-        echo 'Database and extensions created successfully'
-    restart: "no"
-    networks:
-      - hindsight-net
-
-  hindsight:
-    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
-    container_name: hindsight-app
-    ports:
-      - "8888:8888"
-      - "9999:9999"
-    environment:
-      # LLM Configuration
-      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
-      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
-
-      # Database Configuration
-      HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
-
-      # Vector and Text Search Extensions
-      HINDSIGHT_API_VECTOR_EXTENSION: scann
-      HINDSIGHT_API_TEXT_SEARCH_EXTENSION: native
-
-    depends_on:
-      db:
-        condition: service_started
-      alloydb-init:
-        condition: service_completed_successfully
-    networks:
-      - hindsight-net
-
-networks:
-  hindsight-net:
-    driver: bridge
-
-volumes:
-  alloydb_data:

docker/docker-compose/claude-code/README.md

@@ -1,113 +0,0 @@
-# Hindsight with Claude Code (Claude Pro/Max subscription)
-
-Run Hindsight inside Docker using the `claude-code` LLM provider, backed by
-your host machine's Claude Pro or Max subscription credentials.
-
-The standalone Hindsight Docker image ships `claude-agent-sdk` but does **not**
-bundle the host `claude` CLI binary or any Claude credentials. This Compose
-file bind-mounts the host's CLI install and credentials into the container so
-the `claude-code` provider works without an API key.
-
-## When to use this
-
-- You have an active Claude Pro or Max subscription and want to use it for
-  Hindsight without paying separate Anthropic API costs.
-- You want a one-command `docker compose up` instead of a long `docker run`
-  invocation with many flags.
-- You are running on **Linux/amd64** — macOS Docker Desktop and Windows host
-  paths differ and are not yet covered (please open an issue if you'd like to
-  contribute a verified recipe for either).
-
-> **Personal-use only.**  Anthropic's
-> [Agent SDK documentation](https://docs.claude.com/en/api/agent-sdk/overview)
-> states that third-party developers should not offer claude.ai login or rate
-> limits for their products. Hindsight does **not** perform any login on your
-> behalf — it uses credentials you've already authenticated via
-> `claude auth login`. In January 2026, Anthropic
-> [enforced restrictions](https://paddo.dev/blog/anthropic-walled-garden-crackdown/)
-> against tools that spoofed the Claude Code client identity; Hindsight uses
-> the official Claude Agent SDK instead.
->
-> Do not deploy this configuration to shared environments or production. For
-> that, use the `anthropic` provider with an API key from the
-> [Anthropic Console](https://console.anthropic.com/). Usage counts against
-> your Claude Pro/Max subscription limits.
-
-## Prerequisites
-
-- Host has `claude` CLI installed (e.g., `npm install -g @anthropics/claude-code`)
-  and `claude auth login` has been run successfully.
-- `~/.claude.json` and `~/.claude/.credentials.json` exist on the host.
-- Host `claude` CLI version is **2.1.128 or newer** — the version bundled with
-  `claude-agent-sdk` 0.5.x has a protocol incompatibility in containers, so
-  the recipe overrides it with the host binary.
-
-## Quick start
-
-```bash
-# Set your host UID/GID (defaults to 1000:1000 if unset)
-export HOST_UID=$(id -u)
-export HOST_GID=$(id -g)
-
-docker compose -f docker/docker-compose/claude-code/docker-compose.yaml up -d
-```
-
-- API: http://localhost:8888
-- Control Plane: http://localhost:9999
-
-## Post-setup (one-time)
-
-After the container starts for the first time, run these commands to fix
-permissions and symlink the host `claude` binary into `$PATH`:
-
-```bash
-# Make ~/.claude writable by your UID (the CLI writes session/project state)
-docker exec --user 0:0 hindsight-claude-code chown $(id -u):$(id -g) /home/hindsight/.claude
-docker exec --user 0:0 hindsight-claude-code chmod 755 /home/hindsight/.claude
-
-# Symlink the host claude binary into PATH
-docker exec --user 0:0 hindsight-claude-code \
-  ln -sf /home/hindsight/.local/share/claude/versions/2.1.128 /usr/local/bin/claude
-```
-
-If you set `CLAUDE_CLI_VERSION` to a version other than `2.1.128`, update the
-symlink path accordingly.
-
-## Notes on the bind-mount surface (every flag is load-bearing)
-
-- **Host `claude` binary required** — the image ships only `claude-agent-sdk`,
-  not the CLI itself.
-- **SDK bundled-binary override** — the override of
-  `claude_agent_sdk/_bundled/claude` works around a protocol issue in the
-  bundled v2.1.121 binary inside containers. Once `claude-agent-sdk` ships
-  with v2.1.128+ this override can be dropped. Set `CLAUDE_CLI_VERSION` to
-  match your installed version.
-- **Single-file credential mounts** — credentials are mounted as individual
-  `:ro` files rather than a whole-directory `:ro` mount of `~/.claude`,
-  because the CLI writes session/project state at runtime and a read-only
-  directory mount silently breaks it.
-- **`--user` / `user:`** — the `user: ${HOST_UID}:${HOST_GID}` pattern
-  requires `chmod 755 /home/hindsight`, which is built into the image since
-  v0.6.0 (see [#1481](https://github.com/vectorize-io/hindsight/issues/1481)).
-- **`~/.hindsight-docker` data directory** — the pg0 data bind mount must be
-  writable by your host UID (see
-  [#1483](https://github.com/vectorize-io/hindsight/issues/1483)).
-- **Verified** on `linux/amd64` against `ghcr.io/vectorize-io/hindsight:latest`
-  v0.5.6+.
-
-## Using a different Claude CLI version
-
-If your host has a `claude` version other than 2.1.128, set
-`CLAUDE_CLI_VERSION` before starting:
-
-```bash
-export CLAUDE_CLI_VERSION=2.2.0
-docker compose -f docker/docker-compose/claude-code/docker-compose.yaml up -d
-```
-
-Then update the post-setup symlink to match:
-
-```bash
-docker exec --user 0:0 hindsight-claude-code \
-  ln -sf /home/hindsight/.local/share/claude/versions/2.2.0 /usr/local/bin/claude
-```

docker/docker-compose/claude-code/docker-compose.yaml

@@ -1,44 +0,0 @@
-name: hindsight-claude-code
-# Run Hindsight with the claude-code LLM provider, using your host machine's
-# Claude Pro/Max subscription credentials.  Linux/amd64 only for now.
-#
-# Quick start:
-#   docker compose -f docker/docker-compose/claude-code/docker-compose.yaml up -d
-#
-# See README.md for prerequisites, post-setup steps, and important caveats.
-
-services:
-  hindsight:
-    image: ghcr.io/vectorize-io/hindsight:latest
-    container_name: hindsight-claude-code
-    user: "${HOST_UID:-1000}:${HOST_GID:-1000}"
-    ports:
-      - "127.0.0.1:8888:8888"
-      - "127.0.0.1:9999:9999"
-    environment:
-      HOME: /home/hindsight
-      USER: hindsight
-      LOGNAME: hindsight
-      PATH: /usr/local/bin:/usr/bin:/bin:/app/api/.venv/bin
-      HINDSIGHT_API_LLM_PROVIDER: claude-code
-    volumes:
-      # ── Persistent data ────────────────────────────────────────────
-      # Writable pg0 data directory.  Must be writable by HOST_UID.
-      - ${HOME:-.}/.hindsight-docker:/home/hindsight/.pg0
-
-      # ── Claude credentials (read-only, single-file mounts) ────────
-      # A whole-directory :ro mount of ~/.claude silently breaks the
-      # CLI, which writes session/project state at runtime — so we
-      # mount only the two credential files.
-      - ${HOME}/.claude/.credentials.json:/home/hindsight/.claude/.credentials.json:ro
-      - ${HOME}/.claude.json:/home/hindsight/.claude.json:ro
-
-      # ── Claude CLI install (read-only) ─────────────────────────────
-      - ${HOME}/.local/share/claude:/home/hindsight/.local/share/claude:ro
-
-      # ── SDK bundled-binary override ────────────────────────────────
-      # The claude-agent-sdk 0.5.x image bundles v2.1.121 which has a
-      # protocol incompatibility in containers.  Override it with the
-      # host's v2.1.128+ binary.  Drop this mount once claude-agent-sdk
-      # ships with v2.1.128+.
-      - ${HOME}/.local/share/claude/versions/${CLAUDE_CLI_VERSION:-2.1.128}:/app/api/.venv/lib/python3.11/site-packages/claude_agent_sdk/_bundled/claude:ro

docker/docker-compose/custom-models/Dockerfile

@@ -1,34 +0,0 @@
-# Example: custom Hindsight image with non-default local models baked in.
-#
-# Use this pattern in production when you run a non-default embedder or
-# reranker. Baking models into the image removes the runtime dependency on
-# HuggingFace and lets the container registry handle caching per node, so
-# you don't need a model-cache PVC.
-#
-# Built on top of the slim image so only the deps and models you actually
-# use end up in the final image.
-FROM ghcr.io/vectorize-io/hindsight:latest-slim
-
-# Install the local-ml deps required to load sentence-transformers /
-# cross-encoder models at runtime. Pinned ranges mirror hindsight-api-slim's
-# `local-ml` extra in hindsight-api-slim/pyproject.toml. Use `uv pip
-# install` against the image's venv explicitly: the slim image's venv was
-# created by `uv sync` and does not ship its own `pip`, so a bare
-# `pip install` would fall back to user site-packages and not be visible
-# to the runtime python.
-RUN uv pip install --python /app/api/.venv/bin/python --no-cache \
-    'sentence-transformers>=3.3.0' \
-    'transformers>=4.53.0' \
-    'torch>=2.6.0'
-
-# Pre-download the models you want to use. Replace these with your own.
-# The defaults bundled in the full image are BAAI/bge-small-en-v1.5 and
-# cross-encoder/ms-marco-MiniLM-L-6-v2; here we pick multilingual variants
-# as a concrete non-default example.
-ARG EMBEDDER=sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
-ARG RERANKER=cross-encoder/mmarco-mMiniLMv2-L12-H384-v1
-ENV HF_HUB_DOWNLOAD_TIMEOUT=600
-RUN python -c "\
-from sentence_transformers import SentenceTransformer, CrossEncoder; \
-SentenceTransformer('${EMBEDDER}'); \
-CrossEncoder('${RERANKER}')"

docker/docker-compose/custom-models/README.md

@@ -1,81 +0,0 @@
-# Hindsight with Custom Local Models
-
-Example Docker Compose setup that builds a Hindsight image with **non-default
-local embedder and reranker models baked in at build time**.
-
-This is the recommended pattern for production when you use a non-default
-local model: the container registry caches model layers per node, pod
-startup is deterministic, and you don't need a model-cache PVC (or any
-runtime dependency on HuggingFace).
-
-## When to use this
-
-- You override `HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL` or
-  `HINDSIGHT_API_RERANKER_LOCAL_MODEL` to a non-default model.
-- You want pod startup to be deterministic and offline-capable.
-- You'd otherwise reach for a Helm `modelCache` PVC just to avoid
-  re-downloading models.
-
-If you're using the **default** local models, the published full image
-(`ghcr.io/vectorize-io/hindsight:latest`) already bakes them in — you don't
-need this example.
-
-If you're using **external** providers (TEI, OpenAI, Cohere, ...) for
-embeddings and reranking, use the slim image directly — no models are
-needed in the image.
-
-## Quick start
-
-```bash
-export OPENAI_API_KEY=sk-xxx
-
-docker compose -f docker/docker-compose/custom-models/docker-compose.yaml up --build
-```
-
-- API: http://localhost:8888
-- Control Plane: http://localhost:9999
-
-## Using your own models
-
-Override the build args to bake different models:
-
-```bash
-docker compose -f docker/docker-compose/custom-models/docker-compose.yaml build \
-  --build-arg EMBEDDER=your-org/your-embedder \
-  --build-arg RERANKER=your-org/your-reranker
-```
-
-Then update the matching `HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL` and
-`HINDSIGHT_API_RERANKER_LOCAL_MODEL` values in `docker-compose.yaml` so the
-runtime points at the same model IDs.
-
-## Verifying the models are baked in
-
-`docker-compose.yaml` sets `HF_HUB_OFFLINE=1` and `TRANSFORMERS_OFFLINE=1`
-so that any attempt to download a model at runtime fails loudly instead of
-silently re-downloading. If the container starts and serves recall queries
-with these set, the models are correctly baked in.
-
-You can also inspect the image directly:
-
-```bash
-docker run --rm --entrypoint sh hindsight-custom-models-hindsight \
-  -c 'ls ~/.cache/huggingface/hub/'
-```
-
-## Why not a model-cache PVC?
-
-The Helm chart exposes an optional `api.persistence.modelCache` PVC for
-caching downloaded models across pod restarts. Compared to baking models
-into the image:
-
-- A PVC adds storage cost — one PVC per worker replica with
-  `volumeClaimTemplates`.
-- `ReadWriteOnce` (the default) pins pods to a node.
-- The PVC needs lifecycle management on `helm uninstall` / `helm upgrade`
-  — without `helm.sh/resource-policy: keep` it is deleted on uninstall;
-  with it, storage keeps billing forever until manually cleaned up.
-- Pod startup still depends on HuggingFace being reachable on first run.
-
-Image layers, by contrast, are pulled once per node and cached for free by
-the container runtime, with no orphaned-storage cleanup story.

docker/docker-compose/custom-models/docker-compose.yaml

@@ -1,44 +0,0 @@
-name: hindsight-custom-models
-# Example: run a custom Hindsight image with non-default local models baked
-# in at build time, so pod startup does not depend on HuggingFace at runtime.
-#
-# Quick start:
-#   export OPENAI_API_KEY=sk-xxx
-#   docker compose -f docker/docker-compose/custom-models/docker-compose.yaml up --build
-#
-# Required environment variables:
-# - OPENAI_API_KEY (or configure another LLM provider via HINDSIGHT_API_LLM_*)
-
-services:
-  hindsight:
-    build:
-      context: .
-      dockerfile: Dockerfile
-      # Override at build time to bake different models:
-      #   docker compose build --build-arg EMBEDDER=your-org/your-embedder
-      args:
-        EMBEDDER: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
-        RERANKER: cross-encoder/mmarco-mMiniLMv2-L12-H384-v1
-    container_name: hindsight-custom-models
-    ports:
-      - "8888:8888"
-      - "9999:9999"
-    environment:
-      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
-      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
-
-      # Point Hindsight at the models baked into the image above.
-      HINDSIGHT_API_EMBEDDINGS_PROVIDER: local
-      HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
-      HINDSIGHT_API_RERANKER_PROVIDER: local
-      HINDSIGHT_API_RERANKER_LOCAL_MODEL: cross-encoder/mmarco-mMiniLMv2-L12-H384-v1
-
-      # Fail fast if a model is missing from the image instead of silently
-      # falling back to a HuggingFace download at runtime.
-      HF_HUB_OFFLINE: "1"
-      TRANSFORMERS_OFFLINE: "1"
-    volumes:
-      - pg_data:/home/hindsight/.pg0
-
-volumes:
-  pg_data:

docker/docker-compose/external-pg/docker-compose.yaml

@@ -1,54 +0,0 @@
-# Docker Compose file for Hindsight with PostgreSQL and pgvector
-#
-# Make sure to set the required environment variables before running:
-# - HINDSIGHT_DB_PASSWORD: Password for the PostgreSQL user
-# - Configure LLM provider variables as needed (see below in the hindsight service)
-#
-# Usage:
-# docker compose up -d
-#
-# Optional environment variables with defaults:
-# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
-# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
-# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
-# - HINDSIGHT_DB_VERSION: PostgreSQL version (default: 18)
-
-services:
-  db:
-    # Use a PostgreSQL-Image with pgvector extension pre-installed
-    # see https://hub.docker.com/r/pgvector/pgvector 
-    image: pgvector/pgvector:pg${HINDSIGHT_DB_VERSION:-18}
-    container_name: hindsight-db
-    restart: always
-    # Expose PostgreSQL port
-    # ports:
-      # - "5432:5432"
-    environment:
-      POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
-      POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:?Please set the HINDSIGHT_DB_PASSWORD env variable}
-      POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
-    volumes:
-      - pg_data:/var/lib/postgresql/${HINDSIGHT_DB_VERSION:-18}/docker
-    networks:
-      - hindsight-net
-
-  hindsight:
-    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
-    container_name: hindsight-app
-    ports:
-      - "8888:8888"
-      - "9999:9999"
-    environment:
-      - HINDSIGHT_API_LLM_API_KEY=${OPENAI_API_KEY?Please set the OPENAI_API_KEY env variable}
-      - HINDSIGHT_API_DATABASE_URL=postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:?Please set the HINDSIGHT_DB_PASSWORD env variable}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
-    depends_on:
-      - db
-    networks:
-      - hindsight-net
-
-networks:
-  hindsight-net:
-    driver: bridge
-
-volumes:
-  pg_data:

docker/docker-compose/local-llm/README.md

@@ -1,103 +0,0 @@
-# Hindsight with a local llama.cpp server sidecar
-
-Example Docker Compose setup that runs Hindsight against a **local
-llama.cpp server**, fully offline, with no external API key required.
-
-## Architecture
-
-```
-┌────────────┐  HTTP /v1/chat/completions  ┌──────────────────────────────┐
-│ hindsight  │ ──────────────────────────▶ │ llama.cpp server (sidecar)   │
-│ (API + CP) │                             │ ghcr.io/ggml-org/llama.cpp   │
-└────────────┘                             └──────────────────────────────┘
-```
-
-`llama.cpp` runs as its own container and exposes an OpenAI-compatible
-HTTP API. Hindsight talks to it via the standard `openai` LLM provider
-with `HINDSIGHT_API_LLM_BASE_URL` pointed at the sidecar.
-
-This pattern follows
-[*Hosting llama-server with Docker* (ServiceStack)](https://servicestack.net/posts/hosting-llama-server).
-
-### Why a sidecar and not the in-process `llamacpp` provider?
-
-Hindsight does ship an in-process `llamacpp` provider that spawns
-`llama-cpp-python`, but the **published `ghcr.io/vectorize-io/hindsight`
-image deliberately omits `llama-cpp-python`** to keep the image small and
-avoid bundling native inference libraries that most users don't need.
-Trying to set `HINDSIGHT_API_LLM_PROVIDER=llamacpp` against the published
-image fails with `ModuleNotFoundError: No module named 'llama_cpp'`.
-
-The sidecar approach side-steps that entirely: the official llama.cpp
-image is used as-is for inference, Hindsight is used as-is for memory.
-Clean separation, no derived images.
-
-## Quick start
-
-```bash
-docker compose -f docker/docker-compose/local-llm/docker-compose.yaml up
-```
-
-- API: http://localhost:8888
-- Control Plane: http://localhost:9999
-
-**First boot downloads ~3.5 GB** (Gemma 4 E2B Q4_K_M GGUF) into the
-`llama_models` named volume. Subsequent boots reuse it.
-
-Hindsight only starts after llama.cpp's `/health` endpoint reports
-healthy, so the API will appear "stuck" for a few minutes on the first
-run while the model downloads.
-
-## Using a different model
-
-Override the HuggingFace repo / file in `docker-compose.yaml`:
-
-```yaml
-environment:
-  LLAMA_ARG_HF_REPO: bartowski/Qwen2.5-7B-Instruct-GGUF
-  LLAMA_ARG_HF_FILE: Qwen2.5-7B-Instruct-Q4_K_M.gguf
-```
-
-Also update `HINDSIGHT_API_LLM_MODEL` on the `hindsight` service to a
-matching alias (the value is sent to llama-server as the OpenAI `model`
-field — llama-server is lenient about this but it shows up in logs).
-
-## GPU acceleration
-
-The default compose file targets CPU because not everyone has a GPU. On
-CPU, Gemma 4 E2B runs at ~2-3 tokens/sec — fine for a smoke test, but the
-retain pipeline (which makes several multi-hundred-token LLM calls per
-memory) will time out against Hindsight's default LLM timeout. **For any
-real use, run on a GPU.**
-
-### NVIDIA
-
-1. Switch the `llama` service image from `:server` to `:server-cuda`.
-2. Uncomment the `LLAMA_ARG_N_GPU_LAYERS: "999"` env var (offload all
-   layers to GPU).
-3. Uncomment the `deploy.resources.reservations.devices` block.
-4. Install the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
-   on the host.
-
-The compose file has all four spots marked with inline comments.
-
-### Apple Silicon / ROCm / Vulkan
-
-The official `ghcr.io/ggml-org/llama.cpp` image only ships CPU and CUDA
-variants. For Metal (Apple Silicon), ROCm (AMD), or Vulkan backends,
-build llama.cpp yourself with the appropriate flags and reference the
-image you build instead. Docker Desktop on macOS cannot pass through the
-host GPU to a Linux container in any case — for Apple Silicon, run
-llama-server directly on the host and only put Hindsight in Docker.
-
-## Caveats
-
-- llama.cpp's HTTP API is OpenAI-compatible but not 100% feature-parity.
-  Function/tool calling support depends on the chat template baked into
-  the GGUF; some retain/reflect flows may behave differently than against
-  a hosted OpenAI model.
-- Small GGUFs (~3 B params) are useful for smoke testing but will
-  underperform a hosted frontier model on retain quality. Use a larger
-  GGUF (7-13 B params) for production-quality memory.
-- The `llama_models` named volume persists the GGUF across `docker
-  compose down`/`up` so the model is downloaded once, not every restart.

docker/docker-compose/local-llm/docker-compose.yaml

@@ -1,74 +0,0 @@
-name: hindsight-local-llm
-# Example: run Hindsight against a local llama.cpp server sidecar — fully
-# offline, no external API key needed.
-#
-# Pattern follows https://servicestack.net/posts/hosting-llama-server :
-# llama.cpp runs as its own container exposing an OpenAI-compatible HTTP
-# API, and Hindsight talks to it via the `openai` LLM provider with a
-# custom `base_url`. This means we can use the published Hindsight image
-# unchanged — no derived Dockerfile, no `llama-cpp-python` install on top.
-#
-# Quick start:
-#   docker compose -f docker/docker-compose/local-llm/docker-compose.yaml up
-#
-# First boot downloads the default Gemma 4 E2B GGUF (~3.5 GB) into the
-# `llama_models` volume; subsequent boots reuse it.
-
-services:
-  llama:
-    image: ghcr.io/ggml-org/llama.cpp:server
-    container_name: hindsight-local-llm-llama
-    environment:
-      LLAMA_ARG_HOST: 0.0.0.0
-      LLAMA_ARG_PORT: "8080"
-      # Auto-download a small GGUF from HuggingFace on first start.
-      # Override these to use a different model.
-      LLAMA_ARG_HF_REPO: bartowski/google_gemma-4-E2B-it-GGUF
-      LLAMA_ARG_HF_FILE: google_gemma-4-E2B-it-Q4_K_M.gguf
-      LLAMA_ARG_CTX_SIZE: "8192"
-      # Uncomment for NVIDIA GPU (and switch image to :server-cuda):
-      # LLAMA_ARG_N_GPU_LAYERS: "999"
-    volumes:
-      # llama-server stores HuggingFace downloads under ~/.cache/huggingface
-      # (not ~/.cache/llama.cpp), so mount the named volume there to avoid
-      # re-downloading the GGUF on every recreate.
-      - llama_models:/root/.cache/huggingface
-    healthcheck:
-      test: ["CMD-SHELL", "curl -fsS http://localhost:8080/health || exit 1"]
-      interval: 10s
-      timeout: 5s
-      retries: 60
-      start_period: 30s
-    # For NVIDIA GPU acceleration, swap the image above to
-    # `ghcr.io/ggml-org/llama.cpp:server-cuda` and uncomment:
-    # deploy:
-    #   resources:
-    #     reservations:
-    #       devices:
-    #         - driver: nvidia
-    #           count: all
-    #           capabilities: [gpu]
-
-  hindsight:
-    image: ghcr.io/vectorize-io/hindsight:latest
-    container_name: hindsight-local-llm
-    depends_on:
-      llama:
-        condition: service_healthy
-    ports:
-      - "8888:8888"
-      - "9999:9999"
-    environment:
-      # llama-server is OpenAI-compatible, so use the `openai` provider and
-      # point base_url at the sidecar. The API key is unused by llama-server
-      # but Hindsight requires the env var to be set.
-      HINDSIGHT_API_LLM_PROVIDER: openai
-      HINDSIGHT_API_LLM_BASE_URL: http://llama:8080/v1
-      HINDSIGHT_API_LLM_API_KEY: not-needed
-      HINDSIGHT_API_LLM_MODEL: gemma-4-e2b-it
-    volumes:
-      - pg_data:/home/hindsight/.pg0
-
-volumes:
-  pg_data:
-  llama_models:

docker/docker-compose/nginx/README.md

@@ -1,96 +0,0 @@
-# Nginx Reverse Proxy with Custom Base Path
-
-Deploy Hindsight API under `/hindsight` (or any custom path) using Nginx reverse proxy.
-
-## Quick Start (Published Image - API Only)
-
-```bash
-docker-compose up
-```
-
-- **API:** http://localhost:8080/hindsight/docs
-- **Control Plane:** http://localhost:9999 (direct access, not proxied)
-
-## Full Stack with Custom Base Path (Requires Build)
-
-**Important:** You cannot rebuild from the published image with build args. You must build from source.
-
-### Build from Source with Custom Base Path
-
-1. **Clone the repository** (if you haven't):
-```bash
-git clone https://github.com/vectorize-io/hindsight.git
-cd hindsight
-```
-
-2. **Build with base path**:
-```bash
-docker build \
-  --build-arg NEXT_PUBLIC_BASE_PATH=/hindsight \
-  -f docker/standalone/Dockerfile \
-  -t hindsight:custom \
-  .
-```
-
-3. **Update docker-compose.yml** to use your built image:
-```yaml
-services:
-  hindsight:
-    image: hindsight:custom  # ← Change this
-    environment:
-      HINDSIGHT_API_BASE_PATH: /hindsight
-      NEXT_PUBLIC_BASE_PATH: /hindsight
-```
-
-4. **Update nginx.conf** to handle Control Plane routes (see below)
-
-5. **Run**:
-```bash
-docker-compose up
-```
-
-### Required nginx.conf for Full Stack
-
-Replace the current `nginx.conf` with this to proxy both API and Control Plane:
-
-```nginx
-events { worker_connections 1024; }
-
-http {
-    include /etc/nginx/mime.types;
-    default_type application/octet-stream;
-
-    upstream hindsight_api { server hindsight:8888; }
-    upstream hindsight_cp { server hindsight:9999; }
-
-    server {
-        listen 80;
-
-        # API
-        location ~ ^/hindsight/(docs|openapi\.json|health|metrics|v1|mcp) {
-            proxy_pass http://hindsight_api;
-            proxy_set_header Host $http_host;
-        }
-
-        # Control Plane static files
-        location ~ ^/hindsight/_next/ {
-            proxy_pass http://hindsight_cp;
-            proxy_set_header Host $http_host;
-        }
-
-        # Control Plane UI
-        location /hindsight {
-            proxy_pass http://hindsight_cp;
-            proxy_set_header Host $http_host;
-        }
-
-        location = / { return 301 /hindsight; }
-    }
-}
-```
-
-### Why Build is Required
-
-Next.js requires `basePath` at **build time**. The published image was built without a custom base path, so you must rebuild from source with the `NEXT_PUBLIC_BASE_PATH` build arg to deploy the Control Plane under a subpath.
-
-The API works without rebuild because `HINDSIGHT_API_BASE_PATH` is a runtime environment variable.

docker/docker-compose/nginx/docker-compose.yml

@@ -1,90 +0,0 @@
-# Hindsight API deployment with Nginx reverse proxy (API-only)
-#
-# This example deploys Hindsight API under the path /hindsight with:
-# - Hindsight standalone image (API + Control Plane + embedded pg0)
-# - Nginx reverse proxy (API only)
-#
-# Quick Start:
-#   docker-compose -f docker/docker-compose/nginx/docker-compose.yml up
-#
-# Access:
-#   API (via nginx):         http://localhost:8080/hindsight/docs
-#   Control Plane (direct):  http://localhost:9999
-#
-# For full stack deployment (API + Control Plane both under /hindsight):
-#   See README.md in this directory for instructions on building with basePath.
-#
-# Note: This configuration uses the published image (no build required).
-#       Control Plane is served directly because Next.js basePath requires
-#       build-time configuration. See README.md for the full stack option.
-
-services:
-  # Hindsight (API + Control Plane + embedded pg0)
-  hindsight:
-    image: ghcr.io/vectorize-io/hindsight:latest
-    ports:
-      - "9999:9999"  # Control Plane (direct access, not proxied)
-    environment:
-      # API base path for reverse proxy
-      HINDSIGHT_API_BASE_PATH: /hindsight
-
-      # LLM configuration
-      # Using mock provider for testing (no API key needed)
-      # For production, set OPENAI_API_KEY or ANTHROPIC_API_KEY and use a real provider
-      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-mock}
-      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-not-needed-for-mock}
-      HINDSIGHT_API_LLM_MODEL: ${HINDSIGHT_API_LLM_MODEL:-mock-model}
-
-      # Production examples (uncomment and set appropriate API key):
-      # HINDSIGHT_API_LLM_PROVIDER: openai
-      # HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY}
-      # HINDSIGHT_API_LLM_MODEL: gpt-4o-mini
-
-      # HINDSIGHT_API_LLM_PROVIDER: anthropic
-      # HINDSIGHT_API_LLM_API_KEY: ${ANTHROPIC_API_KEY}
-      # HINDSIGHT_API_LLM_MODEL: claude-sonnet-4-20250514
-
-      # Server config
-      HINDSIGHT_API_HOST: 0.0.0.0
-      HINDSIGHT_API_PORT: 8888
-      HINDSIGHT_API_LOG_LEVEL: info
-
-      # Control Plane config
-      HINDSIGHT_CP_DATAPLANE_API_URL: http://localhost:8888
-      # Optional: Require a shared access key for Control Plane UI access
-      # HINDSIGHT_CP_ACCESS_KEY: your-secret-key
-    volumes:
-      # Persist embedded pg0 database
-      - hindsight_data:/app/data
-    # Note: Ports not exposed - access via Nginx at localhost:8080/hindsight/
-    # To debug directly, uncomment these ports:
-    # ports:
-    #   - "8888:8888"  # API
-    #   - "9999:9999"  # Control Plane
-    healthcheck:
-      test: ["CMD", "curl", "-f", "http://localhost:8888/hindsight/health"]
-      interval: 10s
-      timeout: 5s
-      retries: 3
-      start_period: 30s
-    networks:
-      - hindsight
-
-  # Nginx reverse proxy
-  nginx:
-    image: nginx:alpine
-    ports:
-      - "8080:80"
-    volumes:
-      - ./nginx.conf:/etc/nginx/nginx.conf:ro
-    depends_on:
-      hindsight:
-        condition: service_healthy
-    networks:
-      - hindsight
-
-volumes:
-  hindsight_data:
-
-networks:
-  hindsight:

docker/docker-compose/nginx/nginx.conf

@@ -1,40 +0,0 @@
-# Nginx configuration for API-only reverse proxy
-# Control Plane accessed directly (not through nginx)
-
-events {
-    worker_connections 1024;
-}
-
-http {
-    include /etc/nginx/mime.types;
-    default_type application/octet-stream;
-
-    # Logging
-    access_log /var/log/nginx/access.log;
-    error_log /var/log/nginx/error.log;
-
-    # Upstream - Hindsight API
-    upstream hindsight_api {
-        server hindsight:8888;
-    }
-
-    server {
-        listen 80;
-        server_name _;
-
-        # API endpoints - forward with /hindsight prefix
-        location /hindsight/ {
-            proxy_pass http://hindsight_api;
-
-            proxy_set_header Host $http_host;
-            proxy_set_header X-Real-IP $remote_addr;
-            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
-            proxy_set_header X-Forwarded-Proto $scheme;
-        }
-
-        # Redirect root to API docs
-        location = / {
-            return 301 /hindsight/docs;
-        }
-    }
-}

docker/docker-compose/pg_search/Dockerfile

@@ -1,7 +0,0 @@
-# PostgreSQL with pgvector and ParadeDB pg_search extensions.
-#
-# The official ParadeDB image ships PostgreSQL with pg_search and pgvector
-# already installed, so no build steps are required. We pin to the PG17
-# variant for parity with the other Hindsight docker-compose examples
-# (vchord, pg_textsearch).
-FROM paradedb/paradedb:latest-pg17

docker/docker-compose/pg_search/docker-compose.yaml

@@ -1,96 +0,0 @@
-name: hindsight
-# Docker Compose file for Hindsight with PostgreSQL and ParadeDB pg_search.
-#
-# pg_search is the only BM25 backend supported by Hindsight that works with
-# Citus, so this is the recommended setup for horizontally scaled deployments.
-#
-# Usage:
-#   docker compose -f docker/docker-compose/pg_search/docker-compose.yaml up -d
-#
-# Required environment variables:
-# - HINDSIGHT_DB_PASSWORD: Password for the PostgreSQL user
-# - Configure LLM provider variables as needed (see the hindsight service)
-#
-# Optional environment variables with defaults:
-# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
-# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
-# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
-# - HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER: ParadeDB pg_search
-#   tokenizer for new BM25 indexes (default: empty, uses ParadeDB default)
-
-services:
-  db:
-    # Use ParadeDB image which bundles pgvector + pg_search
-    build:
-      context: .
-      dockerfile: Dockerfile
-    container_name: hindsight-db
-    restart: always
-    ports:
-      - "5437:5432"
-    environment:
-      POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
-      POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-      POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
-    volumes:
-      - pg_data:/var/lib/postgresql/data
-    networks:
-      - hindsight-net
-
-  pg-search-init:
-    build:
-      context: .
-      dockerfile: Dockerfile
-    depends_on:
-      - db
-    environment:
-      - PGPASSWORD=${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-    command: >
-      bash -c "
-        echo 'Waiting for PostgreSQL to be ready...';
-        until pg_isready -h hindsight-db -p 5432 -U hindsight_user; do
-          echo 'PostgreSQL is unavailable - sleeping';
-          sleep 2;
-        done;
-        echo 'PostgreSQL is ready - creating hindsight_db database';
-        psql -h hindsight-db -p 5432 -U hindsight_user -c 'CREATE DATABASE hindsight_db;' 2>/dev/null || echo 'Database already exists';
-        echo 'Creating extensions in hindsight_db database';
-        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vector CASCADE;';
-        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS pg_search CASCADE;';
-        echo 'Database and extensions created successfully';
-      "
-    restart: "no"
-    networks:
-      - hindsight-net
-
-  hindsight:
-    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
-    container_name: hindsight-app
-    ports:
-      - "8888:8888"
-      - "9999:9999"
-    environment:
-      # LLM Configuration
-      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
-      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
-
-      # Database Configuration
-      HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
-
-      # Vector and Text Search Extensions
-      HINDSIGHT_API_VECTOR_EXTENSION: pgvector
-      HINDSIGHT_API_TEXT_SEARCH_EXTENSION: pg_search
-      HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER: ${HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER:-}
-
-    depends_on:
-      - db
-    networks:
-      - hindsight-net
-
-
-networks:
-  hindsight-net:
-    driver: bridge
-
-volumes:
-  pg_data:

docker/docker-compose/pg_textsearch/Dockerfile

@@ -1,32 +0,0 @@
-# PostgreSQL with pgvector and pg_textsearch extensions
-# Note: pg_textsearch requires PostgreSQL 17+
-FROM postgres:17
-
-# Install build dependencies
-RUN apt-get update && apt-get install -y \
-    build-essential \
-    git \
-    postgresql-server-dev-17 \
-    libpq-dev \
-    && rm -rf /var/lib/apt/lists/*
-
-# Install pgvector
-RUN cd /tmp && \
-    git clone --branch v0.8.0 https://github.com/pgvector/pgvector.git && \
-    cd pgvector && \
-    make && \
-    make install
-
-# Install pg_textsearch
-RUN cd /tmp && \
-    git clone https://github.com/timescale/pg_textsearch.git && \
-    cd pg_textsearch && \
-    make && \
-    make install
-
-# Clean up source files and build dependencies
-RUN rm -rf /tmp/pgvector /tmp/pg_textsearch && \
-    apt-get purge -y --auto-remove build-essential git postgresql-server-dev-17
-
-# Ensure extensions are preloaded
-RUN echo "shared_preload_libraries = 'pg_textsearch'" >> /usr/share/postgresql/postgresql.conf.sample

docker/docker-compose/pg_textsearch/docker-compose.yaml

@@ -1,91 +0,0 @@
-name: hindsight
-# Docker Compose file for Hindsight with PostgreSQL and Timescale pg_textsearch
-# docker compose -f docker/docker-compose/pg_textsearch/docker-compose.yaml down && sleep 2 && docker compose -f docker/docker-compose/pg_textsearch/docker-compose.yaml up -d
-# Make sure to set the required environment variables before running:
-# - HINDSIGHT_DB_PASSWORD: Password for the PostgreSQL user
-# - Configure LLM provider variables as needed (see below in the hindsight service)
-#
-# Usage:
-# docker compose up -d
-#
-# Optional environment variables with defaults:
-# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
-# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
-# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
-
-services:
-  db:
-    # Use custom PostgreSQL image with pgvector and pg_textsearch extensions
-    build:
-      context: .
-      dockerfile: Dockerfile
-    container_name: hindsight-db
-    restart: always
-    # Expose PostgreSQL port
-    ports:
-      - "5437:5432"
-    environment:
-      POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
-      POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-      POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
-    volumes:
-      - pg_data:/var/lib/postgresql/data
-    networks:
-      - hindsight-net
-
-  pg-textsearch-init:
-    build:
-      context: .
-      dockerfile: Dockerfile
-    depends_on:
-      - db
-    environment:
-      - PGPASSWORD=${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-    command: >
-      bash -c "
-        echo 'Waiting for PostgreSQL to be ready...';
-        until pg_isready -h hindsight-db -p 5432 -U hindsight_user; do
-          echo 'PostgreSQL is unavailable - sleeping';
-          sleep 2;
-        done;
-        echo 'PostgreSQL is ready - creating hindsight_db database';
-        psql -h hindsight-db -p 5432 -U hindsight_user -c 'CREATE DATABASE hindsight_db;' 2>/dev/null || echo 'Database already exists';
-        echo 'Creating extensions in hindsight_db database';
-        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vector CASCADE;';
-        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS pg_textsearch CASCADE;';
-        echo 'Database and extensions created successfully';
-      "
-    restart: "no"
-    networks:
-      - hindsight-net
-
-  hindsight:
-    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
-    container_name: hindsight-app
-    ports:
-      - "8888:8888"
-      - "9999:9999"
-    environment:
-      # LLM Configuration
-      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
-      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
-
-      # Database Configuration
-      HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
-
-      # Vector and Text Search Extensions
-      HINDSIGHT_API_VECTOR_EXTENSION: pgvector
-      HINDSIGHT_API_TEXT_SEARCH_EXTENSION: pg_textsearch
-
-    depends_on:
-      - db
-    networks:
-      - hindsight-net
-
-
-networks:
-  hindsight-net:
-    driver: bridge
-
-volumes:
-  pg_data:

docker/docker-compose/pgroonga/Dockerfile

@@ -1,23 +0,0 @@
-# PostgreSQL with pgvector and pgroonga extensions.
-#
-# pgroonga is a multilingual full-text search extension built on Groonga.
-# It works out of the box for CJK (Chinese, Japanese, Korean) and other
-# non-whitespace-segmented languages via the TokenBigram tokenizer.
-FROM groonga/pgroonga:latest-debian-pg17
-
-# Install pgvector on top of the pgroonga base image (which already provides
-# pgroonga and the Groonga library).
-RUN apt-get update && apt-get install -y --no-install-recommends \
-    build-essential \
-    git \
-    postgresql-server-dev-17 \
-    && rm -rf /var/lib/apt/lists/*
-
-RUN cd /tmp && \
-    git clone --branch v0.8.0 https://github.com/pgvector/pgvector.git && \
-    cd pgvector && \
-    make && \
-    make install
-
-RUN rm -rf /tmp/pgvector && \
-    apt-get purge -y --auto-remove build-essential git postgresql-server-dev-17

docker/docker-compose/pgroonga/docker-compose.yaml

@@ -1,91 +0,0 @@
-name: hindsight
-# Docker Compose file for Hindsight with PostgreSQL and pgroonga
-#
-# pgroonga provides multilingual BM25 indexing that works out of the box for
-# CJK (Chinese, Japanese, Korean) and other non-whitespace-segmented languages.
-# Use this recipe if your bank content is not English/European.
-#
-# docker compose -f docker/docker-compose/pgroonga/docker-compose.yaml down && \
-#   sleep 2 && \
-#   docker compose -f docker/docker-compose/pgroonga/docker-compose.yaml up -d
-#
-# Optional environment variables with defaults:
-# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
-# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
-# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
-# - HINDSIGHT_DB_PASSWORD: PostgreSQL password (default: hindsight_password)
-
-services:
-  db:
-    build:
-      context: .
-      dockerfile: Dockerfile
-    container_name: hindsight-db
-    restart: always
-    ports:
-      - "5439:5432"
-    environment:
-      POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
-      POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-      POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
-    volumes:
-      - pg_data:/var/lib/postgresql/data
-    networks:
-      - hindsight-net
-
-  pgroonga-init:
-    build:
-      context: .
-      dockerfile: Dockerfile
-    depends_on:
-      - db
-    environment:
-      - PGPASSWORD=${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-    command: >
-      bash -c "
-        echo 'Waiting for PostgreSQL to be ready...';
-        until pg_isready -h hindsight-db -p 5432 -U hindsight_user; do
-          echo 'PostgreSQL is unavailable - sleeping';
-          sleep 2;
-        done;
-        echo 'PostgreSQL is ready - creating hindsight_db database';
-        psql -h hindsight-db -p 5432 -U hindsight_user -c 'CREATE DATABASE hindsight_db;' 2>/dev/null || echo 'Database already exists';
-        echo 'Creating extensions in hindsight_db database';
-        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vector CASCADE;';
-        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS pgroonga CASCADE;';
-        echo 'Database and extensions created successfully';
-      "
-    restart: "no"
-    networks:
-      - hindsight-net
-
-  hindsight:
-    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
-    container_name: hindsight-app
-    ports:
-      - "8888:8888"
-      - "9999:9999"
-    environment:
-      # LLM Configuration
-      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
-      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
-
-      # Database Configuration
-      HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
-
-      # Vector and Text Search Extensions
-      HINDSIGHT_API_VECTOR_EXTENSION: pgvector
-      HINDSIGHT_API_TEXT_SEARCH_EXTENSION: pgroonga
-
-    depends_on:
-      - db
-    networks:
-      - hindsight-net
-
-
-networks:
-  hindsight-net:
-    driver: bridge
-
-volumes:
-  pg_data:

docker/docker-compose/s3-file-storage/docker-compose.yaml

@@ -1,83 +0,0 @@
-# Docker Compose file for Hindsight with S3 file storage (SeaweedFS)
-#
-# SeaweedFS (Apache 2.0) provides an S3-compatible object storage backend
-# for storing uploaded files instead of PostgreSQL BYTEA storage.
-#
-# Make sure to set the required environment variables before running:
-# - HINDSIGHT_DB_PASSWORD: Password for the PostgreSQL user
-# - Configure LLM provider variables as needed (see below in the hindsight service)
-#
-# Usage:
-# docker compose up -d
-#
-# Optional environment variables with defaults:
-# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
-# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
-# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
-# - HINDSIGHT_DB_VERSION: PostgreSQL version (default: 18)
-# - SEAWEEDFS_S3_ACCESS_KEY: S3 access key (default: hindsight_s3_key)
-# - SEAWEEDFS_S3_SECRET_KEY: S3 secret key (default: hindsight_s3_secret)
-
-services:
-  db:
-    image: pgvector/pgvector:pg${HINDSIGHT_DB_VERSION:-18}
-    container_name: hindsight-db
-    restart: always
-    environment:
-      POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
-      POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:?Please set the HINDSIGHT_DB_PASSWORD env variable}
-      POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
-    volumes:
-      - pg_data:/var/lib/postgresql/${HINDSIGHT_DB_VERSION:-18}/docker
-    networks:
-      - hindsight-net
-
-  seaweedfs:
-    image: chrislusf/seaweedfs:latest
-    container_name: hindsight-seaweedfs
-    restart: always
-    # Single-node mode: master + volume + filer + S3 gateway all in one process
-    command: >
-      server
-      -s3
-      -s3.port=8333
-      -s3.config=/etc/seaweedfs/s3.json
-      -ip.bind=0.0.0.0
-    volumes:
-      - seaweedfs_data:/data
-      - ./s3.json:/etc/seaweedfs/s3.json:ro
-    # Expose S3 API port (uncomment to access from host)
-    # ports:
-    #   - "8333:8333"
-    networks:
-      - hindsight-net
-
-  hindsight:
-    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
-    container_name: hindsight-app
-    ports:
-      - "8888:8888"
-      - "9999:9999"
-    environment:
-      - HINDSIGHT_API_LLM_API_KEY=${OPENAI_API_KEY?Please set the OPENAI_API_KEY env variable}
-      - HINDSIGHT_API_DATABASE_URL=postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:?Please set the HINDSIGHT_DB_PASSWORD env variable}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
-      # S3 file storage configuration (SeaweedFS)
-      - HINDSIGHT_API_FILE_STORAGE_TYPE=s3
-      - HINDSIGHT_API_FILE_STORAGE_S3_BUCKET=hindsight
-      - HINDSIGHT_API_FILE_STORAGE_S3_ENDPOINT=http://seaweedfs:8333
-      - HINDSIGHT_API_FILE_STORAGE_S3_REGION=us-east-1
-      - HINDSIGHT_API_FILE_STORAGE_S3_ACCESS_KEY_ID=${SEAWEEDFS_S3_ACCESS_KEY:-hindsight_s3_key}
-      - HINDSIGHT_API_FILE_STORAGE_S3_SECRET_ACCESS_KEY=${SEAWEEDFS_S3_SECRET_KEY:-hindsight_s3_secret}
-    depends_on:
-      - db
-      - seaweedfs
-    networks:
-      - hindsight-net
-
-networks:
-  hindsight-net:
-    driver: bridge
-
-volumes:
-  pg_data:
-  seaweedfs_data:

docker/docker-compose/s3-file-storage/s3.json

@@ -1,19 +0,0 @@
-{
-  "identities": [
-    {
-      "name": "hindsight",
-      "credentials": [
-        {
-          "accessKey": "hindsight_s3_key",
-          "secretKey": "hindsight_s3_secret"
-        }
-      ],
-      "actions": [
-        "Admin",
-        "Read",
-        "Write",
-        "List"
-      ]
-    }
-  ]
-}

docker/docker-compose/timescale/.dockerignore

@@ -1,16 +0,0 @@
-# Git
-.git
-.gitignore
-.gitattributes
-
-# Docker
-docker-compose.yaml
-.dockerignore
-
-# Documentation
-README.md
-*.md
-
-# Environment
-.env
-.env.example

docker/docker-compose/timescale/.env.example

@@ -1,25 +0,0 @@
-# PostgreSQL Configuration
-HINDSIGHT_DB_USER=hindsight_user
-HINDSIGHT_DB_PASSWORD=change-me-to-secure-password
-HINDSIGHT_DB_NAME=hindsight_db
-
-# Hindsight Version
-HINDSIGHT_VERSION=latest
-
-# LLM Configuration
-HINDSIGHT_API_LLM_PROVIDER=openai
-OPENAI_API_KEY=your-openai-api-key-here
-
-# Alternative LLM providers (uncomment and configure as needed):
-# HINDSIGHT_API_LLM_PROVIDER=anthropic
-# ANTHROPIC_API_KEY=your-anthropic-api-key
-
-# HINDSIGHT_API_LLM_PROVIDER=gemini
-# GEMINI_API_KEY=your-gemini-api-key
-
-# HINDSIGHT_API_LLM_PROVIDER=groq
-# GROQ_API_KEY=your-groq-api-key
-
-# Vector and Text Search (already configured in docker-compose.yaml)
-# HINDSIGHT_API_VECTOR_EXTENSION=pgvectorscale
-# HINDSIGHT_API_TEXT_SEARCH_EXTENSION=pg_textsearch

docker/docker-compose/timescale/Dockerfile

@@ -1,55 +0,0 @@
-# PostgreSQL with pgvector, pgvectorscale, and pg_textsearch extensions
-# All three extensions from Timescale/pgvector for high-performance vector and text search
-# Note: Requires PostgreSQL 16+
-FROM postgres:17
-
-# Install build dependencies and Rust toolchain
-RUN apt-get update && apt-get install -y \
-    build-essential \
-    git \
-    postgresql-server-dev-17 \
-    libpq-dev \
-    cmake \
-    curl \
-    pkg-config \
-    libssl-dev \
-    && rm -rf /var/lib/apt/lists/*
-
-# Install Rust toolchain (required for pgvectorscale)
-RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
-ENV PATH="/root/.cargo/bin:${PATH}"
-
-# Install pgvector (required by pgvectorscale)
-RUN cd /tmp && \
-    git clone --branch v0.8.0 https://github.com/pgvector/pgvector.git && \
-    cd pgvector && \
-    make && \
-    make install && \
-    rm -rf /tmp/pgvector
-
-# Install cargo-pgrx (PostgreSQL extension framework for Rust)
-RUN cargo install cargo-pgrx --version 0.12.5 --locked && \
-    cargo pgrx init --pg17 /usr/bin/pg_config
-
-# Install pgvectorscale (DiskANN index support)
-RUN cd /tmp && \
-    git clone --branch 0.5.1 https://github.com/timescale/pgvectorscale.git && \
-    cd pgvectorscale/pgvectorscale && \
-    cargo pgrx install --release && \
-    rm -rf /tmp/pgvectorscale
-
-# Install pg_textsearch (BM25 text search)
-RUN cd /tmp && \
-    git clone https://github.com/timescale/pg_textsearch.git && \
-    cd pg_textsearch && \
-    make && \
-    make install && \
-    rm -rf /tmp/pg_textsearch
-
-# Clean up build dependencies (keep runtime dependencies)
-RUN apt-get purge -y --auto-remove git cmake curl && \
-    rm -rf /root/.cargo/registry /root/.cargo/git
-
-# Ensure extensions are preloaded (pg_textsearch requires preloading)
-RUN echo "shared_preload_libraries = 'pg_textsearch'" >> /usr/share/postgresql/postgresql.conf.sample
-

docker/docker-compose/timescale/README.md

@@ -1,101 +0,0 @@
-# Hindsight with Timescale Extensions
-
-This Docker Compose setup provides a complete Hindsight deployment with **Timescale extensions**:
-- **pgvectorscale** - DiskANN algorithm for disk-based scalable vector search  
-- **pg_textsearch** - High-performance BM25 text search
-
-Both extensions are from [Timescale](https://github.com/timescale) and provide production-grade performance.
-
-## Prerequisites
-
-- Docker and Docker Compose installed
-- OpenAI API key (or another LLM provider)
-
-## Quick Start
-
-```bash
-# Set environment variables
-export HINDSIGHT_DB_PASSWORD="your-secure-password"
-export OPENAI_API_KEY="your-openai-api-key"
-
-# Build and start
-docker compose -f docker/docker-compose/timescale/docker-compose.yaml up -d --build
-
-# Check logs
-
-docker compose -f docker/docker-compose/timescale/docker-compose.yaml logs -f
-```
-
-**Access:**
-- API: http://localhost:8888
-- Control Plane: http://localhost:9999
-
-## Stop and Clean Up
-
-```bash
-# Stop services
-docker compose -f docker/docker-compose/timescale/docker-compose.yaml down
-
-# Remove volumes (deletes all data)
-docker compose -f docker/docker-compose/timescale/docker-compose.yaml down -v
-```
-
-## Configuration
-
-### Environment Variables
-
-| Variable | Description | Default |
-|----------|-------------|---------|
-| `HINDSIGHT_DB_PASSWORD` | PostgreSQL password | `hindsight_password` |
-| `HINDSIGHT_DB_USER` | PostgreSQL username | `hindsight_user` |
-| `HINDSIGHT_DB_NAME` | Database name | `hindsight_db` |
-| `HINDSIGHT_VERSION` | Hindsight Docker image version | `latest` |
-| `OPENAI_API_KEY` | OpenAI API key | (required) |
-| `HINDSIGHT_API_LLM_PROVIDER` | LLM provider | `openai` |
-
-### Why Timescale Extensions?
-
-**pgvectorscale (DiskANN):**
-- 28x lower p95 latency vs dedicated vector databases
-- 16x higher query throughput at 99% recall
-- 60-75% cost reduction (disk is cheaper than RAM)
-- Best for large datasets (10M+ vectors)
-
-**pg_textsearch (BM25):**
-- High-performance keyword retrieval
-- Native BM25 ranking algorithm
-- Optimized for full-text search
-
-## Troubleshooting
-
-### Extensions not installed
-
-Check if extensions are available:
-
-```bash
-docker exec -it hindsight-db-timescale psql -U hindsight_user -d hindsight_db -c "\dx"
-```
-
-You should see:
-- `vector` (pgvector)
-- `vectorscale` (pgvectorscale/DiskANN)
-- `pg_textsearch` (BM25 search)
-
-### Build fails
-
-If the Docker build fails during pgvectorscale compilation:
-
-1. Ensure you have sufficient memory (recommended: 4GB+)
-2. Check Docker build logs for Rust compilation errors
-3. Try building with more resources: `docker compose build --no-cache --memory 4g`
-
-### Port conflicts
-
-If port 5438 is already in use, modify the `ports` section in docker-compose.yaml.
-
-## Learn More
-
-- [pgvectorscale GitHub](https://github.com/timescale/pgvectorscale)
-- [pg_textsearch GitHub](https://github.com/timescale/pg_textsearch)
-- [HNSW vs DiskANN](https://www.tigerdata.com/learn/hnsw-vs-diskann)
-- [Hindsight Documentation](https://hindsight.dev)

docker/docker-compose/timescale/docker-compose.yaml

@@ -1,108 +0,0 @@
-name: hindsight
-# Docker Compose file for Hindsight with Timescale extensions
-# - pgvectorscale: DiskANN vector search (disk-based, scalable)
-# - pg_textsearch: BM25 text search (high-performance keyword retrieval)
-#
-# Quick start:
-# docker compose -f docker/docker-compose/timescale/docker-compose.yaml up -d --build
-#
-# Required environment variables:
-# - HINDSIGHT_DB_PASSWORD: Password for the PostgreSQL user
-# - OPENAI_API_KEY (or configure another LLM provider)
-#
-# Optional environment variables with defaults:
-# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
-# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
-# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
-
-services:
-  db:
-    # Custom PostgreSQL image with Timescale extensions (pgvectorscale + pg_textsearch)
-    build:
-      context: .
-      dockerfile: Dockerfile
-    container_name: hindsight-db-timescale
-    restart: always
-    # Expose PostgreSQL port (using 5438 to avoid conflicts with other setups)
-    ports:
-      - "5438:5432"
-    environment:
-      POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
-      POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-      POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
-    volumes:
-      - pg_data:/var/lib/postgresql/data
-    networks:
-      - hindsight-net
-    # Health check to ensure database is ready
-    healthcheck:
-      test: ["CMD-SHELL", "pg_isready -U hindsight_user"]
-      interval: 5s
-      timeout: 5s
-      retries: 5
-
-  timescale-init:
-    build:
-      context: .
-      dockerfile: Dockerfile
-    depends_on:
-      db:
-        condition: service_healthy
-    environment:
-      - PGPASSWORD=${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-    command: >
-      bash -c "
-        echo 'PostgreSQL is ready - creating hindsight_db database';
-        psql -h hindsight-db-timescale -p 5432 -U hindsight_user -c 'CREATE DATABASE hindsight_db;' 2>/dev/null || echo 'Database already exists';
-        echo 'Installing Timescale extensions...';
-        echo '1/3: Installing pgvector (required by pgvectorscale)...';
-        psql -h hindsight-db-timescale -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vector CASCADE;';
-        echo '2/3: Installing pgvectorscale (DiskANN vector search)...';
-        psql -h hindsight-db-timescale -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vectorscale CASCADE;';
-        echo '3/3: Installing pg_textsearch (BM25 text search)...';
-        psql -h hindsight-db-timescale -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS pg_textsearch CASCADE;';
-        echo '';
-        echo '✅ Timescale extensions installed successfully';
-        echo '';
-        echo 'Installed extensions:';
-        psql -h hindsight-db-timescale -p 5432 -U hindsight_user -d hindsight_db -c \"\\dx\" | grep -E '(vector|vectorscale|pg_textsearch)';
-      "
-    restart: "no"
-    networks:
-      - hindsight-net
-
-  hindsight:
-    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
-    container_name: hindsight-app-timescale
-    ports:
-      - "8888:8888"
-      - "9999:9999"
-    environment:
-      # LLM Configuration
-      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
-      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
-
-      # Database Configuration
-      HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
-
-      # Timescale Extensions
-      # pgvectorscale: DiskANN algorithm for disk-based scalable vector search
-      HINDSIGHT_API_VECTOR_EXTENSION: pgvectorscale
-      # pg_textsearch: High-performance BM25 text search
-      HINDSIGHT_API_TEXT_SEARCH_EXTENSION: pg_textsearch
-
-    depends_on:
-      db:
-        condition: service_healthy
-      timescale-init:
-        condition: service_completed_successfully
-    networks:
-      - hindsight-net
-
-
-networks:
-  hindsight-net:
-    driver: bridge
-
-volumes:
-  pg_data:

docker/docker-compose/vchord/docker-compose.yaml

@@ -1,93 +0,0 @@
-name: hindsight
-# Docker Compose file for Hindsight with PostgreSQL and vectorchord
-# docker compose -f docker/docker-compose/vchord/docker-compose.yaml down && sleep 2 && docker compose -f docker/docker-compose/vchord/docker-compose.yaml up -d
-# Make sure to set the required environment variables before running:
-# - HINDSIGHT_DB_PASSWORD: Password for the PostgreSQL user
-# - Configure LLM provider variables as needed (see below in the hindsight service)
-#
-# Usage:
-# docker compose up -d
-#
-# Optional environment variables with defaults:
-# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
-# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
-# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
-# - HINDSIGHT_DB_VERSION: PostgreSQL version (default: 18)
-
-services:
-  db:
-    # Use a PostgreSQL-Image with vectorchord extension pre-installed
-    image: tensorchord/vchord-suite:pg${HINDSIGHT_DB_VERSION:-18-latest}
-    container_name: hindsight-db
-    restart: always
-    # Expose PostgreSQL port
-    ports:
-      - "5436:5432"
-    environment:
-      POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
-      POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-      POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
-    volumes:
-      - pg_data:/var/lib/postgresql/${HINDSIGHT_DB_VERSION:-18}/docker
-    networks:
-      - hindsight-net
-
-  vectorchord-init:
-    image: tensorchord/vchord-suite:pg18-latest
-    #container_name: vectorchord-init
-    depends_on:
-      - db
-    environment:
-      - PGPASSWORD=${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-    command: >
-      bash -c "
-        echo 'Waiting for PostgreSQL to be ready...';
-        until pg_isready -h hindsight-db -p 5432 -U hindsight_user; do
-          echo 'PostgreSQL is unavailable - sleeping';
-          sleep 2;
-        done;
-        echo 'PostgreSQL is ready - creating hindsight_db database';
-        psql -h hindsight-db -p 5432 -U hindsight_user -c 'CREATE DATABASE hindsight_db;' 2>/dev/null || echo 'Database already exists';
-        echo 'Creating extensions in hindsight_db database';
-        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vchord CASCADE;';
-        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS pg_tokenizer CASCADE;';
-        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vchord_bm25 CASCADE;';
-        echo 'Creating llmlingua2 tokenizer';
-        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c \"SELECT create_tokenizer('llmlingua2', \\$\\$ model = \\\"llmlingua2\\\" \\$\\$);\" 2>/dev/null || echo 'Tokenizer already exists or creation skipped';
-        echo 'Database and extensions created successfully';
-      "
-    restart: "no"
-    networks:
-      - hindsight-net
-
-  hindsight:
-    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
-    container_name: hindsight-app
-    ports:
-      - "8888:8888"
-      - "9999:9999"
-    environment:
-      # LLM Configuration (uses OpenAI for testing vchord)
-      # LLM configuration
-      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
-      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
-
-      # Database Configuration
-      HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
-
-      # Vector and Text Search Extensions
-      HINDSIGHT_API_VECTOR_EXTENSION: vchord
-      HINDSIGHT_API_TEXT_SEARCH_EXTENSION: vchord
-
-    depends_on:
-      - db
-    networks:
-      - hindsight-net
-
-
-networks:
-  hindsight-net:
-    driver: bridge
-
-volumes:
-  pg_data:

docker/standalone/Dockerfile

@@ -2,25 +2,19 @@
 # Supports building API-only, Control Plane-only, or both
 #
 # Build args:
-#   INCLUDE_API=true/false         - Include API (default: true)
-#   INCLUDE_CP=true/false          - Include Control Plane (default: true)
-#   INCLUDE_LOCAL_MODELS=true/false - Include local ML models for embeddings/reranking (default: true)
-#                                     Set to false when using external providers (TEI, OpenAI, Cohere)
-#   PRELOAD_ML_MODELS=true/false   - Pre-download ML models during build (default: true)
-#                                     Only effective when INCLUDE_LOCAL_MODELS=true
-#                                     NOTE: tiktoken encodings are ALWAYS preloaded (required for air-gapped deployments)
+#   INCLUDE_API=true/false       - Include API (default: true)
+#   INCLUDE_CP=true/false        - Include Control Plane (default: true)
+#   PRELOAD_ML_MODELS=true/false - Pre-download ML models during build (default: true)
 #
 # Examples:
 #   docker build -t hindsight .                                          # Both (standalone)
 #   docker build -t hindsight-api --build-arg INCLUDE_CP=false .         # API only
 #   docker build -t hindsight-cp --build-arg INCLUDE_API=false .         # Control Plane only
 #   docker build -t hindsight --build-arg PRELOAD_ML_MODELS=false .      # Skip ML model preload
-#   docker build -t hindsight --build-arg INCLUDE_LOCAL_MODELS=false .   # Skip local ML deps (for external providers)
 
 ARG INCLUDE_API=true
 ARG INCLUDE_CP=true
 ARG PRELOAD_ML_MODELS=true
-ARG INCLUDE_LOCAL_MODELS=true
 
 # =============================================================================
 # Stage: API Builder
@@ -28,7 +22,6 @@ ARG INCLUDE_LOCAL_MODELS=true
 FROM python:3.11-slim AS api-builder
 
 ARG INCLUDE_API
-ARG INCLUDE_LOCAL_MODELS
 RUN if [ "$INCLUDE_API" != "true" ]; then echo "Skipping API build" && exit 0; fi
 
 WORKDIR /app
@@ -42,24 +35,16 @@ RUN apt-get update && apt-get install -y \
     && pip install --no-cache-dir uv
 
 # Copy dependency files and README (required by pyproject.toml)
-COPY hindsight-api-slim/pyproject.toml ./api/
-COPY hindsight-api-slim/README.md ./api/
+COPY hindsight-api/pyproject.toml ./api/
+COPY hindsight-api/README.md ./api/
 
 WORKDIR /app/api
 
-# Sync dependencies using appropriate extras based on INCLUDE_LOCAL_MODELS
-# local-ml: torch, sentence-transformers, transformers, einops, flashrank, mlx (optional)
-# embedded-db: pg0-embedded (always included for embedded PostgreSQL support)
-# ONNX Runtime embeddings are intentionally not bundled into the official
-# standalone image; install the local-onnx extra in custom images when needed.
-RUN if [ "$INCLUDE_LOCAL_MODELS" = "true" ]; then \
-        uv sync --extra local-ml --extra embedded-db; \
-    else \
-        uv sync --extra embedded-db; \
-    fi
+# Sync dependencies (will create lock file if needed)
+RUN uv sync
 
 # Copy source code (alembic migrations are inside hindsight_api/)
-COPY hindsight-api-slim/hindsight_api ./hindsight_api
+COPY hindsight-api/hindsight_api ./hindsight_api
 
 # Install the local package (uv sync only installed dependencies, not the package itself)
 RUN uv pip install -e .
@@ -111,10 +96,6 @@ RUN rm -f package-lock.json && sed -i '/"@vectorize-io\/hindsight-client":/d' pa
 # Copy built SDK directly into node_modules (more reliable than npm link in Docker)
 COPY --from=sdk-builder /app/hindsight-clients/typescript ./node_modules/@vectorize-io/hindsight-client
 
-# Accept base path as build argument for reverse proxy deployments
-# Usage: docker build --build-arg NEXT_PUBLIC_BASE_PATH=/hindsight ...
-ARG NEXT_PUBLIC_BASE_PATH=""
-
 # Build Control Plane - run next build first, then custom standalone copy
 # (The build:standalone script expects a specific path structure that differs in Docker)
 RUN npm exec -- next build
@@ -169,69 +150,18 @@ RUN chown -R hindsight:hindsight /app
 
 USER hindsight
 
-# Create pg0 data directory as hindsight user so that Docker seeds new named
-# volumes with correct ownership (UID 1000) on first use, avoiding the
-# "Permission denied" error when mounting a fresh root-owned volume.
-RUN mkdir -p /home/hindsight/.pg0
-
-# Make /home/hindsight traversable when running with --user UID:GID overrides
-# (default 0700 blocks traversal by non-owner UIDs needed for bind-mount ownership matching)
-RUN chmod 755 /home/hindsight
-
 ENV PATH="/app/api/.venv/bin:${PATH}"
 
-# Pre-download tiktoken encoding (ALWAYS - required for token counting even in air-gapped envs)
-# Tiktoken is a core runtime dependency, not an optional ML model
-RUN MAX_RETRIES=3; \
-    RETRY_DELAY=5; \
-    for i in $(seq 1 $MAX_RETRIES); do \
-      echo "Attempt $i/$MAX_RETRIES: Downloading tiktoken encoding..."; \
-      /app/api/.venv/bin/python -c "\
-import tiktoken; \
-print('Downloading cl100k_base encoding...'); \
-tiktoken.get_encoding('cl100k_base'); \
-print('Tiktoken encoding cached successfully')" && break; \
-      if [ $i -lt $MAX_RETRIES ]; then \
-        echo "Attempt $i failed, retrying in ${RETRY_DELAY}s..."; \
-        sleep $RETRY_DELAY; \
-        RETRY_DELAY=$((RETRY_DELAY * 2)); \
-      fi; \
-    done; \
-    if [ $i -eq $MAX_RETRIES ]; then \
-      echo "ERROR: Failed to download tiktoken encoding after $MAX_RETRIES attempts"; \
-      exit 1; \
-    fi
-
 # Pre-download ML models to avoid runtime download (conditional)
-# Only runs if both PRELOAD_ML_MODELS=true AND INCLUDE_LOCAL_MODELS=true
-# Includes retry logic with exponential backoff for transient network failures
 ARG PRELOAD_ML_MODELS
-ARG INCLUDE_LOCAL_MODELS
-ENV HF_HUB_DOWNLOAD_TIMEOUT=600
-RUN if [ "$PRELOAD_ML_MODELS" = "true" ] && [ "$INCLUDE_LOCAL_MODELS" = "true" ]; then \
-    MAX_RETRIES=3; \
-    RETRY_DELAY=10; \
-    for i in $(seq 1 $MAX_RETRIES); do \
-      echo "Attempt $i/$MAX_RETRIES: Downloading ML models..."; \
-      /app/api/.venv/bin/python -c "\
-import os; os.environ['HF_HUB_DOWNLOAD_TIMEOUT'] = '600'; \
+RUN if [ "$PRELOAD_ML_MODELS" = "true" ]; then \
+    /app/api/.venv/bin/python -c "\
 from sentence_transformers import SentenceTransformer, CrossEncoder; \
 print('Downloading embedding model...'); \
 SentenceTransformer('BAAI/bge-small-en-v1.5'); \
 print('Downloading cross-encoder model...'); \
 CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2'); \
-print('Models cached successfully')" && break; \
-      if [ $i -lt $MAX_RETRIES ]; then \
-        echo "Attempt $i failed, retrying in ${RETRY_DELAY}s..."; \
-        sleep $RETRY_DELAY; \
-        RETRY_DELAY=$((RETRY_DELAY * 2)); \
-      fi; \
-    done; \
-    if [ $i -eq $MAX_RETRIES ] && ! /app/api/.venv/bin/python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('BAAI/bge-small-en-v1.5')" 2>/dev/null; then \
-      echo "ERROR: Failed to download models after $MAX_RETRIES attempts"; \
-      exit 1; \
-    fi; \
-    elif [ "$INCLUDE_LOCAL_MODELS" != "true" ]; then echo "Skipping ML model preload (local-models not included)"; \
+print('Models cached successfully')"; \
     else echo "Skipping ML model preload"; fi
 
 EXPOSE 8888
@@ -242,10 +172,6 @@ ENV HINDSIGHT_API_LOG_LEVEL=info
 ENV HINDSIGHT_ENABLE_API=true
 ENV HINDSIGHT_ENABLE_CP=false
 ENV PYTHONUNBUFFERED=1
-# Suppress verbose transformers/HuggingFace model loading warnings
-ENV TRANSFORMERS_VERBOSITY=error
-ENV HF_HUB_VERBOSITY=error
-ENV TOKENIZERS_PARALLELISM=false
 
 CMD ["/app/start-all.sh"]
 
@@ -329,69 +255,18 @@ RUN chown -R hindsight:hindsight /app
 
 USER hindsight
 
-# Create pg0 data directory as hindsight user so that Docker seeds new named
-# volumes with correct ownership (UID 1000) on first use, avoiding the
-# "Permission denied" error when mounting a fresh root-owned volume.
-RUN mkdir -p /home/hindsight/.pg0
-
-# Make /home/hindsight traversable when running with --user UID:GID overrides
-# (default 0700 blocks traversal by non-owner UIDs needed for bind-mount ownership matching)
-RUN chmod 755 /home/hindsight
-
 ENV PATH="/app/api/.venv/bin:${PATH}"
 
-# Pre-download tiktoken encoding (ALWAYS - required for token counting even in air-gapped envs)
-# Tiktoken is a core runtime dependency, not an optional ML model
-RUN MAX_RETRIES=3; \
-    RETRY_DELAY=5; \
-    for i in $(seq 1 $MAX_RETRIES); do \
-      echo "Attempt $i/$MAX_RETRIES: Downloading tiktoken encoding..."; \
-      /app/api/.venv/bin/python -c "\
-import tiktoken; \
-print('Downloading cl100k_base encoding...'); \
-tiktoken.get_encoding('cl100k_base'); \
-print('Tiktoken encoding cached successfully')" && break; \
-      if [ $i -lt $MAX_RETRIES ]; then \
-        echo "Attempt $i failed, retrying in ${RETRY_DELAY}s..."; \
-        sleep $RETRY_DELAY; \
-        RETRY_DELAY=$((RETRY_DELAY * 2)); \
-      fi; \
-    done; \
-    if [ $i -eq $MAX_RETRIES ]; then \
-      echo "ERROR: Failed to download tiktoken encoding after $MAX_RETRIES attempts"; \
-      exit 1; \
-    fi
-
 # Pre-download ML models to avoid runtime download (conditional)
-# Only runs if both PRELOAD_ML_MODELS=true AND INCLUDE_LOCAL_MODELS=true
-# Includes retry logic with exponential backoff for transient network failures
 ARG PRELOAD_ML_MODELS
-ARG INCLUDE_LOCAL_MODELS
-ENV HF_HUB_DOWNLOAD_TIMEOUT=600
-RUN if [ "$PRELOAD_ML_MODELS" = "true" ] && [ "$INCLUDE_LOCAL_MODELS" = "true" ]; then \
-    MAX_RETRIES=3; \
-    RETRY_DELAY=10; \
-    for i in $(seq 1 $MAX_RETRIES); do \
-      echo "Attempt $i/$MAX_RETRIES: Downloading ML models..."; \
-      /app/api/.venv/bin/python -c "\
-import os; os.environ['HF_HUB_DOWNLOAD_TIMEOUT'] = '600'; \
+RUN if [ "$PRELOAD_ML_MODELS" = "true" ]; then \
+    /app/api/.venv/bin/python -c "\
 from sentence_transformers import SentenceTransformer, CrossEncoder; \
 print('Downloading embedding model...'); \
 SentenceTransformer('BAAI/bge-small-en-v1.5'); \
 print('Downloading cross-encoder model...'); \
 CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2'); \
-print('Models cached successfully')" && break; \
-      if [ $i -lt $MAX_RETRIES ]; then \
-        echo "Attempt $i failed, retrying in ${RETRY_DELAY}s..."; \
-        sleep $RETRY_DELAY; \
-        RETRY_DELAY=$((RETRY_DELAY * 2)); \
-      fi; \
-    done; \
-    if [ $i -eq $MAX_RETRIES ] && ! /app/api/.venv/bin/python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('BAAI/bge-small-en-v1.5')" 2>/dev/null; then \
-      echo "ERROR: Failed to download models after $MAX_RETRIES attempts"; \
-      exit 1; \
-    fi; \
-    elif [ "$INCLUDE_LOCAL_MODELS" != "true" ]; then echo "Skipping ML model preload (local-models not included)"; \
+print('Models cached successfully')"; \
     else echo "Skipping ML model preload"; fi
 
 EXPOSE 8888 9999
@@ -404,10 +279,6 @@ ENV HINDSIGHT_CP_DATAPLANE_API_URL=http://localhost:8888
 ENV HINDSIGHT_ENABLE_API=true
 ENV HINDSIGHT_ENABLE_CP=true
 ENV PYTHONUNBUFFERED=1
-# Suppress verbose transformers/HuggingFace model loading warnings
-ENV TRANSFORMERS_VERBOSITY=error
-ENV HF_HUB_VERBOSITY=error
-ENV TOKENIZERS_PARALLELISM=false
 
 CMD ["/app/start-all.sh"]
 

docker/standalone/start-all.sh

@@ -1,99 +1,6 @@
 #!/bin/bash
 set -e
 
-# =============================================================================
-# Embedded pg0 data integrity check (#675)
-#
-# When using embedded pg0, check if the data directory has existing PostgreSQL
-# data before starting. If the directory exists but appears empty/corrupt
-# (e.g., missing PG_VERSION file), log a warning. This helps diagnose data
-# loss scenarios where a container restart caused the data directory to be
-# wiped despite a volume mount being present.
-# =============================================================================
-pg0_has_pg_version() {
-    local pg0_data_dir="$1"
-
-    # pg0 has used more than one on-disk layout. Newer standalone images keep
-    # PostgreSQL data under instances/<name>/data, while older volumes may have
-    # placed PG_VERSION at or one level below the mount.
-    [ -f "$pg0_data_dir/PG_VERSION" ] && return 0
-    compgen -G "$pg0_data_dir"/*/PG_VERSION > /dev/null 2>&1 && return 0
-    compgen -G "$pg0_data_dir"/instances/*/data/PG_VERSION > /dev/null 2>&1 && return 0
-
-    return 1
-}
-
-check_pg0_data_integrity() {
-    local pg0_data_dir="$1"
-
-    if [ ! -d "$pg0_data_dir" ]; then
-        return 0
-    fi
-
-    # Look for actual PostgreSQL data directories (pg0 creates subdirs per instance)
-    if pg0_has_pg_version "$pg0_data_dir"; then
-        echo "✅ Existing pg0 data directory detected at $pg0_data_dir"
-    elif [ "$(ls -A "$pg0_data_dir" 2>/dev/null)" ]; then
-        echo "⚠️  WARNING: pg0 data directory exists at $pg0_data_dir but no PG_VERSION found."
-        echo "   This may indicate data corruption or an incomplete previous shutdown."
-        echo "   If you see all migrations running from scratch after this, your data may have been lost."
-        echo "   See: https://github.com/vectorize-io/hindsight/issues/675"
-    fi
-
-    return 0
-}
-
-# =============================================================================
-# Embedded pg0 writability pre-check (#1483)
-#
-# The container runs as the unprivileged `hindsight` user (UID 1000). When the
-# pg0 data directory is a host bind mount (e.g. `-v $HOME/dir:/home/hindsight/.pg0`)
-# that is not owned by UID 1000 — the default on macOS Docker Desktop and most
-# non-1000 Linux hosts — pg0 fails with the opaque "Permission denied (os error
-# 13)". We cannot chown it ourselves without root (and the image is deliberately
-# rootless), so we surface an actionable message up front instead.
-#
-# Docker *named* volumes are seeded with the image directory's ownership (UID
-# 1000) on first use, so they avoid this entirely — hence the named-volume
-# recommendation below and in the README.
-# =============================================================================
-check_pg0_writable() {
-    local pg0_data_dir="$1"
-
-    # Only relevant for embedded pg0; an external database doesn't use this dir.
-    if [ -n "${HINDSIGHT_API_DATABASE_URL:-}" ]; then
-        return 0
-    fi
-
-    mkdir -p "$pg0_data_dir" 2>/dev/null || true
-    if touch "$pg0_data_dir/.hindsight-write-test" 2>/dev/null; then
-        rm -f "$pg0_data_dir/.hindsight-write-test" 2>/dev/null || true
-        return 0
-    fi
-
-    echo "❌ The embedded database directory $pg0_data_dir is not writable by this container (UID $(id -u))."
-    echo ""
-    echo "   A host directory was bind-mounted but is not owned by the container user (UID 1000)."
-    echo "   Hindsight runs rootless and cannot fix this for you. Choose one:"
-    echo ""
-    echo "   • Recommended — use a Docker named volume (auto-owned by the container):"
-    echo "       -v hindsight-data:/home/hindsight/.pg0"
-    echo ""
-    echo "   • Or keep the host path and run as your host user, chowning it to match:"
-    echo "       sudo chown -R \$(id -u):\$(id -g) <host-directory>"
-    echo "       docker run --user \$(id -u):\$(id -g) -e HOME=/home/hindsight ..."
-    echo ""
-    echo "   See https://github.com/vectorize-io/hindsight/issues/1483"
-    return 1
-}
-
-if [ "${HINDSIGHT_START_ALL_SOURCE_ONLY:-false}" = "true" ]; then
-    return 0 2>/dev/null || exit 0
-fi
-
-check_pg0_data_integrity "${HOME}/.pg0"
-check_pg0_writable "${HOME}/.pg0" || exit 1
-
 # Service flags (default to true if not set)
 ENABLE_API="${HINDSIGHT_ENABLE_API:-true}"
 ENABLE_CP="${HINDSIGHT_ENABLE_CP:-true}"
@@ -164,95 +71,24 @@ if [ "${HINDSIGHT_WAIT_FOR_DEPS:-false}" = "true" ]; then
     done
 fi
 
-# =============================================================================
-# Graceful shutdown handler (#675)
-#
-# Docker sends SIGTERM on `docker stop`/`docker restart`. Without a trap, child
-# processes (hindsight-api + pg0, control-plane) are killed abruptly. For the
-# embedded pg0 database this can cause data loss when the data directory is on
-# a Docker volume that gets remounted after restart.
-#
-# The trap forwards SIGTERM to all tracked child PIDs so that:
-#   - hindsight-api receives the signal and can run its shutdown hooks
-#   - pg0 gets a clean PostgreSQL shutdown (checkpoint + WAL flush)
-#   - The control-plane Node.js process exits cleanly
-# =============================================================================
-# Guard against concurrent cleanup (e.g., child crash + SIGTERM arriving together)
-SHUTTING_DOWN=false
-
-cleanup() {
-    if $SHUTTING_DOWN; then return; fi
-    SHUTTING_DOWN=true
-
-    echo ""
-    echo "🛑 Received shutdown signal, stopping services gracefully..."
-    for pid in "${PIDS[@]}"; do
-        if kill -0 "$pid" 2>/dev/null; then
-            kill -TERM "$pid" 2>/dev/null
-        fi
-    done
-    # Give processes time to shut down cleanly (pg0 needs to flush WAL).
-    # NOTE: Docker's default stop_grace_period is 10s. If you use the default,
-    # either set stop_grace_period: 30s in your compose file / docker stop -t 30,
-    # or Docker will SIGKILL the container before this timeout expires.
-    local timeout=30
-    for ((i=1; i<=timeout; i++)); do
-        local all_stopped=true
-        for pid in "${PIDS[@]}"; do
-            if kill -0 "$pid" 2>/dev/null; then
-                all_stopped=false
-                break
-            fi
-        done
-        if $all_stopped; then
-            echo "✅ All services stopped cleanly"
-            exit 0
-        fi
-        sleep 1
-    done
-    # Force kill if still running after timeout
-    echo "⚠️  Timeout reached, forcing shutdown..."
-    for pid in "${PIDS[@]}"; do
-        if kill -0 "$pid" 2>/dev/null; then
-            kill -9 "$pid" 2>/dev/null
-        fi
-    done
-    exit 1
-}
-trap cleanup SIGTERM SIGINT
-
 # Track PIDs for wait
 PIDS=()
 
 # Start API if enabled
 if [ "$ENABLE_API" = "true" ]; then
     cd /app/api
-    API_HEALTH_URL="${HINDSIGHT_API_HEALTH_URL:-http://localhost:${HINDSIGHT_API_PORT:-8888}/health}"
-    API_STARTUP_WAIT_SECONDS="${HINDSIGHT_API_STARTUP_WAIT_SECONDS:-300}"
-
     # Run API directly - Python's PYTHONUNBUFFERED=1 handles output buffering
     hindsight-api &
     API_PID=$!
     PIDS+=($API_PID)
 
     # Wait for API to be ready
-    api_ready=false
-    for ((i=1; i<=API_STARTUP_WAIT_SECONDS; i++)); do
-        if ! kill -0 "$API_PID" 2>/dev/null; then
-            wait "$API_PID"
-            exit $?
-        fi
-        if curl -sf "$API_HEALTH_URL" &>/dev/null; then
-            api_ready=true
+    for i in {1..60}; do
+        if curl -sf http://localhost:8888/health &>/dev/null; then
             break
         fi
         sleep 1
     done
-
-    if [ "$api_ready" != "true" ]; then
-        echo "❌ API did not become healthy within ${API_STARTUP_WAIT_SECONDS}s"
-        exit 1
-    fi
 else
     echo "API disabled (HINDSIGHT_ENABLE_API=false)"
 fi
@@ -261,8 +97,7 @@ fi
 if [ "$ENABLE_CP" = "true" ]; then
     echo "🎛️  Starting Control Plane..."
     cd /app/control-plane
-    export HOSTNAME="${HINDSIGHT_CP_HOSTNAME:-0.0.0.0}"
-    PORT="${HINDSIGHT_CP_PORT:-9999}" node server.js &
+    PORT=9999 node server.js &
     CP_PID=$!
     PIDS+=($CP_PID)
 else
@@ -275,7 +110,7 @@ echo "✅ Hindsight is running!"
 echo ""
 echo "📍 Access:"
 if [ "$ENABLE_CP" = "true" ]; then
-    echo "   Control Plane: http://localhost:${HINDSIGHT_CP_PORT:-9999}"
+    echo "   Control Plane: http://localhost:9999"
 fi
 if [ "$ENABLE_API" = "true" ]; then
     echo "   API:           http://localhost:8888"
@@ -288,21 +123,8 @@ if [ ${#PIDS[@]} -eq 0 ]; then
     exit 1
 fi
 
-# Wait for any process to exit (use wait -n with trap-safe loop)
-while true; do
-    # wait -n returns when any child exits; it also returns on signal delivery
-    # (the trap handler will run and exit, so this loop is just for robustness).
-    # `&& true` prevents `set -e` from killing the script when wait -n returns
-    # non-zero (child exited with error or no backgrounded children remain).
-    wait -n && true
-    # Check if any tracked PID has exited
-    for pid in "${PIDS[@]}"; do
-        if ! kill -0 "$pid" 2>/dev/null; then
-            wait "$pid" 2>/dev/null
-            exit_code=$?
-            echo "⚠️  Service (PID $pid) exited with code $exit_code"
-            # Trigger cleanup for remaining services
-            cleanup
-        fi
-    done
-done
+# Wait for any process to exit
+wait -n
+
+# Exit with status of first exited process
+exit $?

docker/standalone/test-start-all.sh

@@ -1,121 +0,0 @@
-#!/bin/bash
-set -euo pipefail
-
-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-
-HINDSIGHT_START_ALL_SOURCE_ONLY=true
-source "$SCRIPT_DIR/start-all.sh"
-unset HINDSIGHT_START_ALL_SOURCE_ONLY
-
-TMP_DIR="$(mktemp -d)"
-trap 'chmod -R u+rwx "$TMP_DIR" 2>/dev/null || true; rm -rf "$TMP_DIR"' EXIT
-
-assert_contains() {
-    local output="$1"
-    local expected="$2"
-
-    if [[ "$output" != *"$expected"* ]]; then
-        echo "Expected output to contain: $expected"
-        echo "Actual output:"
-        echo "$output"
-        exit 1
-    fi
-}
-
-assert_not_contains() {
-    local output="$1"
-    local unexpected="$2"
-
-    if [[ "$output" == *"$unexpected"* ]]; then
-        echo "Expected output not to contain: $unexpected"
-        echo "Actual output:"
-        echo "$output"
-        exit 1
-    fi
-}
-
-assert_empty() {
-    local output="$1"
-
-    if [ -n "$output" ]; then
-        echo "Expected no output, got:"
-        echo "$output"
-        exit 1
-    fi
-}
-
-mkdir -p "$TMP_DIR/empty"
-assert_empty "$(check_pg0_data_integrity "$TMP_DIR/empty")"
-
-mkdir -p "$TMP_DIR/direct"
-touch "$TMP_DIR/direct/PG_VERSION"
-direct_output="$(check_pg0_data_integrity "$TMP_DIR/direct")"
-assert_contains "$direct_output" "Existing pg0 data directory detected"
-assert_not_contains "$direct_output" "WARNING"
-
-mkdir -p "$TMP_DIR/legacy/instance"
-touch "$TMP_DIR/legacy/instance/PG_VERSION"
-legacy_output="$(check_pg0_data_integrity "$TMP_DIR/legacy")"
-assert_contains "$legacy_output" "Existing pg0 data directory detected"
-assert_not_contains "$legacy_output" "WARNING"
-
-mkdir -p "$TMP_DIR/nested/instances/hindsight/data"
-touch "$TMP_DIR/nested/instances/hindsight/data/PG_VERSION"
-nested_output="$(check_pg0_data_integrity "$TMP_DIR/nested")"
-assert_contains "$nested_output" "Existing pg0 data directory detected"
-assert_not_contains "$nested_output" "WARNING"
-
-mkdir -p "$TMP_DIR/nonempty/instances/hindsight"
-touch "$TMP_DIR/nonempty/instances/hindsight/instance.json"
-nonempty_output="$(check_pg0_data_integrity "$TMP_DIR/nonempty")"
-assert_contains "$nonempty_output" "WARNING: pg0 data directory exists"
-
-echo "start-all pg0 integrity checks passed"
-
-# =============================================================================
-# check_pg0_writable (#1483)
-# These rely on filesystem permissions, which root bypasses; skip under root.
-# =============================================================================
-if [ "$(id -u)" != "0" ]; then
-    # Writable directory: returns 0, prints nothing, leaves no artifact behind.
-    mkdir -p "$TMP_DIR/writable"
-    writable_output="$(check_pg0_writable "$TMP_DIR/writable")"
-    assert_empty "$writable_output"
-    if [ -e "$TMP_DIR/writable/.hindsight-write-test" ]; then
-        echo "check_pg0_writable left its write-test file behind"
-        exit 1
-    fi
-
-    # Non-writable directory: returns 1 with actionable guidance.
-    mkdir -p "$TMP_DIR/readonly"
-    chmod 000 "$TMP_DIR/readonly"
-    set +e
-    readonly_output="$(check_pg0_writable "$TMP_DIR/readonly" 2>&1)"
-    readonly_rc=$?
-    set -e
-    chmod 755 "$TMP_DIR/readonly"
-    if [ "$readonly_rc" -eq 0 ]; then
-        echo "check_pg0_writable should fail on a non-writable directory"
-        exit 1
-    fi
-    assert_contains "$readonly_output" "not writable"
-    assert_contains "$readonly_output" "hindsight-data:/home/hindsight/.pg0"
-    assert_contains "$readonly_output" "--user"
-
-    # External database configured: skip the check regardless of dir perms.
-    mkdir -p "$TMP_DIR/extdb"
-    chmod 000 "$TMP_DIR/extdb"
-    set +e
-    HINDSIGHT_API_DATABASE_URL="postgres://x" check_pg0_writable "$TMP_DIR/extdb" >/dev/null 2>&1
-    extdb_rc=$?
-    set -e
-    chmod 755 "$TMP_DIR/extdb"
-    if [ "$extdb_rc" -ne 0 ]; then
-        echo "check_pg0_writable should skip when an external database is configured"
-        exit 1
-    fi
-
-    echo "start-all pg0 writability checks passed"
-else
-    echo "⚠️  Running as root; skipping pg0 writability checks (permissions are bypassed)."
-fi

docker/test-image.sh

@@ -1,235 +0,0 @@
-#!/bin/bash
-#
-# Docker Smoke Test Script
-#
-# Tests that a Hindsight Docker image starts correctly and becomes healthy.
-# Can be run locally or in CI pipelines.
-#
-# Usage:
-#   ./docker/test-image.sh <image> [target]
-#
-# Arguments:
-#   image   - Docker image to test (e.g., hindsight-api:test, ghcr.io/vectorize-io/hindsight:latest)
-#   target  - Optional: 'cp-only' for control plane, otherwise assumes API image (default: api)
-#
-# Environment variables:
-#   HINDSIGHT_API_LLM_API_KEY                   - Required for API/standalone images (LLM verification)
-#   HINDSIGHT_API_LLM_PROVIDER                  - LLM provider (default: openai)
-#   HINDSIGHT_API_LLM_MODEL                     - LLM model (default: gpt-4o-mini)
-#   HINDSIGHT_API_EMBEDDINGS_PROVIDER           - Embeddings provider (optional, for slim images: openai, cohere, tei)
-#   HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY     - OpenAI API key for embeddings (optional)
-#   HINDSIGHT_API_RERANKER_PROVIDER             - Reranker provider (optional, for slim images: cohere, tei)
-#   HINDSIGHT_API_COHERE_API_KEY                - Cohere API key for reranking (optional)
-#   SMOKE_TEST_TIMEOUT                          - Timeout in seconds (default: 120)
-#   SMOKE_TEST_CONTAINER_NAME                   - Container name (default: hindsight-smoke-test)
-#
-# Examples:
-#   # Test a locally built full image
-#   ./docker/test-image.sh hindsight-api:test
-#
-#   # Test a released image
-#   ./docker/test-image.sh ghcr.io/vectorize-io/hindsight:latest
-#
-#   # Test control plane image
-#   ./docker/test-image.sh hindsight-control-plane:test cp-only
-#
-#   # Test slim image with external providers
-#   export HINDSIGHT_API_LLM_API_KEY=sk_xxx
-#   export HINDSIGHT_API_EMBEDDINGS_PROVIDER=openai
-#   export HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=sk-xxx
-#   export HINDSIGHT_API_RERANKER_PROVIDER=cohere
-#   export HINDSIGHT_API_COHERE_API_KEY=xxx
-#   ./docker/test-image.sh hindsight-slim:test
-#
-# Exit codes:
-#   0 - Success (container healthy)
-#   1 - Failure (container not healthy within timeout)
-#   2 - Invalid arguments
-#
-
-set -euo pipefail
-
-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-REPO_ROOT="$(dirname "$SCRIPT_DIR")"
-
-# Colors for output
-RED='\033[0;31m'
-GREEN='\033[0;32m'
-YELLOW='\033[0;33m'
-NC='\033[0m' # No Color
-
-# Configuration
-IMAGE="${1:-}"
-TARGET="${2:-api}"
-TIMEOUT="${SMOKE_TEST_TIMEOUT:-120}"
-CONTAINER_NAME="${SMOKE_TEST_CONTAINER_NAME:-hindsight-smoke-test}"
-LLM_PROVIDER="${HINDSIGHT_API_LLM_PROVIDER:-openai}"
-LLM_MODEL="${HINDSIGHT_API_LLM_MODEL:-gpt-4o-mini}"
-
-# Validate arguments
-if [ -z "$IMAGE" ]; then
-    echo -e "${RED}Error: Image argument is required${NC}"
-    echo ""
-    echo "Usage: $0 <image> [target]"
-    echo ""
-    echo "Examples:"
-    echo "  $0 hindsight-api:test"
-    echo "  $0 ghcr.io/vectorize-io/hindsight:latest"
-    echo "  $0 hindsight-control-plane:test cp-only"
-    exit 2
-fi
-
-# Determine health endpoint based on target
-if [ "$TARGET" = "cp-only" ]; then
-    HEALTH_PORT=9999
-    HEALTH_PATH="/api/health"
-    NEEDS_LLM=false
-else
-    HEALTH_PORT=8888
-    HEALTH_PATH="/health"
-    NEEDS_LLM=true
-fi
-
-# Check for required environment variables
-if [ "$NEEDS_LLM" = true ] && [ "$LLM_PROVIDER" != "vertexai" ] && [ -z "${HINDSIGHT_API_LLM_API_KEY:-}" ]; then
-    echo -e "${RED}Error: HINDSIGHT_API_LLM_API_KEY environment variable is required for API/standalone images${NC}"
-    echo "Set it with: export HINDSIGHT_API_LLM_API_KEY=your-api-key"
-    exit 2
-fi
-
-# Cleanup function
-cleanup() {
-    echo "Cleaning up..."
-    docker stop "$CONTAINER_NAME" 2>/dev/null || true
-    docker rm "$CONTAINER_NAME" 2>/dev/null || true
-}
-
-# Set trap to cleanup on exit
-trap cleanup EXIT
-
-echo -e "${YELLOW}Starting smoke test for: ${IMAGE}${NC}"
-echo "  Target: $TARGET"
-echo "  Health endpoint: http://localhost:${HEALTH_PORT}${HEALTH_PATH}"
-echo "  Timeout: ${TIMEOUT}s"
-echo ""
-
-# Remove any existing container with the same name
-docker rm -f "$CONTAINER_NAME" 2>/dev/null || true
-
-# Start container based on target type
-echo "Starting container..."
-if [ "$TARGET" = "cp-only" ]; then
-    docker run -d --name "$CONTAINER_NAME" \
-        -p "${HEALTH_PORT}:${HEALTH_PORT}" \
-        "$IMAGE"
-else
-    # Build docker run command with required and optional env vars
-    DOCKER_CMD="docker run -d --name $CONTAINER_NAME"
-    DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_LLM_PROVIDER=$LLM_PROVIDER"
-    if [ -n "${HINDSIGHT_API_LLM_API_KEY:-}" ]; then
-        DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_LLM_API_KEY=${HINDSIGHT_API_LLM_API_KEY}"
-    fi
-    DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_LLM_MODEL=$LLM_MODEL"
-
-    # Add Vertex AI config if provider is vertexai
-    if [ "$LLM_PROVIDER" = "vertexai" ]; then
-        if [ -n "${HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY:-}" ]; then
-            DOCKER_CMD="$DOCKER_CMD -v ${HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY}:/tmp/gcp-credentials.json:ro"
-            DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY=/tmp/gcp-credentials.json"
-        fi
-        if [ -n "${HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID:-}" ]; then
-            DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=${HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID}"
-        fi
-        if [ -n "${HINDSIGHT_API_LLM_VERTEXAI_REGION:-}" ]; then
-            DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_LLM_VERTEXAI_REGION=${HINDSIGHT_API_LLM_VERTEXAI_REGION}"
-        fi
-    fi
-
-    # Add optional embeddings provider config
-    if [ -n "${HINDSIGHT_API_EMBEDDINGS_PROVIDER:-}" ]; then
-        DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_EMBEDDINGS_PROVIDER=${HINDSIGHT_API_EMBEDDINGS_PROVIDER}"
-    fi
-    if [ -n "${HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY:-}" ]; then
-        DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=${HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY}"
-    fi
-
-    # Add optional reranker provider config
-    if [ -n "${HINDSIGHT_API_RERANKER_PROVIDER:-}" ]; then
-        DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_RERANKER_PROVIDER=${HINDSIGHT_API_RERANKER_PROVIDER}"
-    fi
-    if [ -n "${HINDSIGHT_API_COHERE_API_KEY:-}" ]; then
-        DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_COHERE_API_KEY=${HINDSIGHT_API_COHERE_API_KEY}"
-    fi
-
-    DOCKER_CMD="$DOCKER_CMD -p ${HEALTH_PORT}:${HEALTH_PORT}"
-    DOCKER_CMD="$DOCKER_CMD $IMAGE"
-
-    eval $DOCKER_CMD
-fi
-
-# Wait for health endpoint
-echo "Waiting for health endpoint at http://localhost:${HEALTH_PORT}${HEALTH_PATH}..."
-start_time=$(date +%s)
-
-for i in $(seq 1 "$TIMEOUT"); do
-    if curl -sf "http://localhost:${HEALTH_PORT}${HEALTH_PATH}" > /dev/null 2>&1; then
-        end_time=$(date +%s)
-        duration=$((end_time - start_time))
-        echo ""
-        echo -e "${GREEN}Container is healthy after ${duration}s${NC}"
-        echo ""
-        echo "=== Health Response ==="
-        curl -s "http://localhost:${HEALTH_PORT}${HEALTH_PATH}" | python3 -m json.tool 2>/dev/null || curl -s "http://localhost:${HEALTH_PORT}${HEALTH_PATH}"
-        echo ""
-
-        # Run retain/recall smoke test for API targets
-        if [ "$TARGET" != "cp-only" ]; then
-            echo ""
-            echo "=== Retain/Recall Smoke Test ==="
-            if ! "$REPO_ROOT/scripts/smoke-test-slim.sh" "http://localhost:${HEALTH_PORT}"; then
-                echo ""
-                echo "=== Container Logs (last 50 lines) ==="
-                docker logs "$CONTAINER_NAME" 2>&1 | tail -50
-                echo ""
-                echo -e "${RED}Smoke test FAILED${NC}"
-                exit 1
-            fi
-        fi
-
-        echo ""
-        echo "=== Container Logs (last 50 lines) ==="
-        docker logs "$CONTAINER_NAME" 2>&1 | tail -50
-        echo ""
-        echo -e "${GREEN}Smoke test PASSED${NC}"
-        exit 0
-    fi
-
-    # Show progress every 10 seconds
-    if [ $((i % 10)) -eq 0 ]; then
-        echo "  Still waiting... (${i}s)"
-    fi
-
-    # Check if container is still running
-    if ! docker ps -q -f "name=$CONTAINER_NAME" | grep -q .; then
-        echo ""
-        echo -e "${RED}Container exited unexpectedly!${NC}"
-        echo ""
-        echo "=== Container Logs ==="
-        docker logs "$CONTAINER_NAME" 2>&1
-        echo ""
-        echo -e "${RED}Smoke test FAILED${NC}"
-        exit 1
-    fi
-
-    sleep 1
-done
-
-# Timeout reached
-echo ""
-echo -e "${RED}Container failed to become healthy after ${TIMEOUT}s${NC}"
-echo ""
-echo "=== Container Logs ==="
-docker logs "$CONTAINER_NAME" 2>&1
-echo ""
-echo -e "${RED}Smoke test FAILED${NC}"
-exit 1

docker/test-slim-local.sh

@@ -1,47 +0,0 @@
-#!/bin/bash
-#
-# Local Test Script for Slim Docker Images
-#
-# This script makes it easy to test slim images locally with external providers.
-# It expects API keys to be set in environment variables.
-#
-# Usage:
-#   export OPENAI_API_KEY=sk-xxx
-#   export COHERE_API_KEY=xxx
-#   ./docker/test-slim-local.sh
-#
-# Or inline:
-#   OPENAI_API_KEY=sk_xxx COHERE_API_KEY=xxx ./docker/test-slim-local.sh
-#
-
-set -euo pipefail
-
-# Check for required API keys
-if [ -z "${OPENAI_API_KEY:-}" ]; then
-    echo "❌ Error: OPENAI_API_KEY environment variable is required"
-    echo "Set it with: export OPENAI_API_KEY=sk-xxx"
-    exit 1
-fi
-
-if [ -z "${COHERE_API_KEY:-}" ]; then
-    echo "❌ Error: COHERE_API_KEY environment variable is required"
-    echo "Set it with: export COHERE_API_KEY=xxx"
-    exit 1
-fi
-
-# Configuration
-IMAGE="${1:-hindsight-slim:test}"
-echo "Testing image: $IMAGE"
-echo ""
-
-# Set up LLM and external providers
-export HINDSIGHT_API_LLM_PROVIDER=openai
-export HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY
-export HINDSIGHT_API_LLM_MODEL=gpt-4o-mini
-export HINDSIGHT_API_EMBEDDINGS_PROVIDER=openai
-export HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=$OPENAI_API_KEY
-export HINDSIGHT_API_RERANKER_PROVIDER=cohere
-export HINDSIGHT_API_COHERE_API_KEY=$COHERE_API_KEY
-
-# Run the test
-exec "$(dirname "$0")/test-image.sh" "$IMAGE" standalone

helm/hindsight/Chart.lock

@@ -0,0 +1,6 @@
+dependencies:
+- name: postgresql
+  repository: https://charts.bitnami.com/bitnami
+  version: 15.5.38
+digest: sha256:f67c7612736803ece8a669f8ca6b0555f3b78557bc0ecb732aa2e43f0df7750d
+generated: "2025-12-10T17:20:57.058794+01:00"

helm/hindsight/Chart.yaml

@@ -2,8 +2,8 @@ apiVersion: v2
 name: hindsight
 description: Hindsight helm chart
 type: application
-version: 0.8.4
-appVersion: "0.8.4"
+version: 0.2.1
+appVersion: "0.2.1"
 keywords:
   - ai
   - memory

helm/hindsight/README.md

@@ -66,13 +66,13 @@ helm install hindsight ./helm/hindsight -n hindsight --create-namespace -f value
 
 | Parameter | Description | Default |
 |-----------|-------------|---------|
-| `version` | Default image tag for all components | Chart `appVersion` |
+| `version` | Default image tag for all components | `0.1.0` |
 | `api.enabled` | Enable the API component | `true` |
-| `api.image.repository` | API image repository | `ghcr.io/vectorize-io/hindsight-api` |
+| `api.image.repository` | API image repository | `hindsight/api` |
 | `api.image.tag` | API image tag (defaults to `version`) | - |
 | `api.service.port` | API service port | `8888` |
 | `controlPlane.enabled` | Enable the control plane | `true` |
-| `controlPlane.image.repository` | Control plane image repository | `ghcr.io/vectorize-io/hindsight-control-plane` |
+| `controlPlane.image.repository` | Control plane image repository | `hindsight/control-plane` |
 | `controlPlane.image.tag` | Control plane image tag (defaults to `version`) | - |
 | `controlPlane.service.port` | Control plane service port | `3000` |
 | `postgresql.enabled` | Deploy PostgreSQL as subchart | `true` |

helm/hindsight/templates/_helpers.tpl

@@ -80,22 +80,6 @@ Control plane selector labels
 app.kubernetes.io/component: control-plane
 {{- end }}
 
-{{/*
-Worker labels
-*/}}
-{{- define "hindsight.worker.labels" -}}
-{{ include "hindsight.labels" . }}
-app.kubernetes.io/component: worker
-{{- end }}
-
-{{/*
-Worker selector labels
-*/}}
-{{- define "hindsight.worker.selectorLabels" -}}
-{{ include "hindsight.selectorLabels" . }}
-app.kubernetes.io/component: worker
-{{- end }}
-
 {{/*
 Create the name of the service account to use
 */}}
@@ -127,38 +111,6 @@ API URL for control plane
 {{- printf "http://%s-api:%d" (include "hindsight.fullname" .) (.Values.api.service.port | int) }}
 {{- end }}
 
-{{/*
-TEI reranker labels
-*/}}
-{{- define "hindsight.tei.reranker.labels" -}}
-{{ include "hindsight.labels" . }}
-app.kubernetes.io/component: tei-reranker
-{{- end }}
-
-{{/*
-TEI reranker selector labels
-*/}}
-{{- define "hindsight.tei.reranker.selectorLabels" -}}
-{{ include "hindsight.selectorLabels" . }}
-app.kubernetes.io/component: tei-reranker
-{{- end }}
-
-{{/*
-TEI embedding labels
-*/}}
-{{- define "hindsight.tei.embedding.labels" -}}
-{{ include "hindsight.labels" . }}
-app.kubernetes.io/component: tei-embedding
-{{- end }}
-
-{{/*
-TEI embedding selector labels
-*/}}
-{{- define "hindsight.tei.embedding.selectorLabels" -}}
-{{ include "hindsight.selectorLabels" . }}
-app.kubernetes.io/component: tei-embedding
-{{- end }}
-
 {{/*
 Get the name of the secret to use
 */}}

helm/hindsight/templates/api-deployment.yaml

@@ -33,7 +33,7 @@ spec:
       - name: api
         securityContext:
           {{- toYaml .Values.securityContext | nindent 10 }}
-        image: "{{ .Values.api.image.repository }}:{{ .Values.api.image.tag | default .Values.version | default .Chart.AppVersion }}"
+        image: "{{ .Values.api.image.repository }}:{{ .Values.api.image.tag | default .Values.version }}"
         imagePullPolicy: {{ .Values.api.image.pullPolicy }}
         ports:
         - name: http
@@ -55,30 +55,10 @@ spec:
         {{- end }}
         - name: HINDSIGHT_API_DATABASE_URL
           value: {{ include "hindsight.databaseUrl" . | quote }}
-        {{- /* Disable internal worker when dedicated workers are enabled */}}
-        {{- if .Values.worker.enabled }}
-        - name: HINDSIGHT_API_WORKER_ENABLED
-          value: "false"
-        {{- end }}
-        {{- /* Explicitly set port to override K8s service discovery env var (HINDSIGHT_API_PORT) */}}
-        - name: HINDSIGHT_API_PORT
-          value: {{ .Values.api.service.targetPort | quote }}
         {{- range $key, $value := .Values.api.env }}
         - name: {{ $key }}
           value: {{ $value | quote }}
         {{- end }}
-        {{- if .Values.tei.reranker.enabled }}
-        - name: HINDSIGHT_API_RERANKER_PROVIDER
-          value: "tei"
-        - name: HINDSIGHT_API_RERANKER_TEI_URL
-          value: "http://{{ include "hindsight.fullname" . }}-tei-reranker:{{ .Values.tei.reranker.port }}"
-        {{- end }}
-        {{- if .Values.tei.embedding.enabled }}
-        - name: HINDSIGHT_API_EMBEDDINGS_PROVIDER
-          value: "tei"
-        - name: HINDSIGHT_API_EMBEDDINGS_TEI_URL
-          value: "http://{{ include "hindsight.fullname" . }}-tei-embedding:{{ .Values.tei.embedding.port }}"
-        {{- end }}
         {{- /* Only use api.secrets when not using existingSecret (for chart-managed secrets) */}}
         {{- if not .Values.existingSecret }}
         {{- range $key, $value := .Values.api.secrets }}
@@ -95,32 +75,11 @@ spec:
           {{- toYaml .Values.api.readinessProbe | nindent 10 }}
         resources:
           {{- toYaml .Values.api.resources | nindent 10 }}
-        {{- if or .Values.api.persistence.modelCache.enabled .Values.api.extraVolumeMounts }}
-        volumeMounts:
-        {{- if .Values.api.persistence.modelCache.enabled }}
-        - name: model-cache
-          mountPath: /home/hindsight/.cache
-        {{- end }}
-        {{- with .Values.api.extraVolumeMounts }}
-        {{- toYaml . | nindent 8 }}
-        {{- end }}
-        {{- end }}
-      {{- if or .Values.api.persistence.modelCache.enabled .Values.api.extraVolumes }}
-      volumes:
-      {{- if .Values.api.persistence.modelCache.enabled }}
-      - name: model-cache
-        persistentVolumeClaim:
-          claimName: {{ include "hindsight.fullname" . }}-api-model-cache
-      {{- end }}
-      {{- with .Values.api.extraVolumes }}
-      {{- toYaml . | nindent 6 }}
-      {{- end }}
-      {{- end }}
       {{- with .Values.nodeSelector }}
       nodeSelector:
         {{- toYaml . | nindent 8 }}
       {{- end }}
-      {{- with (.Values.api.affinity | default .Values.affinity) }}
+      {{- with .Values.affinity }}
       affinity:
         {{- toYaml . | nindent 8 }}
       {{- end }}

helm/hindsight/templates/api-model-cache-pvc.yaml

@@ -1,21 +0,0 @@
-{{- if and .Values.api.enabled .Values.api.persistence.modelCache.enabled }}
-apiVersion: v1
-kind: PersistentVolumeClaim
-metadata:
-  name: {{ include "hindsight.fullname" . }}-api-model-cache
-  labels:
-    {{- include "hindsight.api.labels" . | nindent 4 }}
-  {{- with .Values.api.persistence.modelCache.annotations }}
-  annotations:
-    {{- toYaml . | nindent 4 }}
-  {{- end }}
-spec:
-  accessModes:
-    {{- toYaml .Values.api.persistence.modelCache.accessModes | nindent 4 }}
-  {{- if .Values.api.persistence.modelCache.storageClass }}
-  storageClassName: {{ .Values.api.persistence.modelCache.storageClass }}
-  {{- end }}
-  resources:
-    requests:
-      storage: {{ .Values.api.persistence.modelCache.size }}
-{{- end }}

helm/hindsight/templates/controlplane-deployment.yaml

@@ -33,7 +33,7 @@ spec:
       - name: control-plane
         securityContext:
           {{- toYaml .Values.securityContext | nindent 10 }}
-        image: "{{ .Values.controlPlane.image.repository }}:{{ .Values.controlPlane.image.tag | default .Values.version | default .Chart.AppVersion }}"
+        image: "{{ .Values.controlPlane.image.repository }}:{{ .Values.controlPlane.image.tag | default .Values.version }}"
         imagePullPolicy: {{ .Values.controlPlane.image.pullPolicy }}
         ports:
         - name: http
@@ -71,7 +71,7 @@ spec:
       nodeSelector:
         {{- toYaml . | nindent 8 }}
       {{- end }}
-      {{- with (.Values.controlPlane.affinity | default .Values.affinity) }}
+      {{- with .Values.affinity }}
       affinity:
         {{- toYaml . | nindent 8 }}
       {{- end }}

helm/hindsight/templates/pdb.yaml

@@ -1,56 +0,0 @@
-{{- if and .Values.api.enabled .Values.api.podDisruptionBudget.enabled }}
-apiVersion: policy/v1
-kind: PodDisruptionBudget
-metadata:
-  name: {{ include "hindsight.fullname" . }}-api
-  labels:
-    {{- include "hindsight.api.labels" . | nindent 4 }}
-spec:
-  {{- if .Values.api.podDisruptionBudget.minAvailable }}
-  minAvailable: {{ .Values.api.podDisruptionBudget.minAvailable }}
-  {{- end }}
-  {{- if .Values.api.podDisruptionBudget.maxUnavailable }}
-  maxUnavailable: {{ .Values.api.podDisruptionBudget.maxUnavailable }}
-  {{- end }}
-  selector:
-    matchLabels:
-      {{- include "hindsight.api.selectorLabels" . | nindent 6 }}
-{{- end }}
----
-{{- if and .Values.controlPlane.enabled .Values.controlPlane.podDisruptionBudget.enabled }}
-apiVersion: policy/v1
-kind: PodDisruptionBudget
-metadata:
-  name: {{ include "hindsight.fullname" . }}-control-plane
-  labels:
-    {{- include "hindsight.controlPlane.labels" . | nindent 4 }}
-spec:
-  {{- if .Values.controlPlane.podDisruptionBudget.minAvailable }}
-  minAvailable: {{ .Values.controlPlane.podDisruptionBudget.minAvailable }}
-  {{- end }}
-  {{- if .Values.controlPlane.podDisruptionBudget.maxUnavailable }}
-  maxUnavailable: {{ .Values.controlPlane.podDisruptionBudget.maxUnavailable }}
-  {{- end }}
-  selector:
-    matchLabels:
-      {{- include "hindsight.controlPlane.selectorLabels" . | nindent 6 }}
-{{- end }}
----
-{{- if and .Values.worker.enabled .Values.worker.podDisruptionBudget.enabled }}
-apiVersion: policy/v1
-kind: PodDisruptionBudget
-metadata:
-  name: {{ include "hindsight.fullname" . }}-worker
-  labels:
-    {{- include "hindsight.worker.labels" . | nindent 4 }}
-spec:
-  {{- if .Values.worker.podDisruptionBudget.minAvailable }}
-  minAvailable: {{ .Values.worker.podDisruptionBudget.minAvailable }}
-  {{- end }}
-  {{- if .Values.worker.podDisruptionBudget.maxUnavailable }}
-  maxUnavailable: {{ .Values.worker.podDisruptionBudget.maxUnavailable }}
-  {{- end }}
-  selector:
-    matchLabels:
-      {{- include "hindsight.worker.selectorLabels" . | nindent 6 }}
-{{- end }}

helm/hindsight/templates/tei-embedding-deployment.yaml

@@ -1,76 +0,0 @@
-{{- if .Values.tei.embedding.enabled }}
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: {{ include "hindsight.fullname" . }}-tei-embedding
-  labels:
-    {{- include "hindsight.tei.embedding.labels" . | nindent 4 }}
-spec:
-  replicas: {{ .Values.tei.embedding.replicaCount }}
-  selector:
-    matchLabels:
-      {{- include "hindsight.tei.embedding.selectorLabels" . | nindent 6 }}
-  template:
-    metadata:
-      {{- with .Values.podAnnotations }}
-      annotations:
-        {{- toYaml . | nindent 8 }}
-      {{- end }}
-      labels:
-        {{- include "hindsight.tei.embedding.selectorLabels" . | nindent 8 }}
-    spec:
-      {{- if .Values.serviceAccount.create }}
-      serviceAccountName: {{ include "hindsight.serviceAccountName" . }}
-      {{- end }}
-      securityContext:
-        {{- toYaml .Values.podSecurityContext | nindent 8 }}
-      containers:
-      - name: tei-embedding
-        securityContext:
-          {{- toYaml .Values.securityContext | nindent 10 }}
-        image: "{{ .Values.tei.embedding.image.repository }}:{{ .Values.tei.embedding.image.tag }}"
-        imagePullPolicy: {{ .Values.tei.embedding.image.pullPolicy }}
-        args:
-          - "--model-id"
-          - {{ .Values.tei.embedding.model | quote }}
-          - "--hostname"
-          - "0.0.0.0"
-          {{- range .Values.tei.embedding.args }}
-          - {{ . | quote }}
-          {{- end }}
-        ports:
-        - name: http
-          containerPort: {{ .Values.tei.embedding.port }}
-          protocol: TCP
-        env:
-        - name: PORT
-          value: {{ .Values.tei.embedding.port | quote }}
-        {{- range $key, $value := .Values.tei.embedding.env }}
-        - name: {{ $key }}
-          value: {{ $value | quote }}
-        {{- end }}
-        livenessProbe:
-          {{- toYaml .Values.tei.embedding.livenessProbe | nindent 10 }}
-        readinessProbe:
-          {{- toYaml .Values.tei.embedding.readinessProbe | nindent 10 }}
-        resources:
-          {{- toYaml .Values.tei.embedding.resources | nindent 10 }}
-        volumeMounts:
-        - name: model-cache
-          mountPath: /data
-      volumes:
-      - name: model-cache
-        emptyDir: {}
-      {{- with .Values.nodeSelector }}
-      nodeSelector:
-        {{- toYaml . | nindent 8 }}
-      {{- end }}
-      {{- with .Values.affinity }}
-      affinity:
-        {{- toYaml . | nindent 8 }}
-      {{- end }}
-      {{- with .Values.tolerations }}
-      tolerations:
-        {{- toYaml . | nindent 8 }}
-      {{- end }}
-{{- end }}

helm/hindsight/templates/tei-embedding-service.yaml

@@ -1,17 +0,0 @@
-{{- if .Values.tei.embedding.enabled }}
-apiVersion: v1
-kind: Service
-metadata:
-  name: {{ include "hindsight.fullname" . }}-tei-embedding
-  labels:
-    {{- include "hindsight.tei.embedding.labels" . | nindent 4 }}
-spec:
-  type: ClusterIP
-  ports:
-  - port: {{ .Values.tei.embedding.port }}
-    targetPort: http
-    protocol: TCP
-    name: http
-  selector:
-    {{- include "hindsight.tei.embedding.selectorLabels" . | nindent 4 }}
-{{- end }}

helm/hindsight/templates/tei-reranker-deployment.yaml

@@ -1,76 +0,0 @@
-{{- if .Values.tei.reranker.enabled }}
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: {{ include "hindsight.fullname" . }}-tei-reranker
-  labels:
-    {{- include "hindsight.tei.reranker.labels" . | nindent 4 }}
-spec:
-  replicas: {{ .Values.tei.reranker.replicaCount }}
-  selector:
-    matchLabels:
-      {{- include "hindsight.tei.reranker.selectorLabels" . | nindent 6 }}
-  template:
-    metadata:
-      {{- with .Values.podAnnotations }}
-      annotations:
-        {{- toYaml . | nindent 8 }}
-      {{- end }}
-      labels:
-        {{- include "hindsight.tei.reranker.selectorLabels" . | nindent 8 }}
-    spec:
-      {{- if .Values.serviceAccount.create }}
-      serviceAccountName: {{ include "hindsight.serviceAccountName" . }}
-      {{- end }}
-      securityContext:
-        {{- toYaml .Values.podSecurityContext | nindent 8 }}
-      containers:
-      - name: tei-reranker
-        securityContext:
-          {{- toYaml .Values.securityContext | nindent 10 }}
-        image: "{{ .Values.tei.reranker.image.repository }}:{{ .Values.tei.reranker.image.tag }}"
-        imagePullPolicy: {{ .Values.tei.reranker.image.pullPolicy }}
-        args:
-          - "--model-id"
-          - {{ .Values.tei.reranker.model | quote }}
-          - "--hostname"
-          - "0.0.0.0"
-          {{- range .Values.tei.reranker.args }}
-          - {{ . | quote }}
-          {{- end }}
-        ports:
-        - name: http
-          containerPort: {{ .Values.tei.reranker.port }}
-          protocol: TCP
-        env:
-        - name: PORT
-          value: {{ .Values.tei.reranker.port | quote }}
-        {{- range $key, $value := .Values.tei.reranker.env }}
-        - name: {{ $key }}
-          value: {{ $value | quote }}
-        {{- end }}
-        livenessProbe:
-          {{- toYaml .Values.tei.reranker.livenessProbe | nindent 10 }}
-        readinessProbe:
-          {{- toYaml .Values.tei.reranker.readinessProbe | nindent 10 }}
-        resources:
-          {{- toYaml .Values.tei.reranker.resources | nindent 10 }}
-        volumeMounts:
-        - name: model-cache
-          mountPath: /data
-      volumes:
-      - name: model-cache
-        emptyDir: {}
-      {{- with .Values.nodeSelector }}
-      nodeSelector:
-        {{- toYaml . | nindent 8 }}
-      {{- end }}
-      {{- with .Values.affinity }}
-      affinity:
-        {{- toYaml . | nindent 8 }}
-      {{- end }}
-      {{- with .Values.tolerations }}
-      tolerations:
-        {{- toYaml . | nindent 8 }}
-      {{- end }}
-{{- end }}

helm/hindsight/templates/tei-reranker-service.yaml

@@ -1,17 +0,0 @@
-{{- if .Values.tei.reranker.enabled }}
-apiVersion: v1
-kind: Service
-metadata:
-  name: {{ include "hindsight.fullname" . }}-tei-reranker
-  labels:
-    {{- include "hindsight.tei.reranker.labels" . | nindent 4 }}
-spec:
-  type: ClusterIP
-  ports:
-  - port: {{ .Values.tei.reranker.port }}
-    targetPort: http
-    protocol: TCP
-    name: http
-  selector:
-    {{- include "hindsight.tei.reranker.selectorLabels" . | nindent 4 }}
-{{- end }}

helm/hindsight/templates/worker-service.yaml

@@ -1,25 +0,0 @@
-{{- if .Values.worker.enabled }}
-apiVersion: v1
-kind: Service
-metadata:
-  name: {{ include "hindsight.fullname" . }}-worker
-  labels:
-    {{- include "hindsight.worker.labels" . | nindent 4 }}
-  {{- if .Values.podAnnotations }}
-  annotations:
-    {{- /* Common Prometheus annotations for metrics scraping */}}
-    prometheus.io/scrape: "true"
-    prometheus.io/port: {{ .Values.worker.service.port | quote }}
-    prometheus.io/path: "/metrics"
-  {{- end }}
-spec:
-  # Headless service for StatefulSet (enables stable DNS names like worker-0.worker.namespace)
-  clusterIP: None
-  ports:
-  - port: {{ .Values.worker.service.port }}
-    targetPort: {{ .Values.worker.service.targetPort }}
-    protocol: TCP
-    name: http
-  selector:
-    {{- include "hindsight.worker.selectorLabels" . | nindent 4 }}
-{{- end }}

helm/hindsight/templates/worker-statefulset.yaml

@@ -1,142 +0,0 @@
-{{- if .Values.worker.enabled }}
-apiVersion: apps/v1
-kind: StatefulSet
-metadata:
-  name: {{ include "hindsight.fullname" . }}-worker
-  labels:
-    {{- include "hindsight.worker.labels" . | nindent 4 }}
-spec:
-  serviceName: {{ include "hindsight.fullname" . }}-worker
-  replicas: {{ .Values.worker.replicaCount }}
-  selector:
-    matchLabels:
-      {{- include "hindsight.worker.selectorLabels" . | nindent 6 }}
-  template:
-    metadata:
-      annotations:
-        {{- if not .Values.existingSecret }}
-        checksum/secret: {{ include (print $.Template.BasePath "/secret.yaml") . | sha256sum }}
-        {{- end }}
-        {{- with .Values.podAnnotations }}
-        {{- toYaml . | nindent 8 }}
-        {{- end }}
-      labels:
-        {{- include "hindsight.worker.selectorLabels" . | nindent 8 }}
-    spec:
-      {{- if .Values.serviceAccount.create }}
-      serviceAccountName: {{ include "hindsight.serviceAccountName" . }}
-      {{- end }}
-      securityContext:
-        {{- toYaml .Values.podSecurityContext | nindent 8 }}
-      containers:
-      - name: worker
-        securityContext:
-          {{- toYaml .Values.securityContext | nindent 10 }}
-        image: "{{ .Values.worker.image.repository }}:{{ .Values.worker.image.tag | default .Values.version | default .Chart.AppVersion }}"
-        imagePullPolicy: {{ .Values.worker.image.pullPolicy }}
-        command: ["hindsight-worker"]
-        ports:
-        - name: http
-          containerPort: {{ .Values.worker.service.targetPort }}
-          protocol: TCP
-        {{- if .Values.existingSecret }}
-        envFrom:
-        - secretRef:
-            name: {{ .Values.existingSecret }}
-        {{- end }}
-        env:
-        {{- /* POSTGRES_PASSWORD must be defined before DATABASE_URL for $(VAR) interpolation */}}
-        {{- if not .Values.postgresql.enabled }}
-        - name: POSTGRES_PASSWORD
-          valueFrom:
-            secretKeyRef:
-              name: {{ include "hindsight.secretName" . }}
-              key: postgres-password
-        {{- end }}
-        - name: HINDSIGHT_API_DATABASE_URL
-          value: {{ include "hindsight.databaseUrl" . | quote }}
-        {{- /* Worker ID uses pod name (StatefulSet provides stable names like worker-0, worker-1) */}}
-        - name: HINDSIGHT_API_WORKER_ID
-          valueFrom:
-            fieldRef:
-              fieldPath: metadata.name
-        {{- /* Inherit LLM config from api.env */}}
-        {{- range $key, $value := .Values.api.env }}
-        - name: {{ $key }}
-          value: {{ $value | quote }}
-        {{- end }}
-        {{- /* Worker-specific env vars */}}
-        {{- range $key, $value := .Values.worker.env }}
-        - name: {{ $key }}
-          value: {{ $value | quote }}
-        {{- end }}
-        {{- /* Only use secrets when not using existingSecret */}}
-        {{- if not .Values.existingSecret }}
-        {{- /* Inherit secrets from api.secrets */}}
-        {{- range $key, $value := .Values.api.secrets }}
-        - name: {{ $key }}
-          valueFrom:
-            secretKeyRef:
-              name: {{ include "hindsight.secretName" $ }}
-              key: {{ $key }}
-        {{- end }}
-        {{- /* Worker-specific secrets (can override api.secrets) */}}
-        {{- range $key, $value := .Values.worker.secrets }}
-        - name: {{ $key }}
-          valueFrom:
-            secretKeyRef:
-              name: {{ include "hindsight.secretName" $ }}
-              key: {{ $key }}
-        {{- end }}
-        {{- end }}
-        livenessProbe:
-          {{- toYaml .Values.worker.livenessProbe | nindent 10 }}
-        readinessProbe:
-          {{- toYaml .Values.worker.readinessProbe | nindent 10 }}
-        resources:
-          {{- toYaml .Values.worker.resources | nindent 10 }}
-        {{- if or .Values.worker.persistence.modelCache.enabled .Values.worker.extraVolumeMounts }}
-        volumeMounts:
-        {{- if .Values.worker.persistence.modelCache.enabled }}
-        - name: model-cache
-          mountPath: /home/hindsight/.cache
-        {{- end }}
-        {{- with .Values.worker.extraVolumeMounts }}
-        {{- toYaml . | nindent 8 }}
-        {{- end }}
-        {{- end }}
-      {{- with .Values.nodeSelector }}
-      nodeSelector:
-        {{- toYaml . | nindent 8 }}
-      {{- end }}
-      {{- with (.Values.worker.affinity | default .Values.affinity) }}
-      affinity:
-        {{- toYaml . | nindent 8 }}
-      {{- end }}
-      {{- with .Values.tolerations }}
-      tolerations:
-        {{- toYaml . | nindent 8 }}
-      {{- end }}
-      {{- with .Values.worker.extraVolumes }}
-      volumes:
-      {{- toYaml . | nindent 6 }}
-      {{- end }}
-  {{- if .Values.worker.persistence.modelCache.enabled }}
-  volumeClaimTemplates:
-  - metadata:
-      name: model-cache
-      {{- with .Values.worker.persistence.modelCache.annotations }}
-      annotations:
-        {{- toYaml . | nindent 8 }}
-      {{- end }}
-    spec:
-      accessModes:
-        {{- toYaml .Values.worker.persistence.modelCache.accessModes | nindent 8 }}
-      {{- if .Values.worker.persistence.modelCache.storageClass }}
-      storageClassName: {{ .Values.worker.persistence.modelCache.storageClass }}
-      {{- end }}
-      resources:
-        requests:
-          storage: {{ .Values.worker.persistence.modelCache.size }}
-  {{- end }}
-{{- end }}

helm/hindsight/values.yaml

@@ -1,8 +1,7 @@
 # Default values for hindsight
 
-# Global version override - use this to set a consistent image tag across all components
-# If not set, defaults to Chart.appVersion from Chart.yaml
-# version: ""
+# Chart version - use this to set a consistent image tag across all components
+version: "0.1.1"
 
 # Use an existing secret instead of creating one from values
 # When set, all keys from this secret are injected as environment variables via envFrom
@@ -13,6 +12,9 @@
 #   - Any other env vars you want to inject
 # existingSecret: "my-hindsight-secret"
 
+# Global settings
+replicaCount: 1
+
 # Image settings for api
 api:
   enabled: true
@@ -55,48 +57,6 @@ api:
     timeoutSeconds: 3
     failureThreshold: 3
 
-  # Pod disruption budget
-  podDisruptionBudget:
-    enabled: false
-    minAvailable: 1
-    # maxUnavailable: 1
-
-  # Pod affinity/anti-affinity (overrides global affinity for this component)
-  # affinity: {}
-
-  # Persistent volume for local model cache (reranker, embeddings)
-  # Models are downloaded to /home/hindsight/.cache on first use.
-  # Without persistence, models are re-downloaded on every pod restart.
-  #
-  # For production, prefer baking models into a custom image instead of
-  # enabling this PVC: image layers are pulled once per node and cached
-  # for free, while a PVC adds storage cost, pins pods to a node
-  # (ReadWriteOnce), and needs lifecycle management on uninstall/upgrade.
-  # See docs: developer/installation#bundling-custom-models-in-a-custom-image
-  persistence:
-    modelCache:
-      enabled: false
-      size: 5Gi
-      storageClass: ""
-      accessModes:
-        - ReadWriteOnce
-      annotations: {}
-
-  # Extra volume mounts for the api container
-  # e.g.
-  # extraVolumeMounts:
-  #   - name: my-volume
-  #     mountPath: /mnt/my-volume
-  extraVolumeMounts: []
-
-  # Extra volumes for the api pod
-  # e.g.
-  # extraVolumes:
-  #   - name: my-volume
-  #     configMap:
-  #       name: my-configmap
-  extraVolumes: []
-
   # Environment variables
   env:
     #HINDSIGHT_API_LLM_PROVIDER: "groq"
@@ -107,101 +67,6 @@ api:
     # HINDSIGHT_API_LLM_API_KEY: "your-api-key"
     # HINDSIGHT_API_LLM_BASE_URL: "https://api.groq.com/openai/v1"
 
-# Worker settings (distributed task processing)
-# When enabled, dedicated worker pods process tasks and the API's internal worker is disabled
-worker:
-  enabled: false
-  replicaCount: 2
-  image:
-    repository: ghcr.io/vectorize-io/hindsight-api
-    pullPolicy: IfNotPresent
-    # tag: ""  # defaults to .Values.version, then Chart.appVersion if not specified
-
-  service:
-    # Service for metrics scraping (headless for StatefulSet)
-    port: 8889
-    targetPort: 8889
-
-  # Resource limits and requests
-  resources:
-    limits:
-      cpu: 2000m
-      memory: 4Gi
-    requests:
-      cpu: 500m
-      memory: 1Gi
-
-  # Liveness and readiness probes
-  livenessProbe:
-    httpGet:
-      path: /health
-      port: 8889
-    initialDelaySeconds: 30
-    periodSeconds: 10
-    timeoutSeconds: 5
-    failureThreshold: 3
-
-  readinessProbe:
-    httpGet:
-      path: /health
-      port: 8889
-    initialDelaySeconds: 10
-    periodSeconds: 5
-    timeoutSeconds: 3
-    failureThreshold: 3
-
-  # Worker-specific environment variables
-  env:
-    # Poll interval in milliseconds (how often to check for new tasks)
-    HINDSIGHT_API_WORKER_POLL_INTERVAL_MS: "500"
-    # Number of tasks to claim per poll cycle
-    HINDSIGHT_API_WORKER_BATCH_SIZE: "10"
-    # Max retries before marking a task as failed
-    HINDSIGHT_API_WORKER_MAX_RETRIES: "3"
-    # HTTP port for metrics/health (matches service.targetPort)
-    HINDSIGHT_API_WORKER_HTTP_PORT: "8889"
-
-  # Pod disruption budget
-  podDisruptionBudget:
-    enabled: false
-    minAvailable: 1
-    # maxUnavailable: 1
-
-  # Pod affinity/anti-affinity (overrides global affinity for this component)
-  # affinity: {}
-
-  # Persistent volume for local model cache (reranker, embeddings)
-  # Uses volumeClaimTemplates since worker is a StatefulSet — one PVC per
-  # replica. For production, prefer baking models into a custom image; see
-  # api.persistence.modelCache above and docs:
-  # developer/installation#bundling-custom-models-in-a-custom-image
-  persistence:
-    modelCache:
-      enabled: false
-      size: 5Gi
-      storageClass: ""
-      accessModes:
-        - ReadWriteOnce
-      annotations: {}
-
-  # Extra volume mounts for the worker container
-  # e.g.
-  # extraVolumeMounts:
-  #   - name: my-volume
-  #     mountPath: /mnt/my-volume
-  extraVolumeMounts: []
-
-  # Extra volumes for the worker pod
-  # e.g.
-  # extraVolumes:
-  #   - name: my-volume
-  #     configMap:
-  #       name: my-configmap
-  extraVolumes: []
-
-  # Secret environment variables (inherited from api.secrets if not specified)
-  secrets: {}
-
 # Image settings for control plane
 controlPlane:
   enabled: true
@@ -242,15 +107,6 @@ controlPlane:
     timeoutSeconds: 3
     failureThreshold: 3
 
-  # Pod disruption budget
-  podDisruptionBudget:
-    enabled: false
-    minAvailable: 1
-    # maxUnavailable: 1
-
-  # Pod affinity/anti-affinity (overrides global affinity for this component)
-  # affinity: {}
-
   # Environment variables
   env:
     NODE_ENV: "production"
@@ -349,87 +205,9 @@ nodeSelector: {}
 # Tolerations
 tolerations: []
 
-# Affinity (applied to all components unless overridden per-component)
+# Affinity
 affinity: {}
 
-# TEI (Text Embeddings Inference) - optional standalone deployments
-# for reranking and/or embedding models
-tei:
-  reranker:
-    enabled: false
-    replicaCount: 1
-    image:
-      repository: ghcr.io/huggingface/text-embeddings-inference
-      tag: cpu-1.8.3
-      pullPolicy: IfNotPresent
-    model: "cross-encoder/ms-marco-MiniLM-L-6-v2"
-    port: 8090
-    args:
-      - "--auto-truncate"
-    env:
-      PAYLOAD_LIMIT: "10000000"
-      MAX_CLIENT_BATCH_SIZE: "256"
-    resources:
-      limits:
-        cpu: 2000m
-        memory: 2Gi
-      requests:
-        cpu: 500m
-        memory: 1Gi
-    livenessProbe:
-      httpGet:
-        path: /health
-        port: 8090
-      initialDelaySeconds: 30
-      periodSeconds: 10
-      timeoutSeconds: 5
-      failureThreshold: 6
-    readinessProbe:
-      httpGet:
-        path: /health
-        port: 8090
-      initialDelaySeconds: 15
-      periodSeconds: 5
-      timeoutSeconds: 3
-      failureThreshold: 3
-
-  embedding:
-    enabled: false
-    replicaCount: 1
-    image:
-      repository: ghcr.io/huggingface/text-embeddings-inference
-      tag: cpu-1.8.3
-      pullPolicy: IfNotPresent
-    model: "sentence-transformers/all-MiniLM-L6-v2"
-    port: 8091
-    args: []
-    env:
-      PAYLOAD_LIMIT: "10000000"
-      MAX_CLIENT_BATCH_SIZE: "256"
-    resources:
-      limits:
-        cpu: 2000m
-        memory: 2Gi
-      requests:
-        cpu: 500m
-        memory: 1Gi
-    livenessProbe:
-      httpGet:
-        path: /health
-        port: 8091
-      initialDelaySeconds: 30
-      periodSeconds: 10
-      timeoutSeconds: 5
-      failureThreshold: 6
-    readinessProbe:
-      httpGet:
-        path: /health
-        port: 8091
-      initialDelaySeconds: 15
-      periodSeconds: 5
-      timeoutSeconds: 3
-      failureThreshold: 3
-
 # Autoscaling
 autoscaling:
   enabled: false

hindsight-all-npm/.gitignore

@@ -1,4 +0,0 @@
-node_modules
-dist
-*.tgz
-.DS_Store

hindsight-all-npm/README.md

@@ -1,80 +0,0 @@
-# @vectorize-io/hindsight-all
-
-Node.js equivalent of the Python [`hindsight-all`](https://pypi.org/project/hindsight-all/) package — programmatic lifecycle manager for a local Hindsight daemon. Use this when you want to embed Hindsight in a Node application without hand-rolling subprocess management.
-
-This package deliberately does **not** ship an HTTP client. Once the daemon is running, talk to it with [`@vectorize-io/hindsight-client`](https://www.npmjs.com/package/@vectorize-io/hindsight-client) against `server.getBaseUrl()`. The two packages compose — one owns the daemon process, the other owns the HTTP API surface.
-
-## Requirements
-
-- **Node.js >= 22** — uses global `fetch` and `AbortSignal.timeout`.
-- **`uv` / `uvx`** on `PATH` — used to download and run the underlying `hindsight-embed` daemon on first use. Install via <https://docs.astral.sh/uv/>.
-
-## Install
-
-```bash
-npm install @vectorize-io/hindsight-all @vectorize-io/hindsight-client
-```
-
-## Example
-
-```ts
-import { HindsightServer, consoleLogger } from "@vectorize-io/hindsight-all";
-import { HindsightClient } from "@vectorize-io/hindsight-client";
-
-const server = new HindsightServer({
-  profile: "my-app",
-  port: 9077,
-  env: {
-    HINDSIGHT_API_LLM_PROVIDER: "anthropic",
-    HINDSIGHT_API_LLM_API_KEY: process.env.ANTHROPIC_API_KEY,
-    HINDSIGHT_API_LLM_MODEL: "claude-sonnet-4-20250514",
-    HINDSIGHT_EMBED_DAEMON_IDLE_TIMEOUT: "0",
-  },
-  logger: consoleLogger,
-});
-
-await server.start();
-
-const client = new HindsightClient({ baseUrl: server.getBaseUrl() });
-
-await client.retain("user-123", "User prefers dark mode and concise answers.", {
-  documentId: "pref-2026-04-01",
-});
-
-const recall = await client.recall("user-123", "what are the user preferences?");
-console.log(recall.results);
-
-await server.stop();
-```
-
-For a remote Hindsight API, skip `HindsightServer` entirely and just point `HindsightClient` at the remote URL.
-
-## Open config — forward-compatible with new daemon flags
-
-`HindsightServerOptions` is designed so every new environment variable or CLI flag in the underlying Hindsight daemon can be used without waiting for a wrapper release:
-
-- **`env`** accepts an arbitrary `Record<string, string>`. Every entry is exported into the daemon process and written into the profile config via `--env KEY=VALUE`.
-- **`extraProfileCreateArgs`** / **`extraDaemonStartArgs`** append raw args to the respective commands.
-
-## Development against a local checkout
-
-If you're hacking on the Python `hindsight-embed` package in the same monorepo, point the server at the local path — it'll use `uv run --directory <path>` instead of `uvx`:
-
-```ts
-new HindsightServer({
-  embedPackagePath: "/path/to/hindsight-embed",
-  // ...
-});
-```
-
-## API surface
-
-- `HindsightServer` — daemon lifecycle (`start`, `stop`, `checkHealth`, `getBaseUrl`, `getProfile`).
-- `Logger` interface plus `silentLogger` (default) and `consoleLogger` helpers.
-- `getEmbedCommand(opts)` — low-level helper that returns the `[cmd, ...args]` tuple used to invoke the underlying Python CLI.
-
-For memory operations (retain, recall, reflect, bank management, stats) use [`@vectorize-io/hindsight-client`](https://www.npmjs.com/package/@vectorize-io/hindsight-client).
-
-## License
-
-MIT

hindsight-all-npm/package.json

@@ -1,57 +0,0 @@
-{
-  "name": "@vectorize-io/hindsight-all",
-  "version": "0.8.4",
-  "description": "Node.js programmatic lifecycle manager for Hindsight — embeds a local hindsight daemon in a Node application. Pair with @vectorize-io/hindsight-client for memory operations.",
-  "main": "dist/index.js",
-  "types": "dist/index.d.ts",
-  "type": "module",
-  "exports": {
-    ".": {
-      "types": "./dist/index.d.ts",
-      "import": "./dist/index.js"
-    }
-  },
-  "keywords": [
-    "hindsight",
-    "hindsight-all",
-    "memory",
-    "ai",
-    "agent",
-    "long-term-memory",
-    "llm",
-    "embedded-server"
-  ],
-  "author": "Vectorize <[email protected]>",
-  "license": "MIT",
-  "repository": {
-    "type": "git",
-    "url": "https://github.com/vectorize-io/hindsight.git",
-    "directory": "hindsight-all-npm"
-  },
-  "files": [
-    "dist",
-    "README.md"
-  ],
-  "scripts": {
-    "build": "tsup",
-    "dev": "tsup --watch",
-    "clean": "rm -rf dist",
-    "test": "vitest run src",
-    "test:watch": "vitest src",
-    "prepublishOnly": "npm run clean && npm run build"
-  },
-  "devDependencies": {
-    "@types/node": "^22.0.0",
-    "tsup": "^8.5.1",
-    "typescript": "^5.7.0",
-    "vitest": "^4.1.2"
-  },
-  "engines": {
-    "node": ">=22"
-  },
-  "overrides": {
-    "rollup": "^4.59.0",
-    "picomatch": ">=2.3.2 <3.0.0 || >=4.0.4",
-    "vite": ">=8.0.5"
-  }
-}

hindsight-all-npm/src/command.test.ts

@@ -1,36 +0,0 @@
-import { describe, it, expect } from "vitest";
-import { getEmbedCommand } from "./command.js";
-
-describe("getEmbedCommand", () => {
-  it("defaults to uvx hindsight-embed@latest", () => {
-    expect(getEmbedCommand()).toEqual(["uvx", "hindsight-embed@latest"]);
-  });
-
-  it("honours an explicit version", () => {
-    expect(getEmbedCommand({ embedVersion: "0.5.0" })).toEqual(["uvx", "[email protected]"]);
-  });
-
-  it("treats an empty version as latest", () => {
-    expect(getEmbedCommand({ embedVersion: "" })).toEqual(["uvx", "hindsight-embed@latest"]);
-  });
-
-  it("uses uv run --directory when a local path is given", () => {
-    expect(getEmbedCommand({ embedPackagePath: "/abs/path" })).toEqual([
-      "uv",
-      "run",
-      "--directory",
-      "/abs/path",
-      "hindsight-embed",
-    ]);
-  });
-
-  it("local path takes precedence over version", () => {
-    expect(getEmbedCommand({ embedPackagePath: "/abs/path", embedVersion: "0.5.0" })).toEqual([
-      "uv",
-      "run",
-      "--directory",
-      "/abs/path",
-      "hindsight-embed",
-    ]);
-  });
-});

hindsight-all-npm/src/command.ts

@@ -1,25 +0,0 @@
-/**
- * Resolve the command that invokes the `hindsight-embed` Python CLI.
- *
- * - If `embedPackagePath` is set, runs the package from a local checkout via
- *   `uv run --directory <path> hindsight-embed`. Used for in-repo development.
- * - Otherwise runs it via `uvx hindsight-embed@<version>` so no global install
- *   is required.
- *
- * Returns the argv as `[command, ...baseArgs]` suitable for `spawn()` /
- * `execFile()` (never shell-interpolated).
- */
-export interface EmbedCommandOptions {
-  /** Version spec passed to uvx (e.g. "latest", "0.5.0"). Default: "latest". */
-  embedVersion?: string;
-  /** Local checkout path. When set, overrides `embedVersion` and uses `uv run`. */
-  embedPackagePath?: string;
-}
-
-export function getEmbedCommand(opts: EmbedCommandOptions = {}): string[] {
-  if (opts.embedPackagePath) {
-    return ["uv", "run", "--directory", opts.embedPackagePath, "hindsight-embed"];
-  }
-  const version = opts.embedVersion && opts.embedVersion.length > 0 ? opts.embedVersion : "latest";
-  return ["uvx", `hindsight-embed@${version}`];
-}

hindsight-all-npm/src/index.ts

@@ -1,7 +0,0 @@
-export { HindsightServer } from "./server.js";
-export { getEmbedCommand } from "./command.js";
-export { silentLogger, consoleLogger } from "./logger.js";
-
-export type { Logger } from "./logger.js";
-export type { EmbedCommandOptions } from "./command.js";
-export type { HindsightServerOptions } from "./types.js";

hindsight-all-npm/src/logger.ts

@@ -1,29 +0,0 @@
-/**
- * Pluggable logger interface.
- *
- * This package does not own any logging infrastructure — consumers inject
- * whatever they want (console, pino, openclaw's logger, a no-op). The default
- * is silent so embedding this package never adds noise to an unrelated app.
- */
-export interface Logger {
-  debug(msg: string): void;
-  info(msg: string): void;
-  warn(msg: string): void;
-  error(msg: string): void;
-}
-
-/** Logger that drops every call. Used when no logger is passed. */
-export const silentLogger: Logger = {
-  debug: () => {},
-  info: () => {},
-  warn: () => {},
-  error: () => {},
-};
-
-/** Logger that writes to the standard console. Handy for CLIs and tests. */
-export const consoleLogger: Logger = {
-  debug: (msg) => console.debug(msg),
-  info: (msg) => console.log(msg),
-  warn: (msg) => console.warn(msg),
-  error: (msg) => console.error(msg),
-};

hindsight-all-npm/src/server.test.ts

@@ -1,35 +0,0 @@
-import { describe, it, expect } from "vitest";
-import { HindsightServer } from "./server.js";
-
-describe("HindsightServer construction", () => {
-  it("defaults base URL to http://127.0.0.1:8888", () => {
-    const server = new HindsightServer();
-    expect(server.getBaseUrl()).toBe("http://127.0.0.1:8888");
-    expect(server.getProfile()).toBe("default");
-  });
-
-  it("honours custom profile, port, and host", () => {
-    const server = new HindsightServer({ profile: "app", port: 9077, host: "0.0.0.0" });
-    expect(server.getProfile()).toBe("app");
-    expect(server.getBaseUrl()).toBe("http://0.0.0.0:9077");
-  });
-
-  it("accepts open env pass-through without complaining about unknown keys", () => {
-    const server = new HindsightServer({
-      env: {
-        HINDSIGHT_API_LLM_PROVIDER: "openai",
-        HINDSIGHT_API_LLM_MODEL: "gpt-4o-mini",
-        // A field that does not exist today — should still be accepted
-        HINDSIGHT_FUTURE_FLAG: "enabled",
-      },
-    });
-    expect(server).toBeInstanceOf(HindsightServer);
-  });
-
-  it("exposes checkHealth that returns false when no daemon is running", async () => {
-    // Random high port that nothing is listening on.
-    const server = new HindsightServer({ port: 1, readyTimeoutMs: 100 });
-    const healthy = await server.checkHealth();
-    expect(healthy).toBe(false);
-  });
-});

hindsight-all-npm/src/server.ts

@@ -1,322 +0,0 @@
-import { spawn } from "child_process";
-import { getEmbedCommand } from "./command.js";
-import { silentLogger } from "./logger.js";
-import type { Logger } from "./logger.js";
-import type { HindsightServerOptions } from "./types.js";
-
-const DEFAULT_PORT = 8888;
-const DEFAULT_HOST = "127.0.0.1";
-const DEFAULT_PROFILE = "default";
-const DEFAULT_READY_TIMEOUT_MS = 30_000;
-const DEFAULT_READY_POLL_INTERVAL_MS = 1_000;
-
-/**
- * Manages the lifecycle of a local Hindsight daemon from a Node.js process.
- *
- * On {@link start}, this class:
- *   1. Resolves the `hindsight-embed` command (via `uvx` or a local `uv run`).
- *   2. Runs `profile create <name> --merge --port <port> [--env K=V ...]`
- *      with every entry in {@link HindsightServerOptions.env} forwarded as
- *      an `--env` flag.
- *   3. Runs `daemon --profile <name> start` and waits for the start command
- *      to exit.
- *   4. Polls `http://host:port/health` until it returns `200` or the
- *      `readyTimeoutMs` budget is exhausted.
- *
- * On {@link stop}, it runs `daemon --profile <name> stop` and returns once
- * the command exits (or after a short grace period).
- *
- * This is the Node.js equivalent of the Python `hindsight-all` package's
- * `HindsightServer`: a thin programmatic lifecycle wrapper around the
- * Hindsight daemon. It does NOT ship an HTTP client — once `start()`
- * resolves, use `@vectorize-io/hindsight-client` against `getBaseUrl()` for
- * retain / recall / reflect.
- *
- * The class is deliberately transparent about the daemon: new CLI flags or
- * environment variables never require a code change here — callers can pass
- * them via `env`, `extraProfileCreateArgs`, or `extraDaemonStartArgs`.
- */
-export class HindsightServer {
-  private readonly profile: string;
-  private readonly port: number;
-  private readonly host: string;
-  private readonly baseUrl: string;
-  private readonly embedVersion: string | undefined;
-  private readonly embedPackagePath: string | undefined;
-  private readonly userEnv: Record<string, string | undefined>;
-  private readonly extraProfileCreateArgs: string[];
-  private readonly extraDaemonStartArgs: string[];
-  private readonly platformCpuWorkaround: boolean;
-  private readonly readyTimeoutMs: number;
-  private readonly readyPollIntervalMs: number;
-  private readonly logger: Logger;
-
-  constructor(opts: HindsightServerOptions = {}) {
-    this.profile = opts.profile ?? DEFAULT_PROFILE;
-    this.port = opts.port ?? DEFAULT_PORT;
-    this.host = opts.host ?? DEFAULT_HOST;
-    this.baseUrl = `http://${this.host}:${this.port}`;
-    this.embedVersion = opts.embedVersion;
-    this.embedPackagePath = opts.embedPackagePath;
-    this.userEnv = opts.env ?? {};
-    this.extraProfileCreateArgs = opts.extraProfileCreateArgs ?? [];
-    this.extraDaemonStartArgs = opts.extraDaemonStartArgs ?? [];
-    this.platformCpuWorkaround = opts.platformCpuWorkaround ?? process.platform === "darwin";
-    this.readyTimeoutMs = opts.readyTimeoutMs ?? DEFAULT_READY_TIMEOUT_MS;
-    this.readyPollIntervalMs = opts.readyPollIntervalMs ?? DEFAULT_READY_POLL_INTERVAL_MS;
-    this.logger = opts.logger ?? silentLogger;
-  }
-
-  /** The base URL the daemon listens on (`http://host:port`). */
-  getBaseUrl(): string {
-    return this.baseUrl;
-  }
-
-  /** The profile name this server operates on. */
-  getProfile(): string {
-    return this.profile;
-  }
-
-  /**
-   * Ensure the daemon is configured and running. Idempotent — the underlying
-   * `profile create --merge` and `daemon start` commands tolerate re-runs.
-   */
-  async start(): Promise<void> {
-    this.logger.info(`[hindsight] starting daemon for profile "${this.profile}"`);
-
-    const env = this.buildEnv();
-    await this.configureProfile(env);
-    await this.startDaemon(env);
-    await this.waitForReady();
-
-    this.logger.info(`[hindsight] daemon ready at ${this.baseUrl}`);
-  }
-
-  /** Stop the daemon. Never throws — logs and resolves even on failure. */
-  async stop(): Promise<void> {
-    this.logger.info(`[hindsight] stopping daemon for profile "${this.profile}"`);
-
-    const [cmd, ...baseArgs] = getEmbedCommand({
-      embedVersion: this.embedVersion,
-      embedPackagePath: this.embedPackagePath,
-    });
-    const args = [...baseArgs, "daemon", "--profile", this.profile, "stop"];
-
-    const child = spawn(cmd, args, { stdio: "pipe" });
-    this.pipeOutput(child, "daemon.stop");
-
-    await new Promise<void>((resolve) => {
-      const timeout = setTimeout(() => {
-        this.logger.warn(`[hindsight] daemon stop timed out after 5s`);
-        resolve();
-      }, 5_000);
-      child.on("exit", () => {
-        clearTimeout(timeout);
-        this.logger.info(`[hindsight] daemon stopped`);
-        resolve();
-      });
-      child.on("error", (err) => {
-        clearTimeout(timeout);
-        this.logger.warn(`[hindsight] error stopping daemon: ${err.message}`);
-        resolve();
-      });
-    });
-  }
-
-  /** Probe `/health` once with a short timeout. */
-  async checkHealth(): Promise<boolean> {
-    try {
-      const res = await fetch(`${this.baseUrl}/health`, {
-        signal: AbortSignal.timeout(2_000),
-      });
-      return res.ok;
-    } catch {
-      return false;
-    }
-  }
-
-  // -------------------------------------------------------------------------
-  // Internal
-  // -------------------------------------------------------------------------
-
-  /**
-   * Merge the process env, the caller-supplied `env`, and (on macOS) the
-   * embeddings CPU workaround. Caller-supplied values always win over the
-   * workaround; undefined values are dropped.
-   */
-  private buildEnv(): NodeJS.ProcessEnv {
-    const merged: NodeJS.ProcessEnv = { ...process.env };
-
-    if (this.platformCpuWorkaround && process.platform === "darwin") {
-      merged["HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU"] = "1";
-      merged["HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU"] = "1";
-    }
-
-    for (const [key, value] of Object.entries(this.userEnv)) {
-      if (value !== undefined) {
-        merged[key] = value;
-      }
-    }
-
-    return merged;
-  }
-
-  /**
-   * Run `profile create <name> --merge --port <port> [--env K=V ...]`.
-   * Every entry in the merged env that was passed via {@link userEnv} (or
-   * auto-applied by the CPU workaround) is forwarded as `--env`.
-   */
-  private async configureProfile(env: NodeJS.ProcessEnv): Promise<void> {
-    this.logger.info(`[hindsight] configuring profile "${this.profile}"`);
-
-    const [cmd, ...baseArgs] = getEmbedCommand({
-      embedVersion: this.embedVersion,
-      embedPackagePath: this.embedPackagePath,
-    });
-    const createArgs = [
-      ...baseArgs,
-      "profile",
-      "create",
-      this.profile,
-      "--merge",
-      "--port",
-      String(this.port),
-    ];
-
-    // Forward every env var that the caller intended for the daemon as --env.
-    // We only forward keys the caller explicitly set (userEnv) plus the CPU
-    // workaround values — not the entire process.env, to avoid leaking random
-    // host state into profile config.
-    const envForProfile = this.collectProfileEnv(env);
-    for (const [key, value] of Object.entries(envForProfile)) {
-      createArgs.push("--env", `${key}=${value}`);
-    }
-
-    createArgs.push(...this.extraProfileCreateArgs);
-
-    await this.runCommand(cmd, createArgs, env, "profile.create");
-  }
-
-  /** Collect only the env vars that should be written into the profile file. */
-  private collectProfileEnv(env: NodeJS.ProcessEnv): Record<string, string> {
-    const out: Record<string, string> = {};
-
-    // 1. User-supplied env — always forwarded.
-    for (const [key, value] of Object.entries(this.userEnv)) {
-      if (value !== undefined) {
-        out[key] = value;
-      }
-    }
-
-    // 2. CPU workaround — only if auto-applied and not already overridden.
-    if (this.platformCpuWorkaround && process.platform === "darwin") {
-      const cpuKeys = [
-        "HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU",
-        "HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU",
-      ];
-      for (const key of cpuKeys) {
-        if (!(key in out) && env[key] !== undefined) {
-          out[key] = env[key] as string;
-        }
-      }
-    }
-
-    return out;
-  }
-
-  private async startDaemon(env: NodeJS.ProcessEnv): Promise<void> {
-    const [cmd, ...baseArgs] = getEmbedCommand({
-      embedVersion: this.embedVersion,
-      embedPackagePath: this.embedPackagePath,
-    });
-    const args = [
-      ...baseArgs,
-      "daemon",
-      "--profile",
-      this.profile,
-      "start",
-      ...this.extraDaemonStartArgs,
-    ];
-
-    await this.runCommand(cmd, args, env, "daemon.start");
-  }
-
-  /**
-   * Spawn `cmd` with `args`, pipe its output through the logger, and resolve
-   * once it exits with code 0. Rejects on non-zero exit or spawn error.
-   */
-  private async runCommand(
-    cmd: string,
-    args: string[],
-    env: NodeJS.ProcessEnv,
-    label: string
-  ): Promise<void> {
-    const child = spawn(cmd, args, { stdio: "pipe", env });
-    let output = "";
-    child.stdout?.on("data", (data: Buffer) => {
-      const text = data.toString();
-      output += text;
-      for (const line of text.trimEnd().split("\n")) {
-        if (line) this.logger.info(`[hindsight:${label}] ${line}`);
-      }
-    });
-    child.stderr?.on("data", (data: Buffer) => {
-      const text = data.toString();
-      output += text;
-      for (const line of text.trimEnd().split("\n")) {
-        if (line) this.logger.warn(`[hindsight:${label}] ${line}`);
-      }
-    });
-
-    await new Promise<void>((resolve, reject) => {
-      child.on("exit", (code) => {
-        if (code === 0) {
-          resolve();
-        } else {
-          reject(new Error(`${label} failed with code ${code}: ${output.trim()}`));
-        }
-      });
-      child.on("error", (err) => {
-        reject(new Error(`${label} failed to spawn: ${err.message}`, { cause: err }));
-      });
-    });
-  }
-
-  /** Stream a spawned child's stdout/stderr through the logger without blocking. */
-  private pipeOutput(child: ReturnType<typeof spawn>, label: string): void {
-    child.stdout?.on("data", (data: Buffer) => {
-      for (const line of data.toString().trimEnd().split("\n")) {
-        if (line) this.logger.info(`[hindsight:${label}] ${line}`);
-      }
-    });
-    child.stderr?.on("data", (data: Buffer) => {
-      for (const line of data.toString().trimEnd().split("\n")) {
-        if (line) this.logger.warn(`[hindsight:${label}] ${line}`);
-      }
-    });
-  }
-
-  /** Poll `/health` until it succeeds or `readyTimeoutMs` elapses. */
-  private async waitForReady(): Promise<void> {
-    const deadline = Date.now() + this.readyTimeoutMs;
-    let attempt = 0;
-    while (Date.now() < deadline) {
-      attempt++;
-      try {
-        const res = await fetch(`${this.baseUrl}/health`, {
-          signal: AbortSignal.timeout(this.readyPollIntervalMs),
-        });
-        if (res.ok) {
-          this.logger.debug(`[hindsight] health check passed (attempt ${attempt})`);
-          return;
-        }
-      } catch {
-        // expected while the daemon is still booting
-      }
-      await new Promise((resolve) => setTimeout(resolve, this.readyPollIntervalMs));
-    }
-    throw new Error(
-      `Hindsight daemon did not become ready within ${this.readyTimeoutMs}ms at ${this.baseUrl}`
-    );
-  }
-}

hindsight-all-npm/src/types.ts

@@ -1,54 +0,0 @@
-import type { Logger } from "./logger.js";
-
-/**
- * Options for {@link HindsightServer}.
- *
- * The server is intentionally thin and pass-through: anything configurable
- * on the daemon side (env vars or CLI flags) can be set here without needing
- * a new dedicated option. Use {@link env} for `HINDSIGHT_*` / `OPENAI_API_KEY` /
- * custom provider settings, and the two `extra*` arrays to append raw CLI
- * args to `profile create` or `daemon start`.
- *
- * For talking to the daemon after `start()`, use `@vectorize-io/hindsight-client`
- * against `server.getBaseUrl()`. This package does not ship its own HTTP
- * client.
- */
-export interface HindsightServerOptions {
-  /** Profile name used for `--profile <name>` on every sub-command. Default: `"default"`. */
-  profile?: string;
-  /** TCP port the daemon listens on. Default: `8888`. */
-  port?: number;
-  /** Hostname the daemon binds to (for health checks). Default: `127.0.0.1`. */
-  host?: string;
-  /** Version of the underlying `hindsight-embed` PyPI package to run via `uvx`. Default: `"latest"`. */
-  embedVersion?: string;
-  /** Local path to a `hindsight-embed` checkout — takes precedence over `embedVersion`. */
-  embedPackagePath?: string;
-  /**
-   * Environment variables passed to the daemon process AND written into the
-   * profile via repeated `--env KEY=VALUE` flags. This is the preferred way
-   * to surface any `HINDSIGHT_API_*` / `HINDSIGHT_EMBED_*` setting — adding a
-   * new daemon env var never requires a wrapper update.
-   *
-   * Values of `undefined` are dropped (so you can spread conditionally).
-   */
-  env?: Record<string, string | undefined>;
-  /** Extra args appended verbatim to `hindsight-embed profile create <name> --merge ...`. */
-  extraProfileCreateArgs?: string[];
-  /** Extra args appended verbatim to `hindsight-embed daemon --profile <name> start ...`. */
-  extraDaemonStartArgs?: string[];
-  /**
-   * On macOS, automatically set
-   * `HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU=1` and
-   * `HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU=1` to avoid Metal/MPS crashes in
-   * daemon mode. Default: `true` on `darwin`, ignored elsewhere. Any value set
-   * explicitly in {@link env} wins over the auto-applied value.
-   */
-  platformCpuWorkaround?: boolean;
-  /** Max time (ms) to wait for `/health` to return 200. Default: `30_000`. */
-  readyTimeoutMs?: number;
-  /** Polling interval (ms) while waiting for `/health`. Default: `1_000`. */
-  readyPollIntervalMs?: number;
-  /** Optional pluggable logger. Default: silent. */
-  logger?: Logger;
-}

hindsight-all-npm/tsconfig.json

@@ -1,18 +0,0 @@
-{
-  "compilerOptions": {
-    "target": "ES2022",
-    "module": "ES2022",
-    "lib": ["ES2022"],
-    "moduleResolution": "node",
-    "declaration": true,
-    "outDir": "./dist",
-    "rootDir": "./src",
-    "strict": true,
-    "esModuleInterop": true,
-    "skipLibCheck": true,
-    "forceConsistentCasingInFileNames": true,
-    "resolveJsonModule": true
-  },
-  "include": ["src/**/*"],
-  "exclude": ["node_modules", "dist", "src/**/*.test.ts"]
-}

hindsight-all-npm/tsup.config.ts

@@ -1,11 +0,0 @@
-import { defineConfig } from "tsup";
-
-export default defineConfig({
-  entry: ["src/index.ts"],
-  format: ["esm"],
-  dts: true,
-  outDir: "dist",
-  clean: true,
-  sourcemap: true,
-  bundle: true,
-});

hindsight-all-npm/vitest.config.ts

@@ -1,8 +0,0 @@
-import { defineConfig } from "vitest/config";
-
-export default defineConfig({
-  test: {
-    include: ["src/**/*.test.ts"],
-    environment: "node",
-  },
-});

hindsight-all-slim/pyproject.toml

@@ -1,33 +0,0 @@
-[build-system]
-requires = ["setuptools>=61"]
-build-backend = "setuptools.build_meta"
-
-[project]
-name = "hindsight-all-slim"
-version = "0.8.4"
-description = "Hindsight: Agent Memory That Works Like Human Memory - Slim All-in-One Bundle"
-readme = "README.md"
-requires-python = ">=3.11"
-dependencies = [
-    "hindsight-api-slim==0.8.4",
-    "hindsight-client>=0.0.7",
-    "hindsight-embed>=0.1.0",
-]
-
-[tool.uv.sources]
-hindsight-api-slim = { workspace = true }
-hindsight-client = { workspace = true }
-hindsight-embed = { workspace = true }
-
-[project.optional-dependencies]
-test = [
-    "pytest>=7.0.0",
-    "pytest-asyncio>=0.21.0",
-]
-
-[tool.setuptools]
-packages = []
-
-[tool.pytest.ini_options]
-asyncio_mode = "auto"
-asyncio_default_fixture_loop_scope = "function"

hindsight-all/README.md

@@ -1,48 +0,0 @@
-# hindsight-all
-
-All-in-one package for Hindsight - Agent Memory That Works Like Human Memory
-
-## Quick Start
-
-```python
-from hindsight import start_server, HindsightClient
-
-# Start server with embedded PostgreSQL
-server = start_server(
-    llm_provider="groq",
-    llm_api_key="your-api-key",
-    llm_model="openai/gpt-oss-120b"
-)
-
-# Create client
-client = HindsightClient(base_url=server.url)
-
-# Store memories
-client.put(agent_id="assistant", content="User prefers Python for data analysis")
-
-# Search memories
-results = client.search(agent_id="assistant", query="programming preferences")
-
-# Generate contextual response
-response = client.think(agent_id="assistant", query="What languages should I recommend?")
-
-# Stop server when done
-server.stop()
-```
-
-## Using Context Manager
-
-```python
-from hindsight import HindsightServer, HindsightClient
-
-with HindsightServer(llm_provider="groq", llm_api_key="...") as server:
-    client = HindsightClient(base_url=server.url)
-    # ... use client ...
-# Server automatically stops
-```
-
-## Installation
-
-```bash
-pip install hindsight-all
-```

hindsight-all/hindsight/api_namespaces.py

@@ -1,193 +0,0 @@
-"""
-API namespace classes for organizing client methods.
-
-These classes provide organized access to different parts of the Hindsight API
-while ensuring the daemon is running before each call.
-"""
-
-from __future__ import annotations
-
-from typing import TYPE_CHECKING, Any
-
-if TYPE_CHECKING:
-    from .embedded import HindsightEmbedded
-
-
-class BanksAPI:
-    """Namespace for bank-related operations."""
-
-    def __init__(self, embedded: "HindsightEmbedded"):
-        self._embedded = embedded
-
-    def create(
-        self,
-        bank_id: str,
-        name: str | None = None,
-        mission: str | None = None,
-        disposition: dict[str, Any] | None = None,
-    ):
-        """Create a new bank."""
-        self._embedded._ensure_started()
-        return self._embedded._client.create_bank(
-            bank_id=bank_id,
-            name=name,
-            mission=mission,
-            disposition=disposition,
-        )
-
-    def delete(self, bank_id: str):
-        """Delete a bank."""
-        self._embedded._ensure_started()
-        return self._embedded._client.delete_bank(bank_id=bank_id)
-
-    def set_mission(self, bank_id: str, mission: str):
-        """Set or update the mission for a bank."""
-        self._embedded._ensure_started()
-        return self._embedded._client.set_mission(bank_id=bank_id, mission=mission)
-
-    def set_disposition(self, bank_id: str, disposition: dict[str, Any]):
-        """Set or update the disposition for a bank."""
-        self._embedded._ensure_started()
-        return self._embedded._client.set_disposition(bank_id=bank_id, disposition=disposition)
-
-
-class MentalModelsAPI:
-    """Namespace for mental model operations."""
-
-    def __init__(self, embedded: "HindsightEmbedded"):
-        self._embedded = embedded
-
-    def create(
-        self,
-        bank_id: str,
-        name: str,
-        content: str,
-        tags: list[str] | None = None,
-    ):
-        """Create a new mental model."""
-        self._embedded._ensure_started()
-        return self._embedded._client.create_mental_model(
-            bank_id=bank_id,
-            name=name,
-            content=content,
-            tags=tags,
-        )
-
-    def list(self, bank_id: str, tags: list[str] | None = None):
-        """List all mental models for a bank."""
-        self._embedded._ensure_started()
-        return self._embedded._client.list_mental_models(bank_id=bank_id, tags=tags)
-
-    def get(self, bank_id: str, mental_model_id: str):
-        """Get a specific mental model."""
-        self._embedded._ensure_started()
-        return self._embedded._client.get_mental_model(bank_id=bank_id, mental_model_id=mental_model_id)
-
-    def refresh(self, bank_id: str, mental_model_id: str):
-        """Refresh a mental model."""
-        self._embedded._ensure_started()
-        return self._embedded._client.refresh_mental_model(bank_id=bank_id, mental_model_id=mental_model_id)
-
-    def update(
-        self,
-        bank_id: str,
-        mental_model_id: str,
-        name: str | None = None,
-        content: str | None = None,
-        tags: list[str] | None = None,
-    ):
-        """Update a mental model."""
-        self._embedded._ensure_started()
-        return self._embedded._client.update_mental_model(
-            bank_id=bank_id,
-            mental_model_id=mental_model_id,
-            name=name,
-            content=content,
-            tags=tags,
-        )
-
-    def delete(self, bank_id: str, mental_model_id: str):
-        """Delete a mental model."""
-        self._embedded._ensure_started()
-        return self._embedded._client.delete_mental_model(bank_id=bank_id, mental_model_id=mental_model_id)
-
-
-class DirectivesAPI:
-    """Namespace for directive operations."""
-
-    def __init__(self, embedded: "HindsightEmbedded"):
-        self._embedded = embedded
-
-    def create(
-        self,
-        bank_id: str,
-        name: str,
-        content: str,
-        tags: list[str] | None = None,
-    ):
-        """Create a new directive."""
-        self._embedded._ensure_started()
-        return self._embedded._client.create_directive(
-            bank_id=bank_id,
-            name=name,
-            content=content,
-            tags=tags,
-        )
-
-    def list(self, bank_id: str, tags: list[str] | None = None):
-        """List all directives for a bank."""
-        self._embedded._ensure_started()
-        return self._embedded._client.list_directives(bank_id=bank_id, tags=tags)
-
-    def get(self, bank_id: str, directive_id: str):
-        """Get a specific directive."""
-        self._embedded._ensure_started()
-        return self._embedded._client.get_directive(bank_id=bank_id, directive_id=directive_id)
-
-    def update(
-        self,
-        bank_id: str,
-        directive_id: str,
-        name: str | None = None,
-        content: str | None = None,
-        tags: list[str] | None = None,
-    ):
-        """Update a directive."""
-        self._embedded._ensure_started()
-        return self._embedded._client.update_directive(
-            bank_id=bank_id,
-            directive_id=directive_id,
-            name=name,
-            content=content,
-            tags=tags,
-        )
-
-    def delete(self, bank_id: str, directive_id: str):
-        """Delete a directive."""
-        self._embedded._ensure_started()
-        return self._embedded._client.delete_directive(bank_id=bank_id, directive_id=directive_id)
-
-
-class MemoriesAPI:
-    """Namespace for memory operations."""
-
-    def __init__(self, embedded: "HindsightEmbedded"):
-        self._embedded = embedded
-
-    def list(
-        self,
-        bank_id: str,
-        type: str | None = None,
-        search_query: str | None = None,
-        limit: int = 100,
-        offset: int = 0,
-    ):
-        """List memories in a bank."""
-        self._embedded._ensure_started()
-        return self._embedded._client.list_memories(
-            bank_id=bank_id,
-            type=type,
-            search_query=search_query,
-            limit=limit,
-            offset=offset,
-        )

hindsight-all/hindsight/client_wrapper.py

@@ -1,423 +0,0 @@
-"""
-Wrapper for Hindsight client that adds API namespaces.
-
-Provides organized access to different parts of the Hindsight API through
-namespaces like .banks, .mental_models, etc.
-"""
-
-from __future__ import annotations
-
-from typing import Any
-
-from hindsight_client import Hindsight
-
-
-class BanksAPI:
-    """Namespace for bank-related operations.
-    
-    Provides methods to create, delete, and manage memory banks.
-    """
-
-    def __init__(self, client: Hindsight):
-        self._client = client
-
-    def create(
-        self,
-        bank_id: str,
-        name: str | None = None,
-        mission: str | None = None,
-        disposition: dict[str, Any] | None = None,
-    ) -> Any:
-        """Create a new bank.
-        
-        Args:
-            bank_id: Unique identifier for the bank.
-            name: Optional display name for the bank.
-            mission: Optional mission statement for the bank.
-            disposition: Optional disposition configuration dict.
-            
-        Returns:
-            Bank creation response from the API.
-        """
-        return self._client.create_bank(
-            bank_id=bank_id,
-            name=name,
-            mission=mission,
-            disposition=disposition,
-        )
-
-    def delete(self, bank_id: str) -> Any:
-        """Delete a bank.
-        
-        Args:
-            bank_id: The ID of the bank to delete.
-            
-        Returns:
-            Deletion response from the API.
-        """
-        return self._client.delete_bank(bank_id=bank_id)
-
-    def set_mission(self, bank_id: str, mission: str) -> Any:
-        """Set or update the mission for a bank.
-        
-        Args:
-            bank_id: The ID of the bank.
-            mission: The mission statement to set.
-            
-        Returns:
-            API response confirming the update.
-        """
-        return self._client.set_mission(bank_id=bank_id, mission=mission)
-
-    def set_disposition(self, bank_id: str, disposition: dict[str, Any]) -> Any:
-        """Set or update the disposition for a bank.
-        
-        Args:
-            bank_id: The ID of the bank.
-            disposition: The disposition configuration dict.
-            
-        Returns:
-            API response confirming the update.
-        """
-        return self._client.set_disposition(bank_id=bank_id, disposition=disposition)
-
-    def list(self) -> Any:
-        """List all banks.
-        
-        Returns:
-            List of banks from the API.
-        """
-        from hindsight_client.hindsight_client import _run_async
-
-        return _run_async(self._client._banks_api.list_banks())
-
-
-class MentalModelsAPI:
-    """Namespace for mental model operations.
-    
-    Mental models are reusable knowledge structures that guide agent behavior.
-    """
-
-    def __init__(self, client: Hindsight):
-        self._client = client
-
-    def create(
-        self,
-        bank_id: str,
-        name: str,
-        content: str,
-        tags: list[str] | None = None,
-    ) -> Any:
-        """Create a new mental model.
-        
-        Args:
-            bank_id: The ID of the bank to add the model to.
-            name: Name for the mental model.
-            content: The content/instructions for the mental model.
-            tags: Optional list of tags for categorization.
-            
-        Returns:
-            Creation response from the API.
-        """
-        return self._client.create_mental_model(
-            bank_id=bank_id,
-            name=name,
-            content=content,
-            tags=tags,
-        )
-
-    def list(self, bank_id: str, tags: list[str] | None = None) -> Any:
-        """List all mental models for a bank.
-        
-        Args:
-            bank_id: The ID of the bank.
-            tags: Optional filter by tags.
-            
-        Returns:
-            List of mental models.
-        """
-        return self._client.list_mental_models(bank_id=bank_id, tags=tags)
-
-    def get(self, bank_id: str, mental_model_id: str) -> Any:
-        """Get a specific mental model.
-        
-        Args:
-            bank_id: The ID of the bank.
-            mental_model_id: The ID of the mental model.
-            
-        Returns:
-            The mental model details.
-        """
-        return self._client.get_mental_model(bank_id=bank_id, mental_model_id=mental_model_id)
-
-    def refresh(self, bank_id: str, mental_model_id: str) -> Any:
-        """Refresh a mental model.
-        
-        Args:
-            bank_id: The ID of the bank.
-            mental_model_id: The ID of the mental model to refresh.
-            
-        Returns:
-            Refresh response from the API.
-        """
-        return self._client.refresh_mental_model(bank_id=bank_id, mental_model_id=mental_model_id)
-
-    def update(
-        self,
-        bank_id: str,
-        mental_model_id: str,
-        name: str | None = None,
-        content: str | None = None,
-        tags: list[str] | None = None,
-    ) -> Any:
-        """Update a mental model.
-        
-        Args:
-            bank_id: The ID of the bank.
-            mental_model_id: The ID of the mental model to update.
-            name: Optional new name.
-            content: Optional new content.
-            tags: Optional new tags list.
-            
-        Returns:
-            Update response from the API.
-        """
-        return self._client.update_mental_model(
-            bank_id=bank_id,
-            mental_model_id=mental_model_id,
-            name=name,
-            content=content,
-            tags=tags,
-        )
-
-    def delete(self, bank_id: str, mental_model_id: str) -> Any:
-        """Delete a mental model.
-        
-        Args:
-            bank_id: The ID of the bank.
-            mental_model_id: The ID of the mental model to delete.
-            
-        Returns:
-            Deletion response from the API.
-        """
-        return self._client.delete_mental_model(bank_id=bank_id, mental_model_id=mental_model_id)
-
-
-class DirectivesAPI:
-    """Namespace for directive operations.
-    
-    Directives are explicit instructions that guide agent behavior.
-    """
-
-    def __init__(self, client: Hindsight):
-        self._client = client
-
-    def create(
-        self,
-        bank_id: str,
-        name: str,
-        content: str,
-        tags: list[str] | None = None,
-    ) -> Any:
-        """Create a new directive.
-        
-        Args:
-            bank_id: The ID of the bank to add the directive to.
-            name: Name for the directive.
-            content: The directive content/instructions.
-            tags: Optional list of tags for categorization.
-            
-        Returns:
-            Creation response from the API.
-        """
-        return self._client.create_directive(
-            bank_id=bank_id,
-            name=name,
-            content=content,
-            tags=tags,
-        )
-
-    def list(self, bank_id: str, tags: list[str] | None = None) -> Any:
-        """List all directives for a bank.
-        
-        Args:
-            bank_id: The ID of the bank.
-            tags: Optional filter by tags.
-            
-        Returns:
-            List of directives.
-        """
-        return self._client.list_directives(bank_id=bank_id, tags=tags)
-
-    def get(self, bank_id: str, directive_id: str) -> Any:
-        """Get a specific directive.
-        
-        Args:
-            bank_id: The ID of the bank.
-            directive_id: The ID of the directive.
-            
-        Returns:
-            The directive details.
-        """
-        return self._client.get_directive(bank_id=bank_id, directive_id=directive_id)
-
-    def update(
-        self,
-        bank_id: str,
-        directive_id: str,
-        name: str | None = None,
-        content: str | None = None,
-        tags: list[str] | None = None,
-    ) -> Any:
-        """Update a directive.
-        
-        Args:
-            bank_id: The ID of the bank.
-            directive_id: The ID of the directive to update.
-            name: Optional new name.
-            content: Optional new content.
-            tags: Optional new tags list.
-            
-        Returns:
-            Update response from the API.
-        """
-        return self._client.update_directive(
-            bank_id=bank_id,
-            directive_id=directive_id,
-            name=name,
-            content=content,
-            tags=tags,
-        )
-
-    def delete(self, bank_id: str, directive_id: str) -> Any:
-        """Delete a directive.
-        
-        Args:
-            bank_id: The ID of the bank.
-            directive_id: The ID of the directive to delete.
-            
-        Returns:
-            Deletion response from the API.
-        """
-        return self._client.delete_directive(bank_id=bank_id, directive_id=directive_id)
-
-
-class MemoriesAPI:
-    """Namespace for memory operations.
-    
-    Provides methods to query and retrieve stored memories.
-    """
-
-    def __init__(self, client: Hindsight):
-        self._client = client
-
-    def list(
-        self,
-        bank_id: str,
-        type: str | None = None,
-        search_query: str | None = None,
-        limit: int = 100,
-        offset: int = 0,
-    ) -> Any:
-        """List memories in a bank.
-        
-        Args:
-            bank_id: The ID of the bank to query.
-            type: Optional filter by memory type.
-            search_query: Optional search query for filtering.
-            limit: Maximum number of results to return (default: 100).
-            offset: Number of results to skip for pagination (default: 0).
-            
-        Returns:
-            List of memories matching the criteria.
-        """
-        return self._client.list_memories(
-            bank_id=bank_id,
-            type=type,
-            search_query=search_query,
-            limit=limit,
-            offset=offset,
-        )
-
-
-class HindsightClient(Hindsight):
-    """
-    Enhanced Hindsight client with organized API namespaces.
-
-    This wrapper extends the auto-generated Hindsight client with organized
-    access to different parts of the API through namespaces.
-
-    Example:
-        ```python
-        from hindsight import HindsightClient
-
-        client = HindsightClient(base_url="http://localhost:8888")
-
-        # Core operations (inherited from Hindsight)
-        client.retain(bank_id="test", content="Hello")
-        results = client.recall(bank_id="test", query="Hello")
-
-        # Organized API access through namespaces
-        client.banks.create(bank_id="test", name="Test Bank")
-        models = client.mental_models.list(bank_id="test")
-        directives = client.directives.list(bank_id="test")
-        memories = client.memories.list(bank_id="test")
-        ```
-        
-    Attributes:
-        banks: Namespace for bank management operations.
-        mental_models: Namespace for mental model operations.
-        directives: Namespace for directive operations.
-        memories: Namespace for memory listing operations.
-    """
-
-    def __init__(self, *args: Any, **kwargs: Any) -> None:
-        super().__init__(*args, **kwargs)
-        self._banks_namespace: BanksAPI | None = None
-        self._mental_models_namespace: MentalModelsAPI | None = None
-        self._directives_namespace: DirectivesAPI | None = None
-        self._memories_namespace: MemoriesAPI | None = None
-
-    @property
-    def banks(self) -> BanksAPI:
-        """Access bank management operations.
-        
-        Returns:
-            BanksAPI instance for bank operations.
-        """
-        if self._banks_namespace is None:
-            self._banks_namespace = BanksAPI(self)
-        return self._banks_namespace
-
-    @property
-    def mental_models(self) -> MentalModelsAPI:
-        """Access mental model operations.
-        
-        Returns:
-            MentalModelsAPI instance for mental model operations.
-        """
-        if self._mental_models_namespace is None:
-            self._mental_models_namespace = MentalModelsAPI(self)
-        return self._mental_models_namespace
-
-    @property
-    def directives(self) -> DirectivesAPI:
-        """Access directive operations.
-        
-        Returns:
-            DirectivesAPI instance for directive operations.
-        """
-        if self._directives_namespace is None:
-            self._directives_namespace = DirectivesAPI(self)
-        return self._directives_namespace
-
-    @property
-    def memories(self) -> MemoriesAPI:
-        """Access memory listing operations.
-        
-        Returns:
-            MemoriesAPI instance for memory operations.
-        """
-        if self._memories_namespace is None:
-            self._memories_namespace = MemoriesAPI(self)
-        return self._memories_namespace

hindsight-all/hindsight/embedded.py

@@ -1,445 +0,0 @@
-"""
-Embedded Hindsight client with automatic daemon lifecycle management.
-
-This module provides HindsightEmbedded, a client that uses the same daemon
-management interface as hindsight-embed CLI, ensuring full compatibility.
-
-Example:
-    ```python
-    from hindsight import HindsightEmbedded
-
-    # Daemon starts automatically on first use
-    client = HindsightEmbedded(
-        profile="myapp",
-        llm_provider="groq",
-        llm_api_key="your-api-key",
-    )
-
-    # Use just like HindsightClient
-    client.retain(bank_id="alice", content="Alice loves AI")
-    results = client.recall(bank_id="alice", query="What does Alice like?")
-
-    # Optional cleanup
-    client.close()
-    ```
-
-Using context manager:
-    ```python
-    from hindsight import HindsightEmbedded
-
-    with HindsightEmbedded(profile="myapp") as client:
-        client.retain(bank_id="alice", content="Alice loves AI")
-        # Daemon managed automatically
-    ```
-"""
-
-import logging
-import threading
-from typing import Optional
-
-from hindsight_client import Hindsight
-from hindsight_embed import get_embed_manager
-
-from .api_namespaces import BanksAPI, DirectivesAPI, MemoriesAPI, MentalModelsAPI
-
-logger = logging.getLogger(__name__)
-
-
-class HindsightEmbedded:
-    """
-    Hindsight client with automatic daemon lifecycle management.
-
-    This client uses the same daemon management interface as hindsight-embed CLI,
-    ensuring full compatibility and shared profiles. The daemon is started automatically
-    on first use and manages profile-specific databases.
-
-    Profile data is stored in: ~/.pg0/instances/hindsight-embed-{profile}/
-
-    All methods from HindsightClient are available:
-    - retain(), retain_batch()
-    - recall()
-    - reflect()
-    - create_bank(), set_mission(), delete_bank()
-    - create_mental_model(), list_mental_models(), etc.
-    - create_directive(), list_directives(), etc.
-    - And all async variants (aretain, arecall, areflect, etc.)
-
-    Args:
-        profile: Profile name for data isolation (default: "default")
-        llm_provider: LLM provider ("groq", "openai", "ollama", "gemini", "anthropic", "lmstudio")
-        llm_api_key: API key for the LLM provider
-        llm_model: Model name to use
-        llm_base_url: Optional custom base URL for LLM API
-        database_url: Optional database URL override (default: profile-specific pg0)
-        idle_timeout: Seconds before daemon auto-exits when idle (default: 0, disabled)
-        log_level: Daemon log level (default: "info")
-        ui: Whether to start the control plane web UI alongside the daemon (default: False)
-        ui_port: Port for the UI. Defaults to daemon_port + 10000.
-        ui_hostname: Hostname to bind the UI to. Defaults to "0.0.0.0".
-    """
-
-    def __init__(
-        self,
-        profile: str = "default",
-        llm_provider: str = "groq",
-        llm_api_key: str = "",
-        llm_model: str = "openai/gpt-oss-120b",
-        llm_base_url: Optional[str] = None,
-        database_url: Optional[str] = None,
-        idle_timeout: int = 0,
-        log_level: str = "info",
-        ui: bool = False,
-        ui_port: Optional[int] = None,
-        ui_hostname: str = "0.0.0.0",
-    ):
-        """
-        Initialize the embedded client (daemon starts on first use).
-
-        Args:
-            profile: Profile name for data isolation
-            llm_provider: LLM provider
-            llm_api_key: API key for the LLM provider
-            llm_model: Model name to use
-            llm_base_url: Optional custom base URL for LLM API
-            database_url: Optional database URL override
-            idle_timeout: Seconds before daemon auto-exits when idle (0 = disabled)
-            log_level: Daemon log level
-            ui: Whether to start the control plane web UI alongside the daemon
-            ui_port: Port for the UI (defaults to daemon_port + 10000)
-            ui_hostname: Hostname to bind the UI to (defaults to "0.0.0.0")
-        """
-        self.profile = profile
-
-        # Build config dict for daemon (matches CLI format)
-        self.config = {
-            "HINDSIGHT_API_LLM_PROVIDER": llm_provider,
-            "HINDSIGHT_API_LLM_API_KEY": llm_api_key,
-            "HINDSIGHT_API_LLM_MODEL": llm_model,
-            "HINDSIGHT_API_LOG_LEVEL": log_level,
-            "HINDSIGHT_EMBED_DAEMON_IDLE_TIMEOUT": str(idle_timeout),
-        }
-
-        if llm_base_url:
-            self.config["HINDSIGHT_API_LLM_BASE_URL"] = llm_base_url
-
-        if database_url:
-            self.config["HINDSIGHT_EMBED_API_DATABASE_URL"] = database_url
-
-        self._ui = ui
-        self._ui_port = ui_port
-        self._ui_hostname = ui_hostname
-
-        self._client: Optional[Hindsight] = None
-        self._lock = threading.Lock()
-        self._started = False
-        self._closed = False
-        self._manager = get_embed_manager()
-
-        # API namespaces (initialized once, lazily)
-        self._banks_api: Optional[BanksAPI] = None
-        self._mental_models_api: Optional[MentalModelsAPI] = None
-        self._directives_api: Optional[DirectivesAPI] = None
-        self._memories_api: Optional[MemoriesAPI] = None
-
-    def _ensure_started(self):
-        """Ensure daemon is running (thread-safe), restarting if crashed."""
-        if self._started and self._client is not None:
-            if self._manager.is_running(self.profile):
-                return
-            # Daemon crashed — reset state and fall through to restart
-            logger.warning(
-                "Daemon for profile '%s' is no longer responsive, restarting...",
-                self.profile,
-            )
-            try:
-                self._client.close()
-            except Exception:
-                logger.debug("Error closing stale client", exc_info=True)
-            self._client = None
-            self._started = False
-
-        with self._lock:
-            # Double-check after acquiring lock
-            if self._started and self._client is not None:
-                if self._manager.is_running(self.profile):
-                    return
-                logger.warning(
-                    "Daemon for profile '%s' is no longer responsive (lock path), restarting...",
-                    self.profile,
-                )
-                try:
-                    self._client.close()
-                except Exception:
-                    logger.debug("Error closing stale client", exc_info=True)
-                self._client = None
-                self._started = False
-
-            if self._closed:
-                raise RuntimeError(
-                    "Cannot use HindsightEmbedded after it has been closed"
-                )
-
-            # Use embed manager interface for daemon management
-            logger.info(f"Ensuring daemon is running for profile '{self.profile}'...")
-            success = self._manager.ensure_running(self.config, self.profile)
-            if not success:
-                raise RuntimeError(
-                    f"Failed to start daemon for profile '{self.profile}'"
-                )
-
-            # Get daemon URL and create client
-            daemon_url = self._manager.get_url(self.profile)
-            self._client = Hindsight(base_url=daemon_url)
-            self._started = True
-            logger.info(f"Connected to daemon at {daemon_url}")
-
-            # Start UI if requested
-            if self._ui:
-                logger.info(f"Starting UI for profile '{self.profile}'...")
-                ui_started = self._manager.start_ui(
-                    self.profile, self._ui_port, self._ui_hostname
-                )
-                if not ui_started:
-                    logger.warning(f"Failed to start UI for profile '{self.profile}'")
-
-    def _cleanup(self, stop_daemon_on_close: bool = False):
-        """
-        Cleanup client resources (idempotent).
-
-        Args:
-            stop_daemon_on_close: If True, stops the daemon. Otherwise, daemon continues
-                running (it will auto-stop after idle timeout).
-        """
-        if self._closed:
-            return
-
-        acquired = self._lock.acquire(timeout=5.0)
-        if not acquired:
-            # Lock is held by another thread (e.g. _ensure_started).
-            # Mark closed to prevent new operations but skip shared-state
-            # teardown — the daemon's idle timeout handles the rest.
-            logger.warning(
-                "Cleanup lock acquisition timed out for profile '%s'; "
-                "marking closed, daemon will idle-stop on its own",
-                self.profile,
-            )
-            self._closed = True
-            return
-
-        try:
-            if self._closed:
-                return
-
-            if self._client is not None:
-                try:
-                    self._client.close()
-                except Exception:
-                    logger.debug(
-                        "Error closing client for profile '%s'",
-                        self.profile,
-                        exc_info=True,
-                    )
-                self._client = None
-
-            # Stop UI if it was started
-            if self._ui and self._started:
-                logger.info(f"Stopping UI for profile '{self.profile}'...")
-                self._manager.stop_ui(self.profile, self._ui_port)
-
-            # Optionally stop daemon (daemon has idle timeout, so not required)
-            if stop_daemon_on_close and self._started:
-                logger.info(f"Stopping daemon for profile '{self.profile}'...")
-                self._manager.stop(self.profile)
-
-            self._closed = True
-        finally:
-            self._lock.release()
-
-    def close(self, stop_daemon: bool = False):
-        """
-        Explicitly close the client.
-
-        Args:
-            stop_daemon: If True, stops the daemon. Otherwise, daemon continues running
-                and will auto-stop after idle timeout (default: False).
-
-        Note:
-            The daemon may be shared with other clients or the CLI, so stopping it
-            might affect other users. By default, we rely on the daemon's idle timeout.
-        """
-        self._cleanup(stop_daemon_on_close=stop_daemon)
-
-    def __getattr__(self, name: str):
-        """
-        Proxy all method calls to the underlying Hindsight client.
-
-        This allows HindsightEmbedded to expose all HindsightClient methods
-        without manually wrapping each one.
-        """
-        # Ensure server is started (and restart if crashed) before proxying
-        self._ensure_started()
-
-        return getattr(self._client, name)
-
-    def __enter__(self):
-        """Context manager entry - ensures server is started."""
-        self._ensure_started()
-        return self
-
-    def __exit__(self, exc_type, exc_val, exc_tb):
-        """Context manager exit - stops the server."""
-        self.close()
-
-    def __del__(self):
-        """Cleanup on garbage collection."""
-        self._cleanup()
-
-    @property
-    def banks(self) -> BanksAPI:
-        """
-        Access bank management operations.
-
-        Each method call ensures the daemon is running before executing.
-
-        Example:
-            ```python
-            from hindsight import HindsightEmbedded
-
-            embedded = HindsightEmbedded(profile="myapp", ...)
-
-            # Create a bank
-            embedded.banks.create(bank_id="test", name="Test Bank")
-
-            # Set mission
-            embedded.banks.set_mission(bank_id="test", mission="Help users")
-            ```
-        """
-        if self._banks_api is None:
-            self._banks_api = BanksAPI(self)
-        return self._banks_api
-
-    @property
-    def mental_models(self) -> MentalModelsAPI:
-        """
-        Access mental model operations.
-
-        Each method call ensures the daemon is running before executing.
-
-        Example:
-            ```python
-            from hindsight import HindsightEmbedded
-
-            embedded = HindsightEmbedded(profile="myapp", ...)
-
-            # Create a mental model
-            embedded.mental_models.create(
-                bank_id="test",
-                name="User Preferences",
-                content="User prefers dark mode"
-            )
-
-            # List mental models
-            models = embedded.mental_models.list(bank_id="test")
-            ```
-        """
-        if self._mental_models_api is None:
-            self._mental_models_api = MentalModelsAPI(self)
-        return self._mental_models_api
-
-    @property
-    def directives(self) -> DirectivesAPI:
-        """
-        Access directive operations.
-
-        Each method call ensures the daemon is running before executing.
-
-        Example:
-            ```python
-            from hindsight import HindsightEmbedded
-
-            embedded = HindsightEmbedded(profile="myapp", ...)
-
-            # Create a directive
-            embedded.directives.create(
-                bank_id="test",
-                name="Response Style",
-                content="Always be concise and friendly"
-            )
-
-            # List directives
-            directives = embedded.directives.list(bank_id="test")
-            ```
-        """
-        if self._directives_api is None:
-            self._directives_api = DirectivesAPI(self)
-        return self._directives_api
-
-    @property
-    def memories(self) -> MemoriesAPI:
-        """
-        Access memory listing operations.
-
-        Each method call ensures the daemon is running before executing.
-
-        Example:
-            ```python
-            from hindsight import HindsightEmbedded
-
-            embedded = HindsightEmbedded(profile="myapp", ...)
-
-            # List memories
-            memories = embedded.memories.list(
-                bank_id="test",
-                type="world",
-                limit=50
-            )
-            ```
-        """
-        if self._memories_api is None:
-            self._memories_api = MemoriesAPI(self)
-        return self._memories_api
-
-    @property
-    def client(self) -> Hindsight:
-        """
-        Get the underlying Hindsight client for direct access.
-
-        Ensures daemon is started (and restarts it if it has crashed) before
-        returning the client.
-
-        Returns:
-            Hindsight: The underlying client instance
-
-        Example:
-            ```python
-            from hindsight import HindsightEmbedded
-
-            embedded = HindsightEmbedded(profile="myapp", ...)
-
-            client = embedded.client
-            banks = client.list_banks()
-            ```
-        """
-        self._ensure_started()
-        return self._client
-
-    @property
-    def url(self) -> str:
-        """Get the daemon URL (starts daemon if needed)."""
-        self._ensure_started()
-        return self._manager.get_url(self.profile)
-
-    @property
-    def is_running(self) -> bool:
-        """Check if the client is initialized and the daemon is responsive."""
-        return (
-            self._started
-            and not self._closed
-            and self._client is not None
-            and self._manager.is_running(self.profile)
-        )
-
-    @property
-    def ui_url(self) -> str:
-        """Get the UI URL for this profile."""
-        return self._manager.get_ui_url(self.profile)

hindsight-all/tests/test_cleanup_timeout.py

@@ -1,56 +0,0 @@
-"""
-Unit test for _cleanup lock timeout behavior.
-
-Verifies that _cleanup completes even when the lock is held by another thread,
-instead of hanging indefinitely (fixes #952).
-"""
-
-import threading
-import time
-from unittest.mock import MagicMock, patch
-
-import pytest
-
-
-def test_cleanup_completes_when_lock_held():
-    """
-    _cleanup should complete (best-effort) even when self._lock is held
-    by another thread, e.g. during a long _ensure_started call.
-    """
-    with patch.dict("sys.modules", {
-        "hindsight_client": MagicMock(),
-        "hindsight_embed": MagicMock(),
-        "hindsight.api_namespaces": MagicMock(),
-    }):
-        from hindsight.embedded import HindsightEmbedded
-
-        client = HindsightEmbedded.__new__(HindsightEmbedded)
-        client.profile = "test"
-        client._lock = threading.Lock()
-        client._closed = False
-        client._client = None
-        client._started = False
-        client._ui = False
-
-        # Simulate another thread holding the lock
-        client._lock.acquire()
-
-        cleanup_done = threading.Event()
-
-        def run_cleanup():
-            client._cleanup()
-            cleanup_done.set()
-
-        t = threading.Thread(target=run_cleanup)
-        t.start()
-
-        # Cleanup should complete within the timeout (5s) + margin
-        assert cleanup_done.wait(timeout=8.0), (
-            "_cleanup hung instead of timing out on lock acquisition"
-        )
-
-        # Release the lock from the simulating thread
-        client._lock.release()
-        t.join(timeout=1.0)
-
-        assert client._closed, "Client should be marked as closed after cleanup"

hindsight-all/tests/test_embedded.py

@@ -1,442 +0,0 @@
-"""
-Integration tests for HindsightEmbedded client.
-
-Tests the embedded client with automatic server lifecycle management:
-1. Lazy server startup on first use
-2. Server reuse across multiple operations
-3. Context manager support
-4. Method proxying to underlying HindsightClient
-5. Proper cleanup
-
-Note: Each test uses random bank_ids to avoid conflicts and allow safe parallel execution.
-"""
-
-import os
-import uuid
-
-import pytest
-import urllib.request
-import json
-
-from hindsight import HindsightEmbedded
-
-
-@pytest.fixture(scope="session")
-def llm_config():
-    """Get LLM configuration from environment (session-scoped)."""
-    # Try both naming conventions
-    provider = os.getenv("HINDSIGHT_API_LLM_PROVIDER") or os.getenv(
-        "HINDSIGHT_LLM_PROVIDER", "groq"
-    )
-    api_key = os.getenv("HINDSIGHT_API_LLM_API_KEY") or os.getenv(
-        "HINDSIGHT_LLM_API_KEY", ""
-    )
-    model = os.getenv("HINDSIGHT_API_LLM_MODEL") or os.getenv(
-        "HINDSIGHT_LLM_MODEL", "openai/gpt-oss-120b"
-    )
-
-    if not api_key:
-        pytest.skip(
-            "LLM API key not configured. Set HINDSIGHT_API_LLM_API_KEY or HINDSIGHT_LLM_API_KEY."
-        )
-
-    return {
-        "llm_provider": provider,
-        "llm_api_key": api_key,
-        "llm_model": model,
-    }
-
-
-def test_embedded_lazy_start(llm_config):
-    """
-    Test that HindsightEmbedded starts server lazily on first use.
-    """
-    profile = f"test_lazy_{uuid.uuid4().hex[:8]}"
-    bank_id = f"bank_{uuid.uuid4().hex[:8]}"
-
-    # Create client - should NOT start server yet
-    client = HindsightEmbedded(profile=profile, log_level="info", **llm_config)
-    assert not client.is_running, "Server should not be running after initialization"
-
-    # First call should start server
-    result = client.retain(bank_id=bank_id, content="Test content for lazy start")
-
-    # Verify server is now running
-    assert client.is_running, "Server should be running after first call"
-    assert result.success, "Retain should succeed"
-    assert result.items_count >= 1, "Should have stored at least 1 item"
-
-    # Cleanup
-    client.close()
-    assert not client.is_running, "Server should stop after close()"
-
-
-def test_embedded_context_manager(llm_config):
-    """
-    Test HindsightEmbedded with context manager.
-    """
-    profile = f"test_ctx_{uuid.uuid4().hex[:8]}"
-    bank_id = f"bank_{uuid.uuid4().hex[:8]}"
-
-    # Use context manager
-    with HindsightEmbedded(profile=profile, log_level="info", **llm_config) as client:
-        assert client.is_running, "Server should be running inside context"
-
-        # Store memory
-        result = client.retain(bank_id=bank_id, content="Testing context manager")
-        assert result.success, "Retain should succeed"
-
-        # Recall memory
-        recall_results = client.recall(bank_id=bank_id, query="context")
-        assert isinstance(recall_results.results, list), (
-            "Recall should return results list"
-        )
-
-    # Server should be stopped after context exit
-    # Note: We can't check client.is_running here as client is out of scope
-
-
-def test_embedded_complete_workflow(llm_config):
-    """
-    Test complete workflow with HindsightEmbedded.
-
-    This test:
-    1. Creates a client with lazy start
-    2. Creates a memory bank
-    3. Stores multiple memories
-    4. Recalls memories
-    5. Reflects on memories
-    6. Tests cleanup
-    """
-    profile = f"test_workflow_{uuid.uuid4().hex[:8]}"
-    bank_id = f"assistant_{uuid.uuid4().hex[:8]}"
-
-    client = HindsightEmbedded(profile=profile, log_level="info", **llm_config)
-
-    try:
-        # Step 1: Create a memory bank
-        print(f"\n1. Creating memory bank: {bank_id}")
-        bank_response = client.create_bank(
-            bank_id=bank_id,
-            name="Test Assistant",
-            mission="Help with programming tasks",
-        )
-        assert bank_response.bank_id == bank_id
-
-        # Step 2: Store memories (single)
-        print("\n2. Storing single memory...")
-        retain_response = client.retain(
-            bank_id=bank_id,
-            content="User prefers Python for data analysis.",
-            context="Programming preferences",
-        )
-        assert retain_response.success
-        assert retain_response.items_count >= 1
-
-        # Step 3: Store batch memories
-        print("\n3. Storing batch memories...")
-        batch_response = client.retain_batch(
-            bank_id=bank_id,
-            items=[
-                {"content": "User works with pandas and numpy."},
-                {"content": "User likes matplotlib for visualization."},
-                {
-                    "content": "User is interested in machine learning with scikit-learn."
-                },
-            ],
-        )
-        assert batch_response.success
-        assert batch_response.items_count >= 3
-
-        # Step 4: Recall memories
-        print("\n4. Recalling memories...")
-        recall_response = client.recall(
-            bank_id=bank_id, query="What tools does the user prefer?", max_tokens=2000
-        )
-        assert isinstance(recall_response.results, list)
-        assert len(recall_response.results) > 0
-        print(f"   Found {len(recall_response.results)} relevant memories")
-
-        # Step 5: Reflect on memories
-        print("\n5. Reflecting on memories...")
-        reflect_response = client.reflect(
-            bank_id=bank_id,
-            query="What programming tools should I recommend?",
-            budget="low",
-        )
-        assert reflect_response.text
-        assert len(reflect_response.text) > 0
-        print(f"   Answer: {reflect_response.text[:150]}...")
-
-        # Verify answer mentions relevant tools
-        answer_lower = reflect_response.text.lower()
-        assert any(
-            term in answer_lower for term in ["python", "pandas", "numpy", "data"]
-        )
-
-        # Step 6: List memories
-        print("\n6. Listing memories...")
-        list_response = client.list_memories(bank_id=bank_id, limit=10)
-        assert len(list_response.items) > 0
-        print(f"   Listed {len(list_response.items)} memories")
-
-    finally:
-        # Cleanup
-        client.close()
-
-
-def test_embedded_server_reuse(llm_config):
-    """
-    Test that the same server is reused across multiple calls.
-    """
-    profile = f"test_reuse_{uuid.uuid4().hex[:8]}"
-    bank_id = f"bank_{uuid.uuid4().hex[:8]}"
-
-    client = HindsightEmbedded(profile=profile, log_level="info", **llm_config)
-
-    try:
-        # First call starts server
-        result1 = client.retain(bank_id=bank_id, content="First message")
-        url1 = client.url
-        assert client.is_running
-
-        # Second call should reuse the same server
-        result2 = client.retain(bank_id=bank_id, content="Second message")
-        url2 = client.url
-
-        # URLs should be identical (same server)
-        assert url1 == url2, "Server URL should remain the same across calls"
-        assert result1.success and result2.success
-
-        # Third call should also reuse
-        recall_result = client.recall(bank_id=bank_id, query="message")
-        url3 = client.url
-        assert url3 == url1, "Server URL should remain the same for recall"
-        assert isinstance(recall_result.results, list)
-
-    finally:
-        client.close()
-
-
-def test_embedded_method_proxying(llm_config):
-    """
-    Test that all HindsightClient methods are properly proxied.
-
-    This ensures __getattr__ proxying works for various method types.
-    """
-    profile = f"test_proxy_{uuid.uuid4().hex[:8]}"
-    bank_id = f"bank_{uuid.uuid4().hex[:8]}"
-
-    client = HindsightEmbedded(profile=profile, log_level="info", **llm_config)
-
-    try:
-        # Test bank operations
-        bank = client.create_bank(bank_id=bank_id, name="Proxy Test")
-        assert bank.bank_id == bank_id
-
-        # Test mission setting
-        mission_response = client.set_mission(
-            bank_id=bank_id, mission="Test mission for proxying"
-        )
-        assert mission_response.bank_id == bank_id
-
-        # Test retain
-        retain_result = client.retain(bank_id=bank_id, content="Test content")
-        assert retain_result.success
-
-        # Test retain_batch
-        batch_result = client.retain_batch(
-            bank_id=bank_id, items=[{"content": "Item 1"}, {"content": "Item 2"}]
-        )
-        assert batch_result.success
-        assert batch_result.items_count >= 2
-
-        # Test recall
-        recall_result = client.recall(bank_id=bank_id, query="test")
-        assert hasattr(recall_result, "results")
-
-        # Test reflect
-        reflect_result = client.reflect(bank_id=bank_id, query="What is stored?")
-        assert hasattr(reflect_result, "text")
-
-        # Test list_memories
-        list_result = client.list_memories(bank_id=bank_id, limit=5)
-        assert hasattr(list_result, "items")
-
-        print("✓ All methods successfully proxied")
-
-    finally:
-        client.close()
-
-
-def test_embedded_multiple_banks(llm_config):
-    """
-    Test that HindsightEmbedded can work with multiple banks.
-    """
-    profile = f"test_multibank_{uuid.uuid4().hex[:8]}"
-    bank1_id = f"bank1_{uuid.uuid4().hex[:8]}"
-    bank2_id = f"bank2_{uuid.uuid4().hex[:8]}"
-
-    client = HindsightEmbedded(profile=profile, log_level="info", **llm_config)
-
-    try:
-        # Create first bank and store data
-        client.create_bank(bank_id=bank1_id, name="Bank 1")
-        client.retain(bank_id=bank1_id, content="Alice prefers Python for data science")
-
-        # Create second bank and store data
-        client.create_bank(bank_id=bank2_id, name="Bank 2")
-        client.retain(
-            bank_id=bank2_id, content="Bob uses JavaScript for web development"
-        )
-
-        # Recall from both banks
-        results1 = client.recall(bank_id=bank1_id, query="programming language")
-        results2 = client.recall(bank_id=bank2_id, query="programming language")
-
-        assert len(results1.results) > 0
-        assert len(results2.results) > 0
-
-        # Verify banks are isolated (each should only see their own content)
-        # This is a basic check - content isolation is tested more thoroughly in other tests
-        assert results1.results[0].text != results2.results[0].text or len(
-            results1.results
-        ) != len(results2.results)
-
-    finally:
-        client.close()
-
-
-def test_embedded_profile_isolation(llm_config):
-    """
-    Test that different profiles create isolated data stores.
-    """
-    profile1 = f"test_iso1_{uuid.uuid4().hex[:8]}"
-    profile2 = f"test_iso2_{uuid.uuid4().hex[:8]}"
-    bank_id = "shared_bank_name"  # Same bank_id in both profiles
-
-    client1 = HindsightEmbedded(profile=profile1, log_level="info", **llm_config)
-    client2 = HindsightEmbedded(profile=profile2, log_level="info", **llm_config)
-
-    try:
-        # Store data in profile1
-        client1.retain(
-            bank_id=bank_id, content="User likes TypeScript for frontend development"
-        )
-
-        # Store different data in profile2
-        client2.retain(
-            bank_id=bank_id, content="User prefers Rust for systems programming"
-        )
-
-        # Each profile should only see its own data
-        results1 = client1.recall(bank_id=bank_id, query="programming preference")
-        results2 = client2.recall(bank_id=bank_id, query="programming preference")
-
-        # Both should have results
-        assert len(results1.results) > 0
-        assert len(results2.results) > 0
-
-        # Results should be different (basic isolation check)
-        # Note: This is a basic sanity check. Full isolation is ensured by pg0's data directory separation
-
-    finally:
-        client1.close()
-        client2.close()
-
-
-def test_embedded_error_after_close(llm_config):
-    """
-    Test that using HindsightEmbedded after close() raises an error.
-    """
-    profile = f"test_error_{uuid.uuid4().hex[:8]}"
-    bank_id = f"bank_{uuid.uuid4().hex[:8]}"
-
-    client = HindsightEmbedded(profile=profile, log_level="info", **llm_config)
-
-    # Use it once to start server
-    client.retain(bank_id=bank_id, content="Test")
-
-    # Close the client
-    client.close()
-    assert not client.is_running
-
-    # Trying to use it after close should raise an error
-    with pytest.raises(
-        RuntimeError, match="Cannot use HindsightEmbedded after it has been closed"
-    ):
-        client.retain(bank_id=bank_id, content="This should fail")
-
-
-def test_embedded_ui_flag(llm_config):
-    """
-    Test that ui=True starts the control plane UI alongside the daemon,
-    and that the UI's health endpoint reports a connected dataplane.
-    """
-    profile = f"test_ui_{uuid.uuid4().hex[:8]}"
-    bank_id = f"bank_{uuid.uuid4().hex[:8]}"
-
-    client = HindsightEmbedded(profile=profile, log_level="info", ui=True, **llm_config)
-
-    try:
-        # First use triggers daemon + UI startup
-        result = client.retain(bank_id=bank_id, content="UI integration test content")
-        assert result.success, "Retain should succeed"
-        assert client.is_running, "Daemon should be running"
-
-        # Verify UI is reachable and reports connected dataplane
-        ui_url = client.ui_url
-        assert isinstance(ui_url, str) and ui_url, "ui_url should be a non-empty string"
-
-        health_url = f"{ui_url}/api/health"
-        with urllib.request.urlopen(health_url, timeout=10) as resp:
-            health = json.loads(resp.read().decode())
-
-        assert health["status"] == "ok", (
-            f"UI health status should be 'ok', got: {health['status']}"
-        )
-        assert health["dataplane"]["status"] == "connected", (
-            f"Dataplane should be connected, got: {health['dataplane']}"
-        )
-
-    finally:
-        client.close()
-
-
-def test_embedded_daemon_crash_recovery(llm_config):
-    """
-    Test that HindsightEmbedded recovers when the daemon crashes.
-
-    Simulates a crash by stopping the daemon, then verifies
-    that the next operation transparently restarts it.
-    """
-    profile = f"test_crash_{uuid.uuid4().hex[:8]}"
-    bank_id = f"bank_{uuid.uuid4().hex[:8]}"
-
-    client = HindsightEmbedded(profile=profile, log_level="info", **llm_config)
-
-    try:
-        # Start daemon and store a memory
-        result = client.retain(bank_id=bank_id, content="Before crash")
-        assert result.success, "Initial retain should succeed"
-        assert client.is_running, "Daemon should be running"
-
-        original_url = client.url
-
-        # Simulate daemon crash by stopping it
-        client._manager.stop(client.profile)
-        assert not client._manager.is_running(client.profile), (
-            "Daemon should be stopped after simulated crash"
-        )
-
-        # Next operation should transparently restart the daemon
-        result2 = client.retain(bank_id=bank_id, content="After crash recovery")
-        assert result2.success, "Retain after crash recovery should succeed"
-        assert client.is_running, "Daemon should be running again after recovery"
-
-        # Verify recall still works
-        recall_result = client.recall(bank_id=bank_id, query="crash")
-        assert isinstance(recall_result.results, list), "Recall should return results"
-
-    finally:
-        client.close()

hindsight-all/tests/test_embedded_namespaces.py

@@ -1,170 +0,0 @@
-"""Test that API namespaces ensure daemon is started before each call."""
-
-from unittest.mock import Mock, patch
-
-import pytest
-
-from hindsight import HindsightEmbedded
-
-
-@pytest.fixture
-def embedded_client():
-    """Create an embedded client for testing."""
-    return HindsightEmbedded(
-        profile="test",
-        llm_provider="openai",
-        llm_api_key="test-key",
-    )
-
-
-def test_banks_create_ensures_daemon_started(embedded_client):
-    """Test that banks.create() calls _ensure_started()."""
-    # Mock _ensure_started to track calls
-    with patch.object(embedded_client, "_ensure_started") as mock_ensure:
-        # Mock the underlying client to avoid actual API call
-        mock_client = Mock()
-        embedded_client._client = mock_client
-
-        # Call namespace method
-        try:
-            embedded_client.banks.create(bank_id="test", name="Test Bank")
-        except Exception:
-            pass  # We don't care if the actual call fails
-
-        # Verify _ensure_started was called
-        mock_ensure.assert_called_once()
-
-
-def test_mental_models_list_ensures_daemon_started(embedded_client):
-    """Test that mental_models.list() calls _ensure_started()."""
-    with patch.object(embedded_client, "_ensure_started") as mock_ensure:
-        mock_client = Mock()
-        embedded_client._client = mock_client
-
-        try:
-            embedded_client.mental_models.list(bank_id="test")
-        except Exception:
-            pass
-
-        mock_ensure.assert_called_once()
-
-
-def test_directives_list_ensures_daemon_started(embedded_client):
-    """Test that directives.list() calls _ensure_started()."""
-    with patch.object(embedded_client, "_ensure_started") as mock_ensure:
-        mock_client = Mock()
-        embedded_client._client = mock_client
-
-        try:
-            embedded_client.directives.list(bank_id="test")
-        except Exception:
-            pass
-
-        mock_ensure.assert_called_once()
-
-
-def test_memories_list_ensures_daemon_started(embedded_client):
-    """Test that memories.list() calls _ensure_started()."""
-    with patch.object(embedded_client, "_ensure_started") as mock_ensure:
-        mock_client = Mock()
-        embedded_client._client = mock_client
-
-        try:
-            embedded_client.memories.list(bank_id="test")
-        except Exception:
-            pass
-
-        mock_ensure.assert_called_once()
-
-
-def test_multiple_calls_ensure_daemon_each_time(embedded_client):
-    """Test that each namespace call ensures daemon is started."""
-    with patch.object(embedded_client, "_ensure_started") as mock_ensure:
-        mock_client = Mock()
-        embedded_client._client = mock_client
-
-        # Make multiple calls
-        try:
-            embedded_client.banks.create(bank_id="test", name="Test")
-        except Exception:
-            pass
-
-        try:
-            embedded_client.mental_models.list(bank_id="test")
-        except Exception:
-            pass
-
-        try:
-            embedded_client.directives.list(bank_id="test")
-        except Exception:
-            pass
-
-        # Should be called 3 times (once per namespace method call)
-        assert mock_ensure.call_count == 3
-
-
-def test_daemon_restart_handling(embedded_client):
-    """Test that namespace methods can recover from daemon crash."""
-    call_count = 0
-
-    def mock_ensure_started():
-        """Mock that simulates daemon restart."""
-        nonlocal call_count
-        call_count += 1
-        # Create a new mock client each time (simulating daemon restart)
-        embedded_client._client = Mock()
-        embedded_client._started = True
-
-    with patch.object(embedded_client, "_ensure_started", side_effect=mock_ensure_started):
-        # First call - daemon starts
-        embedded_client.banks.create(bank_id="test", name="Test")
-        assert call_count == 1
-
-        # Simulate daemon crash by clearing client
-        embedded_client._client = None
-        embedded_client._started = False
-
-        # Second call - daemon restarts
-        embedded_client.banks.create(bank_id="test", name="Test")
-        assert call_count == 2
-
-
-def test_ensure_started_calls_manager(embedded_client):
-    """Test that _ensure_started actually starts the daemon via manager."""
-    # Mock the manager
-    mock_manager = Mock()
-    mock_manager.ensure_running.return_value = True
-    mock_manager.get_url.return_value = "http://localhost:54321"
-
-    embedded_client._manager = mock_manager
-
-    # Mock Hindsight client constructor
-    with patch("hindsight.embedded.Hindsight") as mock_hindsight_class:
-        mock_client = Mock()
-        mock_hindsight_class.return_value = mock_client
-
-        # Call _ensure_started
-        embedded_client._ensure_started()
-
-        # Verify manager was called
-        mock_manager.ensure_running.assert_called_once_with(
-            embedded_client.config, embedded_client.profile
-        )
-        mock_manager.get_url.assert_called_once_with(embedded_client.profile)
-
-        # Verify Hindsight client was created
-        mock_hindsight_class.assert_called_once_with(base_url="http://localhost:54321")
-
-
-def test_namespace_singleton_behavior(embedded_client):
-    """Test that namespace properties return the same instance."""
-    banks1 = embedded_client.banks
-    banks2 = embedded_client.banks
-
-    # Should be the same instance
-    assert banks1 is banks2
-
-    # Same for other namespaces
-    assert embedded_client.mental_models is embedded_client.mental_models
-    assert embedded_client.directives is embedded_client.directives
-    assert embedded_client.memories is embedded_client.memories

hindsight-api-slim/README.md

@@ -1,137 +0,0 @@
-# Hindsight API
-
-**Memory System for AI Agents** — Temporal + Semantic + Entity Memory Architecture using PostgreSQL with pgvector.
-
-Hindsight gives AI agents persistent memory that works like human memory: it stores facts, tracks entities and relationships, handles temporal reasoning ("what happened last spring?"), and forms opinions based on configurable disposition traits.
-
-## Installation
-
-```bash
-pip install hindsight-api
-```
-
-## Quick Start
-
-### Run the Server
-
-```bash
-# Set your LLM provider
-export HINDSIGHT_API_LLM_PROVIDER=openai
-export HINDSIGHT_API_LLM_API_KEY=sk-xxxxxxxxxxxx
-
-# Start the server (uses embedded PostgreSQL by default)
-hindsight-api
-```
-
-The server starts at http://localhost:8888 with:
-- REST API for memory operations
-- MCP server at `/mcp` for tool-use integration
-
-### Use the Python API
-
-```python
-from hindsight_api import MemoryEngine
-
-# Create and initialize the memory engine
-memory = MemoryEngine()
-await memory.initialize()
-
-# Create a memory bank for your agent
-bank = await memory.create_memory_bank(
-    name="my-assistant",
-    background="A helpful coding assistant"
-)
-
-# Store a memory
-await memory.retain(
-    memory_bank_id=bank.id,
-    content="The user prefers Python for data science projects"
-)
-
-# Recall memories
-results = await memory.recall(
-    memory_bank_id=bank.id,
-    query="What programming language does the user prefer?"
-)
-
-# Reflect with reasoning
-response = await memory.reflect(
-    memory_bank_id=bank.id,
-    query="Should I recommend Python or R for this ML project?"
-)
-```
-
-## CLI Options
-
-```bash
-hindsight-api --help
-
-# Common options
-hindsight-api --port 9000          # Custom port (default: 8888)
-hindsight-api --host 127.0.0.1     # Bind to localhost only
-hindsight-api --workers 4          # Multiple worker processes
-hindsight-api --log-level debug    # Verbose logging
-```
-
-## Configuration
-
-Configure via environment variables:
-
-| Variable | Description | Default |
-|----------|-------------|---------|
-| `HINDSIGHT_API_DATABASE_URL` | PostgreSQL connection string | `pg0` (embedded) |
-| `HINDSIGHT_API_LLM_PROVIDER` | `openai`, `anthropic`, `gemini`, `groq`, `ollama`, `lmstudio` | `openai` |
-| `HINDSIGHT_API_LLM_API_KEY` | API key for LLM provider | - |
-| `HINDSIGHT_API_LLM_MODEL` | Model name | `gpt-4o-mini` |
-| `HINDSIGHT_API_HOST` | Server bind address | `0.0.0.0` |
-| `HINDSIGHT_API_PORT` | Server port | `8888` |
-
-### Example with External PostgreSQL
-
-```bash
-export HINDSIGHT_API_DATABASE_URL=postgresql://user:pass@localhost:5432/hindsight
-export HINDSIGHT_API_LLM_PROVIDER=groq
-export HINDSIGHT_API_LLM_API_KEY=gsk_xxxxxxxxxxxx
-
-hindsight-api
-```
-
-## Docker
-
-```bash
-docker run -it --name hindsight --restart unless-stopped -p 8888:8888 \
-  -e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \
-  -v $HOME/.hindsight-docker:/home/hindsight/.pg0 \
-  ghcr.io/vectorize-io/hindsight:latest
-```
-
-## MCP Server
-
-For local MCP integration without running the full API server:
-
-```bash
-hindsight-local-mcp
-```
-
-This runs a stdio-based MCP server that can be used directly with MCP-compatible clients.
-
-## Key Features
-
-- **Multi-Strategy Retrieval (TEMPR)** — Semantic, keyword, graph, and temporal search combined with RRF fusion
-- **Entity Graph** — Automatic entity extraction and relationship tracking
-- **Temporal Reasoning** — Native support for time-based queries
-- **Disposition Traits** — Configurable skepticism, literalism, and empathy influence opinion formation
-- **Three Memory Types** — World facts, experience facts (the bank's own actions), and observations
-
-## Documentation
-
-Full documentation: [https://hindsight.vectorize.io](https://hindsight.vectorize.io)
-
-- [Installation Guide](https://hindsight.vectorize.io/developer/installation)
-- [Configuration Reference](https://hindsight.vectorize.io/developer/configuration)
-- [API Reference](https://hindsight.vectorize.io/api-reference)
-- [Python SDK](https://hindsight.vectorize.io/sdks/python)
-
-## License
-
-Apache 2.0

hindsight-api-slim/hindsight_api/_pg_search.py

@@ -1,85 +0,0 @@
-"""Helpers for ParadeDB pg_search index configuration."""
-
-from __future__ import annotations
-
-import re
-from collections.abc import Sequence
-
-PG_SEARCH_TOKENIZER_ENV = "HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER"
-
-_SIMPLE_TOKENIZERS = {
-    "unicode_words",
-    "simple",
-    "whitespace",
-    "literal",
-    "literal_normalized",
-    "chinese_compatible",
-    "icu",
-    "jieba",
-    "source_code",
-}
-
-_TOKENIZER_ALIASES = {
-    "chinese_lindera": "lindera(chinese)",
-    "japanese_lindera": "lindera(japanese)",
-    "korean_lindera": "lindera(korean)",
-    "lindera_chinese": "lindera(chinese)",
-    "lindera_japanese": "lindera(japanese)",
-    "lindera_korean": "lindera(korean)",
-}
-
-
-def normalize_pg_search_tokenizer(value: str | None) -> str:
-    """Validate and normalize a ParadeDB pg_search tokenizer setting.
-
-    Returns an empty string when unset. The returned value is safe to embed after
-    ``pdb.`` in a CREATE INDEX expression.
-    """
-
-    tokenizer = (value or "").strip().lower()
-    if not tokenizer:
-        return ""
-
-    if tokenizer in _TOKENIZER_ALIASES:
-        return _TOKENIZER_ALIASES[tokenizer]
-
-    if tokenizer in _SIMPLE_TOKENIZERS:
-        return tokenizer
-
-    lindera_match = re.fullmatch(r"lindera\((chinese|japanese|korean)\)", tokenizer)
-    if lindera_match:
-        return tokenizer
-
-    ngram_match = re.fullmatch(r"(ngram|edge_ngram)\((\d{1,3}),\s*(\d{1,3})\)", tokenizer)
-    if ngram_match:
-        kind, min_gram, max_gram = ngram_match.groups()
-        min_value = int(min_gram)
-        max_value = int(max_gram)
-        if min_value <= 0 or min_value > max_value:
-            raise ValueError(
-                f"Invalid {PG_SEARCH_TOKENIZER_ENV}: {value!r}. "
-                "ngram and edge_ngram require positive min/max gram sizes with min <= max."
-            )
-        return f"{kind}({min_value},{max_value})"
-
-    raise ValueError(
-        f"Invalid {PG_SEARCH_TOKENIZER_ENV}: {value!r}. "
-        "Supported values are: unicode_words, simple, whitespace, literal, "
-        "literal_normalized, chinese_compatible, icu, jieba, source_code, "
-        "chinese_lindera, japanese_lindera, korean_lindera, or "
-        "lindera(chinese|japanese|korean), ngram(min,max), or edge_ngram(min,max)."
-    )
-
-
-def pg_search_bm25_columns(
-    key_field: str,
-    text_fields: Sequence[str],
-    tokenizer: str | None,
-) -> str:
-    """Build a ParadeDB BM25 column list for CREATE INDEX."""
-
-    normalized = normalize_pg_search_tokenizer(tokenizer)
-    if not normalized:
-        return ", ".join([key_field, *text_fields])
-
-    return ", ".join([key_field, *(f"({field}::pdb.{normalized})" for field in text_fields)])

hindsight-api-slim/hindsight_api/_thread_limits.py

@@ -1,107 +0,0 @@
-"""Process-level caps for native ML thread pools.
-
-OpenBLAS, OpenMP, and MKL each spawn a worker pool sized to the host CPU count
-the first time they are loaded (numpy pulls in OpenBLAS eagerly; torch and
-onnxruntime load their pools lazily on first inference). Hindsight already
-parallelizes at the request level via thread-pool executors (embeddings on the
-default executor, the reranker on its own pool), so these native intra-op pools
-oversubscribe the CPU: on a many-core host the process accumulates 100+ native
-threads, which inflates memory and, under contention, can degrade throughput.
-
-We bound each pool to ``_MAX_NATIVE_THREADS`` (or the available CPU count, if
-smaller). "Available" is the CPU budget actually granted to the process, not
-``os.cpu_count()``: in a CPU-limited container ``os.cpu_count()`` still reports
-the host's cores, so sizing pools by it oversubscribes the container's real
-quota — the exact failure mode this guards against. We therefore take the
-smallest of the CPU-affinity set, the cgroup CPU quota, and ``os.cpu_count()``.
-
-Every cap is applied with ``setdefault`` so an operator who has deliberately
-tuned one of these variables keeps their value. This must run *before* numpy,
-torch, or onnxruntime are imported — those libraries read the variables only at
-load time — which is why it is invoked at the very top of
-``hindsight_api/__init__.py``, ahead of the package's other imports.
-"""
-
-from __future__ import annotations
-
-import os
-
-# Native threading env vars, each read by the respective library at load time.
-_NATIVE_THREAD_VARS = (
-    "OMP_NUM_THREADS",  # OpenMP — torch, onnxruntime, some BLAS builds
-    "OPENBLAS_NUM_THREADS",  # OpenBLAS — numpy's default BLAS
-    "MKL_NUM_THREADS",  # Intel MKL — numpy/torch when MKL-backed
-    "NUMEXPR_NUM_THREADS",  # numexpr expression engine
-)
-
-# Upper bound on intra-op threads per native pool. Bounds runaway growth on
-# many-core hosts without serialising single-request inference.
-_MAX_NATIVE_THREADS = 16
-
-
-def _quota_to_cpus(quota: int, period: int) -> int | None:
-    """Whole CPUs from a CFS quota/period pair, or None if unlimited."""
-    if quota > 0 and period > 0:
-        # Floor (never round up) so we never exceed the granted budget.
-        return max(1, quota // period)
-    return None
-
-
-def _parse_cgroup_v2_cpu_max(text: str) -> int | None:
-    """Parse cgroup v2 ``cpu.max`` ("<quota> <period>", or "max <period>")."""
-    parts = text.split()
-    if len(parts) >= 2 and parts[0] != "max":
-        try:
-            return _quota_to_cpus(int(parts[0]), int(parts[1]))
-        except ValueError:
-            return None
-    return None
-
-
-def _cgroup_cpu_quota() -> int | None:
-    """Effective CPUs from the cgroup CPU quota, or None if unlimited/unknown."""
-    try:  # cgroup v2
-        with open("/sys/fs/cgroup/cpu.max") as fh:
-            return _parse_cgroup_v2_cpu_max(fh.read())
-    except OSError:
-        pass
-    try:  # cgroup v1
-        with open("/sys/fs/cgroup/cpu/cpu.cfs_quota_us") as fh:
-            quota = int(fh.read())
-        with open("/sys/fs/cgroup/cpu/cpu.cfs_period_us") as fh:
-            period = int(fh.read())
-        return _quota_to_cpus(quota, period)
-    except (OSError, ValueError):
-        return None
-
-
-def _available_cpu_count() -> int:
-    """CPUs actually available to this process.
-
-    The smallest of the CPU-affinity set (cpuset / ``--cpuset-cpus``), the
-    cgroup CPU quota (``--cpus``), and ``os.cpu_count()`` — each captures a
-    different way the budget can be constrained, and the last alone overcounts
-    inside a limited container.
-    """
-    candidates = [os.cpu_count() or 1]
-    if hasattr(os, "sched_getaffinity"):
-        try:
-            candidates.append(len(os.sched_getaffinity(0)))
-        except OSError:
-            pass
-    quota = _cgroup_cpu_quota()
-    if quota is not None:
-        candidates.append(quota)
-    return max(1, min(candidates))
-
-
-def default_native_thread_count() -> int:
-    """Per-pool cap: ``_MAX_NATIVE_THREADS``, or available CPUs if fewer."""
-    return min(_MAX_NATIVE_THREADS, _available_cpu_count())
-
-
-def apply_default_thread_limits() -> None:
-    """Cap native ML thread pools unless the operator has set the var already."""
-    value = str(default_native_thread_count())
-    for var in _NATIVE_THREAD_VARS:
-        os.environ.setdefault(var, value)

hindsight-api-slim/hindsight_api/_vector_index.py

@@ -1,225 +0,0 @@
-"""Shared PostgreSQL vector-extension dispatch helpers."""
-
-from __future__ import annotations
-
-import logging
-import os
-
-from sqlalchemy import text
-from sqlalchemy.engine import Connection
-
-logger = logging.getLogger(__name__)
-
-# Extensions a user can set via HINDSIGHT_API_VECTOR_EXTENSION.
-CONFIGURABLE_EXTENSIONS = ("pgvector", "pgvectorscale", "vchord", "scann")
-
-# Extensions detect_vector_extension() can return. pg_diskann is a runtime-only
-# resolution from a configured "pgvectorscale" backend on Azure (uses a different
-# WITH clause), never a value the user sets directly.
-RESOLVED_EXTENSIONS = (*CONFIGURABLE_EXTENSIONS, "pg_diskann")
-
-# Backwards-compatible alias for older imports.
-VALID_EXTENSIONS = CONFIGURABLE_EXTENSIONS
-
-SCANN_MIN_ROWS_FOR_AUTO_INDEX = 10_000
-
-
-_EXTENSION_NAMES = {
-    "pgvector": "vector",
-    "pgvectorscale": "vectorscale",
-    "vchord": "vchord",
-    "scann": "alloydb_scann",
-}
-
-_INDEX_USING_CLAUSES = {
-    "pgvector": "USING hnsw (embedding vector_cosine_ops)",
-    "pgvectorscale": "USING diskann (embedding vector_cosine_ops) WITH (num_neighbors = 50)",
-    "pg_diskann": "USING diskann (embedding vector_cosine_ops) WITH (max_neighbors = 50)",
-    "vchord": "USING vchordrq (embedding vector_cosine_ops)",
-    "scann": "USING scann (embedding cosine) WITH (mode = 'AUTO')",
-}
-
-_INDEX_TYPE_KEYWORDS = {
-    "pgvector": "hnsw",
-    "pgvectorscale": "diskann",
-    "pg_diskann": "diskann",
-    "vchord": "vchordrq",
-    "scann": "scann",
-}
-
-# Per-backend ANN search-time tuning GUCs. Each entry is a tuple of
-# (guc_name, value) pairs the caller can apply with SET or SET LOCAL.
-#
-# - pgvector exposes hnsw.ef_search. The 60 / 200 pair is unchanged from the
-#   pre-dispatcher code (internal benchmarks tuned around our embedding count
-#   and recall floor; see the link_utils / pool init call sites for the
-#   latency-vs-recall framing).
-# - vchord exposes vchordrq.probes, but its shape must match the index's
-#   build.internal.lists hierarchy. VectorChord 1.1 added per-index fallback
-#   parameters for this reason: a session GUC overrides every vchordrq index,
-#   and a single value can be invalid for listless or mixed-layout indexes.
-#   Hindsight's built-in vchord clause does not set lists, so the safe default
-#   is no session-level probe override; deployments that partition vchordrq
-#   indexes should attach probes to the index storage parameters instead.
-# - pgvectorscale / pg_diskann / scann do not expose an equivalent per-statement
-#   knob in the engine today, so the dispatcher returns no statements for them.
-_ANN_TUNING_LOW_LATENCY: dict[str, tuple[tuple[str, str], ...]] = {
-    "pgvector": (("hnsw.ef_search", "60"),),
-}
-_ANN_TUNING_HIGH_RECALL: dict[str, tuple[tuple[str, str], ...]] = {
-    "pgvector": (("hnsw.ef_search", "200"),),
-}
-
-_EXTENSION_INSTALL_SQL = {
-    "pgvector": ("CREATE EXTENSION IF NOT EXISTS vector",),
-    "pgvectorscale": (
-        "CREATE EXTENSION IF NOT EXISTS vector",
-        "CREATE EXTENSION IF NOT EXISTS vectorscale CASCADE",
-    ),
-    "vchord": ("CREATE EXTENSION IF NOT EXISTS vchord CASCADE",),
-    "scann": (
-        "CREATE EXTENSION IF NOT EXISTS vector",
-        "CREATE EXTENSION IF NOT EXISTS alloydb_scann CASCADE",
-    ),
-}
-
-_INSTALL_HINTS = {
-    "pgvector": "CREATE EXTENSION vector;",
-    "pgvectorscale": "CREATE EXTENSION vector; then CREATE EXTENSION vectorscale CASCADE; (or pg_diskann on Azure)",
-    "vchord": "CREATE EXTENSION vchord CASCADE;",
-    "scann": "CREATE EXTENSION vector; then CREATE EXTENSION alloydb_scann CASCADE;",
-}
-
-
-def configured_vector_extension() -> str:
-    """Return the user-configured vector backend extension.
-
-    Reads ``HINDSIGHT_API_VECTOR_EXTENSION`` (default ``"pgvector"``) and
-    validates it via :func:`validate_extension`. This is the single source of
-    truth for runtime code that needs to dispatch behaviour by vector backend;
-    callers should prefer this over reading the env var directly, so the
-    default value and the lookup mechanism live in one place.
-    """
-    return validate_extension(os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector"))
-
-
-def validate_extension(name: str) -> str:
-    """Return a normalized configurable vector extension name or raise.
-
-    Used at the user-facing config boundary; pg_diskann is rejected here because
-    it is a detection-time alias, never a value the user sets directly.
-    """
-    ext = name.lower()
-    if ext not in CONFIGURABLE_EXTENSIONS:
-        valid = ", ".join(CONFIGURABLE_EXTENSIONS)
-        raise ValueError(f"Invalid vector_extension: {name}. Must be one of: {valid}")
-    return ext
-
-
-def _normalize_resolved(name: str) -> str:
-    """Normalize either a user-configurable or detect-time extension name."""
-    ext = name.lower()
-    if ext not in RESOLVED_EXTENSIONS:
-        valid = ", ".join(RESOLVED_EXTENSIONS)
-        raise ValueError(f"Unknown vector extension: {name}. Must be one of: {valid}")
-    return ext
-
-
-def pg_extension_name(ext: str) -> str:
-    """Return the PostgreSQL extension name for a configured vector backend."""
-    return _EXTENSION_NAMES[validate_extension(ext)]
-
-
-def index_using_clause(ext: str) -> str:
-    """Return the CREATE INDEX USING clause for the vector backend."""
-    return _INDEX_USING_CLAUSES[_normalize_resolved(ext)]
-
-
-def index_type_keyword(ext: str) -> str:
-    """Return the keyword that identifies this index type in pg_indexes.indexdef."""
-    return _INDEX_TYPE_KEYWORDS[_normalize_resolved(ext)]
-
-
-def minimum_rows_for_index(ext: str) -> int:
-    """Return the minimum populated embedding rows before creating this index type."""
-    return SCANN_MIN_ROWS_FOR_AUTO_INDEX if _normalize_resolved(ext) == "scann" else 0
-
-
-def should_defer_index_creation(ext: str, row_count: int) -> bool:
-    """Return True when index creation should wait for more embeddings."""
-    minimum_rows = minimum_rows_for_index(ext)
-    return minimum_rows > 0 and row_count < minimum_rows
-
-
-def ann_search_tuning_settings(ext: str, *, kind: str) -> tuple[tuple[str, str], ...]:
-    """Return per-backend (guc_name, value) pairs for ANN search-time tuning.
-
-    ``kind`` is ``"low_latency"`` for retain-side link probing (smaller probe
-    count, lower recall, lower latency) and ``"high_recall"`` for connection
-    init in the pool (larger probe count, higher recall). Callers wrap each
-    pair with ``SET LOCAL`` or ``SET`` themselves so the same dispatcher works
-    for both transaction-scoped and session-scoped use. Returns an empty tuple
-    for backends without an equivalent knob.
-    """
-    if kind == "low_latency":
-        table = _ANN_TUNING_LOW_LATENCY
-    elif kind == "high_recall":
-        table = _ANN_TUNING_HIGH_RECALL
-    else:
-        raise ValueError(f"Unknown ANN tuning kind: {kind!r}")
-    return table.get(_normalize_resolved(ext), ())
-
-
-def uses_per_bank_vector_indexes(ext: str) -> bool:
-    """Return whether the backend should create per-bank partial vector indexes."""
-    return _normalize_resolved(ext) != "scann"
-
-
-def bootstrap_extension(conn: Connection, ext: str) -> None:
-    """Install the configured vector extension and any prerequisites if possible."""
-    normalized = validate_extension(ext)
-    for statement in _EXTENSION_INSTALL_SQL[normalized]:
-        conn.execute(text(statement))
-
-
-def detect_vector_extension(conn: Connection, vector_extension: str = "pgvector") -> str:
-    """Validate the configured vector extension exists and return the index backend."""
-    configured_ext = validate_extension(vector_extension)
-
-    if configured_ext == "pgvectorscale":
-        pgvector_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vector'")).scalar()
-        if not pgvector_check:
-            raise RuntimeError(
-                "DiskANN (pgvectorscale/pg_diskann) requires pgvector to be installed. "
-                f"Install it with: {_INSTALL_HINTS['pgvectorscale']}"
-            )
-
-        vectorscale_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vectorscale'")).scalar()
-        pg_diskann_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'pg_diskann'")).scalar()
-
-        if vectorscale_check:
-            logger.debug("Using vector extension: pgvectorscale (DiskANN)")
-            return "pgvectorscale"
-        if pg_diskann_check:
-            logger.debug("Using vector extension: pg_diskann (Azure DiskANN)")
-            return "pg_diskann"
-
-        raise RuntimeError(
-            "Configured vector extension 'pgvectorscale' not found. Install either:\n"
-            "  - pgvectorscale (open source): CREATE EXTENSION vectorscale CASCADE;\n"
-            "  - pg_diskann (Azure): CREATE EXTENSION pg_diskann CASCADE;"
-        )
-
-    extension_name = pg_extension_name(configured_ext)
-    extension_check = conn.execute(
-        text("SELECT 1 FROM pg_extension WHERE extname = :extension_name"),
-        {"extension_name": extension_name},
-    ).scalar()
-    if not extension_check:
-        raise RuntimeError(
-            f"Configured vector extension '{configured_ext}' not found. "
-            f"Install it with: {_INSTALL_HINTS[configured_ext]}"
-        )
-
-    logger.debug("Using configured vector extension: %s", configured_ext)
-    return configured_ext

hindsight-api-slim/hindsight_api/admin/cli.py

@@ -1,672 +0,0 @@
-"""PostgreSQL-only admin utilities (backup, restore, migration, worker management).
-
-Not supported on Oracle backends. Uses asyncpg.connect() directly, binary COPY,
-TRUNCATE CASCADE, and REFRESH MATERIALIZED VIEW — all inherently PG-specific.
-"""
-
-import asyncio
-import io
-import json
-import logging
-import zipfile
-from datetime import datetime, timezone
-from pathlib import Path
-from typing import Any
-
-import asyncpg
-import typer
-
-from ..config import DEFAULT_DATABASE_SCHEMA, HindsightConfig
-from ..engine.memory_engine import _current_schema
-from ..engine.schema import fq_table_explicit as _fq_table
-from ..engine.transfer import export_bank
-from ..extensions import TenantExtension, load_extension
-from ..pg0 import parse_pg0_url, resolve_database_url
-
-# Setup logging
-logging.basicConfig(
-    level=logging.INFO,
-    format="%(message)s",
-)
-logger = logging.getLogger(__name__)
-
-app = typer.Typer(name="hindsight-admin", help="Hindsight administrative commands")
-
-# Tables to backup/restore in foreign-key dependency order (parents first).
-# Restore COPYs in this order and TRUNCATEs in reverse, so every child must
-# appear after the tables it references.
-#
-# This must cover EVERY persistent PostgreSQL table in the schema — a missing
-# entry silently drops that table's data on restore (and, worse, restore's
-# `TRUNCATE banks CASCADE` wipes any FK-to-banks child like mental_models even
-# when it was never backed up). test_admin_backup_restore.py asserts this list
-# equals the live schema's tables, so adding a migration that creates a table
-# without adding it here fails CI. Oracle-only tables (e.g. observation_sources)
-# are intentionally absent — admin backup/restore is PostgreSQL-only.
-BACKUP_TABLES = [
-    "banks",
-    "documents",
-    "entities",
-    "chunks",
-    "memory_units",
-    "invalidated_memory_units",
-    "unit_entities",
-    "entity_cooccurrences",
-    "memory_links",
-    "observation_history",
-    "mental_models",
-    "mental_model_history",
-    "knowledge_pages",
-    "directives",
-    "async_operations",
-    "webhooks",
-    "file_storage",
-    "audit_log",
-    "llm_requests",
-    "graph_maintenance_queue",
-]
-
-MANIFEST_VERSION = "1"
-
-
-async def _admin_connect(db_url: str) -> asyncpg.Connection:
-    """Open a raw asyncpg connection to an admin DB URL.
-
-    ``resolve_database_url`` handles both plain ``postgres://`` (passthrough) and
-    ``pg0://`` (boots the embedded server and returns its real libpq URL), so this
-    is the only step needed to connect. JSON codecs are registered so ``jsonb``
-    columns decode to Python objects (used by the export row dumps).
-    """
-    is_pg0, instance_name, _ = parse_pg0_url(db_url)
-    if is_pg0:
-        typer.echo(f"Starting embedded PostgreSQL (instance: {instance_name})...")
-    conn = await asyncpg.connect(await resolve_database_url(db_url))
-    for type_name in ("json", "jsonb"):
-        await conn.set_type_codec(type_name, encoder=json.dumps, decoder=json.loads, schema="pg_catalog")
-    return conn
-
-
-async def _backup(database_url: str, output_path: Path, schema: str = "public") -> dict[str, Any]:
-    """Backup all tables to a zip file using binary COPY protocol."""
-    conn = await asyncpg.connect(database_url)
-    try:
-        tables: dict[str, Any] = {}
-        manifest: dict[str, Any] = {
-            "version": MANIFEST_VERSION,
-            "created_at": datetime.now(timezone.utc).isoformat(),
-            "schema": schema,
-            "tables": tables,
-        }
-
-        # Use a transaction with REPEATABLE READ isolation to get a consistent
-        # snapshot across all tables. This prevents race conditions where
-        # entity_cooccurrences could reference entities created after the
-        # entities table was backed up.
-        async with conn.transaction(isolation="repeatable_read"):
-            with zipfile.ZipFile(output_path, "w", zipfile.ZIP_DEFLATED) as zf:
-                for i, table in enumerate(BACKUP_TABLES, 1):
-                    typer.echo(f"  [{i}/{len(BACKUP_TABLES)}] Backing up {table}...", nl=False)
-
-                    buffer = io.BytesIO()
-
-                    # Use binary COPY for exact type preservation
-                    # asyncpg requires schema_name as separate parameter
-                    await conn.copy_from_table(table, schema_name=schema, output=buffer, format="binary")
-
-                    data = buffer.getvalue()
-                    zf.writestr(f"{table}.bin", data)
-
-                    # Get row count for manifest
-                    qualified_table = _fq_table(table, schema)
-                    row_count = await conn.fetchval(f"SELECT COUNT(*) FROM {qualified_table}")
-                    tables[table] = {
-                        "rows": row_count,
-                        "size_bytes": len(data),
-                    }
-
-                    typer.echo(f" {row_count} rows")
-
-                zf.writestr("manifest.json", json.dumps(manifest, indent=2))
-
-        return manifest
-    finally:
-        await conn.close()
-
-
-async def _restore(database_url: str, input_path: Path, schema: str = "public") -> dict[str, Any]:
-    """Restore all tables from a zip file using binary COPY protocol."""
-    conn = await asyncpg.connect(database_url)
-    try:
-        with zipfile.ZipFile(input_path, "r") as zf:
-            # Read and validate manifest
-            manifest: dict[str, Any] = json.loads(zf.read("manifest.json"))
-            if manifest.get("version") != MANIFEST_VERSION:
-                raise ValueError(f"Unsupported backup version: {manifest.get('version')}")
-
-            # Use a transaction for atomic restore - either all tables are
-            # restored or none are, preventing partial/inconsistent state.
-            async with conn.transaction():
-                typer.echo("  Clearing existing data...")
-                # Truncate tables in reverse order (respects FK constraints)
-                for table in reversed(BACKUP_TABLES):
-                    qualified_table = _fq_table(table, schema)
-                    await conn.execute(f"TRUNCATE TABLE {qualified_table} CASCADE")
-
-                # Restore tables in forward order
-                for i, table in enumerate(BACKUP_TABLES, 1):
-                    filename = f"{table}.bin"
-                    if filename not in zf.namelist():
-                        typer.echo(f"  [{i}/{len(BACKUP_TABLES)}] {table}: skipped (not in backup)")
-                        continue
-
-                    expected_rows = manifest["tables"].get(table, {}).get("rows", "?")
-                    typer.echo(f"  [{i}/{len(BACKUP_TABLES)}] Restoring {table}... {expected_rows} rows")
-
-                    data = zf.read(filename)
-                    buffer = io.BytesIO(data)
-                    # asyncpg requires schema_name as separate parameter
-                    await conn.copy_to_table(table, schema_name=schema, source=buffer, format="binary")
-
-                # Refresh materialized view
-                typer.echo("  Refreshing materialized views...")
-                await conn.execute(f"REFRESH MATERIALIZED VIEW {_fq_table('memory_units_bm25', schema)}")
-
-        return manifest
-    finally:
-        await conn.close()
-
-
-async def _run_backup(db_url: str, output: Path, schema: str = "public") -> dict[str, Any]:
-    """Resolve database URL and run backup."""
-    is_pg0, instance_name, _ = parse_pg0_url(db_url)
-    if is_pg0:
-        typer.echo(f"Starting embedded PostgreSQL (instance: {instance_name})...")
-    resolved_url = await resolve_database_url(db_url)
-    return await _backup(resolved_url, output, schema)
-
-
-async def _run_restore(db_url: str, input_file: Path, schema: str = "public") -> dict[str, Any]:
-    """Resolve database URL and run restore."""
-    is_pg0, instance_name, _ = parse_pg0_url(db_url)
-    if is_pg0:
-        typer.echo(f"Starting embedded PostgreSQL (instance: {instance_name})...")
-    resolved_url = await resolve_database_url(db_url)
-    return await _restore(resolved_url, input_file, schema)
-
-
-@app.command()
-def backup(
-    output: Path = typer.Argument(..., help="Output file path (.zip)"),
-    schema: str = typer.Option("public", "--schema", "-s", help="Database schema to backup"),
-):
-    """Backup the Hindsight database to a zip file."""
-    config = HindsightConfig.from_env()
-
-    if not config.database_url:
-        typer.echo("Error: Database URL not configured.", err=True)
-        typer.echo("Set HINDSIGHT_API_DATABASE_URL environment variable.", err=True)
-        raise typer.Exit(1)
-
-    if output.suffix != ".zip":
-        output = output.with_suffix(".zip")
-
-    typer.echo(f"Backing up database (schema: {schema}) to {output}...")
-
-    manifest = asyncio.run(_run_backup(config.database_url, output, schema))
-
-    total_rows = sum(t["rows"] for t in manifest["tables"].values())
-    typer.echo(f"Backed up {total_rows} rows across {len(BACKUP_TABLES)} tables")
-    typer.echo(f"Backup saved to {output}")
-
-
-@app.command()
-def restore(
-    input_file: Path = typer.Argument(..., help="Input backup file (.zip)"),
-    schema: str = typer.Option("public", "--schema", "-s", help="Database schema to restore to"),
-    yes: bool = typer.Option(False, "--yes", "-y", help="Skip confirmation prompt"),
-):
-    """Restore the database from a backup file. WARNING: This deletes all existing data."""
-    config = HindsightConfig.from_env()
-
-    if not config.database_url:
-        typer.echo("Error: Database URL not configured.", err=True)
-        typer.echo("Set HINDSIGHT_API_DATABASE_URL environment variable.", err=True)
-        raise typer.Exit(1)
-
-    if not input_file.exists():
-        typer.echo(f"Error: File not found: {input_file}", err=True)
-        raise typer.Exit(1)
-
-    if not yes:
-        typer.confirm(
-            "This will DELETE all existing data and replace it with the backup. Continue?",
-            abort=True,
-        )
-
-    typer.echo(f"Restoring database (schema: {schema}) from {input_file}...")
-
-    manifest = asyncio.run(_run_restore(config.database_url, input_file, schema))
-
-    total_rows = sum(t["rows"] for t in manifest["tables"].values())
-    typer.echo(f"Restored {total_rows} rows across {len(BACKUP_TABLES)} tables")
-    typer.echo("Restore complete")
-
-
-async def _run_migration(
-    db_url: str,
-    schema: str | None = None,
-    base_schema: str = DEFAULT_DATABASE_SCHEMA,
-    embedding_dimension: int | None = None,
-    ensure_extensions: bool = True,
-) -> list[str]:
-    """Resolve database URL and run migrations for one schema or all discovered schemas."""
-    from ..migrations import run_migrations_for_schemas
-
-    is_pg0, instance_name, _ = parse_pg0_url(db_url)
-    if is_pg0:
-        typer.echo(f"Starting embedded PostgreSQL (instance: {instance_name})...")
-    resolved_url = await resolve_database_url(db_url)
-
-    config = HindsightConfig.from_env()
-    if schema:
-        schemas = [schema]
-    else:
-        tenant_extension = load_extension("TENANT", TenantExtension)
-
-        schemas = [base_schema or DEFAULT_DATABASE_SCHEMA]
-        if tenant_extension:
-            tenants = await tenant_extension.list_tenants()
-            schemas.extend(tenant.schema for tenant in tenants if tenant.schema)
-
-        # Preserve order while removing duplicates.
-        schemas = list(dict.fromkeys(schemas))
-
-    # Migrate up to `migration_concurrency` schemas at once (each in its own
-    # process); within a schema the work stays sequential. Run off the event
-    # loop so the process pool's blocking joins don't stall it.
-    await asyncio.to_thread(
-        run_migrations_for_schemas,
-        resolved_url,
-        schemas,
-        concurrency=config.migration_concurrency,
-        migration_database_url=config.migration_database_url,
-        embedding_dimension=embedding_dimension,
-        vector_extension=config.vector_extension,
-        text_search_extension=config.text_search_extension,
-        pg_search_tokenizer=config.text_search_extension_pg_search_tokenizer,
-        ensure_extensions=ensure_extensions,
-    )
-
-    return schemas
-
-
-@app.command(name="run-db-migration")
-def run_db_migration(
-    schema: str | None = typer.Option(
-        None,
-        "--schema",
-        "-s",
-        help="Database schema to run migrations on. If omitted, migrate the base schema and all discovered tenant schemas.",
-    ),
-    embedding_dimension: int | None = typer.Option(
-        None,
-        "--embedding-dimension",
-        help="Expected embedding dimension to enforce after migrations. Omit to skip dimension sync.",
-    ),
-    skip_extension_reconcile: bool = typer.Option(
-        False,
-        "--skip-extension-reconcile",
-        help=(
-            "Skip the post-migration vector / text-search index reconcile. This step only does "
-            "work when the configured backend (HINDSIGHT_API_VECTOR_EXTENSION / "
-            "HINDSIGHT_API_TEXT_SEARCH_EXTENSION) differs from a schema's existing indexes — a "
-            "rare, operator-driven change. Skipping it makes a no-change re-migration over many "
-            "tenant schemas much faster. Only use when you have NOT changed the backend; a "
-            "backend change still needs a normal run to reshape the indexes."
-        ),
-    ),
-):
-    """Run database migrations to the latest version."""
-    config = HindsightConfig.from_env()
-
-    if not config.database_url:
-        typer.echo("Error: Database URL not configured.", err=True)
-        typer.echo("Set HINDSIGHT_API_DATABASE_URL environment variable.", err=True)
-        raise typer.Exit(1)
-
-    if schema:
-        typer.echo(f"Running database migrations for schema: {schema}...")
-    else:
-        typer.echo("Running database migrations for base schema and all discovered tenant schemas...")
-    if skip_extension_reconcile:
-        typer.echo("Skipping post-migration extension reconcile (--skip-extension-reconcile).")
-
-    schemas = asyncio.run(
-        _run_migration(
-            config.database_url,
-            schema=schema,
-            base_schema=config.database_schema,
-            embedding_dimension=embedding_dimension,
-            ensure_extensions=not skip_extension_reconcile,
-        )
-    )
-
-    typer.echo(f"Database migrations completed successfully for {len(schemas)} schema(s)")
-
-
-async def _run_export_bank(db_url: str, bank_id: str, output: Path, schema: str, include_history: bool) -> int:
-    """Export a whole bank to a ZIP archive."""
-    conn = await _admin_connect(db_url)
-    try:
-        # export_bank resolves table names via fq_table (the _current_schema
-        # contextvar); set it so the raw connection targets the right schema.
-        _current_schema.set(schema)
-        data = await export_bank(conn, bank_id, include_history=include_history)
-    finally:
-        await conn.close()
-
-    output.write_bytes(data)
-    return len(data)
-
-
-@app.command(name="export-bank")
-def export_bank_command(
-    bank_id: str = typer.Option(..., "--bank", "-b", help="Bank id to export."),
-    output: Path = typer.Option(..., "--output", "-o", help="Path to write the .zip archive."),
-    schema: str | None = typer.Option(
-        None,
-        "--schema",
-        "-s",
-        help="Database schema the bank lives in. Defaults to the configured base schema.",
-    ),
-    include_history: bool = typer.Option(
-        False,
-        "--include-history",
-        help="Also export operational history (audit_log, llm_requests). Off by default.",
-    ),
-):
-    """Export an entire bank to a portable ZIP (no embeddings — regenerated on import).
-
-    Carries documents, facts, observations, bank config, mental models, directives
-    and webhooks so the bank can be imported into a new instance configured with a
-    different embedding model / vector / text-search backend.
-    """
-    config = HindsightConfig.from_env()
-
-    if not config.database_url:
-        typer.echo("Error: Database URL not configured.", err=True)
-        typer.echo("Set HINDSIGHT_API_DATABASE_URL environment variable.", err=True)
-        raise typer.Exit(1)
-
-    target_schema = schema or config.database_schema or DEFAULT_DATABASE_SCHEMA
-    typer.echo(f"Exporting bank '{bank_id}' from schema '{target_schema}'...")
-
-    size = asyncio.run(_run_export_bank(config.database_url, bank_id, output, target_schema, include_history))
-
-    typer.echo(f"Exported bank '{bank_id}' to {output} ({size} bytes)")
-
-
-async def _run_import_bank(archive_path: Path, schema: str, target_bank_id: str | None, include_history: bool):
-    """Boot a MemoryEngine (for the target's embedding model) and restore a bank archive."""
-    # MemoryEngine is heavy (loads embeddings); import it lazily so other admin
-    # commands don't pay for it. _current_schema is imported at module top.
-    from ..engine.memory_engine import MemoryEngine
-    from ..models import RequestContext
-
-    archive_bytes = archive_path.read_bytes()
-    # run_migrations=True so a fresh target instance is provisioned at this
-    # instance's embedding dimension / vector / text-search backend before restore.
-    engine = MemoryEngine(run_migrations=True)
-    await engine.initialize()
-    try:
-        _current_schema.set(schema)
-        context = RequestContext(internal=True, user_initiated=True)
-        return await engine.import_bank_async(
-            archive_bytes,
-            context,
-            target_bank_id=target_bank_id,
-            include_history=include_history,
-        )
-    finally:
-        await engine.close()
-
-
-@app.command(name="import-bank")
-def import_bank_command(
-    archive: Path = typer.Option(..., "--archive", "-a", help="Path to the .zip produced by export-bank."),
-    schema: str | None = typer.Option(
-        None, "--schema", "-s", help="Target schema. Defaults to the configured base schema."
-    ),
-    target_bank: str | None = typer.Option(
-        None, "--target-bank", help="Override the bank id (defaults to the archive's source bank)."
-    ),
-    include_history: bool = typer.Option(
-        False, "--include-history", help="Also restore operational history if present in the archive."
-    ),
-):
-    """Restore a whole bank from an export-bank archive into THIS instance.
-
-    Re-embeds facts with this instance's configured embedding model and rebuilds
-    links and indexes — the import half of a cross-instance migration. Run against
-    an instance configured with the desired embedding / vector / text-search backend.
-    The target bank must not already exist (import restores a whole bank, not a merge).
-    """
-    config = HindsightConfig.from_env()
-    if not config.database_url:
-        typer.echo("Error: Database URL not configured.", err=True)
-        typer.echo("Set HINDSIGHT_API_DATABASE_URL environment variable.", err=True)
-        raise typer.Exit(1)
-
-    target_schema = schema or config.database_schema or DEFAULT_DATABASE_SCHEMA
-    typer.echo(f"Importing bank archive '{archive}' into schema '{target_schema}'...")
-
-    result = asyncio.run(_run_import_bank(archive, target_schema, target_bank, include_history))
-
-    typer.echo(
-        f"Imported bank '{result.bank_id}': {result.documents_imported} doc(s), "
-        f"{result.facts_imported} fact(s), {result.observations_imported} observation(s), "
-        f"{result.mental_models_imported} mental model(s), "
-        f"{result.mental_model_history_imported} mm-history row(s), {result.directives_imported} directive(s), "
-        f"{result.webhooks_imported} webhook(s), {result.history_rows_imported} history row(s)"
-    )
-
-
-async def _decommission_worker(db_url: str, worker_id: str, schema: str = "public") -> int:
-    """Release all tasks owned by a worker, setting them back to pending status."""
-    is_pg0, instance_name, _ = parse_pg0_url(db_url)
-    if is_pg0:
-        typer.echo(f"Starting embedded PostgreSQL (instance: {instance_name})...")
-    resolved_url = await resolve_database_url(db_url)
-
-    conn = await asyncpg.connect(resolved_url)
-    try:
-        table = _fq_table("async_operations", schema)
-        result = await conn.fetch(
-            f"""
-            UPDATE {table}
-            SET status = 'pending', worker_id = NULL, claimed_at = NULL, updated_at = now()
-            WHERE worker_id = $1 AND status = 'processing'
-            RETURNING operation_id
-            """,
-            worker_id,
-        )
-        return len(result)
-    finally:
-        await conn.close()
-
-
-@app.command(name="decommission-worker")
-def decommission_worker(
-    worker_id: str = typer.Argument(..., help="Worker ID to decommission"),
-    schema: str = typer.Option("public", "--schema", "-s", help="Database schema"),
-    yes: bool = typer.Option(False, "--yes", "-y", help="Skip confirmation prompt"),
-):
-    """Release all tasks owned by a worker (sets status back to pending).
-
-    Use this command when a worker has crashed or been removed without graceful shutdown.
-    All tasks that were being processed by the worker will be released back to the queue
-    so other workers can pick them up.
-    """
-    config = HindsightConfig.from_env()
-
-    if not config.database_url:
-        typer.echo("Error: Database URL not configured.", err=True)
-        typer.echo("Set HINDSIGHT_API_DATABASE_URL environment variable.", err=True)
-        raise typer.Exit(1)
-
-    if not yes:
-        typer.confirm(
-            f"This will release all tasks owned by worker '{worker_id}' back to pending. Continue?",
-            abort=True,
-        )
-
-    typer.echo(f"Decommissioning worker '{worker_id}' (schema: {schema})...")
-
-    count = asyncio.run(_decommission_worker(config.database_url, worker_id, schema))
-
-    if count > 0:
-        typer.echo(f"Released {count} task(s) from worker '{worker_id}'")
-    else:
-        typer.echo(f"No tasks found for worker '{worker_id}'")
-
-
-async def _decommission_all_workers(db_url: str, schema: str = "public") -> list[dict[str, Any]]:
-    """Release all processing tasks from all workers, setting them back to pending status."""
-    is_pg0, instance_name, _ = parse_pg0_url(db_url)
-    if is_pg0:
-        typer.echo(f"Starting embedded PostgreSQL (instance: {instance_name})...")
-    resolved_url = await resolve_database_url(db_url)
-
-    conn = await asyncpg.connect(resolved_url)
-    try:
-        table = _fq_table("async_operations", schema)
-        rows = await conn.fetch(
-            f"""
-            UPDATE {table}
-            SET status = 'pending', worker_id = NULL, claimed_at = NULL, updated_at = now()
-            WHERE status = 'processing'
-            RETURNING operation_id, worker_id, operation_type
-            """,
-        )
-        return [dict(r) for r in rows]
-    finally:
-        await conn.close()
-
-
-@app.command(name="decommission-workers")
-def decommission_workers(
-    schema: str = typer.Option("public", "--schema", "-s", help="Database schema"),
-    yes: bool = typer.Option(False, "--yes", "-y", help="Skip confirmation prompt"),
-):
-    """Release all processing tasks from all workers (sets status back to pending).
-
-    Use this command to recover from situations where one or more workers have crashed
-    or been removed without graceful shutdown. All tasks currently in 'processing' status
-    will be released back to the queue regardless of which worker owns them.
-    """
-    config = HindsightConfig.from_env()
-
-    if not config.database_url:
-        typer.echo("Error: Database URL not configured.", err=True)
-        typer.echo("Set HINDSIGHT_API_DATABASE_URL environment variable.", err=True)
-        raise typer.Exit(1)
-
-    if not yes:
-        typer.confirm(
-            "This will release ALL processing tasks from ALL workers back to pending. Continue?",
-            abort=True,
-        )
-
-    typer.echo(f"Decommissioning all workers (schema: {schema})...")
-
-    released = asyncio.run(_decommission_all_workers(config.database_url, schema))
-
-    if released:
-        # Group by worker_id for summary
-        by_worker: dict[str, int] = {}
-        for row in released:
-            wid = row["worker_id"] or "unknown"
-            by_worker[wid] = by_worker.get(wid, 0) + 1
-
-        typer.echo(f"Released {len(released)} task(s):")
-        for wid, count in by_worker.items():
-            typer.echo(f"  {wid}: {count} task(s)")
-    else:
-        typer.echo("No processing tasks found")
-
-
-async def _worker_status(db_url: str, schema: str = "public") -> list[dict[str, Any]]:
-    """Get all processing tasks grouped by worker with their last update time."""
-    is_pg0, instance_name, _ = parse_pg0_url(db_url)
-    if is_pg0:
-        typer.echo(f"Starting embedded PostgreSQL (instance: {instance_name})...")
-    resolved_url = await resolve_database_url(db_url)
-
-    conn = await asyncpg.connect(resolved_url)
-    try:
-        table = _fq_table("async_operations", schema)
-        rows = await conn.fetch(
-            f"""
-            SELECT worker_id, operation_id, operation_type, bank_id,
-                   claimed_at, updated_at,
-                   now() - claimed_at AS running_for,
-                   now() - updated_at AS last_update_ago
-            FROM {table}
-            WHERE status = 'processing'
-            ORDER BY worker_id, claimed_at
-            """,
-        )
-        return [dict(r) for r in rows]
-    finally:
-        await conn.close()
-
-
-@app.command(name="worker-status")
-def worker_status(
-    schema: str = typer.Option("public", "--schema", "-s", help="Database schema"),
-):
-    """Show all currently processing tasks grouped by worker.
-
-    Displays each worker's active tasks with operation type, bank, how long
-    the task has been running, and when it was last updated. Useful for
-    identifying dead workers with orphaned tasks.
-    """
-    config = HindsightConfig.from_env()
-
-    if not config.database_url:
-        typer.echo("Error: Database URL not configured.", err=True)
-        typer.echo("Set HINDSIGHT_API_DATABASE_URL environment variable.", err=True)
-        raise typer.Exit(1)
-
-    rows = asyncio.run(_worker_status(config.database_url, schema))
-
-    if not rows:
-        typer.echo("No processing tasks found")
-        return
-
-    # Group by worker_id
-    by_worker: dict[str, list[dict[str, Any]]] = {}
-    for row in rows:
-        wid = row["worker_id"] or "unknown"
-        by_worker.setdefault(wid, []).append(row)
-
-    typer.echo(f"Processing tasks across {len(by_worker)} worker(s):\n")
-    for wid, tasks in by_worker.items():
-        typer.echo(f"Worker: {wid} ({len(tasks)} task(s))")
-        for task in tasks:
-            op_id = str(task["operation_id"])[:8]
-            running_for = task["running_for"]
-            last_update = task["last_update_ago"]
-            typer.echo(
-                f"  {op_id}  {task['operation_type']:<20s} bank={task['bank_id']}"
-                f"  running={running_for}  last_update={last_update} ago"
-            )
-        typer.echo("")
-
-
-def main():
-    app()
-
-
-if __name__ == "__main__":
-    main()

hindsight-api-slim/hindsight_api/alembic/_dialect.py

@@ -1,42 +0,0 @@
-"""Dialect dispatcher for Alembic migrations.
-
-Each migration file declares a ``_pg_upgrade``/``_oracle_upgrade`` (and matching
-downgrades) function and routes ``upgrade()``/``downgrade()`` through
-``run_for_dialect``. The helper inspects the live connection's dialect name and
-runs the matching function — or no-ops if the migration doesn't apply to the
-current backend.
-
-Use ``None`` (or omit the kwarg) when a migration intentionally has no effect
-on a dialect; the helper treats it as a no-op.
-"""
-
-from __future__ import annotations
-
-from collections.abc import Callable
-
-from alembic import op
-
-DialectFn = Callable[[], None]
-_SUPPORTED = ("postgresql", "oracle")
-
-
-def run_for_dialect(
-    *,
-    pg: DialectFn | None = None,
-    oracle: DialectFn | None = None,
-) -> None:
-    """Dispatch to the function matching the current bind's dialect.
-
-    Args:
-        pg: Function to run when the active bind is PostgreSQL.
-        oracle: Function to run when the active bind is Oracle.
-
-    Unrecognized dialects raise; an explicit ``None`` for the active dialect
-    is a no-op (the migration deliberately does nothing here).
-    """
-    name = op.get_bind().dialect.name
-    if name not in _SUPPORTED:
-        raise RuntimeError(f"Unsupported dialect for migration dispatch: {name!r}. Expected one of {_SUPPORTED}.")
-    fn = {"postgresql": pg, "oracle": oracle}[name]
-    if fn is not None:
-        fn()

hindsight-api-slim/hindsight_api/alembic/env.py

@@ -1,191 +0,0 @@
-"""
-Alembic environment for Hindsight.
-
-Supports two dialects:
-
-* PostgreSQL (sync psycopg2 driver) — default; uses ``search_path`` for
-  multi-tenant schema isolation and forces read-write transactions to work
-  around Supabase's read-only-by-default sessions.
-* Oracle 23ai (``oracledb`` driver) — uses ``CURRENT_SCHEMA`` for tenant
-  isolation; no equivalent of ``search_path`` or read-only session quirks.
-
-Each migration file dispatches its DDL through ``alembic._dialect.run_for_dialect``
-so a single revision tree serves both backends.
-"""
-
-import logging
-import os
-from pathlib import Path
-from urllib.parse import parse_qsl, urlencode, urlsplit, urlunsplit
-
-from alembic import context
-from dotenv import load_dotenv
-from sqlalchemy import Connection, engine_from_config, pool
-from sqlalchemy.engine import Engine
-
-from hindsight_api.db_url import is_oracle_url, to_libpq_url
-from hindsight_api.models import Base
-
-
-def load_env() -> None:
-    """Load environment variables from .env (skipped if already configured)."""
-    if os.getenv("HINDSIGHT_API_DATABASE_URL"):
-        return
-
-    root_dir = Path(__file__).parent.parent.parent
-    env_file = root_dir / ".env"
-
-    if env_file.exists():
-        load_dotenv(env_file)
-
-
-load_env()
-
-config = context.config
-target_metadata = Base.metadata
-
-
-def _normalize_oracle_url(url: str) -> str:
-    """Coerce an Oracle URL into the SQLAlchemy form the oracledb dialect expects.
-
-    Two issues to handle:
-
-    1. Force the ``oracle+oracledb`` driver — bare ``oracle://`` defaults to
-       cx_Oracle.
-    2. Map a path-style service to ``?service_name=...``. SQLAlchemy's oracledb
-       dialect treats the URL path as a *SID* (legacy), but Oracle Free /
-       Autonomous DB only register a service name. Without this rewrite we get
-       ``DPY-6003: SID "FREEPDB1" is not registered`` even though the listener
-       is happy to accept the same name as a service.
-    """
-    parts = urlsplit(url)
-    if not parts.scheme.startswith("oracle"):
-        return url
-
-    new_scheme = "oracle+oracledb" if parts.scheme == "oracle" else parts.scheme
-    service = parts.path.lstrip("/")
-    new_query = parts.query
-    new_path = parts.path
-
-    # Promote /SERVICE to ?service_name=SERVICE unless the caller already
-    # supplied an explicit ?sid= or ?service_name=.
-    if service and "service_name=" not in new_query and "sid=" not in new_query:
-        params = [(k, v) for k, v in parse_qsl(new_query, keep_blank_values=True)]
-        params.append(("service_name", service))
-        new_query = urlencode(params)
-        new_path = ""
-
-    return urlunsplit((new_scheme, parts.netloc, new_path, new_query, parts.fragment))
-
-
-def get_database_url() -> str:
-    """Resolve the migration URL from Alembic config or env, normalizing per-dialect."""
-    database_url = config.get_main_option("sqlalchemy.url")
-    if not database_url:
-        database_url = os.getenv("HINDSIGHT_API_DATABASE_URL")
-        if not database_url:
-            raise ValueError(
-                "Database URL not found. "
-                "Set HINDSIGHT_API_DATABASE_URL environment variable or pass database_url to run_migrations()."
-            )
-
-    if is_oracle_url(database_url):
-        database_url = _normalize_oracle_url(database_url)
-    else:
-        # PG: convert SQLAlchemy-style asyncpg URLs and ?ssl= params to libpq form
-        # for the sync engine used during migrations.
-        database_url = to_libpq_url(database_url)
-
-    config.set_main_option("sqlalchemy.url", database_url)
-    return database_url
-
-
-def run_migrations_offline() -> None:
-    logging.info("running offline")
-    database_url = get_database_url()
-
-    context.configure(
-        url=database_url,
-        target_metadata=target_metadata,
-        literal_binds=True,
-        dialect_opts={"paramstyle": "named"},
-    )
-
-    with context.begin_transaction():
-        context.run_migrations()
-
-
-def _configure_pg_session(engine: Engine, connection: Connection, target_schema: str | None) -> None:
-    """PG-only: ensure the session is RW (Supabase) and bind ``search_path``."""
-    from sqlalchemy import event, text
-
-    @event.listens_for(engine, "connect")
-    def set_read_write_mode(dbapi_connection, connection_record):
-        cursor = dbapi_connection.cursor()
-        cursor.execute("SET SESSION CHARACTERISTICS AS TRANSACTION READ WRITE")
-        if target_schema:
-            cursor.execute(f'CREATE SCHEMA IF NOT EXISTS "{target_schema}"')
-            cursor.execute(f'SET search_path TO "{target_schema}", public')
-        cursor.close()
-
-    connection.execute(text("SET SESSION CHARACTERISTICS AS TRANSACTION READ WRITE"))
-    if target_schema:
-        connection.execute(text(f'CREATE SCHEMA IF NOT EXISTS "{target_schema}"'))
-        connection.execute(text(f'SET search_path TO "{target_schema}", public'))
-    connection.commit()
-
-
-def _configure_oracle_session(connection: Connection, target_schema: str | None) -> None:
-    """Oracle: switch the session's default schema; tolerate DDL contention."""
-    from sqlalchemy import text
-
-    # Wait up to 30s for DDL locks instead of failing immediately (ORA-00054).
-    connection.execute(text("ALTER SESSION SET DDL_LOCK_TIMEOUT = 30"))
-    if target_schema:
-        connection.execute(text(f'ALTER SESSION SET CURRENT_SCHEMA = "{target_schema}"'))
-
-
-def run_migrations_online() -> None:
-    database_url = get_database_url()
-    target_schema = config.get_main_option("target_schema")
-    is_oracle = is_oracle_url(database_url)
-
-    connectable = engine_from_config(
-        config.get_section(config.config_ini_section, {}),
-        prefix="sqlalchemy.",
-        poolclass=pool.NullPool,
-    )
-
-    with connectable.connect() as connection:
-        if is_oracle:
-            _configure_oracle_session(connection, target_schema)
-        else:
-            _configure_pg_session(connectable, connection, target_schema)
-
-        context_opts = {
-            "connection": connection,
-            "target_metadata": target_metadata,
-        }
-        if target_schema and not is_oracle:
-            # Oracle has no equivalent of PG's per-schema version table; the
-            # ``alembic_version`` table lives in CURRENT_SCHEMA implicitly.
-            context_opts["version_table_schema"] = target_schema
-
-        context.configure(**context_opts)
-
-        with context.begin_transaction():
-            context.run_migrations()
-
-        # Always commit. PG needs it for the explicit RW-mode SET to persist;
-        # Oracle needs it because each DDL auto-commits but the trailing
-        # ``UPDATE alembic_version`` is plain DML that would otherwise stay in
-        # an open transaction and roll back when the connection closes —
-        # producing the "schema is created but the version row is one revision
-        # behind" failure mode.
-        connection.commit()
-
-
-if context.is_offline_mode():
-    run_migrations_offline()
-else:
-    run_migrations_online()

hindsight-api-slim/hindsight_api/alembic/versions/2071c7518f88_add_memory_links_bank_id_index.py

@@ -1,105 +0,0 @@
-"""Add a composite index on memory_links(bank_id, link_type) (PostgreSQL).
-
-``bank_id`` was added to ``memory_links`` in ``c5d6e7f8a9b0`` precisely so that
-bank-scoped reads (e.g. the stats endpoint) could filter on the link table
-directly instead of joining ``memory_units`` — that JOIN took 18+ seconds on
-banks with millions of links. The column landed without an index, so every
-``bank_id = $1`` predicate still falls back to a sequential scan over the whole
-table.
-
-This adds the missing btree. It is composite on ``(bank_id, link_type)`` rather
-than ``bank_id`` alone because the hot query is the stats endpoint's
-``SELECT link_type, COUNT(*) ... WHERE bank_id = $1 GROUP BY link_type``: a
-``(bank_id, link_type)`` index serves that filter, grouping and count as an
-index-only scan, never touching the heap, whereas a ``bank_id``-only index would
-still have to read every matching row to recover ``link_type``. ``link_type`` is
-low-cardinality (only ``temporal``/``semantic``/``caused_by`` are written —
-entity edges were dropped in ``e9b2c7d1f3a4``), so the trailing column adds
-little to the index size while removing the heap fetch.
-
-The Oracle baseline (``o1a2b3c4d5e6``) already creates ``idx_ml_bank_id`` on
-``memory_links(bank_id)``; that single-column index already covers Oracle's
-bank-scoped filter, so the Oracle slot here is intentionally absent and only the
-PostgreSQL dialect gets the composite index.
-
-``memory_links`` can hold tens of millions of rows, so the index is built
-CONCURRENTLY to avoid taking a write lock on the table. CONCURRENTLY cannot run
-inside a transaction block, so the statement runs in an ``autocommit_block()``;
-``IF NOT EXISTS`` keeps it idempotent across retries and re-migrated tenant
-schemas. A CONCURRENTLY build interrupted partway (lock conflict, disk
-pressure, signal) leaves the index behind as *invalid*; ``IF NOT EXISTS`` would
-then skip over it forever, so the upgrade first drops any invalid leftover of
-this name before (re)creating it.
-
-Revision ID: 2071c7518f88
-Revises: a1d3f5b7c9e2
-Create Date: 2026-06-16
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-from sqlalchemy import text
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "2071c7518f88"
-down_revision: str | Sequence[str] | None = "a1d3f5b7c9e2"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-_INDEX_NAME = "idx_memory_links_bank_id_link_type"
-
-
-def _get_schema_prefix() -> str:
-    """Schema-qualifier for raw SQL on PG (multi-tenant search_path)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _pg_upgrade() -> None:
-    bind = op.get_bind()
-    # `or None` collapses an unset option and an explicit empty string into NULL
-    # so the COALESCE below falls back to current_schema() in both cases.
-    target_schema = context.config.get_main_option("target_schema") or None
-    schema = _get_schema_prefix()
-
-    # CREATE INDEX CONCURRENTLY cannot run inside a transaction block; the
-    # autocommit_block runs each statement outside Alembic's migration
-    # transaction.
-    with op.get_context().autocommit_block():
-        # A CONCURRENTLY build that errored on a previous run leaves an INVALID
-        # index of this name behind. `CREATE INDEX ... IF NOT EXISTS` would see
-        # that relation and skip, so bank_id queries would keep seq-scanning.
-        # Drop only the invalid leftover — never a healthy index — so the retry
-        # actually rebuilds a usable one.
-        leftover_invalid = bind.execute(
-            text(
-                "SELECT NOT i.indisvalid "
-                "FROM pg_class c "
-                "JOIN pg_index i ON c.oid = i.indexrelid "
-                "JOIN pg_namespace n ON c.relnamespace = n.oid "
-                "WHERE c.relname = :index_name "
-                "  AND n.nspname = COALESCE(:target_schema, current_schema())"
-            ),
-            {"index_name": _INDEX_NAME, "target_schema": target_schema},
-        ).scalar()
-        if leftover_invalid:
-            op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}{_INDEX_NAME}")
-
-        # IF NOT EXISTS keeps the create idempotent across retries and schemas.
-        op.execute(f"CREATE INDEX CONCURRENTLY IF NOT EXISTS {_INDEX_NAME} ON {schema}memory_links(bank_id, link_type)")
-
-
-def _pg_downgrade() -> None:
-    schema = _get_schema_prefix()
-    with op.get_context().autocommit_block():
-        op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}{_INDEX_NAME}")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/2eee35aa3cfc_case_insensitive_entities_trgm_index.py

@@ -1,55 +0,0 @@
-"""Recreate entities trigram index on LOWER(canonical_name) for case-insensitive matching
-
-The previous GIN trigram index on canonical_name was case-sensitive, causing
-"Alice" and "alice" to have different trigram sets. This recreates it on
-LOWER(canonical_name) so the % operator matches case-insensitively.
-
-Revision ID: 2eee35aa3cfc
-Revises: d6e7f8a9b0c1
-Create Date: 2026-03-31
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "2eee35aa3cfc"
-down_revision: str | Sequence[str] | None = "d6e7f8a9b0c1"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _pg_upgrade() -> None:
-    schema = _get_schema_prefix()
-    # Drop the old case-sensitive trigram index
-    op.execute("DROP INDEX IF EXISTS entities_canonical_name_trgm_idx")
-    # Create case-insensitive trigram index on LOWER(canonical_name)
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS entities_canonical_name_lower_trgm_idx "
-        f"ON {schema}entities USING GIN (LOWER(canonical_name) gin_trgm_ops)"
-    )
-
-
-def _pg_downgrade() -> None:
-    op.execute("DROP INDEX IF EXISTS entities_canonical_name_lower_trgm_idx")
-    schema = _get_schema_prefix()
-    # Restore original case-sensitive index
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS entities_canonical_name_trgm_idx "
-        f"ON {schema}entities USING GIN (canonical_name gin_trgm_ops)"
-    )
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/86f7a033d372_repair_mental_models_subtype_at_head.py

@@ -1,117 +0,0 @@
-"""Repair mental_models.subtype on databases stuck at m3rg3h3ad5f6
-
-Three production deployments reported `column "subtype" of relation
-"mental_models" does not exist` on `create_mental_model` even after their
-container reported `Database migrations completed successfully` and
-`alembic_version` advanced to `m3rg3h3ad5f6` (see issue #1553, #1553#1
-confirmations from @4Lienau and @khanhduyvt0101).
-
-Both `h3c4d5e6f7g8_mental_models_v4` and `d5y6z7a8b9c0_backfill_mental_models_subtype`
-were meant to ensure `subtype` exists, but on databases that came through the
-`reflections -> mental_models` rename chain *and* whose alembic_version
-advanced past `d5y6z7a8b9c0` along an alternate path during the divergent-heads
-reorganization, neither column-add actually fired. The result is a head-tagged
-database with a v3-shaped `mental_models` table missing six columns:
-``subtype``, ``description``, ``entity_id``, ``observations``, ``links``,
-``last_updated``.
-
-This migration sits at the current head (`m3rg3h3ad5f6`) so every affected
-deployment will pick it up on next container start. It mirrors the column-add
-block from `d5y6z7a8b9c0_backfill_mental_models_subtype` using
-``ADD COLUMN IF NOT EXISTS`` so it is a no-op on databases where the columns
-are already present.
-
-Revision ID: 86f7a033d372
-Revises: m3rg3h3ad5f6
-Create Date: 2026-05-14
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "86f7a033d372"
-down_revision: str | Sequence[str] | None = "m3rg3h3ad5f6"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _pg_schema_prefix() -> str:
-    """Schema-qualifier for raw SQL on PG (multi-tenant search_path)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _pg_upgrade() -> None:
-    """Idempotently ensure mental_models has the v4 column set.
-
-    Safe to re-apply on databases that already received the columns via
-    `h3c4d5e6f7g8_mental_models_v4` or `d5y6z7a8b9c0_backfill_mental_models_subtype` —
-    every column-add uses ``IF NOT EXISTS`` and the constraint is recreated
-    from scratch with the canonical v4 allowlist.
-    """
-    schema = _pg_schema_prefix()
-    bare_schema = schema.strip(".").strip('"') if schema else ""
-    schema_clause = f"AND table_schema = '{bare_schema}'" if bare_schema else ""
-
-    # Wrapped in a DO block so the existence check skips databases that
-    # predate the reflections -> mental_models rename chain (no table to
-    # repair). On those, every ALTER below would error.
-    op.execute(
-        f"""
-        DO $$
-        BEGIN
-            IF EXISTS (
-                SELECT 1 FROM information_schema.tables
-                WHERE table_name = 'mental_models'
-                  {schema_clause}
-            ) THEN
-                -- Add the six v4 columns idempotently.
-                ALTER TABLE {schema}mental_models
-                    ADD COLUMN IF NOT EXISTS subtype VARCHAR(32) NOT NULL DEFAULT 'structural';
-                ALTER TABLE {schema}mental_models
-                    ADD COLUMN IF NOT EXISTS description TEXT NOT NULL DEFAULT '';
-                ALTER TABLE {schema}mental_models
-                    ADD COLUMN IF NOT EXISTS entity_id UUID;
-                ALTER TABLE {schema}mental_models
-                    ADD COLUMN IF NOT EXISTS observations JSONB DEFAULT '{{"observations": []}}'::jsonb;
-                ALTER TABLE {schema}mental_models
-                    ADD COLUMN IF NOT EXISTS links VARCHAR[];
-                ALTER TABLE {schema}mental_models
-                    ADD COLUMN IF NOT EXISTS last_updated TIMESTAMP WITH TIME ZONE;
-
-                -- Recreate the CHECK constraint with the canonical v4 allowlist.
-                -- Existing rows with subtype = 'directive' (possible on databases
-                -- that ran the o0j1k2l3m4n5 directive-only path) are rewritten to
-                -- 'structural' first so the constraint add succeeds.
-                UPDATE {schema}mental_models SET subtype = 'structural' WHERE subtype = 'directive';
-
-                ALTER TABLE {schema}mental_models DROP CONSTRAINT IF EXISTS ck_mental_models_subtype;
-                ALTER TABLE {schema}mental_models
-                    ADD CONSTRAINT ck_mental_models_subtype
-                    CHECK (subtype IN ('structural', 'emergent', 'pinned', 'learned'));
-
-                CREATE INDEX IF NOT EXISTS idx_mental_models_subtype
-                    ON {schema}mental_models(bank_id, subtype);
-            END IF;
-        END$$;
-        """
-    )
-
-
-def _pg_downgrade() -> None:
-    """No-op: dropping these columns would corrupt v4 application code."""
-    pass
-
-
-def upgrade() -> None:
-    # PG-only: Oracle's baseline (o1a2b3c4d5e6) creates mental_models with its
-    # own subtype shape (chk_mm_subtype IN ('directive', 'pinned')) and a
-    # different table topology, so this PG-shaped repair does not apply.
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/8c6fa6f7230b_merge_v053_divergent_heads.py

@@ -1,50 +0,0 @@
-"""Merge divergent migration heads for v0.5.3
-
-v0.5.3 shipped with two migration heads that were never unified:
-
-  * ``c4x5y6z7a8b9`` — delta-refresh chain
-    (``add_last_refreshed_source_query`` ->
-     ``add_structured_content_to_mental_models`` ->
-     ``backsweep_orphan_observations_v2``)
-
-  * ``h3i4j5k6l7m8`` — per-bank vector indexes / audit log chain
-    (the ``merge_heads_and_add_unit_entities_index`` subtree)
-
-Both fork from ``z1u2v3w4x5y6``. Upgrades from v0.5.2 still succeed — the
-walker applies the three c4x5 revisions and leaves the database stamped at
-both heads — but the result is a split DAG: ``alembic upgrade head``
-(singular) is ambiguous, and any future migration has to pick one head as
-its parent, orphaning the other.
-
-This revision linearises the DAG into a single head. It has no schema
-effect.
-
-Revision ID: 8c6fa6f7230b
-Revises: c4x5y6z7a8b9, h3i4j5k6l7m8
-Create Date: 2026-04-18
-"""
-
-from collections.abc import Sequence
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "8c6fa6f7230b"
-down_revision: str | Sequence[str] | None = ("c4x5y6z7a8b9", "h3i4j5k6l7m8")
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _pg_upgrade() -> None:
-    pass
-
-
-def _pg_downgrade() -> None:
-    pass
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/9f8e7d6c5b4a_memory_links_deferrable_fk.py

@@ -1,128 +0,0 @@
-"""Make memory_links.from_unit_id and memory_links.to_unit_id FKs deferrable.
-
-Revision ID: 9f8e7d6c5b4a
-Revises: o1a2b3c4d5e6
-Create Date: 2026-05-03
-
-Background
-----------
-Concurrent retain (which INSERTs into ``memory_links``) and any code path
-that DELETEs a row whose deletion cascades into ``memory_links`` (e.g.
-delta-retain superseding chunks, which CASCADEs chunks → memory_units →
-memory_links) can deadlock under sustained single-tenant write load.
-
-The deadlock cycle:
-
-* Tx A: ``DELETE FROM chunks WHERE chunk_id = ANY(...)``
-  → CASCADE acquires row locks on memory_units, then on memory_links rows
-    where ``to_unit_id`` matches the deleted units.
-* Tx B: ``INSERT INTO memory_links (...)`` referencing one of the same
-  memory_units rows.
-  → The immediate FK check takes ``FOR KEY SHARE`` on those memory_units
-    rows.
-
-The two transactions take row locks on the same memory_units rows in
-opposite orders depending on which side started first. PostgreSQL detects
-the cycle and aborts one transaction; the loser is killed mid-batch, the
-winner continues. Workers then retry, but under sustained write load the
-pattern repeats.
-
-Fix
----
-Make both ``memory_links → memory_units`` FKs (``from_unit_id`` and
-``to_unit_id``) ``DEFERRABLE INITIALLY DEFERRED``. This pushes the FK
-check from INSERT time to COMMIT time:
-
-* INSERT no longer takes ``FOR KEY SHARE`` on the memory_units row → no
-  contention with the cascading DELETE's row lock.
-* At COMMIT the engine validates referential integrity in one shot. If a
-  cascade-DELETE has since removed the referenced unit, the INSERT
-  transaction commits OR fails with a clean FK violation (sqlstate
-  23503) instead of a deadlock (sqlstate 40P01).
-
-The ``WHERE EXISTS`` filter already in ``_bulk_insert_links`` continues to
-filter out the typical "stale unit_id" case at INSERT time; the deferred
-FK is only the backstop for the narrow race window between the EXISTS
-probe and COMMIT. ``ON DELETE CASCADE`` semantics are unchanged — only
-the *timing* of the constraint check moves.
-
-The ``entity_id`` FK on ``memory_links`` is not changed; entities are not
-involved in the observed deadlock cycle and leaving the constraint
-immediate keeps the error message specific when an entity row is missing.
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "9f8e7d6c5b4a"
-down_revision: str | Sequence[str] | None = "o1a2b3c4d5e6"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Schema-qualifier for raw SQL on PG (multi-tenant search_path)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-# The two FK constraints installed by the initial schema migration
-# (5a366d414dce_initial_schema), mapped to the column they constrain.
-# They reference memory_units(id) with ON DELETE CASCADE — that
-# semantics is preserved; only the deferral attribute changes.
-_FK_COLUMNS: dict[str, str] = {
-    "fk_memory_links_from_unit_id_memory_units": "from_unit_id",
-    "fk_memory_links_to_unit_id_memory_units": "to_unit_id",
-}
-
-
-def _pg_upgrade() -> None:
-    schema = _get_schema_prefix()
-    # PostgreSQL doesn't allow altering the deferrability of an existing
-    # constraint with ALTER CONSTRAINT — the constraint must be dropped
-    # and recreated. DROP IF EXISTS makes the migration safe to re-run
-    # on schemas where the constraint was already recreated.
-    for fk_name, column in _FK_COLUMNS.items():
-        op.execute(f"ALTER TABLE {schema}memory_links DROP CONSTRAINT IF EXISTS {fk_name}")
-        op.execute(
-            f"""
-            ALTER TABLE {schema}memory_links
-                ADD CONSTRAINT {fk_name}
-                FOREIGN KEY ({column})
-                REFERENCES {schema}memory_units (id)
-                ON DELETE CASCADE
-                DEFERRABLE INITIALLY DEFERRED
-            """
-        )
-
-
-def _pg_downgrade() -> None:
-    schema = _get_schema_prefix()
-    # Revert to the default (NOT DEFERRABLE) form so a downgrade actually
-    # restores the prior schema state, even though that re-introduces the
-    # deadlock window.
-    for fk_name, column in _FK_COLUMNS.items():
-        op.execute(f"ALTER TABLE {schema}memory_links DROP CONSTRAINT IF EXISTS {fk_name}")
-        op.execute(
-            f"""
-            ALTER TABLE {schema}memory_links
-                ADD CONSTRAINT {fk_name}
-                FOREIGN KEY ({column})
-                REFERENCES {schema}memory_units (id)
-                ON DELETE CASCADE
-            """
-        )
-
-
-def upgrade() -> None:
-    # PG-only: Oracle's deferrable-FK semantics differ and the deadlock
-    # cycle was only observed on PostgreSQL. Oracle slot intentionally
-    # absent → no-op there.
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/a1b2c3d4e5f6_add_file_storage_table.py

@@ -1,80 +0,0 @@
-"""Add file_storage table for BYTEA-based file storage
-
-Revision ID: a1b2c3d4e5f6
-Revises: y0t1u2v3w4x5
-Create Date: 2026-02-16
-
-Creates a dedicated table for storing uploaded files using BYTEA.
-This provides zero-config file storage that "just works" for development
-and small deployments. For production/scale, use S3-compatible storage.
-
-Files are stored in a separate table to avoid bloating the documents table.
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "a1b2c3d4e5f6"
-down_revision: str | Sequence[str] | None = "y0t1u2v3w4x5"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Get schema prefix for table names (required for multi-tenant support)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _pg_upgrade() -> None:
-    """Create file_storage table for BYTEA storage."""
-    schema = _get_schema_prefix()
-
-    # Create file_storage table (minimal: just key + data)
-    op.execute(
-        f"""
-        CREATE TABLE IF NOT EXISTS {schema}file_storage (
-            storage_key TEXT PRIMARY KEY,
-            data BYTEA NOT NULL
-        )
-    """
-    )
-
-    # Add file tracking columns to documents table
-    op.execute(
-        f"""
-        ALTER TABLE {schema}documents
-        ADD COLUMN IF NOT EXISTS file_storage_key TEXT,
-        ADD COLUMN IF NOT EXISTS file_original_name TEXT,
-        ADD COLUMN IF NOT EXISTS file_content_type TEXT
-    """
-    )
-
-
-def _pg_downgrade() -> None:
-    """Remove file_storage table and related columns."""
-    schema = _get_schema_prefix()
-
-    # Drop columns from documents table
-    op.execute(
-        f"""
-        ALTER TABLE {schema}documents
-        DROP COLUMN IF EXISTS file_storage_key,
-        DROP COLUMN IF EXISTS file_original_name,
-        DROP COLUMN IF EXISTS file_content_type
-    """
-    )
-
-    # Drop file_storage table
-    op.execute(f"DROP TABLE IF EXISTS {schema}file_storage")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)