fix(oracle): rewriter interval/date_trunc support and NULL timestamp binding

1. Oracle rewriter: add interval literal rewrite (interval '7 days' →
   NUMTODSINTERVAL(7, 'DAY')) and fix date_trunc to handle expressions
   like date_trunc('hour', col AT TIME ZONE 'UTC') by stripping AT
   TIME ZONE and matching non-trivial expressions.

2. Fix ORA-00932 in consolidation: NULL datetime params in COALESCE
   with timestamp columns (occurred_start, occurred_end, mentioned_at)
   now get explicit TIMESTAMP_TZ input sizes via setinputsizes.
   Previously Oracle defaulted NULL to VARCHAR2, causing type mismatch.

3. Config validation: downgrade missing local-ml check from ValueError
   to warning. The hard error broke tests that construct HindsightConfig
   without needing local embeddings.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

.claude-plugin/marketplace.json

@@ -1,7 +1,6 @@
 {
   "$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
   "name": "hindsight",
-  "version": "0.7.5",
   "description": "Official Hindsight integrations for Claude Code",
   "owner": {
     "name": "vectorize-io"
@@ -11,11 +10,6 @@
       "name": "hindsight-memory",
       "description": "Automatic long-term memory for Claude Code via Hindsight",
       "source": "./hindsight-integrations/claude-code"
-    },
-    {
-      "name": "hindsight-zcode",
-      "description": "No-MCP long-term memory for ZCode via Hindsight hooks",
-      "source": "./hindsight-integrations/zcode"
     }
   ]
 }

.claude/skills/code-review/SKILL.md

@@ -73,28 +73,6 @@ results = await asyncio.gather(*tasks, return_exceptions=True)
 results = await asyncio.gather(*tasks, return_exceptions=True)
 ```
 
-### API Layer & Data Access
-- **No direct database access in `api/http.py`** (or any API router). HTTP handlers must not build SQL, call `acquire_with_retry` / `conn.fetch` / `conn.fetchrow` / `conn.execute`, or reference `fq_table(...)`. All persistence and queries live in `MemoryEngine` (the engine layer). A handler parses/validates the request, calls an engine method, shapes the HTTP response, and maps domain results to status codes (e.g. a `None` return → 404).
-- **Authentication/tenancy is enforced inside each engine method, not assumed by the handler.** Every engine method that touches bank-scoped data must authenticate via `request_context` — typically `await self._authenticate_tenant(request_context)` (often indirectly through `get_bank_profile(...)`) — so the correct tenant schema is resolved before any query runs. Handlers must thread `request_context` through to the engine method; never query a tenant-scoped table assuming the schema is already set.
-- Engine methods return typed models (Pydantic/dataclass), not raw dicts (see Type Safety).
-- **Every list endpoint paginates, following the existing ones.** A `GET` that returns a collection whose size grows with the data (banks, documents, memories, entities, operations, webhook deliveries, audit logs, …) must take `limit`/`offset` and bound its result — an unbounded list is an unbounded payload plus unbounded per-row work (per-item counts, config resolution, embedding hydration). Copy the shape `list_documents` uses, don't invent a new one: `limit: int = Query(default=100, ge=0)` and `offset: int = Query(default=0, ge=0)` on the handler, matching keyword args on the engine method, and a response carrying the page **plus `total`, `limit`, `offset`** so a client knows when to stop. Add a `q` search param when the collection is something a user picks from in a UI — client-side filtering only ever sees the loaded page. Bounded-by-construction endpoints are the exception, not the rule: a tree/export that is whole-structure by design, or a table capped at write time (e.g. `observation_history` / `mental_model_history`, trimmed to `*_max_entries` on insert). If it isn't bounded, paginate it.
-
-### Bank/Tenant Isolation in Queries
-- **Bank isolation is a hard security invariant: no query may read, count, update, or delete another bank's rows.** Tenant isolation is enforced at the schema level (the resolved `search_path` / `fq_table(...)` qualifier, gated by `_authenticate_tenant`); bank isolation is enforced *within* a schema by a `bank_id` predicate on every statement that touches a multi-bank table.
-- **Every SQL statement against a multi-bank table must be constrained by `bank_id`** — directly in the `WHERE`, or transitively (see below). Multi-bank tables carry a `bank_id` column: `memory_units`, `documents`, `entities`, `entity_links`, `mental_models`, `knowledge_pages`, `memory_links`, `observation_history`, and similar.
-- **The trap: filtering by a caller-supplied, non-globally-unique key without `bank_id`.** Keys like `document_id` and `mental_models.id` are unique only *per bank* (their PK is composite, e.g. `(id, bank_id)`), so the *same* id legally exists in every bank. A statement like `UPDATE memory_units SET tags = $1 WHERE document_id = $2` — no `bank_id` — silently reads/writes **every** bank's rows that share the id. This is the exact defect from #3429/#3430. Adding `AND bank_id = $n` fixes it.
-- **Three ways a statement is legitimately scoped** (accept these; flag anything that fits none):
-  1. **Explicit** `WHERE ... AND bank_id = $n`.
-  2. **Globally-unique single-column PK.** Filtering by a global uuid PK (`memory_units.id`, `entities.id`, `knowledge_pages.id`) or a bank-encoded key (`chunks.chunk_id` is `{bank_id}_{document_id}_{idx}`) cannot collide across banks. Contrast the *composite*-PK ids (`documents.id`/`document_id`, `mental_models.id`) — those are dangerous and MUST carry `bank_id`.
-  3. **Transitive.** Junction tables without a `bank_id` column (`unit_entities`, `entity_cooccurrences`, `observation_sources`) are safe only when reached through globally-unique unit/entity ids that were themselves selected from a bank-scoped query in the same call, and edges are intra-bank by construction. If the id set could contain another bank's ids, it is not scoped.
-- **Watch two smells:** (a) a caller-supplied id used in the `WHERE` with no adjacent `bank_id`, while a *neighbouring* statement in the same method does carry `bank_id` (asymmetry is the tell); (b) a `bank_id` predicate applied only under `if bank_id:` with a `bank_id: str | None = None` default — latent even if all current callers pass one.
-- **Cross-bank by design must rewrite `bank_id` to the destination.** The transfer/import path is the only one that legitimately crosses banks; verify every write pins the *destination* `bank_id` and never inherits a source row's `bank_id`.
-
-### Database Locking
-- **Never use PostgreSQL advisory locks** (`pg_advisory_lock`, `pg_try_advisory_lock`, `pg_advisory_xact_lock`, `pg_advisory_unlock`, …) in migrations, engine code, or anything else. Hindsight runs against connection poolers and managed/PG-compatible services where advisory locks are unreliable or unsupported: session-level locks silently leak or vanish when a pooler hands the session to another client, and callers can block forever on a lock the server never grants. Reject any new occurrence, including ones that look "safe" because they are transaction-scoped.
-- The pre-existing usage in `hindsight_api/migrations.py` is grandfathered, not a precedent — it is tracked for removal. Don't copy it.
-- Design the concurrency out instead of locking around it: give each process its own object to write (e.g. per-schema DDL rather than a shared `public.` object), make the operation idempotent, or use a real row/table constraint (`INSERT ... ON CONFLICT`, `SELECT ... FOR UPDATE` in a fixed order). See #2690 for a migration that reached for `pg_advisory_xact_lock` and had to be reverted.
-
 ### Branch Hygiene
 - **Always start new feature branches from `origin/main`** — rebase to ensure a clean base.
 - **Only include commits relevant to the PR/branch/feature** — no unrelated changes. If the branch contains commits that don't belong, they must be removed before merging.
@@ -157,35 +135,6 @@ For each new or significantly changed function/endpoint/class:
 
 Flag any new logic that lacks test coverage.
 
-**LLM-behaviour changes need a real-LLM judge test, not MockLLM.** If the change alters how the model interprets a prompt — fact/observation extraction, `fact_type` (world/experience) classification, speaker attribution, instruction-following, prompt wording — there MUST be a test marked `pytest.mark.hs_llm_core` that runs the real pipeline and asserts via `tests.llm_judge.assert_meets_criteria` (not string/enum matching). Flag these as findings:
-- A prompt/classification change verified only by MockLLM or string assertions (MockLLM echoes input — such tests pass spuriously). **Should fix.**
-- A test that hard-asserts `fact_type == "world"/"experience"` (or other model-decided output) instead of judging it — non-deterministic, will flake across providers/runs. **Should fix** (move the classification check into the judge `criteria`; keep only genuinely deterministic structural asserts direct).
-- Deterministic mechanics (prompt assembly, suppression/branching logic) that are covered *only* by a slow LLM test — these should also have fast non-LLM unit tests. **Note.**
-
-See CLAUDE.md → Key Conventions → Testing for the full pattern.
-
-### 6a. Check tests assert memory state via the engine API, not raw SQL
-
-Tests must verify what a retain / recall / consolidation produced by calling the public
-`MemoryEngine` read API — `list_memory_units` (units and their `metadata` / `tags`; counts via
-`total`; `document_id` / `fact_type` / `entity_id` filters), `list_entities` (canonical names,
-mention counts), `get_graph_data` (nodes/edges), `get_bank_stats`, `recall_async` — **not** by
-reaching into the memory tables (`memory_units`, `memory_links`, `unit_entities`) with raw SQL via
-`pool.acquire()` / `conn.fetch*`. Asserting on those tables couples the test to a storage-layer
-detail and checks a proxy instead of the observable property (see **General Principles** → tests
-assert the property, and the handler rule in **7b**).
-
-**Flag as should fix** any added or changed test whose assertion runs a `SELECT` / `COUNT` against
-`memory_units` / `memory_links` / `unit_entities` where an engine read method returns the same
-fact. Prime tell: `async with pool.acquire() as conn:` followed by `SELECT ... FROM memory_units`
-inside a test body; a `fetchval("SELECT count(*) FROM memory_units ...")` that `list_memory_units`
-`["total"]` would return; a `canonical_name` query that `list_entities` covers.
-
-Direct SQL on those tables is legitimate **only** when it forces or inspects internal state the
-public API cannot express — e.g. an `UPDATE documents SET updated_at` that forges a race, or a
-raw `memory_links` row-count that the deduped `get_graph_data` edge list cannot reproduce. Those
-must carry a comment saying why the direct access is necessary; flag any that do not.
-
 ### 7. Check API consistency
 
 If any files in `hindsight-api-slim/hindsight_api/api/` were changed:
@@ -193,44 +142,6 @@ If any files in `hindsight-api-slim/hindsight_api/api/` were changed:
 - Were the client SDKs regenerated? (`./scripts/generate-clients.sh`)
 - Were the control plane proxy routes updated? (`hindsight-control-plane/src/app/api/`)
 
-### 7a. Check TS/Python wrapper-client parity
-
-Two of the generated SDKs ship a **hand-written, maintained convenience wrapper** on top of the auto-generated low-level client — and *only* these two:
-- **TypeScript**: `hindsight-clients/typescript/src/index.ts` (`HindsightClient`)
-- **Python**: `hindsight-clients/python/hindsight_client/hindsight_client.py` (`Hindsight`)
-
-(The Rust/Go/etc. clients are generated-only — no wrapper to keep in sync.)
-
-These wrappers are what most third-party consumers actually call, and they must expose the same surface. **If a change touches one wrapper's method — adds/removes a parameter, changes a default, forwards a new query/body field — the equivalent method in the *other* wrapper must get the same change in the same (or an immediately-following) PR.** A parameter that exists in the generated SDK but is dropped by one wrapper silently strips it for every consumer of that language (this is exactly what #2975 / #3042 fixed for `detail`/`tags_match`/`limit`/`offset` on `listMentalModels`/`getMentalModel`). **Should fix** — flag any wrapper method that gains capabilities in one language but not the other, and add a matching mapping regression test on both sides.
-
-Note: the `client-coverage-check` CI tool only validates **request-body** fields, not GET **query** parameters — so query-param parity gaps are *not* caught automatically and must be checked by hand here.
-
-### 7b. Check API-layer data-access boundary
-
-For each changed handler in `hindsight-api-slim/hindsight_api/api/` (e.g. `http.py`, `mcp.py`):
-- **Flag any direct DB access in the handler** — `acquire_with_retry`, `conn.fetch` / `fetchrow` / `execute`, raw SQL strings, or `fq_table(...)`. These are a **must fix**: the query must be moved into a `MemoryEngine` method that returns a typed model, and the handler must call that method.
-- **Verify authentication is enforced in the engine** — the handler must delegate to an engine method that authenticates via `request_context` (`_authenticate_tenant`, typically through `get_bank_profile`). A handler that reads/writes tenant-scoped data without an engine method enforcing auth is a **must fix** (tenant data could leak across schemas).
-
-### 7c. Check bank/tenant query scoping
-
-For **every SQL statement added or changed** in the diff (grep the diff for `conn.fetch`, `conn.fetchrow`, `conn.fetchval`, `conn.execute`, `executemany`, and any raw `SELECT`/`INSERT`/`UPDATE`/`DELETE` f-strings, including multi-line ones), verify it cannot touch another bank's rows — see **Bank/Tenant Isolation in Queries** above.
-
-For each statement against a multi-bank table (`memory_units`, `documents`, `entities`, `entity_links`, `mental_models`, `knowledge_pages`, `memory_links`, `observation_history`, …), confirm it is scoped by one of the three legitimate mechanisms:
-1. explicit `AND bank_id = $n`;
-2. a globally-unique single-column PK (`*.id` uuid, or the bank-encoded `chunks.chunk_id`) — **not** a composite-PK id like `documents.id`/`document_id` or `mental_models.id`;
-3. transitively, through a globally-unique id set that was itself selected from a bank-scoped query in the same call.
-
-**Flag as a must fix** any statement filtering a multi-bank table by a caller-supplied, non-globally-unique key (`document_id`, `mental_models.id`, an entity name, …) with **no** `bank_id` predicate — construct the concrete two-bank scenario (two banks share the id; the statement reads/counts/updates/deletes the wrong bank's rows or over-reports) to confirm it's real before flagging. Prime tells: a `bank_id`-carrying sibling statement right next to a `bank_id`-less one; a `WHERE bank_id` guarded by `if bank_id:` with a `None` default; an import/transfer write that inherits a source `bank_id` instead of pinning the destination.
-
-### 7d. Check list endpoints paginate
-
-For every added or changed `GET` handler that returns a collection, confirm it takes `limit`/`offset` and returns `total` — see **API Layer & Data Access** above for the exact shape. Then check the fix is real end to end, since a param that nothing enforces is worse than none:
-
-- **The bound reaches the work, not just the response.** Verify the page size actually limits the expensive part — the SQL `LIMIT`/`OFFSET`, or (when paging must happen after an in-process filter, as in `list_banks` where the `filter_bank_list` extension hook can drop any bank) an explicit slice with the per-item work — live store counts, `get_bank_configs`, re-embedding — done for the page only. Paging in SQL *before* a filter that can drop rows is a **must fix**: it hands back short or empty pages and a `total` that counts rows the caller can't see.
-- **Every in-repo consumer pages.** A new default `limit` silently truncates callers that used to get everything: the control plane (`src/lib/api.ts` + the `src/app/api/` proxy route + any context/selector that holds the full list), the CLI (`hindsight-cli/src/api.rs`), MCP tools, and the Zapier dynamic dropdowns. Each must either page through to completion or expose paging in its UI — flag any consumer left on a single default-sized page.
-- **Search moves server-side with it.** A picker that filtered client-side over the full list now only filters the loaded page. If the endpoint gained `q`, the UI must send it (and disable its local filtering, e.g. cmdk's `shouldFilter={false}`); if it didn't, say why the collection is small enough not to need it.
-- **Tests that look up their own row must not depend on landing on page 1** — they should search or pass an explicit `limit`, not rely on default ordering.
-
 ### 8. Check code comments
 
 For each non-trivial change:
@@ -243,52 +154,9 @@ For each non-trivial change:
 If any files in `hindsight-integrations/` were added or changed, verify:
 - **Tests exist** — the integration must have tests that simulate/exercise the external framework (not just pure unit tests of helpers). Check for a `tests/` directory with meaningful test files.
 - **CI job exists** — check `.github/workflows/test.yml` for a corresponding `test-<name>-integration` job. If missing, flag it.
-- **Release process** — check that the integration name is in the `VALID_INTEGRATIONS` array in `scripts/release-integration.sh` AND in the `INTEGRATIONS` dict in `hindsight-dev/hindsight_dev/generate_changelog.py` (the changelog generator keeps its own list; a release fails at the changelog step if the name is missing there). If either is missing, flag it.
-- **Docs gallery + sidebar entry** — the integration must have an entry in `hindsight-docs/src/data/integrations.json`. This file is the **single source of truth** that drives both the integrations gallery and the docs sidebar (the sidebar category is injected from it at render time across all docs versions). The entry needs an internal `/sdks/integrations/<slug>` `link` and a matching page at `hindsight-docs/docs-integrations/<slug>.md(x)`. The `hindsight-docs/scripts/check-integrations.mjs` build step enforces both directions — forward: every internal JSON entry has a doc page; reverse: every released tag (`integrations/<name>/vX.Y.Z`) appears in the JSON (private infra like `cloudflare-oauth-proxy` is in the script's `EXCLUDED` set). Flag any integration that is released (or being released) but missing from `integrations.json`, and any JSON entry without a doc page. Do **not** hand-edit `versioned_sidebars/*.json` to add integration links — they are positional placeholders filled from the JSON.
+- **Release process** — check that the integration name is in the `VALID_INTEGRATIONS` array in `scripts/release-integration.sh`. If missing, flag it.
 - **Code standards** — the integration code must follow all Python style rules (type hints, no raw dicts, no tuple returns, etc.).
 
-### 9a. Check parity across sibling implementations
-
-Whenever the same capability is implemented once per *variant* — one per harness, per language, per
-dialect, per provider — the new or changed variant is where a capability silently goes missing. The
-defect never looks like a bug in the diff: the code that's wrong is the code that **isn't there**,
-and every existing test still passes because the sibling that forgot is by definition the one nobody
-wrote a test for. That is how dsh shipped in daemon mode without ever starting a daemon (#3524):
-`ensureDaemon` sat in the hook-only wrappers, so all five persistent-plugin harnesses lacked it.
-
-Known sibling families in this repo (this is not the whole list — the rule is about the *shape*):
-
-| Family | Where |
-|---|---|
-| Coding-agent harnesses | `hindsight-integrations/coding-agents/src/` (hook harnesses vs. persistent-plugin harnesses: dsh, opencode, Kilo, Cline, Prime Agent) |
-| Wrapper SDK clients | TypeScript + Python wrappers — see step 7a |
-| Alembic migrations | `_pg_upgrade` / `_oracle_upgrade` in every migration |
-| Dataplane ↔ control plane | `api/http.py` params vs `hindsight-control-plane/src/app/api/**` proxy routes + `lib/api.ts` |
-| LLM providers | per-provider branches in `engine/llm_wrapper.py` |
-
-**Procedure — do this by hand; no linter catches it.** When the diff adds a new sibling, or changes
-one sibling of a family:
-
-1. **Enumerate the family.** List every existing sibling (`ls` the directory, grep the registry).
-2. **Diff the capability list, not the code.** For each capability the *other* siblings have —
-   lifecycle hooks called, setup/teardown performed, config flags honoured, opt-outs respected,
-   registry/installer/docs entries — confirm the changed sibling has it, or that its absence is
-   deliberate and commented. Grep is the tool: `grep -rn ensureDaemon src` proves who calls it.
-3. **Prefer hoisting over copying.** If the capability now exists in N places, the fix is usually to
-   move it into the one path every sibling already shares (e.g. `RuntimeCore`, `buildHookOutput`),
-   not to paste an Nth copy that the N+1th sibling will forget again.
-4. **Demand a structural guard, not just a unit test.** A test for the sibling that forgot doesn't
-   exist by construction, so ask for a test that asserts *over the whole family*: enumerate the
-   siblings from the filesystem/registry and assert each satisfies the contract, with an explicit,
-   commented exemption list. Precedents: `registry covers every installable harness`
-   (`harness/registry.test.ts`), `every harness entrypoint reaches a daemon` (`core/daemon.test.ts`),
-   `test_backup_tables_covers_entire_schema`, `test_migration_shape.py`.
-
-Flag a capability present in every sibling but one as a **must fix** — state which siblings have it,
-which doesn't, and what the user-visible symptom is (for #3524: every `hindsight_*` tool call fails
-with ECONNREFUSED and nothing ever starts the daemon). A new sibling family member landing with no
-family-wide guard test is a **should fix**.
-
 ### 10. Check MCP tool registration completeness
 
 If any new MCP tools were added or existing tools renamed in `hindsight-api-slim/hindsight_api/mcp_tools.py`:
@@ -298,34 +166,7 @@ If any new MCP tools were added or existing tools renamed in `hindsight-api-slim
 - **`MCP_TOOL_GROUPS`** in `hindsight-control-plane/src/components/bank-config-view.tsx` — must include the new tool in the appropriate group for the UI tool selector
 - **Tool count assertions** in tests (e.g., `test_mcp_tools.py`) — must be updated to reflect the new count
 
-### 11. Check backup/restore table coverage
-
-If a migration adds a new PostgreSQL table (look for `CREATE TABLE` / `op.create_table` in `hindsight-api-slim/hindsight_api/alembic/versions/`):
-- **`BACKUP_TABLES`** in `hindsight-api-slim/hindsight_api/admin/cli.py` — must include the new table, placed after any table it references via foreign key (parents before children). A missing entry is silent data loss: the table is never backed up, and restore's `TRUNCATE banks CASCADE` wipes any FK-to-banks child (e.g. `mental_models`, `directives`) on restore even though it was never saved.
-- The guard test `test_backup_tables_covers_entire_schema` in `tests/test_admin_backup_restore.py` enforces this — flag it as a **must fix** if a new table is absent from `BACKUP_TABLES`.
-- Oracle-only tables (e.g. `observation_sources`) are intentionally excluded — admin backup/restore is PostgreSQL-only.
-
-### 11b. Check new config flags update the env template
-
-If the diff adds a new configuration field (a new `ENV_*` / `HINDSIGHT_*` env var
-in `hindsight-api-slim/hindsight_api/config.py`):
-- **`.env.example`** (repo root) — must add the variable (commented if optional)
-  alongside the docs entry in `hindsight-docs/docs/developer/configuration.md`.
-  A flag added to `config.py` but absent from `.env.example` is a **should fix**.
-- **`hindsight-embed/hindsight_embed/env.example`** — the bundled copy must stay
-  byte-identical to the repo-root `.env.example` (it seeds embed/profile configs).
-  The `test_bundled_template_matches_repo_root` sync test fails on drift; if the
-  root file changed without re-copying, flag it as a **must fix**.
-
-### 11c. Check for advisory locks
-
-Grep the diff for `advisory` (`git diff main...HEAD | grep -in advisory`). Any new
-`pg_advisory_lock` / `pg_try_advisory_lock` / `pg_advisory_xact_lock` /
-`pg_advisory_unlock` call is a **must fix** — see Database Locking above. Point the
-author at the alternatives (per-process objects, idempotent DDL, row-level
-constraints) rather than just asking them to drop the lock.
-
-### 12. Review against other coding standards
+### 11. Review against other coding standards
 
 Check the diff for violations of the standards listed above:
 - Python files at project root (not allowed)
@@ -337,7 +178,7 @@ Check the diff for violations of the standards listed above:
 - Premature abstractions or speculative helpers
 - Backwards-compatibility hacks (unused vars, re-exports, "removed" comments)
 
-### 13. Report findings
+### 12. Report findings
 
 Present a clear summary organized by severity:
 
@@ -348,14 +189,7 @@ Present a clear summary organized by severity:
 - Raw dict usage for structured data (including internal code)
 - Multi-item tuple returns (including internal code)
 - Missing tests for new endpoints
-- Direct DB access (raw SQL / `acquire_with_retry` / `fq_table`) in an `api/` handler instead of a `MemoryEngine` method
-- Tenant-scoped data accessed without authentication enforced in the engine (`_authenticate_tenant` / `get_bank_profile`)
-- A SQL statement against a multi-bank table filtered by a caller-supplied, non-globally-unique key without a `bank_id` predicate (cross-bank read/write leak — see step 7c)
 - New integration missing tests, CI job, or release-integration.sh entry
-- Released/added integration missing from `hindsight-docs/src/data/integrations.json`, or a JSON entry with no `docs-integrations/<slug>` page (fails the docs build via `check-integrations.mjs`)
-- New PostgreSQL table missing from `BACKUP_TABLES` in `admin/cli.py` (silent data loss on restore)
-- A capability every sibling implementation has except the one in the diff (see step 9a) — a
-  harness, dialect, provider or language variant that skips a lifecycle step the others perform
 
 **Should fix** — issues that hurt code quality:
 - Dead code / unused imports missed by linter

.claude/skills/hs-release/SKILL.md

@@ -1,116 +0,0 @@
----
-name: hs-release
-description: Cut a core Hindsight release (vX.Y.Z) and open the changelog + blog PR. Use when asked to cut/start a release, bump the version, or publish a new Hindsight version.
-user_invocable: true
----
-
-# Hindsight Release
-
-Cut a **core** Hindsight release and open the accompanying changelog/blog PR. This is for the core
-product version (API, clients, CLI, control plane, Helm). **Integrations are versioned
-independently** — use `scripts/release-integration.sh` for those, not this skill.
-
-The release is **irreversible and outward-facing**: it tags a version and pushes it straight to
-`main`, which triggers CI that publishes packages to PyPI / npm / Helm. Confirm the version number
-and that the intended fixes are already merged to `main` before you start.
-
-## Step 0 — Pre-flight
-
-1. **Decide the base.** A release is cut from the latest `origin/main`, never from a feature
-   branch. `git fetch origin --tags` first. Confirm the "couple of fixes" the user means are
-   actually merged to `main` (`git log v<prev>..origin/main --oneline`).
-2. **Find where `main` is checked out.** `main` is often already checked out in a sibling worktree
-   (`git worktree list`). You **cannot** check out `main` in a second worktree — run the release in
-   the worktree that already holds it. If that worktree is dirty with throwaway cruft
-   (`.next-*` tsconfig paths, screenshots), `git stash push -u`, fast-forward to `origin/main`,
-   run the release, then `git stash pop`.
-3. **Pitfall:** never pipe the checkout in an `&&` chain like
-   `git checkout main 2>&1 | tail && git reset --hard ...` — the pipe's exit status is `tail`'s
-   (always 0), so a failed checkout won't stop the chain and the `reset` fires on the **wrong
-   branch**. Check out as its own command and verify `git branch --show-current` before resetting.
-
-## Step 1 — Cut the release
-
-Run from the worktree on a clean `main`:
-
-```bash
-./scripts/release.sh <version>     # e.g. 0.8.1  (no leading v)
-```
-
-`release.sh` bumps the version in every component, regenerates the OpenAPI spec + all client SDKs,
-updates docs versioning, commits `Release v<version>`, tags `v<version>`, and **pushes the commit
-and tag directly to `main`**. The push triggers the `Release` GitHub Actions workflow that builds
-and publishes the packages. It is **not** a PR.
-
-Verify after: `gh run list --limit 5` should show the `Release v<version>` workflow running, and
-`git ls-remote --tags origin v<version>` should return the tag.
-
-## Step 2 — Changelog + blog PR (separate)
-
-Done **after** the tag exists, as its own PR (precedent: v0.8.0 = #2053, v0.8.1 = #2080). Work on a
-branch off the new `main`:
-
-```bash
-git checkout -b docs-changelog-<version> origin/main
-```
-
-Only spin up a separate worktree (`git worktree add ../hindsight-changelog-<version> -b
-docs-changelog-<version> origin/main`) if you can't get a clean checkout otherwise — e.g. `main` is
-held in another worktree and the current one has work you don't want to disturb.
-
-**Branch naming:** use the `docs-` (hyphen) convention, e.g. `docs-changelog-0.8.1`. A remote
-branch literally named `docs` exists, so any `docs/...` branch is rejected on push with
-`directory file conflict`.
-
-### Changelog
-
-```bash
-uv run --directory hindsight-dev generate-changelog <version>
-```
-
-LLM-summarizes the commits between the previous tag and `v<version>` and prepends an entry to
-`hindsight-docs/src/pages/changelog/index.md`. Requires `OPENAI_API_KEY` (already in the repo
-`.env`). It excludes `hindsight-integrations/` source, but new integrations whose commits also
-touched docs will still appear — that matches precedent, leave them in the **changelog**.
-
-### Blog post
-
-Hand-write `hindsight-docs/blog/YYYY-MM-DD-version-X-Y-Z.md` (mirror an existing one; patch
-releases are short — see `2026-06-02-version-0-7-2.md`). Guidance:
-
-- **Explain user impact, not internals/mechanism.** Lead with what the user can now do and what to
-  set. Config/env-var names are fine (developer-facing), code symbols and internals are not.
-- **Do not list integrations in the release blog.** The core blog covers core engine / API /
-  ops changes; each integration ships its own changelog. (Integrations may still appear in the
-  generated `changelog/index.md` — that's fine; just keep them out of the blog.)
-- Call out an upgrade recommendation when there are operational/data-integrity fixes.
-- Validate formatting: `npx prettier --check <blog file>`.
-
-### Sync the docs skill
-
-```bash
-./scripts/generate-docs-skill.sh
-```
-
-Refreshes `skills/hindsight-docs/references/changelog/index.md`. It will also bump
-`skills/hindsight-docs/references/openapi.json` by one version — `release.sh` regenerates the skill
-*before* bumping OpenAPI, so the skill copy lags a version in the release commit; this step syncs
-it. Expect a one-line `version` diff there; keep it.
-
-### Commit, push, PR
-
-```bash
-git add -A
-git commit --no-verify -m "docs: changelog and blog post for v<version>"
-git push -u origin docs-changelog-<version>
-gh pr create --base main --title "docs: changelog and blog post for v<version>" --body "..."
-```
-
-Expected files in the PR: the changelog entry, the new blog post, the regenerated skill changelog
-mirror, and the skill `openapi.json` version sync.
-
-## Cleanup
-
-If you created a temporary worktree, remove it once the PR is up
-(`git worktree remove ../hindsight-changelog-<version>`; the branch stays on origin). Restore any
-stash you popped in Step 0.

.env.example

@@ -2,72 +2,11 @@
 # Copy this file to .env and fill in your values
 
 # LLM Configuration (Required)
-# Supported providers: openai, openai-responses, groq, ollama, gemini, anthropic, lmstudio, vertexai, minimax, deepseek, zai, atlas, volcano
+# Supported providers: openai, groq, ollama, gemini, anthropic, lmstudio, vertexai, minimax, volcano
 HINDSIGHT_API_LLM_PROVIDER=openai
 HINDSIGHT_API_LLM_API_KEY=your-api-key-here
 HINDSIGHT_API_LLM_MODEL=gpt-4o-mini
 HINDSIGHT_API_LLM_BASE_URL=https://api.openai.com/v1
-# Reasoning effort for providers/models that support it. Examples: none, low, medium, high, xhigh.
-# Set it and the value is sent as given, whatever the model is called — use `none` to stop a
-# self-hosted reasoning model (vLLM, Ollama, llama.cpp, TGI) emitting thinking blocks. Unset,
-# no reasoning parameter is sent at all and each model runs at its own default effort.
-# HINDSIGHT_API_LLM_REASONING_EFFORT=low
-
-# Sampling temperature for internal LLM calls. Set a number in [0.0, 2.0], or `none`
-# to omit the temperature parameter entirely (required for models that reject explicit
-# temperatures, e.g. Azure gpt-5.5). The global override below applies to every operation;
-# per-operation overrides (defaults: verification=0.0, retain=0.1, reflect=0.9,
-# consolidation=0.0) take precedence.
-# HINDSIGHT_API_LLM_TEMPERATURE=none
-# HINDSIGHT_API_LLM_TEMPERATURE_VERIFICATION=0.0
-# HINDSIGHT_API_LLM_TEMPERATURE_RETAIN=0.1
-# HINDSIGHT_API_LLM_TEMPERATURE_REFLECT=0.9
-# HINDSIGHT_API_LLM_TEMPERATURE_CONSOLIDATION=0.0
-
-# Grammar-enforce structured output (json_schema strict) instead of the soft
-# schema-in-prompt path. Helps weaker self-hosted models that emit prose preambles
-# or invalid JSON. The global override below applies to every operation;
-# per-operation overrides take precedence, in both directions -- set one to false
-# to opt that operation out while the global flag is on.
-# HINDSIGHT_API_LLM_STRICT_SCHEMA=false
-# HINDSIGHT_API_LLM_STRICT_SCHEMA_RETAIN=true
-# HINDSIGHT_API_LLM_STRICT_SCHEMA_REFLECT=true
-# HINDSIGHT_API_LLM_STRICT_SCHEMA_CONSOLIDATION=true
-
-# Some backends, including Bedrock Converse, reject JSON Schema maxItems.
-# Disable it only for those backends; consolidation still enforces the cap.
-# HINDSIGHT_API_LLM_SUPPORTS_MAX_ITEMS=true
-
-# Pin a conversation to one backend prompt cache (OpenAI-compatible providers only).
-# Server-side prompt caches are per backend server, so the same conversation has to
-# reach the same one to hit. Values: auto (default), xai_conv_id (sends xAI's
-# x-grok-conv-id header), openai_prompt_cache_key (sends OpenAI's prompt_cache_key
-# field), none (sends nothing). "auto" picks from the base URL host and is an
-# allowlist: x.ai / grok.com get the header, native OpenAI / openai.com / Azure
-# OpenAI get the field, and every other backend gets nothing. Per-operation
-# overrides take precedence. Set to none to disable entirely.
-# HINDSIGHT_API_LLM_CACHE_AFFINITY=auto
-# HINDSIGHT_API_RETAIN_LLM_CACHE_AFFINITY=none
-# HINDSIGHT_API_REFLECT_LLM_CACHE_AFFINITY=xai_conv_id
-# HINDSIGHT_API_CONSOLIDATION_LLM_CACHE_AFFINITY=none
-
-# Ask litellm/litellmrouter/bedrock for structured output via a forced tool call
-# instead of response_format. Enable it for backends that reject response_format
-# outright -- e.g. Bedrock Claude in ap-southeast-2 ("Extra inputs are not permitted");
-# the same model in us-east-1 accepts response_format and needs nothing here.
-# HINDSIGHT_API_LLM_STRUCTURED_OUTPUT_FORCED_TOOL=false
-
-# Transport-level output cap for reflect's final synthesis call. Unset = uncapped:
-# the model runs to a natural stop and the reflect/mental-model max_tokens governs
-# visible length via a prompt directive + a post-hoc rewrite (not by truncating the
-# provider call, which on thinking models is eaten by reasoning tokens and cuts pages
-# off mid-word). Set an integer only to enforce a hard cost ceiling on the call.
-# HINDSIGHT_API_REFLECT_MAX_COMPLETION_TOKENS=16000
-
-# Diagnostic: on any LLM 4xx, log the exact assembled request ([LLM_4XX_DUMP]) --
-# serialized request config (message bodies stripped) + capped per-message previews.
-# For debugging otherwise-unreproducible rejected calls. Off by default.
-# HINDSIGHT_API_LLM_DEBUG_DUMP_4XX=false
 
 # Example: Anthropic Claude configuration
 # HINDSIGHT_API_LLM_PROVIDER=anthropic
@@ -84,28 +23,7 @@ HINDSIGHT_API_LLM_BASE_URL=https://api.openai.com/v1
 # Example: MiniMax configuration (1M context window)
 # HINDSIGHT_API_LLM_PROVIDER=minimax
 # HINDSIGHT_API_LLM_API_KEY=your-minimax-api-key
-# HINDSIGHT_API_LLM_MODEL=MiniMax-M3  # or MiniMax-M2.7 for the previous generation
-
-# Example: OpenAI Responses API (/v1/responses) — reasoning + function tools together
-# HINDSIGHT_API_LLM_PROVIDER=openai-responses
-# HINDSIGHT_API_LLM_API_KEY=your-openai-api-key
-# HINDSIGHT_API_LLM_MODEL=gpt-5.6  # reasoning model (gpt-5.x / o-series); e.g. gpt-5.6-terra
-# HINDSIGHT_API_LLM_REASONING_EFFORT=high  # sent alongside tools, unlike chat/completions
-
-# Example: DeepSeek configuration (https://api.deepseek.com)
-# HINDSIGHT_API_LLM_PROVIDER=deepseek
-# HINDSIGHT_API_LLM_API_KEY=your-deepseek-api-key
-# HINDSIGHT_API_LLM_MODEL=deepseek-v4-flash  # or deepseek-v4-pro / deepseek-chat / deepseek-reasoner
-
-# Example: z.ai configuration (Zhipu GLM series, https://z.ai)
-# HINDSIGHT_API_LLM_PROVIDER=zai
-# HINDSIGHT_API_LLM_API_KEY=your-zai-api-key
-# HINDSIGHT_API_LLM_MODEL=glm-4.5-flash  # or glm-4.5-air for the paid tier
-
-# Example: Atlas Cloud configuration (OpenAI-compatible, https://www.atlascloud.ai)
-# HINDSIGHT_API_LLM_PROVIDER=atlas
-# HINDSIGHT_API_LLM_API_KEY=your-atlascloud-api-key
-# HINDSIGHT_API_LLM_MODEL=deepseek-ai/deepseek-v4-pro  # reasoning model; also Qwen / GLM / Kimi / MiniMax, etc.
+# HINDSIGHT_API_LLM_MODEL=MiniMax-M2.7
 
 # Example: LM Studio local configuration (Qwen 2.5 32B recommended)
 # HINDSIGHT_API_LLM_PROVIDER=lmstudio
@@ -113,49 +31,10 @@ HINDSIGHT_API_LLM_BASE_URL=https://api.openai.com/v1
 # HINDSIGHT_API_LLM_BASE_URL=http://localhost:1234/v1
 # HINDSIGHT_API_LLM_MODEL=qwen2.5-32b-instruct
 
-# Example: Ollama local configuration (native provider)
-# HINDSIGHT_API_LLM_PROVIDER=ollama
-# HINDSIGHT_API_LLM_BASE_URL=http://localhost:11434/v1
-# HINDSIGHT_API_LLM_MODEL=gemma3:12b
-# Native Ollama context-window override (num_ctx). Leave unset to let Ollama use
-# the model Modelfile / server default; set a positive integer only to force a
-# specific context size (e.g. 16384 to keep the previous request behavior).
-# HINDSIGHT_API_LLM_OLLAMA_NUM_CTX=16384
-
-# Multi-LLM strategies: configure extra LLMs by index alongside the primary above,
-# then pick a routing strategy. Unset = single primary LLM (default). Members are
-# numbered from 1; indices must be contiguous. Each operation can override with a
-# RETAIN_/REFLECT_/CONSOLIDATION_ prefix (e.g. HINDSIGHT_API_RETAIN_LLM_1_PROVIDER).
-# HINDSIGHT_API_LLM_1_PROVIDER=groq
-# HINDSIGHT_API_LLM_1_API_KEY=your-groq-api-key
-# HINDSIGHT_API_LLM_1_MODEL=openai/gpt-oss-120b
-# HINDSIGHT_API_LLM_2_PROVIDER=anthropic
-# HINDSIGHT_API_LLM_2_API_KEY=your-anthropic-api-key
-# Strategy JSON: {"mode": "failover"} or {"mode": "round-robin"}.
-# Round-robin accepts optional positive-int "weights" (one per member, primary first).
-# HINDSIGHT_API_LLM_STRATEGY={"mode": "failover"}
-
 # API Configuration (Optional)
 HINDSIGHT_API_HOST=0.0.0.0
 HINDSIGHT_API_PORT=8888
 HINDSIGHT_API_LOG_LEVEL=info
-# Optional retain chunking override for structured logs/transcripts.
-# Unset uses HINDSIGHT_API_RETAIN_CHUNK_SIZE as the structured-chunk limit.
-# HINDSIGHT_API_RETAIN_STRUCTURED_CHUNK_SIZE=
-
-# When true, a retain operation that hit any fact-extraction errors is marked
-# 'failed' (not 'completed'), surfacing silently-dropped facts. Default false.
-# HINDSIGHT_API_FAIL_ON_EXTRACTION_ERRORS=false
-
-# Wall-clock ceiling (seconds) for one retain task in the worker. A retain that
-# blocks indefinitely is cancelled and marked 'failed' — and so becomes
-# retryable — instead of holding its worker slot until the process restarts.
-# Set well above your slowest healthy retain; 0 disables. Default 3600.
-# HINDSIGHT_API_RETAIN_WALL_TIMEOUT=3600
-
-# Dry-run extraction preview endpoint (POST /memories/dry-run-extract). Enabled by default; it makes
-# a real LLM call but stores nothing. Set to false to remove the endpoint (returns 404).
-# HINDSIGHT_API_ENABLE_DRY_RUN_EXTRACT=true
 
 # Base Path / Reverse Proxy Support (Optional)
 # Set these when deploying behind a reverse proxy with path-based routing
@@ -165,198 +44,30 @@ HINDSIGHT_API_LOG_LEVEL=info
 
 # Database (Optional - uses embedded pg0 by default)
 # HINDSIGHT_API_DATABASE_URL=postgresql://user:pass@host:5432/db
-# HINDSIGHT_API_READ_DATABASE_URL=  # Optional read-replica URL. When set, recall queries (semantic, BM25, graph, temporal) flow through a separate pool against this URL, offloading the primary. Typically points to a read-only endpoint (CNPG's <cluster>-ro service or Aurora reader endpoint).
 # HINDSIGHT_API_MIGRATION_DATABASE_URL=  # Direct PostgreSQL URL for migrations (bypasses PgBouncer). Falls back to DATABASE_URL.
 # HINDSIGHT_API_DATABASE_SCHEMA=public  # PostgreSQL schema name (default: public)
-# HINDSIGHT_API_DB_MAX_PARALLEL_WORKERS_PER_GATHER=  # Optional cap on Postgres planner parallelism for this process's pool connections. Unset leaves the server default; 0 makes background/bulk queries run serially (useful on worker processes sharing a primary with latency-sensitive traffic).
-# HINDSIGHT_API_DB_SESSION_SETUP_ON_ACQUIRE=true  # Re-apply the per-connection session settings (statement_timeout, planner parallelism, trigram threshold, vector-search tuning, and the vchord search path) every time a connection is taken from the pool, not just when it is opened. Releasing a connection resets it to the server defaults, so turn this off only when the same settings are pinned on the role/database (ALTER ROLE ... SET) — then it is a pure round trip per acquire, worth reclaiming behind a transaction-mode pooler. On the vchord text-search backend the search path is in that set and losing it fails recall outright, so pin it too. application_name is always re-applied regardless.
-# HINDSIGHT_API_ENTITY_TRGM_SIMILARITY_THRESHOLD=0.15  # Postgres pg_trgm.similarity_threshold applied on every pool connection, used by entity resolution's % trigram match. Must be in (0, 1]. Lower catches more substring-ish matches at higher CPU cost on large entity sets; higher is stricter and cheaper.
-# HINDSIGHT_API_ENTITY_INTRABATCH_MERGE_SIMILARITY=0.5  # Trigram similarity (pg_trgm-equivalent, computed in-memory) at/above which two new names created by the SAME retain are merged into one entity (in-batch dedup of surface-form variants). Must be in (0, 1]. A merge cutoff, stricter than the recall threshold above; raise toward 1.0 to merge only near-identical forms.
-# HINDSIGHT_API_RETAIN_ENTITY_RESOLUTION_MAX_CANDIDATES=200  # Max candidates scored per entity mention during retain. The fuzzy lookup keeps only this many best matches per name (ranked by trigram/Jaro-Winkler similarity) before scoring them one by one. On banks holding thousands of near-identical names an uncapped set turns one retain into minutes of CPU that stall the worker's health checks. Raise only if entities that should merge are being duplicated.
-# HINDSIGHT_API_MIGRATION_CONCURRENCY=1  # Tenant schemas to migrate concurrently (PG only, each in its own process; per-schema work stays sequential). Each worker has ~1-2s startup cost + uses ~3 DB connections, so it only pays off with many schemas (tens+) or slow migrations; keep concurrency*3 <= spare max_connections. Default: 1 (sequential).
-# HINDSIGHT_API_OPERATION_RETENTION_DAYS=30  # Prune terminal operation rows, payloads, and metadata after this many days; 0 (the default) keeps them forever.
-# HINDSIGHT_API_OPERATION_CLEANUP_BATCH_SIZE=1000  # Maximum expired terminal rows deleted per tenant schema in each cleanup cycle; must be positive.
-
-# Background maintenance cadences (Optional)
-# Each sweep begins with one cross-tenant discovery call that probes every schema holding the relevant
-# table, in every API/worker process — so its cost scales with tenant count while the work it finds does
-# not. On deployments with thousands of tenants these intervals are the knob to raise.
-# HINDSIGHT_API_RETENTION_SWEEP_INTERVAL_SECONDS=3600  # How often expired audit_log / llm_requests rows are deleted across all tenant schemas. Retention is counted in days, so this only sets how promptly they disappear; 0 disables the sweeps.
-# HINDSIGHT_API_OPERATION_CLEANUP_INTERVAL_SECONDS=900  # How often expired terminal operation rows are pruned; with the batch size above this sets the drain rate for a backlog. 0 disables the job.
-# HINDSIGHT_API_MAINTENANCE_START_JITTER_SECONDS=60  # Upper bound on a random delay before a process runs its FIRST maintenance tick. Every job is due on that tick, so without an offset a fleet started together runs every sweep in every process at once. 0 disables the jitter.
 
 # Vector Extension (Optional - uses pgvector by default)
 # Options: "pgvector" (default), "vchord", "pgvectorscale" (DiskANN)
 # HINDSIGHT_API_VECTOR_EXTENSION=pgvector
-
-# Let a vector index scan resume until the query's LIMIT is satisfied, instead of
-# stopping when its first candidate list drains (pgvector: hnsw.ef_search, 200) — with
-# it off, a larger recall budget cannot retrieve more rows. Needs pgvector 0.8.0+;
-# older servers reject it and it is dropped automatically. Set false and restart as a
-# quick revert to the previous retrieval depth, with no code change.
-# HINDSIGHT_API_ANN_ITERATIVE_SCAN=true
-# Ceiling on tuples one resumed scan may visit. Bounds the CPU and memory a selective
-# query can spend resuming (filters are applied after the scan, so it resumes often).
-# Lower it to trade depth back for latency. pgvector's own default is 20000.
-# HINDSIGHT_API_ANN_MAX_SCAN_TUPLES=4000
 # For Azure PostgreSQL with DiskANN:
 # HINDSIGHT_API_VECTOR_EXTENSION=pgvectorscale  # Auto-detects pg_diskann on Azure
 
-# Per-bank vector indexes (pgvector / pgvectorscale / vchord only; ScaNN and Oracle use one global index)
-# HINDSIGHT_API_VECTOR_INDEX_MIN_ROWS=0  # Memories a bank needs in one fact type before that fact type gets its own vector index. 0 (default) = no minimum, every bank holding memories is indexed. Set ~10000 on deployments with thousands of banks: every index lives on the shared memory_units table and is planned against by every OTHER bank's queries, so unconditional per-bank indexes put a ceiling on bank count. Smaller banks then use exact search, which is faster AND exact.
-
-# Text Search Extension (Optional - uses native PostgreSQL full-text search by default)
-# Backend options: "native" (default), "vchord", "pg_textsearch", "pgroonga", "pg_search"
-# HINDSIGHT_API_TEXT_SEARCH_EXTENSION=native
-# Native backend dictionary (only used by HINDSIGHT_API_TEXT_SEARCH_EXTENSION=native)
-# HINDSIGHT_API_TEXT_SEARCH_EXTENSION_NATIVE_LANGUAGE=english
-# ParadeDB pg_search tokenizer (only used when creating pg_search BM25 indexes).
-# Empty uses ParadeDB's default tokenizer: unicode_words.
-# Supported values: unicode_words, simple, whitespace, literal, literal_normalized,
-# chinese_compatible, icu, jieba, source_code,
-# chinese_lindera/lindera(chinese), japanese_lindera/lindera(japanese),
-# korean_lindera/lindera(korean), ngram(min,max), edge_ngram(min,max)
-# HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER=
-# Cap on the number of terms in the native PostgreSQL BM25 tsquery. Long queries
-# OR-join every normalized token, and native ranking (no IDF, re-ranks every
-# match) can then scan a large fraction of the bank and time out. Over the cap,
-# the most selective terms are kept — lowest tenant-wide document frequency, read
-# for free from pg_stats (no reindex). 0 restores the uncapped behavior; the cap
-# bounds only the native backend (other BM25 backends get the raw query).
-# HINDSIGHT_API_BM25_MAX_QUERY_TERMS=16
-# When the cap above trims a query, keep the most selective terms (lowest
-# document frequency, from pg_stats) instead of the first N. true is strictly
-# better for recall at no extra cost when stats exist; set false to opt out of
-# the catalog read and cap by position. Ignored when the cap is 0.
-# HINDSIGHT_API_BM25_SELECTIVE_TERMS=true
-
-# File Parser (Optional - uses markitdown by default)
-# HINDSIGHT_API_FILE_PARSER=markitdown
-# Enable image OCR for MarkItDown using an OpenAI-compatible OCR/vision endpoint.
-# These OCR settings are independent from HINDSIGHT_API_LLM_* because MarkItDown
-# uses the OpenAI SDK directly and requires Chat Completions image input support.
-# When OCR is enabled, API_KEY, BASE_URL, and MODEL are required.
-# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_ENABLED=false
-# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_API_KEY=
-# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_BASE_URL=
-# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_MODEL=
-# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_PROMPT=
-# Optional JSON dict of custom headers for the OCR OpenAI client (e.g. proxies / request tracing).
-# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_DEFAULT_HEADERS=
-
 # Embeddings Configuration (Optional - uses local by default)
-# Provider: "local" (default), "onnx", "tei", "openai", "cohere", "google", "openrouter", "zeroentropy", "litellm", or "litellm-sdk"
+# Provider: "local" (default) or "tei" (HuggingFace Text Embeddings Inference)
 # HINDSIGHT_API_EMBEDDINGS_PROVIDER=local
 # For local provider:
 # HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL=BAAI/bge-small-en-v1.5
-# Force CPU if local embeddings hit MPS/XPC instability on macOS:
-# HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU=false
-# Opt in to the Apple Silicon MPS GPU (off by default: MPS leaks memory under
-# variable-length workloads). CUDA/XPU still auto-select regardless:
-# HINDSIGHT_API_EMBEDDINGS_LOCAL_ALLOW_MPS=false
-# For ONNX provider (local CPU embeddings without an Ollama/TEI sidecar):
-# HINDSIGHT_API_EMBEDDINGS_PROVIDER=onnx
-# HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_ID=intfloat/multilingual-e5-small
-# HINDSIGHT_API_EMBEDDINGS_ONNX_FILE=onnx/model.onnx
-# HINDSIGHT_API_EMBEDDINGS_ONNX_DIMENSIONS=384
-# HINDSIGHT_API_EMBEDDINGS_ONNX_MAX_TOKENS=512
-# HINDSIGHT_API_EMBEDDINGS_ONNX_POOLING=mean
-# HINDSIGHT_API_EMBEDDINGS_ONNX_NORMALIZE=true
-# HINDSIGHT_API_EMBEDDINGS_ONNX_QUERY_PREFIX="query: "
-# HINDSIGHT_API_EMBEDDINGS_ONNX_PASSAGE_PREFIX="passage: "
-# Optional for local model paths or pre-downloaded artifacts:
-# HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_PATH=/models/multilingual-e5-small/onnx/model.onnx
-# HINDSIGHT_API_EMBEDDINGS_ONNX_TOKENIZER_NAME_OR_PATH=/models/multilingual-e5-small
-# Optional for China network / restricted HF access:
-# HF_ENDPOINT=https://hf-mirror.com
-# Applies to any provider: cap each input at this many tiktoken tokens before
-# embedding, so oversized content is truncated instead of failing the embed call
-# permanently (e.g. Bedrock Titan V2's 8192, or a llama.cpp server's context). Off
-# by default. (Deprecated alias: HINDSIGHT_API_EMBEDDINGS_LITELLM_SDK_MAX_INPUT_TOKENS)
-# HINDSIGHT_API_EMBEDDINGS_MAX_INPUT_TOKENS=8192
-# Asymmetric models (E5, google/embeddinggemma-300m, ...) expect a different instruction
-# in front of a search than in front of stored text. Providers that only accept plain text
-# (tei, openai-compatible, litellm) need it applied client-side; local/zeroentropy handle it
-# themselves and ignore these. Unset = text sent as-is.
-# HINDSIGHT_API_EMBEDDINGS_QUERY_PREFIX="task: search result | query: "
-# HINDSIGHT_API_EMBEDDINGS_PASSAGE_PREFIX="title: none | text: "
 # For TEI provider:
 # HINDSIGHT_API_EMBEDDINGS_TEI_URL=http://localhost:8080
-# For OpenAI-compatible embeddings:
-# HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=sk-xxxx
-# HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL=text-embedding-3-small
-# HINDSIGHT_API_EMBEDDINGS_OPENAI_BASE_URL=https://api.openai.com/v1
-# For ZeroEntropy zembed-1:
-# HINDSIGHT_API_EMBEDDINGS_PROVIDER=zeroentropy
-# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_API_KEY=ze-xxxx
-# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_MODEL=zembed-1
-# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_DIMENSIONS=1280
-# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_ENCODING_FORMAT=float
-# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_LATENCY=fast
-#
-# IMPORTANT: Embedding keys require provider-specific names:
-# HINDSIGHT_API_EMBEDDINGS_{PROVIDER}_{PARAMETER}
-# (for example, HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL).
-#
-# DeepSeek note: DeepSeek is supported for LLM calls, but not for embeddings.
-# If using DeepSeek as LLM provider, keep embeddings on local/openai/cohere/google/etc.
-
-# Embedding similarity thresholds. These defaults preserve the behavior calibrated
-# for BAAI/bge-small-en-v1.5. Recalibrate each threshold independently when changing
-# embedding models because cosine-similarity distributions are model-dependent.
-# HINDSIGHT_API_SEMANTIC_MIN_SIMILARITY=0.3
-# HINDSIGHT_API_GRAPH_SEED_MIN_SIMILARITY=0.3
-# HINDSIGHT_API_TEMPORAL_SEMANTIC_MIN_SIMILARITY=0.1
-# HINDSIGHT_API_SEMANTIC_LINK_MIN_SIMILARITY=0.7
-# HINDSIGHT_API_CONSOLIDATION_DEDUP_THRESHOLD=0.97
-
-# Recall pipeline stages (all on by default). Each is hierarchical, so a single
-# bank can switch a stage off via the config API without changing the server
-# default. Turning all three off leaves semantic + BM25 fused by RRF, the
-# lowest-latency recall path.
-# Temporal retrieval arm, plus the date-aware query analysis that feeds it:
-# HINDSIGHT_API_ENABLE_TEMPORAL_RETRIEVAL=true
-# Entity/link graph traversal arm:
-# HINDSIGHT_API_ENABLE_GRAPH_RETRIEVAL=true
-# Cross-encoder rerank of the fused candidates (false = use the RRF order):
-# HINDSIGHT_API_ENABLE_RERANKING=true
 
 # Reranker Configuration (Optional - uses local by default)
 # Provider: "local" (default) or "tei" (HuggingFace Text Embeddings Inference)
 # HINDSIGHT_API_RERANKER_PROVIDER=local
-# Trusted gateway attribution (disabled by default). When enabled, remote
-# reranker requests include X-Hindsight-Bank-Id with the current bank ID.
-# HINDSIGHT_API_RERANKER_SEND_BANK_AS_HEADER=false
 # For local provider:
 # HINDSIGHT_API_RERANKER_LOCAL_MODEL=cross-encoder/ms-marco-MiniLM-L-6-v2
-# Force CPU if the local reranker hits MPS/XPC instability on macOS:
-# HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU=false
-# Opt in to the Apple Silicon MPS GPU (off by default: MPS leaks memory under
-# variable-length workloads). CUDA/XPU still auto-select regardless:
-# HINDSIGHT_API_RERANKER_LOCAL_ALLOW_MPS=false
 # For TEI provider:
 # HINDSIGHT_API_RERANKER_TEI_URL=http://localhost:8081
-# For flashrank provider: passages scored per ONNX forward pass. Each pass
-# allocates attention tensors sized batch * heads * seq^2, so raising this
-# raises peak memory quadratically in passage length:
-# HINDSIGHT_API_RERANKER_FLASHRANK_BATCH_SIZE=32
-# Max candidates the cross-encoder reranks per recall (RRF pre-filters the rest):
-# HINDSIGHT_API_RERANKER_MAX_CANDIDATES=300
-# Optionally scale that cap by the recall budget level (the cross-encoder dominates
-# a large recall's latency). 0 = fall back to the flat cap above; fully backwards-compatible.
-# HINDSIGHT_API_RERANKER_MAX_CANDIDATES_LOW=0
-# HINDSIGHT_API_RERANKER_MAX_CANDIDATES_MID=0
-# HINDSIGHT_API_RERANKER_MAX_CANDIDATES_HIGH=0
-
-# Reranker failover chain: extra rerankers tried, in order, when the one above
-# fails. Members are numbered from 1 (indices must be contiguous) and every
-# setting of member n carries the same index. A member inherits nothing from the
-# primary, so spell out everything it needs. Unset = no fallback (default): a
-# failing reranker fails the recall. End the chain with "rrf" to fail open and
-# keep the retrieval order instead.
-# HINDSIGHT_API_RERANKER_1_PROVIDER=cohere
-# HINDSIGHT_API_RERANKER_1_COHERE_API_KEY=your-cohere-api-key
-# HINDSIGHT_API_RERANKER_2_PROVIDER=rrf
 
 # Observability & Tracing (Optional - disabled by default)
 # Enable OpenTelemetry tracing for LLM calls (GenAI semantic conventions)
@@ -372,64 +83,3 @@ HINDSIGHT_API_LOG_LEVEL=info
 # Custom service name and environment (optional, defaults: hindsight-api, development)
 # HINDSIGHT_API_OTEL_SERVICE_NAME=hindsight-production
 # HINDSIGHT_API_OTEL_DEPLOYMENT_ENVIRONMENT=production
-#
-# Expose async-operation queue + consolidation-backlog gauges on /metrics.
-# Runs periodic per-schema COUNT queries on a background task (disabled by default).
-# HINDSIGHT_API_METRICS_BACKLOG_ENABLED=true
-#
-# Runtime-stall observability (enabled by default). When a liveness probe fails,
-# these tell you WHY: a blocked event loop vs DB connection-pool exhaustion.
-# The loop watchdog logs the offending stack when the loop is unresponsive; the
-# DB-pool acquire timing logs (and exposes hindsight.db.pool.waiting) when
-# callers queue for a connection. Both are cheap; tune or disable if needed.
-# HINDSIGHT_API_LOOP_WATCHDOG_ENABLED=false
-# HINDSIGHT_API_LOOP_WATCHDOG_STALL_THRESHOLD_MS=1000
-# HINDSIGHT_API_LOOP_WATCHDOG_POLL_INTERVAL_MS=250
-# HINDSIGHT_API_DB_ACQUIRE_WARN_THRESHOLD_MS=1000
-
-# -----------------------------------------------------------------------------
-# Extensions (Optional)
-# -----------------------------------------------------------------------------
-
-# Request headers copied into RequestContext.extra_headers so a custom
-# TenantExtension / OperationValidatorExtension can read them. Comma-separated,
-# matched case-insensitively. Unset by default: extensions see only the
-# Authorization header. Use this when the bearer token identifies a proxy rather
-# than the caller, and per-caller identity arrives in a separate header. A listed
-# header that arrives more than once is dropped, so only list headers the proxy
-# in front of Hindsight sets itself (stripping any client-supplied copy).
-# HINDSIGHT_API_EXTENSION_PASSTHROUGH_HEADERS=x-user-assertion
-
-# -----------------------------------------------------------------------------
-# Webhooks (Optional)
-# -----------------------------------------------------------------------------
-
-# Outbound webhook delivery targets caller-supplied URLs. To prevent SSRF, the
-# delivery worker blocks private, loopback, and link-local destinations
-# (including the cloud metadata address 169.254.169.254) by default. List hosts
-# or IP/CIDR ranges here (comma-separated) to re-permit specific internal
-# destinations — e.g. 127.0.0.1 for local testing, or an internal receiver.
-# HINDSIGHT_API_WEBHOOK_ALLOWED_HOSTS=127.0.0.1,internal-receiver.svc,10.0.0.0/8
-
-# Whether the webhook delivery-history API returns the raw upstream response
-# body. Off by default: returning arbitrary response bodies to callers is an
-# information-exfiltration primitive. The delivery status code is always
-# returned regardless. Enable only if you trust your webhook destinations.
-# HINDSIGHT_API_WEBHOOK_EXPOSE_RESPONSE_BODY=false
-
-# -----------------------------------------------------------------------------
-# Control Plane (Optional)
-# -----------------------------------------------------------------------------
-
-# Dataplane API URL - where the CP proxies requests to
-# HINDSIGHT_CP_DATAPLANE_API_URL=http://localhost:8888
-
-# Optional: Bearer token the CP sends as `Authorization: Bearer <key>` to the
-# dataplane API. Required when the API service is auth-protected; omit for a
-# public/unauthenticated API.
-# HINDSIGHT_CP_DATAPLANE_API_KEY=your-dataplane-bearer-token
-
-# Optional: Require a shared access key to view the Control Plane UI.
-# When set, visitors see a login page and must enter the key before
-# accessing the dashboard or any /api/* routes (except /api/health).
-# HINDSIGHT_CP_ACCESS_KEY=your-shared-secret-key

.github/dependabot.yml

@@ -0,0 +1,6 @@
+version: 2
+updates:
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "weekly"

.github/workflows/deploy-docs.yml

@@ -22,8 +22,6 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v6
-        with:
-          fetch-depth: 0 # fetch tags so check-released-integrations can see them
       - uses: actions/setup-node@v6
         with:
           node-version: 20

.github/workflows/perf-test.yml

@@ -22,31 +22,15 @@ on:
           - ""
           - retain
           - recall
-          - recall-with-observations
-          - recall-temporal
-          - consolidation
-          - graph-maintenance
-        default: ""
-      locomo_conversations:
-        description: "LoComo conversation IDs (space-separated). Blank = curated set (conv-26 conv-30 conv-43)."
-        type: string
         default: ""
+      locomo_max_conversations:
+        description: "LoComo max conversations (0 = skip, blank = all)"
+        type: number
+        default: 0
       locomo_skip:
         description: "Skip LoComo job"
         type: boolean
         default: false
-      obs_skip:
-        description: "Skip observation-dedup benchmark job"
-        type: boolean
-        default: false
-      obs_dataset:
-        description: "Obs benchmark dataset substring (blank = English hermes transcript)."
-        type: string
-        default: ""
-      obs_fraction:
-        description: "Obs benchmark fraction (0-1] of each document to run."
-        type: string
-        default: "1.0"
       ref:
         description: "Git ref to test (branch, tag, or SHA). Defaults to main."
         type: string
@@ -97,7 +81,7 @@ jobs:
       run: |
         cd hindsight-dev && uv sync --frozen --all-extras --index-strategy unsafe-best-match
 
-    - name: Run perf-test
+    - name: Run perf tests
       run: |
         SUITE_ARG=""
         if [ -n "${{ inputs.suite }}" ]; then
@@ -110,23 +94,12 @@ jobs:
 
     - name: Upload perf results
       if: always()
-      uses: actions/upload-artifact@v7
+      uses: actions/upload-artifact@v4
       with:
         name: perf-results-${{ github.sha }}
         path: hindsight-dev/perf-results.json
         retention-days: 90
 
-    # Publish enriched results (perf JSON + commit metadata) to the dashboard
-    # repo's gh-pages branch. The static site at
-    #   https://vectorize-io.github.io/hindsight-continuous-performance-monitor/
-    # reads data/index.json + data/<run>.json and renders charts client-side.
-    - name: Publish to dashboard
-      if: github.event_name == 'schedule' || github.event_name == 'push' || github.event_name == 'workflow_dispatch'
-      env:
-        PERF_DASHBOARD_TOKEN: ${{ secrets.PERF_DASHBOARD_TOKEN }}
-        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      run: ./scripts/benchmarks/publish-perf-results.sh hindsight-dev/perf-results.json
-
   locomo:
     if: inputs.locomo_skip != true
     runs-on: ubuntu-latest
@@ -137,7 +110,7 @@ jobs:
       HINDSIGHT_API_JUDGE_LLM_PROVIDER: vertexai
       HINDSIGHT_API_JUDGE_LLM_MODEL: google/gemini-2.5-flash-lite
       HINDSIGHT_API_ANSWER_LLM_PROVIDER: vertexai
-      HINDSIGHT_API_ANSWER_LLM_MODEL: google/gemini-2.5-flash
+      HINDSIGHT_API_ANSWER_LLM_MODEL: google/gemini-3.1-pro-preview
     steps:
     - uses: actions/checkout@v6
       with:
@@ -183,115 +156,19 @@ jobs:
         cd hindsight-dev && uv sync --frozen --all-extras --index-strategy unsafe-best-match
 
     - name: Run LoComo benchmark
-      # Curated 3-conversation subset (best/middle/worst by accuracy on the
-      # last successful full run): conv-26 (best), conv-30 (middle), conv-43
-      # (worst). Excludes conv-44, the bank with the largest unconsolidated
-      # set that has been pushing scheduled runs over the per-bank
-      # _wait_for_consolidation timeout. Override via workflow_dispatch with
-      # the locomo_conversations input.
       run: |
-        CONVERSATIONS="${{ inputs.locomo_conversations }}"
-        if [ -z "$CONVERSATIONS" ]; then
-          CONVERSATIONS="conv-26 conv-30 conv-43"
+        MAX_CONV_ARG=""
+        if [ "${{ inputs.locomo_max_conversations }}" != "0" ] && [ -n "${{ inputs.locomo_max_conversations }}" ]; then
+          MAX_CONV_ARG="--max-conversations ${{ inputs.locomo_max_conversations }}"
         fi
         uv run python hindsight-dev/benchmarks/locomo/locomo_benchmark.py \
           --wait-consolidation \
-          --conversation $CONVERSATIONS
+          $MAX_CONV_ARG
 
     - name: Upload LoComo results
       if: always()
-      uses: actions/upload-artifact@v7
+      uses: actions/upload-artifact@v4
       with:
         name: locomo-results-${{ github.sha }}
         path: hindsight-dev/benchmarks/locomo/results/
         retention-days: 90
-
-    - name: Publish LoComo to dashboard
-      if: success() && (github.event_name == 'schedule' || github.event_name == 'push' || github.event_name == 'workflow_dispatch')
-      env:
-        PERF_DASHBOARD_TOKEN: ${{ secrets.PERF_DASHBOARD_TOKEN }}
-        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      run: ./scripts/benchmarks/publish-locomo-results.sh hindsight-dev/benchmarks/locomo/results/benchmark_results.json
-
-  obs:
-    # Observation-dedup quality benchmark: ingests a transcript, drains consolidation
-    # (serial SyncTaskBackend + embedded pg0 — no external DB / worker), and reports the
-    # near-duplicate observation rate. Real LLM via VertexAI, mirroring the LoComo job.
-    if: inputs.obs_skip != true
-    runs-on: ubuntu-latest
-    env:
-      HINDSIGHT_API_LLM_PROVIDER: vertexai
-      HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
-      HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
-      HINDSIGHT_API_ENABLE_OBSERVATIONS: "true"
-    steps:
-    - uses: actions/checkout@v6
-      with:
-        ref: ${{ inputs.ref || github.ref }}
-
-    - name: Setup GCP credentials
-      run: |
-        printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
-        PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
-        echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
-
-    - name: Install uv
-      uses: astral-sh/setup-uv@v7
-      with:
-        enable-cache: true
-        prune-cache: false
-
-    - name: Set up Python
-      uses: actions/setup-python@v6
-      with:
-        python-version-file: ".python-version"
-
-    - name: Cache HuggingFace models
-      uses: actions/cache@v5
-      with:
-        path: ~/.cache/huggingface
-        key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
-        restore-keys: |
-          ${{ runner.os }}-huggingface-
-
-    - name: Pre-download models
-      working-directory: ./hindsight-api-slim
-      run: |
-        uv run --frozen --all-extras --index-strategy unsafe-best-match python -c "
-        from sentence_transformers import SentenceTransformer
-        print('Downloading embedding model...')
-        SentenceTransformer('BAAI/bge-small-en-v1.5')
-        print('Model downloaded successfully')
-        "
-
-    - name: Install hindsight-dev dependencies
-      run: |
-        cd hindsight-dev && uv sync --frozen --all-extras --index-strategy unsafe-best-match
-
-    - name: Run obs benchmark
-      # Default to the English hermes transcript at full fraction — a clean, deterministic
-      # consolidation-dedup signal (the Chinese variant adds a cross-lingual embedding
-      # confound). Override dataset/fraction via workflow_dispatch.
-      run: |
-        DATASET="${{ inputs.obs_dataset }}"
-        if [ -z "$DATASET" ]; then DATASET="hermes_session_2026-05-15_en"; fi
-        FRACTION="${{ inputs.obs_fraction }}"
-        if [ -z "$FRACTION" ]; then FRACTION="1.0"; fi
-        cd hindsight-dev
-        uv run python -m benchmarks.obs.obs_benchmark \
-          --dataset "$DATASET" --fraction "$FRACTION" --wipe-bank --output obs-results.json
-
-    - name: Upload obs results
-      if: always()
-      uses: actions/upload-artifact@v7
-      with:
-        name: obs-results-${{ github.sha }}
-        path: hindsight-dev/obs-results.json
-        retention-days: 90
-
-    - name: Publish obs to dashboard
-      if: success() && (github.event_name == 'schedule' || github.event_name == 'push' || github.event_name == 'workflow_dispatch')
-      env:
-        PERF_DASHBOARD_TOKEN: ${{ secrets.PERF_DASHBOARD_TOKEN }}
-        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      run: ./scripts/benchmarks/publish-obs-results.sh hindsight-dev/obs-results.json

.github/workflows/release-integration.yml

@@ -9,11 +9,7 @@ jobs:
   publish:
     runs-on: ubuntu-latest
     permissions:
-      id-token: write  # for PyPI trusted publishing + build-provenance attestations
-      attestations: write  # for actions/attest-build-provenance (Obsidian assets)
-      # No `contents: write`: we never create releases in this repo. The Obsidian
-      # plugin's distribution release is pushed to its dedicated repo using
-      # OBSIDIAN_DIST_TOKEN (see the "Mirror Obsidian plugin" step below).
+      id-token: write  # for PyPI trusted publishing
 
     steps:
     - uses: actions/checkout@v6
@@ -75,7 +71,7 @@ jobs:
       if: steps.type.outputs.type == 'plugin'
       run: |
         echo "Plugin integration ${{ steps.info.outputs.integration }} v${{ steps.info.outputs.version }} — no package to publish."
-        echo "Users install via: claude plugin marketplace add vectorize-io/hindsight"
+        echo "Users install via: claude plugin marketplace add vectorize-io/hindsight --sparse hindsight-integrations"
 
     # ── TypeScript integrations (ai-sdk, chat, openclaw) ────────────────────
 
@@ -85,28 +81,17 @@ jobs:
       with:
         node-version: '22'
         registry-url: 'https://registry.npmjs.org'
-        cache: 'npm'
-        cache-dependency-path: package-lock.json
 
+    # Guard: fail fast if the integration's lockfile resolves any dep from a
+    # monorepo workspace (link=true) or a relative file path. The release
+    # runner has no pre-built workspace `dist/` so `npm run build` would
+    # later fail at tsc with "Cannot find module". See:
+    # https://github.com/vectorize-io/hindsight/issues/… (0.6.0 openclaw retry)
     - name: Check integration lockfile
       if: steps.type.outputs.type == 'typescript'
       run: ./scripts/check-integration-lockfiles.sh
 
-    # Some integrations depend on workspace packages (hindsight-client,
-    # hindsight-all, hindsight-agent-sdk) via file: refs. Install from root
-    # so npm resolves them, then build the workspace deps before the integration.
-    - name: Install root workspace dependencies
-      if: steps.type.outputs.type == 'typescript'
-      run: npm ci
-
-    - name: Build workspace deps (hindsight-client, hindsight-all, hindsight-agent-sdk)
-      if: steps.type.outputs.type == 'typescript'
-      run: |
-        npm run build --workspace=hindsight-clients/typescript
-        npm run build --workspace=hindsight-all-npm
-        npm run build --workspace=hindsight-tools/hindsight-agent-sdk
-
-    - name: Install integration dependencies
+    - name: Install dependencies
       if: steps.type.outputs.type == 'typescript'
       working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
       run: npm ci
@@ -116,86 +101,16 @@ jobs:
       working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
       run: npm run build
 
-    # Build-provenance attestations for the Obsidian release assets (community-store
-    # recommendation). Runs after the build so main.js exists. The assets are
-    # released in the dedicated repo while the build runs here, so users verify at
-    # owner scope: `gh attestation verify main.js --owner vectorize-io`.
-    - name: Attest Obsidian plugin build provenance
-      if: steps.type.outputs.type == 'typescript' && steps.info.outputs.integration == 'obsidian'
-      uses: actions/attest-build-provenance@v2
-      with:
-        subject-path: |
-          hindsight-integrations/obsidian/main.js
-          hindsight-integrations/obsidian/styles.css
-
-    # ── Obsidian plugin — mirror to its dedicated repo + cut the BRAT release ──
-    # We do NOT create a GitHub Release in this monorepo: per-integration
-    # releases pollute the repo's release list (it's for the core product) and
-    # steal the "Latest" badge, and BRAT / the community store read a repo's
-    # *latest* release — not a tag — so they can't target a tag in a monorepo.
-    #
-    # Instead this monorepo stays the source of truth, and on each obsidian
-    # release we mirror hindsight-integrations/obsidian/ → the *root* of
-    # github.com/vectorize-io/hindsight-obsidian (git subtree, history
-    # preserved) and cut the BRAT / community-store release *there*.
-    #
-    # Requires secret OBSIDIAN_DIST_TOKEN — a token with `contents: write` on
-    # vectorize-io/hindsight-obsidian (fine-grained PAT or app installation
-    # token). The dedicated repo is generated; never edit it directly.
-    - name: Mirror Obsidian plugin to its dedicated repo
-      if: steps.type.outputs.type == 'typescript' && steps.info.outputs.integration == 'obsidian'
-      env:
-        DIST_TOKEN: ${{ secrets.OBSIDIAN_DIST_TOKEN }}
-      run: |
-        set -euo pipefail
-        VERSION="${{ steps.info.outputs.version }}"
-        DIST_REPO="vectorize-io/hindsight-obsidian"
-        OBS_DIR="hindsight-integrations/obsidian"
-
-        # `git subtree split` needs full history; the default checkout is shallow.
-        git fetch --unshallow 2>/dev/null || true
-        git config user.name "github-actions[bot]"
-        git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
-
-        # The runner injects the default GITHUB_TOKEN as an http.extraheader via
-        # an *included* config file (/home/runner/work/_temp/git-credentials-*.config),
-        # so `git config --local --unset-all` can't remove it and it authenticates
-        # the push as github-actions[bot] (no access to the dedicated repo → 403).
-        # The documented way to drop an inherited extraheader is to RESET the list
-        # with an empty value: since command-line `-c` is read last, the empty
-        # value clears the accumulated headers (including the included one) at
-        # request-build time. The dist token then comes from the push URL → a
-        # single Authorization header.
-        git subtree split --prefix="$OBS_DIR" -b _obs_dist
-        git -c "http.https://github.com/.extraheader=" \
-          push "https://x-access-token:${DIST_TOKEN}@github.com/${DIST_REPO}.git" _obs_dist:main
-
-        # Cut the BRAT / community-store release. Bare version tag (e.g. 0.1.0)
-        # to match manifest.json — idempotent so re-runs just refresh the assets.
-        export GH_TOKEN="$DIST_TOKEN"
-        ASSETS="$OBS_DIR/main.js $OBS_DIR/manifest.json $OBS_DIR/styles.css"
-        NOTES="Hindsight for Obsidian v${VERSION}. Install via BRAT (add ${DIST_REPO}) or copy main.js/manifest.json/styles.css into <vault>/.obsidian/plugins/hindsight/."
-        if gh release view "$VERSION" --repo "$DIST_REPO" >/dev/null 2>&1; then
-          gh release upload "$VERSION" $ASSETS --repo "$DIST_REPO" --clobber
-        else
-          gh release create "$VERSION" $ASSETS --repo "$DIST_REPO" --title "$VERSION" --notes "$NOTES"
-        fi
-
     - name: Publish TypeScript package to npm
       if: steps.type.outputs.type == 'typescript'
       working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
       run: |
         set +e
-        OUTPUT=$(npm publish --access public --provenance 2>&1)
+        OUTPUT=$(npm publish --access public 2>&1)
         EXIT_CODE=$?
         echo "$OUTPUT"
         if [ $EXIT_CODE -ne 0 ]; then
-          # Treat "already published" as success so re-pointed-tag re-runs stay green.
-          # "cannot publish over" = the version exists. TLOG_CREATE_ENTRY_ERROR / 409
-          # "equivalent entry already exists in the transparency log" = the identical
-          # --provenance artifact was already logged on a prior run (Sigstore tlog is
-          # idempotent); the package is published, so this is benign.
-          if echo "$OUTPUT" | grep -qE "cannot publish over|TLOG_CREATE_ENTRY_ERROR|already exists in the transparency log"; then
+          if echo "$OUTPUT" | grep -q "cannot publish over"; then
             echo "Package version already published, skipping..."
             exit 0
           fi

.github/workflows/release-tool.yml

@@ -1,65 +0,0 @@
-name: Release Tool
-
-on:
-  push:
-    tags:
-      - 'tools/**'
-
-jobs:
-  publish:
-    runs-on: ubuntu-latest
-
-    steps:
-    - uses: actions/checkout@v6
-
-    - name: Extract tool info
-      id: info
-      run: |
-        # refs/tags/tools/self-driving-agents/v0.0.1 → tool=self-driving-agents, version=0.0.1
-        TAG="${GITHUB_REF#refs/tags/}"
-        TOOL=$(echo "$TAG" | cut -d'/' -f2)
-        VERSION=$(echo "$TAG" | cut -d'/' -f3 | sed 's/^v//')
-        echo "tool=$TOOL" >> $GITHUB_OUTPUT
-        echo "version=$VERSION" >> $GITHUB_OUTPUT
-        echo "tag=$TAG" >> $GITHUB_OUTPUT
-        echo "Tool: $TOOL, Version: $VERSION"
-
-    - name: Set up Node.js
-      uses: actions/setup-node@v6
-      with:
-        node-version: '22'
-        registry-url: 'https://registry.npmjs.org'
-        cache: 'npm'
-        cache-dependency-path: package-lock.json
-
-    # Tools live under hindsight-tools/ and may depend on workspace packages
-    # (e.g. @vectorize-io/hindsight-client). Install from root so npm resolves
-    # workspace deps, then build any required workspace packages first.
-    - name: Install root workspace dependencies
-      run: npm ci
-
-    - name: Build hindsight-client (workspace dep)
-      run: npm run build --workspace=hindsight-clients/typescript
-
-    - name: Build hindsight-agent-sdk (workspace dep)
-      run: npm run build --workspace=hindsight-tools/hindsight-agent-sdk
-
-    - name: Build tool
-      run: npm run build --workspace=hindsight-tools/${{ steps.info.outputs.tool }}
-
-    - name: Publish to npm
-      working-directory: ./hindsight-tools/${{ steps.info.outputs.tool }}
-      run: |
-        set +e
-        OUTPUT=$(npm publish --access public 2>&1)
-        EXIT_CODE=$?
-        echo "$OUTPUT"
-        if [ $EXIT_CODE -ne 0 ]; then
-          if echo "$OUTPUT" | grep -q "cannot publish over"; then
-            echo "Package version already published, skipping..."
-            exit 0
-          fi
-          exit $EXIT_CODE
-        fi
-      env:
-        NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

.github/workflows/release.yml

@@ -25,20 +25,6 @@ jobs:
       with:
         python-version-file: ".python-version"
 
-    # Each package is built from its own directory, so stage the repository's
-    # canonical license inside each isolated build context.
-    - name: Stage Python package licenses
-      run: |
-        for package in \
-          hindsight-clients/python \
-          hindsight-api-slim \
-          hindsight-api \
-          hindsight-all \
-          hindsight-all-slim \
-          hindsight-embed; do
-          cp LICENSE "$package/LICENSE"
-        done
-
     # Build all packages
     - name: Build hindsight-client
       working-directory: ./hindsight-clients/python
@@ -64,24 +50,6 @@ jobs:
       working-directory: ./hindsight-embed
       run: uv build --out-dir dist
 
-    - name: Verify Python package licenses
-      run: |
-        for package in \
-          hindsight-clients/python \
-          hindsight-api-slim \
-          hindsight-api \
-          hindsight-all \
-          hindsight-all-slim \
-          hindsight-embed; do
-          wheel=$(find "$package/dist" -maxdepth 1 -name '*.whl' -print -quit)
-          sdist=$(find "$package/dist" -maxdepth 1 -name '*.tar.gz' -print -quit)
-
-          unzip -Z1 "$wheel" | grep -Eq '\.dist-info/licenses/LICENSE$'
-          unzip -p "$wheel" '*/METADATA' | grep -Fxq 'License-Expression: MIT'
-          unzip -p "$wheel" '*/METADATA' | grep -Fxq 'License-File: LICENSE'
-          tar -tzf "$sdist" | grep -Eq '/LICENSE$'
-        done
-
     # Publish in order (client and api-slim first, then api/all wrappers which depend on them)
     - name: Publish hindsight-client to PyPI
       uses: pypa/gh-action-pypi-publish@release/v1
@@ -298,7 +266,7 @@ jobs:
     strategy:
       matrix:
         include:
-          - os: ubuntu-22.04
+          - os: ubuntu-latest
             target: x86_64-unknown-linux-gnu
             artifact_name: hindsight
             asset_name: hindsight-linux-amd64
@@ -310,7 +278,7 @@ jobs:
             target: aarch64-apple-darwin
             artifact_name: hindsight
             asset_name: hindsight-darwin-arm64
-          - os: ubuntu-22.04-arm
+          - os: ubuntu-24.04-arm
             target: aarch64-unknown-linux-gnu
             artifact_name: hindsight
             asset_name: hindsight-linux-arm64
@@ -327,50 +295,17 @@ jobs:
       working-directory: hindsight-cli
       run: cargo build --release --target ${{ matrix.target }}
 
-    - name: Install cargo-about
-      if: matrix.asset_name == 'hindsight-linux-amd64'
-      uses: taiki-e/install-action@v2
-      with:
-        tool: [email protected]
-
-    - name: Verify cargo-about
-      if: matrix.asset_name == 'hindsight-linux-amd64'
-      run: cargo about --version
-
-    # The build above only fetches crates for this target; cargo-about resolves
-    # the graph for every target platform, so fetch for all of them (that is what
-    # `cargo fetch` without --target does) before the --offline generate.
-    - name: Fetch crate sources for the license scan
-      if: matrix.asset_name == 'hindsight-linux-amd64'
-      working-directory: hindsight-cli
-      run: cargo fetch
-
-    - name: Generate license manifest
-      if: matrix.asset_name == 'hindsight-linux-amd64'
-      working-directory: hindsight-cli
-      run: mkdir -p ../artifacts && cargo about generate --offline --manifest-path Cargo.toml --config about.toml about.hbs --output-file ../artifacts/THIRD_PARTY_LICENSES.txt
-
-    - name: Verify license files
-      if: matrix.asset_name == 'hindsight-linux-amd64'
-      run: |
-        test -s LICENSE
-        test -s artifacts/THIRD_PARTY_LICENSES.txt
-        grep -Fq "THIRD-PARTY SOFTWARE LICENSES" artifacts/THIRD_PARTY_LICENSES.txt
-
     - name: Prepare artifact
       run: |
         mkdir -p artifacts
         cp hindsight-cli/target/${{ matrix.target }}/release/${{ matrix.artifact_name }} artifacts/${{ matrix.asset_name }}
-        if [ "${{ matrix.asset_name }}" = "hindsight-linux-amd64" ]; then
-          cp LICENSE artifacts/LICENSE
-        fi
         chmod +x artifacts/${{ matrix.asset_name }}
 
     - name: Upload artifacts
       uses: actions/upload-artifact@v7
       with:
         name: rust-cli-${{ matrix.asset_name }}
-        path: artifacts/*
+        path: artifacts/${{ matrix.asset_name }}
         retention-days: 1
 
   release-docker-images:
@@ -379,7 +314,6 @@ jobs:
     permissions:
       contents: read
       packages: write
-      id-token: write
     strategy:
       matrix:
         include:
@@ -476,7 +410,6 @@ jobs:
 
     # Build multi-platform and push to release tags
     - name: Build and push release images
-      id: build
       uses: docker/build-push-action@v7
       with:
         context: .
@@ -488,31 +421,6 @@ jobs:
         tags: ${{ steps.meta.outputs.tags }}
         labels: ${{ steps.meta.outputs.labels }}
 
-    - name: Install cosign
-      uses: sigstore/cosign-installer@v3
-
-    - name: Sign published images
-      env:
-        TAGS: ${{ steps.meta.outputs.tags }}
-        DIGEST: ${{ steps.build.outputs.digest }}
-      run: |
-        set -euo pipefail
-        refs=()
-        while IFS= read -r tag; do
-          [[ -z "${tag}" ]] && continue
-          refs+=("${tag}@${DIGEST}")
-        done <<< "${TAGS}"
-        cosign sign --yes "${refs[@]}"
-
-    - name: Verify signature on primary tag
-      env:
-        IMAGE: ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}
-        DIGEST: ${{ steps.build.outputs.digest }}
-      run: |
-        cosign verify "${IMAGE}@${DIGEST}" \
-          --certificate-identity-regexp "^https://github\.com/${{ github.repository }}/\.github/workflows/release\.yml@.*" \
-          --certificate-oidc-issuer https://token.actions.githubusercontent.com
-
   release-helm-chart:
     runs-on: ubuntu-latest
     permissions:
@@ -589,12 +497,6 @@ jobs:
         name: rust-cli-hindsight-linux-amd64
         path: ./artifacts/rust-cli-linux
 
-    - name: Download Rust CLI (Linux ARM)
-      uses: actions/download-artifact@v8
-      with:
-        name: rust-cli-hindsight-linux-arm64
-        path: ./artifacts/rust-cli-linux-arm64
-
     - name: Download Rust CLI (macOS Intel)
       uses: actions/download-artifact@v8
       with:
@@ -631,14 +533,8 @@ jobs:
         cp artifacts/control-plane/*.tgz release-assets/ || true
         # Rust CLI binaries
         cp artifacts/rust-cli-linux/hindsight-linux-amd64 release-assets/ || true
-        cp artifacts/rust-cli-linux-arm64/hindsight-linux-arm64 release-assets/ || true
         cp artifacts/rust-cli-darwin-amd64/hindsight-darwin-amd64 release-assets/ || true
         cp artifacts/rust-cli-darwin-arm64/hindsight-darwin-arm64 release-assets/ || true
-        # Rust CLI license files (shared by all four platform binaries)
-        cp artifacts/rust-cli-linux/LICENSE release-assets/
-        cp artifacts/rust-cli-linux/THIRD_PARTY_LICENSES.txt release-assets/
-        test -s release-assets/LICENSE
-        test -s release-assets/THIRD_PARTY_LICENSES.txt
         # Helm chart
         cp artifacts/helm-chart/*.tgz release-assets/ || true
         ls -la release-assets/

.github/workflows/sign-images.yml

@@ -1,71 +0,0 @@
-name: Sign published images
-
-on:
-  workflow_dispatch:
-    inputs:
-      version:
-        description: 'Version to sign (without leading v, e.g. 0.6.0)'
-        required: true
-        type: string
-        default: '0.6.0'
-
-permissions:
-  contents: read
-  packages: write
-  id-token: write
-
-jobs:
-  sign:
-    name: Sign ${{ matrix.image }}:${{ inputs.version }}${{ matrix.suffix }}
-    runs-on: ubuntu-latest
-    strategy:
-      fail-fast: false
-      matrix:
-        include:
-          - { image: hindsight-api, suffix: '' }
-          - { image: hindsight-api, suffix: '-slim' }
-          - { image: hindsight-control-plane, suffix: '' }
-          - { image: hindsight, suffix: '' }
-          - { image: hindsight, suffix: '-slim' }
-
-    steps:
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v4
-
-      - name: Log in to GHCR
-        uses: docker/login-action@v4
-        with:
-          registry: ghcr.io
-          username: ${{ github.actor }}
-          password: ${{ secrets.GITHUB_TOKEN }}
-
-      - name: Install cosign
-        uses: sigstore/cosign-installer@v3
-
-      - name: Resolve image digest
-        id: resolve
-        env:
-          IMAGE: ghcr.io/${{ github.repository_owner }}/${{ matrix.image }}
-          TAG: ${{ inputs.version }}${{ matrix.suffix }}
-        run: |
-          set -euo pipefail
-          DIGEST=$(docker buildx imagetools inspect "${IMAGE}:${TAG}" --format '{{json .Manifest.Digest}}' | tr -d '"')
-          if [[ -z "${DIGEST}" || "${DIGEST}" != sha256:* ]]; then
-            echo "Failed to resolve digest for ${IMAGE}:${TAG} (got: ${DIGEST})" >&2
-            exit 1
-          fi
-          echo "Resolved ${IMAGE}:${TAG} -> ${DIGEST}"
-          echo "ref=${IMAGE}@${DIGEST}" >> "$GITHUB_OUTPUT"
-
-      - name: Sign image
-        env:
-          REF: ${{ steps.resolve.outputs.ref }}
-        run: cosign sign --yes "${REF}"
-
-      - name: Verify signature
-        env:
-          REF: ${{ steps.resolve.outputs.ref }}
-        run: |
-          cosign verify "${REF}" \
-            --certificate-identity-regexp "^https://github\.com/${{ github.repository }}/\.github/workflows/sign-images\.yml@.*" \
-            --certificate-oidc-issuer https://token.actions.githubusercontent.com

.github/workflows/star-history.yml

@@ -1,29 +0,0 @@
-name: Update star history
-
-on:
-  schedule:
-    - cron: '17 3 * * *'
-  workflow_dispatch:
-
-permissions:
-  contents: write
-
-jobs:
-  update:
-    concurrency:
-      group: star-history
-      cancel-in-progress: false
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v4
-      - uses: nicoloboschi/gh-stars@v1
-        with:
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-          line-color: '#14b8a6'
-      - name: Commit chart
-        run: |
-          git config user.name 'github-actions[bot]'
-          git config user.email '41898282+github-actions[bot]@users.noreply.github.com'
-          git add .github/star-history/data.json .github/star-history/chart.svg
-          git diff --cached --quiet || git commit -m 'chore: update star history'
-          git push

.github/workflows/windows-smoke.yml

@@ -1,115 +0,0 @@
-name: Windows Smoke Test
-
-# Daily smoke test that installs the API on Windows and runs the Python client
-# integration tests against a live server. Windows is only exercised by the
-# hindsight-embed jobs in test.yml on PRs; this catches Windows-specific
-# regressions in the API server + client path (e.g. process spawning, console
-# subsystem / ConPTY behaviour, see #1885) that the Linux client jobs miss.
-on:
-  schedule:
-    # 06:00 UTC daily.
-    - cron: "0 6 * * *"
-  workflow_dispatch:
-
-permissions:
-  contents: read
-
-jobs:
-  windows-client-smoke:
-    # Don't run on forks: the job needs the org's Vertex AI credentials.
-    if: github.repository == 'vectorize-io/hindsight'
-    runs-on: windows-latest
-    timeout-minutes: 45
-    env:
-      HINDSIGHT_API_LLM_PROVIDER: vertexai
-      HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: ${{ github.workspace }}/gcp-credentials.json
-      HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
-      HINDSIGHT_API_URL: http://localhost:8888
-      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      # Force UTF-8 I/O so the API/CLI's ✓/box-drawing output doesn't crash the
-      # default Windows cp1252 codec (matches test-embed-windows in test.yml).
-      PYTHONIOENCODING: utf-8
-      PYTHONUTF8: "1"
-
-    steps:
-      - uses: actions/checkout@v6
-
-      - name: Setup GCP credentials
-        shell: bash
-        run: |
-          printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > gcp-credentials.json
-          PROJECT_ID=$(jq -r '.project_id' gcp-credentials.json)
-          echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> "$GITHUB_ENV"
-
-      - name: Install uv
-        uses: astral-sh/setup-uv@v7
-        with:
-          enable-cache: true
-          prune-cache: false
-
-      - name: Set up Python
-        uses: actions/setup-python@v6
-        with:
-          python-version-file: ".python-version"
-
-      - name: Install API dependencies (all extras - local-ml + embedded pg0)
-        working-directory: ./hindsight-api-slim
-        run: uv sync --frozen --all-extras --index-strategy unsafe-best-match
-
-      - name: Install Python client test dependencies
-        working-directory: ./hindsight-clients/python
-        run: uv sync --frozen --extra test --index-strategy unsafe-best-match
-
-      - name: Cache HuggingFace models
-        uses: actions/cache@v5
-        with:
-          path: ~/.cache/huggingface
-          key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
-          restore-keys: |
-            ${{ runner.os }}-huggingface-
-
-      # `uv run` re-syncs the project env to its default (no-extras) state before
-      # running, which drops sentence-transformers / pg0. Pass --all-extras on
-      # every `uv run` so the local-ml + embedded-db deps stay installed (this is
-      # the same reason hindsight-embed launches the daemon with `--extra all`).
-      - name: Pre-download models
-        working-directory: ./hindsight-api-slim
-        run: |
-          uv run --all-extras python -c "from sentence_transformers import SentenceTransformer, CrossEncoder; SentenceTransformer('BAAI/bge-small-en-v1.5'); CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2'); print('Models downloaded')"
-
-      # Start the server and run the client tests in a SINGLE step. On Windows
-      # runners a process backgrounded with `&` in one step is not reliably kept
-      # alive for later steps (unlike Linux, where it reparents to init), so the
-      # server must live in the same shell that runs pytest.
-      - name: Start API server and run Python client tests
-        shell: bash
-        run: |
-          # Config is read straight from the environment (job-level env + the
-          # PROJECT_ID exported to GITHUB_ENV above), so no .env file is needed.
-          # Embedded pg0 is the default when HINDSIGHT_API_DATABASE_URL is unset.
-          ( cd hindsight-api-slim && uv run --all-extras hindsight-api --port 8888 ) > "$RUNNER_TEMP/api-server.log" 2>&1 &
-          server_pid=$!
-          echo "Waiting for API server to be ready (pid $server_pid)..."
-          # pg0 unpacks Postgres + runs initdb on first boot, which is slow on a
-          # cold Windows runner — give it a generous budget before failing.
-          ready=false
-          for i in $(seq 1 300); do
-            if curl -sf http://localhost:8888/health > /dev/null 2>&1; then
-              echo "API server is ready after ${i}s"
-              ready=true
-              break
-            fi
-            sleep 1
-          done
-          if [ "$ready" != true ]; then
-            echo "API server failed to start after 300s"
-            cat "$RUNNER_TEMP/api-server.log"
-            exit 1
-          fi
-
-          cd hindsight-clients/python && uv run --extra test pytest tests -v
-
-      - name: Show API server logs
-        if: always()
-        shell: bash
-        run: cat "$RUNNER_TEMP/api-server.log" || echo "No API server log found"

.gitignore

@@ -5,29 +5,16 @@ build/
 dist/
 wheels/
 *.egg-info
-# Release builds stage the canonical root license in each package context.
-/hindsight-clients/python/LICENSE
-/hindsight-api-slim/LICENSE
-/hindsight-api/LICENSE
-/hindsight-all/LICENSE
-/hindsight-all-slim/LICENSE
-/hindsight-embed/LICENSE
 .mcp.json
-.playwright-mcp/
 .osgrep
 # Virtual environments
 .venv
 
 # Node
 node_modules/
-# Without this, the pattern above matches directories only — a node_modules SYMLINK (what you get
-# pointing a scratch worktree at an installed one) is a file, slips past it, and can be committed.
-node_modules
 
 # Environment variables and local config
 .env
-.env.bak*
-.env.*.bak
 docker-compose.yml
 docker-compose.override.yml
 
@@ -51,7 +38,6 @@ nltk_data/
 logs/
 
 .DS_Store
-.sesskey
 
 # Generated docs files
 hindsight-docs/static/llms-full.txt
@@ -68,10 +54,7 @@ hindsight-clients/rust/target
 !.claude/skills/
 whats-next.md
 TASK.md
-# Parked / draft integrations that aren't ready to ship
-hindsight-integrations/_drafts/
 # Changelog is now tracked in hindsight-docs/src/pages/changelog.md
 # CHANGELOG.md
 
-blog-post*
-.worktrees/
+blog-post*

.sesskey

@@ -0,0 +1 @@
+fcac2839-1db5-432f-91e1-c5dac07d7290

CLAUDE.md

@@ -123,17 +123,12 @@ Key tables: `banks`, `memory_units`, `documents`, `entities`, `entity_links`
 
 ### Adding Database Migrations
 
-Hindsight runs the same Alembic tree against PostgreSQL and Oracle 23ai. Each
-migration file dispatches through `run_for_dialect`, which calls either
-`_pg_upgrade` or `_oracle_upgrade` based on the live connection. A pytest lint
-(`tests/test_migration_shape.py`) fails CI if a migration omits the dispatcher.
-
 1. **Create a new migration file** in `hindsight-api-slim/hindsight_api/alembic/versions/`:
    - File name format: `<revision_id>_<description>.py` (e.g., `f1a2b3c4d5e6_add_new_index.py`)
    - Use a unique hex revision ID (12 chars)
    - Set `down_revision` to the previous migration's revision ID
 
-2. **Migration template** (the `script.py.mako` template scaffolds this; fill in the bodies):
+2. **Migration template**:
    ```python
    """Description of the migration
 
@@ -144,58 +139,25 @@ migration file dispatches through `run_for_dialect`, which calls either
    from collections.abc import Sequence
    from alembic import context, op
 
-   from hindsight_api.alembic._dialect import run_for_dialect
-
    revision: str = "f1a2b3c4d5e6"
    down_revision: str | Sequence[str] | None = "<previous_revision_id>"
    branch_labels: str | Sequence[str] | None = None
    depends_on: str | Sequence[str] | None = None
 
-
-   def _pg_schema_prefix() -> str:
-       """Schema-qualifier for raw SQL on PG (multi-tenant search_path)."""
+   def _get_schema_prefix() -> str:
+       """Get schema prefix for table names (required for multi-tenant support)."""
        schema = context.config.get_main_option("target_schema")
        return f'"{schema}".' if schema else ""
 
-
-   def _pg_upgrade() -> None:
-       schema = _pg_schema_prefix()
+   def upgrade() -> None:
+       schema = _get_schema_prefix()
        op.execute(f"CREATE INDEX ... ON {schema}table_name(...)")
 
-
-   def _pg_downgrade() -> None:
-       schema = _pg_schema_prefix()
-       op.execute(f"DROP INDEX IF EXISTS {schema}index_name")
-
-
-   def _oracle_upgrade() -> None:
-       # Oracle 23ai equivalent. Use op.get_bind().exec_driver_sql for forms
-       # that Alembic core does not model (vector/text indexes, partitions).
-       op.execute("CREATE INDEX ... ON table_name(...)")
-
-
-   def _oracle_downgrade() -> None:
-       op.execute("DROP INDEX IF EXISTS index_name")
-
-
-   def upgrade() -> None:
-       run_for_dialect(pg=_pg_upgrade, oracle=_oracle_upgrade)
-
-
    def downgrade() -> None:
-       run_for_dialect(pg=_pg_downgrade, oracle=_oracle_downgrade)
+       schema = _get_schema_prefix()
+       op.execute(f"DROP INDEX IF EXISTS {schema}index_name")
    ```
 
-   **Dialect-only migrations.** If a change genuinely doesn't apply to one
-   dialect (e.g. enabling `pg_trgm` is PG-only), omit the unused slot:
-   ```python
-   def upgrade() -> None:
-       run_for_dialect(pg=_pg_upgrade)  # oracle slot intentionally absent → no-op
-   ```
-   Make the asymmetry deliberate. Don't leave an Oracle slot empty just because
-   you didn't think about it — copy-pasting a PG migration without the Oracle
-   half is exactly how schemas drift.
-
 3. **Run migrations locally**:
    ```bash
    # Set database URL and run migrations for the base schema plus all tenants
@@ -216,46 +178,10 @@ migration file dispatches through `run_for_dialect`, which calls either
 ./scripts/hooks/lint.sh
 ```
 
-Dead-code detection runs in CI (the `check-unused-code` job) at two levels:
-- **Blocking:** unused imports (ruff `F401`) and variables (`F841`) — `lint.sh` auto-removes
-  them and `verify-generated-files` fails on any leftover diff; and **knip** for orphaned
-  control-plane files / unused (or unlisted) `package.json` dependencies.
-- **Advisory:** whole unused Python functions (vulture) and unused control-plane *exports*
-  (the shadcn/ui surface is kept on purpose) — surfaced, not gated.
-
-Run both locally with:
-```bash
-./scripts/hooks/check-unused.sh
-```
-
 **After completing any implementation work, run `/code-review`** to verify your changes against project standards (missing tests, dead code, type safety, etc.). Fix any "must fix" issues before considering the task done.
 
 **MANDATORY: Run `/code-review` before pushing code or creating a pull request.** Do not push or create a PR until all "must fix" issues are resolved.
 
-### Testing
-
-Most tests are deterministic (MockLLM, pure functions) — assert directly.
-
-**Tests that verify LLM behaviour use a real LLM + an LLM-as-judge.** When the thing under test is *how the model interprets a prompt* (classification, attribution, dimension preservation, instruction-following), MockLLM can't simulate it and exact string/enum asserts flake across providers and runs. Use this pattern instead:
-
-1. Mark the test module `pytestmark = pytest.mark.hs_llm_core` (single-provider; CI runs it in the core-LLM job). Use `hs_llm_mat` only for provider-matrix acceptance tests.
-2. Call the real pipeline (`LLMConfig.from_env()`, `_get_raw_config()`), e.g. `extract_facts_from_text(...)`.
-3. Assert with the judge, not string matching:
-   ```python
-   from tests.llm_judge import assert_meets_criteria
-   facts_summary = "\n".join(f"- [{f.fact_type}] {f.fact}" for f in facts)
-   await assert_meets_criteria(
-       response=facts_summary,
-       criteria="The first-person user statements are classified 'world' and attributed to the user, not the agent.",
-       context="What the input said and who was speaking.",
-   )
-   ```
-
-Rules of thumb:
-- **Judge anything non-deterministic** — including `fact_type` classification and speaker attribution. Do NOT hard-assert `fact_type == "..."`; pass a `[fact_type] fact` summary to the judge instead. Structural facts that ARE deterministic (counts, presence of a field, that a substring was injected into a prompt) stay as direct asserts in fast unit tests.
-- **Split the test surface**: cover the deterministic mechanics (prompt assembly, suppression logic) with fast non-LLM unit tests, and the model-following behaviour with one `hs_llm_core` judge test. (Example pair: `test_narrator_resolution.py` + `test_narrator_context_override.py`.)
-- The judge model is independent of the test provider (defaults to Gemini); never judge with the same call you're testing.
-
 ### Memory Banks
 - Each bank is an isolated memory store (like a "brain" for one user/agent)
 - Banks have dispositions (skepticism, literalism, empathy traits 1-5) affecting reflect
@@ -286,35 +212,6 @@ When adding or modifying parameters in the dataplane API (hindsight-api), you mu
    - Update the client type definition in `lib/api.ts`
    - Update any UI components that need to use the new parameter
 
-### Harness Attribution (which coding agent wrote a document)
-
-`hindsight-integrations/hindsight-coding-agents/` stamps the coding agent on every
-document it retains, so the control plane can show its logo instead of another
-`key=value` chip:
-
-- `metadata.harness = "<id>"` — the authoritative field
-- tag `harness:<id>` — the same value, so the documents list can filter on it
-
-The ids are defined by that integration's HookSpecs
-(`src/harness/hook-lifecycle.ts`) plus the persistent-plugin entrypoints
-registered in `src/harness/registry.ts`, whose id is their
-`createPluginEntry(...)` argument — currently `antigravity-cli`, `claude-code`,
-`cline-cli`, `codex`, `copilot-cli`, `cursor-cli`, `devin-cli`, `grok-build`,
-`kilo`, `opencode`.
-
-The control plane resolves the value in
-`hindsight-control-plane/src/lib/harness-logo.ts` (metadata wins over the tag) and
-renders it with `components/ui/harness-logo.tsx` in the documents table and the
-document detail dialog. **Adding a harness to the integration means adding it to
-that registry in the same change**: copy its icon from
-`hindsight-docs/static/img/icons/` (or take it from the agent's own brand assets
-when the docs site carries none) into
-`hindsight-control-plane/public/img/harness/` and add one entry. Don't register
-ids nothing writes — a test asserts the registry matches the emitted set, plus an
-explicit list of retired ids kept so already-retained documents keep their logo.
-An unregistered harness is not an error: it renders no logo and still shows as
-ordinary metadata.
-
 ### Adding New Integrations
 
 Every new integration in `hindsight-integrations/` must satisfy all of the following before it can be merged:
@@ -356,10 +253,7 @@ Fields must be categorized as either **hierarchical** (can be overridden per-ten
    ```
 
 2. **main.py** (`hindsight-api-slim/hindsight_api/main.py`):
-   - No change is needed for ordinary environment-backed config fields. The CLI starts from `_get_raw_config()`,
-     so new `HindsightConfig` fields are carried through automatically.
-   - If the new field should be overridable by a CLI flag, add the argparse option in `_parse_cli_args()` and include
-     that field in the `dataclasses.replace(config, ...)` call near the "CLI override" comment.
+   - Add field to the manual `HindsightConfig()` constructor call (search for "CLI override")
 
 3. **Use hierarchical config in MemoryEngine**:
    ```python
@@ -379,16 +273,6 @@ Fields must be categorized as either **hierarchical** (can be overridden per-ten
    - Add to appropriate section table with Variable, Description, Default
    - Mark if it's hierarchical (can be overridden per-bank)
 
-6. **Env template** (`.env.example`):
-   - Add the variable to the appropriate section, commented if optional, with a
-     short inline comment describing it (mirror the documentation entry).
-   - This file is the single source of truth for the env template:
-     `scripts/dev/setup.sh` copies it to `.env`, and `hindsight-embed` ships a
-     bundled copy (`hindsight-embed/hindsight_embed/env.example`) that seeds
-     embed/profile configs. After editing `.env.example`, re-copy it to the
-     embed package (`cp .env.example hindsight-embed/hindsight_embed/env.example`)
-     or the `test_bundled_template_matches_repo_root` sync test will fail.
-
 #### Hierarchical vs Static Guidelines
 
 **Hierarchical** (per-bank overridable):
@@ -405,7 +289,7 @@ Fields must be categorized as either **hierarchical** (can be overridden per-ten
 
 ```bash
 cp .env.example .env
-# Edit .env with the LLM provider/model and credentials for your setup
+# Edit .env with LLM API key
 
 # Python deps
 uv sync --directory hindsight-api-slim/
@@ -414,10 +298,10 @@ uv sync --directory hindsight-api-slim/
 npm install
 ```
 
-Common LLM settings:
+Required env vars:
 - `HINDSIGHT_API_LLM_PROVIDER`: openai, anthropic, gemini, groq, minimax, ollama, lmstudio
-- `HINDSIGHT_API_LLM_API_KEY`: API key for providers that require one
-- `HINDSIGHT_API_LLM_MODEL`: Model name (defaults are provider-specific)
+- `HINDSIGHT_API_LLM_API_KEY`: Your API key
+- `HINDSIGHT_API_LLM_MODEL`: Model name (e.g., gpt-4o-mini, claude-sonnet-4-20250514)
 
 Optional (uses local models by default):
 - `HINDSIGHT_API_EMBEDDINGS_PROVIDER`: local (default) or tei

CONTRIBUTING.md

@@ -9,36 +9,13 @@ Thanks for your interest in contributing to Hindsight!
    git clone [email protected]:vectorize-io/hindsight.git
    cd hindsight
    ```
-
-2. Bootstrap your dev environment in one shot:
-   ```bash
-   ./scripts/dev/setup.sh
-   ```
-   This is idempotent (safe to re-run) and gets you ready to develop, including
-   offline. It:
-   - installs the required toolchains if missing (uv/Python, Node/npm, Rust/cargo),
-   - creates `.env` from `.env.example` (remember to add your LLM API key),
-   - configures git hooks,
-   - installs all Python and Node workspace dependencies,
-   - pre-downloads the local ML models + tokenizer so the API runs offline,
-   - builds the TypeScript SDK and the Rust CLI.
-
-   Useful flags: `--skip-build` (deps only), `--skip-models` (skip ML model
-   download), `--with-docs` (also build the docs site), `--force` (rebuild
-   artifacts). Docker image builds are out of scope. Run
-   `./scripts/dev/setup.sh --help` for details.
-
-### Manual setup
-
-If you'd rather set things up by hand instead of running the script above:
-
-1. Set up your environment:
+2. Set up your environment:
    ```bash
    cp .env.example .env
    ```
    Edit the .env to add LLM API key and config as required
 
-2. Install dependencies:
+3. Install dependencies:
    ```bash
    # Python dependencies
    uv sync --directory hindsight-api/

README.md

@@ -2,14 +2,14 @@
 
 ![Hindsight Banner](./hindsight-docs/static/img/hindsight-github-banner.png)
 
-[Documentation](https://hindsight.vectorize.io) • [Integrations](https://hindsight.vectorize.io/integrations) • [Cookbook](https://hindsight.vectorize.io/cookbook) • [Benchmarks](https://benchmarks.hindsight.vectorize.io/) • [Paper](https://arxiv.org/abs/2512.12818) • [Hindsight Cloud](https://ui.hindsight.vectorize.io/signup)
+[Documentation](https://hindsight.vectorize.io) • [Paper](https://arxiv.org/abs/2512.12818) • [Cookbook](https://hindsight.vectorize.io/cookbook) • [Hindsight Cloud](https://ui.hindsight.vectorize.io/signup)
 
-[![Release](https://github.com/vectorize-io/hindsight/actions/workflows/release.yml/badge.svg)](https://github.com/vectorize-io/hindsight/actions/workflows/release.yml)
-[![Version](https://img.shields.io/pypi/v/hindsight-api?logo=python&logoColor=white&label=version&color=blue)](https://pypi.org/project/hindsight-api/)
-[![PyPI Downloads](https://img.shields.io/pypi/dm/hindsight-client?logo=pypi&logoColor=white&label=PyPI&color=blue)](https://pypi.org/project/hindsight-client/)
-[![NPM Downloads](https://img.shields.io/npm/dm/%40vectorize-io%2Fhindsight-client?logo=npm&logoColor=white&label=NPM&color=blue)](https://www.npmjs.com/package/@vectorize-io/hindsight-client)
+[![CI](https://github.com/vectorize-io/hindsight/actions/workflows/release.yml/badge.svg)](https://github.com/vectorize-io/hindsight/actions/workflows/release.yml)
 [![Slack Community](https://img.shields.io/badge/Slack-Join%20Community-4A154B?logo=slack)](https://join.slack.com/t/hindsight-space/shared_invite/zt-3nhbm4w29-LeSJ5Ixi6j8PdiYOCPlOgg)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![gitcgr](https://gitcgr.com/badge/vectorize-io/hindsight.svg)](https://gitcgr.com/vectorize-io/hindsight)
+![PyPI - Downloads](https://img.shields.io/pypi/dm/hindsight-api?label=PyPI)
+![NPM Downloads](https://img.shields.io/npm/dm/%40vectorize-io%2Fhindsight-client?logoColor=orange&label=NPM&color=blue&link=https%3A%2F%2Fwww.npmjs.com%2Fpackage%2F%40vectorize-io%2Fhindsight-client)
 <br/>
 
 <a href="https://trendshift.io/repositories/15603" target="_blank"><img src="https://trendshift.io/api/badge/repositories/15603" alt="vectorize-io%2Fhindsight | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
@@ -21,33 +21,28 @@
 
 Hindsight™ is an agent memory system built to create smarter agents that learn over time. Most agent memory systems focus on recalling conversation history. Hindsight is focused on making agents that learn, not just remember.
 
+
 <video src="https://github.com/user-attachments/assets/923b798d-3581-4897-bb62-9cfa5a931682" controls></video>
 
 It eliminates the shortcomings of alternative techniques such as RAG and knowledge graph and delivers state-of-the-art performance on long term memory tasks.
 
-**Contents**
-
-- [Memory Performance & Accuracy](#memory-performance--accuracy)
-- [Quick Start](#quick-start) — [server](#1-start-a-server) · [clients](#2-connect-a-client) · [platforms](#supported-platforms) · [embedded](#python-embedded-no-server-required)
-- [Adding Hindsight to Your Agent](#adding-hindsight-to-your-agent) — [LLM Wrapper](#llm-wrapper-2-lines-of-code) · [integrations](#integrations) · [coding agents](#coding-agents) · [MCP](#mcp-server)
-- [Core Concepts](#core-concepts) — [memory types](#memory-types) · [retain / recall / reflect](#the-three-operations) · [observations](#observations) · [mental models & knowledge pages](#mental-models--knowledge-pages) · [banks](#memory-banks)
-- [Use Cases](#use-cases)
-- [Running in Production](#running-in-production)
-- [Resources](#resources)
-
----
-
 ## Memory Performance & Accuracy
 
 Hindsight is the most accurate agent memory system ever tested according to benchmark performance. It has achieved state-of-the-art performance on the LongMemEval benchmark, widely used to assess memory system performance across a variety of conversational AI scenarios. The current reported performance of Hindsight and other agent memory solutions as of January 2026 is shown here:
 
-![Overview](./hindsight-docs/static/img/hindsight-benchmarks.png)
-
-> Live, continuously updated results — including per-model accuracy, latency and cost — are published at [benchmarks.hindsight.vectorize.io](https://benchmarks.hindsight.vectorize.io/).
+![Overview](./hindsight-docs/static/img/hindsight-bench.jpg)
 
 The benchmark performance data for Hindsight has been independently reproduced by research collaborators at the Virginia Tech [Sanghani Center for Artificial Intelligence and Data Analytics](https://sanghani.cs.vt.edu/) and The Washington Post. Other scores are self-reported by software vendors.
 
-Hindsight is being used in production at Fortune 500 enterprises and by a growing number of AI startups.
+Hindsight is being used in production at Fortune 500 enterprises and by a growing number of AI startups. 
+
+## Adding Hindsight to Your AI Agents
+
+The easiest way to use Hindsight with an existing agent is with the LLM Wrapper. You can add memory to your agent with 2 lines of code. That will swap your current LLM client out with the Hindsight wrapper. After that, memories will be stored and retrieved automatically as you make LLM calls.
+
+If you need more control over how and when your agent stores and recalls memories, there's also a simple API you can integrate with using the SDKs or directly via HTTP.
+
+![Hindsight Banner](./hindsight-docs/static/img/migration-code.png)
 
 ---
 
@@ -59,70 +54,46 @@ Hindsight is being used in production at Fortune 500 enterprises and by a growin
 
 ---
 
+
 ## Quick Start
 
-### 1. Start a server
-
-#### Docker (recommended)
+### Docker (recommended)
 
 ```bash
 export OPENAI_API_KEY=sk-xxx
 
-docker run -it --pull always --name hindsight --restart unless-stopped -p 8888:8888 -p 9999:9999 \
+docker run --rm -it --pull always -p 8888:8888 -p 9999:9999 \
   -e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \
-  -v hindsight-data:/home/hindsight/.pg0 \
+  -v $HOME/.hindsight-docker:/home/hindsight/.pg0 \
   ghcr.io/vectorize-io/hindsight:latest
 ```
 
 >API: http://localhost:8888
 >UI: http://localhost:9999
 
-Hindsight works with **25+ LLM providers** via `HINDSIGHT_API_LLM_PROVIDER` — hosted (`openai`, `anthropic`, `gemini`, `groq`, `bedrock`, `vertexai`, `minimax`, `deepseek`, `atlas`, …), fully local (`ollama`, `lmstudio`, `llamacpp`), any OpenAI-compatible endpoint, and gateways (`litellm`, `litellmrouter`) that reach the rest. Existing subscriptions work too: `openai-codex` (ChatGPT Plus/Pro) and `claude-code` (Claude Pro/Max) need no API key. See [supported models](https://hindsight.vectorize.io/developer/models).
+You can modify the LLM provider by setting `HINDSIGHT_API_LLM_PROVIDER`. Valid options are `openai`, `anthropic`, `gemini`, `groq`, `ollama`, `lmstudio`, and `minimax`. The documentation provides more details on [supported models](https://hindsight.vectorize.io/developer/models).
 
-#### Docker (external PostgreSQL)
+
+
+### Docker (external PostgreSQL)
 
 ```bash
 export OPENAI_API_KEY=sk-xxx
 export HINDSIGHT_DB_PASSWORD=choose-a-password
 cd docker/docker-compose
-docker compose up
+docker compose up 
 ```
 
-> Oracle AI Database is also supported for enterprise deployments with full feature parity. See the [storage documentation](https://hindsight.vectorize.io/developer/storage) for details.
 
-#### Bare metal (pip)
+>API: http://localhost:8888
+>UI: http://localhost:9999
+
+### Client
 
 ```bash
-pip install hindsight-api
-export HINDSIGHT_API_LLM_API_KEY=sk-xxx
-
-hindsight-api
-```
-
-#### Kubernetes (Helm)
-
-```bash
-helm install hindsight oci://ghcr.io/vectorize-io/charts/hindsight \
-  --set api.llm.provider=openai \
-  --set api.llm.apiKey=sk-xxx \
-  --set postgresql.enabled=true
-```
-
-#### Managed (no server)
-
-[Hindsight Cloud](https://vectorize.io/pricing) is the hosted option: managed infrastructure that scales automatically, plus a dashboard, backups, team collaboration and a 99.9% uptime SLA. Billing is usage-based with free credits to start — no fixed monthly or per-seat fee. Point any client at `https://api.hindsight.vectorize.io` with your API key and skip the deployment entirely.
-
-[Compare self-hosted, Cloud and Enterprise →](https://vectorize.io/pricing) · [Sign up →](https://ui.hindsight.vectorize.io/signup)
-
-All options, including Windows and air-gapped setups, are covered in the [installation guide](https://hindsight.vectorize.io/developer/installation).
-
-### 2. Connect a client
-
-```bash
-pip install hindsight-client -U                                  # Python
-npm install @vectorize-io/hindsight-client                        # Node.js / TypeScript
-go get github.com/vectorize-io/hindsight/hindsight-clients/go     # Go
-curl -fsSL https://hindsight.vectorize.io/get-cli | bash          # CLI
+pip install hindsight-client -U
+# or
+npm install @vectorize-io/hindsight-client
 ```
 
 #### Python
@@ -144,6 +115,10 @@ client.reflect(bank_id="my-bank", query="Tell me about Alice")
 
 #### Node.js / TypeScript
 
+```bash
+npm install @vectorize-io/hindsight-client
+```
+
 ```javascript
 const { HindsightClient } = require('@vectorize-io/hindsight-client');
 
@@ -159,18 +134,6 @@ const main = async () => {
 main();
 ```
 
-Full reference: [Python](https://hindsight.vectorize.io/sdks/python) · [Node.js](https://hindsight.vectorize.io/sdks/nodejs) · [Go](https://hindsight.vectorize.io/sdks/go) · [CLI](https://hindsight.vectorize.io/sdks/cli) · [REST API](https://hindsight.vectorize.io/api-reference)
-
-### Supported Platforms
-
-| Platform | Docker | Bare Metal (pip) | Embedded DB (pg0) |
-|----------|--------|------------------|--------------------|
-| **Linux** (x86_64, ARM64) | ✅ | ✅ | ✅ |
-| **macOS** (Apple Silicon / arm64) | ✅ | ✅ | ✅ |
-| **macOS** (Intel / x86_64) | ✅ | ⚠️ | ✅ |
-| **Windows** (x86_64) | ✅ | ✅ | ✅ |
-
-⚠️ Intel Macs: use `hindsight-all-slim` — see the [installation guide](https://hindsight.vectorize.io/developer/installation#supported-platforms) for details.
 
 ### Python Embedded (no server required)
 
@@ -178,15 +141,13 @@ Full reference: [Python](https://hindsight.vectorize.io/sdks/python) · [Node.js
 pip install hindsight-all -U
 ```
 
-On Intel (x86_64) Macs, install `hindsight-all-slim` instead — see [Supported Platforms](#supported-platforms).
-
 ```python
 import os
 from hindsight import HindsightServer, HindsightClient
 
 with HindsightServer(
     llm_provider="openai",
-    llm_model="gpt-5-mini",
+    llm_model="gpt-5-mini", 
     llm_api_key=os.environ["OPENAI_API_KEY"]
 ) as server:
     client = HindsightClient(base_url=server.url)
@@ -194,182 +155,12 @@ with HindsightServer(
     results = client.recall(bank_id="my-bank", query="Where does Alice work?")
 ```
 
-A [Node.js equivalent](https://hindsight.vectorize.io/sdks/hindsight-all-npm) and a [daemon CLI](https://hindsight.vectorize.io/sdks/embed) are also available.
-
----
-
-## Adding Hindsight to Your Agent
-
-### LLM Wrapper (2 lines of code)
-
-The easiest way to add memory to an existing agent is the LLM Wrapper. Swap your LLM client for a wrapped one — memories are then stored and retrieved automatically on every call, with no other changes to your code.
-
-```bash
-pip install hindsight-litellm
-```
-
-```python
-from openai import OpenAI
-from hindsight_litellm import wrap_openai
-
-# Wrap your existing LLM client and you're done.
-# Defaults to Hindsight Cloud; pass hindsight_api_url for a self-hosted server.
-client = wrap_openai(
-    OpenAI(),
-    bank_id="user-123",
-    hindsight_api_url="http://localhost:8888",
-)
-
-# Hindsight recalls relevant memories before the call
-# and retains the conversation after it.
-response = client.chat.completions.create(
-    model="gpt-5-mini",
-    messages=[{"role": "user", "content": "What do you know about me?"}],
-)
-```
-
-`wrap_anthropic()` does the same for the Anthropic SDK, and every setting — bank, recall budget, fact types, reflect instead of recall — can be overridden per call with `hindsight_*` kwargs. LiteLLM sits underneath, so the same integration covers **100+ models**. See the [LiteLLM integration](https://hindsight.vectorize.io/sdks/integrations/litellm).
-
-If you need explicit control over *when* memories are stored and recalled, use the [SDKs or REST API](#2-connect-a-client) directly instead.
-
-### Integrations
-
-**60+ integrations** — most need no code changes.
-
-| | |
-|---|---|
-| **Coding agents** | [Claude Code](https://hindsight.vectorize.io/sdks/integrations/claude-code) · [Codex](https://hindsight.vectorize.io/sdks/integrations/codex) · [Cursor](https://hindsight.vectorize.io/sdks/integrations/cursor) · [GitHub Copilot](https://hindsight.vectorize.io/sdks/integrations/github-copilot) · [opencode](https://hindsight.vectorize.io/sdks/integrations/opencode) · [Cline](https://hindsight.vectorize.io/sdks/integrations/cline) · [Aider](https://hindsight.vectorize.io/sdks/integrations/aider) · [Zed](https://hindsight.vectorize.io/sdks/integrations/zed) · [Continue](https://hindsight.vectorize.io/sdks/integrations/continue) · [Roo Code](https://hindsight.vectorize.io/sdks/integrations/roo-code) · [OpenHands](https://hindsight.vectorize.io/sdks/integrations/openhands) |
-| **Agent frameworks** | [LangGraph / LangChain](https://hindsight.vectorize.io/sdks/integrations/langgraph) · [LlamaIndex](https://hindsight.vectorize.io/sdks/integrations/llamaindex) · [CrewAI](https://hindsight.vectorize.io/sdks/integrations/crewai) · [Pydantic AI](https://hindsight.vectorize.io/sdks/integrations/pydantic-ai) · [OpenAI Agents SDK](https://hindsight.vectorize.io/sdks/integrations/openai-agents) · [Google ADK](https://hindsight.vectorize.io/sdks/integrations/google-adk) · [Agno](https://hindsight.vectorize.io/sdks/integrations/agno) · [Strands](https://hindsight.vectorize.io/sdks/integrations/strands) · [AutoGen](https://hindsight.vectorize.io/sdks/integrations/autogen) · [Microsoft Agent Framework](https://hindsight.vectorize.io/sdks/integrations/agent-framework) · [Vercel AI SDK](https://hindsight.vectorize.io/sdks/integrations/ai-sdk) · [Haystack](https://hindsight.vectorize.io/sdks/integrations/haystack) |
-| **No-code / low-code** | [n8n](https://hindsight.vectorize.io/sdks/integrations/n8n) · [Zapier](https://hindsight.vectorize.io/sdks/integrations/zapier) · [Dify](https://hindsight.vectorize.io/sdks/integrations/dify) · [Flowise](https://hindsight.vectorize.io/sdks/integrations/flowise) |
-| **Apps & tools** | [ChatGPT](https://hindsight.vectorize.io/sdks/integrations/chatgpt) · [Perplexity](https://hindsight.vectorize.io/sdks/integrations/perplexity) · [Obsidian](https://hindsight.vectorize.io/sdks/integrations/obsidian) · [Pipecat](https://hindsight.vectorize.io/sdks/integrations/pipecat) · [Vapi](https://hindsight.vectorize.io/sdks/integrations/vapi) |
-
-👉 [**Browse all integrations**](https://hindsight.vectorize.io/integrations)
-
-### Coding Agents
-
-One package gives CLI coding agents long-term project memory: a per-repo bank built automatically from git history and past sessions, injected into the agent as it starts working, plus curated knowledge pages covering architecture, conventions and in-flight work.
-
-```bash
-npx @vectorize-io/hindsight-coding-agents install all          # every detected agent, wired natively
-npx @vectorize-io/hindsight-coding-agents install claude-code  # or just one
-```
-
-Supports Claude Code, Codex CLI, Cursor CLI, GitHub Copilot CLI, opencode, Kilo CLI, Cline CLI, Antigravity CLI, Devin CLI, Prime Agent, Grok Build and DeepSeek Harness. Ingestion is automatic — there is no setup command. See the [coding agents integration](https://hindsight.vectorize.io/sdks/integrations/coding-agents).
-
-### MCP Server
-
-Every server ships a built-in [Model Context Protocol](https://modelcontextprotocol.io/) endpoint, one per bank, enabled by default:
-
-```
-http://localhost:8888/mcp/{bank_id}/
-```
-
-Point any MCP client at it to expose retain, recall and reflect as tools. See the [MCP server docs](https://hindsight.vectorize.io/developer/mcp-server).
-
----
-
-## Core Concepts
-
-![Overview](./hindsight-docs/static/img/hindsight-overview.webp)
-
-### Memory Types
-
-Most agent memory implementations rely on basic vector search or sometimes use a knowledge graph. Hindsight uses biomimetic data structures to organize agent memories in a way that is more like how human memory works:
-
-- **World facts:** facts about the world ("The stove gets hot")
-- **Experiences:** the agent's own experiences ("I touched the stove and it really hurt")
-- **Observations:** consolidated, evidence-backed beliefs formed from many memories
-- **Mental models:** learned understanding of the agent's world, synthesized from observations and facts
-
-Memories live in **banks**. When memories are added, they are pushed into either the world facts or the experiences pathway, then represented as a combination of entities, relationships, and time series with sparse/dense vector representations to aid in later recall.
-
-### The Three Operations
-
-#### Retain
-
-The `retain` operation is used to push new memories into Hindsight. It tells Hindsight to _retain_ the information you pass in as an input.
-
-```python
-client.retain(
-    bank_id="my-bank",
-    content="Alice got promoted to senior engineer",
-    context="career update",
-    timestamp="2025-06-15T10:00:00Z",
-)
-```
-
-Behind the scenes, retain uses an LLM to extract key facts, temporal data, entities, and relationships. It passes these through a normalization process to transform extracted data into canonical entities, time series, and search indexes along with metadata. These representations create the pathways for accurate memory retrieval in the recall and reflect operations.
-
-![Retain Operation](hindsight-docs/static/img/retain-operation.webp)
-
-[Retain docs →](https://hindsight.vectorize.io/developer/retain)
-
-#### Recall
-
-The recall operation is used to retrieve memories. These memories can come from any of the memory types (world, experiences, etc.)
-
-```python
-client.recall(bank_id="my-bank", query="What does Alice do?")
-client.recall(bank_id="my-bank", query="What happened in June?")   # temporal
-```
-
-Recall performs 4 retrieval strategies in parallel:
-- Semantic: Vector similarity
-- Keyword: BM25 exact matching
-- Graph: Entity/temporal/causal links
-- Temporal: Time range filtering
-
-![Recall Operation](hindsight-docs/static/img/recall-operation.webp)
-
-The individual results are merged, ordered by relevance using reciprocal rank fusion and a cross-encoder reranking model, then trimmed as needed to fit within the token limit.
-
-[Recall docs →](https://hindsight.vectorize.io/developer/retrieval)
-
-#### Reflect
-
-The reflect operation performs a more thorough analysis of existing memories. This allows the agent to form new connections between memories and build a more thorough understanding of its world — or to answer a question that needs deep thinking rather than lookup.
-
-```python
-client.reflect(bank_id="my-bank", query="What should I know about Alice?")
-```
-
-For example, reflect supports use cases such as:
-
-- An **AI Project Manager** reflecting on what risks need to be mitigated on a project.
-- A **Sales Agent** reflecting on why certain outreach messages have gotten responses while others haven't.
-- A **Support Agent** reflecting on opportunities where customers have questions not answered by current product documentation.
-
-![Reflect Operation](hindsight-docs/static/img/reflect-operation.webp)
-
-[Reflect docs →](https://hindsight.vectorize.io/developer/reflect)
-
-### Observations
-
-Retained facts don't stay a flat pile. In the background, Hindsight consolidates related facts into **observations** — deduplicated beliefs the bank has built up over time. Each observation keeps its supporting evidence with exact quotes and a proof count, and is *refined* rather than overwritten when new evidence arrives, so new information strengthens, weakens or extends an existing belief instead of silently replacing it.
-
-[Observations docs →](https://hindsight.vectorize.io/developer/observations)
-
-### Mental Models & Knowledge Pages
-
-A **mental model** is a standing answer to a question about a bank ("What are this user's preferences?"). You define the question once; Hindsight writes the answer, stores it, and rewrites it in the background as the bank learns more. Reading one is a database read — no retrieval, no LLM call — so an agent can boot with a page of settled knowledge instead of rediscovering it every session.
-
-**Knowledge pages** are mental models with the mechanics hidden: living documents a bank writes about itself, organized in folders like a wiki, searchable, and projectable onto disk as ordinary markdown. Supply a name and a question; every other decision is a default you can override.
-
-[Mental models →](https://hindsight.vectorize.io/developer/mental-models) · [Knowledge pages →](https://hindsight.vectorize.io/developer/knowledge-pages)
-
-### Memory Banks
-
-A **bank** is an isolated memory store — one "brain" for one user, agent, or project. Isolation is strict: no cross-bank leakage. Banks carry background context and **disposition traits** (skepticism, literalism, empathy) that shape how reflect reasons over their memories, and can be created from declarative [bank templates](https://hindsight.vectorize.io/developer/api/bank-templates).
-
-Two more things worth knowing:
-
-- **Multilingual by default.** Input language is detected and preserved end to end — facts stay in their original language and entities keep their native script (张伟 stays 张伟, not "Zhang Wei"). [Docs →](https://hindsight.vectorize.io/developer/multilingual)
-- **Memory Defense.** An opt-in, per-bank policy that scans every retain for secrets and PII against 45 patterns and either redacts the match (`[REDACTED:github_token]`) or blocks the item before it reaches storage. [Docs →](https://hindsight.vectorize.io/developer/memory-defense)
 
 ---
 
 ## Use Cases
 
+
 Hindsight is built to support conversational AI agents as well as agents that are intended to perform tasks autonomously. The ideal use case for Hindsight are agents that require a blend of these features such as AI employees that need to handle open-ended tasks, change behavior based on user feedback, and learn to perform complex tasks to automate work at a level that approximates a human work. Hindsight can be used with simple AI workflows like those built with n8n and other similar tools, but may be overkill for such applications.
 
 ### Per-User Memories and Chat History
@@ -382,47 +173,129 @@ The requirements for this use case usually look something like this:
 
 <video src="https://github.com/user-attachments/assets/4805e8e1-e7d1-47c6-a4f8-2344a5ec8906" controls></video>
 
-Satisfying these requirements in Hindsight is straightforward. When new user inputs and tool calls are ingested into Hindsight using the retain operation, custom metadata can be used to enrich the new memories. Metadata provides a convenient way to isolate memories that need to be restricted to a given user. Once these are fed into the retain operation, any raw memories and mental models that get created can be filtered when retrieving relevant memories.
+Satisfying these requirements in Hindsight is straightforward. When new user inputs and tool calls are ingested into Hindsight using the retain operation, custom metadata can be used to enrich the new memories. Metadata provides a convenient way to isolate memories that need to be restricted to a given user. Once these are fed into the retain operation, any raw memories and mental models that get created can be filtered when retrieving relevant memories. 
 
 ![Per-User Memories](./hindsight-docs/static/img/per-user-memory-howto.png)
 
-More patterns in the [Cookbook](https://hindsight.vectorize.io/cookbook) and [Best Practices](https://hindsight.vectorize.io/best-practices).
-
 ---
 
-## Running in Production
+## Architecture & Operations
 
-| | |
-|---|---|
-| **Storage** | PostgreSQL + pgvector, or Oracle AI Database 23ai with full feature parity — [storage](https://hindsight.vectorize.io/developer/storage) |
-| **Configuration** | Hierarchical: global env vars → per-tenant → per-bank — [configuration](https://hindsight.vectorize.io/developer/configuration) |
-| **Monitoring** | Prometheus metrics and dashboards for LLM calls, tokens and latency — [monitoring](https://hindsight.vectorize.io/developer/monitoring) |
-| **Operations** | Admin CLI for migrations, bank repair and stuck operations — [admin CLI](https://hindsight.vectorize.io/developer/admin-cli) |
-| **Events** | Webhooks for retain, consolidation and refresh lifecycle events — [webhooks](https://hindsight.vectorize.io/developer/api/webhooks) |
-| **Extensibility** | Tenant, auth and storage extension points — [extensions](https://hindsight.vectorize.io/developer/extensions) |
-| **Managed** | Skip all of it with [Hindsight Cloud](https://vectorize.io/pricing) — managed, usage-based, 99.9% uptime SLA |
+![Overview](./hindsight-docs/static/img/hindsight-overview.webp)
+
+Most agent memory implementations rely on basic vector search or sometimes use a knowledge graph. Hindsight uses biomimetic data structures to organize agent memories in a way that is more like how human memory works:
+
+- **World:** Facts about the world ("The stove gets hot")
+- **Experiences:** Agent's own experiences ("I touched the stove and it really hurt")
+- **Mental Models:** Learned understanding of the agent's world formed by reflecting on raw memories and experiences.
+
+Memories in Hindsight are stored in banks (i.e. memory banks). When memories are added to Hindsight, they are pushed into either the world facts or experiences memory pathway. They are then represented as a combination of entities, relationships, and time series with sparse/dense vector representations to aid in later recall.
+
+Hindsight provides three simple methods to interact with the system:
+
+- **Retain:** Provide information to Hindsight that you want it to remember
+- **Recall:** Retrieve memories from Hindsight
+- **Reflect:** Reflect on memories and experiences to generate new observations and insights from existing memories.
+
+### Retain
+
+The `retain` operation is used to push new memories into Hindsight. It tells Hindsight to _retain_ the information you pass in as an input.
+
+```python
+from hindsight_client import Hindsight
+
+client = Hindsight(base_url="http://localhost:8888")
+
+# Simple
+client.retain(
+    bank_id="my-bank",
+    content="Alice works at Google as a software engineer"
+)
+
+# With context and timestamp
+client.retain(
+    bank_id="my-bank",
+    content="Alice got promoted to senior engineer",
+    context="career update",
+    timestamp="2025-06-15T10:00:00Z"
+)
+```
+
+Behind the scenes, the retain operation uses an LLM to extract key facts, temporal data, entities, and relationships. It passes these through a normalization process to transform extracted data into canonical entities, time series, and search indexes along with metadata. These representations create the pathways for accurate memory retrieval in the recall and reflect operations. 
+
+![Retain Operation](hindsight-docs/static/img/retain-operation.webp)
+
+### Recall
+
+The recall operation is used to retrieve memories. These memories can come from any of the memory types (world, experiences, etc.)
+
+```python
+from hindsight_client import Hindsight
+
+client = Hindsight(base_url="http://localhost:8888")
+
+# Simple
+client.recall(bank_id="my-bank", query="What does Alice do?")
+
+# Temporal
+client.recall(bank_id="my-bank", query="What happened in June?")
+```
+
+Recall performs 4 retrieval strategies in parallel:
+- Semantic: Vector similarity
+- Keyword: BM25 exact matching
+- Graph: Entity/temporal/causal links
+- Temporal: Time range filtering
+
+![Retain Operation](hindsight-docs/static/img/recall-operation.webp)
+
+The individual results from the retrievals are merged, then ordered by relevance using reciprocal rank fusion and a cross-encoder reranking model.
+
+The final output is trimmed as needed to fit within the token limit.
+
+### Reflect
+
+The reflect operation is used to perform a more thorough analysis of existing memories. This allows the agent to form new connections between memories and build a more thorough understanding of its world.
+
+For example, the `reflect` operation can be used to support use cases such as:
+
+- An **AI Project Manager** reflecting on what risks need to be mitigated on a project.
+- A **Sales Agent** reflecting on why certain outreach messages have gotten responses while others haven't.
+- A **Support Agent** reflecting on opportunities where customers have questions not answered by current product documentation.
+
+The `reflect` operation can also be used to handle on-demand question answering or analysis which require more deep thinking.
+
+```python
+from hindsight_client import Hindsight
+
+client = Hindsight(base_url="http://localhost:8888")
+
+client.reflect(bank_id="my-bank", query="What should I know about Alice?")
+```
+
+![Retain Operation](hindsight-docs/static/img/reflect-operation.webp)
 
 ---
 
 ## Resources
 
-**Documentation:**
-- [Docs](https://hindsight.vectorize.io) · [FAQ](https://hindsight.vectorize.io/faq) · [Best Practices](https://hindsight.vectorize.io/best-practices) · [Cookbook](https://hindsight.vectorize.io/cookbook) · [Blog](https://hindsight.vectorize.io/blog)
-- [Paper](https://arxiv.org/abs/2512.12818) · [Benchmarks](https://benchmarks.hindsight.vectorize.io/) · [RAG vs Memory](https://hindsight.vectorize.io/developer/rag-vs-hindsight)
+**Documentation:** 
+- [https://hindsight.vectorize.io](https://hindsight.vectorize.io)
 
 **Clients:**
-- [Python](https://hindsight.vectorize.io/sdks/python) · [Node.js](https://hindsight.vectorize.io/sdks/nodejs) · [Go](https://hindsight.vectorize.io/sdks/go) · [CLI](https://hindsight.vectorize.io/sdks/cli) · [REST API](https://hindsight.vectorize.io/api-reference)
+- [Python](http://hindsight.vectorize.io/sdks/python)
+- [Node.js](http://hindsight.vectorize.io/sdks/nodejs)
+- [REST API](https://hindsight.vectorize.io/api-reference)
+- [CLI](https://hindsight.vectorize.io/sdks/cli)
 
 **Community:**
 - [Slack](https://join.slack.com/t/hindsight-space/shared_invite/zt-3nhbm4w29-LeSJ5Ixi6j8PdiYOCPlOgg)
 - [GitHub Issues](https://github.com/vectorize-io/hindsight/issues)
 
 ---
-
 ## Star History
 
-[![Star history](https://raw.githubusercontent.com/vectorize-io/hindsight/main/.github/star-history/chart.svg)](https://github.com/vectorize-io/hindsight/stargazers)
-
+[![Star History Chart](https://api.star-history.com/svg?repos=vectorize-io/hindsight&type=date&legend=top-left)](https://www.star-history.com/#vectorize-io/hindsight&type=date&legend=top-left)
 ---
 
 ## Contributing

deno.lock

@@ -42,16 +42,6 @@
   },
   "workspace": {
     "members": {
-      "hindsight-all-npm": {
-        "packageJson": {
-          "dependencies": [
-            "npm:@types/node@22",
-            "npm:tsup@^8.5.1",
-            "npm:typescript@^5.7.0",
-            "npm:vitest@^4.1.2"
-          ]
-        }
-      },
       "hindsight-clients/typescript": {
         "packageJson": {
           "dependencies": [
@@ -68,7 +58,6 @@
       "hindsight-control-plane": {
         "packageJson": {
           "dependencies": [
-            "npm:@chenglou/pretext@^0.0.3",
             "npm:@eslint/eslintrc@^3.3.3",
             "npm:@eslint/js@^9.39.2",
             "npm:@radix-ui/react-alert-dialog@^1.1.15",
@@ -77,6 +66,7 @@
             "npm:@radix-ui/react-dropdown-menu@^2.1.16",
             "npm:@radix-ui/react-label@^2.1.8",
             "npm:@radix-ui/react-popover@^1.1.15",
+            "npm:@radix-ui/react-radio-group@^1.3.8",
             "npm:@radix-ui/react-select@^2.2.6",
             "npm:@radix-ui/react-slider@^1.3.6",
             "npm:@radix-ui/react-slot@^1.2.4",
@@ -101,12 +91,11 @@
             "npm:eslint@^9.39.1",
             "npm:[email protected]",
             "npm:next-themes@~0.4.6",
-            "npm:next@^16.1.7",
+            "npm:next@^16.1.6",
             "npm:postcss@^8.5.6",
             "npm:prettier@^3.7.4",
             "npm:react-chrono@^2.9.1",
             "npm:react-dom@^19.2.0",
-            "npm:react-is@^19.2.4",
             "npm:react-markdown@^10.1.0",
             "npm:react18-json-view@~0.2.9",
             "npm:react@^19.2.0",
@@ -144,26 +133,6 @@
             "npm:typescript@~5.6.2"
           ]
         }
-      },
-      "hindsight-tools/hindsight-agent-sdk": {
-        "packageJson": {
-          "dependencies": [
-            "npm:@vectorize-io/hindsight-client@~0.5.6",
-            "npm:typescript@^5.4.0",
-            "npm:vitest@^4.1.2"
-          ]
-        }
-      },
-      "hindsight-tools/self-driving-agents": {
-        "packageJson": {
-          "dependencies": [
-            "npm:@clack/prompts@^1.2.0",
-            "npm:@vectorize-io/hindsight-client@~0.5.6",
-            "npm:picocolors@^1.1.0",
-            "npm:typescript@^5.4.0",
-            "npm:vitest@^4.1.2"
-          ]
-        }
       }
     }
   }

docker/docker-compose/alloydb/docker-compose.yaml

@@ -1,90 +0,0 @@
-name: hindsight
-# Docker Compose file for Hindsight with AlloyDB Omni and ScaNN
-# Uses Google's free AlloyDB Omni container image: https://hub.docker.com/r/google/alloydbomni
-#
-# Usage:
-# docker compose -f docker/docker-compose/alloydb/docker-compose.yaml up -d
-#
-# Make sure to set the required environment variables before running:
-# - HINDSIGHT_DB_PASSWORD: password for the AlloyDB Omni/PostgreSQL user
-# - Configure LLM provider variables as needed (see the hindsight service below)
-#
-# Optional environment variables with defaults:
-# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
-# - HINDSIGHT_DB_VERSION: AlloyDB Omni image tag (default: 17)
-# - HINDSIGHT_DB_USER: database user (default: hindsight_user)
-# - HINDSIGHT_DB_NAME: database name (default: hindsight_db)
-
-services:
-  db:
-    image: google/alloydbomni:${HINDSIGHT_DB_VERSION:-17}
-    container_name: hindsight-db-alloydb
-    restart: always
-    ports:
-      - "5438:5432"
-    environment:
-      POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
-      POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-      POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
-    volumes:
-      - alloydb_data:/var/lib/postgresql/data
-    networks:
-      - hindsight-net
-
-  alloydb-init:
-    image: google/alloydbomni:${HINDSIGHT_DB_VERSION:-17}
-    depends_on:
-      - db
-    environment:
-      - PGPASSWORD=${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-    command:
-      - bash
-      - -c
-      - |
-        echo 'Waiting for AlloyDB Omni to be ready...'
-        until pg_isready -h hindsight-db-alloydb -p 5432 -U ${HINDSIGHT_DB_USER:-hindsight_user}; do
-          echo 'AlloyDB Omni is unavailable - sleeping'
-          sleep 2
-        done
-        echo 'AlloyDB Omni is ready - creating ${HINDSIGHT_DB_NAME:-hindsight_db} database'
-        psql -h hindsight-db-alloydb -p 5432 -U ${HINDSIGHT_DB_USER:-hindsight_user} -c 'CREATE DATABASE ${HINDSIGHT_DB_NAME:-hindsight_db};' 2>/dev/null || echo 'Database already exists'
-        echo 'Creating vector and alloydb_scann extensions'
-        psql -h hindsight-db-alloydb -p 5432 -U ${HINDSIGHT_DB_USER:-hindsight_user} -d ${HINDSIGHT_DB_NAME:-hindsight_db} -c 'CREATE EXTENSION IF NOT EXISTS vector;'
-        psql -h hindsight-db-alloydb -p 5432 -U ${HINDSIGHT_DB_USER:-hindsight_user} -d ${HINDSIGHT_DB_NAME:-hindsight_db} -c 'CREATE EXTENSION IF NOT EXISTS alloydb_scann CASCADE;'
-        echo 'Database and extensions created successfully'
-    restart: "no"
-    networks:
-      - hindsight-net
-
-  hindsight:
-    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
-    container_name: hindsight-app
-    ports:
-      - "8888:8888"
-      - "9999:9999"
-    environment:
-      # LLM Configuration
-      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
-      HINDSIGHT_API_LLM_API_KEY: ${HINDSIGHT_API_LLM_API_KEY:-}
-
-      # Database Configuration
-      HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
-
-      # Vector and Text Search Extensions
-      HINDSIGHT_API_VECTOR_EXTENSION: scann
-      HINDSIGHT_API_TEXT_SEARCH_EXTENSION: native
-
-    depends_on:
-      db:
-        condition: service_started
-      alloydb-init:
-        condition: service_completed_successfully
-    networks:
-      - hindsight-net
-
-networks:
-  hindsight-net:
-    driver: bridge
-
-volumes:
-  alloydb_data:

docker/docker-compose/claude-code/README.md

@@ -1,113 +0,0 @@
-# Hindsight with Claude Code (Claude Pro/Max subscription)
-
-Run Hindsight inside Docker using the `claude-code` LLM provider, backed by
-your host machine's Claude Pro or Max subscription credentials.
-
-The standalone Hindsight Docker image ships `claude-agent-sdk` but does **not**
-bundle the host `claude` CLI binary or any Claude credentials. This Compose
-file bind-mounts the host's CLI install and credentials into the container so
-the `claude-code` provider works without an API key.
-
-## When to use this
-
-- You have an active Claude Pro or Max subscription and want to use it for
-  Hindsight without paying separate Anthropic API costs.
-- You want a one-command `docker compose up` instead of a long `docker run`
-  invocation with many flags.
-- You are running on **Linux/amd64** — macOS Docker Desktop and Windows host
-  paths differ and are not yet covered (please open an issue if you'd like to
-  contribute a verified recipe for either).
-
-> **Personal-use only.**  Anthropic's
-> [Agent SDK documentation](https://docs.claude.com/en/api/agent-sdk/overview)
-> states that third-party developers should not offer claude.ai login or rate
-> limits for their products. Hindsight does **not** perform any login on your
-> behalf — it uses credentials you've already authenticated via
-> `claude auth login`. In January 2026, Anthropic
-> [enforced restrictions](https://paddo.dev/blog/anthropic-walled-garden-crackdown/)
-> against tools that spoofed the Claude Code client identity; Hindsight uses
-> the official Claude Agent SDK instead.
->
-> Do not deploy this configuration to shared environments or production. For
-> that, use the `anthropic` provider with an API key from the
-> [Anthropic Console](https://console.anthropic.com/). Usage counts against
-> your Claude Pro/Max subscription limits.
-
-## Prerequisites
-
-- Host has `claude` CLI installed (e.g., `npm install -g @anthropics/claude-code`)
-  and `claude auth login` has been run successfully.
-- `~/.claude.json` and `~/.claude/.credentials.json` exist on the host.
-- Host `claude` CLI version is **2.1.128 or newer** — the version bundled with
-  `claude-agent-sdk` 0.5.x has a protocol incompatibility in containers, so
-  the recipe overrides it with the host binary.
-
-## Quick start
-
-```bash
-# Set your host UID/GID (defaults to 1000:1000 if unset)
-export HOST_UID=$(id -u)
-export HOST_GID=$(id -g)
-
-docker compose -f docker/docker-compose/claude-code/docker-compose.yaml up -d
-```
-
-- API: http://localhost:8888
-- Control Plane: http://localhost:9999
-
-## Post-setup (one-time)
-
-After the container starts for the first time, run these commands to fix
-permissions and symlink the host `claude` binary into `$PATH`:
-
-```bash
-# Make ~/.claude writable by your UID (the CLI writes session/project state)
-docker exec --user 0:0 hindsight-claude-code chown $(id -u):$(id -g) /home/hindsight/.claude
-docker exec --user 0:0 hindsight-claude-code chmod 755 /home/hindsight/.claude
-
-# Symlink the host claude binary into PATH
-docker exec --user 0:0 hindsight-claude-code \
-  ln -sf /home/hindsight/.local/share/claude/versions/2.1.128 /usr/local/bin/claude
-```
-
-If you set `CLAUDE_CLI_VERSION` to a version other than `2.1.128`, update the
-symlink path accordingly.
-
-## Notes on the bind-mount surface (every flag is load-bearing)
-
-- **Host `claude` binary required** — the image ships only `claude-agent-sdk`,
-  not the CLI itself.
-- **SDK bundled-binary override** — the override of
-  `claude_agent_sdk/_bundled/claude` works around a protocol issue in the
-  bundled v2.1.121 binary inside containers. Once `claude-agent-sdk` ships
-  with v2.1.128+ this override can be dropped. Set `CLAUDE_CLI_VERSION` to
-  match your installed version.
-- **Single-file credential mounts** — credentials are mounted as individual
-  `:ro` files rather than a whole-directory `:ro` mount of `~/.claude`,
-  because the CLI writes session/project state at runtime and a read-only
-  directory mount silently breaks it.
-- **`--user` / `user:`** — the `user: ${HOST_UID}:${HOST_GID}` pattern
-  requires `chmod 755 /home/hindsight`, which is built into the image since
-  v0.6.0 (see [#1481](https://github.com/vectorize-io/hindsight/issues/1481)).
-- **`~/.hindsight-docker` data directory** — the pg0 data bind mount must be
-  writable by your host UID (see
-  [#1483](https://github.com/vectorize-io/hindsight/issues/1483)).
-- **Verified** on `linux/amd64` against `ghcr.io/vectorize-io/hindsight:latest`
-  v0.5.6+.
-
-## Using a different Claude CLI version
-
-If your host has a `claude` version other than 2.1.128, set
-`CLAUDE_CLI_VERSION` before starting:
-
-```bash
-export CLAUDE_CLI_VERSION=2.2.0
-docker compose -f docker/docker-compose/claude-code/docker-compose.yaml up -d
-```
-
-Then update the post-setup symlink to match:
-
-```bash
-docker exec --user 0:0 hindsight-claude-code \
-  ln -sf /home/hindsight/.local/share/claude/versions/2.2.0 /usr/local/bin/claude
-```

docker/docker-compose/claude-code/docker-compose.yaml

@@ -1,44 +0,0 @@
-name: hindsight-claude-code
-# Run Hindsight with the claude-code LLM provider, using your host machine's
-# Claude Pro/Max subscription credentials.  Linux/amd64 only for now.
-#
-# Quick start:
-#   docker compose -f docker/docker-compose/claude-code/docker-compose.yaml up -d
-#
-# See README.md for prerequisites, post-setup steps, and important caveats.
-
-services:
-  hindsight:
-    image: ghcr.io/vectorize-io/hindsight:latest
-    container_name: hindsight-claude-code
-    user: "${HOST_UID:-1000}:${HOST_GID:-1000}"
-    ports:
-      - "127.0.0.1:8888:8888"
-      - "127.0.0.1:9999:9999"
-    environment:
-      HOME: /home/hindsight
-      USER: hindsight
-      LOGNAME: hindsight
-      PATH: /usr/local/bin:/usr/bin:/bin:/app/api/.venv/bin
-      HINDSIGHT_API_LLM_PROVIDER: claude-code
-    volumes:
-      # ── Persistent data ────────────────────────────────────────────
-      # Writable pg0 data directory.  Must be writable by HOST_UID.
-      - ${HOME:-.}/.hindsight-docker:/home/hindsight/.pg0
-
-      # ── Claude credentials (read-only, single-file mounts) ────────
-      # A whole-directory :ro mount of ~/.claude silently breaks the
-      # CLI, which writes session/project state at runtime — so we
-      # mount only the two credential files.
-      - ${HOME}/.claude/.credentials.json:/home/hindsight/.claude/.credentials.json:ro
-      - ${HOME}/.claude.json:/home/hindsight/.claude.json:ro
-
-      # ── Claude CLI install (read-only) ─────────────────────────────
-      - ${HOME}/.local/share/claude:/home/hindsight/.local/share/claude:ro
-
-      # ── SDK bundled-binary override ────────────────────────────────
-      # The claude-agent-sdk 0.5.x image bundles v2.1.121 which has a
-      # protocol incompatibility in containers.  Override it with the
-      # host's v2.1.128+ binary.  Drop this mount once claude-agent-sdk
-      # ships with v2.1.128+.
-      - ${HOME}/.local/share/claude/versions/${CLAUDE_CLI_VERSION:-2.1.128}:/app/api/.venv/lib/python3.11/site-packages/claude_agent_sdk/_bundled/claude:ro

docker/docker-compose/custom-models/Dockerfile

@@ -1,34 +0,0 @@
-# Example: custom Hindsight image with non-default local models baked in.
-#
-# Use this pattern in production when you run a non-default embedder or
-# reranker. Baking models into the image removes the runtime dependency on
-# HuggingFace and lets the container registry handle caching per node, so
-# you don't need a model-cache PVC.
-#
-# Built on top of the slim image so only the deps and models you actually
-# use end up in the final image.
-FROM ghcr.io/vectorize-io/hindsight:latest-slim
-
-# Install the local-ml deps required to load sentence-transformers /
-# cross-encoder models at runtime. Pinned ranges mirror hindsight-api-slim's
-# `local-ml` extra in hindsight-api-slim/pyproject.toml. Use `uv pip
-# install` against the image's venv explicitly: the slim image's venv was
-# created by `uv sync` and does not ship its own `pip`, so a bare
-# `pip install` would fall back to user site-packages and not be visible
-# to the runtime python.
-RUN uv pip install --python /app/api/.venv/bin/python --no-cache \
-    'sentence-transformers>=5.0.0' \
-    'transformers>=4.53.0' \
-    'torch>=2.6.0'
-
-# Pre-download the models you want to use. Replace these with your own.
-# The defaults bundled in the full image are BAAI/bge-small-en-v1.5 and
-# cross-encoder/ms-marco-MiniLM-L-6-v2; here we pick multilingual variants
-# as a concrete non-default example.
-ARG EMBEDDER=sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
-ARG RERANKER=cross-encoder/mmarco-mMiniLMv2-L12-H384-v1
-ENV HF_HUB_DOWNLOAD_TIMEOUT=600
-RUN python -c "\
-from sentence_transformers import SentenceTransformer, CrossEncoder; \
-SentenceTransformer('${EMBEDDER}'); \
-CrossEncoder('${RERANKER}')"

docker/docker-compose/custom-models/README.md

@@ -1,81 +0,0 @@
-# Hindsight with Custom Local Models
-
-Example Docker Compose setup that builds a Hindsight image with **non-default
-local embedder and reranker models baked in at build time**.
-
-This is the recommended pattern for production when you use a non-default
-local model: the container registry caches model layers per node, pod
-startup is deterministic, and you don't need a model-cache PVC (or any
-runtime dependency on HuggingFace).
-
-## When to use this
-
-- You override `HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL` or
-  `HINDSIGHT_API_RERANKER_LOCAL_MODEL` to a non-default model.
-- You want pod startup to be deterministic and offline-capable.
-- You'd otherwise reach for a Helm `modelCache` PVC just to avoid
-  re-downloading models.
-
-If you're using the **default** local models, the published full image
-(`ghcr.io/vectorize-io/hindsight:latest`) already bakes them in — you don't
-need this example.
-
-If you're using **external** providers (TEI, OpenAI, Cohere, ...) for
-embeddings and reranking, use the slim image directly — no models are
-needed in the image.
-
-## Quick start
-
-```bash
-export HINDSIGHT_API_LLM_API_KEY=sk-xxx
-
-docker compose -f docker/docker-compose/custom-models/docker-compose.yaml up --build
-```
-
-- API: http://localhost:8888
-- Control Plane: http://localhost:9999
-
-## Using your own models
-
-Override the build args to bake different models:
-
-```bash
-docker compose -f docker/docker-compose/custom-models/docker-compose.yaml build \
-  --build-arg EMBEDDER=your-org/your-embedder \
-  --build-arg RERANKER=your-org/your-reranker
-```
-
-Then update the matching `HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL` and
-`HINDSIGHT_API_RERANKER_LOCAL_MODEL` values in `docker-compose.yaml` so the
-runtime points at the same model IDs.
-
-## Verifying the models are baked in
-
-`docker-compose.yaml` sets `HF_HUB_OFFLINE=1` and `TRANSFORMERS_OFFLINE=1`
-so that any attempt to download a model at runtime fails loudly instead of
-silently re-downloading. If the container starts and serves recall queries
-with these set, the models are correctly baked in.
-
-You can also inspect the image directly:
-
-```bash
-docker run --rm --entrypoint sh hindsight-custom-models-hindsight \
-  -c 'ls ~/.cache/huggingface/hub/'
-```
-
-## Why not a model-cache PVC?
-
-The Helm chart exposes an optional `api.persistence.modelCache` PVC for
-caching downloaded models across pod restarts. Compared to baking models
-into the image:
-
-- A PVC adds storage cost — one PVC per worker replica with
-  `volumeClaimTemplates`.
-- `ReadWriteOnce` (the default) pins pods to a node.
-- The PVC needs lifecycle management on `helm uninstall` / `helm upgrade`
-  — without `helm.sh/resource-policy: keep` it is deleted on uninstall;
-  with it, storage keeps billing forever until manually cleaned up.
-- Pod startup still depends on HuggingFace being reachable on first run.
-
-Image layers, by contrast, are pulled once per node and cached for free by
-the container runtime, with no orphaned-storage cleanup story.

docker/docker-compose/custom-models/docker-compose.yaml

@@ -1,45 +0,0 @@
-name: hindsight-custom-models
-# Example: run a custom Hindsight image with non-default local models baked
-# in at build time, so pod startup does not depend on HuggingFace at runtime.
-#
-# Quick start:
-#   export HINDSIGHT_API_LLM_API_KEY=sk-xxx
-#   docker compose -f docker/docker-compose/custom-models/docker-compose.yaml up --build
-#
-# Required environment variables:
-# - HINDSIGHT_API_LLM_API_KEY (pair it with HINDSIGHT_API_LLM_PROVIDER to use
-#   a provider other than the default openai)
-
-services:
-  hindsight:
-    build:
-      context: .
-      dockerfile: Dockerfile
-      # Override at build time to bake different models:
-      #   docker compose build --build-arg EMBEDDER=your-org/your-embedder
-      args:
-        EMBEDDER: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
-        RERANKER: cross-encoder/mmarco-mMiniLMv2-L12-H384-v1
-    container_name: hindsight-custom-models
-    ports:
-      - "8888:8888"
-      - "9999:9999"
-    environment:
-      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
-      HINDSIGHT_API_LLM_API_KEY: ${HINDSIGHT_API_LLM_API_KEY:-}
-
-      # Point Hindsight at the models baked into the image above.
-      HINDSIGHT_API_EMBEDDINGS_PROVIDER: local
-      HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
-      HINDSIGHT_API_RERANKER_PROVIDER: local
-      HINDSIGHT_API_RERANKER_LOCAL_MODEL: cross-encoder/mmarco-mMiniLMv2-L12-H384-v1
-
-      # Fail fast if a model is missing from the image instead of silently
-      # falling back to a HuggingFace download at runtime.
-      HF_HUB_OFFLINE: "1"
-      TRANSFORMERS_OFFLINE: "1"
-    volumes:
-      - pg_data:/home/hindsight/.pg0
-
-volumes:
-  pg_data:

docker/docker-compose/external-pg/docker-compose.yaml

@@ -39,7 +39,7 @@ services:
       - "8888:8888"
       - "9999:9999"
     environment:
-      - HINDSIGHT_API_LLM_API_KEY=${HINDSIGHT_API_LLM_API_KEY:?Please set the HINDSIGHT_API_LLM_API_KEY env variable}
+      - HINDSIGHT_API_LLM_API_KEY=${OPENAI_API_KEY?Please set the OPENAI_API_KEY env variable}
       - HINDSIGHT_API_DATABASE_URL=postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:?Please set the HINDSIGHT_DB_PASSWORD env variable}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
     depends_on:
       - db

docker/docker-compose/local-llm/README.md

@@ -1,103 +0,0 @@
-# Hindsight with a local llama.cpp server sidecar
-
-Example Docker Compose setup that runs Hindsight against a **local
-llama.cpp server**, fully offline, with no external API key required.
-
-## Architecture
-
-```
-┌────────────┐  HTTP /v1/chat/completions  ┌──────────────────────────────┐
-│ hindsight  │ ──────────────────────────▶ │ llama.cpp server (sidecar)   │
-│ (API + CP) │                             │ ghcr.io/ggml-org/llama.cpp   │
-└────────────┘                             └──────────────────────────────┘
-```
-
-`llama.cpp` runs as its own container and exposes an OpenAI-compatible
-HTTP API. Hindsight talks to it via the standard `openai` LLM provider
-with `HINDSIGHT_API_LLM_BASE_URL` pointed at the sidecar.
-
-This pattern follows
-[*Hosting llama-server with Docker* (ServiceStack)](https://servicestack.net/posts/hosting-llama-server).
-
-### Why a sidecar and not the in-process `llamacpp` provider?
-
-Hindsight does ship an in-process `llamacpp` provider that spawns
-`llama-cpp-python`, but the **published `ghcr.io/vectorize-io/hindsight`
-image deliberately omits `llama-cpp-python`** to keep the image small and
-avoid bundling native inference libraries that most users don't need.
-Trying to set `HINDSIGHT_API_LLM_PROVIDER=llamacpp` against the published
-image fails with `ModuleNotFoundError: No module named 'llama_cpp'`.
-
-The sidecar approach side-steps that entirely: the official llama.cpp
-image is used as-is for inference, Hindsight is used as-is for memory.
-Clean separation, no derived images.
-
-## Quick start
-
-```bash
-docker compose -f docker/docker-compose/local-llm/docker-compose.yaml up
-```
-
-- API: http://localhost:8888
-- Control Plane: http://localhost:9999
-
-**First boot downloads ~3.5 GB** (Gemma 4 E2B Q4_K_M GGUF) into the
-`llama_models` named volume. Subsequent boots reuse it.
-
-Hindsight only starts after llama.cpp's `/health` endpoint reports
-healthy, so the API will appear "stuck" for a few minutes on the first
-run while the model downloads.
-
-## Using a different model
-
-Override the HuggingFace repo / file in `docker-compose.yaml`:
-
-```yaml
-environment:
-  LLAMA_ARG_HF_REPO: bartowski/Qwen2.5-7B-Instruct-GGUF
-  LLAMA_ARG_HF_FILE: Qwen2.5-7B-Instruct-Q4_K_M.gguf
-```
-
-Also update `HINDSIGHT_API_LLM_MODEL` on the `hindsight` service to a
-matching alias (the value is sent to llama-server as the OpenAI `model`
-field — llama-server is lenient about this but it shows up in logs).
-
-## GPU acceleration
-
-The default compose file targets CPU because not everyone has a GPU. On
-CPU, Gemma 4 E2B runs at ~2-3 tokens/sec — fine for a smoke test, but the
-retain pipeline (which makes several multi-hundred-token LLM calls per
-memory) will time out against Hindsight's default LLM timeout. **For any
-real use, run on a GPU.**
-
-### NVIDIA
-
-1. Switch the `llama` service image from `:server` to `:server-cuda`.
-2. Uncomment the `LLAMA_ARG_N_GPU_LAYERS: "999"` env var (offload all
-   layers to GPU).
-3. Uncomment the `deploy.resources.reservations.devices` block.
-4. Install the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
-   on the host.
-
-The compose file has all four spots marked with inline comments.
-
-### Apple Silicon / ROCm / Vulkan
-
-The official `ghcr.io/ggml-org/llama.cpp` image only ships CPU and CUDA
-variants. For Metal (Apple Silicon), ROCm (AMD), or Vulkan backends,
-build llama.cpp yourself with the appropriate flags and reference the
-image you build instead. Docker Desktop on macOS cannot pass through the
-host GPU to a Linux container in any case — for Apple Silicon, run
-llama-server directly on the host and only put Hindsight in Docker.
-
-## Caveats
-
-- llama.cpp's HTTP API is OpenAI-compatible but not 100% feature-parity.
-  Function/tool calling support depends on the chat template baked into
-  the GGUF; some retain/reflect flows may behave differently than against
-  a hosted OpenAI model.
-- Small GGUFs (~3 B params) are useful for smoke testing but will
-  underperform a hosted frontier model on retain quality. Use a larger
-  GGUF (7-13 B params) for production-quality memory.
-- The `llama_models` named volume persists the GGUF across `docker
-  compose down`/`up` so the model is downloaded once, not every restart.

docker/docker-compose/local-llm/docker-compose.yaml

@@ -1,74 +0,0 @@
-name: hindsight-local-llm
-# Example: run Hindsight against a local llama.cpp server sidecar — fully
-# offline, no external API key needed.
-#
-# Pattern follows https://servicestack.net/posts/hosting-llama-server :
-# llama.cpp runs as its own container exposing an OpenAI-compatible HTTP
-# API, and Hindsight talks to it via the `openai` LLM provider with a
-# custom `base_url`. This means we can use the published Hindsight image
-# unchanged — no derived Dockerfile, no `llama-cpp-python` install on top.
-#
-# Quick start:
-#   docker compose -f docker/docker-compose/local-llm/docker-compose.yaml up
-#
-# First boot downloads the default Gemma 4 E2B GGUF (~3.5 GB) into the
-# `llama_models` volume; subsequent boots reuse it.
-
-services:
-  llama:
-    image: ghcr.io/ggml-org/llama.cpp:server
-    container_name: hindsight-local-llm-llama
-    environment:
-      LLAMA_ARG_HOST: 0.0.0.0
-      LLAMA_ARG_PORT: "8080"
-      # Auto-download a small GGUF from HuggingFace on first start.
-      # Override these to use a different model.
-      LLAMA_ARG_HF_REPO: bartowski/google_gemma-4-E2B-it-GGUF
-      LLAMA_ARG_HF_FILE: google_gemma-4-E2B-it-Q4_K_M.gguf
-      LLAMA_ARG_CTX_SIZE: "8192"
-      # Uncomment for NVIDIA GPU (and switch image to :server-cuda):
-      # LLAMA_ARG_N_GPU_LAYERS: "999"
-    volumes:
-      # llama-server stores HuggingFace downloads under ~/.cache/huggingface
-      # (not ~/.cache/llama.cpp), so mount the named volume there to avoid
-      # re-downloading the GGUF on every recreate.
-      - llama_models:/root/.cache/huggingface
-    healthcheck:
-      test: ["CMD-SHELL", "curl -fsS http://localhost:8080/health || exit 1"]
-      interval: 10s
-      timeout: 5s
-      retries: 60
-      start_period: 30s
-    # For NVIDIA GPU acceleration, swap the image above to
-    # `ghcr.io/ggml-org/llama.cpp:server-cuda` and uncomment:
-    # deploy:
-    #   resources:
-    #     reservations:
-    #       devices:
-    #         - driver: nvidia
-    #           count: all
-    #           capabilities: [gpu]
-
-  hindsight:
-    image: ghcr.io/vectorize-io/hindsight:latest
-    container_name: hindsight-local-llm
-    depends_on:
-      llama:
-        condition: service_healthy
-    ports:
-      - "8888:8888"
-      - "9999:9999"
-    environment:
-      # llama-server is OpenAI-compatible, so use the `openai` provider and
-      # point base_url at the sidecar. The API key is unused by llama-server
-      # but Hindsight requires the env var to be set.
-      HINDSIGHT_API_LLM_PROVIDER: openai
-      HINDSIGHT_API_LLM_BASE_URL: http://llama:8080/v1
-      HINDSIGHT_API_LLM_API_KEY: not-needed
-      HINDSIGHT_API_LLM_MODEL: gemma-4-e2b-it
-    volumes:
-      - pg_data:/home/hindsight/.pg0
-
-volumes:
-  pg_data:
-  llama_models:

docker/docker-compose/nginx/docker-compose.yml

@@ -51,8 +51,6 @@ services:
 
       # Control Plane config
       HINDSIGHT_CP_DATAPLANE_API_URL: http://localhost:8888
-      # Optional: Require a shared access key for Control Plane UI access
-      # HINDSIGHT_CP_ACCESS_KEY: your-secret-key
     volumes:
       # Persist embedded pg0 database
       - hindsight_data:/app/data

docker/docker-compose/pg_search/Dockerfile

@@ -1,7 +0,0 @@
-# PostgreSQL with pgvector and ParadeDB pg_search extensions.
-#
-# The official ParadeDB image ships PostgreSQL with pg_search and pgvector
-# already installed, so no build steps are required. We pin to the PG17
-# variant for parity with the other Hindsight docker-compose examples
-# (vchord, pg_textsearch).
-FROM paradedb/paradedb:latest-pg17

docker/docker-compose/pg_search/docker-compose.yaml

@@ -1,96 +0,0 @@
-name: hindsight
-# Docker Compose file for Hindsight with PostgreSQL and ParadeDB pg_search.
-#
-# pg_search is the only BM25 backend supported by Hindsight that works with
-# Citus, so this is the recommended setup for horizontally scaled deployments.
-#
-# Usage:
-#   docker compose -f docker/docker-compose/pg_search/docker-compose.yaml up -d
-#
-# Required environment variables:
-# - HINDSIGHT_DB_PASSWORD: Password for the PostgreSQL user
-# - Configure LLM provider variables as needed (see the hindsight service)
-#
-# Optional environment variables with defaults:
-# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
-# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
-# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
-# - HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER: ParadeDB pg_search
-#   tokenizer for new BM25 indexes (default: empty, uses ParadeDB default)
-
-services:
-  db:
-    # Use ParadeDB image which bundles pgvector + pg_search
-    build:
-      context: .
-      dockerfile: Dockerfile
-    container_name: hindsight-db
-    restart: always
-    ports:
-      - "5437:5432"
-    environment:
-      POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
-      POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-      POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
-    volumes:
-      - pg_data:/var/lib/postgresql/data
-    networks:
-      - hindsight-net
-
-  pg-search-init:
-    build:
-      context: .
-      dockerfile: Dockerfile
-    depends_on:
-      - db
-    environment:
-      - PGPASSWORD=${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-    command: >
-      bash -c "
-        echo 'Waiting for PostgreSQL to be ready...';
-        until pg_isready -h hindsight-db -p 5432 -U hindsight_user; do
-          echo 'PostgreSQL is unavailable - sleeping';
-          sleep 2;
-        done;
-        echo 'PostgreSQL is ready - creating hindsight_db database';
-        psql -h hindsight-db -p 5432 -U hindsight_user -c 'CREATE DATABASE hindsight_db;' 2>/dev/null || echo 'Database already exists';
-        echo 'Creating extensions in hindsight_db database';
-        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vector CASCADE;';
-        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS pg_search CASCADE;';
-        echo 'Database and extensions created successfully';
-      "
-    restart: "no"
-    networks:
-      - hindsight-net
-
-  hindsight:
-    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
-    container_name: hindsight-app
-    ports:
-      - "8888:8888"
-      - "9999:9999"
-    environment:
-      # LLM Configuration
-      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
-      HINDSIGHT_API_LLM_API_KEY: ${HINDSIGHT_API_LLM_API_KEY:-}
-
-      # Database Configuration
-      HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
-
-      # Vector and Text Search Extensions
-      HINDSIGHT_API_VECTOR_EXTENSION: pgvector
-      HINDSIGHT_API_TEXT_SEARCH_EXTENSION: pg_search
-      HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER: ${HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER:-}
-
-    depends_on:
-      - db
-    networks:
-      - hindsight-net
-
-
-networks:
-  hindsight-net:
-    driver: bridge
-
-volumes:
-  pg_data:

docker/docker-compose/pg_textsearch/docker-compose.yaml

@@ -68,7 +68,7 @@ services:
     environment:
       # LLM Configuration
       HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
-      HINDSIGHT_API_LLM_API_KEY: ${HINDSIGHT_API_LLM_API_KEY:-}
+      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
 
       # Database Configuration
       HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}

docker/docker-compose/pgroonga/Dockerfile

@@ -1,13 +0,0 @@
-# PostgreSQL with pgvector and pgroonga extensions.
-#
-# pgroonga is a multilingual full-text search extension built on Groonga.
-# It works out of the box for CJK (Chinese, Japanese, Korean) and other
-# non-whitespace-segmented languages via the TokenBigram tokenizer.
-FROM groonga/pgroonga:4.0.8-debian-17
-
-# Install pgvector on top of the pgroonga base image (which already provides
-# pgroonga, the Groonga library, and the PostgreSQL PGDG package repository).
-RUN apt-get update && apt-get install -y --no-install-recommends \
-    postgresql-17-pgvector=0.8.6-1.pgdg13+1 \
-    && apt-get clean \
-    && rm -rf /var/lib/apt/lists/*

docker/docker-compose/pgroonga/docker-compose.yaml

@@ -1,91 +0,0 @@
-name: hindsight
-# Docker Compose file for Hindsight with PostgreSQL and pgroonga
-#
-# pgroonga provides multilingual BM25 indexing that works out of the box for
-# CJK (Chinese, Japanese, Korean) and other non-whitespace-segmented languages.
-# Use this recipe if your bank content is not English/European.
-#
-# docker compose -f docker/docker-compose/pgroonga/docker-compose.yaml down && \
-#   sleep 2 && \
-#   docker compose -f docker/docker-compose/pgroonga/docker-compose.yaml up -d
-#
-# Optional environment variables with defaults:
-# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
-# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
-# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
-# - HINDSIGHT_DB_PASSWORD: PostgreSQL password (default: hindsight_password)
-
-services:
-  db:
-    build:
-      context: .
-      dockerfile: Dockerfile
-    container_name: hindsight-db
-    restart: always
-    ports:
-      - "5439:5432"
-    environment:
-      POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
-      POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-      POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
-    volumes:
-      - pg_data:/var/lib/postgresql/data
-    networks:
-      - hindsight-net
-
-  pgroonga-init:
-    build:
-      context: .
-      dockerfile: Dockerfile
-    depends_on:
-      - db
-    environment:
-      - PGPASSWORD=${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-    command: >
-      bash -c "
-        echo 'Waiting for PostgreSQL to be ready...';
-        until pg_isready -h hindsight-db -p 5432 -U hindsight_user; do
-          echo 'PostgreSQL is unavailable - sleeping';
-          sleep 2;
-        done;
-        echo 'PostgreSQL is ready - creating hindsight_db database';
-        psql -h hindsight-db -p 5432 -U hindsight_user -c 'CREATE DATABASE hindsight_db;' 2>/dev/null || echo 'Database already exists';
-        echo 'Creating extensions in hindsight_db database';
-        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vector CASCADE;';
-        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS pgroonga CASCADE;';
-        echo 'Database and extensions created successfully';
-      "
-    restart: "no"
-    networks:
-      - hindsight-net
-
-  hindsight:
-    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
-    container_name: hindsight-app
-    ports:
-      - "8888:8888"
-      - "9999:9999"
-    environment:
-      # LLM Configuration
-      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
-      HINDSIGHT_API_LLM_API_KEY: ${HINDSIGHT_API_LLM_API_KEY:-}
-
-      # Database Configuration
-      HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
-
-      # Vector and Text Search Extensions
-      HINDSIGHT_API_VECTOR_EXTENSION: pgvector
-      HINDSIGHT_API_TEXT_SEARCH_EXTENSION: pgroonga
-
-    depends_on:
-      - db
-    networks:
-      - hindsight-net
-
-
-networks:
-  hindsight-net:
-    driver: bridge
-
-volumes:
-  pg_data:

docker/docker-compose/s3-file-storage/docker-compose.yaml

@@ -59,7 +59,7 @@ services:
       - "8888:8888"
       - "9999:9999"
     environment:
-      - HINDSIGHT_API_LLM_API_KEY=${HINDSIGHT_API_LLM_API_KEY:?Please set the HINDSIGHT_API_LLM_API_KEY env variable}
+      - HINDSIGHT_API_LLM_API_KEY=${OPENAI_API_KEY?Please set the OPENAI_API_KEY env variable}
       - HINDSIGHT_API_DATABASE_URL=postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:?Please set the HINDSIGHT_DB_PASSWORD env variable}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
       # S3 file storage configuration (SeaweedFS)
       - HINDSIGHT_API_FILE_STORAGE_TYPE=s3

docker/docker-compose/tei/README.md

@@ -1,107 +0,0 @@
-# Hindsight with TEI embeddings + reranker
-
-Example Docker Compose setup that serves **embeddings and reranking from two
-[HuggingFace Text Embeddings Inference (TEI)](https://github.com/huggingface/text-embeddings-inference)
-sidecars** instead of the in-process local models.
-
-Because embeddings and reranking run outside the API, Hindsight itself needs
-no baked-in models, so this uses the **slim** image
-(`ghcr.io/vectorize-io/hindsight:latest-slim`). Only the LLM — used for
-retain/recall/reflect — still needs a provider and API key.
-
-## When to use this
-
-- You want embeddings/reranking on a dedicated, independently scalable
-  inference server (e.g. a GPU node) rather than in the API process.
-- You run the **slim** image and pull embeddings/reranking from an external
-  service.
-- You want a self-hosted, offline-capable alternative to a cloud embeddings
-  provider (OpenAI, Cohere, ...).
-
-If you just want local models in-process, use the default full image — no
-sidecars required.
-
-## What it runs
-
-| Service         | Image                                                  | Model                                    |
-| --------------- | ------------------------------------------------------ | ---------------------------------------- |
-| `tei-embedding` | `ghcr.io/huggingface/text-embeddings-inference:cpu-1.8.3` | `BAAI/bge-small-en-v1.5` (384-dim)       |
-| `tei-reranker`  | `ghcr.io/huggingface/text-embeddings-inference:cpu-1.8.3` | `BAAI/bge-reranker-base`                 |
-| `hindsight`     | `ghcr.io/vectorize-io/hindsight:latest-slim`           | — (slim; talks to the sidecars)          |
-
-This is a prod-like configuration: the embedding model is Hindsight's default
-(`bge-small-en-v1.5`), the reranker is the `bge-reranker-base` cross-encoder
-commonly paired with it on dedicated inference servers, and both services carry
-throughput flags (`--max-concurrent-requests`, `--max-batch-tokens`,
-`--max-client-batch-size`) tuned for sustained multi-client load instead of
-TEI's bare defaults. The API points at the sidecars with:
-
-```
-HINDSIGHT_API_EMBEDDINGS_PROVIDER=tei
-HINDSIGHT_API_EMBEDDINGS_TEI_URL=http://tei-embedding:80
-HINDSIGHT_API_RERANKER_PROVIDER=tei
-HINDSIGHT_API_RERANKER_TEI_URL=http://tei-reranker:80
-```
-
-## Quick start
-
-```bash
-export HINDSIGHT_API_LLM_API_KEY=sk-xxx
-
-docker compose -f docker/docker-compose/tei/docker-compose.yaml up
-```
-
-- API: http://localhost:8888
-- Control Plane: http://localhost:9999
-- TEI embedding server: http://localhost:8080 (exposed for debugging)
-- TEI reranker server: http://localhost:8081 (exposed for debugging)
-
-`hindsight` waits (via `depends_on: service_healthy`) until both TEI servers
-report healthy, so the first boot pauses while each model downloads into its
-`tei_*_cache` volume. Subsequent boots reuse the cached models.
-
-To use an LLM provider other than the default `openai`:
-
-```bash
-export HINDSIGHT_API_LLM_PROVIDER=gemini
-export HINDSIGHT_API_LLM_API_KEY=...
-docker compose -f docker/docker-compose/tei/docker-compose.yaml up
-```
-
-## Using your own models
-
-Change the `--model-id` in each service's `command` to any TEI-supported
-model. The embedding dimension is **auto-detected from the server** and the
-pgvector schema is adjusted to match on first boot — no dimension env var to
-set. (If you switch the embedding model after data already exists, start from
-a fresh `pg_data` volume, since the stored vectors were built for the old
-dimension.)
-
-## Verifying the servers
-
-```bash
-# Health
-curl 127.0.0.1:8080/health && curl 127.0.0.1:8081/health
-
-# Embedding (returns a 384-length vector for the default model)
-curl 127.0.0.1:8080/embed -H 'content-type: application/json' \
-     -d '{"inputs":"hello world"}'
-
-# Rerank
-curl 127.0.0.1:8081/rerank -H 'content-type: application/json' \
-     -d '{"query":"what is the capital of France?","texts":["Paris is the capital of France.","Bananas are yellow."]}'
-```
-
-## Apple Silicon / arm64
-
-The `cpu-1.8.3` TEI images are published for `linux/amd64` only. On an
-Apple Silicon Mac, run under emulation:
-
-```bash
-export DOCKER_DEFAULT_PLATFORM=linux/amd64
-docker compose -f docker/docker-compose/tei/docker-compose.yaml up
-```
-
-Emulated startup is slow (model load takes a few minutes). For production,
-run on `amd64` hosts — or a GPU node with the CUDA-tagged TEI image and a GPU
-reservation.

docker/docker-compose/tei/docker-compose.yaml

@@ -1,113 +0,0 @@
-name: hindsight-tei
-# Example: run Hindsight with embeddings and reranking served by two
-# HuggingFace Text Embeddings Inference (TEI) sidecars instead of the
-# in-process local models.
-#
-# Because embeddings and reranking are external, Hindsight itself needs no
-# baked-in models — this uses the **slim** image
-# (`ghcr.io/vectorize-io/hindsight:latest-slim`). Only the LLM (used for
-# retain/recall/reflect) still needs a provider + API key.
-#
-# The two TEI services here run a prod-like configuration:
-# `BAAI/bge-small-en-v1.5` embeddings (384-dim, Hindsight's default) and the
-# `BAAI/bge-reranker-base` cross-encoder, with the batching/concurrency flags
-# tuned for sustained multi-client load rather than TEI's bare defaults. Swap
-# the `--model-id` args to serve any TEI-supported model — the embedding
-# dimension is auto-detected from the server, and the pgvector schema is
-# adjusted to match on first boot.
-#
-# Quick start:
-#   export HINDSIGHT_API_LLM_API_KEY=sk-xxx
-#   docker compose -f docker/docker-compose/tei/docker-compose.yaml up
-#
-# First boot downloads the two models into the `tei_*_cache` volumes;
-# subsequent boots reuse them.
-
-services:
-  tei-embedding:
-    image: ghcr.io/huggingface/text-embeddings-inference:cpu-1.8.3
-    container_name: hindsight-tei-embedding
-    # Prod-like tuning: high request concurrency with bounded batch sizes.
-    command:
-      [
-        "--model-id", "BAAI/bge-small-en-v1.5",
-        "--max-concurrent-requests", "512",
-        "--max-batch-tokens", "16384",
-        "--max-client-batch-size", "32",
-        "--auto-truncate",
-      ]
-    environment:
-      # TEI listens on port 80 inside the container by default.
-      PORT: "80"
-    ports:
-      # Exposed on the host so you can curl the server directly, e.g.
-      #   curl 127.0.0.1:8080/embed -H 'content-type: application/json' \
-      #        -d '{"inputs":"hello world"}'
-      - "8080:80"
-    volumes:
-      - tei_embedding_cache:/data
-    healthcheck:
-      # The TEI image ships curl; hit its /health endpoint so Hindsight only
-      # starts once the model is loaded and serving.
-      test: ["CMD", "curl", "-fsS", "http://localhost:80/health"]
-      interval: 10s
-      timeout: 5s
-      retries: 60
-      start_period: 30s
-
-  tei-reranker:
-    image: ghcr.io/huggingface/text-embeddings-inference:cpu-1.8.3
-    container_name: hindsight-tei-reranker
-    # Prod-like tuning: reranking batches are larger than embedding batches
-    # (rerank inputs are query+document pairs scored in bulk during recall).
-    command:
-      [
-        "--model-id", "BAAI/bge-reranker-base",
-        "--max-concurrent-requests", "512",
-        "--max-batch-tokens", "32768",
-        "--max-client-batch-size", "128",
-        "--auto-truncate",
-      ]
-    environment:
-      PORT: "80"
-    ports:
-      - "8081:80"
-    volumes:
-      - tei_reranker_cache:/data
-    healthcheck:
-      test: ["CMD", "curl", "-fsS", "http://localhost:80/health"]
-      interval: 10s
-      timeout: 5s
-      retries: 60
-      start_period: 30s
-
-  hindsight:
-    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest-slim}
-    container_name: hindsight-tei
-    depends_on:
-      tei-embedding:
-        condition: service_healthy
-      tei-reranker:
-        condition: service_healthy
-    ports:
-      - "8888:8888"
-      - "9999:9999"
-    environment:
-      # LLM still runs through a provider — bring your own key. Pair with
-      # HINDSIGHT_API_LLM_PROVIDER to use a provider other than openai.
-      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
-      HINDSIGHT_API_LLM_API_KEY: ${HINDSIGHT_API_LLM_API_KEY:-}
-
-      # Embeddings + reranking served by the TEI sidecars above. Use the
-      # in-cluster service DNS names, not localhost.
-      HINDSIGHT_API_EMBEDDINGS_PROVIDER: tei
-      HINDSIGHT_API_EMBEDDINGS_TEI_URL: http://tei-embedding:80
-      HINDSIGHT_API_RERANKER_PROVIDER: tei
-      HINDSIGHT_API_RERANKER_TEI_URL: http://tei-reranker:80
-    volumes:
-      - pg_data:/home/hindsight/.pg0
-
-volumes:
-  pg_data:
-  tei_embedding_cache:
-  tei_reranker_cache:

docker/docker-compose/timescale/.env.example

@@ -8,12 +8,17 @@ HINDSIGHT_VERSION=latest
 
 # LLM Configuration
 HINDSIGHT_API_LLM_PROVIDER=openai
-HINDSIGHT_API_LLM_API_KEY=your-openai-api-key-here
+OPENAI_API_KEY=your-openai-api-key-here
 
-# Alternative LLM providers (uncomment and set the key above accordingly):
+# Alternative LLM providers (uncomment and configure as needed):
 # HINDSIGHT_API_LLM_PROVIDER=anthropic
+# ANTHROPIC_API_KEY=your-anthropic-api-key
+
 # HINDSIGHT_API_LLM_PROVIDER=gemini
+# GEMINI_API_KEY=your-gemini-api-key
+
 # HINDSIGHT_API_LLM_PROVIDER=groq
+# GROQ_API_KEY=your-groq-api-key
 
 # Vector and Text Search (already configured in docker-compose.yaml)
 # HINDSIGHT_API_VECTOR_EXTENSION=pgvectorscale

docker/docker-compose/timescale/README.md

@@ -9,14 +9,14 @@ Both extensions are from [Timescale](https://github.com/timescale) and provide p
 ## Prerequisites
 
 - Docker and Docker Compose installed
-- An OpenAI API key (or a key for another LLM provider)
+- OpenAI API key (or another LLM provider)
 
 ## Quick Start
 
 ```bash
 # Set environment variables
 export HINDSIGHT_DB_PASSWORD="your-secure-password"
-export HINDSIGHT_API_LLM_API_KEY="your-openai-api-key"
+export OPENAI_API_KEY="your-openai-api-key"
 
 # Build and start
 docker compose -f docker/docker-compose/timescale/docker-compose.yaml up -d --build
@@ -50,7 +50,7 @@ docker compose -f docker/docker-compose/timescale/docker-compose.yaml down -v
 | `HINDSIGHT_DB_USER` | PostgreSQL username | `hindsight_user` |
 | `HINDSIGHT_DB_NAME` | Database name | `hindsight_db` |
 | `HINDSIGHT_VERSION` | Hindsight Docker image version | `latest` |
-| `HINDSIGHT_API_LLM_API_KEY` | API key for the LLM provider | (required) |
+| `OPENAI_API_KEY` | OpenAI API key | (required) |
 | `HINDSIGHT_API_LLM_PROVIDER` | LLM provider | `openai` |
 
 ### Why Timescale Extensions?

docker/docker-compose/timescale/docker-compose.yaml

@@ -8,8 +8,7 @@ name: hindsight
 #
 # Required environment variables:
 # - HINDSIGHT_DB_PASSWORD: Password for the PostgreSQL user
-# - HINDSIGHT_API_LLM_API_KEY (pair it with HINDSIGHT_API_LLM_PROVIDER to use
-#   a provider other than the default openai)
+# - OPENAI_API_KEY (or configure another LLM provider)
 #
 # Optional environment variables with defaults:
 # - HINDSIGHT_VERSION: Hindsight application version (default: latest)
@@ -81,7 +80,7 @@ services:
     environment:
       # LLM Configuration
       HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
-      HINDSIGHT_API_LLM_API_KEY: ${HINDSIGHT_API_LLM_API_KEY:-}
+      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
 
       # Database Configuration
       HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}

docker/docker-compose/vchord/docker-compose.yaml

@@ -1,6 +1,6 @@
 name: hindsight
 # Docker Compose file for Hindsight with PostgreSQL and vectorchord
-# docker compose -f docker/docker-compose/vchord/docker-compose.yaml down && sleep 2 && docker compose -f docker/docker-compose/vchord/docker-compose.yaml up -d
+# docker compose -f docker/docker-compose/docker-compose.yaml down && sleep 2 && docker compose -f docker/docker-compose/docker-compose.yaml up -d
 # Make sure to set the required environment variables before running:
 # - HINDSIGHT_DB_PASSWORD: Password for the PostgreSQL user
 # - Configure LLM provider variables as needed (see below in the hindsight service)
@@ -70,7 +70,7 @@ services:
       # LLM Configuration (uses OpenAI for testing vchord)
       # LLM configuration
       HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
-      HINDSIGHT_API_LLM_API_KEY: ${HINDSIGHT_API_LLM_API_KEY:-}
+      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
 
       # Database Configuration
       HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}

docker/standalone/Dockerfile

@@ -41,43 +41,26 @@ RUN apt-get update && apt-get install -y \
     && rm -rf /var/lib/apt/lists/* \
     && pip install --no-cache-dir uv
 
-# Copy the workspace lock and member metadata before source code so dependency
-# installation stays cacheable while matching the versions tested in CI.
-COPY pyproject.toml uv.lock ./
-COPY hindsight-all/pyproject.toml ./hindsight-all/
-COPY hindsight-api/pyproject.toml ./hindsight-api/
+# Copy dependency files and README (required by pyproject.toml)
 COPY hindsight-api-slim/pyproject.toml ./api/
 COPY hindsight-api-slim/README.md ./api/
-COPY hindsight-all-slim/pyproject.toml ./hindsight-all-slim/
-COPY hindsight-dev/pyproject.toml ./hindsight-dev/
-COPY hindsight-clients/python/pyproject.toml ./hindsight-clients/python/
-COPY hindsight-embed/pyproject.toml ./hindsight-embed/
-RUN ln -s api hindsight-api-slim
+
+WORKDIR /app/api
 
 # Sync dependencies using appropriate extras based on INCLUDE_LOCAL_MODELS
 # local-ml: torch, sentence-transformers, transformers, einops, flashrank, mlx (optional)
 # embedded-db: pg0-embedded (always included for embedded PostgreSQL support)
-# ONNX Runtime embeddings are intentionally not bundled into the official
-# standalone image; install the local-onnx extra in custom images when needed.
-ENV UV_PROJECT_ENVIRONMENT=/app/api/.venv
 RUN if [ "$INCLUDE_LOCAL_MODELS" = "true" ]; then \
-        uv sync --locked --package hindsight-api-slim --no-install-package hindsight-api-slim --extra local-ml --extra embedded-db; \
+        uv sync --extra local-ml --extra embedded-db; \
     else \
-        uv sync --locked --package hindsight-api-slim --no-install-package hindsight-api-slim --extra embedded-db; \
+        uv sync --extra embedded-db; \
     fi
 
 # Copy source code (alembic migrations are inside hindsight_api/)
-WORKDIR /app/api
 COPY hindsight-api-slim/hindsight_api ./hindsight_api
 
-# Install the local package from the same validated lock after source is present.
-WORKDIR /app
-RUN if [ "$INCLUDE_LOCAL_MODELS" = "true" ]; then \
-        uv sync --locked --package hindsight-api-slim --extra local-ml --extra embedded-db; \
-    else \
-        uv sync --locked --package hindsight-api-slim --extra embedded-db; \
-    fi \
-    && uv pip check --python /app/api/.venv/bin/python
+# Install the local package (uv sync only installed dependencies, not the package itself)
+RUN uv pip install -e .
 
 # =============================================================================
 # Stage: SDK Builder (needed for Control Plane)
@@ -160,8 +143,6 @@ FROM python:3.11-slim AS api-only
 WORKDIR /app
 
 # Note: libicu version varies by Debian version - try common versions in order
-# Runtime images use uv directly; remove pip build tooling after installation so
-# vulnerable setuptools-vendored packages and wheel are not shipped in production.
 RUN apt-get update && apt-get install -y \
     curl \
     procps \
@@ -171,8 +152,7 @@ RUN apt-get update && apt-get install -y \
     libossp-uuid16 \
     && (apt-get install -y libicu72 2>/dev/null || apt-get install -y libicu74 2>/dev/null || apt-get install -y libicu76 2>/dev/null || true) \
     && rm -rf /var/lib/apt/lists/* \
-    && pip install --no-cache-dir uv \
-    && pip uninstall --yes setuptools wheel
+    && pip install --no-cache-dir uv
 
 RUN useradd -m -s /bin/bash hindsight
 
@@ -192,10 +172,6 @@ USER hindsight
 # "Permission denied" error when mounting a fresh root-owned volume.
 RUN mkdir -p /home/hindsight/.pg0
 
-# Make /home/hindsight traversable when running with --user UID:GID overrides
-# (default 0700 blocks traversal by non-owner UIDs needed for bind-mount ownership matching)
-RUN chmod 755 /home/hindsight
-
 ENV PATH="/app/api/.venv/bin:${PATH}"
 
 # Pre-download tiktoken encoding (ALWAYS - required for token counting even in air-gapped envs)
@@ -310,8 +286,6 @@ WORKDIR /app
 
 # Install Node.js, curl, uv, and system dependencies
 # Note: libicu version varies by Debian version - try common versions in order
-# Runtime images use uv directly; remove pip build tooling after installation so
-# vulnerable setuptools-vendored packages and wheel are not shipped in production.
 RUN apt-get update && apt-get install -y \
     curl \
     procps \
@@ -323,8 +297,7 @@ RUN apt-get update && apt-get install -y \
     && curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \
     && apt-get install -y nodejs \
     && rm -rf /var/lib/apt/lists/* \
-    && pip install --no-cache-dir uv \
-    && pip uninstall --yes setuptools wheel
+    && pip install --no-cache-dir uv
 
 RUN useradd -m -s /bin/bash hindsight
 
@@ -355,10 +328,6 @@ USER hindsight
 # "Permission denied" error when mounting a fresh root-owned volume.
 RUN mkdir -p /home/hindsight/.pg0
 
-# Make /home/hindsight traversable when running with --user UID:GID overrides
-# (default 0700 blocks traversal by non-owner UIDs needed for bind-mount ownership matching)
-RUN chmod 755 /home/hindsight
-
 ENV PATH="/app/api/.venv/bin:${PATH}"
 
 # Pre-download tiktoken encoding (ALWAYS - required for token counting even in air-gapped envs)

docker/standalone/start-all.sh

@@ -10,90 +10,19 @@ set -e
 # loss scenarios where a container restart caused the data directory to be
 # wiped despite a volume mount being present.
 # =============================================================================
-pg0_has_pg_version() {
-    local pg0_data_dir="$1"
-
-    # pg0 has used more than one on-disk layout. Newer standalone images keep
-    # PostgreSQL data under instances/<name>/data, while older volumes may have
-    # placed PG_VERSION at or one level below the mount.
-    [ -f "$pg0_data_dir/PG_VERSION" ] && return 0
-    compgen -G "$pg0_data_dir"/*/PG_VERSION > /dev/null 2>&1 && return 0
-    compgen -G "$pg0_data_dir"/instances/*/data/PG_VERSION > /dev/null 2>&1 && return 0
-
-    return 1
-}
-
-check_pg0_data_integrity() {
-    local pg0_data_dir="$1"
-
-    if [ ! -d "$pg0_data_dir" ]; then
-        return 0
-    fi
-
+PG0_DATA_DIR="${HOME}/.pg0"
+if [ -d "$PG0_DATA_DIR" ]; then
     # Look for actual PostgreSQL data directories (pg0 creates subdirs per instance)
-    if pg0_has_pg_version "$pg0_data_dir"; then
-        echo "✅ Existing pg0 data directory detected at $pg0_data_dir"
-    elif [ "$(ls -A "$pg0_data_dir" 2>/dev/null)" ]; then
-        echo "⚠️  WARNING: pg0 data directory exists at $pg0_data_dir but no PG_VERSION found."
+    if compgen -G "$PG0_DATA_DIR"/*/PG_VERSION > /dev/null 2>&1; then
+        echo "✅ Existing pg0 data directory detected at $PG0_DATA_DIR"
+    elif [ "$(ls -A "$PG0_DATA_DIR" 2>/dev/null)" ]; then
+        echo "⚠️  WARNING: pg0 data directory exists at $PG0_DATA_DIR but no PG_VERSION found."
         echo "   This may indicate data corruption or an incomplete previous shutdown."
         echo "   If you see all migrations running from scratch after this, your data may have been lost."
         echo "   See: https://github.com/vectorize-io/hindsight/issues/675"
     fi
-
-    return 0
-}
-
-# =============================================================================
-# Embedded pg0 writability pre-check (#1483)
-#
-# The container runs as the unprivileged `hindsight` user (UID 1000). When the
-# pg0 data directory is a host bind mount (e.g. `-v $HOME/dir:/home/hindsight/.pg0`)
-# that is not owned by UID 1000 — the default on macOS Docker Desktop and most
-# non-1000 Linux hosts — pg0 fails with the opaque "Permission denied (os error
-# 13)". We cannot chown it ourselves without root (and the image is deliberately
-# rootless), so we surface an actionable message up front instead.
-#
-# Docker *named* volumes are seeded with the image directory's ownership (UID
-# 1000) on first use, so they avoid this entirely — hence the named-volume
-# recommendation below and in the README.
-# =============================================================================
-check_pg0_writable() {
-    local pg0_data_dir="$1"
-
-    # Only relevant for embedded pg0; an external database doesn't use this dir.
-    if [ -n "${HINDSIGHT_API_DATABASE_URL:-}" ]; then
-        return 0
-    fi
-
-    mkdir -p "$pg0_data_dir" 2>/dev/null || true
-    if touch "$pg0_data_dir/.hindsight-write-test" 2>/dev/null; then
-        rm -f "$pg0_data_dir/.hindsight-write-test" 2>/dev/null || true
-        return 0
-    fi
-
-    echo "❌ The embedded database directory $pg0_data_dir is not writable by this container (UID $(id -u))."
-    echo ""
-    echo "   A host directory was bind-mounted but is not owned by the container user (UID 1000)."
-    echo "   Hindsight runs rootless and cannot fix this for you. Choose one:"
-    echo ""
-    echo "   • Recommended — use a Docker named volume (auto-owned by the container):"
-    echo "       -v hindsight-data:/home/hindsight/.pg0"
-    echo ""
-    echo "   • Or keep the host path and run as your host user, chowning it to match:"
-    echo "       sudo chown -R \$(id -u):\$(id -g) <host-directory>"
-    echo "       docker run --user \$(id -u):\$(id -g) -e HOME=/home/hindsight ..."
-    echo ""
-    echo "   See https://github.com/vectorize-io/hindsight/issues/1483"
-    return 1
-}
-
-if [ "${HINDSIGHT_START_ALL_SOURCE_ONLY:-false}" = "true" ]; then
-    return 0 2>/dev/null || exit 0
 fi
 
-check_pg0_data_integrity "${HOME}/.pg0"
-check_pg0_writable "${HOME}/.pg0" || exit 1
-
 # Service flags (default to true if not set)
 ENABLE_API="${HINDSIGHT_ENABLE_API:-true}"
 ENABLE_CP="${HINDSIGHT_ENABLE_CP:-true}"
@@ -227,7 +156,7 @@ PIDS=()
 # Start API if enabled
 if [ "$ENABLE_API" = "true" ]; then
     cd /app/api
-    API_HEALTH_URL="${HINDSIGHT_API_HEALTH_URL:-http://localhost:${HINDSIGHT_API_PORT:-8888}/health}"
+    API_HEALTH_URL="${HINDSIGHT_API_HEALTH_URL:-http://localhost:8888/health}"
     API_STARTUP_WAIT_SECONDS="${HINDSIGHT_API_STARTUP_WAIT_SECONDS:-300}"
 
     # Run API directly - Python's PYTHONUNBUFFERED=1 handles output buffering

docker/standalone/test-start-all.sh

@@ -1,121 +0,0 @@
-#!/bin/bash
-set -euo pipefail
-
-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-
-HINDSIGHT_START_ALL_SOURCE_ONLY=true
-source "$SCRIPT_DIR/start-all.sh"
-unset HINDSIGHT_START_ALL_SOURCE_ONLY
-
-TMP_DIR="$(mktemp -d)"
-trap 'chmod -R u+rwx "$TMP_DIR" 2>/dev/null || true; rm -rf "$TMP_DIR"' EXIT
-
-assert_contains() {
-    local output="$1"
-    local expected="$2"
-
-    if [[ "$output" != *"$expected"* ]]; then
-        echo "Expected output to contain: $expected"
-        echo "Actual output:"
-        echo "$output"
-        exit 1
-    fi
-}
-
-assert_not_contains() {
-    local output="$1"
-    local unexpected="$2"
-
-    if [[ "$output" == *"$unexpected"* ]]; then
-        echo "Expected output not to contain: $unexpected"
-        echo "Actual output:"
-        echo "$output"
-        exit 1
-    fi
-}
-
-assert_empty() {
-    local output="$1"
-
-    if [ -n "$output" ]; then
-        echo "Expected no output, got:"
-        echo "$output"
-        exit 1
-    fi
-}
-
-mkdir -p "$TMP_DIR/empty"
-assert_empty "$(check_pg0_data_integrity "$TMP_DIR/empty")"
-
-mkdir -p "$TMP_DIR/direct"
-touch "$TMP_DIR/direct/PG_VERSION"
-direct_output="$(check_pg0_data_integrity "$TMP_DIR/direct")"
-assert_contains "$direct_output" "Existing pg0 data directory detected"
-assert_not_contains "$direct_output" "WARNING"
-
-mkdir -p "$TMP_DIR/legacy/instance"
-touch "$TMP_DIR/legacy/instance/PG_VERSION"
-legacy_output="$(check_pg0_data_integrity "$TMP_DIR/legacy")"
-assert_contains "$legacy_output" "Existing pg0 data directory detected"
-assert_not_contains "$legacy_output" "WARNING"
-
-mkdir -p "$TMP_DIR/nested/instances/hindsight/data"
-touch "$TMP_DIR/nested/instances/hindsight/data/PG_VERSION"
-nested_output="$(check_pg0_data_integrity "$TMP_DIR/nested")"
-assert_contains "$nested_output" "Existing pg0 data directory detected"
-assert_not_contains "$nested_output" "WARNING"
-
-mkdir -p "$TMP_DIR/nonempty/instances/hindsight"
-touch "$TMP_DIR/nonempty/instances/hindsight/instance.json"
-nonempty_output="$(check_pg0_data_integrity "$TMP_DIR/nonempty")"
-assert_contains "$nonempty_output" "WARNING: pg0 data directory exists"
-
-echo "start-all pg0 integrity checks passed"
-
-# =============================================================================
-# check_pg0_writable (#1483)
-# These rely on filesystem permissions, which root bypasses; skip under root.
-# =============================================================================
-if [ "$(id -u)" != "0" ]; then
-    # Writable directory: returns 0, prints nothing, leaves no artifact behind.
-    mkdir -p "$TMP_DIR/writable"
-    writable_output="$(check_pg0_writable "$TMP_DIR/writable")"
-    assert_empty "$writable_output"
-    if [ -e "$TMP_DIR/writable/.hindsight-write-test" ]; then
-        echo "check_pg0_writable left its write-test file behind"
-        exit 1
-    fi
-
-    # Non-writable directory: returns 1 with actionable guidance.
-    mkdir -p "$TMP_DIR/readonly"
-    chmod 000 "$TMP_DIR/readonly"
-    set +e
-    readonly_output="$(check_pg0_writable "$TMP_DIR/readonly" 2>&1)"
-    readonly_rc=$?
-    set -e
-    chmod 755 "$TMP_DIR/readonly"
-    if [ "$readonly_rc" -eq 0 ]; then
-        echo "check_pg0_writable should fail on a non-writable directory"
-        exit 1
-    fi
-    assert_contains "$readonly_output" "not writable"
-    assert_contains "$readonly_output" "hindsight-data:/home/hindsight/.pg0"
-    assert_contains "$readonly_output" "--user"
-
-    # External database configured: skip the check regardless of dir perms.
-    mkdir -p "$TMP_DIR/extdb"
-    chmod 000 "$TMP_DIR/extdb"
-    set +e
-    HINDSIGHT_API_DATABASE_URL="postgres://x" check_pg0_writable "$TMP_DIR/extdb" >/dev/null 2>&1
-    extdb_rc=$?
-    set -e
-    chmod 755 "$TMP_DIR/extdb"
-    if [ "$extdb_rc" -ne 0 ]; then
-        echo "check_pg0_writable should skip when an external database is configured"
-        exit 1
-    fi
-
-    echo "start-all pg0 writability checks passed"
-else
-    echo "⚠️  Running as root; skipping pg0 writability checks (permissions are bypassed)."
-fi

helm/hindsight/Chart.lock

@@ -0,0 +1,6 @@
+dependencies:
+- name: postgresql
+  repository: https://charts.bitnami.com/bitnami
+  version: 15.5.38
+digest: sha256:f67c7612736803ece8a669f8ca6b0555f3b78557bc0ecb732aa2e43f0df7750d
+generated: "2025-12-10T17:20:57.058794+01:00"

helm/hindsight/Chart.yaml

@@ -2,8 +2,8 @@ apiVersion: v2
 name: hindsight
 description: Hindsight helm chart
 type: application
-version: 0.9.1
-appVersion: "0.9.1"
+version: 0.5.4
+appVersion: "0.5.4"
 keywords:
   - ai
   - memory

helm/hindsight/README.md

@@ -66,13 +66,13 @@ helm install hindsight ./helm/hindsight -n hindsight --create-namespace -f value
 
 | Parameter | Description | Default |
 |-----------|-------------|---------|
-| `version` | Default image tag for all components | Chart `appVersion` |
+| `version` | Default image tag for all components | `0.1.0` |
 | `api.enabled` | Enable the API component | `true` |
-| `api.image.repository` | API image repository | `ghcr.io/vectorize-io/hindsight-api` |
+| `api.image.repository` | API image repository | `hindsight/api` |
 | `api.image.tag` | API image tag (defaults to `version`) | - |
 | `api.service.port` | API service port | `8888` |
 | `controlPlane.enabled` | Enable the control plane | `true` |
-| `controlPlane.image.repository` | Control plane image repository | `ghcr.io/vectorize-io/hindsight-control-plane` |
+| `controlPlane.image.repository` | Control plane image repository | `hindsight/control-plane` |
 | `controlPlane.image.tag` | Control plane image tag (defaults to `version`) | - |
 | `controlPlane.service.port` | Control plane service port | `3000` |
 | `postgresql.enabled` | Deploy PostgreSQL as subchart | `true` |

helm/hindsight/templates/worker-statefulset.yaml

@@ -60,13 +60,13 @@ spec:
           valueFrom:
             fieldRef:
               fieldPath: metadata.name
-        {{- /* Explicitly set port to override K8s service discovery env var (HINDSIGHT_API_PORT) */}}
-        - name: HINDSIGHT_API_PORT
-          value: {{ .Values.worker.service.targetPort | quote }}
-        {{- /* Inherit LLM config from api.env, then apply worker-specific env.
-               Merge (worker.env wins) so a key set in both does not emit a
-               duplicate env entry, which server-side apply rejects. */}}
-        {{- range $key, $value := merge (deepCopy (.Values.worker.env | default dict)) (.Values.api.env | default dict) }}
+        {{- /* Inherit LLM config from api.env */}}
+        {{- range $key, $value := .Values.api.env }}
+        - name: {{ $key }}
+          value: {{ $value | quote }}
+        {{- end }}
+        {{- /* Worker-specific env vars */}}
+        {{- range $key, $value := .Values.worker.env }}
         - name: {{ $key }}
           value: {{ $value | quote }}
         {{- end }}

helm/hindsight/values.yaml

@@ -13,6 +13,9 @@
 #   - Any other env vars you want to inject
 # existingSecret: "my-hindsight-secret"
 
+# Global settings
+replicaCount: 1
+
 # Image settings for api
 api:
   enabled: true
@@ -36,22 +39,16 @@ api:
       cpu: 500m
       memory: 1Gi
 
-  # Liveness and readiness probes.
-  # Liveness uses /health/live, which performs no database access: a slow or
-  # unreachable database must gate traffic (readiness), never restart pods.
-  # Needs an image from this chart's appVersion or newer — older ones serve
-  # /health only, and would fail this probe with a 404.
+  # Liveness and readiness probes
   livenessProbe:
     httpGet:
-      path: /health/live
+      path: /health
       port: 8888
     initialDelaySeconds: 30
     periodSeconds: 10
     timeoutSeconds: 5
     failureThreshold: 3
 
-  # Readiness checks the database, so a pod that cannot reach it is pulled out
-  # of the Service and put back once the database recovers.
   readinessProbe:
     httpGet:
       path: /health
@@ -73,12 +70,6 @@ api:
   # Persistent volume for local model cache (reranker, embeddings)
   # Models are downloaded to /home/hindsight/.cache on first use.
   # Without persistence, models are re-downloaded on every pod restart.
-  #
-  # For production, prefer baking models into a custom image instead of
-  # enabling this PVC: image layers are pulled once per node and cached
-  # for free, while a PVC adds storage cost, pins pods to a node
-  # (ReadWriteOnce), and needs lifecycle management on uninstall/upgrade.
-  # See docs: developer/installation#bundling-custom-models-in-a-custom-image
   persistence:
     modelCache:
       enabled: false
@@ -137,15 +128,10 @@ worker:
       cpu: 500m
       memory: 1Gi
 
-  # Liveness and readiness probes.
-  # Liveness uses /health/live, which performs no database access. Restarting a
-  # worker whose database is merely slow requeues its claimed operations with
-  # retry_count incremented, so DB checks must stay out of liveness.
-  # Needs an image from this chart's appVersion or newer — older ones serve
-  # /health only, and would fail this probe with a 404.
+  # Liveness and readiness probes
   livenessProbe:
     httpGet:
-      path: /health/live
+      path: /health
       port: 8889
     initialDelaySeconds: 30
     periodSeconds: 10
@@ -182,10 +168,7 @@ worker:
   # affinity: {}
 
   # Persistent volume for local model cache (reranker, embeddings)
-  # Uses volumeClaimTemplates since worker is a StatefulSet — one PVC per
-  # replica. For production, prefer baking models into a custom image; see
-  # api.persistence.modelCache above and docs:
-  # developer/installation#bundling-custom-models-in-a-custom-image
+  # Uses volumeClaimTemplates since worker is a StatefulSet.
   persistence:
     modelCache:
       enabled: false

hindsight-all-npm/README.md

@@ -18,17 +18,17 @@ npm install @vectorize-io/hindsight-all @vectorize-io/hindsight-client
 ## Example
 
 ```ts
-import { HindsightServer, consoleLogger } from "@vectorize-io/hindsight-all";
-import { HindsightClient } from "@vectorize-io/hindsight-client";
+import { HindsightServer, consoleLogger } from '@vectorize-io/hindsight-all';
+import { HindsightClient } from '@vectorize-io/hindsight-client';
 
 const server = new HindsightServer({
-  profile: "my-app",
+  profile: 'my-app',
   port: 9077,
   env: {
-    HINDSIGHT_API_LLM_PROVIDER: "anthropic",
+    HINDSIGHT_API_LLM_PROVIDER: 'anthropic',
     HINDSIGHT_API_LLM_API_KEY: process.env.ANTHROPIC_API_KEY,
-    HINDSIGHT_API_LLM_MODEL: "claude-sonnet-4-20250514",
-    HINDSIGHT_EMBED_DAEMON_IDLE_TIMEOUT: "0",
+    HINDSIGHT_API_LLM_MODEL: 'claude-sonnet-4-20250514',
+    HINDSIGHT_EMBED_DAEMON_IDLE_TIMEOUT: '0',
   },
   logger: consoleLogger,
 });
@@ -37,11 +37,11 @@ await server.start();
 
 const client = new HindsightClient({ baseUrl: server.getBaseUrl() });
 
-await client.retain("user-123", "User prefers dark mode and concise answers.", {
-  documentId: "pref-2026-04-01",
+await client.retain('user-123', 'User prefers dark mode and concise answers.', {
+  documentId: 'pref-2026-04-01',
 });
 
-const recall = await client.recall("user-123", "what are the user preferences?");
+const recall = await client.recall('user-123', 'what are the user preferences?');
 console.log(recall.results);
 
 await server.stop();
@@ -62,7 +62,7 @@ If you're hacking on the Python `hindsight-embed` package in the same monorepo,
 
 ```ts
 new HindsightServer({
-  embedPackagePath: "/path/to/hindsight-embed",
+  embedPackagePath: '/path/to/hindsight-embed',
   // ...
 });
 ```

hindsight-all-npm/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@vectorize-io/hindsight-all",
-  "version": "0.9.1",
+  "version": "0.5.4",
   "description": "Node.js programmatic lifecycle manager for Hindsight — embeds a local hindsight daemon in a Node application. Pair with @vectorize-io/hindsight-client for memory operations.",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",

hindsight-all-npm/src/command.test.ts

@@ -1,36 +1,32 @@
-import { describe, it, expect } from "vitest";
-import { getEmbedCommand } from "./command.js";
+import { describe, it, expect } from 'vitest';
+import { getEmbedCommand } from './command.js';
 
-describe("getEmbedCommand", () => {
-  it("defaults to uvx hindsight-embed@latest", () => {
-    expect(getEmbedCommand()).toEqual(["uvx", "hindsight-embed@latest"]);
+describe('getEmbedCommand', () => {
+  it('defaults to uvx hindsight-embed@latest', () => {
+    expect(getEmbedCommand()).toEqual(['uvx', 'hindsight-embed@latest']);
   });
 
-  it("honours an explicit version", () => {
-    expect(getEmbedCommand({ embedVersion: "0.5.0" })).toEqual(["uvx", "[email protected]"]);
+  it('honours an explicit version', () => {
+    expect(getEmbedCommand({ embedVersion: '0.5.0' })).toEqual(['uvx', '[email protected]']);
   });
 
-  it("treats an empty version as latest", () => {
-    expect(getEmbedCommand({ embedVersion: "" })).toEqual(["uvx", "hindsight-embed@latest"]);
+  it('treats an empty version as latest', () => {
+    expect(getEmbedCommand({ embedVersion: '' })).toEqual(['uvx', 'hindsight-embed@latest']);
   });
 
-  it("uses uv run --directory when a local path is given", () => {
-    expect(getEmbedCommand({ embedPackagePath: "/abs/path" })).toEqual([
-      "uv",
-      "run",
-      "--directory",
-      "/abs/path",
-      "hindsight-embed",
+  it('uses uv run --directory when a local path is given', () => {
+    expect(getEmbedCommand({ embedPackagePath: '/abs/path' })).toEqual([
+      'uv',
+      'run',
+      '--directory',
+      '/abs/path',
+      'hindsight-embed',
     ]);
   });
 
-  it("local path takes precedence over version", () => {
-    expect(getEmbedCommand({ embedPackagePath: "/abs/path", embedVersion: "0.5.0" })).toEqual([
-      "uv",
-      "run",
-      "--directory",
-      "/abs/path",
-      "hindsight-embed",
-    ]);
+  it('local path takes precedence over version', () => {
+    expect(
+      getEmbedCommand({ embedPackagePath: '/abs/path', embedVersion: '0.5.0' }),
+    ).toEqual(['uv', 'run', '--directory', '/abs/path', 'hindsight-embed']);
   });
 });

hindsight-all-npm/src/command.ts

@@ -18,8 +18,8 @@ export interface EmbedCommandOptions {
 
 export function getEmbedCommand(opts: EmbedCommandOptions = {}): string[] {
   if (opts.embedPackagePath) {
-    return ["uv", "run", "--directory", opts.embedPackagePath, "hindsight-embed"];
+    return ['uv', 'run', '--directory', opts.embedPackagePath, 'hindsight-embed'];
   }
-  const version = opts.embedVersion && opts.embedVersion.length > 0 ? opts.embedVersion : "latest";
-  return ["uvx", `hindsight-embed@${version}`];
+  const version = opts.embedVersion && opts.embedVersion.length > 0 ? opts.embedVersion : 'latest';
+  return ['uvx', `hindsight-embed@${version}`];
 }

hindsight-all-npm/src/index.ts

@@ -1,7 +1,7 @@
-export { HindsightServer } from "./server.js";
-export { getEmbedCommand } from "./command.js";
-export { silentLogger, consoleLogger } from "./logger.js";
+export { HindsightServer } from './server.js';
+export { getEmbedCommand } from './command.js';
+export { silentLogger, consoleLogger } from './logger.js';
 
-export type { Logger } from "./logger.js";
-export type { EmbedCommandOptions } from "./command.js";
-export type { HindsightServerOptions } from "./types.js";
+export type { Logger } from './logger.js';
+export type { EmbedCommandOptions } from './command.js';
+export type { HindsightServerOptions } from './types.js';

hindsight-all-npm/src/server.test.ts

@@ -1,32 +1,32 @@
-import { describe, it, expect } from "vitest";
-import { HindsightServer } from "./server.js";
+import { describe, it, expect } from 'vitest';
+import { HindsightServer } from './server.js';
 
-describe("HindsightServer construction", () => {
-  it("defaults base URL to http://127.0.0.1:8888", () => {
+describe('HindsightServer construction', () => {
+  it('defaults base URL to http://127.0.0.1:8888', () => {
     const server = new HindsightServer();
-    expect(server.getBaseUrl()).toBe("http://127.0.0.1:8888");
-    expect(server.getProfile()).toBe("default");
+    expect(server.getBaseUrl()).toBe('http://127.0.0.1:8888');
+    expect(server.getProfile()).toBe('default');
   });
 
-  it("honours custom profile, port, and host", () => {
-    const server = new HindsightServer({ profile: "app", port: 9077, host: "0.0.0.0" });
-    expect(server.getProfile()).toBe("app");
-    expect(server.getBaseUrl()).toBe("http://0.0.0.0:9077");
+  it('honours custom profile, port, and host', () => {
+    const server = new HindsightServer({ profile: 'app', port: 9077, host: '0.0.0.0' });
+    expect(server.getProfile()).toBe('app');
+    expect(server.getBaseUrl()).toBe('http://0.0.0.0:9077');
   });
 
-  it("accepts open env pass-through without complaining about unknown keys", () => {
+  it('accepts open env pass-through without complaining about unknown keys', () => {
     const server = new HindsightServer({
       env: {
-        HINDSIGHT_API_LLM_PROVIDER: "openai",
-        HINDSIGHT_API_LLM_MODEL: "gpt-4o-mini",
+        HINDSIGHT_API_LLM_PROVIDER: 'openai',
+        HINDSIGHT_API_LLM_MODEL: 'gpt-4o-mini',
         // A field that does not exist today — should still be accepted
-        HINDSIGHT_FUTURE_FLAG: "enabled",
+        HINDSIGHT_FUTURE_FLAG: 'enabled',
       },
     });
     expect(server).toBeInstanceOf(HindsightServer);
   });
 
-  it("exposes checkHealth that returns false when no daemon is running", async () => {
+  it('exposes checkHealth that returns false when no daemon is running', async () => {
     // Random high port that nothing is listening on.
     const server = new HindsightServer({ port: 1, readyTimeoutMs: 100 });
     const healthy = await server.checkHealth();

hindsight-all-npm/src/server.ts

@@ -1,12 +1,12 @@
-import { spawn } from "child_process";
-import { getEmbedCommand } from "./command.js";
-import { silentLogger } from "./logger.js";
-import type { Logger } from "./logger.js";
-import type { HindsightServerOptions } from "./types.js";
+import { spawn } from 'child_process';
+import { getEmbedCommand } from './command.js';
+import { silentLogger } from './logger.js';
+import type { Logger } from './logger.js';
+import type { HindsightServerOptions } from './types.js';
 
 const DEFAULT_PORT = 8888;
-const DEFAULT_HOST = "127.0.0.1";
-const DEFAULT_PROFILE = "default";
+const DEFAULT_HOST = '127.0.0.1';
+const DEFAULT_PROFILE = 'default';
 const DEFAULT_READY_TIMEOUT_MS = 30_000;
 const DEFAULT_READY_POLL_INTERVAL_MS = 1_000;
 
@@ -61,7 +61,7 @@ export class HindsightServer {
     this.userEnv = opts.env ?? {};
     this.extraProfileCreateArgs = opts.extraProfileCreateArgs ?? [];
     this.extraDaemonStartArgs = opts.extraDaemonStartArgs ?? [];
-    this.platformCpuWorkaround = opts.platformCpuWorkaround ?? process.platform === "darwin";
+    this.platformCpuWorkaround = opts.platformCpuWorkaround ?? (process.platform === 'darwin');
     this.readyTimeoutMs = opts.readyTimeoutMs ?? DEFAULT_READY_TIMEOUT_MS;
     this.readyPollIntervalMs = opts.readyPollIntervalMs ?? DEFAULT_READY_POLL_INTERVAL_MS;
     this.logger = opts.logger ?? silentLogger;
@@ -100,22 +100,22 @@ export class HindsightServer {
       embedVersion: this.embedVersion,
       embedPackagePath: this.embedPackagePath,
     });
-    const args = [...baseArgs, "daemon", "--profile", this.profile, "stop"];
+    const args = [...baseArgs, 'daemon', '--profile', this.profile, 'stop'];
 
-    const child = spawn(cmd, args, { stdio: "pipe" });
-    this.pipeOutput(child, "daemon.stop");
+    const child = spawn(cmd, args, { stdio: 'pipe' });
+    this.pipeOutput(child, 'daemon.stop');
 
     await new Promise<void>((resolve) => {
       const timeout = setTimeout(() => {
         this.logger.warn(`[hindsight] daemon stop timed out after 5s`);
         resolve();
       }, 5_000);
-      child.on("exit", () => {
+      child.on('exit', () => {
         clearTimeout(timeout);
         this.logger.info(`[hindsight] daemon stopped`);
         resolve();
       });
-      child.on("error", (err) => {
+      child.on('error', (err) => {
         clearTimeout(timeout);
         this.logger.warn(`[hindsight] error stopping daemon: ${err.message}`);
         resolve();
@@ -147,9 +147,9 @@ export class HindsightServer {
   private buildEnv(): NodeJS.ProcessEnv {
     const merged: NodeJS.ProcessEnv = { ...process.env };
 
-    if (this.platformCpuWorkaround && process.platform === "darwin") {
-      merged["HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU"] = "1";
-      merged["HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU"] = "1";
+    if (this.platformCpuWorkaround && process.platform === 'darwin') {
+      merged['HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU'] = '1';
+      merged['HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU'] = '1';
     }
 
     for (const [key, value] of Object.entries(this.userEnv)) {
@@ -175,11 +175,11 @@ export class HindsightServer {
     });
     const createArgs = [
       ...baseArgs,
-      "profile",
-      "create",
+      'profile',
+      'create',
       this.profile,
-      "--merge",
-      "--port",
+      '--merge',
+      '--port',
       String(this.port),
     ];
 
@@ -189,12 +189,12 @@ export class HindsightServer {
     // host state into profile config.
     const envForProfile = this.collectProfileEnv(env);
     for (const [key, value] of Object.entries(envForProfile)) {
-      createArgs.push("--env", `${key}=${value}`);
+      createArgs.push('--env', `${key}=${value}`);
     }
 
     createArgs.push(...this.extraProfileCreateArgs);
 
-    await this.runCommand(cmd, createArgs, env, "profile.create");
+    await this.runCommand(cmd, createArgs, env, 'profile.create');
   }
 
   /** Collect only the env vars that should be written into the profile file. */
@@ -209,10 +209,10 @@ export class HindsightServer {
     }
 
     // 2. CPU workaround — only if auto-applied and not already overridden.
-    if (this.platformCpuWorkaround && process.platform === "darwin") {
+    if (this.platformCpuWorkaround && process.platform === 'darwin') {
       const cpuKeys = [
-        "HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU",
-        "HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU",
+        'HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU',
+        'HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU',
       ];
       for (const key of cpuKeys) {
         if (!(key in out) && env[key] !== undefined) {
@@ -231,14 +231,14 @@ export class HindsightServer {
     });
     const args = [
       ...baseArgs,
-      "daemon",
-      "--profile",
+      'daemon',
+      '--profile',
       this.profile,
-      "start",
+      'start',
       ...this.extraDaemonStartArgs,
     ];
 
-    await this.runCommand(cmd, args, env, "daemon.start");
+    await this.runCommand(cmd, args, env, 'daemon.start');
   }
 
   /**
@@ -249,34 +249,34 @@ export class HindsightServer {
     cmd: string,
     args: string[],
     env: NodeJS.ProcessEnv,
-    label: string
+    label: string,
   ): Promise<void> {
-    const child = spawn(cmd, args, { stdio: "pipe", env });
-    let output = "";
-    child.stdout?.on("data", (data: Buffer) => {
+    const child = spawn(cmd, args, { stdio: 'pipe', env });
+    let output = '';
+    child.stdout?.on('data', (data: Buffer) => {
       const text = data.toString();
       output += text;
-      for (const line of text.trimEnd().split("\n")) {
+      for (const line of text.trimEnd().split('\n')) {
         if (line) this.logger.info(`[hindsight:${label}] ${line}`);
       }
     });
-    child.stderr?.on("data", (data: Buffer) => {
+    child.stderr?.on('data', (data: Buffer) => {
       const text = data.toString();
       output += text;
-      for (const line of text.trimEnd().split("\n")) {
+      for (const line of text.trimEnd().split('\n')) {
         if (line) this.logger.warn(`[hindsight:${label}] ${line}`);
       }
     });
 
     await new Promise<void>((resolve, reject) => {
-      child.on("exit", (code) => {
+      child.on('exit', (code) => {
         if (code === 0) {
           resolve();
         } else {
           reject(new Error(`${label} failed with code ${code}: ${output.trim()}`));
         }
       });
-      child.on("error", (err) => {
+      child.on('error', (err) => {
         reject(new Error(`${label} failed to spawn: ${err.message}`, { cause: err }));
       });
     });
@@ -284,13 +284,13 @@ export class HindsightServer {
 
   /** Stream a spawned child's stdout/stderr through the logger without blocking. */
   private pipeOutput(child: ReturnType<typeof spawn>, label: string): void {
-    child.stdout?.on("data", (data: Buffer) => {
-      for (const line of data.toString().trimEnd().split("\n")) {
+    child.stdout?.on('data', (data: Buffer) => {
+      for (const line of data.toString().trimEnd().split('\n')) {
         if (line) this.logger.info(`[hindsight:${label}] ${line}`);
       }
     });
-    child.stderr?.on("data", (data: Buffer) => {
-      for (const line of data.toString().trimEnd().split("\n")) {
+    child.stderr?.on('data', (data: Buffer) => {
+      for (const line of data.toString().trimEnd().split('\n')) {
         if (line) this.logger.warn(`[hindsight:${label}] ${line}`);
       }
     });
@@ -316,7 +316,7 @@ export class HindsightServer {
       await new Promise((resolve) => setTimeout(resolve, this.readyPollIntervalMs));
     }
     throw new Error(
-      `Hindsight daemon did not become ready within ${this.readyTimeoutMs}ms at ${this.baseUrl}`
+      `Hindsight daemon did not become ready within ${this.readyTimeoutMs}ms at ${this.baseUrl}`,
     );
   }
 }

hindsight-all-npm/src/types.ts

@@ -1,4 +1,4 @@
-import type { Logger } from "./logger.js";
+import type { Logger } from './logger.js';
 
 /**
  * Options for {@link HindsightServer}.

hindsight-all-npm/tsup.config.ts

@@ -1,10 +1,10 @@
-import { defineConfig } from "tsup";
+import { defineConfig } from 'tsup';
 
 export default defineConfig({
-  entry: ["src/index.ts"],
-  format: ["esm"],
+  entry: ['src/index.ts'],
+  format: ['esm'],
   dts: true,
-  outDir: "dist",
+  outDir: 'dist',
   clean: true,
   sourcemap: true,
   bundle: true,

hindsight-all-npm/vitest.config.ts

@@ -1,8 +1,8 @@
-import { defineConfig } from "vitest/config";
+import { defineConfig } from 'vitest/config';
 
 export default defineConfig({
   test: {
-    include: ["src/**/*.test.ts"],
-    environment: "node",
+    include: ['src/**/*.test.ts'],
+    environment: 'node',
   },
 });

hindsight-all-slim/pyproject.toml

@@ -1,18 +1,17 @@
 [build-system]
-requires = ["setuptools>=77"]
+requires = ["setuptools>=61"]
 build-backend = "setuptools.build_meta"
 
 [project]
 name = "hindsight-all-slim"
-version = "0.9.1"
+version = "0.5.4"
 description = "Hindsight: Agent Memory That Works Like Human Memory - Slim All-in-One Bundle"
-license = "MIT"
 readme = "README.md"
 requires-python = ">=3.11"
 dependencies = [
-    "hindsight-api-slim==0.9.1",
+    "hindsight-api-slim>=0.4.17",
     "hindsight-client>=0.0.7",
-    "hindsight-embed==0.9.1",
+    "hindsight-embed>=0.1.0",
 ]
 
 [tool.uv.sources]

hindsight-all/hindsight/embedded.py

@@ -64,26 +64,15 @@ class HindsightEmbedded:
     - create_directive(), list_directives(), etc.
     - And all async variants (aretain, arecall, areflect, etc.)
 
-    Only the settings you pass explicitly are forwarded to the daemon. Anything
-    left at its default is resolved by the daemon instead, in this order: the
-    profile's .env file, then the parent process environment, then the daemon's
-    own default. That is what lets a client constructed without credentials run
-    against a profile (or a shell) that already has them configured, rather than
-    overwriting them with placeholders (#3253).
-
     Args:
         profile: Profile name for data isolation (default: "default")
-        llm_provider: LLM provider ("groq", "openai", "ollama", "gemini", "anthropic",
-            "lmstudio"). Omit to inherit; the server default is "openai".
-        llm_api_key: API key for the LLM provider. Omit to inherit; pass "" to
-            explicitly run without a key (local services that need no auth).
-        llm_model: Model name to use. Omit to inherit; the server picks a default
-            for the resolved provider.
+        llm_provider: LLM provider ("groq", "openai", "ollama", "gemini", "anthropic", "lmstudio")
+        llm_api_key: API key for the LLM provider
+        llm_model: Model name to use
         llm_base_url: Optional custom base URL for LLM API
         database_url: Optional database URL override (default: profile-specific pg0)
-        idle_timeout: Seconds before daemon auto-exits when idle. Omit to inherit
-            (daemon default: 0, disabled).
-        log_level: Daemon log level. Omit to inherit (daemon default: "info").
+        idle_timeout: Seconds before daemon auto-exits when idle (default: 0, disabled)
+        log_level: Daemon log level (default: "info")
         ui: Whether to start the control plane web UI alongside the daemon (default: False)
         ui_port: Port for the UI. Defaults to daemon_port + 10000.
         ui_hostname: Hostname to bind the UI to. Defaults to "0.0.0.0".
@@ -92,13 +81,13 @@ class HindsightEmbedded:
     def __init__(
         self,
         profile: str = "default",
-        llm_provider: Optional[str] = None,
-        llm_api_key: Optional[str] = None,
-        llm_model: Optional[str] = None,
+        llm_provider: str = "groq",
+        llm_api_key: str = "",
+        llm_model: str = "openai/gpt-oss-120b",
         llm_base_url: Optional[str] = None,
         database_url: Optional[str] = None,
-        idle_timeout: Optional[int] = None,
-        log_level: Optional[str] = None,
+        idle_timeout: int = 0,
+        log_level: str = "info",
         ui: bool = False,
         ui_port: Optional[int] = None,
         ui_hostname: str = "0.0.0.0",
@@ -106,50 +95,29 @@ class HindsightEmbedded:
         """
         Initialize the embedded client (daemon starts on first use).
 
-        Every LLM/daemon setting left as None is omitted from the daemon config so
-        the daemon resolves it from the profile .env, then the parent environment,
-        then its own default.
-
         Args:
             profile: Profile name for data isolation
-            llm_provider: LLM provider. Omit to inherit.
-            llm_api_key: API key for the LLM provider. Omit to inherit; pass "" to
-                explicitly run without a key.
-            llm_model: Model name to use. Omit to inherit.
+            llm_provider: LLM provider
+            llm_api_key: API key for the LLM provider
+            llm_model: Model name to use
             llm_base_url: Optional custom base URL for LLM API
             database_url: Optional database URL override
-            idle_timeout: Seconds before daemon auto-exits when idle (0 = disabled).
-                Omit to inherit.
-            log_level: Daemon log level. Omit to inherit.
+            idle_timeout: Seconds before daemon auto-exits when idle (0 = disabled)
+            log_level: Daemon log level
             ui: Whether to start the control plane web UI alongside the daemon
             ui_port: Port for the UI (defaults to daemon_port + 10000)
             ui_hostname: Hostname to bind the UI to (defaults to "0.0.0.0")
         """
         self.profile = profile
 
-        # Build the config dict for the daemon (matches CLI format), omitting
-        # every setting the caller did not specify. An omitted key is inherited
-        # by the daemon from the profile .env / parent environment; sending a
-        # placeholder instead would overwrite it, and _register_profile would
-        # then persist that placeholder into the profile's .env file (#3253).
-        # An explicit "" is still an override — that is how a local LLM service
-        # with no authentication clears an inherited API key.
-        self.config: dict[str, str] = {}
-
-        if llm_provider is not None:
-            self.config["HINDSIGHT_API_LLM_PROVIDER"] = llm_provider
-
-        if llm_api_key is not None:
-            self.config["HINDSIGHT_API_LLM_API_KEY"] = llm_api_key
-
-        if llm_model is not None:
-            self.config["HINDSIGHT_API_LLM_MODEL"] = llm_model
-
-        if log_level is not None:
-            self.config["HINDSIGHT_API_LOG_LEVEL"] = log_level
-
-        if idle_timeout is not None:
-            self.config["HINDSIGHT_EMBED_DAEMON_IDLE_TIMEOUT"] = str(idle_timeout)
+        # Build config dict for daemon (matches CLI format)
+        self.config = {
+            "HINDSIGHT_API_LLM_PROVIDER": llm_provider,
+            "HINDSIGHT_API_LLM_API_KEY": llm_api_key,
+            "HINDSIGHT_API_LLM_MODEL": llm_model,
+            "HINDSIGHT_API_LOG_LEVEL": log_level,
+            "HINDSIGHT_EMBED_DAEMON_IDLE_TIMEOUT": str(idle_timeout),
+        }
 
         if llm_base_url:
             self.config["HINDSIGHT_API_LLM_BASE_URL"] = llm_base_url

hindsight-all/pyproject.toml

@@ -1,18 +1,17 @@
 [build-system]
-requires = ["hatchling>=1.27"]
+requires = ["hatchling"]
 build-backend = "hatchling.build"
 
 [project]
 name = "hindsight-all"
-version = "0.9.1"
+version = "0.5.4"
 description = "Hindsight: Agent Memory That Works Like Human Memory - All-in-One Bundle"
-license = "MIT"
 readme = "README.md"
 requires-python = ">=3.11"
 dependencies = [
-    "hindsight-api-slim[all]==0.9.1",
+    "hindsight-api-slim[all]>=0.4.17",
     "hindsight-client>=0.0.7",
-    "hindsight-embed==0.9.1",
+    "hindsight-embed>=0.1.0",
 ]
 
 [tool.uv.sources]
@@ -22,7 +21,7 @@ hindsight-embed = { workspace = true }
 
 [project.optional-dependencies]
 local-llm = [
-    "hindsight-api-slim[local-llm]==0.9.1",
+    "hindsight-api-slim[local-llm]>=0.4.17",
 ]
 test = [
     "pytest>=7.0.0",

hindsight-all/tests/test_embedded.py

@@ -386,7 +386,7 @@ def test_embedded_ui_flag(llm_config):
 
         # Verify UI is reachable and reports connected dataplane
         ui_url = client.ui_url
-        assert isinstance(ui_url, str) and ui_url, "ui_url should be a non-empty string"
+        assert ui_url, "ui_url should be set"
 
         health_url = f"{ui_url}/api/health"
         with urllib.request.urlopen(health_url, timeout=10) as resp:

hindsight-all/tests/test_embedded_config.py

@@ -1,181 +0,0 @@
-"""Configuration forwarding rules for HindsightEmbedded.
-
-Regression coverage for #3253: a setting the caller does not pass must be left
-out of the daemon config, so the daemon can resolve it from the profile's .env
-file or the parent environment instead of receiving a client-side placeholder
-that overwrites it — and that the daemon then persists back into the profile.
-"""
-
-import json
-from unittest.mock import MagicMock, patch
-
-import pytest
-
-from hindsight import HindsightEmbedded
-from hindsight_embed.daemon_embed_manager import DaemonEmbedManager
-
-LLM_PROVIDER = "HINDSIGHT_API_LLM_PROVIDER"
-LLM_API_KEY = "HINDSIGHT_API_LLM_API_KEY"
-LLM_MODEL = "HINDSIGHT_API_LLM_MODEL"
-LOG_LEVEL = "HINDSIGHT_API_LOG_LEVEL"
-IDLE_TIMEOUT = "HINDSIGHT_EMBED_DAEMON_IDLE_TIMEOUT"
-
-
-@pytest.fixture
-def temp_home(tmp_path, monkeypatch):
-    """Isolate HOME so profile .env files never touch the real user profile.
-
-    USERPROFILE is set as well because Path.home() consults it on Windows.
-    """
-    home = tmp_path / "home"
-    home.mkdir()
-    monkeypatch.setenv("HOME", str(home))
-    monkeypatch.setenv("USERPROFILE", str(home))
-    return home
-
-
-def _write_profile(home, name, port, env_contents=None):
-    """Create a registered profile, optionally with a pre-populated .env file."""
-    profile_dir = home / ".hindsight" / "profiles"
-    profile_dir.mkdir(parents=True, exist_ok=True)
-    (profile_dir / "metadata.json").write_text(
-        json.dumps(
-            {
-                "version": 1,
-                "profiles": {
-                    name: {
-                        "port": port,
-                        "created_at": "2024-01-01T00:00:00+00:00",
-                        "last_used": "2024-01-01T00:00:00+00:00",
-                    }
-                },
-            }
-        )
-    )
-    env_path = profile_dir / f"{name}.env"
-    if env_contents is not None:
-        env_path.write_text(env_contents)
-    return env_path
-
-
-def _daemon_env(client):
-    """Run the real daemon start path with Popen stubbed, returning the child env.
-
-    Asserting on client.config alone would not catch a regression in how the
-    embed manager merges that config with the profile and the parent
-    environment, which is where the reported bug actually surfaced.
-    """
-    manager = DaemonEmbedManager()
-    captured: dict[str, dict[str, str]] = {}
-    spawned = [False]
-
-    def fake_popen(cmd, env, **kwargs):
-        captured["env"] = env
-        spawned[0] = True
-        process = MagicMock()
-        process.pid = 12345
-        return process
-
-    with (
-        patch("hindsight_embed.daemon_embed_manager.subprocess.Popen", side_effect=fake_popen),
-        patch("hindsight_embed.daemon_embed_manager.time.sleep"),
-        patch.object(manager, "_clear_port", return_value=True),
-        patch.object(manager, "_find_api_command", return_value=["hindsight-api"]),
-        patch.object(manager, "is_running", side_effect=lambda profile="": spawned[0]),
-        patch("hindsight_embed.daemon_embed_manager.platform.system", return_value="Linux"),
-    ):
-        assert manager.ensure_running(client.config, client.profile)
-
-    return captured["env"]
-
-
-def test_nothing_is_forwarded_when_nothing_is_specified(temp_home):
-    assert HindsightEmbedded(profile="test").config == {}
-
-
-def test_explicitly_passed_settings_are_forwarded(temp_home):
-    client = HindsightEmbedded(
-        profile="test",
-        llm_provider="openai",
-        llm_api_key="sk-real",
-        llm_model="gpt-4o-mini",
-        log_level="debug",
-        idle_timeout=300,
-    )
-
-    assert client.config == {
-        LLM_PROVIDER: "openai",
-        LLM_API_KEY: "sk-real",
-        LLM_MODEL: "gpt-4o-mini",
-        LOG_LEVEL: "debug",
-        IDLE_TIMEOUT: "300",
-    }
-
-
-def test_empty_api_key_is_forwarded_as_an_override(temp_home):
-    """An empty string is an explicit choice, not an omission.
-
-    Local LLM services that need no authentication rely on it to clear a key
-    inherited from the environment.
-    """
-    assert HindsightEmbedded(profile="test", llm_api_key="").config[LLM_API_KEY] == ""
-
-
-def test_idle_timeout_zero_is_forwarded(temp_home):
-    """0 is falsy but meaningful ("never auto-exit"), so it must survive."""
-    assert HindsightEmbedded(profile="test", idle_timeout=0).config[IDLE_TIMEOUT] == "0"
-
-
-def test_omitted_key_inherits_the_parent_environment(temp_home, monkeypatch):
-    monkeypatch.setenv(LLM_API_KEY, "sk-parent")
-    _write_profile(temp_home, "inherit-env", 9871)
-
-    env = _daemon_env(HindsightEmbedded(profile="inherit-env", llm_provider="openai"))
-
-    assert env[LLM_API_KEY] == "sk-parent"
-
-
-def test_omitted_settings_inherit_the_profile_env(temp_home, monkeypatch):
-    for var in (LLM_PROVIDER, LLM_API_KEY, LLM_MODEL):
-        monkeypatch.delenv(var, raising=False)
-    env_path = _write_profile(
-        temp_home,
-        "prod",
-        9872,
-        "HINDSIGHT_API_LLM_PROVIDER=anthropic\n"
-        "HINDSIGHT_API_LLM_MODEL=claude-sonnet-4-20250514\n"
-        "HINDSIGHT_API_LLM_API_KEY=sk-ant-prod\n",
-    )
-
-    env = _daemon_env(HindsightEmbedded(profile="prod"))
-
-    assert env[LLM_PROVIDER] == "anthropic"
-    assert env[LLM_MODEL] == "claude-sonnet-4-20250514"
-    assert env[LLM_API_KEY] == "sk-ant-prod"
-
-    # A successful start rewrites the profile's .env; it must not come back with
-    # client-side placeholders in place of the configured values.
-    persisted = env_path.read_text()
-    assert "HINDSIGHT_API_LLM_PROVIDER=anthropic" in persisted
-    assert "HINDSIGHT_API_LLM_MODEL=claude-sonnet-4-20250514" in persisted
-    assert "HINDSIGHT_API_LLM_API_KEY=sk-ant-prod" in persisted
-
-
-def test_explicit_empty_key_overrides_the_parent_environment(temp_home, monkeypatch):
-    monkeypatch.setenv(LLM_API_KEY, "sk-parent")
-    _write_profile(temp_home, "no-auth", 9873)
-
-    env = _daemon_env(
-        HindsightEmbedded(profile="no-auth", llm_provider="lmstudio", llm_api_key="")
-    )
-
-    assert env[LLM_API_KEY] == ""
-
-
-def test_explicit_settings_still_win_over_the_profile(temp_home, monkeypatch):
-    monkeypatch.delenv(LLM_PROVIDER, raising=False)
-    _write_profile(temp_home, "override", 9874, "HINDSIGHT_API_LLM_PROVIDER=anthropic\n")
-
-    env = _daemon_env(HindsightEmbedded(profile="override", llm_provider="openai"))
-
-    assert env[LLM_PROVIDER] == "openai"

hindsight-api-slim/README.md

@@ -99,7 +99,7 @@ hindsight-api
 ## Docker
 
 ```bash
-docker run -it --name hindsight --restart unless-stopped -p 8888:8888 \
+docker run --rm -it -p 8888:8888 \
   -e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \
   -v $HOME/.hindsight-docker:/home/hindsight/.pg0 \
   ghcr.io/vectorize-io/hindsight:latest
@@ -121,7 +121,7 @@ This runs a stdio-based MCP server that can be used directly with MCP-compatible
 - **Entity Graph** — Automatic entity extraction and relationship tracking
 - **Temporal Reasoning** — Native support for time-based queries
 - **Disposition Traits** — Configurable skepticism, literalism, and empathy influence opinion formation
-- **Three Memory Types** — World facts, experience facts (the bank's own actions), and observations
+- **Three Memory Types** — World facts, bank actions, and formed opinions with confidence scores
 
 ## Documentation
 

hindsight-api-slim/hindsight_api/__init__.py

@@ -4,13 +4,6 @@ Memory System for AI Agents.
 Temporal + Semantic Memory Architecture using PostgreSQL with pgvector.
 """
 
-# Cap native ML thread pools (OpenBLAS/OpenMP/MKL) before any import pulls in
-# numpy/torch/onnxruntime — they read these env vars only at load time. See
-# hindsight_api/_thread_limits.py for the rationale.
-from ._thread_limits import apply_default_thread_limits
-
-apply_default_thread_limits()
-
 from .config import HindsightConfig, get_config
 from .engine.cross_encoder import CrossEncoderModel, LocalSTCrossEncoder, RemoteTEICrossEncoder
 from .engine.embeddings import Embeddings, LocalSTEmbeddings, RemoteTEIEmbeddings
@@ -53,4 +46,4 @@ __all__ = [
     "RemoteTEICrossEncoder",
     "LLMConfig",
 ]
-__version__ = "0.9.1"
+__version__ = "0.5.4"

hindsight-api-slim/hindsight_api/_pg_search.py

@@ -1,85 +0,0 @@
-"""Helpers for ParadeDB pg_search index configuration."""
-
-from __future__ import annotations
-
-import re
-from collections.abc import Sequence
-
-PG_SEARCH_TOKENIZER_ENV = "HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER"
-
-_SIMPLE_TOKENIZERS = {
-    "unicode_words",
-    "simple",
-    "whitespace",
-    "literal",
-    "literal_normalized",
-    "chinese_compatible",
-    "icu",
-    "jieba",
-    "source_code",
-}
-
-_TOKENIZER_ALIASES = {
-    "chinese_lindera": "lindera(chinese)",
-    "japanese_lindera": "lindera(japanese)",
-    "korean_lindera": "lindera(korean)",
-    "lindera_chinese": "lindera(chinese)",
-    "lindera_japanese": "lindera(japanese)",
-    "lindera_korean": "lindera(korean)",
-}
-
-
-def normalize_pg_search_tokenizer(value: str | None) -> str:
-    """Validate and normalize a ParadeDB pg_search tokenizer setting.
-
-    Returns an empty string when unset. The returned value is safe to embed after
-    ``pdb.`` in a CREATE INDEX expression.
-    """
-
-    tokenizer = (value or "").strip().lower()
-    if not tokenizer:
-        return ""
-
-    if tokenizer in _TOKENIZER_ALIASES:
-        return _TOKENIZER_ALIASES[tokenizer]
-
-    if tokenizer in _SIMPLE_TOKENIZERS:
-        return tokenizer
-
-    lindera_match = re.fullmatch(r"lindera\((chinese|japanese|korean)\)", tokenizer)
-    if lindera_match:
-        return tokenizer
-
-    ngram_match = re.fullmatch(r"(ngram|edge_ngram)\((\d{1,3}),\s*(\d{1,3})\)", tokenizer)
-    if ngram_match:
-        kind, min_gram, max_gram = ngram_match.groups()
-        min_value = int(min_gram)
-        max_value = int(max_gram)
-        if min_value <= 0 or min_value > max_value:
-            raise ValueError(
-                f"Invalid {PG_SEARCH_TOKENIZER_ENV}: {value!r}. "
-                "ngram and edge_ngram require positive min/max gram sizes with min <= max."
-            )
-        return f"{kind}({min_value},{max_value})"
-
-    raise ValueError(
-        f"Invalid {PG_SEARCH_TOKENIZER_ENV}: {value!r}. "
-        "Supported values are: unicode_words, simple, whitespace, literal, "
-        "literal_normalized, chinese_compatible, icu, jieba, source_code, "
-        "chinese_lindera, japanese_lindera, korean_lindera, or "
-        "lindera(chinese|japanese|korean), ngram(min,max), or edge_ngram(min,max)."
-    )
-
-
-def pg_search_bm25_columns(
-    key_field: str,
-    text_fields: Sequence[str],
-    tokenizer: str | None,
-) -> str:
-    """Build a ParadeDB BM25 column list for CREATE INDEX."""
-
-    normalized = normalize_pg_search_tokenizer(tokenizer)
-    if not normalized:
-        return ", ".join([key_field, *text_fields])
-
-    return ", ".join([key_field, *(f"({field}::pdb.{normalized})" for field in text_fields)])

hindsight-api-slim/hindsight_api/_text_search.py

@@ -1,23 +0,0 @@
-"""Text-search SQL shapes shared by index DDL and the queries that must hit it.
-
-A PostgreSQL expression index is only selectable when the query repeats the
-indexed expression verbatim, so the DDL (``migrations.py`` and the Alembic
-versions) and the read arms (``engine/sql/postgresql.py``) cannot be allowed to
-drift. Both sides call the helpers here — same idea as
-``_pg_search.pg_search_bm25_columns``.
-"""
-
-
-def mental_models_text_document(alias: str | None = None) -> str:
-    """The ``mental_models`` full-text document: model/page name + content.
-
-    Mirrors the generating expression of the native tsvector column created by
-    the ``n9i0j1k2l3m4`` (learnings / pinned_reflections) migration, so every
-    backend indexes and queries the exact same document. ``content`` is NOT NULL,
-    hence the deliberate lack of a ``COALESCE`` around it.
-
-    ``alias`` qualifies the columns for queries that join the table (``mm``);
-    leave it unset for DDL, where the expression is already table-scoped.
-    """
-    prefix = f"{alias}." if alias else ""
-    return f"(COALESCE({prefix}name, '') || ' ' || {prefix}content)"

hindsight-api-slim/hindsight_api/_thread_limits.py

@@ -1,107 +0,0 @@
-"""Process-level caps for native ML thread pools.
-
-OpenBLAS, OpenMP, and MKL each spawn a worker pool sized to the host CPU count
-the first time they are loaded (numpy pulls in OpenBLAS eagerly; torch and
-onnxruntime load their pools lazily on first inference). Hindsight already
-parallelizes at the request level via thread-pool executors (embeddings on the
-default executor, the reranker on its own pool), so these native intra-op pools
-oversubscribe the CPU: on a many-core host the process accumulates 100+ native
-threads, which inflates memory and, under contention, can degrade throughput.
-
-We bound each pool to ``_MAX_NATIVE_THREADS`` (or the available CPU count, if
-smaller). "Available" is the CPU budget actually granted to the process, not
-``os.cpu_count()``: in a CPU-limited container ``os.cpu_count()`` still reports
-the host's cores, so sizing pools by it oversubscribes the container's real
-quota — the exact failure mode this guards against. We therefore take the
-smallest of the CPU-affinity set, the cgroup CPU quota, and ``os.cpu_count()``.
-
-Every cap is applied with ``setdefault`` so an operator who has deliberately
-tuned one of these variables keeps their value. This must run *before* numpy,
-torch, or onnxruntime are imported — those libraries read the variables only at
-load time — which is why it is invoked at the very top of
-``hindsight_api/__init__.py``, ahead of the package's other imports.
-"""
-
-from __future__ import annotations
-
-import os
-
-# Native threading env vars, each read by the respective library at load time.
-_NATIVE_THREAD_VARS = (
-    "OMP_NUM_THREADS",  # OpenMP — torch, onnxruntime, some BLAS builds
-    "OPENBLAS_NUM_THREADS",  # OpenBLAS — numpy's default BLAS
-    "MKL_NUM_THREADS",  # Intel MKL — numpy/torch when MKL-backed
-    "NUMEXPR_NUM_THREADS",  # numexpr expression engine
-)
-
-# Upper bound on intra-op threads per native pool. Bounds runaway growth on
-# many-core hosts without serialising single-request inference.
-_MAX_NATIVE_THREADS = 16
-
-
-def _quota_to_cpus(quota: int, period: int) -> int | None:
-    """Whole CPUs from a CFS quota/period pair, or None if unlimited."""
-    if quota > 0 and period > 0:
-        # Floor (never round up) so we never exceed the granted budget.
-        return max(1, quota // period)
-    return None
-
-
-def _parse_cgroup_v2_cpu_max(text: str) -> int | None:
-    """Parse cgroup v2 ``cpu.max`` ("<quota> <period>", or "max <period>")."""
-    parts = text.split()
-    if len(parts) >= 2 and parts[0] != "max":
-        try:
-            return _quota_to_cpus(int(parts[0]), int(parts[1]))
-        except ValueError:
-            return None
-    return None
-
-
-def _cgroup_cpu_quota() -> int | None:
-    """Effective CPUs from the cgroup CPU quota, or None if unlimited/unknown."""
-    try:  # cgroup v2
-        with open("/sys/fs/cgroup/cpu.max") as fh:
-            return _parse_cgroup_v2_cpu_max(fh.read())
-    except OSError:
-        pass
-    try:  # cgroup v1
-        with open("/sys/fs/cgroup/cpu/cpu.cfs_quota_us") as fh:
-            quota = int(fh.read())
-        with open("/sys/fs/cgroup/cpu/cpu.cfs_period_us") as fh:
-            period = int(fh.read())
-        return _quota_to_cpus(quota, period)
-    except (OSError, ValueError):
-        return None
-
-
-def _available_cpu_count() -> int:
-    """CPUs actually available to this process.
-
-    The smallest of the CPU-affinity set (cpuset / ``--cpuset-cpus``), the
-    cgroup CPU quota (``--cpus``), and ``os.cpu_count()`` — each captures a
-    different way the budget can be constrained, and the last alone overcounts
-    inside a limited container.
-    """
-    candidates = [os.cpu_count() or 1]
-    if hasattr(os, "sched_getaffinity"):
-        try:
-            candidates.append(len(os.sched_getaffinity(0)))
-        except OSError:
-            pass
-    quota = _cgroup_cpu_quota()
-    if quota is not None:
-        candidates.append(quota)
-    return max(1, min(candidates))
-
-
-def default_native_thread_count() -> int:
-    """Per-pool cap: ``_MAX_NATIVE_THREADS``, or available CPUs if fewer."""
-    return min(_MAX_NATIVE_THREADS, _available_cpu_count())
-
-
-def apply_default_thread_limits() -> None:
-    """Cap native ML thread pools unless the operator has set the var already."""
-    value = str(default_native_thread_count())
-    for var in _NATIVE_THREAD_VARS:
-        os.environ.setdefault(var, value)

hindsight-api-slim/hindsight_api/_vector_index.py

@@ -1,377 +0,0 @@
-"""Shared PostgreSQL vector-extension dispatch helpers."""
-
-from __future__ import annotations
-
-import logging
-import os
-
-from sqlalchemy import text
-from sqlalchemy.engine import Connection
-
-logger = logging.getLogger(__name__)
-
-# Extensions a user can set via HINDSIGHT_API_VECTOR_EXTENSION.
-CONFIGURABLE_EXTENSIONS = ("pgvector", "pgvectorscale", "vchord", "scann")
-
-# Extensions detect_vector_extension() can return. pg_diskann is a runtime-only
-# resolution from a configured "pgvectorscale" backend on Azure (uses a different
-# WITH clause), never a value the user sets directly.
-RESOLVED_EXTENSIONS = (*CONFIGURABLE_EXTENSIONS, "pg_diskann")
-
-# Backwards-compatible alias for older imports.
-VALID_EXTENSIONS = CONFIGURABLE_EXTENSIONS
-
-SCANN_MIN_ROWS_FOR_AUTO_INDEX = 10_000
-
-
-_EXTENSION_NAMES = {
-    "pgvector": "vector",
-    "pgvectorscale": "vectorscale",
-    "vchord": "vchord",
-    "scann": "alloydb_scann",
-}
-
-_INDEX_USING_CLAUSES = {
-    "pgvector": "USING hnsw (embedding vector_cosine_ops)",
-    "pgvectorscale": "USING diskann (embedding vector_cosine_ops) WITH (num_neighbors = 50)",
-    "pg_diskann": "USING diskann (embedding vector_cosine_ops) WITH (max_neighbors = 50)",
-    "vchord": "USING vchordrq (embedding vector_cosine_ops)",
-    "scann": "USING scann (embedding cosine) WITH (mode = 'AUTO')",
-}
-
-_INDEX_TYPE_KEYWORDS = {
-    "pgvector": "hnsw",
-    "pgvectorscale": "diskann",
-    "pg_diskann": "diskann",
-    "vchord": "vchordrq",
-    "scann": "scann",
-}
-
-# Ceiling on how many tuples one resumed ANN scan may visit (hnsw.max_scan_tuples).
-# Only iterative scans consult it, and it is approximate — the initial round is not
-# counted. pgvector defaults to 20000; this is deliberately lower.
-#
-# The filters that thin a semantic arm (the similarity floor, tags, date ranges) are
-# applied *after* the index scan, so a selective query resumes repeatedly to fill its
-# LIMIT. Unbounded, that turns the cheapest queries today into the most expensive:
-# ~20x the standing batch is enough to fill even a large recall budget on an
-# unfiltered query, and caps the pathological filtered case at a scan that returns
-# short — which is exactly what those queries did before iterative scans were on.
-# The GUCs that make a scan resumable — dropped wholesale when the operator turns the
-# behaviour off, so a connection is left exactly as it was before it existed (and a
-# pgvector too old to define them is never sent them either).
-_ITERATIVE_SCAN_GUCS = frozenset({"hnsw.iterative_scan", "hnsw.max_scan_tuples"})
-
-
-def iterative_scan_enabled() -> bool:
-    """Whether ANN scans may resume to satisfy a query's LIMIT.
-
-    Turning it off restores the previous depth exactly: a scan stops when its first
-    candidate list drains, so no recall retrieves more rows than that list holds,
-    whatever its budget.
-
-    Resolved through the config object rather than read from the environment, so a
-    value set any other way — a CLI override applied with dataclasses.replace, a
-    programmatically built config — is honoured, and the parsing and validation live
-    in one place. Imported inside the function because config imports this module.
-    """
-    from .config import get_config
-
-    return get_config().ann_iterative_scan
-
-
-def ann_max_scan_tuples() -> int:
-    """Ceiling on tuples one resumed scan may visit (hnsw.max_scan_tuples).
-
-    This is the knob that governs the cost of the behaviour. It bounds the CPU a
-    selective query can spend resuming, and with it the scan's memory — pgvector
-    otherwise caps that at ``work_mem * hnsw.scan_mem_multiplier``, but at this
-    default the memory ceiling is never approached: squeezing work_mem to 256kB
-    changes neither the rows returned nor the latency.
-
-    Approximate, and the initial scan is not counted, so even 1 leaves intact the
-    depth a query had before scans could resume.
-    """
-    from .config import get_config
-
-    return get_config().ann_max_scan_tuples
-
-
-# Per-backend ANN search-time tuning GUCs. Each entry is a tuple of
-# (guc_name, value) pairs the caller can apply with SET or SET LOCAL.
-#
-# - pgvector exposes hnsw.ef_search. The 60 / 200 pair is unchanged from the
-#   pre-dispatcher code (internal benchmarks tuned around our embedding count
-#   and recall floor; see the link_utils / pool init call sites for the
-#   latency-vs-recall framing). With iterative scans on (below) the ef value is a
-#   batch size rather than a ceiling, so a query's own LIMIT decides its depth.
-# - vchord exposes vchordrq.probes, but its shape must match the index's
-#   build.internal.lists hierarchy. VectorChord 1.1 added per-index fallback
-#   parameters for this reason: a session GUC overrides every vchordrq index,
-#   and a single value can be invalid for listless or mixed-layout indexes.
-#   Hindsight's built-in vchord clause does not set lists, so the safe default
-#   is no session-level probe override; deployments that partition vchordrq
-#   indexes should attach probes to the index storage parameters instead.
-# - pgvectorscale / pg_diskann / scann do not expose an equivalent per-statement
-#   knob in the engine today, so the dispatcher returns no statements for them.
-#
-# hnsw.iterative_scan is what makes ef_search a *batch* size rather than a ceiling.
-# With it off (pgvector's default, and what Hindsight ran until now) the ground-layer
-# search runs once and the scan ends when its list drains, so a query could never get
-# more rows than ef_search however large its LIMIT — the recall budget moved the SQL
-# and nothing else. With it on, the scan resumes in ef_search-sized rounds until the
-# LIMIT is met, so each query gets the depth it asks for with no per-query setting.
-# strict_order, not relaxed_order: the arms are trimmed in Python on the assumption
-# that rows arrive ordered by distance.
-#
-# Retain-side link probing wants the opposite — it is tuned for latency, not depth,
-# and resuming past its small candidate list would defeat that — so the low-latency
-# profile pins it off. Both profiles set it explicitly rather than relying on the
-# server default, so neither depends on what the other last left on the connection.
-_ANN_TUNING_LOW_LATENCY: dict[str, tuple[tuple[str, str], ...]] = {
-    "pgvector": (("hnsw.ef_search", "60"), ("hnsw.iterative_scan", "off")),
-}
-_ANN_TUNING_HIGH_RECALL: dict[str, tuple[tuple[str, str], ...]] = {
-    "pgvector": (
-        ("hnsw.ef_search", "200"),
-        ("hnsw.iterative_scan", "strict_order"),
-        # Value filled in per call by ann_search_tuning_settings().
-        ("hnsw.max_scan_tuples", ""),
-    ),
-}
-
-_EXTENSION_INSTALL_SQL = {
-    "pgvector": ("CREATE EXTENSION IF NOT EXISTS vector",),
-    "pgvectorscale": (
-        "CREATE EXTENSION IF NOT EXISTS vector",
-        "CREATE EXTENSION IF NOT EXISTS vectorscale CASCADE",
-    ),
-    "vchord": ("CREATE EXTENSION IF NOT EXISTS vchord CASCADE",),
-    "scann": (
-        "CREATE EXTENSION IF NOT EXISTS vector",
-        "CREATE EXTENSION IF NOT EXISTS alloydb_scann CASCADE",
-    ),
-}
-
-_INSTALL_HINTS = {
-    "pgvector": "CREATE EXTENSION vector;",
-    "pgvectorscale": "CREATE EXTENSION vector; then CREATE EXTENSION vectorscale CASCADE; (or pg_diskann on Azure)",
-    "vchord": "CREATE EXTENSION vchord CASCADE;",
-    "scann": "CREATE EXTENSION vector; then CREATE EXTENSION alloydb_scann CASCADE;",
-}
-
-
-def configured_vector_extension() -> str:
-    """Return the user-configured vector backend extension.
-
-    Reads ``HINDSIGHT_API_VECTOR_EXTENSION`` (default ``"pgvector"``) and
-    validates it via :func:`validate_extension`. This is the single source of
-    truth for runtime code that needs to dispatch behaviour by vector backend;
-    callers should prefer this over reading the env var directly, so the
-    default value and the lookup mechanism live in one place.
-    """
-    return validate_extension(os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector"))
-
-
-def validate_extension(name: str) -> str:
-    """Return a normalized configurable vector extension name or raise.
-
-    Used at the user-facing config boundary; pg_diskann is rejected here because
-    it is a detection-time alias, never a value the user sets directly.
-    """
-    ext = name.lower()
-    if ext not in CONFIGURABLE_EXTENSIONS:
-        valid = ", ".join(CONFIGURABLE_EXTENSIONS)
-        raise ValueError(f"Invalid vector_extension: {name}. Must be one of: {valid}")
-    return ext
-
-
-def _normalize_resolved(name: str) -> str:
-    """Normalize either a user-configurable or detect-time extension name."""
-    ext = name.lower()
-    if ext not in RESOLVED_EXTENSIONS:
-        valid = ", ".join(RESOLVED_EXTENSIONS)
-        raise ValueError(f"Unknown vector extension: {name}. Must be one of: {valid}")
-    return ext
-
-
-def pg_extension_name(ext: str) -> str:
-    """Return the PostgreSQL extension name for a configured vector backend."""
-    return _EXTENSION_NAMES[validate_extension(ext)]
-
-
-def index_using_clause(ext: str) -> str:
-    """Return the CREATE INDEX USING clause for the vector backend."""
-    return _INDEX_USING_CLAUSES[_normalize_resolved(ext)]
-
-
-def index_type_keyword(ext: str) -> str:
-    """Return the keyword that identifies this index type in pg_indexes.indexdef."""
-    return _INDEX_TYPE_KEYWORDS[_normalize_resolved(ext)]
-
-
-def minimum_rows_for_index(ext: str) -> int:
-    """Return the minimum populated embedding rows before creating this index type."""
-    return SCANN_MIN_ROWS_FOR_AUTO_INDEX if _normalize_resolved(ext) == "scann" else 0
-
-
-def should_defer_index_creation(ext: str, row_count: int) -> bool:
-    """Return True when index creation should wait for more embeddings."""
-    minimum_rows = minimum_rows_for_index(ext)
-    return minimum_rows > 0 and row_count < minimum_rows
-
-
-def ann_search_tuning_settings(ext: str, *, kind: str) -> tuple[tuple[str, str], ...]:
-    """Return per-backend (guc_name, value) pairs for ANN search-time tuning.
-
-    ``kind`` is ``"low_latency"`` for retain-side link probing (smaller probe
-    count, lower recall, lower latency) and ``"high_recall"`` for connection
-    init in the pool (larger probe count, higher recall). Callers wrap each
-    pair with ``SET LOCAL`` or ``SET`` themselves so the same dispatcher works
-    for both transaction-scoped and session-scoped use. Returns an empty tuple
-    for backends without an equivalent knob.
-    """
-    if kind == "low_latency":
-        table = _ANN_TUNING_LOW_LATENCY
-    elif kind == "high_recall":
-        table = _ANN_TUNING_HIGH_RECALL
-    else:
-        raise ValueError(f"Unknown ANN tuning kind: {kind!r}")
-    settings = table.get(_normalize_resolved(ext), ())
-    if not iterative_scan_enabled():
-        return tuple(pair for pair in settings if pair[0] not in _ITERATIVE_SCAN_GUCS)
-    return tuple(
-        (name, str(ann_max_scan_tuples()) if name == "hnsw.max_scan_tuples" else value) for name, value in settings
-    )
-
-
-def uses_per_bank_vector_indexes(ext: str) -> bool:
-    """Return whether the backend should create per-bank partial vector indexes."""
-    return _normalize_resolved(ext) != "scann"
-
-
-def per_bank_index_min_rows() -> int:
-    """Rows a (bank, fact_type) needs before it earns its own partial vector index.
-
-    Distinct from :func:`minimum_rows_for_index`, which is ScaNN's *build*
-    requirement for its single global index (AlloyDB cannot construct one below
-    a floor). This is a cost policy for the per-bank backends: the indexes sit on
-    the shared ``memory_units`` table, so each one is enumerated and locked at
-    plan time by queries belonging to every *other* bank, and opened by every DML
-    statement against the table. A small bank's index cannot repay that — the
-    ``(bank_id, fact_type)`` B-tree plus a top-N sort answers the same query
-    exactly and faster. See issue #3485.
-
-    Read from config rather than passed in because the write path's pre-check,
-    the maintenance operation and the admin command must all apply the same
-    number; a threshold that differed between the one deciding to queue work and
-    the one deciding what to do would either oscillate or never converge.
-    """
-    from .config import get_config
-
-    return get_config().vector_index_min_rows
-
-
-def per_bank_index_drop_rows() -> int:
-    """Row count below which an existing per-bank vector index is dropped.
-
-    Strictly below :func:`per_bank_index_min_rows` so the build and drop
-    decisions cannot both be true at one row count. Without the gap, a bank
-    hovering at the threshold — consolidation prunes a few facts, retain adds
-    them back — would rebuild and drop the same ANN index on alternating sweeps.
-    """
-    from .config import VECTOR_INDEX_DROP_RATIO
-
-    return int(per_bank_index_min_rows() * VECTOR_INDEX_DROP_RATIO)
-
-
-def should_keep_per_bank_index(row_count: int) -> bool:
-    """Whether an *existing* index on a partition of ``row_count`` rows is kept.
-
-    The counterpart to :func:`qualifies_for_per_bank_index`, and deliberately a
-    separate, lower bound: keeping starts below building, so a partition
-    hovering at the threshold does not rebuild and drop the same ANN index on
-    alternating writes.
-
-    The ``row_count > 0`` term is not redundant with the ratio. At the default
-    threshold of 0 the drop floor is also 0, so a bare ``row_count >= floor``
-    keeps an index over an *emptied* partition forever — every bank ever written
-    to and then cleared would hold three indexes over nothing, which is the
-    accumulation the threshold exists to prevent. An emptied partition loses its
-    index at every threshold.
-    """
-    return row_count > 0 and row_count >= per_bank_index_drop_rows()
-
-
-def qualifies_for_per_bank_index(row_count: int) -> bool:
-    """Whether a (bank, fact_type) holding ``row_count`` rows should have an index.
-
-    At the default threshold of 0 this is true for every partition that holds
-    any rows at all, which is the behaviour before the threshold existed.
-
-    An empty partition is excluded explicitly rather than by arithmetic: at a
-    threshold of 0, ``row_count >= minimum`` alone is true for zero rows, so
-    every bank in the deployment would be entitled to three indexes over nothing
-    the moment it was created — the exact index explosion the threshold exists
-    to prevent, reintroduced by its own default.
-
-    Only the build side: an existing index is kept until the count falls under
-    :func:`per_bank_index_drop_rows`, so callers reconciling live state must
-    consult both bounds rather than treating this as the full policy.
-
-    Takes no extension: the backend question is settled before any reconcile
-    runs (``uses_per_bank_vector_indexes`` gates the maintenance operation and
-    ``_vector_index_clause`` gates the admin command), so re-asking it here
-    would be a second, weaker copy of a decision already made.
-    """
-    return row_count > 0 and row_count >= per_bank_index_min_rows()
-
-
-def bootstrap_extension(conn: Connection, ext: str) -> None:
-    """Install the configured vector extension and any prerequisites if possible."""
-    normalized = validate_extension(ext)
-    for statement in _EXTENSION_INSTALL_SQL[normalized]:
-        conn.execute(text(statement))
-
-
-def detect_vector_extension(conn: Connection, vector_extension: str = "pgvector") -> str:
-    """Validate the configured vector extension exists and return the index backend."""
-    configured_ext = validate_extension(vector_extension)
-
-    if configured_ext == "pgvectorscale":
-        pgvector_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vector'")).scalar()
-        if not pgvector_check:
-            raise RuntimeError(
-                "DiskANN (pgvectorscale/pg_diskann) requires pgvector to be installed. "
-                f"Install it with: {_INSTALL_HINTS['pgvectorscale']}"
-            )
-
-        vectorscale_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vectorscale'")).scalar()
-        pg_diskann_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'pg_diskann'")).scalar()
-
-        if vectorscale_check:
-            logger.debug("Using vector extension: pgvectorscale (DiskANN)")
-            return "pgvectorscale"
-        if pg_diskann_check:
-            logger.debug("Using vector extension: pg_diskann (Azure DiskANN)")
-            return "pg_diskann"
-
-        raise RuntimeError(
-            "Configured vector extension 'pgvectorscale' not found. Install either:\n"
-            "  - pgvectorscale (open source): CREATE EXTENSION vectorscale CASCADE;\n"
-            "  - pg_diskann (Azure): CREATE EXTENSION pg_diskann CASCADE;"
-        )
-
-    extension_name = pg_extension_name(configured_ext)
-    extension_check = conn.execute(
-        text("SELECT 1 FROM pg_extension WHERE extname = :extension_name"),
-        {"extension_name": extension_name},
-    ).scalar()
-    if not extension_check:
-        raise RuntimeError(
-            f"Configured vector extension '{configured_ext}' not found. "
-            f"Install it with: {_INSTALL_HINTS[configured_ext]}"
-        )
-
-    logger.debug("Using configured vector extension: %s", configured_ext)
-    return configured_ext

hindsight-api-slim/hindsight_api/admin/cli.py

@@ -8,9 +8,7 @@ import asyncio
 import io
 import json
 import logging
-import struct
 import zipfile
-from dataclasses import dataclass
 from datetime import datetime, timezone
 from pathlib import Path
 from typing import Any
@@ -18,17 +16,8 @@ from typing import Any
 import asyncpg
 import typer
 
-from ..config import DEFAULT_DATABASE_SCHEMA, HindsightConfig, load_dotenv_for_entrypoint
-from ..engine.memory_engine import _current_schema
-from ..engine.retain.bank_utils import _vector_index_clause
+from ..config import DEFAULT_DATABASE_SCHEMA, HindsightConfig
 from ..engine.schema import fq_table_explicit as _fq_table
-from ..engine.transfer import export_bank
-from ..engine.vector_index_health import (
-    BankIndexResult,
-    drop_orphaned_bank_indexes,
-    list_bank_ids,
-    reconcile_bank_vector_indexes,
-)
 from ..extensions import TenantExtension, load_extension
 from ..pg0 import parse_pg0_url, resolve_database_url
 
@@ -41,240 +30,24 @@ logger = logging.getLogger(__name__)
 
 app = typer.Typer(name="hindsight-admin", help="Hindsight administrative commands")
 
-# Tables to backup/restore in foreign-key dependency order (parents first).
-# Restore COPYs in this order and TRUNCATEs in reverse, so every child must
-# appear after the tables it references.
-#
-# This must cover EVERY persistent PostgreSQL table in the schema — a missing
-# entry silently drops that table's data on restore (and, worse, restore's
-# `TRUNCATE banks CASCADE` wipes any FK-to-banks child like mental_models even
-# when it was never backed up). test_admin_backup_restore.py asserts this list
-# equals the live schema's tables, so adding a migration that creates a table
-# without adding it here fails CI. Oracle-only tables (e.g. observation_sources)
-# are intentionally absent — admin backup/restore is PostgreSQL-only.
+# Tables to backup/restore in dependency order
+# Import must happen in this order due to foreign key constraints
 BACKUP_TABLES = [
     "banks",
     "documents",
     "entities",
     "chunks",
     "memory_units",
-    "invalidated_memory_units",
     "unit_entities",
     "entity_cooccurrences",
     "memory_links",
-    "observation_history",
-    "mental_models",
-    "mental_model_history",
-    "knowledge_pages",
-    "directives",
-    "async_operations",
-    "webhooks",
-    "file_storage",
-    "audit_log",
-    "llm_requests",
-    "graph_maintenance_queue",
-    "entity_maintenance_queue",
 ]
 
-MANIFEST_VERSION = "2"
+MANIFEST_VERSION = "1"
 
 
-@dataclass(frozen=True)
-class BackupColumn:
-    """A PostgreSQL column shape required to decode a binary COPY stream."""
-
-    name: str
-    type_name: str
-
-
-@dataclass(frozen=True)
-class TableRestorePlan:
-    """How one table's backed-up binary COPY stream is replayed onto the target.
-
-    ``columns`` is the target column list handed to ``copy_to_table``, in stream
-    order. When the target no longer has a backed-up column, its field is stripped
-    from every tuple (``dropped_field_indices``) before the stream is replayed —
-    binary COPY is positional, so the column list and the tuple fields must agree.
-    """
-
-    columns: list[str]
-    dropped_field_indices: tuple[int, ...]
-    source_field_count: int
-
-
-# Header of a PostgreSQL binary COPY stream: an 11-byte signature, an int32 flags
-# field, and an int32 header-extension length followed by that many bytes.
-_COPY_BINARY_SIGNATURE = b"PGCOPY\n\xff\r\n\x00"
-_COPY_BINARY_HEADER_LEN = len(_COPY_BINARY_SIGNATURE) + 8
-
-
-def _strip_binary_copy_fields(data: bytes, plan: TableRestorePlan) -> bytes:
-    """Drop `plan.dropped_field_indices` from every tuple of a binary COPY stream.
-
-    Restore used to reject a backup whose columns the target no longer had — the
-    preflight raised "target is missing backup columns …", which made any backup
-    taken before a column-dropping migration unrestorable afterwards. Those columns
-    are now ignored instead, but they cannot simply be left out of the
-    ``copy_to_table`` column list: binary COPY carries no column identities, so each
-    tuple's fields are matched to the column list purely by position and an unedited
-    stream would desynchronise (or, worse, land values in the wrong columns). So the
-    stream itself is rewritten here.
-
-    Tuple format: int16 field count, then per field an int32 length (-1 for NULL)
-    followed by that many bytes. An int16 of -1 is the end-of-data trailer.
-    """
-    if not plan.dropped_field_indices:
-        return data
-    if not data.startswith(_COPY_BINARY_SIGNATURE):
-        raise ValueError("Backup stream is not in PostgreSQL binary COPY format")
-
-    (extension_len,) = struct.unpack_from("!i", data, len(_COPY_BINARY_SIGNATURE) + 4)
-    pos = _COPY_BINARY_HEADER_LEN + extension_len
-    out = bytearray(data[:pos])
-
-    dropped = set(plan.dropped_field_indices)
-    kept_count = plan.source_field_count - len(dropped)
-    while True:
-        (field_count,) = struct.unpack_from("!h", data, pos)
-        pos += 2
-        if field_count == -1:  # end-of-data trailer
-            out += struct.pack("!h", -1)
-            break
-        if field_count != plan.source_field_count:
-            raise ValueError(
-                f"Backup stream tuple has {field_count} fields, manifest declares {plan.source_field_count}"
-            )
-        out += struct.pack("!h", kept_count)
-        for index in range(field_count):
-            (length,) = struct.unpack_from("!i", data, pos)
-            pos += 4
-            payload = b"" if length == -1 else data[pos : pos + length]
-            pos += max(length, 0)
-            if index in dropped:
-                continue
-            out += struct.pack("!i", length)
-            out += payload
-    return bytes(out)
-
-
-async def _table_columns(conn: asyncpg.Connection, schema: str, table: str) -> list[BackupColumn]:
-    rows = await conn.fetch(
-        """
-        SELECT a.attname AS name, pg_catalog.format_type(a.atttypid, a.atttypmod) AS type_name
-        FROM pg_catalog.pg_attribute AS a
-        JOIN pg_catalog.pg_class AS c ON c.oid = a.attrelid
-        JOIN pg_catalog.pg_namespace AS n ON n.oid = c.relnamespace
-        WHERE n.nspname = $1 AND c.relname = $2 AND a.attnum > 0 AND NOT a.attisdropped
-          AND a.attgenerated = ''
-        ORDER BY a.attnum
-        """,
-        schema,
-        table,
-    )
-    return [BackupColumn(name=row["name"], type_name=row["type_name"]) for row in rows]
-
-
-async def _validate_restore_schema(
-    conn: asyncpg.Connection, manifest: dict[str, Any], schema: str
-) -> dict[str, TableRestorePlan]:
-    """Validate every COPY stream against the target before destructive work starts.
-
-    A column the target no longer has is **not** an error: a migration that drops a
-    column would otherwise make every backup taken before it permanently
-    unrestorable. Such columns are skipped (their fields are stripped from the
-    stream by ``_strip_binary_copy_fields``) and reported, so the operator sees what
-    was discarded instead of the restore failing outright.
-
-    Type mismatches remain fatal. Type equality is an exact ``format_type`` string
-    match. This is deliberately stricter than binary-COPY wire compatibility (e.g.
-    ``varchar`` and ``text`` share a binary format yet compare unequal here): we
-    would rather fail a genuinely-restorable backup with a clear, actionable error
-    than silently risk a subtle binary mismatch. Restores blocked this way can be
-    recovered by aligning the target schema.
-    """
-    plans: dict[str, TableRestorePlan] = {}
-    errors: list[str] = []
-    for table, table_manifest in manifest["tables"].items():
-        source_columns = [BackupColumn(**column) for column in table_manifest["columns"]]
-        target_by_name = {column.name: column for column in await _table_columns(conn, schema, table)}
-        unknown = [
-            (index, column.name) for index, column in enumerate(source_columns) if column.name not in target_by_name
-        ]
-        mismatched = [
-            f"{column.name} ({column.type_name} in backup, {target_by_name[column.name].type_name} in target)"
-            for column in source_columns
-            if column.name in target_by_name and target_by_name[column.name].type_name != column.type_name
-        ]
-        if mismatched:
-            errors.append(f"{table}: incompatible column types: {', '.join(mismatched)}")
-        if unknown:
-            typer.echo(
-                f"  {table}: ignoring {len(unknown)} backup column(s) absent from the target schema: "
-                f"{', '.join(name for _, name in unknown)}"
-            )
-        plans[table] = TableRestorePlan(
-            columns=[column.name for column in source_columns if column.name in target_by_name],
-            dropped_field_indices=tuple(index for index, _ in unknown),
-            source_field_count=len(source_columns),
-        )
-
-    if errors:
-        details = "; ".join(errors)
-        raise ValueError(f"Backup schema is incompatible with target schema '{schema}': {details}")
-    return plans
-
-
-def _effective_backup_tables() -> list[str]:
-    """Core backup tables plus any bank-scoped tables a loaded extension declares.
-
-    ``BACKUP_TABLES`` covers only the tables core owns. An extension that
-    provisions its own bank-scoped tables (via ``TenantExtension``) declares
-    them through ``extra_bank_tables()`` so they aren't dropped on restore.
-    Extension tables are appended *after* the core set so restore's forward
-    COPY inserts them after their FK parents (e.g. ``banks``) and the reversed
-    TRUNCATE clears them before those parents.
-    """
-    tables = list(BACKUP_TABLES)
-    tenant_extension = load_extension("TENANT", TenantExtension)
-    if tenant_extension is not None:
-        seen = set(tables)
-        for spec in tenant_extension.extra_bank_tables():
-            if spec.include_in_backup and spec.name not in seen:
-                tables.append(spec.name)
-                seen.add(spec.name)
-    return tables
-
-
-async def _admin_connect(db_url: str) -> asyncpg.Connection:
-    """Open a raw asyncpg connection to an admin DB URL.
-
-    ``resolve_database_url`` handles both plain ``postgres://`` (passthrough) and
-    ``pg0://`` (boots the embedded server and returns its real libpq URL), so this
-    is the only step needed to connect. JSON codecs are registered so ``jsonb``
-    columns decode to Python objects (used by the export row dumps).
-    """
-    _pg0 = parse_pg0_url(db_url)
-    is_pg0, instance_name = _pg0.is_pg0, _pg0.instance_name
-    if is_pg0:
-        typer.echo(f"Starting embedded PostgreSQL (instance: {instance_name})...")
-    conn = await asyncpg.connect(await resolve_database_url(db_url))
-    for type_name in ("json", "jsonb"):
-        await conn.set_type_codec(type_name, encoder=json.dumps, decoder=json.loads, schema="pg_catalog")
-    return conn
-
-
-async def _backup(
-    database_url: str,
-    output_path: Path,
-    schema: str = "public",
-    backup_tables: list[str] | None = None,
-) -> dict[str, Any]:
-    """Backup all tables to a zip file using binary COPY protocol.
-
-    ``backup_tables`` defaults to the core ``BACKUP_TABLES``; callers pass the
-    extension-augmented list from ``_effective_backup_tables()``.
-    """
-    backup_tables = backup_tables if backup_tables is not None else BACKUP_TABLES
+async def _backup(database_url: str, output_path: Path, schema: str = "public") -> dict[str, Any]:
+    """Backup all tables to a zip file using binary COPY protocol."""
     conn = await asyncpg.connect(database_url)
     try:
         tables: dict[str, Any] = {}
@@ -291,24 +64,14 @@ async def _backup(
         # entities table was backed up.
         async with conn.transaction(isolation="repeatable_read"):
             with zipfile.ZipFile(output_path, "w", zipfile.ZIP_DEFLATED) as zf:
-                for i, table in enumerate(backup_tables, 1):
-                    typer.echo(f"  [{i}/{len(backup_tables)}] Backing up {table}...", nl=False)
+                for i, table in enumerate(BACKUP_TABLES, 1):
+                    typer.echo(f"  [{i}/{len(BACKUP_TABLES)}] Backing up {table}...", nl=False)
 
                     buffer = io.BytesIO()
 
-                    columns = await _table_columns(conn, schema, table)
-
-                    # Pin the ordered columns into both the stream and manifest.
-                    # PostgreSQL binary COPY does not encode column identities, so
-                    # restore must validate this shape before truncating any data.
+                    # Use binary COPY for exact type preservation
                     # asyncpg requires schema_name as separate parameter
-                    await conn.copy_from_table(
-                        table,
-                        schema_name=schema,
-                        columns=[column.name for column in columns],
-                        output=buffer,
-                        format="binary",
-                    )
+                    await conn.copy_from_table(table, schema_name=schema, output=buffer, format="binary")
 
                     data = buffer.getvalue()
                     zf.writestr(f"{table}.bin", data)
@@ -319,7 +82,6 @@ async def _backup(
                     tables[table] = {
                         "rows": row_count,
                         "size_bytes": len(data),
-                        "columns": [{"name": column.name, "type_name": column.type_name} for column in columns],
                     }
 
                     typer.echo(f" {row_count} rows")
@@ -331,20 +93,8 @@ async def _backup(
         await conn.close()
 
 
-async def _restore(
-    database_url: str,
-    input_path: Path,
-    schema: str = "public",
-    backup_tables: list[str] | None = None,
-) -> dict[str, Any]:
-    """Restore all tables from a zip file using binary COPY protocol.
-
-    ``backup_tables`` defaults to the core ``BACKUP_TABLES``; callers pass the
-    extension-augmented list from ``_effective_backup_tables()``. Tables named
-    here but absent from the archive are truncated then skipped for restore, so
-    a stale extension registration never leaves pre-restore rows behind.
-    """
-    backup_tables = backup_tables if backup_tables is not None else BACKUP_TABLES
+async def _restore(database_url: str, input_path: Path, schema: str = "public") -> dict[str, Any]:
+    """Restore all tables from a zip file using binary COPY protocol."""
     conn = await asyncpg.connect(database_url)
     try:
         with zipfile.ZipFile(input_path, "r") as zf:
@@ -353,42 +103,29 @@ async def _restore(
             if manifest.get("version") != MANIFEST_VERSION:
                 raise ValueError(f"Unsupported backup version: {manifest.get('version')}")
 
-            # Complete the compatibility check before entering the transaction
-            # that truncates tables. This turns historical schema drift into an
-            # actionable error without risking the target's existing data.
-            restore_plans = await _validate_restore_schema(conn, manifest, schema)
-
             # Use a transaction for atomic restore - either all tables are
             # restored or none are, preventing partial/inconsistent state.
             async with conn.transaction():
                 typer.echo("  Clearing existing data...")
                 # Truncate tables in reverse order (respects FK constraints)
-                for table in reversed(backup_tables):
+                for table in reversed(BACKUP_TABLES):
                     qualified_table = _fq_table(table, schema)
                     await conn.execute(f"TRUNCATE TABLE {qualified_table} CASCADE")
 
                 # Restore tables in forward order
-                for i, table in enumerate(backup_tables, 1):
+                for i, table in enumerate(BACKUP_TABLES, 1):
                     filename = f"{table}.bin"
                     if filename not in zf.namelist():
-                        typer.echo(f"  [{i}/{len(backup_tables)}] {table}: skipped (not in backup)")
+                        typer.echo(f"  [{i}/{len(BACKUP_TABLES)}] {table}: skipped (not in backup)")
                         continue
 
                     expected_rows = manifest["tables"].get(table, {}).get("rows", "?")
-                    typer.echo(f"  [{i}/{len(backup_tables)}] Restoring {table}... {expected_rows} rows")
+                    typer.echo(f"  [{i}/{len(BACKUP_TABLES)}] Restoring {table}... {expected_rows} rows")
 
-                    plan = restore_plans[table]
-                    # Strips the fields of any column the target no longer has;
-                    # a no-op when the schemas still line up.
-                    buffer = io.BytesIO(_strip_binary_copy_fields(zf.read(filename), plan))
+                    data = zf.read(filename)
+                    buffer = io.BytesIO(data)
                     # asyncpg requires schema_name as separate parameter
-                    await conn.copy_to_table(
-                        table,
-                        schema_name=schema,
-                        columns=plan.columns,
-                        source=buffer,
-                        format="binary",
-                    )
+                    await conn.copy_to_table(table, schema_name=schema, source=buffer, format="binary")
 
                 # Refresh materialized view
                 typer.echo("  Refreshing materialized views...")
@@ -401,22 +138,20 @@ async def _restore(
 
 async def _run_backup(db_url: str, output: Path, schema: str = "public") -> dict[str, Any]:
     """Resolve database URL and run backup."""
-    _pg0 = parse_pg0_url(db_url)
-    is_pg0, instance_name = _pg0.is_pg0, _pg0.instance_name
+    is_pg0, instance_name, _ = parse_pg0_url(db_url)
     if is_pg0:
         typer.echo(f"Starting embedded PostgreSQL (instance: {instance_name})...")
     resolved_url = await resolve_database_url(db_url)
-    return await _backup(resolved_url, output, schema, backup_tables=_effective_backup_tables())
+    return await _backup(resolved_url, output, schema)
 
 
 async def _run_restore(db_url: str, input_file: Path, schema: str = "public") -> dict[str, Any]:
     """Resolve database URL and run restore."""
-    _pg0 = parse_pg0_url(db_url)
-    is_pg0, instance_name = _pg0.is_pg0, _pg0.instance_name
+    is_pg0, instance_name, _ = parse_pg0_url(db_url)
     if is_pg0:
         typer.echo(f"Starting embedded PostgreSQL (instance: {instance_name})...")
     resolved_url = await resolve_database_url(db_url)
-    return await _restore(resolved_url, input_file, schema, backup_tables=_effective_backup_tables())
+    return await _restore(resolved_url, input_file, schema)
 
 
 @app.command()
@@ -440,7 +175,7 @@ def backup(
     manifest = asyncio.run(_run_backup(config.database_url, output, schema))
 
     total_rows = sum(t["rows"] for t in manifest["tables"].values())
-    typer.echo(f"Backed up {total_rows} rows across {len(manifest['tables'])} tables")
+    typer.echo(f"Backed up {total_rows} rows across {len(BACKUP_TABLES)} tables")
     typer.echo(f"Backup saved to {output}")
 
 
@@ -473,7 +208,7 @@ def restore(
     manifest = asyncio.run(_run_restore(config.database_url, input_file, schema))
 
     total_rows = sum(t["rows"] for t in manifest["tables"].values())
-    typer.echo(f"Restored {total_rows} rows across {len(manifest['tables'])} tables")
+    typer.echo(f"Restored {total_rows} rows across {len(BACKUP_TABLES)} tables")
     typer.echo("Restore complete")
 
 
@@ -482,22 +217,26 @@ async def _run_migration(
     schema: str | None = None,
     base_schema: str = DEFAULT_DATABASE_SCHEMA,
     embedding_dimension: int | None = None,
-    ensure_extensions: bool = True,
 ) -> list[str]:
     """Resolve database URL and run migrations for one schema or all discovered schemas."""
-    from ..migrations import run_migrations_for_schemas
+    from ..migrations import (
+        ensure_embedding_dimension,
+        ensure_text_search_extension,
+        ensure_vector_extension,
+        run_migrations,
+    )
 
-    _pg0 = parse_pg0_url(db_url)
-    is_pg0, instance_name = _pg0.is_pg0, _pg0.instance_name
+    is_pg0, instance_name, _ = parse_pg0_url(db_url)
     if is_pg0:
         typer.echo(f"Starting embedded PostgreSQL (instance: {instance_name})...")
     resolved_url = await resolve_database_url(db_url)
 
     config = HindsightConfig.from_env()
-    tenant_extension = load_extension("TENANT", TenantExtension)
     if schema:
         schemas = [schema]
     else:
+        tenant_extension = load_extension("TENANT", TenantExtension)
+
         schemas = [base_schema or DEFAULT_DATABASE_SCHEMA]
         if tenant_extension:
             tenants = await tenant_extension.list_tenants()
@@ -506,52 +245,35 @@ async def _run_migration(
         # Preserve order while removing duplicates.
         schemas = list(dict.fromkeys(schemas))
 
-    # Migrate up to `migration_concurrency` schemas at once (each in its own
-    # process); within a schema the work stays sequential. Run off the event
-    # loop so the process pool's blocking joins don't stall it.
-    await asyncio.to_thread(
-        run_migrations_for_schemas,
-        resolved_url,
-        schemas,
-        concurrency=config.migration_concurrency,
-        migration_database_url=config.migration_database_url,
-        embedding_dimension=embedding_dimension,
-        vector_extension=config.vector_extension,
-        text_search_extension=config.text_search_extension,
-        pg_search_tokenizer=config.text_search_extension_pg_search_tokenizer,
-        ensure_extensions=ensure_extensions,
-    )
+    for schema in schemas:
+        run_migrations(resolved_url, schema=schema, migration_database_url=config.migration_database_url)
 
-    # After core migrations, provision any extension-owned bank-scoped tables
-    # per schema so extension schema evolves on the same lifecycle as core
-    # schema (rather than via a lazy first-request path).
-    if tenant_extension is not None:
-        await _provision_extra_bank_tables(resolved_url, schemas, tenant_extension)
+    if embedding_dimension is not None:
+        for schema in schemas:
+            ensure_embedding_dimension(
+                resolved_url,
+                embedding_dimension,
+                schema=schema,
+                vector_extension=config.vector_extension,
+            )
+
+    for schema in schemas:
+        ensure_vector_extension(
+            resolved_url,
+            vector_extension=config.vector_extension,
+            schema=schema,
+        )
+
+    for schema in schemas:
+        ensure_text_search_extension(
+            resolved_url,
+            text_search_extension=config.text_search_extension,
+            schema=schema,
+        )
 
     return schemas
 
 
-async def _provision_extra_bank_tables(
-    resolved_url: str, schemas: list[str], tenant_extension: TenantExtension
-) -> None:
-    """Run the tenant extension's table provisioner for each migrated schema.
-
-    Fires after core migrations complete so extension-owned bank tables are
-    created/evolved on the same lifecycle as core schema. A failure aborts the
-    migration command (and names the offending schema) rather than being
-    swallowed — provisioning is idempotent, so the operator can fix and re-run.
-    """
-    for schema in schemas:
-        conn = await asyncpg.connect(resolved_url)
-        try:
-            await tenant_extension.provision_bank_tables(conn, schema)
-        except Exception as e:
-            typer.echo(f"  Failed to provision extension tables for schema '{schema}': {e}", err=True)
-            raise
-        finally:
-            await conn.close()
-
-
 @app.command(name="run-db-migration")
 def run_db_migration(
     schema: str | None = typer.Option(
@@ -565,18 +287,6 @@ def run_db_migration(
         "--embedding-dimension",
         help="Expected embedding dimension to enforce after migrations. Omit to skip dimension sync.",
     ),
-    skip_extension_reconcile: bool = typer.Option(
-        False,
-        "--skip-extension-reconcile",
-        help=(
-            "Skip the post-migration vector / text-search index reconcile. This step only does "
-            "work when the configured backend (HINDSIGHT_API_VECTOR_EXTENSION / "
-            "HINDSIGHT_API_TEXT_SEARCH_EXTENSION) differs from a schema's existing indexes — a "
-            "rare, operator-driven change. Skipping it makes a no-change re-migration over many "
-            "tenant schemas much faster. Only use when you have NOT changed the backend; a "
-            "backend change still needs a normal run to reshape the indexes."
-        ),
-    ),
 ):
     """Run database migrations to the latest version."""
     config = HindsightConfig.from_env()
@@ -590,8 +300,6 @@ def run_db_migration(
         typer.echo(f"Running database migrations for schema: {schema}...")
     else:
         typer.echo("Running database migrations for base schema and all discovered tenant schemas...")
-    if skip_extension_reconcile:
-        typer.echo("Skipping post-migration extension reconcile (--skip-extension-reconcile).")
 
     schemas = asyncio.run(
         _run_migration(
@@ -599,307 +307,15 @@ def run_db_migration(
             schema=schema,
             base_schema=config.database_schema,
             embedding_dimension=embedding_dimension,
-            ensure_extensions=not skip_extension_reconcile,
         )
     )
 
     typer.echo(f"Database migrations completed successfully for {len(schemas)} schema(s)")
 
 
-async def _resolve_schemas(base_schema: str | None) -> list[str]:
-    """Base schema plus every discovered tenant schema, de-duplicated in order."""
-    schemas = [base_schema or DEFAULT_DATABASE_SCHEMA]
-    tenant_extension = load_extension("TENANT", TenantExtension)
-    if tenant_extension:
-        tenants = await tenant_extension.list_tenants()
-        schemas.extend(tenant.schema for tenant in tenants if tenant.schema)
-    return list(dict.fromkeys(schemas))
-
-
-async def _run_repair_bank(
-    db_url: str,
-    *,
-    base_schema: str,
-    schema: str | None,
-    bank_id: str | None,
-    dry_run: bool,
-) -> list[BankIndexResult]:
-    """Reconcile per-(bank, fact_type) vector index coverage over a raw connection.
-
-    A single autocommit connection is used because ``CREATE INDEX CONCURRENTLY``
-    cannot run inside a transaction block.
-
-    Deliberately unbudgeted, unlike the background operation: this is an operator
-    asking for convergence now, across as many banks as they named.
-    """
-    schemas = [schema] if schema else await _resolve_schemas(base_schema)
-    index_clause = _vector_index_clause()
-    # Guarded by the command, but assert so this helper is never called for a
-    # backend without per-bank indexes.
-    assert index_clause is not None
-
-    conn = await _admin_connect(db_url)
-    results: list[BankIndexResult] = []
-    try:
-        for target_schema in schemas:
-            try:
-                bank_ids = [bank_id] if bank_id else await list_bank_ids(conn, target_schema)
-            except Exception as exc:  # noqa: BLE001 — one bad schema must not abort the sweep
-                typer.echo(f"  schema '{target_schema}': skipped ({exc})", err=True)
-                continue
-            schema_results = [
-                await reconcile_bank_vector_indexes(conn, target_schema, bid, index_clause, dry_run=dry_run)
-                for bid in bank_ids
-            ]
-            results.extend(schema_results)
-            # Only in --all mode: an index whose bank row is gone is unreachable
-            # from every bank-scoped path, so this is the one place that can
-            # collect it. Normally finds nothing — delete_bank drops a bank's
-            # indexes while it still knows their names — but a deployment that
-            # hit the #3485 wall could not run delete_bank at all.
-            orphans = [] if bank_id else await drop_orphaned_bank_indexes(conn, target_schema, dry_run=dry_run)
-            if orphans:
-                typer.echo(
-                    f"  schema '{target_schema}': {len(orphans)} orphaned index(es) "
-                    f"{'to drop (dry-run)' if dry_run else 'dropped'} (no matching bank)"
-                )
-            typer.echo(
-                f"  schema '{target_schema}': {len(bank_ids)} bank(s) scanned, "
-                f"{sum(r.already_present for r in schema_results)} present, "
-                f"{sum(r.created for r in schema_results)} created, "
-                f"{sum(r.dropped for r in schema_results)} dropped, "
-                f"{sum(r.skipped for r in schema_results)} to-create (dry-run), "
-                f"{sum(r.would_drop for r in schema_results)} to-drop (dry-run), "
-                f"{sum(r.failed for r in schema_results)} failed"
-            )
-        return results
-    finally:
-        await conn.close()
-
-
-@app.command(name="repair-bank")
-def repair_bank(
-    bank_id: str | None = typer.Option(
-        None,
-        "--bank",
-        "-b",
-        help="Bank id to repair. Mutually exclusive with --all.",
-    ),
-    all_banks: bool = typer.Option(
-        False,
-        "--all",
-        help="Repair every bank in the base schema and all discovered tenant schemas.",
-    ),
-    schema: str | None = typer.Option(
-        None,
-        "--schema",
-        "-s",
-        help="Limit to a single schema. Defaults to the base schema plus discovered tenant schemas.",
-    ),
-    dry_run: bool = typer.Option(
-        False,
-        "--dry-run",
-        help="Report what would be repaired without creating or dropping any index.",
-    ),
-):
-    """Reconcile per-(bank, fact_type) vector index coverage against the size threshold.
-
-    A (bank, fact_type) earns a partial vector index once it holds
-    HINDSIGHT_API_VECTOR_INDEX_MIN_ROWS rows; below that the planner answers the
-    same query exactly, and faster, from the (bank_id, fact_type) B-tree plus a
-    top-N sort. This command builds what qualifies and drops what no longer does
-    — including indexes orphaned by a deleted bank — detecting invalid coverage
-    too (an INVALID leftover, or an index whose access method drifted after a
-    backend switch, counts as missing). All DDL is CONCURRENTLY, so it never
-    blocks the live fleet.
-
-    Writes keep this converged on their own — every insert that could move a bank
-    across the threshold queues a vector_index_maintenance operation. Reach for
-    the command when you want convergence without waiting for a write: after a
-    restore or upgrade, after a backend switch, or to shed indexes in bulk on a
-    deployment recovering from lock-table exhaustion (#3485). Idempotent and safe
-    to re-run.
-    """
-    if bool(bank_id) == all_banks:
-        typer.echo("Error: pass exactly one of --bank <id> or --all.", err=True)
-        raise typer.Exit(2)
-
-    config = HindsightConfig.from_env()
-    if not config.database_url:
-        typer.echo("Error: Database URL not configured.", err=True)
-        typer.echo("Set HINDSIGHT_API_DATABASE_URL environment variable.", err=True)
-        raise typer.Exit(1)
-
-    # Backend guard: backends with a single global vector index (AlloyDB ScaNN,
-    # Oracle) have no per-bank indexes to repair.
-    if _vector_index_clause() is None:
-        typer.echo("Configured vector backend does not use per-bank vector indexes — nothing to repair.")
-        return
-
-    target = f"bank '{bank_id}'" if bank_id else "all banks"
-    scope = f"schema '{schema}'" if schema else "base schema and all discovered tenant schemas"
-    typer.echo(f"Repairing per-bank vector indexes for {target} across {scope}...")
-    if dry_run:
-        typer.echo("Dry run: no indexes will be created or dropped.")
-
-    results = asyncio.run(
-        _run_repair_bank(
-            config.database_url,
-            base_schema=config.database_schema,
-            schema=schema,
-            bank_id=bank_id,
-            dry_run=dry_run,
-        )
-    )
-
-    total_banks = len(results)
-    total_present = sum(r.already_present for r in results)
-    total_created = sum(r.created for r in results)
-    total_dropped = sum(r.dropped for r in results)
-    total_skipped = sum(r.skipped for r in results)
-    total_would_drop = sum(r.would_drop for r in results)
-    total_failed = sum(r.failed for r in results)
-    typer.echo(
-        f"Done: {len(results)} schema(s), {total_banks} bank(s) scanned, "
-        f"{total_present} already present, {total_created} created, {total_dropped} dropped, "
-        f"{total_skipped} to-create (dry-run), {total_would_drop} to-drop (dry-run), "
-        f"{total_failed} failed"
-    )
-    if total_failed:
-        failed_names = [name for r in results for name in r.failed_indexes]
-        typer.echo(f"Failed indexes (dropped, retry with a re-run): {', '.join(failed_names)}", err=True)
-        raise typer.Exit(1)
-
-
-async def _run_export_bank(db_url: str, bank_id: str, output: Path, schema: str, include_history: bool) -> int:
-    """Export a whole bank to a ZIP archive."""
-    conn = await _admin_connect(db_url)
-    try:
-        # export_bank resolves table names via fq_table (the _current_schema
-        # contextvar); set it so the raw connection targets the right schema.
-        _current_schema.set(schema)
-        # _admin_connect registers JSON codecs, so row dumps already contain
-        # decoded Python values (including JSON scalar strings).
-        data = await export_bank(
-            conn,
-            bank_id,
-            include_history=include_history,
-            bank_rows_json_encoding="decoded",
-        )
-    finally:
-        await conn.close()
-
-    output.write_bytes(data)
-    return len(data)
-
-
-@app.command(name="export-bank")
-def export_bank_command(
-    bank_id: str = typer.Option(..., "--bank", "-b", help="Bank id to export."),
-    output: Path = typer.Option(..., "--output", "-o", help="Path to write the .zip archive."),
-    schema: str | None = typer.Option(
-        None,
-        "--schema",
-        "-s",
-        help="Database schema the bank lives in. Defaults to the configured base schema.",
-    ),
-    include_history: bool = typer.Option(
-        False,
-        "--include-history",
-        help="Also export operational history (audit_log, llm_requests). Off by default.",
-    ),
-):
-    """Export an entire bank to a portable ZIP (no embeddings — regenerated on import).
-
-    Carries documents, facts, observations, bank config, mental models, directives
-    and webhooks so the bank can be imported into a new instance configured with a
-    different embedding model / vector / text-search backend.
-    """
-    config = HindsightConfig.from_env()
-
-    if not config.database_url:
-        typer.echo("Error: Database URL not configured.", err=True)
-        typer.echo("Set HINDSIGHT_API_DATABASE_URL environment variable.", err=True)
-        raise typer.Exit(1)
-
-    target_schema = schema or config.database_schema or DEFAULT_DATABASE_SCHEMA
-    typer.echo(f"Exporting bank '{bank_id}' from schema '{target_schema}'...")
-
-    size = asyncio.run(_run_export_bank(config.database_url, bank_id, output, target_schema, include_history))
-
-    typer.echo(f"Exported bank '{bank_id}' to {output} ({size} bytes)")
-
-
-async def _run_import_bank(archive_path: Path, schema: str, target_bank_id: str | None, include_history: bool):
-    """Boot a MemoryEngine (for the target's embedding model) and restore a bank archive."""
-    # MemoryEngine is heavy (loads embeddings); import it lazily so other admin
-    # commands don't pay for it. _current_schema is imported at module top.
-    from ..engine.memory_engine import MemoryEngine
-    from ..models import RequestContext
-
-    archive_bytes = archive_path.read_bytes()
-    # run_migrations=True so a fresh target instance is provisioned at this
-    # instance's embedding dimension / vector / text-search backend before restore.
-    engine = MemoryEngine(run_migrations=True)
-    await engine.initialize()
-    try:
-        _current_schema.set(schema)
-        context = RequestContext(internal=True, user_initiated=True)
-        return await engine.import_bank_async(
-            archive_bytes,
-            context,
-            target_bank_id=target_bank_id,
-            include_history=include_history,
-        )
-    finally:
-        await engine.close()
-
-
-@app.command(name="import-bank")
-def import_bank_command(
-    archive: Path = typer.Option(..., "--archive", "-a", help="Path to the .zip produced by export-bank."),
-    schema: str | None = typer.Option(
-        None, "--schema", "-s", help="Target schema. Defaults to the configured base schema."
-    ),
-    target_bank: str | None = typer.Option(
-        None, "--target-bank", help="Override the bank id (defaults to the archive's source bank)."
-    ),
-    include_history: bool = typer.Option(
-        False, "--include-history", help="Also restore operational history if present in the archive."
-    ),
-):
-    """Restore a whole bank from an export-bank archive into THIS instance.
-
-    Re-embeds facts with this instance's configured embedding model and rebuilds
-    links and indexes — the import half of a cross-instance migration. Run against
-    an instance configured with the desired embedding / vector / text-search backend.
-    The target bank must not already exist (import restores a whole bank, not a merge).
-    """
-    config = HindsightConfig.from_env()
-    if not config.database_url:
-        typer.echo("Error: Database URL not configured.", err=True)
-        typer.echo("Set HINDSIGHT_API_DATABASE_URL environment variable.", err=True)
-        raise typer.Exit(1)
-
-    target_schema = schema or config.database_schema or DEFAULT_DATABASE_SCHEMA
-    typer.echo(f"Importing bank archive '{archive}' into schema '{target_schema}'...")
-
-    result = asyncio.run(_run_import_bank(archive, target_schema, target_bank, include_history))
-
-    typer.echo(
-        f"Imported bank '{result.bank_id}': {result.documents_imported} doc(s), "
-        f"{result.facts_imported} fact(s), {result.observations_imported} observation(s), "
-        f"{result.mental_models_imported} mental model(s), "
-        f"{result.mental_model_history_imported} mm-history row(s), "
-        f"{result.knowledge_pages_imported} knowledge page(s), {result.directives_imported} directive(s), "
-        f"{result.webhooks_imported} webhook(s), {result.history_rows_imported} history row(s)"
-    )
-
-
 async def _decommission_worker(db_url: str, worker_id: str, schema: str = "public") -> int:
     """Release all tasks owned by a worker, setting them back to pending status."""
-    _pg0 = parse_pg0_url(db_url)
-    is_pg0, instance_name = _pg0.is_pg0, _pg0.instance_name
+    is_pg0, instance_name, _ = parse_pg0_url(db_url)
     if is_pg0:
         typer.echo(f"Starting embedded PostgreSQL (instance: {instance_name})...")
     resolved_url = await resolve_database_url(db_url)
@@ -958,8 +374,7 @@ def decommission_worker(
 
 async def _decommission_all_workers(db_url: str, schema: str = "public") -> list[dict[str, Any]]:
     """Release all processing tasks from all workers, setting them back to pending status."""
-    _pg0 = parse_pg0_url(db_url)
-    is_pg0, instance_name = _pg0.is_pg0, _pg0.instance_name
+    is_pg0, instance_name, _ = parse_pg0_url(db_url)
     if is_pg0:
         typer.echo(f"Starting embedded PostgreSQL (instance: {instance_name})...")
     resolved_url = await resolve_database_url(db_url)
@@ -1024,8 +439,7 @@ def decommission_workers(
 
 async def _worker_status(db_url: str, schema: str = "public") -> list[dict[str, Any]]:
     """Get all processing tasks grouped by worker with their last update time."""
-    _pg0 = parse_pg0_url(db_url)
-    is_pg0, instance_name = _pg0.is_pg0, _pg0.instance_name
+    is_pg0, instance_name, _ = parse_pg0_url(db_url)
     if is_pg0:
         typer.echo(f"Starting embedded PostgreSQL (instance: {instance_name})...")
     resolved_url = await resolve_database_url(db_url)
@@ -1093,7 +507,6 @@ def worker_status(
 
 
 def main():
-    load_dotenv_for_entrypoint()
     app()
 
 

hindsight-api-slim/hindsight_api/alembic/_dialect.py

@@ -1,42 +0,0 @@
-"""Dialect dispatcher for Alembic migrations.
-
-Each migration file declares a ``_pg_upgrade``/``_oracle_upgrade`` (and matching
-downgrades) function and routes ``upgrade()``/``downgrade()`` through
-``run_for_dialect``. The helper inspects the live connection's dialect name and
-runs the matching function — or no-ops if the migration doesn't apply to the
-current backend.
-
-Use ``None`` (or omit the kwarg) when a migration intentionally has no effect
-on a dialect; the helper treats it as a no-op.
-"""
-
-from __future__ import annotations
-
-from collections.abc import Callable
-
-from alembic import op
-
-DialectFn = Callable[[], None]
-_SUPPORTED = ("postgresql", "oracle")
-
-
-def run_for_dialect(
-    *,
-    pg: DialectFn | None = None,
-    oracle: DialectFn | None = None,
-) -> None:
-    """Dispatch to the function matching the current bind's dialect.
-
-    Args:
-        pg: Function to run when the active bind is PostgreSQL.
-        oracle: Function to run when the active bind is Oracle.
-
-    Unrecognized dialects raise; an explicit ``None`` for the active dialect
-    is a no-op (the migration deliberately does nothing here).
-    """
-    name = op.get_bind().dialect.name
-    if name not in _SUPPORTED:
-        raise RuntimeError(f"Unsupported dialect for migration dispatch: {name!r}. Expected one of {_SUPPORTED}.")
-    fn = {"postgresql": pg, "oracle": oracle}[name]
-    if fn is not None:
-        fn()

hindsight-api-slim/hindsight_api/alembic/env.py

@@ -1,37 +1,29 @@
 """
-Alembic environment for Hindsight.
-
-Supports two dialects:
-
-* PostgreSQL (sync psycopg2 driver) — default; uses ``search_path`` for
-  multi-tenant schema isolation and forces read-write transactions to work
-  around Supabase's read-only-by-default sessions.
-* Oracle 23ai (``oracledb`` driver) — uses ``CURRENT_SCHEMA`` for tenant
-  isolation; no equivalent of ``search_path`` or read-only session quirks.
-
-Each migration file dispatches its DDL through ``alembic._dialect.run_for_dialect``
-so a single revision tree serves both backends.
+Alembic environment configuration for SQLAlchemy with pgvector.
+Uses synchronous psycopg2 driver for migrations to avoid pgbouncer issues.
 """
 
 import logging
 import os
 from pathlib import Path
-from urllib.parse import parse_qsl, urlencode, urlsplit, urlunsplit
 
 from alembic import context
 from dotenv import load_dotenv
-from sqlalchemy import Connection, engine_from_config, pool
-from sqlalchemy.engine import Engine
+from sqlalchemy import engine_from_config, pool
 
-from hindsight_api.db_url import is_oracle_url, to_libpq_url
+# Import your models here
+from hindsight_api.db_url import to_libpq_url
 from hindsight_api.models import Base
 
 
-def load_env() -> None:
-    """Load environment variables from .env (skipped if already configured)."""
+# Load environment variables based on HINDSIGHT_API_DATABASE_URL env var or default to local
+def load_env():
+    """Load environment variables from .env"""
+    # Check if HINDSIGHT_API_DATABASE_URL is already set (e.g., by CI/CD)
     if os.getenv("HINDSIGHT_API_DATABASE_URL"):
         return
 
+    # Look for .env file in the parent directory (root of the workspace)
     root_dir = Path(__file__).parent.parent.parent
     env_file = root_dir / ".env"
 
@@ -41,45 +33,30 @@ def load_env() -> None:
 
 load_env()
 
+# this is the Alembic Config object, which provides
+# access to the values within the .ini file in use.
 config = context.config
+
+# Note: We don't call fileConfig() here to avoid overriding the application's logging configuration.
+# Alembic will use the existing logging configuration from the application.
+
+# add your model's MetaData object here
+# for 'autogenerate' support
 target_metadata = Base.metadata
 
-
-def _normalize_oracle_url(url: str) -> str:
-    """Coerce an Oracle URL into the SQLAlchemy form the oracledb dialect expects.
-
-    Two issues to handle:
-
-    1. Force the ``oracle+oracledb`` driver — bare ``oracle://`` defaults to
-       cx_Oracle.
-    2. Map a path-style service to ``?service_name=...``. SQLAlchemy's oracledb
-       dialect treats the URL path as a *SID* (legacy), but Oracle Free /
-       Autonomous DB only register a service name. Without this rewrite we get
-       ``DPY-6003: SID "FREEPDB1" is not registered`` even though the listener
-       is happy to accept the same name as a service.
-    """
-    parts = urlsplit(url)
-    if not parts.scheme.startswith("oracle"):
-        return url
-
-    new_scheme = "oracle+oracledb" if parts.scheme == "oracle" else parts.scheme
-    service = parts.path.lstrip("/")
-    new_query = parts.query
-    new_path = parts.path
-
-    # Promote /SERVICE to ?service_name=SERVICE unless the caller already
-    # supplied an explicit ?sid= or ?service_name=.
-    if service and "service_name=" not in new_query and "sid=" not in new_query:
-        params = [(k, v) for k, v in parse_qsl(new_query, keep_blank_values=True)]
-        params.append(("service_name", service))
-        new_query = urlencode(params)
-        new_path = ""
-
-    return urlunsplit((new_scheme, parts.netloc, new_path, new_query, parts.fragment))
+# other values from the config, defined by the needs of env.py,
+# can be acquired:
+# my_important_option = config.get_main_option("my_important_option")
+# ... etc.
 
 
 def get_database_url() -> str:
-    """Resolve the migration URL from Alembic config or env, normalizing per-dialect."""
+    """
+    Get and process the database URL from config or environment.
+
+    Returns the URL with the correct driver (psycopg2) for migrations.
+    """
+    # Get database URL from config (set programmatically) or environment
     database_url = config.get_main_option("sqlalchemy.url")
     if not database_url:
         database_url = os.getenv("HINDSIGHT_API_DATABASE_URL")
@@ -89,19 +66,30 @@ def get_database_url() -> str:
                 "Set HINDSIGHT_API_DATABASE_URL environment variable or pass database_url to run_migrations()."
             )
 
-    if is_oracle_url(database_url):
-        database_url = _normalize_oracle_url(database_url)
-    else:
-        # PG: convert SQLAlchemy-style asyncpg URLs and ?ssl= params to libpq form
-        # for the sync engine used during migrations.
-        database_url = to_libpq_url(database_url)
+    # For migrations, use the sync psycopg2 driver (avoids pgbouncer prepared
+    # statement issues and is required since create_engine is the sync API).
+    # Also translates ?ssl=require (SQLAlchemy asyncpg style) to ?sslmode=require
+    # (libpq style) for external-PostgreSQL deployments.
+    database_url = to_libpq_url(database_url)
+
+    # Update config with processed URL for engine_from_config to use
+    config.set_main_option("sqlalchemy.url", database_url)
 
-    # Alembic stores options through ConfigParser, where '%' is interpolation.
-    config.set_main_option("sqlalchemy.url", database_url.replace("%", "%%"))
     return database_url
 
 
 def run_migrations_offline() -> None:
+    """Run migrations in 'offline' mode.
+
+    This configures the context with just a URL
+    and not an Engine, though an Engine is acceptable
+    here as well.  By skipping the Engine creation
+    we don't even need a DBAPI to be available.
+
+    Calls to context.execute() here emit the given string to the
+    script output.
+
+    """
     logging.info("running offline")
     database_url = get_database_url()
 
@@ -116,40 +104,14 @@ def run_migrations_offline() -> None:
         context.run_migrations()
 
 
-def _configure_pg_session(engine: Engine, connection: Connection, target_schema: str | None) -> None:
-    """PG-only: ensure the session is RW (Supabase) and bind ``search_path``."""
+def run_migrations_online() -> None:
+    """Run migrations in 'online' mode with synchronous engine."""
     from sqlalchemy import event, text
 
-    @event.listens_for(engine, "connect")
-    def set_read_write_mode(dbapi_connection, connection_record):
-        cursor = dbapi_connection.cursor()
-        cursor.execute("SET SESSION CHARACTERISTICS AS TRANSACTION READ WRITE")
-        if target_schema:
-            cursor.execute(f'CREATE SCHEMA IF NOT EXISTS "{target_schema}"')
-            cursor.execute(f'SET search_path TO "{target_schema}", public')
-        cursor.close()
+    get_database_url()  # Process and set the database URL in config
 
-    connection.execute(text("SET SESSION CHARACTERISTICS AS TRANSACTION READ WRITE"))
-    if target_schema:
-        connection.execute(text(f'CREATE SCHEMA IF NOT EXISTS "{target_schema}"'))
-        connection.execute(text(f'SET search_path TO "{target_schema}", public'))
-    connection.commit()
-
-
-def _configure_oracle_session(connection: Connection, target_schema: str | None) -> None:
-    """Oracle: switch the session's default schema; tolerate DDL contention."""
-    from sqlalchemy import text
-
-    # Wait up to 30s for DDL locks instead of failing immediately (ORA-00054).
-    connection.execute(text("ALTER SESSION SET DDL_LOCK_TIMEOUT = 30"))
-    if target_schema:
-        connection.execute(text(f'ALTER SESSION SET CURRENT_SCHEMA = "{target_schema}"'))
-
-
-def run_migrations_online() -> None:
-    database_url = get_database_url()
+    # Check if we're targeting a specific schema (for multi-tenant isolation)
     target_schema = config.get_main_option("target_schema")
-    is_oracle = is_oracle_url(database_url)
 
     connectable = engine_from_config(
         config.get_section(config.config_ini_section, {}),
@@ -157,19 +119,37 @@ def run_migrations_online() -> None:
         poolclass=pool.NullPool,
     )
 
-    with connectable.connect() as connection:
-        if is_oracle:
-            _configure_oracle_session(connection, target_schema)
-        else:
-            _configure_pg_session(connectable, connection, target_schema)
+    # Add event listener to ensure connection is in read-write mode
+    # This is needed for Supabase which may start connections in read-only mode
+    @event.listens_for(connectable, "connect")
+    def set_read_write_mode(dbapi_connection, connection_record):
+        cursor = dbapi_connection.cursor()
+        cursor.execute("SET SESSION CHARACTERISTICS AS TRANSACTION READ WRITE")
+        # If targeting a specific schema, set search_path
+        # Include public in search_path for access to shared extensions (pgvector)
+        if target_schema:
+            cursor.execute(f'CREATE SCHEMA IF NOT EXISTS "{target_schema}"')
+            cursor.execute(f'SET search_path TO "{target_schema}", public')
+        cursor.close()
 
+    with connectable.connect() as connection:
+        # Also explicitly set read-write mode on this connection
+        connection.execute(text("SET SESSION CHARACTERISTICS AS TRANSACTION READ WRITE"))
+
+        # If targeting a specific schema, set search_path
+        # Include public in search_path for access to shared extensions (pgvector)
+        if target_schema:
+            connection.execute(text(f'CREATE SCHEMA IF NOT EXISTS "{target_schema}"'))
+            connection.execute(text(f'SET search_path TO "{target_schema}", public'))
+
+        connection.commit()  # Commit the SET command
+
+        # Configure context with version_table_schema if using a specific schema
         context_opts = {
             "connection": connection,
             "target_metadata": target_metadata,
         }
-        if target_schema and not is_oracle:
-            # Oracle has no equivalent of PG's per-schema version table; the
-            # ``alembic_version`` table lives in CURRENT_SCHEMA implicitly.
+        if target_schema:
             context_opts["version_table_schema"] = target_schema
 
         context.configure(**context_opts)
@@ -177,12 +157,7 @@ def run_migrations_online() -> None:
         with context.begin_transaction():
             context.run_migrations()
 
-        # Always commit. PG needs it for the explicit RW-mode SET to persist;
-        # Oracle needs it because each DDL auto-commits but the trailing
-        # ``UPDATE alembic_version`` is plain DML that would otherwise stay in
-        # an open transaction and roll back when the connection closes —
-        # producing the "schema is created but the version row is one revision
-        # behind" failure mode.
+        # Explicit commit to ensure changes are persisted (especially for Supabase)
         connection.commit()
 
 

hindsight-api-slim/hindsight_api/alembic/script.py.mako

@@ -11,8 +11,6 @@ from alembic import op
 import sqlalchemy as sa
 ${imports if imports else ""}
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 # revision identifiers, used by Alembic.
 revision: str = ${repr(up_revision)}
 down_revision: Union[str, Sequence[str], None] = ${repr(down_revision)}
@@ -20,27 +18,11 @@ branch_labels: Union[str, Sequence[str], None] = ${repr(branch_labels)}
 depends_on: Union[str, Sequence[str], None] = ${repr(depends_on)}
 
 
-def _pg_upgrade() -> None:
-    """PostgreSQL upgrade. Set to ``None`` below if this migration is Oracle-only."""
+def upgrade() -> None:
+    """Upgrade schema."""
     ${upgrades if upgrades else "pass"}
 
 
-def _pg_downgrade() -> None:
-    ${downgrades if downgrades else "pass"}
-
-
-def _oracle_upgrade() -> None:
-    """Oracle upgrade. Set to ``None`` below if this migration is Postgres-only."""
-    pass
-
-
-def _oracle_downgrade() -> None:
-    pass
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade, oracle=_oracle_upgrade)
-
-
 def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade, oracle=_oracle_downgrade)
+    """Downgrade schema."""
+    ${downgrades if downgrades else "pass"}

hindsight-api-slim/hindsight_api/alembic/versions/2071c7518f88_add_memory_links_bank_id_index.py

@@ -1,105 +0,0 @@
-"""Add a composite index on memory_links(bank_id, link_type) (PostgreSQL).
-
-``bank_id`` was added to ``memory_links`` in ``c5d6e7f8a9b0`` precisely so that
-bank-scoped reads (e.g. the stats endpoint) could filter on the link table
-directly instead of joining ``memory_units`` — that JOIN took 18+ seconds on
-banks with millions of links. The column landed without an index, so every
-``bank_id = $1`` predicate still falls back to a sequential scan over the whole
-table.
-
-This adds the missing btree. It is composite on ``(bank_id, link_type)`` rather
-than ``bank_id`` alone because the hot query is the stats endpoint's
-``SELECT link_type, COUNT(*) ... WHERE bank_id = $1 GROUP BY link_type``: a
-``(bank_id, link_type)`` index serves that filter, grouping and count as an
-index-only scan, never touching the heap, whereas a ``bank_id``-only index would
-still have to read every matching row to recover ``link_type``. ``link_type`` is
-low-cardinality (only ``temporal``/``semantic``/``caused_by`` are written —
-entity edges were dropped in ``e9b2c7d1f3a4``), so the trailing column adds
-little to the index size while removing the heap fetch.
-
-The Oracle baseline (``o1a2b3c4d5e6``) already creates ``idx_ml_bank_id`` on
-``memory_links(bank_id)``; that single-column index already covers Oracle's
-bank-scoped filter, so the Oracle slot here is intentionally absent and only the
-PostgreSQL dialect gets the composite index.
-
-``memory_links`` can hold tens of millions of rows, so the index is built
-CONCURRENTLY to avoid taking a write lock on the table. CONCURRENTLY cannot run
-inside a transaction block, so the statement runs in an ``autocommit_block()``;
-``IF NOT EXISTS`` keeps it idempotent across retries and re-migrated tenant
-schemas. A CONCURRENTLY build interrupted partway (lock conflict, disk
-pressure, signal) leaves the index behind as *invalid*; ``IF NOT EXISTS`` would
-then skip over it forever, so the upgrade first drops any invalid leftover of
-this name before (re)creating it.
-
-Revision ID: 2071c7518f88
-Revises: a1d3f5b7c9e2
-Create Date: 2026-06-16
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-from sqlalchemy import text
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "2071c7518f88"
-down_revision: str | Sequence[str] | None = "a1d3f5b7c9e2"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-_INDEX_NAME = "idx_memory_links_bank_id_link_type"
-
-
-def _get_schema_prefix() -> str:
-    """Schema-qualifier for raw SQL on PG (multi-tenant search_path)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _pg_upgrade() -> None:
-    bind = op.get_bind()
-    # `or None` collapses an unset option and an explicit empty string into NULL
-    # so the COALESCE below falls back to current_schema() in both cases.
-    target_schema = context.config.get_main_option("target_schema") or None
-    schema = _get_schema_prefix()
-
-    # CREATE INDEX CONCURRENTLY cannot run inside a transaction block; the
-    # autocommit_block runs each statement outside Alembic's migration
-    # transaction.
-    with op.get_context().autocommit_block():
-        # A CONCURRENTLY build that errored on a previous run leaves an INVALID
-        # index of this name behind. `CREATE INDEX ... IF NOT EXISTS` would see
-        # that relation and skip, so bank_id queries would keep seq-scanning.
-        # Drop only the invalid leftover — never a healthy index — so the retry
-        # actually rebuilds a usable one.
-        leftover_invalid = bind.execute(
-            text(
-                "SELECT NOT i.indisvalid "
-                "FROM pg_class c "
-                "JOIN pg_index i ON c.oid = i.indexrelid "
-                "JOIN pg_namespace n ON c.relnamespace = n.oid "
-                "WHERE c.relname = :index_name "
-                "  AND n.nspname = COALESCE(:target_schema, current_schema())"
-            ),
-            {"index_name": _INDEX_NAME, "target_schema": target_schema},
-        ).scalar()
-        if leftover_invalid:
-            op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}{_INDEX_NAME}")
-
-        # IF NOT EXISTS keeps the create idempotent across retries and schemas.
-        op.execute(f"CREATE INDEX CONCURRENTLY IF NOT EXISTS {_INDEX_NAME} ON {schema}memory_links(bank_id, link_type)")
-
-
-def _pg_downgrade() -> None:
-    schema = _get_schema_prefix()
-    with op.get_context().autocommit_block():
-        op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}{_INDEX_NAME}")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/2eee35aa3cfc_case_insensitive_entities_trgm_index.py

@@ -13,8 +13,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "2eee35aa3cfc"
 down_revision: str | Sequence[str] | None = "d6e7f8a9b0c1"
 branch_labels: str | Sequence[str] | None = None
@@ -26,7 +24,7 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     schema = _get_schema_prefix()
     # Drop the old case-sensitive trigram index
     op.execute("DROP INDEX IF EXISTS entities_canonical_name_trgm_idx")
@@ -37,7 +35,7 @@ def _pg_upgrade() -> None:
     )
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     op.execute("DROP INDEX IF EXISTS entities_canonical_name_lower_trgm_idx")
     schema = _get_schema_prefix()
     # Restore original case-sensitive index
@@ -45,11 +43,3 @@ def _pg_downgrade() -> None:
         f"CREATE INDEX IF NOT EXISTS entities_canonical_name_trgm_idx "
         f"ON {schema}entities USING GIN (canonical_name gin_trgm_ops)"
     )
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/5a366d414dce_initial_schema.py

@@ -15,13 +15,6 @@ from pgvector.sqlalchemy import Vector
 from sqlalchemy import text
 from sqlalchemy.dialects import postgresql
 
-from hindsight_api._pg_search import (
-    PG_SEARCH_TOKENIZER_ENV,
-    normalize_pg_search_tokenizer,
-    pg_search_bm25_columns,
-)
-from hindsight_api.alembic._dialect import run_for_dialect
-
 # revision identifiers, used by Alembic.
 revision: str = "5a366d414dce"
 down_revision: str | Sequence[str] | None = None
@@ -31,79 +24,59 @@ depends_on: str | Sequence[str] | None = None
 
 def _detect_vector_extension() -> str:
     """
-    Detect or validate vector extension for this immutable migration revision.
+    Detect or validate vector extension: 'pgvector', 'vchord', or 'pgvectorscale'.
     Respects HINDSIGHT_API_VECTOR_EXTENSION env var if set.
     """
     conn = op.get_bind()
     vector_extension = os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector").lower()
 
+    # Validate configured extension is installed
     if vector_extension == "pgvectorscale":
+        # pgvectorscale/DiskANN requires pgvector
         pgvector_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vector'")).scalar()
         if not pgvector_check:
             raise RuntimeError(
                 "DiskANN requires pgvector. Install with: CREATE EXTENSION vector; then vectorscale or pg_diskann CASCADE;"
             )
+        # Check for either vectorscale (open source) or pg_diskann (Azure)
         vectorscale_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vectorscale'")).scalar()
         pg_diskann_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'pg_diskann'")).scalar()
 
         if vectorscale_check:
             return "pgvectorscale"
-        if pg_diskann_check:
+        elif pg_diskann_check:
             return "pg_diskann"
-        raise RuntimeError(
-            "Configured vector extension 'pgvectorscale' not found. Install either:\n"
-            "  - pgvectorscale: CREATE EXTENSION vectorscale CASCADE;\n"
-            "  - pg_diskann (Azure): CREATE EXTENSION pg_diskann CASCADE;"
-        )
-    if vector_extension == "vchord":
+        else:
+            raise RuntimeError(
+                "Configured vector extension 'pgvectorscale' not found. Install either:\n"
+                "  - pgvectorscale: CREATE EXTENSION vectorscale CASCADE;\n"
+                "  - pg_diskann (Azure): CREATE EXTENSION pg_diskann CASCADE;"
+            )
+    elif vector_extension == "vchord":
         vchord_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vchord'")).scalar()
         if not vchord_check:
             raise RuntimeError(
                 "Configured vector extension 'vchord' not found. Install it with: CREATE EXTENSION vchord CASCADE;"
             )
         return "vchord"
-    if vector_extension == "scann":
-        scann_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'alloydb_scann'")).scalar()
-        if not scann_check:
-            raise RuntimeError(
-                "Configured vector extension 'scann' not found. Install it with: CREATE EXTENSION alloydb_scann CASCADE;"
-            )
-        return "scann"
-    if vector_extension == "pgvector":
+    elif vector_extension == "pgvector":
         pgvector_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vector'")).scalar()
         if not pgvector_check:
             raise RuntimeError(
                 "Configured vector extension 'pgvector' not found. Install it with: CREATE EXTENSION vector;"
             )
         return "pgvector"
-    raise ValueError(
-        "Invalid HINDSIGHT_API_VECTOR_EXTENSION: "
-        f"{vector_extension}. Must be 'pgvector', 'vchord', 'pgvectorscale', or 'scann'"
-    )
-
-
-def _vector_index_using_clause(ext: str) -> str:
-    if ext == "pgvectorscale":
-        return "USING diskann (embedding vector_cosine_ops) WITH (num_neighbors = 50)"
-    if ext == "pg_diskann":
-        return "USING diskann (embedding vector_cosine_ops) WITH (max_neighbors = 50)"
-    if ext == "vchord":
-        return "USING vchordrq (embedding vector_cosine_ops)"
-    if ext == "scann":
-        return "USING scann (embedding cosine) WITH (mode = 'AUTO')"
-    return "USING hnsw (embedding vector_cosine_ops)"
+    else:
+        raise ValueError(
+            f"Invalid HINDSIGHT_API_VECTOR_EXTENSION: {vector_extension}. Must be 'pgvector', 'vchord', or 'pgvectorscale'"
+        )
 
 
 def _detect_text_search_extension() -> str:
     """
-    Detect or validate text search extension: 'native', 'vchord', 'pg_textsearch',
-    'pgroonga', or 'pg_search'. Respects HINDSIGHT_API_TEXT_SEARCH_EXTENSION env var.
+    Detect or validate text search extension: 'native', 'vchord', or 'pg_textsearch'.
+    Respects HINDSIGHT_API_TEXT_SEARCH_EXTENSION env var.
     Creates the extension if needed.
-
-    pgroonga is treated as native here so the initial schema still creates valid
-    tsvector columns. ensure_text_search_extension() at startup converts the
-    schema to pgroonga structures (drops the tsvector column, builds a pgroonga
-    index on the base text column).
     """
     text_search_extension = os.getenv("HINDSIGHT_API_TEXT_SEARCH_EXTENSION", "native").lower()
 
@@ -131,36 +104,15 @@ def _detect_text_search_extension() -> str:
                 # Extension truly doesn't exist - re-raise the error
                 raise
         return "pg_textsearch"
-    elif text_search_extension == "pg_search":
-        # ParadeDB pg_search — true BM25 over base columns, Citus-compatible.
-        try:
-            op.execute("CREATE EXTENSION IF NOT EXISTS pg_search CASCADE")
-        except Exception:
-            # Extension might already exist or user lacks permissions - verify it exists
-            conn = op.get_bind()
-            result = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'pg_search'")).fetchone()
-            if not result:
-                # Extension truly doesn't exist - re-raise the error
-                raise
-        return "pg_search"
     elif text_search_extension == "native":
         return "native"
-    elif text_search_extension == "pgroonga":
-        # ensure_text_search_extension() at runtime converts to pgroonga.
-        # Treat as native here so the initial schema still creates valid columns.
-        return "native"
     else:
         raise ValueError(
-            f"Invalid HINDSIGHT_API_TEXT_SEARCH_EXTENSION: {text_search_extension}. "
-            "Must be 'native', 'vchord', 'pg_textsearch', 'pgroonga', or 'pg_search'"
+            f"Invalid HINDSIGHT_API_TEXT_SEARCH_EXTENSION: {text_search_extension}. Must be 'native', 'vchord', or 'pg_textsearch'"
         )
 
 
-def _pg_search_tokenizer() -> str:
-    return normalize_pg_search_tokenizer(os.getenv(PG_SEARCH_TOKENIZER_ENV))
-
-
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     """Upgrade schema - create all tables from scratch."""
 
     # Note: pgvector extension is installed globally BEFORE migrations run
@@ -315,9 +267,8 @@ def _pg_upgrade() -> None:
             ALTER TABLE memory_units
             ADD COLUMN search_vector bm25_catalog.bm25vector
         """)
-    elif text_search_ext in ("pg_textsearch", "pg_search"):
-        # Timescale pg_textsearch / ParadeDB pg_search: dummy TEXT column for
-        # consistency (indexes operate on base columns directly).
+    elif text_search_ext == "pg_textsearch":
+        # Timescale pg_textsearch: dummy TEXT column for consistency (indexes operate on base columns directly)
         op.execute("""
             ALTER TABLE memory_units
             ADD COLUMN search_vector TEXT
@@ -360,11 +311,36 @@ def _pg_upgrade() -> None:
     )
     # Create vector index - conditional based on available extension
     vector_ext = _detect_vector_extension()
-    if vector_ext != "scann":
-        op.execute(f"""
+
+    if vector_ext == "pgvectorscale":
+        # Use DiskANN index for pgvectorscale (disk-based, scalable)
+        op.execute("""
             CREATE INDEX idx_memory_units_embedding ON memory_units
-            {_vector_index_using_clause(vector_ext)}
+            USING diskann (embedding vector_cosine_ops)
+            WITH (num_neighbors = 50)
         """)
+    elif vector_ext == "pg_diskann":
+        # Use DiskANN index for pg_diskann (Azure)
+        op.execute("""
+            CREATE INDEX idx_memory_units_embedding ON memory_units
+            USING diskann (embedding vector_cosine_ops)
+            WITH (max_neighbors = 50)
+        """)
+    elif vector_ext == "vchord":
+        # Use vchordrq index for vchord (supports high-dimensional embeddings)
+        op.execute("""
+            CREATE INDEX idx_memory_units_embedding ON memory_units
+            USING vchordrq (embedding vector_l2_ops)
+        """)
+    else:  # pgvector
+        # Use HNSW index for pgvector
+        op.create_index(
+            "idx_memory_units_embedding",
+            "memory_units",
+            ["embedding"],
+            postgresql_using="hnsw",
+            postgresql_ops={"embedding": "vector_cosine_ops"},
+        )
 
     # Create full-text search index on search_vector
     # Index type depends on text search backend
@@ -382,17 +358,6 @@ def _pg_upgrade() -> None:
             USING bm25(text)
             WITH (text_config='english')
         """)
-    elif text_search_ext == "pg_search":
-        # ParadeDB pg_search BM25 index on (id, text, context). The key_field
-        # reloption is required and must match the table's primary key column.
-        bm25_cols = pg_search_bm25_columns("id", ("text", "context"), _pg_search_tokenizer())
-        op.execute(
-            """
-            CREATE INDEX idx_memory_units_text_search ON memory_units
-            USING bm25 ({bm25_cols})
-            WITH (key_field='id')
-        """.format(bm25_cols=bm25_cols)
-        )
     else:  # native
         # Native PostgreSQL GIN index
         op.execute("""
@@ -498,7 +463,7 @@ def _pg_upgrade() -> None:
     op.create_index("idx_unit_entities_entity", "unit_entities", ["entity_id"])
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     """Downgrade schema - drop all tables."""
 
     # Drop tables in reverse dependency order
@@ -558,11 +523,3 @@ def _pg_downgrade() -> None:
     # Drop extensions (optional - comment out if you want to keep them)
     # op.execute('DROP EXTENSION IF EXISTS vector')
     # op.execute('DROP EXTENSION IF EXISTS "uuid-ossp"')
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/86f7a033d372_repair_mental_models_subtype_at_head.py

@@ -1,117 +0,0 @@
-"""Repair mental_models.subtype on databases stuck at m3rg3h3ad5f6
-
-Three production deployments reported `column "subtype" of relation
-"mental_models" does not exist` on `create_mental_model` even after their
-container reported `Database migrations completed successfully` and
-`alembic_version` advanced to `m3rg3h3ad5f6` (see issue #1553, #1553#1
-confirmations from @4Lienau and @khanhduyvt0101).
-
-Both `h3c4d5e6f7g8_mental_models_v4` and `d5y6z7a8b9c0_backfill_mental_models_subtype`
-were meant to ensure `subtype` exists, but on databases that came through the
-`reflections -> mental_models` rename chain *and* whose alembic_version
-advanced past `d5y6z7a8b9c0` along an alternate path during the divergent-heads
-reorganization, neither column-add actually fired. The result is a head-tagged
-database with a v3-shaped `mental_models` table missing six columns:
-``subtype``, ``description``, ``entity_id``, ``observations``, ``links``,
-``last_updated``.
-
-This migration sits at the current head (`m3rg3h3ad5f6`) so every affected
-deployment will pick it up on next container start. It mirrors the column-add
-block from `d5y6z7a8b9c0_backfill_mental_models_subtype` using
-``ADD COLUMN IF NOT EXISTS`` so it is a no-op on databases where the columns
-are already present.
-
-Revision ID: 86f7a033d372
-Revises: m3rg3h3ad5f6
-Create Date: 2026-05-14
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "86f7a033d372"
-down_revision: str | Sequence[str] | None = "m3rg3h3ad5f6"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _pg_schema_prefix() -> str:
-    """Schema-qualifier for raw SQL on PG (multi-tenant search_path)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _pg_upgrade() -> None:
-    """Idempotently ensure mental_models has the v4 column set.
-
-    Safe to re-apply on databases that already received the columns via
-    `h3c4d5e6f7g8_mental_models_v4` or `d5y6z7a8b9c0_backfill_mental_models_subtype` —
-    every column-add uses ``IF NOT EXISTS`` and the constraint is recreated
-    from scratch with the canonical v4 allowlist.
-    """
-    schema = _pg_schema_prefix()
-    bare_schema = schema.strip(".").strip('"') if schema else ""
-    schema_clause = f"AND table_schema = '{bare_schema}'" if bare_schema else ""
-
-    # Wrapped in a DO block so the existence check skips databases that
-    # predate the reflections -> mental_models rename chain (no table to
-    # repair). On those, every ALTER below would error.
-    op.execute(
-        f"""
-        DO $$
-        BEGIN
-            IF EXISTS (
-                SELECT 1 FROM information_schema.tables
-                WHERE table_name = 'mental_models'
-                  {schema_clause}
-            ) THEN
-                -- Add the six v4 columns idempotently.
-                ALTER TABLE {schema}mental_models
-                    ADD COLUMN IF NOT EXISTS subtype VARCHAR(32) NOT NULL DEFAULT 'structural';
-                ALTER TABLE {schema}mental_models
-                    ADD COLUMN IF NOT EXISTS description TEXT NOT NULL DEFAULT '';
-                ALTER TABLE {schema}mental_models
-                    ADD COLUMN IF NOT EXISTS entity_id UUID;
-                ALTER TABLE {schema}mental_models
-                    ADD COLUMN IF NOT EXISTS observations JSONB DEFAULT '{{"observations": []}}'::jsonb;
-                ALTER TABLE {schema}mental_models
-                    ADD COLUMN IF NOT EXISTS links VARCHAR[];
-                ALTER TABLE {schema}mental_models
-                    ADD COLUMN IF NOT EXISTS last_updated TIMESTAMP WITH TIME ZONE;
-
-                -- Recreate the CHECK constraint with the canonical v4 allowlist.
-                -- Existing rows with subtype = 'directive' (possible on databases
-                -- that ran the o0j1k2l3m4n5 directive-only path) are rewritten to
-                -- 'structural' first so the constraint add succeeds.
-                UPDATE {schema}mental_models SET subtype = 'structural' WHERE subtype = 'directive';
-
-                ALTER TABLE {schema}mental_models DROP CONSTRAINT IF EXISTS ck_mental_models_subtype;
-                ALTER TABLE {schema}mental_models
-                    ADD CONSTRAINT ck_mental_models_subtype
-                    CHECK (subtype IN ('structural', 'emergent', 'pinned', 'learned'));
-
-                CREATE INDEX IF NOT EXISTS idx_mental_models_subtype
-                    ON {schema}mental_models(bank_id, subtype);
-            END IF;
-        END$$;
-        """
-    )
-
-
-def _pg_downgrade() -> None:
-    """No-op: dropping these columns would corrupt v4 application code."""
-    pass
-
-
-def upgrade() -> None:
-    # PG-only: Oracle's baseline (o1a2b3c4d5e6) creates mental_models with its
-    # own subtype shape (chk_mm_subtype IN ('directive', 'pinned')) and a
-    # different table topology, so this PG-shaped repair does not apply.
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/8c6fa6f7230b_merge_v053_divergent_heads.py

@@ -26,25 +26,15 @@ Create Date: 2026-04-18
 
 from collections.abc import Sequence
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "8c6fa6f7230b"
 down_revision: str | Sequence[str] | None = ("c4x5y6z7a8b9", "h3i4j5k6l7m8")
 branch_labels: str | Sequence[str] | None = None
 depends_on: str | Sequence[str] | None = None
 
 
-def _pg_upgrade() -> None:
-    pass
-
-
-def _pg_downgrade() -> None:
-    pass
-
-
 def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
+    pass
 
 
 def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)
+    pass

hindsight-api-slim/hindsight_api/alembic/versions/9f8e7d6c5b4a_memory_links_deferrable_fk.py

@@ -1,128 +0,0 @@
-"""Make memory_links.from_unit_id and memory_links.to_unit_id FKs deferrable.
-
-Revision ID: 9f8e7d6c5b4a
-Revises: o1a2b3c4d5e6
-Create Date: 2026-05-03
-
-Background
-----------
-Concurrent retain (which INSERTs into ``memory_links``) and any code path
-that DELETEs a row whose deletion cascades into ``memory_links`` (e.g.
-delta-retain superseding chunks, which CASCADEs chunks → memory_units →
-memory_links) can deadlock under sustained single-tenant write load.
-
-The deadlock cycle:
-
-* Tx A: ``DELETE FROM chunks WHERE chunk_id = ANY(...)``
-  → CASCADE acquires row locks on memory_units, then on memory_links rows
-    where ``to_unit_id`` matches the deleted units.
-* Tx B: ``INSERT INTO memory_links (...)`` referencing one of the same
-  memory_units rows.
-  → The immediate FK check takes ``FOR KEY SHARE`` on those memory_units
-    rows.
-
-The two transactions take row locks on the same memory_units rows in
-opposite orders depending on which side started first. PostgreSQL detects
-the cycle and aborts one transaction; the loser is killed mid-batch, the
-winner continues. Workers then retry, but under sustained write load the
-pattern repeats.
-
-Fix
----
-Make both ``memory_links → memory_units`` FKs (``from_unit_id`` and
-``to_unit_id``) ``DEFERRABLE INITIALLY DEFERRED``. This pushes the FK
-check from INSERT time to COMMIT time:
-
-* INSERT no longer takes ``FOR KEY SHARE`` on the memory_units row → no
-  contention with the cascading DELETE's row lock.
-* At COMMIT the engine validates referential integrity in one shot. If a
-  cascade-DELETE has since removed the referenced unit, the INSERT
-  transaction commits OR fails with a clean FK violation (sqlstate
-  23503) instead of a deadlock (sqlstate 40P01).
-
-The ``WHERE EXISTS`` filter already in ``_bulk_insert_links`` continues to
-filter out the typical "stale unit_id" case at INSERT time; the deferred
-FK is only the backstop for the narrow race window between the EXISTS
-probe and COMMIT. ``ON DELETE CASCADE`` semantics are unchanged — only
-the *timing* of the constraint check moves.
-
-The ``entity_id`` FK on ``memory_links`` is not changed; entities are not
-involved in the observed deadlock cycle and leaving the constraint
-immediate keeps the error message specific when an entity row is missing.
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "9f8e7d6c5b4a"
-down_revision: str | Sequence[str] | None = "o1a2b3c4d5e6"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Schema-qualifier for raw SQL on PG (multi-tenant search_path)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-# The two FK constraints installed by the initial schema migration
-# (5a366d414dce_initial_schema), mapped to the column they constrain.
-# They reference memory_units(id) with ON DELETE CASCADE — that
-# semantics is preserved; only the deferral attribute changes.
-_FK_COLUMNS: dict[str, str] = {
-    "fk_memory_links_from_unit_id_memory_units": "from_unit_id",
-    "fk_memory_links_to_unit_id_memory_units": "to_unit_id",
-}
-
-
-def _pg_upgrade() -> None:
-    schema = _get_schema_prefix()
-    # PostgreSQL doesn't allow altering the deferrability of an existing
-    # constraint with ALTER CONSTRAINT — the constraint must be dropped
-    # and recreated. DROP IF EXISTS makes the migration safe to re-run
-    # on schemas where the constraint was already recreated.
-    for fk_name, column in _FK_COLUMNS.items():
-        op.execute(f"ALTER TABLE {schema}memory_links DROP CONSTRAINT IF EXISTS {fk_name}")
-        op.execute(
-            f"""
-            ALTER TABLE {schema}memory_links
-                ADD CONSTRAINT {fk_name}
-                FOREIGN KEY ({column})
-                REFERENCES {schema}memory_units (id)
-                ON DELETE CASCADE
-                DEFERRABLE INITIALLY DEFERRED
-            """
-        )
-
-
-def _pg_downgrade() -> None:
-    schema = _get_schema_prefix()
-    # Revert to the default (NOT DEFERRABLE) form so a downgrade actually
-    # restores the prior schema state, even though that re-introduces the
-    # deadlock window.
-    for fk_name, column in _FK_COLUMNS.items():
-        op.execute(f"ALTER TABLE {schema}memory_links DROP CONSTRAINT IF EXISTS {fk_name}")
-        op.execute(
-            f"""
-            ALTER TABLE {schema}memory_links
-                ADD CONSTRAINT {fk_name}
-                FOREIGN KEY ({column})
-                REFERENCES {schema}memory_units (id)
-                ON DELETE CASCADE
-            """
-        )
-
-
-def upgrade() -> None:
-    # PG-only: Oracle's deferrable-FK semantics differ and the deadlock
-    # cycle was only observed on PostgreSQL. Oracle slot intentionally
-    # absent → no-op there.
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/a1b2c3d4e5f6_add_file_storage_table.py

@@ -15,8 +15,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "a1b2c3d4e5f6"
 down_revision: str | Sequence[str] | None = "y0t1u2v3w4x5"
 branch_labels: str | Sequence[str] | None = None
@@ -29,7 +27,7 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     """Create file_storage table for BYTEA storage."""
     schema = _get_schema_prefix()
 
@@ -54,7 +52,7 @@ def _pg_upgrade() -> None:
     )
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     """Remove file_storage table and related columns."""
     schema = _get_schema_prefix()
 
@@ -70,11 +68,3 @@ def _pg_downgrade() -> None:
 
     # Drop file_storage table
     op.execute(f"DROP TABLE IF EXISTS {schema}file_storage")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/a1c9e7f3b2d8_observation_history_drop_memory_units_fk.py

@@ -1,67 +0,0 @@
-"""Drop observation_history's FK to memory_units.
-
-The history table records one snapshot per observation change, keyed by
-``(bank_id, observation_id)``. Its foreign key to ``memory_units`` existed only to
-cascade-delete history when the observation row went away.
-
-That assumes every observation *is* a ``memory_units`` row, which is true only
-while Postgres is the memories store. When another store owns the memories the
-observation lives there and Postgres holds no row for it, so every history insert
-raises a foreign-key violation — swallowed by the writer as "a race with parallel
-consolidation" and logged at warning level. The audit trail goes silently empty.
-
-Dropping the constraint lets history be recorded wherever the observation is
-stored. The cleanup the cascade used to do is now explicit, in the paths that
-delete observations (``_execute_delete_action``, ``clear_observations``,
-``delete_bank``). Rows orphaned by a path that misses — a document delete
-cascading through ``memory_units``, for instance — are invisible to readers,
-which always filter by ``(bank_id, observation_id)``, and are reclaimed when the
-bank is deleted.
-
-Oracle builds this schema through its own DDL runner and never had the
-constraint, so the Oracle slot is a deliberate no-op.
-
-Revision ID: a1c9e7f3b2d8
-Revises: c7d1e9a4b3f2
-"""
-
-from collections.abc import Sequence
-
-from alembic import op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "a1c9e7f3b2d8"
-down_revision: str | Sequence[str] | None = "c7d1e9a4b3f2"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-_CONSTRAINT = "observation_history_observation_id_fkey"
-
-
-def _pg_upgrade() -> None:
-    op.execute(f"ALTER TABLE observation_history DROP CONSTRAINT IF EXISTS {_CONSTRAINT}")
-
-
-def _pg_downgrade() -> None:
-    # Re-adding the FK requires every row to reference a live memory_unit, so
-    # clear any history whose observation is not a Postgres row first — those are
-    # exactly the rows this migration made possible.
-    op.execute(
-        "DELETE FROM observation_history h "
-        "WHERE NOT EXISTS (SELECT 1 FROM memory_units m WHERE m.id = h.observation_id)"
-    )
-    op.execute(
-        f"ALTER TABLE observation_history ADD CONSTRAINT {_CONSTRAINT} "
-        "FOREIGN KEY (observation_id) REFERENCES memory_units(id) ON DELETE CASCADE"
-    )
-
-
-def upgrade() -> None:
-    # Oracle never had the constraint (its schema is built by a separate DDL
-    # runner), so only Postgres has anything to drop.
-    run_for_dialect(pg=_pg_upgrade, oracle=None)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade, oracle=None)

hindsight-api-slim/hindsight_api/alembic/versions/a1d3f5b7c9e2_widen_remaining_bank_id_to_text.py

@@ -1,85 +0,0 @@
-"""Repair: widen the remaining live ``bank_id`` columns from VARCHAR(64) to TEXT on PostgreSQL.
-
-Follow-up to ``c3e5a7b9d1f4`` (issue #2106), which widened the two *history*
-tables (``observation_history``, ``mental_model_history``) to ``TEXT`` after the
-narrow ``VARCHAR(64)`` declaration bricked startup. The same VARCHAR(64) / TEXT
-inconsistency still affects the live tables that store a user-supplied
-``bank_id``:
-
-* ``directives``    -- created VARCHAR(64) in ``p1k2l3m4n5o6``
-* ``mental_models`` -- VARCHAR(64) (origin ``pinned_reflections`` in
-                       ``n9i0j1k2l3m4``; recreated in ``h3c4d5e6f7g8``)
-
-``mental_model_versions`` is intentionally *not* widened here: it is created in
-``j5e6f7g8h9i0`` but dropped (``DROP TABLE ... CASCADE``) in ``o0j1k2l3m4n5`` and
-never recreated on the upgrade path, so it does not exist at head. Issuing
-``ALTER TABLE mental_model_versions ...`` would raise ``UndefinedTable`` and --
-because migrations run inside the lifespan-startup transaction -- roll the whole
-migration back, bricking the API. (It is unrelated to the live
-``mental_model_history`` table widened by ``c3e5a7b9d1f4``.)
-
-``banks.bank_id`` is ``TEXT`` (unbounded), so a deployment can create a bank
-whose id exceeds 64 chars -- the 78-char hierarchical org-unit shape reported in
-issue #2106 -- and the bank insert succeeds. The next write that propagates that
-id (``create_directive``, ``create_mental_model`` / consolidation, or
-mental-model versioning) then aborts with::
-
-    psycopg2.errors.StringDataRightTruncation: value too long for type
-    character varying(64)
-
-i.e. a 500 on core write endpoints, instead of the startup brick that
-``c3e5a7b9d1f4`` already repaired.
-
-``ALTER COLUMN ... TYPE TEXT`` is a no-op on a column that is already ``TEXT``,
-so every upgrade path converges on ``TEXT``. These tables are per-tenant (they
-live in each tenant schema, not ``public``), so this runs for every migrated
-schema via the search-path-aware prefix -- the same mechanism as
-``c3e5a7b9d1f4``.
-
-PostgreSQL only: these tables are created by PostgreSQL-only migrations
-(``run_for_dialect(pg=...)``); on Oracle they are absent or already
-``VARCHAR2(256)`` (consistent, never truncates), so the Oracle slot is
-intentionally absent -- mirroring ``c3e5a7b9d1f4``.
-
-Revision ID: a1d3f5b7c9e2
-Revises: c3e5a7b9d1f4
-Create Date: 2026-06-13
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "a1d3f5b7c9e2"
-down_revision: str | Sequence[str] | None = "c3e5a7b9d1f4"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Schema-qualifier for raw SQL on PG (multi-tenant search_path)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _pg_upgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"ALTER TABLE {schema}directives ALTER COLUMN bank_id TYPE TEXT")
-    op.execute(f"ALTER TABLE {schema}mental_models ALTER COLUMN bank_id TYPE TEXT")
-
-
-def _pg_downgrade() -> None:
-    # No-op: narrowing back to VARCHAR(64) could truncate real data and would
-    # re-introduce the bug this migration repairs. The column types are owned by
-    # the migrations that created the tables.
-    pass
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/a2b3c4d5e6f7_add_text_signals_column.py

@@ -7,7 +7,6 @@ the stored fact text.
 - vchord: text_signals included in tokenize() at insert time
 - native: search_vector GENERATED column regenerated to include text_signals
 - pg_textsearch: no change (index only supports a single base column)
-- pg_search: BM25 index dropped and recreated to include text_signals
 
 Revision ID: a2b3c4d5e6f7
 Revises: z1u2v3w4x5y6
@@ -19,13 +18,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api._pg_search import (
-    PG_SEARCH_TOKENIZER_ENV,
-    normalize_pg_search_tokenizer,
-    pg_search_bm25_columns,
-)
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "a2b3c4d5e6f7"
 down_revision: str | Sequence[str] | None = "aa2b3c4d5e6f"
 branch_labels: str | Sequence[str] | None = None
@@ -41,11 +33,7 @@ def _detect_text_search_extension() -> str:
     return os.getenv("HINDSIGHT_API_TEXT_SEARCH_EXTENSION", "native").lower()
 
 
-def _pg_search_tokenizer() -> str:
-    return normalize_pg_search_tokenizer(os.getenv(PG_SEARCH_TOKENIZER_ENV))
-
-
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     schema = _get_schema_prefix()
     table = f"{schema}memory_units"
     text_search_ext = _detect_text_search_extension()
@@ -72,22 +60,12 @@ def _pg_upgrade() -> None:
             CREATE INDEX IF NOT EXISTS idx_memory_units_text_search
             ON {table} USING gin(search_vector)
         """)
-    elif text_search_ext == "pg_search":
-        # ParadeDB pg_search: drop the existing BM25 index and recreate it
-        # to include text_signals alongside text and context.
-        bm25_cols = pg_search_bm25_columns("id", ("text", "context", "text_signals"), _pg_search_tokenizer())
-        op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_units_text_search")
-        op.execute(f"""
-            CREATE INDEX idx_memory_units_text_search ON {table}
-            USING bm25 ({bm25_cols})
-            WITH (key_field='id')
-        """)
 
     # vchord: tokenize() call in fact_storage.py is updated to include text_signals at insert time
     # pg_textsearch: no change — index operates on the base `text` column only
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     schema = _get_schema_prefix()
     table = f"{schema}memory_units"
     text_search_ext = _detect_text_search_extension()
@@ -106,22 +84,5 @@ def _pg_downgrade() -> None:
             CREATE INDEX idx_memory_units_text_search
             ON {table} USING gin(search_vector)
         """)
-    elif text_search_ext == "pg_search":
-        # Restore the original (id, text, context) BM25 index without text_signals.
-        bm25_cols = pg_search_bm25_columns("id", ("text", "context"), _pg_search_tokenizer())
-        op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_units_text_search")
-        op.execute(f"""
-            CREATE INDEX idx_memory_units_text_search ON {table}
-            USING bm25 ({bm25_cols})
-            WITH (key_field='id')
-        """)
 
     op.execute(f"ALTER TABLE {table} DROP COLUMN IF EXISTS text_signals")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/a2b3c4d5e6f8_add_gin_index_source_memory_ids.py

@@ -24,8 +24,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "a2b3c4d5e6f8"
 down_revision: str | Sequence[str] | None = "f7g8h9i0j1k2"
 branch_labels: str | Sequence[str] | None = None
@@ -37,28 +35,20 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # CREATE INDEX CONCURRENTLY cannot run inside a transaction block; an
-    # autocommit_block runs it outside Alembic's migration transaction.
-    with op.get_context().autocommit_block():
-        op.execute(
-            f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_units_source_memory_ids "
-            f"ON {schema}memory_units USING GIN (source_memory_ids) "
-            f"WHERE source_memory_ids IS NOT NULL"
-        )
-
-
-def _pg_downgrade() -> None:
-    schema = _get_schema_prefix()
-    with op.get_context().autocommit_block():
-        op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_units_source_memory_ids")
-
-
 def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
+    schema = _get_schema_prefix()
+
+    # CREATE INDEX CONCURRENTLY cannot run inside a transaction block.
+    # Commit the current Alembic transaction first.
+    op.execute("COMMIT")
+    op.execute(
+        f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_units_source_memory_ids "
+        f"ON {schema}memory_units USING GIN (source_memory_ids) "
+        f"WHERE source_memory_ids IS NOT NULL"
+    )
 
 
 def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)
+    schema = _get_schema_prefix()
+    op.execute("COMMIT")
+    op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_units_source_memory_ids")

hindsight-api-slim/hindsight_api/alembic/versions/a2v3w4x5y6z7_add_last_refreshed_source_query.py

@@ -14,8 +14,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "a2v3w4x5y6z7"
 down_revision: str | Sequence[str] | None = "z1u2v3w4x5y6"
 branch_labels: str | Sequence[str] | None = None
@@ -27,7 +25,7 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     schema = _get_schema_prefix()
     op.execute(f"""
         ALTER TABLE {schema}mental_models
@@ -35,14 +33,6 @@ def _pg_upgrade() -> None:
     """)
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     schema = _get_schema_prefix()
     op.execute(f"ALTER TABLE {schema}mental_models DROP COLUMN IF EXISTS last_refreshed_source_query")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/a3b4c5d6e7f8_add_consolidation_failed_at_to_memory_units.py

@@ -13,8 +13,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "a3b4c5d6e7f8"
 down_revision: str | Sequence[str] | None = "g7h8i9j0k1l2"
 branch_labels: str | Sequence[str] | None = None
@@ -27,7 +25,7 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     schema = _get_schema_prefix()
 
     op.execute(
@@ -47,16 +45,8 @@ def _pg_upgrade() -> None:
     )
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     schema = _get_schema_prefix()
 
     op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_units_consolidation_failed")
     op.execute(f"ALTER TABLE {schema}memory_units DROP COLUMN IF EXISTS consolidation_failed_at")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/a4b5c6d7e8f9_fix_per_bank_vector_index_type.py

@@ -11,8 +11,7 @@ was configured.
 
 This migration detects the mismatch and recreates the affected indexes with
 the correct type. Skipped entirely when the configured extension is pgvector
-(the default) or scann. ScaNN uses global vector indexes because empty or tiny
-per-bank indexes cannot be built safely on AlloyDB.
+(the default), since those indexes are already correct.
 """
 
 import os
@@ -21,8 +20,6 @@ from collections.abc import Sequence
 from alembic import context, op
 from sqlalchemy import text
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "a4b5c6d7e8f9"
 down_revision: str | Sequence[str] | None = "2eee35aa3cfc"
 branch_labels: str | Sequence[str] | None = None
@@ -40,47 +37,39 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _validate_extension(name: str) -> str:
-    ext = name.lower()
-    if ext not in {"pgvector", "pgvectorscale", "vchord", "scann"}:
-        raise ValueError(
-            f"Invalid HINDSIGHT_API_VECTOR_EXTENSION: {ext}. Must be 'pgvector', 'vchord', 'pgvectorscale', or 'scann'"
-        )
-    return ext
-
-
-def _index_type_keyword(ext: str) -> str:
+def _target_index_type() -> str | None:
+    """Return the target index type, or None if pgvector (no fix needed)."""
+    ext = os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector").lower()
     if ext == "pgvectorscale":
         return "diskann"
-    if ext == "vchord":
+    elif ext == "vchord":
         return "vchordrq"
-    if ext == "scann":
-        return "scann"
-    return "hnsw"
+    return None
 
 
-def _vector_index_using_clause(ext: str) -> str:
+def _vector_index_using_clause() -> str:
+    """Return the USING clause based on the configured vector extension."""
+    ext = os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector").lower()
     if ext == "pgvectorscale":
         return "USING diskann (embedding vector_cosine_ops) WITH (num_neighbors = 50)"
-    if ext == "vchord":
-        return "USING vchordrq (embedding vector_cosine_ops)"
-    if ext == "scann":
-        return "USING scann (embedding cosine) WITH (mode = 'AUTO')"
-    return "USING hnsw (embedding vector_cosine_ops)"
+    elif ext == "vchord":
+        return "USING vchordrq (embedding vector_l2_ops)"
+    else:
+        return "USING hnsw (embedding vector_cosine_ops)"
 
 
-def _pg_upgrade() -> None:
-    ext = _validate_extension(os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector"))
-    if ext in {"pgvector", "scann"}:
+def upgrade() -> None:
+    target = _target_index_type()
+    if target is None:
+        # pgvector — indexes are already HNSW, nothing to fix
         return
-    target = _index_type_keyword(ext)
 
     bind = op.get_bind()
     schema_name = context.config.get_main_option("target_schema")
     schema = _get_schema_prefix()
     table_ref = f'"{schema_name}".memory_units' if schema_name else "memory_units"
     banks_ref = f'"{schema_name}".banks' if schema_name else "banks"
-    using_clause = _vector_index_using_clause(ext)
+    using_clause = _vector_index_using_clause()
     pg_schema = schema_name or "public"
 
     rows = bind.execute(text(f"SELECT bank_id, internal_id FROM {banks_ref}")).fetchall()  # noqa: S608
@@ -124,10 +113,10 @@ def _pg_upgrade() -> None:
             )
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     # Downgrade recreates indexes as HNSW (the original hardcoded behavior)
-    ext = _validate_extension(os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector"))
-    if ext in {"pgvector", "scann"}:
+    target = _target_index_type()
+    if target is None:
         return
 
     bind = op.get_bind()
@@ -151,11 +140,3 @@ def _pg_downgrade() -> None:
                     f"WHERE fact_type = '{ft}' AND bank_id = '{escaped_bank_id}'"
                 )
             )
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/a7b8c9d0e1f2_split_history_into_own_tables.py

@@ -1,253 +0,0 @@
-"""Move mental-model and observation history into dedicated tables.
-
-Both histories were accumulated in a single JSONB/CLOB ``history`` column
-(``mental_models.history`` and ``memory_units.history``), appended to on every
-update. That design has two problems:
-
-1. **Unbounded growth on observations.** The observation write path appended a
-   snapshot on every update with no cap at all, so a frequently-reinforced
-   observation grew its ``history`` array until it crossed Postgres's hard 256MB
-   jsonb limit (SQLSTATE 54000), after which every further UPDATE failed and the
-   row was stuck.
-2. **Wrong-axis cap on mental models.** The mental-model cap bounded the *number*
-   of entries (50), not their *size* — a single large reflect snapshot could
-   still blow the budget — and rewrote the whole array (plus TOAST) on every
-   refresh, defeating HOT updates.
-
-This migration creates one row per history entry in two dedicated tables, with
-an index that makes "most recent N for this item" cheap, then drops the old
-columns. The cap is now enforced at write time as a bounded DELETE of the
-oldest over-cap rows (see config ``*_HISTORY_MAX_ENTRIES``).
-
-Revision ID: a7b8c9d0e1f2
-Revises: d3e4f5a6b7c8
-Create Date: 2026-06-05
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "a7b8c9d0e1f2"
-down_revision: str | Sequence[str] | None = "d3e4f5a6b7c8"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Schema-qualifier for raw SQL on PG (multi-tenant search_path)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-# ---------------------------------------------------------------------------
-# PostgreSQL
-# ---------------------------------------------------------------------------
-def _pg_upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # Both tables share the same shape: surrogate id, FK to the parent, bank_id,
-    # the snapshot payload as a single JSONB ``content`` blob, and changed_at.
-    # The payload is per-row (one change per row) so it stays small — this is NOT
-    # the old single-column-grows-forever design; growth is bounded by row count
-    # plus the write-time cap. Folding the previous_* fields into one JSONB keeps
-    # the schema dialect-simple (no array columns) and flexible.
-
-    # --- mental_model_history -------------------------------------------------
-    # content: {"previous_content": ..., "previous_reflect_response": {...}}
-    op.execute(
-        f"""
-        CREATE TABLE IF NOT EXISTS {schema}mental_model_history (
-            id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
-            mental_model_id VARCHAR(64) NOT NULL,
-            bank_id TEXT NOT NULL,
-            content JSONB NOT NULL,
-            changed_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
-            FOREIGN KEY (mental_model_id, bank_id)
-                REFERENCES {schema}mental_models(id, bank_id) ON DELETE CASCADE
-        )
-        """
-    )
-    op.execute(
-        "CREATE INDEX IF NOT EXISTS idx_mm_history_model "
-        f"ON {schema}mental_model_history (bank_id, mental_model_id, changed_at DESC, id DESC)"
-    )
-
-    # --- observation_history --------------------------------------------------
-    # content: {"previous_text", "previous_tags", "previous_occurred_start",
-    #           "previous_occurred_end", "previous_mentioned_at", "new_source_memory_ids"}
-    op.execute(
-        f"""
-        CREATE TABLE IF NOT EXISTS {schema}observation_history (
-            id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
-            observation_id UUID NOT NULL,
-            bank_id TEXT NOT NULL,
-            content JSONB NOT NULL,
-            changed_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
-            FOREIGN KEY (observation_id)
-                REFERENCES {schema}memory_units(id) ON DELETE CASCADE
-        )
-        """
-    )
-    op.execute(
-        "CREATE INDEX IF NOT EXISTS idx_observation_history_obs "
-        f"ON {schema}observation_history (observation_id, changed_at DESC, id DESC)"
-    )
-
-    # --- backfill mental models ----------------------------------------------
-    # Explode each row's history array into rows, preserving chronological order
-    # via WITH ORDINALITY so the IDENTITY id tie-breaks oldest->newest correctly.
-    # changed_at is promoted to its own column; the rest of the element becomes
-    # ``content`` (the ``- 'changed_at'`` strips the now-redundant key).
-    op.execute(
-        f"""
-        INSERT INTO {schema}mental_model_history (mental_model_id, bank_id, content, changed_at)
-        SELECT mm.id, mm.bank_id,
-               e - 'changed_at',
-               COALESCE(NULLIF(e->>'changed_at', '')::timestamptz, now())
-        FROM {schema}mental_models mm
-             CROSS JOIN LATERAL jsonb_array_elements(mm.history) WITH ORDINALITY a(e, ord)
-        WHERE mm.history IS NOT NULL
-          AND jsonb_typeof(mm.history) = 'array'
-          AND jsonb_array_length(mm.history) > 0
-        ORDER BY mm.id, mm.bank_id, ord
-        """
-    )
-
-    # --- backfill observations -----------------------------------------------
-    op.execute(
-        f"""
-        INSERT INTO {schema}observation_history (observation_id, bank_id, content, changed_at)
-        SELECT mu.id, mu.bank_id,
-               e - 'changed_at',
-               COALESCE(NULLIF(e->>'changed_at', '')::timestamptz, now())
-        FROM {schema}memory_units mu
-             CROSS JOIN LATERAL jsonb_array_elements(mu.history) WITH ORDINALITY a(e, ord)
-        WHERE mu.fact_type = 'observation'
-          AND mu.history IS NOT NULL
-          AND jsonb_typeof(mu.history) = 'array'
-          AND jsonb_array_length(mu.history) > 0
-        ORDER BY mu.id, ord
-        """
-    )
-
-    # --- drop the legacy columns ---------------------------------------------
-    op.execute(f"ALTER TABLE {schema}mental_models DROP COLUMN IF EXISTS history")
-    op.execute(f"ALTER TABLE {schema}memory_units DROP COLUMN IF EXISTS history")
-
-
-def _pg_downgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # Re-add the columns (empty — historical content is not reconstructed back
-    # into the array form; the dedicated tables are dropped below).
-    op.execute(f"ALTER TABLE {schema}mental_models ADD COLUMN IF NOT EXISTS history JSONB DEFAULT '[]'::jsonb")
-    op.execute(f"ALTER TABLE {schema}memory_units ADD COLUMN IF NOT EXISTS history JSONB DEFAULT '[]'::jsonb")
-
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_observation_history_obs")
-    op.execute(f"DROP TABLE IF EXISTS {schema}observation_history")
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_mm_history_model")
-    op.execute(f"DROP TABLE IF EXISTS {schema}mental_model_history")
-
-
-# ---------------------------------------------------------------------------
-# Oracle 23ai
-# ---------------------------------------------------------------------------
-def _oracle_upgrade() -> None:
-    # Same single-JSONB shape as PG: ``content`` holds the snapshot payload as a
-    # CLOB IS JSON. The legacy per-element JSON object (minus changed_at, promoted
-    # to its own column) is carried through verbatim on backfill — the array
-    # columns the previous design needed are gone.
-    op.execute(
-        """
-        CREATE TABLE IF NOT EXISTS mental_model_history (
-            id NUMBER GENERATED BY DEFAULT AS IDENTITY,
-            mental_model_id VARCHAR2(256) NOT NULL,
-            bank_id VARCHAR2(256) NOT NULL,
-            content CLOB NOT NULL
-                CONSTRAINT mmh_content_json CHECK (content IS JSON),
-            changed_at TIMESTAMP WITH TIME ZONE DEFAULT SYSTIMESTAMP NOT NULL,
-            CONSTRAINT pk_mental_model_history PRIMARY KEY (id),
-            CONSTRAINT fk_mmh_model FOREIGN KEY (mental_model_id, bank_id)
-                REFERENCES mental_models(id, bank_id) ON DELETE CASCADE
-        )
-        """
-    )
-    op.execute(
-        "CREATE INDEX idx_mm_history_model ON mental_model_history (bank_id, mental_model_id, changed_at DESC, id DESC)"
-    )
-
-    op.execute(
-        """
-        CREATE TABLE IF NOT EXISTS observation_history (
-            id NUMBER GENERATED BY DEFAULT AS IDENTITY,
-            observation_id RAW(16) NOT NULL,
-            bank_id VARCHAR2(256) NOT NULL,
-            content CLOB NOT NULL
-                CONSTRAINT oh_content_json CHECK (content IS JSON),
-            changed_at TIMESTAMP WITH TIME ZONE DEFAULT SYSTIMESTAMP NOT NULL,
-            CONSTRAINT pk_observation_history PRIMARY KEY (id),
-            CONSTRAINT fk_oh_obs FOREIGN KEY (observation_id)
-                REFERENCES memory_units(id) ON DELETE CASCADE
-        )
-        """
-    )
-    op.execute(
-        "CREATE INDEX idx_observation_history_obs ON observation_history (observation_id, changed_at DESC, id DESC)"
-    )
-
-    bind = op.get_bind()
-
-    # Backfill via JSON_TABLE. ``content`` is the whole element (FORMAT JSON PATH
-    # '$'); changed_at is also promoted to its own column. Backfilled content may
-    # therefore still carry a redundant changed_at key, which the read path
-    # ignores in favour of the column — harmless, and avoids JSON surgery here.
-    bind.exec_driver_sql(
-        """
-        INSERT INTO mental_model_history (mental_model_id, bank_id, content, changed_at)
-        SELECT mm.id, mm.bank_id, jt.content, NVL(jt.changed_at, SYSTIMESTAMP)
-        FROM mental_models mm,
-             JSON_TABLE(mm.history, '$[*]' COLUMNS (
-                 seq FOR ORDINALITY,
-                 content CLOB FORMAT JSON PATH '$',
-                 changed_at TIMESTAMP WITH TIME ZONE PATH '$.changed_at'
-             )) jt
-        WHERE mm.history IS NOT NULL
-        ORDER BY mm.id, mm.bank_id, jt.seq
-        """
-    )
-
-    bind.exec_driver_sql(
-        """
-        INSERT INTO observation_history (observation_id, bank_id, content, changed_at)
-        SELECT mu.id, mu.bank_id, jt.content, NVL(jt.changed_at, SYSTIMESTAMP)
-        FROM memory_units mu,
-             JSON_TABLE(mu.history, '$[*]' COLUMNS (
-                 seq FOR ORDINALITY,
-                 content CLOB FORMAT JSON PATH '$',
-                 changed_at TIMESTAMP WITH TIME ZONE PATH '$.changed_at'
-             )) jt
-        WHERE mu.fact_type = 'observation' AND mu.history IS NOT NULL
-        ORDER BY mu.id, jt.seq
-        """
-    )
-
-    op.execute("ALTER TABLE mental_models DROP COLUMN history")
-    op.execute("ALTER TABLE memory_units DROP COLUMN history")
-
-
-def _oracle_downgrade() -> None:
-    op.execute("ALTER TABLE mental_models ADD history CLOB DEFAULT '[]' NOT NULL")
-    op.execute("ALTER TABLE memory_units ADD history CLOB DEFAULT '[]'")
-    op.execute("DROP TABLE observation_history CASCADE CONSTRAINTS")
-    op.execute("DROP TABLE mental_model_history CASCADE CONSTRAINTS")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade, oracle=_oracle_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade, oracle=_oracle_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/a8c1e4f7b0d3_add_operation_retention_indexes.py

@@ -1,82 +0,0 @@
-"""Add indexes for terminal cleanup and newest-first operation listing.
-
-Revision ID: a8c1e4f7b0d3
-Revises: e7c3a9f1b2d5
-Create Date: 2026-07-14
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "a8c1e4f7b0d3"
-down_revision: str | Sequence[str] | None = "e7c3a9f1b2d5"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _pg_schema_prefix() -> str:
-    """Schema-qualifier for PostgreSQL multi-tenant migration runs."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _pg_upgrade() -> None:
-    schema = _pg_schema_prefix()
-    # These can be large tables in long-running installations. Concurrent DDL
-    # keeps operation submission, polling, and status reads available.
-    with op.get_context().autocommit_block():
-        op.execute(
-            "CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_async_operations_terminal_cleanup "
-            f"ON {schema}async_operations (updated_at, operation_id) "
-            "WHERE status IN ('completed', 'failed', 'cancelled')"
-        )
-        op.execute(
-            "CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_async_operations_bank_created_desc "
-            f"ON {schema}async_operations (bank_id, created_at DESC)"
-        )
-
-
-def _pg_downgrade() -> None:
-    schema = _pg_schema_prefix()
-    with op.get_context().autocommit_block():
-        op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_async_operations_bank_created_desc")
-        op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_async_operations_terminal_cleanup")
-
-
-def _oracle_create_index(sql: str) -> None:
-    """Create an index idempotently for rerun-safe Oracle migrations."""
-    block = (
-        "BEGIN "
-        "EXECUTE IMMEDIATE :stmt; "
-        "EXCEPTION WHEN OTHERS THEN "
-        "IF SQLCODE = -955 THEN NULL; ELSE RAISE; END IF; "
-        "END;"
-    )
-    op.get_bind().exec_driver_sql(block, {"stmt": sql})
-
-
-def _oracle_upgrade() -> None:
-    # Oracle migrations run with CURRENT_SCHEMA set to each tenant, so table
-    # and index names intentionally remain unqualified here.
-    _oracle_create_index(
-        "CREATE INDEX idx_async_operations_terminal_cleanup ON async_operations (updated_at, operation_id, status)"
-    )
-    _oracle_create_index(
-        "CREATE INDEX idx_async_operations_bank_created_desc ON async_operations (bank_id, created_at DESC)"
-    )
-
-
-def _oracle_downgrade() -> None:
-    op.execute("DROP INDEX idx_async_operations_bank_created_desc")
-    op.execute("DROP INDEX idx_async_operations_terminal_cleanup")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade, oracle=_oracle_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade, oracle=_oracle_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/a9b8c7d6e5f4_add_knowledge_pages.py

@@ -1,126 +0,0 @@
-"""Add knowledge_pages table (knowledge-base hierarchy).
-
-The knowledge base organizes synthesized mental models into a navigable tree of
-**folders** and **pages**. A page references the mental model that holds its
-content (``mental_model_id``); a folder is a pure container (``mental_model_id``
-NULL). Hierarchy is a single self-referential ``parent_id`` so folders can nest
-arbitrarily. Content stays in ``mental_models`` — this table is metadata + tree
-structure only.
-
-``managed`` lets a client tag a node as system-owned vs. hand-authored; it
-carries no server-side behaviour. A partial unique index keeps page names unique
-within a folder (case-insensitive; root pages compared under an empty parent).
-
-Revision ID: a9b8c7d6e5f4
-Revises: a1c9e7f3b2d8
-Create Date: 2026-06-25
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "a9b8c7d6e5f4"
-down_revision: str | Sequence[str] | None = "a1c9e7f3b2d8"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _pg_schema_prefix() -> str:
-    """Schema-qualifier for raw SQL on PG (multi-tenant search_path)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _pg_upgrade() -> None:
-    schema = _pg_schema_prefix()
-    # parent_id self-FK cascades so deleting a folder row removes its whole
-    # subtree of rows in one shot. The mental_model FK is composite (matches the
-    # mental_models (id, bank_id) PK) and cascades too, so deleting a page's
-    # mental model removes the page row — folders skip the FK because a NULL
-    # column in a composite FK is not enforced (MATCH SIMPLE).
-    op.execute(
-        f"""
-        CREATE TABLE IF NOT EXISTS {schema}knowledge_pages (
-            id VARCHAR(64) NOT NULL,
-            bank_id TEXT NOT NULL,
-            parent_id VARCHAR(64),
-            kind VARCHAR(16) NOT NULL,
-            name TEXT NOT NULL,
-            mental_model_id VARCHAR(64),
-            sort_order INTEGER NOT NULL DEFAULT 0,
-            managed BOOLEAN NOT NULL DEFAULT false,
-            created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
-            updated_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
-            CONSTRAINT pk_knowledge_pages PRIMARY KEY (id),
-            CONSTRAINT ck_knowledge_pages_kind CHECK (kind IN ('folder', 'page')),
-            CONSTRAINT fk_kp_bank FOREIGN KEY (bank_id)
-                REFERENCES {schema}banks(bank_id) ON DELETE CASCADE,
-            CONSTRAINT fk_kp_parent FOREIGN KEY (parent_id)
-                REFERENCES {schema}knowledge_pages(id) ON DELETE CASCADE,
-            CONSTRAINT fk_kp_mm FOREIGN KEY (mental_model_id, bank_id)
-                REFERENCES {schema}mental_models(id, bank_id) ON DELETE CASCADE
-        )
-        """
-    )
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS idx_kp_bank_parent ON {schema}knowledge_pages (bank_id, parent_id, sort_order)"
-    )
-    # COALESCE(parent_id, '') so root-level pages (NULL parent) are also unique by
-    # name — NULLs would otherwise compare distinct and allow duplicates.
-    op.execute(
-        "CREATE UNIQUE INDEX IF NOT EXISTS uq_kp_folder_pagename "
-        f"ON {schema}knowledge_pages (bank_id, COALESCE(parent_id, ''), lower(name)) "
-        "WHERE kind = 'page'"
-    )
-
-
-def _pg_downgrade() -> None:
-    schema = _pg_schema_prefix()
-    op.execute(f"DROP INDEX IF EXISTS {schema}uq_kp_folder_pagename")
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_kp_bank_parent")
-    op.execute(f"DROP TABLE IF EXISTS {schema}knowledge_pages")
-
-
-def _oracle_upgrade() -> None:
-    # No case-insensitive unique index on Oracle: `name` is a CLOB and cannot be
-    # indexed with lower(); page-name uniqueness is enforced on PG only.
-    op.execute(
-        """
-        CREATE TABLE IF NOT EXISTS knowledge_pages (
-            id VARCHAR2(64) NOT NULL,
-            bank_id VARCHAR2(256) NOT NULL,
-            parent_id VARCHAR2(64),
-            kind VARCHAR2(16) NOT NULL,
-            name CLOB NOT NULL,
-            mental_model_id VARCHAR2(64),
-            sort_order NUMBER DEFAULT 0 NOT NULL,
-            managed NUMBER(1) DEFAULT 0 NOT NULL,
-            created_at TIMESTAMP WITH TIME ZONE DEFAULT SYSTIMESTAMP NOT NULL,
-            updated_at TIMESTAMP WITH TIME ZONE DEFAULT SYSTIMESTAMP NOT NULL,
-            CONSTRAINT pk_knowledge_pages PRIMARY KEY (id),
-            CONSTRAINT ck_knowledge_pages_kind CHECK (kind IN ('folder', 'page')),
-            CONSTRAINT fk_kp_bank FOREIGN KEY (bank_id)
-                REFERENCES banks(bank_id) ON DELETE CASCADE,
-            CONSTRAINT fk_kp_parent FOREIGN KEY (parent_id)
-                REFERENCES knowledge_pages(id) ON DELETE CASCADE,
-            CONSTRAINT fk_kp_mm FOREIGN KEY (mental_model_id, bank_id)
-                REFERENCES mental_models(id, bank_id) ON DELETE CASCADE
-        )
-        """
-    )
-    op.execute("CREATE INDEX idx_kp_bank_parent ON knowledge_pages (bank_id, parent_id, sort_order)")
-
-
-def _oracle_downgrade() -> None:
-    op.execute("DROP TABLE knowledge_pages CASCADE CONSTRAINTS")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade, oracle=_oracle_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade, oracle=_oracle_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/aa2b3c4d5e6f_nullable_event_date.py

@@ -12,8 +12,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "aa2b3c4d5e6f"
 down_revision: str | Sequence[str] | None = "z1u2v3w4x5y6"
 branch_labels: str | Sequence[str] | None = None
@@ -26,21 +24,13 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     schema = _get_schema_prefix()
     op.execute(f"ALTER TABLE {schema}memory_units ALTER COLUMN event_date DROP NOT NULL")
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     schema = _get_schema_prefix()
     # Backfill NULLs with now() before restoring the NOT NULL constraint
     op.execute(f"UPDATE {schema}memory_units SET event_date = now() WHERE event_date IS NULL")
     op.execute(f"ALTER TABLE {schema}memory_units ALTER COLUMN event_date SET NOT NULL")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/b2d4f6a8c1e3_repair_maintenance_routines_public.py

@@ -1,156 +0,0 @@
-"""Repair: install maintenance routines on the ``public`` / base-schema run.
-
-The original maintenance-routines migration (``e5f6a7b8c9d0``) only created the
-shared ``public.banks_needing_consolidation()`` and
-``public.schemas_with_expired_rows(...)`` routines when the run had *no*
-``target_schema`` at all. But the single-tenant runtime always migrates an
-explicit schema — which defaults to ``public`` — so on every default
-PostgreSQL deployment the migration was stamped as applied while the functions
-were never created. Background maintenance then logs::
-
-    Retention sweep failed for llm_requests: function public.schemas_with_expired_rows(...) does not exist
-    Consolidation reconcile discovery failed: function public.banks_needing_consolidation() does not exist
-
-See https://github.com/vectorize-io/hindsight/issues/2056.
-
-Because ``e5f6a7b8c9d0`` is already stamped on affected ``0.8.0`` databases,
-editing it would not re-run it there. This forward migration re-installs the
-functions idempotently (``CREATE OR REPLACE``) on the run that targets the
-shared ``public`` schema (base run with no ``target_schema``, or an explicit
-``target_schema=public``), self-healing already-upgraded deployments and
-covering fresh upgrades from earlier versions.
-
-Per-tenant runs against a non-``public`` schema still skip it: re-issuing
-``CREATE OR REPLACE FUNCTION public....`` from each concurrent tenant migration
-aborts with ``tuple concurrently updated`` on the ``pg_proc`` catalog row, and
-the base/public run has already created the functions for every tenant to use.
-Runs that target ``public`` are serialized by the per-schema migration advisory
-lock, so only one wins the create.
-
-PostgreSQL only — the worker poller and these tables are not wired for Oracle,
-so the Oracle slot is intentionally absent (mirrors ``e5f6a7b8c9d0``).
-
-Revision ID: b2d4f6a8c1e3
-Revises: e5f6a7b8c9d0
-Create Date: 2026-06-08
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "b2d4f6a8c1e3"
-down_revision: str | Sequence[str] | None = "e5f6a7b8c9d0"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _should_install_public_routines(target_schema: str | None) -> bool:
-    """True for the run that must (re)create the shared ``public.*`` routines.
-
-    The routines physically live in ``public`` (hard-coded ``public.`` qualifier
-    in the SQL below), so they must be installed exactly once — on the base run
-    (no ``target_schema``) or on the run that explicitly targets ``public``. A
-    run against any other tenant schema skips it to avoid concurrent
-    ``CREATE OR REPLACE`` on the same ``pg_proc`` row.
-    """
-    return not target_schema or target_schema == "public"
-
-
-def _pg_upgrade() -> None:
-    if not _should_install_public_routines(context.config.get_main_option("target_schema")):
-        return
-    # Banks with eligible-but-unscheduled facts and no in-flight consolidation.
-    # Auto-consolidation is filtered here only at the bank level (cheap prune);
-    # the full hierarchical resolution (global -> tenant -> bank, plus
-    # enable_observations) is done by the caller for the small returned set.
-    op.execute(
-        """
-        CREATE OR REPLACE FUNCTION public.banks_needing_consolidation()
-        RETURNS TABLE(schema_name text, bank_id text)
-        LANGUAGE plpgsql STABLE
-        AS $fn$
-        DECLARE
-            sch text;
-        BEGIN
-            FOR sch IN
-                SELECT n.nspname
-                FROM pg_class c
-                JOIN pg_namespace n ON n.oid = c.relnamespace
-                WHERE c.relname = 'memory_units' AND c.relkind = 'r'
-            LOOP
-                RETURN QUERY EXECUTE format($q$
-                    SELECT %1$L::text, m.bank_id
-                    FROM %1$I.memory_units m
-                    JOIN %1$I.banks b ON b.bank_id = m.bank_id
-                    WHERE m.consolidated_at IS NULL
-                      AND m.consolidation_failed_at IS NULL
-                      AND m.fact_type IN ('experience', 'world')
-                      AND COALESCE(b.config -> 'enable_auto_consolidation', 'true'::jsonb) <> 'false'::jsonb
-                      AND NOT EXISTS (
-                          SELECT 1 FROM %1$I.async_operations o
-                          WHERE o.bank_id = m.bank_id
-                            AND o.operation_type = 'consolidation'
-                            AND o.status IN ('pending', 'processing')
-                      )
-                    GROUP BY m.bank_id
-                $q$, sch);
-            END LOOP;
-        END;
-        $fn$;
-        """
-    )
-
-    # Schemas holding at least one row of p_table older than p_days. p_ts_col is
-    # the timestamp column to compare. Returns nothing when p_days <= 0
-    # (retention disabled).
-    op.execute(
-        """
-        CREATE OR REPLACE FUNCTION public.schemas_with_expired_rows(
-            p_table text, p_ts_col text, p_days int
-        )
-        RETURNS SETOF text
-        LANGUAGE plpgsql STABLE
-        AS $fn$
-        DECLARE
-            sch text;
-            has_expired boolean;
-        BEGIN
-            IF p_days IS NULL OR p_days <= 0 THEN
-                RETURN;
-            END IF;
-            FOR sch IN
-                SELECT n.nspname
-                FROM pg_class c
-                JOIN pg_namespace n ON n.oid = c.relnamespace
-                WHERE c.relname = p_table AND c.relkind = 'r'
-            LOOP
-                EXECUTE format(
-                    'SELECT EXISTS (SELECT 1 FROM %I.%I WHERE %I < NOW() - make_interval(days => $1))',
-                    sch, p_table, p_ts_col
-                ) INTO has_expired USING p_days;
-                IF has_expired THEN
-                    RETURN NEXT sch;
-                END IF;
-            END LOOP;
-        END;
-        $fn$;
-        """
-    )
-
-
-def _pg_downgrade() -> None:
-    # No-op: ``e5f6a7b8c9d0`` owns the lifecycle of these functions and drops
-    # them on its own downgrade. This migration only ever (re)creates them, so
-    # there is nothing to undo without racing that migration's DROP.
-    pass
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/b3c4d5e6f7a8_add_content_hash_to_chunks.py

@@ -9,8 +9,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "b3c4d5e6f7a8"
 down_revision: str | Sequence[str] | None = "a3b4c5d6e7f8"
 branch_labels: str | Sequence[str] | None = None
@@ -23,20 +21,12 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     schema = _get_schema_prefix()
     # Add content_hash column to chunks table for delta comparison
     op.execute(f"ALTER TABLE {schema}chunks ADD COLUMN IF NOT EXISTS content_hash TEXT")
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     schema = _get_schema_prefix()
     op.execute(f"ALTER TABLE {schema}chunks DROP COLUMN IF EXISTS content_hash")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)