fix(opencode): fix message parsing, shared state, and post-compaction retain

Three bugs fixed:
1. msg.role → msg.info.role: OpenCode SDK wraps role inside info, so all
   messages were silently filtered out, breaking retain and recall (#941)
2. Move PluginState to module level so it persists across sessions instead
   of being recreated per plugin instantiation
3. Reset lastRetainedTurn after compaction so idle-retain resumes when the
   message list shrinks

Co-Authored-By: Claude Opus 4.6 <[email protected]>

.claude/skills/code-review/SKILL.md

@@ -73,11 +73,6 @@ results = await asyncio.gather(*tasks, return_exceptions=True)
 results = await asyncio.gather(*tasks, return_exceptions=True)
 ```
 
-### API Layer & Data Access
-- **No direct database access in `api/http.py`** (or any API router). HTTP handlers must not build SQL, call `acquire_with_retry` / `conn.fetch` / `conn.fetchrow` / `conn.execute`, or reference `fq_table(...)`. All persistence and queries live in `MemoryEngine` (the engine layer). A handler parses/validates the request, calls an engine method, shapes the HTTP response, and maps domain results to status codes (e.g. a `None` return → 404).
-- **Authentication/tenancy is enforced inside each engine method, not assumed by the handler.** Every engine method that touches bank-scoped data must authenticate via `request_context` — typically `await self._authenticate_tenant(request_context)` (often indirectly through `get_bank_profile(...)`) — so the correct tenant schema is resolved before any query runs. Handlers must thread `request_context` through to the engine method; never query a tenant-scoped table assuming the schema is already set.
-- Engine methods return typed models (Pydantic/dataclass), not raw dicts (see Type Safety).
-
 ### Branch Hygiene
 - **Always start new feature branches from `origin/main`** — rebase to ensure a clean base.
 - **Only include commits relevant to the PR/branch/feature** — no unrelated changes. If the branch contains commits that don't belong, they must be removed before merging.
@@ -140,13 +135,6 @@ For each new or significantly changed function/endpoint/class:
 
 Flag any new logic that lacks test coverage.
 
-**LLM-behaviour changes need a real-LLM judge test, not MockLLM.** If the change alters how the model interprets a prompt — fact/observation extraction, `fact_type` (world/experience) classification, speaker attribution, instruction-following, prompt wording — there MUST be a test marked `pytest.mark.hs_llm_core` that runs the real pipeline and asserts via `tests.llm_judge.assert_meets_criteria` (not string/enum matching). Flag these as findings:
-- A prompt/classification change verified only by MockLLM or string assertions (MockLLM echoes input — such tests pass spuriously). **Should fix.**
-- A test that hard-asserts `fact_type == "world"/"experience"` (or other model-decided output) instead of judging it — non-deterministic, will flake across providers/runs. **Should fix** (move the classification check into the judge `criteria`; keep only genuinely deterministic structural asserts direct).
-- Deterministic mechanics (prompt assembly, suppression/branching logic) that are covered *only* by a slow LLM test — these should also have fast non-LLM unit tests. **Note.**
-
-See CLAUDE.md → Key Conventions → Testing for the full pattern.
-
 ### 7. Check API consistency
 
 If any files in `hindsight-api-slim/hindsight_api/api/` were changed:
@@ -154,12 +142,6 @@ If any files in `hindsight-api-slim/hindsight_api/api/` were changed:
 - Were the client SDKs regenerated? (`./scripts/generate-clients.sh`)
 - Were the control plane proxy routes updated? (`hindsight-control-plane/src/app/api/`)
 
-### 7b. Check API-layer data-access boundary
-
-For each changed handler in `hindsight-api-slim/hindsight_api/api/` (e.g. `http.py`, `mcp.py`):
-- **Flag any direct DB access in the handler** — `acquire_with_retry`, `conn.fetch` / `fetchrow` / `execute`, raw SQL strings, or `fq_table(...)`. These are a **must fix**: the query must be moved into a `MemoryEngine` method that returns a typed model, and the handler must call that method.
-- **Verify authentication is enforced in the engine** — the handler must delegate to an engine method that authenticates via `request_context` (`_authenticate_tenant`, typically through `get_bank_profile`). A handler that reads/writes tenant-scoped data without an engine method enforcing auth is a **must fix** (tenant data could leak across schemas).
-
 ### 8. Check code comments
 
 For each non-trivial change:
@@ -172,8 +154,7 @@ For each non-trivial change:
 If any files in `hindsight-integrations/` were added or changed, verify:
 - **Tests exist** — the integration must have tests that simulate/exercise the external framework (not just pure unit tests of helpers). Check for a `tests/` directory with meaningful test files.
 - **CI job exists** — check `.github/workflows/test.yml` for a corresponding `test-<name>-integration` job. If missing, flag it.
-- **Release process** — check that the integration name is in the `VALID_INTEGRATIONS` array in `scripts/release-integration.sh` AND in the `INTEGRATIONS` dict in `hindsight-dev/hindsight_dev/generate_changelog.py` (the changelog generator keeps its own list; a release fails at the changelog step if the name is missing there). If either is missing, flag it.
-- **Docs gallery + sidebar entry** — the integration must have an entry in `hindsight-docs/src/data/integrations.json`. This file is the **single source of truth** that drives both the integrations gallery and the docs sidebar (the sidebar category is injected from it at render time across all docs versions). The entry needs an internal `/sdks/integrations/<slug>` `link` and a matching page at `hindsight-docs/docs-integrations/<slug>.md(x)`. The `hindsight-docs/scripts/check-integrations.mjs` build step enforces both directions — forward: every internal JSON entry has a doc page; reverse: every released tag (`integrations/<name>/vX.Y.Z`) appears in the JSON (private infra like `cloudflare-oauth-proxy` is in the script's `EXCLUDED` set). Flag any integration that is released (or being released) but missing from `integrations.json`, and any JSON entry without a doc page. Do **not** hand-edit `versioned_sidebars/*.json` to add integration links — they are positional placeholders filled from the JSON.
+- **Release process** — check that the integration name is in the `VALID_INTEGRATIONS` array in `scripts/release-integration.sh`. If missing, flag it.
 - **Code standards** — the integration code must follow all Python style rules (type hints, no raw dicts, no tuple returns, etc.).
 
 ### 10. Check MCP tool registration completeness
@@ -185,14 +166,7 @@ If any new MCP tools were added or existing tools renamed in `hindsight-api-slim
 - **`MCP_TOOL_GROUPS`** in `hindsight-control-plane/src/components/bank-config-view.tsx` — must include the new tool in the appropriate group for the UI tool selector
 - **Tool count assertions** in tests (e.g., `test_mcp_tools.py`) — must be updated to reflect the new count
 
-### 11. Check backup/restore table coverage
-
-If a migration adds a new PostgreSQL table (look for `CREATE TABLE` / `op.create_table` in `hindsight-api-slim/hindsight_api/alembic/versions/`):
-- **`BACKUP_TABLES`** in `hindsight-api-slim/hindsight_api/admin/cli.py` — must include the new table, placed after any table it references via foreign key (parents before children). A missing entry is silent data loss: the table is never backed up, and restore's `TRUNCATE banks CASCADE` wipes any FK-to-banks child (e.g. `mental_models`, `directives`) on restore even though it was never saved.
-- The guard test `test_backup_tables_covers_entire_schema` in `tests/test_admin_backup_restore.py` enforces this — flag it as a **must fix** if a new table is absent from `BACKUP_TABLES`.
-- Oracle-only tables (e.g. `observation_sources`) are intentionally excluded — admin backup/restore is PostgreSQL-only.
-
-### 12. Review against other coding standards
+### 11. Review against other coding standards
 
 Check the diff for violations of the standards listed above:
 - Python files at project root (not allowed)
@@ -204,7 +178,7 @@ Check the diff for violations of the standards listed above:
 - Premature abstractions or speculative helpers
 - Backwards-compatibility hacks (unused vars, re-exports, "removed" comments)
 
-### 13. Report findings
+### 12. Report findings
 
 Present a clear summary organized by severity:
 
@@ -215,11 +189,7 @@ Present a clear summary organized by severity:
 - Raw dict usage for structured data (including internal code)
 - Multi-item tuple returns (including internal code)
 - Missing tests for new endpoints
-- Direct DB access (raw SQL / `acquire_with_retry` / `fq_table`) in an `api/` handler instead of a `MemoryEngine` method
-- Tenant-scoped data accessed without authentication enforced in the engine (`_authenticate_tenant` / `get_bank_profile`)
 - New integration missing tests, CI job, or release-integration.sh entry
-- Released/added integration missing from `hindsight-docs/src/data/integrations.json`, or a JSON entry with no `docs-integrations/<slug>` page (fails the docs build via `check-integrations.mjs`)
-- New PostgreSQL table missing from `BACKUP_TABLES` in `admin/cli.py` (silent data loss on restore)
 
 **Should fix** — issues that hurt code quality:
 - Dead code / unused imports missed by linter

.env.example

@@ -2,13 +2,11 @@
 # Copy this file to .env and fill in your values
 
 # LLM Configuration (Required)
-# Supported providers: openai, groq, ollama, gemini, anthropic, lmstudio, vertexai, minimax, deepseek, zai, volcano
+# Supported providers: openai, groq, ollama, gemini, anthropic, lmstudio, vertexai, minimax, volcano
 HINDSIGHT_API_LLM_PROVIDER=openai
 HINDSIGHT_API_LLM_API_KEY=your-api-key-here
 HINDSIGHT_API_LLM_MODEL=gpt-4o-mini
 HINDSIGHT_API_LLM_BASE_URL=https://api.openai.com/v1
-# Reasoning effort for providers/models that support it. Examples: low, medium, high, xhigh.
-# HINDSIGHT_API_LLM_REASONING_EFFORT=low
 
 # Example: Anthropic Claude configuration
 # HINDSIGHT_API_LLM_PROVIDER=anthropic
@@ -25,17 +23,7 @@ HINDSIGHT_API_LLM_BASE_URL=https://api.openai.com/v1
 # Example: MiniMax configuration (1M context window)
 # HINDSIGHT_API_LLM_PROVIDER=minimax
 # HINDSIGHT_API_LLM_API_KEY=your-minimax-api-key
-# HINDSIGHT_API_LLM_MODEL=MiniMax-M3  # or MiniMax-M2.7 for the previous generation
-
-# Example: DeepSeek configuration (https://api.deepseek.com)
-# HINDSIGHT_API_LLM_PROVIDER=deepseek
-# HINDSIGHT_API_LLM_API_KEY=your-deepseek-api-key
-# HINDSIGHT_API_LLM_MODEL=deepseek-v4-flash  # or deepseek-v4-pro / deepseek-chat / deepseek-reasoner
-
-# Example: z.ai configuration (Zhipu GLM series, https://z.ai)
-# HINDSIGHT_API_LLM_PROVIDER=zai
-# HINDSIGHT_API_LLM_API_KEY=your-zai-api-key
-# HINDSIGHT_API_LLM_MODEL=glm-4.5-flash  # or glm-4.5-air for the paid tier
+# HINDSIGHT_API_LLM_MODEL=MiniMax-M2.7
 
 # Example: LM Studio local configuration (Qwen 2.5 32B recommended)
 # HINDSIGHT_API_LLM_PROVIDER=lmstudio
@@ -56,7 +44,6 @@ HINDSIGHT_API_LOG_LEVEL=info
 
 # Database (Optional - uses embedded pg0 by default)
 # HINDSIGHT_API_DATABASE_URL=postgresql://user:pass@host:5432/db
-# HINDSIGHT_API_READ_DATABASE_URL=  # Optional read-replica URL. When set, recall queries (semantic, BM25, graph, temporal) flow through a separate pool against this URL, offloading the primary. Typically points to a read-only endpoint (CNPG's <cluster>-ro service or Aurora reader endpoint).
 # HINDSIGHT_API_MIGRATION_DATABASE_URL=  # Direct PostgreSQL URL for migrations (bypasses PgBouncer). Falls back to DATABASE_URL.
 # HINDSIGHT_API_DATABASE_SCHEMA=public  # PostgreSQL schema name (default: public)
 
@@ -66,59 +53,13 @@ HINDSIGHT_API_LOG_LEVEL=info
 # For Azure PostgreSQL with DiskANN:
 # HINDSIGHT_API_VECTOR_EXTENSION=pgvectorscale  # Auto-detects pg_diskann on Azure
 
-# Text Search Extension (Optional - uses native PostgreSQL full-text search by default)
-# Backend options: "native" (default), "vchord", "pg_textsearch", "pgroonga", "pg_search"
-# HINDSIGHT_API_TEXT_SEARCH_EXTENSION=native
-# Native backend dictionary (only used by HINDSIGHT_API_TEXT_SEARCH_EXTENSION=native)
-# HINDSIGHT_API_TEXT_SEARCH_EXTENSION_NATIVE_LANGUAGE=english
-# ParadeDB pg_search tokenizer (only used when creating pg_search BM25 indexes).
-# Empty uses ParadeDB's default tokenizer: unicode_words.
-# Supported values: unicode_words, simple, whitespace, literal, literal_normalized,
-# chinese_compatible, icu, jieba, source_code,
-# chinese_lindera/lindera(chinese), japanese_lindera/lindera(japanese),
-# korean_lindera/lindera(korean), ngram(min,max), edge_ngram(min,max)
-# HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER=
-
 # Embeddings Configuration (Optional - uses local by default)
-# Provider: "local" (default), "onnx", "tei", "openai", "cohere", "google", "openrouter", "zeroentropy", "litellm", or "litellm-sdk"
+# Provider: "local" (default) or "tei" (HuggingFace Text Embeddings Inference)
 # HINDSIGHT_API_EMBEDDINGS_PROVIDER=local
 # For local provider:
 # HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL=BAAI/bge-small-en-v1.5
-# For ONNX provider (local CPU embeddings without an Ollama/TEI sidecar):
-# HINDSIGHT_API_EMBEDDINGS_PROVIDER=onnx
-# HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_ID=intfloat/multilingual-e5-small
-# HINDSIGHT_API_EMBEDDINGS_ONNX_FILE=onnx/model.onnx
-# HINDSIGHT_API_EMBEDDINGS_ONNX_DIMENSIONS=384
-# HINDSIGHT_API_EMBEDDINGS_ONNX_MAX_TOKENS=512
-# HINDSIGHT_API_EMBEDDINGS_ONNX_POOLING=mean
-# HINDSIGHT_API_EMBEDDINGS_ONNX_NORMALIZE=true
-# HINDSIGHT_API_EMBEDDINGS_ONNX_QUERY_PREFIX="query: "
-# HINDSIGHT_API_EMBEDDINGS_ONNX_PASSAGE_PREFIX="passage: "
-# Optional for local model paths or pre-downloaded artifacts:
-# HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_PATH=/models/multilingual-e5-small/onnx/model.onnx
-# HINDSIGHT_API_EMBEDDINGS_ONNX_TOKENIZER_NAME_OR_PATH=/models/multilingual-e5-small
-# Optional for China network / restricted HF access:
-# HF_ENDPOINT=https://hf-mirror.com
 # For TEI provider:
 # HINDSIGHT_API_EMBEDDINGS_TEI_URL=http://localhost:8080
-# For OpenAI-compatible embeddings:
-# HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=sk-xxxx
-# HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL=text-embedding-3-small
-# HINDSIGHT_API_EMBEDDINGS_OPENAI_BASE_URL=https://api.openai.com/v1
-# For ZeroEntropy zembed-1:
-# HINDSIGHT_API_EMBEDDINGS_PROVIDER=zeroentropy
-# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_API_KEY=ze-xxxx
-# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_MODEL=zembed-1
-# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_DIMENSIONS=1280
-# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_ENCODING_FORMAT=float
-# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_LATENCY=fast
-#
-# IMPORTANT: Embedding keys require provider-specific names:
-# HINDSIGHT_API_EMBEDDINGS_{PROVIDER}_{PARAMETER}
-# (for example, HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL).
-#
-# DeepSeek note: DeepSeek is supported for LLM calls, but not for embeddings.
-# If using DeepSeek as LLM provider, keep embeddings on local/openai/cohere/google/etc.
 
 # Reranker Configuration (Optional - uses local by default)
 # Provider: "local" (default) or "tei" (HuggingFace Text Embeddings Inference)
@@ -142,20 +83,3 @@ HINDSIGHT_API_LOG_LEVEL=info
 # Custom service name and environment (optional, defaults: hindsight-api, development)
 # HINDSIGHT_API_OTEL_SERVICE_NAME=hindsight-production
 # HINDSIGHT_API_OTEL_DEPLOYMENT_ENVIRONMENT=production
-
-# -----------------------------------------------------------------------------
-# Control Plane (Optional)
-# -----------------------------------------------------------------------------
-
-# Dataplane API URL - where the CP proxies requests to
-# HINDSIGHT_CP_DATAPLANE_API_URL=http://localhost:8888
-
-# Optional: Bearer token the CP sends as `Authorization: Bearer <key>` to the
-# dataplane API. Required when the API service is auth-protected; omit for a
-# public/unauthenticated API.
-# HINDSIGHT_CP_DATAPLANE_API_KEY=your-dataplane-bearer-token
-
-# Optional: Require a shared access key to view the Control Plane UI.
-# When set, visitors see a login page and must enter the key before
-# accessing the dashboard or any /api/* routes (except /api/health).
-# HINDSIGHT_CP_ACCESS_KEY=your-shared-secret-key

.github/workflows/deploy-docs.yml

@@ -22,8 +22,6 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v6
-        with:
-          fetch-depth: 0 # fetch tags so check-released-integrations can see them
       - uses: actions/setup-node@v6
         with:
           node-version: 20
@@ -36,7 +34,7 @@ jobs:
         env:
           UMAMI_URL: https://analytics.hindsight.vectorize.io
           UMAMI_WEBSITE_ID: ${{ secrets.UMAMI_WEBSITE_ID }}
-      - uses: actions/upload-pages-artifact@v5
+      - uses: actions/upload-pages-artifact@v4
         with:
           path: hindsight-docs/build
   deploy:

.github/workflows/perf-test.yml

@@ -1,297 +0,0 @@
-name: Performance Tests
-
-on:
-  schedule:
-    # Run daily at 06:00 UTC
-    - cron: "0 6 * * *"
-  workflow_dispatch:
-    inputs:
-      scale:
-        description: "Test scale (perf-test)"
-        type: choice
-        options:
-          - tiny
-          - small
-          - medium
-          - large
-        default: large
-      suite:
-        description: "Perf-test suite to run (blank = all)"
-        type: choice
-        options:
-          - ""
-          - retain
-          - recall
-          - recall-with-observations
-          - recall-temporal
-          - consolidation
-          - graph-maintenance
-        default: ""
-      locomo_conversations:
-        description: "LoComo conversation IDs (space-separated). Blank = curated set (conv-26 conv-30 conv-43)."
-        type: string
-        default: ""
-      locomo_skip:
-        description: "Skip LoComo job"
-        type: boolean
-        default: false
-      obs_skip:
-        description: "Skip observation-dedup benchmark job"
-        type: boolean
-        default: false
-      obs_dataset:
-        description: "Obs benchmark dataset substring (blank = English hermes transcript)."
-        type: string
-        default: ""
-      obs_fraction:
-        description: "Obs benchmark fraction (0-1] of each document to run."
-        type: string
-        default: "1.0"
-      ref:
-        description: "Git ref to test (branch, tag, or SHA). Defaults to main."
-        type: string
-        default: ""
-
-concurrency:
-  group: perf-test
-  cancel-in-progress: true
-
-jobs:
-  perf-test:
-    runs-on: ubuntu-latest
-    steps:
-    - uses: actions/checkout@v6
-      with:
-        ref: ${{ inputs.ref || github.ref }}
-
-    - name: Install uv
-      uses: astral-sh/setup-uv@v7
-      with:
-        enable-cache: true
-        prune-cache: false
-
-    - name: Set up Python
-      uses: actions/setup-python@v6
-      with:
-        python-version-file: ".python-version"
-
-    - name: Cache HuggingFace models
-      uses: actions/cache@v5
-      with:
-        path: ~/.cache/huggingface
-        key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
-        restore-keys: |
-          ${{ runner.os }}-huggingface-
-
-    - name: Pre-download models
-      working-directory: ./hindsight-api-slim
-      run: |
-        uv run --frozen --all-extras --index-strategy unsafe-best-match python -c "
-        from sentence_transformers import SentenceTransformer
-        print('Downloading embedding model...')
-        SentenceTransformer('BAAI/bge-small-en-v1.5')
-        print('Model downloaded successfully')
-        "
-
-    - name: Install hindsight-dev dependencies
-      run: |
-        cd hindsight-dev && uv sync --frozen --all-extras --index-strategy unsafe-best-match
-
-    - name: Run perf-test
-      run: |
-        SUITE_ARG=""
-        if [ -n "${{ inputs.suite }}" ]; then
-          SUITE_ARG="--suite ${{ inputs.suite }}"
-        fi
-        ./scripts/benchmarks/run-perf-test.sh \
-          --scale ${{ inputs.scale || 'large' }} \
-          $SUITE_ARG \
-          --output perf-results.json
-
-    - name: Upload perf results
-      if: always()
-      uses: actions/upload-artifact@v7
-      with:
-        name: perf-results-${{ github.sha }}
-        path: hindsight-dev/perf-results.json
-        retention-days: 90
-
-    # Publish enriched results (perf JSON + commit metadata) to the dashboard
-    # repo's gh-pages branch. The static site at
-    #   https://vectorize-io.github.io/hindsight-continuous-performance-monitor/
-    # reads data/index.json + data/<run>.json and renders charts client-side.
-    - name: Publish to dashboard
-      if: github.event_name == 'schedule' || github.event_name == 'push' || github.event_name == 'workflow_dispatch'
-      env:
-        PERF_DASHBOARD_TOKEN: ${{ secrets.PERF_DASHBOARD_TOKEN }}
-        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      run: ./scripts/benchmarks/publish-perf-results.sh hindsight-dev/perf-results.json
-
-  locomo:
-    if: inputs.locomo_skip != true
-    runs-on: ubuntu-latest
-    env:
-      HINDSIGHT_API_LLM_PROVIDER: vertexai
-      HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
-      HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
-      HINDSIGHT_API_JUDGE_LLM_PROVIDER: vertexai
-      HINDSIGHT_API_JUDGE_LLM_MODEL: google/gemini-2.5-flash-lite
-      HINDSIGHT_API_ANSWER_LLM_PROVIDER: vertexai
-      HINDSIGHT_API_ANSWER_LLM_MODEL: google/gemini-2.5-flash
-    steps:
-    - uses: actions/checkout@v6
-      with:
-        ref: ${{ inputs.ref || github.ref }}
-
-    - name: Setup GCP credentials
-      run: |
-        printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
-        PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
-        echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
-
-    - name: Install uv
-      uses: astral-sh/setup-uv@v7
-      with:
-        enable-cache: true
-        prune-cache: false
-
-    - name: Set up Python
-      uses: actions/setup-python@v6
-      with:
-        python-version-file: ".python-version"
-
-    - name: Cache HuggingFace models
-      uses: actions/cache@v5
-      with:
-        path: ~/.cache/huggingface
-        key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
-        restore-keys: |
-          ${{ runner.os }}-huggingface-
-
-    - name: Pre-download models
-      working-directory: ./hindsight-api-slim
-      run: |
-        uv run --frozen --all-extras --index-strategy unsafe-best-match python -c "
-        from sentence_transformers import SentenceTransformer
-        print('Downloading embedding model...')
-        SentenceTransformer('BAAI/bge-small-en-v1.5')
-        print('Model downloaded successfully')
-        "
-
-    - name: Install hindsight-dev dependencies
-      run: |
-        cd hindsight-dev && uv sync --frozen --all-extras --index-strategy unsafe-best-match
-
-    - name: Run LoComo benchmark
-      # Curated 3-conversation subset (best/middle/worst by accuracy on the
-      # last successful full run): conv-26 (best), conv-30 (middle), conv-43
-      # (worst). Excludes conv-44, the bank with the largest unconsolidated
-      # set that has been pushing scheduled runs over the per-bank
-      # _wait_for_consolidation timeout. Override via workflow_dispatch with
-      # the locomo_conversations input.
-      run: |
-        CONVERSATIONS="${{ inputs.locomo_conversations }}"
-        if [ -z "$CONVERSATIONS" ]; then
-          CONVERSATIONS="conv-26 conv-30 conv-43"
-        fi
-        uv run python hindsight-dev/benchmarks/locomo/locomo_benchmark.py \
-          --wait-consolidation \
-          --conversation $CONVERSATIONS
-
-    - name: Upload LoComo results
-      if: always()
-      uses: actions/upload-artifact@v7
-      with:
-        name: locomo-results-${{ github.sha }}
-        path: hindsight-dev/benchmarks/locomo/results/
-        retention-days: 90
-
-    - name: Publish LoComo to dashboard
-      if: success() && (github.event_name == 'schedule' || github.event_name == 'push' || github.event_name == 'workflow_dispatch')
-      env:
-        PERF_DASHBOARD_TOKEN: ${{ secrets.PERF_DASHBOARD_TOKEN }}
-        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      run: ./scripts/benchmarks/publish-locomo-results.sh hindsight-dev/benchmarks/locomo/results/benchmark_results.json
-
-  obs:
-    # Observation-dedup quality benchmark: ingests a transcript, drains consolidation
-    # (serial SyncTaskBackend + embedded pg0 — no external DB / worker), and reports the
-    # near-duplicate observation rate. Real LLM via VertexAI, mirroring the LoComo job.
-    if: inputs.obs_skip != true
-    runs-on: ubuntu-latest
-    env:
-      HINDSIGHT_API_LLM_PROVIDER: vertexai
-      HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
-      HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
-      HINDSIGHT_API_ENABLE_OBSERVATIONS: "true"
-    steps:
-    - uses: actions/checkout@v6
-      with:
-        ref: ${{ inputs.ref || github.ref }}
-
-    - name: Setup GCP credentials
-      run: |
-        printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
-        PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
-        echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
-
-    - name: Install uv
-      uses: astral-sh/setup-uv@v7
-      with:
-        enable-cache: true
-        prune-cache: false
-
-    - name: Set up Python
-      uses: actions/setup-python@v6
-      with:
-        python-version-file: ".python-version"
-
-    - name: Cache HuggingFace models
-      uses: actions/cache@v5
-      with:
-        path: ~/.cache/huggingface
-        key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
-        restore-keys: |
-          ${{ runner.os }}-huggingface-
-
-    - name: Pre-download models
-      working-directory: ./hindsight-api-slim
-      run: |
-        uv run --frozen --all-extras --index-strategy unsafe-best-match python -c "
-        from sentence_transformers import SentenceTransformer
-        print('Downloading embedding model...')
-        SentenceTransformer('BAAI/bge-small-en-v1.5')
-        print('Model downloaded successfully')
-        "
-
-    - name: Install hindsight-dev dependencies
-      run: |
-        cd hindsight-dev && uv sync --frozen --all-extras --index-strategy unsafe-best-match
-
-    - name: Run obs benchmark
-      # Default to the English hermes transcript at full fraction — a clean, deterministic
-      # consolidation-dedup signal (the Chinese variant adds a cross-lingual embedding
-      # confound). Override dataset/fraction via workflow_dispatch.
-      run: |
-        DATASET="${{ inputs.obs_dataset }}"
-        if [ -z "$DATASET" ]; then DATASET="hermes_session_2026-05-15_en"; fi
-        FRACTION="${{ inputs.obs_fraction }}"
-        if [ -z "$FRACTION" ]; then FRACTION="1.0"; fi
-        cd hindsight-dev
-        uv run python -m benchmarks.obs.obs_benchmark \
-          --dataset "$DATASET" --fraction "$FRACTION" --wipe-bank --output obs-results.json
-
-    - name: Upload obs results
-      if: always()
-      uses: actions/upload-artifact@v7
-      with:
-        name: obs-results-${{ github.sha }}
-        path: hindsight-dev/obs-results.json
-        retention-days: 90
-
-    - name: Publish obs to dashboard
-      if: success() && (github.event_name == 'schedule' || github.event_name == 'push' || github.event_name == 'workflow_dispatch')
-      env:
-        PERF_DASHBOARD_TOKEN: ${{ secrets.PERF_DASHBOARD_TOKEN }}
-        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      run: ./scripts/benchmarks/publish-obs-results.sh hindsight-dev/obs-results.json

.github/workflows/release-integration.yml

@@ -81,28 +81,17 @@ jobs:
       with:
         node-version: '22'
         registry-url: 'https://registry.npmjs.org'
-        cache: 'npm'
-        cache-dependency-path: package-lock.json
 
+    # Guard: fail fast if the integration's lockfile resolves any dep from a
+    # monorepo workspace (link=true) or a relative file path. The release
+    # runner has no pre-built workspace `dist/` so `npm run build` would
+    # later fail at tsc with "Cannot find module". See:
+    # https://github.com/vectorize-io/hindsight/issues/… (0.6.0 openclaw retry)
     - name: Check integration lockfile
       if: steps.type.outputs.type == 'typescript'
       run: ./scripts/check-integration-lockfiles.sh
 
-    # Some integrations depend on workspace packages (hindsight-client,
-    # hindsight-all, hindsight-agent-sdk) via file: refs. Install from root
-    # so npm resolves them, then build the workspace deps before the integration.
-    - name: Install root workspace dependencies
-      if: steps.type.outputs.type == 'typescript'
-      run: npm ci
-
-    - name: Build workspace deps (hindsight-client, hindsight-all, hindsight-agent-sdk)
-      if: steps.type.outputs.type == 'typescript'
-      run: |
-        npm run build --workspace=hindsight-clients/typescript
-        npm run build --workspace=hindsight-all-npm
-        npm run build --workspace=hindsight-tools/hindsight-agent-sdk
-
-    - name: Install integration dependencies
+    - name: Install dependencies
       if: steps.type.outputs.type == 'typescript'
       working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
       run: npm ci
@@ -117,7 +106,7 @@ jobs:
       working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
       run: |
         set +e
-        OUTPUT=$(npm publish --access public --provenance 2>&1)
+        OUTPUT=$(npm publish --access public 2>&1)
         EXIT_CODE=$?
         echo "$OUTPUT"
         if [ $EXIT_CODE -ne 0 ]; then

.github/workflows/release-tool.yml

@@ -1,65 +0,0 @@
-name: Release Tool
-
-on:
-  push:
-    tags:
-      - 'tools/**'
-
-jobs:
-  publish:
-    runs-on: ubuntu-latest
-
-    steps:
-    - uses: actions/checkout@v6
-
-    - name: Extract tool info
-      id: info
-      run: |
-        # refs/tags/tools/self-driving-agents/v0.0.1 → tool=self-driving-agents, version=0.0.1
-        TAG="${GITHUB_REF#refs/tags/}"
-        TOOL=$(echo "$TAG" | cut -d'/' -f2)
-        VERSION=$(echo "$TAG" | cut -d'/' -f3 | sed 's/^v//')
-        echo "tool=$TOOL" >> $GITHUB_OUTPUT
-        echo "version=$VERSION" >> $GITHUB_OUTPUT
-        echo "tag=$TAG" >> $GITHUB_OUTPUT
-        echo "Tool: $TOOL, Version: $VERSION"
-
-    - name: Set up Node.js
-      uses: actions/setup-node@v6
-      with:
-        node-version: '22'
-        registry-url: 'https://registry.npmjs.org'
-        cache: 'npm'
-        cache-dependency-path: package-lock.json
-
-    # Tools live under hindsight-tools/ and may depend on workspace packages
-    # (e.g. @vectorize-io/hindsight-client). Install from root so npm resolves
-    # workspace deps, then build any required workspace packages first.
-    - name: Install root workspace dependencies
-      run: npm ci
-
-    - name: Build hindsight-client (workspace dep)
-      run: npm run build --workspace=hindsight-clients/typescript
-
-    - name: Build hindsight-agent-sdk (workspace dep)
-      run: npm run build --workspace=hindsight-tools/hindsight-agent-sdk
-
-    - name: Build tool
-      run: npm run build --workspace=hindsight-tools/${{ steps.info.outputs.tool }}
-
-    - name: Publish to npm
-      working-directory: ./hindsight-tools/${{ steps.info.outputs.tool }}
-      run: |
-        set +e
-        OUTPUT=$(npm publish --access public 2>&1)
-        EXIT_CODE=$?
-        echo "$OUTPUT"
-        if [ $EXIT_CODE -ne 0 ]; then
-          if echo "$OUTPUT" | grep -q "cannot publish over"; then
-            echo "Package version already published, skipping..."
-            exit 0
-          fi
-          exit $EXIT_CODE
-        fi
-      env:
-        NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

.github/workflows/release.yml

@@ -314,7 +314,6 @@ jobs:
     permissions:
       contents: read
       packages: write
-      id-token: write
     strategy:
       matrix:
         include:
@@ -411,7 +410,6 @@ jobs:
 
     # Build multi-platform and push to release tags
     - name: Build and push release images
-      id: build
       uses: docker/build-push-action@v7
       with:
         context: .
@@ -423,31 +421,6 @@ jobs:
         tags: ${{ steps.meta.outputs.tags }}
         labels: ${{ steps.meta.outputs.labels }}
 
-    - name: Install cosign
-      uses: sigstore/cosign-installer@v3
-
-    - name: Sign published images
-      env:
-        TAGS: ${{ steps.meta.outputs.tags }}
-        DIGEST: ${{ steps.build.outputs.digest }}
-      run: |
-        set -euo pipefail
-        refs=()
-        while IFS= read -r tag; do
-          [[ -z "${tag}" ]] && continue
-          refs+=("${tag}@${DIGEST}")
-        done <<< "${TAGS}"
-        cosign sign --yes "${refs[@]}"
-
-    - name: Verify signature on primary tag
-      env:
-        IMAGE: ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}
-        DIGEST: ${{ steps.build.outputs.digest }}
-      run: |
-        cosign verify "${IMAGE}@${DIGEST}" \
-          --certificate-identity-regexp "^https://github\.com/${{ github.repository }}/\.github/workflows/release\.yml@.*" \
-          --certificate-oidc-issuer https://token.actions.githubusercontent.com
-
   release-helm-chart:
     runs-on: ubuntu-latest
     permissions:
@@ -524,12 +497,6 @@ jobs:
         name: rust-cli-hindsight-linux-amd64
         path: ./artifacts/rust-cli-linux
 
-    - name: Download Rust CLI (Linux ARM)
-      uses: actions/download-artifact@v8
-      with:
-        name: rust-cli-hindsight-linux-arm64
-        path: ./artifacts/rust-cli-linux-arm64
-
     - name: Download Rust CLI (macOS Intel)
       uses: actions/download-artifact@v8
       with:
@@ -566,7 +533,6 @@ jobs:
         cp artifacts/control-plane/*.tgz release-assets/ || true
         # Rust CLI binaries
         cp artifacts/rust-cli-linux/hindsight-linux-amd64 release-assets/ || true
-        cp artifacts/rust-cli-linux-arm64/hindsight-linux-arm64 release-assets/ || true
         cp artifacts/rust-cli-darwin-amd64/hindsight-darwin-amd64 release-assets/ || true
         cp artifacts/rust-cli-darwin-arm64/hindsight-darwin-arm64 release-assets/ || true
         # Helm chart

.github/workflows/sign-images.yml

@@ -1,71 +0,0 @@
-name: Sign published images
-
-on:
-  workflow_dispatch:
-    inputs:
-      version:
-        description: 'Version to sign (without leading v, e.g. 0.6.0)'
-        required: true
-        type: string
-        default: '0.6.0'
-
-permissions:
-  contents: read
-  packages: write
-  id-token: write
-
-jobs:
-  sign:
-    name: Sign ${{ matrix.image }}:${{ inputs.version }}${{ matrix.suffix }}
-    runs-on: ubuntu-latest
-    strategy:
-      fail-fast: false
-      matrix:
-        include:
-          - { image: hindsight-api, suffix: '' }
-          - { image: hindsight-api, suffix: '-slim' }
-          - { image: hindsight-control-plane, suffix: '' }
-          - { image: hindsight, suffix: '' }
-          - { image: hindsight, suffix: '-slim' }
-
-    steps:
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v4
-
-      - name: Log in to GHCR
-        uses: docker/login-action@v4
-        with:
-          registry: ghcr.io
-          username: ${{ github.actor }}
-          password: ${{ secrets.GITHUB_TOKEN }}
-
-      - name: Install cosign
-        uses: sigstore/cosign-installer@v3
-
-      - name: Resolve image digest
-        id: resolve
-        env:
-          IMAGE: ghcr.io/${{ github.repository_owner }}/${{ matrix.image }}
-          TAG: ${{ inputs.version }}${{ matrix.suffix }}
-        run: |
-          set -euo pipefail
-          DIGEST=$(docker buildx imagetools inspect "${IMAGE}:${TAG}" --format '{{json .Manifest.Digest}}' | tr -d '"')
-          if [[ -z "${DIGEST}" || "${DIGEST}" != sha256:* ]]; then
-            echo "Failed to resolve digest for ${IMAGE}:${TAG} (got: ${DIGEST})" >&2
-            exit 1
-          fi
-          echo "Resolved ${IMAGE}:${TAG} -> ${DIGEST}"
-          echo "ref=${IMAGE}@${DIGEST}" >> "$GITHUB_OUTPUT"
-
-      - name: Sign image
-        env:
-          REF: ${{ steps.resolve.outputs.ref }}
-        run: cosign sign --yes "${REF}"
-
-      - name: Verify signature
-        env:
-          REF: ${{ steps.resolve.outputs.ref }}
-        run: |
-          cosign verify "${REF}" \
-            --certificate-identity-regexp "^https://github\.com/${{ github.repository }}/\.github/workflows/sign-images\.yml@.*" \
-            --certificate-oidc-issuer https://token.actions.githubusercontent.com

.github/workflows/windows-smoke.yml

@@ -1,115 +0,0 @@
-name: Windows Smoke Test
-
-# Daily smoke test that installs the API on Windows and runs the Python client
-# integration tests against a live server. Windows is only exercised by the
-# hindsight-embed jobs in test.yml on PRs; this catches Windows-specific
-# regressions in the API server + client path (e.g. process spawning, console
-# subsystem / ConPTY behaviour, see #1885) that the Linux client jobs miss.
-on:
-  schedule:
-    # 06:00 UTC daily.
-    - cron: "0 6 * * *"
-  workflow_dispatch:
-
-permissions:
-  contents: read
-
-jobs:
-  windows-client-smoke:
-    # Don't run on forks: the job needs the org's Vertex AI credentials.
-    if: github.repository == 'vectorize-io/hindsight'
-    runs-on: windows-latest
-    timeout-minutes: 45
-    env:
-      HINDSIGHT_API_LLM_PROVIDER: vertexai
-      HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: ${{ github.workspace }}/gcp-credentials.json
-      HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
-      HINDSIGHT_API_URL: http://localhost:8888
-      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      # Force UTF-8 I/O so the API/CLI's ✓/box-drawing output doesn't crash the
-      # default Windows cp1252 codec (matches test-embed-windows in test.yml).
-      PYTHONIOENCODING: utf-8
-      PYTHONUTF8: "1"
-
-    steps:
-      - uses: actions/checkout@v6
-
-      - name: Setup GCP credentials
-        shell: bash
-        run: |
-          printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > gcp-credentials.json
-          PROJECT_ID=$(jq -r '.project_id' gcp-credentials.json)
-          echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> "$GITHUB_ENV"
-
-      - name: Install uv
-        uses: astral-sh/setup-uv@v7
-        with:
-          enable-cache: true
-          prune-cache: false
-
-      - name: Set up Python
-        uses: actions/setup-python@v6
-        with:
-          python-version-file: ".python-version"
-
-      - name: Install API dependencies (all extras - local-ml + embedded pg0)
-        working-directory: ./hindsight-api-slim
-        run: uv sync --frozen --all-extras --index-strategy unsafe-best-match
-
-      - name: Install Python client test dependencies
-        working-directory: ./hindsight-clients/python
-        run: uv sync --frozen --extra test --index-strategy unsafe-best-match
-
-      - name: Cache HuggingFace models
-        uses: actions/cache@v5
-        with:
-          path: ~/.cache/huggingface
-          key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
-          restore-keys: |
-            ${{ runner.os }}-huggingface-
-
-      # `uv run` re-syncs the project env to its default (no-extras) state before
-      # running, which drops sentence-transformers / pg0. Pass --all-extras on
-      # every `uv run` so the local-ml + embedded-db deps stay installed (this is
-      # the same reason hindsight-embed launches the daemon with `--extra all`).
-      - name: Pre-download models
-        working-directory: ./hindsight-api-slim
-        run: |
-          uv run --all-extras python -c "from sentence_transformers import SentenceTransformer, CrossEncoder; SentenceTransformer('BAAI/bge-small-en-v1.5'); CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2'); print('Models downloaded')"
-
-      # Start the server and run the client tests in a SINGLE step. On Windows
-      # runners a process backgrounded with `&` in one step is not reliably kept
-      # alive for later steps (unlike Linux, where it reparents to init), so the
-      # server must live in the same shell that runs pytest.
-      - name: Start API server and run Python client tests
-        shell: bash
-        run: |
-          # Config is read straight from the environment (job-level env + the
-          # PROJECT_ID exported to GITHUB_ENV above), so no .env file is needed.
-          # Embedded pg0 is the default when HINDSIGHT_API_DATABASE_URL is unset.
-          ( cd hindsight-api-slim && uv run --all-extras hindsight-api --port 8888 ) > "$RUNNER_TEMP/api-server.log" 2>&1 &
-          server_pid=$!
-          echo "Waiting for API server to be ready (pid $server_pid)..."
-          # pg0 unpacks Postgres + runs initdb on first boot, which is slow on a
-          # cold Windows runner — give it a generous budget before failing.
-          ready=false
-          for i in $(seq 1 300); do
-            if curl -sf http://localhost:8888/health > /dev/null 2>&1; then
-              echo "API server is ready after ${i}s"
-              ready=true
-              break
-            fi
-            sleep 1
-          done
-          if [ "$ready" != true ]; then
-            echo "API server failed to start after 300s"
-            cat "$RUNNER_TEMP/api-server.log"
-            exit 1
-          fi
-
-          cd hindsight-clients/python && uv run --extra test pytest tests -v
-
-      - name: Show API server logs
-        if: always()
-        shell: bash
-        run: cat "$RUNNER_TEMP/api-server.log" || echo "No API server log found"

.gitignore

@@ -54,10 +54,7 @@ hindsight-clients/rust/target
 !.claude/skills/
 whats-next.md
 TASK.md
-# Parked / draft integrations that aren't ready to ship
-hindsight-integrations/_drafts/
 # Changelog is now tracked in hindsight-docs/src/pages/changelog.md
 # CHANGELOG.md
 
-blog-post*
-.worktrees/
+blog-post*

.prettierrc.json

@@ -1,7 +0,0 @@
-{
-  "semi": true,
-  "singleQuote": false,
-  "tabWidth": 2,
-  "trailingComma": "es5",
-  "printWidth": 100
-}

CLAUDE.md

@@ -68,9 +68,8 @@ cd hindsight-control-plane && npm run dev
 ./scripts/benchmarks/run-locomo.sh
 
 # Performance benchmarks
-./scripts/benchmarks/run-perf-test.sh                      # System perf (mock LLM + pg0)
-./scripts/benchmarks/run-perf-test.sh --scale tiny          # Quick smoke test
 ./scripts/benchmarks/run-consolidation.sh
+./scripts/benchmarks/run-retain-perf.sh --document <path>  # Requires API server running
 
 # Results viewer
 ./scripts/benchmarks/start-visualizer.sh  # View results at localhost:8001
@@ -123,17 +122,12 @@ Key tables: `banks`, `memory_units`, `documents`, `entities`, `entity_links`
 
 ### Adding Database Migrations
 
-Hindsight runs the same Alembic tree against PostgreSQL and Oracle 23ai. Each
-migration file dispatches through `run_for_dialect`, which calls either
-`_pg_upgrade` or `_oracle_upgrade` based on the live connection. A pytest lint
-(`tests/test_migration_shape.py`) fails CI if a migration omits the dispatcher.
-
 1. **Create a new migration file** in `hindsight-api-slim/hindsight_api/alembic/versions/`:
    - File name format: `<revision_id>_<description>.py` (e.g., `f1a2b3c4d5e6_add_new_index.py`)
    - Use a unique hex revision ID (12 chars)
    - Set `down_revision` to the previous migration's revision ID
 
-2. **Migration template** (the `script.py.mako` template scaffolds this; fill in the bodies):
+2. **Migration template**:
    ```python
    """Description of the migration
 
@@ -144,58 +138,25 @@ migration file dispatches through `run_for_dialect`, which calls either
    from collections.abc import Sequence
    from alembic import context, op
 
-   from hindsight_api.alembic._dialect import run_for_dialect
-
    revision: str = "f1a2b3c4d5e6"
    down_revision: str | Sequence[str] | None = "<previous_revision_id>"
    branch_labels: str | Sequence[str] | None = None
    depends_on: str | Sequence[str] | None = None
 
-
-   def _pg_schema_prefix() -> str:
-       """Schema-qualifier for raw SQL on PG (multi-tenant search_path)."""
+   def _get_schema_prefix() -> str:
+       """Get schema prefix for table names (required for multi-tenant support)."""
        schema = context.config.get_main_option("target_schema")
        return f'"{schema}".' if schema else ""
 
-
-   def _pg_upgrade() -> None:
-       schema = _pg_schema_prefix()
+   def upgrade() -> None:
+       schema = _get_schema_prefix()
        op.execute(f"CREATE INDEX ... ON {schema}table_name(...)")
 
-
-   def _pg_downgrade() -> None:
-       schema = _pg_schema_prefix()
-       op.execute(f"DROP INDEX IF EXISTS {schema}index_name")
-
-
-   def _oracle_upgrade() -> None:
-       # Oracle 23ai equivalent. Use op.get_bind().exec_driver_sql for forms
-       # that Alembic core does not model (vector/text indexes, partitions).
-       op.execute("CREATE INDEX ... ON table_name(...)")
-
-
-   def _oracle_downgrade() -> None:
-       op.execute("DROP INDEX IF EXISTS index_name")
-
-
-   def upgrade() -> None:
-       run_for_dialect(pg=_pg_upgrade, oracle=_oracle_upgrade)
-
-
    def downgrade() -> None:
-       run_for_dialect(pg=_pg_downgrade, oracle=_oracle_downgrade)
+       schema = _get_schema_prefix()
+       op.execute(f"DROP INDEX IF EXISTS {schema}index_name")
    ```
 
-   **Dialect-only migrations.** If a change genuinely doesn't apply to one
-   dialect (e.g. enabling `pg_trgm` is PG-only), omit the unused slot:
-   ```python
-   def upgrade() -> None:
-       run_for_dialect(pg=_pg_upgrade)  # oracle slot intentionally absent → no-op
-   ```
-   Make the asymmetry deliberate. Don't leave an Oracle slot empty just because
-   you didn't think about it — copy-pasting a PG migration without the Oracle
-   half is exactly how schemas drift.
-
 3. **Run migrations locally**:
    ```bash
    # Set database URL and run migrations for the base schema plus all tenants
@@ -220,30 +181,6 @@ migration file dispatches through `run_for_dialect`, which calls either
 
 **MANDATORY: Run `/code-review` before pushing code or creating a pull request.** Do not push or create a PR until all "must fix" issues are resolved.
 
-### Testing
-
-Most tests are deterministic (MockLLM, pure functions) — assert directly.
-
-**Tests that verify LLM behaviour use a real LLM + an LLM-as-judge.** When the thing under test is *how the model interprets a prompt* (classification, attribution, dimension preservation, instruction-following), MockLLM can't simulate it and exact string/enum asserts flake across providers and runs. Use this pattern instead:
-
-1. Mark the test module `pytestmark = pytest.mark.hs_llm_core` (single-provider; CI runs it in the core-LLM job). Use `hs_llm_mat` only for provider-matrix acceptance tests.
-2. Call the real pipeline (`LLMConfig.from_env()`, `_get_raw_config()`), e.g. `extract_facts_from_text(...)`.
-3. Assert with the judge, not string matching:
-   ```python
-   from tests.llm_judge import assert_meets_criteria
-   facts_summary = "\n".join(f"- [{f.fact_type}] {f.fact}" for f in facts)
-   await assert_meets_criteria(
-       response=facts_summary,
-       criteria="The first-person user statements are classified 'world' and attributed to the user, not the agent.",
-       context="What the input said and who was speaking.",
-   )
-   ```
-
-Rules of thumb:
-- **Judge anything non-deterministic** — including `fact_type` classification and speaker attribution. Do NOT hard-assert `fact_type == "..."`; pass a `[fact_type] fact` summary to the judge instead. Structural facts that ARE deterministic (counts, presence of a field, that a substring was injected into a prompt) stay as direct asserts in fast unit tests.
-- **Split the test surface**: cover the deterministic mechanics (prompt assembly, suppression logic) with fast non-LLM unit tests, and the model-following behaviour with one `hs_llm_core` judge test. (Example pair: `test_narrator_resolution.py` + `test_narrator_context_override.py`.)
-- The judge model is independent of the test provider (defaults to Gemini); never judge with the same call you're testing.
-
 ### Memory Banks
 - Each bank is an isolated memory store (like a "brain" for one user/agent)
 - Banks have dispositions (skepticism, literalism, empathy traits 1-5) affecting reflect

CONTRIBUTING.md

@@ -9,36 +9,13 @@ Thanks for your interest in contributing to Hindsight!
    git clone [email protected]:vectorize-io/hindsight.git
    cd hindsight
    ```
-
-2. Bootstrap your dev environment in one shot:
-   ```bash
-   ./scripts/dev/setup.sh
-   ```
-   This is idempotent (safe to re-run) and gets you ready to develop, including
-   offline. It:
-   - installs the required toolchains if missing (uv/Python, Node/npm, Rust/cargo),
-   - creates `.env` from `.env.example` (remember to add your LLM API key),
-   - configures git hooks,
-   - installs all Python and Node workspace dependencies,
-   - pre-downloads the local ML models + tokenizer so the API runs offline,
-   - builds the TypeScript SDK and the Rust CLI.
-
-   Useful flags: `--skip-build` (deps only), `--skip-models` (skip ML model
-   download), `--with-docs` (also build the docs site), `--force` (rebuild
-   artifacts). Docker image builds are out of scope. Run
-   `./scripts/dev/setup.sh --help` for details.
-
-### Manual setup
-
-If you'd rather set things up by hand instead of running the script above:
-
-1. Set up your environment:
+2. Set up your environment:
    ```bash
    cp .env.example .env
    ```
    Edit the .env to add LLM API key and config as required
 
-2. Install dependencies:
+3. Install dependencies:
    ```bash
    # Python dependencies
    uv sync --directory hindsight-api/

README.md

@@ -30,7 +30,7 @@ It eliminates the shortcomings of alternative techniques such as RAG and knowled
 
 Hindsight is the most accurate agent memory system ever tested according to benchmark performance. It has achieved state-of-the-art performance on the LongMemEval benchmark, widely used to assess memory system performance across a variety of conversational AI scenarios. The current reported performance of Hindsight and other agent memory solutions as of January 2026 is shown here:
 
-![Overview](./hindsight-docs/static/img/hindsight-benchmarks.png)
+![Overview](./hindsight-docs/static/img/hindsight-bench.jpg)
 
 The benchmark performance data for Hindsight has been independently reproduced by research collaborators at the Virginia Tech [Sanghani Center for Artificial Intelligence and Data Analytics](https://sanghani.cs.vt.edu/) and The Washington Post. Other scores are self-reported by software vendors.
 
@@ -62,9 +62,9 @@ If you need more control over how and when your agent stores and recalls memorie
 ```bash
 export OPENAI_API_KEY=sk-xxx
 
-docker run -it --pull always --name hindsight --restart unless-stopped -p 8888:8888 -p 9999:9999 \
+docker run --rm -it --pull always -p 8888:8888 -p 9999:9999 \
   -e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \
-  -v hindsight-data:/home/hindsight/.pg0 \
+  -v $HOME/.hindsight-docker:/home/hindsight/.pg0 \
   ghcr.io/vectorize-io/hindsight:latest
 ```
 
@@ -84,8 +84,6 @@ cd docker/docker-compose
 docker compose up 
 ```
 
-> Oracle AI Database is also supported for enterprise deployments with full feature parity. See the [storage documentation](https://hindsight.vectorize.io/developer/storage) for details.
-
 
 >API: http://localhost:8888
 >UI: http://localhost:9999

deno.lock

@@ -42,16 +42,6 @@
   },
   "workspace": {
     "members": {
-      "hindsight-all-npm": {
-        "packageJson": {
-          "dependencies": [
-            "npm:@types/node@22",
-            "npm:tsup@^8.5.1",
-            "npm:typescript@^5.7.0",
-            "npm:vitest@^4.1.2"
-          ]
-        }
-      },
       "hindsight-clients/typescript": {
         "packageJson": {
           "dependencies": [
@@ -68,7 +58,6 @@
       "hindsight-control-plane": {
         "packageJson": {
           "dependencies": [
-            "npm:@chenglou/pretext@^0.0.3",
             "npm:@eslint/eslintrc@^3.3.3",
             "npm:@eslint/js@^9.39.2",
             "npm:@radix-ui/react-alert-dialog@^1.1.15",
@@ -102,12 +91,11 @@
             "npm:eslint@^9.39.1",
             "npm:[email protected]",
             "npm:next-themes@~0.4.6",
-            "npm:next@^16.1.7",
+            "npm:next@^16.1.6",
             "npm:postcss@^8.5.6",
             "npm:prettier@^3.7.4",
             "npm:react-chrono@^2.9.1",
             "npm:react-dom@^19.2.0",
-            "npm:react-is@^19.2.4",
             "npm:react-markdown@^10.1.0",
             "npm:react18-json-view@~0.2.9",
             "npm:react@^19.2.0",
@@ -145,26 +133,6 @@
             "npm:typescript@~5.6.2"
           ]
         }
-      },
-      "hindsight-tools/hindsight-agent-sdk": {
-        "packageJson": {
-          "dependencies": [
-            "npm:@vectorize-io/hindsight-client@~0.5.6",
-            "npm:typescript@^5.4.0",
-            "npm:vitest@^4.1.2"
-          ]
-        }
-      },
-      "hindsight-tools/self-driving-agents": {
-        "packageJson": {
-          "dependencies": [
-            "npm:@clack/prompts@^1.2.0",
-            "npm:@vectorize-io/hindsight-client@~0.5.6",
-            "npm:picocolors@^1.1.0",
-            "npm:typescript@^5.4.0",
-            "npm:vitest@^4.1.2"
-          ]
-        }
       }
     }
   }

docker/docker-compose/alloydb/docker-compose.yaml

@@ -1,90 +0,0 @@
-name: hindsight
-# Docker Compose file for Hindsight with AlloyDB Omni and ScaNN
-# Uses Google's free AlloyDB Omni container image: https://hub.docker.com/r/google/alloydbomni
-#
-# Usage:
-# docker compose -f docker/docker-compose/alloydb/docker-compose.yaml up -d
-#
-# Make sure to set the required environment variables before running:
-# - HINDSIGHT_DB_PASSWORD: password for the AlloyDB Omni/PostgreSQL user
-# - Configure LLM provider variables as needed (see the hindsight service below)
-#
-# Optional environment variables with defaults:
-# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
-# - HINDSIGHT_DB_VERSION: AlloyDB Omni image tag (default: 17)
-# - HINDSIGHT_DB_USER: database user (default: hindsight_user)
-# - HINDSIGHT_DB_NAME: database name (default: hindsight_db)
-
-services:
-  db:
-    image: google/alloydbomni:${HINDSIGHT_DB_VERSION:-17}
-    container_name: hindsight-db-alloydb
-    restart: always
-    ports:
-      - "5438:5432"
-    environment:
-      POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
-      POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-      POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
-    volumes:
-      - alloydb_data:/var/lib/postgresql/data
-    networks:
-      - hindsight-net
-
-  alloydb-init:
-    image: google/alloydbomni:${HINDSIGHT_DB_VERSION:-17}
-    depends_on:
-      - db
-    environment:
-      - PGPASSWORD=${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-    command:
-      - bash
-      - -c
-      - |
-        echo 'Waiting for AlloyDB Omni to be ready...'
-        until pg_isready -h hindsight-db-alloydb -p 5432 -U ${HINDSIGHT_DB_USER:-hindsight_user}; do
-          echo 'AlloyDB Omni is unavailable - sleeping'
-          sleep 2
-        done
-        echo 'AlloyDB Omni is ready - creating ${HINDSIGHT_DB_NAME:-hindsight_db} database'
-        psql -h hindsight-db-alloydb -p 5432 -U ${HINDSIGHT_DB_USER:-hindsight_user} -c 'CREATE DATABASE ${HINDSIGHT_DB_NAME:-hindsight_db};' 2>/dev/null || echo 'Database already exists'
-        echo 'Creating vector and alloydb_scann extensions'
-        psql -h hindsight-db-alloydb -p 5432 -U ${HINDSIGHT_DB_USER:-hindsight_user} -d ${HINDSIGHT_DB_NAME:-hindsight_db} -c 'CREATE EXTENSION IF NOT EXISTS vector;'
-        psql -h hindsight-db-alloydb -p 5432 -U ${HINDSIGHT_DB_USER:-hindsight_user} -d ${HINDSIGHT_DB_NAME:-hindsight_db} -c 'CREATE EXTENSION IF NOT EXISTS alloydb_scann CASCADE;'
-        echo 'Database and extensions created successfully'
-    restart: "no"
-    networks:
-      - hindsight-net
-
-  hindsight:
-    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
-    container_name: hindsight-app
-    ports:
-      - "8888:8888"
-      - "9999:9999"
-    environment:
-      # LLM Configuration
-      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
-      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
-
-      # Database Configuration
-      HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
-
-      # Vector and Text Search Extensions
-      HINDSIGHT_API_VECTOR_EXTENSION: scann
-      HINDSIGHT_API_TEXT_SEARCH_EXTENSION: native
-
-    depends_on:
-      db:
-        condition: service_started
-      alloydb-init:
-        condition: service_completed_successfully
-    networks:
-      - hindsight-net
-
-networks:
-  hindsight-net:
-    driver: bridge
-
-volumes:
-  alloydb_data:

docker/docker-compose/claude-code/README.md

@@ -1,113 +0,0 @@
-# Hindsight with Claude Code (Claude Pro/Max subscription)
-
-Run Hindsight inside Docker using the `claude-code` LLM provider, backed by
-your host machine's Claude Pro or Max subscription credentials.
-
-The standalone Hindsight Docker image ships `claude-agent-sdk` but does **not**
-bundle the host `claude` CLI binary or any Claude credentials. This Compose
-file bind-mounts the host's CLI install and credentials into the container so
-the `claude-code` provider works without an API key.
-
-## When to use this
-
-- You have an active Claude Pro or Max subscription and want to use it for
-  Hindsight without paying separate Anthropic API costs.
-- You want a one-command `docker compose up` instead of a long `docker run`
-  invocation with many flags.
-- You are running on **Linux/amd64** — macOS Docker Desktop and Windows host
-  paths differ and are not yet covered (please open an issue if you'd like to
-  contribute a verified recipe for either).
-
-> **Personal-use only.**  Anthropic's
-> [Agent SDK documentation](https://docs.claude.com/en/api/agent-sdk/overview)
-> states that third-party developers should not offer claude.ai login or rate
-> limits for their products. Hindsight does **not** perform any login on your
-> behalf — it uses credentials you've already authenticated via
-> `claude auth login`. In January 2026, Anthropic
-> [enforced restrictions](https://paddo.dev/blog/anthropic-walled-garden-crackdown/)
-> against tools that spoofed the Claude Code client identity; Hindsight uses
-> the official Claude Agent SDK instead.
->
-> Do not deploy this configuration to shared environments or production. For
-> that, use the `anthropic` provider with an API key from the
-> [Anthropic Console](https://console.anthropic.com/). Usage counts against
-> your Claude Pro/Max subscription limits.
-
-## Prerequisites
-
-- Host has `claude` CLI installed (e.g., `npm install -g @anthropics/claude-code`)
-  and `claude auth login` has been run successfully.
-- `~/.claude.json` and `~/.claude/.credentials.json` exist on the host.
-- Host `claude` CLI version is **2.1.128 or newer** — the version bundled with
-  `claude-agent-sdk` 0.5.x has a protocol incompatibility in containers, so
-  the recipe overrides it with the host binary.
-
-## Quick start
-
-```bash
-# Set your host UID/GID (defaults to 1000:1000 if unset)
-export HOST_UID=$(id -u)
-export HOST_GID=$(id -g)
-
-docker compose -f docker/docker-compose/claude-code/docker-compose.yaml up -d
-```
-
-- API: http://localhost:8888
-- Control Plane: http://localhost:9999
-
-## Post-setup (one-time)
-
-After the container starts for the first time, run these commands to fix
-permissions and symlink the host `claude` binary into `$PATH`:
-
-```bash
-# Make ~/.claude writable by your UID (the CLI writes session/project state)
-docker exec --user 0:0 hindsight-claude-code chown $(id -u):$(id -g) /home/hindsight/.claude
-docker exec --user 0:0 hindsight-claude-code chmod 755 /home/hindsight/.claude
-
-# Symlink the host claude binary into PATH
-docker exec --user 0:0 hindsight-claude-code \
-  ln -sf /home/hindsight/.local/share/claude/versions/2.1.128 /usr/local/bin/claude
-```
-
-If you set `CLAUDE_CLI_VERSION` to a version other than `2.1.128`, update the
-symlink path accordingly.
-
-## Notes on the bind-mount surface (every flag is load-bearing)
-
-- **Host `claude` binary required** — the image ships only `claude-agent-sdk`,
-  not the CLI itself.
-- **SDK bundled-binary override** — the override of
-  `claude_agent_sdk/_bundled/claude` works around a protocol issue in the
-  bundled v2.1.121 binary inside containers. Once `claude-agent-sdk` ships
-  with v2.1.128+ this override can be dropped. Set `CLAUDE_CLI_VERSION` to
-  match your installed version.
-- **Single-file credential mounts** — credentials are mounted as individual
-  `:ro` files rather than a whole-directory `:ro` mount of `~/.claude`,
-  because the CLI writes session/project state at runtime and a read-only
-  directory mount silently breaks it.
-- **`--user` / `user:`** — the `user: ${HOST_UID}:${HOST_GID}` pattern
-  requires `chmod 755 /home/hindsight`, which is built into the image since
-  v0.6.0 (see [#1481](https://github.com/vectorize-io/hindsight/issues/1481)).
-- **`~/.hindsight-docker` data directory** — the pg0 data bind mount must be
-  writable by your host UID (see
-  [#1483](https://github.com/vectorize-io/hindsight/issues/1483)).
-- **Verified** on `linux/amd64` against `ghcr.io/vectorize-io/hindsight:latest`
-  v0.5.6+.
-
-## Using a different Claude CLI version
-
-If your host has a `claude` version other than 2.1.128, set
-`CLAUDE_CLI_VERSION` before starting:
-
-```bash
-export CLAUDE_CLI_VERSION=2.2.0
-docker compose -f docker/docker-compose/claude-code/docker-compose.yaml up -d
-```
-
-Then update the post-setup symlink to match:
-
-```bash
-docker exec --user 0:0 hindsight-claude-code \
-  ln -sf /home/hindsight/.local/share/claude/versions/2.2.0 /usr/local/bin/claude
-```

docker/docker-compose/claude-code/docker-compose.yaml

@@ -1,44 +0,0 @@
-name: hindsight-claude-code
-# Run Hindsight with the claude-code LLM provider, using your host machine's
-# Claude Pro/Max subscription credentials.  Linux/amd64 only for now.
-#
-# Quick start:
-#   docker compose -f docker/docker-compose/claude-code/docker-compose.yaml up -d
-#
-# See README.md for prerequisites, post-setup steps, and important caveats.
-
-services:
-  hindsight:
-    image: ghcr.io/vectorize-io/hindsight:latest
-    container_name: hindsight-claude-code
-    user: "${HOST_UID:-1000}:${HOST_GID:-1000}"
-    ports:
-      - "127.0.0.1:8888:8888"
-      - "127.0.0.1:9999:9999"
-    environment:
-      HOME: /home/hindsight
-      USER: hindsight
-      LOGNAME: hindsight
-      PATH: /usr/local/bin:/usr/bin:/bin:/app/api/.venv/bin
-      HINDSIGHT_API_LLM_PROVIDER: claude-code
-    volumes:
-      # ── Persistent data ────────────────────────────────────────────
-      # Writable pg0 data directory.  Must be writable by HOST_UID.
-      - ${HOME:-.}/.hindsight-docker:/home/hindsight/.pg0
-
-      # ── Claude credentials (read-only, single-file mounts) ────────
-      # A whole-directory :ro mount of ~/.claude silently breaks the
-      # CLI, which writes session/project state at runtime — so we
-      # mount only the two credential files.
-      - ${HOME}/.claude/.credentials.json:/home/hindsight/.claude/.credentials.json:ro
-      - ${HOME}/.claude.json:/home/hindsight/.claude.json:ro
-
-      # ── Claude CLI install (read-only) ─────────────────────────────
-      - ${HOME}/.local/share/claude:/home/hindsight/.local/share/claude:ro
-
-      # ── SDK bundled-binary override ────────────────────────────────
-      # The claude-agent-sdk 0.5.x image bundles v2.1.121 which has a
-      # protocol incompatibility in containers.  Override it with the
-      # host's v2.1.128+ binary.  Drop this mount once claude-agent-sdk
-      # ships with v2.1.128+.
-      - ${HOME}/.local/share/claude/versions/${CLAUDE_CLI_VERSION:-2.1.128}:/app/api/.venv/lib/python3.11/site-packages/claude_agent_sdk/_bundled/claude:ro

docker/docker-compose/custom-models/Dockerfile

@@ -1,34 +0,0 @@
-# Example: custom Hindsight image with non-default local models baked in.
-#
-# Use this pattern in production when you run a non-default embedder or
-# reranker. Baking models into the image removes the runtime dependency on
-# HuggingFace and lets the container registry handle caching per node, so
-# you don't need a model-cache PVC.
-#
-# Built on top of the slim image so only the deps and models you actually
-# use end up in the final image.
-FROM ghcr.io/vectorize-io/hindsight:latest-slim
-
-# Install the local-ml deps required to load sentence-transformers /
-# cross-encoder models at runtime. Pinned ranges mirror hindsight-api-slim's
-# `local-ml` extra in hindsight-api-slim/pyproject.toml. Use `uv pip
-# install` against the image's venv explicitly: the slim image's venv was
-# created by `uv sync` and does not ship its own `pip`, so a bare
-# `pip install` would fall back to user site-packages and not be visible
-# to the runtime python.
-RUN uv pip install --python /app/api/.venv/bin/python --no-cache \
-    'sentence-transformers>=3.3.0' \
-    'transformers>=4.53.0' \
-    'torch>=2.6.0'
-
-# Pre-download the models you want to use. Replace these with your own.
-# The defaults bundled in the full image are BAAI/bge-small-en-v1.5 and
-# cross-encoder/ms-marco-MiniLM-L-6-v2; here we pick multilingual variants
-# as a concrete non-default example.
-ARG EMBEDDER=sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
-ARG RERANKER=cross-encoder/mmarco-mMiniLMv2-L12-H384-v1
-ENV HF_HUB_DOWNLOAD_TIMEOUT=600
-RUN python -c "\
-from sentence_transformers import SentenceTransformer, CrossEncoder; \
-SentenceTransformer('${EMBEDDER}'); \
-CrossEncoder('${RERANKER}')"

docker/docker-compose/custom-models/README.md

@@ -1,81 +0,0 @@
-# Hindsight with Custom Local Models
-
-Example Docker Compose setup that builds a Hindsight image with **non-default
-local embedder and reranker models baked in at build time**.
-
-This is the recommended pattern for production when you use a non-default
-local model: the container registry caches model layers per node, pod
-startup is deterministic, and you don't need a model-cache PVC (or any
-runtime dependency on HuggingFace).
-
-## When to use this
-
-- You override `HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL` or
-  `HINDSIGHT_API_RERANKER_LOCAL_MODEL` to a non-default model.
-- You want pod startup to be deterministic and offline-capable.
-- You'd otherwise reach for a Helm `modelCache` PVC just to avoid
-  re-downloading models.
-
-If you're using the **default** local models, the published full image
-(`ghcr.io/vectorize-io/hindsight:latest`) already bakes them in — you don't
-need this example.
-
-If you're using **external** providers (TEI, OpenAI, Cohere, ...) for
-embeddings and reranking, use the slim image directly — no models are
-needed in the image.
-
-## Quick start
-
-```bash
-export OPENAI_API_KEY=sk-xxx
-
-docker compose -f docker/docker-compose/custom-models/docker-compose.yaml up --build
-```
-
-- API: http://localhost:8888
-- Control Plane: http://localhost:9999
-
-## Using your own models
-
-Override the build args to bake different models:
-
-```bash
-docker compose -f docker/docker-compose/custom-models/docker-compose.yaml build \
-  --build-arg EMBEDDER=your-org/your-embedder \
-  --build-arg RERANKER=your-org/your-reranker
-```
-
-Then update the matching `HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL` and
-`HINDSIGHT_API_RERANKER_LOCAL_MODEL` values in `docker-compose.yaml` so the
-runtime points at the same model IDs.
-
-## Verifying the models are baked in
-
-`docker-compose.yaml` sets `HF_HUB_OFFLINE=1` and `TRANSFORMERS_OFFLINE=1`
-so that any attempt to download a model at runtime fails loudly instead of
-silently re-downloading. If the container starts and serves recall queries
-with these set, the models are correctly baked in.
-
-You can also inspect the image directly:
-
-```bash
-docker run --rm --entrypoint sh hindsight-custom-models-hindsight \
-  -c 'ls ~/.cache/huggingface/hub/'
-```
-
-## Why not a model-cache PVC?
-
-The Helm chart exposes an optional `api.persistence.modelCache` PVC for
-caching downloaded models across pod restarts. Compared to baking models
-into the image:
-
-- A PVC adds storage cost — one PVC per worker replica with
-  `volumeClaimTemplates`.
-- `ReadWriteOnce` (the default) pins pods to a node.
-- The PVC needs lifecycle management on `helm uninstall` / `helm upgrade`
-  — without `helm.sh/resource-policy: keep` it is deleted on uninstall;
-  with it, storage keeps billing forever until manually cleaned up.
-- Pod startup still depends on HuggingFace being reachable on first run.
-
-Image layers, by contrast, are pulled once per node and cached for free by
-the container runtime, with no orphaned-storage cleanup story.

docker/docker-compose/custom-models/docker-compose.yaml

@@ -1,44 +0,0 @@
-name: hindsight-custom-models
-# Example: run a custom Hindsight image with non-default local models baked
-# in at build time, so pod startup does not depend on HuggingFace at runtime.
-#
-# Quick start:
-#   export OPENAI_API_KEY=sk-xxx
-#   docker compose -f docker/docker-compose/custom-models/docker-compose.yaml up --build
-#
-# Required environment variables:
-# - OPENAI_API_KEY (or configure another LLM provider via HINDSIGHT_API_LLM_*)
-
-services:
-  hindsight:
-    build:
-      context: .
-      dockerfile: Dockerfile
-      # Override at build time to bake different models:
-      #   docker compose build --build-arg EMBEDDER=your-org/your-embedder
-      args:
-        EMBEDDER: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
-        RERANKER: cross-encoder/mmarco-mMiniLMv2-L12-H384-v1
-    container_name: hindsight-custom-models
-    ports:
-      - "8888:8888"
-      - "9999:9999"
-    environment:
-      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
-      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
-
-      # Point Hindsight at the models baked into the image above.
-      HINDSIGHT_API_EMBEDDINGS_PROVIDER: local
-      HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
-      HINDSIGHT_API_RERANKER_PROVIDER: local
-      HINDSIGHT_API_RERANKER_LOCAL_MODEL: cross-encoder/mmarco-mMiniLMv2-L12-H384-v1
-
-      # Fail fast if a model is missing from the image instead of silently
-      # falling back to a HuggingFace download at runtime.
-      HF_HUB_OFFLINE: "1"
-      TRANSFORMERS_OFFLINE: "1"
-    volumes:
-      - pg_data:/home/hindsight/.pg0
-
-volumes:
-  pg_data:

docker/docker-compose/local-llm/README.md

@@ -1,103 +0,0 @@
-# Hindsight with a local llama.cpp server sidecar
-
-Example Docker Compose setup that runs Hindsight against a **local
-llama.cpp server**, fully offline, with no external API key required.
-
-## Architecture
-
-```
-┌────────────┐  HTTP /v1/chat/completions  ┌──────────────────────────────┐
-│ hindsight  │ ──────────────────────────▶ │ llama.cpp server (sidecar)   │
-│ (API + CP) │                             │ ghcr.io/ggml-org/llama.cpp   │
-└────────────┘                             └──────────────────────────────┘
-```
-
-`llama.cpp` runs as its own container and exposes an OpenAI-compatible
-HTTP API. Hindsight talks to it via the standard `openai` LLM provider
-with `HINDSIGHT_API_LLM_BASE_URL` pointed at the sidecar.
-
-This pattern follows
-[*Hosting llama-server with Docker* (ServiceStack)](https://servicestack.net/posts/hosting-llama-server).
-
-### Why a sidecar and not the in-process `llamacpp` provider?
-
-Hindsight does ship an in-process `llamacpp` provider that spawns
-`llama-cpp-python`, but the **published `ghcr.io/vectorize-io/hindsight`
-image deliberately omits `llama-cpp-python`** to keep the image small and
-avoid bundling native inference libraries that most users don't need.
-Trying to set `HINDSIGHT_API_LLM_PROVIDER=llamacpp` against the published
-image fails with `ModuleNotFoundError: No module named 'llama_cpp'`.
-
-The sidecar approach side-steps that entirely: the official llama.cpp
-image is used as-is for inference, Hindsight is used as-is for memory.
-Clean separation, no derived images.
-
-## Quick start
-
-```bash
-docker compose -f docker/docker-compose/local-llm/docker-compose.yaml up
-```
-
-- API: http://localhost:8888
-- Control Plane: http://localhost:9999
-
-**First boot downloads ~3.5 GB** (Gemma 4 E2B Q4_K_M GGUF) into the
-`llama_models` named volume. Subsequent boots reuse it.
-
-Hindsight only starts after llama.cpp's `/health` endpoint reports
-healthy, so the API will appear "stuck" for a few minutes on the first
-run while the model downloads.
-
-## Using a different model
-
-Override the HuggingFace repo / file in `docker-compose.yaml`:
-
-```yaml
-environment:
-  LLAMA_ARG_HF_REPO: bartowski/Qwen2.5-7B-Instruct-GGUF
-  LLAMA_ARG_HF_FILE: Qwen2.5-7B-Instruct-Q4_K_M.gguf
-```
-
-Also update `HINDSIGHT_API_LLM_MODEL` on the `hindsight` service to a
-matching alias (the value is sent to llama-server as the OpenAI `model`
-field — llama-server is lenient about this but it shows up in logs).
-
-## GPU acceleration
-
-The default compose file targets CPU because not everyone has a GPU. On
-CPU, Gemma 4 E2B runs at ~2-3 tokens/sec — fine for a smoke test, but the
-retain pipeline (which makes several multi-hundred-token LLM calls per
-memory) will time out against Hindsight's default LLM timeout. **For any
-real use, run on a GPU.**
-
-### NVIDIA
-
-1. Switch the `llama` service image from `:server` to `:server-cuda`.
-2. Uncomment the `LLAMA_ARG_N_GPU_LAYERS: "999"` env var (offload all
-   layers to GPU).
-3. Uncomment the `deploy.resources.reservations.devices` block.
-4. Install the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
-   on the host.
-
-The compose file has all four spots marked with inline comments.
-
-### Apple Silicon / ROCm / Vulkan
-
-The official `ghcr.io/ggml-org/llama.cpp` image only ships CPU and CUDA
-variants. For Metal (Apple Silicon), ROCm (AMD), or Vulkan backends,
-build llama.cpp yourself with the appropriate flags and reference the
-image you build instead. Docker Desktop on macOS cannot pass through the
-host GPU to a Linux container in any case — for Apple Silicon, run
-llama-server directly on the host and only put Hindsight in Docker.
-
-## Caveats
-
-- llama.cpp's HTTP API is OpenAI-compatible but not 100% feature-parity.
-  Function/tool calling support depends on the chat template baked into
-  the GGUF; some retain/reflect flows may behave differently than against
-  a hosted OpenAI model.
-- Small GGUFs (~3 B params) are useful for smoke testing but will
-  underperform a hosted frontier model on retain quality. Use a larger
-  GGUF (7-13 B params) for production-quality memory.
-- The `llama_models` named volume persists the GGUF across `docker
-  compose down`/`up` so the model is downloaded once, not every restart.

docker/docker-compose/local-llm/docker-compose.yaml

@@ -1,74 +0,0 @@
-name: hindsight-local-llm
-# Example: run Hindsight against a local llama.cpp server sidecar — fully
-# offline, no external API key needed.
-#
-# Pattern follows https://servicestack.net/posts/hosting-llama-server :
-# llama.cpp runs as its own container exposing an OpenAI-compatible HTTP
-# API, and Hindsight talks to it via the `openai` LLM provider with a
-# custom `base_url`. This means we can use the published Hindsight image
-# unchanged — no derived Dockerfile, no `llama-cpp-python` install on top.
-#
-# Quick start:
-#   docker compose -f docker/docker-compose/local-llm/docker-compose.yaml up
-#
-# First boot downloads the default Gemma 4 E2B GGUF (~3.5 GB) into the
-# `llama_models` volume; subsequent boots reuse it.
-
-services:
-  llama:
-    image: ghcr.io/ggml-org/llama.cpp:server
-    container_name: hindsight-local-llm-llama
-    environment:
-      LLAMA_ARG_HOST: 0.0.0.0
-      LLAMA_ARG_PORT: "8080"
-      # Auto-download a small GGUF from HuggingFace on first start.
-      # Override these to use a different model.
-      LLAMA_ARG_HF_REPO: bartowski/google_gemma-4-E2B-it-GGUF
-      LLAMA_ARG_HF_FILE: google_gemma-4-E2B-it-Q4_K_M.gguf
-      LLAMA_ARG_CTX_SIZE: "8192"
-      # Uncomment for NVIDIA GPU (and switch image to :server-cuda):
-      # LLAMA_ARG_N_GPU_LAYERS: "999"
-    volumes:
-      # llama-server stores HuggingFace downloads under ~/.cache/huggingface
-      # (not ~/.cache/llama.cpp), so mount the named volume there to avoid
-      # re-downloading the GGUF on every recreate.
-      - llama_models:/root/.cache/huggingface
-    healthcheck:
-      test: ["CMD-SHELL", "curl -fsS http://localhost:8080/health || exit 1"]
-      interval: 10s
-      timeout: 5s
-      retries: 60
-      start_period: 30s
-    # For NVIDIA GPU acceleration, swap the image above to
-    # `ghcr.io/ggml-org/llama.cpp:server-cuda` and uncomment:
-    # deploy:
-    #   resources:
-    #     reservations:
-    #       devices:
-    #         - driver: nvidia
-    #           count: all
-    #           capabilities: [gpu]
-
-  hindsight:
-    image: ghcr.io/vectorize-io/hindsight:latest
-    container_name: hindsight-local-llm
-    depends_on:
-      llama:
-        condition: service_healthy
-    ports:
-      - "8888:8888"
-      - "9999:9999"
-    environment:
-      # llama-server is OpenAI-compatible, so use the `openai` provider and
-      # point base_url at the sidecar. The API key is unused by llama-server
-      # but Hindsight requires the env var to be set.
-      HINDSIGHT_API_LLM_PROVIDER: openai
-      HINDSIGHT_API_LLM_BASE_URL: http://llama:8080/v1
-      HINDSIGHT_API_LLM_API_KEY: not-needed
-      HINDSIGHT_API_LLM_MODEL: gemma-4-e2b-it
-    volumes:
-      - pg_data:/home/hindsight/.pg0
-
-volumes:
-  pg_data:
-  llama_models:

docker/docker-compose/nginx/docker-compose.yml

@@ -51,8 +51,6 @@ services:
 
       # Control Plane config
       HINDSIGHT_CP_DATAPLANE_API_URL: http://localhost:8888
-      # Optional: Require a shared access key for Control Plane UI access
-      # HINDSIGHT_CP_ACCESS_KEY: your-secret-key
     volumes:
       # Persist embedded pg0 database
       - hindsight_data:/app/data

docker/docker-compose/pg_search/Dockerfile

@@ -1,7 +0,0 @@
-# PostgreSQL with pgvector and ParadeDB pg_search extensions.
-#
-# The official ParadeDB image ships PostgreSQL with pg_search and pgvector
-# already installed, so no build steps are required. We pin to the PG17
-# variant for parity with the other Hindsight docker-compose examples
-# (vchord, pg_textsearch).
-FROM paradedb/paradedb:latest-pg17

docker/docker-compose/pg_search/docker-compose.yaml

@@ -1,96 +0,0 @@
-name: hindsight
-# Docker Compose file for Hindsight with PostgreSQL and ParadeDB pg_search.
-#
-# pg_search is the only BM25 backend supported by Hindsight that works with
-# Citus, so this is the recommended setup for horizontally scaled deployments.
-#
-# Usage:
-#   docker compose -f docker/docker-compose/pg_search/docker-compose.yaml up -d
-#
-# Required environment variables:
-# - HINDSIGHT_DB_PASSWORD: Password for the PostgreSQL user
-# - Configure LLM provider variables as needed (see the hindsight service)
-#
-# Optional environment variables with defaults:
-# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
-# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
-# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
-# - HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER: ParadeDB pg_search
-#   tokenizer for new BM25 indexes (default: empty, uses ParadeDB default)
-
-services:
-  db:
-    # Use ParadeDB image which bundles pgvector + pg_search
-    build:
-      context: .
-      dockerfile: Dockerfile
-    container_name: hindsight-db
-    restart: always
-    ports:
-      - "5437:5432"
-    environment:
-      POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
-      POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-      POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
-    volumes:
-      - pg_data:/var/lib/postgresql/data
-    networks:
-      - hindsight-net
-
-  pg-search-init:
-    build:
-      context: .
-      dockerfile: Dockerfile
-    depends_on:
-      - db
-    environment:
-      - PGPASSWORD=${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-    command: >
-      bash -c "
-        echo 'Waiting for PostgreSQL to be ready...';
-        until pg_isready -h hindsight-db -p 5432 -U hindsight_user; do
-          echo 'PostgreSQL is unavailable - sleeping';
-          sleep 2;
-        done;
-        echo 'PostgreSQL is ready - creating hindsight_db database';
-        psql -h hindsight-db -p 5432 -U hindsight_user -c 'CREATE DATABASE hindsight_db;' 2>/dev/null || echo 'Database already exists';
-        echo 'Creating extensions in hindsight_db database';
-        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vector CASCADE;';
-        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS pg_search CASCADE;';
-        echo 'Database and extensions created successfully';
-      "
-    restart: "no"
-    networks:
-      - hindsight-net
-
-  hindsight:
-    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
-    container_name: hindsight-app
-    ports:
-      - "8888:8888"
-      - "9999:9999"
-    environment:
-      # LLM Configuration
-      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
-      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
-
-      # Database Configuration
-      HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
-
-      # Vector and Text Search Extensions
-      HINDSIGHT_API_VECTOR_EXTENSION: pgvector
-      HINDSIGHT_API_TEXT_SEARCH_EXTENSION: pg_search
-      HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER: ${HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER:-}
-
-    depends_on:
-      - db
-    networks:
-      - hindsight-net
-
-
-networks:
-  hindsight-net:
-    driver: bridge
-
-volumes:
-  pg_data:

docker/docker-compose/pgroonga/Dockerfile

@@ -1,23 +0,0 @@
-# PostgreSQL with pgvector and pgroonga extensions.
-#
-# pgroonga is a multilingual full-text search extension built on Groonga.
-# It works out of the box for CJK (Chinese, Japanese, Korean) and other
-# non-whitespace-segmented languages via the TokenBigram tokenizer.
-FROM groonga/pgroonga:latest-debian-pg17
-
-# Install pgvector on top of the pgroonga base image (which already provides
-# pgroonga and the Groonga library).
-RUN apt-get update && apt-get install -y --no-install-recommends \
-    build-essential \
-    git \
-    postgresql-server-dev-17 \
-    && rm -rf /var/lib/apt/lists/*
-
-RUN cd /tmp && \
-    git clone --branch v0.8.0 https://github.com/pgvector/pgvector.git && \
-    cd pgvector && \
-    make && \
-    make install
-
-RUN rm -rf /tmp/pgvector && \
-    apt-get purge -y --auto-remove build-essential git postgresql-server-dev-17

docker/docker-compose/pgroonga/docker-compose.yaml

@@ -1,91 +0,0 @@
-name: hindsight
-# Docker Compose file for Hindsight with PostgreSQL and pgroonga
-#
-# pgroonga provides multilingual BM25 indexing that works out of the box for
-# CJK (Chinese, Japanese, Korean) and other non-whitespace-segmented languages.
-# Use this recipe if your bank content is not English/European.
-#
-# docker compose -f docker/docker-compose/pgroonga/docker-compose.yaml down && \
-#   sleep 2 && \
-#   docker compose -f docker/docker-compose/pgroonga/docker-compose.yaml up -d
-#
-# Optional environment variables with defaults:
-# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
-# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
-# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
-# - HINDSIGHT_DB_PASSWORD: PostgreSQL password (default: hindsight_password)
-
-services:
-  db:
-    build:
-      context: .
-      dockerfile: Dockerfile
-    container_name: hindsight-db
-    restart: always
-    ports:
-      - "5439:5432"
-    environment:
-      POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
-      POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-      POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
-    volumes:
-      - pg_data:/var/lib/postgresql/data
-    networks:
-      - hindsight-net
-
-  pgroonga-init:
-    build:
-      context: .
-      dockerfile: Dockerfile
-    depends_on:
-      - db
-    environment:
-      - PGPASSWORD=${HINDSIGHT_DB_PASSWORD:-hindsight_password}
-    command: >
-      bash -c "
-        echo 'Waiting for PostgreSQL to be ready...';
-        until pg_isready -h hindsight-db -p 5432 -U hindsight_user; do
-          echo 'PostgreSQL is unavailable - sleeping';
-          sleep 2;
-        done;
-        echo 'PostgreSQL is ready - creating hindsight_db database';
-        psql -h hindsight-db -p 5432 -U hindsight_user -c 'CREATE DATABASE hindsight_db;' 2>/dev/null || echo 'Database already exists';
-        echo 'Creating extensions in hindsight_db database';
-        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vector CASCADE;';
-        psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS pgroonga CASCADE;';
-        echo 'Database and extensions created successfully';
-      "
-    restart: "no"
-    networks:
-      - hindsight-net
-
-  hindsight:
-    image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
-    container_name: hindsight-app
-    ports:
-      - "8888:8888"
-      - "9999:9999"
-    environment:
-      # LLM Configuration
-      HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
-      HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
-
-      # Database Configuration
-      HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
-
-      # Vector and Text Search Extensions
-      HINDSIGHT_API_VECTOR_EXTENSION: pgvector
-      HINDSIGHT_API_TEXT_SEARCH_EXTENSION: pgroonga
-
-    depends_on:
-      - db
-    networks:
-      - hindsight-net
-
-
-networks:
-  hindsight-net:
-    driver: bridge
-
-volumes:
-  pg_data:

docker/standalone/Dockerfile

@@ -50,8 +50,6 @@ WORKDIR /app/api
 # Sync dependencies using appropriate extras based on INCLUDE_LOCAL_MODELS
 # local-ml: torch, sentence-transformers, transformers, einops, flashrank, mlx (optional)
 # embedded-db: pg0-embedded (always included for embedded PostgreSQL support)
-# ONNX Runtime embeddings are intentionally not bundled into the official
-# standalone image; install the local-onnx extra in custom images when needed.
 RUN if [ "$INCLUDE_LOCAL_MODELS" = "true" ]; then \
         uv sync --extra local-ml --extra embedded-db; \
     else \
@@ -174,10 +172,6 @@ USER hindsight
 # "Permission denied" error when mounting a fresh root-owned volume.
 RUN mkdir -p /home/hindsight/.pg0
 
-# Make /home/hindsight traversable when running with --user UID:GID overrides
-# (default 0700 blocks traversal by non-owner UIDs needed for bind-mount ownership matching)
-RUN chmod 755 /home/hindsight
-
 ENV PATH="/app/api/.venv/bin:${PATH}"
 
 # Pre-download tiktoken encoding (ALWAYS - required for token counting even in air-gapped envs)
@@ -334,10 +328,6 @@ USER hindsight
 # "Permission denied" error when mounting a fresh root-owned volume.
 RUN mkdir -p /home/hindsight/.pg0
 
-# Make /home/hindsight traversable when running with --user UID:GID overrides
-# (default 0700 blocks traversal by non-owner UIDs needed for bind-mount ownership matching)
-RUN chmod 755 /home/hindsight
-
 ENV PATH="/app/api/.venv/bin:${PATH}"
 
 # Pre-download tiktoken encoding (ALWAYS - required for token counting even in air-gapped envs)

docker/standalone/start-all.sh

@@ -10,90 +10,19 @@ set -e
 # loss scenarios where a container restart caused the data directory to be
 # wiped despite a volume mount being present.
 # =============================================================================
-pg0_has_pg_version() {
-    local pg0_data_dir="$1"
-
-    # pg0 has used more than one on-disk layout. Newer standalone images keep
-    # PostgreSQL data under instances/<name>/data, while older volumes may have
-    # placed PG_VERSION at or one level below the mount.
-    [ -f "$pg0_data_dir/PG_VERSION" ] && return 0
-    compgen -G "$pg0_data_dir"/*/PG_VERSION > /dev/null 2>&1 && return 0
-    compgen -G "$pg0_data_dir"/instances/*/data/PG_VERSION > /dev/null 2>&1 && return 0
-
-    return 1
-}
-
-check_pg0_data_integrity() {
-    local pg0_data_dir="$1"
-
-    if [ ! -d "$pg0_data_dir" ]; then
-        return 0
-    fi
-
+PG0_DATA_DIR="${HOME}/.pg0"
+if [ -d "$PG0_DATA_DIR" ]; then
     # Look for actual PostgreSQL data directories (pg0 creates subdirs per instance)
-    if pg0_has_pg_version "$pg0_data_dir"; then
-        echo "✅ Existing pg0 data directory detected at $pg0_data_dir"
-    elif [ "$(ls -A "$pg0_data_dir" 2>/dev/null)" ]; then
-        echo "⚠️  WARNING: pg0 data directory exists at $pg0_data_dir but no PG_VERSION found."
+    if compgen -G "$PG0_DATA_DIR"/*/PG_VERSION > /dev/null 2>&1; then
+        echo "✅ Existing pg0 data directory detected at $PG0_DATA_DIR"
+    elif [ "$(ls -A "$PG0_DATA_DIR" 2>/dev/null)" ]; then
+        echo "⚠️  WARNING: pg0 data directory exists at $PG0_DATA_DIR but no PG_VERSION found."
         echo "   This may indicate data corruption or an incomplete previous shutdown."
         echo "   If you see all migrations running from scratch after this, your data may have been lost."
         echo "   See: https://github.com/vectorize-io/hindsight/issues/675"
     fi
-
-    return 0
-}
-
-# =============================================================================
-# Embedded pg0 writability pre-check (#1483)
-#
-# The container runs as the unprivileged `hindsight` user (UID 1000). When the
-# pg0 data directory is a host bind mount (e.g. `-v $HOME/dir:/home/hindsight/.pg0`)
-# that is not owned by UID 1000 — the default on macOS Docker Desktop and most
-# non-1000 Linux hosts — pg0 fails with the opaque "Permission denied (os error
-# 13)". We cannot chown it ourselves without root (and the image is deliberately
-# rootless), so we surface an actionable message up front instead.
-#
-# Docker *named* volumes are seeded with the image directory's ownership (UID
-# 1000) on first use, so they avoid this entirely — hence the named-volume
-# recommendation below and in the README.
-# =============================================================================
-check_pg0_writable() {
-    local pg0_data_dir="$1"
-
-    # Only relevant for embedded pg0; an external database doesn't use this dir.
-    if [ -n "${HINDSIGHT_API_DATABASE_URL:-}" ]; then
-        return 0
-    fi
-
-    mkdir -p "$pg0_data_dir" 2>/dev/null || true
-    if touch "$pg0_data_dir/.hindsight-write-test" 2>/dev/null; then
-        rm -f "$pg0_data_dir/.hindsight-write-test" 2>/dev/null || true
-        return 0
-    fi
-
-    echo "❌ The embedded database directory $pg0_data_dir is not writable by this container (UID $(id -u))."
-    echo ""
-    echo "   A host directory was bind-mounted but is not owned by the container user (UID 1000)."
-    echo "   Hindsight runs rootless and cannot fix this for you. Choose one:"
-    echo ""
-    echo "   • Recommended — use a Docker named volume (auto-owned by the container):"
-    echo "       -v hindsight-data:/home/hindsight/.pg0"
-    echo ""
-    echo "   • Or keep the host path and run as your host user, chowning it to match:"
-    echo "       sudo chown -R \$(id -u):\$(id -g) <host-directory>"
-    echo "       docker run --user \$(id -u):\$(id -g) -e HOME=/home/hindsight ..."
-    echo ""
-    echo "   See https://github.com/vectorize-io/hindsight/issues/1483"
-    return 1
-}
-
-if [ "${HINDSIGHT_START_ALL_SOURCE_ONLY:-false}" = "true" ]; then
-    return 0 2>/dev/null || exit 0
 fi
 
-check_pg0_data_integrity "${HOME}/.pg0"
-check_pg0_writable "${HOME}/.pg0" || exit 1
-
 # Service flags (default to true if not set)
 ENABLE_API="${HINDSIGHT_ENABLE_API:-true}"
 ENABLE_CP="${HINDSIGHT_ENABLE_CP:-true}"
@@ -227,7 +156,7 @@ PIDS=()
 # Start API if enabled
 if [ "$ENABLE_API" = "true" ]; then
     cd /app/api
-    API_HEALTH_URL="${HINDSIGHT_API_HEALTH_URL:-http://localhost:${HINDSIGHT_API_PORT:-8888}/health}"
+    API_HEALTH_URL="${HINDSIGHT_API_HEALTH_URL:-http://localhost:8888/health}"
     API_STARTUP_WAIT_SECONDS="${HINDSIGHT_API_STARTUP_WAIT_SECONDS:-300}"
 
     # Run API directly - Python's PYTHONUNBUFFERED=1 handles output buffering

docker/standalone/test-start-all.sh

@@ -1,121 +0,0 @@
-#!/bin/bash
-set -euo pipefail
-
-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-
-HINDSIGHT_START_ALL_SOURCE_ONLY=true
-source "$SCRIPT_DIR/start-all.sh"
-unset HINDSIGHT_START_ALL_SOURCE_ONLY
-
-TMP_DIR="$(mktemp -d)"
-trap 'chmod -R u+rwx "$TMP_DIR" 2>/dev/null || true; rm -rf "$TMP_DIR"' EXIT
-
-assert_contains() {
-    local output="$1"
-    local expected="$2"
-
-    if [[ "$output" != *"$expected"* ]]; then
-        echo "Expected output to contain: $expected"
-        echo "Actual output:"
-        echo "$output"
-        exit 1
-    fi
-}
-
-assert_not_contains() {
-    local output="$1"
-    local unexpected="$2"
-
-    if [[ "$output" == *"$unexpected"* ]]; then
-        echo "Expected output not to contain: $unexpected"
-        echo "Actual output:"
-        echo "$output"
-        exit 1
-    fi
-}
-
-assert_empty() {
-    local output="$1"
-
-    if [ -n "$output" ]; then
-        echo "Expected no output, got:"
-        echo "$output"
-        exit 1
-    fi
-}
-
-mkdir -p "$TMP_DIR/empty"
-assert_empty "$(check_pg0_data_integrity "$TMP_DIR/empty")"
-
-mkdir -p "$TMP_DIR/direct"
-touch "$TMP_DIR/direct/PG_VERSION"
-direct_output="$(check_pg0_data_integrity "$TMP_DIR/direct")"
-assert_contains "$direct_output" "Existing pg0 data directory detected"
-assert_not_contains "$direct_output" "WARNING"
-
-mkdir -p "$TMP_DIR/legacy/instance"
-touch "$TMP_DIR/legacy/instance/PG_VERSION"
-legacy_output="$(check_pg0_data_integrity "$TMP_DIR/legacy")"
-assert_contains "$legacy_output" "Existing pg0 data directory detected"
-assert_not_contains "$legacy_output" "WARNING"
-
-mkdir -p "$TMP_DIR/nested/instances/hindsight/data"
-touch "$TMP_DIR/nested/instances/hindsight/data/PG_VERSION"
-nested_output="$(check_pg0_data_integrity "$TMP_DIR/nested")"
-assert_contains "$nested_output" "Existing pg0 data directory detected"
-assert_not_contains "$nested_output" "WARNING"
-
-mkdir -p "$TMP_DIR/nonempty/instances/hindsight"
-touch "$TMP_DIR/nonempty/instances/hindsight/instance.json"
-nonempty_output="$(check_pg0_data_integrity "$TMP_DIR/nonempty")"
-assert_contains "$nonempty_output" "WARNING: pg0 data directory exists"
-
-echo "start-all pg0 integrity checks passed"
-
-# =============================================================================
-# check_pg0_writable (#1483)
-# These rely on filesystem permissions, which root bypasses; skip under root.
-# =============================================================================
-if [ "$(id -u)" != "0" ]; then
-    # Writable directory: returns 0, prints nothing, leaves no artifact behind.
-    mkdir -p "$TMP_DIR/writable"
-    writable_output="$(check_pg0_writable "$TMP_DIR/writable")"
-    assert_empty "$writable_output"
-    if [ -e "$TMP_DIR/writable/.hindsight-write-test" ]; then
-        echo "check_pg0_writable left its write-test file behind"
-        exit 1
-    fi
-
-    # Non-writable directory: returns 1 with actionable guidance.
-    mkdir -p "$TMP_DIR/readonly"
-    chmod 000 "$TMP_DIR/readonly"
-    set +e
-    readonly_output="$(check_pg0_writable "$TMP_DIR/readonly" 2>&1)"
-    readonly_rc=$?
-    set -e
-    chmod 755 "$TMP_DIR/readonly"
-    if [ "$readonly_rc" -eq 0 ]; then
-        echo "check_pg0_writable should fail on a non-writable directory"
-        exit 1
-    fi
-    assert_contains "$readonly_output" "not writable"
-    assert_contains "$readonly_output" "hindsight-data:/home/hindsight/.pg0"
-    assert_contains "$readonly_output" "--user"
-
-    # External database configured: skip the check regardless of dir perms.
-    mkdir -p "$TMP_DIR/extdb"
-    chmod 000 "$TMP_DIR/extdb"
-    set +e
-    HINDSIGHT_API_DATABASE_URL="postgres://x" check_pg0_writable "$TMP_DIR/extdb" >/dev/null 2>&1
-    extdb_rc=$?
-    set -e
-    chmod 755 "$TMP_DIR/extdb"
-    if [ "$extdb_rc" -ne 0 ]; then
-        echo "check_pg0_writable should skip when an external database is configured"
-        exit 1
-    fi
-
-    echo "start-all pg0 writability checks passed"
-else
-    echo "⚠️  Running as root; skipping pg0 writability checks (permissions are bypassed)."
-fi

helm/hindsight/Chart.lock

@@ -0,0 +1,6 @@
+dependencies:
+- name: postgresql
+  repository: https://charts.bitnami.com/bitnami
+  version: 15.5.38
+digest: sha256:f67c7612736803ece8a669f8ca6b0555f3b78557bc0ecb732aa2e43f0df7750d
+generated: "2025-12-10T17:20:57.058794+01:00"

helm/hindsight/Chart.yaml

@@ -2,8 +2,8 @@ apiVersion: v2
 name: hindsight
 description: Hindsight helm chart
 type: application
-version: 0.8.0
-appVersion: "0.8.0"
+version: 0.5.1
+appVersion: "0.5.1"
 keywords:
   - ai
   - memory

helm/hindsight/values.yaml

@@ -70,12 +70,6 @@ api:
   # Persistent volume for local model cache (reranker, embeddings)
   # Models are downloaded to /home/hindsight/.cache on first use.
   # Without persistence, models are re-downloaded on every pod restart.
-  #
-  # For production, prefer baking models into a custom image instead of
-  # enabling this PVC: image layers are pulled once per node and cached
-  # for free, while a PVC adds storage cost, pins pods to a node
-  # (ReadWriteOnce), and needs lifecycle management on uninstall/upgrade.
-  # See docs: developer/installation#bundling-custom-models-in-a-custom-image
   persistence:
     modelCache:
       enabled: false
@@ -174,10 +168,7 @@ worker:
   # affinity: {}
 
   # Persistent volume for local model cache (reranker, embeddings)
-  # Uses volumeClaimTemplates since worker is a StatefulSet — one PVC per
-  # replica. For production, prefer baking models into a custom image; see
-  # api.persistence.modelCache above and docs:
-  # developer/installation#bundling-custom-models-in-a-custom-image
+  # Uses volumeClaimTemplates since worker is a StatefulSet.
   persistence:
     modelCache:
       enabled: false

hindsight-all-npm/README.md

@@ -18,17 +18,17 @@ npm install @vectorize-io/hindsight-all @vectorize-io/hindsight-client
 ## Example
 
 ```ts
-import { HindsightServer, consoleLogger } from "@vectorize-io/hindsight-all";
-import { HindsightClient } from "@vectorize-io/hindsight-client";
+import { HindsightServer, consoleLogger } from '@vectorize-io/hindsight-all';
+import { HindsightClient } from '@vectorize-io/hindsight-client';
 
 const server = new HindsightServer({
-  profile: "my-app",
+  profile: 'my-app',
   port: 9077,
   env: {
-    HINDSIGHT_API_LLM_PROVIDER: "anthropic",
+    HINDSIGHT_API_LLM_PROVIDER: 'anthropic',
     HINDSIGHT_API_LLM_API_KEY: process.env.ANTHROPIC_API_KEY,
-    HINDSIGHT_API_LLM_MODEL: "claude-sonnet-4-20250514",
-    HINDSIGHT_EMBED_DAEMON_IDLE_TIMEOUT: "0",
+    HINDSIGHT_API_LLM_MODEL: 'claude-sonnet-4-20250514',
+    HINDSIGHT_EMBED_DAEMON_IDLE_TIMEOUT: '0',
   },
   logger: consoleLogger,
 });
@@ -37,11 +37,11 @@ await server.start();
 
 const client = new HindsightClient({ baseUrl: server.getBaseUrl() });
 
-await client.retain("user-123", "User prefers dark mode and concise answers.", {
-  documentId: "pref-2026-04-01",
+await client.retain('user-123', 'User prefers dark mode and concise answers.', {
+  documentId: 'pref-2026-04-01',
 });
 
-const recall = await client.recall("user-123", "what are the user preferences?");
+const recall = await client.recall('user-123', 'what are the user preferences?');
 console.log(recall.results);
 
 await server.stop();
@@ -62,7 +62,7 @@ If you're hacking on the Python `hindsight-embed` package in the same monorepo,
 
 ```ts
 new HindsightServer({
-  embedPackagePath: "/path/to/hindsight-embed",
+  embedPackagePath: '/path/to/hindsight-embed',
   // ...
 });
 ```

hindsight-all-npm/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@vectorize-io/hindsight-all",
-  "version": "0.8.0",
+  "version": "0.5.1",
   "description": "Node.js programmatic lifecycle manager for Hindsight — embeds a local hindsight daemon in a Node application. Pair with @vectorize-io/hindsight-client for memory operations.",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",

hindsight-all-npm/src/command.test.ts

@@ -1,36 +1,32 @@
-import { describe, it, expect } from "vitest";
-import { getEmbedCommand } from "./command.js";
+import { describe, it, expect } from 'vitest';
+import { getEmbedCommand } from './command.js';
 
-describe("getEmbedCommand", () => {
-  it("defaults to uvx hindsight-embed@latest", () => {
-    expect(getEmbedCommand()).toEqual(["uvx", "hindsight-embed@latest"]);
+describe('getEmbedCommand', () => {
+  it('defaults to uvx hindsight-embed@latest', () => {
+    expect(getEmbedCommand()).toEqual(['uvx', 'hindsight-embed@latest']);
   });
 
-  it("honours an explicit version", () => {
-    expect(getEmbedCommand({ embedVersion: "0.5.0" })).toEqual(["uvx", "[email protected]"]);
+  it('honours an explicit version', () => {
+    expect(getEmbedCommand({ embedVersion: '0.5.0' })).toEqual(['uvx', '[email protected]']);
   });
 
-  it("treats an empty version as latest", () => {
-    expect(getEmbedCommand({ embedVersion: "" })).toEqual(["uvx", "hindsight-embed@latest"]);
+  it('treats an empty version as latest', () => {
+    expect(getEmbedCommand({ embedVersion: '' })).toEqual(['uvx', 'hindsight-embed@latest']);
   });
 
-  it("uses uv run --directory when a local path is given", () => {
-    expect(getEmbedCommand({ embedPackagePath: "/abs/path" })).toEqual([
-      "uv",
-      "run",
-      "--directory",
-      "/abs/path",
-      "hindsight-embed",
+  it('uses uv run --directory when a local path is given', () => {
+    expect(getEmbedCommand({ embedPackagePath: '/abs/path' })).toEqual([
+      'uv',
+      'run',
+      '--directory',
+      '/abs/path',
+      'hindsight-embed',
     ]);
   });
 
-  it("local path takes precedence over version", () => {
-    expect(getEmbedCommand({ embedPackagePath: "/abs/path", embedVersion: "0.5.0" })).toEqual([
-      "uv",
-      "run",
-      "--directory",
-      "/abs/path",
-      "hindsight-embed",
-    ]);
+  it('local path takes precedence over version', () => {
+    expect(
+      getEmbedCommand({ embedPackagePath: '/abs/path', embedVersion: '0.5.0' }),
+    ).toEqual(['uv', 'run', '--directory', '/abs/path', 'hindsight-embed']);
   });
 });

hindsight-all-npm/src/command.ts

@@ -18,8 +18,8 @@ export interface EmbedCommandOptions {
 
 export function getEmbedCommand(opts: EmbedCommandOptions = {}): string[] {
   if (opts.embedPackagePath) {
-    return ["uv", "run", "--directory", opts.embedPackagePath, "hindsight-embed"];
+    return ['uv', 'run', '--directory', opts.embedPackagePath, 'hindsight-embed'];
   }
-  const version = opts.embedVersion && opts.embedVersion.length > 0 ? opts.embedVersion : "latest";
-  return ["uvx", `hindsight-embed@${version}`];
+  const version = opts.embedVersion && opts.embedVersion.length > 0 ? opts.embedVersion : 'latest';
+  return ['uvx', `hindsight-embed@${version}`];
 }

hindsight-all-npm/src/index.ts

@@ -1,7 +1,7 @@
-export { HindsightServer } from "./server.js";
-export { getEmbedCommand } from "./command.js";
-export { silentLogger, consoleLogger } from "./logger.js";
+export { HindsightServer } from './server.js';
+export { getEmbedCommand } from './command.js';
+export { silentLogger, consoleLogger } from './logger.js';
 
-export type { Logger } from "./logger.js";
-export type { EmbedCommandOptions } from "./command.js";
-export type { HindsightServerOptions } from "./types.js";
+export type { Logger } from './logger.js';
+export type { EmbedCommandOptions } from './command.js';
+export type { HindsightServerOptions } from './types.js';

hindsight-all-npm/src/server.test.ts

@@ -1,32 +1,32 @@
-import { describe, it, expect } from "vitest";
-import { HindsightServer } from "./server.js";
+import { describe, it, expect } from 'vitest';
+import { HindsightServer } from './server.js';
 
-describe("HindsightServer construction", () => {
-  it("defaults base URL to http://127.0.0.1:8888", () => {
+describe('HindsightServer construction', () => {
+  it('defaults base URL to http://127.0.0.1:8888', () => {
     const server = new HindsightServer();
-    expect(server.getBaseUrl()).toBe("http://127.0.0.1:8888");
-    expect(server.getProfile()).toBe("default");
+    expect(server.getBaseUrl()).toBe('http://127.0.0.1:8888');
+    expect(server.getProfile()).toBe('default');
   });
 
-  it("honours custom profile, port, and host", () => {
-    const server = new HindsightServer({ profile: "app", port: 9077, host: "0.0.0.0" });
-    expect(server.getProfile()).toBe("app");
-    expect(server.getBaseUrl()).toBe("http://0.0.0.0:9077");
+  it('honours custom profile, port, and host', () => {
+    const server = new HindsightServer({ profile: 'app', port: 9077, host: '0.0.0.0' });
+    expect(server.getProfile()).toBe('app');
+    expect(server.getBaseUrl()).toBe('http://0.0.0.0:9077');
   });
 
-  it("accepts open env pass-through without complaining about unknown keys", () => {
+  it('accepts open env pass-through without complaining about unknown keys', () => {
     const server = new HindsightServer({
       env: {
-        HINDSIGHT_API_LLM_PROVIDER: "openai",
-        HINDSIGHT_API_LLM_MODEL: "gpt-4o-mini",
+        HINDSIGHT_API_LLM_PROVIDER: 'openai',
+        HINDSIGHT_API_LLM_MODEL: 'gpt-4o-mini',
         // A field that does not exist today — should still be accepted
-        HINDSIGHT_FUTURE_FLAG: "enabled",
+        HINDSIGHT_FUTURE_FLAG: 'enabled',
       },
     });
     expect(server).toBeInstanceOf(HindsightServer);
   });
 
-  it("exposes checkHealth that returns false when no daemon is running", async () => {
+  it('exposes checkHealth that returns false when no daemon is running', async () => {
     // Random high port that nothing is listening on.
     const server = new HindsightServer({ port: 1, readyTimeoutMs: 100 });
     const healthy = await server.checkHealth();

hindsight-all-npm/src/server.ts

@@ -1,12 +1,12 @@
-import { spawn } from "child_process";
-import { getEmbedCommand } from "./command.js";
-import { silentLogger } from "./logger.js";
-import type { Logger } from "./logger.js";
-import type { HindsightServerOptions } from "./types.js";
+import { spawn } from 'child_process';
+import { getEmbedCommand } from './command.js';
+import { silentLogger } from './logger.js';
+import type { Logger } from './logger.js';
+import type { HindsightServerOptions } from './types.js';
 
 const DEFAULT_PORT = 8888;
-const DEFAULT_HOST = "127.0.0.1";
-const DEFAULT_PROFILE = "default";
+const DEFAULT_HOST = '127.0.0.1';
+const DEFAULT_PROFILE = 'default';
 const DEFAULT_READY_TIMEOUT_MS = 30_000;
 const DEFAULT_READY_POLL_INTERVAL_MS = 1_000;
 
@@ -61,7 +61,7 @@ export class HindsightServer {
     this.userEnv = opts.env ?? {};
     this.extraProfileCreateArgs = opts.extraProfileCreateArgs ?? [];
     this.extraDaemonStartArgs = opts.extraDaemonStartArgs ?? [];
-    this.platformCpuWorkaround = opts.platformCpuWorkaround ?? process.platform === "darwin";
+    this.platformCpuWorkaround = opts.platformCpuWorkaround ?? (process.platform === 'darwin');
     this.readyTimeoutMs = opts.readyTimeoutMs ?? DEFAULT_READY_TIMEOUT_MS;
     this.readyPollIntervalMs = opts.readyPollIntervalMs ?? DEFAULT_READY_POLL_INTERVAL_MS;
     this.logger = opts.logger ?? silentLogger;
@@ -100,22 +100,22 @@ export class HindsightServer {
       embedVersion: this.embedVersion,
       embedPackagePath: this.embedPackagePath,
     });
-    const args = [...baseArgs, "daemon", "--profile", this.profile, "stop"];
+    const args = [...baseArgs, 'daemon', '--profile', this.profile, 'stop'];
 
-    const child = spawn(cmd, args, { stdio: "pipe" });
-    this.pipeOutput(child, "daemon.stop");
+    const child = spawn(cmd, args, { stdio: 'pipe' });
+    this.pipeOutput(child, 'daemon.stop');
 
     await new Promise<void>((resolve) => {
       const timeout = setTimeout(() => {
         this.logger.warn(`[hindsight] daemon stop timed out after 5s`);
         resolve();
       }, 5_000);
-      child.on("exit", () => {
+      child.on('exit', () => {
         clearTimeout(timeout);
         this.logger.info(`[hindsight] daemon stopped`);
         resolve();
       });
-      child.on("error", (err) => {
+      child.on('error', (err) => {
         clearTimeout(timeout);
         this.logger.warn(`[hindsight] error stopping daemon: ${err.message}`);
         resolve();
@@ -147,9 +147,9 @@ export class HindsightServer {
   private buildEnv(): NodeJS.ProcessEnv {
     const merged: NodeJS.ProcessEnv = { ...process.env };
 
-    if (this.platformCpuWorkaround && process.platform === "darwin") {
-      merged["HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU"] = "1";
-      merged["HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU"] = "1";
+    if (this.platformCpuWorkaround && process.platform === 'darwin') {
+      merged['HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU'] = '1';
+      merged['HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU'] = '1';
     }
 
     for (const [key, value] of Object.entries(this.userEnv)) {
@@ -175,11 +175,11 @@ export class HindsightServer {
     });
     const createArgs = [
       ...baseArgs,
-      "profile",
-      "create",
+      'profile',
+      'create',
       this.profile,
-      "--merge",
-      "--port",
+      '--merge',
+      '--port',
       String(this.port),
     ];
 
@@ -189,12 +189,12 @@ export class HindsightServer {
     // host state into profile config.
     const envForProfile = this.collectProfileEnv(env);
     for (const [key, value] of Object.entries(envForProfile)) {
-      createArgs.push("--env", `${key}=${value}`);
+      createArgs.push('--env', `${key}=${value}`);
     }
 
     createArgs.push(...this.extraProfileCreateArgs);
 
-    await this.runCommand(cmd, createArgs, env, "profile.create");
+    await this.runCommand(cmd, createArgs, env, 'profile.create');
   }
 
   /** Collect only the env vars that should be written into the profile file. */
@@ -209,10 +209,10 @@ export class HindsightServer {
     }
 
     // 2. CPU workaround — only if auto-applied and not already overridden.
-    if (this.platformCpuWorkaround && process.platform === "darwin") {
+    if (this.platformCpuWorkaround && process.platform === 'darwin') {
       const cpuKeys = [
-        "HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU",
-        "HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU",
+        'HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU',
+        'HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU',
       ];
       for (const key of cpuKeys) {
         if (!(key in out) && env[key] !== undefined) {
@@ -231,14 +231,14 @@ export class HindsightServer {
     });
     const args = [
       ...baseArgs,
-      "daemon",
-      "--profile",
+      'daemon',
+      '--profile',
       this.profile,
-      "start",
+      'start',
       ...this.extraDaemonStartArgs,
     ];
 
-    await this.runCommand(cmd, args, env, "daemon.start");
+    await this.runCommand(cmd, args, env, 'daemon.start');
   }
 
   /**
@@ -249,34 +249,34 @@ export class HindsightServer {
     cmd: string,
     args: string[],
     env: NodeJS.ProcessEnv,
-    label: string
+    label: string,
   ): Promise<void> {
-    const child = spawn(cmd, args, { stdio: "pipe", env });
-    let output = "";
-    child.stdout?.on("data", (data: Buffer) => {
+    const child = spawn(cmd, args, { stdio: 'pipe', env });
+    let output = '';
+    child.stdout?.on('data', (data: Buffer) => {
       const text = data.toString();
       output += text;
-      for (const line of text.trimEnd().split("\n")) {
+      for (const line of text.trimEnd().split('\n')) {
         if (line) this.logger.info(`[hindsight:${label}] ${line}`);
       }
     });
-    child.stderr?.on("data", (data: Buffer) => {
+    child.stderr?.on('data', (data: Buffer) => {
       const text = data.toString();
       output += text;
-      for (const line of text.trimEnd().split("\n")) {
+      for (const line of text.trimEnd().split('\n')) {
         if (line) this.logger.warn(`[hindsight:${label}] ${line}`);
       }
     });
 
     await new Promise<void>((resolve, reject) => {
-      child.on("exit", (code) => {
+      child.on('exit', (code) => {
         if (code === 0) {
           resolve();
         } else {
           reject(new Error(`${label} failed with code ${code}: ${output.trim()}`));
         }
       });
-      child.on("error", (err) => {
+      child.on('error', (err) => {
         reject(new Error(`${label} failed to spawn: ${err.message}`, { cause: err }));
       });
     });
@@ -284,13 +284,13 @@ export class HindsightServer {
 
   /** Stream a spawned child's stdout/stderr through the logger without blocking. */
   private pipeOutput(child: ReturnType<typeof spawn>, label: string): void {
-    child.stdout?.on("data", (data: Buffer) => {
-      for (const line of data.toString().trimEnd().split("\n")) {
+    child.stdout?.on('data', (data: Buffer) => {
+      for (const line of data.toString().trimEnd().split('\n')) {
         if (line) this.logger.info(`[hindsight:${label}] ${line}`);
       }
     });
-    child.stderr?.on("data", (data: Buffer) => {
-      for (const line of data.toString().trimEnd().split("\n")) {
+    child.stderr?.on('data', (data: Buffer) => {
+      for (const line of data.toString().trimEnd().split('\n')) {
         if (line) this.logger.warn(`[hindsight:${label}] ${line}`);
       }
     });
@@ -316,7 +316,7 @@ export class HindsightServer {
       await new Promise((resolve) => setTimeout(resolve, this.readyPollIntervalMs));
     }
     throw new Error(
-      `Hindsight daemon did not become ready within ${this.readyTimeoutMs}ms at ${this.baseUrl}`
+      `Hindsight daemon did not become ready within ${this.readyTimeoutMs}ms at ${this.baseUrl}`,
     );
   }
 }

hindsight-all-npm/src/types.ts

@@ -1,4 +1,4 @@
-import type { Logger } from "./logger.js";
+import type { Logger } from './logger.js';
 
 /**
  * Options for {@link HindsightServer}.

hindsight-all-npm/tsup.config.ts

@@ -1,10 +1,10 @@
-import { defineConfig } from "tsup";
+import { defineConfig } from 'tsup';
 
 export default defineConfig({
-  entry: ["src/index.ts"],
-  format: ["esm"],
+  entry: ['src/index.ts'],
+  format: ['esm'],
   dts: true,
-  outDir: "dist",
+  outDir: 'dist',
   clean: true,
   sourcemap: true,
   bundle: true,

hindsight-all-npm/vitest.config.ts

@@ -1,8 +1,8 @@
-import { defineConfig } from "vitest/config";
+import { defineConfig } from 'vitest/config';
 
 export default defineConfig({
   test: {
-    include: ["src/**/*.test.ts"],
-    environment: "node",
+    include: ['src/**/*.test.ts'],
+    environment: 'node',
   },
 });

hindsight-all-slim/pyproject.toml

@@ -4,12 +4,12 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "hindsight-all-slim"
-version = "0.8.0"
+version = "0.5.1"
 description = "Hindsight: Agent Memory That Works Like Human Memory - Slim All-in-One Bundle"
 readme = "README.md"
 requires-python = ">=3.11"
 dependencies = [
-    "hindsight-api-slim==0.8.0",
+    "hindsight-api-slim>=0.4.17",
     "hindsight-client>=0.0.7",
     "hindsight-embed>=0.1.0",
 ]

hindsight-all/hindsight/embedded.py

@@ -71,7 +71,7 @@ class HindsightEmbedded:
         llm_model: Model name to use
         llm_base_url: Optional custom base URL for LLM API
         database_url: Optional database URL override (default: profile-specific pg0)
-        idle_timeout: Seconds before daemon auto-exits when idle (default: 0, disabled)
+        idle_timeout: Seconds before daemon auto-exits when idle (default: 300)
         log_level: Daemon log level (default: "info")
         ui: Whether to start the control plane web UI alongside the daemon (default: False)
         ui_port: Port for the UI. Defaults to daemon_port + 10000.
@@ -86,7 +86,7 @@ class HindsightEmbedded:
         llm_model: str = "openai/gpt-oss-120b",
         llm_base_url: Optional[str] = None,
         database_url: Optional[str] = None,
-        idle_timeout: int = 0,
+        idle_timeout: int = 300,
         log_level: str = "info",
         ui: bool = False,
         ui_port: Optional[int] = None,
@@ -102,7 +102,7 @@ class HindsightEmbedded:
             llm_model: Model name to use
             llm_base_url: Optional custom base URL for LLM API
             database_url: Optional database URL override
-            idle_timeout: Seconds before daemon auto-exits when idle (0 = disabled)
+            idle_timeout: Seconds before daemon auto-exits when idle
             log_level: Daemon log level
             ui: Whether to start the control plane web UI alongside the daemon
             ui_port: Port for the UI (defaults to daemon_port + 10000)
@@ -142,37 +142,14 @@ class HindsightEmbedded:
         self._memories_api: Optional[MemoriesAPI] = None
 
     def _ensure_started(self):
-        """Ensure daemon is running (thread-safe), restarting if crashed."""
+        """Ensure daemon is running (thread-safe)."""
         if self._started and self._client is not None:
-            if self._manager.is_running(self.profile):
-                return
-            # Daemon crashed — reset state and fall through to restart
-            logger.warning(
-                "Daemon for profile '%s' is no longer responsive, restarting...",
-                self.profile,
-            )
-            try:
-                self._client.close()
-            except Exception:
-                logger.debug("Error closing stale client", exc_info=True)
-            self._client = None
-            self._started = False
+            return
 
         with self._lock:
             # Double-check after acquiring lock
             if self._started and self._client is not None:
-                if self._manager.is_running(self.profile):
-                    return
-                logger.warning(
-                    "Daemon for profile '%s' is no longer responsive (lock path), restarting...",
-                    self.profile,
-                )
-                try:
-                    self._client.close()
-                except Exception:
-                    logger.debug("Error closing stale client", exc_info=True)
-                self._client = None
-                self._started = False
+                return
 
             if self._closed:
                 raise RuntimeError(
@@ -276,10 +253,23 @@ class HindsightEmbedded:
         This allows HindsightEmbedded to expose all HindsightClient methods
         without manually wrapping each one.
         """
-        # Ensure server is started (and restart if crashed) before proxying
+        # Ensure server is started before proxying
         self._ensure_started()
 
-        return getattr(self._client, name)
+        # Get the attribute from the underlying client
+        attr = getattr(self._client, name)
+
+        # If it's a callable, wrap it to ensure server is started
+        # (shouldn't be needed since _ensure_started already called, but defensive)
+        if callable(attr):
+
+            def wrapper(*args, **kwargs):
+                self._ensure_started()
+                return attr(*args, **kwargs)
+
+            return wrapper
+
+        return attr
 
     def __enter__(self):
         """Context manager entry - ensures server is started."""
@@ -404,8 +394,11 @@ class HindsightEmbedded:
         """
         Get the underlying Hindsight client for direct access.
 
-        Ensures daemon is started (and restarts it if it has crashed) before
-        returning the client.
+        WARNING: Using this property directly means daemon restarts won't be
+        handled automatically. Prefer using the API namespaces (banks, mental_models,
+        directives, memories) or direct method calls on HindsightEmbedded instead.
+
+        Ensures daemon is started before returning the client.
 
         Returns:
             Hindsight: The underlying client instance
@@ -416,8 +409,9 @@ class HindsightEmbedded:
 
             embedded = HindsightEmbedded(profile="myapp", ...)
 
+            # Direct access (not recommended - daemon crashes won't be handled)
             client = embedded.client
-            banks = client.list_banks()
+            banks = client.list_banks()  # If daemon crashes, this will fail
             ```
         """
         self._ensure_started()
@@ -431,13 +425,8 @@ class HindsightEmbedded:
 
     @property
     def is_running(self) -> bool:
-        """Check if the client is initialized and the daemon is responsive."""
-        return (
-            self._started
-            and not self._closed
-            and self._client is not None
-            and self._manager.is_running(self.profile)
-        )
+        """Check if the client is initialized."""
+        return self._started and not self._closed and self._client is not None
 
     @property
     def ui_url(self) -> str:

hindsight-all/pyproject.toml

@@ -4,12 +4,12 @@ build-backend = "hatchling.build"
 
 [project]
 name = "hindsight-all"
-version = "0.8.0"
+version = "0.5.1"
 description = "Hindsight: Agent Memory That Works Like Human Memory - All-in-One Bundle"
 readme = "README.md"
 requires-python = ">=3.11"
 dependencies = [
-    "hindsight-api-slim[all]==0.8.0",
+    "hindsight-api-slim[all]>=0.4.17",
     "hindsight-client>=0.0.7",
     "hindsight-embed>=0.1.0",
 ]
@@ -21,7 +21,7 @@ hindsight-embed = { workspace = true }
 
 [project.optional-dependencies]
 local-llm = [
-    "hindsight-api-slim[local-llm]==0.8.0",
+    "hindsight-api-slim[local-llm]>=0.4.17",
 ]
 test = [
     "pytest>=7.0.0",

hindsight-all/tests/test_embedded.py

@@ -386,7 +386,7 @@ def test_embedded_ui_flag(llm_config):
 
         # Verify UI is reachable and reports connected dataplane
         ui_url = client.ui_url
-        assert isinstance(ui_url, str) and ui_url, "ui_url should be a non-empty string"
+        assert ui_url, "ui_url should be set"
 
         health_url = f"{ui_url}/api/health"
         with urllib.request.urlopen(health_url, timeout=10) as resp:
@@ -401,42 +401,3 @@ def test_embedded_ui_flag(llm_config):
 
     finally:
         client.close()
-
-
-def test_embedded_daemon_crash_recovery(llm_config):
-    """
-    Test that HindsightEmbedded recovers when the daemon crashes.
-
-    Simulates a crash by stopping the daemon, then verifies
-    that the next operation transparently restarts it.
-    """
-    profile = f"test_crash_{uuid.uuid4().hex[:8]}"
-    bank_id = f"bank_{uuid.uuid4().hex[:8]}"
-
-    client = HindsightEmbedded(profile=profile, log_level="info", **llm_config)
-
-    try:
-        # Start daemon and store a memory
-        result = client.retain(bank_id=bank_id, content="Before crash")
-        assert result.success, "Initial retain should succeed"
-        assert client.is_running, "Daemon should be running"
-
-        original_url = client.url
-
-        # Simulate daemon crash by stopping it
-        client._manager.stop(client.profile)
-        assert not client._manager.is_running(client.profile), (
-            "Daemon should be stopped after simulated crash"
-        )
-
-        # Next operation should transparently restart the daemon
-        result2 = client.retain(bank_id=bank_id, content="After crash recovery")
-        assert result2.success, "Retain after crash recovery should succeed"
-        assert client.is_running, "Daemon should be running again after recovery"
-
-        # Verify recall still works
-        recall_result = client.recall(bank_id=bank_id, query="crash")
-        assert isinstance(recall_result.results, list), "Recall should return results"
-
-    finally:
-        client.close()

hindsight-api-slim/README.md

@@ -99,7 +99,7 @@ hindsight-api
 ## Docker
 
 ```bash
-docker run -it --name hindsight --restart unless-stopped -p 8888:8888 \
+docker run --rm -it -p 8888:8888 \
   -e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \
   -v $HOME/.hindsight-docker:/home/hindsight/.pg0 \
   ghcr.io/vectorize-io/hindsight:latest

hindsight-api-slim/hindsight_api/__init__.py

@@ -4,13 +4,6 @@ Memory System for AI Agents.
 Temporal + Semantic Memory Architecture using PostgreSQL with pgvector.
 """
 
-# Cap native ML thread pools (OpenBLAS/OpenMP/MKL) before any import pulls in
-# numpy/torch/onnxruntime — they read these env vars only at load time. See
-# hindsight_api/_thread_limits.py for the rationale.
-from ._thread_limits import apply_default_thread_limits
-
-apply_default_thread_limits()
-
 from .config import HindsightConfig, get_config
 from .engine.cross_encoder import CrossEncoderModel, LocalSTCrossEncoder, RemoteTEICrossEncoder
 from .engine.embeddings import Embeddings, LocalSTEmbeddings, RemoteTEIEmbeddings
@@ -53,4 +46,4 @@ __all__ = [
     "RemoteTEICrossEncoder",
     "LLMConfig",
 ]
-__version__ = "0.8.0"
+__version__ = "0.5.1"

hindsight-api-slim/hindsight_api/_pg_search.py

@@ -1,85 +0,0 @@
-"""Helpers for ParadeDB pg_search index configuration."""
-
-from __future__ import annotations
-
-import re
-from collections.abc import Sequence
-
-PG_SEARCH_TOKENIZER_ENV = "HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER"
-
-_SIMPLE_TOKENIZERS = {
-    "unicode_words",
-    "simple",
-    "whitespace",
-    "literal",
-    "literal_normalized",
-    "chinese_compatible",
-    "icu",
-    "jieba",
-    "source_code",
-}
-
-_TOKENIZER_ALIASES = {
-    "chinese_lindera": "lindera(chinese)",
-    "japanese_lindera": "lindera(japanese)",
-    "korean_lindera": "lindera(korean)",
-    "lindera_chinese": "lindera(chinese)",
-    "lindera_japanese": "lindera(japanese)",
-    "lindera_korean": "lindera(korean)",
-}
-
-
-def normalize_pg_search_tokenizer(value: str | None) -> str:
-    """Validate and normalize a ParadeDB pg_search tokenizer setting.
-
-    Returns an empty string when unset. The returned value is safe to embed after
-    ``pdb.`` in a CREATE INDEX expression.
-    """
-
-    tokenizer = (value or "").strip().lower()
-    if not tokenizer:
-        return ""
-
-    if tokenizer in _TOKENIZER_ALIASES:
-        return _TOKENIZER_ALIASES[tokenizer]
-
-    if tokenizer in _SIMPLE_TOKENIZERS:
-        return tokenizer
-
-    lindera_match = re.fullmatch(r"lindera\((chinese|japanese|korean)\)", tokenizer)
-    if lindera_match:
-        return tokenizer
-
-    ngram_match = re.fullmatch(r"(ngram|edge_ngram)\((\d{1,3}),\s*(\d{1,3})\)", tokenizer)
-    if ngram_match:
-        kind, min_gram, max_gram = ngram_match.groups()
-        min_value = int(min_gram)
-        max_value = int(max_gram)
-        if min_value <= 0 or min_value > max_value:
-            raise ValueError(
-                f"Invalid {PG_SEARCH_TOKENIZER_ENV}: {value!r}. "
-                "ngram and edge_ngram require positive min/max gram sizes with min <= max."
-            )
-        return f"{kind}({min_value},{max_value})"
-
-    raise ValueError(
-        f"Invalid {PG_SEARCH_TOKENIZER_ENV}: {value!r}. "
-        "Supported values are: unicode_words, simple, whitespace, literal, "
-        "literal_normalized, chinese_compatible, icu, jieba, source_code, "
-        "chinese_lindera, japanese_lindera, korean_lindera, or "
-        "lindera(chinese|japanese|korean), ngram(min,max), or edge_ngram(min,max)."
-    )
-
-
-def pg_search_bm25_columns(
-    key_field: str,
-    text_fields: Sequence[str],
-    tokenizer: str | None,
-) -> str:
-    """Build a ParadeDB BM25 column list for CREATE INDEX."""
-
-    normalized = normalize_pg_search_tokenizer(tokenizer)
-    if not normalized:
-        return ", ".join([key_field, *text_fields])
-
-    return ", ".join([key_field, *(f"({field}::pdb.{normalized})" for field in text_fields)])

hindsight-api-slim/hindsight_api/_thread_limits.py

@@ -1,107 +0,0 @@
-"""Process-level caps for native ML thread pools.
-
-OpenBLAS, OpenMP, and MKL each spawn a worker pool sized to the host CPU count
-the first time they are loaded (numpy pulls in OpenBLAS eagerly; torch and
-onnxruntime load their pools lazily on first inference). Hindsight already
-parallelizes at the request level via thread-pool executors (embeddings on the
-default executor, the reranker on its own pool), so these native intra-op pools
-oversubscribe the CPU: on a many-core host the process accumulates 100+ native
-threads, which inflates memory and, under contention, can degrade throughput.
-
-We bound each pool to ``_MAX_NATIVE_THREADS`` (or the available CPU count, if
-smaller). "Available" is the CPU budget actually granted to the process, not
-``os.cpu_count()``: in a CPU-limited container ``os.cpu_count()`` still reports
-the host's cores, so sizing pools by it oversubscribes the container's real
-quota — the exact failure mode this guards against. We therefore take the
-smallest of the CPU-affinity set, the cgroup CPU quota, and ``os.cpu_count()``.
-
-Every cap is applied with ``setdefault`` so an operator who has deliberately
-tuned one of these variables keeps their value. This must run *before* numpy,
-torch, or onnxruntime are imported — those libraries read the variables only at
-load time — which is why it is invoked at the very top of
-``hindsight_api/__init__.py``, ahead of the package's other imports.
-"""
-
-from __future__ import annotations
-
-import os
-
-# Native threading env vars, each read by the respective library at load time.
-_NATIVE_THREAD_VARS = (
-    "OMP_NUM_THREADS",  # OpenMP — torch, onnxruntime, some BLAS builds
-    "OPENBLAS_NUM_THREADS",  # OpenBLAS — numpy's default BLAS
-    "MKL_NUM_THREADS",  # Intel MKL — numpy/torch when MKL-backed
-    "NUMEXPR_NUM_THREADS",  # numexpr expression engine
-)
-
-# Upper bound on intra-op threads per native pool. Bounds runaway growth on
-# many-core hosts without serialising single-request inference.
-_MAX_NATIVE_THREADS = 16
-
-
-def _quota_to_cpus(quota: int, period: int) -> int | None:
-    """Whole CPUs from a CFS quota/period pair, or None if unlimited."""
-    if quota > 0 and period > 0:
-        # Floor (never round up) so we never exceed the granted budget.
-        return max(1, quota // period)
-    return None
-
-
-def _parse_cgroup_v2_cpu_max(text: str) -> int | None:
-    """Parse cgroup v2 ``cpu.max`` ("<quota> <period>", or "max <period>")."""
-    parts = text.split()
-    if len(parts) >= 2 and parts[0] != "max":
-        try:
-            return _quota_to_cpus(int(parts[0]), int(parts[1]))
-        except ValueError:
-            return None
-    return None
-
-
-def _cgroup_cpu_quota() -> int | None:
-    """Effective CPUs from the cgroup CPU quota, or None if unlimited/unknown."""
-    try:  # cgroup v2
-        with open("/sys/fs/cgroup/cpu.max") as fh:
-            return _parse_cgroup_v2_cpu_max(fh.read())
-    except OSError:
-        pass
-    try:  # cgroup v1
-        with open("/sys/fs/cgroup/cpu/cpu.cfs_quota_us") as fh:
-            quota = int(fh.read())
-        with open("/sys/fs/cgroup/cpu/cpu.cfs_period_us") as fh:
-            period = int(fh.read())
-        return _quota_to_cpus(quota, period)
-    except (OSError, ValueError):
-        return None
-
-
-def _available_cpu_count() -> int:
-    """CPUs actually available to this process.
-
-    The smallest of the CPU-affinity set (cpuset / ``--cpuset-cpus``), the
-    cgroup CPU quota (``--cpus``), and ``os.cpu_count()`` — each captures a
-    different way the budget can be constrained, and the last alone overcounts
-    inside a limited container.
-    """
-    candidates = [os.cpu_count() or 1]
-    if hasattr(os, "sched_getaffinity"):
-        try:
-            candidates.append(len(os.sched_getaffinity(0)))
-        except OSError:
-            pass
-    quota = _cgroup_cpu_quota()
-    if quota is not None:
-        candidates.append(quota)
-    return max(1, min(candidates))
-
-
-def default_native_thread_count() -> int:
-    """Per-pool cap: ``_MAX_NATIVE_THREADS``, or available CPUs if fewer."""
-    return min(_MAX_NATIVE_THREADS, _available_cpu_count())
-
-
-def apply_default_thread_limits() -> None:
-    """Cap native ML thread pools unless the operator has set the var already."""
-    value = str(default_native_thread_count())
-    for var in _NATIVE_THREAD_VARS:
-        os.environ.setdefault(var, value)

hindsight-api-slim/hindsight_api/_vector_index.py

@@ -1,228 +0,0 @@
-"""Shared PostgreSQL vector-extension dispatch helpers."""
-
-from __future__ import annotations
-
-import logging
-import os
-
-from sqlalchemy import text
-from sqlalchemy.engine import Connection
-
-logger = logging.getLogger(__name__)
-
-# Extensions a user can set via HINDSIGHT_API_VECTOR_EXTENSION.
-CONFIGURABLE_EXTENSIONS = ("pgvector", "pgvectorscale", "vchord", "scann")
-
-# Extensions detect_vector_extension() can return. pg_diskann is a runtime-only
-# resolution from a configured "pgvectorscale" backend on Azure (uses a different
-# WITH clause), never a value the user sets directly.
-RESOLVED_EXTENSIONS = (*CONFIGURABLE_EXTENSIONS, "pg_diskann")
-
-# Backwards-compatible alias for older imports.
-VALID_EXTENSIONS = CONFIGURABLE_EXTENSIONS
-
-SCANN_MIN_ROWS_FOR_AUTO_INDEX = 10_000
-
-
-_EXTENSION_NAMES = {
-    "pgvector": "vector",
-    "pgvectorscale": "vectorscale",
-    "vchord": "vchord",
-    "scann": "alloydb_scann",
-}
-
-_INDEX_USING_CLAUSES = {
-    "pgvector": "USING hnsw (embedding vector_cosine_ops)",
-    "pgvectorscale": "USING diskann (embedding vector_cosine_ops) WITH (num_neighbors = 50)",
-    "pg_diskann": "USING diskann (embedding vector_cosine_ops) WITH (max_neighbors = 50)",
-    "vchord": "USING vchordrq (embedding vector_cosine_ops)",
-    "scann": "USING scann (embedding cosine) WITH (mode = 'AUTO')",
-}
-
-_INDEX_TYPE_KEYWORDS = {
-    "pgvector": "hnsw",
-    "pgvectorscale": "diskann",
-    "pg_diskann": "diskann",
-    "vchord": "vchordrq",
-    "scann": "scann",
-}
-
-# Per-backend ANN search-time tuning GUCs. Each entry is a tuple of
-# (guc_name, value) pairs the caller can apply with SET or SET LOCAL.
-#
-# - pgvector exposes hnsw.ef_search. The 60 / 200 pair is unchanged from the
-#   pre-dispatcher code (internal benchmarks tuned around our embedding count
-#   and recall floor; see the link_utils / pool init call sites for the
-#   latency-vs-recall framing).
-# - vchord exposes vchordrq.probes (no default; see VectorChord issue #392)
-#   and vchordrq.epsilon (default 1.9). probes = 10 / 30 are starting
-#   defaults pending a workload-specific sweep — vchordrq's recall curve
-#   shape differs from HNSW's, so the pgvector numbers don't translate
-#   directly. Revisit with a per-cluster benchmark once we have production
-#   recall data; until then these are deliberately conservative on the
-#   high-recall path. We leave epsilon at its default; tightening it is a
-#   separate trade-off.
-# - pgvectorscale / pg_diskann / scann do not expose an equivalent per-statement
-#   knob in the engine today, so the dispatcher returns no statements for them.
-_ANN_TUNING_LOW_LATENCY: dict[str, tuple[tuple[str, str], ...]] = {
-    "pgvector": (("hnsw.ef_search", "60"),),
-    "vchord": (("vchordrq.probes", "10"),),
-}
-_ANN_TUNING_HIGH_RECALL: dict[str, tuple[tuple[str, str], ...]] = {
-    "pgvector": (("hnsw.ef_search", "200"),),
-    "vchord": (("vchordrq.probes", "30"),),
-}
-
-_EXTENSION_INSTALL_SQL = {
-    "pgvector": ("CREATE EXTENSION IF NOT EXISTS vector",),
-    "pgvectorscale": (
-        "CREATE EXTENSION IF NOT EXISTS vector",
-        "CREATE EXTENSION IF NOT EXISTS vectorscale CASCADE",
-    ),
-    "vchord": ("CREATE EXTENSION IF NOT EXISTS vchord CASCADE",),
-    "scann": (
-        "CREATE EXTENSION IF NOT EXISTS vector",
-        "CREATE EXTENSION IF NOT EXISTS alloydb_scann CASCADE",
-    ),
-}
-
-_INSTALL_HINTS = {
-    "pgvector": "CREATE EXTENSION vector;",
-    "pgvectorscale": "CREATE EXTENSION vector; then CREATE EXTENSION vectorscale CASCADE; (or pg_diskann on Azure)",
-    "vchord": "CREATE EXTENSION vchord CASCADE;",
-    "scann": "CREATE EXTENSION vector; then CREATE EXTENSION alloydb_scann CASCADE;",
-}
-
-
-def configured_vector_extension() -> str:
-    """Return the user-configured vector backend extension.
-
-    Reads ``HINDSIGHT_API_VECTOR_EXTENSION`` (default ``"pgvector"``) and
-    validates it via :func:`validate_extension`. This is the single source of
-    truth for runtime code that needs to dispatch behaviour by vector backend;
-    callers should prefer this over reading the env var directly, so the
-    default value and the lookup mechanism live in one place.
-    """
-    return validate_extension(os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector"))
-
-
-def validate_extension(name: str) -> str:
-    """Return a normalized configurable vector extension name or raise.
-
-    Used at the user-facing config boundary; pg_diskann is rejected here because
-    it is a detection-time alias, never a value the user sets directly.
-    """
-    ext = name.lower()
-    if ext not in CONFIGURABLE_EXTENSIONS:
-        valid = ", ".join(CONFIGURABLE_EXTENSIONS)
-        raise ValueError(f"Invalid vector_extension: {name}. Must be one of: {valid}")
-    return ext
-
-
-def _normalize_resolved(name: str) -> str:
-    """Normalize either a user-configurable or detect-time extension name."""
-    ext = name.lower()
-    if ext not in RESOLVED_EXTENSIONS:
-        valid = ", ".join(RESOLVED_EXTENSIONS)
-        raise ValueError(f"Unknown vector extension: {name}. Must be one of: {valid}")
-    return ext
-
-
-def pg_extension_name(ext: str) -> str:
-    """Return the PostgreSQL extension name for a configured vector backend."""
-    return _EXTENSION_NAMES[validate_extension(ext)]
-
-
-def index_using_clause(ext: str) -> str:
-    """Return the CREATE INDEX USING clause for the vector backend."""
-    return _INDEX_USING_CLAUSES[_normalize_resolved(ext)]
-
-
-def index_type_keyword(ext: str) -> str:
-    """Return the keyword that identifies this index type in pg_indexes.indexdef."""
-    return _INDEX_TYPE_KEYWORDS[_normalize_resolved(ext)]
-
-
-def minimum_rows_for_index(ext: str) -> int:
-    """Return the minimum populated embedding rows before creating this index type."""
-    return SCANN_MIN_ROWS_FOR_AUTO_INDEX if _normalize_resolved(ext) == "scann" else 0
-
-
-def should_defer_index_creation(ext: str, row_count: int) -> bool:
-    """Return True when index creation should wait for more embeddings."""
-    minimum_rows = minimum_rows_for_index(ext)
-    return minimum_rows > 0 and row_count < minimum_rows
-
-
-def ann_search_tuning_settings(ext: str, *, kind: str) -> tuple[tuple[str, str], ...]:
-    """Return per-backend (guc_name, value) pairs for ANN search-time tuning.
-
-    ``kind`` is ``"low_latency"`` for retain-side link probing (smaller probe
-    count, lower recall, lower latency) and ``"high_recall"`` for connection
-    init in the pool (larger probe count, higher recall). Callers wrap each
-    pair with ``SET LOCAL`` or ``SET`` themselves so the same dispatcher works
-    for both transaction-scoped and session-scoped use. Returns an empty tuple
-    for backends without an equivalent knob.
-    """
-    if kind == "low_latency":
-        table = _ANN_TUNING_LOW_LATENCY
-    elif kind == "high_recall":
-        table = _ANN_TUNING_HIGH_RECALL
-    else:
-        raise ValueError(f"Unknown ANN tuning kind: {kind!r}")
-    return table.get(_normalize_resolved(ext), ())
-
-
-def uses_per_bank_vector_indexes(ext: str) -> bool:
-    """Return whether the backend should create per-bank partial vector indexes."""
-    return _normalize_resolved(ext) != "scann"
-
-
-def bootstrap_extension(conn: Connection, ext: str) -> None:
-    """Install the configured vector extension and any prerequisites if possible."""
-    normalized = validate_extension(ext)
-    for statement in _EXTENSION_INSTALL_SQL[normalized]:
-        conn.execute(text(statement))
-
-
-def detect_vector_extension(conn: Connection, vector_extension: str = "pgvector") -> str:
-    """Validate the configured vector extension exists and return the index backend."""
-    configured_ext = validate_extension(vector_extension)
-
-    if configured_ext == "pgvectorscale":
-        pgvector_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vector'")).scalar()
-        if not pgvector_check:
-            raise RuntimeError(
-                "DiskANN (pgvectorscale/pg_diskann) requires pgvector to be installed. "
-                f"Install it with: {_INSTALL_HINTS['pgvectorscale']}"
-            )
-
-        vectorscale_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vectorscale'")).scalar()
-        pg_diskann_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'pg_diskann'")).scalar()
-
-        if vectorscale_check:
-            logger.debug("Using vector extension: pgvectorscale (DiskANN)")
-            return "pgvectorscale"
-        if pg_diskann_check:
-            logger.debug("Using vector extension: pg_diskann (Azure DiskANN)")
-            return "pg_diskann"
-
-        raise RuntimeError(
-            "Configured vector extension 'pgvectorscale' not found. Install either:\n"
-            "  - pgvectorscale (open source): CREATE EXTENSION vectorscale CASCADE;\n"
-            "  - pg_diskann (Azure): CREATE EXTENSION pg_diskann CASCADE;"
-        )
-
-    extension_name = pg_extension_name(configured_ext)
-    extension_check = conn.execute(
-        text("SELECT 1 FROM pg_extension WHERE extname = :extension_name"),
-        {"extension_name": extension_name},
-    ).scalar()
-    if not extension_check:
-        raise RuntimeError(
-            f"Configured vector extension '{configured_ext}' not found. "
-            f"Install it with: {_INSTALL_HINTS[configured_ext]}"
-        )
-
-    logger.debug("Using configured vector extension: %s", configured_ext)
-    return configured_ext

hindsight-api-slim/hindsight_api/admin/cli.py

@@ -1,7 +1,5 @@
-"""PostgreSQL-only admin utilities (backup, restore, migration, worker management).
-
-Not supported on Oracle backends. Uses asyncpg.connect() directly, binary COPY,
-TRUNCATE CASCADE, and REFRESH MATERIALIZED VIEW — all inherently PG-specific.
+"""
+Hindsight Admin CLI - backup and restore operations.
 """
 
 import asyncio
@@ -17,12 +15,15 @@ import asyncpg
 import typer
 
 from ..config import DEFAULT_DATABASE_SCHEMA, HindsightConfig
-from ..engine.memory_engine import _current_schema
-from ..engine.schema import fq_table_explicit as _fq_table
-from ..engine.transfer import export_bank
 from ..extensions import TenantExtension, load_extension
 from ..pg0 import parse_pg0_url, resolve_database_url
 
+
+def _fq_table(table: str, schema: str) -> str:
+    """Get fully-qualified table name with schema prefix."""
+    return f"{schema}.{table}"
+
+
 # Setup logging
 logging.basicConfig(
     level=logging.INFO,
@@ -32,17 +33,8 @@ logger = logging.getLogger(__name__)
 
 app = typer.Typer(name="hindsight-admin", help="Hindsight administrative commands")
 
-# Tables to backup/restore in foreign-key dependency order (parents first).
-# Restore COPYs in this order and TRUNCATEs in reverse, so every child must
-# appear after the tables it references.
-#
-# This must cover EVERY persistent PostgreSQL table in the schema — a missing
-# entry silently drops that table's data on restore (and, worse, restore's
-# `TRUNCATE banks CASCADE` wipes any FK-to-banks child like mental_models even
-# when it was never backed up). test_admin_backup_restore.py asserts this list
-# equals the live schema's tables, so adding a migration that creates a table
-# without adding it here fails CI. Oracle-only tables (e.g. observation_sources)
-# are intentionally absent — admin backup/restore is PostgreSQL-only.
+# Tables to backup/restore in dependency order
+# Import must happen in this order due to foreign key constraints
 BACKUP_TABLES = [
     "banks",
     "documents",
@@ -52,38 +44,11 @@ BACKUP_TABLES = [
     "unit_entities",
     "entity_cooccurrences",
     "memory_links",
-    "observation_history",
-    "mental_models",
-    "mental_model_history",
-    "directives",
-    "async_operations",
-    "webhooks",
-    "file_storage",
-    "audit_log",
-    "llm_requests",
-    "graph_maintenance_queue",
 ]
 
 MANIFEST_VERSION = "1"
 
 
-async def _admin_connect(db_url: str) -> asyncpg.Connection:
-    """Open a raw asyncpg connection to an admin DB URL.
-
-    ``resolve_database_url`` handles both plain ``postgres://`` (passthrough) and
-    ``pg0://`` (boots the embedded server and returns its real libpq URL), so this
-    is the only step needed to connect. JSON codecs are registered so ``jsonb``
-    columns decode to Python objects (used by the export row dumps).
-    """
-    is_pg0, instance_name, _ = parse_pg0_url(db_url)
-    if is_pg0:
-        typer.echo(f"Starting embedded PostgreSQL (instance: {instance_name})...")
-    conn = await asyncpg.connect(await resolve_database_url(db_url))
-    for type_name in ("json", "jsonb"):
-        await conn.set_type_codec(type_name, encoder=json.dumps, decoder=json.loads, schema="pg_catalog")
-    return conn
-
-
 async def _backup(database_url: str, output_path: Path, schema: str = "public") -> dict[str, Any]:
     """Backup all tables to a zip file using binary COPY protocol."""
     conn = await asyncpg.connect(database_url)
@@ -306,7 +271,6 @@ async def _run_migration(
         ensure_text_search_extension(
             resolved_url,
             text_search_extension=config.text_search_extension,
-            pg_search_tokenizer=config.text_search_extension_pg_search_tokenizer,
             schema=schema,
         )
 
@@ -352,123 +316,6 @@ def run_db_migration(
     typer.echo(f"Database migrations completed successfully for {len(schemas)} schema(s)")
 
 
-async def _run_export_bank(db_url: str, bank_id: str, output: Path, schema: str, include_history: bool) -> int:
-    """Export a whole bank to a ZIP archive."""
-    conn = await _admin_connect(db_url)
-    try:
-        # export_bank resolves table names via fq_table (the _current_schema
-        # contextvar); set it so the raw connection targets the right schema.
-        _current_schema.set(schema)
-        data = await export_bank(conn, bank_id, include_history=include_history)
-    finally:
-        await conn.close()
-
-    output.write_bytes(data)
-    return len(data)
-
-
-@app.command(name="export-bank")
-def export_bank_command(
-    bank_id: str = typer.Option(..., "--bank", "-b", help="Bank id to export."),
-    output: Path = typer.Option(..., "--output", "-o", help="Path to write the .zip archive."),
-    schema: str | None = typer.Option(
-        None,
-        "--schema",
-        "-s",
-        help="Database schema the bank lives in. Defaults to the configured base schema.",
-    ),
-    include_history: bool = typer.Option(
-        False,
-        "--include-history",
-        help="Also export operational history (audit_log, llm_requests). Off by default.",
-    ),
-):
-    """Export an entire bank to a portable ZIP (no embeddings — regenerated on import).
-
-    Carries documents, facts, observations, bank config, mental models, directives
-    and webhooks so the bank can be imported into a new instance configured with a
-    different embedding model / vector / text-search backend.
-    """
-    config = HindsightConfig.from_env()
-
-    if not config.database_url:
-        typer.echo("Error: Database URL not configured.", err=True)
-        typer.echo("Set HINDSIGHT_API_DATABASE_URL environment variable.", err=True)
-        raise typer.Exit(1)
-
-    target_schema = schema or config.database_schema or DEFAULT_DATABASE_SCHEMA
-    typer.echo(f"Exporting bank '{bank_id}' from schema '{target_schema}'...")
-
-    size = asyncio.run(_run_export_bank(config.database_url, bank_id, output, target_schema, include_history))
-
-    typer.echo(f"Exported bank '{bank_id}' to {output} ({size} bytes)")
-
-
-async def _run_import_bank(archive_path: Path, schema: str, target_bank_id: str | None, include_history: bool):
-    """Boot a MemoryEngine (for the target's embedding model) and restore a bank archive."""
-    # MemoryEngine is heavy (loads embeddings); import it lazily so other admin
-    # commands don't pay for it. _current_schema is imported at module top.
-    from ..engine.memory_engine import MemoryEngine
-    from ..models import RequestContext
-
-    archive_bytes = archive_path.read_bytes()
-    # run_migrations=True so a fresh target instance is provisioned at this
-    # instance's embedding dimension / vector / text-search backend before restore.
-    engine = MemoryEngine(run_migrations=True)
-    await engine.initialize()
-    try:
-        _current_schema.set(schema)
-        context = RequestContext(internal=True, user_initiated=True)
-        return await engine.import_bank_async(
-            archive_bytes,
-            context,
-            target_bank_id=target_bank_id,
-            include_history=include_history,
-        )
-    finally:
-        await engine.close()
-
-
-@app.command(name="import-bank")
-def import_bank_command(
-    archive: Path = typer.Option(..., "--archive", "-a", help="Path to the .zip produced by export-bank."),
-    schema: str | None = typer.Option(
-        None, "--schema", "-s", help="Target schema. Defaults to the configured base schema."
-    ),
-    target_bank: str | None = typer.Option(
-        None, "--target-bank", help="Override the bank id (defaults to the archive's source bank)."
-    ),
-    include_history: bool = typer.Option(
-        False, "--include-history", help="Also restore operational history if present in the archive."
-    ),
-):
-    """Restore a whole bank from an export-bank archive into THIS instance.
-
-    Re-embeds facts with this instance's configured embedding model and rebuilds
-    links and indexes — the import half of a cross-instance migration. Run against
-    an instance configured with the desired embedding / vector / text-search backend.
-    The target bank must not already exist (import restores a whole bank, not a merge).
-    """
-    config = HindsightConfig.from_env()
-    if not config.database_url:
-        typer.echo("Error: Database URL not configured.", err=True)
-        typer.echo("Set HINDSIGHT_API_DATABASE_URL environment variable.", err=True)
-        raise typer.Exit(1)
-
-    target_schema = schema or config.database_schema or DEFAULT_DATABASE_SCHEMA
-    typer.echo(f"Importing bank archive '{archive}' into schema '{target_schema}'...")
-
-    result = asyncio.run(_run_import_bank(archive, target_schema, target_bank, include_history))
-
-    typer.echo(
-        f"Imported bank '{result.bank_id}': {result.documents_imported} doc(s), "
-        f"{result.facts_imported} fact(s), {result.observations_imported} observation(s), "
-        f"{result.mental_models_imported} mental model(s), "
-        f"{result.mental_model_history_imported} mm-history row(s), {result.directives_imported} directive(s), "
-        f"{result.webhooks_imported} webhook(s), {result.history_rows_imported} history row(s)"
-    )
-
-
 async def _decommission_worker(db_url: str, worker_id: str, schema: str = "public") -> int:
     """Release all tasks owned by a worker, setting them back to pending status."""
     is_pg0, instance_name, _ = parse_pg0_url(db_url)
@@ -528,140 +375,6 @@ def decommission_worker(
         typer.echo(f"No tasks found for worker '{worker_id}'")
 
 
-async def _decommission_all_workers(db_url: str, schema: str = "public") -> list[dict[str, Any]]:
-    """Release all processing tasks from all workers, setting them back to pending status."""
-    is_pg0, instance_name, _ = parse_pg0_url(db_url)
-    if is_pg0:
-        typer.echo(f"Starting embedded PostgreSQL (instance: {instance_name})...")
-    resolved_url = await resolve_database_url(db_url)
-
-    conn = await asyncpg.connect(resolved_url)
-    try:
-        table = _fq_table("async_operations", schema)
-        rows = await conn.fetch(
-            f"""
-            UPDATE {table}
-            SET status = 'pending', worker_id = NULL, claimed_at = NULL, updated_at = now()
-            WHERE status = 'processing'
-            RETURNING operation_id, worker_id, operation_type
-            """,
-        )
-        return [dict(r) for r in rows]
-    finally:
-        await conn.close()
-
-
-@app.command(name="decommission-workers")
-def decommission_workers(
-    schema: str = typer.Option("public", "--schema", "-s", help="Database schema"),
-    yes: bool = typer.Option(False, "--yes", "-y", help="Skip confirmation prompt"),
-):
-    """Release all processing tasks from all workers (sets status back to pending).
-
-    Use this command to recover from situations where one or more workers have crashed
-    or been removed without graceful shutdown. All tasks currently in 'processing' status
-    will be released back to the queue regardless of which worker owns them.
-    """
-    config = HindsightConfig.from_env()
-
-    if not config.database_url:
-        typer.echo("Error: Database URL not configured.", err=True)
-        typer.echo("Set HINDSIGHT_API_DATABASE_URL environment variable.", err=True)
-        raise typer.Exit(1)
-
-    if not yes:
-        typer.confirm(
-            "This will release ALL processing tasks from ALL workers back to pending. Continue?",
-            abort=True,
-        )
-
-    typer.echo(f"Decommissioning all workers (schema: {schema})...")
-
-    released = asyncio.run(_decommission_all_workers(config.database_url, schema))
-
-    if released:
-        # Group by worker_id for summary
-        by_worker: dict[str, int] = {}
-        for row in released:
-            wid = row["worker_id"] or "unknown"
-            by_worker[wid] = by_worker.get(wid, 0) + 1
-
-        typer.echo(f"Released {len(released)} task(s):")
-        for wid, count in by_worker.items():
-            typer.echo(f"  {wid}: {count} task(s)")
-    else:
-        typer.echo("No processing tasks found")
-
-
-async def _worker_status(db_url: str, schema: str = "public") -> list[dict[str, Any]]:
-    """Get all processing tasks grouped by worker with their last update time."""
-    is_pg0, instance_name, _ = parse_pg0_url(db_url)
-    if is_pg0:
-        typer.echo(f"Starting embedded PostgreSQL (instance: {instance_name})...")
-    resolved_url = await resolve_database_url(db_url)
-
-    conn = await asyncpg.connect(resolved_url)
-    try:
-        table = _fq_table("async_operations", schema)
-        rows = await conn.fetch(
-            f"""
-            SELECT worker_id, operation_id, operation_type, bank_id,
-                   claimed_at, updated_at,
-                   now() - claimed_at AS running_for,
-                   now() - updated_at AS last_update_ago
-            FROM {table}
-            WHERE status = 'processing'
-            ORDER BY worker_id, claimed_at
-            """,
-        )
-        return [dict(r) for r in rows]
-    finally:
-        await conn.close()
-
-
-@app.command(name="worker-status")
-def worker_status(
-    schema: str = typer.Option("public", "--schema", "-s", help="Database schema"),
-):
-    """Show all currently processing tasks grouped by worker.
-
-    Displays each worker's active tasks with operation type, bank, how long
-    the task has been running, and when it was last updated. Useful for
-    identifying dead workers with orphaned tasks.
-    """
-    config = HindsightConfig.from_env()
-
-    if not config.database_url:
-        typer.echo("Error: Database URL not configured.", err=True)
-        typer.echo("Set HINDSIGHT_API_DATABASE_URL environment variable.", err=True)
-        raise typer.Exit(1)
-
-    rows = asyncio.run(_worker_status(config.database_url, schema))
-
-    if not rows:
-        typer.echo("No processing tasks found")
-        return
-
-    # Group by worker_id
-    by_worker: dict[str, list[dict[str, Any]]] = {}
-    for row in rows:
-        wid = row["worker_id"] or "unknown"
-        by_worker.setdefault(wid, []).append(row)
-
-    typer.echo(f"Processing tasks across {len(by_worker)} worker(s):\n")
-    for wid, tasks in by_worker.items():
-        typer.echo(f"Worker: {wid} ({len(tasks)} task(s))")
-        for task in tasks:
-            op_id = str(task["operation_id"])[:8]
-            running_for = task["running_for"]
-            last_update = task["last_update_ago"]
-            typer.echo(
-                f"  {op_id}  {task['operation_type']:<20s} bank={task['bank_id']}"
-                f"  running={running_for}  last_update={last_update} ago"
-            )
-        typer.echo("")
-
-
 def main():
     app()
 

hindsight-api-slim/hindsight_api/alembic/_dialect.py

@@ -1,42 +0,0 @@
-"""Dialect dispatcher for Alembic migrations.
-
-Each migration file declares a ``_pg_upgrade``/``_oracle_upgrade`` (and matching
-downgrades) function and routes ``upgrade()``/``downgrade()`` through
-``run_for_dialect``. The helper inspects the live connection's dialect name and
-runs the matching function — or no-ops if the migration doesn't apply to the
-current backend.
-
-Use ``None`` (or omit the kwarg) when a migration intentionally has no effect
-on a dialect; the helper treats it as a no-op.
-"""
-
-from __future__ import annotations
-
-from collections.abc import Callable
-
-from alembic import op
-
-DialectFn = Callable[[], None]
-_SUPPORTED = ("postgresql", "oracle")
-
-
-def run_for_dialect(
-    *,
-    pg: DialectFn | None = None,
-    oracle: DialectFn | None = None,
-) -> None:
-    """Dispatch to the function matching the current bind's dialect.
-
-    Args:
-        pg: Function to run when the active bind is PostgreSQL.
-        oracle: Function to run when the active bind is Oracle.
-
-    Unrecognized dialects raise; an explicit ``None`` for the active dialect
-    is a no-op (the migration deliberately does nothing here).
-    """
-    name = op.get_bind().dialect.name
-    if name not in _SUPPORTED:
-        raise RuntimeError(f"Unsupported dialect for migration dispatch: {name!r}. Expected one of {_SUPPORTED}.")
-    fn = {"postgresql": pg, "oracle": oracle}[name]
-    if fn is not None:
-        fn()

hindsight-api-slim/hindsight_api/alembic/env.py

@@ -1,37 +1,28 @@
 """
-Alembic environment for Hindsight.
-
-Supports two dialects:
-
-* PostgreSQL (sync psycopg2 driver) — default; uses ``search_path`` for
-  multi-tenant schema isolation and forces read-write transactions to work
-  around Supabase's read-only-by-default sessions.
-* Oracle 23ai (``oracledb`` driver) — uses ``CURRENT_SCHEMA`` for tenant
-  isolation; no equivalent of ``search_path`` or read-only session quirks.
-
-Each migration file dispatches its DDL through ``alembic._dialect.run_for_dialect``
-so a single revision tree serves both backends.
+Alembic environment configuration for SQLAlchemy with pgvector.
+Uses synchronous psycopg2 driver for migrations to avoid pgbouncer issues.
 """
 
 import logging
 import os
 from pathlib import Path
-from urllib.parse import parse_qsl, urlencode, urlsplit, urlunsplit
 
 from alembic import context
 from dotenv import load_dotenv
-from sqlalchemy import Connection, engine_from_config, pool
-from sqlalchemy.engine import Engine
+from sqlalchemy import engine_from_config, pool
 
-from hindsight_api.db_url import is_oracle_url, to_libpq_url
+# Import your models here
 from hindsight_api.models import Base
 
 
-def load_env() -> None:
-    """Load environment variables from .env (skipped if already configured)."""
+# Load environment variables based on HINDSIGHT_API_DATABASE_URL env var or default to local
+def load_env():
+    """Load environment variables from .env"""
+    # Check if HINDSIGHT_API_DATABASE_URL is already set (e.g., by CI/CD)
     if os.getenv("HINDSIGHT_API_DATABASE_URL"):
         return
 
+    # Look for .env file in the parent directory (root of the workspace)
     root_dir = Path(__file__).parent.parent.parent
     env_file = root_dir / ".env"
 
@@ -41,45 +32,30 @@ def load_env() -> None:
 
 load_env()
 
+# this is the Alembic Config object, which provides
+# access to the values within the .ini file in use.
 config = context.config
+
+# Note: We don't call fileConfig() here to avoid overriding the application's logging configuration.
+# Alembic will use the existing logging configuration from the application.
+
+# add your model's MetaData object here
+# for 'autogenerate' support
 target_metadata = Base.metadata
 
-
-def _normalize_oracle_url(url: str) -> str:
-    """Coerce an Oracle URL into the SQLAlchemy form the oracledb dialect expects.
-
-    Two issues to handle:
-
-    1. Force the ``oracle+oracledb`` driver — bare ``oracle://`` defaults to
-       cx_Oracle.
-    2. Map a path-style service to ``?service_name=...``. SQLAlchemy's oracledb
-       dialect treats the URL path as a *SID* (legacy), but Oracle Free /
-       Autonomous DB only register a service name. Without this rewrite we get
-       ``DPY-6003: SID "FREEPDB1" is not registered`` even though the listener
-       is happy to accept the same name as a service.
-    """
-    parts = urlsplit(url)
-    if not parts.scheme.startswith("oracle"):
-        return url
-
-    new_scheme = "oracle+oracledb" if parts.scheme == "oracle" else parts.scheme
-    service = parts.path.lstrip("/")
-    new_query = parts.query
-    new_path = parts.path
-
-    # Promote /SERVICE to ?service_name=SERVICE unless the caller already
-    # supplied an explicit ?sid= or ?service_name=.
-    if service and "service_name=" not in new_query and "sid=" not in new_query:
-        params = [(k, v) for k, v in parse_qsl(new_query, keep_blank_values=True)]
-        params.append(("service_name", service))
-        new_query = urlencode(params)
-        new_path = ""
-
-    return urlunsplit((new_scheme, parts.netloc, new_path, new_query, parts.fragment))
+# other values from the config, defined by the needs of env.py,
+# can be acquired:
+# my_important_option = config.get_main_option("my_important_option")
+# ... etc.
 
 
 def get_database_url() -> str:
-    """Resolve the migration URL from Alembic config or env, normalizing per-dialect."""
+    """
+    Get and process the database URL from config or environment.
+
+    Returns the URL with the correct driver (psycopg2) for migrations.
+    """
+    # Get database URL from config (set programmatically) or environment
     database_url = config.get_main_option("sqlalchemy.url")
     if not database_url:
         database_url = os.getenv("HINDSIGHT_API_DATABASE_URL")
@@ -89,18 +65,30 @@ def get_database_url() -> str:
                 "Set HINDSIGHT_API_DATABASE_URL environment variable or pass database_url to run_migrations()."
             )
 
-    if is_oracle_url(database_url):
-        database_url = _normalize_oracle_url(database_url)
-    else:
-        # PG: convert SQLAlchemy-style asyncpg URLs and ?ssl= params to libpq form
-        # for the sync engine used during migrations.
-        database_url = to_libpq_url(database_url)
+    # For migrations, use psycopg2 (sync driver) to avoid pgbouncer prepared statement issues
+    if database_url.startswith("postgresql+asyncpg://"):
+        database_url = database_url.replace("postgresql+asyncpg://", "postgresql://", 1)
+    elif database_url.startswith("postgres+asyncpg://"):
+        database_url = database_url.replace("postgres+asyncpg://", "postgresql://", 1)
 
+    # Update config with processed URL for engine_from_config to use
     config.set_main_option("sqlalchemy.url", database_url)
+
     return database_url
 
 
 def run_migrations_offline() -> None:
+    """Run migrations in 'offline' mode.
+
+    This configures the context with just a URL
+    and not an Engine, though an Engine is acceptable
+    here as well.  By skipping the Engine creation
+    we don't even need a DBAPI to be available.
+
+    Calls to context.execute() here emit the given string to the
+    script output.
+
+    """
     logging.info("running offline")
     database_url = get_database_url()
 
@@ -115,40 +103,14 @@ def run_migrations_offline() -> None:
         context.run_migrations()
 
 
-def _configure_pg_session(engine: Engine, connection: Connection, target_schema: str | None) -> None:
-    """PG-only: ensure the session is RW (Supabase) and bind ``search_path``."""
+def run_migrations_online() -> None:
+    """Run migrations in 'online' mode with synchronous engine."""
     from sqlalchemy import event, text
 
-    @event.listens_for(engine, "connect")
-    def set_read_write_mode(dbapi_connection, connection_record):
-        cursor = dbapi_connection.cursor()
-        cursor.execute("SET SESSION CHARACTERISTICS AS TRANSACTION READ WRITE")
-        if target_schema:
-            cursor.execute(f'CREATE SCHEMA IF NOT EXISTS "{target_schema}"')
-            cursor.execute(f'SET search_path TO "{target_schema}", public')
-        cursor.close()
+    get_database_url()  # Process and set the database URL in config
 
-    connection.execute(text("SET SESSION CHARACTERISTICS AS TRANSACTION READ WRITE"))
-    if target_schema:
-        connection.execute(text(f'CREATE SCHEMA IF NOT EXISTS "{target_schema}"'))
-        connection.execute(text(f'SET search_path TO "{target_schema}", public'))
-    connection.commit()
-
-
-def _configure_oracle_session(connection: Connection, target_schema: str | None) -> None:
-    """Oracle: switch the session's default schema; tolerate DDL contention."""
-    from sqlalchemy import text
-
-    # Wait up to 30s for DDL locks instead of failing immediately (ORA-00054).
-    connection.execute(text("ALTER SESSION SET DDL_LOCK_TIMEOUT = 30"))
-    if target_schema:
-        connection.execute(text(f'ALTER SESSION SET CURRENT_SCHEMA = "{target_schema}"'))
-
-
-def run_migrations_online() -> None:
-    database_url = get_database_url()
+    # Check if we're targeting a specific schema (for multi-tenant isolation)
     target_schema = config.get_main_option("target_schema")
-    is_oracle = is_oracle_url(database_url)
 
     connectable = engine_from_config(
         config.get_section(config.config_ini_section, {}),
@@ -156,19 +118,37 @@ def run_migrations_online() -> None:
         poolclass=pool.NullPool,
     )
 
-    with connectable.connect() as connection:
-        if is_oracle:
-            _configure_oracle_session(connection, target_schema)
-        else:
-            _configure_pg_session(connectable, connection, target_schema)
+    # Add event listener to ensure connection is in read-write mode
+    # This is needed for Supabase which may start connections in read-only mode
+    @event.listens_for(connectable, "connect")
+    def set_read_write_mode(dbapi_connection, connection_record):
+        cursor = dbapi_connection.cursor()
+        cursor.execute("SET SESSION CHARACTERISTICS AS TRANSACTION READ WRITE")
+        # If targeting a specific schema, set search_path
+        # Include public in search_path for access to shared extensions (pgvector)
+        if target_schema:
+            cursor.execute(f'CREATE SCHEMA IF NOT EXISTS "{target_schema}"')
+            cursor.execute(f'SET search_path TO "{target_schema}", public')
+        cursor.close()
 
+    with connectable.connect() as connection:
+        # Also explicitly set read-write mode on this connection
+        connection.execute(text("SET SESSION CHARACTERISTICS AS TRANSACTION READ WRITE"))
+
+        # If targeting a specific schema, set search_path
+        # Include public in search_path for access to shared extensions (pgvector)
+        if target_schema:
+            connection.execute(text(f'CREATE SCHEMA IF NOT EXISTS "{target_schema}"'))
+            connection.execute(text(f'SET search_path TO "{target_schema}", public'))
+
+        connection.commit()  # Commit the SET command
+
+        # Configure context with version_table_schema if using a specific schema
         context_opts = {
             "connection": connection,
             "target_metadata": target_metadata,
         }
-        if target_schema and not is_oracle:
-            # Oracle has no equivalent of PG's per-schema version table; the
-            # ``alembic_version`` table lives in CURRENT_SCHEMA implicitly.
+        if target_schema:
             context_opts["version_table_schema"] = target_schema
 
         context.configure(**context_opts)
@@ -176,12 +156,7 @@ def run_migrations_online() -> None:
         with context.begin_transaction():
             context.run_migrations()
 
-        # Always commit. PG needs it for the explicit RW-mode SET to persist;
-        # Oracle needs it because each DDL auto-commits but the trailing
-        # ``UPDATE alembic_version`` is plain DML that would otherwise stay in
-        # an open transaction and roll back when the connection closes —
-        # producing the "schema is created but the version row is one revision
-        # behind" failure mode.
+        # Explicit commit to ensure changes are persisted (especially for Supabase)
         connection.commit()
 
 

hindsight-api-slim/hindsight_api/alembic/script.py.mako

@@ -11,8 +11,6 @@ from alembic import op
 import sqlalchemy as sa
 ${imports if imports else ""}
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 # revision identifiers, used by Alembic.
 revision: str = ${repr(up_revision)}
 down_revision: Union[str, Sequence[str], None] = ${repr(down_revision)}
@@ -20,27 +18,11 @@ branch_labels: Union[str, Sequence[str], None] = ${repr(branch_labels)}
 depends_on: Union[str, Sequence[str], None] = ${repr(depends_on)}
 
 
-def _pg_upgrade() -> None:
-    """PostgreSQL upgrade. Set to ``None`` below if this migration is Oracle-only."""
+def upgrade() -> None:
+    """Upgrade schema."""
     ${upgrades if upgrades else "pass"}
 
 
-def _pg_downgrade() -> None:
-    ${downgrades if downgrades else "pass"}
-
-
-def _oracle_upgrade() -> None:
-    """Oracle upgrade. Set to ``None`` below if this migration is Postgres-only."""
-    pass
-
-
-def _oracle_downgrade() -> None:
-    pass
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade, oracle=_oracle_upgrade)
-
-
 def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade, oracle=_oracle_downgrade)
+    """Downgrade schema."""
+    ${downgrades if downgrades else "pass"}

hindsight-api-slim/hindsight_api/alembic/versions/2eee35aa3cfc_case_insensitive_entities_trgm_index.py

@@ -4,8 +4,8 @@ The previous GIN trigram index on canonical_name was case-sensitive, causing
 "Alice" and "alice" to have different trigram sets. This recreates it on
 LOWER(canonical_name) so the % operator matches case-insensitively.
 
-Revision ID: 2eee35aa3cfc
-Revises: d6e7f8a9b0c1
+Revision ID: d6e7f8a9b0c1
+Revises: c5d6e7f8a9b0
 Create Date: 2026-03-31
 """
 
@@ -13,10 +13,8 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "2eee35aa3cfc"
-down_revision: str | Sequence[str] | None = "d6e7f8a9b0c1"
+revision: str = "d6e7f8a9b0c1"
+down_revision: str | Sequence[str] | None = "c5d6e7f8a9b0"
 branch_labels: str | Sequence[str] | None = None
 depends_on: str | Sequence[str] | None = None
 
@@ -26,7 +24,7 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     schema = _get_schema_prefix()
     # Drop the old case-sensitive trigram index
     op.execute("DROP INDEX IF EXISTS entities_canonical_name_trgm_idx")
@@ -37,7 +35,7 @@ def _pg_upgrade() -> None:
     )
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     op.execute("DROP INDEX IF EXISTS entities_canonical_name_lower_trgm_idx")
     schema = _get_schema_prefix()
     # Restore original case-sensitive index
@@ -45,11 +43,3 @@ def _pg_downgrade() -> None:
         f"CREATE INDEX IF NOT EXISTS entities_canonical_name_trgm_idx "
         f"ON {schema}entities USING GIN (canonical_name gin_trgm_ops)"
     )
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/5a366d414dce_initial_schema.py

@@ -15,13 +15,6 @@ from pgvector.sqlalchemy import Vector
 from sqlalchemy import text
 from sqlalchemy.dialects import postgresql
 
-from hindsight_api._pg_search import (
-    PG_SEARCH_TOKENIZER_ENV,
-    normalize_pg_search_tokenizer,
-    pg_search_bm25_columns,
-)
-from hindsight_api.alembic._dialect import run_for_dialect
-
 # revision identifiers, used by Alembic.
 revision: str = "5a366d414dce"
 down_revision: str | Sequence[str] | None = None
@@ -31,79 +24,59 @@ depends_on: str | Sequence[str] | None = None
 
 def _detect_vector_extension() -> str:
     """
-    Detect or validate vector extension for this immutable migration revision.
+    Detect or validate vector extension: 'pgvector', 'vchord', or 'pgvectorscale'.
     Respects HINDSIGHT_API_VECTOR_EXTENSION env var if set.
     """
     conn = op.get_bind()
     vector_extension = os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector").lower()
 
+    # Validate configured extension is installed
     if vector_extension == "pgvectorscale":
+        # pgvectorscale/DiskANN requires pgvector
         pgvector_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vector'")).scalar()
         if not pgvector_check:
             raise RuntimeError(
                 "DiskANN requires pgvector. Install with: CREATE EXTENSION vector; then vectorscale or pg_diskann CASCADE;"
             )
+        # Check for either vectorscale (open source) or pg_diskann (Azure)
         vectorscale_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vectorscale'")).scalar()
         pg_diskann_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'pg_diskann'")).scalar()
 
         if vectorscale_check:
             return "pgvectorscale"
-        if pg_diskann_check:
+        elif pg_diskann_check:
             return "pg_diskann"
-        raise RuntimeError(
-            "Configured vector extension 'pgvectorscale' not found. Install either:\n"
-            "  - pgvectorscale: CREATE EXTENSION vectorscale CASCADE;\n"
-            "  - pg_diskann (Azure): CREATE EXTENSION pg_diskann CASCADE;"
-        )
-    if vector_extension == "vchord":
+        else:
+            raise RuntimeError(
+                "Configured vector extension 'pgvectorscale' not found. Install either:\n"
+                "  - pgvectorscale: CREATE EXTENSION vectorscale CASCADE;\n"
+                "  - pg_diskann (Azure): CREATE EXTENSION pg_diskann CASCADE;"
+            )
+    elif vector_extension == "vchord":
         vchord_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vchord'")).scalar()
         if not vchord_check:
             raise RuntimeError(
                 "Configured vector extension 'vchord' not found. Install it with: CREATE EXTENSION vchord CASCADE;"
             )
         return "vchord"
-    if vector_extension == "scann":
-        scann_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'alloydb_scann'")).scalar()
-        if not scann_check:
-            raise RuntimeError(
-                "Configured vector extension 'scann' not found. Install it with: CREATE EXTENSION alloydb_scann CASCADE;"
-            )
-        return "scann"
-    if vector_extension == "pgvector":
+    elif vector_extension == "pgvector":
         pgvector_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vector'")).scalar()
         if not pgvector_check:
             raise RuntimeError(
                 "Configured vector extension 'pgvector' not found. Install it with: CREATE EXTENSION vector;"
             )
         return "pgvector"
-    raise ValueError(
-        "Invalid HINDSIGHT_API_VECTOR_EXTENSION: "
-        f"{vector_extension}. Must be 'pgvector', 'vchord', 'pgvectorscale', or 'scann'"
-    )
-
-
-def _vector_index_using_clause(ext: str) -> str:
-    if ext == "pgvectorscale":
-        return "USING diskann (embedding vector_cosine_ops) WITH (num_neighbors = 50)"
-    if ext == "pg_diskann":
-        return "USING diskann (embedding vector_cosine_ops) WITH (max_neighbors = 50)"
-    if ext == "vchord":
-        return "USING vchordrq (embedding vector_cosine_ops)"
-    if ext == "scann":
-        return "USING scann (embedding cosine) WITH (mode = 'AUTO')"
-    return "USING hnsw (embedding vector_cosine_ops)"
+    else:
+        raise ValueError(
+            f"Invalid HINDSIGHT_API_VECTOR_EXTENSION: {vector_extension}. Must be 'pgvector', 'vchord', or 'pgvectorscale'"
+        )
 
 
 def _detect_text_search_extension() -> str:
     """
-    Detect or validate text search extension: 'native', 'vchord', 'pg_textsearch',
-    'pgroonga', or 'pg_search'. Respects HINDSIGHT_API_TEXT_SEARCH_EXTENSION env var.
+    Detect or validate text search extension: 'native', 'vchord', or 'pg_textsearch'.
+    Respects HINDSIGHT_API_TEXT_SEARCH_EXTENSION env var.
     Creates the extension if needed.
-
-    pgroonga is treated as native here so the initial schema still creates valid
-    tsvector columns. ensure_text_search_extension() at startup converts the
-    schema to pgroonga structures (drops the tsvector column, builds a pgroonga
-    index on the base text column).
     """
     text_search_extension = os.getenv("HINDSIGHT_API_TEXT_SEARCH_EXTENSION", "native").lower()
 
@@ -131,36 +104,15 @@ def _detect_text_search_extension() -> str:
                 # Extension truly doesn't exist - re-raise the error
                 raise
         return "pg_textsearch"
-    elif text_search_extension == "pg_search":
-        # ParadeDB pg_search — true BM25 over base columns, Citus-compatible.
-        try:
-            op.execute("CREATE EXTENSION IF NOT EXISTS pg_search CASCADE")
-        except Exception:
-            # Extension might already exist or user lacks permissions - verify it exists
-            conn = op.get_bind()
-            result = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'pg_search'")).fetchone()
-            if not result:
-                # Extension truly doesn't exist - re-raise the error
-                raise
-        return "pg_search"
     elif text_search_extension == "native":
         return "native"
-    elif text_search_extension == "pgroonga":
-        # ensure_text_search_extension() at runtime converts to pgroonga.
-        # Treat as native here so the initial schema still creates valid columns.
-        return "native"
     else:
         raise ValueError(
-            f"Invalid HINDSIGHT_API_TEXT_SEARCH_EXTENSION: {text_search_extension}. "
-            "Must be 'native', 'vchord', 'pg_textsearch', 'pgroonga', or 'pg_search'"
+            f"Invalid HINDSIGHT_API_TEXT_SEARCH_EXTENSION: {text_search_extension}. Must be 'native', 'vchord', or 'pg_textsearch'"
         )
 
 
-def _pg_search_tokenizer() -> str:
-    return normalize_pg_search_tokenizer(os.getenv(PG_SEARCH_TOKENIZER_ENV))
-
-
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     """Upgrade schema - create all tables from scratch."""
 
     # Note: pgvector extension is installed globally BEFORE migrations run
@@ -315,9 +267,8 @@ def _pg_upgrade() -> None:
             ALTER TABLE memory_units
             ADD COLUMN search_vector bm25_catalog.bm25vector
         """)
-    elif text_search_ext in ("pg_textsearch", "pg_search"):
-        # Timescale pg_textsearch / ParadeDB pg_search: dummy TEXT column for
-        # consistency (indexes operate on base columns directly).
+    elif text_search_ext == "pg_textsearch":
+        # Timescale pg_textsearch: dummy TEXT column for consistency (indexes operate on base columns directly)
         op.execute("""
             ALTER TABLE memory_units
             ADD COLUMN search_vector TEXT
@@ -360,11 +311,36 @@ def _pg_upgrade() -> None:
     )
     # Create vector index - conditional based on available extension
     vector_ext = _detect_vector_extension()
-    if vector_ext != "scann":
-        op.execute(f"""
+
+    if vector_ext == "pgvectorscale":
+        # Use DiskANN index for pgvectorscale (disk-based, scalable)
+        op.execute("""
             CREATE INDEX idx_memory_units_embedding ON memory_units
-            {_vector_index_using_clause(vector_ext)}
+            USING diskann (embedding vector_cosine_ops)
+            WITH (num_neighbors = 50)
         """)
+    elif vector_ext == "pg_diskann":
+        # Use DiskANN index for pg_diskann (Azure)
+        op.execute("""
+            CREATE INDEX idx_memory_units_embedding ON memory_units
+            USING diskann (embedding vector_cosine_ops)
+            WITH (max_neighbors = 50)
+        """)
+    elif vector_ext == "vchord":
+        # Use vchordrq index for vchord (supports high-dimensional embeddings)
+        op.execute("""
+            CREATE INDEX idx_memory_units_embedding ON memory_units
+            USING vchordrq (embedding vector_l2_ops)
+        """)
+    else:  # pgvector
+        # Use HNSW index for pgvector
+        op.create_index(
+            "idx_memory_units_embedding",
+            "memory_units",
+            ["embedding"],
+            postgresql_using="hnsw",
+            postgresql_ops={"embedding": "vector_cosine_ops"},
+        )
 
     # Create full-text search index on search_vector
     # Index type depends on text search backend
@@ -382,17 +358,6 @@ def _pg_upgrade() -> None:
             USING bm25(text)
             WITH (text_config='english')
         """)
-    elif text_search_ext == "pg_search":
-        # ParadeDB pg_search BM25 index on (id, text, context). The key_field
-        # reloption is required and must match the table's primary key column.
-        bm25_cols = pg_search_bm25_columns("id", ("text", "context"), _pg_search_tokenizer())
-        op.execute(
-            """
-            CREATE INDEX idx_memory_units_text_search ON memory_units
-            USING bm25 ({bm25_cols})
-            WITH (key_field='id')
-        """.format(bm25_cols=bm25_cols)
-        )
     else:  # native
         # Native PostgreSQL GIN index
         op.execute("""
@@ -498,7 +463,7 @@ def _pg_upgrade() -> None:
     op.create_index("idx_unit_entities_entity", "unit_entities", ["entity_id"])
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     """Downgrade schema - drop all tables."""
 
     # Drop tables in reverse dependency order
@@ -558,11 +523,3 @@ def _pg_downgrade() -> None:
     # Drop extensions (optional - comment out if you want to keep them)
     # op.execute('DROP EXTENSION IF EXISTS vector')
     # op.execute('DROP EXTENSION IF EXISTS "uuid-ossp"')
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/86f7a033d372_repair_mental_models_subtype_at_head.py

@@ -1,117 +0,0 @@
-"""Repair mental_models.subtype on databases stuck at m3rg3h3ad5f6
-
-Three production deployments reported `column "subtype" of relation
-"mental_models" does not exist` on `create_mental_model` even after their
-container reported `Database migrations completed successfully` and
-`alembic_version` advanced to `m3rg3h3ad5f6` (see issue #1553, #1553#1
-confirmations from @4Lienau and @khanhduyvt0101).
-
-Both `h3c4d5e6f7g8_mental_models_v4` and `d5y6z7a8b9c0_backfill_mental_models_subtype`
-were meant to ensure `subtype` exists, but on databases that came through the
-`reflections -> mental_models` rename chain *and* whose alembic_version
-advanced past `d5y6z7a8b9c0` along an alternate path during the divergent-heads
-reorganization, neither column-add actually fired. The result is a head-tagged
-database with a v3-shaped `mental_models` table missing six columns:
-``subtype``, ``description``, ``entity_id``, ``observations``, ``links``,
-``last_updated``.
-
-This migration sits at the current head (`m3rg3h3ad5f6`) so every affected
-deployment will pick it up on next container start. It mirrors the column-add
-block from `d5y6z7a8b9c0_backfill_mental_models_subtype` using
-``ADD COLUMN IF NOT EXISTS`` so it is a no-op on databases where the columns
-are already present.
-
-Revision ID: 86f7a033d372
-Revises: m3rg3h3ad5f6
-Create Date: 2026-05-14
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "86f7a033d372"
-down_revision: str | Sequence[str] | None = "m3rg3h3ad5f6"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _pg_schema_prefix() -> str:
-    """Schema-qualifier for raw SQL on PG (multi-tenant search_path)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _pg_upgrade() -> None:
-    """Idempotently ensure mental_models has the v4 column set.
-
-    Safe to re-apply on databases that already received the columns via
-    `h3c4d5e6f7g8_mental_models_v4` or `d5y6z7a8b9c0_backfill_mental_models_subtype` —
-    every column-add uses ``IF NOT EXISTS`` and the constraint is recreated
-    from scratch with the canonical v4 allowlist.
-    """
-    schema = _pg_schema_prefix()
-    bare_schema = schema.strip(".").strip('"') if schema else ""
-    schema_clause = f"AND table_schema = '{bare_schema}'" if bare_schema else ""
-
-    # Wrapped in a DO block so the existence check skips databases that
-    # predate the reflections -> mental_models rename chain (no table to
-    # repair). On those, every ALTER below would error.
-    op.execute(
-        f"""
-        DO $$
-        BEGIN
-            IF EXISTS (
-                SELECT 1 FROM information_schema.tables
-                WHERE table_name = 'mental_models'
-                  {schema_clause}
-            ) THEN
-                -- Add the six v4 columns idempotently.
-                ALTER TABLE {schema}mental_models
-                    ADD COLUMN IF NOT EXISTS subtype VARCHAR(32) NOT NULL DEFAULT 'structural';
-                ALTER TABLE {schema}mental_models
-                    ADD COLUMN IF NOT EXISTS description TEXT NOT NULL DEFAULT '';
-                ALTER TABLE {schema}mental_models
-                    ADD COLUMN IF NOT EXISTS entity_id UUID;
-                ALTER TABLE {schema}mental_models
-                    ADD COLUMN IF NOT EXISTS observations JSONB DEFAULT '{{"observations": []}}'::jsonb;
-                ALTER TABLE {schema}mental_models
-                    ADD COLUMN IF NOT EXISTS links VARCHAR[];
-                ALTER TABLE {schema}mental_models
-                    ADD COLUMN IF NOT EXISTS last_updated TIMESTAMP WITH TIME ZONE;
-
-                -- Recreate the CHECK constraint with the canonical v4 allowlist.
-                -- Existing rows with subtype = 'directive' (possible on databases
-                -- that ran the o0j1k2l3m4n5 directive-only path) are rewritten to
-                -- 'structural' first so the constraint add succeeds.
-                UPDATE {schema}mental_models SET subtype = 'structural' WHERE subtype = 'directive';
-
-                ALTER TABLE {schema}mental_models DROP CONSTRAINT IF EXISTS ck_mental_models_subtype;
-                ALTER TABLE {schema}mental_models
-                    ADD CONSTRAINT ck_mental_models_subtype
-                    CHECK (subtype IN ('structural', 'emergent', 'pinned', 'learned'));
-
-                CREATE INDEX IF NOT EXISTS idx_mental_models_subtype
-                    ON {schema}mental_models(bank_id, subtype);
-            END IF;
-        END$$;
-        """
-    )
-
-
-def _pg_downgrade() -> None:
-    """No-op: dropping these columns would corrupt v4 application code."""
-    pass
-
-
-def upgrade() -> None:
-    # PG-only: Oracle's baseline (o1a2b3c4d5e6) creates mental_models with its
-    # own subtype shape (chk_mm_subtype IN ('directive', 'pinned')) and a
-    # different table topology, so this PG-shaped repair does not apply.
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/8c6fa6f7230b_merge_v053_divergent_heads.py

@@ -1,50 +0,0 @@
-"""Merge divergent migration heads for v0.5.3
-
-v0.5.3 shipped with two migration heads that were never unified:
-
-  * ``c4x5y6z7a8b9`` — delta-refresh chain
-    (``add_last_refreshed_source_query`` ->
-     ``add_structured_content_to_mental_models`` ->
-     ``backsweep_orphan_observations_v2``)
-
-  * ``h3i4j5k6l7m8`` — per-bank vector indexes / audit log chain
-    (the ``merge_heads_and_add_unit_entities_index`` subtree)
-
-Both fork from ``z1u2v3w4x5y6``. Upgrades from v0.5.2 still succeed — the
-walker applies the three c4x5 revisions and leaves the database stamped at
-both heads — but the result is a split DAG: ``alembic upgrade head``
-(singular) is ambiguous, and any future migration has to pick one head as
-its parent, orphaning the other.
-
-This revision linearises the DAG into a single head. It has no schema
-effect.
-
-Revision ID: 8c6fa6f7230b
-Revises: c4x5y6z7a8b9, h3i4j5k6l7m8
-Create Date: 2026-04-18
-"""
-
-from collections.abc import Sequence
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "8c6fa6f7230b"
-down_revision: str | Sequence[str] | None = ("c4x5y6z7a8b9", "h3i4j5k6l7m8")
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _pg_upgrade() -> None:
-    pass
-
-
-def _pg_downgrade() -> None:
-    pass
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/9f8e7d6c5b4a_memory_links_deferrable_fk.py

@@ -1,128 +0,0 @@
-"""Make memory_links.from_unit_id and memory_links.to_unit_id FKs deferrable.
-
-Revision ID: 9f8e7d6c5b4a
-Revises: o1a2b3c4d5e6
-Create Date: 2026-05-03
-
-Background
-----------
-Concurrent retain (which INSERTs into ``memory_links``) and any code path
-that DELETEs a row whose deletion cascades into ``memory_links`` (e.g.
-delta-retain superseding chunks, which CASCADEs chunks → memory_units →
-memory_links) can deadlock under sustained single-tenant write load.
-
-The deadlock cycle:
-
-* Tx A: ``DELETE FROM chunks WHERE chunk_id = ANY(...)``
-  → CASCADE acquires row locks on memory_units, then on memory_links rows
-    where ``to_unit_id`` matches the deleted units.
-* Tx B: ``INSERT INTO memory_links (...)`` referencing one of the same
-  memory_units rows.
-  → The immediate FK check takes ``FOR KEY SHARE`` on those memory_units
-    rows.
-
-The two transactions take row locks on the same memory_units rows in
-opposite orders depending on which side started first. PostgreSQL detects
-the cycle and aborts one transaction; the loser is killed mid-batch, the
-winner continues. Workers then retry, but under sustained write load the
-pattern repeats.
-
-Fix
----
-Make both ``memory_links → memory_units`` FKs (``from_unit_id`` and
-``to_unit_id``) ``DEFERRABLE INITIALLY DEFERRED``. This pushes the FK
-check from INSERT time to COMMIT time:
-
-* INSERT no longer takes ``FOR KEY SHARE`` on the memory_units row → no
-  contention with the cascading DELETE's row lock.
-* At COMMIT the engine validates referential integrity in one shot. If a
-  cascade-DELETE has since removed the referenced unit, the INSERT
-  transaction commits OR fails with a clean FK violation (sqlstate
-  23503) instead of a deadlock (sqlstate 40P01).
-
-The ``WHERE EXISTS`` filter already in ``_bulk_insert_links`` continues to
-filter out the typical "stale unit_id" case at INSERT time; the deferred
-FK is only the backstop for the narrow race window between the EXISTS
-probe and COMMIT. ``ON DELETE CASCADE`` semantics are unchanged — only
-the *timing* of the constraint check moves.
-
-The ``entity_id`` FK on ``memory_links`` is not changed; entities are not
-involved in the observed deadlock cycle and leaving the constraint
-immediate keeps the error message specific when an entity row is missing.
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "9f8e7d6c5b4a"
-down_revision: str | Sequence[str] | None = "o1a2b3c4d5e6"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Schema-qualifier for raw SQL on PG (multi-tenant search_path)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-# The two FK constraints installed by the initial schema migration
-# (5a366d414dce_initial_schema), mapped to the column they constrain.
-# They reference memory_units(id) with ON DELETE CASCADE — that
-# semantics is preserved; only the deferral attribute changes.
-_FK_COLUMNS: dict[str, str] = {
-    "fk_memory_links_from_unit_id_memory_units": "from_unit_id",
-    "fk_memory_links_to_unit_id_memory_units": "to_unit_id",
-}
-
-
-def _pg_upgrade() -> None:
-    schema = _get_schema_prefix()
-    # PostgreSQL doesn't allow altering the deferrability of an existing
-    # constraint with ALTER CONSTRAINT — the constraint must be dropped
-    # and recreated. DROP IF EXISTS makes the migration safe to re-run
-    # on schemas where the constraint was already recreated.
-    for fk_name, column in _FK_COLUMNS.items():
-        op.execute(f"ALTER TABLE {schema}memory_links DROP CONSTRAINT IF EXISTS {fk_name}")
-        op.execute(
-            f"""
-            ALTER TABLE {schema}memory_links
-                ADD CONSTRAINT {fk_name}
-                FOREIGN KEY ({column})
-                REFERENCES {schema}memory_units (id)
-                ON DELETE CASCADE
-                DEFERRABLE INITIALLY DEFERRED
-            """
-        )
-
-
-def _pg_downgrade() -> None:
-    schema = _get_schema_prefix()
-    # Revert to the default (NOT DEFERRABLE) form so a downgrade actually
-    # restores the prior schema state, even though that re-introduces the
-    # deadlock window.
-    for fk_name, column in _FK_COLUMNS.items():
-        op.execute(f"ALTER TABLE {schema}memory_links DROP CONSTRAINT IF EXISTS {fk_name}")
-        op.execute(
-            f"""
-            ALTER TABLE {schema}memory_links
-                ADD CONSTRAINT {fk_name}
-                FOREIGN KEY ({column})
-                REFERENCES {schema}memory_units (id)
-                ON DELETE CASCADE
-            """
-        )
-
-
-def upgrade() -> None:
-    # PG-only: Oracle's deferrable-FK semantics differ and the deadlock
-    # cycle was only observed on PostgreSQL. Oracle slot intentionally
-    # absent → no-op there.
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/a1b2c3d4e5f6_add_file_storage_table.py

@@ -15,8 +15,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "a1b2c3d4e5f6"
 down_revision: str | Sequence[str] | None = "y0t1u2v3w4x5"
 branch_labels: str | Sequence[str] | None = None
@@ -29,7 +27,7 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     """Create file_storage table for BYTEA storage."""
     schema = _get_schema_prefix()
 
@@ -54,7 +52,7 @@ def _pg_upgrade() -> None:
     )
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     """Remove file_storage table and related columns."""
     schema = _get_schema_prefix()
 
@@ -70,11 +68,3 @@ def _pg_downgrade() -> None:
 
     # Drop file_storage table
     op.execute(f"DROP TABLE IF EXISTS {schema}file_storage")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/a2b3c4d5e6f7_add_text_signals_column.py

@@ -7,7 +7,6 @@ the stored fact text.
 - vchord: text_signals included in tokenize() at insert time
 - native: search_vector GENERATED column regenerated to include text_signals
 - pg_textsearch: no change (index only supports a single base column)
-- pg_search: BM25 index dropped and recreated to include text_signals
 
 Revision ID: a2b3c4d5e6f7
 Revises: z1u2v3w4x5y6
@@ -19,13 +18,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api._pg_search import (
-    PG_SEARCH_TOKENIZER_ENV,
-    normalize_pg_search_tokenizer,
-    pg_search_bm25_columns,
-)
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "a2b3c4d5e6f7"
 down_revision: str | Sequence[str] | None = "aa2b3c4d5e6f"
 branch_labels: str | Sequence[str] | None = None
@@ -41,11 +33,7 @@ def _detect_text_search_extension() -> str:
     return os.getenv("HINDSIGHT_API_TEXT_SEARCH_EXTENSION", "native").lower()
 
 
-def _pg_search_tokenizer() -> str:
-    return normalize_pg_search_tokenizer(os.getenv(PG_SEARCH_TOKENIZER_ENV))
-
-
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     schema = _get_schema_prefix()
     table = f"{schema}memory_units"
     text_search_ext = _detect_text_search_extension()
@@ -72,22 +60,12 @@ def _pg_upgrade() -> None:
             CREATE INDEX IF NOT EXISTS idx_memory_units_text_search
             ON {table} USING gin(search_vector)
         """)
-    elif text_search_ext == "pg_search":
-        # ParadeDB pg_search: drop the existing BM25 index and recreate it
-        # to include text_signals alongside text and context.
-        bm25_cols = pg_search_bm25_columns("id", ("text", "context", "text_signals"), _pg_search_tokenizer())
-        op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_units_text_search")
-        op.execute(f"""
-            CREATE INDEX idx_memory_units_text_search ON {table}
-            USING bm25 ({bm25_cols})
-            WITH (key_field='id')
-        """)
 
     # vchord: tokenize() call in fact_storage.py is updated to include text_signals at insert time
     # pg_textsearch: no change — index operates on the base `text` column only
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     schema = _get_schema_prefix()
     table = f"{schema}memory_units"
     text_search_ext = _detect_text_search_extension()
@@ -106,22 +84,5 @@ def _pg_downgrade() -> None:
             CREATE INDEX idx_memory_units_text_search
             ON {table} USING gin(search_vector)
         """)
-    elif text_search_ext == "pg_search":
-        # Restore the original (id, text, context) BM25 index without text_signals.
-        bm25_cols = pg_search_bm25_columns("id", ("text", "context"), _pg_search_tokenizer())
-        op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_units_text_search")
-        op.execute(f"""
-            CREATE INDEX idx_memory_units_text_search ON {table}
-            USING bm25 ({bm25_cols})
-            WITH (key_field='id')
-        """)
 
     op.execute(f"ALTER TABLE {table} DROP COLUMN IF EXISTS text_signals")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/a2b3c4d5e6f8_add_gin_index_source_memory_ids.py

@@ -24,8 +24,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "a2b3c4d5e6f8"
 down_revision: str | Sequence[str] | None = "f7g8h9i0j1k2"
 branch_labels: str | Sequence[str] | None = None
@@ -37,28 +35,20 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # CREATE INDEX CONCURRENTLY cannot run inside a transaction block; an
-    # autocommit_block runs it outside Alembic's migration transaction.
-    with op.get_context().autocommit_block():
-        op.execute(
-            f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_units_source_memory_ids "
-            f"ON {schema}memory_units USING GIN (source_memory_ids) "
-            f"WHERE source_memory_ids IS NOT NULL"
-        )
-
-
-def _pg_downgrade() -> None:
-    schema = _get_schema_prefix()
-    with op.get_context().autocommit_block():
-        op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_units_source_memory_ids")
-
-
 def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
+    schema = _get_schema_prefix()
+
+    # CREATE INDEX CONCURRENTLY cannot run inside a transaction block.
+    # Commit the current Alembic transaction first.
+    op.execute("COMMIT")
+    op.execute(
+        f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_units_source_memory_ids "
+        f"ON {schema}memory_units USING GIN (source_memory_ids) "
+        f"WHERE source_memory_ids IS NOT NULL"
+    )
 
 
 def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)
+    schema = _get_schema_prefix()
+    op.execute("COMMIT")
+    op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_units_source_memory_ids")

hindsight-api-slim/hindsight_api/alembic/versions/a2v3w4x5y6z7_add_last_refreshed_source_query.py

@@ -1,48 +0,0 @@
-"""Add last_refreshed_source_query column to mental_models
-
-Revision ID: a2v3w4x5y6z7
-Revises: z1u2v3w4x5y6
-Create Date: 2026-04-15
-
-Tracks the source_query that was used during the most recent refresh.
-Used by delta-mode refresh to detect when the query has changed: if it has,
-delta mode falls back to a full regeneration because the surgical-edit
-assumption (same topic, new facts) no longer holds.
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "a2v3w4x5y6z7"
-down_revision: str | Sequence[str] | None = "z1u2v3w4x5y6"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _pg_upgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"""
-        ALTER TABLE {schema}mental_models
-        ADD COLUMN IF NOT EXISTS last_refreshed_source_query TEXT
-    """)
-
-
-def _pg_downgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"ALTER TABLE {schema}mental_models DROP COLUMN IF EXISTS last_refreshed_source_query")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/a3b4c5d6e7f8_add_consolidation_failed_at_to_memory_units.py

@@ -13,8 +13,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "a3b4c5d6e7f8"
 down_revision: str | Sequence[str] | None = "g7h8i9j0k1l2"
 branch_labels: str | Sequence[str] | None = None
@@ -27,7 +25,7 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     schema = _get_schema_prefix()
 
     op.execute(
@@ -47,16 +45,8 @@ def _pg_upgrade() -> None:
     )
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     schema = _get_schema_prefix()
 
     op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_units_consolidation_failed")
     op.execute(f"ALTER TABLE {schema}memory_units DROP COLUMN IF EXISTS consolidation_failed_at")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/a4b5c6d7e8f9_fix_per_bank_vector_index_type.py

@@ -1,7 +1,7 @@
 """Fix per-bank vector indexes to match configured extension
 
 Revision ID: a4b5c6d7e8f9
-Revises: 2eee35aa3cfc
+Revises: d6e7f8a9b0c1
 Create Date: 2026-04-01
 
 Migration d5e6f7a8b9c0 hardcoded HNSW when creating per-bank partial vector
@@ -11,8 +11,7 @@ was configured.
 
 This migration detects the mismatch and recreates the affected indexes with
 the correct type. Skipped entirely when the configured extension is pgvector
-(the default) or scann. ScaNN uses global vector indexes because empty or tiny
-per-bank indexes cannot be built safely on AlloyDB.
+(the default), since those indexes are already correct.
 """
 
 import os
@@ -21,10 +20,8 @@ from collections.abc import Sequence
 from alembic import context, op
 from sqlalchemy import text
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "a4b5c6d7e8f9"
-down_revision: str | Sequence[str] | None = "2eee35aa3cfc"
+down_revision: str | Sequence[str] | None = "d6e7f8a9b0c1"
 branch_labels: str | Sequence[str] | None = None
 depends_on: str | Sequence[str] | None = None
 
@@ -40,47 +37,39 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _validate_extension(name: str) -> str:
-    ext = name.lower()
-    if ext not in {"pgvector", "pgvectorscale", "vchord", "scann"}:
-        raise ValueError(
-            f"Invalid HINDSIGHT_API_VECTOR_EXTENSION: {ext}. Must be 'pgvector', 'vchord', 'pgvectorscale', or 'scann'"
-        )
-    return ext
-
-
-def _index_type_keyword(ext: str) -> str:
+def _target_index_type() -> str | None:
+    """Return the target index type, or None if pgvector (no fix needed)."""
+    ext = os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector").lower()
     if ext == "pgvectorscale":
         return "diskann"
-    if ext == "vchord":
+    elif ext == "vchord":
         return "vchordrq"
-    if ext == "scann":
-        return "scann"
-    return "hnsw"
+    return None
 
 
-def _vector_index_using_clause(ext: str) -> str:
+def _vector_index_using_clause() -> str:
+    """Return the USING clause based on the configured vector extension."""
+    ext = os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector").lower()
     if ext == "pgvectorscale":
         return "USING diskann (embedding vector_cosine_ops) WITH (num_neighbors = 50)"
-    if ext == "vchord":
-        return "USING vchordrq (embedding vector_cosine_ops)"
-    if ext == "scann":
-        return "USING scann (embedding cosine) WITH (mode = 'AUTO')"
-    return "USING hnsw (embedding vector_cosine_ops)"
+    elif ext == "vchord":
+        return "USING vchordrq (embedding vector_l2_ops)"
+    else:
+        return "USING hnsw (embedding vector_cosine_ops)"
 
 
-def _pg_upgrade() -> None:
-    ext = _validate_extension(os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector"))
-    if ext in {"pgvector", "scann"}:
+def upgrade() -> None:
+    target = _target_index_type()
+    if target is None:
+        # pgvector — indexes are already HNSW, nothing to fix
         return
-    target = _index_type_keyword(ext)
 
     bind = op.get_bind()
     schema_name = context.config.get_main_option("target_schema")
     schema = _get_schema_prefix()
     table_ref = f'"{schema_name}".memory_units' if schema_name else "memory_units"
     banks_ref = f'"{schema_name}".banks' if schema_name else "banks"
-    using_clause = _vector_index_using_clause(ext)
+    using_clause = _vector_index_using_clause()
     pg_schema = schema_name or "public"
 
     rows = bind.execute(text(f"SELECT bank_id, internal_id FROM {banks_ref}")).fetchall()  # noqa: S608
@@ -124,10 +113,10 @@ def _pg_upgrade() -> None:
             )
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     # Downgrade recreates indexes as HNSW (the original hardcoded behavior)
-    ext = _validate_extension(os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector"))
-    if ext in {"pgvector", "scann"}:
+    target = _target_index_type()
+    if target is None:
         return
 
     bind = op.get_bind()
@@ -151,11 +140,3 @@ def _pg_downgrade() -> None:
                     f"WHERE fact_type = '{ft}' AND bank_id = '{escaped_bank_id}'"
                 )
             )
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/a7b8c9d0e1f2_split_history_into_own_tables.py

@@ -1,253 +0,0 @@
-"""Move mental-model and observation history into dedicated tables.
-
-Both histories were accumulated in a single JSONB/CLOB ``history`` column
-(``mental_models.history`` and ``memory_units.history``), appended to on every
-update. That design has two problems:
-
-1. **Unbounded growth on observations.** The observation write path appended a
-   snapshot on every update with no cap at all, so a frequently-reinforced
-   observation grew its ``history`` array until it crossed Postgres's hard 256MB
-   jsonb limit (SQLSTATE 54000), after which every further UPDATE failed and the
-   row was stuck.
-2. **Wrong-axis cap on mental models.** The mental-model cap bounded the *number*
-   of entries (50), not their *size* — a single large reflect snapshot could
-   still blow the budget — and rewrote the whole array (plus TOAST) on every
-   refresh, defeating HOT updates.
-
-This migration creates one row per history entry in two dedicated tables, with
-an index that makes "most recent N for this item" cheap, then drops the old
-columns. The cap is now enforced at write time as a bounded DELETE of the
-oldest over-cap rows (see config ``*_HISTORY_MAX_ENTRIES``).
-
-Revision ID: a7b8c9d0e1f2
-Revises: d3e4f5a6b7c8
-Create Date: 2026-06-05
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "a7b8c9d0e1f2"
-down_revision: str | Sequence[str] | None = "d3e4f5a6b7c8"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Schema-qualifier for raw SQL on PG (multi-tenant search_path)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-# ---------------------------------------------------------------------------
-# PostgreSQL
-# ---------------------------------------------------------------------------
-def _pg_upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # Both tables share the same shape: surrogate id, FK to the parent, bank_id,
-    # the snapshot payload as a single JSONB ``content`` blob, and changed_at.
-    # The payload is per-row (one change per row) so it stays small — this is NOT
-    # the old single-column-grows-forever design; growth is bounded by row count
-    # plus the write-time cap. Folding the previous_* fields into one JSONB keeps
-    # the schema dialect-simple (no array columns) and flexible.
-
-    # --- mental_model_history -------------------------------------------------
-    # content: {"previous_content": ..., "previous_reflect_response": {...}}
-    op.execute(
-        f"""
-        CREATE TABLE IF NOT EXISTS {schema}mental_model_history (
-            id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
-            mental_model_id VARCHAR(64) NOT NULL,
-            bank_id VARCHAR(64) NOT NULL,
-            content JSONB NOT NULL,
-            changed_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
-            FOREIGN KEY (mental_model_id, bank_id)
-                REFERENCES {schema}mental_models(id, bank_id) ON DELETE CASCADE
-        )
-        """
-    )
-    op.execute(
-        "CREATE INDEX IF NOT EXISTS idx_mm_history_model "
-        f"ON {schema}mental_model_history (bank_id, mental_model_id, changed_at DESC, id DESC)"
-    )
-
-    # --- observation_history --------------------------------------------------
-    # content: {"previous_text", "previous_tags", "previous_occurred_start",
-    #           "previous_occurred_end", "previous_mentioned_at", "new_source_memory_ids"}
-    op.execute(
-        f"""
-        CREATE TABLE IF NOT EXISTS {schema}observation_history (
-            id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
-            observation_id UUID NOT NULL,
-            bank_id VARCHAR(64) NOT NULL,
-            content JSONB NOT NULL,
-            changed_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
-            FOREIGN KEY (observation_id)
-                REFERENCES {schema}memory_units(id) ON DELETE CASCADE
-        )
-        """
-    )
-    op.execute(
-        "CREATE INDEX IF NOT EXISTS idx_observation_history_obs "
-        f"ON {schema}observation_history (observation_id, changed_at DESC, id DESC)"
-    )
-
-    # --- backfill mental models ----------------------------------------------
-    # Explode each row's history array into rows, preserving chronological order
-    # via WITH ORDINALITY so the IDENTITY id tie-breaks oldest->newest correctly.
-    # changed_at is promoted to its own column; the rest of the element becomes
-    # ``content`` (the ``- 'changed_at'`` strips the now-redundant key).
-    op.execute(
-        f"""
-        INSERT INTO {schema}mental_model_history (mental_model_id, bank_id, content, changed_at)
-        SELECT mm.id, mm.bank_id,
-               e - 'changed_at',
-               COALESCE(NULLIF(e->>'changed_at', '')::timestamptz, now())
-        FROM {schema}mental_models mm
-             CROSS JOIN LATERAL jsonb_array_elements(mm.history) WITH ORDINALITY a(e, ord)
-        WHERE mm.history IS NOT NULL
-          AND jsonb_typeof(mm.history) = 'array'
-          AND jsonb_array_length(mm.history) > 0
-        ORDER BY mm.id, mm.bank_id, ord
-        """
-    )
-
-    # --- backfill observations -----------------------------------------------
-    op.execute(
-        f"""
-        INSERT INTO {schema}observation_history (observation_id, bank_id, content, changed_at)
-        SELECT mu.id, mu.bank_id,
-               e - 'changed_at',
-               COALESCE(NULLIF(e->>'changed_at', '')::timestamptz, now())
-        FROM {schema}memory_units mu
-             CROSS JOIN LATERAL jsonb_array_elements(mu.history) WITH ORDINALITY a(e, ord)
-        WHERE mu.fact_type = 'observation'
-          AND mu.history IS NOT NULL
-          AND jsonb_typeof(mu.history) = 'array'
-          AND jsonb_array_length(mu.history) > 0
-        ORDER BY mu.id, ord
-        """
-    )
-
-    # --- drop the legacy columns ---------------------------------------------
-    op.execute(f"ALTER TABLE {schema}mental_models DROP COLUMN IF EXISTS history")
-    op.execute(f"ALTER TABLE {schema}memory_units DROP COLUMN IF EXISTS history")
-
-
-def _pg_downgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # Re-add the columns (empty — historical content is not reconstructed back
-    # into the array form; the dedicated tables are dropped below).
-    op.execute(f"ALTER TABLE {schema}mental_models ADD COLUMN IF NOT EXISTS history JSONB DEFAULT '[]'::jsonb")
-    op.execute(f"ALTER TABLE {schema}memory_units ADD COLUMN IF NOT EXISTS history JSONB DEFAULT '[]'::jsonb")
-
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_observation_history_obs")
-    op.execute(f"DROP TABLE IF EXISTS {schema}observation_history")
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_mm_history_model")
-    op.execute(f"DROP TABLE IF EXISTS {schema}mental_model_history")
-
-
-# ---------------------------------------------------------------------------
-# Oracle 23ai
-# ---------------------------------------------------------------------------
-def _oracle_upgrade() -> None:
-    # Same single-JSONB shape as PG: ``content`` holds the snapshot payload as a
-    # CLOB IS JSON. The legacy per-element JSON object (minus changed_at, promoted
-    # to its own column) is carried through verbatim on backfill — the array
-    # columns the previous design needed are gone.
-    op.execute(
-        """
-        CREATE TABLE IF NOT EXISTS mental_model_history (
-            id NUMBER GENERATED BY DEFAULT AS IDENTITY,
-            mental_model_id VARCHAR2(256) NOT NULL,
-            bank_id VARCHAR2(256) NOT NULL,
-            content CLOB NOT NULL
-                CONSTRAINT mmh_content_json CHECK (content IS JSON),
-            changed_at TIMESTAMP WITH TIME ZONE DEFAULT SYSTIMESTAMP NOT NULL,
-            CONSTRAINT pk_mental_model_history PRIMARY KEY (id),
-            CONSTRAINT fk_mmh_model FOREIGN KEY (mental_model_id, bank_id)
-                REFERENCES mental_models(id, bank_id) ON DELETE CASCADE
-        )
-        """
-    )
-    op.execute(
-        "CREATE INDEX idx_mm_history_model ON mental_model_history (bank_id, mental_model_id, changed_at DESC, id DESC)"
-    )
-
-    op.execute(
-        """
-        CREATE TABLE IF NOT EXISTS observation_history (
-            id NUMBER GENERATED BY DEFAULT AS IDENTITY,
-            observation_id RAW(16) NOT NULL,
-            bank_id VARCHAR2(256) NOT NULL,
-            content CLOB NOT NULL
-                CONSTRAINT oh_content_json CHECK (content IS JSON),
-            changed_at TIMESTAMP WITH TIME ZONE DEFAULT SYSTIMESTAMP NOT NULL,
-            CONSTRAINT pk_observation_history PRIMARY KEY (id),
-            CONSTRAINT fk_oh_obs FOREIGN KEY (observation_id)
-                REFERENCES memory_units(id) ON DELETE CASCADE
-        )
-        """
-    )
-    op.execute(
-        "CREATE INDEX idx_observation_history_obs ON observation_history (observation_id, changed_at DESC, id DESC)"
-    )
-
-    bind = op.get_bind()
-
-    # Backfill via JSON_TABLE. ``content`` is the whole element (FORMAT JSON PATH
-    # '$'); changed_at is also promoted to its own column. Backfilled content may
-    # therefore still carry a redundant changed_at key, which the read path
-    # ignores in favour of the column — harmless, and avoids JSON surgery here.
-    bind.exec_driver_sql(
-        """
-        INSERT INTO mental_model_history (mental_model_id, bank_id, content, changed_at)
-        SELECT mm.id, mm.bank_id, jt.content, NVL(jt.changed_at, SYSTIMESTAMP)
-        FROM mental_models mm,
-             JSON_TABLE(mm.history, '$[*]' COLUMNS (
-                 seq FOR ORDINALITY,
-                 content CLOB FORMAT JSON PATH '$',
-                 changed_at TIMESTAMP WITH TIME ZONE PATH '$.changed_at'
-             )) jt
-        WHERE mm.history IS NOT NULL
-        ORDER BY mm.id, mm.bank_id, jt.seq
-        """
-    )
-
-    bind.exec_driver_sql(
-        """
-        INSERT INTO observation_history (observation_id, bank_id, content, changed_at)
-        SELECT mu.id, mu.bank_id, jt.content, NVL(jt.changed_at, SYSTIMESTAMP)
-        FROM memory_units mu,
-             JSON_TABLE(mu.history, '$[*]' COLUMNS (
-                 seq FOR ORDINALITY,
-                 content CLOB FORMAT JSON PATH '$',
-                 changed_at TIMESTAMP WITH TIME ZONE PATH '$.changed_at'
-             )) jt
-        WHERE mu.fact_type = 'observation' AND mu.history IS NOT NULL
-        ORDER BY mu.id, jt.seq
-        """
-    )
-
-    op.execute("ALTER TABLE mental_models DROP COLUMN history")
-    op.execute("ALTER TABLE memory_units DROP COLUMN history")
-
-
-def _oracle_downgrade() -> None:
-    op.execute("ALTER TABLE mental_models ADD history CLOB DEFAULT '[]' NOT NULL")
-    op.execute("ALTER TABLE memory_units ADD history CLOB DEFAULT '[]'")
-    op.execute("DROP TABLE observation_history CASCADE CONSTRAINTS")
-    op.execute("DROP TABLE mental_model_history CASCADE CONSTRAINTS")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade, oracle=_oracle_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade, oracle=_oracle_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/aa2b3c4d5e6f_nullable_event_date.py

@@ -12,8 +12,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "aa2b3c4d5e6f"
 down_revision: str | Sequence[str] | None = "z1u2v3w4x5y6"
 branch_labels: str | Sequence[str] | None = None
@@ -26,21 +24,13 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     schema = _get_schema_prefix()
     op.execute(f"ALTER TABLE {schema}memory_units ALTER COLUMN event_date DROP NOT NULL")
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     schema = _get_schema_prefix()
     # Backfill NULLs with now() before restoring the NOT NULL constraint
     op.execute(f"UPDATE {schema}memory_units SET event_date = now() WHERE event_date IS NULL")
     op.execute(f"ALTER TABLE {schema}memory_units ALTER COLUMN event_date SET NOT NULL")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/b2d4f6a8c1e3_repair_maintenance_routines_public.py

@@ -1,156 +0,0 @@
-"""Repair: install maintenance routines on the ``public`` / base-schema run.
-
-The original maintenance-routines migration (``e5f6a7b8c9d0``) only created the
-shared ``public.banks_needing_consolidation()`` and
-``public.schemas_with_expired_rows(...)`` routines when the run had *no*
-``target_schema`` at all. But the single-tenant runtime always migrates an
-explicit schema — which defaults to ``public`` — so on every default
-PostgreSQL deployment the migration was stamped as applied while the functions
-were never created. Background maintenance then logs::
-
-    Retention sweep failed for llm_requests: function public.schemas_with_expired_rows(...) does not exist
-    Consolidation reconcile discovery failed: function public.banks_needing_consolidation() does not exist
-
-See https://github.com/vectorize-io/hindsight/issues/2056.
-
-Because ``e5f6a7b8c9d0`` is already stamped on affected ``0.8.0`` databases,
-editing it would not re-run it there. This forward migration re-installs the
-functions idempotently (``CREATE OR REPLACE``) on the run that targets the
-shared ``public`` schema (base run with no ``target_schema``, or an explicit
-``target_schema=public``), self-healing already-upgraded deployments and
-covering fresh upgrades from earlier versions.
-
-Per-tenant runs against a non-``public`` schema still skip it: re-issuing
-``CREATE OR REPLACE FUNCTION public....`` from each concurrent tenant migration
-aborts with ``tuple concurrently updated`` on the ``pg_proc`` catalog row, and
-the base/public run has already created the functions for every tenant to use.
-Runs that target ``public`` are serialized by the per-schema migration advisory
-lock, so only one wins the create.
-
-PostgreSQL only — the worker poller and these tables are not wired for Oracle,
-so the Oracle slot is intentionally absent (mirrors ``e5f6a7b8c9d0``).
-
-Revision ID: b2d4f6a8c1e3
-Revises: e5f6a7b8c9d0
-Create Date: 2026-06-08
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "b2d4f6a8c1e3"
-down_revision: str | Sequence[str] | None = "e5f6a7b8c9d0"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _should_install_public_routines(target_schema: str | None) -> bool:
-    """True for the run that must (re)create the shared ``public.*`` routines.
-
-    The routines physically live in ``public`` (hard-coded ``public.`` qualifier
-    in the SQL below), so they must be installed exactly once — on the base run
-    (no ``target_schema``) or on the run that explicitly targets ``public``. A
-    run against any other tenant schema skips it to avoid concurrent
-    ``CREATE OR REPLACE`` on the same ``pg_proc`` row.
-    """
-    return not target_schema or target_schema == "public"
-
-
-def _pg_upgrade() -> None:
-    if not _should_install_public_routines(context.config.get_main_option("target_schema")):
-        return
-    # Banks with eligible-but-unscheduled facts and no in-flight consolidation.
-    # Auto-consolidation is filtered here only at the bank level (cheap prune);
-    # the full hierarchical resolution (global -> tenant -> bank, plus
-    # enable_observations) is done by the caller for the small returned set.
-    op.execute(
-        """
-        CREATE OR REPLACE FUNCTION public.banks_needing_consolidation()
-        RETURNS TABLE(schema_name text, bank_id text)
-        LANGUAGE plpgsql STABLE
-        AS $fn$
-        DECLARE
-            sch text;
-        BEGIN
-            FOR sch IN
-                SELECT n.nspname
-                FROM pg_class c
-                JOIN pg_namespace n ON n.oid = c.relnamespace
-                WHERE c.relname = 'memory_units' AND c.relkind = 'r'
-            LOOP
-                RETURN QUERY EXECUTE format($q$
-                    SELECT %1$L::text, m.bank_id
-                    FROM %1$I.memory_units m
-                    JOIN %1$I.banks b ON b.bank_id = m.bank_id
-                    WHERE m.consolidated_at IS NULL
-                      AND m.consolidation_failed_at IS NULL
-                      AND m.fact_type IN ('experience', 'world')
-                      AND COALESCE(b.config -> 'enable_auto_consolidation', 'true'::jsonb) <> 'false'::jsonb
-                      AND NOT EXISTS (
-                          SELECT 1 FROM %1$I.async_operations o
-                          WHERE o.bank_id = m.bank_id
-                            AND o.operation_type = 'consolidation'
-                            AND o.status IN ('pending', 'processing')
-                      )
-                    GROUP BY m.bank_id
-                $q$, sch);
-            END LOOP;
-        END;
-        $fn$;
-        """
-    )
-
-    # Schemas holding at least one row of p_table older than p_days. p_ts_col is
-    # the timestamp column to compare. Returns nothing when p_days <= 0
-    # (retention disabled).
-    op.execute(
-        """
-        CREATE OR REPLACE FUNCTION public.schemas_with_expired_rows(
-            p_table text, p_ts_col text, p_days int
-        )
-        RETURNS SETOF text
-        LANGUAGE plpgsql STABLE
-        AS $fn$
-        DECLARE
-            sch text;
-            has_expired boolean;
-        BEGIN
-            IF p_days IS NULL OR p_days <= 0 THEN
-                RETURN;
-            END IF;
-            FOR sch IN
-                SELECT n.nspname
-                FROM pg_class c
-                JOIN pg_namespace n ON n.oid = c.relnamespace
-                WHERE c.relname = p_table AND c.relkind = 'r'
-            LOOP
-                EXECUTE format(
-                    'SELECT EXISTS (SELECT 1 FROM %I.%I WHERE %I < NOW() - make_interval(days => $1))',
-                    sch, p_table, p_ts_col
-                ) INTO has_expired USING p_days;
-                IF has_expired THEN
-                    RETURN NEXT sch;
-                END IF;
-            END LOOP;
-        END;
-        $fn$;
-        """
-    )
-
-
-def _pg_downgrade() -> None:
-    # No-op: ``e5f6a7b8c9d0`` owns the lifecycle of these functions and drops
-    # them on its own downgrade. This migration only ever (re)creates them, so
-    # there is nothing to undo without racing that migration's DROP.
-    pass
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/b3c4d5e6f7a8_add_content_hash_to_chunks.py

@@ -9,8 +9,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "b3c4d5e6f7a8"
 down_revision: str | Sequence[str] | None = "a3b4c5d6e7f8"
 branch_labels: str | Sequence[str] | None = None
@@ -23,20 +21,12 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     schema = _get_schema_prefix()
     # Add content_hash column to chunks table for delta comparison
     op.execute(f"ALTER TABLE {schema}chunks ADD COLUMN IF NOT EXISTS content_hash TEXT")
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     schema = _get_schema_prefix()
     op.execute(f"ALTER TABLE {schema}chunks DROP COLUMN IF EXISTS content_hash")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/b3c4d5e6f7g8_add_temporal_date_indexes.py

@@ -22,8 +22,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "b3c4d5e6f7g8"
 down_revision: str | Sequence[str] | None = "c1a2b3d4e5f6"
 branch_labels: str | Sequence[str] | None = None
@@ -35,42 +33,36 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
-    schema = _get_schema_prefix()
-    # CREATE INDEX CONCURRENTLY cannot run inside a transaction block; an
-    # autocommit_block runs each statement outside Alembic's migration transaction.
-    with op.get_context().autocommit_block():
-        # Partial index on occurred_start (covers "occurred_start BETWEEN $4 AND $5")
-        op.execute(
-            f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_units_bank_occurred_start "
-            f"ON {schema}memory_units(bank_id, fact_type, occurred_start) "
-            f"WHERE occurred_start IS NOT NULL"
-        )
-        # Partial index on occurred_end (covers "occurred_end BETWEEN $4 AND $5")
-        op.execute(
-            f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_units_bank_occurred_end "
-            f"ON {schema}memory_units(bank_id, fact_type, occurred_end) "
-            f"WHERE occurred_end IS NOT NULL"
-        )
-        # Partial index on mentioned_at (covers "mentioned_at BETWEEN $4 AND $5")
-        op.execute(
-            f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_units_bank_mentioned_at "
-            f"ON {schema}memory_units(bank_id, fact_type, mentioned_at) "
-            f"WHERE mentioned_at IS NOT NULL"
-        )
-
-
-def _pg_downgrade() -> None:
-    schema = _get_schema_prefix()
-    with op.get_context().autocommit_block():
-        op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_units_bank_mentioned_at")
-        op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_units_bank_occurred_end")
-        op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_units_bank_occurred_start")
-
-
 def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
+    schema = _get_schema_prefix()
+    # Partial index on occurred_start (covers "occurred_start BETWEEN $4 AND $5")
+    op.execute("COMMIT")
+    op.execute(
+        f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_units_bank_occurred_start "
+        f"ON {schema}memory_units(bank_id, fact_type, occurred_start) "
+        f"WHERE occurred_start IS NOT NULL"
+    )
+    # Partial index on occurred_end (covers "occurred_end BETWEEN $4 AND $5")
+    op.execute("COMMIT")
+    op.execute(
+        f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_units_bank_occurred_end "
+        f"ON {schema}memory_units(bank_id, fact_type, occurred_end) "
+        f"WHERE occurred_end IS NOT NULL"
+    )
+    # Partial index on mentioned_at (covers "mentioned_at BETWEEN $4 AND $5")
+    op.execute("COMMIT")
+    op.execute(
+        f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_units_bank_mentioned_at "
+        f"ON {schema}memory_units(bank_id, fact_type, mentioned_at) "
+        f"WHERE mentioned_at IS NOT NULL"
+    )
 
 
 def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)
+    schema = _get_schema_prefix()
+    op.execute("COMMIT")
+    op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_units_bank_mentioned_at")
+    op.execute("COMMIT")
+    op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_units_bank_occurred_end")
+    op.execute("COMMIT")
+    op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_units_bank_occurred_start")

hindsight-api-slim/hindsight_api/alembic/versions/b3w4x5y6z7a8_add_structured_content_to_mental_models.py

@@ -1,54 +0,0 @@
-"""Add structured_content JSONB column to mental_models
-
-Revision ID: b3w4x5y6z7a8
-Revises: a2v3w4x5y6z7
-Create Date: 2026-04-16
-
-Stores the structured representation of a mental model document (sections,
-blocks). The plain ``content`` column remains the rendered markdown shown to
-users. ``structured_content`` is the source of truth for delta-mode refreshes:
-each refresh applies a list of typed operations to the structured doc, then
-re-renders to markdown — so unchanged sections come through byte-identical
-without an LLM round-trip.
-
-Nullable: existing markdown-only mental models continue to work in full mode;
-the column is populated lazily the first time a model is refreshed in delta
-mode.
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "b3w4x5y6z7a8"
-down_revision: str | Sequence[str] | None = "a2v3w4x5y6z7"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _pg_upgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"""
-        ALTER TABLE {schema}mental_models
-        ADD COLUMN IF NOT EXISTS structured_content JSONB
-    """)
-
-
-def _pg_downgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"ALTER TABLE {schema}mental_models DROP COLUMN IF EXISTS structured_content")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/b4c5d6e7f8a9_backfill_observation_scopes.py

@@ -14,8 +14,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "b4c5d6e7f8a9"
 down_revision: str | Sequence[str] | None = "a2b3c4d5e6f7"
 branch_labels: str | Sequence[str] | None = None
@@ -27,18 +25,10 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     schema = _get_schema_prefix()
     op.execute(f"ALTER TABLE {schema}memory_units ADD COLUMN IF NOT EXISTS observation_scopes JSONB")
 
 
-def _pg_downgrade() -> None:
-    pass  # intentionally no-op — safe to leave the column in place
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
 def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)
+    pass  # intentionally no-op — safe to leave the column in place

hindsight-api-slim/hindsight_api/alembic/versions/b5a4c3e2f1d8_add_graph_maintenance_queue.py

@@ -1,106 +0,0 @@
-"""Add graph_maintenance_queue table
-
-Queue of memory_units whose outgoing temporal/semantic links lost a
-neighbour to a delete. Drained by the async graph_maintenance worker,
-which tops the unit's links back up using the same probes retain runs.
-
-The queue only targets the link-recompute pass. The worker also runs
-bank-wide sweeps (orphan-entity prune, stale-cooccurrence prune) on each
-invocation; those don't need per-target queueing.
-
-Revision ID: b5a4c3e2f1d8
-Revises: e9b2c7d1f3a4
-Create Date: 2026-05-27
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "b5a4c3e2f1d8"
-down_revision: str | Sequence[str] | None = "e9b2c7d1f3a4"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _pg_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _pg_upgrade() -> None:
-    schema = _pg_schema_prefix()
-    # Composite PK gives us natural ON CONFLICT DO NOTHING dedup when the same
-    # unit is enqueued from overlapping deletes. No FK to memory_units: if the
-    # unit is deleted between enqueue and drain, the worker observes it's gone
-    # and skips — a cascade would erase the work order, but that work has
-    # already been satisfied (no surviving row to maintain).
-    op.execute(
-        f"""
-        CREATE TABLE IF NOT EXISTS {schema}graph_maintenance_queue (
-            bank_id     TEXT NOT NULL,
-            unit_id     UUID NOT NULL,
-            enqueued_at TIMESTAMPTZ NOT NULL DEFAULT now(),
-            PRIMARY KEY (bank_id, unit_id)
-        )
-        """
-    )
-    op.execute(
-        f"""
-        CREATE INDEX IF NOT EXISTS idx_graph_maintenance_queue_bank_enqueued
-        ON {schema}graph_maintenance_queue (bank_id, enqueued_at)
-        """
-    )
-
-
-def _pg_downgrade() -> None:
-    schema = _pg_schema_prefix()
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_graph_maintenance_queue_bank_enqueued")
-    op.execute(f"DROP TABLE IF EXISTS {schema}graph_maintenance_queue")
-
-
-def _oracle_execute_ignoring_955(sql: str) -> None:
-    """Run a CREATE statement and swallow ORA-00955 (object already exists).
-
-    Mirrors the helper in the Oracle baseline migration so reruns stay safe
-    on a database where the table was created by an earlier partial run.
-    """
-    block = (
-        "BEGIN "
-        "EXECUTE IMMEDIATE :stmt; "
-        "EXCEPTION WHEN OTHERS THEN "
-        "IF SQLCODE = -955 THEN NULL; ELSE RAISE; END IF; "
-        "END;"
-    )
-    op.get_bind().exec_driver_sql(block, {"stmt": sql.strip()})
-
-
-def _oracle_upgrade() -> None:
-    _oracle_execute_ignoring_955(
-        """
-        CREATE TABLE graph_maintenance_queue (
-            bank_id     VARCHAR2(256) NOT NULL,
-            unit_id     RAW(16)       NOT NULL,
-            enqueued_at TIMESTAMP WITH TIME ZONE DEFAULT SYSTIMESTAMP NOT NULL,
-            CONSTRAINT pk_graph_maintenance_queue PRIMARY KEY (bank_id, unit_id)
-        )
-        """
-    )
-    _oracle_execute_ignoring_955(
-        "CREATE INDEX idx_graph_maintenance_queue_bank_enqueued ON graph_maintenance_queue (bank_id, enqueued_at)"
-    )
-
-
-def _oracle_downgrade() -> None:
-    op.execute("DROP INDEX idx_graph_maintenance_queue_bank_enqueued")
-    op.execute("DROP TABLE graph_maintenance_queue")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade, oracle=_oracle_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade, oracle=_oracle_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/b5d4e3f2a1c9_backfill_entity_cooccurrences_event_time.py

@@ -1,91 +0,0 @@
-"""Backfill entity_cooccurrences.last_cooccurred from memory_units event time
-
-Revision ID: b5d4e3f2a1c9
-Revises: o1a2b3c4d5e6
-Create Date: 2026-04-24
-
-The writer path in `entity_resolver.link_units_to_entities_batch` historically
-stamped `entity_cooccurrences.last_cooccurred` with `datetime.now(UTC)` at
-flush time, ignoring the source memory unit's event date. For normal online
-retains that's fine (now ≈ event time), but for any corpus that was
-backfilled in a single session — migrating from another memory system, for
-example — every co-occurrence collapsed to the import moment, which hid the
-underlying knowledge timeline from the dashboard's entity graph recency heat
-and from any downstream consumer of the column.
-
-The writer is fixed in the same change set to propagate the unit's event_date;
-this migration repairs historical rows by reading the true event time off
-`unit_entities × memory_units` (falling back to `created_at` when
-`mentioned_at` / `occurred_start` are NULL, so rows never regress).
-
-Oracle slot is intentionally absent: the Oracle baseline (`o1a2b3c4d5e6`)
-landed days before this fix, so any Oracle deployment runs the corrected
-writer against an effectively empty `entity_cooccurrences` — there is no
-historical residue on Oracle to repair. PG-only matches the asymmetry of
-the data, not negligence.
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "b5d4e3f2a1c9"
-down_revision: str | Sequence[str] | None = "o1a2b3c4d5e6"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Get schema prefix for table names (required for multi-tenant support)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _pg_upgrade() -> None:
-    schema = _get_schema_prefix()
-    # Recompute last_cooccurred from the true event time per entity pair.
-    # COALESCE picks the first non-null of mentioned_at / occurred_start /
-    # created_at so banks without event-time metadata still see a sane value
-    # (equivalent to the pre-fix behaviour) instead of NULL.
-    #
-    # The self-join on `unit_entities` is O(k²) per memory_unit in the number
-    # of distinct entities mentioned (k). For typical units k is small (single
-    # digits), but a bank with units containing hundreds of entities and tens
-    # of millions of co-occurrence rows may want to run this off-hours — the
-    # whole UPDATE is one statement, so it locks every targeted ec row for
-    # the duration. The migration is one-time; subsequent online writes
-    # already carry event time via the writer fix.
-    op.execute(
-        f"""
-        UPDATE {schema}entity_cooccurrences ec
-        SET last_cooccurred = sub.event_time
-        FROM (
-            SELECT
-                LEAST(ue1.entity_id, ue2.entity_id) AS e1,
-                GREATEST(ue1.entity_id, ue2.entity_id) AS e2,
-                MAX(COALESCE(mu.mentioned_at, mu.occurred_start, mu.created_at)) AS event_time
-            FROM {schema}memory_units mu
-            JOIN {schema}unit_entities ue1 ON ue1.unit_id = mu.id
-            JOIN {schema}unit_entities ue2 ON ue2.unit_id = mu.id AND ue1.entity_id <> ue2.entity_id
-            GROUP BY 1, 2
-        ) sub
-        WHERE ec.entity_id_1 = sub.e1 AND ec.entity_id_2 = sub.e2
-        """
-    )
-
-
-def _pg_downgrade() -> None:
-    # No-op: the previous column value was `now()` at the time of write and
-    # isn't recoverable. Rolling back the code is sufficient — new writes will
-    # revert to the old behaviour for subsequent retains.
-    pass
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)  # oracle slot intentionally absent — see header
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/b7c4d8e9f1a2_add_chunks_table.py

@@ -12,8 +12,6 @@ import sqlalchemy as sa
 from alembic import op
 from sqlalchemy.dialects import postgresql
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 # revision identifiers, used by Alembic.
 revision: str = "b7c4d8e9f1a2"
 down_revision: str | Sequence[str] | None = "5a366d414dce"
@@ -21,7 +19,7 @@ branch_labels: str | Sequence[str] | None = None
 depends_on: str | Sequence[str] | None = None
 
 
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     """Add chunks table and link memory_units to chunks."""
 
     # Create chunks table with single text PK (bank_id_document_id_chunk_index)
@@ -58,7 +56,7 @@ def _pg_upgrade() -> None:
     op.create_index("idx_memory_units_chunk_id", "memory_units", ["chunk_id"])
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     """Remove chunks table and chunk_id from memory_units."""
 
     # Drop index and foreign key from memory_units
@@ -70,11 +68,3 @@ def _pg_downgrade() -> None:
     op.drop_index("idx_chunks_bank_id", table_name="chunks")
     op.drop_index("idx_chunks_document_id", table_name="chunks")
     op.drop_table("chunks")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/b8c9d0e1f2a3_vchord_cosine_opclass.py

@@ -1,152 +0,0 @@
-"""Re-create vchord vector indexes with vector_cosine_ops
-
-Revision ID: b8c9d0e1f2a3
-Revises: 86f7a033d372
-Create Date: 2026-05-20
-
-vchordrq operator classes are bound 1:1 to operators in PostgreSQL:
-vector_l2_ops only matches ``<->``, while every Hindsight ANN query uses
-``<=>`` (cosine distance). The previous vchord mapping used vector_l2_ops,
-so vchord deployments could never use the index — every ANN query fell
-back to a sequential scan with per-row cosine computation.
-
-This migration finds any vchordrq index built with vector_l2_ops in the
-target schema and re-creates it with vector_cosine_ops, using
-``CREATE INDEX CONCURRENTLY`` so it can run online. It is a no-op when:
-
-* the configured vector extension is not vchord, or
-* no matching indexes exist (already on cosine ops).
-
-Only PostgreSQL is affected; the Oracle 23ai dialect uses its own native
-vector index and does not depend on this mapping.
-"""
-
-from __future__ import annotations
-
-import re
-from collections.abc import Sequence
-
-from alembic import context, op
-from sqlalchemy import text
-
-from hindsight_api._vector_index import configured_vector_extension
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "b8c9d0e1f2a3"
-down_revision: str | Sequence[str] | None = "86f7a033d372"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _pg_schema_prefix() -> str:
-    """Schema-qualifier for raw SQL on PG (multi-tenant search_path)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _rebuild_vchordrq_indexes(old_ops: str, new_ops: str) -> None:
-    """Rebuild vchordrq indexes using ``old_ops`` so they use ``new_ops``.
-
-    Each index is rebuilt with CREATE INDEX CONCURRENTLY under a fresh name,
-    then the old index is dropped and the new one renamed to take its place.
-    Must be called inside an ``autocommit_block()`` because CONCURRENTLY
-    cannot run inside a transaction.
-    """
-    bind = op.get_bind()
-    # `or None` collapses both unset and explicit empty-string Alembic options
-    # into NULL so the COALESCE below falls back to current_schema() in either
-    # case. Without it, an empty-string option would filter on `schemaname = ''`
-    # and skip every real schema.
-    target_schema = context.config.get_main_option("target_schema") or None
-    prefix = _pg_schema_prefix()
-
-    rows = bind.execute(
-        text(
-            "SELECT indexname, indexdef FROM pg_indexes "
-            "WHERE schemaname = COALESCE(:target_schema, current_schema()) "
-            "AND indexdef ILIKE '%vchordrq%' "
-            "AND indexdef ILIKE :ops_like"
-        ),
-        {"target_schema": target_schema, "ops_like": f"%{old_ops}%"},
-    ).fetchall()
-
-    for idx_name, indexdef in rows:
-        # pg_get_indexdef() emits the canonical form `CREATE INDEX <name> ON …`,
-        # so <name> is the first textual occurrence — both substitutions below
-        # rely on that.
-        new_def = indexdef.replace(old_ops, new_ops, 1)
-        temp_name = f"{idx_name}__opclass_swap"
-        new_def = new_def.replace(idx_name, temp_name, 1)
-        new_def = re.sub(
-            r"^CREATE\s+INDEX\b",
-            "CREATE INDEX CONCURRENTLY IF NOT EXISTS",
-            new_def,
-            count=1,
-        )
-
-        # CREATE INDEX CONCURRENTLY can leave the partial index as INVALID if a
-        # previous run errored (disk pressure, lock conflict, signal). Without
-        # this drop the CONCURRENTLY IF NOT EXISTS below would skip creation,
-        # then we'd drop the original and rename the broken index into its
-        # place — silently restoring the seq-scan bug this migration fixes.
-        op.execute(f'DROP INDEX IF EXISTS {prefix}"{temp_name}"')
-        op.execute(new_def)
-
-        # Even on a clean run CONCURRENTLY can finish with indisvalid = false
-        # (e.g. constraint violation during the second build scan). Refuse to
-        # promote in that case so we never alias an INVALID index over a working
-        # one.
-        is_valid = bind.execute(
-            text(
-                "SELECT i.indisvalid "
-                "FROM pg_class c "
-                "JOIN pg_index i ON c.oid = i.indexrelid "
-                "JOIN pg_namespace n ON c.relnamespace = n.oid "
-                "WHERE c.relname = :name "
-                "  AND n.nspname = COALESCE(:target_schema, current_schema())"
-            ),
-            {"name": temp_name, "target_schema": target_schema},
-        ).scalar()
-        if not is_valid:
-            raise RuntimeError(
-                f"vchordrq index rebuild produced an INVALID index ({temp_name}); "
-                "drop it manually and re-run the migration."
-            )
-
-        # DROP + RENAME atomically. A crash between the two would leave
-        # `temp_name` as a valid orphan and the canonical name missing —
-        # next run's `pg_indexes` filter (looking for vector_l2_ops) wouldn't
-        # find anything to recover from, so the index would stay gone. PG
-        # runs the DO block in its own server-side transaction, so either
-        # both succeed or both roll back.
-        op.execute(
-            f"""
-            DO $$
-            BEGIN
-                DROP INDEX IF EXISTS {prefix}"{idx_name}";
-                ALTER INDEX {prefix}"{temp_name}" RENAME TO "{idx_name}";
-            END $$;
-            """
-        )
-
-
-def _pg_upgrade() -> None:
-    if configured_vector_extension() != "vchord":
-        return
-    with op.get_context().autocommit_block():
-        _rebuild_vchordrq_indexes("vector_l2_ops", "vector_cosine_ops")
-
-
-def _pg_downgrade() -> None:
-    if configured_vector_extension() != "vchord":
-        return
-    with op.get_context().autocommit_block():
-        _rebuild_vchordrq_indexes("vector_cosine_ops", "vector_l2_ops")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/c1a2b3d4e5f6_enable_pg_trgm_and_entities_trgm_index.py

@@ -14,8 +14,6 @@ from collections.abc import Sequence
 import sqlalchemy as sa
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "c1a2b3d4e5f6"
 down_revision: str | Sequence[str] | None = "b4c5d6e7f8a9"
 branch_labels: str | Sequence[str] | None = None
@@ -27,7 +25,7 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     # pg_trgm ships with most PostgreSQL installations as a contrib module.
     # It enables fast similarity lookups via GIN indexes, used for entity name matching.
     # On managed services (e.g. Azure Flexible Server), the extension may not be
@@ -47,24 +45,15 @@ def _pg_upgrade() -> None:
     schema = _get_schema_prefix()
     # GIN index on canonical_name enables sub-millisecond trigram similarity queries
     # (% operator, similarity()) instead of full-table scans across all bank entities.
-    # CREATE INDEX CONCURRENTLY cannot run inside a transaction block.
-    with op.get_context().autocommit_block():
-        op.execute(
-            f"CREATE INDEX CONCURRENTLY IF NOT EXISTS entities_canonical_name_trgm_idx "
-            f"ON {schema}entities USING GIN (canonical_name gin_trgm_ops)"
-        )
-
-
-def _pg_downgrade() -> None:
-    schema = _get_schema_prefix()
-    with op.get_context().autocommit_block():
-        op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}entities_canonical_name_trgm_idx")
-    # Note: not dropping pg_trgm extension as other indexes may depend on it
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
+    op.execute("COMMIT")
+    op.execute(
+        f"CREATE INDEX CONCURRENTLY IF NOT EXISTS entities_canonical_name_trgm_idx "
+        f"ON {schema}entities USING GIN (canonical_name gin_trgm_ops)"
+    )
 
 
 def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)
+    schema = _get_schema_prefix()
+    op.execute("COMMIT")
+    op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}entities_canonical_name_trgm_idx")
+    # Note: not dropping pg_trgm extension as other indexes may depend on it

hindsight-api-slim/hindsight_api/alembic/versions/c1d2e3f4a5b6_merge_graph_queue_and_vchord_heads.py

@@ -1,45 +0,0 @@
-"""Merge graph_maintenance_queue and vchord_cosine_opclass heads.
-
-Revision ID: c1d2e3f4a5b6
-Revises: b5a4c3e2f1d8, b8c9d0e1f2a3
-Create Date: 2026-05-29
-
-PRs #1668 (vchord cosine opclass) and #1772 (async link recompute) both
-branched off the same parent and were merged onto main without rebasing,
-leaving two parallel Alembic heads. This is a structural merge revision
-with no schema changes — its only job is to unify the DAG so
-``alembic upgrade head`` is unambiguous again.
-"""
-
-from collections.abc import Sequence
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "c1d2e3f4a5b6"
-down_revision: str | Sequence[str] | None = ("b5a4c3e2f1d8", "b8c9d0e1f2a3")
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _pg_upgrade() -> None:
-    pass
-
-
-def _pg_downgrade() -> None:
-    pass
-
-
-def _oracle_upgrade() -> None:
-    pass
-
-
-def _oracle_downgrade() -> None:
-    pass
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade, oracle=_oracle_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade, oracle=_oracle_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/c2d3e4f5g6h7_add_audit_log_table.py

@@ -14,8 +14,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "c2d3e4f5g6h7"
 down_revision: str | Sequence[str] | None = ("a3b4c5d6e7f8", "c8e5f2a3b4d1")
 branch_labels: str | Sequence[str] | None = None
@@ -28,7 +26,7 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     schema = _get_schema_prefix()
 
     op.execute(
@@ -54,18 +52,10 @@ def _pg_upgrade() -> None:
     op.execute(f"CREATE INDEX IF NOT EXISTS idx_audit_log_started ON {schema}audit_log (started_at DESC)")
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     schema = _get_schema_prefix()
 
     op.execute(f"DROP INDEX IF EXISTS {schema}idx_audit_log_started")
     op.execute(f"DROP INDEX IF EXISTS {schema}idx_audit_log_bank_started")
     op.execute(f"DROP INDEX IF EXISTS {schema}idx_audit_log_action_started")
     op.execute(f"DROP TABLE IF EXISTS {schema}audit_log")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/c3d4e5f6g7h8_add_history_to_mental_models.py

@@ -9,8 +9,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "c3d4e5f6g7h8"
 down_revision: str | Sequence[str] | None = ("a2b3c4d5e6f7", "a2b3c4d5e6f8")
 branch_labels: str | Sequence[str] | None = None
@@ -22,19 +20,11 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     schema = _get_schema_prefix()
     op.execute(f"ALTER TABLE {schema}mental_models ADD COLUMN IF NOT EXISTS history JSONB DEFAULT '[]'::jsonb")
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     schema = _get_schema_prefix()
     op.execute(f"ALTER TABLE {schema}mental_models DROP COLUMN IF EXISTS history")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/c4x5y6z7a8b9_backsweep_orphan_observations_v2.py

@@ -1,76 +0,0 @@
-"""backsweep_orphan_observations_v2
-
-Re-run of Pass 2 from migration ``g7h8i9j0k1l2_backsweep_orphan_observations``
-to sweep observations that became orphaned between then and now.
-
-Why we need it again:
-  ``fact_storage.handle_document_tracking`` (the retain/upsert path) deleted
-  the existing document via the FK cascade — which removes the source
-  ``memory_units`` — but never invalidated the observations derived from
-  them.  Only the explicit ``MemoryEngine.delete_document`` API called
-  ``_delete_stale_observations_for_memories``.  Every document re-ingest
-  therefore left orphan observations whose ``source_memory_ids`` arrays
-  pointed at IDs that no longer existed in ``memory_units``.
-
-  ``handle_document_tracking`` now calls the same cleanup helper before the
-  cascade, so no new orphans will accumulate going forward.  This migration
-  cleans up the historical residue.
-
-Identical to Pass 2 of g7h8i9j0k1l2.  Pass 1 (memory_units whose bank is
-gone) is intentionally not re-run; that scenario has no fresh source.
-
-Revision ID: c4x5y6z7a8b9
-Revises: b3w4x5y6z7a8
-Create Date: 2026-04-16
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "c4x5y6z7a8b9"
-down_revision: str | Sequence[str] | None = "b3w4x5y6z7a8"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _pg_upgrade() -> None:
-    schema = _get_schema_prefix()
-    mu = f"{schema}memory_units"
-
-    # Delete observations whose every source_memory_id refers to a now-deleted
-    # memory_unit (or the array is empty).  Observations with at least one
-    # surviving source are left alone — the consolidation engine will refresh
-    # their text on the next pass.
-    op.execute(
-        f"""
-        DELETE FROM {mu} orphan
-        WHERE orphan.fact_type = 'observation'
-          AND NOT EXISTS (
-              SELECT 1
-              FROM {mu} src
-              WHERE src.id = ANY(orphan.source_memory_ids)
-                AND src.bank_id = orphan.bank_id
-          )
-        """
-    )
-
-
-def _pg_downgrade() -> None:
-    # Deleted rows cannot be restored.
-    pass
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/c5d6e7f8a9b0_add_bank_id_to_memory_links.py

@@ -13,8 +13,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "c5d6e7f8a9b0"
 down_revision: str | Sequence[str] | None = "b3c4d5e6f7a8"
 branch_labels: str | Sequence[str] | None = None
@@ -26,7 +24,7 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     schema = _get_schema_prefix()
 
     # 1. Add nullable column
@@ -45,14 +43,6 @@ def _pg_upgrade() -> None:
     op.execute(f"ALTER TABLE {schema}memory_links ALTER COLUMN bank_id SET NOT NULL")
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     schema = _get_schema_prefix()
     op.execute(f"ALTER TABLE {schema}memory_links DROP COLUMN IF EXISTS bank_id")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/c8e5f2a3b4d1_add_retain_params_to_documents.py

@@ -12,8 +12,6 @@ import sqlalchemy as sa
 from alembic import op
 from sqlalchemy.dialects import postgresql
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 # revision identifiers, used by Alembic.
 revision: str = "c8e5f2a3b4d1"
 down_revision: str | Sequence[str] | None = "b7c4d8e9f1a2"
@@ -21,7 +19,7 @@ branch_labels: str | Sequence[str] | None = None
 depends_on: str | Sequence[str] | None = None
 
 
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     """Add retain_params JSONB column to documents table."""
 
     # Add retain_params column to store parameters passed during retain
@@ -31,7 +29,7 @@ def _pg_upgrade() -> None:
     op.create_index("idx_documents_retain_params", "documents", ["retain_params"], postgresql_using="gin")
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     """Remove retain_params column from documents table."""
 
     # Drop index
@@ -39,11 +37,3 @@ def _pg_downgrade() -> None:
 
     # Drop column
     op.drop_column("documents", "retain_params")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/d2e3f4a5b6c7_add_memory_links_expansion_indexes.py

@@ -34,8 +34,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "d2e3f4a5b6c7"
 down_revision: str | Sequence[str] | None = "b3c4d5e6f7g8"
 branch_labels: str | Sequence[str] | None = None
@@ -47,43 +45,39 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # CREATE INDEX CONCURRENTLY cannot run inside a transaction block; an
-    # autocommit_block runs each statement outside Alembic's migration
-    # transaction.  IF NOT EXISTS makes each statement idempotent on retry.
-    with op.get_context().autocommit_block():
-        # Index for the semantic *incoming* direction in link_expansion_retrieval.py.
-        # Replaces the BitmapAnd of idx_memory_links_to_unit ∩ idx_memory_links_link_type
-        # with a single composite index scan.
-        op.execute(
-            f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_links_to_type_weight "
-            f"ON {schema}memory_links(to_unit_id, link_type, weight DESC)"
-        )
-
-        # Covering index for entity co-occurrence expansion.
-        # Enables an index-only scan: entity_id and to_unit_id are read from the
-        # index leaf pages instead of the heap, eliminating ~2 500 random heap-page
-        # reads per expansion query.
-        op.execute(
-            f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_links_entity_covering "
-            f"ON {schema}memory_links(from_unit_id) "
-            f"INCLUDE (to_unit_id, entity_id) "
-            f"WHERE link_type = 'entity'"
-        )
-
-
-def _pg_downgrade() -> None:
-    schema = _get_schema_prefix()
-    with op.get_context().autocommit_block():
-        op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_links_entity_covering")
-        op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_links_to_type_weight")
-
-
 def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
+    schema = _get_schema_prefix()
+
+    # CREATE INDEX CONCURRENTLY cannot run inside a transaction block.
+    # Commit the current Alembic transaction, then issue each CONCURRENTLY
+    # statement in its own implicit autocommit transaction.
+    # IF NOT EXISTS makes each statement idempotent if the migration is retried.
+
+    # Index for the semantic *incoming* direction in link_expansion_retrieval.py.
+    # Replaces the BitmapAnd of idx_memory_links_to_unit ∩ idx_memory_links_link_type
+    # with a single composite index scan.
+    op.execute("COMMIT")
+    op.execute(
+        f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_links_to_type_weight "
+        f"ON {schema}memory_links(to_unit_id, link_type, weight DESC)"
+    )
+
+    # Covering index for entity co-occurrence expansion.
+    # Enables an index-only scan: entity_id and to_unit_id are read from the
+    # index leaf pages instead of the heap, eliminating ~2 500 random heap-page
+    # reads per expansion query.
+    op.execute("COMMIT")
+    op.execute(
+        f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_links_entity_covering "
+        f"ON {schema}memory_links(from_unit_id) "
+        f"INCLUDE (to_unit_id, entity_id) "
+        f"WHERE link_type = 'entity'"
+    )
 
 
 def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)
+    schema = _get_schema_prefix()
+    op.execute("COMMIT")
+    op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_links_entity_covering")
+    op.execute("COMMIT")
+    op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_links_to_type_weight")

hindsight-api-slim/hindsight_api/alembic/versions/d3e4f5a6b7c8_add_llm_requests_table.py

@@ -1,96 +0,0 @@
-"""Add llm_requests table for per-bank LLM request tracing.
-
-Stores one row per logical LLM call Hindsight makes (success and failure),
-capturing the input messages, model output, token usage (input/output/cached/
-total), finish reason, and caller metadata. Disabled by default at the
-application layer (HINDSIGHT_API_LLM_TRACE_ENABLED); this migration only
-creates the table.
-
-PostgreSQL only — the tracing subsystem is not wired for Oracle, so the Oracle
-slot is intentionally absent (mirrors the audit_log table).
-
-Revision ID: d3e4f5a6b7c8
-Revises: c1d2e3f4a5b6
-Create Date: 2026-06-01
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "d3e4f5a6b7c8"
-down_revision: str | Sequence[str] | None = "c1d2e3f4a5b6"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Get schema prefix for table names (required for multi-tenant support)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _pg_upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    op.execute(
-        f"""
-        CREATE TABLE IF NOT EXISTS {schema}llm_requests (
-            id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
-            bank_id TEXT,
-            operation TEXT,
-            scope TEXT,
-            -- OTel-style grouping: trace_id is shared by every LLM call of one
-            -- operation invocation (e.g. all calls of a single reflect run);
-            -- parent_span_id is that operation span; span_id is this call.
-            trace_id TEXT,
-            span_id TEXT,
-            parent_span_id TEXT,
-            provider TEXT,
-            model TEXT,
-            status TEXT NOT NULL,
-            started_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
-            ended_at TIMESTAMPTZ,
-            duration_ms INTEGER,
-            input_tokens INTEGER,
-            output_tokens INTEGER,
-            cached_tokens INTEGER,
-            total_tokens INTEGER,
-            input JSONB,
-            output JSONB,
-            error TEXT,
-            llm_info JSONB DEFAULT '{{}}'::jsonb,
-            metadata JSONB DEFAULT '{{}}'::jsonb
-        )
-        """
-    )
-
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS idx_llm_requests_bank_started ON {schema}llm_requests (bank_id, started_at DESC)"
-    )
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS idx_llm_requests_status_started ON {schema}llm_requests (status, started_at DESC)"
-    )
-    op.execute(f"CREATE INDEX IF NOT EXISTS idx_llm_requests_started ON {schema}llm_requests (started_at DESC)")
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS idx_llm_requests_trace ON {schema}llm_requests (bank_id, trace_id, started_at)"
-    )
-
-
-def _pg_downgrade() -> None:
-    schema = _get_schema_prefix()
-
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_llm_requests_started")
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_llm_requests_status_started")
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_llm_requests_bank_started")
-    op.execute(f"DROP TABLE IF EXISTS {schema}llm_requests")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/d4e5f6g7h8i9_gin_source_memory_ids_fastupdate_off.py

@@ -18,8 +18,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "d4e5f6g7h8i9"
 down_revision: str | Sequence[str] | None = "d5e6f7a8b9c0"
 branch_labels: str | Sequence[str] | None = None
@@ -31,34 +29,25 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
-    schema = _get_schema_prefix()
-    # DROP + CREATE CONCURRENTLY must run outside a transaction block; an
-    # autocommit_block runs them outside Alembic's migration transaction.
-    with op.get_context().autocommit_block():
-        op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_units_source_memory_ids")
-        op.execute(
-            f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_units_source_memory_ids "
-            f"ON {schema}memory_units USING GIN (source_memory_ids) "
-            f"WITH (fastupdate=off) "
-            f"WHERE source_memory_ids IS NOT NULL"
-        )
-
-
-def _pg_downgrade() -> None:
-    schema = _get_schema_prefix()
-    with op.get_context().autocommit_block():
-        op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_units_source_memory_ids")
-        op.execute(
-            f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_units_source_memory_ids "
-            f"ON {schema}memory_units USING GIN (source_memory_ids) "
-            f"WHERE source_memory_ids IS NOT NULL"
-        )
-
-
 def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
+    schema = _get_schema_prefix()
+    # DROP + CREATE CONCURRENTLY must run outside a transaction block.
+    op.execute("COMMIT")
+    op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_units_source_memory_ids")
+    op.execute(
+        f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_units_source_memory_ids "
+        f"ON {schema}memory_units USING GIN (source_memory_ids) "
+        f"WITH (fastupdate=off) "
+        f"WHERE source_memory_ids IS NOT NULL"
+    )
 
 
 def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)
+    schema = _get_schema_prefix()
+    op.execute("COMMIT")
+    op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_units_source_memory_ids")
+    op.execute(
+        f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_units_source_memory_ids "
+        f"ON {schema}memory_units USING GIN (source_memory_ids) "
+        f"WHERE source_memory_ids IS NOT NULL"
+    )

hindsight-api-slim/hindsight_api/alembic/versions/d5e6f7a8b9c0_add_bank_internal_id_and_per_bank_hnsw.py

@@ -6,11 +6,10 @@ Create Date: 2026-03-11
 
 This migration:
 1. Adds internal_id UUID column to banks (stable identifier for index naming)
-2. For non-ScaNN backends, drops the global vector index (competes with
-   per-bank partial indexes)
-3. For non-ScaNN backends, creates per-(bank_id, fact_type) partial vector
-   indexes for all existing banks using the configured vector extension
-   (HNSW for pgvector, DiskANN for pgvectorscale, vchordrq for vchord).
+2. Drops the global vector index (competes with per-bank partial indexes)
+3. Creates per-(bank_id, fact_type) partial vector indexes for all existing banks
+   using the configured vector extension (HNSW for pgvector, DiskANN for
+   pgvectorscale, vchordrq for vchord).
    (new banks get indexes created at bank-creation time via bank_utils.create_bank_vector_indexes)
 
 Why per-(bank, fact_type) indexes:
@@ -18,8 +17,6 @@ Why per-(bank, fact_type) indexes:
   clause, because the idx_memory_units_bank_id B-tree index always wins at planning time.
 - Per-(bank, fact_type) partial indexes have both predicates matching → planner selects them.
 - The global vector index competes for larger partitions (world, observation) and must be dropped.
-- AlloyDB ScaNN uses global vector indexes with filtered vector search instead
-  because empty or tiny per-bank indexes cannot be built safely.
 """
 
 import os
@@ -28,8 +25,6 @@ from collections.abc import Sequence
 from alembic import context, op
 from sqlalchemy import text
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "d5e6f7a8b9c0"
 down_revision: str | Sequence[str] | None = "c3d4e5f6g7h8"
 branch_labels: str | Sequence[str] | None = None
@@ -42,31 +37,23 @@ _FACT_TYPES: dict[str, str] = {
 }
 
 
-def _configured_vector_extension() -> str:
-    ext = os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector").lower()
-    if ext not in {"pgvector", "pgvectorscale", "vchord", "scann"}:
-        raise ValueError(
-            f"Invalid HINDSIGHT_API_VECTOR_EXTENSION: {ext}. Must be 'pgvector', 'vchord', 'pgvectorscale', or 'scann'"
-        )
-    return ext
-
-
-def _vector_index_using_clause(ext: str) -> str:
-    if ext == "pgvectorscale":
-        return "USING diskann (embedding vector_cosine_ops) WITH (num_neighbors = 50)"
-    if ext == "vchord":
-        return "USING vchordrq (embedding vector_cosine_ops)"
-    if ext == "scann":
-        return "USING scann (embedding cosine) WITH (mode = 'AUTO')"
-    return "USING hnsw (embedding vector_cosine_ops)"
-
-
 def _get_schema_prefix() -> str:
     schema = context.config.get_main_option("target_schema")
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
+def _vector_index_using_clause() -> str:
+    """Return the USING clause based on the configured vector extension."""
+    ext = os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector").lower()
+    if ext == "pgvectorscale":
+        return "USING diskann (embedding vector_cosine_ops) WITH (num_neighbors = 50)"
+    elif ext == "vchord":
+        return "USING vchordrq (embedding vector_l2_ops)"
+    else:
+        return "USING hnsw (embedding vector_cosine_ops)"
+
+
+def upgrade() -> None:
     schema = _get_schema_prefix()
 
     # 1. Add internal_id column to banks
@@ -75,14 +62,6 @@ def _pg_upgrade() -> None:
     )
     op.execute(f"ALTER TABLE {schema}banks ADD CONSTRAINT banks_internal_id_unique UNIQUE (internal_id)")
 
-    ext = _configured_vector_extension()
-    if ext == "scann":
-        # ScaNN should keep/use a global vector index. Per-bank partial indexes
-        # are created while banks are empty and can fail AlloyDB's ScaNN build
-        # requirements, so this migration leaves vector index reconciliation to
-        # runtime ensure_vector_extension once enough rows exist.
-        return
-
     # 2. Drop any fact_type-only partial indexes that may exist from prior migrations
     #    (bank_id B-tree always wins over them when bank_id is in the WHERE clause)
     op.execute(f"DROP INDEX IF EXISTS {schema}idx_mu_emb_world")
@@ -98,7 +77,7 @@ def _pg_upgrade() -> None:
     schema_name = context.config.get_main_option("target_schema")
     table_ref = f'"{schema_name}".memory_units' if schema_name else "memory_units"
     banks_ref = f'"{schema_name}".banks' if schema_name else "banks"
-    using_clause = _vector_index_using_clause(ext)
+    using_clause = _vector_index_using_clause()
 
     rows = bind.execute(text(f"SELECT bank_id, internal_id FROM {banks_ref}")).fetchall()  # noqa: S608
     for row in rows:
@@ -117,7 +96,7 @@ def _pg_upgrade() -> None:
             )
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     schema = _get_schema_prefix()
 
     # Drop per-bank HNSW indexes (iterate existing banks)
@@ -128,7 +107,7 @@ def _pg_downgrade() -> None:
     rows = bind.execute(text(f"SELECT internal_id FROM {banks_ref}")).fetchall()  # noqa: S608
     for row in rows:
         internal_id = str(row[0]).replace("-", "")[:16]
-        for ft_short in _FACT_TYPES.values():
+        for ft_short in _HNSW_FACT_TYPES.values():
             idx_name = f"idx_mu_emb_{ft_short}_{internal_id}"
             bind.execute(text(f"DROP INDEX IF EXISTS {schema}{idx_name}"))
 
@@ -158,11 +137,3 @@ def _pg_downgrade() -> None:
     # Drop internal_id column
     op.execute(f"ALTER TABLE {schema}banks DROP CONSTRAINT IF EXISTS banks_internal_id_unique")
     op.execute(f"ALTER TABLE {schema}banks DROP COLUMN IF EXISTS internal_id")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/d5y6z7a8b9c0_backfill_mental_models_subtype.py

@@ -1,67 +0,0 @@
-"""Backfill mental_models.subtype for databases that ran h3c4d5e6f7g8 before the fix
-
-Migration h3c4d5e6f7g8 used CREATE TABLE IF NOT EXISTS to create the
-mental_models table with a subtype column. But on databases where the table
-already existed (from the reflections -> mental_models rename chain), the
-CREATE was a no-op and subtype was never added. A fix was later added to
-h3c4d5e6f7g8 (Step 4b), but databases that had already run the migration
-never re-execute it. This migration adds the missing columns idempotently.
-
-Revision ID: d5y6z7a8b9c0
-Revises: 8c6fa6f7230b
-Create Date: 2026-04-18
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "d5y6z7a8b9c0"
-down_revision: str | Sequence[str] | None = "8c6fa6f7230b"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _pg_upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # Add columns that h3c4d5e6f7g8 intended to create but missed when
-    # the table already existed from the reflections rename chain.
-    for col_ddl in [
-        "subtype VARCHAR(32) NOT NULL DEFAULT 'structural'",
-        "description TEXT NOT NULL DEFAULT ''",
-        "entity_id UUID",
-        "observations JSONB DEFAULT '{\"observations\": []}'::jsonb",
-        "links VARCHAR[]",
-        "last_updated TIMESTAMP WITH TIME ZONE",
-    ]:
-        op.execute(f"ALTER TABLE {schema}mental_models ADD COLUMN IF NOT EXISTS {col_ddl}")
-
-    # Ensure the CHECK constraint exists
-    op.execute(f"ALTER TABLE {schema}mental_models DROP CONSTRAINT IF EXISTS ck_mental_models_subtype")
-    op.execute(f"""
-        ALTER TABLE {schema}mental_models
-        ADD CONSTRAINT ck_mental_models_subtype CHECK (subtype IN ('structural', 'emergent', 'pinned', 'learned'))
-    """)
-
-    op.execute(f"CREATE INDEX IF NOT EXISTS idx_mental_models_subtype ON {schema}mental_models(bank_id, subtype)")
-
-
-def _pg_downgrade() -> None:
-    # No-op: these columns are part of the intended schema
-    pass
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/d6e7f8a9b0c1_drop_documents_metadata_column.py

@@ -1,49 +0,0 @@
-"""Drop unused metadata column from documents table
-
-Revision ID: d6e7f8a9b0c1
-Revises: c2d3e4f5g6h7, c5d6e7f8a9b0
-Create Date: 2026-03-30
-
-The metadata column on documents was always stored as an empty dict {}.
-Actual document metadata is stored inside retain_params.metadata.
-
-This migration was originally shipped in v0.4.22, then its file was deleted
-in v0.5.0 (and its revision ID accidentally reused by 2eee35aa3cfc).
-Restoring the file so that databases stamped at this revision can upgrade
-cleanly to v0.5.x+.
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "d6e7f8a9b0c1"
-down_revision: str | Sequence[str] | None = ("c2d3e4f5g6h7", "c5d6e7f8a9b0")
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Get schema prefix for table names (required for multi-tenant support)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _pg_upgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"ALTER TABLE {schema}documents DROP COLUMN IF EXISTS metadata")
-
-
-def _pg_downgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"ALTER TABLE {schema}documents ADD COLUMN IF NOT EXISTS metadata jsonb DEFAULT '{{}}'")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/d9f6a3b4c5e2_rename_bank_to_interactions.py

@@ -8,8 +8,6 @@ Create Date: 2024-12-04 15:00:00.000000
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 # revision identifiers, used by Alembic.
 revision = "d9f6a3b4c5e2"
 down_revision = "c8e5f2a3b4d1"
@@ -23,7 +21,7 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade():
+def upgrade():
     schema = _get_schema_prefix()
 
     # Drop old check constraint FIRST (before updating data)
@@ -40,7 +38,7 @@ def _pg_upgrade():
     )
 
 
-def _pg_downgrade():
+def downgrade():
     schema = _get_schema_prefix()
 
     # Drop new check constraint FIRST
@@ -53,11 +51,3 @@ def _pg_downgrade():
     op.create_check_constraint(
         "memory_units_fact_type_check", "memory_units", "fact_type IN ('world', 'bank', 'opinion', 'observation')"
     )
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/e0a1b2c3d4e5_disposition_to_3_traits.py

@@ -14,8 +14,6 @@ from collections.abc import Sequence
 import sqlalchemy as sa
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 # revision identifiers, used by Alembic.
 revision: str = "e0a1b2c3d4e5"
 down_revision: str | Sequence[str] | None = "rename_personality"
@@ -35,7 +33,7 @@ def _get_target_schema() -> str:
     return schema if schema else "public"
 
 
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     """Convert Big Five disposition to 3-trait disposition."""
     conn = op.get_bind()
     schema = _get_schema_prefix()
@@ -77,7 +75,7 @@ def _pg_upgrade() -> None:
     )
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     """Convert back to Big Five disposition."""
     conn = op.get_bind()
     schema = _get_schema_prefix()
@@ -111,11 +109,3 @@ def _pg_downgrade() -> None:
         ALTER COLUMN disposition SET DEFAULT '{{"openness": 0.5, "conscientiousness": 0.5, "extraversion": 0.5, "agreeableness": 0.5, "neuroticism": 0.5, "bias_strength": 0.5}}'::jsonb
     """)
     )
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/e1b2c3d4f5a6_drop_unused_indexes.py

@@ -1,133 +0,0 @@
-"""Drop indexes that are unused or redundant with composite indexes.
-
-Code audit identified the following indexes as either dead (no code path
-exercises them) or fully covered by composite indexes the planner already
-prefers:
-
-memory_links:
-1. idx_memory_links_entity_covering — entity co-occurrence expansion was
-   rewritten to traverse unit_entities instead of memory_links, so no code
-   path filters memory_links on (link_type = 'entity').
-2. idx_memory_links_from_unit — redundant.  idx_memory_links_from_type_weight
-   (from_unit_id, link_type, weight DESC) leads with the same column and
-   answers every from_unit_id = X query.
-3. idx_memory_links_to_unit — redundant.  idx_memory_links_to_type_weight
-   (to_unit_id, link_type, weight DESC) leads with the same column.
-4. idx_memory_links_link_type — no application query filters on link_type
-   alone; the composite indexes above serve every (from/to + link_type)
-   predicate.
-
-entities:
-5. idx_entities_canonical_name — superseded by
-   entities_canonical_name_lower_trgm_idx (case-insensitive lookups).
-6. entities_canonical_name_trgm_idx — superseded by the lowercase variant
-   in migration 2eee35aa3cfc, but the original was never dropped on schemas
-   that ran the prior migration.
-
-documents:
-7. idx_documents_retain_params — GIN index on retain_params JSONB; no query
-   uses jsonb containment on this column.
-8. idx_documents_content_hash — content-hash lookups happen on the chunks
-   table (chunks.content_hash, indexed separately).
-
-unit_entities:
-9. idx_unit_entities_entity — defensive drop.  Migration h3i4j5k6l7m8 already
-   issues DROP INDEX IF EXISTS for this; this re-runs the drop idempotently
-   to cover any schema that missed the previous migration.
-
-All drops use CONCURRENTLY + IF EXISTS so they neither block writers nor
-fail on schemas where the index is already gone.
-
-Revision ID: e1b2c3d4f5a6
-Revises: p4q5r6s7t8u9
-Create Date: 2026-05-26
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "e1b2c3d4f5a6"
-down_revision: str | Sequence[str] | None = "p4q5r6s7t8u9"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-_PG_INDEXES_TO_DROP: tuple[str, ...] = (
-    "idx_memory_links_entity_covering",
-    "idx_memory_links_from_unit",
-    "idx_memory_links_to_unit",
-    "idx_memory_links_link_type",
-    "idx_entities_canonical_name",
-    "entities_canonical_name_trgm_idx",
-    "idx_documents_retain_params",
-    "idx_documents_content_hash",
-    "idx_unit_entities_entity",
-)
-
-
-def _schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _pg_upgrade() -> None:
-    schema = _schema_prefix()
-    # DROP INDEX CONCURRENTLY cannot run inside a transaction block; an
-    # autocommit_block drops out of Alembic's migration transaction so each
-    # statement runs in its own autocommit.  IF EXISTS makes each statement
-    # idempotent across schemas that already dropped (or never had) the index.
-    with op.get_context().autocommit_block():
-        for index_name in _PG_INDEXES_TO_DROP:
-            op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}{index_name}")
-
-
-def _pg_downgrade() -> None:
-    schema = _schema_prefix()
-
-    # Recreate the dropped indexes in the same shape the prior migrations used,
-    # so a downgrade leaves the schema in the state the previous head expected.
-    # CREATE INDEX CONCURRENTLY cannot run inside a transaction block.
-    with op.get_context().autocommit_block():
-        op.execute(
-            f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_links_entity_covering "
-            f"ON {schema}memory_links(from_unit_id) "
-            f"INCLUDE (to_unit_id, entity_id) "
-            f"WHERE link_type = 'entity'"
-        )
-        op.execute(
-            f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_links_from_unit ON {schema}memory_links(from_unit_id)"
-        )
-        op.execute(
-            f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_links_to_unit ON {schema}memory_links(to_unit_id)"
-        )
-        op.execute(
-            f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_links_link_type ON {schema}memory_links(link_type)"
-        )
-        op.execute(
-            f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_entities_canonical_name ON {schema}entities(canonical_name)"
-        )
-        op.execute(
-            f"CREATE INDEX CONCURRENTLY IF NOT EXISTS entities_canonical_name_trgm_idx "
-            f"ON {schema}entities USING GIN (canonical_name gin_trgm_ops)"
-        )
-        op.execute(
-            f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_documents_retain_params "
-            f"ON {schema}documents USING GIN (retain_params)"
-        )
-        op.execute(
-            f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_documents_content_hash ON {schema}documents(content_hash)"
-        )
-        op.execute(
-            f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_unit_entities_entity ON {schema}unit_entities(entity_id)"
-        )
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/e4f5a6b7c8d9_add_webhooks_tables.py

@@ -12,8 +12,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "e4f5a6b7c8d9"
 down_revision: str | Sequence[str] | None = "d2e3f4a5b6c7"
 branch_labels: str | Sequence[str] | None = None
@@ -25,7 +23,7 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     schema = _get_schema_prefix()
 
     op.execute(
@@ -56,17 +54,9 @@ def _pg_upgrade() -> None:
     )
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     schema = _get_schema_prefix()
     op.execute(f"DROP INDEX IF EXISTS {schema}idx_async_operations_status_retry")
     op.execute(f"ALTER TABLE {schema}async_operations DROP COLUMN IF EXISTS next_retry_at")
     op.execute(f"DROP INDEX IF EXISTS {schema}idx_webhooks_bank_id")
     op.execute(f"DROP TABLE IF EXISTS {schema}webhooks")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/e5f6a7b8c9d0_add_maintenance_routines.py

@@ -1,153 +0,0 @@
-"""Add server-side routines for background maintenance sweeps.
-
-Installs two PL/pgSQL discovery routines in the ``public`` schema. Both loop
-over every schema that actually holds the relevant table (via ``pg_class``), so
-a single function call covers all tenants in one round-trip instead of the
-per-tenant query storm that a client-side loop would create at thousands of
-tenants.
-
-- ``public.banks_needing_consolidation()`` -> (schema_name, bank_id) for banks
-  that have eligible-but-unscheduled facts (``consolidated_at IS NULL AND
-  consolidation_failed_at IS NULL`` for consolidatable fact types), have
-  auto-consolidation not explicitly disabled at the bank level, and have no
-  consolidation operation already pending/processing. Drives the periodic
-  reconcile that re-schedules consolidation after a terminal failure left facts
-  stranded (see HINDSIGHT_API_CONSOLIDATION_RECONCILE_INTERVAL_SECONDS).
-
-- ``public.schemas_with_expired_rows(p_table, p_ts_col, p_days)`` -> schema
-  names that hold at least one ``p_table`` row older than ``p_days``. Drives the
-  cross-tenant retention sweeps for ``audit_log`` and ``llm_requests``; the loop
-  then issues a DELETE only against the returned schemas.
-
-These are read-only (STABLE) discovery routines — the caller performs the
-enqueue/DELETE — so installing them never mutates data.
-
-PostgreSQL only — the worker poller and these tables are not wired for Oracle,
-so the Oracle slot is intentionally absent (mirrors the audit_log / llm_requests
-table migrations). The routines live in ``public`` and are CREATE OR REPLACE, so
-running this migration once per tenant schema is idempotent.
-
-Revision ID: e5f6a7b8c9d0
-Revises: a7b8c9d0e1f2
-Create Date: 2026-06-05
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "e5f6a7b8c9d0"
-down_revision: str | Sequence[str] | None = "a7b8c9d0e1f2"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _is_base_schema_run() -> bool:
-    """True only for the base-schema migration (no per-tenant target_schema).
-
-    These routines live in the shared ``public`` schema, so they must be created
-    exactly once. Running ``CREATE OR REPLACE FUNCTION public....`` again from each
-    concurrent per-tenant migration aborts with ``tuple concurrently updated`` on
-    the ``pg_proc`` catalog row, so tenant runs skip it (the base run already
-    created the function for every tenant to use).
-    """
-    return not context.config.get_main_option("target_schema")
-
-
-def _pg_upgrade() -> None:
-    if not _is_base_schema_run():
-        return
-    # Banks with eligible-but-unscheduled facts and no in-flight consolidation.
-    # Auto-consolidation is filtered here only at the bank level (cheap prune);
-    # the full hierarchical resolution (global -> tenant -> bank, plus
-    # enable_observations) is done by the caller for the small returned set.
-    op.execute(
-        """
-        CREATE OR REPLACE FUNCTION public.banks_needing_consolidation()
-        RETURNS TABLE(schema_name text, bank_id text)
-        LANGUAGE plpgsql STABLE
-        AS $fn$
-        DECLARE
-            sch text;
-        BEGIN
-            FOR sch IN
-                SELECT n.nspname
-                FROM pg_class c
-                JOIN pg_namespace n ON n.oid = c.relnamespace
-                WHERE c.relname = 'memory_units' AND c.relkind = 'r'
-            LOOP
-                RETURN QUERY EXECUTE format($q$
-                    SELECT %1$L::text, m.bank_id
-                    FROM %1$I.memory_units m
-                    JOIN %1$I.banks b ON b.bank_id = m.bank_id
-                    WHERE m.consolidated_at IS NULL
-                      AND m.consolidation_failed_at IS NULL
-                      AND m.fact_type IN ('experience', 'world')
-                      AND COALESCE(b.config -> 'enable_auto_consolidation', 'true'::jsonb) <> 'false'::jsonb
-                      AND NOT EXISTS (
-                          SELECT 1 FROM %1$I.async_operations o
-                          WHERE o.bank_id = m.bank_id
-                            AND o.operation_type = 'consolidation'
-                            AND o.status IN ('pending', 'processing')
-                      )
-                    GROUP BY m.bank_id
-                $q$, sch);
-            END LOOP;
-        END;
-        $fn$;
-        """
-    )
-
-    # Schemas holding at least one row of p_table older than p_days. p_ts_col is
-    # the timestamp column to compare. Returns nothing when p_days <= 0
-    # (retention disabled).
-    op.execute(
-        """
-        CREATE OR REPLACE FUNCTION public.schemas_with_expired_rows(
-            p_table text, p_ts_col text, p_days int
-        )
-        RETURNS SETOF text
-        LANGUAGE plpgsql STABLE
-        AS $fn$
-        DECLARE
-            sch text;
-            has_expired boolean;
-        BEGIN
-            IF p_days IS NULL OR p_days <= 0 THEN
-                RETURN;
-            END IF;
-            FOR sch IN
-                SELECT n.nspname
-                FROM pg_class c
-                JOIN pg_namespace n ON n.oid = c.relnamespace
-                WHERE c.relname = p_table AND c.relkind = 'r'
-            LOOP
-                EXECUTE format(
-                    'SELECT EXISTS (SELECT 1 FROM %I.%I WHERE %I < NOW() - make_interval(days => $1))',
-                    sch, p_table, p_ts_col
-                ) INTO has_expired USING p_days;
-                IF has_expired THEN
-                    RETURN NEXT sch;
-                END IF;
-            END LOOP;
-        END;
-        $fn$;
-        """
-    )
-
-
-def _pg_downgrade() -> None:
-    if not _is_base_schema_run():
-        return
-    op.execute("DROP FUNCTION IF EXISTS public.banks_needing_consolidation()")
-    op.execute("DROP FUNCTION IF EXISTS public.schemas_with_expired_rows(text, text, int)")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/e5f6g7h8i9j0_cascade_delete_ops_on_bank_delete.py

@@ -13,8 +13,6 @@ from collections.abc import Sequence
 
 from alembic import context, op
 
-from hindsight_api.alembic._dialect import run_for_dialect
-
 revision: str = "e5f6g7h8i9j0"
 down_revision: str | Sequence[str] | None = "d4e5f6g7h8i9"
 branch_labels: str | Sequence[str] | None = None
@@ -26,7 +24,7 @@ def _get_schema_prefix() -> str:
     return f'"{schema}".' if schema else ""
 
 
-def _pg_upgrade() -> None:
+def upgrade() -> None:
     schema = _get_schema_prefix()
 
     # Remove orphaned async_operations rows whose bank no longer exists
@@ -69,15 +67,7 @@ def _pg_upgrade() -> None:
     )
 
 
-def _pg_downgrade() -> None:
+def downgrade() -> None:
     schema = _get_schema_prefix()
     op.execute(f"ALTER TABLE {schema}async_operations DROP CONSTRAINT IF EXISTS fk_async_operations_bank_id")
     op.execute(f"ALTER TABLE {schema}webhooks DROP CONSTRAINT IF EXISTS fk_webhooks_bank_id")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)

hindsight-api-slim/hindsight_api/alembic/versions/e9b2c7d1f3a4_drop_entity_memory_links.py

@@ -1,91 +0,0 @@
-"""Drop materialized entity rows from memory_links.
-
-Entity edges are no longer stored in ``memory_links``. The /graph endpoint
-derives them on demand from ``unit_entities``, and recall already used the
-``unit_entities`` self-join. Storing entity rows duplicated state we never
-read from the link table — on a 10k-unit benchmark bank, entity rows were
-53% of all link rows (~190 MB after indexes) and recall never touched them.
-
-This migration deletes ``memory_links`` rows with ``link_type = 'entity'``.
-``idx_memory_links_entity_covering`` was already dropped by migration
-``e1b2c3d4f5a6``; we still issue ``DROP INDEX IF EXISTS`` defensively in case
-this migration runs against an older snapshot that predates that one.
-
-Revision ID: e9b2c7d1f3a4
-Revises: e1b2c3d4f5a6
-Create Date: 2026-05-26
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "e9b2c7d1f3a4"
-down_revision: str | Sequence[str] | None = "e1b2c3d4f5a6"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _pg_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _pg_upgrade() -> None:
-    schema = _pg_schema_prefix()
-
-    # Drop the partial covering index first so the bulk DELETE doesn't churn it.
-    # DROP INDEX CONCURRENTLY, and the DO block's per-batch COMMIT, both require
-    # running outside Alembic's migration transaction — an autocommit_block
-    # commits it and switches the connection to autocommit for the duration.
-    with op.get_context().autocommit_block():
-        op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_links_entity_covering")
-
-        # Delete entity rows. Chunked to keep individual transactions small on
-        # large banks (the perf-medium bench had ~345k entity rows; production
-        # banks can be much larger).
-        op.execute(
-            f"""
-            DO $$
-            DECLARE
-                deleted INTEGER;
-            BEGIN
-                LOOP
-                    DELETE FROM {schema}memory_links
-                    WHERE ctid IN (
-                        SELECT ctid FROM {schema}memory_links
-                        WHERE link_type = 'entity'
-                        LIMIT 50000
-                    );
-                    GET DIAGNOSTICS deleted = ROW_COUNT;
-                    EXIT WHEN deleted = 0;
-                    COMMIT;
-                END LOOP;
-            END$$;
-            """
-        )
-
-
-def _pg_downgrade() -> None:
-    # Cannot reconstruct deleted entity links — the writer was path-dependent
-    # on retain order. New retains will not produce entity rows either, so the
-    # partial index would stay empty. Leave both no-op.
-    pass
-
-
-def _oracle_upgrade() -> None:
-    op.execute("DELETE FROM memory_links WHERE link_type = 'entity'")
-
-
-def _oracle_downgrade() -> None:
-    pass
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade, oracle=_oracle_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade, oracle=_oracle_downgrade)