chore(dev): add one-shot dev environment setup script

Add scripts/dev/setup.sh: an idempotent bootstrap that installs the required toolchains (uv/Python, Node/npm, Rust/cargo) when missing, creates .env, configures git hooks, installs all Python + Node workspace deps, pre-downloads the local ML models + tokenizer for offline use, and builds the TypeScript SDK and Rust CLI. Flags: --skip-build, --skip-models, --with-docs, --force. Document it in CONTRIBUTING.md as the recommended setup, keeping the manual steps as a fallback.
fix(control-plane): force NODE_ENV=production for production build
2026-06-01 18:16:50 +02:00 · 2026-06-01 18:16:36 +02:00
1138 changed files with 23406 additions and 82260 deletions
@@ -73,11 +73,6 @@ results = await asyncio.gather(*tasks, return_exceptions=True)
 results = await asyncio.gather(*tasks, return_exceptions=True)
 ```

-### API Layer & Data Access
- **No direct database access in `api/http.py`** (or any API router). HTTP handlers must not build SQL, call `acquire_with_retry` / `conn.fetch` / `conn.fetchrow` / `conn.execute`, or reference `fq_table(...)`. All persistence and queries live in `MemoryEngine` (the engine layer). A handler parses/validates the request, calls an engine method, shapes the HTTP response, and maps domain results to status codes (e.g. a `None` return → 404).
- **Authentication/tenancy is enforced inside each engine method, not assumed by the handler.** Every engine method that touches bank-scoped data must authenticate via `request_context` — typically `await self._authenticate_tenant(request_context)` (often indirectly through `get_bank_profile(...)`) — so the correct tenant schema is resolved before any query runs. Handlers must thread `request_context` through to the engine method; never query a tenant-scoped table assuming the schema is already set.
- Engine methods return typed models (Pydantic/dataclass), not raw dicts (see Type Safety).
-
 ### Branch Hygiene
 - **Always start new feature branches from `origin/main`** — rebase to ensure a clean base.
 - **Only include commits relevant to the PR/branch/feature** — no unrelated changes. If the branch contains commits that don't belong, they must be removed before merging.
@@ -140,13 +135,6 @@ For each new or significantly changed function/endpoint/class:

 Flag any new logic that lacks test coverage.

-**LLM-behaviour changes need a real-LLM judge test, not MockLLM.** If the change alters how the model interprets a prompt — fact/observation extraction, `fact_type` (world/experience) classification, speaker attribution, instruction-following, prompt wording — there MUST be a test marked `pytest.mark.hs_llm_core` that runs the real pipeline and asserts via `tests.llm_judge.assert_meets_criteria` (not string/enum matching). Flag these as findings:
- A prompt/classification change verified only by MockLLM or string assertions (MockLLM echoes input — such tests pass spuriously). **Should fix.**
- A test that hard-asserts `fact_type == "world"/"experience"` (or other model-decided output) instead of judging it — non-deterministic, will flake across providers/runs. **Should fix** (move the classification check into the judge `criteria`; keep only genuinely deterministic structural asserts direct).
- Deterministic mechanics (prompt assembly, suppression/branching logic) that are covered *only* by a slow LLM test — these should also have fast non-LLM unit tests. **Note.**
-
-See CLAUDE.md → Key Conventions → Testing for the full pattern.
-
 ### 7. Check API consistency

 If any files in `hindsight-api-slim/hindsight_api/api/` were changed:
@@ -154,12 +142,6 @@ If any files in `hindsight-api-slim/hindsight_api/api/` were changed:
 - Were the client SDKs regenerated? (`./scripts/generate-clients.sh`)
 - Were the control plane proxy routes updated? (`hindsight-control-plane/src/app/api/`)

-### 7b. Check API-layer data-access boundary
-
-For each changed handler in `hindsight-api-slim/hindsight_api/api/` (e.g. `http.py`, `mcp.py`):
- **Flag any direct DB access in the handler** — `acquire_with_retry`, `conn.fetch` / `fetchrow` / `execute`, raw SQL strings, or `fq_table(...)`. These are a **must fix**: the query must be moved into a `MemoryEngine` method that returns a typed model, and the handler must call that method.
- **Verify authentication is enforced in the engine** — the handler must delegate to an engine method that authenticates via `request_context` (`_authenticate_tenant`, typically through `get_bank_profile`). A handler that reads/writes tenant-scoped data without an engine method enforcing auth is a **must fix** (tenant data could leak across schemas).
-
 ### 8. Check code comments

 For each non-trivial change:
@@ -172,8 +154,7 @@ For each non-trivial change:
 If any files in `hindsight-integrations/` were added or changed, verify:
 - **Tests exist** — the integration must have tests that simulate/exercise the external framework (not just pure unit tests of helpers). Check for a `tests/` directory with meaningful test files.
 - **CI job exists** — check `.github/workflows/test.yml` for a corresponding `test-<name>-integration` job. If missing, flag it.
- **Release process** — check that the integration name is in the `VALID_INTEGRATIONS` array in `scripts/release-integration.sh` AND in the `INTEGRATIONS` dict in `hindsight-dev/hindsight_dev/generate_changelog.py` (the changelog generator keeps its own list; a release fails at the changelog step if the name is missing there). If either is missing, flag it.
- **Docs gallery + sidebar entry** — the integration must have an entry in `hindsight-docs/src/data/integrations.json`. This file is the **single source of truth** that drives both the integrations gallery and the docs sidebar (the sidebar category is injected from it at render time across all docs versions). The entry needs an internal `/sdks/integrations/<slug>` `link` and a matching page at `hindsight-docs/docs-integrations/<slug>.md(x)`. The `hindsight-docs/scripts/check-integrations.mjs` build step enforces both directions — forward: every internal JSON entry has a doc page; reverse: every released tag (`integrations/<name>/vX.Y.Z`) appears in the JSON (private infra like `cloudflare-oauth-proxy` is in the script's `EXCLUDED` set). Flag any integration that is released (or being released) but missing from `integrations.json`, and any JSON entry without a doc page. Do **not** hand-edit `versioned_sidebars/*.json` to add integration links — they are positional placeholders filled from the JSON.
+- **Release process** — check that the integration name is in the `VALID_INTEGRATIONS` array in `scripts/release-integration.sh`. If missing, flag it.
 - **Code standards** — the integration code must follow all Python style rules (type hints, no raw dicts, no tuple returns, etc.).

 ### 10. Check MCP tool registration completeness
@@ -215,10 +196,7 @@ Present a clear summary organized by severity:
 - Raw dict usage for structured data (including internal code)
 - Multi-item tuple returns (including internal code)
 - Missing tests for new endpoints
- Direct DB access (raw SQL / `acquire_with_retry` / `fq_table`) in an `api/` handler instead of a `MemoryEngine` method
- Tenant-scoped data accessed without authentication enforced in the engine (`_authenticate_tenant` / `get_bank_profile`)
 - New integration missing tests, CI job, or release-integration.sh entry
- Released/added integration missing from `hindsight-docs/src/data/integrations.json`, or a JSON entry with no `docs-integrations/<slug>` page (fails the docs build via `check-integrations.mjs`)
 - New PostgreSQL table missing from `BACKUP_TABLES` in `admin/cli.py` (silent data loss on restore)

 **Should fix** — issues that hurt code quality:
@@ -25,7 +25,7 @@ HINDSIGHT_API_LLM_BASE_URL=https://api.openai.com/v1
 # Example: MiniMax configuration (1M context window)
 # HINDSIGHT_API_LLM_PROVIDER=minimax
 # HINDSIGHT_API_LLM_API_KEY=your-minimax-api-key
-# HINDSIGHT_API_LLM_MODEL=MiniMax-M3  # or MiniMax-M2.7 for the previous generation
+# HINDSIGHT_API_LLM_MODEL=MiniMax-M2.7

 # Example: DeepSeek configuration (https://api.deepseek.com)
 # HINDSIGHT_API_LLM_PROVIDER=deepseek
@@ -80,23 +80,10 @@ HINDSIGHT_API_LOG_LEVEL=info
 # HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER=

 # Embeddings Configuration (Optional - uses local by default)
-# Provider: "local" (default), "onnx", "tei", "openai", "cohere", "google", "openrouter", "zeroentropy", "litellm", or "litellm-sdk"
+# Provider: "local" (default), "tei", "openai", "cohere", "google", "openrouter", "zeroentropy", "litellm", or "litellm-sdk"
 # HINDSIGHT_API_EMBEDDINGS_PROVIDER=local
 # For local provider:
 # HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL=BAAI/bge-small-en-v1.5
-# For ONNX provider (local CPU embeddings without an Ollama/TEI sidecar):
-# HINDSIGHT_API_EMBEDDINGS_PROVIDER=onnx
-# HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_ID=intfloat/multilingual-e5-small
-# HINDSIGHT_API_EMBEDDINGS_ONNX_FILE=onnx/model.onnx
-# HINDSIGHT_API_EMBEDDINGS_ONNX_DIMENSIONS=384
-# HINDSIGHT_API_EMBEDDINGS_ONNX_MAX_TOKENS=512
-# HINDSIGHT_API_EMBEDDINGS_ONNX_POOLING=mean
-# HINDSIGHT_API_EMBEDDINGS_ONNX_NORMALIZE=true
-# HINDSIGHT_API_EMBEDDINGS_ONNX_QUERY_PREFIX="query: "
-# HINDSIGHT_API_EMBEDDINGS_ONNX_PASSAGE_PREFIX="passage: "
-# Optional for local model paths or pre-downloaded artifacts:
-# HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_PATH=/models/multilingual-e5-small/onnx/model.onnx
-# HINDSIGHT_API_EMBEDDINGS_ONNX_TOKENIZER_NAME_OR_PATH=/models/multilingual-e5-small
 # Optional for China network / restricted HF access:
 # HF_ENDPOINT=https://hf-mirror.com
 # For TEI provider:
@@ -159,20 +146,3 @@ HINDSIGHT_API_LOG_LEVEL=info
 # When set, visitors see a login page and must enter the key before
 # accessing the dashboard or any /api/* routes (except /api/health).
 # HINDSIGHT_CP_ACCESS_KEY=your-shared-secret-key
-
-# Optional: Token the CP forwards to the dataplane admin API (/admin/*).
-# Must match HINDSIGHT_API_ADMIN_TOKEN below. Leave unset for an open admin API.
-# HINDSIGHT_CP_ADMIN_TOKEN=your-admin-token
-
-# -----------------------------------------------------------------------------
-# Admin surface (Optional, server-level)
-# -----------------------------------------------------------------------------
-
-# Enable the admin API (GET /admin/config) and the Control Plane /admin page.
-# Off by default — the admin surface is invisible (404) until enabled.
-# HINDSIGHT_API_ENABLE_ADMIN_API=true
-
-# Optional: Require this bearer token for the admin API. When unset, the admin
-# API is open (once enabled). When set, callers must send
-# `Authorization: Bearer <token>`. Independent of the tenant API key.
-# HINDSIGHT_API_ADMIN_TOKEN=your-admin-token
@@ -22,8 +22,6 @@ jobs:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
-        with:
-          fetch-depth: 0 # fetch tags so check-released-integrations can see them
      - uses: actions/setup-node@v6
        with:
          node-version: 20
@@ -23,9 +23,7 @@ on:
          - retain
          - recall
          - recall-with-observations
-          - recall-temporal
          - consolidation
-          - graph-maintenance
        default: ""
      locomo_conversations:
        description: "LoComo conversation IDs (space-separated). Blank = curated set (conv-26 conv-30 conv-43)."
@@ -35,18 +33,6 @@ on:
        description: "Skip LoComo job"
        type: boolean
        default: false
-      obs_skip:
-        description: "Skip observation-dedup benchmark job"
-        type: boolean
-        default: false
-      obs_dataset:
-        description: "Obs benchmark dataset substring (blank = English hermes transcript)."
-        type: string
-        default: ""
-      obs_fraction:
-        description: "Obs benchmark fraction (0-1] of each document to run."
-        type: string
-        default: "1.0"
      ref:
        description: "Git ref to test (branch, tag, or SHA). Defaults to main."
        type: string
@@ -212,86 +198,3 @@ jobs:
        PERF_DASHBOARD_TOKEN: ${{ secrets.PERF_DASHBOARD_TOKEN }}
        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      run: ./scripts/benchmarks/publish-locomo-results.sh hindsight-dev/benchmarks/locomo/results/benchmark_results.json
-
-  obs:
-    # Observation-dedup quality benchmark: ingests a transcript, drains consolidation
-    # (serial SyncTaskBackend + embedded pg0 — no external DB / worker), and reports the
-    # near-duplicate observation rate. Real LLM via VertexAI, mirroring the LoComo job.
-    if: inputs.obs_skip != true
-    runs-on: ubuntu-latest
-    env:
-      HINDSIGHT_API_LLM_PROVIDER: vertexai
-      HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
-      HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
-      HINDSIGHT_API_ENABLE_OBSERVATIONS: "true"
-    steps:
-    - uses: actions/checkout@v6
-      with:
-        ref: ${{ inputs.ref || github.ref }}
-
-    - name: Setup GCP credentials
-      run: |
-        printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
-        PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
-        echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
-
-    - name: Install uv
-      uses: astral-sh/setup-uv@v7
-      with:
-        enable-cache: true
-        prune-cache: false
-
-    - name: Set up Python
-      uses: actions/setup-python@v6
-      with:
-        python-version-file: ".python-version"
-
-    - name: Cache HuggingFace models
-      uses: actions/cache@v5
-      with:
-        path: ~/.cache/huggingface
-        key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
-        restore-keys: |
-          ${{ runner.os }}-huggingface-
-
-    - name: Pre-download models
-      working-directory: ./hindsight-api-slim
-      run: |
-        uv run --frozen --all-extras --index-strategy unsafe-best-match python -c "
-        from sentence_transformers import SentenceTransformer
-        print('Downloading embedding model...')
-        SentenceTransformer('BAAI/bge-small-en-v1.5')
-        print('Model downloaded successfully')
-        "
-
-    - name: Install hindsight-dev dependencies
-      run: |
-        cd hindsight-dev && uv sync --frozen --all-extras --index-strategy unsafe-best-match
-
-    - name: Run obs benchmark
-      # Default to the English hermes transcript at full fraction — a clean, deterministic
-      # consolidation-dedup signal (the Chinese variant adds a cross-lingual embedding
-      # confound). Override dataset/fraction via workflow_dispatch.
-      run: |
-        DATASET="${{ inputs.obs_dataset }}"
-        if [ -z "$DATASET" ]; then DATASET="hermes_session_2026-05-15_en"; fi
-        FRACTION="${{ inputs.obs_fraction }}"
-        if [ -z "$FRACTION" ]; then FRACTION="1.0"; fi
-        cd hindsight-dev
-        uv run python -m benchmarks.obs.obs_benchmark \
-          --dataset "$DATASET" --fraction "$FRACTION" --wipe-bank --output obs-results.json
-
-    - name: Upload obs results
-      if: always()
-      uses: actions/upload-artifact@v7
-      with:
-        name: obs-results-${{ github.sha }}
-        path: hindsight-dev/obs-results.json
-        retention-days: 90
-
-    - name: Publish obs to dashboard
-      if: success() && (github.event_name == 'schedule' || github.event_name == 'push' || github.event_name == 'workflow_dispatch')
-      env:
-        PERF_DASHBOARD_TOKEN: ${{ secrets.PERF_DASHBOARD_TOKEN }}
-        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      run: ./scripts/benchmarks/publish-obs-results.sh hindsight-dev/obs-results.json
@@ -10,7 +10,6 @@ jobs:
    runs-on: ubuntu-latest
    permissions:
      id-token: write  # for PyPI trusted publishing
-      contents: write  # for creating GitHub releases (Obsidian plugin assets)

    steps:
    - uses: actions/checkout@v6
@@ -113,26 +112,6 @@ jobs:
      working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
      run: npm run build

-    # ── Obsidian plugin — attach BRAT / community-store install assets ───────
-    # Obsidian plugins install from GitHub *release assets* (main.js,
-    # manifest.json, styles.css), not from npm — the npm publish below only
-    # gives us a versioned artifact. BRAT and the community store read these
-    # three files off the release for the tag.
-    - name: Attach Obsidian release assets
-      if: steps.type.outputs.type == 'typescript' && steps.info.outputs.integration == 'obsidian'
-      working-directory: ./hindsight-integrations/obsidian
-      env:
-        GH_TOKEN: ${{ github.token }}
-      run: |
-        TAG="${{ steps.info.outputs.tag }}"
-        if gh release view "$TAG" >/dev/null 2>&1; then
-          gh release upload "$TAG" main.js manifest.json styles.css --clobber
-        else
-          gh release create "$TAG" main.js manifest.json styles.css \
-            --title "Obsidian plugin v${{ steps.info.outputs.version }}" \
-            --notes "Hindsight Obsidian plugin v${{ steps.info.outputs.version }}. Install via BRAT (point it at this release) or copy main.js/manifest.json/styles.css into <vault>/.obsidian/plugins/hindsight/."
-        fi
-
    - name: Publish TypeScript package to npm
      if: steps.type.outputs.type == 'typescript'
      working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
@@ -34,35 +34,25 @@ jobs:
      integrations-ai-sdk: ${{ steps.filter.outputs.integrations-ai-sdk }}
      integrations-chat: ${{ steps.filter.outputs.integrations-chat }}
      integrations-claude-code: ${{ steps.filter.outputs.integrations-claude-code }}
-      integrations-cline: ${{ steps.filter.outputs.integrations-cline }}
      integrations-codex: ${{ steps.filter.outputs.integrations-codex }}
-      integrations-cursor-cli: ${{ steps.filter.outputs.integrations-cursor-cli }}
      integrations-crewai: ${{ steps.filter.outputs.integrations-crewai }}
      integrations-litellm: ${{ steps.filter.outputs.integrations-litellm }}
      integrations-pydantic-ai: ${{ steps.filter.outputs.integrations-pydantic-ai }}
      integrations-ag2: ${{ steps.filter.outputs.integrations-ag2 }}
-      integrations-autogen: ${{ steps.filter.outputs.integrations-autogen }}
-      integrations-langgraph: ${{ steps.filter.outputs.integrations-langgraph }}
      integrations-llamaindex: ${{ steps.filter.outputs.integrations-llamaindex }}
      integrations-paperclip: ${{ steps.filter.outputs.integrations-paperclip }}
      integrations-opencode: ${{ steps.filter.outputs.integrations-opencode }}
      integrations-n8n: ${{ steps.filter.outputs.integrations-n8n }}
      integrations-cloudflare-oauth-proxy: ${{ steps.filter.outputs.integrations-cloudflare-oauth-proxy }}
-      integrations-superagent: ${{ steps.filter.outputs.integrations-superagent }}
      integrations-lockfiles: ${{ steps.filter.outputs.integrations-lockfiles }}
      integrations-openai-agents: ${{ steps.filter.outputs.integrations-openai-agents }}
      integrations-pipecat: ${{ steps.filter.outputs.integrations-pipecat }}
      integrations-agentcore: ${{ steps.filter.outputs.integrations-agentcore }}
      integrations-smolagents: ${{ steps.filter.outputs.integrations-smolagents }}
-      integrations-claude-agent-sdk: ${{ steps.filter.outputs.integrations-claude-agent-sdk }}
      integrations-dify: ${{ steps.filter.outputs.integrations-dify }}
      integrations-gemini-spark: ${{ steps.filter.outputs.integrations-gemini-spark }}
      integrations-vapi: ${{ steps.filter.outputs.integrations-vapi }}
      integrations-flowise: ${{ steps.filter.outputs.integrations-flowise }}
-      integrations-google-adk: ${{ steps.filter.outputs.integrations-google-adk }}
-      integrations-obsidian: ${{ steps.filter.outputs.integrations-obsidian }}
-      integrations-omo: ${{ steps.filter.outputs.integrations-omo }}
-      integrations-haystack: ${{ steps.filter.outputs.integrations-haystack }}
      tools-agent-sdk: ${{ steps.filter.outputs.tools-agent-sdk }}
      integrations-roo-code: ${{ steps.filter.outputs.integrations-roo-code }}
      dev: ${{ steps.filter.outputs.dev }}
@@ -106,9 +96,6 @@ jobs:
          docs:
            - 'hindsight-docs/**'
            - '*.md'
-            # Integration changes can add/rename integrations, which the docs
-            # build's integrations check validates against integrations.json.
-            - 'hindsight-integrations/**'
          embed:
            - 'hindsight-embed/**'
          all-npm:
@@ -127,12 +114,8 @@ jobs:
            - 'hindsight-integrations/chat/**'
          integrations-claude-code:
            - 'hindsight-integrations/claude-code/**'
-          integrations-cline:
-            - 'hindsight-integrations/cline/**'
          integrations-codex:
            - 'hindsight-integrations/codex/**'
-          integrations-cursor-cli:
-            - 'hindsight-integrations/cursor-cli/**'
          integrations-crewai:
            - 'hindsight-integrations/crewai/**'
          integrations-litellm:
@@ -141,14 +124,8 @@ jobs:
            - 'hindsight-integrations/pydantic-ai/**'
          integrations-ag2:
            - 'hindsight-integrations/ag2/**'
-          integrations-autogen:
-            - 'hindsight-integrations/autogen/**'
-          integrations-langgraph:
-            - 'hindsight-integrations/langgraph/**'
          integrations-llamaindex:
            - 'hindsight-integrations/llamaindex/**'
-          integrations-haystack:
-            - 'hindsight-integrations/haystack/**'
          integrations-paperclip:
            - 'hindsight-integrations/paperclip/**'
          integrations-opencode:
@@ -157,8 +134,6 @@ jobs:
            - 'hindsight-integrations/n8n/**'
          integrations-cloudflare-oauth-proxy:
            - 'hindsight-integrations/cloudflare-oauth-proxy/**'
-          integrations-superagent:
-            - 'hindsight-integrations/superagent/**'
          integrations-lockfiles:
            - 'hindsight-integrations/*/package-lock.json'
            - 'hindsight-integrations/*/package.json'
@@ -171,8 +146,6 @@ jobs:
            - 'hindsight-integrations/agentcore/**'
          integrations-smolagents:
            - 'hindsight-integrations/smolagents/**'
-          integrations-claude-agent-sdk:
-            - 'hindsight-integrations/claude-agent-sdk/**'
          integrations-dify:
            - 'hindsight-integrations/dify/**'
          integrations-gemini-spark:
@@ -181,12 +154,6 @@ jobs:
            - 'hindsight-integrations/vapi/**'
          integrations-flowise:
            - 'hindsight-integrations/flowise/**'
-          integrations-google-adk:
-            - 'hindsight-integrations/google-adk/**'
-          integrations-obsidian:
-            - 'hindsight-integrations/obsidian/**'
-          integrations-omo:
-            - 'hindsight-integrations/omo/**'
          tools-agent-sdk:
            - 'hindsight-tools/hindsight-agent-sdk/**'
          integrations-roo-code:
@@ -447,58 +414,6 @@ jobs:
      working-directory: ./hindsight-integrations/claude-code
      run: python -m pytest tests/ -v

-  test-omo-integration:
-    needs: [detect-changes]
-    if: >-
-      (github.event_name == 'workflow_dispatch' ||
-      needs.detect-changes.outputs.integrations-omo == 'true' ||
-      needs.detect-changes.outputs.ci == 'true')
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
-
-    steps:
-    - uses: actions/checkout@v6
-      with:
-        ref: ${{ github.event.pull_request.head.sha || '' }}
-
-    - name: Set up Python
-      uses: actions/setup-python@v6
-      with:
-        python-version: '3.11'
-
-    - name: Install pytest
-      run: pip install pytest
-
-    - name: Run tests
-      working-directory: ./hindsight-integrations/omo
-      run: python -m pytest tests/ -v
-
-  test-cline-integration:
-    needs: [detect-changes]
-    if: >-
-      (github.event_name == 'workflow_dispatch' ||
-      needs.detect-changes.outputs.integrations-cline == 'true' ||
-      needs.detect-changes.outputs.ci == 'true')
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
-
-    steps:
-    - uses: actions/checkout@v6
-      with:
-        ref: ${{ github.event.pull_request.head.sha || '' }}
-
-    - name: Set up Python
-      uses: actions/setup-python@v6
-      with:
-        python-version: '3.11'
-
-    - name: Install pytest
-      run: pip install pytest
-
-    - name: Run tests
-      working-directory: ./hindsight-integrations/cline
-      run: python -m pytest tests/ -v
-
  test-codex-integration:
    needs: [detect-changes]
    if: >-
@@ -525,32 +440,6 @@ jobs:
      working-directory: ./hindsight-integrations/codex
      run: python -m pytest tests/ -v

-  test-cursor-cli-integration:
-    needs: [detect-changes]
-    if: >-
-      (github.event_name == 'workflow_dispatch' ||
-      needs.detect-changes.outputs.integrations-cursor-cli == 'true' ||
-      needs.detect-changes.outputs.ci == 'true')
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
-
-    steps:
-    - uses: actions/checkout@v6
-      with:
-        ref: ${{ github.event.pull_request.head.sha || '' }}
-
-    - name: Set up Python
-      uses: actions/setup-python@v6
-      with:
-        python-version: '3.11'
-
-    - name: Install pytest
-      run: pip install pytest
-
-    - name: Run tests
-      working-directory: ./hindsight-integrations/cursor-cli
-      run: python -m pytest tests/ -v
-
  build-ai-sdk-integration:
    needs: [detect-changes]
    if: >-
@@ -839,43 +728,6 @@ jobs:
      working-directory: ./hindsight-integrations/pipecat
      run: uv run pytest tests -v

-  test-google-adk-integration:
-    needs: [detect-changes]
-    if: >-
-      (github.event_name == 'workflow_dispatch' ||
-      needs.detect-changes.outputs.integrations-google-adk == 'true' ||
-      needs.detect-changes.outputs.ci == 'true')
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
-
-    steps:
-    - uses: actions/checkout@v6
-      with:
-        ref: ${{ github.event.pull_request.head.sha || '' }}
-
-    - name: Install uv
-      uses: astral-sh/setup-uv@v7
-      with:
-        enable-cache: true
-        prune-cache: false
-
-    - name: Set up Python
-      uses: actions/setup-python@v6
-      with:
-        python-version-file: ".python-version"
-
-    - name: Build google-adk integration
-      working-directory: ./hindsight-integrations/google-adk
-      run: uv build
-
-    - name: Install dependencies
-      working-directory: ./hindsight-integrations/google-adk
-      run: uv sync --frozen
-
-    - name: Run tests
-      working-directory: ./hindsight-integrations/google-adk
-      run: uv run pytest tests -v
-
  test-gemini-spark-integration:
    needs: [detect-changes]
    if: >-
@@ -924,28 +776,17 @@ jobs:
      with:
        ref: ${{ github.event.pull_request.head.sha || '' }}

-    - name: Install uv
-      uses: astral-sh/setup-uv@v7
-      with:
-        enable-cache: true
-        prune-cache: false
-
    - name: Set up Python
      uses: actions/setup-python@v6
      with:
-        python-version-file: ".python-version"
+        python-version: '3.11'

-    - name: Build roo-code integration
-      working-directory: ./hindsight-integrations/roo-code
-      run: uv build
-
-    - name: Install dependencies
-      working-directory: ./hindsight-integrations/roo-code
-      run: uv sync --frozen
+    - name: Install pytest
+      run: pip install pytest

    - name: Run tests
      working-directory: ./hindsight-integrations/roo-code
-      run: uv run pytest tests -v
+      run: python -m pytest tests/ -v

  build-control-plane:
    needs: [detect-changes]
@@ -1027,7 +868,6 @@ jobs:
    - uses: actions/checkout@v6
      with:
        ref: ${{ github.event.pull_request.head.sha || '' }}
-        fetch-depth: 0 # fetch tags so check-released-integrations can see them

    - name: Set up Node.js
      uses: actions/setup-node@v6
@@ -1036,12 +876,6 @@ jobs:
        cache: 'npm'
        cache-dependency-path: package-lock.json

-    # Fail fast before the (slow) build: every integrations.json entry must have a
-    # doc page, and every released integration tag must be in integrations.json.
-    # Needs no npm install (pure Node) and uses the tags fetched above.
-    - name: Check integrations (single source of truth)
-      run: node hindsight-docs/scripts/check-integrations.mjs
-
    - name: Install dependencies
      run: npm ci --workspace=hindsight-docs

@@ -2927,45 +2761,6 @@ jobs:
      working-directory: ./hindsight-integrations/ag2
      run: uv run pytest tests -v

-  test-autogen-integration:
-    needs: [detect-changes]
-    if: >-
-      (github.event_name == 'workflow_dispatch' ||
-      needs.detect-changes.outputs.integrations-autogen == 'true' ||
-      needs.detect-changes.outputs.ci == 'true')
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
-
-    steps:
-    - uses: actions/checkout@v6
-      with:
-        ref: ${{ github.event.pull_request.head.sha || '' }}
-
-    - name: Install uv
-      uses: astral-sh/setup-uv@v7
-      with:
-        enable-cache: true
-        prune-cache: false
-
-    - name: Set up Python
-      uses: actions/setup-python@v6
-      with:
-        python-version-file: ".python-version"
-
-    - name: Build autogen integration
-      working-directory: ./hindsight-integrations/autogen
-      run: uv build
-
-    - name: Install dependencies
-      working-directory: ./hindsight-integrations/autogen
-      run: uv sync --frozen
-
-    - name: Run tests
-      working-directory: ./hindsight-integrations/autogen
-      # PR CI runs only the deterministic bucket; the real-LLM E2E bucket
-      # (requires_real_llm) needs a live Hindsight server and runs separately.
-      run: uv run pytest tests -v -m "not requires_real_llm"
-
  test-smolagents-integration:
    needs: [detect-changes]
    if: >-
@@ -3069,41 +2864,6 @@ jobs:
      working-directory: ./hindsight-integrations/flowise
      run: npm test

-  test-obsidian-integration:
-    needs: [detect-changes]
-    if: >-
-      github.event_name != 'pull_request_review' &&
-      (github.event_name == 'workflow_dispatch' ||
-      needs.detect-changes.outputs.integrations-obsidian == 'true' ||
-      needs.detect-changes.outputs.ci == 'true')
-    runs-on: ubuntu-latest
-
-    steps:
-    - uses: actions/checkout@v6
-      with:
-        ref: ${{ github.event.pull_request.head.sha || '' }}
-
-    - name: Set up Node.js
-      uses: actions/setup-node@v6
-      with:
-        node-version: '22'
-
-    - name: Install dependencies
-      working-directory: ./hindsight-integrations/obsidian
-      run: npm install --no-audit --no-fund
-
-    - name: Type check
-      working-directory: ./hindsight-integrations/obsidian
-      run: npx tsc --noEmit
-
-    - name: Build
-      working-directory: ./hindsight-integrations/obsidian
-      run: npm run build
-
-    - name: Run tests
-      working-directory: ./hindsight-integrations/obsidian
-      run: npm test
-
  test-crewai-integration:
    needs: [detect-changes]
    if: >-
@@ -3179,45 +2939,6 @@ jobs:
      working-directory: ./hindsight-integrations/vapi
      run: uv run pytest tests -v

-  test-superagent-integration:
-    needs: [detect-changes]
-    if: >-
-      github.event_name != 'pull_request_review' &&
-      (github.event_name == 'workflow_dispatch' ||
-      needs.detect-changes.outputs.integrations-superagent == 'true' ||
-      needs.detect-changes.outputs.ci == 'true')
-    runs-on: ubuntu-latest
-
-    steps:
-    - uses: actions/checkout@v6
-      with:
-        ref: ${{ github.event.pull_request.head.sha || '' }}
-
-    - name: Install uv
-      uses: astral-sh/setup-uv@v7
-      with:
-        enable-cache: true
-        prune-cache: false
-
-    - name: Set up Python
-      uses: actions/setup-python@v6
-      with:
-        python-version-file: ".python-version"
-
-    - name: Build superagent integration
-      working-directory: ./hindsight-integrations/superagent
-      run: uv build
-
-    - name: Install dependencies
-      working-directory: ./hindsight-integrations/superagent
-      run: uv sync --frozen
-
-    - name: Run tests
-      working-directory: ./hindsight-integrations/superagent
-      # PR CI runs only the deterministic bucket; the real-LLM E2E bucket
-      # (requires_real_llm) needs live Hindsight + provider keys and runs separately.
-      run: uv run pytest tests -v -m "not requires_real_llm"
-
  test-litellm-integration:
    needs: [detect-changes]
    if: >-
@@ -3253,9 +2974,7 @@ jobs:

    - name: Run tests
      working-directory: ./hindsight-integrations/litellm
-      # PR CI runs only the deterministic bucket; the real-LLM E2E bucket
-      # (requires_real_llm) needs live Hindsight + provider keys and runs separately.
-      run: uv run pytest tests -v -m "not requires_real_llm"
+      run: uv run pytest tests -v

  test-pydantic-ai-integration:
    needs: [detect-changes]
@@ -3294,45 +3013,6 @@ jobs:
      working-directory: ./hindsight-integrations/pydantic-ai
      run: uv run pytest tests -v

-  test-langgraph-integration:
-    needs: [detect-changes]
-    if: >-
-      (github.event_name == 'workflow_dispatch' ||
-      needs.detect-changes.outputs.integrations-langgraph == 'true' ||
-      needs.detect-changes.outputs.ci == 'true')
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
-
-    steps:
-    - uses: actions/checkout@v6
-      with:
-        ref: ${{ github.event.pull_request.head.sha || '' }}
-
-    - name: Install uv
-      uses: astral-sh/setup-uv@v7
-      with:
-        enable-cache: true
-        prune-cache: false
-
-    - name: Set up Python
-      uses: actions/setup-python@v6
-      with:
-        python-version-file: ".python-version"
-
-    - name: Build langgraph integration
-      working-directory: ./hindsight-integrations/langgraph
-      run: uv build
-
-    - name: Install dependencies
-      working-directory: ./hindsight-integrations/langgraph
-      run: uv sync --frozen
-
-    - name: Run tests
-      working-directory: ./hindsight-integrations/langgraph
-      # PR CI runs only the deterministic bucket; the real-LLM E2E bucket
-      # (requires_real_llm) needs a live Hindsight server and runs separately.
-      run: uv run pytest tests -v -m "not requires_real_llm"
-
  test-llamaindex-integration:
    needs: [detect-changes]
    if: >-
@@ -3368,48 +3048,7 @@ jobs:

    - name: Run tests
      working-directory: ./hindsight-integrations/llamaindex
-      # PR CI runs only the deterministic bucket; the real-LLM E2E bucket
-      # (requires_real_llm) needs a live Hindsight server and runs separately.
-      run: uv run pytest tests -v -m "not requires_real_llm"
-
-  test-haystack-integration:
-    needs: [detect-changes]
-    if: >-
-      github.event_name != 'pull_request_review' &&
-      (github.event_name == 'workflow_dispatch' ||
-      needs.detect-changes.outputs.integrations-haystack == 'true' ||
-      needs.detect-changes.outputs.ci == 'true')
-    runs-on: ubuntu-latest
-
-    steps:
-    - uses: actions/checkout@v6
-      with:
-        ref: ${{ github.event.pull_request.head.sha || '' }}
-
-    - name: Install uv
-      uses: astral-sh/setup-uv@v7
-      with:
-        enable-cache: true
-        prune-cache: false
-
-    - name: Set up Python
-      uses: actions/setup-python@v6
-      with:
-        python-version-file: ".python-version"
-
-    - name: Build haystack integration
-      working-directory: ./hindsight-integrations/haystack
-      run: uv build
-
-    - name: Install dependencies
-      working-directory: ./hindsight-integrations/haystack
-      run: uv sync --frozen
-
-    - name: Run tests
-      working-directory: ./hindsight-integrations/haystack
-      # PR CI runs only the deterministic bucket; the real-LLM E2E bucket
-      # (requires_real_llm) needs a live Hindsight server and runs separately.
-      run: uv run pytest tests -v -m "not requires_real_llm"
+      run: uv run pytest tests -v

  test-openai-agents-integration:
    needs: [detect-changes]
@@ -3446,47 +3085,7 @@ jobs:

    - name: Run tests
      working-directory: ./hindsight-integrations/openai-agents
-      # PR CI runs only the deterministic bucket; the real-LLM E2E bucket
-      # (requires_real_llm) needs a live Hindsight server and runs separately.
-      run: uv run pytest tests -v -m "not requires_real_llm"
-
-  test-claude-agent-sdk-integration:
-    needs: [detect-changes]
-    if: >-
-      (github.event_name == 'workflow_dispatch' ||
-      needs.detect-changes.outputs.integrations-claude-agent-sdk == 'true' ||
-      needs.detect-changes.outputs.ci == 'true')
-    runs-on: ubuntu-latest
-
-    steps:
-    - uses: actions/checkout@v6
-      with:
-        ref: ${{ github.event.pull_request.head.sha || '' }}
-
-    - name: Install uv
-      uses: astral-sh/setup-uv@v7
-      with:
-        enable-cache: true
-        prune-cache: false
-
-    - name: Set up Python
-      uses: actions/setup-python@v6
-      with:
-        python-version-file: ".python-version"
-
-    - name: Build claude-agent-sdk integration
-      working-directory: ./hindsight-integrations/claude-agent-sdk
-      run: uv build
-
-    - name: Install dependencies
-      working-directory: ./hindsight-integrations/claude-agent-sdk
-      run: uv sync --frozen
-
-    - name: Run tests
-      working-directory: ./hindsight-integrations/claude-agent-sdk
-      # PR CI runs only the deterministic bucket; the real-LLM E2E bucket
-      # (requires_real_llm) needs a live Hindsight server and runs separately.
-      run: uv run pytest tests -v -m "not requires_real_llm"
+      run: uv run pytest tests -v

  test-agentcore-integration:
    needs: [detect-changes]
@@ -4351,20 +3950,16 @@ jobs:
      - build-openclaw-integration
      - smoke-openclaw-install
      - test-claude-code-integration
-      - test-cline-integration
      - test-codex-integration
-      - test-cursor-cli-integration
      - build-ai-sdk-integration
      - test-ai-sdk-integration-deno
      - test-opencode-integration
-      - test-omo-integration
      - test-cloudflare-oauth-proxy-integration
      - build-chat-integration
      - test-paperclip-integration
      - test-pipecat-integration
      - test-gemini-spark-integration
      - test-vapi-integration
-      - test-google-adk-integration
      - test-roo-code-integration
      - build-control-plane
      - build-docs
@@ -4385,26 +3980,19 @@ jobs:
      - test-openclaw-integration
      - test-integration
      - test-ag2-integration
-      - test-autogen-integration
      - test-smolagents-integration
      - test-dify-integration
      - test-flowise-integration
-      - test-obsidian-integration
      - test-crewai-integration
-      - test-langgraph-integration
-      - test-superagent-integration
      - test-litellm-integration
      - test-pydantic-ai-integration
      - test-llamaindex-integration
-      - test-openai-agents-integration
      - test-agentcore-integration
-      - test-haystack-integration
      - test-pip-slim
      - test-embed
      - test-embed-windows
      - test-hindsight-all
      - test-hindsight-agent-sdk
-      - test-claude-agent-sdk-integration
      - test-doc-examples
      - test-upgrade
      - verify-generated-files
@@ -59,5 +59,4 @@ hindsight-integrations/_drafts/
 # Changelog is now tracked in hindsight-docs/src/pages/changelog.md
 # CHANGELOG.md

-blog-post*
-.worktrees/
+blog-post*
@@ -220,30 +220,6 @@ migration file dispatches through `run_for_dialect`, which calls either

 **MANDATORY: Run `/code-review` before pushing code or creating a pull request.** Do not push or create a PR until all "must fix" issues are resolved.

-### Testing
-
-Most tests are deterministic (MockLLM, pure functions) — assert directly.
-
-**Tests that verify LLM behaviour use a real LLM + an LLM-as-judge.** When the thing under test is *how the model interprets a prompt* (classification, attribution, dimension preservation, instruction-following), MockLLM can't simulate it and exact string/enum asserts flake across providers and runs. Use this pattern instead:
-
-1. Mark the test module `pytestmark = pytest.mark.hs_llm_core` (single-provider; CI runs it in the core-LLM job). Use `hs_llm_mat` only for provider-matrix acceptance tests.
-2. Call the real pipeline (`LLMConfig.from_env()`, `_get_raw_config()`), e.g. `extract_facts_from_text(...)`.
-3. Assert with the judge, not string matching:
-   ```python
-   from tests.llm_judge import assert_meets_criteria
-   facts_summary = "\n".join(f"- [{f.fact_type}] {f.fact}" for f in facts)
-   await assert_meets_criteria(
-       response=facts_summary,
-       criteria="The first-person user statements are classified 'world' and attributed to the user, not the agent.",
-       context="What the input said and who was speaking.",
-   )
-   ```
-
-Rules of thumb:
- **Judge anything non-deterministic** — including `fact_type` classification and speaker attribution. Do NOT hard-assert `fact_type == "..."`; pass a `[fact_type] fact` summary to the judge instead. Structural facts that ARE deterministic (counts, presence of a field, that a substring was injected into a prompt) stay as direct asserts in fast unit tests.
- **Split the test surface**: cover the deterministic mechanics (prompt assembly, suppression logic) with fast non-LLM unit tests, and the model-following behaviour with one `hs_llm_core` judge test. (Example pair: `test_narrator_resolution.py` + `test_narrator_context_override.py`.)
- The judge model is independent of the test provider (defaults to Gemini); never judge with the same call you're testing.
-
 ### Memory Banks
 - Each bank is an isolated memory store (like a "brain" for one user/agent)
 - Banks have dispositions (skepticism, literalism, empathy traits 1-5) affecting reflect
@@ -62,9 +62,9 @@ If you need more control over how and when your agent stores and recalls memorie
 ```bash
 export OPENAI_API_KEY=sk-xxx

-docker run -it --pull always --name hindsight --restart unless-stopped -p 8888:8888 -p 9999:9999 \
+docker run --rm -it --pull always -p 8888:8888 -p 9999:9999 \
  -e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \
-  -v hindsight-data:/home/hindsight/.pg0 \
+  -v $HOME/.hindsight-docker:/home/hindsight/.pg0 \
  ghcr.io/vectorize-io/hindsight:latest
 ```

@@ -50,8 +50,6 @@ WORKDIR /app/api
 # Sync dependencies using appropriate extras based on INCLUDE_LOCAL_MODELS
 # local-ml: torch, sentence-transformers, transformers, einops, flashrank, mlx (optional)
 # embedded-db: pg0-embedded (always included for embedded PostgreSQL support)
-# ONNX Runtime embeddings are intentionally not bundled into the official
-# standalone image; install the local-onnx extra in custom images when needed.
 RUN if [ "$INCLUDE_LOCAL_MODELS" = "true" ]; then \
        uv sync --extra local-ml --extra embedded-db; \
    else \
@@ -43,56 +43,11 @@ check_pg0_data_integrity() {
    return 0
 }

-# =============================================================================
-# Embedded pg0 writability pre-check (#1483)
-#
-# The container runs as the unprivileged `hindsight` user (UID 1000). When the
-# pg0 data directory is a host bind mount (e.g. `-v $HOME/dir:/home/hindsight/.pg0`)
-# that is not owned by UID 1000 — the default on macOS Docker Desktop and most
-# non-1000 Linux hosts — pg0 fails with the opaque "Permission denied (os error
-# 13)". We cannot chown it ourselves without root (and the image is deliberately
-# rootless), so we surface an actionable message up front instead.
-#
-# Docker *named* volumes are seeded with the image directory's ownership (UID
-# 1000) on first use, so they avoid this entirely — hence the named-volume
-# recommendation below and in the README.
-# =============================================================================
-check_pg0_writable() {
-    local pg0_data_dir="$1"
-
-    # Only relevant for embedded pg0; an external database doesn't use this dir.
-    if [ -n "${HINDSIGHT_API_DATABASE_URL:-}" ]; then
-        return 0
-    fi
-
-    mkdir -p "$pg0_data_dir" 2>/dev/null || true
-    if touch "$pg0_data_dir/.hindsight-write-test" 2>/dev/null; then
-        rm -f "$pg0_data_dir/.hindsight-write-test" 2>/dev/null || true
-        return 0
-    fi
-
-    echo "❌ The embedded database directory $pg0_data_dir is not writable by this container (UID $(id -u))."
-    echo ""
-    echo "   A host directory was bind-mounted but is not owned by the container user (UID 1000)."
-    echo "   Hindsight runs rootless and cannot fix this for you. Choose one:"
-    echo ""
-    echo "   • Recommended — use a Docker named volume (auto-owned by the container):"
-    echo "       -v hindsight-data:/home/hindsight/.pg0"
-    echo ""
-    echo "   • Or keep the host path and run as your host user, chowning it to match:"
-    echo "       sudo chown -R \$(id -u):\$(id -g) <host-directory>"
-    echo "       docker run --user \$(id -u):\$(id -g) -e HOME=/home/hindsight ..."
-    echo ""
-    echo "   See https://github.com/vectorize-io/hindsight/issues/1483"
-    return 1
-}
-
 if [ "${HINDSIGHT_START_ALL_SOURCE_ONLY:-false}" = "true" ]; then
    return 0 2>/dev/null || exit 0
 fi

 check_pg0_data_integrity "${HOME}/.pg0"
-check_pg0_writable "${HOME}/.pg0" || exit 1

 # Service flags (default to true if not set)
 ENABLE_API="${HINDSIGHT_ENABLE_API:-true}"
@@ -8,7 +8,7 @@ source "$SCRIPT_DIR/start-all.sh"
 unset HINDSIGHT_START_ALL_SOURCE_ONLY

 TMP_DIR="$(mktemp -d)"
-trap 'chmod -R u+rwx "$TMP_DIR" 2>/dev/null || true; rm -rf "$TMP_DIR"' EXIT
+trap 'rm -rf "$TMP_DIR"' EXIT

 assert_contains() {
    local output="$1"
@@ -71,51 +71,3 @@ nonempty_output="$(check_pg0_data_integrity "$TMP_DIR/nonempty")"
 assert_contains "$nonempty_output" "WARNING: pg0 data directory exists"

 echo "start-all pg0 integrity checks passed"
-
-# =============================================================================
-# check_pg0_writable (#1483)
-# These rely on filesystem permissions, which root bypasses; skip under root.
-# =============================================================================
-if [ "$(id -u)" != "0" ]; then
-    # Writable directory: returns 0, prints nothing, leaves no artifact behind.
-    mkdir -p "$TMP_DIR/writable"
-    writable_output="$(check_pg0_writable "$TMP_DIR/writable")"
-    assert_empty "$writable_output"
-    if [ -e "$TMP_DIR/writable/.hindsight-write-test" ]; then
-        echo "check_pg0_writable left its write-test file behind"
-        exit 1
-    fi
-
-    # Non-writable directory: returns 1 with actionable guidance.
-    mkdir -p "$TMP_DIR/readonly"
-    chmod 000 "$TMP_DIR/readonly"
-    set +e
-    readonly_output="$(check_pg0_writable "$TMP_DIR/readonly" 2>&1)"
-    readonly_rc=$?
-    set -e
-    chmod 755 "$TMP_DIR/readonly"
-    if [ "$readonly_rc" -eq 0 ]; then
-        echo "check_pg0_writable should fail on a non-writable directory"
-        exit 1
-    fi
-    assert_contains "$readonly_output" "not writable"
-    assert_contains "$readonly_output" "hindsight-data:/home/hindsight/.pg0"
-    assert_contains "$readonly_output" "--user"
-
-    # External database configured: skip the check regardless of dir perms.
-    mkdir -p "$TMP_DIR/extdb"
-    chmod 000 "$TMP_DIR/extdb"
-    set +e
-    HINDSIGHT_API_DATABASE_URL="postgres://x" check_pg0_writable "$TMP_DIR/extdb" >/dev/null 2>&1
-    extdb_rc=$?
-    set -e
-    chmod 755 "$TMP_DIR/extdb"
-    if [ "$extdb_rc" -ne 0 ]; then
-        echo "check_pg0_writable should skip when an external database is configured"
-        exit 1
-    fi
-
-    echo "start-all pg0 writability checks passed"
-else
-    echo "⚠️  Running as root; skipping pg0 writability checks (permissions are bypassed)."
-fi
@@ -2,8 +2,8 @@ apiVersion: v2
 name: hindsight
 description: Hindsight helm chart
 type: application
-version: 0.8.0
-appVersion: "0.8.0"
+version: 0.7.1
+appVersion: "0.7.1"
 keywords:
  - ai
  - memory
@@ -1,6 +1,6 @@
 {
  "name": "@vectorize-io/hindsight-all",
-  "version": "0.8.0",
+  "version": "0.7.1",
  "description": "Node.js programmatic lifecycle manager for Hindsight — embeds a local hindsight daemon in a Node application. Pair with @vectorize-io/hindsight-client for memory operations.",
  "main": "dist/index.js",
  "types": "dist/index.d.ts",
@@ -4,12 +4,12 @@ build-backend = "setuptools.build_meta"

 [project]
 name = "hindsight-all-slim"
-version = "0.8.0"
+version = "0.7.1"
 description = "Hindsight: Agent Memory That Works Like Human Memory - Slim All-in-One Bundle"
 readme = "README.md"
 requires-python = ">=3.11"
 dependencies = [
-    "hindsight-api-slim==0.8.0",
+    "hindsight-api-slim==0.7.1",
    "hindsight-client>=0.0.7",
    "hindsight-embed>=0.1.0",
 ]
@@ -4,12 +4,12 @@ build-backend = "hatchling.build"

 [project]
 name = "hindsight-all"
-version = "0.8.0"
+version = "0.7.1"
 description = "Hindsight: Agent Memory That Works Like Human Memory - All-in-One Bundle"
 readme = "README.md"
 requires-python = ">=3.11"
 dependencies = [
-    "hindsight-api-slim[all]==0.8.0",
+    "hindsight-api-slim[all]==0.7.1",
    "hindsight-client>=0.0.7",
    "hindsight-embed>=0.1.0",
 ]
@@ -21,7 +21,7 @@ hindsight-embed = { workspace = true }

 [project.optional-dependencies]
 local-llm = [
-    "hindsight-api-slim[local-llm]==0.8.0",
+    "hindsight-api-slim[local-llm]==0.7.1",
 ]
 test = [
    "pytest>=7.0.0",
@@ -99,7 +99,7 @@ hindsight-api
 ## Docker

 ```bash
-docker run -it --name hindsight --restart unless-stopped -p 8888:8888 \
+docker run --rm -it -p 8888:8888 \
  -e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \
  -v $HOME/.hindsight-docker:/home/hindsight/.pg0 \
  ghcr.io/vectorize-io/hindsight:latest
@@ -53,4 +53,4 @@ __all__ = [
    "RemoteTEICrossEncoder",
    "LLMConfig",
 ]
-__version__ = "0.8.0"
+__version__ = "0.7.1"
@@ -54,20 +54,23 @@ _INDEX_TYPE_KEYWORDS = {
 #   pre-dispatcher code (internal benchmarks tuned around our embedding count
 #   and recall floor; see the link_utils / pool init call sites for the
 #   latency-vs-recall framing).
-# - vchord exposes vchordrq.probes, but its shape must match the index's
-#   build.internal.lists hierarchy. VectorChord 1.1 added per-index fallback
-#   parameters for this reason: a session GUC overrides every vchordrq index,
-#   and a single value can be invalid for listless or mixed-layout indexes.
-#   Hindsight's built-in vchord clause does not set lists, so the safe default
-#   is no session-level probe override; deployments that partition vchordrq
-#   indexes should attach probes to the index storage parameters instead.
+# - vchord exposes vchordrq.probes (no default; see VectorChord issue #392)
+#   and vchordrq.epsilon (default 1.9). probes = 10 / 30 are starting
+#   defaults pending a workload-specific sweep — vchordrq's recall curve
+#   shape differs from HNSW's, so the pgvector numbers don't translate
+#   directly. Revisit with a per-cluster benchmark once we have production
+#   recall data; until then these are deliberately conservative on the
+#   high-recall path. We leave epsilon at its default; tightening it is a
+#   separate trade-off.
 # - pgvectorscale / pg_diskann / scann do not expose an equivalent per-statement
 #   knob in the engine today, so the dispatcher returns no statements for them.
 _ANN_TUNING_LOW_LATENCY: dict[str, tuple[tuple[str, str], ...]] = {
    "pgvector": (("hnsw.ef_search", "60"),),
+    "vchord": (("vchordrq.probes", "10"),),
 }
 _ANN_TUNING_HIGH_RECALL: dict[str, tuple[tuple[str, str], ...]] = {
    "pgvector": (("hnsw.ef_search", "200"),),
+    "vchord": (("vchordrq.probes", "30"),),
 }

 _EXTENSION_INSTALL_SQL = {
@@ -17,9 +17,7 @@ import asyncpg
 import typer

 from ..config import DEFAULT_DATABASE_SCHEMA, HindsightConfig
-from ..engine.memory_engine import _current_schema
 from ..engine.schema import fq_table_explicit as _fq_table
-from ..engine.transfer import export_bank
 from ..extensions import TenantExtension, load_extension
 from ..pg0 import parse_pg0_url, resolve_database_url

@@ -52,38 +50,18 @@ BACKUP_TABLES = [
    "unit_entities",
    "entity_cooccurrences",
    "memory_links",
-    "observation_history",
    "mental_models",
-    "mental_model_history",
    "directives",
    "async_operations",
    "webhooks",
    "file_storage",
    "audit_log",
-    "llm_requests",
    "graph_maintenance_queue",
 ]

 MANIFEST_VERSION = "1"


-async def _admin_connect(db_url: str) -> asyncpg.Connection:
-    """Open a raw asyncpg connection to an admin DB URL.
-
-    ``resolve_database_url`` handles both plain ``postgres://`` (passthrough) and
-    ``pg0://`` (boots the embedded server and returns its real libpq URL), so this
-    is the only step needed to connect. JSON codecs are registered so ``jsonb``
-    columns decode to Python objects (used by the export row dumps).
-    """
-    is_pg0, instance_name, _ = parse_pg0_url(db_url)
-    if is_pg0:
-        typer.echo(f"Starting embedded PostgreSQL (instance: {instance_name})...")
-    conn = await asyncpg.connect(await resolve_database_url(db_url))
-    for type_name in ("json", "jsonb"):
-        await conn.set_type_codec(type_name, encoder=json.dumps, decoder=json.loads, schema="pg_catalog")
-    return conn
-
-
 async def _backup(database_url: str, output_path: Path, schema: str = "public") -> dict[str, Any]:
    """Backup all tables to a zip file using binary COPY protocol."""
    conn = await asyncpg.connect(database_url)
@@ -352,123 +330,6 @@ def run_db_migration(
    typer.echo(f"Database migrations completed successfully for {len(schemas)} schema(s)")


-async def _run_export_bank(db_url: str, bank_id: str, output: Path, schema: str, include_history: bool) -> int:
-    """Export a whole bank to a ZIP archive."""
-    conn = await _admin_connect(db_url)
-    try:
-        # export_bank resolves table names via fq_table (the _current_schema
-        # contextvar); set it so the raw connection targets the right schema.
-        _current_schema.set(schema)
-        data = await export_bank(conn, bank_id, include_history=include_history)
-    finally:
-        await conn.close()
-
-    output.write_bytes(data)
-    return len(data)
-
-
-@app.command(name="export-bank")
-def export_bank_command(
-    bank_id: str = typer.Option(..., "--bank", "-b", help="Bank id to export."),
-    output: Path = typer.Option(..., "--output", "-o", help="Path to write the .zip archive."),
-    schema: str | None = typer.Option(
-        None,
-        "--schema",
-        "-s",
-        help="Database schema the bank lives in. Defaults to the configured base schema.",
-    ),
-    include_history: bool = typer.Option(
-        False,
-        "--include-history",
-        help="Also export operational history (audit_log, llm_requests). Off by default.",
-    ),
-):
-    """Export an entire bank to a portable ZIP (no embeddings — regenerated on import).
-
-    Carries documents, facts, observations, bank config, mental models, directives
-    and webhooks so the bank can be imported into a new instance configured with a
-    different embedding model / vector / text-search backend.
-    """
-    config = HindsightConfig.from_env()
-
-    if not config.database_url:
-        typer.echo("Error: Database URL not configured.", err=True)
-        typer.echo("Set HINDSIGHT_API_DATABASE_URL environment variable.", err=True)
-        raise typer.Exit(1)
-
-    target_schema = schema or config.database_schema or DEFAULT_DATABASE_SCHEMA
-    typer.echo(f"Exporting bank '{bank_id}' from schema '{target_schema}'...")
-
-    size = asyncio.run(_run_export_bank(config.database_url, bank_id, output, target_schema, include_history))
-
-    typer.echo(f"Exported bank '{bank_id}' to {output} ({size} bytes)")
-
-
-async def _run_import_bank(archive_path: Path, schema: str, target_bank_id: str | None, include_history: bool):
-    """Boot a MemoryEngine (for the target's embedding model) and restore a bank archive."""
-    # MemoryEngine is heavy (loads embeddings); import it lazily so other admin
-    # commands don't pay for it. _current_schema is imported at module top.
-    from ..engine.memory_engine import MemoryEngine
-    from ..models import RequestContext
-
-    archive_bytes = archive_path.read_bytes()
-    # run_migrations=True so a fresh target instance is provisioned at this
-    # instance's embedding dimension / vector / text-search backend before restore.
-    engine = MemoryEngine(run_migrations=True)
-    await engine.initialize()
-    try:
-        _current_schema.set(schema)
-        context = RequestContext(internal=True, user_initiated=True)
-        return await engine.import_bank_async(
-            archive_bytes,
-            context,
-            target_bank_id=target_bank_id,
-            include_history=include_history,
-        )
-    finally:
-        await engine.close()
-
-
-@app.command(name="import-bank")
-def import_bank_command(
-    archive: Path = typer.Option(..., "--archive", "-a", help="Path to the .zip produced by export-bank."),
-    schema: str | None = typer.Option(
-        None, "--schema", "-s", help="Target schema. Defaults to the configured base schema."
-    ),
-    target_bank: str | None = typer.Option(
-        None, "--target-bank", help="Override the bank id (defaults to the archive's source bank)."
-    ),
-    include_history: bool = typer.Option(
-        False, "--include-history", help="Also restore operational history if present in the archive."
-    ),
-):
-    """Restore a whole bank from an export-bank archive into THIS instance.
-
-    Re-embeds facts with this instance's configured embedding model and rebuilds
-    links and indexes — the import half of a cross-instance migration. Run against
-    an instance configured with the desired embedding / vector / text-search backend.
-    The target bank must not already exist (import restores a whole bank, not a merge).
-    """
-    config = HindsightConfig.from_env()
-    if not config.database_url:
-        typer.echo("Error: Database URL not configured.", err=True)
-        typer.echo("Set HINDSIGHT_API_DATABASE_URL environment variable.", err=True)
-        raise typer.Exit(1)
-
-    target_schema = schema or config.database_schema or DEFAULT_DATABASE_SCHEMA
-    typer.echo(f"Importing bank archive '{archive}' into schema '{target_schema}'...")
-
-    result = asyncio.run(_run_import_bank(archive, target_schema, target_bank, include_history))
-
-    typer.echo(
-        f"Imported bank '{result.bank_id}': {result.documents_imported} doc(s), "
-        f"{result.facts_imported} fact(s), {result.observations_imported} observation(s), "
-        f"{result.mental_models_imported} mental model(s), "
-        f"{result.mental_model_history_imported} mm-history row(s), {result.directives_imported} directive(s), "
-        f"{result.webhooks_imported} webhook(s), {result.history_rows_imported} history row(s)"
-    )
-
-
 async def _decommission_worker(db_url: str, worker_id: str, schema: str = "public") -> int:
    """Release all tasks owned by a worker, setting them back to pending status."""
    is_pg0, instance_name, _ = parse_pg0_url(db_url)
@@ -1,253 +0,0 @@
-"""Move mental-model and observation history into dedicated tables.
-
-Both histories were accumulated in a single JSONB/CLOB ``history`` column
-(``mental_models.history`` and ``memory_units.history``), appended to on every
-update. That design has two problems:
-
-1. **Unbounded growth on observations.** The observation write path appended a
-   snapshot on every update with no cap at all, so a frequently-reinforced
-   observation grew its ``history`` array until it crossed Postgres's hard 256MB
-   jsonb limit (SQLSTATE 54000), after which every further UPDATE failed and the
-   row was stuck.
-2. **Wrong-axis cap on mental models.** The mental-model cap bounded the *number*
-   of entries (50), not their *size* — a single large reflect snapshot could
-   still blow the budget — and rewrote the whole array (plus TOAST) on every
-   refresh, defeating HOT updates.
-
-This migration creates one row per history entry in two dedicated tables, with
-an index that makes "most recent N for this item" cheap, then drops the old
-columns. The cap is now enforced at write time as a bounded DELETE of the
-oldest over-cap rows (see config ``*_HISTORY_MAX_ENTRIES``).
-
-Revision ID: a7b8c9d0e1f2
-Revises: d3e4f5a6b7c8
-Create Date: 2026-06-05
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "a7b8c9d0e1f2"
-down_revision: str | Sequence[str] | None = "d3e4f5a6b7c8"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Schema-qualifier for raw SQL on PG (multi-tenant search_path)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-# ---------------------------------------------------------------------------
-# PostgreSQL
-# ---------------------------------------------------------------------------
-def _pg_upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # Both tables share the same shape: surrogate id, FK to the parent, bank_id,
-    # the snapshot payload as a single JSONB ``content`` blob, and changed_at.
-    # The payload is per-row (one change per row) so it stays small — this is NOT
-    # the old single-column-grows-forever design; growth is bounded by row count
-    # plus the write-time cap. Folding the previous_* fields into one JSONB keeps
-    # the schema dialect-simple (no array columns) and flexible.
-
-    # --- mental_model_history -------------------------------------------------
-    # content: {"previous_content": ..., "previous_reflect_response": {...}}
-    op.execute(
-        f"""
-        CREATE TABLE IF NOT EXISTS {schema}mental_model_history (
-            id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
-            mental_model_id VARCHAR(64) NOT NULL,
-            bank_id VARCHAR(64) NOT NULL,
-            content JSONB NOT NULL,
-            changed_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
-            FOREIGN KEY (mental_model_id, bank_id)
-                REFERENCES {schema}mental_models(id, bank_id) ON DELETE CASCADE
-        )
-        """
-    )
-    op.execute(
-        "CREATE INDEX IF NOT EXISTS idx_mm_history_model "
-        f"ON {schema}mental_model_history (bank_id, mental_model_id, changed_at DESC, id DESC)"
-    )
-
-    # --- observation_history --------------------------------------------------
-    # content: {"previous_text", "previous_tags", "previous_occurred_start",
-    #           "previous_occurred_end", "previous_mentioned_at", "new_source_memory_ids"}
-    op.execute(
-        f"""
-        CREATE TABLE IF NOT EXISTS {schema}observation_history (
-            id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
-            observation_id UUID NOT NULL,
-            bank_id VARCHAR(64) NOT NULL,
-            content JSONB NOT NULL,
-            changed_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
-            FOREIGN KEY (observation_id)
-                REFERENCES {schema}memory_units(id) ON DELETE CASCADE
-        )
-        """
-    )
-    op.execute(
-        "CREATE INDEX IF NOT EXISTS idx_observation_history_obs "
-        f"ON {schema}observation_history (observation_id, changed_at DESC, id DESC)"
-    )
-
-    # --- backfill mental models ----------------------------------------------
-    # Explode each row's history array into rows, preserving chronological order
-    # via WITH ORDINALITY so the IDENTITY id tie-breaks oldest->newest correctly.
-    # changed_at is promoted to its own column; the rest of the element becomes
-    # ``content`` (the ``- 'changed_at'`` strips the now-redundant key).
-    op.execute(
-        f"""
-        INSERT INTO {schema}mental_model_history (mental_model_id, bank_id, content, changed_at)
-        SELECT mm.id, mm.bank_id,
-               e - 'changed_at',
-               COALESCE(NULLIF(e->>'changed_at', '')::timestamptz, now())
-        FROM {schema}mental_models mm
-             CROSS JOIN LATERAL jsonb_array_elements(mm.history) WITH ORDINALITY a(e, ord)
-        WHERE mm.history IS NOT NULL
-          AND jsonb_typeof(mm.history) = 'array'
-          AND jsonb_array_length(mm.history) > 0
-        ORDER BY mm.id, mm.bank_id, ord
-        """
-    )
-
-    # --- backfill observations -----------------------------------------------
-    op.execute(
-        f"""
-        INSERT INTO {schema}observation_history (observation_id, bank_id, content, changed_at)
-        SELECT mu.id, mu.bank_id,
-               e - 'changed_at',
-               COALESCE(NULLIF(e->>'changed_at', '')::timestamptz, now())
-        FROM {schema}memory_units mu
-             CROSS JOIN LATERAL jsonb_array_elements(mu.history) WITH ORDINALITY a(e, ord)
-        WHERE mu.fact_type = 'observation'
-          AND mu.history IS NOT NULL
-          AND jsonb_typeof(mu.history) = 'array'
-          AND jsonb_array_length(mu.history) > 0
-        ORDER BY mu.id, ord
-        """
-    )
-
-    # --- drop the legacy columns ---------------------------------------------
-    op.execute(f"ALTER TABLE {schema}mental_models DROP COLUMN IF EXISTS history")
-    op.execute(f"ALTER TABLE {schema}memory_units DROP COLUMN IF EXISTS history")
-
-
-def _pg_downgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # Re-add the columns (empty — historical content is not reconstructed back
-    # into the array form; the dedicated tables are dropped below).
-    op.execute(f"ALTER TABLE {schema}mental_models ADD COLUMN IF NOT EXISTS history JSONB DEFAULT '[]'::jsonb")
-    op.execute(f"ALTER TABLE {schema}memory_units ADD COLUMN IF NOT EXISTS history JSONB DEFAULT '[]'::jsonb")
-
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_observation_history_obs")
-    op.execute(f"DROP TABLE IF EXISTS {schema}observation_history")
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_mm_history_model")
-    op.execute(f"DROP TABLE IF EXISTS {schema}mental_model_history")
-
-
-# ---------------------------------------------------------------------------
-# Oracle 23ai
-# ---------------------------------------------------------------------------
-def _oracle_upgrade() -> None:
-    # Same single-JSONB shape as PG: ``content`` holds the snapshot payload as a
-    # CLOB IS JSON. The legacy per-element JSON object (minus changed_at, promoted
-    # to its own column) is carried through verbatim on backfill — the array
-    # columns the previous design needed are gone.
-    op.execute(
-        """
-        CREATE TABLE IF NOT EXISTS mental_model_history (
-            id NUMBER GENERATED BY DEFAULT AS IDENTITY,
-            mental_model_id VARCHAR2(256) NOT NULL,
-            bank_id VARCHAR2(256) NOT NULL,
-            content CLOB NOT NULL
-                CONSTRAINT mmh_content_json CHECK (content IS JSON),
-            changed_at TIMESTAMP WITH TIME ZONE DEFAULT SYSTIMESTAMP NOT NULL,
-            CONSTRAINT pk_mental_model_history PRIMARY KEY (id),
-            CONSTRAINT fk_mmh_model FOREIGN KEY (mental_model_id, bank_id)
-                REFERENCES mental_models(id, bank_id) ON DELETE CASCADE
-        )
-        """
-    )
-    op.execute(
-        "CREATE INDEX idx_mm_history_model ON mental_model_history (bank_id, mental_model_id, changed_at DESC, id DESC)"
-    )
-
-    op.execute(
-        """
-        CREATE TABLE IF NOT EXISTS observation_history (
-            id NUMBER GENERATED BY DEFAULT AS IDENTITY,
-            observation_id RAW(16) NOT NULL,
-            bank_id VARCHAR2(256) NOT NULL,
-            content CLOB NOT NULL
-                CONSTRAINT oh_content_json CHECK (content IS JSON),
-            changed_at TIMESTAMP WITH TIME ZONE DEFAULT SYSTIMESTAMP NOT NULL,
-            CONSTRAINT pk_observation_history PRIMARY KEY (id),
-            CONSTRAINT fk_oh_obs FOREIGN KEY (observation_id)
-                REFERENCES memory_units(id) ON DELETE CASCADE
-        )
-        """
-    )
-    op.execute(
-        "CREATE INDEX idx_observation_history_obs ON observation_history (observation_id, changed_at DESC, id DESC)"
-    )
-
-    bind = op.get_bind()
-
-    # Backfill via JSON_TABLE. ``content`` is the whole element (FORMAT JSON PATH
-    # '$'); changed_at is also promoted to its own column. Backfilled content may
-    # therefore still carry a redundant changed_at key, which the read path
-    # ignores in favour of the column — harmless, and avoids JSON surgery here.
-    bind.exec_driver_sql(
-        """
-        INSERT INTO mental_model_history (mental_model_id, bank_id, content, changed_at)
-        SELECT mm.id, mm.bank_id, jt.content, NVL(jt.changed_at, SYSTIMESTAMP)
-        FROM mental_models mm,
-             JSON_TABLE(mm.history, '$[*]' COLUMNS (
-                 seq FOR ORDINALITY,
-                 content CLOB FORMAT JSON PATH '$',
-                 changed_at TIMESTAMP WITH TIME ZONE PATH '$.changed_at'
-             )) jt
-        WHERE mm.history IS NOT NULL
-        ORDER BY mm.id, mm.bank_id, jt.seq
-        """
-    )
-
-    bind.exec_driver_sql(
-        """
-        INSERT INTO observation_history (observation_id, bank_id, content, changed_at)
-        SELECT mu.id, mu.bank_id, jt.content, NVL(jt.changed_at, SYSTIMESTAMP)
-        FROM memory_units mu,
-             JSON_TABLE(mu.history, '$[*]' COLUMNS (
-                 seq FOR ORDINALITY,
-                 content CLOB FORMAT JSON PATH '$',
-                 changed_at TIMESTAMP WITH TIME ZONE PATH '$.changed_at'
-             )) jt
-        WHERE mu.fact_type = 'observation' AND mu.history IS NOT NULL
-        ORDER BY mu.id, jt.seq
-        """
-    )
-
-    op.execute("ALTER TABLE mental_models DROP COLUMN history")
-    op.execute("ALTER TABLE memory_units DROP COLUMN history")
-
-
-def _oracle_downgrade() -> None:
-    op.execute("ALTER TABLE mental_models ADD history CLOB DEFAULT '[]' NOT NULL")
-    op.execute("ALTER TABLE memory_units ADD history CLOB DEFAULT '[]'")
-    op.execute("DROP TABLE observation_history CASCADE CONSTRAINTS")
-    op.execute("DROP TABLE mental_model_history CASCADE CONSTRAINTS")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade, oracle=_oracle_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade, oracle=_oracle_downgrade)
@@ -1,156 +0,0 @@
-"""Repair: install maintenance routines on the ``public`` / base-schema run.
-
-The original maintenance-routines migration (``e5f6a7b8c9d0``) only created the
-shared ``public.banks_needing_consolidation()`` and
-``public.schemas_with_expired_rows(...)`` routines when the run had *no*
-``target_schema`` at all. But the single-tenant runtime always migrates an
-explicit schema — which defaults to ``public`` — so on every default
-PostgreSQL deployment the migration was stamped as applied while the functions
-were never created. Background maintenance then logs::
-
-    Retention sweep failed for llm_requests: function public.schemas_with_expired_rows(...) does not exist
-    Consolidation reconcile discovery failed: function public.banks_needing_consolidation() does not exist
-
-See https://github.com/vectorize-io/hindsight/issues/2056.
-
-Because ``e5f6a7b8c9d0`` is already stamped on affected ``0.8.0`` databases,
-editing it would not re-run it there. This forward migration re-installs the
-functions idempotently (``CREATE OR REPLACE``) on the run that targets the
-shared ``public`` schema (base run with no ``target_schema``, or an explicit
-``target_schema=public``), self-healing already-upgraded deployments and
-covering fresh upgrades from earlier versions.
-
-Per-tenant runs against a non-``public`` schema still skip it: re-issuing
-``CREATE OR REPLACE FUNCTION public....`` from each concurrent tenant migration
-aborts with ``tuple concurrently updated`` on the ``pg_proc`` catalog row, and
-the base/public run has already created the functions for every tenant to use.
-Runs that target ``public`` are serialized by the per-schema migration advisory
-lock, so only one wins the create.
-
-PostgreSQL only — the worker poller and these tables are not wired for Oracle,
-so the Oracle slot is intentionally absent (mirrors ``e5f6a7b8c9d0``).
-
-Revision ID: b2d4f6a8c1e3
-Revises: e5f6a7b8c9d0
-Create Date: 2026-06-08
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "b2d4f6a8c1e3"
-down_revision: str | Sequence[str] | None = "e5f6a7b8c9d0"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _should_install_public_routines(target_schema: str | None) -> bool:
-    """True for the run that must (re)create the shared ``public.*`` routines.
-
-    The routines physically live in ``public`` (hard-coded ``public.`` qualifier
-    in the SQL below), so they must be installed exactly once — on the base run
-    (no ``target_schema``) or on the run that explicitly targets ``public``. A
-    run against any other tenant schema skips it to avoid concurrent
-    ``CREATE OR REPLACE`` on the same ``pg_proc`` row.
-    """
-    return not target_schema or target_schema == "public"
-
-
-def _pg_upgrade() -> None:
-    if not _should_install_public_routines(context.config.get_main_option("target_schema")):
-        return
-    # Banks with eligible-but-unscheduled facts and no in-flight consolidation.
-    # Auto-consolidation is filtered here only at the bank level (cheap prune);
-    # the full hierarchical resolution (global -> tenant -> bank, plus
-    # enable_observations) is done by the caller for the small returned set.
-    op.execute(
-        """
-        CREATE OR REPLACE FUNCTION public.banks_needing_consolidation()
-        RETURNS TABLE(schema_name text, bank_id text)
-        LANGUAGE plpgsql STABLE
-        AS $fn$
-        DECLARE
-            sch text;
-        BEGIN
-            FOR sch IN
-                SELECT n.nspname
-                FROM pg_class c
-                JOIN pg_namespace n ON n.oid = c.relnamespace
-                WHERE c.relname = 'memory_units' AND c.relkind = 'r'
-            LOOP
-                RETURN QUERY EXECUTE format($q$
-                    SELECT %1$L::text, m.bank_id
-                    FROM %1$I.memory_units m
-                    JOIN %1$I.banks b ON b.bank_id = m.bank_id
-                    WHERE m.consolidated_at IS NULL
-                      AND m.consolidation_failed_at IS NULL
-                      AND m.fact_type IN ('experience', 'world')
-                      AND COALESCE(b.config -> 'enable_auto_consolidation', 'true'::jsonb) <> 'false'::jsonb
-                      AND NOT EXISTS (
-                          SELECT 1 FROM %1$I.async_operations o
-                          WHERE o.bank_id = m.bank_id
-                            AND o.operation_type = 'consolidation'
-                            AND o.status IN ('pending', 'processing')
-                      )
-                    GROUP BY m.bank_id
-                $q$, sch);
-            END LOOP;
-        END;
-        $fn$;
-        """
-    )
-
-    # Schemas holding at least one row of p_table older than p_days. p_ts_col is
-    # the timestamp column to compare. Returns nothing when p_days <= 0
-    # (retention disabled).
-    op.execute(
-        """
-        CREATE OR REPLACE FUNCTION public.schemas_with_expired_rows(
-            p_table text, p_ts_col text, p_days int
-        )
-        RETURNS SETOF text
-        LANGUAGE plpgsql STABLE
-        AS $fn$
-        DECLARE
-            sch text;
-            has_expired boolean;
-        BEGIN
-            IF p_days IS NULL OR p_days <= 0 THEN
-                RETURN;
-            END IF;
-            FOR sch IN
-                SELECT n.nspname
-                FROM pg_class c
-                JOIN pg_namespace n ON n.oid = c.relnamespace
-                WHERE c.relname = p_table AND c.relkind = 'r'
-            LOOP
-                EXECUTE format(
-                    'SELECT EXISTS (SELECT 1 FROM %I.%I WHERE %I < NOW() - make_interval(days => $1))',
-                    sch, p_table, p_ts_col
-                ) INTO has_expired USING p_days;
-                IF has_expired THEN
-                    RETURN NEXT sch;
-                END IF;
-            END LOOP;
-        END;
-        $fn$;
-        """
-    )
-
-
-def _pg_downgrade() -> None:
-    # No-op: ``e5f6a7b8c9d0`` owns the lifecycle of these functions and drops
-    # them on its own downgrade. This migration only ever (re)creates them, so
-    # there is nothing to undo without racing that migration's DROP.
-    pass
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)
@@ -1,96 +0,0 @@
-"""Add llm_requests table for per-bank LLM request tracing.
-
-Stores one row per logical LLM call Hindsight makes (success and failure),
-capturing the input messages, model output, token usage (input/output/cached/
-total), finish reason, and caller metadata. Disabled by default at the
-application layer (HINDSIGHT_API_LLM_TRACE_ENABLED); this migration only
-creates the table.
-
-PostgreSQL only — the tracing subsystem is not wired for Oracle, so the Oracle
-slot is intentionally absent (mirrors the audit_log table).
-
-Revision ID: d3e4f5a6b7c8
-Revises: c1d2e3f4a5b6
-Create Date: 2026-06-01
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "d3e4f5a6b7c8"
-down_revision: str | Sequence[str] | None = "c1d2e3f4a5b6"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Get schema prefix for table names (required for multi-tenant support)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _pg_upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    op.execute(
-        f"""
-        CREATE TABLE IF NOT EXISTS {schema}llm_requests (
-            id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
-            bank_id TEXT,
-            operation TEXT,
-            scope TEXT,
-            -- OTel-style grouping: trace_id is shared by every LLM call of one
-            -- operation invocation (e.g. all calls of a single reflect run);
-            -- parent_span_id is that operation span; span_id is this call.
-            trace_id TEXT,
-            span_id TEXT,
-            parent_span_id TEXT,
-            provider TEXT,
-            model TEXT,
-            status TEXT NOT NULL,
-            started_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
-            ended_at TIMESTAMPTZ,
-            duration_ms INTEGER,
-            input_tokens INTEGER,
-            output_tokens INTEGER,
-            cached_tokens INTEGER,
-            total_tokens INTEGER,
-            input JSONB,
-            output JSONB,
-            error TEXT,
-            llm_info JSONB DEFAULT '{{}}'::jsonb,
-            metadata JSONB DEFAULT '{{}}'::jsonb
-        )
-        """
-    )
-
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS idx_llm_requests_bank_started ON {schema}llm_requests (bank_id, started_at DESC)"
-    )
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS idx_llm_requests_status_started ON {schema}llm_requests (status, started_at DESC)"
-    )
-    op.execute(f"CREATE INDEX IF NOT EXISTS idx_llm_requests_started ON {schema}llm_requests (started_at DESC)")
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS idx_llm_requests_trace ON {schema}llm_requests (bank_id, trace_id, started_at)"
-    )
-
-
-def _pg_downgrade() -> None:
-    schema = _get_schema_prefix()
-
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_llm_requests_started")
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_llm_requests_status_started")
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_llm_requests_bank_started")
-    op.execute(f"DROP TABLE IF EXISTS {schema}llm_requests")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)
@@ -1,153 +0,0 @@
-"""Add server-side routines for background maintenance sweeps.
-
-Installs two PL/pgSQL discovery routines in the ``public`` schema. Both loop
-over every schema that actually holds the relevant table (via ``pg_class``), so
-a single function call covers all tenants in one round-trip instead of the
-per-tenant query storm that a client-side loop would create at thousands of
-tenants.
-
- ``public.banks_needing_consolidation()`` -> (schema_name, bank_id) for banks
-  that have eligible-but-unscheduled facts (``consolidated_at IS NULL AND
-  consolidation_failed_at IS NULL`` for consolidatable fact types), have
-  auto-consolidation not explicitly disabled at the bank level, and have no
-  consolidation operation already pending/processing. Drives the periodic
-  reconcile that re-schedules consolidation after a terminal failure left facts
-  stranded (see HINDSIGHT_API_CONSOLIDATION_RECONCILE_INTERVAL_SECONDS).
-
- ``public.schemas_with_expired_rows(p_table, p_ts_col, p_days)`` -> schema
-  names that hold at least one ``p_table`` row older than ``p_days``. Drives the
-  cross-tenant retention sweeps for ``audit_log`` and ``llm_requests``; the loop
-  then issues a DELETE only against the returned schemas.
-
-These are read-only (STABLE) discovery routines — the caller performs the
-enqueue/DELETE — so installing them never mutates data.
-
-PostgreSQL only — the worker poller and these tables are not wired for Oracle,
-so the Oracle slot is intentionally absent (mirrors the audit_log / llm_requests
-table migrations). The routines live in ``public`` and are CREATE OR REPLACE, so
-running this migration once per tenant schema is idempotent.
-
-Revision ID: e5f6a7b8c9d0
-Revises: a7b8c9d0e1f2
-Create Date: 2026-06-05
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-from hindsight_api.alembic._dialect import run_for_dialect
-
-revision: str = "e5f6a7b8c9d0"
-down_revision: str | Sequence[str] | None = "a7b8c9d0e1f2"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _is_base_schema_run() -> bool:
-    """True only for the base-schema migration (no per-tenant target_schema).
-
-    These routines live in the shared ``public`` schema, so they must be created
-    exactly once. Running ``CREATE OR REPLACE FUNCTION public....`` again from each
-    concurrent per-tenant migration aborts with ``tuple concurrently updated`` on
-    the ``pg_proc`` catalog row, so tenant runs skip it (the base run already
-    created the function for every tenant to use).
-    """
-    return not context.config.get_main_option("target_schema")
-
-
-def _pg_upgrade() -> None:
-    if not _is_base_schema_run():
-        return
-    # Banks with eligible-but-unscheduled facts and no in-flight consolidation.
-    # Auto-consolidation is filtered here only at the bank level (cheap prune);
-    # the full hierarchical resolution (global -> tenant -> bank, plus
-    # enable_observations) is done by the caller for the small returned set.
-    op.execute(
-        """
-        CREATE OR REPLACE FUNCTION public.banks_needing_consolidation()
-        RETURNS TABLE(schema_name text, bank_id text)
-        LANGUAGE plpgsql STABLE
-        AS $fn$
-        DECLARE
-            sch text;
-        BEGIN
-            FOR sch IN
-                SELECT n.nspname
-                FROM pg_class c
-                JOIN pg_namespace n ON n.oid = c.relnamespace
-                WHERE c.relname = 'memory_units' AND c.relkind = 'r'
-            LOOP
-                RETURN QUERY EXECUTE format($q$
-                    SELECT %1$L::text, m.bank_id
-                    FROM %1$I.memory_units m
-                    JOIN %1$I.banks b ON b.bank_id = m.bank_id
-                    WHERE m.consolidated_at IS NULL
-                      AND m.consolidation_failed_at IS NULL
-                      AND m.fact_type IN ('experience', 'world')
-                      AND COALESCE(b.config -> 'enable_auto_consolidation', 'true'::jsonb) <> 'false'::jsonb
-                      AND NOT EXISTS (
-                          SELECT 1 FROM %1$I.async_operations o
-                          WHERE o.bank_id = m.bank_id
-                            AND o.operation_type = 'consolidation'
-                            AND o.status IN ('pending', 'processing')
-                      )
-                    GROUP BY m.bank_id
-                $q$, sch);
-            END LOOP;
-        END;
-        $fn$;
-        """
-    )
-
-    # Schemas holding at least one row of p_table older than p_days. p_ts_col is
-    # the timestamp column to compare. Returns nothing when p_days <= 0
-    # (retention disabled).
-    op.execute(
-        """
-        CREATE OR REPLACE FUNCTION public.schemas_with_expired_rows(
-            p_table text, p_ts_col text, p_days int
-        )
-        RETURNS SETOF text
-        LANGUAGE plpgsql STABLE
-        AS $fn$
-        DECLARE
-            sch text;
-            has_expired boolean;
-        BEGIN
-            IF p_days IS NULL OR p_days <= 0 THEN
-                RETURN;
-            END IF;
-            FOR sch IN
-                SELECT n.nspname
-                FROM pg_class c
-                JOIN pg_namespace n ON n.oid = c.relnamespace
-                WHERE c.relname = p_table AND c.relkind = 'r'
-            LOOP
-                EXECUTE format(
-                    'SELECT EXISTS (SELECT 1 FROM %I.%I WHERE %I < NOW() - make_interval(days => $1))',
-                    sch, p_table, p_ts_col
-                ) INTO has_expired USING p_days;
-                IF has_expired THEN
-                    RETURN NEXT sch;
-                END IF;
-            END LOOP;
-        END;
-        $fn$;
-        """
-    )
-
-
-def _pg_downgrade() -> None:
-    if not _is_base_schema_run():
-        return
-    op.execute("DROP FUNCTION IF EXISTS public.banks_needing_consolidation()")
-    op.execute("DROP FUNCTION IF EXISTS public.schemas_with_expired_rows(text, text, int)")
-
-
-def upgrade() -> None:
-    run_for_dialect(pg=_pg_upgrade)
-
-
-def downgrade() -> None:
-    run_for_dialect(pg=_pg_downgrade)
@@ -6,8 +6,6 @@ the FastAPI application with all API endpoints.
 """

 import asyncio
-import dataclasses
-import hmac
 import json
 import logging
 import re
@@ -19,13 +17,7 @@ from typing import Any, Literal
 from fastapi import Depends, FastAPI, File, Form, Header, HTTPException, Query, Request, UploadFile
 from fastapi.middleware.gzip import GZipMiddleware

-from hindsight_api.engine.audit import (
-    AuditEntry,
-    AuditLogger,
-    AuditLogListResponse,
-    AuditLogStatsResponse,
-)
-from hindsight_api.engine.llm_trace import LLMRequestListResponse, LLMRequestStatsResponse
+from hindsight_api.engine.audit import AuditEntry, AuditLogger
 from hindsight_api.extensions import AuthenticationError


@@ -81,7 +73,7 @@ def FieldWithDefault(default_factory: Callable, **kwargs) -> Any:
    return Field(default_factory=default_factory, json_schema_extra=json_extra, **kwargs)


-from hindsight_api.config import _get_raw_config, get_config
+from hindsight_api.config import get_config
 from hindsight_api.engine.memory_engine import Budget, _current_schema, _get_tiktoken_encoding, fq_table
 from hindsight_api.engine.providers.none_llm import LLMNotAvailableError
 from hindsight_api.engine.response_models import VALID_RECALL_FACT_TYPES, MemoryFact, TokenUsage
@@ -1208,33 +1200,6 @@ class BankConfigResponse(BaseModel):
    overrides: dict[str, Any] = Field(description="Bank-specific configuration overrides only (Python field names)")


-class AdminConfigResponse(BaseModel):
-    """Response model for the server-level (admin) configuration view.
-
-    Returns the resolved ``HindsightConfig`` as a flat dict keyed by Python field
-    name. Credential fields (API keys, tokens, service-account keys, base URLs) are
-    masked: present as ``"***"`` when set and ``None`` when unset, so an operator can
-    see which credentials are configured without ever seeing their values.
-    """
-
-    config: dict[str, Any] = Field(
-        description="Resolved server-level configuration (Python field names); credentials are redacted"
-    )
-
-
-# Name suffixes that mark a config field as secret-bearing. Used in addition to
-# HindsightConfig._CREDENTIAL_FIELDS so the admin config view never leaks a provider
-# credential even if the (denylist) credential set misses a field. Suffixes are
-# singular on purpose — "_token" must not also match value-bearing "_tokens" fields
-# like recall_max_tokens.
-_SENSITIVE_FIELD_SUFFIXES = ("_api_key", "_token", "_secret", "_access_key", "_account_key", "_password")
-
-
-def _is_sensitive_config_field(field_name: str, credential_fields: set[str]) -> bool:
-    """Whether a config field must be redacted in the admin view."""
-    return field_name in credential_fields or field_name.endswith(_SENSITIVE_FIELD_SUFFIXES)
-
-
 class GraphDataResponse(BaseModel):
    """Response model for graph data endpoint."""

@@ -1479,18 +1444,6 @@ class ReprocessDocumentResponse(BaseModel):
    items_count: int


-class DocumentImportSubmitResponse(BaseModel):
-    """Response for the async document-import endpoint (202).
-
-    The import runs in the background; poll the operations endpoint for status.
-    The imported/skipped counts (documents_imported, facts_imported,
-    observations_imported, etc.) are written to the operation's result_metadata.
-    """
-
-    operation_id: str
-    status: str = "pending"
-
-
 class DeleteResponse(BaseModel):
    """Response model for delete operations."""

@@ -2197,28 +2150,6 @@ async def apply_bank_template_manifest(
    )


-class OperationProgress(BaseModel):
-    """Last-known progress snapshot for a long-running async operation.
-
-    Written at coarse phase/batch boundaries by the worker (consolidation, batch
-    retain). Lets an operator polling the operation status API distinguish a healthy
-    long-running job (``processed`` advancing across polls) from a frozen one (same
-    numbers, no movement in ``at``). Absent (``null``) on operations that never
-    reached a checkpoint — completed-instantly or pre-feature rows.
-    """
-
-    stage: str = Field(description="Coarse phase the operation last reported (e.g. 'processing_batch').")
-    at: str = Field(description="ISO-8601 timestamp when this snapshot was written.")
-    processed: int | None = Field(
-        default=None, description="Units of work finished so far (sub-batches, memories), when known."
-    )
-    total: int | None = Field(default=None, description="Total units of work for the operation, when known.")
-    detail: dict[str, int] | None = Field(
-        default=None,
-        description="Operation-specific counters (e.g. observations_created, round, items_in_sub_batch).",
-    )
-
-
 class OperationResponse(BaseModel):
    """Response model for a single async operation."""

@@ -2243,10 +2174,6 @@ class OperationResponse(BaseModel):
    items_count: int
    document_id: str | None = None
    created_at: str
-    updated_at: str | None = Field(
-        default=None,
-        description="When this operation's row last changed (claim, progress heartbeat, or completion).",
-    )
    status: str
    error_message: str | None
    retry_count: int | None = Field(
@@ -2263,10 +2190,6 @@ class OperationResponse(BaseModel):
            "some backpressure window opens. Always null for completed tasks."
        ),
    )
-    progress: OperationProgress | None = Field(
-        default=None,
-        description="Last-known progress snapshot for a running operation; null if none was recorded.",
-    )


 class ConsolidationRequest(BaseModel):
@@ -2402,10 +2325,6 @@ class OperationStatusResponse(BaseModel):
            "immediate pickup."
        ),
    )
-    progress: OperationProgress | None = Field(
-        default=None,
-        description="Last-known progress snapshot for a running operation; null if none was recorded.",
-    )
    result_metadata: dict[str, Any] | None = Field(
        default=None,
        description="Internal metadata for debugging. Structure may change without notice. Not for production use.",
@@ -2442,12 +2361,7 @@ class FeaturesInfo(BaseModel):
    mcp: bool = Field(description="Whether MCP (Model Context Protocol) server is enabled")
    worker: bool = Field(description="Whether the background worker is enabled")
    bank_config_api: bool = Field(description="Whether per-bank configuration API is enabled")
-    admin_api: bool = Field(description="Whether the admin API (/admin) is enabled")
    file_upload_api: bool = Field(description="Whether file upload/conversion API is enabled")
-    document_export_api: bool = Field(description="Whether the document export endpoint is enabled")
-    document_import_api: bool = Field(description="Whether the document import endpoint is enabled")
-    audit_log: bool = Field(description="Whether audit logging is enabled")
-    llm_trace: bool = Field(description="Whether per-bank LLM request tracing is enabled")


 class VersionResponse(BaseModel):
@@ -2463,8 +2377,6 @@ class VersionResponse(BaseModel):
                    "worker": True,
                    "bank_config_api": False,
                    "file_upload_api": True,
-                    "document_export_api": True,
-                    "document_import_api": True,
                },
            }
        }
@@ -3001,31 +2913,6 @@ def _register_routes(app: FastAPI):
                api_key = authorization.strip()
        return RequestContext(api_key=api_key)

-    def require_admin(authorization: str | None = Header(default=None)) -> None:
-        """Guard for the admin surface.
-
-        - 404 when the admin API is disabled (so the surface is invisible by default).
-        - When ``HINDSIGHT_API_ADMIN_TOKEN`` is set, require it as a bearer token
-          (or a bare token) and reject with 401 otherwise. When unset, the admin API
-          is open (auth is optional) — consistent with the rest of the deployment.
-        """
-        config = _get_raw_config()
-        if not config.enable_admin_api:
-            raise HTTPException(
-                status_code=404,
-                detail="Admin API is disabled. Set HINDSIGHT_API_ENABLE_ADMIN_API=true to enable.",
-            )
-        expected = config.admin_api_token
-        if expected:
-            token = None
-            if authorization:
-                if authorization.lower().startswith("bearer "):
-                    token = authorization[7:].strip()
-                else:
-                    token = authorization.strip()
-            if not token or not hmac.compare_digest(token, expected):
-                raise HTTPException(status_code=401, detail="Invalid or missing admin token")
-
    def precheck_for(operation: str):
        """
        Build a FastAPI dependency that runs ``OperationValidator.precheck``.
@@ -3135,42 +3022,10 @@ def _register_routes(app: FastAPI):
                mcp=config.mcp_enabled,
                worker=config.worker_enabled,
                bank_config_api=config.enable_bank_config_api,
-                admin_api=config.enable_admin_api,
                file_upload_api=config.enable_file_upload_api,
-                document_export_api=config.enable_document_export_api,
-                document_import_api=config.enable_document_import_api,
-                audit_log=config.audit_log_enabled,
-                llm_trace=config.llm_trace_enabled,
            ),
        )

-    @app.get(
-        "/admin/config",
-        response_model=AdminConfigResponse,
-        summary="Get resolved server-level configuration",
-        description="Returns the resolved server-level configuration with credentials redacted. "
-        "Gated by HINDSIGHT_API_ENABLE_ADMIN_API and, when set, HINDSIGHT_API_ADMIN_TOKEN.",
-        tags=["Admin"],
-        operation_id="get_admin_config",
-        dependencies=[Depends(require_admin)],
-    )
-    async def admin_config_endpoint() -> AdminConfigResponse:
-        """Expose the resolved ``HindsightConfig`` for operator inspection.
-
-        Sensitive fields (the credential denylist plus any field whose name ends in a
-        secret-bearing suffix — see ``_is_sensitive_config_field``) are masked so values
-        never leave the server: ``"***"`` when set, ``None`` when unset. All other fields
-        are returned as-is.
-        """
-        config = _get_raw_config()
-        credential_fields = type(config).get_credential_fields()
-        raw = dataclasses.asdict(config)
-        redacted = {
-            key: ("***" if value is not None else None) if _is_sensitive_config_field(key, credential_fields) else value
-            for key, value in raw.items()
-        }
-        return AdminConfigResponse(config=redacted)
-
    @app.get(
        "/metrics",
        summary="Prometheus metrics endpoint",
@@ -5410,124 +5265,6 @@ def _register_routes(app: FastAPI):
            logger.error(f"Error in GET /v1/default/banks/{bank_id}/export: {error_detail}")
            raise HTTPException(status_code=500, detail=str(e))

-    # =====================================================================
-    # Document Transfer (Export / Import between banks — no LLM re-extraction)
-    # =====================================================================
-
-    @app.get(
-        # Dedicated path (not under /documents/) to avoid colliding with the
-        # greedy GET /documents/{document_id:path} route, which would otherwise
-        # capture "export"/"import" as a document id.
-        "/v1/default/banks/{bank_id}/document-transfer",
-        summary="Export documents",
-        description="Export documents (extracted facts, entity names, causal links, chunks) from a bank as a "
-        "transfer ZIP archive. Embeddings and database ids are not included — importing re-embeds with the target "
-        "bank's model and re-resolves entities. Consolidated observations are excluded unless include_observations=true. "
-        "Pass document_id query params to export specific documents, or omit to export the whole bank.",
-        operation_id="export_documents",
-        tags=["Document Transfer"],
-        responses={200: {"content": {"application/zip": {}}, "description": "Transfer archive"}},
-    )
-    async def api_export_documents(
-        bank_id: str,
-        document_id: list[str] | None = Query(default=None, description="Document id(s) to export; omit for all"),
-        include_observations: bool = Query(
-            default=False, description="Also export consolidated observations (restored on import)"
-        ),
-        request_context: RequestContext = Depends(get_request_context),
-    ):
-        """Export documents from a bank into a transfer ZIP archive."""
-        from fastapi.responses import Response
-
-        try:
-            if not get_config().enable_document_export_api:
-                raise HTTPException(
-                    status_code=404,
-                    detail="Document export API is disabled. "
-                    "Set HINDSIGHT_API_ENABLE_DOCUMENT_EXPORT_API=true to enable.",
-                )
-            profile = await app.state.memory.get_bank_profile(
-                bank_id, request_context=request_context, create_if_missing=False
-            )
-            if profile is None:
-                raise HTTPException(status_code=404, detail=f"Bank '{bank_id}' not found")
-
-            try:
-                archive = await app.state.memory.export_documents_async(
-                    bank_id,
-                    request_context,
-                    list(document_id) if document_id else None,
-                    include_observations=include_observations,
-                )
-            except ValueError as e:
-                # e.g. include_observations combined with a document_id subset.
-                raise HTTPException(status_code=400, detail=str(e))
-            return Response(
-                content=archive,
-                media_type="application/zip",
-                headers={"Content-Disposition": f'attachment; filename="{bank_id}-documents.zip"'},
-            )
-        except OperationValidationError as e:
-            raise HTTPException(status_code=e.status_code, detail=e.reason)
-        except (AuthenticationError, HTTPException):
-            raise
-        except Exception as e:
-            import traceback
-
-            logger.error(f"Error in GET /v1/default/banks/{bank_id}/document-transfer: {traceback.format_exc()}")
-            raise HTTPException(status_code=500, detail=str(e))
-
-    @app.post(
-        "/v1/default/banks/{bank_id}/document-transfer",
-        response_model=DocumentImportSubmitResponse,
-        status_code=202,
-        summary="Import documents (async)",
-        description="Submit a transfer archive (produced by the export endpoint) for import into a bank. Runs as a "
-        "background operation: facts are re-embedded with the target bank's embedding model and entities are "
-        "re-resolved — no LLM extraction. Returns an operation_id; poll "
-        "GET /v1/default/banks/{bank_id}/operations/{operation_id} for status and the imported/skipped counts in "
-        "result_metadata. Use on_conflict to control existing document ids: skip (default), replace, or new-id.",
-        operation_id="import_documents",
-        tags=["Document Transfer"],
-    )
-    @audited("import_documents", request_param=None)
-    async def api_import_documents(
-        bank_id: str,
-        file: UploadFile = File(..., description="Transfer ZIP archive"),
-        on_conflict: str = Query(default="skip", description="skip | replace | new-id"),
-        request_context: RequestContext = Depends(get_request_context),
-    ):
-        """Submit a transfer archive for async import into a bank."""
-        try:
-            if not get_config().enable_document_import_api:
-                raise HTTPException(
-                    status_code=404,
-                    detail="Document import API is disabled. "
-                    "Set HINDSIGHT_API_ENABLE_DOCUMENT_IMPORT_API=true to enable.",
-                )
-            if on_conflict not in ("skip", "replace", "new-id"):
-                raise HTTPException(
-                    status_code=400, detail=f"Invalid on_conflict '{on_conflict}' (expected skip|replace|new-id)"
-                )
-            archive_bytes = await file.read()
-            try:
-                submission = await app.state.memory.import_documents_async(
-                    bank_id, archive_bytes, request_context, on_conflict
-                )
-            except ValueError as e:
-                # Invalid archive / unsupported schema version — fail fast.
-                raise HTTPException(status_code=400, detail=str(e))
-            return DocumentImportSubmitResponse(operation_id=submission["operation_id"])
-        except OperationValidationError as e:
-            raise HTTPException(status_code=e.status_code, detail=e.reason)
-        except (AuthenticationError, HTTPException):
-            raise
-        except Exception as e:
-            import traceback
-
-            logger.error(f"Error in POST /v1/default/banks/{bank_id}/document-transfer: {traceback.format_exc()}")
-            raise HTTPException(status_code=500, detail=str(e))
-
    @app.get(
        "/v1/bank-template-schema",
        summary="Get bank template JSON Schema",
@@ -6465,13 +6202,48 @@ def _register_routes(app: FastAPI):
            raise HTTPException(status_code=500, detail=str(e))

    # ---- Audit Logs ----
-    # Response models live in engine/audit.py so the MemoryEngine read methods
-    # (list_audit_logs / audit_log_stats) can build and return them directly.

-    # ---- LLM Request Traces ----
-    # Response models + queries live in the engine (engine/llm_trace.py and
-    # MemoryEngine.list_llm_requests / llm_request_stats). The handlers below
-    # only parse params, delegate to the engine, and map a missing bank to 404.
+    class AuditLogEntry(BaseModel):
+        """A single audit log entry."""
+
+        id: str
+        action: str
+        transport: str
+        bank_id: str | None
+        started_at: str | None
+        ended_at: str | None
+        duration_ms: int | None = Field(
+            default=None,
+            description="Server-computed duration in milliseconds (started_at → ended_at). Null if not yet completed.",
+        )
+        request: dict[str, Any] | None
+        response: dict[str, Any] | None
+        metadata: dict[str, Any]
+
+    class AuditLogListResponse(BaseModel):
+        """Response model for list audit logs endpoint."""
+
+        bank_id: str
+        total: int
+        limit: int
+        offset: int
+        items: list[AuditLogEntry]
+
+    class AuditLogStatsBucket(BaseModel):
+        """A single time bucket in audit log stats."""
+
+        time: str
+        actions: dict[str, int]
+        total: int
+
+    class AuditLogStatsResponse(BaseModel):
+        """Response model for audit log stats endpoint."""
+
+        bank_id: str
+        period: str
+        trunc: str
+        start: str
+        buckets: list[AuditLogStatsBucket]

    @app.get(
        "/v1/default/banks/{bank_id}/audit-logs",
@@ -6493,19 +6265,120 @@ def _register_routes(app: FastAPI):
    ):
        """List audit log entries for a bank."""
        try:
-            result = await app.state.memory.list_audit_logs(
-                bank_id,
-                request_context=request_context,
-                action=action,
-                transport=transport,
-                start_date=datetime.fromisoformat(start_date.replace("Z", "+00:00")) if start_date else None,
-                end_date=datetime.fromisoformat(end_date.replace("Z", "+00:00")) if end_date else None,
-                limit=limit,
-                offset=offset,
-            )
-            if result is None:
+            from hindsight_api.engine.memory_engine import fq_table
+
+            pool = await app.state.memory._get_backend()
+
+            # Read endpoint: verify bank exists without auto-creating it.
+            if (
+                await app.state.memory.get_bank_profile(
+                    bank_id, request_context=request_context, create_if_missing=False
+                )
+                is None
+            ):
                raise HTTPException(status_code=404, detail=f"Bank '{bank_id}' not found")
-            return result
+
+            from hindsight_api.engine.db_utils import acquire_with_retry
+
+            async with acquire_with_retry(pool) as conn:
+                where_clauses = ["bank_id = $1"]
+                params: list[Any] = [bank_id]
+                idx = 2
+
+                if action:
+                    where_clauses.append(f"action = ${idx}")
+                    params.append(action)
+                    idx += 1
+
+                if transport:
+                    where_clauses.append(f"transport = ${idx}")
+                    params.append(transport)
+                    idx += 1
+
+                if start_date:
+                    parsed_start = datetime.fromisoformat(start_date.replace("Z", "+00:00"))
+                    where_clauses.append(f"started_at >= ${idx}")
+                    params.append(parsed_start)
+                    idx += 1
+
+                if end_date:
+                    parsed_end = datetime.fromisoformat(end_date.replace("Z", "+00:00"))
+                    where_clauses.append(f"started_at < ${idx}")
+                    params.append(parsed_end)
+                    idx += 1
+
+                where_sql = " AND ".join(where_clauses)
+                table = fq_table("audit_log")
+
+                # Get total count
+                count_row = await conn.fetchrow(
+                    f"SELECT COUNT(*) as total FROM {table} WHERE {where_sql}",
+                    *params,
+                )
+                total = count_row["total"] if count_row else 0
+
+                # Get paginated results
+                params.append(limit)
+                params.append(offset)
+                rows = await conn.fetch(
+                    f"""
+                    SELECT id, action, transport, bank_id, started_at, ended_at,
+                           request, response, metadata
+                    FROM {table}
+                    WHERE {where_sql}
+                    ORDER BY started_at DESC
+                    LIMIT ${idx} OFFSET ${idx + 1}
+                    """,
+                    *params,
+                )
+
+                items = []
+                for row in rows:
+                    duration_ms = None
+                    started = row["started_at"]
+                    ended = row["ended_at"]
+                    if started and ended and hasattr(started, "total_seconds"):
+                        duration_ms = int((ended - started).total_seconds() * 1000)
+                    elif started and ended:
+                        try:
+                            duration_ms = int((ended - started).total_seconds() * 1000)
+                        except (TypeError, AttributeError):
+                            pass
+
+                    def _safe_iso(val):
+                        if val is None:
+                            return None
+                        return val.isoformat() if hasattr(val, "isoformat") else str(val)
+
+                    def _safe_json(val):
+                        if val is None:
+                            return None
+                        if isinstance(val, dict):
+                            return val
+                        return json.loads(val) if isinstance(val, str) else val
+
+                    items.append(
+                        {
+                            "id": str(row["id"]),
+                            "action": row["action"],
+                            "transport": row["transport"],
+                            "bank_id": row["bank_id"],
+                            "started_at": _safe_iso(started),
+                            "ended_at": _safe_iso(ended),
+                            "duration_ms": duration_ms,
+                            "request": _safe_json(row["request"]),
+                            "response": _safe_json(row["response"]),
+                            "metadata": _safe_json(row["metadata"]) or {},
+                        }
+                    )
+
+                return {
+                    "bank_id": bank_id,
+                    "total": total,
+                    "limit": limit,
+                    "offset": offset,
+                    "items": items,
+                }
        except OperationValidationError as e:
            raise HTTPException(status_code=e.status_code, detail=e.reason)
        except (AuthenticationError, HTTPException):
@@ -6532,15 +6405,72 @@ def _register_routes(app: FastAPI):
    ):
        """Get audit log counts grouped by time bucket."""
        try:
-            result = await app.state.memory.audit_log_stats(
-                bank_id,
-                request_context=request_context,
-                action=action,
-                period=period,
-            )
-            if result is None:
+            from hindsight_api.engine.db_utils import acquire_with_retry
+            from hindsight_api.engine.memory_engine import fq_table
+
+            pool = await app.state.memory._get_backend()
+            # Read endpoint: verify bank exists without auto-creating it.
+            if (
+                await app.state.memory.get_bank_profile(
+                    bank_id, request_context=request_context, create_if_missing=False
+                )
+                is None
+            ):
                raise HTTPException(status_code=404, detail=f"Bank '{bank_id}' not found")
-            return result
+
+            # Determine time range (always per-day buckets)
+            from datetime import timedelta as _td
+
+            now = datetime.now(timezone.utc)
+            trunc = "day"
+            if period == "1d":
+                start = now - _td(days=1)
+            elif period == "30d":
+                start = now - _td(days=30)
+            else:  # 7d default
+                start = now - _td(days=7)
+
+            table = fq_table("audit_log")
+
+            async with acquire_with_retry(pool) as conn:
+                where_clauses = ["bank_id = $1", "started_at >= $2"]
+                params: list[Any] = [bank_id, start]
+                idx = 3
+
+                if action:
+                    where_clauses.append(f"action = ${idx}")
+                    params.append(action)
+                    idx += 1
+
+                where_sql = " AND ".join(where_clauses)
+
+                rows = await conn.fetch(
+                    f"""
+                    SELECT date_trunc('{trunc}', started_at) AS bucket,
+                           action,
+                           COUNT(*) AS count
+                    FROM {table}
+                    WHERE {where_sql}
+                    GROUP BY bucket, action
+                    ORDER BY bucket ASC
+                    """,
+                    *params,
+                )
+
+                buckets: dict[str, dict[str, int]] = {}
+                for row in rows:
+                    bucket_key = row["bucket"].isoformat()
+                    if bucket_key not in buckets:
+                        buckets[bucket_key] = {}
+                    buckets[bucket_key][row["action"]] = row["count"]
+
+                return {
+                    "bank_id": bank_id,
+                    "period": period,
+                    "trunc": trunc,
+                    "start": start.isoformat(),
+                    "buckets": [{"time": k, "actions": v, "total": sum(v.values())} for k, v in buckets.items()],
+                }
        except OperationValidationError as e:
            raise HTTPException(status_code=e.status_code, detail=e.reason)
        except (AuthenticationError, HTTPException):
@@ -6550,98 +6480,3 @@ def _register_routes(app: FastAPI):

            logger.error(f"Error getting audit log stats: {traceback.format_exc()}")
            raise HTTPException(status_code=500, detail=str(e))
-
-    @app.get(
-        "/v1/default/banks/{bank_id}/llm-requests",
-        summary="List LLM request traces",
-        description="List traced LLM requests for a bank, ordered by most recent first. "
-        "Requires LLM request tracing to be enabled (HINDSIGHT_API_LLM_TRACE_ENABLED).",
-        operation_id="list_llm_requests",
-        tags=["LLM Traces"],
-        response_model=LLMRequestListResponse,
-    )
-    async def api_list_llm_requests(
-        bank_id: str,
-        status: str | None = Query(None, description="Filter by status (success, error)"),
-        operation: str | None = Query(None, description="Filter by operation (retain, reflect, consolidation)"),
-        scope: str | None = Query(None, description="Filter by call scope"),
-        provider: str | None = Query(None, description="Filter by LLM provider"),
-        trace_id: str | None = Query(None, description="Filter to one operation run (all LLM calls sharing a trace)"),
-        document_id: str | None = Query(None, description="Filter to LLM calls that processed a given document"),
-        memory_id: str | None = Query(
-            None, description="Filter to the operation run(s) that produced or consumed a given memory_unit"
-        ),
-        group: bool = Query(
-            False, description="Paginate by operation run (trace) instead of by call; returns whole runs"
-        ),
-        start_date: str | None = Query(None, description="Filter from this ISO datetime (inclusive)"),
-        end_date: str | None = Query(None, description="Filter until this ISO datetime (exclusive)"),
-        limit: int = Query(50, ge=1, le=500, description="Max items to return"),
-        offset: int = Query(0, ge=0, description="Offset for pagination"),
-        request_context: RequestContext = Depends(get_request_context),
-    ):
-        """List traced LLM requests for a bank."""
-        try:
-            result = await app.state.memory.list_llm_requests(
-                bank_id,
-                request_context=request_context,
-                status=status,
-                operation=operation,
-                scope=scope,
-                provider=provider,
-                trace_id=trace_id,
-                document_id=document_id,
-                memory_id=memory_id,
-                group=group,
-                start_date=datetime.fromisoformat(start_date.replace("Z", "+00:00")) if start_date else None,
-                end_date=datetime.fromisoformat(end_date.replace("Z", "+00:00")) if end_date else None,
-                limit=limit,
-                offset=offset,
-            )
-            if result is None:
-                raise HTTPException(status_code=404, detail=f"Bank '{bank_id}' not found")
-            return result
-        except OperationValidationError as e:
-            raise HTTPException(status_code=e.status_code, detail=e.reason)
-        except (AuthenticationError, HTTPException):
-            raise
-        except Exception as e:
-            import traceback
-
-            logger.error(f"Error listing LLM requests: {traceback.format_exc()}")
-            raise HTTPException(status_code=500, detail=str(e))
-
-    @app.get(
-        "/v1/default/banks/{bank_id}/llm-requests/stats",
-        summary="LLM request statistics",
-        description="Get LLM request counts grouped by time bucket and status for charting.",
-        operation_id="llm_request_stats",
-        tags=["LLM Traces"],
-        response_model=LLMRequestStatsResponse,
-    )
-    async def api_llm_request_stats(
-        bank_id: str,
-        operation: str | None = Query(None, description="Filter by operation"),
-        period: str = Query("7d", description="Time period: 1d, 7d, or 30d"),
-        request_context: RequestContext = Depends(get_request_context),
-    ):
-        """Get LLM request counts grouped by time bucket and status."""
-        try:
-            result = await app.state.memory.llm_request_stats(
-                bank_id,
-                request_context=request_context,
-                operation=operation,
-                period=period,
-            )
-            if result is None:
-                raise HTTPException(status_code=404, detail=f"Bank '{bank_id}' not found")
-            return result
-        except OperationValidationError as e:
-            raise HTTPException(status_code=e.status_code, detail=e.reason)
-        except (AuthenticationError, HTTPException):
-            raise
-        except Exception as e:
-            import traceback
-
-            logger.error(f"Error getting LLM request stats: {traceback.format_exc()}")
-            raise HTTPException(status_code=500, detail=str(e))
@@ -143,7 +143,6 @@ ENV_LLM_GROQ_SERVICE_TIER = "HINDSIGHT_API_LLM_GROQ_SERVICE_TIER"
 ENV_LLM_OPENAI_SERVICE_TIER = "HINDSIGHT_API_LLM_OPENAI_SERVICE_TIER"
 ENV_LLM_EXTRA_BODY = "HINDSIGHT_API_LLM_EXTRA_BODY"
 ENV_LLM_DEFAULT_HEADERS = "HINDSIGHT_API_LLM_DEFAULT_HEADERS"
-ENV_LLM_STRICT_SCHEMA = "HINDSIGHT_API_LLM_STRICT_SCHEMA"

 # LiteLLM Router chain — provider-specific config consumed by the "litellmrouter"
 # provider. Each entry is a deployment; the Router tries them in declared order and
@@ -210,17 +209,6 @@ ENV_EMBEDDINGS_PROVIDER = "HINDSIGHT_API_EMBEDDINGS_PROVIDER"
 ENV_EMBEDDINGS_LOCAL_MODEL = "HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL"
 ENV_EMBEDDINGS_LOCAL_FORCE_CPU = "HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU"
 ENV_EMBEDDINGS_LOCAL_TRUST_REMOTE_CODE = "HINDSIGHT_API_EMBEDDINGS_LOCAL_TRUST_REMOTE_CODE"
-ENV_EMBEDDINGS_ONNX_MODEL_ID = "HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_ID"
-ENV_EMBEDDINGS_ONNX_MODEL_PATH = "HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_PATH"
-ENV_EMBEDDINGS_ONNX_TOKENIZER_NAME_OR_PATH = "HINDSIGHT_API_EMBEDDINGS_ONNX_TOKENIZER_NAME_OR_PATH"
-ENV_EMBEDDINGS_ONNX_FILE = "HINDSIGHT_API_EMBEDDINGS_ONNX_FILE"
-ENV_EMBEDDINGS_ONNX_DIMENSIONS = "HINDSIGHT_API_EMBEDDINGS_ONNX_DIMENSIONS"
-ENV_EMBEDDINGS_ONNX_MAX_TOKENS = "HINDSIGHT_API_EMBEDDINGS_ONNX_MAX_TOKENS"
-ENV_EMBEDDINGS_ONNX_POOLING = "HINDSIGHT_API_EMBEDDINGS_ONNX_POOLING"
-ENV_EMBEDDINGS_ONNX_NORMALIZE = "HINDSIGHT_API_EMBEDDINGS_ONNX_NORMALIZE"
-ENV_EMBEDDINGS_ONNX_QUERY_PREFIX = "HINDSIGHT_API_EMBEDDINGS_ONNX_QUERY_PREFIX"
-ENV_EMBEDDINGS_ONNX_PASSAGE_PREFIX = "HINDSIGHT_API_EMBEDDINGS_ONNX_PASSAGE_PREFIX"
-ENV_EMBEDDINGS_ONNX_OUTPUT_NAME = "HINDSIGHT_API_EMBEDDINGS_ONNX_OUTPUT_NAME"
 ENV_EMBEDDINGS_TEI_URL = "HINDSIGHT_API_EMBEDDINGS_TEI_URL"
 ENV_EMBEDDINGS_OPENAI_API_KEY = "HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY"
 ENV_EMBEDDINGS_OPENAI_MODEL = "HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL"
@@ -309,7 +297,6 @@ ENV_RERANKER_LITELLM_TIMEOUT = "HINDSIGHT_API_RERANKER_LITELLM_TIMEOUT"
 ENV_RERANKER_LITELLM_SDK_TIMEOUT = "HINDSIGHT_API_RERANKER_LITELLM_SDK_TIMEOUT"
 ENV_RERANKER_GOOGLE_TIMEOUT = "HINDSIGHT_API_RERANKER_GOOGLE_TIMEOUT"
 ENV_RERANKER_MAX_CANDIDATES = "HINDSIGHT_API_RERANKER_MAX_CANDIDATES"
-ENV_SEMANTIC_MIN_SIMILARITY = "HINDSIGHT_API_SEMANTIC_MIN_SIMILARITY"
 ENV_RERANKER_FLASHRANK_MODEL = "HINDSIGHT_API_RERANKER_FLASHRANK_MODEL"
 ENV_RERANKER_FLASHRANK_CACHE_DIR = "HINDSIGHT_API_RERANKER_FLASHRANK_CACHE_DIR"
 ENV_RERANKER_FLASHRANK_CPU_MEM_ARENA = "HINDSIGHT_API_RERANKER_FLASHRANK_CPU_MEM_ARENA"
@@ -351,8 +338,6 @@ ENV_MCP_ENABLED = "HINDSIGHT_API_MCP_ENABLED"
 ENV_MCP_ENABLED_TOOLS = "HINDSIGHT_API_MCP_ENABLED_TOOLS"
 ENV_MCP_STATELESS = "HINDSIGHT_API_MCP_STATELESS"
 ENV_ENABLE_BANK_CONFIG_API = "HINDSIGHT_API_ENABLE_BANK_CONFIG_API"
-ENV_ENABLE_ADMIN_API = "HINDSIGHT_API_ENABLE_ADMIN_API"
-ENV_ADMIN_API_TOKEN = "HINDSIGHT_API_ADMIN_TOKEN"
 ENV_DEFAULT_BANK_TEMPLATE = "HINDSIGHT_API_DEFAULT_BANK_TEMPLATE"
 ENV_GRAPH_RETRIEVER = "HINDSIGHT_API_GRAPH_RETRIEVER"
 ENV_RECALL_MAX_CONCURRENT = "HINDSIGHT_API_RECALL_MAX_CONCURRENT"
@@ -361,8 +346,6 @@ ENV_RECALL_MAX_QUERY_TOKENS = "HINDSIGHT_API_RECALL_MAX_QUERY_TOKENS"
 ENV_MENTAL_MODEL_REFRESH_CONCURRENCY = "HINDSIGHT_API_MENTAL_MODEL_REFRESH_CONCURRENCY"
 ENV_LINK_EXPANSION_PER_ENTITY_LIMIT = "HINDSIGHT_API_LINK_EXPANSION_PER_ENTITY_LIMIT"
 ENV_LINK_EXPANSION_TIMEOUT = "HINDSIGHT_API_LINK_EXPANSION_TIMEOUT"
-ENV_BANK_STATS_CACHE_TTL_SECONDS = "HINDSIGHT_API_BANK_STATS_CACHE_TTL_SECONDS"
-ENV_BANK_STATS_CACHE_MAX_ENTRIES = "HINDSIGHT_API_BANK_STATS_CACHE_MAX_ENTRIES"

 # OpenTelemetry tracing configuration
 ENV_OTEL_TRACES_ENABLED = "HINDSIGHT_API_OTEL_TRACES_ENABLED"
@@ -380,16 +363,6 @@ ENV_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY = "HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOU
 # Gemini safety settings
 ENV_LLM_GEMINI_SAFETY_SETTINGS = "HINDSIGHT_API_LLM_GEMINI_SAFETY_SETTINGS"

-# Gemini prompt caching. When enabled, retain fact-extraction reuses a
-# CachedContent prefix for the static system_instruction + response_schema,
-# cutting per-call input cost on workloads with many small documents.
-# Provider-agnostic prompt-prefix caching. Providers that support it (currently
-# Gemini/Vertex via CachedContent) reuse the large, fixed, bank-agnostic system
-# prefix at the cached-input rate; providers that don't simply ignore it. On by
-# default — the prefix is bank-agnostic so a single cache is shared across all
-# banks, and creation soft-fails to an uncached call, so it never breaks a request.
-ENV_LLM_PROMPT_CACHE_ENABLED = "HINDSIGHT_API_LLM_PROMPT_CACHE_ENABLED"
-
 # Retain settings
 ENV_RETAIN_MAX_COMPLETION_TOKENS = "HINDSIGHT_API_RETAIN_MAX_COMPLETION_TOKENS"
 ENV_RETAIN_CHUNK_SIZE = "HINDSIGHT_API_RETAIN_CHUNK_SIZE"
@@ -427,20 +400,14 @@ ENV_FILE_CONVERSION_MAX_BATCH_SIZE = "HINDSIGHT_API_FILE_CONVERSION_MAX_BATCH_SI
 ENV_ENABLE_FILE_UPLOAD_API = "HINDSIGHT_API_ENABLE_FILE_UPLOAD_API"
 ENV_FILE_DELETE_AFTER_RETAIN = "HINDSIGHT_API_FILE_DELETE_AFTER_RETAIN"

-# Document transfer (export/import documents between banks without re-running the LLM)
-ENV_ENABLE_DOCUMENT_EXPORT_API = "HINDSIGHT_API_ENABLE_DOCUMENT_EXPORT_API"
-ENV_ENABLE_DOCUMENT_IMPORT_API = "HINDSIGHT_API_ENABLE_DOCUMENT_IMPORT_API"
-
 # Observations settings (consolidated knowledge from facts)
 ENV_ENABLE_OBSERVATIONS = "HINDSIGHT_API_ENABLE_OBSERVATIONS"
 ENV_ENABLE_AUTO_CONSOLIDATION = "HINDSIGHT_API_ENABLE_AUTO_CONSOLIDATION"
 ENV_CONSOLIDATION_BATCH_SIZE = "HINDSIGHT_API_CONSOLIDATION_BATCH_SIZE"
 ENV_CONSOLIDATION_MAX_MEMORIES_PER_ROUND = "HINDSIGHT_API_CONSOLIDATION_MAX_MEMORIES_PER_ROUND"
 ENV_CONSOLIDATION_LLM_BATCH_SIZE = "HINDSIGHT_API_CONSOLIDATION_LLM_BATCH_SIZE"
-ENV_CONSOLIDATION_DEDUP_THRESHOLD = "HINDSIGHT_API_CONSOLIDATION_DEDUP_THRESHOLD"
 ENV_CONSOLIDATION_LLM_PARALLELISM = "HINDSIGHT_API_CONSOLIDATION_LLM_PARALLELISM"
 ENV_CONSOLIDATION_MAX_TOKENS = "HINDSIGHT_API_CONSOLIDATION_MAX_TOKENS"
-ENV_CONSOLIDATION_MAX_COMPLETION_TOKENS = "HINDSIGHT_API_CONSOLIDATION_MAX_COMPLETION_TOKENS"
 ENV_CONSOLIDATION_SOURCE_FACTS_MAX_TOKENS = "HINDSIGHT_API_CONSOLIDATION_SOURCE_FACTS_MAX_TOKENS"
 ENV_CONSOLIDATION_SOURCE_FACTS_MAX_TOKENS_PER_OBSERVATION = (
    "HINDSIGHT_API_CONSOLIDATION_SOURCE_FACTS_MAX_TOKENS_PER_OBSERVATION"
@@ -450,7 +417,6 @@ ENV_CONSOLIDATION_MAX_ATTEMPTS = "HINDSIGHT_API_CONSOLIDATION_MAX_ATTEMPTS"
 ENV_OBSERVATIONS_MISSION = "HINDSIGHT_API_OBSERVATIONS_MISSION"
 ENV_MAX_OBSERVATIONS_PER_SCOPE = "HINDSIGHT_API_MAX_OBSERVATIONS_PER_SCOPE"
 ENV_ENABLE_OBSERVATION_HISTORY = "HINDSIGHT_API_ENABLE_OBSERVATION_HISTORY"
-ENV_OBSERVATION_HISTORY_MAX_ENTRIES = "HINDSIGHT_API_OBSERVATION_HISTORY_MAX_ENTRIES"
 ENV_ENABLE_MENTAL_MODEL_HISTORY = "HINDSIGHT_API_ENABLE_MENTAL_MODEL_HISTORY"
 ENV_MENTAL_MODEL_HISTORY_MAX_ENTRIES = "HINDSIGHT_API_MENTAL_MODEL_HISTORY_MAX_ENTRIES"

@@ -482,11 +448,6 @@ ENV_DB_COMMAND_TIMEOUT = "HINDSIGHT_API_DB_COMMAND_TIMEOUT"
 ENV_DB_ACQUIRE_TIMEOUT = "HINDSIGHT_API_DB_ACQUIRE_TIMEOUT"
 ENV_DB_STATEMENT_TIMEOUT = "HINDSIGHT_API_DB_STATEMENT_TIMEOUT"

-# Wall-clock cap on model/connection initialization at startup. If embeddings,
-# cross-encoder, or LLM verification hang (e.g. an offline HuggingFace download
-# or an unreachable provider), the daemon fails fast instead of hanging forever.
-ENV_MODEL_INIT_TIMEOUT = "HINDSIGHT_API_MODEL_INIT_TIMEOUT"
-
 # Worker configuration (distributed task processing)
 ENV_WORKER_ENABLED = "HINDSIGHT_API_WORKER_ENABLED"
 ENV_WORKER_ID = "HINDSIGHT_API_WORKER_ID"
@@ -507,7 +468,6 @@ WORKER_SLOT_RESERVATION_TYPES: dict[str, tuple[str, int]] = {
    "file_convert_retain": ("HINDSIGHT_API_WORKER_FILE_CONVERT_RETAIN_MAX_SLOTS", 0),
    "refresh_mental_model": ("HINDSIGHT_API_WORKER_REFRESH_MENTAL_MODEL_MAX_SLOTS", 0),
    "graph_maintenance": ("HINDSIGHT_API_WORKER_GRAPH_MAINTENANCE_MAX_SLOTS", 0),
-    "import_documents": ("HINDSIGHT_API_WORKER_IMPORT_DOCUMENTS_MAX_SLOTS", 0),
 }
 ENV_WORKER_CONSOLIDATION_BANK_PRIORITY = "HINDSIGHT_API_WORKER_CONSOLIDATION_BANK_PRIORITY"
 ENV_RETAIN_MAX_CONCURRENT = "HINDSIGHT_API_RETAIN_MAX_CONCURRENT"
@@ -533,32 +493,11 @@ ENV_RECALL_BUDGET_ADAPTIVE_HIGH = "HINDSIGHT_API_RECALL_BUDGET_ADAPTIVE_HIGH"
 ENV_RECALL_BUDGET_MIN = "HINDSIGHT_API_RECALL_BUDGET_MIN"
 ENV_RECALL_BUDGET_MAX = "HINDSIGHT_API_RECALL_BUDGET_MAX"

-# Recall candidate gating (per-source cap + BM25 score floor)
-ENV_BM25_MIN_SCORE = "HINDSIGHT_API_BM25_MIN_SCORE"
-ENV_RECALL_MAX_CANDIDATES_PER_SOURCE = "HINDSIGHT_API_RECALL_MAX_CANDIDATES_PER_SOURCE"
-# Per-strategy recall boost. Prioritises specific retrieval arms (semantic,
-# bm25, graph, temporal) on recall via a human priority level — e.g.
-# "graph:high" to strongly favour graph hits, or "graph:high,semantic:low".
-# Valid levels: low | medium | high. The level (not a raw number) is the knob
-# because the boost is applied on two different score scales — see
-# engine/search/recall_boost.py for the level -> magnitude mapping and rationale.
-# Empty disables the feature.
-ENV_RECALL_STRATEGY_BOOSTS = "HINDSIGHT_API_RECALL_STRATEGY_BOOSTS"
-
 # Audit log settings
 ENV_AUDIT_LOG_ENABLED = "HINDSIGHT_API_AUDIT_LOG_ENABLED"
 ENV_AUDIT_LOG_ACTIONS = "HINDSIGHT_API_AUDIT_LOG_ACTIONS"
 ENV_AUDIT_LOG_RETENTION_DAYS = "HINDSIGHT_API_AUDIT_LOG_RETENTION_DAYS"

-# LLM request tracing settings
-ENV_LLM_TRACE_ENABLED = "HINDSIGHT_API_LLM_TRACE_ENABLED"
-ENV_LLM_TRACE_SCOPES = "HINDSIGHT_API_LLM_TRACE_SCOPES"
-ENV_LLM_TRACE_RETENTION_DAYS = "HINDSIGHT_API_LLM_TRACE_RETENTION_DAYS"
-ENV_LLM_TRACE_MAX_CHARS = "HINDSIGHT_API_LLM_TRACE_MAX_CHARS"
-
-# Background maintenance settings
-ENV_CONSOLIDATION_RECONCILE_INTERVAL_SECONDS = "HINDSIGHT_API_CONSOLIDATION_RECONCILE_INTERVAL_SECONDS"
-
 # Disposition settings
 ENV_DISPOSITION_SKEPTICISM = "HINDSIGHT_API_DISPOSITION_SKEPTICISM"
 ENV_DISPOSITION_LITERALISM = "HINDSIGHT_API_DISPOSITION_LITERALISM"
@@ -574,9 +513,9 @@ DEFAULT_LLM_PROVIDER = "openai"
 PROVIDER_DEFAULT_MODELS = {
    "openai": "gpt-4o-mini",
    "anthropic": "claude-haiku-4-5",
-    "gemini": "gemini-3.5-flash",
+    "gemini": "gemini-2.5-flash",
    "groq": "openai/gpt-oss-120b",
-    "minimax": "MiniMax-M3",
+    "minimax": "MiniMax-M2.7",
    "deepseek": "deepseek-v4-flash",
    "zai": "glm-4.5-flash",
    "opencode-go": "deepseek-v4-flash",
@@ -584,7 +523,7 @@ PROVIDER_DEFAULT_MODELS = {
    "ollama-cloud": "gemma3:12b",
    "llamacpp": "gemma-4-e2b-it",
    "lmstudio": "local-model",
-    "vertexai": "google/gemini-3.1-flash-lite",
+    "vertexai": "google/gemini-2.5-flash-lite",
    "openai-codex": "gpt-5.4-mini",
    "claude-code": "claude-sonnet-4-5-20250929",
    "mock": "mock-model",
@@ -603,14 +542,6 @@ DEFAULT_LLAMACPP_CHAT_FORMAT = None  # None = auto-detect from GGUF metadata
 DEFAULT_LLAMACPP_NO_GRAMMAR = False  # True = disable JSON grammar enforcement (faster but less reliable)
 DEFAULT_LLAMACPP_EXTRA_ARGS = None  # Space-separated extra CLI args for llama.cpp server

-# True = ask schema-capable backends to grammar-enforce structured output via
-# json_schema strict (OpenAI-compatible, LiteLLM; Gemini already enforces its
-# native response_schema). Default False keeps the soft "schema-in-prompt +
-# json_object" path, which weaker self-hosted instruction-followers can violate
-# (prose preambles, markdown fences, invalid JSON) — wedging retain/consolidation
-# on parse retries.
-DEFAULT_LLM_STRICT_SCHEMA = False
-
 DEFAULT_LLM_MAX_CONCURRENT = 32
 DEFAULT_LLM_MAX_RETRIES = 3  # Max retry attempts for LLM API calls
 DEFAULT_LLM_INITIAL_BACKOFF = 1.0  # Initial backoff in seconds for retry exponential backoff
@@ -630,13 +561,6 @@ DEFAULT_EMBEDDINGS_PROVIDER = "local"
 DEFAULT_EMBEDDINGS_LOCAL_MODEL = "BAAI/bge-small-en-v1.5"
 DEFAULT_EMBEDDINGS_LOCAL_FORCE_CPU = False  # Force CPU mode for local embeddings
 DEFAULT_EMBEDDINGS_LOCAL_TRUST_REMOTE_CODE = False  # Security: disabled by default, required for some models
-DEFAULT_EMBEDDINGS_ONNX_MODEL_ID = "intfloat/multilingual-e5-small"
-DEFAULT_EMBEDDINGS_ONNX_FILE = "onnx/model.onnx"
-DEFAULT_EMBEDDINGS_ONNX_MAX_TOKENS = 512
-DEFAULT_EMBEDDINGS_ONNX_POOLING = "mean"
-DEFAULT_EMBEDDINGS_ONNX_NORMALIZE = True
-DEFAULT_EMBEDDINGS_ONNX_QUERY_PREFIX = "query: "
-DEFAULT_EMBEDDINGS_ONNX_PASSAGE_PREFIX = "passage: "
 DEFAULT_EMBEDDINGS_OPENAI_MODEL = "text-embedding-3-small"
 DEFAULT_EMBEDDINGS_OPENAI_BATCH_SIZE = 100
 DEFAULT_EMBEDDINGS_GEMINI_MODEL = "gemini-embedding-001"
@@ -668,65 +592,6 @@ DEFAULT_RERANKER_LITELLM_TIMEOUT = 60.0
 DEFAULT_RERANKER_LITELLM_SDK_TIMEOUT = 60.0
 DEFAULT_RERANKER_GOOGLE_TIMEOUT = 60.0
 DEFAULT_RERANKER_MAX_CANDIDATES = 300
-DEFAULT_SEMANTIC_MIN_SIMILARITY = 0.3
-# Minimum BM25 score a row must exceed to enter fusion. 0.0 gates out
-# zero-score (non-matching) rows on backends — notably VectorChord — whose
-# operator ranks every document rather than pre-filtering to term matches.
-DEFAULT_BM25_MIN_SCORE = 0.0
-# Per-source candidate cap applied to each retrieval arm (semantic, BM25, graph,
-# temporal) before RRF, so a single over-expanding backend cannot fill the
-# reranker's global candidate budget on its own. 0 disables the cap.
-DEFAULT_RECALL_MAX_CANDIDATES_PER_SOURCE = 0
-# Per-strategy recall boost, as a comma-separated "strategy:level" list (e.g.
-# "graph:high,semantic:low"). Empty disables the feature. See
-# ENV_RECALL_STRATEGY_BOOSTS for the full rationale.
-DEFAULT_RECALL_STRATEGY_BOOSTS = ""
-# Retrieval arms that can be boosted; mirrors fusion.py source_names.
-RECALL_STRATEGY_NAMES = ("semantic", "bm25", "graph", "temporal")
-# User-facing priority levels. Kept in sync with recall_boost.BOOST_LEVELS by a
-# guard test; defined here (not imported) so config stays free of the heavy
-# engine.search import graph.
-RECALL_BOOST_LEVELS = ("low", "medium", "high")
-# Level applied when a strategy is listed without one (e.g. "graph" or "graph:").
-DEFAULT_RECALL_BOOST_LEVEL = "medium"
-
-
-def _parse_strategy_boosts(raw: str | None) -> dict[str, str]:
-    """Parse a "strategy:level,strategy:level" string into a boost map.
-
-    A strategy listed without a level (``"graph"`` or ``"graph:"``) defaults to
-    ``medium``. Only the strategies you list are boosted; any strategy you omit
-    keeps its normal, unboosted weight. Unknown strategy names, unknown levels,
-    and malformed entries are skipped with a warning so a typo degrades to a
-    no-op boost rather than breaking recall.
-    """
-    if not raw or not raw.strip():
-        return {}
-    boosts: dict[str, str] = {}
-    for entry in raw.split(","):
-        entry = entry.strip()
-        if not entry:
-            continue
-        name, _sep, level = entry.partition(":")
-        name = name.strip().lower()
-        level = level.strip().lower() or DEFAULT_RECALL_BOOST_LEVEL
-        if name not in RECALL_STRATEGY_NAMES:
-            logger.warning(
-                "Ignoring unknown recall strategy %r in boost (valid: %s)", name, ", ".join(RECALL_STRATEGY_NAMES)
-            )
-            continue
-        if level not in RECALL_BOOST_LEVELS:
-            logger.warning(
-                "Ignoring unknown recall boost level %r for %r (valid: %s)",
-                level,
-                name,
-                ", ".join(RECALL_BOOST_LEVELS),
-            )
-            continue
-        boosts[name] = level
-    return boosts
-
-
 DEFAULT_RERANKER_FLASHRANK_MODEL = "ms-marco-MiniLM-L-12-v2"  # Best balance of speed and quality
 DEFAULT_RERANKER_FLASHRANK_CACHE_DIR = None  # Use default cache directory
 DEFAULT_RERANKER_FLASHRANK_CPU_MEM_ARENA = False  # Disable ONNX CPU memory arena to bound RSS
@@ -794,8 +659,6 @@ DEFAULT_MCP_ENABLED = True
 DEFAULT_MCP_ENABLED_TOOLS: list[str] | None = None  # None = all tools enabled
 DEFAULT_MCP_STATELESS = False  # False = stateful (supports SSE/GET); True = stateless (POST-only)
 DEFAULT_ENABLE_BANK_CONFIG_API = True
-DEFAULT_ENABLE_ADMIN_API = False  # Admin surface (server config view) is off unless explicitly enabled
-DEFAULT_ADMIN_API_TOKEN: str | None = None  # None = admin API open (when enabled); set = required bearer token
 DEFAULT_DEFAULT_BANK_TEMPLATE: dict | None = None  # BankTemplateManifest dict applied to newly-created banks
 DEFAULT_GRAPH_RETRIEVER = "link_expansion"
 DEFAULT_RECALL_MAX_CONCURRENT = 32  # Max concurrent recall operations per worker
@@ -804,8 +667,6 @@ DEFAULT_RECALL_MAX_QUERY_TOKENS = 500  # Maximum tokens allowed in recall query
 DEFAULT_MENTAL_MODEL_REFRESH_CONCURRENCY = 8  # Max concurrent mental model refreshes
 DEFAULT_LINK_EXPANSION_PER_ENTITY_LIMIT = 200  # Max target units per entity in graph expansion
 DEFAULT_LINK_EXPANSION_TIMEOUT = 10.0  # Timeout (seconds) for entity expansion query
-DEFAULT_BANK_STATS_CACHE_TTL_SECONDS = 60.0  # TTL for get_bank_stats result cache; 0 disables
-DEFAULT_BANK_STATS_CACHE_MAX_ENTRIES = 1024  # LRU bound across (schema, bank) keys

 # Retain settings
 DEFAULT_RETAIN_MAX_COMPLETION_TOKENS = 64000  # Max tokens for fact extraction LLM call
@@ -824,7 +685,6 @@ DEFAULT_RETAIN_BATCH_TOKENS = 10_000  # ~40KB of text  # Max chars per sub-batch
 DEFAULT_RETAIN_ENTITY_LOOKUP = "trigram"  # "full" or "trigram"
 DEFAULT_RETAIN_ENTITY_RESOLUTION_BATCH_SIZE = 100  # Unique entity names per pg_trgm candidate lookup query
 DEFAULT_RETAIN_BATCH_ENABLED = False  # Use LLM Batch API for fact extraction (only when async=True)
-DEFAULT_LLM_PROMPT_CACHE_ENABLED = True  # Reuse the fixed system prefix via provider prompt caching
 DEFAULT_RETAIN_BATCH_POLL_INTERVAL_SECONDS = 60  # Batch API polling interval in seconds

 # File storage defaults
@@ -836,45 +696,28 @@ DEFAULT_FILE_CONVERSION_MAX_BATCH_SIZE = 10  # Max files per batch upload
 DEFAULT_ENABLE_FILE_UPLOAD_API = True  # Enable file upload endpoint
 DEFAULT_FILE_DELETE_AFTER_RETAIN = True  # Delete file bytes after retain (saves storage)

-# Document transfer defaults (export/import enabled by default; gated independently)
-DEFAULT_ENABLE_DOCUMENT_EXPORT_API = True
-DEFAULT_ENABLE_DOCUMENT_IMPORT_API = True
-
 # Observations defaults (consolidated knowledge from facts)
 DEFAULT_ENABLE_OBSERVATIONS = True  # Observations enabled by default
 DEFAULT_ENABLE_AUTO_CONSOLIDATION = True  # Auto-consolidation after retain enabled by default
 DEFAULT_ENABLE_OBSERVATION_HISTORY = True  # Observation history tracking enabled by default
 DEFAULT_ENABLE_MENTAL_MODEL_HISTORY = True  # Mental model history tracking enabled by default
-# History (mental-model refresh snapshots and observation update snapshots) lives in
-# the dedicated mental_model_history / observation_history tables, one row per change.
-# On every write we insert the new entry and delete the oldest rows beyond the cap,
-# so the per-item history can never grow unboundedly (the old single-JSONB-column
-# design hit Postgres's hard 256MB jsonb limit -> SQLSTATE 54000 and stuck rows).
-# 50 preserves enough recent history for meaningful audit / rollback per item.
-# A cap <= 0 removes the trim (unbounded growth) — to turn history OFF use the
-# enable_* flag, not a zero cap.
+# Each history entry snapshots previous_content + previous_reflect_response. Without
+# a cap, sustained mental-model refresh load grows the jsonb array unboundedly until
+# it crosses Postgres's hard 256MB jsonb limit and subsequent UPDATEs fail with
+# SQLSTATE 54000. 50 keeps the array well under 100MB even with large reflect
+# responses, while preserving enough recent history for meaningful audit / rollback.
 DEFAULT_MENTAL_MODEL_HISTORY_MAX_ENTRIES = 50
-DEFAULT_OBSERVATION_HISTORY_MAX_ENTRIES = 50
 DEFAULT_CONSOLIDATION_MAX_ATTEMPTS = 3  # Outer retry attempts for consolidation LLM batch calls
 DEFAULT_CONSOLIDATION_BATCH_SIZE = 50  # Memories to load per batch (internal memory optimization)
 DEFAULT_CONSOLIDATION_MAX_MEMORIES_PER_ROUND = (
    100  # Max memories per consolidation round (0 = unlimited). Limits how long one bank holds a worker slot.
 )
 DEFAULT_CONSOLIDATION_LLM_BATCH_SIZE = 8  # Facts per LLM call (1 = no batching; >1 = batch mode)
-# Cosine >= this between a newly-created or freshly-updated observation and an existing one
-# triggers a focused 1-by-1 LLM "merge or keep" pass (the LLM reads both, so numbers/negation/
-# entities are respected). Enabled by default; set to 1.0 to disable. Postgres only — the merge
-# path uses Postgres-only SQL, so consolidation skips it on Oracle regardless of this value.
-DEFAULT_CONSOLIDATION_DEDUP_THRESHOLD = 0.97
 DEFAULT_CONSOLIDATION_LLM_PARALLELISM = (
    4  # Max tag groups consolidated concurrently per op. Locks on overlapping write
    # scopes degrade to sequential automatically; matches retain_max_concurrent.
 )
 DEFAULT_CONSOLIDATION_MAX_TOKENS = 512  # Max tokens for recall when finding related observations
-# Unset by default: the key is omitted from the LLM call so every provider keeps its current implicit output
-# budget — 100% backwards compatible. Operators on providers with a low hidden default (notably Bedrock imported
-# models, which cap at 4096 and truncate structured consolidation JSON) set this explicitly to fix #1939.
-DEFAULT_CONSOLIDATION_MAX_COMPLETION_TOKENS = None
 DEFAULT_CONSOLIDATION_RECALL_BUDGET = "low"  # Budget level for consolidation recall (low/mid/high)
 DEFAULT_CONSOLIDATION_SOURCE_FACTS_MAX_TOKENS = (
    4096  # Total token budget for source facts in consolidation recall (-1 = unlimited)
@@ -894,7 +737,6 @@ DEFAULT_DB_POOL_MAX_SIZE = 100
 DEFAULT_DB_COMMAND_TIMEOUT = 60  # seconds
 DEFAULT_DB_ACQUIRE_TIMEOUT = 30  # seconds
 DEFAULT_DB_STATEMENT_TIMEOUT = 600  # seconds (Postgres statement_timeout applied on every pool connection; 0 disables)
-DEFAULT_MODEL_INIT_TIMEOUT = 300  # seconds (cap on startup model/connection init; covers first-time downloads)

 # Worker configuration (distributed task processing)
 DEFAULT_WORKER_ENABLED = True  # API runs worker by default (standalone mode)
@@ -947,18 +789,6 @@ DEFAULT_AUDIT_LOG_ENABLED = False  # Disabled by default
 DEFAULT_AUDIT_LOG_ACTIONS = ""  # Empty = audit all eligible actions
 DEFAULT_AUDIT_LOG_RETENTION_DAYS = -1  # -1 = keep forever

-# LLM request tracing defaults
-DEFAULT_LLM_TRACE_ENABLED = True  # Enabled by default
-DEFAULT_LLM_TRACE_SCOPES = ""  # Empty = trace all call scopes
-DEFAULT_LLM_TRACE_RETENTION_DAYS = 1  # Retain trace rows for 1 day by default
-DEFAULT_LLM_TRACE_MAX_CHARS = 50000  # Truncate stored input/output beyond this many chars
-
-# Background maintenance defaults
-# Periodic reconcile that re-schedules consolidation for banks with eligible-but-unscheduled
-# facts (e.g. after a consolidation operation failed terminally and left them unscheduled).
-# 0 disables the reconcile sweep.
-DEFAULT_CONSOLIDATION_RECONCILE_INTERVAL_SECONDS = 300
-
 # Default MCP tool descriptions (can be customized via env vars)
 DEFAULT_MCP_RETAIN_DESCRIPTION = """Store important information to long-term memory.

@@ -1222,7 +1052,6 @@ class HindsightConfig:
    llm_default_headers: (
        dict | None
    )  # Custom headers passed as default_headers to provider SDK clients (e.g. {"X-Component-Id": "hindsight"} for proxies / request tracing)
-    llm_strict_schema: bool  # Grammar-enforce structured output via the provider's strongest schema mode (see DEFAULT_LLM_STRICT_SCHEMA)

    # LiteLLM Router chain (provider-specific; consumed by the "litellmrouter" provider).
    # List of deployment dicts evaluated in order with fallback on transient errors.
@@ -1238,10 +1067,6 @@ class HindsightConfig:
    # Gemini safety settings (None = use Gemini defaults; list of dicts with category/threshold)
    llm_gemini_safety_settings: list | None

-    # Gemini prompt caching toggle. When True, retain extraction reuses a
-    # CachedContent prefix for its system prompt + response schema.
-    llm_prompt_cache_enabled: bool
-
    # Built-in llama.cpp configuration (for provider=llamacpp)
    llamacpp_model_path: str | None  # Path to GGUF file (None = auto-download default)
    llamacpp_gpu_layers: int  # -1 = all layers on GPU, 0 = CPU only
@@ -1294,17 +1119,6 @@ class HindsightConfig:
    embeddings_local_model: str
    embeddings_local_force_cpu: bool
    embeddings_local_trust_remote_code: bool
-    embeddings_onnx_model_id: str
-    embeddings_onnx_model_path: str | None
-    embeddings_onnx_tokenizer_name_or_path: str | None
-    embeddings_onnx_file: str
-    embeddings_onnx_dimensions: int | None
-    embeddings_onnx_max_tokens: int
-    embeddings_onnx_pooling: str
-    embeddings_onnx_normalize: bool
-    embeddings_onnx_query_prefix: str
-    embeddings_onnx_passage_prefix: str
-    embeddings_onnx_output_name: str | None
    embeddings_tei_url: str | None
    embeddings_openai_base_url: str | None
    embeddings_cohere_api_key: str | None
@@ -1344,10 +1158,6 @@ class HindsightConfig:
    reranker_tei_max_concurrent: int
    reranker_tei_http_timeout: float
    reranker_max_candidates: int
-    semantic_min_similarity: float
-    bm25_min_score: float
-    recall_max_candidates_per_source: int
-    recall_strategy_boosts: dict[str, str]
    reranker_cohere_api_key: str | None
    reranker_cohere_model: str
    reranker_cohere_base_url: str | None
@@ -1391,10 +1201,6 @@ class HindsightConfig:
    mcp_enabled_tools: list[str] | None  # None = all tools; explicit list = allowlist
    mcp_stateless: bool  # True = stateless HTTP (POST-only); False = stateful (supports GET/SSE)
    enable_bank_config_api: bool
-    # Admin surface (static, server-level only). enable_admin_api gates the /admin API +
-    # control-plane page; admin_api_token (when set) is the required bearer token.
-    enable_admin_api: bool
-    admin_api_token: str | None
    # Default bank template (static, server-level only). When set, the manifest is applied
    # to every newly-created bank, overriding the env/config defaults for any fields it sets.
    default_bank_template: dict | None
@@ -1407,8 +1213,6 @@ class HindsightConfig:
    mental_model_refresh_concurrency: int
    link_expansion_per_entity_limit: int
    link_expansion_timeout: float
-    bank_stats_cache_ttl_seconds: float
-    bank_stats_cache_max_entries: int

    # Retain settings
    retain_max_completion_tokens: int
@@ -1447,23 +1251,18 @@ class HindsightConfig:
    file_conversion_max_batch_size: int  # Max files per request
    enable_file_upload_api: bool
    file_delete_after_retain: bool
-    enable_document_export_api: bool
-    enable_document_import_api: bool

    # Observations settings (consolidated knowledge from facts)
    enable_observations: bool
    enable_auto_consolidation: bool
    enable_observation_history: bool
-    observation_history_max_entries: int
    enable_mental_model_history: bool
    mental_model_history_max_entries: int
    consolidation_batch_size: int
-    consolidation_dedup_threshold: float
    consolidation_max_memories_per_round: int
    consolidation_llm_batch_size: int
    consolidation_llm_parallelism: int
    consolidation_max_tokens: int
-    consolidation_max_completion_tokens: int | None
    consolidation_recall_budget: str
    consolidation_source_facts_max_tokens: int
    consolidation_source_facts_max_tokens_per_observation: int
@@ -1519,7 +1318,6 @@ class HindsightConfig:
    db_command_timeout: int
    db_acquire_timeout: int
    db_statement_timeout: int
-    model_init_timeout: float

    # Worker configuration (distributed task processing)
    worker_enabled: bool
@@ -1551,17 +1349,6 @@ class HindsightConfig:
    audit_log_actions: list[str]  # Allowlist of action types (empty = all)
    audit_log_retention_days: int  # -1 = keep forever, >0 = delete after N days

-    # LLM request tracing configuration (static - server-level only)
-    llm_trace_enabled: bool  # Master switch for per-bank LLM request tracing
-    llm_trace_scopes: list[str]  # Allowlist of call scopes to trace (empty = all)
-    llm_trace_retention_days: int  # -1 = keep forever, >0 = delete after N days
-    llm_trace_max_chars: int  # Truncate stored input/output beyond this many chars
-
-    # Background maintenance configuration (static - server-level only)
-    # Interval for the periodic sweep that re-schedules consolidation for banks with
-    # eligible-but-unscheduled facts. 0 = disabled.
-    consolidation_reconcile_interval_seconds: int
-
    # Webhook configuration (static - server-level only, not per-bank)
    webhook_url: str | None  # Global webhook URL (None = disabled)
    webhook_secret: str | None  # HMAC signing secret (None = unsigned)
@@ -1620,8 +1407,6 @@ class HindsightConfig:
        # File parser credentials
        "file_parser_iris_token",
        "file_parser_llama_parse_api_key",
-        # Admin surface token (never exposed via the admin config view itself)
-        "admin_api_token",
    }

    # CONFIGURABLE_FIELDS: Safe behavioral settings that can be customized per-tenant/bank
@@ -1764,11 +1549,6 @@ class HindsightConfig:
            self.text_search_extension_pg_search_tokenizer
        )

-        if not 0.0 <= self.semantic_min_similarity <= 1.0:
-            raise ValueError(
-                f"Invalid semantic_min_similarity: {self.semantic_min_similarity}. Must be between 0.0 and 1.0"
-            )
-
        # When LLM provider is "none", force chunks-only mode and disable LLM-dependent features
        if self.llm_provider == "none":
            self.retain_extraction_mode = "chunks"
@@ -1817,21 +1597,6 @@ class HindsightConfig:
                    " and ".join(missing),
                )

-        if self.embeddings_provider == "onnx":
-            try:
-                import importlib
-
-                importlib.import_module("onnxruntime")
-                importlib.import_module("transformers")
-            except ImportError:
-                logger.warning(
-                    "ONNX embeddings provider configured, but 'onnxruntime' and/or "
-                    "'transformers' is not installed. The API will fail at model init time. Either:\n"
-                    "  1. Install ONNX deps: pip install hindsight-api-slim[local-onnx]\n"
-                    "  2. Use a different embeddings provider, e.g. HINDSIGHT_API_EMBEDDINGS_PROVIDER=local "
-                    "or openai"
-                )
-
        # Validate that sum of per-operation slot reservations does not exceed max_slots
        total_reserved = sum(self.worker_slot_reservations.values())
        if total_reserved > self.worker_max_slots:
@@ -1884,7 +1649,6 @@ class HindsightConfig:
            llm_openai_service_tier=os.getenv(ENV_LLM_OPENAI_SERVICE_TIER, DEFAULT_LLM_OPENAI_SERVICE_TIER),
            llm_extra_body=json.loads(os.getenv(ENV_LLM_EXTRA_BODY, "null")),
            llm_default_headers=json.loads(os.getenv(ENV_LLM_DEFAULT_HEADERS, "null")),
-            llm_strict_schema=os.getenv(ENV_LLM_STRICT_SCHEMA, str(DEFAULT_LLM_STRICT_SCHEMA)).lower() in ("true", "1"),
            llm_litellmrouter_config=_parse_llm_router_config(ENV_LLM_LITELLMROUTER_CONFIG),
            # Vertex AI
            llm_vertexai_project_id=os.getenv(ENV_LLM_VERTEXAI_PROJECT_ID) or DEFAULT_LLM_VERTEXAI_PROJECT_ID,
@@ -1893,10 +1657,6 @@ class HindsightConfig:
            or DEFAULT_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY,
            # Gemini safety settings (JSON-encoded list of {category, threshold} dicts)
            llm_gemini_safety_settings=json.loads(os.getenv(ENV_LLM_GEMINI_SAFETY_SETTINGS, "null")),
-            llm_prompt_cache_enabled=os.getenv(
-                ENV_LLM_PROMPT_CACHE_ENABLED, str(DEFAULT_LLM_PROMPT_CACHE_ENABLED)
-            ).lower()
-            in ("1", "true", "yes", "on"),
            # Built-in llama.cpp configuration
            llamacpp_model_path=os.getenv(ENV_LLAMACPP_MODEL_PATH) or None,
            llamacpp_gpu_layers=int(os.getenv(ENV_LLAMACPP_GPU_LAYERS, str(DEFAULT_LLAMACPP_GPU_LAYERS))),
@@ -1995,36 +1755,6 @@ class HindsightConfig:
                ENV_EMBEDDINGS_LOCAL_TRUST_REMOTE_CODE, str(DEFAULT_EMBEDDINGS_LOCAL_TRUST_REMOTE_CODE)
            ).lower()
            in ("true", "1"),
-            embeddings_onnx_model_id=os.getenv(ENV_EMBEDDINGS_ONNX_MODEL_ID, DEFAULT_EMBEDDINGS_ONNX_MODEL_ID),
-            embeddings_onnx_model_path=os.getenv(ENV_EMBEDDINGS_ONNX_MODEL_PATH) or None,
-            embeddings_onnx_tokenizer_name_or_path=os.getenv(ENV_EMBEDDINGS_ONNX_TOKENIZER_NAME_OR_PATH) or None,
-            embeddings_onnx_file=os.getenv(ENV_EMBEDDINGS_ONNX_FILE, DEFAULT_EMBEDDINGS_ONNX_FILE),
-            embeddings_onnx_dimensions=_parse_optional_positive_int(
-                ENV_EMBEDDINGS_ONNX_DIMENSIONS,
-                os.getenv(ENV_EMBEDDINGS_ONNX_DIMENSIONS),
-            ),
-            embeddings_onnx_max_tokens=_parse_positive_int(
-                ENV_EMBEDDINGS_ONNX_MAX_TOKENS,
-                os.getenv(ENV_EMBEDDINGS_ONNX_MAX_TOKENS),
-                DEFAULT_EMBEDDINGS_ONNX_MAX_TOKENS,
-            ),
-            embeddings_onnx_pooling=_parse_optional_choice(
-                ENV_EMBEDDINGS_ONNX_POOLING,
-                os.getenv(ENV_EMBEDDINGS_ONNX_POOLING),
-                frozenset({"mean", "cls"}),
-            )
-            or DEFAULT_EMBEDDINGS_ONNX_POOLING,
-            embeddings_onnx_normalize=os.getenv(
-                ENV_EMBEDDINGS_ONNX_NORMALIZE, str(DEFAULT_EMBEDDINGS_ONNX_NORMALIZE)
-            ).lower()
-            in ("true", "1"),
-            embeddings_onnx_query_prefix=os.getenv(
-                ENV_EMBEDDINGS_ONNX_QUERY_PREFIX, DEFAULT_EMBEDDINGS_ONNX_QUERY_PREFIX
-            ),
-            embeddings_onnx_passage_prefix=os.getenv(
-                ENV_EMBEDDINGS_ONNX_PASSAGE_PREFIX, DEFAULT_EMBEDDINGS_ONNX_PASSAGE_PREFIX
-            ),
-            embeddings_onnx_output_name=os.getenv(ENV_EMBEDDINGS_ONNX_OUTPUT_NAME) or None,
            embeddings_tei_url=os.getenv(ENV_EMBEDDINGS_TEI_URL),
            embeddings_openai_base_url=os.getenv(ENV_EMBEDDINGS_OPENAI_BASE_URL) or None,
            embeddings_openai_batch_size=_parse_positive_int(
@@ -2146,14 +1876,6 @@ class HindsightConfig:
                os.getenv(ENV_RERANKER_TEI_HTTP_TIMEOUT, str(DEFAULT_RERANKER_TEI_HTTP_TIMEOUT))
            ),
            reranker_max_candidates=int(os.getenv(ENV_RERANKER_MAX_CANDIDATES, str(DEFAULT_RERANKER_MAX_CANDIDATES))),
-            semantic_min_similarity=float(os.getenv(ENV_SEMANTIC_MIN_SIMILARITY, str(DEFAULT_SEMANTIC_MIN_SIMILARITY))),
-            bm25_min_score=float(os.getenv(ENV_BM25_MIN_SCORE, str(DEFAULT_BM25_MIN_SCORE))),
-            recall_max_candidates_per_source=int(
-                os.getenv(ENV_RECALL_MAX_CANDIDATES_PER_SOURCE, str(DEFAULT_RECALL_MAX_CANDIDATES_PER_SOURCE))
-            ),
-            recall_strategy_boosts=_parse_strategy_boosts(
-                os.getenv(ENV_RECALL_STRATEGY_BOOSTS, DEFAULT_RECALL_STRATEGY_BOOSTS)
-            ),
            # Cohere reranker (with backward-compatible fallback to shared API key)
            reranker_cohere_api_key=os.getenv(ENV_RERANKER_COHERE_API_KEY) or os.getenv(ENV_COHERE_API_KEY),
            reranker_cohere_model=os.getenv(ENV_RERANKER_COHERE_MODEL, DEFAULT_RERANKER_COHERE_MODEL),
@@ -2228,8 +1950,6 @@ class HindsightConfig:
            mcp_stateless=os.getenv(ENV_MCP_STATELESS, str(DEFAULT_MCP_STATELESS)).lower() == "true",
            enable_bank_config_api=os.getenv(ENV_ENABLE_BANK_CONFIG_API, str(DEFAULT_ENABLE_BANK_CONFIG_API)).lower()
            == "true",
-            enable_admin_api=os.getenv(ENV_ENABLE_ADMIN_API, str(DEFAULT_ENABLE_ADMIN_API)).lower() == "true",
-            admin_api_token=os.getenv(ENV_ADMIN_API_TOKEN) or DEFAULT_ADMIN_API_TOKEN,
            default_bank_template=_parse_default_bank_template(os.getenv(ENV_DEFAULT_BANK_TEMPLATE)),
            # Recall
            graph_retriever=os.getenv(ENV_GRAPH_RETRIEVER, DEFAULT_GRAPH_RETRIEVER),
@@ -2245,12 +1965,6 @@ class HindsightConfig:
                os.getenv(ENV_LINK_EXPANSION_PER_ENTITY_LIMIT, str(DEFAULT_LINK_EXPANSION_PER_ENTITY_LIMIT))
            ),
            link_expansion_timeout=float(os.getenv(ENV_LINK_EXPANSION_TIMEOUT, str(DEFAULT_LINK_EXPANSION_TIMEOUT))),
-            bank_stats_cache_ttl_seconds=float(
-                os.getenv(ENV_BANK_STATS_CACHE_TTL_SECONDS, str(DEFAULT_BANK_STATS_CACHE_TTL_SECONDS))
-            ),
-            bank_stats_cache_max_entries=int(
-                os.getenv(ENV_BANK_STATS_CACHE_MAX_ENTRIES, str(DEFAULT_BANK_STATS_CACHE_MAX_ENTRIES))
-            ),
            # Optimization flags
            skip_llm_verification=os.getenv(ENV_SKIP_LLM_VERIFICATION, "false").lower() == "true",
            lazy_reranker=os.getenv(ENV_LAZY_RERANKER, "false").lower() == "true",
@@ -2314,14 +2028,6 @@ class HindsightConfig:
                ENV_FILE_DELETE_AFTER_RETAIN, str(DEFAULT_FILE_DELETE_AFTER_RETAIN)
            ).lower()
            == "true",
-            enable_document_export_api=os.getenv(
-                ENV_ENABLE_DOCUMENT_EXPORT_API, str(DEFAULT_ENABLE_DOCUMENT_EXPORT_API)
-            ).lower()
-            == "true",
-            enable_document_import_api=os.getenv(
-                ENV_ENABLE_DOCUMENT_IMPORT_API, str(DEFAULT_ENABLE_DOCUMENT_IMPORT_API)
-            ).lower()
-            == "true",
            # Observations settings (consolidated knowledge from facts)
            enable_observations=os.getenv(ENV_ENABLE_OBSERVATIONS, str(DEFAULT_ENABLE_OBSERVATIONS)).lower() == "true",
            enable_auto_consolidation=os.getenv(
@@ -2332,12 +2038,6 @@ class HindsightConfig:
                ENV_ENABLE_OBSERVATION_HISTORY, str(DEFAULT_ENABLE_OBSERVATION_HISTORY)
            ).lower()
            == "true",
-            observation_history_max_entries=int(
-                os.getenv(
-                    ENV_OBSERVATION_HISTORY_MAX_ENTRIES,
-                    str(DEFAULT_OBSERVATION_HISTORY_MAX_ENTRIES),
-                )
-            ),
            enable_mental_model_history=os.getenv(
                ENV_ENABLE_MENTAL_MODEL_HISTORY, str(DEFAULT_ENABLE_MENTAL_MODEL_HISTORY)
            ).lower()
@@ -2357,9 +2057,6 @@ class HindsightConfig:
                    str(DEFAULT_CONSOLIDATION_MAX_MEMORIES_PER_ROUND),
                )
            ),
-            consolidation_dedup_threshold=float(
-                os.getenv(ENV_CONSOLIDATION_DEDUP_THRESHOLD, str(DEFAULT_CONSOLIDATION_DEDUP_THRESHOLD))
-            ),
            consolidation_llm_batch_size=int(
                os.getenv(ENV_CONSOLIDATION_LLM_BATCH_SIZE, str(DEFAULT_CONSOLIDATION_LLM_BATCH_SIZE))
            ),
@@ -2375,11 +2072,6 @@ class HindsightConfig:
            consolidation_max_tokens=int(
                os.getenv(ENV_CONSOLIDATION_MAX_TOKENS, str(DEFAULT_CONSOLIDATION_MAX_TOKENS))
            ),
-            consolidation_max_completion_tokens=(
-                int(os.getenv(ENV_CONSOLIDATION_MAX_COMPLETION_TOKENS))
-                if os.getenv(ENV_CONSOLIDATION_MAX_COMPLETION_TOKENS)
-                else DEFAULT_CONSOLIDATION_MAX_COMPLETION_TOKENS
-            ),
            consolidation_recall_budget=os.getenv(ENV_CONSOLIDATION_RECALL_BUDGET, DEFAULT_CONSOLIDATION_RECALL_BUDGET),
            consolidation_source_facts_max_tokens=int(
                os.getenv(ENV_CONSOLIDATION_SOURCE_FACTS_MAX_TOKENS, str(DEFAULT_CONSOLIDATION_SOURCE_FACTS_MAX_TOKENS))
@@ -2407,7 +2099,6 @@ class HindsightConfig:
            db_command_timeout=int(os.getenv(ENV_DB_COMMAND_TIMEOUT, str(DEFAULT_DB_COMMAND_TIMEOUT))),
            db_acquire_timeout=int(os.getenv(ENV_DB_ACQUIRE_TIMEOUT, str(DEFAULT_DB_ACQUIRE_TIMEOUT))),
            db_statement_timeout=int(os.getenv(ENV_DB_STATEMENT_TIMEOUT, str(DEFAULT_DB_STATEMENT_TIMEOUT))),
-            model_init_timeout=float(os.getenv(ENV_MODEL_INIT_TIMEOUT, str(DEFAULT_MODEL_INIT_TIMEOUT))),
            # Worker configuration
            worker_enabled=os.getenv(ENV_WORKER_ENABLED, str(DEFAULT_WORKER_ENABLED)).lower() == "true",
            worker_id=os.getenv(ENV_WORKER_ID) or DEFAULT_WORKER_ID,
@@ -2492,22 +2183,6 @@ class HindsightConfig:
            audit_log_retention_days=int(
                os.getenv(ENV_AUDIT_LOG_RETENTION_DAYS, str(DEFAULT_AUDIT_LOG_RETENTION_DAYS))
            ),
-            # LLM request tracing configuration (static, server-level only)
-            llm_trace_enabled=os.getenv(ENV_LLM_TRACE_ENABLED, str(DEFAULT_LLM_TRACE_ENABLED)).lower() == "true",
-            llm_trace_scopes=[
-                s.strip() for s in os.getenv(ENV_LLM_TRACE_SCOPES, DEFAULT_LLM_TRACE_SCOPES).split(",") if s.strip()
-            ],
-            llm_trace_retention_days=int(
-                os.getenv(ENV_LLM_TRACE_RETENTION_DAYS, str(DEFAULT_LLM_TRACE_RETENTION_DAYS))
-            ),
-            llm_trace_max_chars=int(os.getenv(ENV_LLM_TRACE_MAX_CHARS, str(DEFAULT_LLM_TRACE_MAX_CHARS))),
-            # Background maintenance configuration (static, server-level only)
-            consolidation_reconcile_interval_seconds=int(
-                os.getenv(
-                    ENV_CONSOLIDATION_RECONCILE_INTERVAL_SECONDS,
-                    str(DEFAULT_CONSOLIDATION_RECONCILE_INTERVAL_SECONDS),
-                )
-            ),
            # Webhook configuration (static, server-level only)
            webhook_url=os.getenv(ENV_WEBHOOK_URL) or DEFAULT_WEBHOOK_URL,
            webhook_secret=os.getenv(ENV_WEBHOOK_SECRET) or DEFAULT_WEBHOOK_SECRET,
@@ -266,20 +266,12 @@ class ConfigResolver:
        # Validate recall budget fields
        _validate_recall_budget_updates(normalized_updates)

-        # Persist the override. Banks are created lazily (on first retain), so a
-        # PATCH that precedes any ingestion would otherwise UPDATE zero rows and
-        # silently no-op while returning 200. Ensure the bank row exists first
-        # (this also creates its per-bank vector indexes), then merge defensively:
-        # COALESCE guards against a NULL config column (NULL || jsonb is NULL),
-        # which would drop the override even when a row is updated.
-        from .engine.retain.fact_storage import ensure_bank_exists
-
+        # Merge with existing config (JSONB || operator)
        async with self._backend.acquire() as conn:
-            await ensure_bank_exists(conn, bank_id, ops=self._backend.ops)
            await conn.execute(
                f"""
                UPDATE {fq_table("banks")}
-                SET config = COALESCE(config, '{{}}'::jsonb) || $1::jsonb,
+                SET config = config || $1::jsonb,
                    updated_at = now()
                WHERE bank_id = $2
                """,
@@ -16,59 +16,11 @@ from dataclasses import dataclass, field
 from datetime import datetime, timezone
 from typing import Any

-from pydantic import BaseModel, Field
-
 from ..engine.db_utils import acquire_with_retry

 logger = logging.getLogger(__name__)


-class AuditLogEntry(BaseModel):
-    """A single audit log entry."""
-
-    id: str
-    action: str
-    transport: str
-    bank_id: str | None
-    started_at: str | None
-    ended_at: str | None
-    duration_ms: int | None = Field(
-        default=None,
-        description="Server-computed duration in milliseconds (started_at → ended_at). Null if not yet completed.",
-    )
-    request: dict[str, Any] | None
-    response: dict[str, Any] | None
-    metadata: dict[str, Any]
-
-
-class AuditLogListResponse(BaseModel):
-    """Response model for list audit logs endpoint."""
-
-    bank_id: str
-    total: int
-    limit: int
-    offset: int
-    items: list[AuditLogEntry]
-
-
-class AuditLogStatsBucket(BaseModel):
-    """A single time bucket in audit log stats."""
-
-    time: str
-    actions: dict[str, int]
-    total: int
-
-
-class AuditLogStatsResponse(BaseModel):
-    """Response model for audit log stats endpoint."""
-
-    bank_id: str
-    period: str
-    trunc: str
-    start: str
-    buckets: list[AuditLogStatsBucket]
-
-
@dataclass
 class AuditEntry:
    """A single audit log entry."""
@@ -107,11 +59,11 @@ def _safe_json(data: Any) -> str | None:
        return None


-class AuditLogger:
-    """Fire-and-forget audit log writer.
+_SWEEP_INTERVAL_SECONDS = 3600  # Run retention sweep every hour

-    Retention of old rows is handled by the background :class:`MaintenanceLoop`.
-    """
+
+class AuditLogger:
+    """Fire-and-forget audit log writer with optional retention sweep."""

    def __init__(
        self,
@@ -119,11 +71,14 @@ class AuditLogger:
        schema_getter: Callable[[], str],
        enabled: bool,
        allowed_actions: list[str],
+        retention_days: int = -1,
    ) -> None:
        self._pool_getter = pool_getter
        self._schema_getter = schema_getter
        self._enabled = enabled
        self._allowed_actions: frozenset[str] | None = frozenset(allowed_actions) if allowed_actions else None
+        self._retention_days = retention_days
+        self._sweep_task: asyncio.Task | None = None

    def is_enabled(self, action: str) -> bool:
        """Check if audit logging is enabled for this action."""
@@ -173,6 +128,48 @@ class AuditLogger:
        except Exception as e:
            logger.warning(f"Audit log write failed for action={entry.action}: {e}")

+    def start_retention_sweep(self) -> None:
+        """Start the periodic retention sweep if retention is configured."""
+        if self._retention_days <= 0 or not self._enabled:
+            return
+        try:
+            self._sweep_task = asyncio.create_task(self._sweep_loop())
+        except RuntimeError:
+            logger.debug("Cannot start retention sweep: no running event loop")
+
+    async def stop_retention_sweep(self) -> None:
+        """Stop the periodic retention sweep."""
+        if self._sweep_task and not self._sweep_task.done():
+            self._sweep_task.cancel()
+            try:
+                await self._sweep_task
+            except asyncio.CancelledError:
+                pass
+            self._sweep_task = None
+
+    async def _sweep_loop(self) -> None:
+        """Periodically delete audit log entries older than retention_days."""
+        while True:
+            await self._run_sweep()
+            await asyncio.sleep(_SWEEP_INTERVAL_SECONDS)
+
+    async def _run_sweep(self) -> None:
+        """Delete expired audit log entries. Concurrent-safe via row-level deletes."""
+        pool = self._pool_getter()
+        if pool is None:
+            return
+        try:
+            schema = self._schema_getter()
+            table = f"{schema}.audit_log"
+            async with acquire_with_retry(pool, max_retries=1) as conn:
+                result = await conn.execute(
+                    f"DELETE FROM {table} WHERE started_at < NOW() - INTERVAL '{self._retention_days} days'"
+                )
+                if result and result != "DELETE 0":
+                    logger.info(f"Audit log retention sweep: {result}")
+        except Exception as e:
+            logger.warning(f"Audit log retention sweep failed: {e}")
+

@asynccontextmanager
 async def audit_context(
@@ -1,122 +0,0 @@
-"""TTL + coalescing cache for `get_bank_stats`.
-
-`get_bank_stats` aggregates over `memory_links` (and joins to `memory_units`),
-which can be a multi-second parallel sequential scan on banks with millions of
-rows. The result is intentionally approximate (it powers a UI widget and a
-freshness hint inside `reflect`), so caching it for a few tens of seconds is
-safe and dramatically reduces planner-driven thrash from clients that poll.
-
-The cache also coalesces concurrent misses on the same key onto a single
-in-flight task so that N concurrent callers produce one query rather than N.
-"""
-
-from __future__ import annotations
-
-import asyncio
-import time
-from collections import OrderedDict
-from typing import Any, Awaitable, Callable
-
-
-class BankStatsCache:
-    """Per-process TTL cache keyed on (schema, bank_id).
-
-    `ttl_seconds <= 0` disables caching: each call passes straight through to
-    the loader. `max_entries` bounds memory in environments with many banks.
-    """
-
-    def __init__(self, *, ttl_seconds: float, max_entries: int) -> None:
-        self._ttl = float(ttl_seconds)
-        self._max_entries = int(max_entries) if max_entries and max_entries > 0 else 0
-        self._entries: OrderedDict[tuple[str, str], tuple[float, dict[str, Any]]] = OrderedDict()
-        self._in_flight: dict[tuple[str, str], asyncio.Future[dict[str, Any]]] = {}
-        self._lock = asyncio.Lock()
-
-    @property
-    def enabled(self) -> bool:
-        return self._ttl > 0
-
-    def _now(self) -> float:
-        return time.monotonic()
-
-    def _get_fresh_unlocked(self, key: tuple[str, str]) -> dict[str, Any] | None:
-        entry = self._entries.get(key)
-        if entry is None:
-            return None
-        expires_at, value = entry
-        if expires_at <= self._now():
-            # Expired — drop so the loader runs again.
-            self._entries.pop(key, None)
-            return None
-        # Mark as recently used for LRU eviction.
-        self._entries.move_to_end(key)
-        return value
-
-    def _store_unlocked(self, key: tuple[str, str], value: dict[str, Any]) -> None:
-        if not self.enabled:
-            return
-        self._entries[key] = (self._now() + self._ttl, value)
-        self._entries.move_to_end(key)
-        if self._max_entries:
-            while len(self._entries) > self._max_entries:
-                self._entries.popitem(last=False)
-
-    async def get_or_load(
-        self,
-        schema: str,
-        bank_id: str,
-        loader: Callable[[], Awaitable[dict[str, Any]]],
-    ) -> dict[str, Any]:
-        """Return cached stats for `(schema, bank_id)` or call `loader()`.
-
-        Concurrent misses on the same key are coalesced onto a single
-        in-flight loader.
-        """
-        if not self.enabled:
-            return await loader()
-
-        key = (schema, bank_id)
-
-        async with self._lock:
-            cached = self._get_fresh_unlocked(key)
-            if cached is not None:
-                return cached
-            in_flight = self._in_flight.get(key)
-            if in_flight is None:
-                in_flight = asyncio.get_running_loop().create_future()
-                self._in_flight[key] = in_flight
-                is_owner = True
-            else:
-                is_owner = False
-
-        if not is_owner:
-            return await asyncio.shield(in_flight)
-
-        try:
-            value = await loader()
-        except BaseException as exc:
-            async with self._lock:
-                self._in_flight.pop(key, None)
-            if not in_flight.done():
-                in_flight.set_exception(exc)
-            # Suppress "Future exception was never retrieved" when no other
-            # caller was waiting on this loader — we re-raise to the owner
-            # immediately and the future is a no-op in that case.
-            in_flight.exception()
-            raise
-
-        async with self._lock:
-            self._store_unlocked(key, value)
-            self._in_flight.pop(key, None)
-        if not in_flight.done():
-            in_flight.set_result(value)
-        return value
-
-    async def invalidate(self, schema: str, bank_id: str) -> None:
-        """Drop any cached stats for `(schema, bank_id)`."""
-        async with self._lock:
-            self._entries.pop((schema, bank_id), None)
-
-    async def clear(self) -> None:
-        async with self._lock:
-            self._entries.clear()
@@ -22,30 +22,19 @@ import time
 import uuid
 from collections import defaultdict
 from contextlib import AsyncExitStack
-from dataclasses import asdict, dataclass, field
+from dataclasses import dataclass, field
 from datetime import datetime, timezone
 from itertools import combinations
-from typing import TYPE_CHECKING, Any, Literal
+from typing import TYPE_CHECKING, Any

 from pydantic import BaseModel, field_validator

 from ...config import get_config
-from ...worker.stage import set_stage
 from ..db_utils import acquire_with_retry
-from ..llm_trace import (
-    record_created_memory_ids,
-    record_source_memory_ids,
-    reset_trace_context,
-    set_trace_context,
-    trace_context_of,
-)
 from ..llm_wrapper import sanitize_llm_output
 from ..memory_engine import Budget, fq_table
 from ..retain import embedding_utils
-from .prompts import (
-    build_consolidation_input,
-    build_consolidation_system_prompt,
-)
+from .prompts import build_batch_consolidation_prompt

 if TYPE_CHECKING:
    from asyncpg import Connection
@@ -57,254 +46,6 @@ if TYPE_CHECKING:
 logger = logging.getLogger(__name__)


-def _norm_obs_text(text: str) -> str:
-    """Whitespace-normalised observation text for exact-duplicate matching.
-
-    Collapses runs of whitespace only; case is preserved. The reconciliation guard
-    drops a CREATE on the premise that an exact-text match loses no information — but
-    case-folding would also drop a create differing only in case (e.g. "TLS" vs "tls"),
-    which *does* lose information, so we match case-sensitively.
-    """
-    return " ".join((text or "").split()).strip()
-
-
-def _duplicate_create_target(
-    create_text: str,
-    shown_obs_by_text: "dict[str, MemoryFact]",
-    update_texts: set[str],
-) -> str | None:
-    """Return a human label for what ``create_text`` duplicates, or None if novel.
-
-    A CREATE is a duplicate when its normalised text matches an observation that was
-    already shown to the LLM, or the text of an UPDATE issued in the same response
-    (the model occasionally UPDATEs the twin to text X and also CREATEs X). Exact-text
-    match means no information is lost by dropping the CREATE.
-    """
-    norm = _norm_obs_text(create_text)
-    matched = shown_obs_by_text.get(norm)
-    if matched is not None:
-        return f"shown observation {str(matched.id)[:8]}"
-    if norm in update_texts:
-        return "an UPDATE in this response"
-    return None
-
-
-# Top-K existing observations probed (by the new observation's own embedding) when
-# semantic dedup is enabled. Small: we only need the nearest few candidates.
-_DEDUP_TOP_K = 5
-
-
-class _DedupDecision(BaseModel):
-    """Focused 1-by-1 verdict for whether a new observation duplicates an existing one."""
-
-    action: Literal["merge", "keep"]
-    text: str = ""  # the synthesized merged observation (when action == "merge")
-    reason: str = ""
-
-
-_DEDUP_PROMPT = """You reconcile long-term memory observations. A NEW observation is about to be \
-stored, and it is highly similar to an EXISTING one:
-
-[NEW] {new}
-[EXISTING] {existing}
-
-If they assert the SAME fact (wording aside), respond action="merge" and provide `text`: a single \
-observation that preserves EVERY detail from both. If they differ in ANY important detail — a \
-number/quantity, a named entity or language, a negation, or a condition — respond action="keep"."""
-
-
-def _dedup_active(config: Any) -> bool:
-    """Whether create/update semantic dedup runs for this consolidation.
-
-    Enabled when the resolved threshold is < 1.0, EXCEPT on Oracle: the merge path uses
-    Postgres-only SQL (``unnest``/``array_agg``, ``UPDATE ... FROM``), so on Oracle dedup is
-    skipped — it behaves exactly as it did before this feature, regardless of the configured
-    threshold. This is why the feature can ship enabled-by-default without breaking Oracle.
-    """
-    if config is None or getattr(config, "consolidation_dedup_threshold", 1.0) >= 1.0:
-        return False
-    return get_config().database_backend != "oracle"
-
-
-@dataclass
-class _DedupOutcome:
-    """Result of probing one observation against its in-scope neighbours.
-
-    ``best_id`` is the nearest observation at/above the threshold (None if none),
-    ``merged_text`` is the LLM-synthesized union text (set only when ``should_merge``).
-    """
-
-    best_id: str | None
-    merged_text: str
-    should_merge: bool
-
-
-async def _dedup_adjudicate(
-    conn: "Connection",
-    memory_engine: "MemoryEngine",
-    bank_id: str,
-    config: Any,
-    dedup_llm_config: Any,
-    anchor_text: str,
-    anchor_emb_str: str | None,
-    tags: list[str] | None,
-    exclude_id: str | None,
-) -> _DedupOutcome:
-    """Probe one observation's embedding against in-scope observations and adjudicate a merge.
-
-    Anchored on the observation text — the correct obs<->obs comparison, unlike consolidation
-    recall which is anchored on the raw fact. Returns the nearest observation at/above
-    ``consolidation_dedup_threshold`` and, when found, the LLM's focused 1-by-1 merge-or-keep
-    verdict (scope ``consolidation_dedup``): the LLM reads both texts, so a word-level difference
-    (number / negation / entity) is respected. ``exclude_id`` skips the anchor observation itself
-    (used by the UPDATE path, where the anchor row already exists and would self-match at 1.0).
-    ``anchor_emb_str`` reuses an already-computed embedding (the UPDATE path just embedded it);
-    pass None to embed ``anchor_text`` here (the CREATE path).
-    """
-    from ..search.retrieval import retrieve_semantic_bm25_combined
-
-    threshold = config.consolidation_dedup_threshold
-    if anchor_emb_str is None:
-        embs = await embedding_utils.generate_embeddings_batch(memory_engine.embeddings, [anchor_text])
-        if not embs:
-            return _DedupOutcome(best_id=None, merged_text="", should_merge=False)
-        anchor_emb_str = str(embs[0])
-    tags_match = "all_strict" if tags else "any"
-    grouped = await retrieve_semantic_bm25_combined(
-        conn, anchor_emb_str, anchor_text, bank_id, ["observation"], _DEDUP_TOP_K, tags=tags, tags_match=tags_match
-    )
-    results = grouped.get("observation", ([], []))[0]
-    best_id: str | None = None
-    best_text = ""
-    best_sim = threshold  # only candidates at/above the threshold are considered
-    for r in results:
-        rid = str(r.id)
-        if exclude_id is not None and rid == exclude_id:
-            continue  # never match the anchor observation against itself
-        sim = r.similarity or 0.0
-        if sim >= best_sim:
-            best_id, best_text, best_sim = rid, r.text, sim
-
-    if best_id is None:
-        return _DedupOutcome(best_id=None, merged_text="", should_merge=False)
-
-    decision: _DedupDecision = await dedup_llm_config.call(
-        messages=[{"role": "user", "content": _DEDUP_PROMPT.format(new=anchor_text, existing=best_text)}],
-        response_format=_DedupDecision,
-        scope="consolidation_dedup",
-    )
-    if decision.action != "merge":
-        return _DedupOutcome(best_id=best_id, merged_text="", should_merge=False)
-    return _DedupOutcome(best_id=best_id, merged_text=decision.text.strip() or best_text, should_merge=True)
-
-
-async def _dedup_reconcile_create(
-    conn: "Connection",
-    memory_engine: "MemoryEngine",
-    bank_id: str,
-    config: Any,
-    dedup_llm_config: Any,
-    create_text: str,
-    create_source_ids: list[uuid.UUID],
-    tags: list[str] | None,
-) -> str | None:
-    """Semantic dedup for a single CREATE (create-time, focused 1-by-1).
-
-    On "merge", folds the new source facts + the synthesized text into the existing
-    observation and returns its id (caller skips the CREATE). Returns None when there is
-    no near twin or the LLM keeps them distinct.
-    """
-    outcome = await _dedup_adjudicate(
-        conn, memory_engine, bank_id, config, dedup_llm_config, create_text, None, tags, exclude_id=None
-    )
-    if not outcome.should_merge or outcome.best_id is None:
-        return None
-
-    # Fold the new source facts into the twin and persist the merged text. We keep the twin's
-    # existing embedding: the merged text is >= threshold similar, so the stored vector stays
-    # representative and we avoid a re-embed + a dialect-specific vector UPDATE.
-    await conn.execute(
-        f"""
-        UPDATE {fq_table("memory_units")}
-        SET text = $1,
-            source_memory_ids = (SELECT array_agg(DISTINCT e) FROM unnest(source_memory_ids || $2::uuid[]) e),
-            proof_count = (SELECT count(DISTINCT e) FROM unnest(source_memory_ids || $2::uuid[]) e),
-            updated_at = now()
-        WHERE id = $3::uuid
-        """,
-        outcome.merged_text,
-        create_source_ids,
-        uuid.UUID(outcome.best_id),
-    )
-    return outcome.best_id
-
-
-async def _dedup_reconcile_update(
-    conn: "Connection",
-    memory_engine: "MemoryEngine",
-    bank_id: str,
-    config: Any,
-    dedup_llm_config: Any,
-    updated_id: str,
-    updated_text: str,
-    updated_emb_str: str | None,
-    tags: list[str] | None,
-) -> None:
-    """Semantic dedup for an UPDATE (after the observation was rewritten + re-embedded).
-
-    An UPDATE rewrites an observation's text and re-embeds it, so its vector can drift to
-    within threshold of a DIFFERENT existing observation. The create-time guard never sees
-    this (it only runs on CREATE), so without this the two persist as a near-duplicate pair —
-    the measured residual-duplicate source. Probe the updated observation's new embedding
-    against the others (excluding itself); on "merge", fold the just-updated observation's
-    sources into the twin, persist the merged text, and DELETE the updated row. Unlike the
-    CREATE path the row already exists, so reconciliation is a fold-and-delete, not a skip.
-    """
-    outcome = await _dedup_adjudicate(
-        conn,
-        memory_engine,
-        bank_id,
-        config,
-        dedup_llm_config,
-        updated_text,
-        updated_emb_str,
-        tags,
-        exclude_id=updated_id,
-    )
-    if not outcome.should_merge or outcome.best_id is None:
-        return
-
-    # Fold the updated observation's sources into the twin (keeping the twin's embedding, as in
-    # the create path) then delete the now-redundant updated row. The all_strict/any tag match
-    # guarantees twin and updated share scope, so dropping the updated row's tags loses no
-    # visibility. Temporal fields follow the surviving twin (minimal scope; matches create).
-    await conn.execute(
-        f"""
-        UPDATE {fq_table("memory_units")} t
-        SET text = $1,
-            source_memory_ids = (
-                SELECT array_agg(DISTINCT e) FROM unnest(t.source_memory_ids || u.source_memory_ids) e
-            ),
-            proof_count = (
-                SELECT count(DISTINCT e) FROM unnest(t.source_memory_ids || u.source_memory_ids) e
-            ),
-            updated_at = now()
-        FROM {fq_table("memory_units")} u
-        WHERE t.id = $2::uuid AND u.id = $3::uuid
-        """,
-        outcome.merged_text,
-        uuid.UUID(outcome.best_id),
-        uuid.UUID(updated_id),
-    )
-    await _execute_delete_action(conn, bank_id, updated_id)
-    logger.info(
-        "[CONSOLIDATION] dedup-merged updated observation %s into %s (cosine>=%.2f)",
-        updated_id[:8],
-        outcome.best_id[:8],
-        config.consolidation_dedup_threshold,
-    )
-
-
@dataclass
 class _BatchDeltas:
    """Per-LLM-batch deltas, merged into the job's running stats after dispatch.
@@ -426,9 +167,6 @@ async def _filter_live_source_memories(
 class _CreateAction(BaseModel):
    text: str
    source_fact_ids: list[str]  # memory UUIDs from the NEW FACTS list
-    # One-sentence justification from the LLM (why CREATE vs UPDATE). Diagnostic
-    # only — surfaced in the consolidation trace to explain duplicate creates.
-    reason: str = ""

    @field_validator("text", mode="before")
    @classmethod
@@ -440,7 +178,6 @@ class _UpdateAction(BaseModel):
    text: str
    observation_id: str  # UUID of the existing observation to update
    source_fact_ids: list[str]  # memory UUIDs from the NEW FACTS list
-    reason: str = ""  # LLM's one-sentence justification (diagnostic only)

    @field_validator("text", mode="before")
    @classmethod
@@ -450,7 +187,6 @@ class _UpdateAction(BaseModel):

 class _DeleteAction(BaseModel):
    observation_id: str  # UUID of the observation to remove
-    reason: str = ""  # LLM's one-sentence justification (diagnostic only)


 class _ConsolidationBatchResponse(BaseModel):
@@ -625,36 +361,8 @@ async def run_consolidation_job(

    # Build a configured LLM wrapper that applies per-bank settings (e.g. safety settings)
    # to every call without leaking across operations.
-    llm_config = memory_engine._consolidation_llm_config.with_config(config, bank_id=bank_id, operation="consolidation")
+    llm_config = memory_engine._consolidation_llm_config.with_config(config)

-    # Bind the operation trace context for the whole run so the create/update DB
-    # sites (deep inside _process_memory_batch) can accumulate the observations
-    # this consolidation produced and the source memories it consumed onto the
-    # trace — flushed onto every trace row on exit by attach_memory_ids.
-    trace_ctx = trace_context_of(llm_config)
-    trace_token = set_trace_context(trace_ctx) if trace_ctx is not None else None
-    try:
-        return await _run_consolidation_job(
-            memory_engine, bank_id, request_context, config, llm_config, operation_id, observation_scopes
-        )
-    finally:
-        if trace_token is not None:
-            reset_trace_context(trace_token)
-            # Fire-and-forget: patched on a background task, off the consolidation
-            # critical path.
-            memory_engine._llm_recorder.attach_memory_ids(trace_ctx)
-
-
-async def _run_consolidation_job(
-    memory_engine: "MemoryEngine",
-    bank_id: str,
-    request_context: "RequestContext",
-    config: Any,
-    llm_config: Any,
-    operation_id: str | None = None,
-    observation_scopes: list[list[str]] | None = None,
-) -> dict[str, Any]:
-    """Core consolidation flow. See ``run_consolidation_job`` for the public entrypoint."""
    perf = ConsolidationPerfLog(bank_id)
    max_memories_per_batch = config.consolidation_batch_size
    max_memories_per_round = config.consolidation_max_memories_per_round
@@ -718,44 +426,6 @@ async def _run_consolidation_job(
    logger.info(f"[CONSOLIDATION] bank={bank_id} total_unconsolidated={total_count}")
    perf.log(f"[1] Found {total_count} pending memories to consolidate")

-    # Initial durable progress snapshot so an operator polling the operation status
-    # API sees the job has started and how much work it found, before the first batch
-    # of LLM work completes (which can take minutes on a dense bank). Uses the same
-    # "consolidating" stage as the per-batch heartbeat so the operator sees a single
-    # phase advancing 0/N -> N/N rather than an opaque "scanning" -> "processing" hop.
-    set_stage("consolidation.consolidating")
-    await memory_engine._write_operation_progress(operation_id, stage="consolidating", processed=0, total=total_count)
-
-    async def _count_unconsolidated() -> int:
-        """Re-count memories still pending consolidation in this job's scope.
-
-        ``total_count`` is a point-in-time estimate from job start; memories retained
-        while consolidation runs get picked up by later fetches, so processed can pass
-        it. When that happens we re-count to report a real total (processed + remaining)
-        instead of pinning the bar at 100%."""
-        async with acquire_with_retry(pool) as count_conn:
-            pending = await count_conn.fetchval(
-                f"""
-                SELECT COUNT(*)
-                FROM {fq_table("memory_units")}
-                WHERE bank_id = $1
-                  AND consolidated_at IS NULL
-                  AND consolidation_failed_at IS NULL
-                  AND fact_type IN ('experience', 'world')
-                  {scope_clause}
-                """,
-                *scope_params,
-            )
-        return pending or 0
-
-    async def _progress_total(processed: int) -> int:
-        # Cheap path: while we're still within the start-of-job estimate it's exact, so
-        # no extra query. Only re-count once the estimate is exhausted (≈the final batch
-        # normally, or repeatedly only if memories keep arriving mid-run).
-        if processed < total_count:
-            return total_count
-        return processed + await _count_unconsolidated()
-
    # Process each memory with individual commits for crash recovery
    stats: dict[str, int] = {
        "memories_processed": 0,
@@ -776,18 +446,10 @@ async def _run_consolidation_job(
    hit_round_limit = False

    llm_batch_num = 0
-    # Cumulative counters across the whole job, shared by the per-batch log and the
-    # durable progress snapshot so both report processed/total (and observation
-    # tallies) under parallelism. Mutable container so the inner closure can update
-    # without a `nonlocal`.
-    cumulative_progress = {
-        "processed": 0,
-        "observations_created": 0,
-        "observations_updated": 0,
-        "observations_merged": 0,
-        "observations_deleted": 0,
-        "memories_failed": 0,
-    }
+    # Cumulative count of memories processed across the whole job, shared by
+    # the per-batch log so it can still report processed/total under parallelism.
+    # Mutable container so the inner closure can update without a `nonlocal`.
+    cumulative_progress = {"processed": 0}
    while True:
        # Cap fetch size by remaining round budget
        fetch_limit = (
@@ -1008,18 +670,12 @@ async def _run_consolidation_job(
                    local_stats["memories_failed"] += 1

            # Maintain the cumulative-progress indicator under parallelism:
-            # increment shared counters and snapshot under the same statements so
-            # the snapshot includes this batch. No await between the reads and
-            # writes, so single-threaded asyncio gives us atomicity for free —
-            # no lock needed.
+            # increment a shared counter and snapshot under the same statement
+            # so the snapshot includes this batch. No await between the read
+            # and write, so single-threaded asyncio gives us atomicity for free
+            # — no lock needed.
            cumulative_progress["processed"] += local_stats["memories_processed"]
-            cumulative_progress["observations_created"] += local_stats["observations_created"]
-            cumulative_progress["observations_updated"] += local_stats["observations_updated"]
-            cumulative_progress["observations_merged"] += local_stats["observations_merged"]
-            cumulative_progress["observations_deleted"] += local_stats["observations_deleted"]
-            cumulative_progress["memories_failed"] += local_stats["memories_failed"]
            cum_processed = cumulative_progress["processed"]
-            cum_snapshot = dict(cumulative_progress)

            # Per-batch log uses batch_perf so timings/llm-calls/tokens reflect
            # only this batch's own work, even when other batches are running
@@ -1047,27 +703,6 @@ async def _run_consolidation_job(
                f" | avg={llm_batch_time / max(1, len(llm_batch_local)):.3f}s/memory"
            )

-            # Durable progress snapshot per LLM batch — this is the heartbeat an
-            # operator polls. The whole fetched batch is processed inside one outer
-            # round, so a round-boundary write would sit at the pre-round count for
-            # the entire (often minutes-long) LLM phase; writing here advances
-            # processed/total as each batch commits. set_stage mirrors it for the
-            # live worker log.
-            set_stage(f"consolidation.llm_batch.{batch_num_local}")
-            await memory_engine._write_operation_progress(
-                operation_id,
-                stage="consolidating",
-                processed=cum_processed,
-                total=await _progress_total(cum_processed),
-                detail={
-                    "observations_created": cum_snapshot["observations_created"],
-                    "observations_updated": cum_snapshot["observations_updated"],
-                    "observations_merged": cum_snapshot["observations_merged"],
-                    "observations_deleted": cum_snapshot["observations_deleted"],
-                    "memories_failed": cum_snapshot["memories_failed"],
-                },
-            )
-
            # Fold batch counters into the job-level perf so the final summary
            # (perf.flush) totals every batch correctly. Safe without a lock —
            # ConsolidationPerfLog.merge_from is a series of += on Python ints
@@ -1206,13 +841,6 @@ async def _run_consolidation_job(
        stats["mental_models_refreshed"] = 0
        logger.info(f"[CONSOLIDATION] bank={bank_id} skipping mental model refresh (round limit hit, re-queued)")
    else:
-        set_stage("consolidation.refreshing_mental_models")
-        await memory_engine._write_operation_progress(
-            operation_id,
-            stage="refreshing_mental_models",
-            processed=stats["memories_processed"],
-            total=await _progress_total(stats["memories_processed"]),
-        )
        # SECURITY: Only refresh mental models with matching tags (or all if no tags were consolidated)
        mental_models_refreshed = await _trigger_mental_model_refreshes(
            memory_engine=memory_engine,
@@ -1356,9 +984,6 @@ async def _process_memory_batch(
    """
    import asyncio

-    # Map the source memories this batch consumes onto the consolidation trace.
-    record_source_memory_ids([str(m["id"]) for m in memories])
-
    # 1. Parallel recalls — one per fact
    # When obs_tags_override is set, use it as the observation scope for all facts.
    t0 = time.time()
@@ -1438,20 +1063,6 @@ async def _process_memory_batch(

    mem_by_id = {str(m["id"]): m for m in memories}

-    # Semantic dedup: when enabled, an observation that is >= the threshold cosine to a DIFFERENT
-    # existing observation is reconciled by a focused 1-by-1 LLM merge (anchored on the observation
-    # text, not the source fact). It runs on both CREATE (a near-dup emitted despite the twin being
-    # in context — weak-model failure mode) and UPDATE (a rewrite+re-embed that drifts an existing
-    # observation into a twin — the create-time guard can't see this). The trace operation/scope is
-    # "consolidation_dedup" (routes through the consolidation concurrency bucket via llm_wrapper's
-    # "consolidation" prefix; recorded distinctly in llm_requests).
-    dedup_enabled = _dedup_active(config)
-    dedup_llm_config = (
-        memory_engine._consolidation_llm_config.with_config(config, bank_id=bank_id, operation="consolidation_dedup")
-        if dedup_enabled
-        else None
-    )
-
    # Execute deletes first to free observation slots before creates consume them
    deleted_count = 0
    for delete in llm_result.deletes:
@@ -1476,7 +1087,7 @@ async def _process_memory_batch(
            )
            continue
        agg = _aggregate_source_fields(source_mems, tags=fact_tags)
-        updated_emb_str = await _execute_update_action(
+        await _execute_update_action(
            conn=conn,
            memory_engine=memory_engine,
            bank_id=bank_id,
@@ -1492,76 +1103,17 @@ async def _process_memory_batch(
        )
        for m in source_mems:
            per_memory_updated.add(str(m["id"]))
-        # Reconcile the rewritten observation against its neighbours: the re-embed may have
-        # drifted it into a near-twin of another existing observation (the residual-duplicate
-        # source). updated_emb_str is None when the update was skipped — nothing to reconcile.
-        if dedup_enabled and updated_emb_str is not None:
-            await _dedup_reconcile_update(
-                conn,
-                memory_engine,
-                bank_id,
-                config,
-                dedup_llm_config,
-                update.observation_id,
-                update.text,
-                updated_emb_str,
-                agg.tags,
-            )
-
-    # Deterministic dedup guard: map the observations the LLM was SHOWN by their
-    # normalised text. The model intermittently emits a CREATE whose text is identical
-    # to an observation already in its context (over-aggregation / incoherence — it even
-    # UPDATEs the twin and creates a sibling). When that happens we drop the duplicate
-    # CREATE instead of inserting a redundant row. No extra LLM/embedding cost — the
-    # match is exact text against the in-memory set.
-    shown_obs_by_text = {_norm_obs_text(o.text): o for o in union_observations}
-    # Also collapse a CREATE that reproduces the text of an UPDATE issued in the SAME
-    # response (the model occasionally UPDATEs the twin to text X and also CREATEs X).
-    update_texts = {_norm_obs_text(u.text) for u in llm_result.updates if u.text}

    for create in llm_result.creates:
        source_mems = [mem_by_id[fid] for fid in create.source_fact_ids if fid in mem_by_id]
        if not source_mems:
            continue
        agg = _aggregate_source_fields(source_mems, tags=fact_tags)
-        create_source_ids = [m["id"] for m in source_mems]
-
-        # Reconcile against observations shown to the LLM: an exact-text match means
-        # this CREATE reproduces verbatim an observation the model already had in context.
-        # Since that observation already carries this exact text, drop the duplicate CREATE
-        # — no row is inserted, nothing is lost. We deliberately do NOT also UPDATE the twin
-        # here: the LLM frequently UPDATEd it earlier in this same batch, and a second update
-        # would run off the pre-LLM snapshot and clobber that change (see _dedupe_updates).
-        duplicate_of = _duplicate_create_target(create.text, shown_obs_by_text, update_texts)
-        if duplicate_of is not None:
-            logger.warning(
-                "[CONSOLIDATION] dropped duplicate observation CREATE — verbatim match of %s; llm_reason=%r",
-                duplicate_of,
-                create.reason or "(none given)",
-            )
-            continue
-
-        # Semantic near-duplicate reconciliation: merge this CREATE into an existing
-        # near-identical observation (LLM-adjudicated, 1-by-1) instead of inserting a dup.
-        if dedup_enabled:
-            merged_into = await _dedup_reconcile_create(
-                conn, memory_engine, bank_id, config, dedup_llm_config, create.text, create_source_ids, agg.tags
-            )
-            if merged_into is not None:
-                logger.info(
-                    "[CONSOLIDATION] dedup-merged observation CREATE into %s (cosine>=%.2f)",
-                    merged_into[:8],
-                    config.consolidation_dedup_threshold,
-                )
-                for m in source_mems:
-                    per_memory_created.add(str(m["id"]))
-                continue
-
        await _execute_create_action(
            conn=conn,
            memory_engine=memory_engine,
            bank_id=bank_id,
-            source_memory_ids=create_source_ids,
+            source_memory_ids=[m["id"] for m in source_mems],
            text=create.text,
            source_fact_tags=agg.tags,
            event_date=agg.event_date,
@@ -1601,64 +1153,6 @@ def _max_date(dates: "Any") -> "datetime | None":
    return max((d for d in dates if d is not None), default=None)


-@dataclass(frozen=True)
-class _ObservationHistorySnapshot:
-    """Pre-update state of an observation, persisted as the ``content`` JSON blob
-    of one observation_history row.
-
-    Temporal fields are the ISO strings carried on MemoryFact; new_source_memory_ids
-    are the ids added by the update.
-    """
-
-    previous_text: str | None
-    previous_tags: list[str]
-    previous_occurred_start: str | None
-    previous_occurred_end: str | None
-    previous_mentioned_at: str | None
-    new_source_memory_ids: list[str]
-
-
-async def _append_observation_history(
-    conn: "Connection",
-    bank_id: str,
-    observation_id: str,
-    snapshot: _ObservationHistorySnapshot,
-    max_entries: int,
-) -> None:
-    """Insert one pre-update snapshot into ``observation_history``, then delete the
-    oldest rows beyond ``max_entries`` for this observation.
-
-    The snapshot is stored as a single JSONB ``content`` blob (per-row, so it stays
-    small). Bounding by row count keeps a frequently-reinforced observation's
-    history from growing without bound.
-    """
-    obs_uuid = uuid.UUID(observation_id)
-    await conn.execute(
-        f"""
-        INSERT INTO {fq_table("observation_history")} (observation_id, bank_id, content, changed_at)
-        VALUES ($1, $2, $3::jsonb, now())
-        """,
-        obs_uuid,
-        bank_id,
-        json.dumps(asdict(snapshot)),
-    )
-    if max_entries and max_entries > 0:
-        await conn.execute(
-            f"""
-            DELETE FROM {fq_table("observation_history")}
-            WHERE observation_id = $1
-              AND id NOT IN (
-                  SELECT id FROM {fq_table("observation_history")}
-                  WHERE observation_id = $1
-                  ORDER BY changed_at DESC, id DESC
-                  LIMIT $2
-              )
-            """,
-            obs_uuid,
-            max_entries,
-        )
-
-
 async def _execute_update_action(
    conn: "Connection",
    memory_engine: "MemoryEngine",
@@ -1672,15 +1166,12 @@ async def _execute_update_action(
    source_occurred_end: datetime | None = None,
    source_mentioned_at: datetime | None = None,
    perf: ConsolidationPerfLog | None = None,
-) -> str | None:
+) -> None:
    """
    Update an existing observation.

    Extends source_memory_ids with all contributing memories, updates temporal fields
    (LEAST for occurred_start, GREATEST for occurred_end / mentioned_at), and merges tags.
-
-    Returns the observation's freshly-computed embedding (pgvector literal) so the caller can
-    run UPDATE-path dedup without re-embedding, or None when the update was skipped.
    """
    model = next((m for m in observations if str(m.id) == observation_id), None)
    if not model:
@@ -1698,14 +1189,15 @@ async def _execute_update_action(

    from ...config import get_config

-    history_entry = _ObservationHistorySnapshot(
-        previous_text=model.text,
-        previous_tags=list(model.tags or []),
-        previous_occurred_start=model.occurred_start,
-        previous_occurred_end=model.occurred_end,
-        previous_mentioned_at=model.mentioned_at,
-        new_source_memory_ids=[str(mid) for mid in source_memory_ids],
-    )
+    history_entry = {
+        "previous_text": model.text,
+        "previous_tags": list(model.tags or []),
+        "previous_occurred_start": model.occurred_start,
+        "previous_occurred_end": model.occurred_end,
+        "previous_mentioned_at": model.mentioned_at,
+        "changed_at": datetime.now(timezone.utc).isoformat(),
+        "new_source_memory_ids": [str(mid) for mid in source_memory_ids],
+    }

    source_ids = list(model.source_fact_ids or []) + source_memory_ids

@@ -1721,6 +1213,9 @@ async def _execute_update_action(
        perf.record_timing("embedding", time.time() - t0)

    config = get_config()
+    history_clause = (
+        "history = COALESCE(history, '[]'::jsonb) || $3::jsonb," if config.enable_observation_history else ""
+    )

    t0 = time.time()
    await conn.execute(
@@ -1728,17 +1223,19 @@ async def _execute_update_action(
        UPDATE {fq_table("memory_units")}
        SET text = $1,
            embedding = $2::vector,
-            source_memory_ids = $3,
-            proof_count = $4,
-            tags = $9,
+            {history_clause}
+            source_memory_ids = $4,
+            proof_count = $5,
+            tags = $10,
            updated_at = now(),
-            occurred_start = LEAST(occurred_start, COALESCE($6, occurred_start)),
-            occurred_end = GREATEST(occurred_end, COALESCE($7, occurred_end)),
-            mentioned_at = GREATEST(mentioned_at, COALESCE($8, mentioned_at))
-        WHERE id = $5
+            occurred_start = LEAST(occurred_start, COALESCE($7, occurred_start)),
+            occurred_end = GREATEST(occurred_end, COALESCE($8, occurred_end)),
+            mentioned_at = GREATEST(mentioned_at, COALESCE($9, mentioned_at))
+        WHERE id = $6
        """,
        new_text,
        embedding_str,
+        json.dumps([history_entry]),
        source_ids,
        len(source_ids),
        uuid.UUID(observation_id),
@@ -1748,15 +1245,6 @@ async def _execute_update_action(
        merged_tags,
    )

-    # Record the pre-update snapshot in the dedicated observation_history table
-    # (one row per change), then trim to the configured cap. History lived in a
-    # single unbounded JSONB column before; an often-reinforced observation grew
-    # it until it crossed Postgres's 256MB jsonb limit and got stuck.
-    if config.enable_observation_history:
-        await _append_observation_history(
-            conn, bank_id, observation_id, history_entry, config.observation_history_max_entries
-        )
-
    # Sync observation_sources junction table (Oracle only — PG uses native array ops).
    if memory_engine._backend.ops.uses_observation_sources_table:
        obs_uuid = uuid.UUID(observation_id)
@@ -1777,10 +1265,7 @@ async def _execute_update_action(
    if perf:
        perf.record_timing("db_write", time.time() - t0)

-    # Map the updated observation onto the consolidation trace as a produced memory.
-    record_created_memory_ids([observation_id])
    logger.debug(f"Updated observation {observation_id} from {len(source_memory_ids)} source memories")
-    return embedding_str


 async def _execute_create_action(
@@ -1802,7 +1287,7 @@ async def _execute_create_action(
    Tags are inherited from the source facts (determined algorithmically, not by LLM)
    to maintain visibility scope.
    """
-    created = await _create_observation_directly(
+    await _create_observation_directly(
        conn=conn,
        memory_engine=memory_engine,
        bank_id=bank_id,
@@ -1815,10 +1300,6 @@ async def _execute_create_action(
        mentioned_at=mentioned_at,
        perf=perf,
    )
-    # Map the new observation onto the consolidation trace as a produced memory.
-    new_id = created.get("observation_id")
-    if new_id:
-        record_created_memory_ids([new_id])
    logger.debug(f"Created observation from {len(source_memory_ids)} source memories")


@@ -1917,13 +1398,6 @@ async def _find_related_observations(
            include_source_facts=True,  # Embed source facts so we avoid a separate DB fetch
            max_source_facts_tokens=config.consolidation_source_facts_max_tokens,
            max_source_facts_tokens_per_observation=config.consolidation_source_facts_max_tokens_per_observation,
-            # Round-robin interleave fusion (no cross-encoder): consolidation is looking
-            # for an existing near-identical observation to merge into. Both the
-            # cross-encoder (semantic #1 -> reranked #37) and RRF (semantic #1 -> outside
-            # the 512-token budget) were measured to bury that twin; interleave guarantees
-            # each retrieval arm's top hits a slot, so the semantic-#1 twin is always shown
-            # to the LLM, which then UPDATEs instead of creating a duplicate.
-            reranking="interleave",
            _quiet=True,  # Suppress logging
        )
    finally:
@@ -2058,38 +1532,16 @@ async def _consolidate_batch_with_llm(
                f"(out of {max_observations_per_scope}). Prefer UPDATE over CREATE when possible."
            )

-    # Split the prompt: a bank-agnostic system instruction (rules + input format +
-    # decision guide + output format) that is byte-identical across batches AND
-    # across banks, and a per-batch user message (mission + capacity note + facts +
-    # existing observations). The split lets the system prefix be served from a
-    # single Gemini context cache shared by every bank — the bank mission, capacity
-    # note, and response_schema (all bank/batch-variable) are kept OUT of the
-    # cached prefix so one cache serves all and it never busts within a run.
-    system_prompt = build_consolidation_system_prompt(
+    prompt_template = build_batch_consolidation_prompt(
+        config.observations_mission,
+        observation_capacity_note,
        llm_output_language=getattr(config, "llm_output_language", None),
    )
-    user_content = build_consolidation_input(
+    prompt = prompt_template.format(
        facts_text=facts_lines,
        observations_text=observations_text,
-        observations_mission=config.observations_mission,
-        observation_capacity_note=observation_capacity_note,
    )

-    # Opt into context caching of the stable system prefix when the provider
-    # supports it (gemini/vertexai with the flag on). response_schema is NOT
-    # passed to the fingerprint: it varies per batch (max_creates) but is not
-    # part of the cached prefix, so keying on it would needlessly bust the cache.
-    cached_prefix_name: str | None = None
-    provider_impl = getattr(llm_config, "_provider_impl", None)
-    if provider_impl is not None and provider_impl.supports_prompt_caching():
-        try:
-            cached_prefix_name = await provider_impl.get_or_create_cached_prefix(
-                system_instruction=system_prompt,
-            )
-        except Exception:
-            logger.exception("Consolidation cache prefix lookup failed; falling back to uncached call")
-            cached_prefix_name = None
-
    # Use a constrained response model when observation limit is active
    response_model = _build_response_model(max_creates=remaining_observation_slots)

@@ -2109,23 +1561,12 @@ async def _consolidate_batch_with_llm(
    for attempt in range(1, max_attempts + 1):
        try:
            call_kwargs: dict[str, Any] = {
-                "messages": [
-                    {"role": "system", "content": system_prompt},
-                    {"role": "user", "content": user_content},
-                ],
+                "messages": [{"role": "user", "content": prompt}],
                "response_format": response_model,
                "scope": "consolidation",
            }
-            # Only request an explicit output budget when configured. Left unset by default the key is
-            # omitted, so each provider keeps its implicit default (backwards compatible). Operators on
-            # providers with a low hidden cap (notably Bedrock imported models, which truncate structured
-            # consolidation JSON) set HINDSIGHT_API_CONSOLIDATION_MAX_COMPLETION_TOKENS to fix it.
-            if config.consolidation_max_completion_tokens is not None:
-                call_kwargs["max_completion_tokens"] = config.consolidation_max_completion_tokens
            if inner_max_retries is not None:
                call_kwargs["max_retries"] = inner_max_retries
-            if cached_prefix_name is not None:
-                call_kwargs["cached_prefix"] = cached_prefix_name
            response: _ConsolidationBatchResponse = await llm_config.call(**call_kwargs)
            # Defensive truncation: some LLM providers may not enforce JSON schema max_length
            creates = response.creates
@@ -2142,7 +1583,7 @@ async def _consolidate_batch_with_llm(
                updates=updates,
                deletes=response.deletes,
                obs_count=len(union_observations),
-                prompt_chars=len(system_prompt) + len(user_content),
+                prompt_chars=len(prompt),
            )
        except Exception as exc:
            last_exc = exc
@@ -2154,9 +1595,7 @@ async def _consolidate_batch_with_llm(
        f"[CONSOLIDATION] LLM batch call failed after {max_attempts} attempts for {batch_label}, "
        f"skipping batch. Last error: {last_exc}"
    )
-    return _BatchLLMResult(
-        obs_count=len(union_observations), prompt_chars=len(system_prompt) + len(user_content), failed=True
-    )
+    return _BatchLLMResult(obs_count=len(union_observations), prompt_chars=len(prompt), failed=True)


 async def _create_observation_directly(
@@ -2203,10 +1642,10 @@ async def _create_observation_directly(
        # VectorChord: manually tokenize and insert search_vector
        query = f"""
            INSERT INTO {fq_table("memory_units")} (
-                id, bank_id, text, fact_type, embedding, proof_count, source_memory_ids,
+                id, bank_id, text, fact_type, embedding, proof_count, source_memory_ids, history,
                tags, event_date, occurred_start, occurred_end, mentioned_at, search_vector
            )
-            VALUES ($1, $2, $3, 'observation', $4::vector, 1, $5, $6, $7, $8, $9, $10,
+            VALUES ($1, $2, $3, 'observation', $4::vector, 1, $5, '[]'::jsonb, $6, $7, $8, $9, $10,
                    tokenize($3, 'llmlingua2')::bm25_catalog.bm25vector)
            RETURNING id
        """
@@ -2222,10 +1661,10 @@ async def _create_observation_directly(
        # re-ingested. Tracking a separate fix for that gap.
        query = f"""
            INSERT INTO {fq_table("memory_units")} (
-                id, bank_id, text, fact_type, embedding, proof_count, source_memory_ids,
+                id, bank_id, text, fact_type, embedding, proof_count, source_memory_ids, history,
                tags, event_date, occurred_start, occurred_end, mentioned_at
            )
-            VALUES ($1, $2, $3, 'observation', $4::vector, 1, $5, $6, $7, $8, $9, $10)
+            VALUES ($1, $2, $3, 'observation', $4::vector, 1, $5, '[]'::jsonb, $6, $7, $8, $9, $10)
            RETURNING id
        """

@@ -37,33 +37,6 @@ _PROCESSING_RULES = """## PROCESSING RULES

 9. KEEP DISTINCT TOPICS DISTINCT: do not merge observations about different people, entities, or unrelated topics. Merging is for the same canonical fact recurring — not for related-but-distinct claims."""

-# Stable description of the input shape. For the cached split path this lives in
-# the system prefix (build_consolidation_system_prompt) so it is not re-sent on
-# every batch; the per-batch user message then carries only the actual data.
-_INPUT_FORMAT_NOTE = """## INPUT FORMAT
-
-Each request provides new facts and existing observations:
- New facts: one per line, each prefixed with its `[uuid]`, followed by the fact text and optional temporal fields.
- Existing observations: a JSON array pooled from recalls across the new facts. Each entry has:
-  - `id`: unique identifier — copy this exactly when issuing an UPDATE or DELETE
-  - `text`: the observation content
-  - `proof_count`: number of supporting memories
-  - `occurred_start` / `occurred_end`: temporal range of source facts
-  - `source_memories`: array of supporting facts with their text and dates"""
-
-# Per-batch data section for the cached split path — the stable format
-# explanation above is omitted here (it lives in the cached prefix); only the
-# variable facts/observations remain. Placeholders substituted at call time.
-_SPLIT_INPUT_SECTION = """## INPUT
-
-### New facts
-
-{facts_text}
-
-### Existing observations
-
-{observations_text}"""
-
 # Data section — format placeholders {facts_text} and {observations_text} are substituted at call time
 _INPUT_SECTION = """## INPUT

@@ -92,7 +65,7 @@ _DECISION_GUIDE = """## DECISION GUIDE
 # Output format — JSON braces escaped as {{ }} so .format() leaves them literal
 _OUTPUT_SECTION = """## OUTPUT FORMAT

-Return a JSON object with three arrays: `creates`, `updates`, `deletes`. Every entry must include a `reason`.
+Return a JSON object with three arrays: `creates`, `updates`, `deletes`.

 ### Example 1 — Merging recurring claims into an existing observation

@@ -106,7 +79,7 @@ Existing observation:
 Expected output (one UPDATE, no creates — both new facts are additional evidence for the same canonical decision):

 {{"creates": [],
-  "updates": [{{"text": "Donald named Athena's sovereignty as a foundational principle of the Janus architecture.", "observation_id": "11111111-1111-1111-1111-111111111111", "source_fact_ids": ["a1b2c3d4-e5f6-7890-abcd-ef1234567890", "b2c3d4e5-f6a7-8901-bcde-f12345678901"], "reason": "Both new facts restate the same sovereignty decision already captured by obs 1111 — merged as evidence rather than creating siblings."}}],
+  "updates": [{{"text": "Donald named Athena's sovereignty as a foundational principle of the Janus architecture.", "observation_id": "11111111-1111-1111-1111-111111111111", "source_fact_ids": ["a1b2c3d4-e5f6-7890-abcd-ef1234567890", "b2c3d4e5-f6a7-8901-bcde-f12345678901"]}}],
  "deletes": []}}

 ### Example 2 — State change updates one observation; unrelated fact creates a new one
@@ -120,8 +93,8 @@ Existing observation:

 Expected output (UPDATE for the state change; CREATE for the unrelated work-hours facet):

-{{"creates": [{{"text": "Alice works long hours, often past midnight.", "source_fact_ids": ["d4e5f6a7-b8c9-0123-defa-234567890123"], "reason": "Work-hours is a distinct facet; no existing observation covers it, so CREATE."}}],
-  "updates": [{{"text": "Alice owned a 2019 Honda Civic; sold it on March 15, 2025.", "observation_id": "22222222-2222-2222-2222-222222222222", "source_fact_ids": ["c3d4e5f6-a7b8-9012-cdef-123456789012"], "reason": "State change to the existing Honda Civic observation 2222 — UPDATE, not a new sibling."}}],
+{{"creates": [{{"text": "Alice works long hours, often past midnight.", "source_fact_ids": ["d4e5f6a7-b8c9-0123-defa-234567890123"]}}],
+  "updates": [{{"text": "Alice owned a 2019 Honda Civic; sold it on March 15, 2025.", "observation_id": "22222222-2222-2222-2222-222222222222", "source_fact_ids": ["c3d4e5f6-a7b8-9012-cdef-123456789012"]}}],
  "deletes": []}}

 ### Observation text rules
@@ -137,7 +110,6 @@ Expected output (UPDATE for the state change; CREATE for the unrelated work-hour
 - One create or update may reference multiple facts when they jointly support the observation.
 - **AT MOST ONE UPDATE PER `observation_id`**: if several new facts all update the same existing observation, emit a single `updates` entry that lists all contributing `source_fact_ids` and a single consolidated `text`. Never emit two `updates` entries with the same `observation_id` in one response — they would silently overwrite each other.
 - `deletes`: only when an observation is directly superseded or contradicted by new facts.
- `reason`: REQUIRED on every create/update/delete — one sentence explaining the choice. For a CREATE, state which existing observation(s) you considered and why none matched (a near-identical existing observation means you should UPDATE, not CREATE). This is audited to catch duplicate creates.
 - Do NOT include `tags` — handled automatically.
 - Return `{{"creates": [], "updates": [], "deletes": []}}` if nothing durable is found."""

@@ -173,55 +145,3 @@ def build_batch_consolidation_prompt(
        f"{_DECISION_GUIDE}\n\n"
        f"{_OUTPUT_SECTION}" + output_language_directive(llm_output_language)
    )
-
-
-def build_consolidation_system_prompt(
-    llm_output_language: str | None = None,
-) -> str:
-    """Bank-agnostic, cacheable system instruction for batch consolidation.
-
-    Holds only what is constant across banks: processing rules, input format,
-    decision guide, and output format. The bank's MISSION is deliberately NOT
-    here — baking it in would make the prefix bank-specific and force a separate
-    Gemini context cache per mission. The mission, the per-batch INPUT, and any
-    capacity constraint all ride in the user message (see
-    :func:`build_consolidation_input`), so this prefix is identical for every
-    bank and a single CachedContent serves them all. Returns final text
-    (brace-escaped examples already unescaped) for verbatim use as system message
-    and cached prefix.
-    """
-    template = (
-        "You are a memory consolidation system. Synthesize new facts into "
-        "observations, merging with existing observations when appropriate.\n\n"
-        f"{_MISSION_PRIORITY_NOTE}\n\n"
-        f"{_PROCESSING_RULES}\n\n"
-        f"{_INPUT_FORMAT_NOTE}\n\n"
-        f"{_DECISION_GUIDE}\n\n"
-        f"{_OUTPUT_SECTION}" + output_language_directive(llm_output_language)
-    )
-    # No {facts_text}/{observations_text} placeholders here — the only braces are
-    # the doubled {{ }} in the OUTPUT examples, which .format() unescapes.
-    return template.format()
-
-
-def build_consolidation_input(
-    facts_text: str,
-    observations_text: str,
-    observations_mission: str | None = None,
-    observation_capacity_note: str | None = None,
-) -> str:
-    """Per-batch user message: MISSION + INPUT data + any capacity constraint.
-
-    The MISSION lives here (not in the cached system prefix) so the prefix stays
-    bank-agnostic and one CachedContent serves every bank. The capacity note also
-    lives here since it varies as observation slots fill.
-    """
-    mission = escape_for_prompt(observations_mission or _DEFAULT_MISSION)
-    mission_section = f"## MISSION\n\n{mission}\n\n"
-    capacity_section = ""
-    if observation_capacity_note:
-        capacity_section = f"## CAPACITY CONSTRAINT\n\n{escape_for_prompt(observation_capacity_note)}\n\n"
-    # _SPLIT_INPUT_SECTION omits the stable observation-format explanation (now in
-    # the cached system prefix) — only the variable facts/observations remain.
-    template = mission_section + capacity_section + _SPLIT_INPUT_SECTION
-    return template.format(facts_text=facts_text, observations_text=observations_text)
@@ -46,6 +46,7 @@ from ..config import (
    ENV_RERANKER_FLASHRANK_CPU_MEM_ARENA,
    ENV_RERANKER_FLASHRANK_MODEL,
    ENV_RERANKER_GOOGLE_PROJECT_ID,
+    ENV_RERANKER_LITELLM_SDK_API_KEY,
    ENV_RERANKER_LOCAL_FORCE_CPU,
    ENV_RERANKER_LOCAL_MAX_CONCURRENT,
    ENV_RERANKER_LOCAL_MODEL,
@@ -1198,7 +1199,7 @@ class LiteLLMSDKCrossEncoder(CrossEncoderModel):

    def __init__(
        self,
-        api_key: str | None = None,
+        api_key: str,
        model: str = DEFAULT_RERANKER_LITELLM_SDK_MODEL,
        api_base: str | None = None,
        timeout: float = 60.0,
@@ -1208,8 +1209,7 @@ class LiteLLMSDKCrossEncoder(CrossEncoderModel):
        Initialize LiteLLM SDK cross-encoder client.

        Args:
-            api_key: API key for the reranking provider (optional — omit for
-                     providers that use ambient credentials, e.g. AWS Bedrock with IAM)
+            api_key: API key for the reranking provider
            model: Model name with provider prefix (e.g., "deepinfra/Qwen3-reranker-8B")
            api_base: Custom base URL for API (optional)
            timeout: Request timeout in seconds (default: 60.0)
@@ -1284,9 +1284,8 @@ class LiteLLMSDKCrossEncoder(CrossEncoderModel):
                "model": self.model,
                "query": query,
                "documents": texts,
+                "api_key": self.api_key,
            }
-            if self.api_key:
-                rerank_kwargs["api_key"] = self.api_key
            if self.api_base:
                rerank_kwargs["api_base"] = self.api_base

@@ -1698,8 +1697,13 @@ def create_cross_encoder_from_env() -> CrossEncoderModel:
            timeout=config.reranker_litellm_timeout,
        )
    elif provider == "litellm-sdk":
+        api_key = config.reranker_litellm_sdk_api_key
+        if not api_key:
+            raise ValueError(
+                f"{ENV_RERANKER_LITELLM_SDK_API_KEY} is required when {ENV_RERANKER_PROVIDER} is 'litellm-sdk'"
+            )
        return LiteLLMSDKCrossEncoder(
-            api_key=config.reranker_litellm_sdk_api_key or None,
+            api_key=api_key,
            model=config.reranker_litellm_sdk_model,
            api_base=config.reranker_litellm_sdk_api_base,
            max_tokens_per_doc=config.reranker_litellm_max_tokens_per_doc,
@@ -72,30 +72,6 @@ class DataAccessOps(ABC):
        """
        ...

-    @abstractmethod
-    async def lock_document_for_write(
-        self,
-        conn: DatabaseConnection,
-        table: str,
-        doc_id: str,
-        bank_id: str,
-    ) -> str | None:
-        """Ensure the document row exists, take a row lock on it, and return its
-        pre-existing ``content_hash``.
-
-        This serializes all concurrent writers for ``doc_id`` at the DB level
-        (so interleaved same-document retains can't corrupt each other), while
-        creating the row on first write. The returned hash is ``'__pending__'``
-        for a freshly inserted row, the stored hash for an existing one, or
-        ``None`` if the row could not be read back.
-
-        PG does this in a single statement (``INSERT ... ON CONFLICT DO UPDATE
-        ... RETURNING``), which always takes the row lock as part of the upsert.
-        Oracle can't (``MERGE`` doesn't support ``RETURNING``), so it splits the
-        work into an idempotent insert plus a ``SELECT ... FOR UPDATE``.
-        """
-        ...
-
    @abstractmethod
    async def insert_facts_batch(
        self,
@@ -47,37 +47,6 @@ class OracleOps(DataAccessOps):
            column_types=["text[]", "text[]", "text[]", "text[]", "integer[]", "text[]"],
        )

-    async def lock_document_for_write(
-        self,
-        conn: DatabaseConnection,
-        table: str,
-        doc_id: str,
-        bank_id: str,
-    ) -> str | None:
-        # Oracle can't express the PG "INSERT ... ON CONFLICT DO UPDATE ...
-        # RETURNING" upsert in one statement — MERGE doesn't support RETURNING,
-        # so the single-statement form rewrites to a MERGE that returns no rows
-        # (DPY-1003). Split it into two statements instead:
-        #   1. Idempotent insert that silently skips an existing row. The
-        #      IGNORE_ROW_ON_DUPKEY_INDEX hint suppresses ORA-00001 server-side;
-        #      a concurrent uncommitted insert of the same key blocks here until
-        #      the other writer commits, so writers still serialize.
-        #   2. SELECT ... FOR UPDATE to take the row lock and read the hash
-        #      ('__pending__' for a row we just inserted, the stored hash for an
-        #      existing one).
-        await conn.execute(
-            f"INSERT /*+ IGNORE_ROW_ON_DUPKEY_INDEX({table}, pk_documents) */ "
-            f"INTO {table} (id, bank_id, original_text, content_hash) "
-            f"VALUES ($1, $2, '', '__pending__')",
-            doc_id,
-            bank_id,
-        )
-        return await conn.fetchval(
-            f"SELECT content_hash FROM {table} WHERE id = $1 AND bank_id = $2 FOR UPDATE",
-            doc_id,
-            bank_id,
-        )
-
    async def insert_facts_batch(
        self,
        conn: DatabaseConnection,
@@ -49,30 +49,6 @@ class PostgreSQLOps(DataAccessOps):
            content_hashes,
        )

-    async def lock_document_for_write(
-        self,
-        conn: DatabaseConnection,
-        table: str,
-        doc_id: str,
-        bank_id: str,
-    ) -> str | None:
-        # Single upsert that both creates the row (if absent) and locks it (if
-        # present) atomically. ON CONFLICT DO UPDATE always takes the row lock as
-        # part of the statement, so all concurrent same-document writers serialize
-        # on the document row in one consistent step (the earlier two-step form —
-        # DO NOTHING + a separate SELECT FOR UPDATE — could deadlock because
-        # DO NOTHING takes no lock on an existing row). The SET is a no-op
-        # self-assignment used only to acquire the lock; RETURNING yields the
-        # pre-existing hash (or '__pending__' for a freshly inserted row).
-        return await conn.fetchval(
-            f"INSERT INTO {table} (id, bank_id, original_text, content_hash) "
-            f"VALUES ($1, $2, '', '__pending__') "
-            f"ON CONFLICT (id, bank_id) DO UPDATE SET content_hash = {table}.content_hash "
-            f"RETURNING content_hash",
-            doc_id,
-            bank_id,
-        )
-
    async def insert_facts_batch(
        self,
        conn: DatabaseConnection,
@@ -14,7 +14,6 @@ Supports multi-tenant schema isolation via ALTER SESSION SET CURRENT_SCHEMA.
 """

 import datetime
-import inspect
 import json
 import logging
 import re
@@ -156,23 +155,6 @@ _JSON_COL_NAMES = {
    "task_payload",
    "history",
 }
-# NOTE: the history tables' JSON payload column is named ``content`` — deliberately
-# NOT added here, because ``mental_models.content`` is plain text (adding "content"
-# would corrupt those reads). The history read paths json.loads ``content`` directly.
-
-# Columns backed by CLOB in Oracle (large text or JSON). When such a column is
-# returned via a ``RETURNING`` clause it must be bound as DB_TYPE_CLOB; binding
-# it as VARCHAR raises ORA-22835 ("buffer too small for CLOB to CHAR") once the
-# value exceeds 4000 bytes. Union of the JSON-CLOB columns above and the
-# large-text CLOB columns.
-_CLOB_RETURNING_COLS = _JSON_COL_NAMES | {
-    "content",
-    "text",
-    "context",
-    "structured_content",
-    "text_signals",
-    "search_vector",
-}


 def _is_uuid_column(col: str) -> bool:
@@ -703,11 +685,6 @@ class OracleConnection(DatabaseConnection):
                    params[f"ret_{i}"] = cursor.var(oracledb.DB_TYPE_TIMESTAMP_TZ, arraysize=1)
                elif clean in _NUMERIC_COLS:
                    params[f"ret_{i}"] = cursor.var(oracledb.DB_TYPE_NUMBER, arraysize=1)
-                elif clean in _CLOB_RETURNING_COLS:
-                    # CLOB-backed column: a VARCHAR out-bind caps at 4000 bytes and
-                    # raises ORA-22835 for larger values. Read back as a LOB in
-                    # _read_returning_values.
-                    params[f"ret_{i}"] = cursor.var(oracledb.DB_TYPE_CLOB, arraysize=1)
                else:
                    params[f"ret_{i}"] = cursor.var(oracledb.DB_TYPE_VARCHAR, arraysize=1)

@@ -885,7 +862,7 @@ class OracleConnection(DatabaseConnection):

        return query, params

-    async def _read_returning_values(self, returning_cols: list[str], params: dict[str, Any]) -> dict[str, Any] | None:
+    def _read_returning_values(self, returning_cols: list[str], params: dict[str, Any]) -> dict[str, Any] | None:
        """Read values from RETURNING INTO output variables after execute."""
        row: dict[str, Any] = {}
        for i, col in enumerate(returning_cols):
@@ -895,14 +872,6 @@ class OracleConnection(DatabaseConnection):
                return None
            val = values[0] if isinstance(values, list) else values

-            # CLOB-bound columns return a LOB handle; read it to a string. The
-            # async pool yields AsyncLOB whose read() is a coroutine.
-            if val is not None and not isinstance(val, (str, bytes, int, float)) and hasattr(val, "read"):
-                data = val.read()
-                if inspect.isawaitable(data):
-                    data = await data
-                val = data
-
            # Clean alias: "LOWER(canonical_name) AS name_lower" → "name_lower"
            clean_col = col.strip()
            upper = clean_col.upper()
@@ -1090,7 +1059,7 @@ class OracleConnection(DatabaseConnection):
                raise

            if ret_cols is not None:
-                row_dict = await self._read_returning_values(ret_cols, params)
+                row_dict = self._read_returning_values(ret_cols, params)
                return [ResultRow(row_dict)] if row_dict else []

            columns = [col[0].lower() for col in cursor.description or []]
@@ -1128,7 +1097,7 @@ class OracleConnection(DatabaseConnection):
                raise

            if ret_cols is not None:
-                row_dict = await self._read_returning_values(ret_cols, params)
+                row_dict = self._read_returning_values(ret_cols, params)
                return ResultRow(row_dict) if row_dict else None

            columns = [col[0].lower() for col in cursor.description or []]
@@ -1161,7 +1130,7 @@ class OracleConnection(DatabaseConnection):
            await cursor.execute(query, params)

            if ret_cols is not None:
-                row_dict = await self._read_returning_values(ret_cols, params)
+                row_dict = self._read_returning_values(ret_cols, params)
                if row_dict is None:
                    return None
                vals = list(row_dict.values())
@@ -43,10 +43,6 @@ from ..config import (
    ENV_EMBEDDINGS_LOCAL_FORCE_CPU,
    ENV_EMBEDDINGS_LOCAL_MODEL,
    ENV_EMBEDDINGS_LOCAL_TRUST_REMOTE_CODE,
-    ENV_EMBEDDINGS_ONNX_DIMENSIONS,
-    ENV_EMBEDDINGS_ONNX_MODEL_ID,
-    ENV_EMBEDDINGS_ONNX_MODEL_PATH,
-    ENV_EMBEDDINGS_ONNX_TOKENIZER_NAME_OR_PATH,
    ENV_EMBEDDINGS_OPENAI_API_KEY,
    ENV_EMBEDDINGS_OPENAI_BASE_URL,
    ENV_EMBEDDINGS_OPENAI_MODEL,
@@ -256,172 +252,6 @@ class LocalSTEmbeddings(Embeddings):
        return [emb.tolist() for emb in embeddings]


-class OnnxEmbeddings(Embeddings):
-    """Local ONNX Runtime embeddings provider.
-
-    This provider runs transformer embedding models in-process with ONNX Runtime,
-    avoiding a sidecar Ollama/TEI server or a remote embeddings API. It supports
-    sentence-transformer style mean pooling and E5-style asymmetric prefixes.
-    """
-
-    def __init__(
-        self,
-        model_id: str,
-        model_path: str | None = None,
-        tokenizer_name_or_path: str | None = None,
-        onnx_file: str = "onnx/model.onnx",
-        dimensions: int | None = None,
-        max_tokens: int = 512,
-        pooling: str = "mean",
-        normalize: bool = True,
-        query_prefix: str = "query: ",
-        passage_prefix: str = "passage: ",
-        output_name: str | None = None,
-    ):
-        self.model_id = model_id
-        self.model_path = model_path
-        if model_path and tokenizer_name_or_path is None:
-            logger.warning(
-                "Embeddings: ONNX model_path is set without tokenizer_name_or_path; "
-                "falling back to tokenizer from model_id %s. Set "
-                "HINDSIGHT_API_EMBEDDINGS_ONNX_TOKENIZER_NAME_OR_PATH when using local ONNX artifacts.",
-                model_id,
-            )
-        self.tokenizer_name_or_path = tokenizer_name_or_path or model_id
-        self.onnx_file = onnx_file
-        self.configured_dimensions = dimensions
-        self.max_tokens = max_tokens
-        self.pooling = pooling.lower()
-        if self.pooling not in {"mean", "cls"}:
-            raise ValueError("ONNX embeddings pooling must be 'mean' or 'cls'")
-        self.normalize = normalize
-        self.query_prefix = query_prefix
-        self.passage_prefix = passage_prefix
-        self.output_name = output_name
-        self._session = None
-        self._tokenizer = None
-        self._dimension: int | None = dimensions
-
-    @property
-    def provider_name(self) -> str:
-        return "onnx"
-
-    @property
-    def dimension(self) -> int:
-        if self._dimension is None:
-            raise RuntimeError("Embeddings not initialized. Call initialize() first.")
-        return self._dimension
-
-    async def initialize(self) -> None:
-        if self._session is not None and self._tokenizer is not None:
-            return
-
-        try:
-            import onnxruntime as ort
-            from transformers import AutoTokenizer
-        except ImportError as exc:
-            raise ImportError(
-                "onnxruntime and transformers are required for OnnxEmbeddings. "
-                "Install with: pip install 'hindsight-api-slim[local-onnx]'"
-            ) from exc
-
-        model_path = self.model_path
-        if not model_path:
-            try:
-                from huggingface_hub import snapshot_download
-            except ImportError as exc:
-                raise ImportError(
-                    "huggingface-hub is required to download ONNX embedding models. "
-                    "Set HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_PATH or install local-onnx."
-                ) from exc
-            # Some large ONNX exports, for example BAAI/bge-m3, store weights in
-            # an external sidecar file next to model.onnx. Download both the
-            # requested graph and its conventional *_data sidecar when present.
-            snapshot_dir = snapshot_download(
-                repo_id=self.model_id,
-                allow_patterns=[self.onnx_file, f"{self.onnx_file}_data"],
-            )
-            model_path = os.path.join(snapshot_dir, self.onnx_file)
-
-        logger.info(
-            "Embeddings: initializing ONNX provider with model %s (%s)",
-            self.model_id,
-            model_path,
-        )
-        logger.info(
-            "Embeddings: ONNX query_prefix=%r passage_prefix=%r pooling=%s normalize=%s",
-            self.query_prefix,
-            self.passage_prefix,
-            self.pooling,
-            self.normalize,
-        )
-        self._tokenizer = AutoTokenizer.from_pretrained(self.tokenizer_name_or_path)
-        self._session = ort.InferenceSession(model_path, providers=["CPUExecutionProvider"])
-
-        detected = len(self.encode(["test"])[0])
-        if self.configured_dimensions is not None and detected != self.configured_dimensions:
-            raise ValueError(
-                f"Configured ONNX embedding dimension {self.configured_dimensions} does not match model output {detected}"
-            )
-        self._dimension = detected
-        logger.info("Embeddings: ONNX provider initialized (dim: %s)", self._dimension)
-
-    def _encode_prefixed(self, texts: list[str], prefix: str) -> list[list[float]]:
-        if prefix:
-            return self.encode([f"{prefix}{text}" for text in texts])
-        return self.encode(texts)
-
-    def encode_query(self, texts: list[str]) -> list[list[float]]:
-        return self._encode_prefixed(texts, self.query_prefix)
-
-    def encode_documents(self, texts: list[str]) -> list[list[float]]:
-        return self._encode_prefixed(texts, self.passage_prefix)
-
-    def encode(self, texts: list[str]) -> list[list[float]]:
-        if self._session is None or self._tokenizer is None:
-            raise RuntimeError("Embeddings not initialized. Call initialize() first.")
-        if not texts:
-            return []
-
-        import numpy as np
-
-        encoded = self._tokenizer(
-            texts,
-            padding=True,
-            truncation=True,
-            max_length=self.max_tokens,
-            return_tensors="np",
-        )
-        input_names = {inp.name for inp in self._session.get_inputs()}
-        ort_inputs = {name: value for name, value in encoded.items() if name in input_names}
-        if "token_type_ids" in input_names and "token_type_ids" not in ort_inputs:
-            ort_inputs["token_type_ids"] = np.zeros_like(encoded["input_ids"])
-
-        outputs = self._session.run([self.output_name] if self.output_name else None, ort_inputs)
-        token_embeddings = outputs[0]
-
-        # Some exported models expose a pooled 2-D embedding as their first output.
-        if getattr(token_embeddings, "ndim", 0) == 2:
-            embeddings = token_embeddings
-        elif self.pooling == "cls":
-            embeddings = token_embeddings[:, 0]
-        else:
-            attention_mask = encoded.get("attention_mask")
-            if attention_mask is None:
-                attention_mask = np.ones(token_embeddings.shape[:2], dtype=np.float32)
-            mask = attention_mask[..., None].astype(np.float32)
-            summed = (token_embeddings * mask).sum(axis=1)
-            counts = np.clip(mask.sum(axis=1), a_min=1e-9, a_max=None)
-            embeddings = summed / counts
-
-        if self.normalize:
-            norms = np.linalg.norm(embeddings, axis=1, keepdims=True)
-            norms[norms == 0] = 1
-            embeddings = embeddings / norms
-
-        return embeddings.astype(float).tolist()
-
-
 class RemoteTEIEmbeddings(Embeddings):
    """
    Remote embeddings implementation using HuggingFace Text Embeddings Inference (TEI) HTTP API.
@@ -1561,20 +1391,6 @@ def create_embeddings_from_env() -> Embeddings:
            force_cpu=config.embeddings_local_force_cpu,
            trust_remote_code=config.embeddings_local_trust_remote_code,
        )
-    elif provider == "onnx":
-        return OnnxEmbeddings(
-            model_id=config.embeddings_onnx_model_id,
-            model_path=config.embeddings_onnx_model_path,
-            tokenizer_name_or_path=config.embeddings_onnx_tokenizer_name_or_path,
-            onnx_file=config.embeddings_onnx_file,
-            dimensions=config.embeddings_onnx_dimensions,
-            max_tokens=config.embeddings_onnx_max_tokens,
-            pooling=config.embeddings_onnx_pooling,
-            normalize=config.embeddings_onnx_normalize,
-            query_prefix=config.embeddings_onnx_query_prefix,
-            passage_prefix=config.embeddings_onnx_passage_prefix,
-            output_name=config.embeddings_onnx_output_name,
-        )
    elif provider == "openai":
        # Use dedicated embeddings API key, or fall back to LLM API key
        api_key = os.environ.get(ENV_EMBEDDINGS_OPENAI_API_KEY) or os.environ.get(ENV_LLM_API_KEY)
@@ -1676,6 +1492,6 @@ def create_embeddings_from_env() -> Embeddings:
    else:
        raise ValueError(
            f"Unknown embeddings provider: {provider}. "
-            f"Supported: 'local', 'onnx', 'tei', 'openai', 'openai-codex', 'openrouter', 'cohere', 'google', "
+            f"Supported: 'local', 'tei', 'openai', 'openai-codex', 'openrouter', 'cohere', 'google', "
            f"'zeroentropy', 'litellm', 'litellm-sdk'"
        )
@@ -458,28 +458,8 @@ class MemoryEngineInterface(ABC):
            request_context: Request context for authentication.

        Returns:
-            Dict with node_counts, link_counts, link_counts_by_fact_type
-            (deprecated, returns empty), link_breakdown (deprecated, returns
-            empty), and operations stats.
-        """
-        ...
-
-    @abstractmethod
-    async def get_bank_freshness(
-        self,
-        bank_id: str,
-        *,
-        request_context: "RequestContext",
-    ) -> dict[str, Any]:
-        """
-        Get consolidation freshness for a bank.
-
-        Cheap alternative to get_bank_stats when callers only need
-        last_consolidated_at / pending_consolidation / failed_consolidation.
-
-        Returns:
-            Dict with last_consolidated_at (ISO-8601 string or None),
-            pending_consolidation (int), and failed_consolidation (int).
+            Dict with node_counts, link_counts, link_counts_by_fact_type,
+            link_breakdown, and operations stats.
        """
        ...

@@ -69,7 +69,6 @@ class LLMInterface(ABC):
        skip_validation: bool = False,
        strict_schema: bool = False,
        return_usage: bool = False,
-        cached_prefix: str | None = None,
    ) -> Any:
        """
        Make an LLM API call with retry logic.
@@ -84,13 +83,8 @@ class LLMInterface(ABC):
            initial_backoff: Initial backoff time in seconds.
            max_backoff: Maximum backoff time in seconds.
            skip_validation: Return raw JSON without Pydantic validation.
-            strict_schema: Grammar-enforce structured output via json_schema strict
-                (OpenAI-compatible, LiteLLM) instead of the soft json_object path. Gemini
-                enforces its response_schema natively; providers without a strict mode ignore it.
+            strict_schema: Use strict JSON schema enforcement (OpenAI only).
            return_usage: If True, return tuple (result, TokenUsage) instead of just result.
-            cached_prefix: Opaque handle from ``get_or_create_cached_prefix`` for the
-                cacheable system prefix, or None. Providers without explicit prompt
-                caching ignore it (and the wrapper only forwards it when set).

        Returns:
            If return_usage=False: Parsed response if response_format is provided, otherwise text content.
@@ -114,7 +108,6 @@ class LLMInterface(ABC):
        initial_backoff: float = 1.0,
        max_backoff: float = 30.0,
        tool_choice: str | dict[str, Any] = "auto",
-        cached_prefix: str | None = None,
    ) -> LLMToolCallResult:
        """
        Make an LLM API call with tool/function calling support.
@@ -144,46 +137,6 @@ class LLMInterface(ABC):
        """
        return False

-    # ── Prompt prefix caching (optional, per-provider) ─────────────────────────
-
-    def supports_prompt_caching(self) -> bool:
-        """Whether this provider can cache a reusable prompt prefix.
-
-        Default False. Providers that return True must implement
-        ``get_or_create_cached_prefix`` and honour the ``cached_prefix`` argument
-        of ``call`` / ``call_with_tools``.
-        """
-        return False
-
-    async def get_or_create_cached_prefix(
-        self,
-        *,
-        system_instruction: str,
-        response_schema: Any | None = None,
-        tools: list[dict[str, Any]] | None = None,
-    ) -> str | None:
-        """Cache a reusable prompt prefix and return an opaque handle, or None.
-
-        The engine has already decided WHAT is cacheable: it puts the stable,
-        bank-agnostic instructions in ``system_instruction`` (plus ``tools``) and
-        keeps all per-request / per-bank data (documents, facts, the bank mission)
-        in the user message. A provider only chooses HOW to cache that prefix:
-
-        - Explicit-cache providers (e.g. Gemini ``CachedContent``): create the
-          cache, return its handle; the engine passes the handle back via
-          ``call(cached_prefix=...)`` and the provider then drops the prefix from
-          the request, billing it at the cached rate.
-        - Automatic-cache providers (e.g. OpenAI): no handle needed — caching is
-          transparent as long as the prefix is a stable leading block, which it
-          already is. They can keep this default (return None) and still benefit.
-        - Inline-marker providers (e.g. Anthropic ``cache_control``): mark the
-          prefix block inside ``call`` instead; may also keep this default.
-
-        Returns None when caching is disabled/unsupported or the prefix is too
-        small; callers MUST fall back to an uncached call in that case.
-        """
-        return None
-
    async def submit_batch(
        self,
        requests: list[dict[str, Any]],
@@ -1,540 +0,0 @@
-"""Per-bank LLM request tracing.
-
-Opt-in, fire-and-forget recording of every LLM call Hindsight makes (both
-successes and failures) into the ``llm_requests`` table, per bank. Each row
-captures the input messages, the model output, token usage (input / output /
-cached / total), finish reason, and caller metadata. Disabled by default —
-controlled by ``HINDSIGHT_API_LLM_TRACE_ENABLED``.
-
-This plugs into the OpenTelemetry **GenAI** recording pattern: providers already
-call ``tracing.get_span_recorder().record_llm_call(...)`` on success, so the DB
-tracer is registered as one of those recorders (alongside the OTLP span
-exporter) rather than hooking the call path with custom code. Failures, which
-providers don't report to the recorder, are forwarded from the LLM wrapper.
-
-Bank/operation attribution is carried via a ContextVar set by
-``ConfiguredLLMProvider`` (see ``llm_wrapper.py``); outside a traced context
-``bank_id`` is recorded as NULL.
-"""
-
-from __future__ import annotations
-
-import asyncio
-import json
-import logging
-import uuid
-from collections.abc import Callable, Iterable
-from contextvars import ContextVar, Token
-from dataclasses import dataclass, field
-from datetime import datetime, timedelta, timezone
-from typing import Any
-
-from pydantic import BaseModel
-
-from .db_utils import acquire_with_retry
-
-logger = logging.getLogger(__name__)
-
-
-# ── bank/operation attribution (carried across the async call chain) ──────────
-
-
-@dataclass
-class LLMTraceContext:
-    """Attribution for in-flight LLM calls, bound by ``ConfiguredLLMProvider``.
-
-    ``trace_id`` and ``operation_span_id`` are generated once per operation
-    invocation (one ``with_config`` call), so every LLM call of a single
-    reflect/retain/consolidation run shares them — reproducing the OTel
-    parent (operation span) → children (LLM calls) hierarchy in the DB.
-    """
-
-    bank_id: str | None = None
-    operation: str | None = None  # "retain" | "reflect" | "consolidation" | ...
-    metadata: dict[str, Any] = field(default_factory=dict)
-    trace_id: str | None = None
-    operation_span_id: str | None = None
-    # Memory_units this operation produced/consumed, accumulated at the DB-write
-    # sites and flushed onto every row of the trace at operation end (see
-    # LLMTraceRecorder.attach_memory_ids). Lets a retain/consolidation trace map
-    # to the memories it created (outputs) and consumed (source inputs).
-    created_memory_ids: list[str] = field(default_factory=list)
-    source_memory_ids: list[str] = field(default_factory=list)
-
-
-_trace_ctx: ContextVar[LLMTraceContext | None] = ContextVar("hindsight_llm_trace_ctx", default=None)
-
-# Per-call requested parameters (max_completion_tokens, temperature, response
-# schema, tool_choice). Set by ``LLMProvider.call`` around the provider
-# delegation so the recorder can attach them even though success is reported by
-# the provider. Only includes values the caller actually set — never nulls.
-_request_ctx: ContextVar[dict[str, Any] | None] = ContextVar("hindsight_llm_request_ctx", default=None)
-
-# Per-call caller metadata (e.g. document_id for retain extraction). Set by
-# engine code around a specific LLM call; merged into the row's metadata on top
-# of the operation-level LLMTraceContext.metadata.
-_call_metadata_ctx: ContextVar[dict[str, Any] | None] = ContextVar("hindsight_llm_call_metadata_ctx", default=None)
-
-
-def set_trace_context(ctx: LLMTraceContext | None) -> Token:
-    """Bind trace attribution to the current context. Returns a reset token."""
-    return _trace_ctx.set(ctx)
-
-
-def reset_trace_context(token: Token) -> None:
-    """Unwind a binding made by :func:`set_trace_context`."""
-    _trace_ctx.reset(token)
-
-
-def set_request_context(params: dict[str, Any] | None) -> Token:
-    """Bind the current LLM call's requested parameters. Returns a reset token."""
-    return _request_ctx.set(params)
-
-
-def reset_request_context(token: Token) -> None:
-    """Unwind a binding made by :func:`set_request_context`."""
-    _request_ctx.reset(token)
-
-
-def current_request_context() -> dict[str, Any] | None:
-    """Return the active call's requested parameters, or None."""
-    return _request_ctx.get()
-
-
-def set_call_metadata(metadata: dict[str, Any] | None) -> Token:
-    """Bind per-call caller metadata (e.g. ``{"document_id": ...}``)."""
-    return _call_metadata_ctx.set(metadata)
-
-
-def reset_call_metadata(token: Token) -> None:
-    """Unwind a binding made by :func:`set_call_metadata`."""
-    _call_metadata_ctx.reset(token)
-
-
-def current_call_metadata() -> dict[str, Any] | None:
-    """Return the active call's caller metadata, or None."""
-    return _call_metadata_ctx.get()
-
-
-def current_trace_context() -> LLMTraceContext | None:
-    """Return the active trace attribution, or None outside a traced context."""
-    return _trace_ctx.get()
-
-
-def trace_context_of(llm_config: Any) -> LLMTraceContext | None:
-    """Return a configured provider's operation trace context, or None.
-
-    Real providers expose ``trace_context()`` (``ConfiguredLLMProvider``); test
-    or mock substitutes may not, so this degrades gracefully rather than raising
-    — tracing is best-effort and must never break an operation.
-    """
-    getter = getattr(llm_config, "trace_context", None)
-    return getter() if callable(getter) else None
-
-
-def record_created_memory_ids(ids: Iterable[str]) -> None:
-    """Accumulate output memory_units onto the active operation trace.
-
-    No-op outside a traced operation context (e.g. tracing disabled). Child
-    asyncio tasks inherit the same ``LLMTraceContext`` object, so appends from
-    parallel consolidation batches land on one shared list.
-    """
-    ctx = _trace_ctx.get()
-    if ctx is not None:
-        ctx.created_memory_ids.extend(str(i) for i in ids)
-
-
-def record_source_memory_ids(ids: Iterable[str]) -> None:
-    """Accumulate consumed/source memory_units onto the active operation trace.
-
-    No-op outside a traced operation context.
-    """
-    ctx = _trace_ctx.get()
-    if ctx is not None:
-        ctx.source_memory_ids.extend(str(i) for i in ids)
-
-
-# ── serialization helpers ─────────────────────────────────────────────────────
-
-
-def _json_default(obj: Any) -> Any:
-    """JSON serializer for objects not serializable by default."""
-    if isinstance(obj, datetime):
-        return obj.isoformat()
-    if isinstance(obj, uuid.UUID):
-        return str(obj)
-    if isinstance(obj, bytes):
-        return "<bytes>"
-    if isinstance(obj, set):
-        return list(obj)
-    model_dump = getattr(obj, "model_dump", None)
-    if callable(model_dump):
-        try:
-            return model_dump(mode="json")
-        except Exception:
-            return str(obj)
-    return str(obj)
-
-
-def _safe_json(data: Any, max_chars: int) -> str | None:
-    """Serialize ``data`` to a JSON string, truncating beyond ``max_chars``.
-
-    Returns None on total failure. Truncation preserves valid JSON by wrapping
-    the oversized payload in a marker object with a preview.
-    """
-    if data is None:
-        return None
-    try:
-        serialized = json.dumps(data, default=_json_default)
-    except Exception:
-        logger.debug("Failed to serialize llm trace data", exc_info=True)
-        try:
-            serialized = json.dumps(str(data))
-        except Exception:
-            return None
-    if max_chars and max_chars > 0 and len(serialized) > max_chars:
-        return json.dumps({"_truncated": True, "_original_chars": len(serialized), "preview": serialized[:max_chars]})
-    return serialized
-
-
-# ── record ────────────────────────────────────────────────────────────────────
-
-
-@dataclass
-class LLMRequestRecord:
-    """A single LLM request trace row."""
-
-    provider: str
-    model: str | None
-    scope: str
-    status: str  # "success" | "error"
-    started_at: datetime
-    ended_at: datetime
-    bank_id: str | None = None
-    operation: str | None = None
-    trace_id: str | None = None
-    span_id: str | None = None
-    parent_span_id: str | None = None
-    input: Any = None
-    output: Any = None
-    error: str | None = None
-    input_tokens: int | None = None
-    output_tokens: int | None = None
-    cached_tokens: int | None = None
-    total_tokens: int | None = None
-    llm_info: dict[str, Any] = field(default_factory=dict)
-    metadata: dict[str, Any] = field(default_factory=dict)
-
-    @property
-    def duration_ms(self) -> int:
-        return int((self.ended_at - self.started_at).total_seconds() * 1000)
-
-
-# ── read models (returned by MemoryEngine query methods, served by the API) ───
-
-
-class LLMRequestEntry(BaseModel):
-    """A single LLM request trace row, as returned by the read API."""
-
-    id: str
-    bank_id: str | None
-    operation: str | None
-    scope: str | None
-    trace_id: str | None
-    span_id: str | None
-    parent_span_id: str | None
-    provider: str | None
-    model: str | None
-    status: str
-    started_at: str | None
-    ended_at: str | None
-    duration_ms: int | None
-    input_tokens: int | None
-    output_tokens: int | None
-    cached_tokens: int | None
-    total_tokens: int | None
-    # Arbitrary JSON (message list, string, or object) — open `Any` so the
-    # OpenAPI schema stays a plain open type the Go SDK generator can model.
-    input: Any = None
-    output: Any = None
-    error: str | None
-    llm_info: dict[str, Any]
-    metadata: dict[str, Any]
-
-
-class LLMRequestListResponse(BaseModel):
-    """Paginated list of LLM request traces for a bank."""
-
-    bank_id: str
-    total: int
-    limit: int
-    offset: int
-    items: list[LLMRequestEntry]
-
-
-class LLMRequestTokenSums(BaseModel):
-    """Token totals for a time bucket."""
-
-    input: int
-    output: int
-    cached: int
-    total: int
-
-
-class LLMRequestStatsBucket(BaseModel):
-    """A single time bucket in LLM request stats."""
-
-    time: str
-    statuses: dict[str, int]
-    total: int
-    tokens: LLMRequestTokenSums
-
-
-class LLMRequestStatsResponse(BaseModel):
-    """LLM request counts and token sums grouped by time bucket."""
-
-    bank_id: str
-    period: str
-    trunc: str
-    start: str
-    buckets: list[LLMRequestStatsBucket]
-
-
-# ── recorder / writer ─────────────────────────────────────────────────────────
-
-
-class LLMTraceRecorder:
-    """GenAI span recorder that writes per-bank LLM traces to ``llm_requests``.
-
-    Implements ``record_llm_call`` so it can be registered with
-    :func:`hindsight_api.tracing.register_span_recorder`. Writes are
-    fire-and-forget and never surface errors into the calling path. Retention of
-    old rows is handled by the background :class:`MaintenanceLoop`.
-    """
-
-    def __init__(
-        self,
-        pool_getter: Callable[[], Any],
-        schema_getter: Callable[[], str],
-        enabled: bool,
-        allowed_scopes: list[str],
-        max_chars: int = 50000,
-    ) -> None:
-        self._pool_getter = pool_getter
-        self._schema_getter = schema_getter
-        self._enabled = enabled
-        self._allowed_scopes: frozenset[str] | None = frozenset(allowed_scopes) if allowed_scopes else None
-        self._max_chars = max_chars
-        # In-flight fire-and-forget write tasks, bucketed by trace_id so
-        # attach_memory_ids can await only *its own* operation's writes before the
-        # post-operation UPDATE (otherwise the UPDATE could race ahead of the
-        # INSERTs it patches — but it must not block on unrelated operations).
-        self._pending: dict[str | None, set[asyncio.Task]] = {}
-
-    def is_enabled(self, scope: str) -> bool:
-        """Whether tracing is active for the given call scope."""
-        if not self._enabled:
-            return False
-        if self._allowed_scopes is not None:
-            return scope in self._allowed_scopes
-        return True
-
-    # ── GenAI recorder interface ──────────────────────────────────────────────
-
-    def record_llm_call(
-        self,
-        provider: str,
-        model: str,
-        scope: str,
-        messages: list[dict[str, Any]],
-        response_content: Any = None,
-        input_tokens: int = 0,
-        output_tokens: int = 0,
-        duration: float = 0.0,
-        finish_reason: str | None = None,
-        error: BaseException | None = None,
-        tool_calls: list[dict[str, Any]] | None = None,
-        cached_tokens: int = 0,
-        **_extra: Any,
-    ) -> None:
-        """Build a trace record from a GenAI call and schedule a DB write."""
-        if not self.is_enabled(scope):
-            return
-
-        ctx = current_trace_context()
-        ended_at = datetime.now(timezone.utc)
-        started_at = ended_at - timedelta(seconds=max(0.0, duration))
-
-        # Operation-level metadata + any per-call metadata (e.g. document_id).
-        metadata = dict(ctx.metadata) if ctx else {}
-        call_metadata = current_call_metadata()
-        if call_metadata:
-            metadata.update(call_metadata)
-
-        llm_info: dict[str, Any] = {}
-        request_params = current_request_context()
-        if request_params:
-            llm_info["request"] = dict(request_params)
-        if finish_reason:
-            llm_info["finish_reason"] = finish_reason
-        if tool_calls:
-            llm_info["tool_calls"] = [tc.get("name", "") for tc in tool_calls]
-
-        record = LLMRequestRecord(
-            provider=provider,
-            model=model,
-            scope=scope,
-            status="error" if error is not None else "success",
-            started_at=started_at,
-            ended_at=ended_at,
-            bank_id=ctx.bank_id if ctx else None,
-            operation=ctx.operation if ctx else None,
-            # OTel-style hierarchy: all calls of one operation invocation share
-            # the context's trace_id and point at its operation span; this call
-            # gets its own span_id.
-            trace_id=ctx.trace_id if ctx else None,
-            span_id=str(uuid.uuid4()),
-            parent_span_id=ctx.operation_span_id if ctx else None,
-            input=messages,
-            output=None if error is not None else response_content,
-            error=f"{type(error).__name__}: {error}" if error is not None else None,
-            input_tokens=input_tokens or None,
-            output_tokens=output_tokens or None,
-            cached_tokens=cached_tokens or None,
-            total_tokens=(input_tokens + output_tokens) or None,
-            llm_info=llm_info,
-            metadata=metadata,
-        )
-        self._record_fire_and_forget(record)
-
-    def _record_fire_and_forget(self, record: LLMRequestRecord) -> None:
-        """Schedule a trace write as a background task."""
-        try:
-            task = asyncio.create_task(self._safe_write(record))
-        except RuntimeError:
-            # No running event loop (e.g. during shutdown)
-            logger.debug("Cannot schedule llm trace write: no running event loop")
-            return
-        key = record.trace_id
-        self._pending.setdefault(key, set()).add(task)
-        task.add_done_callback(lambda t, k=key: self._discard_pending(k, t))
-
-    def _discard_pending(self, key: str | None, task: asyncio.Task) -> None:
-        bucket = self._pending.get(key)
-        if bucket is not None:
-            bucket.discard(task)
-            if not bucket:
-                self._pending.pop(key, None)
-
-    async def _safe_write(self, record: LLMRequestRecord) -> None:
-        """Write a trace row. Errors are logged, never raised."""
-        pool = self._pool_getter()
-        if pool is None:
-            logger.debug("LLM trace skipped: pool not available")
-            return
-        try:
-            schema = self._schema_getter()
-            table = f"{schema}.llm_requests"
-            async with acquire_with_retry(pool, max_retries=1) as conn:
-                await conn.execute(
-                    f"""
-                    INSERT INTO {table}
-                        (id, bank_id, operation, scope, trace_id, span_id, parent_span_id,
-                         provider, model, status,
-                         started_at, ended_at, duration_ms,
-                         input_tokens, output_tokens, cached_tokens, total_tokens,
-                         input, output, error, llm_info, metadata)
-                    VALUES
-                        ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10,
-                         $11, $12, $13, $14, $15, $16, $17,
-                         $18::jsonb, $19::jsonb, $20, $21::jsonb, $22::jsonb)
-                    """,
-                    uuid.uuid4(),
-                    record.bank_id,
-                    record.operation,
-                    record.scope,
-                    record.trace_id,
-                    record.span_id,
-                    record.parent_span_id,
-                    record.provider,
-                    record.model,
-                    record.status,
-                    record.started_at,
-                    record.ended_at,
-                    record.duration_ms,
-                    record.input_tokens,
-                    record.output_tokens,
-                    record.cached_tokens,
-                    record.total_tokens,
-                    _safe_json(record.input, self._max_chars),
-                    _safe_json(record.output, self._max_chars),
-                    record.error,
-                    _safe_json(record.llm_info, self._max_chars) or "{}",
-                    _safe_json(record.metadata, self._max_chars) or "{}",
-                )
-        except Exception as e:
-            logger.warning(f"LLM trace write failed for scope={record.scope}: {e}")
-
-    async def _flush_pending(self, trace_id: str) -> None:
-        """Await this trace's in-flight writes so its rows exist before an UPDATE."""
-        pending = [t for t in self._pending.get(trace_id, ()) if not t.done()]
-        if pending:
-            await asyncio.gather(*pending, return_exceptions=True)
-
-    def attach_memory_ids(
-        self,
-        trace_ctx: LLMTraceContext | None,
-        *,
-        created: list[str] | None = None,
-        source: list[str] | None = None,
-    ) -> None:
-        """Map a finished operation's memory_units onto every row of its trace.
-
-        Merges the explicitly passed ids with any accumulated on the context
-        (``record_created_memory_ids`` / ``record_source_memory_ids``), de-dupes
-        preserving order, and patches ``metadata.memory_ids`` (outputs created)
-        and ``metadata.source_memory_ids`` (inputs consumed) on all rows sharing
-        the trace_id. No-op when tracing is off or nothing was produced.
-
-        Fire-and-forget: the snapshotted patch is applied on a background task so
-        the retain/consolidation operation never waits on the trace write. The
-        ids are snapshotted synchronously here because the caller may reset the
-        context immediately after.
-        """
-        if not self._enabled or trace_ctx is None or not trace_ctx.trace_id:
-            return
-        created_ids = list(dict.fromkeys([*(created or []), *trace_ctx.created_memory_ids]))
-        source_ids = list(dict.fromkeys([*(source or []), *trace_ctx.source_memory_ids]))
-        patch: dict[str, Any] = {}
-        if created_ids:
-            patch["memory_ids"] = created_ids
-        if source_ids:
-            patch["source_memory_ids"] = source_ids
-        if not patch:
-            return
-        try:
-            asyncio.create_task(self._attach_memory_ids(trace_ctx.bank_id, trace_ctx.trace_id, patch))
-        except RuntimeError:
-            logger.debug("Cannot schedule llm trace memory_id attach: no running event loop")
-
-    async def _attach_memory_ids(self, bank_id: str | None, trace_id: str, patch: dict[str, Any]) -> None:
-        """Background worker: flush this trace's writes, then patch its rows."""
-        # The trace-row INSERTs are fire-and-forget; flush *this trace's* writes
-        # so the UPDATE patches rows that already exist rather than racing ahead
-        # of them (without blocking on unrelated operations' pending writes).
-        await self._flush_pending(trace_id)
-        pool = self._pool_getter()
-        if pool is None:
-            return
-        try:
-            schema = self._schema_getter()
-            table = f"{schema}.llm_requests"
-            async with acquire_with_retry(pool, max_retries=1) as conn:
-                await conn.execute(
-                    f"UPDATE {table} SET metadata = metadata || $3::jsonb WHERE bank_id = $1 AND trace_id = $2",
-                    bank_id,
-                    trace_id,
-                    json.dumps(patch),
-                )
-        except Exception as e:
-            logger.warning(f"LLM trace memory_id attach failed for trace={trace_id}: {e}")
@@ -114,32 +114,6 @@ def _semaphores_for_scope(scope: str) -> list[asyncio.Semaphore]:
    return [per_op, _global_llm_semaphore]


-def _request_params(
-    *,
-    max_completion_tokens: int | None = None,
-    temperature: float | None = None,
-    scope: str | None = None,
-    response_format: Any | None = None,
-    tool_choice: str | dict[str, Any] | None = None,
-) -> dict[str, Any] | None:
-    """Build the requested-params bag for tracing — only values the caller set.
-
-    Omitting unset values avoids the misleading nulls we used to record (e.g.
-    consolidation, which passes no token cap), while surfacing the real cap for
-    callers that do set one (e.g. retain's ``retain_max_completion_tokens``).
-    """
-    params: dict[str, Any] = {}
-    if max_completion_tokens is not None:
-        params["max_completion_tokens"] = max_completion_tokens
-    if temperature is not None:
-        params["temperature"] = temperature
-    if response_format is not None:
-        params["response_schema"] = getattr(response_format, "__name__", None) or "structured"
-    if tool_choice is not None and tool_choice != "auto":
-        params["tool_choice"] = tool_choice if isinstance(tool_choice, str) else "named"
-    return params or None
-
-
 def sanitize_text(text: str | None) -> str | None:
    """
    Sanitize text by removing characters that break downstream systems.
@@ -255,7 +229,6 @@ def create_llm_provider(
    vertexai_region: str | None = None,
    vertexai_credentials: Any = None,
    gemini_safety_settings: list | None = None,
-    prompt_cache_enabled: bool = False,
    litellmrouter_config: dict[str, Any] | None = None,
 ) -> Any:  # Returns LLMInterface
    """
@@ -269,11 +242,7 @@ def create_llm_provider(
        reasoning_effort: Reasoning effort level for supported providers.
        groq_service_tier: Groq service tier (for Groq provider) - "on_demand", "flex", or "auto".
        openai_service_tier: OpenAI service tier (for OpenAI provider) - None (default) or "flex" (50% cheaper).
-        extra_body: Extra request-body params merged into the provider's native
-            call. Threaded into OpenAI-compatible, Fireworks, Anthropic, Gemini/
-            VertexAI and LiteLLM providers (each merges them in its own parameter
-            space). Keys must use each provider's native names (e.g. ``max_tokens``
-            for OpenAI/Anthropic vs ``max_output_tokens`` for Gemini).
+        extra_body: Extra body params merged into OpenAI-compatible API calls.
        default_headers: Custom headers passed as ``default_headers`` to provider SDK clients
            (used by operators routing through proxies / request-tracing middleware). Currently
            wired into the Anthropic provider; other providers may opt in as needed.
@@ -348,8 +317,6 @@ def create_llm_provider(
            vertexai_region=vertexai_region,
            vertexai_credentials=vertexai_credentials,
            gemini_safety_settings=gemini_safety_settings,
-            prompt_cache_enabled=prompt_cache_enabled,
-            extra_body=extra_body,
        )

    elif provider_lower == "anthropic":
@@ -360,7 +327,6 @@ def create_llm_provider(
            model=model,
            reasoning_effort=reasoning_effort,
            default_headers=default_headers,
-            extra_body=extra_body,
        )

    elif provider_lower == "litellm":
@@ -370,7 +336,6 @@ def create_llm_provider(
            base_url=base_url,
            model=model,
            reasoning_effort=reasoning_effort,
-            extra_body=extra_body,
        )

    elif provider_lower == "litellmrouter":
@@ -388,7 +353,6 @@ def create_llm_provider(
            model=model,
            config=litellmrouter_config,
            reasoning_effort=reasoning_effort,
-            extra_body=extra_body,
        )

    elif provider_lower == "bedrock":
@@ -400,7 +364,6 @@ def create_llm_provider(
            base_url=base_url,
            model=bedrock_model,
            reasoning_effort=reasoning_effort,
-            extra_body=extra_body,
        )

    elif provider_lower == "llamacpp":
@@ -479,7 +442,6 @@ class LLMProvider:
        groq_service_tier: str | None = None,
        openai_service_tier: str | None = None,
        gemini_safety_settings: list | None = None,
-        prompt_cache_enabled: bool = False,
        extra_body: dict[str, Any] | None = None,
        default_headers: dict[str, str] | None = None,
        litellmrouter_config: dict[str, Any] | None = None,
@@ -496,8 +458,7 @@ class LLMProvider:
            groq_service_tier: Groq service tier ("on_demand", "flex", "auto") - from config.
            openai_service_tier: OpenAI service tier (None or "flex") - from config.
            gemini_safety_settings: Safety settings for Gemini/VertexAI providers.
-            extra_body: Extra request-body params merged into the provider's native call
-                (OpenAI-compatible, Fireworks, Anthropic, Gemini/VertexAI, LiteLLM).
+            extra_body: Extra body params merged into OpenAI-compatible API calls.
            default_headers: Custom headers passed as ``default_headers`` to provider SDK clients.
                Used by operators routing through proxies / request-tracing middleware. Falls
                back to ``HindsightConfig.llm_default_headers`` (env: ``HINDSIGHT_API_LLM_DEFAULT_HEADERS``)
@@ -519,11 +480,6 @@ class LLMProvider:
        self.openai_service_tier = openai_service_tier
        # Gemini safety settings (instance default; can be overridden per-request via context var)
        self.gemini_safety_settings = gemini_safety_settings
-        # Gemini prompt caching: when True, retain extraction (and any future
-        # caller that opts in) will reuse a CachedContent prefix to cut
-        # input-token cost. Off by default so the change is observable behind
-        # a flip rather than a silent behaviour change on upgrade.
-        self.prompt_cache_enabled = prompt_cache_enabled
        # Extra body params for OpenAI-compatible providers (e.g. chat_template_kwargs)
        self.extra_body = extra_body
        # Default headers passed to provider SDK clients (e.g. proxy auth, request tracing).
@@ -642,21 +598,6 @@ class LLMProvider:
            except Exception:
                pass  # Config may not be initialized in test environments

-        # Prompt-prefix caching is a provider-agnostic toggle (default on): resolve
-        # it from the static server config for every provider when the caller didn't
-        # pass an explicit override. Providers that don't support caching ignore the
-        # value; only those that implement get_or_create_cached_prefix act on it.
-        if not self.prompt_cache_enabled:
-            from ..config import DEFAULT_LLM_PROMPT_CACHE_ENABLED, _get_raw_config
-
-            try:
-                raw_config = _get_raw_config()
-                self.prompt_cache_enabled = bool(
-                    getattr(raw_config, "llm_prompt_cache_enabled", DEFAULT_LLM_PROMPT_CACHE_ENABLED)
-                )
-            except Exception:
-                pass  # Config may not be initialized in test environments
-
        # For litellmrouter: prefer an explicit chain from the caller (per-op
        # construction in MemoryEngine threads the right chain through). If the caller
        # didn't supply one, fall back to the global ``llm_litellmrouter_config`` so
@@ -685,7 +626,6 @@ class LLMProvider:
            vertexai_region=vertexai_region,
            vertexai_credentials=vertexai_credentials,
            gemini_safety_settings=self.gemini_safety_settings,
-            prompt_cache_enabled=self.prompt_cache_enabled,
            litellmrouter_config=router_config,
        )

@@ -749,7 +689,6 @@ class LLMProvider:
        skip_validation: bool = False,
        strict_schema: bool = False,
        return_usage: bool = False,
-        cached_prefix: str | None = None,
    ) -> Any:
        """
        Make an LLM API call with retry logic.
@@ -764,10 +703,7 @@ class LLMProvider:
            initial_backoff: Initial backoff time in seconds.
            max_backoff: Maximum backoff time in seconds.
            skip_validation: Return raw JSON without Pydantic validation.
-            strict_schema: Per-call override requesting grammar-enforced (json_schema strict)
-                structured output instead of the soft json_object path. The server-level
-                HINDSIGHT_API_LLM_STRICT_SCHEMA flag is OR-ed in here so it applies to every call;
-                providers without a strict mode ignore it.
+            strict_schema: Use strict JSON schema enforcement (OpenAI only). Guarantees all required fields.
            return_usage: If True, return tuple (result, TokenUsage) instead of just result.

        Returns:
@@ -787,83 +723,33 @@ class LLMProvider:
        structured = "+structured" if response_format is not None else ""
        set_stage(f"llm.{self.provider}.{scope}{structured}")

-        # Resolve strict-schema once, here, rather than in each provider: the
-        # per-call argument OR the server-level HINDSIGHT_API_LLM_STRICT_SCHEMA
-        # flag. Providers with a json_schema response_format (OpenAI-compatible,
-        # LiteLLM) then grammar-enforce structured output instead of the fragile
-        # soft json_object path; Gemini already enforces its native response_schema,
-        # and providers without a strict mode simply ignore the flag.
-        from ..config import get_config
+        async with AsyncExitStack() as stack:
+            for sem in _semaphores_for_scope(scope):
+                await stack.enter_async_context(sem)

-        strict_schema = strict_schema or get_config().llm_strict_schema
-
-        # LLM call observability flows through the OTel GenAI recorder
-        # (tracing.get_span_recorder().record_llm_call). Provider implementations
-        # record successful calls; we forward failures here since they don't.
-        # The requested params are stashed in a contextvar (only what the caller
-        # actually set) so the recorder can attach them to either path.
-        from ..tracing import get_span_recorder
-        from .llm_trace import reset_request_context, set_request_context
-
-        call_start = time.monotonic()
-        request_token = set_request_context(
-            _request_params(
+            # Delegate to provider implementation
+            result = await self._provider_impl.call(
+                messages=messages,
+                response_format=response_format,
                max_completion_tokens=max_completion_tokens,
                temperature=temperature,
                scope=scope,
-                response_format=response_format,
+                max_retries=max_retries,
+                initial_backoff=initial_backoff,
+                max_backoff=max_backoff,
+                skip_validation=skip_validation,
+                strict_schema=strict_schema,
+                return_usage=return_usage,
            )
-        )
-        try:
-            async with AsyncExitStack() as stack:
-                for sem in _semaphores_for_scope(scope):
-                    await stack.enter_async_context(sem)

-                # cached_prefix is only set for providers that returned a handle
-                # from get_or_create_cached_prefix() (e.g. Gemini); it's None for
-                # the rest. Forward it only when present so providers that don't
-                # implement caching keep their call() signature untouched.
-                cache_kwarg = {"cached_prefix": cached_prefix} if cached_prefix is not None else {}
-                try:
-                    # Delegate to provider implementation
-                    result = await self._provider_impl.call(
-                        messages=messages,
-                        response_format=response_format,
-                        max_completion_tokens=max_completion_tokens,
-                        temperature=temperature,
-                        scope=scope,
-                        max_retries=max_retries,
-                        initial_backoff=initial_backoff,
-                        max_backoff=max_backoff,
-                        skip_validation=skip_validation,
-                        strict_schema=strict_schema,
-                        return_usage=return_usage,
-                        **cache_kwarg,
-                    )
-                except Exception as e:
-                    get_span_recorder().record_llm_call(
-                        provider=self.provider,
-                        model=self.model,
-                        scope=scope,
-                        messages=messages,
-                        response_content=None,
-                        input_tokens=0,
-                        output_tokens=0,
-                        duration=time.monotonic() - call_start,
-                        error=e,
-                    )
-                    raise
+            # Backward compatibility: Update mock call tracking for mock provider
+            # This allows existing tests using LLMProvider._mock_calls to continue working
+            if self.provider == "mock":
+                from .providers.mock_llm import MockLLM

-                # Backward compatibility: Update mock call tracking for mock provider
-                # This allows existing tests using LLMProvider._mock_calls to continue working
-                if self.provider == "mock":
-                    from .providers.mock_llm import MockLLM
-
-                    if isinstance(self._provider_impl, MockLLM):
-                        # Sync the mock calls from provider implementation to wrapper
-                        self._mock_calls = self._provider_impl.get_mock_calls()
-        finally:
-            reset_request_context(request_token)
+                if isinstance(self._provider_impl, MockLLM):
+                    # Sync the mock calls from provider implementation to wrapper
+                    self._mock_calls = self._provider_impl.get_mock_calls()

        return result

@@ -878,7 +764,6 @@ class LLMProvider:
        initial_backoff: float = 1.0,
        max_backoff: float = 30.0,
        tool_choice: str | dict[str, Any] = "auto",
-        cached_prefix: str | None = None,
    ) -> "LLMToolCallResult":
        """
        Make an LLM API call with tool/function calling support.
@@ -901,66 +786,31 @@ class LLMProvider:

        set_stage(f"llm.{self.provider}.{scope}+tools")

-        # Failures forwarded to the GenAI recorder; successes recorded by providers.
-        from ..tracing import get_span_recorder
-        from .llm_trace import reset_request_context, set_request_context
+        async with AsyncExitStack() as stack:
+            for sem in _semaphores_for_scope(scope):
+                await stack.enter_async_context(sem)

-        call_start = time.monotonic()
-        request_token = set_request_context(
-            _request_params(
+            # Delegate to provider implementation
+            result = await self._provider_impl.call_with_tools(
+                messages=messages,
+                tools=tools,
                max_completion_tokens=max_completion_tokens,
                temperature=temperature,
                scope=scope,
+                max_retries=max_retries,
+                initial_backoff=initial_backoff,
+                max_backoff=max_backoff,
                tool_choice=tool_choice,
            )
-        )
-        try:
-            async with AsyncExitStack() as stack:
-                for sem in _semaphores_for_scope(scope):
-                    await stack.enter_async_context(sem)

-                # cached_prefix is only set for providers that returned a handle
-                # from get_or_create_cached_prefix(); forward it only when present
-                # so non-caching providers keep their signature (same as call()).
-                cache_kwarg = {"cached_prefix": cached_prefix} if cached_prefix is not None else {}
-                try:
-                    # Delegate to provider implementation
-                    result = await self._provider_impl.call_with_tools(
-                        messages=messages,
-                        tools=tools,
-                        max_completion_tokens=max_completion_tokens,
-                        temperature=temperature,
-                        scope=scope,
-                        max_retries=max_retries,
-                        initial_backoff=initial_backoff,
-                        max_backoff=max_backoff,
-                        tool_choice=tool_choice,
-                        **cache_kwarg,
-                    )
-                except Exception as e:
-                    get_span_recorder().record_llm_call(
-                        provider=self.provider,
-                        model=self.model,
-                        scope=scope,
-                        messages=messages,
-                        response_content=None,
-                        input_tokens=0,
-                        output_tokens=0,
-                        duration=time.monotonic() - call_start,
-                        error=e,
-                    )
-                    raise
+            # Backward compatibility: Update mock call tracking for mock provider
+            # This allows existing tests using LLMProvider._mock_calls to continue working
+            if self.provider == "mock":
+                from .providers.mock_llm import MockLLM

-                # Backward compatibility: Update mock call tracking for mock provider
-                # This allows existing tests using LLMProvider._mock_calls to continue working
-                if self.provider == "mock":
-                    from .providers.mock_llm import MockLLM
-
-                    if isinstance(self._provider_impl, MockLLM):
-                        # Sync the mock calls from provider implementation to wrapper
-                        self._mock_calls = self._provider_impl.get_mock_calls()
-        finally:
-            reset_request_context(request_token)
+                if isinstance(self._provider_impl, MockLLM):
+                    # Sync the mock calls from provider implementation to wrapper
+                    self._mock_calls = self._provider_impl.get_mock_calls()

        return result

@@ -1064,14 +914,7 @@ class LLMProvider:
        # SDK will automatically check for authentication when first used
        # No need to verify here - let it fail gracefully on first call with helpful error

-    def with_config(
-        self,
-        config: Any,
-        *,
-        bank_id: str | None = None,
-        operation: str | None = None,
-        metadata: dict[str, Any] | None = None,
-    ) -> "ConfiguredLLMProvider":
+    def with_config(self, config: Any) -> "ConfiguredLLMProvider":
        """
        Return a configured wrapper for a specific bank operation.

@@ -1081,31 +924,12 @@ class LLMProvider:

        Args:
            config: Resolved ``HindsightConfig`` for the current bank/request.
-            bank_id: Bank the operation runs for; attributed to LLM trace rows.
-            operation: Logical operation label ("retain", "reflect", ...) for
-                LLM trace rows.
-            metadata: Optional extra caller metadata stored on trace rows.

        Returns:
            A ``ConfiguredLLMProvider`` that delegates to this provider with
            the supplied config applied.
        """
-        trace_ctx = None
-        if bank_id is not None or operation is not None or metadata:
-            from .llm_trace import LLMTraceContext
-
-            # One trace + operation span per with_config() call — i.e. per
-            # operation invocation. Every LLM call made through this wrapper
-            # shares them, so a reflect/retain/consolidation run groups its
-            # calls as parent (operation) → children (LLM calls).
-            trace_ctx = LLMTraceContext(
-                bank_id=bank_id,
-                operation=operation,
-                metadata=dict(metadata or {}),
-                trace_id=str(uuid.uuid4()),
-                operation_span_id=str(uuid.uuid4()),
-            )
-        return ConfiguredLLMProvider(self, config.llm_gemini_safety_settings, trace_ctx)
+        return ConfiguredLLMProvider(self, config.llm_gemini_safety_settings)

    async def cleanup(self) -> None:
        """Clean up resources (e.g. stop llamacpp subprocess)."""
@@ -1169,16 +993,10 @@ class ConfiguredLLMProvider:
    any changes.
    """

-    def __init__(
-        self,
-        provider: "LLMProvider",
-        gemini_safety_settings: list | None,
-        trace_ctx: Any | None = None,
-    ) -> None:
+    def __init__(self, provider: "LLMProvider", gemini_safety_settings: list | None) -> None:
        # Use object.__setattr__ to avoid triggering __getattr__
        object.__setattr__(self, "_provider", provider)
        object.__setattr__(self, "_gemini_safety_settings", gemini_safety_settings)
-        object.__setattr__(self, "_trace_ctx", trace_ctx)

    # ── attribute passthrough ──────────────────────────────────────────────────

@@ -1191,12 +1009,10 @@ class ConfiguredLLMProvider:
        from .providers.gemini_llm import _safety_settings_ctx

        token = _safety_settings_ctx.set(object.__getattribute__(self, "_gemini_safety_settings"))
-        trace_token = self._bind_trace_context()
        try:
            return await object.__getattribute__(self, "_provider").call(messages=messages, **kwargs)
        finally:
            _safety_settings_ctx.reset(token)
-            self._reset_trace_context(trace_token)

    async def call_with_tools(
        self,
@@ -1207,38 +1023,12 @@ class ConfiguredLLMProvider:
        from .providers.gemini_llm import _safety_settings_ctx

        token = _safety_settings_ctx.set(object.__getattribute__(self, "_gemini_safety_settings"))
-        trace_token = self._bind_trace_context()
        try:
            return await object.__getattribute__(self, "_provider").call_with_tools(
                messages=messages, tools=tools, **kwargs
            )
        finally:
            _safety_settings_ctx.reset(token)
-            self._reset_trace_context(trace_token)
-
-    def trace_context(self) -> Any | None:
-        """The operation-level LLM trace context (or None when untraced).
-
-        Lets the engine attach the operation's produced/consumed memory_ids to
-        this run's trace rows once they're known (after the LLM calls).
-        """
-        return object.__getattribute__(self, "_trace_ctx")
-
-    def _bind_trace_context(self) -> Any | None:
-        """Bind bank/operation attribution for the duration of one call."""
-        trace_ctx = object.__getattribute__(self, "_trace_ctx")
-        if trace_ctx is None:
-            return None
-        from .llm_trace import set_trace_context
-
-        return set_trace_context(trace_ctx)
-
-    def _reset_trace_context(self, trace_token: Any | None) -> None:
-        if trace_token is None:
-            return
-        from .llm_trace import reset_trace_context
-
-        reset_trace_context(trace_token)


 # Backwards compatibility alias
@@ -1,214 +0,0 @@
-"""Background maintenance loop.
-
-A single periodic loop that drives all of Hindsight's recurring housekeeping
-from one place, so we don't spawn a separate ``asyncio`` task per concern:
-
- **Retention sweeps** (hourly): delete ``audit_log`` and ``llm_requests`` rows
-  older than their configured retention, across *all* tenant schemas.
- **Consolidation reconcile** (configurable, default 5 min): re-schedule
-  consolidation for banks that have eligible-but-unscheduled facts and no
-  in-flight consolidation. This recovers facts that were stranded when a
-  consolidation operation failed terminally and left them with
-  ``consolidated_at IS NULL AND consolidation_failed_at IS NULL`` and nothing to
-  re-trigger them.
-
-The loop wakes on a short fixed tick and runs each job when its own
-``last_run + interval`` is due (run-at-start, then on interval), so adding jobs
-with different cadences doesn't burst CPU. Cross-tenant discovery goes through
-server-side PL/pgSQL routines (``public.schemas_with_expired_rows`` and
-``public.banks_needing_consolidation``) — one round-trip each — instead of a
-per-schema query storm, which matters at thousands of tenants.
-"""
-
-from __future__ import annotations
-
-import asyncio
-import logging
-import time
-from typing import TYPE_CHECKING
-
-from ..config import HindsightConfig, get_config
-from ..models import RequestContext
-from .db_utils import acquire_with_retry
-from .schema import _is_oracle
-
-if TYPE_CHECKING:
-    from .memory_engine import MemoryEngine
-
-logger = logging.getLogger(__name__)
-
-# Short tick so jobs with different cadences share one loop without per-job tasks.
-_TICK_SECONDS = 60
-# Retention sweeps are not time-sensitive; hourly matches the previous per-sweep cadence.
-_RETENTION_INTERVAL_SECONDS = 3600
-
-
-class MaintenanceLoop:
-    """Owns the single periodic maintenance task for a :class:`MemoryEngine`."""
-
-    def __init__(self, engine: "MemoryEngine") -> None:
-        self._engine = engine
-        self._task: asyncio.Task | None = None
-        self._stop = asyncio.Event()
-        # Monotonic timestamps of the last run per job, keyed by job name.
-        self._last_run: dict[str, float] = {}
-
-    # ── lifecycle ──────────────────────────────────────────────────────────
-
-    def start(self) -> None:
-        """Start the loop if any maintenance job is enabled. Idempotent."""
-        if self._task and not self._task.done():
-            return
-        # PostgreSQL-only: the retention sweeps target PG-only tables (audit_log,
-        # llm_requests) and the reconcile relies on PG-only PL/pgSQL routines
-        # installed by the maintenance-routines migration. Oracle support is
-        # intentionally absent (mirrors that PG-only migration).
-        if _is_oracle():
-            logger.debug("Maintenance loop not started: PostgreSQL-only")
-            return
-        if not self._any_job_enabled():
-            logger.debug("Maintenance loop not started: no jobs enabled")
-            return
-        self._stop.clear()
-        try:
-            self._task = asyncio.create_task(self._run())
-        except RuntimeError:
-            logger.debug("Cannot start maintenance loop: no running event loop")
-
-    async def stop(self) -> None:
-        """Stop the loop and wait for the current tick to finish."""
-        self._stop.set()
-        if self._task and not self._task.done():
-            try:
-                await self._task
-            except asyncio.CancelledError:
-                pass
-        self._task = None
-
-    @staticmethod
-    def _any_job_enabled() -> bool:
-        cfg = get_config()
-        reconcile_on = cfg.consolidation_reconcile_interval_seconds > 0
-        audit_on = cfg.audit_log_enabled and cfg.audit_log_retention_days > 0
-        llm_on = cfg.llm_trace_enabled and cfg.llm_trace_retention_days > 0
-        return reconcile_on or audit_on or llm_on
-
-    # ── loop ───────────────────────────────────────────────────────────────
-
-    async def _run(self) -> None:
-        while not self._stop.is_set():
-            try:
-                await self._tick()
-            except Exception:
-                logger.exception("Maintenance tick failed")
-            try:
-                await asyncio.wait_for(self._stop.wait(), timeout=_TICK_SECONDS)
-            except asyncio.TimeoutError:
-                pass
-
-    def _is_due(self, job: str, interval_seconds: int) -> bool:
-        """True if ``job`` has never run or its interval has elapsed; marks it run now."""
-        now = time.monotonic()
-        last = self._last_run.get(job)
-        if last is not None and (now - last) < interval_seconds:
-            return False
-        self._last_run[job] = now
-        return True
-
-    async def _tick(self) -> None:
-        cfg = get_config()
-        if self._is_due("retention", _RETENTION_INTERVAL_SECONDS):
-            await self._run_retention(cfg)
-        interval = cfg.consolidation_reconcile_interval_seconds
-        if interval > 0 and self._is_due("reconcile", interval):
-            await self._run_reconcile()
-
-    # ── retention ──────────────────────────────────────────────────────────
-
-    async def _run_retention(self, cfg: HindsightConfig) -> None:
-        # Retention days are static server-level config, so one global cutoff
-        # applies to every tenant schema (the routine sweeps them all).
-        if cfg.audit_log_enabled and cfg.audit_log_retention_days > 0:
-            await self._purge_expired("audit_log", "started_at", cfg.audit_log_retention_days)
-        if cfg.llm_trace_enabled and cfg.llm_trace_retention_days > 0:
-            await self._purge_expired("llm_requests", "started_at", cfg.llm_trace_retention_days)
-
-    async def _purge_expired(self, table: str, ts_col: str, days: int) -> None:
-        """Delete rows older than ``days`` from ``table`` across every tenant schema."""
-        backend = self._engine._backend
-        try:
-            async with acquire_with_retry(backend, max_retries=1) as conn:
-                rows = await conn.fetch(
-                    "SELECT * FROM public.schemas_with_expired_rows($1, $2, $3)", table, ts_col, days
-                )
-                for row in rows:
-                    schema = row[0]
-                    # schema names come from pg_class; quote defensively all the same.
-                    qschema = '"' + schema.replace('"', '""') + '"'
-                    result = await conn.execute(
-                        f"DELETE FROM {qschema}.{table} WHERE {ts_col} < NOW() - make_interval(days => $1)",
-                        days,
-                    )
-                    if result and result != "DELETE 0":
-                        logger.info(f"Retention sweep {schema}.{table}: {result}")
-        except Exception as e:
-            logger.warning(f"Retention sweep failed for {table}: {e}")
-
-    # ── consolidation reconcile ──────────────────────────────────────────────
-
-    async def _run_reconcile(self) -> None:
-        """Re-schedule consolidation for banks with eligible-but-unscheduled facts."""
-        engine = self._engine
-        try:
-            async with acquire_with_retry(engine._backend, max_retries=1) as conn:
-                rows = await conn.fetch("SELECT schema_name, bank_id FROM public.banks_needing_consolidation()")
-        except Exception as e:
-            logger.warning(f"Consolidation reconcile discovery failed: {e}")
-            return
-        if not rows:
-            return
-
-        # Only enqueue into schemas the worker actually polls (tenant discovery),
-        # otherwise the op would never be claimed and would block future reconciles
-        # for that bank. The tenant_id (when the extension provides one) lets
-        # config resolution honor tenant-level overrides.
-        try:
-            tenants = await engine._tenant_extension.list_tenants()
-        except Exception as e:
-            logger.warning(f"Consolidation reconcile tenant discovery failed: {e}")
-            return
-        tenant_by_schema = {t.schema: t for t in tenants}
-        default_schema = get_config().database_schema
-
-        from .memory_engine import _current_schema
-
-        submitted = 0
-        skipped_unknown = 0
-        for row in rows:
-            schema = row["schema_name"]
-            bank_id = row["bank_id"]
-            tenant = tenant_by_schema.get(schema)
-            if tenant is None and schema != default_schema:
-                skipped_unknown += 1
-                continue
-            tenant_id = tenant.tenant_id if tenant else None
-            token = _current_schema.set(schema)
-            try:
-                context = RequestContext(internal=True, tenant_id=tenant_id)
-                resolved = await engine._config_resolver.resolve_full_config(bank_id, context)
-                # Mirror the retain-time auto-consolidation gate (memory_engine): both
-                # observations and auto-consolidation must be enabled for this bank.
-                if not (resolved.enable_observations and resolved.enable_auto_consolidation):
-                    continue
-                await engine.submit_async_consolidation(bank_id=bank_id, request_context=context)
-                submitted += 1
-            except Exception as e:
-                logger.warning(f"Consolidation reconcile failed for bank {bank_id} in {schema}: {e}")
-            finally:
-                _current_schema.reset(token)
-
-        if submitted or skipped_unknown:
-            logger.info(
-                f"Consolidation reconcile: scheduled {submitted} bank(s)"
-                + (f", skipped {skipped_unknown} in unrecognized schema(s)" if skipped_unknown else "")
-            )
@@ -5,10 +5,8 @@ These dataclasses define the structure of result_metadata for different operatio
 The metadata is exposed in the API for debugging purposes and may change without notice.
 """

-from dataclasses import asdict, dataclass, field
-from typing import Any, Mapping
-
-MAX_EXTRACTION_ERROR_SAMPLES = 5
+from dataclasses import asdict, dataclass
+from typing import Any


@dataclass
@@ -50,79 +48,6 @@ class RetainMetadata:
        return asdict(self)


-@dataclass
-class RetainExtractionErrors:
-    """Non-fatal fact extraction failures observed inside one retain operation."""
-
-    count: int = 0
-    sample: list[str] = field(default_factory=list)
-
-    def add(self, message: str) -> None:
-        """Record one extraction error while keeping the stored sample bounded."""
-        self.count += 1
-        if len(self.sample) < MAX_EXTRACTION_ERROR_SAMPLES:
-            self.sample.append(message[:500])
-
-    def merge_metadata(self, metadata: Mapping[str, Any]) -> None:
-        """Merge errors already present on an operation result_metadata object."""
-        self.count += int(metadata.get("extraction_errors_count") or 0)
-
-        sample = metadata.get("extraction_errors_sample") or []
-        if isinstance(sample, str):
-            sample = [sample]
-        if isinstance(sample, list):
-            for entry in sample:
-                if isinstance(entry, str) and len(self.sample) < MAX_EXTRACTION_ERROR_SAMPLES:
-                    self.sample.append(entry[:500])
-
-    def to_dict(self) -> dict[str, Any]:
-        """Convert to the public result_metadata field shape."""
-        data: dict[str, Any] = {"extraction_errors_count": self.count}
-        if self.sample:
-            data["extraction_errors_sample"] = self.sample
-        return data
-
-
-@dataclass
-class RetainOutcomeMetadata:
-    """Machine-readable outcome metadata for a completed retain operation."""
-
-    unit_ids_count: int
-    extraction_errors_count: int = 0
-    extraction_errors_sample: list[str] = field(default_factory=list)
-
-    def to_dict(self) -> dict[str, Any]:
-        """Convert to dict for JSON serialization, omitting empty optional samples."""
-        data: dict[str, Any] = {
-            "unit_ids_count": self.unit_ids_count,
-            "extraction_errors_count": self.extraction_errors_count,
-        }
-        if self.extraction_errors_sample:
-            data["extraction_errors_sample"] = self.extraction_errors_sample[:MAX_EXTRACTION_ERROR_SAMPLES]
-        return data
-
-
-@dataclass
-class RetainOutcomeAggregate:
-    """Aggregate retain outcome metadata from child retain operations."""
-
-    unit_ids_count: int = 0
-    extraction_errors: RetainExtractionErrors = field(default_factory=RetainExtractionErrors)
-
-    def add_metadata(self, metadata: Mapping[str, Any]) -> None:
-        """Fold one child operation's result_metadata into the aggregate."""
-        self.unit_ids_count += int(metadata.get("unit_ids_count") or 0)
-        self.extraction_errors.merge_metadata(metadata)
-
-    def to_outcome_metadata(self) -> RetainOutcomeMetadata:
-        """Return the aggregate in the public result_metadata field shape."""
-        return RetainOutcomeMetadata(
-            unit_ids_count=self.unit_ids_count,
-            extraction_errors_count=self.extraction_errors.count,
-            extraction_errors_sample=self.extraction_errors.sample,
-        )
-
-
@dataclass
 class ConsolidationMetadata:
    """Metadata for consolidation operations."""
@@ -38,7 +38,6 @@ class AnthropicLLM(LLMInterface):
        reasoning_effort: str = "low",
        timeout: float = 300.0,
        default_headers: dict[str, str] | None = None,
-        extra_body: dict[str, Any] | None = None,
        **kwargs: Any,
    ):
        """
@@ -55,10 +54,6 @@ class AnthropicLLM(LLMInterface):
                the Anthropic SDK client. Used by operators routing through proxies
                or request-tracing middleware. Sourced from ``llm_default_headers`` in
                ``HindsightConfig`` (env: ``HINDSIGHT_API_LLM_DEFAULT_HEADERS``).
-            extra_body: Extra request-body params (e.g. ``{"temperature": 0.2,
-                "top_p": 0.9, "top_k": 40}``) passed via the Anthropic SDK's
-                ``extra_body`` so they merge into the JSON sent to the Messages API.
-                Sourced from ``llm_extra_body`` (env: ``HINDSIGHT_API_LLM_EXTRA_BODY``).
            **kwargs: Additional provider-specific parameters.
        """
        super().__init__(provider, api_key, base_url, model, reasoning_effort, **kwargs)
@@ -66,9 +61,6 @@ class AnthropicLLM(LLMInterface):
        if not self.api_key:
            raise ValueError("API key is required for Anthropic provider")

-        # User-configured extra body params (merged into every Messages API call)
-        self._extra_body = extra_body or {}
-
        # Import and initialize Anthropic client
        try:
            from anthropic import AsyncAnthropic
@@ -186,9 +178,6 @@ class AnthropicLLM(LLMInterface):
        if system_prompt:
            call_params["system"] = system_prompt

-        if self._extra_body:
-            call_params["extra_body"] = self._extra_body
-
        last_exception = None

        for attempt in range(max_retries + 1):
@@ -227,7 +216,6 @@ class AnthropicLLM(LLMInterface):
                input_tokens = response.usage.input_tokens or 0 if response.usage else 0
                output_tokens = response.usage.output_tokens or 0 if response.usage else 0
                total_tokens = input_tokens + output_tokens
-                cached_tokens = getattr(response.usage, "cache_read_input_tokens", 0) or 0 if response.usage else 0

                # Record LLM metrics
                metrics = get_metrics_collector()
@@ -257,7 +245,6 @@ class AnthropicLLM(LLMInterface):
                    duration=duration,
                    finish_reason=finish_reason,
                    error=None,
-                    cached_tokens=cached_tokens,
                )

                # Log slow calls
@@ -273,7 +260,6 @@ class AnthropicLLM(LLMInterface):
                        input_tokens=input_tokens,
                        output_tokens=output_tokens,
                        total_tokens=total_tokens,
-                        cached_tokens=cached_tokens,
                    )
                    return result, token_usage
                return result
@@ -408,9 +394,6 @@ class AnthropicLLM(LLMInterface):
        if system_prompt:
            call_params["system"] = system_prompt

-        if self._extra_body:
-            call_params["extra_body"] = self._extra_body
-
        last_exception = None
        for attempt in range(max_retries + 1):
            try:
@@ -1,316 +0,0 @@
-"""Gemini context-cache manager.
-
-Wraps the ``google-genai`` SDK's CachedContent API to let callers reuse a
-stable system_instruction + response_schema prefix across many requests.
-
-Cached input tokens are billed at ~10× lower than fresh input tokens
-(check the current Gemini pricing for the exact ratio per model), so for
-workloads that repeatedly send a large fixed prefix with a small variable
-user message — fact extraction, structured tagging, classification — the
-input-cost savings are substantial.
-
-This module owns only the create/refresh/lookup lifecycle. It is up to
-the caller to (a) decide that the prefix is stable enough to cache, and
-(b) pass the returned cache name to ``GeminiLLM.call()``. When the
-returned name is ``None`` (because Gemini rejected the create — most
-commonly because the prefix is smaller than the model's minimum), the
-caller MUST fall back to a non-cached call.
-
-Cardinality
-----------
-The intended cache count per process is small (≲100 entries). Each
-entry corresponds to one combination of (model, system_instruction,
-response_schema). If a caller sees the cache grow unboundedly it
-indicates the system_instruction contains per-request data that should
-move into the user message instead.
-
-TTL
---
-Gemini's CachedContent has a TTL bounded by the model (currently 1h
-for most generally-available models). This manager refreshes proactively
-at ``ttl_safety_margin`` before expiry. If a cached entry has expired
-between refreshes the next call will recreate it transparently.
-"""
-
-from __future__ import annotations
-
-import asyncio
-import hashlib
-import json
-import logging
-import time
-from dataclasses import dataclass
-from typing import Any
-
-logger = logging.getLogger(__name__)
-
-
-# Default TTL: 55 minutes. Gemini's hard max for CachedContent is 1 hour
-# for most models; we refresh 5 minutes early so a request landing right
-# at the boundary doesn't race against expiry.
-_DEFAULT_TTL_SECONDS = 55 * 60
-_DEFAULT_REFRESH_MARGIN_SECONDS = 5 * 60
-# Cap on the cache-create network call. It runs while holding the manager lock, so
-# a hung create would block every concurrent caller (e.g. all chunks of a 10-chunk
-# retain batch waiting on the cold-start create). On timeout the create soft-fails
-# to None and callers proceed uncached, rather than stalling the whole batch.
-_DEFAULT_CREATE_TIMEOUT_SECONDS = 30.0
-
-
-@dataclass
-class _CacheEntry:
-    name: str  # The CachedContent resource name returned by Gemini.
-    created_at: float
-    ttl_seconds: int
-
-
-class GeminiCacheManager:
-    """Per-process map of (prefix fingerprint) → CachedContent name.
-
-    Thread-safe across asyncio tasks via a single ``asyncio.Lock``. The
-    create/refresh calls are serialised; this is fine because cache
-    creation is a one-shot warm-up per fingerprint (subsequent reads are
-    pure dict lookups outside the lock).
-
-    Not shared across pods — each worker / api replica builds its own
-    cache. The cost of cold-starting one extra full-price call per pod
-    per fingerprint per hour is negligible compared to the steady-state
-    savings.
-    """
-
-    def __init__(
-        self,
-        client: Any,
-        *,
-        ttl_seconds: int = _DEFAULT_TTL_SECONDS,
-        refresh_margin_seconds: int = _DEFAULT_REFRESH_MARGIN_SECONDS,
-        create_timeout_seconds: float = _DEFAULT_CREATE_TIMEOUT_SECONDS,
-    ) -> None:
-        self._client = client
-        self._ttl_seconds = ttl_seconds
-        self._refresh_margin_seconds = refresh_margin_seconds
-        self._create_timeout_seconds = create_timeout_seconds
-        self._entries: dict[str, _CacheEntry] = {}
-        self._lock = asyncio.Lock()
-
-    @staticmethod
-    def fingerprint(
-        model: str,
-        system_instruction: str,
-        response_schema: Any | None = None,
-        tools: list[dict[str, Any]] | None = None,
-    ) -> str:
-        """Stable hash of the cacheable surface.
-
-        ``response_schema`` may be a Pydantic class, a dict, or ``None``.
-        Pydantic schemas are normalised by serialising via
-        ``model_json_schema()`` and stripping the auto-generated
-        ``"title"`` fields so two dynamically-built models with the same
-        shape but different class names hash identically. This matters
-        for callers (e.g. fact extraction) that rebuild the schema
-        class on every request via a builder helper — without the
-        normalisation the cache would never hit.
-
-        ``tools`` is the OpenAI-style tools list (each entry has a
-        ``"function"`` dict with name/description/parameters). When
-        supplied, the tool definitions become part of the cache key so a
-        loop that adds or renames a tool gets a fresh cache and doesn't
-        silently use a stale schema. Tools are serialised with
-        ``sort_keys=True`` to neutralise dict-ordering drift.
-        """
-        hasher = hashlib.sha256()
-        hasher.update(model.encode("utf-8"))
-        hasher.update(b"\x00")
-        hasher.update(system_instruction.encode("utf-8"))
-        hasher.update(b"\x00")
-        if response_schema is None:
-            hasher.update(b"none")
-        elif hasattr(response_schema, "model_json_schema"):
-            try:
-                schema = response_schema.model_json_schema()
-                _strip_titles(schema)
-                hasher.update(json.dumps(schema, sort_keys=True).encode("utf-8"))
-            except Exception:
-                # Fall back to class identity if the schema can't be serialised.
-                hasher.update(repr(response_schema).encode("utf-8"))
-        else:
-            try:
-                hasher.update(json.dumps(response_schema, sort_keys=True).encode("utf-8"))
-            except (TypeError, ValueError):
-                hasher.update(repr(response_schema).encode("utf-8"))
-        hasher.update(b"\x00")
-        if tools:
-            try:
-                hasher.update(json.dumps(tools, sort_keys=True).encode("utf-8"))
-            except (TypeError, ValueError):
-                hasher.update(repr(tools).encode("utf-8"))
-        else:
-            hasher.update(b"no-tools")
-        return hasher.hexdigest()
-
-    async def get_or_create(
-        self,
-        *,
-        model: str,
-        system_instruction: str,
-        response_schema: Any | None = None,
-        tools: list[dict[str, Any]] | None = None,
-    ) -> str | None:
-        """Return a CachedContent resource name for the given prefix, or
-        ``None`` if Gemini rejects the create (prefix too small, model
-        does not support caching, etc.).
-
-        ``tools`` is the OpenAI-style tools list. When supplied, the tool
-        definitions are baked into the CachedContent so the caller's
-        ``call_with_tools`` doesn't need to resend them on every
-        iteration. Pass ``None`` for non-tool calls.
-
-        ``None`` return is a normal, expected value — the caller falls
-        back to an uncached call and the system continues to work.
-        """
-        key = self.fingerprint(model, system_instruction, response_schema, tools)
-
-        async with self._lock:
-            entry = self._entries.get(key)
-            if entry is not None and self._is_fresh(entry):
-                return entry.name
-
-            # Need to (re)create. Pop the stale entry first so a failed
-            # create doesn't leave a name we'd return on the next call.
-            self._entries.pop(key, None)
-
-            try:
-                cache_name = await self._create_cache(
-                    model=model,
-                    system_instruction=system_instruction,
-                    tools=tools,
-                )
-            except _CacheNotEligible as e:
-                logger.debug(
-                    "GeminiCacheManager: prefix not eligible for caching (model=%s, reason=%s) — caller will fall back",
-                    model,
-                    e,
-                )
-                return None
-            except Exception:
-                logger.exception(
-                    "GeminiCacheManager: failed to create cached content "
-                    "(model=%s); caller will fall back to uncached call",
-                    model,
-                )
-                return None
-
-            if cache_name is None:
-                return None
-
-            self._entries[key] = _CacheEntry(
-                name=cache_name,
-                created_at=time.monotonic(),
-                ttl_seconds=self._ttl_seconds,
-            )
-            return cache_name
-
-    def _is_fresh(self, entry: _CacheEntry) -> bool:
-        """An entry is fresh if it's young enough that the next request
-        won't race against the TTL expiry."""
-        age = time.monotonic() - entry.created_at
-        return age < (entry.ttl_seconds - self._refresh_margin_seconds)
-
-    def invalidate(self, name: str) -> None:
-        """Forget a cache name that the server rejected (expired/deleted/invalid).
-
-        Called by the provider when a generate request using this CachedContent
-        fails, so the next ``get_or_create`` recreates it instead of handing back
-        the dead name again. Best-effort and sync — drops the matching entry from
-        the in-process map; the orphaned server-side cache (if any) ages out on
-        its own TTL.
-        """
-        for key, entry in list(self._entries.items()):
-            if entry.name == name:
-                self._entries.pop(key, None)
-
-    async def _create_cache(
-        self,
-        *,
-        model: str,
-        system_instruction: str,
-        tools: list[dict[str, Any]] | None = None,
-    ) -> str | None:
-        """Wrap ``client.aio.caches.create`` with the config we want.
-
-        The SDK surface differs slightly across google-genai versions;
-        this implementation targets the >=1.0.0 line where caches live
-        under ``client.aio.caches``.
-        """
-        # Lazy import so this module doesn't require the SDK at import time.
-        from google.genai import types as genai_types
-
-        # A CachedContent only holds reusable *input* — system_instruction,
-        # contents, tools, ttl. ``response_schema``/``response_mime_type`` are
-        # generation-time output constraints and the SDK rejects them here
-        # (``CreateCachedContentConfig`` forbids those fields). They are applied
-        # per-request on the GenerateContentConfig instead — see the call sites,
-        # which set them alongside ``cached_content``. ``response_schema`` is
-        # still part of the fingerprint so a schema change keys a fresh cache.
-        config_kwargs: dict[str, Any] = {
-            "system_instruction": system_instruction,
-            "ttl": f"{self._ttl_seconds}s",
-        }
-        if tools:
-            # OpenAI-style {"function": {...}} entries must be converted to
-            # Gemini's Tool/FunctionDeclaration shape before caching.
-            gemini_tools = []
-            for tool in tools:
-                func = tool.get("function", {})
-                gemini_tools.append(
-                    genai_types.Tool(
-                        function_declarations=[
-                            genai_types.FunctionDeclaration(
-                                name=func.get("name", ""),
-                                description=func.get("description", ""),
-                                parameters=func.get("parameters"),
-                            )
-                        ]
-                    )
-                )
-            config_kwargs["tools"] = gemini_tools
-
-        try:
-            cached = await asyncio.wait_for(
-                self._client.aio.caches.create(
-                    model=model,
-                    config=genai_types.CreateCachedContentConfig(**config_kwargs),
-                ),
-                timeout=self._create_timeout_seconds,
-            )
-        except Exception as e:
-            # Gemini returns a 400 with a "minimum token count" message
-            # when the prefix is too small. We treat this as a soft
-            # "not eligible" signal rather than a real error so callers
-            # silently fall back to non-cached.
-            msg = str(e).lower()
-            if "minimum" in msg or "too small" in msg or "too short" in msg:
-                raise _CacheNotEligible(str(e)) from e
-            raise
-
-        return getattr(cached, "name", None)
-
-
-class _CacheNotEligible(Exception):
-    """Raised when Gemini rejects the cache create because the prefix
-    is below the model's minimum cacheable size. Treated as a soft
-    fallback by the caller, not an error."""
-
-
-def _strip_titles(node: Any) -> None:
-    """Recursively remove auto-generated ``"title"`` keys from a JSON
-    Schema-like dict tree, in place. Pydantic seeds these from the
-    Python class name, which means structurally-identical schemas built
-    from differently-named classes look distinct to a naive hash."""
-    if isinstance(node, dict):
-        node.pop("title", None)
-        for v in node.values():
-            _strip_titles(v)
-    elif isinstance(node, list):
-        for item in node:
-            _strip_titles(item)
@@ -70,22 +70,6 @@ class GeminiLLM(LLMInterface):
        # Safety settings: None means use Gemini's defaults
        self._safety_settings: list | None = kwargs.get("gemini_safety_settings")

-        # User-configured extra params merged into the GenerateContentConfig of
-        # every call. Gemini's request body nests generation params, so we expose
-        # them in the SDK's native config space rather than as a raw body merge:
-        # keys must be GenerateContentConfig fields (e.g. temperature, top_p,
-        # top_k, max_output_tokens, seed). Sourced from llm_extra_body
-        # (env: HINDSIGHT_API_LLM_EXTRA_BODY).
-        self._extra_body: dict[str, Any] = kwargs.get("extra_body") or {}
-
-        # Context-cache manager. Lazy-initialized on first cache lookup so
-        # nothing happens for models/workloads that never reach it. The instance
-        # default here is off (a directly-constructed GeminiLLM doesn't cache); the
-        # server-level default is on and flows in via the prompt_cache_enabled kwarg
-        # resolved from config in LLMProvider.
-        self._cache_manager: Any | None = None
-        self._prompt_cache_enabled: bool = bool(kwargs.get("prompt_cache_enabled", False))
-
        if self._is_vertexai:
            self._init_vertexai(**kwargs)
        else:
@@ -184,7 +168,6 @@ class GeminiLLM(LLMInterface):
        skip_validation: bool = False,
        strict_schema: bool = False,
        return_usage: bool = False,
-        cached_prefix: str | None = None,
    ) -> Any:
        """
        Make a Gemini/VertexAI API call with retry logic.
@@ -199,17 +182,8 @@ class GeminiLLM(LLMInterface):
            initial_backoff: Initial backoff time in seconds.
            max_backoff: Maximum backoff time in seconds.
            skip_validation: Return raw JSON without Pydantic validation.
-            strict_schema: Ignored — Gemini always grammar-enforces structured output via its
-                native response_schema, so it is strict regardless of this flag.
+            strict_schema: Use strict JSON schema enforcement (not supported by Gemini).
            return_usage: If True, return tuple (result, TokenUsage).
-            cached_prefix: Optional CachedContent resource name (from
-                ``GeminiCacheManager.get_or_create``). When set, the
-                system_instruction is assumed to live in the cache; this call
-                skips resending it and the cached prefix is billed at the
-                cached-input rate instead of the standard input rate. The
-                response_schema is still sent per-request (it is not cacheable).
-                Pass ``None`` to use the
-                normal uncached path.

        Returns:
            If return_usage=False: Parsed response if response_format provided, else text.
@@ -217,14 +191,9 @@ class GeminiLLM(LLMInterface):
        """
        start_time = time.time()

-        # Convert OpenAI-style messages to Gemini format. We ALWAYS build
-        # system_instruction (even when a cache is in use): the config builder
-        # below omits it from the request while the cache carries the prefix, but
-        # it must be available so the cached-call-failed safety net can re-send it
-        # inline. Whether it's actually sent is decided in _build_generation_config.
+        # Convert OpenAI-style messages to Gemini format
        system_instruction = None
        gemini_contents = []
-        using_cache = cached_prefix is not None

        for msg in messages:
            role = msg.get("role", "user")
@@ -240,9 +209,7 @@ class GeminiLLM(LLMInterface):
            else:
                gemini_contents.append(genai_types.Content(role="user", parts=[genai_types.Part(text=content)]))

-        # Add the JSON schema as a textual hint in the system_instruction (matching
-        # the normal uncached path). Structured output is still enforced via
-        # response_schema regardless; this is just guidance text.
+        # Add JSON schema instruction if response_format is provided
        if response_format is not None and hasattr(response_format, "model_json_schema"):
            schema = response_format.model_json_schema()
            schema_msg = f"\n\nYou must respond with valid JSON matching this schema:\n{json.dumps(schema, indent=2, ensure_ascii=False)}"
@@ -251,44 +218,32 @@ class GeminiLLM(LLMInterface):
            else:
                system_instruction = schema_msg

+        # Build generation config
+        config_kwargs: dict[str, Any] = {}
+        if system_instruction:
+            config_kwargs["system_instruction"] = system_instruction
+        if response_format is not None:
+            config_kwargs["response_mime_type"] = "application/json"
+            config_kwargs["response_schema"] = response_format
+        if temperature is not None:
+            config_kwargs["temperature"] = temperature
+        # Gemini's equivalent of OpenAI-style max_completion_tokens is max_output_tokens.
+        # Without it the model can produce arbitrarily long responses, ignoring the
+        # caller's intended cap (e.g. mental_models max_tokens during refresh).
+        if max_completion_tokens is not None:
+            config_kwargs["max_output_tokens"] = max_completion_tokens
+
        # Apply safety settings: context var (per-request bank override) takes precedence over instance default
        effective_safety_settings = _safety_settings_ctx.get()
        if effective_safety_settings is None:
            effective_safety_settings = self._safety_settings
+        if effective_safety_settings is not None:
+            config_kwargs["safety_settings"] = [
+                genai_types.SafetySetting(category=s["category"], threshold=s["threshold"])
+                for s in effective_safety_settings
+            ]

-        # Build generation config. ``cached_content`` and ``system_instruction``
-        # are mutually exclusive (the cache IS the prefix; the SDK rejects
-        # re-sending it). ``response_schema``/``response_mime_type`` are
-        # request-level output constraints — NOT cacheable — so they're set on
-        # every structured call, including cached ones where they ride alongside
-        # ``cached_content``. Built as a closure so we can rebuild it WITHOUT the
-        # cache and retry inline if a stale/invalid CachedContent makes the call fail.
-        def _build_generation_config(use_cache: bool) -> "genai_types.GenerateContentConfig | None":
-            # Seed with user-configured extra params; explicit settings below win.
-            config_kwargs: dict[str, Any] = dict(self._extra_body)
-            if use_cache:
-                config_kwargs["cached_content"] = cached_prefix
-            elif system_instruction:
-                config_kwargs["system_instruction"] = system_instruction
-            if response_format is not None:
-                config_kwargs["response_mime_type"] = "application/json"
-                config_kwargs["response_schema"] = response_format
-            if temperature is not None:
-                config_kwargs["temperature"] = temperature
-            # Gemini's equivalent of OpenAI-style max_completion_tokens is max_output_tokens.
-            # Without it the model can produce arbitrarily long responses, ignoring the
-            # caller's intended cap (e.g. mental_models max_tokens during refresh).
-            if max_completion_tokens is not None:
-                config_kwargs["max_output_tokens"] = max_completion_tokens
-            if effective_safety_settings is not None:
-                config_kwargs["safety_settings"] = [
-                    genai_types.SafetySetting(category=s["category"], threshold=s["threshold"])
-                    for s in effective_safety_settings
-                ]
-            return genai_types.GenerateContentConfig(**config_kwargs) if config_kwargs else None
-
-        cache_active = using_cache
-        generation_config = _build_generation_config(cache_active)
+        generation_config = genai_types.GenerateContentConfig(**config_kwargs) if config_kwargs else None

        last_exception = None

@@ -333,24 +288,13 @@ class GeminiLLM(LLMInterface):
                else:
                    result = content

-                # Extract token usage. ``cached_content_token_count`` and
-                # ``thoughts_token_count`` are populated on the Gemini 2.5+
-                # family; treat missing fields as 0 so older models still
-                # record sensible metrics.
+                # Extract token usage
                input_tokens = 0
                output_tokens = 0
-                cached_input_tokens = 0
-                thoughts_tokens = 0
-                cached_tokens = 0
                if hasattr(response, "usage_metadata") and response.usage_metadata:
                    usage = response.usage_metadata
                    input_tokens = usage.prompt_token_count or 0
                    output_tokens = usage.candidates_token_count or 0
-                    cached_input_tokens = getattr(usage, "cached_content_token_count", 0) or 0
-                    thoughts_tokens = getattr(usage, "thoughts_token_count", 0) or 0
-                    # Tracing/TokenUsage consume ``cached_tokens``; metrics consume
-                    # ``cached_input_tokens`` — same value, two downstream names.
-                    cached_tokens = cached_input_tokens

                # Record metrics
                duration = time.time() - start_time
@@ -363,8 +307,6 @@ class GeminiLLM(LLMInterface):
                    input_tokens=input_tokens,
                    output_tokens=output_tokens,
                    success=True,
-                    cached_input_tokens=cached_input_tokens,
-                    thoughts_tokens=thoughts_tokens,
                )

                # Record trace span
@@ -388,7 +330,6 @@ class GeminiLLM(LLMInterface):
                    duration=duration,
                    finish_reason=finish_reason,
                    error=None,
-                    cached_tokens=cached_tokens,
                )

                # Log slow calls
@@ -404,7 +345,6 @@ class GeminiLLM(LLMInterface):
                        input_tokens=input_tokens,
                        output_tokens=output_tokens,
                        total_tokens=input_tokens + output_tokens,
-                        cached_tokens=cached_tokens,
                    )
                    return result, token_usage
                return result
@@ -426,20 +366,6 @@ class GeminiLLM(LLMInterface):
                    logger.error(f"Gemini auth error (HTTP {e.code}), not retrying: {str(e)}")
                    raise

-                # Cached-request safety net: a stale/invalid/expired CachedContent
-                # (or an incompatibility like cache + tool_config) surfaces as a 400.
-                # Retrying the same cached request can't recover, so on the first
-                # such failure drop the cache, invalidate it so later operations
-                # recreate it, and retry THIS call inline with the prefix inlined.
-                # Caching must never break a request.
-                if cache_active and e.code == 400:
-                    logger.warning(f"Gemini cached call failed (400); retrying uncached. Reason: {str(e)}")
-                    if self._cache_manager is not None and cached_prefix is not None:
-                        self._cache_manager.invalidate(cached_prefix)
-                    cache_active = False
-                    generation_config = _build_generation_config(cache_active)
-                    continue
-
                # Retry on retryable errors (rate limits, server errors, client errors)
                if e.code in (400, 429, 500, 502, 503, 504) or (e.code and e.code >= 500):
                    last_exception = e
@@ -473,7 +399,6 @@ class GeminiLLM(LLMInterface):
        initial_backoff: float = 1.0,
        max_backoff: float = 30.0,
        tool_choice: str | dict[str, Any] = "auto",
-        cached_prefix: str | None = None,
    ) -> LLMToolCallResult:
        """
        Make a Gemini/VertexAI API call with tool/function calling support.
@@ -488,39 +413,27 @@ class GeminiLLM(LLMInterface):
            initial_backoff: Initial backoff time in seconds.
            max_backoff: Maximum backoff time in seconds.
            tool_choice: How to choose tools (Gemini uses "auto" only).
-            cached_prefix: Optional CachedContent resource name (from
-                ``GeminiCacheManager.get_or_create`` with ``tools=...``). When
-                set, the system_instruction and tool definitions are assumed
-                to live in the cache; this call will skip resending them and
-                the cached prefix is billed at the cached-input rate. The
-                ``tools`` argument is still required (the caller may pass
-                an empty list when the cache holds them) so existing call
-                sites don't break.

        Returns:
            LLMToolCallResult with content and/or tool_calls.
        """
        start_time = time.time()
-        using_cache = cached_prefix is not None

-        # Convert tools to Gemini format. When the cache is in use, the
-        # tool definitions are baked into the CachedContent at create time
-        # and the SDK rejects re-sending them alongside ``cached_content``.
+        # Convert tools to Gemini format
        gemini_tools = []
-        if not using_cache:
-            for tool in tools:
-                func = tool.get("function", {})
-                gemini_tools.append(
-                    genai_types.Tool(
-                        function_declarations=[
-                            genai_types.FunctionDeclaration(
-                                name=func.get("name", ""),
-                                description=func.get("description", ""),
-                                parameters=func.get("parameters"),
-                            )
-                        ]
-                    )
+        for tool in tools:
+            func = tool.get("function", {})
+            gemini_tools.append(
+                genai_types.Tool(
+                    function_declarations=[
+                        genai_types.FunctionDeclaration(
+                            name=func.get("name", ""),
+                            description=func.get("description", ""),
+                            parameters=func.get("parameters"),
+                        )
+                    ]
                )
+            )

        # Convert messages
        system_instruction = None
@@ -533,10 +446,6 @@ class GeminiLLM(LLMInterface):
            content = msg.get("content", "")

            if role == "system":
-                # Always capture system_instruction. _build_tools_config omits it
-                # (and tools) from the request while the cache carries the prefix,
-                # but it must be available so the cached-call-failed safety net can
-                # re-send the prefix + tools inline.
                system_instruction = (system_instruction + "\n\n" + content) if system_instruction else content
                i += 1
            elif role == "tool":
@@ -584,63 +493,49 @@ class GeminiLLM(LLMInterface):
                gemini_contents.append(genai_types.Content(role="user", parts=[genai_types.Part(text=content)]))
                i += 1

+        config_kwargs: dict[str, Any] = {"tools": gemini_tools}
+        if system_instruction:
+            config_kwargs["system_instruction"] = system_instruction
+        if temperature is not None:
+            config_kwargs["temperature"] = temperature
+        # See note in `call`: Gemini's max_output_tokens is the equivalent of
+        # OpenAI-style max_completion_tokens.
+        if max_completion_tokens is not None:
+            config_kwargs["max_output_tokens"] = max_completion_tokens
+
+        # Map OpenAI-style tool_choice to Gemini FunctionCallingConfig
+        if tool_choice == "required":
+            config_kwargs["tool_config"] = genai_types.ToolConfig(
+                function_calling_config=genai_types.FunctionCallingConfig(
+                    mode="ANY",
+                )
+            )
+        elif isinstance(tool_choice, dict) and tool_choice.get("type") == "function":
+            fn_name = tool_choice.get("function", {}).get("name")
+            if fn_name:
+                config_kwargs["tool_config"] = genai_types.ToolConfig(
+                    function_calling_config=genai_types.FunctionCallingConfig(
+                        mode="ANY",
+                        allowed_function_names=[fn_name],
+                    )
+                )
+        elif tool_choice == "none":
+            config_kwargs["tool_config"] = genai_types.ToolConfig(
+                function_calling_config=genai_types.FunctionCallingConfig(mode="NONE")
+            )
+        # "auto" is the default (no tool_config needed)
+
        # Apply safety settings: context var (per-request bank override) takes precedence over instance default
        effective_safety_settings = _safety_settings_ctx.get()
        if effective_safety_settings is None:
            effective_safety_settings = self._safety_settings
+        if effective_safety_settings is not None:
+            config_kwargs["safety_settings"] = [
+                genai_types.SafetySetting(category=s["category"], threshold=s["threshold"])
+                for s in effective_safety_settings
+            ]

-        # When using a cached prefix, the SDK rejects re-sending system_instruction
-        # or tools alongside ``cached_content`` — the cache IS the prefix.
-        # tool_config (mode / allowed_function_names) is a per-request decision and
-        # stays out of the cache. Built as a closure so we can rebuild it WITHOUT
-        # the cache and retry inline if a stale/invalid cache makes the call fail.
-        def _build_tools_config(use_cache: bool) -> "genai_types.GenerateContentConfig":
-            # Seed with user-configured extra params; explicit settings below win.
-            config_kwargs: dict[str, Any] = dict(self._extra_body)
-            if use_cache:
-                config_kwargs["cached_content"] = cached_prefix
-            else:
-                config_kwargs["tools"] = gemini_tools
-                if system_instruction:
-                    config_kwargs["system_instruction"] = system_instruction
-            if temperature is not None:
-                config_kwargs["temperature"] = temperature
-            # See note in `call`: Gemini's max_output_tokens is the equivalent of
-            # OpenAI-style max_completion_tokens.
-            if max_completion_tokens is not None:
-                config_kwargs["max_output_tokens"] = max_completion_tokens
-
-            # Map OpenAI-style tool_choice to Gemini FunctionCallingConfig
-            if tool_choice == "required":
-                config_kwargs["tool_config"] = genai_types.ToolConfig(
-                    function_calling_config=genai_types.FunctionCallingConfig(
-                        mode="ANY",
-                    )
-                )
-            elif isinstance(tool_choice, dict) and tool_choice.get("type") == "function":
-                fn_name = tool_choice.get("function", {}).get("name")
-                if fn_name:
-                    config_kwargs["tool_config"] = genai_types.ToolConfig(
-                        function_calling_config=genai_types.FunctionCallingConfig(
-                            mode="ANY",
-                            allowed_function_names=[fn_name],
-                        )
-                    )
-            elif tool_choice == "none":
-                config_kwargs["tool_config"] = genai_types.ToolConfig(
-                    function_calling_config=genai_types.FunctionCallingConfig(mode="NONE")
-                )
-            # "auto" is the default (no tool_config needed)
-
-            if effective_safety_settings is not None:
-                config_kwargs["safety_settings"] = [
-                    genai_types.SafetySetting(category=s["category"], threshold=s["threshold"])
-                    for s in effective_safety_settings
-                ]
-            return genai_types.GenerateContentConfig(**config_kwargs)
-
-        cache_active = using_cache
-        config = _build_tools_config(cache_active)
+        config = genai_types.GenerateContentConfig(**config_kwargs)

        last_exception = None
        for attempt in range(max_retries + 1):
@@ -683,18 +578,12 @@ class GeminiLLM(LLMInterface):

                finish_reason = "tool_calls" if tool_calls else "stop"

-                # Extract token usage. ``cached_content_token_count`` and
-                # ``thoughts_token_count`` are populated on the Gemini 2.5+
-                # family; absent fields are treated as 0.
+                # Extract token usage
                input_tokens = 0
                output_tokens = 0
-                cached_input_tokens = 0
-                thoughts_tokens = 0
                if response.usage_metadata:
                    input_tokens = response.usage_metadata.prompt_token_count or 0
                    output_tokens = response.usage_metadata.candidates_token_count or 0
-                    cached_input_tokens = getattr(response.usage_metadata, "cached_content_token_count", 0) or 0
-                    thoughts_tokens = getattr(response.usage_metadata, "thoughts_token_count", 0) or 0

                # Record metrics
                duration = time.time() - start_time
@@ -707,8 +596,6 @@ class GeminiLLM(LLMInterface):
                    input_tokens=input_tokens,
                    output_tokens=output_tokens,
                    success=True,
-                    cached_input_tokens=cached_input_tokens,
-                    thoughts_tokens=thoughts_tokens,
                )

                # Record OpenTelemetry span
@@ -733,7 +620,6 @@ class GeminiLLM(LLMInterface):
                    finish_reason=finish_reason,
                    error=None,
                    tool_calls=tool_calls_dict,
-                    cached_tokens=cached_input_tokens,
                )

                return LLMToolCallResult(
@@ -750,18 +636,6 @@ class GeminiLLM(LLMInterface):
                    logger.error(f"Gemini auth error (HTTP {e.code}), not retrying: {str(e)}")
                    raise

-                # Cached-request safety net (see ``call``): a stale/invalid cache or
-                # a cache+tool_config conflict surfaces as a 400. Drop the cache,
-                # invalidate it for later operations, and retry THIS call inline
-                # with the prefix + tools re-sent. Caching must never break a call.
-                if cache_active and e.code == 400:
-                    logger.warning(f"Gemini cached tool call failed (400); retrying uncached. Reason: {str(e)}")
-                    if self._cache_manager is not None and cached_prefix is not None:
-                        self._cache_manager.invalidate(cached_prefix)
-                    cache_active = False
-                    config = _build_tools_config(cache_active)
-                    continue
-
                # Retry on retryable errors
                last_exception = e
                if attempt < max_retries:
@@ -778,54 +652,6 @@ class GeminiLLM(LLMInterface):
            raise last_exception
        raise RuntimeError("Gemini tool call failed")

-    def supports_prompt_caching(self) -> bool:
-        """True when explicit Gemini context caching is enabled for this instance.
-
-        Reflects the opt-in flag so callers skip the cache lookup entirely when
-        it's off; ``get_or_create_cached_prefix`` also returns None in that case.
-        """
-        return self._prompt_cache_enabled
-
-    async def get_or_create_cached_prefix(
-        self,
-        *,
-        system_instruction: str,
-        response_schema: Any | None = None,
-        tools: list[dict[str, Any]] | None = None,
-    ) -> str | None:
-        """Return a CachedContent resource name for the given prefix, or
-        ``None`` if context caching is disabled, the provider doesn't
-        support it, or Gemini rejects the create (prefix too small, etc.).
-
-        ``tools`` is the OpenAI-style tools list; pass it when caching a
-        prefix that will be used by ``call_with_tools()``. The fingerprint
-        includes the tool definitions so a loop that swaps a tool gets a
-        fresh cache automatically.
-
-        Callers pass the returned name to ``call(cached_prefix=...)``
-        or ``call_with_tools(cached_prefix=...)`` and treat ``None``
-        as "cache unavailable — use the normal path". That fallback is
-        essential: the system must continue to work if caching is disabled,
-        if Gemini's caching API has an outage, or if the prefix is below
-        the model's minimum cacheable size.
-        """
-        if not self._prompt_cache_enabled:
-            return None
-        if self._client is None:
-            return None
-        if self._cache_manager is None:
-            # Lazy import so the cache module is only loaded when caching
-            # is actually used.
-            from hindsight_api.engine.providers.gemini_cache import GeminiCacheManager
-
-            self._cache_manager = GeminiCacheManager(self._client)
-        return await self._cache_manager.get_or_create(
-            model=self.model,
-            system_instruction=system_instruction,
-            response_schema=response_schema,
-            tools=tools,
-        )
-
    async def cleanup(self) -> None:
        """Clean up resources (close connections, etc.)."""
        # Gemini client doesn't require explicit cleanup
@@ -48,18 +48,11 @@ class LiteLLMLLM(LLMInterface):
        model: str,
        reasoning_effort: str = "low",
        timeout: float = 300.0,
-        extra_body: dict[str, Any] | None = None,
        **kwargs: Any,
    ):
        super().__init__(provider, api_key, base_url, model, reasoning_effort, **kwargs)
        self.timeout = timeout
        self._litellm: Any = None
-        # User-configured extra params merged as top-level kwargs into every
-        # completion call so LiteLLM normalizes them per-provider (e.g. maps
-        # temperature/top_p/max_tokens across OpenAI, Anthropic, Bedrock, …) and
-        # drops any the target model rejects (litellm.drop_params=True below).
-        # Sourced from llm_extra_body (env: HINDSIGHT_API_LLM_EXTRA_BODY).
-        self._extra_body: dict[str, Any] = extra_body or {}

        try:
            import litellm
@@ -114,11 +107,6 @@ class LiteLLMLLM(LLMInterface):
        if temperature is not None:
            kwargs["temperature"] = temperature

-        # User-configured extras fill in only where the caller didn't set a value,
-        # so explicit per-call params (model, messages, temperature, …) always win.
-        for key, value in self._extra_body.items():
-            kwargs.setdefault(key, value)
-
        return kwargs

    # ── per-model output-tokens cap (shared with Router subclass) ────────────
@@ -162,12 +162,6 @@ class MockLLM(LLMInterface):
            # Consolidation: produce a single observation from the input facts
            # so the full pipeline (retain → consolidation → observation → recall) works.
            result = self._build_mock_consolidation(messages, response_format)
-        elif scope == "consolidation_dedup" and response_format is not None:
-            # Observation dedup adjudication. Default to "keep" so mock-LLM consolidation never
-            # spuriously merges observations — this preserves the pre-dedup behaviour that
-            # deterministic consolidation tests assert (the generic branch below can't construct
-            # the model because its "action" field is required and has no default).
-            result = response_format(action="keep", reason="mock")
        elif scope == "memory_think":
            # Reflect: return a plausible text answer
            result = "Based on the available information, the answer is related to the context provided."
@@ -7,7 +7,7 @@ This provider handles all OpenAI API-compatible models including:
 - Groq: Fast inference with seed control and service tiers
 - Ollama: Local models with native streaming API support
 - LMStudio: Local models with OpenAI-compatible API
- MiniMax: MiniMax-M3 / MiniMax-M2.7 models with 1M context window
+- MiniMax: MiniMax-M2.7 models with 1M context window
 - DeepSeek: deepseek-v4-flash / deepseek-v4-pro / deepseek-chat / deepseek-reasoner via api.deepseek.com
 - Opencode Go: deepseek-v4-flash via https://opencode.ai/zen/go/v1

@@ -44,16 +44,6 @@ logger = logging.getLogger(__name__)
 DEFAULT_LLM_SEED = 4242
 JSON_MODE_USER_HINT = "Return valid json only."

-# Self-hosted OpenAI-compatible servers that advertise tool_choice="required"
-# but silently ignore it: instead of forcing a tool call they return
-# finish_reason "stop"/"tool_calls" with an EMPTY tool_calls array and no error.
-# Reflect's agent loop then sees no tool call, runs synthesis with no retrieval,
-# and answers "I don't have information" even when the bank holds the answer.
-# See issues #1563 (LM Studio), #1179 (LM Studio + Qwen), #1877 (vLLM with
-# --enable-auto-tool-choice). llama-server (the "llamacpp" provider) honors
-# "required" correctly and is intentionally excluded (#1179).
-_TOOL_CHOICE_REQUIRED_UNSUPPORTED_PROVIDERS = frozenset({"lmstudio", "ollama"})
-

 class ProviderResponseError(RuntimeError):
    """Raised when a provider returns a success response without usable content."""
@@ -242,7 +232,7 @@ class OpenAICompatibleLLM(LLMInterface):
    - Groq: Fast inference with seed control and service tiers
    - Ollama: Local models with native streaming API for better structured output
    - LMStudio: Local models with OpenAI-compatible API
-    - MiniMax: MiniMax-M3 / MiniMax-M2.7 models via OpenAI-compatible API (https://api.minimax.io/v1)
+    - MiniMax: MiniMax-M2.7 models via OpenAI-compatible API (https://api.minimax.io/v1)
    - DeepSeek: deepseek-v4-flash / deepseek-v4-pro / deepseek-chat / deepseek-reasoner via https://api.deepseek.com
    - opencode-go: deepseek-v4-flash via https://opencode.ai/zen/go/v1
    """
@@ -370,21 +360,6 @@ class OpenAICompatibleLLM(LLMInterface):
            f"base_url={self.base_url or 'default'}"
        )

-    def _drops_tool_choice_required(self) -> bool:
-        """Whether this endpoint silently ignores ``tool_choice="required"``.
-
-        True for self-hosted OpenAI-compatible servers known to return an empty
-        tool_calls array for "required" instead of forcing a call (#1563/#1179/
-        #1877). Covers LM Studio / Ollama directly, plus any server reached via
-        the generic "openai" provider with a custom ``base_url`` (e.g. a local
-        vLLM endpoint). The real OpenAI API (no base_url override) honors
-        "required", and cloud providers keep their own default base_urls, so both
-        are left untouched.
-        """
-        if self.provider in _TOOL_CHOICE_REQUIRED_UNSUPPORTED_PROVIDERS:
-            return True
-        return self.provider == "openai" and bool(self.base_url)
-
    async def verify_connection(self) -> None:
        """
        Verify that the provider is configured correctly by making a simple test call.
@@ -485,9 +460,7 @@ class OpenAICompatibleLLM(LLMInterface):
            initial_backoff: Initial backoff time in seconds.
            max_backoff: Maximum backoff time in seconds.
            skip_validation: Return raw JSON without Pydantic validation.
-            strict_schema: Use strict json_schema (grammar-enforced) response_format instead of
-                the soft json_object path. Supported by OpenAI and schema-capable self-hosted
-                backends (llama.cpp, vLLM). Server-wide via HINDSIGHT_API_LLM_STRICT_SCHEMA.
+            strict_schema: Use strict JSON schema enforcement (OpenAI only).
            return_usage: If True, return tuple (result, TokenUsage) instead of just result.

        Returns:
@@ -677,9 +650,6 @@ class OpenAICompatibleLLM(LLMInterface):
                input_tokens = usage.prompt_tokens or 0 if usage else 0
                output_tokens = usage.completion_tokens or 0 if usage else 0
                total_tokens = usage.total_tokens or 0 if usage else 0
-                cached_tokens = 0
-                if usage and getattr(usage, "prompt_tokens_details", None):
-                    cached_tokens = getattr(usage.prompt_tokens_details, "cached_tokens", 0) or 0

                # Record LLM metrics
                metrics = get_metrics_collector()
@@ -709,12 +679,14 @@ class OpenAICompatibleLLM(LLMInterface):
                    duration=duration,
                    finish_reason=finish_reason,
                    error=None,
-                    cached_tokens=cached_tokens,
                )

                # Log slow calls
                if duration > 10.0 and usage:
                    ratio = max(1, output_tokens) / max(1, input_tokens)
+                    cached_tokens = 0
+                    if hasattr(usage, "prompt_tokens_details") and usage.prompt_tokens_details:
+                        cached_tokens = getattr(usage.prompt_tokens_details, "cached_tokens", 0) or 0
                    cache_info = f", cached_tokens={cached_tokens}" if cached_tokens > 0 else ""
                    logger.info(
                        f"slow llm call: scope={scope}, model={self.provider}/{self.model}, "
@@ -727,7 +699,6 @@ class OpenAICompatibleLLM(LLMInterface):
                        input_tokens=input_tokens,
                        output_tokens=output_tokens,
                        total_tokens=total_tokens,
-                        cached_tokens=cached_tokens,
                    )
                    return result, token_usage
                return result
@@ -896,16 +867,6 @@ class OpenAICompatibleLLM(LLMInterface):
        if request_tool_choice == "auto":
            request_tool_choice = None

-        # vLLM (--enable-auto-tool-choice), LM Studio, Ollama and similar
-        # self-hosted servers silently drop tool_choice="required", returning an
-        # empty tool_calls array instead of forcing a call (#1563/#1179/#1877).
-        # Downgrade to auto (None) so the model still gets to call a tool. Named
-        # tool_choice dicts were already normalized to "required" + a single
-        # filtered tool above, so the call stays practically forced even under
-        # auto. The real OpenAI API honors "required" and is left untouched.
-        if request_tool_choice == "required" and self._drops_tool_choice_required():
-            request_tool_choice = None
-
        # DeepSeek tool-call replies can carry provider-specific reasoning_content.
        # The normalized tool result does not retain it, but replaying assistant
        # tool_calls without the field can trigger a 400. DeepSeek accepts an
@@ -302,24 +302,6 @@ def _is_context_overflow_error(exc: Exception) -> bool:
    )


-def _all_mental_models_are_usable_and_fresh(tool_output: dict[str, Any]) -> bool:
-    """Return whether every retrieved mental model is explicitly fresh and has answerable content.
-
-    Used to decide — without an extra LLM call — whether a forced
-    ``search_mental_models`` result is trustworthy enough to hand control back
-    to the agent. A model is usable only when it is explicitly ``is_stale ==
-    False`` (an unknown/missing staleness flag is treated as unsafe) and has
-    non-empty content.
-    """
-    models = tool_output.get("mental_models") or []
-    for model in models:
-        if model.get("is_stale") is not False:
-            return False
-        if not str(model.get("content") or "").strip():
-            return False
-    return True
-
-
 async def run_reflect_agent(
    llm_config: "LLMProvider",
    bank_id: str,
@@ -400,28 +382,6 @@ async def run_reflect_agent(
        {"role": "user", "content": query},
    ]

-    # Opt into context caching for the agentic tool loop. The system
-    # prompt and tool definitions are stable for the duration of this
-    # reflect call (and across reflects against the same bank), so
-    # caching them once and reusing across every iteration of the loop
-    # collapses the dominant input cost — the prefix repeated on every
-    # turn. ``get_or_create_cached_prefix`` returns None when caching is
-    # disabled, unsupported, or the prefix is too small; the
-    # ``call_with_tools`` invocation below transparently falls back to
-    # the uncached path in that case.
-    cached_prefix_name: str | None = None
-    provider_impl = getattr(llm_config, "_provider_impl", None)
-    if provider_impl is not None and provider_impl.supports_prompt_caching():
-        try:
-            cached_prefix_name = await provider_impl.get_or_create_cached_prefix(
-                system_instruction=system_prompt,
-                tools=tools,
-            )
-        except Exception:
-            # Caching is a soft optimisation; never let a cache-side
-            # error block a reflect.
-            cached_prefix_name = None
-
    # Tracking
    total_tools_called = 0
    tool_trace: list[ToolCall] = []
@@ -482,11 +442,6 @@ async def run_reflect_agent(
        )

    consecutive_errors = 0
-    # When a forced ``search_mental_models`` returns fresh, usable models on a
-    # low/mid-budget call, we stop forcing the lower retrieval layers from this
-    # iteration onward and let the agent answer (or retrieve deeper itself)
-    # under ``auto`` tool choice. None means the full forced path still applies.
-    stop_forcing_from_iteration: int | None = None
    for iteration in range(max_iterations):
        is_last = iteration == max_iterations - 1

@@ -615,31 +570,18 @@ async def run_reflect_agent(
        if include_recall:
            forced_sequence.append("recall")

-        if stop_forcing_from_iteration is not None and iteration >= stop_forcing_from_iteration:
-            # A fresh mental model already short-circuited the forced path.
-            iter_tool_choice: str | dict = "auto"
-        elif iteration < len(forced_sequence):
-            iter_tool_choice = {"type": "function", "function": {"name": forced_sequence[iteration]}}
+        if iteration < len(forced_sequence):
+            iter_tool_choice: str | dict = {"type": "function", "function": {"name": forced_sequence[iteration]}}
        else:
            iter_tool_choice = "auto"

        try:
-            ct_kwargs: dict[str, Any] = dict(
+            result = await llm_config.call_with_tools(
                messages=messages,
                tools=tools,
                scope="reflect_tool_call",
                tool_choice=iter_tool_choice,
            )
-            # Gemini rejects ``cached_content`` alongside a per-request
-            # ``tool_config`` (forced tool choice): "CachedContent can not be used
-            # with GenerateContent request setting system_instruction, tools or
-            # tool_config." The forced-sequence iterations set tool_config, so only
-            # the ``auto`` iterations can reference the cache; forced iterations send
-            # the prefix inline. The cache (tools + system prompt) is identical
-            # either way, so this just limits *which* iterations are billed cached.
-            if cached_prefix_name is not None and iter_tool_choice == "auto":
-                ct_kwargs["cached_prefix"] = cached_prefix_name
-            result = await llm_config.call_with_tools(**ct_kwargs)
            llm_duration = int((time.time() - llm_start) * 1000)
            consecutive_errors = 0
            total_input_tokens += result.input_tokens
@@ -982,25 +924,6 @@ async def run_reflect_agent(
                    for mm in output["mental_models"]:
                        if "id" in mm:
                            available_mental_model_ids.add(mm["id"])
-                    # Deterministic short-circuit (no extra LLM call): on a
-                    # low/mid-budget call, if every retrieved mental model is
-                    # fresh and has usable content, stop forcing the lower
-                    # retrieval layers. The next iteration runs under ``auto``
-                    # tool choice, so the agent can answer directly when the
-                    # mental model suffices, or — having just read it — issue a
-                    # targeted ``search_observations``/``recall`` itself. Stale,
-                    # empty, or missing mental models keep the full forced path.
-                    if (
-                        stop_forcing_from_iteration is None
-                        and (budget or "low").lower() != "high"
-                        and output.get("mental_models")
-                        and _all_mental_models_are_usable_and_fresh(output)
-                    ):
-                        stop_forcing_from_iteration = iteration + 1
-                        logger.info(
-                            f"[REFLECT {reflect_id}] Fresh mental models sufficient on iteration {iteration + 1}; "
-                            "releasing forced lower-level retrieval to auto."
-                        )

                if (
                    normalized_tool_name == "search_observations"
@@ -93,7 +93,6 @@ class TokenUsage(BaseModel):
    input_tokens: int = Field(default=0, description="Number of input/prompt tokens consumed")
    output_tokens: int = Field(default=0, description="Number of output/completion tokens generated")
    total_tokens: int = Field(default=0, description="Total tokens (input + output)")
-    cached_tokens: int = Field(default=0, description="Cached/cache-read prompt tokens, when reported by the provider")

    def __add__(self, other: "TokenUsage") -> "TokenUsage":
        """Allow aggregating token usage from multiple calls."""
@@ -101,7 +100,6 @@ class TokenUsage(BaseModel):
            input_tokens=self.input_tokens + other.input_tokens,
            output_tokens=self.output_tokens + other.output_tokens,
            total_tokens=self.total_tokens + other.total_tokens,
-            cached_tokens=self.cached_tokens + other.cached_tokens,
        )


@@ -6,7 +6,6 @@ import json
 import logging
 import re
 import uuid
-from dataclasses import dataclass
 from typing import TypedDict

 from pydantic import BaseModel, Field
@@ -106,18 +105,6 @@ class BankProfile(TypedDict):
    mission: str


-@dataclass
-class BankProfileResult:
-    """Result of a get-or-create bank lookup.
-
-    ``created`` is True when the bank row was freshly inserted on this call,
-    which callers use to drive the one-time HINDSIGHT_API_DEFAULT_BANK_TEMPLATE hook.
-    """
-
-    profile: BankProfile
-    created: bool
-
-
 class MissionMergeResponse(BaseModel):
    """LLM response for mission merge."""

@@ -136,8 +123,8 @@ async def get_bank_profile(pool, bank_id: str) -> BankProfile:
    Returns:
        BankProfile with name, typed DispositionTraits, and mission
    """
-    result = await get_or_create_bank_profile(pool, bank_id)
-    return result.profile
+    profile, _ = await get_or_create_bank_profile(pool, bank_id)
+    return profile


 async def get_bank_profile_if_exists(pool, bank_id: str) -> BankProfile | None:
@@ -175,89 +162,70 @@ async def get_bank_profile_if_exists(pool, bank_id: str) -> BankProfile | None:
        )


-async def get_or_create_bank_profile(pool, bank_id: str) -> BankProfileResult:
+async def get_or_create_bank_profile(pool, bank_id: str) -> tuple[BankProfile, bool]:
    """
    Get bank profile, auto-creating with defaults if it doesn't exist.

-    Same as get_bank_profile, but also reports whether the bank was freshly
-    created on this call (``BankProfileResult.created``). Used by the memory
-    engine to apply the HINDSIGHT_API_DEFAULT_BANK_TEMPLATE hook on first bank
-    creation.
+    Same as get_bank_profile, but also returns a flag indicating whether the
+    bank was freshly created on this call. Used by the memory engine to apply
+    the HINDSIGHT_API_DEFAULT_BANK_TEMPLATE hook on first bank creation.

-    Acquires its own connection. When the caller already holds a connection and
-    wants the bank row to share its transaction (so the lazy bank-create commits
-    or rolls back atomically with the caller's write), use
-    ``get_or_create_bank_profile_on_conn`` instead.
+    Returns:
+        Tuple of (BankProfile, created) where created is True if the bank
+        did not exist before this call.
    """
    async with acquire_with_retry(pool) as conn:
-        return await get_or_create_bank_profile_on_conn(conn, bank_id, ops=pool.ops)
-
-
-async def get_or_create_bank_profile_on_conn(conn, bank_id: str, *, ops) -> BankProfileResult:
-    """
-    Connection-bound variant of ``get_or_create_bank_profile``.
-
-    Runs the SELECT, the ``INSERT ... ON CONFLICT DO NOTHING`` and the per-bank
-    vector index creation on the caller-supplied ``conn``. When ``conn`` is
-    inside an open transaction, the lazy bank-create therefore commits (or rolls
-    back) atomically with whatever bank-scoped write the caller performs on the
-    same connection — closing the window where a freshly-created bank could
-    outlive a write that ultimately failed.
-
-    ``ops`` is the backend's dialect ops object (``backend.ops``), needed for
-    per-bank vector index DDL.
-    """
-    # Try to get existing bank
-    row = await conn.fetchrow(
-        f"""
-        SELECT name, disposition, mission
-        FROM {fq_table("banks")} WHERE bank_id = $1
-        """,
-        bank_id,
-    )
-
-    if row:
-        # asyncpg returns JSONB as a string, so parse it
-        disposition_data = row["disposition"]
-        if isinstance(disposition_data, str):
-            disposition_data = json.loads(disposition_data)
-
-        return BankProfileResult(
-            profile=BankProfile(
-                name=row["name"],
-                disposition=DispositionTraits(**disposition_data),
-                mission=row["mission"] or "",
-            ),
-            created=False,
+        # Try to get existing bank
+        row = await conn.fetchrow(
+            f"""
+            SELECT name, disposition, mission
+            FROM {fq_table("banks")} WHERE bank_id = $1
+            """,
+            bank_id,
        )

-    # Bank doesn't exist, create with defaults.
-    # Generate internal_id here so we control the value and can use it
-    # immediately for vector index creation without a RETURNING round-trip.
-    internal_id = uuid.uuid4()
-    inserted = await conn.fetchval(
-        f"""
-        INSERT INTO {fq_table("banks")} (bank_id, name, disposition, mission, internal_id)
-        VALUES ($1, $2, $3::jsonb, $4, $5)
-        ON CONFLICT (bank_id) DO NOTHING
-        RETURNING bank_id
-        """,
-        bank_id,
-        bank_id,  # Default name is the bank_id
-        json.dumps(DEFAULT_DISPOSITION),
-        "",
-        internal_id,
-    )
+        if row:
+            # asyncpg returns JSONB as a string, so parse it
+            disposition_data = row["disposition"]
+            if isinstance(disposition_data, str):
+                disposition_data = json.loads(disposition_data)

-    created = inserted is not None
-    if created:
-        # Fresh insert — create per-bank vector indexes (instant on empty bank)
-        await create_bank_vector_indexes(conn, bank_id, str(internal_id), ops=ops)
+            return (
+                BankProfile(
+                    name=row["name"],
+                    disposition=DispositionTraits(**disposition_data),
+                    mission=row["mission"] or "",
+                ),
+                False,
+            )

-    return BankProfileResult(
-        profile=BankProfile(name=bank_id, disposition=DispositionTraits(**DEFAULT_DISPOSITION), mission=""),
-        created=created,
-    )
+        # Bank doesn't exist, create with defaults.
+        # Generate internal_id here so we control the value and can use it
+        # immediately for vector index creation without a RETURNING round-trip.
+        internal_id = uuid.uuid4()
+        inserted = await conn.fetchval(
+            f"""
+            INSERT INTO {fq_table("banks")} (bank_id, name, disposition, mission, internal_id)
+            VALUES ($1, $2, $3::jsonb, $4, $5)
+            ON CONFLICT (bank_id) DO NOTHING
+            RETURNING bank_id
+            """,
+            bank_id,
+            bank_id,  # Default name is the bank_id
+            json.dumps(DEFAULT_DISPOSITION),
+            "",
+            internal_id,
+        )
+
+        created = inserted is not None
+        if created:
+            # Fresh insert — create per-bank vector indexes (instant on empty bank)
+            await create_bank_vector_indexes(conn, bank_id, str(internal_id), ops=pool.ops)
+
+        return (
+            BankProfile(name=bank_id, disposition=DispositionTraits(**DEFAULT_DISPOSITION), mission=""),
+            created,
+        )


 async def update_bank_disposition(pool, bank_id: str, disposition: dict[str, int]) -> None:
@@ -15,23 +15,11 @@ class EmbeddingsBackend(Protocol):
    """Minimal duck-typed surface used by retain/recall — the concrete `Embeddings`
    ABC supplies default implementations that delegate to `encode()`."""

-    @property
-    def dimension(self) -> int: ...
-
    def encode_query(self, texts: list[str]) -> list[list[float]]: ...

    def encode_documents(self, texts: list[str]) -> list[list[float]]: ...


-def _validate_embedding_vector(vector: list[float], *, index: int, expected_dimension: int) -> list[float]:
-    actual_dimension = len(vector)
-    if actual_dimension == 0:
-        raise RuntimeError(f"embedding {index} has dimension 0; expected {expected_dimension}")
-    if actual_dimension != expected_dimension:
-        raise RuntimeError(f"embedding {index} has dimension {actual_dimension}; expected {expected_dimension}")
-    return vector
-
-
 def generate_embedding(
    embeddings_backend: EmbeddingsBackend, text: str, input_type: EmbeddingInputType = "document"
 ) -> list[float]:
@@ -48,19 +36,10 @@ def generate_embedding(
    """
    try:
        embeddings = _encode_with_input_type(embeddings_backend, [text], input_type)
+        return embeddings[0]
    except Exception as e:
        raise Exception(f"Failed to generate embedding: {str(e)}")

-    if len(embeddings) != 1:
-        raise RuntimeError(
-            f"Embeddings backend returned {len(embeddings)} vectors for 1 input text; expected exact 1:1 alignment"
-        )
-    return _validate_embedding_vector(
-        embeddings[0],
-        index=0,
-        expected_dimension=embeddings_backend.dimension,
-    )
-

 def _encode_with_input_type(
    embeddings_backend: EmbeddingsBackend, texts: list[str], input_type: EmbeddingInputType
@@ -102,7 +81,4 @@ async def generate_embeddings_batch(
            "expected exact 1:1 alignment"
        )

-    return [
-        _validate_embedding_vector(embedding, index=index, expected_dimension=embeddings_backend.dimension)
-        for index, embedding in enumerate(embeddings)
-    ]
+    return embeddings
@@ -10,13 +10,12 @@ import json
 import logging
 import re
 from datetime import datetime, timedelta
-from typing import Any, Literal, cast
+from typing import Literal, cast

 from pydantic import BaseModel, ConfigDict, Field, create_model, field_validator

 from ...config import get_config
 from ..llm_wrapper import LLMConfig, OutputTooLongError, sanitize_llm_output
-from ..operation_metadata import RetainExtractionErrors
 from ..response_models import TokenUsage
 from .entity_labels import (
    EntityLabelsConfig,
@@ -511,11 +510,11 @@ LANGUAGE: MANDATORY — Detect the language of the input text and produce ALL ou
 FACT FORMAT - BE CONCISE
 ══════════════════════════════════════════════════════════════════════════

-1. "what": Core fact - concise but complete (1-2 sentences max)
-2. "when": Temporal info if mentioned. "N/A" if none. Use day name when known.
-3. "where": Location if relevant. "N/A" if none.
-4. "who": People involved with relationships. "N/A" if just general info.
-5. "why": Context/significance ONLY if important. "N/A" if obvious.
+1. **what**: Core fact - concise but complete (1-2 sentences max)
+2. **when**: Temporal info if mentioned. "N/A" if none. Use day name when known.
+3. **where**: Location if relevant. "N/A" if none.
+4. **who**: People involved with relationships. "N/A" if just general info.
+5. **why**: Context/significance ONLY if important. "N/A" if obvious.

 CONCISENESS: Capture the essence, not every word. One good sentence beats three mediocre ones.

@@ -888,15 +887,20 @@ def _build_extraction_prompt_and_schema(config) -> tuple[str, type]:
    extraction_mode = config.retain_extraction_mode
    extract_causal_links = config.retain_extract_causal_links

-    # The per-bank retain mission is NOT baked into this system prompt: it would
-    # make the prompt bank-specific and force a separate Gemini context cache per
-    # mission (one per bank). Instead the prompt is bank-agnostic so a single
-    # CachedContent serves every bank, and the mission rides in the per-request
-    # user message via _retain_mission_preamble(). The {retain_mission_section}
-    # placeholder is kept (templates still reference it) but always empty here.
+    # Build retain_mission section if set - injected before the mode-specific guidelines
+    # Escape braces so user-supplied text survives str.format() on the prompt template.
    from hindsight_api.engine.prompt_utils import escape_for_prompt

-    retain_mission_section = ""
+    retain_mission = getattr(config, "retain_mission", None)
+    if retain_mission:
+        retain_mission_section = (
+            f"══════════════════════════════════════════════════════════════════════════\n"
+            f"FOCUS — What to retain for this bank\n"
+            f"══════════════════════════════════════════════════════════════════════════\n\n"
+            f"{escape_for_prompt(retain_mission)}\n\n"
+        )
+    else:
+        retain_mission_section = ""

    # Select base prompt based on extraction mode
    if extraction_mode == "custom":
@@ -993,26 +997,6 @@ def _build_extraction_prompt_and_schema(config) -> tuple[str, type]:
    return prompt, response_schema


-def _retain_mission_preamble(config) -> str:
-    """The bank's retain mission, formatted for the per-request user message.
-
-    Kept OUT of the cached system prompt (which must stay bank-agnostic so one
-    CachedContent serves every bank — otherwise each distinct mission spawns its
-    own cache) and prepended to the user message instead. Returns "" when unset.
-    No brace-escaping needed: unlike the system template, the user message is
-    used verbatim, not passed through str.format().
-    """
-    retain_mission = getattr(config, "retain_mission", None)
-    if not retain_mission:
-        return ""
-    return (
-        "══════════════════════════════════════════════════════════════════════════\n"
-        "FOCUS — What to retain for this bank (takes priority over the general guidelines)\n"
-        "══════════════════════════════════════════════════════════════════════════\n\n"
-        f"{retain_mission}\n\n"
-    )
-
-
 def _build_user_message(
    chunk: str,
    chunk_index: int,
@@ -1021,14 +1005,8 @@ def _build_user_message(
    context: str,
    metadata: dict[str, str] | None = None,
    agent_name: str | None = None,
-    mission_preamble: str = "",
 ) -> str:
-    """Build user message for fact extraction.
-
-    ``mission_preamble`` (the bank's retain mission, possibly empty) is prepended
-    so the bank-specific focus lives in the variable user turn rather than the
-    cached, bank-agnostic system prompt.
-    """
+    """Build user message for fact extraction."""
    from .orchestrator import parse_datetime_flexible

    sanitized_chunk = _sanitize_text(chunk)
@@ -1047,21 +1025,9 @@ def _build_user_message(

    narrator_section = ""
    if agent_name:
-        narrator_section = (
-            f"\nNarrator: {agent_name} (the AI agent whose memory this is). By default, "
-            f'first-person statements like "I did X" are {agent_name}\'s own actions → classify as '
-            f'"assistant".'
-        )
-        # Only defer to the Context when one was actually provided — otherwise this
-        # clause points at a "Context: none" line and just adds noise.
-        if context:
-            narrator_section += (
-                " BUT the Context above takes precedence: if it identifies a different "
-                "first-person speaker (e.g. a user or customer in a transcript), attribute those "
-                'statements to that speaker and classify them as "world", not "assistant".'
-            )
+        narrator_section = f'\nNarrator: {agent_name} (AI agent — first-person statements like "I did X" are the agent\'s own actions; classify as "assistant")'

-    return f"""{mission_preamble}Extract facts from the following text chunk.
+    return f"""Extract facts from the following text chunk.

 Chunk: {chunk_index + 1}/{total_chunks}
 Event Date: {event_date_str}
@@ -1087,15 +1053,12 @@ def _build_request_body(llm_config, config, prompt: str, user_message: str, resp
    if llm_config.provider == "openai" and llm_config._provider_impl.openai_service_tier:
        request_body["service_tier"] = llm_config._provider_impl.openai_service_tier

-    # Add response_format (JSON schema). The batch path builds the request body
-    # directly instead of going through LLMProvider.call(), so honour
-    # HINDSIGHT_API_LLM_STRICT_SCHEMA here too: strict=True grammar-enforces the
-    # output on capable backends rather than relying on the model to emit clean JSON.
+    # Add response_format (JSON schema)
    if hasattr(response_schema, "model_json_schema"):
        schema = response_schema.model_json_schema()
        request_body["response_format"] = {
            "type": "json_schema",
-            "json_schema": {"name": "facts", "schema": schema, "strict": config.llm_strict_schema},
+            "json_schema": {"name": "facts", "schema": schema},
        }

    return request_body
@@ -1131,38 +1094,8 @@ async def _extract_facts_from_chunk(
    extraction_mode = config.retain_extraction_mode
    extract_causal_links = config.retain_extract_causal_links

-    # Build user message — the bank mission rides here (not in the cached prefix).
-    user_message = _build_user_message(
-        chunk,
-        chunk_index,
-        total_chunks,
-        event_date,
-        context,
-        metadata,
-        agent_name,
-        mission_preamble=_retain_mission_preamble(config),
-    )
-
-    # Opt into context caching when the provider supports it. The prompt and
-    # response_schema are bank-agnostic (the mission lives in the user message),
-    # so one cached prefix serves every bank; reusing it across many small-payload
-    # retain calls dramatically lowers per-call input
-    # cost. ``get_or_create_cached_prefix`` returns None when caching is
-    # disabled, unsupported, or the prefix is too small; the LLM call
-    # transparently falls back to the uncached path in that case.
-    cached_prefix_name: str | None = None
-    provider_impl = getattr(llm_config, "_provider_impl", None)
-    if provider_impl is not None and provider_impl.supports_prompt_caching():
-        try:
-            cached_prefix_name = await provider_impl.get_or_create_cached_prefix(
-                system_instruction=prompt,
-                response_schema=response_schema,
-            )
-        except Exception:
-            # Caching is a soft optimisation — never let a cache-side
-            # error block a retain operation.
-            logger.exception("Cache prefix lookup failed; falling back to uncached call")
-            cached_prefix_name = None
+    # Build user message using helper function
+    user_message = _build_user_message(chunk, chunk_index, total_chunks, event_date, context, metadata, agent_name)

    # Retry logic for JSON validation errors
    # Use retain-specific overrides if set, otherwise fall back to global LLM config
@@ -1183,7 +1116,7 @@ async def _extract_facts_from_chunk(
                config.retain_llm_max_backoff if config.retain_llm_max_backoff is not None else config.llm_max_backoff
            )

-            call_kwargs: dict[str, Any] = dict(
+            extraction_response_json, call_usage = await llm_config.call(
                messages=[{"role": "system", "content": prompt}, {"role": "user", "content": user_message}],
                response_format=response_schema,
                scope="retain_extract_facts",
@@ -1195,10 +1128,6 @@ async def _extract_facts_from_chunk(
                skip_validation=True,  # Get raw JSON, we'll validate leniently
                return_usage=True,
            )
-            if cached_prefix_name is not None:
-                call_kwargs["cached_prefix"] = cached_prefix_name
-
-            extraction_response_json, call_usage = await llm_config.call(**call_kwargs)
            usage = usage + call_usage  # Aggregate usage across retries

            # Lenient parsing of facts from raw JSON
@@ -1711,39 +1640,6 @@ logger = logging.getLogger(__name__)
 SECONDS_PER_FACT = 0.01


-async def _write_batch_extraction_errors(
-    pool: Any,
-    operation_id: str | None,
-    schema: str | None,
-    errors: RetainExtractionErrors,
-) -> None:
-    """Persist non-fatal Batch API extraction errors into operation result_metadata."""
-    if not pool or not operation_id or errors.count == 0:
-        return
-
-    from ..db_utils import acquire_with_retry
-    from ..task_backend import fq_table
-
-    # `errors` is the complete set for this extraction run, so overwrite the
-    # extraction_errors_* keys rather than folding in what's already stored. On
-    # batch crash recovery the resumed batch reprocesses every result and
-    # recomputes `errors` from scratch; reading + merging the prior run's
-    # counters here would double-count them. The SQL `||` merge still preserves
-    # unrelated keys (e.g. batch_id) already on result_metadata.
-    table = fq_table("async_operations", schema)
-    async with acquire_with_retry(pool) as conn:
-        await conn.execute(
-            f"""
-            UPDATE {table}
-            SET result_metadata = COALESCE(result_metadata, '{{}}'::jsonb) || $2::jsonb,
-                updated_at = now()
-            WHERE operation_id = $1
-            """,
-            operation_id,
-            json.dumps(errors.to_dict()),
-        )
-
-
 async def extract_facts_from_contents_batch_api(
    contents: list[RetainContent],
    llm_config,
@@ -1835,7 +1731,6 @@ async def extract_facts_from_contents_batch_api(
                item.context,
                item.metadata or None,
                agent_name,
-                mission_preamble=_retain_mission_preamble(config),
            )

            # Build request body using helper function
@@ -1921,7 +1816,6 @@ async def extract_facts_from_contents_batch_api(
    all_facts_from_llm = []
    chunks_metadata = []
    total_usage = TokenUsage()
-    extraction_errors = RetainExtractionErrors()

    for chunk_idx, (chunk_content, content_index, chunk_index_in_content, event_date, context) in enumerate(
        all_chunks_info
@@ -1930,9 +1824,7 @@ async def extract_facts_from_contents_batch_api(
        result = results_by_id.get(custom_id)

        if not result:
-            message = f"{custom_id}: missing batch result"
-            logger.warning(message)
-            extraction_errors.add(message)
+            logger.warning(f"Missing result for {custom_id}, skipping")
            chunks_metadata.append(
                ChunkMetadata(
                    chunk_text=chunk_content, fact_count=0, content_index=content_index, chunk_index=chunk_idx
@@ -1942,9 +1834,7 @@ async def extract_facts_from_contents_batch_api(

        # Check for errors
        if result.get("error"):
-            message = f"{custom_id}: {result['error']}"
-            logger.error(f"Error in {message}")
-            extraction_errors.add(message)
+            logger.error(f"Error in {custom_id}: {result['error']}")
            chunks_metadata.append(
                ChunkMetadata(
                    chunk_text=chunk_content, fact_count=0, content_index=content_index, chunk_index=chunk_idx
@@ -1957,9 +1847,7 @@ async def extract_facts_from_contents_batch_api(
        choices = response_body.get("choices", [])

        if not choices:
-            message = f"{custom_id}: no choices in response"
-            logger.warning(message)
-            extraction_errors.add(message)
+            logger.warning(f"No choices in response for {custom_id}")
            chunks_metadata.append(
                ChunkMetadata(
                    chunk_text=chunk_content, fact_count=0, content_index=content_index, chunk_index=chunk_idx
@@ -1974,9 +1862,7 @@ async def extract_facts_from_contents_batch_api(
        try:
            extraction_response_json = json.loads(content_str)
        except json.JSONDecodeError as e:
-            message = f"{custom_id}: failed to parse JSON: {e}"
-            logger.error(message)
-            extraction_errors.add(message)
+            logger.error(f"Failed to parse JSON for {custom_id}: {e}")
            chunks_metadata.append(
                ChunkMetadata(
                    chunk_text=chunk_content, fact_count=0, content_index=content_index, chunk_index=chunk_idx
@@ -2149,9 +2035,7 @@ async def extract_facts_from_contents_batch_api(
                fact = Fact(fact=combined_text, fact_type=fact_type, **fact_data)
                chunk_facts.append(fact)
            except Exception as e:
-                message = f"{custom_id}: failed to create Fact model for fact {i}: {e}"
-                logger.error(message)
-                extraction_errors.add(message)
+                logger.error(f"Failed to create Fact model for fact {i}: {e}")
                continue

        all_facts_from_llm.extend(chunk_facts)
@@ -2216,8 +2100,6 @@ async def extract_facts_from_contents_batch_api(
    # Step 8: Auto-tag facts from label groups with tag=True
    _inject_label_tags(extracted_facts, config)

-    await _write_batch_extraction_errors(pool, operation_id, schema, extraction_errors)
-
    logger.info(f"Batch API extracted {len(extracted_facts)} facts from {len(all_chunks_info)} chunks")

    return extracted_facts, chunks_metadata, total_usage
@@ -147,13 +147,12 @@ async def ensure_bank_exists(conn, bank_id: str, ops=None) -> None:
    internal_id = uuid.uuid4()
    inserted = await conn.fetchval(
        f"""
-        INSERT INTO {fq_table("banks")} (bank_id, name, disposition, mission, internal_id)
-        VALUES ($1, $2, $3::jsonb, $4, $5)
+        INSERT INTO {fq_table("banks")} (bank_id, disposition, mission, internal_id)
+        VALUES ($1, $2::jsonb, $3, $4)
        ON CONFLICT (bank_id) DO NOTHING
        RETURNING bank_id
        """,
        bank_id,
-        bank_id,  # Default name is the bank_id (matches get_or_create_bank_profile)
        json.dumps(DEFAULT_DISPOSITION),
        "",
        internal_id,
@@ -574,10 +574,12 @@ async def compute_semantic_links_ann(
    # the transaction end handles both.
    rows: list = []
    async with conn.transaction():
-        # Transaction-local ANN tuning. The dispatcher only returns GUCs that
-        # are safe to apply at session/transaction scope for the configured
-        # backend. VectorChord probe values are index-shaped, so vchordrq uses
-        # index storage fallback parameters instead of a blanket SET LOCAL.
+        # Transaction-local ANN tuning. Each supported backend exposes its own
+        # GUC (hnsw.ef_search on pgvector, vchordrq.probes on vchord); the
+        # dispatcher returns the right knob for the configured backend with a
+        # value tuned for top-50 semantic link creation (lower recall but much
+        # lower latency than the recall-side default). SET LOCAL auto-reverts
+        # at commit, so we don't pollute the pool for subsequent queries.
        for guc, value in ann_search_tuning_settings(configured_vector_extension(), kind="low_latency"):
            await conn.execute(f"SET LOCAL {guc} = {value}")

@@ -597,35 +599,23 @@ async def compute_semantic_links_ann(
            t_query = time_mod.time()
            seed_count = sum(1 for ft in fact_types if ft == fact_type)
            logger.debug(f"[ANN] Querying fact_type={fact_type}: {seed_count} seeds")
-            # Cast each seed's text embedding to `vector` exactly once in a
-            # MATERIALIZED CTE. Casting inside the LATERAL (s.emb_text::vector)
-            # re-parses the ~5KB embedding string for every candidate row the
-            # probe touches — seeds × bank_units text-parses per batch, which
-            # dominated the whole job on small banks (see #1919: ~50 seeds over
-            # ~1k units took 1.5-3.7s, ~25-48x slower than casting once). The
-            # stable `vector` column also lets the planner consider an HNSW
-            # index scan, which a cast expression inhibits.
            ft_rows = await conn.fetch(
                f"""
-                WITH seeds AS MATERIALIZED (
-                    SELECT unit_id, emb_text::vector AS emb
-                    FROM _ann_seeds
-                    WHERE fact_type = $2
-                )
                SELECT s.unit_id       AS from_id,
                       n.id::text      AS to_id,
                       n.similarity
-                FROM seeds s
+                FROM _ann_seeds s
                CROSS JOIN LATERAL (
                    SELECT mu.id,
-                           1 - (mu.embedding <=> s.emb) AS similarity
+                           1 - (mu.embedding <=> s.emb_text::vector) AS similarity
                    FROM {fq_table("memory_units")} mu
                    WHERE mu.bank_id = $1
                      AND mu.fact_type = $2
                      AND mu.embedding IS NOT NULL
-                    ORDER BY mu.embedding <=> s.emb
+                    ORDER BY mu.embedding <=> s.emb_text::vector
                    LIMIT $3
                ) n
+                WHERE s.fact_type = $2
                """,
                bank_id,
                fact_type,
@@ -634,7 +624,7 @@ async def compute_semantic_links_ann(
            logger.debug(f"[ANN] fact_type={fact_type}: {len(ft_rows)} rows in {time_mod.time() - t_query:.3f}s")
            rows.extend(ft_rows)
    # Transaction commits here. _ann_seeds is dropped (ON COMMIT DROP).
-    # Transaction-local ANN tuning reverts (SET LOCAL).
+    # hnsw.ef_search reverts (SET LOCAL).

    for row in rows:
        sim = float(min(1.0, max(0.0, row["similarity"])))
@@ -11,7 +11,6 @@ import logging
 import time
 import uuid
 from collections.abc import Awaitable, Callable
-from dataclasses import dataclass
 from datetime import UTC, datetime
 from typing import Any

@@ -112,25 +111,6 @@ RetainOutboxCallback = Callable[[asyncpg.Connection], Awaitable[None]]
 RetainOutboxCallbackFactory = Callable[[list[RetainContentDict]], RetainOutboxCallback | None]


-def _resolve_narrator(profile_name: str, bank_id: str) -> str | None:
-    """Resolve the narrator (memory owner) used to prime fact extraction.
-
-    The narrator is injected as a "Narrator: {name}" line in fact extraction and
-    is stamped into the who-dimension of every first-person fact — and the
-    observations later consolidated from those facts. That is correct for a named
-    agent retaining its own logs, but harmful when ``name`` is just the bank_id:
-    on auto-create the bank ``name`` defaults to ``bank_id``, which is typically a
-    routing key (e.g. ``my-agent::channel-456::user-789``), not a speaker. Priming
-    extraction with a routing key embeds that string into stored fact text and
-    pollutes downstream observations (issue #1680). Suppress it in that case.
-
-    Returns the narrator name, or ``None`` to omit the Narrator line entirely.
-    """
-    if profile_name == bank_id:
-        return None
-    return profile_name
-
-
 def _build_retain_params(contents_dicts, document_tags=None, doc_contents=None):
    """Build retain_params and merged_tags from content dicts."""
    if doc_contents is not None:
@@ -424,7 +404,6 @@ async def retain_batch(
    db_semaphore: "asyncio.Semaphore | None" = None,
    document_body_override: str | None = None,
    chunk_index_offset: int = 0,
-    progress_callback: "Callable[..., Awaitable[None]] | None" = None,
 ) -> tuple[list[list[str]], TokenUsage, int | None]:
    """
    Process a batch of content through the retain pipeline.
@@ -460,9 +439,7 @@ async def retain_batch(

    # Get bank profile
    profile = await bank_utils.get_bank_profile(pool, bank_id)
-    # Suppress the narrator when name == bank_id (auto-create default) — see
-    # _resolve_narrator for why a routing-key narrator pollutes extraction (#1680).
-    agent_name = _resolve_narrator(profile["name"], bank_id)
+    agent_name = profile["name"]

    # Convert dicts to RetainContent objects
    contents = _build_contents(contents_dicts, document_tags)
@@ -715,7 +692,6 @@ async def retain_batch(
        db_semaphore=db_semaphore,
        document_body_override=document_body_override,
        chunk_index_offset=chunk_index_offset,
-        progress_callback=progress_callback,
    )


@@ -855,7 +831,6 @@ async def _streaming_retain_batch(
    db_semaphore: "asyncio.Semaphore | None" = None,
    document_body_override: str | None = None,
    chunk_index_offset: int = 0,
-    progress_callback: "Callable[..., Awaitable[None]] | None" = None,
 ) -> tuple[list[list[str]], TokenUsage]:
    """
    Process a large document in streaming mini-batches to bound memory usage.
@@ -977,29 +952,19 @@ async def _streaming_retain_batch(
                tags=source.tags,
                observation_scopes=source.observation_scopes,
            )
-            # Attribute this chunk's extraction LLM call to its document, so the
-            # trace row carries document_id (a document accrues one such trace
-            # per retain/re-retain). Per-call: the operation-level trace context
-            # is shared across a batch's documents.
-            from ..llm_trace import reset_call_metadata, set_call_metadata
-
-            meta_token = set_call_metadata({"document_id": effective_doc_id})
-            try:
-                extracted, processed, chunk_meta, usage = await _extract_and_embed(
-                    [content],
-                    llm_config,
-                    agent_name,
-                    config,
-                    embeddings_model,
-                    format_date_fn,
-                    fact_type_override,
-                    log_buffer,
-                    pool,
-                    operation_id,
-                    schema,
-                )
-            finally:
-                reset_call_metadata(meta_token)
+            extracted, processed, chunk_meta, usage = await _extract_and_embed(
+                [content],
+                llm_config,
+                agent_name,
+                config,
+                embeddings_model,
+                format_date_fn,
+                fact_type_override,
+                log_buffer,
+                pool,
+                operation_id,
+                schema,
+            )
            await chunk_queue.put((global_idx, content, extracted, processed, chunk_meta, usage))
            # Memory: release the chunk text from the shared list now that it's
            # been extracted and queued. The queued RetainContent holds its own copy.
@@ -1034,25 +999,6 @@ async def _streaming_retain_batch(
    async def _db_consumer() -> None:
        batch: list[tuple] = []
        consumer_batch_idx = 0
-        chunks_committed = 0
-
-        # Best-effort durable progress: how many chunks of this document have been
-        # extracted+committed so far. Written per consumer batch so an operator polling
-        # the retain operation sees "storing 200/1200 chunks" advancing instead of a
-        # single opaque sub-batch tick. Never lets a heartbeat failure break retain.
-        async def _emit_chunk_progress() -> None:
-            if not (progress_callback and operation_id):
-                return
-            try:
-                await progress_callback(
-                    operation_id,
-                    stage="storing",
-                    processed=chunks_committed,
-                    total=total_chunks,
-                    detail={"facts_committed": len(all_unit_ids)},
-                )
-            except Exception:
-                logger.debug("retain chunk-progress write failed", exc_info=True)

        while True:
            item = await chunk_queue.get()
@@ -1064,8 +1010,6 @@ async def _streaming_retain_batch(
                        consumer_batch_idx,
                        is_last=True,
                    )
-                    chunks_committed += len(batch)
-                    await _emit_chunk_progress()
                break

            batch.append(item)
@@ -1085,8 +1029,6 @@ async def _streaming_retain_batch(
                    is_last=False,
                )
                consumer_batch_idx += 1
-                chunks_committed += len(batch)
-                await _emit_chunk_progress()
                batch = []

    async def _process_db_batch(
@@ -1239,17 +1181,20 @@ async def _streaming_retain_batch(
            async with acquire_with_retry(pool) as conn:
                async with conn.transaction():
                    # --- Document ownership gate ---
-                    # Ensure the document row exists, lock it to serialize all
-                    # concurrent same-document writers, and read its pre-existing
-                    # hash. The lock prevents interleaved retains from corrupting
-                    # each other in handle_document_tracking; the returned hash
-                    # ('__pending__' for a freshly inserted row) drives the
-                    # takeover check for later batches below. The PG/Oracle split
-                    # lives in the ops layer because Oracle can't do this upsert +
-                    # RETURNING in a single statement.
-                    existing_hash = await pool.ops.lock_document_for_write(
-                        conn,
-                        fq_table("documents"),
+                    # Lock the document row to serialize all concurrent writers.
+                    # SELECT ... FOR UPDATE doesn't lock non-existent rows, so we
+                    # first ensure the row exists with a lightweight upsert, THEN lock it.
+                    # The content_hash='__pending__' placeholder is immediately overwritten
+                    # by handle_document_tracking or upsert_document_metadata below.
+                    await conn.execute(
+                        f"INSERT INTO {fq_table('documents')} (id, bank_id, original_text, content_hash) "
+                        f"VALUES ($1, $2, '', '__pending__') "
+                        f"ON CONFLICT (id, bank_id) DO NOTHING",
+                        effective_doc_id,
+                        bank_id,
+                    )
+                    existing_hash = await conn.fetchval(
+                        f"SELECT content_hash FROM {fq_table('documents')} WHERE id = $1 AND bank_id = $2 FOR UPDATE",
                        effective_doc_id,
                        bank_id,
                    )
@@ -1556,35 +1501,6 @@ async def _streaming_retain_batch(
 # ---------------------------------------------------------------------------


-@dataclass
-class _ChunkDiff:
-    """Classification of chunk indices when diffing new content vs stored chunks."""
-
-    unchanged: list[int]
-    changed: list[int]
-    new: list[int]
-    removed: list[int]
-
-
-def _classify_chunk_diff(existing_by_index: dict[int, Any], new_hashes: dict[int, str]) -> _ChunkDiff:
-    """Classify chunk indices by comparing freshly computed ``new_hashes``
-    (index -> content hash) against the currently stored chunks
-    (``existing_by_index``: index -> chunk row)."""
-    diff = _ChunkDiff(unchanged=[], changed=[], new=[], removed=[])
-    for idx, new_hash in new_hashes.items():
-        existing = existing_by_index.get(idx)
-        if existing and existing.content_hash == new_hash:
-            diff.unchanged.append(idx)
-        elif existing:
-            diff.changed.append(idx)
-        else:
-            diff.new.append(idx)
-    for idx in existing_by_index:
-        if idx not in new_hashes:
-            diff.removed.append(idx)
-    return diff
-
-
 async def _try_delta_retain(
    pool: Any,
    embeddings_model,
@@ -1654,11 +1570,18 @@ async def _try_delta_retain(
    existing_by_index = {c.chunk_index: c for c in existing_chunks}
    new_hashes = {idx: chunk_storage.compute_chunk_hash(text) for idx, text in new_chunks_with_contents.items()}

-    diff = _classify_chunk_diff(existing_by_index, new_hashes)
-    unchanged_indices = diff.unchanged
-    changed_indices = diff.changed
-    new_indices = diff.new
-    removed_indices = diff.removed
+    unchanged_indices, changed_indices, new_indices, removed_indices = [], [], [], []
+    for idx, new_hash in new_hashes.items():
+        existing = existing_by_index.get(idx)
+        if existing and existing.content_hash == new_hash:
+            unchanged_indices.append(idx)
+        elif existing:
+            changed_indices.append(idx)
+        else:
+            new_indices.append(idx)
+    for idx in existing_by_index:
+        if idx not in new_hashes:
+            removed_indices.append(idx)

    log_buffer.append(
        f"[delta] Chunk diff: {len(unchanged_indices)} unchanged, "
@@ -1705,85 +1628,20 @@ async def _try_delta_retain(
            document_body_override=document_body_override,
        )

-    # Freshness recheck BEFORE the (expensive) LLM extraction.
-    #
-    # We snapshotted the document hash and chunks outside any lock. A concurrent
-    # retain for the same document may have committed a new version while we were
-    # chunking and diffing. Re-read the current hash; if it changed, recompute the
-    # diff against the now-committed chunk state. If the concurrent writer already
-    # produced content identical to ours, there is nothing left to extract — skip
-    # the LLM call entirely (metadata-only). If it still differs, fall back to the
-    # streaming path (which dedups per-chunk and re-locks the document).
-    #
-    # This narrows — but cannot fully close — the race window: a writer can still
-    # commit during our extraction. The post-extraction hash gate inside the write
-    # transaction remains the correctness backstop; this check exists purely to
-    # avoid burning LLM tokens on work a concurrent request already did.
-    async with acquire_with_retry(pool) as conn:
-        recheck_hash = await conn.fetchval(
-            f"SELECT content_hash FROM {fq_table('documents')} WHERE id = $1 AND bank_id = $2",
-            effective_doc_id,
-            bank_id,
-        )
-    if recheck_hash is not None and doc_hash_at_load is not None and recheck_hash != doc_hash_at_load:
-        log_buffer.append(
-            f"[delta] Document {effective_doc_id} changed before extraction "
-            f"(concurrent retain) — rechecking diff against current state"
-        )
-        async with acquire_with_retry(pool) as conn:
-            current_chunks = await chunk_storage.load_existing_chunks(conn, bank_id, effective_doc_id)
-        if not current_chunks or any(c.content_hash is None for c in current_chunks):
-            log_buffer.append("[delta] Recheck: current chunks unavailable — falling back to full retain")
-            logger.info("\n" + "\n".join(log_buffer) + "\n")
-            return None
-        current_by_index = {c.chunk_index: c for c in current_chunks}
-        recheck = _classify_chunk_diff(current_by_index, new_hashes)
-        if not (recheck.changed or recheck.new or recheck.removed):
-            log_buffer.append(
-                "[delta] Recheck: concurrent retain already stored identical content — "
-                "skipping extraction, updating metadata only"
-            )
-            return await _delta_metadata_only(
-                pool,
-                bank_id,
-                contents_dicts,
-                contents,
-                effective_doc_id,
-                document_tags,
-                log_buffer,
-                start_time,
-                outbox_callback,
-                document_body_override=document_body_override,
-            )
-        log_buffer.append(
-            f"[delta] Recheck: {len(recheck.changed) + len(recheck.new) + len(recheck.removed)} chunks still differ — "
-            f"falling back to full retain"
-        )
-        logger.info("\n" + "\n".join(log_buffer) + "\n")
-        return None
-
-    # Extract facts and generate embeddings (shared pipeline). Attribute these
-    # extraction calls to the document so the delta re-retain's trace also binds
-    # to it (a document accrues one trace per full/delta retain).
-    from ..llm_trace import reset_call_metadata, set_call_metadata
-
-    meta_token = set_call_metadata({"document_id": effective_doc_id})
-    try:
-        extracted_facts, processed_facts, new_chunk_metadata, usage = await _extract_and_embed(
-            delta_contents,
-            llm_config,
-            agent_name,
-            config,
-            embeddings_model,
-            format_date_fn,
-            fact_type_override,
-            log_buffer,
-            pool,
-            operation_id,
-            schema,
-        )
-    finally:
-        reset_call_metadata(meta_token)
+    # Extract facts and generate embeddings (shared pipeline)
+    extracted_facts, processed_facts, new_chunk_metadata, usage = await _extract_and_embed(
+        delta_contents,
+        llm_config,
+        agent_name,
+        config,
+        embeddings_model,
+        format_date_fn,
+        fact_type_override,
+        log_buffer,
+        pool,
+        operation_id,
+        schema,
+    )

    # Database transaction
    result_unit_ids: list[list[str]] = []
@@ -7,27 +7,6 @@ from typing import Any
 from .types import MergedCandidate, RetrievalResult


-def cap_per_source(results: list[RetrievalResult], cap: int) -> list[RetrievalResult]:
-    """Truncate a single retrieval arm to its top-``cap`` results.
-
-    Applied per source (semantic, BM25, graph, temporal) before fusion so that
-    one over-expanding backend cannot crowd out the others when the merged pool
-    is later trimmed to the reranker's global candidate budget. The caller is
-    responsible for sorting ``results`` by relevance first; this only slices.
-
-    Args:
-        results: Results for a single source, already sorted best-first.
-        cap: Maximum results to keep. ``0`` (or negative) disables the cap.
-
-    Returns:
-        The original list when the cap is disabled or not exceeded, otherwise a
-        truncated copy of the top ``cap`` results.
-    """
-    if cap <= 0 or len(results) <= cap:
-        return results
-    return results[:cap]
-
-
 def reciprocal_rank_fusion(result_lists: list[list[RetrievalResult]], k: int = 60) -> list[MergedCandidate]:
    """
    Merge multiple ranked result lists using Reciprocal Rank Fusion.
@@ -98,66 +77,6 @@ def reciprocal_rank_fusion(result_lists: list[list[RetrievalResult]], k: int = 6
    return merged_results


-def interleave_fusion(result_lists: list[list[RetrievalResult]]) -> list[MergedCandidate]:
-    """Round-robin (interleaved) fusion — an alternative to RRF for dedup-style recall.
-
-    RRF scores a doc by the *sum* of its reciprocal ranks across arms, so a result
-    that is #1 in one arm but absent/low in the others gets averaged down. That is
-    exactly the consolidation-dedup failure mode: the near-identical existing
-    observation (the "twin" to merge into) is semantic rank #1, yet shares no
-    source-fact graph link and little lexical overlap, so RRF drops it below the
-    recall budget cutoff and the LLM never sees it → creates a duplicate.
-
-    Interleave instead *guarantees every arm's top hits a slot*: take each arm's
-    #1, then each arm's #2, … in arm-priority order, de-duplicating, until all
-    results are placed. The arm priority is the order of ``result_lists``
-    (semantic, bm25, graph, temporal), so semantic #1 is always first.
-
-    ``rrf_score`` is assigned strictly decreasing by final interleave position so
-    downstream order-by-score sorts preserve the interleave order; ``source_ranks``
-    mirrors the RRF bookkeeping (each doc's rank within every arm it appears in).
-    """
-    source_names = ["semantic", "bm25", "graph", "temporal"]
-    source_ranks: dict[str, dict[str, int]] = {}
-    all_retrievals: dict[str, RetrievalResult] = {}
-
-    for source_idx, results in enumerate(result_lists):
-        source_name = source_names[source_idx] if source_idx < len(source_names) else f"source_{source_idx}"
-        for rank, retrieval in enumerate(results, start=1):
-            if not isinstance(retrieval, RetrievalResult):
-                raise TypeError(
-                    f"Expected RetrievalResult but got {type(retrieval).__name__} in {source_name} results at rank {rank}"
-                )
-            doc_id = retrieval.id
-            all_retrievals.setdefault(doc_id, retrieval)
-            source_ranks.setdefault(doc_id, {})[f"{source_name}_rank"] = rank
-
-    # Round-robin pick across arms in priority order: all #1s, then all #2s, ...
-    ordered_ids: list[str] = []
-    seen: set[str] = set()
-    max_len = max((len(r) for r in result_lists), default=0)
-    for r in range(max_len):
-        for results in result_lists:
-            if r < len(results):
-                doc_id = results[r].id
-                if doc_id not in seen:
-                    seen.add(doc_id)
-                    ordered_ids.append(doc_id)
-
-    n = len(ordered_ids)
-    return [
-        MergedCandidate(
-            retrieval=all_retrievals[doc_id],
-            # Strictly decreasing by interleave position → sorting desc by rrf_score
-            # reproduces the interleave order downstream.
-            rrf_score=float(n - pos),
-            rrf_rank=pos + 1,
-            source_ranks=source_ranks[doc_id],
-        )
-        for pos, doc_id in enumerate(ordered_ids)
-    ]
-
-
 def normalize_scores_on_deltas(results: list[dict[str, Any]], score_keys: list[str]) -> list[dict[str, Any]]:
    """
    Normalize scores based on deltas (min-max normalization within result set).
@@ -1,117 +0,0 @@
-"""Per-strategy recall boosting.
-
-A deployment can prioritise one retrieval arm (semantic, bm25, graph, temporal)
-over the others via ``HINDSIGHT_API_RECALL_STRATEGY_BOOSTS``, expressed as a
-human priority *level* rather than an opaque number — e.g. ``graph:high`` to
-strongly favour graph hits.
-
-A level is chosen instead of a raw weight because the boost is applied in two
-structurally different places that live on different score scales, so a single
-number could not mean the same thing in both. The level maps to a tuned
-:class:`BoostWeights` pair:
-
-1. **Before the reranker cap** — :func:`boosted_rrf_score` uses ``BoostWeights.rrf``
-   as a weighted-RRF multiplier on the boosted arm's rank contribution, so its
-   candidates survive the global reranker candidate budget instead of being
-   trimmed by raw RRF score. Rank-aware: a candidate ranked #1 in the boosted
-   arm is protected more than one ranked #200.
-
-2. **After the reranker** — :func:`additive_strategy_boost` uses
-   ``BoostWeights.additive`` as a flat bump to the final ranking weight (which
-   sits in ~[0, 1] after cross-encoder + recency/temporal scoring), nudging the
-   boosted arm's candidates up the final ordering.
-
-Both functions are no-ops when ``boosts`` is empty, preserving current behaviour.
-"""
-
-from dataclasses import dataclass
-
-from .types import MergedCandidate
-
-
-@dataclass(frozen=True)
-class BoostWeights:
-    """Per-stage boost magnitudes for one priority level.
-
-    The two fields live on different scales on purpose (see module docstring):
-    ``rrf`` multiplies an arm's ``1/(k+rank)`` RRF contribution; ``additive`` is
-    added directly to the post-rerank weight in ~[0, 1].
-    """
-
-    rrf: float
-    additive: float
-
-
-# Priority level -> per-stage boost magnitudes. Tuned against real recall traces
-# (LoCoMo bank, 336 merged candidates → 300-cap, local ms-marco cross-encoder):
-#
-# Stage 1 (rrf, weighted-RRF multiplier on the arm's 1/(k+rank) contribution).
-# The observed 300-cap boundary RRF score was ~0.0055; a graph-only candidate
-# falls below it past graph-rank ~120. The multipliers map to that boundary:
-#   low=1.0   doubles the arm's vote — rescues at-risk candidates from the cut
-#             (graph-rank 150: 0.0048 → 0.0095) without reshuffling much.
-#   medium=3.0 promotes them into the middle of the pool (~rank 60).
-#   high=6.0   makes the boosted arm dominate the top of the candidate pool.
-#
-# Stage 2 (additive, flat bump to the post-rerank weight in [0, 1]). The local
-# cross-encoder is sharply bimodal: strong direct matches score 0.5–0.999, while
-# everything else — including graph hits the CE undervalues, which is exactly
-# what we boost — collapses near 0. So the additive lifts a ~0 candidate up the
-# weight scale. Levels are calibrated as relevance thresholds it can outrank:
-#   low=0.05  nudges above the near-0 tail; loses to any real CE match.
-#   medium=0.2 competes with weak/moderate matches.
-#   high=0.5  wins over most semantic matches (honouring "prioritise graph over
-#             semantic"); only a strong direct match (>0.5 normalized) still wins.
-#
-# The keys are the user-facing contract; config.py validates env input against
-# them (kept in sync by a guard test).
-BOOST_LEVELS: dict[str, BoostWeights] = {
-    "low": BoostWeights(rrf=1.0, additive=0.05),
-    "medium": BoostWeights(rrf=3.0, additive=0.2),
-    "high": BoostWeights(rrf=6.0, additive=0.5),
-}
-
-
-def boosted_rrf_score(candidate: MergedCandidate, boosts: dict[str, str], k: int = 60) -> float:
-    """Return ``candidate``'s RRF score plus a weighted-RRF boost delta.
-
-    For each boosted arm the candidate appeared in, adds ``level.rrf * 1/(k+rank)``
-    — i.e. scales that arm's RRF contribution by the level's multiplier. Staying
-    in RRF units keeps the boost comparable to the base score and rank-aware.
-
-    Args:
-        candidate: Merged candidate carrying ``rrf_score`` and ``source_ranks``.
-        boosts: Map of strategy name -> priority level. Empty means no boost.
-        k: RRF constant; must match the value used during fusion.
-
-    Returns:
-        The (possibly) boosted score to sort by. Equal to ``rrf_score`` when no
-        boosted arm surfaced this candidate.
-    """
-    if not boosts:
-        return candidate.rrf_score
-    delta = 0.0
-    for strategy, level in boosts.items():
-        rank = candidate.source_ranks.get(f"{strategy}_rank")
-        if rank is not None:
-            delta += BOOST_LEVELS[level].rrf * (1.0 / (k + rank))
-    return candidate.rrf_score + delta
-
-
-def additive_strategy_boost(source_ranks: dict[str, int], boosts: dict[str, str]) -> float:
-    """Return the flat additive boost for a candidate given its source ranks.
-
-    Sums the ``additive`` magnitude of every boosted arm that surfaced the
-    candidate. Flat by design: the bump does not depend on the candidate's rank
-    within the arm, matching the post-rerank "additive boost" semantics.
-
-    Args:
-        source_ranks: ``{"graph_rank": 3, "semantic_rank": 50, ...}`` from RRF.
-        boosts: Map of strategy name -> priority level. Empty means no boost.
-
-    Returns:
-        The additive boost (0.0 when no boosted arm surfaced this candidate).
-    """
-    if not boosts:
-        return 0.0
-    return sum(BOOST_LEVELS[level].additive for strategy, level in boosts.items() if f"{strategy}_rank" in source_ranks)
@@ -160,29 +160,13 @@ class CrossEncoderReranker:

        import asyncio

-        from hindsight_api.config import ENV_MODEL_INIT_TIMEOUT, get_config
-
        cross_encoder = self.cross_encoder
        # For local providers, run in thread pool to avoid blocking event loop
        if cross_encoder.provider_name == "local":
            loop = asyncio.get_event_loop()
-            init = loop.run_in_executor(None, lambda: asyncio.run(cross_encoder.initialize()))
+            await loop.run_in_executor(None, lambda: asyncio.run(cross_encoder.initialize()))
        else:
-            init = cross_encoder.initialize()
-
-        # Cap lazy init with the same wall-clock timeout used at startup so a
-        # hung model download surfaces as a clear error on the request that
-        # triggered it, rather than hanging the caller forever.
-        init_timeout = get_config().model_init_timeout
-        try:
-            await asyncio.wait_for(init, timeout=init_timeout)
-        except TimeoutError as e:
-            raise RuntimeError(
-                f"Cross-encoder initialization did not complete within {init_timeout:g}s. "
-                f"The reranker model is likely blocked loading — e.g. an offline model "
-                f"download. Increase {ENV_MODEL_INIT_TIMEOUT} if the first-time download "
-                f"legitimately needs more time."
-            ) from e
+            await cross_encoder.initialize()
        self._initialized = True

    async def rerank(self, query: str, candidates: list[MergedCandidate]) -> list[ScoredResult]:
@@ -137,7 +137,6 @@ async def retrieve_semantic_bm25_combined(
    """
    result_dict: dict[str, tuple[list[RetrievalResult], list[RetrievalResult]]] = {ft: ([], []) for ft in fact_types}

-    config = get_config()
    tokens = tokenize_query(query_text)

    # Over-fetch for HNSW approximation; semantic results trimmed to limit in Python.
@@ -149,6 +148,8 @@ async def retrieve_semantic_bm25_combined(
    )
    table = fq_table("memory_units")

+    config = get_config()
+
    # Use the SQL dialect to build backend-specific query arms, avoiding
    # inline if/else branches for each database.
    # Use getattr for backward compat: raw asyncpg connections (used in some
@@ -200,7 +201,6 @@ async def retrieve_semantic_bm25_combined(
            embedding_param="$1",
            bank_id_param="$2",
            fetch_limit=hnsw_fetch,
-            min_similarity=config.semantic_min_similarity,
            tags_clause=tags_clause,
            groups_clause=groups_clause,
            extra_where=created_range_clause,
@@ -226,7 +226,6 @@ async def retrieve_semantic_bm25_combined(
                    arm_index=i,
                    text_search_extension=text_ext,
                    bm25_language=config.text_search_extension_native_language,
-                    bm25_min_score=config.bm25_min_score,
                    extra_where=created_range_clause,
                )
            )
@@ -274,7 +273,6 @@ async def retrieve_semantic_bm25_combined(
                    embedding_param="$1",
                    bank_id_param="$2",
                    fetch_limit=hnsw_fetch,
-                    min_similarity=config.semantic_min_similarity,
                    tags_clause=fb_tags_clause,
                    groups_clause=fb_groups_clause,
                    extra_where=fb_created_clause,
@@ -309,66 +307,6 @@ async def retrieve_semantic_bm25_combined(
    return result_dict


-# Temporal entry-point selection tuning.
-_TEMPORAL_POOL_SIZE = 60  # ANN candidates fetched per fact_type before coverage selection
-_TEMPORAL_ENTRY_POINTS = 10  # entry points kept per fact_type after coverage selection
-_TEMPORAL_COVERAGE_BUCKETS = 8  # time-buckets the window is divided into for coverage
-
-
-def _coalesce_date(row: Any) -> datetime | None:
-    """The unit's effective time — matches COALESCE(occurred_start, mentioned_at, occurred_end)."""
-    return row["occurred_start"] or row["mentioned_at"] or row["occurred_end"]
-
-
-def _select_with_temporal_coverage(
-    pool: list,
-    start_date: datetime,
-    end_date: datetime,
-    limit: int,
-    n_buckets: int,
-) -> list:
-    """Pick `limit` entry points from a similarity-ranked pool, spread across the window.
-
-    The window [start_date, end_date] is split into `n_buckets` equal time-buckets.
-    Candidates are taken round-robin across the buckets that contain them — the
-    best-similarity item from each populated bucket first, then the second-best from each,
-    and so on — so every populated slice of the window is represented before any slice
-    contributes a second item. Within a tier, higher-similarity items lead. When the
-    in-window dates are degenerate (all in one bucket — e.g. a batch stamped with a single
-    date) this collapses to plain similarity order.
-    """
-    if len(pool) <= limit:
-        return list(pool)
-
-    ranked = sorted(pool, key=lambda r: r["similarity"], reverse=True)
-    span = (end_date - start_date).total_seconds()
-
-    def _bucket(row: Any) -> int:
-        d = _coalesce_date(row)
-        if d is None or span <= 0:
-            return 0
-        if d.tzinfo is None:
-            d = d.replace(tzinfo=UTC)
-        frac = (d - start_date).total_seconds() / span
-        return max(0, min(int(frac * n_buckets), n_buckets - 1))
-
-    buckets: dict[int, list] = {}
-    for row in ranked:  # ranked is similarity-desc, so each bucket list inherits that order
-        buckets.setdefault(_bucket(row), []).append(row)
-
-    selected: list = []
-    tier = 0
-    while len(selected) < limit and any(len(b) > tier for b in buckets.values()):
-        # The tier-th best item from every bucket that still has one, strongest first.
-        tier_rows = [b[tier] for b in buckets.values() if len(b) > tier]
-        tier_rows.sort(key=lambda r: r["similarity"], reverse=True)
-        for row in tier_rows:
-            if len(selected) < limit:
-                selected.append(row)
-        tier += 1
-    return selected
-
-
 async def retrieve_temporal_combined(
    conn,
    query_emb_str: str,
@@ -412,12 +350,9 @@ async def retrieve_temporal_combined(
        end_date = end_date.replace(tzinfo=UTC)

    # Build tags clause
-    # Entry-point query: fixed params are $1-$5 (emb, bank, start, end, threshold), tags at $6.
-    # fact_type is inlined as a literal per UNION ALL arm (not a bind) — this avoids `unnest`,
-    # which has no Oracle equivalent (the `<=>` operator and LIMIT are translated to Oracle by
-    # the backend on execute, but `unnest` is not). Mirrors retrieve_semantic_bm25_combined.
-    tags_clause = build_tags_where_clause_simple(tags, 6, match=tags_match)
-    tag_groups_param_start = 6 + (1 if tags else 0)
+    # Entry point query: fixed params are $1-$6, tags at $7
+    tags_clause = build_tags_where_clause_simple(tags, 7, match=tags_match)
+    tag_groups_param_start = 7 + (1 if tags else 0)
    groups_clause, groups_params, _ = build_tag_groups_where_clause(tag_groups, tag_groups_param_start)

    # created_at time range filter (after tags/groups)
@@ -433,88 +368,69 @@ async def retrieve_temporal_combined(
        created_range_clause += f" AND updated_at < ${_next_idx}"
        _next_idx += 1

-    params: list = [query_emb_str, bank_id, start_date, end_date, semantic_threshold]
+    params: list = [query_emb_str, bank_id, fact_types, start_date, end_date, semantic_threshold]
    if tags:
        params.append(tags)
    params.extend(groups_params)
    params.extend(created_range_params)

-    # Entry-point selection: similarity-gated, window-filtered, then narrowed for coverage.
-    #
-    # For each fact_type, ANN-rank the units whose time overlaps the window
-    # (ORDER BY embedding <=> query) and keep a pool of the most relevant
-    # (_TEMPORAL_POOL_SIZE). The planner serves this from the per-(bank, fact_type) vector
-    # index when the window is broad — the dense-metadata case, where the window matches
-    # most rows — and from the partial date indexes plus an exact sort when the window is
-    # narrow. Either way the work is bounded; neither path is a scan-and-sort of the whole
-    # match set.
-    #
-    # Selecting by *similarity* (not recency) is deliberate. The earlier form ranked the
-    # entire match set by COALESCE(occurred_start, mentioned_at, occurred_end) and kept the
-    # 50 most recent: that biased results toward the end of the window and, on banks with
-    # dense/near-uniform dates (e.g. a retain batch stamped with one date), the date key was
-    # degenerate so the "50 most recent" became a near-random sample that could drop the
-    # single most relevant in-window memory — and it degraded to a full scan + disk-spilling
-    # sort (30s+ on a 660k-row bank). The pool is then narrowed to _TEMPORAL_ENTRY_POINTS per
-    # fact_type by _select_with_temporal_coverage so the entry points span the window's range
-    # rather than clustering in one slice.
-    if not fact_types:
-        return {}
-
-    # One similarity-ranked, window-filtered arm per fact_type, UNION ALL'd — each arm has its
-    # own ORDER BY ... LIMIT so the per-(bank, fact_type) vector index can serve it. fact_type
-    # is inlined as a literal (controlled internal enum, never user input), matching
-    # retrieve_semantic_bm25_combined; this keeps the query free of `unnest`/LATERAL, which the
-    # Oracle backend cannot translate.
-    pool_cols = (
-        "id, text, context, event_date, occurred_start, occurred_end, mentioned_at, "
-        "fact_type, proof_count, document_id, chunk_id, tags, metadata"
+    # Two-phase entry point query:
+    # Phase 1 (date_ranked): rank by date only — no embedding computation — for all units in
+    #   the temporal window. This lets the planner use date indexes for filtering.
+    # Phase 2 (sim_ranked): join back to memory_units for only the top-50-per-type candidates
+    #   and compute embedding similarity for that small set (≤ 50 × len(fact_types) rows).
+    # This avoids computing embedding distances for potentially thousands of date-range rows.
+    entry_points = await conn.fetch(
+        f"""
+        WITH date_ranked AS MATERIALIZED (
+            SELECT id, fact_type,
+                   ROW_NUMBER() OVER (
+                       PARTITION BY fact_type
+                       ORDER BY COALESCE(occurred_start, mentioned_at, occurred_end) DESC NULLS LAST
+                   ) AS rn
+            FROM {fq_table("memory_units")}
+            WHERE bank_id = $2
+              AND fact_type = ANY($3)
+              AND embedding IS NOT NULL
+              AND (
+                  (occurred_start IS NOT NULL AND occurred_end IS NOT NULL
+                   AND occurred_start <= $5 AND occurred_end >= $4)
+                  OR
+                  (mentioned_at IS NOT NULL AND mentioned_at BETWEEN $4 AND $5)
+                  OR
+                  (occurred_start IS NOT NULL AND occurred_start BETWEEN $4 AND $5)
+                  OR
+                  (occurred_end IS NOT NULL AND occurred_end BETWEEN $4 AND $5)
+              )
+              {tags_clause}
+              {groups_clause}
+              {created_range_clause}
+        ),
+        sim_ranked AS (
+            SELECT mu.id, mu.text, mu.context, mu.event_date, mu.occurred_start, mu.occurred_end, mu.mentioned_at, mu.fact_type, mu.proof_count, mu.document_id, mu.chunk_id, mu.tags, mu.metadata,
+                   1 - (mu.embedding <=> $1::vector) AS similarity,
+                   ROW_NUMBER() OVER (PARTITION BY mu.fact_type ORDER BY mu.embedding <=> $1::vector) AS sim_rn
+            FROM date_ranked dr
+            JOIN {fq_table("memory_units")} mu ON mu.id = dr.id
+            WHERE dr.rn <= 50
+              AND (1 - (mu.embedding <=> $1::vector)) >= $6
+        )
+        SELECT id, text, context, event_date, occurred_start, occurred_end, mentioned_at, fact_type, proof_count, document_id, chunk_id, tags, metadata, similarity
+        FROM sim_ranked
+        WHERE sim_rn <= 10
+        """,
+        *params,
    )
-    table = fq_table("memory_units")
-    arms = [
-        f"""(
-        SELECT {pool_cols}, 1 - (embedding <=> $1::vector) AS similarity
-        FROM {table}
-        WHERE bank_id = $2
-          AND fact_type = '{ft}'
-          AND embedding IS NOT NULL
-          AND (
-              (occurred_start IS NOT NULL AND occurred_end IS NOT NULL
-               AND occurred_start <= $4 AND occurred_end >= $3)
-              OR
-              (mentioned_at IS NOT NULL AND mentioned_at BETWEEN $3 AND $4)
-              OR
-              (occurred_start IS NOT NULL AND occurred_start BETWEEN $3 AND $4)
-              OR
-              (occurred_end IS NOT NULL AND occurred_end BETWEEN $3 AND $4)
-          )
-          AND (1 - (embedding <=> $1::vector)) >= $5
-          {tags_clause}
-          {groups_clause}
-          {created_range_clause}
-        ORDER BY embedding <=> $1::vector
-        LIMIT {_TEMPORAL_POOL_SIZE}
-        )"""
-        for ft in fact_types
-    ]
-    pool_rows = await conn.fetch("\nUNION ALL\n".join(arms), *params)

-    if not pool_rows:
+    if not entry_points:
        return {ft: [] for ft in fact_types}

-    # Group the ANN pool by fact type, then narrow each to coverage-spread entry points.
-    pool_by_ft: dict[str, list] = {ft: [] for ft in fact_types}
-    for row in pool_rows:
-        ft = row["fact_type"]
-        if ft in pool_by_ft:
-            pool_by_ft[ft].append(row)
-
-    entries_by_ft: dict[str, list] = {
-        ft: _select_with_temporal_coverage(
-            rows, start_date, end_date, _TEMPORAL_ENTRY_POINTS, _TEMPORAL_COVERAGE_BUCKETS
-        )
-        for ft, rows in pool_by_ft.items()
-    }
+    # Group entry points by fact type
+    entries_by_ft: dict[str, list] = {ft: [] for ft in fact_types}
+    for ep in entry_points:
+        ft = ep["fact_type"]
+        if ft in entries_by_ft:
+            entries_by_ft[ft].append(ep)

    # Calculate shared temporal parameters
    total_days = (end_date - start_date).total_seconds() / 86400
@@ -582,13 +498,7 @@ async def retrieve_temporal_combined(
            tag_groups, spreading_groups_param_start, table_alias="mu."
        )

-        # Multi-hop temporal spreading expands a batch of seed ids with
-        # ``FROM unnest($2::uuid[])``, which has no Oracle equivalent. On backends
-        # without unnest, skip the spread: the temporal entry points are still
-        # returned above, and the semantic/keyword/graph retrievers cover the rest.
-        supports_unnest = getattr(conn, "backend_type", "postgresql") != "oracle"
-
-        while frontier and budget_remaining > 0 and iteration < max_iterations and supports_unnest:
+        while frontier and budget_remaining > 0 and iteration < max_iterations:
            iteration += 1
            batch_ids = frontier[:batch_size]
            frontier = frontier[batch_size:]
@@ -358,15 +358,12 @@ class SearchTracer:
        """
        self.rrf_merged = []
        for rank, (doc_id, data, rrf_meta) in enumerate(merged_results, start=1):
-            source_ranks = rrf_meta.get("source_ranks")
-            if source_ranks is None:
-                source_ranks = {key: value for key, value in rrf_meta.items() if key.endswith("_rank")}
            self.rrf_merged.append(
                RRFMergeResult(
                    node_id=doc_id,
                    text=data.get("text", ""),
                    rrf_score=rrf_meta.get("rrf_score", 0.0),
-                    source_ranks=source_ranks,
+                    source_ranks=rrf_meta.get("source_ranks", {}),
                    final_rrf_rank=rank,
                )
            )
@@ -371,7 +371,6 @@ class SQLDialect(ABC):
        embedding_param: str,
        bank_id_param: str,
        fetch_limit: int,
-        min_similarity: float,
        tags_clause: str = "",
        groups_clause: str = "",
        extra_where: str = "",
@@ -388,7 +387,6 @@ class SQLDialect(ABC):
            embedding_param: Parameter placeholder for query embedding.
            bank_id_param: Parameter placeholder for bank_id.
            fetch_limit: Max rows to fetch (over-fetched for HNSW approximation).
-            min_similarity: Minimum cosine similarity to include.
            tags_clause: Optional WHERE clause fragment for tag filtering.
            groups_clause: Optional WHERE clause fragment for tag group filtering.
            extra_where: Optional additional WHERE clause fragment (e.g. time range filter).
@@ -410,7 +408,6 @@ class SQLDialect(ABC):
        arm_index: int = 0,
        text_search_extension: str = "native",
        bm25_language: str = "english",
-        bm25_min_score: float = 0.0,
        extra_where: str = "",
    ) -> str:
        """Build a BM25/full-text search subquery arm.
@@ -433,11 +430,6 @@ class SQLDialect(ABC):
                                   "pg_textsearch", "pgroonga"). Only relevant for PostgreSQL.
            bm25_language: PostgreSQL text search dictionary used by the native
                           backend (e.g. "english", "french"). Ignored by other backends.
-            bm25_min_score: Minimum BM25 relevance score a row must exceed to be
-                            returned. Gates out non-matching rows on backends whose
-                            operator (e.g. VectorChord) ranks every document instead
-                            of pre-filtering to query-term matches. Backends that
-                            already apply a boolean match gate ignore this.
            extra_where: Optional additional WHERE clause fragment (e.g. time range filter).
        """
        ...
@@ -234,7 +234,6 @@ class OracleDialect(SQLDialect):
        embedding_param: str,
        bank_id_param: str,
        fetch_limit: int,
-        min_similarity: float,
        tags_clause: str = "",
        groups_clause: str = "",
        extra_where: str = "",
@@ -250,7 +249,7 @@ class OracleDialect(SQLDialect):
            f" WHERE bank_id = {bank_id_param}"
            f"   AND fact_type = '{fact_type}'"
            f"   AND embedding IS NOT NULL"
-            f"   AND (1 - VECTOR_DISTANCE(embedding, {embedding_param}, COSINE)) >= {min_similarity}"
+            f"   AND (1 - VECTOR_DISTANCE(embedding, {embedding_param}, COSINE)) >= 0.3"
            f"   {tags_clause}"
            f"   {groups_clause}"
            f"   {extra_where}"
@@ -272,7 +271,6 @@ class OracleDialect(SQLDialect):
        arm_index: int = 0,
        text_search_extension: str = "native",
        bm25_language: str = "english",
-        bm25_min_score: float = 0.0,
        extra_where: str = "",
    ) -> str:
        # Oracle Text: CONTAINS() / SCORE() with the CTXSYS.CONTEXT index.
@@ -287,9 +285,7 @@ class OracleDialect(SQLDialect):
            f" FROM {table}"
            f" WHERE bank_id = {bank_id_param}"
            f"   AND fact_type = '{fact_type}'"
-            # CONTAINS already gates to genuine matches; the configurable floor
-            # (default 0) keeps the threshold semantics uniform across backends.
-            f"   AND CONTAINS(text, {text_param}, {label}) > {bm25_min_score:g}"
+            f"   AND CONTAINS(text, {text_param}, {label}) > 0"
            f"   {tags_clause}"
            f"   {groups_clause}"
            f"   {extra_where}"
@@ -148,7 +148,6 @@ class PostgreSQLDialect(SQLDialect):
        embedding_param: str,
        bank_id_param: str,
        fetch_limit: int,
-        min_similarity: float,
        tags_clause: str = "",
        groups_clause: str = "",
        extra_where: str = "",
@@ -162,7 +161,7 @@ class PostgreSQLDialect(SQLDialect):
            f" WHERE bank_id = {bank_id_param}"
            f"   AND fact_type = '{fact_type}'"
            f"   AND embedding IS NOT NULL"
-            f"   AND (1 - (embedding <=> {embedding_param}::vector)) >= {min_similarity}"
+            f"   AND (1 - (embedding <=> {embedding_param}::vector)) >= 0.3"
            f"   {tags_clause}"
            f"   {groups_clause}"
            f"   {extra_where}"
@@ -184,32 +183,25 @@ class PostgreSQLDialect(SQLDialect):
        arm_index: int = 0,
        text_search_extension: str = "native",
        bm25_language: str = "english",
-        bm25_min_score: float = 0.0,
        extra_where: str = "",
    ) -> str:
        if text_search_extension == "vchord":
-            # <&> returns the NEGATIVE BM25 score (lower = more relevant), negate
-            # for a positive score where higher = more relevant.
+            # <&> returns a distance (lower = more relevant), negate for score
            bm25_score_expr = f"-(search_vector <&> to_bm25query('idx_memory_units_text_search', tokenize({text_param}, 'llmlingua2')))"
            bm25_order_by = f"{bm25_score_expr} DESC"
-            # Unlike native tsvector (which has a boolean `@@` match gate), the
-            # VectorChord operator ranks *every* document, so a bare ORDER BY ...
-            # LIMIT pads the result with zero-score, non-matching rows. Gate on the
-            # score so only genuine term matches survive into fusion/reranking.
-            bm25_where_filter = f"AND -(search_vector <&> to_bm25query('idx_memory_units_text_search', tokenize({text_param}, 'llmlingua2'))) > {bm25_min_score:g}"
+            bm25_where_filter = ""
        elif text_search_extension == "pg_textsearch":
            bm25_score_expr = f"-({text_param} <@> to_bm25query({text_param}, 'idx_memory_units_text_search'))"
            bm25_order_by = f"text <@> to_bm25query({text_param}, 'idx_memory_units_text_search') ASC"
            bm25_where_filter = ""
        elif text_search_extension == "pgroonga":
-            # &@~ accepts pgroonga's query syntax. Escape the bind parameter so
-            # literal memory text containing operators like ">" or "(" is not
-            # parsed as a malformed query expression.
+            # &@~ accepts pgroonga's query syntax (raw query text). pgroonga_score
+            # returns a non-negative relevance score (higher = better).
            bm25_score_expr = "pgroonga_score(tableoid, ctid)"
            bm25_order_by = f"{bm25_score_expr} DESC"
            bm25_where_filter = (
                f"AND (COALESCE(text, '') || ' ' || COALESCE(context, '') || ' ' || COALESCE(text_signals, '')) "
-                f"&@~ pgroonga_query_escape({text_param})"
+                f"&@~ {text_param}"
            )
        elif text_search_extension == "pg_search":
            # ParadeDB pg_search: BM25 index over (id, text, context, text_signals)
@@ -1,43 +0,0 @@
-"""Document transfer: export/import documents between banks without re-running the LLM.
-
-An export is a ZIP of already-extracted facts (text, entities by canonical name,
-causal relations, chunks) — never embeddings or DB ids. An import replays the
-deterministic half of the retain pipeline against the target bank: it re-embeds
-locally with the target bank's embedding model, re-resolves entities, and
-recreates temporal/semantic/causal links relative to the target bank's existing
-memories. No LLM fact-extraction is involved.
-
-Consolidated observations (``fact_type='observation'``) are intentionally
-excluded from export — they are derived by consolidation and are regenerated in
-the target bank.
-"""
-
-from .export import export_bank, export_documents
-from .importer import BankImportResult, ImportResult, import_bank, import_documents
-from .schema import (
-    SCHEMA_VERSION,
-    TransferCausalRelation,
-    TransferChunk,
-    TransferDocument,
-    TransferFact,
-    TransferManifest,
-    TransferObservation,
-    TransferObservationSource,
-)
-
-__all__ = [
-    "SCHEMA_VERSION",
-    "BankImportResult",
-    "ImportResult",
-    "TransferCausalRelation",
-    "TransferChunk",
-    "TransferDocument",
-    "TransferFact",
-    "TransferManifest",
-    "TransferObservation",
-    "TransferObservationSource",
-    "export_bank",
-    "export_documents",
-    "import_bank",
-    "import_documents",
-]
@@ -1,555 +0,0 @@
-"""Export documents (with extracted facts, entities, causal links, chunks) to a ZIP archive.
-
-Reads directly from the database via the backend connection. Embeddings and
-database ids are deliberately omitted — they are regenerated/re-resolved on
-import. Consolidated observations are excluded unless ``include_observations``
-is set, in which case they are written to ``observations.json``.
-"""
-
-from __future__ import annotations
-
-import base64
-import io
-import json
-import logging
-import zipfile
-from dataclasses import dataclass, field
-from datetime import UTC, date, datetime
-from decimal import Decimal
-from typing import Any
-from uuid import UUID
-
-from ..db_utils import acquire_with_retry
-from ..schema import fq_table
-from .schema import (
-    SCHEMA_VERSION,
-    TransferCausalRelation,
-    TransferChunk,
-    TransferDocument,
-    TransferFact,
-    TransferManifest,
-    TransferObservation,
-    TransferObservationSource,
-)
-
-logger = logging.getLogger(__name__)
-
-# Whole-bank export classification. Every bank-scoped table (admin.cli.BACKUP_TABLES)
-# must fall into exactly one bucket below; tests/test_document_transfer.py's
-# test_export_bank_covers_schema enforces this so a table added by a future
-# migration can't be silently dropped from a migration archive.
-
-# NOT written to the archive — rebuilt on import by replaying the document/fact/
-# observation payload through the import pipeline:
-#   * documents / chunks / memory_units carry their *text* in the logical document
-#     payload (TransferDocument) and are re-embedded with the target model;
-#   * entities / unit_entities / memory_links / entity_cooccurrences are derived
-#     data — the pipeline re-resolves entities and rebuilds links/cooccurrence
-#     stats against the target bank, so they are never exported.
-# Listed here only so the coverage guard can assert every table is classified.
-_REPLAYED_TABLES = frozenset(
-    {
-        "documents",
-        "chunks",
-        "memory_units",
-        "entities",
-        "unit_entities",
-        "memory_links",
-        "entity_cooccurrences",
-        # observation_history FKs to a memory_units observation, but observations
-        # are derived: they're regenerated with FRESH ids when consolidation is
-        # replayed on import (see _EXPORTED_FACT_TYPES — observations are excluded).
-        # There is no stable observation id to re-attach history to, so it is not
-        # carried; the target rebuilds observation history as it re-consolidates.
-        "observation_history",
-    }
-)
-# Carried verbatim as JSON rows (bank config + synthesized state). Embedding-bearing
-# rows have their vector stripped (see _DERIVED_COLUMNS) and are re-embedded on import.
-_BANK_ROW_TABLES = ("banks", "mental_models", "directives", "webhooks")
-# Bank-scoped child-history carried verbatim. Unlike observations, mental models
-# keep their (id, bank_id) across export/import, so their refresh history can be
-# re-attached. The surrogate ``id`` is dropped on dump so the target reassigns it
-# (see _dump_history_rows); restored after its parent table (mental_models).
-_CARRIED_HISTORY_TABLES = ("mental_model_history",)
-# Operational history — only carried with include_history=True.
-_HISTORY_TABLES = ("audit_log", "llm_requests")
-# Intentionally never exported.
-_SKIP_TABLES = frozenset(
-    {
-        "async_operations",  # in-flight ops; drain on the source before migrating
-        "graph_maintenance_queue",  # transient work queue; regenerated on import
-        "file_storage",  # raw uploads; documents.original_text is already carried
-    }
-)
-# Derived columns dropped from carried rows so the target regenerates them with
-# its own embedding model / text-search backend.
-_DERIVED_COLUMNS = ("embedding", "search_vector")
-
-
-@dataclass
-class _UnitLocation:
-    """Where a memory unit's fact lives in the assembled export (document + ordinal)."""
-
-    document_id: str
-    ordinal: int
-
-
-@dataclass
-class _LoadedFacts:
-    """Facts grouped by document plus an index from unit id to its location.
-
-    ``facts_by_doc`` and ``unit_index`` share the same fixed ordering so that
-    causal ``target_fact_index`` ordinals stay consistent across both.
-    """
-
-    facts_by_doc: dict[str, list[TransferFact]] = field(default_factory=dict)
-    unit_index: dict[Any, _UnitLocation] = field(default_factory=dict)
-
-
-@dataclass
-class _LoadedExport:
-    """Assembled documents plus the unit-id → location index.
-
-    ``unit_index`` is retained so observation source unit ids can be resolved to
-    (document_id, fact_index) references when observations are exported.
-    """
-
-    documents: list[TransferDocument] = field(default_factory=list)
-    unit_index: dict[Any, _UnitLocation] = field(default_factory=dict)
-
-
-# Causal link types that retain persists between facts. Only these travel in the
-# archive; temporal/semantic/entity links are regenerated against the target bank.
-_CAUSAL_LINK_TYPES = ("caused_by", "causes", "enables", "prevents")
-
-# Facts of these types are exported; observations are derived and excluded.
-_EXPORTED_FACT_TYPES = ("world", "experience")
-
-
-def _as_jsonb(value: Any) -> Any:
-    """Coerce an asyncpg JSONB column (str or already-decoded) to a Python object."""
-    if value is None:
-        return None
-    if isinstance(value, str):
-        return json.loads(value)
-    return value
-
-
-def _chunk_index_from_chunk_id(chunk_id: str | None) -> int | None:
-    """Recover the chunk ordinal from a ``{bank_id}_{document_id}_{index}`` chunk_id.
-
-    The index is always the final underscore-delimited segment, so rsplit is
-    correct even when bank/document ids themselves contain underscores.
-    """
-    if not chunk_id:
-        return None
-    try:
-        return int(chunk_id.rsplit("_", 1)[1])
-    except (IndexError, ValueError):
-        return None
-
-
-async def export_documents(
-    backend: Any,
-    bank_id: str,
-    document_ids: list[str] | None = None,
-    *,
-    include_observations: bool = False,
-) -> bytes:
-    """Export documents from ``bank_id`` into an in-memory ZIP archive.
-
-    Args:
-        backend: Database backend (provides ``acquire()``).
-        bank_id: Source bank.
-        document_ids: Specific document ids to export. ``None`` exports every
-            document in the bank.
-        include_observations: Also export consolidated observations (written to
-            ``observations.json``). Only valid for a whole-bank export.
-
-    Returns:
-        The ZIP archive as bytes.
-
-    Raises:
-        ValueError: if ``include_observations`` is combined with ``document_ids``.
-    """
-    # Observations are bank-level and can be derived from facts spanning several
-    # documents, so they're only coherent when the whole bank is exported. For a
-    # document subset we'd have to silently drop every cross-document observation
-    # — reject the combination instead so the caller isn't surprised.
-    if include_observations and document_ids is not None:
-        raise ValueError("include_observations is only supported when exporting the whole bank (omit document_id)")
-
-    async with acquire_with_retry(backend) as conn:
-        loaded = await _load_documents(conn, bank_id, document_ids)
-        documents = loaded.documents
-        observations = await _load_observations(conn, bank_id, loaded.unit_index) if include_observations else []
-
-    archive = io.BytesIO()
-    fact_total = 0
-    with zipfile.ZipFile(archive, "w", zipfile.ZIP_DEFLATED) as zf:
-        for index, document in enumerate(documents):
-            fact_total += len(document.facts)
-            zf.writestr(
-                f"documents/{index:06d}.json",
-                document.model_dump_json(indent=2, exclude_none=False),
-            )
-
-        if observations:
-            payload = "[\n" + ",\n".join(o.model_dump_json(indent=2) for o in observations) + "\n]\n"
-            zf.writestr("observations.json", payload)
-
-        manifest = TransferManifest(
-            schema_version=SCHEMA_VERSION,
-            source_bank_id=bank_id,
-            exported_at=datetime.now(UTC),
-            document_count=len(documents),
-            fact_count=fact_total,
-            observation_count=len(observations),
-        )
-        zf.writestr("manifest.json", manifest.model_dump_json(indent=2))
-
-    logger.info(
-        "[transfer] Exported %d document(s), %d fact(s), %d observation(s) from bank %s",
-        len(documents),
-        fact_total,
-        len(observations),
-        bank_id,
-    )
-    return archive.getvalue()
-
-
-def _row_json_default(obj: Any) -> Any:
-    """JSON serializer for the value types asyncpg returns from bank rows."""
-    if isinstance(obj, datetime):
-        return obj.isoformat()
-    if isinstance(obj, date):
-        return obj.isoformat()
-    if isinstance(obj, UUID):
-        return str(obj)
-    if isinstance(obj, Decimal):
-        # str preserves precision; import casts back to numeric.
-        return str(obj)
-    if isinstance(obj, (bytes, bytearray, memoryview)):
-        return base64.b64encode(bytes(obj)).decode("ascii")
-    raise TypeError(f"Object of type {type(obj).__name__} is not JSON serializable")
-
-
-async def _dump_bank_rows(conn: Any, table: str, bank_id: str) -> list[dict]:
-    """Dump all rows of a bank-scoped table as JSON-ready dicts (derived columns stripped).
-
-    Embedding/search-vector columns are omitted so the target instance
-    regenerates them with its own model/backend on import.
-    """
-    rows = await conn.fetch(f"SELECT * FROM {fq_table(table)} WHERE bank_id = $1", bank_id)
-    return [{k: v for k, v in dict(row).items() if k not in _DERIVED_COLUMNS} for row in rows]
-
-
-async def _dump_history_rows(conn: Any, table: str, bank_id: str) -> list[dict]:
-    """Dump a bank-scoped child-history table for carrying across instances.
-
-    Drops the surrogate ``id`` so the target reassigns it from its own IDENTITY
-    sequence (carrying explicit ids would leave the sequence un-advanced and
-    collide with later writes). Ordered oldest-first so the reassigned ids keep
-    the same chronological tie-break order the read path relies on.
-    """
-    rows = await conn.fetch(
-        f"SELECT * FROM {fq_table(table)} WHERE bank_id = $1 ORDER BY changed_at, id",
-        bank_id,
-    )
-    return [{k: v for k, v in dict(row).items() if k not in _DERIVED_COLUMNS and k != "id"} for row in rows]
-
-
-async def export_bank(conn: Any, bank_id: str, *, include_history: bool = False) -> bytes:
-    """Export an entire bank into a portable ZIP archive (no embeddings).
-
-    Produces a superset of the documents archive: the logical
-    document/fact/observation export (replayed and re-embedded on import) plus
-    the bank's config, mental models, directives and webhooks as JSON rows. With
-    ``include_history`` the operational tails (audit_log, llm_requests) are also
-    carried. Intended for migrating a bank to a new instance configured with a
-    different embedding model / vector / text-search backend — every vector is
-    regenerated on the target, so nothing here is encoder-specific.
-
-    ``conn`` is a live connection scoped to the bank's schema (the admin CLI sets
-    ``_current_schema`` and passes its raw connection; the engine acquires one
-    after tenant auth).
-    """
-    loaded = await _load_documents(conn, bank_id, None)
-    documents = loaded.documents
-    # Whole-bank export always carries observations (they're bank-level state).
-    observations = await _load_observations(conn, bank_id, loaded.unit_index)
-
-    bank_rows = {table: await _dump_bank_rows(conn, table, bank_id) for table in _BANK_ROW_TABLES}
-    for table in _CARRIED_HISTORY_TABLES:
-        bank_rows[table] = await _dump_history_rows(conn, table, bank_id)
-    history_rows: dict[str, list[dict]] = {}
-    if include_history:
-        history_rows = {table: await _dump_bank_rows(conn, table, bank_id) for table in _HISTORY_TABLES}
-
-    archive = io.BytesIO()
-    fact_total = 0
-    with zipfile.ZipFile(archive, "w", zipfile.ZIP_DEFLATED) as zf:
-        for index, document in enumerate(documents):
-            fact_total += len(document.facts)
-            zf.writestr(f"documents/{index:06d}.json", document.model_dump_json(indent=2, exclude_none=False))
-
-        if observations:
-            payload = "[\n" + ",\n".join(o.model_dump_json(indent=2) for o in observations) + "\n]\n"
-            zf.writestr("observations.json", payload)
-
-        for table, rows in bank_rows.items():
-            zf.writestr(f"{table}.json", json.dumps(rows, indent=2, default=_row_json_default))
-        for table, rows in history_rows.items():
-            zf.writestr(f"history/{table}.json", json.dumps(rows, indent=2, default=_row_json_default))
-
-        manifest = TransferManifest(
-            schema_version=SCHEMA_VERSION,
-            source_bank_id=bank_id,
-            exported_at=datetime.now(UTC),
-            document_count=len(documents),
-            fact_count=fact_total,
-            observation_count=len(observations),
-            archive_type="bank",
-            mental_model_count=len(bank_rows.get("mental_models", [])),
-            directive_count=len(bank_rows.get("directives", [])),
-            webhook_count=len(bank_rows.get("webhooks", [])),
-            includes_history=include_history,
-        )
-        zf.writestr("manifest.json", manifest.model_dump_json(indent=2))
-
-    logger.info(
-        "[transfer] Exported bank %s: %d document(s), %d fact(s), %d observation(s), "
-        "%d mental model(s), %d directive(s), %d webhook(s)%s",
-        bank_id,
-        len(documents),
-        fact_total,
-        len(observations),
-        len(bank_rows.get("mental_models", [])),
-        len(bank_rows.get("directives", [])),
-        len(bank_rows.get("webhooks", [])),
-        " (with history)" if include_history else "",
-    )
-    return archive.getvalue()
-
-
-async def _load_documents(
-    conn: Any,
-    bank_id: str,
-    document_ids: list[str] | None,
-) -> _LoadedExport:
-    """Load and assemble TransferDocument payloads for the requested documents."""
-    doc_filter = "AND id = ANY($2)" if document_ids else ""
-    params: list[Any] = [bank_id]
-    if document_ids:
-        params.append(document_ids)
-    doc_rows = await conn.fetch(
-        f"""
-        SELECT id, original_text, retain_params, tags, created_at
-        FROM {fq_table("documents")}
-        WHERE bank_id = $1 {doc_filter}
-        ORDER BY created_at, id
-        """,
-        *params,
-    )
-    if not doc_rows:
-        return _LoadedExport()
-
-    selected_ids = [row["id"] for row in doc_rows]
-
-    chunks_by_doc = await _load_chunks(conn, bank_id, selected_ids)
-    loaded = await _load_facts(conn, bank_id, selected_ids)
-    await _attach_entities(conn, loaded)
-    await _attach_causal_relations(conn, loaded)
-
-    documents: list[TransferDocument] = []
-    for row in doc_rows:
-        doc_id = row["id"]
-        documents.append(
-            TransferDocument(
-                id=doc_id,
-                original_text=row["original_text"],
-                retain_params=_as_jsonb(row["retain_params"]),
-                tags=list(row["tags"] or []),
-                created_at=row["created_at"],
-                chunks=chunks_by_doc.get(doc_id, []),
-                facts=loaded.facts_by_doc.get(doc_id, []),
-            )
-        )
-    return _LoadedExport(documents=documents, unit_index=loaded.unit_index)
-
-
-async def _load_observations(
-    conn: Any,
-    bank_id: str,
-    unit_index: dict[Any, _UnitLocation],
-) -> list[TransferObservation]:
-    """Load observations whose source facts are all present in the exported set.
-
-    Each source unit id is rewritten to its (document_id, fact_index) reference
-    via ``unit_index``. Only called for a whole-bank export, so every live source
-    fact is present; an observation is skipped only if a source no longer exists
-    (stale reference) — that keeps every exported observation resolvable on import.
-    """
-    rows = await conn.fetch(
-        f"""
-        SELECT id, text, tags, event_date, occurred_start, occurred_end,
-               mentioned_at, observation_scopes, proof_count, source_memory_ids
-        FROM {fq_table("memory_units")}
-        WHERE bank_id = $1 AND fact_type = 'observation'
-        ORDER BY created_at, id
-        """,
-        bank_id,
-    )
-
-    observations: list[TransferObservation] = []
-    skipped = 0
-    for row in rows:
-        source_ids = list(row["source_memory_ids"] or [])
-        locations = [unit_index.get(sid) for sid in source_ids]
-        if not source_ids or any(loc is None for loc in locations):
-            # An observation with sources outside the exported documents would be
-            # incoherent on import — skip it rather than emit dangling refs.
-            skipped += 1
-            continue
-        observations.append(
-            TransferObservation(
-                text=row["text"],
-                tags=list(row["tags"] or []),
-                event_date=row["event_date"],
-                occurred_start=row["occurred_start"],
-                occurred_end=row["occurred_end"],
-                mentioned_at=row["mentioned_at"],
-                observation_scopes=_as_jsonb(row["observation_scopes"]),
-                proof_count=row["proof_count"] or len(source_ids),
-                sources=[
-                    TransferObservationSource(document_id=loc.document_id, fact_index=loc.ordinal)
-                    for loc in locations
-                    if loc is not None
-                ],
-            )
-        )
-    if skipped:
-        logger.info("[transfer] Skipped %d observation(s) with sources outside the exported documents", skipped)
-    return observations
-
-
-async def _load_chunks(conn: Any, bank_id: str, doc_ids: list[str]) -> dict[str, list[TransferChunk]]:
-    rows = await conn.fetch(
-        f"""
-        SELECT document_id, chunk_index, chunk_text
-        FROM {fq_table("chunks")}
-        WHERE bank_id = $1 AND document_id = ANY($2)
-        ORDER BY document_id, chunk_index
-        """,
-        bank_id,
-        doc_ids,
-    )
-    chunks_by_doc: dict[str, list[TransferChunk]] = {}
-    for row in rows:
-        chunks_by_doc.setdefault(row["document_id"], []).append(
-            TransferChunk(chunk_index=row["chunk_index"], chunk_text=row["chunk_text"])
-        )
-    return chunks_by_doc
-
-
-async def _load_facts(conn: Any, bank_id: str, doc_ids: list[str]) -> _LoadedFacts:
-    """Load non-observation facts grouped by document, with a unit-id location index.
-
-    The ordering is fixed (created_at, id) so that
-    ``causal_relations.target_fact_index`` ordinals stay consistent.
-    """
-    rows = await conn.fetch(
-        f"""
-        SELECT id, document_id, text, fact_type, context, event_date,
-               occurred_start, occurred_end, mentioned_at, metadata,
-               chunk_id, tags, observation_scopes
-        FROM {fq_table("memory_units")}
-        WHERE bank_id = $1
-          AND document_id = ANY($2)
-          AND fact_type = ANY($3)
-        ORDER BY document_id, created_at, id
-        """,
-        bank_id,
-        doc_ids,
-        list(_EXPORTED_FACT_TYPES),
-    )
-
-    loaded = _LoadedFacts()
-    for row in rows:
-        doc_id = row["document_id"]
-        bucket = loaded.facts_by_doc.setdefault(doc_id, [])
-        ordinal = len(bucket)
-        fact = TransferFact(
-            text=row["text"],
-            fact_type=row["fact_type"],
-            context=row["context"],
-            event_date=row["event_date"],
-            occurred_start=row["occurred_start"],
-            occurred_end=row["occurred_end"],
-            mentioned_at=row["mentioned_at"],
-            metadata=_as_jsonb(row["metadata"]) or {},
-            tags=list(row["tags"] or []),
-            observation_scopes=_as_jsonb(row["observation_scopes"]),
-            chunk_index=_chunk_index_from_chunk_id(row["chunk_id"]),
-        )
-        bucket.append(fact)
-        loaded.unit_index[row["id"]] = _UnitLocation(document_id=doc_id, ordinal=ordinal)
-    return loaded
-
-
-async def _attach_entities(conn: Any, loaded: _LoadedFacts) -> None:
-    """Populate each fact's ``entities`` list with its entities' canonical names."""
-    if not loaded.unit_index:
-        return
-    rows = await conn.fetch(
-        f"""
-        SELECT ue.unit_id, e.canonical_name
-        FROM {fq_table("unit_entities")} ue
-        JOIN {fq_table("entities")} e ON e.id = ue.entity_id
-        WHERE ue.unit_id = ANY($1)
-        ORDER BY e.canonical_name
-        """,
-        list(loaded.unit_index.keys()),
-    )
-    for row in rows:
-        location = loaded.unit_index.get(row["unit_id"])
-        if location is None:
-            continue
-        loaded.facts_by_doc[location.document_id][location.ordinal].entities.append(row["canonical_name"])
-
-
-async def _attach_causal_relations(conn: Any, loaded: _LoadedFacts) -> None:
-    """Reconstruct causal edges as fact ordinals within each document.
-
-    A memory_link (from_unit -> to_unit, link_type) means ``from_unit`` carries
-    the relation pointing at ``to_unit``, so the edge is attached to the source
-    fact with the target's ordinal. Edges spanning two documents are skipped
-    (causal links are created within a single retain batch in practice).
-    """
-    if not loaded.unit_index:
-        return
-    rows = await conn.fetch(
-        f"""
-        SELECT from_unit_id, to_unit_id, link_type
-        FROM {fq_table("memory_links")}
-        WHERE link_type = ANY($1)
-          AND from_unit_id = ANY($2)
-          AND to_unit_id = ANY($2)
-        """,
-        list(_CAUSAL_LINK_TYPES),
-        list(loaded.unit_index.keys()),
-    )
-    for row in rows:
-        source = loaded.unit_index.get(row["from_unit_id"])
-        target = loaded.unit_index.get(row["to_unit_id"])
-        if source is None or target is None:
-            continue
-        if source.document_id != target.document_id:
-            continue
-        loaded.facts_by_doc[source.document_id][source.ordinal].causal_relations.append(
-            TransferCausalRelation(
-                relation_type=row["link_type"],
-                target_fact_index=target.ordinal,
-            )
-        )
@@ -1,716 +0,0 @@
-"""Import documents from a transfer archive by replaying the deterministic retain pipeline.
-
-For each document the importer rebuilds the extracted facts, re-embeds them with
-the *target* bank's embedding model, then runs entity resolution (Phase 1) and
-the fact/link insert (Phase 2) — exactly the steps retain runs after LLM
-extraction. No LLM is called. Temporal/semantic/causal links and entity merges
-are therefore computed relative to the target bank's existing memories.
-"""
-
-from __future__ import annotations
-
-import io
-import json
-import logging
-import uuid
-import zipfile
-from dataclasses import dataclass, field
-from datetime import UTC, date, datetime
-from typing import Any, Literal
-
-from ..db_utils import acquire_with_retry
-from ..retain import bank_utils, chunk_storage, embedding_processing, fact_storage, orchestrator
-from ..retain.types import (
-    CausalRelation,
-    ChunkMetadata,
-    ExtractedFact,
-    ProcessedFact,
-    RetainContent,
-)
-from ..schema import fq_table
-from .schema import (
-    SCHEMA_VERSION,
-    TransferDocument,
-    TransferFact,
-    TransferManifest,
-    TransferObservation,
-)
-
-logger = logging.getLogger(__name__)
-
-OnConflict = Literal["skip", "replace", "new-id"]
-_VALID_CONFLICT_MODES: tuple[OnConflict, ...] = ("skip", "replace", "new-id")
-
-
-@dataclass
-class ImportedDocument:
-    """A single document successfully imported, with the units it produced.
-
-    Carried back so the engine can fire the post-retain extension hook
-    (usage tracking / metrics / notifications) once per imported document,
-    mirroring how retain reports each completed document.
-    """
-
-    document_id: str
-    unit_ids: list[str]
-    content: str
-    tags: list[str]
-
-
-@dataclass
-class ImportResult:
-    """Outcome of importing a transfer archive into a bank."""
-
-    documents_imported: int = 0
-    documents_skipped: int = 0
-    facts_imported: int = 0
-    observations_imported: int = 0
-    # Observations dropped because some source fact was not imported in this run.
-    observations_skipped: int = 0
-    skipped_document_ids: list[str] = field(default_factory=list)
-    # Original id -> freshly generated id, for documents imported under "new-id".
-    remapped_document_ids: dict[str, str] = field(default_factory=dict)
-    # Per-document outcomes, for the engine's post-retain hook. Not serialized
-    # into operation result_metadata (the worker handler writes counts only).
-    imported_documents: list[ImportedDocument] = field(default_factory=list)
-
-
-@dataclass
-class _ObservationOutcome:
-    """Counts from the observation import pass."""
-
-    imported: int = 0
-    skipped: int = 0
-
-
-@dataclass
-class ParsedArchive:
-    """A transfer archive after parsing/validation."""
-
-    manifest: TransferManifest
-    documents: list[TransferDocument]
-    observations: list[TransferObservation] = field(default_factory=list)
-
-
-def parse_archive(archive_bytes: bytes) -> ParsedArchive:
-    """Parse and validate a transfer ZIP archive produced by ``export_documents``."""
-    with zipfile.ZipFile(io.BytesIO(archive_bytes), "r") as zf:
-        names = set(zf.namelist())
-        if "manifest.json" not in names:
-            raise ValueError("Invalid transfer archive: manifest.json is missing")
-        manifest = TransferManifest.model_validate_json(zf.read("manifest.json"))
-        if manifest.schema_version != SCHEMA_VERSION:
-            raise ValueError(
-                f"Unsupported transfer archive schema version {manifest.schema_version} "
-                f"(this build supports {SCHEMA_VERSION})"
-            )
-        doc_names = sorted(n for n in names if n.startswith("documents/") and n.endswith(".json"))
-        documents = [TransferDocument.model_validate_json(zf.read(name)) for name in doc_names]
-        observations: list[TransferObservation] = []
-        if "observations.json" in names:
-            observations = [TransferObservation.model_validate(o) for o in json.loads(zf.read("observations.json"))]
-    return ParsedArchive(manifest=manifest, documents=documents, observations=observations)
-
-
-async def import_documents(
-    *,
-    backend: Any,
-    embeddings_model: Any,
-    entity_resolver: Any,
-    config: Any,
-    format_date_fn: Any,
-    bank_id: str,
-    archive_bytes: bytes,
-    on_conflict: OnConflict = "skip",
-    ops: Any = None,
-    outbox_callback_factory: Any = None,
-) -> ImportResult:
-    """Import every document in ``archive_bytes`` into ``bank_id``.
-
-    Args:
-        backend: Database backend (provides ``acquire()`` and ``ops``).
-        embeddings_model: Target bank's embedding model (used to re-embed facts).
-        entity_resolver: Shared entity resolver for the target bank.
-        config: Resolved bank config for the target bank.
-        format_date_fn: Date formatter used when augmenting fact text for embedding
-            (must match retain so embeddings are consistent).
-        bank_id: Target bank.
-        archive_bytes: A ZIP archive produced by ``export_documents``.
-        on_conflict: How to handle a document id that already exists in the target
-            bank — ``skip`` (default), ``replace`` (delete old data and re-import),
-            or ``new-id`` (import under a freshly generated id).
-        ops: Backend ``DataAccessOps``. Defaults to ``backend.ops``.
-
-    Returns:
-        An :class:`ImportResult` with per-document counts.
-    """
-    if on_conflict not in _VALID_CONFLICT_MODES:
-        raise ValueError(f"Invalid on_conflict '{on_conflict}'; expected one of {_VALID_CONFLICT_MODES}")
-    if ops is None:
-        ops = backend.ops
-
-    parsed = parse_archive(archive_bytes)
-    result = ImportResult()
-
-    # (original document_id, fact ordinal) -> freshly inserted unit id. Used to
-    # resolve observation source references after all facts exist.
-    ref_map: dict[tuple[str, int], str] = {}
-
-    for document in parsed.documents:
-        target_id = await _resolve_target_id(backend, bank_id, document.id, on_conflict)
-        if target_id is None:
-            result.documents_skipped += 1
-            result.skipped_document_ids.append(document.id)
-            continue
-        if target_id != document.id:
-            result.remapped_document_ids[document.id] = target_id
-
-        unit_ids = await _import_one_document(
-            backend=backend,
-            embeddings_model=embeddings_model,
-            entity_resolver=entity_resolver,
-            config=config,
-            format_date_fn=format_date_fn,
-            bank_id=bank_id,
-            document=document,
-            target_id=target_id,
-            ops=ops,
-            outbox_callback_factory=outbox_callback_factory,
-        )
-        result.documents_imported += 1
-        result.facts_imported += len(unit_ids)
-        result.imported_documents.append(
-            ImportedDocument(
-                document_id=target_id,
-                unit_ids=unit_ids,
-                content=document.original_text or "",
-                tags=list(document.tags),
-            )
-        )
-        for ordinal, unit_id in enumerate(unit_ids):
-            ref_map[(document.id, ordinal)] = unit_id
-
-    if parsed.observations:
-        outcome = await _import_observations(
-            backend=backend,
-            embeddings_model=embeddings_model,
-            bank_id=bank_id,
-            observations=parsed.observations,
-            ref_map=ref_map,
-            ops=ops,
-        )
-        result.observations_imported = outcome.imported
-        result.observations_skipped = outcome.skipped
-
-    logger.info(
-        "[transfer] Imported %d document(s), %d fact(s), %d observation(s) into bank %s "
-        "(%d docs skipped, %d observations skipped)",
-        result.documents_imported,
-        result.facts_imported,
-        result.observations_imported,
-        bank_id,
-        result.documents_skipped,
-        result.observations_skipped,
-    )
-    return result
-
-
-# Bank-level config/state tables restored verbatim from a whole-bank archive.
-# Order matters for foreign keys: banks (parent) is restored before any child.
-_BANK_CHILD_TABLES = ("mental_models", "directives", "webhooks")
-# Child-history carried verbatim; restored after its parent (mental_models) so the
-# foreign key resolves. Surrogate ids were dropped on export (the target reassigns
-# them), so these restore via fresh IDENTITY values.
-_CARRIED_HISTORY_TABLES = ("mental_model_history",)
-_HISTORY_TABLES = ("audit_log", "llm_requests")
-
-
-@dataclass
-class BankImportResult:
-    """Outcome of importing a whole-bank archive."""
-
-    bank_id: str
-    documents_imported: int = 0
-    facts_imported: int = 0
-    observations_imported: int = 0
-    mental_models_imported: int = 0
-    mental_model_history_imported: int = 0
-    directives_imported: int = 0
-    webhooks_imported: int = 0
-    history_rows_imported: int = 0
-
-
-@dataclass
-class ParsedBankArchive:
-    """The bank-level sections of a whole-bank archive (documents read separately)."""
-
-    manifest: TransferManifest
-    # table name -> list of verbatim row dicts (banks, mental_models, directives, webhooks)
-    bank_rows: dict[str, list[dict]] = field(default_factory=dict)
-    # table name -> rows (audit_log, llm_requests), present only with --include-history
-    history_rows: dict[str, list[dict]] = field(default_factory=dict)
-
-
-def parse_bank_archive(archive_bytes: bytes) -> ParsedBankArchive:
-    """Parse the bank-level sections of a whole-bank archive (``archive_type='bank'``)."""
-    with zipfile.ZipFile(io.BytesIO(archive_bytes), "r") as zf:
-        names = set(zf.namelist())
-        if "manifest.json" not in names:
-            raise ValueError("Invalid transfer archive: manifest.json is missing")
-        manifest = TransferManifest.model_validate_json(zf.read("manifest.json"))
-        if manifest.archive_type != "bank":
-            raise ValueError(
-                f"Not a whole-bank archive (archive_type={manifest.archive_type!r}); use import_documents instead"
-            )
-        bank_rows: dict[str, list[dict]] = {}
-        for table in ("banks", *_BANK_CHILD_TABLES, *_CARRIED_HISTORY_TABLES):
-            fname = f"{table}.json"
-            bank_rows[table] = json.loads(zf.read(fname)) if fname in names else []
-        history_rows: dict[str, list[dict]] = {}
-        for table in _HISTORY_TABLES:
-            fname = f"history/{table}.json"
-            if fname in names:
-                history_rows[table] = json.loads(zf.read(fname))
-    return ParsedBankArchive(manifest=manifest, bank_rows=bank_rows, history_rows=history_rows)
-
-
-async def _restore_rows(conn: Any, table: str, rows: list[dict]) -> int:
-    """Insert verbatim rows into a bank-scoped table, coercing JSON-encoded values
-    back to the column's type (timestamps, uuids, jsonb). ``ON CONFLICT DO NOTHING``
-    keeps an import idempotent and safe to re-run against a partially-filled target."""
-    if not rows:
-        return 0
-    from ..memory_engine import get_current_schema
-
-    schema = get_current_schema()
-    col_types = {
-        r["column_name"]: r["data_type"]
-        for r in await conn.fetch(
-            "SELECT column_name, data_type FROM information_schema.columns WHERE table_schema = $1 AND table_name = $2",
-            schema,
-            table,
-        )
-    }
-    inserted = 0
-    for row in rows:
-        cols = [c for c in row if c in col_types]
-        placeholders: list[str] = []
-        values: list[Any] = []
-        for position, col in enumerate(cols, start=1):
-            data_type = col_types[col]
-            value = row[col]
-            if data_type in ("jsonb", "json"):
-                # asyncpg has no JSON codec on these raw connections; pass JSON
-                # text and cast. Values may already be str (no codec on export) or
-                # a Python object (codec on export) — normalize to text either way.
-                values.append(value if isinstance(value, str) or value is None else json.dumps(value))
-                placeholders.append(f"${position}::jsonb")
-                continue
-            if value is not None and isinstance(value, str):
-                if data_type in ("timestamp with time zone", "timestamp without time zone"):
-                    value = datetime.fromisoformat(value)
-                elif data_type == "date":
-                    value = date.fromisoformat(value)
-                elif data_type == "uuid":
-                    value = uuid.UUID(value)
-            placeholders.append(f"${position}")
-            values.append(value)
-        col_list = ", ".join(f'"{c}"' for c in cols)
-        await conn.execute(
-            f"INSERT INTO {fq_table(table)} ({col_list}) VALUES ({', '.join(placeholders)}) ON CONFLICT DO NOTHING",
-            *values,
-        )
-        inserted += 1
-    return inserted
-
-
-async def import_bank(
-    *,
-    backend: Any,
-    embeddings_model: Any,
-    entity_resolver: Any,
-    config: Any,
-    format_date_fn: Any,
-    archive_bytes: bytes,
-    target_bank_id: str | None = None,
-    include_history: bool = False,
-    ops: Any = None,
-) -> BankImportResult:
-    """Restore a whole bank from a ``export_bank`` archive into the target instance.
-
-    Re-embeds facts with the *target* instance's embedding model and rebuilds links,
-    entities and search/vector indexes — the path for migrating a bank to an instance
-    configured with a different embedding model / vector / text-search backend.
-
-    The **target bank must not already exist**: import restores a complete bank
-    (config + facts + mental models + …) and is not a merge. If a bank with the
-    target id is present, this raises — delete it first or pass ``target_bank_id``
-    for a fresh id. A migration restores *exact* state, so unlike the document
-    import it fires no retain webhooks and triggers no consolidation/graph
-    maintenance: observations and mental models are restored as exported.
-    """
-    if ops is None:
-        ops = backend.ops
-    parsed = parse_bank_archive(archive_bytes)
-    source_bank_id = parsed.manifest.source_bank_id
-    bank_id = target_bank_id or source_bank_id
-
-    # Remapping to a different id: rewrite the carried bank_id on every row so FKs
-    # and PKs line up with the (also-remapped) documents/facts.
-    if bank_id != source_bank_id:
-        for rows in (*parsed.bank_rows.values(), *parsed.history_rows.values()):
-            for row in rows:
-                if "bank_id" in row:
-                    row["bank_id"] = bank_id
-
-    async with acquire_with_retry(backend) as conn:
-        # Refuse to import into an existing bank — this restores a whole bank, it
-        # does not merge. Merging would silently mix the archive's config/mental
-        # models/webhooks with whatever is already there (and global-unique ids
-        # like webhooks/directives would collide).
-        if await conn.fetchval(f"SELECT 1 FROM {fq_table('banks')} WHERE bank_id = $1", bank_id):
-            raise ValueError(
-                f"Target bank '{bank_id}' already exists; import-bank restores into a fresh bank "
-                f"(it is not a merge). Delete the bank first, or pass a different target bank id."
-            )
-        # Bank row first — children (documents, mental_models, …) FK to it.
-        await _restore_rows(conn, "banks", parsed.bank_rows.get("banks", []))
-    # Ensure the bank's per-bank vector indexes exist (no-op for global-index
-    # extensions); idempotent and keeps the restored banks row (ON CONFLICT DO NOTHING).
-    await bank_utils.get_or_create_bank_profile(backend, bank_id)
-
-    doc_result = await import_documents(
-        backend=backend,
-        embeddings_model=embeddings_model,
-        entity_resolver=entity_resolver,
-        config=config,
-        format_date_fn=format_date_fn,
-        bank_id=bank_id,
-        archive_bytes=archive_bytes,
-        ops=ops,
-        outbox_callback_factory=None,
-    )
-
-    result = BankImportResult(
-        bank_id=bank_id,
-        documents_imported=doc_result.documents_imported,
-        facts_imported=doc_result.facts_imported,
-        observations_imported=doc_result.observations_imported,
-    )
-    async with acquire_with_retry(backend) as conn:
-        result.mental_models_imported = await _restore_rows(
-            conn, "mental_models", parsed.bank_rows.get("mental_models", [])
-        )
-        # Restored after mental_models so the (mental_model_id, bank_id) FK resolves.
-        result.mental_model_history_imported = await _restore_rows(
-            conn, "mental_model_history", parsed.bank_rows.get("mental_model_history", [])
-        )
-        result.directives_imported = await _restore_rows(conn, "directives", parsed.bank_rows.get("directives", []))
-        result.webhooks_imported = await _restore_rows(conn, "webhooks", parsed.bank_rows.get("webhooks", []))
-        if include_history:
-            for table in _HISTORY_TABLES:
-                result.history_rows_imported += await _restore_rows(conn, table, parsed.history_rows.get(table, []))
-
-    logger.info(
-        "[transfer] Imported bank %s: %d doc(s), %d fact(s), %d observation(s), "
-        "%d mental model(s), %d mm-history row(s), %d directive(s), %d webhook(s), %d history row(s)",
-        bank_id,
-        result.documents_imported,
-        result.facts_imported,
-        result.observations_imported,
-        result.mental_models_imported,
-        result.mental_model_history_imported,
-        result.directives_imported,
-        result.webhooks_imported,
-        result.history_rows_imported,
-    )
-    return result
-
-
-async def _resolve_target_id(backend: Any, bank_id: str, document_id: str, on_conflict: OnConflict) -> str | None:
-    """Decide the document id to write under, or ``None`` to skip.
-
-    Returns the original id when there is no conflict, a fresh id under
-    ``new-id``, the original id under ``replace`` (the insert path cascades the
-    old data away), or ``None`` under ``skip`` when the document already exists.
-    """
-    async with acquire_with_retry(backend) as conn:
-        exists = await conn.fetchval(
-            f"SELECT 1 FROM {fq_table('documents')} WHERE id = $1 AND bank_id = $2",
-            document_id,
-            bank_id,
-        )
-    if not exists:
-        return document_id
-    if on_conflict == "skip":
-        return None
-    if on_conflict == "new-id":
-        return str(uuid.uuid4())
-    return document_id  # replace
-
-
-async def _import_one_document(
-    *,
-    backend: Any,
-    embeddings_model: Any,
-    entity_resolver: Any,
-    config: Any,
-    format_date_fn: Any,
-    bank_id: str,
-    document: TransferDocument,
-    target_id: str,
-    ops: Any,
-    outbox_callback_factory: Any = None,
-) -> list[str]:
-    """Re-embed and insert a single document; returns the new unit ids in fact order."""
-    log_buffer: list[str] = []
-
-    # Fire the same retain.completed webhook retain emits, transactionally inside
-    # this document's insert. Factory returns None when no webhook manager exists.
-    outbox_callback = (
-        outbox_callback_factory([{"document_id": target_id, "tags": list(document.tags)}])
-        if outbox_callback_factory
-        else None
-    )
-
-    extracted_facts = [_to_extracted_fact(fact) for fact in document.facts]
-
-    processed_facts: list[ProcessedFact] = []
-    if extracted_facts:
-        augmented = embedding_processing.augment_texts_with_dates(extracted_facts, format_date_fn)
-        embeddings = await embedding_processing.generate_embeddings_batch(embeddings_model, augmented)
-        processed_facts = [ProcessedFact.from_extracted_fact(ef, emb) for ef, emb in zip(extracted_facts, embeddings)]
-
-    contents = [RetainContent(content=document.original_text or "")]
-    chunk_meta = [
-        ChunkMetadata(chunk_text=chunk.chunk_text, fact_count=0, content_index=0, chunk_index=chunk.chunk_index)
-        for chunk in document.chunks
-    ]
-
-    # Phase 1 (entity resolution + semantic ANN) on its own connection, outside
-    # the write transaction — mirrors the retain pipeline.
-    entity_resolver.discard_pending_stats()
-    phase1 = await orchestrator._pre_resolve_phase1(
-        backend,
-        entity_resolver,
-        bank_id,
-        contents,
-        processed_facts,
-        config,
-        log_buffer,
-        skip_semantic_ann=False,
-    )
-
-    async with acquire_with_retry(backend) as conn:
-        async with conn.transaction():
-            # is_first_batch=True: cascade-delete any existing data for this id
-            # (the "replace" path) and (re)insert the document row.
-            await fact_storage.handle_document_tracking(
-                conn,
-                bank_id,
-                target_id,
-                document.original_text or "",
-                True,
-                document.retain_params,
-                document.tags,
-                ops=ops,
-            )
-
-            chunk_id_map: dict[int, str] = {}
-            if chunk_meta:
-                chunk_id_map = await chunk_storage.store_chunks_batch(conn, bank_id, target_id, chunk_meta, ops=ops)
-
-            for extracted, processed in zip(extracted_facts, processed_facts):
-                processed.document_id = target_id
-                if chunk_id_map and extracted.chunk_index is not None:
-                    chunk_id = chunk_id_map.get(extracted.chunk_index)
-                    if chunk_id:
-                        processed.chunk_id = chunk_id
-
-            result_unit_ids = await orchestrator._insert_facts_and_links(
-                conn,
-                entity_resolver,
-                bank_id,
-                contents,
-                extracted_facts,
-                processed_facts,
-                config,
-                log_buffer,
-                resolved_entity_ids=phase1.entities.resolved_entity_ids,
-                entity_to_unit=phase1.entities.entity_to_unit,
-                unit_to_entity_ids=phase1.entities.unit_to_entity_ids,
-                semantic_ann_links=phase1.semantic_ann_links,
-                skip_semantic_links=False,
-                outbox_callback=outbox_callback,
-                ops=ops,
-            )
-
-        try:
-            await entity_resolver.flush_pending_stats()
-        except Exception:
-            logger.warning("[transfer] Entity stats flush failed for document %s", target_id, exc_info=True)
-
-    logger.debug("[transfer] Imported document %s:\n%s", target_id, "\n".join(log_buffer))
-    # Single content item -> result_unit_ids[0] holds the new unit ids in fact order.
-    return list(result_unit_ids[0]) if result_unit_ids else []
-
-
-async def _import_observations(
-    *,
-    backend: Any,
-    embeddings_model: Any,
-    bank_id: str,
-    observations: list[TransferObservation],
-    ref_map: dict[tuple[str, int], str],
-    ops: Any,
-) -> _ObservationOutcome:
-    """Insert observations whose source facts were all imported in this run.
-
-    Observations carry no embedding, links, or entity rows — only the unit row
-    plus ``source_memory_ids`` (remapped to the freshly inserted source units)
-    and ``proof_count``. Their source facts are marked ``consolidated_at`` so the
-    target bank's consolidator won't re-process them. Mirrors what consolidation
-    writes, but driven from the archive instead of the LLM.
-
-    Inserted as-is: imported observations are NOT merged or deduplicated against
-    observations that already exist in the target bank (unlike consolidation,
-    which merges related observations). Importing into a bank that already has
-    observations — or importing the same archive twice — can therefore produce
-    overlapping observations over the same facts.
-    """
-    outcome = _ObservationOutcome()
-
-    # Resolve each observation's sources to new unit ids; drop any whose sources
-    # weren't all imported (e.g. a subset/skip import).
-    resolved: list[tuple[TransferObservation, list[str]]] = []
-    for obs in observations:
-        source_ids = [ref_map.get((s.document_id, s.fact_index)) for s in obs.sources]
-        if not source_ids or any(sid is None for sid in source_ids):
-            outcome.skipped += 1
-            continue
-        resolved.append((obs, [sid for sid in source_ids if sid is not None]))
-
-    if not resolved:
-        return outcome
-
-    # Observations embed the raw text (matching consolidation), not the
-    # date-augmented text used for facts.
-    embeddings = await embedding_processing.generate_embeddings_batch(
-        embeddings_model, [obs.text for obs, _ in resolved]
-    )
-    processed = [
-        ProcessedFact(
-            fact_text=obs.text,
-            fact_type="observation",
-            embedding=embedding,
-            occurred_start=obs.occurred_start,
-            occurred_end=obs.occurred_end,
-            mentioned_at=_observation_mentioned_at(obs),
-            context="",
-            metadata={},
-            tags=list(obs.tags),
-            observation_scopes=obs.observation_scopes,
-            document_id=None,
-            chunk_id=None,
-        )
-        for (obs, _sources), embedding in zip(resolved, embeddings)
-    ]
-
-    async with acquire_with_retry(backend) as conn:
-        async with conn.transaction():
-            obs_unit_ids = await fact_storage.insert_facts_batch(conn, bank_id, processed, ops=ops)
-
-            all_source_ids: set[uuid.UUID] = set()
-            for (obs, sources), obs_unit_id in zip(resolved, obs_unit_ids):
-                source_uuids = [uuid.UUID(s) for s in sources]
-                all_source_ids.update(source_uuids)
-                await _link_observation_sources(
-                    conn, ops, bank_id, uuid.UUID(obs_unit_id), source_uuids, obs.proof_count
-                )
-
-            # Mark source facts consolidated so the target consolidator skips them.
-            if all_source_ids:
-                await conn.execute(
-                    f"UPDATE {fq_table('memory_units')} SET consolidated_at = now() "
-                    f"WHERE bank_id = $1 AND id = ANY($2)",
-                    bank_id,
-                    list(all_source_ids),
-                )
-
-    outcome.imported = len(resolved)
-    return outcome
-
-
-async def _link_observation_sources(
-    conn: Any,
-    ops: Any,
-    bank_id: str,
-    observation_id: uuid.UUID,
-    source_ids: list[uuid.UUID],
-    proof_count: int,
-) -> None:
-    """Attach source ids + proof_count to a freshly inserted observation row.
-
-    PG stores the sources in the ``source_memory_ids`` array column; Oracle uses
-    the ``observation_sources`` junction table (same split as consolidation).
-    """
-    if ops.uses_observation_sources_table:
-        await conn.executemany(
-            f"INSERT INTO {fq_table('observation_sources')} (observation_id, source_id) "
-            f"VALUES ($1, $2) ON CONFLICT (observation_id, source_id) DO NOTHING",
-            [(observation_id, sid) for sid in dict.fromkeys(source_ids)],
-        )
-        await conn.execute(
-            f"UPDATE {fq_table('memory_units')} SET proof_count = $1 WHERE id = $2 AND bank_id = $3",
-            proof_count,
-            observation_id,
-            bank_id,
-        )
-    else:
-        await conn.execute(
-            f"UPDATE {fq_table('memory_units')} SET source_memory_ids = $1, proof_count = $2 "
-            f"WHERE id = $3 AND bank_id = $4",
-            source_ids,
-            proof_count,
-            observation_id,
-            bank_id,
-        )
-
-
-def _observation_mentioned_at(obs: TransferObservation) -> datetime | None:
-    """event_date (NOT NULL) is derived from occurred_start or mentioned_at on
-    insert; fall back so the column stays populated for observations too."""
-    mentioned_at = obs.mentioned_at
-    if obs.occurred_start is None and mentioned_at is None:
-        mentioned_at = obs.event_date or datetime.now(UTC)
-    return mentioned_at
-
-
-def _to_extracted_fact(fact: TransferFact) -> ExtractedFact:
-    """Rebuild the retain pipeline's ExtractedFact from a serialized transfer fact."""
-    # event_date is NOT NULL in the schema and is derived from occurred_start or
-    # mentioned_at on insert. When neither is present, fall back to the carried
-    # event_date (or now) via mentioned_at so the column stays populated.
-    mentioned_at = fact.mentioned_at
-    if fact.occurred_start is None and mentioned_at is None:
-        mentioned_at = fact.event_date or datetime.now(UTC)
-
-    return ExtractedFact(
-        fact_text=fact.text,
-        fact_type=fact.fact_type,
-        entities=list(fact.entities),
-        occurred_start=fact.occurred_start,
-        occurred_end=fact.occurred_end,
-        where=None,
-        causal_relations=[
-            CausalRelation(relation_type=rel.relation_type, target_fact_index=rel.target_fact_index)
-            for rel in fact.causal_relations
-        ],
-        content_index=0,
-        chunk_index=fact.chunk_index,
-        context=fact.context or "",
-        mentioned_at=mentioned_at,
-        metadata=dict(fact.metadata),
-        tags=list(fact.tags),
-        observation_scopes=fact.observation_scopes,
-    )
@@ -1,138 +0,0 @@
-"""Serialization schema for the document transfer archive (manifest + per-document payloads).
-
-The archive is a ZIP:
-
-    manifest.json              -- TransferManifest
-    documents/000000.json      -- TransferDocument (one file per document)
-    documents/000001.json
-    ...
-
-Documents are stored under a zero-padded index rather than their id so that
-arbitrary document ids (which may contain path-unsafe characters) never leak
-into archive entry names. The real id lives inside each payload.
-"""
-
-from __future__ import annotations
-
-from datetime import datetime
-from typing import Literal
-
-from pydantic import BaseModel, Field
-
-# Bump when the archive layout changes in a backward-incompatible way.
-SCHEMA_VERSION = 1
-
-ObservationScopes = Literal["per_tag", "combined", "all_combinations"] | list[list[str]]
-
-
-class TransferCausalRelation(BaseModel):
-    """A causal edge from this fact to an earlier fact in the same document.
-
-    ``target_fact_index`` is the ordinal of the target fact within the document's
-    ``facts`` list (not a database id), so it survives transfer to a new bank.
-    """
-
-    relation_type: str
-    target_fact_index: int
-
-
-class TransferFact(BaseModel):
-    """One extracted fact (memory unit) without its embedding or database id.
-
-    Everything here is reused verbatim on import except the embedding, which is
-    regenerated by the target bank's model, and the entity ids, which are
-    re-resolved against the target bank by canonical name.
-    """
-
-    text: str
-    fact_type: str
-    context: str | None = None
-    # event_date is a fallback used only when both occurred_start and
-    # mentioned_at are absent, to satisfy the NOT NULL event_date column.
-    event_date: datetime | None = None
-    occurred_start: datetime | None = None
-    occurred_end: datetime | None = None
-    mentioned_at: datetime | None = None
-    metadata: dict[str, str] = Field(default_factory=dict)
-    tags: list[str] = Field(default_factory=list)
-    observation_scopes: ObservationScopes | None = None
-    # Ordinal of the source chunk within the document (parsed from chunk_id).
-    chunk_index: int | None = None
-    # Entity canonical names; re-resolved against the target bank on import.
-    entities: list[str] = Field(default_factory=list)
-    causal_relations: list[TransferCausalRelation] = Field(default_factory=list)
-
-
-class TransferChunk(BaseModel):
-    """A raw text chunk of the source document, reused verbatim."""
-
-    chunk_index: int
-    chunk_text: str
-
-
-class TransferObservationSource(BaseModel):
-    """A reference to a source fact of an observation, by document + ordinal.
-
-    Observations span documents and reference their source facts by unit id;
-    those ids don't survive transfer, so each source is carried as the
-    (document_id, fact_index) of the fact within the exported document set.
-    """
-
-    document_id: str
-    fact_index: int
-
-
-class TransferObservation(BaseModel):
-    """A consolidated observation (``fact_type='observation'``).
-
-    Observations are bank-level (not tied to one document), carry no embedding
-    (re-generated on import) and no entity/link associations — retrieval reaches
-    entities/links through their source facts. Only exported when explicitly
-    requested, and only when every source resolves within the archive.
-    """
-
-    text: str
-    tags: list[str] = Field(default_factory=list)
-    event_date: datetime | None = None
-    occurred_start: datetime | None = None
-    occurred_end: datetime | None = None
-    mentioned_at: datetime | None = None
-    observation_scopes: ObservationScopes | None = None
-    proof_count: int = 1
-    sources: list[TransferObservationSource] = Field(default_factory=list)
-
-
-class TransferDocument(BaseModel):
-    """A single document plus its chunks and extracted facts."""
-
-    id: str
-    original_text: str | None = None
-    retain_params: dict | None = None
-    tags: list[str] = Field(default_factory=list)
-    created_at: datetime | None = None
-    chunks: list[TransferChunk] = Field(default_factory=list)
-    facts: list[TransferFact] = Field(default_factory=list)
-
-
-class TransferManifest(BaseModel):
-    """Top-level archive descriptor (``manifest.json``).
-
-    The bank-level fields default to a documents-only archive so older
-    document-only archives (and the document import path) keep parsing
-    unchanged; ``export_bank`` populates them for a whole-bank archive.
-    """
-
-    schema_version: int = SCHEMA_VERSION
-    source_bank_id: str
-    exported_at: datetime | None = None
-    document_count: int = 0
-    fact_count: int = 0
-    observation_count: int = 0
-    # "documents" = doc/fact/observation subset; "bank" = whole-bank export
-    # (also carries bank config, mental models, directives, webhooks).
-    archive_type: Literal["documents", "bank"] = "documents"
-    mental_model_count: int = 0
-    directive_count: int = 0
-    webhook_count: int = 0
-    # True when --include-history carried audit_log / llm_requests.
-    includes_history: bool = False
@@ -40,11 +40,6 @@ class Tenant:
    """

    schema: str
-    # Optional tenant identifier. When provided, background maintenance (e.g. the
-    # consolidation reconcile sweep) can build a RequestContext carrying this id so
-    # tenant-level config overrides are honored. Leave as None for single-tenant
-    # setups or extensions that do not key config by tenant id.
-    tenant_id: str | None = None


 class TenantExtension(Extension, ABC):
@@ -184,8 +184,6 @@ class MetricsCollectorBase:
        input_tokens: int = 0,
        output_tokens: int = 0,
        success: bool = True,
-        cached_input_tokens: int = 0,
-        thoughts_tokens: int = 0,
    ):
        """
        Record metrics for an LLM call.
@@ -195,11 +193,9 @@ class MetricsCollectorBase:
            model: Model name
            scope: Scope identifier (e.g., "memory", "reflect", "consolidation")
            duration: Call duration in seconds
-            input_tokens: Number of input/prompt tokens (total)
-            output_tokens: Number of output/completion tokens visible in candidates
+            input_tokens: Number of input/prompt tokens
+            output_tokens: Number of output/completion tokens
            success: Whether the call was successful
-            cached_input_tokens: Subset of input_tokens billed at the cached rate
-            thoughts_tokens: Reasoning tokens (billed as output, hidden from candidates)
        """
        raise NotImplementedError

@@ -237,8 +233,6 @@ class NoOpMetricsCollector(MetricsCollectorBase):
        input_tokens: int = 0,
        output_tokens: int = 0,
        success: bool = True,
-        cached_input_tokens: int = 0,
-        thoughts_tokens: int = 0,
    ):
        """No-op LLM call recording."""
        pass
@@ -293,27 +287,6 @@ class MetricsCollector(MetricsCollectorBase):
            name="hindsight.llm.calls.total", description="Total number of LLM API calls", unit="calls"
        )

-        # Cached input tokens (subset of input_tokens billed at the cached rate).
-        # Useful for tracking prompt-cache hit-rate independently of total
-        # input volume. provider.scope.model labels matche llm_tokens_input.
-        self.llm_tokens_cached_input = self.meter.create_counter(
-            name="hindsight.llm.tokens.cached_input",
-            description="Number of cached input tokens (billed at cached rate) for LLM calls",
-            unit="tokens",
-        )
-
-        # Thinking / reasoning tokens (Gemini 2.5+ family). Billed at the
-        # output rate by the provider but invisible to candidates_token_count.
-        # Surfacing them as a distinct counter is required for honest cost
-        # attribution: a workload that "looks cheap" by output volume can be
-        # silently expensive if the model is doing long reasoning chains.
-        self.llm_tokens_thoughts = self.meter.create_counter(
-            name="hindsight.llm.tokens.thoughts",
-            description="Number of reasoning/thinking tokens emitted by the model "
-            "(billed as output but not surfaced in candidates)",
-            unit="tokens",
-        )
-
        # HTTP request metrics
        self.http_request_duration = self.meter.create_histogram(
            name="hindsight.http.duration", description="Duration of HTTP requests in seconds", unit="s"
@@ -397,8 +370,6 @@ class MetricsCollector(MetricsCollectorBase):
        input_tokens: int = 0,
        output_tokens: int = 0,
        success: bool = True,
-        cached_input_tokens: int = 0,
-        thoughts_tokens: int = 0,
    ):
        """
        Record metrics for an LLM call.
@@ -408,15 +379,9 @@ class MetricsCollector(MetricsCollectorBase):
            model: Model name
            scope: Scope identifier (e.g., "memory", "reflect", "consolidation")
            duration: Call duration in seconds
-            input_tokens: Number of input/prompt tokens (total, including cached portion)
-            output_tokens: Number of output/completion tokens visible in candidates
+            input_tokens: Number of input/prompt tokens
+            output_tokens: Number of output/completion tokens
            success: Whether the call was successful
-            cached_input_tokens: Subset of input_tokens billed at the cached
-                rate (Gemini context caching). Defaults to 0 when caching is
-                disabled or the provider doesn't surface this field.
-            thoughts_tokens: Reasoning/thinking tokens (Gemini 2.5+ family).
-                Billed at the output rate but not counted in candidates.
-                Defaults to 0 for providers that don't emit thoughts.
        """
        # Base attributes for all metrics
        base_attributes = {
@@ -448,18 +413,6 @@ class MetricsCollector(MetricsCollectorBase):
            }
            self.llm_tokens_output.add(output_tokens, output_attributes)

-        if cached_input_tokens > 0:
-            self.llm_tokens_cached_input.add(
-                cached_input_tokens,
-                {**base_attributes, "token_bucket": get_token_bucket(cached_input_tokens)},
-            )
-
-        if thoughts_tokens > 0:
-            self.llm_tokens_thoughts.add(
-                thoughts_tokens,
-                {**base_attributes, "token_bucket": get_token_bucket(thoughts_tokens)},
-            )
-
    @contextmanager
    def record_http_request(self, method: str, endpoint: str, status_code_getter: Callable[[], int]):
        """
@@ -273,8 +273,6 @@ class LLMSpanRecorder:
        finish_reason: Optional[str] = None,
        error: Optional[Exception] = None,
        tool_calls: Optional[list[dict[str, Any]]] = None,
-        cached_tokens: int = 0,
-        **_extra: Any,
    ) -> None:
        """
        Record a completed LLM call as a span with GenAI semantic conventions.
@@ -295,8 +293,6 @@ class LLMSpanRecorder:
            finish_reason: Reason the model stopped (stop, length, tool_calls, etc.)
            error: Exception if call failed
            tool_calls: List of tool calls made (for function calling)
-            cached_tokens: Cached/cache-read prompt tokens, when reported by the provider.
-            _extra: Tolerated forward-compatible kwargs from other recorders.
        """
        try:
            # Map provider name to GenAI semantic convention
@@ -330,8 +326,6 @@ class LLMSpanRecorder:
                span.set_attribute(GenAIAttributes.RESPONSE_MODEL, model)
                span.set_attribute(GenAIAttributes.USAGE_INPUT_TOKENS, input_tokens)
                span.set_attribute(GenAIAttributes.USAGE_OUTPUT_TOKENS, output_tokens)
-                if cached_tokens:
-                    span.set_attribute("gen_ai.usage.cached_tokens", cached_tokens)

                # Add custom attributes for Hindsight context
                span.set_attribute("hindsight.scope", scope)
@@ -466,61 +460,22 @@ class NoOpLLMSpanRecorder:
        pass


-class CompositeSpanRecorder:
-    """Fans out ``record_llm_call`` to every registered recorder.
-
-    This lets multiple GenAI consumers observe the same LLM calls — e.g. the
-    OpenTelemetry span exporter and the per-bank DB tracer — through the single
-    ``record_llm_call`` chokepoint each provider already calls. A failure in one
-    recorder never affects the others or the LLM call itself.
-    """
-
-    def __init__(self) -> None:
-        self._recorders: list[Any] = []
-
-    def register(self, recorder: Any) -> None:
-        if recorder not in self._recorders:
-            self._recorders.append(recorder)
-
-    def unregister(self, recorder: Any) -> None:
-        if recorder in self._recorders:
-            self._recorders.remove(recorder)
-
-    def record_llm_call(self, **kwargs: Any) -> None:
-        for recorder in self._recorders:
-            try:
-                recorder.record_llm_call(**kwargs)
-            except Exception as e:  # never let one recorder break others
-                logger.debug(f"Span recorder {type(recorder).__name__} failed: {e}", exc_info=True)
-
-
-# Global composite recorder — always present; fans out to whatever is registered.
-_composite_recorder = CompositeSpanRecorder()
-# Backward-compat reference to the OTel recorder (if created).
+# Global span recorder instance
 _span_recorder: Optional[LLMSpanRecorder] = None


-def get_span_recorder() -> CompositeSpanRecorder:
-    """Get the global composite span recorder (fans out to all registered recorders)."""
-    return _composite_recorder
-
-
-def register_span_recorder(recorder: Any) -> None:
-    """Register an additional GenAI recorder (e.g. the per-bank DB tracer)."""
-    _composite_recorder.register(recorder)
-
-
-def unregister_span_recorder(recorder: Any) -> None:
-    """Remove a previously registered recorder."""
-    _composite_recorder.unregister(recorder)
+def get_span_recorder() -> LLMSpanRecorder | NoOpLLMSpanRecorder:
+    """Get the global span recorder (NoOp if tracing disabled)."""
+    if _span_recorder is None:
+        return NoOpLLMSpanRecorder()
+    return _span_recorder


 def create_span_recorder() -> LLMSpanRecorder:
-    """Create and register the OpenTelemetry span recorder."""
+    """Create and set the global span recorder."""
    global _span_recorder
    tracer = get_tracer()
    if tracer is None:
        raise RuntimeError("Tracing not initialized. Call initialize_tracing() first.")
    _span_recorder = LLMSpanRecorder(tracer)
-    register_span_recorder(_span_recorder)
    return _span_recorder
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"

 [project]
 name = "hindsight-api-slim"
-version = "0.8.0"
+version = "0.7.1"
 description = "Hindsight: Agent Memory That Works Like Human Memory"
 readme = "README.md"
 requires-python = ">=3.11"
@@ -81,10 +81,6 @@ local-ml = [
    # Local ML models for embeddings/reranking
    "sentence-transformers>=3.3.0",
    "transformers>=4.53.0", # Security fixes for ReDoS vulnerabilities
-    # transformers (incl. latest 5.x) hard-requires tokenizers<=0.23.0 via a
-    # runtime check; without this cap an in-place upgrade can pull tokenizers
-    # 0.23.1 and break local embeddings/reranker startup. See issue #2055.
-    "tokenizers>=0.22.0,<=0.23.0",
    "torch>=2.6.0", # CVE fix for remote code execution
    "einops>=0.8.2",
    "flashrank>=0.2.0",
@@ -100,14 +96,6 @@ local-llm = [
    "llama-cpp-python[server]>=0.3.0",
    "huggingface-hub>=0.20.0",
 ]
-local-onnx = [
-    # In-process ONNX Runtime embeddings without an Ollama/TEI sidecar
-    "onnxruntime>=1.17.0",
-    "transformers>=4.53.0",
-    "tokenizers>=0.22.0,<=0.23.0", # See issue #2055 (transformers caps tokenizers<=0.23.0)
-    "huggingface-hub>=0.20.0",
-    "numpy>=1.26.0",
-]
 embedded-db = [
    "pg0-embedded>=0.14.2",
 ]
@@ -115,7 +103,7 @@ oracle = [
    "oracledb>=2.5.0",
 ]
 all = [
-    "hindsight-api-slim[local-ml,local-onnx,embedded-db]",
+    "hindsight-api-slim[local-ml,embedded-db]",
 ]
 test = [
    "pytest>=7.0.0",
@@ -187,14 +175,12 @@ dev = [
 [tool.ruff]
 line-length = 120
 target-version = "py311"
+exclude = [
+    "tests/",
+    "**/tests/",
+]

 [tool.ruff.lint]
-# Tests are formatted (via `ruff format`) but excluded from lint rules, which
-# are too noisy for test code (unused imports/vars, import ordering).
-exclude = [
-    "tests/**",
-    "**/tests/**",
-]
 select = [
    "E",      # pycodestyle errors
    "W",      # pycodestyle warnings
@@ -1,7 +1,6 @@
 """
 Pytest configuration and shared fixtures.
 """
-
 import asyncio
 import os
 from pathlib import Path
@@ -21,16 +20,6 @@ from hindsight_api.pg0 import EmbeddedPostgres
 DEFAULT_PG0_INSTANCE_NAME = "hindsight-test"
 DEFAULT_PG0_PORT = int(os.environ.get("HINDSIGHT_TEST_PG_PORT", "5556"))

-# Keep the background MaintenanceLoop from auto-starting during tests. In
-# production it sweeps retention and re-schedules consolidation, but its timers
-# would race shared-pg0 test data (e.g. delete llm_requests/audit_log rows a test
-# just inserted). Disabling the reconcile interval and llm-trace retention — with
-# audit retention already off by default — leaves no job enabled, so the loop
-# never starts. Tests that exercise it call MaintenanceLoop methods
-# (_run_reconcile / _purge_expired) directly.
-os.environ.setdefault("HINDSIGHT_API_CONSOLIDATION_RECONCILE_INTERVAL_SECONDS", "0")
-os.environ.setdefault("HINDSIGHT_API_LLM_TRACE_RETENTION_DAYS", "-1")
-

 # Load environment variables from .env at the start of test session
 def pytest_configure(config):
@@ -77,7 +66,6 @@ def pg0_db_url(db_url, tmp_path_factory, worker_id):
    if db_url and not _parse_pg0_url(db_url)[0]:
        # Plain postgresql:// URL - use it directly but still run migrations
        from hindsight_api.migrations import run_migrations
-
        run_migrations(db_url)
        return db_url

@@ -129,7 +117,6 @@ def pg0_db_url(db_url, tmp_path_factory, worker_id):
    # Run migrations - uses PostgreSQL advisory lock internally,
    # so safe to call from multiple workers (only one will actually run migrations)
    from hindsight_api.migrations import run_migrations
-
    run_migrations(url)

    # Clean up stale test data from previous sessions. Per-bank vector indexes
@@ -160,7 +147,8 @@ def _cleanup_stale_test_data(db_url: str) -> None:
        conn = await asyncpg.connect(db_url)
        try:
            idx_rows = await conn.fetch(
-                "SELECT indexname FROM pg_indexes WHERE schemaname = 'public' AND indexname LIKE 'idx_mu_emb_%'"
+                "SELECT indexname FROM pg_indexes "
+                "WHERE schemaname = 'public' AND indexname LIKE 'idx_mu_emb_%'"
            )
            if idx_rows:
                for row in idx_rows:
@@ -168,20 +156,10 @@ def _cleanup_stale_test_data(db_url: str) -> None:

            # Truncate test data in dependency order
            for table in [
-                "entity_cooccurrences",
-                "unit_entities",
-                "memory_links",
-                "entities",
-                "memory_units",
-                "chunks",
-                "documents",
-                "mental_models",
-                "directives",
-                "async_operations",
-                "audit_log",
-                "webhooks",
-                "file_storage",
-                "banks",
+                "entity_cooccurrences", "unit_entities", "memory_links",
+                "entities", "memory_units", "chunks", "documents",
+                "mental_models", "directives", "async_operations",
+                "audit_log", "webhooks", "file_storage", "banks",
            ]:
                try:
                    await conn.execute(f"TRUNCATE {table} CASCADE")
@@ -264,7 +242,8 @@ def oracle_db_url(_oracle_admin_dsn):
        # Create test user (idempotent — skip if already exists)
        try:
            cursor.execute(
-                f'CREATE USER {test_user} IDENTIFIED BY "{test_pass}" DEFAULT TABLESPACE USERS QUOTA UNLIMITED ON USERS'
+                f'CREATE USER {test_user} IDENTIFIED BY "{test_pass}" '
+                f"DEFAULT TABLESPACE USERS QUOTA UNLIMITED ON USERS"
            )
        except oracledb.DatabaseError as e:
            if hasattr(e.args[0], "code") and e.args[0].code == 1920:
@@ -431,12 +410,13 @@ def cross_encoder(tmp_path_factory, worker_id):

    return ce

-
@pytest.fixture(scope="session")
 def query_analyzer():
    return DateparserQueryAnalyzer()


+
+
@pytest_asyncio.fixture(scope="function")
 async def memory(pg0_db_url, embeddings, cross_encoder, query_analyzer):
    """
@@ -23,7 +23,6 @@ from urllib.parse import urlparse
 # Helpers
 # ---------------------------------------------------------------------------

-
 def _log(step: int, total: int, msg: str) -> None:
    print(f"  [{step}/{total}] {msg}")

@@ -65,7 +64,8 @@ def _bootstrap_test_user(admin_dsn: dict[str, str]) -> str:
        # Create user (skip if already exists - ORA-01920)
        try:
            cursor.execute(
-                f'CREATE USER {test_user} IDENTIFIED BY "{test_pass}" DEFAULT TABLESPACE USERS QUOTA UNLIMITED ON USERS'
+                f'CREATE USER {test_user} IDENTIFIED BY "{test_pass}" '
+                f"DEFAULT TABLESPACE USERS QUOTA UNLIMITED ON USERS"
            )
        except oracledb.DatabaseError as e:
            if hasattr(e.args[0], "code") and e.args[0].code == 1920:
@@ -100,7 +100,6 @@ def _bootstrap_test_user(admin_dsn: dict[str, str]) -> str:
 # Main
 # ---------------------------------------------------------------------------

-
 async def _run() -> None:
    total_steps = 8

@@ -291,7 +290,6 @@ def main() -> int:
    except Exception as exc:
        print(f"\nFAILED: {exc}", file=sys.stderr)
        import traceback
-
        traceback.print_exc()
        return 1

@@ -12,7 +12,6 @@ Usage in tests:
    )
 """

-import asyncio
 import json
 import logging
 import os
@@ -37,19 +36,6 @@ _JUDGE_API_KEY = os.getenv(
 )
 _JUDGE_BASE_URL = os.getenv("HINDSIGHT_TEST_JUDGE_BASE_URL", "")

-# Flakiness hardening. A single temperature-0 judge call still occasionally flips
-# its verdict on borderline phrasing — the dominant source of hs_llm_core
-# flakiness. When the primary verdict is "not met", we ask for a few independent
-# second opinions (at a higher temperature so the samples genuinely differ) and
-# uphold the failure only if the majority agrees. Verdicts that pass on the first
-# call are returned immediately, so passing tests are unaffected in cost or
-# behaviour, and genuine failures (where every judge agrees) still fail.
-_JUDGE_CONFIRMATIONS = int(os.getenv("HINDSIGHT_TEST_JUDGE_CONFIRMATIONS", "2"))
-_JUDGE_CONFIRM_TEMPERATURE = float(os.getenv("HINDSIGHT_TEST_JUDGE_CONFIRM_TEMPERATURE", "0.5"))
-# Retry transient judge-call errors (rate limits, 5xx) so judge infrastructure
-# hiccups never fail the test under evaluation.
-_JUDGE_CALL_ATTEMPTS = int(os.getenv("HINDSIGHT_TEST_JUDGE_CALL_ATTEMPTS", "3"))
-

 class JudgeVerdict(BaseModel):
    meets_criteria: bool
@@ -72,58 +58,6 @@ def _get_judge():
    return _judge_instance


-async def _judge_once(
-    response: str,
-    criteria: str,
-    context: str | None,
-    temperature: float,
-) -> JudgeVerdict:
-    """Run a single judge verdict, retrying transient call errors."""
-    judge = _get_judge()
-    context_block = f"\n\nContext provided to the system:\n{context}" if context else ""
-    messages = [
-        {
-            "role": "system",
-            "content": (
-                "You are a test evaluation judge. Given a response and evaluation criteria, "
-                "determine whether the response meets the criteria. "
-                'Respond with JSON: {"meets_criteria": true/false, "reasoning": "brief explanation"}'
-            ),
-        },
-        {
-            "role": "user",
-            "content": (
-                f"## Response to evaluate\n{response}\n"
-                f"{context_block}\n"
-                f"## Criteria\n{criteria}\n\n"
-                "Does the response meet the criteria?"
-            ),
-        },
-    ]
-
-    last_error: Exception | None = None
-    for attempt in range(max(1, _JUDGE_CALL_ATTEMPTS)):
-        try:
-            result = await judge.call(
-                messages=messages,
-                response_format=JudgeVerdict,
-                max_completion_tokens=256,
-                temperature=temperature,
-                scope="test_judge",
-            )
-            if isinstance(result, JudgeVerdict):
-                return result
-            if isinstance(result, dict):
-                return JudgeVerdict(**result)
-            return JudgeVerdict(**json.loads(str(result)))
-        except Exception as e:  # transient provider error — retry before giving up
-            last_error = e
-            logger.warning(f"Judge call failed (attempt {attempt + 1}/{_JUDGE_CALL_ATTEMPTS}): {e}")
-            await asyncio.sleep(1.0 * (attempt + 1))
-
-    raise RuntimeError(f"Judge call failed after {_JUDGE_CALL_ATTEMPTS} attempts: {last_error}") from last_error
-
-
 async def evaluate(
    response: str,
    criteria: str,
@@ -131,12 +65,6 @@ async def evaluate(
 ) -> JudgeVerdict:
    """Ask the judge LLM whether a response meets the given criteria.

-    The primary verdict is deterministic (temperature 0). If it says the criteria
-    are NOT met, we collect a few independent higher-temperature second opinions
-    and overrule the failure only when the majority disagrees — smoothing out the
-    single-call noise that makes these tests flaky. See the module-level
-    ``_JUDGE_CONFIRMATIONS`` notes.
-
    Args:
        response: The LLM-generated text to evaluate.
        criteria: Plain-English description of what the response should contain/satisfy.
@@ -145,34 +73,44 @@ async def evaluate(
    Returns:
        JudgeVerdict with meets_criteria bool and reasoning string.
    """
-    primary = await _judge_once(response, criteria, context, temperature=0.0)
-    if primary.meets_criteria or _JUDGE_CONFIRMATIONS <= 0:
-        return primary
+    judge = _get_judge()

-    # Primary says "not met": get independent second opinions before trusting it.
-    confirmations = await asyncio.gather(
-        *(
-            _judge_once(response, criteria, context, temperature=_JUDGE_CONFIRM_TEMPERATURE)
-            for _ in range(_JUDGE_CONFIRMATIONS)
-        ),
-        return_exceptions=True,
-    )
-    verdicts = [primary] + [c for c in confirmations if isinstance(c, JudgeVerdict)]
-    met = sum(1 for v in verdicts if v.meets_criteria)
-    not_met = len(verdicts) - met
+    context_block = f"\n\nContext provided to the system:\n{context}" if context else ""

-    if met > not_met:
-        agreeing = next(v for v in verdicts if v.meets_criteria)
-        logger.info(f"Judge: primary 'not met' overruled by majority ({met}/{len(verdicts)} met). Criteria: {criteria}")
-        return JudgeVerdict(
-            meets_criteria=True,
-            reasoning=f"Majority of {len(verdicts)} judges met criteria (primary verdict overruled as noise). {agreeing.reasoning}",
-        )
-    return JudgeVerdict(
-        meets_criteria=False,
-        reasoning=f"{not_met}/{len(verdicts)} judges agree criteria not met. {primary.reasoning}",
+    result = await judge.call(
+        messages=[
+            {
+                "role": "system",
+                "content": (
+                    "You are a test evaluation judge. Given a response and evaluation criteria, "
+                    "determine whether the response meets the criteria. "
+                    "Respond with JSON: {\"meets_criteria\": true/false, \"reasoning\": \"brief explanation\"}"
+                ),
+            },
+            {
+                "role": "user",
+                "content": (
+                    f"## Response to evaluate\n{response}\n"
+                    f"{context_block}\n"
+                    f"## Criteria\n{criteria}\n\n"
+                    "Does the response meet the criteria?"
+                ),
+            },
+        ],
+        response_format=JudgeVerdict,
+        max_completion_tokens=256,
+        temperature=0.0,
+        scope="test_judge",
    )

+    if isinstance(result, JudgeVerdict):
+        return result
+
+    # Fallback: parse raw dict/string
+    if isinstance(result, dict):
+        return JudgeVerdict(**result)
+    return JudgeVerdict(**json.loads(str(result)))
+

 async def assert_meets_criteria(
    response: str,
@@ -186,7 +124,7 @@ async def assert_meets_criteria(
    """
    verdict = await evaluate(response=response, criteria=criteria, context=context)
    if not verdict.meets_criteria:
-        fail_msg = msg or "LLM judge: criteria not met"
+        fail_msg = msg or f"LLM judge: criteria not met"
        raise AssertionError(
            f"{fail_msg}\n"
            f"  Criteria: {criteria}\n"
@@ -1,128 +0,0 @@
-"""Tests for the admin surface: GET /admin/config + the admin_api feature flag.
-
-These are deterministic (no LLM): the endpoint only reads server-level config. We
-toggle env vars + clear the config cache to exercise the enable flag, the optional
-admin token, and credential redaction.
-"""
-
-import httpx
-import pytest
-import pytest_asyncio
-
-from hindsight_api.api import create_app
-from hindsight_api.config import clear_config_cache
-
-
-@pytest_asyncio.fixture
-async def admin_client(memory):
-    """Async test client for the FastAPI app (mock LLM)."""
-    app = create_app(memory, initialize_memory=False)
-    transport = httpx.ASGITransport(app=app)
-    async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
-        yield client
-
-
-def _set_env(monkeypatch, **values: str | None) -> None:
-    """Set/unset env vars and reset the cached config so the next read reflects them."""
-    for key, value in values.items():
-        if value is None:
-            monkeypatch.delenv(key, raising=False)
-        else:
-            monkeypatch.setenv(key, value)
-    clear_config_cache()
-
-
-@pytest.fixture(autouse=True)
-def _restore_config_cache():
-    """Ensure the global config cache is reset after each test."""
-    yield
-    clear_config_cache()
-
-
-@pytest.mark.asyncio
-async def test_admin_config_disabled_by_default(admin_client, monkeypatch):
-    """When the admin API is disabled (default), the endpoint is invisible (404)."""
-    _set_env(monkeypatch, HINDSIGHT_API_ENABLE_ADMIN_API=None, HINDSIGHT_API_ADMIN_TOKEN=None)
-
-    response = await admin_client.get("/admin/config")
-
-    assert response.status_code == 404
-
-
-@pytest.mark.asyncio
-async def test_admin_config_enabled_no_token(admin_client, monkeypatch):
-    """When enabled without a token, the endpoint is open and returns config."""
-    _set_env(monkeypatch, HINDSIGHT_API_ENABLE_ADMIN_API="true", HINDSIGHT_API_ADMIN_TOKEN=None)
-
-    response = await admin_client.get("/admin/config")
-
-    assert response.status_code == 200
-    config = response.json()["config"]
-    # A representative spread of non-credential fields should be present.
-    assert "llm_provider" in config
-    assert "enable_admin_api" in config
-    assert config["enable_admin_api"] is True
-
-
-@pytest.mark.asyncio
-async def test_admin_config_redacts_credentials(admin_client, monkeypatch):
-    """Credential fields are masked, never returned in cleartext."""
-    _set_env(
-        monkeypatch,
-        HINDSIGHT_API_ENABLE_ADMIN_API="true",
-        HINDSIGHT_API_ADMIN_TOKEN="s3cret-token",
-        HINDSIGHT_API_LLM_API_KEY="super-secret-key",
-    )
-
-    response = await admin_client.get("/admin/config", headers={"Authorization": "Bearer s3cret-token"})
-
-    assert response.status_code == 200
-    config = response.json()["config"]
-    # The configured LLM key is present but masked.
-    assert config["llm_api_key"] == "***"
-    assert "super-secret-key" not in response.text
-    # Provider keys that fall back to the LLM key (and aren't in the credential
-    # denylist) must also be masked — the view redacts by name, not just the set.
-    assert config["embeddings_openrouter_api_key"] == "***"
-    assert config["reranker_openrouter_api_key"] == "***"
-    # The admin token must never leak through its own config view.
-    assert config["admin_api_token"] == "***"
-    assert "s3cret-token" not in response.text
-    # Value-bearing fields that merely contain "token" in their name (plural) are
-    # NOT redacted — they carry useful config, not secrets.
-    assert config["recall_max_tokens"] != "***"
-
-
-@pytest.mark.asyncio
-async def test_admin_config_requires_token_when_set(admin_client, monkeypatch):
-    """With a token configured, missing/wrong tokens are rejected; the right one passes."""
-    _set_env(
-        monkeypatch,
-        HINDSIGHT_API_ENABLE_ADMIN_API="true",
-        HINDSIGHT_API_ADMIN_TOKEN="right-token",
-    )
-
-    missing = await admin_client.get("/admin/config")
-    assert missing.status_code == 401
-
-    wrong = await admin_client.get("/admin/config", headers={"Authorization": "Bearer wrong-token"})
-    assert wrong.status_code == 401
-
-    bearer = await admin_client.get("/admin/config", headers={"Authorization": "Bearer right-token"})
-    assert bearer.status_code == 200
-
-    # A bare token (no "Bearer " prefix) is also accepted.
-    bare = await admin_client.get("/admin/config", headers={"Authorization": "right-token"})
-    assert bare.status_code == 200
-
-
-@pytest.mark.asyncio
-async def test_version_reports_admin_api_flag(admin_client, monkeypatch):
-    """The /version feature flags track the admin enable flag."""
-    _set_env(monkeypatch, HINDSIGHT_API_ENABLE_ADMIN_API="true")
-    enabled = await admin_client.get("/version")
-    assert enabled.json()["features"]["admin_api"] is True
-
-    _set_env(monkeypatch, HINDSIGHT_API_ENABLE_ADMIN_API="false")
-    disabled = await admin_client.get("/version")
-    assert disabled.json()["features"]["admin_api"] is False
@@ -1,7 +1,6 @@
 """
 Tests for agent management API (profile, disposition).
 """
-
 import pytest
 import uuid
 from hindsight_api import MemoryEngine, RequestContext
@@ -18,7 +17,9 @@ class TestAgentProfile:
    """Tests for agent profile management."""

    @pytest.mark.asyncio
-    async def test_get_bank_profile_no_auto_create_returns_none(self, memory: MemoryEngine, request_context):
+    async def test_get_bank_profile_no_auto_create_returns_none(
+        self, memory: MemoryEngine, request_context
+    ):
        """When create_if_missing=False is passed, a missing bank returns None
        rather than being silently auto-created. This is what read-only
        endpoints (HTTP GET, polling, etc.) must use to avoid creating banks
@@ -26,20 +27,28 @@ class TestAgentProfile:
        bank_id = unique_agent_id("test_no_auto_create")

        # First call with create_if_missing=False on a non-existent bank
-        result = await memory.get_bank_profile(bank_id, request_context=request_context, create_if_missing=False)
+        result = await memory.get_bank_profile(
+            bank_id, request_context=request_context, create_if_missing=False
+        )
        assert result is None, "Expected None for missing bank with create_if_missing=False"

        # Verify the bank was NOT created as a side effect
-        result_again = await memory.get_bank_profile(bank_id, request_context=request_context, create_if_missing=False)
+        result_again = await memory.get_bank_profile(
+            bank_id, request_context=request_context, create_if_missing=False
+        )
        assert result_again is None, "Bank must not exist after read-only call"

        # And explicit auto-create still works
-        created = await memory.get_bank_profile(bank_id, request_context=request_context, create_if_missing=True)
+        created = await memory.get_bank_profile(
+            bank_id, request_context=request_context, create_if_missing=True
+        )
        assert created is not None
        assert created["disposition"]["skepticism"] == 3

        # Now read-only call sees it
-        seen = await memory.get_bank_profile(bank_id, request_context=request_context, create_if_missing=False)
+        seen = await memory.get_bank_profile(
+            bank_id, request_context=request_context, create_if_missing=False
+        )
        assert seen is not None
        assert seen["disposition"]["skepticism"] == 3

@@ -113,7 +122,11 @@ class TestAgentEndpoint:
        bank_id = unique_agent_id("test_put_create")

        request = CreateBankRequest(
-            disposition=DispositionTraits(skepticism=4, literalism=5, empathy=2),
+            disposition=DispositionTraits(
+                skepticism=4,
+                literalism=5,
+                empathy=2
+            ),
        )

        profile = await memory.get_bank_profile(bank_id, request_context=request_context)
@@ -142,7 +155,7 @@ class TestAgentDispositionIntegration:
        disposition = {
            "skepticism": 5,  # Very skeptical
            "literalism": 4,  # High literalism
-            "empathy": 2,  # Low empathy
+            "empathy": 2,     # Low empathy
        }
        await memory.update_bank_disposition(bank_id, disposition, request_context=request_context)

@@ -150,7 +163,7 @@ class TestAgentDispositionIntegration:
            bank_id=bank_id,
            contents=[
                {"content": "Traditional painting techniques have been used for centuries"},
-                {"content": "Modern digital art is changing the art world"},
+                {"content": "Modern digital art is changing the art world"}
            ],
            request_context=request_context,
        )
@@ -103,11 +103,6 @@ async def test_small_async_batch_no_splitting(memory, request_context):
    assert status["result_metadata"]["num_sub_batches"] == 1  # Single sub-batch
    assert len(status["child_operations"]) == 1
    assert status["child_operations"][0]["status"] == "completed"
-    child_meta = await _child_metadata(memory, bank_id, operation_id, request_context)
-    assert child_meta["unit_ids_count"] > 0
-    assert child_meta["extraction_errors_count"] == 0
-    assert status["result_metadata"]["unit_ids_count"] == child_meta["unit_ids_count"]
-    assert status["result_metadata"]["extraction_errors_count"] == 0


@pytest.mark.asyncio
@@ -171,19 +166,6 @@ async def test_large_async_batch_auto_splits(memory, request_context):

    # Parent status should be aggregated as "completed"
    assert parent_status["status"] == "completed"
-    child_unit_counts = []
-    for child in child_ops:
-        child_status = await memory.get_operation_status(
-            bank_id=bank_id,
-            operation_id=child["operation_id"],
-            request_context=request_context,
-        )
-        child_meta = child_status["result_metadata"]
-        assert child_meta["unit_ids_count"] > 0
-        assert child_meta["extraction_errors_count"] == 0
-        child_unit_counts.append(child_meta["unit_ids_count"])
-    assert parent_status["result_metadata"]["unit_ids_count"] == sum(child_unit_counts)
-    assert parent_status["result_metadata"]["extraction_errors_count"] == 0


@pytest.mark.asyncio
@@ -479,42 +461,6 @@ async def _child_metadata(memory, bank_id: str, parent_operation_id: str, reques
    return child["result_metadata"]


-@pytest.mark.asyncio
-async def test_retain_outcome_metadata_records_zero_counts(memory, request_context, monkeypatch):
-    """Completed retain operations expose explicit zero outcome counters."""
-    from hindsight_api.engine.response_models import TokenUsage
-    from hindsight_api.engine.retain import fact_extraction
-
-    async def empty_extract_facts_from_contents(
-        *args: object, **kwargs: object
-    ) -> tuple[list[object], list[object], TokenUsage]:
-        return [], [], TokenUsage()
-
-    monkeypatch.setattr(fact_extraction, "extract_facts_from_contents", empty_extract_facts_from_contents)
-
-    bank_id = "test_retain_outcome_zero_counts"
-    result = await memory.submit_async_retain(
-        bank_id=bank_id,
-        contents=[{"content": "No extracted facts for this item."}],
-        request_context=request_context,
-    )
-    await asyncio.sleep(0.2)
-
-    parent = await memory.get_operation_status(
-        bank_id=bank_id,
-        operation_id=result["operation_id"],
-        request_context=request_context,
-    )
-    child_meta = await _child_metadata(memory, bank_id, result["operation_id"], request_context)
-
-    assert child_meta["unit_ids_count"] == 0
-    assert child_meta["extraction_errors_count"] == 0
-    assert "extraction_errors_sample" not in child_meta
-    assert parent["result_metadata"]["unit_ids_count"] == 0
-    assert parent["result_metadata"]["extraction_errors_count"] == 0
-    assert "extraction_errors_sample" not in parent["result_metadata"]
-
-
@pytest.mark.asyncio
 async def test_retain_records_user_provided_document_ids(memory, request_context):
    """User-supplied document_ids land in child op result_metadata.document_ids."""
@@ -1114,67 +1060,3 @@ async def test_submit_async_batch_retain_rolls_back_parent_on_child_failure(
        f"{[(r['operation_type'], r['status'], r['task_payload'] is not None) for r in rows]}. "
        "The parent INSERT must be transactionally coupled to the child INSERTs."
    )
-
-
-@pytest.mark.asyncio
-async def test_submit_async_batch_retain_creates_missing_bank(memory, request_context, monkeypatch):
-    """First async retain to a new bank lazily creates the bank (async_operations
-    has an FK to banks) instead of raising a constraint error."""
-
-    async def noop_submit_task(_task_dict):
-        return None
-
-    monkeypatch.setattr(memory._task_backend, "submit_task", noop_submit_task)
-
-    bank_id = f"test_batch_newbank_{uuid.uuid4().hex[:8]}"
-    pool = await memory._get_pool()
-
-    await memory.submit_async_retain(
-        bank_id=bank_id,
-        contents=[{"content": "Alice works at Google.", "document_id": "doc1"}],
-        request_context=request_context,
-    )
-
-    bank = await pool.fetchrow("SELECT bank_id FROM banks WHERE bank_id = $1", bank_id)
-    assert bank is not None, "submit_async_retain should have lazily created the bank"
-
-
-@pytest.mark.asyncio
-async def test_submit_async_batch_retain_rolls_back_missing_bank_on_child_failure(
-    memory_no_llm_verify, request_context, monkeypatch
-):
-    """The lazy bank-create shares the parent+child transaction. When the child
-    loop fails for a bank that did not previously exist, the freshly-created bank
-    must roll back together with the operation rows — no orphan bank."""
-    import hindsight_api.engine.memory_engine as me
-    from hindsight_api.engine.memory_engine import count_tokens
-
-    bank_id = f"test_batch_bank_rollback_{uuid.uuid4().hex[:8]}"
-    pool = await memory_no_llm_verify._get_pool()
-    # Intentionally do NOT pre-create the bank — it must be created (and then
-    # rolled back) inside submit_async_retain's transaction.
-
-    large_content = "The quick brown fox jumps over the lazy dog. " * 500
-    contents = [{"content": large_content + f" item {i}", "document_id": f"doc{i}"} for i in range(2)]
-    assert sum(count_tokens(item["content"]) for item in contents) > 10_000
-
-    real_class = me.BatchRetainChildMetadata
-    call_count = {"n": 0}
-
-    def failing_child_metadata(*args, **kwargs):
-        call_count["n"] += 1
-        if call_count["n"] == 2:
-            raise RuntimeError("Simulated child-step failure mid-batch")
-        return real_class(*args, **kwargs)
-
-    monkeypatch.setattr(me, "BatchRetainChildMetadata", failing_child_metadata)
-
-    with pytest.raises(RuntimeError, match="Simulated child-step failure"):
-        await memory_no_llm_verify.submit_async_retain(
-            bank_id=bank_id,
-            contents=contents,
-            request_context=request_context,
-        )
-
-    bank = await pool.fetchrow("SELECT bank_id FROM banks WHERE bank_id = $1", bank_id)
-    assert bank is None, "the lazily-created bank must roll back with the failed operation inserts"
@@ -1,6 +1,6 @@
 """Unit tests for async retain tag propagation."""

-from unittest.mock import AsyncMock, MagicMock
+from unittest.mock import AsyncMock, MagicMock, patch

 import pytest

@@ -56,18 +56,19 @@ async def test_submit_async_retain_includes_document_tags_in_task_payload():
    contents = [{"content": "Async retain payload test."}]
    document_tags = ["scope:tools", "user:alice"]

-    # Stub the lazy bank-create/default-template hook to a no-op (created=False)
-    # so the inline transaction path runs against the mock connection without
-    # real DB work. The hook itself is covered by dedicated tests.
-    engine._ensure_bank_exists = AsyncMock(return_value=False)
-
-    result = await MemoryEngine.submit_async_retain(
-        engine,
-        bank_id="bank-1",
-        contents=contents,
-        document_tags=document_tags,
-        request_context=request_context,
-    )
+    # Return (profile, created=False) so the default-template-on-create hook is skipped.
+    with patch(
+        "hindsight_api.engine.memory_engine.bank_utils.get_or_create_bank_profile",
+        new_callable=AsyncMock,
+        return_value=(MagicMock(), False),
+    ):
+        result = await MemoryEngine.submit_async_retain(
+            engine,
+            bank_id="bank-1",
+            contents=contents,
+            document_tags=document_tags,
+            request_context=request_context,
+        )

    # Check result structure
    assert "operation_id" in result
@@ -5,7 +5,6 @@ Covers the new fields exposed by GET /v1/default/banks/{bank_id}/stats
 (operations_by_status) and the new endpoint
 GET /v1/default/banks/{bank_id}/stats/memories-timeseries.
 """
-
 import uuid
 from datetime import datetime

@@ -84,7 +83,9 @@ async def test_bank_stats_exposes_operations_by_status(api_client, test_bank_id)
        ("90d", 90, "day"),
    ],
 )
-async def test_memories_timeseries_periods(api_client, test_bank_id, period, expected_count, expected_trunc):
+async def test_memories_timeseries_periods(
+    api_client, test_bank_id, period, expected_count, expected_trunc
+):
    """Every period must return the full expected bucket count and trunc."""
    try:
        response = await api_client.post(
@@ -138,7 +139,9 @@ async def test_memories_timeseries_invalid_period_falls_back(api_client, test_ba


@pytest.mark.asyncio
-async def test_memories_timeseries_empty_bank_returns_zero_filled_buckets(api_client, test_bank_id):
+async def test_memories_timeseries_empty_bank_returns_zero_filled_buckets(
+    api_client, test_bank_id
+):
    """A bank with no memories must still return the full zero-filled bucket set."""
    try:
        response = await api_client.get(
@@ -239,146 +242,3 @@ async def test_list_memories_filter_by_consolidation_state_rejects_unknown(api_c
        assert response.status_code == 400
    finally:
        await api_client.delete(f"/v1/default/banks/{test_bank_id}")
-
-
-@pytest.mark.asyncio
-async def test_bank_stats_link_counts_have_no_join(api_client, test_bank_id):
-    """link_counts must be populated; the deprecated breakdown fields must be empty.
-
-    Confirms the simplified single-table aggregation still produces the totals
-    the UI reads (`links_by_link_type`) without the historical
-    memory_links⇒memory_units join that powered the 2D `links_breakdown` no
-    consumer reads.
-    """
-    try:
-        response = await api_client.post(
-            f"/v1/default/banks/{test_bank_id}/memories",
-            json={"items": [{"content": "Carol leads platform engineering.", "context": "team"}]},
-        )
-        assert response.status_code == 200
-
-        response = await api_client.get(f"/v1/default/banks/{test_bank_id}/stats")
-        assert response.status_code == 200
-        stats = response.json()
-
-        # link totals must still come back so the UI overview cards render.
-        assert isinstance(stats["links_by_link_type"], dict)
-        assert stats["total_links"] >= 0
-
-        # Deprecated breakdown fields stay in the response shape but are empty.
-        assert stats["links_breakdown"] == {}
-        assert stats["links_by_fact_type"] == {}
-    finally:
-        await api_client.delete(f"/v1/default/banks/{test_bank_id}")
-
-
-@pytest.mark.asyncio
-async def test_get_bank_freshness_returns_only_consolidation_fields(memory, test_bank_id):
-    """get_bank_freshness must return just the freshness keys, no link aggregation."""
-    from hindsight_api.extensions import RequestContext
-
-    try:
-        await _insert_memory(memory, test_bank_id, "Headed for consolidation.", failed=False)
-        await _insert_memory(memory, test_bank_id, "Also pending.", failed=True)
-
-        freshness = await memory.get_bank_freshness(
-            test_bank_id,
-            request_context=RequestContext(internal=True),
-        )
-
-        assert set(freshness.keys()) == {
-            "last_consolidated_at",
-            "pending_consolidation",
-            "failed_consolidation",
-        }
-        assert freshness["pending_consolidation"] >= 2
-        assert freshness["failed_consolidation"] >= 1
-    finally:
-        await memory._bank_stats_cache.clear()
-        async with memory._pool.acquire() as conn:
-            await conn.execute("DELETE FROM memory_units WHERE bank_id = $1", test_bank_id)
-
-
-@pytest.mark.asyncio
-async def test_reflect_uses_freshness_not_bank_stats(memory, test_bank_id):
-    """reflect() must call the cheap freshness query, not get_bank_stats.
-
-    Counts calls to `_compute_bank_stats` (the heavy loader) during a reflect
-    invocation; it must stay at zero — reflect should route through
-    `get_bank_freshness` instead.
-    """
-    from hindsight_api.extensions import RequestContext
-
-    try:
-        # Seed a single memory so reflect has something to inspect.
-        await _insert_memory(memory, test_bank_id, "Reflect seed.", failed=False)
-
-        compute_calls = 0
-        original_compute = memory._compute_bank_stats
-
-        async def counting_compute(bank_id: str):
-            nonlocal compute_calls
-            compute_calls += 1
-            return await original_compute(bank_id)
-
-        memory._compute_bank_stats = counting_compute  # type: ignore[method-assign]
-        try:
-            await memory._bank_stats_cache.clear()
-            try:
-                await memory.reflect(
-                    test_bank_id,
-                    "What do you know about this bank?",
-                    request_context=RequestContext(internal=True),
-                )
-            except Exception:
-                # reflect may fail without a configured LLM in this test env;
-                # we only care that it did not invoke the heavy stats loader
-                # before failing.
-                pass
-            assert compute_calls == 0
-        finally:
-            memory._compute_bank_stats = original_compute  # type: ignore[method-assign]
-    finally:
-        await memory._bank_stats_cache.clear()
-        async with memory._pool.acquire() as conn:
-            await conn.execute("DELETE FROM memory_units WHERE bank_id = $1", test_bank_id)
-
-
-@pytest.mark.asyncio
-async def test_bank_stats_served_from_cache_on_repeat_call(api_client, memory, test_bank_id):
-    """A second /stats call within the TTL must not re-run the aggregations.
-
-    The cache layer wraps the DB-heavy `_compute_bank_stats` body; counting
-    its invocations is the cleanest way to prove the wiring works without
-    relying on timing.
-    """
-    try:
-        response = await api_client.post(
-            f"/v1/default/banks/{test_bank_id}/memories",
-            json={"items": [{"content": "Bob is a project manager.", "context": "team"}]},
-        )
-        assert response.status_code == 200
-
-        original = memory._compute_bank_stats
-        call_count = 0
-
-        async def counting_compute(bank_id: str):
-            nonlocal call_count
-            call_count += 1
-            return await original(bank_id)
-
-        # Make sure no stale entry exists from prior test ordering.
-        await memory._bank_stats_cache.clear()
-        memory._compute_bank_stats = counting_compute  # type: ignore[method-assign]
-        try:
-            first = await api_client.get(f"/v1/default/banks/{test_bank_id}/stats")
-            second = await api_client.get(f"/v1/default/banks/{test_bank_id}/stats")
-            assert first.status_code == 200
-            assert second.status_code == 200
-            assert first.json() == second.json()
-            assert call_count == 1
-        finally:
-            memory._compute_bank_stats = original  # type: ignore[method-assign]
-            await memory._bank_stats_cache.clear()
-    finally:
-        await api_client.delete(f"/v1/default/banks/{test_bank_id}")
@@ -1,201 +0,0 @@
-"""Unit tests for `BankStatsCache` — TTL, eviction, and concurrent coalescing.
-
-These tests don't touch the database; they exercise the cache wrapper
-directly so the semantics are checked in isolation from `MemoryEngine`.
-"""
-
-from __future__ import annotations
-
-import asyncio
-from typing import Any
-
-import pytest
-
-from hindsight_api.engine.bank_stats_cache import BankStatsCache
-
-
-def make_loader(return_value: dict[str, Any]) -> tuple[Any, list[int]]:
-    """Returns (loader_fn, call_count_list). `call_count_list[0]` is the count."""
-    calls = [0]
-
-    async def loader() -> dict[str, Any]:
-        calls[0] += 1
-        return return_value
-
-    return loader, calls
-
-
-@pytest.mark.asyncio
-async def test_cache_disabled_passes_through() -> None:
-    cache = BankStatsCache(ttl_seconds=0, max_entries=100)
-    loader, calls = make_loader({"v": 1})
-
-    for _ in range(3):
-        result = await cache.get_or_load("schema", "bank", loader)
-        assert result == {"v": 1}
-    assert calls[0] == 3
-
-
-@pytest.mark.asyncio
-async def test_cache_serves_hits_within_ttl() -> None:
-    cache = BankStatsCache(ttl_seconds=60, max_entries=100)
-    loader, calls = make_loader({"v": 1})
-
-    first = await cache.get_or_load("schema", "bank", loader)
-    second = await cache.get_or_load("schema", "bank", loader)
-    assert first == second == {"v": 1}
-    assert calls[0] == 1
-
-
-@pytest.mark.asyncio
-async def test_cache_reloads_after_ttl_expires(monkeypatch) -> None:
-    cache = BankStatsCache(ttl_seconds=0.05, max_entries=100)
-    loader, calls = make_loader({"v": 1})
-
-    fake_time = [1000.0]
-    monkeypatch.setattr(cache, "_now", lambda: fake_time[0])
-
-    await cache.get_or_load("schema", "bank", loader)
-    fake_time[0] += 0.1  # advance past TTL
-    await cache.get_or_load("schema", "bank", loader)
-    assert calls[0] == 2
-
-
-@pytest.mark.asyncio
-async def test_cache_isolates_by_schema_and_bank() -> None:
-    cache = BankStatsCache(ttl_seconds=60, max_entries=100)
-    loader, calls = make_loader({"v": 1})
-
-    await cache.get_or_load("schema_a", "bank", loader)
-    await cache.get_or_load("schema_b", "bank", loader)
-    await cache.get_or_load("schema_a", "other", loader)
-    # 3 distinct keys → 3 loader calls.
-    assert calls[0] == 3
-
-
-@pytest.mark.asyncio
-async def test_concurrent_misses_are_coalesced() -> None:
-    """6 concurrent callers on the same cold key must trigger exactly one loader."""
-    cache = BankStatsCache(ttl_seconds=60, max_entries=100)
-    calls = [0]
-    started = asyncio.Event()
-    release = asyncio.Event()
-
-    async def slow_loader() -> dict[str, Any]:
-        calls[0] += 1
-        started.set()
-        await release.wait()
-        return {"v": calls[0]}
-
-    tasks = [asyncio.create_task(cache.get_or_load("schema", "bank", slow_loader)) for _ in range(6)]
-    await started.wait()
-    # All other tasks should now be queued behind the in-flight loader.
-    release.set()
-    results = await asyncio.gather(*tasks)
-
-    assert calls[0] == 1
-    assert all(r == {"v": 1} for r in results)
-
-
-@pytest.mark.asyncio
-async def test_loader_exception_does_not_poison_cache() -> None:
-    cache = BankStatsCache(ttl_seconds=60, max_entries=100)
-    calls = [0]
-
-    async def flaky_loader() -> dict[str, Any]:
-        calls[0] += 1
-        if calls[0] == 1:
-            raise RuntimeError("boom")
-        return {"v": calls[0]}
-
-    with pytest.raises(RuntimeError, match="boom"):
-        await cache.get_or_load("schema", "bank", flaky_loader)
-
-    # Second call should still attempt the loader (cache wasn't populated).
-    result = await cache.get_or_load("schema", "bank", flaky_loader)
-    assert result == {"v": 2}
-    assert calls[0] == 2
-
-
-@pytest.mark.asyncio
-async def test_concurrent_loader_exception_propagates_to_waiters() -> None:
-    cache = BankStatsCache(ttl_seconds=60, max_entries=100)
-    started = asyncio.Event()
-    release = asyncio.Event()
-
-    async def failing_loader() -> dict[str, Any]:
-        started.set()
-        await release.wait()
-        raise RuntimeError("loader failed")
-
-    tasks = [asyncio.create_task(cache.get_or_load("schema", "bank", failing_loader)) for _ in range(3)]
-    await started.wait()
-    release.set()
-    results = await asyncio.gather(*tasks, return_exceptions=True)
-    assert all(isinstance(r, RuntimeError) for r in results)
-
-
-@pytest.mark.asyncio
-async def test_lru_eviction_respects_max_entries() -> None:
-    cache = BankStatsCache(ttl_seconds=60, max_entries=2)
-
-    async def loader_for(value: int):
-        async def _loader() -> dict[str, Any]:
-            return {"v": value}
-
-        return _loader
-
-    await cache.get_or_load("s", "a", await loader_for(1))
-    await cache.get_or_load("s", "b", await loader_for(2))
-    # Touch "a" so it's most-recently-used.
-    await cache.get_or_load("s", "a", await loader_for(99))
-    # Insert "c" — should evict "b" (the LRU), not "a".
-    await cache.get_or_load("s", "c", await loader_for(3))
-
-    # "a" is still cached (loader for "a" with value=99 must NOT be called again).
-    miss_check_calls = [0]
-
-    async def should_not_run() -> dict[str, Any]:
-        miss_check_calls[0] += 1
-        return {"v": -1}
-
-    cached_a = await cache.get_or_load("s", "a", should_not_run)
-    assert cached_a == {"v": 1}
-    assert miss_check_calls[0] == 0
-
-    # "b" was evicted; the loader must run on the next get.
-    new_b_calls = [0]
-
-    async def new_b() -> dict[str, Any]:
-        new_b_calls[0] += 1
-        return {"v": 200}
-
-    fetched_b = await cache.get_or_load("s", "b", new_b)
-    assert fetched_b == {"v": 200}
-    assert new_b_calls[0] == 1
-
-
-@pytest.mark.asyncio
-async def test_invalidate_drops_entry() -> None:
-    cache = BankStatsCache(ttl_seconds=60, max_entries=100)
-    loader, calls = make_loader({"v": 1})
-
-    await cache.get_or_load("schema", "bank", loader)
-    await cache.invalidate("schema", "bank")
-    await cache.get_or_load("schema", "bank", loader)
-    assert calls[0] == 2
-
-
-@pytest.mark.asyncio
-async def test_clear_drops_all_entries() -> None:
-    cache = BankStatsCache(ttl_seconds=60, max_entries=100)
-    loader, calls = make_loader({"v": 1})
-
-    await cache.get_or_load("s", "a", loader)
-    await cache.get_or_load("s", "b", loader)
-    assert calls[0] == 2
-
-    await cache.clear()
-    await cache.get_or_load("s", "a", loader)
-    await cache.get_or_load("s", "b", loader)
-    assert calls[0] == 4
@@ -643,7 +643,9 @@ class TestDefaultBankTemplateEnvVar:
        yield default_template

    @pytest.mark.asyncio
-    async def test_default_template_applied_on_new_bank(self, api_client, bank_id, _patched_default_template):
+    async def test_default_template_applied_on_new_bank(
+        self, api_client, bank_id, _patched_default_template
+    ):
        """Creating a new bank applies the default template (config + mental models + directives)."""
        # Trigger bank auto-creation via GET profile
        resp = await api_client.put(f"/v1/default/banks/{bank_id}", json={})
@@ -728,7 +730,9 @@ class TestDefaultBankTemplateEnvVar:
        assert config_resp.json()["overrides"] == {}

    @pytest.mark.asyncio
-    async def test_default_template_malformed_is_swallowed(self, api_client, bank_id, monkeypatch):
+    async def test_default_template_malformed_is_swallowed(
+        self, api_client, bank_id, monkeypatch
+    ):
        """A malformed default template is logged and ignored — bank creation still succeeds."""
        from hindsight_api.config import _get_raw_config

@@ -4,7 +4,6 @@ Integration test for API base path support.
 Tests that the API works correctly when deployed with a base path (e.g., /hindsight)
 for reverse proxy deployments.
 """
-
 import os
 import pytest
 import pytest_asyncio
@@ -28,7 +27,10 @@ async def api_client_with_base_path(memory):

    # Use base_url with base path
    transport = httpx.ASGITransport(app=app)
-    async with httpx.AsyncClient(transport=transport, base_url=f"http://test{base_path}") as client:
+    async with httpx.AsyncClient(
+        transport=transport,
+        base_url=f"http://test{base_path}"
+    ) as client:
        yield client

    # Cleanup: unset base path
@@ -120,10 +122,10 @@ async def test_base_path_full_workflow(api_client_with_base_path):
            "items": [
                {
                    "content": "The API supports base path deployment for reverse proxy use cases.",
-                    "context": "testing base path feature",
+                    "context": "testing base path feature"
                }
            ]
-        },
+        }
    )
    assert response.status_code == 200
    result = response.json()
@@ -131,7 +133,10 @@ async def test_base_path_full_workflow(api_client_with_base_path):

    # 3. Recall the memory
    response = await api_client_with_base_path.post(
-        f"/v1/default/banks/{bank_id}/memories/recall", json={"query": "base path support"}
+        f"/v1/default/banks/{bank_id}/memories/recall",
+        json={
+            "query": "base path support"
+        }
    )
    assert response.status_code == 200
    recall_result = response.json()
@@ -7,24 +7,21 @@ Tests cover:
 - Hard error when provider doesn't support the batch API (no silent fallback)
 - Worker recovery on restart
 """
-
+import pytest
 import asyncio
-import json
 import logging
+import json
 import uuid
 from datetime import datetime, timezone
 from unittest.mock import AsyncMock, MagicMock, patch
-
-import pytest
-
 from hindsight_api import RequestContext
+from hindsight_api.engine.retain.fact_extraction import (
+    extract_facts_from_contents_batch_api,
+    extract_facts_from_contents,
+    RetainContent,
+)
 from hindsight_api.config import HindsightConfig
 from hindsight_api.engine.llm_wrapper import create_llm_provider
-from hindsight_api.engine.retain.fact_extraction import (
-    RetainContent,
-    extract_facts_from_contents,
-    extract_facts_from_contents_batch_api,
-)
 from hindsight_api.worker.poller import WorkerPoller

 logger = logging.getLogger(__name__)
@@ -106,21 +103,19 @@ async def test_batch_api_normal_flow(mock_llm_config, test_contents, hindsight_c
                        "choices": [
                            {
                                "message": {
-                                    "content": json.dumps(
-                                        {
-                                            "facts": [
-                                                {
-                                                    "what": "Alice is a senior software engineer at TechCorp",
-                                                    "when": "present",
-                                                    "where": "TechCorp",
-                                                    "who": "Alice",
-                                                    "why": "Professional background information",
-                                                    "fact_type": "world",
-                                                    "fact_kind": "conversation",
-                                                }
-                                            ]
-                                        }
-                                    )
+                                    "content": json.dumps({
+                                        "facts": [
+                                            {
+                                                "what": "Alice is a senior software engineer at TechCorp",
+                                                "when": "present",
+                                                "where": "TechCorp",
+                                                "who": "Alice",
+                                                "why": "Professional background information",
+                                                "fact_type": "world",
+                                                "fact_kind": "conversation",
+                                            }
+                                        ]
+                                    })
                                }
                            }
                        ],
@@ -135,21 +130,19 @@ async def test_batch_api_normal_flow(mock_llm_config, test_contents, hindsight_c
                        "choices": [
                            {
                                "message": {
-                                    "content": json.dumps(
-                                        {
-                                            "facts": [
-                                                {
-                                                    "what": "Bob joined the team last month as a junior developer",
-                                                    "when": "last month",
-                                                    "where": "team",
-                                                    "who": "Bob",
-                                                    "why": "New team member information",
-                                                    "fact_type": "world",
-                                                    "fact_kind": "conversation",
-                                                }
-                                            ]
-                                        }
-                                    )
+                                    "content": json.dumps({
+                                        "facts": [
+                                            {
+                                                "what": "Bob joined the team last month as a junior developer",
+                                                "when": "last month",
+                                                "where": "team",
+                                                "who": "Bob",
+                                                "why": "New team member information",
+                                                "fact_type": "world",
+                                                "fact_kind": "conversation",
+                                            }
+                                        ]
+                                    })
                                }
                            }
                        ],
@@ -218,7 +211,6 @@ async def test_batch_api_crash_recovery(mock_llm_config, test_contents, hindsigh
        schema = request_context.tenant_id

        from hindsight_api.engine.task_backend import fq_table
-
        table = fq_table("async_operations", schema)

        # Create operation with batch_id already stored
@@ -229,13 +221,11 @@ async def test_batch_api_crash_recovery(mock_llm_config, test_contents, hindsigh
            """,
            operation_id,
            bank_id,
-            json.dumps(
-                {
-                    "batch_id": batch_id,
-                    "batch_provider": "openai",
-                    "chunk_count": 2,
-                }
-            ),
+            json.dumps({
+                "batch_id": batch_id,
+                "batch_provider": "openai",
+                "chunk_count": 2,
+            }),
        )

        # Mock batch API responses for resume scenario
@@ -258,21 +248,19 @@ async def test_batch_api_crash_recovery(mock_llm_config, test_contents, hindsigh
                        "choices": [
                            {
                                "message": {
-                                    "content": json.dumps(
-                                        {
-                                            "facts": [
-                                                {
-                                                    "what": "Alice is a senior software engineer",
-                                                    "when": "present",
-                                                    "where": "TechCorp",
-                                                    "who": "Alice",
-                                                    "why": "Background",
-                                                    "fact_type": "world",
-                                                    "fact_kind": "conversation",
-                                                }
-                                            ]
-                                        }
-                                    )
+                                    "content": json.dumps({
+                                        "facts": [
+                                            {
+                                                "what": "Alice is a senior software engineer",
+                                                "when": "present",
+                                                "where": "TechCorp",
+                                                "who": "Alice",
+                                                "why": "Background",
+                                                "fact_type": "world",
+                                                "fact_kind": "conversation",
+                                            }
+                                        ]
+                                    })
                                }
                            }
                        ],
@@ -287,21 +275,19 @@ async def test_batch_api_crash_recovery(mock_llm_config, test_contents, hindsigh
                        "choices": [
                            {
                                "message": {
-                                    "content": json.dumps(
-                                        {
-                                            "facts": [
-                                                {
-                                                    "what": "Bob is a junior developer",
-                                                    "when": "last month",
-                                                    "where": "team",
-                                                    "who": "Bob",
-                                                    "why": "New member",
-                                                    "fact_type": "world",
-                                                    "fact_kind": "conversation",
-                                                }
-                                            ]
-                                        }
-                                    )
+                                    "content": json.dumps({
+                                        "facts": [
+                                            {
+                                                "what": "Bob is a junior developer",
+                                                "when": "last month",
+                                                "where": "team",
+                                                "who": "Bob",
+                                                "why": "New member",
+                                                "fact_type": "world",
+                                                "fact_kind": "conversation",
+                                            }
+                                        ]
+                                    })
                                }
                            }
                        ],
@@ -345,105 +331,6 @@ async def test_batch_api_crash_recovery(mock_llm_config, test_contents, hindsigh
            pass


-@pytest.mark.asyncio
-async def test_batch_api_records_non_fatal_extraction_errors(
-    mock_llm_config, test_contents, hindsight_config, memory, request_context
-):
-    """Batch API skipped chunks are surfaced in operation result_metadata."""
-    bank_id = f"test_batch_errors_{datetime.now(timezone.utc).timestamp()}"
-    operation_id = str(uuid.uuid4())
-
-    try:
-        await memory.get_bank_profile(bank_id, request_context=request_context)
-        pool = memory._pool
-        schema = request_context.tenant_id
-
-        from hindsight_api.engine.task_backend import fq_table
-
-        table = fq_table("async_operations", schema)
-        await pool.execute(
-            f"""
-            INSERT INTO {table} (operation_id, operation_type, bank_id, status, result_metadata)
-            VALUES ($1, 'retain', $2, 'processing', $3::jsonb)
-            """,
-            operation_id,
-            bank_id,
-            json.dumps({}),
-        )
-
-        batch_id = "batch_partial_errors"
-        mock_llm_config._provider_impl.supports_batch_api = AsyncMock(return_value=True)
-        mock_llm_config._provider_impl.submit_batch = AsyncMock(return_value={"batch_id": batch_id})
-        mock_llm_config._provider_impl.get_batch_status = AsyncMock(
-            return_value={
-                "status": "completed",
-                "request_counts": {"total": 2, "completed": 2, "failed": 0},
-            }
-        )
-        mock_llm_config._provider_impl.retrieve_batch_results = AsyncMock(
-            return_value=[
-                {
-                    "custom_id": "chunk_0",
-                    "response": {
-                        "body": {
-                            "choices": [
-                                {
-                                    "message": {
-                                        "content": json.dumps(
-                                            {
-                                                "facts": [
-                                                    {
-                                                        "what": "Alice is a senior software engineer",
-                                                        "when": "present",
-                                                        "where": "TechCorp",
-                                                        "who": "Alice",
-                                                        "why": "Background",
-                                                        "fact_type": "world",
-                                                        "fact_kind": "conversation",
-                                                    }
-                                                ]
-                                            }
-                                        )
-                                    }
-                                }
-                            ],
-                            "usage": {"prompt_tokens": 100, "completion_tokens": 50, "total_tokens": 150},
-                        }
-                    },
-                }
-            ]
-        )
-
-        facts, chunks, usage = await extract_facts_from_contents_batch_api(
-            contents=test_contents,
-            llm_config=mock_llm_config,
-            agent_name="test_agent",
-            config=hindsight_config,
-            pool=pool,
-            operation_id=operation_id,
-            schema=schema,
-        )
-
-        assert len(facts) == 1
-        assert len(chunks) == 2
-        assert chunks[1].fact_count == 0
-        assert usage.total_tokens == 150
-
-        row = await pool.fetchrow(f"SELECT result_metadata FROM {table} WHERE operation_id = $1", operation_id)
-        metadata = (
-            json.loads(row["result_metadata"]) if isinstance(row["result_metadata"], str) else row["result_metadata"]
-        )
-        assert metadata["batch_id"] == batch_id
-        assert metadata["extraction_errors_count"] == 1
-        assert metadata["extraction_errors_sample"] == ["chunk_1: missing batch result"]
-
-    finally:
-        try:
-            await memory.delete_bank(bank_id, request_context=request_context)
-        except Exception:
-            pass
-
-
@pytest.mark.asyncio
 async def test_batch_api_raises_for_unsupported_provider(mock_llm_config, test_contents, hindsight_config):
    """Batch extraction must surface a hard error (not silently fall back) when
@@ -485,7 +372,6 @@ async def test_worker_batch_recovery(memory, request_context):
        schema = request_context.tenant_id

        from hindsight_api.engine.task_backend import fq_table
-
        table = fq_table("async_operations", schema)

        # Create orphaned batch operation (simulates worker crash during polling)
@@ -503,19 +389,16 @@ async def test_worker_batch_recovery(memory, request_context):
            """,
            operation_id,
            bank_id,
-            json.dumps(
-                {
-                    "batch_id": batch_id,
-                    "batch_provider": "openai",
-                    "chunk_count": 1,
-                }
-            ),
+            json.dumps({
+                "batch_id": batch_id,
+                "batch_provider": "openai",
+                "chunk_count": 1,
+            }),
            json.dumps(task_payload),
        )

        # Create WorkerPoller
        from hindsight_api.extensions.builtin.tenant import DefaultTenantExtension
-
        tenant_extension = DefaultTenantExtension(config={"schema": schema} if schema else {})

        poller = WorkerPoller(
@@ -579,7 +462,13 @@ async def test_batch_api_via_extract_facts_from_contents(
                    "custom_id": "chunk_0",
                    "response": {
                        "body": {
-                            "choices": [{"message": {"content": json.dumps({"facts": []})}}],
+                            "choices": [
+                                {
+                                    "message": {
+                                        "content": json.dumps({"facts": []})
+                                    }
+                                }
+                            ],
                            "usage": {"prompt_tokens": 10, "completion_tokens": 5, "total_tokens": 15},
                        }
                    },
@@ -10,7 +10,6 @@ To run:
 To skip in CI:
    Add @pytest.mark.skip at the test level
 """
-
 import pytest
 import os
 import asyncio
@@ -116,9 +115,7 @@ def integration_config():
    return config


-@pytest.mark.skip(
-    reason="Real API test - takes minutes and costs money. Run manually with: pytest tests/test_batch_api_integration.py::test_real_openai_batch_api -v -s"
-)
+@pytest.mark.skip(reason="Real API test - takes minutes and costs money. Run manually with: pytest tests/test_batch_api_integration.py::test_real_openai_batch_api -v -s")
@pytest.mark.integration  # Mark as integration test
@pytest.mark.slow  # Mark as slow test
@pytest.mark.asyncio
@@ -177,21 +174,17 @@ async def test_real_openai_batch_api(real_llm_config, test_contents_real, integr
        logger.info("\n" + "=" * 80)
        logger.info("✅ BATCH COMPLETED SUCCESSFULLY")
        logger.info("=" * 80)
-        logger.info(f"Total duration: {total_duration:.1f} seconds ({total_duration / 60:.1f} minutes)")
+        logger.info(f"Total duration: {total_duration:.1f} seconds ({total_duration/60:.1f} minutes)")
        logger.info(f"Facts extracted: {len(facts)}")
        logger.info(f"Chunks processed: {len(chunks)}")
-        logger.info(
-            f"Token usage: {usage.input_tokens} input + {usage.output_tokens} output = {usage.total_tokens} total"
-        )
-        logger.info(
-            f"Estimated cost: ${(usage.input_tokens * 0.00015 / 1000 + usage.output_tokens * 0.0006 / 1000):.4f}"
-        )
+        logger.info(f"Token usage: {usage.input_tokens} input + {usage.output_tokens} output = {usage.total_tokens} total")
+        logger.info(f"Estimated cost: ${(usage.input_tokens * 0.00015 / 1000 + usage.output_tokens * 0.0006 / 1000):.4f}")
        logger.info("=" * 80)

        # Log sample facts
        logger.info("\n📋 Sample extracted facts:")
        for i, fact in enumerate(facts[:5]):  # Show first 5 facts
-            logger.info(f"\nFact {i + 1}:")
+            logger.info(f"\nFact {i+1}:")
            logger.info(f"  Type: {fact.fact_type}")
            logger.info(f"  Text: {fact.fact_text[:100]}...")
            logger.info(f"  Entities: {fact.entities}")
@@ -219,13 +212,11 @@ async def test_real_openai_batch_api(real_llm_config, test_contents_real, integr
            f.write(f"Contents: {len(test_contents_real)} items\n")
            f.write(f"Poll Interval: {integration_config.retain_batch_poll_interval_seconds}s\n\n")
            f.write(f"Results:\n")
-            f.write(f"  Total Duration: {total_duration:.1f}s ({total_duration / 60:.1f} min)\n")
+            f.write(f"  Total Duration: {total_duration:.1f}s ({total_duration/60:.1f} min)\n")
            f.write(f"  Facts Extracted: {len(facts)}\n")
            f.write(f"  Chunks Processed: {len(chunks)}\n")
            f.write(f"  Token Usage: {usage.total_tokens} ({usage.input_tokens} in + {usage.output_tokens} out)\n")
-            f.write(
-                f"  Estimated Cost: ${(usage.input_tokens * 0.00015 / 1000 + usage.output_tokens * 0.0006 / 1000):.4f}\n"
-            )
+            f.write(f"  Estimated Cost: ${(usage.input_tokens * 0.00015 / 1000 + usage.output_tokens * 0.0006 / 1000):.4f}\n")

        logger.info(f"\n📄 Timing report written to: {report_path}")

@@ -174,7 +174,9 @@ def test_async_children_packs_small_items_by_budget():
    num_items = max(4, (tokens_per_batch // max(item_tokens, 1)) * 3)
    contents = [{"content": item_text, "document_id": f"doc-{i}"} for i in range(num_items)]
    total = sum(count_tokens(c["content"]) for c in contents)
-    assert total > tokens_per_batch, f"Test setup error: {total} tokens does not exceed budget {tokens_per_batch}"
+    assert total > tokens_per_batch, (
+        f"Test setup error: {total} tokens does not exceed budget {tokens_per_batch}"
+    )

    children = _split_contents_into_async_children(contents, tokens_per_batch)

@@ -104,7 +104,8 @@ class TestCausalRelationsValidation:
            for rel in facts[0].causal_relations:
                # This should never happen due to validation
                assert False, (
-                    f"First fact should not have causal relations, but found: target_index={rel.target_fact_index}"
+                    f"First fact should not have causal relations, "
+                    f"but found: target_index={rel.target_fact_index}"
                )

    @pytest.mark.asyncio
@@ -138,13 +139,11 @@ class TestCausalRelationsValidation:
        for i, fact in enumerate(facts):
            if fact.causal_relations:
                for rel in fact.causal_relations:
-                    all_relations.append(
-                        {
-                            "from_fact": i,
-                            "to_fact": rel.target_fact_index,
-                            "type": rel.relation_type,
-                        }
-                    )
+                    all_relations.append({
+                        "from_fact": i,
+                        "to_fact": rel.target_fact_index,
+                        "type": rel.relation_type,
+                    })

        # If causal relations were extracted, verify they form a valid chain
        if all_relations:
@@ -227,5 +226,6 @@ class TestCausalRelationsValidation:
            if fact.causal_relations:
                for rel in fact.causal_relations:
                    assert rel.relation_type in valid_types, (
-                        f"Invalid relation_type '{rel.relation_type}'. Must be one of: {valid_types}"
+                        f"Invalid relation_type '{rel.relation_type}'. "
+                        f"Must be one of: {valid_types}"
                    )
@@ -40,11 +40,7 @@ After searching for weeks, I finally found a cheaper apartment in Brooklyn.
        llm_config = LLMConfig.from_env()

        facts, _, _ = await extract_facts_from_text(
-            text=text,
-            event_date=datetime(2024, 3, 15),
-            context=context,
-            llm_config=llm_config,
-            agent_name="TestUser",
+            text=text, event_date=datetime(2024, 3, 15), context=context, llm_config=llm_config, agent_name="TestUser",
            config=_get_raw_config(),
        )

@@ -113,11 +109,7 @@ The renovation took three months and cost $15,000.
        llm_config = LLMConfig.from_env()

        facts, _, _ = await extract_facts_from_text(
-            text=text,
-            event_date=datetime(2024, 6, 1),
-            context=context,
-            llm_config=llm_config,
-            agent_name="TestUser",
+            text=text, event_date=datetime(2024, 6, 1), context=context, llm_config=llm_config, agent_name="TestUser",
            config=_get_raw_config(),
        )

@@ -148,11 +140,7 @@ Machine learning fascinated me so much that I changed my career to data science.
        llm_config = LLMConfig.from_env()

        facts, _, _ = await extract_facts_from_text(
-            text=text,
-            event_date=datetime(2024, 1, 1),
-            context=context,
-            llm_config=llm_config,
-            agent_name="TestUser",
+            text=text, event_date=datetime(2024, 1, 1), context=context, llm_config=llm_config, agent_name="TestUser",
            config=_get_raw_config(),
        )

@@ -180,11 +168,7 @@ The new role enabled me to lead a team of engineers.
        llm_config = LLMConfig.from_env()

        facts, _, _ = await extract_facts_from_text(
-            text=text,
-            event_date=datetime(2024, 2, 15),
-            context=context,
-            llm_config=llm_config,
-            agent_name="TestUser",
+            text=text, event_date=datetime(2024, 2, 15), context=context, llm_config=llm_config, agent_name="TestUser",
            config=_get_raw_config(),
        )

@@ -196,3 +180,4 @@ The new role enabled me to lead a team of engineers.
                        f"Invalid target_fact_index {rel.target_fact_index} in fact {i}. "
                        f"Must reference previous facts only (valid range: 0 to {i - 1})"
                    )
+
@@ -130,7 +130,8 @@ async def test_store_chunks_batch_second_call_with_identical_payload(memory):
            await _seed_bank_and_document(conn, bank_id, document_id)

            chunks = [
-                ChunkMetadata(chunk_text=f"chunk-{i}", fact_count=1, content_index=0, chunk_index=i) for i in range(5)
+                ChunkMetadata(chunk_text=f"chunk-{i}", fact_count=1, content_index=0, chunk_index=i)
+                for i in range(5)
            ]

            await chunk_storage.store_chunks_batch(conn, bank_id, document_id, chunks, ops=ops)
@@ -1,7 +1,6 @@
 """
 Test chunking functionality for large documents.
 """
-
 import pytest
 from hindsight_api.engine.retain.fact_extraction import chunk_text

@@ -54,3 +53,4 @@ def test_chunk_text_64k():
    # Verify we didn't lose content
    combined_length = sum(len(chunk) for chunk in chunks)
    assert combined_length >= len(text) * 0.95, "Lost too much content during chunking"
+
@@ -344,7 +344,4 @@ class TestFactoryFunction:
                assert isinstance(encoder, CohereCrossEncoder)
                assert encoder.api_key == "test_key"
                assert encoder.model == "cohere-rerank-v3-english"
-                assert (
-                    encoder.base_url
-                    == "https://my-endpoint.inference.ai.azure.com/models/cohere-rerank-v3-english/invoke"
-                )
+                assert encoder.base_url == "https://my-endpoint.inference.ai.azure.com/models/cohere-rerank-v3-english/invoke"
@@ -18,12 +18,10 @@ def setup_test_env():
    # Save original environment values
    env_vars_to_save = [
        "HINDSIGHT_API_RETAIN_MAX_COMPLETION_TOKENS",
-        "HINDSIGHT_API_CONSOLIDATION_MAX_COMPLETION_TOKENS",
        "HINDSIGHT_API_RETAIN_CHUNK_SIZE",
        "HINDSIGHT_API_LLM_PROVIDER",
        "HINDSIGHT_API_LLM_MODEL",
        "HINDSIGHT_API_LLM_REASONING_EFFORT",
-        "HINDSIGHT_API_SEMANTIC_MIN_SIMILARITY",
        "HINDSIGHT_API_DATABASE_URL",
        "HINDSIGHT_API_MIGRATION_DATABASE_URL",
    ]
@@ -105,48 +103,6 @@ def test_valid_retain_config_succeeds():
    assert config.retain_chunk_size == 3000


-def test_semantic_min_similarity_reads_from_env():
-    """Semantic retrieval min similarity can be configured at the server level."""
-    from hindsight_api.config import HindsightConfig
-
-    os.environ["HINDSIGHT_API_SEMANTIC_MIN_SIMILARITY"] = "0.58"
-
-    config = HindsightConfig.from_env()
-    assert config.semantic_min_similarity == 0.58
-
-
-def test_semantic_min_similarity_must_be_between_zero_and_one():
-    """Invalid semantic min similarity fails fast during configuration loading."""
-    from hindsight_api.config import HindsightConfig
-
-    os.environ["HINDSIGHT_API_SEMANTIC_MIN_SIMILARITY"] = "1.5"
-
-    with pytest.raises(ValueError, match="semantic_min_similarity"):
-        HindsightConfig.from_env()
-
-
-def test_consolidation_max_completion_tokens_defaults_to_unset():
-    """By default consolidation sends no explicit output budget (backwards compatible)."""
-    from hindsight_api.config import HindsightConfig
-
-    os.environ["HINDSIGHT_API_LLM_PROVIDER"] = "mock"
-    os.environ.pop("HINDSIGHT_API_CONSOLIDATION_MAX_COMPLETION_TOKENS", None)
-
-    config = HindsightConfig.from_env()
-    assert config.consolidation_max_completion_tokens is None
-
-
-def test_consolidation_max_completion_tokens_env_override():
-    """HINDSIGHT_API_CONSOLIDATION_MAX_COMPLETION_TOKENS controls consolidation LLM output budget."""
-    from hindsight_api.config import HindsightConfig
-
-    os.environ["HINDSIGHT_API_LLM_PROVIDER"] = "mock"
-    os.environ["HINDSIGHT_API_CONSOLIDATION_MAX_COMPLETION_TOKENS"] = "8192"
-
-    config = HindsightConfig.from_env()
-    assert config.consolidation_max_completion_tokens == 8192
-
-
 def test_log_config_masks_database_urls(caplog):
    """Config startup logs must not expose database credentials."""
    from hindsight_api.config import HindsightConfig
@@ -420,48 +376,3 @@ def test_llm_reasoning_effort_loaded_from_env(monkeypatch):

    config = HindsightConfig.from_env()
    assert config.llm_reasoning_effort == "xhigh"
-
-
-# ---------------------------------------------------------------------------
-# Recall candidate gating (BM25 score floor + per-source cap) — issue #1707
-# ---------------------------------------------------------------------------
-
-
-def test_bm25_min_score_defaults_to_zero(monkeypatch):
-    from hindsight_api.config import HindsightConfig
-
-    monkeypatch.delenv("HINDSIGHT_API_BM25_MIN_SCORE", raising=False)
-    monkeypatch.setenv("HINDSIGHT_API_LLM_PROVIDER", "mock")
-
-    config = HindsightConfig.from_env()
-    assert config.bm25_min_score == 0.0
-
-
-def test_bm25_min_score_loaded_from_env(monkeypatch):
-    from hindsight_api.config import HindsightConfig
-
-    monkeypatch.setenv("HINDSIGHT_API_BM25_MIN_SCORE", "1.5")
-    monkeypatch.setenv("HINDSIGHT_API_LLM_PROVIDER", "mock")
-
-    config = HindsightConfig.from_env()
-    assert config.bm25_min_score == 1.5
-
-
-def test_recall_max_candidates_per_source_defaults_to_disabled(monkeypatch):
-    from hindsight_api.config import HindsightConfig
-
-    monkeypatch.delenv("HINDSIGHT_API_RECALL_MAX_CANDIDATES_PER_SOURCE", raising=False)
-    monkeypatch.setenv("HINDSIGHT_API_LLM_PROVIDER", "mock")
-
-    config = HindsightConfig.from_env()
-    assert config.recall_max_candidates_per_source == 0
-
-
-def test_recall_max_candidates_per_source_loaded_from_env(monkeypatch):
-    from hindsight_api.config import HindsightConfig
-
-    monkeypatch.setenv("HINDSIGHT_API_RECALL_MAX_CANDIDATES_PER_SOURCE", "150")
-    monkeypatch.setenv("HINDSIGHT_API_LLM_PROVIDER", "mock")
-
-    config = HindsightConfig.from_env()
-    assert config.recall_max_candidates_per_source == 150
@@ -464,7 +464,7 @@ class TestConsolidationIntegration:
        async with memory._pool.acquire() as conn:
            observations = await conn.fetch(
                """
-                SELECT id, text, source_memory_ids
+                SELECT id, text, source_memory_ids, history
                FROM memory_units
                WHERE bank_id = $1 AND fact_type = 'observation'
                """,
@@ -3040,13 +3040,10 @@ def _make_mock_llm_one_obs_per_fact():
    def callback(messages, scope):
        if scope != "consolidation":
            return _ConsolidationBatchResponse()
-        # Parse all fact UUIDs from the prompt — one create per fact. Read only
-        # the user message(s): consolidation sends the facts there, while the
-        # stable (cacheable) system message carries example UUIDs in its OUTPUT
-        # FORMAT samples that must not be mistaken for real facts.
+        # Parse all fact UUIDs from the prompt — one create per fact
        import re

-        prompt = "\n".join(m.get("content", "") for m in messages if m.get("role") == "user")
+        prompt = messages[0]["content"] if messages else ""
        fact_ids = re.findall(r"\[([0-9a-f-]{36})\]", prompt)
        creates = [_CreateAction(text=f"Observation about fact {fid[:8]}", source_fact_ids=[fid]) for fid in fact_ids]
        return _ConsolidationBatchResponse(creates=creates)
@@ -3148,9 +3145,7 @@ async def test_max_observations_per_scope_allows_updates_at_capacity(memory: Mem
            call_count += 1
            import re

-            # Facts live in the user message; the system message (stable, cached)
-            # carries example UUIDs in its OUTPUT samples — read user only.
-            prompt = "\n".join(m.get("content", "") for m in messages if m.get("role") == "user")
+            prompt = messages[0]["content"] if messages else ""
            fact_ids = re.findall(r"\[([0-9a-f-]{36})\]", prompt)
            if call_count == 1 and fact_ids:
                # First call: create an observation
@@ -3517,52 +3512,3 @@ async def test_enable_auto_consolidation_flag(memory: MemoryEngine, request_cont
    finally:
        memory._config_resolver._global_config = original_global_config
        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-def test_consolidation_prompt_split_is_cacheable_and_complete():
-    """The split consolidation prompt: bank-agnostic system prefix + per-batch user.
-
-    The system prefix must be byte-identical across batches AND across banks (the
-    property that lets a single Gemini context cache serve every bank), carry only
-    stable instructions, and the per-batch/per-bank data (mission, facts,
-    observations, capacity note) must live in the user message — never in the
-    cached prefix.
-    """
-    from hindsight_api.engine.consolidation.prompts import (
-        build_consolidation_input,
-        build_consolidation_system_prompt,
-    )
-
-    sys_prompt = build_consolidation_system_prompt()
-    # Byte-stable across calls and independent of any mission → one cache for all banks.
-    assert sys_prompt == build_consolidation_system_prompt()
-    # Instructions only: no per-batch placeholders leaked into the prefix.
-    assert "{facts_text}" not in sys_prompt
-    assert "{observations_text}" not in sys_prompt
-    # JSON examples are unescaped (single braces), i.e. .format() ran.
-    assert '{"creates"' in sys_prompt
-    assert "{{" not in sys_prompt
-    # The stable observation-format boilerplate lives in the cached prefix.
-    assert "proof_count" in sys_prompt
-
-    # Two banks with DIFFERENT missions share the identical cached prefix; the
-    # mission rides in the per-batch user message instead.
-    user_a = build_consolidation_input(
-        facts_text="[id-a] Fact A.", observations_text="[]", observations_mission="Track widgets."
-    )
-    user_b = build_consolidation_input(
-        facts_text="[id-b] Fact B.", observations_text="[]", observations_mission="Track gadgets."
-    )
-    assert "Track widgets." in user_a
-    assert "Track widgets." not in sys_prompt  # mission NOT in the cached prefix
-    assert "Fact A." in user_a
-    assert user_a != user_b
-    # The format boilerplate is NOT re-sent per batch (it's in the cached prefix).
-    assert "proof_count" not in user_a
-
-    # The capacity note is per-batch too — kept out of the cached prefix.
-    capped = build_consolidation_input(
-        facts_text="[id] F.", observations_text="[]", observation_capacity_note="OBSERVATION LIMIT REACHED"
-    )
-    assert "OBSERVATION LIMIT REACHED" in capped
-    assert "OBSERVATION LIMIT REACHED" not in sys_prompt
@@ -1,259 +0,0 @@
-"""Deterministic unit tests for the consolidation duplicate-create guard.
-
-These exercise the dedup decision directly (no LLM, no DB), so they reliably
-guard the fix in CI — unlike the real-LLM integration test, which only triggers
-the path stochastically.
-"""
-
-import types
-import uuid
-from dataclasses import dataclass
-from unittest.mock import AsyncMock, patch
-
-from hindsight_api.engine.consolidation.consolidator import (
-    _dedup_active,
-    _dedup_reconcile_create,
-    _dedup_reconcile_update,
-    _DedupDecision,
-    _duplicate_create_target,
-    _norm_obs_text,
-)
-from hindsight_api.engine.search.types import RetrievalResult
-
-
-@dataclass
-class _FakeObs:
-    id: str
-    text: str
-
-
-def _shown(*observations: _FakeObs) -> dict[str, _FakeObs]:
-    return {_norm_obs_text(o.text): o for o in observations}
-
-
-def test_norm_obs_text_collapses_whitespace_preserves_case() -> None:
-    # Whitespace (incl. newlines) collapses; case is preserved.
-    assert _norm_obs_text("  The  User  likes BASIL.\n") == "The User likes BASIL."
-    assert _norm_obs_text(None) == ""
-
-
-def test_create_matching_shown_observation_is_duplicate() -> None:
-    shown = _shown(_FakeObs(id="11111111-aaaa", text="User waters the herbs early in the morning."))
-    # Same text with only-whitespace differences still matches.
-    target = _duplicate_create_target("User waters the   herbs early in the morning.", shown, set())
-    assert target is not None
-    assert target.startswith("shown observation 11111111")
-
-
-def test_create_differing_only_in_case_is_not_duplicate() -> None:
-    # Case-folding would lose information (e.g. acronyms), so a case-only difference
-    # is treated as novel rather than silently dropped.
-    shown = _shown(_FakeObs(id="22222222-bbbb", text="The user prefers TLS."))
-    assert _duplicate_create_target("The user prefers tls.", shown, set()) is None
-
-
-def test_create_matching_inresponse_update_is_duplicate() -> None:
-    update_texts = {_norm_obs_text("Mint is kept in its own separate bed.")}
-    target = _duplicate_create_target("Mint is kept in its own separate bed.", {}, update_texts)
-    assert target == "an UPDATE in this response"
-
-
-def test_novel_create_is_not_duplicate() -> None:
-    shown = _shown(_FakeObs(id="22222222-bbbb", text="User waters the herbs early in the morning."))
-    assert _duplicate_create_target("Rosemary is drought-tolerant.", shown, set()) is None
-    assert _duplicate_create_target("", {}, set()) is None
-
-
-# ── semantic dedup (_dedup_reconcile_create) ──────────────────────────────────
-#
-# Mocks the embedder, the obs-anchored ANN probe, and the LLM so the decision logic is
-# tested without a DB or a real model.
-
-_TWIN_ID = "33333333-3333-4333-8333-333333333333"
-
-
-def _obs(text: str, sim: float, oid: str = _TWIN_ID) -> RetrievalResult:
-    return RetrievalResult(id=oid, text=text, fact_type="observation", similarity=sim)
-
-
-def _ctx(threshold: float = 0.97):
-    """Return (kwargs, conn_mock, llm_mock) for a _dedup_reconcile_create call."""
-    conn = AsyncMock()
-    llm = types.SimpleNamespace(call=AsyncMock())
-    kwargs = dict(
-        conn=conn,
-        memory_engine=types.SimpleNamespace(embeddings=object()),
-        bank_id="bank1",
-        config=types.SimpleNamespace(consolidation_dedup_threshold=threshold),
-        dedup_llm_config=llm,
-        create_text="YouTube content in Uzbek is very rich.",
-        create_source_ids=[uuid.uuid4()],
-        tags=["t1"],
-    )
-    return kwargs, conn, llm
-
-
-def _patch_probe(results):
-    return patch(
-        "hindsight_api.engine.search.retrieval.retrieve_semantic_bm25_combined",
-        AsyncMock(return_value={"observation": (results, [])}),
-    )
-
-
-def _patch_embed():
-    return patch(
-        "hindsight_api.engine.retain.embedding_utils.generate_embeddings_batch",
-        AsyncMock(return_value=[[0.1, 0.2, 0.3]]),
-    )
-
-
-async def test_dedup_no_twin_above_threshold_returns_none() -> None:
-    kwargs, conn, llm = _ctx(threshold=0.97)
-    with _patch_embed(), _patch_probe([_obs("something loosely related", 0.81)]):
-        result = await _dedup_reconcile_create(**kwargs)
-    assert result is None
-    llm.call.assert_not_called()  # below threshold → no LLM call
-    conn.execute.assert_not_called()  # no merge
-
-
-async def test_dedup_llm_keep_does_not_merge() -> None:
-    kwargs, conn, llm = _ctx()
-    llm.call.return_value = _DedupDecision(action="keep", reason="different language")
-    with _patch_embed(), _patch_probe([_obs("Uzbek content on YouTube is described as very rich.", 0.98)]):
-        result = await _dedup_reconcile_create(**kwargs)
-    assert result is None
-    llm.call.assert_awaited_once()
-    conn.execute.assert_not_called()  # kept distinct → no merge
-
-
-async def test_dedup_llm_merge_folds_into_twin() -> None:
-    kwargs, conn, llm = _ctx()
-    kwargs["create_source_ids"] = [uuid.uuid4(), uuid.uuid4()]
-    llm.call.return_value = _DedupDecision(action="merge", text="Uzbek content on YouTube is very rich.")
-    with _patch_embed(), _patch_probe([_obs("Uzbek content on YouTube is described as very rich.", 0.99)]):
-        result = await _dedup_reconcile_create(**kwargs)
-    assert result == _TWIN_ID  # merged into the twin; caller skips the CREATE
-    conn.execute.assert_awaited_once()
-    args = conn.execute.await_args.args
-    assert args[1] == "Uzbek content on YouTube is very rich."  # merged text persisted
-    assert args[2] == kwargs["create_source_ids"]  # new source facts folded in
-    assert args[3] == uuid.UUID(_TWIN_ID)  # onto the twin row
-
-
-async def test_dedup_picks_highest_above_threshold_skips_below() -> None:
-    # Only the >=threshold candidate is considered; a 0.95 result is ignored at threshold 0.97.
-    kwargs, conn, llm = _ctx(threshold=0.97)
-    llm.call.return_value = _DedupDecision(action="keep")
-    with _patch_embed(), _patch_probe([_obs("near but distinct", 0.95), _obs("the real twin", 0.98)]):
-        await _dedup_reconcile_create(**kwargs)
-    # the twin passed to the LLM is the >=0.97 one, not the 0.95
-    sent = llm.call.await_args.kwargs["messages"][0]["content"]
-    assert "the real twin" in sent
-    assert "near but distinct" not in sent
-
-
-# ── UPDATE-path dedup (_dedup_reconcile_update) ───────────────────────────────
-#
-# An UPDATE rewrites+re-embeds an observation, which can drift it into a near-twin of a
-# DIFFERENT existing observation. These cover the fold-and-delete reconciliation (unlike
-# CREATE, both rows already exist), the self-exclusion, and the keep/no-twin no-ops.
-
-_UPDATED_ID = "44444444-4444-4444-8444-444444444444"
-
-
-def _update_ctx(threshold: float = 0.97):
-    """Return (kwargs, conn_mock, llm_mock) for a _dedup_reconcile_update call."""
-    conn = AsyncMock()
-    llm = types.SimpleNamespace(call=AsyncMock())
-    kwargs = dict(
-        conn=conn,
-        memory_engine=types.SimpleNamespace(embeddings=object()),
-        bank_id="bank1",
-        config=types.SimpleNamespace(consolidation_dedup_threshold=threshold),
-        dedup_llm_config=llm,
-        updated_id=_UPDATED_ID,
-        updated_text="Uzbek content on YouTube is very rich and growing.",
-        updated_emb_str="[0.1, 0.2, 0.3]",  # already embedded by _execute_update_action
-        tags=["t1"],
-    )
-    return kwargs, conn, llm
-
-
-async def test_dedup_update_merge_folds_into_twin_and_deletes_updated() -> None:
-    kwargs, conn, llm = _update_ctx()
-    llm.call.return_value = _DedupDecision(action="merge", text="Uzbek YouTube content is very rich and growing.")
-    with _patch_probe([_obs("Uzbek content on YouTube is described as very rich.", 0.98)]):
-        await _dedup_reconcile_update(**kwargs)
-    llm.call.assert_awaited_once()
-    # Two writes: fold-into-twin UPDATE, then DELETE of the updated row.
-    assert conn.execute.await_count == 2
-    fold_args = conn.execute.await_args_list[0].args
-    assert fold_args[1] == "Uzbek YouTube content is very rich and growing."  # merged text on the twin
-    assert fold_args[2] == uuid.UUID(_TWIN_ID)  # survivor = the twin
-    assert fold_args[3] == uuid.UUID(_UPDATED_ID)  # folded-from = the updated row
-    delete_args = conn.execute.await_args_list[1].args
-    assert delete_args[1] == uuid.UUID(_UPDATED_ID)  # the updated row is deleted
-
-
-async def test_dedup_update_keep_does_not_merge() -> None:
-    kwargs, conn, llm = _update_ctx()
-    llm.call.return_value = _DedupDecision(action="keep", reason="different growth claim")
-    with _patch_probe([_obs("Uzbek content on YouTube is described as very rich.", 0.98)]):
-        await _dedup_reconcile_update(**kwargs)
-    llm.call.assert_awaited_once()
-    conn.execute.assert_not_called()  # kept distinct → neither fold nor delete
-
-
-async def test_dedup_update_excludes_self() -> None:
-    # The probe surfaces the updated observation itself at 1.0; it must be excluded so we don't
-    # "merge" a row into itself. With no other candidate, there is no twin → no LLM, no writes.
-    kwargs, conn, llm = _update_ctx()
-    with _patch_probe([_obs("its own current text", 1.0, oid=_UPDATED_ID)]):
-        await _dedup_reconcile_update(**kwargs)
-    llm.call.assert_not_called()
-    conn.execute.assert_not_called()
-
-
-async def test_dedup_update_no_twin_above_threshold() -> None:
-    kwargs, conn, llm = _update_ctx(threshold=0.97)
-    with _patch_probe([_obs("loosely related", 0.8)]):
-        await _dedup_reconcile_update(**kwargs)
-    llm.call.assert_not_called()
-    conn.execute.assert_not_called()
-
-
-# ── dedup activation gate (_dedup_active) ─────────────────────────────────────
-#
-# Enabled by default (threshold < 1.0), but skipped on Oracle because the merge path is
-# Postgres-only — so the feature can ship on-by-default without breaking Oracle.
-
-
-def _gate_cfg(threshold: float):
-    return types.SimpleNamespace(consolidation_dedup_threshold=threshold)
-
-
-def _patch_backend(name: str):
-    return patch(
-        "hindsight_api.engine.consolidation.consolidator.get_config",
-        return_value=types.SimpleNamespace(database_backend=name),
-    )
-
-
-def test_dedup_active_enabled_on_postgres() -> None:
-    with _patch_backend("postgresql"):
-        assert _dedup_active(_gate_cfg(0.97)) is True
-
-
-def test_dedup_active_disabled_when_threshold_is_one() -> None:
-    with _patch_backend("postgresql"):
-        assert _dedup_active(_gate_cfg(1.0)) is False
-
-
-def test_dedup_active_skipped_on_oracle() -> None:
-    # PG-only merge path → dedup is skipped on Oracle even with a sub-1.0 threshold.
-    with _patch_backend("oracle"):
-        assert _dedup_active(_gate_cfg(0.97)) is False
-
-
-def test_dedup_active_none_config() -> None:
-    assert _dedup_active(None) is False
@@ -1,41 +0,0 @@
-import uuid
-
-import pytest
-
-from hindsight_api.engine.consolidation import consolidator
-
-
-class _ZeroLengthEmbeddings:
-    dimension = 384
-
-    def encode_documents(self, texts):
-        assert texts == ["Consolidated observation text."]
-        return [[]]
-
-
-class _FakeMemoryEngine:
-    embeddings = _ZeroLengthEmbeddings()
-
-
-class _FailingConn:
-    async def fetchrow(self, *args, **kwargs):
-        raise AssertionError("zero-length embedding should be rejected before database insert")
-
-
-@pytest.mark.asyncio
-async def test_create_observation_rejects_zero_length_embedding_before_insert(monkeypatch):
-    source_id = uuid.uuid4()
-
-    async def fake_filter_live_source_memories(conn, bank_id, source_memory_ids):
-        return source_memory_ids
-
-    monkeypatch.setattr(consolidator, "_filter_live_source_memories", fake_filter_live_source_memories)
-
-    with pytest.raises(RuntimeError, match="embedding 0 has dimension 0; expected 384"):
-        await consolidator._create_observation_directly(
-            conn=_FailingConn(),
-            memory_engine=_FakeMemoryEngine(),
-            bank_id="test-bank",
-            source_memory_ids=[source_id],
-            observation_text="Consolidated observation text.",
-        )
@@ -371,7 +371,9 @@ class TestRecoverConsolidation:
                    mem_id,
                )

-        result = await memory_no_llm_verify.retry_failed_consolidation(bank_id, request_context=request_context)
+        result = await memory_no_llm_verify.retry_failed_consolidation(
+            bank_id, request_context=request_context
+        )

        assert result["retried_count"] == 2

@@ -392,7 +394,9 @@ class TestRecoverConsolidation:
        bank_id = f"test-recover-zero-{uuid.uuid4().hex[:8]}"
        await memory_no_llm_verify.get_bank_profile(bank_id=bank_id, request_context=request_context)

-        result = await memory_no_llm_verify.retry_failed_consolidation(bank_id, request_context=request_context)
+        result = await memory_no_llm_verify.retry_failed_consolidation(
+            bank_id, request_context=request_context
+        )

        assert result["retried_count"] == 0

@@ -406,10 +410,14 @@ class TestRecoverConsolidation:

        async with memory_no_llm_verify._pool.acquire() as conn:
            (mem_id,) = await _insert_memories(conn, bank_id, ["Grace is an expert rock climber."])
-            await conn.execute("UPDATE memory_units SET consolidation_failed_at = NOW() WHERE id = $1", mem_id)
+            await conn.execute(
+                "UPDATE memory_units SET consolidation_failed_at = NOW() WHERE id = $1", mem_id
+            )

        # Recover
-        recover_result = await memory_no_llm_verify.retry_failed_consolidation(bank_id, request_context=request_context)
+        recover_result = await memory_no_llm_verify.retry_failed_consolidation(
+            bank_id, request_context=request_context
+        )
        assert recover_result["retried_count"] == 1

        # Now consolidate with a healthy LLM
@@ -452,7 +460,9 @@ class TestRecoverConsolidation:
                ["Henry is a professional chef.", "Henry trained at Le Cordon Bleu."],
            )
            for mem_id in ids:
-                await conn.execute("UPDATE memory_units SET consolidation_failed_at = NOW() WHERE id = $1", mem_id)
+                await conn.execute(
+                    "UPDATE memory_units SET consolidation_failed_at = NOW() WHERE id = $1", mem_id
+                )

        app = create_app(memory_no_llm_verify, initialize_memory=False)
        transport = httpx.ASGITransport(app=app)
@@ -80,7 +80,9 @@ async def _pending_consolidation_ops(memory, bank_id: str) -> list[str]:


@pytest.mark.asyncio
-async def test_round_limited_consolidation_leaves_followup_pending_op(memory: MemoryEngine, request_context):
+async def test_round_limited_consolidation_leaves_followup_pending_op(
+    memory: MemoryEngine, request_context
+):
    """A round-limited consolidation must leave a new ``pending`` consolidation
    op in ``async_operations`` for the same bank so the worker poller can
    drain the backlog without external intervention."""
@@ -145,7 +147,9 @@ async def test_round_limited_consolidation_leaves_followup_pending_op(memory: Me
            op_id,
        )
    assert row is not None
-    assert row["status"] == "completed", f"first consolidation op should be marked completed, got {row['status']}"
+    assert row["status"] == "completed", (
+        f"first consolidation op should be marked completed, got {row['status']}"
+    )

    # 4. Backlog must remain (round limit kept one round under the total)
    unconsolidated_after = await _count_unconsolidated(memory, bank_id)
@@ -165,6 +169,8 @@ async def test_round_limited_consolidation_leaves_followup_pending_op(memory: Me
        f"backlog. Found {len(pending_ops)} pending ops; backlog still has "
        f"{unconsolidated_after} unconsolidated memory_units."
    )
-    assert pending_ops[0] != str(op_id), "The pending op must be a NEW row, not the original op we just executed."
+    assert pending_ops[0] != str(op_id), (
+        "The pending op must be a NEW row, not the original op we just executed."
+    )

    await memory.delete_bank(bank_id, request_context=request_context)
@@ -1,9 +1,9 @@
 """Tests for consolidation retry budget configurability (issue #1042)."""

-from unittest.mock import AsyncMock, MagicMock
-
 import pytest

+from unittest.mock import AsyncMock, MagicMock
+
 from hindsight_api.engine.consolidation.consolidator import _consolidate_batch_with_llm


@@ -24,7 +24,6 @@ def mock_config():
    config.observations_mission = None
    config.consolidation_max_attempts = 3
    config.consolidation_llm_max_retries = None
-    config.consolidation_max_completion_tokens = None
    return config


@@ -69,32 +68,6 @@ class TestConsolidationRetryBudget:
        )
        assert mock_llm_config.call.call_args.kwargs.get("max_retries") == 3

-    @pytest.mark.asyncio
-    async def test_max_completion_tokens_threaded_to_call(self, mock_llm_config, mock_config):
-        """consolidation_max_completion_tokens is passed to llm_config.call()."""
-        mock_config.consolidation_max_completion_tokens = 8192
-        await _consolidate_batch_with_llm(
-            llm_config=mock_llm_config,
-            memories=[{"id": "m1", "text": "test"}],
-            union_observations=[],
-            union_source_facts={},
-            config=mock_config,
-        )
-        assert mock_llm_config.call.call_args.kwargs.get("max_completion_tokens") == 8192
-
-    @pytest.mark.asyncio
-    async def test_max_completion_tokens_not_passed_when_none(self, mock_llm_config, mock_config):
-        """When consolidation_max_completion_tokens is None, max_completion_tokens is omitted (no regression)."""
-        mock_config.consolidation_max_completion_tokens = None
-        await _consolidate_batch_with_llm(
-            llm_config=mock_llm_config,
-            memories=[{"id": "m1", "text": "test"}],
-            union_observations=[],
-            union_source_facts={},
-            config=mock_config,
-        )
-        assert "max_completion_tokens" not in mock_llm_config.call.call_args.kwargs
-
    @pytest.mark.asyncio
    async def test_max_retries_not_passed_when_none(self, mock_llm_config, mock_config):
        """When consolidation_llm_max_retries is None, max_retries is not passed."""
--- a/Show More
+++ b/Show More