fix

.claude-plugin/marketplace.json

@@ -1,15 +0,0 @@
-{
-  "$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
-  "name": "hindsight",
-  "description": "Official Hindsight integrations for Claude Code",
-  "owner": {
-    "name": "vectorize-io"
-  },
-  "plugins": [
-    {
-      "name": "hindsight-memory",
-      "description": "Automatic long-term memory for Claude Code via Hindsight",
-      "source": "./hindsight-integrations/claude-code"
-    }
-  ]
-}

.claude/skills/code-review/SKILL.md

@@ -1,205 +0,0 @@
----
-name: code-review
-description: Review changed code against project standards. Checks for missing tests, dead code, type safety, lint issues, and coding conventions. Run after completing any implementation work.
-user_invocable: true
----
-
-# Code Review
-
-Review all changed code against the project's quality standards and coding conventions.
-
-## Code Standards
-
-Read and internalize these standards before writing code. The review steps below verify compliance.
-
-### Python Style
-- Python 3.11+, type hints required
-- Async throughout (asyncpg, async FastAPI)
-- Pydantic models for request/response
-- Ruff for linting (line-length 120)
-- No Python files at project root - maintain clean directory structure
-- **Never use multi-item tuple return values** — not even for internal/private functions. Always use a dataclass or Pydantic model. No exceptions, no "it's just two values" shortcuts. If a function returns more than one value, define a named type for it.
-
-### Type Safety with Pydantic Models
-**NEVER use raw `dict` types for structured data** — this applies to all code, including internal helpers and private functions. If the dict has known keys, it must be a dataclass or Pydantic model:
-- Use Pydantic `BaseModel` for all data structures passed between functions
-- Use `@dataclass` for lightweight internal data containers when Pydantic validation isn't needed
-- Add `@field_validator` for type coercion (e.g., ensuring datetimes are timezone-aware)
-- Avoid `dict.get()` patterns - use typed model attributes instead
-- Parse external data (JSON, API responses) into Pydantic models at the boundary
-- This catches type errors at parse time, not deep in business logic
-- The only acceptable `dict` usage is for truly dynamic/unknown keys (e.g., arbitrary metadata, JSON blobs with no fixed schema)
-
-```python
-# BAD - error-prone dict access
-def process(data: dict) -> str:
-    return data.get("name", "")  # No validation, silent failures
-
-# GOOD - typed and validated
-class UserData(BaseModel):
-    name: str
-    created_at: datetime
-
-    @field_validator("created_at", mode="before")
-    @classmethod
-    def ensure_tz_aware(cls, v):
-        if isinstance(v, str):
-            v = datetime.fromisoformat(v.replace("Z", "+00:00"))
-        if v.tzinfo is None:
-            return v.replace(tzinfo=timezone.utc)
-        return v
-
-def process(data: UserData) -> str:
-    return data.name  # Type-safe, validated at construction
-```
-
-### TypeScript Style
-- Next.js App Router for control plane
-- Tailwind CSS with shadcn/ui components
-
-### Code Comments
-- **Always comment non-trivial technical decisions** with the reasoning behind the choice. If someone would ask "why is it done this way?", there should be a comment.
-- **Keep comments up to date with history** — when changing an approach, update the comment to explain what was tried before and why it was changed. Comments serve as a tracker of previous implementations that likely had problems.
-- Don't comment obvious code — only where the "why" isn't self-evident from the code itself.
-
-```python
-# BAD - no context for future readers
-results = await asyncio.gather(*tasks, return_exceptions=True)
-
-# GOOD - explains the non-obvious choice
-# Use return_exceptions=True to avoid cancelling sibling tasks on failure.
-# Previously we used TaskGroup but it cancelled all tasks when one failed,
-# causing partial writes that left orphaned entity links (see #412).
-results = await asyncio.gather(*tasks, return_exceptions=True)
-```
-
-### Branch Hygiene
-- **Always start new feature branches from `origin/main`** — rebase to ensure a clean base.
-- **Only include commits relevant to the PR/branch/feature** — no unrelated changes. If the branch contains commits that don't belong, they must be removed before merging.
-
-### General Principles
-- Don't add features, refactor code, or make "improvements" beyond what was asked
-- Don't add unnecessary error handling for impossible scenarios
-- Don't create helpers or abstractions for one-time operations
-- No backwards-compatibility hacks (unused vars, re-exports, "removed" comments)
-- Three similar lines of code is better than a premature abstraction
-
-## Review Steps
-
-### 1. Check branch hygiene
-
-- Run `git log --oneline main..HEAD` to list all commits on the branch.
-- Verify every commit is relevant to the feature/PR. Flag any unrelated commits.
-- Check the branch is based on a recent `origin/main` (no stale base).
-
-### 2. Identify changed files
-
-Run `git diff --name-only HEAD` (unstaged) and `git diff --cached --name-only` (staged) to get all changed files. If there are no local changes, diff against the base branch using `git diff main...HEAD --name-only` and `git diff main...HEAD` to review all commits on the current branch.
-
-### 3. Run linters
-
-```bash
-./scripts/hooks/lint.sh
-```
-
-Report any failures. Do NOT fix them yourself — just report.
-
-### 4. Check for dead code
-
-For each changed Python file, check for:
-- Unused imports (Ruff should catch these, but verify)
-- Functions/methods/classes that were added but are never called from anywhere
-- Variables assigned but never read
-- Commented-out code blocks that should be removed
-
-For each changed TypeScript file, check for:
-- Unused imports
-- Unused variables or functions
-- Commented-out code
-
-### 5. Check type safety (Python)
-
-For each changed Python file, check for violations:
-- **No raw `dict` for structured data** — must use Pydantic model or dataclass, even for internal/private functions (only exception: truly dynamic/unknown keys)
-- **No multi-item tuple returns** — must use dataclass or Pydantic model, even for internal/private functions (no exceptions)
-- **Missing type hints** on function parameters and return types
-- **Missing `@field_validator`** for datetime fields that should be timezone-aware
-
-### 6. Check for missing tests
-
-For each new or significantly changed function/endpoint/class:
-- Check if there is a corresponding test addition or update
-- New API endpoints MUST have integration tests
-- New utility functions MUST have unit tests
-- Bug fixes SHOULD have a regression test
-
-Flag any new logic that lacks test coverage.
-
-### 7. Check API consistency
-
-If any files in `hindsight-api-slim/hindsight_api/api/` were changed:
-- Were the OpenAPI specs regenerated? (`./scripts/generate-openapi.sh`)
-- Were the client SDKs regenerated? (`./scripts/generate-clients.sh`)
-- Were the control plane proxy routes updated? (`hindsight-control-plane/src/app/api/`)
-
-### 8. Check code comments
-
-For each non-trivial change:
-- **New non-obvious logic** — is there a comment explaining the reasoning?
-- **Changed approach** — does the comment include what was done before and why it changed?
-- **Stale comments** — do existing comments near the changed code still accurately describe the behavior?
-
-### 9. Check integration completeness
-
-If any files in `hindsight-integrations/` were added or changed, verify:
-- **Tests exist** — the integration must have tests that simulate/exercise the external framework (not just pure unit tests of helpers). Check for a `tests/` directory with meaningful test files.
-- **CI job exists** — check `.github/workflows/test.yml` for a corresponding `test-<name>-integration` job. If missing, flag it.
-- **Release process** — check that the integration name is in the `VALID_INTEGRATIONS` array in `scripts/release-integration.sh`. If missing, flag it.
-- **Code standards** — the integration code must follow all Python style rules (type hints, no raw dicts, no tuple returns, etc.).
-
-### 10. Check MCP tool registration completeness
-
-If any new MCP tools were added or existing tools renamed in `hindsight-api-slim/hindsight_api/mcp_tools.py`:
-- **`_ALL_TOOLS` set** in `mcp_tools.py` — must include the new tool name
-- **`tools_to_register` default set** in `register_mcp_tools()` in `mcp_tools.py` — must include the new tool name
-- **`_SINGLE_BANK_TOOLS` set** in `hindsight-api-slim/hindsight_api/api/mcp.py` — must include the new tool if it is bank-scoped (not a bank-management tool like `list_banks`/`create_bank`)
-- **`MCP_TOOL_GROUPS`** in `hindsight-control-plane/src/components/bank-config-view.tsx` — must include the new tool in the appropriate group for the UI tool selector
-- **Tool count assertions** in tests (e.g., `test_mcp_tools.py`) — must be updated to reflect the new count
-
-### 11. Review against other coding standards
-
-Check the diff for violations of the standards listed above:
-- Python files at project root (not allowed)
-- Missing async patterns (should be async throughout)
-- Pydantic models for request/response
-- Line length > 120 chars
-- New features/code beyond what was asked (over-engineering)
-- Unnecessary error handling for impossible scenarios
-- Premature abstractions or speculative helpers
-- Backwards-compatibility hacks (unused vars, re-exports, "removed" comments)
-
-### 12. Report findings
-
-Present a clear summary organized by severity:
-
-**Must fix** — issues that will break CI or violate hard project rules:
-- Unrelated commits on the branch
-- Lint failures
-- Missing type hints on public functions
-- Raw dict usage for structured data (including internal code)
-- Multi-item tuple returns (including internal code)
-- Missing tests for new endpoints
-- New integration missing tests, CI job, or release-integration.sh entry
-
-**Should fix** — issues that hurt code quality:
-- Dead code / unused imports missed by linter
-- Missing tests for non-trivial utility functions
-- Over-engineering beyond the task scope
-
-**Note** — observations that may or may not need action:
-- API changes that might need client regeneration
-- Patterns that deviate from nearby code style
-
-For each finding, include the file path, line number, and a brief explanation.
-
-Do NOT auto-fix any issues. Report all findings and let the user decide what to address. If there are no findings, confirm the code looks good.

.env.example

@@ -2,10 +2,10 @@
 # Copy this file to .env and fill in your values
 
 # LLM Configuration (Required)
-# Supported providers: openai, groq, ollama, gemini, anthropic, lmstudio, vertexai, minimax, volcano
+# Supported providers: openai, groq, ollama, gemini, anthropic, lmstudio, vertexai
 HINDSIGHT_API_LLM_PROVIDER=openai
 HINDSIGHT_API_LLM_API_KEY=your-api-key-here
-HINDSIGHT_API_LLM_MODEL=gpt-4o-mini
+HINDSIGHT_API_LLM_MODEL=o3-mini
 HINDSIGHT_API_LLM_BASE_URL=https://api.openai.com/v1
 
 # Example: Anthropic Claude configuration
@@ -20,11 +20,6 @@ HINDSIGHT_API_LLM_BASE_URL=https://api.openai.com/v1
 # HINDSIGHT_API_LLM_VERTEXAI_REGION=us-central1
 # HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY=/path/to/service-account-key.json  # Optional, uses ADC if not set
 
-# Example: MiniMax configuration (1M context window)
-# HINDSIGHT_API_LLM_PROVIDER=minimax
-# HINDSIGHT_API_LLM_API_KEY=your-minimax-api-key
-# HINDSIGHT_API_LLM_MODEL=MiniMax-M2.7
-
 # Example: LM Studio local configuration (Qwen 2.5 32B recommended)
 # HINDSIGHT_API_LLM_PROVIDER=lmstudio
 # HINDSIGHT_API_LLM_API_KEY=lmstudio
@@ -44,7 +39,6 @@ HINDSIGHT_API_LOG_LEVEL=info
 
 # Database (Optional - uses embedded pg0 by default)
 # HINDSIGHT_API_DATABASE_URL=postgresql://user:pass@host:5432/db
-# HINDSIGHT_API_MIGRATION_DATABASE_URL=  # Direct PostgreSQL URL for migrations (bypasses PgBouncer). Falls back to DATABASE_URL.
 # HINDSIGHT_API_DATABASE_SCHEMA=public  # PostgreSQL schema name (default: public)
 
 # Vector Extension (Optional - uses pgvector by default)

.github/dependabot.yml

@@ -1,6 +0,0 @@
-version: 2
-updates:
-  - package-ecosystem: "github-actions"
-    directory: "/"
-    schedule:
-      interval: "weekly"

.github/workflows/deploy-docs.yml

@@ -21,20 +21,17 @@ jobs:
   build:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v6
-      - uses: actions/setup-node@v6
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
         with:
           node-version: 20
           cache: npm
           cache-dependency-path: package-lock.json
-      - uses: astral-sh/setup-uv@v7
+      - uses: astral-sh/setup-uv@v4
       - run: npm ci --workspace=hindsight-docs
       - run: uv run generate-llms-full
       - run: npm run build --workspace=hindsight-docs
-        env:
-          UMAMI_URL: https://analytics.hindsight.vectorize.io
-          UMAMI_WEBSITE_ID: ${{ secrets.UMAMI_WEBSITE_ID }}
-      - uses: actions/upload-pages-artifact@v4
+      - uses: actions/upload-pages-artifact@v3
         with:
           path: hindsight-docs/build
   deploy:
@@ -44,5 +41,5 @@ jobs:
     runs-on: ubuntu-latest
     needs: build
     steps:
-      - uses: actions/deploy-pages@v5
+      - uses: actions/deploy-pages@v4
         id: deployment

.github/workflows/release-integration.yml

@@ -1,111 +0,0 @@
-name: Release Integration
-
-on:
-  push:
-    tags:
-      - 'integrations/**'
-
-jobs:
-  publish:
-    runs-on: ubuntu-latest
-    permissions:
-      id-token: write  # for PyPI trusted publishing
-
-    steps:
-    - uses: actions/checkout@v6
-
-    - name: Extract integration info
-      id: info
-      run: |
-        # refs/tags/integrations/litellm/v0.1.0 → integration=litellm, version=0.1.0
-        TAG="${GITHUB_REF#refs/tags/}"
-        INTEGRATION=$(echo "$TAG" | cut -d'/' -f2)
-        VERSION=$(echo "$TAG" | cut -d'/' -f3 | sed 's/^v//')
-        echo "integration=$INTEGRATION" >> $GITHUB_OUTPUT
-        echo "version=$VERSION" >> $GITHUB_OUTPUT
-        echo "tag=$TAG" >> $GITHUB_OUTPUT
-        echo "Integration: $INTEGRATION, Version: $VERSION"
-
-    - name: Detect integration type
-      id: type
-      run: |
-        if [ -f "hindsight-integrations/${{ steps.info.outputs.integration }}/pyproject.toml" ]; then
-          echo "type=python" >> $GITHUB_OUTPUT
-        elif [ -f "hindsight-integrations/${{ steps.info.outputs.integration }}/package.json" ]; then
-          echo "type=typescript" >> $GITHUB_OUTPUT
-        else
-          echo "type=plugin" >> $GITHUB_OUTPUT
-        fi
-
-    # ── Python integrations (litellm, pydantic-ai, crewai) ──────────────────
-
-    - name: Install uv
-      if: steps.type.outputs.type == 'python'
-      uses: astral-sh/setup-uv@v7
-      with:
-        enable-cache: true
-
-    - name: Set up Python
-      if: steps.type.outputs.type == 'python'
-      uses: actions/setup-python@v6
-      with:
-        python-version-file: ".python-version"
-
-    - name: Build Python package
-      if: steps.type.outputs.type == 'python'
-      working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
-      run: uv build --out-dir dist
-
-    - name: Publish Python package to PyPI
-      if: steps.type.outputs.type == 'python'
-      uses: pypa/gh-action-pypi-publish@release/v1
-      with:
-        packages-dir: ./hindsight-integrations/${{ steps.info.outputs.integration }}/dist
-        skip-existing: true
-
-    # ── TypeScript integrations (ai-sdk, chat, openclaw) ────────────────────
-
-    # ── Plugin integrations (claude-code) — no package to publish ───────────
-
-    - name: Plugin release
-      if: steps.type.outputs.type == 'plugin'
-      run: |
-        echo "Plugin integration ${{ steps.info.outputs.integration }} v${{ steps.info.outputs.version }} — no package to publish."
-        echo "Users install via: claude plugin marketplace add vectorize-io/hindsight --sparse hindsight-integrations"
-
-    # ── TypeScript integrations (ai-sdk, chat, openclaw) ────────────────────
-
-    - name: Set up Node.js
-      if: steps.type.outputs.type == 'typescript'
-      uses: actions/setup-node@v6
-      with:
-        node-version: '22'
-        registry-url: 'https://registry.npmjs.org'
-
-    - name: Install dependencies
-      if: steps.type.outputs.type == 'typescript'
-      working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
-      run: npm ci
-
-    - name: Build TypeScript package
-      if: steps.type.outputs.type == 'typescript'
-      working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
-      run: npm run build
-
-    - name: Publish TypeScript package to npm
-      if: steps.type.outputs.type == 'typescript'
-      working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
-      run: |
-        set +e
-        OUTPUT=$(npm publish --access public 2>&1)
-        EXIT_CODE=$?
-        echo "$OUTPUT"
-        if [ $EXIT_CODE -ne 0 ]; then
-          if echo "$OUTPUT" | grep -q "cannot publish over"; then
-            echo "Package version already published, skipping..."
-            exit 0
-          fi
-          exit $EXIT_CODE
-        fi
-      env:
-        NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

.github/workflows/release.yml

@@ -13,15 +13,15 @@ jobs:
       id-token: write
 
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v4
 
     - name: Install uv
-      uses: astral-sh/setup-uv@v7
+      uses: astral-sh/setup-uv@v5
       with:
         enable-cache: true
 
     - name: Set up Python
-      uses: actions/setup-python@v6
+      uses: actions/setup-python@v5
       with:
         python-version-file: ".python-version"
 
@@ -30,39 +30,29 @@ jobs:
       working-directory: ./hindsight-clients/python
       run: uv build --out-dir dist
 
-    - name: Build hindsight-api-slim
-      working-directory: ./hindsight-api-slim
-      run: uv build --out-dir dist
-
     - name: Build hindsight-api
       working-directory: ./hindsight-api
       run: uv build --out-dir dist
 
     - name: Build hindsight-all
-      working-directory: ./hindsight-all
+      working-directory: ./hindsight
       run: uv build --out-dir dist
 
-    - name: Build hindsight-all-slim
-      working-directory: ./hindsight-all-slim
+    - name: Build hindsight-litellm
+      working-directory: ./hindsight-integrations/litellm
       run: uv build --out-dir dist
 
     - name: Build hindsight-embed
       working-directory: ./hindsight-embed
       run: uv build --out-dir dist
 
-    # Publish in order (client and api-slim first, then api/all wrappers which depend on them)
+    # Publish in order (client and api first, then hindsight-all which depends on them)
     - name: Publish hindsight-client to PyPI
       uses: pypa/gh-action-pypi-publish@release/v1
       with:
         packages-dir: ./hindsight-clients/python/dist
         skip-existing: true
 
-    - name: Publish hindsight-api-slim to PyPI
-      uses: pypa/gh-action-pypi-publish@release/v1
-      with:
-        packages-dir: ./hindsight-api-slim/dist
-        skip-existing: true
-
     - name: Publish hindsight-api to PyPI
       uses: pypa/gh-action-pypi-publish@release/v1
       with:
@@ -72,13 +62,13 @@ jobs:
     - name: Publish hindsight-all to PyPI
       uses: pypa/gh-action-pypi-publish@release/v1
       with:
-        packages-dir: ./hindsight-all/dist
+        packages-dir: ./hindsight/dist
         skip-existing: true
 
-    - name: Publish hindsight-all-slim to PyPI
+    - name: Publish hindsight-litellm to PyPI
       uses: pypa/gh-action-pypi-publish@release/v1
       with:
-        packages-dir: ./hindsight-all-slim/dist
+        packages-dir: ./hindsight-integrations/litellm/dist
         skip-existing: true
 
     - name: Publish hindsight-embed to PyPI
@@ -89,15 +79,14 @@ jobs:
 
     # Upload artifacts for GitHub release
     - name: Upload artifacts
-      uses: actions/upload-artifact@v7
+      uses: actions/upload-artifact@v4
       with:
         name: python-packages
         path: |
           hindsight-clients/python/dist/*
-          hindsight-api-slim/dist/*
           hindsight-api/dist/*
-          hindsight-all/dist/*
-          hindsight-all-slim/dist/*
+          hindsight/dist/*
+          hindsight-integrations/litellm/dist/*
           hindsight-embed/dist/*
         retention-days: 1
 
@@ -106,10 +95,10 @@ jobs:
     environment: npm
 
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v4
 
     - name: Set up Node.js
-      uses: actions/setup-node@v6
+      uses: actions/setup-node@v4
       with:
         node-version: '20'
         registry-url: 'https://registry.npmjs.org'
@@ -144,21 +133,119 @@ jobs:
       run: npm pack
 
     - name: Upload artifacts
-      uses: actions/upload-artifact@v7
+      uses: actions/upload-artifact@v4
       with:
         name: typescript-client
         path: hindsight-clients/typescript/*.tgz
         retention-days: 1
 
+  release-openclaw-integration:
+    runs-on: ubuntu-latest
+    environment: npm
+
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: Set up Node.js
+      uses: actions/setup-node@v4
+      with:
+        node-version: '22'
+        registry-url: 'https://registry.npmjs.org'
+
+    - name: Install dependencies
+      working-directory: ./hindsight-integrations/openclaw
+      run: npm ci
+
+    - name: Build
+      working-directory: ./hindsight-integrations/openclaw
+      run: npm run build
+
+    - name: Publish to npm
+      working-directory: ./hindsight-integrations/openclaw
+      run: |
+        set +e
+        OUTPUT=$(npm publish --access public 2>&1)
+        EXIT_CODE=$?
+        echo "$OUTPUT"
+        if [ $EXIT_CODE -ne 0 ]; then
+          if echo "$OUTPUT" | grep -q "cannot publish over"; then
+            echo "Package version already published, skipping..."
+            exit 0
+          fi
+          exit $EXIT_CODE
+        fi
+      env:
+        NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
+
+    - name: Pack for GitHub release
+      working-directory: ./hindsight-integrations/openclaw
+      run: npm pack
+
+    - name: Upload artifacts
+      uses: actions/upload-artifact@v4
+      with:
+        name: openclaw-integration
+        path: hindsight-integrations/openclaw/*.tgz
+        retention-days: 1
+
+  release-ai-sdk-integration:
+    runs-on: ubuntu-latest
+    environment: npm
+
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: Set up Node.js
+      uses: actions/setup-node@v4
+      with:
+        node-version: '22'
+        registry-url: 'https://registry.npmjs.org'
+
+    - name: Install dependencies
+      working-directory: ./hindsight-integrations/ai-sdk
+      run: npm ci
+
+    - name: Build
+      working-directory: ./hindsight-integrations/ai-sdk
+      run: npm run build
+
+    - name: Publish to npm
+      working-directory: ./hindsight-integrations/ai-sdk
+      run: |
+        set +e
+        OUTPUT=$(npm publish --access public 2>&1)
+        EXIT_CODE=$?
+        echo "$OUTPUT"
+        if [ $EXIT_CODE -ne 0 ]; then
+          if echo "$OUTPUT" | grep -q "cannot publish over"; then
+            echo "Package version already published, skipping..."
+            exit 0
+          fi
+          exit $EXIT_CODE
+        fi
+      env:
+        NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
+
+    - name: Pack for GitHub release
+      working-directory: ./hindsight-integrations/ai-sdk
+      run: npm pack
+
+    - name: Upload artifacts
+      uses: actions/upload-artifact@v4
+      with:
+        name: ai-sdk-integration
+        path: hindsight-integrations/ai-sdk/*.tgz
+        retention-days: 1
+
   release-control-plane:
     runs-on: ubuntu-latest
     environment: npm
 
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v4
 
     - name: Set up Node.js
-      uses: actions/setup-node@v6
+      uses: actions/setup-node@v4
       with:
         node-version: '20'
         registry-url: 'https://registry.npmjs.org'
@@ -181,14 +268,11 @@ jobs:
     - name: Build
       run: npm run build --workspace=hindsight-control-plane
 
-    - name: Verify standalone build
-      run: test -f hindsight-control-plane/standalone/server.js || (echo 'standalone/server.js missing - build failed' && exit 1)
-
     - name: Publish to npm
       working-directory: ./hindsight-control-plane
       run: |
         set +e
-        OUTPUT=$(npm publish --access public --ignore-scripts 2>&1)
+        OUTPUT=$(npm publish --access public 2>&1)
         EXIT_CODE=$?
         echo "$OUTPUT"
         if [ $EXIT_CODE -ne 0 ]; then
@@ -206,7 +290,7 @@ jobs:
       run: npm pack
 
     - name: Upload artifacts
-      uses: actions/upload-artifact@v7
+      uses: actions/upload-artifact@v4
       with:
         name: control-plane
         path: hindsight-control-plane/*.tgz
@@ -229,13 +313,9 @@ jobs:
             target: aarch64-apple-darwin
             artifact_name: hindsight
             asset_name: hindsight-darwin-arm64
-          - os: ubuntu-24.04-arm
-            target: aarch64-unknown-linux-gnu
-            artifact_name: hindsight
-            asset_name: hindsight-linux-arm64
 
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v4
 
     - name: Install Rust
       uses: dtolnay/rust-toolchain@stable
@@ -253,7 +333,7 @@ jobs:
         chmod +x artifacts/${{ matrix.asset_name }}
 
     - name: Upload artifacts
-      uses: actions/upload-artifact@v7
+      uses: actions/upload-artifact@v4
       with:
         name: rust-cli-${{ matrix.asset_name }}
         path: artifacts/${{ matrix.asset_name }}
@@ -294,7 +374,7 @@ jobs:
               PRELOAD_ML_MODELS=false
 
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v4
 
     - name: Free Disk Space
       uses: jlumbroso/free-disk-space@main
@@ -308,13 +388,13 @@ jobs:
         swap-storage: true
 
     - name: Set up QEMU
-      uses: docker/setup-qemu-action@v4
+      uses: docker/setup-qemu-action@v3
 
     - name: Set up Docker Buildx
-      uses: docker/setup-buildx-action@v4
+      uses: docker/setup-buildx-action@v3
 
     - name: Log in to GitHub Container Registry
-      uses: docker/login-action@v4
+      uses: docker/login-action@v3
       with:
         registry: ghcr.io
         username: ${{ github.actor }}
@@ -326,7 +406,7 @@ jobs:
 
     - name: Extract metadata for release tags
       id: meta
-      uses: docker/metadata-action@v6
+      uses: docker/metadata-action@v5
       with:
         images: ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}
         flavor: |
@@ -342,7 +422,7 @@ jobs:
     # # Step 1: Build for local testing (single platform, no push)
     # # This creates an identical image to what will be released, just for one platform
     # - name: Build image for testing
-    #   uses: docker/build-push-action@v7
+    #   uses: docker/build-push-action@v6
     #   with:
     #     context: .
     #     file: docker/standalone/Dockerfile
@@ -361,7 +441,7 @@ jobs:
 
     # Build multi-platform and push to release tags
     - name: Build and push release images
-      uses: docker/build-push-action@v7
+      uses: docker/build-push-action@v6
       with:
         context: .
         file: docker/standalone/Dockerfile
@@ -379,10 +459,10 @@ jobs:
       packages: write
 
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v4
 
     - name: Install Helm
-      uses: azure/setup-helm@v5
+      uses: azure/setup-helm@v4
       with:
         version: 'latest'
 
@@ -399,7 +479,7 @@ jobs:
       run: helm push helm-packages/*.tgz oci://ghcr.io/${{ github.repository_owner }}/charts
 
     - name: Upload artifacts
-      uses: actions/upload-artifact@v7
+      uses: actions/upload-artifact@v4
       with:
         name: helm-chart
         path: helm-packages/*.tgz
@@ -407,55 +487,67 @@ jobs:
 
   create-github-release:
     runs-on: ubuntu-latest
-    needs: [release-python-packages, release-typescript-client, release-control-plane, release-rust-cli, release-docker-images, release-helm-chart]
+    needs: [release-python-packages, release-typescript-client, release-openclaw-integration, release-ai-sdk-integration, release-control-plane, release-rust-cli, release-docker-images, release-helm-chart]
     permissions:
       contents: write
 
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v4
 
     - name: Extract version from tag
       id: get_version
       run: echo "VERSION=${GITHUB_REF#refs/tags/v}" >> $GITHUB_OUTPUT
 
     - name: Download Python packages
-      uses: actions/download-artifact@v8
+      uses: actions/download-artifact@v4
       with:
         name: python-packages
         path: ./artifacts/python-packages
 
     - name: Download TypeScript client
-      uses: actions/download-artifact@v8
+      uses: actions/download-artifact@v4
       with:
         name: typescript-client
         path: ./artifacts/typescript-client
 
+    - name: Download OpenClaw Integration
+      uses: actions/download-artifact@v4
+      with:
+        name: openclaw-integration
+        path: ./artifacts/openclaw-integration
+
+    - name: Download AI SDK Integration
+      uses: actions/download-artifact@v4
+      with:
+        name: ai-sdk-integration
+        path: ./artifacts/ai-sdk-integration
+
     - name: Download Control Plane
-      uses: actions/download-artifact@v8
+      uses: actions/download-artifact@v4
       with:
         name: control-plane
         path: ./artifacts/control-plane
 
     - name: Download Rust CLI (Linux)
-      uses: actions/download-artifact@v8
+      uses: actions/download-artifact@v4
       with:
         name: rust-cli-hindsight-linux-amd64
         path: ./artifacts/rust-cli-linux
 
     - name: Download Rust CLI (macOS Intel)
-      uses: actions/download-artifact@v8
+      uses: actions/download-artifact@v4
       with:
         name: rust-cli-hindsight-darwin-amd64
         path: ./artifacts/rust-cli-darwin-amd64
 
     - name: Download Rust CLI (macOS ARM)
-      uses: actions/download-artifact@v8
+      uses: actions/download-artifact@v4
       with:
         name: rust-cli-hindsight-darwin-arm64
         path: ./artifacts/rust-cli-darwin-arm64
 
     - name: Download Helm chart
-      uses: actions/download-artifact@v8
+      uses: actions/download-artifact@v4
       with:
         name: helm-chart
         path: ./artifacts/helm-chart
@@ -465,13 +557,16 @@ jobs:
         mkdir -p release-assets
         # Python packages
         cp artifacts/python-packages/hindsight-clients/python/dist/* release-assets/ || true
-        cp artifacts/python-packages/hindsight-api-slim/dist/* release-assets/ || true
         cp artifacts/python-packages/hindsight-api/dist/* release-assets/ || true
-        cp artifacts/python-packages/hindsight-all/dist/* release-assets/ || true
-        cp artifacts/python-packages/hindsight-all-slim/dist/* release-assets/ || true
+        cp artifacts/python-packages/hindsight/dist/* release-assets/ || true
+        cp artifacts/python-packages/hindsight-integrations/litellm/dist/* release-assets/ || true
         cp artifacts/python-packages/hindsight-embed/dist/* release-assets/ || true
         # TypeScript client
         cp artifacts/typescript-client/*.tgz release-assets/ || true
+        # OpenClaw Integration
+        cp artifacts/openclaw-integration/*.tgz release-assets/ || true
+        # AI SDK Integration
+        cp artifacts/ai-sdk-integration/*.tgz release-assets/ || true
         # Control Plane
         cp artifacts/control-plane/*.tgz release-assets/ || true
         # Rust CLI binaries

.gitignore

@@ -50,8 +50,7 @@ hindsight-dev/benchmarks/perf/results/
 benchmarks/results/
 hindsight-cli/target
 hindsight-clients/rust/target
-.claude/*
-!.claude/skills/
+.claude
 whats-next.md
 TASK.md
 # Changelog is now tracked in hindsight-docs/src/pages/changelog.md

CLAUDE.md

@@ -11,32 +11,26 @@ Hindsight is an agent memory system that provides long-term memory for AI agents
 
 ## Development Commands
 
-### Local Development (API + UI)
-```bash
-# Start both API server and control plane UI
-./scripts/dev/start.sh
-```
-
 ### API Server (Python/FastAPI)
 ```bash
-# Start API server only (loads .env automatically)
+# Start API server (loads .env automatically)
 ./scripts/dev/start-api.sh
 
 # Run all tests (parallelized with pytest-xdist)
-cd hindsight-api-slim && uv run pytest tests/
+cd hindsight-api && uv run pytest tests/
 
 # Run specific test file
-cd hindsight-api-slim && uv run pytest tests/test_http_api_integration.py -v
+cd hindsight-api && uv run pytest tests/test_http_api_integration.py -v
 
 # Run single test function
-cd hindsight-api-slim && uv run pytest tests/test_retain.py::test_retain_simple -v
+cd hindsight-api && uv run pytest tests/test_retain.py::test_retain_simple -v
 
 # Lint and format
-cd hindsight-api-slim && uv run ruff check .
-cd hindsight-api-slim && uv run ruff format .
+cd hindsight-api && uv run ruff check .
+cd hindsight-api && uv run ruff format .
 
 # Type checking (uses ty - extremely fast type checker from Astral)
-cd hindsight-api-slim && uv run ty check hindsight_api/
+cd hindsight-api && uv run ty check hindsight_api/
 ```
 
 ### Control Plane (Next.js)
@@ -78,17 +72,18 @@ cd hindsight-control-plane && npm run dev
 ## Architecture
 
 ### Monorepo Structure
-- **hindsight-api-slim/**: Core FastAPI server with memory engine (Python, uv)
+- **hindsight-api/**: Core FastAPI server with memory engine (Python, uv)
+- **hindsight/**: Embedded Python bundle (hindsight-all package)
 - **hindsight-control-plane/**: Admin UI (Next.js, npm)
 - **hindsight-cli/**: CLI tool (Rust, cargo, uses progenitor for API client)
 - **hindsight-clients/**: Generated SDK clients (Python, TypeScript, Rust)
 - **hindsight-docs/**: Docusaurus documentation site
-- **hindsight-integrations/**: Framework integrations (LiteLLM, CrewAI, LangGraph, Pydantic AI, AG2, Claude Code, etc.)
+- **hindsight-integrations/**: Framework integrations (LiteLLM, OpenAI)
 - **hindsight-dev/**: Development tools and benchmarks
 
-### Core Engine (hindsight-api-slim/hindsight_api/engine/)
-- `memory_engine.py`: Main orchestrator for retain/recall/reflect operations
-- `llm_wrapper.py`: LLM abstraction supporting OpenAI, Anthropic, Gemini, VertexAI, Groq, MiniMax, Ollama, LM Studio, LiteLLM, Claude Code
+### Core Engine (hindsight-api/hindsight_api/engine/)
+- `memory_engine.py`: Main orchestrator (~170KB) for retain/recall/reflect operations
+- `llm_wrapper.py`: LLM abstraction supporting OpenAI, Anthropic, Gemini, Groq, Ollama, LM Studio
 - `embeddings.py`: Embedding generation (local sentence-transformers or TEI)
 - `cross_encoder.py`: Reranking (local or TEI)
 - `entity_resolver.py`: Entity extraction and normalization
@@ -101,13 +96,13 @@ cd hindsight-control-plane && npm run dev
 
 **search/**: Multi-strategy retrieval
 - `retrieval.py`: Main retrieval orchestrator
-- `graph_retrieval.py`: Graph retrieval abstract base class
-- `link_expansion_retrieval.py`: Link expansion graph retrieval
+- `graph_retrieval.py`: Entity/relationship graph traversal
+- `mpfp_retrieval.py`: Multi-Path Fact Propagation retrieval
 - `fusion.py`: Reciprocal rank fusion for combining results
 - `reranking.py`: Cross-encoder reranking
 
-### API Layer (hindsight-api-slim/hindsight_api/api/)
-- `http.py`: FastAPI HTTP routers for all REST endpoints
+### API Layer (hindsight-api/hindsight_api/api/)
+- `http.py`: FastAPI HTTP routers (~80KB) for all REST endpoints
 - `mcp.py`: Model Context Protocol server implementation
 
 Main operations:
@@ -116,13 +111,13 @@ Main operations:
 - **Reflect**: Disposition-aware reasoning using memories and mental models.
 
 ### Database
-PostgreSQL with pgvector. Schema managed via Alembic migrations in `hindsight-api-slim/hindsight_api/alembic/`. Migrations run automatically on API startup.
+PostgreSQL with pgvector. Schema managed via Alembic migrations in `hindsight-api/hindsight_api/alembic/`. Migrations run automatically on API startup.
 
 Key tables: `banks`, `memory_units`, `documents`, `entities`, `entity_links`
 
 ### Adding Database Migrations
 
-1. **Create a new migration file** in `hindsight-api-slim/hindsight_api/alembic/versions/`:
+1. **Create a new migration file** in `hindsight-api/hindsight_api/alembic/versions/`:
    - File name format: `<revision_id>_<description>.py` (e.g., `f1a2b3c4d5e6_add_new_index.py`)
    - Use a unique hex revision ID (12 chars)
    - Set `down_revision` to the previous migration's revision ID
@@ -159,7 +154,7 @@ Key tables: `banks`, `memory_units`, `documents`, `entities`, `entity_links`
 
 3. **Run migrations locally**:
    ```bash
-   # Set database URL and run migrations for the base schema plus all tenants
+   # Set database URL and run migrations
    uv run hindsight-admin run-db-migration
 
    # Run on a specific tenant schema
@@ -169,17 +164,11 @@ Key tables: `banks`, `memory_units`, `documents`, `entities`, `entity_links`
 ## Key Conventions
 
 ### Code Quality
-
-**Before writing code, read `.claude/skills/code-review/SKILL.md`** for the full coding standards (Python style, type safety, TypeScript style, general principles).
-
 **Always run the lint script after making Python or TypeScript/Node changes:**
 ```bash
 ./scripts/hooks/lint.sh
 ```
-
-**After completing any implementation work, run `/code-review`** to verify your changes against project standards (missing tests, dead code, type safety, etc.). Fix any "must fix" issues before considering the task done.
-
-**MANDATORY: Run `/code-review` before pushing code or creating a pull request.** Do not push or create a PR until all "must fix" issues are resolved.
+This runs the same checks as the pre-commit hook (Ruff for Python, ESLint/Prettier for TypeScript).
 
 ### Memory Banks
 - Each bank is an isolated memory store (like a "brain" for one user/agent)
@@ -211,16 +200,48 @@ When adding or modifying parameters in the dataplane API (hindsight-api), you mu
    - Update the client type definition in `lib/api.ts`
    - Update any UI components that need to use the new parameter
 
-### Adding New Integrations
+### Python Style
+- Python 3.11+, type hints required
+- Async throughout (asyncpg, async FastAPI)
+- Pydantic models for request/response
+- Ruff for linting (line-length 120)
+- No Python files at project root - maintain clean directory structure
+- **Never use multi-item tuple return values** - prefer dataclass or Pydantic model for structured returns
 
-Every new integration in `hindsight-integrations/` must satisfy all of the following before it can be merged:
+### Type Safety with Pydantic Models
+**NEVER use raw `dict` types for structured data.** Always use Pydantic models:
+- Use Pydantic `BaseModel` for all data structures passed between functions
+- Add `@field_validator` for type coercion (e.g., ensuring datetimes are timezone-aware)
+- Avoid `dict.get()` patterns - use typed model attributes instead
+- Parse external data (JSON, API responses) into Pydantic models at the boundary
+- This catches type errors at parse time, not deep in business logic
 
-1. **Tests are required** — tests must simulate or exercise the external system (mock the framework's interfaces and verify the integration actually calls Hindsight correctly). Pure unit tests of helper functions are not sufficient.
-2. **CI job** — add a test job in `.github/workflows/test.yml` following the existing pattern (e.g., `test-crewai-integration`). The job must build, install deps, and run `uv run pytest tests -v`. Also add the integration to `detect-changes` outputs so it only runs when its files change.
-3. **Release process** — add the integration name to the `VALID_INTEGRATIONS` array in `scripts/release-integration.sh` so it can be released via the standard release workflow.
-4. **Follow project code standards** — Python style, type safety, no raw dicts for structured data, no multi-item tuple returns (see `.claude/skills/code-review/SKILL.md`).
+```python
+# BAD - error-prone dict access
+def process(data: dict) -> str:
+    return data.get("name", "")  # No validation, silent failures
 
-If any of these are missing, the integration is incomplete and must not be pushed or merged.
+# GOOD - typed and validated
+class UserData(BaseModel):
+    name: str
+    created_at: datetime
+
+    @field_validator("created_at", mode="before")
+    @classmethod
+    def ensure_tz_aware(cls, v):
+        if isinstance(v, str):
+            v = datetime.fromisoformat(v.replace("Z", "+00:00"))
+        if v.tzinfo is None:
+            return v.replace(tzinfo=timezone.utc)
+        return v
+
+def process(data: UserData) -> str:
+    return data.name  # Type-safe, validated at construction
+```
+
+### TypeScript Style
+- Next.js App Router for control plane
+- Tailwind CSS with shadcn/ui components
 
 ### Adding New API Configuration Flags
 
@@ -230,24 +251,24 @@ Fields must be categorized as either **hierarchical** (can be overridden per-ten
 
 #### Adding a New Configuration Field
 
-1. **config.py** (`hindsight-api-slim/hindsight_api/config.py`):
+1. **config.py** (`hindsight-api/hindsight_api/config.py`):
    - Add `ENV_*` constant for the environment variable name (e.g., `ENV_MY_SETTING = "HINDSIGHT_API_MY_SETTING"`)
    - Add `DEFAULT_*` constant for the default value
    - Add field to `HindsightConfig` dataclass with type annotation
-   - **Mark as configurable** by adding to `_CONFIGURABLE_FIELDS` set if the field should be overridable per-tenant/bank via API
+   - **Mark as hierarchical or static** by adding to `_HIERARCHICAL_FIELDS` set (hierarchical) or leaving it out (static)
    - Add initialization in `from_env()` method
 
    ```python
-   # Configurable field (can be overridden per-tenant/bank via API)
-   _CONFIGURABLE_FIELDS = {
+   # Hierarchical field (can be overridden per-bank)
+   _HIERARCHICAL_FIELDS = {
        ...,
-       "my_setting",  # Add here for configurable
+       "my_setting",  # Add here for hierarchical
    }
 
-   # Static field - just don't add to _CONFIGURABLE_FIELDS
+   # Static field - just don't add to _HIERARCHICAL_FIELDS
    ```
 
-2. **main.py** (`hindsight-api-slim/hindsight_api/main.py`):
+2. **main.py** (`hindsight-api/hindsight_api/main.py`):
    - Add field to the manual `HindsightConfig()` constructor call (search for "CLI override")
 
 3. **Use hierarchical config in MemoryEngine**:
@@ -287,19 +308,19 @@ cp .env.example .env
 # Edit .env with LLM API key
 
 # Python deps
-uv sync --directory hindsight-api-slim/
+uv sync --directory hindsight-api/
 
 # Node deps (uses npm workspaces)
 npm install
 ```
 
 Required env vars:
-- `HINDSIGHT_API_LLM_PROVIDER`: openai, anthropic, gemini, groq, minimax, ollama, lmstudio
+- `HINDSIGHT_API_LLM_PROVIDER`: openai, anthropic, gemini, groq, ollama, lmstudio
 - `HINDSIGHT_API_LLM_API_KEY`: Your API key
-- `HINDSIGHT_API_LLM_MODEL`: Model name (e.g., gpt-4o-mini, claude-sonnet-4-20250514)
+- `HINDSIGHT_API_LLM_MODEL`: Model name (e.g., o3-mini, claude-sonnet-4-20250514)
 
 Optional (uses local models by default):
 - `HINDSIGHT_API_EMBEDDINGS_PROVIDER`: local (default) or tei
 - `HINDSIGHT_API_RERANKER_PROVIDER`: local (default) or tei
 - `HINDSIGHT_API_DATABASE_URL`: External PostgreSQL (uses embedded pg0 by default)
-- `HINDSIGHT_API_ENABLE_BANK_CONFIG_API`: Enable per-bank config API (default: true)
+- `HINDSIGHT_API_ENABLE_BANK_CONFIG_API`: Enable per-bank config API (default: false, disabled for security)

README.md

@@ -7,12 +7,10 @@
 [![CI](https://github.com/vectorize-io/hindsight/actions/workflows/release.yml/badge.svg)](https://github.com/vectorize-io/hindsight/actions/workflows/release.yml)
 [![Slack Community](https://img.shields.io/badge/Slack-Join%20Community-4A154B?logo=slack)](https://join.slack.com/t/hindsight-space/shared_invite/zt-3nhbm4w29-LeSJ5Ixi6j8PdiYOCPlOgg)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
-[![gitcgr](https://gitcgr.com/badge/vectorize-io/hindsight.svg)](https://gitcgr.com/vectorize-io/hindsight)
 ![PyPI - Downloads](https://img.shields.io/pypi/dm/hindsight-api?label=PyPI)
 ![NPM Downloads](https://img.shields.io/npm/dm/%40vectorize-io%2Fhindsight-client?logoColor=orange&label=NPM&color=blue&link=https%3A%2F%2Fwww.npmjs.com%2Fpackage%2F%40vectorize-io%2Fhindsight-client)
-<br/>
 
-<a href="https://trendshift.io/repositories/15603" target="_blank"><img src="https://trendshift.io/api/badge/repositories/15603" alt="vectorize-io%2Fhindsight | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
+
 </div>
 
 ---
@@ -38,7 +36,7 @@ Hindsight is being used in production at Fortune 500 enterprises and by a growin
 
 ## Adding Hindsight to Your AI Agents
 
-The easiest way to use Hindsight with an existing agent is with the LLM Wrapper. You can add memory to your agent with 2 lines of code. That will swap your current LLM client out with the Hindsight wrapper. After that, memories will be stored and retrieved automatically as you make LLM calls.
+The easiest way use Hindsight with an existing agent is with the LLM Wrapper. You can add memory to your agent with 2 lines of code. That will swap your current LLM client out with the Hindsight wrapper. After that, memories will be stored and retrieved automatically as you make LLM calls.
 
 If you need more control over how and when your agent stores and recalls memories, there's also a simple API you can integrate with using the SDKs or directly via HTTP.
 
@@ -71,7 +69,7 @@ docker run --rm -it --pull always -p 8888:8888 -p 9999:9999 \
 >API: http://localhost:8888
 >UI: http://localhost:9999
 
-You can modify the LLM provider by setting `HINDSIGHT_API_LLM_PROVIDER`. Valid options are `openai`, `anthropic`, `gemini`, `groq`, `ollama`, `lmstudio`, and `minimax`. The documentation provides more details on [supported models](https://hindsight.vectorize.io/developer/models).
+You can modify the LLM provider by setting `HINDSIGHT_API_LLM_PROVIDER`. Valid options are `openai`, `anthropic`, `gemini`, `groq`, `ollama`, and `lmstudio`. The documentation provides more details on [supported models](https://hindsight.vectorize.io/developer/models).
 
 
 
@@ -183,7 +181,7 @@ Satisfying these requirements in Hindsight is straightforward. When new user inp
 
 ![Overview](./hindsight-docs/static/img/hindsight-overview.webp)
 
-Most agent memory implementations rely on basic vector search or sometimes use a knowledge graph. Hindsight uses biomimetic data structures to organize agent memories in a way that is more like how human memory works:
+Most agent memory implementation rely on basic vector search or sometimes use a knowledge graph. Hindsight uses biomimetic data structures to organize agent memories in a way that is more like how human memory works:
 
 - **World:** Facts about the world ("The stove gets hot")
 - **Experiences:** Agent's own experiences ("I touched the stove and it really hurt")
@@ -309,5 +307,3 @@ MIT — see [LICENSE](./LICENSE)
 ---
 
 Built by [Vectorize.io](https://vectorize.io)
-
-<img src="https://umami-pixel.chris-latimer.workers.dev/?id=a8b043e6-6964-454d-80df-69b69d3f0d50&host=github.com&url=/vectorize-io/hindsight" width="1" height="1" alt="" />

deno.lock

@@ -1,139 +0,0 @@
-{
-  "version": "5",
-  "specifiers": {
-    "jsr:@std/assert@^1.0.17": "1.0.19",
-    "jsr:@std/assert@^1.0.19": "1.0.19",
-    "jsr:@std/expect@*": "1.0.18",
-    "jsr:@std/internal@^1.0.12": "1.0.12",
-    "jsr:@std/path@^1.1.4": "1.1.4",
-    "jsr:@std/testing@*": "1.0.17"
-  },
-  "jsr": {
-    "@std/[email protected]": {
-      "integrity": "eaada96ee120cb980bc47e040f82814d786fe8162ecc53c91d8df60b8755991e",
-      "dependencies": [
-        "jsr:@std/internal"
-      ]
-    },
-    "@std/[email protected]": {
-      "integrity": "8566eab35200466f8609eb7e7aed062ed0db314e9a258d5d201b1b8997ce801a",
-      "dependencies": [
-        "jsr:@std/assert@^1.0.19",
-        "jsr:@std/internal",
-        "jsr:@std/path"
-      ]
-    },
-    "@std/[email protected]": {
-      "integrity": "972a634fd5bc34b242024402972cd5143eac68d8dffaca5eaa4dba30ce17b027"
-    },
-    "@std/[email protected]": {
-      "integrity": "1d2d43f39efb1b42f0b1882a25486647cb851481862dc7313390b2bb044314b5",
-      "dependencies": [
-        "jsr:@std/internal"
-      ]
-    },
-    "@std/[email protected]": {
-      "integrity": "87bdc2700fa98249d48a17cd72413352d3d3680dcfbdb64947fd0982d6bbf681",
-      "dependencies": [
-        "jsr:@std/assert@^1.0.17",
-        "jsr:@std/internal"
-      ]
-    }
-  },
-  "workspace": {
-    "members": {
-      "hindsight-clients/typescript": {
-        "packageJson": {
-          "dependencies": [
-            "npm:@hey-api/[email protected]",
-            "npm:@types/jest@29",
-            "npm:@types/node@20",
-            "npm:jest@29",
-            "npm:ts-jest@29",
-            "npm:tsup@^8.5.1",
-            "npm:typescript@5"
-          ]
-        }
-      },
-      "hindsight-control-plane": {
-        "packageJson": {
-          "dependencies": [
-            "npm:@eslint/eslintrc@^3.3.3",
-            "npm:@eslint/js@^9.39.2",
-            "npm:@radix-ui/react-alert-dialog@^1.1.15",
-            "npm:@radix-ui/react-checkbox@^1.3.3",
-            "npm:@radix-ui/react-dialog@^1.1.15",
-            "npm:@radix-ui/react-dropdown-menu@^2.1.16",
-            "npm:@radix-ui/react-label@^2.1.8",
-            "npm:@radix-ui/react-popover@^1.1.15",
-            "npm:@radix-ui/react-radio-group@^1.3.8",
-            "npm:@radix-ui/react-select@^2.2.6",
-            "npm:@radix-ui/react-slider@^1.3.6",
-            "npm:@radix-ui/react-slot@^1.2.4",
-            "npm:@radix-ui/react-switch@^1.2.6",
-            "npm:@radix-ui/react-tabs@^1.1.13",
-            "npm:@radix-ui/react-tooltip@^1.2.8",
-            "npm:@tailwindcss/postcss@^4.1.17",
-            "npm:@tailwindcss/typography@~0.5.19",
-            "npm:@types/cytoscape@^3.21.9",
-            "npm:@types/node@^24.10.0",
-            "npm:@types/react-dom@^19.2.2",
-            "npm:@types/react@^19.2.2",
-            "npm:autoprefixer@^10.4.21",
-            "npm:class-variance-authority@~0.7.1",
-            "npm:clsx@^2.1.1",
-            "npm:cmdk@^1.1.1",
-            "npm:cytoscape-fcose@^2.2.0",
-            "npm:cytoscape@^3.33.1",
-            "npm:eslint-config-next@^16.0.1",
-            "npm:eslint-plugin-react-hooks@^7.0.1",
-            "npm:eslint-plugin-react@^7.37.5",
-            "npm:eslint@^9.39.1",
-            "npm:[email protected]",
-            "npm:next-themes@~0.4.6",
-            "npm:next@^16.1.6",
-            "npm:postcss@^8.5.6",
-            "npm:prettier@^3.7.4",
-            "npm:react-chrono@^2.9.1",
-            "npm:react-dom@^19.2.0",
-            "npm:react-markdown@^10.1.0",
-            "npm:react18-json-view@~0.2.9",
-            "npm:react@^19.2.0",
-            "npm:recharts@^3.5.1",
-            "npm:remark-gfm@^4.0.1",
-            "npm:sonner@^2.0.7",
-            "npm:tailwind-merge@^3.4.0",
-            "npm:tailwindcss-animate@^1.0.7",
-            "npm:tailwindcss@^4.1.17",
-            "npm:[email protected]",
-            "npm:typescript-eslint@^8.50.0",
-            "npm:typescript@^5.9.3"
-          ]
-        }
-      },
-      "hindsight-docs": {
-        "packageJson": {
-          "dependencies": [
-            "npm:@docusaurus/[email protected]",
-            "npm:@docusaurus/[email protected]",
-            "npm:@docusaurus/[email protected]",
-            "npm:@docusaurus/theme-common@^3.9.2",
-            "npm:@docusaurus/theme-mermaid@^3.9.2",
-            "npm:@docusaurus/[email protected]",
-            "npm:@docusaurus/[email protected]",
-            "npm:@easyops-cn/docusaurus-search-local@~0.52.2",
-            "npm:@mdx-js/react@3",
-            "npm:clsx@2",
-            "npm:prism-react-renderer@^2.3.0",
-            "npm:raw-loader@^4.0.2",
-            "npm:react-dom@19",
-            "npm:react-icons@^5.6.0",
-            "npm:react@19",
-            "npm:redocusaurus@^2.5.0",
-            "npm:typescript@~5.6.2"
-          ]
-        }
-      }
-    }
-  }
-}

docker/standalone/Dockerfile

@@ -42,22 +42,25 @@ RUN apt-get update && apt-get install -y \
     && pip install --no-cache-dir uv
 
 # Copy dependency files and README (required by pyproject.toml)
-COPY hindsight-api-slim/pyproject.toml ./api/
-COPY hindsight-api-slim/README.md ./api/
+COPY hindsight-api/pyproject.toml ./api/
+COPY hindsight-api/README.md ./api/
 
 WORKDIR /app/api
 
-# Sync dependencies using appropriate extras based on INCLUDE_LOCAL_MODELS
-# local-ml: torch, sentence-transformers, transformers, einops, flashrank, mlx (optional)
-# embedded-db: pg0-embedded (always included for embedded PostgreSQL support)
-RUN if [ "$INCLUDE_LOCAL_MODELS" = "true" ]; then \
-        uv sync --extra local-ml --extra embedded-db; \
-    else \
-        uv sync --extra embedded-db; \
+# Remove local ML model dependencies if INCLUDE_LOCAL_MODELS=false
+# This creates a smaller image when using external providers (TEI, OpenAI, Cohere)
+RUN if [ "$INCLUDE_LOCAL_MODELS" != "true" ]; then \
+        echo "Removing local-models dependencies (sentence-transformers, torch, transformers)..." && \
+        sed -i '/"sentence-transformers/d' pyproject.toml && \
+        sed -i '/"transformers/d' pyproject.toml && \
+        sed -i '/"torch/d' pyproject.toml; \
     fi
 
+# Sync dependencies (will create lock file if needed)
+RUN uv sync
+
 # Copy source code (alembic migrations are inside hindsight_api/)
-COPY hindsight-api-slim/hindsight_api ./hindsight_api
+COPY hindsight-api/hindsight_api ./hindsight_api
 
 # Install the local package (uv sync only installed dependencies, not the package itself)
 RUN uv pip install -e .
@@ -167,11 +170,6 @@ RUN chown -R hindsight:hindsight /app
 
 USER hindsight
 
-# Create pg0 data directory as hindsight user so that Docker seeds new named
-# volumes with correct ownership (UID 1000) on first use, avoiding the
-# "Permission denied" error when mounting a fresh root-owned volume.
-RUN mkdir -p /home/hindsight/.pg0
-
 ENV PATH="/app/api/.venv/bin:${PATH}"
 
 # Pre-download tiktoken encoding (ALWAYS - required for token counting even in air-gapped envs)
@@ -323,11 +321,6 @@ RUN chown -R hindsight:hindsight /app
 
 USER hindsight
 
-# Create pg0 data directory as hindsight user so that Docker seeds new named
-# volumes with correct ownership (UID 1000) on first use, avoiding the
-# "Permission denied" error when mounting a fresh root-owned volume.
-RUN mkdir -p /home/hindsight/.pg0
-
 ENV PATH="/app/api/.venv/bin:${PATH}"
 
 # Pre-download tiktoken encoding (ALWAYS - required for token counting even in air-gapped envs)

docker/standalone/start-all.sh

@@ -1,28 +1,6 @@
 #!/bin/bash
 set -e
 
-# =============================================================================
-# Embedded pg0 data integrity check (#675)
-#
-# When using embedded pg0, check if the data directory has existing PostgreSQL
-# data before starting. If the directory exists but appears empty/corrupt
-# (e.g., missing PG_VERSION file), log a warning. This helps diagnose data
-# loss scenarios where a container restart caused the data directory to be
-# wiped despite a volume mount being present.
-# =============================================================================
-PG0_DATA_DIR="${HOME}/.pg0"
-if [ -d "$PG0_DATA_DIR" ]; then
-    # Look for actual PostgreSQL data directories (pg0 creates subdirs per instance)
-    if compgen -G "$PG0_DATA_DIR"/*/PG_VERSION > /dev/null 2>&1; then
-        echo "✅ Existing pg0 data directory detected at $PG0_DATA_DIR"
-    elif [ "$(ls -A "$PG0_DATA_DIR" 2>/dev/null)" ]; then
-        echo "⚠️  WARNING: pg0 data directory exists at $PG0_DATA_DIR but no PG_VERSION found."
-        echo "   This may indicate data corruption or an incomplete previous shutdown."
-        echo "   If you see all migrations running from scratch after this, your data may have been lost."
-        echo "   See: https://github.com/vectorize-io/hindsight/issues/675"
-    fi
-fi
-
 # Service flags (default to true if not set)
 ENABLE_API="${HINDSIGHT_ENABLE_API:-true}"
 ENABLE_CP="${HINDSIGHT_ENABLE_CP:-true}"
@@ -93,95 +71,24 @@ if [ "${HINDSIGHT_WAIT_FOR_DEPS:-false}" = "true" ]; then
     done
 fi
 
-# =============================================================================
-# Graceful shutdown handler (#675)
-#
-# Docker sends SIGTERM on `docker stop`/`docker restart`. Without a trap, child
-# processes (hindsight-api + pg0, control-plane) are killed abruptly. For the
-# embedded pg0 database this can cause data loss when the data directory is on
-# a Docker volume that gets remounted after restart.
-#
-# The trap forwards SIGTERM to all tracked child PIDs so that:
-#   - hindsight-api receives the signal and can run its shutdown hooks
-#   - pg0 gets a clean PostgreSQL shutdown (checkpoint + WAL flush)
-#   - The control-plane Node.js process exits cleanly
-# =============================================================================
-# Guard against concurrent cleanup (e.g., child crash + SIGTERM arriving together)
-SHUTTING_DOWN=false
-
-cleanup() {
-    if $SHUTTING_DOWN; then return; fi
-    SHUTTING_DOWN=true
-
-    echo ""
-    echo "🛑 Received shutdown signal, stopping services gracefully..."
-    for pid in "${PIDS[@]}"; do
-        if kill -0 "$pid" 2>/dev/null; then
-            kill -TERM "$pid" 2>/dev/null
-        fi
-    done
-    # Give processes time to shut down cleanly (pg0 needs to flush WAL).
-    # NOTE: Docker's default stop_grace_period is 10s. If you use the default,
-    # either set stop_grace_period: 30s in your compose file / docker stop -t 30,
-    # or Docker will SIGKILL the container before this timeout expires.
-    local timeout=30
-    for ((i=1; i<=timeout; i++)); do
-        local all_stopped=true
-        for pid in "${PIDS[@]}"; do
-            if kill -0 "$pid" 2>/dev/null; then
-                all_stopped=false
-                break
-            fi
-        done
-        if $all_stopped; then
-            echo "✅ All services stopped cleanly"
-            exit 0
-        fi
-        sleep 1
-    done
-    # Force kill if still running after timeout
-    echo "⚠️  Timeout reached, forcing shutdown..."
-    for pid in "${PIDS[@]}"; do
-        if kill -0 "$pid" 2>/dev/null; then
-            kill -9 "$pid" 2>/dev/null
-        fi
-    done
-    exit 1
-}
-trap cleanup SIGTERM SIGINT
-
 # Track PIDs for wait
 PIDS=()
 
 # Start API if enabled
 if [ "$ENABLE_API" = "true" ]; then
     cd /app/api
-    API_HEALTH_URL="${HINDSIGHT_API_HEALTH_URL:-http://localhost:8888/health}"
-    API_STARTUP_WAIT_SECONDS="${HINDSIGHT_API_STARTUP_WAIT_SECONDS:-300}"
-
     # Run API directly - Python's PYTHONUNBUFFERED=1 handles output buffering
     hindsight-api &
     API_PID=$!
     PIDS+=($API_PID)
 
     # Wait for API to be ready
-    api_ready=false
-    for ((i=1; i<=API_STARTUP_WAIT_SECONDS; i++)); do
-        if ! kill -0 "$API_PID" 2>/dev/null; then
-            wait "$API_PID"
-            exit $?
-        fi
-        if curl -sf "$API_HEALTH_URL" &>/dev/null; then
-            api_ready=true
+    for i in {1..60}; do
+        if curl -sf http://localhost:8888/health &>/dev/null; then
             break
         fi
         sleep 1
     done
-
-    if [ "$api_ready" != "true" ]; then
-        echo "❌ API did not become healthy within ${API_STARTUP_WAIT_SECONDS}s"
-        exit 1
-    fi
 else
     echo "API disabled (HINDSIGHT_ENABLE_API=false)"
 fi
@@ -190,8 +97,7 @@ fi
 if [ "$ENABLE_CP" = "true" ]; then
     echo "🎛️  Starting Control Plane..."
     cd /app/control-plane
-    export HOSTNAME="${HINDSIGHT_CP_HOSTNAME:-0.0.0.0}"
-    PORT="${HINDSIGHT_CP_PORT:-9999}" node server.js &
+    PORT=9999 node server.js &
     CP_PID=$!
     PIDS+=($CP_PID)
 else
@@ -204,7 +110,7 @@ echo "✅ Hindsight is running!"
 echo ""
 echo "📍 Access:"
 if [ "$ENABLE_CP" = "true" ]; then
-    echo "   Control Plane: http://localhost:${HINDSIGHT_CP_PORT:-9999}"
+    echo "   Control Plane: http://localhost:9999"
 fi
 if [ "$ENABLE_API" = "true" ]; then
     echo "   API:           http://localhost:8888"
@@ -217,21 +123,8 @@ if [ ${#PIDS[@]} -eq 0 ]; then
     exit 1
 fi
 
-# Wait for any process to exit (use wait -n with trap-safe loop)
-while true; do
-    # wait -n returns when any child exits; it also returns on signal delivery
-    # (the trap handler will run and exit, so this loop is just for robustness).
-    # `&& true` prevents `set -e` from killing the script when wait -n returns
-    # non-zero (child exited with error or no backgrounded children remain).
-    wait -n && true
-    # Check if any tracked PID has exited
-    for pid in "${PIDS[@]}"; do
-        if ! kill -0 "$pid" 2>/dev/null; then
-            wait "$pid" 2>/dev/null
-            exit_code=$?
-            echo "⚠️  Service (PID $pid) exited with code $exit_code"
-            # Trigger cleanup for remaining services
-            cleanup
-        fi
-    done
-done
+# Wait for any process to exit
+wait -n
+
+# Exit with status of first exited process
+exit $?

docker/test-image.sh

@@ -13,9 +13,9 @@
 #   target  - Optional: 'cp-only' for control plane, otherwise assumes API image (default: api)
 #
 # Environment variables:
-#   HINDSIGHT_API_LLM_API_KEY                   - Required for API/standalone images (LLM verification)
-#   HINDSIGHT_API_LLM_PROVIDER                  - LLM provider (default: openai)
-#   HINDSIGHT_API_LLM_MODEL                     - LLM model (default: gpt-4o-mini)
+#   GROQ_API_KEY                                - Required for API/standalone images (LLM verification)
+#   HINDSIGHT_API_LLM_PROVIDER                  - LLM provider (default: groq)
+#   HINDSIGHT_API_LLM_MODEL                     - LLM model (default: llama-3.3-70b-versatile)
 #   HINDSIGHT_API_EMBEDDINGS_PROVIDER           - Embeddings provider (optional, for slim images: openai, cohere, tei)
 #   HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY     - OpenAI API key for embeddings (optional)
 #   HINDSIGHT_API_RERANKER_PROVIDER             - Reranker provider (optional, for slim images: cohere, tei)
@@ -34,7 +34,7 @@
 #   ./docker/test-image.sh hindsight-control-plane:test cp-only
 #
 #   # Test slim image with external providers
-#   export HINDSIGHT_API_LLM_API_KEY=sk_xxx
+#   export GROQ_API_KEY=gsk_xxx
 #   export HINDSIGHT_API_EMBEDDINGS_PROVIDER=openai
 #   export HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=sk-xxx
 #   export HINDSIGHT_API_RERANKER_PROVIDER=cohere
@@ -49,9 +49,6 @@
 
 set -euo pipefail
 
-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-REPO_ROOT="$(dirname "$SCRIPT_DIR")"
-
 # Colors for output
 RED='\033[0;31m'
 GREEN='\033[0;32m'
@@ -63,8 +60,8 @@ IMAGE="${1:-}"
 TARGET="${2:-api}"
 TIMEOUT="${SMOKE_TEST_TIMEOUT:-120}"
 CONTAINER_NAME="${SMOKE_TEST_CONTAINER_NAME:-hindsight-smoke-test}"
-LLM_PROVIDER="${HINDSIGHT_API_LLM_PROVIDER:-openai}"
-LLM_MODEL="${HINDSIGHT_API_LLM_MODEL:-gpt-4o-mini}"
+LLM_PROVIDER="${HINDSIGHT_API_LLM_PROVIDER:-groq}"
+LLM_MODEL="${HINDSIGHT_API_LLM_MODEL:-llama-3.3-70b-versatile}"
 
 # Validate arguments
 if [ -z "$IMAGE" ]; then
@@ -91,9 +88,9 @@ else
 fi
 
 # Check for required environment variables
-if [ "$NEEDS_LLM" = true ] && [ "$LLM_PROVIDER" != "vertexai" ] && [ -z "${HINDSIGHT_API_LLM_API_KEY:-}" ]; then
-    echo -e "${RED}Error: HINDSIGHT_API_LLM_API_KEY environment variable is required for API/standalone images${NC}"
-    echo "Set it with: export HINDSIGHT_API_LLM_API_KEY=your-api-key"
+if [ "$NEEDS_LLM" = true ] && [ -z "${GROQ_API_KEY:-}" ]; then
+    echo -e "${RED}Error: GROQ_API_KEY environment variable is required for API/standalone images${NC}"
+    echo "Set it with: export GROQ_API_KEY=your-api-key"
     exit 2
 fi
 
@@ -126,25 +123,9 @@ else
     # Build docker run command with required and optional env vars
     DOCKER_CMD="docker run -d --name $CONTAINER_NAME"
     DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_LLM_PROVIDER=$LLM_PROVIDER"
-    if [ -n "${HINDSIGHT_API_LLM_API_KEY:-}" ]; then
-        DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_LLM_API_KEY=${HINDSIGHT_API_LLM_API_KEY}"
-    fi
+    DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_LLM_API_KEY=${GROQ_API_KEY}"
     DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_LLM_MODEL=$LLM_MODEL"
 
-    # Add Vertex AI config if provider is vertexai
-    if [ "$LLM_PROVIDER" = "vertexai" ]; then
-        if [ -n "${HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY:-}" ]; then
-            DOCKER_CMD="$DOCKER_CMD -v ${HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY}:/tmp/gcp-credentials.json:ro"
-            DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY=/tmp/gcp-credentials.json"
-        fi
-        if [ -n "${HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID:-}" ]; then
-            DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=${HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID}"
-        fi
-        if [ -n "${HINDSIGHT_API_LLM_VERTEXAI_REGION:-}" ]; then
-            DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_LLM_VERTEXAI_REGION=${HINDSIGHT_API_LLM_VERTEXAI_REGION}"
-        fi
-    fi
-
     # Add optional embeddings provider config
     if [ -n "${HINDSIGHT_API_EMBEDDINGS_PROVIDER:-}" ]; then
         DOCKER_CMD="$DOCKER_CMD -e HINDSIGHT_API_EMBEDDINGS_PROVIDER=${HINDSIGHT_API_EMBEDDINGS_PROVIDER}"
@@ -181,21 +162,6 @@ for i in $(seq 1 "$TIMEOUT"); do
         echo "=== Health Response ==="
         curl -s "http://localhost:${HEALTH_PORT}${HEALTH_PATH}" | python3 -m json.tool 2>/dev/null || curl -s "http://localhost:${HEALTH_PORT}${HEALTH_PATH}"
         echo ""
-
-        # Run retain/recall smoke test for API targets
-        if [ "$TARGET" != "cp-only" ]; then
-            echo ""
-            echo "=== Retain/Recall Smoke Test ==="
-            if ! "$REPO_ROOT/scripts/smoke-test-slim.sh" "http://localhost:${HEALTH_PORT}"; then
-                echo ""
-                echo "=== Container Logs (last 50 lines) ==="
-                docker logs "$CONTAINER_NAME" 2>&1 | tail -50
-                echo ""
-                echo -e "${RED}Smoke test FAILED${NC}"
-                exit 1
-            fi
-        fi
-
         echo ""
         echo "=== Container Logs (last 50 lines) ==="
         docker logs "$CONTAINER_NAME" 2>&1 | tail -50

docker/test-slim-local.sh

@@ -6,17 +6,24 @@
 # It expects API keys to be set in environment variables.
 #
 # Usage:
+#   export GROQ_API_KEY=gsk_xxx
 #   export OPENAI_API_KEY=sk-xxx
 #   export COHERE_API_KEY=xxx
 #   ./docker/test-slim-local.sh
 #
 # Or inline:
-#   OPENAI_API_KEY=sk_xxx COHERE_API_KEY=xxx ./docker/test-slim-local.sh
+#   GROQ_API_KEY=gsk_xxx OPENAI_API_KEY=sk_xxx COHERE_API_KEY=xxx ./docker/test-slim-local.sh
 #
 
 set -euo pipefail
 
 # Check for required API keys
+if [ -z "${GROQ_API_KEY:-}" ]; then
+    echo "❌ Error: GROQ_API_KEY environment variable is required"
+    echo "Set it with: export GROQ_API_KEY=gsk_xxx"
+    exit 1
+fi
+
 if [ -z "${OPENAI_API_KEY:-}" ]; then
     echo "❌ Error: OPENAI_API_KEY environment variable is required"
     echo "Set it with: export OPENAI_API_KEY=sk-xxx"
@@ -34,10 +41,7 @@ IMAGE="${1:-hindsight-slim:test}"
 echo "Testing image: $IMAGE"
 echo ""
 
-# Set up LLM and external providers
-export HINDSIGHT_API_LLM_PROVIDER=openai
-export HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY
-export HINDSIGHT_API_LLM_MODEL=gpt-4o-mini
+# Set up external providers
 export HINDSIGHT_API_EMBEDDINGS_PROVIDER=openai
 export HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=$OPENAI_API_KEY
 export HINDSIGHT_API_RERANKER_PROVIDER=cohere

helm/hindsight/Chart.yaml

@@ -2,8 +2,8 @@ apiVersion: v2
 name: hindsight
 description: Hindsight helm chart
 type: application
-version: 0.4.22
-appVersion: "0.4.22"
+version: 0.4.11
+appVersion: "0.4.11"
 keywords:
   - ai
   - memory

helm/hindsight/templates/api-deployment.yaml

@@ -95,27 +95,6 @@ spec:
           {{- toYaml .Values.api.readinessProbe | nindent 10 }}
         resources:
           {{- toYaml .Values.api.resources | nindent 10 }}
-        {{- if or .Values.api.persistence.modelCache.enabled .Values.api.extraVolumeMounts }}
-        volumeMounts:
-        {{- if .Values.api.persistence.modelCache.enabled }}
-        - name: model-cache
-          mountPath: /home/hindsight/.cache
-        {{- end }}
-        {{- with .Values.api.extraVolumeMounts }}
-        {{- toYaml . | nindent 8 }}
-        {{- end }}
-        {{- end }}
-      {{- if or .Values.api.persistence.modelCache.enabled .Values.api.extraVolumes }}
-      volumes:
-      {{- if .Values.api.persistence.modelCache.enabled }}
-      - name: model-cache
-        persistentVolumeClaim:
-          claimName: {{ include "hindsight.fullname" . }}-api-model-cache
-      {{- end }}
-      {{- with .Values.api.extraVolumes }}
-      {{- toYaml . | nindent 6 }}
-      {{- end }}
-      {{- end }}
       {{- with .Values.nodeSelector }}
       nodeSelector:
         {{- toYaml . | nindent 8 }}

helm/hindsight/templates/api-model-cache-pvc.yaml

@@ -1,21 +0,0 @@
-{{- if and .Values.api.enabled .Values.api.persistence.modelCache.enabled }}
-apiVersion: v1
-kind: PersistentVolumeClaim
-metadata:
-  name: {{ include "hindsight.fullname" . }}-api-model-cache
-  labels:
-    {{- include "hindsight.api.labels" . | nindent 4 }}
-  {{- with .Values.api.persistence.modelCache.annotations }}
-  annotations:
-    {{- toYaml . | nindent 4 }}
-  {{- end }}
-spec:
-  accessModes:
-    {{- toYaml .Values.api.persistence.modelCache.accessModes | nindent 4 }}
-  {{- if .Values.api.persistence.modelCache.storageClass }}
-  storageClassName: {{ .Values.api.persistence.modelCache.storageClass }}
-  {{- end }}
-  resources:
-    requests:
-      storage: {{ .Values.api.persistence.modelCache.size }}
-{{- end }}

helm/hindsight/templates/worker-statefulset.yaml

@@ -95,16 +95,6 @@ spec:
           {{- toYaml .Values.worker.readinessProbe | nindent 10 }}
         resources:
           {{- toYaml .Values.worker.resources | nindent 10 }}
-        {{- if or .Values.worker.persistence.modelCache.enabled .Values.worker.extraVolumeMounts }}
-        volumeMounts:
-        {{- if .Values.worker.persistence.modelCache.enabled }}
-        - name: model-cache
-          mountPath: /home/hindsight/.cache
-        {{- end }}
-        {{- with .Values.worker.extraVolumeMounts }}
-        {{- toYaml . | nindent 8 }}
-        {{- end }}
-        {{- end }}
       {{- with .Values.nodeSelector }}
       nodeSelector:
         {{- toYaml . | nindent 8 }}
@@ -117,26 +107,4 @@ spec:
       tolerations:
         {{- toYaml . | nindent 8 }}
       {{- end }}
-      {{- with .Values.worker.extraVolumes }}
-      volumes:
-      {{- toYaml . | nindent 6 }}
-      {{- end }}
-  {{- if .Values.worker.persistence.modelCache.enabled }}
-  volumeClaimTemplates:
-  - metadata:
-      name: model-cache
-      {{- with .Values.worker.persistence.modelCache.annotations }}
-      annotations:
-        {{- toYaml . | nindent 8 }}
-      {{- end }}
-    spec:
-      accessModes:
-        {{- toYaml .Values.worker.persistence.modelCache.accessModes | nindent 8 }}
-      {{- if .Values.worker.persistence.modelCache.storageClass }}
-      storageClassName: {{ .Values.worker.persistence.modelCache.storageClass }}
-      {{- end }}
-      resources:
-        requests:
-          storage: {{ .Values.worker.persistence.modelCache.size }}
-  {{- end }}
 {{- end }}

helm/hindsight/values.yaml

@@ -67,33 +67,6 @@ api:
   # Pod affinity/anti-affinity (overrides global affinity for this component)
   # affinity: {}
 
-  # Persistent volume for local model cache (reranker, embeddings)
-  # Models are downloaded to /home/hindsight/.cache on first use.
-  # Without persistence, models are re-downloaded on every pod restart.
-  persistence:
-    modelCache:
-      enabled: false
-      size: 5Gi
-      storageClass: ""
-      accessModes:
-        - ReadWriteOnce
-      annotations: {}
-
-  # Extra volume mounts for the api container
-  # e.g.
-  # extraVolumeMounts:
-  #   - name: my-volume
-  #     mountPath: /mnt/my-volume
-  extraVolumeMounts: []
-
-  # Extra volumes for the api pod
-  # e.g.
-  # extraVolumes:
-  #   - name: my-volume
-  #     configMap:
-  #       name: my-configmap
-  extraVolumes: []
-
   # Environment variables
   env:
     #HINDSIGHT_API_LLM_PROVIDER: "groq"
@@ -167,32 +140,6 @@ worker:
   # Pod affinity/anti-affinity (overrides global affinity for this component)
   # affinity: {}
 
-  # Persistent volume for local model cache (reranker, embeddings)
-  # Uses volumeClaimTemplates since worker is a StatefulSet.
-  persistence:
-    modelCache:
-      enabled: false
-      size: 5Gi
-      storageClass: ""
-      accessModes:
-        - ReadWriteOnce
-      annotations: {}
-
-  # Extra volume mounts for the worker container
-  # e.g.
-  # extraVolumeMounts:
-  #   - name: my-volume
-  #     mountPath: /mnt/my-volume
-  extraVolumeMounts: []
-
-  # Extra volumes for the worker pod
-  # e.g.
-  # extraVolumes:
-  #   - name: my-volume
-  #     configMap:
-  #       name: my-configmap
-  extraVolumes: []
-
   # Secret environment variables (inherited from api.secrets if not specified)
   secrets: {}
 

hindsight-all-slim/pyproject.toml

@@ -1,33 +0,0 @@
-[build-system]
-requires = ["setuptools>=61"]
-build-backend = "setuptools.build_meta"
-
-[project]
-name = "hindsight-all-slim"
-version = "0.4.22"
-description = "Hindsight: Agent Memory That Works Like Human Memory - Slim All-in-One Bundle"
-readme = "README.md"
-requires-python = ">=3.11"
-dependencies = [
-    "hindsight-api-slim>=0.4.17",
-    "hindsight-client>=0.0.7",
-    "hindsight-embed>=0.1.0",
-]
-
-[tool.uv.sources]
-hindsight-api-slim = { workspace = true }
-hindsight-client = { workspace = true }
-hindsight-embed = { workspace = true }
-
-[project.optional-dependencies]
-test = [
-    "pytest>=7.0.0",
-    "pytest-asyncio>=0.21.0",
-]
-
-[tool.setuptools]
-packages = []
-
-[tool.pytest.ini_options]
-asyncio_mode = "auto"
-asyncio_default_fixture_loop_scope = "function"

hindsight-all/README.md

@@ -1,48 +0,0 @@
-# hindsight-all
-
-All-in-one package for Hindsight - Agent Memory That Works Like Human Memory
-
-## Quick Start
-
-```python
-from hindsight import start_server, HindsightClient
-
-# Start server with embedded PostgreSQL
-server = start_server(
-    llm_provider="groq",
-    llm_api_key="your-api-key",
-    llm_model="openai/gpt-oss-120b"
-)
-
-# Create client
-client = HindsightClient(base_url=server.url)
-
-# Store memories
-client.put(agent_id="assistant", content="User prefers Python for data analysis")
-
-# Search memories
-results = client.search(agent_id="assistant", query="programming preferences")
-
-# Generate contextual response
-response = client.think(agent_id="assistant", query="What languages should I recommend?")
-
-# Stop server when done
-server.stop()
-```
-
-## Using Context Manager
-
-```python
-from hindsight import HindsightServer, HindsightClient
-
-with HindsightServer(llm_provider="groq", llm_api_key="...") as server:
-    client = HindsightClient(base_url=server.url)
-    # ... use client ...
-# Server automatically stops
-```
-
-## Installation
-
-```bash
-pip install hindsight-all
-```

hindsight-all/hindsight/client_wrapper.py

@@ -1,423 +0,0 @@
-"""
-Wrapper for Hindsight client that adds API namespaces.
-
-Provides organized access to different parts of the Hindsight API through
-namespaces like .banks, .mental_models, etc.
-"""
-
-from __future__ import annotations
-
-from typing import Any
-
-from hindsight_client import Hindsight
-
-
-class BanksAPI:
-    """Namespace for bank-related operations.
-    
-    Provides methods to create, delete, and manage memory banks.
-    """
-
-    def __init__(self, client: Hindsight):
-        self._client = client
-
-    def create(
-        self,
-        bank_id: str,
-        name: str | None = None,
-        mission: str | None = None,
-        disposition: dict[str, Any] | None = None,
-    ) -> Any:
-        """Create a new bank.
-        
-        Args:
-            bank_id: Unique identifier for the bank.
-            name: Optional display name for the bank.
-            mission: Optional mission statement for the bank.
-            disposition: Optional disposition configuration dict.
-            
-        Returns:
-            Bank creation response from the API.
-        """
-        return self._client.create_bank(
-            bank_id=bank_id,
-            name=name,
-            mission=mission,
-            disposition=disposition,
-        )
-
-    def delete(self, bank_id: str) -> Any:
-        """Delete a bank.
-        
-        Args:
-            bank_id: The ID of the bank to delete.
-            
-        Returns:
-            Deletion response from the API.
-        """
-        return self._client.delete_bank(bank_id=bank_id)
-
-    def set_mission(self, bank_id: str, mission: str) -> Any:
-        """Set or update the mission for a bank.
-        
-        Args:
-            bank_id: The ID of the bank.
-            mission: The mission statement to set.
-            
-        Returns:
-            API response confirming the update.
-        """
-        return self._client.set_mission(bank_id=bank_id, mission=mission)
-
-    def set_disposition(self, bank_id: str, disposition: dict[str, Any]) -> Any:
-        """Set or update the disposition for a bank.
-        
-        Args:
-            bank_id: The ID of the bank.
-            disposition: The disposition configuration dict.
-            
-        Returns:
-            API response confirming the update.
-        """
-        return self._client.set_disposition(bank_id=bank_id, disposition=disposition)
-
-    def list(self) -> Any:
-        """List all banks.
-        
-        Returns:
-            List of banks from the API.
-        """
-        from hindsight_client.hindsight_client import _run_async
-
-        return _run_async(self._client._banks_api.list_banks())
-
-
-class MentalModelsAPI:
-    """Namespace for mental model operations.
-    
-    Mental models are reusable knowledge structures that guide agent behavior.
-    """
-
-    def __init__(self, client: Hindsight):
-        self._client = client
-
-    def create(
-        self,
-        bank_id: str,
-        name: str,
-        content: str,
-        tags: list[str] | None = None,
-    ) -> Any:
-        """Create a new mental model.
-        
-        Args:
-            bank_id: The ID of the bank to add the model to.
-            name: Name for the mental model.
-            content: The content/instructions for the mental model.
-            tags: Optional list of tags for categorization.
-            
-        Returns:
-            Creation response from the API.
-        """
-        return self._client.create_mental_model(
-            bank_id=bank_id,
-            name=name,
-            content=content,
-            tags=tags,
-        )
-
-    def list(self, bank_id: str, tags: list[str] | None = None) -> Any:
-        """List all mental models for a bank.
-        
-        Args:
-            bank_id: The ID of the bank.
-            tags: Optional filter by tags.
-            
-        Returns:
-            List of mental models.
-        """
-        return self._client.list_mental_models(bank_id=bank_id, tags=tags)
-
-    def get(self, bank_id: str, mental_model_id: str) -> Any:
-        """Get a specific mental model.
-        
-        Args:
-            bank_id: The ID of the bank.
-            mental_model_id: The ID of the mental model.
-            
-        Returns:
-            The mental model details.
-        """
-        return self._client.get_mental_model(bank_id=bank_id, mental_model_id=mental_model_id)
-
-    def refresh(self, bank_id: str, mental_model_id: str) -> Any:
-        """Refresh a mental model.
-        
-        Args:
-            bank_id: The ID of the bank.
-            mental_model_id: The ID of the mental model to refresh.
-            
-        Returns:
-            Refresh response from the API.
-        """
-        return self._client.refresh_mental_model(bank_id=bank_id, mental_model_id=mental_model_id)
-
-    def update(
-        self,
-        bank_id: str,
-        mental_model_id: str,
-        name: str | None = None,
-        content: str | None = None,
-        tags: list[str] | None = None,
-    ) -> Any:
-        """Update a mental model.
-        
-        Args:
-            bank_id: The ID of the bank.
-            mental_model_id: The ID of the mental model to update.
-            name: Optional new name.
-            content: Optional new content.
-            tags: Optional new tags list.
-            
-        Returns:
-            Update response from the API.
-        """
-        return self._client.update_mental_model(
-            bank_id=bank_id,
-            mental_model_id=mental_model_id,
-            name=name,
-            content=content,
-            tags=tags,
-        )
-
-    def delete(self, bank_id: str, mental_model_id: str) -> Any:
-        """Delete a mental model.
-        
-        Args:
-            bank_id: The ID of the bank.
-            mental_model_id: The ID of the mental model to delete.
-            
-        Returns:
-            Deletion response from the API.
-        """
-        return self._client.delete_mental_model(bank_id=bank_id, mental_model_id=mental_model_id)
-
-
-class DirectivesAPI:
-    """Namespace for directive operations.
-    
-    Directives are explicit instructions that guide agent behavior.
-    """
-
-    def __init__(self, client: Hindsight):
-        self._client = client
-
-    def create(
-        self,
-        bank_id: str,
-        name: str,
-        content: str,
-        tags: list[str] | None = None,
-    ) -> Any:
-        """Create a new directive.
-        
-        Args:
-            bank_id: The ID of the bank to add the directive to.
-            name: Name for the directive.
-            content: The directive content/instructions.
-            tags: Optional list of tags for categorization.
-            
-        Returns:
-            Creation response from the API.
-        """
-        return self._client.create_directive(
-            bank_id=bank_id,
-            name=name,
-            content=content,
-            tags=tags,
-        )
-
-    def list(self, bank_id: str, tags: list[str] | None = None) -> Any:
-        """List all directives for a bank.
-        
-        Args:
-            bank_id: The ID of the bank.
-            tags: Optional filter by tags.
-            
-        Returns:
-            List of directives.
-        """
-        return self._client.list_directives(bank_id=bank_id, tags=tags)
-
-    def get(self, bank_id: str, directive_id: str) -> Any:
-        """Get a specific directive.
-        
-        Args:
-            bank_id: The ID of the bank.
-            directive_id: The ID of the directive.
-            
-        Returns:
-            The directive details.
-        """
-        return self._client.get_directive(bank_id=bank_id, directive_id=directive_id)
-
-    def update(
-        self,
-        bank_id: str,
-        directive_id: str,
-        name: str | None = None,
-        content: str | None = None,
-        tags: list[str] | None = None,
-    ) -> Any:
-        """Update a directive.
-        
-        Args:
-            bank_id: The ID of the bank.
-            directive_id: The ID of the directive to update.
-            name: Optional new name.
-            content: Optional new content.
-            tags: Optional new tags list.
-            
-        Returns:
-            Update response from the API.
-        """
-        return self._client.update_directive(
-            bank_id=bank_id,
-            directive_id=directive_id,
-            name=name,
-            content=content,
-            tags=tags,
-        )
-
-    def delete(self, bank_id: str, directive_id: str) -> Any:
-        """Delete a directive.
-        
-        Args:
-            bank_id: The ID of the bank.
-            directive_id: The ID of the directive to delete.
-            
-        Returns:
-            Deletion response from the API.
-        """
-        return self._client.delete_directive(bank_id=bank_id, directive_id=directive_id)
-
-
-class MemoriesAPI:
-    """Namespace for memory operations.
-    
-    Provides methods to query and retrieve stored memories.
-    """
-
-    def __init__(self, client: Hindsight):
-        self._client = client
-
-    def list(
-        self,
-        bank_id: str,
-        type: str | None = None,
-        search_query: str | None = None,
-        limit: int = 100,
-        offset: int = 0,
-    ) -> Any:
-        """List memories in a bank.
-        
-        Args:
-            bank_id: The ID of the bank to query.
-            type: Optional filter by memory type.
-            search_query: Optional search query for filtering.
-            limit: Maximum number of results to return (default: 100).
-            offset: Number of results to skip for pagination (default: 0).
-            
-        Returns:
-            List of memories matching the criteria.
-        """
-        return self._client.list_memories(
-            bank_id=bank_id,
-            type=type,
-            search_query=search_query,
-            limit=limit,
-            offset=offset,
-        )
-
-
-class HindsightClient(Hindsight):
-    """
-    Enhanced Hindsight client with organized API namespaces.
-
-    This wrapper extends the auto-generated Hindsight client with organized
-    access to different parts of the API through namespaces.
-
-    Example:
-        ```python
-        from hindsight import HindsightClient
-
-        client = HindsightClient(base_url="http://localhost:8888")
-
-        # Core operations (inherited from Hindsight)
-        client.retain(bank_id="test", content="Hello")
-        results = client.recall(bank_id="test", query="Hello")
-
-        # Organized API access through namespaces
-        client.banks.create(bank_id="test", name="Test Bank")
-        models = client.mental_models.list(bank_id="test")
-        directives = client.directives.list(bank_id="test")
-        memories = client.memories.list(bank_id="test")
-        ```
-        
-    Attributes:
-        banks: Namespace for bank management operations.
-        mental_models: Namespace for mental model operations.
-        directives: Namespace for directive operations.
-        memories: Namespace for memory listing operations.
-    """
-
-    def __init__(self, *args: Any, **kwargs: Any) -> None:
-        super().__init__(*args, **kwargs)
-        self._banks_namespace: BanksAPI | None = None
-        self._mental_models_namespace: MentalModelsAPI | None = None
-        self._directives_namespace: DirectivesAPI | None = None
-        self._memories_namespace: MemoriesAPI | None = None
-
-    @property
-    def banks(self) -> BanksAPI:
-        """Access bank management operations.
-        
-        Returns:
-            BanksAPI instance for bank operations.
-        """
-        if self._banks_namespace is None:
-            self._banks_namespace = BanksAPI(self)
-        return self._banks_namespace
-
-    @property
-    def mental_models(self) -> MentalModelsAPI:
-        """Access mental model operations.
-        
-        Returns:
-            MentalModelsAPI instance for mental model operations.
-        """
-        if self._mental_models_namespace is None:
-            self._mental_models_namespace = MentalModelsAPI(self)
-        return self._mental_models_namespace
-
-    @property
-    def directives(self) -> DirectivesAPI:
-        """Access directive operations.
-        
-        Returns:
-            DirectivesAPI instance for directive operations.
-        """
-        if self._directives_namespace is None:
-            self._directives_namespace = DirectivesAPI(self)
-        return self._directives_namespace
-
-    @property
-    def memories(self) -> MemoriesAPI:
-        """Access memory listing operations.
-        
-        Returns:
-            MemoriesAPI instance for memory operations.
-        """
-        if self._memories_namespace is None:
-            self._memories_namespace = MemoriesAPI(self)
-        return self._memories_namespace

hindsight-api-slim/README.md

@@ -1,137 +0,0 @@
-# Hindsight API
-
-**Memory System for AI Agents** — Temporal + Semantic + Entity Memory Architecture using PostgreSQL with pgvector.
-
-Hindsight gives AI agents persistent memory that works like human memory: it stores facts, tracks entities and relationships, handles temporal reasoning ("what happened last spring?"), and forms opinions based on configurable disposition traits.
-
-## Installation
-
-```bash
-pip install hindsight-api
-```
-
-## Quick Start
-
-### Run the Server
-
-```bash
-# Set your LLM provider
-export HINDSIGHT_API_LLM_PROVIDER=openai
-export HINDSIGHT_API_LLM_API_KEY=sk-xxxxxxxxxxxx
-
-# Start the server (uses embedded PostgreSQL by default)
-hindsight-api
-```
-
-The server starts at http://localhost:8888 with:
-- REST API for memory operations
-- MCP server at `/mcp` for tool-use integration
-
-### Use the Python API
-
-```python
-from hindsight_api import MemoryEngine
-
-# Create and initialize the memory engine
-memory = MemoryEngine()
-await memory.initialize()
-
-# Create a memory bank for your agent
-bank = await memory.create_memory_bank(
-    name="my-assistant",
-    background="A helpful coding assistant"
-)
-
-# Store a memory
-await memory.retain(
-    memory_bank_id=bank.id,
-    content="The user prefers Python for data science projects"
-)
-
-# Recall memories
-results = await memory.recall(
-    memory_bank_id=bank.id,
-    query="What programming language does the user prefer?"
-)
-
-# Reflect with reasoning
-response = await memory.reflect(
-    memory_bank_id=bank.id,
-    query="Should I recommend Python or R for this ML project?"
-)
-```
-
-## CLI Options
-
-```bash
-hindsight-api --help
-
-# Common options
-hindsight-api --port 9000          # Custom port (default: 8888)
-hindsight-api --host 127.0.0.1     # Bind to localhost only
-hindsight-api --workers 4          # Multiple worker processes
-hindsight-api --log-level debug    # Verbose logging
-```
-
-## Configuration
-
-Configure via environment variables:
-
-| Variable | Description | Default |
-|----------|-------------|---------|
-| `HINDSIGHT_API_DATABASE_URL` | PostgreSQL connection string | `pg0` (embedded) |
-| `HINDSIGHT_API_LLM_PROVIDER` | `openai`, `anthropic`, `gemini`, `groq`, `ollama`, `lmstudio` | `openai` |
-| `HINDSIGHT_API_LLM_API_KEY` | API key for LLM provider | - |
-| `HINDSIGHT_API_LLM_MODEL` | Model name | `gpt-4o-mini` |
-| `HINDSIGHT_API_HOST` | Server bind address | `0.0.0.0` |
-| `HINDSIGHT_API_PORT` | Server port | `8888` |
-
-### Example with External PostgreSQL
-
-```bash
-export HINDSIGHT_API_DATABASE_URL=postgresql://user:pass@localhost:5432/hindsight
-export HINDSIGHT_API_LLM_PROVIDER=groq
-export HINDSIGHT_API_LLM_API_KEY=gsk_xxxxxxxxxxxx
-
-hindsight-api
-```
-
-## Docker
-
-```bash
-docker run --rm -it -p 8888:8888 \
-  -e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \
-  -v $HOME/.hindsight-docker:/home/hindsight/.pg0 \
-  ghcr.io/vectorize-io/hindsight:latest
-```
-
-## MCP Server
-
-For local MCP integration without running the full API server:
-
-```bash
-hindsight-local-mcp
-```
-
-This runs a stdio-based MCP server that can be used directly with MCP-compatible clients.
-
-## Key Features
-
-- **Multi-Strategy Retrieval (TEMPR)** — Semantic, keyword, graph, and temporal search combined with RRF fusion
-- **Entity Graph** — Automatic entity extraction and relationship tracking
-- **Temporal Reasoning** — Native support for time-based queries
-- **Disposition Traits** — Configurable skepticism, literalism, and empathy influence opinion formation
-- **Three Memory Types** — World facts, bank actions, and formed opinions with confidence scores
-
-## Documentation
-
-Full documentation: [https://hindsight.vectorize.io](https://hindsight.vectorize.io)
-
-- [Installation Guide](https://hindsight.vectorize.io/developer/installation)
-- [Configuration Reference](https://hindsight.vectorize.io/developer/configuration)
-- [API Reference](https://hindsight.vectorize.io/api-reference)
-- [Python SDK](https://hindsight.vectorize.io/sdks/python)
-
-## License
-
-Apache 2.0

hindsight-api-slim/hindsight_api/alembic/versions/2eee35aa3cfc_case_insensitive_entities_trgm_index.py

@@ -1,45 +0,0 @@
-"""Recreate entities trigram index on LOWER(canonical_name) for case-insensitive matching
-
-The previous GIN trigram index on canonical_name was case-sensitive, causing
-"Alice" and "alice" to have different trigram sets. This recreates it on
-LOWER(canonical_name) so the % operator matches case-insensitively.
-
-Revision ID: d6e7f8a9b0c1
-Revises: c5d6e7f8a9b0
-Create Date: 2026-03-31
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "d6e7f8a9b0c1"
-down_revision: str | Sequence[str] | None = "c5d6e7f8a9b0"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-    # Drop the old case-sensitive trigram index
-    op.execute("DROP INDEX IF EXISTS entities_canonical_name_trgm_idx")
-    # Create case-insensitive trigram index on LOWER(canonical_name)
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS entities_canonical_name_lower_trgm_idx "
-        f"ON {schema}entities USING GIN (LOWER(canonical_name) gin_trgm_ops)"
-    )
-
-
-def downgrade() -> None:
-    op.execute("DROP INDEX IF EXISTS entities_canonical_name_lower_trgm_idx")
-    schema = _get_schema_prefix()
-    # Restore original case-sensitive index
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS entities_canonical_name_trgm_idx "
-        f"ON {schema}entities USING GIN (canonical_name gin_trgm_ops)"
-    )

hindsight-api-slim/hindsight_api/alembic/versions/a2b3c4d5e6f7_add_text_signals_column.py

@@ -1,88 +0,0 @@
-"""Add text_signals column to memory_units for enriched BM25 indexing.
-
-text_signals stores a denormalized space-separated string of entity names
-(and future signals) to improve full-text search recall without polluting
-the stored fact text.
-
-- vchord: text_signals included in tokenize() at insert time
-- native: search_vector GENERATED column regenerated to include text_signals
-- pg_textsearch: no change (index only supports a single base column)
-
-Revision ID: a2b3c4d5e6f7
-Revises: z1u2v3w4x5y6
-Create Date: 2026-02-28
-"""
-
-import os
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "a2b3c4d5e6f7"
-down_revision: str | Sequence[str] | None = "aa2b3c4d5e6f"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _detect_text_search_extension() -> str:
-    return os.getenv("HINDSIGHT_API_TEXT_SEARCH_EXTENSION", "native").lower()
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-    table = f"{schema}memory_units"
-    text_search_ext = _detect_text_search_extension()
-
-    # Add text_signals column (nullable TEXT, populated at retain time)
-    op.execute(f"ALTER TABLE {table} ADD COLUMN IF NOT EXISTS text_signals TEXT")
-
-    if text_search_ext == "native":
-        # Native PostgreSQL: drop and recreate the GENERATED tsvector column to include text_signals
-        op.execute(f"ALTER TABLE {table} DROP COLUMN IF EXISTS search_vector")
-        op.execute(f"""
-            ALTER TABLE {table}
-            ADD COLUMN search_vector tsvector
-            GENERATED ALWAYS AS (
-                to_tsvector('english',
-                    COALESCE(text, '') || ' ' ||
-                    COALESCE(context, '') || ' ' ||
-                    COALESCE(text_signals, '')
-                )
-            ) STORED
-        """)
-        # Recreate GIN index (was dropped with the column)
-        op.execute(f"""
-            CREATE INDEX IF NOT EXISTS idx_memory_units_text_search
-            ON {table} USING gin(search_vector)
-        """)
-
-    # vchord: tokenize() call in fact_storage.py is updated to include text_signals at insert time
-    # pg_textsearch: no change — index operates on the base `text` column only
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-    table = f"{schema}memory_units"
-    text_search_ext = _detect_text_search_extension()
-
-    if text_search_ext == "native":
-        op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_units_text_search")
-        op.execute(f"ALTER TABLE {table} DROP COLUMN IF EXISTS search_vector")
-        op.execute(f"""
-            ALTER TABLE {table}
-            ADD COLUMN search_vector tsvector
-            GENERATED ALWAYS AS (
-                to_tsvector('english', COALESCE(text, '') || ' ' || COALESCE(context, ''))
-            ) STORED
-        """)
-        op.execute(f"""
-            CREATE INDEX idx_memory_units_text_search
-            ON {table} USING gin(search_vector)
-        """)
-
-    op.execute(f"ALTER TABLE {table} DROP COLUMN IF EXISTS text_signals")

hindsight-api-slim/hindsight_api/alembic/versions/a2b3c4d5e6f8_add_gin_index_source_memory_ids.py

@@ -1,54 +0,0 @@
-"""Add GIN index on source_memory_ids for observation lookup performance
-
-Without this index, queries using the array overlap operator (&&) or array
-containment (@>) on source_memory_ids require a full sequential scan over all
-observation memory_units. At ~77k observations this was measured at 45ms per
-query, becoming a bottleneck during consolidation recall (57-64s timeouts) and
-user recall (18-27s average).
-
-The GIN index reduces these queries to index scans: 45ms → 0.049ms (927x
-speedup). Recall dropped from 18-27s to ~6s, and consolidation recall
-stabilised from timeout to ~15s.
-
-Created with CONCURRENTLY so the migration does not block reads or writes.
-CONCURRENTLY requires running outside a transaction block, so the migration
-emits an explicit COMMIT before the statement and uses IF NOT EXISTS for
-idempotency.
-
-Revision ID: a2b3c4d5e6f8
-Revises: f7g8h9i0j1k2
-Create Date: 2026-03-04
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "a2b3c4d5e6f8"
-down_revision: str | Sequence[str] | None = "f7g8h9i0j1k2"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # CREATE INDEX CONCURRENTLY cannot run inside a transaction block.
-    # Commit the current Alembic transaction first.
-    op.execute("COMMIT")
-    op.execute(
-        f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_units_source_memory_ids "
-        f"ON {schema}memory_units USING GIN (source_memory_ids) "
-        f"WHERE source_memory_ids IS NOT NULL"
-    )
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute("COMMIT")
-    op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_units_source_memory_ids")

hindsight-api-slim/hindsight_api/alembic/versions/a3b4c5d6e7f8_add_consolidation_failed_at_to_memory_units.py

@@ -1,52 +0,0 @@
-"""Add consolidation_failed_at column to memory_units for tracking persistent LLM failures.
-
-When all LLM retries are exhausted on a single-memory batch, the memory is marked
-with consolidation_failed_at instead of consolidated_at, so it is not silently lost
-and can be retried later via the API.
-
-Revision ID: a3b4c5d6e7f8
-Revises: g7h8i9j0k1l2
-Create Date: 2026-03-17
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "a3b4c5d6e7f8"
-down_revision: str | Sequence[str] | None = "g7h8i9j0k1l2"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Get schema prefix for table names (required for multi-tenant support)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    op.execute(
-        f"""
-        ALTER TABLE {schema}memory_units
-        ADD COLUMN IF NOT EXISTS consolidation_failed_at TIMESTAMPTZ DEFAULT NULL
-        """
-    )
-
-    # Index to efficiently query memories that failed consolidation for a given bank
-    op.execute(
-        f"""
-        CREATE INDEX IF NOT EXISTS idx_memory_units_consolidation_failed
-        ON {schema}memory_units (bank_id, consolidation_failed_at)
-        WHERE consolidation_failed_at IS NOT NULL AND fact_type IN ('experience', 'world')
-        """
-    )
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_units_consolidation_failed")
-    op.execute(f"ALTER TABLE {schema}memory_units DROP COLUMN IF EXISTS consolidation_failed_at")

hindsight-api-slim/hindsight_api/alembic/versions/a4b5c6d7e8f9_fix_per_bank_vector_index_type.py

@@ -1,142 +0,0 @@
-"""Fix per-bank vector indexes to match configured extension
-
-Revision ID: a4b5c6d7e8f9
-Revises: d6e7f8a9b0c1
-Create Date: 2026-04-01
-
-Migration d5e6f7a8b9c0 hardcoded HNSW when creating per-bank partial vector
-indexes, ignoring HINDSIGHT_API_VECTOR_EXTENSION. Banks that existed when that
-migration ran got HNSW indexes even when pgvectorscale (DiskANN) or vchord
-was configured.
-
-This migration detects the mismatch and recreates the affected indexes with
-the correct type. Skipped entirely when the configured extension is pgvector
-(the default), since those indexes are already correct.
-"""
-
-import os
-from collections.abc import Sequence
-
-from alembic import context, op
-from sqlalchemy import text
-
-revision: str = "a4b5c6d7e8f9"
-down_revision: str | Sequence[str] | None = "d6e7f8a9b0c1"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-_FACT_TYPES: dict[str, str] = {
-    "world": "worl",
-    "experience": "expr",
-    "observation": "obsv",
-}
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _target_index_type() -> str | None:
-    """Return the target index type, or None if pgvector (no fix needed)."""
-    ext = os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector").lower()
-    if ext == "pgvectorscale":
-        return "diskann"
-    elif ext == "vchord":
-        return "vchordrq"
-    return None
-
-
-def _vector_index_using_clause() -> str:
-    """Return the USING clause based on the configured vector extension."""
-    ext = os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector").lower()
-    if ext == "pgvectorscale":
-        return "USING diskann (embedding vector_cosine_ops) WITH (num_neighbors = 50)"
-    elif ext == "vchord":
-        return "USING vchordrq (embedding vector_l2_ops)"
-    else:
-        return "USING hnsw (embedding vector_cosine_ops)"
-
-
-def upgrade() -> None:
-    target = _target_index_type()
-    if target is None:
-        # pgvector — indexes are already HNSW, nothing to fix
-        return
-
-    bind = op.get_bind()
-    schema_name = context.config.get_main_option("target_schema")
-    schema = _get_schema_prefix()
-    table_ref = f'"{schema_name}".memory_units' if schema_name else "memory_units"
-    banks_ref = f'"{schema_name}".banks' if schema_name else "banks"
-    using_clause = _vector_index_using_clause()
-    pg_schema = schema_name or "public"
-
-    rows = bind.execute(text(f"SELECT bank_id, internal_id FROM {banks_ref}")).fetchall()  # noqa: S608
-    for row in rows:
-        bank_id = row[0]
-        internal_id = str(row[1]).replace("-", "")[:16]
-        escaped_bank_id = bank_id.replace("'", "''")
-        for ft, ft_short in _FACT_TYPES.items():
-            idx_name = f"idx_mu_emb_{ft_short}_{internal_id}"
-
-            # Check if this index exists and what type it is
-            idx_info = bind.execute(
-                text("SELECT indexdef FROM pg_indexes WHERE schemaname = :schema AND indexname = :idx"),
-                {"schema": pg_schema, "idx": idx_name},
-            ).fetchone()
-
-            if idx_info is None:
-                # Index doesn't exist — create it with the correct type
-                bind.execute(
-                    text(
-                        f"CREATE INDEX IF NOT EXISTS {idx_name} "
-                        f"ON {table_ref} {using_clause} "
-                        f"WHERE fact_type = '{ft}' AND bank_id = '{escaped_bank_id}'"
-                    )
-                )
-                continue
-
-            indexdef = idx_info[0].lower()
-            if target in indexdef:
-                # Already the correct type
-                continue
-
-            # Wrong type — drop and recreate
-            bind.execute(text(f"DROP INDEX IF EXISTS {schema}{idx_name}"))
-            bind.execute(
-                text(
-                    f"CREATE INDEX IF NOT EXISTS {idx_name} "
-                    f"ON {table_ref} {using_clause} "
-                    f"WHERE fact_type = '{ft}' AND bank_id = '{escaped_bank_id}'"
-                )
-            )
-
-
-def downgrade() -> None:
-    # Downgrade recreates indexes as HNSW (the original hardcoded behavior)
-    target = _target_index_type()
-    if target is None:
-        return
-
-    bind = op.get_bind()
-    schema_name = context.config.get_main_option("target_schema")
-    schema = _get_schema_prefix()
-    table_ref = f'"{schema_name}".memory_units' if schema_name else "memory_units"
-    banks_ref = f'"{schema_name}".banks' if schema_name else "banks"
-
-    rows = bind.execute(text(f"SELECT bank_id, internal_id FROM {banks_ref}")).fetchall()  # noqa: S608
-    for row in rows:
-        bank_id = row[0]
-        internal_id = str(row[1]).replace("-", "")[:16]
-        escaped_bank_id = bank_id.replace("'", "''")
-        for ft, ft_short in _FACT_TYPES.items():
-            idx_name = f"idx_mu_emb_{ft_short}_{internal_id}"
-            bind.execute(text(f"DROP INDEX IF EXISTS {schema}{idx_name}"))
-            bind.execute(
-                text(
-                    f"CREATE INDEX IF NOT EXISTS {idx_name} "
-                    f"ON {table_ref} USING hnsw (embedding vector_cosine_ops) "
-                    f"WHERE fact_type = '{ft}' AND bank_id = '{escaped_bank_id}'"
-                )
-            )

hindsight-api-slim/hindsight_api/alembic/versions/aa2b3c4d5e6f_nullable_event_date.py

@@ -1,36 +0,0 @@
-"""Make event_date nullable in memory_units to support timestamp-free content
-
-Revision ID: aa2b3c4d5e6f
-Revises: z1u2v3w4x5y6
-Create Date: 2026-03-02
-
-When callers retain content without a timestamp (e.g. fictional documents, static text),
-the event_date column should be allowed to be NULL rather than defaulting to utcnow().
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "aa2b3c4d5e6f"
-down_revision: str | Sequence[str] | None = "z1u2v3w4x5y6"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Get schema prefix for table names (required for multi-tenant support)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"ALTER TABLE {schema}memory_units ALTER COLUMN event_date DROP NOT NULL")
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-    # Backfill NULLs with now() before restoring the NOT NULL constraint
-    op.execute(f"UPDATE {schema}memory_units SET event_date = now() WHERE event_date IS NULL")
-    op.execute(f"ALTER TABLE {schema}memory_units ALTER COLUMN event_date SET NOT NULL")

hindsight-api-slim/hindsight_api/alembic/versions/b3c4d5e6f7a8_add_content_hash_to_chunks.py

@@ -1,32 +0,0 @@
-"""add content_hash to chunks table for delta retain
-
-Revision ID: b3c4d5e6f7a8
-Revises: a3b4c5d6e7f8
-Create Date: 2026-03-25
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "b3c4d5e6f7a8"
-down_revision: str | Sequence[str] | None = "a3b4c5d6e7f8"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Get schema prefix for table names (required for multi-tenant support)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-    # Add content_hash column to chunks table for delta comparison
-    op.execute(f"ALTER TABLE {schema}chunks ADD COLUMN IF NOT EXISTS content_hash TEXT")
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"ALTER TABLE {schema}chunks DROP COLUMN IF EXISTS content_hash")

hindsight-api-slim/hindsight_api/alembic/versions/b3c4d5e6f7g8_add_temporal_date_indexes.py

@@ -1,68 +0,0 @@
-"""Add partial indexes on memory_units temporal date fields for fast temporal retrieval
-
-Revision ID: b3c4d5e6f7g8
-Revises: c1a2b3d4e5f6
-Create Date: 2026-03-02
-
-The temporal retrieval entry-point query filters memory_units by occurred_start,
-occurred_end, and mentioned_at using OR conditions. Without dedicated indexes the
-planner falls back to a sequential scan of all bank rows after applying the
-(bank_id, fact_type) index, then re-checks each date field.
-
-These three partial indexes give the planner bitmap-index scan options for the
-three most common date predicates, dramatically reducing the row set before any
-embedding computation is required.
-
-All indexes are created CONCURRENTLY so the migration does not block writes on
-memory_units during production deployments. CONCURRENTLY requires running outside
-a transaction block; see migrations.py for how this is handled safely.
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "b3c4d5e6f7g8"
-down_revision: str | Sequence[str] | None = "c1a2b3d4e5f6"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-    # Partial index on occurred_start (covers "occurred_start BETWEEN $4 AND $5")
-    op.execute("COMMIT")
-    op.execute(
-        f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_units_bank_occurred_start "
-        f"ON {schema}memory_units(bank_id, fact_type, occurred_start) "
-        f"WHERE occurred_start IS NOT NULL"
-    )
-    # Partial index on occurred_end (covers "occurred_end BETWEEN $4 AND $5")
-    op.execute("COMMIT")
-    op.execute(
-        f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_units_bank_occurred_end "
-        f"ON {schema}memory_units(bank_id, fact_type, occurred_end) "
-        f"WHERE occurred_end IS NOT NULL"
-    )
-    # Partial index on mentioned_at (covers "mentioned_at BETWEEN $4 AND $5")
-    op.execute("COMMIT")
-    op.execute(
-        f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_units_bank_mentioned_at "
-        f"ON {schema}memory_units(bank_id, fact_type, mentioned_at) "
-        f"WHERE mentioned_at IS NOT NULL"
-    )
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute("COMMIT")
-    op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_units_bank_mentioned_at")
-    op.execute("COMMIT")
-    op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_units_bank_occurred_end")
-    op.execute("COMMIT")
-    op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_units_bank_occurred_start")

hindsight-api-slim/hindsight_api/alembic/versions/b4c5d6e7f8a9_backfill_observation_scopes.py

@@ -1,34 +0,0 @@
-"""Backfill observation_scopes column if missing.
-
-This migration ensures observation_scopes exists even on databases that had
-revision z1u2v3w4x5y6 applied when it referred to the old text_signals migration
-(before it was renamed to a2b3c4d5e6f7). The ADD COLUMN IF NOT EXISTS makes this
-a no-op on databases that already have the column.
-
-Revision ID: b4c5d6e7f8a9
-Revises: a2b3c4d5e6f7
-Create Date: 2026-03-02
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "b4c5d6e7f8a9"
-down_revision: str | Sequence[str] | None = "a2b3c4d5e6f7"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"ALTER TABLE {schema}memory_units ADD COLUMN IF NOT EXISTS observation_scopes JSONB")
-
-
-def downgrade() -> None:
-    pass  # intentionally no-op — safe to leave the column in place

hindsight-api-slim/hindsight_api/alembic/versions/c1a2b3d4e5f6_enable_pg_trgm_and_entities_trgm_index.py

@@ -1,59 +0,0 @@
-"""Enable pg_trgm extension and add GIN trigram index on entities.canonical_name
-
-Revision ID: c1a2b3d4e5f6
-Revises: b4c5d6e7f8a9
-Create Date: 2026-03-02
-
-Index is created CONCURRENTLY so the migration does not block writes on entities
-during production deployments. CONCURRENTLY requires running outside a transaction
-block; see migrations.py for how this is handled safely.
-"""
-
-from collections.abc import Sequence
-
-import sqlalchemy as sa
-from alembic import context, op
-
-revision: str = "c1a2b3d4e5f6"
-down_revision: str | Sequence[str] | None = "b4c5d6e7f8a9"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    # pg_trgm ships with most PostgreSQL installations as a contrib module.
-    # It enables fast similarity lookups via GIN indexes, used for entity name matching.
-    # On managed services (e.g. Azure Flexible Server), the extension may not be
-    # available or may require manual enablement.  We gracefully skip the index
-    # creation if the extension cannot be loaded — the entity resolver will
-    # auto-detect and fall back to the "full" lookup strategy at runtime.  See #626.
-    conn = op.get_bind()
-    try:
-        conn.execute(sa.text("CREATE EXTENSION IF NOT EXISTS pg_trgm"))
-    except Exception:
-        # Extension not available (managed Postgres, insufficient privileges, etc.)
-        # Roll back the failed statement and skip index creation.
-        conn.execute(sa.text("ROLLBACK"))
-        conn.execute(sa.text("BEGIN"))
-        return
-
-    schema = _get_schema_prefix()
-    # GIN index on canonical_name enables sub-millisecond trigram similarity queries
-    # (% operator, similarity()) instead of full-table scans across all bank entities.
-    op.execute("COMMIT")
-    op.execute(
-        f"CREATE INDEX CONCURRENTLY IF NOT EXISTS entities_canonical_name_trgm_idx "
-        f"ON {schema}entities USING GIN (canonical_name gin_trgm_ops)"
-    )
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute("COMMIT")
-    op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}entities_canonical_name_trgm_idx")
-    # Note: not dropping pg_trgm extension as other indexes may depend on it

hindsight-api-slim/hindsight_api/alembic/versions/c2d3e4f5g6h7_add_audit_log_table.py

@@ -1,61 +0,0 @@
-"""Add audit_log table for feature usage tracking.
-
-Merge migration that combines the two existing heads (a3b4c5d6e7f8 + c8e5f2a3b4d1).
-
-Stores raw request/response as JSONB for expandability without future migrations.
-The metadata JSONB column allows adding arbitrary fields in the future.
-
-Revision ID: c2d3e4f5g6h7
-Revises: a3b4c5d6e7f8, c8e5f2a3b4d1
-Create Date: 2026-03-26
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "c2d3e4f5g6h7"
-down_revision: str | Sequence[str] | None = ("a3b4c5d6e7f8", "c8e5f2a3b4d1")
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Get schema prefix for table names (required for multi-tenant support)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    op.execute(
-        f"""
-        CREATE TABLE IF NOT EXISTS {schema}audit_log (
-            id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
-            action TEXT NOT NULL,
-            transport TEXT NOT NULL,
-            bank_id TEXT,
-            started_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
-            ended_at TIMESTAMPTZ,
-            request JSONB,
-            response JSONB,
-            metadata JSONB DEFAULT '{{}}'::jsonb
-        )
-        """
-    )
-
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS idx_audit_log_action_started ON {schema}audit_log (action, started_at DESC)"
-    )
-    op.execute(f"CREATE INDEX IF NOT EXISTS idx_audit_log_bank_started ON {schema}audit_log (bank_id, started_at DESC)")
-    op.execute(f"CREATE INDEX IF NOT EXISTS idx_audit_log_started ON {schema}audit_log (started_at DESC)")
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_audit_log_started")
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_audit_log_bank_started")
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_audit_log_action_started")
-    op.execute(f"DROP TABLE IF EXISTS {schema}audit_log")

hindsight-api-slim/hindsight_api/alembic/versions/c3d4e5f6g7h8_add_history_to_mental_models.py

@@ -1,30 +0,0 @@
-"""Add history column to mental_models
-
-Revision ID: c3d4e5f6g7h8
-Revises: a2b3c4d5e6f7, a2b3c4d5e6f8
-Create Date: 2026-03-06
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "c3d4e5f6g7h8"
-down_revision: str | Sequence[str] | None = ("a2b3c4d5e6f7", "a2b3c4d5e6f8")
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"ALTER TABLE {schema}mental_models ADD COLUMN IF NOT EXISTS history JSONB DEFAULT '[]'::jsonb")
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"ALTER TABLE {schema}mental_models DROP COLUMN IF EXISTS history")

hindsight-api-slim/hindsight_api/alembic/versions/c5d6e7f8a9b0_add_bank_id_to_memory_links.py

@@ -1,48 +0,0 @@
-"""Add bank_id column to memory_links for direct filtering
-
-The stats endpoint JOINs memory_links to memory_units just to filter by
-bank_id.  With millions of links this takes 18+ seconds.  Adding bank_id
-directly to memory_links lets Postgres push the filter down before the JOIN.
-
-Revision ID: c5d6e7f8a9b0
-Revises: b3c4d5e6f7a8
-Create Date: 2026-03-26
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "c5d6e7f8a9b0"
-down_revision: str | Sequence[str] | None = "b3c4d5e6f7a8"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # 1. Add nullable column
-    op.execute(f"ALTER TABLE {schema}memory_links ADD COLUMN IF NOT EXISTS bank_id TEXT")
-
-    # 2. Backfill from memory_units
-    op.execute(f"""
-        UPDATE {schema}memory_links ml
-        SET bank_id = mu.bank_id
-        FROM {schema}memory_units mu
-        WHERE ml.from_unit_id = mu.id
-          AND ml.bank_id IS NULL
-    """)
-
-    # 3. Set NOT NULL
-    op.execute(f"ALTER TABLE {schema}memory_links ALTER COLUMN bank_id SET NOT NULL")
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"ALTER TABLE {schema}memory_links DROP COLUMN IF EXISTS bank_id")

hindsight-api-slim/hindsight_api/alembic/versions/d2e3f4a5b6c7_add_memory_links_expansion_indexes.py

@@ -1,83 +0,0 @@
-"""Add covering and composite indexes to speed up link expansion graph retrieval.
-
-Two indexes target the two bottlenecks identified by EXPLAIN ANALYZE on a 17M-row
-memory_links table:
-
-1. idx_memory_links_to_type_weight  (to_unit_id, link_type, weight DESC)
-   The semantic incoming direction — finding facts that consider seeds as their
-   nearest neighbour — currently hits an expensive BitmapAnd of two separate
-   bitmap scans (to_unit_id bitmap ∩ link_type bitmap).  A composite index
-   on (to_unit_id, link_type) turns this into a single index scan and reduces
-   latency from ~36 ms to < 5 ms per query.
-
-2. idx_memory_links_entity_covering  (from_unit_id) INCLUDE (to_unit_id, entity_id)
-   WHERE link_type = 'entity'
-   The entity co-occurrence expansion uses COUNT(DISTINCT ml.entity_id) and
-   joins on ml.to_unit_id.  Without a covering index the planner must read
-   ~2 500 heap pages to fetch entity_id and to_unit_id after the bitmap index
-   scan, adding ~230 ms of random I/O.  INCLUDE adds those two columns to the
-   index leaf pages so the entire query can be served from the index (index-only
-   scan), eliminating the heap reads entirely.
-   Partial index (WHERE link_type = 'entity') keeps index size ~40 % smaller.
-
-Both indexes are created with CONCURRENTLY so the migration does not block
-concurrent reads or writes on memory_links.  CONCURRENTLY requires running
-outside a transaction block, so the migration emits an explicit COMMIT before
-each statement and uses IF NOT EXISTS for idempotency.
-
-Revision ID: d2e3f4a5b6c7
-Revises: b3c4d5e6f7g8
-Create Date: 2026-03-02
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "d2e3f4a5b6c7"
-down_revision: str | Sequence[str] | None = "b3c4d5e6f7g8"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # CREATE INDEX CONCURRENTLY cannot run inside a transaction block.
-    # Commit the current Alembic transaction, then issue each CONCURRENTLY
-    # statement in its own implicit autocommit transaction.
-    # IF NOT EXISTS makes each statement idempotent if the migration is retried.
-
-    # Index for the semantic *incoming* direction in link_expansion_retrieval.py.
-    # Replaces the BitmapAnd of idx_memory_links_to_unit ∩ idx_memory_links_link_type
-    # with a single composite index scan.
-    op.execute("COMMIT")
-    op.execute(
-        f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_links_to_type_weight "
-        f"ON {schema}memory_links(to_unit_id, link_type, weight DESC)"
-    )
-
-    # Covering index for entity co-occurrence expansion.
-    # Enables an index-only scan: entity_id and to_unit_id are read from the
-    # index leaf pages instead of the heap, eliminating ~2 500 random heap-page
-    # reads per expansion query.
-    op.execute("COMMIT")
-    op.execute(
-        f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_links_entity_covering "
-        f"ON {schema}memory_links(from_unit_id) "
-        f"INCLUDE (to_unit_id, entity_id) "
-        f"WHERE link_type = 'entity'"
-    )
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute("COMMIT")
-    op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_links_entity_covering")
-    op.execute("COMMIT")
-    op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_links_to_type_weight")

hindsight-api-slim/hindsight_api/alembic/versions/d4e5f6g7h8i9_gin_source_memory_ids_fastupdate_off.py

@@ -1,53 +0,0 @@
-"""Recreate idx_memory_units_source_memory_ids GIN index with fastupdate=off
-
-GIN indexes use a "fastupdate" pending list by default: small writes are
-buffered there and flushed to the main GIN tree in bulk. Flushing requires
-AccessExclusiveLock on the index. Under high insert concurrency (e.g. 8
-parallel pytest-xdist workers all calling retain_async) two transactions can
-each trigger a flush simultaneously and deadlock.
-
-Disabling fastupdate makes every insert write directly to the GIN tree
-(slightly slower per insert, but no pending-list lock cycles).
-
-Revision ID: d4e5f6g7h8i9
-Revises: d5e6f7a8b9c0
-Create Date: 2026-03-11
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "d4e5f6g7h8i9"
-down_revision: str | Sequence[str] | None = "d5e6f7a8b9c0"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-    # DROP + CREATE CONCURRENTLY must run outside a transaction block.
-    op.execute("COMMIT")
-    op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_units_source_memory_ids")
-    op.execute(
-        f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_units_source_memory_ids "
-        f"ON {schema}memory_units USING GIN (source_memory_ids) "
-        f"WITH (fastupdate=off) "
-        f"WHERE source_memory_ids IS NOT NULL"
-    )
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute("COMMIT")
-    op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_units_source_memory_ids")
-    op.execute(
-        f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_units_source_memory_ids "
-        f"ON {schema}memory_units USING GIN (source_memory_ids) "
-        f"WHERE source_memory_ids IS NOT NULL"
-    )

hindsight-api-slim/hindsight_api/alembic/versions/d5e6f7a8b9c0_add_bank_internal_id_and_per_bank_hnsw.py

@@ -1,139 +0,0 @@
-"""Add internal_id to banks and per-(bank, fact_type) partial vector indexes
-
-Revision ID: d5e6f7a8b9c0
-Revises: a3b4c5d6e7f8
-Create Date: 2026-03-11
-
-This migration:
-1. Adds internal_id UUID column to banks (stable identifier for index naming)
-2. Drops the global vector index (competes with per-bank partial indexes)
-3. Creates per-(bank_id, fact_type) partial vector indexes for all existing banks
-   using the configured vector extension (HNSW for pgvector, DiskANN for
-   pgvectorscale, vchordrq for vchord).
-   (new banks get indexes created at bank-creation time via bank_utils.create_bank_vector_indexes)
-
-Why per-(bank, fact_type) indexes:
-- fact_type-only partial indexes are never chosen by the planner when bank_id is in the WHERE
-  clause, because the idx_memory_units_bank_id B-tree index always wins at planning time.
-- Per-(bank, fact_type) partial indexes have both predicates matching → planner selects them.
-- The global vector index competes for larger partitions (world, observation) and must be dropped.
-"""
-
-import os
-from collections.abc import Sequence
-
-from alembic import context, op
-from sqlalchemy import text
-
-revision: str = "d5e6f7a8b9c0"
-down_revision: str | Sequence[str] | None = "c3d4e5f6g7h8"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-_FACT_TYPES: dict[str, str] = {
-    "world": "worl",
-    "experience": "expr",
-    "observation": "obsv",
-}
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _vector_index_using_clause() -> str:
-    """Return the USING clause based on the configured vector extension."""
-    ext = os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector").lower()
-    if ext == "pgvectorscale":
-        return "USING diskann (embedding vector_cosine_ops) WITH (num_neighbors = 50)"
-    elif ext == "vchord":
-        return "USING vchordrq (embedding vector_l2_ops)"
-    else:
-        return "USING hnsw (embedding vector_cosine_ops)"
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # 1. Add internal_id column to banks
-    op.execute(
-        f"ALTER TABLE {schema}banks ADD COLUMN IF NOT EXISTS internal_id UUID DEFAULT gen_random_uuid() NOT NULL"
-    )
-    op.execute(f"ALTER TABLE {schema}banks ADD CONSTRAINT banks_internal_id_unique UNIQUE (internal_id)")
-
-    # 2. Drop any fact_type-only partial indexes that may exist from prior migrations
-    #    (bank_id B-tree always wins over them when bank_id is in the WHERE clause)
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_mu_emb_world")
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_mu_emb_observation")
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_mu_emb_experience")
-
-    # 4. Drop global vector index (competes with per-bank partial indexes)
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_units_embedding")
-
-    # 5. Create per-(bank, fact_type) partial vector indexes for all existing banks
-    #    using the configured extension (HNSW / DiskANN / vchordrq)
-    bind = op.get_bind()
-    schema_name = context.config.get_main_option("target_schema")
-    table_ref = f'"{schema_name}".memory_units' if schema_name else "memory_units"
-    banks_ref = f'"{schema_name}".banks' if schema_name else "banks"
-    using_clause = _vector_index_using_clause()
-
-    rows = bind.execute(text(f"SELECT bank_id, internal_id FROM {banks_ref}")).fetchall()  # noqa: S608
-    for row in rows:
-        bank_id = row[0]
-        internal_id = str(row[1]).replace("-", "")[:16]
-        escaped_bank_id = bank_id.replace("'", "''")
-        for ft, ft_short in _FACT_TYPES.items():
-            idx_name = f"idx_mu_emb_{ft_short}_{internal_id}"
-            # Index name is schema-unqualified (indexes live in the schema of their table)
-            bind.execute(
-                text(
-                    f"CREATE INDEX IF NOT EXISTS {idx_name} "
-                    f"ON {table_ref} {using_clause} "
-                    f"WHERE fact_type = '{ft}' AND bank_id = '{escaped_bank_id}'"
-                )
-            )
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # Drop per-bank HNSW indexes (iterate existing banks)
-    bind = op.get_bind()
-    schema_name = context.config.get_main_option("target_schema")
-    banks_ref = f'"{schema_name}".banks' if schema_name else "banks"
-
-    rows = bind.execute(text(f"SELECT internal_id FROM {banks_ref}")).fetchall()  # noqa: S608
-    for row in rows:
-        internal_id = str(row[0]).replace("-", "")[:16]
-        for ft_short in _HNSW_FACT_TYPES.values():
-            idx_name = f"idx_mu_emb_{ft_short}_{internal_id}"
-            bind.execute(text(f"DROP INDEX IF EXISTS {schema}{idx_name}"))
-
-    # Restore the global HNSW index
-    table_ref = f'"{schema_name}".memory_units' if schema_name else "memory_units"
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS idx_memory_units_embedding ON {table_ref} USING hnsw (embedding vector_cosine_ops)"
-    )
-
-    # Restore old fact_type-only partial indexes
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS idx_mu_emb_world "
-        f"ON {table_ref} USING hnsw (embedding vector_cosine_ops) "
-        f"WHERE fact_type = 'world'"
-    )
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS idx_mu_emb_observation "
-        f"ON {table_ref} USING hnsw (embedding vector_cosine_ops) "
-        f"WHERE fact_type = 'observation'"
-    )
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS idx_mu_emb_experience "
-        f"ON {table_ref} USING hnsw (embedding vector_cosine_ops) "
-        f"WHERE fact_type = 'experience'"
-    )
-
-    # Drop internal_id column
-    op.execute(f"ALTER TABLE {schema}banks DROP CONSTRAINT IF EXISTS banks_internal_id_unique")
-    op.execute(f"ALTER TABLE {schema}banks DROP COLUMN IF EXISTS internal_id")

hindsight-api-slim/hindsight_api/alembic/versions/e4f5a6b7c8d9_add_webhooks_tables.py

@@ -1,62 +0,0 @@
-"""Add webhooks table and next_retry_at to async_operations.
-
-Webhook deliveries are handled as async_operations tasks (operation_type='webhook_delivery')
-rather than a dedicated webhook_deliveries table.
-
-Revision ID: e4f5a6b7c8d9
-Revises: d2e3f4a5b6c7
-Create Date: 2026-03-04
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "e4f5a6b7c8d9"
-down_revision: str | Sequence[str] | None = "d2e3f4a5b6c7"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    op.execute(
-        f"""
-        CREATE TABLE IF NOT EXISTS {schema}webhooks (
-            id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
-            bank_id TEXT,
-            url TEXT NOT NULL,
-            secret TEXT,
-            event_types TEXT[] NOT NULL DEFAULT '{{}}',
-            enabled BOOLEAN NOT NULL DEFAULT TRUE,
-            created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
-            updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
-        )
-        """
-    )
-
-    # Index for bank-scoped webhook lookup
-    op.execute(f"CREATE INDEX IF NOT EXISTS idx_webhooks_bank_id ON {schema}webhooks(bank_id)")
-
-    # Add next_retry_at to async_operations for task-owned retry scheduling
-    op.execute(f"ALTER TABLE {schema}async_operations ADD COLUMN IF NOT EXISTS next_retry_at TIMESTAMPTZ NULL")
-
-    # Index for polling: status + next_retry_at
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS idx_async_operations_status_retry "
-        f"ON {schema}async_operations(status, next_retry_at)"
-    )
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_async_operations_status_retry")
-    op.execute(f"ALTER TABLE {schema}async_operations DROP COLUMN IF EXISTS next_retry_at")
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_webhooks_bank_id")
-    op.execute(f"DROP TABLE IF EXISTS {schema}webhooks")

hindsight-api-slim/hindsight_api/alembic/versions/e5f6g7h8i9j0_cascade_delete_ops_on_bank_delete.py

@@ -1,73 +0,0 @@
-"""Add CASCADE DELETE FK from async_operations and webhooks to banks.
-
-When a bank is deleted, all its async_operations and webhooks rows are
-automatically deleted by the database. This ensures that any in-flight
-worker tasks detect the deletion via _check_op_alive() and abort early.
-
-Revision ID: e5f6g7h8i9j0
-Revises: d4e5f6g7h8i9
-Create Date: 2026-03-11
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "e5f6g7h8i9j0"
-down_revision: str | Sequence[str] | None = "d4e5f6g7h8i9"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # Remove orphaned async_operations rows whose bank no longer exists
-    # (can happen because there was no FK before this migration).
-    op.execute(
-        f"""
-        DELETE FROM {schema}async_operations
-        WHERE bank_id IS NOT NULL
-          AND bank_id NOT IN (SELECT bank_id FROM {schema}banks)
-        """
-    )
-
-    # Remove orphaned webhooks rows whose bank no longer exists.
-    op.execute(
-        f"""
-        DELETE FROM {schema}webhooks
-        WHERE bank_id IS NOT NULL
-          AND bank_id NOT IN (SELECT bank_id FROM {schema}banks)
-        """
-    )
-
-    # Add FK with ON DELETE CASCADE so that deleting a bank automatically
-    # cleans up all its pending/processing operations and webhook configs.
-    op.execute(
-        f"""
-        ALTER TABLE {schema}async_operations
-            ADD CONSTRAINT fk_async_operations_bank_id
-            FOREIGN KEY (bank_id) REFERENCES {schema}banks(bank_id)
-            ON DELETE CASCADE
-        """
-    )
-
-    op.execute(
-        f"""
-        ALTER TABLE {schema}webhooks
-            ADD CONSTRAINT fk_webhooks_bank_id
-            FOREIGN KEY (bank_id) REFERENCES {schema}banks(bank_id)
-            ON DELETE CASCADE
-        """
-    )
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"ALTER TABLE {schema}async_operations DROP CONSTRAINT IF EXISTS fk_async_operations_bank_id")
-    op.execute(f"ALTER TABLE {schema}webhooks DROP CONSTRAINT IF EXISTS fk_webhooks_bank_id")

hindsight-api-slim/hindsight_api/alembic/versions/f6g7h8i9j0k1_chunk_fk_cascade_delete.py

@@ -1,57 +0,0 @@
-"""chunk_fk_cascade_delete
-
-Revision ID: f6g7h8i9j0k1
-Revises: e5f6g7h8i9j0
-Create Date: 2026-03-16 00:00:00.000000
-
-"""
-
-from collections.abc import Sequence
-
-from alembic import op
-
-# revision identifiers, used by Alembic.
-revision: str = "f6g7h8i9j0k1"
-down_revision: str | Sequence[str] | None = "e5f6g7h8i9j0"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def upgrade() -> None:
-    """Change memory_units.chunk_id FK from SET NULL to CASCADE.
-
-    When a document is deleted the CASCADE reaches chunks first; with SET NULL
-    the memory_units rows survived with chunk_id = NULL, leaving ghost records.
-    Switching to CASCADE ensures they are removed together with their chunk.
-    """
-    from alembic import context
-
-    schema = context.config.get_main_option("target_schema")
-    schema_prefix = f'"{schema}".' if schema else ""
-    # Use raw SQL with IF EXISTS so this is safe on schemas where the FK was
-    # already dropped or never existed under this name.
-    op.execute(f"ALTER TABLE {schema_prefix}memory_units DROP CONSTRAINT IF EXISTS memory_units_chunk_fkey")
-    # Use a DO block so the ADD is also idempotent: if the FK already exists (e.g.
-    # the schema was provisioned after the base migration already added it) the
-    # duplicate_object exception is swallowed rather than failing the migration.
-    op.execute(
-        f"""
-        DO $$ BEGIN
-            ALTER TABLE {schema_prefix}memory_units
-                ADD CONSTRAINT memory_units_chunk_fkey
-                FOREIGN KEY (chunk_id)
-                REFERENCES {schema_prefix}chunks (chunk_id)
-                ON DELETE CASCADE;
-        EXCEPTION
-            WHEN duplicate_object THEN NULL;
-        END $$;
-        """
-    )
-
-
-def downgrade() -> None:
-    """Revert to SET NULL behaviour."""
-    op.drop_constraint("memory_units_chunk_fkey", "memory_units", type_="foreignkey")
-    op.create_foreign_key(
-        "memory_units_chunk_fkey", "memory_units", "chunks", ["chunk_id"], ["chunk_id"], ondelete="SET NULL"
-    )

hindsight-api-slim/hindsight_api/alembic/versions/f7g8h9i0j1k2_add_webhook_http_config.py

@@ -1,33 +0,0 @@
-"""Add http_config JSONB column to webhooks table.
-
-Stores HTTP delivery configuration (method, timeout, headers, params) as a
-single JSONB column rather than separate columns.
-
-Revision ID: f7g8h9i0j1k2
-Revises: e4f5a6b7c8d9
-Create Date: 2026-03-04
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "f7g8h9i0j1k2"
-down_revision: str | Sequence[str] | None = "e4f5a6b7c8d9"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"ALTER TABLE {schema}webhooks ADD COLUMN IF NOT EXISTS http_config JSONB NOT NULL DEFAULT '{{}}'")
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"ALTER TABLE {schema}webhooks DROP COLUMN IF EXISTS http_config")

hindsight-api-slim/hindsight_api/alembic/versions/g2h3i4j5k6l7_remove_opinion_fact_type.py

@@ -1,83 +0,0 @@
-"""remove_opinion_fact_type
-
-Revision ID: g2h3i4j5k6l7
-Revises: f1a2b3c4d5e6
-Create Date: 2026-04-02
-
-Remove the deprecated 'opinion' fact type: drop opinion-specific indexes,
-update CHECK constraints, delete any remaining opinion rows, and drop the
-confidence_score column (was only used for opinions, always NULL otherwise).
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-# revision identifiers, used by Alembic.
-revision: str = "g2h3i4j5k6l7"
-down_revision: str | Sequence[str] | None = "f1a2b3c4d5e6"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Get schema prefix for table names (required for multi-tenant support)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # 1. Delete any remaining opinion rows
-    op.execute(f"DELETE FROM {schema}memory_units WHERE fact_type = 'opinion'")
-
-    # 2. Drop opinion-specific indexes
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_units_opinion_confidence")
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_units_opinion_date")
-
-    # 3. Drop confidence_score constraints and column (only used for opinions, always NULL otherwise)
-    op.execute(f"ALTER TABLE {schema}memory_units DROP CONSTRAINT IF EXISTS confidence_score_fact_type_check")
-    op.execute(f"ALTER TABLE {schema}memory_units DROP CONSTRAINT IF EXISTS memory_units_confidence_score_check")
-    op.execute(f"ALTER TABLE {schema}memory_units DROP COLUMN IF EXISTS confidence_score")
-
-    # 4. Replace fact_type CHECK constraint
-    op.execute(f"ALTER TABLE {schema}memory_units DROP CONSTRAINT IF EXISTS memory_units_fact_type_check")
-    op.execute(
-        f"ALTER TABLE {schema}memory_units ADD CONSTRAINT memory_units_fact_type_check "
-        f"CHECK (fact_type IN ('world', 'experience', 'observation'))"
-    )
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # Restore confidence_score column
-    op.execute(f"ALTER TABLE {schema}memory_units ADD COLUMN IF NOT EXISTS confidence_score float")
-    op.execute(
-        f"ALTER TABLE {schema}memory_units ADD CONSTRAINT memory_units_confidence_score_check "
-        f"CHECK (confidence_score IS NULL OR (confidence_score >= 0.0 AND confidence_score <= 1.0))"
-    )
-    op.execute(
-        f"ALTER TABLE {schema}memory_units ADD CONSTRAINT confidence_score_fact_type_check "
-        f"CHECK ((fact_type = 'opinion' AND confidence_score IS NOT NULL) OR "
-        f"(fact_type = 'observation') OR "
-        f"(fact_type NOT IN ('opinion', 'observation') AND confidence_score IS NULL))"
-    )
-
-    # Restore original fact_type CHECK constraint (with opinion)
-    op.execute(f"ALTER TABLE {schema}memory_units DROP CONSTRAINT IF EXISTS memory_units_fact_type_check")
-    op.execute(
-        f"ALTER TABLE {schema}memory_units ADD CONSTRAINT memory_units_fact_type_check "
-        f"CHECK (fact_type IN ('world', 'experience', 'opinion', 'observation'))"
-    )
-
-    # Recreate opinion indexes
-    op.execute(
-        f"CREATE INDEX idx_memory_units_opinion_confidence ON {schema}memory_units "
-        f"(bank_id, confidence_score DESC) WHERE fact_type = 'opinion'"
-    )
-    op.execute(
-        f"CREATE INDEX idx_memory_units_opinion_date ON {schema}memory_units "
-        f"(bank_id, event_date DESC) WHERE fact_type = 'opinion'"
-    )

hindsight-api-slim/hindsight_api/alembic/versions/g7h8i9j0k1l2_backsweep_orphan_observations.py

@@ -1,71 +0,0 @@
-"""backsweep_orphan_memory_units
-
-Two-pass cleanup of memory_units rows that were never removed by earlier bugs:
-
-Pass 1 — any fact_type, bank gone:
-  memory_units whose bank_id no longer exists in banks.  These accumulate when
-  a bank is deleted without a proper cascade (no FK from memory_units to banks
-  exists in the schema).
-
-Pass 2 — observations only, all sources gone:
-  observation rows whose bank still exists but every source_memory_id points
-  to a deleted memory unit.  These were left behind before PR #580 fixed the
-  chunk FK cascade and before delete_document() called
-  _delete_stale_observations_for_memories.
-
-Revision ID: g7h8i9j0k1l2
-Revises: f6g7h8i9j0k1
-Create Date: 2026-03-16
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "g7h8i9j0k1l2"
-down_revision: str | Sequence[str] | None = "f6g7h8i9j0k1"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-    mu = f"{schema}memory_units"
-    banks = f"{schema}banks"
-
-    # Pass 1: delete all memory_units (any fact_type) whose bank no longer exists.
-    # There is no FK from memory_units to banks, so these never cascade away.
-    op.execute(
-        f"""
-        DELETE FROM {mu}
-        WHERE NOT EXISTS (
-            SELECT 1 FROM {banks} b WHERE b.bank_id = {mu}.bank_id
-        )
-        """
-    )
-
-    # Pass 2: delete orphaned observations whose bank still exists but every
-    # source_memory_id refers to a now-deleted memory unit (or the array is
-    # empty).  Observations with at least one surviving source are left alone.
-    op.execute(
-        f"""
-        DELETE FROM {mu} orphan
-        WHERE orphan.fact_type = 'observation'
-          AND NOT EXISTS (
-              SELECT 1
-              FROM {mu} src
-              WHERE src.id = ANY(orphan.source_memory_ids)
-                AND src.bank_id = orphan.bank_id
-          )
-        """
-    )
-
-
-def downgrade() -> None:
-    # Deleted rows cannot be restored.
-    pass

hindsight-api-slim/hindsight_api/alembic/versions/h3i4j5k6l7m8_merge_heads_and_add_unit_entities_index.py

@@ -1,42 +0,0 @@
-"""Merge 3 migration heads and add unit_entities composite index
-
-Revision ID: h3i4j5k6l7m8
-Revises: a4b5c6d7e8f9, c2d3e4f5g6h7, g2h3i4j5k6l7
-Create Date: 2026-04-07
-
-Merges three unmerged migration heads into one, and adds a composite index
-(entity_id, unit_id) on unit_entities for index-only scans in the LATERAL
-entity expansion query.
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "h3i4j5k6l7m8"
-down_revision: str | Sequence[str] | None = ("a4b5c6d7e8f9", "c2d3e4f5g6h7", "g2h3i4j5k6l7")
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Get schema prefix for table names (required for multi-tenant support)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-    # Composite index enables index-only scans for entity_id -> unit_id lookups
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS idx_unit_entities_entity_unit ON {schema}unit_entities (entity_id, unit_id)"
-    )
-    # Drop the now-redundant single-column index
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_unit_entities_entity")
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_unit_entities_entity_unit")
-    # Restore the single-column index
-    op.execute(f"CREATE INDEX IF NOT EXISTS idx_unit_entities_entity ON {schema}unit_entities (entity_id)")

hindsight-api-slim/hindsight_api/alembic/versions/z1u2v3w4x5y6_add_observation_tags_to_memory_units.py

@@ -1,35 +0,0 @@
-"""Add observation_scopes column to memory_units table
-
-Revision ID: z1u2v3w4x5y6
-Revises: a1b2c3d4e5f6
-Create Date: 2026-02-25
-
-Adds observation_scopes JSONB column to memory_units to control how observations
-are scoped during consolidation. Accepts "per_tag", "combined", or an explicit
-list of tag-set lists for custom multi-pass consolidation.
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "z1u2v3w4x5y6"
-down_revision: str | Sequence[str] | None = "a1b2c3d4e5f6"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Get schema prefix for table names (required for multi-tenant support)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"ALTER TABLE {schema}memory_units ADD COLUMN IF NOT EXISTS observation_scopes JSONB")
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"ALTER TABLE {schema}memory_units DROP COLUMN IF EXISTS observation_scopes")

hindsight-api-slim/hindsight_api/engine/audit.py

@@ -1,209 +0,0 @@
-"""Audit logging for feature usage tracking.
-
-Provides fire-and-forget audit logging of all mutating and core operations
-(retain, recall, reflect, bank CRUD, etc.) across HTTP, MCP, and system transports.
-"""
-
-from __future__ import annotations
-
-import asyncio
-import json
-import logging
-import uuid
-from collections.abc import Callable
-from contextlib import asynccontextmanager
-from dataclasses import dataclass, field
-from datetime import datetime, timezone
-from typing import Any
-
-import asyncpg
-
-from ..engine.db_utils import acquire_with_retry
-
-logger = logging.getLogger(__name__)
-
-
-@dataclass
-class AuditEntry:
-    """A single audit log entry."""
-
-    action: str
-    transport: str  # "http", "mcp", "system"
-    bank_id: str | None = None
-    started_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
-    ended_at: datetime | None = None
-    request: dict[str, Any] | None = None
-    response: dict[str, Any] | None = None
-    metadata: dict[str, Any] = field(default_factory=dict)
-
-
-def _json_default(obj: Any) -> str:
-    """JSON serializer for objects not serializable by default."""
-    if isinstance(obj, datetime):
-        return obj.isoformat()
-    if isinstance(obj, uuid.UUID):
-        return str(obj)
-    if isinstance(obj, bytes):
-        return "<bytes>"
-    if isinstance(obj, set):
-        return list(obj)
-    return str(obj)
-
-
-def _safe_json(data: Any) -> str | None:
-    """Serialize data to JSON string, returning None on failure."""
-    if data is None:
-        return None
-    try:
-        return json.dumps(data, default=_json_default)
-    except Exception:
-        logger.debug("Failed to serialize audit data", exc_info=True)
-        return None
-
-
-_SWEEP_INTERVAL_SECONDS = 3600  # Run retention sweep every hour
-
-
-class AuditLogger:
-    """Fire-and-forget audit log writer with optional retention sweep."""
-
-    def __init__(
-        self,
-        pool_getter: Callable[[], asyncpg.Pool | None],
-        schema_getter: Callable[[], str],
-        enabled: bool,
-        allowed_actions: list[str],
-        retention_days: int = -1,
-    ) -> None:
-        self._pool_getter = pool_getter
-        self._schema_getter = schema_getter
-        self._enabled = enabled
-        self._allowed_actions: frozenset[str] | None = frozenset(allowed_actions) if allowed_actions else None
-        self._retention_days = retention_days
-        self._sweep_task: asyncio.Task | None = None
-
-    def is_enabled(self, action: str) -> bool:
-        """Check if audit logging is enabled for this action."""
-        if not self._enabled:
-            return False
-        if self._allowed_actions is not None:
-            return action in self._allowed_actions
-        return True
-
-    def log_fire_and_forget(self, entry: AuditEntry) -> None:
-        """Schedule an audit write as a background task."""
-        if not self.is_enabled(entry.action):
-            return
-        try:
-            asyncio.create_task(self._safe_log(entry))
-        except RuntimeError:
-            # No running event loop (e.g. during shutdown)
-            logger.debug("Cannot schedule audit log write: no running event loop")
-
-    async def _safe_log(self, entry: AuditEntry) -> None:
-        """Write audit entry to DB. Errors are logged, never raised."""
-        pool = self._pool_getter()
-        if pool is None:
-            logger.debug("Audit log skipped: pool not available")
-            return
-        try:
-            schema = self._schema_getter()
-            table = f"{schema}.audit_log"
-            async with acquire_with_retry(pool, max_retries=1) as conn:
-                await conn.execute(
-                    f"""
-                    INSERT INTO {table}
-                        (id, action, transport, bank_id, started_at, ended_at, request, response, metadata)
-                    VALUES
-                        ($1, $2, $3, $4, $5, $6, $7::jsonb, $8::jsonb, $9::jsonb)
-                    """,
-                    uuid.uuid4(),
-                    entry.action,
-                    entry.transport,
-                    entry.bank_id,
-                    entry.started_at,
-                    entry.ended_at,
-                    _safe_json(entry.request),
-                    _safe_json(entry.response),
-                    _safe_json(entry.metadata) or "{}",
-                )
-        except Exception as e:
-            logger.warning(f"Audit log write failed for action={entry.action}: {e}")
-
-    def start_retention_sweep(self) -> None:
-        """Start the periodic retention sweep if retention is configured."""
-        if self._retention_days <= 0 or not self._enabled:
-            return
-        try:
-            self._sweep_task = asyncio.create_task(self._sweep_loop())
-        except RuntimeError:
-            logger.debug("Cannot start retention sweep: no running event loop")
-
-    async def stop_retention_sweep(self) -> None:
-        """Stop the periodic retention sweep."""
-        if self._sweep_task and not self._sweep_task.done():
-            self._sweep_task.cancel()
-            try:
-                await self._sweep_task
-            except asyncio.CancelledError:
-                pass
-            self._sweep_task = None
-
-    async def _sweep_loop(self) -> None:
-        """Periodically delete audit log entries older than retention_days."""
-        while True:
-            await self._run_sweep()
-            await asyncio.sleep(_SWEEP_INTERVAL_SECONDS)
-
-    async def _run_sweep(self) -> None:
-        """Delete expired audit log entries. Concurrent-safe via row-level deletes."""
-        pool = self._pool_getter()
-        if pool is None:
-            return
-        try:
-            schema = self._schema_getter()
-            table = f"{schema}.audit_log"
-            async with acquire_with_retry(pool, max_retries=1) as conn:
-                result = await conn.execute(
-                    f"DELETE FROM {table} WHERE started_at < NOW() - INTERVAL '{self._retention_days} days'"
-                )
-                if result and result != "DELETE 0":
-                    logger.info(f"Audit log retention sweep: {result}")
-        except Exception as e:
-            logger.warning(f"Audit log retention sweep failed: {e}")
-
-
-@asynccontextmanager
-async def audit_context(
-    audit_logger: AuditLogger | None,
-    action: str,
-    transport: str,
-    bank_id: str | None = None,
-    request: dict[str, Any] | None = None,
-    metadata: dict[str, Any] | None = None,
-):
-    """Async context manager that times the operation and writes audit on exit.
-
-    Usage:
-        async with audit_context(logger, "retain", "http", bank_id, request_dict) as entry:
-            result = await do_work()
-            entry.response = result_dict
-    """
-    if audit_logger is None or not audit_logger.is_enabled(action):
-        entry = AuditEntry(action=action, transport=transport, bank_id=bank_id)
-        yield entry
-        return
-
-    entry = AuditEntry(
-        action=action,
-        transport=transport,
-        bank_id=bank_id,
-        started_at=datetime.now(timezone.utc),
-        request=request,
-        metadata=metadata or {},
-    )
-    try:
-        yield entry
-    finally:
-        entry.ended_at = datetime.now(timezone.utc)
-        audit_logger.log_fire_and_forget(entry)

hindsight-api-slim/hindsight_api/engine/consolidation/prompts.py

@@ -1,104 +0,0 @@
-"""Prompts for the consolidation engine."""
-
-# Default mission when no bank-specific mission is set
-_DEFAULT_MISSION = "Track every detail: names, numbers, dates, places, and relationships. Prefer specifics over abstractions, never generalise."
-
-# Processing rules — always present regardless of mission
-_PROCESSING_RULES = """Processing rules (always apply):
-
-1. ONE OBSERVATION PER DISTINCT FACET: each observation tracks exactly one specific facet — a count ("has 3 items"), a named entity ("has a dog named Rex"), a relationship ("works at Google"), etc. Never merge different facets into one observation.
-
-2. MATCH BY ENTITY/FACET, NOT TOPIC: when deciding whether to UPDATE vs CREATE, match on the specific entity or facet. "Sold item X" updates only the X observation. "Now has 5 items" updates only the count observation. Do not update observations about different entities just because they share a general topic.
-
-3. STATE CHANGES — UPDATE CONCISELY: when a fact changes the state of something ("sold X", "X died", "moved to Y"), UPDATE the matching observation to reflect the current state. Include dates when available. Keep it concise — only information about THAT specific facet. Example: "User owned a dog named Rex who died on March 15, 2025". Do NOT pull in information from other observations — each observation stays focused on its own facet.
-
-4. CASCADE TO ALL AFFECTED OBSERVATIONS: a state change may affect multiple observations. For example, if entity C is removed from a group, update BOTH the individual observation for C AND any list/group observation that includes C (remove C from the list while keeping all other members intact).
-
-5. NO COMPUTATION: you do not have the full picture — never calculate, derive, or adjust numeric values. If the user says "I have 2 dogs" and then "I have a dog named Rex", do NOT update the count to 3 — you don't know if Rex is one of the 2 or a new one. If the user says "I sold X", do NOT decrement a count. Only update a count when the user explicitly states a new count. Synthesize and consolidate what was stated, but never do arithmetic or logical deductions.
-
-6. SAME FACET → UPDATE, NOT CREATE: a new count supersedes the old count — UPDATE the existing count observation, don't create a second one. If there's an existing observation for the same specific facet, always UPDATE it rather than creating a duplicate.
-
-7. PRESERVE HISTORY: observations that record significant events (sold, died, moved, changed) are important history — never DELETE them. Only delete an observation when it is restated identically or truly meaningless. Be very conservative with deletes.
-
-8. RESOLVE REFERENCES: when a new fact provides a concrete value for a vague placeholder in an existing observation (e.g., "home country" → "Sweden"), UPDATE to embed the resolved value.
-
-9. NEVER merge observations about different people or unrelated topics."""
-
-# Data section — format placeholders {facts_text} and {observations_text} are substituted at call time
-_BATCH_DATA_SECTION = """
-NEW FACTS:
-{facts_text}
-
-EXISTING OBSERVATIONS (JSON array, pooled from recalls across all facts above):
-{observations_text}
-
-Each observation includes:
-- id: unique identifier for updating
-- text: the observation content
-- proof_count: number of supporting memories
-- occurred_start/occurred_end: temporal range of source facts
-- source_memories: array of supporting facts with their text and dates
-
-Compare the facts against existing observations:
-- Same facet as an existing observation → UPDATE it (observation_id + source_fact_ids)
-- New facet with durable knowledge → CREATE a new observation (source_fact_ids)
-- Cross-reference facts within the batch: a later fact may resolve a vague reference in an earlier one
-- Purely ephemeral facts → omit them unless the MISSION above explicitly targets such data (e.g. timestamped events, session state, screen content)"""
-
-# Output format — JSON braces escaped as {{ }} so .format() leaves them literal
-_BATCH_OUTPUT_FORMAT = """
-Output a JSON object with three arrays.
-
-## EXAMPLE
-
-Input facts:
-[a1b2c3d4-e5f6-7890-abcd-ef1234567890] Alice mentioned she works long hours, often past midnight | Involving: Alice (occurred_start=2024-01-15, mentioned_at=2024-01-15)
-[b2c3d4e5-f6a7-8901-bcde-f12345678901] Alice said she's exhausted from the project deadlines | Involving: Alice (occurred_start=2024-01-20, mentioned_at=2024-01-20)
-
-Good observation text — clean prose, no metadata, each fact tracked distinctly:
-  "Alice works long hours, often past midnight."
-  "Alice feels exhausted from project deadlines."
-
-Bad observation text — NEVER do this (verbatim copy of fact text with metadata):
-  "Alice mentioned she works long hours, often past midnight | Involving: Alice (occurred_start=2024-01-15, mentioned_at=2024-01-15)"
-
-Observation text rules:
-- Write clean prose — NEVER copy raw fact lines or their metadata (temporal fields, "Involving:", "When:" labels, UUIDs).
-- Parenthesized metadata like (occurred_start=...) and pipe-separated labels like "| Involving: ..." are fact formatting — strip them entirely from observation text.
-- How many observations to create and how much to aggregate is driven by the MISSION above.
-
-{{"creates": [{{"text": "Alice works long hours, often past midnight.", "source_fact_ids": ["a1b2c3d4-e5f6-7890-abcd-ef1234567890"]}}, {{"text": "Alice feels exhausted from project deadlines.", "source_fact_ids": ["b2c3d4e5-f6a7-8901-bcde-f12345678901"]}}],
-  "updates": [{{"text": "Alice works at Acme Corp as a senior engineer", "observation_id": "c3d4e5f6-a7b8-9012-cdef-123456789012", "source_fact_ids": ["d4e5f6a7-b8c9-0123-defa-234567890123"]}}],
-  "deletes": [{{"observation_id": "e5f6a7b8-c9d0-1234-efab-345678901234"}}]}}
-
-Rules:
-- "source_fact_ids": copy the EXACT UUID strings shown in brackets [uuid] from NEW FACTS — never use integers or positions.
-- "observation_id": copy the EXACT "id" UUID string from EXISTING OBSERVATIONS.
-- One create/update may reference multiple facts when they jointly support the observation.
-- "deletes": only when an observation is directly superseded or contradicted by new facts.
-- Do NOT include "tags" — handled automatically.
-- Return {{"creates": [], "updates": [], "deletes": []}} if nothing durable is found."""
-
-
-def build_batch_consolidation_prompt(
-    observations_mission: str | None = None,
-    observation_capacity_note: str | None = None,
-) -> str:
-    """
-    Build the consolidation prompt for batch mode (multiple facts per LLM call).
-
-    The mission defines *what* to track (customisable per bank).
-    Processing rules and output format are always present regardless of mission.
-    """
-    mission = observations_mission or _DEFAULT_MISSION
-
-    capacity_section = ""
-    if observation_capacity_note:
-        capacity_section = f"\n\n## CAPACITY CONSTRAINT\n{observation_capacity_note}"
-
-    return (
-        "You are a memory consolidation system. Synthesize facts into observations "
-        "and merge with existing observations when appropriate.\n\n"
-        f"## MISSION\n{mission}{capacity_section}\n\n"
-        f"{_PROCESSING_RULES}" + _BATCH_DATA_SECTION + _BATCH_OUTPUT_FORMAT
-    )

hindsight-api-slim/hindsight_api/engine/jina_mlx_reranker.py

@@ -1,144 +0,0 @@
-"""
-MLX implementation of jina-reranker-v3 for Apple Silicon.
-
-This file is adapted from the official model repository:
-  https://huggingface.co/jinaai/jina-reranker-v3-mlx/blob/main/rerank.py
-
-License: CC BY-NC 4.0 (contact Jina AI for commercial usage)
-
-Changes from upstream:
-- Removed the __main__ example block
-- Type annotations added to public methods
-- top_n parameter added to rerank() (upstream only exposed it implicitly)
-"""
-
-import numpy as np
-
-
-class _MLPProjector:
-    def __init__(self):
-        import mlx.nn as nn
-
-        self.linear1 = nn.Linear(1024, 512, bias=False)
-        self.linear2 = nn.Linear(512, 512, bias=False)
-
-    def __call__(self, x):
-        import mlx.nn as nn
-
-        x = self.linear1(x)
-        x = nn.relu(x)
-        x = self.linear2(x)
-        return x
-
-
-def _load_projector(projector_path: str) -> _MLPProjector:
-    import mlx.core as mx
-    from safetensors import safe_open
-
-    projector = _MLPProjector()
-    with safe_open(projector_path, framework="numpy") as f:
-        projector.linear1.weight = mx.array(f.get_tensor("linear1.weight"))
-        projector.linear2.weight = mx.array(f.get_tensor("linear2.weight"))
-    return projector
-
-
-def _sanitize(text: str, special_tokens: dict[str, str]) -> str:
-    for token in special_tokens.values():
-        text = text.replace(token, "")
-    return text
-
-
-def _format_prompt(query: str, docs: list[str], special_tokens: dict[str, str]) -> str:
-    query = _sanitize(query, special_tokens)
-    docs = [_sanitize(d, special_tokens) for d in docs]
-
-    doc_token = special_tokens["doc_embed_token"]
-    query_token = special_tokens["query_embed_token"]
-
-    prefix = (
-        "<|im_start|>system\n"
-        "You are a search relevance expert who can determine a ranking of the passages based on how relevant they are to the query. "
-        "If the query is a question, how relevant a passage is depends on how well it answers the question. "
-        "If not, try to analyze the intent of the query and assess how well each passage satisfies the intent. "
-        "If an instruction is provided, you should follow the instruction when determining the ranking."
-        "<|im_end|>\n<|im_start|>user\n"
-    )
-    suffix = "<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"
-
-    body = (
-        f"I will provide you with {len(docs)} passages, each indicated by a numerical identifier. "
-        f"Rank the passages based on their relevance to query: {query}\n"
-    )
-    body += "\n".join(f'<passage id="{i}">\n{doc}{doc_token}\n</passage>' for i, doc in enumerate(docs))
-    body += f"\n<query>\n{query}{query_token}\n</query>"
-    return prefix + body + suffix
-
-
-class MLXReranker:
-    """
-    MLX-accelerated jina-reranker-v3 for Apple Silicon.
-
-    Loads the model from a local directory (use huggingface_hub.snapshot_download
-    to fetch jinaai/jina-reranker-v3-mlx if you don't have it already).
-    """
-
-    _SPECIAL_TOKENS = {
-        "query_embed_token": "<|rerank_token|>",
-        "doc_embed_token": "<|embed_token|>",
-    }
-    _DOC_TOKEN_ID = 151670
-    _QUERY_TOKEN_ID = 151671
-
-    def __init__(self, model_path: str, projector_path: str):
-        from mlx_lm import load
-
-        self.model, self.tokenizer = load(model_path)
-        self.model.eval()
-        self.projector = _load_projector(projector_path)
-
-    def rerank(self, query: str, documents: list[str], top_n: int | None = None) -> list[dict]:
-        """
-        Rank documents by relevance to a query.
-
-        Returns a list of dicts with keys: document, relevance_score, index.
-        Sorted by descending relevance_score.
-        """
-        import mlx.core as mx
-
-        prompt = _format_prompt(query, documents, self._SPECIAL_TOKENS)
-        input_ids = self.tokenizer.encode(prompt)
-        hidden_states = self.model.model([input_ids])[0]  # [seq_len, hidden_size]
-
-        input_ids_np = np.array(input_ids)
-        query_positions = np.where(input_ids_np == self._QUERY_TOKEN_ID)[0]
-        doc_positions = np.where(input_ids_np == self._DOC_TOKEN_ID)[0]
-
-        if len(query_positions) == 0:
-            raise ValueError("Query embed token not found in prompt")
-        if len(doc_positions) == 0:
-            raise ValueError("Document embed tokens not found in prompt")
-
-        query_hidden = mx.expand_dims(hidden_states[int(query_positions[0])], axis=0)
-        doc_hidden = mx.stack([hidden_states[int(p)] for p in doc_positions])
-
-        query_emb = self.projector(query_hidden)  # [1, 512]
-        doc_emb = self.projector(doc_hidden)  # [num_docs, 512]
-
-        query_exp = mx.broadcast_to(mx.expand_dims(query_emb, 0), (1, len(documents), 512))
-        doc_exp = mx.expand_dims(doc_emb, 0)
-
-        scores = mx.sum(doc_exp * query_exp, axis=-1) / (
-            mx.sqrt(mx.sum(doc_exp * doc_exp, axis=-1)) * mx.sqrt(mx.sum(query_exp * query_exp, axis=-1))
-        )  # [1, num_docs]
-        scores_np = np.array(scores[0])
-
-        order = np.argsort(scores_np)[::-1]
-        n = min(top_n, len(documents)) if top_n is not None else len(documents)
-        return [
-            {
-                "document": documents[order[i]],
-                "relevance_score": float(scores_np[order[i]]),
-                "index": int(order[i]),
-            }
-            for i in range(n)
-        ]

hindsight-api-slim/hindsight_api/engine/parsers/__init__.py

@@ -1,128 +0,0 @@
-"""File parser implementations."""
-
-import logging
-from dataclasses import dataclass
-
-from .base import FileParser, UnsupportedFileTypeError
-from .iris import IrisParser
-from .markitdown import MarkitdownParser
-
-__all__ = [
-    "FileParser",
-    "UnsupportedFileTypeError",
-    "IrisParser",
-    "MarkitdownParser",
-    "FileParserRegistry",
-    "ConvertResult",
-]
-
-
-@dataclass
-class ConvertResult:
-    """Result of a successful file conversion."""
-
-    content: str
-    parser_name: str
-
-
-logger = logging.getLogger(__name__)
-
-
-class FileParserRegistry:
-    """Registry for file parsers with auto-detection."""
-
-    def __init__(self):
-        """Initialize empty parser registry."""
-        self._parsers: dict[str, FileParser] = {}
-
-    def register(self, parser: FileParser):
-        """
-        Register a parser.
-
-        Args:
-            parser: FileParser instance
-        """
-        self._parsers[parser.name()] = parser
-
-    def get_parser(
-        self,
-        name: str | None,
-        filename: str,
-        content_type: str | None = None,
-    ) -> FileParser:
-        """
-        Get parser by name or auto-detect.
-
-        Args:
-            name: Parser name (e.g., "markitdown") or None for auto-detect
-            filename: File name for auto-detection
-            content_type: MIME type (optional)
-
-        Returns:
-            FileParser instance
-
-        Raises:
-            ValueError: If no suitable parser found
-        """
-        if name:
-            # Explicit parser requested — return it directly, let the parser
-            # raise UnsupportedFileTypeError from convert() if needed
-            if name not in self._parsers:
-                raise ValueError(f"Parser '{name}' not found. Available: {list(self._parsers.keys())}")
-            return self._parsers[name]
-
-        # Auto-detect parser
-        for parser in self._parsers.values():
-            if parser.supports(filename, content_type):
-                return parser
-
-        raise ValueError(f"No parser found for {filename}. Available parsers: {list(self._parsers.keys())}")
-
-    async def convert_with_fallback(
-        self,
-        parsers: list[str],
-        file_data: bytes,
-        filename: str,
-        content_type: str | None = None,
-    ) -> ConvertResult:
-        """
-        Try each parser in order, falling back on failure or empty content.
-
-        Moves to the next parser if the current one raises UnsupportedFileTypeError
-        or returns empty content. Any other exception (RuntimeError, network error,
-        etc.) also triggers a fallback so the chain is exhausted before failing.
-
-        Args:
-            parsers: Ordered list of parser names to try
-            file_data: Raw file bytes
-            filename: Original filename
-            content_type: MIME type (optional)
-
-        Returns:
-            ConvertResult with the parsed content and the name of the parser that succeeded
-
-        Raises:
-            ValueError: If a parser name is not registered
-            RuntimeError: If all parsers fail or return empty content
-        """
-        last_error: Exception | None = None
-        for name in parsers:
-            parser = self.get_parser(name, filename, content_type)
-            try:
-                content = await parser.convert(file_data, filename)
-                if content and content.strip():
-                    return ConvertResult(content=content, parser_name=name)
-                logger.warning(f"Parser '{name}' returned empty content for '{filename}', trying next")
-                last_error = RuntimeError(f"Parser '{name}' returned no content for '{filename}'")
-            except UnsupportedFileTypeError as e:
-                logger.warning(f"Parser '{name}' does not support '{filename}', trying next: {e}")
-                last_error = e
-            except Exception as e:
-                logger.warning(f"Parser '{name}' failed for '{filename}', trying next: {e}")
-                last_error = e
-
-        raise last_error or RuntimeError(f"No parsers available for '{filename}'")
-
-    def list_parsers(self) -> list[str]:
-        """Get list of registered parser names."""
-        return list(self._parsers.keys())

hindsight-api-slim/hindsight_api/engine/providers/litellm_llm.py

@@ -1,380 +0,0 @@
-"""
-LiteLLM LLM provider for universal model support.
-
-This provider enables using 100+ LLM providers via the LiteLLM SDK, including:
-- AWS Bedrock (bedrock/anthropic.claude-3-5-sonnet-...)
-- Azure OpenAI (azure/gpt-4o)
-- Together AI (together_ai/meta-llama/...)
-- Any other LiteLLM-supported provider
-
-Uses litellm.acompletion() for async chat completions.
-Authentication for cloud providers (e.g., AWS Bedrock via boto3 credential chain)
-is handled automatically by LiteLLM.
-"""
-
-import asyncio
-import json
-import logging
-import time
-from typing import Any
-
-from hindsight_api.engine.llm_interface import LLMInterface, OutputTooLongError
-from hindsight_api.engine.response_models import LLMToolCall, LLMToolCallResult, TokenUsage
-from hindsight_api.metrics import get_metrics_collector
-
-logger = logging.getLogger(__name__)
-
-
-class LiteLLMLLM(LLMInterface):
-    """
-    LLM provider using the LiteLLM SDK for universal model support.
-
-    Supports any model accessible via litellm.acompletion(), including AWS Bedrock,
-    Azure OpenAI, Together AI, Fireworks AI, and more.
-
-    Model names follow LiteLLM conventions with provider prefixes:
-    - bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0
-    - azure/gpt-4o
-    - together_ai/meta-llama/Llama-3-70b-chat-hf
-    - fireworks_ai/accounts/fireworks/models/llama-v3p1-70b-instruct
-    """
-
-    def __init__(
-        self,
-        provider: str,
-        api_key: str,
-        base_url: str,
-        model: str,
-        reasoning_effort: str = "low",
-        timeout: float = 300.0,
-        **kwargs: Any,
-    ):
-        super().__init__(provider, api_key, base_url, model, reasoning_effort, **kwargs)
-        self.timeout = timeout
-        self._litellm: Any = None
-
-        try:
-            import litellm
-
-            self._litellm = litellm
-            # Suppress LiteLLM's verbose logging
-            litellm.suppress_debug_info = True  # type: ignore[assignment]
-            # Drop unsupported params instead of raising errors (e.g. tool_choice on some Bedrock models)
-            litellm.drop_params = True  # type: ignore[assignment]
-            logging.getLogger("LiteLLM").setLevel(logging.WARNING)
-            logger.info(f"LiteLLM SDK initialized for model: {self.model}")
-        except ImportError as e:
-            raise RuntimeError("LiteLLM SDK not installed. Run: uv add litellm or pip install litellm") from e
-
-    async def verify_connection(self) -> None:
-        try:
-            test_messages = [{"role": "user", "content": "test"}]
-            await self.call(
-                messages=test_messages,
-                max_completion_tokens=50,
-                temperature=0.0,
-                scope="verification",
-                max_retries=0,
-            )
-            logger.info("LiteLLM connection verified successfully")
-        except OutputTooLongError:
-            # Truncation is fine for verification — it means the connection works
-            logger.info("LiteLLM connection verified successfully (response truncated)")
-        except Exception as e:
-            logger.error(f"LiteLLM connection verification failed: {e}")
-            raise RuntimeError(f"Failed to verify LiteLLM connection: {e}") from e
-
-    def _build_common_kwargs(
-        self,
-        messages: list[dict[str, Any]],
-        max_completion_tokens: int | None = None,
-        temperature: float | None = None,
-    ) -> dict[str, Any]:
-        """Build common kwargs for litellm calls."""
-        kwargs: dict[str, Any] = {
-            "model": self.model,
-            "messages": messages,
-            "timeout": self.timeout,
-        }
-
-        if self.api_key:
-            kwargs["api_key"] = self.api_key
-        if self.base_url:
-            kwargs["api_base"] = self.base_url
-        if max_completion_tokens is not None:
-            kwargs["max_completion_tokens"] = max_completion_tokens
-        if temperature is not None:
-            kwargs["temperature"] = temperature
-
-        return kwargs
-
-    async def call(
-        self,
-        messages: list[dict[str, str]],
-        response_format: Any | None = None,
-        max_completion_tokens: int | None = None,
-        temperature: float | None = None,
-        scope: str = "memory",
-        max_retries: int = 10,
-        initial_backoff: float = 1.0,
-        max_backoff: float = 60.0,
-        skip_validation: bool = False,
-        strict_schema: bool = False,
-        return_usage: bool = False,
-    ) -> Any:
-        start_time = time.time()
-
-        call_kwargs = self._build_common_kwargs(messages, max_completion_tokens, temperature)
-
-        # Add JSON schema response format if provided
-        if response_format is not None and hasattr(response_format, "model_json_schema"):
-            schema = response_format.model_json_schema()
-            call_kwargs["response_format"] = {
-                "type": "json_schema",
-                "json_schema": {
-                    "name": response_format.__name__ if hasattr(response_format, "__name__") else "response",
-                    "schema": schema,
-                    "strict": strict_schema,
-                },
-            }
-
-        last_exception = None
-
-        for attempt in range(max_retries + 1):
-            try:
-                response = await self._litellm.acompletion(**call_kwargs)
-
-                content = response.choices[0].message.content or ""
-                finish_reason = response.choices[0].finish_reason
-
-                # Check for length-limited output
-                if finish_reason == "length":
-                    raise OutputTooLongError("LiteLLM response was truncated due to token limit")
-
-                if response_format is not None:
-                    # Strip markdown code fences if present
-                    clean_content = content
-                    if "```json" in content:
-                        clean_content = content.split("```json")[1].split("```")[0].strip()
-                    elif "```" in content:
-                        clean_content = content.split("```")[1].split("```")[0].strip()
-
-                    try:
-                        json_data = json.loads(clean_content)
-                    except json.JSONDecodeError:
-                        json_data = json.loads(content)
-
-                    if skip_validation:
-                        result = json_data
-                    else:
-                        result = response_format.model_validate(json_data)
-                else:
-                    result = content
-
-                # Extract usage
-                input_tokens = getattr(response.usage, "prompt_tokens", 0) or 0
-                output_tokens = getattr(response.usage, "completion_tokens", 0) or 0
-                total_tokens = input_tokens + output_tokens
-
-                # Record metrics
-                duration = time.time() - start_time
-                metrics = get_metrics_collector()
-                metrics.record_llm_call(
-                    provider=self.provider,
-                    model=self.model,
-                    scope=scope,
-                    duration=duration,
-                    input_tokens=input_tokens,
-                    output_tokens=output_tokens,
-                    success=True,
-                )
-
-                # Record trace span
-                from hindsight_api.tracing import _serialize_for_span, get_span_recorder
-
-                span_recorder = get_span_recorder()
-                span_recorder.record_llm_call(
-                    provider=self.provider,
-                    model=self.model,
-                    scope=scope,
-                    messages=messages,
-                    response_content=_serialize_for_span(result),
-                    input_tokens=input_tokens,
-                    output_tokens=output_tokens,
-                    duration=duration,
-                    finish_reason=finish_reason,
-                    error=None,
-                )
-
-                if duration > 10.0:
-                    logger.info(
-                        f"slow llm call: scope={scope}, model={self.provider}/{self.model}, "
-                        f"input_tokens={input_tokens}, output_tokens={output_tokens}, "
-                        f"time={duration:.3f}s"
-                    )
-
-                if return_usage:
-                    token_usage = TokenUsage(
-                        input_tokens=input_tokens,
-                        output_tokens=output_tokens,
-                        total_tokens=total_tokens,
-                    )
-                    return result, token_usage
-                return result
-
-            except OutputTooLongError:
-                raise
-
-            except json.JSONDecodeError as e:
-                last_exception = e
-                if attempt < max_retries:
-                    logger.warning("LiteLLM returned invalid JSON, retrying...")
-                    backoff = min(initial_backoff * (2**attempt), max_backoff)
-                    await asyncio.sleep(backoff)
-                    continue
-                else:
-                    logger.error(f"LiteLLM returned invalid JSON after {max_retries + 1} attempts")
-                    raise
-
-            except Exception as e:
-                error_str = str(e).lower()
-                # Fast fail on auth errors
-                if "401" in error_str or "403" in error_str or "unauthorized" in error_str:
-                    logger.error(f"LiteLLM auth error, not retrying: {e}")
-                    raise
-
-                last_exception = e
-                if attempt < max_retries:
-                    # Retry on rate limits, connection errors, server errors
-                    is_retryable = any(
-                        keyword in error_str
-                        for keyword in ("rate", "limit", "timeout", "connection", "500", "502", "503", "529")
-                    )
-                    if is_retryable:
-                        backoff = min(initial_backoff * (2**attempt), max_backoff)
-                        jitter = backoff * 0.2 * (2 * (time.time() % 1) - 1)
-                        await asyncio.sleep(backoff + jitter)
-                        continue
-
-                logger.error(f"LiteLLM API error after {attempt + 1} attempts: {e}")
-                raise
-
-        if last_exception:
-            raise last_exception
-        raise RuntimeError("LiteLLM call failed after all retries")
-
-    async def call_with_tools(
-        self,
-        messages: list[dict[str, Any]],
-        tools: list[dict[str, Any]],
-        max_completion_tokens: int | None = None,
-        temperature: float | None = None,
-        scope: str = "tools",
-        max_retries: int = 5,
-        initial_backoff: float = 1.0,
-        max_backoff: float = 30.0,
-        tool_choice: str | dict[str, Any] = "auto",
-    ) -> LLMToolCallResult:
-        start_time = time.time()
-
-        call_kwargs = self._build_common_kwargs(messages, max_completion_tokens, temperature)
-        call_kwargs["tools"] = tools
-        call_kwargs["tool_choice"] = tool_choice
-
-        last_exception = None
-        for attempt in range(max_retries + 1):
-            try:
-                response = await self._litellm.acompletion(**call_kwargs)
-
-                message = response.choices[0].message
-                content = message.content
-                finish_reason = response.choices[0].finish_reason
-
-                # Extract tool calls
-                tool_calls: list[LLMToolCall] = []
-                if message.tool_calls:
-                    for tc in message.tool_calls:
-                        arguments = tc.function.arguments
-                        if isinstance(arguments, str):
-                            arguments = json.loads(arguments)
-                        tool_calls.append(
-                            LLMToolCall(
-                                id=tc.id,
-                                name=tc.function.name,
-                                arguments=arguments,
-                            )
-                        )
-
-                # Extract usage
-                input_tokens = getattr(response.usage, "prompt_tokens", 0) or 0
-                output_tokens = getattr(response.usage, "completion_tokens", 0) or 0
-
-                # Record metrics
-                duration = time.time() - start_time
-                metrics = get_metrics_collector()
-                metrics.record_llm_call(
-                    provider=self.provider,
-                    model=self.model,
-                    scope=scope,
-                    duration=duration,
-                    input_tokens=input_tokens,
-                    output_tokens=output_tokens,
-                    success=True,
-                )
-
-                # Record trace span
-                from hindsight_api.tracing import get_span_recorder
-
-                span_recorder = get_span_recorder()
-                tool_calls_dict = (
-                    [{"id": tc.id, "name": tc.name, "arguments": tc.arguments} for tc in tool_calls]
-                    if tool_calls
-                    else None
-                )
-                span_recorder.record_llm_call(
-                    provider=self.provider,
-                    model=self.model,
-                    scope=scope,
-                    messages=messages,
-                    response_content=content,
-                    input_tokens=input_tokens,
-                    output_tokens=output_tokens,
-                    duration=duration,
-                    finish_reason=finish_reason,
-                    error=None,
-                    tool_calls=tool_calls_dict,
-                )
-
-                return LLMToolCallResult(
-                    content=content,
-                    tool_calls=tool_calls,
-                    finish_reason=finish_reason or ("tool_calls" if tool_calls else "stop"),
-                    input_tokens=input_tokens,
-                    output_tokens=output_tokens,
-                )
-
-            except Exception as e:
-                error_str = str(e).lower()
-                if "401" in error_str or "403" in error_str or "unauthorized" in error_str:
-                    raise
-
-                last_exception = e
-                if attempt < max_retries:
-                    is_retryable = any(
-                        keyword in error_str
-                        for keyword in ("rate", "limit", "timeout", "connection", "500", "502", "503", "529")
-                    )
-                    if is_retryable:
-                        await asyncio.sleep(min(initial_backoff * (2**attempt), max_backoff))
-                        continue
-
-                logger.error(f"LiteLLM tool call error after {attempt + 1} attempts: {e}")
-                raise
-
-        if last_exception:
-            raise last_exception
-        raise RuntimeError("LiteLLM tool call failed after all retries")
-
-    async def cleanup(self) -> None:
-        """Clean up resources."""
-        pass

hindsight-api-slim/hindsight_api/engine/providers/none_llm.py

@@ -1,78 +0,0 @@
-"""
-No-op LLM provider for chunk-only storage mode.
-
-When the LLM provider is set to "none", the system operates without any LLM dependency.
-Retain uses chunks mode (no fact extraction), and reflect/consolidation are disabled.
-This provider acts as a safety net — if any code path unexpectedly tries to call the LLM,
-it raises a clear error instead of a confusing connection failure.
-"""
-
-import logging
-from typing import Any
-
-from ..llm_interface import LLMInterface
-from ..response_models import LLMToolCallResult
-
-logger = logging.getLogger(__name__)
-
-
-class LLMNotAvailableError(Exception):
-    """Raised when an operation requires an LLM but the provider is set to 'none'."""
-
-    pass
-
-
-class NoneLLM(LLMInterface):
-    """
-    No-op LLM provider that rejects all LLM calls.
-
-    Used when HINDSIGHT_API_LLM_PROVIDER=none to run Hindsight as a chunk store
-    with semantic search but without LLM-based features (fact extraction, reflect,
-    consolidation).
-    """
-
-    async def verify_connection(self) -> None:
-        """No-op — no LLM connection to verify."""
-        logger.debug("NoneLLM: no LLM connection to verify (provider=none)")
-
-    async def call(
-        self,
-        messages: list[dict[str, str]],
-        response_format: Any | None = None,
-        max_completion_tokens: int | None = None,
-        temperature: float | None = None,
-        scope: str = "memory",
-        max_retries: int = 10,
-        initial_backoff: float = 1.0,
-        max_backoff: float = 60.0,
-        skip_validation: bool = False,
-        strict_schema: bool = False,
-        return_usage: bool = False,
-    ) -> Any:
-        """Raise LLMNotAvailableError — no LLM is configured."""
-        raise LLMNotAvailableError(
-            "LLM provider is set to 'none'. This operation requires an LLM. "
-            "Set HINDSIGHT_API_LLM_PROVIDER to a real provider (e.g., openai, anthropic, gemini)."
-        )
-
-    async def call_with_tools(
-        self,
-        messages: list[dict[str, Any]],
-        tools: list[dict[str, Any]],
-        max_completion_tokens: int | None = None,
-        temperature: float | None = None,
-        scope: str = "tools",
-        max_retries: int = 5,
-        initial_backoff: float = 1.0,
-        max_backoff: float = 30.0,
-        tool_choice: str | dict[str, Any] = "auto",
-    ) -> LLMToolCallResult:
-        """Raise LLMNotAvailableError — no LLM is configured."""
-        raise LLMNotAvailableError(
-            "LLM provider is set to 'none'. This operation requires an LLM. "
-            "Set HINDSIGHT_API_LLM_PROVIDER to a real provider (e.g., openai, anthropic, gemini)."
-        )
-
-    async def cleanup(self) -> None:
-        """No-op — nothing to clean up."""
-        pass

hindsight-api-slim/hindsight_api/engine/retain/entity_labels.py

@@ -1,194 +0,0 @@
-"""
-Entity labels models and helpers for retain pipeline.
-
-Defines a controlled vocabulary of key:value classification labels
-(e.g., 'pedagogy:scaffolding', 'interest:active') that are extracted
-at retain time and stored as entities.
-"""
-
-from typing import Literal
-
-from pydantic import BaseModel, Field, create_model
-
-
-class LabelValue(BaseModel):
-    """A single allowed value for a label group."""
-
-    value: str
-    description: str = ""
-
-
-class LabelGroup(BaseModel):
-    """A label group (dimension) with its type and allowed values."""
-
-    key: str
-    description: str = ""
-    type: Literal["value", "multi-values", "text"] = "value"
-    optional: bool = True
-    tag: bool = False
-    values: list[LabelValue] = []
-
-
-class EntityLabelsConfig(BaseModel):
-    """Entity labels configuration for a bank (controlled vocabulary)."""
-
-    attributes: list[LabelGroup] = []
-
-
-def parse_entity_labels(raw: dict | list | None) -> EntityLabelsConfig | None:
-    """
-    Parse raw entity labels config into EntityLabelsConfig.
-
-    Accepts:
-    - None → returns None
-    - list → list of attribute dicts (each may use legacy free_values/multi_value or new type field)
-    - dict → {attributes: [...]}
-
-    Legacy migration (backward-compat):
-    - free_values=True          → type="text"
-    - multi_value=True          → type="multi-values"
-    - neither / free_values=False → type="value"
-
-    Args:
-        raw: Raw entity labels config from bank config
-
-    Returns:
-        EntityLabelsConfig or None if raw is None/empty
-    """
-    if raw is None:
-        return None
-
-    if isinstance(raw, list):
-        if not raw:
-            return None
-        attributes = [LabelGroup.model_validate(_migrate_label_group(a)) for a in raw]
-        return EntityLabelsConfig(attributes=attributes)
-
-    if isinstance(raw, dict):
-        attrs_raw = raw.get("attributes", [])
-        if not attrs_raw:
-            return None
-        attributes = [LabelGroup.model_validate(_migrate_label_group(a)) for a in attrs_raw]
-        return EntityLabelsConfig(attributes=attributes)
-
-    return None
-
-
-def _migrate_label_group(raw: dict) -> dict:
-    """Migrate legacy free_values/multi_value fields to the new type field."""
-    if not isinstance(raw, dict) or "type" in raw:
-        return raw
-    patched = dict(raw)
-    if patched.get("free_values"):
-        patched["type"] = "text"
-    elif patched.get("multi_value"):
-        patched["type"] = "multi-values"
-    else:
-        patched["type"] = "value"
-    # Remove legacy keys so Pydantic doesn't error on unknown fields
-    patched.pop("free_values", None)
-    patched.pop("multi_value", None)
-    return patched
-
-
-def build_labels_model(labels_cfg: EntityLabelsConfig) -> type[BaseModel] | None:
-    """
-    Build a dynamic Pydantic model for structured label extraction.
-
-    Each LabelGroup becomes a typed field based on its type:
-    - type="text"                        → str | None  (always optional)
-    - type="value",   optional=True      → Literal["v1","v2"] | None
-    - type="value",   optional=False     → Literal["v1","v2"]  (required)
-    - type="multi-values"                → list[Literal["v1","v2"]]
-
-    Args:
-        labels_cfg: Parsed EntityLabelsConfig
-
-    Returns:
-        Dynamic Pydantic model class, or None if no groups defined
-    """
-    fields: dict = {}
-    for group in labels_cfg.attributes:
-        if not group.key:
-            continue
-        description = group.description or group.key
-
-        if group.type == "text":
-            # Free-form: any string value accepted, always optional
-            fields[group.key] = (str | None, Field(default=None, description=description))
-        else:
-            # Enum-constrained: must have defined values
-            if not group.values:
-                continue
-            values = tuple(v.value for v in group.values if v.value)
-            if not values:
-                continue
-            # Literal[("v1", "v2")] is equivalent to Literal["v1", "v2"] in Python 3.11+
-            literal_type = Literal[values]  # type: ignore[valid-type]
-            if group.type == "multi-values":
-                fields[group.key] = (
-                    list[literal_type],  # type: ignore[valid-type]
-                    Field(default_factory=list, description=description),
-                )
-            elif group.optional:
-                fields[group.key] = (
-                    literal_type | None,  # type: ignore[valid-type]
-                    Field(default=None, description=description),
-                )
-            else:
-                fields[group.key] = (
-                    literal_type,  # type: ignore[valid-type]
-                    Field(description=description),
-                )
-
-    if not fields:
-        return None
-
-    return create_model("Labels", **fields)
-
-
-def is_label_entity(text: str, labels_cfg: EntityLabelsConfig, labels_lookup: set[str]) -> bool:
-    """
-    Return True if entity text belongs to any configured label group.
-
-    For enum groups: checks the pre-built lookup set.
-    For text groups: checks that the text starts with a known key prefix.
-    """
-    if text.lower() in labels_lookup:
-        return True
-    for group in labels_cfg.attributes:
-        if group.type == "text" and group.key and text.lower().startswith(f"{group.key.lower()}:"):
-            return True
-    return False
-
-
-def build_labels_lookup(labels_cfg: EntityLabelsConfig | list | None) -> set[str]:
-    """
-    Build a set of valid 'key:value' label strings (lowercase) for fast lookup.
-
-    Accepts either EntityLabelsConfig or raw list/None for backwards compatibility.
-
-    Args:
-        labels_cfg: EntityLabelsConfig, raw list of attribute dicts, or None
-
-    Returns:
-        Set of lowercase 'key:value' strings
-    """
-    if labels_cfg is None:
-        return set()
-
-    # Accept raw list/dict for backwards compatibility
-    if not isinstance(labels_cfg, EntityLabelsConfig):
-        parsed = parse_entity_labels(labels_cfg)
-        if parsed is None:
-            return set()
-        labels_cfg = parsed
-
-    valid = set()
-    for group in labels_cfg.attributes:
-        if group.type == "text":
-            continue  # No fixed vocabulary — all values accepted in post-processing
-        for v in group.values:
-            if group.key and v.value:
-                valid.add(f"{group.key}:{v.value}".lower())
-    return valid

hindsight-api-slim/hindsight_api/engine/retain/entity_processing.py

@@ -1,162 +0,0 @@
-"""
-Entity processing for retain pipeline.
-
-Handles entity extraction, resolution, and link creation for stored facts.
-"""
-
-import logging
-
-from . import link_utils
-from .types import EntityLink, ProcessedFact
-
-logger = logging.getLogger(__name__)
-
-
-def _prepare_facts_for_entity_processing(
-    facts: list[ProcessedFact],
-    user_entities_per_content: dict[int, list[dict]] | None = None,
-) -> tuple[list[str], list, list[list[dict]]]:
-    """
-    Extract fact texts, dates, and merged entity lists from ProcessedFact objects.
-
-    Returns:
-        Tuple of (fact_texts, fact_dates, entities_per_fact)
-    """
-    user_entities_per_content = user_entities_per_content or {}
-
-    fact_texts = [fact.fact_text for fact in facts]
-    fact_dates = [fact.occurred_start if fact.occurred_start is not None else fact.mentioned_at for fact in facts]
-
-    entities_per_fact = []
-    for fact in facts:
-        llm_entities = [{"text": entity.name, "type": "CONCEPT"} for entity in (fact.entities or [])]
-
-        user_entities = user_entities_per_content.get(fact.content_index, [])
-
-        seen_texts = {e["text"].lower() for e in llm_entities}
-        for user_entity in user_entities:
-            if user_entity["text"].lower() not in seen_texts:
-                llm_entities.append(
-                    {
-                        "text": user_entity["text"],
-                        "type": user_entity.get("type", "CONCEPT"),
-                    }
-                )
-                seen_texts.add(user_entity["text"].lower())
-
-        entities_per_fact.append(llm_entities)
-
-    return fact_texts, fact_dates, entities_per_fact
-
-
-async def resolve_entities(
-    entity_resolver,
-    conn,
-    bank_id: str,
-    unit_ids: list[str],
-    facts: list[ProcessedFact],
-    log_buffer: list[str] = None,
-    user_entities_per_content: dict[int, list[dict]] = None,
-    entity_labels: list | None = None,
-) -> tuple[list[str], list[tuple], dict[str, list[str]]]:
-    """
-    Phase 1: Resolve entity names to canonical IDs (read-heavy).
-
-    Should be called on a SEPARATE connection OUTSIDE the main write transaction
-    to avoid holding the transaction open during expensive trigram scans.
-
-    Args:
-        entity_resolver: EntityResolver instance
-        conn: Database connection (separate from the main write transaction)
-        bank_id: Bank identifier
-        unit_ids: Placeholder unit IDs (used only for grouping)
-        facts: List of ProcessedFact objects
-        log_buffer: Optional buffer for detailed logging
-        user_entities_per_content: Dict mapping content_index to user-provided entities
-        entity_labels: Optional entity label taxonomy
-
-    Returns:
-        Tuple of (resolved_entity_ids, entity_to_unit, unit_to_entity_ids)
-        to pass to build_entity_links().
-    """
-    if not unit_ids or not facts:
-        return [], [], {}
-
-    if len(unit_ids) != len(facts):
-        raise ValueError(f"Mismatch between unit_ids ({len(unit_ids)}) and facts ({len(facts)})")
-
-    fact_texts, fact_dates, entities_per_fact = _prepare_facts_for_entity_processing(facts, user_entities_per_content)
-
-    return await link_utils.resolve_entities_only(
-        entity_resolver,
-        conn,
-        bank_id,
-        unit_ids,
-        fact_texts,
-        "",  # context (not used in current implementation)
-        fact_dates,
-        entities_per_fact,
-        log_buffer,
-        entity_labels=entity_labels,
-    )
-
-
-async def build_entity_links(
-    entity_resolver,
-    conn,
-    bank_id: str,
-    unit_ids: list[str],
-    resolved_entity_ids: list[str],
-    entity_to_unit: list[tuple],
-    unit_to_entity_ids: dict[str, list[str]],
-    log_buffer: list[str] = None,
-    skip_unit_entities_insert: bool = False,
-) -> list[EntityLink]:
-    """
-    Build entity links for UI graph visualization.
-
-    Queries unit_entities to find shared entities between new and existing units,
-    then generates EntityLink objects. When called from Phase 3 (post-transaction),
-    set skip_unit_entities_insert=True since unit_entities were already inserted
-    in Phase 2.
-
-    Args:
-        entity_resolver: EntityResolver instance
-        conn: Database connection
-        bank_id: Bank identifier
-        unit_ids: Actual unit IDs (must already be inserted in the DB)
-        resolved_entity_ids: From resolve_entities()
-        entity_to_unit: From resolve_entities()
-        unit_to_entity_ids: From resolve_entities()
-        log_buffer: Optional buffer for detailed logging
-        skip_unit_entities_insert: Skip unit_entities INSERT (already done in Phase 2)
-
-    Returns:
-        List of EntityLink objects for batch insertion
-    """
-    return await link_utils.build_entity_links_from_resolved(
-        entity_resolver,
-        conn,
-        bank_id,
-        unit_ids,
-        resolved_entity_ids,
-        entity_to_unit,
-        unit_to_entity_ids,
-        log_buffer,
-        skip_unit_entities_insert=skip_unit_entities_insert,
-    )
-
-
-async def insert_entity_links_batch(conn, entity_links: list[EntityLink], bank_id: str) -> None:
-    """
-    Insert entity links in batch.
-
-    Args:
-        conn: Database connection
-        entity_links: List of EntityLink objects
-        bank_id: Bank identifier (stored directly on memory_links for fast filtering)
-    """
-    if not entity_links:
-        return
-
-    await link_utils.insert_entity_links_batch(conn, entity_links, bank_id)

hindsight-api-slim/hindsight_api/engine/search/graph_retrieval.py

@@ -1,67 +0,0 @@
-"""
-Graph retrieval strategies for memory recall.
-
-This module provides an abstraction for graph-based memory retrieval,
-allowing different algorithms to be swapped without changing the rest
-of the recall pipeline.
-"""
-
-import logging
-from abc import ABC, abstractmethod
-
-from .tags import TagGroup, TagsMatch
-from .types import GraphRetrievalTimings, RetrievalResult
-
-logger = logging.getLogger(__name__)
-
-
-class GraphRetriever(ABC):
-    """
-    Abstract base class for graph-based memory retrieval.
-
-    Implementations traverse the memory graph (entity links, temporal links,
-    causal links) to find relevant facts that might not be found by
-    semantic or keyword search alone.
-    """
-
-    @property
-    @abstractmethod
-    def name(self) -> str:
-        """Return identifier for this retrieval strategy (e.g., 'link_expansion')."""
-        pass
-
-    @abstractmethod
-    async def retrieve(
-        self,
-        pool,
-        query_embedding_str: str,
-        bank_id: str,
-        fact_type: str,
-        budget: int,
-        query_text: str | None = None,
-        semantic_seeds: list[RetrievalResult] | None = None,
-        temporal_seeds: list[RetrievalResult] | None = None,
-        adjacency=None,  # TypedAdjacency, optional pre-loaded graph
-        tags: list[str] | None = None,  # Visibility scope tags for filtering
-        tags_match: TagsMatch = "any",  # How to match tags: 'any' (OR) or 'all' (AND)
-        tag_groups: list[TagGroup] | None = None,  # Compound boolean tag filter groups
-    ) -> tuple[list[RetrievalResult], GraphRetrievalTimings | None]:
-        """
-        Retrieve relevant facts via graph traversal.
-
-        Args:
-            pool: Database connection pool
-            query_embedding_str: Query embedding as string (for finding entry points)
-            bank_id: Memory bank identifier
-            fact_type: Fact type to filter ('world', 'experience', 'observation')
-            budget: Maximum number of nodes to explore/return
-            query_text: Original query text (optional, for some strategies)
-            semantic_seeds: Pre-computed semantic entry points (from semantic retrieval)
-            temporal_seeds: Pre-computed temporal entry points (from temporal retrieval)
-            adjacency: Pre-loaded typed adjacency graph (optional)
-            tags: Optional list of tags for visibility filtering (OR matching)
-
-        Returns:
-            Tuple of (List of RetrievalResult with activation scores, optional timing info)
-        """
-        pass

hindsight-api-slim/hindsight_api/engine/search/link_expansion_retrieval.py

@@ -1,553 +0,0 @@
-"""
-Link Expansion graph retrieval.
-
-Expands from semantic/temporal seeds through three parallel, first-class signals
-stored in memory_links:
-
-1. Entity links  — query-time self-join through unit_entities. Score = number of distinct
-                   shared entities between the seed set and each candidate, computed via
-                   COUNT(DISTINCT entity_id). Uses a LATERAL per-entity cap
-                   (graph_per_entity_limit, default 200) to prevent high-fanout entities
-                   from exploding the self-join intermediate rows.
-2. Semantic links — precomputed kNN graph (each new fact linked to its top-5 most
-                    similar existing facts at insert time, similarity >= 0.7). Checked
-                    in both directions since the graph is not symmetric. Score = weight.
-3. Causal links  — explicit causal chains (causes/caused_by/enables/prevents).
-                   Score = weight + 1.0 (boosted as highest-quality signal).
-
-Entity expansion is bounded by graph_per_entity_limit (LATERAL cap per entity).
-A timeout fallback (graph_expansion_timeout) drops entity expansion entirely if the
-query still exceeds the budget.
-
-For non-observation fact types the three expansions are issued as a single CTE query
-(one roundtrip, one connection) with a `source` discriminator column so the Python
-merge step can apply per-signal score transformations.
-"""
-
-import asyncio
-import logging
-import math
-import time
-
-from ...config import get_config
-from ..db_utils import acquire_with_retry
-from ..memory_engine import fq_table
-from .graph_retrieval import GraphRetriever
-from .tags import TagGroup, TagsMatch, filter_results_by_tag_groups, filter_results_by_tags
-from .types import GraphRetrievalTimings, RetrievalResult
-
-logger = logging.getLogger(__name__)
-
-
-async def _find_semantic_seeds(
-    conn,
-    query_embedding_str: str,
-    bank_id: str,
-    fact_type: str,
-    limit: int = 20,
-    threshold: float = 0.3,
-    tags: list[str] | None = None,
-    tags_match: TagsMatch = "any",
-    tag_groups: list[TagGroup] | None = None,
-) -> list[RetrievalResult]:
-    """Find semantic seeds via embedding search."""
-    from .tags import build_tag_groups_where_clause, build_tags_where_clause_simple
-
-    tags_clause = build_tags_where_clause_simple(tags, 6, match=tags_match)
-    tag_groups_param_start = 6 + (1 if tags else 0)
-    groups_clause, groups_params, _ = build_tag_groups_where_clause(tag_groups, tag_groups_param_start)
-    params = [query_embedding_str, bank_id, fact_type, threshold, limit]
-    if tags:
-        params.append(tags)
-    params.extend(groups_params)
-
-    rows = await conn.fetch(
-        f"""
-        SELECT id, text, context, event_date, occurred_start, occurred_end,
-               mentioned_at, fact_type, document_id, chunk_id, tags, proof_count,
-               1 - (embedding <=> $1::vector) AS similarity
-        FROM {fq_table("memory_units")}
-        WHERE bank_id = $2
-          AND embedding IS NOT NULL
-          AND fact_type = $3
-          AND (1 - (embedding <=> $1::vector)) >= $4
-          {tags_clause}
-          {groups_clause}
-        ORDER BY embedding <=> $1::vector
-        LIMIT $5
-        """,
-        *params,
-    )
-    return [RetrievalResult.from_db_row(dict(r)) for r in rows]
-
-
-class LinkExpansionRetriever(GraphRetriever):
-    """
-    Graph retrieval via direct link expansion from seeds.
-
-    Runs three expansions through precomputed memory_links: entity co-occurrence,
-    semantic kNN, and causal chains, all bounded at retain time.
-
-    For non-observation fact types the three expansions are issued as a single CTE
-    query (one roundtrip, one connection slot) with a `source` discriminator column.
-    The Python merge step applies per-signal score transformations.
-    """
-
-    def __init__(
-        self,
-        causal_weight_threshold: float = 0.3,
-    ):
-        """
-        Args:
-            causal_weight_threshold: Minimum weight for causal links to follow.
-        """
-        self.causal_weight_threshold = causal_weight_threshold
-
-    @property
-    def name(self) -> str:
-        return "link_expansion"
-
-    async def retrieve(
-        self,
-        pool,
-        query_embedding_str: str,
-        bank_id: str,
-        fact_type: str,
-        budget: int,
-        query_text: str | None = None,
-        semantic_seeds: list[RetrievalResult] | None = None,
-        temporal_seeds: list[RetrievalResult] | None = None,
-        adjacency=None,
-        tags: list[str] | None = None,
-        tags_match: TagsMatch = "any",
-        tag_groups: list[TagGroup] | None = None,
-    ) -> tuple[list[RetrievalResult], GraphRetrievalTimings | None]:
-        """
-        Retrieve facts by expanding links from seeds.
-
-        Args:
-            pool: Database connection pool
-            query_embedding_str: Query embedding as string
-            bank_id: Memory bank ID
-            fact_type: Fact type to filter
-            budget: Maximum results to return
-            query_text: Original query text (unused)
-            semantic_seeds: Pre-computed semantic entry points
-            temporal_seeds: Pre-computed temporal entry points
-            adjacency: Unused, kept for interface compatibility
-            tags: Optional list of tags for visibility filtering
-
-        Returns:
-            Tuple of (results, timings)
-        """
-        start_time = time.time()
-        timings = GraphRetrievalTimings(fact_type=fact_type)
-
-        async with acquire_with_retry(pool) as conn:
-            # Find seeds if not provided
-            if semantic_seeds:
-                all_seeds = list(semantic_seeds)
-            else:
-                seeds_start = time.time()
-                all_seeds = await _find_semantic_seeds(
-                    conn,
-                    query_embedding_str,
-                    bank_id,
-                    fact_type,
-                    limit=20,
-                    threshold=0.3,
-                    tags=tags,
-                    tags_match=tags_match,
-                    tag_groups=tag_groups,
-                )
-                timings.seeds_time = time.time() - seeds_start
-                logger.debug(
-                    f"[LinkExpansion] Found {len(all_seeds)} semantic seeds for fact_type={fact_type} "
-                    f"(tags={tags}, tags_match={tags_match})"
-                )
-
-            if temporal_seeds:
-                all_seeds.extend(temporal_seeds)
-
-            if not all_seeds:
-                return [], timings
-
-            seed_ids = list({s.id for s in all_seeds})
-            timings.pattern_count = len(seed_ids)
-
-            query_start = time.time()
-
-            if fact_type == "observation":
-                entity_rows, semantic_rows, causal_rows = await self._expand_observations(conn, seed_ids, budget)
-            else:
-                entity_rows, semantic_rows, causal_rows = await self._expand_combined(conn, seed_ids, fact_type, budget)
-
-            timings.edge_load_time = time.time() - query_start
-            timings.db_queries = 1
-            timings.edge_count = len(entity_rows) + len(semantic_rows) + len(causal_rows)
-
-        # Merge results with additive intra-score: entity + semantic + causal ∈ [0, 3].
-        #
-        # Entity score: tanh(count × 0.5) maps shared-entity count to [0, 1]:
-        #   1 entity → 0.46,  2 → 0.76,  3 → 0.91,  4 → 0.96  (saturates naturally)
-        # Semantic score: similarity weight, already ∈ [0.7, 1.0].
-        # Causal score:   link weight, already ∈ [0, 1].
-        #
-        # Facts appearing in multiple signals accumulate higher scores, rewarding
-        # convergent evidence. The outer RRF uses rank position from this sorted list.
-        entity_scores: dict[str, float] = {}
-        semantic_scores: dict[str, float] = {}
-        causal_scores: dict[str, float] = {}
-        row_map: dict[str, dict] = {}
-
-        for row in entity_rows:
-            fact_id = str(row["id"])
-            entity_scores[fact_id] = math.tanh(row["score"] * 0.5)
-            row_map[fact_id] = dict(row)
-
-        for row in semantic_rows:
-            fact_id = str(row["id"])
-            semantic_scores[fact_id] = max(semantic_scores.get(fact_id, 0.0), row["score"])
-            row_map.setdefault(fact_id, dict(row))
-
-        for row in causal_rows:
-            fact_id = str(row["id"])
-            causal_scores[fact_id] = max(causal_scores.get(fact_id, 0.0), row["score"])
-            row_map.setdefault(fact_id, dict(row))
-
-        all_ids = set(entity_scores) | set(semantic_scores) | set(causal_scores)
-        score_map = {
-            fid: entity_scores.get(fid, 0.0) + semantic_scores.get(fid, 0.0) + causal_scores.get(fid, 0.0)
-            for fid in all_ids
-        }
-
-        sorted_ids = sorted(score_map.keys(), key=lambda x: score_map[x], reverse=True)[:budget]
-        rows = [row_map[fact_id] for fact_id in sorted_ids]
-
-        results = []
-        for row in rows:
-            result = RetrievalResult.from_db_row(dict(row))
-            result.activation = row["score"]
-            results.append(result)
-
-        if tags:
-            results = filter_results_by_tags(results, tags, match=tags_match)
-
-        if tag_groups:
-            results = filter_results_by_tag_groups(results, tag_groups)
-
-        timings.result_count = len(results)
-        timings.traverse = time.time() - start_time
-
-        logger.debug(
-            f"LinkExpansion: {len(results)} results from {len(seed_ids)} seeds "
-            f"in {timings.traverse * 1000:.1f}ms (query: {timings.edge_load_time * 1000:.1f}ms)"
-        )
-
-        return results, timings
-
-    async def _expand_combined(
-        self,
-        conn,
-        seed_ids: list,
-        fact_type: str,
-        budget: int,
-    ) -> tuple[list, list, list]:
-        """
-        Single-roundtrip CTE query combining entity, semantic, and causal expansions.
-
-        Uses a `source` discriminator column so the caller can apply per-signal
-        score transformations.  The three CTEs share one connection slot — important
-        for asyncpg which does not allow concurrent queries on the same connection.
-
-        Index coverage (requires migration d2e3f4a5b6c7):
-          entity:   idx_memory_links_entity_covering (from_unit_id) INCLUDE (to_unit_id, entity_id)
-                    WHERE link_type = 'entity'  → index-only scan, no heap reads
-          semantic incoming:
-                    idx_memory_links_to_type_weight (to_unit_id, link_type, weight DESC)
-                    → replaces costly BitmapAnd of two separate scans
-        """
-        config = get_config()
-        ml = fq_table("memory_links")
-        mu = fq_table("memory_units")
-        ue = fq_table("unit_entities")
-
-        per_entity_limit = config.link_expansion_per_entity_limit
-
-        # Entity CTE with LATERAL fanout cap.
-        # Every seed entity (including high-frequency ones) is kept, but each
-        # entity's expansion is capped to per_entity_limit target units.  The
-        # LATERAL subquery orders by unit_id DESC so the most recently inserted
-        # units are preferred (a recency proxy that is free — it rides the PK
-        # index with no extra sort).
-        entity_cte = f"""
-            seed_entities AS (
-                SELECT DISTINCT ue.entity_id
-                FROM {ue} ue
-                WHERE ue.unit_id = ANY($1::uuid[])
-            ),
-            entity_expanded AS (
-                SELECT mu.id, mu.text, mu.context, mu.event_date, mu.occurred_start,
-                       mu.occurred_end, mu.mentioned_at,
-                       mu.fact_type, mu.document_id, mu.chunk_id, mu.tags, mu.proof_count,
-                       COUNT(DISTINCT se.entity_id)::float AS score,
-                       'entity'::text AS source
-                FROM seed_entities se
-                CROSS JOIN LATERAL (
-                    SELECT ue_target.unit_id
-                    FROM {ue} ue_target
-                    WHERE ue_target.entity_id = se.entity_id
-                      AND ue_target.unit_id != ALL($1::uuid[])
-                    ORDER BY ue_target.unit_id DESC
-                    LIMIT {per_entity_limit}
-                ) t
-                JOIN {mu} mu ON mu.id = t.unit_id
-                WHERE mu.fact_type = $2
-                GROUP BY mu.id
-                ORDER BY score DESC
-                LIMIT $3
-            )"""
-
-        semantic_causal_cte = f"""
-            semantic_expanded AS (
-                -- Semantic kNN: both outgoing (seeds → their kNN at insert time) and
-                -- incoming (facts inserted after seeds that found seeds as kNN).
-                -- Score = max similarity weight across both directions.
-                SELECT
-                    id, text, context, event_date, occurred_start,
-                    occurred_end, mentioned_at,
-                    fact_type, document_id, chunk_id, tags, proof_count,
-                    MAX(weight) AS score,
-                    'semantic'::text AS source
-                FROM (
-                    SELECT
-                        mu.id, mu.text, mu.context, mu.event_date, mu.occurred_start,
-                        mu.occurred_end, mu.mentioned_at,
-                        mu.fact_type, mu.document_id, mu.chunk_id, mu.tags, mu.proof_count,
-                        ml.weight
-                    FROM {ml} ml
-                    JOIN {mu} mu ON mu.id = ml.to_unit_id
-                    WHERE ml.from_unit_id = ANY($1::uuid[])
-                      AND ml.link_type = 'semantic'
-                      AND mu.fact_type = $2
-                      AND mu.id != ALL($1::uuid[])
-                    UNION ALL
-                    SELECT
-                        mu.id, mu.text, mu.context, mu.event_date, mu.occurred_start,
-                        mu.occurred_end, mu.mentioned_at,
-                        mu.fact_type, mu.document_id, mu.chunk_id, mu.tags, mu.proof_count,
-                        ml.weight
-                    FROM {ml} ml
-                    JOIN {mu} mu ON mu.id = ml.from_unit_id
-                    WHERE ml.to_unit_id = ANY($1::uuid[])
-                      AND ml.link_type = 'semantic'
-                      AND mu.fact_type = $2
-                      AND mu.id != ALL($1::uuid[])
-                ) sem_raw
-                GROUP BY id, text, context, event_date, occurred_start,
-                         occurred_end, mentioned_at,
-                         fact_type, document_id, chunk_id, tags, proof_count
-                ORDER BY score DESC
-                LIMIT $3
-            ),
-            causal_expanded AS (
-                -- Causal chains: explicit causes/enables/prevents links from seeds.
-                -- DISTINCT ON handles the case where a seed has multiple causal links
-                -- to the same target; best weight wins.
-                SELECT DISTINCT ON (mu.id)
-                    mu.id, mu.text, mu.context, mu.event_date, mu.occurred_start,
-                    mu.occurred_end, mu.mentioned_at,
-                    mu.fact_type, mu.document_id, mu.chunk_id, mu.tags, mu.proof_count,
-                    ml.weight AS score,
-                    'causal'::text AS source
-                FROM {ml} ml
-                JOIN {mu} mu ON ml.to_unit_id = mu.id
-                WHERE ml.from_unit_id = ANY($1::uuid[])
-                  AND ml.link_type IN ('causes', 'caused_by', 'enables', 'prevents')
-                  AND ml.weight >= $4
-                  AND mu.fact_type = $2
-                ORDER BY mu.id, ml.weight DESC
-                LIMIT $3
-            )"""
-
-        full_query = f"""
-            WITH {entity_cte},
-            {semantic_causal_cte}
-            SELECT * FROM entity_expanded
-            UNION ALL
-            SELECT * FROM semantic_expanded
-            UNION ALL
-            SELECT * FROM causal_expanded
-            """
-
-        params = [seed_ids, fact_type, budget, self.causal_weight_threshold]
-
-        try:
-            all_rows = await asyncio.wait_for(
-                conn.fetch(full_query, *params),
-                timeout=config.link_expansion_timeout,
-            )
-        except asyncio.TimeoutError:
-            logger.warning(
-                f"[LinkExpansion] Entity expansion timed out after {config.link_expansion_timeout}s "
-                f"for fact_type={fact_type}, falling back to semantic+causal only"
-            )
-            fallback_query = f"""
-                WITH {semantic_causal_cte}
-                SELECT * FROM semantic_expanded
-                UNION ALL
-                SELECT * FROM causal_expanded
-                """
-            all_rows = await conn.fetch(fallback_query, *params)
-
-        entity_rows = [r for r in all_rows if r["source"] == "entity"]
-        semantic_rows = [r for r in all_rows if r["source"] == "semantic"]
-        causal_rows = [r for r in all_rows if r["source"] == "causal"]
-        return entity_rows, semantic_rows, causal_rows
-
-    async def _expand_observations(
-        self,
-        conn,
-        seed_ids: list,
-        budget: int,
-    ) -> tuple[list, list, list]:
-        """
-        Observation-specific expansion.
-
-        Observations don't have direct entity links in memory_links (they're created
-        by consolidation, not retain).  Instead, traverse source_memory_ids → world
-        facts → entities → other world facts → their observations.
-
-        Semantic and causal expansions run as a second combined CTE query.
-        """
-        source_ids_found: list = []
-        if logger.isEnabledFor(logging.DEBUG):
-            debug_rows = await conn.fetch(
-                f"""
-                SELECT id, source_memory_ids
-                FROM {fq_table("memory_units")}
-                WHERE id = ANY($1::uuid[])
-                """,
-                seed_ids,
-            )
-            for row in debug_rows:
-                if row["source_memory_ids"]:
-                    source_ids_found.extend(row["source_memory_ids"])
-            logger.debug(
-                f"[LinkExpansion] observation graph: {len(seed_ids)} seeds, "
-                f"{len(source_ids_found)} source_memory_ids found"
-            )
-
-        config = get_config()
-        ue = fq_table("unit_entities")
-        per_entity_limit = config.link_expansion_per_entity_limit
-
-        connected_sources_cte = f"""
-            source_entities AS (
-                SELECT DISTINCT ue_seed.entity_id
-                FROM seed_sources ss
-                JOIN {ue} ue_seed ON ue_seed.unit_id = ss.source_id
-            ),
-            connected_sources AS (
-                -- Find sources sharing entities with seed observation sources
-                -- via LATERAL-capped self-join (prevents hub entity fanout).
-                SELECT DISTINCT t.unit_id AS source_id
-                FROM source_entities se
-                CROSS JOIN LATERAL (
-                    SELECT ue_target.unit_id
-                    FROM {ue} ue_target
-                    WHERE ue_target.entity_id = se.entity_id
-                    ORDER BY ue_target.unit_id DESC
-                    LIMIT {per_entity_limit}
-                ) t
-                WHERE NOT EXISTS (
-                    SELECT 1 FROM seed_sources ss WHERE ss.source_id = t.unit_id
-                )
-            )"""
-
-        entity_rows = await conn.fetch(
-            f"""
-            WITH seed_sources AS (
-                SELECT DISTINCT unnest(source_memory_ids) AS source_id
-                FROM {fq_table("memory_units")}
-                WHERE id = ANY($1::uuid[])
-                  AND source_memory_ids IS NOT NULL
-            ),
-            {connected_sources_cte},
-            connected_array AS (
-                SELECT array_agg(source_id) AS source_ids FROM connected_sources
-            )
-            SELECT
-                mu.id, mu.text, mu.context, mu.event_date, mu.occurred_start,
-                mu.occurred_end, mu.mentioned_at,
-                mu.fact_type, mu.document_id, mu.chunk_id, mu.tags, mu.proof_count,
-                (SELECT COUNT(DISTINCT s) FROM unnest(mu.source_memory_ids) s WHERE s = ANY(ca.source_ids))::float AS score
-            FROM {fq_table("memory_units")} mu, connected_array ca
-            WHERE mu.fact_type = 'observation'
-              AND mu.id != ALL($1::uuid[])
-              AND ca.source_ids IS NOT NULL
-              AND mu.source_memory_ids && ca.source_ids
-            ORDER BY score DESC
-            LIMIT $2
-            """,
-            seed_ids,
-            budget,
-        )
-        logger.debug(f"[LinkExpansion] observation graph: found {len(entity_rows)} connected observations")
-
-        # Semantic + causal for observations in one query
-        ml = fq_table("memory_links")
-        mu = fq_table("memory_units")
-        sem_causal_rows = await conn.fetch(
-            f"""
-            WITH semantic_expanded AS (
-                SELECT
-                    id, text, context, event_date, occurred_start,
-                    occurred_end, mentioned_at,
-                    fact_type, document_id, chunk_id, tags, proof_count,
-                    MAX(weight) AS score,
-                    'semantic'::text AS source
-                FROM (
-                    SELECT mu.id, mu.text, mu.context, mu.event_date, mu.occurred_start,
-                           mu.occurred_end, mu.mentioned_at, mu.fact_type, mu.document_id,
-                           mu.chunk_id, mu.tags, mu.proof_count, ml.weight
-                    FROM {ml} ml JOIN {mu} mu ON mu.id = ml.to_unit_id
-                    WHERE ml.from_unit_id = ANY($1::uuid[])
-                      AND ml.link_type = 'semantic' AND mu.fact_type = 'observation'
-                      AND mu.id != ALL($1::uuid[])
-                    UNION ALL
-                    SELECT mu.id, mu.text, mu.context, mu.event_date, mu.occurred_start,
-                           mu.occurred_end, mu.mentioned_at, mu.fact_type, mu.document_id,
-                           mu.chunk_id, mu.tags, mu.proof_count, ml.weight
-                    FROM {ml} ml JOIN {mu} mu ON mu.id = ml.from_unit_id
-                    WHERE ml.to_unit_id = ANY($1::uuid[])
-                      AND ml.link_type = 'semantic' AND mu.fact_type = 'observation'
-                      AND mu.id != ALL($1::uuid[])
-                ) sem_raw
-                GROUP BY id, text, context, event_date, occurred_start, occurred_end,
-                         mentioned_at, fact_type, document_id, chunk_id, tags, proof_count
-                ORDER BY score DESC LIMIT $2
-            ),
-            causal_expanded AS (
-                SELECT DISTINCT ON (mu.id)
-                    mu.id, mu.text, mu.context, mu.event_date, mu.occurred_start,
-                    mu.occurred_end, mu.mentioned_at, mu.fact_type, mu.document_id,
-                    mu.chunk_id, mu.tags, mu.proof_count, ml.weight AS score, 'causal'::text AS source
-                FROM {ml} ml JOIN {mu} mu ON ml.to_unit_id = mu.id
-                WHERE ml.from_unit_id = ANY($1::uuid[])
-                  AND ml.link_type IN ('causes', 'caused_by', 'enables', 'prevents')
-                  AND ml.weight >= $3 AND mu.fact_type = 'observation'
-                ORDER BY mu.id, ml.weight DESC LIMIT $2
-            )
-            SELECT * FROM semantic_expanded
-            UNION ALL
-            SELECT * FROM causal_expanded
-            """,
-            seed_ids,
-            budget,
-            self.causal_weight_threshold,
-        )
-
-        semantic_rows = [r for r in sem_causal_rows if r["source"] == "semantic"]
-        causal_rows = [r for r in sem_causal_rows if r["source"] == "causal"]
-        return entity_rows, semantic_rows, causal_rows

hindsight-api-slim/hindsight_api/engine/search/reranking.py

@@ -1,212 +0,0 @@
-"""
-Cross-encoder neural reranking for search results.
-"""
-
-import math
-from datetime import datetime, timezone
-
-from .types import MergedCandidate, ScoredResult
-
-UTC = timezone.utc
-
-# Multiplicative boost alphas for recency and temporal proximity.
-# Each signal contributes at most ±(alpha/2) relative adjustment to the base CE score,
-# so the max combined boost is (1 + alpha/2)^2 ≈ +21% and min is (1 - alpha/2)^2 ≈ -19%.
-_RECENCY_ALPHA: float = 0.2
-_TEMPORAL_ALPHA: float = 0.2
-_PROOF_COUNT_ALPHA: float = 0.1  # Conservative: max ±5% for evidence strength
-
-
-def apply_combined_scoring(
-    scored_results: list[ScoredResult],
-    now: datetime,
-    recency_alpha: float = _RECENCY_ALPHA,
-    temporal_alpha: float = _TEMPORAL_ALPHA,
-    proof_count_alpha: float = _PROOF_COUNT_ALPHA,
-) -> None:
-    """Apply combined scoring to a list of ScoredResults in-place.
-
-    Uses the cross-encoder score as the primary relevance signal, with recency,
-    temporal proximity, and proof count applied as multiplicative boosts. This
-    ensures the influence of these secondary signals is always proportional to
-    the base relevance score, regardless of the cross-encoder model's score
-    calibration.
-
-    Formula::
-
-        recency_boost     = 1 + recency_alpha     * (recency     - 0.5)   # in [1-α/2, 1+α/2]
-        temporal_boost    = 1 + temporal_alpha    * (temporal    - 0.5)   # in [1-α/2, 1+α/2]
-        proof_count_boost = 1 + proof_count_alpha * (proof_norm  - 0.5)   # in [1-α/2, 1+α/2]
-        combined_score    = CE_normalized * recency_boost * temporal_boost * proof_count_boost
-
-    proof_norm maps proof_count using a smooth logarithmic curve centered at 0.5,
-    clamped to [0, 1]:
-      proof_count=1 → 0.5 + 0 = 0.5 (neutral multiplier)
-      proof_count=150 → clamped to 1.0 (max +5% boost)
-
-    Temporal proximity is treated as neutral (0.5) when not set by temporal retrieval,
-    so temporal_boost collapses to 1.0 for non-temporal queries.
-
-    Proof count is treated as neutral (0.5) when not available (non-observation facts),
-    so proof_count_boost collapses to 1.0 for world/experience/opinion facts.
-
-    Args:
-        scored_results: Results from the cross-encoder reranker. Mutated in place.
-        now: Current UTC datetime for recency calculation.
-        recency_alpha: Max relative recency adjustment (default 0.2 → ±10%).
-        temporal_alpha: Max relative temporal adjustment (default 0.2 → ±10%).
-        proof_count_alpha: Max relative proof count adjustment (default 0.1 → ±5%).
-    """
-    if now.tzinfo is None:
-        now = now.replace(tzinfo=UTC)
-
-    for sr in scored_results:
-        # Recency: linear decay over 365 days → [0.1, 1.0]; neutral 0.5 if no date.
-        sr.recency = 0.5
-        if sr.retrieval.occurred_start:
-            occurred = sr.retrieval.occurred_start
-            if occurred.tzinfo is None:
-                occurred = occurred.replace(tzinfo=UTC)
-            days_ago = (now - occurred).total_seconds() / 86400
-            sr.recency = max(0.1, min(1.0, 1.0 - (days_ago / 365)))
-
-        # Temporal proximity: meaningful only for temporal queries; neutral otherwise.
-        sr.temporal = sr.retrieval.temporal_proximity if sr.retrieval.temporal_proximity is not None else 0.5
-
-        # Proof count: log-normalized evidence strength; neutral for non-observations.
-        proof_count = sr.retrieval.proof_count
-        if proof_count is not None and proof_count >= 1:
-            # Clamp to [0, 1] so extreme counts stay within documented ±5% range
-            proof_norm = min(1.0, max(0.0, 0.5 + (math.log(proof_count) / 10.0)))
-        else:
-            # Neutral baseline is precisely 0.5, ensuring neutral multiplier (1.0)
-            proof_norm = 0.5
-
-        # RRF: kept at 0.0 for trace continuity but excluded from scoring.
-        # RRF is batch-relative (min-max normalised) and redundant after reranking.
-        sr.rrf_normalized = 0.0
-
-        recency_boost = 1.0 + recency_alpha * (sr.recency - 0.5)
-        temporal_boost = 1.0 + temporal_alpha * (sr.temporal - 0.5)
-        proof_count_boost = 1.0 + proof_count_alpha * (proof_norm - 0.5)
-        sr.combined_score = sr.cross_encoder_score_normalized * recency_boost * temporal_boost * proof_count_boost
-        sr.weight = sr.combined_score
-
-
-class CrossEncoderReranker:
-    """
-    Neural reranking using a cross-encoder model.
-
-    Configured via environment variables (see cross_encoder.py).
-    Default local model is cross-encoder/ms-marco-MiniLM-L-6-v2.
-    """
-
-    def __init__(self, cross_encoder=None):
-        """
-        Initialize cross-encoder reranker.
-
-        Args:
-            cross_encoder: CrossEncoderModel instance. If None, creates one from
-                          environment variables (defaults to local provider)
-        """
-        if cross_encoder is None:
-            from hindsight_api.engine.cross_encoder import create_cross_encoder_from_env
-
-            cross_encoder = create_cross_encoder_from_env()
-        self.cross_encoder = cross_encoder
-        self._initialized = False
-
-    async def ensure_initialized(self):
-        """Ensure the cross-encoder model is initialized (for lazy initialization)."""
-        if self._initialized:
-            return
-
-        import asyncio
-
-        cross_encoder = self.cross_encoder
-        # For local providers, run in thread pool to avoid blocking event loop
-        if cross_encoder.provider_name == "local":
-            loop = asyncio.get_event_loop()
-            await loop.run_in_executor(None, lambda: asyncio.run(cross_encoder.initialize()))
-        else:
-            await cross_encoder.initialize()
-        self._initialized = True
-
-    async def rerank(self, query: str, candidates: list[MergedCandidate]) -> list[ScoredResult]:
-        """
-        Rerank candidates using cross-encoder scores.
-
-        Args:
-            query: Search query
-            candidates: Merged candidates from RRF
-
-        Returns:
-            List of ScoredResult objects sorted by cross-encoder score
-        """
-        if not candidates:
-            return []
-
-        # Prepare query-document pairs with date information
-        pairs = []
-        for candidate in candidates:
-            retrieval = candidate.retrieval
-
-            # Use text + context for better ranking
-            doc_text = retrieval.text
-            if retrieval.context:
-                doc_text = f"{retrieval.context}: {doc_text}"
-
-            # Add formatted date information for temporal awareness
-            if retrieval.occurred_start:
-                occurred_start = retrieval.occurred_start
-
-                # Format in two styles for better model understanding
-                # 1. ISO format: YYYY-MM-DD
-                date_iso = occurred_start.strftime("%Y-%m-%d")
-
-                # 2. Human-readable: "June 5, 2022"
-                date_readable = occurred_start.strftime("%B %d, %Y")
-
-                # Prepend date to document text
-                doc_text = f"[Date: {date_readable} ({date_iso})] {doc_text}"
-
-            pairs.append([query, doc_text])
-
-        # Get cross-encoder scores
-        scores = await self.cross_encoder.predict(pairs)
-
-        # Normalize scores using sigmoid to [0, 1] range
-        # Cross-encoder returns logits which can be negative
-        import math
-
-        import numpy as np
-
-        def sigmoid(x):
-            return 1 / (1 + np.exp(-x))
-
-        normalized_scores = [sigmoid(score) for score in scores]
-
-        # Create ScoredResult objects with cross-encoder scores
-        scored_results = []
-        for candidate, raw_score, norm_score in zip(candidates, scores, normalized_scores):
-            # Sanitize NaN scores (cross-encoder can return NaN for certain inputs).
-            # NaN propagates through all downstream scoring and Pydantic serializes
-            # NaN as JSON null, which breaks clients expecting numeric values.
-            raw = float(raw_score)
-            norm = float(norm_score)
-            if math.isnan(raw):
-                raw = 0.0
-            if math.isnan(norm):
-                norm = 0.0
-            scored_result = ScoredResult(
-                candidate=candidate,
-                cross_encoder_score=raw,
-                cross_encoder_score_normalized=norm,
-                weight=norm,  # Initial weight is just cross-encoder score
-            )
-            scored_results.append(scored_result)
-
-        # Sort by cross-encoder score
-        scored_results.sort(key=lambda x: x.weight, reverse=True)
-
-        return scored_results

hindsight-api-slim/hindsight_api/engine/search/tags.py

@@ -1,390 +0,0 @@
-"""
-Tags filtering utilities for retrieval.
-
-Provides SQL building functions for filtering memories by tags.
-Supports four matching modes via TagsMatch enum:
-- "any": OR matching, includes untagged memories (default, backward compatible)
-- "all": AND matching, includes untagged memories
-- "any_strict": OR matching, excludes untagged memories
-- "all_strict": AND matching, excludes untagged memories
-
-OR matching (any/any_strict): Memory matches if ANY of its tags overlap with request tags
-AND matching (all/all_strict): Memory matches if ALL request tags are present in its tags
-"""
-
-from __future__ import annotations
-
-from typing import Annotated, Literal
-
-from pydantic import BaseModel, ConfigDict, Field
-
-TagsMatch = Literal["any", "all", "any_strict", "all_strict"]
-
-
-def _parse_tags_match(match: TagsMatch) -> tuple[str, bool]:
-    """
-    Parse TagsMatch into operator and include_untagged flag.
-
-    Returns:
-        Tuple of (operator, include_untagged)
-        - operator: "&&" for any/any_strict, "@>" for all/all_strict
-        - include_untagged: True for any/all, False for any_strict/all_strict
-    """
-    if match == "any":
-        return "&&", True
-    elif match == "all":
-        return "@>", True
-    elif match == "any_strict":
-        return "&&", False
-    elif match == "all_strict":
-        return "@>", False
-    else:
-        # Default to "any" behavior
-        return "&&", True
-
-
-def build_tags_where_clause(
-    tags: list[str] | None,
-    param_offset: int = 1,
-    table_alias: str = "",
-    match: TagsMatch = "any",
-) -> tuple[str, list, int]:
-    """
-    Build a SQL WHERE clause for filtering by tags.
-
-    Supports four matching modes:
-    - "any" (default): OR matching, includes untagged memories
-    - "all": AND matching, includes untagged memories
-    - "any_strict": OR matching, excludes untagged memories
-    - "all_strict": AND matching, excludes untagged memories
-
-    Args:
-        tags: List of tags to filter by. If None or empty, returns empty clause (no filtering).
-        param_offset: Starting parameter number for SQL placeholders (default 1).
-        table_alias: Optional table alias prefix (e.g., "mu." for "memory_units mu").
-        match: Matching mode. Defaults to "any".
-
-    Returns:
-        Tuple of (sql_clause, params, next_param_offset):
-        - sql_clause: SQL WHERE clause string
-        - params: List of parameter values to bind
-        - next_param_offset: Next available parameter number
-
-    Example:
-        >>> clause, params, next_offset = build_tags_where_clause(['user_a'], 3, 'mu.', 'any_strict')
-        >>> print(clause)  # "AND mu.tags IS NOT NULL AND mu.tags != '{}' AND mu.tags && $3"
-    """
-    if not tags:
-        return "", [], param_offset
-
-    column = f"{table_alias}tags" if table_alias else "tags"
-    operator, include_untagged = _parse_tags_match(match)
-
-    if include_untagged:
-        # Include untagged memories (NULL or empty array) OR matching tags
-        clause = f"AND ({column} IS NULL OR {column} = '{{}}' OR {column} {operator} ${param_offset})"
-    else:
-        # Strict: only memories with matching tags (exclude NULL and empty)
-        clause = f"AND {column} IS NOT NULL AND {column} != '{{}}' AND {column} {operator} ${param_offset}"
-
-    return clause, [tags], param_offset + 1
-
-
-def build_tags_where_clause_simple(
-    tags: list[str] | None,
-    param_num: int,
-    table_alias: str = "",
-    match: TagsMatch = "any",
-) -> str:
-    """
-    Build a simple SQL WHERE clause for tags filtering.
-
-    This is a convenience version that returns just the clause string,
-    assuming the caller will add the tags array to their params list.
-
-    Args:
-        tags: List of tags to filter by. If None or empty, returns empty string.
-        param_num: Parameter number to use in the clause.
-        table_alias: Optional table alias prefix.
-        match: Matching mode. Defaults to "any".
-
-    Returns:
-        SQL clause string or empty string.
-    """
-    if not tags:
-        return ""
-
-    column = f"{table_alias}tags" if table_alias else "tags"
-    operator, include_untagged = _parse_tags_match(match)
-
-    if include_untagged:
-        # Include untagged memories (NULL or empty array) OR matching tags
-        return f"AND ({column} IS NULL OR {column} = '{{}}' OR {column} {operator} ${param_num})"
-    else:
-        # Strict: only memories with matching tags (exclude NULL and empty)
-        return f"AND {column} IS NOT NULL AND {column} != '{{}}' AND {column} {operator} ${param_num}"
-
-
-def filter_results_by_tags(
-    results: list,
-    tags: list[str] | None,
-    match: TagsMatch = "any",
-) -> list:
-    """
-    Filter retrieval results by tags in Python (for post-processing).
-
-    Used when SQL filtering isn't possible (e.g., graph traversal results).
-
-    Args:
-        results: List of RetrievalResult objects with a 'tags' attribute.
-        tags: List of tags to filter by. If None or empty, returns all results.
-        match: Matching mode. Defaults to "any".
-
-    Returns:
-        Filtered list of results.
-    """
-    if not tags:
-        return results
-
-    _, include_untagged = _parse_tags_match(match)
-    is_any_match = match in ("any", "any_strict")
-
-    tags_set = set(tags)
-    filtered = []
-
-    for result in results:
-        result_tags = getattr(result, "tags", None)
-
-        # Check if untagged
-        is_untagged = result_tags is None or len(result_tags) == 0
-
-        if is_untagged:
-            if include_untagged:
-                filtered.append(result)
-            # else: skip untagged
-        else:
-            result_tags_set = set(result_tags)
-            if is_any_match:
-                # Any overlap
-                if result_tags_set & tags_set:
-                    filtered.append(result)
-            else:
-                # All tags must be present
-                if tags_set <= result_tags_set:
-                    filtered.append(result)
-
-    return filtered
-
-
-# =============================================================================
-# Compound tag group models (recursive boolean expressions)
-# =============================================================================
-
-
-class TagGroupLeaf(BaseModel):
-    """A leaf tag filter: matches memories by tag list and match mode."""
-
-    tags: list[str]
-    match: TagsMatch = "any_strict"
-
-
-class TagGroupAnd(BaseModel):
-    """Compound AND group: all child filters must match."""
-
-    model_config = ConfigDict(populate_by_name=True)
-    filters: list[TagGroup] = Field(alias="and")
-
-
-class TagGroupOr(BaseModel):
-    """Compound OR group: at least one child filter must match."""
-
-    model_config = ConfigDict(populate_by_name=True)
-    filters: list[TagGroup] = Field(alias="or")
-
-
-class TagGroupNot(BaseModel):
-    """Compound NOT group: child filter must NOT match."""
-
-    model_config = ConfigDict(populate_by_name=True)
-    filter: TagGroup = Field(alias="not")
-
-
-# TagGroup is a discriminated union; Pydantic will try left-to-right.
-# TagGroupLeaf is identified by the presence of 'tags'.
-# TagGroupAnd / TagGroupOr / TagGroupNot are compound (no 'tags' key).
-TagGroup = Annotated[
-    TagGroupLeaf | TagGroupAnd | TagGroupOr | TagGroupNot,
-    Field(union_mode="left_to_right"),
-]
-
-# Rebuild forward-reference models so recursive TagGroup is resolved.
-TagGroupAnd.model_rebuild()
-TagGroupOr.model_rebuild()
-TagGroupNot.model_rebuild()
-
-
-# =============================================================================
-# SQL builder for compound tag groups
-# =============================================================================
-
-
-def _build_group_clause(
-    group: TagGroup,
-    param_offset: int,
-    table_alias: str,
-) -> tuple[str, list, int]:
-    """
-    Recursively build an inner SQL clause (no leading AND/OR) for a single TagGroup.
-
-    Returns:
-        (inner_clause, params, next_param_offset)
-    """
-    if isinstance(group, TagGroupLeaf):
-        column = f"{table_alias}tags" if table_alias else "tags"
-        operator, include_untagged = _parse_tags_match(group.match)
-        if include_untagged:
-            clause = f"({column} IS NULL OR {column} = '{{}}' OR {column} {operator} ${param_offset})"
-        else:
-            clause = f"({column} IS NOT NULL AND {column} != '{{}}' AND {column} {operator} ${param_offset})"
-        return clause, [group.tags], param_offset + 1
-
-    elif isinstance(group, TagGroupAnd):
-        parts = []
-        params: list = []
-        offset = param_offset
-        for child in group.filters:
-            child_clause, child_params, offset = _build_group_clause(child, offset, table_alias)
-            parts.append(child_clause)
-            params.extend(child_params)
-        inner = " AND ".join(parts)
-        return f"({inner})", params, offset
-
-    elif isinstance(group, TagGroupOr):
-        parts = []
-        params = []
-        offset = param_offset
-        for child in group.filters:
-            child_clause, child_params, offset = _build_group_clause(child, offset, table_alias)
-            parts.append(child_clause)
-            params.extend(child_params)
-        inner = " OR ".join(parts)
-        return f"({inner})", params, offset
-
-    elif isinstance(group, TagGroupNot):
-        child_clause, child_params, next_offset = _build_group_clause(group.filter, param_offset, table_alias)
-        return f"NOT {child_clause}", child_params, next_offset
-
-    else:
-        # Should never happen with proper Pydantic validation
-        return "", [], param_offset
-
-
-def build_tag_groups_where_clause(
-    tag_groups: list[TagGroup] | None,
-    param_offset: int,
-    table_alias: str = "",
-) -> tuple[str, list, int]:
-    """
-    Build a SQL WHERE clause for compound tag group filtering.
-
-    Top-level groups are AND-ed together. Each group is a recursive boolean
-    expression (leaf, and, or, not).
-
-    Args:
-        tag_groups: List of TagGroup objects. If None or empty, returns empty clause.
-        param_offset: Starting parameter number for SQL placeholders.
-        table_alias: Optional table alias prefix (e.g., "mu." for "memory_units mu").
-
-    Returns:
-        Tuple of (sql_clause, params, next_param_offset):
-        - sql_clause: SQL WHERE clause string starting with "AND" (or empty string)
-        - params: List of parameter values to bind (one per leaf node)
-        - next_param_offset: Next available parameter number
-
-    Example:
-        >>> groups = [TagGroupLeaf(tags=["user:alice"], match="all_strict")]
-        >>> clause, params, next_offset = build_tag_groups_where_clause(groups, 3)
-        >>> print(clause)  # "AND (tags IS NOT NULL AND tags != '{}' AND tags @> $3)"
-    """
-    if not tag_groups:
-        return "", [], param_offset
-
-    all_params: list = []
-    all_clauses: list[str] = []
-    offset = param_offset
-
-    for group in tag_groups:
-        inner_clause, group_params, offset = _build_group_clause(group, offset, table_alias)
-        all_clauses.append(inner_clause)
-        all_params.extend(group_params)
-
-    combined = " AND ".join(all_clauses)
-    return f"AND {combined}", all_params, offset
-
-
-# =============================================================================
-# Python-side filter for compound tag groups (post-retrieval filtering)
-# =============================================================================
-
-
-def _match_group(result: object, group: TagGroup) -> bool:
-    """
-    Recursively evaluate a TagGroup against a retrieval result.
-
-    Args:
-        result: Any object with a 'tags' attribute (list[str] or None).
-        group: The TagGroup to evaluate.
-
-    Returns:
-        True if the result matches the group, False otherwise.
-    """
-    if isinstance(group, TagGroupLeaf):
-        result_tags = getattr(result, "tags", None)
-        is_untagged = result_tags is None or len(result_tags) == 0
-        _, include_untagged = _parse_tags_match(group.match)
-        is_any_match = group.match in ("any", "any_strict")
-        tags_set = set(group.tags)
-
-        if is_untagged:
-            return include_untagged
-        else:
-            result_tags_set = set(result_tags)
-            if is_any_match:
-                return bool(result_tags_set & tags_set)
-            else:
-                return tags_set <= result_tags_set
-
-    elif isinstance(group, TagGroupAnd):
-        return all(_match_group(result, child) for child in group.filters)
-
-    elif isinstance(group, TagGroupOr):
-        return any(_match_group(result, child) for child in group.filters)
-
-    elif isinstance(group, TagGroupNot):
-        return not _match_group(result, group.filter)
-
-    else:
-        return True
-
-
-def filter_results_by_tag_groups(
-    results: list,
-    tag_groups: list[TagGroup] | None,
-) -> list:
-    """
-    Filter retrieval results by compound tag groups in Python (for post-processing).
-
-    Used when SQL filtering isn't possible (e.g., graph traversal results).
-    Top-level groups are AND-ed together.
-
-    Args:
-        results: List of RetrievalResult objects with a 'tags' attribute.
-        tag_groups: List of TagGroup objects. If None or empty, returns all results.
-
-    Returns:
-        Filtered list of results where ALL top-level groups match.
-    """
-    if not tag_groups:
-        return results
-
-    return [r for r in results if all(_match_group(r, group) for group in tag_groups)]

hindsight-api-slim/hindsight_api/engine/storage/gcs.py

@@ -1,105 +0,0 @@
-"""Google Cloud Storage backend using obstore."""
-
-import logging
-import os
-from datetime import datetime, timedelta, timezone
-
-import obstore as obs
-from obstore.store import GCSStore
-
-from .base import FileStorage
-
-logger = logging.getLogger(__name__)
-
-
-def _make_google_auth_credential_provider():
-    """Create a credential provider using google.auth (supports all credential types).
-
-    obstore's built-in credential parsing only supports service_account and
-    authorized_user JSON types.  This provider uses the google-auth library
-    which additionally handles external_account (Workload Identity Federation),
-    impersonated credentials, and metadata-server credentials.
-    """
-    import google.auth
-    import google.auth.transport.requests
-
-    credentials, _ = google.auth.default(scopes=["https://www.googleapis.com/auth/cloud-platform"])
-    request = google.auth.transport.requests.Request()
-
-    def _provide():
-        credentials.refresh(request)
-        expiry = credentials.expiry
-        if expiry and expiry.tzinfo is None:
-            expiry = expiry.replace(tzinfo=timezone.utc)
-        return {"token": credentials.token, "expires_at": expiry}
-
-    return _provide
-
-
-class GCSFileStorage(FileStorage):
-    """
-    Google Cloud Storage backend.
-
-    Uses obstore (Rust-backed) for high-throughput async access to GCS.
-    Supports Application Default Credentials, service account keys, and explicit credentials.
-    """
-
-    def __init__(
-        self,
-        bucket: str,
-        service_account_key: str | None = None,
-    ):
-        kwargs: dict = {}
-        if service_account_key:
-            kwargs["service_account_key"] = service_account_key
-        else:
-            # Use google.auth credential provider for broad credential type support
-            # (service_account, authorized_user, external_account, metadata server, etc.)
-            try:
-                kwargs["credential_provider"] = _make_google_auth_credential_provider()
-                logger.info("Using google.auth credential provider for GCS")
-            except Exception as e:
-                logger.warning(
-                    f"Failed to create google.auth credential provider, falling back to obstore defaults: {e}"
-                )
-
-        # Workaround for https://github.com/developmentseed/obstore/issues/605
-        # obstore's Rust layer doesn't support external_account credentials (Workload
-        # Identity Federation) and eagerly parses GOOGLE_APPLICATION_CREDENTIALS even
-        # when credential_provider is given. Per the obstore maintainer's guidance,
-        # remove env vars so the Rust code doesn't try to authenticate itself.
-        # google.auth (used by credential_provider above) has already loaded credentials.
-        gac = os.environ.pop("GOOGLE_APPLICATION_CREDENTIALS", None)
-        try:
-            self._store = GCSStore(bucket, **kwargs)
-        finally:
-            if gac is not None:
-                os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = gac
-        logger.info(f"Initialized GCS file storage: bucket={bucket}")
-
-    async def store(self, file_data: bytes, key: str, metadata: dict[str, str] | None = None) -> str:
-        await obs.put_async(self._store, key, file_data)
-        logger.debug(f"Stored file {key} ({len(file_data)} bytes) in GCS")
-        return key
-
-    async def retrieve(self, key: str) -> bytes:
-        try:
-            response = await obs.get_async(self._store, key)
-            return await response.bytes_async()
-        except Exception as e:
-            if "not found" in str(e).lower():
-                raise FileNotFoundError(f"File not found: {key}") from e
-            raise
-
-    async def delete(self, key: str) -> None:
-        await obs.delete_async(self._store, key)
-
-    async def exists(self, key: str) -> bool:
-        try:
-            await obs.head_async(self._store, key)
-            return True
-        except Exception:
-            return False
-
-    async def get_download_url(self, key: str, expires_in: int = 3600) -> str:
-        return await obs.sign_async(self._store, "GET", key, timedelta(seconds=expires_in))

hindsight-api-slim/hindsight_api/main.py

@@ -1,302 +0,0 @@
-"""
-Command-line interface for Hindsight API.
-
-Run the server with:
-    hindsight-api
-
-Run as background daemon:
-    hindsight-api --daemon
-
-Stop with Ctrl+C.
-"""
-
-import argparse
-import asyncio
-import atexit
-import dataclasses
-import os
-import signal
-import sys
-import warnings
-
-import uvicorn
-
-from . import MemoryEngine, __version__
-from .api import create_app
-from .banner import print_banner
-from .config import DEFAULT_WORKERS, ENV_WORKERS, HindsightConfig, _get_raw_config
-from .daemon import (
-    DEFAULT_DAEMON_PORT,
-    DEFAULT_IDLE_TIMEOUT,
-    IdleTimeoutMiddleware,
-    daemonize,
-)
-from .extensions import DefaultExtensionContext, OperationValidatorExtension, TenantExtension, load_extension
-
-# Filter deprecation warnings from third-party libraries
-warnings.filterwarnings("ignore", message="websockets.legacy is deprecated")
-warnings.filterwarnings("ignore", message="websockets.server.WebSocketServerProtocol is deprecated")
-
-# Disable tokenizers parallelism to avoid warnings
-os.environ["TOKENIZERS_PARALLELISM"] = "false"
-
-# Global reference for cleanup
-_memory: MemoryEngine | None = None
-
-
-def _cleanup():
-    """Synchronous cleanup function to stop resources on exit."""
-    global _memory
-    if _memory is not None and _memory._pg0 is not None:
-        try:
-            loop = asyncio.new_event_loop()
-            loop.run_until_complete(_memory._pg0.stop())
-            loop.close()
-            print("\npg0 stopped.")
-        except Exception as e:
-            print(f"\nError stopping pg0: {e}")
-
-
-def _signal_handler(signum, frame):
-    """Handle SIGINT/SIGTERM to ensure cleanup."""
-    print(f"\nReceived signal {signum}, shutting down...")
-    _cleanup()
-    sys.exit(0)
-
-
-def main():
-    """Main entry point for the CLI."""
-    global _memory
-
-    # Load configuration from environment (for CLI args defaults)
-    config = _get_raw_config()
-
-    parser = argparse.ArgumentParser(
-        prog="hindsight-api",
-        description="Hindsight API Server",
-    )
-
-    # Server options
-    parser.add_argument(
-        "--host", default=config.host, help=f"Host to bind to (default: {config.host}, env: HINDSIGHT_API_HOST)"
-    )
-    parser.add_argument(
-        "--port",
-        type=int,
-        default=config.port,
-        help=f"Port to bind to (default: {config.port}, env: HINDSIGHT_API_PORT)",
-    )
-    parser.add_argument(
-        "--log-level",
-        default=config.log_level,
-        choices=["critical", "error", "warning", "info", "debug", "trace"],
-        help=f"Log level (default: {config.log_level}, env: HINDSIGHT_API_LOG_LEVEL)",
-    )
-
-    # Development options
-    parser.add_argument("--reload", action="store_true", help="Enable auto-reload on code changes (development only)")
-    parser.add_argument(
-        "--workers",
-        type=int,
-        default=int(os.getenv(ENV_WORKERS, str(DEFAULT_WORKERS))),
-        help=f"Number of worker processes (env: {ENV_WORKERS}, default: {DEFAULT_WORKERS})",
-    )
-
-    # Access log options
-    parser.add_argument("--access-log", action="store_true", help="Enable access log")
-    parser.add_argument("--no-access-log", dest="access_log", action="store_false", help="Disable access log (default)")
-    parser.set_defaults(access_log=False)
-
-    # Proxy options
-    parser.add_argument(
-        "--proxy-headers", action="store_true", help="Enable X-Forwarded-Proto, X-Forwarded-For headers"
-    )
-    parser.add_argument(
-        "--forwarded-allow-ips", default=None, help="Comma separated list of IPs to trust with proxy headers"
-    )
-
-    # SSL options
-    parser.add_argument("--ssl-keyfile", default=None, help="SSL key file")
-    parser.add_argument("--ssl-certfile", default=None, help="SSL certificate file")
-
-    # Daemon mode options
-    parser.add_argument(
-        "--daemon",
-        action="store_true",
-        help=f"Run as background daemon (uses port {DEFAULT_DAEMON_PORT}, auto-exits after idle)",
-    )
-    parser.add_argument(
-        "--idle-timeout",
-        type=int,
-        default=DEFAULT_IDLE_TIMEOUT,
-        help=f"Idle timeout in seconds before auto-exit in daemon mode (default: {DEFAULT_IDLE_TIMEOUT})",
-    )
-
-    args = parser.parse_args()
-
-    # Daemon mode handling
-    if args.daemon:
-        # Use port from args (may be custom for profiles)
-        if args.port == config.port:  # No custom port specified
-            args.port = DEFAULT_DAEMON_PORT
-        args.host = "127.0.0.1"  # Only bind to localhost for security
-
-        # Fork into background
-        # No lockfile needed - port binding prevents duplicate daemons
-        daemonize()
-
-    # Print banner (not in daemon mode)
-    if not args.daemon:
-        print()
-        print_banner()
-
-    # Configure Python logging based on log level
-    # Update config with CLI override if provided
-    if args.log_level != config.log_level:
-        config = dataclasses.replace(config, host=args.host, port=args.port, log_level=args.log_level)
-    config.configure_logging()
-    if not args.daemon:
-        config.log_config()
-
-    # Register cleanup handlers
-    atexit.register(_cleanup)
-    signal.signal(signal.SIGINT, _signal_handler)
-    signal.signal(signal.SIGTERM, _signal_handler)
-
-    # Load operation validator extension if configured
-    operation_validator = load_extension("OPERATION_VALIDATOR", OperationValidatorExtension)
-    if operation_validator:
-        import logging
-
-        logging.info(f"Loaded operation validator: {operation_validator.__class__.__name__}")
-
-    # Load tenant extension if configured
-    tenant_extension = load_extension("TENANT", TenantExtension)
-    if tenant_extension:
-        import logging
-
-        logging.info(f"Loaded tenant extension: {tenant_extension.__class__.__name__}")
-
-    # Create MemoryEngine (reads configuration from environment)
-    _memory = MemoryEngine(
-        operation_validator=operation_validator,
-        tenant_extension=tenant_extension,
-        run_migrations=config.run_migrations_on_startup,
-    )
-
-    # Set extension context on tenant extension (needed for schema provisioning)
-    if tenant_extension:
-        extension_context = DefaultExtensionContext(
-            database_url=config.database_url,
-            memory_engine=_memory,
-        )
-        tenant_extension.set_context(extension_context)
-        logging.info("Extension context set on tenant extension")
-
-    # Create FastAPI app
-    app = create_app(
-        memory=_memory,
-        http_api_enabled=True,
-        mcp_api_enabled=config.mcp_enabled,
-        mcp_mount_path="/mcp",
-        initialize_memory=True,
-    )
-
-    # Wrap with idle timeout middleware in daemon mode
-    idle_middleware = None
-    if args.daemon:
-        idle_middleware = IdleTimeoutMiddleware(app, idle_timeout=args.idle_timeout)
-        app = idle_middleware
-
-    # Prepare uvicorn config
-    # When using workers or reload, we must use import string so each worker can import the app
-    use_import_string = args.workers > 1 or args.reload
-    # Check for uvloop/winloop availability
-    import sys
-
-    loop_impl = "asyncio"
-    if sys.platform == "win32":
-        try:
-            import winloop
-
-            winloop.install()  # Patches asyncio globally — uvicorn uses "asyncio" but gets winloop
-            loop_impl = "asyncio"  # Tell uvicorn "asyncio" — it's now winloop underneath
-            print("winloop installed as asyncio event loop policy (Windows uvloop port)")
-        except ImportError:
-            print("winloop not installed, using default asyncio event loop")
-    else:
-        try:
-            import uvloop  # noqa: F401
-
-            loop_impl = "uvloop"
-            print("uvloop available, will use for event loop")
-        except ImportError:
-            print("uvloop not installed, using default asyncio event loop")
-
-    uvicorn_config = {
-        "app": "hindsight_api.server:app" if use_import_string else app,
-        "host": args.host,
-        "port": args.port,
-        "log_level": args.log_level,
-        "access_log": args.access_log,
-        "proxy_headers": args.proxy_headers,
-        "ws": "wsproto",  # Use wsproto instead of websockets to avoid deprecation warnings
-        "loop": loop_impl,  # Explicitly set event loop implementation
-        "timeout_keep_alive": 30,  # Exceed aiohttp's 15s client timeout so the client always closes first
-        "timeout_graceful_shutdown": 5,  # Cap graceful shutdown at 5s; also enables force-kill on second Ctrl+C
-    }
-
-    # Add optional parameters if provided
-    if args.reload:
-        uvicorn_config["reload"] = True
-    if args.workers > 1:
-        uvicorn_config["workers"] = args.workers
-    if args.forwarded_allow_ips:
-        uvicorn_config["forwarded_allow_ips"] = args.forwarded_allow_ips
-    if args.ssl_keyfile:
-        uvicorn_config["ssl_keyfile"] = args.ssl_keyfile
-    if args.ssl_certfile:
-        uvicorn_config["ssl_certfile"] = args.ssl_certfile
-
-    # Print startup info (not in daemon mode)
-    if not args.daemon:
-        from .banner import print_startup_info
-
-        print_startup_info(
-            host=args.host,
-            port=args.port,
-            database_url=config.database_url,
-            llm_provider=config.llm_provider,
-            llm_model=config.llm_model,
-            embeddings_provider=config.embeddings_provider,
-            reranker_provider=config.reranker_provider,
-            mcp_enabled=config.mcp_enabled,
-            version=__version__,
-            vector_extension=config.vector_extension,
-            text_search_extension=config.text_search_extension,
-        )
-
-    # Start idle checker in daemon mode
-    if idle_middleware is not None:
-        # Start the idle checker in a background thread with its own event loop
-        import logging
-        import threading
-
-        def run_idle_checker():
-            import time
-
-            time.sleep(2)  # Wait for uvicorn to start
-            try:
-                loop = asyncio.new_event_loop()
-                asyncio.set_event_loop(loop)
-                loop.run_until_complete(idle_middleware._check_idle())
-            except Exception as e:
-                logging.error(f"Idle checker error: {e}", exc_info=True)
-
-        threading.Thread(target=run_idle_checker, daemon=True).start()
-
-    uvicorn.run(**uvicorn_config)
-
-
-if __name__ == "__main__":
-    main()

hindsight-api-slim/hindsight_api/mcp_local.py

@@ -1,40 +0,0 @@
-"""
-Local MCP server entry point for use with Claude Code (HTTP transport).
-
-This is a thin wrapper around the main hindsight-api server that pre-configures
-sensible defaults for local use (embedded PostgreSQL via pg0, warning log level).
-
-The full API runs on localhost:8888. Configure Claude Code's MCP settings:
-    claude mcp add --transport http hindsight http://localhost:8888/mcp/
-
-Or pinned to a specific bank (single-bank mode):
-    claude mcp add --transport http hindsight http://localhost:8888/mcp/default/
-
-Run with:
-    hindsight-local-mcp
-
-Or with uvx:
-    uvx hindsight-api@latest hindsight-local-mcp
-
-Environment variables:
-    HINDSIGHT_API_LLM_API_KEY: Required. API key for LLM provider.
-    HINDSIGHT_API_LLM_PROVIDER: Optional. LLM provider (default: "openai").
-    HINDSIGHT_API_LLM_MODEL: Optional. LLM model (default: "gpt-4o-mini").
-    HINDSIGHT_API_DATABASE_URL: Optional. Override database URL (default: pg0://hindsight-mcp).
-"""
-
-import os
-
-
-def main() -> None:
-    """Start the Hindsight API server with local defaults."""
-    # Set local defaults (only if not already configured by the user)
-    os.environ.setdefault("HINDSIGHT_API_DATABASE_URL", "pg0://hindsight-mcp")
-
-    from hindsight_api.main import main as api_main
-
-    api_main()
-
-
-if __name__ == "__main__":
-    main()

hindsight-api-slim/hindsight_api/webhooks/__init__.py

@@ -1,13 +0,0 @@
-"""Webhook system for Hindsight API event notifications."""
-
-from .manager import WebhookManager
-from .models import ConsolidationEventData, RetainEventData, WebhookConfig, WebhookEvent, WebhookEventType
-
-__all__ = [
-    "WebhookManager",
-    "WebhookConfig",
-    "WebhookEvent",
-    "WebhookEventType",
-    "ConsolidationEventData",
-    "RetainEventData",
-]

hindsight-api-slim/hindsight_api/webhooks/manager.py

@@ -1,242 +0,0 @@
-"""Webhook manager for delivering event notifications."""
-
-import hashlib
-import hmac
-import json
-import logging
-import uuid
-from datetime import datetime, timezone
-from typing import TYPE_CHECKING
-
-import asyncpg
-
-from .models import WebhookConfig, WebhookEvent, WebhookHttpConfig
-
-if TYPE_CHECKING:
-    from hindsight_api.extensions.tenant import TenantExtension
-
-logger = logging.getLogger(__name__)
-
-# Retry delay schedule in seconds: 5 retries after the first attempt.
-# Fast early retries catch transient failures; later retries handle longer outages.
-RETRY_DELAYS = [5, 300, 1800, 7200, 18000]
-MAX_ATTEMPTS = len(RETRY_DELAYS) + 1  # first attempt + len(RETRY_DELAYS) retries
-
-
-def _fq_table(table: str, schema: str | None = None) -> str:
-    """Get fully-qualified table name with optional schema prefix."""
-    if schema:
-        return f'"{schema}".{table}'
-    return table
-
-
-def _parse_http_config(value: str | dict | None) -> WebhookHttpConfig:
-    """Parse http_config column value (JSONB returned as text or dict) into a model."""
-    if value is None:
-        return WebhookHttpConfig()
-    if isinstance(value, str):
-        return WebhookHttpConfig.model_validate_json(value)
-    return WebhookHttpConfig.model_validate(value)
-
-
-class WebhookManager:
-    """
-    Manages webhook registration and event firing.
-
-    Supports both global webhooks (configured via env vars) and per-bank
-    webhooks stored in the database. Deliveries are queued as async_operations
-    tasks (operation_type='webhook_delivery') and picked up by the worker poller.
-    """
-
-    def __init__(
-        self,
-        pool: asyncpg.Pool,
-        global_webhooks: list[WebhookConfig],
-        tenant_extension: "TenantExtension | None" = None,
-    ):
-        self._pool = pool
-        self._global_webhooks = global_webhooks
-        self._tenant_extension = tenant_extension
-
-    def _sign_payload(self, secret: str, payload_bytes: bytes) -> str:
-        """Compute HMAC-SHA256 signature for a payload."""
-        return "sha256=" + hmac.new(secret.encode(), payload_bytes, hashlib.sha256).hexdigest()
-
-    async def fire_event(self, event: WebhookEvent, schema: str | None = None) -> None:
-        """
-        Queue webhook deliveries for an event as async_operations tasks.
-
-        Loads per-bank and global webhooks, inserts pending webhook_delivery tasks for
-        any webhook whose event_types list matches the fired event type. The worker
-        poller picks these up and calls MemoryEngine._handle_webhook_delivery().
-
-        Args:
-            event: The event to deliver.
-            schema: Database schema (for multi-tenant). None = default schema.
-        """
-        webhook_table = _fq_table("webhooks", schema)
-        ops_table = _fq_table("async_operations", schema)
-        now = datetime.now(timezone.utc)
-        payload_str = event.model_dump_json()
-
-        try:
-            # Load per-bank webhooks from DB (bank-specific + global NULL rows)
-            rows = await self._pool.fetch(
-                f"""
-                SELECT id, bank_id, url, secret, event_types, enabled, http_config::text
-                FROM {webhook_table}
-                WHERE (bank_id = $1 OR bank_id IS NULL) AND enabled = true
-                """,
-                event.bank_id,
-            )
-
-            db_webhooks = [
-                WebhookConfig(
-                    id=str(row["id"]),
-                    bank_id=row["bank_id"],
-                    url=row["url"],
-                    secret=row["secret"],
-                    event_types=list(row["event_types"]) if row["event_types"] else [],
-                    enabled=row["enabled"],
-                    http_config=_parse_http_config(row["http_config"]),
-                )
-                for row in rows
-            ]
-
-            # Merge with global webhooks from env config
-            all_webhooks = self._global_webhooks + db_webhooks
-            matched = 0
-
-            for webhook in all_webhooks:
-                if not webhook.enabled:
-                    continue
-                if event.event.value not in webhook.event_types:
-                    continue
-
-                operation_id = uuid.uuid4()
-                webhook_id = webhook.id if webhook.id else None
-
-                task_payload = json.dumps(
-                    {
-                        "type": "webhook_delivery",
-                        "operation_id": str(operation_id),
-                        "bank_id": event.bank_id,
-                        "url": webhook.url,
-                        "secret": webhook.secret,
-                        "event_type": event.event.value,
-                        "payload": payload_str,
-                        "webhook_id": webhook_id,
-                        "http_config": webhook.http_config.model_dump(),
-                    }
-                )
-
-                await self._pool.execute(
-                    f"""
-                    INSERT INTO {ops_table}
-                      (operation_id, bank_id, operation_type, status, task_payload, result_metadata, created_at, updated_at)
-                    VALUES ($1, $2, 'webhook_delivery', 'pending', $3::jsonb, '{{}}'::jsonb, $4, $4)
-                    """,
-                    operation_id,
-                    event.bank_id,
-                    task_payload,
-                    now,
-                )
-                matched += 1
-
-            logger.debug(f"Fired webhook event {event.event} for bank {event.bank_id}: {matched} delivery(ies) queued")
-
-        except Exception as e:
-            logger.error(f"Failed to queue webhook deliveries for event {event.event}: {e}")
-
-    async def fire_event_with_conn(
-        self, event: WebhookEvent, conn: asyncpg.Connection, schema: str | None = None
-    ) -> None:
-        """
-        Queue webhook deliveries within an existing database connection/transaction.
-
-        Identical to fire_event() but uses the provided connection instead of acquiring
-        one from the pool. Use this to atomically insert delivery tasks in the same
-        transaction as the primary operation (transactional outbox pattern).
-
-        Args:
-            event: The event to deliver.
-            conn: Existing asyncpg connection (may be inside an active transaction).
-            schema: Database schema (for multi-tenant). None = default schema.
-        """
-        webhook_table = _fq_table("webhooks", schema)
-        ops_table = _fq_table("async_operations", schema)
-        now = datetime.now(timezone.utc)
-        payload_str = event.model_dump_json()
-
-        try:
-            rows = await conn.fetch(
-                f"""
-                SELECT id, bank_id, url, secret, event_types, enabled, http_config::text
-                FROM {webhook_table}
-                WHERE (bank_id = $1 OR bank_id IS NULL) AND enabled = true
-                """,
-                event.bank_id,
-            )
-
-            db_webhooks = [
-                WebhookConfig(
-                    id=str(row["id"]),
-                    bank_id=row["bank_id"],
-                    url=row["url"],
-                    secret=row["secret"],
-                    event_types=list(row["event_types"]) if row["event_types"] else [],
-                    enabled=row["enabled"],
-                    http_config=_parse_http_config(row["http_config"]),
-                )
-                for row in rows
-            ]
-
-            all_webhooks = self._global_webhooks + db_webhooks
-            matched = 0
-
-            for webhook in all_webhooks:
-                if not webhook.enabled:
-                    continue
-                if event.event.value not in webhook.event_types:
-                    continue
-
-                operation_id = uuid.uuid4()
-                webhook_id = webhook.id if webhook.id else None
-
-                task_payload = json.dumps(
-                    {
-                        "type": "webhook_delivery",
-                        "operation_id": str(operation_id),
-                        "bank_id": event.bank_id,
-                        "url": webhook.url,
-                        "secret": webhook.secret,
-                        "event_type": event.event.value,
-                        "payload": payload_str,
-                        "webhook_id": webhook_id,
-                        "http_config": webhook.http_config.model_dump(),
-                    }
-                )
-
-                await conn.execute(
-                    f"""
-                    INSERT INTO {ops_table}
-                      (operation_id, bank_id, operation_type, status, task_payload, result_metadata, created_at, updated_at)
-                    VALUES ($1, $2, 'webhook_delivery', 'pending', $3::jsonb, '{{}}'::jsonb, $4, $4)
-                    """,
-                    operation_id,
-                    event.bank_id,
-                    task_payload,
-                    now,
-                )
-                matched += 1
-
-            logger.debug(
-                f"Fired webhook event {event.event} for bank {event.bank_id}: {matched} delivery(ies) queued (in-transaction)"
-            )
-
-        except Exception as e:
-            logger.error(
-                f"Failed to queue webhook deliveries (in-transaction) for event {event.event}: {e}. "
-                "CRITICAL: The enclosing database transaction is now aborted and will roll back all changes."
-            )
-            raise

hindsight-api-slim/hindsight_api/webhooks/models.py

@@ -1,51 +0,0 @@
-"""Pydantic models for the webhook system."""
-
-from datetime import datetime
-from enum import StrEnum
-
-from pydantic import BaseModel, Field
-
-
-class WebhookEventType(StrEnum):
-    CONSOLIDATION_COMPLETED = "consolidation.completed"
-    RETAIN_COMPLETED = "retain.completed"
-
-
-class ConsolidationEventData(BaseModel):
-    observations_created: int | None = None
-    observations_updated: int | None = None
-    observations_deleted: int | None = None
-    error_message: str | None = None
-
-
-class RetainEventData(BaseModel):
-    document_id: str | None = None
-    tags: list[str] | None = None
-
-
-class WebhookEvent(BaseModel):
-    event: WebhookEventType
-    bank_id: str
-    operation_id: str
-    status: str  # "completed" or "failed"
-    timestamp: datetime
-    data: ConsolidationEventData | RetainEventData
-
-
-class WebhookHttpConfig(BaseModel):
-    """HTTP delivery configuration for a webhook."""
-
-    method: str = Field(default="POST", description="HTTP method: GET or POST")
-    timeout_seconds: int = Field(default=30, description="HTTP request timeout in seconds")
-    headers: dict[str, str] = Field(default_factory=dict, description="Custom HTTP headers")
-    params: dict[str, str] = Field(default_factory=dict, description="Custom HTTP query parameters")
-
-
-class WebhookConfig(BaseModel):
-    id: str
-    bank_id: str | None
-    url: str
-    secret: str | None
-    event_types: list[str]
-    enabled: bool
-    http_config: WebhookHttpConfig = Field(default_factory=WebhookHttpConfig)

hindsight-api-slim/hindsight_api/worker/exceptions.py

@@ -1,9 +0,0 @@
-from datetime import datetime
-
-
-class RetryTaskAt(Exception):
-    """Raise from a task handler to schedule a retry at a specific time."""
-
-    def __init__(self, retry_at: datetime, message: str = ""):
-        self.retry_at = retry_at
-        super().__init__(message)

hindsight-api-slim/pyproject.toml

@@ -1,213 +0,0 @@
-[build-system]
-requires = ["hatchling"]
-build-backend = "hatchling.build"
-
-[project]
-name = "hindsight-api-slim"
-version = "0.4.22"
-description = "Hindsight: Agent Memory That Works Like Human Memory"
-readme = "README.md"
-requires-python = ">=3.11"
-dependencies = [
-    "asyncpg>=0.29.0",
-    "python-dotenv>=1.0.0",
-    "openai>=1.0.0",
-    "pydantic>=2.0.0",
-    "rich>=13.0.0",
-    "langchain-text-splitters>=0.3.0",
-    "fastapi[standard]>=0.120.3",
-    "uvicorn>=0.38.0",
-    "wsproto>=1.0.0",
-    "sqlalchemy>=2.0.44",
-    "alembic>=1.17.1",
-    "pgvector>=0.4.1",
-    "greenlet>=3.2.4",
-    "psycopg2-binary>=2.9.11",
-    "tiktoken>=0.12.0",
-    "httpx>=0.27.0",
-    "PyJWT[crypto]>=2.8.0",
-    "fastmcp>=3.2.0", # SSRF/path traversal, OAuth confused deputy, command injection fixes
-    "python-dateutil>=2.8.0",
-    "opentelemetry-api>=1.20.0",
-    "opentelemetry-sdk>=1.20.0",
-    "opentelemetry-instrumentation-fastapi>=0.41b0",
-    "opentelemetry-exporter-prometheus>=0.41b0",
-    "opentelemetry-exporter-otlp-proto-http>=1.20.0",
-    "opentelemetry-semantic-conventions>=0.41b0",
-    "dateparser>=1.2.2",
-    "google-genai>=1.0.0",
-    "google-auth>=2.0.0",
-    "anthropic>=0.40.0",
-    "typer>=0.9.0",
-    "cohere>=5.0.0",
-    "litellm>=1.83.0", # 1.82.7/1.82.8 had a supply chain compromise (yanked); 1.83.0+ also fixes GHSA-jjhc-v7c2-5hh6 / GHSA-53mr-6c8q-9789
-    "markitdown[pdf,docx,pptx,xlsx,xls]>=0.1.4", # File to markdown conversion
-    "obstore>=0.4.0", # S3/GCS/Azure object storage client (Rust-backed)
-    "winloop>=0.1.0; sys_platform == 'win32'",
-    "uvloop>=0.22.1; sys_platform != 'win32'",
-    # Transitive dependency security fixes
-    "pyasn1>=0.6.3", # DoS vulnerability fix
-    "urllib3>=2.6.3", # Decompression-bomb safeguards bypass fix
-    "langchain-core>=1.2.22", # Path traversal in legacy load_prompt functions fix
-    "langsmith>=0.6.3", # SSRF via tracing header injection fix
-    "protobuf>=6.33.5", # JSON recursion depth bypass fix
-    "pillow>=12.1.1", # Out-of-bounds write in PSD image loading fix
-    "cryptography>=46.0.6", # Incomplete DNS name constraint enforcement fix
-    "filelock>=3.20.1", # TOCTOU race condition fix
-    "authlib>=1.6.9", # Account takeover/JWS header injection vulnerability fix
-    "pyjwt>=2.12.0", # Accepts unknown crit header extensions fix
-    "orjson>=3.11.6", # Unbounded recursion DoS fix
-    "python-multipart>=0.0.22", # Arbitrary file write via non-default configuration fix
-    "tornado>=6.5.5", # DoS multipart/incomplete cookie validation fix
-    "aiohttp>=3.13.3", # Multiple DoS vulnerabilities
-    "pygments>=2.20.0", # ReDoS via inefficient GUID regex fix
-    "claude-agent-sdk>=0.1.27",
-    "boto3>=1.42.74",
-]
-
-[project.optional-dependencies]
-local-ml = [
-    # Local ML models for embeddings/reranking
-    "sentence-transformers>=3.3.0",
-    "transformers>=4.53.0", # Security fixes for ReDoS vulnerabilities
-    "torch>=2.6.0", # CVE fix for remote code execution
-    "einops>=0.8.2",
-    "flashrank>=0.2.0",
-    # Apple Silicon local inference
-    "mlx>=0.31.0",
-    "mlx-lm>=0.31.1",
-    "safetensors>=0.6.2",
-]
-embedded-db = [
-    "pg0-embedded>=0.11.0",
-]
-all = [
-    "hindsight-api-slim[local-ml,embedded-db]",
-]
-test = [
-    "pytest>=7.0.0",
-    "pytest-asyncio>=0.21.0",
-    "pytest-timeout>=2.4.0",
-    "pytest-xdist>=3.0.0",
-    "filelock>=3.20.1",  # TOCTOU race condition fix
-    "testcontainers>=4.0.0",
-]
-
-[project.scripts]
-hindsight-api = "hindsight_api.main:main"
-hindsight-worker = "hindsight_api.worker.main:main"
-hindsight-local-mcp = "hindsight_api.mcp_local:main"
-hindsight-admin = "hindsight_api.admin.cli:main"
-
-[tool.hatch.build.targets.wheel]
-packages = ["hindsight_api"]
-
-[tool.hatch.build.targets.wheel.sources]
-"hindsight_api" = "hindsight_api"
-
-[tool.hatch.build.targets.sdist]
-include = [
-    "hindsight_api/**/*",
-]
-
-[tool.hatch.build]
-include = [
-    "hindsight_api/**/*.py",
-    "hindsight_api/alembic/**/*",
-]
-
-[tool.pytest.ini_options]
-log_cli = true
-log_cli_level = "INFO"
-log_cli_format = "%(asctime)s - %(levelname)s - %(name)s - %(message)s"
-log_cli_date_format = "%Y-%m-%d %H:%M:%S"
-addopts = "--timeout 300 -n 8 --dist loadgroup --durations=10 -v"
-asyncio_mode = "auto"
-asyncio_default_fixture_loop_scope = "function"
-log_auto_indent = true
-filterwarnings = [
-    "ignore:The @wait_container_is_ready decorator is deprecated:DeprecationWarning",
-    "ignore::RuntimeWarning:asyncio",
-]
-
-[dependency-groups]
-dev = [
-    "pytest>=9.0.0",
-    "pytest-asyncio>=1.3.0",
-    "pytest-timeout>=2.4.0",
-    "pytest-xdist>=3.8.0",
-    "pytest-rerunfailures>=15.0",
-    "python-dotenv>=1.2.1",
-    "filelock>=3.20.1",  # TOCTOU race condition fix
-    "ruff>=0.8.0",
-    "ty>=0.0.1",
-    "testcontainers>=4.0.0",
-]
-
-[tool.ruff]
-line-length = 120
-target-version = "py311"
-exclude = [
-    "tests/",
-    "**/tests/",
-]
-
-[tool.ruff.lint]
-select = [
-    "E",      # pycodestyle errors
-    "W",      # pycodestyle warnings
-    "F",      # Pyflakes
-    "I",      # isort
-]
-ignore = [
-    "E501",   # line too long (handled by formatter)
-    "E402",   # module import not at top of file
-    "F401",   # unused import (too noisy during development)
-    "F841",   # unused variable (too noisy during development)
-    "F811",   # redefined while unused
-    "F821",   # undefined name (forward references in type hints)
-]
-
-[tool.ruff.lint.isort]
-known-third-party = ["alembic"]
-
-[tool.ruff.format]
-quote-style = "double"
-indent-style = "space"
-
-[tool.uv]
-# Use explicit index for PyTorch to prevent the pytorch index from serving
-# non-pytorch packages (e.g. markupsafe) with incompatible wheels
-[[tool.uv.index]]
-name = "pytorch-cpu"
-url = "https://download.pytorch.org/whl/cpu"
-explicit = true
-
-[tool.uv.sources]
-# Route torch to the CPU-only PyTorch index; everything else uses PyPI
-torch = { index = "pytorch-cpu" }
-
-[tool.ty]
-# Type checking configuration
-# ty is an extremely fast Python type checker from Astral (same team as ruff/uv)
-
-[tool.ty.environment]
-python-version = "3.11"
-
-[tool.ty.src]
-exclude = [
-    "tests/",
-    "hindsight_api/alembic/",
-]
-
-[tool.ty.rules]
-# Disable noisy rules while keeping important ones
-invalid-argument-type = "ignore"    # False positives with **kwargs patterns
-invalid-return-type = "ignore"      # Often intentional in async code
-invalid-parameter-default = "ignore" # Optional params with None default
-possibly-missing-attribute = "ignore" # Common with Optional types
-invalid-raise = "ignore"            # False positives with exception tracking
-call-non-callable = "ignore"        # False positives with Optional types
-invalid-key = "ignore"              # Pydantic ConfigDict not understood
-invalid-method-override = "ignore"  # Intentional signature differences
-unresolved-reference = "ignore"     # Forward references not always resolved

hindsight-api-slim/tests/test_audit_log.py

@@ -1,449 +0,0 @@
-"""
-Tests for the audit log feature.
-
-Tests the audit log list, stats, filtering, and pagination endpoints.
-Verifies that audit entries are created for operations when audit logging is enabled.
-"""
-
-import asyncio
-
-import httpx
-import pytest
-import pytest_asyncio
-
-from hindsight_api.api import create_app
-from hindsight_api.config import get_config
-
-
-@pytest_asyncio.fixture
-async def audit_api_client(memory):
-    """Create a test client with audit logging enabled."""
-    # Enable audit logging on the memory engine's audit logger
-    memory._audit_logger._enabled = True
-    memory._audit_logger._allowed_actions = None  # All actions
-
-    app = create_app(memory, initialize_memory=False)
-    transport = httpx.ASGITransport(app=app)
-    async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
-        yield client
-
-
-@pytest.fixture
-def bank_id():
-    """Provide a unique bank ID for audit tests."""
-    from datetime import datetime
-
-    return f"audit_test_{datetime.now().timestamp()}"
-
-
-@pytest.mark.asyncio
-async def test_audit_log_list_empty(audit_api_client, bank_id):
-    """Test listing audit logs for a bank with no entries returns empty."""
-    # Create the bank first
-    await audit_api_client.put(
-        f"/v1/default/banks/{bank_id}",
-        json={"name": "Audit Test Bank"},
-    )
-
-    # Small delay for fire-and-forget audit writes
-    await asyncio.sleep(0.5)
-
-    response = await audit_api_client.get(f"/v1/default/banks/{bank_id}/audit-logs")
-    assert response.status_code == 200
-    data = response.json()
-    assert data["bank_id"] == bank_id
-    assert "total" in data
-    assert "items" in data
-    assert "limit" in data
-    assert "offset" in data
-    assert isinstance(data["items"], list)
-
-
-@pytest.mark.asyncio
-async def test_audit_log_created_for_retain(audit_api_client, bank_id):
-    """Test that a retain operation creates an audit log entry."""
-    # Create bank
-    await audit_api_client.put(
-        f"/v1/default/banks/{bank_id}",
-        json={"name": "Audit Test Bank"},
-    )
-
-    # Perform a retain
-    response = await audit_api_client.post(
-        f"/v1/default/banks/{bank_id}/memories",
-        json={
-            "items": [{"content": "Alice likes cats", "context": "preferences"}],
-        },
-    )
-    assert response.status_code == 200
-
-    # Wait for fire-and-forget audit writes
-    await asyncio.sleep(1.0)
-
-    # List audit logs - should have entries for create_bank and retain
-    response = await audit_api_client.get(f"/v1/default/banks/{bank_id}/audit-logs")
-    assert response.status_code == 200
-    data = response.json()
-    assert data["total"] >= 1
-
-    actions = [item["action"] for item in data["items"]]
-    assert "retain" in actions, f"Expected 'retain' in audit actions, got: {actions}"
-
-
-@pytest.mark.asyncio
-async def test_audit_log_entry_fields(audit_api_client, bank_id):
-    """Test that audit log entries have all expected fields."""
-    # Create bank + recall to generate entries
-    await audit_api_client.put(
-        f"/v1/default/banks/{bank_id}",
-        json={"name": "Audit Test Bank"},
-    )
-
-    await audit_api_client.post(
-        f"/v1/default/banks/{bank_id}/memories/recall",
-        json={"query": "test query"},
-    )
-
-    await asyncio.sleep(1.0)
-
-    response = await audit_api_client.get(f"/v1/default/banks/{bank_id}/audit-logs")
-    assert response.status_code == 200
-    data = response.json()
-    assert data["total"] >= 1
-
-    # Check the recall entry has all fields
-    recall_entries = [item for item in data["items"] if item["action"] == "recall"]
-    assert len(recall_entries) >= 1, f"Expected recall entry, got actions: {[i['action'] for i in data['items']]}"
-
-    entry = recall_entries[0]
-    assert entry["id"] is not None
-    assert entry["action"] == "recall"
-    assert entry["transport"] == "http"
-    assert entry["bank_id"] == bank_id
-    assert entry["started_at"] is not None
-    assert entry["ended_at"] is not None
-    # Request should contain the recall parameters
-    assert entry["request"] is not None
-    assert "query" in entry["request"]
-    # Response should contain the recall results
-    assert entry["response"] is not None
-
-
-@pytest.mark.asyncio
-async def test_audit_log_filter_by_action(audit_api_client, bank_id):
-    """Test filtering audit logs by action type."""
-    # Create bank and do retain + recall
-    await audit_api_client.put(
-        f"/v1/default/banks/{bank_id}",
-        json={"name": "Audit Test Bank"},
-    )
-    await audit_api_client.post(
-        f"/v1/default/banks/{bank_id}/memories",
-        json={"items": [{"content": "test content", "context": "test"}]},
-    )
-    await audit_api_client.post(
-        f"/v1/default/banks/{bank_id}/memories/recall",
-        json={"query": "test"},
-    )
-
-    await asyncio.sleep(1.0)
-
-    # Filter by retain only
-    response = await audit_api_client.get(
-        f"/v1/default/banks/{bank_id}/audit-logs",
-        params={"action": "retain"},
-    )
-    assert response.status_code == 200
-    data = response.json()
-    for item in data["items"]:
-        assert item["action"] == "retain"
-
-    # Filter by recall only
-    response = await audit_api_client.get(
-        f"/v1/default/banks/{bank_id}/audit-logs",
-        params={"action": "recall"},
-    )
-    assert response.status_code == 200
-    data = response.json()
-    for item in data["items"]:
-        assert item["action"] == "recall"
-
-
-@pytest.mark.asyncio
-async def test_audit_log_filter_by_transport(audit_api_client, bank_id):
-    """Test filtering audit logs by transport type."""
-    await audit_api_client.put(
-        f"/v1/default/banks/{bank_id}",
-        json={"name": "Audit Test Bank"},
-    )
-
-    await asyncio.sleep(0.5)
-
-    # Filter by http transport
-    response = await audit_api_client.get(
-        f"/v1/default/banks/{bank_id}/audit-logs",
-        params={"transport": "http"},
-    )
-    assert response.status_code == 200
-    data = response.json()
-    for item in data["items"]:
-        assert item["transport"] == "http"
-
-    # Filter by mcp transport - should be empty (no MCP calls in this test)
-    response = await audit_api_client.get(
-        f"/v1/default/banks/{bank_id}/audit-logs",
-        params={"transport": "mcp"},
-    )
-    assert response.status_code == 200
-    data = response.json()
-    assert data["total"] == 0
-
-
-@pytest.mark.asyncio
-async def test_audit_log_filter_by_date_range(audit_api_client, bank_id):
-    """Test filtering audit logs by date range."""
-    from datetime import datetime, timedelta, timezone
-
-    await audit_api_client.put(
-        f"/v1/default/banks/{bank_id}",
-        json={"name": "Audit Test Bank"},
-    )
-
-    await asyncio.sleep(0.5)
-
-    now = datetime.now(timezone.utc)
-
-    # Filter with start_date in the past - should include entries
-    past = (now - timedelta(hours=1)).isoformat()
-    response = await audit_api_client.get(
-        f"/v1/default/banks/{bank_id}/audit-logs",
-        params={"start_date": past},
-    )
-    assert response.status_code == 200
-    data = response.json()
-    assert data["total"] >= 1
-
-    # Filter with start_date in the future - should be empty
-    future = (now + timedelta(hours=1)).isoformat()
-    response = await audit_api_client.get(
-        f"/v1/default/banks/{bank_id}/audit-logs",
-        params={"start_date": future},
-    )
-    assert response.status_code == 200
-    data = response.json()
-    assert data["total"] == 0
-
-
-@pytest.mark.asyncio
-async def test_audit_log_pagination(audit_api_client, bank_id):
-    """Test audit log pagination with limit and offset."""
-    await audit_api_client.put(
-        f"/v1/default/banks/{bank_id}",
-        json={"name": "Audit Test Bank"},
-    )
-
-    # Generate multiple audit entries
-    for i in range(5):
-        await audit_api_client.post(
-            f"/v1/default/banks/{bank_id}/memories/recall",
-            json={"query": f"test query {i}"},
-        )
-
-    await asyncio.sleep(1.5)
-
-    # Get first page
-    response = await audit_api_client.get(
-        f"/v1/default/banks/{bank_id}/audit-logs",
-        params={"limit": 2, "offset": 0},
-    )
-    assert response.status_code == 200
-    page1 = response.json()
-    assert len(page1["items"]) == 2
-    assert page1["limit"] == 2
-    assert page1["offset"] == 0
-    assert page1["total"] >= 5  # At least 5 recall + 1 create_bank
-
-    # Get second page
-    response = await audit_api_client.get(
-        f"/v1/default/banks/{bank_id}/audit-logs",
-        params={"limit": 2, "offset": 2},
-    )
-    assert response.status_code == 200
-    page2 = response.json()
-    assert len(page2["items"]) == 2
-    assert page2["offset"] == 2
-
-    # Entries should be different between pages
-    page1_ids = {item["id"] for item in page1["items"]}
-    page2_ids = {item["id"] for item in page2["items"]}
-    assert page1_ids.isdisjoint(page2_ids), "Pages should not overlap"
-
-
-@pytest.mark.asyncio
-async def test_audit_log_stats(audit_api_client, bank_id):
-    """Test the audit log stats endpoint returns correct structure."""
-    await audit_api_client.put(
-        f"/v1/default/banks/{bank_id}",
-        json={"name": "Audit Test Bank"},
-    )
-
-    await audit_api_client.post(
-        f"/v1/default/banks/{bank_id}/memories/recall",
-        json={"query": "stats test"},
-    )
-
-    await asyncio.sleep(1.0)
-
-    # Get stats for last 24h
-    response = await audit_api_client.get(
-        f"/v1/default/banks/{bank_id}/audit-logs/stats",
-        params={"period": "1d"},
-    )
-    assert response.status_code == 200
-    data = response.json()
-    assert data["bank_id"] == bank_id
-    assert data["period"] == "1d"
-    assert data["trunc"] == "day"
-    assert "buckets" in data
-    assert isinstance(data["buckets"], list)
-
-    # Should have at least one bucket with our operations
-    assert len(data["buckets"]) >= 1
-    bucket = data["buckets"][0]
-    assert "time" in bucket
-    assert "actions" in bucket
-    assert "total" in bucket
-    assert bucket["total"] >= 1
-
-
-@pytest.mark.asyncio
-async def test_audit_log_stats_filter_by_action(audit_api_client, bank_id):
-    """Test stats endpoint filters by action."""
-    await audit_api_client.put(
-        f"/v1/default/banks/{bank_id}",
-        json={"name": "Audit Test Bank"},
-    )
-
-    await audit_api_client.post(
-        f"/v1/default/banks/{bank_id}/memories/recall",
-        json={"query": "test"},
-    )
-
-    await asyncio.sleep(1.0)
-
-    # Stats filtered by recall
-    response = await audit_api_client.get(
-        f"/v1/default/banks/{bank_id}/audit-logs/stats",
-        params={"period": "1d", "action": "recall"},
-    )
-    assert response.status_code == 200
-    data = response.json()
-    for bucket in data["buckets"]:
-        # All actions in buckets should be "recall" only
-        for action_name in bucket["actions"]:
-            assert action_name == "recall"
-
-
-@pytest.mark.asyncio
-async def test_audit_log_stats_periods(audit_api_client, bank_id):
-    """Test stats endpoint supports different periods."""
-    await audit_api_client.put(
-        f"/v1/default/banks/{bank_id}",
-        json={"name": "Audit Test Bank"},
-    )
-
-    await asyncio.sleep(0.5)
-
-    for period, expected_trunc in [("1d", "day"), ("7d", "day"), ("30d", "day")]:
-        response = await audit_api_client.get(
-            f"/v1/default/banks/{bank_id}/audit-logs/stats",
-            params={"period": period},
-        )
-        assert response.status_code == 200
-        data = response.json()
-        assert data["period"] == period
-        assert data["trunc"] == expected_trunc
-
-
-@pytest.mark.asyncio
-async def test_audit_log_disabled(memory):
-    """Test that no audit logs are created when audit logging is disabled."""
-    # Ensure audit logging is disabled
-    memory._audit_logger._enabled = False
-
-    app = create_app(memory, initialize_memory=False)
-    transport = httpx.ASGITransport(app=app)
-    async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
-        from datetime import datetime
-
-        bid = f"audit_disabled_test_{datetime.now().timestamp()}"
-
-        await client.put(f"/v1/default/banks/{bid}", json={"name": "No Audit"})
-        await client.post(
-            f"/v1/default/banks/{bid}/memories/recall",
-            json={"query": "test"},
-        )
-
-        await asyncio.sleep(0.5)
-
-        response = await client.get(f"/v1/default/banks/{bid}/audit-logs")
-        assert response.status_code == 200
-        data = response.json()
-        assert data["total"] == 0, "No audit entries should exist when audit logging is disabled"
-
-
-@pytest.mark.asyncio
-async def test_audit_log_action_allowlist(memory):
-    """Test that only allowed actions are audited when allowlist is set."""
-    memory._audit_logger._enabled = True
-    memory._audit_logger._allowed_actions = frozenset({"recall"})  # Only audit recall
-
-    app = create_app(memory, initialize_memory=False)
-    transport = httpx.ASGITransport(app=app)
-    async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
-        from datetime import datetime
-
-        bid = f"audit_allowlist_test_{datetime.now().timestamp()}"
-
-        # create_bank should NOT be audited
-        await client.put(f"/v1/default/banks/{bid}", json={"name": "Allowlist Test"})
-        # recall should be audited
-        await client.post(
-            f"/v1/default/banks/{bid}/memories/recall",
-            json={"query": "allowlist test"},
-        )
-
-        await asyncio.sleep(1.0)
-
-        response = await client.get(f"/v1/default/banks/{bid}/audit-logs")
-        assert response.status_code == 200
-        data = response.json()
-        actions = [item["action"] for item in data["items"]]
-        assert "recall" in actions, "recall should be audited"
-        assert "create_bank" not in actions, "create_bank should NOT be audited (not in allowlist)"
-
-
-@pytest.mark.asyncio
-async def test_audit_log_ordered_by_most_recent(audit_api_client, bank_id):
-    """Test that audit logs are returned ordered by most recent first."""
-    await audit_api_client.put(
-        f"/v1/default/banks/{bank_id}",
-        json={"name": "Order Test Bank"},
-    )
-
-    for i in range(3):
-        await audit_api_client.post(
-            f"/v1/default/banks/{bank_id}/memories/recall",
-            json={"query": f"order test {i}"},
-        )
-        await asyncio.sleep(0.2)  # Small gap between requests
-
-    await asyncio.sleep(1.0)
-
-    response = await audit_api_client.get(f"/v1/default/banks/{bank_id}/audit-logs")
-    assert response.status_code == 200
-    data = response.json()
-
-    # Check descending order by started_at
-    timestamps = [item["started_at"] for item in data["items"] if item["started_at"]]
-    assert timestamps == sorted(timestamps, reverse=True), "Audit logs should be ordered most recent first"

hindsight-api-slim/tests/test_bank_templates.py

@@ -1,600 +0,0 @@
-"""Integration tests for bank template import/export endpoints."""
-
-import pytest
-import pytest_asyncio
-import httpx
-from datetime import datetime
-from hindsight_api.api import create_app
-
-
-@pytest_asyncio.fixture
-async def api_client(memory):
-    """Create an async test client for the FastAPI app."""
-    app = create_app(memory, initialize_memory=False)
-    transport = httpx.ASGITransport(app=app)
-    async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
-        yield client
-
-
-@pytest.fixture
-def bank_id():
-    return f"template_test_{datetime.now().timestamp()}"
-
-
-@pytest.fixture
-def sample_template():
-    return {
-        "version": "1",
-        "bank": {
-            "reflect_mission": "Test mission for reflect",
-            "retain_mission": "Extract test data carefully",
-            "retain_extraction_mode": "verbose",
-            "disposition_empathy": 5,
-            "disposition_skepticism": 2,
-            "enable_observations": True,
-            "observations_mission": "Track test patterns",
-        },
-        "mental_models": [
-            {
-                "id": "test-model-one",
-                "name": "Test Model One",
-                "source_query": "What are the key patterns?",
-                "tags": ["test"],
-                "max_tokens": 1024,
-                "trigger": {"refresh_after_consolidation": True},
-            },
-            {
-                "id": "test-model-two",
-                "name": "Test Model Two",
-                "source_query": "What are the common issues?",
-            },
-        ],
-        "directives": [
-            {
-                "name": "Be concise",
-                "content": "Always respond concisely.",
-                "priority": 10,
-            },
-            {
-                "name": "Use examples",
-                "content": "Include examples when explaining concepts.",
-                "tags": ["style"],
-            },
-        ],
-    }
-
-
-class TestImportValidation:
-    """Test template manifest validation."""
-
-    @pytest.mark.asyncio
-    async def test_import_dry_run_valid(self, api_client, bank_id, sample_template):
-        """dry_run=true with a valid manifest returns what would happen."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import?dry_run=true",
-            json=sample_template,
-        )
-        assert resp.status_code == 200
-        data = resp.json()
-        assert data["dry_run"] is True
-        assert data["config_applied"] is True
-        assert set(data["mental_models_created"]) == {"test-model-one", "test-model-two"}
-        assert set(data["directives_created"]) == {"Be concise", "Use examples"}
-
-    @pytest.mark.asyncio
-    async def test_import_invalid_version(self, api_client, bank_id):
-        """Reject manifest with unsupported version."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={"version": "999"},
-        )
-        assert resp.status_code == 400
-
-    @pytest.mark.asyncio
-    async def test_import_invalid_extraction_mode(self, api_client, bank_id):
-        """Semantic validation catches bad extraction mode."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "bank": {"retain_extraction_mode": "invalid_mode"},
-            },
-        )
-        assert resp.status_code == 400
-        assert "retain_extraction_mode" in resp.json()["detail"]
-
-    @pytest.mark.asyncio
-    async def test_import_custom_instructions_without_custom_mode(self, api_client, bank_id):
-        """Validate that custom_instructions requires extraction_mode=custom."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "bank": {
-                    "retain_extraction_mode": "verbose",
-                    "retain_custom_instructions": "some custom prompt",
-                },
-            },
-        )
-        assert resp.status_code == 400
-        assert "retain_custom_instructions" in resp.json()["detail"]
-
-    @pytest.mark.asyncio
-    async def test_import_duplicate_mental_model_ids(self, api_client, bank_id):
-        """Reject manifest with duplicate mental model IDs."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "mental_models": [
-                    {"id": "dup-id", "name": "First", "source_query": "q1"},
-                    {"id": "dup-id", "name": "Second", "source_query": "q2"},
-                ],
-            },
-        )
-        assert resp.status_code == 400
-
-    @pytest.mark.asyncio
-    async def test_import_duplicate_directive_names(self, api_client, bank_id):
-        """Reject manifest with duplicate directive names."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "directives": [
-                    {"name": "Same Name", "content": "First"},
-                    {"name": "Same Name", "content": "Second"},
-                ],
-            },
-        )
-        assert resp.status_code == 400
-
-    @pytest.mark.asyncio
-    async def test_import_missing_mental_model_id(self, api_client, bank_id):
-        """Mental model without id is rejected."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "mental_models": [
-                    {"name": "No ID Model", "source_query": "test query"},
-                ],
-            },
-        )
-        assert resp.status_code == 400
-
-    @pytest.mark.asyncio
-    async def test_import_invalid_mental_model_id_format(self, api_client, bank_id):
-        """Mental model with invalid ID format is rejected."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "mental_models": [
-                    {"id": "UPPERCASE-NOT-ALLOWED", "name": "Bad", "source_query": "q"},
-                ],
-            },
-        )
-        assert resp.status_code == 400
-
-    @pytest.mark.asyncio
-    async def test_import_empty_manifest(self, api_client, bank_id):
-        """Import with no bank or mental_models is valid (no-op)."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={"version": "1"},
-        )
-        assert resp.status_code == 200
-        data = resp.json()
-        assert data["config_applied"] is False
-        assert data["mental_models_created"] == []
-        assert data["directives_created"] == []
-
-    @pytest.mark.asyncio
-    async def test_import_empty_mental_model_name(self, api_client, bank_id):
-        """Semantic validation catches empty mental model name."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "mental_models": [
-                    {"id": "test-mm", "name": "  ", "source_query": "q"},
-                ],
-            },
-        )
-        assert resp.status_code == 400
-        assert "name" in resp.json()["detail"]
-
-    @pytest.mark.asyncio
-    async def test_import_empty_directive_content(self, api_client, bank_id):
-        """Semantic validation catches empty directive content."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "directives": [
-                    {"name": "Bad Directive", "content": "  "},
-                ],
-            },
-        )
-        assert resp.status_code == 400
-        assert "content" in resp.json()["detail"]
-
-
-class TestImportApply:
-    """Test that import actually applies config, mental models, and directives."""
-
-    @pytest.mark.asyncio
-    async def test_import_applies_config(self, api_client, bank_id):
-        """Import with bank config applies config overrides on a new bank."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "bank": {
-                    "reflect_mission": "Imported mission",
-                    "disposition_empathy": 4,
-                },
-            },
-        )
-        assert resp.status_code == 200
-        data = resp.json()
-        assert data["config_applied"] is True
-        assert data["dry_run"] is False
-
-        # Verify config was actually applied
-        config_resp = await api_client.get(f"/v1/default/banks/{bank_id}/config")
-        assert config_resp.status_code == 200
-        config = config_resp.json()
-        assert config["overrides"]["reflect_mission"] == "Imported mission"
-        assert config["overrides"]["disposition_empathy"] == 4
-
-    @pytest.mark.asyncio
-    async def test_import_into_existing_bank(self, api_client, bank_id):
-        """Import into an already-existing bank applies config and creates resources."""
-        # Pre-create the bank
-        await api_client.put(f"/v1/default/banks/{bank_id}", json={})
-
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "bank": {"reflect_mission": "Existing bank mission"},
-                "mental_models": [
-                    {"id": "existing-bank-mm", "name": "MM", "source_query": "q"},
-                ],
-                "directives": [
-                    {"name": "Existing Bank Directive", "content": "Be helpful"},
-                ],
-            },
-        )
-        assert resp.status_code == 200
-        data = resp.json()
-        assert data["config_applied"] is True
-        assert "existing-bank-mm" in data["mental_models_created"]
-        assert "Existing Bank Directive" in data["directives_created"]
-
-        # Verify everything exists
-        config_resp = await api_client.get(f"/v1/default/banks/{bank_id}/config")
-        assert config_resp.json()["overrides"]["reflect_mission"] == "Existing bank mission"
-
-        mm_resp = await api_client.get(f"/v1/default/banks/{bank_id}/mental-models/existing-bank-mm")
-        assert mm_resp.status_code == 200
-
-        dir_resp = await api_client.get(f"/v1/default/banks/{bank_id}/directives")
-        assert dir_resp.status_code == 200
-        names = [d["name"] for d in dir_resp.json()["items"]]
-        assert "Existing Bank Directive" in names
-
-    @pytest.mark.asyncio
-    async def test_import_creates_mental_models(self, api_client, bank_id):
-        """Import creates mental models and returns operation IDs."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "mental_models": [
-                    {
-                        "id": "import-mm-1",
-                        "name": "Imported Model",
-                        "source_query": "What patterns exist?",
-                        "tags": ["imported"],
-                    },
-                ],
-            },
-        )
-        assert resp.status_code == 200
-        data = resp.json()
-        assert "import-mm-1" in data["mental_models_created"]
-        assert len(data["operation_ids"]) == 1
-
-        # Verify mental model exists
-        mm_resp = await api_client.get(f"/v1/default/banks/{bank_id}/mental-models/import-mm-1")
-        assert mm_resp.status_code == 200
-        mm = mm_resp.json()
-        assert mm["name"] == "Imported Model"
-        assert mm["source_query"] == "What patterns exist?"
-        assert mm["tags"] == ["imported"]
-
-    @pytest.mark.asyncio
-    async def test_import_updates_existing_mental_models(self, api_client, bank_id):
-        """Re-importing updates existing mental models matched by ID."""
-        # First import
-        await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "mental_models": [
-                    {
-                        "id": "reusable-mm",
-                        "name": "Original Name",
-                        "source_query": "Original query",
-                    },
-                ],
-            },
-        )
-
-        # Second import with same ID but different content
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "mental_models": [
-                    {
-                        "id": "reusable-mm",
-                        "name": "Updated Name",
-                        "source_query": "Updated query",
-                    },
-                ],
-            },
-        )
-        assert resp.status_code == 200
-        data = resp.json()
-        assert "reusable-mm" in data["mental_models_updated"]
-        assert data["mental_models_created"] == []
-
-        # Verify update
-        mm_resp = await api_client.get(f"/v1/default/banks/{bank_id}/mental-models/reusable-mm")
-        assert mm_resp.status_code == 200
-        mm = mm_resp.json()
-        assert mm["name"] == "Updated Name"
-        assert mm["source_query"] == "Updated query"
-
-    @pytest.mark.asyncio
-    async def test_import_creates_directives(self, api_client, bank_id):
-        """Import creates directives."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "directives": [
-                    {
-                        "name": "Test Directive",
-                        "content": "Always be helpful and precise.",
-                        "priority": 5,
-                        "tags": ["test"],
-                    },
-                ],
-            },
-        )
-        assert resp.status_code == 200
-        data = resp.json()
-        assert "Test Directive" in data["directives_created"]
-        assert data["directives_updated"] == []
-
-        # Verify directive exists
-        dir_resp = await api_client.get(f"/v1/default/banks/{bank_id}/directives")
-        assert dir_resp.status_code == 200
-        items = dir_resp.json()["items"]
-        assert len(items) == 1
-        assert items[0]["name"] == "Test Directive"
-        assert items[0]["content"] == "Always be helpful and precise."
-        assert items[0]["priority"] == 5
-        assert items[0]["tags"] == ["test"]
-
-    @pytest.mark.asyncio
-    async def test_import_updates_existing_directives(self, api_client, bank_id):
-        """Re-importing updates existing directives matched by name."""
-        # First import
-        await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "directives": [
-                    {"name": "Reusable Directive", "content": "Original content", "priority": 1},
-                ],
-            },
-        )
-
-        # Second import with same name but different content
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "directives": [
-                    {"name": "Reusable Directive", "content": "Updated content", "priority": 10},
-                ],
-            },
-        )
-        assert resp.status_code == 200
-        data = resp.json()
-        assert "Reusable Directive" in data["directives_updated"]
-        assert data["directives_created"] == []
-
-        # Verify update
-        dir_resp = await api_client.get(f"/v1/default/banks/{bank_id}/directives")
-        items = dir_resp.json()["items"]
-        directive = [d for d in items if d["name"] == "Reusable Directive"][0]
-        assert directive["content"] == "Updated content"
-        assert directive["priority"] == 10
-
-    @pytest.mark.asyncio
-    async def test_import_config_only(self, api_client, bank_id):
-        """Import with only bank config (no mental_models or directives) works."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "bank": {"retain_extraction_mode": "verbose"},
-            },
-        )
-        assert resp.status_code == 200
-        data = resp.json()
-        assert data["config_applied"] is True
-        assert data["mental_models_created"] == []
-        assert data["directives_created"] == []
-        assert data["operation_ids"] == []
-
-    @pytest.mark.asyncio
-    async def test_import_mental_models_only(self, api_client, bank_id):
-        """Import with only mental_models works."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "mental_models": [
-                    {"id": "mm-only", "name": "MM Only", "source_query": "test"},
-                ],
-            },
-        )
-        assert resp.status_code == 200
-        data = resp.json()
-        assert data["config_applied"] is False
-        assert "mm-only" in data["mental_models_created"]
-        assert data["directives_created"] == []
-
-    @pytest.mark.asyncio
-    async def test_import_directives_only(self, api_client, bank_id):
-        """Import with only directives works."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "directives": [
-                    {"name": "Dir Only", "content": "test directive"},
-                ],
-            },
-        )
-        assert resp.status_code == 200
-        data = resp.json()
-        assert data["config_applied"] is False
-        assert data["mental_models_created"] == []
-        assert "Dir Only" in data["directives_created"]
-
-
-class TestExport:
-    """Test bank template export."""
-
-    @pytest.mark.asyncio
-    async def test_export_empty_bank(self, api_client, bank_id):
-        """Export a bank with no overrides returns minimal manifest."""
-        # Create bank
-        await api_client.put(f"/v1/default/banks/{bank_id}", json={})
-
-        resp = await api_client.get(f"/v1/default/banks/{bank_id}/export")
-        assert resp.status_code == 200
-        data = resp.json()
-        assert data["version"] == "1"
-        assert data["bank"] is None
-        assert data["mental_models"] is None
-        assert data["directives"] is None
-
-    @pytest.mark.asyncio
-    async def test_export_after_import(self, api_client, bank_id):
-        """Export after import returns the imported config, mental models, and directives."""
-        template = {
-            "version": "1",
-            "bank": {
-                "reflect_mission": "Roundtrip mission",
-                "disposition_empathy": 3,
-            },
-            "mental_models": [
-                {
-                    "id": "roundtrip-mm",
-                    "name": "Roundtrip Model",
-                    "source_query": "What happened?",
-                    "tags": ["roundtrip"],
-                    "max_tokens": 512,
-                },
-            ],
-            "directives": [
-                {
-                    "name": "Roundtrip Directive",
-                    "content": "Be thorough.",
-                    "priority": 3,
-                    "tags": ["roundtrip"],
-                },
-            ],
-        }
-
-        # Import
-        import_resp = await api_client.post(f"/v1/default/banks/{bank_id}/import", json=template)
-        assert import_resp.status_code == 200
-
-        # Export
-        resp = await api_client.get(f"/v1/default/banks/{bank_id}/export")
-        assert resp.status_code == 200
-        data = resp.json()
-
-        assert data["version"] == "1"
-        assert data["bank"]["reflect_mission"] == "Roundtrip mission"
-        assert data["bank"]["disposition_empathy"] == 3
-
-        assert len(data["mental_models"]) == 1
-        mm = data["mental_models"][0]
-        assert mm["id"] == "roundtrip-mm"
-        assert mm["name"] == "Roundtrip Model"
-        assert mm["source_query"] == "What happened?"
-        assert mm["tags"] == ["roundtrip"]
-        assert mm["max_tokens"] == 512
-
-        assert len(data["directives"]) == 1
-        d = data["directives"][0]
-        assert d["name"] == "Roundtrip Directive"
-        assert d["content"] == "Be thorough."
-        assert d["priority"] == 3
-        assert d["tags"] == ["roundtrip"]
-
-    @pytest.mark.asyncio
-    async def test_export_reimport_roundtrip(self, api_client, bank_id):
-        """Exported manifest can be re-imported into a new bank."""
-        # Set up source bank
-        await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "bank": {"retain_mission": "Roundtrip test"},
-                "mental_models": [
-                    {"id": "rt-mm", "name": "RT Model", "source_query": "test query"},
-                ],
-                "directives": [
-                    {"name": "RT Directive", "content": "test directive"},
-                ],
-            },
-        )
-
-        # Export
-        export_resp = await api_client.get(f"/v1/default/banks/{bank_id}/export")
-        assert export_resp.status_code == 200
-        exported = export_resp.json()
-
-        # Import into a new bank
-        new_bank_id = f"{bank_id}_clone"
-        import_resp = await api_client.post(
-            f"/v1/default/banks/{new_bank_id}/import",
-            json=exported,
-        )
-        assert import_resp.status_code == 200
-        data = import_resp.json()
-        assert data["config_applied"] is True
-        assert "rt-mm" in data["mental_models_created"]
-        assert "RT Directive" in data["directives_created"]
-
-    @pytest.mark.asyncio
-    async def test_export_nonexistent_bank(self, api_client):
-        """Export from a nonexistent bank returns the bank with defaults (auto-created)."""
-        resp = await api_client.get("/v1/default/banks/nonexistent-export-test/export")
-        # get_bank_profile auto-creates, so this returns a valid empty manifest
-        assert resp.status_code == 200
-        data = resp.json()
-        assert data["version"] == "1"

hindsight-api-slim/tests/test_codex_tool_choice.py

@@ -1,88 +0,0 @@
-"""
-Regression tests for Codex provider tool_choice normalization.
-
-The reflect agent forces tool selection via named tool_choice dicts on early iterations:
-  {"type": "function", "function": {"name": "recall"}}
-
-The Codex Responses API expects the function name at the top level instead:
-  {"type": "function", "name": "recall"}
-
-Without normalization, Codex rejects the request with:
-  400 Unknown parameter: 'tool_choice.function'
-"""
-
-from unittest.mock import AsyncMock, MagicMock, patch
-
-import pytest
-
-from hindsight_api.engine.providers.codex_llm import CodexLLM
-
-TOOLS = [
-    {
-        "type": "function",
-        "function": {
-            "name": "recall",
-            "description": "Recall semantic memories",
-            "parameters": {
-                "type": "object",
-                "properties": {"query": {"type": "string"}},
-                "required": ["query"],
-            },
-        },
-    }
-]
-
-
-def build_llm() -> CodexLLM:
-    with patch.object(CodexLLM, "_load_codex_auth", return_value=("token", "account")):
-        return CodexLLM(
-            provider="openai-codex",
-            api_key="ignored",
-            base_url="https://chatgpt.com/backend-api",
-            model="gpt-5.4-mini",
-        )
-
-
-@pytest.mark.asyncio
-async def test_codex_normalizes_legacy_named_tool_choice_shape():
-    llm = build_llm()
-    response = MagicMock()
-    response.status_code = 200
-    response.raise_for_status.return_value = None
-    with patch.object(llm._client, "post", new_callable=AsyncMock) as mock_post:
-        mock_post.return_value = response
-        with patch.object(llm, "_parse_sse_tool_stream", new_callable=AsyncMock) as mock_parse:
-            mock_parse.return_value = (None, [])
-            await llm.call_with_tools(
-                messages=[{"role": "user", "content": "recall the memory"}],
-                tools=TOOLS,
-                tool_choice={"type": "function", "function": {"name": "recall"}},
-                max_retries=0,
-            )
-        sent_payload = mock_post.call_args.kwargs["json"]
-
-    assert sent_payload["tool_choice"] == {"type": "function", "name": "recall"}
-
-
-@pytest.mark.asyncio
-async def test_codex_forced_tool_choice_still_yields_tool_calls():
-    llm = build_llm()
-    response = MagicMock()
-    response.status_code = 200
-    response.raise_for_status.return_value = None
-    tool_call = {"id": "call-1", "name": "recall", "arguments": {"query": "memory"}}
-    with patch.object(llm._client, "post", new_callable=AsyncMock) as mock_post:
-        mock_post.return_value = response
-        with patch.object(llm, "_parse_sse_tool_stream", new_callable=AsyncMock) as mock_parse:
-            mock_parse.return_value = (None, [tool_call])
-            result = await llm.call_with_tools(
-                messages=[{"role": "user", "content": "recall the memory"}],
-                tools=TOOLS,
-                tool_choice={"type": "function", "function": {"name": "recall"}},
-                max_retries=0,
-            )
-        sent_payload = mock_post.call_args.kwargs["json"]
-
-    assert len(result.tool_calls) == 1
-    assert result.tool_calls[0].name == "recall"
-    assert sent_payload["tool_choice"] == {"type": "function", "name": "recall"}

hindsight-api-slim/tests/test_cohere_cross_encoder.py

@@ -1,339 +0,0 @@
-"""
-Tests for CohereCrossEncoder.
-
-Tests the Cohere cross-encoder implementation, including Azure AI Foundry endpoint support.
-"""
-
-import os
-from unittest.mock import MagicMock, patch
-
-import httpx
-import pytest
-
-from hindsight_api.engine.cross_encoder import CohereCrossEncoder, create_cross_encoder_from_env
-
-
-class TestCohereCrossEncoder:
-    """Test suite for CohereCrossEncoder class."""
-
-    @pytest.mark.asyncio
-    async def test_initialization_native_cohere(self):
-        """Test successful initialization with native Cohere API (no base_url)."""
-        encoder = CohereCrossEncoder(
-            api_key="test_key",
-            model="rerank-english-v3.0",
-        )
-
-        assert encoder.provider_name == "cohere"
-        assert encoder.api_key == "test_key"
-        assert encoder.model == "rerank-english-v3.0"
-        assert encoder._client is None
-        assert encoder._httpx_client is None
-
-        # Mock the cohere import
-        mock_cohere = MagicMock()
-        mock_cohere.Client = MagicMock()
-        with patch.dict("sys.modules", {"cohere": mock_cohere}):
-            await encoder.initialize()
-            assert encoder._client is not None
-            assert encoder._httpx_client is None
-            mock_cohere.Client.assert_called_once_with(api_key="test_key", timeout=60.0)
-
-    @pytest.mark.asyncio
-    async def test_initialization_azure_endpoint(self):
-        """Test initialization with Azure AI Foundry endpoint (uses httpx)."""
-        encoder = CohereCrossEncoder(
-            api_key="test_key",
-            model="cohere-rerank-v3-english",
-            base_url="https://my-endpoint.inference.ai.azure.com/models/cohere-rerank-v3-english/invoke",
-        )
-
-        assert encoder.base_url == "https://my-endpoint.inference.ai.azure.com/models/cohere-rerank-v3-english/invoke"
-
-        await encoder.initialize()
-
-        assert encoder._httpx_client is not None
-        assert encoder._client is None
-        assert isinstance(encoder._httpx_client, httpx.Client)
-
-    @pytest.mark.asyncio
-    async def test_initialization_missing_package(self):
-        """Test initialization fails when cohere package is missing (native API)."""
-        encoder = CohereCrossEncoder(
-            api_key="test_key",
-            model="rerank-english-v3.0",
-        )
-
-        with patch.dict("sys.modules", {"cohere": None}):
-            with pytest.raises(ImportError, match="cohere is required"):
-                await encoder.initialize()
-
-    @pytest.mark.asyncio
-    async def test_initialization_idempotent(self):
-        """Test that calling initialize() multiple times is safe."""
-        encoder = CohereCrossEncoder(
-            api_key="test_key",
-            model="rerank-english-v3.0",
-        )
-
-        mock_cohere = MagicMock()
-        mock_cohere.Client = MagicMock()
-        with patch.dict("sys.modules", {"cohere": mock_cohere}):
-            await encoder.initialize()
-            assert encoder._client is not None
-
-            # Second call should be no-op
-            await encoder.initialize()
-            # Should only create client once
-            mock_cohere.Client.assert_called_once()
-
-    @pytest.mark.asyncio
-    async def test_predict_native_cohere_single_query(self):
-        """Test prediction with native Cohere SDK."""
-        encoder = CohereCrossEncoder(
-            api_key="test_key",
-            model="rerank-english-v3.0",
-        )
-
-        # Create mock Cohere response
-        mock_result_1 = MagicMock()
-        mock_result_1.index = 0
-        mock_result_1.relevance_score = 0.9
-
-        mock_result_2 = MagicMock()
-        mock_result_2.index = 1
-        mock_result_2.relevance_score = 0.7
-
-        mock_result_3 = MagicMock()
-        mock_result_3.index = 2
-        mock_result_3.relevance_score = 0.5
-
-        mock_response = MagicMock()
-        mock_response.results = [mock_result_1, mock_result_2, mock_result_3]
-
-        mock_cohere_client = MagicMock()
-        mock_cohere_client.rerank = MagicMock(return_value=mock_response)
-
-        mock_cohere = MagicMock()
-        mock_cohere.Client = MagicMock(return_value=mock_cohere_client)
-
-        with patch.dict("sys.modules", {"cohere": mock_cohere}):
-            await encoder.initialize()
-
-            pairs = [
-                ("What is Python?", "Python is a programming language"),
-                ("What is Python?", "Python is a snake"),
-                ("What is Python?", "Python is a British comedy group"),
-            ]
-
-            scores = await encoder.predict(pairs)
-
-            assert len(scores) == 3
-            assert scores == [0.9, 0.7, 0.5]
-
-            # Verify rerank was called correctly
-            mock_cohere_client.rerank.assert_called_once()
-            call_args = mock_cohere_client.rerank.call_args
-            assert call_args.kwargs["model"] == "rerank-english-v3.0"
-            assert call_args.kwargs["query"] == "What is Python?"
-            assert len(call_args.kwargs["documents"]) == 3
-            assert call_args.kwargs["return_documents"] is False
-
-    @pytest.mark.asyncio
-    async def test_predict_azure_endpoint_single_query(self):
-        """Test prediction with Azure AI Foundry endpoint (httpx direct call)."""
-        encoder = CohereCrossEncoder(
-            api_key="test_key",
-            model="cohere-rerank-v3-english",
-            base_url="https://my-endpoint.inference.ai.azure.com/models/cohere-rerank-v3-english/invoke",
-        )
-
-        await encoder.initialize()
-
-        # Mock httpx response
-        mock_response = MagicMock()
-        mock_response.json.return_value = {
-            "results": [
-                {"index": 0, "relevance_score": 0.9},
-                {"index": 1, "relevance_score": 0.7},
-                {"index": 2, "relevance_score": 0.5},
-            ]
-        }
-
-        encoder._httpx_client.post = MagicMock(return_value=mock_response)
-
-        pairs = [
-            ("What is Python?", "Python is a programming language"),
-            ("What is Python?", "Python is a snake"),
-            ("What is Python?", "Python is a British comedy group"),
-        ]
-
-        scores = await encoder.predict(pairs)
-
-        assert len(scores) == 3
-        assert scores == [0.9, 0.7, 0.5]
-
-        # Verify httpx.post was called with correct URL and payload
-        encoder._httpx_client.post.assert_called_once()
-        call_args = encoder._httpx_client.post.call_args
-        assert call_args[0][0] == "https://my-endpoint.inference.ai.azure.com/models/cohere-rerank-v3-english/invoke"
-        assert call_args.kwargs["json"]["model"] == "cohere-rerank-v3-english"
-        assert call_args.kwargs["json"]["query"] == "What is Python?"
-        assert len(call_args.kwargs["json"]["documents"]) == 3
-        assert call_args.kwargs["json"]["return_documents"] is False
-
-    @pytest.mark.asyncio
-    async def test_predict_multiple_queries(self):
-        """Test prediction with multiple different queries (grouped efficiently)."""
-        encoder = CohereCrossEncoder(
-            api_key="test_key",
-            model="rerank-english-v3.0",
-        )
-
-        # First query response
-        mock_result_1_1 = MagicMock()
-        mock_result_1_1.index = 0
-        mock_result_1_1.relevance_score = 0.9
-
-        mock_result_1_2 = MagicMock()
-        mock_result_1_2.index = 1
-        mock_result_1_2.relevance_score = 0.7
-
-        mock_response1 = MagicMock()
-        mock_response1.results = [mock_result_1_1, mock_result_1_2]
-
-        # Second query response
-        mock_result_2_1 = MagicMock()
-        mock_result_2_1.index = 0
-        mock_result_2_1.relevance_score = 0.8
-
-        mock_response2 = MagicMock()
-        mock_response2.results = [mock_result_2_1]
-
-        mock_cohere_client = MagicMock()
-        mock_cohere_client.rerank = MagicMock(side_effect=[mock_response1, mock_response2])
-
-        mock_cohere = MagicMock()
-        mock_cohere.Client = MagicMock(return_value=mock_cohere_client)
-
-        with patch.dict("sys.modules", {"cohere": mock_cohere}):
-            await encoder.initialize()
-
-            pairs = [
-                ("What is Python?", "Python is a programming language"),
-                ("What is Python?", "Python is a snake"),
-                ("What is Java?", "Java is a programming language"),
-            ]
-
-            scores = await encoder.predict(pairs)
-
-            assert len(scores) == 3
-            assert scores[0] == 0.9  # First query, first doc
-            assert scores[1] == 0.7  # First query, second doc
-            assert scores[2] == 0.8  # Second query, first doc
-
-            # Verify rerank was called twice (once per unique query)
-            assert mock_cohere_client.rerank.call_count == 2
-
-    @pytest.mark.asyncio
-    async def test_predict_empty_pairs(self):
-        """Test prediction with empty input."""
-        encoder = CohereCrossEncoder(
-            api_key="test_key",
-            model="rerank-english-v3.0",
-        )
-
-        mock_cohere = MagicMock()
-        mock_cohere.Client = MagicMock()
-        with patch.dict("sys.modules", {"cohere": mock_cohere}):
-            await encoder.initialize()
-            scores = await encoder.predict([])
-            assert scores == []
-
-    @pytest.mark.asyncio
-    async def test_predict_not_initialized(self):
-        """Test that predict fails if encoder not initialized."""
-        encoder = CohereCrossEncoder(
-            api_key="test_key",
-            model="rerank-english-v3.0",
-        )
-
-        pairs = [("query", "document")]
-
-        with pytest.raises(RuntimeError, match="not initialized"):
-            await encoder.predict(pairs)
-
-    @pytest.mark.asyncio
-    async def test_azure_endpoint_http_error(self):
-        """Test that HTTP errors from Azure endpoint are raised."""
-        encoder = CohereCrossEncoder(
-            api_key="test_key",
-            model="cohere-rerank-v3-english",
-            base_url="https://my-endpoint.inference.ai.azure.com/models/cohere-rerank-v3-english/invoke",
-        )
-
-        await encoder.initialize()
-
-        # Mock httpx to raise HTTP error
-        mock_response = MagicMock()
-        mock_response.raise_for_status.side_effect = httpx.HTTPStatusError(
-            "404 Not Found",
-            request=MagicMock(),
-            response=MagicMock(status_code=404),
-        )
-        encoder._httpx_client.post = MagicMock(return_value=mock_response)
-
-        pairs = [("What is Python?", "Python is a programming language")]
-
-        # Should raise the HTTP error
-        with pytest.raises(httpx.HTTPStatusError):
-            await encoder.predict(pairs)
-
-
-class TestFactoryFunction:
-    """Test suite for create_cross_encoder_from_env factory function."""
-
-    @pytest.mark.asyncio
-    async def test_create_cohere_from_env(self):
-        """Test creating Cohere cross-encoder from environment variables."""
-        env_vars = {
-            "HINDSIGHT_API_RERANKER_PROVIDER": "cohere",
-            "HINDSIGHT_API_RERANKER_COHERE_API_KEY": "test_key",
-            "HINDSIGHT_API_RERANKER_COHERE_MODEL": "rerank-english-v3.0",
-        }
-
-        with patch.dict(os.environ, env_vars, clear=False):
-            from hindsight_api.config import HindsightConfig
-
-            config = HindsightConfig.from_env()
-
-            with patch("hindsight_api.config.get_config", return_value=config):
-                encoder = create_cross_encoder_from_env()
-
-                assert isinstance(encoder, CohereCrossEncoder)
-                assert encoder.api_key == "test_key"
-                assert encoder.model == "rerank-english-v3.0"
-                assert encoder.base_url is None
-
-    @pytest.mark.asyncio
-    async def test_create_cohere_with_azure_base_url_from_env(self):
-        """Test creating Cohere cross-encoder with Azure base URL from environment."""
-        env_vars = {
-            "HINDSIGHT_API_RERANKER_PROVIDER": "cohere",
-            "HINDSIGHT_API_RERANKER_COHERE_API_KEY": "test_key",
-            "HINDSIGHT_API_RERANKER_COHERE_MODEL": "cohere-rerank-v3-english",
-            "HINDSIGHT_API_RERANKER_COHERE_BASE_URL": "https://my-endpoint.inference.ai.azure.com/models/cohere-rerank-v3-english/invoke",
-        }
-
-        with patch.dict(os.environ, env_vars, clear=False):
-            from hindsight_api.config import HindsightConfig
-
-            config = HindsightConfig.from_env()
-
-            with patch("hindsight_api.config.get_config", return_value=config):
-                encoder = create_cross_encoder_from_env()
-
-                assert isinstance(encoder, CohereCrossEncoder)
-                assert encoder.api_key == "test_key"
-                assert encoder.model == "cohere-rerank-v3-english"
-                assert encoder.base_url == "https://my-endpoint.inference.ai.azure.com/models/cohere-rerank-v3-english/invoke"

hindsight-api-slim/tests/test_combined_scoring.py

@@ -1,175 +0,0 @@
-"""
-Tests for combined scoring (apply_combined_scoring).
-
-The function applies multiplicative recency/temporal boosts to the cross-encoder
-score so that the relative influence of these signals is proportional to the base
-relevance score, independent of the cross-encoder model's score calibration.
-"""
-
-from datetime import datetime, timedelta, timezone
-
-import pytest
-
-from hindsight_api.engine.search.reranking import apply_combined_scoring, _RECENCY_ALPHA, _TEMPORAL_ALPHA
-from hindsight_api.engine.search.types import MergedCandidate, RetrievalResult, ScoredResult
-
-UTC = timezone.utc
-NOW = datetime(2024, 6, 1, tzinfo=UTC)
-
-
-def _make_result(
-    ce_norm: float,
-    occurred_start: datetime | None = None,
-    temporal_proximity: float | None = None,
-) -> ScoredResult:
-    retrieval = RetrievalResult(
-        id="test",
-        text="test",
-        fact_type="world",
-        occurred_start=occurred_start,
-        temporal_proximity=temporal_proximity,
-    )
-
-    candidate = MergedCandidate(
-        retrieval=retrieval,
-        rrf_score=0.05,
-    )
-
-    return ScoredResult(
-        candidate=candidate,
-        cross_encoder_score=1.0,
-        cross_encoder_score_normalized=ce_norm,
-        weight=ce_norm,
-    )
-
-
-class TestBoostFormula:
-    def test_neutral_signals_leave_score_unchanged(self):
-        """recency=0.5 and temporal=0.5 both produce boost=1.0, so weight == ce."""
-        sr = _make_result(ce_norm=0.6)
-        apply_combined_scoring([sr], now=NOW)
-        assert abs(sr.weight - 0.6) < 1e-9
-
-    def test_max_recency_boost(self):
-        """A memory from today (recency≈1.0) should boost by (1 + alpha*0.5)."""
-        sr = _make_result(ce_norm=0.5, occurred_start=NOW)
-        apply_combined_scoring([sr], now=NOW)
-        expected = 0.5 * (1.0 + _RECENCY_ALPHA * 0.5) * 1.0  # temporal neutral
-        assert abs(sr.weight - expected) < 1e-6
-
-    def test_min_recency_penalty(self):
-        """A memory from >365 days ago (recency=0.1) should penalise score."""
-        old = NOW - timedelta(days=400)
-        sr = _make_result(ce_norm=0.5, occurred_start=old)
-        apply_combined_scoring([sr], now=NOW)
-        expected = 0.5 * (1.0 + _RECENCY_ALPHA * (0.1 - 0.5)) * 1.0
-        assert abs(sr.weight - expected) < 1e-6
-
-    def test_max_temporal_boost(self):
-        """temporal_proximity=1.0 should boost by (1 + alpha*0.5)."""
-        sr = _make_result(ce_norm=0.5, temporal_proximity=1.0)
-        apply_combined_scoring([sr], now=NOW)
-        expected = 0.5 * 1.0 * (1.0 + _TEMPORAL_ALPHA * 0.5)  # recency neutral
-        assert abs(sr.weight - expected) < 1e-6
-
-    def test_temporal_none_is_neutral(self):
-        """temporal_proximity=None must be treated as 0.5 (no boost/penalty)."""
-        sr_none = _make_result(ce_norm=0.5, temporal_proximity=None)
-        sr_half = _make_result(ce_norm=0.5, temporal_proximity=0.5)
-        apply_combined_scoring([sr_none], now=NOW)
-        apply_combined_scoring([sr_half], now=NOW)
-        assert abs(sr_none.weight - sr_half.weight) < 1e-9
-
-    def test_both_signals_combined(self):
-        """Both boosts are applied multiplicatively."""
-        sr = _make_result(ce_norm=0.5, occurred_start=NOW, temporal_proximity=1.0)
-        apply_combined_scoring([sr], now=NOW)
-        recency_boost = 1.0 + _RECENCY_ALPHA * (1.0 - 0.5)
-        temporal_boost = 1.0 + _TEMPORAL_ALPHA * (1.0 - 0.5)
-        expected = 0.5 * recency_boost * temporal_boost
-        assert abs(sr.weight - expected) < 1e-6
-
-    def test_boost_is_proportional_to_ce(self):
-        """The absolute boost from recency scales with the CE score."""
-        sr_high = _make_result(ce_norm=0.9, occurred_start=NOW)
-        sr_low = _make_result(ce_norm=0.3, occurred_start=NOW)
-        apply_combined_scoring([sr_high, sr_low], now=NOW)
-
-        # Both get the same recency boost factor — absolute gain is proportional to CE
-        boost_factor = 1.0 + _RECENCY_ALPHA * 0.5
-        assert abs(sr_high.weight - 0.9 * boost_factor) < 1e-6
-        assert abs(sr_low.weight - 0.3 * boost_factor) < 1e-6
-
-    def test_boost_capped(self):
-        """Max boost: recency=1.0 + temporal=1.0 gives ≤21% uplift on CE."""
-        sr = _make_result(ce_norm=1.0, occurred_start=NOW, temporal_proximity=1.0)
-        apply_combined_scoring([sr], now=NOW)
-        assert sr.weight <= 1.0 * (1 + _RECENCY_ALPHA / 2) * (1 + _TEMPORAL_ALPHA / 2) + 1e-9
-
-    def test_rrf_normalized_always_zero(self):
-        """RRF is excluded from scoring; rrf_normalized is set to 0.0 for trace clarity."""
-        sr = _make_result(ce_norm=0.5)
-        apply_combined_scoring([sr], now=NOW)
-        assert sr.rrf_normalized == 0.0
-
-    def test_combined_score_equals_weight(self):
-        """combined_score and weight must stay in sync."""
-        sr = _make_result(ce_norm=0.7, occurred_start=NOW, temporal_proximity=0.8)
-        apply_combined_scoring([sr], now=NOW)
-        assert sr.combined_score == sr.weight
-
-    def test_model_calibration_independence(self):
-        """
-        A low-calibration model (low CE scores) and a high-calibration model
-        (high CE scores) should produce the same ranking for identical content.
-
-        With additive scoring the recency term would dominate for low-CE models;
-        with multiplicative boosting the relative ranking is stable.
-        """
-        recent = NOW - timedelta(days=10)
-        old = NOW - timedelta(days=300)
-
-        # High-calibration model: clear winner is #1 (more relevant, slightly older)
-        h_relevant = _make_result(ce_norm=0.85, occurred_start=old)
-        h_recent = _make_result(ce_norm=0.60, occurred_start=recent)
-        apply_combined_scoring([h_relevant, h_recent], now=NOW)
-        assert h_relevant.weight > h_recent.weight, "High-CE model: relevance should win"
-
-        # Low-calibration model: same relative difference, just compressed scores
-        l_relevant = _make_result(ce_norm=0.34, occurred_start=old)
-        l_recent = _make_result(ce_norm=0.24, occurred_start=recent)
-        apply_combined_scoring([l_relevant, l_recent], now=NOW)
-        assert l_relevant.weight > l_recent.weight, "Low-CE model: relevance should still win"
-
-    def test_no_occurred_start_defaults_recency_neutral(self):
-        """Missing occurred_start → recency=0.5 → no boost/penalty."""
-        sr = _make_result(ce_norm=0.5, occurred_start=None)
-        apply_combined_scoring([sr], now=NOW)
-        assert sr.recency == 0.5
-        assert abs(sr.weight - 0.5) < 1e-9
-
-    def test_timezone_naive_occurred_start_handled(self):
-        """Naive datetimes in occurred_start should not raise."""
-        naive_date = datetime(2024, 1, 1)  # no tzinfo
-        sr = _make_result(ce_norm=0.5, occurred_start=naive_date)
-        apply_combined_scoring([sr], now=NOW)  # must not raise
-        assert 0.0 < sr.weight < 1.0
-
-    def test_custom_alpha_values(self):
-        """Custom alpha parameters are respected."""
-        sr = _make_result(ce_norm=0.5, occurred_start=NOW)
-        apply_combined_scoring([sr], now=NOW, recency_alpha=0.4, temporal_alpha=0.0)
-        expected = 0.5 * (1.0 + 0.4 * 0.5) * 1.0
-        assert abs(sr.weight - expected) < 1e-6
-
-    def test_future_event_recency_capped_at_one(self):
-        """Events in the future must not produce recency > 1.0, keeping boost within bounds."""
-        future = NOW + timedelta(days=180)
-        sr = _make_result(ce_norm=0.5, occurred_start=future)
-        apply_combined_scoring([sr], now=NOW)
-        assert sr.recency == 1.0
-        expected_max_boost = 1.0 + _RECENCY_ALPHA * 0.5
-        assert sr.weight <= 0.5 * expected_max_boost + 1e-9
-
-    def test_empty_list_is_noop(self):
-        apply_combined_scoring([], now=NOW)  # must not raise

hindsight-api-slim/tests/test_consolidation_failure_recovery.py

@@ -1,476 +0,0 @@
-"""Tests for consolidation failure handling: adaptive batch splitting, consolidation_failed_at,
-and the recovery API.
-
-These tests use a mock LLM to simulate LLM failures deterministically, without making real
-API calls. All tests insert memories directly into the database to bypass retain's LLM calls
-and focus exclusively on the consolidation code paths.
-"""
-
-import uuid
-from unittest.mock import MagicMock
-
-import pytest
-import pytest_asyncio
-
-from hindsight_api.engine.consolidation.consolidator import run_consolidation_job
-from hindsight_api.engine.memory_engine import MemoryEngine
-from hindsight_api.engine.providers.mock_llm import MockLLM
-from hindsight_api.engine.task_backend import SyncTaskBackend
-
-
-@pytest_asyncio.fixture(scope="function")
-async def memory_no_llm_verify(pg0_db_url, embeddings, cross_encoder, query_analyzer):
-    """MemoryEngine with mock LLM.
-
-    Migrations are already applied by the session-scoped pg0_db_url fixture, so
-    run_migrations=False avoids advisory-lock serialization overhead per test.
-    """
-    mem = MemoryEngine(
-        db_url=pg0_db_url,
-        memory_llm_provider="mock",
-        memory_llm_api_key="",
-        memory_llm_model="mock",
-        embeddings=embeddings,
-        cross_encoder=cross_encoder,
-        query_analyzer=query_analyzer,
-        pool_min_size=1,
-        pool_max_size=5,
-        run_migrations=False,
-        task_backend=SyncTaskBackend(),
-        skip_llm_verification=True,
-    )
-    await mem.initialize()
-    yield mem
-    try:
-        if mem._pool and not mem._pool._closing:
-            await mem.close()
-    except Exception:
-        pass
-
-
-@pytest.fixture(autouse=True)
-def enable_observations():
-    """Enable observations for all tests in this module."""
-    from hindsight_api.config import _get_raw_config
-
-    config = _get_raw_config()
-    original = config.enable_observations
-    config.enable_observations = True
-    yield
-    config.enable_observations = original
-
-
-def _make_failing_mock_llm(*, fail_first_n: int = 999) -> MockLLM:
-    """Return a MockLLM that raises ValueError for the first `fail_first_n` consolidation calls."""
-    mock_llm = MockLLM(provider="mock", api_key="", base_url="", model="mock-model")
-    call_count = 0
-
-    def callback(messages, scope):
-        nonlocal call_count
-        if scope == "consolidation":
-            call_count += 1
-            if call_count <= fail_first_n:
-                raise ValueError(f"Simulated LLM failure (call {call_count})")
-        # Return empty response — no creates/updates/deletes
-        from hindsight_api.engine.consolidation.consolidator import _ConsolidationBatchResponse
-
-        return _ConsolidationBatchResponse()
-
-    mock_llm.set_response_callback(callback)
-    return mock_llm
-
-
-def _make_always_success_mock_llm() -> MockLLM:
-    """Return a MockLLM that always succeeds with an empty consolidation response."""
-    mock_llm = MockLLM(provider="mock", api_key="", base_url="", model="mock-model")
-
-    def callback(messages, scope):
-        from hindsight_api.engine.consolidation.consolidator import _ConsolidationBatchResponse
-
-        return _ConsolidationBatchResponse()
-
-    mock_llm.set_response_callback(callback)
-    return mock_llm
-
-
-def _inject_mock_llm(memory: MemoryEngine, mock_llm: MockLLM) -> None:
-    """Replace memory._consolidation_llm_config with a wrapper that returns mock_llm from with_config."""
-    wrapper = MagicMock()
-    wrapper.with_config.return_value = mock_llm
-    memory._consolidation_llm_config = wrapper
-
-
-async def _insert_memories(conn, bank_id: str, texts: list[str]) -> list[uuid.UUID]:
-    """Insert experience memories directly, bypassing LLM-based retain."""
-    ids = []
-    for text in texts:
-        mem_id = uuid.uuid4()
-        await conn.execute(
-            """
-            INSERT INTO memory_units (id, bank_id, text, fact_type, created_at)
-            VALUES ($1, $2, $3, 'experience', now())
-            """,
-            mem_id,
-            bank_id,
-            text,
-        )
-        ids.append(mem_id)
-    return ids
-
-
-class TestAdaptiveBatchSplitting:
-    """Verify that a failing batch is halved and retried until batch_size=1 succeeds."""
-
-    @pytest.mark.asyncio
-    async def test_splitting_recovers_all_memories(self, memory_no_llm_verify: MemoryEngine, request_context):
-        """When a batch of 2 fails, both are retried individually and succeed."""
-        bank_id = f"test-split-recovery-{uuid.uuid4().hex[:8]}"
-        await memory_no_llm_verify.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            mem_ids = await _insert_memories(
-                conn,
-                bank_id,
-                [
-                    "Alice runs marathons every spring.",
-                    "Alice trained for six months for her last race.",
-                ],
-            )
-
-        # Exhaust all 3 retries for batch=2 (calls 1-3 fail), then each batch=1 succeeds (calls 4-5)
-        mock_llm = _make_failing_mock_llm(fail_first_n=3)
-        _inject_mock_llm(memory_no_llm_verify, mock_llm)
-
-        result = await run_consolidation_job(
-            memory_engine=memory_no_llm_verify,
-            bank_id=bank_id,
-            request_context=request_context,
-        )
-
-        assert result["status"] == "completed"
-        assert result["memories_processed"] == 2
-        assert result["memories_failed"] == 0
-
-        # Both memories must have consolidated_at set and consolidation_failed_at NULL
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            rows = await conn.fetch(
-                """
-                SELECT id, consolidated_at, consolidation_failed_at
-                FROM memory_units
-                WHERE bank_id = $1 AND fact_type = 'experience'
-                """,
-                bank_id,
-            )
-        assert len(rows) == 2
-        for row in rows:
-            assert row["consolidated_at"] is not None, f"Memory {row['id']} should have consolidated_at set"
-            assert row["consolidation_failed_at"] is None, (
-                f"Memory {row['id']} should NOT have consolidation_failed_at set"
-            )
-
-        # LLM called 5 times: 3 retries failed (batch=2) + 1 succeeded (batch=1) + 1 succeeded (batch=1)
-        consolidation_calls = [c for c in mock_llm.get_mock_calls() if c["scope"] == "consolidation"]
-        assert len(consolidation_calls) == 5
-
-        await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)
-
-    @pytest.mark.asyncio
-    async def test_splitting_with_larger_batch(self, memory_no_llm_verify: MemoryEngine, request_context):
-        """A batch of 4 that always fails at size>1 resolves to 4 individual calls."""
-        bank_id = f"test-split-large-{uuid.uuid4().hex[:8]}"
-        await memory_no_llm_verify.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            await _insert_memories(
-                conn,
-                bank_id,
-                [
-                    "Bob plays chess competitively.",
-                    "Bob won a regional chess tournament.",
-                    "Bob practices tactics every morning.",
-                    "Bob coaches youth chess on weekends.",
-                ],
-            )
-
-        # Exhaust all 3 retries for batch=4 (calls 1-3 fail), then both batch=2 halves succeed
-        # (calls 4-5). This verifies that halving once is sufficient when batch=2 works.
-        mock_llm = _make_failing_mock_llm(fail_first_n=3)
-        _inject_mock_llm(memory_no_llm_verify, mock_llm)
-
-        result = await run_consolidation_job(
-            memory_engine=memory_no_llm_verify,
-            bank_id=bank_id,
-            request_context=request_context,
-        )
-
-        assert result["memories_processed"] == 4
-        assert result["memories_failed"] == 0
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            rows = await conn.fetch(
-                "SELECT consolidated_at, consolidation_failed_at FROM memory_units "
-                "WHERE bank_id = $1 AND fact_type = 'experience'",
-                bank_id,
-            )
-        assert all(r["consolidated_at"] is not None for r in rows)
-        assert all(r["consolidation_failed_at"] is None for r in rows)
-
-        await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)
-
-
-class TestConsolidationFailedAt:
-    """Verify that consolidation_failed_at is set — and consolidated_at is NOT — when all retries fail."""
-
-    @pytest.mark.asyncio
-    async def test_single_memory_permanent_failure(self, memory_no_llm_verify: MemoryEngine, request_context):
-        """A single memory that exhausts all LLM retries gets consolidation_failed_at, not consolidated_at."""
-        bank_id = f"test-perm-fail-{uuid.uuid4().hex[:8]}"
-        await memory_no_llm_verify.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            (mem_id,) = await _insert_memories(conn, bank_id, ["Carol enjoys painting watercolors."])
-
-        # Always fail
-        mock_llm = _make_failing_mock_llm(fail_first_n=999)
-        _inject_mock_llm(memory_no_llm_verify, mock_llm)
-
-        result = await run_consolidation_job(
-            memory_engine=memory_no_llm_verify,
-            bank_id=bank_id,
-            request_context=request_context,
-        )
-
-        assert result["memories_failed"] == 1
-        assert result["memories_processed"] == 1
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            row = await conn.fetchrow(
-                "SELECT consolidated_at, consolidation_failed_at FROM memory_units WHERE id = $1",
-                mem_id,
-            )
-
-        assert row["consolidated_at"] is None, "consolidated_at must NOT be set for a permanently failed memory"
-        assert row["consolidation_failed_at"] is not None, "consolidation_failed_at must be set"
-
-        await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)
-
-    @pytest.mark.asyncio
-    async def test_failed_memory_excluded_from_next_run(self, memory_no_llm_verify: MemoryEngine, request_context):
-        """A memory marked consolidation_failed_at is not re-processed on the next consolidation run."""
-        bank_id = f"test-excluded-{uuid.uuid4().hex[:8]}"
-        await memory_no_llm_verify.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            (mem_id,) = await _insert_memories(conn, bank_id, ["Dave collects vinyl records."])
-            # Manually stamp consolidation_failed_at to simulate a prior failed run
-            await conn.execute(
-                "UPDATE memory_units SET consolidation_failed_at = NOW() WHERE id = $1",
-                mem_id,
-            )
-
-        # Even with a healthy LLM, the memory should be skipped
-        mock_llm = _make_always_success_mock_llm()
-        _inject_mock_llm(memory_no_llm_verify, mock_llm)
-
-        result = await run_consolidation_job(
-            memory_engine=memory_no_llm_verify,
-            bank_id=bank_id,
-            request_context=request_context,
-        )
-
-        # No unconsolidated memories to pick up (consolidation_failed_at ≠ NULL, consolidated_at = NULL
-        # but the SELECT filters on consolidated_at IS NULL AND fact_type IN ('experience','world'))
-        assert result["status"] in ("no_new_memories", "completed")
-        if result["status"] == "completed":
-            assert result["memories_processed"] == 0
-
-        # Memory still has consolidation_failed_at set and consolidated_at NULL
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            row = await conn.fetchrow(
-                "SELECT consolidated_at, consolidation_failed_at FROM memory_units WHERE id = $1",
-                mem_id,
-            )
-        assert row["consolidated_at"] is None
-        assert row["consolidation_failed_at"] is not None
-
-        await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)
-
-    @pytest.mark.asyncio
-    async def test_partial_batch_failure(self, memory_no_llm_verify: MemoryEngine, request_context):
-        """In a batch of 2, if only the first individual retry fails, the second still succeeds."""
-        bank_id = f"test-partial-fail-{uuid.uuid4().hex[:8]}"
-        await memory_no_llm_verify.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            mem_ids = await _insert_memories(
-                conn,
-                bank_id,
-                [
-                    "Eve speaks three languages fluently.",
-                    "Eve learned Japanese in two years.",
-                ],
-            )
-
-        # Exhaust 3 retries for batch=2 (calls 1-3), exhaust 3 retries for first batch=1 (calls 4-6),
-        # second batch=1 succeeds (call 7)
-        mock_llm = _make_failing_mock_llm(fail_first_n=6)
-        _inject_mock_llm(memory_no_llm_verify, mock_llm)
-
-        result = await run_consolidation_job(
-            memory_engine=memory_no_llm_verify,
-            bank_id=bank_id,
-            request_context=request_context,
-        )
-
-        assert result["memories_processed"] == 2
-        assert result["memories_failed"] == 1
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            rows = {
-                str(r["id"]): r
-                for r in await conn.fetch(
-                    "SELECT id, consolidated_at, consolidation_failed_at FROM memory_units "
-                    "WHERE bank_id = $1 AND fact_type = 'experience'",
-                    bank_id,
-                )
-            }
-
-        # One should have failed, one should have succeeded
-        failed = [r for r in rows.values() if r["consolidation_failed_at"] is not None]
-        succeeded = [r for r in rows.values() if r["consolidated_at"] is not None]
-        assert len(failed) == 1
-        assert len(succeeded) == 1
-        # They must be different memories
-        assert str(failed[0]["id"]) != str(succeeded[0]["id"])
-
-        await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)
-
-
-class TestRecoverConsolidation:
-    """Verify the retry_failed_consolidation() method and the /consolidation/recover endpoint."""
-
-    @pytest.mark.asyncio
-    async def test_recover_resets_failed_memories(self, memory_no_llm_verify: MemoryEngine, request_context):
-        """retry_failed_consolidation resets consolidation_failed_at and consolidated_at."""
-        bank_id = f"test-recover-reset-{uuid.uuid4().hex[:8]}"
-        await memory_no_llm_verify.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            ids = await _insert_memories(
-                conn,
-                bank_id,
-                [
-                    "Frank is a competitive cyclist.",
-                    "Frank completed the Tour de France route.",
-                ],
-            )
-            # Mark both as failed
-            for mem_id in ids:
-                await conn.execute(
-                    "UPDATE memory_units SET consolidation_failed_at = NOW() WHERE id = $1",
-                    mem_id,
-                )
-
-        result = await memory_no_llm_verify.retry_failed_consolidation(
-            bank_id, request_context=request_context
-        )
-
-        assert result["retried_count"] == 2
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            rows = await conn.fetch(
-                "SELECT consolidated_at, consolidation_failed_at FROM memory_units "
-                "WHERE bank_id = $1 AND fact_type = 'experience'",
-                bank_id,
-            )
-        assert all(r["consolidation_failed_at"] is None for r in rows), "consolidation_failed_at must be cleared"
-        assert all(r["consolidated_at"] is None for r in rows), "consolidated_at must also be cleared"
-
-        await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)
-
-    @pytest.mark.asyncio
-    async def test_recover_returns_zero_when_none_failed(self, memory_no_llm_verify: MemoryEngine, request_context):
-        """retry_failed_consolidation returns 0 when no memories have failed."""
-        bank_id = f"test-recover-zero-{uuid.uuid4().hex[:8]}"
-        await memory_no_llm_verify.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-        result = await memory_no_llm_verify.retry_failed_consolidation(
-            bank_id, request_context=request_context
-        )
-
-        assert result["retried_count"] == 0
-
-        await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)
-
-    @pytest.mark.asyncio
-    async def test_recover_then_consolidate_succeeds(self, memory_no_llm_verify: MemoryEngine, request_context):
-        """After recovery, the memory is picked up by the next consolidation run."""
-        bank_id = f"test-recover-consolidate-{uuid.uuid4().hex[:8]}"
-        await memory_no_llm_verify.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            (mem_id,) = await _insert_memories(conn, bank_id, ["Grace is an expert rock climber."])
-            await conn.execute(
-                "UPDATE memory_units SET consolidation_failed_at = NOW() WHERE id = $1", mem_id
-            )
-
-        # Recover
-        recover_result = await memory_no_llm_verify.retry_failed_consolidation(
-            bank_id, request_context=request_context
-        )
-        assert recover_result["retried_count"] == 1
-
-        # Now consolidate with a healthy LLM
-        mock_llm = _make_always_success_mock_llm()
-        _inject_mock_llm(memory_no_llm_verify, mock_llm)
-
-        run_result = await run_consolidation_job(
-            memory_engine=memory_no_llm_verify,
-            bank_id=bank_id,
-            request_context=request_context,
-        )
-
-        assert run_result["memories_processed"] == 1
-        assert run_result["memories_failed"] == 0
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            row = await conn.fetchrow(
-                "SELECT consolidated_at, consolidation_failed_at FROM memory_units WHERE id = $1",
-                mem_id,
-            )
-        assert row["consolidated_at"] is not None, "Memory should be consolidated after recovery"
-        assert row["consolidation_failed_at"] is None
-
-        await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)
-
-    @pytest.mark.asyncio
-    async def test_recover_endpoint_via_http(self, memory_no_llm_verify: MemoryEngine, request_context):
-        """The POST /consolidation/recover endpoint returns the correct retried_count."""
-        import httpx
-
-        from hindsight_api.api.http import create_app
-
-        bank_id = f"test-recover-http-{uuid.uuid4().hex[:8]}"
-        await memory_no_llm_verify.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            ids = await _insert_memories(
-                conn,
-                bank_id,
-                ["Henry is a professional chef.", "Henry trained at Le Cordon Bleu."],
-            )
-            for mem_id in ids:
-                await conn.execute(
-                    "UPDATE memory_units SET consolidation_failed_at = NOW() WHERE id = $1", mem_id
-                )
-
-        app = create_app(memory_no_llm_verify, initialize_memory=False)
-        transport = httpx.ASGITransport(app=app)
-        async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
-            response = await client.post(f"/v1/default/banks/{bank_id}/consolidation/recover")
-
-        assert response.status_code == 200
-        body = response.json()
-        assert body["retried_count"] == 2
-
-        await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)

hindsight-api-slim/tests/test_delta_retain.py

@@ -1,842 +0,0 @@
-"""
-Tests for delta retain — upsert optimization that only re-processes changed chunks.
-"""
-
-import logging
-from datetime import datetime, timezone
-
-import pytest
-
-from hindsight_api import RequestContext
-from hindsight_api.engine.memory_engine import Budget
-
-logger = logging.getLogger(__name__)
-
-
-def _ts():
-    return datetime.now(timezone.utc).timestamp()
-
-
-# ============================================================
-# Core Delta Retain Tests
-# ============================================================
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_unchanged_content_skips_llm(memory, request_context):
-    """
-    When upserting a document with identical content, no new facts should be
-    extracted (LLM is not called for unchanged chunks). The existing facts
-    should be preserved.
-    """
-    bank_id = f"test_delta_unchanged_{_ts()}"
-    document_id = "conversation-001"
-
-    try:
-        content = "Alice works at Google. Bob works at Microsoft."
-
-        # First retain — full processing
-        v1_units = await memory.retain_async(
-            bank_id=bank_id,
-            content=content,
-            context="team info",
-            document_id=document_id,
-            request_context=request_context,
-        )
-        assert len(v1_units) > 0, "v1 should create facts"
-
-        # Get v1 document state
-        doc_v1 = await memory.get_document(document_id, bank_id, request_context=request_context)
-        v1_unit_count = doc_v1["memory_unit_count"]
-
-        # Second retain — same content, should use delta path (no new facts)
-        v2_units = await memory.retain_async(
-            bank_id=bank_id,
-            content=content,
-            context="team info",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        # No new units should be returned (nothing changed)
-        assert v2_units == [], "Delta retain with unchanged content should return empty unit list"
-
-        # Existing facts should still be there
-        doc_v2 = await memory.get_document(document_id, bank_id, request_context=request_context)
-        assert doc_v2["memory_unit_count"] == v1_unit_count, "Existing facts should be preserved"
-
-        # Verify recall still works
-        result = await memory.recall_async(
-            bank_id=bank_id,
-            query="Where does Alice work?",
-            budget=Budget.MID,
-            max_tokens=1000,
-            request_context=request_context,
-        )
-        assert len(result.results) > 0, "Should still recall facts after delta retain"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_appended_content(memory, request_context):
-    """
-    When a conversation grows (new content appended), only new chunks should
-    be processed. Facts from unchanged chunks should be preserved.
-    """
-    bank_id = f"test_delta_append_{_ts()}"
-    document_id = "growing-conversation"
-
-    try:
-        # First version — short content (single chunk)
-        v1_content = "Alice is a software engineer at Google. She works on search infrastructure."
-
-        v1_units = await memory.retain_async(
-            bank_id=bank_id,
-            content=v1_content,
-            context="profile",
-            document_id=document_id,
-            request_context=request_context,
-        )
-        assert len(v1_units) > 0
-
-        # Get v1 facts via recall
-        v1_recall = await memory.recall_async(
-            bank_id=bank_id,
-            query="What does Alice do?",
-            budget=Budget.MID,
-            max_tokens=2000,
-            request_context=request_context,
-        )
-        v1_fact_texts = {r.text for r in v1_recall.results}
-
-        # Second version — original content + new content appended
-        # This should preserve facts from the first chunk and add new ones
-        v2_content = v1_content + "\n\nBob joined Google as a product manager in 2024. He previously worked at Meta on AR/VR products."
-
-        v2_units = await memory.retain_async(
-            bank_id=bank_id,
-            content=v2_content,
-            context="profile",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        # Should have facts about Bob from the new content
-        v2_recall = await memory.recall_async(
-            bank_id=bank_id,
-            query="What does Bob do?",
-            budget=Budget.MID,
-            max_tokens=2000,
-            request_context=request_context,
-        )
-        bob_facts = [r for r in v2_recall.results if "bob" in r.text.lower()]
-        assert len(bob_facts) > 0, "Should have facts about Bob from appended content"
-
-        # Should still have facts about Alice from original content
-        alice_recall = await memory.recall_async(
-            bank_id=bank_id,
-            query="What does Alice do?",
-            budget=Budget.MID,
-            max_tokens=2000,
-            request_context=request_context,
-        )
-        assert len(alice_recall.results) > 0, "Should still have Alice facts from original content"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_modified_chunk(memory, request_context):
-    """
-    When content in the middle changes, that chunk should be re-processed
-    while other chunks are preserved.
-    """
-    bank_id = f"test_delta_modified_{_ts()}"
-    document_id = "changing-doc"
-
-    try:
-        # v1: Alice works at Google
-        v1_content = "Alice works at Google as a senior engineer."
-        v1_units = await memory.retain_async(
-            bank_id=bank_id,
-            content=v1_content,
-            context="team",
-            document_id=document_id,
-            request_context=request_context,
-        )
-        assert len(v1_units) > 0
-
-        # v2: Alice works at Microsoft (changed)
-        v2_content = "Alice works at Microsoft as a principal engineer."
-        v2_units = await memory.retain_async(
-            bank_id=bank_id,
-            content=v2_content,
-            context="team",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        # New facts should reflect the updated content
-        result = await memory.recall_async(
-            bank_id=bank_id,
-            query="Where does Alice work?",
-            budget=Budget.MID,
-            max_tokens=2000,
-            request_context=request_context,
-        )
-        all_texts = " ".join(r.text.lower() for r in result.results)
-        assert "microsoft" in all_texts, f"Should have updated fact about Microsoft, got: {all_texts}"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-# ============================================================
-# Entity & Link Tests
-# ============================================================
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_entities_preserved_for_unchanged_chunks(memory, request_context):
-    """
-    Entities linked to unchanged chunks should be preserved after delta retain.
-    """
-    bank_id = f"test_delta_entities_{_ts()}"
-    document_id = "entity-doc"
-
-    try:
-        v1_content = "Alice works at Google. She is a senior engineer in the Cloud division."
-        v1_units = await memory.retain_async(
-            bank_id=bank_id,
-            content=v1_content,
-            context="team",
-            document_id=document_id,
-            request_context=request_context,
-        )
-        assert len(v1_units) > 0
-
-        # Check entities exist
-        pool = await memory._get_pool()
-        async with pool.acquire() as conn:
-            v1_entities = await conn.fetch(
-                "SELECT canonical_name FROM entities WHERE bank_id = $1",
-                bank_id,
-            )
-        v1_entity_names = {e["canonical_name"].lower() for e in v1_entities}
-        assert len(v1_entity_names) > 0, "Should have entities after v1 retain"
-
-        # Upsert with same content — entities should persist
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=v1_content,
-            context="team",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        async with pool.acquire() as conn:
-            v2_entities = await conn.fetch(
-                "SELECT canonical_name FROM entities WHERE bank_id = $1",
-                bank_id,
-            )
-        v2_entity_names = {e["canonical_name"].lower() for e in v2_entities}
-
-        # All v1 entities should still exist
-        assert v1_entity_names.issubset(v2_entity_names), (
-            f"v1 entities {v1_entity_names} should be preserved, got {v2_entity_names}"
-        )
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_new_entities_created_for_new_chunks(memory, request_context):
-    """
-    New entities should be created for newly added chunks during delta retain.
-    """
-    bank_id = f"test_delta_new_entities_{_ts()}"
-    document_id = "entity-growth-doc"
-
-    try:
-        v1_content = "Alice works at Google."
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=v1_content,
-            context="team",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        pool = await memory._get_pool()
-        async with pool.acquire() as conn:
-            v1_entities = await conn.fetch(
-                "SELECT canonical_name FROM entities WHERE bank_id = $1",
-                bank_id,
-            )
-        v1_entity_names = {e["canonical_name"].lower() for e in v1_entities}
-
-        # Append content mentioning new entities
-        v2_content = v1_content + "\n\nBob joined Facebook. He works with Charlie on the Reality Labs project."
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=v2_content,
-            context="team",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        async with pool.acquire() as conn:
-            v2_entities = await conn.fetch(
-                "SELECT canonical_name FROM entities WHERE bank_id = $1",
-                bank_id,
-            )
-        v2_entity_names = {e["canonical_name"].lower() for e in v2_entities}
-
-        # Should have more entities after adding content with new people/orgs
-        assert len(v2_entity_names) > len(v1_entity_names), (
-            f"Should have more entities after append: v1={v1_entity_names}, v2={v2_entity_names}"
-        )
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_links_preserved_for_unchanged_chunks(memory, request_context):
-    """
-    Memory links (temporal, semantic, entity) for unchanged chunks should be preserved.
-    """
-    bank_id = f"test_delta_links_{_ts()}"
-    document_id = "links-doc"
-
-    try:
-        content = "Alice is a senior engineer at Google Cloud. She mentors junior engineers and reviews their code."
-        v1_units = await memory.retain_async(
-            bank_id=bank_id,
-            content=content,
-            context="team",
-            document_id=document_id,
-            request_context=request_context,
-        )
-        assert len(v1_units) > 0
-
-        # Count links after v1
-        pool = await memory._get_pool()
-        async with pool.acquire() as conn:
-            v1_link_count = await conn.fetchval(
-                """SELECT COUNT(*) FROM memory_links ml
-                   JOIN memory_units mu ON ml.from_unit_id = mu.id
-                   WHERE mu.bank_id = $1 AND mu.document_id = $2""",
-                bank_id,
-                document_id,
-            )
-
-        # Upsert with same content
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=content,
-            context="team",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        # Links should be preserved
-        async with pool.acquire() as conn:
-            v2_link_count = await conn.fetchval(
-                """SELECT COUNT(*) FROM memory_links ml
-                   JOIN memory_units mu ON ml.from_unit_id = mu.id
-                   WHERE mu.bank_id = $1 AND mu.document_id = $2""",
-                bank_id,
-                document_id,
-            )
-
-        assert v2_link_count == v1_link_count, (
-            f"Links should be preserved: v1={v1_link_count}, v2={v2_link_count}"
-        )
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-# ============================================================
-# Document Metadata & Tags Tests
-# ============================================================
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_document_metadata_updated(memory, request_context):
-    """
-    Document metadata (retain_params, tags) should be updated even when
-    chunk content hasn't changed.
-    """
-    bank_id = f"test_delta_meta_{_ts()}"
-    document_id = "metadata-doc"
-
-    try:
-        content = "Alice works at Google."
-
-        # v1 with initial tags
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=content,
-            context="initial context",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        doc_v1 = await memory.get_document(document_id, bank_id, request_context=request_context)
-        assert doc_v1 is not None
-
-        # v2 with updated context (same content — triggers delta path)
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=content,
-            context="updated context",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        doc_v2 = await memory.get_document(document_id, bank_id, request_context=request_context)
-        assert doc_v2 is not None
-        assert doc_v2["updated_at"] >= doc_v1["updated_at"], "Document should have updated timestamp"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_tags_propagated_to_existing_units(memory, request_context):
-    """
-    When tags change during an upsert with unchanged content, the new tags
-    should be propagated to all existing memory units.
-    """
-    bank_id = f"test_delta_tags_{_ts()}"
-    document_id = "tags-doc"
-
-    try:
-        content = "Alice works at Google."
-
-        # v1 with tag "team-a"
-        await memory.retain_batch_async(
-            bank_id=bank_id,
-            contents=[{
-                "content": content,
-                "document_id": document_id,
-                "tags": ["team-a"],
-            }],
-            request_context=request_context,
-        )
-
-        pool = await memory._get_pool()
-        async with pool.acquire() as conn:
-            v1_tags = await conn.fetch(
-                "SELECT tags FROM memory_units WHERE bank_id = $1 AND document_id = $2",
-                bank_id,
-                document_id,
-            )
-        assert all("team-a" in row["tags"] for row in v1_tags), "v1 units should have team-a tag"
-
-        # v2 with same content but different tags
-        await memory.retain_batch_async(
-            bank_id=bank_id,
-            contents=[{
-                "content": content,
-                "document_id": document_id,
-                "tags": ["team-b", "important"],
-            }],
-            request_context=request_context,
-        )
-
-        async with pool.acquire() as conn:
-            v2_tags = await conn.fetch(
-                "SELECT tags FROM memory_units WHERE bank_id = $1 AND document_id = $2",
-                bank_id,
-                document_id,
-            )
-        for row in v2_tags:
-            assert "team-b" in row["tags"], f"v2 units should have team-b tag, got {row['tags']}"
-            assert "important" in row["tags"], f"v2 units should have important tag, got {row['tags']}"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-# ============================================================
-# Chunk Management Tests
-# ============================================================
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_removed_chunks_delete_facts(memory, request_context):
-    """
-    When content is shortened (chunks removed), facts from the removed
-    chunks should be deleted.
-    """
-    bank_id = f"test_delta_removed_{_ts()}"
-    document_id = "shrinking-doc"
-
-    try:
-        # v1: longer content with facts about Alice and Bob
-        v1_content = (
-            "Alice is a senior engineer at Google Cloud. "
-            "She leads the infrastructure team and has been there for 5 years.\n\n"
-            "Bob is a product manager at Facebook Reality Labs. "
-            "He previously worked at Amazon on Alexa voice products."
-        )
-
-        v1_units = await memory.retain_async(
-            bank_id=bank_id,
-            content=v1_content,
-            context="profiles",
-            document_id=document_id,
-            request_context=request_context,
-        )
-        assert len(v1_units) > 0
-
-        doc_v1 = await memory.get_document(document_id, bank_id, request_context=request_context)
-        v1_count = doc_v1["memory_unit_count"]
-
-        # v2: Completely different content — all chunks change
-        v2_content = "Charlie works at Netflix as a data scientist."
-        v2_units = await memory.retain_async(
-            bank_id=bank_id,
-            content=v2_content,
-            context="profiles",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        doc_v2 = await memory.get_document(document_id, bank_id, request_context=request_context)
-        assert doc_v2 is not None
-
-        # Should have facts about Charlie
-        result = await memory.recall_async(
-            bank_id=bank_id,
-            query="Who works at Netflix?",
-            budget=Budget.MID,
-            max_tokens=2000,
-            request_context=request_context,
-        )
-        all_texts = " ".join(r.text.lower() for r in result.results)
-        assert "charlie" in all_texts or "netflix" in all_texts, (
-            f"Should have facts about Charlie/Netflix after replacing content, got: {all_texts}"
-        )
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_chunks_have_content_hash(memory, request_context):
-    """
-    After retain, chunks should have content_hash populated.
-    """
-    bank_id = f"test_delta_hash_{_ts()}"
-    document_id = "hash-doc"
-
-    try:
-        content = "Alice works at Google as a software engineer."
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=content,
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        pool = await memory._get_pool()
-        async with pool.acquire() as conn:
-            chunks = await conn.fetch(
-                "SELECT chunk_id, content_hash FROM chunks WHERE document_id = $1 AND bank_id = $2",
-                document_id,
-                bank_id,
-            )
-
-        assert len(chunks) > 0, "Should have stored chunks"
-        for chunk in chunks:
-            assert chunk["content_hash"] is not None, f"Chunk {chunk['chunk_id']} should have content_hash"
-            assert len(chunk["content_hash"]) == 64, "content_hash should be SHA256 hex (64 chars)"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-# ============================================================
-# Backward Compatibility Tests
-# ============================================================
-
-
-@pytest.mark.asyncio
-async def test_retain_without_document_id_still_works(memory, request_context):
-    """
-    Retain without document_id should still work normally (no delta path).
-    """
-    bank_id = f"test_no_docid_{_ts()}"
-
-    try:
-        units = await memory.retain_async(
-            bank_id=bank_id,
-            content="Alice works at Google.",
-            context="test",
-            request_context=request_context,
-        )
-        assert len(units) > 0, "Should create facts without document_id"
-
-        result = await memory.recall_async(
-            bank_id=bank_id,
-            query="Where does Alice work?",
-            budget=Budget.MID,
-            max_tokens=1000,
-            request_context=request_context,
-        )
-        assert len(result.results) > 0
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_first_retain_full_path(memory, request_context):
-    """
-    First retain of a new document should use the full path (no delta possible).
-    """
-    bank_id = f"test_first_retain_{_ts()}"
-    document_id = "new-doc"
-
-    try:
-        units = await memory.retain_async(
-            bank_id=bank_id,
-            content="Alice works at Google.",
-            context="test",
-            document_id=document_id,
-            request_context=request_context,
-        )
-        assert len(units) > 0, "First retain should create facts via full path"
-
-        doc = await memory.get_document(document_id, bank_id, request_context=request_context)
-        assert doc is not None
-        assert doc["memory_unit_count"] > 0
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-# ============================================================
-# Edge Cases
-# ============================================================
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_empty_to_content(memory, request_context):
-    """
-    Going from gibberish (zero facts) to real content should work.
-    """
-    bank_id = f"test_delta_empty_{_ts()}"
-    document_id = "empty-to-content"
-
-    try:
-        # v1: content that probably produces zero facts
-        await memory.retain_async(
-            bank_id=bank_id,
-            content="!!!###$$$%%%",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        doc_v1 = await memory.get_document(document_id, bank_id, request_context=request_context)
-        assert doc_v1 is not None
-
-        # v2: real content
-        v2_units = await memory.retain_async(
-            bank_id=bank_id,
-            content="Alice works at Google as a senior engineer.",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        doc_v2 = await memory.get_document(document_id, bank_id, request_context=request_context)
-        assert doc_v2 is not None
-        assert doc_v2["memory_unit_count"] > 0 or len(v2_units) > 0, "Should have facts after updating with real content"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_multiple_upserts(memory, request_context):
-    """
-    Multiple sequential upserts should work correctly, with delta optimization
-    kicking in after the first retain.
-    """
-    bank_id = f"test_delta_multi_{_ts()}"
-    document_id = "multi-upsert"
-
-    try:
-        # v1: initial
-        v1_content = "Alice works at Google."
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=v1_content,
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        # v2: same content (delta: no changes)
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=v1_content,
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        # v3: append
-        v3_content = v1_content + "\n\nBob works at Microsoft."
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=v3_content,
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        # v4: same as v3 (delta: no changes again)
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=v3_content,
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        # Final check: should have facts about both Alice and Bob
-        result = await memory.recall_async(
-            bank_id=bank_id,
-            query="Who works where?",
-            budget=Budget.MID,
-            max_tokens=2000,
-            request_context=request_context,
-        )
-        all_texts = " ".join(r.text.lower() for r in result.results)
-        assert "alice" in all_texts or "google" in all_texts, f"Should have Alice/Google facts, got: {all_texts}"
-
-        doc = await memory.get_document(document_id, bank_id, request_context=request_context)
-        assert doc is not None
-        assert doc["memory_unit_count"] > 0
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_with_user_entities(memory, request_context):
-    """
-    User-provided entities should work correctly with delta retain.
-    """
-    bank_id = f"test_delta_user_entities_{_ts()}"
-    document_id = "user-entity-doc"
-
-    try:
-        content = "The project is going well."
-
-        # v1 with user entities
-        await memory.retain_batch_async(
-            bank_id=bank_id,
-            contents=[{
-                "content": content,
-                "document_id": document_id,
-                "entities": [{"text": "Project Alpha", "type": "PROJECT"}],
-            }],
-            request_context=request_context,
-        )
-
-        pool = await memory._get_pool()
-        async with pool.acquire() as conn:
-            v1_entities = await conn.fetch(
-                "SELECT canonical_name FROM entities WHERE bank_id = $1",
-                bank_id,
-            )
-        v1_names = {e["canonical_name"].lower() for e in v1_entities}
-
-        # v2 with additional entity, same content
-        # Note: same content = delta path (no re-extraction)
-        # The user entities for NEW chunks only get processed
-        v2_content = content + "\n\nThe timeline is on track for Q2 delivery."
-        await memory.retain_batch_async(
-            bank_id=bank_id,
-            contents=[{
-                "content": v2_content,
-                "document_id": document_id,
-                "entities": [
-                    {"text": "Project Alpha", "type": "PROJECT"},
-                    {"text": "Q2 Deadline", "type": "MILESTONE"},
-                ],
-            }],
-            request_context=request_context,
-        )
-
-        # Should have entities from both v1 and v2
-        async with pool.acquire() as conn:
-            v2_entities = await conn.fetch(
-                "SELECT canonical_name FROM entities WHERE bank_id = $1",
-                bank_id,
-            )
-        v2_names = {e["canonical_name"].lower() for e in v2_entities}
-
-        # v1 entities should be preserved
-        assert v1_names.issubset(v2_names), f"v1 entities should be preserved: {v1_names} not in {v2_names}"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_recall_with_chunks(memory, request_context):
-    """
-    After delta retain, recall with include_chunks should return correct chunk data.
-    """
-    bank_id = f"test_delta_recall_chunks_{_ts()}"
-    document_id = "recall-chunks-doc"
-
-    try:
-        content = "Alice is a senior engineer at Google Cloud. She designs distributed systems."
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=content,
-            context="profile",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        # Upsert with same content (delta: no changes)
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=content,
-            context="profile",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        # Recall with chunks
-        result = await memory.recall_async(
-            bank_id=bank_id,
-            query="What does Alice do?",
-            budget=Budget.MID,
-            max_tokens=2000,
-            include_chunks=True,
-            max_chunk_tokens=8192,
-            request_context=request_context,
-        )
-
-        assert len(result.results) > 0, "Should recall facts"
-
-        # Facts with chunk_ids should have corresponding chunks
-        facts_with_chunks = [r for r in result.results if r.chunk_id]
-        if facts_with_chunks and result.chunks:
-            for fact in facts_with_chunks:
-                assert fact.chunk_id in result.chunks, (
-                    f"Chunk {fact.chunk_id} should be in returned chunks"
-                )
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)

hindsight-api-slim/tests/test_entity_resolver.py

@@ -1,114 +0,0 @@
-"""
-Tests for EntityResolver edge cases.
-"""
-
-import uuid
-from datetime import datetime, timezone
-
-import asyncpg
-import pytest
-
-from hindsight_api.engine.entity_resolver import EntityResolver
-from hindsight_api.pg0 import resolve_database_url
-
-# ---------------------------------------------------------------------------
-# Unit tests for discard_pending_stats() — no database required
-# ---------------------------------------------------------------------------
-
-
-@pytest.mark.asyncio
-async def test_discard_pending_stats_clears_both_dicts():
-    """discard_pending_stats() must remove entries for the current task from
-    both _pending_stats and _pending_cooccurrences."""
-    resolver = EntityResolver(pool=None)  # type: ignore[arg-type]
-    key = resolver._task_key()
-
-    resolver._pending_stats[key] = [object()]  # type: ignore[list-item]
-    resolver._pending_cooccurrences[key] = [object()]  # type: ignore[list-item]
-
-    resolver.discard_pending_stats()
-
-    assert key not in resolver._pending_stats
-    assert key not in resolver._pending_cooccurrences
-
-
-@pytest.mark.asyncio
-async def test_discard_pending_stats_is_idempotent():
-    """Calling discard_pending_stats() when nothing is pending must not raise."""
-    resolver = EntityResolver(pool=None)  # type: ignore[arg-type]
-    resolver.discard_pending_stats()
-    resolver.discard_pending_stats()  # second call — still safe
-
-
-@pytest.mark.asyncio
-async def test_discard_pending_stats_does_not_affect_other_task_keys():
-    """discard_pending_stats() must only remove the current task's entries,
-    leaving entries keyed under other task IDs untouched."""
-    resolver = EntityResolver(pool=None)  # type: ignore[arg-type]
-    other_key = -1  # A fake key that can never be a real task id
-
-    resolver._pending_stats[other_key] = [object()]  # type: ignore[list-item]
-    resolver._pending_cooccurrences[other_key] = [object()]  # type: ignore[list-item]
-
-    resolver.discard_pending_stats()  # discards current task's key only
-
-    assert other_key in resolver._pending_stats, "other task's stats must be preserved"
-    assert other_key in resolver._pending_cooccurrences, "other task's cooccurrences must be preserved"
-
-
-@pytest.mark.asyncio
-async def test_resolve_entities_batch_handles_unicode_lower_conflicts(pg0_db_url):
-    """
-    Existing entities with PostgreSQL/Python lowercase mismatches should resolve
-    to the conflicted row instead of leaving a missing entity_id.
-    """
-    resolved_url = await resolve_database_url(pg0_db_url)
-    pool = await asyncpg.create_pool(resolved_url, min_size=1, max_size=2, command_timeout=30)
-    bank_id = f"test-entity-resolver-{uuid.uuid4().hex[:8]}"
-    event_date = datetime(2024, 1, 15, tzinfo=timezone.utc)
-    resolver = EntityResolver(pool=pool, entity_lookup="full")
-
-    try:
-        async with pool.acquire() as conn:
-            existing_entity_id = await conn.fetchval(
-                """
-                INSERT INTO entities (bank_id, canonical_name, first_seen, last_seen, mention_count)
-                VALUES ($1, $2, $3, $3, 1)
-                RETURNING id
-                """,
-                bank_id,
-                "İstanbul",
-                event_date,
-            )
-
-            resolved_ids = await resolver.resolve_entities_batch(
-                bank_id=bank_id,
-                entities_data=[
-                    {
-                        "text": "istanbul",
-                        "nearby_entities": [],
-                        "event_date": event_date,
-                    }
-                ],
-                context="unicode case mismatch",
-                unit_event_date=event_date,
-                conn=conn,
-            )
-
-            entity_rows = await conn.fetch(
-                """
-                SELECT id, canonical_name
-                FROM entities
-                WHERE bank_id = $1
-                ORDER BY canonical_name
-                """,
-                bank_id,
-            )
-
-        assert resolved_ids == [existing_entity_id]
-        assert len(entity_rows) == 1
-        assert entity_rows[0]["id"] == existing_entity_id
-        assert entity_rows[0]["canonical_name"] == "İstanbul"
-    finally:
-        await pool.execute("DELETE FROM entities WHERE bank_id = $1", bank_id)
-        await pool.close()

hindsight-api-slim/tests/test_entity_resolver_pg_trgm.py

@@ -1,163 +0,0 @@
-"""
-Unit tests for EntityResolver pg_trgm auto-detection (PR #626/#649).
-
-These tests verify:
-1. When entity_lookup="trigram" and pg_trgm IS available, the trigram path is used.
-2. When entity_lookup="trigram" and pg_trgm is NOT available, the resolver falls back
-   to entity_lookup="full" and uses the full-scan path.
-3. The pg_trgm check is only performed once (_pg_trgm_checked flag prevents re-checking).
-4. When entity_lookup="full" from the start, the trgm check is never performed.
-"""
-
-from unittest.mock import AsyncMock, MagicMock, patch
-
-import pytest
-
-from hindsight_api.engine.entity_resolver import EntityResolver
-
-
-def _make_conn(pg_trgm_available: bool) -> MagicMock:
-    """Create a minimal mock asyncpg connection for the pg_trgm availability check."""
-    conn = MagicMock()
-    conn.fetchval = AsyncMock(return_value=pg_trgm_available)
-    conn.fetch = AsyncMock(return_value=[])
-    conn.executemany = AsyncMock()
-    conn.fetchrow = AsyncMock(return_value=None)
-    return conn
-
-
-def _make_resolver(entity_lookup: str = "trigram") -> EntityResolver:
-    """Return an EntityResolver with a None pool (not needed for unit tests)."""
-    return EntityResolver(pool=None, entity_lookup=entity_lookup)  # type: ignore[arg-type]
-
-
-class TestPgTrgmAutoDetection:
-    """Unit tests for pg_trgm detection logic inside _resolve_entities_batch_impl."""
-
-    @pytest.mark.asyncio
-    async def test_falls_back_to_full_when_pg_trgm_unavailable(self):
-        """When pg_trgm is absent the resolver switches to 'full' and calls the full-scan path."""
-        resolver = _make_resolver(entity_lookup="trigram")
-        conn = _make_conn(pg_trgm_available=False)
-
-        with (
-            patch.object(resolver, "_resolve_entities_batch_full", new=AsyncMock(return_value=[])) as mock_full,
-            patch.object(resolver, "_resolve_entities_batch_trigram", new=AsyncMock(return_value=[])) as mock_trgm,
-        ):
-            await resolver._resolve_entities_batch_impl(
-                conn=conn,
-                bank_id="test-bank",
-                entities_data=[],
-                context="",
-                unit_event_date=None,
-            )
-
-        # Trigram path must NOT be called
-        mock_trgm.assert_not_called()
-        # Full-scan path must be called as the fallback
-        mock_full.assert_called_once()
-        # Strategy is permanently downgraded
-        assert resolver.entity_lookup == "full"
-        assert resolver._pg_trgm_checked is True
-
-    @pytest.mark.asyncio
-    async def test_uses_trigram_when_pg_trgm_available(self):
-        """When pg_trgm is present the trigram path is used."""
-        resolver = _make_resolver(entity_lookup="trigram")
-        conn = _make_conn(pg_trgm_available=True)
-
-        with (
-            patch.object(resolver, "_resolve_entities_batch_full", new=AsyncMock(return_value=[])) as mock_full,
-            patch.object(resolver, "_resolve_entities_batch_trigram", new=AsyncMock(return_value=[])) as mock_trgm,
-        ):
-            await resolver._resolve_entities_batch_impl(
-                conn=conn,
-                bank_id="test-bank",
-                entities_data=[],
-                context="",
-                unit_event_date=None,
-            )
-
-        mock_trgm.assert_called_once()
-        mock_full.assert_not_called()
-        assert resolver.entity_lookup == "trigram"
-        assert resolver._pg_trgm_checked is True
-
-    @pytest.mark.asyncio
-    async def test_pg_trgm_check_performed_only_once(self):
-        """The fetchval check is only issued on the first call; subsequent calls skip it."""
-        resolver = _make_resolver(entity_lookup="trigram")
-        conn = _make_conn(pg_trgm_available=True)
-
-        with patch.object(resolver, "_resolve_entities_batch_trigram", new=AsyncMock(return_value=[])):
-            # First call — check is issued
-            await resolver._resolve_entities_batch_impl(
-                conn=conn,
-                bank_id="test-bank",
-                entities_data=[],
-                context="",
-                unit_event_date=None,
-            )
-            # Second call — check must NOT be issued again
-            await resolver._resolve_entities_batch_impl(
-                conn=conn,
-                bank_id="test-bank",
-                entities_data=[],
-                context="",
-                unit_event_date=None,
-            )
-
-        # fetchval (the pg_trgm availability query) should be called exactly once
-        assert conn.fetchval.call_count == 1
-
-    @pytest.mark.asyncio
-    async def test_full_strategy_skips_pg_trgm_check(self):
-        """When entity_lookup='full' from the start, no pg_trgm check is ever issued."""
-        resolver = _make_resolver(entity_lookup="full")
-        conn = _make_conn(pg_trgm_available=False)
-
-        with patch.object(resolver, "_resolve_entities_batch_full", new=AsyncMock(return_value=[])):
-            await resolver._resolve_entities_batch_impl(
-                conn=conn,
-                bank_id="test-bank",
-                entities_data=[],
-                context="",
-                unit_event_date=None,
-            )
-
-        # fetchval should never be called when entity_lookup is already "full"
-        conn.fetchval.assert_not_called()
-
-    @pytest.mark.asyncio
-    async def test_fallback_is_sticky_across_calls(self):
-        """After falling back to 'full', subsequent calls also use the full path."""
-        resolver = _make_resolver(entity_lookup="trigram")
-        conn = _make_conn(pg_trgm_available=False)
-
-        with (
-            patch.object(resolver, "_resolve_entities_batch_full", new=AsyncMock(return_value=[])) as mock_full,
-            patch.object(resolver, "_resolve_entities_batch_trigram", new=AsyncMock(return_value=[])) as mock_trgm,
-        ):
-            # First call triggers the fallback
-            await resolver._resolve_entities_batch_impl(
-                conn=conn,
-                bank_id="b",
-                entities_data=[],
-                context="",
-                unit_event_date=None,
-            )
-            # Second call — _pg_trgm_checked is True so no re-check; entity_lookup=="full"
-            await resolver._resolve_entities_batch_impl(
-                conn=conn,
-                bank_id="b",
-                entities_data=[],
-                context="",
-                unit_event_date=None,
-            )
-
-        # Trigram path is never called
-        mock_trgm.assert_not_called()
-        # Full-scan path is called both times
-        assert mock_full.call_count == 2
-        # pg_trgm check was issued exactly once
-        assert conn.fetchval.call_count == 1

hindsight-api-slim/tests/test_experience_fact_type_passthrough.py

@@ -1,59 +0,0 @@
-"""
-Regression test for experience fact_type preservation.
-
-The LLM extraction layer normalizes raw "assistant" → "experience" early in parsing.
-The subsequent conversion to ExtractedFactType must pass through the already-normalized
-fact_type rather than re-checking for "assistant" (which would remap experience → world).
-
-See: https://github.com/vectorize-io/hindsight/pull/839
-"""
-
-from datetime import datetime, timezone
-from unittest.mock import AsyncMock, patch
-
-import pytest
-
-from hindsight_api.config import _get_raw_config
-from hindsight_api.engine.response_models import TokenUsage
-from hindsight_api.engine.retain.fact_extraction import (
-    Fact,
-    RetainContent,
-    extract_facts_from_contents,
-    extract_facts_from_contents_batch_api,
-)
-
-
-@pytest.mark.asyncio
-async def test_extract_facts_preserves_experience_type():
-    """
-    When extract_facts_from_text returns a Fact with fact_type="experience",
-    extract_facts_from_contents must preserve it (not remap to "world").
-    """
-    contents = [
-        RetainContent(
-            content="I fixed the failing tests after discovering they mocked the wrong interface.",
-            event_date=datetime(2026, 4, 1, tzinfo=timezone.utc),
-            context="assistant work log",
-        )
-    ]
-    extracted_fact = Fact(
-        fact="Fixed the failing tests after discovering they mocked the wrong interface.",
-        fact_type="experience",
-    )
-
-    with patch(
-        "hindsight_api.engine.retain.fact_extraction.extract_facts_from_text",
-        new=AsyncMock(return_value=([extracted_fact], [(contents[0].content, 1)], TokenUsage())),
-    ):
-        facts, _chunks, _usage = await extract_facts_from_contents(
-            contents=contents,
-            llm_config=None,
-            agent_name="TestAgent",
-            config=_get_raw_config(),
-        )
-
-    assert len(facts) == 1
-    assert facts[0].fact_type == "experience", (
-        f"Expected 'experience' but got '{facts[0].fact_type}' — "
-        f"the conversion layer is remapping the already-normalized fact_type"
-    )

hindsight-api-slim/tests/test_fact_extraction_agent_experience.py

@@ -1,131 +0,0 @@
-"""
-Test that first-person agent experiences are classified as 'experience' fact_type,
-not 'world'. This is critical for AI agent systems that store their own operational
-experiences (debugging, code changes, user interactions) separately from world knowledge.
-"""
-from datetime import datetime
-
-import pytest
-
-from hindsight_api import LLMConfig
-from hindsight_api.config import _get_raw_config
-from hindsight_api.engine.retain.fact_extraction import extract_facts_from_text
-
-
-class TestAgentExperienceClassification:
-    """Tests that first-person coding agent experiences get classified as 'experience'."""
-
-    @pytest.mark.asyncio
-    async def test_code_changes_classified_as_experience(self):
-        """First-person code change descriptions should be experience, not world."""
-        text = """
-I changed the return type of the `process_request` function from `dict` to `ResponseModel`.
-After that, I updated the three callers in `api/handlers.py` to destructure the new model fields.
-The type checker was happy after the change but I noticed one test was still using the old dict keys.
-"""
-        llm_config = LLMConfig.from_env()
-        facts, _, _ = await extract_facts_from_text(
-            text=text,
-            event_date=datetime(2025, 3, 28),
-            llm_config=llm_config,
-            agent_name="coding-agent",
-            context="agent work log",
-            config=_get_raw_config(),
-        )
-
-        assert len(facts) > 0, "Should extract at least one fact"
-        world_facts = [f for f in facts if f.fact_type == "world"]
-        experience_facts = [f for f in facts if f.fact_type == "experience"]
-        assert len(experience_facts) > len(world_facts), (
-            f"First-person code changes should be mostly 'experience', "
-            f"got {len(experience_facts)} experience vs {len(world_facts)} world. "
-            f"Facts: {[(f.fact, f.fact_type) for f in facts]}"
-        )
-
-    @pytest.mark.asyncio
-    async def test_debugging_session_classified_as_experience(self):
-        """First-person debugging narrative should be experience, not world."""
-        text = """
-The tests were failing with a ConnectionRefusedError on the Redis integration suite.
-I traced it to the connection pool not being initialized before the first test ran.
-I added a setup fixture that ensures the pool is warmed up, and all 47 tests pass now.
-"""
-        llm_config = LLMConfig.from_env()
-        facts, _, _ = await extract_facts_from_text(
-            text=text,
-            event_date=datetime(2025, 3, 28),
-            llm_config=llm_config,
-            agent_name="coding-agent",
-            context="agent work log",
-            config=_get_raw_config(),
-        )
-
-        assert len(facts) > 0, "Should extract at least one fact"
-        world_facts = [f for f in facts if f.fact_type == "world"]
-        experience_facts = [f for f in facts if f.fact_type == "experience"]
-        assert len(experience_facts) > len(world_facts), (
-            f"First-person debugging should be mostly 'experience', "
-            f"got {len(experience_facts)} experience vs {len(world_facts)} world. "
-            f"Facts: {[(f.fact, f.fact_type) for f in facts]}"
-        )
-
-    @pytest.mark.asyncio
-    async def test_user_interaction_classified_as_experience(self):
-        """Agent describing interactions with the user should be experience."""
-        text = """
-The user asked me to refactor the authentication middleware to support JWT tokens.
-I proposed splitting it into two modules: token_validation.py and session_management.py.
-The user approved my approach and I started with the token validation logic.
-I discovered that the existing tests were mocking the wrong interface, so I had to rewrite them first.
-"""
-        llm_config = LLMConfig.from_env()
-        facts, _, _ = await extract_facts_from_text(
-            text=text,
-            event_date=datetime(2025, 3, 28),
-            llm_config=llm_config,
-            agent_name="coding-agent",
-            context="agent work log",
-            config=_get_raw_config(),
-        )
-
-        assert len(facts) > 0, "Should extract at least one fact"
-        world_facts = [f for f in facts if f.fact_type == "world"]
-        experience_facts = [f for f in facts if f.fact_type == "experience"]
-        assert len(experience_facts) > len(world_facts), (
-            f"Agent-user interactions should be mostly 'experience', "
-            f"got {len(experience_facts)} experience vs {len(world_facts)} world. "
-            f"Facts: {[(f.fact, f.fact_type) for f in facts]}"
-        )
-
-    @pytest.mark.asyncio
-    async def test_mixed_agent_and_world_facts(self):
-        """Mix of agent experiences and world knowledge should be classified correctly."""
-        text = """
-Python 3.12 introduced a new type parameter syntax for generic classes.
-I migrated our codebase from the old TypeVar approach to the new syntax.
-The migration touched 23 files but was mostly mechanical.
-PEP 695 defines the new type statement that makes generics more readable.
-"""
-        llm_config = LLMConfig.from_env()
-        facts, _, _ = await extract_facts_from_text(
-            text=text,
-            event_date=datetime(2025, 3, 28),
-            llm_config=llm_config,
-            agent_name="coding-agent",
-            context="agent work log",
-            config=_get_raw_config(),
-        )
-
-        assert len(facts) > 0, "Should extract at least one fact"
-        world_facts = [f for f in facts if f.fact_type == "world"]
-        experience_facts = [f for f in facts if f.fact_type == "experience"]
-        # Should have both types - world facts about Python 3.12/PEP 695,
-        # experience facts about the migration work
-        assert len(world_facts) >= 1, (
-            f"Should have at least 1 world fact about Python 3.12/PEP 695. "
-            f"Facts: {[(f.fact, f.fact_type) for f in facts]}"
-        )
-        assert len(experience_facts) >= 1, (
-            f"Should have at least 1 experience fact about the migration. "
-            f"Facts: {[(f.fact, f.fact_type) for f in facts]}"
-        )

hindsight-api-slim/tests/test_fact_extraction_metadata.py

@@ -1,61 +0,0 @@
-"""
-Unit tests for metadata inclusion in fact extraction LLM prompt.
-"""
-from datetime import datetime
-
-from hindsight_api.engine.retain.fact_extraction import _build_user_message
-
-
-def test_build_user_message_includes_metadata():
-    """Metadata key-value pairs should appear in the user message."""
-    event_date = datetime(2024, 6, 15, 12, 0, 0)
-    metadata = {"title": "Q2 Planning Doc", "source": "confluence", "author": "Alice"}
-
-    msg = _build_user_message(
-        chunk="Some content.",
-        chunk_index=0,
-        total_chunks=1,
-        event_date=event_date,
-        context="planning meeting",
-        metadata=metadata,
-    )
-
-    assert "title" in msg
-    assert "Q2 Planning Doc" in msg
-    assert "source" in msg
-    assert "confluence" in msg
-    assert "author" in msg
-    assert "Alice" in msg
-
-
-def test_build_user_message_no_metadata():
-    """When metadata is empty, the message should still be valid and not include a metadata section."""
-    event_date = datetime(2024, 6, 15, 12, 0, 0)
-
-    msg = _build_user_message(
-        chunk="Some content.",
-        chunk_index=0,
-        total_chunks=1,
-        event_date=event_date,
-        context="planning meeting",
-        metadata={},
-    )
-
-    assert "Some content." in msg
-    assert "Metadata:" not in msg
-
-
-def test_build_user_message_without_metadata_arg():
-    """Calling without metadata (default) should behave the same as empty metadata."""
-    event_date = datetime(2024, 6, 15, 12, 0, 0)
-
-    msg = _build_user_message(
-        chunk="Some content.",
-        chunk_index=0,
-        total_chunks=1,
-        event_date=event_date,
-        context="none",
-    )
-
-    assert "Some content." in msg
-    assert "Metadata:" not in msg

hindsight-api-slim/tests/test_fact_extraction_retry.py

@@ -1,214 +0,0 @@
-"""
-Unit tests for fact extraction retry logic.
-
-Tests the fix for the TypeError when LLM returns invalid JSON across all retries.
-Previously, `raise last_error` would raise None (TypeError) because last_error was
-only set in the BadRequestError handler, not when the LLM returned non-dict JSON.
-"""
-
-from datetime import datetime, timezone
-from unittest.mock import AsyncMock, MagicMock, patch
-
-import pytest
-
-
-def _make_config(llm_max_retries: int = 3, retain_llm_max_retries: int | None = None):
-    """Build a minimal HindsightConfig for fact extraction tests."""
-    from hindsight_api.config import HindsightConfig
-
-    cfg = MagicMock(spec=HindsightConfig)
-    cfg.retain_llm_max_retries = retain_llm_max_retries
-    cfg.llm_max_retries = llm_max_retries
-    cfg.retain_llm_initial_backoff = None
-    cfg.llm_initial_backoff = 0.0
-    cfg.retain_llm_max_backoff = None
-    cfg.llm_max_backoff = 0.0
-    cfg.retain_max_completion_tokens = 8192
-    cfg.retain_extraction_mode = "concise"
-    cfg.retain_extract_causal_links = False
-    cfg.retain_mission = None
-    return cfg
-
-
-def _make_llm_config(mock_response):
-    """Build a mock LLMProvider that returns the given response."""
-    from hindsight_api.engine.llm_wrapper import LLMProvider
-
-    llm = MagicMock(spec=LLMProvider)
-    llm.provider = "mock"
-    token_usage = MagicMock()
-    token_usage.__add__ = lambda self, other: self
-    llm.call = AsyncMock(return_value=(mock_response, token_usage))
-    return llm
-
-
-@pytest.mark.asyncio
-async def test_non_dict_json_all_retries_returns_empty():
-    """
-    When LLM returns non-dict JSON on every attempt, extraction should return []
-    without raising TypeError ('exceptions must derive from BaseException').
-
-    This was the bug: the loop ran range(2) times (hardcoded), but comparisons
-    used config.llm_max_retries (default 10). On the last loop iteration (attempt=1),
-    `attempt < 10 - 1` was True, so the code called `continue`, the loop
-    exhausted, and `raise last_error` raised None → TypeError.
-    """
-    from hindsight_api.engine.retain.fact_extraction import _extract_facts_from_chunk
-
-    # llm_max_retries=3 ensures the bug triggers with the old code (3 != 2 hardcoded)
-    config = _make_config(llm_max_retries=3, retain_llm_max_retries=None)
-
-    # Mock: always returns a list (non-dict), which is invalid
-    llm_config = _make_llm_config(mock_response=[{"invalid": "response"}])
-
-    with patch(
-        "hindsight_api.engine.retain.fact_extraction._build_extraction_prompt_and_schema",
-        return_value=("system prompt", MagicMock()),
-    ):
-        facts, usage = await _extract_facts_from_chunk(
-            chunk="Alice visited Paris in 2023.",
-            chunk_index=0,
-            total_chunks=1,
-            event_date=datetime(2023, 1, 1, tzinfo=timezone.utc),
-            context="travel notes",
-            llm_config=llm_config,
-            config=config,
-            agent_name="test-agent",
-        )
-
-    assert facts == []
-
-
-@pytest.mark.asyncio
-async def test_non_dict_json_with_default_max_retries_returns_empty():
-    """
-    Same scenario with the default llm_max_retries=10 (matching real default config).
-    The old code ran range(2) but checked against 10, always continuing until
-    the loop exhausted, then raised None → TypeError.
-    """
-    from hindsight_api.engine.retain.fact_extraction import _extract_facts_from_chunk
-
-    config = _make_config(llm_max_retries=10, retain_llm_max_retries=None)
-    llm_config = _make_llm_config(mock_response="not a dict at all")
-
-    with patch(
-        "hindsight_api.engine.retain.fact_extraction._build_extraction_prompt_and_schema",
-        return_value=("system prompt", MagicMock()),
-    ):
-        facts, usage = await _extract_facts_from_chunk(
-            chunk="Some text.",
-            chunk_index=0,
-            total_chunks=1,
-            event_date=datetime(2023, 6, 1, tzinfo=timezone.utc),
-            context="",
-            llm_config=llm_config,
-            config=config,
-            agent_name="agent",
-        )
-
-    assert facts == []
-
-
-@pytest.mark.asyncio
-async def test_retain_llm_max_retries_overrides_global():
-    """
-    When retain_llm_max_retries is set, it should be used for the loop range
-    and all comparisons (no shadowing bug).
-    """
-    from hindsight_api.engine.retain.fact_extraction import _extract_facts_from_chunk
-
-    # retain_llm_max_retries=5 should override llm_max_retries=10
-    config = _make_config(llm_max_retries=10, retain_llm_max_retries=5)
-    llm_config = _make_llm_config(mock_response=42)  # non-dict: integer
-
-    with patch(
-        "hindsight_api.engine.retain.fact_extraction._build_extraction_prompt_and_schema",
-        return_value=("system prompt", MagicMock()),
-    ):
-        facts, usage = await _extract_facts_from_chunk(
-            chunk="Bob likes Python.",
-            chunk_index=0,
-            total_chunks=1,
-            event_date=datetime(2024, 1, 1, tzinfo=timezone.utc),
-            context="",
-            llm_config=llm_config,
-            config=config,
-            agent_name="agent",
-        )
-
-    assert facts == []
-    # Verify it retried exactly retain_llm_max_retries times
-    assert llm_config.call.call_count == 5
-
-
-@pytest.mark.asyncio
-async def test_none_event_date_with_empty_facts_no_crash():
-    """
-    When event_date is None and the LLM returns an empty facts list,
-    the debug log should not crash with AttributeError on .isoformat().
-
-    Regression test for https://github.com/vectorize-io/hindsight/issues/874
-    """
-    from hindsight_api.engine.retain.fact_extraction import _extract_facts_from_chunk
-
-    config = _make_config(llm_max_retries=1)
-
-    # LLM returns a valid dict but with no facts — triggers the debug log path
-    llm_config = _make_llm_config(mock_response={"facts": []})
-
-    with patch(
-        "hindsight_api.engine.retain.fact_extraction._build_extraction_prompt_and_schema",
-        return_value=("system prompt", MagicMock()),
-    ):
-        facts, usage = await _extract_facts_from_chunk(
-            chunk="A plain text document with no timestamp.",
-            chunk_index=0,
-            total_chunks=1,
-            event_date=None,
-            context="",
-            llm_config=llm_config,
-            config=config,
-            agent_name="test-agent",
-        )
-
-    assert facts == []
-
-
-@pytest.mark.asyncio
-async def test_none_event_date_with_valid_facts_no_crash():
-    """
-    When event_date is None but the LLM returns valid facts,
-    extraction should succeed without errors.
-    """
-    from hindsight_api.engine.retain.fact_extraction import _extract_facts_from_chunk
-
-    config = _make_config(llm_max_retries=1)
-
-    llm_config = _make_llm_config(mock_response={
-        "facts": [
-            {
-                "what": "Alice visited Paris",
-                "when": "2023",
-                "who": "Alice",
-                "why": "vacation",
-            }
-        ]
-    })
-
-    with patch(
-        "hindsight_api.engine.retain.fact_extraction._build_extraction_prompt_and_schema",
-        return_value=("system prompt", MagicMock()),
-    ):
-        facts, usage = await _extract_facts_from_chunk(
-            chunk="Alice visited Paris in 2023.",
-            chunk_index=0,
-            total_chunks=1,
-            event_date=None,
-            context="",
-            llm_config=llm_config,
-            config=config,
-            agent_name="test-agent",
-        )
-
-    assert len(facts) == 1
-    assert "Alice visited Paris" in facts[0].fact

hindsight-api-slim/tests/test_gemini_embeddings.py

@@ -1,336 +0,0 @@
-"""
-Tests for Google embeddings implementation (Gemini API + Vertex AI).
-
-These tests cover:
-1. Initialization (Gemini API key, Vertex AI with ADC/service account)
-2. Dimension detection via test embedding
-3. Output dimensionality configuration
-4. Encode (single text, multiple texts, batching, empty list, uninitialized)
-5. Provider name and model name normalization
-6. Factory function (create from env, validation errors)
-"""
-
-from typing import Any
-from unittest.mock import MagicMock, patch
-
-import pytest
-
-from hindsight_api.config import (
-    ENV_EMBEDDINGS_GEMINI_API_KEY,
-    ENV_EMBEDDINGS_PROVIDER,
-    HindsightConfig,
-)
-from hindsight_api.engine.embeddings import GeminiEmbeddings, create_embeddings_from_env
-
-
-def _make_mock_embedding(values: list[float]) -> MagicMock:
-    emb = MagicMock()
-    emb.values = values
-    return emb
-
-
-def _make_mock_embed_result(embeddings_data: list[list[float]]) -> MagicMock:
-    result = MagicMock()
-    result.embeddings = [_make_mock_embedding(v) for v in embeddings_data]
-    return result
-
-
-def _make_mock_genai(embed_result: Any = None) -> MagicMock:
-    if embed_result is None:
-        embed_result = _make_mock_embed_result([[0.1] * 768])
-    mock_genai = MagicMock()
-    mock_client = MagicMock()
-    mock_client.models.embed_content = MagicMock(return_value=embed_result)
-    mock_genai.Client = MagicMock(return_value=mock_client)
-    return mock_genai
-
-
-def _make_mock_google_module(mock_genai: MagicMock) -> MagicMock:
-    mod = MagicMock()
-    mod.genai = mock_genai
-    mod.genai.types.EmbedContentConfig = MagicMock(side_effect=lambda **kw: MagicMock(**kw))
-    return mod
-
-
-def _patch_google_import(mock_genai: MagicMock):
-    original_import = __import__
-
-    def mock_import(name, *args, **kwargs):
-        if name == "google":
-            return _make_mock_google_module(mock_genai)
-        if name == "google.genai":
-            return mock_genai
-        return original_import(name, *args, **kwargs)
-
-    return patch("builtins.__import__", side_effect=mock_import)
-
-
-class TestGeminiEmbeddings:
-    """Unit tests for GeminiEmbeddings with mocked google.genai."""
-
-    async def test_initialization_api_key_success(self):
-        """Test successful Gemini API key initialization."""
-        mock_genai = _make_mock_genai()
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key="test-key")
-
-        with _patch_google_import(mock_genai):
-            await emb.initialize()
-
-        assert emb._client is not None
-        assert emb.dimension == 768
-        assert emb.provider_name == "google"
-        assert emb._is_vertexai is False
-        mock_genai.Client.return_value.models.embed_content.assert_called_once()
-
-    async def test_initialization_vertexai_success(self):
-        """Test successful Vertex AI initialization."""
-        mock_genai = _make_mock_genai()
-        emb = GeminiEmbeddings(
-            model="gemini-embedding-001",
-            vertexai_project_id="test-project",
-            vertexai_region="us-central1",
-        )
-
-        with _patch_google_import(mock_genai):
-            await emb.initialize()
-
-        assert emb._client is not None
-        assert emb.dimension == 768
-        assert emb.provider_name == "google"
-        assert emb._is_vertexai is True
-        mock_genai.Client.assert_called_once_with(
-            vertexai=True,
-            project="test-project",
-            location="us-central1",
-        )
-
-    async def test_initialization_missing_api_key(self):
-        """Test that missing API key raises ValueError when no vertexai_project_id."""
-        mock_genai = _make_mock_genai()
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key=None)
-
-        with _patch_google_import(mock_genai):
-            with pytest.raises(ValueError, match="requires an API key"):
-                await emb.initialize()
-
-    async def test_initialization_vertexai_missing_project_id(self):
-        """Test that Vertex AI mode requires project_id."""
-        mock_genai = _make_mock_genai()
-        emb = GeminiEmbeddings(model="gemini-embedding-001", vertexai_project_id="temp")
-        emb.vertexai_project_id = None  # Simulate misconfiguration
-
-        with _patch_google_import(mock_genai):
-            with pytest.raises(ValueError, match="is required for Vertex AI"):
-                await emb.initialize()
-
-    async def test_initialization_idempotent(self):
-        """Test that calling initialize() twice is a no-op."""
-        mock_genai = _make_mock_genai()
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key="test-key")
-
-        with _patch_google_import(mock_genai):
-            await emb.initialize()
-            first_client = emb._client
-            await emb.initialize()
-            assert emb._client is first_client
-
-    async def test_dimension_detection_via_test_embedding(self):
-        """Test that dimension is detected via a test embedding call."""
-        test_embed = _make_mock_embed_result([[0.5] * 256])
-        mock_genai = _make_mock_genai(embed_result=test_embed)
-        emb = GeminiEmbeddings(model="some-new-model", api_key="test-key")
-
-        with _patch_google_import(mock_genai):
-            await emb.initialize()
-
-        assert emb.dimension == 256
-
-    async def test_output_dimensionality(self):
-        """Test that output_dimensionality is passed via EmbedContentConfig."""
-        test_embed = _make_mock_embed_result([[0.1] * 256])
-        mock_genai = _make_mock_genai(embed_result=test_embed)
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key="test-key", output_dimensionality=256)
-
-        with _patch_google_import(mock_genai):
-            await emb.initialize()
-
-        assert emb.dimension == 256
-        assert emb._embed_config is not None
-        call_kwargs = mock_genai.Client.return_value.models.embed_content.call_args
-        assert "config" in call_kwargs.kwargs
-
-    async def test_no_output_dimensionality(self):
-        """Test that no EmbedContentConfig is built when output_dimensionality is None."""
-        mock_genai = _make_mock_genai()
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key="test-key", output_dimensionality=None)
-
-        with _patch_google_import(mock_genai):
-            await emb.initialize()
-
-        assert emb._embed_config is None
-        call_kwargs = mock_genai.Client.return_value.models.embed_content.call_args
-        assert "config" not in call_kwargs.kwargs
-
-    def test_auto_detect_vertexai(self):
-        """Test that _is_vertexai is auto-detected from vertexai_project_id."""
-        assert GeminiEmbeddings(model="m", api_key="k")._is_vertexai is False
-        assert GeminiEmbeddings(model="m", vertexai_project_id="p")._is_vertexai is True
-
-    def test_encode_single_text(self):
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key="test-key")
-        mock_client = MagicMock()
-        mock_client.models.embed_content = MagicMock(return_value=_make_mock_embed_result([[0.1, 0.2, 0.3]]))
-        emb._client = mock_client
-        emb._dimension = 3
-
-        assert emb.encode(["hello"]) == [[0.1, 0.2, 0.3]]
-
-    def test_encode_multiple_texts(self):
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key="test-key")
-        mock_client = MagicMock()
-        mock_client.models.embed_content = MagicMock(
-            return_value=_make_mock_embed_result([[0.1, 0.2], [0.3, 0.4], [0.5, 0.6]])
-        )
-        emb._client = mock_client
-        emb._dimension = 2
-
-        result = emb.encode(["a", "b", "c"])
-        assert len(result) == 3
-        assert result[1] == [0.3, 0.4]
-
-    def test_encode_batching(self):
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key="test-key", batch_size=2)
-        mock_client = MagicMock()
-        mock_client.models.embed_content = MagicMock(
-            side_effect=[_make_mock_embed_result([[0.1], [0.2]]), _make_mock_embed_result([[0.3]])]
-        )
-        emb._client = mock_client
-        emb._dimension = 1
-
-        assert emb.encode(["a", "b", "c"]) == [[0.1], [0.2], [0.3]]
-        assert mock_client.models.embed_content.call_count == 2
-
-    def test_encode_passes_config(self):
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key="test-key")
-        mock_client = MagicMock()
-        mock_client.models.embed_content = MagicMock(return_value=_make_mock_embed_result([[0.1, 0.2]]))
-        emb._client = mock_client
-        emb._dimension = 2
-        emb._embed_config = MagicMock()
-
-        emb.encode(["hello"])
-        assert mock_client.models.embed_content.call_args.kwargs["config"] is emb._embed_config
-
-    def test_encode_empty_list(self):
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key="test-key")
-        emb._client = MagicMock()
-        emb._dimension = 768
-        assert emb.encode([]) == []
-
-    def test_encode_before_initialization(self):
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key="test-key")
-        with pytest.raises(RuntimeError, match="not initialized"):
-            emb.encode(["test"])
-
-    def test_dimension_before_initialization(self):
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key="test-key")
-        with pytest.raises(RuntimeError, match="not initialized"):
-            _ = emb.dimension
-
-    def test_provider_name_always_google(self):
-        assert GeminiEmbeddings(model="m", api_key="k").provider_name == "google"
-        assert GeminiEmbeddings(model="m", vertexai_project_id="p").provider_name == "google"
-
-    def test_vertexai_strips_google_prefix(self):
-        mock_genai = _make_mock_genai()
-        emb = GeminiEmbeddings(model="google/gemini-embedding-001", vertexai_project_id="test-project")
-        emb._init_vertexai(mock_genai)
-        assert emb.model == "gemini-embedding-001"
-
-    def test_default_region(self):
-        emb = GeminiEmbeddings(model="m", vertexai_project_id="proj")
-        assert emb.vertexai_region == "us-central1"
-
-    def test_custom_region(self):
-        emb = GeminiEmbeddings(model="m", vertexai_project_id="proj", vertexai_region="europe-west1")
-        assert emb.vertexai_region == "europe-west1"
-
-
-class TestGeminiEmbeddingsFactory:
-    """Tests for create_embeddings_from_env() with 'google' provider."""
-
-    def _make_config(self, **overrides) -> HindsightConfig:
-        from dataclasses import fields
-
-        defaults = {}
-        for f in fields(HindsightConfig):
-            if f.type == "str":
-                defaults[f.name] = ""
-            elif f.type == "str | None":
-                defaults[f.name] = None
-            elif f.type == "int":
-                defaults[f.name] = 0
-            elif f.type == "int | None":
-                defaults[f.name] = None
-            elif f.type == "float":
-                defaults[f.name] = 0.0
-            elif f.type == "float | None":
-                defaults[f.name] = None
-            elif f.type == "bool":
-                defaults[f.name] = False
-            elif f.type == "list | None":
-                defaults[f.name] = None
-            else:
-                defaults[f.name] = None
-
-        defaults["embeddings_provider"] = "google"
-        defaults["embeddings_gemini_api_key"] = "test-key"
-        defaults["embeddings_gemini_model"] = "gemini-embedding-001"
-        defaults["embeddings_gemini_output_dimensionality"] = 768
-        defaults["embeddings_vertexai_project_id"] = None
-        defaults["embeddings_vertexai_region"] = None
-        defaults["embeddings_vertexai_service_account_key"] = None
-
-        defaults.update(overrides)
-        return HindsightConfig(**defaults)
-
-    def test_create_with_api_key(self):
-        config = self._make_config()
-        with patch("hindsight_api.config.get_config", return_value=config):
-            emb = create_embeddings_from_env()
-        assert isinstance(emb, GeminiEmbeddings)
-        assert emb.provider_name == "google"
-        assert emb.api_key == "test-key"
-        assert emb._is_vertexai is False
-
-    def test_create_with_vertexai(self):
-        config = self._make_config(
-            embeddings_gemini_api_key=None,
-            embeddings_vertexai_project_id="my-project",
-            embeddings_vertexai_region="us-east1",
-        )
-        with patch("hindsight_api.config.get_config", return_value=config):
-            emb = create_embeddings_from_env()
-        assert isinstance(emb, GeminiEmbeddings)
-        assert emb._is_vertexai is True
-        assert emb.api_key is None
-        assert emb.vertexai_project_id == "my-project"
-
-    def test_create_missing_all_credentials(self):
-        config = self._make_config(embeddings_gemini_api_key=None, embeddings_vertexai_project_id=None)
-        with patch("hindsight_api.config.get_config", return_value=config):
-            with pytest.raises(ValueError, match="is required"):
-                create_embeddings_from_env()
-
-    def test_vertexai_takes_priority(self):
-        config = self._make_config(embeddings_gemini_api_key="key", embeddings_vertexai_project_id="proj")
-        with patch("hindsight_api.config.get_config", return_value=config):
-            emb = create_embeddings_from_env()
-        assert emb._is_vertexai is True
-        assert emb.api_key is None
-
-    def test_create_with_custom_dimensionality(self):
-        config = self._make_config(embeddings_gemini_output_dimensionality=256)
-        with patch("hindsight_api.config.get_config", return_value=config):
-            emb = create_embeddings_from_env()
-        assert emb.output_dimensionality == 256

hindsight-api-slim/tests/test_gemini_safety_settings.py

@@ -1,340 +0,0 @@
-"""
-Tests for Gemini safety settings feature.
-
-Verifies that:
-- Safety settings are read from env var and stored on GeminiLLM instances
-- Settings are applied to GenerateContentConfig in call() and call_with_tools()
-- The context variable override allows per-bank settings at request time
-- None (unset) means Gemini's default safety settings are used (no override)
-"""
-
-import os
-from unittest.mock import AsyncMock, MagicMock, patch
-
-import pytest
-
-pytest.importorskip("google.genai")
-
-
-SAMPLE_SAFETY_SETTINGS = [
-    {"category": "HARM_CATEGORY_HARASSMENT", "threshold": "BLOCK_NONE"},
-    {"category": "HARM_CATEGORY_HATE_SPEECH", "threshold": "BLOCK_NONE"},
-    {"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT", "threshold": "BLOCK_NONE"},
-    {"category": "HARM_CATEGORY_DANGEROUS_CONTENT", "threshold": "BLOCK_NONE"},
-]
-
-
-# ─── Config / env var parsing ─────────────────────────────────────────────────
-
-
-def test_gemini_safety_settings_parsed_from_env():
-    """Safety settings JSON from env var is parsed into HindsightConfig."""
-    import json
-
-    from hindsight_api.config import ENV_LLM_GEMINI_SAFETY_SETTINGS, HindsightConfig, clear_config_cache
-
-    settings_json = json.dumps(SAMPLE_SAFETY_SETTINGS)
-    with patch.dict(os.environ, {ENV_LLM_GEMINI_SAFETY_SETTINGS: settings_json}, clear=False):
-        clear_config_cache()
-        config = HindsightConfig.from_env()
-        assert config.llm_gemini_safety_settings == SAMPLE_SAFETY_SETTINGS
-        clear_config_cache()
-
-
-def test_gemini_safety_settings_default_is_none():
-    """When env var is not set, llm_gemini_safety_settings defaults to None."""
-    from hindsight_api.config import ENV_LLM_GEMINI_SAFETY_SETTINGS, HindsightConfig, clear_config_cache
-
-    env = {k: v for k, v in os.environ.items() if k != ENV_LLM_GEMINI_SAFETY_SETTINGS}
-    with patch.dict(os.environ, env, clear=True):
-        clear_config_cache()
-        config = HindsightConfig.from_env()
-        assert config.llm_gemini_safety_settings is None
-        clear_config_cache()
-
-
-def test_gemini_safety_settings_is_configurable_field():
-    """llm_gemini_safety_settings appears in configurable (per-bank) fields."""
-    from hindsight_api.config import HindsightConfig
-
-    assert "llm_gemini_safety_settings" in HindsightConfig.get_configurable_fields()
-
-
-def test_gemini_safety_settings_not_in_credential_fields():
-    """llm_gemini_safety_settings is NOT a credential — it is safe to expose via API."""
-    from hindsight_api.config import HindsightConfig
-
-    assert "llm_gemini_safety_settings" not in HindsightConfig.get_credential_fields()
-
-
-# ─── GeminiLLM instance ───────────────────────────────────────────────────────
-
-
-def _make_gemini_provider(safety_settings=None):
-    """Return a GeminiLLM instance with a mocked genai.Client."""
-    with patch("google.genai.Client") as mock_client_cls:
-        mock_client_cls.return_value = MagicMock()
-        from hindsight_api.engine.providers.gemini_llm import GeminiLLM
-
-        provider = GeminiLLM(
-            provider="gemini",
-            api_key="fake-api-key",
-            base_url="",
-            model="gemini-2.5-flash",
-            gemini_safety_settings=safety_settings,
-        )
-        # Replace client with a fresh mock so we can inspect calls
-        provider._client = MagicMock()
-        return provider
-
-
-def test_gemini_llm_stores_safety_settings():
-    """GeminiLLM stores safety settings passed at construction."""
-    provider = _make_gemini_provider(safety_settings=SAMPLE_SAFETY_SETTINGS)
-    assert provider._safety_settings == SAMPLE_SAFETY_SETTINGS
-
-
-def test_gemini_llm_no_safety_settings_is_none():
-    """GeminiLLM._safety_settings is None when not provided."""
-    provider = _make_gemini_provider(safety_settings=None)
-    assert provider._safety_settings is None
-
-
-# ─── call() applies safety settings ──────────────────────────────────────────
-
-
-@pytest.mark.asyncio
-async def test_call_applies_safety_settings():
-    """call() includes safety_settings in GenerateContentConfig when configured."""
-    from google.genai import types as genai_types
-
-    provider = _make_gemini_provider(safety_settings=SAMPLE_SAFETY_SETTINGS)
-
-    # Build a fake successful response
-    fake_response = MagicMock()
-    fake_response.text = "hello"
-    fake_response.candidates = [MagicMock(finish_reason="STOP")]
-    fake_response.usage_metadata = MagicMock(prompt_token_count=5, candidates_token_count=2)
-
-    provider._client.aio.models.generate_content = AsyncMock(return_value=fake_response)
-
-    await provider.call(
-        messages=[{"role": "user", "content": "hi"}],
-        scope="test",
-    )
-
-    # Inspect the config passed to generate_content
-    call_args = provider._client.aio.models.generate_content.call_args
-    config_arg = call_args.kwargs.get("config") or call_args.args[0] if call_args.args else None
-    # config may be in kwargs or positional; grab from kwargs
-    config_arg = call_args.kwargs.get("config")
-
-    assert config_arg is not None, "GenerateContentConfig should have been passed"
-    assert hasattr(config_arg, "safety_settings"), "Config should have safety_settings"
-    assert config_arg.safety_settings is not None
-
-    categories = [s.category.value if hasattr(s.category, "value") else str(s.category) for s in config_arg.safety_settings]
-    assert "HARM_CATEGORY_HARASSMENT" in categories
-    assert "HARM_CATEGORY_HATE_SPEECH" in categories
-    assert "HARM_CATEGORY_SEXUALLY_EXPLICIT" in categories
-    assert "HARM_CATEGORY_DANGEROUS_CONTENT" in categories
-
-    thresholds = [s.threshold.value if hasattr(s.threshold, "value") else str(s.threshold) for s in config_arg.safety_settings]
-    assert all(t == "BLOCK_NONE" for t in thresholds)
-
-
-@pytest.mark.asyncio
-async def test_call_no_safety_settings_omits_key():
-    """call() does NOT add safety_settings to GenerateContentConfig when none configured."""
-    provider = _make_gemini_provider(safety_settings=None)
-
-    fake_response = MagicMock()
-    fake_response.text = "hello"
-    fake_response.candidates = [MagicMock(finish_reason="STOP")]
-    fake_response.usage_metadata = MagicMock(prompt_token_count=5, candidates_token_count=2)
-
-    provider._client.aio.models.generate_content = AsyncMock(return_value=fake_response)
-
-    await provider.call(
-        messages=[{"role": "user", "content": "hi"}],
-        scope="test",
-    )
-
-    call_args = provider._client.aio.models.generate_content.call_args
-    config_arg = call_args.kwargs.get("config")
-
-    # When no safety settings, config is either None or lacks safety_settings
-    if config_arg is not None:
-        assert not hasattr(config_arg, "safety_settings") or config_arg.safety_settings is None
-
-
-# ─── call_with_tools() applies safety settings ────────────────────────────────
-
-
-@pytest.mark.asyncio
-async def test_call_with_tools_applies_safety_settings():
-    """call_with_tools() includes safety_settings in GenerateContentConfig."""
-    provider = _make_gemini_provider(safety_settings=SAMPLE_SAFETY_SETTINGS)
-
-    # Build a fake tool-use response (no tool calls, just text)
-    fake_part = MagicMock()
-    fake_part.text = "answer"
-    fake_part.function_call = None
-
-    fake_candidate = MagicMock()
-    fake_candidate.content = MagicMock(parts=[fake_part])
-
-    fake_response = MagicMock()
-    fake_response.candidates = [fake_candidate]
-    fake_response.usage_metadata = MagicMock(prompt_token_count=5, candidates_token_count=3)
-
-    provider._client.aio.models.generate_content = AsyncMock(return_value=fake_response)
-
-    tools = [
-        {
-            "type": "function",
-            "function": {
-                "name": "test_tool",
-                "description": "A test tool",
-                "parameters": {"type": "object", "properties": {}, "required": []},
-            },
-        }
-    ]
-
-    await provider.call_with_tools(
-        messages=[{"role": "user", "content": "hi"}],
-        tools=tools,
-        scope="test",
-    )
-
-    call_args = provider._client.aio.models.generate_content.call_args
-    config_arg = call_args.kwargs.get("config")
-
-    assert config_arg is not None
-    assert config_arg.safety_settings is not None
-    categories = [s.category.value if hasattr(s.category, "value") else str(s.category) for s in config_arg.safety_settings]
-    assert "HARM_CATEGORY_HARASSMENT" in categories
-
-
-# ─── with_config() override ───────────────────────────────────────────────────
-
-
-def _make_llm_provider(safety_settings=None):
-    """Return an LLMProvider (wrapping GeminiLLM) with a mocked genai.Client."""
-    with patch("google.genai.Client") as mock_client_cls:
-        mock_client_cls.return_value = MagicMock()
-        from hindsight_api.engine.llm_wrapper import LLMProvider
-
-        provider = LLMProvider(
-            provider="gemini",
-            api_key="fake-api-key",
-            base_url="",
-            model="gemini-2.5-flash",
-            gemini_safety_settings=safety_settings,
-        )
-        # Replace the underlying Gemini client with a fresh mock
-        provider._provider_impl._client = MagicMock()
-        return provider
-
-
-def _fake_response():
-    r = MagicMock()
-    r.text = "hello"
-    r.candidates = [MagicMock(finish_reason="STOP")]
-    r.usage_metadata = MagicMock(prompt_token_count=5, candidates_token_count=2)
-    return r
-
-
-def _make_config(safety_settings):
-    """Return a minimal config-like object with llm_gemini_safety_settings."""
-    cfg = MagicMock()
-    cfg.llm_gemini_safety_settings = safety_settings
-    return cfg
-
-
-@pytest.mark.asyncio
-async def test_with_config_overrides_instance_settings():
-    """with_config() settings take precedence over the provider instance defaults."""
-    instance_settings = [{"category": "HARM_CATEGORY_HARASSMENT", "threshold": "BLOCK_ONLY_HIGH"}]
-    override_settings = [{"category": "HARM_CATEGORY_HATE_SPEECH", "threshold": "BLOCK_NONE"}]
-
-    provider = _make_llm_provider(safety_settings=instance_settings)
-    provider._provider_impl._client.aio.models.generate_content = AsyncMock(return_value=_fake_response())
-
-    configured = provider.with_config(_make_config(override_settings))
-    await configured.call(messages=[{"role": "user", "content": "hi"}], scope="test")
-
-    config_arg = provider._provider_impl._client.aio.models.generate_content.call_args.kwargs.get("config")
-    assert config_arg is not None
-    categories = [s.category.value if hasattr(s.category, "value") else str(s.category) for s in config_arg.safety_settings]
-    # Should use override_settings (HATE_SPEECH), not instance_settings (HARASSMENT)
-    assert "HARM_CATEGORY_HATE_SPEECH" in categories
-    assert "HARM_CATEGORY_HARASSMENT" not in categories
-
-
-@pytest.mark.asyncio
-async def test_with_config_none_falls_back_to_instance():
-    """When with_config() supplies None, the instance default is used."""
-    instance_settings = [{"category": "HARM_CATEGORY_HARASSMENT", "threshold": "BLOCK_NONE"}]
-
-    provider = _make_llm_provider(safety_settings=instance_settings)
-    provider._provider_impl._client.aio.models.generate_content = AsyncMock(return_value=_fake_response())
-
-    configured = provider.with_config(_make_config(None))
-    await configured.call(messages=[{"role": "user", "content": "hi"}], scope="test")
-
-    config_arg = provider._provider_impl._client.aio.models.generate_content.call_args.kwargs.get("config")
-    assert config_arg is not None
-    categories = [s.category.value if hasattr(s.category, "value") else str(s.category) for s in config_arg.safety_settings]
-    assert "HARM_CATEGORY_HARASSMENT" in categories
-
-
-@pytest.mark.asyncio
-async def test_with_config_resets_after_call():
-    """The ContextVar is properly reset after a with_config() call (no leakage)."""
-    from hindsight_api.engine.providers.gemini_llm import _safety_settings_ctx
-
-    settings = [{"category": "HARM_CATEGORY_HARASSMENT", "threshold": "BLOCK_NONE"}]
-    provider = _make_llm_provider(safety_settings=None)
-    provider._provider_impl._client.aio.models.generate_content = AsyncMock(return_value=_fake_response())
-
-    before = _safety_settings_ctx.get()
-    configured = provider.with_config(_make_config(settings))
-    await configured.call(messages=[{"role": "user", "content": "hi"}], scope="test")
-    after = _safety_settings_ctx.get()
-
-    assert after == before  # ContextVar restored to its original value
-
-
-# ─── LLMProvider reads safety settings from config ────────────────────────────
-
-
-def test_llm_provider_reads_safety_settings_from_config():
-    """LLMProvider reads llm_gemini_safety_settings from global config for Gemini provider."""
-    import json
-
-    from hindsight_api.config import ENV_LLM_GEMINI_SAFETY_SETTINGS, clear_config_cache
-
-    settings_json = json.dumps(SAMPLE_SAFETY_SETTINGS)
-    env_overrides = {
-        "HINDSIGHT_API_LLM_PROVIDER": "gemini",
-        "HINDSIGHT_API_LLM_API_KEY": "fake-key",
-        ENV_LLM_GEMINI_SAFETY_SETTINGS: settings_json,
-    }
-
-    with patch.dict(os.environ, env_overrides, clear=False):
-        clear_config_cache()
-        with patch("google.genai.Client") as mock_client_cls:
-            mock_client_cls.return_value = MagicMock()
-            from hindsight_api.engine.llm_wrapper import LLMProvider
-
-            provider = LLMProvider(
-                provider="gemini",
-                api_key="fake-key",
-                base_url="",
-                model="gemini-2.5-flash",
-            )
-
-            assert provider.gemini_safety_settings == SAMPLE_SAFETY_SETTINGS
-
-        clear_config_cache()

hindsight-api-slim/tests/test_google_cross_encoder.py

@@ -1,275 +0,0 @@
-"""
-Tests for Google Discovery Engine cross-encoder (Ranking REST API).
-
-These tests cover:
-1. Initialization (service account, ADC, missing project_id)
-2. Predict (single query, multiple queries, batching, empty pairs, uninitialized)
-3. Provider name
-4. Factory function (create from env, validation errors)
-"""
-
-from unittest.mock import MagicMock, patch
-
-import httpx
-import pytest
-
-from hindsight_api.config import (
-    ENV_RERANKER_GOOGLE_PROJECT_ID,
-    ENV_RERANKER_PROVIDER,
-    HindsightConfig,
-)
-from hindsight_api.engine.cross_encoder import GoogleCrossEncoder, create_cross_encoder_from_env
-
-
-def _make_rank_response(records: list[tuple[str, float]]) -> dict:
-    """Build a JSON response matching the Discovery Engine REST API format."""
-    return {"records": [{"id": rid, "score": score} for rid, score in records]}
-
-
-def _make_mock_httpx_client(responses: list[dict] | None = None) -> MagicMock:
-    """Create a mock httpx.Client that returns predefined responses."""
-    mock_client = MagicMock(spec=httpx.Client)
-    if responses:
-        side_effects = []
-        for resp_json in responses:
-            mock_resp = MagicMock(spec=httpx.Response)
-            mock_resp.json.return_value = resp_json
-            mock_resp.raise_for_status.return_value = None
-            side_effects.append(mock_resp)
-        mock_client.post.side_effect = side_effects
-    return mock_client
-
-
-def _make_mock_credentials() -> MagicMock:
-    """Create mock credentials with a valid token."""
-    creds = MagicMock()
-    creds.valid = True
-    creds.token = "mock-token"
-    return creds
-
-
-class TestGoogleCrossEncoder:
-    """Unit tests for GoogleCrossEncoder with mocked httpx + google-auth."""
-
-    async def test_initialization_adc_success(self):
-        """Test successful initialization with ADC (no service account key)."""
-        mock_creds = _make_mock_credentials()
-
-        encoder = GoogleCrossEncoder(project_id="test-project")
-
-        with patch("google.auth.default", return_value=(mock_creds, "test-project")):
-            await encoder.initialize()
-
-        assert encoder._client is not None
-        assert encoder._credentials is mock_creds
-        assert encoder.provider_name == "google"
-        assert "test-project" in encoder._rank_url
-
-    async def test_initialization_service_account(self):
-        """Test initialization with service account key."""
-        mock_creds = _make_mock_credentials()
-
-        encoder = GoogleCrossEncoder(
-            project_id="test-project",
-            service_account_key="/path/to/key.json",
-        )
-
-        with patch(
-            "google.oauth2.service_account.Credentials.from_service_account_file",
-            return_value=mock_creds,
-        ):
-            await encoder.initialize()
-
-        assert encoder._client is not None
-        assert encoder._credentials is mock_creds
-
-    async def test_initialization_idempotent(self):
-        """Test that calling initialize() twice is a no-op."""
-        mock_creds = _make_mock_credentials()
-        encoder = GoogleCrossEncoder(project_id="test-project")
-
-        with patch("google.auth.default", return_value=(mock_creds, "test-project")):
-            await encoder.initialize()
-            first_client = encoder._client
-            await encoder.initialize()
-            assert encoder._client is first_client
-
-    async def test_predict_single_query(self):
-        """Test prediction with a single query and multiple documents."""
-        mock_creds = _make_mock_credentials()
-        mock_client = _make_mock_httpx_client([
-            _make_rank_response([("1", 0.95), ("0", 0.30)]),
-        ])
-
-        encoder = GoogleCrossEncoder(project_id="test-project")
-        with patch("google.auth.default", return_value=(mock_creds, "p")):
-            await encoder.initialize()
-        encoder._client = mock_client
-
-        scores = await encoder.predict([
-            ("What is AI?", "AI is artificial intelligence"),
-            ("What is AI?", "The sky is blue"),
-        ])
-
-        assert len(scores) == 2
-        assert scores[0] == 0.30  # id="0" -> index 0
-        assert scores[1] == 0.95  # id="1" -> index 1
-        mock_client.post.assert_called_once()
-
-    async def test_predict_multiple_queries(self):
-        """Test prediction with multiple distinct queries."""
-        mock_creds = _make_mock_credentials()
-        mock_client = _make_mock_httpx_client([
-            _make_rank_response([("0", 0.9), ("1", 0.1)]),
-            _make_rank_response([("0", 0.8)]),
-        ])
-
-        encoder = GoogleCrossEncoder(project_id="test-project")
-        with patch("google.auth.default", return_value=(mock_creds, "p")):
-            await encoder.initialize()
-        encoder._client = mock_client
-
-        scores = await encoder.predict([
-            ("Query A", "Doc A1"),
-            ("Query A", "Doc A2"),
-            ("Query B", "Doc B1"),
-        ])
-
-        assert len(scores) == 3
-        assert scores[0] == 0.9
-        assert scores[1] == 0.1
-        assert scores[2] == 0.8
-        assert mock_client.post.call_count == 2
-
-    async def test_predict_empty_pairs(self):
-        """Test that empty pairs returns empty list."""
-        mock_creds = _make_mock_credentials()
-        encoder = GoogleCrossEncoder(project_id="test-project")
-
-        with patch("google.auth.default", return_value=(mock_creds, "p")):
-            await encoder.initialize()
-
-        scores = await encoder.predict([])
-        assert scores == []
-
-    async def test_predict_not_initialized(self):
-        """Test that predict raises if not initialized."""
-        encoder = GoogleCrossEncoder(project_id="test-project")
-        with pytest.raises(RuntimeError, match="not initialized"):
-            await encoder.predict([("q", "d")])
-
-    async def test_predict_batching(self):
-        """Test that >200 records are split into batches."""
-        mock_creds = _make_mock_credentials()
-        mock_client = _make_mock_httpx_client([
-            _make_rank_response([(str(i), 0.5) for i in range(200)]),
-            _make_rank_response([(str(i), 0.3) for i in range(50)]),
-        ])
-
-        encoder = GoogleCrossEncoder(project_id="test-project")
-        with patch("google.auth.default", return_value=(mock_creds, "p")):
-            await encoder.initialize()
-        encoder._client = mock_client
-
-        pairs = [("same query", f"doc {i}") for i in range(250)]
-        scores = await encoder.predict(pairs)
-
-        assert len(scores) == 250
-        assert mock_client.post.call_count == 2
-
-    async def test_auth_header_sent(self):
-        """Test that Authorization header is sent with requests."""
-        mock_creds = _make_mock_credentials()
-        mock_creds.token = "test-bearer-token"
-        mock_client = _make_mock_httpx_client([
-            _make_rank_response([("0", 0.9)]),
-        ])
-
-        encoder = GoogleCrossEncoder(project_id="test-project")
-        with patch("google.auth.default", return_value=(mock_creds, "p")):
-            await encoder.initialize()
-        encoder._client = mock_client
-
-        await encoder.predict([("q", "d")])
-
-        call_kwargs = mock_client.post.call_args
-        assert call_kwargs.kwargs["headers"]["Authorization"] == "Bearer test-bearer-token"
-
-    def test_provider_name(self):
-        assert GoogleCrossEncoder(project_id="p").provider_name == "google"
-
-    def test_default_model(self):
-        encoder = GoogleCrossEncoder(project_id="p")
-        assert encoder.model == "semantic-ranker-default-004"
-
-    def test_custom_model(self):
-        encoder = GoogleCrossEncoder(project_id="p", model="semantic-ranker-fast-004")
-        assert encoder.model == "semantic-ranker-fast-004"
-
-    def test_default_location(self):
-        encoder = GoogleCrossEncoder(project_id="p")
-        assert encoder.location == "global"
-
-
-class TestGoogleCrossEncoderFactory:
-    """Tests for create_cross_encoder_from_env() with 'google' provider."""
-
-    def _make_config(self, **overrides) -> HindsightConfig:
-        from dataclasses import fields
-
-        defaults = {}
-        for f in fields(HindsightConfig):
-            if f.type == "str":
-                defaults[f.name] = ""
-            elif f.type == "str | None":
-                defaults[f.name] = None
-            elif f.type == "int":
-                defaults[f.name] = 0
-            elif f.type == "int | None":
-                defaults[f.name] = None
-            elif f.type == "float":
-                defaults[f.name] = 0.0
-            elif f.type == "float | None":
-                defaults[f.name] = None
-            elif f.type == "bool":
-                defaults[f.name] = False
-            elif f.type == "list | None":
-                defaults[f.name] = None
-            else:
-                defaults[f.name] = None
-
-        defaults["reranker_provider"] = "google"
-        defaults["reranker_google_model"] = "semantic-ranker-default-004"
-        defaults["reranker_google_project_id"] = "test-project"
-        defaults["reranker_google_service_account_key"] = None
-
-        defaults.update(overrides)
-        return HindsightConfig(**defaults)
-
-    def test_create_with_project_id(self):
-        config = self._make_config()
-        with patch("hindsight_api.config.get_config", return_value=config):
-            encoder = create_cross_encoder_from_env()
-        assert isinstance(encoder, GoogleCrossEncoder)
-        assert encoder.provider_name == "google"
-        assert encoder.project_id == "test-project"
-        assert encoder.service_account_key is None
-
-    def test_create_with_service_account(self):
-        config = self._make_config(reranker_google_service_account_key="/path/to/key.json")
-        with patch("hindsight_api.config.get_config", return_value=config):
-            encoder = create_cross_encoder_from_env()
-        assert isinstance(encoder, GoogleCrossEncoder)
-        assert encoder.service_account_key == "/path/to/key.json"
-
-    def test_create_missing_project_id(self):
-        config = self._make_config(reranker_google_project_id=None)
-        with patch("hindsight_api.config.get_config", return_value=config):
-            with pytest.raises(ValueError, match="is required"):
-                create_cross_encoder_from_env()
-
-    def test_create_with_custom_model(self):
-        config = self._make_config(reranker_google_model="semantic-ranker-fast-004")
-        with patch("hindsight_api.config.get_config", return_value=config):
-            encoder = create_cross_encoder_from_env()
-        assert encoder.model == "semantic-ranker-fast-004"

hindsight-api-slim/tests/test_graph_entity_fanout_cap.py

@@ -1,211 +0,0 @@
-"""
-Tests for LATERAL entity fanout cap in graph expansion.
-
-Verifies that the per-entity LIMIT in _expand_combined prevents high-fanout
-entities from exploding the self-join, while still returning entity-based
-graph results.
-"""
-
-import asyncio
-from datetime import datetime, timezone
-
-import pytest
-
-
-@pytest.mark.asyncio
-async def test_high_fanout_entity_returns_results(memory, request_context):
-    """
-    A high-fanout entity (appearing in many facts) should still produce
-    graph retrieval results — the LATERAL cap limits rows per entity but
-    does not drop the entity entirely.
-    """
-    bank_id = f"test_fanout_cap_{datetime.now(timezone.utc).timestamp()}"
-
-    try:
-        # Create many facts sharing one common entity ("Acme Corp") plus
-        # a few with a unique entity so we can query for the unique one
-        # and verify graph expansion finds siblings via "Acme Corp".
-        contents = [
-            # Target: unique entity "Zara" shares "Acme Corp" with the rest
-            {
-                "content": "Zara joined Acme Corp as a senior engineer last month",
-                "context": "hr update",
-                "entities": [{"text": "Zara"}, {"text": "Acme Corp"}],
-            },
-        ]
-        # Add many facts that all share "Acme Corp" — creates a high-fanout entity
-        for i in range(60):
-            contents.append(
-                {
-                    "content": f"Employee {i} completed onboarding at Acme Corp in department {i % 5}",
-                    "context": "hr update",
-                    "entities": [{"text": f"Employee {i}"}, {"text": "Acme Corp"}],
-                }
-            )
-
-        await memory.retain_batch_async(
-            bank_id=bank_id,
-            contents=contents,
-            request_context=request_context,
-        )
-
-        from hindsight_api.engine.memory_engine import Budget
-
-        # Query for "Zara" — semantic search finds Zara's fact as a seed,
-        # then graph expansion should find other Acme Corp facts via the
-        # shared entity, even though "Acme Corp" has 60+ mentions.
-        result = await memory.recall_async(
-            bank_id=bank_id,
-            query="Zara",
-            budget=Budget.HIGH,
-            max_tokens=4096,
-            enable_trace=True,
-            request_context=request_context,
-            _quiet=True,
-        )
-
-        assert result.results is not None
-        assert len(result.results) > 0
-
-        # Verify graph retrieval ran and found results
-        retrieval_results = result.trace.get("retrieval_results", [])
-        graph_results = [r for r in retrieval_results if r.get("method_name") == "graph"]
-        assert len(graph_results) > 0, "Graph retrieval should have run"
-
-        # At least one graph result should contain Acme Corp content
-        # (found via shared entity, not just semantic similarity)
-        all_texts = [r.text for r in result.results]
-        acme_found = any("Acme Corp" in t for t in all_texts)
-        assert acme_found, "Should find Acme Corp facts via entity graph expansion"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_entity_expansion_timeout_fallback(memory, request_context):
-    """
-    When graph_expansion_timeout is set very low, entity expansion should
-    time out gracefully and fall back to semantic+causal links only,
-    rather than failing the entire recall.
-    """
-    bank_id = f"test_timeout_fallback_{datetime.now(timezone.utc).timestamp()}"
-
-    try:
-        await memory.retain_batch_async(
-            bank_id=bank_id,
-            contents=[
-                {
-                    "content": "Alice works on the backend API at TechCorp",
-                    "context": "team info",
-                    "entities": [{"text": "Alice"}, {"text": "TechCorp"}],
-                },
-                {
-                    "content": "Bob maintains the frontend at TechCorp",
-                    "context": "team info",
-                    "entities": [{"text": "Bob"}, {"text": "TechCorp"}],
-                },
-            ],
-            request_context=request_context,
-        )
-
-        from hindsight_api.config import _get_raw_config
-        from hindsight_api.engine.memory_engine import Budget
-
-        config = _get_raw_config()
-        original_timeout = config.link_expansion_timeout
-
-        try:
-            # Set an impossibly low timeout to force the fallback path
-            config.link_expansion_timeout = 0.0001
-
-            result = await memory.recall_async(
-                bank_id=bank_id,
-                query="Alice",
-                budget=Budget.MID,
-                max_tokens=2048,
-                enable_trace=True,
-                request_context=request_context,
-                _quiet=True,
-            )
-
-            # Recall should succeed even when entity expansion times out
-            assert result.results is not None
-            assert len(result.results) > 0
-
-            # Alice should still be found via semantic search
-            result_texts = [r.text for r in result.results]
-            alice_found = any("Alice" in t for t in result_texts)
-            assert alice_found, "Should find Alice via semantic search despite graph timeout"
-        finally:
-            config.link_expansion_timeout = original_timeout
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_per_entity_limit_caps_expansion(memory, request_context):
-    """
-    With graph_per_entity_limit set to a small value, entity expansion should
-    still work but return fewer results from high-fanout entities.
-    """
-    bank_id = f"test_per_entity_limit_{datetime.now(timezone.utc).timestamp()}"
-
-    try:
-        # Create facts with a shared entity
-        contents = [
-            {
-                "content": "Lead engineer Dana oversees the Widgets project at MegaCorp",
-                "context": "project info",
-                "entities": [{"text": "Dana"}, {"text": "MegaCorp"}],
-            },
-        ]
-        for i in range(30):
-            contents.append(
-                {
-                    "content": f"MegaCorp hired contractor {i} for the Q4 push",
-                    "context": "hiring info",
-                    "entities": [{"text": f"Contractor {i}"}, {"text": "MegaCorp"}],
-                }
-            )
-
-        await memory.retain_batch_async(
-            bank_id=bank_id,
-            contents=contents,
-            request_context=request_context,
-        )
-
-        from hindsight_api.config import _get_raw_config
-        from hindsight_api.engine.memory_engine import Budget
-
-        config = _get_raw_config()
-        original_limit = config.link_expansion_per_entity_limit
-
-        try:
-            # Set a very small per-entity limit
-            config.link_expansion_per_entity_limit = 5
-
-            result = await memory.recall_async(
-                bank_id=bank_id,
-                query="Dana",
-                budget=Budget.HIGH,
-                max_tokens=4096,
-                enable_trace=True,
-                request_context=request_context,
-                _quiet=True,
-            )
-
-            # Recall should succeed with the cap
-            assert result.results is not None
-            assert len(result.results) > 0
-
-            # Graph retrieval should have run
-            retrieval_results = result.trace.get("retrieval_results", [])
-            graph_results = [r for r in retrieval_results if r.get("method_name") == "graph"]
-            assert len(graph_results) > 0, "Graph retrieval should have run"
-        finally:
-            config.link_expansion_per_entity_limit = original_limit
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)

hindsight-api-slim/tests/test_graph_filtering.py

@@ -1,171 +0,0 @@
-"""
-Tests for server-side filtering in the graph API endpoint.
-
-Verifies that q (text search) and tags filters work correctly
-when passed as query parameters to GET /v1/default/banks/{bank_id}/graph.
-"""
-from datetime import datetime
-
-import httpx
-import pytest
-import pytest_asyncio
-
-from hindsight_api.api import create_app
-
-
-@pytest_asyncio.fixture
-async def api_client(memory):
-    """Create an async test client for the FastAPI app."""
-    app = create_app(memory, initialize_memory=False)
-    transport = httpx.ASGITransport(app=app)
-    async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
-        yield client
-
-
-@pytest.fixture
-def test_bank_id():
-    """Provide a unique bank ID for this test run."""
-    return f"graph_filter_test_{datetime.now().timestamp()}"
-
-
-@pytest.mark.asyncio
-async def test_graph_no_filter_returns_all(api_client, test_bank_id):
-    """Without filters the graph endpoint returns all memories."""
-    response = await api_client.post(
-        f"/v1/default/banks/{test_bank_id}/memories",
-        json={
-            "items": [
-                {"content": "Alice loves hiking in the mountains.", "tags": ["user_alice"]},
-                {"content": "Bob enjoys swimming at the beach.", "tags": ["user_bob"]},
-            ]
-        },
-    )
-    assert response.status_code == 200
-
-    response = await api_client.get(f"/v1/default/banks/{test_bank_id}/graph")
-    assert response.status_code == 200
-    data = response.json()
-    assert "table_rows" in data
-    texts = [row["text"] for row in data["table_rows"]]
-    assert any("Alice" in t for t in texts)
-    assert any("Bob" in t for t in texts)
-
-
-@pytest.mark.asyncio
-async def test_graph_q_filter_returns_matching(api_client, test_bank_id):
-    """The q parameter filters memories by text content."""
-    response = await api_client.post(
-        f"/v1/default/banks/{test_bank_id}/memories",
-        json={
-            "items": [
-                {"content": "Alice loves hiking in the mountains."},
-                {"content": "Bob enjoys swimming at the beach."},
-            ]
-        },
-    )
-    assert response.status_code == 200
-
-    response = await api_client.get(f"/v1/default/banks/{test_bank_id}/graph", params={"q": "Alice"})
-    assert response.status_code == 200
-    data = response.json()
-    texts = [row["text"] for row in data["table_rows"]]
-    assert all("Alice" in t or "alice" in t.lower() for t in texts), (
-        f"Expected only Alice memories, got: {texts}"
-    )
-    assert not any("Bob" in t for t in texts)
-
-
-@pytest.mark.asyncio
-async def test_graph_q_filter_case_insensitive(api_client, test_bank_id):
-    """The q filter is case-insensitive."""
-    response = await api_client.post(
-        f"/v1/default/banks/{test_bank_id}/memories",
-        json={
-            "items": [
-                {"content": "Alice loves hiking in the mountains."},
-                {"content": "Bob enjoys swimming at the beach."},
-            ]
-        },
-    )
-    assert response.status_code == 200
-
-    response = await api_client.get(f"/v1/default/banks/{test_bank_id}/graph", params={"q": "alice"})
-    assert response.status_code == 200
-    data = response.json()
-    texts = [row["text"] for row in data["table_rows"]]
-    assert any("Alice" in t for t in texts)
-    assert not any("Bob" in t for t in texts)
-
-
-@pytest.mark.asyncio
-async def test_graph_tags_filter_returns_matching(api_client, test_bank_id):
-    """The tags parameter filters memories to only those with matching tags."""
-    response = await api_client.post(
-        f"/v1/default/banks/{test_bank_id}/memories",
-        json={
-            "items": [
-                {"content": "Alice loves hiking.", "tags": ["user_alice"]},
-                {"content": "Bob enjoys swimming.", "tags": ["user_bob"]},
-            ]
-        },
-    )
-    assert response.status_code == 200
-
-    response = await api_client.get(
-        f"/v1/default/banks/{test_bank_id}/graph",
-        params={"tags": "user_alice", "tags_match": "all_strict"},
-    )
-    assert response.status_code == 200
-    data = response.json()
-    texts = [row["text"] for row in data["table_rows"]]
-    assert any("Alice" in t for t in texts)
-    assert not any("Bob" in t for t in texts)
-
-
-@pytest.mark.asyncio
-async def test_graph_q_and_tags_filter_combined(api_client, test_bank_id):
-    """Combining q and tags filters applies both server-side."""
-    response = await api_client.post(
-        f"/v1/default/banks/{test_bank_id}/memories",
-        json={
-            "items": [
-                {"content": "Alice loves hiking.", "tags": ["user_alice"]},
-                {"content": "Alice also loves coding.", "tags": ["user_alice"]},
-                {"content": "Bob enjoys swimming.", "tags": ["user_bob"]},
-            ]
-        },
-    )
-    assert response.status_code == 200
-
-    response = await api_client.get(
-        f"/v1/default/banks/{test_bank_id}/graph",
-        params={"q": "hiking", "tags": "user_alice", "tags_match": "all_strict"},
-    )
-    assert response.status_code == 200
-    data = response.json()
-    texts = [row["text"] for row in data["table_rows"]]
-    assert any("hiking" in t.lower() for t in texts)
-    assert not any("coding" in t.lower() for t in texts)
-    assert not any("Bob" in t for t in texts)
-
-
-@pytest.mark.asyncio
-async def test_graph_q_filter_empty_results(api_client, test_bank_id):
-    """The q filter returns empty results when no memory matches."""
-    response = await api_client.post(
-        f"/v1/default/banks/{test_bank_id}/memories",
-        json={
-            "items": [
-                {"content": "Alice loves hiking."},
-            ]
-        },
-    )
-    assert response.status_code == 200
-
-    response = await api_client.get(
-        f"/v1/default/banks/{test_bank_id}/graph",
-        params={"q": "zzznomatchzzz"},
-    )
-    assert response.status_code == 200
-    data = response.json()
-    assert data["table_rows"] == []

hindsight-api-slim/tests/test_hnsw_indexes.py

@@ -1,193 +0,0 @@
-"""
-Tests for per-bank vector index lifecycle and UNION ALL retrieval.
-
-Covers:
-- _bank_index_name deterministic naming
-- Per-bank vector indexes created on bank creation (retain_async / ensure_bank_exists)
-- Per-bank vector indexes dropped on bank deletion
-- retrieve_semantic_bm25_combined groups results correctly by fact_type and source
-"""
-import uuid
-from datetime import datetime, timezone
-
-import pytest
-
-from hindsight_api.engine.retain.bank_utils import _BANK_INDEX_FACT_TYPES, _bank_index_name
-
-
-# ---------------------------------------------------------------------------
-# Unit tests — no DB required
-# ---------------------------------------------------------------------------
-
-
-class TestBankIndexName:
-    def test_deterministic(self):
-        uid = "550e8400-e29b-41d4-a716-446655440000"
-        assert _bank_index_name("world", uid) == _bank_index_name("world", uid)
-
-    def test_strips_dashes(self):
-        uid = "550e8400-e29b-41d4-a716-446655440000"
-        name = _bank_index_name("world", uid)
-        # uid16 should be hex chars only
-        assert "-" not in name
-
-    def test_uses_first_16_hex_chars(self):
-        uid = "550e8400-e29b-41d4-a716-446655440000"
-        uid16 = uid.replace("-", "")[:16]  # "550e8400e29b41d4"
-        assert name_ends_with(name=_bank_index_name("world", uid), suffix=uid16)
-
-    def test_suffix_per_fact_type(self):
-        uid = "550e8400-e29b-41d4-a716-446655440000"
-        names = {ft: _bank_index_name(ft, uid) for ft in _BANK_INDEX_FACT_TYPES}
-        # All three names must be distinct
-        assert len(set(names.values())) == 3
-
-    def test_all_fact_types_covered(self):
-        assert set(_BANK_INDEX_FACT_TYPES) == {"world", "experience", "observation"}
-
-    def test_fits_pg_identifier_limit(self):
-        # PostgreSQL max identifier length is 63 chars
-        uid = "f" * 32  # simulated UUID without dashes
-        for ft in _BANK_INDEX_FACT_TYPES:
-            assert len(_bank_index_name(ft, uid)) <= 63
-
-
-def name_ends_with(name: str, suffix: str) -> bool:
-    return name.endswith(suffix)
-
-
-# ---------------------------------------------------------------------------
-# Integration tests — require DB (memory fixture)
-# ---------------------------------------------------------------------------
-
-
-async def _get_bank_vector_indexes(pool, bank_id: str) -> list[str]:
-    """Return index names for memory_units that match the per-bank pattern."""
-    async with pool.acquire() as conn:
-        rows = await conn.fetch(
-            """
-            SELECT indexname
-            FROM pg_indexes
-            WHERE tablename = 'memory_units'
-              AND indexname LIKE 'idx_mu_emb_%'
-              AND indexdef LIKE $1
-            ORDER BY indexname
-            """,
-            f"%bank_id = '{bank_id}'%",
-        )
-    return [row["indexname"] for row in rows]
-
-
-@pytest.mark.asyncio
-async def test_retain_creates_per_bank_vector_indexes(memory, request_context):
-    """retain_async on a new bank must create 3 per-(bank, fact_type) vector indexes."""
-    bank_id = f"test_hnsw_create_{uuid.uuid4().hex[:8]}"
-    try:
-        await memory.retain_async(
-            bank_id=bank_id,
-            content="Alice is a software engineer.",
-            request_context=request_context,
-        )
-        indexes = await _get_bank_vector_indexes(memory._pool, bank_id)
-        assert len(indexes) == 3, f"Expected 3 per-bank vector indexes, got: {indexes}"
-        for ft_short in _BANK_INDEX_FACT_TYPES.values():
-            assert any(ft_short in idx for idx in indexes), (
-                f"Missing index for fact_type short '{ft_short}' in {indexes}"
-            )
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_delete_bank_drops_vector_indexes(memory, request_context):
-    """delete_bank must drop all per-bank vector indexes."""
-    bank_id = f"test_hnsw_drop_{uuid.uuid4().hex[:8]}"
-
-    await memory.retain_async(
-        bank_id=bank_id,
-        content="Bob is a data scientist.",
-        request_context=request_context,
-    )
-    # Verify indexes exist before deletion
-    indexes_before = await _get_bank_vector_indexes(memory._pool, bank_id)
-    assert len(indexes_before) == 3
-
-    await memory.delete_bank(bank_id, request_context=request_context)
-
-    indexes_after = await _get_bank_vector_indexes(memory._pool, bank_id)
-    assert indexes_after == [], f"Indexes should be dropped after bank deletion, got: {indexes_after}"
-
-
-@pytest.mark.asyncio
-async def test_retain_idempotent_bank_creation(memory, request_context):
-    """Retaining into the same bank twice must not error and still have exactly 3 indexes."""
-    bank_id = f"test_hnsw_idem_{uuid.uuid4().hex[:8]}"
-    try:
-        await memory.retain_async(
-            bank_id=bank_id,
-            content="Carol is a product manager.",
-            request_context=request_context,
-        )
-        await memory.retain_async(
-            bank_id=bank_id,
-            content="Carol joined the company in 2022.",
-            request_context=request_context,
-        )
-        indexes = await _get_bank_vector_indexes(memory._pool, bank_id)
-        assert len(indexes) == 3
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_retrieve_semantic_bm25_grouped_by_fact_type(memory, request_context):
-    """
-    retrieve_semantic_bm25_combined must return a dict keyed by fact_type with
-    (semantic_list, bm25_list) tuples.  All returned facts must belong to their
-    declared fact_type.
-    """
-    from hindsight_api.engine.search.retrieval import retrieve_semantic_bm25_combined
-
-    bank_id = f"test_retrieval_{uuid.uuid4().hex[:8]}"
-    try:
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=(
-                "Alice is a software engineer at TechCorp. "
-                "She visited Paris in 2023 for a conference."
-            ),
-            context="background",
-            event_date=datetime(2023, 6, 1, tzinfo=timezone.utc),
-            request_context=request_context,
-        )
-
-        query_emb = memory.embeddings.encode(["software engineer Alice"])
-        query_emb_str = str(query_emb[0])
-
-        fact_types = ["world", "experience"]
-        async with memory._pool.acquire() as conn:
-            results = await retrieve_semantic_bm25_combined(
-                conn=conn,
-                query_emb_str=query_emb_str,
-                query_text="software engineer Alice",
-                bank_id=bank_id,
-                fact_types=fact_types,
-                limit=5,
-            )
-
-        # Must return an entry for every requested fact_type
-        assert set(results.keys()) == set(fact_types)
-
-        for ft, (sem, bm25) in results.items():
-            # Semantic and BM25 lists must be lists
-            assert isinstance(sem, list)
-            assert isinstance(bm25, list)
-            # All semantic results must declare the correct fact_type
-            for r in sem:
-                assert r.fact_type == ft, f"Semantic result has wrong fact_type: {r.fact_type}"
-            # All BM25 results must declare the correct fact_type
-            for r in bm25:
-                assert r.fact_type == ft, f"BM25 result has wrong fact_type: {r.fact_type}"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)

hindsight-api-slim/tests/test_horse_observations.py

@@ -1,353 +0,0 @@
-"""Test observation tracking for a sequence of horse-related memories.
-
-This test retains a series of facts about horses on a farm and inspects
-how observations track the evolving state over time, with full prompt debugging.
-"""
-
-import json
-import uuid
-from dataclasses import dataclass, field
-from typing import Any
-
-import pytest
-
-from hindsight_api.config import _get_raw_config
-from hindsight_api.engine.consolidation import consolidator as consolidator_mod
-from hindsight_api.engine.memory_engine import MemoryEngine
-
-
-@pytest.fixture(autouse=True)
-def enable_observations():
-    """Enable observations for all tests in this module."""
-    config = _get_raw_config()
-    original_value = config.enable_observations
-    config.enable_observations = True
-    yield
-    config.enable_observations = original_value
-
-
-@dataclass
-class _ActionLog:
-    text: str
-    source_fact_ids: list[str] = field(default_factory=list)
-    observation_id: str = ""
-
-
-@dataclass
-class _ConsolidationResponse:
-    creates: list[_ActionLog] = field(default_factory=list)
-    updates: list[_ActionLog] = field(default_factory=list)
-    deletes: list[_ActionLog] = field(default_factory=list)
-
-
-@dataclass
-class _ConsolidationDebugEntry:
-    facts: str
-    observations_text: str
-    response: _ConsolidationResponse
-
-
-# Store prompts/responses for debugging
-_debug_log: list[_ConsolidationDebugEntry] = []
-
-
-def _fact_line(m: dict[str, Any]) -> str:
-    text = f"[{m['id']}] {m['text']}"
-    temporal_parts = []
-    if m.get("occurred_start"):
-        temporal_parts.append(f"occurred_start={m['occurred_start']}")
-    if m.get("occurred_end"):
-        temporal_parts.append(f"occurred_end={m['occurred_end']}")
-    if m.get("mentioned_at"):
-        temporal_parts.append(f"mentioned_at={m['mentioned_at']}")
-    if temporal_parts:
-        text += f" ({', '.join(temporal_parts)})"
-    return text
-
-
-async def _instrumented_consolidate(
-    original_fn: Any,
-    *,
-    llm_config: Any,
-    memories: list[dict[str, Any]],
-    union_observations: Any,
-    union_source_facts: Any,
-    config: Any = None,
-    remaining_observation_slots: int | None = None,
-    max_observations_per_scope: int = -1,
-) -> Any:
-    """Wrapper that captures the prompt and response for debugging."""
-    if union_observations:
-        obs_list = consolidator_mod._build_observations_for_llm(union_observations, union_source_facts)
-        observations_text = json.dumps(obs_list, indent=2)
-    else:
-        observations_text = "[]"
-
-    facts_lines = "\n".join(_fact_line(m) for m in memories)
-
-    result = await original_fn(
-        llm_config=llm_config,
-        memories=memories,
-        union_observations=union_observations,
-        union_source_facts=union_source_facts,
-        config=config,
-        remaining_observation_slots=remaining_observation_slots,
-        max_observations_per_scope=max_observations_per_scope,
-    )
-
-    _debug_log.append(_ConsolidationDebugEntry(
-        facts=facts_lines,
-        observations_text=observations_text,
-        response=_ConsolidationResponse(
-            creates=[_ActionLog(text=c.text, source_fact_ids=c.source_fact_ids) for c in result.creates],
-            updates=[
-                _ActionLog(text=u.text, observation_id=u.observation_id, source_fact_ids=u.source_fact_ids)
-                for u in result.updates
-            ],
-            deletes=[_ActionLog(text="", observation_id=d.observation_id) for d in result.deletes],
-        ),
-    ))
-
-    return result
-
-
-def _print_consolidation_debug(entry: _ConsolidationDebugEntry, index: int) -> None:
-    """Print a single consolidation LLM call for debugging."""
-    print(f"\n  --- LLM Call #{index} ---")
-    print("  FACTS sent to LLM:")
-    for line in entry.facts.split("\n"):
-        print(f"    {line}")
-    print("\n  EXISTING OBSERVATIONS sent to LLM:")
-    obs_data = json.loads(entry.observations_text)
-    if obs_data:
-        for obs in obs_data:
-            src_summary = ""
-            if obs.get("source_memories"):
-                src_texts = [sm["text"] for sm in obs["source_memories"]]
-                src_summary = f" (sources: {src_texts})"
-            print(f"    [{obs['id'][:8]}..] proof={obs.get('proof_count', '?')}: {obs['text']}{src_summary}")
-    else:
-        print("    (none)")
-
-    resp = entry.response
-    print("\n  LLM RESPONSE:")
-    if resp.creates:
-        for c in resp.creates:
-            print(f"    CREATE: \"{c.text}\" (from facts: {[fid[:8] + '..' for fid in c.source_fact_ids]})")
-    if resp.updates:
-        for u in resp.updates:
-            print(
-                f"    UPDATE [{u.observation_id[:8]}..]: \"{u.text}\""
-                f" (from facts: {[fid[:8] + '..' for fid in u.source_fact_ids]})"
-            )
-    if resp.deletes:
-        for d in resp.deletes:
-            print(f"    DELETE [{d.observation_id[:8]}..]")
-    if not resp.creates and not resp.updates and not resp.deletes:
-        print("    (no actions)")
-
-
-def _parse_history(hist: Any) -> list[str]:
-    """Parse observation history from DB (may be list of dicts or JSON strings)."""
-    if not hist:
-        return []
-    parsed = hist if isinstance(hist, list) else json.loads(hist)
-    prev_texts = []
-    for h in parsed:
-        if isinstance(h, str):
-            h = json.loads(h)
-        prev_texts.append(h.get("previous_text", "?"))
-    return prev_texts
-
-
-@pytest.mark.asyncio
-@pytest.mark.flaky(reruns=2, reruns_delay=5)
-async def test_horse_farm_observation_history(memory: MemoryEngine, request_context: Any) -> None:
-    """Retain a sequence of horse facts and inspect how observations evolve."""
-    bank_id = f"test-horses-{uuid.uuid4().hex[:8]}"
-
-    await memory.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-    messages = [
-        "I have a farm.",
-        "I have 2 horses.",
-        "I have a horse named Daisy.",
-        "I have a horse named Buttercup.",
-        "I sold Buttercup.",
-        "I now have 1 horse.",
-        "I have 5 horses on my farm.",
-        "I have a horse named Midnight.",
-        "I have horses named Midnight and Shadow.",
-        "I have horses named Shadow and Twister.",
-        "I am sad to report that Shadow has died.",
-    ]
-
-    # Monkey-patch to intercept consolidation LLM calls
-    _original_consolidate = consolidator_mod._consolidate_batch_with_llm
-
-    async def _patched(**kwargs: Any) -> Any:
-        return await _instrumented_consolidate(_original_consolidate, **kwargs)
-
-    consolidator_mod._consolidate_batch_with_llm = _patched
-    _debug_log.clear()
-
-    try:
-        for i, content in enumerate(messages):
-            print(f"\n{'='*80}")
-            print(f"RETAIN #{i+1}: {content}")
-            print(f"{'='*80}")
-
-            log_start = len(_debug_log)
-
-            await memory.retain_async(
-                bank_id=bank_id,
-                content=content,
-                request_context=request_context,
-            )
-            await memory.wait_for_background_tasks()
-
-            for j, entry in enumerate(_debug_log[log_start:]):
-                _print_consolidation_debug(entry, j + 1)
-
-            # Dump current observations
-            pool = await memory._get_pool()
-            async with pool.acquire() as conn:
-                observations = await conn.fetch(
-                    """
-                    SELECT id, text, proof_count, source_memory_ids, history
-                    FROM memory_units
-                    WHERE bank_id = $1 AND fact_type = 'observation'
-                    ORDER BY created_at
-                    """,
-                    bank_id,
-                )
-
-                print(f"\n  CURRENT OBSERVATIONS ({len(observations)}):")
-                for obs in observations:
-                    prev_texts = _parse_history(obs["history"])
-                    hist_str = f" (was: {' -> '.join(prev_texts)})" if prev_texts else ""
-                    print(f"    [{str(obs['id'])[:8]}..] proof={obs['proof_count']}: {obs['text']}{hist_str}")
-    finally:
-        consolidator_mod._consolidate_batch_with_llm = _original_consolidate
-
-    # Final summary
-    print(f"\n{'='*80}")
-    print("FINAL STATE")
-    print(f"{'='*80}")
-    pool = await memory._get_pool()
-    async with pool.acquire() as conn:
-        observations = await conn.fetch(
-            """
-            SELECT id, text, proof_count, source_memory_ids, history
-            FROM memory_units
-            WHERE bank_id = $1 AND fact_type = 'observation'
-            ORDER BY created_at
-            """,
-            bank_id,
-        )
-        print(f"\nFinal observations ({len(observations)}):")
-        for obs in observations:
-            prev_texts = _parse_history(obs["history"])
-            if prev_texts:
-                chain = prev_texts + [obs["text"]]
-                print(f"  - [proof={obs['proof_count']}] {obs['text']}")
-                print(f"    evolution: {' -> '.join(chain)}")
-            else:
-                print(f"  - [proof={obs['proof_count']}] {obs['text']}")
-
-    # Create a mental model to synthesize the observations
-    print(f"\n{'='*80}")
-    print("MENTAL MODEL")
-    print(f"{'='*80}")
-
-    # Patch reflect _execute_tool to log tool inputs/outputs
-    from hindsight_api.engine.reflect import agent as reflect_agent_mod
-
-    _original_execute = reflect_agent_mod._execute_tool
-
-    async def _logging_execute(tool_name: str, args: dict[str, Any], *a: Any, **kw: Any) -> dict[str, Any]:
-        result = await _original_execute(tool_name, args, *a, **kw)
-        normalized = reflect_agent_mod._normalize_tool_name(tool_name)
-        print(f"\n  [REFLECT TOOL] {normalized}(args={args})")
-        if isinstance(result, dict):
-            if "observations" in result:
-                print(f"    Observations returned ({result.get('count', '?')}, freshness={result.get('freshness', '?')}):")
-                for obs in result.get("observations", []):
-                    print(f"      - [proof={obs.get('proof_count', '?')}] {obs.get('text', '?')}")
-            if "memories" in result:
-                print(f"    Memories returned ({result.get('count', '?')}):")
-                for mem in result.get("memories", []):
-                    chunk = mem.get("chunk_text", "")
-                    chunk_preview = f" | chunk: {chunk[:80]}..." if chunk else ""
-                    print(f"      - [{mem.get('fact_type', '?')}] {mem.get('text', '?')}{chunk_preview}")
-            if "mental_models" in result:
-                print(f"    Mental models returned ({result.get('count', '?')}):")
-                for mm_item in result.get("mental_models", []):
-                    print(f"      - {mm_item.get('name', '?')}: {str(mm_item.get('content', '?'))[:120]}")
-            if "error" in result:
-                print(f"    ERROR: {result['error']}")
-        return result
-
-    reflect_agent_mod._execute_tool = _logging_execute
-
-    source_query = (
-        "Produce a structured summary of all animals on the farm. Include:\n"
-        "1. A chronological timeline of events (acquisitions, sales, deaths) with dates\n"
-        "2. The list of all known horse names and their current status (alive, sold, died)\n"
-        "3. The current number of horses on the farm, accounting for all events\n"
-        "Reason step by step from the facts. If a horse died or was sold, subtract from the count."
-    )
-
-    try:
-        mm = await memory.create_mental_model(
-            bank_id=bank_id,
-            name="Farm Animals",
-            source_query=source_query,
-            content="(initial — awaiting refresh)",
-            request_context=request_context,
-        )
-
-        refreshed = await memory.refresh_mental_model(
-            bank_id=bank_id,
-            mental_model_id=mm["id"],
-            request_context=request_context,
-        )
-        content = refreshed["content"]
-    finally:
-        reflect_agent_mod._execute_tool = _original_execute
-    print(f"\nMental model content:\n{content}")
-
-    reflect_resp = refreshed.get("reflect_response")
-    if reflect_resp and isinstance(reflect_resp, str) and reflect_resp.strip():
-        try:
-            reflect_resp = json.loads(reflect_resp)
-        except json.JSONDecodeError:
-            reflect_resp = None
-    if isinstance(reflect_resp, dict):
-        based_on = reflect_resp.get("based_on", [])
-        if based_on:
-            print("\nBased on:")
-            for item in based_on:
-                if isinstance(item, str):
-                    try:
-                        item = json.loads(item)
-                    except json.JSONDecodeError:
-                        continue
-                print(f"  - [{item.get('fact_type', '?')}] {item.get('text', '?')}")
-
-    # Verify the mental model captures key facts
-    content_lower = content.lower()
-
-    for name in ["daisy", "buttercup", "midnight", "shadow", "twister"]:
-        assert name in content_lower, f"Mental model should mention {name}. Got:\n{content}"
-
-    assert "sold" in content_lower or "sale" in content_lower, (
-        f"Mental model should mention Buttercup was sold. Got:\n{content}"
-    )
-
-    assert "died" in content_lower or "passed" in content_lower or "death" in content_lower, (
-        f"Mental model should mention Shadow's death. Got:\n{content}"
-    )
-
-    # Cleanup
-    await memory.delete_bank(bank_id, request_context=request_context)

hindsight-api-slim/tests/test_list_documents.py

@@ -1,174 +0,0 @@
-"""
-Tests for list_documents pagination and tags filtering.
-"""
-from datetime import datetime, timezone
-
-import pytest
-
-
-async def _retain_doc(memory, bank_id, document_id, tags, request_context):
-    """Helper to retain a document with given tags. Uses gibberish content to avoid LLM
-    fact extraction (documents are persisted even with zero facts)."""
-    await memory.retain_batch_async(
-        bank_id=bank_id,
-        contents=[{"content": f"xyzabc123 !@# $$$ {document_id}"}],
-        document_id=document_id,
-        document_tags=tags or None,
-        request_context=request_context,
-    )
-
-
-@pytest.mark.asyncio
-async def test_list_documents_offset_pagination(memory, request_context):
-    """offset parameter returns the correct slice of documents."""
-    bank_id = f"test_list_docs_offset_{datetime.now(timezone.utc).timestamp()}"
-
-    try:
-        for i in range(4):
-            await _retain_doc(memory, bank_id, f"doc-{i:02d}", [], request_context)
-
-        # All documents, ordered by created_at DESC → doc-03, doc-02, doc-01, doc-00
-        all_docs = await memory.list_documents(
-            bank_id=bank_id, limit=10, offset=0, request_context=request_context
-        )
-        assert all_docs["total"] == 4
-        assert len(all_docs["items"]) == 4
-        all_ids = [d["id"] for d in all_docs["items"]]
-
-        # offset=2 should skip the first two and return the remaining two
-        page2 = await memory.list_documents(
-            bank_id=bank_id, limit=10, offset=2, request_context=request_context
-        )
-        assert page2["total"] == 4  # total is always the full count
-        assert len(page2["items"]) == 2
-        assert [d["id"] for d in page2["items"]] == all_ids[2:]
-
-        # offset beyond total returns empty items but correct total
-        beyond = await memory.list_documents(
-            bank_id=bank_id, limit=10, offset=10, request_context=request_context
-        )
-        assert beyond["total"] == 4
-        assert beyond["items"] == []
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_list_documents_tags_filter_any_strict(memory, request_context):
-    """tags filter with any_strict returns only tagged documents that match."""
-    bank_id = f"test_list_docs_tags_{datetime.now(timezone.utc).timestamp()}"
-
-    try:
-        await _retain_doc(memory, bank_id, "doc-alpha", ["team-a"], request_context)
-        await _retain_doc(memory, bank_id, "doc-beta", ["team-b"], request_context)
-        await _retain_doc(memory, bank_id, "doc-both", ["team-a", "team-b"], request_context)
-        await _retain_doc(memory, bank_id, "doc-untagged", [], request_context)
-
-        # any_strict: only docs with at least one of the given tags, untagged excluded
-        result = await memory.list_documents(
-            bank_id=bank_id,
-            tags=["team-a"],
-            tags_match="any_strict",
-            request_context=request_context,
-        )
-        ids = {d["id"] for d in result["items"]}
-        assert ids == {"doc-alpha", "doc-both"}
-        assert result["total"] == 2
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_list_documents_tags_filter_any_includes_untagged(memory, request_context):
-    """tags filter with 'any' mode includes untagged documents."""
-    bank_id = f"test_list_docs_tags_any_{datetime.now(timezone.utc).timestamp()}"
-
-    try:
-        await _retain_doc(memory, bank_id, "doc-tagged", ["team-a"], request_context)
-        await _retain_doc(memory, bank_id, "doc-other", ["team-b"], request_context)
-        await _retain_doc(memory, bank_id, "doc-untagged", [], request_context)
-
-        result = await memory.list_documents(
-            bank_id=bank_id,
-            tags=["team-a"],
-            tags_match="any",
-            request_context=request_context,
-        )
-        ids = {d["id"] for d in result["items"]}
-        # "any" includes untagged + matching tagged
-        assert "doc-tagged" in ids
-        assert "doc-untagged" in ids
-        assert "doc-other" not in ids
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_list_documents_tags_filter_all_strict(memory, request_context):
-    """tags filter with all_strict returns only docs that have ALL the specified tags."""
-    bank_id = f"test_list_docs_tags_all_{datetime.now(timezone.utc).timestamp()}"
-
-    try:
-        await _retain_doc(memory, bank_id, "doc-a-only", ["team-a"], request_context)
-        await _retain_doc(memory, bank_id, "doc-a-and-b", ["team-a", "team-b"], request_context)
-        await _retain_doc(memory, bank_id, "doc-untagged", [], request_context)
-
-        result = await memory.list_documents(
-            bank_id=bank_id,
-            tags=["team-a", "team-b"],
-            tags_match="all_strict",
-            request_context=request_context,
-        )
-        ids = {d["id"] for d in result["items"]}
-        assert ids == {"doc-a-and-b"}
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_list_documents_no_tags_filter_returns_all(memory, request_context):
-    """When no tags filter is specified, all documents are returned."""
-    bank_id = f"test_list_docs_no_tags_{datetime.now(timezone.utc).timestamp()}"
-
-    try:
-        await _retain_doc(memory, bank_id, "doc-tagged", ["team-a"], request_context)
-        await _retain_doc(memory, bank_id, "doc-untagged", [], request_context)
-
-        result = await memory.list_documents(
-            bank_id=bank_id,
-            tags=None,
-            request_context=request_context,
-        )
-        ids = {d["id"] for d in result["items"]}
-        assert ids == {"doc-tagged", "doc-untagged"}
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_list_documents_tags_and_search_query_combined(memory, request_context):
-    """tags filter and q (search_query) can be combined."""
-    bank_id = f"test_list_docs_tags_q_{datetime.now(timezone.utc).timestamp()}"
-
-    try:
-        await _retain_doc(memory, bank_id, "report-2024", ["team-a"], request_context)
-        await _retain_doc(memory, bank_id, "report-2025", ["team-b"], request_context)
-        await _retain_doc(memory, bank_id, "summary-2024", ["team-a"], request_context)
-
-        result = await memory.list_documents(
-            bank_id=bank_id,
-            search_query="report",
-            tags=["team-a"],
-            tags_match="any_strict",
-            request_context=request_context,
-        )
-        ids = {d["id"] for d in result["items"]}
-        assert ids == {"report-2024"}
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)

hindsight-api-slim/tests/test_llm_wrapper.py

@@ -1,37 +0,0 @@
-import pytest
-
-from hindsight_api.engine.llm_wrapper import sanitize_llm_output
-
-
-@pytest.mark.parametrize(
-    "input_text, expected",
-    [
-        # Null bytes stripped
-        ("hello\x00world", "helloworld"),
-        ("FIRST\u0000PAGE", "FIRSTPAGE"),
-        # Multiple null bytes
-        ("\x00\x00text\x00", "text"),
-        # Other control characters stripped (non-whitespace)
-        ("text\x01\x02\x03end", "textend"),
-        ("text\x08end", "textend"),  # backspace
-        ("text\x0cend", "textend"),  # form feed
-        ("text\x0bend", "textend"),  # vertical tab
-        ("text\x1fend", "textend"),  # unit separator
-        ("text\x7fend", "textend"),  # DEL
-        # Whitespace preserved
-        ("hello\tworld", "hello\tworld"),
-        ("hello\nworld", "hello\nworld"),
-        ("hello\r\nworld", "hello\r\nworld"),
-        # Unicode surrogates stripped
-        ("text\ud800end", "textend"),
-        ("text\udfffend", "textend"),
-        # Clean text unchanged
-        ("normal text", "normal text"),
-        ("unicode: café naïve", "unicode: café naïve"),
-        # Edge cases
-        ("", ""),
-        (None, None),
-    ],
-)
-def test_sanitize_llm_output(input_text, expected):
-    assert sanitize_llm_output(input_text) == expected

hindsight-api-slim/tests/test_lmstudio_tool_choice.py

@@ -1,321 +0,0 @@
-"""
-Reproduce issue #520: Reflect fails with LM Studio due to unsupported tool_choice format.
-
-The reflect agent forces tool selection via named tool_choice dicts on the first few iterations:
-  {"type": "function", "function": {"name": "search_mental_models"}}
-
-LM Studio (and Ollama) reject this format with HTTP 400:
-  "Tool choice of type 'function' is not supported. Use 'auto', 'none', or 'required'."
-
-The fix should convert named tool_choice to "required" and filter the tools list
-to only the requested tool for providers that don't support named tool_choice.
-"""
-
-import json
-from unittest.mock import AsyncMock, MagicMock, patch
-
-import pytest
-from openai import APIStatusError
-
-from hindsight_api.engine.providers.openai_compatible_llm import OpenAICompatibleLLM
-
-# Reflect agent tools (subset matching what agent.py uses)
-REFLECT_TOOLS = [
-    {
-        "type": "function",
-        "function": {
-            "name": "search_mental_models",
-            "description": "Search consolidated mental models",
-            "parameters": {
-                "type": "object",
-                "properties": {"query": {"type": "string"}},
-                "required": ["query"],
-            },
-        },
-    },
-    {
-        "type": "function",
-        "function": {
-            "name": "search_observations",
-            "description": "Search raw observations",
-            "parameters": {
-                "type": "object",
-                "properties": {"query": {"type": "string"}},
-                "required": ["query"],
-            },
-        },
-    },
-    {
-        "type": "function",
-        "function": {
-            "name": "recall",
-            "description": "Recall semantic memories",
-            "parameters": {
-                "type": "object",
-                "properties": {"query": {"type": "string"}},
-                "required": ["query"],
-            },
-        },
-    },
-    {
-        "type": "function",
-        "function": {
-            "name": "done",
-            "description": "Finish and return the answer",
-            "parameters": {
-                "type": "object",
-                "properties": {"answer": {"type": "string"}},
-                "required": ["answer"],
-            },
-        },
-    },
-]
-
-
-def _make_lmstudio_llm() -> OpenAICompatibleLLM:
-    return OpenAICompatibleLLM(
-        provider="lmstudio",
-        api_key="local",
-        base_url="http://localhost:1234/v1",
-        model="openai/gpt-oss-20b",
-    )
-
-
-def _lmstudio_400_error(msg: str = "Tool choice of type 'function' is not supported. Use 'auto', 'none', or 'required'.") -> APIStatusError:
-    """Simulate the HTTP 400 LM Studio returns for unsupported tool_choice format."""
-    mock_response = MagicMock()
-    mock_response.status_code = 400
-    mock_response.headers = {}
-    return APIStatusError(
-        message=msg,
-        response=mock_response,
-        body={"error": {"message": msg, "type": "invalid_request_error"}},
-    )
-
-
-def _make_tool_call_response(tool_name: str, arguments: dict) -> MagicMock:
-    """Build a mock successful tool call response from the LLM API."""
-    mock_tc = MagicMock()
-    mock_tc.id = "call_abc123"
-    mock_tc.function.name = tool_name
-    mock_tc.function.arguments = json.dumps(arguments)
-
-    mock_response = MagicMock()
-    mock_response.usage.prompt_tokens = 120
-    mock_response.usage.completion_tokens = 40
-    mock_response.usage.total_tokens = 160
-    mock_response.choices[0].finish_reason = "tool_calls"
-    mock_response.choices[0].message.content = None
-    mock_response.choices[0].message.tool_calls = [mock_tc]
-    return mock_response
-
-
-class TestLMStudioNamedToolChoiceBug:
-    """
-    Reproduces issue #520.
-
-    The reflect agent (agent.py lines 546-555) sets tool_choice to a named dict
-    on the first iterations to force sequential retrieval:
-
-        iteration=0, has_mental_models=True  → {"type": "function", "function": {"name": "search_mental_models"}}
-        iteration=0, has_mental_models=False → {"type": "function", "function": {"name": "search_observations"}}
-        iteration=1, has_mental_models=True  → {"type": "function", "function": {"name": "search_observations"}}
-        iteration=1 or (2 with models)       → {"type": "function", "function": {"name": "recall"}}
-
-    LM Studio rejects these dict formats with HTTP 400.
-    """
-
-    @pytest.mark.asyncio
-    async def test_lmstudio_named_tool_choice_no_longer_causes_400(self):
-        """
-        Regression test for issue #520: named tool_choice dict is converted to
-        "required" + filtered tools before the API call, so LM Studio never
-        sees the unsupported format and the 400 error no longer occurs.
-        """
-        llm = _make_lmstudio_llm()
-        named_tool_choice = {"type": "function", "function": {"name": "search_mental_models"}}
-        success_response = _make_tool_call_response("search_mental_models", {"query": "user name"})
-
-        with patch.object(llm._client.chat.completions, "create", new_callable=AsyncMock) as mock_create:
-            mock_create.return_value = success_response
-
-            # Should succeed — no 400 because the dict is converted before sending
-            result = await llm.call_with_tools(
-                messages=[{"role": "user", "content": "What is the user's name?"}],
-                tools=REFLECT_TOOLS,
-                tool_choice=named_tool_choice,
-                max_retries=0,
-            )
-
-        assert len(result.tool_calls) == 1
-        assert result.tool_calls[0].name == "search_mental_models"
-
-        sent_kwargs = mock_create.call_args.kwargs
-        assert sent_kwargs["tool_choice"] == "required"
-        assert len(sent_kwargs["tools"]) == 1
-        assert sent_kwargs["tools"][0]["function"]["name"] == "search_mental_models"
-
-    @pytest.mark.asyncio
-    @pytest.mark.parametrize(
-        "forced_tool_name",
-        ["search_mental_models", "search_observations", "recall"],
-    )
-    async def test_all_reflect_forced_tools_fail_on_lmstudio(self, forced_tool_name: str):
-        """
-        Each named tool_choice the reflect agent uses on iterations 0-2 triggers
-        the same 400 error on LM Studio.
-        """
-        llm = _make_lmstudio_llm()
-        named_tool_choice = {"type": "function", "function": {"name": forced_tool_name}}
-
-        with patch.object(llm._client.chat.completions, "create", new_callable=AsyncMock) as mock_create:
-            mock_create.side_effect = _lmstudio_400_error()
-
-            with pytest.raises(APIStatusError) as exc_info:
-                await llm.call_with_tools(
-                    messages=[{"role": "user", "content": "Test query"}],
-                    tools=REFLECT_TOOLS,
-                    tool_choice=named_tool_choice,
-                    max_retries=0,
-                )
-
-            assert exc_info.value.status_code == 400
-
-    @pytest.mark.asyncio
-    async def test_lmstudio_string_tool_choice_works_fine(self):
-        """
-        String tool_choice values ("auto", "none", "required") ARE supported by LM Studio.
-        Only the dict format {"type": "function", "function": {"name": "..."}} fails.
-        This test confirms the control case works.
-        """
-        llm = _make_lmstudio_llm()
-        success_response = _make_tool_call_response("search_mental_models", {"query": "user name"})
-
-        with patch.object(llm._client.chat.completions, "create", new_callable=AsyncMock) as mock_create:
-            mock_create.return_value = success_response
-
-            result = await llm.call_with_tools(
-                messages=[{"role": "user", "content": "What is the user's name?"}],
-                tools=REFLECT_TOOLS,
-                tool_choice="required",  # string form — LM Studio accepts this
-                max_retries=0,
-            )
-
-        assert len(result.tool_calls) == 1
-        assert result.tool_calls[0].name == "search_mental_models"
-
-        # Confirm "required" was sent, not a dict
-        sent_kwargs = mock_create.call_args.kwargs
-        assert sent_kwargs["tool_choice"] == "required"
-
-
-class TestExpectedFixBehavior:
-    """
-    Tests that document the EXPECTED behavior after the fix is applied.
-
-    For lmstudio (and ollama) providers, when tool_choice is a named dict:
-      {"type": "function", "function": {"name": "search_mental_models"}}
-
-    The fix should:
-      1. Convert tool_choice to "required"
-      2. Filter tools to only the requested tool
-
-    These tests currently FAIL (because the fix is not yet implemented).
-    After the fix is applied, they should PASS.
-    """
-
-    @pytest.mark.asyncio
-    async def test_fix_converts_named_tool_choice_to_required(self):
-        """
-        After fix: named tool_choice dict is converted to "required" for lmstudio.
-        The API receives tool_choice="required" instead of the unsupported dict.
-        """
-        llm = _make_lmstudio_llm()
-        named_tool_choice = {"type": "function", "function": {"name": "search_mental_models"}}
-        success_response = _make_tool_call_response("search_mental_models", {"query": "user name"})
-
-        with patch.object(llm._client.chat.completions, "create", new_callable=AsyncMock) as mock_create:
-            mock_create.return_value = success_response
-
-            result = await llm.call_with_tools(
-                messages=[{"role": "user", "content": "What is the user's name?"}],
-                tools=REFLECT_TOOLS,
-                tool_choice=named_tool_choice,
-                max_retries=0,
-            )
-
-        assert len(result.tool_calls) == 1
-        assert result.tool_calls[0].name == "search_mental_models"
-
-        sent_kwargs = mock_create.call_args.kwargs
-        # Fix: dict was converted to "required"
-        assert sent_kwargs["tool_choice"] == "required", (
-            f"Expected tool_choice='required', got {sent_kwargs['tool_choice']!r}"
-        )
-        # Fix: tools filtered to just the requested one
-        assert len(sent_kwargs["tools"]) == 1
-        assert sent_kwargs["tools"][0]["function"]["name"] == "search_mental_models"
-
-    @pytest.mark.asyncio
-    @pytest.mark.parametrize(
-        "forced_tool_name",
-        ["search_mental_models", "search_observations", "recall"],
-    )
-    async def test_fix_filters_tools_to_requested_tool(self, forced_tool_name: str):
-        """
-        After fix: tools list is filtered to only the forced tool so the model
-        can only call that one tool (equivalent to the named tool_choice behavior).
-        """
-        llm = _make_lmstudio_llm()
-        named_tool_choice = {"type": "function", "function": {"name": forced_tool_name}}
-        success_response = _make_tool_call_response(forced_tool_name, {"query": "test"})
-
-        with patch.object(llm._client.chat.completions, "create", new_callable=AsyncMock) as mock_create:
-            mock_create.return_value = success_response
-
-            await llm.call_with_tools(
-                messages=[{"role": "user", "content": "Test query"}],
-                tools=REFLECT_TOOLS,
-                tool_choice=named_tool_choice,
-                max_retries=0,
-            )
-
-        sent_kwargs = mock_create.call_args.kwargs
-        assert sent_kwargs["tool_choice"] == "required"
-        assert len(sent_kwargs["tools"]) == 1
-        assert sent_kwargs["tools"][0]["function"]["name"] == forced_tool_name
-
-    @pytest.mark.asyncio
-    async def test_fix_also_applies_to_openai_provider(self):
-        """
-        The fix is generalized: all providers convert named tool_choice to
-        "required" + filtered tools.  OpenAI natively supports the dict format
-        too, so the behaviour is semantically identical either way.
-        """
-        from hindsight_api.engine.providers.openai_compatible_llm import OpenAICompatibleLLM
-
-        openai_llm = OpenAICompatibleLLM(
-            provider="openai",
-            api_key="sk-test",
-            base_url="",
-            model="gpt-4o-mini",
-        )
-
-        named_tool_choice = {"type": "function", "function": {"name": "search_mental_models"}}
-        success_response = _make_tool_call_response("search_mental_models", {"query": "test"})
-
-        with patch.object(openai_llm._client.chat.completions, "create", new_callable=AsyncMock) as mock_create:
-            mock_create.return_value = success_response
-
-            await openai_llm.call_with_tools(
-                messages=[{"role": "user", "content": "Test"}],
-                tools=REFLECT_TOOLS,
-                tool_choice=named_tool_choice,
-                max_retries=0,
-            )
-
-        sent_kwargs = mock_create.call_args.kwargs
-        # Generalized fix applies to OpenAI too
-        assert sent_kwargs["tool_choice"] == "required"
-        assert len(sent_kwargs["tools"]) == 1
-        assert sent_kwargs["tools"][0]["function"]["name"] == "search_mental_models"

hindsight-api-slim/tests/test_mcp_string_coercion.py

@@ -1,330 +0,0 @@
-"""Tests for MCP tool argument string-to-JSON coercion (issue #849)."""
-
-import json
-from unittest.mock import AsyncMock, MagicMock
-
-import pytest
-
-from hindsight_api.api.mcp import (
-    _coerce_string_json,
-    _collect_coercible_types,
-    _get_mcp_tools,
-    _make_tools_tolerant,
-)
-
-
-# ---------------------------------------------------------------------------
-# _collect_coercible_types — schema type detection
-# ---------------------------------------------------------------------------
-
-
-class TestCollectCoercibleTypes:
-    """Tests for _collect_coercible_types schema detection."""
-
-    def _run(self, schema: dict, param_name: str = "p") -> tuple[set[str], set[str]]:
-        array_params: set[str] = set()
-        object_params: set[str] = set()
-        _collect_coercible_types(schema, param_name, array_params, object_params)
-        return array_params, object_params
-
-    # --- array types ---
-
-    def test_direct_array_type(self):
-        arrays, objects = self._run({"type": "array", "items": {"type": "string"}})
-        assert "p" in arrays and not objects
-
-    def test_anyof_nullable_array(self):
-        """list[str] | None → anyOf with array and null."""
-        arrays, objects = self._run(
-            {"anyOf": [{"type": "array", "items": {"type": "string"}}, {"type": "null"}]}
-        )
-        assert "p" in arrays
-
-    def test_oneof_nullable_array(self):
-        """oneOf variant."""
-        arrays, objects = self._run(
-            {"oneOf": [{"type": "array", "items": {"type": "string"}}, {"type": "null"}]}
-        )
-        assert "p" in arrays
-
-    # --- object types ---
-
-    def test_direct_object_type(self):
-        arrays, objects = self._run({"type": "object"})
-        assert "p" in objects and not arrays
-
-    def test_anyof_nullable_object(self):
-        """dict[str, str] | None → anyOf with object and null."""
-        arrays, objects = self._run({"anyOf": [{"type": "object"}, {"type": "null"}]})
-        assert "p" in objects
-
-    def test_oneof_nullable_object(self):
-        arrays, objects = self._run({"oneOf": [{"type": "object"}, {"type": "null"}]})
-        assert "p" in objects
-
-    # --- non-coercible types (should be ignored) ---
-
-    def test_string_type_ignored(self):
-        arrays, objects = self._run({"type": "string"})
-        assert not arrays and not objects
-
-    def test_integer_type_ignored(self):
-        arrays, objects = self._run({"type": "integer"})
-        assert not arrays and not objects
-
-    def test_number_type_ignored(self):
-        arrays, objects = self._run({"type": "number"})
-        assert not arrays and not objects
-
-    def test_boolean_type_ignored(self):
-        arrays, objects = self._run({"type": "boolean"})
-        assert not arrays and not objects
-
-    def test_null_type_ignored(self):
-        arrays, objects = self._run({"type": "null"})
-        assert not arrays and not objects
-
-    def test_anyof_string_or_null_ignored(self):
-        """str | None should not be collected."""
-        arrays, objects = self._run({"anyOf": [{"type": "string"}, {"type": "null"}]})
-        assert not arrays and not objects
-
-    def test_anyof_integer_or_null_ignored(self):
-        arrays, objects = self._run({"anyOf": [{"type": "integer"}, {"type": "null"}]})
-        assert not arrays and not objects
-
-
-# ---------------------------------------------------------------------------
-# _coerce_string_json — value coercion
-# ---------------------------------------------------------------------------
-
-
-class TestCoerceStringJson:
-    """Tests for _coerce_string_json argument coercion."""
-
-    # --- list coercion ---
-
-    def test_coerce_string_to_list(self):
-        result = _coerce_string_json(
-            {"tags": '["tag1", "tag2"]', "query": "hello"},
-            array_params={"tags"},
-            object_params=set(),
-        )
-        assert result["tags"] == ["tag1", "tag2"]
-        assert result["query"] == "hello"
-
-    def test_coerce_empty_list_string(self):
-        result = _coerce_string_json({"tags": "[]"}, array_params={"tags"}, object_params=set())
-        assert result["tags"] == []
-
-    def test_native_list_passthrough(self):
-        result = _coerce_string_json({"tags": ["a", "b"]}, array_params={"tags"}, object_params=set())
-        assert result["tags"] == ["a", "b"]
-
-    # --- dict coercion ---
-
-    def test_coerce_string_to_dict(self):
-        result = _coerce_string_json(
-            {"metadata": '{"key": "value"}'},
-            array_params=set(),
-            object_params={"metadata"},
-        )
-        assert result["metadata"] == {"key": "value"}
-
-    def test_coerce_empty_dict_string(self):
-        result = _coerce_string_json({"metadata": "{}"}, array_params=set(), object_params={"metadata"})
-        assert result["metadata"] == {}
-
-    def test_native_dict_passthrough(self):
-        result = _coerce_string_json(
-            {"metadata": {"key": "value"}}, array_params=set(), object_params={"metadata"}
-        )
-        assert result["metadata"] == {"key": "value"}
-
-    # --- non-coercible values left untouched ---
-
-    def test_none_passthrough(self):
-        result = _coerce_string_json({"tags": None}, array_params={"tags"}, object_params=set())
-        assert result["tags"] is None
-
-    def test_invalid_json_string_passthrough(self):
-        result = _coerce_string_json({"tags": "not-json"}, array_params={"tags"}, object_params=set())
-        assert result["tags"] == "not-json"
-
-    def test_wrong_json_type_not_coerced_list(self):
-        """String that parses to a dict should NOT be coerced for an array param."""
-        result = _coerce_string_json(
-            {"tags": '{"key": "value"}'}, array_params={"tags"}, object_params=set()
-        )
-        assert result["tags"] == '{"key": "value"}'
-
-    def test_wrong_json_type_not_coerced_dict(self):
-        """String that parses to a list should NOT be coerced for an object param."""
-        result = _coerce_string_json(
-            {"metadata": '["a", "b"]'}, array_params=set(), object_params={"metadata"}
-        )
-        assert result["metadata"] == '["a", "b"]'
-
-    def test_string_param_not_touched(self):
-        """Strings not in array_params/object_params are never modified."""
-        result = _coerce_string_json(
-            {"query": '["looks", "like", "json"]'},
-            array_params=set(),
-            object_params=set(),
-        )
-        assert result["query"] == '["looks", "like", "json"]'
-
-    def test_integer_param_not_touched(self):
-        result = _coerce_string_json(
-            {"max_tokens": 4096}, array_params=set(), object_params=set()
-        )
-        assert result["max_tokens"] == 4096
-
-    def test_boolean_param_not_touched(self):
-        result = _coerce_string_json(
-            {"verbose": True}, array_params=set(), object_params=set()
-        )
-        assert result["verbose"] is True
-
-    def test_missing_param_no_error(self):
-        result = _coerce_string_json(
-            {"query": "hello"},
-            array_params={"tags"},
-            object_params={"metadata"},
-        )
-        assert result == {"query": "hello"}
-
-    # --- multiple params coerced at once ---
-
-    def test_multiple_params_coerced(self):
-        result = _coerce_string_json(
-            {
-                "tags": '["a", "b"]',
-                "types": '["world"]',
-                "metadata": '{"source": "test"}',
-                "query": "hello",
-                "max_tokens": 4096,
-            },
-            array_params={"tags", "types"},
-            object_params={"metadata"},
-        )
-        assert result["tags"] == ["a", "b"]
-        assert result["types"] == ["world"]
-        assert result["metadata"] == {"source": "test"}
-        assert result["query"] == "hello"
-        assert result["max_tokens"] == 4096
-
-
-# ---------------------------------------------------------------------------
-# _make_tools_tolerant — integration test with a real FastMCP tool
-# ---------------------------------------------------------------------------
-
-
-class TestMakeToolsTolerantIntegration:
-    """Test that _make_tools_tolerant correctly wraps real FastMCP tool functions."""
-
-    def _create_mcp_with_tool(self):
-        """Create a FastMCP instance with a tool that uses various parameter types."""
-        from fastmcp import FastMCP
-
-        mcp = FastMCP("test")
-        captured = {}
-
-        @mcp.tool(description="test tool with diverse param types")
-        async def test_tool(
-            query: str,
-            max_tokens: int = 100,
-            verbose: bool = False,
-            tags: list[str] | None = None,
-            metadata: dict[str, str] | None = None,
-        ) -> dict:
-            """Test tool.
-
-            Args:
-                query: a string param
-                max_tokens: an integer param
-                verbose: a boolean param
-                tags: an array param
-                metadata: an object param
-            """
-            captured["query"] = query
-            captured["max_tokens"] = max_tokens
-            captured["verbose"] = verbose
-            captured["tags"] = tags
-            captured["metadata"] = metadata
-            return {"ok": True}
-
-        return mcp, captured
-
-    @pytest.mark.asyncio
-    async def test_coerces_string_encoded_list(self):
-        mcp, captured = self._create_mcp_with_tool()
-        _make_tools_tolerant(mcp)
-        tool = _get_mcp_tools(mcp)["test_tool"]
-        await tool.run({"query": "hi", "tags": '["a", "b"]'})
-        assert captured["tags"] == ["a", "b"]
-
-    @pytest.mark.asyncio
-    async def test_coerces_string_encoded_dict(self):
-        mcp, captured = self._create_mcp_with_tool()
-        _make_tools_tolerant(mcp)
-        tool = _get_mcp_tools(mcp)["test_tool"]
-        await tool.run({"query": "hi", "metadata": '{"k": "v"}'})
-        assert captured["metadata"] == {"k": "v"}
-
-    @pytest.mark.asyncio
-    async def test_native_types_pass_through(self):
-        mcp, captured = self._create_mcp_with_tool()
-        _make_tools_tolerant(mcp)
-        tool = _get_mcp_tools(mcp)["test_tool"]
-        await tool.run({
-            "query": "hi",
-            "max_tokens": 200,
-            "verbose": True,
-            "tags": ["x"],
-            "metadata": {"a": "b"},
-        })
-        assert captured["query"] == "hi"
-        assert captured["max_tokens"] == 200
-        assert captured["verbose"] is True
-        assert captured["tags"] == ["x"]
-        assert captured["metadata"] == {"a": "b"}
-
-    @pytest.mark.asyncio
-    async def test_strips_extra_args_and_coerces(self):
-        """Both extra-arg stripping and coercion work together."""
-        mcp, captured = self._create_mcp_with_tool()
-        _make_tools_tolerant(mcp)
-        tool = _get_mcp_tools(mcp)["test_tool"]
-        await tool.run({
-            "query": "hi",
-            "tags": '["x"]',
-            "explanation": "LLM added this",
-        })
-        assert captured["tags"] == ["x"]
-        assert "explanation" not in captured
-
-    @pytest.mark.asyncio
-    async def test_string_param_not_coerced(self):
-        """A string param whose value happens to look like JSON is NOT coerced."""
-        mcp, captured = self._create_mcp_with_tool()
-        _make_tools_tolerant(mcp)
-        tool = _get_mcp_tools(mcp)["test_tool"]
-        await tool.run({"query": '["this", "is", "a", "string"]'})
-        assert captured["query"] == '["this", "is", "a", "string"]'
-
-    @pytest.mark.asyncio
-    async def test_integer_param_not_coerced(self):
-        mcp, captured = self._create_mcp_with_tool()
-        _make_tools_tolerant(mcp)
-        tool = _get_mcp_tools(mcp)["test_tool"]
-        await tool.run({"query": "hi", "max_tokens": 50})
-        assert captured["max_tokens"] == 50
-
-    @pytest.mark.asyncio
-    async def test_boolean_param_not_coerced(self):
-        mcp, captured = self._create_mcp_with_tool()
-        _make_tools_tolerant(mcp)
-        tool = _get_mcp_tools(mcp)["test_tool"]
-        await tool.run({"query": "hi", "verbose": True})
-        assert captured["verbose"] is True

hindsight-api-slim/tests/test_mcp_tool_filtering.py

@@ -1,338 +0,0 @@
-"""Tests for filter_mcp_tools on OperationValidatorExtension."""
-
-from unittest.mock import AsyncMock, MagicMock
-
-import pytest
-
-from hindsight_api.api.mcp import (
-    _current_api_key,
-    _current_api_key_id,
-    _current_bank_id,
-    _current_mcp_authenticated,
-    _current_tenant_id,
-    create_mcp_server,
-)
-from hindsight_api.extensions.operation_validator import OperationValidatorExtension, ValidationResult
-from hindsight_api.models import RequestContext
-
-
-class MinimalValidator(OperationValidatorExtension):
-    """Minimal concrete subclass — only implements abstract methods."""
-
-    async def validate_retain(self, ctx):
-        return ValidationResult.accept()
-
-    async def validate_recall(self, ctx):
-        return ValidationResult.accept()
-
-    async def validate_reflect(self, ctx):
-        return ValidationResult.accept()
-
-
-class FilteringValidator(OperationValidatorExtension):
-    """Validator that removes retain from the tool set."""
-
-    async def validate_retain(self, ctx):
-        return ValidationResult.accept()
-
-    async def validate_recall(self, ctx):
-        return ValidationResult.accept()
-
-    async def validate_reflect(self, ctx):
-        return ValidationResult.accept()
-
-    async def filter_mcp_tools(self, bank_id, request_context, tools):
-        return tools - {"retain"}
-
-
-@pytest.mark.asyncio
-async def test_filter_mcp_tools_default_returns_all():
-    """Default implementation returns the input unchanged."""
-    validator = MinimalValidator({})
-    tools = frozenset({"retain", "recall", "reflect", "list_memories"})
-    ctx = RequestContext()
-
-    result = await validator.filter_mcp_tools("test-bank", ctx, tools)
-
-    assert result == tools
-    assert isinstance(result, frozenset)
-
-
-@pytest.mark.asyncio
-async def test_filter_mcp_tools_subclass_removes_tools():
-    """Subclass can remove tools from the set."""
-    validator = FilteringValidator({})
-    tools = frozenset({"retain", "recall", "reflect"})
-    ctx = RequestContext()
-
-    result = await validator.filter_mcp_tools("test-bank", ctx, tools)
-
-    assert result == frozenset({"recall", "reflect"})
-    assert "retain" not in result
-
-
-@pytest.mark.asyncio
-async def test_filter_mcp_tools_returns_empty_set():
-    """Validator can return empty set — no tools visible."""
-
-    class DenyAllValidator(OperationValidatorExtension):
-        async def validate_retain(self, ctx):
-            return ValidationResult.accept()
-        async def validate_recall(self, ctx):
-            return ValidationResult.accept()
-        async def validate_reflect(self, ctx):
-            return ValidationResult.accept()
-        async def filter_mcp_tools(self, bank_id, request_context, tools):
-            return frozenset()
-
-    validator = DenyAllValidator({})
-    tools = frozenset({"retain", "recall", "reflect"})
-    ctx = RequestContext()
-
-    result = await validator.filter_mcp_tools("test-bank", ctx, tools)
-
-    assert result == frozenset()
-    assert len(result) == 0
-
-
-@pytest.mark.asyncio
-async def test_validator_filters_tools_list():
-    """Validator filter is applied during tools/list via _get_enabled_tools."""
-    mock_memory = MagicMock()
-    mock_memory._tenant_extension = MagicMock()
-    mock_memory._tenant_extension.authenticate_mcp = AsyncMock()
-    mock_memory.retain_batch_async = AsyncMock()
-    mock_memory.submit_async_retain = AsyncMock()
-    mock_memory.recall_async = AsyncMock()
-    mock_memory.reflect_async = AsyncMock()
-    mock_memory.list_banks = AsyncMock(return_value=[])
-
-    validator = FilteringValidator({})
-    mock_memory._operation_validator = validator
-
-    mock_config = {"mcp_enabled_tools": None}
-    mock_memory._config_resolver = MagicMock()
-    mock_memory._config_resolver.get_bank_config = AsyncMock(return_value=mock_config)
-
-    mcp_server = create_mcp_server(mock_memory, multi_bank=False)
-
-    bank_token = _current_bank_id.set("test-bank")
-    api_key_token = _current_api_key.set("hsk_test_key")
-    tenant_token = _current_tenant_id.set("alice")
-    key_id_token = _current_api_key_id.set("key-uuid")
-    mcp_auth_token = _current_mcp_authenticated.set(False)
-    try:
-        if hasattr(mcp_server, "list_tools"):
-            tools = await mcp_server.list_tools()
-            tool_names = {t.name for t in tools}
-        else:
-            tools = await mcp_server._tool_manager.get_tools()
-            tool_names = set(tools.keys())
-
-        assert "recall" in tool_names
-        assert "reflect" in tool_names
-        assert "retain" not in tool_names
-    finally:
-        _current_bank_id.reset(bank_token)
-        _current_api_key.reset(api_key_token)
-        _current_tenant_id.reset(tenant_token)
-        _current_api_key_id.reset(key_id_token)
-        _current_mcp_authenticated.reset(mcp_auth_token)
-
-
-@pytest.mark.asyncio
-async def test_bank_config_and_validator_compose():
-    """Bank config sets ceiling, validator narrows further."""
-    mock_memory = MagicMock()
-    mock_memory._tenant_extension = MagicMock()
-    mock_memory._tenant_extension.authenticate_mcp = AsyncMock()
-    mock_memory.retain_batch_async = AsyncMock()
-    mock_memory.submit_async_retain = AsyncMock()
-    mock_memory.recall_async = AsyncMock()
-    mock_memory.reflect_async = AsyncMock()
-    mock_memory.list_banks = AsyncMock(return_value=[])
-
-    mock_memory._operation_validator = FilteringValidator({})
-
-    mock_config = {"mcp_enabled_tools": ["recall", "retain", "reflect"]}
-    mock_memory._config_resolver = MagicMock()
-    mock_memory._config_resolver.get_bank_config = AsyncMock(return_value=mock_config)
-
-    mcp_server = create_mcp_server(mock_memory, multi_bank=False)
-
-    bank_token = _current_bank_id.set("test-bank")
-    api_key_token = _current_api_key.set("hsk_test")
-    tenant_token = _current_tenant_id.set("alice")
-    key_id_token = _current_api_key_id.set("key-1")
-    mcp_auth_token = _current_mcp_authenticated.set(False)
-    try:
-        if hasattr(mcp_server, "list_tools"):
-            tools = await mcp_server.list_tools()
-            tool_names = {t.name for t in tools}
-        else:
-            tools = await mcp_server._tool_manager.get_tools()
-            tool_names = set(tools.keys())
-
-        assert tool_names == {"recall", "reflect"}
-    finally:
-        _current_bank_id.reset(bank_token)
-        _current_api_key.reset(api_key_token)
-        _current_tenant_id.reset(tenant_token)
-        _current_api_key_id.reset(key_id_token)
-        _current_mcp_authenticated.reset(mcp_auth_token)
-
-
-@pytest.mark.asyncio
-async def test_validator_cannot_add_tools_beyond_bank_config():
-    """Validator returning tools not in bank config doesn't expand the set."""
-
-    class PermissiveValidator(OperationValidatorExtension):
-        async def validate_retain(self, ctx):
-            return ValidationResult.accept()
-        async def validate_recall(self, ctx):
-            return ValidationResult.accept()
-        async def validate_reflect(self, ctx):
-            return ValidationResult.accept()
-        async def filter_mcp_tools(self, bank_id, request_context, tools):
-            return tools | {"retain", "delete_bank"}
-
-    mock_memory = MagicMock()
-    mock_memory._tenant_extension = MagicMock()
-    mock_memory._tenant_extension.authenticate_mcp = AsyncMock()
-    mock_memory.retain_batch_async = AsyncMock()
-    mock_memory.submit_async_retain = AsyncMock()
-    mock_memory.recall_async = AsyncMock()
-    mock_memory.reflect_async = AsyncMock()
-    mock_memory.list_banks = AsyncMock(return_value=[])
-
-    mock_memory._operation_validator = PermissiveValidator({})
-
-    mock_config = {"mcp_enabled_tools": ["recall"]}
-    mock_memory._config_resolver = MagicMock()
-    mock_memory._config_resolver.get_bank_config = AsyncMock(return_value=mock_config)
-
-    mcp_server = create_mcp_server(mock_memory, multi_bank=False)
-
-    bank_token = _current_bank_id.set("test-bank")
-    api_key_token = _current_api_key.set("hsk_test")
-    tenant_token = _current_tenant_id.set("alice")
-    key_id_token = _current_api_key_id.set("key-1")
-    mcp_auth_token = _current_mcp_authenticated.set(False)
-    try:
-        if hasattr(mcp_server, "list_tools"):
-            tools = await mcp_server.list_tools()
-            tool_names = {t.name for t in tools}
-        else:
-            tools = await mcp_server._tool_manager.get_tools()
-            tool_names = set(tools.keys())
-
-        assert "recall" in tool_names
-        assert "retain" not in tool_names
-        assert "delete_bank" not in tool_names
-    finally:
-        _current_bank_id.reset(bank_token)
-        _current_api_key.reset(api_key_token)
-        _current_tenant_id.reset(tenant_token)
-        _current_api_key_id.reset(key_id_token)
-        _current_mcp_authenticated.reset(mcp_auth_token)
-
-
-@pytest.mark.asyncio
-async def test_validator_exception_fails_open(caplog):
-    """If filter_mcp_tools raises, all tools remain visible and warning is logged."""
-    import logging
-    caplog.set_level(logging.WARNING)
-
-    class BrokenValidator(OperationValidatorExtension):
-        async def validate_retain(self, ctx):
-            return ValidationResult.accept()
-        async def validate_recall(self, ctx):
-            return ValidationResult.accept()
-        async def validate_reflect(self, ctx):
-            return ValidationResult.accept()
-        async def filter_mcp_tools(self, bank_id, request_context, tools):
-            raise RuntimeError("Policy backend unreachable")
-
-    mock_memory = MagicMock()
-    mock_memory._tenant_extension = MagicMock()
-    mock_memory._tenant_extension.authenticate_mcp = AsyncMock()
-    mock_memory.retain_batch_async = AsyncMock()
-    mock_memory.submit_async_retain = AsyncMock()
-    mock_memory.recall_async = AsyncMock()
-    mock_memory.reflect_async = AsyncMock()
-    mock_memory.list_banks = AsyncMock(return_value=[])
-
-    mock_memory._operation_validator = BrokenValidator({})
-    mock_config = {"mcp_enabled_tools": None}
-    mock_memory._config_resolver = MagicMock()
-    mock_memory._config_resolver.get_bank_config = AsyncMock(return_value=mock_config)
-
-    mcp_server = create_mcp_server(mock_memory, multi_bank=False)
-
-    bank_token = _current_bank_id.set("test-bank")
-    api_key_token = _current_api_key.set("hsk_test")
-    tenant_token = _current_tenant_id.set("alice")
-    key_id_token = _current_api_key_id.set("key-1")
-    mcp_auth_token = _current_mcp_authenticated.set(False)
-    try:
-        if hasattr(mcp_server, "list_tools"):
-            tools = await mcp_server.list_tools()
-            tool_names = {t.name for t in tools}
-        else:
-            tools = await mcp_server._tool_manager.get_tools()
-            tool_names = set(tools.keys())
-
-        assert "retain" in tool_names
-        assert "recall" in tool_names
-        assert "reflect" in tool_names
-
-        assert any("filter_mcp_tools raised" in r.message for r in caplog.records)
-    finally:
-        _current_bank_id.reset(bank_token)
-        _current_api_key.reset(api_key_token)
-        _current_tenant_id.reset(tenant_token)
-        _current_api_key_id.reset(key_id_token)
-        _current_mcp_authenticated.reset(mcp_auth_token)
-
-
-@pytest.mark.asyncio
-async def test_no_validator_returns_unfiltered():
-    """Without an operation validator, tools/list returns all tools."""
-    mock_memory = MagicMock()
-    mock_memory._tenant_extension = MagicMock()
-    mock_memory._tenant_extension.authenticate_mcp = AsyncMock()
-    mock_memory.retain_batch_async = AsyncMock()
-    mock_memory.submit_async_retain = AsyncMock()
-    mock_memory.recall_async = AsyncMock()
-    mock_memory.reflect_async = AsyncMock()
-    mock_memory.list_banks = AsyncMock(return_value=[])
-
-    mock_memory._operation_validator = None
-    mock_config = {"mcp_enabled_tools": None}
-    mock_memory._config_resolver = MagicMock()
-    mock_memory._config_resolver.get_bank_config = AsyncMock(return_value=mock_config)
-
-    mcp_server = create_mcp_server(mock_memory, multi_bank=False)
-
-    bank_token = _current_bank_id.set("test-bank")
-    api_key_token = _current_api_key.set("hsk_test")
-    tenant_token = _current_tenant_id.set("alice")
-    key_id_token = _current_api_key_id.set("key-1")
-    mcp_auth_token = _current_mcp_authenticated.set(False)
-    try:
-        if hasattr(mcp_server, "list_tools"):
-            tools = await mcp_server.list_tools()
-            tool_names = {t.name for t in tools}
-        else:
-            tools = await mcp_server._tool_manager.get_tools()
-            tool_names = set(tools.keys())
-
-        assert "retain" in tool_names
-        assert "recall" in tool_names
-        assert "reflect" in tool_names
-    finally:
-        _current_bank_id.reset(bank_token)
-        _current_api_key.reset(api_key_token)
-        _current_tenant_id.reset(tenant_token)
-        _current_api_key_id.reset(key_id_token)
-        _current_mcp_authenticated.reset(mcp_auth_token)