fix: normalize named tool_choice to required + filtered tools for OpenAI-compatible providers

LM Studio (and Ollama) reject the named tool_choice dict format
{"type": "function", "function": {"name": "..."}} with HTTP 400.

The reflect agent uses this format on iterations 0-2 to force sequential
tool selection, causing reflect to fail entirely on LM Studio.

The fix converts named tool_choice dicts to tool_choice="required" with
the tools list filtered to just the requested tool — semantically identical
and accepted by all providers including LM Studio and Ollama.

Closes #520

.claude-plugin/marketplace.json

@@ -1,15 +0,0 @@
-{
-  "$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
-  "name": "hindsight",
-  "description": "Official Hindsight integrations for Claude Code",
-  "owner": {
-    "name": "vectorize-io"
-  },
-  "plugins": [
-    {
-      "name": "hindsight-memory",
-      "description": "Automatic long-term memory for Claude Code via Hindsight",
-      "source": "./hindsight-integrations/claude-code"
-    }
-  ]
-}

.claude/skills/code-review/SKILL.md

@@ -1,205 +0,0 @@
----
-name: code-review
-description: Review changed code against project standards. Checks for missing tests, dead code, type safety, lint issues, and coding conventions. Run after completing any implementation work.
-user_invocable: true
----
-
-# Code Review
-
-Review all changed code against the project's quality standards and coding conventions.
-
-## Code Standards
-
-Read and internalize these standards before writing code. The review steps below verify compliance.
-
-### Python Style
-- Python 3.11+, type hints required
-- Async throughout (asyncpg, async FastAPI)
-- Pydantic models for request/response
-- Ruff for linting (line-length 120)
-- No Python files at project root - maintain clean directory structure
-- **Never use multi-item tuple return values** — not even for internal/private functions. Always use a dataclass or Pydantic model. No exceptions, no "it's just two values" shortcuts. If a function returns more than one value, define a named type for it.
-
-### Type Safety with Pydantic Models
-**NEVER use raw `dict` types for structured data** — this applies to all code, including internal helpers and private functions. If the dict has known keys, it must be a dataclass or Pydantic model:
-- Use Pydantic `BaseModel` for all data structures passed between functions
-- Use `@dataclass` for lightweight internal data containers when Pydantic validation isn't needed
-- Add `@field_validator` for type coercion (e.g., ensuring datetimes are timezone-aware)
-- Avoid `dict.get()` patterns - use typed model attributes instead
-- Parse external data (JSON, API responses) into Pydantic models at the boundary
-- This catches type errors at parse time, not deep in business logic
-- The only acceptable `dict` usage is for truly dynamic/unknown keys (e.g., arbitrary metadata, JSON blobs with no fixed schema)
-
-```python
-# BAD - error-prone dict access
-def process(data: dict) -> str:
-    return data.get("name", "")  # No validation, silent failures
-
-# GOOD - typed and validated
-class UserData(BaseModel):
-    name: str
-    created_at: datetime
-
-    @field_validator("created_at", mode="before")
-    @classmethod
-    def ensure_tz_aware(cls, v):
-        if isinstance(v, str):
-            v = datetime.fromisoformat(v.replace("Z", "+00:00"))
-        if v.tzinfo is None:
-            return v.replace(tzinfo=timezone.utc)
-        return v
-
-def process(data: UserData) -> str:
-    return data.name  # Type-safe, validated at construction
-```
-
-### TypeScript Style
-- Next.js App Router for control plane
-- Tailwind CSS with shadcn/ui components
-
-### Code Comments
-- **Always comment non-trivial technical decisions** with the reasoning behind the choice. If someone would ask "why is it done this way?", there should be a comment.
-- **Keep comments up to date with history** — when changing an approach, update the comment to explain what was tried before and why it was changed. Comments serve as a tracker of previous implementations that likely had problems.
-- Don't comment obvious code — only where the "why" isn't self-evident from the code itself.
-
-```python
-# BAD - no context for future readers
-results = await asyncio.gather(*tasks, return_exceptions=True)
-
-# GOOD - explains the non-obvious choice
-# Use return_exceptions=True to avoid cancelling sibling tasks on failure.
-# Previously we used TaskGroup but it cancelled all tasks when one failed,
-# causing partial writes that left orphaned entity links (see #412).
-results = await asyncio.gather(*tasks, return_exceptions=True)
-```
-
-### Branch Hygiene
-- **Always start new feature branches from `origin/main`** — rebase to ensure a clean base.
-- **Only include commits relevant to the PR/branch/feature** — no unrelated changes. If the branch contains commits that don't belong, they must be removed before merging.
-
-### General Principles
-- Don't add features, refactor code, or make "improvements" beyond what was asked
-- Don't add unnecessary error handling for impossible scenarios
-- Don't create helpers or abstractions for one-time operations
-- No backwards-compatibility hacks (unused vars, re-exports, "removed" comments)
-- Three similar lines of code is better than a premature abstraction
-
-## Review Steps
-
-### 1. Check branch hygiene
-
-- Run `git log --oneline main..HEAD` to list all commits on the branch.
-- Verify every commit is relevant to the feature/PR. Flag any unrelated commits.
-- Check the branch is based on a recent `origin/main` (no stale base).
-
-### 2. Identify changed files
-
-Run `git diff --name-only HEAD` (unstaged) and `git diff --cached --name-only` (staged) to get all changed files. If there are no local changes, diff against the base branch using `git diff main...HEAD --name-only` and `git diff main...HEAD` to review all commits on the current branch.
-
-### 3. Run linters
-
-```bash
-./scripts/hooks/lint.sh
-```
-
-Report any failures. Do NOT fix them yourself — just report.
-
-### 4. Check for dead code
-
-For each changed Python file, check for:
-- Unused imports (Ruff should catch these, but verify)
-- Functions/methods/classes that were added but are never called from anywhere
-- Variables assigned but never read
-- Commented-out code blocks that should be removed
-
-For each changed TypeScript file, check for:
-- Unused imports
-- Unused variables or functions
-- Commented-out code
-
-### 5. Check type safety (Python)
-
-For each changed Python file, check for violations:
-- **No raw `dict` for structured data** — must use Pydantic model or dataclass, even for internal/private functions (only exception: truly dynamic/unknown keys)
-- **No multi-item tuple returns** — must use dataclass or Pydantic model, even for internal/private functions (no exceptions)
-- **Missing type hints** on function parameters and return types
-- **Missing `@field_validator`** for datetime fields that should be timezone-aware
-
-### 6. Check for missing tests
-
-For each new or significantly changed function/endpoint/class:
-- Check if there is a corresponding test addition or update
-- New API endpoints MUST have integration tests
-- New utility functions MUST have unit tests
-- Bug fixes SHOULD have a regression test
-
-Flag any new logic that lacks test coverage.
-
-### 7. Check API consistency
-
-If any files in `hindsight-api-slim/hindsight_api/api/` were changed:
-- Were the OpenAPI specs regenerated? (`./scripts/generate-openapi.sh`)
-- Were the client SDKs regenerated? (`./scripts/generate-clients.sh`)
-- Were the control plane proxy routes updated? (`hindsight-control-plane/src/app/api/`)
-
-### 8. Check code comments
-
-For each non-trivial change:
-- **New non-obvious logic** — is there a comment explaining the reasoning?
-- **Changed approach** — does the comment include what was done before and why it changed?
-- **Stale comments** — do existing comments near the changed code still accurately describe the behavior?
-
-### 9. Check integration completeness
-
-If any files in `hindsight-integrations/` were added or changed, verify:
-- **Tests exist** — the integration must have tests that simulate/exercise the external framework (not just pure unit tests of helpers). Check for a `tests/` directory with meaningful test files.
-- **CI job exists** — check `.github/workflows/test.yml` for a corresponding `test-<name>-integration` job. If missing, flag it.
-- **Release process** — check that the integration name is in the `VALID_INTEGRATIONS` array in `scripts/release-integration.sh`. If missing, flag it.
-- **Code standards** — the integration code must follow all Python style rules (type hints, no raw dicts, no tuple returns, etc.).
-
-### 10. Check MCP tool registration completeness
-
-If any new MCP tools were added or existing tools renamed in `hindsight-api-slim/hindsight_api/mcp_tools.py`:
-- **`_ALL_TOOLS` set** in `mcp_tools.py` — must include the new tool name
-- **`tools_to_register` default set** in `register_mcp_tools()` in `mcp_tools.py` — must include the new tool name
-- **`_SINGLE_BANK_TOOLS` set** in `hindsight-api-slim/hindsight_api/api/mcp.py` — must include the new tool if it is bank-scoped (not a bank-management tool like `list_banks`/`create_bank`)
-- **`MCP_TOOL_GROUPS`** in `hindsight-control-plane/src/components/bank-config-view.tsx` — must include the new tool in the appropriate group for the UI tool selector
-- **Tool count assertions** in tests (e.g., `test_mcp_tools.py`) — must be updated to reflect the new count
-
-### 11. Review against other coding standards
-
-Check the diff for violations of the standards listed above:
-- Python files at project root (not allowed)
-- Missing async patterns (should be async throughout)
-- Pydantic models for request/response
-- Line length > 120 chars
-- New features/code beyond what was asked (over-engineering)
-- Unnecessary error handling for impossible scenarios
-- Premature abstractions or speculative helpers
-- Backwards-compatibility hacks (unused vars, re-exports, "removed" comments)
-
-### 12. Report findings
-
-Present a clear summary organized by severity:
-
-**Must fix** — issues that will break CI or violate hard project rules:
-- Unrelated commits on the branch
-- Lint failures
-- Missing type hints on public functions
-- Raw dict usage for structured data (including internal code)
-- Multi-item tuple returns (including internal code)
-- Missing tests for new endpoints
-- New integration missing tests, CI job, or release-integration.sh entry
-
-**Should fix** — issues that hurt code quality:
-- Dead code / unused imports missed by linter
-- Missing tests for non-trivial utility functions
-- Over-engineering beyond the task scope
-
-**Note** — observations that may or may not need action:
-- API changes that might need client regeneration
-- Patterns that deviate from nearby code style
-
-For each finding, include the file path, line number, and a brief explanation.
-
-Do NOT auto-fix any issues. Report all findings and let the user decide what to address. If there are no findings, confirm the code looks good.

.env.example

@@ -2,7 +2,7 @@
 # Copy this file to .env and fill in your values
 
 # LLM Configuration (Required)
-# Supported providers: openai, groq, ollama, gemini, anthropic, lmstudio, vertexai, minimax, volcano
+# Supported providers: openai, groq, ollama, gemini, anthropic, lmstudio, vertexai
 HINDSIGHT_API_LLM_PROVIDER=openai
 HINDSIGHT_API_LLM_API_KEY=your-api-key-here
 HINDSIGHT_API_LLM_MODEL=gpt-4o-mini
@@ -20,11 +20,6 @@ HINDSIGHT_API_LLM_BASE_URL=https://api.openai.com/v1
 # HINDSIGHT_API_LLM_VERTEXAI_REGION=us-central1
 # HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY=/path/to/service-account-key.json  # Optional, uses ADC if not set
 
-# Example: MiniMax configuration (1M context window)
-# HINDSIGHT_API_LLM_PROVIDER=minimax
-# HINDSIGHT_API_LLM_API_KEY=your-minimax-api-key
-# HINDSIGHT_API_LLM_MODEL=MiniMax-M2.7
-
 # Example: LM Studio local configuration (Qwen 2.5 32B recommended)
 # HINDSIGHT_API_LLM_PROVIDER=lmstudio
 # HINDSIGHT_API_LLM_API_KEY=lmstudio
@@ -44,7 +39,6 @@ HINDSIGHT_API_LOG_LEVEL=info
 
 # Database (Optional - uses embedded pg0 by default)
 # HINDSIGHT_API_DATABASE_URL=postgresql://user:pass@host:5432/db
-# HINDSIGHT_API_MIGRATION_DATABASE_URL=  # Direct PostgreSQL URL for migrations (bypasses PgBouncer). Falls back to DATABASE_URL.
 # HINDSIGHT_API_DATABASE_SCHEMA=public  # PostgreSQL schema name (default: public)
 
 # Vector Extension (Optional - uses pgvector by default)

.github/dependabot.yml

@@ -1,6 +0,0 @@
-version: 2
-updates:
-  - package-ecosystem: "github-actions"
-    directory: "/"
-    schedule:
-      interval: "weekly"

.github/workflows/deploy-docs.yml

@@ -21,20 +21,20 @@ jobs:
   build:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v6
-      - uses: actions/setup-node@v6
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
         with:
           node-version: 20
           cache: npm
           cache-dependency-path: package-lock.json
-      - uses: astral-sh/setup-uv@v7
+      - uses: astral-sh/setup-uv@v4
       - run: npm ci --workspace=hindsight-docs
       - run: uv run generate-llms-full
       - run: npm run build --workspace=hindsight-docs
         env:
           UMAMI_URL: https://analytics.hindsight.vectorize.io
           UMAMI_WEBSITE_ID: ${{ secrets.UMAMI_WEBSITE_ID }}
-      - uses: actions/upload-pages-artifact@v4
+      - uses: actions/upload-pages-artifact@v3
         with:
           path: hindsight-docs/build
   deploy:
@@ -44,5 +44,5 @@ jobs:
     runs-on: ubuntu-latest
     needs: build
     steps:
-      - uses: actions/deploy-pages@v5
+      - uses: actions/deploy-pages@v4
         id: deployment

.github/workflows/release-integration.yml

@@ -1,120 +0,0 @@
-name: Release Integration
-
-on:
-  push:
-    tags:
-      - 'integrations/**'
-
-jobs:
-  publish:
-    runs-on: ubuntu-latest
-    permissions:
-      id-token: write  # for PyPI trusted publishing
-
-    steps:
-    - uses: actions/checkout@v6
-
-    - name: Extract integration info
-      id: info
-      run: |
-        # refs/tags/integrations/litellm/v0.1.0 → integration=litellm, version=0.1.0
-        TAG="${GITHUB_REF#refs/tags/}"
-        INTEGRATION=$(echo "$TAG" | cut -d'/' -f2)
-        VERSION=$(echo "$TAG" | cut -d'/' -f3 | sed 's/^v//')
-        echo "integration=$INTEGRATION" >> $GITHUB_OUTPUT
-        echo "version=$VERSION" >> $GITHUB_OUTPUT
-        echo "tag=$TAG" >> $GITHUB_OUTPUT
-        echo "Integration: $INTEGRATION, Version: $VERSION"
-
-    - name: Detect integration type
-      id: type
-      run: |
-        if [ -f "hindsight-integrations/${{ steps.info.outputs.integration }}/pyproject.toml" ]; then
-          echo "type=python" >> $GITHUB_OUTPUT
-        elif [ -f "hindsight-integrations/${{ steps.info.outputs.integration }}/package.json" ]; then
-          echo "type=typescript" >> $GITHUB_OUTPUT
-        else
-          echo "type=plugin" >> $GITHUB_OUTPUT
-        fi
-
-    # ── Python integrations (litellm, pydantic-ai, crewai) ──────────────────
-
-    - name: Install uv
-      if: steps.type.outputs.type == 'python'
-      uses: astral-sh/setup-uv@v7
-      with:
-        enable-cache: true
-
-    - name: Set up Python
-      if: steps.type.outputs.type == 'python'
-      uses: actions/setup-python@v6
-      with:
-        python-version-file: ".python-version"
-
-    - name: Build Python package
-      if: steps.type.outputs.type == 'python'
-      working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
-      run: uv build --out-dir dist
-
-    - name: Publish Python package to PyPI
-      if: steps.type.outputs.type == 'python'
-      uses: pypa/gh-action-pypi-publish@release/v1
-      with:
-        packages-dir: ./hindsight-integrations/${{ steps.info.outputs.integration }}/dist
-        skip-existing: true
-
-    # ── TypeScript integrations (ai-sdk, chat, openclaw) ────────────────────
-
-    # ── Plugin integrations (claude-code) — no package to publish ───────────
-
-    - name: Plugin release
-      if: steps.type.outputs.type == 'plugin'
-      run: |
-        echo "Plugin integration ${{ steps.info.outputs.integration }} v${{ steps.info.outputs.version }} — no package to publish."
-        echo "Users install via: claude plugin marketplace add vectorize-io/hindsight --sparse hindsight-integrations"
-
-    # ── TypeScript integrations (ai-sdk, chat, openclaw) ────────────────────
-
-    - name: Set up Node.js
-      if: steps.type.outputs.type == 'typescript'
-      uses: actions/setup-node@v6
-      with:
-        node-version: '22'
-        registry-url: 'https://registry.npmjs.org'
-
-    # Guard: fail fast if the integration's lockfile resolves any dep from a
-    # monorepo workspace (link=true) or a relative file path. The release
-    # runner has no pre-built workspace `dist/` so `npm run build` would
-    # later fail at tsc with "Cannot find module". See:
-    # https://github.com/vectorize-io/hindsight/issues/… (0.6.0 openclaw retry)
-    - name: Check integration lockfile
-      if: steps.type.outputs.type == 'typescript'
-      run: ./scripts/check-integration-lockfiles.sh
-
-    - name: Install dependencies
-      if: steps.type.outputs.type == 'typescript'
-      working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
-      run: npm ci
-
-    - name: Build TypeScript package
-      if: steps.type.outputs.type == 'typescript'
-      working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
-      run: npm run build
-
-    - name: Publish TypeScript package to npm
-      if: steps.type.outputs.type == 'typescript'
-      working-directory: ./hindsight-integrations/${{ steps.info.outputs.integration }}
-      run: |
-        set +e
-        OUTPUT=$(npm publish --access public 2>&1)
-        EXIT_CODE=$?
-        echo "$OUTPUT"
-        if [ $EXIT_CODE -ne 0 ]; then
-          if echo "$OUTPUT" | grep -q "cannot publish over"; then
-            echo "Package version already published, skipping..."
-            exit 0
-          fi
-          exit $EXIT_CODE
-        fi
-      env:
-        NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

.github/workflows/release.yml

@@ -13,15 +13,15 @@ jobs:
       id-token: write
 
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v4
 
     - name: Install uv
-      uses: astral-sh/setup-uv@v7
+      uses: astral-sh/setup-uv@v5
       with:
         enable-cache: true
 
     - name: Set up Python
-      uses: actions/setup-python@v6
+      uses: actions/setup-python@v5
       with:
         python-version-file: ".python-version"
 
@@ -30,39 +30,37 @@ jobs:
       working-directory: ./hindsight-clients/python
       run: uv build --out-dir dist
 
-    - name: Build hindsight-api-slim
-      working-directory: ./hindsight-api-slim
-      run: uv build --out-dir dist
-
     - name: Build hindsight-api
       working-directory: ./hindsight-api
       run: uv build --out-dir dist
 
     - name: Build hindsight-all
-      working-directory: ./hindsight-all
+      working-directory: ./hindsight
       run: uv build --out-dir dist
 
-    - name: Build hindsight-all-slim
-      working-directory: ./hindsight-all-slim
+    - name: Build hindsight-litellm
+      working-directory: ./hindsight-integrations/litellm
       run: uv build --out-dir dist
 
     - name: Build hindsight-embed
       working-directory: ./hindsight-embed
       run: uv build --out-dir dist
 
-    # Publish in order (client and api-slim first, then api/all wrappers which depend on them)
+    - name: Build hindsight-crewai
+      working-directory: ./hindsight-integrations/crewai
+      run: uv build --out-dir dist
+
+    - name: Build hindsight-pydantic-ai
+      working-directory: ./hindsight-integrations/pydantic-ai
+      run: uv build --out-dir dist
+
+    # Publish in order (client and api first, then hindsight-all which depends on them)
     - name: Publish hindsight-client to PyPI
       uses: pypa/gh-action-pypi-publish@release/v1
       with:
         packages-dir: ./hindsight-clients/python/dist
         skip-existing: true
 
-    - name: Publish hindsight-api-slim to PyPI
-      uses: pypa/gh-action-pypi-publish@release/v1
-      with:
-        packages-dir: ./hindsight-api-slim/dist
-        skip-existing: true
-
     - name: Publish hindsight-api to PyPI
       uses: pypa/gh-action-pypi-publish@release/v1
       with:
@@ -72,13 +70,13 @@ jobs:
     - name: Publish hindsight-all to PyPI
       uses: pypa/gh-action-pypi-publish@release/v1
       with:
-        packages-dir: ./hindsight-all/dist
+        packages-dir: ./hindsight/dist
         skip-existing: true
 
-    - name: Publish hindsight-all-slim to PyPI
+    - name: Publish hindsight-litellm to PyPI
       uses: pypa/gh-action-pypi-publish@release/v1
       with:
-        packages-dir: ./hindsight-all-slim/dist
+        packages-dir: ./hindsight-integrations/litellm/dist
         skip-existing: true
 
     - name: Publish hindsight-embed to PyPI
@@ -87,18 +85,31 @@ jobs:
         packages-dir: ./hindsight-embed/dist
         skip-existing: true
 
+    - name: Publish hindsight-crewai to PyPI
+      uses: pypa/gh-action-pypi-publish@release/v1
+      with:
+        packages-dir: ./hindsight-integrations/crewai/dist
+        skip-existing: true
+
+    - name: Publish hindsight-pydantic-ai to PyPI
+      uses: pypa/gh-action-pypi-publish@release/v1
+      with:
+        packages-dir: ./hindsight-integrations/pydantic-ai/dist
+        skip-existing: true
+
     # Upload artifacts for GitHub release
     - name: Upload artifacts
-      uses: actions/upload-artifact@v7
+      uses: actions/upload-artifact@v4
       with:
         name: python-packages
         path: |
           hindsight-clients/python/dist/*
-          hindsight-api-slim/dist/*
           hindsight-api/dist/*
-          hindsight-all/dist/*
-          hindsight-all-slim/dist/*
+          hindsight/dist/*
+          hindsight-integrations/litellm/dist/*
           hindsight-embed/dist/*
+          hindsight-integrations/crewai/dist/*
+          hindsight-integrations/pydantic-ai/dist/*
         retention-days: 1
 
   release-typescript-client:
@@ -106,10 +117,10 @@ jobs:
     environment: npm
 
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v4
 
     - name: Set up Node.js
-      uses: actions/setup-node@v6
+      uses: actions/setup-node@v4
       with:
         node-version: '20'
         registry-url: 'https://registry.npmjs.org'
@@ -144,35 +155,35 @@ jobs:
       run: npm pack
 
     - name: Upload artifacts
-      uses: actions/upload-artifact@v7
+      uses: actions/upload-artifact@v4
       with:
         name: typescript-client
         path: hindsight-clients/typescript/*.tgz
         retention-days: 1
 
-  release-hindsight-all-npm:
+  release-openclaw-integration:
     runs-on: ubuntu-latest
     environment: npm
 
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v4
 
     - name: Set up Node.js
-      uses: actions/setup-node@v6
+      uses: actions/setup-node@v4
       with:
         node-version: '22'
         registry-url: 'https://registry.npmjs.org'
-        cache: 'npm'
-        cache-dependency-path: package-lock.json
 
     - name: Install dependencies
-      run: npm ci --workspace=hindsight-all-npm
+      working-directory: ./hindsight-integrations/openclaw
+      run: npm ci
 
     - name: Build
-      run: npm run build --workspace=hindsight-all-npm
+      working-directory: ./hindsight-integrations/openclaw
+      run: npm run build
 
     - name: Publish to npm
-      working-directory: ./hindsight-all-npm
+      working-directory: ./hindsight-integrations/openclaw
       run: |
         set +e
         OUTPUT=$(npm publish --access public 2>&1)
@@ -189,14 +200,112 @@ jobs:
         NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
 
     - name: Pack for GitHub release
-      working-directory: ./hindsight-all-npm
+      working-directory: ./hindsight-integrations/openclaw
       run: npm pack
 
     - name: Upload artifacts
-      uses: actions/upload-artifact@v7
+      uses: actions/upload-artifact@v4
       with:
-        name: hindsight-all-npm
-        path: hindsight-all-npm/*.tgz
+        name: openclaw-integration
+        path: hindsight-integrations/openclaw/*.tgz
+        retention-days: 1
+
+  release-ai-sdk-integration:
+    runs-on: ubuntu-latest
+    environment: npm
+
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: Set up Node.js
+      uses: actions/setup-node@v4
+      with:
+        node-version: '22'
+        registry-url: 'https://registry.npmjs.org'
+
+    - name: Install dependencies
+      working-directory: ./hindsight-integrations/ai-sdk
+      run: npm ci
+
+    - name: Build
+      working-directory: ./hindsight-integrations/ai-sdk
+      run: npm run build
+
+    - name: Publish to npm
+      working-directory: ./hindsight-integrations/ai-sdk
+      run: |
+        set +e
+        OUTPUT=$(npm publish --access public 2>&1)
+        EXIT_CODE=$?
+        echo "$OUTPUT"
+        if [ $EXIT_CODE -ne 0 ]; then
+          if echo "$OUTPUT" | grep -q "cannot publish over"; then
+            echo "Package version already published, skipping..."
+            exit 0
+          fi
+          exit $EXIT_CODE
+        fi
+      env:
+        NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
+
+    - name: Pack for GitHub release
+      working-directory: ./hindsight-integrations/ai-sdk
+      run: npm pack
+
+    - name: Upload artifacts
+      uses: actions/upload-artifact@v4
+      with:
+        name: ai-sdk-integration
+        path: hindsight-integrations/ai-sdk/*.tgz
+        retention-days: 1
+
+  release-chat-integration:
+    runs-on: ubuntu-latest
+    environment: npm
+
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: Set up Node.js
+      uses: actions/setup-node@v4
+      with:
+        node-version: '22'
+        registry-url: 'https://registry.npmjs.org'
+
+    - name: Install dependencies
+      working-directory: ./hindsight-integrations/chat
+      run: npm ci
+
+    - name: Build
+      working-directory: ./hindsight-integrations/chat
+      run: npm run build
+
+    - name: Publish to npm
+      working-directory: ./hindsight-integrations/chat
+      run: |
+        set +e
+        OUTPUT=$(npm publish --access public 2>&1)
+        EXIT_CODE=$?
+        echo "$OUTPUT"
+        if [ $EXIT_CODE -ne 0 ]; then
+          if echo "$OUTPUT" | grep -q "cannot publish over"; then
+            echo "Package version already published, skipping..."
+            exit 0
+          fi
+          exit $EXIT_CODE
+        fi
+      env:
+        NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
+
+    - name: Pack for GitHub release
+      working-directory: ./hindsight-integrations/chat
+      run: npm pack
+
+    - name: Upload artifacts
+      uses: actions/upload-artifact@v4
+      with:
+        name: chat-integration
+        path: hindsight-integrations/chat/*.tgz
         retention-days: 1
 
   release-control-plane:
@@ -204,10 +313,10 @@ jobs:
     environment: npm
 
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v4
 
     - name: Set up Node.js
-      uses: actions/setup-node@v6
+      uses: actions/setup-node@v4
       with:
         node-version: '20'
         registry-url: 'https://registry.npmjs.org'
@@ -255,7 +364,7 @@ jobs:
       run: npm pack
 
     - name: Upload artifacts
-      uses: actions/upload-artifact@v7
+      uses: actions/upload-artifact@v4
       with:
         name: control-plane
         path: hindsight-control-plane/*.tgz
@@ -284,7 +393,7 @@ jobs:
             asset_name: hindsight-linux-arm64
 
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v4
 
     - name: Install Rust
       uses: dtolnay/rust-toolchain@stable
@@ -302,7 +411,7 @@ jobs:
         chmod +x artifacts/${{ matrix.asset_name }}
 
     - name: Upload artifacts
-      uses: actions/upload-artifact@v7
+      uses: actions/upload-artifact@v4
       with:
         name: rust-cli-${{ matrix.asset_name }}
         path: artifacts/${{ matrix.asset_name }}
@@ -343,7 +452,7 @@ jobs:
               PRELOAD_ML_MODELS=false
 
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v4
 
     - name: Free Disk Space
       uses: jlumbroso/free-disk-space@main
@@ -357,13 +466,13 @@ jobs:
         swap-storage: true
 
     - name: Set up QEMU
-      uses: docker/setup-qemu-action@v4
+      uses: docker/setup-qemu-action@v3
 
     - name: Set up Docker Buildx
-      uses: docker/setup-buildx-action@v4
+      uses: docker/setup-buildx-action@v3
 
     - name: Log in to GitHub Container Registry
-      uses: docker/login-action@v4
+      uses: docker/login-action@v3
       with:
         registry: ghcr.io
         username: ${{ github.actor }}
@@ -375,7 +484,7 @@ jobs:
 
     - name: Extract metadata for release tags
       id: meta
-      uses: docker/metadata-action@v6
+      uses: docker/metadata-action@v5
       with:
         images: ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}
         flavor: |
@@ -391,7 +500,7 @@ jobs:
     # # Step 1: Build for local testing (single platform, no push)
     # # This creates an identical image to what will be released, just for one platform
     # - name: Build image for testing
-    #   uses: docker/build-push-action@v7
+    #   uses: docker/build-push-action@v6
     #   with:
     #     context: .
     #     file: docker/standalone/Dockerfile
@@ -410,7 +519,7 @@ jobs:
 
     # Build multi-platform and push to release tags
     - name: Build and push release images
-      uses: docker/build-push-action@v7
+      uses: docker/build-push-action@v6
       with:
         context: .
         file: docker/standalone/Dockerfile
@@ -428,10 +537,10 @@ jobs:
       packages: write
 
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v4
 
     - name: Install Helm
-      uses: azure/setup-helm@v5
+      uses: azure/setup-helm@v4
       with:
         version: 'latest'
 
@@ -448,7 +557,7 @@ jobs:
       run: helm push helm-packages/*.tgz oci://ghcr.io/${{ github.repository_owner }}/charts
 
     - name: Upload artifacts
-      uses: actions/upload-artifact@v7
+      uses: actions/upload-artifact@v4
       with:
         name: helm-chart
         path: helm-packages/*.tgz
@@ -456,61 +565,73 @@ jobs:
 
   create-github-release:
     runs-on: ubuntu-latest
-    needs: [release-python-packages, release-typescript-client, release-hindsight-all-npm, release-control-plane, release-rust-cli, release-docker-images, release-helm-chart]
+    needs: [release-python-packages, release-typescript-client, release-openclaw-integration, release-ai-sdk-integration, release-chat-integration, release-control-plane, release-rust-cli, release-docker-images, release-helm-chart]
     permissions:
       contents: write
 
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v4
 
     - name: Extract version from tag
       id: get_version
       run: echo "VERSION=${GITHUB_REF#refs/tags/v}" >> $GITHUB_OUTPUT
 
     - name: Download Python packages
-      uses: actions/download-artifact@v8
+      uses: actions/download-artifact@v4
       with:
         name: python-packages
         path: ./artifacts/python-packages
 
     - name: Download TypeScript client
-      uses: actions/download-artifact@v8
+      uses: actions/download-artifact@v4
       with:
         name: typescript-client
         path: ./artifacts/typescript-client
 
+    - name: Download OpenClaw Integration
+      uses: actions/download-artifact@v4
+      with:
+        name: openclaw-integration
+        path: ./artifacts/openclaw-integration
+
+    - name: Download AI SDK Integration
+      uses: actions/download-artifact@v4
+      with:
+        name: ai-sdk-integration
+        path: ./artifacts/ai-sdk-integration
+
+    - name: Download Chat Integration
+      uses: actions/download-artifact@v4
+      with:
+        name: chat-integration
+        path: ./artifacts/chat-integration
+
     - name: Download Control Plane
-      uses: actions/download-artifact@v8
+      uses: actions/download-artifact@v4
       with:
         name: control-plane
         path: ./artifacts/control-plane
 
-    - name: Download hindsight-embed npm wrapper
-      uses: actions/download-artifact@v8
-      with:
-        name: hindsight-all-npm
-        path: ./artifacts/hindsight-all-npm
-
     - name: Download Rust CLI (Linux)
-      uses: actions/download-artifact@v8
+      uses: actions/download-artifact@v4
       with:
         name: rust-cli-hindsight-linux-amd64
         path: ./artifacts/rust-cli-linux
 
     - name: Download Rust CLI (macOS Intel)
-      uses: actions/download-artifact@v8
+      uses: actions/download-artifact@v4
       with:
         name: rust-cli-hindsight-darwin-amd64
         path: ./artifacts/rust-cli-darwin-amd64
 
     - name: Download Rust CLI (macOS ARM)
-      uses: actions/download-artifact@v8
+      uses: actions/download-artifact@v4
       with:
         name: rust-cli-hindsight-darwin-arm64
         path: ./artifacts/rust-cli-darwin-arm64
 
     - name: Download Helm chart
-      uses: actions/download-artifact@v8
+      uses: actions/download-artifact@v4
       with:
         name: helm-chart
         path: ./artifacts/helm-chart
@@ -520,15 +641,19 @@ jobs:
         mkdir -p release-assets
         # Python packages
         cp artifacts/python-packages/hindsight-clients/python/dist/* release-assets/ || true
-        cp artifacts/python-packages/hindsight-api-slim/dist/* release-assets/ || true
         cp artifacts/python-packages/hindsight-api/dist/* release-assets/ || true
-        cp artifacts/python-packages/hindsight-all/dist/* release-assets/ || true
-        cp artifacts/python-packages/hindsight-all-slim/dist/* release-assets/ || true
+        cp artifacts/python-packages/hindsight/dist/* release-assets/ || true
+        cp artifacts/python-packages/hindsight-integrations/litellm/dist/* release-assets/ || true
+        cp artifacts/python-packages/hindsight-integrations/pydantic-ai/dist/* release-assets/ || true
         cp artifacts/python-packages/hindsight-embed/dist/* release-assets/ || true
         # TypeScript client
         cp artifacts/typescript-client/*.tgz release-assets/ || true
-        # hindsight-embed npm wrapper
-        cp artifacts/hindsight-all-npm/*.tgz release-assets/ || true
+        # OpenClaw Integration
+        cp artifacts/openclaw-integration/*.tgz release-assets/ || true
+        # AI SDK Integration
+        cp artifacts/ai-sdk-integration/*.tgz release-assets/ || true
+        # Chat Integration
+        cp artifacts/chat-integration/*.tgz release-assets/ || true
         # Control Plane
         cp artifacts/control-plane/*.tgz release-assets/ || true
         # Rust CLI binaries
@@ -540,7 +665,7 @@ jobs:
         ls -la release-assets/
 
     - name: Create GitHub Release
-      uses: softprops/action-gh-release@v3
+      uses: softprops/action-gh-release@v2
       with:
         files: release-assets/*
         generate_release_notes: true

.gitignore

@@ -50,8 +50,7 @@ hindsight-dev/benchmarks/perf/results/
 benchmarks/results/
 hindsight-cli/target
 hindsight-clients/rust/target
-.claude/*
-!.claude/skills/
+.claude
 whats-next.md
 TASK.md
 # Changelog is now tracked in hindsight-docs/src/pages/changelog.md

CLAUDE.md

@@ -11,32 +11,26 @@ Hindsight is an agent memory system that provides long-term memory for AI agents
 
 ## Development Commands
 
-### Local Development (API + UI)
-```bash
-# Start both API server and control plane UI
-./scripts/dev/start.sh
-```
-
 ### API Server (Python/FastAPI)
 ```bash
-# Start API server only (loads .env automatically)
+# Start API server (loads .env automatically)
 ./scripts/dev/start-api.sh
 
 # Run all tests (parallelized with pytest-xdist)
-cd hindsight-api-slim && uv run pytest tests/
+cd hindsight-api && uv run pytest tests/
 
 # Run specific test file
-cd hindsight-api-slim && uv run pytest tests/test_http_api_integration.py -v
+cd hindsight-api && uv run pytest tests/test_http_api_integration.py -v
 
 # Run single test function
-cd hindsight-api-slim && uv run pytest tests/test_retain.py::test_retain_simple -v
+cd hindsight-api && uv run pytest tests/test_retain.py::test_retain_simple -v
 
 # Lint and format
-cd hindsight-api-slim && uv run ruff check .
-cd hindsight-api-slim && uv run ruff format .
+cd hindsight-api && uv run ruff check .
+cd hindsight-api && uv run ruff format .
 
 # Type checking (uses ty - extremely fast type checker from Astral)
-cd hindsight-api-slim && uv run ty check hindsight_api/
+cd hindsight-api && uv run ty check hindsight_api/
 ```
 
 ### Control Plane (Next.js)
@@ -78,17 +72,18 @@ cd hindsight-control-plane && npm run dev
 ## Architecture
 
 ### Monorepo Structure
-- **hindsight-api-slim/**: Core FastAPI server with memory engine (Python, uv)
+- **hindsight-api/**: Core FastAPI server with memory engine (Python, uv)
+- **hindsight/**: Embedded Python bundle (hindsight-all package)
 - **hindsight-control-plane/**: Admin UI (Next.js, npm)
 - **hindsight-cli/**: CLI tool (Rust, cargo, uses progenitor for API client)
 - **hindsight-clients/**: Generated SDK clients (Python, TypeScript, Rust)
 - **hindsight-docs/**: Docusaurus documentation site
-- **hindsight-integrations/**: Framework integrations (LiteLLM, CrewAI, LangGraph, Pydantic AI, AG2, Claude Code, etc.)
+- **hindsight-integrations/**: Framework integrations (LiteLLM, OpenAI)
 - **hindsight-dev/**: Development tools and benchmarks
 
-### Core Engine (hindsight-api-slim/hindsight_api/engine/)
-- `memory_engine.py`: Main orchestrator for retain/recall/reflect operations
-- `llm_wrapper.py`: LLM abstraction supporting OpenAI, Anthropic, Gemini, VertexAI, Groq, MiniMax, Ollama, LM Studio, LiteLLM, Claude Code
+### Core Engine (hindsight-api/hindsight_api/engine/)
+- `memory_engine.py`: Main orchestrator (~170KB) for retain/recall/reflect operations
+- `llm_wrapper.py`: LLM abstraction supporting OpenAI, Anthropic, Gemini, Groq, Ollama, LM Studio
 - `embeddings.py`: Embedding generation (local sentence-transformers or TEI)
 - `cross_encoder.py`: Reranking (local or TEI)
 - `entity_resolver.py`: Entity extraction and normalization
@@ -101,13 +96,13 @@ cd hindsight-control-plane && npm run dev
 
 **search/**: Multi-strategy retrieval
 - `retrieval.py`: Main retrieval orchestrator
-- `graph_retrieval.py`: Graph retrieval abstract base class
-- `link_expansion_retrieval.py`: Link expansion graph retrieval
+- `graph_retrieval.py`: Entity/relationship graph traversal
+- `mpfp_retrieval.py`: Multi-Path Fact Propagation retrieval
 - `fusion.py`: Reciprocal rank fusion for combining results
 - `reranking.py`: Cross-encoder reranking
 
-### API Layer (hindsight-api-slim/hindsight_api/api/)
-- `http.py`: FastAPI HTTP routers for all REST endpoints
+### API Layer (hindsight-api/hindsight_api/api/)
+- `http.py`: FastAPI HTTP routers (~80KB) for all REST endpoints
 - `mcp.py`: Model Context Protocol server implementation
 
 Main operations:
@@ -116,13 +111,13 @@ Main operations:
 - **Reflect**: Disposition-aware reasoning using memories and mental models.
 
 ### Database
-PostgreSQL with pgvector. Schema managed via Alembic migrations in `hindsight-api-slim/hindsight_api/alembic/`. Migrations run automatically on API startup.
+PostgreSQL with pgvector. Schema managed via Alembic migrations in `hindsight-api/hindsight_api/alembic/`. Migrations run automatically on API startup.
 
 Key tables: `banks`, `memory_units`, `documents`, `entities`, `entity_links`
 
 ### Adding Database Migrations
 
-1. **Create a new migration file** in `hindsight-api-slim/hindsight_api/alembic/versions/`:
+1. **Create a new migration file** in `hindsight-api/hindsight_api/alembic/versions/`:
    - File name format: `<revision_id>_<description>.py` (e.g., `f1a2b3c4d5e6_add_new_index.py`)
    - Use a unique hex revision ID (12 chars)
    - Set `down_revision` to the previous migration's revision ID
@@ -159,7 +154,7 @@ Key tables: `banks`, `memory_units`, `documents`, `entities`, `entity_links`
 
 3. **Run migrations locally**:
    ```bash
-   # Set database URL and run migrations for the base schema plus all tenants
+   # Set database URL and run migrations
    uv run hindsight-admin run-db-migration
 
    # Run on a specific tenant schema
@@ -169,17 +164,11 @@ Key tables: `banks`, `memory_units`, `documents`, `entities`, `entity_links`
 ## Key Conventions
 
 ### Code Quality
-
-**Before writing code, read `.claude/skills/code-review/SKILL.md`** for the full coding standards (Python style, type safety, TypeScript style, general principles).
-
 **Always run the lint script after making Python or TypeScript/Node changes:**
 ```bash
 ./scripts/hooks/lint.sh
 ```
-
-**After completing any implementation work, run `/code-review`** to verify your changes against project standards (missing tests, dead code, type safety, etc.). Fix any "must fix" issues before considering the task done.
-
-**MANDATORY: Run `/code-review` before pushing code or creating a pull request.** Do not push or create a PR until all "must fix" issues are resolved.
+This runs the same checks as the pre-commit hook (Ruff for Python, ESLint/Prettier for TypeScript).
 
 ### Memory Banks
 - Each bank is an isolated memory store (like a "brain" for one user/agent)
@@ -211,20 +200,48 @@ When adding or modifying parameters in the dataplane API (hindsight-api), you mu
    - Update the client type definition in `lib/api.ts`
    - Update any UI components that need to use the new parameter
 
-### Adding New Integrations
+### Python Style
+- Python 3.11+, type hints required
+- Async throughout (asyncpg, async FastAPI)
+- Pydantic models for request/response
+- Ruff for linting (line-length 120)
+- No Python files at project root - maintain clean directory structure
+- **Never use multi-item tuple return values** - prefer dataclass or Pydantic model for structured returns
 
-Every new integration in `hindsight-integrations/` must satisfy all of the following before it can be merged:
+### Type Safety with Pydantic Models
+**NEVER use raw `dict` types for structured data.** Always use Pydantic models:
+- Use Pydantic `BaseModel` for all data structures passed between functions
+- Add `@field_validator` for type coercion (e.g., ensuring datetimes are timezone-aware)
+- Avoid `dict.get()` patterns - use typed model attributes instead
+- Parse external data (JSON, API responses) into Pydantic models at the boundary
+- This catches type errors at parse time, not deep in business logic
 
-1. **Tests are required** — tests must simulate or exercise the external system (mock the framework's interfaces and verify the integration actually calls Hindsight correctly). Pure unit tests of helper functions are not sufficient.
-2. **CI job** — add a test job in `.github/workflows/test.yml` following the existing pattern (e.g., `test-crewai-integration`). The job must build, install deps, and run `uv run pytest tests -v`. Also add the integration to `detect-changes` outputs so it only runs when its files change.
-3. **Release process** — add the integration name to the `VALID_INTEGRATIONS` array in `scripts/release-integration.sh` so it can be released via the standard release workflow.
-4. **Follow project code standards** — Python style, type safety, no raw dicts for structured data, no multi-item tuple returns (see `.claude/skills/code-review/SKILL.md`).
+```python
+# BAD - error-prone dict access
+def process(data: dict) -> str:
+    return data.get("name", "")  # No validation, silent failures
 
-If any of these are missing, the integration is incomplete and must not be pushed or merged.
+# GOOD - typed and validated
+class UserData(BaseModel):
+    name: str
+    created_at: datetime
 
-### Changelogs
+    @field_validator("created_at", mode="before")
+    @classmethod
+    def ensure_tz_aware(cls, v):
+        if isinstance(v, str):
+            v = datetime.fromisoformat(v.replace("Z", "+00:00"))
+        if v.tzinfo is None:
+            return v.replace(tzinfo=timezone.utc)
+        return v
 
-Never add "Unreleased" entries to changelogs (e.g. `hindsight-docs/src/pages/changelog/**`). Changelog entries are written by the release script (`./scripts/release-integration.sh`) when a version is actually cut. If a bug fix or feature needs documenting before release, describe it in the PR/commit — the release tooling will surface it in the published changelog section.
+def process(data: UserData) -> str:
+    return data.name  # Type-safe, validated at construction
+```
+
+### TypeScript Style
+- Next.js App Router for control plane
+- Tailwind CSS with shadcn/ui components
 
 ### Adding New API Configuration Flags
 
@@ -234,24 +251,24 @@ Fields must be categorized as either **hierarchical** (can be overridden per-ten
 
 #### Adding a New Configuration Field
 
-1. **config.py** (`hindsight-api-slim/hindsight_api/config.py`):
+1. **config.py** (`hindsight-api/hindsight_api/config.py`):
    - Add `ENV_*` constant for the environment variable name (e.g., `ENV_MY_SETTING = "HINDSIGHT_API_MY_SETTING"`)
    - Add `DEFAULT_*` constant for the default value
    - Add field to `HindsightConfig` dataclass with type annotation
-   - **Mark as configurable** by adding to `_CONFIGURABLE_FIELDS` set if the field should be overridable per-tenant/bank via API
+   - **Mark as hierarchical or static** by adding to `_HIERARCHICAL_FIELDS` set (hierarchical) or leaving it out (static)
    - Add initialization in `from_env()` method
 
    ```python
-   # Configurable field (can be overridden per-tenant/bank via API)
-   _CONFIGURABLE_FIELDS = {
+   # Hierarchical field (can be overridden per-bank)
+   _HIERARCHICAL_FIELDS = {
        ...,
-       "my_setting",  # Add here for configurable
+       "my_setting",  # Add here for hierarchical
    }
 
-   # Static field - just don't add to _CONFIGURABLE_FIELDS
+   # Static field - just don't add to _HIERARCHICAL_FIELDS
    ```
 
-2. **main.py** (`hindsight-api-slim/hindsight_api/main.py`):
+2. **main.py** (`hindsight-api/hindsight_api/main.py`):
    - Add field to the manual `HindsightConfig()` constructor call (search for "CLI override")
 
 3. **Use hierarchical config in MemoryEngine**:
@@ -291,14 +308,14 @@ cp .env.example .env
 # Edit .env with LLM API key
 
 # Python deps
-uv sync --directory hindsight-api-slim/
+uv sync --directory hindsight-api/
 
 # Node deps (uses npm workspaces)
 npm install
 ```
 
 Required env vars:
-- `HINDSIGHT_API_LLM_PROVIDER`: openai, anthropic, gemini, groq, minimax, ollama, lmstudio
+- `HINDSIGHT_API_LLM_PROVIDER`: openai, anthropic, gemini, groq, ollama, lmstudio
 - `HINDSIGHT_API_LLM_API_KEY`: Your API key
 - `HINDSIGHT_API_LLM_MODEL`: Model name (e.g., gpt-4o-mini, claude-sonnet-4-20250514)
 

README.md

@@ -7,12 +7,10 @@
 [![CI](https://github.com/vectorize-io/hindsight/actions/workflows/release.yml/badge.svg)](https://github.com/vectorize-io/hindsight/actions/workflows/release.yml)
 [![Slack Community](https://img.shields.io/badge/Slack-Join%20Community-4A154B?logo=slack)](https://join.slack.com/t/hindsight-space/shared_invite/zt-3nhbm4w29-LeSJ5Ixi6j8PdiYOCPlOgg)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
-[![gitcgr](https://gitcgr.com/badge/vectorize-io/hindsight.svg)](https://gitcgr.com/vectorize-io/hindsight)
 ![PyPI - Downloads](https://img.shields.io/pypi/dm/hindsight-api?label=PyPI)
 ![NPM Downloads](https://img.shields.io/npm/dm/%40vectorize-io%2Fhindsight-client?logoColor=orange&label=NPM&color=blue&link=https%3A%2F%2Fwww.npmjs.com%2Fpackage%2F%40vectorize-io%2Fhindsight-client)
-<br/>
 
-<a href="https://trendshift.io/repositories/15603" target="_blank"><img src="https://trendshift.io/api/badge/repositories/15603" alt="vectorize-io%2Fhindsight | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
+
 </div>
 
 ---
@@ -71,7 +69,7 @@ docker run --rm -it --pull always -p 8888:8888 -p 9999:9999 \
 >API: http://localhost:8888
 >UI: http://localhost:9999
 
-You can modify the LLM provider by setting `HINDSIGHT_API_LLM_PROVIDER`. Valid options are `openai`, `anthropic`, `gemini`, `groq`, `ollama`, `lmstudio`, and `minimax`. The documentation provides more details on [supported models](https://hindsight.vectorize.io/developer/models).
+You can modify the LLM provider by setting `HINDSIGHT_API_LLM_PROVIDER`. Valid options are `openai`, `anthropic`, `gemini`, `groq`, `ollama`, and `lmstudio`. The documentation provides more details on [supported models](https://hindsight.vectorize.io/developer/models).
 
 
 

deno.lock

@@ -1,139 +0,0 @@
-{
-  "version": "5",
-  "specifiers": {
-    "jsr:@std/assert@^1.0.17": "1.0.19",
-    "jsr:@std/assert@^1.0.19": "1.0.19",
-    "jsr:@std/expect@*": "1.0.18",
-    "jsr:@std/internal@^1.0.12": "1.0.12",
-    "jsr:@std/path@^1.1.4": "1.1.4",
-    "jsr:@std/testing@*": "1.0.17"
-  },
-  "jsr": {
-    "@std/[email protected]": {
-      "integrity": "eaada96ee120cb980bc47e040f82814d786fe8162ecc53c91d8df60b8755991e",
-      "dependencies": [
-        "jsr:@std/internal"
-      ]
-    },
-    "@std/[email protected]": {
-      "integrity": "8566eab35200466f8609eb7e7aed062ed0db314e9a258d5d201b1b8997ce801a",
-      "dependencies": [
-        "jsr:@std/assert@^1.0.19",
-        "jsr:@std/internal",
-        "jsr:@std/path"
-      ]
-    },
-    "@std/[email protected]": {
-      "integrity": "972a634fd5bc34b242024402972cd5143eac68d8dffaca5eaa4dba30ce17b027"
-    },
-    "@std/[email protected]": {
-      "integrity": "1d2d43f39efb1b42f0b1882a25486647cb851481862dc7313390b2bb044314b5",
-      "dependencies": [
-        "jsr:@std/internal"
-      ]
-    },
-    "@std/[email protected]": {
-      "integrity": "87bdc2700fa98249d48a17cd72413352d3d3680dcfbdb64947fd0982d6bbf681",
-      "dependencies": [
-        "jsr:@std/assert@^1.0.17",
-        "jsr:@std/internal"
-      ]
-    }
-  },
-  "workspace": {
-    "members": {
-      "hindsight-clients/typescript": {
-        "packageJson": {
-          "dependencies": [
-            "npm:@hey-api/[email protected]",
-            "npm:@types/jest@29",
-            "npm:@types/node@20",
-            "npm:jest@29",
-            "npm:ts-jest@29",
-            "npm:tsup@^8.5.1",
-            "npm:typescript@5"
-          ]
-        }
-      },
-      "hindsight-control-plane": {
-        "packageJson": {
-          "dependencies": [
-            "npm:@eslint/eslintrc@^3.3.3",
-            "npm:@eslint/js@^9.39.2",
-            "npm:@radix-ui/react-alert-dialog@^1.1.15",
-            "npm:@radix-ui/react-checkbox@^1.3.3",
-            "npm:@radix-ui/react-dialog@^1.1.15",
-            "npm:@radix-ui/react-dropdown-menu@^2.1.16",
-            "npm:@radix-ui/react-label@^2.1.8",
-            "npm:@radix-ui/react-popover@^1.1.15",
-            "npm:@radix-ui/react-radio-group@^1.3.8",
-            "npm:@radix-ui/react-select@^2.2.6",
-            "npm:@radix-ui/react-slider@^1.3.6",
-            "npm:@radix-ui/react-slot@^1.2.4",
-            "npm:@radix-ui/react-switch@^1.2.6",
-            "npm:@radix-ui/react-tabs@^1.1.13",
-            "npm:@radix-ui/react-tooltip@^1.2.8",
-            "npm:@tailwindcss/postcss@^4.1.17",
-            "npm:@tailwindcss/typography@~0.5.19",
-            "npm:@types/cytoscape@^3.21.9",
-            "npm:@types/node@^24.10.0",
-            "npm:@types/react-dom@^19.2.2",
-            "npm:@types/react@^19.2.2",
-            "npm:autoprefixer@^10.4.21",
-            "npm:class-variance-authority@~0.7.1",
-            "npm:clsx@^2.1.1",
-            "npm:cmdk@^1.1.1",
-            "npm:cytoscape-fcose@^2.2.0",
-            "npm:cytoscape@^3.33.1",
-            "npm:eslint-config-next@^16.0.1",
-            "npm:eslint-plugin-react-hooks@^7.0.1",
-            "npm:eslint-plugin-react@^7.37.5",
-            "npm:eslint@^9.39.1",
-            "npm:[email protected]",
-            "npm:next-themes@~0.4.6",
-            "npm:next@^16.1.6",
-            "npm:postcss@^8.5.6",
-            "npm:prettier@^3.7.4",
-            "npm:react-chrono@^2.9.1",
-            "npm:react-dom@^19.2.0",
-            "npm:react-markdown@^10.1.0",
-            "npm:react18-json-view@~0.2.9",
-            "npm:react@^19.2.0",
-            "npm:recharts@^3.5.1",
-            "npm:remark-gfm@^4.0.1",
-            "npm:sonner@^2.0.7",
-            "npm:tailwind-merge@^3.4.0",
-            "npm:tailwindcss-animate@^1.0.7",
-            "npm:tailwindcss@^4.1.17",
-            "npm:[email protected]",
-            "npm:typescript-eslint@^8.50.0",
-            "npm:typescript@^5.9.3"
-          ]
-        }
-      },
-      "hindsight-docs": {
-        "packageJson": {
-          "dependencies": [
-            "npm:@docusaurus/[email protected]",
-            "npm:@docusaurus/[email protected]",
-            "npm:@docusaurus/[email protected]",
-            "npm:@docusaurus/theme-common@^3.9.2",
-            "npm:@docusaurus/theme-mermaid@^3.9.2",
-            "npm:@docusaurus/[email protected]",
-            "npm:@docusaurus/[email protected]",
-            "npm:@easyops-cn/docusaurus-search-local@~0.52.2",
-            "npm:@mdx-js/react@3",
-            "npm:clsx@2",
-            "npm:prism-react-renderer@^2.3.0",
-            "npm:raw-loader@^4.0.2",
-            "npm:react-dom@19",
-            "npm:react-icons@^5.6.0",
-            "npm:react@19",
-            "npm:redocusaurus@^2.5.0",
-            "npm:typescript@~5.6.2"
-          ]
-        }
-      }
-    }
-  }
-}

docker/standalone/Dockerfile

@@ -42,22 +42,25 @@ RUN apt-get update && apt-get install -y \
     && pip install --no-cache-dir uv
 
 # Copy dependency files and README (required by pyproject.toml)
-COPY hindsight-api-slim/pyproject.toml ./api/
-COPY hindsight-api-slim/README.md ./api/
+COPY hindsight-api/pyproject.toml ./api/
+COPY hindsight-api/README.md ./api/
 
 WORKDIR /app/api
 
-# Sync dependencies using appropriate extras based on INCLUDE_LOCAL_MODELS
-# local-ml: torch, sentence-transformers, transformers, einops, flashrank, mlx (optional)
-# embedded-db: pg0-embedded (always included for embedded PostgreSQL support)
-RUN if [ "$INCLUDE_LOCAL_MODELS" = "true" ]; then \
-        uv sync --extra local-ml --extra embedded-db; \
-    else \
-        uv sync --extra embedded-db; \
+# Remove local ML model dependencies if INCLUDE_LOCAL_MODELS=false
+# This creates a smaller image when using external providers (TEI, OpenAI, Cohere)
+RUN if [ "$INCLUDE_LOCAL_MODELS" != "true" ]; then \
+        echo "Removing local-models dependencies (sentence-transformers, torch, transformers)..." && \
+        sed -i '/"sentence-transformers/d' pyproject.toml && \
+        sed -i '/"transformers/d' pyproject.toml && \
+        sed -i '/"torch/d' pyproject.toml; \
     fi
 
+# Sync dependencies (will create lock file if needed)
+RUN uv sync
+
 # Copy source code (alembic migrations are inside hindsight_api/)
-COPY hindsight-api-slim/hindsight_api ./hindsight_api
+COPY hindsight-api/hindsight_api ./hindsight_api
 
 # Install the local package (uv sync only installed dependencies, not the package itself)
 RUN uv pip install -e .

docker/standalone/start-all.sh

@@ -1,28 +1,6 @@
 #!/bin/bash
 set -e
 
-# =============================================================================
-# Embedded pg0 data integrity check (#675)
-#
-# When using embedded pg0, check if the data directory has existing PostgreSQL
-# data before starting. If the directory exists but appears empty/corrupt
-# (e.g., missing PG_VERSION file), log a warning. This helps diagnose data
-# loss scenarios where a container restart caused the data directory to be
-# wiped despite a volume mount being present.
-# =============================================================================
-PG0_DATA_DIR="${HOME}/.pg0"
-if [ -d "$PG0_DATA_DIR" ]; then
-    # Look for actual PostgreSQL data directories (pg0 creates subdirs per instance)
-    if compgen -G "$PG0_DATA_DIR"/*/PG_VERSION > /dev/null 2>&1; then
-        echo "✅ Existing pg0 data directory detected at $PG0_DATA_DIR"
-    elif [ "$(ls -A "$PG0_DATA_DIR" 2>/dev/null)" ]; then
-        echo "⚠️  WARNING: pg0 data directory exists at $PG0_DATA_DIR but no PG_VERSION found."
-        echo "   This may indicate data corruption or an incomplete previous shutdown."
-        echo "   If you see all migrations running from scratch after this, your data may have been lost."
-        echo "   See: https://github.com/vectorize-io/hindsight/issues/675"
-    fi
-fi
-
 # Service flags (default to true if not set)
 ENABLE_API="${HINDSIGHT_ENABLE_API:-true}"
 ENABLE_CP="${HINDSIGHT_ENABLE_CP:-true}"
@@ -93,95 +71,24 @@ if [ "${HINDSIGHT_WAIT_FOR_DEPS:-false}" = "true" ]; then
     done
 fi
 
-# =============================================================================
-# Graceful shutdown handler (#675)
-#
-# Docker sends SIGTERM on `docker stop`/`docker restart`. Without a trap, child
-# processes (hindsight-api + pg0, control-plane) are killed abruptly. For the
-# embedded pg0 database this can cause data loss when the data directory is on
-# a Docker volume that gets remounted after restart.
-#
-# The trap forwards SIGTERM to all tracked child PIDs so that:
-#   - hindsight-api receives the signal and can run its shutdown hooks
-#   - pg0 gets a clean PostgreSQL shutdown (checkpoint + WAL flush)
-#   - The control-plane Node.js process exits cleanly
-# =============================================================================
-# Guard against concurrent cleanup (e.g., child crash + SIGTERM arriving together)
-SHUTTING_DOWN=false
-
-cleanup() {
-    if $SHUTTING_DOWN; then return; fi
-    SHUTTING_DOWN=true
-
-    echo ""
-    echo "🛑 Received shutdown signal, stopping services gracefully..."
-    for pid in "${PIDS[@]}"; do
-        if kill -0 "$pid" 2>/dev/null; then
-            kill -TERM "$pid" 2>/dev/null
-        fi
-    done
-    # Give processes time to shut down cleanly (pg0 needs to flush WAL).
-    # NOTE: Docker's default stop_grace_period is 10s. If you use the default,
-    # either set stop_grace_period: 30s in your compose file / docker stop -t 30,
-    # or Docker will SIGKILL the container before this timeout expires.
-    local timeout=30
-    for ((i=1; i<=timeout; i++)); do
-        local all_stopped=true
-        for pid in "${PIDS[@]}"; do
-            if kill -0 "$pid" 2>/dev/null; then
-                all_stopped=false
-                break
-            fi
-        done
-        if $all_stopped; then
-            echo "✅ All services stopped cleanly"
-            exit 0
-        fi
-        sleep 1
-    done
-    # Force kill if still running after timeout
-    echo "⚠️  Timeout reached, forcing shutdown..."
-    for pid in "${PIDS[@]}"; do
-        if kill -0 "$pid" 2>/dev/null; then
-            kill -9 "$pid" 2>/dev/null
-        fi
-    done
-    exit 1
-}
-trap cleanup SIGTERM SIGINT
-
 # Track PIDs for wait
 PIDS=()
 
 # Start API if enabled
 if [ "$ENABLE_API" = "true" ]; then
     cd /app/api
-    API_HEALTH_URL="${HINDSIGHT_API_HEALTH_URL:-http://localhost:8888/health}"
-    API_STARTUP_WAIT_SECONDS="${HINDSIGHT_API_STARTUP_WAIT_SECONDS:-300}"
-
     # Run API directly - Python's PYTHONUNBUFFERED=1 handles output buffering
     hindsight-api &
     API_PID=$!
     PIDS+=($API_PID)
 
     # Wait for API to be ready
-    api_ready=false
-    for ((i=1; i<=API_STARTUP_WAIT_SECONDS; i++)); do
-        if ! kill -0 "$API_PID" 2>/dev/null; then
-            wait "$API_PID"
-            exit $?
-        fi
-        if curl -sf "$API_HEALTH_URL" &>/dev/null; then
-            api_ready=true
+    for i in {1..60}; do
+        if curl -sf http://localhost:8888/health &>/dev/null; then
             break
         fi
         sleep 1
     done
-
-    if [ "$api_ready" != "true" ]; then
-        echo "❌ API did not become healthy within ${API_STARTUP_WAIT_SECONDS}s"
-        exit 1
-    fi
 else
     echo "API disabled (HINDSIGHT_ENABLE_API=false)"
 fi
@@ -190,7 +97,6 @@ fi
 if [ "$ENABLE_CP" = "true" ]; then
     echo "🎛️  Starting Control Plane..."
     cd /app/control-plane
-    export HOSTNAME="${HINDSIGHT_CP_HOSTNAME:-0.0.0.0}"
     PORT="${HINDSIGHT_CP_PORT:-9999}" node server.js &
     CP_PID=$!
     PIDS+=($CP_PID)
@@ -217,21 +123,8 @@ if [ ${#PIDS[@]} -eq 0 ]; then
     exit 1
 fi
 
-# Wait for any process to exit (use wait -n with trap-safe loop)
-while true; do
-    # wait -n returns when any child exits; it also returns on signal delivery
-    # (the trap handler will run and exit, so this loop is just for robustness).
-    # `&& true` prevents `set -e` from killing the script when wait -n returns
-    # non-zero (child exited with error or no backgrounded children remain).
-    wait -n && true
-    # Check if any tracked PID has exited
-    for pid in "${PIDS[@]}"; do
-        if ! kill -0 "$pid" 2>/dev/null; then
-            wait "$pid" 2>/dev/null
-            exit_code=$?
-            echo "⚠️  Service (PID $pid) exited with code $exit_code"
-            # Trigger cleanup for remaining services
-            cleanup
-        fi
-    done
-done
+# Wait for any process to exit
+wait -n
+
+# Exit with status of first exited process
+exit $?

docker/test-image.sh

@@ -49,9 +49,6 @@
 
 set -euo pipefail
 
-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-REPO_ROOT="$(dirname "$SCRIPT_DIR")"
-
 # Colors for output
 RED='\033[0;31m'
 GREEN='\033[0;32m'
@@ -181,21 +178,6 @@ for i in $(seq 1 "$TIMEOUT"); do
         echo "=== Health Response ==="
         curl -s "http://localhost:${HEALTH_PORT}${HEALTH_PATH}" | python3 -m json.tool 2>/dev/null || curl -s "http://localhost:${HEALTH_PORT}${HEALTH_PATH}"
         echo ""
-
-        # Run retain/recall smoke test for API targets
-        if [ "$TARGET" != "cp-only" ]; then
-            echo ""
-            echo "=== Retain/Recall Smoke Test ==="
-            if ! "$REPO_ROOT/scripts/smoke-test-slim.sh" "http://localhost:${HEALTH_PORT}"; then
-                echo ""
-                echo "=== Container Logs (last 50 lines) ==="
-                docker logs "$CONTAINER_NAME" 2>&1 | tail -50
-                echo ""
-                echo -e "${RED}Smoke test FAILED${NC}"
-                exit 1
-            fi
-        fi
-
         echo ""
         echo "=== Container Logs (last 50 lines) ==="
         docker logs "$CONTAINER_NAME" 2>&1 | tail -50

helm/hindsight/Chart.yaml

@@ -2,8 +2,8 @@ apiVersion: v2
 name: hindsight
 description: Hindsight helm chart
 type: application
-version: 0.5.1
-appVersion: "0.5.1"
+version: 0.4.16
+appVersion: "0.4.16"
 keywords:
   - ai
   - memory

helm/hindsight/templates/api-deployment.yaml

@@ -95,27 +95,6 @@ spec:
           {{- toYaml .Values.api.readinessProbe | nindent 10 }}
         resources:
           {{- toYaml .Values.api.resources | nindent 10 }}
-        {{- if or .Values.api.persistence.modelCache.enabled .Values.api.extraVolumeMounts }}
-        volumeMounts:
-        {{- if .Values.api.persistence.modelCache.enabled }}
-        - name: model-cache
-          mountPath: /home/hindsight/.cache
-        {{- end }}
-        {{- with .Values.api.extraVolumeMounts }}
-        {{- toYaml . | nindent 8 }}
-        {{- end }}
-        {{- end }}
-      {{- if or .Values.api.persistence.modelCache.enabled .Values.api.extraVolumes }}
-      volumes:
-      {{- if .Values.api.persistence.modelCache.enabled }}
-      - name: model-cache
-        persistentVolumeClaim:
-          claimName: {{ include "hindsight.fullname" . }}-api-model-cache
-      {{- end }}
-      {{- with .Values.api.extraVolumes }}
-      {{- toYaml . | nindent 6 }}
-      {{- end }}
-      {{- end }}
       {{- with .Values.nodeSelector }}
       nodeSelector:
         {{- toYaml . | nindent 8 }}

helm/hindsight/templates/api-model-cache-pvc.yaml

@@ -1,21 +0,0 @@
-{{- if and .Values.api.enabled .Values.api.persistence.modelCache.enabled }}
-apiVersion: v1
-kind: PersistentVolumeClaim
-metadata:
-  name: {{ include "hindsight.fullname" . }}-api-model-cache
-  labels:
-    {{- include "hindsight.api.labels" . | nindent 4 }}
-  {{- with .Values.api.persistence.modelCache.annotations }}
-  annotations:
-    {{- toYaml . | nindent 4 }}
-  {{- end }}
-spec:
-  accessModes:
-    {{- toYaml .Values.api.persistence.modelCache.accessModes | nindent 4 }}
-  {{- if .Values.api.persistence.modelCache.storageClass }}
-  storageClassName: {{ .Values.api.persistence.modelCache.storageClass }}
-  {{- end }}
-  resources:
-    requests:
-      storage: {{ .Values.api.persistence.modelCache.size }}
-{{- end }}

helm/hindsight/templates/worker-statefulset.yaml

@@ -95,16 +95,6 @@ spec:
           {{- toYaml .Values.worker.readinessProbe | nindent 10 }}
         resources:
           {{- toYaml .Values.worker.resources | nindent 10 }}
-        {{- if or .Values.worker.persistence.modelCache.enabled .Values.worker.extraVolumeMounts }}
-        volumeMounts:
-        {{- if .Values.worker.persistence.modelCache.enabled }}
-        - name: model-cache
-          mountPath: /home/hindsight/.cache
-        {{- end }}
-        {{- with .Values.worker.extraVolumeMounts }}
-        {{- toYaml . | nindent 8 }}
-        {{- end }}
-        {{- end }}
       {{- with .Values.nodeSelector }}
       nodeSelector:
         {{- toYaml . | nindent 8 }}
@@ -117,26 +107,4 @@ spec:
       tolerations:
         {{- toYaml . | nindent 8 }}
       {{- end }}
-      {{- with .Values.worker.extraVolumes }}
-      volumes:
-      {{- toYaml . | nindent 6 }}
-      {{- end }}
-  {{- if .Values.worker.persistence.modelCache.enabled }}
-  volumeClaimTemplates:
-  - metadata:
-      name: model-cache
-      {{- with .Values.worker.persistence.modelCache.annotations }}
-      annotations:
-        {{- toYaml . | nindent 8 }}
-      {{- end }}
-    spec:
-      accessModes:
-        {{- toYaml .Values.worker.persistence.modelCache.accessModes | nindent 8 }}
-      {{- if .Values.worker.persistence.modelCache.storageClass }}
-      storageClassName: {{ .Values.worker.persistence.modelCache.storageClass }}
-      {{- end }}
-      resources:
-        requests:
-          storage: {{ .Values.worker.persistence.modelCache.size }}
-  {{- end }}
 {{- end }}

helm/hindsight/values.yaml

@@ -67,33 +67,6 @@ api:
   # Pod affinity/anti-affinity (overrides global affinity for this component)
   # affinity: {}
 
-  # Persistent volume for local model cache (reranker, embeddings)
-  # Models are downloaded to /home/hindsight/.cache on first use.
-  # Without persistence, models are re-downloaded on every pod restart.
-  persistence:
-    modelCache:
-      enabled: false
-      size: 5Gi
-      storageClass: ""
-      accessModes:
-        - ReadWriteOnce
-      annotations: {}
-
-  # Extra volume mounts for the api container
-  # e.g.
-  # extraVolumeMounts:
-  #   - name: my-volume
-  #     mountPath: /mnt/my-volume
-  extraVolumeMounts: []
-
-  # Extra volumes for the api pod
-  # e.g.
-  # extraVolumes:
-  #   - name: my-volume
-  #     configMap:
-  #       name: my-configmap
-  extraVolumes: []
-
   # Environment variables
   env:
     #HINDSIGHT_API_LLM_PROVIDER: "groq"
@@ -167,32 +140,6 @@ worker:
   # Pod affinity/anti-affinity (overrides global affinity for this component)
   # affinity: {}
 
-  # Persistent volume for local model cache (reranker, embeddings)
-  # Uses volumeClaimTemplates since worker is a StatefulSet.
-  persistence:
-    modelCache:
-      enabled: false
-      size: 5Gi
-      storageClass: ""
-      accessModes:
-        - ReadWriteOnce
-      annotations: {}
-
-  # Extra volume mounts for the worker container
-  # e.g.
-  # extraVolumeMounts:
-  #   - name: my-volume
-  #     mountPath: /mnt/my-volume
-  extraVolumeMounts: []
-
-  # Extra volumes for the worker pod
-  # e.g.
-  # extraVolumes:
-  #   - name: my-volume
-  #     configMap:
-  #       name: my-configmap
-  extraVolumes: []
-
   # Secret environment variables (inherited from api.secrets if not specified)
   secrets: {}
 

hindsight-all-npm/.gitignore

@@ -1,4 +0,0 @@
-node_modules
-dist
-*.tgz
-.DS_Store

hindsight-all-npm/README.md

@@ -1,80 +0,0 @@
-# @vectorize-io/hindsight-all
-
-Node.js equivalent of the Python [`hindsight-all`](https://pypi.org/project/hindsight-all/) package — programmatic lifecycle manager for a local Hindsight daemon. Use this when you want to embed Hindsight in a Node application without hand-rolling subprocess management.
-
-This package deliberately does **not** ship an HTTP client. Once the daemon is running, talk to it with [`@vectorize-io/hindsight-client`](https://www.npmjs.com/package/@vectorize-io/hindsight-client) against `server.getBaseUrl()`. The two packages compose — one owns the daemon process, the other owns the HTTP API surface.
-
-## Requirements
-
-- **Node.js >= 22** — uses global `fetch` and `AbortSignal.timeout`.
-- **`uv` / `uvx`** on `PATH` — used to download and run the underlying `hindsight-embed` daemon on first use. Install via <https://docs.astral.sh/uv/>.
-
-## Install
-
-```bash
-npm install @vectorize-io/hindsight-all @vectorize-io/hindsight-client
-```
-
-## Example
-
-```ts
-import { HindsightServer, consoleLogger } from '@vectorize-io/hindsight-all';
-import { HindsightClient } from '@vectorize-io/hindsight-client';
-
-const server = new HindsightServer({
-  profile: 'my-app',
-  port: 9077,
-  env: {
-    HINDSIGHT_API_LLM_PROVIDER: 'anthropic',
-    HINDSIGHT_API_LLM_API_KEY: process.env.ANTHROPIC_API_KEY,
-    HINDSIGHT_API_LLM_MODEL: 'claude-sonnet-4-20250514',
-    HINDSIGHT_EMBED_DAEMON_IDLE_TIMEOUT: '0',
-  },
-  logger: consoleLogger,
-});
-
-await server.start();
-
-const client = new HindsightClient({ baseUrl: server.getBaseUrl() });
-
-await client.retain('user-123', 'User prefers dark mode and concise answers.', {
-  documentId: 'pref-2026-04-01',
-});
-
-const recall = await client.recall('user-123', 'what are the user preferences?');
-console.log(recall.results);
-
-await server.stop();
-```
-
-For a remote Hindsight API, skip `HindsightServer` entirely and just point `HindsightClient` at the remote URL.
-
-## Open config — forward-compatible with new daemon flags
-
-`HindsightServerOptions` is designed so every new environment variable or CLI flag in the underlying Hindsight daemon can be used without waiting for a wrapper release:
-
-- **`env`** accepts an arbitrary `Record<string, string>`. Every entry is exported into the daemon process and written into the profile config via `--env KEY=VALUE`.
-- **`extraProfileCreateArgs`** / **`extraDaemonStartArgs`** append raw args to the respective commands.
-
-## Development against a local checkout
-
-If you're hacking on the Python `hindsight-embed` package in the same monorepo, point the server at the local path — it'll use `uv run --directory <path>` instead of `uvx`:
-
-```ts
-new HindsightServer({
-  embedPackagePath: '/path/to/hindsight-embed',
-  // ...
-});
-```
-
-## API surface
-
-- `HindsightServer` — daemon lifecycle (`start`, `stop`, `checkHealth`, `getBaseUrl`, `getProfile`).
-- `Logger` interface plus `silentLogger` (default) and `consoleLogger` helpers.
-- `getEmbedCommand(opts)` — low-level helper that returns the `[cmd, ...args]` tuple used to invoke the underlying Python CLI.
-
-For memory operations (retain, recall, reflect, bank management, stats) use [`@vectorize-io/hindsight-client`](https://www.npmjs.com/package/@vectorize-io/hindsight-client).
-
-## License
-
-MIT

hindsight-all-npm/package.json

@@ -1,57 +0,0 @@
-{
-  "name": "@vectorize-io/hindsight-all",
-  "version": "0.5.1",
-  "description": "Node.js programmatic lifecycle manager for Hindsight — embeds a local hindsight daemon in a Node application. Pair with @vectorize-io/hindsight-client for memory operations.",
-  "main": "dist/index.js",
-  "types": "dist/index.d.ts",
-  "type": "module",
-  "exports": {
-    ".": {
-      "types": "./dist/index.d.ts",
-      "import": "./dist/index.js"
-    }
-  },
-  "keywords": [
-    "hindsight",
-    "hindsight-all",
-    "memory",
-    "ai",
-    "agent",
-    "long-term-memory",
-    "llm",
-    "embedded-server"
-  ],
-  "author": "Vectorize <[email protected]>",
-  "license": "MIT",
-  "repository": {
-    "type": "git",
-    "url": "https://github.com/vectorize-io/hindsight.git",
-    "directory": "hindsight-all-npm"
-  },
-  "files": [
-    "dist",
-    "README.md"
-  ],
-  "scripts": {
-    "build": "tsup",
-    "dev": "tsup --watch",
-    "clean": "rm -rf dist",
-    "test": "vitest run src",
-    "test:watch": "vitest src",
-    "prepublishOnly": "npm run clean && npm run build"
-  },
-  "devDependencies": {
-    "@types/node": "^22.0.0",
-    "tsup": "^8.5.1",
-    "typescript": "^5.7.0",
-    "vitest": "^4.1.2"
-  },
-  "engines": {
-    "node": ">=22"
-  },
-  "overrides": {
-    "rollup": "^4.59.0",
-    "picomatch": ">=2.3.2 <3.0.0 || >=4.0.4",
-    "vite": ">=8.0.5"
-  }
-}

hindsight-all-npm/src/command.test.ts

@@ -1,32 +0,0 @@
-import { describe, it, expect } from 'vitest';
-import { getEmbedCommand } from './command.js';
-
-describe('getEmbedCommand', () => {
-  it('defaults to uvx hindsight-embed@latest', () => {
-    expect(getEmbedCommand()).toEqual(['uvx', 'hindsight-embed@latest']);
-  });
-
-  it('honours an explicit version', () => {
-    expect(getEmbedCommand({ embedVersion: '0.5.0' })).toEqual(['uvx', '[email protected]']);
-  });
-
-  it('treats an empty version as latest', () => {
-    expect(getEmbedCommand({ embedVersion: '' })).toEqual(['uvx', 'hindsight-embed@latest']);
-  });
-
-  it('uses uv run --directory when a local path is given', () => {
-    expect(getEmbedCommand({ embedPackagePath: '/abs/path' })).toEqual([
-      'uv',
-      'run',
-      '--directory',
-      '/abs/path',
-      'hindsight-embed',
-    ]);
-  });
-
-  it('local path takes precedence over version', () => {
-    expect(
-      getEmbedCommand({ embedPackagePath: '/abs/path', embedVersion: '0.5.0' }),
-    ).toEqual(['uv', 'run', '--directory', '/abs/path', 'hindsight-embed']);
-  });
-});

hindsight-all-npm/src/command.ts

@@ -1,25 +0,0 @@
-/**
- * Resolve the command that invokes the `hindsight-embed` Python CLI.
- *
- * - If `embedPackagePath` is set, runs the package from a local checkout via
- *   `uv run --directory <path> hindsight-embed`. Used for in-repo development.
- * - Otherwise runs it via `uvx hindsight-embed@<version>` so no global install
- *   is required.
- *
- * Returns the argv as `[command, ...baseArgs]` suitable for `spawn()` /
- * `execFile()` (never shell-interpolated).
- */
-export interface EmbedCommandOptions {
-  /** Version spec passed to uvx (e.g. "latest", "0.5.0"). Default: "latest". */
-  embedVersion?: string;
-  /** Local checkout path. When set, overrides `embedVersion` and uses `uv run`. */
-  embedPackagePath?: string;
-}
-
-export function getEmbedCommand(opts: EmbedCommandOptions = {}): string[] {
-  if (opts.embedPackagePath) {
-    return ['uv', 'run', '--directory', opts.embedPackagePath, 'hindsight-embed'];
-  }
-  const version = opts.embedVersion && opts.embedVersion.length > 0 ? opts.embedVersion : 'latest';
-  return ['uvx', `hindsight-embed@${version}`];
-}

hindsight-all-npm/src/index.ts

@@ -1,7 +0,0 @@
-export { HindsightServer } from './server.js';
-export { getEmbedCommand } from './command.js';
-export { silentLogger, consoleLogger } from './logger.js';
-
-export type { Logger } from './logger.js';
-export type { EmbedCommandOptions } from './command.js';
-export type { HindsightServerOptions } from './types.js';

hindsight-all-npm/src/logger.ts

@@ -1,29 +0,0 @@
-/**
- * Pluggable logger interface.
- *
- * This package does not own any logging infrastructure — consumers inject
- * whatever they want (console, pino, openclaw's logger, a no-op). The default
- * is silent so embedding this package never adds noise to an unrelated app.
- */
-export interface Logger {
-  debug(msg: string): void;
-  info(msg: string): void;
-  warn(msg: string): void;
-  error(msg: string): void;
-}
-
-/** Logger that drops every call. Used when no logger is passed. */
-export const silentLogger: Logger = {
-  debug: () => {},
-  info: () => {},
-  warn: () => {},
-  error: () => {},
-};
-
-/** Logger that writes to the standard console. Handy for CLIs and tests. */
-export const consoleLogger: Logger = {
-  debug: (msg) => console.debug(msg),
-  info: (msg) => console.log(msg),
-  warn: (msg) => console.warn(msg),
-  error: (msg) => console.error(msg),
-};

hindsight-all-npm/src/server.test.ts

@@ -1,35 +0,0 @@
-import { describe, it, expect } from 'vitest';
-import { HindsightServer } from './server.js';
-
-describe('HindsightServer construction', () => {
-  it('defaults base URL to http://127.0.0.1:8888', () => {
-    const server = new HindsightServer();
-    expect(server.getBaseUrl()).toBe('http://127.0.0.1:8888');
-    expect(server.getProfile()).toBe('default');
-  });
-
-  it('honours custom profile, port, and host', () => {
-    const server = new HindsightServer({ profile: 'app', port: 9077, host: '0.0.0.0' });
-    expect(server.getProfile()).toBe('app');
-    expect(server.getBaseUrl()).toBe('http://0.0.0.0:9077');
-  });
-
-  it('accepts open env pass-through without complaining about unknown keys', () => {
-    const server = new HindsightServer({
-      env: {
-        HINDSIGHT_API_LLM_PROVIDER: 'openai',
-        HINDSIGHT_API_LLM_MODEL: 'gpt-4o-mini',
-        // A field that does not exist today — should still be accepted
-        HINDSIGHT_FUTURE_FLAG: 'enabled',
-      },
-    });
-    expect(server).toBeInstanceOf(HindsightServer);
-  });
-
-  it('exposes checkHealth that returns false when no daemon is running', async () => {
-    // Random high port that nothing is listening on.
-    const server = new HindsightServer({ port: 1, readyTimeoutMs: 100 });
-    const healthy = await server.checkHealth();
-    expect(healthy).toBe(false);
-  });
-});

hindsight-all-npm/src/server.ts

@@ -1,322 +0,0 @@
-import { spawn } from 'child_process';
-import { getEmbedCommand } from './command.js';
-import { silentLogger } from './logger.js';
-import type { Logger } from './logger.js';
-import type { HindsightServerOptions } from './types.js';
-
-const DEFAULT_PORT = 8888;
-const DEFAULT_HOST = '127.0.0.1';
-const DEFAULT_PROFILE = 'default';
-const DEFAULT_READY_TIMEOUT_MS = 30_000;
-const DEFAULT_READY_POLL_INTERVAL_MS = 1_000;
-
-/**
- * Manages the lifecycle of a local Hindsight daemon from a Node.js process.
- *
- * On {@link start}, this class:
- *   1. Resolves the `hindsight-embed` command (via `uvx` or a local `uv run`).
- *   2. Runs `profile create <name> --merge --port <port> [--env K=V ...]`
- *      with every entry in {@link HindsightServerOptions.env} forwarded as
- *      an `--env` flag.
- *   3. Runs `daemon --profile <name> start` and waits for the start command
- *      to exit.
- *   4. Polls `http://host:port/health` until it returns `200` or the
- *      `readyTimeoutMs` budget is exhausted.
- *
- * On {@link stop}, it runs `daemon --profile <name> stop` and returns once
- * the command exits (or after a short grace period).
- *
- * This is the Node.js equivalent of the Python `hindsight-all` package's
- * `HindsightServer`: a thin programmatic lifecycle wrapper around the
- * Hindsight daemon. It does NOT ship an HTTP client — once `start()`
- * resolves, use `@vectorize-io/hindsight-client` against `getBaseUrl()` for
- * retain / recall / reflect.
- *
- * The class is deliberately transparent about the daemon: new CLI flags or
- * environment variables never require a code change here — callers can pass
- * them via `env`, `extraProfileCreateArgs`, or `extraDaemonStartArgs`.
- */
-export class HindsightServer {
-  private readonly profile: string;
-  private readonly port: number;
-  private readonly host: string;
-  private readonly baseUrl: string;
-  private readonly embedVersion: string | undefined;
-  private readonly embedPackagePath: string | undefined;
-  private readonly userEnv: Record<string, string | undefined>;
-  private readonly extraProfileCreateArgs: string[];
-  private readonly extraDaemonStartArgs: string[];
-  private readonly platformCpuWorkaround: boolean;
-  private readonly readyTimeoutMs: number;
-  private readonly readyPollIntervalMs: number;
-  private readonly logger: Logger;
-
-  constructor(opts: HindsightServerOptions = {}) {
-    this.profile = opts.profile ?? DEFAULT_PROFILE;
-    this.port = opts.port ?? DEFAULT_PORT;
-    this.host = opts.host ?? DEFAULT_HOST;
-    this.baseUrl = `http://${this.host}:${this.port}`;
-    this.embedVersion = opts.embedVersion;
-    this.embedPackagePath = opts.embedPackagePath;
-    this.userEnv = opts.env ?? {};
-    this.extraProfileCreateArgs = opts.extraProfileCreateArgs ?? [];
-    this.extraDaemonStartArgs = opts.extraDaemonStartArgs ?? [];
-    this.platformCpuWorkaround = opts.platformCpuWorkaround ?? (process.platform === 'darwin');
-    this.readyTimeoutMs = opts.readyTimeoutMs ?? DEFAULT_READY_TIMEOUT_MS;
-    this.readyPollIntervalMs = opts.readyPollIntervalMs ?? DEFAULT_READY_POLL_INTERVAL_MS;
-    this.logger = opts.logger ?? silentLogger;
-  }
-
-  /** The base URL the daemon listens on (`http://host:port`). */
-  getBaseUrl(): string {
-    return this.baseUrl;
-  }
-
-  /** The profile name this server operates on. */
-  getProfile(): string {
-    return this.profile;
-  }
-
-  /**
-   * Ensure the daemon is configured and running. Idempotent — the underlying
-   * `profile create --merge` and `daemon start` commands tolerate re-runs.
-   */
-  async start(): Promise<void> {
-    this.logger.info(`[hindsight] starting daemon for profile "${this.profile}"`);
-
-    const env = this.buildEnv();
-    await this.configureProfile(env);
-    await this.startDaemon(env);
-    await this.waitForReady();
-
-    this.logger.info(`[hindsight] daemon ready at ${this.baseUrl}`);
-  }
-
-  /** Stop the daemon. Never throws — logs and resolves even on failure. */
-  async stop(): Promise<void> {
-    this.logger.info(`[hindsight] stopping daemon for profile "${this.profile}"`);
-
-    const [cmd, ...baseArgs] = getEmbedCommand({
-      embedVersion: this.embedVersion,
-      embedPackagePath: this.embedPackagePath,
-    });
-    const args = [...baseArgs, 'daemon', '--profile', this.profile, 'stop'];
-
-    const child = spawn(cmd, args, { stdio: 'pipe' });
-    this.pipeOutput(child, 'daemon.stop');
-
-    await new Promise<void>((resolve) => {
-      const timeout = setTimeout(() => {
-        this.logger.warn(`[hindsight] daemon stop timed out after 5s`);
-        resolve();
-      }, 5_000);
-      child.on('exit', () => {
-        clearTimeout(timeout);
-        this.logger.info(`[hindsight] daemon stopped`);
-        resolve();
-      });
-      child.on('error', (err) => {
-        clearTimeout(timeout);
-        this.logger.warn(`[hindsight] error stopping daemon: ${err.message}`);
-        resolve();
-      });
-    });
-  }
-
-  /** Probe `/health` once with a short timeout. */
-  async checkHealth(): Promise<boolean> {
-    try {
-      const res = await fetch(`${this.baseUrl}/health`, {
-        signal: AbortSignal.timeout(2_000),
-      });
-      return res.ok;
-    } catch {
-      return false;
-    }
-  }
-
-  // -------------------------------------------------------------------------
-  // Internal
-  // -------------------------------------------------------------------------
-
-  /**
-   * Merge the process env, the caller-supplied `env`, and (on macOS) the
-   * embeddings CPU workaround. Caller-supplied values always win over the
-   * workaround; undefined values are dropped.
-   */
-  private buildEnv(): NodeJS.ProcessEnv {
-    const merged: NodeJS.ProcessEnv = { ...process.env };
-
-    if (this.platformCpuWorkaround && process.platform === 'darwin') {
-      merged['HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU'] = '1';
-      merged['HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU'] = '1';
-    }
-
-    for (const [key, value] of Object.entries(this.userEnv)) {
-      if (value !== undefined) {
-        merged[key] = value;
-      }
-    }
-
-    return merged;
-  }
-
-  /**
-   * Run `profile create <name> --merge --port <port> [--env K=V ...]`.
-   * Every entry in the merged env that was passed via {@link userEnv} (or
-   * auto-applied by the CPU workaround) is forwarded as `--env`.
-   */
-  private async configureProfile(env: NodeJS.ProcessEnv): Promise<void> {
-    this.logger.info(`[hindsight] configuring profile "${this.profile}"`);
-
-    const [cmd, ...baseArgs] = getEmbedCommand({
-      embedVersion: this.embedVersion,
-      embedPackagePath: this.embedPackagePath,
-    });
-    const createArgs = [
-      ...baseArgs,
-      'profile',
-      'create',
-      this.profile,
-      '--merge',
-      '--port',
-      String(this.port),
-    ];
-
-    // Forward every env var that the caller intended for the daemon as --env.
-    // We only forward keys the caller explicitly set (userEnv) plus the CPU
-    // workaround values — not the entire process.env, to avoid leaking random
-    // host state into profile config.
-    const envForProfile = this.collectProfileEnv(env);
-    for (const [key, value] of Object.entries(envForProfile)) {
-      createArgs.push('--env', `${key}=${value}`);
-    }
-
-    createArgs.push(...this.extraProfileCreateArgs);
-
-    await this.runCommand(cmd, createArgs, env, 'profile.create');
-  }
-
-  /** Collect only the env vars that should be written into the profile file. */
-  private collectProfileEnv(env: NodeJS.ProcessEnv): Record<string, string> {
-    const out: Record<string, string> = {};
-
-    // 1. User-supplied env — always forwarded.
-    for (const [key, value] of Object.entries(this.userEnv)) {
-      if (value !== undefined) {
-        out[key] = value;
-      }
-    }
-
-    // 2. CPU workaround — only if auto-applied and not already overridden.
-    if (this.platformCpuWorkaround && process.platform === 'darwin') {
-      const cpuKeys = [
-        'HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU',
-        'HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU',
-      ];
-      for (const key of cpuKeys) {
-        if (!(key in out) && env[key] !== undefined) {
-          out[key] = env[key] as string;
-        }
-      }
-    }
-
-    return out;
-  }
-
-  private async startDaemon(env: NodeJS.ProcessEnv): Promise<void> {
-    const [cmd, ...baseArgs] = getEmbedCommand({
-      embedVersion: this.embedVersion,
-      embedPackagePath: this.embedPackagePath,
-    });
-    const args = [
-      ...baseArgs,
-      'daemon',
-      '--profile',
-      this.profile,
-      'start',
-      ...this.extraDaemonStartArgs,
-    ];
-
-    await this.runCommand(cmd, args, env, 'daemon.start');
-  }
-
-  /**
-   * Spawn `cmd` with `args`, pipe its output through the logger, and resolve
-   * once it exits with code 0. Rejects on non-zero exit or spawn error.
-   */
-  private async runCommand(
-    cmd: string,
-    args: string[],
-    env: NodeJS.ProcessEnv,
-    label: string,
-  ): Promise<void> {
-    const child = spawn(cmd, args, { stdio: 'pipe', env });
-    let output = '';
-    child.stdout?.on('data', (data: Buffer) => {
-      const text = data.toString();
-      output += text;
-      for (const line of text.trimEnd().split('\n')) {
-        if (line) this.logger.info(`[hindsight:${label}] ${line}`);
-      }
-    });
-    child.stderr?.on('data', (data: Buffer) => {
-      const text = data.toString();
-      output += text;
-      for (const line of text.trimEnd().split('\n')) {
-        if (line) this.logger.warn(`[hindsight:${label}] ${line}`);
-      }
-    });
-
-    await new Promise<void>((resolve, reject) => {
-      child.on('exit', (code) => {
-        if (code === 0) {
-          resolve();
-        } else {
-          reject(new Error(`${label} failed with code ${code}: ${output.trim()}`));
-        }
-      });
-      child.on('error', (err) => {
-        reject(new Error(`${label} failed to spawn: ${err.message}`, { cause: err }));
-      });
-    });
-  }
-
-  /** Stream a spawned child's stdout/stderr through the logger without blocking. */
-  private pipeOutput(child: ReturnType<typeof spawn>, label: string): void {
-    child.stdout?.on('data', (data: Buffer) => {
-      for (const line of data.toString().trimEnd().split('\n')) {
-        if (line) this.logger.info(`[hindsight:${label}] ${line}`);
-      }
-    });
-    child.stderr?.on('data', (data: Buffer) => {
-      for (const line of data.toString().trimEnd().split('\n')) {
-        if (line) this.logger.warn(`[hindsight:${label}] ${line}`);
-      }
-    });
-  }
-
-  /** Poll `/health` until it succeeds or `readyTimeoutMs` elapses. */
-  private async waitForReady(): Promise<void> {
-    const deadline = Date.now() + this.readyTimeoutMs;
-    let attempt = 0;
-    while (Date.now() < deadline) {
-      attempt++;
-      try {
-        const res = await fetch(`${this.baseUrl}/health`, {
-          signal: AbortSignal.timeout(this.readyPollIntervalMs),
-        });
-        if (res.ok) {
-          this.logger.debug(`[hindsight] health check passed (attempt ${attempt})`);
-          return;
-        }
-      } catch {
-        // expected while the daemon is still booting
-      }
-      await new Promise((resolve) => setTimeout(resolve, this.readyPollIntervalMs));
-    }
-    throw new Error(
-      `Hindsight daemon did not become ready within ${this.readyTimeoutMs}ms at ${this.baseUrl}`,
-    );
-  }
-}

hindsight-all-npm/src/types.ts

@@ -1,54 +0,0 @@
-import type { Logger } from './logger.js';
-
-/**
- * Options for {@link HindsightServer}.
- *
- * The server is intentionally thin and pass-through: anything configurable
- * on the daemon side (env vars or CLI flags) can be set here without needing
- * a new dedicated option. Use {@link env} for `HINDSIGHT_*` / `OPENAI_API_KEY` /
- * custom provider settings, and the two `extra*` arrays to append raw CLI
- * args to `profile create` or `daemon start`.
- *
- * For talking to the daemon after `start()`, use `@vectorize-io/hindsight-client`
- * against `server.getBaseUrl()`. This package does not ship its own HTTP
- * client.
- */
-export interface HindsightServerOptions {
-  /** Profile name used for `--profile <name>` on every sub-command. Default: `"default"`. */
-  profile?: string;
-  /** TCP port the daemon listens on. Default: `8888`. */
-  port?: number;
-  /** Hostname the daemon binds to (for health checks). Default: `127.0.0.1`. */
-  host?: string;
-  /** Version of the underlying `hindsight-embed` PyPI package to run via `uvx`. Default: `"latest"`. */
-  embedVersion?: string;
-  /** Local path to a `hindsight-embed` checkout — takes precedence over `embedVersion`. */
-  embedPackagePath?: string;
-  /**
-   * Environment variables passed to the daemon process AND written into the
-   * profile via repeated `--env KEY=VALUE` flags. This is the preferred way
-   * to surface any `HINDSIGHT_API_*` / `HINDSIGHT_EMBED_*` setting — adding a
-   * new daemon env var never requires a wrapper update.
-   *
-   * Values of `undefined` are dropped (so you can spread conditionally).
-   */
-  env?: Record<string, string | undefined>;
-  /** Extra args appended verbatim to `hindsight-embed profile create <name> --merge ...`. */
-  extraProfileCreateArgs?: string[];
-  /** Extra args appended verbatim to `hindsight-embed daemon --profile <name> start ...`. */
-  extraDaemonStartArgs?: string[];
-  /**
-   * On macOS, automatically set
-   * `HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU=1` and
-   * `HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU=1` to avoid Metal/MPS crashes in
-   * daemon mode. Default: `true` on `darwin`, ignored elsewhere. Any value set
-   * explicitly in {@link env} wins over the auto-applied value.
-   */
-  platformCpuWorkaround?: boolean;
-  /** Max time (ms) to wait for `/health` to return 200. Default: `30_000`. */
-  readyTimeoutMs?: number;
-  /** Polling interval (ms) while waiting for `/health`. Default: `1_000`. */
-  readyPollIntervalMs?: number;
-  /** Optional pluggable logger. Default: silent. */
-  logger?: Logger;
-}

hindsight-all-npm/tsconfig.json

@@ -1,18 +0,0 @@
-{
-  "compilerOptions": {
-    "target": "ES2022",
-    "module": "ES2022",
-    "lib": ["ES2022"],
-    "moduleResolution": "node",
-    "declaration": true,
-    "outDir": "./dist",
-    "rootDir": "./src",
-    "strict": true,
-    "esModuleInterop": true,
-    "skipLibCheck": true,
-    "forceConsistentCasingInFileNames": true,
-    "resolveJsonModule": true
-  },
-  "include": ["src/**/*"],
-  "exclude": ["node_modules", "dist", "src/**/*.test.ts"]
-}

hindsight-all-npm/tsup.config.ts

@@ -1,11 +0,0 @@
-import { defineConfig } from 'tsup';
-
-export default defineConfig({
-  entry: ['src/index.ts'],
-  format: ['esm'],
-  dts: true,
-  outDir: 'dist',
-  clean: true,
-  sourcemap: true,
-  bundle: true,
-});

hindsight-all-npm/vitest.config.ts

@@ -1,8 +0,0 @@
-import { defineConfig } from 'vitest/config';
-
-export default defineConfig({
-  test: {
-    include: ['src/**/*.test.ts'],
-    environment: 'node',
-  },
-});

hindsight-all-slim/pyproject.toml

@@ -1,33 +0,0 @@
-[build-system]
-requires = ["setuptools>=61"]
-build-backend = "setuptools.build_meta"
-
-[project]
-name = "hindsight-all-slim"
-version = "0.5.1"
-description = "Hindsight: Agent Memory That Works Like Human Memory - Slim All-in-One Bundle"
-readme = "README.md"
-requires-python = ">=3.11"
-dependencies = [
-    "hindsight-api-slim>=0.4.17",
-    "hindsight-client>=0.0.7",
-    "hindsight-embed>=0.1.0",
-]
-
-[tool.uv.sources]
-hindsight-api-slim = { workspace = true }
-hindsight-client = { workspace = true }
-hindsight-embed = { workspace = true }
-
-[project.optional-dependencies]
-test = [
-    "pytest>=7.0.0",
-    "pytest-asyncio>=0.21.0",
-]
-
-[tool.setuptools]
-packages = []
-
-[tool.pytest.ini_options]
-asyncio_mode = "auto"
-asyncio_default_fixture_loop_scope = "function"

hindsight-all/README.md

@@ -1,48 +0,0 @@
-# hindsight-all
-
-All-in-one package for Hindsight - Agent Memory That Works Like Human Memory
-
-## Quick Start
-
-```python
-from hindsight import start_server, HindsightClient
-
-# Start server with embedded PostgreSQL
-server = start_server(
-    llm_provider="groq",
-    llm_api_key="your-api-key",
-    llm_model="openai/gpt-oss-120b"
-)
-
-# Create client
-client = HindsightClient(base_url=server.url)
-
-# Store memories
-client.put(agent_id="assistant", content="User prefers Python for data analysis")
-
-# Search memories
-results = client.search(agent_id="assistant", query="programming preferences")
-
-# Generate contextual response
-response = client.think(agent_id="assistant", query="What languages should I recommend?")
-
-# Stop server when done
-server.stop()
-```
-
-## Using Context Manager
-
-```python
-from hindsight import HindsightServer, HindsightClient
-
-with HindsightServer(llm_provider="groq", llm_api_key="...") as server:
-    client = HindsightClient(base_url=server.url)
-    # ... use client ...
-# Server automatically stops
-```
-
-## Installation
-
-```bash
-pip install hindsight-all
-```

hindsight-all/hindsight/client_wrapper.py

@@ -1,423 +0,0 @@
-"""
-Wrapper for Hindsight client that adds API namespaces.
-
-Provides organized access to different parts of the Hindsight API through
-namespaces like .banks, .mental_models, etc.
-"""
-
-from __future__ import annotations
-
-from typing import Any
-
-from hindsight_client import Hindsight
-
-
-class BanksAPI:
-    """Namespace for bank-related operations.
-    
-    Provides methods to create, delete, and manage memory banks.
-    """
-
-    def __init__(self, client: Hindsight):
-        self._client = client
-
-    def create(
-        self,
-        bank_id: str,
-        name: str | None = None,
-        mission: str | None = None,
-        disposition: dict[str, Any] | None = None,
-    ) -> Any:
-        """Create a new bank.
-        
-        Args:
-            bank_id: Unique identifier for the bank.
-            name: Optional display name for the bank.
-            mission: Optional mission statement for the bank.
-            disposition: Optional disposition configuration dict.
-            
-        Returns:
-            Bank creation response from the API.
-        """
-        return self._client.create_bank(
-            bank_id=bank_id,
-            name=name,
-            mission=mission,
-            disposition=disposition,
-        )
-
-    def delete(self, bank_id: str) -> Any:
-        """Delete a bank.
-        
-        Args:
-            bank_id: The ID of the bank to delete.
-            
-        Returns:
-            Deletion response from the API.
-        """
-        return self._client.delete_bank(bank_id=bank_id)
-
-    def set_mission(self, bank_id: str, mission: str) -> Any:
-        """Set or update the mission for a bank.
-        
-        Args:
-            bank_id: The ID of the bank.
-            mission: The mission statement to set.
-            
-        Returns:
-            API response confirming the update.
-        """
-        return self._client.set_mission(bank_id=bank_id, mission=mission)
-
-    def set_disposition(self, bank_id: str, disposition: dict[str, Any]) -> Any:
-        """Set or update the disposition for a bank.
-        
-        Args:
-            bank_id: The ID of the bank.
-            disposition: The disposition configuration dict.
-            
-        Returns:
-            API response confirming the update.
-        """
-        return self._client.set_disposition(bank_id=bank_id, disposition=disposition)
-
-    def list(self) -> Any:
-        """List all banks.
-        
-        Returns:
-            List of banks from the API.
-        """
-        from hindsight_client.hindsight_client import _run_async
-
-        return _run_async(self._client._banks_api.list_banks())
-
-
-class MentalModelsAPI:
-    """Namespace for mental model operations.
-    
-    Mental models are reusable knowledge structures that guide agent behavior.
-    """
-
-    def __init__(self, client: Hindsight):
-        self._client = client
-
-    def create(
-        self,
-        bank_id: str,
-        name: str,
-        content: str,
-        tags: list[str] | None = None,
-    ) -> Any:
-        """Create a new mental model.
-        
-        Args:
-            bank_id: The ID of the bank to add the model to.
-            name: Name for the mental model.
-            content: The content/instructions for the mental model.
-            tags: Optional list of tags for categorization.
-            
-        Returns:
-            Creation response from the API.
-        """
-        return self._client.create_mental_model(
-            bank_id=bank_id,
-            name=name,
-            content=content,
-            tags=tags,
-        )
-
-    def list(self, bank_id: str, tags: list[str] | None = None) -> Any:
-        """List all mental models for a bank.
-        
-        Args:
-            bank_id: The ID of the bank.
-            tags: Optional filter by tags.
-            
-        Returns:
-            List of mental models.
-        """
-        return self._client.list_mental_models(bank_id=bank_id, tags=tags)
-
-    def get(self, bank_id: str, mental_model_id: str) -> Any:
-        """Get a specific mental model.
-        
-        Args:
-            bank_id: The ID of the bank.
-            mental_model_id: The ID of the mental model.
-            
-        Returns:
-            The mental model details.
-        """
-        return self._client.get_mental_model(bank_id=bank_id, mental_model_id=mental_model_id)
-
-    def refresh(self, bank_id: str, mental_model_id: str) -> Any:
-        """Refresh a mental model.
-        
-        Args:
-            bank_id: The ID of the bank.
-            mental_model_id: The ID of the mental model to refresh.
-            
-        Returns:
-            Refresh response from the API.
-        """
-        return self._client.refresh_mental_model(bank_id=bank_id, mental_model_id=mental_model_id)
-
-    def update(
-        self,
-        bank_id: str,
-        mental_model_id: str,
-        name: str | None = None,
-        content: str | None = None,
-        tags: list[str] | None = None,
-    ) -> Any:
-        """Update a mental model.
-        
-        Args:
-            bank_id: The ID of the bank.
-            mental_model_id: The ID of the mental model to update.
-            name: Optional new name.
-            content: Optional new content.
-            tags: Optional new tags list.
-            
-        Returns:
-            Update response from the API.
-        """
-        return self._client.update_mental_model(
-            bank_id=bank_id,
-            mental_model_id=mental_model_id,
-            name=name,
-            content=content,
-            tags=tags,
-        )
-
-    def delete(self, bank_id: str, mental_model_id: str) -> Any:
-        """Delete a mental model.
-        
-        Args:
-            bank_id: The ID of the bank.
-            mental_model_id: The ID of the mental model to delete.
-            
-        Returns:
-            Deletion response from the API.
-        """
-        return self._client.delete_mental_model(bank_id=bank_id, mental_model_id=mental_model_id)
-
-
-class DirectivesAPI:
-    """Namespace for directive operations.
-    
-    Directives are explicit instructions that guide agent behavior.
-    """
-
-    def __init__(self, client: Hindsight):
-        self._client = client
-
-    def create(
-        self,
-        bank_id: str,
-        name: str,
-        content: str,
-        tags: list[str] | None = None,
-    ) -> Any:
-        """Create a new directive.
-        
-        Args:
-            bank_id: The ID of the bank to add the directive to.
-            name: Name for the directive.
-            content: The directive content/instructions.
-            tags: Optional list of tags for categorization.
-            
-        Returns:
-            Creation response from the API.
-        """
-        return self._client.create_directive(
-            bank_id=bank_id,
-            name=name,
-            content=content,
-            tags=tags,
-        )
-
-    def list(self, bank_id: str, tags: list[str] | None = None) -> Any:
-        """List all directives for a bank.
-        
-        Args:
-            bank_id: The ID of the bank.
-            tags: Optional filter by tags.
-            
-        Returns:
-            List of directives.
-        """
-        return self._client.list_directives(bank_id=bank_id, tags=tags)
-
-    def get(self, bank_id: str, directive_id: str) -> Any:
-        """Get a specific directive.
-        
-        Args:
-            bank_id: The ID of the bank.
-            directive_id: The ID of the directive.
-            
-        Returns:
-            The directive details.
-        """
-        return self._client.get_directive(bank_id=bank_id, directive_id=directive_id)
-
-    def update(
-        self,
-        bank_id: str,
-        directive_id: str,
-        name: str | None = None,
-        content: str | None = None,
-        tags: list[str] | None = None,
-    ) -> Any:
-        """Update a directive.
-        
-        Args:
-            bank_id: The ID of the bank.
-            directive_id: The ID of the directive to update.
-            name: Optional new name.
-            content: Optional new content.
-            tags: Optional new tags list.
-            
-        Returns:
-            Update response from the API.
-        """
-        return self._client.update_directive(
-            bank_id=bank_id,
-            directive_id=directive_id,
-            name=name,
-            content=content,
-            tags=tags,
-        )
-
-    def delete(self, bank_id: str, directive_id: str) -> Any:
-        """Delete a directive.
-        
-        Args:
-            bank_id: The ID of the bank.
-            directive_id: The ID of the directive to delete.
-            
-        Returns:
-            Deletion response from the API.
-        """
-        return self._client.delete_directive(bank_id=bank_id, directive_id=directive_id)
-
-
-class MemoriesAPI:
-    """Namespace for memory operations.
-    
-    Provides methods to query and retrieve stored memories.
-    """
-
-    def __init__(self, client: Hindsight):
-        self._client = client
-
-    def list(
-        self,
-        bank_id: str,
-        type: str | None = None,
-        search_query: str | None = None,
-        limit: int = 100,
-        offset: int = 0,
-    ) -> Any:
-        """List memories in a bank.
-        
-        Args:
-            bank_id: The ID of the bank to query.
-            type: Optional filter by memory type.
-            search_query: Optional search query for filtering.
-            limit: Maximum number of results to return (default: 100).
-            offset: Number of results to skip for pagination (default: 0).
-            
-        Returns:
-            List of memories matching the criteria.
-        """
-        return self._client.list_memories(
-            bank_id=bank_id,
-            type=type,
-            search_query=search_query,
-            limit=limit,
-            offset=offset,
-        )
-
-
-class HindsightClient(Hindsight):
-    """
-    Enhanced Hindsight client with organized API namespaces.
-
-    This wrapper extends the auto-generated Hindsight client with organized
-    access to different parts of the API through namespaces.
-
-    Example:
-        ```python
-        from hindsight import HindsightClient
-
-        client = HindsightClient(base_url="http://localhost:8888")
-
-        # Core operations (inherited from Hindsight)
-        client.retain(bank_id="test", content="Hello")
-        results = client.recall(bank_id="test", query="Hello")
-
-        # Organized API access through namespaces
-        client.banks.create(bank_id="test", name="Test Bank")
-        models = client.mental_models.list(bank_id="test")
-        directives = client.directives.list(bank_id="test")
-        memories = client.memories.list(bank_id="test")
-        ```
-        
-    Attributes:
-        banks: Namespace for bank management operations.
-        mental_models: Namespace for mental model operations.
-        directives: Namespace for directive operations.
-        memories: Namespace for memory listing operations.
-    """
-
-    def __init__(self, *args: Any, **kwargs: Any) -> None:
-        super().__init__(*args, **kwargs)
-        self._banks_namespace: BanksAPI | None = None
-        self._mental_models_namespace: MentalModelsAPI | None = None
-        self._directives_namespace: DirectivesAPI | None = None
-        self._memories_namespace: MemoriesAPI | None = None
-
-    @property
-    def banks(self) -> BanksAPI:
-        """Access bank management operations.
-        
-        Returns:
-            BanksAPI instance for bank operations.
-        """
-        if self._banks_namespace is None:
-            self._banks_namespace = BanksAPI(self)
-        return self._banks_namespace
-
-    @property
-    def mental_models(self) -> MentalModelsAPI:
-        """Access mental model operations.
-        
-        Returns:
-            MentalModelsAPI instance for mental model operations.
-        """
-        if self._mental_models_namespace is None:
-            self._mental_models_namespace = MentalModelsAPI(self)
-        return self._mental_models_namespace
-
-    @property
-    def directives(self) -> DirectivesAPI:
-        """Access directive operations.
-        
-        Returns:
-            DirectivesAPI instance for directive operations.
-        """
-        if self._directives_namespace is None:
-            self._directives_namespace = DirectivesAPI(self)
-        return self._directives_namespace
-
-    @property
-    def memories(self) -> MemoriesAPI:
-        """Access memory listing operations.
-        
-        Returns:
-            MemoriesAPI instance for memory operations.
-        """
-        if self._memories_namespace is None:
-            self._memories_namespace = MemoriesAPI(self)
-        return self._memories_namespace

hindsight-all/tests/test_cleanup_timeout.py

@@ -1,56 +0,0 @@
-"""
-Unit test for _cleanup lock timeout behavior.
-
-Verifies that _cleanup completes even when the lock is held by another thread,
-instead of hanging indefinitely (fixes #952).
-"""
-
-import threading
-import time
-from unittest.mock import MagicMock, patch
-
-import pytest
-
-
-def test_cleanup_completes_when_lock_held():
-    """
-    _cleanup should complete (best-effort) even when self._lock is held
-    by another thread, e.g. during a long _ensure_started call.
-    """
-    with patch.dict("sys.modules", {
-        "hindsight_client": MagicMock(),
-        "hindsight_embed": MagicMock(),
-        "hindsight.api_namespaces": MagicMock(),
-    }):
-        from hindsight.embedded import HindsightEmbedded
-
-        client = HindsightEmbedded.__new__(HindsightEmbedded)
-        client.profile = "test"
-        client._lock = threading.Lock()
-        client._closed = False
-        client._client = None
-        client._started = False
-        client._ui = False
-
-        # Simulate another thread holding the lock
-        client._lock.acquire()
-
-        cleanup_done = threading.Event()
-
-        def run_cleanup():
-            client._cleanup()
-            cleanup_done.set()
-
-        t = threading.Thread(target=run_cleanup)
-        t.start()
-
-        # Cleanup should complete within the timeout (5s) + margin
-        assert cleanup_done.wait(timeout=8.0), (
-            "_cleanup hung instead of timing out on lock acquisition"
-        )
-
-        # Release the lock from the simulating thread
-        client._lock.release()
-        t.join(timeout=1.0)
-
-        assert client._closed, "Client should be marked as closed after cleanup"

hindsight-api-slim/README.md

@@ -1,137 +0,0 @@
-# Hindsight API
-
-**Memory System for AI Agents** — Temporal + Semantic + Entity Memory Architecture using PostgreSQL with pgvector.
-
-Hindsight gives AI agents persistent memory that works like human memory: it stores facts, tracks entities and relationships, handles temporal reasoning ("what happened last spring?"), and forms opinions based on configurable disposition traits.
-
-## Installation
-
-```bash
-pip install hindsight-api
-```
-
-## Quick Start
-
-### Run the Server
-
-```bash
-# Set your LLM provider
-export HINDSIGHT_API_LLM_PROVIDER=openai
-export HINDSIGHT_API_LLM_API_KEY=sk-xxxxxxxxxxxx
-
-# Start the server (uses embedded PostgreSQL by default)
-hindsight-api
-```
-
-The server starts at http://localhost:8888 with:
-- REST API for memory operations
-- MCP server at `/mcp` for tool-use integration
-
-### Use the Python API
-
-```python
-from hindsight_api import MemoryEngine
-
-# Create and initialize the memory engine
-memory = MemoryEngine()
-await memory.initialize()
-
-# Create a memory bank for your agent
-bank = await memory.create_memory_bank(
-    name="my-assistant",
-    background="A helpful coding assistant"
-)
-
-# Store a memory
-await memory.retain(
-    memory_bank_id=bank.id,
-    content="The user prefers Python for data science projects"
-)
-
-# Recall memories
-results = await memory.recall(
-    memory_bank_id=bank.id,
-    query="What programming language does the user prefer?"
-)
-
-# Reflect with reasoning
-response = await memory.reflect(
-    memory_bank_id=bank.id,
-    query="Should I recommend Python or R for this ML project?"
-)
-```
-
-## CLI Options
-
-```bash
-hindsight-api --help
-
-# Common options
-hindsight-api --port 9000          # Custom port (default: 8888)
-hindsight-api --host 127.0.0.1     # Bind to localhost only
-hindsight-api --workers 4          # Multiple worker processes
-hindsight-api --log-level debug    # Verbose logging
-```
-
-## Configuration
-
-Configure via environment variables:
-
-| Variable | Description | Default |
-|----------|-------------|---------|
-| `HINDSIGHT_API_DATABASE_URL` | PostgreSQL connection string | `pg0` (embedded) |
-| `HINDSIGHT_API_LLM_PROVIDER` | `openai`, `anthropic`, `gemini`, `groq`, `ollama`, `lmstudio` | `openai` |
-| `HINDSIGHT_API_LLM_API_KEY` | API key for LLM provider | - |
-| `HINDSIGHT_API_LLM_MODEL` | Model name | `gpt-4o-mini` |
-| `HINDSIGHT_API_HOST` | Server bind address | `0.0.0.0` |
-| `HINDSIGHT_API_PORT` | Server port | `8888` |
-
-### Example with External PostgreSQL
-
-```bash
-export HINDSIGHT_API_DATABASE_URL=postgresql://user:pass@localhost:5432/hindsight
-export HINDSIGHT_API_LLM_PROVIDER=groq
-export HINDSIGHT_API_LLM_API_KEY=gsk_xxxxxxxxxxxx
-
-hindsight-api
-```
-
-## Docker
-
-```bash
-docker run --rm -it -p 8888:8888 \
-  -e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \
-  -v $HOME/.hindsight-docker:/home/hindsight/.pg0 \
-  ghcr.io/vectorize-io/hindsight:latest
-```
-
-## MCP Server
-
-For local MCP integration without running the full API server:
-
-```bash
-hindsight-local-mcp
-```
-
-This runs a stdio-based MCP server that can be used directly with MCP-compatible clients.
-
-## Key Features
-
-- **Multi-Strategy Retrieval (TEMPR)** — Semantic, keyword, graph, and temporal search combined with RRF fusion
-- **Entity Graph** — Automatic entity extraction and relationship tracking
-- **Temporal Reasoning** — Native support for time-based queries
-- **Disposition Traits** — Configurable skepticism, literalism, and empathy influence opinion formation
-- **Three Memory Types** — World facts, bank actions, and formed opinions with confidence scores
-
-## Documentation
-
-Full documentation: [https://hindsight.vectorize.io](https://hindsight.vectorize.io)
-
-- [Installation Guide](https://hindsight.vectorize.io/developer/installation)
-- [Configuration Reference](https://hindsight.vectorize.io/developer/configuration)
-- [API Reference](https://hindsight.vectorize.io/api-reference)
-- [Python SDK](https://hindsight.vectorize.io/sdks/python)
-
-## License
-
-Apache 2.0

hindsight-api-slim/hindsight_api/alembic/versions/2eee35aa3cfc_case_insensitive_entities_trgm_index.py

@@ -1,45 +0,0 @@
-"""Recreate entities trigram index on LOWER(canonical_name) for case-insensitive matching
-
-The previous GIN trigram index on canonical_name was case-sensitive, causing
-"Alice" and "alice" to have different trigram sets. This recreates it on
-LOWER(canonical_name) so the % operator matches case-insensitively.
-
-Revision ID: d6e7f8a9b0c1
-Revises: c5d6e7f8a9b0
-Create Date: 2026-03-31
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "d6e7f8a9b0c1"
-down_revision: str | Sequence[str] | None = "c5d6e7f8a9b0"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-    # Drop the old case-sensitive trigram index
-    op.execute("DROP INDEX IF EXISTS entities_canonical_name_trgm_idx")
-    # Create case-insensitive trigram index on LOWER(canonical_name)
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS entities_canonical_name_lower_trgm_idx "
-        f"ON {schema}entities USING GIN (LOWER(canonical_name) gin_trgm_ops)"
-    )
-
-
-def downgrade() -> None:
-    op.execute("DROP INDEX IF EXISTS entities_canonical_name_lower_trgm_idx")
-    schema = _get_schema_prefix()
-    # Restore original case-sensitive index
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS entities_canonical_name_trgm_idx "
-        f"ON {schema}entities USING GIN (canonical_name gin_trgm_ops)"
-    )

hindsight-api-slim/hindsight_api/alembic/versions/a3b4c5d6e7f8_add_consolidation_failed_at_to_memory_units.py

@@ -1,52 +0,0 @@
-"""Add consolidation_failed_at column to memory_units for tracking persistent LLM failures.
-
-When all LLM retries are exhausted on a single-memory batch, the memory is marked
-with consolidation_failed_at instead of consolidated_at, so it is not silently lost
-and can be retried later via the API.
-
-Revision ID: a3b4c5d6e7f8
-Revises: g7h8i9j0k1l2
-Create Date: 2026-03-17
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "a3b4c5d6e7f8"
-down_revision: str | Sequence[str] | None = "g7h8i9j0k1l2"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Get schema prefix for table names (required for multi-tenant support)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    op.execute(
-        f"""
-        ALTER TABLE {schema}memory_units
-        ADD COLUMN IF NOT EXISTS consolidation_failed_at TIMESTAMPTZ DEFAULT NULL
-        """
-    )
-
-    # Index to efficiently query memories that failed consolidation for a given bank
-    op.execute(
-        f"""
-        CREATE INDEX IF NOT EXISTS idx_memory_units_consolidation_failed
-        ON {schema}memory_units (bank_id, consolidation_failed_at)
-        WHERE consolidation_failed_at IS NOT NULL AND fact_type IN ('experience', 'world')
-        """
-    )
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_units_consolidation_failed")
-    op.execute(f"ALTER TABLE {schema}memory_units DROP COLUMN IF EXISTS consolidation_failed_at")

hindsight-api-slim/hindsight_api/alembic/versions/a4b5c6d7e8f9_fix_per_bank_vector_index_type.py

@@ -1,142 +0,0 @@
-"""Fix per-bank vector indexes to match configured extension
-
-Revision ID: a4b5c6d7e8f9
-Revises: d6e7f8a9b0c1
-Create Date: 2026-04-01
-
-Migration d5e6f7a8b9c0 hardcoded HNSW when creating per-bank partial vector
-indexes, ignoring HINDSIGHT_API_VECTOR_EXTENSION. Banks that existed when that
-migration ran got HNSW indexes even when pgvectorscale (DiskANN) or vchord
-was configured.
-
-This migration detects the mismatch and recreates the affected indexes with
-the correct type. Skipped entirely when the configured extension is pgvector
-(the default), since those indexes are already correct.
-"""
-
-import os
-from collections.abc import Sequence
-
-from alembic import context, op
-from sqlalchemy import text
-
-revision: str = "a4b5c6d7e8f9"
-down_revision: str | Sequence[str] | None = "d6e7f8a9b0c1"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-_FACT_TYPES: dict[str, str] = {
-    "world": "worl",
-    "experience": "expr",
-    "observation": "obsv",
-}
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _target_index_type() -> str | None:
-    """Return the target index type, or None if pgvector (no fix needed)."""
-    ext = os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector").lower()
-    if ext == "pgvectorscale":
-        return "diskann"
-    elif ext == "vchord":
-        return "vchordrq"
-    return None
-
-
-def _vector_index_using_clause() -> str:
-    """Return the USING clause based on the configured vector extension."""
-    ext = os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector").lower()
-    if ext == "pgvectorscale":
-        return "USING diskann (embedding vector_cosine_ops) WITH (num_neighbors = 50)"
-    elif ext == "vchord":
-        return "USING vchordrq (embedding vector_l2_ops)"
-    else:
-        return "USING hnsw (embedding vector_cosine_ops)"
-
-
-def upgrade() -> None:
-    target = _target_index_type()
-    if target is None:
-        # pgvector — indexes are already HNSW, nothing to fix
-        return
-
-    bind = op.get_bind()
-    schema_name = context.config.get_main_option("target_schema")
-    schema = _get_schema_prefix()
-    table_ref = f'"{schema_name}".memory_units' if schema_name else "memory_units"
-    banks_ref = f'"{schema_name}".banks' if schema_name else "banks"
-    using_clause = _vector_index_using_clause()
-    pg_schema = schema_name or "public"
-
-    rows = bind.execute(text(f"SELECT bank_id, internal_id FROM {banks_ref}")).fetchall()  # noqa: S608
-    for row in rows:
-        bank_id = row[0]
-        internal_id = str(row[1]).replace("-", "")[:16]
-        escaped_bank_id = bank_id.replace("'", "''")
-        for ft, ft_short in _FACT_TYPES.items():
-            idx_name = f"idx_mu_emb_{ft_short}_{internal_id}"
-
-            # Check if this index exists and what type it is
-            idx_info = bind.execute(
-                text("SELECT indexdef FROM pg_indexes WHERE schemaname = :schema AND indexname = :idx"),
-                {"schema": pg_schema, "idx": idx_name},
-            ).fetchone()
-
-            if idx_info is None:
-                # Index doesn't exist — create it with the correct type
-                bind.execute(
-                    text(
-                        f"CREATE INDEX IF NOT EXISTS {idx_name} "
-                        f"ON {table_ref} {using_clause} "
-                        f"WHERE fact_type = '{ft}' AND bank_id = '{escaped_bank_id}'"
-                    )
-                )
-                continue
-
-            indexdef = idx_info[0].lower()
-            if target in indexdef:
-                # Already the correct type
-                continue
-
-            # Wrong type — drop and recreate
-            bind.execute(text(f"DROP INDEX IF EXISTS {schema}{idx_name}"))
-            bind.execute(
-                text(
-                    f"CREATE INDEX IF NOT EXISTS {idx_name} "
-                    f"ON {table_ref} {using_clause} "
-                    f"WHERE fact_type = '{ft}' AND bank_id = '{escaped_bank_id}'"
-                )
-            )
-
-
-def downgrade() -> None:
-    # Downgrade recreates indexes as HNSW (the original hardcoded behavior)
-    target = _target_index_type()
-    if target is None:
-        return
-
-    bind = op.get_bind()
-    schema_name = context.config.get_main_option("target_schema")
-    schema = _get_schema_prefix()
-    table_ref = f'"{schema_name}".memory_units' if schema_name else "memory_units"
-    banks_ref = f'"{schema_name}".banks' if schema_name else "banks"
-
-    rows = bind.execute(text(f"SELECT bank_id, internal_id FROM {banks_ref}")).fetchall()  # noqa: S608
-    for row in rows:
-        bank_id = row[0]
-        internal_id = str(row[1]).replace("-", "")[:16]
-        escaped_bank_id = bank_id.replace("'", "''")
-        for ft, ft_short in _FACT_TYPES.items():
-            idx_name = f"idx_mu_emb_{ft_short}_{internal_id}"
-            bind.execute(text(f"DROP INDEX IF EXISTS {schema}{idx_name}"))
-            bind.execute(
-                text(
-                    f"CREATE INDEX IF NOT EXISTS {idx_name} "
-                    f"ON {table_ref} USING hnsw (embedding vector_cosine_ops) "
-                    f"WHERE fact_type = '{ft}' AND bank_id = '{escaped_bank_id}'"
-                )
-            )

hindsight-api-slim/hindsight_api/alembic/versions/b3c4d5e6f7a8_add_content_hash_to_chunks.py

@@ -1,32 +0,0 @@
-"""add content_hash to chunks table for delta retain
-
-Revision ID: b3c4d5e6f7a8
-Revises: a3b4c5d6e7f8
-Create Date: 2026-03-25
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "b3c4d5e6f7a8"
-down_revision: str | Sequence[str] | None = "a3b4c5d6e7f8"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Get schema prefix for table names (required for multi-tenant support)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-    # Add content_hash column to chunks table for delta comparison
-    op.execute(f"ALTER TABLE {schema}chunks ADD COLUMN IF NOT EXISTS content_hash TEXT")
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"ALTER TABLE {schema}chunks DROP COLUMN IF EXISTS content_hash")

hindsight-api-slim/hindsight_api/alembic/versions/c2d3e4f5g6h7_add_audit_log_table.py

@@ -1,61 +0,0 @@
-"""Add audit_log table for feature usage tracking.
-
-Merge migration that combines the two existing heads (a3b4c5d6e7f8 + c8e5f2a3b4d1).
-
-Stores raw request/response as JSONB for expandability without future migrations.
-The metadata JSONB column allows adding arbitrary fields in the future.
-
-Revision ID: c2d3e4f5g6h7
-Revises: a3b4c5d6e7f8, c8e5f2a3b4d1
-Create Date: 2026-03-26
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "c2d3e4f5g6h7"
-down_revision: str | Sequence[str] | None = ("a3b4c5d6e7f8", "c8e5f2a3b4d1")
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Get schema prefix for table names (required for multi-tenant support)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    op.execute(
-        f"""
-        CREATE TABLE IF NOT EXISTS {schema}audit_log (
-            id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
-            action TEXT NOT NULL,
-            transport TEXT NOT NULL,
-            bank_id TEXT,
-            started_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
-            ended_at TIMESTAMPTZ,
-            request JSONB,
-            response JSONB,
-            metadata JSONB DEFAULT '{{}}'::jsonb
-        )
-        """
-    )
-
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS idx_audit_log_action_started ON {schema}audit_log (action, started_at DESC)"
-    )
-    op.execute(f"CREATE INDEX IF NOT EXISTS idx_audit_log_bank_started ON {schema}audit_log (bank_id, started_at DESC)")
-    op.execute(f"CREATE INDEX IF NOT EXISTS idx_audit_log_started ON {schema}audit_log (started_at DESC)")
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_audit_log_started")
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_audit_log_bank_started")
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_audit_log_action_started")
-    op.execute(f"DROP TABLE IF EXISTS {schema}audit_log")

hindsight-api-slim/hindsight_api/alembic/versions/c5d6e7f8a9b0_add_bank_id_to_memory_links.py

@@ -1,48 +0,0 @@
-"""Add bank_id column to memory_links for direct filtering
-
-The stats endpoint JOINs memory_links to memory_units just to filter by
-bank_id.  With millions of links this takes 18+ seconds.  Adding bank_id
-directly to memory_links lets Postgres push the filter down before the JOIN.
-
-Revision ID: c5d6e7f8a9b0
-Revises: b3c4d5e6f7a8
-Create Date: 2026-03-26
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "c5d6e7f8a9b0"
-down_revision: str | Sequence[str] | None = "b3c4d5e6f7a8"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # 1. Add nullable column
-    op.execute(f"ALTER TABLE {schema}memory_links ADD COLUMN IF NOT EXISTS bank_id TEXT")
-
-    # 2. Backfill from memory_units
-    op.execute(f"""
-        UPDATE {schema}memory_links ml
-        SET bank_id = mu.bank_id
-        FROM {schema}memory_units mu
-        WHERE ml.from_unit_id = mu.id
-          AND ml.bank_id IS NULL
-    """)
-
-    # 3. Set NOT NULL
-    op.execute(f"ALTER TABLE {schema}memory_links ALTER COLUMN bank_id SET NOT NULL")
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"ALTER TABLE {schema}memory_links DROP COLUMN IF EXISTS bank_id")

hindsight-api-slim/hindsight_api/alembic/versions/d4e5f6g7h8i9_gin_source_memory_ids_fastupdate_off.py

@@ -1,53 +0,0 @@
-"""Recreate idx_memory_units_source_memory_ids GIN index with fastupdate=off
-
-GIN indexes use a "fastupdate" pending list by default: small writes are
-buffered there and flushed to the main GIN tree in bulk. Flushing requires
-AccessExclusiveLock on the index. Under high insert concurrency (e.g. 8
-parallel pytest-xdist workers all calling retain_async) two transactions can
-each trigger a flush simultaneously and deadlock.
-
-Disabling fastupdate makes every insert write directly to the GIN tree
-(slightly slower per insert, but no pending-list lock cycles).
-
-Revision ID: d4e5f6g7h8i9
-Revises: d5e6f7a8b9c0
-Create Date: 2026-03-11
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "d4e5f6g7h8i9"
-down_revision: str | Sequence[str] | None = "d5e6f7a8b9c0"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-    # DROP + CREATE CONCURRENTLY must run outside a transaction block.
-    op.execute("COMMIT")
-    op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_units_source_memory_ids")
-    op.execute(
-        f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_units_source_memory_ids "
-        f"ON {schema}memory_units USING GIN (source_memory_ids) "
-        f"WITH (fastupdate=off) "
-        f"WHERE source_memory_ids IS NOT NULL"
-    )
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute("COMMIT")
-    op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {schema}idx_memory_units_source_memory_ids")
-    op.execute(
-        f"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_memory_units_source_memory_ids "
-        f"ON {schema}memory_units USING GIN (source_memory_ids) "
-        f"WHERE source_memory_ids IS NOT NULL"
-    )

hindsight-api-slim/hindsight_api/alembic/versions/d5e6f7a8b9c0_add_bank_internal_id_and_per_bank_hnsw.py

@@ -1,139 +0,0 @@
-"""Add internal_id to banks and per-(bank, fact_type) partial vector indexes
-
-Revision ID: d5e6f7a8b9c0
-Revises: a3b4c5d6e7f8
-Create Date: 2026-03-11
-
-This migration:
-1. Adds internal_id UUID column to banks (stable identifier for index naming)
-2. Drops the global vector index (competes with per-bank partial indexes)
-3. Creates per-(bank_id, fact_type) partial vector indexes for all existing banks
-   using the configured vector extension (HNSW for pgvector, DiskANN for
-   pgvectorscale, vchordrq for vchord).
-   (new banks get indexes created at bank-creation time via bank_utils.create_bank_vector_indexes)
-
-Why per-(bank, fact_type) indexes:
-- fact_type-only partial indexes are never chosen by the planner when bank_id is in the WHERE
-  clause, because the idx_memory_units_bank_id B-tree index always wins at planning time.
-- Per-(bank, fact_type) partial indexes have both predicates matching → planner selects them.
-- The global vector index competes for larger partitions (world, observation) and must be dropped.
-"""
-
-import os
-from collections.abc import Sequence
-
-from alembic import context, op
-from sqlalchemy import text
-
-revision: str = "d5e6f7a8b9c0"
-down_revision: str | Sequence[str] | None = "c3d4e5f6g7h8"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-_FACT_TYPES: dict[str, str] = {
-    "world": "worl",
-    "experience": "expr",
-    "observation": "obsv",
-}
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def _vector_index_using_clause() -> str:
-    """Return the USING clause based on the configured vector extension."""
-    ext = os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector").lower()
-    if ext == "pgvectorscale":
-        return "USING diskann (embedding vector_cosine_ops) WITH (num_neighbors = 50)"
-    elif ext == "vchord":
-        return "USING vchordrq (embedding vector_l2_ops)"
-    else:
-        return "USING hnsw (embedding vector_cosine_ops)"
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # 1. Add internal_id column to banks
-    op.execute(
-        f"ALTER TABLE {schema}banks ADD COLUMN IF NOT EXISTS internal_id UUID DEFAULT gen_random_uuid() NOT NULL"
-    )
-    op.execute(f"ALTER TABLE {schema}banks ADD CONSTRAINT banks_internal_id_unique UNIQUE (internal_id)")
-
-    # 2. Drop any fact_type-only partial indexes that may exist from prior migrations
-    #    (bank_id B-tree always wins over them when bank_id is in the WHERE clause)
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_mu_emb_world")
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_mu_emb_observation")
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_mu_emb_experience")
-
-    # 4. Drop global vector index (competes with per-bank partial indexes)
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_units_embedding")
-
-    # 5. Create per-(bank, fact_type) partial vector indexes for all existing banks
-    #    using the configured extension (HNSW / DiskANN / vchordrq)
-    bind = op.get_bind()
-    schema_name = context.config.get_main_option("target_schema")
-    table_ref = f'"{schema_name}".memory_units' if schema_name else "memory_units"
-    banks_ref = f'"{schema_name}".banks' if schema_name else "banks"
-    using_clause = _vector_index_using_clause()
-
-    rows = bind.execute(text(f"SELECT bank_id, internal_id FROM {banks_ref}")).fetchall()  # noqa: S608
-    for row in rows:
-        bank_id = row[0]
-        internal_id = str(row[1]).replace("-", "")[:16]
-        escaped_bank_id = bank_id.replace("'", "''")
-        for ft, ft_short in _FACT_TYPES.items():
-            idx_name = f"idx_mu_emb_{ft_short}_{internal_id}"
-            # Index name is schema-unqualified (indexes live in the schema of their table)
-            bind.execute(
-                text(
-                    f"CREATE INDEX IF NOT EXISTS {idx_name} "
-                    f"ON {table_ref} {using_clause} "
-                    f"WHERE fact_type = '{ft}' AND bank_id = '{escaped_bank_id}'"
-                )
-            )
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # Drop per-bank HNSW indexes (iterate existing banks)
-    bind = op.get_bind()
-    schema_name = context.config.get_main_option("target_schema")
-    banks_ref = f'"{schema_name}".banks' if schema_name else "banks"
-
-    rows = bind.execute(text(f"SELECT internal_id FROM {banks_ref}")).fetchall()  # noqa: S608
-    for row in rows:
-        internal_id = str(row[0]).replace("-", "")[:16]
-        for ft_short in _HNSW_FACT_TYPES.values():
-            idx_name = f"idx_mu_emb_{ft_short}_{internal_id}"
-            bind.execute(text(f"DROP INDEX IF EXISTS {schema}{idx_name}"))
-
-    # Restore the global HNSW index
-    table_ref = f'"{schema_name}".memory_units' if schema_name else "memory_units"
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS idx_memory_units_embedding ON {table_ref} USING hnsw (embedding vector_cosine_ops)"
-    )
-
-    # Restore old fact_type-only partial indexes
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS idx_mu_emb_world "
-        f"ON {table_ref} USING hnsw (embedding vector_cosine_ops) "
-        f"WHERE fact_type = 'world'"
-    )
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS idx_mu_emb_observation "
-        f"ON {table_ref} USING hnsw (embedding vector_cosine_ops) "
-        f"WHERE fact_type = 'observation'"
-    )
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS idx_mu_emb_experience "
-        f"ON {table_ref} USING hnsw (embedding vector_cosine_ops) "
-        f"WHERE fact_type = 'experience'"
-    )
-
-    # Drop internal_id column
-    op.execute(f"ALTER TABLE {schema}banks DROP CONSTRAINT IF EXISTS banks_internal_id_unique")
-    op.execute(f"ALTER TABLE {schema}banks DROP COLUMN IF EXISTS internal_id")

hindsight-api-slim/hindsight_api/alembic/versions/e5f6g7h8i9j0_cascade_delete_ops_on_bank_delete.py

@@ -1,73 +0,0 @@
-"""Add CASCADE DELETE FK from async_operations and webhooks to banks.
-
-When a bank is deleted, all its async_operations and webhooks rows are
-automatically deleted by the database. This ensures that any in-flight
-worker tasks detect the deletion via _check_op_alive() and abort early.
-
-Revision ID: e5f6g7h8i9j0
-Revises: d4e5f6g7h8i9
-Create Date: 2026-03-11
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "e5f6g7h8i9j0"
-down_revision: str | Sequence[str] | None = "d4e5f6g7h8i9"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # Remove orphaned async_operations rows whose bank no longer exists
-    # (can happen because there was no FK before this migration).
-    op.execute(
-        f"""
-        DELETE FROM {schema}async_operations
-        WHERE bank_id IS NOT NULL
-          AND bank_id NOT IN (SELECT bank_id FROM {schema}banks)
-        """
-    )
-
-    # Remove orphaned webhooks rows whose bank no longer exists.
-    op.execute(
-        f"""
-        DELETE FROM {schema}webhooks
-        WHERE bank_id IS NOT NULL
-          AND bank_id NOT IN (SELECT bank_id FROM {schema}banks)
-        """
-    )
-
-    # Add FK with ON DELETE CASCADE so that deleting a bank automatically
-    # cleans up all its pending/processing operations and webhook configs.
-    op.execute(
-        f"""
-        ALTER TABLE {schema}async_operations
-            ADD CONSTRAINT fk_async_operations_bank_id
-            FOREIGN KEY (bank_id) REFERENCES {schema}banks(bank_id)
-            ON DELETE CASCADE
-        """
-    )
-
-    op.execute(
-        f"""
-        ALTER TABLE {schema}webhooks
-            ADD CONSTRAINT fk_webhooks_bank_id
-            FOREIGN KEY (bank_id) REFERENCES {schema}banks(bank_id)
-            ON DELETE CASCADE
-        """
-    )
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"ALTER TABLE {schema}async_operations DROP CONSTRAINT IF EXISTS fk_async_operations_bank_id")
-    op.execute(f"ALTER TABLE {schema}webhooks DROP CONSTRAINT IF EXISTS fk_webhooks_bank_id")

hindsight-api-slim/hindsight_api/alembic/versions/f6g7h8i9j0k1_chunk_fk_cascade_delete.py

@@ -1,57 +0,0 @@
-"""chunk_fk_cascade_delete
-
-Revision ID: f6g7h8i9j0k1
-Revises: e5f6g7h8i9j0
-Create Date: 2026-03-16 00:00:00.000000
-
-"""
-
-from collections.abc import Sequence
-
-from alembic import op
-
-# revision identifiers, used by Alembic.
-revision: str = "f6g7h8i9j0k1"
-down_revision: str | Sequence[str] | None = "e5f6g7h8i9j0"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def upgrade() -> None:
-    """Change memory_units.chunk_id FK from SET NULL to CASCADE.
-
-    When a document is deleted the CASCADE reaches chunks first; with SET NULL
-    the memory_units rows survived with chunk_id = NULL, leaving ghost records.
-    Switching to CASCADE ensures they are removed together with their chunk.
-    """
-    from alembic import context
-
-    schema = context.config.get_main_option("target_schema")
-    schema_prefix = f'"{schema}".' if schema else ""
-    # Use raw SQL with IF EXISTS so this is safe on schemas where the FK was
-    # already dropped or never existed under this name.
-    op.execute(f"ALTER TABLE {schema_prefix}memory_units DROP CONSTRAINT IF EXISTS memory_units_chunk_fkey")
-    # Use a DO block so the ADD is also idempotent: if the FK already exists (e.g.
-    # the schema was provisioned after the base migration already added it) the
-    # duplicate_object exception is swallowed rather than failing the migration.
-    op.execute(
-        f"""
-        DO $$ BEGIN
-            ALTER TABLE {schema_prefix}memory_units
-                ADD CONSTRAINT memory_units_chunk_fkey
-                FOREIGN KEY (chunk_id)
-                REFERENCES {schema_prefix}chunks (chunk_id)
-                ON DELETE CASCADE;
-        EXCEPTION
-            WHEN duplicate_object THEN NULL;
-        END $$;
-        """
-    )
-
-
-def downgrade() -> None:
-    """Revert to SET NULL behaviour."""
-    op.drop_constraint("memory_units_chunk_fkey", "memory_units", type_="foreignkey")
-    op.create_foreign_key(
-        "memory_units_chunk_fkey", "memory_units", "chunks", ["chunk_id"], ["chunk_id"], ondelete="SET NULL"
-    )

hindsight-api-slim/hindsight_api/alembic/versions/g2h3i4j5k6l7_remove_opinion_fact_type.py

@@ -1,83 +0,0 @@
-"""remove_opinion_fact_type
-
-Revision ID: g2h3i4j5k6l7
-Revises: f1a2b3c4d5e6
-Create Date: 2026-04-02
-
-Remove the deprecated 'opinion' fact type: drop opinion-specific indexes,
-update CHECK constraints, delete any remaining opinion rows, and drop the
-confidence_score column (was only used for opinions, always NULL otherwise).
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-# revision identifiers, used by Alembic.
-revision: str = "g2h3i4j5k6l7"
-down_revision: str | Sequence[str] | None = "f1a2b3c4d5e6"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Get schema prefix for table names (required for multi-tenant support)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # 1. Delete any remaining opinion rows
-    op.execute(f"DELETE FROM {schema}memory_units WHERE fact_type = 'opinion'")
-
-    # 2. Drop opinion-specific indexes
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_units_opinion_confidence")
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_units_opinion_date")
-
-    # 3. Drop confidence_score constraints and column (only used for opinions, always NULL otherwise)
-    op.execute(f"ALTER TABLE {schema}memory_units DROP CONSTRAINT IF EXISTS confidence_score_fact_type_check")
-    op.execute(f"ALTER TABLE {schema}memory_units DROP CONSTRAINT IF EXISTS memory_units_confidence_score_check")
-    op.execute(f"ALTER TABLE {schema}memory_units DROP COLUMN IF EXISTS confidence_score")
-
-    # 4. Replace fact_type CHECK constraint
-    op.execute(f"ALTER TABLE {schema}memory_units DROP CONSTRAINT IF EXISTS memory_units_fact_type_check")
-    op.execute(
-        f"ALTER TABLE {schema}memory_units ADD CONSTRAINT memory_units_fact_type_check "
-        f"CHECK (fact_type IN ('world', 'experience', 'observation'))"
-    )
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-
-    # Restore confidence_score column
-    op.execute(f"ALTER TABLE {schema}memory_units ADD COLUMN IF NOT EXISTS confidence_score float")
-    op.execute(
-        f"ALTER TABLE {schema}memory_units ADD CONSTRAINT memory_units_confidence_score_check "
-        f"CHECK (confidence_score IS NULL OR (confidence_score >= 0.0 AND confidence_score <= 1.0))"
-    )
-    op.execute(
-        f"ALTER TABLE {schema}memory_units ADD CONSTRAINT confidence_score_fact_type_check "
-        f"CHECK ((fact_type = 'opinion' AND confidence_score IS NOT NULL) OR "
-        f"(fact_type = 'observation') OR "
-        f"(fact_type NOT IN ('opinion', 'observation') AND confidence_score IS NULL))"
-    )
-
-    # Restore original fact_type CHECK constraint (with opinion)
-    op.execute(f"ALTER TABLE {schema}memory_units DROP CONSTRAINT IF EXISTS memory_units_fact_type_check")
-    op.execute(
-        f"ALTER TABLE {schema}memory_units ADD CONSTRAINT memory_units_fact_type_check "
-        f"CHECK (fact_type IN ('world', 'experience', 'opinion', 'observation'))"
-    )
-
-    # Recreate opinion indexes
-    op.execute(
-        f"CREATE INDEX idx_memory_units_opinion_confidence ON {schema}memory_units "
-        f"(bank_id, confidence_score DESC) WHERE fact_type = 'opinion'"
-    )
-    op.execute(
-        f"CREATE INDEX idx_memory_units_opinion_date ON {schema}memory_units "
-        f"(bank_id, event_date DESC) WHERE fact_type = 'opinion'"
-    )

hindsight-api-slim/hindsight_api/alembic/versions/g7h8i9j0k1l2_backsweep_orphan_observations.py

@@ -1,71 +0,0 @@
-"""backsweep_orphan_memory_units
-
-Two-pass cleanup of memory_units rows that were never removed by earlier bugs:
-
-Pass 1 — any fact_type, bank gone:
-  memory_units whose bank_id no longer exists in banks.  These accumulate when
-  a bank is deleted without a proper cascade (no FK from memory_units to banks
-  exists in the schema).
-
-Pass 2 — observations only, all sources gone:
-  observation rows whose bank still exists but every source_memory_id points
-  to a deleted memory unit.  These were left behind before PR #580 fixed the
-  chunk FK cascade and before delete_document() called
-  _delete_stale_observations_for_memories.
-
-Revision ID: g7h8i9j0k1l2
-Revises: f6g7h8i9j0k1
-Create Date: 2026-03-16
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "g7h8i9j0k1l2"
-down_revision: str | Sequence[str] | None = "f6g7h8i9j0k1"
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-    mu = f"{schema}memory_units"
-    banks = f"{schema}banks"
-
-    # Pass 1: delete all memory_units (any fact_type) whose bank no longer exists.
-    # There is no FK from memory_units to banks, so these never cascade away.
-    op.execute(
-        f"""
-        DELETE FROM {mu}
-        WHERE NOT EXISTS (
-            SELECT 1 FROM {banks} b WHERE b.bank_id = {mu}.bank_id
-        )
-        """
-    )
-
-    # Pass 2: delete orphaned observations whose bank still exists but every
-    # source_memory_id refers to a now-deleted memory unit (or the array is
-    # empty).  Observations with at least one surviving source are left alone.
-    op.execute(
-        f"""
-        DELETE FROM {mu} orphan
-        WHERE orphan.fact_type = 'observation'
-          AND NOT EXISTS (
-              SELECT 1
-              FROM {mu} src
-              WHERE src.id = ANY(orphan.source_memory_ids)
-                AND src.bank_id = orphan.bank_id
-          )
-        """
-    )
-
-
-def downgrade() -> None:
-    # Deleted rows cannot be restored.
-    pass

hindsight-api-slim/hindsight_api/alembic/versions/h3i4j5k6l7m8_merge_heads_and_add_unit_entities_index.py

@@ -1,42 +0,0 @@
-"""Merge 3 migration heads and add unit_entities composite index
-
-Revision ID: h3i4j5k6l7m8
-Revises: a4b5c6d7e8f9, c2d3e4f5g6h7, g2h3i4j5k6l7
-Create Date: 2026-04-07
-
-Merges three unmerged migration heads into one, and adds a composite index
-(entity_id, unit_id) on unit_entities for index-only scans in the LATERAL
-entity expansion query.
-"""
-
-from collections.abc import Sequence
-
-from alembic import context, op
-
-revision: str = "h3i4j5k6l7m8"
-down_revision: str | Sequence[str] | None = ("a4b5c6d7e8f9", "c2d3e4f5g6h7", "g2h3i4j5k6l7")
-branch_labels: str | Sequence[str] | None = None
-depends_on: str | Sequence[str] | None = None
-
-
-def _get_schema_prefix() -> str:
-    """Get schema prefix for table names (required for multi-tenant support)."""
-    schema = context.config.get_main_option("target_schema")
-    return f'"{schema}".' if schema else ""
-
-
-def upgrade() -> None:
-    schema = _get_schema_prefix()
-    # Composite index enables index-only scans for entity_id -> unit_id lookups
-    op.execute(
-        f"CREATE INDEX IF NOT EXISTS idx_unit_entities_entity_unit ON {schema}unit_entities (entity_id, unit_id)"
-    )
-    # Drop the now-redundant single-column index
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_unit_entities_entity")
-
-
-def downgrade() -> None:
-    schema = _get_schema_prefix()
-    op.execute(f"DROP INDEX IF EXISTS {schema}idx_unit_entities_entity_unit")
-    # Restore the single-column index
-    op.execute(f"CREATE INDEX IF NOT EXISTS idx_unit_entities_entity ON {schema}unit_entities (entity_id)")

hindsight-api-slim/hindsight_api/engine/audit.py

@@ -1,209 +0,0 @@
-"""Audit logging for feature usage tracking.
-
-Provides fire-and-forget audit logging of all mutating and core operations
-(retain, recall, reflect, bank CRUD, etc.) across HTTP, MCP, and system transports.
-"""
-
-from __future__ import annotations
-
-import asyncio
-import json
-import logging
-import uuid
-from collections.abc import Callable
-from contextlib import asynccontextmanager
-from dataclasses import dataclass, field
-from datetime import datetime, timezone
-from typing import Any
-
-import asyncpg
-
-from ..engine.db_utils import acquire_with_retry
-
-logger = logging.getLogger(__name__)
-
-
-@dataclass
-class AuditEntry:
-    """A single audit log entry."""
-
-    action: str
-    transport: str  # "http", "mcp", "system"
-    bank_id: str | None = None
-    started_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
-    ended_at: datetime | None = None
-    request: dict[str, Any] | None = None
-    response: dict[str, Any] | None = None
-    metadata: dict[str, Any] = field(default_factory=dict)
-
-
-def _json_default(obj: Any) -> str:
-    """JSON serializer for objects not serializable by default."""
-    if isinstance(obj, datetime):
-        return obj.isoformat()
-    if isinstance(obj, uuid.UUID):
-        return str(obj)
-    if isinstance(obj, bytes):
-        return "<bytes>"
-    if isinstance(obj, set):
-        return list(obj)
-    return str(obj)
-
-
-def _safe_json(data: Any) -> str | None:
-    """Serialize data to JSON string, returning None on failure."""
-    if data is None:
-        return None
-    try:
-        return json.dumps(data, default=_json_default)
-    except Exception:
-        logger.debug("Failed to serialize audit data", exc_info=True)
-        return None
-
-
-_SWEEP_INTERVAL_SECONDS = 3600  # Run retention sweep every hour
-
-
-class AuditLogger:
-    """Fire-and-forget audit log writer with optional retention sweep."""
-
-    def __init__(
-        self,
-        pool_getter: Callable[[], asyncpg.Pool | None],
-        schema_getter: Callable[[], str],
-        enabled: bool,
-        allowed_actions: list[str],
-        retention_days: int = -1,
-    ) -> None:
-        self._pool_getter = pool_getter
-        self._schema_getter = schema_getter
-        self._enabled = enabled
-        self._allowed_actions: frozenset[str] | None = frozenset(allowed_actions) if allowed_actions else None
-        self._retention_days = retention_days
-        self._sweep_task: asyncio.Task | None = None
-
-    def is_enabled(self, action: str) -> bool:
-        """Check if audit logging is enabled for this action."""
-        if not self._enabled:
-            return False
-        if self._allowed_actions is not None:
-            return action in self._allowed_actions
-        return True
-
-    def log_fire_and_forget(self, entry: AuditEntry) -> None:
-        """Schedule an audit write as a background task."""
-        if not self.is_enabled(entry.action):
-            return
-        try:
-            asyncio.create_task(self._safe_log(entry))
-        except RuntimeError:
-            # No running event loop (e.g. during shutdown)
-            logger.debug("Cannot schedule audit log write: no running event loop")
-
-    async def _safe_log(self, entry: AuditEntry) -> None:
-        """Write audit entry to DB. Errors are logged, never raised."""
-        pool = self._pool_getter()
-        if pool is None:
-            logger.debug("Audit log skipped: pool not available")
-            return
-        try:
-            schema = self._schema_getter()
-            table = f"{schema}.audit_log"
-            async with acquire_with_retry(pool, max_retries=1) as conn:
-                await conn.execute(
-                    f"""
-                    INSERT INTO {table}
-                        (id, action, transport, bank_id, started_at, ended_at, request, response, metadata)
-                    VALUES
-                        ($1, $2, $3, $4, $5, $6, $7::jsonb, $8::jsonb, $9::jsonb)
-                    """,
-                    uuid.uuid4(),
-                    entry.action,
-                    entry.transport,
-                    entry.bank_id,
-                    entry.started_at,
-                    entry.ended_at,
-                    _safe_json(entry.request),
-                    _safe_json(entry.response),
-                    _safe_json(entry.metadata) or "{}",
-                )
-        except Exception as e:
-            logger.warning(f"Audit log write failed for action={entry.action}: {e}")
-
-    def start_retention_sweep(self) -> None:
-        """Start the periodic retention sweep if retention is configured."""
-        if self._retention_days <= 0 or not self._enabled:
-            return
-        try:
-            self._sweep_task = asyncio.create_task(self._sweep_loop())
-        except RuntimeError:
-            logger.debug("Cannot start retention sweep: no running event loop")
-
-    async def stop_retention_sweep(self) -> None:
-        """Stop the periodic retention sweep."""
-        if self._sweep_task and not self._sweep_task.done():
-            self._sweep_task.cancel()
-            try:
-                await self._sweep_task
-            except asyncio.CancelledError:
-                pass
-            self._sweep_task = None
-
-    async def _sweep_loop(self) -> None:
-        """Periodically delete audit log entries older than retention_days."""
-        while True:
-            await self._run_sweep()
-            await asyncio.sleep(_SWEEP_INTERVAL_SECONDS)
-
-    async def _run_sweep(self) -> None:
-        """Delete expired audit log entries. Concurrent-safe via row-level deletes."""
-        pool = self._pool_getter()
-        if pool is None:
-            return
-        try:
-            schema = self._schema_getter()
-            table = f"{schema}.audit_log"
-            async with acquire_with_retry(pool, max_retries=1) as conn:
-                result = await conn.execute(
-                    f"DELETE FROM {table} WHERE started_at < NOW() - INTERVAL '{self._retention_days} days'"
-                )
-                if result and result != "DELETE 0":
-                    logger.info(f"Audit log retention sweep: {result}")
-        except Exception as e:
-            logger.warning(f"Audit log retention sweep failed: {e}")
-
-
-@asynccontextmanager
-async def audit_context(
-    audit_logger: AuditLogger | None,
-    action: str,
-    transport: str,
-    bank_id: str | None = None,
-    request: dict[str, Any] | None = None,
-    metadata: dict[str, Any] | None = None,
-):
-    """Async context manager that times the operation and writes audit on exit.
-
-    Usage:
-        async with audit_context(logger, "retain", "http", bank_id, request_dict) as entry:
-            result = await do_work()
-            entry.response = result_dict
-    """
-    if audit_logger is None or not audit_logger.is_enabled(action):
-        entry = AuditEntry(action=action, transport=transport, bank_id=bank_id)
-        yield entry
-        return
-
-    entry = AuditEntry(
-        action=action,
-        transport=transport,
-        bank_id=bank_id,
-        started_at=datetime.now(timezone.utc),
-        request=request,
-        metadata=metadata or {},
-    )
-    try:
-        yield entry
-    finally:
-        entry.ended_at = datetime.now(timezone.utc)
-        audit_logger.log_fire_and_forget(entry)

hindsight-api-slim/hindsight_api/engine/jina_mlx_reranker.py

@@ -1,144 +0,0 @@
-"""
-MLX implementation of jina-reranker-v3 for Apple Silicon.
-
-This file is adapted from the official model repository:
-  https://huggingface.co/jinaai/jina-reranker-v3-mlx/blob/main/rerank.py
-
-License: CC BY-NC 4.0 (contact Jina AI for commercial usage)
-
-Changes from upstream:
-- Removed the __main__ example block
-- Type annotations added to public methods
-- top_n parameter added to rerank() (upstream only exposed it implicitly)
-"""
-
-import numpy as np
-
-
-class _MLPProjector:
-    def __init__(self):
-        import mlx.nn as nn
-
-        self.linear1 = nn.Linear(1024, 512, bias=False)
-        self.linear2 = nn.Linear(512, 512, bias=False)
-
-    def __call__(self, x):
-        import mlx.nn as nn
-
-        x = self.linear1(x)
-        x = nn.relu(x)
-        x = self.linear2(x)
-        return x
-
-
-def _load_projector(projector_path: str) -> _MLPProjector:
-    import mlx.core as mx
-    from safetensors import safe_open
-
-    projector = _MLPProjector()
-    with safe_open(projector_path, framework="numpy") as f:
-        projector.linear1.weight = mx.array(f.get_tensor("linear1.weight"))
-        projector.linear2.weight = mx.array(f.get_tensor("linear2.weight"))
-    return projector
-
-
-def _sanitize(text: str, special_tokens: dict[str, str]) -> str:
-    for token in special_tokens.values():
-        text = text.replace(token, "")
-    return text
-
-
-def _format_prompt(query: str, docs: list[str], special_tokens: dict[str, str]) -> str:
-    query = _sanitize(query, special_tokens)
-    docs = [_sanitize(d, special_tokens) for d in docs]
-
-    doc_token = special_tokens["doc_embed_token"]
-    query_token = special_tokens["query_embed_token"]
-
-    prefix = (
-        "<|im_start|>system\n"
-        "You are a search relevance expert who can determine a ranking of the passages based on how relevant they are to the query. "
-        "If the query is a question, how relevant a passage is depends on how well it answers the question. "
-        "If not, try to analyze the intent of the query and assess how well each passage satisfies the intent. "
-        "If an instruction is provided, you should follow the instruction when determining the ranking."
-        "<|im_end|>\n<|im_start|>user\n"
-    )
-    suffix = "<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"
-
-    body = (
-        f"I will provide you with {len(docs)} passages, each indicated by a numerical identifier. "
-        f"Rank the passages based on their relevance to query: {query}\n"
-    )
-    body += "\n".join(f'<passage id="{i}">\n{doc}{doc_token}\n</passage>' for i, doc in enumerate(docs))
-    body += f"\n<query>\n{query}{query_token}\n</query>"
-    return prefix + body + suffix
-
-
-class MLXReranker:
-    """
-    MLX-accelerated jina-reranker-v3 for Apple Silicon.
-
-    Loads the model from a local directory (use huggingface_hub.snapshot_download
-    to fetch jinaai/jina-reranker-v3-mlx if you don't have it already).
-    """
-
-    _SPECIAL_TOKENS = {
-        "query_embed_token": "<|rerank_token|>",
-        "doc_embed_token": "<|embed_token|>",
-    }
-    _DOC_TOKEN_ID = 151670
-    _QUERY_TOKEN_ID = 151671
-
-    def __init__(self, model_path: str, projector_path: str):
-        from mlx_lm import load
-
-        self.model, self.tokenizer = load(model_path)
-        self.model.eval()
-        self.projector = _load_projector(projector_path)
-
-    def rerank(self, query: str, documents: list[str], top_n: int | None = None) -> list[dict]:
-        """
-        Rank documents by relevance to a query.
-
-        Returns a list of dicts with keys: document, relevance_score, index.
-        Sorted by descending relevance_score.
-        """
-        import mlx.core as mx
-
-        prompt = _format_prompt(query, documents, self._SPECIAL_TOKENS)
-        input_ids = self.tokenizer.encode(prompt)
-        hidden_states = self.model.model([input_ids])[0]  # [seq_len, hidden_size]
-
-        input_ids_np = np.array(input_ids)
-        query_positions = np.where(input_ids_np == self._QUERY_TOKEN_ID)[0]
-        doc_positions = np.where(input_ids_np == self._DOC_TOKEN_ID)[0]
-
-        if len(query_positions) == 0:
-            raise ValueError("Query embed token not found in prompt")
-        if len(doc_positions) == 0:
-            raise ValueError("Document embed tokens not found in prompt")
-
-        query_hidden = mx.expand_dims(hidden_states[int(query_positions[0])], axis=0)
-        doc_hidden = mx.stack([hidden_states[int(p)] for p in doc_positions])
-
-        query_emb = self.projector(query_hidden)  # [1, 512]
-        doc_emb = self.projector(doc_hidden)  # [num_docs, 512]
-
-        query_exp = mx.broadcast_to(mx.expand_dims(query_emb, 0), (1, len(documents), 512))
-        doc_exp = mx.expand_dims(doc_emb, 0)
-
-        scores = mx.sum(doc_exp * query_exp, axis=-1) / (
-            mx.sqrt(mx.sum(doc_exp * doc_exp, axis=-1)) * mx.sqrt(mx.sum(query_exp * query_exp, axis=-1))
-        )  # [1, num_docs]
-        scores_np = np.array(scores[0])
-
-        order = np.argsort(scores_np)[::-1]
-        n = min(top_n, len(documents)) if top_n is not None else len(documents)
-        return [
-            {
-                "document": documents[order[i]],
-                "relevance_score": float(scores_np[order[i]]),
-                "index": int(order[i]),
-            }
-            for i in range(n)
-        ]

hindsight-api-slim/hindsight_api/engine/providers/litellm_llm.py

@@ -1,385 +0,0 @@
-"""
-LiteLLM LLM provider for universal model support.
-
-This provider enables using 100+ LLM providers via the LiteLLM SDK, including:
-- AWS Bedrock (bedrock/anthropic.claude-3-5-sonnet-...)
-- Azure OpenAI (azure/gpt-4o)
-- Together AI (together_ai/meta-llama/...)
-- Any other LiteLLM-supported provider
-
-Uses litellm.acompletion() for async chat completions.
-Authentication for cloud providers (e.g., AWS Bedrock via boto3 credential chain)
-is handled automatically by LiteLLM.
-"""
-
-import asyncio
-import json
-import logging
-import time
-from typing import Any
-
-from hindsight_api.engine.llm_interface import LLMInterface, OutputTooLongError
-from hindsight_api.engine.response_models import LLMToolCall, LLMToolCallResult, TokenUsage
-from hindsight_api.metrics import get_metrics_collector
-from hindsight_api.worker.stage import set_stage
-
-logger = logging.getLogger(__name__)
-
-
-class LiteLLMLLM(LLMInterface):
-    """
-    LLM provider using the LiteLLM SDK for universal model support.
-
-    Supports any model accessible via litellm.acompletion(), including AWS Bedrock,
-    Azure OpenAI, Together AI, Fireworks AI, and more.
-
-    Model names follow LiteLLM conventions with provider prefixes:
-    - bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0
-    - azure/gpt-4o
-    - together_ai/meta-llama/Llama-3-70b-chat-hf
-    - fireworks_ai/accounts/fireworks/models/llama-v3p1-70b-instruct
-    """
-
-    def __init__(
-        self,
-        provider: str,
-        api_key: str,
-        base_url: str,
-        model: str,
-        reasoning_effort: str = "low",
-        timeout: float = 300.0,
-        **kwargs: Any,
-    ):
-        super().__init__(provider, api_key, base_url, model, reasoning_effort, **kwargs)
-        self.timeout = timeout
-        self._litellm: Any = None
-
-        try:
-            import litellm
-
-            self._litellm = litellm
-            # Suppress LiteLLM's verbose logging
-            litellm.suppress_debug_info = True  # type: ignore[assignment]
-            # Drop unsupported params instead of raising errors (e.g. tool_choice on some Bedrock models)
-            litellm.drop_params = True  # type: ignore[assignment]
-            logging.getLogger("LiteLLM").setLevel(logging.WARNING)
-            logger.info(f"LiteLLM SDK initialized for model: {self.model}")
-        except ImportError as e:
-            raise RuntimeError("LiteLLM SDK not installed. Run: uv add litellm or pip install litellm") from e
-
-    async def verify_connection(self) -> None:
-        try:
-            test_messages = [{"role": "user", "content": "test"}]
-            await self.call(
-                messages=test_messages,
-                max_completion_tokens=50,
-                temperature=0.0,
-                scope="verification",
-                max_retries=0,
-            )
-            logger.info("LiteLLM connection verified successfully")
-        except OutputTooLongError:
-            # Truncation is fine for verification — it means the connection works
-            logger.info("LiteLLM connection verified successfully (response truncated)")
-        except Exception as e:
-            logger.error(f"LiteLLM connection verification failed: {e}")
-            raise RuntimeError(f"Failed to verify LiteLLM connection: {e}") from e
-
-    def _build_common_kwargs(
-        self,
-        messages: list[dict[str, Any]],
-        max_completion_tokens: int | None = None,
-        temperature: float | None = None,
-    ) -> dict[str, Any]:
-        """Build common kwargs for litellm calls."""
-        kwargs: dict[str, Any] = {
-            "model": self.model,
-            "messages": messages,
-            "timeout": self.timeout,
-        }
-
-        if self.api_key:
-            kwargs["api_key"] = self.api_key
-        if self.base_url:
-            kwargs["api_base"] = self.base_url
-        if max_completion_tokens is not None:
-            kwargs["max_completion_tokens"] = max_completion_tokens
-        if temperature is not None:
-            kwargs["temperature"] = temperature
-
-        return kwargs
-
-    async def call(
-        self,
-        messages: list[dict[str, str]],
-        response_format: Any | None = None,
-        max_completion_tokens: int | None = None,
-        temperature: float | None = None,
-        scope: str = "memory",
-        max_retries: int = 10,
-        initial_backoff: float = 1.0,
-        max_backoff: float = 60.0,
-        skip_validation: bool = False,
-        strict_schema: bool = False,
-        return_usage: bool = False,
-    ) -> Any:
-        start_time = time.time()
-
-        call_kwargs = self._build_common_kwargs(messages, max_completion_tokens, temperature)
-
-        # Add JSON schema response format if provided
-        if response_format is not None and hasattr(response_format, "model_json_schema"):
-            schema = response_format.model_json_schema()
-            call_kwargs["response_format"] = {
-                "type": "json_schema",
-                "json_schema": {
-                    "name": response_format.__name__ if hasattr(response_format, "__name__") else "response",
-                    "schema": schema,
-                    "strict": strict_schema,
-                },
-            }
-
-        last_exception = None
-
-        for attempt in range(max_retries + 1):
-            if attempt > 0:
-                set_stage(f"llm.litellm.{scope}.attempt={attempt + 1}/{max_retries + 1}")
-            try:
-                response = await self._litellm.acompletion(**call_kwargs)
-
-                content = response.choices[0].message.content or ""
-                finish_reason = response.choices[0].finish_reason
-
-                # Check for length-limited output
-                if finish_reason == "length":
-                    raise OutputTooLongError("LiteLLM response was truncated due to token limit")
-
-                if response_format is not None:
-                    # Strip markdown code fences if present
-                    clean_content = content
-                    if "```json" in content:
-                        clean_content = content.split("```json")[1].split("```")[0].strip()
-                    elif "```" in content:
-                        clean_content = content.split("```")[1].split("```")[0].strip()
-
-                    try:
-                        json_data = json.loads(clean_content)
-                    except json.JSONDecodeError:
-                        json_data = json.loads(content)
-
-                    if skip_validation:
-                        result = json_data
-                    else:
-                        result = response_format.model_validate(json_data)
-                else:
-                    result = content
-
-                # Extract usage
-                input_tokens = getattr(response.usage, "prompt_tokens", 0) or 0
-                output_tokens = getattr(response.usage, "completion_tokens", 0) or 0
-                total_tokens = input_tokens + output_tokens
-
-                # Record metrics
-                duration = time.time() - start_time
-                metrics = get_metrics_collector()
-                metrics.record_llm_call(
-                    provider=self.provider,
-                    model=self.model,
-                    scope=scope,
-                    duration=duration,
-                    input_tokens=input_tokens,
-                    output_tokens=output_tokens,
-                    success=True,
-                )
-
-                # Record trace span
-                from hindsight_api.tracing import _serialize_for_span, get_span_recorder
-
-                span_recorder = get_span_recorder()
-                span_recorder.record_llm_call(
-                    provider=self.provider,
-                    model=self.model,
-                    scope=scope,
-                    messages=messages,
-                    response_content=_serialize_for_span(result),
-                    input_tokens=input_tokens,
-                    output_tokens=output_tokens,
-                    duration=duration,
-                    finish_reason=finish_reason,
-                    error=None,
-                )
-
-                if duration > 10.0:
-                    logger.info(
-                        f"slow llm call: scope={scope}, model={self.provider}/{self.model}, "
-                        f"input_tokens={input_tokens}, output_tokens={output_tokens}, "
-                        f"time={duration:.3f}s"
-                    )
-
-                if return_usage:
-                    token_usage = TokenUsage(
-                        input_tokens=input_tokens,
-                        output_tokens=output_tokens,
-                        total_tokens=total_tokens,
-                    )
-                    return result, token_usage
-                return result
-
-            except OutputTooLongError:
-                raise
-
-            except json.JSONDecodeError as e:
-                last_exception = e
-                if attempt < max_retries:
-                    logger.warning("LiteLLM returned invalid JSON, retrying...")
-                    backoff = min(initial_backoff * (2**attempt), max_backoff)
-                    await asyncio.sleep(backoff)
-                    continue
-                else:
-                    logger.error(f"LiteLLM returned invalid JSON after {max_retries + 1} attempts")
-                    raise
-
-            except Exception as e:
-                error_str = str(e).lower()
-                # Fast fail on auth errors
-                if "401" in error_str or "403" in error_str or "unauthorized" in error_str:
-                    logger.error(f"LiteLLM auth error, not retrying: {e}")
-                    raise
-
-                last_exception = e
-                if attempt < max_retries:
-                    # Retry on rate limits, connection errors, server errors
-                    is_retryable = any(
-                        keyword in error_str
-                        for keyword in ("rate", "limit", "timeout", "connection", "500", "502", "503", "529")
-                    )
-                    if is_retryable:
-                        backoff = min(initial_backoff * (2**attempt), max_backoff)
-                        jitter = backoff * 0.2 * (2 * (time.time() % 1) - 1)
-                        await asyncio.sleep(backoff + jitter)
-                        continue
-
-                logger.error(f"LiteLLM API error after {attempt + 1} attempts: {e}")
-                raise
-
-        if last_exception:
-            raise last_exception
-        raise RuntimeError("LiteLLM call failed after all retries")
-
-    async def call_with_tools(
-        self,
-        messages: list[dict[str, Any]],
-        tools: list[dict[str, Any]],
-        max_completion_tokens: int | None = None,
-        temperature: float | None = None,
-        scope: str = "tools",
-        max_retries: int = 5,
-        initial_backoff: float = 1.0,
-        max_backoff: float = 30.0,
-        tool_choice: str | dict[str, Any] = "auto",
-    ) -> LLMToolCallResult:
-        start_time = time.time()
-
-        call_kwargs = self._build_common_kwargs(messages, max_completion_tokens, temperature)
-        call_kwargs["tools"] = tools
-        call_kwargs["tool_choice"] = tool_choice
-
-        last_exception = None
-        for attempt in range(max_retries + 1):
-            if attempt > 0:
-                set_stage(f"llm.litellm.tools.attempt={attempt + 1}/{max_retries + 1}")
-            try:
-                response = await self._litellm.acompletion(**call_kwargs)
-
-                message = response.choices[0].message
-                content = message.content
-                finish_reason = response.choices[0].finish_reason
-
-                # Extract tool calls
-                tool_calls: list[LLMToolCall] = []
-                if message.tool_calls:
-                    for tc in message.tool_calls:
-                        arguments = tc.function.arguments
-                        if isinstance(arguments, str):
-                            arguments = json.loads(arguments)
-                        tool_calls.append(
-                            LLMToolCall(
-                                id=tc.id,
-                                name=tc.function.name,
-                                arguments=arguments,
-                            )
-                        )
-
-                # Extract usage
-                input_tokens = getattr(response.usage, "prompt_tokens", 0) or 0
-                output_tokens = getattr(response.usage, "completion_tokens", 0) or 0
-
-                # Record metrics
-                duration = time.time() - start_time
-                metrics = get_metrics_collector()
-                metrics.record_llm_call(
-                    provider=self.provider,
-                    model=self.model,
-                    scope=scope,
-                    duration=duration,
-                    input_tokens=input_tokens,
-                    output_tokens=output_tokens,
-                    success=True,
-                )
-
-                # Record trace span
-                from hindsight_api.tracing import get_span_recorder
-
-                span_recorder = get_span_recorder()
-                tool_calls_dict = (
-                    [{"id": tc.id, "name": tc.name, "arguments": tc.arguments} for tc in tool_calls]
-                    if tool_calls
-                    else None
-                )
-                span_recorder.record_llm_call(
-                    provider=self.provider,
-                    model=self.model,
-                    scope=scope,
-                    messages=messages,
-                    response_content=content,
-                    input_tokens=input_tokens,
-                    output_tokens=output_tokens,
-                    duration=duration,
-                    finish_reason=finish_reason,
-                    error=None,
-                    tool_calls=tool_calls_dict,
-                )
-
-                return LLMToolCallResult(
-                    content=content,
-                    tool_calls=tool_calls,
-                    finish_reason=finish_reason or ("tool_calls" if tool_calls else "stop"),
-                    input_tokens=input_tokens,
-                    output_tokens=output_tokens,
-                )
-
-            except Exception as e:
-                error_str = str(e).lower()
-                if "401" in error_str or "403" in error_str or "unauthorized" in error_str:
-                    raise
-
-                last_exception = e
-                if attempt < max_retries:
-                    is_retryable = any(
-                        keyword in error_str
-                        for keyword in ("rate", "limit", "timeout", "connection", "500", "502", "503", "529")
-                    )
-                    if is_retryable:
-                        await asyncio.sleep(min(initial_backoff * (2**attempt), max_backoff))
-                        continue
-
-                logger.error(f"LiteLLM tool call error after {attempt + 1} attempts: {e}")
-                raise
-
-        if last_exception:
-            raise last_exception
-        raise RuntimeError("LiteLLM tool call failed after all retries")
-
-    async def cleanup(self) -> None:
-        """Clean up resources."""
-        pass

hindsight-api-slim/hindsight_api/engine/providers/llamacpp_llm.py

@@ -1,428 +0,0 @@
-"""
-Built-in llama.cpp LLM provider for fully offline operation.
-
-Manages a llama-cpp-python server as a subprocess, downloads GGUF models
-from HuggingFace on first use, and delegates inference to the OpenAI-compatible API.
-
-Usage:
-    HINDSIGHT_API_LLM_PROVIDER=llamacpp
-    HINDSIGHT_API_LLAMACPP_MODEL_PATH=~/.hindsight/models/gemma-4-E2B-it-Q4_K_M.gguf
-    HINDSIGHT_API_LLAMACPP_GPU_LAYERS=-1  # -1 = all layers on GPU
-    HINDSIGHT_API_LLAMACPP_CONTEXT_SIZE=8192
-"""
-
-import asyncio
-import logging
-import os
-import signal
-import socket
-import subprocess
-import sys
-import time
-from pathlib import Path
-from typing import Any
-
-from hindsight_api.engine.llm_interface import LLMInterface
-from hindsight_api.engine.response_models import LLMToolCallResult
-
-logger = logging.getLogger(__name__)
-
-# Default GGUF model for offline mode
-DEFAULT_LLAMACPP_HF_REPO = "bartowski/google_gemma-4-E2B-it-GGUF"
-DEFAULT_LLAMACPP_HF_FILENAME = "google_gemma-4-E2B-it-Q4_K_M.gguf"
-DEFAULT_LLAMACPP_MODEL_ALIAS = "gemma-4-e2b-it"
-
-MODELS_DIR = Path.home() / ".hindsight" / "models"
-
-# Singleton server instance — shared across all LlamaCppLLM instances
-# (retain, reflect, consolidation each create their own LLMProvider,
-# but they should all share one llama.cpp server process)
-_shared_server: "LlamaCppServer | None" = None
-_shared_server_lock = asyncio.Lock()
-
-
-def _find_free_port() -> int:
-    """Find a free TCP port on localhost."""
-    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
-        s.bind(("127.0.0.1", 0))
-        return s.getsockname()[1]
-
-
-def _download_default_model() -> Path:
-    """Download the default GGUF model from HuggingFace if not already cached.
-
-    Returns:
-        Path to the downloaded GGUF file.
-    """
-    try:
-        from huggingface_hub import hf_hub_download
-    except ImportError:
-        raise ImportError(
-            "huggingface-hub is required for automatic model download. "
-            "Install with: pip install 'hindsight-api-slim[local-llm]'"
-        )
-
-    MODELS_DIR.mkdir(parents=True, exist_ok=True)
-    target = MODELS_DIR / DEFAULT_LLAMACPP_HF_FILENAME
-
-    if target.exists():
-        logger.info(f"Using cached model: {target}")
-        return target
-
-    logger.info(
-        f"Downloading {DEFAULT_LLAMACPP_HF_FILENAME} from {DEFAULT_LLAMACPP_HF_REPO} (~3.5 GB, first run only)..."
-    )
-
-    downloaded = hf_hub_download(
-        repo_id=DEFAULT_LLAMACPP_HF_REPO,
-        filename=DEFAULT_LLAMACPP_HF_FILENAME,
-        local_dir=str(MODELS_DIR),
-    )
-
-    logger.info(f"Model downloaded: {downloaded}")
-    return Path(downloaded)
-
-
-def _resolve_model_path(model_path: str | None) -> Path:
-    """Resolve the model path, downloading the default if needed.
-
-    Args:
-        model_path: Explicit path to a GGUF file, or None to use the default.
-
-    Returns:
-        Resolved Path to the GGUF file.
-    """
-    if model_path:
-        p = Path(model_path).expanduser()
-        if not p.exists():
-            raise FileNotFoundError(
-                f"GGUF model not found: {p}\n"
-                f"Set HINDSIGHT_API_LLAMACPP_MODEL_PATH to a valid .gguf file, "
-                f"or remove the setting to auto-download the default model."
-            )
-        return p
-
-    return _download_default_model()
-
-
-class LlamaCppServer:
-    """Manages a llama-cpp-python OpenAI-compatible server as a subprocess."""
-
-    def __init__(
-        self,
-        model_path: Path,
-        port: int,
-        gpu_layers: int = -1,
-        context_size: int = 8192,
-        chat_format: str | None = None,
-        extra_args: str | None = None,
-    ):
-        self.model_path = model_path
-        self.port = port
-        self.gpu_layers = gpu_layers
-        self.context_size = context_size
-        self.chat_format = chat_format
-        self.extra_args = extra_args
-        self._process: subprocess.Popen | None = None
-
-    @property
-    def base_url(self) -> str:
-        return f"http://127.0.0.1:{self.port}/v1"
-
-    async def start(self) -> None:
-        """Start the llama.cpp server subprocess."""
-        cmd = [
-            sys.executable,
-            "-m",
-            "llama_cpp.server",
-            "--model",
-            str(self.model_path),
-            "--host",
-            "127.0.0.1",
-            "--port",
-            str(self.port),
-            "--n_gpu_layers",
-            str(self.gpu_layers),
-            "--n_ctx",
-            str(self.context_size),
-            "--flash_attn",
-            "true",
-            "--n_batch",
-            "2048",
-            # Prompt cache: reuse KV cache for repeated system prompts
-            "--cache",
-            "true",
-        ]
-        # Only pass chat_format if explicitly set (most GGUF models have it embedded)
-        if self.chat_format:
-            cmd.extend(["--chat_format", self.chat_format])
-        # User-provided extra args (e.g. "--type_k 1 --type_v 1 --n_threads 8")
-        if self.extra_args:
-            cmd.extend(self.extra_args.split())
-
-        logger.info(f"Starting llama.cpp server: {' '.join(cmd)}")
-
-        # Write stderr to a log file to avoid pipe buffer deadlock
-        # (llama.cpp outputs a lot of model metadata on stderr during loading)
-        self._log_path = MODELS_DIR / "llamacpp_server.log"
-        self._log_file = open(self._log_path, "w")
-
-        self._process = subprocess.Popen(
-            cmd,
-            stdout=subprocess.DEVNULL,
-            stderr=self._log_file,
-            # Ensure the subprocess is killed when the parent exits
-            preexec_fn=os.setsid if hasattr(os, "setsid") else None,
-        )
-
-        # Wait for the server to be ready
-        await self._wait_for_ready()
-
-    async def _wait_for_ready(self, timeout: float = 120.0) -> None:
-        """Wait for the llama.cpp server to accept connections."""
-        import httpx
-
-        start = time.monotonic()
-        url = f"http://127.0.0.1:{self.port}/v1/models"
-        last_log = start
-
-        while time.monotonic() - start < timeout:
-            # Check if process died
-            if self._process and self._process.poll() is not None:
-                stderr = ""
-                try:
-                    stderr = self._log_path.read_text()[-2000:]
-                except Exception:
-                    pass
-                raise RuntimeError(f"llama.cpp server exited with code {self._process.returncode}.\nstderr: {stderr}")
-
-            try:
-                async with httpx.AsyncClient() as client:
-                    resp = await client.get(url, timeout=5.0)
-                    if resp.status_code == 200:
-                        logger.info(f"llama.cpp server ready on port {self.port}")
-                        return
-            except (httpx.ConnectError, httpx.TimeoutException, httpx.ConnectTimeout):
-                pass
-
-            # Log progress every 15s
-            now = time.monotonic()
-            if now - last_log > 15:
-                elapsed = int(now - start)
-                logger.info(f"Waiting for llama.cpp server to load model... ({elapsed}s)")
-                last_log = now
-
-            await asyncio.sleep(1.0)
-
-        # Timeout — read the log to help debug
-        stderr = ""
-        try:
-            stderr = self._log_path.read_text()[-2000:]
-        except Exception:
-            pass
-        raise TimeoutError(
-            f"llama.cpp server did not become ready within {timeout}s.\n"
-            f"Check model compatibility and available memory.\n"
-            f"Server log: {stderr}"
-        )
-
-    async def stop(self) -> None:
-        """Stop the llama.cpp server subprocess."""
-        if self._process is None:
-            return
-
-        logger.info("Stopping llama.cpp server...")
-        try:
-            # Send SIGTERM to the process group
-            if hasattr(os, "killpg"):
-                os.killpg(os.getpgid(self._process.pid), signal.SIGTERM)
-            else:
-                self._process.terminate()
-
-            # Wait up to 10s for graceful shutdown
-            try:
-                self._process.wait(timeout=10)
-            except subprocess.TimeoutExpired:
-                if hasattr(os, "killpg"):
-                    os.killpg(os.getpgid(self._process.pid), signal.SIGKILL)
-                else:
-                    self._process.kill()
-                self._process.wait(timeout=5)
-        except (ProcessLookupError, OSError):
-            pass  # Process already exited
-        finally:
-            self._process = None
-            if hasattr(self, "_log_file") and self._log_file:
-                self._log_file.close()
-                self._log_file = None
-            logger.info("llama.cpp server stopped")
-
-
-class LlamaCppLLM(LLMInterface):
-    """
-    Built-in llama.cpp provider.
-
-    Manages a llama-cpp-python server subprocess and delegates to OpenAICompatibleLLM
-    for actual inference calls. Handles model downloading and server lifecycle.
-    """
-
-    def __init__(
-        self,
-        provider: str,
-        api_key: str,
-        base_url: str,
-        model: str,
-        reasoning_effort: str = "low",
-        model_path: str | None = None,
-        gpu_layers: int = -1,
-        context_size: int = 8192,
-        chat_format: str | None = None,
-        no_grammar: bool = False,
-        extra_args: str | None = None,
-        **kwargs: Any,
-    ):
-        super().__init__(
-            provider=provider,
-            api_key=api_key or "llamacpp",
-            base_url=base_url or "",
-            model=model or DEFAULT_LLAMACPP_MODEL_ALIAS,
-            reasoning_effort=reasoning_effort,
-        )
-        self._model_path_str = model_path
-        self._gpu_layers = gpu_layers
-        self._context_size = context_size
-        self._chat_format = chat_format
-        self._no_grammar = no_grammar
-        self._extra_args = extra_args
-        self._server: LlamaCppServer | None = None
-        self._delegate: Any = None  # OpenAICompatibleLLM, created after server starts
-        self._initialized = False
-
-    async def _ensure_initialized(self) -> None:
-        """Lazy initialization: download model + start shared server on first use."""
-        if self._initialized:
-            return
-
-        global _shared_server
-
-        from .openai_compatible_llm import OpenAICompatibleLLM
-
-        async with _shared_server_lock:
-            if _shared_server is None:
-                # Resolve and potentially download the model
-                model_path = _resolve_model_path(self._model_path_str)
-                logger.info(f"Using GGUF model: {model_path}")
-
-                # Start the shared llama.cpp server
-                port = _find_free_port()
-                _shared_server = LlamaCppServer(
-                    model_path=model_path,
-                    port=port,
-                    gpu_layers=self._gpu_layers,
-                    context_size=self._context_size,
-                    chat_format=self._chat_format,
-                    extra_args=self._extra_args,
-                )
-                await _shared_server.start()
-
-        self._server = _shared_server
-
-        # Create the delegate that talks to the shared server's OpenAI-compatible API
-        if self._no_grammar:
-            logger.info("Grammar enforcement disabled (HINDSIGHT_API_LLAMACPP_NO_GRAMMAR=true)")
-        self._delegate = OpenAICompatibleLLM(
-            provider="llamacpp",
-            api_key="llamacpp",
-            base_url=self._server.base_url,
-            model=self.model,
-            reasoning_effort=self.reasoning_effort,
-        )
-
-        self._initialized = True
-
-    async def verify_connection(self) -> None:
-        """Verify the llama.cpp server is running and can generate text."""
-        await self._ensure_initialized()
-        # Make a simple test call to verify the model can actually generate
-        await self._delegate.call(
-            messages=[{"role": "user", "content": "Say 'ok'"}],
-            max_completion_tokens=10,
-            max_retries=2,
-            initial_backoff=0.5,
-            max_backoff=2.0,
-            scope="verification",
-        )
-        logger.info("llama.cpp LLM verification passed")
-
-    async def call(
-        self,
-        messages: list[dict[str, str]],
-        response_format: Any | None = None,
-        max_completion_tokens: int | None = None,
-        temperature: float | None = None,
-        scope: str = "memory",
-        max_retries: int = 10,
-        initial_backoff: float = 1.0,
-        max_backoff: float = 60.0,
-        skip_validation: bool = False,
-        strict_schema: bool = False,
-        return_usage: bool = False,
-    ) -> Any:
-        """Delegate call to the OpenAI-compatible API."""
-        await self._ensure_initialized()
-        return await self._delegate.call(
-            messages=messages,
-            response_format=response_format,
-            max_completion_tokens=max_completion_tokens,
-            temperature=temperature,
-            scope=scope,
-            max_retries=max_retries,
-            initial_backoff=initial_backoff,
-            max_backoff=max_backoff,
-            skip_validation=skip_validation,
-            strict_schema=strict_schema,
-            return_usage=return_usage,
-        )
-
-    async def call_with_tools(
-        self,
-        messages: list[dict[str, Any]],
-        tools: list[dict[str, Any]],
-        max_completion_tokens: int | None = None,
-        temperature: float | None = None,
-        scope: str = "tools",
-        max_retries: int = 5,
-        initial_backoff: float = 1.0,
-        max_backoff: float = 30.0,
-        tool_choice: str | dict[str, Any] = "auto",
-    ) -> LLMToolCallResult:
-        """Delegate tool calls to the OpenAI-compatible API."""
-        await self._ensure_initialized()
-        return await self._delegate.call_with_tools(
-            messages=messages,
-            tools=tools,
-            max_completion_tokens=max_completion_tokens,
-            temperature=temperature,
-            scope=scope,
-            max_retries=max_retries,
-            initial_backoff=initial_backoff,
-            max_backoff=max_backoff,
-            tool_choice=tool_choice,
-        )
-
-    async def cleanup(self) -> None:
-        """Stop the shared llama.cpp server."""
-        global _shared_server
-
-        if self._delegate:
-            await self._delegate.cleanup()
-            self._delegate = None
-
-        # Stop the shared server (only the first cleanup call actually stops it)
-        async with _shared_server_lock:
-            if _shared_server is not None:
-                await _shared_server.stop()
-                _shared_server = None
-
-        self._server = None
-        self._initialized = False

hindsight-api-slim/hindsight_api/engine/providers/none_llm.py

@@ -1,78 +0,0 @@
-"""
-No-op LLM provider for chunk-only storage mode.
-
-When the LLM provider is set to "none", the system operates without any LLM dependency.
-Retain uses chunks mode (no fact extraction), and reflect/consolidation are disabled.
-This provider acts as a safety net — if any code path unexpectedly tries to call the LLM,
-it raises a clear error instead of a confusing connection failure.
-"""
-
-import logging
-from typing import Any
-
-from ..llm_interface import LLMInterface
-from ..response_models import LLMToolCallResult
-
-logger = logging.getLogger(__name__)
-
-
-class LLMNotAvailableError(Exception):
-    """Raised when an operation requires an LLM but the provider is set to 'none'."""
-
-    pass
-
-
-class NoneLLM(LLMInterface):
-    """
-    No-op LLM provider that rejects all LLM calls.
-
-    Used when HINDSIGHT_API_LLM_PROVIDER=none to run Hindsight as a chunk store
-    with semantic search but without LLM-based features (fact extraction, reflect,
-    consolidation).
-    """
-
-    async def verify_connection(self) -> None:
-        """No-op — no LLM connection to verify."""
-        logger.debug("NoneLLM: no LLM connection to verify (provider=none)")
-
-    async def call(
-        self,
-        messages: list[dict[str, str]],
-        response_format: Any | None = None,
-        max_completion_tokens: int | None = None,
-        temperature: float | None = None,
-        scope: str = "memory",
-        max_retries: int = 10,
-        initial_backoff: float = 1.0,
-        max_backoff: float = 60.0,
-        skip_validation: bool = False,
-        strict_schema: bool = False,
-        return_usage: bool = False,
-    ) -> Any:
-        """Raise LLMNotAvailableError — no LLM is configured."""
-        raise LLMNotAvailableError(
-            "LLM provider is set to 'none'. This operation requires an LLM. "
-            "Set HINDSIGHT_API_LLM_PROVIDER to a real provider (e.g., openai, anthropic, gemini)."
-        )
-
-    async def call_with_tools(
-        self,
-        messages: list[dict[str, Any]],
-        tools: list[dict[str, Any]],
-        max_completion_tokens: int | None = None,
-        temperature: float | None = None,
-        scope: str = "tools",
-        max_retries: int = 5,
-        initial_backoff: float = 1.0,
-        max_backoff: float = 30.0,
-        tool_choice: str | dict[str, Any] = "auto",
-    ) -> LLMToolCallResult:
-        """Raise LLMNotAvailableError — no LLM is configured."""
-        raise LLMNotAvailableError(
-            "LLM provider is set to 'none'. This operation requires an LLM. "
-            "Set HINDSIGHT_API_LLM_PROVIDER to a real provider (e.g., openai, anthropic, gemini)."
-        )
-
-    async def cleanup(self) -> None:
-        """No-op — nothing to clean up."""
-        pass

hindsight-api-slim/hindsight_api/engine/retain/chunk_storage.py

@@ -1,144 +0,0 @@
-"""
-Chunk storage for retain pipeline.
-
-Handles storage of document chunks in the database.
-"""
-
-import hashlib
-import logging
-from dataclasses import dataclass
-
-from ..memory_engine import fq_table
-from .types import ChunkMetadata
-
-logger = logging.getLogger(__name__)
-
-
-def compute_chunk_hash(chunk_text: str) -> str:
-    """Compute SHA256 hash of chunk text for delta comparison."""
-    return hashlib.sha256(chunk_text.encode()).hexdigest()
-
-
-@dataclass
-class ExistingChunk:
-    """Represents a chunk already stored in the database."""
-
-    chunk_id: str
-    chunk_index: int
-    content_hash: str | None
-
-
-async def load_existing_chunks(conn, bank_id: str, document_id: str) -> list[ExistingChunk]:
-    """
-    Load existing chunk metadata for a document.
-
-    Returns list of ExistingChunk with chunk_id, chunk_index, and content_hash.
-    """
-    rows = await conn.fetch(
-        f"""
-        SELECT chunk_id, chunk_index, content_hash
-        FROM {fq_table("chunks")}
-        WHERE document_id = $1 AND bank_id = $2
-        ORDER BY chunk_index
-        """,
-        document_id,
-        bank_id,
-    )
-    return [
-        ExistingChunk(
-            chunk_id=row["chunk_id"],
-            chunk_index=row["chunk_index"],
-            content_hash=row["content_hash"],
-        )
-        for row in rows
-    ]
-
-
-async def delete_chunks_by_ids(conn, chunk_ids: list[str]) -> None:
-    """
-    Delete specific chunks by their IDs.
-
-    This cascades to memory_units (via FK with CASCADE delete)
-    and their links.
-    """
-    if not chunk_ids:
-        return
-    await conn.execute(
-        f"DELETE FROM {fq_table('chunks')} WHERE chunk_id = ANY($1::text[])",
-        chunk_ids,
-    )
-
-
-async def store_chunks_batch(conn, bank_id: str, document_id: str, chunks: list[ChunkMetadata]) -> dict[int, str]:
-    """
-    Store document chunks in the database.
-
-    Args:
-        conn: Database connection
-        bank_id: Bank identifier
-        document_id: Document identifier
-        chunks: List of ChunkMetadata objects
-
-    Returns:
-        Dictionary mapping global chunk index to chunk_id
-    """
-    if not chunks:
-        return {}
-
-    # Prepare chunk data for batch insert
-    chunk_ids = []
-    chunk_texts = []
-    chunk_indices = []
-    content_hashes = []
-    chunk_id_map = {}
-
-    for chunk in chunks:
-        chunk_id = f"{bank_id}_{document_id}_{chunk.chunk_index}"
-        chunk_ids.append(chunk_id)
-        chunk_texts.append(chunk.chunk_text)
-        chunk_indices.append(chunk.chunk_index)
-        content_hashes.append(compute_chunk_hash(chunk.chunk_text))
-        chunk_id_map[chunk.chunk_index] = chunk_id
-
-    # Batch upsert all chunks. ON CONFLICT makes this idempotent: re-submitting
-    # a retain under the same document_id (the pattern in vectorize-io/hindsight#977)
-    # may produce chunk_ids that already exist when upstream cascade-delete or
-    # delta-retain paths don't run (or race with a concurrent task). Overwriting
-    # is the correct behavior per the document_id grouping semantics — the caller
-    # intends this chunk to hold the latest content at that (document_id, index).
-    await conn.execute(
-        f"""
-        INSERT INTO {fq_table("chunks")} (chunk_id, document_id, bank_id, chunk_text, chunk_index, content_hash)
-        SELECT * FROM unnest($1::text[], $2::text[], $3::text[], $4::text[], $5::integer[], $6::text[])
-        ON CONFLICT (chunk_id) DO UPDATE SET
-            chunk_text = EXCLUDED.chunk_text,
-            chunk_index = EXCLUDED.chunk_index,
-            content_hash = EXCLUDED.content_hash
-        """,
-        chunk_ids,
-        [document_id] * len(chunk_texts),
-        [bank_id] * len(chunk_texts),
-        chunk_texts,
-        chunk_indices,
-        content_hashes,
-    )
-
-    return chunk_id_map
-
-
-def map_facts_to_chunks(facts_chunk_indices: list[int], chunk_id_map: dict[int, str]) -> list[str | None]:
-    """
-    Map fact chunk indices to chunk IDs.
-
-    Args:
-        facts_chunk_indices: List of chunk indices for each fact
-        chunk_id_map: Dictionary mapping chunk index to chunk_id
-
-    Returns:
-        List of chunk_ids (same length as facts_chunk_indices)
-    """
-    chunk_ids = []
-    for chunk_idx in facts_chunk_indices:
-        chunk_id = chunk_id_map.get(chunk_idx)
-        chunk_ids.append(chunk_id)
-    return chunk_ids

hindsight-api-slim/hindsight_api/engine/retain/entity_processing.py

@@ -1,162 +0,0 @@
-"""
-Entity processing for retain pipeline.
-
-Handles entity extraction, resolution, and link creation for stored facts.
-"""
-
-import logging
-
-from . import link_utils
-from .types import EntityLink, ProcessedFact
-
-logger = logging.getLogger(__name__)
-
-
-def _prepare_facts_for_entity_processing(
-    facts: list[ProcessedFact],
-    user_entities_per_content: dict[int, list[dict]] | None = None,
-) -> tuple[list[str], list, list[list[dict]]]:
-    """
-    Extract fact texts, dates, and merged entity lists from ProcessedFact objects.
-
-    Returns:
-        Tuple of (fact_texts, fact_dates, entities_per_fact)
-    """
-    user_entities_per_content = user_entities_per_content or {}
-
-    fact_texts = [fact.fact_text for fact in facts]
-    fact_dates = [fact.occurred_start if fact.occurred_start is not None else fact.mentioned_at for fact in facts]
-
-    entities_per_fact = []
-    for fact in facts:
-        llm_entities = [{"text": entity.name, "type": "CONCEPT"} for entity in (fact.entities or [])]
-
-        user_entities = user_entities_per_content.get(fact.content_index, [])
-
-        seen_texts = {e["text"].lower() for e in llm_entities}
-        for user_entity in user_entities:
-            if user_entity["text"].lower() not in seen_texts:
-                llm_entities.append(
-                    {
-                        "text": user_entity["text"],
-                        "type": user_entity.get("type", "CONCEPT"),
-                    }
-                )
-                seen_texts.add(user_entity["text"].lower())
-
-        entities_per_fact.append(llm_entities)
-
-    return fact_texts, fact_dates, entities_per_fact
-
-
-async def resolve_entities(
-    entity_resolver,
-    conn,
-    bank_id: str,
-    unit_ids: list[str],
-    facts: list[ProcessedFact],
-    log_buffer: list[str] = None,
-    user_entities_per_content: dict[int, list[dict]] = None,
-    entity_labels: list | None = None,
-) -> tuple[list[str], list[tuple], dict[str, list[str]]]:
-    """
-    Phase 1: Resolve entity names to canonical IDs (read-heavy).
-
-    Should be called on a SEPARATE connection OUTSIDE the main write transaction
-    to avoid holding the transaction open during expensive trigram scans.
-
-    Args:
-        entity_resolver: EntityResolver instance
-        conn: Database connection (separate from the main write transaction)
-        bank_id: Bank identifier
-        unit_ids: Placeholder unit IDs (used only for grouping)
-        facts: List of ProcessedFact objects
-        log_buffer: Optional buffer for detailed logging
-        user_entities_per_content: Dict mapping content_index to user-provided entities
-        entity_labels: Optional entity label taxonomy
-
-    Returns:
-        Tuple of (resolved_entity_ids, entity_to_unit, unit_to_entity_ids)
-        to pass to build_entity_links().
-    """
-    if not unit_ids or not facts:
-        return [], [], {}
-
-    if len(unit_ids) != len(facts):
-        raise ValueError(f"Mismatch between unit_ids ({len(unit_ids)}) and facts ({len(facts)})")
-
-    fact_texts, fact_dates, entities_per_fact = _prepare_facts_for_entity_processing(facts, user_entities_per_content)
-
-    return await link_utils.resolve_entities_only(
-        entity_resolver,
-        conn,
-        bank_id,
-        unit_ids,
-        fact_texts,
-        "",  # context (not used in current implementation)
-        fact_dates,
-        entities_per_fact,
-        log_buffer,
-        entity_labels=entity_labels,
-    )
-
-
-async def build_entity_links(
-    entity_resolver,
-    conn,
-    bank_id: str,
-    unit_ids: list[str],
-    resolved_entity_ids: list[str],
-    entity_to_unit: list[tuple],
-    unit_to_entity_ids: dict[str, list[str]],
-    log_buffer: list[str] = None,
-    skip_unit_entities_insert: bool = False,
-) -> list[EntityLink]:
-    """
-    Build entity links for UI graph visualization.
-
-    Queries unit_entities to find shared entities between new and existing units,
-    then generates EntityLink objects. When called from Phase 3 (post-transaction),
-    set skip_unit_entities_insert=True since unit_entities were already inserted
-    in Phase 2.
-
-    Args:
-        entity_resolver: EntityResolver instance
-        conn: Database connection
-        bank_id: Bank identifier
-        unit_ids: Actual unit IDs (must already be inserted in the DB)
-        resolved_entity_ids: From resolve_entities()
-        entity_to_unit: From resolve_entities()
-        unit_to_entity_ids: From resolve_entities()
-        log_buffer: Optional buffer for detailed logging
-        skip_unit_entities_insert: Skip unit_entities INSERT (already done in Phase 2)
-
-    Returns:
-        List of EntityLink objects for batch insertion
-    """
-    return await link_utils.build_entity_links_from_resolved(
-        entity_resolver,
-        conn,
-        bank_id,
-        unit_ids,
-        resolved_entity_ids,
-        entity_to_unit,
-        unit_to_entity_ids,
-        log_buffer,
-        skip_unit_entities_insert=skip_unit_entities_insert,
-    )
-
-
-async def insert_entity_links_batch(conn, entity_links: list[EntityLink], bank_id: str) -> None:
-    """
-    Insert entity links in batch.
-
-    Args:
-        conn: Database connection
-        entity_links: List of EntityLink objects
-        bank_id: Bank identifier (stored directly on memory_links for fast filtering)
-    """
-    if not entity_links:
-        return
-
-    await link_utils.insert_entity_links_batch(conn, entity_links, bank_id)

hindsight-api-slim/hindsight_api/engine/search/graph_retrieval.py

@@ -1,67 +0,0 @@
-"""
-Graph retrieval strategies for memory recall.
-
-This module provides an abstraction for graph-based memory retrieval,
-allowing different algorithms to be swapped without changing the rest
-of the recall pipeline.
-"""
-
-import logging
-from abc import ABC, abstractmethod
-
-from .tags import TagGroup, TagsMatch
-from .types import GraphRetrievalTimings, RetrievalResult
-
-logger = logging.getLogger(__name__)
-
-
-class GraphRetriever(ABC):
-    """
-    Abstract base class for graph-based memory retrieval.
-
-    Implementations traverse the memory graph (entity links, temporal links,
-    causal links) to find relevant facts that might not be found by
-    semantic or keyword search alone.
-    """
-
-    @property
-    @abstractmethod
-    def name(self) -> str:
-        """Return identifier for this retrieval strategy (e.g., 'link_expansion')."""
-        pass
-
-    @abstractmethod
-    async def retrieve(
-        self,
-        pool,
-        query_embedding_str: str,
-        bank_id: str,
-        fact_type: str,
-        budget: int,
-        query_text: str | None = None,
-        semantic_seeds: list[RetrievalResult] | None = None,
-        temporal_seeds: list[RetrievalResult] | None = None,
-        adjacency=None,  # TypedAdjacency, optional pre-loaded graph
-        tags: list[str] | None = None,  # Visibility scope tags for filtering
-        tags_match: TagsMatch = "any",  # How to match tags: 'any' (OR) or 'all' (AND)
-        tag_groups: list[TagGroup] | None = None,  # Compound boolean tag filter groups
-    ) -> tuple[list[RetrievalResult], GraphRetrievalTimings | None]:
-        """
-        Retrieve relevant facts via graph traversal.
-
-        Args:
-            pool: Database connection pool
-            query_embedding_str: Query embedding as string (for finding entry points)
-            bank_id: Memory bank identifier
-            fact_type: Fact type to filter ('world', 'experience', 'observation')
-            budget: Maximum number of nodes to explore/return
-            query_text: Original query text (optional, for some strategies)
-            semantic_seeds: Pre-computed semantic entry points (from semantic retrieval)
-            temporal_seeds: Pre-computed temporal entry points (from temporal retrieval)
-            adjacency: Pre-loaded typed adjacency graph (optional)
-            tags: Optional list of tags for visibility filtering (OR matching)
-
-        Returns:
-            Tuple of (List of RetrievalResult with activation scores, optional timing info)
-        """
-        pass

hindsight-api-slim/hindsight_api/engine/search/tags.py

@@ -1,390 +0,0 @@
-"""
-Tags filtering utilities for retrieval.
-
-Provides SQL building functions for filtering memories by tags.
-Supports four matching modes via TagsMatch enum:
-- "any": OR matching, includes untagged memories (default, backward compatible)
-- "all": AND matching, includes untagged memories
-- "any_strict": OR matching, excludes untagged memories
-- "all_strict": AND matching, excludes untagged memories
-
-OR matching (any/any_strict): Memory matches if ANY of its tags overlap with request tags
-AND matching (all/all_strict): Memory matches if ALL request tags are present in its tags
-"""
-
-from __future__ import annotations
-
-from typing import Annotated, Literal
-
-from pydantic import BaseModel, ConfigDict, Field
-
-TagsMatch = Literal["any", "all", "any_strict", "all_strict"]
-
-
-def _parse_tags_match(match: TagsMatch) -> tuple[str, bool]:
-    """
-    Parse TagsMatch into operator and include_untagged flag.
-
-    Returns:
-        Tuple of (operator, include_untagged)
-        - operator: "&&" for any/any_strict, "@>" for all/all_strict
-        - include_untagged: True for any/all, False for any_strict/all_strict
-    """
-    if match == "any":
-        return "&&", True
-    elif match == "all":
-        return "@>", True
-    elif match == "any_strict":
-        return "&&", False
-    elif match == "all_strict":
-        return "@>", False
-    else:
-        # Default to "any" behavior
-        return "&&", True
-
-
-def build_tags_where_clause(
-    tags: list[str] | None,
-    param_offset: int = 1,
-    table_alias: str = "",
-    match: TagsMatch = "any",
-) -> tuple[str, list, int]:
-    """
-    Build a SQL WHERE clause for filtering by tags.
-
-    Supports four matching modes:
-    - "any" (default): OR matching, includes untagged memories
-    - "all": AND matching, includes untagged memories
-    - "any_strict": OR matching, excludes untagged memories
-    - "all_strict": AND matching, excludes untagged memories
-
-    Args:
-        tags: List of tags to filter by. If None or empty, returns empty clause (no filtering).
-        param_offset: Starting parameter number for SQL placeholders (default 1).
-        table_alias: Optional table alias prefix (e.g., "mu." for "memory_units mu").
-        match: Matching mode. Defaults to "any".
-
-    Returns:
-        Tuple of (sql_clause, params, next_param_offset):
-        - sql_clause: SQL WHERE clause string
-        - params: List of parameter values to bind
-        - next_param_offset: Next available parameter number
-
-    Example:
-        >>> clause, params, next_offset = build_tags_where_clause(['user_a'], 3, 'mu.', 'any_strict')
-        >>> print(clause)  # "AND mu.tags IS NOT NULL AND mu.tags != '{}' AND mu.tags && $3"
-    """
-    if not tags:
-        return "", [], param_offset
-
-    column = f"{table_alias}tags" if table_alias else "tags"
-    operator, include_untagged = _parse_tags_match(match)
-
-    if include_untagged:
-        # Include untagged memories (NULL or empty array) OR matching tags
-        clause = f"AND ({column} IS NULL OR {column} = '{{}}' OR {column} {operator} ${param_offset})"
-    else:
-        # Strict: only memories with matching tags (exclude NULL and empty)
-        clause = f"AND {column} IS NOT NULL AND {column} != '{{}}' AND {column} {operator} ${param_offset}"
-
-    return clause, [tags], param_offset + 1
-
-
-def build_tags_where_clause_simple(
-    tags: list[str] | None,
-    param_num: int,
-    table_alias: str = "",
-    match: TagsMatch = "any",
-) -> str:
-    """
-    Build a simple SQL WHERE clause for tags filtering.
-
-    This is a convenience version that returns just the clause string,
-    assuming the caller will add the tags array to their params list.
-
-    Args:
-        tags: List of tags to filter by. If None or empty, returns empty string.
-        param_num: Parameter number to use in the clause.
-        table_alias: Optional table alias prefix.
-        match: Matching mode. Defaults to "any".
-
-    Returns:
-        SQL clause string or empty string.
-    """
-    if not tags:
-        return ""
-
-    column = f"{table_alias}tags" if table_alias else "tags"
-    operator, include_untagged = _parse_tags_match(match)
-
-    if include_untagged:
-        # Include untagged memories (NULL or empty array) OR matching tags
-        return f"AND ({column} IS NULL OR {column} = '{{}}' OR {column} {operator} ${param_num})"
-    else:
-        # Strict: only memories with matching tags (exclude NULL and empty)
-        return f"AND {column} IS NOT NULL AND {column} != '{{}}' AND {column} {operator} ${param_num}"
-
-
-def filter_results_by_tags(
-    results: list,
-    tags: list[str] | None,
-    match: TagsMatch = "any",
-) -> list:
-    """
-    Filter retrieval results by tags in Python (for post-processing).
-
-    Used when SQL filtering isn't possible (e.g., graph traversal results).
-
-    Args:
-        results: List of RetrievalResult objects with a 'tags' attribute.
-        tags: List of tags to filter by. If None or empty, returns all results.
-        match: Matching mode. Defaults to "any".
-
-    Returns:
-        Filtered list of results.
-    """
-    if not tags:
-        return results
-
-    _, include_untagged = _parse_tags_match(match)
-    is_any_match = match in ("any", "any_strict")
-
-    tags_set = set(tags)
-    filtered = []
-
-    for result in results:
-        result_tags = getattr(result, "tags", None)
-
-        # Check if untagged
-        is_untagged = result_tags is None or len(result_tags) == 0
-
-        if is_untagged:
-            if include_untagged:
-                filtered.append(result)
-            # else: skip untagged
-        else:
-            result_tags_set = set(result_tags)
-            if is_any_match:
-                # Any overlap
-                if result_tags_set & tags_set:
-                    filtered.append(result)
-            else:
-                # All tags must be present
-                if tags_set <= result_tags_set:
-                    filtered.append(result)
-
-    return filtered
-
-
-# =============================================================================
-# Compound tag group models (recursive boolean expressions)
-# =============================================================================
-
-
-class TagGroupLeaf(BaseModel):
-    """A leaf tag filter: matches memories by tag list and match mode."""
-
-    tags: list[str]
-    match: TagsMatch = "any_strict"
-
-
-class TagGroupAnd(BaseModel):
-    """Compound AND group: all child filters must match."""
-
-    model_config = ConfigDict(populate_by_name=True)
-    filters: list[TagGroup] = Field(alias="and")
-
-
-class TagGroupOr(BaseModel):
-    """Compound OR group: at least one child filter must match."""
-
-    model_config = ConfigDict(populate_by_name=True)
-    filters: list[TagGroup] = Field(alias="or")
-
-
-class TagGroupNot(BaseModel):
-    """Compound NOT group: child filter must NOT match."""
-
-    model_config = ConfigDict(populate_by_name=True)
-    filter: TagGroup = Field(alias="not")
-
-
-# TagGroup is a discriminated union; Pydantic will try left-to-right.
-# TagGroupLeaf is identified by the presence of 'tags'.
-# TagGroupAnd / TagGroupOr / TagGroupNot are compound (no 'tags' key).
-TagGroup = Annotated[
-    TagGroupLeaf | TagGroupAnd | TagGroupOr | TagGroupNot,
-    Field(union_mode="left_to_right"),
-]
-
-# Rebuild forward-reference models so recursive TagGroup is resolved.
-TagGroupAnd.model_rebuild()
-TagGroupOr.model_rebuild()
-TagGroupNot.model_rebuild()
-
-
-# =============================================================================
-# SQL builder for compound tag groups
-# =============================================================================
-
-
-def _build_group_clause(
-    group: TagGroup,
-    param_offset: int,
-    table_alias: str,
-) -> tuple[str, list, int]:
-    """
-    Recursively build an inner SQL clause (no leading AND/OR) for a single TagGroup.
-
-    Returns:
-        (inner_clause, params, next_param_offset)
-    """
-    if isinstance(group, TagGroupLeaf):
-        column = f"{table_alias}tags" if table_alias else "tags"
-        operator, include_untagged = _parse_tags_match(group.match)
-        if include_untagged:
-            clause = f"({column} IS NULL OR {column} = '{{}}' OR {column} {operator} ${param_offset})"
-        else:
-            clause = f"({column} IS NOT NULL AND {column} != '{{}}' AND {column} {operator} ${param_offset})"
-        return clause, [group.tags], param_offset + 1
-
-    elif isinstance(group, TagGroupAnd):
-        parts = []
-        params: list = []
-        offset = param_offset
-        for child in group.filters:
-            child_clause, child_params, offset = _build_group_clause(child, offset, table_alias)
-            parts.append(child_clause)
-            params.extend(child_params)
-        inner = " AND ".join(parts)
-        return f"({inner})", params, offset
-
-    elif isinstance(group, TagGroupOr):
-        parts = []
-        params = []
-        offset = param_offset
-        for child in group.filters:
-            child_clause, child_params, offset = _build_group_clause(child, offset, table_alias)
-            parts.append(child_clause)
-            params.extend(child_params)
-        inner = " OR ".join(parts)
-        return f"({inner})", params, offset
-
-    elif isinstance(group, TagGroupNot):
-        child_clause, child_params, next_offset = _build_group_clause(group.filter, param_offset, table_alias)
-        return f"NOT {child_clause}", child_params, next_offset
-
-    else:
-        # Should never happen with proper Pydantic validation
-        return "", [], param_offset
-
-
-def build_tag_groups_where_clause(
-    tag_groups: list[TagGroup] | None,
-    param_offset: int,
-    table_alias: str = "",
-) -> tuple[str, list, int]:
-    """
-    Build a SQL WHERE clause for compound tag group filtering.
-
-    Top-level groups are AND-ed together. Each group is a recursive boolean
-    expression (leaf, and, or, not).
-
-    Args:
-        tag_groups: List of TagGroup objects. If None or empty, returns empty clause.
-        param_offset: Starting parameter number for SQL placeholders.
-        table_alias: Optional table alias prefix (e.g., "mu." for "memory_units mu").
-
-    Returns:
-        Tuple of (sql_clause, params, next_param_offset):
-        - sql_clause: SQL WHERE clause string starting with "AND" (or empty string)
-        - params: List of parameter values to bind (one per leaf node)
-        - next_param_offset: Next available parameter number
-
-    Example:
-        >>> groups = [TagGroupLeaf(tags=["user:alice"], match="all_strict")]
-        >>> clause, params, next_offset = build_tag_groups_where_clause(groups, 3)
-        >>> print(clause)  # "AND (tags IS NOT NULL AND tags != '{}' AND tags @> $3)"
-    """
-    if not tag_groups:
-        return "", [], param_offset
-
-    all_params: list = []
-    all_clauses: list[str] = []
-    offset = param_offset
-
-    for group in tag_groups:
-        inner_clause, group_params, offset = _build_group_clause(group, offset, table_alias)
-        all_clauses.append(inner_clause)
-        all_params.extend(group_params)
-
-    combined = " AND ".join(all_clauses)
-    return f"AND {combined}", all_params, offset
-
-
-# =============================================================================
-# Python-side filter for compound tag groups (post-retrieval filtering)
-# =============================================================================
-
-
-def _match_group(result: object, group: TagGroup) -> bool:
-    """
-    Recursively evaluate a TagGroup against a retrieval result.
-
-    Args:
-        result: Any object with a 'tags' attribute (list[str] or None).
-        group: The TagGroup to evaluate.
-
-    Returns:
-        True if the result matches the group, False otherwise.
-    """
-    if isinstance(group, TagGroupLeaf):
-        result_tags = getattr(result, "tags", None)
-        is_untagged = result_tags is None or len(result_tags) == 0
-        _, include_untagged = _parse_tags_match(group.match)
-        is_any_match = group.match in ("any", "any_strict")
-        tags_set = set(group.tags)
-
-        if is_untagged:
-            return include_untagged
-        else:
-            result_tags_set = set(result_tags)
-            if is_any_match:
-                return bool(result_tags_set & tags_set)
-            else:
-                return tags_set <= result_tags_set
-
-    elif isinstance(group, TagGroupAnd):
-        return all(_match_group(result, child) for child in group.filters)
-
-    elif isinstance(group, TagGroupOr):
-        return any(_match_group(result, child) for child in group.filters)
-
-    elif isinstance(group, TagGroupNot):
-        return not _match_group(result, group.filter)
-
-    else:
-        return True
-
-
-def filter_results_by_tag_groups(
-    results: list,
-    tag_groups: list[TagGroup] | None,
-) -> list:
-    """
-    Filter retrieval results by compound tag groups in Python (for post-processing).
-
-    Used when SQL filtering isn't possible (e.g., graph traversal results).
-    Top-level groups are AND-ed together.
-
-    Args:
-        results: List of RetrievalResult objects with a 'tags' attribute.
-        tag_groups: List of TagGroup objects. If None or empty, returns all results.
-
-    Returns:
-        Filtered list of results where ALL top-level groups match.
-    """
-    if not tag_groups:
-        return results
-
-    return [r for r in results if all(_match_group(r, group) for group in tag_groups)]

hindsight-api-slim/hindsight_api/main.py

@@ -1,302 +0,0 @@
-"""
-Command-line interface for Hindsight API.
-
-Run the server with:
-    hindsight-api
-
-Run as background daemon:
-    hindsight-api --daemon
-
-Stop with Ctrl+C.
-"""
-
-import argparse
-import asyncio
-import atexit
-import dataclasses
-import os
-import signal
-import sys
-import warnings
-
-import uvicorn
-
-from . import MemoryEngine, __version__
-from .api import create_app
-from .banner import print_banner
-from .config import DEFAULT_WORKERS, ENV_WORKERS, HindsightConfig, _get_raw_config
-from .daemon import (
-    DEFAULT_DAEMON_PORT,
-    DEFAULT_IDLE_TIMEOUT,
-    IdleTimeoutMiddleware,
-    daemonize,
-)
-from .extensions import DefaultExtensionContext, OperationValidatorExtension, TenantExtension, load_extension
-
-# Filter deprecation warnings from third-party libraries
-warnings.filterwarnings("ignore", message="websockets.legacy is deprecated")
-warnings.filterwarnings("ignore", message="websockets.server.WebSocketServerProtocol is deprecated")
-
-# Disable tokenizers parallelism to avoid warnings
-os.environ["TOKENIZERS_PARALLELISM"] = "false"
-
-# Global reference for cleanup
-_memory: MemoryEngine | None = None
-
-
-def _cleanup():
-    """Synchronous cleanup function to stop resources on exit."""
-    global _memory
-    if _memory is not None and _memory._pg0 is not None:
-        try:
-            loop = asyncio.new_event_loop()
-            loop.run_until_complete(_memory._pg0.stop())
-            loop.close()
-            print("\npg0 stopped.")
-        except Exception as e:
-            print(f"\nError stopping pg0: {e}")
-
-
-def _signal_handler(signum, frame):
-    """Handle SIGINT/SIGTERM to ensure cleanup."""
-    print(f"\nReceived signal {signum}, shutting down...")
-    _cleanup()
-    sys.exit(0)
-
-
-def main():
-    """Main entry point for the CLI."""
-    global _memory
-
-    # Load configuration from environment (for CLI args defaults)
-    config = _get_raw_config()
-
-    parser = argparse.ArgumentParser(
-        prog="hindsight-api",
-        description="Hindsight API Server",
-    )
-
-    # Server options
-    parser.add_argument(
-        "--host", default=config.host, help=f"Host to bind to (default: {config.host}, env: HINDSIGHT_API_HOST)"
-    )
-    parser.add_argument(
-        "--port",
-        type=int,
-        default=config.port,
-        help=f"Port to bind to (default: {config.port}, env: HINDSIGHT_API_PORT)",
-    )
-    parser.add_argument(
-        "--log-level",
-        default=config.log_level,
-        choices=["critical", "error", "warning", "info", "debug", "trace"],
-        help=f"Log level (default: {config.log_level}, env: HINDSIGHT_API_LOG_LEVEL)",
-    )
-
-    # Development options
-    parser.add_argument("--reload", action="store_true", help="Enable auto-reload on code changes (development only)")
-    parser.add_argument(
-        "--workers",
-        type=int,
-        default=int(os.getenv(ENV_WORKERS, str(DEFAULT_WORKERS))),
-        help=f"Number of worker processes (env: {ENV_WORKERS}, default: {DEFAULT_WORKERS})",
-    )
-
-    # Access log options
-    parser.add_argument("--access-log", action="store_true", help="Enable access log")
-    parser.add_argument("--no-access-log", dest="access_log", action="store_false", help="Disable access log (default)")
-    parser.set_defaults(access_log=False)
-
-    # Proxy options
-    parser.add_argument(
-        "--proxy-headers", action="store_true", help="Enable X-Forwarded-Proto, X-Forwarded-For headers"
-    )
-    parser.add_argument(
-        "--forwarded-allow-ips", default=None, help="Comma separated list of IPs to trust with proxy headers"
-    )
-
-    # SSL options
-    parser.add_argument("--ssl-keyfile", default=None, help="SSL key file")
-    parser.add_argument("--ssl-certfile", default=None, help="SSL certificate file")
-
-    # Daemon mode options
-    parser.add_argument(
-        "--daemon",
-        action="store_true",
-        help=f"Run as background daemon (uses port {DEFAULT_DAEMON_PORT}, auto-exits after idle)",
-    )
-    parser.add_argument(
-        "--idle-timeout",
-        type=int,
-        default=DEFAULT_IDLE_TIMEOUT,
-        help=f"Idle timeout in seconds before auto-exit in daemon mode (default: {DEFAULT_IDLE_TIMEOUT})",
-    )
-
-    args = parser.parse_args()
-
-    # Daemon mode handling
-    if args.daemon:
-        # Use port from args (may be custom for profiles)
-        if args.port == config.port:  # No custom port specified
-            args.port = DEFAULT_DAEMON_PORT
-        args.host = "127.0.0.1"  # Only bind to localhost for security
-
-        # Fork into background
-        # No lockfile needed - port binding prevents duplicate daemons
-        daemonize()
-
-    # Print banner (not in daemon mode)
-    if not args.daemon:
-        print()
-        print_banner()
-
-    # Configure Python logging based on log level
-    # Update config with CLI override if provided
-    if args.log_level != config.log_level:
-        config = dataclasses.replace(config, host=args.host, port=args.port, log_level=args.log_level)
-    config.configure_logging()
-    if not args.daemon:
-        config.log_config()
-
-    # Register cleanup handlers
-    atexit.register(_cleanup)
-    signal.signal(signal.SIGINT, _signal_handler)
-    signal.signal(signal.SIGTERM, _signal_handler)
-
-    # Load operation validator extension if configured
-    operation_validator = load_extension("OPERATION_VALIDATOR", OperationValidatorExtension)
-    if operation_validator:
-        import logging
-
-        logging.info(f"Loaded operation validator: {operation_validator.__class__.__name__}")
-
-    # Load tenant extension if configured
-    tenant_extension = load_extension("TENANT", TenantExtension)
-    if tenant_extension:
-        import logging
-
-        logging.info(f"Loaded tenant extension: {tenant_extension.__class__.__name__}")
-
-    # Create MemoryEngine (reads configuration from environment)
-    _memory = MemoryEngine(
-        operation_validator=operation_validator,
-        tenant_extension=tenant_extension,
-        run_migrations=config.run_migrations_on_startup,
-    )
-
-    # Set extension context on tenant extension (needed for schema provisioning)
-    if tenant_extension:
-        extension_context = DefaultExtensionContext(
-            database_url=config.database_url,
-            memory_engine=_memory,
-        )
-        tenant_extension.set_context(extension_context)
-        logging.info("Extension context set on tenant extension")
-
-    # Create FastAPI app
-    app = create_app(
-        memory=_memory,
-        http_api_enabled=True,
-        mcp_api_enabled=config.mcp_enabled,
-        mcp_mount_path="/mcp",
-        initialize_memory=True,
-    )
-
-    # Wrap with idle timeout middleware in daemon mode
-    idle_middleware = None
-    if args.daemon:
-        idle_middleware = IdleTimeoutMiddleware(app, idle_timeout=args.idle_timeout)
-        app = idle_middleware
-
-    # Prepare uvicorn config
-    # When using workers or reload, we must use import string so each worker can import the app
-    use_import_string = args.workers > 1 or args.reload
-    # Check for uvloop/winloop availability
-    import sys
-
-    loop_impl = "asyncio"
-    if sys.platform == "win32":
-        try:
-            import winloop
-
-            winloop.install()  # Patches asyncio globally — uvicorn uses "asyncio" but gets winloop
-            loop_impl = "asyncio"  # Tell uvicorn "asyncio" — it's now winloop underneath
-            print("winloop installed as asyncio event loop policy (Windows uvloop port)")
-        except ImportError:
-            print("winloop not installed, using default asyncio event loop")
-    else:
-        try:
-            import uvloop  # noqa: F401
-
-            loop_impl = "uvloop"
-            print("uvloop available, will use for event loop")
-        except ImportError:
-            print("uvloop not installed, using default asyncio event loop")
-
-    uvicorn_config = {
-        "app": "hindsight_api.server:app" if use_import_string else app,
-        "host": args.host,
-        "port": args.port,
-        "log_level": args.log_level,
-        "access_log": args.access_log,
-        "proxy_headers": args.proxy_headers,
-        "ws": "wsproto",  # Use wsproto instead of websockets to avoid deprecation warnings
-        "loop": loop_impl,  # Explicitly set event loop implementation
-        "timeout_keep_alive": 30,  # Exceed aiohttp's 15s client timeout so the client always closes first
-        "timeout_graceful_shutdown": 5,  # Cap graceful shutdown at 5s; also enables force-kill on second Ctrl+C
-    }
-
-    # Add optional parameters if provided
-    if args.reload:
-        uvicorn_config["reload"] = True
-    if args.workers > 1:
-        uvicorn_config["workers"] = args.workers
-    if args.forwarded_allow_ips:
-        uvicorn_config["forwarded_allow_ips"] = args.forwarded_allow_ips
-    if args.ssl_keyfile:
-        uvicorn_config["ssl_keyfile"] = args.ssl_keyfile
-    if args.ssl_certfile:
-        uvicorn_config["ssl_certfile"] = args.ssl_certfile
-
-    # Print startup info (not in daemon mode)
-    if not args.daemon:
-        from .banner import print_startup_info
-
-        print_startup_info(
-            host=args.host,
-            port=args.port,
-            database_url=config.database_url,
-            llm_provider=config.llm_provider,
-            llm_model=config.llm_model,
-            embeddings_provider=config.embeddings_provider,
-            reranker_provider=config.reranker_provider,
-            mcp_enabled=config.mcp_enabled,
-            version=__version__,
-            vector_extension=config.vector_extension,
-            text_search_extension=config.text_search_extension,
-        )
-
-    # Start idle checker in daemon mode
-    if idle_middleware is not None:
-        # Start the idle checker in a background thread with its own event loop
-        import logging
-        import threading
-
-        def run_idle_checker():
-            import time
-
-            time.sleep(2)  # Wait for uvicorn to start
-            try:
-                loop = asyncio.new_event_loop()
-                asyncio.set_event_loop(loop)
-                loop.run_until_complete(idle_middleware._check_idle())
-            except Exception as e:
-                logging.error(f"Idle checker error: {e}", exc_info=True)
-
-        threading.Thread(target=run_idle_checker, daemon=True).start()
-
-    uvicorn.run(**uvicorn_config)
-
-
-if __name__ == "__main__":
-    main()

hindsight-api-slim/hindsight_api/worker/stage.py

@@ -1,59 +0,0 @@
-"""Stage breadcrumbs for in-flight worker tasks.
-
-The worker poller binds a `StageHolder` to each task's contextvar scope.
-Engine code calls `set_stage("retain.facts.llm")` at phase boundaries; the
-poller reads the holder periodically to surface what each in-flight task is
-currently doing in `WORKER_STATS` / `WORKER_TASK` log lines.
-
-Outside a worker context the contextvar is unset and `set_stage` is a no-op,
-so engine code is safe to call from sync HTTP requests, tests, or the CLI
-without any setup.
-"""
-
-from __future__ import annotations
-
-import time
-from contextvars import ContextVar
-from dataclasses import dataclass, field
-
-
-@dataclass
-class StageHolder:
-    """Mutable container for the current task's stage label."""
-
-    stage: str = "init"
-    updated_at: float = field(default_factory=time.monotonic)
-
-
-_current_holder: ContextVar[StageHolder | None] = ContextVar("hindsight_stage_holder", default=None)
-
-
-def bind_holder(holder: StageHolder):
-    """Bind a holder to the current async context.
-
-    Must be called from inside the task coroutine itself (not from the
-    spawning code) so the binding lives in the task's own contextvar scope.
-
-    Returns the token that can be passed to `_current_holder.reset()` if
-    the binding ever needs to be unwound.
-    """
-    return _current_holder.set(holder)
-
-
-def set_stage(name: str) -> None:
-    """Update the current task's stage label.
-
-    No-op when called outside a worker task context (e.g. from a sync HTTP
-    request, a test, or the CLI). Cheap enough to call per-phase.
-    """
-    holder = _current_holder.get()
-    if holder is None:
-        return
-    holder.stage = name
-    holder.updated_at = time.monotonic()
-
-
-def get_stage() -> str | None:
-    """Return the current stage label, or None if no holder is bound."""
-    holder = _current_holder.get()
-    return holder.stage if holder is not None else None

hindsight-api-slim/pyproject.toml

@@ -1,218 +0,0 @@
-[build-system]
-requires = ["hatchling"]
-build-backend = "hatchling.build"
-
-[project]
-name = "hindsight-api-slim"
-version = "0.5.1"
-description = "Hindsight: Agent Memory That Works Like Human Memory"
-readme = "README.md"
-requires-python = ">=3.11"
-dependencies = [
-    "asyncpg>=0.29.0",
-    "python-dotenv>=1.0.0",
-    "openai>=1.0.0",
-    "pydantic>=2.0.0",
-    "rich>=13.0.0",
-    "langchain-text-splitters>=0.3.0",
-    "fastapi[standard]>=0.120.3",
-    "uvicorn>=0.38.0",
-    "wsproto>=1.0.0",
-    "sqlalchemy>=2.0.44",
-    "alembic>=1.17.1",
-    "pgvector>=0.4.1",
-    "greenlet>=3.2.4,<3.4.0",  # 3.4.0 lacks arm64 wheels for manylinux_2_41
-    "psycopg2-binary>=2.9.11",
-    "tiktoken>=0.12.0",
-    "httpx>=0.27.0",
-    "PyJWT[crypto]>=2.8.0",
-    "fastmcp>=3.2.0", # SSRF/path traversal, OAuth confused deputy, command injection fixes
-    "python-dateutil>=2.8.0",
-    "opentelemetry-api>=1.20.0",
-    "opentelemetry-sdk>=1.20.0",
-    "opentelemetry-instrumentation-fastapi>=0.41b0",
-    "opentelemetry-exporter-prometheus>=0.41b0",
-    "opentelemetry-exporter-otlp-proto-http>=1.20.0",
-    "opentelemetry-semantic-conventions>=0.41b0",
-    "dateparser>=1.2.2",
-    "google-genai>=1.0.0",
-    "google-auth>=2.0.0",
-    "anthropic>=0.40.0",
-    "typer>=0.9.0",
-    "cohere>=5.0.0",
-    "litellm>=1.83.0", # 1.82.7/1.82.8 had a supply chain compromise (yanked); 1.83.0+ also fixes GHSA-jjhc-v7c2-5hh6 / GHSA-53mr-6c8q-9789
-    "markitdown[pdf,docx,pptx,xlsx,xls]>=0.1.4", # File to markdown conversion
-    "obstore>=0.4.0", # S3/GCS/Azure object storage client (Rust-backed)
-    "winloop>=0.1.0; sys_platform == 'win32'",
-    "uvloop>=0.22.1; sys_platform != 'win32'",
-    # Transitive dependency security fixes
-    "pyasn1>=0.6.3", # DoS vulnerability fix
-    "urllib3>=2.6.3", # Decompression-bomb safeguards bypass fix
-    "langchain-core>=1.2.22", # Path traversal in legacy load_prompt functions fix
-    "langsmith>=0.6.3", # SSRF via tracing header injection fix
-    "protobuf>=6.33.5", # JSON recursion depth bypass fix
-    "pillow>=12.1.1", # Out-of-bounds write in PSD image loading fix
-    "cryptography>=46.0.6", # Incomplete DNS name constraint enforcement fix
-    "filelock>=3.20.1", # TOCTOU race condition fix
-    "authlib>=1.6.9", # Account takeover/JWS header injection vulnerability fix
-    "pyjwt>=2.12.0", # Accepts unknown crit header extensions fix
-    "orjson>=3.11.6", # Unbounded recursion DoS fix
-    "python-multipart>=0.0.22", # Arbitrary file write via non-default configuration fix
-    "tornado>=6.5.5", # DoS multipart/incomplete cookie validation fix
-    "aiohttp>=3.13.3", # Multiple DoS vulnerabilities
-    "pygments>=2.20.0", # ReDoS via inefficient GUID regex fix
-    "claude-agent-sdk>=0.1.27",
-    "boto3>=1.42.74",
-]
-
-[project.optional-dependencies]
-local-ml = [
-    # Local ML models for embeddings/reranking
-    "sentence-transformers>=3.3.0",
-    "transformers>=4.53.0", # Security fixes for ReDoS vulnerabilities
-    "torch>=2.6.0", # CVE fix for remote code execution
-    "einops>=0.8.2",
-    "flashrank>=0.2.0",
-    # Apple Silicon local inference
-    "mlx>=0.31.0",
-    "mlx-lm>=0.31.1",
-    "safetensors>=0.6.2",
-]
-local-llm = [
-    # Built-in llama.cpp inference for fully offline operation
-    "llama-cpp-python[server]>=0.3.0",
-    "huggingface-hub>=0.20.0",
-]
-embedded-db = [
-    "pg0-embedded>=0.11.0",
-]
-all = [
-    "hindsight-api-slim[local-ml,embedded-db]",
-]
-test = [
-    "pytest>=7.0.0",
-    "pytest-asyncio>=0.21.0",
-    "pytest-timeout>=2.4.0",
-    "pytest-xdist>=3.0.0",
-    "filelock>=3.20.1",  # TOCTOU race condition fix
-    "testcontainers>=4.0.0",
-]
-
-[project.scripts]
-hindsight-api = "hindsight_api.main:main"
-hindsight-worker = "hindsight_api.worker.main:main"
-hindsight-local-mcp = "hindsight_api.mcp_local:main"
-hindsight-admin = "hindsight_api.admin.cli:main"
-
-[tool.hatch.build.targets.wheel]
-packages = ["hindsight_api"]
-
-[tool.hatch.build.targets.wheel.sources]
-"hindsight_api" = "hindsight_api"
-
-[tool.hatch.build.targets.sdist]
-include = [
-    "hindsight_api/**/*",
-]
-
-[tool.hatch.build]
-include = [
-    "hindsight_api/**/*.py",
-    "hindsight_api/alembic/**/*",
-]
-
-[tool.pytest.ini_options]
-log_cli = true
-log_cli_level = "INFO"
-log_cli_format = "%(asctime)s - %(levelname)s - %(name)s - %(message)s"
-log_cli_date_format = "%Y-%m-%d %H:%M:%S"
-addopts = "--timeout 300 -n 8 --dist loadgroup --durations=10 -v"
-asyncio_mode = "auto"
-asyncio_default_fixture_loop_scope = "function"
-log_auto_indent = true
-filterwarnings = [
-    "ignore:The @wait_container_is_ready decorator is deprecated:DeprecationWarning",
-    "ignore::RuntimeWarning:asyncio",
-]
-
-[dependency-groups]
-dev = [
-    "pytest>=9.0.0",
-    "pytest-asyncio>=1.3.0",
-    "pytest-timeout>=2.4.0",
-    "pytest-xdist>=3.8.0",
-    "pytest-rerunfailures>=15.0",
-    "python-dotenv>=1.2.1",
-    "filelock>=3.20.1",  # TOCTOU race condition fix
-    "ruff>=0.8.0",
-    "ty>=0.0.1",
-    "testcontainers>=4.0.0",
-]
-
-[tool.ruff]
-line-length = 120
-target-version = "py311"
-exclude = [
-    "tests/",
-    "**/tests/",
-]
-
-[tool.ruff.lint]
-select = [
-    "E",      # pycodestyle errors
-    "W",      # pycodestyle warnings
-    "F",      # Pyflakes
-    "I",      # isort
-]
-ignore = [
-    "E501",   # line too long (handled by formatter)
-    "E402",   # module import not at top of file
-    "F401",   # unused import (too noisy during development)
-    "F841",   # unused variable (too noisy during development)
-    "F811",   # redefined while unused
-    "F821",   # undefined name (forward references in type hints)
-]
-
-[tool.ruff.lint.isort]
-known-third-party = ["alembic"]
-
-[tool.ruff.format]
-quote-style = "double"
-indent-style = "space"
-
-[tool.uv]
-# Use explicit index for PyTorch to prevent the pytorch index from serving
-# non-pytorch packages (e.g. markupsafe) with incompatible wheels
-[[tool.uv.index]]
-name = "pytorch-cpu"
-url = "https://download.pytorch.org/whl/cpu"
-explicit = true
-
-[tool.uv.sources]
-# Route torch to the CPU-only PyTorch index; everything else uses PyPI
-torch = { index = "pytorch-cpu" }
-
-[tool.ty]
-# Type checking configuration
-# ty is an extremely fast Python type checker from Astral (same team as ruff/uv)
-
-[tool.ty.environment]
-python-version = "3.11"
-
-[tool.ty.src]
-exclude = [
-    "tests/",
-    "hindsight_api/alembic/",
-]
-
-[tool.ty.rules]
-# Disable noisy rules while keeping important ones
-invalid-argument-type = "ignore"    # False positives with **kwargs patterns
-invalid-return-type = "ignore"      # Often intentional in async code
-invalid-parameter-default = "ignore" # Optional params with None default
-possibly-missing-attribute = "ignore" # Common with Optional types
-invalid-raise = "ignore"            # False positives with exception tracking
-call-non-callable = "ignore"        # False positives with Optional types
-invalid-key = "ignore"              # Pydantic ConfigDict not understood
-invalid-method-override = "ignore"  # Intentional signature differences
-unresolved-reference = "ignore"     # Forward references not always resolved

hindsight-api-slim/tests/test_audit_log.py

@@ -1,449 +0,0 @@
-"""
-Tests for the audit log feature.
-
-Tests the audit log list, stats, filtering, and pagination endpoints.
-Verifies that audit entries are created for operations when audit logging is enabled.
-"""
-
-import asyncio
-
-import httpx
-import pytest
-import pytest_asyncio
-
-from hindsight_api.api import create_app
-from hindsight_api.config import get_config
-
-
-@pytest_asyncio.fixture
-async def audit_api_client(memory):
-    """Create a test client with audit logging enabled."""
-    # Enable audit logging on the memory engine's audit logger
-    memory._audit_logger._enabled = True
-    memory._audit_logger._allowed_actions = None  # All actions
-
-    app = create_app(memory, initialize_memory=False)
-    transport = httpx.ASGITransport(app=app)
-    async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
-        yield client
-
-
-@pytest.fixture
-def bank_id():
-    """Provide a unique bank ID for audit tests."""
-    from datetime import datetime
-
-    return f"audit_test_{datetime.now().timestamp()}"
-
-
-@pytest.mark.asyncio
-async def test_audit_log_list_empty(audit_api_client, bank_id):
-    """Test listing audit logs for a bank with no entries returns empty."""
-    # Create the bank first
-    await audit_api_client.put(
-        f"/v1/default/banks/{bank_id}",
-        json={"name": "Audit Test Bank"},
-    )
-
-    # Small delay for fire-and-forget audit writes
-    await asyncio.sleep(0.5)
-
-    response = await audit_api_client.get(f"/v1/default/banks/{bank_id}/audit-logs")
-    assert response.status_code == 200
-    data = response.json()
-    assert data["bank_id"] == bank_id
-    assert "total" in data
-    assert "items" in data
-    assert "limit" in data
-    assert "offset" in data
-    assert isinstance(data["items"], list)
-
-
-@pytest.mark.asyncio
-async def test_audit_log_created_for_retain(audit_api_client, bank_id):
-    """Test that a retain operation creates an audit log entry."""
-    # Create bank
-    await audit_api_client.put(
-        f"/v1/default/banks/{bank_id}",
-        json={"name": "Audit Test Bank"},
-    )
-
-    # Perform a retain
-    response = await audit_api_client.post(
-        f"/v1/default/banks/{bank_id}/memories",
-        json={
-            "items": [{"content": "Alice likes cats", "context": "preferences"}],
-        },
-    )
-    assert response.status_code == 200
-
-    # Wait for fire-and-forget audit writes
-    await asyncio.sleep(1.0)
-
-    # List audit logs - should have entries for create_bank and retain
-    response = await audit_api_client.get(f"/v1/default/banks/{bank_id}/audit-logs")
-    assert response.status_code == 200
-    data = response.json()
-    assert data["total"] >= 1
-
-    actions = [item["action"] for item in data["items"]]
-    assert "retain" in actions, f"Expected 'retain' in audit actions, got: {actions}"
-
-
-@pytest.mark.asyncio
-async def test_audit_log_entry_fields(audit_api_client, bank_id):
-    """Test that audit log entries have all expected fields."""
-    # Create bank + recall to generate entries
-    await audit_api_client.put(
-        f"/v1/default/banks/{bank_id}",
-        json={"name": "Audit Test Bank"},
-    )
-
-    await audit_api_client.post(
-        f"/v1/default/banks/{bank_id}/memories/recall",
-        json={"query": "test query"},
-    )
-
-    await asyncio.sleep(1.0)
-
-    response = await audit_api_client.get(f"/v1/default/banks/{bank_id}/audit-logs")
-    assert response.status_code == 200
-    data = response.json()
-    assert data["total"] >= 1
-
-    # Check the recall entry has all fields
-    recall_entries = [item for item in data["items"] if item["action"] == "recall"]
-    assert len(recall_entries) >= 1, f"Expected recall entry, got actions: {[i['action'] for i in data['items']]}"
-
-    entry = recall_entries[0]
-    assert entry["id"] is not None
-    assert entry["action"] == "recall"
-    assert entry["transport"] == "http"
-    assert entry["bank_id"] == bank_id
-    assert entry["started_at"] is not None
-    assert entry["ended_at"] is not None
-    # Request should contain the recall parameters
-    assert entry["request"] is not None
-    assert "query" in entry["request"]
-    # Response should contain the recall results
-    assert entry["response"] is not None
-
-
-@pytest.mark.asyncio
-async def test_audit_log_filter_by_action(audit_api_client, bank_id):
-    """Test filtering audit logs by action type."""
-    # Create bank and do retain + recall
-    await audit_api_client.put(
-        f"/v1/default/banks/{bank_id}",
-        json={"name": "Audit Test Bank"},
-    )
-    await audit_api_client.post(
-        f"/v1/default/banks/{bank_id}/memories",
-        json={"items": [{"content": "test content", "context": "test"}]},
-    )
-    await audit_api_client.post(
-        f"/v1/default/banks/{bank_id}/memories/recall",
-        json={"query": "test"},
-    )
-
-    await asyncio.sleep(1.0)
-
-    # Filter by retain only
-    response = await audit_api_client.get(
-        f"/v1/default/banks/{bank_id}/audit-logs",
-        params={"action": "retain"},
-    )
-    assert response.status_code == 200
-    data = response.json()
-    for item in data["items"]:
-        assert item["action"] == "retain"
-
-    # Filter by recall only
-    response = await audit_api_client.get(
-        f"/v1/default/banks/{bank_id}/audit-logs",
-        params={"action": "recall"},
-    )
-    assert response.status_code == 200
-    data = response.json()
-    for item in data["items"]:
-        assert item["action"] == "recall"
-
-
-@pytest.mark.asyncio
-async def test_audit_log_filter_by_transport(audit_api_client, bank_id):
-    """Test filtering audit logs by transport type."""
-    await audit_api_client.put(
-        f"/v1/default/banks/{bank_id}",
-        json={"name": "Audit Test Bank"},
-    )
-
-    await asyncio.sleep(0.5)
-
-    # Filter by http transport
-    response = await audit_api_client.get(
-        f"/v1/default/banks/{bank_id}/audit-logs",
-        params={"transport": "http"},
-    )
-    assert response.status_code == 200
-    data = response.json()
-    for item in data["items"]:
-        assert item["transport"] == "http"
-
-    # Filter by mcp transport - should be empty (no MCP calls in this test)
-    response = await audit_api_client.get(
-        f"/v1/default/banks/{bank_id}/audit-logs",
-        params={"transport": "mcp"},
-    )
-    assert response.status_code == 200
-    data = response.json()
-    assert data["total"] == 0
-
-
-@pytest.mark.asyncio
-async def test_audit_log_filter_by_date_range(audit_api_client, bank_id):
-    """Test filtering audit logs by date range."""
-    from datetime import datetime, timedelta, timezone
-
-    await audit_api_client.put(
-        f"/v1/default/banks/{bank_id}",
-        json={"name": "Audit Test Bank"},
-    )
-
-    await asyncio.sleep(0.5)
-
-    now = datetime.now(timezone.utc)
-
-    # Filter with start_date in the past - should include entries
-    past = (now - timedelta(hours=1)).isoformat()
-    response = await audit_api_client.get(
-        f"/v1/default/banks/{bank_id}/audit-logs",
-        params={"start_date": past},
-    )
-    assert response.status_code == 200
-    data = response.json()
-    assert data["total"] >= 1
-
-    # Filter with start_date in the future - should be empty
-    future = (now + timedelta(hours=1)).isoformat()
-    response = await audit_api_client.get(
-        f"/v1/default/banks/{bank_id}/audit-logs",
-        params={"start_date": future},
-    )
-    assert response.status_code == 200
-    data = response.json()
-    assert data["total"] == 0
-
-
-@pytest.mark.asyncio
-async def test_audit_log_pagination(audit_api_client, bank_id):
-    """Test audit log pagination with limit and offset."""
-    await audit_api_client.put(
-        f"/v1/default/banks/{bank_id}",
-        json={"name": "Audit Test Bank"},
-    )
-
-    # Generate multiple audit entries
-    for i in range(5):
-        await audit_api_client.post(
-            f"/v1/default/banks/{bank_id}/memories/recall",
-            json={"query": f"test query {i}"},
-        )
-
-    await asyncio.sleep(1.5)
-
-    # Get first page
-    response = await audit_api_client.get(
-        f"/v1/default/banks/{bank_id}/audit-logs",
-        params={"limit": 2, "offset": 0},
-    )
-    assert response.status_code == 200
-    page1 = response.json()
-    assert len(page1["items"]) == 2
-    assert page1["limit"] == 2
-    assert page1["offset"] == 0
-    assert page1["total"] >= 5  # At least 5 recall + 1 create_bank
-
-    # Get second page
-    response = await audit_api_client.get(
-        f"/v1/default/banks/{bank_id}/audit-logs",
-        params={"limit": 2, "offset": 2},
-    )
-    assert response.status_code == 200
-    page2 = response.json()
-    assert len(page2["items"]) == 2
-    assert page2["offset"] == 2
-
-    # Entries should be different between pages
-    page1_ids = {item["id"] for item in page1["items"]}
-    page2_ids = {item["id"] for item in page2["items"]}
-    assert page1_ids.isdisjoint(page2_ids), "Pages should not overlap"
-
-
-@pytest.mark.asyncio
-async def test_audit_log_stats(audit_api_client, bank_id):
-    """Test the audit log stats endpoint returns correct structure."""
-    await audit_api_client.put(
-        f"/v1/default/banks/{bank_id}",
-        json={"name": "Audit Test Bank"},
-    )
-
-    await audit_api_client.post(
-        f"/v1/default/banks/{bank_id}/memories/recall",
-        json={"query": "stats test"},
-    )
-
-    await asyncio.sleep(1.0)
-
-    # Get stats for last 24h
-    response = await audit_api_client.get(
-        f"/v1/default/banks/{bank_id}/audit-logs/stats",
-        params={"period": "1d"},
-    )
-    assert response.status_code == 200
-    data = response.json()
-    assert data["bank_id"] == bank_id
-    assert data["period"] == "1d"
-    assert data["trunc"] == "day"
-    assert "buckets" in data
-    assert isinstance(data["buckets"], list)
-
-    # Should have at least one bucket with our operations
-    assert len(data["buckets"]) >= 1
-    bucket = data["buckets"][0]
-    assert "time" in bucket
-    assert "actions" in bucket
-    assert "total" in bucket
-    assert bucket["total"] >= 1
-
-
-@pytest.mark.asyncio
-async def test_audit_log_stats_filter_by_action(audit_api_client, bank_id):
-    """Test stats endpoint filters by action."""
-    await audit_api_client.put(
-        f"/v1/default/banks/{bank_id}",
-        json={"name": "Audit Test Bank"},
-    )
-
-    await audit_api_client.post(
-        f"/v1/default/banks/{bank_id}/memories/recall",
-        json={"query": "test"},
-    )
-
-    await asyncio.sleep(1.0)
-
-    # Stats filtered by recall
-    response = await audit_api_client.get(
-        f"/v1/default/banks/{bank_id}/audit-logs/stats",
-        params={"period": "1d", "action": "recall"},
-    )
-    assert response.status_code == 200
-    data = response.json()
-    for bucket in data["buckets"]:
-        # All actions in buckets should be "recall" only
-        for action_name in bucket["actions"]:
-            assert action_name == "recall"
-
-
-@pytest.mark.asyncio
-async def test_audit_log_stats_periods(audit_api_client, bank_id):
-    """Test stats endpoint supports different periods."""
-    await audit_api_client.put(
-        f"/v1/default/banks/{bank_id}",
-        json={"name": "Audit Test Bank"},
-    )
-
-    await asyncio.sleep(0.5)
-
-    for period, expected_trunc in [("1d", "day"), ("7d", "day"), ("30d", "day")]:
-        response = await audit_api_client.get(
-            f"/v1/default/banks/{bank_id}/audit-logs/stats",
-            params={"period": period},
-        )
-        assert response.status_code == 200
-        data = response.json()
-        assert data["period"] == period
-        assert data["trunc"] == expected_trunc
-
-
-@pytest.mark.asyncio
-async def test_audit_log_disabled(memory):
-    """Test that no audit logs are created when audit logging is disabled."""
-    # Ensure audit logging is disabled
-    memory._audit_logger._enabled = False
-
-    app = create_app(memory, initialize_memory=False)
-    transport = httpx.ASGITransport(app=app)
-    async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
-        from datetime import datetime
-
-        bid = f"audit_disabled_test_{datetime.now().timestamp()}"
-
-        await client.put(f"/v1/default/banks/{bid}", json={"name": "No Audit"})
-        await client.post(
-            f"/v1/default/banks/{bid}/memories/recall",
-            json={"query": "test"},
-        )
-
-        await asyncio.sleep(0.5)
-
-        response = await client.get(f"/v1/default/banks/{bid}/audit-logs")
-        assert response.status_code == 200
-        data = response.json()
-        assert data["total"] == 0, "No audit entries should exist when audit logging is disabled"
-
-
-@pytest.mark.asyncio
-async def test_audit_log_action_allowlist(memory):
-    """Test that only allowed actions are audited when allowlist is set."""
-    memory._audit_logger._enabled = True
-    memory._audit_logger._allowed_actions = frozenset({"recall"})  # Only audit recall
-
-    app = create_app(memory, initialize_memory=False)
-    transport = httpx.ASGITransport(app=app)
-    async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
-        from datetime import datetime
-
-        bid = f"audit_allowlist_test_{datetime.now().timestamp()}"
-
-        # create_bank should NOT be audited
-        await client.put(f"/v1/default/banks/{bid}", json={"name": "Allowlist Test"})
-        # recall should be audited
-        await client.post(
-            f"/v1/default/banks/{bid}/memories/recall",
-            json={"query": "allowlist test"},
-        )
-
-        await asyncio.sleep(1.0)
-
-        response = await client.get(f"/v1/default/banks/{bid}/audit-logs")
-        assert response.status_code == 200
-        data = response.json()
-        actions = [item["action"] for item in data["items"]]
-        assert "recall" in actions, "recall should be audited"
-        assert "create_bank" not in actions, "create_bank should NOT be audited (not in allowlist)"
-
-
-@pytest.mark.asyncio
-async def test_audit_log_ordered_by_most_recent(audit_api_client, bank_id):
-    """Test that audit logs are returned ordered by most recent first."""
-    await audit_api_client.put(
-        f"/v1/default/banks/{bank_id}",
-        json={"name": "Order Test Bank"},
-    )
-
-    for i in range(3):
-        await audit_api_client.post(
-            f"/v1/default/banks/{bank_id}/memories/recall",
-            json={"query": f"order test {i}"},
-        )
-        await asyncio.sleep(0.2)  # Small gap between requests
-
-    await asyncio.sleep(1.0)
-
-    response = await audit_api_client.get(f"/v1/default/banks/{bank_id}/audit-logs")
-    assert response.status_code == 200
-    data = response.json()
-
-    # Check descending order by started_at
-    timestamps = [item["started_at"] for item in data["items"] if item["started_at"]]
-    assert timestamps == sorted(timestamps, reverse=True), "Audit logs should be ordered most recent first"

hindsight-api-slim/tests/test_bank_templates.py

@@ -1,779 +0,0 @@
-"""Integration tests for bank template import/export endpoints."""
-
-import pytest
-import pytest_asyncio
-import httpx
-from datetime import datetime
-from hindsight_api.api import create_app
-
-
-@pytest_asyncio.fixture
-async def api_client(memory):
-    """Create an async test client for the FastAPI app."""
-    app = create_app(memory, initialize_memory=False)
-    transport = httpx.ASGITransport(app=app)
-    async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
-        yield client
-
-
-@pytest.fixture
-def bank_id():
-    return f"template_test_{datetime.now().timestamp()}"
-
-
-@pytest.fixture
-def sample_template():
-    return {
-        "version": "1",
-        "bank": {
-            "reflect_mission": "Test mission for reflect",
-            "retain_mission": "Extract test data carefully",
-            "retain_extraction_mode": "verbose",
-            "disposition_empathy": 5,
-            "disposition_skepticism": 2,
-            "enable_observations": True,
-            "observations_mission": "Track test patterns",
-        },
-        "mental_models": [
-            {
-                "id": "test-model-one",
-                "name": "Test Model One",
-                "source_query": "What are the key patterns?",
-                "tags": ["test"],
-                "max_tokens": 1024,
-                "trigger": {"refresh_after_consolidation": True},
-            },
-            {
-                "id": "test-model-two",
-                "name": "Test Model Two",
-                "source_query": "What are the common issues?",
-            },
-        ],
-        "directives": [
-            {
-                "name": "Be concise",
-                "content": "Always respond concisely.",
-                "priority": 10,
-            },
-            {
-                "name": "Use examples",
-                "content": "Include examples when explaining concepts.",
-                "tags": ["style"],
-            },
-        ],
-    }
-
-
-class TestImportValidation:
-    """Test template manifest validation."""
-
-    @pytest.mark.asyncio
-    async def test_import_dry_run_valid(self, api_client, bank_id, sample_template):
-        """dry_run=true with a valid manifest returns what would happen."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import?dry_run=true",
-            json=sample_template,
-        )
-        assert resp.status_code == 200
-        data = resp.json()
-        assert data["dry_run"] is True
-        assert data["config_applied"] is True
-        assert set(data["mental_models_created"]) == {"test-model-one", "test-model-two"}
-        assert set(data["directives_created"]) == {"Be concise", "Use examples"}
-
-    @pytest.mark.asyncio
-    async def test_import_invalid_version(self, api_client, bank_id):
-        """Reject manifest with unsupported version."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={"version": "999"},
-        )
-        assert resp.status_code == 400
-
-    @pytest.mark.asyncio
-    async def test_import_invalid_extraction_mode(self, api_client, bank_id):
-        """Semantic validation catches bad extraction mode."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "bank": {"retain_extraction_mode": "invalid_mode"},
-            },
-        )
-        assert resp.status_code == 400
-        assert "retain_extraction_mode" in resp.json()["detail"]
-
-    @pytest.mark.asyncio
-    async def test_import_custom_instructions_without_custom_mode(self, api_client, bank_id):
-        """Validate that custom_instructions requires extraction_mode=custom."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "bank": {
-                    "retain_extraction_mode": "verbose",
-                    "retain_custom_instructions": "some custom prompt",
-                },
-            },
-        )
-        assert resp.status_code == 400
-        assert "retain_custom_instructions" in resp.json()["detail"]
-
-    @pytest.mark.asyncio
-    async def test_import_duplicate_mental_model_ids(self, api_client, bank_id):
-        """Reject manifest with duplicate mental model IDs."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "mental_models": [
-                    {"id": "dup-id", "name": "First", "source_query": "q1"},
-                    {"id": "dup-id", "name": "Second", "source_query": "q2"},
-                ],
-            },
-        )
-        assert resp.status_code == 400
-
-    @pytest.mark.asyncio
-    async def test_import_duplicate_directive_names(self, api_client, bank_id):
-        """Reject manifest with duplicate directive names."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "directives": [
-                    {"name": "Same Name", "content": "First"},
-                    {"name": "Same Name", "content": "Second"},
-                ],
-            },
-        )
-        assert resp.status_code == 400
-
-    @pytest.mark.asyncio
-    async def test_import_missing_mental_model_id(self, api_client, bank_id):
-        """Mental model without id is rejected."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "mental_models": [
-                    {"name": "No ID Model", "source_query": "test query"},
-                ],
-            },
-        )
-        assert resp.status_code == 400
-
-    @pytest.mark.asyncio
-    async def test_import_invalid_mental_model_id_format(self, api_client, bank_id):
-        """Mental model with invalid ID format is rejected."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "mental_models": [
-                    {"id": "UPPERCASE-NOT-ALLOWED", "name": "Bad", "source_query": "q"},
-                ],
-            },
-        )
-        assert resp.status_code == 400
-
-    @pytest.mark.asyncio
-    async def test_import_empty_manifest(self, api_client, bank_id):
-        """Import with no bank or mental_models is valid (no-op)."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={"version": "1"},
-        )
-        assert resp.status_code == 200
-        data = resp.json()
-        assert data["config_applied"] is False
-        assert data["mental_models_created"] == []
-        assert data["directives_created"] == []
-
-    @pytest.mark.asyncio
-    async def test_import_empty_mental_model_name(self, api_client, bank_id):
-        """Semantic validation catches empty mental model name."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "mental_models": [
-                    {"id": "test-mm", "name": "  ", "source_query": "q"},
-                ],
-            },
-        )
-        assert resp.status_code == 400
-        assert "name" in resp.json()["detail"]
-
-    @pytest.mark.asyncio
-    async def test_import_empty_directive_content(self, api_client, bank_id):
-        """Semantic validation catches empty directive content."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "directives": [
-                    {"name": "Bad Directive", "content": "  "},
-                ],
-            },
-        )
-        assert resp.status_code == 400
-        assert "content" in resp.json()["detail"]
-
-
-class TestImportApply:
-    """Test that import actually applies config, mental models, and directives."""
-
-    @pytest.mark.asyncio
-    async def test_import_applies_config(self, api_client, bank_id):
-        """Import with bank config applies config overrides on a new bank."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "bank": {
-                    "reflect_mission": "Imported mission",
-                    "disposition_empathy": 4,
-                },
-            },
-        )
-        assert resp.status_code == 200
-        data = resp.json()
-        assert data["config_applied"] is True
-        assert data["dry_run"] is False
-
-        # Verify config was actually applied
-        config_resp = await api_client.get(f"/v1/default/banks/{bank_id}/config")
-        assert config_resp.status_code == 200
-        config = config_resp.json()
-        assert config["overrides"]["reflect_mission"] == "Imported mission"
-        assert config["overrides"]["disposition_empathy"] == 4
-
-    @pytest.mark.asyncio
-    async def test_import_into_existing_bank(self, api_client, bank_id):
-        """Import into an already-existing bank applies config and creates resources."""
-        # Pre-create the bank
-        await api_client.put(f"/v1/default/banks/{bank_id}", json={})
-
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "bank": {"reflect_mission": "Existing bank mission"},
-                "mental_models": [
-                    {"id": "existing-bank-mm", "name": "MM", "source_query": "q"},
-                ],
-                "directives": [
-                    {"name": "Existing Bank Directive", "content": "Be helpful"},
-                ],
-            },
-        )
-        assert resp.status_code == 200
-        data = resp.json()
-        assert data["config_applied"] is True
-        assert "existing-bank-mm" in data["mental_models_created"]
-        assert "Existing Bank Directive" in data["directives_created"]
-
-        # Verify everything exists
-        config_resp = await api_client.get(f"/v1/default/banks/{bank_id}/config")
-        assert config_resp.json()["overrides"]["reflect_mission"] == "Existing bank mission"
-
-        mm_resp = await api_client.get(f"/v1/default/banks/{bank_id}/mental-models/existing-bank-mm")
-        assert mm_resp.status_code == 200
-
-        dir_resp = await api_client.get(f"/v1/default/banks/{bank_id}/directives")
-        assert dir_resp.status_code == 200
-        names = [d["name"] for d in dir_resp.json()["items"]]
-        assert "Existing Bank Directive" in names
-
-    @pytest.mark.asyncio
-    async def test_import_creates_mental_models(self, api_client, bank_id):
-        """Import creates mental models and returns operation IDs."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "mental_models": [
-                    {
-                        "id": "import-mm-1",
-                        "name": "Imported Model",
-                        "source_query": "What patterns exist?",
-                        "tags": ["imported"],
-                    },
-                ],
-            },
-        )
-        assert resp.status_code == 200
-        data = resp.json()
-        assert "import-mm-1" in data["mental_models_created"]
-        assert len(data["operation_ids"]) == 1
-
-        # Verify mental model exists
-        mm_resp = await api_client.get(f"/v1/default/banks/{bank_id}/mental-models/import-mm-1")
-        assert mm_resp.status_code == 200
-        mm = mm_resp.json()
-        assert mm["name"] == "Imported Model"
-        assert mm["source_query"] == "What patterns exist?"
-        assert mm["tags"] == ["imported"]
-
-    @pytest.mark.asyncio
-    async def test_import_updates_existing_mental_models(self, api_client, bank_id):
-        """Re-importing updates existing mental models matched by ID."""
-        # First import
-        await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "mental_models": [
-                    {
-                        "id": "reusable-mm",
-                        "name": "Original Name",
-                        "source_query": "Original query",
-                    },
-                ],
-            },
-        )
-
-        # Second import with same ID but different content
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "mental_models": [
-                    {
-                        "id": "reusable-mm",
-                        "name": "Updated Name",
-                        "source_query": "Updated query",
-                    },
-                ],
-            },
-        )
-        assert resp.status_code == 200
-        data = resp.json()
-        assert "reusable-mm" in data["mental_models_updated"]
-        assert data["mental_models_created"] == []
-
-        # Verify update
-        mm_resp = await api_client.get(f"/v1/default/banks/{bank_id}/mental-models/reusable-mm")
-        assert mm_resp.status_code == 200
-        mm = mm_resp.json()
-        assert mm["name"] == "Updated Name"
-        assert mm["source_query"] == "Updated query"
-
-    @pytest.mark.asyncio
-    async def test_import_creates_directives(self, api_client, bank_id):
-        """Import creates directives."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "directives": [
-                    {
-                        "name": "Test Directive",
-                        "content": "Always be helpful and precise.",
-                        "priority": 5,
-                        "tags": ["test"],
-                    },
-                ],
-            },
-        )
-        assert resp.status_code == 200
-        data = resp.json()
-        assert "Test Directive" in data["directives_created"]
-        assert data["directives_updated"] == []
-
-        # Verify directive exists
-        dir_resp = await api_client.get(f"/v1/default/banks/{bank_id}/directives")
-        assert dir_resp.status_code == 200
-        items = dir_resp.json()["items"]
-        assert len(items) == 1
-        assert items[0]["name"] == "Test Directive"
-        assert items[0]["content"] == "Always be helpful and precise."
-        assert items[0]["priority"] == 5
-        assert items[0]["tags"] == ["test"]
-
-    @pytest.mark.asyncio
-    async def test_import_updates_existing_directives(self, api_client, bank_id):
-        """Re-importing updates existing directives matched by name."""
-        # First import
-        await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "directives": [
-                    {"name": "Reusable Directive", "content": "Original content", "priority": 1},
-                ],
-            },
-        )
-
-        # Second import with same name but different content
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "directives": [
-                    {"name": "Reusable Directive", "content": "Updated content", "priority": 10},
-                ],
-            },
-        )
-        assert resp.status_code == 200
-        data = resp.json()
-        assert "Reusable Directive" in data["directives_updated"]
-        assert data["directives_created"] == []
-
-        # Verify update
-        dir_resp = await api_client.get(f"/v1/default/banks/{bank_id}/directives")
-        items = dir_resp.json()["items"]
-        directive = [d for d in items if d["name"] == "Reusable Directive"][0]
-        assert directive["content"] == "Updated content"
-        assert directive["priority"] == 10
-
-    @pytest.mark.asyncio
-    async def test_import_config_only(self, api_client, bank_id):
-        """Import with only bank config (no mental_models or directives) works."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "bank": {"retain_extraction_mode": "verbose"},
-            },
-        )
-        assert resp.status_code == 200
-        data = resp.json()
-        assert data["config_applied"] is True
-        assert data["mental_models_created"] == []
-        assert data["directives_created"] == []
-        assert data["operation_ids"] == []
-
-    @pytest.mark.asyncio
-    async def test_import_mental_models_only(self, api_client, bank_id):
-        """Import with only mental_models works."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "mental_models": [
-                    {"id": "mm-only", "name": "MM Only", "source_query": "test"},
-                ],
-            },
-        )
-        assert resp.status_code == 200
-        data = resp.json()
-        assert data["config_applied"] is False
-        assert "mm-only" in data["mental_models_created"]
-        assert data["directives_created"] == []
-
-    @pytest.mark.asyncio
-    async def test_import_directives_only(self, api_client, bank_id):
-        """Import with only directives works."""
-        resp = await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "directives": [
-                    {"name": "Dir Only", "content": "test directive"},
-                ],
-            },
-        )
-        assert resp.status_code == 200
-        data = resp.json()
-        assert data["config_applied"] is False
-        assert data["mental_models_created"] == []
-        assert "Dir Only" in data["directives_created"]
-
-
-class TestExport:
-    """Test bank template export."""
-
-    @pytest.mark.asyncio
-    async def test_export_empty_bank(self, api_client, bank_id):
-        """Export a bank with no overrides returns minimal manifest."""
-        # Create bank
-        await api_client.put(f"/v1/default/banks/{bank_id}", json={})
-
-        resp = await api_client.get(f"/v1/default/banks/{bank_id}/export")
-        assert resp.status_code == 200
-        data = resp.json()
-        assert data["version"] == "1"
-        assert data["bank"] is None
-        assert data["mental_models"] is None
-        assert data["directives"] is None
-
-    @pytest.mark.asyncio
-    async def test_export_after_import(self, api_client, bank_id):
-        """Export after import returns the imported config, mental models, and directives."""
-        template = {
-            "version": "1",
-            "bank": {
-                "reflect_mission": "Roundtrip mission",
-                "disposition_empathy": 3,
-            },
-            "mental_models": [
-                {
-                    "id": "roundtrip-mm",
-                    "name": "Roundtrip Model",
-                    "source_query": "What happened?",
-                    "tags": ["roundtrip"],
-                    "max_tokens": 512,
-                },
-            ],
-            "directives": [
-                {
-                    "name": "Roundtrip Directive",
-                    "content": "Be thorough.",
-                    "priority": 3,
-                    "tags": ["roundtrip"],
-                },
-            ],
-        }
-
-        # Import
-        import_resp = await api_client.post(f"/v1/default/banks/{bank_id}/import", json=template)
-        assert import_resp.status_code == 200
-
-        # Export
-        resp = await api_client.get(f"/v1/default/banks/{bank_id}/export")
-        assert resp.status_code == 200
-        data = resp.json()
-
-        assert data["version"] == "1"
-        assert data["bank"]["reflect_mission"] == "Roundtrip mission"
-        assert data["bank"]["disposition_empathy"] == 3
-
-        assert len(data["mental_models"]) == 1
-        mm = data["mental_models"][0]
-        assert mm["id"] == "roundtrip-mm"
-        assert mm["name"] == "Roundtrip Model"
-        assert mm["source_query"] == "What happened?"
-        assert mm["tags"] == ["roundtrip"]
-        assert mm["max_tokens"] == 512
-
-        assert len(data["directives"]) == 1
-        d = data["directives"][0]
-        assert d["name"] == "Roundtrip Directive"
-        assert d["content"] == "Be thorough."
-        assert d["priority"] == 3
-        assert d["tags"] == ["roundtrip"]
-
-    @pytest.mark.asyncio
-    async def test_export_reimport_roundtrip(self, api_client, bank_id):
-        """Exported manifest can be re-imported into a new bank."""
-        # Set up source bank
-        await api_client.post(
-            f"/v1/default/banks/{bank_id}/import",
-            json={
-                "version": "1",
-                "bank": {"retain_mission": "Roundtrip test"},
-                "mental_models": [
-                    {"id": "rt-mm", "name": "RT Model", "source_query": "test query"},
-                ],
-                "directives": [
-                    {"name": "RT Directive", "content": "test directive"},
-                ],
-            },
-        )
-
-        # Export
-        export_resp = await api_client.get(f"/v1/default/banks/{bank_id}/export")
-        assert export_resp.status_code == 200
-        exported = export_resp.json()
-
-        # Import into a new bank
-        new_bank_id = f"{bank_id}_clone"
-        import_resp = await api_client.post(
-            f"/v1/default/banks/{new_bank_id}/import",
-            json=exported,
-        )
-        assert import_resp.status_code == 200
-        data = import_resp.json()
-        assert data["config_applied"] is True
-        assert "rt-mm" in data["mental_models_created"]
-        assert "RT Directive" in data["directives_created"]
-
-    @pytest.mark.asyncio
-    async def test_export_nonexistent_bank(self, api_client):
-        """Export from a nonexistent bank returns the bank with defaults (auto-created)."""
-        resp = await api_client.get("/v1/default/banks/nonexistent-export-test/export")
-        # get_bank_profile auto-creates, so this returns a valid empty manifest
-        assert resp.status_code == 200
-        data = resp.json()
-        assert data["version"] == "1"
-
-
-class TestDefaultBankTemplateEnvVar:
-    """Tests for HINDSIGHT_API_DEFAULT_BANK_TEMPLATE — a server-level env var
-    whose manifest is applied automatically to every newly-created bank."""
-
-    @pytest.fixture
-    def default_template(self):
-        return {
-            "version": "1",
-            "bank": {
-                "reflect_mission": "default-env-mission",
-                "retain_extraction_mode": "verbose",
-                "disposition_empathy": 5,
-                "disposition_skepticism": 1,
-            },
-            "mental_models": [
-                {
-                    "id": "default-env-model",
-                    "name": "Default Env Model",
-                    "source_query": "What is the default?",
-                },
-            ],
-            "directives": [
-                {
-                    "name": "Default Env Directive",
-                    "content": "Follow the default behavior.",
-                    "priority": 7,
-                },
-            ],
-        }
-
-    @pytest.fixture
-    def _patched_default_template(self, monkeypatch, default_template):
-        """Install the default template on the already-initialized global config.
-
-        We can't rely on env-var resolution here: MemoryEngine (and its
-        ConfigResolver) snapshot the global config at fixture init time.
-        Patching the field directly keeps the test deterministic while still
-        exercising the same code path that reads `get_config().default_bank_template`.
-        """
-        from hindsight_api.config import _get_raw_config
-
-        raw = _get_raw_config()
-        monkeypatch.setattr(raw, "default_bank_template", default_template)
-        yield default_template
-
-    @pytest.mark.asyncio
-    async def test_default_template_applied_on_new_bank(
-        self, api_client, bank_id, _patched_default_template
-    ):
-        """Creating a new bank applies the default template (config + mental models + directives)."""
-        # Trigger bank auto-creation via GET profile
-        resp = await api_client.put(f"/v1/default/banks/{bank_id}", json={})
-        assert resp.status_code == 200
-
-        # Config from template should be present as bank overrides
-        config_resp = await api_client.get(f"/v1/default/banks/{bank_id}/config")
-        assert config_resp.status_code == 200
-        overrides = config_resp.json()["overrides"]
-        assert overrides["reflect_mission"] == "default-env-mission"
-        assert overrides["retain_extraction_mode"] == "verbose"
-        assert overrides["disposition_empathy"] == 5
-        assert overrides["disposition_skepticism"] == 1
-
-        # Mental model from template should exist
-        mm_resp = await api_client.get(f"/v1/default/banks/{bank_id}/mental-models/default-env-model")
-        assert mm_resp.status_code == 200
-        assert mm_resp.json()["name"] == "Default Env Model"
-
-        # Directive from template should exist
-        dir_resp = await api_client.get(f"/v1/default/banks/{bank_id}/directives")
-        assert dir_resp.status_code == 200
-        names = [d["name"] for d in dir_resp.json()["items"]]
-        assert "Default Env Directive" in names
-
-    @pytest.mark.asyncio
-    async def test_default_template_overrides_env_config_defaults(
-        self, api_client, bank_id, monkeypatch, default_template
-    ):
-        """Fields set by the default template override server-level env-var defaults.
-
-        We point both HINDSIGHT_API_RETAIN_EXTRACTION_MODE (env) and the
-        default template at different values, then confirm the template wins
-        via the per-bank config overrides layer (highest precedence).
-        """
-        from hindsight_api.config import _get_raw_config
-
-        raw = _get_raw_config()
-        # Simulate an env-level default of "concise", overridden by a template that sets "verbose".
-        monkeypatch.setattr(raw, "retain_extraction_mode", "concise")
-        monkeypatch.setattr(raw, "default_bank_template", default_template)
-
-        resp = await api_client.put(f"/v1/default/banks/{bank_id}", json={})
-        assert resp.status_code == 200
-
-        config_resp = await api_client.get(f"/v1/default/banks/{bank_id}/config")
-        overrides = config_resp.json()["overrides"]
-        # Template value wins at the bank-override layer.
-        assert overrides["retain_extraction_mode"] == "verbose"
-
-    @pytest.mark.asyncio
-    async def test_default_template_not_reapplied_on_existing_bank(
-        self, api_client, bank_id, _patched_default_template
-    ):
-        """Template only applies on FIRST creation; subsequent puts are no-ops."""
-        # First hit creates the bank and applies the template
-        resp = await api_client.put(f"/v1/default/banks/{bank_id}", json={})
-        assert resp.status_code == 200
-
-        # User explicitly overrides a template-set field
-        patch_resp = await api_client.patch(
-            f"/v1/default/banks/{bank_id}/config",
-            json={"updates": {"reflect_mission": "user-override"}},
-        )
-        assert patch_resp.status_code == 200
-
-        # Second put — template must NOT be reapplied (would clobber the override)
-        resp = await api_client.put(f"/v1/default/banks/{bank_id}", json={})
-        assert resp.status_code == 200
-
-        config_resp = await api_client.get(f"/v1/default/banks/{bank_id}/config")
-        assert config_resp.json()["overrides"]["reflect_mission"] == "user-override"
-
-    @pytest.mark.asyncio
-    async def test_default_template_unset_is_noop(self, api_client, bank_id):
-        """With the env var unset (fixture default), bank creation behaves as before."""
-        resp = await api_client.put(f"/v1/default/banks/{bank_id}", json={})
-        assert resp.status_code == 200
-
-        # No template = no overrides
-        config_resp = await api_client.get(f"/v1/default/banks/{bank_id}/config")
-        assert config_resp.json()["overrides"] == {}
-
-    @pytest.mark.asyncio
-    async def test_default_template_malformed_is_swallowed(
-        self, api_client, bank_id, monkeypatch
-    ):
-        """A malformed default template is logged and ignored — bank creation still succeeds."""
-        from hindsight_api.config import _get_raw_config
-
-        raw = _get_raw_config()
-        # Wrong version number fails Pydantic validation.
-        monkeypatch.setattr(raw, "default_bank_template", {"version": "999"})
-
-        resp = await api_client.put(f"/v1/default/banks/{bank_id}", json={})
-        # Bank creation must not fail even though the template is broken.
-        assert resp.status_code == 200
-
-    def test_parse_default_bank_template_valid_json(self, monkeypatch):
-        """_parse_default_bank_template parses a valid JSON object env var."""
-        from hindsight_api.config import _parse_default_bank_template
-
-        parsed = _parse_default_bank_template('{"version": "1", "bank": {"disposition_empathy": 4}}')
-        assert parsed == {"version": "1", "bank": {"disposition_empathy": 4}}
-
-    def test_parse_default_bank_template_none_or_empty(self):
-        """Unset / empty env var resolves to None."""
-        from hindsight_api.config import _parse_default_bank_template
-
-        assert _parse_default_bank_template(None) is None
-        assert _parse_default_bank_template("") is None
-        assert _parse_default_bank_template("   ") is None
-
-    def test_parse_default_bank_template_invalid_json_raises(self):
-        """Invalid JSON fails fast with a clear error."""
-        from hindsight_api.config import _parse_default_bank_template
-
-        with pytest.raises(ValueError, match="HINDSIGHT_API_DEFAULT_BANK_TEMPLATE"):
-            _parse_default_bank_template("not-json")
-
-    def test_parse_default_bank_template_non_object_raises(self):
-        """Non-object JSON (e.g. array, string) fails fast."""
-        from hindsight_api.config import _parse_default_bank_template
-
-        with pytest.raises(ValueError, match="expected a JSON object"):
-            _parse_default_bank_template("[1, 2, 3]")
-        with pytest.raises(ValueError, match="expected a JSON object"):
-            _parse_default_bank_template('"just a string"')

hindsight-api-slim/tests/test_chunk_storage_upsert.py

@@ -1,149 +0,0 @@
-"""
-Regression tests for chunk_storage.store_chunks_batch idempotency.
-
-Covers vectorize-io/hindsight#977: re-submitting a retain under the same
-document_id must not fail with ``UniqueViolationError`` on ``pk_chunks``.
-The upstream retain paths (cascade delete on first batch, delta retain)
-should usually prevent a chunk_id collision, but any bug in those paths
-used to surface as a raw Postgres constraint violation. ``store_chunks_batch``
-is now idempotent: inserting the same ``chunk_id`` twice overwrites the
-existing row rather than raising.
-"""
-
-from datetime import datetime, timezone
-
-import pytest
-
-from hindsight_api.engine.retain import chunk_storage
-from hindsight_api.engine.retain.types import ChunkMetadata
-
-
-def _ts() -> float:
-    return datetime.now(timezone.utc).timestamp()
-
-
-async def _seed_bank_and_document(conn, bank_id: str, document_id: str) -> None:
-    """Insert the minimum rows required for the chunks FK to pass."""
-    await conn.execute(
-        "INSERT INTO banks (bank_id, name) VALUES ($1, $2) ON CONFLICT DO NOTHING",
-        bank_id,
-        bank_id,
-    )
-    await conn.execute(
-        """
-        INSERT INTO documents (id, bank_id, original_text, content_hash)
-        VALUES ($1, $2, $3, $4)
-        ON CONFLICT (id, bank_id) DO NOTHING
-        """,
-        document_id,
-        bank_id,
-        "seed",
-        "seed-hash",
-    )
-
-
-@pytest.mark.asyncio
-async def test_store_chunks_batch_is_idempotent_for_same_chunk_id(memory):
-    """
-    Regression for #977.
-
-    Directly exercises the chunk insert path: inserting a ChunkMetadata with
-    a chunk_index that already exists (i.e., the same chunk_id) must not
-    raise. The new content should overwrite the old one.
-    """
-    bank_id = f"test_chunk_upsert_{_ts()}"
-    document_id = "doc-upsert-regression"
-
-    pool = await memory._get_pool()
-    try:
-        async with pool.acquire() as conn:
-            await _seed_bank_and_document(conn, bank_id, document_id)
-
-            # First insert — fresh chunks at indices 0, 1, 2.
-            v1 = [
-                ChunkMetadata(chunk_text="alpha", fact_count=1, content_index=0, chunk_index=0),
-                ChunkMetadata(chunk_text="beta", fact_count=1, content_index=0, chunk_index=1),
-                ChunkMetadata(chunk_text="gamma", fact_count=1, content_index=0, chunk_index=2),
-            ]
-            v1_map = await chunk_storage.store_chunks_batch(conn, bank_id, document_id, v1)
-            assert set(v1_map.keys()) == {0, 1, 2}
-
-            # Second insert — overlapping chunk_index (1 and 2) with new text,
-            # plus a fresh chunk at index 3. Before the fix this raised
-            # asyncpg.exceptions.UniqueViolationError on pk_chunks; after the
-            # fix the conflicting rows are overwritten and the new one is
-            # inserted.
-            v2 = [
-                ChunkMetadata(chunk_text="beta-updated", fact_count=1, content_index=0, chunk_index=1),
-                ChunkMetadata(chunk_text="gamma-updated", fact_count=1, content_index=0, chunk_index=2),
-                ChunkMetadata(chunk_text="delta", fact_count=1, content_index=0, chunk_index=3),
-            ]
-            v2_map = await chunk_storage.store_chunks_batch(conn, bank_id, document_id, v2)
-            assert set(v2_map.keys()) == {1, 2, 3}
-
-            # Verify the stored state matches the upserted content.
-            rows = await conn.fetch(
-                """
-                SELECT chunk_index, chunk_text, content_hash
-                FROM chunks
-                WHERE document_id = $1 AND bank_id = $2
-                ORDER BY chunk_index
-                """,
-                document_id,
-                bank_id,
-            )
-            by_index = {row["chunk_index"]: row for row in rows}
-
-            assert set(by_index.keys()) == {0, 1, 2, 3}, (
-                "Expected four chunks total after upsert (0 untouched, 1-2 overwritten, 3 new)"
-            )
-            assert by_index[0]["chunk_text"] == "alpha", "Untouched chunk must be preserved"
-            assert by_index[1]["chunk_text"] == "beta-updated", "Conflicting chunk must be overwritten"
-            assert by_index[2]["chunk_text"] == "gamma-updated", "Conflicting chunk must be overwritten"
-            assert by_index[3]["chunk_text"] == "delta", "New chunk must be inserted"
-
-            # content_hash should reflect the new text, not the original.
-            assert by_index[1]["content_hash"] == chunk_storage.compute_chunk_hash("beta-updated")
-            assert by_index[2]["content_hash"] == chunk_storage.compute_chunk_hash("gamma-updated")
-    finally:
-        async with pool.acquire() as conn:
-            await conn.execute("DELETE FROM chunks WHERE bank_id = $1", bank_id)
-            await conn.execute("DELETE FROM documents WHERE bank_id = $1", bank_id)
-            await conn.execute("DELETE FROM banks WHERE bank_id = $1", bank_id)
-
-
-@pytest.mark.asyncio
-async def test_store_chunks_batch_second_call_with_identical_payload(memory):
-    """
-    The exact #977 shape: ``store_chunks_batch`` called twice with the same
-    chunks must succeed both times (the second call is a no-op in terms of
-    stored content, but must not raise).
-    """
-    bank_id = f"test_chunk_upsert_identical_{_ts()}"
-    document_id = "doc-upsert-identical"
-
-    pool = await memory._get_pool()
-    try:
-        async with pool.acquire() as conn:
-            await _seed_bank_and_document(conn, bank_id, document_id)
-
-            chunks = [
-                ChunkMetadata(chunk_text=f"chunk-{i}", fact_count=1, content_index=0, chunk_index=i)
-                for i in range(5)
-            ]
-
-            await chunk_storage.store_chunks_batch(conn, bank_id, document_id, chunks)
-            # Second call with identical chunks — must not raise.
-            await chunk_storage.store_chunks_batch(conn, bank_id, document_id, chunks)
-
-            count = await conn.fetchval(
-                "SELECT COUNT(*) FROM chunks WHERE document_id = $1 AND bank_id = $2",
-                document_id,
-                bank_id,
-            )
-            assert count == 5, "Second identical insert should not duplicate rows"
-    finally:
-        async with pool.acquire() as conn:
-            await conn.execute("DELETE FROM chunks WHERE bank_id = $1", bank_id)
-            await conn.execute("DELETE FROM documents WHERE bank_id = $1", bank_id)
-            await conn.execute("DELETE FROM banks WHERE bank_id = $1", bank_id)

hindsight-api-slim/tests/test_codex_tool_choice.py

@@ -1,88 +0,0 @@
-"""
-Regression tests for Codex provider tool_choice normalization.
-
-The reflect agent forces tool selection via named tool_choice dicts on early iterations:
-  {"type": "function", "function": {"name": "recall"}}
-
-The Codex Responses API expects the function name at the top level instead:
-  {"type": "function", "name": "recall"}
-
-Without normalization, Codex rejects the request with:
-  400 Unknown parameter: 'tool_choice.function'
-"""
-
-from unittest.mock import AsyncMock, MagicMock, patch
-
-import pytest
-
-from hindsight_api.engine.providers.codex_llm import CodexLLM
-
-TOOLS = [
-    {
-        "type": "function",
-        "function": {
-            "name": "recall",
-            "description": "Recall semantic memories",
-            "parameters": {
-                "type": "object",
-                "properties": {"query": {"type": "string"}},
-                "required": ["query"],
-            },
-        },
-    }
-]
-
-
-def build_llm() -> CodexLLM:
-    with patch.object(CodexLLM, "_load_codex_auth", return_value=("token", "account")):
-        return CodexLLM(
-            provider="openai-codex",
-            api_key="ignored",
-            base_url="https://chatgpt.com/backend-api",
-            model="gpt-5.4-mini",
-        )
-
-
-@pytest.mark.asyncio
-async def test_codex_normalizes_legacy_named_tool_choice_shape():
-    llm = build_llm()
-    response = MagicMock()
-    response.status_code = 200
-    response.raise_for_status.return_value = None
-    with patch.object(llm._client, "post", new_callable=AsyncMock) as mock_post:
-        mock_post.return_value = response
-        with patch.object(llm, "_parse_sse_tool_stream", new_callable=AsyncMock) as mock_parse:
-            mock_parse.return_value = (None, [])
-            await llm.call_with_tools(
-                messages=[{"role": "user", "content": "recall the memory"}],
-                tools=TOOLS,
-                tool_choice={"type": "function", "function": {"name": "recall"}},
-                max_retries=0,
-            )
-        sent_payload = mock_post.call_args.kwargs["json"]
-
-    assert sent_payload["tool_choice"] == {"type": "function", "name": "recall"}
-
-
-@pytest.mark.asyncio
-async def test_codex_forced_tool_choice_still_yields_tool_calls():
-    llm = build_llm()
-    response = MagicMock()
-    response.status_code = 200
-    response.raise_for_status.return_value = None
-    tool_call = {"id": "call-1", "name": "recall", "arguments": {"query": "memory"}}
-    with patch.object(llm._client, "post", new_callable=AsyncMock) as mock_post:
-        mock_post.return_value = response
-        with patch.object(llm, "_parse_sse_tool_stream", new_callable=AsyncMock) as mock_parse:
-            mock_parse.return_value = (None, [tool_call])
-            result = await llm.call_with_tools(
-                messages=[{"role": "user", "content": "recall the memory"}],
-                tools=TOOLS,
-                tool_choice={"type": "function", "function": {"name": "recall"}},
-                max_retries=0,
-            )
-        sent_payload = mock_post.call_args.kwargs["json"]
-
-    assert len(result.tool_calls) == 1
-    assert result.tool_calls[0].name == "recall"
-    assert sent_payload["tool_choice"] == {"type": "function", "name": "recall"}

hindsight-api-slim/tests/test_cohere_cross_encoder.py

@@ -1,347 +0,0 @@
-"""
-Tests for CohereCrossEncoder.
-
-Tests the Cohere cross-encoder implementation, including Azure AI Foundry endpoint support.
-"""
-
-import os
-from unittest.mock import AsyncMock, MagicMock, patch
-
-import httpx
-import pytest
-
-from hindsight_api.engine.cross_encoder import CohereCrossEncoder, create_cross_encoder_from_env
-
-
-class TestCohereCrossEncoder:
-    """Test suite for CohereCrossEncoder class."""
-
-    @pytest.mark.asyncio
-    async def test_initialization_native_cohere(self):
-        """Test successful initialization with native Cohere API (no base_url)."""
-        encoder = CohereCrossEncoder(
-            api_key="test_key",
-            model="rerank-english-v3.0",
-        )
-
-        assert encoder.provider_name == "cohere"
-        assert encoder.api_key == "test_key"
-        assert encoder.model == "rerank-english-v3.0"
-        assert encoder._client is None
-        assert encoder._http_client is None
-
-        # Mock the cohere import
-        mock_cohere = MagicMock()
-        mock_cohere.Client = MagicMock()
-        with patch.dict("sys.modules", {"cohere": mock_cohere}):
-            await encoder.initialize()
-            assert encoder._client is not None
-            assert encoder._http_client is None
-            mock_cohere.Client.assert_called_once_with(api_key="test_key", timeout=60.0)
-
-    @pytest.mark.asyncio
-    async def test_initialization_azure_endpoint(self):
-        """Test initialization with Azure AI Foundry endpoint (uses httpx)."""
-        encoder = CohereCrossEncoder(
-            api_key="test_key",
-            model="cohere-rerank-v3-english",
-            base_url="https://my-endpoint.inference.ai.azure.com/models/cohere-rerank-v3-english/invoke",
-        )
-
-        assert encoder.base_url == "https://my-endpoint.inference.ai.azure.com/models/cohere-rerank-v3-english/invoke"
-
-        await encoder.initialize()
-
-        assert encoder._http_client is not None
-        assert encoder._client is None
-        assert isinstance(encoder._http_client._async_client, httpx.AsyncClient)
-        assert encoder._http_client.include_top_n is False
-        assert (
-            encoder._http_client.rerank_url
-            == "https://my-endpoint.inference.ai.azure.com/models/cohere-rerank-v3-english/invoke"
-        )
-
-    @pytest.mark.asyncio
-    async def test_initialization_missing_package(self):
-        """Test initialization fails when cohere package is missing (native API)."""
-        encoder = CohereCrossEncoder(
-            api_key="test_key",
-            model="rerank-english-v3.0",
-        )
-
-        with patch.dict("sys.modules", {"cohere": None}):
-            with pytest.raises(ImportError, match="cohere is required"):
-                await encoder.initialize()
-
-    @pytest.mark.asyncio
-    async def test_initialization_idempotent(self):
-        """Test that calling initialize() multiple times is safe."""
-        encoder = CohereCrossEncoder(
-            api_key="test_key",
-            model="rerank-english-v3.0",
-        )
-
-        mock_cohere = MagicMock()
-        mock_cohere.Client = MagicMock()
-        with patch.dict("sys.modules", {"cohere": mock_cohere}):
-            await encoder.initialize()
-            assert encoder._client is not None
-
-            # Second call should be no-op
-            await encoder.initialize()
-            # Should only create client once
-            mock_cohere.Client.assert_called_once()
-
-    @pytest.mark.asyncio
-    async def test_predict_native_cohere_single_query(self):
-        """Test prediction with native Cohere SDK."""
-        encoder = CohereCrossEncoder(
-            api_key="test_key",
-            model="rerank-english-v3.0",
-        )
-
-        # Create mock Cohere response
-        mock_result_1 = MagicMock()
-        mock_result_1.index = 0
-        mock_result_1.relevance_score = 0.9
-
-        mock_result_2 = MagicMock()
-        mock_result_2.index = 1
-        mock_result_2.relevance_score = 0.7
-
-        mock_result_3 = MagicMock()
-        mock_result_3.index = 2
-        mock_result_3.relevance_score = 0.5
-
-        mock_response = MagicMock()
-        mock_response.results = [mock_result_1, mock_result_2, mock_result_3]
-
-        mock_cohere_client = MagicMock()
-        mock_cohere_client.rerank = MagicMock(return_value=mock_response)
-
-        mock_cohere = MagicMock()
-        mock_cohere.Client = MagicMock(return_value=mock_cohere_client)
-
-        with patch.dict("sys.modules", {"cohere": mock_cohere}):
-            await encoder.initialize()
-
-            pairs = [
-                ("What is Python?", "Python is a programming language"),
-                ("What is Python?", "Python is a snake"),
-                ("What is Python?", "Python is a British comedy group"),
-            ]
-
-            scores = await encoder.predict(pairs)
-
-            assert len(scores) == 3
-            assert scores == [0.9, 0.7, 0.5]
-
-            # Verify rerank was called correctly
-            mock_cohere_client.rerank.assert_called_once()
-            call_args = mock_cohere_client.rerank.call_args
-            assert call_args.kwargs["model"] == "rerank-english-v3.0"
-            assert call_args.kwargs["query"] == "What is Python?"
-            assert len(call_args.kwargs["documents"]) == 3
-            assert call_args.kwargs["return_documents"] is False
-
-    @pytest.mark.asyncio
-    async def test_predict_azure_endpoint_single_query(self):
-        """Test prediction with Azure AI Foundry endpoint (httpx direct call)."""
-        encoder = CohereCrossEncoder(
-            api_key="test_key",
-            model="cohere-rerank-v3-english",
-            base_url="https://my-endpoint.inference.ai.azure.com/models/cohere-rerank-v3-english/invoke",
-        )
-
-        await encoder.initialize()
-
-        # Mock async httpx response
-        mock_response = MagicMock()
-        mock_response.json.return_value = {
-            "results": [
-                {"index": 0, "relevance_score": 0.9},
-                {"index": 1, "relevance_score": 0.7},
-                {"index": 2, "relevance_score": 0.5},
-            ]
-        }
-        mock_response.raise_for_status = MagicMock()
-
-        encoder._http_client._async_client.post = AsyncMock(return_value=mock_response)
-
-        pairs = [
-            ("What is Python?", "Python is a programming language"),
-            ("What is Python?", "Python is a snake"),
-            ("What is Python?", "Python is a British comedy group"),
-        ]
-
-        scores = await encoder.predict(pairs)
-
-        assert len(scores) == 3
-        assert scores == [0.9, 0.7, 0.5]
-
-        # Verify httpx.post was called with correct URL and payload
-        encoder._http_client._async_client.post.assert_called_once()
-        call_args = encoder._http_client._async_client.post.call_args
-        assert call_args[0][0] == "https://my-endpoint.inference.ai.azure.com/models/cohere-rerank-v3-english/invoke"
-        assert call_args.kwargs["json"]["model"] == "cohere-rerank-v3-english"
-        assert call_args.kwargs["json"]["query"] == "What is Python?"
-        assert len(call_args.kwargs["json"]["documents"]) == 3
-        assert call_args.kwargs["json"]["return_documents"] is False
-        # Azure endpoints expect no top_n in the body
-        assert "top_n" not in call_args.kwargs["json"]
-
-    @pytest.mark.asyncio
-    async def test_predict_multiple_queries(self):
-        """Test prediction with multiple different queries (grouped efficiently)."""
-        encoder = CohereCrossEncoder(
-            api_key="test_key",
-            model="rerank-english-v3.0",
-        )
-
-        # First query response
-        mock_result_1_1 = MagicMock()
-        mock_result_1_1.index = 0
-        mock_result_1_1.relevance_score = 0.9
-
-        mock_result_1_2 = MagicMock()
-        mock_result_1_2.index = 1
-        mock_result_1_2.relevance_score = 0.7
-
-        mock_response1 = MagicMock()
-        mock_response1.results = [mock_result_1_1, mock_result_1_2]
-
-        # Second query response
-        mock_result_2_1 = MagicMock()
-        mock_result_2_1.index = 0
-        mock_result_2_1.relevance_score = 0.8
-
-        mock_response2 = MagicMock()
-        mock_response2.results = [mock_result_2_1]
-
-        mock_cohere_client = MagicMock()
-        mock_cohere_client.rerank = MagicMock(side_effect=[mock_response1, mock_response2])
-
-        mock_cohere = MagicMock()
-        mock_cohere.Client = MagicMock(return_value=mock_cohere_client)
-
-        with patch.dict("sys.modules", {"cohere": mock_cohere}):
-            await encoder.initialize()
-
-            pairs = [
-                ("What is Python?", "Python is a programming language"),
-                ("What is Python?", "Python is a snake"),
-                ("What is Java?", "Java is a programming language"),
-            ]
-
-            scores = await encoder.predict(pairs)
-
-            assert len(scores) == 3
-            assert scores[0] == 0.9  # First query, first doc
-            assert scores[1] == 0.7  # First query, second doc
-            assert scores[2] == 0.8  # Second query, first doc
-
-            # Verify rerank was called twice (once per unique query)
-            assert mock_cohere_client.rerank.call_count == 2
-
-    @pytest.mark.asyncio
-    async def test_predict_empty_pairs(self):
-        """Test prediction with empty input."""
-        encoder = CohereCrossEncoder(
-            api_key="test_key",
-            model="rerank-english-v3.0",
-        )
-
-        mock_cohere = MagicMock()
-        mock_cohere.Client = MagicMock()
-        with patch.dict("sys.modules", {"cohere": mock_cohere}):
-            await encoder.initialize()
-            scores = await encoder.predict([])
-            assert scores == []
-
-    @pytest.mark.asyncio
-    async def test_predict_not_initialized(self):
-        """Test that predict fails if encoder not initialized."""
-        encoder = CohereCrossEncoder(
-            api_key="test_key",
-            model="rerank-english-v3.0",
-        )
-
-        pairs = [("query", "document")]
-
-        with pytest.raises(RuntimeError, match="not initialized"):
-            await encoder.predict(pairs)
-
-    @pytest.mark.asyncio
-    async def test_azure_endpoint_http_error(self):
-        """Test that HTTP errors from Azure endpoint are raised."""
-        encoder = CohereCrossEncoder(
-            api_key="test_key",
-            model="cohere-rerank-v3-english",
-            base_url="https://my-endpoint.inference.ai.azure.com/models/cohere-rerank-v3-english/invoke",
-        )
-
-        await encoder.initialize()
-
-        # Mock httpx to raise HTTP error
-        mock_response = MagicMock()
-        mock_response.raise_for_status.side_effect = httpx.HTTPStatusError(
-            "404 Not Found",
-            request=MagicMock(),
-            response=MagicMock(status_code=404),
-        )
-        encoder._http_client._async_client.post = AsyncMock(return_value=mock_response)
-
-        pairs = [("What is Python?", "Python is a programming language")]
-
-        # Should raise the HTTP error
-        with pytest.raises(httpx.HTTPStatusError):
-            await encoder.predict(pairs)
-
-
-class TestFactoryFunction:
-    """Test suite for create_cross_encoder_from_env factory function."""
-
-    @pytest.mark.asyncio
-    async def test_create_cohere_from_env(self):
-        """Test creating Cohere cross-encoder from environment variables."""
-        env_vars = {
-            "HINDSIGHT_API_RERANKER_PROVIDER": "cohere",
-            "HINDSIGHT_API_RERANKER_COHERE_API_KEY": "test_key",
-            "HINDSIGHT_API_RERANKER_COHERE_MODEL": "rerank-english-v3.0",
-        }
-
-        with patch.dict(os.environ, env_vars, clear=False):
-            from hindsight_api.config import HindsightConfig
-
-            config = HindsightConfig.from_env()
-
-            with patch("hindsight_api.config.get_config", return_value=config):
-                encoder = create_cross_encoder_from_env()
-
-                assert isinstance(encoder, CohereCrossEncoder)
-                assert encoder.api_key == "test_key"
-                assert encoder.model == "rerank-english-v3.0"
-                assert encoder.base_url is None
-
-    @pytest.mark.asyncio
-    async def test_create_cohere_with_azure_base_url_from_env(self):
-        """Test creating Cohere cross-encoder with Azure base URL from environment."""
-        env_vars = {
-            "HINDSIGHT_API_RERANKER_PROVIDER": "cohere",
-            "HINDSIGHT_API_RERANKER_COHERE_API_KEY": "test_key",
-            "HINDSIGHT_API_RERANKER_COHERE_MODEL": "cohere-rerank-v3-english",
-            "HINDSIGHT_API_RERANKER_COHERE_BASE_URL": "https://my-endpoint.inference.ai.azure.com/models/cohere-rerank-v3-english/invoke",
-        }
-
-        with patch.dict(os.environ, env_vars, clear=False):
-            from hindsight_api.config import HindsightConfig
-
-            config = HindsightConfig.from_env()
-
-            with patch("hindsight_api.config.get_config", return_value=config):
-                encoder = create_cross_encoder_from_env()
-
-                assert isinstance(encoder, CohereCrossEncoder)
-                assert encoder.api_key == "test_key"
-                assert encoder.model == "cohere-rerank-v3-english"
-                assert encoder.base_url == "https://my-endpoint.inference.ai.azure.com/models/cohere-rerank-v3-english/invoke"

hindsight-api-slim/tests/test_consolidation_failure_recovery.py

@@ -1,476 +0,0 @@
-"""Tests for consolidation failure handling: adaptive batch splitting, consolidation_failed_at,
-and the recovery API.
-
-These tests use a mock LLM to simulate LLM failures deterministically, without making real
-API calls. All tests insert memories directly into the database to bypass retain's LLM calls
-and focus exclusively on the consolidation code paths.
-"""
-
-import uuid
-from unittest.mock import MagicMock
-
-import pytest
-import pytest_asyncio
-
-from hindsight_api.engine.consolidation.consolidator import run_consolidation_job
-from hindsight_api.engine.memory_engine import MemoryEngine
-from hindsight_api.engine.providers.mock_llm import MockLLM
-from hindsight_api.engine.task_backend import SyncTaskBackend
-
-
-@pytest_asyncio.fixture(scope="function")
-async def memory_no_llm_verify(pg0_db_url, embeddings, cross_encoder, query_analyzer):
-    """MemoryEngine with mock LLM.
-
-    Migrations are already applied by the session-scoped pg0_db_url fixture, so
-    run_migrations=False avoids advisory-lock serialization overhead per test.
-    """
-    mem = MemoryEngine(
-        db_url=pg0_db_url,
-        memory_llm_provider="mock",
-        memory_llm_api_key="",
-        memory_llm_model="mock",
-        embeddings=embeddings,
-        cross_encoder=cross_encoder,
-        query_analyzer=query_analyzer,
-        pool_min_size=1,
-        pool_max_size=5,
-        run_migrations=False,
-        task_backend=SyncTaskBackend(),
-        skip_llm_verification=True,
-    )
-    await mem.initialize()
-    yield mem
-    try:
-        if mem._pool and not mem._pool._closing:
-            await mem.close()
-    except Exception:
-        pass
-
-
-@pytest.fixture(autouse=True)
-def enable_observations():
-    """Enable observations for all tests in this module."""
-    from hindsight_api.config import _get_raw_config
-
-    config = _get_raw_config()
-    original = config.enable_observations
-    config.enable_observations = True
-    yield
-    config.enable_observations = original
-
-
-def _make_failing_mock_llm(*, fail_first_n: int = 999) -> MockLLM:
-    """Return a MockLLM that raises ValueError for the first `fail_first_n` consolidation calls."""
-    mock_llm = MockLLM(provider="mock", api_key="", base_url="", model="mock-model")
-    call_count = 0
-
-    def callback(messages, scope):
-        nonlocal call_count
-        if scope == "consolidation":
-            call_count += 1
-            if call_count <= fail_first_n:
-                raise ValueError(f"Simulated LLM failure (call {call_count})")
-        # Return empty response — no creates/updates/deletes
-        from hindsight_api.engine.consolidation.consolidator import _ConsolidationBatchResponse
-
-        return _ConsolidationBatchResponse()
-
-    mock_llm.set_response_callback(callback)
-    return mock_llm
-
-
-def _make_always_success_mock_llm() -> MockLLM:
-    """Return a MockLLM that always succeeds with an empty consolidation response."""
-    mock_llm = MockLLM(provider="mock", api_key="", base_url="", model="mock-model")
-
-    def callback(messages, scope):
-        from hindsight_api.engine.consolidation.consolidator import _ConsolidationBatchResponse
-
-        return _ConsolidationBatchResponse()
-
-    mock_llm.set_response_callback(callback)
-    return mock_llm
-
-
-def _inject_mock_llm(memory: MemoryEngine, mock_llm: MockLLM) -> None:
-    """Replace memory._consolidation_llm_config with a wrapper that returns mock_llm from with_config."""
-    wrapper = MagicMock()
-    wrapper.with_config.return_value = mock_llm
-    memory._consolidation_llm_config = wrapper
-
-
-async def _insert_memories(conn, bank_id: str, texts: list[str]) -> list[uuid.UUID]:
-    """Insert experience memories directly, bypassing LLM-based retain."""
-    ids = []
-    for text in texts:
-        mem_id = uuid.uuid4()
-        await conn.execute(
-            """
-            INSERT INTO memory_units (id, bank_id, text, fact_type, created_at)
-            VALUES ($1, $2, $3, 'experience', now())
-            """,
-            mem_id,
-            bank_id,
-            text,
-        )
-        ids.append(mem_id)
-    return ids
-
-
-class TestAdaptiveBatchSplitting:
-    """Verify that a failing batch is halved and retried until batch_size=1 succeeds."""
-
-    @pytest.mark.asyncio
-    async def test_splitting_recovers_all_memories(self, memory_no_llm_verify: MemoryEngine, request_context):
-        """When a batch of 2 fails, both are retried individually and succeed."""
-        bank_id = f"test-split-recovery-{uuid.uuid4().hex[:8]}"
-        await memory_no_llm_verify.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            mem_ids = await _insert_memories(
-                conn,
-                bank_id,
-                [
-                    "Alice runs marathons every spring.",
-                    "Alice trained for six months for her last race.",
-                ],
-            )
-
-        # Exhaust all 3 retries for batch=2 (calls 1-3 fail), then each batch=1 succeeds (calls 4-5)
-        mock_llm = _make_failing_mock_llm(fail_first_n=3)
-        _inject_mock_llm(memory_no_llm_verify, mock_llm)
-
-        result = await run_consolidation_job(
-            memory_engine=memory_no_llm_verify,
-            bank_id=bank_id,
-            request_context=request_context,
-        )
-
-        assert result["status"] == "completed"
-        assert result["memories_processed"] == 2
-        assert result["memories_failed"] == 0
-
-        # Both memories must have consolidated_at set and consolidation_failed_at NULL
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            rows = await conn.fetch(
-                """
-                SELECT id, consolidated_at, consolidation_failed_at
-                FROM memory_units
-                WHERE bank_id = $1 AND fact_type = 'experience'
-                """,
-                bank_id,
-            )
-        assert len(rows) == 2
-        for row in rows:
-            assert row["consolidated_at"] is not None, f"Memory {row['id']} should have consolidated_at set"
-            assert row["consolidation_failed_at"] is None, (
-                f"Memory {row['id']} should NOT have consolidation_failed_at set"
-            )
-
-        # LLM called 5 times: 3 retries failed (batch=2) + 1 succeeded (batch=1) + 1 succeeded (batch=1)
-        consolidation_calls = [c for c in mock_llm.get_mock_calls() if c["scope"] == "consolidation"]
-        assert len(consolidation_calls) == 5
-
-        await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)
-
-    @pytest.mark.asyncio
-    async def test_splitting_with_larger_batch(self, memory_no_llm_verify: MemoryEngine, request_context):
-        """A batch of 4 that always fails at size>1 resolves to 4 individual calls."""
-        bank_id = f"test-split-large-{uuid.uuid4().hex[:8]}"
-        await memory_no_llm_verify.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            await _insert_memories(
-                conn,
-                bank_id,
-                [
-                    "Bob plays chess competitively.",
-                    "Bob won a regional chess tournament.",
-                    "Bob practices tactics every morning.",
-                    "Bob coaches youth chess on weekends.",
-                ],
-            )
-
-        # Exhaust all 3 retries for batch=4 (calls 1-3 fail), then both batch=2 halves succeed
-        # (calls 4-5). This verifies that halving once is sufficient when batch=2 works.
-        mock_llm = _make_failing_mock_llm(fail_first_n=3)
-        _inject_mock_llm(memory_no_llm_verify, mock_llm)
-
-        result = await run_consolidation_job(
-            memory_engine=memory_no_llm_verify,
-            bank_id=bank_id,
-            request_context=request_context,
-        )
-
-        assert result["memories_processed"] == 4
-        assert result["memories_failed"] == 0
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            rows = await conn.fetch(
-                "SELECT consolidated_at, consolidation_failed_at FROM memory_units "
-                "WHERE bank_id = $1 AND fact_type = 'experience'",
-                bank_id,
-            )
-        assert all(r["consolidated_at"] is not None for r in rows)
-        assert all(r["consolidation_failed_at"] is None for r in rows)
-
-        await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)
-
-
-class TestConsolidationFailedAt:
-    """Verify that consolidation_failed_at is set — and consolidated_at is NOT — when all retries fail."""
-
-    @pytest.mark.asyncio
-    async def test_single_memory_permanent_failure(self, memory_no_llm_verify: MemoryEngine, request_context):
-        """A single memory that exhausts all LLM retries gets consolidation_failed_at, not consolidated_at."""
-        bank_id = f"test-perm-fail-{uuid.uuid4().hex[:8]}"
-        await memory_no_llm_verify.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            (mem_id,) = await _insert_memories(conn, bank_id, ["Carol enjoys painting watercolors."])
-
-        # Always fail
-        mock_llm = _make_failing_mock_llm(fail_first_n=999)
-        _inject_mock_llm(memory_no_llm_verify, mock_llm)
-
-        result = await run_consolidation_job(
-            memory_engine=memory_no_llm_verify,
-            bank_id=bank_id,
-            request_context=request_context,
-        )
-
-        assert result["memories_failed"] == 1
-        assert result["memories_processed"] == 1
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            row = await conn.fetchrow(
-                "SELECT consolidated_at, consolidation_failed_at FROM memory_units WHERE id = $1",
-                mem_id,
-            )
-
-        assert row["consolidated_at"] is None, "consolidated_at must NOT be set for a permanently failed memory"
-        assert row["consolidation_failed_at"] is not None, "consolidation_failed_at must be set"
-
-        await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)
-
-    @pytest.mark.asyncio
-    async def test_failed_memory_excluded_from_next_run(self, memory_no_llm_verify: MemoryEngine, request_context):
-        """A memory marked consolidation_failed_at is not re-processed on the next consolidation run."""
-        bank_id = f"test-excluded-{uuid.uuid4().hex[:8]}"
-        await memory_no_llm_verify.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            (mem_id,) = await _insert_memories(conn, bank_id, ["Dave collects vinyl records."])
-            # Manually stamp consolidation_failed_at to simulate a prior failed run
-            await conn.execute(
-                "UPDATE memory_units SET consolidation_failed_at = NOW() WHERE id = $1",
-                mem_id,
-            )
-
-        # Even with a healthy LLM, the memory should be skipped
-        mock_llm = _make_always_success_mock_llm()
-        _inject_mock_llm(memory_no_llm_verify, mock_llm)
-
-        result = await run_consolidation_job(
-            memory_engine=memory_no_llm_verify,
-            bank_id=bank_id,
-            request_context=request_context,
-        )
-
-        # No unconsolidated memories to pick up (consolidation_failed_at ≠ NULL, consolidated_at = NULL
-        # but the SELECT filters on consolidated_at IS NULL AND fact_type IN ('experience','world'))
-        assert result["status"] in ("no_new_memories", "completed")
-        if result["status"] == "completed":
-            assert result["memories_processed"] == 0
-
-        # Memory still has consolidation_failed_at set and consolidated_at NULL
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            row = await conn.fetchrow(
-                "SELECT consolidated_at, consolidation_failed_at FROM memory_units WHERE id = $1",
-                mem_id,
-            )
-        assert row["consolidated_at"] is None
-        assert row["consolidation_failed_at"] is not None
-
-        await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)
-
-    @pytest.mark.asyncio
-    async def test_partial_batch_failure(self, memory_no_llm_verify: MemoryEngine, request_context):
-        """In a batch of 2, if only the first individual retry fails, the second still succeeds."""
-        bank_id = f"test-partial-fail-{uuid.uuid4().hex[:8]}"
-        await memory_no_llm_verify.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            mem_ids = await _insert_memories(
-                conn,
-                bank_id,
-                [
-                    "Eve speaks three languages fluently.",
-                    "Eve learned Japanese in two years.",
-                ],
-            )
-
-        # Exhaust 3 retries for batch=2 (calls 1-3), exhaust 3 retries for first batch=1 (calls 4-6),
-        # second batch=1 succeeds (call 7)
-        mock_llm = _make_failing_mock_llm(fail_first_n=6)
-        _inject_mock_llm(memory_no_llm_verify, mock_llm)
-
-        result = await run_consolidation_job(
-            memory_engine=memory_no_llm_verify,
-            bank_id=bank_id,
-            request_context=request_context,
-        )
-
-        assert result["memories_processed"] == 2
-        assert result["memories_failed"] == 1
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            rows = {
-                str(r["id"]): r
-                for r in await conn.fetch(
-                    "SELECT id, consolidated_at, consolidation_failed_at FROM memory_units "
-                    "WHERE bank_id = $1 AND fact_type = 'experience'",
-                    bank_id,
-                )
-            }
-
-        # One should have failed, one should have succeeded
-        failed = [r for r in rows.values() if r["consolidation_failed_at"] is not None]
-        succeeded = [r for r in rows.values() if r["consolidated_at"] is not None]
-        assert len(failed) == 1
-        assert len(succeeded) == 1
-        # They must be different memories
-        assert str(failed[0]["id"]) != str(succeeded[0]["id"])
-
-        await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)
-
-
-class TestRecoverConsolidation:
-    """Verify the retry_failed_consolidation() method and the /consolidation/recover endpoint."""
-
-    @pytest.mark.asyncio
-    async def test_recover_resets_failed_memories(self, memory_no_llm_verify: MemoryEngine, request_context):
-        """retry_failed_consolidation resets consolidation_failed_at and consolidated_at."""
-        bank_id = f"test-recover-reset-{uuid.uuid4().hex[:8]}"
-        await memory_no_llm_verify.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            ids = await _insert_memories(
-                conn,
-                bank_id,
-                [
-                    "Frank is a competitive cyclist.",
-                    "Frank completed the Tour de France route.",
-                ],
-            )
-            # Mark both as failed
-            for mem_id in ids:
-                await conn.execute(
-                    "UPDATE memory_units SET consolidation_failed_at = NOW() WHERE id = $1",
-                    mem_id,
-                )
-
-        result = await memory_no_llm_verify.retry_failed_consolidation(
-            bank_id, request_context=request_context
-        )
-
-        assert result["retried_count"] == 2
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            rows = await conn.fetch(
-                "SELECT consolidated_at, consolidation_failed_at FROM memory_units "
-                "WHERE bank_id = $1 AND fact_type = 'experience'",
-                bank_id,
-            )
-        assert all(r["consolidation_failed_at"] is None for r in rows), "consolidation_failed_at must be cleared"
-        assert all(r["consolidated_at"] is None for r in rows), "consolidated_at must also be cleared"
-
-        await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)
-
-    @pytest.mark.asyncio
-    async def test_recover_returns_zero_when_none_failed(self, memory_no_llm_verify: MemoryEngine, request_context):
-        """retry_failed_consolidation returns 0 when no memories have failed."""
-        bank_id = f"test-recover-zero-{uuid.uuid4().hex[:8]}"
-        await memory_no_llm_verify.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-        result = await memory_no_llm_verify.retry_failed_consolidation(
-            bank_id, request_context=request_context
-        )
-
-        assert result["retried_count"] == 0
-
-        await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)
-
-    @pytest.mark.asyncio
-    async def test_recover_then_consolidate_succeeds(self, memory_no_llm_verify: MemoryEngine, request_context):
-        """After recovery, the memory is picked up by the next consolidation run."""
-        bank_id = f"test-recover-consolidate-{uuid.uuid4().hex[:8]}"
-        await memory_no_llm_verify.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            (mem_id,) = await _insert_memories(conn, bank_id, ["Grace is an expert rock climber."])
-            await conn.execute(
-                "UPDATE memory_units SET consolidation_failed_at = NOW() WHERE id = $1", mem_id
-            )
-
-        # Recover
-        recover_result = await memory_no_llm_verify.retry_failed_consolidation(
-            bank_id, request_context=request_context
-        )
-        assert recover_result["retried_count"] == 1
-
-        # Now consolidate with a healthy LLM
-        mock_llm = _make_always_success_mock_llm()
-        _inject_mock_llm(memory_no_llm_verify, mock_llm)
-
-        run_result = await run_consolidation_job(
-            memory_engine=memory_no_llm_verify,
-            bank_id=bank_id,
-            request_context=request_context,
-        )
-
-        assert run_result["memories_processed"] == 1
-        assert run_result["memories_failed"] == 0
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            row = await conn.fetchrow(
-                "SELECT consolidated_at, consolidation_failed_at FROM memory_units WHERE id = $1",
-                mem_id,
-            )
-        assert row["consolidated_at"] is not None, "Memory should be consolidated after recovery"
-        assert row["consolidation_failed_at"] is None
-
-        await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)
-
-    @pytest.mark.asyncio
-    async def test_recover_endpoint_via_http(self, memory_no_llm_verify: MemoryEngine, request_context):
-        """The POST /consolidation/recover endpoint returns the correct retried_count."""
-        import httpx
-
-        from hindsight_api.api.http import create_app
-
-        bank_id = f"test-recover-http-{uuid.uuid4().hex[:8]}"
-        await memory_no_llm_verify.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-        async with memory_no_llm_verify._pool.acquire() as conn:
-            ids = await _insert_memories(
-                conn,
-                bank_id,
-                ["Henry is a professional chef.", "Henry trained at Le Cordon Bleu."],
-            )
-            for mem_id in ids:
-                await conn.execute(
-                    "UPDATE memory_units SET consolidation_failed_at = NOW() WHERE id = $1", mem_id
-                )
-
-        app = create_app(memory_no_llm_verify, initialize_memory=False)
-        transport = httpx.ASGITransport(app=app)
-        async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
-            response = await client.post(f"/v1/default/banks/{bank_id}/consolidation/recover")
-
-        assert response.status_code == 200
-        body = response.json()
-        assert body["retried_count"] == 2
-
-        await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)

hindsight-api-slim/tests/test_delta_retain.py

@@ -1,842 +0,0 @@
-"""
-Tests for delta retain — upsert optimization that only re-processes changed chunks.
-"""
-
-import logging
-from datetime import datetime, timezone
-
-import pytest
-
-from hindsight_api import RequestContext
-from hindsight_api.engine.memory_engine import Budget
-
-logger = logging.getLogger(__name__)
-
-
-def _ts():
-    return datetime.now(timezone.utc).timestamp()
-
-
-# ============================================================
-# Core Delta Retain Tests
-# ============================================================
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_unchanged_content_skips_llm(memory, request_context):
-    """
-    When upserting a document with identical content, no new facts should be
-    extracted (LLM is not called for unchanged chunks). The existing facts
-    should be preserved.
-    """
-    bank_id = f"test_delta_unchanged_{_ts()}"
-    document_id = "conversation-001"
-
-    try:
-        content = "Alice works at Google. Bob works at Microsoft."
-
-        # First retain — full processing
-        v1_units = await memory.retain_async(
-            bank_id=bank_id,
-            content=content,
-            context="team info",
-            document_id=document_id,
-            request_context=request_context,
-        )
-        assert len(v1_units) > 0, "v1 should create facts"
-
-        # Get v1 document state
-        doc_v1 = await memory.get_document(document_id, bank_id, request_context=request_context)
-        v1_unit_count = doc_v1["memory_unit_count"]
-
-        # Second retain — same content, should use delta path (no new facts)
-        v2_units = await memory.retain_async(
-            bank_id=bank_id,
-            content=content,
-            context="team info",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        # No new units should be returned (nothing changed)
-        assert v2_units == [], "Delta retain with unchanged content should return empty unit list"
-
-        # Existing facts should still be there
-        doc_v2 = await memory.get_document(document_id, bank_id, request_context=request_context)
-        assert doc_v2["memory_unit_count"] == v1_unit_count, "Existing facts should be preserved"
-
-        # Verify recall still works
-        result = await memory.recall_async(
-            bank_id=bank_id,
-            query="Where does Alice work?",
-            budget=Budget.MID,
-            max_tokens=1000,
-            request_context=request_context,
-        )
-        assert len(result.results) > 0, "Should still recall facts after delta retain"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_appended_content(memory, request_context):
-    """
-    When a conversation grows (new content appended), only new chunks should
-    be processed. Facts from unchanged chunks should be preserved.
-    """
-    bank_id = f"test_delta_append_{_ts()}"
-    document_id = "growing-conversation"
-
-    try:
-        # First version — short content (single chunk)
-        v1_content = "Alice is a software engineer at Google. She works on search infrastructure."
-
-        v1_units = await memory.retain_async(
-            bank_id=bank_id,
-            content=v1_content,
-            context="profile",
-            document_id=document_id,
-            request_context=request_context,
-        )
-        assert len(v1_units) > 0
-
-        # Get v1 facts via recall
-        v1_recall = await memory.recall_async(
-            bank_id=bank_id,
-            query="What does Alice do?",
-            budget=Budget.MID,
-            max_tokens=2000,
-            request_context=request_context,
-        )
-        v1_fact_texts = {r.text for r in v1_recall.results}
-
-        # Second version — original content + new content appended
-        # This should preserve facts from the first chunk and add new ones
-        v2_content = v1_content + "\n\nBob joined Google as a product manager in 2024. He previously worked at Meta on AR/VR products."
-
-        v2_units = await memory.retain_async(
-            bank_id=bank_id,
-            content=v2_content,
-            context="profile",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        # Should have facts about Bob from the new content
-        v2_recall = await memory.recall_async(
-            bank_id=bank_id,
-            query="What does Bob do?",
-            budget=Budget.MID,
-            max_tokens=2000,
-            request_context=request_context,
-        )
-        bob_facts = [r for r in v2_recall.results if "bob" in r.text.lower()]
-        assert len(bob_facts) > 0, "Should have facts about Bob from appended content"
-
-        # Should still have facts about Alice from original content
-        alice_recall = await memory.recall_async(
-            bank_id=bank_id,
-            query="What does Alice do?",
-            budget=Budget.MID,
-            max_tokens=2000,
-            request_context=request_context,
-        )
-        assert len(alice_recall.results) > 0, "Should still have Alice facts from original content"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_modified_chunk(memory, request_context):
-    """
-    When content in the middle changes, that chunk should be re-processed
-    while other chunks are preserved.
-    """
-    bank_id = f"test_delta_modified_{_ts()}"
-    document_id = "changing-doc"
-
-    try:
-        # v1: Alice works at Google
-        v1_content = "Alice works at Google as a senior engineer."
-        v1_units = await memory.retain_async(
-            bank_id=bank_id,
-            content=v1_content,
-            context="team",
-            document_id=document_id,
-            request_context=request_context,
-        )
-        assert len(v1_units) > 0
-
-        # v2: Alice works at Microsoft (changed)
-        v2_content = "Alice works at Microsoft as a principal engineer."
-        v2_units = await memory.retain_async(
-            bank_id=bank_id,
-            content=v2_content,
-            context="team",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        # New facts should reflect the updated content
-        result = await memory.recall_async(
-            bank_id=bank_id,
-            query="Where does Alice work?",
-            budget=Budget.MID,
-            max_tokens=2000,
-            request_context=request_context,
-        )
-        all_texts = " ".join(r.text.lower() for r in result.results)
-        assert "microsoft" in all_texts, f"Should have updated fact about Microsoft, got: {all_texts}"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-# ============================================================
-# Entity & Link Tests
-# ============================================================
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_entities_preserved_for_unchanged_chunks(memory, request_context):
-    """
-    Entities linked to unchanged chunks should be preserved after delta retain.
-    """
-    bank_id = f"test_delta_entities_{_ts()}"
-    document_id = "entity-doc"
-
-    try:
-        v1_content = "Alice works at Google. She is a senior engineer in the Cloud division."
-        v1_units = await memory.retain_async(
-            bank_id=bank_id,
-            content=v1_content,
-            context="team",
-            document_id=document_id,
-            request_context=request_context,
-        )
-        assert len(v1_units) > 0
-
-        # Check entities exist
-        pool = await memory._get_pool()
-        async with pool.acquire() as conn:
-            v1_entities = await conn.fetch(
-                "SELECT canonical_name FROM entities WHERE bank_id = $1",
-                bank_id,
-            )
-        v1_entity_names = {e["canonical_name"].lower() for e in v1_entities}
-        assert len(v1_entity_names) > 0, "Should have entities after v1 retain"
-
-        # Upsert with same content — entities should persist
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=v1_content,
-            context="team",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        async with pool.acquire() as conn:
-            v2_entities = await conn.fetch(
-                "SELECT canonical_name FROM entities WHERE bank_id = $1",
-                bank_id,
-            )
-        v2_entity_names = {e["canonical_name"].lower() for e in v2_entities}
-
-        # All v1 entities should still exist
-        assert v1_entity_names.issubset(v2_entity_names), (
-            f"v1 entities {v1_entity_names} should be preserved, got {v2_entity_names}"
-        )
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_new_entities_created_for_new_chunks(memory, request_context):
-    """
-    New entities should be created for newly added chunks during delta retain.
-    """
-    bank_id = f"test_delta_new_entities_{_ts()}"
-    document_id = "entity-growth-doc"
-
-    try:
-        v1_content = "Alice works at Google."
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=v1_content,
-            context="team",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        pool = await memory._get_pool()
-        async with pool.acquire() as conn:
-            v1_entities = await conn.fetch(
-                "SELECT canonical_name FROM entities WHERE bank_id = $1",
-                bank_id,
-            )
-        v1_entity_names = {e["canonical_name"].lower() for e in v1_entities}
-
-        # Append content mentioning new entities
-        v2_content = v1_content + "\n\nBob joined Facebook. He works with Charlie on the Reality Labs project."
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=v2_content,
-            context="team",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        async with pool.acquire() as conn:
-            v2_entities = await conn.fetch(
-                "SELECT canonical_name FROM entities WHERE bank_id = $1",
-                bank_id,
-            )
-        v2_entity_names = {e["canonical_name"].lower() for e in v2_entities}
-
-        # Should have more entities after adding content with new people/orgs
-        assert len(v2_entity_names) > len(v1_entity_names), (
-            f"Should have more entities after append: v1={v1_entity_names}, v2={v2_entity_names}"
-        )
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_links_preserved_for_unchanged_chunks(memory, request_context):
-    """
-    Memory links (temporal, semantic, entity) for unchanged chunks should be preserved.
-    """
-    bank_id = f"test_delta_links_{_ts()}"
-    document_id = "links-doc"
-
-    try:
-        content = "Alice is a senior engineer at Google Cloud. She mentors junior engineers and reviews their code."
-        v1_units = await memory.retain_async(
-            bank_id=bank_id,
-            content=content,
-            context="team",
-            document_id=document_id,
-            request_context=request_context,
-        )
-        assert len(v1_units) > 0
-
-        # Count links after v1
-        pool = await memory._get_pool()
-        async with pool.acquire() as conn:
-            v1_link_count = await conn.fetchval(
-                """SELECT COUNT(*) FROM memory_links ml
-                   JOIN memory_units mu ON ml.from_unit_id = mu.id
-                   WHERE mu.bank_id = $1 AND mu.document_id = $2""",
-                bank_id,
-                document_id,
-            )
-
-        # Upsert with same content
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=content,
-            context="team",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        # Links should be preserved
-        async with pool.acquire() as conn:
-            v2_link_count = await conn.fetchval(
-                """SELECT COUNT(*) FROM memory_links ml
-                   JOIN memory_units mu ON ml.from_unit_id = mu.id
-                   WHERE mu.bank_id = $1 AND mu.document_id = $2""",
-                bank_id,
-                document_id,
-            )
-
-        assert v2_link_count == v1_link_count, (
-            f"Links should be preserved: v1={v1_link_count}, v2={v2_link_count}"
-        )
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-# ============================================================
-# Document Metadata & Tags Tests
-# ============================================================
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_document_metadata_updated(memory, request_context):
-    """
-    Document metadata (retain_params, tags) should be updated even when
-    chunk content hasn't changed.
-    """
-    bank_id = f"test_delta_meta_{_ts()}"
-    document_id = "metadata-doc"
-
-    try:
-        content = "Alice works at Google."
-
-        # v1 with initial tags
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=content,
-            context="initial context",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        doc_v1 = await memory.get_document(document_id, bank_id, request_context=request_context)
-        assert doc_v1 is not None
-
-        # v2 with updated context (same content — triggers delta path)
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=content,
-            context="updated context",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        doc_v2 = await memory.get_document(document_id, bank_id, request_context=request_context)
-        assert doc_v2 is not None
-        assert doc_v2["updated_at"] >= doc_v1["updated_at"], "Document should have updated timestamp"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_tags_propagated_to_existing_units(memory, request_context):
-    """
-    When tags change during an upsert with unchanged content, the new tags
-    should be propagated to all existing memory units.
-    """
-    bank_id = f"test_delta_tags_{_ts()}"
-    document_id = "tags-doc"
-
-    try:
-        content = "Alice works at Google."
-
-        # v1 with tag "team-a"
-        await memory.retain_batch_async(
-            bank_id=bank_id,
-            contents=[{
-                "content": content,
-                "document_id": document_id,
-                "tags": ["team-a"],
-            }],
-            request_context=request_context,
-        )
-
-        pool = await memory._get_pool()
-        async with pool.acquire() as conn:
-            v1_tags = await conn.fetch(
-                "SELECT tags FROM memory_units WHERE bank_id = $1 AND document_id = $2",
-                bank_id,
-                document_id,
-            )
-        assert all("team-a" in row["tags"] for row in v1_tags), "v1 units should have team-a tag"
-
-        # v2 with same content but different tags
-        await memory.retain_batch_async(
-            bank_id=bank_id,
-            contents=[{
-                "content": content,
-                "document_id": document_id,
-                "tags": ["team-b", "important"],
-            }],
-            request_context=request_context,
-        )
-
-        async with pool.acquire() as conn:
-            v2_tags = await conn.fetch(
-                "SELECT tags FROM memory_units WHERE bank_id = $1 AND document_id = $2",
-                bank_id,
-                document_id,
-            )
-        for row in v2_tags:
-            assert "team-b" in row["tags"], f"v2 units should have team-b tag, got {row['tags']}"
-            assert "important" in row["tags"], f"v2 units should have important tag, got {row['tags']}"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-# ============================================================
-# Chunk Management Tests
-# ============================================================
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_removed_chunks_delete_facts(memory, request_context):
-    """
-    When content is shortened (chunks removed), facts from the removed
-    chunks should be deleted.
-    """
-    bank_id = f"test_delta_removed_{_ts()}"
-    document_id = "shrinking-doc"
-
-    try:
-        # v1: longer content with facts about Alice and Bob
-        v1_content = (
-            "Alice is a senior engineer at Google Cloud. "
-            "She leads the infrastructure team and has been there for 5 years.\n\n"
-            "Bob is a product manager at Facebook Reality Labs. "
-            "He previously worked at Amazon on Alexa voice products."
-        )
-
-        v1_units = await memory.retain_async(
-            bank_id=bank_id,
-            content=v1_content,
-            context="profiles",
-            document_id=document_id,
-            request_context=request_context,
-        )
-        assert len(v1_units) > 0
-
-        doc_v1 = await memory.get_document(document_id, bank_id, request_context=request_context)
-        v1_count = doc_v1["memory_unit_count"]
-
-        # v2: Completely different content — all chunks change
-        v2_content = "Charlie works at Netflix as a data scientist."
-        v2_units = await memory.retain_async(
-            bank_id=bank_id,
-            content=v2_content,
-            context="profiles",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        doc_v2 = await memory.get_document(document_id, bank_id, request_context=request_context)
-        assert doc_v2 is not None
-
-        # Should have facts about Charlie
-        result = await memory.recall_async(
-            bank_id=bank_id,
-            query="Who works at Netflix?",
-            budget=Budget.MID,
-            max_tokens=2000,
-            request_context=request_context,
-        )
-        all_texts = " ".join(r.text.lower() for r in result.results)
-        assert "charlie" in all_texts or "netflix" in all_texts, (
-            f"Should have facts about Charlie/Netflix after replacing content, got: {all_texts}"
-        )
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_chunks_have_content_hash(memory, request_context):
-    """
-    After retain, chunks should have content_hash populated.
-    """
-    bank_id = f"test_delta_hash_{_ts()}"
-    document_id = "hash-doc"
-
-    try:
-        content = "Alice works at Google as a software engineer."
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=content,
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        pool = await memory._get_pool()
-        async with pool.acquire() as conn:
-            chunks = await conn.fetch(
-                "SELECT chunk_id, content_hash FROM chunks WHERE document_id = $1 AND bank_id = $2",
-                document_id,
-                bank_id,
-            )
-
-        assert len(chunks) > 0, "Should have stored chunks"
-        for chunk in chunks:
-            assert chunk["content_hash"] is not None, f"Chunk {chunk['chunk_id']} should have content_hash"
-            assert len(chunk["content_hash"]) == 64, "content_hash should be SHA256 hex (64 chars)"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-# ============================================================
-# Backward Compatibility Tests
-# ============================================================
-
-
-@pytest.mark.asyncio
-async def test_retain_without_document_id_still_works(memory, request_context):
-    """
-    Retain without document_id should still work normally (no delta path).
-    """
-    bank_id = f"test_no_docid_{_ts()}"
-
-    try:
-        units = await memory.retain_async(
-            bank_id=bank_id,
-            content="Alice works at Google.",
-            context="test",
-            request_context=request_context,
-        )
-        assert len(units) > 0, "Should create facts without document_id"
-
-        result = await memory.recall_async(
-            bank_id=bank_id,
-            query="Where does Alice work?",
-            budget=Budget.MID,
-            max_tokens=1000,
-            request_context=request_context,
-        )
-        assert len(result.results) > 0
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_first_retain_full_path(memory, request_context):
-    """
-    First retain of a new document should use the full path (no delta possible).
-    """
-    bank_id = f"test_first_retain_{_ts()}"
-    document_id = "new-doc"
-
-    try:
-        units = await memory.retain_async(
-            bank_id=bank_id,
-            content="Alice works at Google.",
-            context="test",
-            document_id=document_id,
-            request_context=request_context,
-        )
-        assert len(units) > 0, "First retain should create facts via full path"
-
-        doc = await memory.get_document(document_id, bank_id, request_context=request_context)
-        assert doc is not None
-        assert doc["memory_unit_count"] > 0
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-# ============================================================
-# Edge Cases
-# ============================================================
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_empty_to_content(memory, request_context):
-    """
-    Going from gibberish (zero facts) to real content should work.
-    """
-    bank_id = f"test_delta_empty_{_ts()}"
-    document_id = "empty-to-content"
-
-    try:
-        # v1: content that probably produces zero facts
-        await memory.retain_async(
-            bank_id=bank_id,
-            content="!!!###$$$%%%",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        doc_v1 = await memory.get_document(document_id, bank_id, request_context=request_context)
-        assert doc_v1 is not None
-
-        # v2: real content
-        v2_units = await memory.retain_async(
-            bank_id=bank_id,
-            content="Alice works at Google as a senior engineer.",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        doc_v2 = await memory.get_document(document_id, bank_id, request_context=request_context)
-        assert doc_v2 is not None
-        assert doc_v2["memory_unit_count"] > 0 or len(v2_units) > 0, "Should have facts after updating with real content"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_multiple_upserts(memory, request_context):
-    """
-    Multiple sequential upserts should work correctly, with delta optimization
-    kicking in after the first retain.
-    """
-    bank_id = f"test_delta_multi_{_ts()}"
-    document_id = "multi-upsert"
-
-    try:
-        # v1: initial
-        v1_content = "Alice works at Google."
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=v1_content,
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        # v2: same content (delta: no changes)
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=v1_content,
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        # v3: append
-        v3_content = v1_content + "\n\nBob works at Microsoft."
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=v3_content,
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        # v4: same as v3 (delta: no changes again)
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=v3_content,
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        # Final check: should have facts about both Alice and Bob
-        result = await memory.recall_async(
-            bank_id=bank_id,
-            query="Who works where?",
-            budget=Budget.MID,
-            max_tokens=2000,
-            request_context=request_context,
-        )
-        all_texts = " ".join(r.text.lower() for r in result.results)
-        assert "alice" in all_texts or "google" in all_texts, f"Should have Alice/Google facts, got: {all_texts}"
-
-        doc = await memory.get_document(document_id, bank_id, request_context=request_context)
-        assert doc is not None
-        assert doc["memory_unit_count"] > 0
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_with_user_entities(memory, request_context):
-    """
-    User-provided entities should work correctly with delta retain.
-    """
-    bank_id = f"test_delta_user_entities_{_ts()}"
-    document_id = "user-entity-doc"
-
-    try:
-        content = "The project is going well."
-
-        # v1 with user entities
-        await memory.retain_batch_async(
-            bank_id=bank_id,
-            contents=[{
-                "content": content,
-                "document_id": document_id,
-                "entities": [{"text": "Project Alpha", "type": "PROJECT"}],
-            }],
-            request_context=request_context,
-        )
-
-        pool = await memory._get_pool()
-        async with pool.acquire() as conn:
-            v1_entities = await conn.fetch(
-                "SELECT canonical_name FROM entities WHERE bank_id = $1",
-                bank_id,
-            )
-        v1_names = {e["canonical_name"].lower() for e in v1_entities}
-
-        # v2 with additional entity, same content
-        # Note: same content = delta path (no re-extraction)
-        # The user entities for NEW chunks only get processed
-        v2_content = content + "\n\nThe timeline is on track for Q2 delivery."
-        await memory.retain_batch_async(
-            bank_id=bank_id,
-            contents=[{
-                "content": v2_content,
-                "document_id": document_id,
-                "entities": [
-                    {"text": "Project Alpha", "type": "PROJECT"},
-                    {"text": "Q2 Deadline", "type": "MILESTONE"},
-                ],
-            }],
-            request_context=request_context,
-        )
-
-        # Should have entities from both v1 and v2
-        async with pool.acquire() as conn:
-            v2_entities = await conn.fetch(
-                "SELECT canonical_name FROM entities WHERE bank_id = $1",
-                bank_id,
-            )
-        v2_names = {e["canonical_name"].lower() for e in v2_entities}
-
-        # v1 entities should be preserved
-        assert v1_names.issubset(v2_names), f"v1 entities should be preserved: {v1_names} not in {v2_names}"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_delta_retain_recall_with_chunks(memory, request_context):
-    """
-    After delta retain, recall with include_chunks should return correct chunk data.
-    """
-    bank_id = f"test_delta_recall_chunks_{_ts()}"
-    document_id = "recall-chunks-doc"
-
-    try:
-        content = "Alice is a senior engineer at Google Cloud. She designs distributed systems."
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=content,
-            context="profile",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        # Upsert with same content (delta: no changes)
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=content,
-            context="profile",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        # Recall with chunks
-        result = await memory.recall_async(
-            bank_id=bank_id,
-            query="What does Alice do?",
-            budget=Budget.MID,
-            max_tokens=2000,
-            include_chunks=True,
-            max_chunk_tokens=8192,
-            request_context=request_context,
-        )
-
-        assert len(result.results) > 0, "Should recall facts"
-
-        # Facts with chunk_ids should have corresponding chunks
-        facts_with_chunks = [r for r in result.results if r.chunk_id]
-        if facts_with_chunks and result.chunks:
-            for fact in facts_with_chunks:
-                assert fact.chunk_id in result.chunks, (
-                    f"Chunk {fact.chunk_id} should be in returned chunks"
-                )
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)

hindsight-api-slim/tests/test_entity_resolver.py

@@ -1,114 +0,0 @@
-"""
-Tests for EntityResolver edge cases.
-"""
-
-import uuid
-from datetime import datetime, timezone
-
-import asyncpg
-import pytest
-
-from hindsight_api.engine.entity_resolver import EntityResolver
-from hindsight_api.pg0 import resolve_database_url
-
-# ---------------------------------------------------------------------------
-# Unit tests for discard_pending_stats() — no database required
-# ---------------------------------------------------------------------------
-
-
-@pytest.mark.asyncio
-async def test_discard_pending_stats_clears_both_dicts():
-    """discard_pending_stats() must remove entries for the current task from
-    both _pending_stats and _pending_cooccurrences."""
-    resolver = EntityResolver(pool=None)  # type: ignore[arg-type]
-    key = resolver._task_key()
-
-    resolver._pending_stats[key] = [object()]  # type: ignore[list-item]
-    resolver._pending_cooccurrences[key] = [object()]  # type: ignore[list-item]
-
-    resolver.discard_pending_stats()
-
-    assert key not in resolver._pending_stats
-    assert key not in resolver._pending_cooccurrences
-
-
-@pytest.mark.asyncio
-async def test_discard_pending_stats_is_idempotent():
-    """Calling discard_pending_stats() when nothing is pending must not raise."""
-    resolver = EntityResolver(pool=None)  # type: ignore[arg-type]
-    resolver.discard_pending_stats()
-    resolver.discard_pending_stats()  # second call — still safe
-
-
-@pytest.mark.asyncio
-async def test_discard_pending_stats_does_not_affect_other_task_keys():
-    """discard_pending_stats() must only remove the current task's entries,
-    leaving entries keyed under other task IDs untouched."""
-    resolver = EntityResolver(pool=None)  # type: ignore[arg-type]
-    other_key = -1  # A fake key that can never be a real task id
-
-    resolver._pending_stats[other_key] = [object()]  # type: ignore[list-item]
-    resolver._pending_cooccurrences[other_key] = [object()]  # type: ignore[list-item]
-
-    resolver.discard_pending_stats()  # discards current task's key only
-
-    assert other_key in resolver._pending_stats, "other task's stats must be preserved"
-    assert other_key in resolver._pending_cooccurrences, "other task's cooccurrences must be preserved"
-
-
-@pytest.mark.asyncio
-async def test_resolve_entities_batch_handles_unicode_lower_conflicts(pg0_db_url):
-    """
-    Existing entities with PostgreSQL/Python lowercase mismatches should resolve
-    to the conflicted row instead of leaving a missing entity_id.
-    """
-    resolved_url = await resolve_database_url(pg0_db_url)
-    pool = await asyncpg.create_pool(resolved_url, min_size=1, max_size=2, command_timeout=30)
-    bank_id = f"test-entity-resolver-{uuid.uuid4().hex[:8]}"
-    event_date = datetime(2024, 1, 15, tzinfo=timezone.utc)
-    resolver = EntityResolver(pool=pool, entity_lookup="full")
-
-    try:
-        async with pool.acquire() as conn:
-            existing_entity_id = await conn.fetchval(
-                """
-                INSERT INTO entities (bank_id, canonical_name, first_seen, last_seen, mention_count)
-                VALUES ($1, $2, $3, $3, 1)
-                RETURNING id
-                """,
-                bank_id,
-                "İstanbul",
-                event_date,
-            )
-
-            resolved_ids = await resolver.resolve_entities_batch(
-                bank_id=bank_id,
-                entities_data=[
-                    {
-                        "text": "istanbul",
-                        "nearby_entities": [],
-                        "event_date": event_date,
-                    }
-                ],
-                context="unicode case mismatch",
-                unit_event_date=event_date,
-                conn=conn,
-            )
-
-            entity_rows = await conn.fetch(
-                """
-                SELECT id, canonical_name
-                FROM entities
-                WHERE bank_id = $1
-                ORDER BY canonical_name
-                """,
-                bank_id,
-            )
-
-        assert resolved_ids == [existing_entity_id]
-        assert len(entity_rows) == 1
-        assert entity_rows[0]["id"] == existing_entity_id
-        assert entity_rows[0]["canonical_name"] == "İstanbul"
-    finally:
-        await pool.execute("DELETE FROM entities WHERE bank_id = $1", bank_id)
-        await pool.close()

hindsight-api-slim/tests/test_entity_resolver_pg_trgm.py

@@ -1,163 +0,0 @@
-"""
-Unit tests for EntityResolver pg_trgm auto-detection (PR #626/#649).
-
-These tests verify:
-1. When entity_lookup="trigram" and pg_trgm IS available, the trigram path is used.
-2. When entity_lookup="trigram" and pg_trgm is NOT available, the resolver falls back
-   to entity_lookup="full" and uses the full-scan path.
-3. The pg_trgm check is only performed once (_pg_trgm_checked flag prevents re-checking).
-4. When entity_lookup="full" from the start, the trgm check is never performed.
-"""
-
-from unittest.mock import AsyncMock, MagicMock, patch
-
-import pytest
-
-from hindsight_api.engine.entity_resolver import EntityResolver
-
-
-def _make_conn(pg_trgm_available: bool) -> MagicMock:
-    """Create a minimal mock asyncpg connection for the pg_trgm availability check."""
-    conn = MagicMock()
-    conn.fetchval = AsyncMock(return_value=pg_trgm_available)
-    conn.fetch = AsyncMock(return_value=[])
-    conn.executemany = AsyncMock()
-    conn.fetchrow = AsyncMock(return_value=None)
-    return conn
-
-
-def _make_resolver(entity_lookup: str = "trigram") -> EntityResolver:
-    """Return an EntityResolver with a None pool (not needed for unit tests)."""
-    return EntityResolver(pool=None, entity_lookup=entity_lookup)  # type: ignore[arg-type]
-
-
-class TestPgTrgmAutoDetection:
-    """Unit tests for pg_trgm detection logic inside _resolve_entities_batch_impl."""
-
-    @pytest.mark.asyncio
-    async def test_falls_back_to_full_when_pg_trgm_unavailable(self):
-        """When pg_trgm is absent the resolver switches to 'full' and calls the full-scan path."""
-        resolver = _make_resolver(entity_lookup="trigram")
-        conn = _make_conn(pg_trgm_available=False)
-
-        with (
-            patch.object(resolver, "_resolve_entities_batch_full", new=AsyncMock(return_value=[])) as mock_full,
-            patch.object(resolver, "_resolve_entities_batch_trigram", new=AsyncMock(return_value=[])) as mock_trgm,
-        ):
-            await resolver._resolve_entities_batch_impl(
-                conn=conn,
-                bank_id="test-bank",
-                entities_data=[],
-                context="",
-                unit_event_date=None,
-            )
-
-        # Trigram path must NOT be called
-        mock_trgm.assert_not_called()
-        # Full-scan path must be called as the fallback
-        mock_full.assert_called_once()
-        # Strategy is permanently downgraded
-        assert resolver.entity_lookup == "full"
-        assert resolver._pg_trgm_checked is True
-
-    @pytest.mark.asyncio
-    async def test_uses_trigram_when_pg_trgm_available(self):
-        """When pg_trgm is present the trigram path is used."""
-        resolver = _make_resolver(entity_lookup="trigram")
-        conn = _make_conn(pg_trgm_available=True)
-
-        with (
-            patch.object(resolver, "_resolve_entities_batch_full", new=AsyncMock(return_value=[])) as mock_full,
-            patch.object(resolver, "_resolve_entities_batch_trigram", new=AsyncMock(return_value=[])) as mock_trgm,
-        ):
-            await resolver._resolve_entities_batch_impl(
-                conn=conn,
-                bank_id="test-bank",
-                entities_data=[],
-                context="",
-                unit_event_date=None,
-            )
-
-        mock_trgm.assert_called_once()
-        mock_full.assert_not_called()
-        assert resolver.entity_lookup == "trigram"
-        assert resolver._pg_trgm_checked is True
-
-    @pytest.mark.asyncio
-    async def test_pg_trgm_check_performed_only_once(self):
-        """The fetchval check is only issued on the first call; subsequent calls skip it."""
-        resolver = _make_resolver(entity_lookup="trigram")
-        conn = _make_conn(pg_trgm_available=True)
-
-        with patch.object(resolver, "_resolve_entities_batch_trigram", new=AsyncMock(return_value=[])):
-            # First call — check is issued
-            await resolver._resolve_entities_batch_impl(
-                conn=conn,
-                bank_id="test-bank",
-                entities_data=[],
-                context="",
-                unit_event_date=None,
-            )
-            # Second call — check must NOT be issued again
-            await resolver._resolve_entities_batch_impl(
-                conn=conn,
-                bank_id="test-bank",
-                entities_data=[],
-                context="",
-                unit_event_date=None,
-            )
-
-        # fetchval (the pg_trgm availability query) should be called exactly once
-        assert conn.fetchval.call_count == 1
-
-    @pytest.mark.asyncio
-    async def test_full_strategy_skips_pg_trgm_check(self):
-        """When entity_lookup='full' from the start, no pg_trgm check is ever issued."""
-        resolver = _make_resolver(entity_lookup="full")
-        conn = _make_conn(pg_trgm_available=False)
-
-        with patch.object(resolver, "_resolve_entities_batch_full", new=AsyncMock(return_value=[])):
-            await resolver._resolve_entities_batch_impl(
-                conn=conn,
-                bank_id="test-bank",
-                entities_data=[],
-                context="",
-                unit_event_date=None,
-            )
-
-        # fetchval should never be called when entity_lookup is already "full"
-        conn.fetchval.assert_not_called()
-
-    @pytest.mark.asyncio
-    async def test_fallback_is_sticky_across_calls(self):
-        """After falling back to 'full', subsequent calls also use the full path."""
-        resolver = _make_resolver(entity_lookup="trigram")
-        conn = _make_conn(pg_trgm_available=False)
-
-        with (
-            patch.object(resolver, "_resolve_entities_batch_full", new=AsyncMock(return_value=[])) as mock_full,
-            patch.object(resolver, "_resolve_entities_batch_trigram", new=AsyncMock(return_value=[])) as mock_trgm,
-        ):
-            # First call triggers the fallback
-            await resolver._resolve_entities_batch_impl(
-                conn=conn,
-                bank_id="b",
-                entities_data=[],
-                context="",
-                unit_event_date=None,
-            )
-            # Second call — _pg_trgm_checked is True so no re-check; entity_lookup=="full"
-            await resolver._resolve_entities_batch_impl(
-                conn=conn,
-                bank_id="b",
-                entities_data=[],
-                context="",
-                unit_event_date=None,
-            )
-
-        # Trigram path is never called
-        mock_trgm.assert_not_called()
-        # Full-scan path is called both times
-        assert mock_full.call_count == 2
-        # pg_trgm check was issued exactly once
-        assert conn.fetchval.call_count == 1

hindsight-api-slim/tests/test_experience_fact_type_passthrough.py

@@ -1,59 +0,0 @@
-"""
-Regression test for experience fact_type preservation.
-
-The LLM extraction layer normalizes raw "assistant" → "experience" early in parsing.
-The subsequent conversion to ExtractedFactType must pass through the already-normalized
-fact_type rather than re-checking for "assistant" (which would remap experience → world).
-
-See: https://github.com/vectorize-io/hindsight/pull/839
-"""
-
-from datetime import datetime, timezone
-from unittest.mock import AsyncMock, patch
-
-import pytest
-
-from hindsight_api.config import _get_raw_config
-from hindsight_api.engine.response_models import TokenUsage
-from hindsight_api.engine.retain.fact_extraction import (
-    Fact,
-    RetainContent,
-    extract_facts_from_contents,
-    extract_facts_from_contents_batch_api,
-)
-
-
-@pytest.mark.asyncio
-async def test_extract_facts_preserves_experience_type():
-    """
-    When extract_facts_from_text returns a Fact with fact_type="experience",
-    extract_facts_from_contents must preserve it (not remap to "world").
-    """
-    contents = [
-        RetainContent(
-            content="I fixed the failing tests after discovering they mocked the wrong interface.",
-            event_date=datetime(2026, 4, 1, tzinfo=timezone.utc),
-            context="assistant work log",
-        )
-    ]
-    extracted_fact = Fact(
-        fact="Fixed the failing tests after discovering they mocked the wrong interface.",
-        fact_type="experience",
-    )
-
-    with patch(
-        "hindsight_api.engine.retain.fact_extraction.extract_facts_from_text",
-        new=AsyncMock(return_value=([extracted_fact], [(contents[0].content, 1)], TokenUsage())),
-    ):
-        facts, _chunks, _usage = await extract_facts_from_contents(
-            contents=contents,
-            llm_config=None,
-            agent_name="TestAgent",
-            config=_get_raw_config(),
-        )
-
-    assert len(facts) == 1
-    assert facts[0].fact_type == "experience", (
-        f"Expected 'experience' but got '{facts[0].fact_type}' — "
-        f"the conversion layer is remapping the already-normalized fact_type"
-    )

hindsight-api-slim/tests/test_fact_extraction_agent_experience.py

@@ -1,131 +0,0 @@
-"""
-Test that first-person agent experiences are classified as 'experience' fact_type,
-not 'world'. This is critical for AI agent systems that store their own operational
-experiences (debugging, code changes, user interactions) separately from world knowledge.
-"""
-from datetime import datetime
-
-import pytest
-
-from hindsight_api import LLMConfig
-from hindsight_api.config import _get_raw_config
-from hindsight_api.engine.retain.fact_extraction import extract_facts_from_text
-
-
-class TestAgentExperienceClassification:
-    """Tests that first-person coding agent experiences get classified as 'experience'."""
-
-    @pytest.mark.asyncio
-    async def test_code_changes_classified_as_experience(self):
-        """First-person code change descriptions should be experience, not world."""
-        text = """
-I changed the return type of the `process_request` function from `dict` to `ResponseModel`.
-After that, I updated the three callers in `api/handlers.py` to destructure the new model fields.
-The type checker was happy after the change but I noticed one test was still using the old dict keys.
-"""
-        llm_config = LLMConfig.from_env()
-        facts, _, _ = await extract_facts_from_text(
-            text=text,
-            event_date=datetime(2025, 3, 28),
-            llm_config=llm_config,
-            agent_name="coding-agent",
-            context="agent work log",
-            config=_get_raw_config(),
-        )
-
-        assert len(facts) > 0, "Should extract at least one fact"
-        world_facts = [f for f in facts if f.fact_type == "world"]
-        experience_facts = [f for f in facts if f.fact_type == "experience"]
-        assert len(experience_facts) > len(world_facts), (
-            f"First-person code changes should be mostly 'experience', "
-            f"got {len(experience_facts)} experience vs {len(world_facts)} world. "
-            f"Facts: {[(f.fact, f.fact_type) for f in facts]}"
-        )
-
-    @pytest.mark.asyncio
-    async def test_debugging_session_classified_as_experience(self):
-        """First-person debugging narrative should be experience, not world."""
-        text = """
-The tests were failing with a ConnectionRefusedError on the Redis integration suite.
-I traced it to the connection pool not being initialized before the first test ran.
-I added a setup fixture that ensures the pool is warmed up, and all 47 tests pass now.
-"""
-        llm_config = LLMConfig.from_env()
-        facts, _, _ = await extract_facts_from_text(
-            text=text,
-            event_date=datetime(2025, 3, 28),
-            llm_config=llm_config,
-            agent_name="coding-agent",
-            context="agent work log",
-            config=_get_raw_config(),
-        )
-
-        assert len(facts) > 0, "Should extract at least one fact"
-        world_facts = [f for f in facts if f.fact_type == "world"]
-        experience_facts = [f for f in facts if f.fact_type == "experience"]
-        assert len(experience_facts) > len(world_facts), (
-            f"First-person debugging should be mostly 'experience', "
-            f"got {len(experience_facts)} experience vs {len(world_facts)} world. "
-            f"Facts: {[(f.fact, f.fact_type) for f in facts]}"
-        )
-
-    @pytest.mark.asyncio
-    async def test_user_interaction_classified_as_experience(self):
-        """Agent describing interactions with the user should be experience."""
-        text = """
-The user asked me to refactor the authentication middleware to support JWT tokens.
-I proposed splitting it into two modules: token_validation.py and session_management.py.
-The user approved my approach and I started with the token validation logic.
-I discovered that the existing tests were mocking the wrong interface, so I had to rewrite them first.
-"""
-        llm_config = LLMConfig.from_env()
-        facts, _, _ = await extract_facts_from_text(
-            text=text,
-            event_date=datetime(2025, 3, 28),
-            llm_config=llm_config,
-            agent_name="coding-agent",
-            context="agent work log",
-            config=_get_raw_config(),
-        )
-
-        assert len(facts) > 0, "Should extract at least one fact"
-        world_facts = [f for f in facts if f.fact_type == "world"]
-        experience_facts = [f for f in facts if f.fact_type == "experience"]
-        assert len(experience_facts) > len(world_facts), (
-            f"Agent-user interactions should be mostly 'experience', "
-            f"got {len(experience_facts)} experience vs {len(world_facts)} world. "
-            f"Facts: {[(f.fact, f.fact_type) for f in facts]}"
-        )
-
-    @pytest.mark.asyncio
-    async def test_mixed_agent_and_world_facts(self):
-        """Mix of agent experiences and world knowledge should be classified correctly."""
-        text = """
-Python 3.12 introduced a new type parameter syntax for generic classes.
-I migrated our codebase from the old TypeVar approach to the new syntax.
-The migration touched 23 files but was mostly mechanical.
-PEP 695 defines the new type statement that makes generics more readable.
-"""
-        llm_config = LLMConfig.from_env()
-        facts, _, _ = await extract_facts_from_text(
-            text=text,
-            event_date=datetime(2025, 3, 28),
-            llm_config=llm_config,
-            agent_name="coding-agent",
-            context="agent work log",
-            config=_get_raw_config(),
-        )
-
-        assert len(facts) > 0, "Should extract at least one fact"
-        world_facts = [f for f in facts if f.fact_type == "world"]
-        experience_facts = [f for f in facts if f.fact_type == "experience"]
-        # Should have both types - world facts about Python 3.12/PEP 695,
-        # experience facts about the migration work
-        assert len(world_facts) >= 1, (
-            f"Should have at least 1 world fact about Python 3.12/PEP 695. "
-            f"Facts: {[(f.fact, f.fact_type) for f in facts]}"
-        )
-        assert len(experience_facts) >= 1, (
-            f"Should have at least 1 experience fact about the migration. "
-            f"Facts: {[(f.fact, f.fact_type) for f in facts]}"
-        )

hindsight-api-slim/tests/test_format_facts_for_prompt.py

@@ -1,65 +0,0 @@
-"""
-Tests for format_facts_for_prompt in think_utils.
-"""
-
-import json
-
-from hindsight_api.engine.response_models import MemoryFact
-from hindsight_api.engine.search.think_utils import format_facts_for_prompt
-
-
-def test_format_facts_includes_temporal_fields():
-    """All temporal fields (occurred_start, occurred_end, mentioned_at) should appear in the JSON."""
-    facts = [
-        MemoryFact(
-            id="fact-1",
-            text="Team offsite in February",
-            fact_type="experience",
-            occurred_start="2024-02-01T00:00:00Z",
-            occurred_end="2024-02-28T23:59:59Z",
-            mentioned_at="2024-03-05T10:00:00Z",
-        )
-    ]
-    result = json.loads(format_facts_for_prompt(facts))
-    assert len(result) == 1
-    assert result[0]["text"] == "Team offsite in February"
-    assert result[0]["occurred_start"] == "2024-02-01T00:00:00Z"
-    assert result[0]["occurred_end"] == "2024-02-28T23:59:59Z"
-    assert result[0]["mentioned_at"] == "2024-03-05T10:00:00Z"
-
-
-def test_format_facts_omits_null_temporal_fields():
-    """Null temporal fields should not appear in the JSON."""
-    facts = [
-        MemoryFact(
-            id="fact-2",
-            text="The sky is blue",
-            fact_type="world",
-        )
-    ]
-    result = json.loads(format_facts_for_prompt(facts))
-    assert len(result) == 1
-    assert "occurred_start" not in result[0]
-    assert "occurred_end" not in result[0]
-    assert "mentioned_at" not in result[0]
-
-
-def test_format_facts_partial_temporal_fields():
-    """Only non-null temporal fields should appear."""
-    facts = [
-        MemoryFact(
-            id="fact-3",
-            text="Meeting happened",
-            fact_type="experience",
-            occurred_start="2024-06-01T09:00:00Z",
-        )
-    ]
-    result = json.loads(format_facts_for_prompt(facts))
-    assert result[0]["occurred_start"] == "2024-06-01T09:00:00Z"
-    assert "occurred_end" not in result[0]
-    assert "mentioned_at" not in result[0]
-
-
-def test_format_facts_empty_list():
-    """Empty list should return '[]'."""
-    assert format_facts_for_prompt([]) == "[]"

hindsight-api-slim/tests/test_gemini_embeddings.py

@@ -1,336 +0,0 @@
-"""
-Tests for Google embeddings implementation (Gemini API + Vertex AI).
-
-These tests cover:
-1. Initialization (Gemini API key, Vertex AI with ADC/service account)
-2. Dimension detection via test embedding
-3. Output dimensionality configuration
-4. Encode (single text, multiple texts, batching, empty list, uninitialized)
-5. Provider name and model name normalization
-6. Factory function (create from env, validation errors)
-"""
-
-from typing import Any
-from unittest.mock import MagicMock, patch
-
-import pytest
-
-from hindsight_api.config import (
-    ENV_EMBEDDINGS_GEMINI_API_KEY,
-    ENV_EMBEDDINGS_PROVIDER,
-    HindsightConfig,
-)
-from hindsight_api.engine.embeddings import GeminiEmbeddings, create_embeddings_from_env
-
-
-def _make_mock_embedding(values: list[float]) -> MagicMock:
-    emb = MagicMock()
-    emb.values = values
-    return emb
-
-
-def _make_mock_embed_result(embeddings_data: list[list[float]]) -> MagicMock:
-    result = MagicMock()
-    result.embeddings = [_make_mock_embedding(v) for v in embeddings_data]
-    return result
-
-
-def _make_mock_genai(embed_result: Any = None) -> MagicMock:
-    if embed_result is None:
-        embed_result = _make_mock_embed_result([[0.1] * 768])
-    mock_genai = MagicMock()
-    mock_client = MagicMock()
-    mock_client.models.embed_content = MagicMock(return_value=embed_result)
-    mock_genai.Client = MagicMock(return_value=mock_client)
-    return mock_genai
-
-
-def _make_mock_google_module(mock_genai: MagicMock) -> MagicMock:
-    mod = MagicMock()
-    mod.genai = mock_genai
-    mod.genai.types.EmbedContentConfig = MagicMock(side_effect=lambda **kw: MagicMock(**kw))
-    return mod
-
-
-def _patch_google_import(mock_genai: MagicMock):
-    original_import = __import__
-
-    def mock_import(name, *args, **kwargs):
-        if name == "google":
-            return _make_mock_google_module(mock_genai)
-        if name == "google.genai":
-            return mock_genai
-        return original_import(name, *args, **kwargs)
-
-    return patch("builtins.__import__", side_effect=mock_import)
-
-
-class TestGeminiEmbeddings:
-    """Unit tests for GeminiEmbeddings with mocked google.genai."""
-
-    async def test_initialization_api_key_success(self):
-        """Test successful Gemini API key initialization."""
-        mock_genai = _make_mock_genai()
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key="test-key")
-
-        with _patch_google_import(mock_genai):
-            await emb.initialize()
-
-        assert emb._client is not None
-        assert emb.dimension == 768
-        assert emb.provider_name == "google"
-        assert emb._is_vertexai is False
-        mock_genai.Client.return_value.models.embed_content.assert_called_once()
-
-    async def test_initialization_vertexai_success(self):
-        """Test successful Vertex AI initialization."""
-        mock_genai = _make_mock_genai()
-        emb = GeminiEmbeddings(
-            model="gemini-embedding-001",
-            vertexai_project_id="test-project",
-            vertexai_region="us-central1",
-        )
-
-        with _patch_google_import(mock_genai):
-            await emb.initialize()
-
-        assert emb._client is not None
-        assert emb.dimension == 768
-        assert emb.provider_name == "google"
-        assert emb._is_vertexai is True
-        mock_genai.Client.assert_called_once_with(
-            vertexai=True,
-            project="test-project",
-            location="us-central1",
-        )
-
-    async def test_initialization_missing_api_key(self):
-        """Test that missing API key raises ValueError when no vertexai_project_id."""
-        mock_genai = _make_mock_genai()
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key=None)
-
-        with _patch_google_import(mock_genai):
-            with pytest.raises(ValueError, match="requires an API key"):
-                await emb.initialize()
-
-    async def test_initialization_vertexai_missing_project_id(self):
-        """Test that Vertex AI mode requires project_id."""
-        mock_genai = _make_mock_genai()
-        emb = GeminiEmbeddings(model="gemini-embedding-001", vertexai_project_id="temp")
-        emb.vertexai_project_id = None  # Simulate misconfiguration
-
-        with _patch_google_import(mock_genai):
-            with pytest.raises(ValueError, match="is required for Vertex AI"):
-                await emb.initialize()
-
-    async def test_initialization_idempotent(self):
-        """Test that calling initialize() twice is a no-op."""
-        mock_genai = _make_mock_genai()
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key="test-key")
-
-        with _patch_google_import(mock_genai):
-            await emb.initialize()
-            first_client = emb._client
-            await emb.initialize()
-            assert emb._client is first_client
-
-    async def test_dimension_detection_via_test_embedding(self):
-        """Test that dimension is detected via a test embedding call."""
-        test_embed = _make_mock_embed_result([[0.5] * 256])
-        mock_genai = _make_mock_genai(embed_result=test_embed)
-        emb = GeminiEmbeddings(model="some-new-model", api_key="test-key")
-
-        with _patch_google_import(mock_genai):
-            await emb.initialize()
-
-        assert emb.dimension == 256
-
-    async def test_output_dimensionality(self):
-        """Test that output_dimensionality is passed via EmbedContentConfig."""
-        test_embed = _make_mock_embed_result([[0.1] * 256])
-        mock_genai = _make_mock_genai(embed_result=test_embed)
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key="test-key", output_dimensionality=256)
-
-        with _patch_google_import(mock_genai):
-            await emb.initialize()
-
-        assert emb.dimension == 256
-        assert emb._embed_config is not None
-        call_kwargs = mock_genai.Client.return_value.models.embed_content.call_args
-        assert "config" in call_kwargs.kwargs
-
-    async def test_no_output_dimensionality(self):
-        """Test that no EmbedContentConfig is built when output_dimensionality is None."""
-        mock_genai = _make_mock_genai()
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key="test-key", output_dimensionality=None)
-
-        with _patch_google_import(mock_genai):
-            await emb.initialize()
-
-        assert emb._embed_config is None
-        call_kwargs = mock_genai.Client.return_value.models.embed_content.call_args
-        assert "config" not in call_kwargs.kwargs
-
-    def test_auto_detect_vertexai(self):
-        """Test that _is_vertexai is auto-detected from vertexai_project_id."""
-        assert GeminiEmbeddings(model="m", api_key="k")._is_vertexai is False
-        assert GeminiEmbeddings(model="m", vertexai_project_id="p")._is_vertexai is True
-
-    def test_encode_single_text(self):
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key="test-key")
-        mock_client = MagicMock()
-        mock_client.models.embed_content = MagicMock(return_value=_make_mock_embed_result([[0.1, 0.2, 0.3]]))
-        emb._client = mock_client
-        emb._dimension = 3
-
-        assert emb.encode(["hello"]) == [[0.1, 0.2, 0.3]]
-
-    def test_encode_multiple_texts(self):
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key="test-key")
-        mock_client = MagicMock()
-        mock_client.models.embed_content = MagicMock(
-            return_value=_make_mock_embed_result([[0.1, 0.2], [0.3, 0.4], [0.5, 0.6]])
-        )
-        emb._client = mock_client
-        emb._dimension = 2
-
-        result = emb.encode(["a", "b", "c"])
-        assert len(result) == 3
-        assert result[1] == [0.3, 0.4]
-
-    def test_encode_batching(self):
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key="test-key", batch_size=2)
-        mock_client = MagicMock()
-        mock_client.models.embed_content = MagicMock(
-            side_effect=[_make_mock_embed_result([[0.1], [0.2]]), _make_mock_embed_result([[0.3]])]
-        )
-        emb._client = mock_client
-        emb._dimension = 1
-
-        assert emb.encode(["a", "b", "c"]) == [[0.1], [0.2], [0.3]]
-        assert mock_client.models.embed_content.call_count == 2
-
-    def test_encode_passes_config(self):
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key="test-key")
-        mock_client = MagicMock()
-        mock_client.models.embed_content = MagicMock(return_value=_make_mock_embed_result([[0.1, 0.2]]))
-        emb._client = mock_client
-        emb._dimension = 2
-        emb._embed_config = MagicMock()
-
-        emb.encode(["hello"])
-        assert mock_client.models.embed_content.call_args.kwargs["config"] is emb._embed_config
-
-    def test_encode_empty_list(self):
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key="test-key")
-        emb._client = MagicMock()
-        emb._dimension = 768
-        assert emb.encode([]) == []
-
-    def test_encode_before_initialization(self):
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key="test-key")
-        with pytest.raises(RuntimeError, match="not initialized"):
-            emb.encode(["test"])
-
-    def test_dimension_before_initialization(self):
-        emb = GeminiEmbeddings(model="gemini-embedding-001", api_key="test-key")
-        with pytest.raises(RuntimeError, match="not initialized"):
-            _ = emb.dimension
-
-    def test_provider_name_always_google(self):
-        assert GeminiEmbeddings(model="m", api_key="k").provider_name == "google"
-        assert GeminiEmbeddings(model="m", vertexai_project_id="p").provider_name == "google"
-
-    def test_vertexai_strips_google_prefix(self):
-        mock_genai = _make_mock_genai()
-        emb = GeminiEmbeddings(model="google/gemini-embedding-001", vertexai_project_id="test-project")
-        emb._init_vertexai(mock_genai)
-        assert emb.model == "gemini-embedding-001"
-
-    def test_default_region(self):
-        emb = GeminiEmbeddings(model="m", vertexai_project_id="proj")
-        assert emb.vertexai_region == "us-central1"
-
-    def test_custom_region(self):
-        emb = GeminiEmbeddings(model="m", vertexai_project_id="proj", vertexai_region="europe-west1")
-        assert emb.vertexai_region == "europe-west1"
-
-
-class TestGeminiEmbeddingsFactory:
-    """Tests for create_embeddings_from_env() with 'google' provider."""
-
-    def _make_config(self, **overrides) -> HindsightConfig:
-        from dataclasses import fields
-
-        defaults = {}
-        for f in fields(HindsightConfig):
-            if f.type == "str":
-                defaults[f.name] = ""
-            elif f.type == "str | None":
-                defaults[f.name] = None
-            elif f.type == "int":
-                defaults[f.name] = 0
-            elif f.type == "int | None":
-                defaults[f.name] = None
-            elif f.type == "float":
-                defaults[f.name] = 0.0
-            elif f.type == "float | None":
-                defaults[f.name] = None
-            elif f.type == "bool":
-                defaults[f.name] = False
-            elif f.type == "list | None":
-                defaults[f.name] = None
-            else:
-                defaults[f.name] = None
-
-        defaults["embeddings_provider"] = "google"
-        defaults["embeddings_gemini_api_key"] = "test-key"
-        defaults["embeddings_gemini_model"] = "gemini-embedding-001"
-        defaults["embeddings_gemini_output_dimensionality"] = 768
-        defaults["embeddings_vertexai_project_id"] = None
-        defaults["embeddings_vertexai_region"] = None
-        defaults["embeddings_vertexai_service_account_key"] = None
-
-        defaults.update(overrides)
-        return HindsightConfig(**defaults)
-
-    def test_create_with_api_key(self):
-        config = self._make_config()
-        with patch("hindsight_api.config.get_config", return_value=config):
-            emb = create_embeddings_from_env()
-        assert isinstance(emb, GeminiEmbeddings)
-        assert emb.provider_name == "google"
-        assert emb.api_key == "test-key"
-        assert emb._is_vertexai is False
-
-    def test_create_with_vertexai(self):
-        config = self._make_config(
-            embeddings_gemini_api_key=None,
-            embeddings_vertexai_project_id="my-project",
-            embeddings_vertexai_region="us-east1",
-        )
-        with patch("hindsight_api.config.get_config", return_value=config):
-            emb = create_embeddings_from_env()
-        assert isinstance(emb, GeminiEmbeddings)
-        assert emb._is_vertexai is True
-        assert emb.api_key is None
-        assert emb.vertexai_project_id == "my-project"
-
-    def test_create_missing_all_credentials(self):
-        config = self._make_config(embeddings_gemini_api_key=None, embeddings_vertexai_project_id=None)
-        with patch("hindsight_api.config.get_config", return_value=config):
-            with pytest.raises(ValueError, match="is required"):
-                create_embeddings_from_env()
-
-    def test_vertexai_takes_priority(self):
-        config = self._make_config(embeddings_gemini_api_key="key", embeddings_vertexai_project_id="proj")
-        with patch("hindsight_api.config.get_config", return_value=config):
-            emb = create_embeddings_from_env()
-        assert emb._is_vertexai is True
-        assert emb.api_key is None
-
-    def test_create_with_custom_dimensionality(self):
-        config = self._make_config(embeddings_gemini_output_dimensionality=256)
-        with patch("hindsight_api.config.get_config", return_value=config):
-            emb = create_embeddings_from_env()
-        assert emb.output_dimensionality == 256

hindsight-api-slim/tests/test_google_cross_encoder.py

@@ -1,275 +0,0 @@
-"""
-Tests for Google Discovery Engine cross-encoder (Ranking REST API).
-
-These tests cover:
-1. Initialization (service account, ADC, missing project_id)
-2. Predict (single query, multiple queries, batching, empty pairs, uninitialized)
-3. Provider name
-4. Factory function (create from env, validation errors)
-"""
-
-from unittest.mock import MagicMock, patch
-
-import httpx
-import pytest
-
-from hindsight_api.config import (
-    ENV_RERANKER_GOOGLE_PROJECT_ID,
-    ENV_RERANKER_PROVIDER,
-    HindsightConfig,
-)
-from hindsight_api.engine.cross_encoder import GoogleCrossEncoder, create_cross_encoder_from_env
-
-
-def _make_rank_response(records: list[tuple[str, float]]) -> dict:
-    """Build a JSON response matching the Discovery Engine REST API format."""
-    return {"records": [{"id": rid, "score": score} for rid, score in records]}
-
-
-def _make_mock_httpx_client(responses: list[dict] | None = None) -> MagicMock:
-    """Create a mock httpx.Client that returns predefined responses."""
-    mock_client = MagicMock(spec=httpx.Client)
-    if responses:
-        side_effects = []
-        for resp_json in responses:
-            mock_resp = MagicMock(spec=httpx.Response)
-            mock_resp.json.return_value = resp_json
-            mock_resp.raise_for_status.return_value = None
-            side_effects.append(mock_resp)
-        mock_client.post.side_effect = side_effects
-    return mock_client
-
-
-def _make_mock_credentials() -> MagicMock:
-    """Create mock credentials with a valid token."""
-    creds = MagicMock()
-    creds.valid = True
-    creds.token = "mock-token"
-    return creds
-
-
-class TestGoogleCrossEncoder:
-    """Unit tests for GoogleCrossEncoder with mocked httpx + google-auth."""
-
-    async def test_initialization_adc_success(self):
-        """Test successful initialization with ADC (no service account key)."""
-        mock_creds = _make_mock_credentials()
-
-        encoder = GoogleCrossEncoder(project_id="test-project")
-
-        with patch("google.auth.default", return_value=(mock_creds, "test-project")):
-            await encoder.initialize()
-
-        assert encoder._client is not None
-        assert encoder._credentials is mock_creds
-        assert encoder.provider_name == "google"
-        assert "test-project" in encoder._rank_url
-
-    async def test_initialization_service_account(self):
-        """Test initialization with service account key."""
-        mock_creds = _make_mock_credentials()
-
-        encoder = GoogleCrossEncoder(
-            project_id="test-project",
-            service_account_key="/path/to/key.json",
-        )
-
-        with patch(
-            "google.oauth2.service_account.Credentials.from_service_account_file",
-            return_value=mock_creds,
-        ):
-            await encoder.initialize()
-
-        assert encoder._client is not None
-        assert encoder._credentials is mock_creds
-
-    async def test_initialization_idempotent(self):
-        """Test that calling initialize() twice is a no-op."""
-        mock_creds = _make_mock_credentials()
-        encoder = GoogleCrossEncoder(project_id="test-project")
-
-        with patch("google.auth.default", return_value=(mock_creds, "test-project")):
-            await encoder.initialize()
-            first_client = encoder._client
-            await encoder.initialize()
-            assert encoder._client is first_client
-
-    async def test_predict_single_query(self):
-        """Test prediction with a single query and multiple documents."""
-        mock_creds = _make_mock_credentials()
-        mock_client = _make_mock_httpx_client([
-            _make_rank_response([("1", 0.95), ("0", 0.30)]),
-        ])
-
-        encoder = GoogleCrossEncoder(project_id="test-project")
-        with patch("google.auth.default", return_value=(mock_creds, "p")):
-            await encoder.initialize()
-        encoder._client = mock_client
-
-        scores = await encoder.predict([
-            ("What is AI?", "AI is artificial intelligence"),
-            ("What is AI?", "The sky is blue"),
-        ])
-
-        assert len(scores) == 2
-        assert scores[0] == 0.30  # id="0" -> index 0
-        assert scores[1] == 0.95  # id="1" -> index 1
-        mock_client.post.assert_called_once()
-
-    async def test_predict_multiple_queries(self):
-        """Test prediction with multiple distinct queries."""
-        mock_creds = _make_mock_credentials()
-        mock_client = _make_mock_httpx_client([
-            _make_rank_response([("0", 0.9), ("1", 0.1)]),
-            _make_rank_response([("0", 0.8)]),
-        ])
-
-        encoder = GoogleCrossEncoder(project_id="test-project")
-        with patch("google.auth.default", return_value=(mock_creds, "p")):
-            await encoder.initialize()
-        encoder._client = mock_client
-
-        scores = await encoder.predict([
-            ("Query A", "Doc A1"),
-            ("Query A", "Doc A2"),
-            ("Query B", "Doc B1"),
-        ])
-
-        assert len(scores) == 3
-        assert scores[0] == 0.9
-        assert scores[1] == 0.1
-        assert scores[2] == 0.8
-        assert mock_client.post.call_count == 2
-
-    async def test_predict_empty_pairs(self):
-        """Test that empty pairs returns empty list."""
-        mock_creds = _make_mock_credentials()
-        encoder = GoogleCrossEncoder(project_id="test-project")
-
-        with patch("google.auth.default", return_value=(mock_creds, "p")):
-            await encoder.initialize()
-
-        scores = await encoder.predict([])
-        assert scores == []
-
-    async def test_predict_not_initialized(self):
-        """Test that predict raises if not initialized."""
-        encoder = GoogleCrossEncoder(project_id="test-project")
-        with pytest.raises(RuntimeError, match="not initialized"):
-            await encoder.predict([("q", "d")])
-
-    async def test_predict_batching(self):
-        """Test that >200 records are split into batches."""
-        mock_creds = _make_mock_credentials()
-        mock_client = _make_mock_httpx_client([
-            _make_rank_response([(str(i), 0.5) for i in range(200)]),
-            _make_rank_response([(str(i), 0.3) for i in range(50)]),
-        ])
-
-        encoder = GoogleCrossEncoder(project_id="test-project")
-        with patch("google.auth.default", return_value=(mock_creds, "p")):
-            await encoder.initialize()
-        encoder._client = mock_client
-
-        pairs = [("same query", f"doc {i}") for i in range(250)]
-        scores = await encoder.predict(pairs)
-
-        assert len(scores) == 250
-        assert mock_client.post.call_count == 2
-
-    async def test_auth_header_sent(self):
-        """Test that Authorization header is sent with requests."""
-        mock_creds = _make_mock_credentials()
-        mock_creds.token = "test-bearer-token"
-        mock_client = _make_mock_httpx_client([
-            _make_rank_response([("0", 0.9)]),
-        ])
-
-        encoder = GoogleCrossEncoder(project_id="test-project")
-        with patch("google.auth.default", return_value=(mock_creds, "p")):
-            await encoder.initialize()
-        encoder._client = mock_client
-
-        await encoder.predict([("q", "d")])
-
-        call_kwargs = mock_client.post.call_args
-        assert call_kwargs.kwargs["headers"]["Authorization"] == "Bearer test-bearer-token"
-
-    def test_provider_name(self):
-        assert GoogleCrossEncoder(project_id="p").provider_name == "google"
-
-    def test_default_model(self):
-        encoder = GoogleCrossEncoder(project_id="p")
-        assert encoder.model == "semantic-ranker-default-004"
-
-    def test_custom_model(self):
-        encoder = GoogleCrossEncoder(project_id="p", model="semantic-ranker-fast-004")
-        assert encoder.model == "semantic-ranker-fast-004"
-
-    def test_default_location(self):
-        encoder = GoogleCrossEncoder(project_id="p")
-        assert encoder.location == "global"
-
-
-class TestGoogleCrossEncoderFactory:
-    """Tests for create_cross_encoder_from_env() with 'google' provider."""
-
-    def _make_config(self, **overrides) -> HindsightConfig:
-        from dataclasses import fields
-
-        defaults = {}
-        for f in fields(HindsightConfig):
-            if f.type == "str":
-                defaults[f.name] = ""
-            elif f.type == "str | None":
-                defaults[f.name] = None
-            elif f.type == "int":
-                defaults[f.name] = 0
-            elif f.type == "int | None":
-                defaults[f.name] = None
-            elif f.type == "float":
-                defaults[f.name] = 0.0
-            elif f.type == "float | None":
-                defaults[f.name] = None
-            elif f.type == "bool":
-                defaults[f.name] = False
-            elif f.type == "list | None":
-                defaults[f.name] = None
-            else:
-                defaults[f.name] = None
-
-        defaults["reranker_provider"] = "google"
-        defaults["reranker_google_model"] = "semantic-ranker-default-004"
-        defaults["reranker_google_project_id"] = "test-project"
-        defaults["reranker_google_service_account_key"] = None
-
-        defaults.update(overrides)
-        return HindsightConfig(**defaults)
-
-    def test_create_with_project_id(self):
-        config = self._make_config()
-        with patch("hindsight_api.config.get_config", return_value=config):
-            encoder = create_cross_encoder_from_env()
-        assert isinstance(encoder, GoogleCrossEncoder)
-        assert encoder.provider_name == "google"
-        assert encoder.project_id == "test-project"
-        assert encoder.service_account_key is None
-
-    def test_create_with_service_account(self):
-        config = self._make_config(reranker_google_service_account_key="/path/to/key.json")
-        with patch("hindsight_api.config.get_config", return_value=config):
-            encoder = create_cross_encoder_from_env()
-        assert isinstance(encoder, GoogleCrossEncoder)
-        assert encoder.service_account_key == "/path/to/key.json"
-
-    def test_create_missing_project_id(self):
-        config = self._make_config(reranker_google_project_id=None)
-        with patch("hindsight_api.config.get_config", return_value=config):
-            with pytest.raises(ValueError, match="is required"):
-                create_cross_encoder_from_env()
-
-    def test_create_with_custom_model(self):
-        config = self._make_config(reranker_google_model="semantic-ranker-fast-004")
-        with patch("hindsight_api.config.get_config", return_value=config):
-            encoder = create_cross_encoder_from_env()
-        assert encoder.model == "semantic-ranker-fast-004"

hindsight-api-slim/tests/test_graph_entity_fanout_cap.py

@@ -1,211 +0,0 @@
-"""
-Tests for LATERAL entity fanout cap in graph expansion.
-
-Verifies that the per-entity LIMIT in _expand_combined prevents high-fanout
-entities from exploding the self-join, while still returning entity-based
-graph results.
-"""
-
-import asyncio
-from datetime import datetime, timezone
-
-import pytest
-
-
-@pytest.mark.asyncio
-async def test_high_fanout_entity_returns_results(memory, request_context):
-    """
-    A high-fanout entity (appearing in many facts) should still produce
-    graph retrieval results — the LATERAL cap limits rows per entity but
-    does not drop the entity entirely.
-    """
-    bank_id = f"test_fanout_cap_{datetime.now(timezone.utc).timestamp()}"
-
-    try:
-        # Create many facts sharing one common entity ("Acme Corp") plus
-        # a few with a unique entity so we can query for the unique one
-        # and verify graph expansion finds siblings via "Acme Corp".
-        contents = [
-            # Target: unique entity "Zara" shares "Acme Corp" with the rest
-            {
-                "content": "Zara joined Acme Corp as a senior engineer last month",
-                "context": "hr update",
-                "entities": [{"text": "Zara"}, {"text": "Acme Corp"}],
-            },
-        ]
-        # Add many facts that all share "Acme Corp" — creates a high-fanout entity
-        for i in range(60):
-            contents.append(
-                {
-                    "content": f"Employee {i} completed onboarding at Acme Corp in department {i % 5}",
-                    "context": "hr update",
-                    "entities": [{"text": f"Employee {i}"}, {"text": "Acme Corp"}],
-                }
-            )
-
-        await memory.retain_batch_async(
-            bank_id=bank_id,
-            contents=contents,
-            request_context=request_context,
-        )
-
-        from hindsight_api.engine.memory_engine import Budget
-
-        # Query for "Zara" — semantic search finds Zara's fact as a seed,
-        # then graph expansion should find other Acme Corp facts via the
-        # shared entity, even though "Acme Corp" has 60+ mentions.
-        result = await memory.recall_async(
-            bank_id=bank_id,
-            query="Zara",
-            budget=Budget.HIGH,
-            max_tokens=4096,
-            enable_trace=True,
-            request_context=request_context,
-            _quiet=True,
-        )
-
-        assert result.results is not None
-        assert len(result.results) > 0
-
-        # Verify graph retrieval ran and found results
-        retrieval_results = result.trace.get("retrieval_results", [])
-        graph_results = [r for r in retrieval_results if r.get("method_name") == "graph"]
-        assert len(graph_results) > 0, "Graph retrieval should have run"
-
-        # At least one graph result should contain Acme Corp content
-        # (found via shared entity, not just semantic similarity)
-        all_texts = [r.text for r in result.results]
-        acme_found = any("Acme Corp" in t for t in all_texts)
-        assert acme_found, "Should find Acme Corp facts via entity graph expansion"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_entity_expansion_timeout_fallback(memory, request_context):
-    """
-    When graph_expansion_timeout is set very low, entity expansion should
-    time out gracefully and fall back to semantic+causal links only,
-    rather than failing the entire recall.
-    """
-    bank_id = f"test_timeout_fallback_{datetime.now(timezone.utc).timestamp()}"
-
-    try:
-        await memory.retain_batch_async(
-            bank_id=bank_id,
-            contents=[
-                {
-                    "content": "Alice works on the backend API at TechCorp",
-                    "context": "team info",
-                    "entities": [{"text": "Alice"}, {"text": "TechCorp"}],
-                },
-                {
-                    "content": "Bob maintains the frontend at TechCorp",
-                    "context": "team info",
-                    "entities": [{"text": "Bob"}, {"text": "TechCorp"}],
-                },
-            ],
-            request_context=request_context,
-        )
-
-        from hindsight_api.config import _get_raw_config
-        from hindsight_api.engine.memory_engine import Budget
-
-        config = _get_raw_config()
-        original_timeout = config.link_expansion_timeout
-
-        try:
-            # Set an impossibly low timeout to force the fallback path
-            config.link_expansion_timeout = 0.0001
-
-            result = await memory.recall_async(
-                bank_id=bank_id,
-                query="Alice",
-                budget=Budget.MID,
-                max_tokens=2048,
-                enable_trace=True,
-                request_context=request_context,
-                _quiet=True,
-            )
-
-            # Recall should succeed even when entity expansion times out
-            assert result.results is not None
-            assert len(result.results) > 0
-
-            # Alice should still be found via semantic search
-            result_texts = [r.text for r in result.results]
-            alice_found = any("Alice" in t for t in result_texts)
-            assert alice_found, "Should find Alice via semantic search despite graph timeout"
-        finally:
-            config.link_expansion_timeout = original_timeout
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_per_entity_limit_caps_expansion(memory, request_context):
-    """
-    With graph_per_entity_limit set to a small value, entity expansion should
-    still work but return fewer results from high-fanout entities.
-    """
-    bank_id = f"test_per_entity_limit_{datetime.now(timezone.utc).timestamp()}"
-
-    try:
-        # Create facts with a shared entity
-        contents = [
-            {
-                "content": "Lead engineer Dana oversees the Widgets project at MegaCorp",
-                "context": "project info",
-                "entities": [{"text": "Dana"}, {"text": "MegaCorp"}],
-            },
-        ]
-        for i in range(30):
-            contents.append(
-                {
-                    "content": f"MegaCorp hired contractor {i} for the Q4 push",
-                    "context": "hiring info",
-                    "entities": [{"text": f"Contractor {i}"}, {"text": "MegaCorp"}],
-                }
-            )
-
-        await memory.retain_batch_async(
-            bank_id=bank_id,
-            contents=contents,
-            request_context=request_context,
-        )
-
-        from hindsight_api.config import _get_raw_config
-        from hindsight_api.engine.memory_engine import Budget
-
-        config = _get_raw_config()
-        original_limit = config.link_expansion_per_entity_limit
-
-        try:
-            # Set a very small per-entity limit
-            config.link_expansion_per_entity_limit = 5
-
-            result = await memory.recall_async(
-                bank_id=bank_id,
-                query="Dana",
-                budget=Budget.HIGH,
-                max_tokens=4096,
-                enable_trace=True,
-                request_context=request_context,
-                _quiet=True,
-            )
-
-            # Recall should succeed with the cap
-            assert result.results is not None
-            assert len(result.results) > 0
-
-            # Graph retrieval should have run
-            retrieval_results = result.trace.get("retrieval_results", [])
-            graph_results = [r for r in retrieval_results if r.get("method_name") == "graph"]
-            assert len(graph_results) > 0, "Graph retrieval should have run"
-        finally:
-            config.link_expansion_per_entity_limit = original_limit
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)

hindsight-api-slim/tests/test_hnsw_indexes.py

@@ -1,193 +0,0 @@
-"""
-Tests for per-bank vector index lifecycle and UNION ALL retrieval.
-
-Covers:
-- _bank_index_name deterministic naming
-- Per-bank vector indexes created on bank creation (retain_async / ensure_bank_exists)
-- Per-bank vector indexes dropped on bank deletion
-- retrieve_semantic_bm25_combined groups results correctly by fact_type and source
-"""
-import uuid
-from datetime import datetime, timezone
-
-import pytest
-
-from hindsight_api.engine.retain.bank_utils import _BANK_INDEX_FACT_TYPES, _bank_index_name
-
-
-# ---------------------------------------------------------------------------
-# Unit tests — no DB required
-# ---------------------------------------------------------------------------
-
-
-class TestBankIndexName:
-    def test_deterministic(self):
-        uid = "550e8400-e29b-41d4-a716-446655440000"
-        assert _bank_index_name("world", uid) == _bank_index_name("world", uid)
-
-    def test_strips_dashes(self):
-        uid = "550e8400-e29b-41d4-a716-446655440000"
-        name = _bank_index_name("world", uid)
-        # uid16 should be hex chars only
-        assert "-" not in name
-
-    def test_uses_first_16_hex_chars(self):
-        uid = "550e8400-e29b-41d4-a716-446655440000"
-        uid16 = uid.replace("-", "")[:16]  # "550e8400e29b41d4"
-        assert name_ends_with(name=_bank_index_name("world", uid), suffix=uid16)
-
-    def test_suffix_per_fact_type(self):
-        uid = "550e8400-e29b-41d4-a716-446655440000"
-        names = {ft: _bank_index_name(ft, uid) for ft in _BANK_INDEX_FACT_TYPES}
-        # All three names must be distinct
-        assert len(set(names.values())) == 3
-
-    def test_all_fact_types_covered(self):
-        assert set(_BANK_INDEX_FACT_TYPES) == {"world", "experience", "observation"}
-
-    def test_fits_pg_identifier_limit(self):
-        # PostgreSQL max identifier length is 63 chars
-        uid = "f" * 32  # simulated UUID without dashes
-        for ft in _BANK_INDEX_FACT_TYPES:
-            assert len(_bank_index_name(ft, uid)) <= 63
-
-
-def name_ends_with(name: str, suffix: str) -> bool:
-    return name.endswith(suffix)
-
-
-# ---------------------------------------------------------------------------
-# Integration tests — require DB (memory fixture)
-# ---------------------------------------------------------------------------
-
-
-async def _get_bank_vector_indexes(pool, bank_id: str) -> list[str]:
-    """Return index names for memory_units that match the per-bank pattern."""
-    async with pool.acquire() as conn:
-        rows = await conn.fetch(
-            """
-            SELECT indexname
-            FROM pg_indexes
-            WHERE tablename = 'memory_units'
-              AND indexname LIKE 'idx_mu_emb_%'
-              AND indexdef LIKE $1
-            ORDER BY indexname
-            """,
-            f"%bank_id = '{bank_id}'%",
-        )
-    return [row["indexname"] for row in rows]
-
-
-@pytest.mark.asyncio
-async def test_retain_creates_per_bank_vector_indexes(memory, request_context):
-    """retain_async on a new bank must create 3 per-(bank, fact_type) vector indexes."""
-    bank_id = f"test_hnsw_create_{uuid.uuid4().hex[:8]}"
-    try:
-        await memory.retain_async(
-            bank_id=bank_id,
-            content="Alice is a software engineer.",
-            request_context=request_context,
-        )
-        indexes = await _get_bank_vector_indexes(memory._pool, bank_id)
-        assert len(indexes) == 3, f"Expected 3 per-bank vector indexes, got: {indexes}"
-        for ft_short in _BANK_INDEX_FACT_TYPES.values():
-            assert any(ft_short in idx for idx in indexes), (
-                f"Missing index for fact_type short '{ft_short}' in {indexes}"
-            )
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_delete_bank_drops_vector_indexes(memory, request_context):
-    """delete_bank must drop all per-bank vector indexes."""
-    bank_id = f"test_hnsw_drop_{uuid.uuid4().hex[:8]}"
-
-    await memory.retain_async(
-        bank_id=bank_id,
-        content="Bob is a data scientist.",
-        request_context=request_context,
-    )
-    # Verify indexes exist before deletion
-    indexes_before = await _get_bank_vector_indexes(memory._pool, bank_id)
-    assert len(indexes_before) == 3
-
-    await memory.delete_bank(bank_id, request_context=request_context)
-
-    indexes_after = await _get_bank_vector_indexes(memory._pool, bank_id)
-    assert indexes_after == [], f"Indexes should be dropped after bank deletion, got: {indexes_after}"
-
-
-@pytest.mark.asyncio
-async def test_retain_idempotent_bank_creation(memory, request_context):
-    """Retaining into the same bank twice must not error and still have exactly 3 indexes."""
-    bank_id = f"test_hnsw_idem_{uuid.uuid4().hex[:8]}"
-    try:
-        await memory.retain_async(
-            bank_id=bank_id,
-            content="Carol is a product manager.",
-            request_context=request_context,
-        )
-        await memory.retain_async(
-            bank_id=bank_id,
-            content="Carol joined the company in 2022.",
-            request_context=request_context,
-        )
-        indexes = await _get_bank_vector_indexes(memory._pool, bank_id)
-        assert len(indexes) == 3
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_retrieve_semantic_bm25_grouped_by_fact_type(memory, request_context):
-    """
-    retrieve_semantic_bm25_combined must return a dict keyed by fact_type with
-    (semantic_list, bm25_list) tuples.  All returned facts must belong to their
-    declared fact_type.
-    """
-    from hindsight_api.engine.search.retrieval import retrieve_semantic_bm25_combined
-
-    bank_id = f"test_retrieval_{uuid.uuid4().hex[:8]}"
-    try:
-        await memory.retain_async(
-            bank_id=bank_id,
-            content=(
-                "Alice is a software engineer at TechCorp. "
-                "She visited Paris in 2023 for a conference."
-            ),
-            context="background",
-            event_date=datetime(2023, 6, 1, tzinfo=timezone.utc),
-            request_context=request_context,
-        )
-
-        query_emb = memory.embeddings.encode(["software engineer Alice"])
-        query_emb_str = str(query_emb[0])
-
-        fact_types = ["world", "experience"]
-        async with memory._pool.acquire() as conn:
-            results = await retrieve_semantic_bm25_combined(
-                conn=conn,
-                query_emb_str=query_emb_str,
-                query_text="software engineer Alice",
-                bank_id=bank_id,
-                fact_types=fact_types,
-                limit=5,
-            )
-
-        # Must return an entry for every requested fact_type
-        assert set(results.keys()) == set(fact_types)
-
-        for ft, (sem, bm25) in results.items():
-            # Semantic and BM25 lists must be lists
-            assert isinstance(sem, list)
-            assert isinstance(bm25, list)
-            # All semantic results must declare the correct fact_type
-            for r in sem:
-                assert r.fact_type == ft, f"Semantic result has wrong fact_type: {r.fact_type}"
-            # All BM25 results must declare the correct fact_type
-            for r in bm25:
-                assert r.fact_type == ft, f"BM25 result has wrong fact_type: {r.fact_type}"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)

hindsight-api-slim/tests/test_horse_observations.py

@@ -1,353 +0,0 @@
-"""Test observation tracking for a sequence of horse-related memories.
-
-This test retains a series of facts about horses on a farm and inspects
-how observations track the evolving state over time, with full prompt debugging.
-"""
-
-import json
-import uuid
-from dataclasses import dataclass, field
-from typing import Any
-
-import pytest
-
-from hindsight_api.config import _get_raw_config
-from hindsight_api.engine.consolidation import consolidator as consolidator_mod
-from hindsight_api.engine.memory_engine import MemoryEngine
-
-
-@pytest.fixture(autouse=True)
-def enable_observations():
-    """Enable observations for all tests in this module."""
-    config = _get_raw_config()
-    original_value = config.enable_observations
-    config.enable_observations = True
-    yield
-    config.enable_observations = original_value
-
-
-@dataclass
-class _ActionLog:
-    text: str
-    source_fact_ids: list[str] = field(default_factory=list)
-    observation_id: str = ""
-
-
-@dataclass
-class _ConsolidationResponse:
-    creates: list[_ActionLog] = field(default_factory=list)
-    updates: list[_ActionLog] = field(default_factory=list)
-    deletes: list[_ActionLog] = field(default_factory=list)
-
-
-@dataclass
-class _ConsolidationDebugEntry:
-    facts: str
-    observations_text: str
-    response: _ConsolidationResponse
-
-
-# Store prompts/responses for debugging
-_debug_log: list[_ConsolidationDebugEntry] = []
-
-
-def _fact_line(m: dict[str, Any]) -> str:
-    text = f"[{m['id']}] {m['text']}"
-    temporal_parts = []
-    if m.get("occurred_start"):
-        temporal_parts.append(f"occurred_start={m['occurred_start']}")
-    if m.get("occurred_end"):
-        temporal_parts.append(f"occurred_end={m['occurred_end']}")
-    if m.get("mentioned_at"):
-        temporal_parts.append(f"mentioned_at={m['mentioned_at']}")
-    if temporal_parts:
-        text += f" ({', '.join(temporal_parts)})"
-    return text
-
-
-async def _instrumented_consolidate(
-    original_fn: Any,
-    *,
-    llm_config: Any,
-    memories: list[dict[str, Any]],
-    union_observations: Any,
-    union_source_facts: Any,
-    config: Any = None,
-    remaining_observation_slots: int | None = None,
-    max_observations_per_scope: int = -1,
-) -> Any:
-    """Wrapper that captures the prompt and response for debugging."""
-    if union_observations:
-        obs_list = consolidator_mod._build_observations_for_llm(union_observations, union_source_facts)
-        observations_text = json.dumps(obs_list, indent=2)
-    else:
-        observations_text = "[]"
-
-    facts_lines = "\n".join(_fact_line(m) for m in memories)
-
-    result = await original_fn(
-        llm_config=llm_config,
-        memories=memories,
-        union_observations=union_observations,
-        union_source_facts=union_source_facts,
-        config=config,
-        remaining_observation_slots=remaining_observation_slots,
-        max_observations_per_scope=max_observations_per_scope,
-    )
-
-    _debug_log.append(_ConsolidationDebugEntry(
-        facts=facts_lines,
-        observations_text=observations_text,
-        response=_ConsolidationResponse(
-            creates=[_ActionLog(text=c.text, source_fact_ids=c.source_fact_ids) for c in result.creates],
-            updates=[
-                _ActionLog(text=u.text, observation_id=u.observation_id, source_fact_ids=u.source_fact_ids)
-                for u in result.updates
-            ],
-            deletes=[_ActionLog(text="", observation_id=d.observation_id) for d in result.deletes],
-        ),
-    ))
-
-    return result
-
-
-def _print_consolidation_debug(entry: _ConsolidationDebugEntry, index: int) -> None:
-    """Print a single consolidation LLM call for debugging."""
-    print(f"\n  --- LLM Call #{index} ---")
-    print("  FACTS sent to LLM:")
-    for line in entry.facts.split("\n"):
-        print(f"    {line}")
-    print("\n  EXISTING OBSERVATIONS sent to LLM:")
-    obs_data = json.loads(entry.observations_text)
-    if obs_data:
-        for obs in obs_data:
-            src_summary = ""
-            if obs.get("source_memories"):
-                src_texts = [sm["text"] for sm in obs["source_memories"]]
-                src_summary = f" (sources: {src_texts})"
-            print(f"    [{obs['id'][:8]}..] proof={obs.get('proof_count', '?')}: {obs['text']}{src_summary}")
-    else:
-        print("    (none)")
-
-    resp = entry.response
-    print("\n  LLM RESPONSE:")
-    if resp.creates:
-        for c in resp.creates:
-            print(f"    CREATE: \"{c.text}\" (from facts: {[fid[:8] + '..' for fid in c.source_fact_ids]})")
-    if resp.updates:
-        for u in resp.updates:
-            print(
-                f"    UPDATE [{u.observation_id[:8]}..]: \"{u.text}\""
-                f" (from facts: {[fid[:8] + '..' for fid in u.source_fact_ids]})"
-            )
-    if resp.deletes:
-        for d in resp.deletes:
-            print(f"    DELETE [{d.observation_id[:8]}..]")
-    if not resp.creates and not resp.updates and not resp.deletes:
-        print("    (no actions)")
-
-
-def _parse_history(hist: Any) -> list[str]:
-    """Parse observation history from DB (may be list of dicts or JSON strings)."""
-    if not hist:
-        return []
-    parsed = hist if isinstance(hist, list) else json.loads(hist)
-    prev_texts = []
-    for h in parsed:
-        if isinstance(h, str):
-            h = json.loads(h)
-        prev_texts.append(h.get("previous_text", "?"))
-    return prev_texts
-
-
-@pytest.mark.asyncio
-@pytest.mark.flaky(reruns=2, reruns_delay=5)
-async def test_horse_farm_observation_history(memory: MemoryEngine, request_context: Any) -> None:
-    """Retain a sequence of horse facts and inspect how observations evolve."""
-    bank_id = f"test-horses-{uuid.uuid4().hex[:8]}"
-
-    await memory.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-    messages = [
-        "I have a farm.",
-        "I have 2 horses.",
-        "I have a horse named Daisy.",
-        "I have a horse named Buttercup.",
-        "I sold Buttercup.",
-        "I now have 1 horse.",
-        "I have 5 horses on my farm.",
-        "I have a horse named Midnight.",
-        "I have horses named Midnight and Shadow.",
-        "I have horses named Shadow and Twister.",
-        "I am sad to report that Shadow has died.",
-    ]
-
-    # Monkey-patch to intercept consolidation LLM calls
-    _original_consolidate = consolidator_mod._consolidate_batch_with_llm
-
-    async def _patched(**kwargs: Any) -> Any:
-        return await _instrumented_consolidate(_original_consolidate, **kwargs)
-
-    consolidator_mod._consolidate_batch_with_llm = _patched
-    _debug_log.clear()
-
-    try:
-        for i, content in enumerate(messages):
-            print(f"\n{'='*80}")
-            print(f"RETAIN #{i+1}: {content}")
-            print(f"{'='*80}")
-
-            log_start = len(_debug_log)
-
-            await memory.retain_async(
-                bank_id=bank_id,
-                content=content,
-                request_context=request_context,
-            )
-            await memory.wait_for_background_tasks()
-
-            for j, entry in enumerate(_debug_log[log_start:]):
-                _print_consolidation_debug(entry, j + 1)
-
-            # Dump current observations
-            pool = await memory._get_pool()
-            async with pool.acquire() as conn:
-                observations = await conn.fetch(
-                    """
-                    SELECT id, text, proof_count, source_memory_ids, history
-                    FROM memory_units
-                    WHERE bank_id = $1 AND fact_type = 'observation'
-                    ORDER BY created_at
-                    """,
-                    bank_id,
-                )
-
-                print(f"\n  CURRENT OBSERVATIONS ({len(observations)}):")
-                for obs in observations:
-                    prev_texts = _parse_history(obs["history"])
-                    hist_str = f" (was: {' -> '.join(prev_texts)})" if prev_texts else ""
-                    print(f"    [{str(obs['id'])[:8]}..] proof={obs['proof_count']}: {obs['text']}{hist_str}")
-    finally:
-        consolidator_mod._consolidate_batch_with_llm = _original_consolidate
-
-    # Final summary
-    print(f"\n{'='*80}")
-    print("FINAL STATE")
-    print(f"{'='*80}")
-    pool = await memory._get_pool()
-    async with pool.acquire() as conn:
-        observations = await conn.fetch(
-            """
-            SELECT id, text, proof_count, source_memory_ids, history
-            FROM memory_units
-            WHERE bank_id = $1 AND fact_type = 'observation'
-            ORDER BY created_at
-            """,
-            bank_id,
-        )
-        print(f"\nFinal observations ({len(observations)}):")
-        for obs in observations:
-            prev_texts = _parse_history(obs["history"])
-            if prev_texts:
-                chain = prev_texts + [obs["text"]]
-                print(f"  - [proof={obs['proof_count']}] {obs['text']}")
-                print(f"    evolution: {' -> '.join(chain)}")
-            else:
-                print(f"  - [proof={obs['proof_count']}] {obs['text']}")
-
-    # Create a mental model to synthesize the observations
-    print(f"\n{'='*80}")
-    print("MENTAL MODEL")
-    print(f"{'='*80}")
-
-    # Patch reflect _execute_tool to log tool inputs/outputs
-    from hindsight_api.engine.reflect import agent as reflect_agent_mod
-
-    _original_execute = reflect_agent_mod._execute_tool
-
-    async def _logging_execute(tool_name: str, args: dict[str, Any], *a: Any, **kw: Any) -> dict[str, Any]:
-        result = await _original_execute(tool_name, args, *a, **kw)
-        normalized = reflect_agent_mod._normalize_tool_name(tool_name)
-        print(f"\n  [REFLECT TOOL] {normalized}(args={args})")
-        if isinstance(result, dict):
-            if "observations" in result:
-                print(f"    Observations returned ({result.get('count', '?')}, freshness={result.get('freshness', '?')}):")
-                for obs in result.get("observations", []):
-                    print(f"      - [proof={obs.get('proof_count', '?')}] {obs.get('text', '?')}")
-            if "memories" in result:
-                print(f"    Memories returned ({result.get('count', '?')}):")
-                for mem in result.get("memories", []):
-                    chunk = mem.get("chunk_text", "")
-                    chunk_preview = f" | chunk: {chunk[:80]}..." if chunk else ""
-                    print(f"      - [{mem.get('fact_type', '?')}] {mem.get('text', '?')}{chunk_preview}")
-            if "mental_models" in result:
-                print(f"    Mental models returned ({result.get('count', '?')}):")
-                for mm_item in result.get("mental_models", []):
-                    print(f"      - {mm_item.get('name', '?')}: {str(mm_item.get('content', '?'))[:120]}")
-            if "error" in result:
-                print(f"    ERROR: {result['error']}")
-        return result
-
-    reflect_agent_mod._execute_tool = _logging_execute
-
-    source_query = (
-        "Produce a structured summary of all animals on the farm. Include:\n"
-        "1. A chronological timeline of events (acquisitions, sales, deaths) with dates\n"
-        "2. The list of all known horse names and their current status (alive, sold, died)\n"
-        "3. The current number of horses on the farm, accounting for all events\n"
-        "Reason step by step from the facts. If a horse died or was sold, subtract from the count."
-    )
-
-    try:
-        mm = await memory.create_mental_model(
-            bank_id=bank_id,
-            name="Farm Animals",
-            source_query=source_query,
-            content="(initial — awaiting refresh)",
-            request_context=request_context,
-        )
-
-        refreshed = await memory.refresh_mental_model(
-            bank_id=bank_id,
-            mental_model_id=mm["id"],
-            request_context=request_context,
-        )
-        content = refreshed["content"]
-    finally:
-        reflect_agent_mod._execute_tool = _original_execute
-    print(f"\nMental model content:\n{content}")
-
-    reflect_resp = refreshed.get("reflect_response")
-    if reflect_resp and isinstance(reflect_resp, str) and reflect_resp.strip():
-        try:
-            reflect_resp = json.loads(reflect_resp)
-        except json.JSONDecodeError:
-            reflect_resp = None
-    if isinstance(reflect_resp, dict):
-        based_on = reflect_resp.get("based_on", [])
-        if based_on:
-            print("\nBased on:")
-            for item in based_on:
-                if isinstance(item, str):
-                    try:
-                        item = json.loads(item)
-                    except json.JSONDecodeError:
-                        continue
-                print(f"  - [{item.get('fact_type', '?')}] {item.get('text', '?')}")
-
-    # Verify the mental model captures key facts
-    content_lower = content.lower()
-
-    for name in ["daisy", "buttercup", "midnight", "shadow", "twister"]:
-        assert name in content_lower, f"Mental model should mention {name}. Got:\n{content}"
-
-    assert "sold" in content_lower or "sale" in content_lower, (
-        f"Mental model should mention Buttercup was sold. Got:\n{content}"
-    )
-
-    assert "died" in content_lower or "passed" in content_lower or "death" in content_lower, (
-        f"Mental model should mention Shadow's death. Got:\n{content}"
-    )
-
-    # Cleanup
-    await memory.delete_bank(bank_id, request_context=request_context)

hindsight-api-slim/tests/test_integrity_violation_not_retried.py

@@ -1,184 +0,0 @@
-"""
-Regression tests for vectorize-io/hindsight#980.
-
-Deterministic Postgres integrity-constraint violations (UniqueViolationError,
-ForeignKeyViolationError, CheckViolationError, NotNullViolationError,
-ExclusionViolationError) must NOT be retried by the worker — they will never
-succeed on retry, and retrying just burns worker capacity for ~3 minutes
-(3 retries × 60s) before finally giving up.
-
-These tests verify that ``MemoryEngine.execute_task`` classifies
-``asyncpg.exceptions.IntegrityConstraintViolationError`` as non-retryable
-and marks the operation as failed on the first occurrence.
-"""
-
-import json
-import uuid
-from unittest.mock import AsyncMock, patch
-
-import asyncpg
-import pytest
-
-from hindsight_api.worker.exceptions import RetryTaskAt
-
-
-async def _ensure_bank(pool, bank_id: str) -> None:
-    """Upsert a minimal bank row so FK on async_operations passes."""
-    await pool.execute(
-        "INSERT INTO banks (bank_id, name) VALUES ($1, $2) ON CONFLICT DO NOTHING",
-        bank_id,
-        bank_id,
-    )
-
-
-async def _create_pending_operation(pool, bank_id: str, operation_id: uuid.UUID) -> None:
-    """Insert a pending batch_retain operation row for the test."""
-    payload = json.dumps(
-        {
-            "type": "batch_retain",
-            "operation_id": str(operation_id),
-            "bank_id": bank_id,
-            "contents": [{"content": "test", "document_id": "doc-1"}],
-        }
-    )
-    await pool.execute(
-        """
-        INSERT INTO async_operations (operation_id, bank_id, operation_type, status, task_payload)
-        VALUES ($1, $2, 'retain', 'pending', $3::jsonb)
-        """,
-        operation_id,
-        bank_id,
-        payload,
-    )
-
-
-@pytest.mark.asyncio
-async def test_unique_violation_marks_failed_without_retry(memory):
-    """
-    UniqueViolationError must mark the operation as failed immediately, not
-    raise RetryTaskAt. This is the primary symptom from #977: re-submitting
-    retain caused PK collisions that the poller retried ~3 times before
-    giving up. With #980's fix, the first collision fails the task.
-    """
-    bank_id = f"test-worker-{uuid.uuid4().hex[:8]}"
-    operation_id = uuid.uuid4()
-
-    pool = await memory._get_pool()
-    await _ensure_bank(pool, bank_id)
-    await _create_pending_operation(pool, bank_id, operation_id)
-
-    # Synthesize a real asyncpg UniqueViolationError the way the server would
-    # raise it (matches the error observed in the bug report's logs).
-    unique_violation = asyncpg.exceptions.UniqueViolationError(
-        'duplicate key value violates unique constraint "pk_chunks"'
-    )
-
-    task_dict = {
-        "type": "batch_retain",
-        "operation_id": str(operation_id),
-        "bank_id": bank_id,
-        "contents": [{"content": "test", "document_id": "doc-1"}],
-    }
-
-    # Force _handle_batch_retain to raise the integrity error, isolating the
-    # execute_task exception-classification path.
-    with patch.object(memory, "_handle_batch_retain", side_effect=unique_violation):
-        # Must not raise RetryTaskAt — the whole point of the fix.
-        try:
-            await memory.execute_task(task_dict)
-        except RetryTaskAt as exc:
-            pytest.fail(
-                f"IntegrityConstraintViolationError must not be retried, but execute_task raised {exc!r}"
-            )
-
-    # The operation must be marked 'failed' (not left pending / retrying).
-    row = await pool.fetchrow(
-        "SELECT status, error_message FROM async_operations WHERE operation_id = $1",
-        operation_id,
-    )
-    assert row is not None, "Operation row disappeared"
-    assert row["status"] == "failed", (
-        f"Expected status='failed' after integrity violation, got {row['status']!r}"
-    )
-    assert row["error_message"] is not None
-    assert "pk_chunks" in row["error_message"]
-
-    # Cleanup
-    await pool.execute("DELETE FROM async_operations WHERE operation_id = $1", operation_id)
-    await pool.execute("DELETE FROM banks WHERE bank_id = $1", bank_id)
-
-
-@pytest.mark.asyncio
-async def test_foreign_key_violation_also_not_retried(memory):
-    """
-    All subclasses of IntegrityConstraintViolationError are non-retryable —
-    verify ForeignKeyViolationError is classified the same way as
-    UniqueViolationError.
-    """
-    bank_id = f"test-worker-{uuid.uuid4().hex[:8]}"
-    operation_id = uuid.uuid4()
-
-    pool = await memory._get_pool()
-    await _ensure_bank(pool, bank_id)
-    await _create_pending_operation(pool, bank_id, operation_id)
-
-    fk_violation = asyncpg.exceptions.ForeignKeyViolationError(
-        "insert or update on table \"memory_units\" violates foreign key constraint \"fk_bank\""
-    )
-
-    task_dict = {
-        "type": "batch_retain",
-        "operation_id": str(operation_id),
-        "bank_id": bank_id,
-        "contents": [{"content": "test", "document_id": "doc-1"}],
-    }
-
-    with patch.object(memory, "_handle_batch_retain", side_effect=fk_violation):
-        try:
-            await memory.execute_task(task_dict)
-        except RetryTaskAt as exc:
-            pytest.fail(
-                f"ForeignKeyViolationError must not be retried, but execute_task raised {exc!r}"
-            )
-
-    row = await pool.fetchrow(
-        "SELECT status FROM async_operations WHERE operation_id = $1",
-        operation_id,
-    )
-    assert row["status"] == "failed"
-
-    await pool.execute("DELETE FROM async_operations WHERE operation_id = $1", operation_id)
-    await pool.execute("DELETE FROM banks WHERE bank_id = $1", bank_id)
-
-
-@pytest.mark.asyncio
-async def test_non_integrity_error_still_retried(memory):
-    """
-    Sanity check: non-integrity errors (network errors, timeouts, value errors)
-    should STILL use the existing retry path — i.e., raise RetryTaskAt when
-    ``_retry_count < 3``. Only integrity violations are the new non-retryable
-    class.
-    """
-    bank_id = f"test-worker-{uuid.uuid4().hex[:8]}"
-    operation_id = uuid.uuid4()
-
-    pool = await memory._get_pool()
-    await _ensure_bank(pool, bank_id)
-    await _create_pending_operation(pool, bank_id, operation_id)
-
-    task_dict = {
-        "type": "batch_retain",
-        "operation_id": str(operation_id),
-        "bank_id": bank_id,
-        "contents": [{"content": "test", "document_id": "doc-1"}],
-        # _retry_count = 0 (first attempt), so the existing retry path should fire.
-    }
-
-    transient_error = RuntimeError("transient connection blip")
-
-    with patch.object(memory, "_handle_batch_retain", side_effect=transient_error):
-        with pytest.raises(RetryTaskAt):
-            await memory.execute_task(task_dict)
-
-    await pool.execute("DELETE FROM async_operations WHERE operation_id = $1", operation_id)
-    await pool.execute("DELETE FROM banks WHERE bank_id = $1", bank_id)

hindsight-api-slim/tests/test_jina_mlx_import_error.py

@@ -1,78 +0,0 @@
-"""
-Regression test for the JinaMLXCrossEncoder import-error handling.
-
-See: https://github.com/vectorize-io/hindsight/issues/994
-
-Before the fix, the bare `except ImportError` around `import mlx_lm` masked
-*any* ImportError raised transitively during mlx_lm's own initialization
-(e.g. transformers 5.x's _LazyModule race producing
-`ImportError: cannot import name 'AutoTokenizer' from 'transformers'`),
-replacing it with a misleading "install mlx" message.
-
-These tests verify:
-1. A transitive ImportError raised from inside mlx_lm surfaces verbatim.
-2. A genuine "package not installed" ImportError still produces the install hint.
-"""
-
-import sys
-import types
-from unittest.mock import patch
-
-import pytest
-
-from hindsight_api.engine.cross_encoder import JinaMLXCrossEncoder
-
-
-def _stub_mlx_modules() -> dict[str, types.ModuleType]:
-    """Stub mlx + mlx.core so `import mlx.core` succeeds even without mlx installed."""
-    import importlib.machinery
-
-    mlx = types.ModuleType("mlx")
-    mlx.__spec__ = importlib.machinery.ModuleSpec("mlx", loader=None)
-    mlx_core = types.ModuleType("mlx.core")
-    mlx_core.__spec__ = importlib.machinery.ModuleSpec("mlx.core", loader=None)
-    mlx.core = mlx_core
-    return {"mlx": mlx, "mlx.core": mlx_core}
-
-
-@pytest.mark.asyncio
-async def test_initialize_surfaces_transitive_import_error():
-    """A transformers-lazy-load-style failure must propagate, not be masked."""
-    encoder = JinaMLXCrossEncoder()
-
-    real_import = __import__
-
-    def fake_import(name, *args, **kwargs):
-        if name == "mlx_lm" or name.startswith("mlx_lm."):
-            raise ImportError("cannot import name 'AutoTokenizer' from 'transformers'")
-        return real_import(name, *args, **kwargs)
-
-    sys.modules.pop("mlx_lm", None)
-
-    with patch.dict(sys.modules, _stub_mlx_modules()):
-        with patch("builtins.__import__", side_effect=fake_import):
-            with pytest.raises(ImportError, match="AutoTokenizer"):
-                await encoder.initialize()
-
-
-@pytest.mark.asyncio
-async def test_initialize_reports_install_hint_when_mlx_missing():
-    """A genuine 'package not installed' error still gets the friendly install hint."""
-    encoder = JinaMLXCrossEncoder()
-
-    real_import = __import__
-
-    def fake_import(name, *args, **kwargs):
-        if name == "mlx_lm" or name.startswith("mlx_lm."):
-            raise ImportError("No module named 'mlx_lm'")
-        if name == "mlx" or name.startswith("mlx."):
-            raise ImportError("No module named 'mlx'")
-        return real_import(name, *args, **kwargs)
-
-    sys.modules.pop("mlx_lm", None)
-    sys.modules.pop("mlx", None)
-    sys.modules.pop("mlx.core", None)
-
-    with patch("builtins.__import__", side_effect=fake_import):
-        with pytest.raises(ImportError, match="mlx and mlx-lm are required"):
-            await encoder.initialize()

hindsight-api-slim/tests/test_link_utils.py

@@ -1,534 +0,0 @@
-"""Tests for link_utils datetime handling, temporal link computation, and semantic link splitting."""
-import numpy as np
-import pytest
-from datetime import datetime, timezone, timedelta
-from unittest.mock import AsyncMock, MagicMock
-
-from hindsight_api.engine.retain.link_utils import (
-    _normalize_datetime,
-    _cap_links_per_unit,
-    compute_temporal_links,
-    compute_temporal_query_bounds,
-    compute_semantic_links_ann,
-    compute_semantic_links_within_batch,
-    MAX_TEMPORAL_LINKS_PER_UNIT,
-)
-
-
-class TestNormalizeDatetime:
-    """Tests for the _normalize_datetime helper function."""
-
-    def test_none_returns_none(self):
-        """Test that None input returns None."""
-        assert _normalize_datetime(None) is None
-
-    def test_naive_datetime_becomes_utc(self):
-        """Test that naive datetimes are converted to UTC."""
-        naive_dt = datetime(2024, 6, 15, 10, 30, 0)
-        result = _normalize_datetime(naive_dt)
-
-        assert result.tzinfo is not None
-        assert result.tzinfo == timezone.utc
-        assert result.year == 2024
-        assert result.month == 6
-        assert result.day == 15
-        assert result.hour == 10
-        assert result.minute == 30
-
-    def test_aware_datetime_unchanged(self):
-        """Test that timezone-aware datetimes are returned unchanged."""
-        aware_dt = datetime(2024, 6, 15, 10, 30, 0, tzinfo=timezone.utc)
-        result = _normalize_datetime(aware_dt)
-
-        assert result == aware_dt
-        assert result.tzinfo == timezone.utc
-
-    def test_mixed_datetimes_can_be_compared(self):
-        """Test that normalized naive and aware datetimes can be compared."""
-        naive_dt = datetime(2024, 6, 15, 10, 30, 0)
-        aware_dt = datetime(2024, 6, 15, 10, 30, 0, tzinfo=timezone.utc)
-
-        normalized_naive = _normalize_datetime(naive_dt)
-        normalized_aware = _normalize_datetime(aware_dt)
-
-        # Should be able to compare without TypeError
-        assert normalized_naive == normalized_aware
-
-
-class TestComputeTemporalQueryBounds:
-    """Tests for compute_temporal_query_bounds function."""
-
-    def test_empty_units_returns_none(self):
-        """Test that empty input returns (None, None)."""
-        min_date, max_date = compute_temporal_query_bounds({})
-        assert min_date is None
-        assert max_date is None
-
-    def test_single_unit_normal_date(self):
-        """Test bounds for a single unit with normal date."""
-        units = {"unit-1": datetime(2024, 6, 15, 12, 0, 0, tzinfo=timezone.utc)}
-        min_date, max_date = compute_temporal_query_bounds(units, time_window_hours=24)
-
-        assert min_date == datetime(2024, 6, 14, 12, 0, 0, tzinfo=timezone.utc)
-        assert max_date == datetime(2024, 6, 16, 12, 0, 0, tzinfo=timezone.utc)
-
-    def test_multiple_units(self):
-        """Test bounds span across multiple units."""
-        units = {
-            "unit-1": datetime(2024, 6, 10, 12, 0, 0, tzinfo=timezone.utc),
-            "unit-2": datetime(2024, 6, 15, 12, 0, 0, tzinfo=timezone.utc),
-            "unit-3": datetime(2024, 6, 20, 12, 0, 0, tzinfo=timezone.utc),
-        }
-        min_date, max_date = compute_temporal_query_bounds(units, time_window_hours=24)
-
-        # min should be Jun 10 - 24h = Jun 9
-        assert min_date == datetime(2024, 6, 9, 12, 0, 0, tzinfo=timezone.utc)
-        # max should be Jun 20 + 24h = Jun 21
-        assert max_date == datetime(2024, 6, 21, 12, 0, 0, tzinfo=timezone.utc)
-
-    def test_mixed_naive_and_aware_datetimes(self):
-        """Test that mixed naive/aware datetimes work correctly."""
-        units = {
-            "unit-1": datetime(2024, 6, 10, 12, 0, 0),  # naive
-            "unit-2": datetime(2024, 6, 15, 12, 0, 0, tzinfo=timezone.utc),  # aware
-        }
-        # Should not raise TypeError
-        min_date, max_date = compute_temporal_query_bounds(units, time_window_hours=24)
-
-        assert min_date is not None
-        assert max_date is not None
-        assert min_date.tzinfo is not None
-        assert max_date.tzinfo is not None
-
-    def test_overflow_near_datetime_min(self):
-        """Test overflow protection near datetime.min."""
-        units = {"unit-1": datetime(1, 1, 2, 0, 0, tzinfo=timezone.utc)}
-        min_date, max_date = compute_temporal_query_bounds(units, time_window_hours=48)
-
-        # Should handle overflow gracefully
-        assert min_date == datetime.min.replace(tzinfo=timezone.utc)
-        assert max_date is not None
-
-    def test_overflow_near_datetime_max(self):
-        """Test overflow protection near datetime.max."""
-        units = {"unit-1": datetime(9999, 12, 30, 0, 0, tzinfo=timezone.utc)}
-        min_date, max_date = compute_temporal_query_bounds(units, time_window_hours=48)
-
-        # Should handle overflow gracefully
-        assert min_date is not None
-        assert max_date == datetime.max.replace(tzinfo=timezone.utc)
-
-
-class TestComputeTemporalLinks:
-    """Tests for compute_temporal_links function."""
-
-    def test_empty_units_returns_empty(self):
-        """Test that empty input returns empty list."""
-        links = compute_temporal_links({}, [])
-        assert links == []
-
-    def test_no_candidates_returns_empty(self):
-        """Test that no candidates means no links."""
-        units = {"unit-1": datetime(2024, 6, 15, 12, 0, 0, tzinfo=timezone.utc)}
-        links = compute_temporal_links(units, [])
-        assert links == []
-
-    def test_candidate_within_window_creates_link(self):
-        """Test that candidates within time window create links."""
-        units = {"unit-1": datetime(2024, 6, 15, 12, 0, 0, tzinfo=timezone.utc)}
-        candidates = [
-            {"id": "candidate-1", "event_date": datetime(2024, 6, 15, 10, 0, 0, tzinfo=timezone.utc)},
-        ]
-
-        links = compute_temporal_links(units, candidates, time_window_hours=24)
-
-        assert len(links) == 1
-        assert links[0][0] == "unit-1"
-        assert links[0][1] == "candidate-1"
-        assert links[0][2] == "temporal"
-        assert links[0][4] is None
-        # Weight should be high since they're close (2 hours apart)
-        assert links[0][3] > 0.9
-
-    def test_candidate_outside_window_no_link(self):
-        """Test that candidates outside time window don't create links."""
-        units = {"unit-1": datetime(2024, 6, 15, 12, 0, 0, tzinfo=timezone.utc)}
-        candidates = [
-            {"id": "candidate-1", "event_date": datetime(2024, 6, 10, 12, 0, 0, tzinfo=timezone.utc)},
-        ]
-
-        links = compute_temporal_links(units, candidates, time_window_hours=24)
-
-        assert len(links) == 0
-
-    def test_weight_decreases_with_distance(self):
-        """Test that weight decreases as time difference increases."""
-        units = {"unit-1": datetime(2024, 6, 15, 12, 0, 0, tzinfo=timezone.utc)}
-        candidates = [
-            {"id": "close", "event_date": datetime(2024, 6, 15, 11, 0, 0, tzinfo=timezone.utc)},  # 1 hour
-            {"id": "far", "event_date": datetime(2024, 6, 14, 18, 0, 0, tzinfo=timezone.utc)},  # 18 hours
-        ]
-
-        links = compute_temporal_links(units, candidates, time_window_hours=24)
-
-        assert len(links) == 2
-        close_link = next(l for l in links if l[1] == "close")
-        far_link = next(l for l in links if l[1] == "far")
-
-        assert close_link[3] > far_link[3]
-
-    def test_max_10_links_per_unit(self):
-        """Test that at most 10 links are created per unit."""
-        units = {"unit-1": datetime(2024, 6, 15, 12, 0, 0, tzinfo=timezone.utc)}
-        # Create 15 candidates all within window
-        candidates = [
-            {"id": f"candidate-{i}", "event_date": datetime(2024, 6, 15, 11, 0, 0, tzinfo=timezone.utc)}
-            for i in range(15)
-        ]
-
-        links = compute_temporal_links(units, candidates, time_window_hours=24)
-
-        assert len(links) == 10
-
-    def test_multiple_units_multiple_candidates(self):
-        """Test with multiple units and candidates."""
-        units = {
-            "unit-1": datetime(2024, 6, 15, 12, 0, 0, tzinfo=timezone.utc),
-            "unit-2": datetime(2024, 6, 20, 12, 0, 0, tzinfo=timezone.utc),
-        }
-        candidates = [
-            {"id": "c1", "event_date": datetime(2024, 6, 15, 10, 0, 0, tzinfo=timezone.utc)},  # near unit-1
-            {"id": "c2", "event_date": datetime(2024, 6, 20, 10, 0, 0, tzinfo=timezone.utc)},  # near unit-2
-            {"id": "c3", "event_date": datetime(2024, 6, 17, 12, 0, 0, tzinfo=timezone.utc)},  # between, near neither
-        ]
-
-        links = compute_temporal_links(units, candidates, time_window_hours=24)
-
-        # unit-1 should link to c1 only
-        # unit-2 should link to c2 only
-        unit1_links = [l for l in links if l[0] == "unit-1"]
-        unit2_links = [l for l in links if l[0] == "unit-2"]
-
-        assert len(unit1_links) == 1
-        assert unit1_links[0][1] == "c1"
-
-        assert len(unit2_links) == 1
-        assert unit2_links[0][1] == "c2"
-
-    def test_mixed_naive_and_aware_datetimes(self):
-        """Test that mixed naive/aware datetimes work correctly."""
-        units = {"unit-1": datetime(2024, 6, 15, 12, 0, 0)}  # naive
-        candidates = [
-            {"id": "c1", "event_date": datetime(2024, 6, 15, 10, 0, 0, tzinfo=timezone.utc)},  # aware
-        ]
-
-        # Should not raise TypeError
-        links = compute_temporal_links(units, candidates, time_window_hours=24)
-        assert len(links) == 1
-
-    def test_overflow_near_datetime_min(self):
-        """Test overflow protection when unit date is near datetime.min."""
-        units = {"unit-1": datetime(1, 1, 2, 0, 0, tzinfo=timezone.utc)}
-        candidates = [
-            {"id": "c1", "event_date": datetime(1, 1, 1, 12, 0, 0, tzinfo=timezone.utc)},
-        ]
-
-        # Should not raise OverflowError
-        links = compute_temporal_links(units, candidates, time_window_hours=48)
-        assert len(links) == 1
-
-    def test_overflow_near_datetime_max(self):
-        """Test overflow protection when unit date is near datetime.max."""
-        units = {"unit-1": datetime(9999, 12, 30, 0, 0, tzinfo=timezone.utc)}
-        candidates = [
-            {"id": "c1", "event_date": datetime(9999, 12, 31, 12, 0, 0, tzinfo=timezone.utc)},
-        ]
-
-        # Should not raise OverflowError
-        links = compute_temporal_links(units, candidates, time_window_hours=48)
-        assert len(links) == 1
-
-    def test_weight_minimum_is_0_3(self):
-        """Test that weight doesn't go below 0.3."""
-        units = {"unit-1": datetime(2024, 6, 15, 12, 0, 0, tzinfo=timezone.utc)}
-        candidates = [
-            # 23 hours apart - should be just within 24h window but low weight
-            {"id": "c1", "event_date": datetime(2024, 6, 14, 13, 0, 0, tzinfo=timezone.utc)},
-        ]
-
-        links = compute_temporal_links(units, candidates, time_window_hours=24)
-
-        assert len(links) == 1
-        assert links[0][3] >= 0.3
-
-
-class TestCapLinksPerUnit:
-    """Tests for the _cap_links_per_unit helper function."""
-
-    def test_empty_links(self):
-        assert _cap_links_per_unit([]) == []
-
-    def test_under_cap_unchanged(self):
-        links = [
-            ("unit_a", "unit_x", "temporal", 0.9, None),
-            ("unit_a", "unit_y", "temporal", 0.8, None),
-        ]
-        result = _cap_links_per_unit(links, max_per_unit=5)
-        assert len(result) == 2
-
-    def test_caps_to_max_per_unit(self):
-        # Create 30 links from the same unit with descending weights
-        links = [("unit_a", f"unit_{i}", "temporal", 1.0 - i * 0.01, None) for i in range(30)]
-        result = _cap_links_per_unit(links, max_per_unit=10)
-        assert len(result) == 10
-        # Should keep the highest-weight links
-        weights = [lnk[3] for lnk in result]
-        assert weights == sorted(weights, reverse=True)
-        assert weights[0] == 1.0  # Highest weight kept
-
-    def test_caps_independently_per_unit(self):
-        links_a = [("unit_a", f"target_{i}", "temporal", 0.9 - i * 0.01, None) for i in range(10)]
-        links_b = [("unit_b", f"target_{i}", "temporal", 0.8 - i * 0.01, None) for i in range(10)]
-        result = _cap_links_per_unit(links_a + links_b, max_per_unit=5)
-        # 5 from unit_a + 5 from unit_b
-        assert len(result) == 10
-        from_a = [lnk for lnk in result if lnk[0] == "unit_a"]
-        from_b = [lnk for lnk in result if lnk[0] == "unit_b"]
-        assert len(from_a) == 5
-        assert len(from_b) == 5
-
-    def test_default_max_is_temporal_constant(self):
-        links = [("unit_a", f"target_{i}", "temporal", 1.0 - i * 0.01, None) for i in range(50)]
-        result = _cap_links_per_unit(links)
-        assert len(result) == MAX_TEMPORAL_LINKS_PER_UNIT
-
-    def test_preserves_tuple_structure(self):
-        links = [("from_id", "to_id", "temporal", 0.95, "entity_id")]
-        result = _cap_links_per_unit(links, max_per_unit=5)
-        assert result[0] == ("from_id", "to_id", "temporal", 0.95, "entity_id")
-
-
-class TestComputeSemanticLinksWithinBatch:
-    """Tests for compute_semantic_links_within_batch.
-
-    This function computes semantic links between units in the same batch
-    using numpy dot product (no DB access). It runs in Phase 2 (write
-    transaction) while the expensive ANN search against existing units runs
-    in Phase 1 on a separate connection to avoid TimeoutErrors from HNSW
-    index contention under concurrent load.
-    """
-
-    def test_empty_returns_empty(self):
-        assert compute_semantic_links_within_batch([], []) == []
-
-    def test_single_unit_returns_empty(self):
-        emb = [np.random.randn(384).tolist()]
-        assert compute_semantic_links_within_batch(["u1"], emb) == []
-
-    def test_identical_embeddings_produce_links(self):
-        """Two identical embeddings should have similarity=1.0 (above 0.7 threshold)."""
-        emb = [0.1] * 384
-        links = compute_semantic_links_within_batch(["u1", "u2"], [emb, emb])
-        assert len(links) == 2  # bidirectional: u1→u2, u2→u1
-        from_ids = {lnk[0] for lnk in links}
-        to_ids = {lnk[1] for lnk in links}
-        assert from_ids == {"u1", "u2"}
-        assert to_ids == {"u1", "u2"}
-        for lnk in links:
-            assert lnk[2] == "semantic"
-            assert lnk[3] >= 0.99  # near-1.0 similarity
-            assert lnk[4] is None  # no entity_id
-
-    def test_orthogonal_embeddings_no_links(self):
-        """Orthogonal embeddings should have similarity=0 (below 0.7 threshold)."""
-        emb1 = [1.0] + [0.0] * 383
-        emb2 = [0.0] + [1.0] + [0.0] * 382
-        links = compute_semantic_links_within_batch(["u1", "u2"], [emb1, emb2])
-        assert len(links) == 0
-
-    def test_respects_threshold(self):
-        """Links below threshold should be excluded."""
-        emb1 = np.random.randn(384).tolist()
-        # Create a slightly similar embedding (add noise)
-        emb2 = [x + np.random.randn() * 0.5 for x in emb1]
-        # Normalize both
-        norm1 = np.linalg.norm(emb1)
-        norm2 = np.linalg.norm(emb2)
-        emb1 = [x / norm1 for x in emb1]
-        emb2 = [x / norm2 for x in emb2]
-
-        links_low = compute_semantic_links_within_batch(["u1", "u2"], [emb1, emb2], threshold=0.0)
-        links_high = compute_semantic_links_within_batch(["u1", "u2"], [emb1, emb2], threshold=0.99)
-        # Low threshold should have more links than high threshold
-        assert len(links_low) >= len(links_high)
-
-    def test_top_k_limits_per_unit(self):
-        """Each unit should link to at most top_k other units."""
-        n = 10
-        # Create similar embeddings (all close to the same vector)
-        base = np.random.randn(384)
-        base = base / np.linalg.norm(base)
-        embs = [(base + np.random.randn(384) * 0.01).tolist() for _ in range(n)]
-        unit_ids = [f"u{i}" for i in range(n)]
-
-        links = compute_semantic_links_within_batch(unit_ids, embs, top_k=3, threshold=0.5)
-        # Each unit should have at most 3 outgoing links
-        from collections import Counter
-        from_counts = Counter(lnk[0] for lnk in links)
-        for count in from_counts.values():
-            assert count <= 3
-
-    def test_link_tuple_structure(self):
-        """Verify the tuple format matches what _bulk_insert_links expects."""
-        emb = [0.1] * 384
-        links = compute_semantic_links_within_batch(["u1", "u2"], [emb, emb])
-        for lnk in links:
-            assert len(lnk) == 5
-            from_id, to_id, link_type, weight, entity_id = lnk
-            assert isinstance(from_id, str)
-            assert isinstance(to_id, str)
-            assert link_type == "semantic"
-            assert 0.0 <= weight <= 1.0
-            assert entity_id is None
-
-
-class TestComputeSemanticLinksAnnPgBouncerSafety:
-    """Regression tests ensuring compute_semantic_links_ann stays in a single
-    transaction so that the `_ann_seeds` temp table remains visible when the
-    caller's connection goes through pgBouncer in `transaction` pool mode.
-
-    In pgBouncer transaction mode, the backend is only pinned to the client
-    for the duration of an actual PostgreSQL transaction. Outside a
-    transaction, consecutive statements can land on different backends, and
-    session-scoped temp tables (which are bound to the backend that created
-    them) become invisible. The observed failure mode was an intermittent
-    `relation "_ann_seeds" does not exist` on the statement immediately
-    following the CREATE TEMP TABLE.
-    """
-
-    @pytest.fixture
-    def mock_conn(self):
-        """An asyncpg-like connection mock with an async `transaction()`
-        context manager and awaitable execute/fetch/copy helpers."""
-        conn = MagicMock()
-
-        txn_cm = MagicMock()
-        txn_cm.__aenter__ = AsyncMock(return_value=None)
-        txn_cm.__aexit__ = AsyncMock(return_value=None)
-        conn.transaction = MagicMock(return_value=txn_cm)
-
-        conn.execute = AsyncMock()
-        conn.copy_records_to_table = AsyncMock()
-        conn.fetch = AsyncMock(return_value=[])
-        return conn
-
-    @pytest.mark.asyncio
-    async def test_empty_inputs_skip_transaction(self, mock_conn):
-        """No seeds -> no work, no transaction, no temp-table churn."""
-        result = await compute_semantic_links_ann(
-            conn=mock_conn,
-            bank_id="bank-1",
-            unit_ids=[],
-            embeddings=[],
-        )
-        assert result == []
-        mock_conn.transaction.assert_not_called()
-        mock_conn.execute.assert_not_called()
-
-    @pytest.mark.asyncio
-    async def test_runs_inside_a_transaction(self, mock_conn):
-        """The full CREATE TEMP TABLE -> COPY -> SELECT sequence must happen
-        inside a single `async with conn.transaction():` block."""
-        emb = [0.1] * 384
-        await compute_semantic_links_ann(
-            conn=mock_conn,
-            bank_id="bank-1",
-            unit_ids=["u1", "u2"],
-            embeddings=[emb, emb],
-            fact_types=["world", "world"],
-        )
-
-        # Transaction context manager was entered.
-        mock_conn.transaction.assert_called_once()
-        txn_cm = mock_conn.transaction.return_value
-        txn_cm.__aenter__.assert_awaited_once()
-        txn_cm.__aexit__.assert_awaited_once()
-
-    @pytest.mark.asyncio
-    async def test_temp_table_uses_on_commit_drop(self, mock_conn):
-        """The CREATE TEMP TABLE statement must use ON COMMIT DROP so the
-        table is transaction-scoped. Without ON COMMIT DROP the table would
-        be session-scoped and would not survive pgBouncer backend rebinding
-        between transactions."""
-        emb = [0.1] * 384
-        await compute_semantic_links_ann(
-            conn=mock_conn,
-            bank_id="bank-1",
-            unit_ids=["u1"],
-            embeddings=[emb],
-            fact_types=["world"],
-        )
-
-        executed_sql = [call.args[0] for call in mock_conn.execute.call_args_list]
-        create_statements = [s for s in executed_sql if "CREATE TEMP TABLE" in s]
-        assert len(create_statements) == 1, "Should create _ann_seeds exactly once"
-        assert "_ann_seeds" in create_statements[0]
-        assert "ON COMMIT DROP" in create_statements[0], (
-            "CREATE TEMP TABLE must use ON COMMIT DROP so the table is cleaned "
-            "up at transaction end and is transaction-scoped"
-        )
-
-        # Must not use IF NOT EXISTS — the table is fresh each transaction.
-        assert "IF NOT EXISTS" not in create_statements[0], (
-            "With ON COMMIT DROP the table is always fresh at transaction start, "
-            "so IF NOT EXISTS is both unnecessary and misleading (suggests the "
-            "table might persist across transactions)"
-        )
-
-    @pytest.mark.asyncio
-    async def test_no_manual_drop_or_truncate(self, mock_conn):
-        """With ON COMMIT DROP we must not re-add manual TRUNCATE or DROP
-        statements — they were the source of the original pgBouncer bug."""
-        emb = [0.1] * 384
-        await compute_semantic_links_ann(
-            conn=mock_conn,
-            bank_id="bank-1",
-            unit_ids=["u1"],
-            embeddings=[emb],
-            fact_types=["world"],
-        )
-
-        executed_sql = [call.args[0] for call in mock_conn.execute.call_args_list]
-        assert not any("TRUNCATE _ann_seeds" in s for s in executed_sql), (
-            "TRUNCATE is unnecessary with ON COMMIT DROP and was previously "
-            "the statement that failed with 'relation does not exist' when "
-            "pgBouncer rebound the backend"
-        )
-        assert not any("DROP TABLE" in s and "_ann_seeds" in s for s in executed_sql), (
-            "Explicit DROP is unnecessary with ON COMMIT DROP"
-        )
-
-    @pytest.mark.asyncio
-    async def test_uses_set_local_for_ef_search(self, mock_conn):
-        """hnsw.ef_search must be set with SET LOCAL so the change is scoped
-        to the transaction. Without SET LOCAL, the setting would leak onto
-        the pooled backend and affect subsequent recall queries that land
-        on the same backend."""
-        emb = [0.1] * 384
-        await compute_semantic_links_ann(
-            conn=mock_conn,
-            bank_id="bank-1",
-            unit_ids=["u1"],
-            embeddings=[emb],
-            fact_types=["world"],
-        )
-
-        executed_sql = [call.args[0] for call in mock_conn.execute.call_args_list]
-        ef_statements = [s for s in executed_sql if "hnsw.ef_search" in s]
-        assert ef_statements, "ef_search must be tuned down for retain ANN"
-        for stmt in ef_statements:
-            assert stmt.strip().startswith("SET LOCAL"), (
-                f"hnsw.ef_search must use SET LOCAL, got: {stmt}"
-            )
-        # And there must not be a RESET — SET LOCAL handles it at commit.
-        assert not any("RESET hnsw.ef_search" in s for s in executed_sql)

hindsight-api-slim/tests/test_llm_wrapper.py

@@ -1,37 +0,0 @@
-import pytest
-
-from hindsight_api.engine.llm_wrapper import sanitize_llm_output
-
-
-@pytest.mark.parametrize(
-    "input_text, expected",
-    [
-        # Null bytes stripped
-        ("hello\x00world", "helloworld"),
-        ("FIRST\u0000PAGE", "FIRSTPAGE"),
-        # Multiple null bytes
-        ("\x00\x00text\x00", "text"),
-        # Other control characters stripped (non-whitespace)
-        ("text\x01\x02\x03end", "textend"),
-        ("text\x08end", "textend"),  # backspace
-        ("text\x0cend", "textend"),  # form feed
-        ("text\x0bend", "textend"),  # vertical tab
-        ("text\x1fend", "textend"),  # unit separator
-        ("text\x7fend", "textend"),  # DEL
-        # Whitespace preserved
-        ("hello\tworld", "hello\tworld"),
-        ("hello\nworld", "hello\nworld"),
-        ("hello\r\nworld", "hello\r\nworld"),
-        # Unicode surrogates stripped
-        ("text\ud800end", "textend"),
-        ("text\udfffend", "textend"),
-        # Clean text unchanged
-        ("normal text", "normal text"),
-        ("unicode: café naïve", "unicode: café naïve"),
-        # Edge cases
-        ("", ""),
-        (None, None),
-    ],
-)
-def test_sanitize_llm_output(input_text, expected):
-    assert sanitize_llm_output(input_text) == expected

hindsight-api-slim/tests/test_mcp_string_coercion.py

@@ -1,330 +0,0 @@
-"""Tests for MCP tool argument string-to-JSON coercion (issue #849)."""
-
-import json
-from unittest.mock import AsyncMock, MagicMock
-
-import pytest
-
-from hindsight_api.api.mcp import (
-    _coerce_string_json,
-    _collect_coercible_types,
-    _get_mcp_tools,
-    _make_tools_tolerant,
-)
-
-
-# ---------------------------------------------------------------------------
-# _collect_coercible_types — schema type detection
-# ---------------------------------------------------------------------------
-
-
-class TestCollectCoercibleTypes:
-    """Tests for _collect_coercible_types schema detection."""
-
-    def _run(self, schema: dict, param_name: str = "p") -> tuple[set[str], set[str]]:
-        array_params: set[str] = set()
-        object_params: set[str] = set()
-        _collect_coercible_types(schema, param_name, array_params, object_params)
-        return array_params, object_params
-
-    # --- array types ---
-
-    def test_direct_array_type(self):
-        arrays, objects = self._run({"type": "array", "items": {"type": "string"}})
-        assert "p" in arrays and not objects
-
-    def test_anyof_nullable_array(self):
-        """list[str] | None → anyOf with array and null."""
-        arrays, objects = self._run(
-            {"anyOf": [{"type": "array", "items": {"type": "string"}}, {"type": "null"}]}
-        )
-        assert "p" in arrays
-
-    def test_oneof_nullable_array(self):
-        """oneOf variant."""
-        arrays, objects = self._run(
-            {"oneOf": [{"type": "array", "items": {"type": "string"}}, {"type": "null"}]}
-        )
-        assert "p" in arrays
-
-    # --- object types ---
-
-    def test_direct_object_type(self):
-        arrays, objects = self._run({"type": "object"})
-        assert "p" in objects and not arrays
-
-    def test_anyof_nullable_object(self):
-        """dict[str, str] | None → anyOf with object and null."""
-        arrays, objects = self._run({"anyOf": [{"type": "object"}, {"type": "null"}]})
-        assert "p" in objects
-
-    def test_oneof_nullable_object(self):
-        arrays, objects = self._run({"oneOf": [{"type": "object"}, {"type": "null"}]})
-        assert "p" in objects
-
-    # --- non-coercible types (should be ignored) ---
-
-    def test_string_type_ignored(self):
-        arrays, objects = self._run({"type": "string"})
-        assert not arrays and not objects
-
-    def test_integer_type_ignored(self):
-        arrays, objects = self._run({"type": "integer"})
-        assert not arrays and not objects
-
-    def test_number_type_ignored(self):
-        arrays, objects = self._run({"type": "number"})
-        assert not arrays and not objects
-
-    def test_boolean_type_ignored(self):
-        arrays, objects = self._run({"type": "boolean"})
-        assert not arrays and not objects
-
-    def test_null_type_ignored(self):
-        arrays, objects = self._run({"type": "null"})
-        assert not arrays and not objects
-
-    def test_anyof_string_or_null_ignored(self):
-        """str | None should not be collected."""
-        arrays, objects = self._run({"anyOf": [{"type": "string"}, {"type": "null"}]})
-        assert not arrays and not objects
-
-    def test_anyof_integer_or_null_ignored(self):
-        arrays, objects = self._run({"anyOf": [{"type": "integer"}, {"type": "null"}]})
-        assert not arrays and not objects
-
-
-# ---------------------------------------------------------------------------
-# _coerce_string_json — value coercion
-# ---------------------------------------------------------------------------
-
-
-class TestCoerceStringJson:
-    """Tests for _coerce_string_json argument coercion."""
-
-    # --- list coercion ---
-
-    def test_coerce_string_to_list(self):
-        result = _coerce_string_json(
-            {"tags": '["tag1", "tag2"]', "query": "hello"},
-            array_params={"tags"},
-            object_params=set(),
-        )
-        assert result["tags"] == ["tag1", "tag2"]
-        assert result["query"] == "hello"
-
-    def test_coerce_empty_list_string(self):
-        result = _coerce_string_json({"tags": "[]"}, array_params={"tags"}, object_params=set())
-        assert result["tags"] == []
-
-    def test_native_list_passthrough(self):
-        result = _coerce_string_json({"tags": ["a", "b"]}, array_params={"tags"}, object_params=set())
-        assert result["tags"] == ["a", "b"]
-
-    # --- dict coercion ---
-
-    def test_coerce_string_to_dict(self):
-        result = _coerce_string_json(
-            {"metadata": '{"key": "value"}'},
-            array_params=set(),
-            object_params={"metadata"},
-        )
-        assert result["metadata"] == {"key": "value"}
-
-    def test_coerce_empty_dict_string(self):
-        result = _coerce_string_json({"metadata": "{}"}, array_params=set(), object_params={"metadata"})
-        assert result["metadata"] == {}
-
-    def test_native_dict_passthrough(self):
-        result = _coerce_string_json(
-            {"metadata": {"key": "value"}}, array_params=set(), object_params={"metadata"}
-        )
-        assert result["metadata"] == {"key": "value"}
-
-    # --- non-coercible values left untouched ---
-
-    def test_none_passthrough(self):
-        result = _coerce_string_json({"tags": None}, array_params={"tags"}, object_params=set())
-        assert result["tags"] is None
-
-    def test_invalid_json_string_passthrough(self):
-        result = _coerce_string_json({"tags": "not-json"}, array_params={"tags"}, object_params=set())
-        assert result["tags"] == "not-json"
-
-    def test_wrong_json_type_not_coerced_list(self):
-        """String that parses to a dict should NOT be coerced for an array param."""
-        result = _coerce_string_json(
-            {"tags": '{"key": "value"}'}, array_params={"tags"}, object_params=set()
-        )
-        assert result["tags"] == '{"key": "value"}'
-
-    def test_wrong_json_type_not_coerced_dict(self):
-        """String that parses to a list should NOT be coerced for an object param."""
-        result = _coerce_string_json(
-            {"metadata": '["a", "b"]'}, array_params=set(), object_params={"metadata"}
-        )
-        assert result["metadata"] == '["a", "b"]'
-
-    def test_string_param_not_touched(self):
-        """Strings not in array_params/object_params are never modified."""
-        result = _coerce_string_json(
-            {"query": '["looks", "like", "json"]'},
-            array_params=set(),
-            object_params=set(),
-        )
-        assert result["query"] == '["looks", "like", "json"]'
-
-    def test_integer_param_not_touched(self):
-        result = _coerce_string_json(
-            {"max_tokens": 4096}, array_params=set(), object_params=set()
-        )
-        assert result["max_tokens"] == 4096
-
-    def test_boolean_param_not_touched(self):
-        result = _coerce_string_json(
-            {"verbose": True}, array_params=set(), object_params=set()
-        )
-        assert result["verbose"] is True
-
-    def test_missing_param_no_error(self):
-        result = _coerce_string_json(
-            {"query": "hello"},
-            array_params={"tags"},
-            object_params={"metadata"},
-        )
-        assert result == {"query": "hello"}
-
-    # --- multiple params coerced at once ---
-
-    def test_multiple_params_coerced(self):
-        result = _coerce_string_json(
-            {
-                "tags": '["a", "b"]',
-                "types": '["world"]',
-                "metadata": '{"source": "test"}',
-                "query": "hello",
-                "max_tokens": 4096,
-            },
-            array_params={"tags", "types"},
-            object_params={"metadata"},
-        )
-        assert result["tags"] == ["a", "b"]
-        assert result["types"] == ["world"]
-        assert result["metadata"] == {"source": "test"}
-        assert result["query"] == "hello"
-        assert result["max_tokens"] == 4096
-
-
-# ---------------------------------------------------------------------------
-# _make_tools_tolerant — integration test with a real FastMCP tool
-# ---------------------------------------------------------------------------
-
-
-class TestMakeToolsTolerantIntegration:
-    """Test that _make_tools_tolerant correctly wraps real FastMCP tool functions."""
-
-    def _create_mcp_with_tool(self):
-        """Create a FastMCP instance with a tool that uses various parameter types."""
-        from fastmcp import FastMCP
-
-        mcp = FastMCP("test")
-        captured = {}
-
-        @mcp.tool(description="test tool with diverse param types")
-        async def test_tool(
-            query: str,
-            max_tokens: int = 100,
-            verbose: bool = False,
-            tags: list[str] | None = None,
-            metadata: dict[str, str] | None = None,
-        ) -> dict:
-            """Test tool.
-
-            Args:
-                query: a string param
-                max_tokens: an integer param
-                verbose: a boolean param
-                tags: an array param
-                metadata: an object param
-            """
-            captured["query"] = query
-            captured["max_tokens"] = max_tokens
-            captured["verbose"] = verbose
-            captured["tags"] = tags
-            captured["metadata"] = metadata
-            return {"ok": True}
-
-        return mcp, captured
-
-    @pytest.mark.asyncio
-    async def test_coerces_string_encoded_list(self):
-        mcp, captured = self._create_mcp_with_tool()
-        _make_tools_tolerant(mcp)
-        tool = _get_mcp_tools(mcp)["test_tool"]
-        await tool.run({"query": "hi", "tags": '["a", "b"]'})
-        assert captured["tags"] == ["a", "b"]
-
-    @pytest.mark.asyncio
-    async def test_coerces_string_encoded_dict(self):
-        mcp, captured = self._create_mcp_with_tool()
-        _make_tools_tolerant(mcp)
-        tool = _get_mcp_tools(mcp)["test_tool"]
-        await tool.run({"query": "hi", "metadata": '{"k": "v"}'})
-        assert captured["metadata"] == {"k": "v"}
-
-    @pytest.mark.asyncio
-    async def test_native_types_pass_through(self):
-        mcp, captured = self._create_mcp_with_tool()
-        _make_tools_tolerant(mcp)
-        tool = _get_mcp_tools(mcp)["test_tool"]
-        await tool.run({
-            "query": "hi",
-            "max_tokens": 200,
-            "verbose": True,
-            "tags": ["x"],
-            "metadata": {"a": "b"},
-        })
-        assert captured["query"] == "hi"
-        assert captured["max_tokens"] == 200
-        assert captured["verbose"] is True
-        assert captured["tags"] == ["x"]
-        assert captured["metadata"] == {"a": "b"}
-
-    @pytest.mark.asyncio
-    async def test_strips_extra_args_and_coerces(self):
-        """Both extra-arg stripping and coercion work together."""
-        mcp, captured = self._create_mcp_with_tool()
-        _make_tools_tolerant(mcp)
-        tool = _get_mcp_tools(mcp)["test_tool"]
-        await tool.run({
-            "query": "hi",
-            "tags": '["x"]',
-            "explanation": "LLM added this",
-        })
-        assert captured["tags"] == ["x"]
-        assert "explanation" not in captured
-
-    @pytest.mark.asyncio
-    async def test_string_param_not_coerced(self):
-        """A string param whose value happens to look like JSON is NOT coerced."""
-        mcp, captured = self._create_mcp_with_tool()
-        _make_tools_tolerant(mcp)
-        tool = _get_mcp_tools(mcp)["test_tool"]
-        await tool.run({"query": '["this", "is", "a", "string"]'})
-        assert captured["query"] == '["this", "is", "a", "string"]'
-
-    @pytest.mark.asyncio
-    async def test_integer_param_not_coerced(self):
-        mcp, captured = self._create_mcp_with_tool()
-        _make_tools_tolerant(mcp)
-        tool = _get_mcp_tools(mcp)["test_tool"]
-        await tool.run({"query": "hi", "max_tokens": 50})
-        assert captured["max_tokens"] == 50
-
-    @pytest.mark.asyncio
-    async def test_boolean_param_not_coerced(self):
-        mcp, captured = self._create_mcp_with_tool()
-        _make_tools_tolerant(mcp)
-        tool = _get_mcp_tools(mcp)["test_tool"]
-        await tool.run({"query": "hi", "verbose": True})
-        assert captured["verbose"] is True

hindsight-api-slim/tests/test_mcp_tool_filtering.py

@@ -1,338 +0,0 @@
-"""Tests for filter_mcp_tools on OperationValidatorExtension."""
-
-from unittest.mock import AsyncMock, MagicMock
-
-import pytest
-
-from hindsight_api.api.mcp import (
-    _current_api_key,
-    _current_api_key_id,
-    _current_bank_id,
-    _current_mcp_authenticated,
-    _current_tenant_id,
-    create_mcp_server,
-)
-from hindsight_api.extensions.operation_validator import OperationValidatorExtension, ValidationResult
-from hindsight_api.models import RequestContext
-
-
-class MinimalValidator(OperationValidatorExtension):
-    """Minimal concrete subclass — only implements abstract methods."""
-
-    async def validate_retain(self, ctx):
-        return ValidationResult.accept()
-
-    async def validate_recall(self, ctx):
-        return ValidationResult.accept()
-
-    async def validate_reflect(self, ctx):
-        return ValidationResult.accept()
-
-
-class FilteringValidator(OperationValidatorExtension):
-    """Validator that removes retain from the tool set."""
-
-    async def validate_retain(self, ctx):
-        return ValidationResult.accept()
-
-    async def validate_recall(self, ctx):
-        return ValidationResult.accept()
-
-    async def validate_reflect(self, ctx):
-        return ValidationResult.accept()
-
-    async def filter_mcp_tools(self, bank_id, request_context, tools):
-        return tools - {"retain"}
-
-
-@pytest.mark.asyncio
-async def test_filter_mcp_tools_default_returns_all():
-    """Default implementation returns the input unchanged."""
-    validator = MinimalValidator({})
-    tools = frozenset({"retain", "recall", "reflect", "list_memories"})
-    ctx = RequestContext()
-
-    result = await validator.filter_mcp_tools("test-bank", ctx, tools)
-
-    assert result == tools
-    assert isinstance(result, frozenset)
-
-
-@pytest.mark.asyncio
-async def test_filter_mcp_tools_subclass_removes_tools():
-    """Subclass can remove tools from the set."""
-    validator = FilteringValidator({})
-    tools = frozenset({"retain", "recall", "reflect"})
-    ctx = RequestContext()
-
-    result = await validator.filter_mcp_tools("test-bank", ctx, tools)
-
-    assert result == frozenset({"recall", "reflect"})
-    assert "retain" not in result
-
-
-@pytest.mark.asyncio
-async def test_filter_mcp_tools_returns_empty_set():
-    """Validator can return empty set — no tools visible."""
-
-    class DenyAllValidator(OperationValidatorExtension):
-        async def validate_retain(self, ctx):
-            return ValidationResult.accept()
-        async def validate_recall(self, ctx):
-            return ValidationResult.accept()
-        async def validate_reflect(self, ctx):
-            return ValidationResult.accept()
-        async def filter_mcp_tools(self, bank_id, request_context, tools):
-            return frozenset()
-
-    validator = DenyAllValidator({})
-    tools = frozenset({"retain", "recall", "reflect"})
-    ctx = RequestContext()
-
-    result = await validator.filter_mcp_tools("test-bank", ctx, tools)
-
-    assert result == frozenset()
-    assert len(result) == 0
-
-
-@pytest.mark.asyncio
-async def test_validator_filters_tools_list():
-    """Validator filter is applied during tools/list via _get_enabled_tools."""
-    mock_memory = MagicMock()
-    mock_memory._tenant_extension = MagicMock()
-    mock_memory._tenant_extension.authenticate_mcp = AsyncMock()
-    mock_memory.retain_batch_async = AsyncMock()
-    mock_memory.submit_async_retain = AsyncMock()
-    mock_memory.recall_async = AsyncMock()
-    mock_memory.reflect_async = AsyncMock()
-    mock_memory.list_banks = AsyncMock(return_value=[])
-
-    validator = FilteringValidator({})
-    mock_memory._operation_validator = validator
-
-    mock_config = {"mcp_enabled_tools": None}
-    mock_memory._config_resolver = MagicMock()
-    mock_memory._config_resolver.get_bank_config = AsyncMock(return_value=mock_config)
-
-    mcp_server = create_mcp_server(mock_memory, multi_bank=False)
-
-    bank_token = _current_bank_id.set("test-bank")
-    api_key_token = _current_api_key.set("hsk_test_key")
-    tenant_token = _current_tenant_id.set("alice")
-    key_id_token = _current_api_key_id.set("key-uuid")
-    mcp_auth_token = _current_mcp_authenticated.set(False)
-    try:
-        if hasattr(mcp_server, "list_tools"):
-            tools = await mcp_server.list_tools()
-            tool_names = {t.name for t in tools}
-        else:
-            tools = await mcp_server._tool_manager.get_tools()
-            tool_names = set(tools.keys())
-
-        assert "recall" in tool_names
-        assert "reflect" in tool_names
-        assert "retain" not in tool_names
-    finally:
-        _current_bank_id.reset(bank_token)
-        _current_api_key.reset(api_key_token)
-        _current_tenant_id.reset(tenant_token)
-        _current_api_key_id.reset(key_id_token)
-        _current_mcp_authenticated.reset(mcp_auth_token)
-
-
-@pytest.mark.asyncio
-async def test_bank_config_and_validator_compose():
-    """Bank config sets ceiling, validator narrows further."""
-    mock_memory = MagicMock()
-    mock_memory._tenant_extension = MagicMock()
-    mock_memory._tenant_extension.authenticate_mcp = AsyncMock()
-    mock_memory.retain_batch_async = AsyncMock()
-    mock_memory.submit_async_retain = AsyncMock()
-    mock_memory.recall_async = AsyncMock()
-    mock_memory.reflect_async = AsyncMock()
-    mock_memory.list_banks = AsyncMock(return_value=[])
-
-    mock_memory._operation_validator = FilteringValidator({})
-
-    mock_config = {"mcp_enabled_tools": ["recall", "retain", "reflect"]}
-    mock_memory._config_resolver = MagicMock()
-    mock_memory._config_resolver.get_bank_config = AsyncMock(return_value=mock_config)
-
-    mcp_server = create_mcp_server(mock_memory, multi_bank=False)
-
-    bank_token = _current_bank_id.set("test-bank")
-    api_key_token = _current_api_key.set("hsk_test")
-    tenant_token = _current_tenant_id.set("alice")
-    key_id_token = _current_api_key_id.set("key-1")
-    mcp_auth_token = _current_mcp_authenticated.set(False)
-    try:
-        if hasattr(mcp_server, "list_tools"):
-            tools = await mcp_server.list_tools()
-            tool_names = {t.name for t in tools}
-        else:
-            tools = await mcp_server._tool_manager.get_tools()
-            tool_names = set(tools.keys())
-
-        assert tool_names == {"recall", "reflect"}
-    finally:
-        _current_bank_id.reset(bank_token)
-        _current_api_key.reset(api_key_token)
-        _current_tenant_id.reset(tenant_token)
-        _current_api_key_id.reset(key_id_token)
-        _current_mcp_authenticated.reset(mcp_auth_token)
-
-
-@pytest.mark.asyncio
-async def test_validator_cannot_add_tools_beyond_bank_config():
-    """Validator returning tools not in bank config doesn't expand the set."""
-
-    class PermissiveValidator(OperationValidatorExtension):
-        async def validate_retain(self, ctx):
-            return ValidationResult.accept()
-        async def validate_recall(self, ctx):
-            return ValidationResult.accept()
-        async def validate_reflect(self, ctx):
-            return ValidationResult.accept()
-        async def filter_mcp_tools(self, bank_id, request_context, tools):
-            return tools | {"retain", "delete_bank"}
-
-    mock_memory = MagicMock()
-    mock_memory._tenant_extension = MagicMock()
-    mock_memory._tenant_extension.authenticate_mcp = AsyncMock()
-    mock_memory.retain_batch_async = AsyncMock()
-    mock_memory.submit_async_retain = AsyncMock()
-    mock_memory.recall_async = AsyncMock()
-    mock_memory.reflect_async = AsyncMock()
-    mock_memory.list_banks = AsyncMock(return_value=[])
-
-    mock_memory._operation_validator = PermissiveValidator({})
-
-    mock_config = {"mcp_enabled_tools": ["recall"]}
-    mock_memory._config_resolver = MagicMock()
-    mock_memory._config_resolver.get_bank_config = AsyncMock(return_value=mock_config)
-
-    mcp_server = create_mcp_server(mock_memory, multi_bank=False)
-
-    bank_token = _current_bank_id.set("test-bank")
-    api_key_token = _current_api_key.set("hsk_test")
-    tenant_token = _current_tenant_id.set("alice")
-    key_id_token = _current_api_key_id.set("key-1")
-    mcp_auth_token = _current_mcp_authenticated.set(False)
-    try:
-        if hasattr(mcp_server, "list_tools"):
-            tools = await mcp_server.list_tools()
-            tool_names = {t.name for t in tools}
-        else:
-            tools = await mcp_server._tool_manager.get_tools()
-            tool_names = set(tools.keys())
-
-        assert "recall" in tool_names
-        assert "retain" not in tool_names
-        assert "delete_bank" not in tool_names
-    finally:
-        _current_bank_id.reset(bank_token)
-        _current_api_key.reset(api_key_token)
-        _current_tenant_id.reset(tenant_token)
-        _current_api_key_id.reset(key_id_token)
-        _current_mcp_authenticated.reset(mcp_auth_token)
-
-
-@pytest.mark.asyncio
-async def test_validator_exception_fails_open(caplog):
-    """If filter_mcp_tools raises, all tools remain visible and warning is logged."""
-    import logging
-    caplog.set_level(logging.WARNING)
-
-    class BrokenValidator(OperationValidatorExtension):
-        async def validate_retain(self, ctx):
-            return ValidationResult.accept()
-        async def validate_recall(self, ctx):
-            return ValidationResult.accept()
-        async def validate_reflect(self, ctx):
-            return ValidationResult.accept()
-        async def filter_mcp_tools(self, bank_id, request_context, tools):
-            raise RuntimeError("Policy backend unreachable")
-
-    mock_memory = MagicMock()
-    mock_memory._tenant_extension = MagicMock()
-    mock_memory._tenant_extension.authenticate_mcp = AsyncMock()
-    mock_memory.retain_batch_async = AsyncMock()
-    mock_memory.submit_async_retain = AsyncMock()
-    mock_memory.recall_async = AsyncMock()
-    mock_memory.reflect_async = AsyncMock()
-    mock_memory.list_banks = AsyncMock(return_value=[])
-
-    mock_memory._operation_validator = BrokenValidator({})
-    mock_config = {"mcp_enabled_tools": None}
-    mock_memory._config_resolver = MagicMock()
-    mock_memory._config_resolver.get_bank_config = AsyncMock(return_value=mock_config)
-
-    mcp_server = create_mcp_server(mock_memory, multi_bank=False)
-
-    bank_token = _current_bank_id.set("test-bank")
-    api_key_token = _current_api_key.set("hsk_test")
-    tenant_token = _current_tenant_id.set("alice")
-    key_id_token = _current_api_key_id.set("key-1")
-    mcp_auth_token = _current_mcp_authenticated.set(False)
-    try:
-        if hasattr(mcp_server, "list_tools"):
-            tools = await mcp_server.list_tools()
-            tool_names = {t.name for t in tools}
-        else:
-            tools = await mcp_server._tool_manager.get_tools()
-            tool_names = set(tools.keys())
-
-        assert "retain" in tool_names
-        assert "recall" in tool_names
-        assert "reflect" in tool_names
-
-        assert any("filter_mcp_tools raised" in r.message for r in caplog.records)
-    finally:
-        _current_bank_id.reset(bank_token)
-        _current_api_key.reset(api_key_token)
-        _current_tenant_id.reset(tenant_token)
-        _current_api_key_id.reset(key_id_token)
-        _current_mcp_authenticated.reset(mcp_auth_token)
-
-
-@pytest.mark.asyncio
-async def test_no_validator_returns_unfiltered():
-    """Without an operation validator, tools/list returns all tools."""
-    mock_memory = MagicMock()
-    mock_memory._tenant_extension = MagicMock()
-    mock_memory._tenant_extension.authenticate_mcp = AsyncMock()
-    mock_memory.retain_batch_async = AsyncMock()
-    mock_memory.submit_async_retain = AsyncMock()
-    mock_memory.recall_async = AsyncMock()
-    mock_memory.reflect_async = AsyncMock()
-    mock_memory.list_banks = AsyncMock(return_value=[])
-
-    mock_memory._operation_validator = None
-    mock_config = {"mcp_enabled_tools": None}
-    mock_memory._config_resolver = MagicMock()
-    mock_memory._config_resolver.get_bank_config = AsyncMock(return_value=mock_config)
-
-    mcp_server = create_mcp_server(mock_memory, multi_bank=False)
-
-    bank_token = _current_bank_id.set("test-bank")
-    api_key_token = _current_api_key.set("hsk_test")
-    tenant_token = _current_tenant_id.set("alice")
-    key_id_token = _current_api_key_id.set("key-1")
-    mcp_auth_token = _current_mcp_authenticated.set(False)
-    try:
-        if hasattr(mcp_server, "list_tools"):
-            tools = await mcp_server.list_tools()
-            tool_names = {t.name for t in tools}
-        else:
-            tools = await mcp_server._tool_manager.get_tools()
-            tool_names = set(tools.keys())
-
-        assert "retain" in tool_names
-        assert "recall" in tool_names
-        assert "reflect" in tool_names
-    finally:
-        _current_bank_id.reset(bank_token)
-        _current_api_key.reset(api_key_token)
-        _current_tenant_id.reset(tenant_token)
-        _current_api_key_id.reset(key_id_token)
-        _current_mcp_authenticated.reset(mcp_auth_token)

hindsight-api-slim/tests/test_migration_backsweep.py

@@ -1,154 +0,0 @@
-"""Tests for migration g7h8i9j0k1l2 (backsweep orphaned memory_units).
-
-Uses a dedicated pg0 instance (port 5562) so the test can control exactly
-which migrations have run before inserting the orphan seed data.
-"""
-
-import asyncio
-import uuid
-from pathlib import Path
-
-import pytest
-from alembic import command
-from alembic.config import Config
-from sqlalchemy import create_engine, text
-
-# ---------------------------------------------------------------------------
-# Helpers
-# ---------------------------------------------------------------------------
-
-_SCRIPT_LOCATION = str(Path(__file__).parent.parent / "hindsight_api" / "alembic")
-
-
-def _alembic_cfg(db_url: str) -> Config:
-    cfg = Config()
-    cfg.set_main_option("script_location", _SCRIPT_LOCATION)
-    cfg.set_main_option("sqlalchemy.url", db_url)
-    cfg.set_main_option("prepend_sys_path", ".")
-    cfg.set_main_option("path_separator", "os")
-    return cfg
-
-
-def _upgrade(db_url: str, revision: str) -> None:
-    command.upgrade(_alembic_cfg(db_url), revision)
-
-
-# ---------------------------------------------------------------------------
-# Fixture: fresh database at the revision just before the backsweep
-# ---------------------------------------------------------------------------
-
-@pytest.fixture(scope="module")
-def pre_backsweep_db_url():
-    """
-    Spin up a dedicated pg0 instance and run all migrations up to (but not
-    including) the backsweep revision so each test can seed orphan data and
-    then apply the backsweep itself.
-    """
-    from hindsight_api.pg0 import EmbeddedPostgres
-
-    pg0 = EmbeddedPostgres(name="hindsight-backsweep-test", port=5562)
-    loop = asyncio.new_event_loop()
-    try:
-        url = loop.run_until_complete(pg0.ensure_running())
-    finally:
-        loop.close()
-
-    # Migrate up to the revision just before the backsweep.
-    _upgrade(url, "f6g7h8i9j0k1")
-    return url
-
-
-# ---------------------------------------------------------------------------
-# The test
-# ---------------------------------------------------------------------------
-
-def test_backsweep_removes_orphans_and_preserves_legit_rows(pre_backsweep_db_url):
-    """
-    Seed four kinds of rows then apply the backsweep migration and verify:
-
-    Rows that MUST be deleted
-    ─────────────────────────
-    A. Any fact_type, bank_id missing from banks
-       → Pass 1 deletes these regardless of fact_type or source links.
-
-    B. observation, bank exists, but ALL source_memory_ids are gone
-       → Pass 2 deletes these.
-
-    Rows that MUST survive
-    ──────────────────────
-    C. observation, bank exists, at least ONE source_memory_id still live
-       → Pass 2 must not touch these.
-
-    D. Non-observation (world), bank exists, no sources (not relevant)
-       → Pass 1 must not touch these (bank exists).
-    """
-    db_url = pre_backsweep_db_url
-    engine = create_engine(db_url)
-
-    alive_bank = f"bank_{uuid.uuid4().hex[:8]}"
-    ghost_bank = f"bank_{uuid.uuid4().hex[:8]}"  # never inserted into banks
-
-    # UUIDs for memory units
-    id_pass1_world = uuid.uuid4()       # A: world unit, ghost bank
-    id_pass1_obs = uuid.uuid4()         # A: observation, ghost bank
-    id_pass2_obs = uuid.uuid4()         # B: observation, all sources gone
-    id_keep_obs = uuid.uuid4()          # C: observation with one live source
-    id_keep_world = uuid.uuid4()        # D: world unit, alive bank
-    id_live_source = uuid.uuid4()       # live source for C
-
-    with engine.connect() as conn:
-        # --- banks ---
-        conn.execute(text("INSERT INTO banks (bank_id) VALUES (:b)"), {"b": alive_bank})
-
-        # --- seed memory_units ---
-        def insert_mu(uid, bank, fact_type, sources=None):
-            src_arr = "{" + ",".join(str(s) for s in (sources or [])) + "}"
-            conn.execute(
-                text(
-                    """
-                    INSERT INTO memory_units
-                        (id, bank_id, text, fact_type, source_memory_ids)
-                    VALUES
-                        (:id, :bank, :text, :ft, CAST(:src AS uuid[]))
-                    """
-                ),
-                {"id": uid, "bank": bank, "text": "test", "ft": fact_type, "src": src_arr},
-            )
-
-        # A: ghost-bank rows (Pass 1 targets)
-        insert_mu(id_pass1_world, ghost_bank, "world")
-        insert_mu(id_pass1_obs, ghost_bank, "observation", sources=[uuid.uuid4()])
-
-        # B: observation with all-dead sources (Pass 2 target)
-        insert_mu(id_pass2_obs, alive_bank, "observation", sources=[uuid.uuid4(), uuid.uuid4()])
-
-        # C: observation with one live source (must survive)
-        insert_mu(id_live_source, alive_bank, "world")
-        insert_mu(id_keep_obs, alive_bank, "observation", sources=[id_live_source, uuid.uuid4()])
-
-        # D: world unit in alive bank (must survive)
-        insert_mu(id_keep_world, alive_bank, "world")
-
-        conn.commit()
-
-    # --- apply the backsweep ---
-    _upgrade(db_url, "g7h8i9j0k1l2")
-
-    # --- verify ---
-    with engine.connect() as conn:
-        def exists(uid):
-            return conn.execute(
-                text("SELECT 1 FROM memory_units WHERE id = :id"), {"id": uid}
-            ).fetchone() is not None
-
-        # Must be gone
-        assert not exists(id_pass1_world), "Pass 1: world unit with ghost bank should be deleted"
-        assert not exists(id_pass1_obs), "Pass 1: observation with ghost bank should be deleted"
-        assert not exists(id_pass2_obs), "Pass 2: observation with all-dead sources should be deleted"
-
-        # Must survive
-        assert exists(id_keep_obs), "observation with a live source must not be deleted"
-        assert exists(id_keep_world), "world unit in alive bank must not be deleted"
-        assert exists(id_live_source), "live source memory unit must not be deleted"
-
-    engine.dispose()

hindsight-api-slim/tests/test_none_llm_provider.py

@@ -1,232 +0,0 @@
-"""
-Tests for the 'none' LLM provider mode.
-
-Verifies that when HINDSIGHT_API_LLM_PROVIDER=none:
-- Retain defaults to chunks mode (no LLM calls)
-- Reflect returns 400
-- Mental model refresh returns 400
-- Consolidation is skipped
-- NoneLLM.call() raises LLMNotAvailableError
-"""
-
-import os
-from datetime import datetime, timezone
-
-import httpx
-import pytest
-import pytest_asyncio
-
-from hindsight_api import LLMConfig, LocalSTEmbeddings, MemoryEngine, RequestContext
-from hindsight_api.api import create_app
-from hindsight_api.engine.cross_encoder import LocalSTCrossEncoder
-from hindsight_api.engine.memory_engine import Budget
-from hindsight_api.engine.providers.none_llm import LLMNotAvailableError, NoneLLM
-from hindsight_api.engine.query_analyzer import DateparserQueryAnalyzer
-from hindsight_api.engine.task_backend import SyncTaskBackend
-
-
-@pytest.fixture(scope="function")
-def request_context():
-    return RequestContext()
-
-
-@pytest_asyncio.fixture(scope="function")
-async def none_memory(pg0_db_url, embeddings, cross_encoder, query_analyzer):
-    """MemoryEngine with provider=none."""
-    mem = MemoryEngine(
-        db_url=pg0_db_url,
-        memory_llm_provider="none",
-        memory_llm_api_key=None,
-        memory_llm_model="none",
-        embeddings=embeddings,
-        cross_encoder=cross_encoder,
-        query_analyzer=query_analyzer,
-        pool_min_size=1,
-        pool_max_size=5,
-        run_migrations=False,
-        task_backend=SyncTaskBackend(),
-    )
-    await mem.initialize()
-    yield mem
-    try:
-        if mem._pool and not mem._pool._closing:
-            await mem.close()
-    except Exception:
-        pass
-
-
-@pytest_asyncio.fixture
-async def none_api_client(none_memory):
-    """HTTP test client backed by a none-provider MemoryEngine."""
-    app = create_app(none_memory, initialize_memory=False)
-    transport = httpx.ASGITransport(app=app)
-    async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
-        yield client
-
-
-# -- Unit tests for NoneLLM ---------------------------------------------------
-
-
-@pytest.mark.asyncio
-async def test_none_llm_call_raises():
-    """NoneLLM.call() should raise LLMNotAvailableError."""
-    llm = NoneLLM(provider="none", api_key="", base_url="", model="none")
-    with pytest.raises(LLMNotAvailableError):
-        await llm.call(messages=[{"role": "user", "content": "hello"}])
-
-
-@pytest.mark.asyncio
-async def test_none_llm_call_with_tools_raises():
-    """NoneLLM.call_with_tools() should raise LLMNotAvailableError."""
-    llm = NoneLLM(provider="none", api_key="", base_url="", model="none")
-    with pytest.raises(LLMNotAvailableError):
-        await llm.call_with_tools(
-            messages=[{"role": "user", "content": "hello"}],
-            tools=[{"type": "function", "function": {"name": "test", "parameters": {}}}],
-        )
-
-
-@pytest.mark.asyncio
-async def test_none_llm_verify_connection_succeeds():
-    """NoneLLM.verify_connection() should be a no-op."""
-    llm = NoneLLM(provider="none", api_key="", base_url="", model="none")
-    await llm.verify_connection()  # Should not raise
-
-
-# -- Config validation tests ---------------------------------------------------
-
-
-def test_config_forces_chunks_mode():
-    """When provider is 'none', config.validate() forces retain_extraction_mode='chunks'."""
-    from hindsight_api.config import HindsightConfig
-
-    config = HindsightConfig.from_env()
-    # Override to none for test
-    config.llm_provider = "none"
-    config.retain_extraction_mode = "facts"
-    config.enable_observations = True
-    config.validate()
-
-    assert config.retain_extraction_mode == "chunks"
-    assert config.enable_observations is False
-
-
-# -- Integration tests (require database) -------------------------------------
-
-
-@pytest.mark.asyncio
-async def test_retain_works_with_none_provider(none_memory, request_context):
-    """Retain should work with provider=none, storing chunks without LLM calls."""
-    bank_id = f"test_none_retain_{datetime.now(timezone.utc).timestamp()}"
-
-    unit_ids = await none_memory.retain_async(
-        bank_id=bank_id,
-        content="Alice is a software engineer. She works at TechCorp and loves Python.",
-        context="team info",
-        request_context=request_context,
-    )
-
-    assert len(unit_ids) > 0, "Should store chunks even without an LLM"
-
-
-@pytest.mark.asyncio
-async def test_recall_works_with_none_provider(none_memory, request_context):
-    """Recall should work with provider=none (uses embeddings, not LLM)."""
-    bank_id = f"test_none_recall_{datetime.now(timezone.utc).timestamp()}"
-
-    await none_memory.retain_async(
-        bank_id=bank_id,
-        content="Alice is a software engineer at TechCorp.",
-        context="team info",
-        request_context=request_context,
-    )
-
-    result = await none_memory.recall_async(
-        bank_id=bank_id,
-        query="Who is Alice?",
-        budget=Budget.LOW,
-        request_context=request_context,
-    )
-
-    assert len(result.results) > 0, "Should find results via semantic search"
-
-
-@pytest.mark.asyncio
-async def test_reflect_raises_with_none_provider(none_memory, request_context):
-    """Reflect should raise LLMNotAvailableError with provider=none."""
-    bank_id = f"test_none_reflect_{datetime.now(timezone.utc).timestamp()}"
-
-    with pytest.raises(LLMNotAvailableError):
-        await none_memory.reflect_async(
-            bank_id=bank_id,
-            query="What do you know?",
-            request_context=request_context,
-        )
-
-
-@pytest.mark.asyncio
-async def test_consolidation_skipped_with_none_provider(none_memory, request_context):
-    """Consolidation handler should skip when provider=none."""
-    result = await none_memory._handle_consolidation({"bank_id": "test_bank"})
-    assert result["skipped"] is True
-    assert result["memories_processed"] == 0
-
-
-@pytest.mark.asyncio
-async def test_mental_model_refresh_raises_with_none_provider(none_memory, request_context):
-    """Mental model refresh should raise LLMNotAvailableError with provider=none."""
-    bank_id = f"test_none_mm_{datetime.now(timezone.utc).timestamp()}"
-
-    with pytest.raises(LLMNotAvailableError):
-        await none_memory.submit_async_refresh_mental_model(
-            bank_id=bank_id,
-            mental_model_id="fake-id",
-            request_context=request_context,
-        )
-
-
-# -- HTTP API tests -----------------------------------------------------------
-
-
-@pytest.mark.asyncio
-async def test_http_reflect_returns_400(none_api_client):
-    """Reflect endpoint should return 400 when LLM provider is none."""
-    bank_id = f"test_none_http_{datetime.now(timezone.utc).timestamp()}"
-
-    response = await none_api_client.post(
-        f"/v1/default/banks/{bank_id}/reflect",
-        json={"query": "What do you know?"},
-    )
-    assert response.status_code == 400
-    assert "none" in response.json()["detail"].lower()
-
-
-@pytest.mark.asyncio
-async def test_http_retain_works(none_api_client):
-    """Retain endpoint should work with provider=none (chunks mode)."""
-    bank_id = f"test_none_http_retain_{datetime.now(timezone.utc).timestamp()}"
-
-    response = await none_api_client.post(
-        f"/v1/default/banks/{bank_id}/memories",
-        json={"items": [{"content": "Hello world", "context": "test"}]},
-    )
-    assert response.status_code == 200
-
-
-@pytest.mark.asyncio
-async def test_http_recall_works(none_api_client):
-    """Recall endpoint should work with provider=none."""
-    bank_id = f"test_none_http_recall_{datetime.now(timezone.utc).timestamp()}"
-
-    # Retain first
-    await none_api_client.post(
-        f"/v1/default/banks/{bank_id}/memories",
-        json={"items": [{"content": "Alice is an engineer.", "context": "test"}]},
-    )
-
-    # Recall
-    response = await none_api_client.post(
-        f"/v1/default/banks/{bank_id}/memories/recall",
-        json={"query": "Alice"},
-    )
-    assert response.status_code == 200

hindsight-api-slim/tests/test_op_cancellation.py

@@ -1,311 +0,0 @@
-"""Tests for operation cancellation when a bank is deleted.
-
-Covers:
-- CASCADE DELETE: deleting a bank removes async_operations and webhooks rows
-- _check_op_alive: returns True when op exists, False when deleted
-- _mark_operation_completed / _mark_operation_failed: graceful no-op when row is gone
-- Consolidation checkpoint: stops early after a batch commit if op was deleted
-- Retain checkpoint: stops between sub-batches if op was deleted
-"""
-
-import uuid
-from unittest.mock import AsyncMock, patch
-
-import pytest
-import pytest_asyncio
-
-from hindsight_api.engine.memory_engine import MemoryEngine
-
-
-pytestmark = pytest.mark.xdist_group("op_cancellation_tests")
-
-_BANK_PREFIX = "test-op-cancel"
-
-
-@pytest_asyncio.fixture
-async def pool(pg0_db_url):
-    import asyncpg
-    from hindsight_api.pg0 import resolve_database_url
-
-    resolved_url = await resolve_database_url(pg0_db_url)
-    p = await asyncpg.create_pool(resolved_url, min_size=1, max_size=5, command_timeout=30)
-    yield p
-    await p.close()
-
-
-@pytest_asyncio.fixture(autouse=True)
-async def cleanup(pool):
-    """Remove test rows before and after each test."""
-    await pool.execute(f"DELETE FROM banks WHERE bank_id LIKE '{_BANK_PREFIX}%'")
-    yield
-    await pool.execute(f"DELETE FROM banks WHERE bank_id LIKE '{_BANK_PREFIX}%'")
-
-
-async def _insert_bank(pool, bank_id: str):
-    await pool.execute(
-        "INSERT INTO banks (bank_id, name) VALUES ($1, $2) ON CONFLICT DO NOTHING",
-        bank_id,
-        bank_id,
-    )
-
-
-async def _insert_op(pool, bank_id: str, op_id: uuid.UUID | None = None) -> uuid.UUID:
-    op_id = op_id or uuid.uuid4()
-    await pool.execute(
-        """
-        INSERT INTO async_operations (operation_id, bank_id, operation_type, status)
-        VALUES ($1, $2, 'consolidation', 'processing')
-        """,
-        op_id,
-        bank_id,
-    )
-    return op_id
-
-
-# ---------------------------------------------------------------------------
-# CASCADE DELETE tests
-# ---------------------------------------------------------------------------
-
-
-class TestCascadeDeleteOnBankDeletion:
-    @pytest.mark.asyncio
-    async def test_bank_deletion_cascades_to_async_operations(self, pool):
-        bank_id = f"{_BANK_PREFIX}-{uuid.uuid4().hex[:8]}"
-        await _insert_bank(pool, bank_id)
-        op_id = await _insert_op(pool, bank_id)
-
-        # Verify op exists
-        row = await pool.fetchrow("SELECT operation_id FROM async_operations WHERE operation_id = $1", op_id)
-        assert row is not None
-
-        # Delete the bank — should cascade to async_operations
-        await pool.execute("DELETE FROM banks WHERE bank_id = $1", bank_id)
-
-        row = await pool.fetchrow("SELECT operation_id FROM async_operations WHERE operation_id = $1", op_id)
-        assert row is None, "async_operations row should be deleted by CASCADE"
-
-    @pytest.mark.asyncio
-    async def test_bank_deletion_cascades_to_webhooks(self, pool):
-        bank_id = f"{_BANK_PREFIX}-{uuid.uuid4().hex[:8]}"
-        await _insert_bank(pool, bank_id)
-        webhook_id = uuid.uuid4()
-        await pool.execute(
-            """
-            INSERT INTO webhooks (id, bank_id, url, event_types)
-            VALUES ($1, $2, 'https://example.com/hook', '{}')
-            """,
-            webhook_id,
-            bank_id,
-        )
-
-        row = await pool.fetchrow("SELECT id FROM webhooks WHERE id = $1", webhook_id)
-        assert row is not None
-
-        await pool.execute("DELETE FROM banks WHERE bank_id = $1", bank_id)
-
-        row = await pool.fetchrow("SELECT id FROM webhooks WHERE id = $1", webhook_id)
-        assert row is None, "webhooks row should be deleted by CASCADE"
-
-
-# ---------------------------------------------------------------------------
-# _check_op_alive tests
-# ---------------------------------------------------------------------------
-
-
-class TestCheckOpAlive:
-    @pytest.mark.asyncio
-    async def test_returns_true_when_op_exists(self, memory: MemoryEngine, request_context):
-        bank_id = f"{_BANK_PREFIX}-{uuid.uuid4().hex[:8]}"
-        await memory.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-        op_id = uuid.uuid4()
-        async with memory._pool.acquire() as conn:
-            await conn.execute(
-                """
-                INSERT INTO async_operations (operation_id, bank_id, operation_type, status)
-                VALUES ($1, $2, 'consolidation', 'processing')
-                """,
-                op_id,
-                bank_id,
-            )
-
-        assert await memory._check_op_alive(str(op_id)) is True
-
-    @pytest.mark.asyncio
-    async def test_returns_false_when_op_deleted(self, memory: MemoryEngine, request_context):
-        bank_id = f"{_BANK_PREFIX}-{uuid.uuid4().hex[:8]}"
-        await memory.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-        op_id = uuid.uuid4()
-        async with memory._pool.acquire() as conn:
-            await conn.execute(
-                """
-                INSERT INTO async_operations (operation_id, bank_id, operation_type, status)
-                VALUES ($1, $2, 'consolidation', 'processing')
-                """,
-                op_id,
-                bank_id,
-            )
-            await conn.execute("DELETE FROM async_operations WHERE operation_id = $1", op_id)
-
-        assert await memory._check_op_alive(str(op_id)) is False
-
-    @pytest.mark.asyncio
-    async def test_returns_false_after_bank_cascade_delete(self, memory: MemoryEngine, request_context):
-        bank_id = f"{_BANK_PREFIX}-{uuid.uuid4().hex[:8]}"
-        await memory.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-        op_id = uuid.uuid4()
-        async with memory._pool.acquire() as conn:
-            await conn.execute(
-                """
-                INSERT INTO async_operations (operation_id, bank_id, operation_type, status)
-                VALUES ($1, $2, 'consolidation', 'processing')
-                """,
-                op_id,
-                bank_id,
-            )
-
-        # Delete the bank — cascades to the op row
-        await memory.delete_bank(bank_id=bank_id, request_context=request_context)
-
-        assert await memory._check_op_alive(str(op_id)) is False
-
-
-# ---------------------------------------------------------------------------
-# _mark_operation_completed / _mark_operation_failed graceful no-op
-# ---------------------------------------------------------------------------
-
-
-class TestMarkOperationGracefulOnMissingRow:
-    @pytest.mark.asyncio
-    async def test_mark_completed_does_not_raise_when_row_missing(self, memory: MemoryEngine):
-        # Row never existed — should log and return cleanly
-        missing_id = str(uuid.uuid4())
-        await memory._mark_operation_completed(missing_id)  # no exception
-
-    @pytest.mark.asyncio
-    async def test_mark_failed_does_not_raise_when_row_missing(self, memory: MemoryEngine):
-        missing_id = str(uuid.uuid4())
-        await memory._mark_operation_failed(missing_id, "some error", "traceback here")  # no exception
-
-    @pytest.mark.asyncio
-    async def test_mark_completed_and_fire_webhook_does_not_raise_when_row_missing(
-        self, memory: MemoryEngine
-    ):
-        missing_id = str(uuid.uuid4())
-        await memory._mark_operation_completed_and_fire_webhook(
-            operation_id=missing_id,
-            bank_id="nonexistent-bank",
-            status="completed",
-            result=None,
-        )  # no exception
-
-
-# ---------------------------------------------------------------------------
-# Consolidation checkpoint
-# ---------------------------------------------------------------------------
-
-
-class TestConsolidationCheckpoint:
-    @pytest.mark.asyncio
-    async def test_consolidation_stops_early_when_op_cancelled(self, memory: MemoryEngine, request_context):
-        """Consolidation returns 'cancelled' status after the first batch if _check_op_alive is False."""
-        from hindsight_api.config import _get_raw_config
-        from hindsight_api.engine.consolidation.consolidator import run_consolidation_job
-
-        config = _get_raw_config()
-        original = config.enable_observations
-        config.enable_observations = True
-
-        try:
-            bank_id = f"{_BANK_PREFIX}-{uuid.uuid4().hex[:8]}"
-            await memory.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-            # Insert a few unconsolidated memories directly so we control the batch without LLM
-            async with memory._pool.acquire() as conn:
-                for i in range(3):
-                    await conn.execute(
-                        """
-                        INSERT INTO memory_units
-                            (id, bank_id, text, fact_type, created_at, updated_at)
-                        VALUES (gen_random_uuid(), $1, $2, 'experience', NOW(), NOW())
-                        """,
-                        bank_id,
-                        f"Test memory {i} for cancellation test",
-                    )
-
-            op_id = str(uuid.uuid4())
-            call_count = 0
-
-            async def _fake_check(operation_id: str) -> bool:
-                nonlocal call_count
-                call_count += 1
-                # Return False on the very first checkpoint call
-                return False
-
-            with patch.object(memory, "_check_op_alive", side_effect=_fake_check):
-                result = await run_consolidation_job(
-                    memory_engine=memory,
-                    bank_id=bank_id,
-                    request_context=request_context,
-                    operation_id=op_id,
-                )
-
-            assert result["status"] == "cancelled"
-            assert call_count >= 1
-        finally:
-            config.enable_observations = original
-
-
-# ---------------------------------------------------------------------------
-# Retain checkpoint
-# ---------------------------------------------------------------------------
-
-
-class TestRetainCheckpoint:
-    @pytest.mark.asyncio
-    async def test_retain_stops_between_sub_batches_when_cancelled(
-        self, memory: MemoryEngine, request_context
-    ):
-        """retain_batch_async returns partial results if _check_op_alive is False between sub-batches."""
-        from hindsight_api.config import _get_raw_config
-
-        bank_id = f"{_BANK_PREFIX}-{uuid.uuid4().hex[:8]}"
-        await memory.get_bank_profile(bank_id=bank_id, request_context=request_context)
-
-        # Force sub-batch splitting by temporarily lowering the token threshold
-        config = _get_raw_config()
-        original_tokens = config.retain_batch_tokens
-        # Set threshold very low so each item becomes its own sub-batch
-        config.retain_batch_tokens = 1
-
-        try:
-            op_id = str(uuid.uuid4())
-            check_calls = 0
-
-            async def _fake_check(operation_id: str) -> bool:
-                nonlocal check_calls
-                check_calls += 1
-                # Cancel after the first sub-batch completes
-                return check_calls <= 1
-
-            contents = [
-                {"content": f"Memory item {i} about something interesting."} for i in range(4)
-            ]
-
-            with patch.object(memory, "_check_op_alive", side_effect=_fake_check):
-                result = await memory.retain_batch_async(
-                    bank_id=bank_id,
-                    contents=contents,
-                    request_context=request_context,
-                    operation_id=op_id,
-                )
-
-            # Should have stopped early: fewer results than total items
-            assert len(result) < len(contents), (
-                f"Expected early stop but got {len(result)}/{len(contents)} results"
-            )
-            assert check_calls >= 1
-        finally:
-            config.retain_batch_tokens = original_tokens

hindsight-api-slim/tests/test_openai_max_tokens_param.py

@@ -1,72 +0,0 @@
-"""
-Tests for OpenAICompatibleLLM._max_tokens_param_name.
-
-Regression coverage for issue #978: Azure OpenAI + GPT-5 models were failing with
-"'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead."
-because PR #858 started sending 'max_tokens' whenever the openai provider had a
-custom base_url. Reasoning models only accept 'max_completion_tokens', and Azure
-OpenAI is fully OpenAI-API-compatible, so both cases must keep using the new
-parameter name.
-"""
-
-from hindsight_api.engine.providers.openai_compatible_llm import OpenAICompatibleLLM
-
-
-def _make(provider: str, model: str, base_url: str = "") -> OpenAICompatibleLLM:
-    return OpenAICompatibleLLM(
-        provider=provider,
-        api_key="test-key",
-        base_url=base_url,
-        model=model,
-    )
-
-
-class TestMaxTokensParamName:
-    def test_native_openai_uses_max_completion_tokens(self):
-        llm = _make("openai", "gpt-4o-mini")
-        assert llm._max_tokens_param_name() == "max_completion_tokens"
-
-    def test_openai_custom_base_url_falls_back_to_max_tokens(self):
-        """Mistral/Together-style OpenAI-compatible endpoints need max_tokens (PR #858)."""
-        llm = _make("openai", "mistral-large-latest", base_url="https://api.mistral.ai/v1")
-        assert llm._max_tokens_param_name() == "max_tokens"
-
-    def test_azure_openai_uses_max_completion_tokens(self):
-        """Regression for #978: Azure is fully OpenAI-API-compatible, not a third-party clone."""
-        llm = _make(
-            "openai",
-            "gpt-4o-mini",
-            base_url="https://my-resource.openai.azure.com/openai/v1/",
-        )
-        assert llm._max_tokens_param_name() == "max_completion_tokens"
-
-    def test_reasoning_model_always_uses_max_completion_tokens(self):
-        """Regression for #978: GPT-5/o1/o3 reject max_tokens outright, base_url must not matter."""
-        # Azure + GPT-5 (exact reporter setup)
-        azure_gpt5 = _make(
-            "openai",
-            "gpt-5.4-nano",
-            base_url="https://my-resource.openai.azure.com/openai/v1/",
-        )
-        assert azure_gpt5._max_tokens_param_name() == "max_completion_tokens"
-
-        # Even a Mistral-style custom base_url must not downgrade a reasoning model
-        for model in ("gpt-5", "gpt-5-mini", "o1-mini", "o3", "deepseek-r1"):
-            llm = _make("openai", model, base_url="https://some-proxy.example.com/v1")
-            assert llm._max_tokens_param_name() == "max_completion_tokens", model
-
-    def test_groq_uses_max_completion_tokens(self):
-        llm = _make("groq", "openai/gpt-oss-120b", base_url="https://api.groq.com/openai/v1")
-        assert llm._max_tokens_param_name() == "max_completion_tokens"
-
-    def test_llamacpp_uses_max_completion_tokens(self):
-        llm = _make("llamacpp", "some-model", base_url="http://localhost:8080/v1")
-        assert llm._max_tokens_param_name() == "max_completion_tokens"
-
-    def test_ollama_uses_max_tokens(self):
-        llm = _make("ollama", "gemma3:12b", base_url="http://localhost:11434/v1")
-        assert llm._max_tokens_param_name() == "max_tokens"
-
-    def test_lmstudio_uses_max_tokens(self):
-        llm = _make("lmstudio", "openai/gpt-oss-20b", base_url="http://localhost:1234/v1")
-        assert llm._max_tokens_param_name() == "max_tokens"

hindsight-api-slim/tests/test_reflect_internal_billing.py

@@ -1,80 +0,0 @@
-"""Regression test for #972: reflect sub-recalls must be marked internal.
-
-When reflect calls search_observations or recall, the sub-recalls must use
-``request_context.internal=True`` to avoid double-billing.  The reflect caller
-is already billed for the overall operation; sub-recalls are implementation
-details that should not generate additional billing events.
-"""
-
-from dataclasses import dataclass, field
-from unittest.mock import AsyncMock, MagicMock
-
-import pytest
-
-from hindsight_api.engine.reflect.tools import tool_recall, tool_search_observations
-from hindsight_api.engine.response_models import RecallResult
-
-
-@dataclass
-class _FakeRequestContext:
-    """Dataclass stand-in matching the fields used by ``dataclasses.replace``."""
-
-    api_key: str | None = None
-    api_key_id: str | None = None
-    tenant_id: str | None = None
-    internal: bool = False
-    mcp_authenticated: bool = False
-    user_initiated: bool = False
-    allowed_bank_ids: list[str] | None = None
-
-
-def _mock_engine():
-    engine = MagicMock()
-    engine.recall_async = AsyncMock(
-        return_value=RecallResult(results=[], source_facts={})
-    )
-    return engine
-
-
-class TestReflectInternalBilling:
-    """Verify that reflect sub-recalls are marked internal (#972)."""
-
-    @pytest.mark.asyncio
-    async def test_search_observations_marks_recall_internal(self):
-        engine = _mock_engine()
-        ctx = _FakeRequestContext(api_key="k", internal=False)
-
-        await tool_search_observations(engine, "bank-1", "query", ctx)
-
-        engine.recall_async.assert_called_once()
-        passed_ctx = engine.recall_async.call_args.kwargs["request_context"]
-        assert passed_ctx.internal is True, "sub-recall must be internal"
-
-    @pytest.mark.asyncio
-    async def test_search_observations_preserves_original_context(self):
-        engine = _mock_engine()
-        ctx = _FakeRequestContext(api_key="k", internal=False)
-
-        await tool_search_observations(engine, "bank-1", "query", ctx)
-
-        assert ctx.internal is False, "original context must not be mutated"
-
-    @pytest.mark.asyncio
-    async def test_recall_marks_recall_internal(self):
-        engine = _mock_engine()
-        ctx = _FakeRequestContext(api_key="k", internal=False)
-
-        await tool_recall(engine, "bank-1", "query", ctx)
-
-        engine.recall_async.assert_called_once()
-        passed_ctx = engine.recall_async.call_args.kwargs["request_context"]
-        assert passed_ctx.internal is True, "sub-recall must be internal"
-
-    @pytest.mark.asyncio
-    async def test_recall_preserves_original_context(self):
-        engine = _mock_engine()
-        ctx = _FakeRequestContext(api_key="k", internal=False)
-
-        await tool_recall(engine, "bank-1", "query", ctx)
-
-        assert ctx.internal is False, "original context must not be mutated"

hindsight-api-slim/tests/test_reflect_source_facts_config.py

@@ -1,134 +0,0 @@
-"""
-Tests for reflect search_observations source_facts_max_tokens configuration.
-
-Verifies that the source_facts_max_tokens parameter correctly controls
-whether source facts are included in search_observations recall calls.
-"""
-
-from unittest.mock import AsyncMock, MagicMock, patch
-
-import pytest
-
-from hindsight_api.engine.reflect.tools import tool_search_observations
-from hindsight_api.engine.response_models import RecallResult
-from hindsight_api.models import RequestContext
-
-
-def _make_mock_engine(recall_result=None):
-    """Create a mock memory engine with a recall_async method."""
-    if recall_result is None:
-        recall_result = RecallResult(results=[], source_facts={})
-    engine = MagicMock()
-    engine.recall_async = AsyncMock(return_value=recall_result)
-    return engine
-
-
-@pytest.fixture
-def mock_request_context():
-    # Use a real dataclass instance — tool_search_observations calls
-    # dataclasses.replace(request_context, internal=True), which fails on
-    # MagicMock. The fields don't matter for these tests; we only inspect
-    # the kwargs passed to the mocked recall_async.
-    return RequestContext()
-
-
-class TestSearchObservationsSourceFacts:
-    """Test source_facts_max_tokens parameter in tool_search_observations."""
-
-    @pytest.mark.asyncio
-    async def test_default_disables_source_facts(self, mock_request_context):
-        """Default source_facts_max_tokens=-1 should disable source facts."""
-        engine = _make_mock_engine()
-
-        await tool_search_observations(
-            engine, "bank-1", "test query", mock_request_context
-        )
-
-        engine.recall_async.assert_called_once()
-        call_kwargs = engine.recall_async.call_args.kwargs
-        assert call_kwargs["include_source_facts"] is False
-        assert "max_source_facts_tokens" not in call_kwargs
-
-    @pytest.mark.asyncio
-    async def test_zero_enables_source_facts_unlimited(self, mock_request_context):
-        """source_facts_max_tokens=0 should enable source facts with no token limit."""
-        engine = _make_mock_engine()
-
-        await tool_search_observations(
-            engine, "bank-1", "test query", mock_request_context,
-            source_facts_max_tokens=0,
-        )
-
-        engine.recall_async.assert_called_once()
-        call_kwargs = engine.recall_async.call_args.kwargs
-        assert call_kwargs["include_source_facts"] is True
-        assert "max_source_facts_tokens" not in call_kwargs
-
-    @pytest.mark.asyncio
-    async def test_positive_enables_source_facts_with_limit(self, mock_request_context):
-        """source_facts_max_tokens>0 should enable source facts with a token budget."""
-        engine = _make_mock_engine()
-
-        await tool_search_observations(
-            engine, "bank-1", "test query", mock_request_context,
-            source_facts_max_tokens=5000,
-        )
-
-        engine.recall_async.assert_called_once()
-        call_kwargs = engine.recall_async.call_args.kwargs
-        assert call_kwargs["include_source_facts"] is True
-        assert call_kwargs["max_source_facts_tokens"] == 5000
-
-    @pytest.mark.asyncio
-    async def test_negative_one_disables_source_facts(self, mock_request_context):
-        """Explicit -1 should disable source facts (same as default)."""
-        engine = _make_mock_engine()
-
-        await tool_search_observations(
-            engine, "bank-1", "test query", mock_request_context,
-            source_facts_max_tokens=-1,
-        )
-
-        engine.recall_async.assert_called_once()
-        call_kwargs = engine.recall_async.call_args.kwargs
-        assert call_kwargs["include_source_facts"] is False
-        assert "max_source_facts_tokens" not in call_kwargs
-
-
-class TestReflectSourceFactsConfig:
-    """Test that reflect_source_facts_max_tokens is properly wired in HindsightConfig."""
-
-    def test_config_field_exists(self):
-        """reflect_source_facts_max_tokens should be a valid config field."""
-        from hindsight_api.config import HindsightConfig
-
-        import dataclasses
-
-        field_names = {f.name for f in dataclasses.fields(HindsightConfig)}
-        assert "reflect_source_facts_max_tokens" in field_names
-
-    def test_config_is_configurable(self):
-        """reflect_source_facts_max_tokens should be a configurable (per-bank) field."""
-        from hindsight_api.config import HindsightConfig
-
-        assert "reflect_source_facts_max_tokens" in HindsightConfig.get_configurable_fields()
-
-    def test_default_value_is_disabled(self):
-        """Default should be -1 (disabled)."""
-        from hindsight_api.config import DEFAULT_REFLECT_SOURCE_FACTS_MAX_TOKENS
-
-        assert DEFAULT_REFLECT_SOURCE_FACTS_MAX_TOKENS == -1
-
-    def test_env_var_constant_exists(self):
-        """Env var constant should be defined."""
-        from hindsight_api.config import ENV_REFLECT_SOURCE_FACTS_MAX_TOKENS
-
-        assert ENV_REFLECT_SOURCE_FACTS_MAX_TOKENS == "HINDSIGHT_API_REFLECT_SOURCE_FACTS_MAX_TOKENS"
-
-    @patch.dict("os.environ", {"HINDSIGHT_API_REFLECT_SOURCE_FACTS_MAX_TOKENS": "8000"})
-    def test_from_env_reads_value(self):
-        """from_env should parse the env var."""
-        from hindsight_api.config import HindsightConfig
-
-        config = HindsightConfig.from_env()
-        assert config.reflect_source_facts_max_tokens == 8000

hindsight-api-slim/tests/test_reranking_proof_count.py

@@ -1,91 +0,0 @@
-"""
-Unit tests for proof_count boost in reranking.
-"""
-
-from datetime import datetime, timezone
-import pytest
-from uuid import uuid4
-
-from hindsight_api.engine.search.types import RetrievalResult, MergedCandidate, ScoredResult
-from hindsight_api.engine.search.reranking import apply_combined_scoring
-
-UTC = timezone.utc
-
-def create_mock_scored_result(proof_count: int | None = None, ce_score: float = 0.8) -> ScoredResult:
-    """Helper to create a minimal ScoredResult suitable for scoring tests."""
-    retrieval = RetrievalResult(
-        id=str(uuid4()),
-        text="Test mock fact",
-        fact_type="observation" if proof_count is not None else "world",
-        document_id=str(uuid4()),
-        chunk_id=str(uuid4()),
-        proof_count=proof_count,
-        # Use None for neutral recency so only proof_count changes score
-        occurred_start=None,
-        occurred_end=None
-    )
-    candidate = MergedCandidate(
-        retrieval=retrieval,
-        rrf_score=0.1,
-    )
-    return ScoredResult(
-        candidate=candidate,
-        cross_encoder_score=ce_score,
-        cross_encoder_score_normalized=ce_score,
-        weight=ce_score,
-    )
-
-def test_proof_count_neutral_when_none():
-    """Test that when proof_count is None (e.g. non-observation), it gets neutral 0.5 norm."""
-    sr = create_mock_scored_result(proof_count=None, ce_score=0.8)
-    now = datetime.now(UTC)
-    
-    apply_combined_scoring([sr], now, proof_count_alpha=0.1)
-    
-    # Neutral multiplier means score shouldn't be boosted by proof_count
-    # Since recency is neutral (just created) and temporal is neutral, score should remain unchanged
-    assert sr.combined_score == pytest.approx(0.8, rel=1e-3)
-
-def test_proof_count_neutral_at_one():
-    """Test that proof_count=1 gives neutral multiplier."""
-    sr = create_mock_scored_result(proof_count=1, ce_score=0.8)
-    now = datetime.now(UTC)
-    
-    apply_combined_scoring([sr], now, proof_count_alpha=0.1)
-    
-    # proof_count=1 -> math.log(1) = 0 -> 0.5 + 0/10 = 0.5 (neutral) -> multiplier 1.0
-    assert sr.combined_score == pytest.approx(0.8, rel=1e-3)
-
-def test_proof_count_increases_with_higher_counts():
-    """Test that higher proof counts yield strictly higher scores."""
-    now = datetime.now(UTC)
-    
-    # Create results with increasing proof counts
-    sr_5 = create_mock_scored_result(proof_count=5, ce_score=0.8)
-    sr_50 = create_mock_scored_result(proof_count=50, ce_score=0.8)
-    sr_100 = create_mock_scored_result(proof_count=100, ce_score=0.8)
-    
-    # Process them
-    apply_combined_scoring([sr_5, sr_50, sr_100], now, proof_count_alpha=0.1)
-    
-    # Assure scores strictly increase
-    assert sr_5.combined_score > 0.8
-    assert sr_50.combined_score > sr_5.combined_score
-    assert sr_100.combined_score > sr_50.combined_score
-
-def test_proof_count_no_hardcoded_cap_at_100():
-    """Test that proof_count continues to scale within the clamped [0, 1] range."""
-    now = datetime.now(UTC)
-    
-    # Use values that stay below the clamp ceiling (proof_norm < 1.0)
-    # log(5)/10=0.16, log(20)/10=0.30, log(100)/10=0.46 → all below 0.5 headroom
-    sr_5 = create_mock_scored_result(proof_count=5, ce_score=0.8)
-    sr_20 = create_mock_scored_result(proof_count=20, ce_score=0.8)
-    sr_100 = create_mock_scored_result(proof_count=100, ce_score=0.8)
-    
-    apply_combined_scoring([sr_5, sr_20, sr_100], now, proof_count_alpha=0.1)
-    
-    # Must strictly increase within the valid range
-    assert sr_20.combined_score > sr_5.combined_score
-    assert sr_100.combined_score > sr_20.combined_score
-

hindsight-api-slim/tests/test_retain_append_mode.py

@@ -1,240 +0,0 @@
-"""
-Tests for retain update_mode='append' — appends new content to existing documents.
-"""
-
-import logging
-from datetime import datetime, timezone
-
-import pytest
-
-from hindsight_api.engine.memory_engine import Budget
-
-logger = logging.getLogger(__name__)
-
-
-def _ts():
-    return datetime.now(timezone.utc).timestamp()
-
-
-@pytest.mark.asyncio
-async def test_append_mode_concatenates_content(memory, request_context):
-    """
-    When update_mode='append', new content should be appended to the existing
-    document and the full document should be reprocessed. Facts from both
-    old and new content should be recallable.
-    """
-    bank_id = f"test_append_{_ts()}"
-    document_id = "conversation-append"
-
-    try:
-        # First retain — initial content
-        v1_units = await memory.retain_async(
-            bank_id=bank_id,
-            content="Alice works at Google as a software engineer.",
-            context="team info",
-            document_id=document_id,
-            request_context=request_context,
-        )
-        assert len(v1_units) > 0, "v1 should create facts"
-
-        doc_v1 = await memory.get_document(document_id, bank_id, request_context=request_context)
-        v1_text = doc_v1["original_text"]
-        assert "Alice works at Google" in v1_text
-
-        # Second retain with append — add new content
-        v2_units = await memory.retain_batch_async(
-            bank_id=bank_id,
-            contents=[
-                {
-                    "content": "Bob works at Microsoft as a data scientist.",
-                    "context": "team info",
-                    "document_id": document_id,
-                    "update_mode": "append",
-                }
-            ],
-            request_context=request_context,
-        )
-
-        # Verify document now contains both old and new content
-        doc_v2 = await memory.get_document(document_id, bank_id, request_context=request_context)
-        v2_text = doc_v2["original_text"]
-        assert "Alice works at Google" in v2_text, "Original content should be preserved"
-        assert "Bob works at Microsoft" in v2_text, "New content should be appended"
-
-        # Verify facts from both old and new content are recallable
-        result_alice = await memory.recall_async(
-            bank_id=bank_id,
-            query="Where does Alice work?",
-            budget=Budget.MID,
-            max_tokens=1000,
-            request_context=request_context,
-        )
-        assert len(result_alice.results) > 0, "Should recall facts about Alice"
-
-        result_bob = await memory.recall_async(
-            bank_id=bank_id,
-            query="Where does Bob work?",
-            budget=Budget.MID,
-            max_tokens=1000,
-            request_context=request_context,
-        )
-        assert len(result_bob.results) > 0, "Should recall facts about Bob"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_append_mode_no_existing_document(memory, request_context):
-    """
-    When update_mode='append' but no existing document exists,
-    it should behave like a normal retain (no content to prepend).
-    """
-    bank_id = f"test_append_new_{_ts()}"
-    document_id = "new-doc-append"
-
-    try:
-        units = await memory.retain_batch_async(
-            bank_id=bank_id,
-            contents=[
-                {
-                    "content": "Charlie is a product manager at Stripe.",
-                    "context": "team info",
-                    "document_id": document_id,
-                    "update_mode": "append",
-                }
-            ],
-            request_context=request_context,
-        )
-
-        assert len(units) > 0, "Should create facts even with no existing document"
-        # Flatten if nested
-        flat_units = units[0] if units and isinstance(units[0], list) else units
-        assert len(flat_units) > 0
-
-        doc = await memory.get_document(document_id, bank_id, request_context=request_context)
-        assert "Charlie is a product manager" in doc["original_text"]
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_append_mode_requires_document_id(memory, request_context):
-    """update_mode='append' without document_id should raise ValueError."""
-    bank_id = f"test_append_no_docid_{_ts()}"
-
-    with pytest.raises(ValueError, match="update_mode='append' requires a document_id"):
-        await memory.retain_batch_async(
-            bank_id=bank_id,
-            contents=[
-                {
-                    "content": "Some content",
-                    "update_mode": "append",
-                }
-            ],
-            request_context=request_context,
-        )
-
-
-@pytest.mark.asyncio
-async def test_append_mode_multiple_appends(memory, request_context):
-    """Multiple appends should accumulate content over successive retains."""
-    bank_id = f"test_multi_append_{_ts()}"
-    document_id = "multi-append-doc"
-
-    try:
-        # Initial retain
-        await memory.retain_async(
-            bank_id=bank_id,
-            content="Day 1: Alice joined the team.",
-            context="journal",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        # First append
-        await memory.retain_batch_async(
-            bank_id=bank_id,
-            contents=[
-                {
-                    "content": "Day 2: Alice completed her onboarding.",
-                    "context": "journal",
-                    "document_id": document_id,
-                    "update_mode": "append",
-                }
-            ],
-            request_context=request_context,
-        )
-
-        # Second append
-        await memory.retain_batch_async(
-            bank_id=bank_id,
-            contents=[
-                {
-                    "content": "Day 3: Alice shipped her first feature.",
-                    "context": "journal",
-                    "document_id": document_id,
-                    "update_mode": "append",
-                }
-            ],
-            request_context=request_context,
-        )
-
-        # Verify all content is present
-        doc = await memory.get_document(document_id, bank_id, request_context=request_context)
-        text = doc["original_text"]
-        assert "Day 1" in text, "Original content should be present"
-        assert "Day 2" in text, "First append should be present"
-        assert "Day 3" in text, "Second append should be present"
-
-        # All days should be recallable
-        result = await memory.recall_async(
-            bank_id=bank_id,
-            query="What happened on Alice's first days?",
-            budget=Budget.MID,
-            max_tokens=1000,
-            request_context=request_context,
-        )
-        assert len(result.results) > 0, "Should recall facts from all appends"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)
-
-
-@pytest.mark.asyncio
-async def test_replace_mode_is_default(memory, request_context):
-    """Without update_mode (or update_mode='replace'), retain should replace content."""
-    bank_id = f"test_replace_default_{_ts()}"
-    document_id = "replace-doc"
-
-    try:
-        await memory.retain_async(
-            bank_id=bank_id,
-            content="Alice works at Google.",
-            context="team info",
-            document_id=document_id,
-            request_context=request_context,
-        )
-
-        # Retain again without update_mode — should replace
-        await memory.retain_batch_async(
-            bank_id=bank_id,
-            contents=[
-                {
-                    "content": "Bob works at Microsoft.",
-                    "context": "team info",
-                    "document_id": document_id,
-                }
-            ],
-            request_context=request_context,
-        )
-
-        doc = await memory.get_document(document_id, bank_id, request_context=request_context)
-        text = doc["original_text"]
-        # With replace, only new content should remain
-        assert "Bob works at Microsoft" in text, "New content should be present"
-        assert "Alice works at Google" not in text, "Old content should be replaced"
-
-    finally:
-        await memory.delete_bank(bank_id, request_context=request_context)

hindsight-api-slim/tests/test_stage.py

@@ -1,71 +0,0 @@
-"""Unit tests for the worker stage breadcrumb module."""
-
-import asyncio
-
-import pytest
-
-from hindsight_api.worker.stage import StageHolder, bind_holder, get_stage, set_stage
-
-
-def test_set_stage_is_noop_without_holder():
-    # No holder bound in this context: must not raise, and get_stage returns None.
-    set_stage("anything")
-    assert get_stage() is None
-
-
-@pytest.mark.asyncio
-async def test_holder_bound_inside_task_is_visible_to_called_code():
-    holder = StageHolder()
-
-    async def inner():
-        # The poller binds the holder from inside the task coroutine so it
-        # lives in that task's contextvar scope; mirror that here.
-        bind_holder(holder)
-        set_stage("phase1")
-        # Engine code further down the call stack reads via set_stage.
-        set_stage("phase2")
-        assert get_stage() == "phase2"
-
-    await asyncio.create_task(inner())
-
-    # Holder is mutable: the spawning context sees the latest stage written
-    # by the child task without needing access to the contextvar.
-    assert holder.stage == "phase2"
-
-
-@pytest.mark.asyncio
-async def test_holder_does_not_leak_across_tasks():
-    # Each asyncio.create_task copies the parent's context. Binding inside
-    # one task must not affect a sibling task's view.
-    holder_a = StageHolder()
-    holder_b = StageHolder()
-
-    async def task_a():
-        bind_holder(holder_a)
-        set_stage("a")
-
-    async def task_b():
-        bind_holder(holder_b)
-        set_stage("b")
-
-    await asyncio.gather(asyncio.create_task(task_a()), asyncio.create_task(task_b()))
-
-    assert holder_a.stage == "a"
-    assert holder_b.stage == "b"
-    # Outside both tasks, no holder is bound.
-    assert get_stage() is None
-
-
-@pytest.mark.asyncio
-async def test_set_stage_updates_timestamp():
-    holder = StageHolder()
-
-    async def inner():
-        bind_holder(holder)
-        first = holder.updated_at
-        # asyncio.sleep guarantees monotonic clock advances on next set.
-        await asyncio.sleep(0.01)
-        set_stage("next")
-        assert holder.updated_at > first
-
-    await asyncio.create_task(inner())

hindsight-api-slim/tests/test_strip_code_fences.py

@@ -1,92 +0,0 @@
-"""Tests for _strip_code_fences helper in OpenAI-compatible LLM provider."""
-
-import pytest
-
-from hindsight_api.engine.providers.openai_compatible_llm import _strip_code_fences
-
-
-class TestStripCodeFences:
-    """Test markdown code fence stripping from LLM responses."""
-
-    def test_bare_json_unchanged(self):
-        """Bare JSON passes through unchanged."""
-        content = '{"facts": [{"what": "test"}]}'
-        assert _strip_code_fences(content) == content
-
-    def test_json_fence_stripped(self):
-        """```json ... ``` fences are stripped."""
-        content = '```json\n{"facts": [{"what": "test"}]}\n```'
-        assert _strip_code_fences(content) == '{"facts": [{"what": "test"}]}'
-
-    def test_plain_fence_stripped(self):
-        """``` ... ``` fences without language tag are stripped."""
-        content = '```\n{"facts": [{"what": "test"}]}\n```'
-        assert _strip_code_fences(content) == '{"facts": [{"what": "test"}]}'
-
-    def test_fence_with_trailing_whitespace(self):
-        """Fences with extra whitespace are handled."""
-        content = '```json\n{"facts": []}\n```\n'
-        result = _strip_code_fences(content)
-        assert result == '{"facts": []}'
-
-    def test_fence_with_leading_whitespace(self):
-        """Content with leading whitespace before fence."""
-        content = '  ```json\n{"facts": []}\n```'
-        # The function checks for ``` in content, not startswith
-        result = _strip_code_fences(content)
-        assert '{"facts": []}' in result
-
-    def test_no_fences_no_change(self):
-        """Content without any backticks passes through."""
-        content = "Just some text without fences"
-        assert _strip_code_fences(content) == content
-
-    def test_empty_string(self):
-        """Empty string passes through."""
-        assert _strip_code_fences("") == ""
-
-    def test_multiline_json(self):
-        """Multi-line JSON inside fences is preserved."""
-        content = '```json\n{\n  "facts": [\n    {"what": "line1"},\n    {"what": "line2"}\n  ]\n}\n```'
-        result = _strip_code_fences(content)
-        assert '"line1"' in result
-        assert '"line2"' in result
-        assert "```" not in result
-
-    def test_malformed_fence_returns_original(self):
-        """Malformed fences (missing closing) return something parseable."""
-        content = '```json\n{"facts": []}'
-        result = _strip_code_fences(content)
-        # Should attempt to strip and return best effort
-        assert isinstance(result, str)
-
-    def test_minimax_style_response(self):
-        """Real-world MiniMax response format."""
-        content = (
-            "```json\n"
-            "{\n"
-            '  "facts": [\n'
-            "    {\n"
-            '      "what": "Sebastian switched the Hindsight extraction LLM",\n'
-            '      "when": "2026-03-21",\n'
-            '      "where": "N/A",\n'
-            '      "who": "Sebastian",\n'
-            '      "why": "MiniMax wraps JSON in code fences",\n'
-            '      "fact_kind": "event",\n'
-            '      "fact_type": "world",\n'
-            '      "entities": [{"text": "Sebastian"}, {"text": "Hindsight"}],\n'
-            '      "labels": {"source_type": "stated", "domain": ["infrastructure"]}\n'
-            "    }\n"
-            "  ]\n"
-            "}\n"
-            "```"
-        )
-        result = _strip_code_fences(content)
-        assert not result.startswith("```")
-        assert not result.endswith("```")
-        # Should be valid JSON
-        import json
-
-        parsed = json.loads(result)
-        assert len(parsed["facts"]) == 1
-        assert parsed["facts"][0]["who"] == "Sebastian"