fix

feat: add metrics for llm call latency
2026-01-08 11:22:27 +01:00 · 2026-01-08 11:22:27 +01:00 · 2026-01-08 11:22:27 +01:00 · 2026-01-08 11:22:10 +01:00 · 2026-01-08 11:17:02 +01:00 · 2026-01-08 10:36:58 +01:00
288 changed files with 28126 additions and 22458 deletions
@@ -2,11 +2,23 @@
 # Copy this file to .env and fill in your values

 # LLM Configuration (Required)
+# Supported providers: openai, groq, ollama, gemini, anthropic, lmstudio
 HINDSIGHT_API_LLM_PROVIDER=openai
 HINDSIGHT_API_LLM_API_KEY=your-api-key-here
 HINDSIGHT_API_LLM_MODEL=o3-mini
 HINDSIGHT_API_LLM_BASE_URL=https://api.openai.com/v1

+# Example: Anthropic Claude configuration
+# HINDSIGHT_API_LLM_PROVIDER=anthropic
+# HINDSIGHT_API_LLM_API_KEY=your-anthropic-api-key
+# HINDSIGHT_API_LLM_MODEL=claude-sonnet-4-20250514
+
+# Example: LM Studio local configuration (Qwen 2.5 32B recommended)
+# HINDSIGHT_API_LLM_PROVIDER=lmstudio
+# HINDSIGHT_API_LLM_API_KEY=lmstudio
+# HINDSIGHT_API_LLM_BASE_URL=http://localhost:1234/v1
+# HINDSIGHT_API_LLM_MODEL=qwen2.5-32b-instruct
+
 # API Configuration (Optional)
 HINDSIGHT_API_HOST=0.0.0.0
 HINDSIGHT_API_PORT=8888
@@ -0,0 +1,71 @@
+name: Bug Report
+description: Report a bug or unexpected behavior
+labels: ["bug", "triage"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for taking the time to report a bug! Please fill out the sections below.
+
+  - type: textarea
+    id: description
+    attributes:
+      label: Bug Description
+      description: A clear and concise description of the bug
+      placeholder: What happened?
+    validations:
+      required: true
+
+  - type: textarea
+    id: reproduction
+    attributes:
+      label: Steps to Reproduce
+      description: Steps to reproduce the behavior
+      placeholder: |
+        1. Configure '...'
+        2. Call '...'
+        3. See error
+    validations:
+      required: true
+
+  - type: textarea
+    id: expected
+    attributes:
+      label: Expected Behavior
+      description: What did you expect to happen?
+    validations:
+      required: true
+
+  - type: textarea
+    id: actual
+    attributes:
+      label: Actual Behavior
+      description: What actually happened?
+    validations:
+      required: true
+
+  - type: input
+    id: version
+    attributes:
+      label: Version
+      description: What version are you using?
+      placeholder: e.g., 0.1.0 or commit hash
+    validations:
+      required: false
+
+  - type: dropdown
+    id: llm-provider
+    attributes:
+      label: LLM Provider
+      description: Which LLM provider are you using?
+      options:
+        - OpenAI
+        - Anthropic
+        - Gemini
+        - Groq
+        - Ollama
+        - LM Studio
+        - Other
+    validations:
+      required: false
+
@@ -0,0 +1,8 @@
+blank_issues_enabled: false
+contact_links:
+  - name: Questions & Help
+    url: https://github.com/vectorize-io/hindsight/discussions/categories/q-a
+    about: Please ask questions and get help in Discussions instead of opening an issue.
+  - name: Ideas & Feedback
+    url: https://github.com/vectorize-io/hindsight/discussions/categories/ideas
+    about: Share ideas or give feedback in Discussions.
@@ -0,0 +1,82 @@
+name: Feature Request
+description: Suggest a new feature or enhancement
+labels: ["enhancement", "triage"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for suggesting a feature! Please describe what you'd like to see added.
+
+  - type: textarea
+    id: use-case
+    attributes:
+      label: Use Case
+      description: Describe your specific use case. What are you building? What's your goal?
+      placeholder: |
+        I'm building an AI agent that needs to...
+        My application handles...
+    validations:
+      required: true
+
+  - type: textarea
+    id: problem
+    attributes:
+      label: Problem Statement
+      description: What problem are you facing? What's missing or difficult today?
+      placeholder: Currently I have to... which causes...
+    validations:
+      required: true
+
+  - type: textarea
+    id: benefit
+    attributes:
+      label: How This Feature Would Help
+      description: Explain how this feature would improve your workflow or solve your problem
+      placeholder: With this feature, I would be able to...
+    validations:
+      required: true
+
+  - type: textarea
+    id: solution
+    attributes:
+      label: Proposed Solution
+      description: Describe your ideal solution (optional - we may have ideas too!)
+      placeholder: It would be great if Hindsight could...
+    validations:
+      required: false
+
+  - type: textarea
+    id: alternatives
+    attributes:
+      label: Alternatives Considered
+      description: Have you considered any alternative solutions or workarounds?
+    validations:
+      required: false
+
+  - type: dropdown
+    id: priority
+    attributes:
+      label: Priority
+      description: How important is this feature to you?
+      options:
+        - Nice to have
+        - Important - affects my workflow
+        - Critical - blocking my use case
+    validations:
+      required: true
+
+  - type: textarea
+    id: additional
+    attributes:
+      label: Additional Context
+      description: Any other context, mockups, or examples?
+    validations:
+      required: false
+
+  - type: checkboxes
+    id: checklist
+    attributes:
+      label: Checklist
+      options:
+        - label: I would be willing to contribute this feature
+          required: false
@@ -182,6 +182,83 @@ jobs:
        path: hindsight-cli/target/release/hindsight
        retention-days: 1

+  test-rust-cli:
+    runs-on: ubuntu-latest
+    needs: build-rust-cli
+    env:
+      HINDSIGHT_API_LLM_PROVIDER: groq
+      HINDSIGHT_API_LLM_API_KEY: ${{ secrets.GROQ_API_KEY }}
+      HINDSIGHT_API_LLM_MODEL: openai/gpt-oss-20b
+      HINDSIGHT_API_URL: http://localhost:8888
+      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      UV_INDEX: pytorch=https://download.pytorch.org/whl/cpu
+
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: Download CLI artifact
+      uses: actions/download-artifact@v4
+      with:
+        name: hindsight-cli
+        path: /tmp/cli
+
+    - name: Make CLI executable
+      run: chmod +x /tmp/cli/hindsight
+
+    - name: Install uv
+      uses: astral-sh/setup-uv@v5
+      with:
+        enable-cache: true
+        prune-cache: false
+
+    - name: Set up Python
+      uses: actions/setup-python@v5
+      with:
+        python-version-file: ".python-version"
+
+    - name: Build API
+      working-directory: ./hindsight-api
+      run: uv build
+
+    - name: Install API dependencies
+      working-directory: ./hindsight-api
+      run: uv sync --no-install-project --index-strategy unsafe-best-match
+
+    - name: Create .env file
+      run: |
+        cat > .env << EOF
+        HINDSIGHT_API_LLM_PROVIDER=${{ env.HINDSIGHT_API_LLM_PROVIDER }}
+        HINDSIGHT_API_LLM_API_KEY=${{ env.HINDSIGHT_API_LLM_API_KEY }}
+        HINDSIGHT_API_LLM_MODEL=${{ env.HINDSIGHT_API_LLM_MODEL }}
+        EOF
+
+    - name: Start API server
+      run: |
+        ./scripts/dev/start-api.sh > /tmp/api-server.log 2>&1 &
+        echo "Waiting for API server to be ready..."
+        for i in {1..60}; do
+          if curl -sf http://localhost:8888/health > /dev/null 2>&1; then
+            echo "API server is ready after ${i}s"
+            break
+          fi
+          if [ $i -eq 60 ]; then
+            echo "API server failed to start after 60s"
+            cat /tmp/api-server.log
+            exit 1
+          fi
+          sleep 1
+        done
+
+    - name: Run CLI smoke test
+      run: |
+        HINDSIGHT_CLI=/tmp/cli/hindsight ./hindsight-cli/smoke-test.sh
+
+    - name: Show API server logs
+      if: always()
+      run: |
+        echo "=== API Server Logs ==="
+        cat /tmp/api-server.log || echo "No API server log found"
+
  lint-helm-chart:
    runs-on: ubuntu-latest

@@ -248,6 +325,7 @@ jobs:
      GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
      GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
      OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
+      HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
      HINDSIGHT_API_LLM_MODEL: openai/gpt-oss-20b
      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      # Prefer CPU-only PyTorch in CI (but keep PyPI for everything else)
@@ -536,6 +614,97 @@ jobs:
        echo "=== API Server Logs ==="
        cat /tmp/api-server.log || echo "No API server log found"

+  test-integration:
+    runs-on: ubuntu-latest
+    env:
+      HINDSIGHT_API_LLM_PROVIDER: groq
+      HINDSIGHT_API_LLM_API_KEY: ${{ secrets.GROQ_API_KEY }}
+      HINDSIGHT_API_LLM_MODEL: openai/gpt-oss-20b
+      HINDSIGHT_API_URL: http://localhost:8888
+      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      UV_INDEX: pytorch=https://download.pytorch.org/whl/cpu
+
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: Install uv
+      uses: astral-sh/setup-uv@v5
+      with:
+        enable-cache: true
+        prune-cache: false
+
+    - name: Set up Python
+      uses: actions/setup-python@v5
+      with:
+        python-version-file: ".python-version"
+
+    - name: Build API
+      working-directory: ./hindsight-api
+      run: uv build
+
+    - name: Install API dependencies
+      working-directory: ./hindsight-api
+      run: uv sync --no-install-project --index-strategy unsafe-best-match
+
+    - name: Install integration test dependencies
+      working-directory: ./hindsight-integration-tests
+      run: uv sync
+
+    - name: Cache HuggingFace models
+      uses: actions/cache@v4
+      with:
+        path: ~/.cache/huggingface
+        key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api/pyproject.toml') }}
+        restore-keys: |
+          ${{ runner.os }}-huggingface-
+
+    - name: Pre-download models
+      working-directory: ./hindsight-api
+      run: |
+        uv run python -c "
+        from sentence_transformers import SentenceTransformer, CrossEncoder
+        print('Downloading embedding model...')
+        SentenceTransformer('BAAI/bge-small-en-v1.5')
+        print('Downloading cross-encoder model...')
+        CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
+        print('Models downloaded successfully')
+        "
+
+    - name: Create .env file
+      run: |
+        cat > .env << EOF
+        HINDSIGHT_API_LLM_PROVIDER=${{ env.HINDSIGHT_API_LLM_PROVIDER }}
+        HINDSIGHT_API_LLM_API_KEY=${{ env.HINDSIGHT_API_LLM_API_KEY }}
+        HINDSIGHT_API_LLM_MODEL=${{ env.HINDSIGHT_API_LLM_MODEL }}
+        EOF
+
+    - name: Start API server
+      run: |
+        ./scripts/dev/start-api.sh > /tmp/api-server.log 2>&1 &
+        echo "Waiting for API server to be ready..."
+        for i in {1..60}; do
+          if curl -sf http://localhost:8888/health > /dev/null 2>&1; then
+            echo "API server is ready after ${i}s"
+            break
+          fi
+          if [ $i -eq 60 ]; then
+            echo "API server failed to start after 60s"
+            cat /tmp/api-server.log
+            exit 1
+          fi
+          sleep 1
+        done
+
+    - name: Run integration tests
+      working-directory: ./hindsight-integration-tests
+      run: uv run pytest tests/ -v
+
+    - name: Show API server logs
+      if: always()
+      run: |
+        echo "=== API Server Logs ==="
+        cat /tmp/api-server.log || echo "No API server log found"
+
  test-litellm-integration:
    runs-on: ubuntu-latest

@@ -715,4 +884,76 @@ jobs:
      if: always()
      run: |
        echo "=== API Server Logs ==="
-        cat /tmp/api-server.log || echo "No API server log found"
+        cat /tmp/api-server.log || echo "No API server log found"
+
+  verify-generated-files:
+    runs-on: ubuntu-latest
+    env:
+      UV_INDEX: pytorch=https://download.pytorch.org/whl/cpu
+
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: Install uv
+      uses: astral-sh/setup-uv@v5
+      with:
+        enable-cache: true
+
+    - name: Set up Python
+      uses: actions/setup-python@v5
+      with:
+        python-version-file: ".python-version"
+
+    - name: Set up Node.js
+      uses: actions/setup-node@v4
+      with:
+        node-version: '20'
+        cache: 'npm'
+        cache-dependency-path: package-lock.json
+
+    - name: Install Rust
+      uses: dtolnay/rust-toolchain@stable
+
+    - name: Cache cargo
+      uses: actions/cache@v4
+      with:
+        path: |
+          ~/.cargo/registry
+          ~/.cargo/git
+        key: ${{ runner.os }}-cargo-gen-${{ hashFiles('**/Cargo.lock') }}
+
+    - name: Install Node dependencies
+      run: npm ci
+
+    - name: Install Python dependencies
+      run: |
+        cd hindsight-dev && uv sync --index-strategy unsafe-best-match
+        cd ../hindsight-api && uv sync --index-strategy unsafe-best-match
+        cd ../hindsight-embed && uv sync --index-strategy unsafe-best-match
+
+    - name: Run generate-openapi
+      run: ./scripts/generate-openapi.sh
+
+    - name: Run generate-clients
+      run: ./scripts/generate-clients.sh
+
+    - name: Run lint
+      run: ./scripts/hooks/lint.sh
+
+    - name: Verify no uncommitted changes
+      run: |
+        if [ -n "$(git status --porcelain)" ]; then
+          echo "❌ Error: Generated files are out of sync with committed files."
+          echo ""
+          echo "The following files have changed after running generation scripts:"
+          git status --porcelain
+          echo ""
+          echo "Please run the following commands locally and commit the changes:"
+          echo "  ./scripts/generate-openapi.sh"
+          echo "  ./scripts/generate-clients.sh"
+          echo "  ./scripts/hooks/lint.sh"
+          echo ""
+          git diff --stat
+          exit 1
+        fi
+        echo "✓ All generated files are up to date"
@@ -5,15 +5,18 @@ build/
 dist/
 wheels/
 *.egg-info
-
+.mcp.json
+.osgrep
 # Virtual environments
 .venv

 # Node
 node_modules/

-# Environment variables
+# Environment variables and local config
 .env
+docker-compose.yml
+docker-compose.override.yml

 # IDE
 .idea/
@@ -39,4 +42,8 @@ hindsight-docs/static/llms-full.txt
 hindsight-dev/benchmarks/locomo/results/
 hindsight-dev/benchmarks/longmemeval/results/
 hindsight-cli/target
-hindsight-clients/rust/target
+hindsight-clients/rust/target
+.claude
+whats-next.md
+TASK.md
+CHANGELOG.md
@@ -0,0 +1,186 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+Hindsight is an agent memory system that provides long-term memory for AI agents using biomimetic data structures. Memories are organized as:
+- **World facts**: General knowledge ("The sky is blue")
+- **Experience facts**: Personal experiences ("I visited Paris in 2023")
+- **Opinion facts**: Beliefs with confidence scores ("Paris is beautiful" - 0.9 confidence)
+- **Observations**: Complex mental models derived from reflection
+
+## Development Commands
+
+### API Server (Python/FastAPI)
+```bash
+# Start API server (loads .env automatically)
+./scripts/dev/start-api.sh
+
+# Run all tests (parallelized with pytest-xdist)
+cd hindsight-api && uv run pytest tests/
+
+# Run specific test file
+cd hindsight-api && uv run pytest tests/test_http_api_integration.py -v
+
+# Run single test function
+cd hindsight-api && uv run pytest tests/test_retain.py::test_retain_simple -v
+
+# Lint and format
+cd hindsight-api && uv run ruff check .
+cd hindsight-api && uv run ruff format .
+
+# Type checking (uses ty - extremely fast type checker from Astral)
+cd hindsight-api && uv run ty check hindsight_api/
+```
+
+### Control Plane (Next.js)
+```bash
+./scripts/dev/start-control-plane.sh
+# Or manually:
+cd hindsight-control-plane && npm run dev
+```
+
+### Documentation Site (Docusaurus)
+```bash
+./scripts/dev/start-docs.sh
+```
+
+### Generating Clients/OpenAPI
+```bash
+# Regenerate OpenAPI spec after API changes (REQUIRED after changing endpoints)
+./scripts/generate-openapi.sh
+
+# Regenerate all client SDKs (Python, TypeScript, Rust)
+./scripts/generate-clients.sh
+```
+
+### Benchmarks
+```bash
+./scripts/benchmarks/run-longmemeval.sh
+./scripts/benchmarks/run-locomo.sh
+./scripts/benchmarks/start-visualizer.sh  # View results at localhost:8001
+```
+
+## Architecture
+
+### Monorepo Structure
+- **hindsight-api/**: Core FastAPI server with memory engine (Python, uv)
+- **hindsight/**: Embedded Python bundle (hindsight-all package)
+- **hindsight-control-plane/**: Admin UI (Next.js, npm)
+- **hindsight-cli/**: CLI tool (Rust, cargo, uses progenitor for API client)
+- **hindsight-clients/**: Generated SDK clients (Python, TypeScript, Rust)
+- **hindsight-docs/**: Docusaurus documentation site
+- **hindsight-integrations/**: Framework integrations (LiteLLM, OpenAI)
+- **hindsight-dev/**: Development tools and benchmarks
+
+### Core Engine (hindsight-api/hindsight_api/engine/)
+- `memory_engine.py`: Main orchestrator (~170KB) for retain/recall/reflect operations
+- `llm_wrapper.py`: LLM abstraction supporting OpenAI, Anthropic, Gemini, Groq, Ollama, LM Studio
+- `embeddings.py`: Embedding generation (local sentence-transformers or TEI)
+- `cross_encoder.py`: Reranking (local or TEI)
+- `entity_resolver.py`: Entity extraction and normalization
+- `query_analyzer.py`: Query intent analysis
+
+**retain/**: Memory ingestion pipeline
+- `orchestrator.py`: Coordinates the retain flow
+- `fact_extraction.py`: LLM-based fact extraction from content
+- `link_utils.py`: Entity link creation and management
+
+**search/**: Multi-strategy retrieval
+- `retrieval.py`: Main retrieval orchestrator
+- `graph_retrieval.py`: Entity/relationship graph traversal
+- `mpfp_retrieval.py`: Multi-Path Fact Propagation retrieval
+- `fusion.py`: Reciprocal rank fusion for combining results
+- `reranking.py`: Cross-encoder reranking
+
+### API Layer (hindsight-api/hindsight_api/api/)
+- `http.py`: FastAPI HTTP routers (~80KB) for all REST endpoints
+- `mcp.py`: Model Context Protocol server implementation
+
+Main operations:
+- **Retain**: Store memories, extracts facts/entities/relationships
+- **Recall**: Retrieve memories via 4 parallel strategies (semantic, BM25, graph, temporal) + reranking
+- **Reflect**: Deep analysis forming new opinions/observations (disposition-aware)
+
+### Database
+PostgreSQL with pgvector. Schema managed via Alembic migrations in `hindsight-api/hindsight_api/alembic/`. Migrations run automatically on API startup.
+
+Key tables: `banks`, `memory_units`, `documents`, `entities`, `entity_links`
+
+## Key Conventions
+
+### Code Quality
+**Always run the lint script after making Python or TypeScript/Node changes:**
+```bash
+./scripts/hooks/lint.sh
+```
+This runs the same checks as the pre-commit hook (Ruff for Python, ESLint/Prettier for TypeScript).
+
+### Memory Banks
+- Each bank is an isolated memory store (like a "brain" for one user/agent)
+- Banks have dispositions (skepticism, literalism, empathy traits 1-5) affecting reflect
+- Banks can have background context
+- Bank isolation is strict - no cross-bank data leakage
+
+### API Design
+- All endpoints operate on a single bank per request
+- Multi-bank queries are client responsibility to orchestrate
+- Disposition traits only affect reflect, not recall
+
+### Python Style
+- Python 3.11+, type hints required
+- Async throughout (asyncpg, async FastAPI)
+- Pydantic models for request/response
+- Ruff for linting (line-length 120)
+- No Python files at project root - maintain clean directory structure
+
+### TypeScript Style
+- Next.js App Router for control plane
+- Tailwind CSS with shadcn/ui components
+
+### Adding New API Configuration Flags
+
+When adding a new environment variable configuration:
+
+1. **config.py** (`hindsight-api/hindsight_api/config.py`):
+   - Add `ENV_*` constant for the environment variable name
+   - Add `DEFAULT_*` constant for the default value
+   - Add field to `HindsightConfig` dataclass
+   - Add initialization in `from_env()` method
+
+2. **main.py** (`hindsight-api/hindsight_api/main.py`):
+   - Add field to the manual `HindsightConfig()` constructor call (search for "CLI override")
+
+3. **Use the config** in code:
+   ```python
+   from ...config import get_config
+   config = get_config()
+   value = config.your_new_field
+   ```
+
+4. **Documentation** (`hindsight-docs/docs/developer/configuration.md`):
+   - Add to appropriate section table with Variable, Description, Default
+
+## Environment Setup
+
+```bash
+cp .env.example .env
+# Edit .env with LLM API key
+
+# Python deps
+uv sync --directory hindsight-api/
+
+# Node deps (uses npm workspaces)
+npm install
+```
+
+Required env vars:
+- `HINDSIGHT_API_LLM_PROVIDER`: openai, anthropic, gemini, groq, ollama, lmstudio
+- `HINDSIGHT_API_LLM_API_KEY`: Your API key
+- `HINDSIGHT_API_LLM_MODEL`: Model name (e.g., o3-mini, claude-sonnet-4-20250514)
+
+Optional (uses local models by default):
+- `HINDSIGHT_API_EMBEDDINGS_PROVIDER`: local (default) or tei
+- `HINDSIGHT_API_RERANKER_PROVIDER`: local (default) or tei
+- `HINDSIGHT_API_DATABASE_URL`: External PostgreSQL (uses embedded pg0 by default)
@@ -51,7 +51,36 @@ cd hindsight-api
 uv run pytest tests/
 ```

-### Code style
+### Code Style
+
+We use [Ruff](https://docs.astral.sh/ruff/) for Python linting and formatting, and ESLint/Prettier for TypeScript.
+
+#### Setting up git hooks (recommended)
+
+Set up git hooks to automatically lint and format code before each commit:
+
+```bash
+./scripts/setup-hooks.sh
+```
+
+This configures git to use the hooks in `.githooks/`, which run all scripts in `scripts/hooks/` on commit. The lint hook runs in parallel:
+- **Python**: `ruff check --fix`, `ruff format`, `ty check`
+- **TypeScript**: `eslint --fix`, `prettier`
+
+#### Manual linting and formatting
+
+```bash
+# Run all lints (same as pre-commit)
+./scripts/hooks/lint.sh
+
+# Or run individually for Python:
+cd hindsight-api
+uv run ruff check --fix .   # Lint and auto-fix
+uv run ruff format .        # Format code
+uv run ty check hindsight_api  # Type check
+```
+
+#### Style guidelines

 - Use Python type hints
 - Follow existing code patterns
@@ -81,6 +81,8 @@ docker run --rm -it --pull always -p 8888:8888 -p 9999:9999 \
  ghcr.io/vectorize-io/hindsight:latest
 ```

+You can modify the LLM provider by setting `HINDSIGHT_API_LLM_PROVIDER`. Valid options are `openai`, `anthropic`, `gemini`, `groq`, `ollama`, and `lmstudio`. The documentation provides more details on [supported models](https://hindsight.vectorize.io/developer/models).
+
 API: http://localhost:8888  
 UI: http://localhost:9999

@@ -125,7 +125,6 @@ FROM python:3.11-slim AS api-only

 WORKDIR /app

-# Install pg0 dependencies (procps provides 'kill' command needed by pg0)
 # Note: libicu version varies by Debian version - try common versions in order
 RUN apt-get update && apt-get install -y \
    curl \
@@ -138,7 +137,6 @@ RUN apt-get update && apt-get install -y \
    && rm -rf /var/lib/apt/lists/* \
    && pip install --no-cache-dir uv

-# Create non-root user (PostgreSQL cannot run as root)
 RUN useradd -m -s /bin/bash hindsight

 # Copy API with virtual environment from builder
@@ -148,20 +146,12 @@ COPY --from=api-builder /app/api /app/api
 COPY docker/standalone/start-all.sh /app/start-all.sh
 RUN chmod +x /app/start-all.sh

-# Create data directory for pg0 and set ownership
-RUN mkdir -p /app/data && chown -R hindsight:hindsight /app
+RUN chown -R hindsight:hindsight /app

-# Switch to non-root user
 USER hindsight

-# Set PATH for hindsight user
 ENV PATH="/app/api/.venv/bin:${PATH}"

-# Pre-cache PostgreSQL binaries by starting/stopping pg0-embedded
-ENV PG0_HOME=/home/hindsight/.pg0-cache
-
-ENV PG0_HOME=/home/hindsight/.pg0
-
 # Pre-download ML models to avoid runtime download (conditional)
 ARG PRELOAD_ML_MODELS
 RUN if [ "$PRELOAD_ML_MODELS" = "true" ]; then \
@@ -226,7 +216,7 @@ FROM python:3.11-slim AS standalone

 WORKDIR /app

-# Install Node.js, curl, uv, and pg0 dependencies (procps provides 'kill' command needed by pg0)
+# Install Node.js, curl, uv, and system dependencies
 # Note: libicu version varies by Debian version - try common versions in order
 RUN apt-get update && apt-get install -y \
    curl \
@@ -241,7 +231,6 @@ RUN apt-get update && apt-get install -y \
    && rm -rf /var/lib/apt/lists/* \
    && pip install --no-cache-dir uv

-# Create non-root user (PostgreSQL cannot run as root)
 RUN useradd -m -s /bin/bash hindsight

 # Copy API with virtual environment from builder
@@ -262,27 +251,12 @@ WORKDIR /app
 COPY docker/standalone/start-all.sh /app/start-all.sh
 RUN chmod +x /app/start-all.sh

-# Create data directory for pg0 and set ownership
-RUN mkdir -p /app/data && chown -R hindsight:hindsight /app
+RUN chown -R hindsight:hindsight /app

-# Switch to non-root user
 USER hindsight

-# Set PATH for hindsight user
 ENV PATH="/app/api/.venv/bin:${PATH}"

-# Pre-cache PostgreSQL binaries by starting/stopping pg0-embedded
-ENV PG0_HOME=/home/hindsight/.pg0-cache
-RUN /app/api/.venv/bin/python -c "\
-from pg0 import Pg0; \
-print('Pre-caching PostgreSQL binaries...'); \
-pg = Pg0(name='hindsight', port=5555, username='hindsight', password='hindsight', database='hindsight'); \
-pg.start(); \
-pg.stop(); \
-print('PostgreSQL pre-cached to PG0_HOME')" || echo "Pre-download skipped"
-
-ENV PG0_HOME=/home/hindsight/.pg0
-
 # Pre-download ML models to avoid runtime download (conditional)
 ARG PRELOAD_ML_MODELS
 RUN if [ "$PRELOAD_ML_MODELS" = "true" ]; then \
@@ -5,16 +5,70 @@ set -e
 ENABLE_API="${HINDSIGHT_ENABLE_API:-true}"
 ENABLE_CP="${HINDSIGHT_ENABLE_CP:-true}"

-# Copy pre-cached PostgreSQL data if runtime directory is empty (first run with volume)
-if [ "$ENABLE_API" = "true" ]; then
-    PG0_CACHE="/home/hindsight/.pg0-cache"
-    PG0_HOME="/home/hindsight/.pg0"
-    if [ -d "$PG0_CACHE" ] && [ "$(ls -A $PG0_CACHE 2>/dev/null)" ]; then
-        if [ ! "$(ls -A $PG0_HOME 2>/dev/null)" ]; then
-            echo "📦 Copying pre-cached PostgreSQL data..."
-            cp -r "$PG0_CACHE"/* "$PG0_HOME"/ 2>/dev/null || true
-        fi
+# =============================================================================
+# Dependency waiting (opt-in via HINDSIGHT_WAIT_FOR_DEPS=true)
+#
+# Problem: When running with LM Studio, the LLM may take time to load models.
+# If Hindsight starts before LM Studio is ready, it fails on LLM verification.
+# This wait loop ensures dependencies are ready before starting.
+# =============================================================================
+if [ "${HINDSIGHT_WAIT_FOR_DEPS:-false}" = "true" ]; then
+    LLM_BASE_URL="${HINDSIGHT_API_LLM_BASE_URL:-http://host.docker.internal:1234/v1}"
+    MAX_RETRIES="${HINDSIGHT_RETRY_MAX:-0}"  # 0 = infinite
+    RETRY_INTERVAL="${HINDSIGHT_RETRY_INTERVAL:-10}"
+
+    # Check if external database is configured (skip check for embedded pg0)
+    SKIP_DB_CHECK=false
+    if [ -z "${HINDSIGHT_API_DATABASE_URL}" ]; then
+        SKIP_DB_CHECK=true
+    else
+        DB_CHECK_HOST=$(echo "$HINDSIGHT_API_DATABASE_URL" | sed -E 's|.*@([^:/]+):([0-9]+)/.*|\1 \2|')
    fi
+
+    check_db() {
+        if $SKIP_DB_CHECK; then
+            return 0
+        fi
+        if command -v pg_isready &> /dev/null; then
+            pg_isready -h $(echo $DB_CHECK_HOST | cut -d' ' -f1) -p $(echo $DB_CHECK_HOST | cut -d' ' -f2) &>/dev/null
+        else
+            python3 -c "import socket; s=socket.socket(); s.settimeout(5); exit(0 if s.connect_ex(('$(echo $DB_CHECK_HOST | cut -d' ' -f1)', $(echo $DB_CHECK_HOST | cut -d' ' -f2))) == 0 else 1)" 2>/dev/null
+        fi
+    }
+
+    check_llm() {
+        curl -sf "${LLM_BASE_URL}/models" --connect-timeout 5 &>/dev/null
+    }
+
+    echo "⏳ Waiting for dependencies to be ready..."
+    attempt=1
+
+    while true; do
+        db_ok=false
+        llm_ok=false
+
+        if check_db; then
+            db_ok=true
+        fi
+
+        if check_llm; then
+            llm_ok=true
+        fi
+
+        if $db_ok && $llm_ok; then
+            echo "✅ Dependencies ready!"
+            break
+        fi
+
+        if [ "$MAX_RETRIES" -ne 0 ] && [ "$attempt" -ge "$MAX_RETRIES" ]; then
+            echo "❌ Max retries ($MAX_RETRIES) reached. Dependencies not available."
+            exit 1
+        fi
+
+        echo "   Attempt $attempt: DB=$( $db_ok && echo 'ok' || echo 'waiting' ), LLM=$( $llm_ok && echo 'ok' || echo 'waiting' )"
+        sleep "$RETRY_INTERVAL"
+        ((attempt++))
+    done
 fi

 # Track PIDs for wait
@@ -2,8 +2,8 @@ apiVersion: v2
 name: hindsight
 description: Hindsight helm chart
 type: application
-version: 0.1.12
-appVersion: "0.1.12"
+version: 0.2.1
+appVersion: "0.2.1"
 keywords:
  - ai
  - memory
@@ -110,3 +110,14 @@ API URL for control plane
 {{- define "hindsight.apiUrl" -}}
 {{- printf "http://%s-api:%d" (include "hindsight.fullname" .) (.Values.api.service.port | int) }}
 {{- end }}
+
+{{/*
+Get the name of the secret to use
+*/}}
+{{- define "hindsight.secretName" -}}
+{{- if .Values.existingSecret }}
+{{- .Values.existingSecret }}
+{{- else }}
+{{- printf "%s-secret" (include "hindsight.fullname" .) }}
+{{- end }}
+{{- end }}
@@ -15,7 +15,9 @@ spec:
  template:
    metadata:
      annotations:
+        {{- if not .Values.existingSecret }}
        checksum/secret: {{ include (print $.Template.BasePath "/secret.yaml") . | sha256sum }}
+        {{- end }}
        {{- with .Values.podAnnotations }}
        {{- toYaml . | nindent 8 }}
        {{- end }}
@@ -37,27 +39,36 @@ spec:
        - name: http
          containerPort: {{ .Values.api.service.targetPort }}
          protocol: TCP
+        {{- if .Values.existingSecret }}
+        envFrom:
+        - secretRef:
+            name: {{ .Values.existingSecret }}
+        {{- end }}
        env:
-        - name: HINDSIGHT_API_DATABASE_URL
-          value: {{ include "hindsight.databaseUrl" . | quote }}
+        {{- /* POSTGRES_PASSWORD must be defined before DATABASE_URL for $(VAR) interpolation */}}
        {{- if not .Values.postgresql.enabled }}
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
-              name: {{ include "hindsight.fullname" . }}-secret
+              name: {{ include "hindsight.secretName" . }}
              key: postgres-password
        {{- end }}
+        - name: HINDSIGHT_API_DATABASE_URL
+          value: {{ include "hindsight.databaseUrl" . | quote }}
        {{- range $key, $value := .Values.api.env }}
        - name: {{ $key }}
          value: {{ $value | quote }}
        {{- end }}
+        {{- /* Only use api.secrets when not using existingSecret (for chart-managed secrets) */}}
+        {{- if not .Values.existingSecret }}
        {{- range $key, $value := .Values.api.secrets }}
        - name: {{ $key }}
          valueFrom:
            secretKeyRef:
-              name: {{ include "hindsight.fullname" $ }}-secret
+              name: {{ include "hindsight.secretName" $ }}
              key: {{ $key }}
        {{- end }}
+        {{- end }}
        livenessProbe:
          {{- toYaml .Values.api.livenessProbe | nindent 10 }}
        readinessProbe:
@@ -15,7 +15,9 @@ spec:
  template:
    metadata:
      annotations:
+        {{- if not .Values.existingSecret }}
        checksum/secret: {{ include (print $.Template.BasePath "/secret.yaml") . | sha256sum }}
+        {{- end }}
        {{- with .Values.podAnnotations }}
        {{- toYaml . | nindent 8 }}
        {{- end }}
@@ -37,6 +39,11 @@ spec:
        - name: http
          containerPort: {{ .Values.controlPlane.service.targetPort }}
          protocol: TCP
+        {{- if .Values.existingSecret }}
+        envFrom:
+        - secretRef:
+            name: {{ .Values.existingSecret }}
+        {{- end }}
        env:
        - name: HINDSIGHT_CP_DATAPLANE_API_URL
          value: {{ include "hindsight.apiUrl" . | quote }}
@@ -44,13 +51,16 @@ spec:
        - name: {{ $key }}
          value: {{ $value | quote }}
        {{- end }}
+        {{- /* Only use controlPlane.secrets when not using existingSecret (for chart-managed secrets) */}}
+        {{- if not .Values.existingSecret }}
        {{- range $key, $value := .Values.controlPlane.secrets }}
        - name: {{ $key }}
          valueFrom:
            secretKeyRef:
-              name: {{ include "hindsight.fullname" $ }}-secret
+              name: {{ include "hindsight.secretName" $ }}
              key: {{ $key }}
        {{- end }}
+        {{- end }}
        livenessProbe:
          {{- toYaml .Values.controlPlane.livenessProbe | nindent 10 }}
        readinessProbe:
@@ -1,7 +1,8 @@
+{{- if not .Values.existingSecret }}
 apiVersion: v1
 kind: Secret
 metadata:
-  name: {{ include "hindsight.fullname" . }}-secret
+  name: {{ include "hindsight.secretName" . }}
  labels:
    {{- include "hindsight.labels" . | nindent 4 }}
 type: Opaque
@@ -15,3 +16,4 @@ data:
  {{- if and (not .Values.postgresql.enabled) .Values.postgresql.external.password }}
  postgres-password: {{ .Values.postgresql.external.password | b64enc | quote }}
  {{- end }}
+{{- end }}
@@ -3,6 +3,15 @@
 # Chart version - use this to set a consistent image tag across all components
 version: "0.1.1"

+# Use an existing secret instead of creating one from values
+# When set, all keys from this secret are injected as environment variables via envFrom
+# Required keys:
+#   - postgres-password: PostgreSQL password (when postgresql.enabled=false)
+# Optional keys (any key becomes an env var):
+#   - HINDSIGHT_API_LLM_API_KEY: API key for LLM provider
+#   - Any other env vars you want to inject
+# existingSecret: "my-hindsight-secret"
+
 # Global settings
 replicaCount: 1

@@ -80,7 +80,7 @@ Configure via environment variables:
 | Variable | Description | Default |
 |----------|-------------|---------|
 | `HINDSIGHT_API_DATABASE_URL` | PostgreSQL connection string | `pg0` (embedded) |
-| `HINDSIGHT_API_LLM_PROVIDER` | `openai`, `groq`, `gemini`, `ollama` | `openai` |
+| `HINDSIGHT_API_LLM_PROVIDER` | `openai`, `anthropic`, `gemini`, `groq`, `ollama`, `lmstudio` | `openai` |
 | `HINDSIGHT_API_LLM_API_KEY` | API key for LLM provider | - |
 | `HINDSIGHT_API_LLM_MODEL` | Model name | `gpt-4o-mini` |
 | `HINDSIGHT_API_HOST` | Server bind address | `0.0.0.0` |
@@ -0,0 +1 @@
+# Admin CLI for Hindsight
@@ -0,0 +1,252 @@
+"""
+Hindsight Admin CLI - backup and restore operations.
+"""
+
+import asyncio
+import io
+import json
+import logging
+import zipfile
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any
+
+import asyncpg
+import typer
+
+from ..config import HindsightConfig
+from ..pg0 import parse_pg0_url, resolve_database_url
+
+
+def _fq_table(table: str, schema: str) -> str:
+    """Get fully-qualified table name with schema prefix."""
+    return f"{schema}.{table}"
+
+
+# Setup logging
+logging.basicConfig(
+    level=logging.INFO,
+    format="%(message)s",
+)
+logger = logging.getLogger(__name__)
+
+app = typer.Typer(name="hindsight-admin", help="Hindsight administrative commands")
+
+# Tables to backup/restore in dependency order
+# Import must happen in this order due to foreign key constraints
+BACKUP_TABLES = [
+    "banks",
+    "documents",
+    "entities",
+    "chunks",
+    "memory_units",
+    "unit_entities",
+    "entity_cooccurrences",
+    "memory_links",
+]
+
+MANIFEST_VERSION = "1"
+
+
+async def _backup(database_url: str, output_path: Path, schema: str = "public") -> dict[str, Any]:
+    """Backup all tables to a zip file using binary COPY protocol."""
+    conn = await asyncpg.connect(database_url)
+    try:
+        tables: dict[str, Any] = {}
+        manifest: dict[str, Any] = {
+            "version": MANIFEST_VERSION,
+            "created_at": datetime.now(timezone.utc).isoformat(),
+            "schema": schema,
+            "tables": tables,
+        }
+
+        # Use a transaction with REPEATABLE READ isolation to get a consistent
+        # snapshot across all tables. This prevents race conditions where
+        # entity_cooccurrences could reference entities created after the
+        # entities table was backed up.
+        async with conn.transaction(isolation="repeatable_read"):
+            with zipfile.ZipFile(output_path, "w", zipfile.ZIP_DEFLATED) as zf:
+                for i, table in enumerate(BACKUP_TABLES, 1):
+                    typer.echo(f"  [{i}/{len(BACKUP_TABLES)}] Backing up {table}...", nl=False)
+
+                    buffer = io.BytesIO()
+
+                    # Use binary COPY for exact type preservation
+                    # asyncpg requires schema_name as separate parameter
+                    await conn.copy_from_table(table, schema_name=schema, output=buffer, format="binary")
+
+                    data = buffer.getvalue()
+                    zf.writestr(f"{table}.bin", data)
+
+                    # Get row count for manifest
+                    qualified_table = _fq_table(table, schema)
+                    row_count = await conn.fetchval(f"SELECT COUNT(*) FROM {qualified_table}")
+                    tables[table] = {
+                        "rows": row_count,
+                        "size_bytes": len(data),
+                    }
+
+                    typer.echo(f" {row_count} rows")
+
+                zf.writestr("manifest.json", json.dumps(manifest, indent=2))
+
+        return manifest
+    finally:
+        await conn.close()
+
+
+async def _restore(database_url: str, input_path: Path, schema: str = "public") -> dict[str, Any]:
+    """Restore all tables from a zip file using binary COPY protocol."""
+    conn = await asyncpg.connect(database_url)
+    try:
+        with zipfile.ZipFile(input_path, "r") as zf:
+            # Read and validate manifest
+            manifest: dict[str, Any] = json.loads(zf.read("manifest.json"))
+            if manifest.get("version") != MANIFEST_VERSION:
+                raise ValueError(f"Unsupported backup version: {manifest.get('version')}")
+
+            # Use a transaction for atomic restore - either all tables are
+            # restored or none are, preventing partial/inconsistent state.
+            async with conn.transaction():
+                typer.echo("  Clearing existing data...")
+                # Truncate tables in reverse order (respects FK constraints)
+                for table in reversed(BACKUP_TABLES):
+                    qualified_table = _fq_table(table, schema)
+                    await conn.execute(f"TRUNCATE TABLE {qualified_table} CASCADE")
+
+                # Restore tables in forward order
+                for i, table in enumerate(BACKUP_TABLES, 1):
+                    filename = f"{table}.bin"
+                    if filename not in zf.namelist():
+                        typer.echo(f"  [{i}/{len(BACKUP_TABLES)}] {table}: skipped (not in backup)")
+                        continue
+
+                    expected_rows = manifest["tables"].get(table, {}).get("rows", "?")
+                    typer.echo(f"  [{i}/{len(BACKUP_TABLES)}] Restoring {table}... {expected_rows} rows")
+
+                    data = zf.read(filename)
+                    buffer = io.BytesIO(data)
+                    # asyncpg requires schema_name as separate parameter
+                    await conn.copy_to_table(table, schema_name=schema, source=buffer, format="binary")
+
+                # Refresh materialized view
+                typer.echo("  Refreshing materialized views...")
+                await conn.execute(f"REFRESH MATERIALIZED VIEW {_fq_table('memory_units_bm25', schema)}")
+
+        return manifest
+    finally:
+        await conn.close()
+
+
+async def _run_backup(db_url: str, output: Path, schema: str = "public") -> dict[str, Any]:
+    """Resolve database URL and run backup."""
+    is_pg0, instance_name, _ = parse_pg0_url(db_url)
+    if is_pg0:
+        typer.echo(f"Starting embedded PostgreSQL (instance: {instance_name})...")
+    resolved_url = await resolve_database_url(db_url)
+    return await _backup(resolved_url, output, schema)
+
+
+async def _run_restore(db_url: str, input_file: Path, schema: str = "public") -> dict[str, Any]:
+    """Resolve database URL and run restore."""
+    is_pg0, instance_name, _ = parse_pg0_url(db_url)
+    if is_pg0:
+        typer.echo(f"Starting embedded PostgreSQL (instance: {instance_name})...")
+    resolved_url = await resolve_database_url(db_url)
+    return await _restore(resolved_url, input_file, schema)
+
+
+@app.command()
+def backup(
+    output: Path = typer.Argument(..., help="Output file path (.zip)"),
+    schema: str = typer.Option("public", "--schema", "-s", help="Database schema to backup"),
+):
+    """Backup the Hindsight database to a zip file."""
+    config = HindsightConfig.from_env()
+
+    if not config.database_url:
+        typer.echo("Error: Database URL not configured.", err=True)
+        typer.echo("Set HINDSIGHT_API_DATABASE_URL environment variable.", err=True)
+        raise typer.Exit(1)
+
+    if output.suffix != ".zip":
+        output = output.with_suffix(".zip")
+
+    typer.echo(f"Backing up database (schema: {schema}) to {output}...")
+
+    manifest = asyncio.run(_run_backup(config.database_url, output, schema))
+
+    total_rows = sum(t["rows"] for t in manifest["tables"].values())
+    typer.echo(f"Backed up {total_rows} rows across {len(BACKUP_TABLES)} tables")
+    typer.echo(f"Backup saved to {output}")
+
+
+@app.command()
+def restore(
+    input_file: Path = typer.Argument(..., help="Input backup file (.zip)"),
+    schema: str = typer.Option("public", "--schema", "-s", help="Database schema to restore to"),
+    yes: bool = typer.Option(False, "--yes", "-y", help="Skip confirmation prompt"),
+):
+    """Restore the database from a backup file. WARNING: This deletes all existing data."""
+    config = HindsightConfig.from_env()
+
+    if not config.database_url:
+        typer.echo("Error: Database URL not configured.", err=True)
+        typer.echo("Set HINDSIGHT_API_DATABASE_URL environment variable.", err=True)
+        raise typer.Exit(1)
+
+    if not input_file.exists():
+        typer.echo(f"Error: File not found: {input_file}", err=True)
+        raise typer.Exit(1)
+
+    if not yes:
+        typer.confirm(
+            "This will DELETE all existing data and replace it with the backup. Continue?",
+            abort=True,
+        )
+
+    typer.echo(f"Restoring database (schema: {schema}) from {input_file}...")
+
+    manifest = asyncio.run(_run_restore(config.database_url, input_file, schema))
+
+    total_rows = sum(t["rows"] for t in manifest["tables"].values())
+    typer.echo(f"Restored {total_rows} rows across {len(BACKUP_TABLES)} tables")
+    typer.echo("Restore complete")
+
+
+async def _run_migration(db_url: str, schema: str = "public") -> None:
+    """Resolve database URL and run migrations."""
+    from ..migrations import run_migrations
+
+    is_pg0, instance_name, _ = parse_pg0_url(db_url)
+    if is_pg0:
+        typer.echo(f"Starting embedded PostgreSQL (instance: {instance_name})...")
+    resolved_url = await resolve_database_url(db_url)
+    run_migrations(resolved_url, schema=schema)
+
+
+@app.command(name="run-db-migration")
+def run_db_migration(
+    schema: str = typer.Option("public", "--schema", "-s", help="Database schema to run migrations on"),
+):
+    """Run database migrations to the latest version."""
+    config = HindsightConfig.from_env()
+
+    if not config.database_url:
+        typer.echo("Error: Database URL not configured.", err=True)
+        typer.echo("Set HINDSIGHT_API_DATABASE_URL environment variable.", err=True)
+        raise typer.Exit(1)
+
+    typer.echo(f"Running database migrations (schema: {schema})...")
+
+    asyncio.run(_run_migration(config.database_url, schema))
+
+    typer.echo("Database migrations completed successfully")
+
+
+def main():
+    app()
+
+
+if __name__ == "__main__":
+    main()
@@ -5,6 +5,7 @@ Provides both HTTP REST API and MCP (Model Context Protocol) server.
 """

 import logging
+from contextlib import asynccontextmanager
 from typing import Optional

 from fastapi import FastAPI
@@ -45,6 +46,18 @@ def create_app(
        # Both HTTP and MCP
        app = create_app(memory, mcp_api_enabled=True)
    """
+    mcp_app = None
+
+    # Create MCP app first if enabled (we need its lifespan for chaining)
+    if mcp_api_enabled:
+        try:
+            from .mcp import create_mcp_app
+
+            mcp_app = create_mcp_app(memory=memory)
+        except ImportError as e:
+            logger.error(f"MCP server requested but dependencies not available: {e}")
+            logger.error("Install with: pip install hindsight-api[mcp]")
+            raise

    # Import and create HTTP API if enabled
    if http_api_enabled:
@@ -57,20 +70,31 @@ def create_app(
        app = FastAPI(title="Hindsight API", version="0.0.7")
        logger.info("HTTP REST API disabled")

-    # Mount MCP server if enabled
-    if mcp_api_enabled:
-        try:
-            from .mcp import create_mcp_app
+    # Mount MCP server and chain its lifespan if enabled
+    if mcp_app is not None:
+        # Get the MCP app's underlying Starlette app for lifespan access
+        mcp_starlette_app = mcp_app.mcp_app

-            # Create MCP app with dynamic bank_id support
-            # Supports: /mcp/{bank_id}/sse (bank-specific SSE endpoint)
-            mcp_app = create_mcp_app(memory=memory)
-            app.mount(mcp_mount_path, mcp_app)
-            logger.info(f"MCP server enabled at {mcp_mount_path}/{{bank_id}}/sse")
-        except ImportError as e:
-            logger.error(f"MCP server requested but dependencies not available: {e}")
-            logger.error("Install with: pip install hindsight-api[mcp]")
-            raise
+        # Store the original lifespan
+        original_lifespan = app.router.lifespan_context
+
+        @asynccontextmanager
+        async def chained_lifespan(app_instance: FastAPI):
+            """Chain the MCP lifespan with the main app lifespan."""
+            # Start MCP lifespan first
+            async with mcp_starlette_app.router.lifespan_context(mcp_starlette_app):
+                logger.info("MCP lifespan started")
+                # Then start the original app lifespan
+                async with original_lifespan(app_instance):
+                    yield
+            logger.info("MCP lifespan stopped")
+
+        # Replace the app's lifespan with the chained version
+        app.router.lifespan_context = chained_lifespan
+
+        # Mount the MCP middleware
+        app.mount(mcp_mount_path, mcp_app)
+        logger.info(f"MCP server enabled at {mcp_mount_path}/")

    return app

@@ -14,6 +14,8 @@ from typing import Any

 from fastapi import Depends, FastAPI, Header, HTTPException, Query

+from hindsight_api.extensions import AuthenticationError
+

 def _parse_metadata(metadata: Any) -> dict[str, Any]:
    """Parse metadata that may be a dict, JSON string, or None."""
@@ -29,13 +31,13 @@ def _parse_metadata(metadata: Any) -> dict[str, Any]:
    return {}


-from pydantic import BaseModel, ConfigDict, Field
+from pydantic import BaseModel, ConfigDict, Field, field_validator

 from hindsight_api import MemoryEngine
 from hindsight_api.engine.db_utils import acquire_with_retry
 from hindsight_api.engine.memory_engine import Budget, fq_table
-from hindsight_api.engine.response_models import VALID_RECALL_FACT_TYPES
-from hindsight_api.extensions import HttpExtension, load_extension
+from hindsight_api.engine.response_models import VALID_RECALL_FACT_TYPES, TokenUsage
+from hindsight_api.extensions import HttpExtension, OperationValidationError, load_extension
 from hindsight_api.metrics import create_metrics_collector, get_metrics_collector, initialize_metrics
 from hindsight_api.models import RequestContext

@@ -279,6 +281,13 @@ class RecallResponse(BaseModel):
    chunks: dict[str, ChunkData] | None = Field(default=None, description="Chunks for facts, keyed by chunk_id")


+class EntityInput(BaseModel):
+    """Entity to associate with retained content."""
+
+    text: str = Field(description="The entity name/text")
+    type: str | None = Field(default=None, description="Optional entity type (e.g., 'PERSON', 'ORG', 'CONCEPT')")
+
+
 class MemoryItem(BaseModel):
    """Single memory item for retain."""

@@ -290,8 +299,9 @@ class MemoryItem(BaseModel):
                "context": "team meeting",
                "metadata": {"source": "slack", "channel": "engineering"},
                "document_id": "meeting_notes_2024_01_15",
+                "entities": [{"text": "Alice"}, {"text": "ML model", "type": "CONCEPT"}],
            }
-        }
+        },
    )

    content: str
@@ -299,6 +309,27 @@ class MemoryItem(BaseModel):
    context: str | None = None
    metadata: dict[str, str] | None = None
    document_id: str | None = Field(default=None, description="Optional document ID for this memory item.")
+    entities: list[EntityInput] | None = Field(
+        default=None,
+        description="Optional entities to combine with auto-extracted entities.",
+    )
+
+    @field_validator("timestamp", mode="before")
+    @classmethod
+    def validate_timestamp(cls, v):
+        if v is None or v == "":
+            return None
+        if isinstance(v, datetime):
+            return v
+        if isinstance(v, str):
+            try:
+                # Try parsing as ISO format
+                return datetime.fromisoformat(v.replace("Z", "+00:00"))
+            except ValueError as e:
+                raise ValueError(
+                    f"Invalid timestamp/event_date format: '{v}'. Expected ISO format like '2024-01-15T10:30:00' or '2024-01-15T10:30:00Z'"
+                ) from e
+        raise ValueError(f"timestamp must be a string or datetime, got {type(v).__name__}")


 class RetainRequest(BaseModel):
@@ -333,7 +364,15 @@ class RetainResponse(BaseModel):

    model_config = ConfigDict(
        populate_by_name=True,
-        json_schema_extra={"example": {"success": True, "bank_id": "user123", "items_count": 2, "async": False}},
+        json_schema_extra={
+            "example": {
+                "success": True,
+                "bank_id": "user123",
+                "items_count": 2,
+                "async": False,
+                "usage": {"input_tokens": 500, "output_tokens": 100, "total_tokens": 600},
+            }
+        },
    )

    success: bool
@@ -342,6 +381,10 @@ class RetainResponse(BaseModel):
    is_async: bool = Field(
        alias="async", serialization_alias="async", description="Whether the operation was processed asynchronously"
    )
+    usage: TokenUsage | None = Field(
+        default=None,
+        description="Token usage metrics for LLM calls during fact extraction (only present for synchronous operations)",
+    )


 class FactsIncludeOptions(BaseModel):
@@ -368,7 +411,16 @@ class ReflectRequest(BaseModel):
                "query": "What do you think about artificial intelligence?",
                "budget": "low",
                "context": "This is for a research paper on AI ethics",
+                "max_tokens": 4096,
                "include": {"facts": {}},
+                "response_schema": {
+                    "type": "object",
+                    "properties": {
+                        "summary": {"type": "string"},
+                        "key_points": {"type": "array", "items": {"type": "string"}},
+                    },
+                    "required": ["summary", "key_points"],
+                },
            }
        }
    )
@@ -376,9 +428,14 @@ class ReflectRequest(BaseModel):
    query: str
    budget: Budget = Budget.LOW
    context: str | None = None
+    max_tokens: int = Field(default=4096, description="Maximum tokens for the response")
    include: ReflectIncludeOptions = Field(
        default_factory=ReflectIncludeOptions, description="Options for including additional data (disabled by default)"
    )
+    response_schema: dict | None = Field(
+        default=None,
+        description="Optional JSON Schema for structured output. When provided, the response will include a 'structured_output' field with the LLM response parsed according to this schema.",
+    )


 class OpinionItem(BaseModel):
@@ -423,12 +480,25 @@ class ReflectResponse(BaseModel):
                    {"id": "123", "text": "AI is used in healthcare", "type": "world"},
                    {"id": "456", "text": "I discussed AI applications last week", "type": "experience"},
                ],
+                "structured_output": {
+                    "summary": "AI is transformative",
+                    "key_points": ["Used in healthcare", "Discussed recently"],
+                },
+                "usage": {"input_tokens": 1500, "output_tokens": 500, "total_tokens": 2000},
            }
        }
    )

    text: str
    based_on: list[ReflectFact] = []  # Facts used to generate the response
+    structured_output: dict | None = Field(
+        default=None,
+        description="Structured output parsed according to the request's response_schema. Only present when response_schema was provided in the request.",
+    )
+    usage: TokenUsage | None = Field(
+        default=None,
+        description="Token usage metrics for LLM calls during reflection.",
+    )


 class BanksResponse(BaseModel):
@@ -577,6 +647,7 @@ class GraphDataResponse(BaseModel):
                    }
                ],
                "total_units": 2,
+                "limit": 1000,
            }
        }
    )
@@ -585,6 +656,7 @@ class GraphDataResponse(BaseModel):
    edges: list[dict[str, Any]]
    table_rows: list[dict[str, Any]]
    total_units: int
+    limit: int


 class ListMemoryUnitsResponse(BaseModel):
@@ -672,6 +744,26 @@ class DocumentResponse(BaseModel):
    memory_unit_count: int


+class DeleteDocumentResponse(BaseModel):
+    """Response model for delete document endpoint."""
+
+    model_config = ConfigDict(
+        json_schema_extra={
+            "example": {
+                "success": True,
+                "message": "Document 'session_1' and 5 associated memory units deleted successfully",
+                "document_id": "session_1",
+                "memory_units_deleted": 5,
+            }
+        }
+    )
+
+    success: bool
+    message: str
+    document_id: str
+    memory_units_deleted: int
+
+
 class ChunkResponse(BaseModel):
    """Response model for get chunk endpoint."""

@@ -708,6 +800,108 @@ class DeleteResponse(BaseModel):
    deleted_count: int | None = None


+class BankStatsResponse(BaseModel):
+    """Response model for bank statistics endpoint."""
+
+    model_config = ConfigDict(
+        json_schema_extra={
+            "example": {
+                "bank_id": "user123",
+                "total_nodes": 150,
+                "total_links": 300,
+                "total_documents": 10,
+                "nodes_by_fact_type": {"fact": 100, "preference": 30, "observation": 20},
+                "links_by_link_type": {"temporal": 150, "semantic": 100, "entity": 50},
+                "links_by_fact_type": {"fact": 200, "preference": 60, "observation": 40},
+                "links_breakdown": {"fact": {"temporal": 100, "semantic": 60, "entity": 40}},
+                "pending_operations": 2,
+                "failed_operations": 0,
+            }
+        }
+    )
+
+    bank_id: str
+    total_nodes: int
+    total_links: int
+    total_documents: int
+    nodes_by_fact_type: dict[str, int]
+    links_by_link_type: dict[str, int]
+    links_by_fact_type: dict[str, int]
+    links_breakdown: dict[str, dict[str, int]]
+    pending_operations: int
+    failed_operations: int
+
+
+class OperationResponse(BaseModel):
+    """Response model for a single async operation."""
+
+    model_config = ConfigDict(
+        json_schema_extra={
+            "example": {
+                "id": "550e8400-e29b-41d4-a716-446655440000",
+                "task_type": "retain",
+                "items_count": 5,
+                "document_id": "meeting-notes-2024",
+                "created_at": "2024-01-15T10:30:00Z",
+                "status": "pending",
+                "error_message": None,
+            }
+        }
+    )
+
+    id: str
+    task_type: str
+    items_count: int
+    document_id: str | None
+    created_at: str
+    status: str
+    error_message: str | None
+
+
+class OperationsListResponse(BaseModel):
+    """Response model for list operations endpoint."""
+
+    model_config = ConfigDict(
+        json_schema_extra={
+            "example": {
+                "bank_id": "user123",
+                "operations": [
+                    {
+                        "id": "550e8400-e29b-41d4-a716-446655440000",
+                        "task_type": "retain",
+                        "items_count": 5,
+                        "document_id": None,
+                        "created_at": "2024-01-15T10:30:00Z",
+                        "status": "pending",
+                        "error_message": None,
+                    }
+                ],
+            }
+        }
+    )
+
+    bank_id: str
+    operations: list[OperationResponse]
+
+
+class CancelOperationResponse(BaseModel):
+    """Response model for cancel operation endpoint."""
+
+    model_config = ConfigDict(
+        json_schema_extra={
+            "example": {
+                "success": True,
+                "message": "Operation 550e8400-e29b-41d4-a716-446655440000 cancelled",
+                "operation_id": "550e8400-e29b-41d4-a716-446655440000",
+            }
+        }
+    )
+
+    success: bool
+    message: str
+    operation_id: str
+
+
 def create_app(
    memory: MemoryEngine,
    initialize_memory: bool = True,
@@ -828,6 +1022,16 @@ def _register_routes(app: FastAPI):
                api_key = authorization.strip()
        return RequestContext(api_key=api_key)

+    # Global exception handler for authentication errors
+    @app.exception_handler(AuthenticationError)
+    async def authentication_error_handler(request, exc: AuthenticationError):
+        from fastapi.responses import JSONResponse
+
+        return JSONResponse(
+            status_code=401,
+            content={"detail": str(exc)},
+        )
+
    @app.get(
        "/health",
        summary="Health check endpoint",
@@ -864,17 +1068,22 @@ def _register_routes(app: FastAPI):
        "/v1/default/banks/{bank_id}/graph",
        response_model=GraphDataResponse,
        summary="Get memory graph data",
-        description="Retrieve graph data for visualization, optionally filtered by type (world/experience/opinion). Limited to 1000 most recent items.",
+        description="Retrieve graph data for visualization, optionally filtered by type (world/experience/opinion).",
        operation_id="get_graph",
        tags=["Memory"],
    )
    async def api_graph(
-        bank_id: str, type: str | None = None, request_context: RequestContext = Depends(get_request_context)
+        bank_id: str,
+        type: str | None = None,
+        limit: int = 1000,
+        request_context: RequestContext = Depends(get_request_context),
    ):
        """Get graph data from database, filtered by bank_id and optionally by type."""
        try:
-            data = await app.state.memory.get_graph_data(bank_id, type, request_context=request_context)
+            data = await app.state.memory.get_graph_data(bank_id, type, limit=limit, request_context=request_context)
            return data
+        except (AuthenticationError, HTTPException):
+            raise
        except Exception as e:
            import traceback

@@ -921,6 +1130,8 @@ def _register_routes(app: FastAPI):
                request_context=request_context,
            )
            return data
+        except (AuthenticationError, HTTPException):
+            raise
        except Exception as e:
            import traceback

@@ -1037,6 +1248,10 @@ def _register_routes(app: FastAPI):
            )
        except HTTPException:
            raise
+        except OperationValidationError as e:
+            raise HTTPException(status_code=e.status_code, detail=e.reason)
+        except (AuthenticationError, HTTPException):
+            raise
        except Exception as e:
            import traceback

@@ -1072,6 +1287,8 @@ def _register_routes(app: FastAPI):
                    query=request.query,
                    budget=request.budget,
                    context=request.context,
+                    max_tokens=request.max_tokens,
+                    response_schema=request.response_schema,
                    request_context=request_context,
                )

@@ -1094,8 +1311,14 @@ def _register_routes(app: FastAPI):
            return ReflectResponse(
                text=core_result.text,
                based_on=based_on_facts,
+                structured_output=core_result.structured_output,
+                usage=core_result.usage,
            )

+        except OperationValidationError as e:
+            raise HTTPException(status_code=e.status_code, detail=e.reason)
+        except (AuthenticationError, HTTPException):
+            raise
        except Exception as e:
            import traceback

@@ -1116,6 +1339,8 @@ def _register_routes(app: FastAPI):
        try:
            banks = await app.state.memory.list_banks(request_context=request_context)
            return BankListResponse(banks=banks)
+        except (AuthenticationError, HTTPException):
+            raise
        except Exception as e:
            import traceback

@@ -1125,6 +1350,7 @@ def _register_routes(app: FastAPI):

    @app.get(
        "/v1/default/banks/{bank_id}/stats",
+        response_model=BankStatsResponse,
        summary="Get statistics for memory bank",
        description="Get statistics about nodes and links for a specific agent",
        operation_id="get_agent_stats",
@@ -1225,19 +1451,21 @@ def _register_routes(app: FastAPI):
                total_nodes = sum(nodes_by_type.values())
                total_links = sum(links_by_type.values())

-                return {
-                    "bank_id": bank_id,
-                    "total_nodes": total_nodes,
-                    "total_links": total_links,
-                    "total_documents": total_documents,
-                    "nodes_by_fact_type": nodes_by_type,
-                    "links_by_link_type": links_by_type,
-                    "links_by_fact_type": links_by_fact_type,
-                    "links_breakdown": links_breakdown,
-                    "pending_operations": pending_operations,
-                    "failed_operations": failed_operations,
-                }
+                return BankStatsResponse(
+                    bank_id=bank_id,
+                    total_nodes=total_nodes,
+                    total_links=total_links,
+                    total_documents=total_documents,
+                    nodes_by_fact_type=nodes_by_type,
+                    links_by_link_type=links_by_type,
+                    links_by_fact_type=links_by_fact_type,
+                    links_breakdown=links_breakdown,
+                    pending_operations=pending_operations,
+                    failed_operations=failed_operations,
+                )

+        except (AuthenticationError, HTTPException):
+            raise
        except Exception as e:
            import traceback

@@ -1262,6 +1490,8 @@ def _register_routes(app: FastAPI):
        try:
            entities = await app.state.memory.list_entities(bank_id, limit=limit, request_context=request_context)
            return EntityListResponse(items=[EntityListItem(**e) for e in entities])
+        except (AuthenticationError, HTTPException):
+            raise
        except Exception as e:
            import traceback

@@ -1299,7 +1529,7 @@ def _register_routes(app: FastAPI):
                    for obs in entity["observations"]
                ],
            )
-        except HTTPException:
+        except (AuthenticationError, HTTPException):
            raise
        except Exception as e:
            import traceback
@@ -1352,7 +1582,7 @@ def _register_routes(app: FastAPI):
                    for obs in entity["observations"]
                ],
            )
-        except HTTPException:
+        except (AuthenticationError, HTTPException):
            raise
        except Exception as e:
            import traceback
@@ -1390,6 +1620,8 @@ def _register_routes(app: FastAPI):
                bank_id=bank_id, search_query=q, limit=limit, offset=offset, request_context=request_context
            )
            return data
+        except (AuthenticationError, HTTPException):
+            raise
        except Exception as e:
            import traceback

@@ -1398,7 +1630,7 @@ def _register_routes(app: FastAPI):
            raise HTTPException(status_code=500, detail=str(e))

    @app.get(
-        "/v1/default/banks/{bank_id}/documents/{document_id}",
+        "/v1/default/banks/{bank_id}/documents/{document_id:path}",
        response_model=DocumentResponse,
        summary="Get document details",
        description="Get a specific document including its original text",
@@ -1420,7 +1652,7 @@ def _register_routes(app: FastAPI):
            if not document:
                raise HTTPException(status_code=404, detail="Document not found")
            return document
-        except HTTPException:
+        except (AuthenticationError, HTTPException):
            raise
        except Exception as e:
            import traceback
@@ -1430,7 +1662,7 @@ def _register_routes(app: FastAPI):
            raise HTTPException(status_code=500, detail=str(e))

    @app.get(
-        "/v1/default/chunks/{chunk_id}",
+        "/v1/default/chunks/{chunk_id:path}",
        response_model=ChunkResponse,
        summary="Get chunk details",
        description="Get a specific chunk by its ID",
@@ -1449,7 +1681,7 @@ def _register_routes(app: FastAPI):
            if not chunk:
                raise HTTPException(status_code=404, detail="Chunk not found")
            return chunk
-        except HTTPException:
+        except (AuthenticationError, HTTPException):
            raise
        except Exception as e:
            import traceback
@@ -1459,7 +1691,8 @@ def _register_routes(app: FastAPI):
            raise HTTPException(status_code=500, detail=str(e))

    @app.delete(
-        "/v1/default/banks/{bank_id}/documents/{document_id}",
+        "/v1/default/banks/{bank_id}/documents/{document_id:path}",
+        response_model=DeleteDocumentResponse,
        summary="Delete a document",
        description="Delete a document and all its associated memory units and links.\n\n"
        "This will cascade delete:\n"
@@ -1486,13 +1719,13 @@ def _register_routes(app: FastAPI):
            if result["document_deleted"] == 0:
                raise HTTPException(status_code=404, detail="Document not found")

-            return {
-                "success": True,
-                "message": f"Document '{document_id}' and {result['memory_units_deleted']} associated memory units deleted successfully",
-                "document_id": document_id,
-                "memory_units_deleted": result["memory_units_deleted"],
-            }
-        except HTTPException:
+            return DeleteDocumentResponse(
+                success=True,
+                message=f"Document '{document_id}' and {result['memory_units_deleted']} associated memory units deleted successfully",
+                document_id=document_id,
+                memory_units_deleted=result["memory_units_deleted"],
+            )
+        except (AuthenticationError, HTTPException):
            raise
        except Exception as e:
            import traceback
@@ -1503,6 +1736,7 @@ def _register_routes(app: FastAPI):

    @app.get(
        "/v1/default/banks/{bank_id}/operations",
+        response_model=OperationsListResponse,
        summary="List async operations",
        description="Get a list of all async operations (pending and failed) for a specific agent, including error messages for failed operations",
        operation_id="list_operations",
@@ -1512,10 +1746,12 @@ def _register_routes(app: FastAPI):
        """List all async operations (pending and failed) for a memory bank."""
        try:
            operations = await app.state.memory.list_operations(bank_id, request_context=request_context)
-            return {
-                "bank_id": bank_id,
-                "operations": operations,
-            }
+            return OperationsListResponse(
+                bank_id=bank_id,
+                operations=[OperationResponse(**op) for op in operations],
+            )
+        except (AuthenticationError, HTTPException):
+            raise
        except Exception as e:
            import traceback

@@ -1525,6 +1761,7 @@ def _register_routes(app: FastAPI):

    @app.delete(
        "/v1/default/banks/{bank_id}/operations/{operation_id}",
+        response_model=CancelOperationResponse,
        summary="Cancel a pending async operation",
        description="Cancel a pending async operation by removing it from the queue",
        operation_id="cancel_operation",
@@ -1542,9 +1779,11 @@ def _register_routes(app: FastAPI):
                raise HTTPException(status_code=400, detail=f"Invalid operation_id format: {operation_id}")

            result = await app.state.memory.cancel_operation(bank_id, operation_id, request_context=request_context)
-            return result
+            return CancelOperationResponse(**result)
        except ValueError as e:
            raise HTTPException(status_code=404, detail=str(e))
+        except (AuthenticationError, HTTPException):
+            raise
        except Exception as e:
            import traceback

@@ -1576,6 +1815,8 @@ def _register_routes(app: FastAPI):
                disposition=DispositionTraits(**disposition_dict),
                background=profile["background"],
            )
+        except (AuthenticationError, HTTPException):
+            raise
        except Exception as e:
            import traceback

@@ -1614,6 +1855,8 @@ def _register_routes(app: FastAPI):
                disposition=DispositionTraits(**disposition_dict),
                background=profile["background"],
            )
+        except (AuthenticationError, HTTPException):
+            raise
        except Exception as e:
            import traceback

@@ -1643,6 +1886,8 @@ def _register_routes(app: FastAPI):
                response.disposition = DispositionTraits(**result["disposition"])

            return response
+        except (AuthenticationError, HTTPException):
+            raise
        except Exception as e:
            import traceback

@@ -1694,6 +1939,8 @@ def _register_routes(app: FastAPI):
                disposition=DispositionTraits(**disposition_dict),
                background=final_profile["background"],
            )
+        except (AuthenticationError, HTTPException):
+            raise
        except Exception as e:
            import traceback

@@ -1721,6 +1968,8 @@ def _register_routes(app: FastAPI):
                + result.get("entities_deleted", 0)
                + result.get("documents_deleted", 0),
            )
+        except (AuthenticationError, HTTPException):
+            raise
        except Exception as e:
            import traceback

@@ -1772,6 +2021,8 @@ def _register_routes(app: FastAPI):
                    content_dict["metadata"] = item.metadata
                if item.document_id:
                    content_dict["document_id"] = item.document_id
+                if item.entities:
+                    content_dict["entities"] = [{"text": e.text, "type": e.type or "CONCEPT"} for e in item.entities]
                contents.append(content_dict)

            if request.async_:
@@ -1788,13 +2039,17 @@ def _register_routes(app: FastAPI):
            else:
                # Synchronous processing: wait for completion (record metrics)
                with metrics.record_operation("retain", bank_id=bank_id):
-                    result = await app.state.memory.retain_batch_async(
-                        bank_id=bank_id, contents=contents, request_context=request_context
+                    result, usage = await app.state.memory.retain_batch_async(
+                        bank_id=bank_id, contents=contents, request_context=request_context, return_usage=True
                    )

                return RetainResponse.model_validate(
-                    {"success": True, "bank_id": bank_id, "items_count": len(contents), "async": False}
+                    {"success": True, "bank_id": bank_id, "items_count": len(contents), "async": False, "usage": usage}
                )
+        except OperationValidationError as e:
+            raise HTTPException(status_code=e.status_code, detail=e.reason)
+        except (AuthenticationError, HTTPException):
+            raise
        except Exception as e:
            import traceback

@@ -1833,6 +2088,8 @@ def _register_routes(app: FastAPI):
            await app.state.memory.delete_bank(bank_id, fact_type=type, request_context=request_context)

            return DeleteResponse(success=True)
+        except (AuthenticationError, HTTPException):
+            raise
        except Exception as e:
            import traceback

@@ -27,12 +27,15 @@ logging.basicConfig(
 )
 logger = logging.getLogger(__name__)

-# Context variable to hold the current bank_id from the URL path
+# Default bank_id from environment variable
+DEFAULT_BANK_ID = os.environ.get("HINDSIGHT_MCP_BANK_ID", "default")
+
+# Context variable to hold the current bank_id
 _current_bank_id: ContextVar[str | None] = ContextVar("current_bank_id", default=None)


 def get_current_bank_id() -> str | None:
-    """Get the current bank_id from context (set from URL path)."""
+    """Get the current bank_id from context."""
    return _current_bank_id.get()


@@ -44,12 +47,18 @@ def create_mcp_server(memory: MemoryEngine) -> FastMCP:
        memory: MemoryEngine instance (required)

    Returns:
-        Configured FastMCP server instance
+        Configured FastMCP server instance with stateless_http enabled
    """
-    mcp = FastMCP("hindsight-mcp-server")
+    # Use stateless_http=True for Claude Code compatibility
+    mcp = FastMCP("hindsight-mcp-server", stateless_http=True)

    @mcp.tool()
-    async def retain(content: str, context: str = "general") -> str:
+    async def retain(
+        content: str,
+        context: str = "general",
+        async_processing: bool = True,
+        bank_id: str | None = None,
+    ) -> str:
        """
        Store important information to long-term memory.

@@ -65,21 +74,34 @@ def create_mcp_server(memory: MemoryEngine) -> FastMCP:
        Args:
            content: The fact/memory to store (be specific and include relevant details)
            context: Category for the memory (e.g., 'preferences', 'work', 'hobbies', 'family'). Default: 'general'
+            async_processing: If True, queue for background processing and return immediately. If False, wait for completion. Default: True
+            bank_id: Optional bank to store in (defaults to session bank). Use for cross-bank operations.
        """
        try:
-            bank_id = get_current_bank_id()
-            if bank_id is None:
+            target_bank = bank_id or get_current_bank_id()
+            if target_bank is None:
                return "Error: No bank_id configured"
-            await memory.retain_batch_async(
-                bank_id=bank_id, contents=[{"content": content, "context": context}], request_context=RequestContext()
-            )
-            return "Memory stored successfully"
+            contents = [{"content": content, "context": context}]
+            if async_processing:
+                # Queue for background processing and return immediately
+                result = await memory.submit_async_retain(
+                    bank_id=target_bank, contents=contents, request_context=RequestContext()
+                )
+                return f"Memory queued for background processing (operation_id: {result.get('operation_id', 'N/A')})"
+            else:
+                # Wait for completion
+                await memory.retain_batch_async(
+                    bank_id=target_bank,
+                    contents=contents,
+                    request_context=RequestContext(),
+                )
+                return f"Memory stored successfully in bank '{target_bank}'"
        except Exception as e:
            logger.error(f"Error storing memory: {e}", exc_info=True)
            return f"Error: {str(e)}"

    @mcp.tool()
-    async def recall(query: str, max_results: int = 10) -> str:
+    async def recall(query: str, max_tokens: int = 4096, bank_id: str | None = None) -> str:
        """
        Search memories to provide personalized, context-aware responses.

@@ -91,49 +113,111 @@ def create_mcp_server(memory: MemoryEngine) -> FastMCP:

        Args:
            query: Natural language search query (e.g., "user's food preferences", "what projects is user working on")
-            max_results: Maximum number of results to return (default: 10)
+            max_tokens: Maximum tokens in the response (default: 4096)
+            bank_id: Optional bank to search in (defaults to session bank). Use for cross-bank operations.
        """
        try:
-            bank_id = get_current_bank_id()
-            if bank_id is None:
+            target_bank = bank_id or get_current_bank_id()
+            if target_bank is None:
                return "Error: No bank_id configured"
            from hindsight_api.engine.memory_engine import Budget

-            search_result = await memory.recall_async(
-                bank_id=bank_id,
+            recall_result = await memory.recall_async(
+                bank_id=target_bank,
                query=query,
                fact_type=list(VALID_RECALL_FACT_TYPES),
-                budget=Budget.LOW,
+                budget=Budget.HIGH,
+                max_tokens=max_tokens,
                request_context=RequestContext(),
            )

-            results = [
-                {
-                    "id": fact.id,
-                    "text": fact.text,
-                    "type": fact.fact_type,
-                    "context": fact.context,
-                    "occurred_start": fact.occurred_start,
-                }
-                for fact in search_result.results[:max_results]
-            ]
-
-            return json.dumps({"results": results}, indent=2)
+            # Use model's JSON serialization
+            return recall_result.model_dump_json(indent=2)
        except Exception as e:
            logger.error(f"Error searching: {e}", exc_info=True)
-            return json.dumps({"error": str(e), "results": []})
+            return f'{{"error": "{e}", "results": []}}'
+
+    @mcp.tool()
+    async def reflect(query: str, context: str | None = None, budget: str = "low", bank_id: str | None = None) -> str:
+        """
+        Generate thoughtful analysis by synthesizing stored memories with the bank's personality.
+
+        WHEN TO USE THIS TOOL:
+        Use reflect when you need reasoned analysis, not just fact retrieval. This tool
+        thinks through the question using everything the bank knows and its personality traits.
+
+        EXAMPLES OF GOOD QUERIES:
+        - "What patterns have emerged in how I approach debugging?"
+        - "Based on my past decisions, what architectural style do I prefer?"
+        - "What might be the best approach for this problem given what you know about me?"
+        - "How should I prioritize these tasks based on my goals?"
+
+        HOW IT DIFFERS FROM RECALL:
+        - recall: Returns raw facts matching your search (fast lookup)
+        - reflect: Reasons across memories to form a synthesized answer (deeper analysis)
+
+        Use recall for "what did I say about X?" and reflect for "what should I do about X?"
+
+        Args:
+            query: The question or topic to reflect on
+            context: Optional context about why this reflection is needed
+            budget: Search budget - 'low', 'mid', or 'high' (default: 'low')
+            bank_id: Optional bank to reflect in (defaults to session bank). Use for cross-bank operations.
+        """
+        try:
+            target_bank = bank_id or get_current_bank_id()
+            if target_bank is None:
+                return "Error: No bank_id configured"
+            from hindsight_api.engine.memory_engine import Budget
+
+            # Map string budget to enum
+            budget_map = {"low": Budget.LOW, "mid": Budget.MID, "high": Budget.HIGH}
+            budget_enum = budget_map.get(budget.lower(), Budget.LOW)
+
+            reflect_result = await memory.reflect_async(
+                bank_id=target_bank,
+                query=query,
+                budget=budget_enum,
+                context=context,
+                request_context=RequestContext(),
+            )
+
+            return reflect_result.model_dump_json(indent=2)
+        except Exception as e:
+            logger.error(f"Error reflecting: {e}", exc_info=True)
+            return f'{{"error": "{e}", "text": ""}}'

    return mcp


 class MCPMiddleware:
-    """ASGI middleware that extracts bank_id from path and sets context."""
+    """ASGI middleware that extracts bank_id from header or path and sets context.
+
+    Bank ID can be provided via:
+    1. X-Bank-Id header (recommended for Claude Code)
+    2. URL path: /mcp/{bank_id}/
+    3. Environment variable HINDSIGHT_MCP_BANK_ID (fallback default)
+
+    For Claude Code, configure with:
+        claude mcp add --transport http hindsight http://localhost:8888/mcp \\
+            --header "X-Bank-Id: my-bank"
+    """

    def __init__(self, app, memory: MemoryEngine):
        self.app = app
        self.memory = memory
        self.mcp_server = create_mcp_server(memory)
-        self.mcp_app = self.mcp_server.http_app()
+        self.mcp_app = self.mcp_server.http_app(path="/")
+        # Expose the lifespan for the parent app to chain
+        self.lifespan = self.mcp_app.lifespan_handler if hasattr(self.mcp_app, "lifespan_handler") else None
+
+    def _get_header(self, scope: dict, name: str) -> str | None:
+        """Extract a header value from ASGI scope."""
+        name_lower = name.lower().encode()
+        for header_name, header_value in scope.get("headers", []):
+            if header_name.lower() == name_lower:
+                return header_value.decode()
+        return None

    async def __call__(self, scope, receive, send):
        if scope["type"] != "http":
@@ -150,32 +234,39 @@ class MCPMiddleware:
        # Also handle case where mount path wasn't stripped (e.g., /mcp/...)
        if path.startswith("/mcp/"):
            path = path[4:]  # Remove /mcp prefix
+        elif path == "/mcp":
+            path = "/"

-        # Extract bank_id from path: /{bank_id}/ or /{bank_id}
-        # http_app expects requests at /
-        if not path.startswith("/") or len(path) <= 1:
-            # No bank_id in path - return error
-            await self._send_error(send, 400, "bank_id required in path: /mcp/{bank_id}/")
-            return
+        # Try to get bank_id from header first (for Claude Code compatibility)
+        bank_id = self._get_header(scope, "X-Bank-Id")

-        # Extract bank_id from first path segment
-        parts = path[1:].split("/", 1)
-        if not parts[0]:
-            await self._send_error(send, 400, "bank_id required in path: /mcp/{bank_id}/")
-            return
+        # MCP endpoint paths that should not be treated as bank_ids
+        MCP_ENDPOINTS = {"sse", "messages"}

-        bank_id = parts[0]
-        new_path = "/" + parts[1] if len(parts) > 1 else "/"
+        # If no header, try to extract from path: /{bank_id}/...
+        new_path = path
+        if not bank_id and path.startswith("/") and len(path) > 1:
+            parts = path[1:].split("/", 1)
+            # Don't treat MCP endpoints as bank_ids
+            if parts[0] and parts[0] not in MCP_ENDPOINTS:
+                # First segment looks like a bank_id
+                bank_id = parts[0]
+                new_path = "/" + parts[1] if len(parts) > 1 else "/"
+
+        # Fall back to default bank_id
+        if not bank_id:
+            bank_id = DEFAULT_BANK_ID
+            logger.debug(f"Using default bank_id: {bank_id}")

        # Set bank_id context
        token = _current_bank_id.set(bank_id)
        try:
            new_scope = scope.copy()
            new_scope["path"] = new_path
+            # Clear root_path since we're passing directly to the app
+            new_scope["root_path"] = ""

-            # Wrap send to rewrite the SSE endpoint URL to include bank_id
-            # The SSE app sends "event: endpoint\ndata: /messages\n" but we need
-            # the client to POST to /{bank_id}/messages instead
+            # Wrap send to rewrite the SSE endpoint URL to include bank_id if using path-based routing
            async def send_wrapper(message):
                if message["type"] == "http.response.body":
                    body = message.get("body", b"")
@@ -211,9 +302,10 @@ def create_mcp_app(memory: MemoryEngine):
    """
    Create an ASGI app that handles MCP requests.

-    URL pattern: /mcp/{bank_id}/
-
-    The bank_id is extracted from the URL path and made available to tools.
+    Bank ID can be provided via:
+    1. X-Bank-Id header: claude mcp add --transport http hindsight http://localhost:8888/mcp --header "X-Bank-Id: my-bank"
+    2. URL path: /mcp/{bank_id}/
+    3. Environment variable HINDSIGHT_MCP_BANK_ID (fallback, default: "default")

    Args:
        memory: MemoryEngine instance
@@ -8,6 +8,11 @@ import logging
 import os
 from dataclasses import dataclass

+from dotenv import find_dotenv, load_dotenv
+
+# Load .env file, searching current and parent directories (overrides existing env vars)
+load_dotenv(find_dotenv(usecwd=True), override=True)
+
 logger = logging.getLogger(__name__)

 # Environment variable names
@@ -16,10 +21,15 @@ ENV_LLM_PROVIDER = "HINDSIGHT_API_LLM_PROVIDER"
 ENV_LLM_API_KEY = "HINDSIGHT_API_LLM_API_KEY"
 ENV_LLM_MODEL = "HINDSIGHT_API_LLM_MODEL"
 ENV_LLM_BASE_URL = "HINDSIGHT_API_LLM_BASE_URL"
+ENV_LLM_MAX_CONCURRENT = "HINDSIGHT_API_LLM_MAX_CONCURRENT"
+ENV_LLM_TIMEOUT = "HINDSIGHT_API_LLM_TIMEOUT"
+ENV_LLM_GROQ_SERVICE_TIER = "HINDSIGHT_API_LLM_GROQ_SERVICE_TIER"

 ENV_EMBEDDINGS_PROVIDER = "HINDSIGHT_API_EMBEDDINGS_PROVIDER"
 ENV_EMBEDDINGS_LOCAL_MODEL = "HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL"
 ENV_EMBEDDINGS_TEI_URL = "HINDSIGHT_API_EMBEDDINGS_TEI_URL"
+ENV_EMBEDDINGS_OPENAI_API_KEY = "HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY"
+ENV_EMBEDDINGS_OPENAI_MODEL = "HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL"

 ENV_RERANKER_PROVIDER = "HINDSIGHT_API_RERANKER_PROVIDER"
 ENV_RERANKER_LOCAL_MODEL = "HINDSIGHT_API_RERANKER_LOCAL_MODEL"
@@ -33,13 +43,31 @@ ENV_GRAPH_RETRIEVER = "HINDSIGHT_API_GRAPH_RETRIEVER"
 ENV_MCP_LOCAL_BANK_ID = "HINDSIGHT_API_MCP_LOCAL_BANK_ID"
 ENV_MCP_INSTRUCTIONS = "HINDSIGHT_API_MCP_INSTRUCTIONS"

+# Observation thresholds
+ENV_OBSERVATION_MIN_FACTS = "HINDSIGHT_API_OBSERVATION_MIN_FACTS"
+ENV_OBSERVATION_TOP_ENTITIES = "HINDSIGHT_API_OBSERVATION_TOP_ENTITIES"
+
+# Retain settings
+ENV_RETAIN_MAX_COMPLETION_TOKENS = "HINDSIGHT_API_RETAIN_MAX_COMPLETION_TOKENS"
+
+# Optimization flags
+ENV_SKIP_LLM_VERIFICATION = "HINDSIGHT_API_SKIP_LLM_VERIFICATION"
+ENV_LAZY_RERANKER = "HINDSIGHT_API_LAZY_RERANKER"
+
+# Database migrations
+ENV_RUN_MIGRATIONS_ON_STARTUP = "HINDSIGHT_API_RUN_MIGRATIONS_ON_STARTUP"
+
 # Default values
 DEFAULT_DATABASE_URL = "pg0"
 DEFAULT_LLM_PROVIDER = "openai"
 DEFAULT_LLM_MODEL = "gpt-5-mini"
+DEFAULT_LLM_MAX_CONCURRENT = 32
+DEFAULT_LLM_TIMEOUT = 120.0  # seconds

 DEFAULT_EMBEDDINGS_PROVIDER = "local"
 DEFAULT_EMBEDDINGS_LOCAL_MODEL = "BAAI/bge-small-en-v1.5"
+DEFAULT_EMBEDDINGS_OPENAI_MODEL = "text-embedding-3-small"
+DEFAULT_EMBEDDING_DIMENSION = 384

 DEFAULT_RERANKER_PROVIDER = "local"
 DEFAULT_RERANKER_LOCAL_MODEL = "cross-encoder/ms-marco-MiniLM-L-6-v2"
@@ -51,6 +79,16 @@ DEFAULT_MCP_ENABLED = True
 DEFAULT_GRAPH_RETRIEVER = "bfs"  # Options: "bfs", "mpfp"
 DEFAULT_MCP_LOCAL_BANK_ID = "mcp"

+# Observation thresholds
+DEFAULT_OBSERVATION_MIN_FACTS = 5  # Min facts required to generate entity observations
+DEFAULT_OBSERVATION_TOP_ENTITIES = 5  # Max entities to process per retain batch
+
+# Retain settings
+DEFAULT_RETAIN_MAX_COMPLETION_TOKENS = 64000  # Max tokens for fact extraction LLM call
+
+# Database migrations
+DEFAULT_RUN_MIGRATIONS_ON_STARTUP = True
+
 # Default MCP tool descriptions (can be customized via env vars)
 DEFAULT_MCP_RETAIN_DESCRIPTION = """Store important information to long-term memory.

@@ -71,8 +109,8 @@ Use this tool PROACTIVELY to:
 - Remember user's goals and context
 - Personalize responses based on past interactions"""

-# Required embedding dimension for database schema
-EMBEDDING_DIMENSION = 384
+# Default embedding dimension (used by initial migration, adjusted at runtime)
+EMBEDDING_DIMENSION = DEFAULT_EMBEDDING_DIMENSION


@dataclass
@@ -87,6 +125,8 @@ class HindsightConfig:
    llm_api_key: str | None
    llm_model: str
    llm_base_url: str | None
+    llm_max_concurrent: int
+    llm_timeout: float

    # Embeddings
    embeddings_provider: str
@@ -107,6 +147,20 @@ class HindsightConfig:
    # Recall
    graph_retriever: str

+    # Observation thresholds
+    observation_min_facts: int
+    observation_top_entities: int
+
+    # Retain settings
+    retain_max_completion_tokens: int
+
+    # Optimization flags
+    skip_llm_verification: bool
+    lazy_reranker: bool
+
+    # Database migrations
+    run_migrations_on_startup: bool
+
    @classmethod
    def from_env(cls) -> "HindsightConfig":
        """Create configuration from environment variables."""
@@ -118,6 +172,8 @@ class HindsightConfig:
            llm_api_key=os.getenv(ENV_LLM_API_KEY),
            llm_model=os.getenv(ENV_LLM_MODEL, DEFAULT_LLM_MODEL),
            llm_base_url=os.getenv(ENV_LLM_BASE_URL) or None,
+            llm_max_concurrent=int(os.getenv(ENV_LLM_MAX_CONCURRENT, str(DEFAULT_LLM_MAX_CONCURRENT))),
+            llm_timeout=float(os.getenv(ENV_LLM_TIMEOUT, str(DEFAULT_LLM_TIMEOUT))),
            # Embeddings
            embeddings_provider=os.getenv(ENV_EMBEDDINGS_PROVIDER, DEFAULT_EMBEDDINGS_PROVIDER),
            embeddings_local_model=os.getenv(ENV_EMBEDDINGS_LOCAL_MODEL, DEFAULT_EMBEDDINGS_LOCAL_MODEL),
@@ -133,6 +189,20 @@ class HindsightConfig:
            mcp_enabled=os.getenv(ENV_MCP_ENABLED, str(DEFAULT_MCP_ENABLED)).lower() == "true",
            # Recall
            graph_retriever=os.getenv(ENV_GRAPH_RETRIEVER, DEFAULT_GRAPH_RETRIEVER),
+            # Optimization flags
+            skip_llm_verification=os.getenv(ENV_SKIP_LLM_VERIFICATION, "false").lower() == "true",
+            lazy_reranker=os.getenv(ENV_LAZY_RERANKER, "false").lower() == "true",
+            # Observation thresholds
+            observation_min_facts=int(os.getenv(ENV_OBSERVATION_MIN_FACTS, str(DEFAULT_OBSERVATION_MIN_FACTS))),
+            observation_top_entities=int(
+                os.getenv(ENV_OBSERVATION_TOP_ENTITIES, str(DEFAULT_OBSERVATION_TOP_ENTITIES))
+            ),
+            # Retain settings
+            retain_max_completion_tokens=int(
+                os.getenv(ENV_RETAIN_MAX_COMPLETION_TOKENS, str(DEFAULT_RETAIN_MAX_COMPLETION_TOKENS))
+            ),
+            # Database migrations
+            run_migrations_on_startup=os.getenv(ENV_RUN_MIGRATIONS_ON_STARTUP, "true").lower() == "true",
        )

    def get_llm_base_url(self) -> str:
@@ -145,6 +215,8 @@ class HindsightConfig:
            return "https://api.groq.com/openai/v1"
        elif provider == "ollama":
            return "http://localhost:11434/v1"
+        elif provider == "lmstudio":
+            return "http://localhost:1234/v1"
        else:
            return ""

@@ -177,6 +249,19 @@ class HindsightConfig:
        logger.info(f"Graph retriever: {self.graph_retriever}")


+# Cached config instance
+_config_cache: HindsightConfig | None = None
+
+
 def get_config() -> HindsightConfig:
-    """Get the current configuration from environment variables."""
-    return HindsightConfig.from_env()
+    """Get the cached configuration, loading from environment on first call."""
+    global _config_cache
+    if _config_cache is None:
+        _config_cache = HindsightConfig.from_env()
+    return _config_cache
+
+
+def clear_config_cache() -> None:
+    """Clear the config cache. Useful for testing or reloading config."""
+    global _config_cache
+    _config_cache = None
@@ -0,0 +1,204 @@
+"""
+Daemon mode support for Hindsight API.
+
+Provides idle timeout and lockfile management for running as a background daemon.
+"""
+
+import asyncio
+import fcntl
+import logging
+import os
+import sys
+import time
+from pathlib import Path
+
+logger = logging.getLogger(__name__)
+
+# Default daemon configuration
+DEFAULT_DAEMON_PORT = 8889
+DEFAULT_IDLE_TIMEOUT = 0  # 0 = no auto-exit (hindsight-embed passes its own timeout)
+LOCKFILE_PATH = Path.home() / ".hindsight" / "daemon.lock"
+DAEMON_LOG_PATH = Path.home() / ".hindsight" / "daemon.log"
+
+
+class IdleTimeoutMiddleware:
+    """ASGI middleware that tracks activity and exits after idle timeout."""
+
+    def __init__(self, app, idle_timeout: int = DEFAULT_IDLE_TIMEOUT):
+        self.app = app
+        self.idle_timeout = idle_timeout
+        self.last_activity = time.time()
+        self._checker_task = None
+
+    async def __call__(self, scope, receive, send):
+        # Update activity timestamp on each request
+        self.last_activity = time.time()
+        await self.app(scope, receive, send)
+
+    def start_idle_checker(self):
+        """Start the background task that checks for idle timeout."""
+        self._checker_task = asyncio.create_task(self._check_idle())
+
+    async def _check_idle(self):
+        """Background task that exits the process after idle timeout."""
+        # If idle_timeout is 0, don't auto-exit
+        if self.idle_timeout <= 0:
+            return
+
+        while True:
+            await asyncio.sleep(30)  # Check every 30 seconds
+            idle_time = time.time() - self.last_activity
+            if idle_time > self.idle_timeout:
+                logger.info(f"Idle timeout reached ({self.idle_timeout}s), shutting down daemon")
+                # Give a moment for any in-flight requests
+                await asyncio.sleep(1)
+                os._exit(0)
+
+
+class DaemonLock:
+    """
+    File-based lock to prevent multiple daemon instances.
+
+    Uses fcntl.flock for atomic locking on Unix systems.
+    """
+
+    def __init__(self, lockfile: Path = LOCKFILE_PATH):
+        self.lockfile = lockfile
+        self._fd = None
+
+    def acquire(self) -> bool:
+        """
+        Try to acquire the daemon lock.
+
+        Returns True if lock acquired, False if another daemon is running.
+        """
+        self.lockfile.parent.mkdir(parents=True, exist_ok=True)
+
+        try:
+            self._fd = open(self.lockfile, "w")
+            fcntl.flock(self._fd.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
+            # Write PID for debugging
+            self._fd.write(str(os.getpid()))
+            self._fd.flush()
+            return True
+        except (IOError, OSError):
+            # Lock is held by another process
+            if self._fd:
+                self._fd.close()
+                self._fd = None
+            return False
+
+    def release(self):
+        """Release the daemon lock."""
+        if self._fd:
+            try:
+                fcntl.flock(self._fd.fileno(), fcntl.LOCK_UN)
+                self._fd.close()
+            except Exception:
+                pass
+            finally:
+                self._fd = None
+                # Remove lockfile
+                try:
+                    self.lockfile.unlink()
+                except Exception:
+                    pass
+
+    def is_locked(self) -> bool:
+        """Check if the lock is held by another process."""
+        if not self.lockfile.exists():
+            return False
+
+        try:
+            fd = open(self.lockfile, "r")
+            fcntl.flock(fd.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
+            # We got the lock, so no one else has it
+            fcntl.flock(fd.fileno(), fcntl.LOCK_UN)
+            fd.close()
+            return False
+        except (IOError, OSError):
+            return True
+
+    def get_pid(self) -> int | None:
+        """Get the PID of the daemon holding the lock."""
+        if not self.lockfile.exists():
+            return None
+        try:
+            with open(self.lockfile, "r") as f:
+                return int(f.read().strip())
+        except (ValueError, IOError):
+            return None
+
+
+def daemonize():
+    """
+    Fork the current process into a background daemon.
+
+    Uses double-fork technique to properly detach from terminal.
+    """
+    # First fork
+    pid = os.fork()
+    if pid > 0:
+        # Parent exits
+        sys.exit(0)
+
+    # Create new session
+    os.setsid()
+
+    # Second fork to prevent zombie processes
+    pid = os.fork()
+    if pid > 0:
+        sys.exit(0)
+
+    # Redirect standard file descriptors to log file
+    DAEMON_LOG_PATH.parent.mkdir(parents=True, exist_ok=True)
+
+    sys.stdout.flush()
+    sys.stderr.flush()
+
+    # Redirect stdin to /dev/null
+    with open("/dev/null", "r") as devnull:
+        os.dup2(devnull.fileno(), sys.stdin.fileno())
+
+    # Redirect stdout/stderr to log file
+    log_fd = open(DAEMON_LOG_PATH, "a")
+    os.dup2(log_fd.fileno(), sys.stdout.fileno())
+    os.dup2(log_fd.fileno(), sys.stderr.fileno())
+
+
+def check_daemon_running(port: int = DEFAULT_DAEMON_PORT) -> bool:
+    """Check if a daemon is running and responsive on the given port."""
+    import socket
+
+    try:
+        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+        sock.settimeout(1)
+        result = sock.connect_ex(("127.0.0.1", port))
+        sock.close()
+        return result == 0
+    except Exception:
+        return False
+
+
+def stop_daemon(port: int = DEFAULT_DAEMON_PORT) -> bool:
+    """Stop a running daemon by sending SIGTERM to the process."""
+    lock = DaemonLock()
+    pid = lock.get_pid()
+
+    if pid is None:
+        return False
+
+    try:
+        import signal
+
+        os.kill(pid, signal.SIGTERM)
+        # Wait for process to exit
+        for _ in range(50):  # Wait up to 5 seconds
+            time.sleep(0.1)
+            try:
+                os.kill(pid, 0)  # Check if process exists
+            except OSError:
+                return True  # Process exited
+        return False
+    except OSError:
+        return False
@@ -3,8 +3,8 @@ Embeddings abstraction for the memory system.

 Provides an interface for generating embeddings with different backends.

-IMPORTANT: All embeddings must produce 384-dimensional vectors to match
-the database schema (pgvector column defined as vector(384)).
+The embedding dimension is auto-detected from the model at initialization.
+The database schema is automatically adjusted to match the model's dimension.

 Configuration via environment variables - see hindsight_api.config for all env var names.
 """
@@ -17,11 +17,14 @@ import httpx

 from ..config import (
    DEFAULT_EMBEDDINGS_LOCAL_MODEL,
+    DEFAULT_EMBEDDINGS_OPENAI_MODEL,
    DEFAULT_EMBEDDINGS_PROVIDER,
-    EMBEDDING_DIMENSION,
    ENV_EMBEDDINGS_LOCAL_MODEL,
+    ENV_EMBEDDINGS_OPENAI_API_KEY,
+    ENV_EMBEDDINGS_OPENAI_MODEL,
    ENV_EMBEDDINGS_PROVIDER,
    ENV_EMBEDDINGS_TEI_URL,
+    ENV_LLM_API_KEY,
 )

 logger = logging.getLogger(__name__)
@@ -31,8 +34,8 @@ class Embeddings(ABC):
    """
    Abstract base class for embedding generation.

-    All implementations MUST generate 384-dimensional embeddings to match
-    the database schema.
+    The embedding dimension is determined by the model and detected at initialization.
+    The database schema is automatically adjusted to match the model's dimension.
    """

    @property
@@ -41,6 +44,12 @@ class Embeddings(ABC):
        """Return a human-readable name for this provider (e.g., 'local', 'tei')."""
        pass

+    @property
+    @abstractmethod
+    def dimension(self) -> int:
+        """Return the embedding dimension produced by this model."""
+        pass
+
    @abstractmethod
    async def initialize(self) -> None:
        """
@@ -54,13 +63,13 @@ class Embeddings(ABC):
    @abstractmethod
    def encode(self, texts: list[str]) -> list[list[float]]:
        """
-        Generate 384-dimensional embeddings for a list of texts.
+        Generate embeddings for a list of texts.

        Args:
            texts: List of text strings to encode

        Returns:
-            List of 384-dimensional embedding vectors (each is a list of floats)
+            List of embedding vectors (each is a list of floats)
        """
        pass

@@ -70,9 +79,7 @@ class LocalSTEmbeddings(Embeddings):
    Local embeddings implementation using SentenceTransformers.

    Call initialize() during startup to load the model and avoid cold starts.
-
-    Default model is BAAI/bge-small-en-v1.5 which produces 384-dimensional
-    embeddings matching the database schema.
+    The embedding dimension is auto-detected from the model.
    """

    def __init__(self, model_name: str | None = None):
@@ -81,16 +88,22 @@ class LocalSTEmbeddings(Embeddings):

        Args:
            model_name: Name of the SentenceTransformer model to use.
-                       Must produce 384-dimensional embeddings.
                       Default: BAAI/bge-small-en-v1.5
        """
        self.model_name = model_name or DEFAULT_EMBEDDINGS_LOCAL_MODEL
        self._model = None
+        self._dimension: int | None = None

    @property
    def provider_name(self) -> str:
        return "local"

+    @property
+    def dimension(self) -> int:
+        if self._dimension is None:
+            raise RuntimeError("Embeddings not initialized. Call initialize() first.")
+        return self._dimension
+
    async def initialize(self) -> None:
        """Load the embedding model."""
        if self._model is not None:
@@ -112,26 +125,18 @@ class LocalSTEmbeddings(Embeddings):
            model_kwargs={"low_cpu_mem_usage": False, "device_map": None},
        )

-        # Validate dimension matches database schema
-        model_dim = self._model.get_sentence_embedding_dimension()
-        if model_dim != EMBEDDING_DIMENSION:
-            raise ValueError(
-                f"Model {self.model_name} produces {model_dim}-dimensional embeddings, "
-                f"but database schema requires {EMBEDDING_DIMENSION} dimensions. "
-                f"Use a model that produces {EMBEDDING_DIMENSION}-dimensional embeddings."
-            )
-
-        logger.info(f"Embeddings: local provider initialized (dim: {model_dim})")
+        self._dimension = self._model.get_sentence_embedding_dimension()
+        logger.info(f"Embeddings: local provider initialized (dim: {self._dimension})")

    def encode(self, texts: list[str]) -> list[list[float]]:
        """
-        Generate 384-dimensional embeddings for a list of texts.
+        Generate embeddings for a list of texts.

        Args:
            texts: List of text strings to encode

        Returns:
-            List of 384-dimensional embedding vectors
+            List of embedding vectors
        """
        if self._model is None:
            raise RuntimeError("Embeddings not initialized. Call initialize() first.")
@@ -146,7 +151,7 @@ class RemoteTEIEmbeddings(Embeddings):
    TEI provides a high-performance inference server for embedding models.
    See: https://github.com/huggingface/text-embeddings-inference

-    The server should be running a model that produces 384-dimensional embeddings.
+    The embedding dimension is auto-detected from the server at initialization.
    """

    def __init__(
@@ -174,11 +179,18 @@ class RemoteTEIEmbeddings(Embeddings):
        self.retry_delay = retry_delay
        self._client: httpx.Client | None = None
        self._model_id: str | None = None
+        self._dimension: int | None = None

    @property
    def provider_name(self) -> str:
        return "tei"

+    @property
+    def dimension(self) -> int:
+        if self._dimension is None:
+            raise RuntimeError("Embeddings not initialized. Call initialize() first.")
+        return self._dimension
+
    def _request_with_retry(self, method: str, url: str, **kwargs) -> httpx.Response:
        """Make an HTTP request with automatic retries on transient errors."""
        import time
@@ -229,7 +241,24 @@ class RemoteTEIEmbeddings(Embeddings):
            response = self._request_with_retry("GET", f"{self.base_url}/info")
            info = response.json()
            self._model_id = info.get("model_id", "unknown")
-            logger.info(f"Embeddings: TEI provider initialized (model: {self._model_id})")
+
+            # Get dimension from server info or by doing a test embedding
+            if "max_input_length" in info and "model_dtype" in info:
+                # Try to get dimension from info endpoint (some TEI versions expose it)
+                # If not available, do a test embedding
+                pass
+
+            # Do a test embedding to detect dimension
+            test_response = self._request_with_retry(
+                "POST",
+                f"{self.base_url}/embed",
+                json={"inputs": ["test"]},
+            )
+            test_embeddings = test_response.json()
+            if test_embeddings and len(test_embeddings) > 0:
+                self._dimension = len(test_embeddings[0])
+
+            logger.info(f"Embeddings: TEI provider initialized (model: {self._model_id}, dim: {self._dimension})")
        except httpx.HTTPError as e:
            raise RuntimeError(f"Failed to connect to TEI server at {self.base_url}: {e}")

@@ -269,6 +298,117 @@ class RemoteTEIEmbeddings(Embeddings):
        return all_embeddings


+class OpenAIEmbeddings(Embeddings):
+    """
+    OpenAI embeddings implementation using the OpenAI API.
+
+    Supports text-embedding-3-small (1536 dims), text-embedding-3-large (3072 dims),
+    and text-embedding-ada-002 (1536 dims, legacy).
+
+    The embedding dimension is auto-detected from the model at initialization.
+    """
+
+    # Known dimensions for OpenAI embedding models
+    MODEL_DIMENSIONS = {
+        "text-embedding-3-small": 1536,
+        "text-embedding-3-large": 3072,
+        "text-embedding-ada-002": 1536,
+    }
+
+    def __init__(
+        self,
+        api_key: str,
+        model: str = DEFAULT_EMBEDDINGS_OPENAI_MODEL,
+        batch_size: int = 100,
+        max_retries: int = 3,
+    ):
+        """
+        Initialize OpenAI embeddings client.
+
+        Args:
+            api_key: OpenAI API key
+            model: OpenAI embedding model name (default: text-embedding-3-small)
+            batch_size: Maximum batch size for embedding requests (default: 100)
+            max_retries: Maximum number of retries for failed requests (default: 3)
+        """
+        self.api_key = api_key
+        self.model = model
+        self.batch_size = batch_size
+        self.max_retries = max_retries
+        self._client = None
+        self._dimension: int | None = None
+
+    @property
+    def provider_name(self) -> str:
+        return "openai"
+
+    @property
+    def dimension(self) -> int:
+        if self._dimension is None:
+            raise RuntimeError("Embeddings not initialized. Call initialize() first.")
+        return self._dimension
+
+    async def initialize(self) -> None:
+        """Initialize the OpenAI client and detect dimension."""
+        if self._client is not None:
+            return
+
+        try:
+            from openai import OpenAI
+        except ImportError:
+            raise ImportError("openai is required for OpenAIEmbeddings. Install it with: pip install openai")
+
+        logger.info(f"Embeddings: initializing OpenAI provider with model {self.model}")
+        self._client = OpenAI(api_key=self.api_key, max_retries=self.max_retries)
+
+        # Try to get dimension from known models, otherwise do a test embedding
+        if self.model in self.MODEL_DIMENSIONS:
+            self._dimension = self.MODEL_DIMENSIONS[self.model]
+        else:
+            # Do a test embedding to detect dimension
+            response = self._client.embeddings.create(
+                model=self.model,
+                input=["test"],
+            )
+            if response.data:
+                self._dimension = len(response.data[0].embedding)
+
+        logger.info(f"Embeddings: OpenAI provider initialized (model: {self.model}, dim: {self._dimension})")
+
+    def encode(self, texts: list[str]) -> list[list[float]]:
+        """
+        Generate embeddings using the OpenAI API.
+
+        Args:
+            texts: List of text strings to encode
+
+        Returns:
+            List of embedding vectors
+        """
+        if self._client is None:
+            raise RuntimeError("Embeddings not initialized. Call initialize() first.")
+
+        if not texts:
+            return []
+
+        all_embeddings = []
+
+        # Process in batches
+        for i in range(0, len(texts), self.batch_size):
+            batch = texts[i : i + self.batch_size]
+
+            response = self._client.embeddings.create(
+                model=self.model,
+                input=batch,
+            )
+
+            # Sort by index to ensure correct order
+            batch_embeddings = sorted(response.data, key=lambda x: x.index)
+            all_embeddings.extend([e.embedding for e in batch_embeddings])
+
+        return all_embeddings
+
+
 def create_embeddings_from_env() -> Embeddings:
    """
    Create an Embeddings instance based on environment variables.
@@ -289,5 +429,15 @@ def create_embeddings_from_env() -> Embeddings:
        model = os.environ.get(ENV_EMBEDDINGS_LOCAL_MODEL)
        model_name = model or DEFAULT_EMBEDDINGS_LOCAL_MODEL
        return LocalSTEmbeddings(model_name=model_name)
+    elif provider == "openai":
+        # Use dedicated embeddings API key, or fall back to LLM API key
+        api_key = os.environ.get(ENV_EMBEDDINGS_OPENAI_API_KEY) or os.environ.get(ENV_LLM_API_KEY)
+        if not api_key:
+            raise ValueError(
+                f"{ENV_EMBEDDINGS_OPENAI_API_KEY} or {ENV_LLM_API_KEY} is required "
+                f"when {ENV_EMBEDDINGS_PROVIDER} is 'openai'"
+            )
+        model = os.environ.get(ENV_EMBEDDINGS_OPENAI_MODEL, DEFAULT_EMBEDDINGS_OPENAI_MODEL)
+        return OpenAIEmbeddings(api_key=api_key, model=model)
    else:
-        raise ValueError(f"Unknown embeddings provider: {provider}. Supported: 'local', 'tei'")
+        raise ValueError(f"Unknown embeddings provider: {provider}. Supported: 'local', 'tei', 'openai'")
@@ -110,6 +110,8 @@ class MemoryEngineInterface(ABC):
        *,
        budget: "Budget | None" = None,
        context: str | None = None,
+        max_tokens: int = 4096,
+        response_schema: dict | None = None,
        request_context: "RequestContext",
    ) -> "ReflectResult":
        """
@@ -120,6 +122,8 @@ class MemoryEngineInterface(ABC):
            query: The question to reflect on.
            budget: Search budget for retrieving context.
            context: Additional context for the reflection.
+            max_tokens: Maximum tokens for the response.
+            response_schema: Optional JSON Schema for structured output.
            request_context: Request context for authentication.

        Returns:
@@ -285,6 +289,7 @@ class MemoryEngineInterface(ABC):
        bank_id: str,
        *,
        fact_type: str | None = None,
+        limit: int = 1000,
        request_context: "RequestContext",
    ) -> dict[str, Any]:
        """
@@ -293,10 +298,11 @@ class MemoryEngineInterface(ABC):
        Args:
            bank_id: The memory bank ID.
            fact_type: Filter by fact type.
+            limit: Maximum number of items to return (default: 1000).
            request_context: Request context for authentication.

        Returns:
-            Dict with nodes, edges, table_rows, total_units.
+            Dict with nodes, edges, table_rows, total_units, limit.
        """
        ...

@@ -6,6 +6,7 @@ import asyncio
 import json
 import logging
 import os
+import re
 import time
 from typing import Any

@@ -15,6 +16,16 @@ from google.genai import errors as genai_errors
 from google.genai import types as genai_types
 from openai import APIConnectionError, APIStatusError, AsyncOpenAI, LengthFinishReasonError

+from ..config import (
+    DEFAULT_LLM_MAX_CONCURRENT,
+    DEFAULT_LLM_TIMEOUT,
+    ENV_LLM_GROQ_SERVICE_TIER,
+    ENV_LLM_MAX_CONCURRENT,
+    ENV_LLM_TIMEOUT,
+)
+from ..metrics import get_metrics_collector
+from .response_models import TokenUsage
+
 # Seed applied to every Groq request for deterministic behavior.
 DEFAULT_LLM_SEED = 4242

@@ -24,7 +35,9 @@ logger = logging.getLogger(__name__)
 logging.getLogger("httpx").setLevel(logging.WARNING)

 # Global semaphore to limit concurrent LLM requests across all instances
-_global_llm_semaphore = asyncio.Semaphore(32)
+# Set HINDSIGHT_API_LLM_MAX_CONCURRENT=1 for local LLMs (LM Studio, Ollama)
+_llm_max_concurrent = int(os.getenv(ENV_LLM_MAX_CONCURRENT, str(DEFAULT_LLM_MAX_CONCURRENT)))
+_global_llm_semaphore = asyncio.Semaphore(_llm_max_concurrent)


 class OutputTooLongError(Exception):
@@ -53,25 +66,29 @@ class LLMProvider:
        base_url: str,
        model: str,
        reasoning_effort: str = "low",
+        groq_service_tier: str | None = None,
    ):
        """
        Initialize LLM provider.

        Args:
-            provider: Provider name ("openai", "groq", "ollama", "gemini").
+            provider: Provider name ("openai", "groq", "ollama", "gemini", "anthropic", "lmstudio").
            api_key: API key.
            base_url: Base URL for the API.
            model: Model name.
            reasoning_effort: Reasoning effort level for supported providers.
+            groq_service_tier: Groq service tier ("on_demand", "flex", "auto"). Default: None (uses Groq's default).
        """
        self.provider = provider.lower()
        self.api_key = api_key
        self.base_url = base_url
        self.model = model
        self.reasoning_effort = reasoning_effort
+        # Default to 'auto' for best performance, users can override to 'on_demand' for free tier
+        self.groq_service_tier = groq_service_tier or os.getenv(ENV_LLM_GROQ_SERVICE_TIER, "auto")

        # Validate provider
-        valid_providers = ["openai", "groq", "ollama", "gemini"]
+        valid_providers = ["openai", "groq", "ollama", "gemini", "anthropic", "lmstudio"]
        if self.provider not in valid_providers:
            raise ValueError(f"Invalid LLM provider: {self.provider}. Must be one of: {', '.join(valid_providers)}")

@@ -81,25 +98,48 @@ class LLMProvider:
                self.base_url = "https://api.groq.com/openai/v1"
            elif self.provider == "ollama":
                self.base_url = "http://localhost:11434/v1"
+            elif self.provider == "lmstudio":
+                self.base_url = "http://localhost:1234/v1"

-        # Validate API key (not needed for ollama)
-        if self.provider != "ollama" and not self.api_key:
+        # Validate API key (not needed for ollama or lmstudio)
+        if self.provider not in ("ollama", "lmstudio") and not self.api_key:
            raise ValueError(f"API key not found for {self.provider}")

+        # Get timeout config (set HINDSIGHT_API_LLM_TIMEOUT for local LLMs that need longer timeouts)
+        self.timeout = float(os.getenv(ENV_LLM_TIMEOUT, str(DEFAULT_LLM_TIMEOUT)))
+
        # Create client based on provider
+        self._client = None
+        self._gemini_client = None
+        self._anthropic_client = None
+
        if self.provider == "gemini":
            self._gemini_client = genai.Client(api_key=self.api_key)
-            self._client = None
-        elif self.provider == "ollama":
-            self._client = AsyncOpenAI(api_key="ollama", base_url=self.base_url, max_retries=0)
-            self._gemini_client = None
+        elif self.provider == "anthropic":
+            from anthropic import AsyncAnthropic
+
+            # Only pass base_url if it's set (Anthropic uses default URL otherwise)
+            anthropic_kwargs = {"api_key": self.api_key}
+            if self.base_url:
+                anthropic_kwargs["base_url"] = self.base_url
+            if self.timeout:
+                anthropic_kwargs["timeout"] = self.timeout
+            self._anthropic_client = AsyncAnthropic(**anthropic_kwargs)
+        elif self.provider in ("ollama", "lmstudio"):
+            # Use dummy key if not provided for local
+            api_key = self.api_key or "local"
+            client_kwargs = {"api_key": api_key, "base_url": self.base_url, "max_retries": 0}
+            if self.timeout:
+                client_kwargs["timeout"] = self.timeout
+            self._client = AsyncOpenAI(**client_kwargs)
        else:
            # Only pass base_url if it's set (OpenAI uses default URL otherwise)
            client_kwargs = {"api_key": self.api_key, "max_retries": 0}
            if self.base_url:
                client_kwargs["base_url"] = self.base_url
-            self._client = AsyncOpenAI(**client_kwargs)  # type: ignore[invalid-argument-type] - dict kwargs
-            self._gemini_client = None
+            if self.timeout:
+                client_kwargs["timeout"] = self.timeout
+            self._client = AsyncOpenAI(**client_kwargs)

    async def verify_connection(self) -> None:
        """
@@ -135,6 +175,8 @@ class LLMProvider:
        initial_backoff: float = 1.0,
        max_backoff: float = 60.0,
        skip_validation: bool = False,
+        strict_schema: bool = False,
+        return_usage: bool = False,
    ) -> Any:
        """
        Make an LLM API call with retry logic.
@@ -149,9 +191,12 @@ class LLMProvider:
            initial_backoff: Initial backoff time in seconds.
            max_backoff: Maximum backoff time in seconds.
            skip_validation: Return raw JSON without Pydantic validation.
+            strict_schema: Use strict JSON schema enforcement (OpenAI only). Guarantees all required fields.
+            return_usage: If True, return tuple (result, TokenUsage) instead of just result.

        Returns:
-            Parsed response if response_format is provided, otherwise text content.
+            If return_usage=False: Parsed response if response_format is provided, otherwise text content.
+            If return_usage=True: Tuple of (result, TokenUsage) with token counts from the LLM call.

        Raises:
            OutputTooLongError: If output exceeds token limits.
@@ -163,7 +208,28 @@ class LLMProvider:
            # Handle Gemini provider separately
            if self.provider == "gemini":
                return await self._call_gemini(
-                    messages, response_format, max_retries, initial_backoff, max_backoff, skip_validation, start_time
+                    messages,
+                    response_format,
+                    max_retries,
+                    initial_backoff,
+                    max_backoff,
+                    skip_validation,
+                    start_time,
+                    return_usage,
+                )
+
+            # Handle Anthropic provider separately
+            if self.provider == "anthropic":
+                return await self._call_anthropic(
+                    messages,
+                    response_format,
+                    max_completion_tokens,
+                    max_retries,
+                    initial_backoff,
+                    max_backoff,
+                    skip_validation,
+                    start_time,
+                    return_usage,
                )

            # Handle Ollama with native API for structured output (better schema enforcement)
@@ -178,6 +244,7 @@ class LLMProvider:
                    max_backoff,
                    skip_validation,
                    start_time,
+                    return_usage,
                )

            call_params = {
@@ -215,58 +282,108 @@ class LLMProvider:
            # Provider-specific parameters
            if self.provider == "groq":
                call_params["seed"] = DEFAULT_LLM_SEED
-                extra_body = {"service_tier": "auto"}
-                # Only add reasoning parameters for reasoning models
+                extra_body: dict[str, Any] = {}
+                # Add service_tier if configured (requires paid plan for flex/auto)
+                if self.groq_service_tier:
+                    extra_body["service_tier"] = self.groq_service_tier
+                # Add reasoning parameters for reasoning models
                if is_reasoning_model:
                    extra_body["include_reasoning"] = False
-                call_params["extra_body"] = extra_body
+                if extra_body:
+                    call_params["extra_body"] = extra_body

            last_exception = None

            for attempt in range(max_retries + 1):
                try:
                    if response_format is not None:
-                        # Add schema to system message for JSON mode
+                        schema = None
                        if hasattr(response_format, "model_json_schema"):
                            schema = response_format.model_json_schema()
-                            schema_msg = f"\n\nYou must respond with valid JSON matching this schema:\n{json.dumps(schema, indent=2)}"

-                            if call_params["messages"] and call_params["messages"][0].get("role") == "system":
-                                call_params["messages"][0]["content"] += schema_msg
-                            elif call_params["messages"]:
-                                call_params["messages"][0]["content"] = (
-                                    schema_msg + "\n\n" + call_params["messages"][0]["content"]
-                                )
+                        if strict_schema and schema is not None:
+                            # Use OpenAI's strict JSON schema enforcement
+                            # This guarantees all required fields are returned
+                            call_params["response_format"] = {
+                                "type": "json_schema",
+                                "json_schema": {
+                                    "name": "response",
+                                    "strict": True,
+                                    "schema": schema,
+                                },
+                            }
+                        else:
+                            # Soft enforcement: add schema to prompt and use json_object mode
+                            if schema is not None:
+                                schema_msg = f"\n\nYou must respond with valid JSON matching this schema:\n{json.dumps(schema, indent=2)}"

-                        call_params["response_format"] = {"type": "json_object"}
+                                if call_params["messages"] and call_params["messages"][0].get("role") == "system":
+                                    call_params["messages"][0]["content"] += schema_msg
+                                elif call_params["messages"]:
+                                    call_params["messages"][0]["content"] = (
+                                        schema_msg + "\n\n" + call_params["messages"][0]["content"]
+                                    )
+                            if self.provider not in ("lmstudio", "ollama"):
+                                # LM Studio and Ollama don't support json_object response format reliably
+                                # We rely on the schema in the system message instead
+                                call_params["response_format"] = {"type": "json_object"}
+
+                        logger.debug(f"Sending request to {self.provider}/{self.model} (timeout={self.timeout})")
                        response = await self._client.chat.completions.create(**call_params)
+                        logger.debug(f"Received response from {self.provider}/{self.model}")

                        content = response.choices[0].message.content

-                        # Log raw LLM response for debugging JSON parse issues
-                        try:
-                            json_data = json.loads(content)
-                        except json.JSONDecodeError as json_err:
-                            # Truncate content for logging (first 500 and last 200 chars)
-                            content_preview = content[:500] if content else "<empty>"
-                            if content and len(content) > 700:
-                                content_preview = f"{content[:500]}...TRUNCATED...{content[-200:]}"
-                            logger.warning(
-                                f"JSON parse error from LLM response (attempt {attempt + 1}/{max_retries + 1}): {json_err}\n"
-                                f"  Model: {self.provider}/{self.model}\n"
-                                f"  Content length: {len(content) if content else 0} chars\n"
-                                f"  Content preview: {content_preview!r}\n"
-                                f"  Finish reason: {response.choices[0].finish_reason if response.choices else 'unknown'}"
-                            )
-                            # Retry on JSON parse errors - LLM may return valid JSON on next attempt
-                            if attempt < max_retries:
-                                backoff = min(initial_backoff * (2**attempt), max_backoff)
-                                await asyncio.sleep(backoff)
-                                last_exception = json_err
-                                continue
-                            else:
-                                logger.error(f"JSON parse error after {max_retries + 1} attempts, giving up")
-                                raise
+                        # Strip reasoning model thinking tags
+                        # Supports: <think>, <thinking>, <reasoning>, |startthink|/|endthink|
+                        # for reasoning models that embed thinking in their output (e.g., Qwen3, DeepSeek)
+                        if content:
+                            original_len = len(content)
+                            content = re.sub(r"<think>.*?</think>", "", content, flags=re.DOTALL)
+                            content = re.sub(r"<thinking>.*?</thinking>", "", content, flags=re.DOTALL)
+                            content = re.sub(r"<reasoning>.*?</reasoning>", "", content, flags=re.DOTALL)
+                            content = re.sub(r"\|startthink\|.*?\|endthink\|", "", content, flags=re.DOTALL)
+                            content = content.strip()
+                            if len(content) < original_len:
+                                logger.debug(f"Stripped {original_len - len(content)} chars of reasoning tokens")
+
+                        # For local models, they may wrap JSON in markdown code blocks
+                        if self.provider in ("lmstudio", "ollama"):
+                            clean_content = content
+                            if "```json" in content:
+                                clean_content = content.split("```json")[1].split("```")[0].strip()
+                            elif "```" in content:
+                                clean_content = content.split("```")[1].split("```")[0].strip()
+                            try:
+                                json_data = json.loads(clean_content)
+                            except json.JSONDecodeError:
+                                # Fallback to parsing raw content
+                                json_data = json.loads(content)
+                        else:
+                            # Log raw LLM response for debugging JSON parse issues
+                            try:
+                                json_data = json.loads(content)
+                            except json.JSONDecodeError as json_err:
+                                # Truncate content for logging (first 500 and last 200 chars)
+                                content_preview = content[:500] if content else "<empty>"
+                                if content and len(content) > 700:
+                                    content_preview = f"{content[:500]}...TRUNCATED...{content[-200:]}"
+                                logger.warning(
+                                    f"JSON parse error from LLM response (attempt {attempt + 1}/{max_retries + 1}): {json_err}\n"
+                                    f"  Model: {self.provider}/{self.model}\n"
+                                    f"  Content length: {len(content) if content else 0} chars\n"
+                                    f"  Content preview: {content_preview!r}\n"
+                                    f"  Finish reason: {response.choices[0].finish_reason if response.choices else 'unknown'}"
+                                )
+                                # Retry on JSON parse errors - LLM may return valid JSON on next attempt
+                                if attempt < max_retries:
+                                    backoff = min(initial_backoff * (2**attempt), max_backoff)
+                                    await asyncio.sleep(backoff)
+                                    last_exception = json_err
+                                    continue
+                                else:
+                                    logger.error(f"JSON parse error after {max_retries + 1} attempts, giving up")
+                                    raise

                        if skip_validation:
                            result = json_data
@@ -276,21 +393,45 @@ class LLMProvider:
                        response = await self._client.chat.completions.create(**call_params)
                        result = response.choices[0].message.content

-                    # Log slow calls
+                    # Record token usage metrics
                    duration = time.time() - start_time
                    usage = response.usage
-                    if duration > 10.0:
-                        ratio = max(1, usage.completion_tokens) / usage.prompt_tokens
+                    input_tokens = usage.prompt_tokens or 0 if usage else 0
+                    output_tokens = usage.completion_tokens or 0 if usage else 0
+                    total_tokens = usage.total_tokens or 0 if usage else 0
+
+                    # Record LLM metrics
+                    metrics = get_metrics_collector()
+                    metrics.record_llm_call(
+                        provider=self.provider,
+                        model=self.model,
+                        scope=scope,
+                        duration=duration,
+                        input_tokens=input_tokens,
+                        output_tokens=output_tokens,
+                        success=True,
+                    )
+
+                    # Log slow calls
+                    if duration > 10.0 and usage:
+                        ratio = max(1, output_tokens) / max(1, input_tokens)
                        cached_tokens = 0
                        if hasattr(usage, "prompt_tokens_details") and usage.prompt_tokens_details:
                            cached_tokens = getattr(usage.prompt_tokens_details, "cached_tokens", 0) or 0
                        cache_info = f", cached_tokens={cached_tokens}" if cached_tokens > 0 else ""
                        logger.info(
                            f"slow llm call: model={self.provider}/{self.model}, "
-                            f"input_tokens={usage.prompt_tokens}, output_tokens={usage.completion_tokens}, "
-                            f"total_tokens={usage.total_tokens}{cache_info}, time={duration:.3f}s, ratio out/in={ratio:.2f}"
+                            f"input_tokens={input_tokens}, output_tokens={output_tokens}, "
+                            f"total_tokens={total_tokens}{cache_info}, time={duration:.3f}s, ratio out/in={ratio:.2f}"
                        )

+                    if return_usage:
+                        token_usage = TokenUsage(
+                            input_tokens=input_tokens,
+                            output_tokens=output_tokens,
+                            total_tokens=total_tokens,
+                        )
+                        return result, token_usage
                    return result

                except LengthFinishReasonError as e:
@@ -339,6 +480,165 @@ class LLMProvider:
                raise last_exception
            raise RuntimeError("LLM call failed after all retries with no exception captured")

+    async def _call_anthropic(
+        self,
+        messages: list[dict[str, str]],
+        response_format: Any | None,
+        max_completion_tokens: int | None,
+        max_retries: int,
+        initial_backoff: float,
+        max_backoff: float,
+        skip_validation: bool,
+        start_time: float,
+        return_usage: bool = False,
+    ) -> Any:
+        """Handle Anthropic-specific API calls."""
+        from anthropic import APIConnectionError, APIStatusError, RateLimitError
+
+        # Convert OpenAI-style messages to Anthropic format
+        system_prompt = None
+        anthropic_messages = []
+
+        for msg in messages:
+            role = msg.get("role", "user")
+            content = msg.get("content", "")
+
+            if role == "system":
+                if system_prompt:
+                    system_prompt += "\n\n" + content
+                else:
+                    system_prompt = content
+            else:
+                anthropic_messages.append({"role": role, "content": content})
+
+        # Add JSON schema instruction if response_format is provided
+        if response_format is not None and hasattr(response_format, "model_json_schema"):
+            schema = response_format.model_json_schema()
+            schema_msg = f"\n\nYou must respond with valid JSON matching this schema:\n{json.dumps(schema, indent=2)}"
+            if system_prompt:
+                system_prompt += schema_msg
+            else:
+                system_prompt = schema_msg
+
+        # Prepare parameters
+        call_params = {
+            "model": self.model,
+            "messages": anthropic_messages,
+            "max_tokens": max_completion_tokens if max_completion_tokens is not None else 4096,
+        }
+
+        if system_prompt:
+            call_params["system"] = system_prompt
+
+        last_exception = None
+
+        for attempt in range(max_retries + 1):
+            try:
+                response = await self._anthropic_client.messages.create(**call_params)
+
+                # Anthropic response content is a list of blocks
+                content = ""
+                for block in response.content:
+                    if block.type == "text":
+                        content += block.text
+
+                if response_format is not None:
+                    # Models may wrap JSON in markdown code blocks
+                    clean_content = content
+                    if "```json" in content:
+                        clean_content = content.split("```json")[1].split("```")[0].strip()
+                    elif "```" in content:
+                        clean_content = content.split("```")[1].split("```")[0].strip()
+
+                    try:
+                        json_data = json.loads(clean_content)
+                    except json.JSONDecodeError:
+                        # Fallback to parsing raw content if markdown stripping failed
+                        json_data = json.loads(content)
+
+                    if skip_validation:
+                        result = json_data
+                    else:
+                        result = response_format.model_validate(json_data)
+                else:
+                    result = content
+
+                # Record metrics and log slow calls
+                duration = time.time() - start_time
+                input_tokens = response.usage.input_tokens or 0 if response.usage else 0
+                output_tokens = response.usage.output_tokens or 0 if response.usage else 0
+                total_tokens = input_tokens + output_tokens
+
+                # Record LLM metrics
+                metrics = get_metrics_collector()
+                metrics.record_llm_call(
+                    provider=self.provider,
+                    model=self.model,
+                    scope="memory",
+                    duration=duration,
+                    input_tokens=input_tokens,
+                    output_tokens=output_tokens,
+                    success=True,
+                )
+
+                # Log slow calls
+                if duration > 10.0:
+                    logger.info(
+                        f"slow llm call: scope=memory, model={self.provider}/{self.model}, "
+                        f"input_tokens={input_tokens}, output_tokens={output_tokens}, "
+                        f"time={duration:.3f}s"
+                    )
+
+                if return_usage:
+                    token_usage = TokenUsage(
+                        input_tokens=input_tokens,
+                        output_tokens=output_tokens,
+                        total_tokens=total_tokens,
+                    )
+                    return result, token_usage
+                return result
+
+            except json.JSONDecodeError as e:
+                last_exception = e
+                if attempt < max_retries:
+                    logger.warning("Anthropic returned invalid JSON, retrying...")
+                    backoff = min(initial_backoff * (2**attempt), max_backoff)
+                    await asyncio.sleep(backoff)
+                    continue
+                else:
+                    logger.error(f"Anthropic returned invalid JSON after {max_retries + 1} attempts")
+                    raise
+
+            except (APIConnectionError, RateLimitError, APIStatusError) as e:
+                # Fast fail on 401/403
+                if isinstance(e, APIStatusError) and e.status_code in (401, 403):
+                    logger.error(f"Anthropic auth error (HTTP {e.status_code}), not retrying: {str(e)}")
+                    raise
+
+                last_exception = e
+                if attempt < max_retries:
+                    # Check if it's a rate limit or server error
+                    should_retry = isinstance(e, (APIConnectionError, RateLimitError)) or (
+                        isinstance(e, APIStatusError) and e.status_code >= 500
+                    )
+
+                    if should_retry:
+                        backoff = min(initial_backoff * (2**attempt), max_backoff)
+                        jitter = backoff * 0.2 * (2 * (time.time() % 1) - 1)
+                        await asyncio.sleep(backoff + jitter)
+                        continue
+
+                logger.error(f"Anthropic API error after {max_retries + 1} attempts: {str(e)}")
+                raise
+
+            except Exception as e:
+                logger.error(f"Unexpected error during Anthropic call: {type(e).__name__}: {str(e)}")
+                raise
+
+        if last_exception:
+            raise last_exception
+        raise RuntimeError("Anthropic call failed after all retries")
+
    async def _call_ollama_native(
        self,
        messages: list[dict[str, str]],
@@ -350,6 +650,7 @@ class LLMProvider:
        max_backoff: float,
        skip_validation: bool,
        start_time: float,
+        return_usage: bool = False,
    ) -> Any:
        """
        Call Ollama using native API with JSON schema enforcement.
@@ -424,11 +725,39 @@ class LLMProvider:
                        else:
                            raise

+                    # Extract token usage from Ollama response
+                    # Ollama returns prompt_eval_count (input) and eval_count (output)
+                    duration = time.time() - start_time
+                    input_tokens = result.get("prompt_eval_count", 0) or 0
+                    output_tokens = result.get("eval_count", 0) or 0
+                    total_tokens = input_tokens + output_tokens
+
+                    # Record LLM metrics
+                    metrics = get_metrics_collector()
+                    metrics.record_llm_call(
+                        provider=self.provider,
+                        model=self.model,
+                        scope="memory",
+                        duration=duration,
+                        input_tokens=input_tokens,
+                        output_tokens=output_tokens,
+                        success=True,
+                    )
+
                    # Validate against Pydantic model or return raw JSON
                    if skip_validation:
-                        return json_data
+                        validated_result = json_data
                    else:
-                        return response_format.model_validate(json_data)
+                        validated_result = response_format.model_validate(json_data)
+
+                    if return_usage:
+                        token_usage = TokenUsage(
+                            input_tokens=input_tokens,
+                            output_tokens=output_tokens,
+                            total_tokens=total_tokens,
+                        )
+                        return validated_result, token_usage
+                    return validated_result

                except httpx.HTTPStatusError as e:
                    last_exception = e
@@ -471,6 +800,7 @@ class LLMProvider:
        max_backoff: float,
        skip_validation: bool,
        start_time: float,
+        return_usage: bool = False,
    ) -> Any:
        """Handle Gemini-specific API calls."""
        # Convert OpenAI-style messages to Gemini format
@@ -547,16 +877,42 @@ class LLMProvider:
                else:
                    result = content

-                # Log slow calls
+                # Record metrics and log slow calls
                duration = time.time() - start_time
-                if duration > 10.0 and hasattr(response, "usage_metadata") and response.usage_metadata:
+                input_tokens = 0
+                output_tokens = 0
+                if hasattr(response, "usage_metadata") and response.usage_metadata:
                    usage = response.usage_metadata
+                    input_tokens = usage.prompt_token_count or 0
+                    output_tokens = usage.candidates_token_count or 0
+
+                # Record LLM metrics
+                metrics = get_metrics_collector()
+                metrics.record_llm_call(
+                    provider=self.provider,
+                    model=self.model,
+                    scope="memory",
+                    duration=duration,
+                    input_tokens=input_tokens,
+                    output_tokens=output_tokens,
+                    success=True,
+                )
+
+                # Log slow calls
+                if duration > 10.0 and input_tokens > 0:
                    logger.info(
-                        f"slow llm call: model={self.provider}/{self.model}, "
-                        f"input_tokens={usage.prompt_token_count}, output_tokens={usage.candidates_token_count}, "
+                        f"slow llm call: scope=memory, model={self.provider}/{self.model}, "
+                        f"input_tokens={input_tokens}, output_tokens={output_tokens}, "
                        f"time={duration:.3f}s"
                    )

+                if return_usage:
+                    token_usage = TokenUsage(
+                        input_tokens=input_tokens,
+                        output_tokens=output_tokens,
+                        total_tokens=input_tokens + output_tokens,
+                    )
+                    return result, token_usage
                return result

            except json.JSONDecodeError as e:
@@ -17,6 +17,9 @@ import uuid
 from datetime import UTC, datetime, timedelta
 from typing import TYPE_CHECKING, Any

+from ..config import get_config
+from ..metrics import get_metrics_collector
+
 # Context variable for current schema (async-safe, per-task isolation)
 _current_schema: contextvars.ContextVar[str] = contextvars.ContextVar("current_schema", default="public")

@@ -130,11 +133,18 @@ if TYPE_CHECKING:

 from enum import Enum

-from ..pg0 import EmbeddedPostgres
+from ..pg0 import EmbeddedPostgres, parse_pg0_url
 from .entity_resolver import EntityResolver
 from .llm_wrapper import LLMConfig
 from .query_analyzer import QueryAnalyzer
-from .response_models import VALID_RECALL_FACT_TYPES, EntityObservation, EntityState, MemoryFact, ReflectResult
+from .response_models import (
+    VALID_RECALL_FACT_TYPES,
+    EntityObservation,
+    EntityState,
+    MemoryFact,
+    ReflectResult,
+    TokenUsage,
+)
 from .response_models import RecallResult as RecallResultModel
 from .retain import bank_utils, embedding_utils
 from .retain.types import RetainContentDict
@@ -202,6 +212,8 @@ class MemoryEngine(MemoryEngineInterface):
        run_migrations: bool = True,
        operation_validator: "OperationValidatorExtension | None" = None,
        tenant_extension: "TenantExtension | None" = None,
+        skip_llm_verification: bool | None = None,
+        lazy_reranker: bool | None = None,
    ):
        """
        Initialize the temporal + semantic memory system.
@@ -227,12 +239,23 @@ class MemoryEngine(MemoryEngineInterface):
                                If provided, retain/recall/reflect operations will be validated.
            tenant_extension: Optional extension for multi-tenancy and API key authentication.
                             If provided, operations require a RequestContext for authentication.
+            skip_llm_verification: Skip LLM connection verification during initialization.
+                                  Defaults to HINDSIGHT_API_SKIP_LLM_VERIFICATION env var or False.
+            lazy_reranker: Delay reranker initialization until first use. Useful for retain-only
+                          operations that don't need the cross-encoder. Defaults to
+                          HINDSIGHT_API_LAZY_RERANKER env var or False.
        """
        # Load config from environment for any missing parameters
        from ..config import get_config

        config = get_config()

+        # Apply optimization flags from config if not explicitly provided
+        self._skip_llm_verification = (
+            skip_llm_verification if skip_llm_verification is not None else config.skip_llm_verification
+        )
+        self._lazy_reranker = lazy_reranker if lazy_reranker is not None else config.lazy_reranker
+
        # Apply defaults from config
        db_url = db_url or config.database_url
        memory_llm_provider = memory_llm_provider or config.llm_provider
@@ -244,31 +267,14 @@ class MemoryEngine(MemoryEngineInterface):
        memory_llm_base_url = memory_llm_base_url or config.get_llm_base_url() or None
        # Track pg0 instance (if used)
        self._pg0: EmbeddedPostgres | None = None
-        self._pg0_instance_name: str | None = None

        # Initialize PostgreSQL connection URL
        # The actual URL will be set during initialize() after starting the server
        # Supports: "pg0" (default instance), "pg0://instance-name" (named instance), or regular postgresql:// URL
-        if db_url == "pg0":
-            self._use_pg0 = True
-            self._pg0_instance_name = "hindsight"
-            self._pg0_port = None  # Use default port
-            self.db_url = None
-        elif db_url.startswith("pg0://"):
-            self._use_pg0 = True
-            # Parse instance name and optional port: pg0://instance-name or pg0://instance-name:port
-            url_part = db_url[6:]  # Remove "pg0://"
-            if ":" in url_part:
-                self._pg0_instance_name, port_str = url_part.rsplit(":", 1)
-                self._pg0_port = int(port_str)
-            else:
-                self._pg0_instance_name = url_part or "hindsight"
-                self._pg0_port = None  # Use default port
+        self._use_pg0, self._pg0_instance_name, self._pg0_port = parse_pg0_url(db_url)
+        if self._use_pg0:
            self.db_url = None
        else:
-            self._use_pg0 = False
-            self._pg0_instance_name = None
-            self._pg0_port = None
            self.db_url = db_url

        # Set default base URL if not provided
@@ -359,7 +365,7 @@ class MemoryEngine(MemoryEngineInterface):

        result = await validation_coro
        if not result.allowed:
-            raise OperationValidationError(result.reason or "Operation not allowed")
+            raise OperationValidationError(result.reason or "Operation not allowed", result.status_code)

    async def _authenticate_tenant(self, request_context: "RequestContext | None") -> str:
        """
@@ -386,7 +392,9 @@ class MemoryEngine(MemoryEngineInterface):
        if request_context is None:
            raise AuthenticationError("RequestContext is required when tenant extension is configured")

+        # Let AuthenticationError propagate - HTTP layer will convert to 401
        tenant_context = await self._tenant_extension.authenticate(request_context)
+
        _current_schema.set(tenant_context.schema_name)
        return tenant_context.schema_name

@@ -396,22 +404,22 @@ class MemoryEngine(MemoryEngineInterface):

        Args:
            task_dict: Dict with 'node_ids' key containing list of node IDs to update
+
+        Raises:
+            Exception: Any exception from database operations (propagates to execute_task for retry)
        """
        node_ids = task_dict.get("node_ids", [])
        if not node_ids:
            return

        pool = await self._get_pool()
-        try:
-            # Convert string UUIDs to UUID type for faster matching
-            uuid_list = [uuid.UUID(nid) for nid in node_ids]
-            async with acquire_with_retry(pool) as conn:
-                await conn.execute(
-                    f"UPDATE {fq_table('memory_units')} SET access_count = access_count + 1 WHERE id = ANY($1::uuid[])",
-                    uuid_list,
-                )
-        except Exception as e:
-            logger.error(f"Access count handler: Error updating access counts: {e}")
+        # Convert string UUIDs to UUID type for faster matching
+        uuid_list = [uuid.UUID(nid) for nid in node_ids]
+        async with acquire_with_retry(pool) as conn:
+            await conn.execute(
+                f"UPDATE {fq_table('memory_units')} SET access_count = access_count + 1 WHERE id = ANY($1::uuid[])",
+                uuid_list,
+            )

    async def _handle_batch_retain(self, task_dict: dict[str, Any]):
        """
@@ -419,29 +427,27 @@ class MemoryEngine(MemoryEngineInterface):

        Args:
            task_dict: Dict with 'bank_id', 'contents'
+
+        Raises:
+            ValueError: If bank_id is missing
+            Exception: Any exception from retain_batch_async (propagates to execute_task for retry)
        """
-        try:
-            bank_id = task_dict.get("bank_id")
-            if not bank_id:
-                raise ValueError("bank_id is required for batch retain task")
-            contents = task_dict.get("contents", [])
+        bank_id = task_dict.get("bank_id")
+        if not bank_id:
+            raise ValueError("bank_id is required for batch retain task")
+        contents = task_dict.get("contents", [])

-            logger.info(
-                f"[BATCH_RETAIN_TASK] Starting background batch retain for bank_id={bank_id}, {len(contents)} items"
-            )
+        logger.info(
+            f"[BATCH_RETAIN_TASK] Starting background batch retain for bank_id={bank_id}, {len(contents)} items"
+        )

-            # Use internal request context for background tasks
-            from hindsight_api.models import RequestContext
+        # Use internal request context for background tasks
+        from hindsight_api.models import RequestContext

-            internal_context = RequestContext()
-            await self.retain_batch_async(bank_id=bank_id, contents=contents, request_context=internal_context)
+        internal_context = RequestContext()
+        await self.retain_batch_async(bank_id=bank_id, contents=contents, request_context=internal_context)

-            logger.info(f"[BATCH_RETAIN_TASK] Completed background batch retain for bank_id={bank_id}")
-        except Exception as e:
-            logger.error(f"Batch retain handler: Error processing batch retain: {e}")
-            import traceback
-
-            traceback.print_exc()
+        logger.info(f"[BATCH_RETAIN_TASK] Completed background batch retain for bank_id={bank_id}")

    async def execute_task(self, task_dict: dict[str, Any]):
        """
@@ -594,6 +600,8 @@ class MemoryEngine(MemoryEngineInterface):
                await loop.run_in_executor(None, lambda: asyncio.run(cross_encoder.initialize()))
            else:
                await cross_encoder.initialize()
+            # Mark reranker as initialized
+            self._cross_encoder_reranker._initialized = True

        async def init_query_analyzer():
            """Initialize query analyzer model."""
@@ -602,26 +610,40 @@ class MemoryEngine(MemoryEngineInterface):

        async def verify_llm():
            """Verify LLM connection is working."""
-            await self._llm_config.verify_connection()
+            if not self._skip_llm_verification:
+                await self._llm_config.verify_connection()

-        # Run pg0 and all model initializations in parallel
-        await asyncio.gather(
+        # Build list of initialization tasks
+        init_tasks = [
            start_pg0(),
            init_embeddings(),
-            init_cross_encoder(),
            init_query_analyzer(),
-            verify_llm(),
-        )
+        ]
+
+        # Only init cross-encoder eagerly if not using lazy initialization
+        if not self._lazy_reranker:
+            init_tasks.append(init_cross_encoder())
+
+        # Only verify LLM if not skipping
+        if not self._skip_llm_verification:
+            init_tasks.append(verify_llm())
+
+        # Run pg0 and selected model initializations in parallel
+        await asyncio.gather(*init_tasks)

        # Run database migrations if enabled
        if self._run_migrations:
-            from ..migrations import run_migrations
+            from ..migrations import ensure_embedding_dimension, run_migrations

            if not self.db_url:
                raise ValueError("Database URL is required for migrations")
            logger.info("Running database migrations...")
            run_migrations(self.db_url)

+            # Ensure embedding column dimension matches the model's dimension
+            # This is done after migrations and after embeddings.initialize()
+            ensure_embedding_dimension(self.db_url, self.embeddings.dimension)
+
        logger.info(f"Connecting to PostgreSQL at {self.db_url}")

        # Create connection pool
@@ -672,7 +694,14 @@ class MemoryEngine(MemoryEngineInterface):

        Returns:
            dict with status and optional error message
+
+        Note:
+            Returns unhealthy until initialize() has completed successfully.
        """
+        # Not healthy until fully initialized
+        if not self._initialized:
+            return {"status": "unhealthy", "reason": "not_initialized"}
+
        try:
            pool = await self._get_pool()
            async with pool.acquire() as conn:
@@ -933,7 +962,8 @@ class MemoryEngine(MemoryEngineInterface):
        document_id: str | None = None,
        fact_type_override: str | None = None,
        confidence_score: float | None = None,
-    ) -> list[list[str]]:
+        return_usage: bool = False,
+    ):
        """
        Store multiple content items as memory units in ONE batch operation.

@@ -954,9 +984,11 @@ class MemoryEngine(MemoryEngineInterface):
                        Applies the same document_id to ALL content items that don't specify their own.
            fact_type_override: Override fact type for all facts ('world', 'experience', 'opinion')
            confidence_score: Confidence score for opinions (0.0 to 1.0)
+            return_usage: If True, returns tuple of (unit_ids, TokenUsage). Default False for backward compatibility.

        Returns:
-            List of lists of unit IDs (one list per content item)
+            If return_usage=False: List of lists of unit IDs (one list per content item)
+            If return_usage=True: Tuple of (unit_ids, TokenUsage)

        Example (new style - per-content document_id):
            unit_ids = await memory.retain_batch_async(
@@ -983,6 +1015,8 @@ class MemoryEngine(MemoryEngineInterface):
        start_time = time.time()

        if not contents:
+            if return_usage:
+                return [], TokenUsage()
            return []

        # Authenticate tenant and set schema in context (for fq_table())
@@ -1012,6 +1046,7 @@ class MemoryEngine(MemoryEngineInterface):
        # Auto-chunk large batches by character count to avoid timeouts and memory issues
        # Calculate total character count
        total_chars = sum(len(item.get("content", "")) for item in contents)
+        total_usage = TokenUsage()

        CHARS_PER_BATCH = 600_000

@@ -1052,7 +1087,7 @@ class MemoryEngine(MemoryEngineInterface):
                    f"Processing sub-batch {i}/{len(sub_batches)}: {len(sub_batch)} items, {sub_batch_chars:,} chars"
                )

-                sub_results = await self._retain_batch_async_internal(
+                sub_results, sub_usage = await self._retain_batch_async_internal(
                    bank_id=bank_id,
                    contents=sub_batch,
                    document_id=document_id,
@@ -1061,6 +1096,7 @@ class MemoryEngine(MemoryEngineInterface):
                    confidence_score=confidence_score,
                )
                all_results.extend(sub_results)
+                total_usage = total_usage + sub_usage

            total_time = time.time() - start_time
            logger.info(
@@ -1069,7 +1105,7 @@ class MemoryEngine(MemoryEngineInterface):
            result = all_results
        else:
            # Small batch - use internal method directly
-            result = await self._retain_batch_async_internal(
+            result, total_usage = await self._retain_batch_async_internal(
                bank_id=bank_id,
                contents=contents,
                document_id=document_id,
@@ -1098,6 +1134,8 @@ class MemoryEngine(MemoryEngineInterface):
            except Exception as e:
                logger.warning(f"Post-retain hook error (non-fatal): {e}")

+        if return_usage:
+            return result, total_usage
        return result

    async def _retain_batch_async_internal(
@@ -1108,7 +1146,7 @@ class MemoryEngine(MemoryEngineInterface):
        is_first_batch: bool = True,
        fact_type_override: str | None = None,
        confidence_score: float | None = None,
-    ) -> list[list[str]]:
+    ) -> tuple[list[list[str]], "TokenUsage"]:
        """
        Internal method for batch processing without chunking logic.

@@ -1124,6 +1162,9 @@ class MemoryEngine(MemoryEngineInterface):
            is_first_batch: Whether this is the first batch (for chunked operations, only delete on first batch)
            fact_type_override: Override fact type for all facts
            confidence_score: Confidence score for opinions
+
+        Returns:
+            Tuple of (unit ID lists, token usage for fact extraction)
        """
        # Backpressure: limit concurrent retains to prevent database contention
        async with self._put_semaphore:
@@ -1641,6 +1682,9 @@ class MemoryEngine(MemoryEngineInterface):
            step_start = time.time()
            reranker_instance = self._cross_encoder_reranker

+            # Ensure reranker is initialized (for lazy initialization mode)
+            await reranker_instance.ensure_initialized()
+
            # Rerank using cross-encoder
            scored_results = reranker_instance.rerank(query, merged_candidates)

@@ -2221,6 +2265,7 @@ class MemoryEngine(MemoryEngineInterface):
        bank_id: str | None = None,
        fact_type: str | None = None,
        *,
+        limit: int = 1000,
        request_context: "RequestContext",
    ):
        """
@@ -2229,10 +2274,11 @@ class MemoryEngine(MemoryEngineInterface):
        Args:
            bank_id: Filter by bank ID
            fact_type: Filter by fact type (world, experience, opinion)
+            limit: Maximum number of items to return (default: 1000)
            request_context: Request context for authentication.

        Returns:
-            Dict with nodes, edges, and table_rows
+            Dict with nodes, edges, table_rows, total_units, and limit
        """
        await self._authenticate_tenant(request_context)
        pool = await self._get_pool()
@@ -2254,15 +2300,29 @@ class MemoryEngine(MemoryEngineInterface):

            where_clause = "WHERE " + " AND ".join(query_conditions) if query_conditions else ""

+            # Get total count first
+            total_count_result = await conn.fetchrow(
+                f"""
+                SELECT COUNT(*) as total
+                FROM {fq_table("memory_units")}
+                {where_clause}
+            """,
+                *query_params,
+            )
+            total_count = total_count_result["total"] if total_count_result else 0
+
+            # Get units with limit
+            param_count += 1
            units = await conn.fetch(
                f"""
                SELECT id, text, event_date, context, occurred_start, occurred_end, mentioned_at, document_id, chunk_id, fact_type
                FROM {fq_table("memory_units")}
                {where_clause}
                ORDER BY mentioned_at DESC NULLS LAST, event_date DESC
-                LIMIT 1000
+                LIMIT ${param_count}
            """,
                *query_params,
+                limit,
            )

            # Get links, filtering to only include links between units of the selected agent
@@ -2399,7 +2459,7 @@ class MemoryEngine(MemoryEngineInterface):
                }
            )

-        return {"nodes": nodes, "edges": edges, "table_rows": table_rows, "total_units": len(units)}
+        return {"nodes": nodes, "edges": edges, "table_rows": table_rows, "total_units": total_count, "limit": limit}

    async def list_memory_units(
        self,
@@ -2792,13 +2852,16 @@ Guidelines:
        Handler for form opinion tasks.

        Args:
-            task_dict: Dict with keys: 'bank_id', 'answer_text', 'query'
+            task_dict: Dict with keys: 'bank_id', 'answer_text', 'query', 'tenant_id'
        """
        bank_id = task_dict["bank_id"]
        answer_text = task_dict["answer_text"]
        query = task_dict["query"]
+        tenant_id = task_dict.get("tenant_id")

-        await self._extract_and_store_opinions_async(bank_id=bank_id, answer_text=answer_text, query=query)
+        await self._extract_and_store_opinions_async(
+            bank_id=bank_id, answer_text=answer_text, query=query, tenant_id=tenant_id
+        )

    async def _handle_reinforce_opinion(self, task_dict: dict[str, Any]):
        """
@@ -3043,6 +3106,8 @@ Guidelines:
        *,
        budget: Budget | None = None,
        context: str | None = None,
+        max_tokens: int = 4096,
+        response_schema: dict | None = None,
        request_context: "RequestContext",
    ) -> ReflectResult:
        """
@@ -3054,19 +3119,22 @@ Guidelines:
        3. Retrieves existing opinions (bank's formed perspectives)
        4. Uses LLM to formulate an answer
        5. Extracts and stores any new opinions formed during reflection
-        6. Returns plain text answer and the facts used
+        6. Optionally generates structured output based on response_schema
+        7. Returns plain text answer and the facts used

        Args:
            bank_id: bank identifier
            query: Question to answer
            budget: Budget level for memory exploration (low=100, mid=300, high=600 units)
            context: Additional context string to include in LLM prompt (not used in recall)
+            response_schema: Optional JSON Schema for structured output

        Returns:
            ReflectResult containing:
                - text: Plain text answer (no markdown)
                - based_on: Dict with 'world', 'experience', and 'opinion' fact lists (MemoryFact objects)
                - new_opinions: List of newly formed opinions
+                - structured_output: Optional dict if response_schema was provided
        """
        # Use cached LLM config
        if self._llm_config is None:
@@ -3095,16 +3163,20 @@ Guidelines:

        # Steps 1-3: Run multi-fact-type search (12-way retrieval: 4 methods × 3 fact types)
        recall_start = time.time()
-        search_result = await self.recall_async(
-            bank_id=bank_id,
-            query=query,
-            budget=budget,
-            max_tokens=4096,
-            enable_trace=False,
-            fact_type=["experience", "world", "opinion"],
-            include_entities=True,
-            request_context=request_context,
-        )
+        metrics = get_metrics_collector()
+        with metrics.record_operation(
+            "recall", bank_id=bank_id, source="reflect", budget=budget.value if budget else None
+        ):
+            search_result = await self.recall_async(
+                bank_id=bank_id,
+                query=query,
+                budget=budget,
+                max_tokens=4096,
+                enable_trace=False,
+                fact_type=["experience", "world", "opinion"],
+                include_entities=True,
+                request_context=request_context,
+            )
        recall_time = time.time() - recall_start

        all_results = search_result.results
@@ -3144,21 +3216,54 @@ Guidelines:
        log_buffer.append(f"[REFLECT {reflect_id}] Prompt: {len(prompt)} chars")

        system_message = think_utils.get_system_message(disposition)
+        messages = [{"role": "system", "content": system_message}, {"role": "user", "content": prompt}]
+
+        # Prepare response_format if schema provided
+        response_format = None
+        if response_schema is not None:
+            # Wrapper class to provide Pydantic-like interface for raw JSON schemas
+            class JsonSchemaWrapper:
+                def __init__(self, schema: dict):
+                    self._schema = schema
+
+                def model_json_schema(self):
+                    return self._schema
+
+            response_format = JsonSchemaWrapper(response_schema)

        llm_start = time.time()
-        answer_text = await self._llm_config.call(
-            messages=[{"role": "system", "content": system_message}, {"role": "user", "content": prompt}],
-            scope="memory_think",
-            temperature=0.9,
-            max_completion_tokens=1000,
+        llm_result, usage = await self._llm_config.call(
+            messages=messages,
+            scope="memory_reflect",
+            max_completion_tokens=max_tokens,
+            response_format=response_format,
+            skip_validation=True if response_format else False,
+            # Don't enforce strict_schema - not all providers support it and may retry forever
+            # Soft enforcement (schema in prompt + json_object mode) is sufficient
+            strict_schema=False,
+            return_usage=True,
        )
        llm_time = time.time() - llm_start

-        answer_text = answer_text.strip()
+        # Handle response based on whether structured output was requested
+        if response_schema is not None:
+            structured_output = llm_result
+            answer_text = ""  # Empty for backward compatibility
+            log_buffer.append(f"[REFLECT {reflect_id}] Structured output generated")
+        else:
+            structured_output = None
+            answer_text = llm_result.strip()

        # Submit form_opinion task for background processing
+        # Pass tenant_id from request context for internal authentication in background task
        await self._task_backend.submit_task(
-            {"type": "form_opinion", "bank_id": bank_id, "answer_text": answer_text, "query": query}
+            {
+                "type": "form_opinion",
+                "bank_id": bank_id,
+                "answer_text": answer_text,
+                "query": query,
+                "tenant_id": getattr(request_context, "tenant_id", None) if request_context else None,
+            }
        )

        total_time = time.time() - reflect_start
@@ -3172,6 +3277,8 @@ Guidelines:
            text=answer_text,
            based_on={"world": world_results, "experience": agent_results, "opinion": opinion_results},
            new_opinions=[],  # Opinions are being extracted asynchronously
+            structured_output=structured_output,
+            usage=usage,
        )

        # Call post-operation hook if validator is configured
@@ -3195,7 +3302,9 @@ Guidelines:

        return result

-    async def _extract_and_store_opinions_async(self, bank_id: str, answer_text: str, query: str):
+    async def _extract_and_store_opinions_async(
+        self, bank_id: str, answer_text: str, query: str, tenant_id: str | None = None
+    ):
        """
        Background task to extract and store opinions from think response.

@@ -3205,6 +3314,7 @@ Guidelines:
            bank_id: bank IDentifier
            answer_text: The generated answer text
            query: The original query
+            tenant_id: Tenant identifier for internal authentication
        """
        try:
            # Extract opinions from the answer
@@ -3215,10 +3325,11 @@ Guidelines:
                from datetime import datetime

                current_time = datetime.now(UTC)
-                # Use internal request context for background tasks
+                # Use internal context with tenant_id for background authentication
+                # Extension can check internal=True to bypass normal auth
                from hindsight_api.models import RequestContext

-                internal_context = RequestContext()
+                internal_context = RequestContext(tenant_id=tenant_id, internal=True)
                for opinion in new_opinions:
                    await self.retain_async(
                        bank_id=bank_id,
@@ -3539,7 +3650,7 @@ Guidelines:
        self,
        bank_id: str,
        entity_ids: list[str],
-        min_facts: int = 5,
+        min_facts: int | None = None,
        conn=None,
        request_context: "RequestContext | None" = None,
    ) -> None:
@@ -3551,12 +3662,16 @@ Guidelines:
        Args:
            bank_id: Bank identifier
            entity_ids: List of entity IDs to process
-            min_facts: Minimum facts required to regenerate observations
+            min_facts: Minimum facts required to regenerate observations (uses config default if None)
            conn: Optional database connection (for transactional atomicity)
        """
        if not bank_id or not entity_ids:
            return

+        # Use config default if min_facts not specified
+        if min_facts is None:
+            min_facts = get_config().observation_min_facts
+
        # Convert to UUIDs
        entity_uuids = [uuid.UUID(eid) if isinstance(eid, str) else eid for eid in entity_ids]

@@ -3649,90 +3764,87 @@ Guidelines:
            task_dict: Dict with 'bank_id' and either:
                       - 'entity_ids' (list): Process multiple entities
                       - 'entity_id', 'entity_name': Process single entity (legacy)
+
+        Raises:
+            ValueError: If required fields are missing
+            Exception: Any exception from regenerate_entity_observations (propagates to execute_task for retry)
        """
-        try:
-            bank_id = task_dict.get("bank_id")
-            # Use internal request context for background tasks
-            from hindsight_api.models import RequestContext
+        bank_id = task_dict.get("bank_id")
+        # Use internal request context for background tasks
+        from hindsight_api.models import RequestContext

-            internal_context = RequestContext()
+        internal_context = RequestContext()

-            # New format: multiple entity_ids
-            if "entity_ids" in task_dict:
-                entity_ids = task_dict.get("entity_ids", [])
-                min_facts = task_dict.get("min_facts", 5)
+        # New format: multiple entity_ids
+        if "entity_ids" in task_dict:
+            entity_ids = task_dict.get("entity_ids", [])
+            min_facts = task_dict.get("min_facts", 5)

-                if not bank_id or not entity_ids:
-                    logger.error(f"[OBSERVATIONS] Missing required fields in task: {task_dict}")
-                    return
+            if not bank_id or not entity_ids:
+                raise ValueError(f"[OBSERVATIONS] Missing required fields in task: {task_dict}")

-                # Process each entity
-                pool = await self._get_pool()
-                async with pool.acquire() as conn:
-                    for entity_id in entity_ids:
-                        try:
-                            # Fetch entity name and check fact count
-                            import uuid as uuid_module
+            # Process each entity
+            pool = await self._get_pool()
+            async with pool.acquire() as conn:
+                for entity_id in entity_ids:
+                    try:
+                        # Fetch entity name and check fact count
+                        import uuid as uuid_module

-                            entity_uuid = uuid_module.UUID(entity_id) if isinstance(entity_id, str) else entity_id
+                        entity_uuid = uuid_module.UUID(entity_id) if isinstance(entity_id, str) else entity_id

-                            # First check if entity exists
-                            entity_exists = await conn.fetchrow(
-                                f"SELECT canonical_name FROM {fq_table('entities')} WHERE id = $1 AND bank_id = $2",
-                                entity_uuid,
-                                bank_id,
-                            )
+                        # First check if entity exists
+                        entity_exists = await conn.fetchrow(
+                            f"SELECT canonical_name FROM {fq_table('entities')} WHERE id = $1 AND bank_id = $2",
+                            entity_uuid,
+                            bank_id,
+                        )

-                            if not entity_exists:
-                                logger.debug(f"[OBSERVATIONS] Entity {entity_id} not yet in bank {bank_id}, skipping")
-                                continue
-
-                            entity_name = entity_exists["canonical_name"]
-
-                            # Count facts linked to this entity
-                            fact_count = (
-                                await conn.fetchval(
-                                    f"SELECT COUNT(*) FROM {fq_table('unit_entities')} WHERE entity_id = $1",
-                                    entity_uuid,
-                                )
-                                or 0
-                            )
-
-                            # Only regenerate if entity has enough facts
-                            if fact_count >= min_facts:
-                                await self.regenerate_entity_observations(
-                                    bank_id, entity_id, entity_name, version=None, request_context=internal_context
-                                )
-                            else:
-                                logger.debug(
-                                    f"[OBSERVATIONS] Skipping {entity_name} ({fact_count} facts < {min_facts} threshold)"
-                                )
-
-                        except Exception as e:
-                            logger.error(f"[OBSERVATIONS] Error processing entity {entity_id}: {e}")
+                        if not entity_exists:
+                            logger.debug(f"[OBSERVATIONS] Entity {entity_id} not yet in bank {bank_id}, skipping")
                            continue

-            # Legacy format: single entity
-            else:
-                entity_id = task_dict.get("entity_id")
-                entity_name = task_dict.get("entity_name")
-                version = task_dict.get("version")
+                        entity_name = entity_exists["canonical_name"]

-                if not all([bank_id, entity_id, entity_name]):
-                    logger.error(f"[OBSERVATIONS] Missing required fields in task: {task_dict}")
-                    return
+                        # Count facts linked to this entity
+                        fact_count = (
+                            await conn.fetchval(
+                                f"SELECT COUNT(*) FROM {fq_table('unit_entities')} WHERE entity_id = $1",
+                                entity_uuid,
+                            )
+                            or 0
+                        )

-                # Type assertions after validation
-                assert isinstance(bank_id, str) and isinstance(entity_id, str) and isinstance(entity_name, str)
-                await self.regenerate_entity_observations(
-                    bank_id, entity_id, entity_name, version=version, request_context=internal_context
-                )
+                        # Only regenerate if entity has enough facts
+                        if fact_count >= min_facts:
+                            await self.regenerate_entity_observations(
+                                bank_id, entity_id, entity_name, version=None, request_context=internal_context
+                            )
+                        else:
+                            logger.debug(
+                                f"[OBSERVATIONS] Skipping {entity_name} ({fact_count} facts < {min_facts} threshold)"
+                            )

-        except Exception as e:
-            logger.error(f"[OBSERVATIONS] Error regenerating observations: {e}")
-            import traceback
+                    except Exception as e:
+                        # Log but continue processing other entities - individual entity failures
+                        # shouldn't fail the whole batch
+                        logger.error(f"[OBSERVATIONS] Error processing entity {entity_id}: {e}")
+                        continue

-            traceback.print_exc()
+        # Legacy format: single entity
+        else:
+            entity_id = task_dict.get("entity_id")
+            entity_name = task_dict.get("entity_name")
+            version = task_dict.get("version")
+
+            if not all([bank_id, entity_id, entity_name]):
+                raise ValueError(f"[OBSERVATIONS] Missing required fields in task: {task_dict}")
+
+            # Type assertions after validation
+            assert isinstance(bank_id, str) and isinstance(entity_id, str) and isinstance(entity_name, str)
+            await self.regenerate_entity_observations(
+                bank_id, entity_id, entity_name, version=version, request_context=internal_context
+            )

    # =========================================================================
    # Statistics & Operations (for HTTP API layer)
@@ -14,6 +14,37 @@ from pydantic import BaseModel, ConfigDict, Field
 VALID_RECALL_FACT_TYPES = frozenset(["world", "experience", "opinion"])


+class TokenUsage(BaseModel):
+    """
+    Token usage metrics for LLM calls.
+
+    Tracks input/output tokens for a single request to enable
+    per-request cost tracking and monitoring.
+    """
+
+    model_config = ConfigDict(
+        json_schema_extra={
+            "example": {
+                "input_tokens": 1500,
+                "output_tokens": 500,
+                "total_tokens": 2000,
+            }
+        }
+    )
+
+    input_tokens: int = Field(default=0, description="Number of input/prompt tokens consumed")
+    output_tokens: int = Field(default=0, description="Number of output/completion tokens generated")
+    total_tokens: int = Field(default=0, description="Total tokens (input + output)")
+
+    def __add__(self, other: "TokenUsage") -> "TokenUsage":
+        """Allow aggregating token usage from multiple calls."""
+        return TokenUsage(
+            input_tokens=self.input_tokens + other.input_tokens,
+            output_tokens=self.output_tokens + other.output_tokens,
+            total_tokens=self.total_tokens + other.total_tokens,
+        )
+
+
 class DispositionTraits(BaseModel):
    """
    Disposition traits for a memory bank.
@@ -123,7 +154,8 @@ class ReflectResult(BaseModel):
    Result from a reflect operation.

    Contains the formulated answer, the facts it was based on (organized by type),
-    and any new opinions that were formed during the reflection process.
+    any new opinions that were formed during the reflection process, and optionally
+    structured output if a response schema was provided.
    """

    model_config = ConfigDict(
@@ -145,6 +177,8 @@ class ReflectResult(BaseModel):
                    "opinion": [],
                },
                "new_opinions": ["Machine learning has great potential in healthcare"],
+                "structured_output": {"summary": "ML in healthcare", "confidence": 0.9},
+                "usage": {"input_tokens": 1500, "output_tokens": 500, "total_tokens": 2000},
            }
        }
    )
@@ -154,6 +188,14 @@ class ReflectResult(BaseModel):
        description="Facts used to formulate the answer, organized by type (world, experience, opinion)"
    )
    new_opinions: list[str] = Field(default_factory=list, description="List of newly formed opinions during reflection")
+    structured_output: dict[str, Any] | None = Field(
+        default=None,
+        description="Structured output parsed according to the provided response schema. Only present when response_schema was provided.",
+    )
+    usage: TokenUsage | None = Field(
+        default=None,
+        description="Token usage metrics for the LLM calls made during this reflect operation.",
+    )


 class Opinion(BaseModel):
@@ -13,16 +13,23 @@ logger = logging.getLogger(__name__)


 async def process_entities_batch(
-    entity_resolver, conn, bank_id: str, unit_ids: list[str], facts: list[ProcessedFact], log_buffer: list[str] = None
+    entity_resolver,
+    conn,
+    bank_id: str,
+    unit_ids: list[str],
+    facts: list[ProcessedFact],
+    log_buffer: list[str] = None,
+    user_entities_per_content: dict[int, list[dict]] = None,
 ) -> list[EntityLink]:
    """
    Process entities for all facts and create entity links.

    This function:
    1. Extracts entity mentions from fact texts
-    2. Resolves entity names to canonical entities
-    3. Creates entity records in the database
-    4. Returns entity links ready for insertion
+    2. Merges user-provided entities with LLM-extracted entities
+    3. Resolves entity names to canonical entities
+    4. Creates entity records in the database
+    5. Returns entity links ready for insertion

    Args:
        entity_resolver: EntityResolver instance for entity resolution
@@ -31,6 +38,7 @@ async def process_entities_batch(
        unit_ids: List of unit IDs (same length as facts)
        facts: List of ProcessedFact objects
        log_buffer: Optional buffer for detailed logging
+        user_entities_per_content: Dict mapping content_index to list of user-provided entities

    Returns:
        List of EntityLink objects for batch insertion
@@ -41,14 +49,35 @@ async def process_entities_batch(
    if len(unit_ids) != len(facts):
        raise ValueError(f"Mismatch between unit_ids ({len(unit_ids)}) and facts ({len(facts)})")

+    user_entities_per_content = user_entities_per_content or {}
+
    # Extract data for link_utils function
    fact_texts = [fact.fact_text for fact in facts]
    # Use occurred_start if available, otherwise use mentioned_at for entity timestamps
    fact_dates = [fact.occurred_start if fact.occurred_start is not None else fact.mentioned_at for fact in facts]
-    # Convert EntityRef objects to dict format expected by link_utils
-    entities_per_fact = [
-        [{"text": entity.name, "type": "CONCEPT"} for entity in (fact.entities or [])] for fact in facts
-    ]
+
+    # Convert EntityRef objects to dict format and merge with user-provided entities
+    entities_per_fact = []
+    for fact in facts:
+        # Start with LLM-extracted entities
+        llm_entities = [{"text": entity.name, "type": "CONCEPT"} for entity in (fact.entities or [])]
+
+        # Get user entities for this content (use content_index from fact)
+        user_entities = user_entities_per_content.get(fact.content_index, [])
+
+        # Merge with case-insensitive deduplication
+        seen_texts = {e["text"].lower() for e in llm_entities}
+        for user_entity in user_entities:
+            if user_entity["text"].lower() not in seen_texts:
+                llm_entities.append(
+                    {
+                        "text": user_entity["text"],
+                        "type": user_entity.get("type", "CONCEPT"),
+                    }
+                )
+                seen_texts.add(user_entity["text"].lower())
+
+        entities_per_fact.append(llm_entities)

    # Use existing link_utils function for entity processing
    entity_links = await link_utils.extract_entities_batch_optimized(
@@ -14,7 +14,47 @@ from typing import Literal

 from pydantic import BaseModel, ConfigDict, Field, field_validator

+from ...config import get_config
 from ..llm_wrapper import LLMConfig, OutputTooLongError
+from ..response_models import TokenUsage
+
+
+def _infer_temporal_date(fact_text: str, event_date: datetime) -> str | None:
+    """
+    Infer a temporal date from fact text when LLM didn't provide occurred_start.
+
+    This is a fallback for when the LLM fails to extract temporal information
+    from relative time expressions like "last night", "yesterday", etc.
+    """
+    import re
+
+    fact_lower = fact_text.lower()
+
+    # Map relative time expressions to day offsets
+    temporal_patterns = {
+        r"\blast night\b": -1,
+        r"\byesterday\b": -1,
+        r"\btoday\b": 0,
+        r"\bthis morning\b": 0,
+        r"\bthis afternoon\b": 0,
+        r"\bthis evening\b": 0,
+        r"\btonigh?t\b": 0,
+        r"\btomorrow\b": 1,
+        r"\blast week\b": -7,
+        r"\bthis week\b": 0,
+        r"\bnext week\b": 7,
+        r"\blast month\b": -30,
+        r"\bthis month\b": 0,
+        r"\bnext month\b": 30,
+    }
+
+    for pattern, offset_days in temporal_patterns.items():
+        if re.search(pattern, fact_lower):
+            target_date = event_date + timedelta(days=offset_days)
+            return target_date.replace(hour=0, minute=0, second=0, microsecond=0).isoformat()
+
+    # If no relative time expression found, return None
+    return None


 def _sanitize_text(text: str) -> str:
@@ -71,7 +111,7 @@ class Fact(BaseModel):


 class CausalRelation(BaseModel):
-    """Causal relationship between facts."""
+    """Causal relationship between facts (legacy - embedded in each fact)."""

    target_fact_index: int = Field(
        description="Index of the related fact in the facts array (0-based). "
@@ -93,6 +133,36 @@ class CausalRelation(BaseModel):
    )


+class TopLevelCausalRelation(BaseModel):
+    """
+    Causal relationship between two facts (top-level schema).
+
+    This is the preferred format - defined AFTER all facts are extracted,
+    allowing the LLM to see the full list of facts before specifying relationships.
+    """
+
+    from_fact_index: int = Field(
+        description="Index of the source fact (0-based). The fact that causes/enables/prevents."
+    )
+    to_fact_index: int = Field(
+        description="Index of the target fact (0-based). The fact that is caused/enabled/prevented."
+    )
+    relation_type: Literal["causes", "caused_by", "enables", "prevents"] = Field(
+        description="Type of causal relationship: "
+        "'causes' = source fact directly causes the target fact, "
+        "'caused_by' = source fact was caused by the target fact, "
+        "'enables' = source fact enables/allows the target fact, "
+        "'prevents' = source fact prevents/blocks the target fact"
+    )
+    strength: float = Field(
+        description="Strength of causal relationship (0.0 to 1.0). "
+        "1.0 = direct/strong causation, 0.5 = moderate, 0.3 = weak/indirect",
+        ge=0.0,
+        le=1.0,
+        default=1.0,
+    )
+
+
 class ExtractedFact(BaseModel):
    """A single extracted fact with 5 required dimensions for comprehensive capture."""

@@ -215,9 +285,15 @@ class ExtractedFact(BaseModel):


 class FactExtractionResponse(BaseModel):
-    """Response containing all extracted facts."""
+    """Response containing all extracted facts and their causal relationships."""

    facts: list[ExtractedFact] = Field(description="List of extracted factual statements")
+    causal_relationships: list[TopLevelCausalRelation] | None = Field(
+        default=None,
+        description="Causal relationships between facts. Define these AFTER listing all facts. "
+        "Each relationship specifies from_fact_index -> to_fact_index with a relation type. "
+        "Indices must be valid (0 to N-1 where N is the number of facts).",
+    )


 def chunk_text(text: str, max_chars: int) -> list[str]:
@@ -318,7 +394,7 @@ async def _extract_facts_from_chunk(
    llm_config: "LLMConfig",
    agent_name: str = None,
    extract_opinions: bool = False,
-) -> list[dict[str, str]]:
+) -> tuple[list[dict[str, str]], TokenUsage]:
    """
    Extract facts from a single chunk (internal helper for parallel processing).

@@ -534,7 +610,53 @@ WHAT TO EXTRACT vs SKIP
 ══════════════════════════════════════════════════════════════════════════

 ✅ EXTRACT: User preferences (ALWAYS as separate facts!), feelings, plans, events, relationships, achievements
-❌ SKIP: Greetings, filler ("thanks", "cool"), purely structural statements"""
+❌ SKIP: Greetings, filler ("thanks", "cool"), purely structural statements
+
+══════════════════════════════════════════════════════════════════════════
+CAUSAL RELATIONSHIPS (CRITICAL - DEFINE AFTER ALL FACTS)
+══════════════════════════════════════════════════════════════════════════
+
+⚠️ IMPORTANT: Causal relationships are defined at the TOP LEVEL, AFTER listing all facts!
+
+The `causal_relationships` array goes at the root of your response (NOT inside each fact).
+This allows you to see all facts first before defining how they relate.
+
+Format:
+```json
+{{
+  "facts": [...all your extracted facts...],
+  "causal_relationships": [
+    {{"from_fact_index": 0, "to_fact_index": 1, "relation_type": "causes", "strength": 0.9}},
+    {{"from_fact_index": 1, "to_fact_index": 2, "relation_type": "enables", "strength": 0.7}}
+  ]
+}}
+```
+
+Relationship types:
+- "causes": Fact A directly causes Fact B (A → B)
+- "caused_by": Fact A was caused by Fact B (A ← B)
+- "enables": Fact A enables/allows Fact B to happen
+- "prevents": Fact A prevents/blocks Fact B from happening
+
+⚠️ INDEX VALIDATION: If you extract N facts (indices 0 to N-1), both from_fact_index and to_fact_index MUST be in range [0, N-1].
+
+Example (Event Date: March 15, 2024):
+Input: "I lost my job in January. Because of that, I couldn't pay rent. So I had to move to a cheaper apartment."
+
+Facts extracted:
+- Fact 0: "User lost their job in January due to layoffs"
+- Fact 1: "User couldn't pay rent because of job loss"
+- Fact 2: "User moved to a cheaper apartment"
+
+Causal relationships (at root level):
+```json
+"causal_relationships": [
+  {{"from_fact_index": 0, "to_fact_index": 1, "relation_type": "causes", "strength": 1.0}},
+  {{"from_fact_index": 1, "to_fact_index": 2, "relation_type": "causes", "strength": 0.9}}
+]
+```
+
+This creates a chain: Job loss (0) → Can't pay rent (1) → Moved to cheaper apartment (2)"""

    import logging

@@ -545,6 +667,7 @@ WHAT TO EXTRACT vs SKIP
    # Retry logic for JSON validation errors
    max_retries = 2
    last_error = None
+    config = get_config()

    # Sanitize input text to prevent Unicode encoding errors (e.g., unpaired surrogates)
    sanitized_chunk = _sanitize_text(chunk)
@@ -563,16 +686,19 @@ Context: {sanitized_context}
 Text:
 {sanitized_chunk}"""

+    usage = TokenUsage()  # Track cumulative usage across retries
    for attempt in range(max_retries):
        try:
-            extraction_response_json = await llm_config.call(
+            extraction_response_json, call_usage = await llm_config.call(
                messages=[{"role": "system", "content": prompt}, {"role": "user", "content": user_message}],
                response_format=FactExtractionResponse,
                scope="memory_extract_facts",
                temperature=0.1,
-                max_completion_tokens=65000,
+                max_completion_tokens=config.retain_max_completion_tokens,
                skip_validation=True,  # Get raw JSON, we'll validate leniently
+                return_usage=True,
            )
+            usage = usage + call_usage  # Aggregate usage across retries

            # Lenient parsing of facts from raw JSON
            chunk_facts = []
@@ -590,9 +716,12 @@ Text:
                        f"LLM returned non-dict JSON after {max_retries} attempts: {type(extraction_response_json).__name__}. "
                        f"Raw: {str(extraction_response_json)[:500]}"
                    )
-                    return []
+                    return [], usage

            raw_facts = extraction_response_json.get("facts", [])
+            # Get top-level causal relationships (new schema)
+            top_level_causal_relations = extraction_response_json.get("causal_relationships", [])
+
            if not raw_facts:
                logger.debug(
                    f"LLM response missing 'facts' field or returned empty list. "
@@ -603,6 +732,47 @@ Text:
                    f"text: {chunk}"
                )

+            # Build a map from fact index to causal relations (from top-level field)
+            # This converts from_fact_index -> [{target_fact_index, relation_type, strength}]
+            causal_relations_by_fact: dict[int, list[dict]] = {}
+            if top_level_causal_relations:
+                num_facts = len(raw_facts)
+                for rel in top_level_causal_relations:
+                    if not isinstance(rel, dict):
+                        continue
+                    from_idx = rel.get("from_fact_index")
+                    to_idx = rel.get("to_fact_index")
+                    relation_type = rel.get("relation_type")
+                    strength = rel.get("strength", 1.0)
+
+                    # Validate indices
+                    if from_idx is None or to_idx is None or relation_type is None:
+                        logger.warning(f"Skipping malformed top-level causal relation: {rel}")
+                        continue
+                    if from_idx < 0 or from_idx >= num_facts:
+                        logger.warning(
+                            f"Invalid from_fact_index {from_idx} in top-level causal relation "
+                            f"(valid range: 0-{num_facts - 1}). Skipping."
+                        )
+                        continue
+                    if to_idx < 0 or to_idx >= num_facts:
+                        logger.warning(
+                            f"Invalid to_fact_index {to_idx} in top-level causal relation "
+                            f"(valid range: 0-{num_facts - 1}). Skipping."
+                        )
+                        continue
+
+                    # Add to the map for the from_fact_index
+                    if from_idx not in causal_relations_by_fact:
+                        causal_relations_by_fact[from_idx] = []
+                    causal_relations_by_fact[from_idx].append(
+                        {
+                            "target_fact_index": to_idx,
+                            "relation_type": relation_type,
+                            "strength": strength,
+                        }
+                    )
+
            for i, llm_fact in enumerate(raw_facts):
                # Skip non-dict entries but track them for retry
                if not isinstance(llm_fact, dict):
@@ -676,13 +846,18 @@ Text:
                if fact_kind == "event":
                    occurred_start = get_value("occurred_start")
                    occurred_end = get_value("occurred_end")
-                    if occurred_start:
+
+                    # If LLM didn't set temporal fields, try to extract them from the fact text
+                    if not occurred_start:
+                        fact_data["occurred_start"] = _infer_temporal_date(combined_text, event_date)
+                    else:
                        fact_data["occurred_start"] = occurred_start
-                        # For point events: if occurred_end not set, default to occurred_start
-                        if occurred_end:
-                            fact_data["occurred_end"] = occurred_end
-                        else:
-                            fact_data["occurred_end"] = occurred_start
+
+                    # For point events: if occurred_end not set, default to occurred_start
+                    if occurred_end:
+                        fact_data["occurred_end"] = occurred_end
+                    elif fact_data.get("occurred_start"):
+                        fact_data["occurred_end"] = fact_data["occurred_start"]

                # Add entities if present (validate as Entity objects)
                # LLM sometimes returns strings instead of {"text": "..."} format
@@ -702,19 +877,40 @@ Text:
                    if validated_entities:
                        fact_data["entities"] = validated_entities

-                # Add causal relations if present (validate as CausalRelation objects)
-                # Filter out invalid relations (missing required fields)
-                causal_relations = get_value("causal_relations")
-                if causal_relations:
-                    validated_relations = []
-                    for rel in causal_relations:
+                # Add causal relations from both sources:
+                # 1. Top-level causal_relationships (preferred, new schema)
+                # 2. Per-fact causal_relations (legacy, for backward compatibility)
+                validated_relations = []
+
+                # First, add relations from top-level (already validated above)
+                if i in causal_relations_by_fact:
+                    for rel in causal_relations_by_fact[i]:
+                        try:
+                            validated_relations.append(CausalRelation.model_validate(rel))
+                        except Exception as e:
+                            logger.warning(f"Invalid top-level causal relation for fact {i}: {rel}: {e}")
+
+                # Then, add any legacy per-fact relations (with index validation)
+                legacy_causal_relations = get_value("causal_relations")
+                if legacy_causal_relations:
+                    num_facts = len(raw_facts)
+                    for rel in legacy_causal_relations:
                        if isinstance(rel, dict) and "target_fact_index" in rel and "relation_type" in rel:
-                            try:
-                                validated_relations.append(CausalRelation.model_validate(rel))
-                            except Exception as e:
-                                logger.warning(f"Invalid causal relation {rel}: {e}")
-                    if validated_relations:
-                        fact_data["causal_relations"] = validated_relations
+                            target_idx = rel.get("target_fact_index")
+                            # Validate target index for legacy format too
+                            if target_idx is not None and 0 <= target_idx < num_facts:
+                                try:
+                                    validated_relations.append(CausalRelation.model_validate(rel))
+                                except Exception as e:
+                                    logger.warning(f"Invalid causal relation {rel}: {e}")
+                            else:
+                                logger.warning(
+                                    f"Invalid target_fact_index {target_idx} in per-fact causal relation "
+                                    f"from fact {i} (valid range: 0-{num_facts - 1}). Skipping."
+                                )
+
+                if validated_relations:
+                    fact_data["causal_relations"] = validated_relations

                # Always set mentioned_at to the event_date (when the conversation/document occurred)
                fact_data["mentioned_at"] = event_date.isoformat()
@@ -735,7 +931,7 @@ Text:
                )
                continue

-            return chunk_facts
+            return chunk_facts, usage

        except BadRequestError as e:
            last_error = e
@@ -762,7 +958,7 @@ async def _extract_facts_with_auto_split(
    llm_config: LLMConfig,
    agent_name: str = None,
    extract_opinions: bool = False,
-) -> list[dict[str, str]]:
+) -> tuple[list[dict[str, str]], TokenUsage]:
    """
    Extract facts from a chunk with automatic splitting if output exceeds token limits.

@@ -780,7 +976,7 @@ async def _extract_facts_with_auto_split(
        extract_opinions: If True, extract ONLY opinions. If False, extract world and agent facts (no opinions)

    Returns:
-        List of fact dictionaries extracted from the chunk (possibly from sub-chunks)
+        Tuple of (facts list, token usage) extracted from the chunk (possibly from sub-chunks)
    """
    import logging

@@ -859,12 +1055,14 @@ async def _extract_facts_with_auto_split(

        # Combine results from both halves
        all_facts = []
-        for sub_result in sub_results:
-            all_facts.extend(sub_result)
+        total_usage = TokenUsage()
+        for sub_facts, sub_usage in sub_results:
+            all_facts.extend(sub_facts)
+            total_usage = total_usage + sub_usage

        logger.info(f"Successfully extracted {len(all_facts)} facts from split chunk {chunk_index + 1}")

-        return all_facts
+        return all_facts, total_usage


 async def extract_facts_from_text(
@@ -874,7 +1072,7 @@ async def extract_facts_from_text(
    agent_name: str,
    context: str = "",
    extract_opinions: bool = False,
-) -> tuple[list[Fact], list[tuple[str, int]]]:
+) -> tuple[list[Fact], list[tuple[str, int]], TokenUsage]:
    """
    Extract semantic facts from conversational or narrative text using LLM.

@@ -893,9 +1091,10 @@ async def extract_facts_from_text(
        extract_opinions: If True, extract ONLY opinions. If False, extract world and bank facts (no opinions)

    Returns:
-        Tuple of (facts, chunks) where:
+        Tuple of (facts, chunks, usage) where:
        - facts: List of Fact model instances
        - chunks: List of tuples (chunk_text, fact_count) for each chunk
+        - usage: Aggregated token usage across all LLM calls
    """
    chunks = chunk_text(text, max_chars=3000)
    tasks = [
@@ -914,10 +1113,12 @@ async def extract_facts_from_text(
    chunk_results = await asyncio.gather(*tasks)
    all_facts = []
    chunk_metadata = []  # [(chunk_text, fact_count), ...]
-    for chunk, chunk_facts in zip(chunks, chunk_results):
+    total_usage = TokenUsage()
+    for chunk, (chunk_facts, chunk_usage) in zip(chunks, chunk_results):
        all_facts.extend(chunk_facts)
        chunk_metadata.append((chunk, len(chunk_facts)))
-    return all_facts, chunk_metadata
+        total_usage = total_usage + chunk_usage
+    return all_facts, chunk_metadata, total_usage


 # ============================================================================
@@ -938,7 +1139,7 @@ SECONDS_PER_FACT = 10

 async def extract_facts_from_contents(
    contents: list[RetainContent], llm_config, agent_name: str, extract_opinions: bool = False
-) -> tuple[list[ExtractedFactType], list[ChunkMetadata]]:
+) -> tuple[list[ExtractedFactType], list[ChunkMetadata], TokenUsage]:
    """
    Extract facts from multiple content items in parallel.

@@ -955,10 +1156,10 @@ async def extract_facts_from_contents(
        extract_opinions: If True, extract only opinions; otherwise world/bank facts

    Returns:
-        Tuple of (extracted_facts, chunks_metadata)
+        Tuple of (extracted_facts, chunks_metadata, usage)
    """
    if not contents:
-        return [], []
+        return [], [], TokenUsage()

    # Step 1: Create parallel fact extraction tasks
    fact_extraction_tasks = []
@@ -981,11 +1182,15 @@ async def extract_facts_from_contents(
    # Step 3: Flatten and convert to typed objects
    extracted_facts: list[ExtractedFactType] = []
    chunks_metadata: list[ChunkMetadata] = []
+    total_usage = TokenUsage()

    global_chunk_idx = 0
    global_fact_idx = 0

-    for content_index, (content, (facts_from_llm, chunks_from_llm)) in enumerate(zip(contents, all_fact_results)):
+    for content_index, (content, (facts_from_llm, chunks_from_llm, content_usage)) in enumerate(
+        zip(contents, all_fact_results)
+    ):
+        total_usage = total_usage + content_usage
        chunk_start_idx = global_chunk_idx

        # Convert chunk tuples to ChunkMetadata objects
@@ -1039,7 +1244,7 @@ async def extract_facts_from_contents(
    # Step 4: Add time offsets to preserve ordering within each content
    _add_temporal_offsets(extracted_facts, contents)

-    return extracted_facts, chunks_metadata
+    return extracted_facts, chunks_metadata, total_usage


 def _parse_datetime(date_str: str):
@@ -9,6 +9,7 @@ import time
 import uuid
 from datetime import UTC, datetime

+from ...config import get_config
 from ..memory_engine import fq_table
 from ..search import observation_utils
 from . import embedding_utils
@@ -49,8 +50,9 @@ async def regenerate_observations_batch(
        entity_links: Entity links from this batch
        log_buffer: Optional log buffer for timing
    """
-    TOP_N_ENTITIES = 5
-    MIN_FACTS_THRESHOLD = 5
+    config = get_config()
+    TOP_N_ENTITIES = config.observation_top_entities
+    MIN_FACTS_THRESHOLD = config.observation_min_facts

    if not entity_links:
        return
@@ -18,6 +18,7 @@ def utcnow():
    return datetime.now(UTC)


+from ..response_models import TokenUsage
 from . import (
    chunk_storage,
    deduplication,
@@ -47,7 +48,7 @@ async def retain_batch(
    is_first_batch: bool = True,
    fact_type_override: str | None = None,
    confidence_score: float | None = None,
-) -> list[list[str]]:
+) -> tuple[list[list[str]], TokenUsage]:
    """
    Process a batch of content through the retain pipeline.

@@ -67,7 +68,7 @@ async def retain_batch(
        confidence_score: Confidence score for opinions

    Returns:
-        List of unit ID lists (one list per content item)
+        Tuple of (unit ID lists, token usage for fact extraction)
    """
    start_time = time.time()
    total_chars = sum(len(item.get("content", "")) for item in contents_dicts)
@@ -91,6 +92,7 @@ async def retain_batch(
            context=item.get("context", ""),
            event_date=item.get("event_date") or utcnow(),
            metadata=item.get("metadata", {}),
+            entities=item.get("entities", []),
        )
        contents.append(content)

@@ -98,7 +100,7 @@ async def retain_batch(
    step_start = time.time()
    extract_opinions = fact_type_override == "opinion"

-    extracted_facts, chunks = await fact_extraction.extract_facts_from_contents(
+    extracted_facts, chunks, usage = await fact_extraction.extract_facts_from_contents(
        contents, llm_config, agent_name, extract_opinions
    )
    log_buffer.append(
@@ -106,11 +108,64 @@ async def retain_batch(
    )

    if not extracted_facts:
+        # Still need to create document if document_id was provided
+        async with acquire_with_retry(pool) as conn:
+            async with conn.transaction():
+                await fact_storage.ensure_bank_exists(conn, bank_id)
+
+                # Handle document tracking even with no facts
+                if document_id:
+                    combined_content = "\n".join([c.get("content", "") for c in contents_dicts])
+                    retain_params = {}
+                    if contents_dicts:
+                        first_item = contents_dicts[0]
+                        if first_item.get("context"):
+                            retain_params["context"] = first_item["context"]
+                        if first_item.get("event_date"):
+                            retain_params["event_date"] = (
+                                first_item["event_date"].isoformat()
+                                if hasattr(first_item["event_date"], "isoformat")
+                                else str(first_item["event_date"])
+                            )
+                        if first_item.get("metadata"):
+                            retain_params["metadata"] = first_item["metadata"]
+                    await fact_storage.handle_document_tracking(
+                        conn, bank_id, document_id, combined_content, is_first_batch, retain_params
+                    )
+                else:
+                    # Check for per-item document_ids
+                    from collections import defaultdict
+
+                    contents_by_doc = defaultdict(list)
+                    for idx, content_dict in enumerate(contents_dicts):
+                        doc_id = content_dict.get("document_id")
+                        if doc_id:
+                            contents_by_doc[doc_id].append((idx, content_dict))
+
+                    for doc_id, doc_contents in contents_by_doc.items():
+                        combined_content = "\n".join([c.get("content", "") for _, c in doc_contents])
+                        retain_params = {}
+                        if doc_contents:
+                            first_item = doc_contents[0][1]
+                            if first_item.get("context"):
+                                retain_params["context"] = first_item["context"]
+                            if first_item.get("event_date"):
+                                retain_params["event_date"] = (
+                                    first_item["event_date"].isoformat()
+                                    if hasattr(first_item["event_date"], "isoformat")
+                                    else str(first_item["event_date"])
+                                )
+                            if first_item.get("metadata"):
+                                retain_params["metadata"] = first_item["metadata"]
+                        await fact_storage.handle_document_tracking(
+                            conn, bank_id, doc_id, combined_content, is_first_batch, retain_params
+                        )
+
        total_time = time.time() - start_time
        logger.info(
-            f"RETAIN_BATCH COMPLETE: 0 facts extracted from {len(contents)} contents in {total_time:.3f}s (nothing to store)"
+            f"RETAIN_BATCH COMPLETE: 0 facts extracted from {len(contents)} contents in {total_time:.3f}s (document tracked, no facts)"
        )
-        return [[] for _ in contents]
+        return [[] for _ in contents], usage

    # Apply fact_type_override if provided
    if fact_type_override:
@@ -290,7 +345,7 @@ async def retain_batch(
            non_duplicate_facts = deduplication.filter_duplicates(processed_facts, is_duplicate_flags)

            if not non_duplicate_facts:
-                return [[] for _ in contents]
+                return [[] for _ in contents], usage

            # Insert facts (document_id is now stored per-fact)
            step_start = time.time()
@@ -299,8 +354,18 @@ async def retain_batch(

            # Process entities
            step_start = time.time()
+            # Build map of content_index -> user entities for merging
+            user_entities_per_content = {
+                idx: content.entities for idx, content in enumerate(contents) if content.entities
+            }
            entity_links = await entity_processing.process_entities_batch(
-                entity_resolver, conn, bank_id, unit_ids, non_duplicate_facts, log_buffer
+                entity_resolver,
+                conn,
+                bank_id,
+                unit_ids,
+                non_duplicate_facts,
+                log_buffer,
+                user_entities_per_content=user_entities_per_content,
            )
            log_buffer.append(f"[6] Process entities: {len(entity_links)} links in {time.time() - step_start:.3f}s")

@@ -351,7 +416,7 @@ async def retain_batch(

        logger.info("\n" + "\n".join(log_buffer) + "\n")

-        return result_unit_ids
+        return result_unit_ids, usage


 def _map_results_to_contents(
@@ -20,6 +20,7 @@ class RetainContentDict(TypedDict, total=False):
        event_date: When the content occurred (optional, defaults to now)
        metadata: Custom key-value metadata (optional)
        document_id: Document ID for this content item (optional)
+        entities: User-provided entities to merge with extracted entities (optional)
    """

    content: str  # Required
@@ -27,6 +28,7 @@ class RetainContentDict(TypedDict, total=False):
    event_date: datetime
    metadata: dict[str, str]
    document_id: str
+    entities: list[dict[str, str]]  # [{"text": "...", "type": "..."}]


 def _now_utc() -> datetime:
@@ -46,6 +48,7 @@ class RetainContent:
    context: str = ""
    event_date: datetime = field(default_factory=_now_utc)
    metadata: dict[str, str] = field(default_factory=dict)
+    entities: list[dict[str, str]] = field(default_factory=list)  # User-provided entities


@dataclass
@@ -152,6 +155,9 @@ class ProcessedFact:
    # DB fields (set after insertion)
    unit_id: UUID | None = None

+    # Track which content this fact came from (for user entity merging)
+    content_index: int = 0
+
    @property
    def is_duplicate(self) -> bool:
        """Check if this fact was marked as a duplicate."""
@@ -194,6 +200,7 @@ class ProcessedFact:
            entities=entities,
            causal_relations=extracted_fact.causal_relations,
            chunk_id=chunk_id,
+            content_index=extracted_fact.content_index,
        )


@@ -26,6 +26,23 @@ class CrossEncoderReranker:

            cross_encoder = create_cross_encoder_from_env()
        self.cross_encoder = cross_encoder
+        self._initialized = False
+
+    async def ensure_initialized(self):
+        """Ensure the cross-encoder model is initialized (for lazy initialization)."""
+        if self._initialized:
+            return
+
+        import asyncio
+
+        cross_encoder = self.cross_encoder
+        # For local providers, run in thread pool to avoid blocking event loop
+        if cross_encoder.provider_name == "local":
+            loop = asyncio.get_event_loop()
+            await loop.run_in_executor(None, lambda: asyncio.run(cross_encoder.initialize()))
+        else:
+            await cross_encoder.initialize()
+        self._initialized = True

    def rerank(self, query: str, candidates: list[MergedCandidate]) -> list[ScoredResult]:
        """
@@ -49,7 +49,7 @@ async def extract_facts(
    if not text or not text.strip():
        return [], []

-    facts, chunks = await extract_facts_from_text(
+    facts, chunks, _ = await extract_facts_from_text(
        text,
        event_date,
        context=context,
@@ -98,7 +98,14 @@ class DefaultExtensionContext(ExtensionContext):
        """Run migrations for a specific schema."""
        from hindsight_api.migrations import run_migrations

-        run_migrations(self._database_url, schema=schema)
+        # Prefer getting URL from memory engine (handles pg0 case where URL is set after init)
+        db_url = self._database_url
+        if self._memory_engine is not None:
+            engine_url = getattr(self._memory_engine, "db_url", None)
+            if engine_url:
+                db_url = engine_url
+
+        run_migrations(db_url, schema=schema)

    def get_memory_engine(self) -> "MemoryEngineInterface":
        """Get the memory engine interface."""
@@ -17,8 +17,9 @@ if TYPE_CHECKING:
 class OperationValidationError(Exception):
    """Raised when an operation fails validation."""

-    def __init__(self, reason: str):
+    def __init__(self, reason: str, status_code: int = 403):
        self.reason = reason
+        self.status_code = status_code
        super().__init__(f"Operation validation failed: {reason}")


@@ -28,6 +29,7 @@ class ValidationResult:

    allowed: bool
    reason: str | None = None
+    status_code: int = 403  # Default to Forbidden

    @classmethod
    def accept(cls) -> "ValidationResult":
@@ -35,9 +37,9 @@ class ValidationResult:
        return cls(allowed=True)

    @classmethod
-    def reject(cls, reason: str) -> "ValidationResult":
-        """Create a rejected validation result with a reason."""
-        return cls(allowed=False, reason=reason)
+    def reject(cls, reason: str, status_code: int = 403) -> "ValidationResult":
+        """Create a rejected validation result with a reason and HTTP status code."""
+        return cls(allowed=False, reason=reason, status_code=status_code)


 # =============================================================================
@@ -4,6 +4,9 @@ Command-line interface for Hindsight API.
 Run the server with:
    hindsight-api

+Run as background daemon:
+    hindsight-api --daemon
+
 Stop with Ctrl+C.
 """

@@ -21,9 +24,14 @@ from . import MemoryEngine
 from .api import create_app
 from .banner import print_banner
 from .config import HindsightConfig, get_config
-
-print()
-print_banner()
+from .daemon import (
+    DEFAULT_DAEMON_PORT,
+    DEFAULT_IDLE_TIMEOUT,
+    DaemonLock,
+    IdleTimeoutMiddleware,
+    daemonize,
+)
+from .extensions import DefaultExtensionContext, OperationValidatorExtension, TenantExtension, load_extension

 # Filter deprecation warnings from third-party libraries
 warnings.filterwarnings("ignore", message="websockets.legacy is deprecated")
@@ -106,8 +114,52 @@ def main():
    parser.add_argument("--ssl-keyfile", default=None, help="SSL key file")
    parser.add_argument("--ssl-certfile", default=None, help="SSL certificate file")

+    # Daemon mode options
+    parser.add_argument(
+        "--daemon",
+        action="store_true",
+        help=f"Run as background daemon (uses port {DEFAULT_DAEMON_PORT}, auto-exits after idle)",
+    )
+    parser.add_argument(
+        "--idle-timeout",
+        type=int,
+        default=DEFAULT_IDLE_TIMEOUT,
+        help=f"Idle timeout in seconds before auto-exit in daemon mode (default: {DEFAULT_IDLE_TIMEOUT})",
+    )
+
    args = parser.parse_args()

+    # Daemon mode handling
+    if args.daemon:
+        # Use fixed daemon port
+        args.port = DEFAULT_DAEMON_PORT
+        args.host = "127.0.0.1"  # Only bind to localhost for security
+
+        # Check if another daemon is already running
+        daemon_lock = DaemonLock()
+        if not daemon_lock.acquire():
+            print(f"Daemon already running (PID: {daemon_lock.get_pid()})", file=sys.stderr)
+            sys.exit(1)
+
+        # Fork into background
+        daemonize()
+
+        # Re-acquire lock in child process
+        daemon_lock = DaemonLock()
+        if not daemon_lock.acquire():
+            sys.exit(1)
+
+        # Register cleanup to release lock
+        def release_lock():
+            daemon_lock.release()
+
+        atexit.register(release_lock)
+
+    # Print banner (not in daemon mode)
+    if not args.daemon:
+        print()
+        print_banner()
+
    # Configure Python logging based on log level
    # Update config with CLI override if provided
    if args.log_level != config.log_level:
@@ -117,6 +169,8 @@ def main():
            llm_api_key=config.llm_api_key,
            llm_model=config.llm_model,
            llm_base_url=config.llm_base_url,
+            llm_max_concurrent=config.llm_max_concurrent,
+            llm_timeout=config.llm_timeout,
            embeddings_provider=config.embeddings_provider,
            embeddings_local_model=config.embeddings_local_model,
            embeddings_tei_url=config.embeddings_tei_url,
@@ -128,17 +182,51 @@ def main():
            log_level=args.log_level,
            mcp_enabled=config.mcp_enabled,
            graph_retriever=config.graph_retriever,
+            observation_min_facts=config.observation_min_facts,
+            observation_top_entities=config.observation_top_entities,
+            retain_max_completion_tokens=config.retain_max_completion_tokens,
+            skip_llm_verification=config.skip_llm_verification,
+            lazy_reranker=config.lazy_reranker,
+            run_migrations_on_startup=config.run_migrations_on_startup,
        )
    config.configure_logging()
-    config.log_config()
+    if not args.daemon:
+        config.log_config()

    # Register cleanup handlers
    atexit.register(_cleanup)
    signal.signal(signal.SIGINT, _signal_handler)
    signal.signal(signal.SIGTERM, _signal_handler)

+    # Load operation validator extension if configured
+    operation_validator = load_extension("OPERATION_VALIDATOR", OperationValidatorExtension)
+    if operation_validator:
+        import logging
+
+        logging.info(f"Loaded operation validator: {operation_validator.__class__.__name__}")
+
+    # Load tenant extension if configured
+    tenant_extension = load_extension("TENANT", TenantExtension)
+    if tenant_extension:
+        import logging
+
+        logging.info(f"Loaded tenant extension: {tenant_extension.__class__.__name__}")
+
    # Create MemoryEngine (reads configuration from environment)
-    _memory = MemoryEngine()
+    _memory = MemoryEngine(
+        operation_validator=operation_validator,
+        tenant_extension=tenant_extension,
+        run_migrations=config.run_migrations_on_startup,
+    )
+
+    # Set extension context on tenant extension (needed for schema provisioning)
+    if tenant_extension:
+        extension_context = DefaultExtensionContext(
+            database_url=config.database_url,
+            memory_engine=_memory,
+        )
+        tenant_extension.set_context(extension_context)
+        logging.info("Extension context set on tenant extension")

    # Create FastAPI app
    app = create_app(
@@ -149,6 +237,12 @@ def main():
        initialize_memory=True,
    )

+    # Wrap with idle timeout middleware in daemon mode
+    idle_middleware = None
+    if args.daemon:
+        idle_middleware = IdleTimeoutMiddleware(app, idle_timeout=args.idle_timeout)
+        app = idle_middleware
+
    # Prepare uvicorn config
    uvicorn_config = {
        "app": app,
@@ -172,18 +266,38 @@ def main():
    if args.ssl_certfile:
        uvicorn_config["ssl_certfile"] = args.ssl_certfile

-    from .banner import print_startup_info
+    # Print startup info (not in daemon mode)
+    if not args.daemon:
+        from .banner import print_startup_info

-    print_startup_info(
-        host=args.host,
-        port=args.port,
-        database_url=config.database_url,
-        llm_provider=config.llm_provider,
-        llm_model=config.llm_model,
-        embeddings_provider=config.embeddings_provider,
-        reranker_provider=config.reranker_provider,
-        mcp_enabled=config.mcp_enabled,
-    )
+        print_startup_info(
+            host=args.host,
+            port=args.port,
+            database_url=config.database_url,
+            llm_provider=config.llm_provider,
+            llm_model=config.llm_model,
+            embeddings_provider=config.embeddings_provider,
+            reranker_provider=config.reranker_provider,
+            mcp_enabled=config.mcp_enabled,
+        )
+
+    # Start idle checker in daemon mode
+    if idle_middleware is not None:
+        # Start the idle checker in a background thread with its own event loop
+        import threading
+
+        def run_idle_checker():
+            import time
+
+            time.sleep(2)  # Wait for uvicorn to start
+            try:
+                loop = asyncio.new_event_loop()
+                asyncio.set_event_loop(loop)
+                loop.run_until_complete(idle_middleware._check_idle())
+            except Exception:
+                pass
+
+        threading.Thread(target=run_idle_checker, daemon=True).start()

    uvicorn.run(**uvicorn_config)  # type: ignore[invalid-argument-type] - dict kwargs

@@ -5,6 +5,7 @@ This module provides metrics for:
 - Operation latency (retain, recall, reflect) with percentiles
 - Token usage (input/output) per operation
 - Per-bank granularity via labels
+- LLM call latency and token usage with scope dimension
 """

 import logging
@@ -14,8 +15,54 @@ from contextlib import contextmanager
 from opentelemetry import metrics
 from opentelemetry.exporter.prometheus import PrometheusMetricReader
 from opentelemetry.sdk.metrics import MeterProvider
+from opentelemetry.sdk.metrics.view import ExplicitBucketHistogramAggregation, View
 from opentelemetry.sdk.resources import Resource

+# Custom bucket boundaries for operation duration (in seconds)
+# Fine granularity in 0-30s range where most operations complete
+DURATION_BUCKETS = (0.1, 0.25, 0.5, 0.75, 1.0, 2.0, 3.0, 5.0, 7.5, 10.0, 15.0, 20.0, 30.0, 60.0, 120.0)
+
+# LLM duration buckets (finer granularity for faster LLM calls)
+LLM_DURATION_BUCKETS = (0.1, 0.25, 0.5, 1.0, 2.0, 3.0, 5.0, 10.0, 15.0, 30.0, 60.0, 120.0)
+
+
+def get_token_bucket(token_count: int) -> str:
+    """
+    Convert a token count to a bucket label for use as a dimension.
+
+    This allows analyzing token usage patterns without high-cardinality issues.
+
+    Buckets:
+    - "0-100": Very small requests/responses
+    - "100-500": Small requests/responses
+    - "500-1k": Medium requests/responses
+    - "1k-5k": Large requests/responses
+    - "5k-10k": Very large requests/responses
+    - "10k-50k": Huge requests/responses
+    - "50k+": Extremely large requests/responses
+
+    Args:
+        token_count: Number of tokens
+
+    Returns:
+        Bucket label string
+    """
+    if token_count < 100:
+        return "0-100"
+    elif token_count < 500:
+        return "100-500"
+    elif token_count < 1000:
+        return "500-1k"
+    elif token_count < 5000:
+        return "1k-5k"
+    elif token_count < 10000:
+        return "5k-10k"
+    elif token_count < 50000:
+        return "10k-50k"
+    else:
+        return "50k+"
+
+
 logger = logging.getLogger(__name__)

 # Global meter instance
@@ -48,8 +95,22 @@ def initialize_metrics(service_name: str = "hindsight-api", service_version: str
    # Create Prometheus metric reader
    prometheus_reader = PrometheusMetricReader()

-    # Create meter provider with Prometheus exporter
-    provider = MeterProvider(resource=resource, metric_readers=[prometheus_reader])
+    # Create view with custom bucket boundaries for duration histogram
+    duration_view = View(
+        instrument_name="hindsight.operation.duration",
+        aggregation=ExplicitBucketHistogramAggregation(boundaries=DURATION_BUCKETS),
+    )
+
+    # Create view with custom bucket boundaries for LLM duration histogram
+    llm_duration_view = View(
+        instrument_name="hindsight.llm.duration",
+        aggregation=ExplicitBucketHistogramAggregation(boundaries=LLM_DURATION_BUCKETS),
+    )
+
+    # Create meter provider with Prometheus exporter and custom views
+    provider = MeterProvider(
+        resource=resource, metric_readers=[prometheus_reader], views=[duration_view, llm_duration_view]
+    )

    # Set the global meter provider
    metrics.set_meter_provider(provider)
@@ -71,20 +132,39 @@ class MetricsCollectorBase:
    """Base class for metrics collectors."""

    @contextmanager
-    def record_operation(self, operation: str, bank_id: str, budget: str | None = None, max_tokens: int | None = None):
-        """Context manager to record operation duration and status."""
-        raise NotImplementedError
-
-    def record_tokens(
+    def record_operation(
        self,
        operation: str,
        bank_id: str,
-        input_tokens: int = 0,
-        output_tokens: int = 0,
+        source: str = "api",
        budget: str | None = None,
        max_tokens: int | None = None,
    ):
-        """Record token usage for an operation."""
+        """Context manager to record operation duration and status."""
+        raise NotImplementedError
+
+    def record_llm_call(
+        self,
+        provider: str,
+        model: str,
+        scope: str,
+        duration: float,
+        input_tokens: int = 0,
+        output_tokens: int = 0,
+        success: bool = True,
+    ):
+        """
+        Record metrics for an LLM call.
+
+        Args:
+            provider: LLM provider name (openai, anthropic, gemini, groq, ollama, lmstudio)
+            model: Model name
+            scope: Scope identifier (e.g., "memory", "reflect", "entity_observation")
+            duration: Call duration in seconds
+            input_tokens: Number of input/prompt tokens
+            output_tokens: Number of output/completion tokens
+            success: Whether the call was successful
+        """
        raise NotImplementedError


@@ -92,20 +172,28 @@ class NoOpMetricsCollector(MetricsCollectorBase):
    """No-op metrics collector that does nothing. Used when metrics are disabled."""

    @contextmanager
-    def record_operation(self, operation: str, bank_id: str, budget: str | None = None, max_tokens: int | None = None):
-        """No-op context manager."""
-        yield
-
-    def record_tokens(
+    def record_operation(
        self,
        operation: str,
        bank_id: str,
-        input_tokens: int = 0,
-        output_tokens: int = 0,
+        source: str = "api",
        budget: str | None = None,
        max_tokens: int | None = None,
    ):
-        """No-op token recording."""
+        """No-op context manager."""
+        yield
+
+    def record_llm_call(
+        self,
+        provider: str,
+        model: str,
+        scope: str,
+        duration: float,
+        input_tokens: int = 0,
+        output_tokens: int = 0,
+        success: bool = True,
+    ):
+        """No-op LLM call recording."""
        pass


@@ -125,33 +213,52 @@ class MetricsCollector(MetricsCollectorBase):
            name="hindsight.operation.duration", description="Duration of Hindsight operations in seconds", unit="s"
        )

-        # Token usage counters
-        self.tokens_input = self.meter.create_counter(
-            name="hindsight.tokens.input", description="Number of input tokens consumed", unit="tokens"
-        )
-
-        self.tokens_output = self.meter.create_counter(
-            name="hindsight.tokens.output", description="Number of output tokens generated", unit="tokens"
-        )
-
        # Operation counter (success/failure)
        self.operation_total = self.meter.create_counter(
            name="hindsight.operation.total", description="Total number of operations executed", unit="operations"
        )

+        # LLM call latency histogram (in seconds)
+        # Records duration of LLM API calls with provider, model, and scope dimensions
+        self.llm_duration = self.meter.create_histogram(
+            name="hindsight.llm.duration", description="Duration of LLM API calls in seconds", unit="s"
+        )
+
+        # LLM token usage counters with bucket labels
+        self.llm_tokens_input = self.meter.create_counter(
+            name="hindsight.llm.tokens.input", description="Number of input tokens for LLM calls", unit="tokens"
+        )
+
+        self.llm_tokens_output = self.meter.create_counter(
+            name="hindsight.llm.tokens.output", description="Number of output tokens from LLM calls", unit="tokens"
+        )
+
+        # LLM call counter (success/failure)
+        self.llm_calls_total = self.meter.create_counter(
+            name="hindsight.llm.calls.total", description="Total number of LLM API calls", unit="calls"
+        )
+
    @contextmanager
-    def record_operation(self, operation: str, bank_id: str, budget: str | None = None, max_tokens: int | None = None):
+    def record_operation(
+        self,
+        operation: str,
+        bank_id: str,
+        source: str = "api",
+        budget: str | None = None,
+        max_tokens: int | None = None,
+    ):
        """
        Context manager to record operation duration and status.

        Usage:
-            with metrics.record_operation("recall", bank_id="user123", budget="mid", max_tokens=4096):
+            with metrics.record_operation("recall", bank_id="user123", source="api", budget="mid", max_tokens=4096):
                # ... perform operation
                pass

        Args:
-            operation: Operation name (retain, recall, reflect)
+            operation: Operation name (retain, recall, reflect, entity_observation)
            bank_id: Memory bank ID
+            source: Source of the operation (api, reflect, internal)
            budget: Optional budget level (low, mid, high)
            max_tokens: Optional max tokens for the operation
        """
@@ -159,6 +266,7 @@ class MetricsCollector(MetricsCollectorBase):
        attributes = {
            "operation": operation,
            "bank_id": bank_id,
+            "source": source,
        }
        if budget:
            attributes["budget"] = budget
@@ -181,40 +289,56 @@ class MetricsCollector(MetricsCollectorBase):
            # Record operation count
            self.operation_total.add(1, attributes)

-    def record_tokens(
+    def record_llm_call(
        self,
-        operation: str,
-        bank_id: str,
+        provider: str,
+        model: str,
+        scope: str,
+        duration: float,
        input_tokens: int = 0,
        output_tokens: int = 0,
-        budget: str | None = None,
-        max_tokens: int | None = None,
+        success: bool = True,
    ):
        """
-        Record token usage for an operation.
+        Record metrics for an LLM call.

        Args:
-            operation: Operation name (retain, recall, reflect)
-            bank_id: Memory bank ID
-            input_tokens: Number of input tokens
-            output_tokens: Number of output tokens
-            budget: Optional budget level
-            max_tokens: Optional max tokens for the operation
+            provider: LLM provider name (openai, anthropic, gemini, groq, ollama, lmstudio)
+            model: Model name
+            scope: Scope identifier (e.g., "memory", "reflect", "entity_observation")
+            duration: Call duration in seconds
+            input_tokens: Number of input/prompt tokens
+            output_tokens: Number of output/completion tokens
+            success: Whether the call was successful
        """
-        attributes = {
-            "operation": operation,
-            "bank_id": bank_id,
+        # Base attributes for all metrics
+        base_attributes = {
+            "provider": provider,
+            "model": model,
+            "scope": scope,
+            "success": str(success).lower(),
        }
-        if budget:
-            attributes["budget"] = budget
-        if max_tokens:
-            attributes["max_tokens"] = str(max_tokens)

+        # Record duration
+        self.llm_duration.record(duration, base_attributes)
+
+        # Record call count
+        self.llm_calls_total.add(1, base_attributes)
+
+        # Record tokens with bucket labels for cardinality control
        if input_tokens > 0:
-            self.tokens_input.add(input_tokens, attributes)
+            input_attributes = {
+                **base_attributes,
+                "token_bucket": get_token_bucket(input_tokens),
+            }
+            self.llm_tokens_input.add(input_tokens, input_attributes)

        if output_tokens > 0:
-            self.tokens_output.add(output_tokens, attributes)
+            output_attributes = {
+                **base_attributes,
+                "token_bucket": get_token_bucket(output_tokens),
+            }
+            self.llm_tokens_output.add(output_tokens, output_attributes)


 # Global metrics collector instance (defaults to no-op)
@@ -229,3 +229,131 @@ def check_migration_status(
    except Exception as e:
        logger.warning(f"Unable to check migration status: {e}")
        return None, None
+
+
+def ensure_embedding_dimension(
+    database_url: str,
+    required_dimension: int,
+    schema: str | None = None,
+) -> None:
+    """
+    Ensure the embedding column dimension matches the model's dimension.
+
+    This function checks the current vector column dimension in the database
+    and adjusts it if necessary:
+    - If dimensions match: no action needed
+    - If dimensions differ and table is empty: ALTER COLUMN to new dimension
+    - If dimensions differ and table has data: raise error with migration guidance
+
+    Args:
+        database_url: SQLAlchemy database URL
+        required_dimension: The embedding dimension required by the model
+        schema: Target PostgreSQL schema name (None for public)
+
+    Raises:
+        RuntimeError: If dimension mismatch with existing data
+    """
+    schema_name = schema or "public"
+
+    engine = create_engine(database_url)
+    with engine.connect() as conn:
+        # Check if memory_units table exists
+        table_exists = conn.execute(
+            text("""
+                SELECT EXISTS (
+                    SELECT 1 FROM information_schema.tables
+                    WHERE table_schema = :schema AND table_name = 'memory_units'
+                )
+            """),
+            {"schema": schema_name},
+        ).scalar()
+
+        if not table_exists:
+            logger.debug(f"memory_units table does not exist in schema '{schema_name}', skipping dimension check")
+            return
+
+        # Get current column dimension from pg_attribute
+        # pgvector stores dimension in atttypmod
+        current_dim = conn.execute(
+            text("""
+                SELECT atttypmod
+                FROM pg_attribute a
+                JOIN pg_class c ON a.attrelid = c.oid
+                JOIN pg_namespace n ON c.relnamespace = n.oid
+                WHERE n.nspname = :schema
+                  AND c.relname = 'memory_units'
+                  AND a.attname = 'embedding'
+            """),
+            {"schema": schema_name},
+        ).scalar()
+
+        if current_dim is None:
+            logger.warning("Could not determine current embedding dimension, skipping check")
+            return
+
+        # pgvector stores dimension directly in atttypmod (no offset like other types)
+        current_dimension = current_dim
+
+        if current_dimension == required_dimension:
+            logger.debug(f"Embedding dimension OK: {current_dimension}")
+            return
+
+        logger.info(
+            f"Embedding dimension mismatch: database has {current_dimension}, model requires {required_dimension}"
+        )
+
+        # Check if table has data
+        row_count = conn.execute(
+            text(f"SELECT COUNT(*) FROM {schema_name}.memory_units WHERE embedding IS NOT NULL")
+        ).scalar()
+
+        if row_count > 0:
+            raise RuntimeError(
+                f"Cannot change embedding dimension from {current_dimension} to {required_dimension}: "
+                f"memory_units table contains {row_count} rows with embeddings. "
+                f"To change dimensions, you must either:\n"
+                f"  1. Re-embed all data: DELETE FROM {schema_name}.memory_units; then restart\n"
+                f"  2. Use a model with {current_dimension}-dimensional embeddings"
+            )
+
+        # Table is empty, safe to alter column
+        logger.info(f"Altering embedding column dimension from {current_dimension} to {required_dimension}")
+
+        # Drop the HNSW index on embedding column if it exists
+        # Only drop indexes that use 'hnsw' and reference the 'embedding' column
+        conn.execute(
+            text(f"""
+                DO $$
+                DECLARE idx_name TEXT;
+                BEGIN
+                    FOR idx_name IN
+                        SELECT indexname FROM pg_indexes
+                        WHERE schemaname = '{schema_name}'
+                          AND tablename = 'memory_units'
+                          AND indexdef LIKE '%hnsw%'
+                          AND indexdef LIKE '%embedding%'
+                    LOOP
+                        EXECUTE 'DROP INDEX IF EXISTS {schema_name}.' || idx_name;
+                    END LOOP;
+                END $$;
+            """)
+        )
+
+        # Alter the column type
+        conn.execute(
+            text(f"ALTER TABLE {schema_name}.memory_units ALTER COLUMN embedding TYPE vector({required_dimension})")
+        )
+        conn.commit()
+
+        # Recreate the HNSW index
+        conn.execute(
+            text(f"""
+                CREATE INDEX IF NOT EXISTS idx_memory_units_embedding_hnsw
+                ON {schema_name}.memory_units
+                USING hnsw (embedding vector_cosine_ops)
+                WITH (m = 16, ef_construction = 64)
+            """)
+        )
+        conn.commit()
+
+        logger.info(f"Successfully changed embedding dimension to {required_dimension}")
@@ -18,6 +18,9 @@ class RequestContext:
    """

    api_key: str | None = None
+    api_key_id: str | None = None  # UUID of the API key used for authentication
+    tenant_id: str | None = None  # Tenant identifier (set by extension after auth)
+    internal: bool = False  # True for background/internal operations (not user-visible)


 from pgvector.sqlalchemy import Vector
@@ -38,6 +41,8 @@ from sqlalchemy.dialects.postgresql import JSONB, TIMESTAMP, UUID
 from sqlalchemy.ext.asyncio import AsyncAttrs
 from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column, relationship

+from .config import EMBEDDING_DIMENSION
+

 class Base(AsyncAttrs, DeclarativeBase):
    """Base class for all models."""
@@ -78,7 +83,7 @@ class MemoryUnit(Base):
    bank_id: Mapped[str] = mapped_column(Text, nullable=False)
    document_id: Mapped[str | None] = mapped_column(Text)
    text: Mapped[str] = mapped_column(Text, nullable=False)
-    embedding = mapped_column(Vector(384))  # pgvector type
+    embedding = mapped_column(Vector(EMBEDDING_DIMENSION))  # pgvector type
    context: Mapped[str | None] = mapped_column(Text)
    event_date: Mapped[datetime] = mapped_column(
        TIMESTAMP(timezone=True), nullable=False
@@ -132,3 +132,56 @@ async def stop_embedded_postgres() -> None:
    global _default_instance
    if _default_instance:
        await _default_instance.stop()
+
+
+def parse_pg0_url(db_url: str) -> tuple[bool, str | None, int | None]:
+    """
+    Parse a database URL and check if it's a pg0:// embedded database URL.
+
+    Supports:
+    - "pg0" -> default instance "hindsight"
+    - "pg0://instance-name" -> named instance
+    - "pg0://instance-name:port" -> named instance with explicit port
+    - Any other URL (e.g., postgresql://) -> not a pg0 URL
+
+    Args:
+        db_url: The database URL to parse
+
+    Returns:
+        Tuple of (is_pg0, instance_name, port)
+        - is_pg0: True if this is a pg0 URL
+        - instance_name: The instance name (or None if not pg0)
+        - port: The explicit port (or None for auto-assign)
+    """
+    if db_url == "pg0":
+        return True, "hindsight", None
+
+    if db_url.startswith("pg0://"):
+        url_part = db_url[6:]  # Remove "pg0://"
+        if ":" in url_part:
+            instance_name, port_str = url_part.rsplit(":", 1)
+            return True, instance_name or "hindsight", int(port_str)
+        else:
+            return True, url_part or "hindsight", None
+
+    return False, None, None
+
+
+async def resolve_database_url(db_url: str) -> str:
+    """
+    Resolve a database URL, handling pg0:// embedded database URLs.
+
+    If the URL is a pg0:// URL, starts the embedded PostgreSQL and returns
+    the actual postgresql:// connection URL. Otherwise, returns the URL unchanged.
+
+    Args:
+        db_url: Database URL (pg0://, pg0, or postgresql://)
+
+    Returns:
+        The resolved postgresql:// connection URL
+    """
+    is_pg0, instance_name, port = parse_pg0_url(db_url)
+    if is_pg0:
+        pg0 = EmbeddedPostgres(name=instance_name, port=port)
+        return await pg0.ensure_running()
+    return db_url
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"

 [project]
 name = "hindsight-api"
-version = "0.1.12"
+version = "0.2.1"
 description = "Hindsight: Agent Memory That Works Like Human Memory"
 readme = "README.md"
 requires-python = ">=3.11"
@@ -37,6 +37,8 @@ dependencies = [
    "opentelemetry-exporter-prometheus>=0.41b0",
    "dateparser>=1.2.2",
    "google-genai>=1.0.0",
+    "anthropic>=0.40.0",
+    "typer>=0.9.0",
 ]

 [project.optional-dependencies]
@@ -51,6 +53,7 @@ test = [
 [project.scripts]
 hindsight-api = "hindsight_api.main:main"
 hindsight-local-mcp = "hindsight_api.mcp_local:main"
+hindsight-admin = "hindsight_api.admin.cli:main"

 [tool.hatch.build.targets.wheel]
 packages = ["hindsight_api"]
@@ -74,7 +77,7 @@ log_cli = true
 log_cli_level = "INFO"
 log_cli_format = "%(asctime)s - %(levelname)s - %(name)s - %(message)s"
 log_cli_date_format = "%Y-%m-%d %H:%M:%S"
-addopts = "--timeout 120 -n 8 --durations=10 -v"
+addopts = "--timeout 120 -n 8 --dist loadgroup --durations=10 -v"
 asyncio_mode = "auto"
 asyncio_default_fixture_loop_scope = "function"
 log_auto_indent = true
@@ -0,0 +1,292 @@
+"""
+Tests for admin backup and restore functionality.
+
+These tests use an isolated schema to avoid interfering with other tests.
+The backup/restore operations truncate tables, which would cause deadlocks
+and race conditions if run against the shared public schema.
+"""
+
+import tempfile
+import uuid
+import zipfile
+from pathlib import Path
+
+import asyncpg
+import pytest
+import pytest_asyncio
+
+from hindsight_api.admin.cli import _backup, _restore, BACKUP_TABLES
+from hindsight_api.migrations import run_migrations
+
+
+# Run these tests sequentially since they do full DB backup/restore
+pytestmark = pytest.mark.xdist_group(name="backup_restore")
+
+
+@pytest_asyncio.fixture(scope="function")
+async def backup_test_schema(pg0_db_url, embeddings):
+    """Create an isolated schema for backup/restore tests.
+
+    Uses a unique schema name per test invocation to avoid conflicts with
+    parallel test runs or leftover state from interrupted runs.
+
+    Returns a tuple of (db_url, schema_name, fq_helper, embeddings).
+    """
+    # Initialize embeddings if not already done
+    await embeddings.initialize()
+
+    # Use unique schema name to avoid conflicts
+    schema_name = f"backup_test_{uuid.uuid4().hex[:8]}"
+
+    def _fq(table: str) -> str:
+        """Get fully-qualified table name in test schema."""
+        return f"{schema_name}.{table}"
+
+    conn = await asyncpg.connect(pg0_db_url)
+    try:
+        await conn.execute(f"CREATE SCHEMA {schema_name}")
+    finally:
+        await conn.close()
+
+    # Run migrations on the isolated schema
+    run_migrations(pg0_db_url, schema=schema_name)
+
+    yield pg0_db_url, schema_name, _fq, embeddings
+
+    # Cleanup after test
+    conn = await asyncpg.connect(pg0_db_url)
+    try:
+        await conn.execute(f"DROP SCHEMA IF EXISTS {schema_name} CASCADE")
+    finally:
+        await conn.close()
+
+
+@pytest.mark.asyncio
+async def test_backup_restore_roundtrip(backup_test_schema):
+    """Test that backup and restore preserves all data correctly."""
+    db_url, schema_name, _fq, embeddings = backup_test_schema
+    bank_id = f"test-backup-{uuid.uuid4().hex[:8]}"
+    conn = await asyncpg.connect(db_url)
+
+    try:
+        # Create a bank
+        await conn.execute(
+            f"INSERT INTO {_fq('banks')} (bank_id) VALUES ($1) ON CONFLICT DO NOTHING",
+            bank_id,
+        )
+
+        # Create some test memory units with embeddings
+        # Convert embedding list to pgvector format string
+        embedding_list = embeddings.encode(["Test content about Alice"])[0]
+        embedding_str = "[" + ",".join(str(x) for x in embedding_list) + "]"
+        for text in [
+            "Alice is a software engineer who loves Python.",
+            "Bob works with Alice on the backend team.",
+            "The team uses PostgreSQL for their database.",
+        ]:
+            await conn.execute(
+                f"""INSERT INTO {_fq('memory_units')}
+                    (bank_id, text, fact_type, embedding, event_date)
+                    VALUES ($1, $2, 'world', $3::vector, NOW())""",
+                bank_id,
+                text,
+                embedding_str,
+            )
+
+        # Get counts before backup
+        counts_before = {}
+        for table in BACKUP_TABLES:
+            counts_before[table] = await conn.fetchval(f"SELECT COUNT(*) FROM {_fq(table)}")
+
+        # Verify we have data
+        assert counts_before["banks"] > 0
+        assert counts_before["memory_units"] > 0
+
+    finally:
+        await conn.close()
+
+    # Backup to a temp file
+    with tempfile.NamedTemporaryFile(suffix=".zip", delete=False) as f:
+        backup_path = Path(f.name)
+
+    try:
+        manifest = await _backup(db_url, backup_path, schema=schema_name)
+
+        # Verify backup file exists and is valid
+        assert backup_path.exists()
+        assert backup_path.stat().st_size > 0
+
+        # Verify manifest
+        assert manifest["version"] == "1"
+        assert "created_at" in manifest
+        for table in BACKUP_TABLES:
+            assert table in manifest["tables"]
+            assert manifest["tables"][table]["rows"] == counts_before[table]
+
+        # Verify zip contents
+        with zipfile.ZipFile(backup_path, "r") as zf:
+            assert "manifest.json" in zf.namelist()
+            for table in BACKUP_TABLES:
+                assert f"{table}.bin" in zf.namelist()
+
+        # Clear all data
+        conn = await asyncpg.connect(db_url)
+        try:
+            for table in reversed(BACKUP_TABLES):
+                await conn.execute(f"TRUNCATE TABLE {_fq(table)} CASCADE")
+
+            # Verify data is gone
+            for table in BACKUP_TABLES:
+                count = await conn.fetchval(f"SELECT COUNT(*) FROM {_fq(table)}")
+                assert count == 0, f"Table {table} should be empty after truncate"
+        finally:
+            await conn.close()
+
+        # Restore from backup
+        await _restore(db_url, backup_path, schema=schema_name)
+
+        # Verify counts match original
+        conn = await asyncpg.connect(db_url)
+        try:
+            for table in BACKUP_TABLES:
+                count = await conn.fetchval(f"SELECT COUNT(*) FROM {_fq(table)}")
+                assert count == counts_before[table], f"Table {table} count mismatch after restore"
+
+            # Verify data content is preserved
+            texts = await conn.fetch(
+                f"SELECT text FROM {_fq('memory_units')} WHERE bank_id = $1",
+                bank_id,
+            )
+            text_content = " ".join(r["text"] for r in texts)
+            assert "Alice" in text_content or "software" in text_content
+        finally:
+            await conn.close()
+
+    finally:
+        # Cleanup
+        if backup_path.exists():
+            backup_path.unlink()
+
+
+@pytest.mark.asyncio
+async def test_backup_restore_preserves_all_column_types(backup_test_schema):
+    """Test that all column types are preserved: vectors, UUIDs, timestamps, JSONB."""
+    db_url, schema_name, _fq, embeddings = backup_test_schema
+    bank_id = f"test-types-{uuid.uuid4().hex[:8]}"
+    conn = await asyncpg.connect(db_url)
+
+    try:
+        # Create a bank
+        await conn.execute(
+            f"INSERT INTO {_fq('banks')} (bank_id) VALUES ($1) ON CONFLICT DO NOTHING",
+            bank_id,
+        )
+
+        # Create a memory unit with all column types
+        # Convert embedding list to pgvector format string
+        embedding_list = embeddings.encode(["John Smith engineer"])[0]
+        embedding_str = "[" + ",".join(str(x) for x in embedding_list) + "]"
+        await conn.execute(
+            f"""INSERT INTO {_fq('memory_units')}
+                (bank_id, text, fact_type, embedding, event_date, metadata)
+                VALUES ($1, $2, 'world', $3::vector, NOW(), $4)""",
+            bank_id,
+            "John Smith is a senior engineer at Acme Corp since 2020.",
+            embedding_str,
+            '{"key": "value"}',
+        )
+
+        # Create an entity
+        await conn.execute(
+            f"""INSERT INTO {_fq('entities')}
+                (bank_id, canonical_name, metadata)
+                VALUES ($1, $2, $3)""",
+            bank_id,
+            "John Smith",
+            '{"role": "engineer"}',
+        )
+
+        # Get original data
+        original_unit = await conn.fetchrow(
+            f"""SELECT id, embedding, event_date, created_at, metadata, text
+               FROM {_fq('memory_units')} WHERE bank_id = $1 LIMIT 1""",
+            bank_id,
+        )
+        original_entity = await conn.fetchrow(
+            f"""SELECT id, first_seen, last_seen, metadata, canonical_name
+               FROM {_fq('entities')} WHERE bank_id = $1 LIMIT 1""",
+            bank_id,
+        )
+        original_bank = await conn.fetchrow(
+            f"SELECT bank_id, created_at, updated_at FROM {_fq('banks')} WHERE bank_id = $1",
+            bank_id,
+        )
+    finally:
+        await conn.close()
+
+    assert original_unit is not None, "Should have created memory units"
+    assert original_unit["embedding"] is not None, "Should have embedding"
+    assert original_unit["id"] is not None, "Should have UUID"
+    assert original_entity is not None, "Should have created entities"
+
+    with tempfile.NamedTemporaryFile(suffix=".zip", delete=False) as f:
+        backup_path = Path(f.name)
+
+    try:
+        await _backup(db_url, backup_path, schema=schema_name)
+
+        # Clear all data
+        conn = await asyncpg.connect(db_url)
+        try:
+            for table in reversed(BACKUP_TABLES):
+                await conn.execute(f"TRUNCATE TABLE {_fq(table)} CASCADE")
+        finally:
+            await conn.close()
+
+        await _restore(db_url, backup_path, schema=schema_name)
+
+        # Verify all column types are preserved exactly
+        conn = await asyncpg.connect(db_url)
+        try:
+            restored_unit = await conn.fetchrow(
+                f"""SELECT id, embedding, event_date, created_at, metadata, text
+                   FROM {_fq('memory_units')} WHERE bank_id = $1 LIMIT 1""",
+                bank_id,
+            )
+            restored_entity = await conn.fetchrow(
+                f"""SELECT id, first_seen, last_seen, metadata, canonical_name
+                   FROM {_fq('entities')} WHERE bank_id = $1 LIMIT 1""",
+                bank_id,
+            )
+            restored_bank = await conn.fetchrow(
+                f"SELECT bank_id, created_at, updated_at FROM {_fq('banks')} WHERE bank_id = $1",
+                bank_id,
+            )
+        finally:
+            await conn.close()
+
+        # Verify memory_units
+        assert restored_unit is not None, "Should have restored memory unit"
+        assert restored_unit["id"] == original_unit["id"], "UUID should match exactly"
+        assert restored_unit["text"] == original_unit["text"], "Text should match"
+        assert list(restored_unit["embedding"]) == list(original_unit["embedding"]), "Vector embedding should match exactly"
+        assert restored_unit["event_date"] == original_unit["event_date"], "Timestamp should match exactly"
+        assert restored_unit["created_at"] == original_unit["created_at"], "Created timestamp should match"
+        assert restored_unit["metadata"] == original_unit["metadata"], "JSONB metadata should match"
+
+        # Verify entities
+        assert restored_entity is not None, "Should have restored entity"
+        assert restored_entity["id"] == original_entity["id"], "Entity UUID should match"
+        assert restored_entity["canonical_name"] == original_entity["canonical_name"], "Entity name should match"
+        assert restored_entity["first_seen"] == original_entity["first_seen"], "Entity first_seen should match"
+        assert restored_entity["last_seen"] == original_entity["last_seen"], "Entity last_seen should match"
+        assert restored_entity["metadata"] == original_entity["metadata"], "Entity metadata should match"
+
+        # Verify banks
+        assert restored_bank is not None, "Should have restored bank"
+        assert restored_bank["bank_id"] == original_bank["bank_id"], "Bank ID should match"
+        assert restored_bank["created_at"] == original_bank["created_at"], "Bank created_at should match"
+
+    finally:
+        if backup_path.exists():
+            backup_path.unlink()
@@ -0,0 +1,222 @@
+"""
+Test suite for causal relationship extraction.
+
+Tests that the fact extraction system correctly identifies and validates
+causal relationships between facts, with valid indices.
+"""
+
+from datetime import datetime
+
+import pytest
+
+from hindsight_api import LLMConfig
+from hindsight_api.engine.retain.fact_extraction import extract_facts_from_text
+
+
+class TestCausalRelationships:
+    """Tests for causal relationship extraction and validation."""
+
+    @pytest.mark.asyncio
+    async def test_causal_chain_extraction(self):
+        """
+        Test that a clear causal chain is extracted with valid relationships.
+
+        Story: Lost job -> couldn't pay rent -> had to move -> found new apartment
+
+        This is a 4-fact causal chain where each fact causes the next.
+        The extracted causal relations should have valid indices (0-3).
+        """
+        text = """
+I lost my job at the tech company in January because of layoffs.
+Because I lost my job, I couldn't pay my rent anymore.
+Since I couldn't afford rent, I had to move out of my apartment.
+After searching for weeks, I finally found a cheaper apartment in Brooklyn.
+"""
+
+        context = "Personal story about housing change"
+        llm_config = LLMConfig.for_memory()
+
+        facts, _, _ = await extract_facts_from_text(
+            text=text,
+            event_date=datetime(2024, 3, 15),
+            context=context,
+            llm_config=llm_config,
+            agent_name="TestUser"
+        )
+
+        assert len(facts) >= 3, f"Should extract at least 3 facts from the causal chain. Got {len(facts)}"
+
+        # Collect all causal relations from all facts
+        all_causal_relations = []
+        for i, fact in enumerate(facts):
+            if fact.causal_relations:
+                for rel in fact.causal_relations:
+                    all_causal_relations.append({
+                        "from_fact_index": i,
+                        "to_fact_index": rel.target_fact_index,
+                        "relation_type": rel.relation_type,
+                        "strength": rel.strength,
+                        "from_fact_text": fact.fact[:50],
+                    })
+
+        # Verify that ALL causal relation indices are valid
+        num_facts = len(facts)
+        invalid_relations = []
+        for rel in all_causal_relations:
+            if rel["to_fact_index"] < 0 or rel["to_fact_index"] >= num_facts:
+                invalid_relations.append(rel)
+
+        assert len(invalid_relations) == 0, (
+            f"Found {len(invalid_relations)} causal relations with invalid indices! "
+            f"Valid range is 0-{num_facts - 1}. "
+            f"Invalid relations: {invalid_relations}"
+        )
+
+        # Should have at least some causal relations extracted
+        assert len(all_causal_relations) >= 2, (
+            f"Should extract at least 2 causal relationships from this clear chain. "
+            f"Got {len(all_causal_relations)}: {all_causal_relations}"
+        )
+
+        # Verify relation types are valid
+        valid_types = {"causes", "caused_by", "enables", "prevents"}
+        for rel in all_causal_relations:
+            assert rel["relation_type"] in valid_types, (
+                f"Invalid relation_type '{rel['relation_type']}'. Must be one of {valid_types}"
+            )
+
+    @pytest.mark.asyncio
+    async def test_complex_causal_web(self):
+        """
+        Test a more complex scenario with multiple interconnected causes.
+
+        This tests the LLM's ability to identify multiple causal links and
+        ensure all referenced indices exist.
+        """
+        text = """
+The heavy rain caused flooding in the basement.
+The flooding damaged the electrical system.
+Because of the electrical damage, we had to call an electrician.
+The electrician found that the wiring was old and needed replacement.
+We decided to renovate the entire basement while fixing the wiring.
+The renovation took three months and cost $15,000.
+"""
+
+        context = "Home repair story"
+        llm_config = LLMConfig.for_memory()
+
+        facts, _, _ = await extract_facts_from_text(
+            text=text,
+            event_date=datetime(2024, 6, 1),
+            context=context,
+            llm_config=llm_config,
+            agent_name="TestUser"
+        )
+
+        assert len(facts) >= 4, f"Should extract at least 4 facts. Got {len(facts)}"
+
+        # Validate all causal relation indices
+        num_facts = len(facts)
+        for i, fact in enumerate(facts):
+            if fact.causal_relations:
+                for rel in fact.causal_relations:
+                    assert 0 <= rel.target_fact_index < num_facts, (
+                        f"Fact {i} has causal relation to invalid index {rel.target_fact_index}. "
+                        f"Valid range is 0-{num_facts - 1}. "
+                        f"Fact text: {fact.fact[:80]}..."
+                    )
+
+    @pytest.mark.asyncio
+    async def test_no_self_referencing_causal_relations(self):
+        """
+        Test that facts don't have causal relations pointing to themselves.
+        """
+        text = """
+I started learning Python because I wanted to automate my work tasks.
+Learning Python led me to discover machine learning.
+Machine learning fascinated me so much that I changed my career to data science.
+"""
+
+        context = "Career change story"
+        llm_config = LLMConfig.for_memory()
+
+        facts, _, _ = await extract_facts_from_text(
+            text=text,
+            event_date=datetime(2024, 1, 1),
+            context=context,
+            llm_config=llm_config,
+            agent_name="TestUser"
+        )
+
+        # Check no fact references itself
+        for i, fact in enumerate(facts):
+            if fact.causal_relations:
+                for rel in fact.causal_relations:
+                    assert rel.target_fact_index != i, (
+                        f"Fact {i} has a self-referencing causal relation! "
+                        f"Fact text: {fact.fact}"
+                    )
+
+    @pytest.mark.asyncio
+    async def test_bidirectional_causal_relationships(self):
+        """
+        Test that bidirectional causal relationships (causes and caused_by)
+        are handled correctly.
+        """
+        text = """
+My promotion at work caused me to move to New York.
+Moving to New York was caused by my promotion at work.
+The new role enabled me to lead a team of engineers.
+"""
+
+        context = "Work promotion story"
+        llm_config = LLMConfig.for_memory()
+
+        facts, _, _ = await extract_facts_from_text(
+            text=text,
+            event_date=datetime(2024, 2, 15),
+            context=context,
+            llm_config=llm_config,
+            agent_name="TestUser"
+        )
+
+        num_facts = len(facts)
+
+        # Validate all indices
+        for i, fact in enumerate(facts):
+            if fact.causal_relations:
+                for rel in fact.causal_relations:
+                    assert 0 <= rel.target_fact_index < num_facts, (
+                        f"Invalid target_fact_index {rel.target_fact_index} in fact {i}. "
+                        f"Valid range: 0-{num_facts - 1}"
+                    )
+
+    @pytest.mark.asyncio
+    async def test_causal_relation_strength_values(self):
+        """
+        Test that causal relation strength values are within valid range [0.0, 1.0].
+        """
+        text = """
+The stock market crash directly caused the company to lay off employees.
+The layoffs indirectly led to reduced consumer spending in the area.
+Reduced spending somewhat affected local businesses.
+"""
+
+        context = "Economic impact story"
+        llm_config = LLMConfig.for_memory()
+
+        facts, _, _ = await extract_facts_from_text(
+            text=text,
+            event_date=datetime(2024, 4, 1),
+            context=context,
+            llm_config=llm_config,
+            agent_name="TestUser"
+        )
+
+        for i, fact in enumerate(facts):
+            if fact.causal_relations:
+                for rel in fact.causal_relations:
+                    assert 0.0 <= rel.strength <= 1.0, (
+                        f"Causal relation strength {rel.strength} is outside valid range [0.0, 1.0]. "
+                        f"Fact {i}: {fact.fact[:50]}..."
+                    )
@@ -0,0 +1,428 @@
+"""
+Tests for custom embedding dimensions and automatic dimension detection.
+
+Uses isolated PostgreSQL schemas to avoid affecting other tests.
+Includes tests for:
+- Automatic embedding dimension detection and database schema adjustment
+- OpenAI embeddings provider with 1536 dimensions
+"""
+
+import asyncio
+import os
+import pytest
+from datetime import datetime
+from sqlalchemy import create_engine, text
+
+from hindsight_api import MemoryEngine, RequestContext
+from hindsight_api.engine.embeddings import LocalSTEmbeddings, OpenAIEmbeddings
+from hindsight_api.engine.cross_encoder import LocalSTCrossEncoder
+from hindsight_api.engine.query_analyzer import DateparserQueryAnalyzer
+from hindsight_api.extensions import TenantExtension, TenantContext
+from hindsight_api.migrations import run_migrations, ensure_embedding_dimension
+
+
+# =============================================================================
+# Shared Utilities
+# =============================================================================
+
+
+class SchemaTenantExtension(TenantExtension):
+    """Tenant extension that routes all requests to a specific schema (for testing)."""
+
+    def __init__(self, schema_name: str):
+        self.schema_name = schema_name
+
+    async def authenticate(self, request_context: RequestContext) -> TenantContext:
+        return TenantContext(schema_name=self.schema_name)
+
+
+def get_test_schema(prefix: str, worker_id: str) -> str:
+    """Get unique schema name per xdist worker."""
+    if worker_id == "master" or not worker_id:
+        return prefix
+    return f"{prefix}_{worker_id}"
+
+
+def create_isolated_schema(db_url: str, schema_name: str, dimension: int | None = None):
+    """Create an isolated schema with migrations and optional dimension adjustment."""
+    engine = create_engine(db_url)
+
+    # Create schema (drop first if exists from previous failed run)
+    with engine.connect() as conn:
+        conn.execute(text(f"DROP SCHEMA IF EXISTS {schema_name} CASCADE"))
+        conn.execute(text(f"CREATE SCHEMA {schema_name}"))
+        conn.commit()
+
+    # Run migrations in the isolated schema
+    run_migrations(db_url, schema=schema_name)
+
+    # Adjust embedding dimension if specified
+    if dimension is not None:
+        ensure_embedding_dimension(db_url, dimension, schema=schema_name)
+
+
+def drop_schema(db_url: str, schema_name: str):
+    """Drop an isolated schema."""
+    engine = create_engine(db_url)
+    with engine.connect() as conn:
+        conn.execute(text(f"DROP SCHEMA IF EXISTS {schema_name} CASCADE"))
+        conn.commit()
+
+
+def get_column_dimension(db_url: str, schema: str = "public") -> int | None:
+    """Get the current embedding column dimension from the database."""
+    engine = create_engine(db_url)
+    with engine.connect() as conn:
+        result = conn.execute(
+            text("""
+                SELECT atttypmod
+                FROM pg_attribute a
+                JOIN pg_class c ON a.attrelid = c.oid
+                JOIN pg_namespace n ON c.relnamespace = n.oid
+                WHERE n.nspname = :schema
+                  AND c.relname = 'memory_units'
+                  AND a.attname = 'embedding'
+            """),
+            {"schema": schema},
+        ).scalar()
+        return result
+
+
+def get_row_count(db_url: str, schema: str = "public") -> int:
+    """Get the number of rows with embeddings in memory_units."""
+    engine = create_engine(db_url)
+    with engine.connect() as conn:
+        return conn.execute(
+            text(f"SELECT COUNT(*) FROM {schema}.memory_units WHERE embedding IS NOT NULL")
+        ).scalar()
+
+
+def insert_test_embedding(db_url: str, schema: str, dimension: int):
+    """Insert a test row with a dummy embedding."""
+    engine = create_engine(db_url)
+    embedding = [0.1] * dimension
+    embedding_str = "[" + ",".join(str(x) for x in embedding) + "]"
+
+    with engine.connect() as conn:
+        conn.execute(
+            text(f"""
+                INSERT INTO {schema}.memory_units (bank_id, text, embedding, event_date, fact_type)
+                VALUES ('test-bank', 'test text', '{embedding_str}'::vector, NOW(), 'world')
+            """)
+        )
+        conn.commit()
+
+
+def clear_embeddings(db_url: str, schema: str):
+    """Clear all rows from memory_units."""
+    engine = create_engine(db_url)
+    with engine.connect() as conn:
+        conn.execute(text(f"DELETE FROM {schema}.memory_units"))
+        conn.commit()
+
+
+# =============================================================================
+# Embedding Dimension Tests (Local Embeddings)
+# =============================================================================
+
+
+@pytest.fixture(scope="class")
+def dimension_test_schema(pg0_db_url, worker_id):
+    """Create an isolated schema for dimension tests."""
+    schema_name = get_test_schema("test_embed_dim", worker_id)
+    create_isolated_schema(pg0_db_url, schema_name)
+    yield pg0_db_url, schema_name
+    drop_schema(pg0_db_url, schema_name)
+
+
+class TestEmbeddingDimension:
+    """Tests for embedding dimension detection and adjustment."""
+
+    def test_dimension_matches_no_change(self, dimension_test_schema):
+        """When dimension matches, no changes should be made."""
+        db_url, schema = dimension_test_schema
+
+        # Get initial dimension (should be 384 from migration)
+        initial_dim = get_column_dimension(db_url, schema)
+        assert initial_dim == 384, f"Expected 384, got {initial_dim}"
+
+        # Call ensure_embedding_dimension with matching dimension
+        ensure_embedding_dimension(db_url, 384, schema=schema)
+
+        # Dimension should still be 384
+        assert get_column_dimension(db_url, schema) == 384
+
+    def test_dimension_change_empty_table(self, dimension_test_schema):
+        """When table is empty, dimension can be changed."""
+        db_url, schema = dimension_test_schema
+
+        # Ensure table is empty
+        clear_embeddings(db_url, schema)
+        assert get_row_count(db_url, schema) == 0
+
+        # Change dimension to 768
+        ensure_embedding_dimension(db_url, 768, schema=schema)
+
+        # Verify dimension changed
+        new_dim = get_column_dimension(db_url, schema)
+        assert new_dim == 768, f"Expected 768, got {new_dim}"
+
+        # Change back to 384 for other tests
+        ensure_embedding_dimension(db_url, 384, schema=schema)
+        assert get_column_dimension(db_url, schema) == 384
+
+    def test_dimension_change_blocked_with_data(self, dimension_test_schema):
+        """When table has data, dimension change should be blocked."""
+        db_url, schema = dimension_test_schema
+
+        # Ensure table is empty first
+        clear_embeddings(db_url, schema)
+
+        # Insert a test row with 384-dim embedding
+        insert_test_embedding(db_url, schema, 384)
+        assert get_row_count(db_url, schema) == 1
+
+        # Try to change dimension - should raise error
+        with pytest.raises(RuntimeError) as exc_info:
+            ensure_embedding_dimension(db_url, 768, schema=schema)
+
+        assert "Cannot change embedding dimension" in str(exc_info.value)
+        assert "1 rows with embeddings" in str(exc_info.value)
+
+        # Dimension should be unchanged
+        assert get_column_dimension(db_url, schema) == 384
+
+        # Cleanup
+        clear_embeddings(db_url, schema)
+
+    def test_local_embeddings_dimension_detection(self, embeddings):
+        """Test that LocalSTEmbeddings correctly detects dimension."""
+        # Initialize embeddings if not already done
+        loop = asyncio.new_event_loop()
+        try:
+            loop.run_until_complete(embeddings.initialize())
+        finally:
+            loop.close()
+
+        # bge-small-en-v1.5 produces 384-dim embeddings
+        assert embeddings.dimension == 384
+
+        # Verify by generating an actual embedding
+        result = embeddings.encode(["test"])
+        assert len(result) == 1
+        assert len(result[0]) == 384
+
+
+# =============================================================================
+# OpenAI Embeddings Tests
+# =============================================================================
+
+
+def has_openai_api_key() -> bool:
+    """Check if OpenAI API key is available."""
+    return bool(os.environ.get("HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY"))
+
+
+def get_openai_api_key() -> str:
+    """Get OpenAI API key from environment."""
+    return os.environ.get("HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY", "")
+
+
+@pytest.fixture(scope="module")
+def openai_embeddings():
+    """Create OpenAI embeddings instance."""
+    if not has_openai_api_key():
+        pytest.skip("OpenAI API key not available (set HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY)")
+
+    embeddings = OpenAIEmbeddings(
+        api_key=get_openai_api_key(),
+        model="text-embedding-3-small",
+    )
+    loop = asyncio.new_event_loop()
+    try:
+        loop.run_until_complete(embeddings.initialize())
+    finally:
+        loop.close()
+    return embeddings
+
+
+@pytest.fixture(scope="module")
+def openai_test_schema(pg0_db_url, worker_id, openai_embeddings):
+    """Create an isolated schema for OpenAI embedding tests."""
+    schema_name = get_test_schema("test_openai_embed", worker_id)
+    create_isolated_schema(pg0_db_url, schema_name, dimension=openai_embeddings.dimension)
+    yield pg0_db_url, schema_name
+    drop_schema(pg0_db_url, schema_name)
+
+
+@pytest.fixture
+def cross_encoder():
+    """Provide a cross encoder for tests."""
+    return LocalSTCrossEncoder()
+
+
+@pytest.fixture
+def query_analyzer():
+    """Provide a query analyzer for tests."""
+    return DateparserQueryAnalyzer()
+
+
+@pytest.fixture
+def test_bank_id():
+    """Provide a unique bank ID for this test run."""
+    return f"openai_test_{datetime.now().timestamp()}"
+
+
+@pytest.fixture
+def request_context():
+    """Provide a default RequestContext for tests."""
+    return RequestContext()
+
+
+class TestOpenAIEmbeddings:
+    """Tests for OpenAI embeddings provider."""
+
+    def test_openai_embeddings_initialization(self, openai_embeddings):
+        """Test that OpenAI embeddings initializes correctly."""
+        assert openai_embeddings.dimension == 1536
+        assert openai_embeddings.provider_name == "openai"
+
+    def test_openai_embeddings_encode(self, openai_embeddings):
+        """Test that OpenAI embeddings can encode text."""
+        texts = ["Hello, world!", "This is a test."]
+        embeddings = openai_embeddings.encode(texts)
+
+        assert len(embeddings) == 2
+        assert len(embeddings[0]) == 1536
+        assert len(embeddings[1]) == 1536
+        assert all(isinstance(x, float) for x in embeddings[0])
+
+    @pytest.mark.asyncio
+    async def test_openai_embeddings_retain_recall(
+        self,
+        openai_test_schema,
+        openai_embeddings,
+        cross_encoder,
+        query_analyzer,
+        test_bank_id,
+        request_context,
+    ):
+        """Test retain and recall operations with OpenAI embeddings."""
+        db_url, schema_name = openai_test_schema
+
+        memory = MemoryEngine(
+            db_url=db_url,
+            memory_llm_provider=os.getenv("HINDSIGHT_API_LLM_PROVIDER", "groq"),
+            memory_llm_api_key=os.getenv("HINDSIGHT_API_LLM_API_KEY"),
+            memory_llm_model=os.getenv("HINDSIGHT_API_LLM_MODEL", "openai/gpt-oss-120b"),
+            memory_llm_base_url=os.getenv("HINDSIGHT_API_LLM_BASE_URL") or None,
+            embeddings=openai_embeddings,
+            cross_encoder=cross_encoder,
+            query_analyzer=query_analyzer,
+            pool_min_size=1,
+            pool_max_size=3,
+            run_migrations=False,
+            tenant_extension=SchemaTenantExtension(schema_name),
+        )
+
+        try:
+            await memory.initialize()
+
+            # Store some memories
+            await memory.retain_async(
+                bank_id=test_bank_id,
+                content="Alice works as a software engineer at Google.",
+                context="career discussion",
+                request_context=request_context,
+            )
+
+            await memory.retain_async(
+                bank_id=test_bank_id,
+                content="Bob is a data scientist specializing in machine learning.",
+                context="team introductions",
+                request_context=request_context,
+            )
+
+            # Recall memories
+            result = await memory.recall_async(
+                bank_id=test_bank_id,
+                query="Who works in technology?",
+                request_context=request_context,
+            )
+
+            assert result is not None
+            assert len(result.results) > 0
+
+            memory_texts = [m.text for m in result.results]
+            assert any(
+                "Alice" in text or "Bob" in text or "software" in text or "data scientist" in text
+                for text in memory_texts
+            ), f"Expected to find relevant memories, got: {memory_texts}"
+
+        finally:
+            try:
+                if memory._pool and not memory._pool._closing:
+                    await memory.close()
+            except Exception:
+                pass
+
+    @pytest.mark.asyncio
+    async def test_openai_embeddings_batch_retain(
+        self,
+        openai_test_schema,
+        openai_embeddings,
+        cross_encoder,
+        query_analyzer,
+        test_bank_id,
+        request_context,
+    ):
+        """Test batch retain with OpenAI embeddings."""
+        db_url, schema_name = openai_test_schema
+
+        memory = MemoryEngine(
+            db_url=db_url,
+            memory_llm_provider=os.getenv("HINDSIGHT_API_LLM_PROVIDER", "groq"),
+            memory_llm_api_key=os.getenv("HINDSIGHT_API_LLM_API_KEY"),
+            memory_llm_model=os.getenv("HINDSIGHT_API_LLM_MODEL", "openai/gpt-oss-120b"),
+            memory_llm_base_url=os.getenv("HINDSIGHT_API_LLM_BASE_URL") or None,
+            embeddings=openai_embeddings,
+            cross_encoder=cross_encoder,
+            query_analyzer=query_analyzer,
+            pool_min_size=1,
+            pool_max_size=3,
+            run_migrations=False,
+            tenant_extension=SchemaTenantExtension(schema_name),
+        )
+
+        try:
+            await memory.initialize()
+
+            contents = [
+                {"content": "Python is my favorite programming language.", "context": "preferences"},
+                {"content": "I prefer dark mode for all my applications.", "context": "preferences"},
+                {"content": "Coffee is essential for morning productivity.", "context": "habits"},
+            ]
+
+            result = await memory.retain_batch_async(
+                bank_id=test_bank_id,
+                contents=contents,
+                request_context=request_context,
+            )
+
+            assert len(result) == 3
+
+            recall_result = await memory.recall_async(
+                bank_id=test_bank_id,
+                query="What are my preferences?",
+                request_context=request_context,
+            )
+
+            assert recall_result is not None
+            assert len(recall_result.results) > 0
+
+        finally:
+            try:
+                if memory._pool and not memory._pool._closing:
+                    await memory.close()
+            except Exception:
+                pass
@@ -43,7 +43,7 @@ Marcus felt anxious about the upcoming interview.
        context = "Personal journal entry"
        llm_config = LLMConfig.for_memory()

-        facts, _ = await extract_facts_from_text(
+        facts, _, _ = await extract_facts_from_text(
            text=text,
            event_date=datetime(2024, 11, 13),
            context=context,
@@ -75,7 +75,7 @@ The music was so loud I could barely hear myself think.
        context = "Personal experience"
        llm_config = LLMConfig.for_memory()

-        facts, _ = await extract_facts_from_text(
+        facts, _, _ = await extract_facts_from_text(
            text=text,
            event_date=datetime(2024, 11, 13),
            context=context,
@@ -108,7 +108,7 @@ Maybe we should reconsider the timeline.
        context = "Team discussion"
        llm_config = LLMConfig.for_memory()

-        facts, _ = await extract_facts_from_text(
+        facts, _, _ = await extract_facts_from_text(
            text=text,
            event_date=datetime(2024, 11, 13),
            context=context,
@@ -141,7 +141,7 @@ I'm unable to attend the conference due to scheduling conflicts.
        context = "Personal profile discussion"
        llm_config = LLMConfig.for_memory()

-        facts, _ = await extract_facts_from_text(
+        facts, _, _ = await extract_facts_from_text(
            text=text,
            event_date=datetime(2024, 11, 13),
            context=context,
@@ -173,7 +173,7 @@ Unlike last year, we're ahead of schedule.
        context = "Project review"
        llm_config = LLMConfig.for_memory()

-        facts, _ = await extract_facts_from_text(
+        facts, _, _ = await extract_facts_from_text(
            text=text,
            event_date=datetime(2024, 11, 13),
            context=context,
@@ -206,7 +206,7 @@ She's enthusiastic about the opportunity.
        context = "Team meeting"
        llm_config = LLMConfig.for_memory()

-        facts, _ = await extract_facts_from_text(
+        facts, _, _ = await extract_facts_from_text(
            text=text,
            event_date=datetime(2024, 11, 13),
            context=context,
@@ -239,7 +239,7 @@ I'm planning to switch careers because I'm not fulfilled in my current role.
        context = "Personal goals discussion"
        llm_config = LLMConfig.for_memory()

-        facts, _ = await extract_facts_from_text(
+        facts, _, _ = await extract_facts_from_text(
            text=text,
            event_date=datetime(2024, 11, 13),
            context=context,
@@ -276,7 +276,7 @@ Family is the most important thing to her.
        context = "Personal values discussion"
        llm_config = LLMConfig.for_memory()

-        facts, _ = await extract_facts_from_text(
+        facts, _, _ = await extract_facts_from_text(
            text=text,
            event_date=datetime(2024, 11, 13),
            context=context,
@@ -310,7 +310,7 @@ I prefer presenting in person rather than virtually because I can read the room

        event_date = datetime(2024, 11, 13)

-        facts, _ = await extract_facts_from_text(
+        facts, _, _ = await extract_facts_from_text(
            text=text,
            event_date=event_date,
            context=context,
@@ -366,7 +366,7 @@ I'm planning to visit Tokyo next month.

        event_date = datetime(2024, 11, 13)

-        facts, _ = await extract_facts_from_text(
+        facts, _, _ = await extract_facts_from_text(
            text=text,
            event_date=event_date,
            context=context,
@@ -402,6 +402,8 @@ I'm planning to visit Tokyo next month.

        Ideally: If conversation is on August 14, 2023 and text says "last night",
        the date field should be August 13. We accept 13 or 14 as LLM may vary.
+
+        Retries up to 3 times to account for LLM inconsistencies.
        """
        text = """
 Melanie: Hey Caroline! Last night was amazing! We celebrated my daughter's birthday
@@ -410,41 +412,69 @@ with a concert surrounded by music, joy and the warm summer breeze.

        context = "Conversation between Melanie and Caroline"
        llm_config = LLMConfig.for_memory()
-
        event_date = datetime(2023, 8, 14, 14, 24)

-        facts, _ = await extract_facts_from_text(
-            text=text,
-            event_date=event_date,
-            context=context,
-            llm_config=llm_config,
-            agent_name="Melanie"
-        )
+        last_error = None
+        max_retries = 3

-        assert len(facts) > 0, "Should extract at least one fact"
+        for attempt in range(max_retries):
+            try:
+                facts, _, _ = await extract_facts_from_text(
+                    text=text,
+                    event_date=event_date,
+                    context=context,
+                    llm_config=llm_config,
+                    agent_name="Melanie"
+                )

-        birthday_fact = None
-        for fact in facts:
-            if "birthday" in fact.fact.lower() or "concert" in fact.fact.lower():
-                birthday_fact = fact
-                break
+                assert len(facts) > 0, "Should extract at least one fact"

-        assert birthday_fact is not None, "Should extract fact about birthday celebration"
+                birthday_fact = None
+                for fact in facts:
+                    if "birthday" in fact.fact.lower() or "concert" in fact.fact.lower():
+                        birthday_fact = fact
+                        break

-        fact_date_str = birthday_fact.occurred_start
-        assert fact_date_str is not None, "occurred_start should not be None for temporal events"
+                assert birthday_fact is not None, "Should extract fact about birthday celebration"

-        if 'T' in fact_date_str:
-            fact_date = datetime.fromisoformat(fact_date_str.replace('Z', '+00:00'))
-        else:
-            fact_date = datetime.fromisoformat(fact_date_str)
+                fact_date_str = birthday_fact.occurred_start
+                assert fact_date_str is not None, "occurred_start should not be None for temporal events"

-        assert fact_date.year == 2023, "Year should be 2023"
-        assert fact_date.month == 8, "Month should be August"
-        # Accept day 13 (ideal: last night) or 14 (conversation date) as valid
-        assert fact_date.day in (13, 14), (
-            f"Day should be 13 or 14 (around Aug 14 event), but got {fact_date.day}."
-        )
+                if 'T' in fact_date_str:
+                    fact_date = datetime.fromisoformat(fact_date_str.replace('Z', '+00:00'))
+                else:
+                    fact_date = datetime.fromisoformat(fact_date_str)
+
+                assert fact_date.year == 2023, "Year should be 2023"
+                assert fact_date.month == 8, "Month should be August"
+                # Accept day 13 (ideal: last night) or 14 (conversation date) as valid
+                assert fact_date.day in (13, 14), (
+                    f"Day should be 13 or 14 (around Aug 14 event), but got {fact_date.day}."
+                )
+
+                # If we reach here, test passed
+                return
+
+            except AssertionError as e:
+                last_error = e
+                if attempt < max_retries - 1:
+                    print(f"Test attempt {attempt + 1} failed: {e}. Retrying...")
+                    continue
+                else:
+                    # Last attempt failed, re-raise the error
+                    raise e
+            except Exception as e:
+                last_error = e
+                if attempt < max_retries - 1:
+                    print(f"Test attempt {attempt + 1} failed with exception: {e}. Retrying...")
+                    continue
+                else:
+                    # Last attempt failed, re-raise the error
+                    raise e
+
+        # Should not reach here, but just in case
+        if last_error:
+            raise last_error

    @pytest.mark.asyncio
    async def test_date_field_calculation_yesterday(self):
@@ -458,7 +488,7 @@ Yesterday I went for a morning jog for the first time in a nearby park.

        event_date = datetime(2024, 11, 13)

-        facts, _ = await extract_facts_from_text(
+        facts, _, _ = await extract_facts_from_text(
            text=text,
            event_date=event_date,
            context=context,
@@ -507,7 +537,7 @@ Yesterday I went for a morning jog for the first time in a nearby park.
        This morning I had coffee with Alice.
        """

-        facts, _ = await extract_facts_from_text(
+        facts, _, _ = await extract_facts_from_text(
            text=text,
            event_date=reference_date,
            llm_config=llm_config,
@@ -537,7 +567,7 @@ Yesterday I went for a morning jog for the first time in a nearby park.

        text = "Alice works at Google. She loves Python programming."

-        facts, _ = await extract_facts_from_text(
+        facts, _, _ = await extract_facts_from_text(
            text=text,
            event_date=reference_date,
            llm_config=llm_config,
@@ -564,7 +594,7 @@ Yesterday I went for a morning jog for the first time in a nearby park.
        Bob will start his vacation on April 1st.
        """

-        facts, _ = await extract_facts_from_text(
+        facts, _, _ = await extract_facts_from_text(
            text=text,
            event_date=reference_date,
            llm_config=llm_config,
@@ -615,7 +645,7 @@ great time! Every time I see it, I can't help but smile.

        event_date = datetime(2023, 2, 23)

-        facts, _ = await extract_facts_from_text(
+        facts, _, _ = await extract_facts_from_text(
            text=text,
            event_date=event_date,
            context=context,
@@ -665,7 +695,7 @@ I've learned so much from it.
        context = "Personal update"
        llm_config = LLMConfig.for_memory()

-        facts, _ = await extract_facts_from_text(
+        facts, _, _ = await extract_facts_from_text(
            text=text,
            event_date=datetime(2024, 11, 13),
            context=context,
@@ -728,7 +758,7 @@ Jamie: Congratulations! I'd love to read it.

        llm_config = LLMConfig.for_memory()

-        facts, _ = await extract_facts_from_text(
+        facts, _, _ = await extract_facts_from_text(
            text=transcript,
            event_date=datetime(2024, 11, 13),
            llm_config=llm_config,
@@ -773,7 +803,7 @@ We presented our findings to the team yesterday.

        llm_config = LLMConfig.for_memory()

-        facts, _ = await extract_facts_from_text(
+        facts, _, _ = await extract_facts_from_text(
            text=text,
            event_date=datetime(2024, 11, 13),
            llm_config=llm_config,
@@ -808,7 +838,7 @@ Jamie: [teasing] We'll see who's right, my Niners pick is solid.

        llm_config = LLMConfig.for_memory()

-        facts, _ = await extract_facts_from_text(
+        facts, _, _ = await extract_facts_from_text(
            text=transcript,
            event_date=datetime(2024, 11, 14),
            context=context,
@@ -867,7 +897,7 @@ so the algorithm learns to box out. See you next week!

        llm_config = LLMConfig.for_memory()

-        facts, _ = await extract_facts_from_text(
+        facts, _, _ = await extract_facts_from_text(
            text=transcript,
            event_date=datetime(2024, 11, 13),
            llm_config=llm_config,
@@ -428,6 +428,66 @@ async def test_document_deletion(api_client):
    assert response.status_code == 404


+@pytest.mark.asyncio
+async def test_document_deletion_with_slashes_in_id(api_client):
+    """
+    Test document deletion when document_id contains forward slashes.
+
+    Regression test for https://github.com/vectorize-io/hindsight/issues/92
+
+    Document IDs with slashes (e.g., "folder/file.md") should work correctly
+    for all operations including creation, listing, retrieval, and deletion.
+    """
+    import urllib.parse
+
+    test_bank_id = f"doc_slash_test_{datetime.now().timestamp()}"
+    document_id_with_slash = "reports/quarterly/q1-2024.md"
+
+    try:
+        # 1. Create a document with slashes in its ID
+        response = await api_client.post(
+            f"/v1/default/banks/{test_bank_id}/memories",
+            json={
+                "items": [
+                    {
+                        "content": "The Q1 2024 report shows significant growth in user engagement.",
+                        "context": "quarterly report",
+                        "document_id": document_id_with_slash
+                    }
+                ]
+            }
+        )
+        assert response.status_code == 200, f"Failed to create document: {response.text}"
+
+        # 2. Verify document exists via list endpoint
+        response = await api_client.get(f"/v1/default/banks/{test_bank_id}/documents")
+        assert response.status_code == 200
+        documents = response.json()
+        doc_ids = [doc["id"] for doc in documents["items"]]
+        assert document_id_with_slash in doc_ids, f"Document should be in list: {doc_ids}"
+
+        # 3. Delete the document (slashes in document_id should work with :path converter)
+        encoded_doc_id = urllib.parse.quote(document_id_with_slash, safe="")
+        response = await api_client.delete(
+            f"/v1/default/banks/{test_bank_id}/documents/{encoded_doc_id}"
+        )
+        assert response.status_code == 200, (
+            f"Failed to delete document with slashes in ID. "
+            f"Status: {response.status_code}, Response: {response.text}"
+        )
+
+        # Verify document is deleted
+        response = await api_client.get(f"/v1/default/banks/{test_bank_id}/documents")
+        assert response.status_code == 200
+        documents = response.json()
+        doc_ids = [doc["id"] for doc in documents["items"]]
+        assert document_id_with_slash not in doc_ids, "Document should be deleted"
+
+    finally:
+        # Cleanup - delete the bank
+        await api_client.delete(f"/v1/default/banks/{test_bank_id}")
+
+
@pytest.mark.asyncio
 async def test_async_retain(api_client):
    """Test asynchronous retain functionality.
@@ -608,3 +668,298 @@ async def test_async_retain_parallel(api_client):
        assert response.status_code == 200
        results = response.json()["results"]
        assert len(results) > 0, f"Should find memories for document {i}"
+
+
+@pytest.mark.asyncio
+async def test_reflect_structured_output(api_client):
+    """Test reflect endpoint with structured output via response_schema.
+
+    When response_schema is provided, the reflect endpoint should return
+    both the natural language text response and a structured_output field
+    containing the response parsed according to the provided JSON schema.
+    """
+    test_bank_id = f"reflect_structured_test_{datetime.now().timestamp()}"
+
+    # Store some memories to reflect on
+    response = await api_client.post(
+        f"/v1/default/banks/{test_bank_id}/memories",
+        json={
+            "items": [
+                {
+                    "content": "Alice is a senior machine learning engineer with 8 years of experience.",
+                    "context": "team member info"
+                },
+                {
+                    "content": "Bob is a junior data scientist who joined last month.",
+                    "context": "team member info"
+                },
+                {
+                    "content": "The team uses Python and TensorFlow for most projects.",
+                    "context": "tech stack"
+                }
+            ]
+        }
+    )
+    assert response.status_code == 200
+
+    # Define a JSON schema for structured output
+    response_schema = {
+        "type": "object",
+        "properties": {
+            "team_members": {
+                "type": "array",
+                "items": {
+                    "type": "object",
+                    "properties": {
+                        "name": {"type": "string"},
+                        "role": {"type": "string"},
+                        "experience_level": {"type": "string"}
+                    }
+                }
+            },
+            "technologies": {
+                "type": "array",
+                "items": {"type": "string"}
+            },
+            "summary": {"type": "string"}
+        },
+        "required": ["team_members", "summary"]
+    }
+
+    # Call reflect with response_schema
+    response = await api_client.post(
+        f"/v1/default/banks/{test_bank_id}/reflect",
+        json={
+            "query": "Give me an overview of the team and their tech stack",
+            "response_schema": response_schema
+        }
+    )
+    assert response.status_code == 200
+    result = response.json()
+
+    # Verify text field exists (empty when using structured output)
+    assert "text" in result
+    assert result["text"] == ""
+
+    # Verify structured output exists and has expected structure
+    assert "structured_output" in result
+    assert result["structured_output"] is not None
+
+    structured = result["structured_output"]
+    assert "team_members" in structured
+    assert "summary" in structured
+    assert isinstance(structured["team_members"], list)
+    assert isinstance(structured["summary"], str)
+
+    # Verify team members have the expected fields
+    if len(structured["team_members"]) > 0:
+        member = structured["team_members"][0]
+        assert "name" in member or "role" in member  # At least some fields should be present
+
+
+@pytest.mark.asyncio
+async def test_reflect_without_structured_output(api_client):
+    """Test that reflect works normally without response_schema.
+
+    When response_schema is not provided, the structured_output field
+    should be null/None in the response.
+    """
+    test_bank_id = f"reflect_no_structured_test_{datetime.now().timestamp()}"
+
+    # Store a memory
+    response = await api_client.post(
+        f"/v1/default/banks/{test_bank_id}/memories",
+        json={
+            "items": [
+                {
+                    "content": "The project deadline is next Friday.",
+                    "context": "project timeline"
+                }
+            ]
+        }
+    )
+    assert response.status_code == 200
+
+    # Call reflect without response_schema
+    response = await api_client.post(
+        f"/v1/default/banks/{test_bank_id}/reflect",
+        json={
+            "query": "When is the project deadline?"
+        }
+    )
+    assert response.status_code == 200
+    result = response.json()
+
+    # Verify response has text but structured_output is null
+    assert "text" in result
+    assert len(result["text"]) > 0
+    assert result.get("structured_output") is None
+
+
+@pytest.mark.asyncio
+async def test_reflect_with_max_tokens(api_client):
+    """Test reflect endpoint with custom max_tokens parameter.
+
+    The max_tokens parameter controls the maximum tokens for the LLM response.
+    """
+    test_bank_id = f"reflect_max_tokens_test_{datetime.now().timestamp()}"
+
+    # Store a memory
+    response = await api_client.post(
+        f"/v1/default/banks/{test_bank_id}/memories",
+        json={
+            "items": [
+                {
+                    "content": "Python is a popular programming language for data science and machine learning.",
+                    "context": "tech"
+                }
+            ]
+        }
+    )
+    assert response.status_code == 200
+
+    # Call reflect with custom max_tokens
+    response = await api_client.post(
+        f"/v1/default/banks/{test_bank_id}/reflect",
+        json={
+            "query": "What is Python used for?",
+            "max_tokens": 500
+        }
+    )
+    assert response.status_code == 200
+    result = response.json()
+
+    # Verify response has text
+    assert "text" in result
+    assert len(result["text"]) > 0
+
+
+@pytest.mark.asyncio
+async def test_reflect_returns_token_usage(api_client):
+    """Test that reflect endpoint returns token usage metrics.
+
+    The usage field should contain input_tokens, output_tokens, and total_tokens
+    from the LLM call made during reflection.
+    """
+    test_bank_id = f"reflect_usage_test_{datetime.now().timestamp()}"
+
+    # Store a memory to reflect on
+    response = await api_client.post(
+        f"/v1/default/banks/{test_bank_id}/memories",
+        json={
+            "items": [
+                {
+                    "content": "The capital of France is Paris.",
+                    "context": "geography"
+                }
+            ]
+        }
+    )
+    assert response.status_code == 200
+
+    # Call reflect
+    response = await api_client.post(
+        f"/v1/default/banks/{test_bank_id}/reflect",
+        json={
+            "query": "What is the capital of France?"
+        }
+    )
+    assert response.status_code == 200
+    result = response.json()
+
+    # Verify response has text
+    assert "text" in result
+    assert len(result["text"]) > 0
+
+    # Verify usage field exists and has expected structure
+    assert "usage" in result, "Response should include 'usage' field"
+    usage = result["usage"]
+    assert usage is not None, "Usage should not be None for reflect"
+    assert "input_tokens" in usage, "Usage should have 'input_tokens'"
+    assert "output_tokens" in usage, "Usage should have 'output_tokens'"
+    assert "total_tokens" in usage, "Usage should have 'total_tokens'"
+
+    # Verify token counts are valid
+    assert usage["input_tokens"] > 0, f"Expected input_tokens > 0, got {usage['input_tokens']}"
+    assert usage["output_tokens"] >= 0, f"Expected output_tokens >= 0, got {usage['output_tokens']}"
+    assert usage["total_tokens"] == usage["input_tokens"] + usage["output_tokens"]
+
+    print(f"Reflect token usage: input={usage['input_tokens']}, output={usage['output_tokens']}, total={usage['total_tokens']}")
+
+
+@pytest.mark.asyncio
+async def test_retain_returns_token_usage(api_client):
+    """Test that retain endpoint returns token usage metrics for synchronous operations.
+
+    The usage field should contain input_tokens, output_tokens, and total_tokens
+    from the LLM calls made during fact extraction.
+    """
+    test_bank_id = f"retain_usage_test_{datetime.now().timestamp()}"
+
+    # Store memory synchronously (async=false is default)
+    response = await api_client.post(
+        f"/v1/default/banks/{test_bank_id}/memories",
+        json={
+            "items": [
+                {
+                    "content": "Alice is a software engineer at TechCorp. She specializes in machine learning.",
+                    "context": "team introduction"
+                }
+            ]
+        }
+    )
+    assert response.status_code == 200
+    result = response.json()
+
+    # Verify basic response
+    assert result["success"] is True
+    assert result["items_count"] == 1
+    assert result["async"] is False
+
+    # Verify usage field exists and has expected structure
+    assert "usage" in result, "Response should include 'usage' field"
+    usage = result["usage"]
+    assert usage is not None, "Usage should not be None for synchronous retain"
+    assert "input_tokens" in usage, "Usage should have 'input_tokens'"
+    assert "output_tokens" in usage, "Usage should have 'output_tokens'"
+    assert "total_tokens" in usage, "Usage should have 'total_tokens'"
+
+    # Verify token counts are valid
+    assert usage["input_tokens"] > 0, f"Expected input_tokens > 0, got {usage['input_tokens']}"
+    assert usage["output_tokens"] >= 0, f"Expected output_tokens >= 0, got {usage['output_tokens']}"
+    assert usage["total_tokens"] == usage["input_tokens"] + usage["output_tokens"]
+
+    print(f"Retain token usage: input={usage['input_tokens']}, output={usage['output_tokens']}, total={usage['total_tokens']}")
+
+
+@pytest.mark.asyncio
+async def test_retain_async_no_usage(api_client):
+    """Test that async retain does not return usage (as it's processed in background).
+
+    When async=true, the usage field should be None since the actual
+    fact extraction happens asynchronously.
+    """
+    test_bank_id = f"retain_async_no_usage_test_{datetime.now().timestamp()}"
+
+    # Store memory asynchronously
+    response = await api_client.post(
+        f"/v1/default/banks/{test_bank_id}/memories",
+        json={
+            "async": True,
+            "items": [
+                {
+                    "content": "Bob is a data scientist.",
+                    "context": "team introduction"
+                }
+            ]
+        }
+    )
+    assert response.status_code == 200
+    result = response.json()
+
+    # Verify async response
+    assert result["success"] is True
+    assert result["async"] is True
+
+    # Usage should be None for async operations
+    assert result.get("usage") is None, "Async retain should not include usage"
@@ -0,0 +1,245 @@
+"""
+Test that LLM calls record token metrics via the metrics collector.
+"""
+import os
+from unittest.mock import MagicMock, patch
+import pytest
+from hindsight_api.engine.llm_wrapper import LLMProvider
+from hindsight_api.metrics import (
+    MetricsCollector,
+    NoOpMetricsCollector,
+    get_metrics_collector,
+)
+
+
+def get_groq_api_key() -> str | None:
+    """Get Groq API key from environment."""
+    return os.getenv("GROQ_API_KEY")
+
+
+@pytest.mark.asyncio
+async def test_llm_metrics_recorded_for_groq():
+    """
+    Test that LLM metrics are recorded when making LLM calls via Groq.
+    Uses openai/gpt-oss-20b as recommended by Hindsight.
+    """
+    api_key = get_groq_api_key()
+    if not api_key:
+        pytest.skip("Skipping: GROQ_API_KEY not set")
+
+    # Create a mock metrics collector to track record_llm_call calls
+    mock_collector = MagicMock(spec=MetricsCollector)
+
+    with patch("hindsight_api.engine.llm_wrapper.get_metrics_collector", return_value=mock_collector):
+        llm = LLMProvider(
+            provider="groq",
+            api_key=api_key,
+            base_url="",
+            model="openai/gpt-oss-20b",
+        )
+
+        # Make an LLM call with clear instruction
+        response = await llm.call(
+            messages=[
+                {"role": "system", "content": "You are a helpful assistant. Always respond."},
+                {"role": "user", "content": "What is 2+2? Reply with just the number."}
+            ],
+            max_completion_tokens=50,
+            scope="test_metrics",
+        )
+
+        # Verify record_llm_call was called - this is the main test
+        assert mock_collector.record_llm_call.called, "record_llm_call should have been called"
+
+        # Get the call arguments
+        call_kwargs = mock_collector.record_llm_call.call_args.kwargs
+
+        # Verify the call had correct structure
+        assert call_kwargs["provider"] == "groq", f"Expected provider='groq', got {call_kwargs}"
+        assert call_kwargs["model"] == "openai/gpt-oss-20b", f"Expected model='openai/gpt-oss-20b', got {call_kwargs}"
+        assert call_kwargs["scope"] == "test_metrics", f"Expected scope='test_metrics', got {call_kwargs}"
+        assert call_kwargs["duration"] > 0, f"Expected duration > 0, got {call_kwargs['duration']}"
+        assert call_kwargs["input_tokens"] > 0, f"Expected input_tokens > 0, got {call_kwargs['input_tokens']}"
+        assert call_kwargs["output_tokens"] >= 0, f"Expected output_tokens >= 0, got {call_kwargs['output_tokens']}"
+        assert call_kwargs["success"] is True, f"Expected success=True, got {call_kwargs['success']}"
+
+        print(f"\nLLM metrics recorded:")
+        print(f"  provider: {call_kwargs['provider']}")
+        print(f"  model: {call_kwargs['model']}")
+        print(f"  scope: {call_kwargs['scope']}")
+        print(f"  duration: {call_kwargs['duration']:.3f}s")
+        print(f"  input_tokens: {call_kwargs['input_tokens']}")
+        print(f"  output_tokens: {call_kwargs['output_tokens']}")
+        print(f"  response: {response}")
+
+
+@pytest.mark.asyncio
+async def test_llm_metrics_recorded_for_structured_output():
+    """
+    Test that LLM metrics are recorded for structured output (JSON) calls.
+    """
+    api_key = get_groq_api_key()
+    if not api_key:
+        pytest.skip("Skipping: GROQ_API_KEY not set")
+
+    from pydantic import BaseModel
+
+    class SimpleResponse(BaseModel):
+        greeting: str
+        language: str
+
+    mock_collector = MagicMock(spec=MetricsCollector)
+
+    with patch("hindsight_api.engine.llm_wrapper.get_metrics_collector", return_value=mock_collector):
+        llm = LLMProvider(
+            provider="groq",
+            api_key=api_key,
+            base_url="",
+            model="openai/gpt-oss-20b",
+        )
+
+        # Make a structured output call
+        response = await llm.call(
+            messages=[{"role": "user", "content": "Say hello in French. Return greeting and language."}],
+            response_format=SimpleResponse,
+            max_completion_tokens=100,
+            scope="structured_output_test",
+        )
+
+        # Verify structured response
+        assert isinstance(response, SimpleResponse)
+        assert response.greeting is not None
+        assert response.language is not None
+
+        # Verify record_llm_call was called
+        assert mock_collector.record_llm_call.called, "record_llm_call should have been called"
+
+        call_kwargs = mock_collector.record_llm_call.call_args.kwargs
+        assert call_kwargs["input_tokens"] > 0
+        assert call_kwargs["output_tokens"] > 0
+
+        print(f"\nStructured output LLM metrics:")
+        print(f"  greeting: {response.greeting}")
+        print(f"  language: {response.language}")
+        print(f"  input_tokens: {call_kwargs['input_tokens']}")
+        print(f"  output_tokens: {call_kwargs['output_tokens']}")
+
+
+@pytest.mark.asyncio
+async def test_noop_collector_when_metrics_disabled():
+    """
+    Test that NoOpMetricsCollector is returned when metrics are not initialized.
+    This verifies the fallback behavior doesn't break LLM calls.
+    """
+    api_key = get_groq_api_key()
+    if not api_key:
+        pytest.skip("Skipping: GROQ_API_KEY not set")
+
+    # Without initializing metrics, get_metrics_collector returns NoOpMetricsCollector
+    collector = get_metrics_collector()
+    assert isinstance(collector, NoOpMetricsCollector), "Should return NoOpMetricsCollector when not initialized"
+
+    # Make an LLM call - should work fine with NoOp collector
+    llm = LLMProvider(
+        provider="groq",
+        api_key=api_key,
+        base_url="",
+        model="openai/gpt-oss-20b",
+    )
+
+    response = await llm.call(
+        messages=[{"role": "user", "content": "Say 'test' in one word."}],
+        max_completion_tokens=50,
+    )
+
+    assert response is not None
+    print(f"\nLLM call succeeded with NoOpMetricsCollector: {response}")
+
+
+@pytest.mark.asyncio
+async def test_return_usage_returns_tuple():
+    """
+    Test that return_usage=True returns (result, TokenUsage) tuple.
+    """
+    from hindsight_api.engine.response_models import TokenUsage
+
+    api_key = get_groq_api_key()
+    if not api_key:
+        pytest.skip("Skipping: GROQ_API_KEY not set")
+
+    llm = LLMProvider(
+        provider="groq",
+        api_key=api_key,
+        base_url="",
+        model="openai/gpt-oss-20b",
+    )
+
+    # Call with return_usage=True
+    result, usage = await llm.call(
+        messages=[
+            {"role": "system", "content": "You are a helpful assistant."},
+            {"role": "user", "content": "What is 2+2? Reply with just the number."}
+        ],
+        max_completion_tokens=50,
+        return_usage=True,
+    )
+
+    # Verify result is the response text
+    assert result is not None
+    assert isinstance(result, str)
+
+    # Verify usage is TokenUsage model with valid counts
+    assert isinstance(usage, TokenUsage)
+    assert usage.input_tokens > 0, f"Expected input_tokens > 0, got {usage.input_tokens}"
+    assert usage.output_tokens >= 0, f"Expected output_tokens >= 0, got {usage.output_tokens}"
+    assert usage.total_tokens == usage.input_tokens + usage.output_tokens
+
+    print(f"\nreturn_usage=True test:")
+    print(f"  result: {result}")
+    print(f"  usage: {usage}")
+
+
+@pytest.mark.asyncio
+async def test_return_usage_with_structured_output():
+    """
+    Test that return_usage=True works with structured output (JSON).
+    """
+    from pydantic import BaseModel
+    from hindsight_api.engine.response_models import TokenUsage
+
+    api_key = get_groq_api_key()
+    if not api_key:
+        pytest.skip("Skipping: GROQ_API_KEY not set")
+
+    class MathAnswer(BaseModel):
+        answer: int
+        explanation: str
+
+    llm = LLMProvider(
+        provider="groq",
+        api_key=api_key,
+        base_url="",
+        model="openai/gpt-oss-20b",
+    )
+
+    # Call with return_usage=True and structured output
+    result, usage = await llm.call(
+        messages=[{"role": "user", "content": "What is 5+3? Return the answer and a brief explanation."}],
+        response_format=MathAnswer,
+        max_completion_tokens=100,
+        return_usage=True,
+    )
+
+    # Verify result is the parsed response
+    assert isinstance(result, MathAnswer)
+    assert result.answer == 8
+    assert result.explanation is not None
+
+    # Verify usage is TokenUsage model
+    assert isinstance(usage, TokenUsage)
+    assert usage.input_tokens > 0
+    assert usage.output_tokens > 0
+
+    print(f"\nStructured output with return_usage=True:")
+    print(f"  result: {result}")
+    print(f"  usage: {usage}")
@@ -1,177 +0,0 @@
-"""
-Integration test for the MCP (Model Context Protocol) server.
-
-Tests MCP endpoints by starting a FastAPI server with MCP enabled and using the MCP client.
-
-Note: MCP server is integrated with the web server. These tests require HINDSIGHT_API_MCP_ENABLED=true.
-"""
-import asyncio
-import pytest
-import pytest_asyncio
-import httpx
-from mcp import ClientSession
-from mcp.client.sse import sse_client
-from hindsight_api.api import create_app
-
-
-@pytest_asyncio.fixture
-async def mcp_server(memory):
-    """Start the FastAPI app with MCP enabled and return the SSE URL."""
-    # Memory is already initialized by the conftest fixture (with migrations)
-    app = create_app(
-        memory,
-        initialize_memory=False,
-        mcp_api_enabled=True
-    )
-
-    # Use httpx to create a test server
-    transport = httpx.ASGITransport(app=app)
-    async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
-        # The MCP SSE endpoint is at /mcp/sse
-        # We need to yield the base URL for sse_client to connect
-        # However, sse_client expects a real URL, not a test client
-        # So we'll start a real server on a random port
-        pass
-
-    # For now, skip these tests as they require a real server
-    # The sse_client doesn't work with ASGI test transport
-    pytest.skip("MCP tests require a real running server. Run: HINDSIGHT_API_MCP_ENABLED=true uvicorn hindsight_api.api:app")
-
-
-@pytest.mark.asyncio
-async def test_mcp_server_tools_via_sse(mcp_server):
-    """Test MCP server tools via SSE transport using proper MCP client."""
-    sse_url = mcp_server
-
-    async with sse_client(sse_url) as (read, write):
-        async with ClientSession(read, write) as session:
-            await session.initialize()
-
-            # Test 1: List tools
-            tools_list = await session.list_tools()
-            print(f"Tools: {tools_list}")
-            tool_names = [t.name for t in tools_list.tools]
-            assert "hindsight_search" in tool_names
-            assert "hindsight_put" in tool_names
-
-            # Test 2: Call hindsight_put
-            put_result = await session.call_tool(
-                "hindsight_put",
-                arguments={
-                    "content": "User loves Python programming",
-                    "context": "programming_preferences",
-                    "explanation": "Storing user's programming language preference"
-                }
-            )
-            print(f"Put result: {put_result}")
-            assert put_result is not None
-
-            # Wait a bit for indexing
-            await asyncio.sleep(1)
-
-            # Test 3: Call hindsight_search
-            search_result = await session.call_tool(
-                "hindsight_search",
-                arguments={
-                    "query": "What programming languages does the user like?",
-                    "max_tokens": 4096,
-                    "explanation": "Searching for programming preferences"
-                }
-            )
-            print(f"Search result: {search_result}")
-            assert search_result is not None
-
-
-@pytest.mark.asyncio
-async def test_multiple_concurrent_requests(mcp_server):
-    """Test multiple concurrent requests from a single session."""
-    sse_url = mcp_server
-
-    async with sse_client(sse_url) as (read, write):
-        async with ClientSession(read, write) as session:
-            await session.initialize()
-
-            # Fire off 10 concurrent search requests from same session
-            async def make_search(idx):
-                try:
-                    result = await session.call_tool(
-                        "hindsight_search",
-                        arguments={
-                            "query": f"test query {idx}",
-                            "explanation": f"Concurrent test {idx}"
-                        }
-                    )
-                    return idx, "success", result
-                except Exception as e:
-                    return idx, "error", str(e)
-
-            tasks = [make_search(i) for i in range(10)]
-            results = await asyncio.gather(*tasks, return_exceptions=True)
-
-            # Check results
-            successes = 0
-            failures = 0
-
-            for result in results:
-                if isinstance(result, Exception):
-                    print(f"Request failed with exception: {result}")
-                    failures += 1
-                else:
-                    idx, status, data = result
-                    if status == "success":
-                        successes += 1
-                    else:
-                        print(f"Request {idx} failed: {data}")
-                        failures += 1
-
-            print(f"Successes: {successes}, Failures: {failures}")
-
-            # We expect all requests to succeed
-            assert successes >= 8, f"Too many failures: {failures}/10"
-
-
-@pytest.mark.asyncio
-async def test_race_condition_with_rapid_requests(mcp_server):
-    """Test rapid-fire requests with multiple sessions to trigger race condition."""
-    sse_url = mcp_server
-
-    async def rapid_session_search(idx):
-        """Create a new session and immediately make a request."""
-        try:
-            async with sse_client(sse_url) as (read, write):
-                async with ClientSession(read, write) as session:
-                    await session.initialize()
-
-                    # Make request immediately after initialization
-                    result = await session.call_tool(
-                        "hindsight_search",
-                        arguments={
-                            "query": f"rapid query {idx}",
-                            "max_tokens": 2048
-                        }
-                    )
-                    return idx, "success", result
-        except Exception as e:
-            return idx, "error", str(e)
-
-    # Fire 20 requests with minimal delay, each with its own session
-    tasks = [rapid_session_search(i) for i in range(20)]
-    results = await asyncio.gather(*tasks)
-
-    # Analyze results
-    errors = []
-    for idx, status, data in results:
-        if status == "error":
-            errors.append((idx, data))
-
-    if errors:
-        print(f"Found {len(errors)} errors:")
-        for idx, error_msg in errors:
-            print(f"  Request {idx}: {error_msg}")
-
-    # Most requests should succeed
-    assert len(errors) < 5, f"Too many errors: {len(errors)}/20"
-
-
-if __name__ == "__main__":
-    pytest.main([__file__, "-v", "-s"])
@@ -0,0 +1,391 @@
+"""Tests for metrics instrumentation."""
+import pytest
+from unittest.mock import MagicMock, patch
+
+from hindsight_api.metrics import (
+    MetricsCollector,
+    MetricsCollectorBase,
+    NoOpMetricsCollector,
+    get_metrics_collector,
+    get_token_bucket,
+    create_metrics_collector,
+    initialize_metrics,
+)
+
+
+class TestNoOpMetricsCollector:
+    """Tests for the no-op metrics collector."""
+
+    def test_record_operation_is_noop(self):
+        """Test that record_operation does nothing."""
+        collector = NoOpMetricsCollector()
+
+        # Should not raise any exception
+        with collector.record_operation("recall", bank_id="test_bank", source="api"):
+            pass
+
+    def test_nested_contexts_work(self):
+        """Test that nested context managers work correctly."""
+        collector = NoOpMetricsCollector()
+
+        # Nested contexts should work without issues
+        with collector.record_operation("reflect", bank_id="test_bank", source="api"):
+            with collector.record_operation("recall", bank_id="test_bank", source="reflect"):
+                pass
+
+    def test_exception_propagates(self):
+        """Test that exceptions inside context are propagated."""
+        collector = NoOpMetricsCollector()
+
+        with pytest.raises(ValueError, match="test error"):
+            with collector.record_operation("recall", bank_id="test_bank"):
+                raise ValueError("test error")
+
+    def test_record_llm_call_is_noop(self):
+        """Test that record_llm_call does nothing."""
+        collector = NoOpMetricsCollector()
+
+        # Should not raise any exception
+        collector.record_llm_call(
+            provider="openai",
+            model="gpt-4",
+            scope="memory",
+            duration=1.5,
+            input_tokens=100,
+            output_tokens=50,
+            success=True,
+        )
+
+
+class TestMetricsCollector:
+    """Tests for the real metrics collector."""
+
+    @pytest.fixture
+    def mock_meter(self):
+        """Create a mock meter for testing."""
+        meter = MagicMock()
+        # Create separate mocks for each histogram (operation_duration, llm_duration)
+        histogram_mocks = [MagicMock(), MagicMock()]
+        meter.create_histogram.side_effect = histogram_mocks
+        # Create separate mocks for each counter
+        # (operation_total, llm_tokens_input, llm_tokens_output, llm_calls_total)
+        counter_mocks = [MagicMock() for _ in range(4)]
+        meter.create_counter.side_effect = counter_mocks
+        return meter
+
+    @pytest.fixture
+    def collector(self, mock_meter):
+        """Create a MetricsCollector with a mock meter."""
+        with patch("hindsight_api.metrics.get_meter", return_value=mock_meter):
+            return MetricsCollector()
+
+    def test_record_operation_records_duration(self, collector):
+        """Test that record_operation records duration."""
+        with collector.record_operation("recall", bank_id="test_bank", source="api"):
+            pass
+
+        # Histogram should have been called
+        collector.operation_duration.record.assert_called_once()
+        call_args = collector.operation_duration.record.call_args
+
+        # First arg is duration (should be > 0)
+        duration = call_args[0][0]
+        assert duration >= 0
+
+        # Second arg is attributes dict
+        attributes = call_args[0][1]
+        assert attributes["operation"] == "recall"
+        assert attributes["bank_id"] == "test_bank"
+        assert attributes["source"] == "api"
+        assert attributes["success"] == "true"
+
+    def test_record_operation_records_failure_on_exception(self, collector):
+        """Test that record_operation records failure when exception occurs."""
+        with pytest.raises(RuntimeError):
+            with collector.record_operation("retain", bank_id="test_bank", source="api"):
+                raise RuntimeError("Test error")
+
+        # Should have recorded with success=false
+        call_args = collector.operation_duration.record.call_args
+        attributes = call_args[0][1]
+        assert attributes["success"] == "false"
+
+    def test_record_operation_with_budget(self, collector):
+        """Test that budget is included in attributes when provided."""
+        with collector.record_operation("recall", bank_id="test_bank", source="api", budget="mid"):
+            pass
+
+        call_args = collector.operation_duration.record.call_args
+        attributes = call_args[0][1]
+        assert attributes["budget"] == "mid"
+
+    def test_record_operation_with_max_tokens(self, collector):
+        """Test that max_tokens is included in attributes when provided."""
+        with collector.record_operation("recall", bank_id="test_bank", source="api", max_tokens=4096):
+            pass
+
+        call_args = collector.operation_duration.record.call_args
+        attributes = call_args[0][1]
+        assert attributes["max_tokens"] == "4096"
+
+    def test_record_operation_source_values(self, collector):
+        """Test different source values: api, reflect, internal."""
+        sources = ["api", "reflect", "internal"]
+
+        for source in sources:
+            collector.operation_duration.record.reset_mock()
+
+            with collector.record_operation("recall", bank_id="test_bank", source=source):
+                pass
+
+            call_args = collector.operation_duration.record.call_args
+            attributes = call_args[0][1]
+            assert attributes["source"] == source
+
+    def test_nested_contexts_track_separately(self, collector):
+        """Test that nested operations are tracked separately with different sources."""
+        # Simulate reflect (api) calling recall (reflect)
+        with collector.record_operation("reflect", bank_id="test_bank", source="api"):
+            with collector.record_operation("recall", bank_id="test_bank", source="reflect"):
+                pass
+
+        # Should have 2 calls to record
+        assert collector.operation_duration.record.call_count == 2
+        assert collector.operation_total.add.call_count == 2
+
+        # Check the calls
+        calls = collector.operation_duration.record.call_args_list
+
+        # First call should be recall (inner context exits first)
+        recall_attrs = calls[0][0][1]
+        assert recall_attrs["operation"] == "recall"
+        assert recall_attrs["source"] == "reflect"
+
+        # Second call should be reflect (outer context exits last)
+        reflect_attrs = calls[1][0][1]
+        assert reflect_attrs["operation"] == "reflect"
+        assert reflect_attrs["source"] == "api"
+
+
+class TestGetMetricsCollector:
+    """Tests for the get_metrics_collector function."""
+
+    def test_returns_noop_by_default(self):
+        """Test that get_metrics_collector returns NoOpMetricsCollector by default."""
+        # Reset global state
+        import hindsight_api.metrics as metrics_module
+        original_collector = metrics_module._metrics_collector
+
+        try:
+            metrics_module._metrics_collector = NoOpMetricsCollector()
+            collector = get_metrics_collector()
+            assert isinstance(collector, NoOpMetricsCollector)
+        finally:
+            metrics_module._metrics_collector = original_collector
+
+
+class TestMetricsCollectorBase:
+    """Tests for the MetricsCollectorBase abstract class."""
+
+    def test_is_abstract(self):
+        """Test that MetricsCollectorBase methods are abstract."""
+        # Create a class that inherits but doesn't implement
+        class IncompleteCollector(MetricsCollectorBase):
+            pass
+
+        collector = IncompleteCollector()
+
+        # Abstract methods should raise NotImplementedError
+        with pytest.raises(NotImplementedError):
+            with collector.record_operation("test", "test"):
+                pass
+
+        with pytest.raises(NotImplementedError):
+            collector.record_llm_call("test", "test", "test", 1.0)
+
+
+class TestGetTokenBucket:
+    """Tests for the get_token_bucket function."""
+
+    def test_bucket_0_100(self):
+        """Test tokens < 100 return '0-100' bucket."""
+        assert get_token_bucket(0) == "0-100"
+        assert get_token_bucket(50) == "0-100"
+        assert get_token_bucket(99) == "0-100"
+
+    def test_bucket_100_500(self):
+        """Test tokens 100-499 return '100-500' bucket."""
+        assert get_token_bucket(100) == "100-500"
+        assert get_token_bucket(250) == "100-500"
+        assert get_token_bucket(499) == "100-500"
+
+    def test_bucket_500_1k(self):
+        """Test tokens 500-999 return '500-1k' bucket."""
+        assert get_token_bucket(500) == "500-1k"
+        assert get_token_bucket(750) == "500-1k"
+        assert get_token_bucket(999) == "500-1k"
+
+    def test_bucket_1k_5k(self):
+        """Test tokens 1000-4999 return '1k-5k' bucket."""
+        assert get_token_bucket(1000) == "1k-5k"
+        assert get_token_bucket(2500) == "1k-5k"
+        assert get_token_bucket(4999) == "1k-5k"
+
+    def test_bucket_5k_10k(self):
+        """Test tokens 5000-9999 return '5k-10k' bucket."""
+        assert get_token_bucket(5000) == "5k-10k"
+        assert get_token_bucket(7500) == "5k-10k"
+        assert get_token_bucket(9999) == "5k-10k"
+
+    def test_bucket_10k_50k(self):
+        """Test tokens 10000-49999 return '10k-50k' bucket."""
+        assert get_token_bucket(10000) == "10k-50k"
+        assert get_token_bucket(25000) == "10k-50k"
+        assert get_token_bucket(49999) == "10k-50k"
+
+    def test_bucket_50k_plus(self):
+        """Test tokens >= 50000 return '50k+' bucket."""
+        assert get_token_bucket(50000) == "50k+"
+        assert get_token_bucket(100000) == "50k+"
+        assert get_token_bucket(1000000) == "50k+"
+
+
+class TestLLMMetrics:
+    """Tests for LLM-specific metrics recording."""
+
+    @pytest.fixture
+    def mock_meter(self):
+        """Create a mock meter for testing."""
+        meter = MagicMock()
+        # Create separate mocks for each histogram (operation_duration, llm_duration)
+        histogram_mocks = [MagicMock(), MagicMock()]
+        meter.create_histogram.side_effect = histogram_mocks
+        # Create separate mocks for each counter
+        # (operation_total, llm_tokens_input, llm_tokens_output, llm_calls_total)
+        counter_mocks = [MagicMock() for _ in range(4)]
+        meter.create_counter.side_effect = counter_mocks
+        return meter
+
+    @pytest.fixture
+    def collector(self, mock_meter):
+        """Create a MetricsCollector with a mock meter."""
+        with patch("hindsight_api.metrics.get_meter", return_value=mock_meter):
+            return MetricsCollector()
+
+    def test_record_llm_call_records_duration(self, collector):
+        """Test that record_llm_call records duration."""
+        collector.record_llm_call(
+            provider="openai",
+            model="gpt-4",
+            scope="memory",
+            duration=1.5,
+            input_tokens=100,
+            output_tokens=50,
+            success=True,
+        )
+
+        # LLM duration histogram should be called
+        collector.llm_duration.record.assert_called_once()
+        call_args = collector.llm_duration.record.call_args
+
+        # First arg is duration
+        assert call_args[0][0] == 1.5
+
+        # Second arg is attributes dict
+        attributes = call_args[0][1]
+        assert attributes["provider"] == "openai"
+        assert attributes["model"] == "gpt-4"
+        assert attributes["scope"] == "memory"
+        assert attributes["success"] == "true"
+
+    def test_record_llm_call_records_failure(self, collector):
+        """Test that record_llm_call records failure status."""
+        collector.record_llm_call(
+            provider="anthropic",
+            model="claude-3",
+            scope="reflect",
+            duration=0.5,
+            success=False,
+        )
+
+        # Check success is false
+        call_args = collector.llm_duration.record.call_args
+        attributes = call_args[0][1]
+        assert attributes["success"] == "false"
+
+    def test_record_llm_call_records_tokens_with_buckets(self, collector):
+        """Test that record_llm_call records tokens with bucket labels."""
+        collector.record_llm_call(
+            provider="openai",
+            model="gpt-4",
+            scope="memory",
+            duration=1.0,
+            input_tokens=2500,  # Should be "1k-5k" bucket
+            output_tokens=150,  # Should be "100-500" bucket
+            success=True,
+        )
+
+        # Input tokens should be recorded with bucket
+        collector.llm_tokens_input.add.assert_called_once()
+        input_call = collector.llm_tokens_input.add.call_args
+        assert input_call[0][0] == 2500
+        assert input_call[0][1]["token_bucket"] == "1k-5k"
+
+        # Output tokens should be recorded with bucket
+        collector.llm_tokens_output.add.assert_called_once()
+        output_call = collector.llm_tokens_output.add.call_args
+        assert output_call[0][0] == 150
+        assert output_call[0][1]["token_bucket"] == "100-500"
+
+    def test_record_llm_call_skips_zero_tokens(self, collector):
+        """Test that zero token values don't record."""
+        collector.record_llm_call(
+            provider="openai",
+            model="gpt-4",
+            scope="memory",
+            duration=1.0,
+            input_tokens=0,
+            output_tokens=0,
+            success=True,
+        )
+
+        # Token counters should not be called
+        collector.llm_tokens_input.add.assert_not_called()
+        collector.llm_tokens_output.add.assert_not_called()
+
+    def test_record_llm_call_increments_call_counter(self, collector):
+        """Test that record_llm_call increments the call counter."""
+        collector.record_llm_call(
+            provider="gemini",
+            model="gemini-pro",
+            scope="entity_observation",
+            duration=2.0,
+            success=True,
+        )
+
+        # Call counter should be incremented
+        collector.llm_calls_total.add.assert_called_once()
+        call_args = collector.llm_calls_total.add.call_args
+        assert call_args[0][0] == 1
+        assert call_args[0][1]["provider"] == "gemini"
+        assert call_args[0][1]["model"] == "gemini-pro"
+        assert call_args[0][1]["scope"] == "entity_observation"
+
+    def test_record_llm_call_different_scopes(self, collector):
+        """Test recording LLM calls with different scopes."""
+        scopes = ["memory", "reflect", "entity_observation", "answer"]
+
+        for scope in scopes:
+            collector.llm_duration.record.reset_mock()
+
+            collector.record_llm_call(
+                provider="openai",
+                model="gpt-4",
+                scope=scope,
+                duration=1.0,
+                success=True,
+            )
+
+            call_args = collector.llm_duration.record.call_args
+            attributes = call_args[0][1]
+            assert attributes["scope"] == scope
@@ -173,6 +173,107 @@ async def test_regenerate_entity_observations(memory, request_context):
            await conn.execute("DELETE FROM entities WHERE bank_id = $1", bank_id)


+@pytest.mark.asyncio
+async def test_manual_regenerate_with_few_facts(memory, request_context):
+    """
+    Test that manual regeneration works even with fewer than 5 facts.
+
+    This is important because:
+    - Automatic generation during retain requires MIN_FACTS_THRESHOLD (5)
+    - But manual regeneration via API should work with any number of facts
+    - The UI triggers manual regeneration, so it should work regardless of fact count
+    """
+    bank_id = f"test_manual_regen_{datetime.now(timezone.utc).timestamp()}"
+
+    try:
+        # Store only 2 facts - below the automatic threshold
+        await memory.retain_async(
+            bank_id=bank_id,
+            content="Alice works at Google as a senior software engineer.",
+            context="work info",
+            event_date=datetime(2024, 1, 15, tzinfo=timezone.utc),
+            request_context=request_context,
+        )
+        await memory.retain_async(
+            bank_id=bank_id,
+            content="Alice loves hiking and outdoor photography.",
+            context="hobbies",
+            event_date=datetime(2024, 1, 16, tzinfo=timezone.utc),
+            request_context=request_context,
+        )
+
+        # Find the Alice entity
+        pool = await memory._get_pool()
+        async with pool.acquire() as conn:
+            entity_row = await conn.fetchrow(
+                """
+                SELECT id, canonical_name
+                FROM entities
+                WHERE bank_id = $1 AND LOWER(canonical_name) LIKE '%alice%'
+                LIMIT 1
+                """,
+                bank_id
+            )
+
+        assert entity_row is not None, "Alice entity should have been extracted"
+
+        entity_id = str(entity_row['id'])
+        entity_name = entity_row['canonical_name']
+
+        # Check fact count - should be < 5
+        async with pool.acquire() as conn:
+            fact_count = await conn.fetchval(
+                "SELECT COUNT(*) FROM unit_entities WHERE entity_id = $1",
+                entity_row['id']
+            )
+
+        print(f"\n=== Manual Regeneration Test ===")
+        print(f"Entity: {entity_name} (id: {entity_id})")
+        print(f"Linked facts: {fact_count}")
+
+        # Verify we're testing with fewer than the automatic threshold
+        assert fact_count < 5, f"Test requires < 5 facts, but entity has {fact_count}"
+
+        # Before regeneration - should have no observations (auto threshold not met)
+        obs_before = await memory.get_entity_observations(bank_id, entity_id, limit=10, request_context=request_context)
+        print(f"Observations before manual regenerate: {len(obs_before)}")
+
+        # Manually regenerate observations - this should work regardless of fact count
+        created_ids = await memory.regenerate_entity_observations(
+            bank_id=bank_id,
+            entity_id=entity_id,
+            entity_name=entity_name,
+            request_context=request_context,
+        )
+
+        print(f"Observations created by manual regenerate: {len(created_ids)}")
+
+        # Get observations after regeneration
+        observations = await memory.get_entity_observations(bank_id, entity_id, limit=10, request_context=request_context)
+        print(f"Observations after manual regenerate: {len(observations)}")
+        for obs in observations:
+            print(f"  - {obs.text}")
+
+        # Manual regeneration should create observations even with < 5 facts
+        assert len(observations) > 0, \
+            f"Manual regeneration should create observations even with only {fact_count} facts. " \
+            f"The LLM should synthesize at least 1 observation from the available facts."
+
+        # Verify observations contain relevant content
+        obs_texts = " ".join([o.text.lower() for o in observations])
+        assert any(keyword in obs_texts for keyword in ["google", "engineer", "hiking", "photography", "alice"]), \
+            "Observations should contain relevant information about Alice"
+
+        print(f"✓ Manual regeneration works with {fact_count} facts (below automatic threshold of 5)")
+
+    finally:
+        # Cleanup
+        pool = await memory._get_pool()
+        async with pool.acquire() as conn:
+            await conn.execute("DELETE FROM memory_units WHERE bank_id = $1", bank_id)
+            await conn.execute("DELETE FROM entities WHERE bank_id = $1", bank_id)
+
+
@pytest.mark.asyncio
 async def test_search_with_include_entities(memory, request_context):
    """
@@ -1778,3 +1778,71 @@ async def test_temporal_links_within_same_batch(memory, request_context):

    finally:
        await memory.delete_bank(bank_id, request_context=request_context)
+
+
+@pytest.mark.asyncio
+async def test_user_provided_entities(memory, request_context):
+    """
+    Test that user-provided entities are merged with auto-extracted entities.
+
+    This tests the feature added in PR #91 where users can provide entities
+    via the 'entities' field in the retain request. These should be combined
+    with LLM-extracted entities, with case-insensitive deduplication.
+    """
+    bank_id = f"test_user_entities_{datetime.now(timezone.utc).timestamp()}"
+
+    try:
+        # Store content with user-provided entities
+        # The content mentions "Alice" which LLM might extract,
+        # but we also provide "ProjectX" and "ACME Corp" which may not be in the text
+        contents = [
+            {
+                "content": "Alice completed the quarterly report.",
+                "context": "work update",
+                "entities": [
+                    {"text": "ProjectX", "type": "PROJECT"},
+                    {"text": "ACME Corp", "type": "ORG"},
+                    {"text": "Alice"},  # May also be extracted by LLM (dedup test)
+                ],
+            }
+        ]
+
+        result = await memory.retain_batch_async(
+            bank_id=bank_id,
+            contents=contents,
+            request_context=request_context,
+        )
+
+        # Flatten the list of lists
+        unit_ids = [uid for sublist in result for uid in sublist]
+        assert len(unit_ids) > 0, "Should have created at least one fact"
+
+        logger.info(f"Created {len(unit_ids)} facts with user-provided entities")
+
+        # Query entity links to verify user-provided entities were stored
+        async with memory._pool.acquire() as conn:
+            # Get all entities linked to our facts via the unit_entities junction table
+            entity_rows = await conn.fetch(
+                """
+                SELECT DISTINCT e.canonical_name
+                FROM entities e
+                JOIN unit_entities ue ON e.id = ue.entity_id
+                WHERE ue.unit_id::text = ANY($1)
+                """,
+                unit_ids
+            )
+
+            entity_names = {row['canonical_name'].lower() for row in entity_rows}
+            logger.info(f"Found entities linked to facts: {[row['canonical_name'] for row in entity_rows]}")
+
+            # Verify user-provided entities are present
+            assert "projectx" in entity_names, "User-provided entity 'ProjectX' should be linked"
+            assert "acme corp" in entity_names, "User-provided entity 'ACME Corp' should be linked"
+
+            # Alice should be present (either from LLM extraction or user-provided)
+            assert "alice" in entity_names, "Entity 'Alice' should be linked"
+
+            logger.info("✓ User-provided entities successfully merged with extracted entities")
+
+    finally:
+        await memory.delete_bank(bank_id, request_context=request_context)
@@ -1,102 +0,0 @@
-name: Release
-
-on:
-  push:
-    tags:
-      - 'v*'
-  workflow_dispatch:
-
-jobs:
-  build:
-    name: Build ${{ matrix.target }}
-    runs-on: ${{ matrix.os }}
-    strategy:
-      matrix:
-        include:
-          - os: ubuntu-latest
-            target: x86_64-unknown-linux-gnu
-            artifact_name: memora
-            release_name: memora-linux-x86_64
-          - os: ubuntu-latest
-            target: aarch64-unknown-linux-gnu
-            artifact_name: memora
-            release_name: memora-linux-arm64
-          - os: macos-latest
-            target: x86_64-apple-darwin
-            artifact_name: memora
-            release_name: memora-macos-x86_64
-          - os: macos-latest
-            target: aarch64-apple-darwin
-            artifact_name: memora
-            release_name: memora-macos-arm64
-
-    steps:
-      - name: Checkout code
-        uses: actions/checkout@v4
-
-      - name: Install Rust
-        uses: dtolnay/rust-toolchain@stable
-        with:
-          targets: ${{ matrix.target }}
-
-      - name: Install cross-compilation tools (Linux ARM64)
-        if: matrix.target == 'aarch64-unknown-linux-gnu'
-        run: |
-          sudo apt-get update
-          sudo apt-get install -y gcc-aarch64-linux-gnu
-
-      - name: Build
-        run: cargo build --release --target ${{ matrix.target }}
-
-      - name: Strip binary (Linux)
-        if: runner.os == 'Linux'
-        run: strip target/${{ matrix.target }}/release/${{ matrix.artifact_name }}
-
-      - name: Strip binary (macOS)
-        if: runner.os == 'macOS'
-        run: strip target/${{ matrix.target }}/release/${{ matrix.artifact_name }}
-
-      - name: Upload artifact
-        uses: actions/upload-artifact@v4
-        with:
-          name: ${{ matrix.release_name }}
-          path: target/${{ matrix.target }}/release/${{ matrix.artifact_name }}
-
-  release:
-    name: Create Release
-    needs: build
-    runs-on: ubuntu-latest
-    if: startsWith(github.ref, 'refs/tags/')
-
-    steps:
-      - name: Checkout code
-        uses: actions/checkout@v4
-
-      - name: Download all artifacts
-        uses: actions/download-artifact@v4
-        with:
-          path: artifacts
-
-      - name: Create checksums
-        run: |
-          cd artifacts
-          for dir in */; do
-            cd "$dir"
-            sha256sum * > SHA256SUMS
-            cd ..
-          done
-
-      - name: Create Release
-        uses: softprops/action-gh-release@v1
-        with:
-          files: |
-            artifacts/memora-linux-x86_64/memora
-            artifacts/memora-linux-arm64/memora
-            artifacts/memora-macos-x86_64/memora
-            artifacts/memora-macos-arm64/memora
-            artifacts/*/SHA256SUMS
-          draft: false
-          prerelease: false
-          generate_release_notes: true
-        env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@@ -1,6 +1,6 @@
 [package]
 name = "hindsight-cli"
-version = "0.1.12"
+version = "0.2.1"
 edition = "2021"
 authors = ["Hindsight Team"]
 description = "A beautiful CLI for Hindsight - semantic memory system"
@@ -0,0 +1,128 @@
+#!/bin/bash
+# CLI smoke test - verifies basic CLI functionality against a running API server
+#
+# Prerequisites:
+#   - hindsight CLI must be in PATH or HINDSIGHT_CLI env var set
+#   - API server must be running at HINDSIGHT_API_URL (default: http://localhost:8888)
+#
+# Usage:
+#   ./hindsight-cli/smoke-test.sh
+#   HINDSIGHT_CLI=/path/to/hindsight ./hindsight-cli/smoke-test.sh
+
+set -e
+
+# Configuration
+HINDSIGHT_CLI="${HINDSIGHT_CLI:-hindsight}"
+export HINDSIGHT_API_URL="${HINDSIGHT_API_URL:-http://localhost:8888}"
+TEST_BANK="cli-smoke-test-$(date +%s)"
+
+echo "=== Hindsight CLI Smoke Test ==="
+echo "CLI: $HINDSIGHT_CLI"
+echo "API URL: $HINDSIGHT_API_URL"
+echo "Test bank: $TEST_BANK"
+echo ""
+
+# Helper function
+run_test() {
+    local name="$1"
+    shift
+    echo -n "Testing: $name... "
+    if "$@" > /tmp/cli-test-output.txt 2>&1; then
+        echo "OK"
+        return 0
+    else
+        echo "FAILED"
+        echo "  Command: $*"
+        echo "  Output:"
+        cat /tmp/cli-test-output.txt | sed 's/^/    /'
+        return 1
+    fi
+}
+
+run_test_output() {
+    local name="$1"
+    local expected="$2"
+    shift 2
+    echo -n "Testing: $name... "
+    if "$@" > /tmp/cli-test-output.txt 2>&1; then
+        if grep -qi "$expected" /tmp/cli-test-output.txt; then
+            echo "OK"
+            return 0
+        else
+            echo "FAILED (expected '$expected' not found)"
+            echo "  Command: $*"
+            echo "  Output:"
+            cat /tmp/cli-test-output.txt | sed 's/^/    /'
+            return 1
+        fi
+    else
+        echo "FAILED"
+        echo "  Command: $*"
+        echo "  Output:"
+        cat /tmp/cli-test-output.txt | sed 's/^/    /'
+        return 1
+    fi
+}
+
+cleanup() {
+    echo ""
+    echo "Cleaning up test bank..."
+    "$HINDSIGHT_CLI" bank delete "$TEST_BANK" 2>/dev/null || true
+}
+trap cleanup EXIT
+
+FAILED=0
+
+# Test 1: Version
+run_test "version" "$HINDSIGHT_CLI" --version || FAILED=1
+
+# Test 2: Help
+run_test "help" "$HINDSIGHT_CLI" --help || FAILED=1
+
+# Test 3: Configure help
+run_test "configure help" "$HINDSIGHT_CLI" configure --help || FAILED=1
+
+# Test 4: List banks (JSON output)
+run_test "list banks" "$HINDSIGHT_CLI" bank list -o json || FAILED=1
+
+# Test 5: Set bank name (creates the bank)
+run_test "set bank name" "$HINDSIGHT_CLI" bank name "$TEST_BANK" "CLI Smoke Test Bank" || FAILED=1
+
+# Test 6: Get bank disposition
+run_test_output "get bank disposition" "CLI Smoke Test Bank" "$HINDSIGHT_CLI" bank disposition "$TEST_BANK" || FAILED=1
+
+# Test 7: Retain memory
+run_test "retain memory" "$HINDSIGHT_CLI" memory retain "$TEST_BANK" "Alice is a software engineer who loves Rust programming" || FAILED=1
+
+# Test 8: Retain more memories
+run_test "retain more memories" "$HINDSIGHT_CLI" memory retain "$TEST_BANK" "Bob is Alice's colleague who prefers Python" || FAILED=1
+
+# Test 9: Recall memories
+run_test_output "recall memories" "Alice" "$HINDSIGHT_CLI" memory recall "$TEST_BANK" "Who is Alice?" || FAILED=1
+
+# Test 10: Reflect on memories
+run_test_output "reflect" "Alice" "$HINDSIGHT_CLI" memory reflect "$TEST_BANK" "What do you know about Alice?" || FAILED=1
+
+# Test 11: Get bank stats
+run_test "bank stats" "$HINDSIGHT_CLI" bank stats "$TEST_BANK" || FAILED=1
+
+# Test 12: List entities
+run_test "list entities" "$HINDSIGHT_CLI" entity list "$TEST_BANK" || FAILED=1
+
+# Test 13: List documents
+run_test "list documents" "$HINDSIGHT_CLI" document list "$TEST_BANK" || FAILED=1
+
+# Test 14: Clear memories
+run_test "clear memories" "$HINDSIGHT_CLI" memory clear "$TEST_BANK" || FAILED=1
+
+# Test 15: Delete bank
+run_test "delete bank" "$HINDSIGHT_CLI" bank delete "$TEST_BANK" || FAILED=1
+
+echo ""
+if [ $FAILED -eq 0 ]; then
+    echo "=== All smoke tests passed! ==="
+    exit 0
+else
+    echo "=== Some smoke tests failed ==="
+    exit 1
+fi
@@ -89,14 +89,14 @@ impl ApiClient {

    pub fn list_agents(&self, _verbose: bool) -> Result<Vec<types::BankListItem>> {
        self.runtime.block_on(async {
-            let response = self.client.list_banks().await?;
+            let response = self.client.list_banks(None).await?;
            Ok(response.into_inner().banks)
        })
    }

    pub fn get_profile(&self, agent_id: &str, _verbose: bool) -> Result<types::BankProfileResponse> {
        self.runtime.block_on(async {
-            let response = self.client.get_bank_profile(agent_id).await?;
+            let response = self.client.get_bank_profile(agent_id, None).await?;
            Ok(response.into_inner())
        })
    }
@@ -105,7 +105,9 @@ impl ApiClient {
        self.runtime.block_on(async {
            let response = self.client.get_agent_stats(agent_id).await?;
            let value = response.into_inner();
-            let stats: AgentStats = serde_json::from_value(value)?;
+            // Convert to JSON Value first, then parse into our type
+            let json_value = serde_json::to_value(&value)?;
+            let stats: AgentStats = serde_json::from_value(json_value)?;
            Ok(stats)
        })
    }
@@ -117,7 +119,7 @@ impl ApiClient {
                background: None,
                disposition: None,
            };
-            let response = self.client.create_or_update_bank(agent_id, &request).await?;
+            let response = self.client.create_or_update_bank(agent_id, None, &request).await?;
            Ok(response.into_inner())
        })
    }
@@ -128,7 +130,7 @@ impl ApiClient {
                content: content.to_string(),
                update_disposition,
            };
-            let response = self.client.add_bank_background(agent_id, &request).await?;
+            let response = self.client.add_bank_background(agent_id, None, &request).await?;
            Ok(response.into_inner())
        })
    }
@@ -138,21 +140,21 @@ impl ApiClient {
            eprintln!("Request body: {}", serde_json::to_string_pretty(request).unwrap_or_default());
        }
        self.runtime.block_on(async {
-            let response = self.client.recall_memories(agent_id, request).await?;
+            let response = self.client.recall_memories(agent_id, None, request).await?;
            Ok(response.into_inner())
        })
    }

    pub fn reflect(&self, agent_id: &str, request: &types::ReflectRequest, _verbose: bool) -> Result<types::ReflectResponse> {
        self.runtime.block_on(async {
-            let response = self.client.reflect(agent_id, request).await?;
+            let response = self.client.reflect(agent_id, None, request).await?;
            Ok(response.into_inner())
        })
    }

    pub fn retain(&self, agent_id: &str, request: &types::RetainRequest, _async_mode: bool, _verbose: bool) -> Result<MemoryPutResult> {
        self.runtime.block_on(async {
-            let response = self.client.retain_memories(agent_id, request).await?;
+            let response = self.client.retain_memories(agent_id, None, request).await?;
            let result = response.into_inner();
            Ok(MemoryPutResult {
                success: result.success,
@@ -170,7 +172,7 @@ impl ApiClient {

    pub fn clear_memories(&self, agent_id: &str, fact_type: Option<&str>, _verbose: bool) -> Result<types::DeleteResponse> {
        self.runtime.block_on(async {
-            let response = self.client.clear_bank_memories(agent_id, fact_type).await?;
+            let response = self.client.clear_bank_memories(agent_id, None, Some(fact_type)).await?;
            Ok(response.into_inner())
        })
    }
@@ -181,7 +183,8 @@ impl ApiClient {
                agent_id,
                limit.map(|l| l as i64),
                offset.map(|o| o as i64),
-                q
+                q,
+                None,
            ).await?;
            Ok(response.into_inner())
        })
@@ -189,69 +192,79 @@ impl ApiClient {

    pub fn get_document(&self, agent_id: &str, document_id: &str, _verbose: bool) -> Result<types::DocumentResponse> {
        self.runtime.block_on(async {
-            let response = self.client.get_document(agent_id, document_id).await?;
+            let response = self.client.get_document(agent_id, document_id, None).await?;
            Ok(response.into_inner())
        })
    }

    pub fn delete_document(&self, agent_id: &str, document_id: &str, _verbose: bool) -> Result<types::DeleteResponse> {
        self.runtime.block_on(async {
-            let response = self.client.delete_document(agent_id, document_id).await?;
+            let response = self.client.delete_document(agent_id, document_id, None).await?;
            let value = response.into_inner();
-            let result: types::DeleteResponse = serde_json::from_value(value)?;
-            Ok(result)
+            // Convert typed response to DeleteResponse
+            Ok(types::DeleteResponse {
+                deleted_count: Some(value.memory_units_deleted),
+                message: Some(value.message),
+                success: value.success,
+            })
        })
    }

    pub fn list_operations(&self, agent_id: &str, _verbose: bool) -> Result<OperationsResponse> {
        self.runtime.block_on(async {
-            let response = self.client.list_operations(agent_id).await?;
+            let response = self.client.list_operations(agent_id, None).await?;
            let value = response.into_inner();
-            let ops: OperationsResponse = serde_json::from_value(value)?;
+            // Convert to JSON Value first, then parse into our type
+            let json_value = serde_json::to_value(&value)?;
+            let ops: OperationsResponse = serde_json::from_value(json_value)?;
            Ok(ops)
        })
    }

    pub fn cancel_operation(&self, agent_id: &str, operation_id: &str, _verbose: bool) -> Result<types::DeleteResponse> {
        self.runtime.block_on(async {
-            let response = self.client.cancel_operation(agent_id, operation_id).await?;
+            let response = self.client.cancel_operation(agent_id, operation_id, None).await?;
            let value = response.into_inner();
-            let result: types::DeleteResponse = serde_json::from_value(value)?;
-            Ok(result)
+            // Convert typed response to DeleteResponse
+            Ok(types::DeleteResponse {
+                deleted_count: None,
+                message: Some(value.message),
+                success: value.success,
+            })
        })
    }

    pub fn list_memories(&self, bank_id: &str, type_filter: Option<&str>, q: Option<&str>, limit: Option<i64>, offset: Option<i64>, _verbose: bool) -> Result<types::ListMemoryUnitsResponse> {
        self.runtime.block_on(async {
-            let response = self.client.list_memories(bank_id, limit, offset, q, type_filter).await?;
+            let response = self.client.list_memories(bank_id, limit, offset, q, type_filter, None).await?;
            Ok(response.into_inner())
        })
    }

    pub fn list_entities(&self, bank_id: &str, limit: Option<i64>, _verbose: bool) -> Result<types::EntityListResponse> {
        self.runtime.block_on(async {
-            let response = self.client.list_entities(bank_id, limit).await?;
+            let response = self.client.list_entities(bank_id, limit, None).await?;
            Ok(response.into_inner())
        })
    }

    pub fn get_entity(&self, bank_id: &str, entity_id: &str, _verbose: bool) -> Result<types::EntityDetailResponse> {
        self.runtime.block_on(async {
-            let response = self.client.get_entity(bank_id, entity_id).await?;
+            let response = self.client.get_entity(bank_id, entity_id, None).await?;
            Ok(response.into_inner())
        })
    }

    pub fn regenerate_entity(&self, bank_id: &str, entity_id: &str, _verbose: bool) -> Result<types::EntityDetailResponse> {
        self.runtime.block_on(async {
-            let response = self.client.regenerate_entity_observations(bank_id, entity_id).await?;
+            let response = self.client.regenerate_entity_observations(bank_id, entity_id, None).await?;
            Ok(response.into_inner())
        })
    }

    pub fn delete_bank(&self, bank_id: &str, _verbose: bool) -> Result<types::DeleteResponse> {
        self.runtime.block_on(async {
-            let response = self.client.delete_bank(bank_id).await?;
+            let response = self.client.delete_bank(bank_id, None).await?;
            Ok(response.into_inner())
        })
    }
@@ -354,7 +354,9 @@ impl App {
                            query: query_text,
                            budget: Some(query_budget),
                            context: None,
+                            max_tokens: 4096,
                            include: None,
+                            response_schema: None,
                        };

                        let result = client.reflect(&bank_id, &request, false)
@@ -10,6 +10,7 @@ use crate::ui;

 // Import types from generated client
 use hindsight_client::types::{Budget, ChunkIncludeOptions, IncludeOptions};
+use serde_json;

 // Helper function to parse budget string to Budget enum
 fn parse_budget(budget: &str) -> Budget {
@@ -86,6 +87,8 @@ pub fn reflect(
    query: String,
    budget: String,
    context: Option<String>,
+    max_tokens: Option<i64>,
+    schema_path: Option<PathBuf>,
    verbose: bool,
    output_format: OutputFormat,
 ) -> Result<()> {
@@ -95,11 +98,24 @@ pub fn reflect(
        None
    };

+    // Load and parse schema if provided
+    let response_schema = if let Some(path) = schema_path {
+        let schema_content = fs::read_to_string(&path)
+            .with_context(|| format!("Failed to read schema file: {}", path.display()))?;
+        let schema: serde_json::Map<String, serde_json::Value> = serde_json::from_str(&schema_content)
+            .with_context(|| format!("Failed to parse JSON schema from: {}", path.display()))?;
+        Some(schema)
+    } else {
+        None
+    };
+
    let request = ReflectRequest {
        query,
        budget: Some(parse_budget(&budget)),
        context,
+        max_tokens: max_tokens.unwrap_or(4096),
        include: None,
+        response_schema,
    };

    let response = client.reflect(agent_id, &request, verbose);
@@ -145,6 +161,7 @@ pub fn retain(
        metadata: None,
        timestamp: None,
        document_id: Some(doc_id.clone()),
+        entities: None,
    };

    let request = RetainRequest {
@@ -254,6 +271,7 @@ pub fn retain_files(
            metadata: None,
            timestamp: None,
            document_id: Some(doc_id),
+            entities: None,
        });

        pb.inc(1);
@@ -206,6 +206,14 @@ enum MemoryCommands {
        /// Additional context
        #[arg(short = 'c', long)]
        context: Option<String>,
+
+        /// Maximum tokens for the response (server default: 4096)
+        #[arg(short = 'm', long)]
+        max_tokens: Option<i64>,
+
+        /// Path to JSON schema file for structured output
+        #[arg(short = 's', long)]
+        schema: Option<PathBuf>,
    },

    /// Store (retain) a single memory
@@ -421,8 +429,8 @@ fn run() -> Result<()> {
            MemoryCommands::Recall { bank_id, query, fact_type, budget, max_tokens, trace, include_chunks, chunk_max_tokens } => {
                commands::memory::recall(&client, &bank_id, query, fact_type, budget, max_tokens, trace, include_chunks, chunk_max_tokens, verbose, output_format)
            }
-            MemoryCommands::Reflect { bank_id, query, budget, context } => {
-                commands::memory::reflect(&client, &bank_id, query, budget, context, verbose, output_format)
+            MemoryCommands::Reflect { bank_id, query, budget, context, max_tokens, schema } => {
+                commands::memory::reflect(&client, &bank_id, query, budget, context, max_tokens, schema, verbose, output_format)
            }
            MemoryCommands::Retain { bank_id, content, doc_id, context, r#async } => {
                commands::memory::retain(&client, &bank_id, content, doc_id, context, r#async, verbose, output_format)
@@ -175,6 +175,16 @@ pub fn print_think_response(response: &ReflectResponse) {
    if !response.based_on.is_empty() {
        println!("{}", dim(&format!("Based on {} memory units", response.based_on.len())));
    }
+
+    // Display structured output if present
+    if let Some(structured) = &response.structured_output {
+        println!();
+        println!("{}", gradient_text("─── Structured Output ───"));
+        println!();
+        if let Ok(json) = serde_json::to_string_pretty(structured) {
+            println!("{}", json);
+        }
+    }
 }

 pub fn print_trace_info(trace: &serde_json::Map<String, serde_json::Value>) {
@@ -1,49 +1,14 @@
 hindsight_client_api/__init__.py
 hindsight_client_api/api/__init__.py
-hindsight_client_api/api/default_api.py
+hindsight_client_api/api/banks_api.py
+hindsight_client_api/api/documents_api.py
+hindsight_client_api/api/entities_api.py
+hindsight_client_api/api/memory_api.py
 hindsight_client_api/api/monitoring_api.py
+hindsight_client_api/api/operations_api.py
 hindsight_client_api/api_client.py
 hindsight_client_api/api_response.py
 hindsight_client_api/configuration.py
-hindsight_client_api/docs/AddBackgroundRequest.md
-hindsight_client_api/docs/BackgroundResponse.md
-hindsight_client_api/docs/BankListItem.md
-hindsight_client_api/docs/BankListResponse.md
-hindsight_client_api/docs/BankProfileResponse.md
-hindsight_client_api/docs/Budget.md
-hindsight_client_api/docs/ChunkData.md
-hindsight_client_api/docs/ChunkIncludeOptions.md
-hindsight_client_api/docs/ChunkResponse.md
-hindsight_client_api/docs/CreateBankRequest.md
-hindsight_client_api/docs/DefaultApi.md
-hindsight_client_api/docs/DeleteResponse.md
-hindsight_client_api/docs/DispositionTraits.md
-hindsight_client_api/docs/DocumentResponse.md
-hindsight_client_api/docs/EntityDetailResponse.md
-hindsight_client_api/docs/EntityIncludeOptions.md
-hindsight_client_api/docs/EntityListItem.md
-hindsight_client_api/docs/EntityListResponse.md
-hindsight_client_api/docs/EntityObservationResponse.md
-hindsight_client_api/docs/EntityStateResponse.md
-hindsight_client_api/docs/GraphDataResponse.md
-hindsight_client_api/docs/HTTPValidationError.md
-hindsight_client_api/docs/IncludeOptions.md
-hindsight_client_api/docs/ListDocumentsResponse.md
-hindsight_client_api/docs/ListMemoryUnitsResponse.md
-hindsight_client_api/docs/MemoryItem.md
-hindsight_client_api/docs/MonitoringApi.md
-hindsight_client_api/docs/RecallRequest.md
-hindsight_client_api/docs/RecallResponse.md
-hindsight_client_api/docs/RecallResult.md
-hindsight_client_api/docs/ReflectFact.md
-hindsight_client_api/docs/ReflectIncludeOptions.md
-hindsight_client_api/docs/ReflectRequest.md
-hindsight_client_api/docs/ReflectResponse.md
-hindsight_client_api/docs/RetainRequest.md
-hindsight_client_api/docs/RetainResponse.md
-hindsight_client_api/docs/UpdateDispositionRequest.md
-hindsight_client_api/docs/ValidationError.md
-hindsight_client_api/docs/ValidationErrorLocInner.md
 hindsight_client_api/exceptions.py
 hindsight_client_api/models/__init__.py
 hindsight_client_api/models/add_background_request.py
@@ -51,16 +16,20 @@ hindsight_client_api/models/background_response.py
 hindsight_client_api/models/bank_list_item.py
 hindsight_client_api/models/bank_list_response.py
 hindsight_client_api/models/bank_profile_response.py
+hindsight_client_api/models/bank_stats_response.py
 hindsight_client_api/models/budget.py
+hindsight_client_api/models/cancel_operation_response.py
 hindsight_client_api/models/chunk_data.py
 hindsight_client_api/models/chunk_include_options.py
 hindsight_client_api/models/chunk_response.py
 hindsight_client_api/models/create_bank_request.py
+hindsight_client_api/models/delete_document_response.py
 hindsight_client_api/models/delete_response.py
 hindsight_client_api/models/disposition_traits.py
 hindsight_client_api/models/document_response.py
 hindsight_client_api/models/entity_detail_response.py
 hindsight_client_api/models/entity_include_options.py
+hindsight_client_api/models/entity_input.py
 hindsight_client_api/models/entity_list_item.py
 hindsight_client_api/models/entity_list_response.py
 hindsight_client_api/models/entity_observation_response.py
@@ -71,6 +40,8 @@ hindsight_client_api/models/include_options.py
 hindsight_client_api/models/list_documents_response.py
 hindsight_client_api/models/list_memory_units_response.py
 hindsight_client_api/models/memory_item.py
+hindsight_client_api/models/operation_response.py
+hindsight_client_api/models/operations_list_response.py
 hindsight_client_api/models/recall_request.py
 hindsight_client_api/models/recall_response.py
 hindsight_client_api/models/recall_result.py
@@ -80,48 +51,9 @@ hindsight_client_api/models/reflect_request.py
 hindsight_client_api/models/reflect_response.py
 hindsight_client_api/models/retain_request.py
 hindsight_client_api/models/retain_response.py
+hindsight_client_api/models/token_usage.py
 hindsight_client_api/models/update_disposition_request.py
 hindsight_client_api/models/validation_error.py
 hindsight_client_api/models/validation_error_loc_inner.py
 hindsight_client_api/rest.py
-hindsight_client_api/test/__init__.py
-hindsight_client_api/test/test_add_background_request.py
-hindsight_client_api/test/test_background_response.py
-hindsight_client_api/test/test_bank_list_item.py
-hindsight_client_api/test/test_bank_list_response.py
-hindsight_client_api/test/test_bank_profile_response.py
-hindsight_client_api/test/test_budget.py
-hindsight_client_api/test/test_chunk_data.py
-hindsight_client_api/test/test_chunk_include_options.py
-hindsight_client_api/test/test_chunk_response.py
-hindsight_client_api/test/test_create_bank_request.py
-hindsight_client_api/test/test_default_api.py
-hindsight_client_api/test/test_delete_response.py
-hindsight_client_api/test/test_disposition_traits.py
-hindsight_client_api/test/test_document_response.py
-hindsight_client_api/test/test_entity_detail_response.py
-hindsight_client_api/test/test_entity_include_options.py
-hindsight_client_api/test/test_entity_list_item.py
-hindsight_client_api/test/test_entity_list_response.py
-hindsight_client_api/test/test_entity_observation_response.py
-hindsight_client_api/test/test_entity_state_response.py
-hindsight_client_api/test/test_graph_data_response.py
-hindsight_client_api/test/test_http_validation_error.py
-hindsight_client_api/test/test_include_options.py
-hindsight_client_api/test/test_list_documents_response.py
-hindsight_client_api/test/test_list_memory_units_response.py
-hindsight_client_api/test/test_memory_item.py
-hindsight_client_api/test/test_monitoring_api.py
-hindsight_client_api/test/test_recall_request.py
-hindsight_client_api/test/test_recall_response.py
-hindsight_client_api/test/test_recall_result.py
-hindsight_client_api/test/test_reflect_fact.py
-hindsight_client_api/test/test_reflect_include_options.py
-hindsight_client_api/test/test_reflect_request.py
-hindsight_client_api/test/test_reflect_response.py
-hindsight_client_api/test/test_retain_request.py
-hindsight_client_api/test/test_retain_response.py
-hindsight_client_api/test/test_update_disposition_request.py
-hindsight_client_api/test/test_validation_error.py
-hindsight_client_api/test/test_validation_error_loc_inner.py
 hindsight_client_api_README.md
@@ -1 +1 @@
-7.18.0-SNAPSHOT
+7.10.0
@@ -10,7 +10,7 @@ from typing import Optional, List, Dict, Any
 from datetime import datetime

 import hindsight_client_api
-from hindsight_client_api.api import default_api
+from hindsight_client_api.api import memory_api, banks_api
 from hindsight_client_api.models import (
    recall_request,
    retain_request,
@@ -74,7 +74,10 @@ class Hindsight:
        """
        config = hindsight_client_api.Configuration(host=base_url, access_token=api_key)
        self._api_client = hindsight_client_api.ApiClient(config)
-        self._api = default_api.DefaultApi(self._api_client)
+        if api_key:
+            self._api_client.set_default_header("Authorization", f"Bearer {api_key}")
+        self._memory_api = memory_api.MemoryApi(self._api_client)
+        self._banks_api = banks_api.BanksApi(self._api_client)

    def __enter__(self):
        """Context manager entry."""
@@ -85,9 +88,21 @@ class Hindsight:
        self.close()

    def close(self):
-        """Close the API client."""
+        """Close the API client (sync version - use aclose() in async code)."""
        if self._api_client:
-            _run_async(self._api_client.close())
+            try:
+                loop = asyncio.get_running_loop()
+                # We're in an async context - schedule but don't wait
+                # The caller should use aclose() instead
+                loop.create_task(self._api_client.close())
+            except RuntimeError:
+                # No running loop - safe to run synchronously
+                _run_async(self._api_client.close())
+
+    async def aclose(self):
+        """Close the API client (async version)."""
+        if self._api_client:
+            await self._api_client.close()

    # Simplified methods for main operations

@@ -99,6 +114,7 @@ class Hindsight:
        context: Optional[str] = None,
        document_id: Optional[str] = None,
        metadata: Optional[Dict[str, str]] = None,
+        entities: Optional[List[Dict[str, str]]] = None,
    ) -> RetainResponse:
        """
        Store a single memory (simplified interface).
@@ -110,13 +126,14 @@ class Hindsight:
            context: Optional context description
            document_id: Optional document ID for grouping
            metadata: Optional user-defined metadata
+            entities: Optional list of entities [{"text": "...", "type": "..."}]

        Returns:
            RetainResponse with success status
        """
        return self.retain_batch(
            bank_id=bank_id,
-            items=[{"content": content, "timestamp": timestamp, "context": context, "metadata": metadata}],
+            items=[{"content": content, "timestamp": timestamp, "context": context, "metadata": metadata, "entities": entities}],
            document_id=document_id,
        )

@@ -132,31 +149,41 @@ class Hindsight:

        Args:
            bank_id: The memory bank ID
-            items: List of memory items with 'content' and optional 'timestamp', 'context', 'metadata', 'document_id'
+            items: List of memory items with 'content' and optional 'timestamp', 'context', 'metadata', 'document_id', 'entities'
            document_id: Optional document ID for grouping memories (applied to items that don't have their own)
            retain_async: If True, process asynchronously in background (default: False)

        Returns:
            RetainResponse with success status and item count
        """
-        memory_items = [
-            memory_item.MemoryItem(
-                content=item["content"],
-                timestamp=item.get("timestamp"),
-                context=item.get("context"),
-                metadata=item.get("metadata"),
-                # Use item's document_id if provided, otherwise fall back to batch-level document_id
-                document_id=item.get("document_id") or document_id,
+        from hindsight_client_api.models.entity_input import EntityInput
+
+        memory_items = []
+        for item in items:
+            entities = None
+            if item.get("entities"):
+                entities = [
+                    EntityInput(text=e["text"], type=e.get("type"))
+                    for e in item["entities"]
+                ]
+            memory_items.append(
+                memory_item.MemoryItem(
+                    content=item["content"],
+                    timestamp=item.get("timestamp"),
+                    context=item.get("context"),
+                    metadata=item.get("metadata"),
+                    # Use item's document_id if provided, otherwise fall back to batch-level document_id
+                    document_id=item.get("document_id") or document_id,
+                    entities=entities,
+                )
            )
-            for item in items
-        ]

        request_obj = retain_request.RetainRequest(
            items=memory_items,
            async_=retain_async,
        )

-        return _run_async(self._api.retain_memories(bank_id, request_obj))
+        return _run_async(self._memory_api.retain_memories(bank_id, request_obj))

    def recall(
        self,
@@ -208,7 +235,7 @@ class Hindsight:
            include=include_opts,
        )

-        return _run_async(self._api.recall_memories(bank_id, request_obj))
+        return _run_async(self._memory_api.recall_memories(bank_id, request_obj))

    def reflect(
        self,
@@ -216,6 +243,8 @@ class Hindsight:
        query: str,
        budget: str = "low",
        context: Optional[str] = None,
+        max_tokens: Optional[int] = None,
+        response_schema: Optional[Dict[str, Any]] = None,
    ) -> ReflectResponse:
        """
        Generate a contextual answer based on bank identity and memories.
@@ -225,17 +254,24 @@ class Hindsight:
            query: The question or prompt
            budget: Budget level for reflection - "low", "mid", or "high" (default: "low")
            context: Optional additional context
+            max_tokens: Maximum tokens for the response (server default: 4096)
+            response_schema: Optional JSON Schema for structured output. When provided,
+                the response will include a 'structured_output' field with the LLM
+                response parsed according to this schema.

        Returns:
-            ReflectResponse with answer text and optionally facts used
+            ReflectResponse with answer text, optionally facts used, and optionally
+            structured_output if response_schema was provided
        """
        request_obj = reflect_request.ReflectRequest(
            query=query,
            budget=budget,
            context=context,
+            max_tokens=max_tokens,
+            response_schema=response_schema,
        )

-        return _run_async(self._api.reflect(bank_id, request_obj))
+        return _run_async(self._memory_api.reflect(bank_id, request_obj))

    def list_memories(
        self,
@@ -246,7 +282,7 @@ class Hindsight:
        offset: int = 0,
    ) -> ListMemoryUnitsResponse:
        """List memory units with pagination."""
-        return _run_async(self._api.list_memories(
+        return _run_async(self._memory_api.list_memories(
            bank_id=bank_id,
            type=type,
            q=search_query,
@@ -274,7 +310,7 @@ class Hindsight:
            disposition=disposition_obj,
        )

-        return _run_async(self._api.create_or_update_bank(bank_id, request_obj))
+        return _run_async(self._banks_api.create_or_update_bank(bank_id, request_obj))

    # Async methods (native async, no _run_async wrapper)

@@ -290,31 +326,41 @@ class Hindsight:

        Args:
            bank_id: The memory bank ID
-            items: List of memory items with 'content' and optional 'timestamp', 'context', 'metadata', 'document_id'
+            items: List of memory items with 'content' and optional 'timestamp', 'context', 'metadata', 'document_id', 'entities'
            document_id: Optional document ID for grouping memories (applied to items that don't have their own)
            retain_async: If True, process asynchronously in background (default: False)

        Returns:
            RetainResponse with success status and item count
        """
-        memory_items = [
-            memory_item.MemoryItem(
-                content=item["content"],
-                timestamp=item.get("timestamp"),
-                context=item.get("context"),
-                metadata=item.get("metadata"),
-                # Use item's document_id if provided, otherwise fall back to batch-level document_id
-                document_id=item.get("document_id") or document_id,
+        from hindsight_client_api.models.entity_input import EntityInput
+
+        memory_items = []
+        for item in items:
+            entities = None
+            if item.get("entities"):
+                entities = [
+                    EntityInput(text=e["text"], type=e.get("type"))
+                    for e in item["entities"]
+                ]
+            memory_items.append(
+                memory_item.MemoryItem(
+                    content=item["content"],
+                    timestamp=item.get("timestamp"),
+                    context=item.get("context"),
+                    metadata=item.get("metadata"),
+                    # Use item's document_id if provided, otherwise fall back to batch-level document_id
+                    document_id=item.get("document_id") or document_id,
+                    entities=entities,
+                )
            )
-            for item in items
-        ]

        request_obj = retain_request.RetainRequest(
            items=memory_items,
            async_=retain_async,
        )

-        return await self._api.retain_memories(bank_id, request_obj)
+        return await self._memory_api.retain_memories(bank_id, request_obj)

    async def aretain(
        self,
@@ -324,6 +370,7 @@ class Hindsight:
        context: Optional[str] = None,
        document_id: Optional[str] = None,
        metadata: Optional[Dict[str, str]] = None,
+        entities: Optional[List[Dict[str, str]]] = None,
    ) -> RetainResponse:
        """
        Store a single memory (async).
@@ -335,13 +382,14 @@ class Hindsight:
            context: Optional context description
            document_id: Optional document ID for grouping
            metadata: Optional user-defined metadata
+            entities: Optional list of entities [{"text": "...", "type": "..."}]

        Returns:
            RetainResponse with success status
        """
        return await self.aretain_batch(
            bank_id=bank_id,
-            items=[{"content": content, "timestamp": timestamp, "context": context, "metadata": metadata}],
+            items=[{"content": content, "timestamp": timestamp, "context": context, "metadata": metadata, "entities": entities}],
            document_id=document_id,
        )

@@ -374,7 +422,7 @@ class Hindsight:
            trace=False,
        )

-        response = await self._api.recall_memories(bank_id, request_obj)
+        response = await self._memory_api.recall_memories(bank_id, request_obj)
        return response.results if hasattr(response, 'results') else []

    async def areflect(
@@ -402,4 +450,4 @@ class Hindsight:
            context=context,
        )

-        return await self._api.reflect(bank_id, request_obj)
+        return await self._memory_api.reflect(bank_id, request_obj)
@@ -7,7 +7,7 @@

    HTTP API for Hindsight

-    The version of the OpenAPI document: 1.0.0
+    The version of the OpenAPI document: 0.1.0
    Generated by OpenAPI Generator (https://openapi-generator.tech)

    Do not edit the class manually.
@@ -16,109 +16,67 @@

 __version__ = "0.0.7"

-# Define package exports
-__all__ = [
-    "MonitoringApi",
-    "DefaultApi",
-    "ApiResponse",
-    "ApiClient",
-    "Configuration",
-    "OpenApiException",
-    "ApiTypeError",
-    "ApiValueError",
-    "ApiKeyError",
-    "ApiAttributeError",
-    "ApiException",
-    "AddBackgroundRequest",
-    "BackgroundResponse",
-    "BankListItem",
-    "BankListResponse",
-    "BankProfileResponse",
-    "Budget",
-    "ChunkData",
-    "ChunkIncludeOptions",
-    "ChunkResponse",
-    "CreateBankRequest",
-    "DeleteResponse",
-    "DispositionTraits",
-    "DocumentResponse",
-    "EntityDetailResponse",
-    "EntityIncludeOptions",
-    "EntityListItem",
-    "EntityListResponse",
-    "EntityObservationResponse",
-    "EntityStateResponse",
-    "GraphDataResponse",
-    "HTTPValidationError",
-    "IncludeOptions",
-    "ListDocumentsResponse",
-    "ListMemoryUnitsResponse",
-    "MemoryItem",
-    "RecallRequest",
-    "RecallResponse",
-    "RecallResult",
-    "ReflectFact",
-    "ReflectIncludeOptions",
-    "ReflectRequest",
-    "ReflectResponse",
-    "RetainRequest",
-    "RetainResponse",
-    "UpdateDispositionRequest",
-    "ValidationError",
-    "ValidationErrorLocInner",
-]
-
 # import apis into sdk package
-from hindsight_client_api.api.monitoring_api import MonitoringApi as MonitoringApi
-from hindsight_client_api.api.default_api import DefaultApi as DefaultApi
+from hindsight_client_api.api.banks_api import BanksApi
+from hindsight_client_api.api.documents_api import DocumentsApi
+from hindsight_client_api.api.entities_api import EntitiesApi
+from hindsight_client_api.api.memory_api import MemoryApi
+from hindsight_client_api.api.monitoring_api import MonitoringApi
+from hindsight_client_api.api.operations_api import OperationsApi

 # import ApiClient
-from hindsight_client_api.api_response import ApiResponse as ApiResponse
-from hindsight_client_api.api_client import ApiClient as ApiClient
-from hindsight_client_api.configuration import Configuration as Configuration
-from hindsight_client_api.exceptions import OpenApiException as OpenApiException
-from hindsight_client_api.exceptions import ApiTypeError as ApiTypeError
-from hindsight_client_api.exceptions import ApiValueError as ApiValueError
-from hindsight_client_api.exceptions import ApiKeyError as ApiKeyError
-from hindsight_client_api.exceptions import ApiAttributeError as ApiAttributeError
-from hindsight_client_api.exceptions import ApiException as ApiException
+from hindsight_client_api.api_response import ApiResponse
+from hindsight_client_api.api_client import ApiClient
+from hindsight_client_api.configuration import Configuration
+from hindsight_client_api.exceptions import OpenApiException
+from hindsight_client_api.exceptions import ApiTypeError
+from hindsight_client_api.exceptions import ApiValueError
+from hindsight_client_api.exceptions import ApiKeyError
+from hindsight_client_api.exceptions import ApiAttributeError
+from hindsight_client_api.exceptions import ApiException

 # import models into sdk package
-from hindsight_client_api.models.add_background_request import AddBackgroundRequest as AddBackgroundRequest
-from hindsight_client_api.models.background_response import BackgroundResponse as BackgroundResponse
-from hindsight_client_api.models.bank_list_item import BankListItem as BankListItem
-from hindsight_client_api.models.bank_list_response import BankListResponse as BankListResponse
-from hindsight_client_api.models.bank_profile_response import BankProfileResponse as BankProfileResponse
-from hindsight_client_api.models.budget import Budget as Budget
-from hindsight_client_api.models.chunk_data import ChunkData as ChunkData
-from hindsight_client_api.models.chunk_include_options import ChunkIncludeOptions as ChunkIncludeOptions
-from hindsight_client_api.models.chunk_response import ChunkResponse as ChunkResponse
-from hindsight_client_api.models.create_bank_request import CreateBankRequest as CreateBankRequest
-from hindsight_client_api.models.delete_response import DeleteResponse as DeleteResponse
-from hindsight_client_api.models.disposition_traits import DispositionTraits as DispositionTraits
-from hindsight_client_api.models.document_response import DocumentResponse as DocumentResponse
-from hindsight_client_api.models.entity_detail_response import EntityDetailResponse as EntityDetailResponse
-from hindsight_client_api.models.entity_include_options import EntityIncludeOptions as EntityIncludeOptions
-from hindsight_client_api.models.entity_list_item import EntityListItem as EntityListItem
-from hindsight_client_api.models.entity_list_response import EntityListResponse as EntityListResponse
-from hindsight_client_api.models.entity_observation_response import EntityObservationResponse as EntityObservationResponse
-from hindsight_client_api.models.entity_state_response import EntityStateResponse as EntityStateResponse
-from hindsight_client_api.models.graph_data_response import GraphDataResponse as GraphDataResponse
-from hindsight_client_api.models.http_validation_error import HTTPValidationError as HTTPValidationError
-from hindsight_client_api.models.include_options import IncludeOptions as IncludeOptions
-from hindsight_client_api.models.list_documents_response import ListDocumentsResponse as ListDocumentsResponse
-from hindsight_client_api.models.list_memory_units_response import ListMemoryUnitsResponse as ListMemoryUnitsResponse
-from hindsight_client_api.models.memory_item import MemoryItem as MemoryItem
-from hindsight_client_api.models.recall_request import RecallRequest as RecallRequest
-from hindsight_client_api.models.recall_response import RecallResponse as RecallResponse
-from hindsight_client_api.models.recall_result import RecallResult as RecallResult
-from hindsight_client_api.models.reflect_fact import ReflectFact as ReflectFact
-from hindsight_client_api.models.reflect_include_options import ReflectIncludeOptions as ReflectIncludeOptions
-from hindsight_client_api.models.reflect_request import ReflectRequest as ReflectRequest
-from hindsight_client_api.models.reflect_response import ReflectResponse as ReflectResponse
-from hindsight_client_api.models.retain_request import RetainRequest as RetainRequest
-from hindsight_client_api.models.retain_response import RetainResponse as RetainResponse
-from hindsight_client_api.models.update_disposition_request import UpdateDispositionRequest as UpdateDispositionRequest
-from hindsight_client_api.models.validation_error import ValidationError as ValidationError
-from hindsight_client_api.models.validation_error_loc_inner import ValidationErrorLocInner as ValidationErrorLocInner
-
+from hindsight_client_api.models.add_background_request import AddBackgroundRequest
+from hindsight_client_api.models.background_response import BackgroundResponse
+from hindsight_client_api.models.bank_list_item import BankListItem
+from hindsight_client_api.models.bank_list_response import BankListResponse
+from hindsight_client_api.models.bank_profile_response import BankProfileResponse
+from hindsight_client_api.models.bank_stats_response import BankStatsResponse
+from hindsight_client_api.models.budget import Budget
+from hindsight_client_api.models.cancel_operation_response import CancelOperationResponse
+from hindsight_client_api.models.chunk_data import ChunkData
+from hindsight_client_api.models.chunk_include_options import ChunkIncludeOptions
+from hindsight_client_api.models.chunk_response import ChunkResponse
+from hindsight_client_api.models.create_bank_request import CreateBankRequest
+from hindsight_client_api.models.delete_document_response import DeleteDocumentResponse
+from hindsight_client_api.models.delete_response import DeleteResponse
+from hindsight_client_api.models.disposition_traits import DispositionTraits
+from hindsight_client_api.models.document_response import DocumentResponse
+from hindsight_client_api.models.entity_detail_response import EntityDetailResponse
+from hindsight_client_api.models.entity_include_options import EntityIncludeOptions
+from hindsight_client_api.models.entity_input import EntityInput
+from hindsight_client_api.models.entity_list_item import EntityListItem
+from hindsight_client_api.models.entity_list_response import EntityListResponse
+from hindsight_client_api.models.entity_observation_response import EntityObservationResponse
+from hindsight_client_api.models.entity_state_response import EntityStateResponse
+from hindsight_client_api.models.graph_data_response import GraphDataResponse
+from hindsight_client_api.models.http_validation_error import HTTPValidationError
+from hindsight_client_api.models.include_options import IncludeOptions
+from hindsight_client_api.models.list_documents_response import ListDocumentsResponse
+from hindsight_client_api.models.list_memory_units_response import ListMemoryUnitsResponse
+from hindsight_client_api.models.memory_item import MemoryItem
+from hindsight_client_api.models.operation_response import OperationResponse
+from hindsight_client_api.models.operations_list_response import OperationsListResponse
+from hindsight_client_api.models.recall_request import RecallRequest
+from hindsight_client_api.models.recall_response import RecallResponse
+from hindsight_client_api.models.recall_result import RecallResult
+from hindsight_client_api.models.reflect_fact import ReflectFact
+from hindsight_client_api.models.reflect_include_options import ReflectIncludeOptions
+from hindsight_client_api.models.reflect_request import ReflectRequest
+from hindsight_client_api.models.reflect_response import ReflectResponse
+from hindsight_client_api.models.retain_request import RetainRequest
+from hindsight_client_api.models.retain_response import RetainResponse
+from hindsight_client_api.models.token_usage import TokenUsage
+from hindsight_client_api.models.update_disposition_request import UpdateDispositionRequest
+from hindsight_client_api.models.validation_error import ValidationError
+from hindsight_client_api.models.validation_error_loc_inner import ValidationErrorLocInner
@@ -1,6 +1,10 @@
 # flake8: noqa

 # import apis into api package
+from hindsight_client_api.api.banks_api import BanksApi
+from hindsight_client_api.api.documents_api import DocumentsApi
+from hindsight_client_api.api.entities_api import EntitiesApi
+from hindsight_client_api.api.memory_api import MemoryApi
 from hindsight_client_api.api.monitoring_api import MonitoringApi
-from hindsight_client_api.api.default_api import DefaultApi
+from hindsight_client_api.api.operations_api import OperationsApi

@@ -0,0 +1,921 @@
+# coding: utf-8
+
+"""
+    Hindsight HTTP API
+
+    HTTP API for Hindsight
+
+    The version of the OpenAPI document: 0.1.0
+    Generated by OpenAPI Generator (https://openapi-generator.tech)
+
+    Do not edit the class manually.
+"""  # noqa: E501
+
+import warnings
+from pydantic import validate_call, Field, StrictFloat, StrictStr, StrictInt
+from typing import Any, Dict, List, Optional, Tuple, Union
+from typing_extensions import Annotated
+
+from pydantic import Field, StrictInt, StrictStr
+from typing import Optional
+from typing_extensions import Annotated
+from hindsight_client_api.models.entity_detail_response import EntityDetailResponse
+from hindsight_client_api.models.entity_list_response import EntityListResponse
+
+from hindsight_client_api.api_client import ApiClient, RequestSerialized
+from hindsight_client_api.api_response import ApiResponse
+from hindsight_client_api.rest import RESTResponseType
+
+
+class EntitiesApi:
+    """NOTE: This class is auto generated by OpenAPI Generator
+    Ref: https://openapi-generator.tech
+
+    Do not edit the class manually.
+    """
+
+    def __init__(self, api_client=None) -> None:
+        if api_client is None:
+            api_client = ApiClient.get_default()
+        self.api_client = api_client
+
+
+    @validate_call
+    async def get_entity(
+        self,
+        bank_id: StrictStr,
+        entity_id: StrictStr,
+        authorization: Optional[StrictStr] = None,
+        _request_timeout: Union[
+            None,
+            Annotated[StrictFloat, Field(gt=0)],
+            Tuple[
+                Annotated[StrictFloat, Field(gt=0)],
+                Annotated[StrictFloat, Field(gt=0)]
+            ]
+        ] = None,
+        _request_auth: Optional[Dict[StrictStr, Any]] = None,
+        _content_type: Optional[StrictStr] = None,
+        _headers: Optional[Dict[StrictStr, Any]] = None,
+        _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0,
+    ) -> EntityDetailResponse:
+        """Get entity details
+
+        Get detailed information about an entity including observations (mental model).
+
+        :param bank_id: (required)
+        :type bank_id: str
+        :param entity_id: (required)
+        :type entity_id: str
+        :param authorization:
+        :type authorization: str
+        :param _request_timeout: timeout setting for this request. If one
+                                 number provided, it will be total request
+                                 timeout. It can also be a pair (tuple) of
+                                 (connection, read) timeouts.
+        :type _request_timeout: int, tuple(int, int), optional
+        :param _request_auth: set to override the auth_settings for an a single
+                              request; this effectively ignores the
+                              authentication in the spec for a single request.
+        :type _request_auth: dict, optional
+        :param _content_type: force content-type for the request.
+        :type _content_type: str, Optional
+        :param _headers: set to override the headers for a single
+                         request; this effectively ignores the headers
+                         in the spec for a single request.
+        :type _headers: dict, optional
+        :param _host_index: set to override the host_index for a single
+                            request; this effectively ignores the host_index
+                            in the spec for a single request.
+        :type _host_index: int, optional
+        :return: Returns the result object.
+        """ # noqa: E501
+
+        _param = self._get_entity_serialize(
+            bank_id=bank_id,
+            entity_id=entity_id,
+            authorization=authorization,
+            _request_auth=_request_auth,
+            _content_type=_content_type,
+            _headers=_headers,
+            _host_index=_host_index
+        )
+
+        _response_types_map: Dict[str, Optional[str]] = {
+            '200': "EntityDetailResponse",
+            '422': "HTTPValidationError",
+        }
+        response_data = await self.api_client.call_api(
+            *_param,
+            _request_timeout=_request_timeout
+        )
+        await response_data.read()
+        return self.api_client.response_deserialize(
+            response_data=response_data,
+            response_types_map=_response_types_map,
+        ).data
+
+
+    @validate_call
+    async def get_entity_with_http_info(
+        self,
+        bank_id: StrictStr,
+        entity_id: StrictStr,
+        authorization: Optional[StrictStr] = None,
+        _request_timeout: Union[
+            None,
+            Annotated[StrictFloat, Field(gt=0)],
+            Tuple[
+                Annotated[StrictFloat, Field(gt=0)],
+                Annotated[StrictFloat, Field(gt=0)]
+            ]
+        ] = None,
+        _request_auth: Optional[Dict[StrictStr, Any]] = None,
+        _content_type: Optional[StrictStr] = None,
+        _headers: Optional[Dict[StrictStr, Any]] = None,
+        _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0,
+    ) -> ApiResponse[EntityDetailResponse]:
+        """Get entity details
+
+        Get detailed information about an entity including observations (mental model).
+
+        :param bank_id: (required)
+        :type bank_id: str
+        :param entity_id: (required)
+        :type entity_id: str
+        :param authorization:
+        :type authorization: str
+        :param _request_timeout: timeout setting for this request. If one
+                                 number provided, it will be total request
+                                 timeout. It can also be a pair (tuple) of
+                                 (connection, read) timeouts.
+        :type _request_timeout: int, tuple(int, int), optional
+        :param _request_auth: set to override the auth_settings for an a single
+                              request; this effectively ignores the
+                              authentication in the spec for a single request.
+        :type _request_auth: dict, optional
+        :param _content_type: force content-type for the request.
+        :type _content_type: str, Optional
+        :param _headers: set to override the headers for a single
+                         request; this effectively ignores the headers
+                         in the spec for a single request.
+        :type _headers: dict, optional
+        :param _host_index: set to override the host_index for a single
+                            request; this effectively ignores the host_index
+                            in the spec for a single request.
+        :type _host_index: int, optional
+        :return: Returns the result object.
+        """ # noqa: E501
+
+        _param = self._get_entity_serialize(
+            bank_id=bank_id,
+            entity_id=entity_id,
+            authorization=authorization,
+            _request_auth=_request_auth,
+            _content_type=_content_type,
+            _headers=_headers,
+            _host_index=_host_index
+        )
+
+        _response_types_map: Dict[str, Optional[str]] = {
+            '200': "EntityDetailResponse",
+            '422': "HTTPValidationError",
+        }
+        response_data = await self.api_client.call_api(
+            *_param,
+            _request_timeout=_request_timeout
+        )
+        await response_data.read()
+        return self.api_client.response_deserialize(
+            response_data=response_data,
+            response_types_map=_response_types_map,
+        )
+
+
+    @validate_call
+    async def get_entity_without_preload_content(
+        self,
+        bank_id: StrictStr,
+        entity_id: StrictStr,
+        authorization: Optional[StrictStr] = None,
+        _request_timeout: Union[
+            None,
+            Annotated[StrictFloat, Field(gt=0)],
+            Tuple[
+                Annotated[StrictFloat, Field(gt=0)],
+                Annotated[StrictFloat, Field(gt=0)]
+            ]
+        ] = None,
+        _request_auth: Optional[Dict[StrictStr, Any]] = None,
+        _content_type: Optional[StrictStr] = None,
+        _headers: Optional[Dict[StrictStr, Any]] = None,
+        _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0,
+    ) -> RESTResponseType:
+        """Get entity details
+
+        Get detailed information about an entity including observations (mental model).
+
+        :param bank_id: (required)
+        :type bank_id: str
+        :param entity_id: (required)
+        :type entity_id: str
+        :param authorization:
+        :type authorization: str
+        :param _request_timeout: timeout setting for this request. If one
+                                 number provided, it will be total request
+                                 timeout. It can also be a pair (tuple) of
+                                 (connection, read) timeouts.
+        :type _request_timeout: int, tuple(int, int), optional
+        :param _request_auth: set to override the auth_settings for an a single
+                              request; this effectively ignores the
+                              authentication in the spec for a single request.
+        :type _request_auth: dict, optional
+        :param _content_type: force content-type for the request.
+        :type _content_type: str, Optional
+        :param _headers: set to override the headers for a single
+                         request; this effectively ignores the headers
+                         in the spec for a single request.
+        :type _headers: dict, optional
+        :param _host_index: set to override the host_index for a single
+                            request; this effectively ignores the host_index
+                            in the spec for a single request.
+        :type _host_index: int, optional
+        :return: Returns the result object.
+        """ # noqa: E501
+
+        _param = self._get_entity_serialize(
+            bank_id=bank_id,
+            entity_id=entity_id,
+            authorization=authorization,
+            _request_auth=_request_auth,
+            _content_type=_content_type,
+            _headers=_headers,
+            _host_index=_host_index
+        )
+
+        _response_types_map: Dict[str, Optional[str]] = {
+            '200': "EntityDetailResponse",
+            '422': "HTTPValidationError",
+        }
+        response_data = await self.api_client.call_api(
+            *_param,
+            _request_timeout=_request_timeout
+        )
+        return response_data.response
+
+
+    def _get_entity_serialize(
+        self,
+        bank_id,
+        entity_id,
+        authorization,
+        _request_auth,
+        _content_type,
+        _headers,
+        _host_index,
+    ) -> RequestSerialized:
+
+        _host = None
+
+        _collection_formats: Dict[str, str] = {
+        }
+
+        _path_params: Dict[str, str] = {}
+        _query_params: List[Tuple[str, str]] = []
+        _header_params: Dict[str, Optional[str]] = _headers or {}
+        _form_params: List[Tuple[str, str]] = []
+        _files: Dict[
+            str, Union[str, bytes, List[str], List[bytes], List[Tuple[str, bytes]]]
+        ] = {}
+        _body_params: Optional[bytes] = None
+
+        # process the path parameters
+        if bank_id is not None:
+            _path_params['bank_id'] = bank_id
+        if entity_id is not None:
+            _path_params['entity_id'] = entity_id
+        # process the query parameters
+        # process the header parameters
+        if authorization is not None:
+            _header_params['authorization'] = authorization
+        # process the form parameters
+        # process the body parameter
+
+
+        # set the HTTP header `Accept`
+        if 'Accept' not in _header_params:
+            _header_params['Accept'] = self.api_client.select_header_accept(
+                [
+                    'application/json'
+                ]
+            )
+
+
+        # authentication setting
+        _auth_settings: List[str] = [
+        ]
+
+        return self.api_client.param_serialize(
+            method='GET',
+            resource_path='/v1/default/banks/{bank_id}/entities/{entity_id}',
+            path_params=_path_params,
+            query_params=_query_params,
+            header_params=_header_params,
+            body=_body_params,
+            post_params=_form_params,
+            files=_files,
+            auth_settings=_auth_settings,
+            collection_formats=_collection_formats,
+            _host=_host,
+            _request_auth=_request_auth
+        )
+
+
+
+
+    @validate_call
+    async def list_entities(
+        self,
+        bank_id: StrictStr,
+        limit: Annotated[Optional[StrictInt], Field(description="Maximum number of entities to return")] = None,
+        authorization: Optional[StrictStr] = None,
+        _request_timeout: Union[
+            None,
+            Annotated[StrictFloat, Field(gt=0)],
+            Tuple[
+                Annotated[StrictFloat, Field(gt=0)],
+                Annotated[StrictFloat, Field(gt=0)]
+            ]
+        ] = None,
+        _request_auth: Optional[Dict[StrictStr, Any]] = None,
+        _content_type: Optional[StrictStr] = None,
+        _headers: Optional[Dict[StrictStr, Any]] = None,
+        _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0,
+    ) -> EntityListResponse:
+        """List entities
+
+        List all entities (people, organizations, etc.) known by the bank, ordered by mention count.
+
+        :param bank_id: (required)
+        :type bank_id: str
+        :param limit: Maximum number of entities to return
+        :type limit: int
+        :param authorization:
+        :type authorization: str
+        :param _request_timeout: timeout setting for this request. If one
+                                 number provided, it will be total request
+                                 timeout. It can also be a pair (tuple) of
+                                 (connection, read) timeouts.
+        :type _request_timeout: int, tuple(int, int), optional
+        :param _request_auth: set to override the auth_settings for an a single
+                              request; this effectively ignores the
+                              authentication in the spec for a single request.
+        :type _request_auth: dict, optional
+        :param _content_type: force content-type for the request.
+        :type _content_type: str, Optional
+        :param _headers: set to override the headers for a single
+                         request; this effectively ignores the headers
+                         in the spec for a single request.
+        :type _headers: dict, optional
+        :param _host_index: set to override the host_index for a single
+                            request; this effectively ignores the host_index
+                            in the spec for a single request.
+        :type _host_index: int, optional
+        :return: Returns the result object.
+        """ # noqa: E501
+
+        _param = self._list_entities_serialize(
+            bank_id=bank_id,
+            limit=limit,
+            authorization=authorization,
+            _request_auth=_request_auth,
+            _content_type=_content_type,
+            _headers=_headers,
+            _host_index=_host_index
+        )
+
+        _response_types_map: Dict[str, Optional[str]] = {
+            '200': "EntityListResponse",
+            '422': "HTTPValidationError",
+        }
+        response_data = await self.api_client.call_api(
+            *_param,
+            _request_timeout=_request_timeout
+        )
+        await response_data.read()
+        return self.api_client.response_deserialize(
+            response_data=response_data,
+            response_types_map=_response_types_map,
+        ).data
+
+
+    @validate_call
+    async def list_entities_with_http_info(
+        self,
+        bank_id: StrictStr,
+        limit: Annotated[Optional[StrictInt], Field(description="Maximum number of entities to return")] = None,
+        authorization: Optional[StrictStr] = None,
+        _request_timeout: Union[
+            None,
+            Annotated[StrictFloat, Field(gt=0)],
+            Tuple[
+                Annotated[StrictFloat, Field(gt=0)],
+                Annotated[StrictFloat, Field(gt=0)]
+            ]
+        ] = None,
+        _request_auth: Optional[Dict[StrictStr, Any]] = None,
+        _content_type: Optional[StrictStr] = None,
+        _headers: Optional[Dict[StrictStr, Any]] = None,
+        _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0,
+    ) -> ApiResponse[EntityListResponse]:
+        """List entities
+
+        List all entities (people, organizations, etc.) known by the bank, ordered by mention count.
+
+        :param bank_id: (required)
+        :type bank_id: str
+        :param limit: Maximum number of entities to return
+        :type limit: int
+        :param authorization:
+        :type authorization: str
+        :param _request_timeout: timeout setting for this request. If one
+                                 number provided, it will be total request
+                                 timeout. It can also be a pair (tuple) of
+                                 (connection, read) timeouts.
+        :type _request_timeout: int, tuple(int, int), optional
+        :param _request_auth: set to override the auth_settings for an a single
+                              request; this effectively ignores the
+                              authentication in the spec for a single request.
+        :type _request_auth: dict, optional
+        :param _content_type: force content-type for the request.
+        :type _content_type: str, Optional
+        :param _headers: set to override the headers for a single
+                         request; this effectively ignores the headers
+                         in the spec for a single request.
+        :type _headers: dict, optional
+        :param _host_index: set to override the host_index for a single
+                            request; this effectively ignores the host_index
+                            in the spec for a single request.
+        :type _host_index: int, optional
+        :return: Returns the result object.
+        """ # noqa: E501
+
+        _param = self._list_entities_serialize(
+            bank_id=bank_id,
+            limit=limit,
+            authorization=authorization,
+            _request_auth=_request_auth,
+            _content_type=_content_type,
+            _headers=_headers,
+            _host_index=_host_index
+        )
+
+        _response_types_map: Dict[str, Optional[str]] = {
+            '200': "EntityListResponse",
+            '422': "HTTPValidationError",
+        }
+        response_data = await self.api_client.call_api(
+            *_param,
+            _request_timeout=_request_timeout
+        )
+        await response_data.read()
+        return self.api_client.response_deserialize(
+            response_data=response_data,
+            response_types_map=_response_types_map,
+        )
+
+
+    @validate_call
+    async def list_entities_without_preload_content(
+        self,
+        bank_id: StrictStr,
+        limit: Annotated[Optional[StrictInt], Field(description="Maximum number of entities to return")] = None,
+        authorization: Optional[StrictStr] = None,
+        _request_timeout: Union[
+            None,
+            Annotated[StrictFloat, Field(gt=0)],
+            Tuple[
+                Annotated[StrictFloat, Field(gt=0)],
+                Annotated[StrictFloat, Field(gt=0)]
+            ]
+        ] = None,
+        _request_auth: Optional[Dict[StrictStr, Any]] = None,
+        _content_type: Optional[StrictStr] = None,
+        _headers: Optional[Dict[StrictStr, Any]] = None,
+        _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0,
+    ) -> RESTResponseType:
+        """List entities
+
+        List all entities (people, organizations, etc.) known by the bank, ordered by mention count.
+
+        :param bank_id: (required)
+        :type bank_id: str
+        :param limit: Maximum number of entities to return
+        :type limit: int
+        :param authorization:
+        :type authorization: str
+        :param _request_timeout: timeout setting for this request. If one
+                                 number provided, it will be total request
+                                 timeout. It can also be a pair (tuple) of
+                                 (connection, read) timeouts.
+        :type _request_timeout: int, tuple(int, int), optional
+        :param _request_auth: set to override the auth_settings for an a single
+                              request; this effectively ignores the
+                              authentication in the spec for a single request.
+        :type _request_auth: dict, optional
+        :param _content_type: force content-type for the request.
+        :type _content_type: str, Optional
+        :param _headers: set to override the headers for a single
+                         request; this effectively ignores the headers
+                         in the spec for a single request.
+        :type _headers: dict, optional
+        :param _host_index: set to override the host_index for a single
+                            request; this effectively ignores the host_index
+                            in the spec for a single request.
+        :type _host_index: int, optional
+        :return: Returns the result object.
+        """ # noqa: E501
+
+        _param = self._list_entities_serialize(
+            bank_id=bank_id,
+            limit=limit,
+            authorization=authorization,
+            _request_auth=_request_auth,
+            _content_type=_content_type,
+            _headers=_headers,
+            _host_index=_host_index
+        )
+
+        _response_types_map: Dict[str, Optional[str]] = {
+            '200': "EntityListResponse",
+            '422': "HTTPValidationError",
+        }
+        response_data = await self.api_client.call_api(
+            *_param,
+            _request_timeout=_request_timeout
+        )
+        return response_data.response
+
+
+    def _list_entities_serialize(
+        self,
+        bank_id,
+        limit,
+        authorization,
+        _request_auth,
+        _content_type,
+        _headers,
+        _host_index,
+    ) -> RequestSerialized:
+
+        _host = None
+
+        _collection_formats: Dict[str, str] = {
+        }
+
+        _path_params: Dict[str, str] = {}
+        _query_params: List[Tuple[str, str]] = []
+        _header_params: Dict[str, Optional[str]] = _headers or {}
+        _form_params: List[Tuple[str, str]] = []
+        _files: Dict[
+            str, Union[str, bytes, List[str], List[bytes], List[Tuple[str, bytes]]]
+        ] = {}
+        _body_params: Optional[bytes] = None
+
+        # process the path parameters
+        if bank_id is not None:
+            _path_params['bank_id'] = bank_id
+        # process the query parameters
+        if limit is not None:
+            
+            _query_params.append(('limit', limit))
+            
+        # process the header parameters
+        if authorization is not None:
+            _header_params['authorization'] = authorization
+        # process the form parameters
+        # process the body parameter
+
+
+        # set the HTTP header `Accept`
+        if 'Accept' not in _header_params:
+            _header_params['Accept'] = self.api_client.select_header_accept(
+                [
+                    'application/json'
+                ]
+            )
+
+
+        # authentication setting
+        _auth_settings: List[str] = [
+        ]
+
+        return self.api_client.param_serialize(
+            method='GET',
+            resource_path='/v1/default/banks/{bank_id}/entities',
+            path_params=_path_params,
+            query_params=_query_params,
+            header_params=_header_params,
+            body=_body_params,
+            post_params=_form_params,
+            files=_files,
+            auth_settings=_auth_settings,
+            collection_formats=_collection_formats,
+            _host=_host,
+            _request_auth=_request_auth
+        )
+
+
+
+
+    @validate_call
+    async def regenerate_entity_observations(
+        self,
+        bank_id: StrictStr,
+        entity_id: StrictStr,
+        authorization: Optional[StrictStr] = None,
+        _request_timeout: Union[
+            None,
+            Annotated[StrictFloat, Field(gt=0)],
+            Tuple[
+                Annotated[StrictFloat, Field(gt=0)],
+                Annotated[StrictFloat, Field(gt=0)]
+            ]
+        ] = None,
+        _request_auth: Optional[Dict[StrictStr, Any]] = None,
+        _content_type: Optional[StrictStr] = None,
+        _headers: Optional[Dict[StrictStr, Any]] = None,
+        _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0,
+    ) -> EntityDetailResponse:
+        """Regenerate entity observations
+
+        Regenerate observations for an entity based on all facts mentioning it.
+
+        :param bank_id: (required)
+        :type bank_id: str
+        :param entity_id: (required)
+        :type entity_id: str
+        :param authorization:
+        :type authorization: str
+        :param _request_timeout: timeout setting for this request. If one
+                                 number provided, it will be total request
+                                 timeout. It can also be a pair (tuple) of
+                                 (connection, read) timeouts.
+        :type _request_timeout: int, tuple(int, int), optional
+        :param _request_auth: set to override the auth_settings for an a single
+                              request; this effectively ignores the
+                              authentication in the spec for a single request.
+        :type _request_auth: dict, optional
+        :param _content_type: force content-type for the request.
+        :type _content_type: str, Optional
+        :param _headers: set to override the headers for a single
+                         request; this effectively ignores the headers
+                         in the spec for a single request.
+        :type _headers: dict, optional
+        :param _host_index: set to override the host_index for a single
+                            request; this effectively ignores the host_index
+                            in the spec for a single request.
+        :type _host_index: int, optional
+        :return: Returns the result object.
+        """ # noqa: E501
+
+        _param = self._regenerate_entity_observations_serialize(
+            bank_id=bank_id,
+            entity_id=entity_id,
+            authorization=authorization,
+            _request_auth=_request_auth,
+            _content_type=_content_type,
+            _headers=_headers,
+            _host_index=_host_index
+        )
+
+        _response_types_map: Dict[str, Optional[str]] = {
+            '200': "EntityDetailResponse",
+            '422': "HTTPValidationError",
+        }
+        response_data = await self.api_client.call_api(
+            *_param,
+            _request_timeout=_request_timeout
+        )
+        await response_data.read()
+        return self.api_client.response_deserialize(
+            response_data=response_data,
+            response_types_map=_response_types_map,
+        ).data
+
+
+    @validate_call
+    async def regenerate_entity_observations_with_http_info(
+        self,
+        bank_id: StrictStr,
+        entity_id: StrictStr,
+        authorization: Optional[StrictStr] = None,
+        _request_timeout: Union[
+            None,
+            Annotated[StrictFloat, Field(gt=0)],
+            Tuple[
+                Annotated[StrictFloat, Field(gt=0)],
+                Annotated[StrictFloat, Field(gt=0)]
+            ]
+        ] = None,
+        _request_auth: Optional[Dict[StrictStr, Any]] = None,
+        _content_type: Optional[StrictStr] = None,
+        _headers: Optional[Dict[StrictStr, Any]] = None,
+        _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0,
+    ) -> ApiResponse[EntityDetailResponse]:
+        """Regenerate entity observations
+
+        Regenerate observations for an entity based on all facts mentioning it.
+
+        :param bank_id: (required)
+        :type bank_id: str
+        :param entity_id: (required)
+        :type entity_id: str
+        :param authorization:
+        :type authorization: str
+        :param _request_timeout: timeout setting for this request. If one
+                                 number provided, it will be total request
+                                 timeout. It can also be a pair (tuple) of
+                                 (connection, read) timeouts.
+        :type _request_timeout: int, tuple(int, int), optional
+        :param _request_auth: set to override the auth_settings for an a single
+                              request; this effectively ignores the
+                              authentication in the spec for a single request.
+        :type _request_auth: dict, optional
+        :param _content_type: force content-type for the request.
+        :type _content_type: str, Optional
+        :param _headers: set to override the headers for a single
+                         request; this effectively ignores the headers
+                         in the spec for a single request.
+        :type _headers: dict, optional
+        :param _host_index: set to override the host_index for a single
+                            request; this effectively ignores the host_index
+                            in the spec for a single request.
+        :type _host_index: int, optional
+        :return: Returns the result object.
+        """ # noqa: E501
+
+        _param = self._regenerate_entity_observations_serialize(
+            bank_id=bank_id,
+            entity_id=entity_id,
+            authorization=authorization,
+            _request_auth=_request_auth,
+            _content_type=_content_type,
+            _headers=_headers,
+            _host_index=_host_index
+        )
+
+        _response_types_map: Dict[str, Optional[str]] = {
+            '200': "EntityDetailResponse",
+            '422': "HTTPValidationError",
+        }
+        response_data = await self.api_client.call_api(
+            *_param,
+            _request_timeout=_request_timeout
+        )
+        await response_data.read()
+        return self.api_client.response_deserialize(
+            response_data=response_data,
+            response_types_map=_response_types_map,
+        )
+
+
+    @validate_call
+    async def regenerate_entity_observations_without_preload_content(
+        self,
+        bank_id: StrictStr,
+        entity_id: StrictStr,
+        authorization: Optional[StrictStr] = None,
+        _request_timeout: Union[
+            None,
+            Annotated[StrictFloat, Field(gt=0)],
+            Tuple[
+                Annotated[StrictFloat, Field(gt=0)],
+                Annotated[StrictFloat, Field(gt=0)]
+            ]
+        ] = None,
+        _request_auth: Optional[Dict[StrictStr, Any]] = None,
+        _content_type: Optional[StrictStr] = None,
+        _headers: Optional[Dict[StrictStr, Any]] = None,
+        _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0,
+    ) -> RESTResponseType:
+        """Regenerate entity observations
+
+        Regenerate observations for an entity based on all facts mentioning it.
+
+        :param bank_id: (required)
+        :type bank_id: str
+        :param entity_id: (required)
+        :type entity_id: str
+        :param authorization:
+        :type authorization: str
+        :param _request_timeout: timeout setting for this request. If one
+                                 number provided, it will be total request
+                                 timeout. It can also be a pair (tuple) of
+                                 (connection, read) timeouts.
+        :type _request_timeout: int, tuple(int, int), optional
+        :param _request_auth: set to override the auth_settings for an a single
+                              request; this effectively ignores the
+                              authentication in the spec for a single request.
+        :type _request_auth: dict, optional
+        :param _content_type: force content-type for the request.
+        :type _content_type: str, Optional
+        :param _headers: set to override the headers for a single
+                         request; this effectively ignores the headers
+                         in the spec for a single request.
+        :type _headers: dict, optional
+        :param _host_index: set to override the host_index for a single
+                            request; this effectively ignores the host_index
+                            in the spec for a single request.
+        :type _host_index: int, optional
+        :return: Returns the result object.
+        """ # noqa: E501
+
+        _param = self._regenerate_entity_observations_serialize(
+            bank_id=bank_id,
+            entity_id=entity_id,
+            authorization=authorization,
+            _request_auth=_request_auth,
+            _content_type=_content_type,
+            _headers=_headers,
+            _host_index=_host_index
+        )
+
+        _response_types_map: Dict[str, Optional[str]] = {
+            '200': "EntityDetailResponse",
+            '422': "HTTPValidationError",
+        }
+        response_data = await self.api_client.call_api(
+            *_param,
+            _request_timeout=_request_timeout
+        )
+        return response_data.response
+
+
+    def _regenerate_entity_observations_serialize(
+        self,
+        bank_id,
+        entity_id,
+        authorization,
+        _request_auth,
+        _content_type,
+        _headers,
+        _host_index,
+    ) -> RequestSerialized:
+
+        _host = None
+
+        _collection_formats: Dict[str, str] = {
+        }
+
+        _path_params: Dict[str, str] = {}
+        _query_params: List[Tuple[str, str]] = []
+        _header_params: Dict[str, Optional[str]] = _headers or {}
+        _form_params: List[Tuple[str, str]] = []
+        _files: Dict[
+            str, Union[str, bytes, List[str], List[bytes], List[Tuple[str, bytes]]]
+        ] = {}
+        _body_params: Optional[bytes] = None
+
+        # process the path parameters
+        if bank_id is not None:
+            _path_params['bank_id'] = bank_id
+        if entity_id is not None:
+            _path_params['entity_id'] = entity_id
+        # process the query parameters
+        # process the header parameters
+        if authorization is not None:
+            _header_params['authorization'] = authorization
+        # process the form parameters
+        # process the body parameter
+
+
+        # set the HTTP header `Accept`
+        if 'Accept' not in _header_params:
+            _header_params['Accept'] = self.api_client.select_header_accept(
+                [
+                    'application/json'
+                ]
+            )
+
+
+        # authentication setting
+        _auth_settings: List[str] = [
+        ]
+
+        return self.api_client.param_serialize(
+            method='POST',
+            resource_path='/v1/default/banks/{bank_id}/entities/{entity_id}/regenerate',
+            path_params=_path_params,
+            query_params=_query_params,
+            header_params=_header_params,
+            body=_body_params,
+            post_params=_form_params,
+            files=_files,
+            auth_settings=_auth_settings,
+            collection_formats=_collection_formats,
+            _host=_host,
+            _request_auth=_request_auth
+        )
+
+
@@ -5,7 +5,7 @@

    HTTP API for Hindsight

-    The version of the OpenAPI document: 1.0.0
+    The version of the OpenAPI document: 0.1.0
    Generated by OpenAPI Generator (https://openapi-generator.tech)

    Do not edit the class manually.
@@ -0,0 +1,610 @@
+# coding: utf-8
+
+"""
+    Hindsight HTTP API
+
+    HTTP API for Hindsight
+
+    The version of the OpenAPI document: 0.1.0
+    Generated by OpenAPI Generator (https://openapi-generator.tech)
+
+    Do not edit the class manually.
+"""  # noqa: E501
+
+import warnings
+from pydantic import validate_call, Field, StrictFloat, StrictStr, StrictInt
+from typing import Any, Dict, List, Optional, Tuple, Union
+from typing_extensions import Annotated
+
+from pydantic import StrictStr
+from typing import Optional
+from hindsight_client_api.models.cancel_operation_response import CancelOperationResponse
+from hindsight_client_api.models.operations_list_response import OperationsListResponse
+
+from hindsight_client_api.api_client import ApiClient, RequestSerialized
+from hindsight_client_api.api_response import ApiResponse
+from hindsight_client_api.rest import RESTResponseType
+
+
+class OperationsApi:
+    """NOTE: This class is auto generated by OpenAPI Generator
+    Ref: https://openapi-generator.tech
+
+    Do not edit the class manually.
+    """
+
+    def __init__(self, api_client=None) -> None:
+        if api_client is None:
+            api_client = ApiClient.get_default()
+        self.api_client = api_client
+
+
+    @validate_call
+    async def cancel_operation(
+        self,
+        bank_id: StrictStr,
+        operation_id: StrictStr,
+        authorization: Optional[StrictStr] = None,
+        _request_timeout: Union[
+            None,
+            Annotated[StrictFloat, Field(gt=0)],
+            Tuple[
+                Annotated[StrictFloat, Field(gt=0)],
+                Annotated[StrictFloat, Field(gt=0)]
+            ]
+        ] = None,
+        _request_auth: Optional[Dict[StrictStr, Any]] = None,
+        _content_type: Optional[StrictStr] = None,
+        _headers: Optional[Dict[StrictStr, Any]] = None,
+        _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0,
+    ) -> CancelOperationResponse:
+        """Cancel a pending async operation
+
+        Cancel a pending async operation by removing it from the queue
+
+        :param bank_id: (required)
+        :type bank_id: str
+        :param operation_id: (required)
+        :type operation_id: str
+        :param authorization:
+        :type authorization: str
+        :param _request_timeout: timeout setting for this request. If one
+                                 number provided, it will be total request
+                                 timeout. It can also be a pair (tuple) of
+                                 (connection, read) timeouts.
+        :type _request_timeout: int, tuple(int, int), optional
+        :param _request_auth: set to override the auth_settings for an a single
+                              request; this effectively ignores the
+                              authentication in the spec for a single request.
+        :type _request_auth: dict, optional
+        :param _content_type: force content-type for the request.
+        :type _content_type: str, Optional
+        :param _headers: set to override the headers for a single
+                         request; this effectively ignores the headers
+                         in the spec for a single request.
+        :type _headers: dict, optional
+        :param _host_index: set to override the host_index for a single
+                            request; this effectively ignores the host_index
+                            in the spec for a single request.
+        :type _host_index: int, optional
+        :return: Returns the result object.
+        """ # noqa: E501
+
+        _param = self._cancel_operation_serialize(
+            bank_id=bank_id,
+            operation_id=operation_id,
+            authorization=authorization,
+            _request_auth=_request_auth,
+            _content_type=_content_type,
+            _headers=_headers,
+            _host_index=_host_index
+        )
+
+        _response_types_map: Dict[str, Optional[str]] = {
+            '200': "CancelOperationResponse",
+            '422': "HTTPValidationError",
+        }
+        response_data = await self.api_client.call_api(
+            *_param,
+            _request_timeout=_request_timeout
+        )
+        await response_data.read()
+        return self.api_client.response_deserialize(
+            response_data=response_data,
+            response_types_map=_response_types_map,
+        ).data
+
+
+    @validate_call
+    async def cancel_operation_with_http_info(
+        self,
+        bank_id: StrictStr,
+        operation_id: StrictStr,
+        authorization: Optional[StrictStr] = None,
+        _request_timeout: Union[
+            None,
+            Annotated[StrictFloat, Field(gt=0)],
+            Tuple[
+                Annotated[StrictFloat, Field(gt=0)],
+                Annotated[StrictFloat, Field(gt=0)]
+            ]
+        ] = None,
+        _request_auth: Optional[Dict[StrictStr, Any]] = None,
+        _content_type: Optional[StrictStr] = None,
+        _headers: Optional[Dict[StrictStr, Any]] = None,
+        _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0,
+    ) -> ApiResponse[CancelOperationResponse]:
+        """Cancel a pending async operation
+
+        Cancel a pending async operation by removing it from the queue
+
+        :param bank_id: (required)
+        :type bank_id: str
+        :param operation_id: (required)
+        :type operation_id: str
+        :param authorization:
+        :type authorization: str
+        :param _request_timeout: timeout setting for this request. If one
+                                 number provided, it will be total request
+                                 timeout. It can also be a pair (tuple) of
+                                 (connection, read) timeouts.
+        :type _request_timeout: int, tuple(int, int), optional
+        :param _request_auth: set to override the auth_settings for an a single
+                              request; this effectively ignores the
+                              authentication in the spec for a single request.
+        :type _request_auth: dict, optional
+        :param _content_type: force content-type for the request.
+        :type _content_type: str, Optional
+        :param _headers: set to override the headers for a single
+                         request; this effectively ignores the headers
+                         in the spec for a single request.
+        :type _headers: dict, optional
+        :param _host_index: set to override the host_index for a single
+                            request; this effectively ignores the host_index
+                            in the spec for a single request.
+        :type _host_index: int, optional
+        :return: Returns the result object.
+        """ # noqa: E501
+
+        _param = self._cancel_operation_serialize(
+            bank_id=bank_id,
+            operation_id=operation_id,
+            authorization=authorization,
+            _request_auth=_request_auth,
+            _content_type=_content_type,
+            _headers=_headers,
+            _host_index=_host_index
+        )
+
+        _response_types_map: Dict[str, Optional[str]] = {
+            '200': "CancelOperationResponse",
+            '422': "HTTPValidationError",
+        }
+        response_data = await self.api_client.call_api(
+            *_param,
+            _request_timeout=_request_timeout
+        )
+        await response_data.read()
+        return self.api_client.response_deserialize(
+            response_data=response_data,
+            response_types_map=_response_types_map,
+        )
+
+
+    @validate_call
+    async def cancel_operation_without_preload_content(
+        self,
+        bank_id: StrictStr,
+        operation_id: StrictStr,
+        authorization: Optional[StrictStr] = None,
+        _request_timeout: Union[
+            None,
+            Annotated[StrictFloat, Field(gt=0)],
+            Tuple[
+                Annotated[StrictFloat, Field(gt=0)],
+                Annotated[StrictFloat, Field(gt=0)]
+            ]
+        ] = None,
+        _request_auth: Optional[Dict[StrictStr, Any]] = None,
+        _content_type: Optional[StrictStr] = None,
+        _headers: Optional[Dict[StrictStr, Any]] = None,
+        _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0,
+    ) -> RESTResponseType:
+        """Cancel a pending async operation
+
+        Cancel a pending async operation by removing it from the queue
+
+        :param bank_id: (required)
+        :type bank_id: str
+        :param operation_id: (required)
+        :type operation_id: str
+        :param authorization:
+        :type authorization: str
+        :param _request_timeout: timeout setting for this request. If one
+                                 number provided, it will be total request
+                                 timeout. It can also be a pair (tuple) of
+                                 (connection, read) timeouts.
+        :type _request_timeout: int, tuple(int, int), optional
+        :param _request_auth: set to override the auth_settings for an a single
+                              request; this effectively ignores the
+                              authentication in the spec for a single request.
+        :type _request_auth: dict, optional
+        :param _content_type: force content-type for the request.
+        :type _content_type: str, Optional
+        :param _headers: set to override the headers for a single
+                         request; this effectively ignores the headers
+                         in the spec for a single request.
+        :type _headers: dict, optional
+        :param _host_index: set to override the host_index for a single
+                            request; this effectively ignores the host_index
+                            in the spec for a single request.
+        :type _host_index: int, optional
+        :return: Returns the result object.
+        """ # noqa: E501
+
+        _param = self._cancel_operation_serialize(
+            bank_id=bank_id,
+            operation_id=operation_id,
+            authorization=authorization,
+            _request_auth=_request_auth,
+            _content_type=_content_type,
+            _headers=_headers,
+            _host_index=_host_index
+        )
+
+        _response_types_map: Dict[str, Optional[str]] = {
+            '200': "CancelOperationResponse",
+            '422': "HTTPValidationError",
+        }
+        response_data = await self.api_client.call_api(
+            *_param,
+            _request_timeout=_request_timeout
+        )
+        return response_data.response
+
+
+    def _cancel_operation_serialize(
+        self,
+        bank_id,
+        operation_id,
+        authorization,
+        _request_auth,
+        _content_type,
+        _headers,
+        _host_index,
+    ) -> RequestSerialized:
+
+        _host = None
+
+        _collection_formats: Dict[str, str] = {
+        }
+
+        _path_params: Dict[str, str] = {}
+        _query_params: List[Tuple[str, str]] = []
+        _header_params: Dict[str, Optional[str]] = _headers or {}
+        _form_params: List[Tuple[str, str]] = []
+        _files: Dict[
+            str, Union[str, bytes, List[str], List[bytes], List[Tuple[str, bytes]]]
+        ] = {}
+        _body_params: Optional[bytes] = None
+
+        # process the path parameters
+        if bank_id is not None:
+            _path_params['bank_id'] = bank_id
+        if operation_id is not None:
+            _path_params['operation_id'] = operation_id
+        # process the query parameters
+        # process the header parameters
+        if authorization is not None:
+            _header_params['authorization'] = authorization
+        # process the form parameters
+        # process the body parameter
+
+
+        # set the HTTP header `Accept`
+        if 'Accept' not in _header_params:
+            _header_params['Accept'] = self.api_client.select_header_accept(
+                [
+                    'application/json'
+                ]
+            )
+
+
+        # authentication setting
+        _auth_settings: List[str] = [
+        ]
+
+        return self.api_client.param_serialize(
+            method='DELETE',
+            resource_path='/v1/default/banks/{bank_id}/operations/{operation_id}',
+            path_params=_path_params,
+            query_params=_query_params,
+            header_params=_header_params,
+            body=_body_params,
+            post_params=_form_params,
+            files=_files,
+            auth_settings=_auth_settings,
+            collection_formats=_collection_formats,
+            _host=_host,
+            _request_auth=_request_auth
+        )
+
+
+
+
+    @validate_call
+    async def list_operations(
+        self,
+        bank_id: StrictStr,
+        authorization: Optional[StrictStr] = None,
+        _request_timeout: Union[
+            None,
+            Annotated[StrictFloat, Field(gt=0)],
+            Tuple[
+                Annotated[StrictFloat, Field(gt=0)],
+                Annotated[StrictFloat, Field(gt=0)]
+            ]
+        ] = None,
+        _request_auth: Optional[Dict[StrictStr, Any]] = None,
+        _content_type: Optional[StrictStr] = None,
+        _headers: Optional[Dict[StrictStr, Any]] = None,
+        _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0,
+    ) -> OperationsListResponse:
+        """List async operations
+
+        Get a list of all async operations (pending and failed) for a specific agent, including error messages for failed operations
+
+        :param bank_id: (required)
+        :type bank_id: str
+        :param authorization:
+        :type authorization: str
+        :param _request_timeout: timeout setting for this request. If one
+                                 number provided, it will be total request
+                                 timeout. It can also be a pair (tuple) of
+                                 (connection, read) timeouts.
+        :type _request_timeout: int, tuple(int, int), optional
+        :param _request_auth: set to override the auth_settings for an a single
+                              request; this effectively ignores the
+                              authentication in the spec for a single request.
+        :type _request_auth: dict, optional
+        :param _content_type: force content-type for the request.
+        :type _content_type: str, Optional
+        :param _headers: set to override the headers for a single
+                         request; this effectively ignores the headers
+                         in the spec for a single request.
+        :type _headers: dict, optional
+        :param _host_index: set to override the host_index for a single
+                            request; this effectively ignores the host_index
+                            in the spec for a single request.
+        :type _host_index: int, optional
+        :return: Returns the result object.
+        """ # noqa: E501
+
+        _param = self._list_operations_serialize(
+            bank_id=bank_id,
+            authorization=authorization,
+            _request_auth=_request_auth,
+            _content_type=_content_type,
+            _headers=_headers,
+            _host_index=_host_index
+        )
+
+        _response_types_map: Dict[str, Optional[str]] = {
+            '200': "OperationsListResponse",
+            '422': "HTTPValidationError",
+        }
+        response_data = await self.api_client.call_api(
+            *_param,
+            _request_timeout=_request_timeout
+        )
+        await response_data.read()
+        return self.api_client.response_deserialize(
+            response_data=response_data,
+            response_types_map=_response_types_map,
+        ).data
+
+
+    @validate_call
+    async def list_operations_with_http_info(
+        self,
+        bank_id: StrictStr,
+        authorization: Optional[StrictStr] = None,
+        _request_timeout: Union[
+            None,
+            Annotated[StrictFloat, Field(gt=0)],
+            Tuple[
+                Annotated[StrictFloat, Field(gt=0)],
+                Annotated[StrictFloat, Field(gt=0)]
+            ]
+        ] = None,
+        _request_auth: Optional[Dict[StrictStr, Any]] = None,
+        _content_type: Optional[StrictStr] = None,
+        _headers: Optional[Dict[StrictStr, Any]] = None,
+        _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0,
+    ) -> ApiResponse[OperationsListResponse]:
+        """List async operations
+
+        Get a list of all async operations (pending and failed) for a specific agent, including error messages for failed operations
+
+        :param bank_id: (required)
+        :type bank_id: str
+        :param authorization:
+        :type authorization: str
+        :param _request_timeout: timeout setting for this request. If one
+                                 number provided, it will be total request
+                                 timeout. It can also be a pair (tuple) of
+                                 (connection, read) timeouts.
+        :type _request_timeout: int, tuple(int, int), optional
+        :param _request_auth: set to override the auth_settings for an a single
+                              request; this effectively ignores the
+                              authentication in the spec for a single request.
+        :type _request_auth: dict, optional
+        :param _content_type: force content-type for the request.
+        :type _content_type: str, Optional
+        :param _headers: set to override the headers for a single
+                         request; this effectively ignores the headers
+                         in the spec for a single request.
+        :type _headers: dict, optional
+        :param _host_index: set to override the host_index for a single
+                            request; this effectively ignores the host_index
+                            in the spec for a single request.
+        :type _host_index: int, optional
+        :return: Returns the result object.
+        """ # noqa: E501
+
+        _param = self._list_operations_serialize(
+            bank_id=bank_id,
+            authorization=authorization,
+            _request_auth=_request_auth,
+            _content_type=_content_type,
+            _headers=_headers,
+            _host_index=_host_index
+        )
+
+        _response_types_map: Dict[str, Optional[str]] = {
+            '200': "OperationsListResponse",
+            '422': "HTTPValidationError",
+        }
+        response_data = await self.api_client.call_api(
+            *_param,
+            _request_timeout=_request_timeout
+        )
+        await response_data.read()
+        return self.api_client.response_deserialize(
+            response_data=response_data,
+            response_types_map=_response_types_map,
+        )
+
+
+    @validate_call
+    async def list_operations_without_preload_content(
+        self,
+        bank_id: StrictStr,
+        authorization: Optional[StrictStr] = None,
+        _request_timeout: Union[
+            None,
+            Annotated[StrictFloat, Field(gt=0)],
+            Tuple[
+                Annotated[StrictFloat, Field(gt=0)],
+                Annotated[StrictFloat, Field(gt=0)]
+            ]
+        ] = None,
+        _request_auth: Optional[Dict[StrictStr, Any]] = None,
+        _content_type: Optional[StrictStr] = None,
+        _headers: Optional[Dict[StrictStr, Any]] = None,
+        _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0,
+    ) -> RESTResponseType:
+        """List async operations
+
+        Get a list of all async operations (pending and failed) for a specific agent, including error messages for failed operations
+
+        :param bank_id: (required)
+        :type bank_id: str
+        :param authorization:
+        :type authorization: str
+        :param _request_timeout: timeout setting for this request. If one
+                                 number provided, it will be total request
+                                 timeout. It can also be a pair (tuple) of
+                                 (connection, read) timeouts.
+        :type _request_timeout: int, tuple(int, int), optional
+        :param _request_auth: set to override the auth_settings for an a single
+                              request; this effectively ignores the
+                              authentication in the spec for a single request.
+        :type _request_auth: dict, optional
+        :param _content_type: force content-type for the request.
+        :type _content_type: str, Optional
+        :param _headers: set to override the headers for a single
+                         request; this effectively ignores the headers
+                         in the spec for a single request.
+        :type _headers: dict, optional
+        :param _host_index: set to override the host_index for a single
+                            request; this effectively ignores the host_index
+                            in the spec for a single request.
+        :type _host_index: int, optional
+        :return: Returns the result object.
+        """ # noqa: E501
+
+        _param = self._list_operations_serialize(
+            bank_id=bank_id,
+            authorization=authorization,
+            _request_auth=_request_auth,
+            _content_type=_content_type,
+            _headers=_headers,
+            _host_index=_host_index
+        )
+
+        _response_types_map: Dict[str, Optional[str]] = {
+            '200': "OperationsListResponse",
+            '422': "HTTPValidationError",
+        }
+        response_data = await self.api_client.call_api(
+            *_param,
+            _request_timeout=_request_timeout
+        )
+        return response_data.response
+
+
+    def _list_operations_serialize(
+        self,
+        bank_id,
+        authorization,
+        _request_auth,
+        _content_type,
+        _headers,
+        _host_index,
+    ) -> RequestSerialized:
+
+        _host = None
+
+        _collection_formats: Dict[str, str] = {
+        }
+
+        _path_params: Dict[str, str] = {}
+        _query_params: List[Tuple[str, str]] = []
+        _header_params: Dict[str, Optional[str]] = _headers or {}
+        _form_params: List[Tuple[str, str]] = []
+        _files: Dict[
+            str, Union[str, bytes, List[str], List[bytes], List[Tuple[str, bytes]]]
+        ] = {}
+        _body_params: Optional[bytes] = None
+
+        # process the path parameters
+        if bank_id is not None:
+            _path_params['bank_id'] = bank_id
+        # process the query parameters
+        # process the header parameters
+        if authorization is not None:
+            _header_params['authorization'] = authorization
+        # process the form parameters
+        # process the body parameter
+
+
+        # set the HTTP header `Accept`
+        if 'Accept' not in _header_params:
+            _header_params['Accept'] = self.api_client.select_header_accept(
+                [
+                    'application/json'
+                ]
+            )
+
+
+        # authentication setting
+        _auth_settings: List[str] = [
+        ]
+
+        return self.api_client.param_serialize(
+            method='GET',
+            resource_path='/v1/default/banks/{bank_id}/operations',
+            path_params=_path_params,
+            query_params=_query_params,
+            header_params=_header_params,
+            body=_body_params,
+            post_params=_form_params,
+            files=_files,
+            auth_settings=_auth_settings,
+            collection_formats=_collection_formats,
+            _host=_host,
+            _request_auth=_request_auth
+        )
+
+
@@ -5,7 +5,7 @@

    HTTP API for Hindsight

-    The version of the OpenAPI document: 1.0.0
+    The version of the OpenAPI document: 0.1.0
    Generated by OpenAPI Generator (https://openapi-generator.tech)

    Do not edit the class manually.
@@ -21,7 +21,6 @@ import mimetypes
 import os
 import re
 import tempfile
-import uuid

 from urllib.parse import quote
 from typing import Tuple, Optional, List, Dict, Union
@@ -360,8 +359,6 @@ class ApiClient:
            return obj.get_secret_value()
        elif isinstance(obj, self.PRIMITIVE_TYPES):
            return obj
-        elif isinstance(obj, uuid.UUID):
-            return str(obj)
        elif isinstance(obj, list):
            return [
                self.sanitize_for_serialization(sub_obj) for sub_obj in obj
@@ -388,10 +385,6 @@ class ApiClient:
            else:
                obj_dict = obj.__dict__

-        if isinstance(obj_dict, list):
-            # here we handle instances that can either be a list or something else, and only became a real list by calling to_dict()
-            return self.sanitize_for_serialization(obj_dict)
-
        return {
            key: self.sanitize_for_serialization(val)
            for key, val in obj_dict.items()
@@ -414,7 +407,7 @@ class ApiClient:
                data = json.loads(response_text)
            except ValueError:
                data = response_text
-        elif re.match(r'^application/(json|[\w!#$&.+\-^_]+\+json)\s*(;|$)', content_type, re.IGNORECASE):
+        elif re.match(r'^application/(json|[\w!#$&.+-^_]+\+json)\s*(;|$)', content_type, re.IGNORECASE):
            if response_text == "":
                data = ""
            else:
@@ -463,13 +456,13 @@ class ApiClient:

        if klass in self.PRIMITIVE_TYPES:
            return self.__deserialize_primitive(data, klass)
-        elif klass is object:
+        elif klass == object:
            return self.__deserialize_object(data)
-        elif klass is datetime.date:
+        elif klass == datetime.date:
            return self.__deserialize_date(data)
-        elif klass is datetime.datetime:
+        elif klass == datetime.datetime:
            return self.__deserialize_datetime(data)
-        elif klass is decimal.Decimal:
+        elif klass == decimal.Decimal:
            return decimal.Decimal(data)
        elif issubclass(klass, Enum):
            return self.__deserialize_enum(data, klass)
@@ -527,7 +520,7 @@ class ApiClient:
            if k in collection_formats:
                collection_format = collection_formats[k]
                if collection_format == 'multi':
-                    new_params.extend((k, quote(str(value))) for value in v)
+                    new_params.extend((k, str(value)) for value in v)
                else:
                    if collection_format == 'ssv':
                        delimiter = ' '
@@ -5,7 +5,7 @@

    HTTP API for Hindsight

-    The version of the OpenAPI document: 1.0.0
+    The version of the OpenAPI document: 0.1.0
    Generated by OpenAPI Generator (https://openapi-generator.tech)

    Do not edit the class manually.
@@ -17,7 +17,7 @@ import http.client as httplib
 import logging
 from logging import FileHandler
 import sys
-from typing import Any, ClassVar, Dict, List, Literal, Optional, TypedDict, Union
+from typing import Any, ClassVar, Dict, List, Literal, Optional, TypedDict
 from typing_extensions import NotRequired, Self

 import urllib3
@@ -159,10 +159,6 @@ class Configuration:
    :param ssl_ca_cert: str - the path to a file of concatenated CA certificates
      in PEM format.
    :param retries: Number of retries for API requests.
-    :param ca_cert_data: verify the peer using concatenated CA certificate data
-      in PEM (str) or DER (bytes) format.
-    :param cert_file: the path to a client certificate file, for mTLS.
-    :param key_file: the path to a client key file, for mTLS. 

    """

@@ -176,16 +172,13 @@ class Configuration:
        username: Optional[str]=None,
        password: Optional[str]=None,
        access_token: Optional[str]=None,
-        server_index: Optional[int]=None,
+        server_index: Optional[int]=None, 
        server_variables: Optional[ServerVariablesT]=None,
        server_operation_index: Optional[Dict[int, int]]=None,
        server_operation_variables: Optional[Dict[int, ServerVariablesT]]=None,
        ignore_operation_servers: bool=False,
        ssl_ca_cert: Optional[str]=None,
        retries: Optional[int] = None,
-        ca_cert_data: Optional[Union[str, bytes]] = None,
-        cert_file: Optional[str]=None,
-        key_file: Optional[str]=None,
        *,
        debug: Optional[bool] = None,
    ) -> None:
@@ -263,14 +256,10 @@ class Configuration:
        self.ssl_ca_cert = ssl_ca_cert
        """Set this to customize the certificate file to verify the peer.
        """
-        self.ca_cert_data = ca_cert_data
-        """Set this to verify the peer using PEM (str) or DER (bytes)
-           certificate data.
-        """
-        self.cert_file = cert_file
+        self.cert_file = None
        """client certificate file
        """
-        self.key_file = key_file
+        self.key_file = None
        """client key file
        """
        self.assert_hostname = None
@@ -500,7 +489,7 @@ class Configuration:
        return "Python SDK Debug Report:\n"\
               "OS: {env}\n"\
               "Python Version: {pyversion}\n"\
-               "Version of the API: 1.0.0\n"\
+               "Version of the API: 0.1.0\n"\
               "SDK Package Version: 0.0.7".\
               format(env=sys.platform, pyversion=sys.version)

@@ -1,31 +0,0 @@
-# AddBackgroundRequest
-
-Request model for adding/merging background information.
-
-## Properties
-
-Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
-**content** | **str** | New background information to add or merge | 
-**update_disposition** | **bool** | If true, infer disposition traits from the merged background (default: true) | [optional] [default to True]
-
-## Example
-
-```python
-from hindsight_client_api.models.add_background_request import AddBackgroundRequest
-
-# TODO update the JSON string below
-json = "{}"
-# create an instance of AddBackgroundRequest from a JSON string
-add_background_request_instance = AddBackgroundRequest.from_json(json)
-# print the JSON string representation of the object
-print(AddBackgroundRequest.to_json())
-
-# convert the object into a dict
-add_background_request_dict = add_background_request_instance.to_dict()
-# create an instance of AddBackgroundRequest from a dict
-add_background_request_from_dict = AddBackgroundRequest.from_dict(add_background_request_dict)
-```
-[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
-
-
@@ -1,31 +0,0 @@
-# BackgroundResponse
-
-Response model for background update.
-
-## Properties
-
-Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
-**background** | **str** |  | 
-**disposition** | [**DispositionTraits**](DispositionTraits.md) |  | [optional] 
-
-## Example
-
-```python
-from hindsight_client_api.models.background_response import BackgroundResponse
-
-# TODO update the JSON string below
-json = "{}"
-# create an instance of BackgroundResponse from a JSON string
-background_response_instance = BackgroundResponse.from_json(json)
-# print the JSON string representation of the object
-print(BackgroundResponse.to_json())
-
-# convert the object into a dict
-background_response_dict = background_response_instance.to_dict()
-# create an instance of BackgroundResponse from a dict
-background_response_from_dict = BackgroundResponse.from_dict(background_response_dict)
-```
-[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
-
-
@@ -1,35 +0,0 @@
-# BankListItem
-
-Bank list item with profile summary.
-
-## Properties
-
-Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
-**bank_id** | **str** |  | 
-**name** | **str** |  | 
-**disposition** | [**DispositionTraits**](DispositionTraits.md) |  | 
-**background** | **str** |  | 
-**created_at** | **str** |  | [optional] 
-**updated_at** | **str** |  | [optional] 
-
-## Example
-
-```python
-from hindsight_client_api.models.bank_list_item import BankListItem
-
-# TODO update the JSON string below
-json = "{}"
-# create an instance of BankListItem from a JSON string
-bank_list_item_instance = BankListItem.from_json(json)
-# print the JSON string representation of the object
-print(BankListItem.to_json())
-
-# convert the object into a dict
-bank_list_item_dict = bank_list_item_instance.to_dict()
-# create an instance of BankListItem from a dict
-bank_list_item_from_dict = BankListItem.from_dict(bank_list_item_dict)
-```
-[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
-
-
@@ -1,30 +0,0 @@
-# BankListResponse
-
-Response model for listing all banks.
-
-## Properties
-
-Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
-**banks** | [**List[BankListItem]**](BankListItem.md) |  | 
-
-## Example
-
-```python
-from hindsight_client_api.models.bank_list_response import BankListResponse
-
-# TODO update the JSON string below
-json = "{}"
-# create an instance of BankListResponse from a JSON string
-bank_list_response_instance = BankListResponse.from_json(json)
-# print the JSON string representation of the object
-print(BankListResponse.to_json())
-
-# convert the object into a dict
-bank_list_response_dict = bank_list_response_instance.to_dict()
-# create an instance of BankListResponse from a dict
-bank_list_response_from_dict = BankListResponse.from_dict(bank_list_response_dict)
-```
-[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
-
-
@@ -1,33 +0,0 @@
-# BankProfileResponse
-
-Response model for bank profile.
-
-## Properties
-
-Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
-**bank_id** | **str** |  | 
-**name** | **str** |  | 
-**disposition** | [**DispositionTraits**](DispositionTraits.md) |  | 
-**background** | **str** |  | 
-
-## Example
-
-```python
-from hindsight_client_api.models.bank_profile_response import BankProfileResponse
-
-# TODO update the JSON string below
-json = "{}"
-# create an instance of BankProfileResponse from a JSON string
-bank_profile_response_instance = BankProfileResponse.from_json(json)
-# print the JSON string representation of the object
-print(BankProfileResponse.to_json())
-
-# convert the object into a dict
-bank_profile_response_dict = bank_profile_response_instance.to_dict()
-# create an instance of BankProfileResponse from a dict
-bank_profile_response_from_dict = BankProfileResponse.from_dict(bank_profile_response_dict)
-```
-[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
-
-
@@ -1,15 +0,0 @@
-# Budget
-
-Budget levels for recall/reflect operations.
-
-## Enum
-
-* `LOW` (value: `'low'`)
-
-* `MID` (value: `'mid'`)
-
-* `HIGH` (value: `'high'`)
-
-[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
-
-
@@ -1,33 +0,0 @@
-# ChunkData
-
-Chunk data for a single chunk.
-
-## Properties
-
-Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
-**id** | **str** |  | 
-**text** | **str** |  | 
-**chunk_index** | **int** |  | 
-**truncated** | **bool** | Whether the chunk text was truncated due to token limits | [optional] [default to False]
-
-## Example
-
-```python
-from hindsight_client_api.models.chunk_data import ChunkData
-
-# TODO update the JSON string below
-json = "{}"
-# create an instance of ChunkData from a JSON string
-chunk_data_instance = ChunkData.from_json(json)
-# print the JSON string representation of the object
-print(ChunkData.to_json())
-
-# convert the object into a dict
-chunk_data_dict = chunk_data_instance.to_dict()
-# create an instance of ChunkData from a dict
-chunk_data_from_dict = ChunkData.from_dict(chunk_data_dict)
-```
-[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
-
-
@@ -1,30 +0,0 @@
-# ChunkIncludeOptions
-
-Options for including chunks in recall results.
-
-## Properties
-
-Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
-**max_tokens** | **int** | Maximum tokens for chunks (chunks may be truncated) | [optional] [default to 8192]
-
-## Example
-
-```python
-from hindsight_client_api.models.chunk_include_options import ChunkIncludeOptions
-
-# TODO update the JSON string below
-json = "{}"
-# create an instance of ChunkIncludeOptions from a JSON string
-chunk_include_options_instance = ChunkIncludeOptions.from_json(json)
-# print the JSON string representation of the object
-print(ChunkIncludeOptions.to_json())
-
-# convert the object into a dict
-chunk_include_options_dict = chunk_include_options_instance.to_dict()
-# create an instance of ChunkIncludeOptions from a dict
-chunk_include_options_from_dict = ChunkIncludeOptions.from_dict(chunk_include_options_dict)
-```
-[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
-
-
@@ -1,35 +0,0 @@
-# ChunkResponse
-
-Response model for get chunk endpoint.
-
-## Properties
-
-Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
-**chunk_id** | **str** |  | 
-**document_id** | **str** |  | 
-**bank_id** | **str** |  | 
-**chunk_index** | **int** |  | 
-**chunk_text** | **str** |  | 
-**created_at** | **str** |  | 
-
-## Example
-
-```python
-from hindsight_client_api.models.chunk_response import ChunkResponse
-
-# TODO update the JSON string below
-json = "{}"
-# create an instance of ChunkResponse from a JSON string
-chunk_response_instance = ChunkResponse.from_json(json)
-# print the JSON string representation of the object
-print(ChunkResponse.to_json())
-
-# convert the object into a dict
-chunk_response_dict = chunk_response_instance.to_dict()
-# create an instance of ChunkResponse from a dict
-chunk_response_from_dict = ChunkResponse.from_dict(chunk_response_dict)
-```
-[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
-
-
@@ -1,32 +0,0 @@
-# CreateBankRequest
-
-Request model for creating/updating a bank.
-
-## Properties
-
-Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
-**name** | **str** |  | [optional] 
-**disposition** | [**DispositionTraits**](DispositionTraits.md) |  | [optional] 
-**background** | **str** |  | [optional] 
-
-## Example
-
-```python
-from hindsight_client_api.models.create_bank_request import CreateBankRequest
-
-# TODO update the JSON string below
-json = "{}"
-# create an instance of CreateBankRequest from a JSON string
-create_bank_request_instance = CreateBankRequest.from_json(json)
-# print the JSON string representation of the object
-print(CreateBankRequest.to_json())
-
-# convert the object into a dict
-create_bank_request_dict = create_bank_request_instance.to_dict()
-# create an instance of CreateBankRequest from a dict
-create_bank_request_from_dict = CreateBankRequest.from_dict(create_bank_request_dict)
-```
-[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
-
-
@@ -1,30 +0,0 @@
-# DeleteResponse
-
-Response model for delete operations.
-
-## Properties
-
-Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
-**success** | **bool** |  | 
-
-## Example
-
-```python
-from hindsight_client_api.models.delete_response import DeleteResponse
-
-# TODO update the JSON string below
-json = "{}"
-# create an instance of DeleteResponse from a JSON string
-delete_response_instance = DeleteResponse.from_json(json)
-# print the JSON string representation of the object
-print(DeleteResponse.to_json())
-
-# convert the object into a dict
-delete_response_dict = delete_response_instance.to_dict()
-# create an instance of DeleteResponse from a dict
-delete_response_from_dict = DeleteResponse.from_dict(delete_response_dict)
-```
-[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
-
-
@@ -1,32 +0,0 @@
-# DispositionTraits
-
-Disposition traits that influence how memories are formed and interpreted.
-
-## Properties
-
-Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
-**skepticism** | **int** | How skeptical vs trusting (1&#x3D;trusting, 5&#x3D;skeptical) | 
-**literalism** | **int** | How literally to interpret information (1&#x3D;flexible, 5&#x3D;literal) | 
-**empathy** | **int** | How much to consider emotional context (1&#x3D;detached, 5&#x3D;empathetic) | 
-
-## Example
-
-```python
-from hindsight_client_api.models.disposition_traits import DispositionTraits
-
-# TODO update the JSON string below
-json = "{}"
-# create an instance of DispositionTraits from a JSON string
-disposition_traits_instance = DispositionTraits.from_json(json)
-# print the JSON string representation of the object
-print(DispositionTraits.to_json())
-
-# convert the object into a dict
-disposition_traits_dict = disposition_traits_instance.to_dict()
-# create an instance of DispositionTraits from a dict
-disposition_traits_from_dict = DispositionTraits.from_dict(disposition_traits_dict)
-```
-[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
-
-
@@ -1,36 +0,0 @@
-# DocumentResponse
-
-Response model for get document endpoint.
-
-## Properties
-
-Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
-**id** | **str** |  | 
-**bank_id** | **str** |  | 
-**original_text** | **str** |  | 
-**content_hash** | **str** |  | 
-**created_at** | **str** |  | 
-**updated_at** | **str** |  | 
-**memory_unit_count** | **int** |  | 
-
-## Example
-
-```python
-from hindsight_client_api.models.document_response import DocumentResponse
-
-# TODO update the JSON string below
-json = "{}"
-# create an instance of DocumentResponse from a JSON string
-document_response_instance = DocumentResponse.from_json(json)
-# print the JSON string representation of the object
-print(DocumentResponse.to_json())
-
-# convert the object into a dict
-document_response_dict = document_response_instance.to_dict()
-# create an instance of DocumentResponse from a dict
-document_response_from_dict = DocumentResponse.from_dict(document_response_dict)
-```
-[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
-
-
@@ -1,36 +0,0 @@
-# EntityDetailResponse
-
-Response model for entity detail endpoint.
-
-## Properties
-
-Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
-**id** | **str** |  | 
-**canonical_name** | **str** |  | 
-**mention_count** | **int** |  | 
-**first_seen** | **str** |  | [optional] 
-**last_seen** | **str** |  | [optional] 
-**metadata** | **Dict[str, object]** |  | [optional] 
-**observations** | [**List[EntityObservationResponse]**](EntityObservationResponse.md) |  | 
-
-## Example
-
-```python
-from hindsight_client_api.models.entity_detail_response import EntityDetailResponse
-
-# TODO update the JSON string below
-json = "{}"
-# create an instance of EntityDetailResponse from a JSON string
-entity_detail_response_instance = EntityDetailResponse.from_json(json)
-# print the JSON string representation of the object
-print(EntityDetailResponse.to_json())
-
-# convert the object into a dict
-entity_detail_response_dict = entity_detail_response_instance.to_dict()
-# create an instance of EntityDetailResponse from a dict
-entity_detail_response_from_dict = EntityDetailResponse.from_dict(entity_detail_response_dict)
-```
-[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
-
-
@@ -1,30 +0,0 @@
-# EntityIncludeOptions
-
-Options for including entity observations in recall results.
-
-## Properties
-
-Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
-**max_tokens** | **int** | Maximum tokens for entity observations | [optional] [default to 500]
-
-## Example
-
-```python
-from hindsight_client_api.models.entity_include_options import EntityIncludeOptions
-
-# TODO update the JSON string below
-json = "{}"
-# create an instance of EntityIncludeOptions from a JSON string
-entity_include_options_instance = EntityIncludeOptions.from_json(json)
-# print the JSON string representation of the object
-print(EntityIncludeOptions.to_json())
-
-# convert the object into a dict
-entity_include_options_dict = entity_include_options_instance.to_dict()
-# create an instance of EntityIncludeOptions from a dict
-entity_include_options_from_dict = EntityIncludeOptions.from_dict(entity_include_options_dict)
-```
-[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
-
-
@@ -1,35 +0,0 @@
-# EntityListItem
-
-Entity list item with summary.
-
-## Properties
-
-Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
-**id** | **str** |  | 
-**canonical_name** | **str** |  | 
-**mention_count** | **int** |  | 
-**first_seen** | **str** |  | [optional] 
-**last_seen** | **str** |  | [optional] 
-**metadata** | **Dict[str, object]** |  | [optional] 
-
-## Example
-
-```python
-from hindsight_client_api.models.entity_list_item import EntityListItem
-
-# TODO update the JSON string below
-json = "{}"
-# create an instance of EntityListItem from a JSON string
-entity_list_item_instance = EntityListItem.from_json(json)
-# print the JSON string representation of the object
-print(EntityListItem.to_json())
-
-# convert the object into a dict
-entity_list_item_dict = entity_list_item_instance.to_dict()
-# create an instance of EntityListItem from a dict
-entity_list_item_from_dict = EntityListItem.from_dict(entity_list_item_dict)
-```
-[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
-
-
@@ -1,30 +0,0 @@
-# EntityListResponse
-
-Response model for entity list endpoint.
-
-## Properties
-
-Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
-**items** | [**List[EntityListItem]**](EntityListItem.md) |  | 
-
-## Example
-
-```python
-from hindsight_client_api.models.entity_list_response import EntityListResponse
-
-# TODO update the JSON string below
-json = "{}"
-# create an instance of EntityListResponse from a JSON string
-entity_list_response_instance = EntityListResponse.from_json(json)
-# print the JSON string representation of the object
-print(EntityListResponse.to_json())
-
-# convert the object into a dict
-entity_list_response_dict = entity_list_response_instance.to_dict()
-# create an instance of EntityListResponse from a dict
-entity_list_response_from_dict = EntityListResponse.from_dict(entity_list_response_dict)
-```
-[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
-
-
@@ -1,31 +0,0 @@
-# EntityObservationResponse
-
-An observation about an entity.
-
-## Properties
-
-Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
-**text** | **str** |  | 
-**mentioned_at** | **str** |  | [optional] 
-
-## Example
-
-```python
-from hindsight_client_api.models.entity_observation_response import EntityObservationResponse
-
-# TODO update the JSON string below
-json = "{}"
-# create an instance of EntityObservationResponse from a JSON string
-entity_observation_response_instance = EntityObservationResponse.from_json(json)
-# print the JSON string representation of the object
-print(EntityObservationResponse.to_json())
-
-# convert the object into a dict
-entity_observation_response_dict = entity_observation_response_instance.to_dict()
-# create an instance of EntityObservationResponse from a dict
-entity_observation_response_from_dict = EntityObservationResponse.from_dict(entity_observation_response_dict)
-```
-[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
-
-
@@ -1,32 +0,0 @@
-# EntityStateResponse
-
-Current mental model of an entity.
-
-## Properties
-
-Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
-**entity_id** | **str** |  | 
-**canonical_name** | **str** |  | 
-**observations** | [**List[EntityObservationResponse]**](EntityObservationResponse.md) |  | 
-
-## Example
-
-```python
-from hindsight_client_api.models.entity_state_response import EntityStateResponse
-
-# TODO update the JSON string below
-json = "{}"
-# create an instance of EntityStateResponse from a JSON string
-entity_state_response_instance = EntityStateResponse.from_json(json)
-# print the JSON string representation of the object
-print(EntityStateResponse.to_json())
-
-# convert the object into a dict
-entity_state_response_dict = entity_state_response_instance.to_dict()
-# create an instance of EntityStateResponse from a dict
-entity_state_response_from_dict = EntityStateResponse.from_dict(entity_state_response_dict)
-```
-[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
-
-
@@ -1,33 +0,0 @@
-# GraphDataResponse
-
-Response model for graph data endpoint.
-
-## Properties
-
-Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
-**nodes** | **List[Dict[str, object]]** |  | 
-**edges** | **List[Dict[str, object]]** |  | 
-**table_rows** | **List[Dict[str, object]]** |  | 
-**total_units** | **int** |  | 
-
-## Example
-
-```python
-from hindsight_client_api.models.graph_data_response import GraphDataResponse
-
-# TODO update the JSON string below
-json = "{}"
-# create an instance of GraphDataResponse from a JSON string
-graph_data_response_instance = GraphDataResponse.from_json(json)
-# print the JSON string representation of the object
-print(GraphDataResponse.to_json())
-
-# convert the object into a dict
-graph_data_response_dict = graph_data_response_instance.to_dict()
-# create an instance of GraphDataResponse from a dict
-graph_data_response_from_dict = GraphDataResponse.from_dict(graph_data_response_dict)
-```
-[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
-
-
@@ -1,29 +0,0 @@
-# HTTPValidationError
-
-
-## Properties
-
-Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
-**detail** | [**List[ValidationError]**](ValidationError.md) |  | [optional] 
-
-## Example
-
-```python
-from hindsight_client_api.models.http_validation_error import HTTPValidationError
-
-# TODO update the JSON string below
-json = "{}"
-# create an instance of HTTPValidationError from a JSON string
-http_validation_error_instance = HTTPValidationError.from_json(json)
-# print the JSON string representation of the object
-print(HTTPValidationError.to_json())
-
-# convert the object into a dict
-http_validation_error_dict = http_validation_error_instance.to_dict()
-# create an instance of HTTPValidationError from a dict
-http_validation_error_from_dict = HTTPValidationError.from_dict(http_validation_error_dict)
-```
-[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
-
-
@@ -1,31 +0,0 @@
-# IncludeOptions
-
-Options for including additional data in recall results.
-
-## Properties
-
-Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
-**entities** | [**EntityIncludeOptions**](EntityIncludeOptions.md) |  | [optional] 
-**chunks** | [**ChunkIncludeOptions**](ChunkIncludeOptions.md) |  | [optional] 
-
-## Example
-
-```python
-from hindsight_client_api.models.include_options import IncludeOptions
-
-# TODO update the JSON string below
-json = "{}"
-# create an instance of IncludeOptions from a JSON string
-include_options_instance = IncludeOptions.from_json(json)
-# print the JSON string representation of the object
-print(IncludeOptions.to_json())
-
-# convert the object into a dict
-include_options_dict = include_options_instance.to_dict()
-# create an instance of IncludeOptions from a dict
-include_options_from_dict = IncludeOptions.from_dict(include_options_dict)
-```
-[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
-
-
--- a/Show More
+++ b/Show More