Compare commits

..
1 Commits
Author SHA1 Message Date
Nicolò Boschi df849fba76 feat: support cohere as embeddings and reranker 2026-01-08 11:32:22 +01:00
934 changed files with 16972 additions and 189103 deletions
+1 -36
View File
@@ -2,7 +2,7 @@
# Copy this file to .env and fill in your values
# LLM Configuration (Required)
# Supported providers: openai, groq, ollama, gemini, anthropic, lmstudio, vertexai
# Supported providers: openai, groq, ollama, gemini, anthropic, lmstudio
HINDSIGHT_API_LLM_PROVIDER=openai
HINDSIGHT_API_LLM_API_KEY=your-api-key-here
HINDSIGHT_API_LLM_MODEL=o3-mini
@@ -13,13 +13,6 @@ HINDSIGHT_API_LLM_BASE_URL=https://api.openai.com/v1
# HINDSIGHT_API_LLM_API_KEY=your-anthropic-api-key
# HINDSIGHT_API_LLM_MODEL=claude-sonnet-4-20250514
# Example: Google Vertex AI configuration
# HINDSIGHT_API_LLM_PROVIDER=vertexai
# HINDSIGHT_API_LLM_MODEL=google/gemini-2.0-flash-001
# HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=your-gcp-project-id
# HINDSIGHT_API_LLM_VERTEXAI_REGION=us-central1
# HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY=/path/to/service-account-key.json # Optional, uses ADC if not set
# Example: LM Studio local configuration (Qwen 2.5 32B recommended)
# HINDSIGHT_API_LLM_PROVIDER=lmstudio
# HINDSIGHT_API_LLM_API_KEY=lmstudio
@@ -31,21 +24,8 @@ HINDSIGHT_API_HOST=0.0.0.0
HINDSIGHT_API_PORT=8888
HINDSIGHT_API_LOG_LEVEL=info
# Base Path / Reverse Proxy Support (Optional)
# Set these when deploying behind a reverse proxy with path-based routing
# Example: To deploy at example.com/hindsight/, set both to "/hindsight"
# HINDSIGHT_API_BASE_PATH=/hindsight
# NEXT_PUBLIC_BASE_PATH=/hindsight
# Database (Optional - uses embedded pg0 by default)
# HINDSIGHT_API_DATABASE_URL=postgresql://user:pass@host:5432/db
# HINDSIGHT_API_DATABASE_SCHEMA=public # PostgreSQL schema name (default: public)
# Vector Extension (Optional - uses pgvector by default)
# Options: "pgvector" (default), "vchord", "pgvectorscale" (DiskANN)
# HINDSIGHT_API_VECTOR_EXTENSION=pgvector
# For Azure PostgreSQL with DiskANN:
# HINDSIGHT_API_VECTOR_EXTENSION=pgvectorscale # Auto-detects pg_diskann on Azure
# Embeddings Configuration (Optional - uses local by default)
# Provider: "local" (default) or "tei" (HuggingFace Text Embeddings Inference)
@@ -62,18 +42,3 @@ HINDSIGHT_API_LOG_LEVEL=info
# HINDSIGHT_API_RERANKER_LOCAL_MODEL=cross-encoder/ms-marco-MiniLM-L-6-v2
# For TEI provider:
# HINDSIGHT_API_RERANKER_TEI_URL=http://localhost:8081
# Observability & Tracing (Optional - disabled by default)
# Enable OpenTelemetry tracing for LLM calls (GenAI semantic conventions)
# HINDSIGHT_API_OTEL_TRACES_ENABLED=true
#
# Local development with Grafana LGTM stack (recommended - see scripts/dev/grafana/README.md)
# HINDSIGHT_API_OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
#
# Cloud backends (Grafana Cloud, Langfuse, DataDog, etc.)
# HINDSIGHT_API_OTEL_EXPORTER_OTLP_ENDPOINT=https://your-backend-url
# HINDSIGHT_API_OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer your-token"
#
# Custom service name and environment (optional, defaults: hindsight-api, development)
# HINDSIGHT_API_OTEL_SERVICE_NAME=hindsight-production
# HINDSIGHT_API_OTEL_DEPLOYMENT_ENVIRONMENT=production
+2 -196
View File
@@ -139,153 +139,6 @@ jobs:
path: hindsight-clients/typescript/*.tgz
retention-days: 1
release-openclaw-integration:
runs-on: ubuntu-latest
environment: npm
steps:
- uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '22'
registry-url: 'https://registry.npmjs.org'
- name: Install dependencies
working-directory: ./hindsight-integrations/openclaw
run: npm ci
- name: Build
working-directory: ./hindsight-integrations/openclaw
run: npm run build
- name: Publish to npm
working-directory: ./hindsight-integrations/openclaw
run: |
set +e
OUTPUT=$(npm publish --access public 2>&1)
EXIT_CODE=$?
echo "$OUTPUT"
if [ $EXIT_CODE -ne 0 ]; then
if echo "$OUTPUT" | grep -q "cannot publish over"; then
echo "Package version already published, skipping..."
exit 0
fi
exit $EXIT_CODE
fi
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
- name: Pack for GitHub release
working-directory: ./hindsight-integrations/openclaw
run: npm pack
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: openclaw-integration
path: hindsight-integrations/openclaw/*.tgz
retention-days: 1
release-nemoclaw-integration:
runs-on: ubuntu-latest
environment: npm
steps:
- uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '22'
registry-url: 'https://registry.npmjs.org'
- name: Install dependencies
working-directory: ./hindsight-integrations/nemoclaw
run: npm ci
- name: Build
working-directory: ./hindsight-integrations/nemoclaw
run: npm run build
- name: Publish to npm
working-directory: ./hindsight-integrations/nemoclaw
run: |
set +e
OUTPUT=$(npm publish --access public 2>&1)
EXIT_CODE=$?
echo "$OUTPUT"
if [ $EXIT_CODE -ne 0 ]; then
if echo "$OUTPUT" | grep -q "cannot publish over"; then
echo "Package version already published, skipping..."
exit 0
fi
exit $EXIT_CODE
fi
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
- name: Pack for GitHub release
working-directory: ./hindsight-integrations/nemoclaw
run: npm pack
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: nemoclaw-integration
path: hindsight-integrations/nemoclaw/*.tgz
retention-days: 1
release-ai-sdk-integration:
runs-on: ubuntu-latest
environment: npm
steps:
- uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '22'
registry-url: 'https://registry.npmjs.org'
- name: Install dependencies
working-directory: ./hindsight-integrations/ai-sdk
run: npm ci
- name: Build
working-directory: ./hindsight-integrations/ai-sdk
run: npm run build
- name: Publish to npm
working-directory: ./hindsight-integrations/ai-sdk
run: |
set +e
OUTPUT=$(npm publish --access public 2>&1)
EXIT_CODE=$?
echo "$OUTPUT"
if [ $EXIT_CODE -ne 0 ]; then
if echo "$OUTPUT" | grep -q "cannot publish over"; then
echo "Package version already published, skipping..."
exit 0
fi
exit $EXIT_CODE
fi
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
- name: Pack for GitHub release
working-directory: ./hindsight-integrations/ai-sdk
run: npm pack
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: ai-sdk-integration
path: hindsight-integrations/ai-sdk/*.tgz
retention-days: 1
release-control-plane:
runs-on: ubuntu-latest
environment: npm
@@ -389,7 +242,6 @@ jobs:
retention-days: 1
release-docker-images:
name: Release Docker (${{ matrix.image_name }}${{ matrix.tag_suffix }})
runs-on: ubuntu-latest
permissions:
contents: read
@@ -399,28 +251,10 @@ jobs:
include:
- target: api-only
image_name: hindsight-api
tag_suffix: ""
build_args: ""
- target: api-only
image_name: hindsight-api
tag_suffix: "-slim"
build_args: |
INCLUDE_LOCAL_MODELS=false
PRELOAD_ML_MODELS=false
- target: cp-only
image_name: hindsight-control-plane
tag_suffix: ""
build_args: ""
- target: standalone
image_name: hindsight
tag_suffix: ""
build_args: ""
- target: standalone
image_name: hindsight
tag_suffix: "-slim"
build_args: |
INCLUDE_LOCAL_MODELS=false
PRELOAD_ML_MODELS=false
steps:
- uses: actions/checkout@v4
@@ -458,9 +292,6 @@ jobs:
uses: docker/metadata-action@v5
with:
images: ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}
flavor: |
latest=auto
suffix=${{ matrix.tag_suffix }}
tags: |
type=semver,pattern={{version}},value=${{ steps.get_version.outputs.VERSION }}
type=semver,pattern={{major}}.{{minor}},value=${{ steps.get_version.outputs.VERSION }}
@@ -486,7 +317,7 @@ jobs:
# - name: Smoke test - verify container starts
# env:
# GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
# run: ./docker/test-image.sh "${{ matrix.image_name }}:test" "${{ matrix.target }}"
# run: ./scripts/docker-smoke-test.sh "${{ matrix.image_name }}:test" "${{ matrix.target }}"
# Build multi-platform and push to release tags
- name: Build and push release images
@@ -495,7 +326,6 @@ jobs:
context: .
file: docker/standalone/Dockerfile
target: ${{ matrix.target }}
build-args: ${{ matrix.build_args }}
push: true
platforms: linux/amd64,linux/arm64
tags: ${{ steps.meta.outputs.tags }}
@@ -536,7 +366,7 @@ jobs:
create-github-release:
runs-on: ubuntu-latest
needs: [release-python-packages, release-typescript-client, release-openclaw-integration, release-nemoclaw-integration, release-ai-sdk-integration, release-control-plane, release-rust-cli, release-docker-images, release-helm-chart]
needs: [release-python-packages, release-typescript-client, release-control-plane, release-rust-cli, release-docker-images, release-helm-chart]
permissions:
contents: write
@@ -559,24 +389,6 @@ jobs:
name: typescript-client
path: ./artifacts/typescript-client
- name: Download OpenClaw Integration
uses: actions/download-artifact@v4
with:
name: openclaw-integration
path: ./artifacts/openclaw-integration
- name: Download NemoClaw Integration
uses: actions/download-artifact@v4
with:
name: nemoclaw-integration
path: ./artifacts/nemoclaw-integration
- name: Download AI SDK Integration
uses: actions/download-artifact@v4
with:
name: ai-sdk-integration
path: ./artifacts/ai-sdk-integration
- name: Download Control Plane
uses: actions/download-artifact@v4
with:
@@ -618,12 +430,6 @@ jobs:
cp artifacts/python-packages/hindsight-embed/dist/* release-assets/ || true
# TypeScript client
cp artifacts/typescript-client/*.tgz release-assets/ || true
# OpenClaw Integration
cp artifacts/openclaw-integration/*.tgz release-assets/ || true
# NemoClaw Integration
cp artifacts/nemoclaw-integration/*.tgz release-assets/ || true
# AI SDK Integration
cp artifacts/ai-sdk-integration/*.tgz release-assets/ || true
# Control Plane
cp artifacts/control-plane/*.tgz release-assets/ || true
# Rust CLI binaries
+105 -470
View File
@@ -9,11 +9,42 @@ concurrency:
cancel-in-progress: true
jobs:
build-python-packages:
runs-on: ubuntu-latest
strategy:
matrix:
include:
- name: hindsight-all
path: hindsight
- name: hindsight-api
path: hindsight-api
- name: hindsight-client
path: hindsight-clients/python
- name: hindsight-embed
path: hindsight-embed
steps:
- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
enable-cache: true
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version-file: ".python-version"
- name: Build ${{ matrix.name }}
working-directory: ./${{ matrix.path }}
run: uv build
build-api-python-versions:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.11', '3.12', '3.13', '3.14']
python-version: ['3.11', '3.12', '3.13']
steps:
- uses: actions/checkout@v4
@@ -51,52 +82,6 @@ jobs:
- name: Build TypeScript client
run: npm run build --workspace=hindsight-clients/typescript
build-openclaw-integration:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '22'
- name: Install dependencies
working-directory: ./hindsight-integrations/openclaw
run: npm ci
- name: Run tests
working-directory: ./hindsight-integrations/openclaw
run: npm test
- name: Build
working-directory: ./hindsight-integrations/openclaw
run: npm run build
build-ai-sdk-integration:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '22'
- name: Install dependencies
working-directory: ./hindsight-integrations/ai-sdk
run: npm ci
- name: Run tests
working-directory: ./hindsight-integrations/ai-sdk
run: npm test
- name: Build
working-directory: ./hindsight-integrations/ai-sdk
run: npm run build
build-control-plane:
runs-on: ubuntu-latest
@@ -168,15 +153,8 @@ jobs:
- name: Build docs
run: npm run build --workspace=hindsight-docs
test-rust-cli:
build-rust-cli:
runs-on: ubuntu-latest
env:
HINDSIGHT_API_LLM_PROVIDER: groq
HINDSIGHT_API_LLM_API_KEY: ${{ secrets.GROQ_API_KEY }}
HINDSIGHT_API_LLM_MODEL: openai/gpt-oss-20b
HINDSIGHT_API_URL: http://localhost:8888
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
UV_INDEX: pytorch=https://download.pytorch.org/whl/cpu
steps:
- uses: actions/checkout@v4
@@ -193,10 +171,6 @@ jobs:
hindsight-cli/target
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
- name: Run unit tests
working-directory: hindsight-cli
run: cargo test
- name: Build CLI
working-directory: hindsight-cli
run: cargo build --release
@@ -208,6 +182,29 @@ jobs:
path: hindsight-cli/target/release/hindsight
retention-days: 1
test-rust-cli:
runs-on: ubuntu-latest
needs: build-rust-cli
env:
HINDSIGHT_API_LLM_PROVIDER: groq
HINDSIGHT_API_LLM_API_KEY: ${{ secrets.GROQ_API_KEY }}
HINDSIGHT_API_LLM_MODEL: openai/gpt-oss-20b
HINDSIGHT_API_URL: http://localhost:8888
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
UV_INDEX: pytorch=https://download.pytorch.org/whl/cpu
steps:
- uses: actions/checkout@v4
- name: Download CLI artifact
uses: actions/download-artifact@v4
with:
name: hindsight-cli
path: /tmp/cli
- name: Make CLI executable
run: chmod +x /tmp/cli/hindsight
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
@@ -225,7 +222,7 @@ jobs:
- name: Install API dependencies
working-directory: ./hindsight-api
run: uv sync --frozen --no-install-project --index-strategy unsafe-best-match
run: uv sync --no-install-project --index-strategy unsafe-best-match
- name: Create .env file
run: |
@@ -254,7 +251,7 @@ jobs:
- name: Run CLI smoke test
run: |
HINDSIGHT_CLI=hindsight-cli/target/release/hindsight ./hindsight-cli/smoke-test.sh
HINDSIGHT_CLI=/tmp/cli/hindsight ./hindsight-cli/smoke-test.sh
- name: Show API server logs
if: always()
@@ -277,35 +274,16 @@ jobs:
run: helm lint helm/hindsight
build-docker-images:
name: Build Docker (${{ matrix.name }})
runs-on: ubuntu-latest
strategy:
matrix:
include:
- target: api-only
name: api
variant: full
build_args: ""
- target: api-only
name: api-slim
variant: slim
build_args: |
INCLUDE_LOCAL_MODELS=false
PRELOAD_ML_MODELS=false
- target: cp-only
name: control-plane
variant: full
build_args: ""
- target: standalone
name: standalone
variant: full
build_args: ""
- target: standalone
name: standalone-slim
variant: slim
build_args: |
INCLUDE_LOCAL_MODELS=false
PRELOAD_ML_MODELS=false
steps:
- uses: actions/checkout@v4
@@ -324,31 +302,20 @@ jobs:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build ${{ matrix.name }} image (${{ matrix.variant }})
- name: Build ${{ matrix.name }} image
uses: docker/build-push-action@v6
with:
context: .
file: docker/standalone/Dockerfile
target: ${{ matrix.target }}
build-args: ${{ matrix.build_args }}
push: false
load: ${{ matrix.variant == 'slim' }}
tags: hindsight-${{ matrix.name }}:test
# Removed GitHub Actions cache (type=gha) - it frequently returns 502 errors
# causing buildx to fail with "failed to parse error response 502"
# Build will be slower but more reliable
load: false
# Only test slim variants to save disk space (they're much smaller)
# Slim variants require external embedding providers
- name: Smoke test - verify container starts
if: matrix.variant == 'slim'
env:
GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
HINDSIGHT_API_EMBEDDINGS_PROVIDER: openai
HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
HINDSIGHT_API_RERANKER_PROVIDER: cohere
HINDSIGHT_API_COHERE_API_KEY: ${{ secrets.COHERE_API_KEY }}
run: ./docker/test-image.sh "hindsight-${{ matrix.name }}:test" "${{ matrix.target }}"
# TODO: Re-enable smoke test when disk space issue is resolved
# - name: Smoke test - verify container starts
# env:
# GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
# run: ./scripts/docker-smoke-test.sh "hindsight-${{ matrix.name }}:test" "${{ matrix.target }}"
test-api:
runs-on: ubuntu-latest
@@ -385,7 +352,7 @@ jobs:
- name: Install dependencies
working-directory: ./hindsight-api
run: uv sync --frozen --extra test --no-install-project --index-strategy unsafe-best-match
run: uv sync --extra test --no-install-project --index-strategy unsafe-best-match
- name: Cache HuggingFace models
uses: actions/cache@v4
@@ -446,11 +413,11 @@ jobs:
- name: Install client test dependencies
working-directory: ./hindsight-clients/python
run: uv sync --frozen --extra test --index-strategy unsafe-best-match
run: uv sync --extra test --index-strategy unsafe-best-match
- name: Install API dependencies
working-directory: ./hindsight-api
run: uv sync --frozen --no-install-project --index-strategy unsafe-best-match
run: uv sync --no-install-project --index-strategy unsafe-best-match
- name: Create .env file
run: |
@@ -523,7 +490,7 @@ jobs:
- name: Install API dependencies
working-directory: ./hindsight-api
run: uv sync --frozen --no-install-project --index-strategy unsafe-best-match
run: uv sync --no-install-project --index-strategy unsafe-best-match
- name: Install TypeScript client dependencies
working-directory: ./hindsight-clients/typescript
@@ -611,7 +578,7 @@ jobs:
- name: Install API dependencies
working-directory: ./hindsight-api
run: uv sync --frozen --no-install-project --index-strategy unsafe-best-match
run: uv sync --no-install-project --index-strategy unsafe-best-match
- name: Create .env file
run: |
@@ -648,185 +615,6 @@ jobs:
echo "=== API Server Logs ==="
cat /tmp/api-server.log || echo "No API server log found"
test-go-client:
runs-on: ubuntu-latest
env:
HINDSIGHT_API_LLM_PROVIDER: groq
HINDSIGHT_API_LLM_API_KEY: ${{ secrets.GROQ_API_KEY }}
HINDSIGHT_API_LLM_MODEL: openai/gpt-oss-20b
HINDSIGHT_API_URL: http://localhost:8888
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Prefer CPU-only PyTorch in CI (but keep PyPI for everything else)
UV_INDEX: pytorch=https://download.pytorch.org/whl/cpu
steps:
- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
enable-cache: true
prune-cache: false
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version-file: ".python-version"
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: '1.23'
cache-dependency-path: hindsight-clients/go/go.sum
- name: Build API
working-directory: ./hindsight-api
run: uv build
- name: Install API dependencies
working-directory: ./hindsight-api
run: uv sync --frozen --no-install-project --index-strategy unsafe-best-match
- name: Create .env file
run: |
cat > .env << EOF
HINDSIGHT_API_LLM_PROVIDER=${{ env.HINDSIGHT_API_LLM_PROVIDER }}
HINDSIGHT_API_LLM_API_KEY=${{ env.HINDSIGHT_API_LLM_API_KEY }}
HINDSIGHT_API_LLM_MODEL=${{ env.HINDSIGHT_API_LLM_MODEL }}
EOF
- name: Start API server
run: |
./scripts/dev/start-api.sh > /tmp/api-server.log 2>&1 &
echo "Waiting for API server to be ready..."
for i in {1..60}; do
if curl -sf http://localhost:8888/health > /dev/null 2>&1; then
echo "API server is ready after ${i}s"
break
fi
if [ $i -eq 60 ]; then
echo "API server failed to start after 60s"
cat /tmp/api-server.log
exit 1
fi
sleep 1
done
- name: Build Go client
working-directory: ./hindsight-clients/go
run: go build ./...
- name: Run Go client tests
working-directory: ./hindsight-clients/go
run: go test -v -tags=integration
- name: Show API server logs
if: always()
run: |
echo "=== API Server Logs ==="
cat /tmp/api-server.log || echo "No API server log found"
test-openclaw-integration:
runs-on: ubuntu-latest
env:
HINDSIGHT_API_LLM_PROVIDER: groq
HINDSIGHT_API_LLM_API_KEY: ${{ secrets.GROQ_API_KEY }}
HINDSIGHT_API_LLM_MODEL: openai/gpt-oss-20b
HINDSIGHT_API_URL: http://localhost:8888
HINDSIGHT_EMBED_PACKAGE_PATH: ${{ github.workspace }}/hindsight-embed
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
UV_INDEX: pytorch=https://download.pytorch.org/whl/cpu
steps:
- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
enable-cache: true
prune-cache: false
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version-file: ".python-version"
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '22'
- name: Build API
working-directory: ./hindsight-api
run: uv build
- name: Install API dependencies
working-directory: ./hindsight-api
run: uv sync --frozen --no-install-project --index-strategy unsafe-best-match
- name: Install embed dependencies
working-directory: ./hindsight-embed
run: uv sync --frozen --index-strategy unsafe-best-match
- name: Cache HuggingFace models
uses: actions/cache@v4
with:
path: ~/.cache/huggingface
key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api/pyproject.toml') }}
restore-keys: |
${{ runner.os }}-huggingface-
- name: Pre-download models
working-directory: ./hindsight-api
run: |
uv run python -c "
from sentence_transformers import SentenceTransformer, CrossEncoder
print('Downloading embedding model...')
SentenceTransformer('BAAI/bge-small-en-v1.5')
print('Downloading cross-encoder model...')
CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
print('Models downloaded successfully')
"
- name: Install openclaw integration dependencies
working-directory: ./hindsight-integrations/openclaw
run: npm ci
- name: Create .env file
run: |
cat > .env << EOF
HINDSIGHT_API_LLM_PROVIDER=${{ env.HINDSIGHT_API_LLM_PROVIDER }}
HINDSIGHT_API_LLM_API_KEY=${{ env.HINDSIGHT_API_LLM_API_KEY }}
HINDSIGHT_API_LLM_MODEL=${{ env.HINDSIGHT_API_LLM_MODEL }}
EOF
- name: Start API server
run: |
./scripts/dev/start-api.sh > /tmp/api-server.log 2>&1 &
echo "Waiting for API server to be ready..."
for i in {1..60}; do
if curl -sf http://localhost:8888/health > /dev/null 2>&1; then
echo "API server is ready after ${i}s"
break
fi
if [ $i -eq 60 ]; then
echo "API server failed to start after 60s"
cat /tmp/api-server.log
exit 1
fi
sleep 1
done
- name: Run openclaw integration tests
working-directory: ./hindsight-integrations/openclaw
run: npm run test:integration
- name: Show API server logs
if: always()
run: |
echo "=== API Server Logs ==="
cat /tmp/api-server.log || echo "No API server log found"
test-integration:
runs-on: ubuntu-latest
env:
@@ -857,11 +645,11 @@ jobs:
- name: Install API dependencies
working-directory: ./hindsight-api
run: uv sync --frozen --no-install-project --index-strategy unsafe-best-match
run: uv sync --no-install-project --index-strategy unsafe-best-match
- name: Install integration test dependencies
working-directory: ./hindsight-integration-tests
run: uv sync --frozen
run: uv sync
- name: Cache HuggingFace models
uses: actions/cache@v4
@@ -941,7 +729,7 @@ jobs:
- name: Install dependencies
working-directory: ./hindsight-integrations/litellm
run: uv sync --frozen --extra dev
run: uv sync --extra dev
- name: Run tests
working-directory: ./hindsight-integrations/litellm
@@ -950,9 +738,9 @@ jobs:
test-embed:
runs-on: ubuntu-latest
env:
HINDSIGHT_API_LLM_PROVIDER: groq
HINDSIGHT_API_LLM_API_KEY: ${{ secrets.GROQ_API_KEY }}
HINDSIGHT_API_LLM_MODEL: openai/gpt-oss-20b
HINDSIGHT_EMBED_LLM_PROVIDER: groq
HINDSIGHT_EMBED_LLM_API_KEY: ${{ secrets.GROQ_API_KEY }}
HINDSIGHT_EMBED_LLM_MODEL: openai/gpt-oss-20b
# Prefer CPU-only PyTorch in CI
UV_INDEX: pytorch=https://download.pytorch.org/whl/cpu
@@ -972,7 +760,7 @@ jobs:
- name: Install dependencies
working-directory: ./hindsight-embed
run: uv sync --frozen --index-strategy unsafe-best-match
run: uv sync --index-strategy unsafe-best-match
- name: Cache HuggingFace models
uses: actions/cache@v4
@@ -983,65 +771,13 @@ jobs:
${{ runner.os }}-huggingface-embed-
${{ runner.os }}-huggingface-
- name: Run unit and integration tests
working-directory: ./hindsight-embed
run: uv run pytest tests/ -v
- name: Run smoke test
working-directory: ./hindsight-embed
run: ./test.sh
test-hindsight-all:
runs-on: ubuntu-latest
env:
HINDSIGHT_API_LLM_PROVIDER: groq
HINDSIGHT_API_LLM_API_KEY: ${{ secrets.GROQ_API_KEY }}
HINDSIGHT_API_LLM_MODEL: openai/gpt-oss-20b
# For test_server_integration.py compatibility
HINDSIGHT_LLM_PROVIDER: groq
HINDSIGHT_LLM_API_KEY: ${{ secrets.GROQ_API_KEY }}
HINDSIGHT_LLM_MODEL: openai/gpt-oss-20b
# Prefer CPU-only PyTorch in CI
UV_INDEX: pytorch=https://download.pytorch.org/whl/cpu
steps:
- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
enable-cache: true
prune-cache: false
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version-file: ".python-version"
- name: Build hindsight-all
working-directory: ./hindsight
run: uv build
- name: Install dependencies
working-directory: ./hindsight
run: uv sync --frozen --extra test --index-strategy unsafe-best-match
- name: Cache HuggingFace models
uses: actions/cache@v4
with:
path: ~/.cache/huggingface
key: ${{ runner.os }}-huggingface-all-${{ hashFiles('hindsight/pyproject.toml') }}
restore-keys: |
${{ runner.os }}-huggingface-all-
${{ runner.os }}-huggingface-
- name: Run unit tests
working-directory: ./hindsight
run: uv run pytest tests/ -v
test-doc-examples:
runs-on: ubuntu-latest
needs: test-rust-cli
needs: build-rust-cli
env:
HINDSIGHT_API_LLM_PROVIDER: groq
HINDSIGHT_API_LLM_API_KEY: ${{ secrets.GROQ_API_KEY }}
@@ -1084,11 +820,11 @@ jobs:
working-directory: ./hindsight-api
run: |
uv build
uv sync --frozen --no-install-project --index-strategy unsafe-best-match
uv sync --no-install-project --index-strategy unsafe-best-match
- name: Install Python client dependencies
working-directory: ./hindsight-clients/python
run: uv sync --frozen --extra test --index-strategy unsafe-best-match
run: uv sync --extra test --index-strategy unsafe-best-match
- name: Install TypeScript client
run: |
@@ -1120,11 +856,30 @@ jobs:
sleep 1
done
- name: Run Python doc examples
working-directory: ./hindsight-clients/python
run: |
for f in ../../hindsight-docs/examples/api/*.py; do
echo "Running $f..."
uv run python "$f"
done
- name: Run Node.js doc examples
run: |
for f in hindsight-docs/examples/api/*.mjs; do
echo "Running $f..."
node "$f"
done
- name: Configure CLI
run: hindsight configure --api-url http://localhost:8888
- name: Run all doc examples
run: ./scripts/test-doc-examples.sh
- name: Run CLI doc examples
run: |
for f in hindsight-docs/examples/api/*.sh; do
echo "Running $f..."
bash "$f"
done
- name: Show API server logs
if: always()
@@ -1132,78 +887,6 @@ jobs:
echo "=== API Server Logs ==="
cat /tmp/api-server.log || echo "No API server log found"
test-upgrade:
runs-on: ubuntu-latest
env:
HINDSIGHT_API_LLM_PROVIDER: groq
HINDSIGHT_API_LLM_API_KEY: ${{ secrets.GROQ_API_KEY }}
HINDSIGHT_API_LLM_MODEL: openai/gpt-oss-20b
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
UV_INDEX: pytorch=https://download.pytorch.org/whl/cpu
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Full history needed for git clone of tags
- name: Fetch tags
run: git fetch --tags
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
enable-cache: true
prune-cache: false
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version-file: ".python-version"
- name: Cache HuggingFace models
uses: actions/cache@v4
with:
path: ~/.cache/huggingface
key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api/pyproject.toml') }}
restore-keys: |
${{ runner.os }}-huggingface-
- name: Install hindsight-dev dependencies
working-directory: ./hindsight-dev
run: uv sync --frozen --extra test --index-strategy unsafe-best-match
- name: Install current hindsight-api
working-directory: ./hindsight-api
run: uv sync --frozen --index-strategy unsafe-best-match
- name: Pre-download models
working-directory: ./hindsight-api
run: |
uv run python -c "
from sentence_transformers import SentenceTransformer, CrossEncoder
print('Downloading embedding model...')
SentenceTransformer('BAAI/bge-small-en-v1.5')
print('Downloading cross-encoder model...')
CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
print('Models downloaded successfully')
"
- name: Run upgrade tests
working-directory: ./hindsight-dev
run: uv run pytest upgrade_tests/ -v --tb=short
- name: Show upgrade test logs
if: always()
run: |
echo "=== Upgrade Test Server Logs ==="
for log in /tmp/upgrade-test-*.log; do
if [ -f "$log" ]; then
echo ""
echo "--- $log ---"
tail -500 "$log"
fi
done
verify-generated-files:
runs-on: ubuntu-latest
env:
@@ -1245,9 +928,9 @@ jobs:
- name: Install Python dependencies
run: |
cd hindsight-dev && uv sync --frozen --index-strategy unsafe-best-match
cd ../hindsight-api && uv sync --frozen --index-strategy unsafe-best-match
cd ../hindsight-embed && uv sync --frozen --index-strategy unsafe-best-match
cd hindsight-dev && uv sync --index-strategy unsafe-best-match
cd ../hindsight-api && uv sync --index-strategy unsafe-best-match
cd ../hindsight-embed && uv sync --index-strategy unsafe-best-match
- name: Run generate-openapi
run: ./scripts/generate-openapi.sh
@@ -1274,52 +957,4 @@ jobs:
git diff --stat
exit 1
fi
echo "✓ All generated files are up to date"
check-openapi-compatibility:
runs-on: ubuntu-latest
env:
UV_INDEX: pytorch=https://download.pytorch.org/whl/cpu
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Fetch full git history to access base branch
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
enable-cache: true
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version-file: ".python-version"
- name: Install hindsight-dev dependencies
run: |
cd hindsight-dev && uv sync --frozen --index-strategy unsafe-best-match
- name: Check OpenAPI compatibility with base branch
run: |
# Get the base branch (usually main)
BASE_BRANCH="${{ github.base_ref }}"
if [ -z "$BASE_BRANCH" ]; then
echo "⚠️ Warning: No base branch found (not a PR?). Skipping compatibility check."
exit 0
fi
echo "Checking OpenAPI compatibility against base branch: $BASE_BRANCH"
# Extract the old OpenAPI spec from base branch
git show "origin/$BASE_BRANCH:hindsight-docs/static/openapi.json" > /tmp/old-openapi.json
if [ ! -s /tmp/old-openapi.json ]; then
echo "⚠️ Warning: Could not find OpenAPI spec in base branch. Skipping compatibility check."
exit 0
fi
# Check compatibility using our tool
cd hindsight-dev
uv run check-openapi-compatibility /tmp/old-openapi.json ../hindsight-docs/static/openapi.json
echo "✓ All generated files are up to date"
+1 -11
View File
@@ -27,10 +27,6 @@ docker-compose.override.yml
# NLTK data (will be downloaded automatically)
nltk_data/
# Monitoring stack (Prometheus/Grafana binaries and data)
.monitoring/
.pgbouncer/
# Large benchmark datasets (will be downloaded automatically)
**/longmemeval_s_cleaned.json
@@ -45,15 +41,9 @@ hindsight-docs/static/llms-full.txt
hindsight-dev/benchmarks/locomo/results/
hindsight-dev/benchmarks/longmemeval/results/
hindsight-dev/benchmarks/consolidation/results/
hindsight-dev/benchmarks/perf/results/
benchmarks/results/
hindsight-cli/target
hindsight-clients/rust/target
.claude
whats-next.md
TASK.md
# Changelog is now tracked in hindsight-docs/src/pages/changelog.md
# CHANGELOG.md
blog-post*
CHANGELOG.md
+151 -1
View File
@@ -1,3 +1,153 @@
# AGENTS.md
See [CLAUDE.md](./CLAUDE.md) for project documentation and coding conventions.
This document captures architectural decisions and coding conventions for the Hindsight project.
## Documentation
- **Main documentation**: [hindsight-docs/docs/developer/](./hindsight-docs/docs/developer/)
- **Use case patterns**: [hindsight-docs/docs/cookbook/](./hindsight-docs/docs/cookbook/)
- **API reference**: Auto-generated from OpenAPI spec
## Project Structure
```
hindsight/ # Python package for embedded usage
hindsight-api/ # FastAPI server (core memory engine)
hindsight-cli/ # Rust CLI client
hindsight-embed/ # Embedded CLI (no server needed)
hindsight-control-plane/ # Next.js admin UI
hindsight-docs/ # Docusaurus documentation site
hindsight-dev/ # Development tools and benchmarks
hindsight-integrations/ # Framework integrations (LangChain, etc.)
hindsight-clients/ # Generated API clients (Python, TypeScript, Rust)
```
## Core Concepts
### Memory Banks
- Each bank is an isolated memory store (like a "brain" for one user/agent)
- Banks contain: memory units (facts), entities, documents, entity links
- Banks have a **disposition** (personality traits) and **background** (context)
- Bank isolation is strict - no cross-bank data leakage
### Memory Types
- **World facts**: General knowledge ("The sky is blue")
- **Experience facts**: Personal experiences ("I visited Paris in 2023")
- **Opinion facts**: Beliefs with confidence scores ("Paris is beautiful" - 0.9 confidence)
### Operations
- **Retain**: Store new memories (extracts facts, entities, relationships)
- **Recall**: Retrieve memories (semantic, BM25, graph, temporal search)
- **Reflect**: Deep analysis to form new insights/opinions
## API Design Decisions
### Single Bank Per Request
- All API endpoints (`recall`, `reflect`, `retain`) operate on a single bank
- Multi-bank queries are the **client/agent's responsibility** to orchestrate
- This keeps the API simple and the isolation model clear
### Disposition Traits (3-trait system)
- **Skepticism** (1-5): How skeptical vs trusting when forming opinions
- **Literalism** (1-5): How literally to interpret information
- **Empathy** (1-5): How much to consider emotional context
- These influence the `reflect` operation, not `recall`
- Background info also only affects `reflect` (opinion formation)
## Multi-Bank Architecture Patterns
See [hindsight-docs/docs/cookbook/](./hindsight-docs/docs/cookbook/) for detailed guides:
- **Per-User Memory**: One bank per user, simplest pattern
- **Support Agent + Shared Knowledge**: User bank + shared docs bank, client orchestrates
## Developer Guide
### Running the API Server
```bash
# From project root
./scripts/dev/start-api.sh
# With options
./scripts/dev/start-api.sh --reload --port 8888 --log-level debug
```
### Running Tests
```bash
# API tests
cd hindsight-api
uv run pytest tests/
# Specific test
uv run pytest tests/test_http_api_integration.py -v
```
### Generating OpenAPI Spec
After changing API endpoints, regenerate the OpenAPI spec and docs:
```bash
./scripts/generate-openapi.sh
```
This will:
1. Generate `openapi.json` at project root
2. Copy to `hindsight-docs/openapi.json`
3. Regenerate API reference documentation
### Generating API Clients
After updating the OpenAPI spec, regenerate all clients:
```bash
./scripts/generate-clients.sh
```
This generates:
- **Rust client**: `hindsight-clients/rust/` (via progenitor in build.rs)
- **Python client**: `hindsight-clients/python/` (via openapi-generator Docker)
- **TypeScript client**: `hindsight-clients/typescript/` (via @hey-api/openapi-ts)
Note: The maintained wrapper `hindsight_client.py` and `README.md` are preserved during regeneration.
### Running the Documentation Site
```bash
./scripts/dev/start-docs.sh
```
### Running the Control Plane
```bash
./scripts/dev/start-control-plane.sh
```
## Code Style
### Python (hindsight-api)
- Use `uv` for package management
- Async throughout (asyncpg, async FastAPI endpoints)
- Pydantic models for request/response validation
- No py files at project root - maintain clean directory structure
### TypeScript (control-plane, clients)
- Next.js with App Router for control plane
- Tailwind CSS with shadcn/ui components
### Rust (CLI)
- Async with tokio
- reqwest for HTTP client
- progenitor for API client generation
## Database
- PostgreSQL with pgvector extension
- Schema managed via Alembic migrations in `hindsight-api/alembic/`, db migrations happen during api startup, no manual commands
- Key tables: `banks`, `memory_units`, `documents`, `entities`, `entity_links`
# Branding
## Colors
- Primary: gradient from #0074d9 to #009296
+9 -149
View File
@@ -7,7 +7,8 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
Hindsight is an agent memory system that provides long-term memory for AI agents using biomimetic data structures. Memories are organized as:
- **World facts**: General knowledge ("The sky is blue")
- **Experience facts**: Personal experiences ("I visited Paris in 2023")
- **Mental models**: Consolidated knowledge synthesized from facts ("User prefers functional programming patterns")
- **Opinion facts**: Beliefs with confidence scores ("Paris is beautiful" - 0.9 confidence)
- **Observations**: Complex mental models derived from reflection
## Development Commands
@@ -45,7 +46,6 @@ cd hindsight-control-plane && npm run dev
./scripts/dev/start-docs.sh
```
### Generating Clients/OpenAPI
```bash
# Regenerate OpenAPI spec after API changes (REQUIRED after changing endpoints)
@@ -57,15 +57,8 @@ cd hindsight-control-plane && npm run dev
### Benchmarks
```bash
# Accuracy benchmarks
./scripts/benchmarks/run-longmemeval.sh
./scripts/benchmarks/run-locomo.sh
# Performance benchmarks
./scripts/benchmarks/run-consolidation.sh
./scripts/benchmarks/run-retain-perf.sh --document <path> # Requires API server running
# Results viewer
./scripts/benchmarks/start-visualizer.sh # View results at localhost:8001
```
@@ -108,59 +101,13 @@ cd hindsight-control-plane && npm run dev
Main operations:
- **Retain**: Store memories, extracts facts/entities/relationships
- **Recall**: Retrieve memories via 4 parallel strategies (semantic, BM25, graph, temporal) + reranking
- **Reflect**: Disposition-aware reasoning using memories and mental models.
- **Reflect**: Deep analysis forming new opinions/observations (disposition-aware)
### Database
PostgreSQL with pgvector. Schema managed via Alembic migrations in `hindsight-api/hindsight_api/alembic/`. Migrations run automatically on API startup.
Key tables: `banks`, `memory_units`, `documents`, `entities`, `entity_links`
### Adding Database Migrations
1. **Create a new migration file** in `hindsight-api/hindsight_api/alembic/versions/`:
- File name format: `<revision_id>_<description>.py` (e.g., `f1a2b3c4d5e6_add_new_index.py`)
- Use a unique hex revision ID (12 chars)
- Set `down_revision` to the previous migration's revision ID
2. **Migration template**:
```python
"""Description of the migration
Revision ID: f1a2b3c4d5e6
Revises: <previous_revision_id>
Create Date: YYYY-MM-DD
"""
from collections.abc import Sequence
from alembic import context, op
revision: str = "f1a2b3c4d5e6"
down_revision: str | Sequence[str] | None = "<previous_revision_id>"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _get_schema_prefix() -> str:
"""Get schema prefix for table names (required for multi-tenant support)."""
schema = context.config.get_main_option("target_schema")
return f'"{schema}".' if schema else ""
def upgrade() -> None:
schema = _get_schema_prefix()
op.execute(f"CREATE INDEX ... ON {schema}table_name(...)")
def downgrade() -> None:
schema = _get_schema_prefix()
op.execute(f"DROP INDEX IF EXISTS {schema}index_name")
```
3. **Run migrations locally**:
```bash
# Set database URL and run migrations
uv run hindsight-admin run-db-migration
# Run on a specific tenant schema
uv run hindsight-admin run-db-migration --schema tenant_xyz
```
## Key Conventions
### Code Quality
@@ -181,63 +128,12 @@ This runs the same checks as the pre-commit hook (Ruff for Python, ESLint/Pretti
- Multi-bank queries are client responsibility to orchestrate
- Disposition traits only affect reflect, not recall
### Control Plane API Routes
When adding or modifying parameters in the dataplane API (hindsight-api), you must also update the control plane routes that proxy to it:
1. **API Routes** (`hindsight-control-plane/src/app/api/`):
- `recall/route.ts` - proxies to `/v1/default/banks/{bank_id}/memories/recall`
- `reflect/route.ts` - proxies to `/v1/default/banks/{bank_id}/reflect`
- `memories/retain/route.ts` - proxies to `/v1/default/banks/{bank_id}/memories/retain`
- Other routes follow the same pattern
2. **Client types** (`hindsight-control-plane/src/lib/api.ts`):
- Update the TypeScript type definitions for `recall()`, `reflect()`, `retain()` etc.
3. **Checklist when adding new API parameters**:
- Add parameter extraction in the route handler (destructure from `body`)
- Pass the parameter to the SDK call
- Update the client type definition in `lib/api.ts`
- Update any UI components that need to use the new parameter
### Python Style
- Python 3.11+, type hints required
- Async throughout (asyncpg, async FastAPI)
- Pydantic models for request/response
- Ruff for linting (line-length 120)
- No Python files at project root - maintain clean directory structure
- **Never use multi-item tuple return values** - prefer dataclass or Pydantic model for structured returns
### Type Safety with Pydantic Models
**NEVER use raw `dict` types for structured data.** Always use Pydantic models:
- Use Pydantic `BaseModel` for all data structures passed between functions
- Add `@field_validator` for type coercion (e.g., ensuring datetimes are timezone-aware)
- Avoid `dict.get()` patterns - use typed model attributes instead
- Parse external data (JSON, API responses) into Pydantic models at the boundary
- This catches type errors at parse time, not deep in business logic
```python
# BAD - error-prone dict access
def process(data: dict) -> str:
return data.get("name", "") # No validation, silent failures
# GOOD - typed and validated
class UserData(BaseModel):
name: str
created_at: datetime
@field_validator("created_at", mode="before")
@classmethod
def ensure_tz_aware(cls, v):
if isinstance(v, str):
v = datetime.fromisoformat(v.replace("Z", "+00:00"))
if v.tzinfo is None:
return v.replace(tzinfo=timezone.utc)
return v
def process(data: UserData) -> str:
return data.name # Type-safe, validated at construction
```
### TypeScript Style
- Next.js App Router for control plane
@@ -245,61 +141,26 @@ def process(data: UserData) -> str:
### Adding New API Configuration Flags
Configuration follows a hierarchical system: **Global (env vars) → Tenant (via extension) → Bank (database)**.
Fields must be categorized as either **hierarchical** (can be overridden per-tenant/bank) or **static** (server-level only).
#### Adding a New Configuration Field
When adding a new environment variable configuration:
1. **config.py** (`hindsight-api/hindsight_api/config.py`):
- Add `ENV_*` constant for the environment variable name (e.g., `ENV_MY_SETTING = "HINDSIGHT_API_MY_SETTING"`)
- Add `ENV_*` constant for the environment variable name
- Add `DEFAULT_*` constant for the default value
- Add field to `HindsightConfig` dataclass with type annotation
- **Mark as hierarchical or static** by adding to `_HIERARCHICAL_FIELDS` set (hierarchical) or leaving it out (static)
- Add field to `HindsightConfig` dataclass
- Add initialization in `from_env()` method
```python
# Hierarchical field (can be overridden per-bank)
_HIERARCHICAL_FIELDS = {
...,
"my_setting", # Add here for hierarchical
}
# Static field - just don't add to _HIERARCHICAL_FIELDS
```
2. **main.py** (`hindsight-api/hindsight_api/main.py`):
- Add field to the manual `HindsightConfig()` constructor call (search for "CLI override")
3. **Use hierarchical config in MemoryEngine**:
```python
# Config is resolved automatically per bank via ConfigResolver
config_dict = await self._config_resolver.get_bank_config(bank_id, context)
value = config_dict["my_setting"]
```
4. **Use static config** (non-hierarchical):
3. **Use the config** in code:
```python
from ...config import get_config
config = get_config()
value = config.my_static_field
value = config.your_new_field
```
5. **Documentation** (`hindsight-docs/docs/developer/configuration.md`):
4. **Documentation** (`hindsight-docs/docs/developer/configuration.md`):
- Add to appropriate section table with Variable, Description, Default
- Mark if it's hierarchical (can be overridden per-bank)
#### Hierarchical vs Static Guidelines
**Hierarchical** (per-bank overridable):
- LLM settings (provider, model, API key, base URL)
- Operation-specific settings (retain mode, chunk size, etc.)
- Feature flags that vary by customer/bank
**Static** (server-level only):
- Infrastructure settings (database URL, port, host)
- Global limits (max concurrent operations)
- System-wide feature flags
## Environment Setup
@@ -323,4 +184,3 @@ Optional (uses local models by default):
- `HINDSIGHT_API_EMBEDDINGS_PROVIDER`: local (default) or tei
- `HINDSIGHT_API_RERANKER_PROVIDER`: local (default) or tei
- `HINDSIGHT_API_DATABASE_URL`: External PostgreSQL (uses embedded pg0 by default)
- `HINDSIGHT_API_ENABLE_BANK_CONFIG_API`: Enable per-bank config API (default: false, disabled for security)
-28
View File
@@ -93,34 +93,6 @@ uv run ty check hindsight_api # Type check
3. Run tests to ensure nothing breaks
4. Submit a PR with a clear description of changes
## Release Process
The project uses `scripts/release.sh` for creating releases. This script automates the entire release workflow:
1. Bumps version in all components (API, clients, CLI, control plane, Helm)
2. **Regenerates OpenAPI spec and client SDKs** (Python, TypeScript, Rust)
3. Updates documentation versioning
4. Creates a commit and git tag
5. Pushes to GitHub (triggers CI/CD to publish packages)
### Usage
```bash
./scripts/release.sh <version>
```
**Example:**
```bash
./scripts/release.sh 0.5.0
```
### Important for Developers
- During development, version bumps in `__init__.py` do NOT require client regeneration
- Clients are only regenerated during releases
- Do not manually run `./scripts/generate-clients.sh` unless testing generation changes
- Client version comments will reflect the API version from the latest release
## Reporting Issues
Open an issue on GitHub with:
+60 -105
View File
@@ -1,11 +1,11 @@
<div align="center">
![Hindsight Banner](./hindsight-docs/static/img/hindsight-github-banner.png)
![Hindsight Banner](./hindsight-docs/static/img/banner.svg)
[Documentation](https://hindsight.vectorize.io) • [Paper](https://arxiv.org/abs/2512.12818) • [Cookbook](https://hindsight.vectorize.io/cookbook) • [Hindsight Cloud](https://ui.hindsight.vectorize.io/signup)
[Documentation](https://hindsight.vectorize.io) • [Paper](https://arxiv.org/abs/2512.12818) • [Cookbook](https://hindsight.vectorize.io/cookbook) • [Hindsight Cloud](https://vectorize.io/hindsight/cloud)
[![CI](https://github.com/vectorize-io/hindsight/actions/workflows/release.yml/badge.svg)](https://github.com/vectorize-io/hindsight/actions/workflows/release.yml)
[![Slack Community](https://img.shields.io/badge/Slack-Join%20Community-4A154B?logo=slack)](https://join.slack.com/t/hindsight-space/shared_invite/zt-3nhbm4w29-LeSJ5Ixi6j8PdiYOCPlOgg)
[![Slack Community](https://img.shields.io/badge/Slack-Join%20Community-4A154B?logo=slack)](https://join.slack.com/t/hindsight-space/shared_invite/zt-3klo21kua-VUCC_zHP5rIcXFB1_5yw6A)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
![PyPI - Downloads](https://img.shields.io/pypi/dm/hindsight-api?label=PyPI)
![NPM Downloads](https://img.shields.io/npm/dm/%40vectorize-io%2Fhindsight-client?logoColor=orange&label=NPM&color=blue&link=https%3A%2F%2Fwww.npmjs.com%2Fpackage%2F%40vectorize-io%2Fhindsight-client)
@@ -17,76 +17,76 @@
## What is Hindsight?
Hindsight™ is an agent memory system built to create smarter agents that learn over time. Most agent memory systems focus on recalling conversation history. Hindsight is focused on making agents that learn, not just remember.
Hindsight™ is an agent memory system built to create smarter agents that learn over time. It eliminates the shortcomings of alternative techniques such as RAG and knowledge graph and delivers state-of-the-art performance on long term memory tasks.
Hindsight addresses common challenges that have frustrated AI engineers building agents to automate tasks and assist users with conversational interfaces. Many of these challenges stem directly from a lack of memory.
<video src="https://github.com/user-attachments/assets/923b798d-3581-4897-bb62-9cfa5a931682" controls></video>
- **Inconsistency:** Agents complete tasks successfully one time, then fail when asked to complete the same task again. Memory gives the agent a mechanism to remember what worked and what didn't and to use that information to reduce errors and improve consistency.
- **Hallucinations:** Long term memory can be seeded with external knowledge to ground agent behavior in reliable sources to augment training data.
- **Cognitive Overload:** As workflows get complex, retrievals, tool calls, user messages and agent responses can grow to fill the context window leading to context rot. Short term memory optimization allows agents to reduce tokens and focus context by removing irrelevant details.
It eliminates the shortcomings of alternative techniques such as RAG and knowledge graph and delivers state-of-the-art performance on long term memory tasks.
## How is Hindsight Different From Other Memory Systems?
![Overview](./hindsight-docs/static/img/hindsight-overview.webp)
Most agent memory implementation rely on basic vector search or sometimes use a knowledge graph. Hindsight uses biomimetic data structures to organize agent memories in a way that is more like how human memory works:
- **World:** Facts about the world ("The stove gets hot")
- **Experiences:** Agent's own experiences ("I touched the stove and it really hurt")
- **Opinion:** Beliefs with confidence scores ("I shouldn't touch the stove again" - .99 confidence)
- **Observation:** Complex mental models derived by reflecting on facts and experiences ("Curling irons, ovens, and fire are also hot. I shouldn't touch those either.")
Memories in Hindsight are stored in banks (i.e. memory banks). When memories are added to Hindsight, they are pushed into either the world facts or experiences memory pathway. They are then represented as a combination of entities, relationships, and time series with sparse/dense vector representations to aid in later recall.
Hindsight provides three simple methods to interact with the system:
- **Retain:** Provide information to Hindsight that you want it to remember
- **Recall:** Retrieve memories from Hindsight
- **Reflect:** Reflect on memories and experiences to generate new observations and insights from existing memories.
### Agent Memory That Learns
A key goal of Hindsight is to build agent memory that enables agents to learn and improve over time. This is the role of the `reflect` operation which provides the agent to form broader opinions and observations over time.
For example, imagine a product support agent that is helping a user troubleshoot a problem. It uses a `search-documentation` tool it found on an MCP server. Later in the conversation, the agent discovers that the documentation returned from the tool wasn't for the product the user was asking about. The agent now has an experience in its memory bank. And just like humans, we want that agent to learn from its experience.
As the agent gains more experiences, `reflect` allows the agent to form observations about what worked, what didn't, and what to do differently the next time it encounters a similar task.
---
## Memory Performance & Accuracy
Hindsight is the most accurate agent memory system ever tested according to benchmark performance. It has achieved state-of-the-art performance on the LongMemEval benchmark, widely used to assess memory system performance across a variety of conversational AI scenarios. The current reported performance of Hindsight and other agent memory solutions as of January 2026 is shown here:
Hindsight has achieved state-of-the-art performance on the LongMemEval benchmark, widely used to assess memory system performance across a variety of conversational
AI scenarios. The current reported performance of Hindsight and other agent memory solutions as of December 2025 is shown here:
![Overview](./hindsight-docs/static/img/hindsight-bench.jpg)
The benchmark performance data for Hindsight has been independently reproduced by research collaborators at the Virginia Tech [Sanghani Center for Artificial Intelligence and Data Analytics](https://sanghani.cs.vt.edu/) and The Washington Post. Other scores are self-reported by software vendors.
The benchmark performance data for Hindsight and GPT-4o (full context) have been reproduced by research collaborators at the Virginia Tech [Sanghani Center for Artificial Intelligence and Data Analytics](https://sanghani.cs.vt.edu/) and The Washington Post. Other scores are self-reported by software vendors.
Hindsight is being used in production at Fortune 500 enterprises and by a growing number of AI startups.
## Adding Hindsight to Your AI Agents
The easiest way use Hindsight with an existing agent is with the LLM Wrapper. You can add memory to your agent with 2 lines of code. That will swap your current LLM client out with the Hindsight wrapper. After that, memories will be stored and retrieved automatically as you make LLM calls.
If you need more control over how and when your agent stores and recalls memories, there's also a simple API you can integrate with using the SDKs or directly via HTTP.
![Hindsight Banner](./hindsight-docs/static/img/migration-code.png)
---
> 🤖 **Using a coding agent?** Install the Hindsight documentation skill for instant access to docs while you code:
> ```bash
> npx skills add https://github.com/vectorize-io/hindsight --skill hindsight-docs
> ```
> Works with Claude Code, Cursor, and other AI coding assistants.
---
A thorough examination of the techniques implemented in Hindsight and detailed breakdowns of benchmark performance are [available on arXiv](https://arxiv.org/abs/2512.12818). This research is currently being prepared for conference submission and the wider peer review process.
The benchmark results from this research can be inspected in our [visual benchmark explorer](https://hindsight-benchmarks.vercel.app). As additional improvements are made to Hindsight, new benchmark data will be available for review using this same tool.
## Quick Start
### Docker (recommended)
```bash
export OPENAI_API_KEY=sk-xxx
export OPENAI_API_KEY=your-key
docker run --rm -it --pull always -p 8888:8888 -p 9999:9999 \
-e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \
-e HINDSIGHT_API_LLM_MODEL=o3-mini \
-v $HOME/.hindsight-docker:/home/hindsight/.pg0 \
ghcr.io/vectorize-io/hindsight:latest
```
>API: http://localhost:8888
>UI: http://localhost:9999
You can modify the LLM provider by setting `HINDSIGHT_API_LLM_PROVIDER`. Valid options are `openai`, `anthropic`, `gemini`, `groq`, `ollama`, and `lmstudio`. The documentation provides more details on [supported models](https://hindsight.vectorize.io/developer/models).
API: http://localhost:8888
UI: http://localhost:9999
### Docker (external PostgreSQL)
```bash
export OPENAI_API_KEY=sk-xxx
export HINDSIGHT_DB_PASSWORD=choose-a-password
cd docker/docker-compose
docker compose up
```
>API: http://localhost:8888
>UI: http://localhost:9999
### Client
Install client:
```bash
pip install hindsight-client -U
@@ -94,7 +94,7 @@ pip install hindsight-client -U
npm install @vectorize-io/hindsight-client
```
#### Python
Python example:
```python
from hindsight_client import Hindsight
@@ -111,29 +111,7 @@ client.recall(bank_id="my-bank", query="What does Alice do?")
client.reflect(bank_id="my-bank", query="Tell me about Alice")
```
#### Node.js / TypeScript
```bash
npm install @vectorize-io/hindsight-client
```
```javascript
const { HindsightClient } = require('@vectorize-io/hindsight-client');
const main = async () => {
const client = new HindsightClient({ baseUrl: 'http://localhost:8888' });
await client.retain('my-bank', 'Alice loves hiking in Yosemite');
const results = await client.recall('my-bank', 'What does Alice like?');
console.log(results);
}
main();
```
### Python Embedded (no server required)
### Python (embedded, no Docker)
```bash
pip install hindsight-all -U
@@ -153,48 +131,25 @@ with HindsightServer(
results = client.recall(bank_id="my-bank", query="Where does Alice work?")
```
### Node.js / TypeScript
---
```bash
npm install @vectorize-io/hindsight-client
```
## Use Cases
```javascript
const { HindsightClient } = require('@vectorize-io/hindsight-client');
const client = new HindsightClient({ baseUrl: 'http://localhost:8888' });
Hindsight is built to support conversational AI agents as well as agents that are intended to perform tasks autonomously. The ideal use case for Hindsight are agents that require a blend of these features such as AI employees that need to handle open-ended tasks, change behavior based on user feedback, and learn to perform complex tasks to automate work at a level that approximates a human work. Hindsight can be used with simple AI workflows like those built with n8n and other similar tools, but may be overkill for such applications.
### Per-User Memories and Chat History
One of the simpler use cases you can use Hindsight for is to personalize AI chatbots and other conversational agents by storing and recalling memories associated with individual users.
The requirements for this use case usually look something like this:
![Per-User Memories](./hindsight-docs/static/img/per-user-memory-requirements.png)
<video src="https://github.com/user-attachments/assets/4805e8e1-e7d1-47c6-a4f8-2344a5ec8906" controls></video>
Satisfying these requirements in Hindsight is straightforward. When new user inputs and tool calls are ingested into Hindsight using the retain operation, custom metadata can be used to enrich the new memories. Metadata provides a convenient way to isolate memories that need to be restricted to a given user. Once these are fed into the retain operation, any raw memories and mental models that get created can be filtered when retrieving relevant memories.
![Per-User Memories](./hindsight-docs/static/img/per-user-memory-howto.png)
await client.retain('my-bank', 'Alice loves hiking in Yosemite');
await client.recall('my-bank', 'What does Alice like?');
```
---
## Architecture & Operations
![Overview](./hindsight-docs/static/img/hindsight-overview.webp)
Most agent memory implementation rely on basic vector search or sometimes use a knowledge graph. Hindsight uses biomimetic data structures to organize agent memories in a way that is more like how human memory works:
- **World:** Facts about the world ("The stove gets hot")
- **Experiences:** Agent's own experiences ("I touched the stove and it really hurt")
- **Mental Models:** Learned understanding of the agent's world formed by reflecting on raw memories and experiences.
Memories in Hindsight are stored in banks (i.e. memory banks). When memories are added to Hindsight, they are pushed into either the world facts or experiences memory pathway. They are then represented as a combination of entities, relationships, and time series with sparse/dense vector representations to aid in later recall.
Hindsight provides three simple methods to interact with the system:
- **Retain:** Provide information to Hindsight that you want it to remember
- **Recall:** Retrieve memories from Hindsight
- **Reflect:** Reflect on memories and experiences to generate new observations and insights from existing memories.
### Retain
The `retain` operation is used to push new memories into Hindsight. It tells Hindsight to _retain_ the information you pass in as an input.
@@ -253,7 +208,7 @@ The final output is trimmed as needed to fit within the token limit.
### Reflect
The reflect operation is used to perform a more thorough analysis of existing memories. This allows the agent to form new connections between memories and build a more thorough understanding of its world.
The reflect operation is used to perform a more thorough analysis of existing memories. This allows the agent to form new connections between memories which are then persisted as opinions and/or observations. When building agents, the reflect operation is a key capability to enable the agent to learn from its experiences.
For example, the `reflect` operation can be used to support use cases such as:
@@ -287,7 +242,7 @@ client.reflect(bank_id="my-bank", query="What should I know about Alice?")
- [CLI](https://hindsight.vectorize.io/sdks/cli)
**Community:**
- [Slack](https://join.slack.com/t/hindsight-space/shared_invite/zt-3nhbm4w29-LeSJ5Ixi6j8PdiYOCPlOgg)
- [Slack](https://join.slack.com/t/hindsight-space/shared_invite/zt-3klo21kua-VUCC_zHP5rIcXFB1_5yw6A)
- [GitHub Issues](https://github.com/vectorize-io/hindsight/issues)
---
@@ -1,54 +0,0 @@
# Docker Compose file for Hindsight with PostgreSQL and pgvector
#
# Make sure to set the required environment variables before running:
# - HINDSIGHT_DB_PASSWORD: Password for the PostgreSQL user
# - Configure LLM provider variables as needed (see below in the hindsight service)
#
# Usage:
# docker compose up -d
#
# Optional environment variables with defaults:
# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
# - HINDSIGHT_DB_VERSION: PostgreSQL version (default: 18)
services:
db:
# Use a PostgreSQL-Image with pgvector extension pre-installed
# see https://hub.docker.com/r/pgvector/pgvector
image: pgvector/pgvector:pg${HINDSIGHT_DB_VERSION:-18}
container_name: hindsight-db
restart: always
# Expose PostgreSQL port
# ports:
# - "5432:5432"
environment:
POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:?Please set the HINDSIGHT_DB_PASSWORD env variable}
POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
volumes:
- pg_data:/var/lib/postgresql/${HINDSIGHT_DB_VERSION:-18}/docker
networks:
- hindsight-net
hindsight:
image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
container_name: hindsight-app
ports:
- "8888:8888"
- "9999:9999"
environment:
- HINDSIGHT_API_LLM_API_KEY=${OPENAI_API_KEY?Please set the OPENAI_API_KEY env variable}
- HINDSIGHT_API_DATABASE_URL=postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:?Please set the HINDSIGHT_DB_PASSWORD env variable}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
depends_on:
- db
networks:
- hindsight-net
networks:
hindsight-net:
driver: bridge
volumes:
pg_data:
-96
View File
@@ -1,96 +0,0 @@
# Nginx Reverse Proxy with Custom Base Path
Deploy Hindsight API under `/hindsight` (or any custom path) using Nginx reverse proxy.
## Quick Start (Published Image - API Only)
```bash
docker-compose up
```
- **API:** http://localhost:8080/hindsight/docs
- **Control Plane:** http://localhost:9999 (direct access, not proxied)
## Full Stack with Custom Base Path (Requires Build)
**Important:** You cannot rebuild from the published image with build args. You must build from source.
### Build from Source with Custom Base Path
1. **Clone the repository** (if you haven't):
```bash
git clone https://github.com/vectorize-io/hindsight.git
cd hindsight
```
2. **Build with base path**:
```bash
docker build \
--build-arg NEXT_PUBLIC_BASE_PATH=/hindsight \
-f docker/standalone/Dockerfile \
-t hindsight:custom \
.
```
3. **Update docker-compose.yml** to use your built image:
```yaml
services:
hindsight:
image: hindsight:custom # ← Change this
environment:
HINDSIGHT_API_BASE_PATH: /hindsight
NEXT_PUBLIC_BASE_PATH: /hindsight
```
4. **Update nginx.conf** to handle Control Plane routes (see below)
5. **Run**:
```bash
docker-compose up
```
### Required nginx.conf for Full Stack
Replace the current `nginx.conf` with this to proxy both API and Control Plane:
```nginx
events { worker_connections 1024; }
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
upstream hindsight_api { server hindsight:8888; }
upstream hindsight_cp { server hindsight:9999; }
server {
listen 80;
# API
location ~ ^/hindsight/(docs|openapi\.json|health|metrics|v1|mcp) {
proxy_pass http://hindsight_api;
proxy_set_header Host $http_host;
}
# Control Plane static files
location ~ ^/hindsight/_next/ {
proxy_pass http://hindsight_cp;
proxy_set_header Host $http_host;
}
# Control Plane UI
location /hindsight {
proxy_pass http://hindsight_cp;
proxy_set_header Host $http_host;
}
location = / { return 301 /hindsight; }
}
}
```
### Why Build is Required
Next.js requires `basePath` at **build time**. The published image was built without a custom base path, so you must rebuild from source with the `NEXT_PUBLIC_BASE_PATH` build arg to deploy the Control Plane under a subpath.
The API works without rebuild because `HINDSIGHT_API_BASE_PATH` is a runtime environment variable.
@@ -1,88 +0,0 @@
# Hindsight API deployment with Nginx reverse proxy (API-only)
#
# This example deploys Hindsight API under the path /hindsight with:
# - Hindsight standalone image (API + Control Plane + embedded pg0)
# - Nginx reverse proxy (API only)
#
# Quick Start:
# docker-compose -f docker/docker-compose/nginx/docker-compose.yml up
#
# Access:
# API (via nginx): http://localhost:8080/hindsight/docs
# Control Plane (direct): http://localhost:9999
#
# For full stack deployment (API + Control Plane both under /hindsight):
# See README.md in this directory for instructions on building with basePath.
#
# Note: This configuration uses the published image (no build required).
# Control Plane is served directly because Next.js basePath requires
# build-time configuration. See README.md for the full stack option.
services:
# Hindsight (API + Control Plane + embedded pg0)
hindsight:
image: ghcr.io/vectorize-io/hindsight:latest
ports:
- "9999:9999" # Control Plane (direct access, not proxied)
environment:
# API base path for reverse proxy
HINDSIGHT_API_BASE_PATH: /hindsight
# LLM configuration
# Using mock provider for testing (no API key needed)
# For production, set OPENAI_API_KEY or ANTHROPIC_API_KEY and use a real provider
HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-mock}
HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-not-needed-for-mock}
HINDSIGHT_API_LLM_MODEL: ${HINDSIGHT_API_LLM_MODEL:-mock-model}
# Production examples (uncomment and set appropriate API key):
# HINDSIGHT_API_LLM_PROVIDER: openai
# HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY}
# HINDSIGHT_API_LLM_MODEL: gpt-4o-mini
# HINDSIGHT_API_LLM_PROVIDER: anthropic
# HINDSIGHT_API_LLM_API_KEY: ${ANTHROPIC_API_KEY}
# HINDSIGHT_API_LLM_MODEL: claude-sonnet-4-20250514
# Server config
HINDSIGHT_API_HOST: 0.0.0.0
HINDSIGHT_API_PORT: 8888
HINDSIGHT_API_LOG_LEVEL: info
# Control Plane config
HINDSIGHT_CP_DATAPLANE_API_URL: http://localhost:8888
volumes:
# Persist embedded pg0 database
- hindsight_data:/app/data
# Note: Ports not exposed - access via Nginx at localhost:8080/hindsight/
# To debug directly, uncomment these ports:
# ports:
# - "8888:8888" # API
# - "9999:9999" # Control Plane
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8888/hindsight/health"]
interval: 10s
timeout: 5s
retries: 3
start_period: 30s
networks:
- hindsight
# Nginx reverse proxy
nginx:
image: nginx:alpine
ports:
- "8080:80"
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf:ro
depends_on:
hindsight:
condition: service_healthy
networks:
- hindsight
volumes:
hindsight_data:
networks:
hindsight:
-40
View File
@@ -1,40 +0,0 @@
# Nginx configuration for API-only reverse proxy
# Control Plane accessed directly (not through nginx)
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Logging
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
# Upstream - Hindsight API
upstream hindsight_api {
server hindsight:8888;
}
server {
listen 80;
server_name _;
# API endpoints - forward with /hindsight prefix
location /hindsight/ {
proxy_pass http://hindsight_api;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
# Redirect root to API docs
location = / {
return 301 /hindsight/docs;
}
}
}
@@ -1,32 +0,0 @@
# PostgreSQL with pgvector and pg_textsearch extensions
# Note: pg_textsearch requires PostgreSQL 17+
FROM postgres:17
# Install build dependencies
RUN apt-get update && apt-get install -y \
build-essential \
git \
postgresql-server-dev-17 \
libpq-dev \
&& rm -rf /var/lib/apt/lists/*
# Install pgvector
RUN cd /tmp && \
git clone --branch v0.8.0 https://github.com/pgvector/pgvector.git && \
cd pgvector && \
make && \
make install
# Install pg_textsearch
RUN cd /tmp && \
git clone https://github.com/timescale/pg_textsearch.git && \
cd pg_textsearch && \
make && \
make install
# Clean up source files and build dependencies
RUN rm -rf /tmp/pgvector /tmp/pg_textsearch && \
apt-get purge -y --auto-remove build-essential git postgresql-server-dev-17
# Ensure extensions are preloaded
RUN echo "shared_preload_libraries = 'pg_textsearch'" >> /usr/share/postgresql/postgresql.conf.sample
@@ -1,91 +0,0 @@
name: hindsight
# Docker Compose file for Hindsight with PostgreSQL and Timescale pg_textsearch
# docker compose -f docker/docker-compose/pg_textsearch/docker-compose.yaml down && sleep 2 && docker compose -f docker/docker-compose/pg_textsearch/docker-compose.yaml up -d
# Make sure to set the required environment variables before running:
# - HINDSIGHT_DB_PASSWORD: Password for the PostgreSQL user
# - Configure LLM provider variables as needed (see below in the hindsight service)
#
# Usage:
# docker compose up -d
#
# Optional environment variables with defaults:
# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
services:
db:
# Use custom PostgreSQL image with pgvector and pg_textsearch extensions
build:
context: .
dockerfile: Dockerfile
container_name: hindsight-db
restart: always
# Expose PostgreSQL port
ports:
- "5437:5432"
environment:
POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:-hindsight_password}
POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
volumes:
- pg_data:/var/lib/postgresql/data
networks:
- hindsight-net
pg-textsearch-init:
build:
context: .
dockerfile: Dockerfile
depends_on:
- db
environment:
- PGPASSWORD=${HINDSIGHT_DB_PASSWORD:-hindsight_password}
command: >
bash -c "
echo 'Waiting for PostgreSQL to be ready...';
until pg_isready -h hindsight-db -p 5432 -U hindsight_user; do
echo 'PostgreSQL is unavailable - sleeping';
sleep 2;
done;
echo 'PostgreSQL is ready - creating hindsight_db database';
psql -h hindsight-db -p 5432 -U hindsight_user -c 'CREATE DATABASE hindsight_db;' 2>/dev/null || echo 'Database already exists';
echo 'Creating extensions in hindsight_db database';
psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vector CASCADE;';
psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS pg_textsearch CASCADE;';
echo 'Database and extensions created successfully';
"
restart: "no"
networks:
- hindsight-net
hindsight:
image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
container_name: hindsight-app
ports:
- "8888:8888"
- "9999:9999"
environment:
# LLM Configuration
HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
# Database Configuration
HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
# Vector and Text Search Extensions
HINDSIGHT_API_VECTOR_EXTENSION: pgvector
HINDSIGHT_API_TEXT_SEARCH_EXTENSION: pg_textsearch
depends_on:
- db
networks:
- hindsight-net
networks:
hindsight-net:
driver: bridge
volumes:
pg_data:
@@ -1,83 +0,0 @@
# Docker Compose file for Hindsight with S3 file storage (SeaweedFS)
#
# SeaweedFS (Apache 2.0) provides an S3-compatible object storage backend
# for storing uploaded files instead of PostgreSQL BYTEA storage.
#
# Make sure to set the required environment variables before running:
# - HINDSIGHT_DB_PASSWORD: Password for the PostgreSQL user
# - Configure LLM provider variables as needed (see below in the hindsight service)
#
# Usage:
# docker compose up -d
#
# Optional environment variables with defaults:
# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
# - HINDSIGHT_DB_VERSION: PostgreSQL version (default: 18)
# - SEAWEEDFS_S3_ACCESS_KEY: S3 access key (default: hindsight_s3_key)
# - SEAWEEDFS_S3_SECRET_KEY: S3 secret key (default: hindsight_s3_secret)
services:
db:
image: pgvector/pgvector:pg${HINDSIGHT_DB_VERSION:-18}
container_name: hindsight-db
restart: always
environment:
POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:?Please set the HINDSIGHT_DB_PASSWORD env variable}
POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
volumes:
- pg_data:/var/lib/postgresql/${HINDSIGHT_DB_VERSION:-18}/docker
networks:
- hindsight-net
seaweedfs:
image: chrislusf/seaweedfs:latest
container_name: hindsight-seaweedfs
restart: always
# Single-node mode: master + volume + filer + S3 gateway all in one process
command: >
server
-s3
-s3.port=8333
-s3.config=/etc/seaweedfs/s3.json
-ip.bind=0.0.0.0
volumes:
- seaweedfs_data:/data
- ./s3.json:/etc/seaweedfs/s3.json:ro
# Expose S3 API port (uncomment to access from host)
# ports:
# - "8333:8333"
networks:
- hindsight-net
hindsight:
image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
container_name: hindsight-app
ports:
- "8888:8888"
- "9999:9999"
environment:
- HINDSIGHT_API_LLM_API_KEY=${OPENAI_API_KEY?Please set the OPENAI_API_KEY env variable}
- HINDSIGHT_API_DATABASE_URL=postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:?Please set the HINDSIGHT_DB_PASSWORD env variable}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
# S3 file storage configuration (SeaweedFS)
- HINDSIGHT_API_FILE_STORAGE_TYPE=s3
- HINDSIGHT_API_FILE_STORAGE_S3_BUCKET=hindsight
- HINDSIGHT_API_FILE_STORAGE_S3_ENDPOINT=http://seaweedfs:8333
- HINDSIGHT_API_FILE_STORAGE_S3_REGION=us-east-1
- HINDSIGHT_API_FILE_STORAGE_S3_ACCESS_KEY_ID=${SEAWEEDFS_S3_ACCESS_KEY:-hindsight_s3_key}
- HINDSIGHT_API_FILE_STORAGE_S3_SECRET_ACCESS_KEY=${SEAWEEDFS_S3_SECRET_KEY:-hindsight_s3_secret}
depends_on:
- db
- seaweedfs
networks:
- hindsight-net
networks:
hindsight-net:
driver: bridge
volumes:
pg_data:
seaweedfs_data:
@@ -1,19 +0,0 @@
{
"identities": [
{
"name": "hindsight",
"credentials": [
{
"accessKey": "hindsight_s3_key",
"secretKey": "hindsight_s3_secret"
}
],
"actions": [
"Admin",
"Read",
"Write",
"List"
]
}
]
}
@@ -1,16 +0,0 @@
# Git
.git
.gitignore
.gitattributes
# Docker
docker-compose.yaml
.dockerignore
# Documentation
README.md
*.md
# Environment
.env
.env.example
@@ -1,25 +0,0 @@
# PostgreSQL Configuration
HINDSIGHT_DB_USER=hindsight_user
HINDSIGHT_DB_PASSWORD=change-me-to-secure-password
HINDSIGHT_DB_NAME=hindsight_db
# Hindsight Version
HINDSIGHT_VERSION=latest
# LLM Configuration
HINDSIGHT_API_LLM_PROVIDER=openai
OPENAI_API_KEY=your-openai-api-key-here
# Alternative LLM providers (uncomment and configure as needed):
# HINDSIGHT_API_LLM_PROVIDER=anthropic
# ANTHROPIC_API_KEY=your-anthropic-api-key
# HINDSIGHT_API_LLM_PROVIDER=gemini
# GEMINI_API_KEY=your-gemini-api-key
# HINDSIGHT_API_LLM_PROVIDER=groq
# GROQ_API_KEY=your-groq-api-key
# Vector and Text Search (already configured in docker-compose.yaml)
# HINDSIGHT_API_VECTOR_EXTENSION=pgvectorscale
# HINDSIGHT_API_TEXT_SEARCH_EXTENSION=pg_textsearch
@@ -1,55 +0,0 @@
# PostgreSQL with pgvector, pgvectorscale, and pg_textsearch extensions
# All three extensions from Timescale/pgvector for high-performance vector and text search
# Note: Requires PostgreSQL 16+
FROM postgres:17
# Install build dependencies and Rust toolchain
RUN apt-get update && apt-get install -y \
build-essential \
git \
postgresql-server-dev-17 \
libpq-dev \
cmake \
curl \
pkg-config \
libssl-dev \
&& rm -rf /var/lib/apt/lists/*
# Install Rust toolchain (required for pgvectorscale)
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
ENV PATH="/root/.cargo/bin:${PATH}"
# Install pgvector (required by pgvectorscale)
RUN cd /tmp && \
git clone --branch v0.8.0 https://github.com/pgvector/pgvector.git && \
cd pgvector && \
make && \
make install && \
rm -rf /tmp/pgvector
# Install cargo-pgrx (PostgreSQL extension framework for Rust)
RUN cargo install cargo-pgrx --version 0.12.5 --locked && \
cargo pgrx init --pg17 /usr/bin/pg_config
# Install pgvectorscale (DiskANN index support)
RUN cd /tmp && \
git clone --branch 0.5.1 https://github.com/timescale/pgvectorscale.git && \
cd pgvectorscale/pgvectorscale && \
cargo pgrx install --release && \
rm -rf /tmp/pgvectorscale
# Install pg_textsearch (BM25 text search)
RUN cd /tmp && \
git clone https://github.com/timescale/pg_textsearch.git && \
cd pg_textsearch && \
make && \
make install && \
rm -rf /tmp/pg_textsearch
# Clean up build dependencies (keep runtime dependencies)
RUN apt-get purge -y --auto-remove git cmake curl && \
rm -rf /root/.cargo/registry /root/.cargo/git
# Ensure extensions are preloaded (pg_textsearch requires preloading)
RUN echo "shared_preload_libraries = 'pg_textsearch'" >> /usr/share/postgresql/postgresql.conf.sample
-101
View File
@@ -1,101 +0,0 @@
# Hindsight with Timescale Extensions
This Docker Compose setup provides a complete Hindsight deployment with **Timescale extensions**:
- **pgvectorscale** - DiskANN algorithm for disk-based scalable vector search
- **pg_textsearch** - High-performance BM25 text search
Both extensions are from [Timescale](https://github.com/timescale) and provide production-grade performance.
## Prerequisites
- Docker and Docker Compose installed
- OpenAI API key (or another LLM provider)
## Quick Start
```bash
# Set environment variables
export HINDSIGHT_DB_PASSWORD="your-secure-password"
export OPENAI_API_KEY="your-openai-api-key"
# Build and start
docker compose -f docker/docker-compose/timescale/docker-compose.yaml up -d --build
# Check logs
docker compose -f docker/docker-compose/timescale/docker-compose.yaml logs -f
```
**Access:**
- API: http://localhost:8888
- Control Plane: http://localhost:9999
## Stop and Clean Up
```bash
# Stop services
docker compose -f docker/docker-compose/timescale/docker-compose.yaml down
# Remove volumes (deletes all data)
docker compose -f docker/docker-compose/timescale/docker-compose.yaml down -v
```
## Configuration
### Environment Variables
| Variable | Description | Default |
|----------|-------------|---------|
| `HINDSIGHT_DB_PASSWORD` | PostgreSQL password | `hindsight_password` |
| `HINDSIGHT_DB_USER` | PostgreSQL username | `hindsight_user` |
| `HINDSIGHT_DB_NAME` | Database name | `hindsight_db` |
| `HINDSIGHT_VERSION` | Hindsight Docker image version | `latest` |
| `OPENAI_API_KEY` | OpenAI API key | (required) |
| `HINDSIGHT_API_LLM_PROVIDER` | LLM provider | `openai` |
### Why Timescale Extensions?
**pgvectorscale (DiskANN):**
- 28x lower p95 latency vs dedicated vector databases
- 16x higher query throughput at 99% recall
- 60-75% cost reduction (disk is cheaper than RAM)
- Best for large datasets (10M+ vectors)
**pg_textsearch (BM25):**
- High-performance keyword retrieval
- Native BM25 ranking algorithm
- Optimized for full-text search
## Troubleshooting
### Extensions not installed
Check if extensions are available:
```bash
docker exec -it hindsight-db-timescale psql -U hindsight_user -d hindsight_db -c "\dx"
```
You should see:
- `vector` (pgvector)
- `vectorscale` (pgvectorscale/DiskANN)
- `pg_textsearch` (BM25 search)
### Build fails
If the Docker build fails during pgvectorscale compilation:
1. Ensure you have sufficient memory (recommended: 4GB+)
2. Check Docker build logs for Rust compilation errors
3. Try building with more resources: `docker compose build --no-cache --memory 4g`
### Port conflicts
If port 5438 is already in use, modify the `ports` section in docker-compose.yaml.
## Learn More
- [pgvectorscale GitHub](https://github.com/timescale/pgvectorscale)
- [pg_textsearch GitHub](https://github.com/timescale/pg_textsearch)
- [HNSW vs DiskANN](https://www.tigerdata.com/learn/hnsw-vs-diskann)
- [Hindsight Documentation](https://hindsight.dev)
@@ -1,108 +0,0 @@
name: hindsight
# Docker Compose file for Hindsight with Timescale extensions
# - pgvectorscale: DiskANN vector search (disk-based, scalable)
# - pg_textsearch: BM25 text search (high-performance keyword retrieval)
#
# Quick start:
# docker compose -f docker/docker-compose/timescale/docker-compose.yaml up -d --build
#
# Required environment variables:
# - HINDSIGHT_DB_PASSWORD: Password for the PostgreSQL user
# - OPENAI_API_KEY (or configure another LLM provider)
#
# Optional environment variables with defaults:
# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
services:
db:
# Custom PostgreSQL image with Timescale extensions (pgvectorscale + pg_textsearch)
build:
context: .
dockerfile: Dockerfile
container_name: hindsight-db-timescale
restart: always
# Expose PostgreSQL port (using 5438 to avoid conflicts with other setups)
ports:
- "5438:5432"
environment:
POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:-hindsight_password}
POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
volumes:
- pg_data:/var/lib/postgresql/data
networks:
- hindsight-net
# Health check to ensure database is ready
healthcheck:
test: ["CMD-SHELL", "pg_isready -U hindsight_user"]
interval: 5s
timeout: 5s
retries: 5
timescale-init:
build:
context: .
dockerfile: Dockerfile
depends_on:
db:
condition: service_healthy
environment:
- PGPASSWORD=${HINDSIGHT_DB_PASSWORD:-hindsight_password}
command: >
bash -c "
echo 'PostgreSQL is ready - creating hindsight_db database';
psql -h hindsight-db-timescale -p 5432 -U hindsight_user -c 'CREATE DATABASE hindsight_db;' 2>/dev/null || echo 'Database already exists';
echo 'Installing Timescale extensions...';
echo '1/3: Installing pgvector (required by pgvectorscale)...';
psql -h hindsight-db-timescale -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vector CASCADE;';
echo '2/3: Installing pgvectorscale (DiskANN vector search)...';
psql -h hindsight-db-timescale -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vectorscale CASCADE;';
echo '3/3: Installing pg_textsearch (BM25 text search)...';
psql -h hindsight-db-timescale -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS pg_textsearch CASCADE;';
echo '';
echo '✅ Timescale extensions installed successfully';
echo '';
echo 'Installed extensions:';
psql -h hindsight-db-timescale -p 5432 -U hindsight_user -d hindsight_db -c \"\\dx\" | grep -E '(vector|vectorscale|pg_textsearch)';
"
restart: "no"
networks:
- hindsight-net
hindsight:
image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
container_name: hindsight-app-timescale
ports:
- "8888:8888"
- "9999:9999"
environment:
# LLM Configuration
HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
# Database Configuration
HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
# Timescale Extensions
# pgvectorscale: DiskANN algorithm for disk-based scalable vector search
HINDSIGHT_API_VECTOR_EXTENSION: pgvectorscale
# pg_textsearch: High-performance BM25 text search
HINDSIGHT_API_TEXT_SEARCH_EXTENSION: pg_textsearch
depends_on:
db:
condition: service_healthy
timescale-init:
condition: service_completed_successfully
networks:
- hindsight-net
networks:
hindsight-net:
driver: bridge
volumes:
pg_data:
@@ -1,93 +0,0 @@
name: hindsight
# Docker Compose file for Hindsight with PostgreSQL and vectorchord
# docker compose -f docker/docker-compose/docker-compose.yaml down && sleep 2 && docker compose -f docker/docker-compose/docker-compose.yaml up -d
# Make sure to set the required environment variables before running:
# - HINDSIGHT_DB_PASSWORD: Password for the PostgreSQL user
# - Configure LLM provider variables as needed (see below in the hindsight service)
#
# Usage:
# docker compose up -d
#
# Optional environment variables with defaults:
# - HINDSIGHT_VERSION: Hindsight application version (default: latest)
# - HINDSIGHT_DB_USER: PostgreSQL user (default: hindsight_user)
# - HINDSIGHT_DB_NAME: PostgreSQL database name (default: hindsight_db)
# - HINDSIGHT_DB_VERSION: PostgreSQL version (default: 18)
services:
db:
# Use a PostgreSQL-Image with vectorchord extension pre-installed
image: tensorchord/vchord-suite:pg${HINDSIGHT_DB_VERSION:-18-latest}
container_name: hindsight-db
restart: always
# Expose PostgreSQL port
ports:
- "5436:5432"
environment:
POSTGRES_USER: ${HINDSIGHT_DB_USER:-hindsight_user}
POSTGRES_PASSWORD: ${HINDSIGHT_DB_PASSWORD:-hindsight_password}
POSTGRES_DB: ${HINDSIGHT_DB_NAME:-hindsight_db}
volumes:
- pg_data:/var/lib/postgresql/${HINDSIGHT_DB_VERSION:-18}/docker
networks:
- hindsight-net
vectorchord-init:
image: tensorchord/vchord-suite:pg18-latest
#container_name: vectorchord-init
depends_on:
- db
environment:
- PGPASSWORD=${HINDSIGHT_DB_PASSWORD:-hindsight_password}
command: >
bash -c "
echo 'Waiting for PostgreSQL to be ready...';
until pg_isready -h hindsight-db -p 5432 -U hindsight_user; do
echo 'PostgreSQL is unavailable - sleeping';
sleep 2;
done;
echo 'PostgreSQL is ready - creating hindsight_db database';
psql -h hindsight-db -p 5432 -U hindsight_user -c 'CREATE DATABASE hindsight_db;' 2>/dev/null || echo 'Database already exists';
echo 'Creating extensions in hindsight_db database';
psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vchord CASCADE;';
psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS pg_tokenizer CASCADE;';
psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c 'CREATE EXTENSION IF NOT EXISTS vchord_bm25 CASCADE;';
echo 'Creating llmlingua2 tokenizer';
psql -h hindsight-db -p 5432 -U hindsight_user -d hindsight_db -c \"SELECT create_tokenizer('llmlingua2', \\$\\$ model = \\\"llmlingua2\\\" \\$\\$);\" 2>/dev/null || echo 'Tokenizer already exists or creation skipped';
echo 'Database and extensions created successfully';
"
restart: "no"
networks:
- hindsight-net
hindsight:
image: ghcr.io/vectorize-io/hindsight:${HINDSIGHT_VERSION:-latest}
container_name: hindsight-app
ports:
- "8888:8888"
- "9999:9999"
environment:
# LLM Configuration (uses OpenAI for testing vchord)
# LLM configuration
HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-openai}
HINDSIGHT_API_LLM_API_KEY: ${OPENAI_API_KEY:-your-api-key}
# Database Configuration
HINDSIGHT_API_DATABASE_URL: postgresql://${HINDSIGHT_DB_USER:-hindsight_user}:${HINDSIGHT_DB_PASSWORD:-hindsight_password}@db:5432/${HINDSIGHT_DB_NAME:-hindsight_db}
# Vector and Text Search Extensions
HINDSIGHT_API_VECTOR_EXTENSION: vchord
HINDSIGHT_API_TEXT_SEARCH_EXTENSION: vchord
depends_on:
- db
networks:
- hindsight-net
networks:
hindsight-net:
driver: bridge
volumes:
pg_data:
+9 -121
View File
@@ -2,25 +2,19 @@
# Supports building API-only, Control Plane-only, or both
#
# Build args:
# INCLUDE_API=true/false - Include API (default: true)
# INCLUDE_CP=true/false - Include Control Plane (default: true)
# INCLUDE_LOCAL_MODELS=true/false - Include local ML models for embeddings/reranking (default: true)
# Set to false when using external providers (TEI, OpenAI, Cohere)
# PRELOAD_ML_MODELS=true/false - Pre-download ML models during build (default: true)
# Only effective when INCLUDE_LOCAL_MODELS=true
# NOTE: tiktoken encodings are ALWAYS preloaded (required for air-gapped deployments)
# INCLUDE_API=true/false - Include API (default: true)
# INCLUDE_CP=true/false - Include Control Plane (default: true)
# PRELOAD_ML_MODELS=true/false - Pre-download ML models during build (default: true)
#
# Examples:
# docker build -t hindsight . # Both (standalone)
# docker build -t hindsight-api --build-arg INCLUDE_CP=false . # API only
# docker build -t hindsight-cp --build-arg INCLUDE_API=false . # Control Plane only
# docker build -t hindsight --build-arg PRELOAD_ML_MODELS=false . # Skip ML model preload
# docker build -t hindsight --build-arg INCLUDE_LOCAL_MODELS=false . # Skip local ML deps (for external providers)
ARG INCLUDE_API=true
ARG INCLUDE_CP=true
ARG PRELOAD_ML_MODELS=true
ARG INCLUDE_LOCAL_MODELS=true
# =============================================================================
# Stage: API Builder
@@ -28,7 +22,6 @@ ARG INCLUDE_LOCAL_MODELS=true
FROM python:3.11-slim AS api-builder
ARG INCLUDE_API
ARG INCLUDE_LOCAL_MODELS
RUN if [ "$INCLUDE_API" != "true" ]; then echo "Skipping API build" && exit 0; fi
WORKDIR /app
@@ -47,15 +40,6 @@ COPY hindsight-api/README.md ./api/
WORKDIR /app/api
# Remove local ML model dependencies if INCLUDE_LOCAL_MODELS=false
# This creates a smaller image when using external providers (TEI, OpenAI, Cohere)
RUN if [ "$INCLUDE_LOCAL_MODELS" != "true" ]; then \
echo "Removing local-models dependencies (sentence-transformers, torch, transformers)..." && \
sed -i '/"sentence-transformers/d' pyproject.toml && \
sed -i '/"transformers/d' pyproject.toml && \
sed -i '/"torch/d' pyproject.toml; \
fi
# Sync dependencies (will create lock file if needed)
RUN uv sync
@@ -112,10 +96,6 @@ RUN rm -f package-lock.json && sed -i '/"@vectorize-io\/hindsight-client":/d' pa
# Copy built SDK directly into node_modules (more reliable than npm link in Docker)
COPY --from=sdk-builder /app/hindsight-clients/typescript ./node_modules/@vectorize-io/hindsight-client
# Accept base path as build argument for reverse proxy deployments
# Usage: docker build --build-arg NEXT_PUBLIC_BASE_PATH=/hindsight ...
ARG NEXT_PUBLIC_BASE_PATH=""
# Build Control Plane - run next build first, then custom standalone copy
# (The build:standalone script expects a specific path structure that differs in Docker)
RUN npm exec -- next build
@@ -172,58 +152,16 @@ USER hindsight
ENV PATH="/app/api/.venv/bin:${PATH}"
# Pre-download tiktoken encoding (ALWAYS - required for token counting even in air-gapped envs)
# Tiktoken is a core runtime dependency, not an optional ML model
RUN MAX_RETRIES=3; \
RETRY_DELAY=5; \
for i in $(seq 1 $MAX_RETRIES); do \
echo "Attempt $i/$MAX_RETRIES: Downloading tiktoken encoding..."; \
/app/api/.venv/bin/python -c "\
import tiktoken; \
print('Downloading cl100k_base encoding...'); \
tiktoken.get_encoding('cl100k_base'); \
print('Tiktoken encoding cached successfully')" && break; \
if [ $i -lt $MAX_RETRIES ]; then \
echo "Attempt $i failed, retrying in ${RETRY_DELAY}s..."; \
sleep $RETRY_DELAY; \
RETRY_DELAY=$((RETRY_DELAY * 2)); \
fi; \
done; \
if [ $i -eq $MAX_RETRIES ]; then \
echo "ERROR: Failed to download tiktoken encoding after $MAX_RETRIES attempts"; \
exit 1; \
fi
# Pre-download ML models to avoid runtime download (conditional)
# Only runs if both PRELOAD_ML_MODELS=true AND INCLUDE_LOCAL_MODELS=true
# Includes retry logic with exponential backoff for transient network failures
ARG PRELOAD_ML_MODELS
ARG INCLUDE_LOCAL_MODELS
ENV HF_HUB_DOWNLOAD_TIMEOUT=600
RUN if [ "$PRELOAD_ML_MODELS" = "true" ] && [ "$INCLUDE_LOCAL_MODELS" = "true" ]; then \
MAX_RETRIES=3; \
RETRY_DELAY=10; \
for i in $(seq 1 $MAX_RETRIES); do \
echo "Attempt $i/$MAX_RETRIES: Downloading ML models..."; \
/app/api/.venv/bin/python -c "\
import os; os.environ['HF_HUB_DOWNLOAD_TIMEOUT'] = '600'; \
RUN if [ "$PRELOAD_ML_MODELS" = "true" ]; then \
/app/api/.venv/bin/python -c "\
from sentence_transformers import SentenceTransformer, CrossEncoder; \
print('Downloading embedding model...'); \
SentenceTransformer('BAAI/bge-small-en-v1.5'); \
print('Downloading cross-encoder model...'); \
CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2'); \
print('Models cached successfully')" && break; \
if [ $i -lt $MAX_RETRIES ]; then \
echo "Attempt $i failed, retrying in ${RETRY_DELAY}s..."; \
sleep $RETRY_DELAY; \
RETRY_DELAY=$((RETRY_DELAY * 2)); \
fi; \
done; \
if [ $i -eq $MAX_RETRIES ] && ! /app/api/.venv/bin/python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('BAAI/bge-small-en-v1.5')" 2>/dev/null; then \
echo "ERROR: Failed to download models after $MAX_RETRIES attempts"; \
exit 1; \
fi; \
elif [ "$INCLUDE_LOCAL_MODELS" != "true" ]; then echo "Skipping ML model preload (local-models not included)"; \
print('Models cached successfully')"; \
else echo "Skipping ML model preload"; fi
EXPOSE 8888
@@ -234,10 +172,6 @@ ENV HINDSIGHT_API_LOG_LEVEL=info
ENV HINDSIGHT_ENABLE_API=true
ENV HINDSIGHT_ENABLE_CP=false
ENV PYTHONUNBUFFERED=1
# Suppress verbose transformers/HuggingFace model loading warnings
ENV TRANSFORMERS_VERBOSITY=error
ENV HF_HUB_VERBOSITY=error
ENV TOKENIZERS_PARALLELISM=false
CMD ["/app/start-all.sh"]
@@ -323,58 +257,16 @@ USER hindsight
ENV PATH="/app/api/.venv/bin:${PATH}"
# Pre-download tiktoken encoding (ALWAYS - required for token counting even in air-gapped envs)
# Tiktoken is a core runtime dependency, not an optional ML model
RUN MAX_RETRIES=3; \
RETRY_DELAY=5; \
for i in $(seq 1 $MAX_RETRIES); do \
echo "Attempt $i/$MAX_RETRIES: Downloading tiktoken encoding..."; \
/app/api/.venv/bin/python -c "\
import tiktoken; \
print('Downloading cl100k_base encoding...'); \
tiktoken.get_encoding('cl100k_base'); \
print('Tiktoken encoding cached successfully')" && break; \
if [ $i -lt $MAX_RETRIES ]; then \
echo "Attempt $i failed, retrying in ${RETRY_DELAY}s..."; \
sleep $RETRY_DELAY; \
RETRY_DELAY=$((RETRY_DELAY * 2)); \
fi; \
done; \
if [ $i -eq $MAX_RETRIES ]; then \
echo "ERROR: Failed to download tiktoken encoding after $MAX_RETRIES attempts"; \
exit 1; \
fi
# Pre-download ML models to avoid runtime download (conditional)
# Only runs if both PRELOAD_ML_MODELS=true AND INCLUDE_LOCAL_MODELS=true
# Includes retry logic with exponential backoff for transient network failures
ARG PRELOAD_ML_MODELS
ARG INCLUDE_LOCAL_MODELS
ENV HF_HUB_DOWNLOAD_TIMEOUT=600
RUN if [ "$PRELOAD_ML_MODELS" = "true" ] && [ "$INCLUDE_LOCAL_MODELS" = "true" ]; then \
MAX_RETRIES=3; \
RETRY_DELAY=10; \
for i in $(seq 1 $MAX_RETRIES); do \
echo "Attempt $i/$MAX_RETRIES: Downloading ML models..."; \
/app/api/.venv/bin/python -c "\
import os; os.environ['HF_HUB_DOWNLOAD_TIMEOUT'] = '600'; \
RUN if [ "$PRELOAD_ML_MODELS" = "true" ]; then \
/app/api/.venv/bin/python -c "\
from sentence_transformers import SentenceTransformer, CrossEncoder; \
print('Downloading embedding model...'); \
SentenceTransformer('BAAI/bge-small-en-v1.5'); \
print('Downloading cross-encoder model...'); \
CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2'); \
print('Models cached successfully')" && break; \
if [ $i -lt $MAX_RETRIES ]; then \
echo "Attempt $i failed, retrying in ${RETRY_DELAY}s..."; \
sleep $RETRY_DELAY; \
RETRY_DELAY=$((RETRY_DELAY * 2)); \
fi; \
done; \
if [ $i -eq $MAX_RETRIES ] && ! /app/api/.venv/bin/python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('BAAI/bge-small-en-v1.5')" 2>/dev/null; then \
echo "ERROR: Failed to download models after $MAX_RETRIES attempts"; \
exit 1; \
fi; \
elif [ "$INCLUDE_LOCAL_MODELS" != "true" ]; then echo "Skipping ML model preload (local-models not included)"; \
print('Models cached successfully')"; \
else echo "Skipping ML model preload"; fi
EXPOSE 8888 9999
@@ -387,10 +279,6 @@ ENV HINDSIGHT_CP_DATAPLANE_API_URL=http://localhost:8888
ENV HINDSIGHT_ENABLE_API=true
ENV HINDSIGHT_ENABLE_CP=true
ENV PYTHONUNBUFFERED=1
# Suppress verbose transformers/HuggingFace model loading warnings
ENV TRANSFORMERS_VERBOSITY=error
ENV HF_HUB_VERBOSITY=error
ENV TOKENIZERS_PARALLELISM=false
CMD ["/app/start-all.sh"]
-51
View File
@@ -1,51 +0,0 @@
#!/bin/bash
#
# Local Test Script for Slim Docker Images
#
# This script makes it easy to test slim images locally with external providers.
# It expects API keys to be set in environment variables.
#
# Usage:
# export GROQ_API_KEY=gsk_xxx
# export OPENAI_API_KEY=sk-xxx
# export COHERE_API_KEY=xxx
# ./docker/test-slim-local.sh
#
# Or inline:
# GROQ_API_KEY=gsk_xxx OPENAI_API_KEY=sk_xxx COHERE_API_KEY=xxx ./docker/test-slim-local.sh
#
set -euo pipefail
# Check for required API keys
if [ -z "${GROQ_API_KEY:-}" ]; then
echo "❌ Error: GROQ_API_KEY environment variable is required"
echo "Set it with: export GROQ_API_KEY=gsk_xxx"
exit 1
fi
if [ -z "${OPENAI_API_KEY:-}" ]; then
echo "❌ Error: OPENAI_API_KEY environment variable is required"
echo "Set it with: export OPENAI_API_KEY=sk-xxx"
exit 1
fi
if [ -z "${COHERE_API_KEY:-}" ]; then
echo "❌ Error: COHERE_API_KEY environment variable is required"
echo "Set it with: export COHERE_API_KEY=xxx"
exit 1
fi
# Configuration
IMAGE="${1:-hindsight-slim:test}"
echo "Testing image: $IMAGE"
echo ""
# Set up external providers
export HINDSIGHT_API_EMBEDDINGS_PROVIDER=openai
export HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=$OPENAI_API_KEY
export HINDSIGHT_API_RERANKER_PROVIDER=cohere
export HINDSIGHT_API_COHERE_API_KEY=$COHERE_API_KEY
# Run the test
exec "$(dirname "$0")/test-image.sh" "$IMAGE" standalone
+2 -2
View File
@@ -2,8 +2,8 @@ apiVersion: v2
name: hindsight
description: Hindsight helm chart
type: application
version: 0.4.11
appVersion: "0.4.11"
version: 0.2.1
appVersion: "0.2.1"
keywords:
- ai
- memory
-48
View File
@@ -80,22 +80,6 @@ Control plane selector labels
app.kubernetes.io/component: control-plane
{{- end }}
{{/*
Worker labels
*/}}
{{- define "hindsight.worker.labels" -}}
{{ include "hindsight.labels" . }}
app.kubernetes.io/component: worker
{{- end }}
{{/*
Worker selector labels
*/}}
{{- define "hindsight.worker.selectorLabels" -}}
{{ include "hindsight.selectorLabels" . }}
app.kubernetes.io/component: worker
{{- end }}
{{/*
Create the name of the service account to use
*/}}
@@ -127,38 +111,6 @@ API URL for control plane
{{- printf "http://%s-api:%d" (include "hindsight.fullname" .) (.Values.api.service.port | int) }}
{{- end }}
{{/*
TEI reranker labels
*/}}
{{- define "hindsight.tei.reranker.labels" -}}
{{ include "hindsight.labels" . }}
app.kubernetes.io/component: tei-reranker
{{- end }}
{{/*
TEI reranker selector labels
*/}}
{{- define "hindsight.tei.reranker.selectorLabels" -}}
{{ include "hindsight.selectorLabels" . }}
app.kubernetes.io/component: tei-reranker
{{- end }}
{{/*
TEI embedding labels
*/}}
{{- define "hindsight.tei.embedding.labels" -}}
{{ include "hindsight.labels" . }}
app.kubernetes.io/component: tei-embedding
{{- end }}
{{/*
TEI embedding selector labels
*/}}
{{- define "hindsight.tei.embedding.selectorLabels" -}}
{{ include "hindsight.selectorLabels" . }}
app.kubernetes.io/component: tei-embedding
{{- end }}
{{/*
Get the name of the secret to use
*/}}
+2 -22
View File
@@ -33,7 +33,7 @@ spec:
- name: api
securityContext:
{{- toYaml .Values.securityContext | nindent 10 }}
image: "{{ .Values.api.image.repository }}:{{ .Values.api.image.tag | default .Values.version | default .Chart.AppVersion }}"
image: "{{ .Values.api.image.repository }}:{{ .Values.api.image.tag | default .Values.version }}"
imagePullPolicy: {{ .Values.api.image.pullPolicy }}
ports:
- name: http
@@ -55,30 +55,10 @@ spec:
{{- end }}
- name: HINDSIGHT_API_DATABASE_URL
value: {{ include "hindsight.databaseUrl" . | quote }}
{{- /* Disable internal worker when dedicated workers are enabled */}}
{{- if .Values.worker.enabled }}
- name: HINDSIGHT_API_WORKER_ENABLED
value: "false"
{{- end }}
{{- /* Explicitly set port to override K8s service discovery env var (HINDSIGHT_API_PORT) */}}
- name: HINDSIGHT_API_PORT
value: {{ .Values.api.service.targetPort | quote }}
{{- range $key, $value := .Values.api.env }}
- name: {{ $key }}
value: {{ $value | quote }}
{{- end }}
{{- if .Values.tei.reranker.enabled }}
- name: HINDSIGHT_API_RERANKER_PROVIDER
value: "tei"
- name: HINDSIGHT_API_RERANKER_TEI_URL
value: "http://{{ include "hindsight.fullname" . }}-tei-reranker:{{ .Values.tei.reranker.port }}"
{{- end }}
{{- if .Values.tei.embedding.enabled }}
- name: HINDSIGHT_API_EMBEDDINGS_PROVIDER
value: "tei"
- name: HINDSIGHT_API_EMBEDDINGS_TEI_URL
value: "http://{{ include "hindsight.fullname" . }}-tei-embedding:{{ .Values.tei.embedding.port }}"
{{- end }}
{{- /* Only use api.secrets when not using existingSecret (for chart-managed secrets) */}}
{{- if not .Values.existingSecret }}
{{- range $key, $value := .Values.api.secrets }}
@@ -99,7 +79,7 @@ spec:
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with (.Values.api.affinity | default .Values.affinity) }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
@@ -33,7 +33,7 @@ spec:
- name: control-plane
securityContext:
{{- toYaml .Values.securityContext | nindent 10 }}
image: "{{ .Values.controlPlane.image.repository }}:{{ .Values.controlPlane.image.tag | default .Values.version | default .Chart.AppVersion }}"
image: "{{ .Values.controlPlane.image.repository }}:{{ .Values.controlPlane.image.tag | default .Values.version }}"
imagePullPolicy: {{ .Values.controlPlane.image.pullPolicy }}
ports:
- name: http
@@ -71,7 +71,7 @@ spec:
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with (.Values.controlPlane.affinity | default .Values.affinity) }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
-56
View File
@@ -1,56 +0,0 @@
{{- if and .Values.api.enabled .Values.api.podDisruptionBudget.enabled }}
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: {{ include "hindsight.fullname" . }}-api
labels:
{{- include "hindsight.api.labels" . | nindent 4 }}
spec:
{{- if .Values.api.podDisruptionBudget.minAvailable }}
minAvailable: {{ .Values.api.podDisruptionBudget.minAvailable }}
{{- end }}
{{- if .Values.api.podDisruptionBudget.maxUnavailable }}
maxUnavailable: {{ .Values.api.podDisruptionBudget.maxUnavailable }}
{{- end }}
selector:
matchLabels:
{{- include "hindsight.api.selectorLabels" . | nindent 6 }}
{{- end }}
---
{{- if and .Values.controlPlane.enabled .Values.controlPlane.podDisruptionBudget.enabled }}
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: {{ include "hindsight.fullname" . }}-control-plane
labels:
{{- include "hindsight.controlPlane.labels" . | nindent 4 }}
spec:
{{- if .Values.controlPlane.podDisruptionBudget.minAvailable }}
minAvailable: {{ .Values.controlPlane.podDisruptionBudget.minAvailable }}
{{- end }}
{{- if .Values.controlPlane.podDisruptionBudget.maxUnavailable }}
maxUnavailable: {{ .Values.controlPlane.podDisruptionBudget.maxUnavailable }}
{{- end }}
selector:
matchLabels:
{{- include "hindsight.controlPlane.selectorLabels" . | nindent 6 }}
{{- end }}
---
{{- if and .Values.worker.enabled .Values.worker.podDisruptionBudget.enabled }}
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: {{ include "hindsight.fullname" . }}-worker
labels:
{{- include "hindsight.worker.labels" . | nindent 4 }}
spec:
{{- if .Values.worker.podDisruptionBudget.minAvailable }}
minAvailable: {{ .Values.worker.podDisruptionBudget.minAvailable }}
{{- end }}
{{- if .Values.worker.podDisruptionBudget.maxUnavailable }}
maxUnavailable: {{ .Values.worker.podDisruptionBudget.maxUnavailable }}
{{- end }}
selector:
matchLabels:
{{- include "hindsight.worker.selectorLabels" . | nindent 6 }}
{{- end }}
@@ -1,76 +0,0 @@
{{- if .Values.tei.embedding.enabled }}
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "hindsight.fullname" . }}-tei-embedding
labels:
{{- include "hindsight.tei.embedding.labels" . | nindent 4 }}
spec:
replicas: {{ .Values.tei.embedding.replicaCount }}
selector:
matchLabels:
{{- include "hindsight.tei.embedding.selectorLabels" . | nindent 6 }}
template:
metadata:
{{- with .Values.podAnnotations }}
annotations:
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "hindsight.tei.embedding.selectorLabels" . | nindent 8 }}
spec:
{{- if .Values.serviceAccount.create }}
serviceAccountName: {{ include "hindsight.serviceAccountName" . }}
{{- end }}
securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }}
containers:
- name: tei-embedding
securityContext:
{{- toYaml .Values.securityContext | nindent 10 }}
image: "{{ .Values.tei.embedding.image.repository }}:{{ .Values.tei.embedding.image.tag }}"
imagePullPolicy: {{ .Values.tei.embedding.image.pullPolicy }}
args:
- "--model-id"
- {{ .Values.tei.embedding.model | quote }}
- "--hostname"
- "0.0.0.0"
{{- range .Values.tei.embedding.args }}
- {{ . | quote }}
{{- end }}
ports:
- name: http
containerPort: {{ .Values.tei.embedding.port }}
protocol: TCP
env:
- name: PORT
value: {{ .Values.tei.embedding.port | quote }}
{{- range $key, $value := .Values.tei.embedding.env }}
- name: {{ $key }}
value: {{ $value | quote }}
{{- end }}
livenessProbe:
{{- toYaml .Values.tei.embedding.livenessProbe | nindent 10 }}
readinessProbe:
{{- toYaml .Values.tei.embedding.readinessProbe | nindent 10 }}
resources:
{{- toYaml .Values.tei.embedding.resources | nindent 10 }}
volumeMounts:
- name: model-cache
mountPath: /data
volumes:
- name: model-cache
emptyDir: {}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}
@@ -1,17 +0,0 @@
{{- if .Values.tei.embedding.enabled }}
apiVersion: v1
kind: Service
metadata:
name: {{ include "hindsight.fullname" . }}-tei-embedding
labels:
{{- include "hindsight.tei.embedding.labels" . | nindent 4 }}
spec:
type: ClusterIP
ports:
- port: {{ .Values.tei.embedding.port }}
targetPort: http
protocol: TCP
name: http
selector:
{{- include "hindsight.tei.embedding.selectorLabels" . | nindent 4 }}
{{- end }}
@@ -1,76 +0,0 @@
{{- if .Values.tei.reranker.enabled }}
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "hindsight.fullname" . }}-tei-reranker
labels:
{{- include "hindsight.tei.reranker.labels" . | nindent 4 }}
spec:
replicas: {{ .Values.tei.reranker.replicaCount }}
selector:
matchLabels:
{{- include "hindsight.tei.reranker.selectorLabels" . | nindent 6 }}
template:
metadata:
{{- with .Values.podAnnotations }}
annotations:
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "hindsight.tei.reranker.selectorLabels" . | nindent 8 }}
spec:
{{- if .Values.serviceAccount.create }}
serviceAccountName: {{ include "hindsight.serviceAccountName" . }}
{{- end }}
securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }}
containers:
- name: tei-reranker
securityContext:
{{- toYaml .Values.securityContext | nindent 10 }}
image: "{{ .Values.tei.reranker.image.repository }}:{{ .Values.tei.reranker.image.tag }}"
imagePullPolicy: {{ .Values.tei.reranker.image.pullPolicy }}
args:
- "--model-id"
- {{ .Values.tei.reranker.model | quote }}
- "--hostname"
- "0.0.0.0"
{{- range .Values.tei.reranker.args }}
- {{ . | quote }}
{{- end }}
ports:
- name: http
containerPort: {{ .Values.tei.reranker.port }}
protocol: TCP
env:
- name: PORT
value: {{ .Values.tei.reranker.port | quote }}
{{- range $key, $value := .Values.tei.reranker.env }}
- name: {{ $key }}
value: {{ $value | quote }}
{{- end }}
livenessProbe:
{{- toYaml .Values.tei.reranker.livenessProbe | nindent 10 }}
readinessProbe:
{{- toYaml .Values.tei.reranker.readinessProbe | nindent 10 }}
resources:
{{- toYaml .Values.tei.reranker.resources | nindent 10 }}
volumeMounts:
- name: model-cache
mountPath: /data
volumes:
- name: model-cache
emptyDir: {}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}
@@ -1,17 +0,0 @@
{{- if .Values.tei.reranker.enabled }}
apiVersion: v1
kind: Service
metadata:
name: {{ include "hindsight.fullname" . }}-tei-reranker
labels:
{{- include "hindsight.tei.reranker.labels" . | nindent 4 }}
spec:
type: ClusterIP
ports:
- port: {{ .Values.tei.reranker.port }}
targetPort: http
protocol: TCP
name: http
selector:
{{- include "hindsight.tei.reranker.selectorLabels" . | nindent 4 }}
{{- end }}
@@ -1,25 +0,0 @@
{{- if .Values.worker.enabled }}
apiVersion: v1
kind: Service
metadata:
name: {{ include "hindsight.fullname" . }}-worker
labels:
{{- include "hindsight.worker.labels" . | nindent 4 }}
{{- if .Values.podAnnotations }}
annotations:
{{- /* Common Prometheus annotations for metrics scraping */}}
prometheus.io/scrape: "true"
prometheus.io/port: {{ .Values.worker.service.port | quote }}
prometheus.io/path: "/metrics"
{{- end }}
spec:
# Headless service for StatefulSet (enables stable DNS names like worker-0.worker.namespace)
clusterIP: None
ports:
- port: {{ .Values.worker.service.port }}
targetPort: {{ .Values.worker.service.targetPort }}
protocol: TCP
name: http
selector:
{{- include "hindsight.worker.selectorLabels" . | nindent 4 }}
{{- end }}
@@ -1,110 +0,0 @@
{{- if .Values.worker.enabled }}
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: {{ include "hindsight.fullname" . }}-worker
labels:
{{- include "hindsight.worker.labels" . | nindent 4 }}
spec:
serviceName: {{ include "hindsight.fullname" . }}-worker
replicas: {{ .Values.worker.replicaCount }}
selector:
matchLabels:
{{- include "hindsight.worker.selectorLabels" . | nindent 6 }}
template:
metadata:
annotations:
{{- if not .Values.existingSecret }}
checksum/secret: {{ include (print $.Template.BasePath "/secret.yaml") . | sha256sum }}
{{- end }}
{{- with .Values.podAnnotations }}
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "hindsight.worker.selectorLabels" . | nindent 8 }}
spec:
{{- if .Values.serviceAccount.create }}
serviceAccountName: {{ include "hindsight.serviceAccountName" . }}
{{- end }}
securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }}
containers:
- name: worker
securityContext:
{{- toYaml .Values.securityContext | nindent 10 }}
image: "{{ .Values.worker.image.repository }}:{{ .Values.worker.image.tag | default .Values.version | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.worker.image.pullPolicy }}
command: ["hindsight-worker"]
ports:
- name: http
containerPort: {{ .Values.worker.service.targetPort }}
protocol: TCP
{{- if .Values.existingSecret }}
envFrom:
- secretRef:
name: {{ .Values.existingSecret }}
{{- end }}
env:
{{- /* POSTGRES_PASSWORD must be defined before DATABASE_URL for $(VAR) interpolation */}}
{{- if not .Values.postgresql.enabled }}
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: {{ include "hindsight.secretName" . }}
key: postgres-password
{{- end }}
- name: HINDSIGHT_API_DATABASE_URL
value: {{ include "hindsight.databaseUrl" . | quote }}
{{- /* Worker ID uses pod name (StatefulSet provides stable names like worker-0, worker-1) */}}
- name: HINDSIGHT_API_WORKER_ID
valueFrom:
fieldRef:
fieldPath: metadata.name
{{- /* Inherit LLM config from api.env */}}
{{- range $key, $value := .Values.api.env }}
- name: {{ $key }}
value: {{ $value | quote }}
{{- end }}
{{- /* Worker-specific env vars */}}
{{- range $key, $value := .Values.worker.env }}
- name: {{ $key }}
value: {{ $value | quote }}
{{- end }}
{{- /* Only use secrets when not using existingSecret */}}
{{- if not .Values.existingSecret }}
{{- /* Inherit secrets from api.secrets */}}
{{- range $key, $value := .Values.api.secrets }}
- name: {{ $key }}
valueFrom:
secretKeyRef:
name: {{ include "hindsight.secretName" $ }}
key: {{ $key }}
{{- end }}
{{- /* Worker-specific secrets (can override api.secrets) */}}
{{- range $key, $value := .Values.worker.secrets }}
- name: {{ $key }}
valueFrom:
secretKeyRef:
name: {{ include "hindsight.secretName" $ }}
key: {{ $key }}
{{- end }}
{{- end }}
livenessProbe:
{{- toYaml .Values.worker.livenessProbe | nindent 10 }}
readinessProbe:
{{- toYaml .Values.worker.readinessProbe | nindent 10 }}
resources:
{{- toYaml .Values.worker.resources | nindent 10 }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with (.Values.worker.affinity | default .Values.affinity) }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}
+3 -166
View File
@@ -1,8 +1,7 @@
# Default values for hindsight
# Global version override - use this to set a consistent image tag across all components
# If not set, defaults to Chart.appVersion from Chart.yaml
# version: ""
# Chart version - use this to set a consistent image tag across all components
version: "0.1.1"
# Use an existing secret instead of creating one from values
# When set, all keys from this secret are injected as environment variables via envFrom
@@ -58,15 +57,6 @@ api:
timeoutSeconds: 3
failureThreshold: 3
# Pod disruption budget
podDisruptionBudget:
enabled: false
minAvailable: 1
# maxUnavailable: 1
# Pod affinity/anti-affinity (overrides global affinity for this component)
# affinity: {}
# Environment variables
env:
#HINDSIGHT_API_LLM_PROVIDER: "groq"
@@ -77,72 +67,6 @@ api:
# HINDSIGHT_API_LLM_API_KEY: "your-api-key"
# HINDSIGHT_API_LLM_BASE_URL: "https://api.groq.com/openai/v1"
# Worker settings (distributed task processing)
# When enabled, dedicated worker pods process tasks and the API's internal worker is disabled
worker:
enabled: false
replicaCount: 2
image:
repository: ghcr.io/vectorize-io/hindsight-api
pullPolicy: IfNotPresent
# tag: "" # defaults to .Values.version, then Chart.appVersion if not specified
service:
# Service for metrics scraping (headless for StatefulSet)
port: 8889
targetPort: 8889
# Resource limits and requests
resources:
limits:
cpu: 2000m
memory: 4Gi
requests:
cpu: 500m
memory: 1Gi
# Liveness and readiness probes
livenessProbe:
httpGet:
path: /health
port: 8889
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
readinessProbe:
httpGet:
path: /health
port: 8889
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 3
failureThreshold: 3
# Worker-specific environment variables
env:
# Poll interval in milliseconds (how often to check for new tasks)
HINDSIGHT_API_WORKER_POLL_INTERVAL_MS: "500"
# Number of tasks to claim per poll cycle
HINDSIGHT_API_WORKER_BATCH_SIZE: "10"
# Max retries before marking a task as failed
HINDSIGHT_API_WORKER_MAX_RETRIES: "3"
# HTTP port for metrics/health (matches service.targetPort)
HINDSIGHT_API_WORKER_HTTP_PORT: "8889"
# Pod disruption budget
podDisruptionBudget:
enabled: false
minAvailable: 1
# maxUnavailable: 1
# Pod affinity/anti-affinity (overrides global affinity for this component)
# affinity: {}
# Secret environment variables (inherited from api.secrets if not specified)
secrets: {}
# Image settings for control plane
controlPlane:
enabled: true
@@ -183,15 +107,6 @@ controlPlane:
timeoutSeconds: 3
failureThreshold: 3
# Pod disruption budget
podDisruptionBudget:
enabled: false
minAvailable: 1
# maxUnavailable: 1
# Pod affinity/anti-affinity (overrides global affinity for this component)
# affinity: {}
# Environment variables
env:
NODE_ENV: "production"
@@ -290,87 +205,9 @@ nodeSelector: {}
# Tolerations
tolerations: []
# Affinity (applied to all components unless overridden per-component)
# Affinity
affinity: {}
# TEI (Text Embeddings Inference) - optional standalone deployments
# for reranking and/or embedding models
tei:
reranker:
enabled: false
replicaCount: 1
image:
repository: ghcr.io/huggingface/text-embeddings-inference
tag: cpu-1.8.3
pullPolicy: IfNotPresent
model: "cross-encoder/ms-marco-MiniLM-L-6-v2"
port: 8090
args:
- "--auto-truncate"
env:
PAYLOAD_LIMIT: "10000000"
MAX_CLIENT_BATCH_SIZE: "256"
resources:
limits:
cpu: 2000m
memory: 2Gi
requests:
cpu: 500m
memory: 1Gi
livenessProbe:
httpGet:
path: /health
port: 8090
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 6
readinessProbe:
httpGet:
path: /health
port: 8090
initialDelaySeconds: 15
periodSeconds: 5
timeoutSeconds: 3
failureThreshold: 3
embedding:
enabled: false
replicaCount: 1
image:
repository: ghcr.io/huggingface/text-embeddings-inference
tag: cpu-1.8.3
pullPolicy: IfNotPresent
model: "sentence-transformers/all-MiniLM-L6-v2"
port: 8091
args: []
env:
PAYLOAD_LIMIT: "10000000"
MAX_CLIENT_BATCH_SIZE: "256"
resources:
limits:
cpu: 2000m
memory: 2Gi
requests:
cpu: 500m
memory: 1Gi
livenessProbe:
httpGet:
path: /health
port: 8091
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 6
readinessProbe:
httpGet:
path: /health
port: 8091
initialDelaySeconds: 15
periodSeconds: 5
timeoutSeconds: 3
failureThreshold: 3
# Autoscaling
autoscaling:
enabled: false
+1 -1
View File
@@ -46,4 +46,4 @@ __all__ = [
"RemoteTEICrossEncoder",
"LLMConfig",
]
__version__ = "0.4.11"
__version__ = "0.1.0"
-59
View File
@@ -244,65 +244,6 @@ def run_db_migration(
typer.echo("Database migrations completed successfully")
async def _decommission_worker(db_url: str, worker_id: str, schema: str = "public") -> int:
"""Release all tasks owned by a worker, setting them back to pending status."""
is_pg0, instance_name, _ = parse_pg0_url(db_url)
if is_pg0:
typer.echo(f"Starting embedded PostgreSQL (instance: {instance_name})...")
resolved_url = await resolve_database_url(db_url)
conn = await asyncpg.connect(resolved_url)
try:
table = _fq_table("async_operations", schema)
result = await conn.fetch(
f"""
UPDATE {table}
SET status = 'pending', worker_id = NULL, claimed_at = NULL, updated_at = now()
WHERE worker_id = $1 AND status = 'processing'
RETURNING operation_id
""",
worker_id,
)
return len(result)
finally:
await conn.close()
@app.command(name="decommission-worker")
def decommission_worker(
worker_id: str = typer.Argument(..., help="Worker ID to decommission"),
schema: str = typer.Option("public", "--schema", "-s", help="Database schema"),
yes: bool = typer.Option(False, "--yes", "-y", help="Skip confirmation prompt"),
):
"""Release all tasks owned by a worker (sets status back to pending).
Use this command when a worker has crashed or been removed without graceful shutdown.
All tasks that were being processed by the worker will be released back to the queue
so other workers can pick them up.
"""
config = HindsightConfig.from_env()
if not config.database_url:
typer.echo("Error: Database URL not configured.", err=True)
typer.echo("Set HINDSIGHT_API_DATABASE_URL environment variable.", err=True)
raise typer.Exit(1)
if not yes:
typer.confirm(
f"This will release all tasks owned by worker '{worker_id}' back to pending. Continue?",
abort=True,
)
typer.echo(f"Decommissioning worker '{worker_id}' (schema: {schema})...")
count = asyncio.run(_decommission_worker(config.database_url, worker_id, schema))
if count > 0:
typer.echo(f"Released {count} task(s) from worker '{worker_id}'")
else:
typer.echo(f"No tasks found for worker '{worker_id}'")
def main():
app()
@@ -6,13 +6,11 @@ Create Date: 2025-11-27 11:54:19.228030
"""
import os
from collections.abc import Sequence
import sqlalchemy as sa
from alembic import op
from pgvector.sqlalchemy import Vector
from sqlalchemy import text
from sqlalchemy.dialects import postgresql
# revision identifiers, used by Alembic.
@@ -22,114 +20,11 @@ branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _detect_vector_extension() -> str:
"""
Detect or validate vector extension: 'pgvector', 'vchord', or 'pgvectorscale'.
Respects HINDSIGHT_API_VECTOR_EXTENSION env var if set.
"""
conn = op.get_bind()
vector_extension = os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector").lower()
# Validate configured extension is installed
if vector_extension == "pgvectorscale":
# pgvectorscale/DiskANN requires pgvector
pgvector_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vector'")).scalar()
if not pgvector_check:
raise RuntimeError(
"DiskANN requires pgvector. Install with: CREATE EXTENSION vector; then vectorscale or pg_diskann CASCADE;"
)
# Check for either vectorscale (open source) or pg_diskann (Azure)
vectorscale_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vectorscale'")).scalar()
pg_diskann_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'pg_diskann'")).scalar()
if vectorscale_check:
return "pgvectorscale"
elif pg_diskann_check:
return "pg_diskann"
else:
raise RuntimeError(
"Configured vector extension 'pgvectorscale' not found. Install either:\n"
" - pgvectorscale: CREATE EXTENSION vectorscale CASCADE;\n"
" - pg_diskann (Azure): CREATE EXTENSION pg_diskann CASCADE;"
)
elif vector_extension == "vchord":
vchord_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vchord'")).scalar()
if not vchord_check:
raise RuntimeError(
"Configured vector extension 'vchord' not found. Install it with: CREATE EXTENSION vchord CASCADE;"
)
return "vchord"
elif vector_extension == "pgvector":
pgvector_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vector'")).scalar()
if not pgvector_check:
raise RuntimeError(
"Configured vector extension 'pgvector' not found. Install it with: CREATE EXTENSION vector;"
)
return "pgvector"
else:
raise ValueError(
f"Invalid HINDSIGHT_API_VECTOR_EXTENSION: {vector_extension}. Must be 'pgvector', 'vchord', or 'pgvectorscale'"
)
def _detect_text_search_extension() -> str:
"""
Detect or validate text search extension: 'native', 'vchord', or 'pg_textsearch'.
Respects HINDSIGHT_API_TEXT_SEARCH_EXTENSION env var.
Creates the extension if needed.
"""
text_search_extension = os.getenv("HINDSIGHT_API_TEXT_SEARCH_EXTENSION", "native").lower()
if text_search_extension == "vchord":
# Create vchord_bm25 extension if not exists
try:
op.execute("CREATE EXTENSION IF NOT EXISTS vchord_bm25 CASCADE")
except Exception:
# Extension might already exist or user lacks permissions - verify it exists
conn = op.get_bind()
result = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vchord_bm25'")).fetchone()
if not result:
# Extension truly doesn't exist - re-raise the error
raise
return "vchord"
elif text_search_extension == "pg_textsearch":
# Create pg_textsearch extension if not exists
try:
op.execute("CREATE EXTENSION IF NOT EXISTS pg_textsearch CASCADE")
except Exception:
# Extension might already exist or user lacks permissions - verify it exists
conn = op.get_bind()
result = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'pg_textsearch'")).fetchone()
if not result:
# Extension truly doesn't exist - re-raise the error
raise
return "pg_textsearch"
elif text_search_extension == "native":
return "native"
else:
raise ValueError(
f"Invalid HINDSIGHT_API_TEXT_SEARCH_EXTENSION: {text_search_extension}. Must be 'native', 'vchord', or 'pg_textsearch'"
)
def upgrade() -> None:
"""Upgrade schema - create all tables from scratch."""
# Note: pgvector extension is installed globally BEFORE migrations run
# See migrations.py:run_migrations() - this ensures the extension is available
# to all schemas, not just the one being migrated
# We keep this here as a fallback for backwards compatibility
# This may fail if user lacks permissions, which is fine if extension already exists
try:
op.execute("CREATE EXTENSION IF NOT EXISTS vector")
except Exception:
# Extension might already exist or user lacks permissions - verify it exists
conn = op.get_bind()
result = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vector'")).fetchone()
if not result:
# Extension truly doesn't exist - re-raise the error
raise
# Enable required extensions
op.execute("CREATE EXTENSION IF NOT EXISTS vector")
# Create banks table
op.create_table(
@@ -257,29 +152,11 @@ def upgrade() -> None:
)
# Add search_vector column for full-text search
# Type depends on configured text search backend
text_search_ext = _detect_text_search_extension()
if text_search_ext == "vchord":
# VectorChord BM25: bm25vector type (no GENERATED - tokenization happens on INSERT)
# Note: vchord_bm25 extension creates types in bm25_catalog schema
op.execute("""
ALTER TABLE memory_units
ADD COLUMN search_vector bm25_catalog.bm25vector
""")
elif text_search_ext == "pg_textsearch":
# Timescale pg_textsearch: dummy TEXT column for consistency (indexes operate on base columns directly)
op.execute("""
ALTER TABLE memory_units
ADD COLUMN search_vector TEXT
""")
else: # native
# Native PostgreSQL: tsvector with automatic generation
op.execute("""
ALTER TABLE memory_units
ADD COLUMN search_vector tsvector
GENERATED ALWAYS AS (to_tsvector('english', COALESCE(text, '') || ' ' || COALESCE(context, ''))) STORED
""")
op.execute("""
ALTER TABLE memory_units
ADD COLUMN search_vector tsvector
GENERATED ALWAYS AS (to_tsvector('english', COALESCE(text, '') || ' ' || COALESCE(context, ''))) STORED
""")
op.create_index("idx_memory_units_bank_id", "memory_units", ["bank_id"])
op.create_index("idx_memory_units_document_id", "memory_units", ["document_id"])
@@ -309,61 +186,19 @@ def upgrade() -> None:
["bank_id", sa.text("event_date DESC")],
postgresql_where=sa.text("fact_type = 'observation'"),
)
# Create vector index - conditional based on available extension
vector_ext = _detect_vector_extension()
op.create_index(
"idx_memory_units_embedding",
"memory_units",
["embedding"],
postgresql_using="hnsw",
postgresql_ops={"embedding": "vector_cosine_ops"},
)
if vector_ext == "pgvectorscale":
# Use DiskANN index for pgvectorscale (disk-based, scalable)
op.execute("""
CREATE INDEX idx_memory_units_embedding ON memory_units
USING diskann (embedding vector_cosine_ops)
WITH (num_neighbors = 50)
""")
elif vector_ext == "pg_diskann":
# Use DiskANN index for pg_diskann (Azure)
op.execute("""
CREATE INDEX idx_memory_units_embedding ON memory_units
USING diskann (embedding vector_cosine_ops)
WITH (max_neighbors = 50)
""")
elif vector_ext == "vchord":
# Use vchordrq index for vchord (supports high-dimensional embeddings)
op.execute("""
CREATE INDEX idx_memory_units_embedding ON memory_units
USING vchordrq (embedding vector_l2_ops)
""")
else: # pgvector
# Use HNSW index for pgvector
op.create_index(
"idx_memory_units_embedding",
"memory_units",
["embedding"],
postgresql_using="hnsw",
postgresql_ops={"embedding": "vector_cosine_ops"},
)
# Create full-text search index on search_vector
# Index type depends on text search backend
if text_search_ext == "vchord":
# VectorChord BM25 index
op.execute("""
CREATE INDEX idx_memory_units_text_search ON memory_units
USING bm25 (search_vector bm25_catalog.bm25_ops)
""")
elif text_search_ext == "pg_textsearch":
# Timescale pg_textsearch BM25 index on text column
# Note: pg_textsearch doesn't support expressions, so we index the main text column
op.execute("""
CREATE INDEX idx_memory_units_text_search ON memory_units
USING bm25(text)
WITH (text_config='english')
""")
else: # native
# Native PostgreSQL GIN index
op.execute("""
CREATE INDEX idx_memory_units_text_search ON memory_units
USING gin(search_vector)
""")
# Create BM25 full-text search index on search_vector
op.execute("""
CREATE INDEX idx_memory_units_text_search ON memory_units
USING gin(search_vector)
""")
op.execute("""
CREATE MATERIALIZED VIEW memory_units_bm25 AS
@@ -1,70 +0,0 @@
"""Add file_storage table for BYTEA-based file storage
Revision ID: a1b2c3d4e5f6
Revises: y0t1u2v3w4x5
Create Date: 2026-02-16
Creates a dedicated table for storing uploaded files using BYTEA.
This provides zero-config file storage that "just works" for development
and small deployments. For production/scale, use S3-compatible storage.
Files are stored in a separate table to avoid bloating the documents table.
"""
from collections.abc import Sequence
from alembic import context, op
revision: str = "a1b2c3d4e5f6"
down_revision: str | Sequence[str] | None = "y0t1u2v3w4x5"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _get_schema_prefix() -> str:
"""Get schema prefix for table names (required for multi-tenant support)."""
schema = context.config.get_main_option("target_schema")
return f'"{schema}".' if schema else ""
def upgrade() -> None:
"""Create file_storage table for BYTEA storage."""
schema = _get_schema_prefix()
# Create file_storage table (minimal: just key + data)
op.execute(
f"""
CREATE TABLE {schema}file_storage (
storage_key TEXT PRIMARY KEY,
data BYTEA NOT NULL
)
"""
)
# Add file tracking columns to documents table
op.execute(
f"""
ALTER TABLE {schema}documents
ADD COLUMN IF NOT EXISTS file_storage_key TEXT,
ADD COLUMN IF NOT EXISTS file_original_name TEXT,
ADD COLUMN IF NOT EXISTS file_content_type TEXT
"""
)
def downgrade() -> None:
"""Remove file_storage table and related columns."""
schema = _get_schema_prefix()
# Drop columns from documents table
op.execute(
f"""
ALTER TABLE {schema}documents
DROP COLUMN IF EXISTS file_storage_key,
DROP COLUMN IF EXISTS file_original_name,
DROP COLUMN IF EXISTS file_content_type
"""
)
# Drop file_storage table
op.execute(f"DROP TABLE IF EXISTS {schema}file_storage")
@@ -1,44 +0,0 @@
"""add_memory_links_from_type_weight_index
Revision ID: f1a2b3c4d5e6
Revises: e0a1b2c3d4e5
Create Date: 2025-01-12
Add composite index on memory_links (from_unit_id, link_type, weight DESC)
to optimize MPFP graph traversal queries that need top-k edges per type.
"""
from collections.abc import Sequence
from alembic import context, op
# revision identifiers, used by Alembic.
revision: str = "f1a2b3c4d5e6"
down_revision: str | Sequence[str] | None = "e0a1b2c3d4e5"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _get_schema_prefix() -> str:
"""Get schema prefix for table names (e.g., 'tenant_x.' or '' for public)."""
schema = context.config.get_main_option("target_schema")
return f'"{schema}".' if schema else ""
def upgrade() -> None:
"""Add composite index for efficient MPFP edge loading."""
schema = _get_schema_prefix()
# Create composite index for efficient top-k per (from_node, link_type) queries
# This enables LATERAL joins to use index-only scans with early termination
# Note: Not using CONCURRENTLY here as it requires running outside a transaction
# For production with large tables, consider running this manually with CONCURRENTLY
op.execute(
f"CREATE INDEX IF NOT EXISTS idx_memory_links_from_type_weight "
f"ON {schema}memory_links(from_unit_id, link_type, weight DESC)"
)
def downgrade() -> None:
"""Remove the composite index."""
schema = _get_schema_prefix()
op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_links_from_type_weight")
@@ -1,48 +0,0 @@
"""add_tags_column
Revision ID: g2a3b4c5d6e7
Revises: f1a2b3c4d5e6
Create Date: 2025-01-13
Add tags column to memory_units and documents tables for visibility scoping.
Tags enable filtering memories by scope (e.g., user IDs, session IDs) during recall/reflect.
"""
from collections.abc import Sequence
from alembic import context, op
# revision identifiers, used by Alembic.
revision: str = "g2a3b4c5d6e7"
down_revision: str | Sequence[str] | None = "f1a2b3c4d5e6"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _get_schema_prefix() -> str:
"""Get schema prefix for table names (e.g., 'tenant_x.' or '' for public)."""
schema = context.config.get_main_option("target_schema")
return f'"{schema}".' if schema else ""
def upgrade() -> None:
"""Add tags column to memory_units and documents tables."""
schema = _get_schema_prefix()
# Add tags column to memory_units table
op.execute(f"ALTER TABLE {schema}memory_units ADD COLUMN IF NOT EXISTS tags VARCHAR[] NOT NULL DEFAULT '{{}}'")
# Create GIN index for efficient array containment queries (tags && ARRAY['x'])
op.execute(f"CREATE INDEX IF NOT EXISTS idx_memory_units_tags ON {schema}memory_units USING GIN (tags)")
# Add tags column to documents table for document-level tags
op.execute(f"ALTER TABLE {schema}documents ADD COLUMN IF NOT EXISTS tags VARCHAR[] NOT NULL DEFAULT '{{}}'")
def downgrade() -> None:
"""Remove tags columns and index."""
schema = _get_schema_prefix()
op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_units_tags")
op.execute(f"ALTER TABLE {schema}memory_units DROP COLUMN IF EXISTS tags")
op.execute(f"ALTER TABLE {schema}documents DROP COLUMN IF EXISTS tags")
@@ -1,112 +0,0 @@
"""mental_models_v4
Revision ID: h3c4d5e6f7g8
Revises: g2a3b4c5d6e7
Create Date: 2026-01-08 00:00:00.000000
This migration implements the v4 mental models system:
1. Deletes existing observation memory_units (observations now in mental models)
2. Adds mission column to banks (replacing background)
3. Creates mental_models table with final schema
Mental models can reference entities when an entity is "promoted" to a mental model.
Summary content is stored as JSONB observations with per-observation fact attribution.
"""
from collections.abc import Sequence
from alembic import context, op
# revision identifiers, used by Alembic.
revision: str = "h3c4d5e6f7g8"
down_revision: str | Sequence[str] | None = "g2a3b4c5d6e7"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _get_schema_prefix() -> str:
"""Get schema prefix for table names (required for multi-tenant support)."""
schema = context.config.get_main_option("target_schema")
return f'"{schema}".' if schema else ""
def upgrade() -> None:
"""Apply mental models v4 changes."""
schema = _get_schema_prefix()
# Step 1: Delete observation memory_units (cascades to unit_entities links)
# Observations are now handled through mental models, not memory_units
op.execute(f"DELETE FROM {schema}memory_units WHERE fact_type = 'observation'")
# Step 2: Drop observation-specific index (if it exists)
op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_units_observation_date")
# Step 3: Add mission column to banks (replacing background)
op.execute(f"ALTER TABLE {schema}banks ADD COLUMN IF NOT EXISTS mission TEXT")
# Migrate: copy background to mission if background column exists
# Use DO block to check column existence first (idempotent for re-runs)
schema_name = context.config.get_main_option("target_schema") or "public"
op.execute(f"""
DO $$
BEGIN
IF EXISTS (
SELECT 1 FROM information_schema.columns
WHERE table_schema = '{schema_name}' AND table_name = 'banks' AND column_name = 'background'
) THEN
UPDATE {schema}banks
SET mission = background
WHERE mission IS NULL;
END IF;
END $$;
""")
# Remove background column (replaced by mission)
op.execute(f"ALTER TABLE {schema}banks DROP COLUMN IF EXISTS background")
# Step 4: Create mental_models table with final v4 schema (if not exists)
op.execute(f"""
CREATE TABLE IF NOT EXISTS {schema}mental_models (
id VARCHAR(64) NOT NULL,
bank_id VARCHAR(64) NOT NULL,
subtype VARCHAR(32) NOT NULL,
name VARCHAR(256) NOT NULL,
description TEXT NOT NULL,
entity_id UUID,
observations JSONB DEFAULT '{{"observations": []}}'::jsonb,
links VARCHAR[],
tags VARCHAR[] DEFAULT '{{}}',
last_updated TIMESTAMP WITH TIME ZONE,
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
PRIMARY KEY (id, bank_id),
FOREIGN KEY (bank_id) REFERENCES {schema}banks(bank_id) ON DELETE CASCADE,
FOREIGN KEY (entity_id) REFERENCES {schema}entities(id) ON DELETE SET NULL,
CONSTRAINT ck_mental_models_subtype CHECK (subtype IN ('structural', 'emergent', 'pinned', 'learned'))
)
""")
# Step 5: Create indexes for efficient queries (if not exist)
op.execute(f"CREATE INDEX IF NOT EXISTS idx_mental_models_bank_id ON {schema}mental_models(bank_id)")
op.execute(f"CREATE INDEX IF NOT EXISTS idx_mental_models_subtype ON {schema}mental_models(bank_id, subtype)")
op.execute(f"CREATE INDEX IF NOT EXISTS idx_mental_models_entity_id ON {schema}mental_models(entity_id)")
# GIN index for efficient tags array filtering
op.execute(f"CREATE INDEX IF NOT EXISTS idx_mental_models_tags ON {schema}mental_models USING GIN(tags)")
def downgrade() -> None:
"""Revert mental models v4 changes."""
schema = _get_schema_prefix()
# Drop mental_models table (cascades to indexes)
op.execute(f"DROP TABLE IF EXISTS {schema}mental_models CASCADE")
# Add back background column to banks
op.execute(f"ALTER TABLE {schema}banks ADD COLUMN IF NOT EXISTS background TEXT")
# Migrate mission back to background
op.execute(f"UPDATE {schema}banks SET background = mission WHERE background IS NULL")
# Remove mission column
op.execute(f"ALTER TABLE {schema}banks DROP COLUMN IF EXISTS mission")
# Note: Cannot restore deleted observations - they are lost on downgrade
@@ -1,41 +0,0 @@
"""delete_opinions
Revision ID: i4d5e6f7g8h9
Revises: h3c4d5e6f7g8
Create Date: 2026-01-15 00:00:00.000000
This migration removes opinion facts from memory_units.
Opinions are no longer a separate fact type - they are now represented
through mental model observations with confidence scores.
"""
from collections.abc import Sequence
from alembic import context, op
# revision identifiers, used by Alembic.
revision: str = "i4d5e6f7g8h9"
down_revision: str | Sequence[str] | None = "h3c4d5e6f7g8"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _get_schema_prefix() -> str:
"""Get schema prefix for table names (required for multi-tenant support)."""
schema = context.config.get_main_option("target_schema")
return f'"{schema}".' if schema else ""
def upgrade() -> None:
"""Delete opinion memory_units."""
schema = _get_schema_prefix()
# Delete opinion memory_units (cascades to unit_entities links)
# Opinions are now handled through mental model observations
op.execute(f"DELETE FROM {schema}memory_units WHERE fact_type = 'opinion'")
def downgrade() -> None:
"""Cannot restore deleted opinions."""
# Note: Cannot restore deleted opinions - they are lost on downgrade
pass
@@ -1,95 +0,0 @@
"""mental_model_versions
Revision ID: j5e6f7g8h9i0
Revises: i4d5e6f7g8h9
Create Date: 2026-01-16 00:00:00.000000
This migration adds versioning support for mental models:
1. Creates mental_model_versions table to store observation snapshots
2. Adds version column to mental_models for tracking current version
This enables changelog/diff functionality for mental model observations.
"""
from collections.abc import Sequence
from alembic import context, op
# revision identifiers, used by Alembic.
revision: str = "j5e6f7g8h9i0"
down_revision: str | Sequence[str] | None = "i4d5e6f7g8h9"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _get_schema_prefix() -> str:
"""Get schema prefix for table names (required for multi-tenant support)."""
schema = context.config.get_main_option("target_schema")
return f'"{schema}".' if schema else ""
def upgrade() -> None:
"""Create mental_model_versions table and add version tracking."""
schema = _get_schema_prefix()
# Create mental_model_versions table for storing observation snapshots
op.execute(f"""
CREATE TABLE {schema}mental_model_versions (
id SERIAL PRIMARY KEY,
mental_model_id VARCHAR(64) NOT NULL,
bank_id VARCHAR(64) NOT NULL,
version INT NOT NULL,
observations JSONB NOT NULL DEFAULT '{{"observations": []}}'::jsonb,
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
FOREIGN KEY (mental_model_id, bank_id)
REFERENCES {schema}mental_models(id, bank_id) ON DELETE CASCADE,
UNIQUE (mental_model_id, bank_id, version)
)
""")
# Index for efficient version queries (get latest, list versions)
op.execute(f"""
CREATE INDEX idx_mental_model_versions_lookup
ON {schema}mental_model_versions(mental_model_id, bank_id, version DESC)
""")
# Add version column to mental_models to track current version
op.execute(f"""
ALTER TABLE {schema}mental_models
ADD COLUMN IF NOT EXISTS version INT NOT NULL DEFAULT 0
""")
# Migrate existing mental models: create version 1 for any that have observations
op.execute(f"""
INSERT INTO {schema}mental_model_versions (mental_model_id, bank_id, version, observations, created_at)
SELECT id, bank_id, 1, observations, COALESCE(last_updated, created_at)
FROM {schema}mental_models
WHERE observations IS NOT NULL
AND observations != '{{"observations": []}}'::jsonb
AND (observations->'observations') IS NOT NULL
AND jsonb_array_length(observations->'observations') > 0
""")
# Update version to 1 for migrated mental models
op.execute(f"""
UPDATE {schema}mental_models
SET version = 1
WHERE observations IS NOT NULL
AND observations != '{{"observations": []}}'::jsonb
AND (observations->'observations') IS NOT NULL
AND jsonb_array_length(observations->'observations') > 0
""")
def downgrade() -> None:
"""Remove mental_model_versions table and version column."""
schema = _get_schema_prefix()
# Drop index
op.execute(f"DROP INDEX IF EXISTS {schema}idx_mental_model_versions_lookup")
# Drop versions table
op.execute(f"DROP TABLE IF EXISTS {schema}mental_model_versions")
# Remove version column from mental_models
op.execute(f"ALTER TABLE {schema}mental_models DROP COLUMN IF EXISTS version")
@@ -1,58 +0,0 @@
"""add_directive_subtype
Revision ID: k6f7g8h9i0j1
Revises: j5e6f7g8h9i0
Create Date: 2026-01-16 00:00:00.000000
This migration adds 'directive' to the mental_models subtype constraint.
Directives are hard rules with user-provided observations that the reflect agent must follow.
"""
from collections.abc import Sequence
from alembic import context, op
# revision identifiers, used by Alembic.
revision: str = "k6f7g8h9i0j1"
down_revision: str | Sequence[str] | None = "j5e6f7g8h9i0"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _get_schema_prefix() -> str:
"""Get schema prefix for table names (required for multi-tenant support)."""
schema = context.config.get_main_option("target_schema")
return f'"{schema}".' if schema else ""
def upgrade() -> None:
"""Add 'directive' to mental_models subtype constraint."""
schema = _get_schema_prefix()
# Drop existing constraint
op.execute(f"ALTER TABLE {schema}mental_models DROP CONSTRAINT IF EXISTS ck_mental_models_subtype")
# Create new constraint with 'directive' added
op.execute(f"""
ALTER TABLE {schema}mental_models
ADD CONSTRAINT ck_mental_models_subtype
CHECK (subtype IN ('structural', 'emergent', 'pinned', 'learned', 'directive'))
""")
def downgrade() -> None:
"""Remove 'directive' from mental_models subtype constraint."""
schema = _get_schema_prefix()
# First delete any directives (cannot downgrade if they exist)
op.execute(f"DELETE FROM {schema}mental_models WHERE subtype = 'directive'")
# Drop constraint with directive
op.execute(f"ALTER TABLE {schema}mental_models DROP CONSTRAINT IF EXISTS ck_mental_models_subtype")
# Recreate original constraint without directive
op.execute(f"""
ALTER TABLE {schema}mental_models
ADD CONSTRAINT ck_mental_models_subtype
CHECK (subtype IN ('structural', 'emergent', 'pinned', 'learned'))
""")
@@ -1,109 +0,0 @@
"""add_worker_columns
Revision ID: l7g8h9i0j1k2
Revises: k6f7g8h9i0j1
Create Date: 2026-01-19 00:00:00.000000
This migration adds columns to async_operations for distributed worker support:
- worker_id: ID of the worker that claimed the task
- claimed_at: When the task was claimed
- retry_count: Number of retry attempts
- task_payload: The serialized task dictionary
"""
from collections.abc import Sequence
import sqlalchemy as sa
from alembic import context, op
from sqlalchemy.dialects import postgresql
# revision identifiers, used by Alembic.
revision: str = "l7g8h9i0j1k2"
down_revision: str | Sequence[str] | None = "k6f7g8h9i0j1"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _get_schema_prefix() -> str:
"""Get schema prefix for table names (required for multi-tenant support)."""
schema = context.config.get_main_option("target_schema")
return f'"{schema}".' if schema else ""
def upgrade() -> None:
"""Add worker columns to async_operations."""
schema = _get_schema_prefix()
# Add worker_id column (ID of worker that claimed the task)
op.add_column(
"async_operations",
sa.Column("worker_id", sa.Text(), nullable=True),
schema=context.config.get_main_option("target_schema") or None,
)
# Add claimed_at column (when task was claimed by worker)
op.add_column(
"async_operations",
sa.Column("claimed_at", postgresql.TIMESTAMP(timezone=True), nullable=True),
schema=context.config.get_main_option("target_schema") or None,
)
# Add retry_count column (number of retry attempts)
op.add_column(
"async_operations",
sa.Column("retry_count", sa.Integer(), server_default="0", nullable=False),
schema=context.config.get_main_option("target_schema") or None,
)
# Add task_payload column (serialized task dictionary)
op.add_column(
"async_operations",
sa.Column(
"task_payload",
postgresql.JSONB(astext_type=sa.Text()),
nullable=True,
),
schema=context.config.get_main_option("target_schema") or None,
)
# Add index for efficient worker polling (pending tasks ordered by creation time)
op.execute(
f"CREATE INDEX idx_async_operations_pending_claim ON {schema}async_operations (status, created_at) "
f"WHERE status = 'pending' AND task_payload IS NOT NULL"
)
# Add index for finding tasks by worker_id (for decommissioning)
op.execute(
f"CREATE INDEX idx_async_operations_worker_id ON {schema}async_operations (worker_id) WHERE worker_id IS NOT NULL"
)
def downgrade() -> None:
"""Remove worker columns from async_operations."""
schema = _get_schema_prefix()
# Drop indexes
op.execute(f"DROP INDEX IF EXISTS {schema}idx_async_operations_pending_claim")
op.execute(f"DROP INDEX IF EXISTS {schema}idx_async_operations_worker_id")
# Drop columns
op.drop_column(
"async_operations",
"task_payload",
schema=context.config.get_main_option("target_schema") or None,
)
op.drop_column(
"async_operations",
"retry_count",
schema=context.config.get_main_option("target_schema") or None,
)
op.drop_column(
"async_operations",
"claimed_at",
schema=context.config.get_main_option("target_schema") or None,
)
op.drop_column(
"async_operations",
"worker_id",
schema=context.config.get_main_option("target_schema") or None,
)
@@ -1,41 +0,0 @@
"""mental_model_id_to_text
Revision ID: m8h9i0j1k2l3
Revises: l7g8h9i0j1k2
Create Date: 2026-01-19 00:00:00.000000
This migration changes the mental_models.id column from VARCHAR(64) to TEXT
to support longer model IDs (e.g., entity names that exceed 64 characters).
"""
from collections.abc import Sequence
from alembic import context, op
# revision identifiers, used by Alembic.
revision: str = "m8h9i0j1k2l3"
down_revision: str | Sequence[str] | None = "l7g8h9i0j1k2"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _get_schema_prefix() -> str:
"""Get schema prefix for table names (required for multi-tenant support)."""
schema = context.config.get_main_option("target_schema")
return f'"{schema}".' if schema else ""
def upgrade() -> None:
"""Change mental_models.id from VARCHAR(64) to TEXT."""
schema = _get_schema_prefix()
# Alter the id column type from VARCHAR(64) to TEXT
op.execute(f"ALTER TABLE {schema}mental_models ALTER COLUMN id TYPE TEXT")
def downgrade() -> None:
"""Revert mental_models.id from TEXT to VARCHAR(64)."""
schema = _get_schema_prefix()
# Note: This may fail if any id values exceed 64 characters
op.execute(f"ALTER TABLE {schema}mental_models ALTER COLUMN id TYPE VARCHAR(64)")
@@ -1,317 +0,0 @@
"""learnings_and_pinned_reflections
Revision ID: n9i0j1k2l3m4
Revises: m8h9i0j1k2l3
Create Date: 2026-01-21 00:00:00.000000
This migration:
1. Creates the 'learnings' table for automatic bottom-up consolidation
2. Creates the 'pinned_reflections' table for user-curated living documents
3. Adds consolidation tracking columns to the 'banks' table
"""
import os
from collections.abc import Sequence
from alembic import context, op
from sqlalchemy import text
# revision identifiers, used by Alembic.
revision: str = "n9i0j1k2l3m4"
down_revision: str | Sequence[str] | None = "m8h9i0j1k2l3"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _get_schema_prefix() -> str:
"""Get schema prefix for table names (required for multi-tenant support)."""
schema = context.config.get_main_option("target_schema")
return f'"{schema}".' if schema else ""
def _detect_vector_extension() -> str:
"""
Detect or validate vector extension: 'pgvector', 'vchord', or 'pgvectorscale'.
Respects HINDSIGHT_API_VECTOR_EXTENSION env var if set.
"""
conn = op.get_bind()
vector_extension = os.getenv("HINDSIGHT_API_VECTOR_EXTENSION", "pgvector").lower()
# Validate configured extension is installed
if vector_extension == "pgvectorscale":
# pgvectorscale/DiskANN requires pgvector
pgvector_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vector'")).scalar()
if not pgvector_check:
raise RuntimeError(
"DiskANN requires pgvector. Install with: CREATE EXTENSION vector; then vectorscale or pg_diskann CASCADE;"
)
# Check for either vectorscale (open source) or pg_diskann (Azure)
vectorscale_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vectorscale'")).scalar()
pg_diskann_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'pg_diskann'")).scalar()
if vectorscale_check:
return "pgvectorscale"
elif pg_diskann_check:
return "pg_diskann"
else:
raise RuntimeError(
"Configured vector extension 'pgvectorscale' not found. Install either:\n"
" - pgvectorscale: CREATE EXTENSION vectorscale CASCADE;\n"
" - pg_diskann (Azure): CREATE EXTENSION pg_diskann CASCADE;"
)
elif vector_extension == "vchord":
vchord_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vchord'")).scalar()
if not vchord_check:
raise RuntimeError(
"Configured vector extension 'vchord' not found. Install it with: CREATE EXTENSION vchord CASCADE;"
)
return "vchord"
elif vector_extension == "pgvector":
pgvector_check = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vector'")).scalar()
if not pgvector_check:
raise RuntimeError(
"Configured vector extension 'pgvector' not found. Install it with: CREATE EXTENSION vector;"
)
return "pgvector"
else:
raise ValueError(
f"Invalid HINDSIGHT_API_VECTOR_EXTENSION: {vector_extension}. Must be 'pgvector', 'vchord', or 'pgvectorscale'"
)
def _detect_text_search_extension() -> str:
"""
Detect or validate text search extension: 'native', 'vchord', or 'pg_textsearch'.
Respects HINDSIGHT_API_TEXT_SEARCH_EXTENSION env var.
Creates the extension if needed.
"""
text_search_extension = os.getenv("HINDSIGHT_API_TEXT_SEARCH_EXTENSION", "native").lower()
if text_search_extension == "vchord":
# Create vchord_bm25 extension if not exists
try:
op.execute("CREATE EXTENSION IF NOT EXISTS vchord_bm25 CASCADE")
except Exception:
# Extension might already exist or user lacks permissions - verify it exists
conn = op.get_bind()
result = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'vchord_bm25'")).fetchone()
if not result:
# Extension truly doesn't exist - re-raise the error
raise
return "vchord"
elif text_search_extension == "pg_textsearch":
# Create pg_textsearch extension if not exists
try:
op.execute("CREATE EXTENSION IF NOT EXISTS pg_textsearch CASCADE")
except Exception:
# Extension might already exist or user lacks permissions - verify it exists
conn = op.get_bind()
result = conn.execute(text("SELECT 1 FROM pg_extension WHERE extname = 'pg_textsearch'")).fetchone()
if not result:
# Extension truly doesn't exist - re-raise the error
raise
return "pg_textsearch"
elif text_search_extension == "native":
return "native"
else:
raise ValueError(
f"Invalid HINDSIGHT_API_TEXT_SEARCH_EXTENSION: {text_search_extension}. Must be 'native', 'vchord', or 'pg_textsearch'"
)
def upgrade() -> None:
"""Create learnings and pinned_reflections tables."""
schema = _get_schema_prefix()
# Detect which vector extension is available
vector_ext = _detect_vector_extension()
# Detect which text search extension to use
text_search_ext = _detect_text_search_extension()
# 1. Create learnings table
op.execute(f"""
CREATE TABLE {schema}learnings (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
bank_id VARCHAR(64) NOT NULL,
text TEXT NOT NULL,
proof_count INT NOT NULL DEFAULT 1,
history JSONB DEFAULT '[]'::jsonb,
mission_context VARCHAR(64),
pre_mission_change BOOLEAN DEFAULT FALSE,
embedding vector(384),
tags VARCHAR[] DEFAULT ARRAY[]::VARCHAR[],
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
updated_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now()
)
""")
# Add foreign key constraint
op.execute(f"""
ALTER TABLE {schema}learnings
ADD CONSTRAINT fk_learnings_bank_id
FOREIGN KEY (bank_id) REFERENCES {schema}banks(bank_id) ON DELETE CASCADE
""")
# Indexes for learnings
op.execute(f"CREATE INDEX idx_learnings_bank_id ON {schema}learnings(bank_id)")
# Create vector index based on detected extension
if vector_ext == "pgvectorscale":
op.execute(f"""
CREATE INDEX idx_learnings_embedding ON {schema}learnings
USING diskann (embedding vector_cosine_ops)
WITH (num_neighbors = 50)
""")
elif vector_ext == "pg_diskann":
op.execute(f"""
CREATE INDEX idx_learnings_embedding ON {schema}learnings
USING diskann (embedding vector_cosine_ops)
WITH (max_neighbors = 50)
""")
elif vector_ext == "vchord":
op.execute(f"""
CREATE INDEX idx_learnings_embedding ON {schema}learnings
USING vchordrq (embedding vector_l2_ops)
""")
else: # pgvector
op.execute(f"""
CREATE INDEX idx_learnings_embedding ON {schema}learnings
USING hnsw (embedding vector_cosine_ops)
""")
op.execute(f"CREATE INDEX idx_learnings_tags ON {schema}learnings USING GIN(tags)")
# Full-text search for learnings
if text_search_ext == "vchord":
# VectorChord BM25: bm25vector type (no GENERATED - tokenization happens on INSERT)
# Note: vchord_bm25 extension creates types in bm25_catalog schema
op.execute(f"""
ALTER TABLE {schema}learnings ADD COLUMN search_vector bm25_catalog.bm25vector
""")
op.execute(f"""
CREATE INDEX idx_learnings_text_search ON {schema}learnings
USING bm25 (search_vector bm25_catalog.bm25_ops)
""")
elif text_search_ext == "pg_textsearch":
# Timescale pg_textsearch: dummy TEXT column for consistency (indexes operate on base columns directly)
op.execute(f"""
ALTER TABLE {schema}learnings ADD COLUMN search_vector TEXT
""")
op.execute(f"""
CREATE INDEX idx_learnings_text_search ON {schema}learnings
USING bm25(text) WITH (text_config='english')
""")
else: # native
# Native PostgreSQL: tsvector with automatic generation
op.execute(f"""
ALTER TABLE {schema}learnings ADD COLUMN search_vector tsvector
GENERATED ALWAYS AS (to_tsvector('english', text)) STORED
""")
op.execute(f"CREATE INDEX idx_learnings_text_search ON {schema}learnings USING gin(search_vector)")
# 2. Create pinned_reflections table
op.execute(f"""
CREATE TABLE {schema}pinned_reflections (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
bank_id VARCHAR(64) NOT NULL,
name VARCHAR(256) NOT NULL,
source_query TEXT NOT NULL,
content TEXT NOT NULL,
embedding vector(384),
tags VARCHAR[] DEFAULT ARRAY[]::VARCHAR[],
last_refreshed_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now()
)
""")
# Add foreign key constraint
op.execute(f"""
ALTER TABLE {schema}pinned_reflections
ADD CONSTRAINT fk_pinned_reflections_bank_id
FOREIGN KEY (bank_id) REFERENCES {schema}banks(bank_id) ON DELETE CASCADE
""")
# Indexes for pinned_reflections
op.execute(f"CREATE INDEX idx_pinned_reflections_bank_id ON {schema}pinned_reflections(bank_id)")
# Create vector index based on detected extension
if vector_ext == "pgvectorscale":
op.execute(f"""
CREATE INDEX idx_pinned_reflections_embedding ON {schema}pinned_reflections
USING diskann (embedding vector_cosine_ops)
WITH (num_neighbors = 50)
""")
elif vector_ext == "pg_diskann":
op.execute(f"""
CREATE INDEX idx_pinned_reflections_embedding ON {schema}pinned_reflections
USING diskann (embedding vector_cosine_ops)
WITH (max_neighbors = 50)
""")
elif vector_ext == "vchord":
op.execute(f"""
CREATE INDEX idx_pinned_reflections_embedding ON {schema}pinned_reflections
USING vchordrq (embedding vector_l2_ops)
""")
else: # pgvector
op.execute(f"""
CREATE INDEX idx_pinned_reflections_embedding ON {schema}pinned_reflections
USING hnsw (embedding vector_cosine_ops)
""")
op.execute(f"CREATE INDEX idx_pinned_reflections_tags ON {schema}pinned_reflections USING GIN(tags)")
# Full-text search for pinned_reflections
if text_search_ext == "vchord":
# VectorChord BM25: bm25vector type (no GENERATED - tokenization happens on INSERT/UPDATE)
# Note: vchord_bm25 extension creates types in bm25_catalog schema
op.execute(f"""
ALTER TABLE {schema}pinned_reflections ADD COLUMN search_vector bm25_catalog.bm25vector
""")
op.execute(f"""
CREATE INDEX idx_pinned_reflections_text_search ON {schema}pinned_reflections
USING bm25 (search_vector bm25_catalog.bm25_ops)
""")
elif text_search_ext == "pg_textsearch":
# Timescale pg_textsearch: dummy TEXT column for consistency (indexes operate on base columns directly)
op.execute(f"""
ALTER TABLE {schema}pinned_reflections ADD COLUMN search_vector TEXT
""")
op.execute(f"""
CREATE INDEX idx_pinned_reflections_text_search ON {schema}pinned_reflections
USING bm25(content)
WITH (text_config='english')
""")
else: # native
# Native PostgreSQL: tsvector with automatic generation
op.execute(f"""
ALTER TABLE {schema}pinned_reflections ADD COLUMN search_vector tsvector
GENERATED ALWAYS AS (to_tsvector('english', COALESCE(name, '') || ' ' || content)) STORED
""")
op.execute(f"""
CREATE INDEX idx_pinned_reflections_text_search ON {schema}pinned_reflections
USING gin(search_vector)
""")
# 3. Add consolidation tracking columns to banks table
op.execute(f"""
ALTER TABLE {schema}banks
ADD COLUMN IF NOT EXISTS last_consolidated_at TIMESTAMP WITH TIME ZONE
""")
op.execute(f"""
ALTER TABLE {schema}banks
ADD COLUMN IF NOT EXISTS mission_changed_at TIMESTAMP WITH TIME ZONE
""")
def downgrade() -> None:
"""Drop learnings and pinned_reflections tables."""
schema = _get_schema_prefix()
# Drop tables
op.execute(f"DROP TABLE IF EXISTS {schema}learnings CASCADE")
op.execute(f"DROP TABLE IF EXISTS {schema}pinned_reflections CASCADE")
# Remove columns from banks
op.execute(f"ALTER TABLE {schema}banks DROP COLUMN IF EXISTS last_consolidated_at")
op.execute(f"ALTER TABLE {schema}banks DROP COLUMN IF EXISTS mission_changed_at")
@@ -1,113 +0,0 @@
"""migrate_mental_models_data
Revision ID: o0j1k2l3m4n5
Revises: n9i0j1k2l3m4
Create Date: 2026-01-21 00:00:00.000000
This migration:
1. Migrates existing 'pinned' mental models to the new 'pinned_reflections' table
2. Migrates existing 'learned' mental models to the new 'learnings' table
3. Deletes non-directive mental models (structural, emergent, pinned, learned)
4. Drops the mental_model_versions table (no longer used)
5. Adds a CHECK constraint that only 'directive' subtype is allowed
"""
from collections.abc import Sequence
from alembic import context, op
# revision identifiers, used by Alembic.
revision: str = "o0j1k2l3m4n5"
down_revision: str | Sequence[str] | None = "n9i0j1k2l3m4"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _get_schema_prefix() -> str:
"""Get schema prefix for table names (required for multi-tenant support)."""
schema = context.config.get_main_option("target_schema")
return f'"{schema}".' if schema else ""
def upgrade() -> None:
"""Migrate data and clean up old mental models."""
schema = _get_schema_prefix()
# 1. Migrate 'pinned' mental models to pinned_reflections
# For pinned models, the first observation's content becomes the pinned reflection content
op.execute(f"""
INSERT INTO {schema}pinned_reflections (bank_id, name, source_query, content, tags, created_at)
SELECT
bank_id,
name,
description AS source_query,
COALESCE(
observations->'observations'->0->>'content',
description,
''
) AS content,
tags,
created_at
FROM {schema}mental_models
WHERE subtype = 'pinned'
ON CONFLICT DO NOTHING
""")
# 2. Migrate 'learned' mental models to learnings
# Each observation in a learned model becomes a separate learning
op.execute(f"""
INSERT INTO {schema}learnings (bank_id, text, proof_count, tags, created_at)
SELECT
mm.bank_id,
obs->>'content' AS text,
GREATEST(1, COALESCE(jsonb_array_length(obs->'evidence'), 1)) AS proof_count,
mm.tags,
mm.created_at
FROM {schema}mental_models mm,
LATERAL jsonb_array_elements(mm.observations->'observations') AS obs
WHERE mm.subtype = 'learned'
AND obs->>'content' IS NOT NULL
AND obs->>'content' != ''
ON CONFLICT DO NOTHING
""")
# 3. Delete all non-directive mental models (they've been migrated or are obsolete)
op.execute(f"""
DELETE FROM {schema}mental_models
WHERE subtype != 'directive'
""")
# 4. Drop the mental_model_versions table (no longer used)
op.execute(f"DROP TABLE IF EXISTS {schema}mental_model_versions CASCADE")
# 5. Drop old constraints and add new one that only allows 'directive'
op.execute(f"ALTER TABLE {schema}mental_models DROP CONSTRAINT IF EXISTS ck_mental_models_subtype")
op.execute(f"""
ALTER TABLE {schema}mental_models
ADD CONSTRAINT ck_mental_models_subtype CHECK (subtype = 'directive')
""")
def downgrade() -> None:
"""Reverse the migration (data migration is one-way, so this just removes constraints)."""
schema = _get_schema_prefix()
# Remove the directive-only constraint
op.execute(f"ALTER TABLE {schema}mental_models DROP CONSTRAINT IF EXISTS ck_mental_models_subtype")
# Re-create mental_model_versions table
op.execute(f"""
CREATE TABLE IF NOT EXISTS {schema}mental_model_versions (
id SERIAL PRIMARY KEY,
bank_id VARCHAR(64) NOT NULL,
model_id VARCHAR(128) NOT NULL,
version INT NOT NULL,
observations JSONB NOT NULL,
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now()
)
""")
op.execute(
f"CREATE INDEX IF NOT EXISTS idx_mm_versions_lookup ON {schema}mental_model_versions(bank_id, model_id, version DESC)"
)
# Note: Data migration cannot be reversed - pinned_reflections and learnings data remains
@@ -1,194 +0,0 @@
"""new_knowledge_architecture
Revision ID: p1k2l3m4n5o6
Revises: o0j1k2l3m4n5
Create Date: 2026-01-21 00:00:00.000000
This migration implements the new knowledge architecture:
1. Drops the 'learnings' table (mental models are now in memory_units)
2. Renames 'pinned_reflections' to 'reflections'
3. Drops the 'mental_models' table completely
4. Creates 'directives' table for hard rules
5. Adds mental model support columns to 'memory_units' (proof_count, source_memory_ids, history)
The new architecture:
- Directives: Hard rules in their own table
- Mental Models: Stored in memory_units with fact_type='mental_model'
- Reflections: User-curated documents (renamed from pinned_reflections)
"""
from collections.abc import Sequence
from alembic import context, op
# revision identifiers, used by Alembic.
revision: str = "p1k2l3m4n5o6"
down_revision: str | Sequence[str] | None = "o0j1k2l3m4n5"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _get_schema_prefix() -> str:
"""Get schema prefix for table names (required for multi-tenant support)."""
schema = context.config.get_main_option("target_schema")
return f'"{schema}".' if schema else ""
def upgrade() -> None:
"""Implement new knowledge architecture."""
schema = _get_schema_prefix()
# 1. Drop the learnings table (mental models will be in memory_units)
op.execute(f"DROP TABLE IF EXISTS {schema}learnings CASCADE")
# 2. Rename pinned_reflections to reflections
op.execute(f"ALTER TABLE IF EXISTS {schema}pinned_reflections RENAME TO reflections")
# Rename indexes for reflections
op.execute(f"ALTER INDEX IF EXISTS {schema}idx_pinned_reflections_bank_id RENAME TO idx_reflections_bank_id")
op.execute(f"ALTER INDEX IF EXISTS {schema}idx_pinned_reflections_embedding RENAME TO idx_reflections_embedding")
op.execute(f"ALTER INDEX IF EXISTS {schema}idx_pinned_reflections_tags RENAME TO idx_reflections_tags")
op.execute(
f"ALTER INDEX IF EXISTS {schema}idx_pinned_reflections_text_search RENAME TO idx_reflections_text_search"
)
# Rename foreign key constraint
op.execute(f"""
ALTER TABLE {schema}reflections
DROP CONSTRAINT IF EXISTS fk_pinned_reflections_bank_id
""")
op.execute(f"""
ALTER TABLE {schema}reflections
ADD CONSTRAINT fk_reflections_bank_id
FOREIGN KEY (bank_id) REFERENCES {schema}banks(bank_id) ON DELETE CASCADE
""")
# 3. Drop the mental_models table completely
op.execute(f"DROP TABLE IF EXISTS {schema}mental_models CASCADE")
# 4. Create directives table
op.execute(f"""
CREATE TABLE {schema}directives (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
bank_id VARCHAR(64) NOT NULL,
name VARCHAR(256) NOT NULL,
content TEXT NOT NULL,
priority INT NOT NULL DEFAULT 0,
is_active BOOLEAN NOT NULL DEFAULT TRUE,
tags VARCHAR[] DEFAULT ARRAY[]::VARCHAR[],
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
updated_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now()
)
""")
# Add foreign key and indexes for directives
op.execute(f"""
ALTER TABLE {schema}directives
ADD CONSTRAINT fk_directives_bank_id
FOREIGN KEY (bank_id) REFERENCES {schema}banks(bank_id) ON DELETE CASCADE
""")
op.execute(f"CREATE INDEX idx_directives_bank_id ON {schema}directives(bank_id)")
op.execute(f"CREATE INDEX idx_directives_bank_active ON {schema}directives(bank_id, is_active)")
op.execute(f"CREATE INDEX idx_directives_tags ON {schema}directives USING GIN(tags)")
# 5. Add mental model support columns to memory_units
# proof_count: Number of memories that support this mental model
op.execute(f"""
ALTER TABLE {schema}memory_units
ADD COLUMN IF NOT EXISTS proof_count INT DEFAULT 1
""")
# source_memory_ids: Array of memory IDs that consolidated into this mental model
op.execute(f"""
ALTER TABLE {schema}memory_units
ADD COLUMN IF NOT EXISTS source_memory_ids UUID[] DEFAULT ARRAY[]::UUID[]
""")
# history: JSONB array tracking changes to mental models
op.execute(f"""
ALTER TABLE {schema}memory_units
ADD COLUMN IF NOT EXISTS history JSONB DEFAULT '[]'::jsonb
""")
# Add index for finding mental models
op.execute(f"""
CREATE INDEX IF NOT EXISTS idx_memory_units_mental_models
ON {schema}memory_units(bank_id, fact_type)
WHERE fact_type = 'mental_model'
""")
# 6. Update fact_type check constraint to include 'mental_model'
op.execute(f"ALTER TABLE {schema}memory_units DROP CONSTRAINT IF EXISTS memory_units_fact_type_check")
op.execute(f"""
ALTER TABLE {schema}memory_units
ADD CONSTRAINT memory_units_fact_type_check
CHECK (fact_type IN ('world', 'experience', 'opinion', 'observation', 'mental_model'))
""")
def downgrade() -> None:
"""Reverse the migration."""
schema = _get_schema_prefix()
# Restore original fact_type check constraint (without 'mental_model')
op.execute(f"ALTER TABLE {schema}memory_units DROP CONSTRAINT IF EXISTS memory_units_fact_type_check")
op.execute(f"""
ALTER TABLE {schema}memory_units
ADD CONSTRAINT memory_units_fact_type_check
CHECK (fact_type IN ('world', 'experience', 'opinion', 'observation'))
""")
# Drop mental model columns from memory_units
op.execute(f"ALTER TABLE {schema}memory_units DROP COLUMN IF EXISTS proof_count")
op.execute(f"ALTER TABLE {schema}memory_units DROP COLUMN IF EXISTS source_memory_ids")
op.execute(f"ALTER TABLE {schema}memory_units DROP COLUMN IF EXISTS history")
op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_units_mental_models")
# Drop directives table
op.execute(f"DROP TABLE IF EXISTS {schema}directives CASCADE")
# Rename reflections back to pinned_reflections
op.execute(f"ALTER TABLE IF EXISTS {schema}reflections RENAME TO pinned_reflections")
# Restore indexes
op.execute(f"ALTER INDEX IF EXISTS {schema}idx_reflections_bank_id RENAME TO idx_pinned_reflections_bank_id")
op.execute(f"ALTER INDEX IF EXISTS {schema}idx_reflections_embedding RENAME TO idx_pinned_reflections_embedding")
op.execute(f"ALTER INDEX IF EXISTS {schema}idx_reflections_tags RENAME TO idx_pinned_reflections_tags")
op.execute(
f"ALTER INDEX IF EXISTS {schema}idx_reflections_text_search RENAME TO idx_pinned_reflections_text_search"
)
# Restore foreign key
op.execute(f"""
ALTER TABLE {schema}pinned_reflections
DROP CONSTRAINT IF EXISTS fk_reflections_bank_id
""")
op.execute(f"""
ALTER TABLE {schema}pinned_reflections
ADD CONSTRAINT fk_pinned_reflections_bank_id
FOREIGN KEY (bank_id) REFERENCES {schema}banks(bank_id) ON DELETE CASCADE
""")
# Re-create learnings table
op.execute(f"""
CREATE TABLE IF NOT EXISTS {schema}learnings (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
bank_id VARCHAR(64) NOT NULL,
text TEXT NOT NULL,
proof_count INT NOT NULL DEFAULT 1,
history JSONB DEFAULT '[]'::jsonb,
mission_context VARCHAR(64),
pre_mission_change BOOLEAN DEFAULT FALSE,
embedding vector(384),
tags VARCHAR[] DEFAULT ARRAY[]::VARCHAR[],
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
updated_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now()
)
""")
op.execute(f"""
ALTER TABLE {schema}learnings
ADD CONSTRAINT fk_learnings_bank_id
FOREIGN KEY (bank_id) REFERENCES {schema}banks(bank_id) ON DELETE CASCADE
""")
# Note: mental_models table recreation is complex and would need separate handling
@@ -1,50 +0,0 @@
"""fix_mental_model_fact_type
Revision ID: q2l3m4n5o6p7
Revises: p1k2l3m4n5o6
Create Date: 2026-01-21 13:30:00.000000
Fix the fact_type check constraint to include 'mental_model'.
This is a fix for p1k2l3m4n5o6 which should have included this change.
"""
from collections.abc import Sequence
from alembic import context, op
# revision identifiers, used by Alembic.
revision: str = "q2l3m4n5o6p7"
down_revision: str | Sequence[str] | None = "p1k2l3m4n5o6"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _get_schema_prefix() -> str:
"""Get schema prefix for table names (required for multi-tenant support)."""
schema = context.config.get_main_option("target_schema")
return f'"{schema}".' if schema else ""
def upgrade() -> None:
"""Add 'mental_model' to the fact_type check constraint."""
schema = _get_schema_prefix()
# Drop the old constraint and add the new one with mental_model included
op.execute(f"ALTER TABLE {schema}memory_units DROP CONSTRAINT IF EXISTS memory_units_fact_type_check")
op.execute(f"""
ALTER TABLE {schema}memory_units
ADD CONSTRAINT memory_units_fact_type_check
CHECK (fact_type IN ('world', 'experience', 'opinion', 'observation', 'mental_model'))
""")
def downgrade() -> None:
"""Remove 'mental_model' from the fact_type check constraint."""
schema = _get_schema_prefix()
op.execute(f"ALTER TABLE {schema}memory_units DROP CONSTRAINT IF EXISTS memory_units_fact_type_check")
op.execute(f"""
ALTER TABLE {schema}memory_units
ADD CONSTRAINT memory_units_fact_type_check
CHECK (fact_type IN ('world', 'experience', 'opinion', 'observation'))
""")
@@ -1,47 +0,0 @@
"""Add reflect_response JSONB column to reflections
Revision ID: r3m4n5o6p7q8
Revises: q2l3m4n5o6p7
Create Date: 2026-01-21
This migration adds a reflect_response JSONB column to store the full
reflect API response payload, including based_on facts and trace data.
Note: Table was renamed from pinned_reflections to reflections in p1k2l3m4n5o6.
"""
from collections.abc import Sequence
from alembic import context, op
revision: str = "r3m4n5o6p7q8"
down_revision: str | Sequence[str] | None = "q2l3m4n5o6p7"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _get_schema_prefix() -> str:
"""Get schema prefix for table names (required for multi-tenant support)."""
schema = context.config.get_main_option("target_schema")
return f'"{schema}".' if schema else ""
def upgrade() -> None:
"""Add reflect_response JSONB column to reflections."""
schema = _get_schema_prefix()
# Add reflect_response column to store the full reflect API response
op.execute(f"""
ALTER TABLE {schema}reflections
ADD COLUMN IF NOT EXISTS reflect_response JSONB
""")
def downgrade() -> None:
"""Remove reflect_response column from reflections."""
schema = _get_schema_prefix()
op.execute(f"""
ALTER TABLE {schema}reflections
DROP COLUMN IF EXISTS reflect_response
""")
@@ -1,53 +0,0 @@
"""Add consolidated_at column to memory_units for incremental consolidation tracking.
This allows consolidation to track progress at the memory level rather than
using a bank-level watermark. If consolidation crashes, already-processed
memories won't be reprocessed.
Revision ID: s4n5o6p7q8r9
Revises: r3m4n5o6p7q8
Create Date: 2025-01-22
"""
from collections.abc import Sequence
from alembic import context, op
revision: str = "s4n5o6p7q8r9"
down_revision: str | Sequence[str] | None = "r3m4n5o6p7q8"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _get_schema_prefix() -> str:
"""Get schema prefix for table names (required for multi-tenant support)."""
schema = context.config.get_main_option("target_schema")
return f'"{schema}".' if schema else ""
def upgrade() -> None:
schema = _get_schema_prefix()
# Add consolidated_at column to memory_units
op.execute(
f"""
ALTER TABLE {schema}memory_units
ADD COLUMN IF NOT EXISTS consolidated_at TIMESTAMPTZ DEFAULT NULL
"""
)
# Create index for efficient querying of unconsolidated memories
op.execute(
f"""
CREATE INDEX IF NOT EXISTS idx_memory_units_unconsolidated
ON {schema}memory_units (bank_id, created_at)
WHERE consolidated_at IS NULL AND fact_type IN ('experience', 'world')
"""
)
def downgrade() -> None:
schema = _get_schema_prefix()
op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_units_unconsolidated")
op.execute(f"ALTER TABLE {schema}memory_units DROP COLUMN IF EXISTS consolidated_at")
@@ -1,134 +0,0 @@
"""Rename mental_model fact_type to observation and reflections table to mental_models
Revision ID: t5o6p7q8r9s0
Revises: s4n5o6p7q8r9
Create Date: 2026-01-26
This migration implements the terminology rename:
1. mental_model (fact_type in memory_units) -> observation
2. reflections table -> mental_models table
The new terminology:
- Observations: Consolidated knowledge synthesized from facts (was mental_model)
- Mental Models: Stored reflect responses (was reflections)
"""
from collections.abc import Sequence
from alembic import context, op
revision: str = "t5o6p7q8r9s0"
down_revision: str | Sequence[str] | None = "s4n5o6p7q8r9"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _get_schema_prefix() -> str:
"""Get schema prefix for table names (required for multi-tenant support)."""
schema = context.config.get_main_option("target_schema")
return f'"{schema}".' if schema else ""
def upgrade() -> None:
"""Rename mental_model -> observation and reflections -> mental_models."""
schema = _get_schema_prefix()
# 1. Update fact_type values: mental_model -> observation
op.execute(f"""
UPDATE {schema}memory_units
SET fact_type = 'observation'
WHERE fact_type = 'mental_model'
""")
# 2. Update the CHECK constraint - remove mental_model, keep observation
op.execute(f"ALTER TABLE {schema}memory_units DROP CONSTRAINT IF EXISTS memory_units_fact_type_check")
op.execute(f"""
ALTER TABLE {schema}memory_units
ADD CONSTRAINT memory_units_fact_type_check
CHECK (fact_type IN ('world', 'experience', 'opinion', 'observation'))
""")
# 3. Rename the index for observations (was for mental_models)
op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_units_mental_models")
op.execute(f"""
CREATE INDEX IF NOT EXISTS idx_memory_units_observations
ON {schema}memory_units(bank_id, fact_type)
WHERE fact_type = 'observation'
""")
# 4. Update the unconsolidated index to not filter by fact_type since observations
# are now the consolidated type
op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_units_unconsolidated")
op.execute(f"""
CREATE INDEX IF NOT EXISTS idx_memory_units_unconsolidated
ON {schema}memory_units (bank_id, created_at)
WHERE consolidated_at IS NULL AND fact_type IN ('experience', 'world')
""")
# 5. Rename reflections table to mental_models
op.execute(f"ALTER TABLE IF EXISTS {schema}reflections RENAME TO mental_models")
# 6. Rename indexes for mental_models (was reflections)
op.execute(f"ALTER INDEX IF EXISTS {schema}idx_reflections_bank_id RENAME TO idx_mental_models_bank_id")
op.execute(f"ALTER INDEX IF EXISTS {schema}idx_reflections_embedding RENAME TO idx_mental_models_embedding")
op.execute(f"ALTER INDEX IF EXISTS {schema}idx_reflections_tags RENAME TO idx_mental_models_tags")
op.execute(f"ALTER INDEX IF EXISTS {schema}idx_reflections_text_search RENAME TO idx_mental_models_text_search")
# 7. Rename foreign key constraint
op.execute(f"""
ALTER TABLE {schema}mental_models
DROP CONSTRAINT IF EXISTS fk_reflections_bank_id
""")
op.execute(f"""
ALTER TABLE {schema}mental_models
ADD CONSTRAINT fk_mental_models_bank_id
FOREIGN KEY (bank_id) REFERENCES {schema}banks(bank_id) ON DELETE CASCADE
""")
def downgrade() -> None:
"""Reverse: observation -> mental_model and mental_models -> reflections."""
schema = _get_schema_prefix()
# 1. Rename mental_models table back to reflections
op.execute(f"ALTER TABLE IF EXISTS {schema}mental_models RENAME TO reflections")
# 2. Rename indexes back
op.execute(f"ALTER INDEX IF EXISTS {schema}idx_mental_models_bank_id RENAME TO idx_reflections_bank_id")
op.execute(f"ALTER INDEX IF EXISTS {schema}idx_mental_models_embedding RENAME TO idx_reflections_embedding")
op.execute(f"ALTER INDEX IF EXISTS {schema}idx_mental_models_tags RENAME TO idx_reflections_tags")
op.execute(f"ALTER INDEX IF EXISTS {schema}idx_mental_models_text_search RENAME TO idx_reflections_text_search")
# 3. Rename foreign key back
op.execute(f"""
ALTER TABLE {schema}reflections
DROP CONSTRAINT IF EXISTS fk_mental_models_bank_id
""")
op.execute(f"""
ALTER TABLE {schema}reflections
ADD CONSTRAINT fk_reflections_bank_id
FOREIGN KEY (bank_id) REFERENCES {schema}banks(bank_id) ON DELETE CASCADE
""")
# 4. Update fact_type values: observation -> mental_model
op.execute(f"""
UPDATE {schema}memory_units
SET fact_type = 'mental_model'
WHERE fact_type = 'observation'
""")
# 5. Update the CHECK constraint back
op.execute(f"ALTER TABLE {schema}memory_units DROP CONSTRAINT IF EXISTS memory_units_fact_type_check")
op.execute(f"""
ALTER TABLE {schema}memory_units
ADD CONSTRAINT memory_units_fact_type_check
CHECK (fact_type IN ('world', 'experience', 'opinion', 'observation', 'mental_model'))
""")
# 6. Rename index back
op.execute(f"DROP INDEX IF EXISTS {schema}idx_memory_units_observations")
op.execute(f"""
CREATE INDEX IF NOT EXISTS idx_memory_units_mental_models
ON {schema}memory_units(bank_id, fact_type)
WHERE fact_type = 'mental_model'
""")
@@ -1,41 +0,0 @@
"""Change mental_models.id from UUID to TEXT
Revision ID: u6p7q8r9s0t1
Revises: t5o6p7q8r9s0
Create Date: 2026-01-27
This migration changes the mental_models.id column from UUID to TEXT
to support user-defined text identifiers like 'team-communication' instead of UUIDs.
"""
from collections.abc import Sequence
from alembic import context, op
revision: str = "u6p7q8r9s0t1"
down_revision: str | Sequence[str] | None = "t5o6p7q8r9s0"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _get_schema_prefix() -> str:
"""Get schema prefix for table names (required for multi-tenant support)."""
schema = context.config.get_main_option("target_schema")
return f'"{schema}".' if schema else ""
def upgrade() -> None:
"""Change mental_models.id from UUID to TEXT."""
schema = _get_schema_prefix()
# Change the id column type from UUID to TEXT
# Existing UUIDs will be converted to their string representation
op.execute(f"ALTER TABLE {schema}mental_models ALTER COLUMN id TYPE TEXT USING id::TEXT")
def downgrade() -> None:
"""Revert mental_models.id from TEXT to UUID."""
schema = _get_schema_prefix()
# Note: This will fail if any id values are not valid UUIDs
op.execute(f"ALTER TABLE {schema}mental_models ALTER COLUMN id TYPE UUID USING id::UUID")
@@ -1,50 +0,0 @@
"""Add max_tokens and trigger columns to mental_models
Revision ID: v7q8r9s0t1u2
Revises: u6p7q8r9s0t1
Create Date: 2026-01-27
This migration adds:
- max_tokens column: token limit for content generation during refresh
- trigger column: JSONB for trigger settings (e.g., refresh_after_consolidation)
"""
from collections.abc import Sequence
from alembic import context, op
revision: str = "v7q8r9s0t1u2"
down_revision: str | Sequence[str] | None = "u6p7q8r9s0t1"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _get_schema_prefix() -> str:
"""Get schema prefix for table names (required for multi-tenant support)."""
schema = context.config.get_main_option("target_schema")
return f'"{schema}".' if schema else ""
def upgrade() -> None:
"""Add max_tokens and trigger columns to mental_models."""
schema = _get_schema_prefix()
op.execute(f"""
ALTER TABLE {schema}mental_models
ADD COLUMN IF NOT EXISTS max_tokens INT NOT NULL DEFAULT 2048
""")
# trigger column stores trigger settings as JSONB
# Default: refresh_after_consolidation = false (not "real time")
op.execute(f"""
ALTER TABLE {schema}mental_models
ADD COLUMN IF NOT EXISTS trigger JSONB NOT NULL DEFAULT '{{"refresh_after_consolidation": false}}'::jsonb
""")
def downgrade() -> None:
"""Remove max_tokens and trigger columns from mental_models."""
schema = _get_schema_prefix()
op.execute(f"ALTER TABLE {schema}mental_models DROP COLUMN IF EXISTS max_tokens")
op.execute(f"ALTER TABLE {schema}mental_models DROP COLUMN IF EXISTS trigger")
@@ -1,60 +0,0 @@
"""Fix mental_models primary key to be scoped per bank
Revision ID: w8r9s0t1u2v3
Revises: v7q8r9s0t1u2
Create Date: 2026-02-05
This migration fixes a critical bank isolation bug where mental_models.id was
globally unique across all banks instead of being scoped per bank. This caused
conflicts when different banks tried to use the same custom ID.
CRITICAL FIX: Changes primary key from (id) to (bank_id, id) to ensure proper isolation.
"""
from collections.abc import Sequence
from alembic import context, op
revision: str = "w8r9s0t1u2v3"
down_revision: str | Sequence[str] | None = "v7q8r9s0t1u2"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _get_schema_prefix() -> str:
"""Get schema prefix for table names (required for multi-tenant support)."""
schema = context.config.get_main_option("target_schema")
return f'"{schema}".' if schema else ""
def upgrade() -> None:
"""Change mental_models primary key from (id) to (bank_id, id) for proper bank isolation."""
schema = _get_schema_prefix()
# Drop the old primary key constraint (just id)
# Note: The constraint might be named differently on different DBs
# Try both old names (pinned_reflections_pkey from original, mental_models_pkey from rename)
op.execute(f"ALTER TABLE {schema}mental_models DROP CONSTRAINT IF EXISTS pinned_reflections_pkey")
op.execute(f"ALTER TABLE {schema}mental_models DROP CONSTRAINT IF EXISTS mental_models_pkey")
# Create the new composite primary key (bank_id, id)
# This ensures IDs are scoped per bank, not globally
op.execute(f"""
ALTER TABLE {schema}mental_models
ADD CONSTRAINT mental_models_pkey PRIMARY KEY (bank_id, id)
""")
def downgrade() -> None:
"""Revert mental_models primary key from (bank_id, id) to (id)."""
schema = _get_schema_prefix()
# Drop the composite primary key
op.execute(f"ALTER TABLE {schema}mental_models DROP CONSTRAINT IF EXISTS mental_models_pkey")
# Restore the old primary key (just id)
# WARNING: This downgrade will fail if there are duplicate IDs across banks
op.execute(f"""
ALTER TABLE {schema}mental_models
ADD CONSTRAINT mental_models_pkey PRIMARY KEY (id)
""")
@@ -1,64 +0,0 @@
"""Add config JSONB column to banks table for hierarchical configuration
Revision ID: x9s0t1u2v3w4
Revises: w8r9s0t1u2v3
Create Date: 2026-02-09
This migration adds a `config` JSONB column to the banks table to support
per-bank configuration overrides. This enables hierarchical configuration where:
- Global config is loaded from environment variables
- Tenant config is provided via TenantExtension
- Bank config overrides are stored in banks.config JSONB column
The config column stores overrides for hierarchical fields (LLM settings,
retention parameters, retrieval settings, etc.) in Python field name format.
"""
from collections.abc import Sequence
import sqlalchemy as sa
from alembic import context, op
from sqlalchemy.dialects.postgresql import JSONB
revision: str = "x9s0t1u2v3w4"
down_revision: str | Sequence[str] | None = "w8r9s0t1u2v3"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _get_schema_prefix() -> str:
"""Get schema prefix for table names (required for multi-tenant support)."""
schema = context.config.get_main_option("target_schema")
return f'"{schema}".' if schema else ""
def upgrade() -> None:
"""Add config JSONB column to banks table with GIN index."""
schema = _get_schema_prefix()
# Add config column to banks table
op.execute(f"""
ALTER TABLE {schema}banks
ADD COLUMN config JSONB NOT NULL DEFAULT '{{}}'::jsonb
""")
# Add GIN index for efficient JSONB queries
op.execute(f"""
CREATE INDEX idx_banks_config
ON {schema}banks
USING gin(config)
""")
def downgrade() -> None:
"""Remove config column and index from banks table."""
schema = _get_schema_prefix()
# Drop index first
op.execute(f"DROP INDEX IF EXISTS {schema}idx_banks_config")
# Drop column
op.execute(f"""
ALTER TABLE {schema}banks
DROP COLUMN IF EXISTS config
""")
@@ -1,49 +0,0 @@
"""Add GIN index on async_operations.result_metadata for parent_operation_id queries
Revision ID: y0t1u2v3w4x5
Revises: x9s0t1u2v3w4
Create Date: 2026-02-13
This migration adds a GIN index on the result_metadata JSONB column in the
async_operations table to support efficient queries for child operations by
parent_operation_id.
The index enables fast lookups when querying for child operations:
SELECT * FROM async_operations
WHERE result_metadata::jsonb @> '{"parent_operation_id": "uuid"}'::jsonb
"""
from collections.abc import Sequence
from alembic import context, op
revision: str = "y0t1u2v3w4x5"
down_revision: str | Sequence[str] | None = "x9s0t1u2v3w4"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _get_schema_prefix() -> str:
"""Get schema prefix for table names (required for multi-tenant support)."""
schema = context.config.get_main_option("target_schema")
return f'"{schema}".' if schema else ""
def upgrade() -> None:
"""Add GIN index on result_metadata for efficient parent_operation_id queries."""
schema = _get_schema_prefix()
# Add GIN index for JSONB containment queries (@> operator)
op.execute(f"""
CREATE INDEX idx_async_operations_result_metadata
ON {schema}async_operations
USING gin(result_metadata)
""")
def downgrade() -> None:
"""Remove GIN index on result_metadata."""
schema = _get_schema_prefix()
# Drop index
op.execute(f"DROP INDEX IF EXISTS {schema}idx_async_operations_result_metadata")
+19 -29
View File
@@ -6,6 +6,7 @@ Provides both HTTP REST API and MCP (Model Context Protocol) server.
import logging
from contextlib import asynccontextmanager
from typing import Optional
from fastapi import FastAPI
@@ -45,14 +46,14 @@ def create_app(
# Both HTTP and MCP
app = create_app(memory, mcp_api_enabled=True)
"""
mcp_servers = None
mcp_app = None
# Create MCP servers first if enabled (we need their lifespans for chaining)
# Create MCP app first if enabled (we need its lifespan for chaining)
if mcp_api_enabled:
try:
from .mcp import MCPMiddleware, create_mcp_servers
from .mcp import create_mcp_app
mcp_servers = create_mcp_servers(memory=memory)
mcp_app = create_mcp_app(memory=memory)
except ImportError as e:
logger.error(f"MCP server requested but dependencies not available: {e}")
logger.error("Install with: pip install hindsight-api[mcp]")
@@ -69,41 +70,30 @@ def create_app(
app = FastAPI(title="Hindsight API", version="0.0.7")
logger.info("HTTP REST API disabled")
# Add MCP middleware and chain its lifespan if enabled
if mcp_servers is not None:
multi_bank_server, single_bank_server, multi_bank_starlette_app, single_bank_starlette_app = mcp_servers
# Mount MCP server and chain its lifespan if enabled
if mcp_app is not None:
# Get the MCP app's underlying Starlette app for lifespan access
mcp_starlette_app = mcp_app.mcp_app
# Store the original lifespan
original_lifespan = app.router.lifespan_context
@asynccontextmanager
async def chained_lifespan(app_instance: FastAPI):
"""Chain both MCP lifespans with the main app lifespan."""
# Start both MCP lifespans (multi-bank and single-bank)
async with multi_bank_starlette_app.router.lifespan_context(multi_bank_starlette_app):
async with single_bank_starlette_app.router.lifespan_context(single_bank_starlette_app):
logger.info("MCP lifespans started (multi-bank and single-bank)")
# Then start the original app lifespan
async with original_lifespan(app_instance):
yield
logger.info("MCP lifespans stopped")
"""Chain the MCP lifespan with the main app lifespan."""
# Start MCP lifespan first
async with mcp_starlette_app.router.lifespan_context(mcp_starlette_app):
logger.info("MCP lifespan started")
# Then start the original app lifespan
async with original_lifespan(app_instance):
yield
logger.info("MCP lifespan stopped")
# Replace the app's lifespan with the chained version
app.router.lifespan_context = chained_lifespan
# Add MCP as a wrapping middleware — intercepts /mcp* requests directly,
# passes everything else through to the FastAPI app. No Starlette Mount
# means no 307 redirect for /mcp (no trailing slash).
app.add_middleware(
MCPMiddleware,
memory=memory,
prefix=mcp_mount_path,
multi_bank_app=multi_bank_starlette_app,
single_bank_app=single_bank_starlette_app,
multi_bank_server=multi_bank_server,
single_bank_server=single_bank_server,
)
# Mount the MCP middleware
app.mount(mcp_mount_path, mcp_app)
logger.info(f"MCP server enabled at {mcp_mount_path}/")
return app
File diff suppressed because it is too large Load Diff
+183 -254
View File
@@ -1,4 +1,4 @@
"""Hindsight MCP Server implementation using FastMCP (HTTP transport)."""
"""Hindsight MCP Server implementation using FastMCP."""
import json
import logging
@@ -8,10 +8,7 @@ from contextvars import ContextVar
from fastmcp import FastMCP
from hindsight_api import MemoryEngine
from hindsight_api.engine.memory_engine import _current_schema
from hindsight_api.extensions import MCPExtension, load_extension
from hindsight_api.extensions.tenant import AuthenticationError
from hindsight_api.mcp_tools import MCPToolsConfig, register_mcp_tools
from hindsight_api.engine.response_models import VALID_RECALL_FACT_TYPES
from hindsight_api.models import RequestContext
# Configure logging from HINDSIGHT_API_LOG_LEVEL environment variable
@@ -33,49 +30,21 @@ logger = logging.getLogger(__name__)
# Default bank_id from environment variable
DEFAULT_BANK_ID = os.environ.get("HINDSIGHT_MCP_BANK_ID", "default")
# Legacy MCP authentication token (for backwards compatibility)
# If set, this token is checked first before TenantExtension auth
MCP_AUTH_TOKEN = os.environ.get("HINDSIGHT_API_MCP_AUTH_TOKEN")
# Context variable to hold the current bank_id
_current_bank_id: ContextVar[str | None] = ContextVar("current_bank_id", default=None)
# Context variable to hold the current API key (for tenant auth propagation)
_current_api_key: ContextVar[str | None] = ContextVar("current_api_key", default=None)
# Context variables for tenant_id and api_key_id (set by authenticate, used by usage metering)
_current_tenant_id: ContextVar[str | None] = ContextVar("current_tenant_id", default=None)
_current_api_key_id: ContextVar[str | None] = ContextVar("current_api_key_id", default=None)
def get_current_bank_id() -> str | None:
"""Get the current bank_id from context."""
return _current_bank_id.get()
def get_current_api_key() -> str | None:
"""Get the current API key from context."""
return _current_api_key.get()
def get_current_tenant_id() -> str | None:
"""Get the current tenant_id from context."""
return _current_tenant_id.get()
def get_current_api_key_id() -> str | None:
"""Get the current api_key_id from context."""
return _current_api_key_id.get()
def create_mcp_server(memory: MemoryEngine, multi_bank: bool = True) -> FastMCP:
def create_mcp_server(memory: MemoryEngine) -> FastMCP:
"""
Create and configure the Hindsight MCP server.
Args:
memory: MemoryEngine instance (required)
multi_bank: If True, expose all tools with bank_id parameters (default).
If False, only expose bank-scoped tools without bank_id parameters.
Returns:
Configured FastMCP server instance with stateless_http enabled
@@ -83,137 +52,164 @@ def create_mcp_server(memory: MemoryEngine, multi_bank: bool = True) -> FastMCP:
# Use stateless_http=True for Claude Code compatibility
mcp = FastMCP("hindsight-mcp-server", stateless_http=True)
# Configure and register tools using shared module
config = MCPToolsConfig(
bank_id_resolver=get_current_bank_id,
api_key_resolver=get_current_api_key, # Propagate API key for tenant auth
tenant_id_resolver=get_current_tenant_id, # Propagate tenant_id for usage metering
api_key_id_resolver=get_current_api_key_id, # Propagate api_key_id for usage metering
include_bank_id_param=multi_bank,
tools=None
if multi_bank
else {
"retain",
"recall",
"reflect",
"list_mental_models",
"get_mental_model",
"create_mental_model",
"update_mental_model",
"delete_mental_model",
"refresh_mental_model",
}, # Scoped tools for single-bank mode (excludes bank management: list_banks, create_bank)
retain_fire_and_forget=False, # HTTP MCP supports sync/async modes
)
@mcp.tool()
async def retain(
content: str,
context: str = "general",
async_processing: bool = True,
bank_id: str | None = None,
) -> str:
"""
Store important information to long-term memory.
register_mcp_tools(mcp, memory, config)
Use this tool PROACTIVELY whenever the user shares:
- Personal facts, preferences, or interests
- Important events or milestones
- User history, experiences, or background
- Decisions, opinions, or stated preferences
- Goals, plans, or future intentions
- Relationships or people mentioned
- Work context, projects, or responsibilities
# Load and register additional tools from MCP extension if configured
mcp_extension = load_extension("MCP", MCPExtension)
if mcp_extension:
logger.info(f"Loading MCP extension: {mcp_extension.__class__.__name__}")
mcp_extension.register_tools(mcp, memory)
Args:
content: The fact/memory to store (be specific and include relevant details)
context: Category for the memory (e.g., 'preferences', 'work', 'hobbies', 'family'). Default: 'general'
async_processing: If True, queue for background processing and return immediately. If False, wait for completion. Default: True
bank_id: Optional bank to store in (defaults to session bank). Use for cross-bank operations.
"""
try:
target_bank = bank_id or get_current_bank_id()
if target_bank is None:
return "Error: No bank_id configured"
contents = [{"content": content, "context": context}]
if async_processing:
# Queue for background processing and return immediately
result = await memory.submit_async_retain(
bank_id=target_bank, contents=contents, request_context=RequestContext()
)
return f"Memory queued for background processing (operation_id: {result.get('operation_id', 'N/A')})"
else:
# Wait for completion
await memory.retain_batch_async(
bank_id=target_bank,
contents=contents,
request_context=RequestContext(),
)
return f"Memory stored successfully in bank '{target_bank}'"
except Exception as e:
logger.error(f"Error storing memory: {e}", exc_info=True)
return f"Error: {str(e)}"
# Make all tools tolerant of extra arguments from LLMs (e.g., "explanation")
_make_tools_tolerant(mcp)
@mcp.tool()
async def recall(query: str, max_tokens: int = 4096, bank_id: str | None = None) -> str:
"""
Search memories to provide personalized, context-aware responses.
Use this tool PROACTIVELY to:
- Check user's preferences before making suggestions
- Recall user's history to provide continuity
- Remember user's goals and context
- Personalize responses based on past interactions
Args:
query: Natural language search query (e.g., "user's food preferences", "what projects is user working on")
max_tokens: Maximum tokens in the response (default: 4096)
bank_id: Optional bank to search in (defaults to session bank). Use for cross-bank operations.
"""
try:
target_bank = bank_id or get_current_bank_id()
if target_bank is None:
return "Error: No bank_id configured"
from hindsight_api.engine.memory_engine import Budget
recall_result = await memory.recall_async(
bank_id=target_bank,
query=query,
fact_type=list(VALID_RECALL_FACT_TYPES),
budget=Budget.HIGH,
max_tokens=max_tokens,
request_context=RequestContext(),
)
# Use model's JSON serialization
return recall_result.model_dump_json(indent=2)
except Exception as e:
logger.error(f"Error searching: {e}", exc_info=True)
return f'{{"error": "{e}", "results": []}}'
@mcp.tool()
async def reflect(query: str, context: str | None = None, budget: str = "low", bank_id: str | None = None) -> str:
"""
Generate thoughtful analysis by synthesizing stored memories with the bank's personality.
WHEN TO USE THIS TOOL:
Use reflect when you need reasoned analysis, not just fact retrieval. This tool
thinks through the question using everything the bank knows and its personality traits.
EXAMPLES OF GOOD QUERIES:
- "What patterns have emerged in how I approach debugging?"
- "Based on my past decisions, what architectural style do I prefer?"
- "What might be the best approach for this problem given what you know about me?"
- "How should I prioritize these tasks based on my goals?"
HOW IT DIFFERS FROM RECALL:
- recall: Returns raw facts matching your search (fast lookup)
- reflect: Reasons across memories to form a synthesized answer (deeper analysis)
Use recall for "what did I say about X?" and reflect for "what should I do about X?"
Args:
query: The question or topic to reflect on
context: Optional context about why this reflection is needed
budget: Search budget - 'low', 'mid', or 'high' (default: 'low')
bank_id: Optional bank to reflect in (defaults to session bank). Use for cross-bank operations.
"""
try:
target_bank = bank_id or get_current_bank_id()
if target_bank is None:
return "Error: No bank_id configured"
from hindsight_api.engine.memory_engine import Budget
# Map string budget to enum
budget_map = {"low": Budget.LOW, "mid": Budget.MID, "high": Budget.HIGH}
budget_enum = budget_map.get(budget.lower(), Budget.LOW)
reflect_result = await memory.reflect_async(
bank_id=target_bank,
query=query,
budget=budget_enum,
context=context,
request_context=RequestContext(),
)
return reflect_result.model_dump_json(indent=2)
except Exception as e:
logger.error(f"Error reflecting: {e}", exc_info=True)
return f'{{"error": "{e}", "text": ""}}'
return mcp
def _make_tools_tolerant(mcp: FastMCP) -> None:
"""Wrap all tool run methods to strip unknown arguments before validation.
LLMs frequently add extra fields like "explanation" or "reasoning" to tool calls.
FastMCP's Pydantic TypeAdapter rejects these with "Unexpected keyword argument".
This wraps each tool's run() to filter arguments to only known parameters.
"""
try:
for name, tool in mcp._tool_manager._tools.items():
if hasattr(tool, "parameters") and tool.parameters:
allowed = set(tool.parameters.get("properties", {}).keys())
original_run = tool.run
async def _tolerant_run(arguments, _allowed=allowed, _orig=original_run):
extra_keys = set(arguments.keys()) - _allowed
if extra_keys:
logger.debug(f"Stripping unknown arguments from tool call: {extra_keys}")
arguments = {k: v for k, v in arguments.items() if k in _allowed}
return await _orig(arguments)
# FunctionTool is a Pydantic model with extra='forbid', so use
# object.__setattr__ to bypass Pydantic's setter validation.
object.__setattr__(tool, "run", _tolerant_run)
except (AttributeError, KeyError) as e:
logger.warning(f"Could not make tools tolerant of extra arguments: {e}")
class MCPMiddleware:
"""ASGI middleware that intercepts MCP requests and routes to appropriate MCP server.
"""ASGI middleware that extracts bank_id from header or path and sets context.
This middleware wraps the main FastAPI app and intercepts requests matching the
configured prefix (default: /mcp). Non-MCP requests pass through to the inner app.
Bank ID can be provided via:
1. X-Bank-Id header (recommended for Claude Code)
2. URL path: /mcp/{bank_id}/
3. Environment variable HINDSIGHT_MCP_BANK_ID (fallback default)
Authentication:
1. If HINDSIGHT_API_MCP_AUTH_TOKEN is set (legacy), validates against that token
2. Otherwise, uses TenantExtension.authenticate_mcp() from the MemoryEngine
- DefaultTenantExtension: no auth required (local dev)
- ApiKeyTenantExtension: validates against env var
Two modes based on URL structure:
1. Multi-bank mode (for /mcp/ root endpoint):
- Exposes all tools: retain, recall, reflect, list_banks, create_bank
- All tools include optional bank_id parameter for cross-bank operations
- Bank ID from: X-Bank-Id header or HINDSIGHT_MCP_BANK_ID env var
2. Single-bank mode (for /mcp/{bank_id}/ endpoints):
- Exposes bank-scoped tools only: retain, recall, reflect
- No bank_id parameter (comes from URL)
- No bank management tools (list_banks, create_bank)
- Recommended for agent isolation
Bank ID resolution priority:
1. URL path (e.g., /mcp/{bank_id}/) → single-bank mode
2. X-Bank-Id header → multi-bank mode
3. HINDSIGHT_MCP_BANK_ID env var → multi-bank mode (default: "default")
Examples:
# Single-bank mode (recommended for agent isolation)
claude mcp add --transport http my-agent http://localhost:8888/mcp/my-agent-bank/ \\
--header "Authorization: Bearer <token>"
# Multi-bank mode (for cross-bank operations)
For Claude Code, configure with:
claude mcp add --transport http hindsight http://localhost:8888/mcp \\
--header "X-Bank-Id: my-bank" --header "Authorization: Bearer <token>"
--header "X-Bank-Id: my-bank"
"""
def __init__(
self,
app,
memory: MemoryEngine,
prefix: str = "/mcp",
multi_bank_app=None,
single_bank_app=None,
multi_bank_server=None,
single_bank_server=None,
):
def __init__(self, app, memory: MemoryEngine):
self.app = app
self.prefix = prefix
self.memory = memory
self.tenant_extension = memory._tenant_extension
if multi_bank_app and single_bank_app:
# Pre-created servers (used when called via add_middleware from create_app)
self.multi_bank_app = multi_bank_app
self.single_bank_app = single_bank_app
self.multi_bank_server = multi_bank_server
self.single_bank_server = single_bank_server
else:
# Create servers internally (for direct construction / tests)
self.multi_bank_server = create_mcp_server(memory, multi_bank=True)
self.multi_bank_app = self.multi_bank_server.http_app(path="/")
self.single_bank_server = create_mcp_server(memory, multi_bank=False)
self.single_bank_app = self.single_bank_server.http_app(path="/")
self.mcp_server = create_mcp_server(memory)
self.mcp_app = self.mcp_server.http_app(path="/")
# Expose the lifespan for the parent app to chain
self.lifespan = self.mcp_app.lifespan_handler if hasattr(self.mcp_app, "lifespan_handler") else None
def _get_header(self, scope: dict, name: str) -> str | None:
"""Extract a header value from ASGI scope."""
@@ -225,113 +221,54 @@ class MCPMiddleware:
async def __call__(self, scope, receive, send):
if scope["type"] != "http":
await self.app(scope, receive, send)
await self.mcp_app(scope, receive, send)
return
path = scope.get("path", "")
# Check if this is an MCP request (matches prefix)
if not (path == self.prefix or path.startswith(self.prefix + "/")):
# Not an MCP request — pass through to the inner app
await self.app(scope, receive, send)
return
# Strip any mount prefix (e.g., /mcp) that FastAPI might not have stripped
root_path = scope.get("root_path", "")
if root_path and path.startswith(root_path):
path = path[len(root_path) :] or "/"
# Strip prefix from path
path = path[len(self.prefix) :] or "/"
# Also handle case where mount path wasn't stripped (e.g., /mcp/...)
if path.startswith("/mcp/"):
path = path[4:] # Remove /mcp prefix
elif path == "/mcp":
path = "/"
# Extract auth token from header (for tenant auth propagation)
auth_header = self._get_header(scope, "Authorization")
auth_token: str | None = None
if auth_header:
# Support both "Bearer <token>" and direct token
auth_token = auth_header[7:].strip() if auth_header.startswith("Bearer ") else auth_header.strip()
# Try to get bank_id from header first (for Claude Code compatibility)
bank_id = self._get_header(scope, "X-Bank-Id")
# Authenticate: check legacy MCP_AUTH_TOKEN first, then TenantExtension
tenant_context = None
auth_tenant_id: str | None = None
auth_api_key_id: str | None = None
if MCP_AUTH_TOKEN:
# Legacy authentication mode - validate against static token
if not auth_token:
await self._send_error(send, 401, "Authorization header required")
return
if auth_token != MCP_AUTH_TOKEN:
await self._send_error(send, 401, "Invalid authentication token")
return
# Legacy mode doesn't use tenant schemas
tenant_context = None
else:
# Use TenantExtension.authenticate_mcp() for auth
try:
auth_context = RequestContext(api_key=auth_token)
tenant_context = await self.tenant_extension.authenticate_mcp(auth_context)
# Capture tenant_id and api_key_id set by authenticate() for usage metering
auth_tenant_id = auth_context.tenant_id
auth_api_key_id = auth_context.api_key_id
except AuthenticationError as e:
await self._send_error(send, 401, str(e))
return
# MCP endpoint paths that should not be treated as bank_ids
MCP_ENDPOINTS = {"sse", "messages"}
# Set schema from tenant context so downstream DB queries use the correct schema
schema_token = (
_current_schema.set(tenant_context.schema_name) if tenant_context and tenant_context.schema_name else None
)
# Resolve bank_id: path takes priority over header.
# Path = user's explicit connection endpoint (e.g., /mcp/my-bank/).
# X-Bank-Id header = per-request override for multi-bank mode only.
bank_id = None
bank_id_from_path = False
# If no header, try to extract from path: /{bank_id}/...
new_path = path
# First, try to extract from path: /{bank_id}/...
if path.startswith("/") and len(path) > 1:
if not bank_id and path.startswith("/") and len(path) > 1:
parts = path[1:].split("/", 1)
if parts[0]:
# Don't treat MCP endpoints as bank_ids
if parts[0] and parts[0] not in MCP_ENDPOINTS:
# First segment looks like a bank_id
bank_id = parts[0]
bank_id_from_path = True
new_path = "/" + parts[1] if len(parts) > 1 else "/"
# If no path-based bank_id, try X-Bank-Id header (multi-bank mode)
if not bank_id:
bank_id = self._get_header(scope, "X-Bank-Id")
# Fall back to default bank_id
if not bank_id:
bank_id = DEFAULT_BANK_ID
logger.debug(f"Using default bank_id: {bank_id}")
# Select the appropriate MCP app based on how bank_id was provided:
# - Path-based bank_id → single-bank app (no bank_id param, scoped tools)
# - Header/env bank_id → multi-bank app (bank_id param, all tools)
target_app = self.single_bank_app if bank_id_from_path else self.multi_bank_app
# Set bank_id, api_key, tenant_id, and api_key_id context
bank_id_token = _current_bank_id.set(bank_id)
# Store the auth token for tenant extension to validate
api_key_token = _current_api_key.set(auth_token) if auth_token else None
# Store tenant_id and api_key_id from authentication for usage metering
tenant_id_token = _current_tenant_id.set(auth_tenant_id) if auth_tenant_id else None
api_key_id_token = _current_api_key_id.set(auth_api_key_id) if auth_api_key_id else None
# Set bank_id context
token = _current_bank_id.set(bank_id)
try:
new_scope = scope.copy()
new_scope["path"] = new_path
# Clear root_path since we're passing directly to the app
new_scope["root_path"] = ""
# Wrap send to rewrite the SSE endpoint URL to include bank_id if using path-based routing.
# Only rewrite SSE (text/event-stream) responses to avoid corrupting tool results
# that might contain the literal string "data: /messages".
is_sse_response = False
# Wrap send to rewrite the SSE endpoint URL to include bank_id if using path-based routing
async def send_wrapper(message):
nonlocal is_sse_response
if message["type"] == "http.response.start":
for header_name, header_value in message.get("headers", []):
if header_name == b"content-type" and b"text/event-stream" in header_value:
is_sse_response = True
break
if message["type"] == "http.response.body" and bank_id_from_path and is_sse_response:
if message["type"] == "http.response.body":
body = message.get("body", b"")
if body and b"/messages" in body:
# Rewrite /messages to /{bank_id}/messages in SSE endpoint event
@@ -339,17 +276,9 @@ class MCPMiddleware:
message = {**message, "body": body}
await send(message)
await target_app(new_scope, receive, send_wrapper)
await self.mcp_app(new_scope, receive, send_wrapper)
finally:
_current_bank_id.reset(bank_id_token)
if api_key_token is not None:
_current_api_key.reset(api_key_token)
if tenant_id_token is not None:
_current_tenant_id.reset(tenant_id_token)
if api_key_id_token is not None:
_current_api_key_id.reset(api_key_id_token)
if schema_token is not None:
_current_schema.reset(schema_token)
_current_bank_id.reset(token)
async def _send_error(self, send, status: int, message: str):
"""Send an error response."""
@@ -369,19 +298,19 @@ class MCPMiddleware:
)
def create_mcp_servers(memory: MemoryEngine):
"""Create multi-bank and single-bank MCP servers and their Starlette apps.
def create_mcp_app(memory: MemoryEngine):
"""
Create an ASGI app that handles MCP requests.
Returns the servers and apps separately so lifespans can be chained before
the middleware wraps the main app.
Bank ID can be provided via:
1. X-Bank-Id header: claude mcp add --transport http hindsight http://localhost:8888/mcp --header "X-Bank-Id: my-bank"
2. URL path: /mcp/{bank_id}/
3. Environment variable HINDSIGHT_MCP_BANK_ID (fallback, default: "default")
Args:
memory: MemoryEngine instance
Returns:
Tuple of (multi_bank_server, single_bank_server, multi_bank_app, single_bank_app)
ASGI application
"""
multi_bank_server = create_mcp_server(memory, multi_bank=True)
multi_bank_app = multi_bank_server.http_app(path="/")
single_bank_server = create_mcp_server(memory, multi_bank=False)
single_bank_app = single_bank_server.http_app(path="/")
return multi_bank_server, single_bank_server, multi_bank_app, single_bank_app
return MCPMiddleware(None, memory)
+1 -10
View File
@@ -4,8 +4,6 @@ Banner display for Hindsight API startup.
Shows the logo and tagline with gradient colors.
"""
from .utils import mask_network_location
# Gradient colors: #0074d9 -> #009296
GRADIENT_START = (0, 116, 217) # #0074d9
GRADIENT_END = (0, 146, 150) # #009296
@@ -85,21 +83,14 @@ def print_startup_info(
embeddings_provider: str,
reranker_provider: str,
mcp_enabled: bool = False,
version: str | None = None,
vector_extension: str | None = None,
text_search_extension: str | None = None,
):
"""Print styled startup information."""
print(color_start("Starting Hindsight API..."))
if version:
print(f" {dim('Version:')} {color(f'v{version}', 0.1)}")
print(f" {dim('URL:')} {color(f'http://{host}:{port}', 0.2)}")
print(f" {dim('Database:')} {color(mask_network_location(database_url), 0.4)}")
print(f" {dim('Database:')} {color(database_url, 0.4)}")
print(f" {dim('LLM:')} {color(f'{llm_provider} / {llm_model}', 0.6)}")
print(f" {dim('Embeddings:')} {color(embeddings_provider, 0.8)}")
print(f" {dim('Reranker:')} {color(reranker_provider, 1.0)}")
extensions = f"{vector_extension or 'default'} (vector) / {text_search_extension or 'default'} (text)"
print(f" {dim('Extensions:')} {color(extensions, 0.4)}")
if mcp_enabled:
print(f" {dim('MCP:')} {color_end('enabled at /mcp')}")
print()
File diff suppressed because it is too large Load Diff
@@ -1,274 +0,0 @@
"""
Configuration resolution with hierarchical overrides.
Resolves config values through the hierarchy:
Global (env vars) → Tenant config (via extension) → Bank config (database)
Config values are resolved on every request to ensure consistency across
multiple API servers.
"""
import json
import logging
from dataclasses import asdict
from typing import Any
import asyncpg
from hindsight_api.config import HindsightConfig, _get_raw_config, normalize_config_dict
from hindsight_api.extensions.tenant import TenantExtension
from hindsight_api.models import RequestContext
logger = logging.getLogger(__name__)
class ConfigResolver:
"""Resolves hierarchical configuration with tenant/bank overrides."""
def __init__(self, pool: asyncpg.Pool, tenant_extension: TenantExtension | None = None):
"""
Initialize config resolver.
Args:
pool: Database connection pool
tenant_extension: Optional tenant extension for tenant-level config and permissions
"""
self.pool = pool
self.tenant_extension = tenant_extension
self._global_config = _get_raw_config()
self._configurable_fields = HindsightConfig.get_configurable_fields()
self._credential_fields = HindsightConfig.get_credential_fields()
async def resolve_full_config(self, bank_id: str, context: RequestContext | None = None) -> HindsightConfig:
"""
Resolve full HindsightConfig for a bank with hierarchical overrides applied.
This is for INTERNAL USE ONLY. Returns the complete config object with all fields
including credentials and static fields. Use get_bank_config() for API responses.
Resolution order:
1. Global config (from environment variables)
2. Tenant config overrides (from TenantExtension.get_tenant_config())
3. Bank config overrides (from banks.config JSONB)
Args:
bank_id: Bank identifier
context: Request context for tenant config resolution
Returns:
Complete HindsightConfig with hierarchical overrides applied
"""
# Start with global config (all fields)
config_dict = asdict(self._global_config)
# Load tenant config overrides (if tenant extension available)
if self.tenant_extension and context:
try:
tenant_overrides = await self.tenant_extension.get_tenant_config(context)
if tenant_overrides:
# Normalize keys and filter to configurable fields only
normalized_tenant = normalize_config_dict(tenant_overrides)
configurable_tenant = {k: v for k, v in normalized_tenant.items() if k in self._configurable_fields}
config_dict.update(configurable_tenant)
logger.debug(
f"Applied tenant config overrides for bank {bank_id}: {list(configurable_tenant.keys())}"
)
except Exception as e:
logger.warning(f"Failed to load tenant config for bank {bank_id}: {e}")
# Load bank config overrides
bank_overrides = await self._load_bank_config(bank_id)
if bank_overrides:
config_dict.update(bank_overrides)
logger.debug(f"Applied bank config overrides for bank {bank_id}: {list(bank_overrides.keys())}")
# Return full config object (dataclass doesn't have __init__ that accepts kwargs, so we update the object)
# Create a new config instance by copying the global config and updating fields
resolved_config = HindsightConfig(**config_dict)
return resolved_config
async def get_bank_config(self, bank_id: str, context: RequestContext | None = None) -> dict[str, Any]:
"""
Get fully resolved config for a bank (filtered by permissions).
Resolution order:
1. Global config (from environment variables)
2. Tenant config overrides (from TenantExtension.get_tenant_config())
3. Bank config overrides (from banks.config JSONB)
Note: Config is resolved on every call (not cached) to ensure consistency
across multiple API servers.
SECURITY:
- Only returns configurable fields (excludes static/infrastructure fields)
- Filters out ALL credential fields (API keys, base URLs, etc.)
- Further filtered by tenant/bank permissions if extension provides them
Args:
bank_id: Bank identifier
context: Request context for tenant config resolution and permissions
Returns:
Dict of allowed configurable fields only (never includes credentials or static fields)
"""
# Resolve full config with all hierarchical overrides
resolved_config = await self.resolve_full_config(bank_id, context)
config_dict = asdict(resolved_config)
# SECURITY: Filter to only configurable fields (exclude static/infrastructure)
filtered = {k: v for k, v in config_dict.items() if k in self._configurable_fields}
# SECURITY: Remove ALL credential fields (API keys, base URLs, etc.)
filtered = {k: v for k, v in filtered.items() if k not in self._credential_fields}
# PERMISSIONS: Further filter based on tenant/bank permissions
if self.tenant_extension and context:
try:
allowed_fields = await self.tenant_extension.get_allowed_config_fields(context, bank_id)
if allowed_fields is not None: # None means "allow all"
filtered = {k: v for k, v in filtered.items() if k in allowed_fields}
logger.debug(
f"Applied permission filter for bank {bank_id}: allowed={len(allowed_fields)} fields, "
f"returned={len(filtered)} fields"
)
except Exception as e:
logger.warning(f"Failed to load permissions for bank {bank_id}: {e}")
return filtered
async def _load_bank_config(self, bank_id: str) -> dict[str, Any]:
"""
Load bank config overrides from banks.config JSONB column.
Args:
bank_id: Bank identifier
Returns:
Dict of config overrides (only configurable fields, normalized keys)
"""
try:
async with self.pool.acquire() as conn:
row = await conn.fetchrow(
"""
SELECT config FROM banks WHERE bank_id = $1
""",
bank_id,
)
if row and row["config"]:
config_data = row["config"]
# Handle case where JSONB is returned as JSON string
if isinstance(config_data, str):
config_data = json.loads(config_data)
# Normalize keys (handle both env var format and Python field format)
normalized = normalize_config_dict(config_data)
# Only return overrides for configurable fields
return {k: v for k, v in normalized.items() if k in self._configurable_fields}
except Exception as e:
logger.error(f"Failed to load bank config for {bank_id}: {e}")
return {}
async def update_bank_config(
self, bank_id: str, updates: dict[str, Any], context: RequestContext | None = None
) -> None:
"""
Update bank configuration overrides (with permission checking).
Args:
bank_id: Bank identifier
updates: Dict of config field names to new values.
Keys can be in env var format (HINDSIGHT_API_LLM_PROVIDER)
or Python field format (llm_provider).
Only configurable fields are allowed.
context: Request context for permission checking
Raises:
ValueError: If attempting to override invalid/disallowed fields
"""
# Normalize keys
normalized_updates = normalize_config_dict(updates)
# SECURITY: Reject credential fields explicitly
credential_attempts = set(normalized_updates.keys()) & self._credential_fields
if credential_attempts:
raise ValueError(
f"Cannot set credential fields via API: {sorted(credential_attempts)}. "
f"Credentials (API keys, base URLs) must be set at server level only."
)
# Validate all fields are configurable
invalid_fields = set(normalized_updates.keys()) - self._configurable_fields
if invalid_fields:
static_fields = HindsightConfig.get_static_fields()
invalid_static = invalid_fields & static_fields
if invalid_static:
raise ValueError(
f"Cannot override static (server-level) fields: {sorted(invalid_static)}. "
f"Only configurable fields can be overridden per-bank. "
f"Configurable fields include: {sorted(list(self._configurable_fields)[:10])}... "
f"(total: {len(self._configurable_fields)} fields)"
)
else:
raise ValueError(
f"Unknown configuration fields: {sorted(invalid_fields)}. "
f"Valid configurable fields: {sorted(list(self._configurable_fields)[:10])}..."
)
# PERMISSIONS: Check tenant/bank permissions
if self.tenant_extension and context:
try:
allowed_fields = await self.tenant_extension.get_allowed_config_fields(context, bank_id)
if allowed_fields is not None: # None means "allow all"
disallowed = set(normalized_updates.keys()) - allowed_fields
if disallowed:
raise ValueError(
f"Not allowed to modify fields: {sorted(disallowed)}. "
f"Your permissions allow: {sorted(list(allowed_fields)[:10])}..."
if allowed_fields
else "Not allowed to modify fields: {sorted(disallowed)}. "
"Your permissions do not allow any config modifications."
)
except ValueError:
raise # Re-raise permission errors
except Exception as e:
logger.warning(f"Failed to check permissions for bank {bank_id}: {e}")
# Continue without permission check (fail open for backward compatibility)
# Merge with existing config (JSONB || operator)
async with self.pool.acquire() as conn:
await conn.execute(
"""
UPDATE banks
SET config = config || $1::jsonb,
updated_at = now()
WHERE bank_id = $2
""",
json.dumps(normalized_updates),
bank_id,
)
logger.info(f"Updated bank config for {bank_id}: {list(normalized_updates.keys())}")
async def reset_bank_config(self, bank_id: str) -> None:
"""
Reset bank configuration to defaults (remove all overrides).
Args:
bank_id: Bank identifier
"""
async with self.pool.acquire() as conn:
await conn.execute(
"""
UPDATE banks
SET config = '{}'::jsonb,
updated_at = now()
WHERE bank_id = $1
""",
bank_id,
)
logger.info(f"Reset bank config for {bank_id} to defaults")
+111 -20
View File
@@ -1,10 +1,11 @@
"""
Daemon mode support for Hindsight API.
Provides idle timeout for running as a background daemon.
Provides idle timeout and lockfile management for running as a background daemon.
"""
import asyncio
import fcntl
import logging
import os
import sys
@@ -14,11 +15,10 @@ from pathlib import Path
logger = logging.getLogger(__name__)
# Default daemon configuration
DEFAULT_DAEMON_PORT = 8888
DEFAULT_DAEMON_PORT = 8889
DEFAULT_IDLE_TIMEOUT = 0 # 0 = no auto-exit (hindsight-embed passes its own timeout)
# Allow override via environment variable for profile-specific logs
DAEMON_LOG_PATH = Path(os.getenv("HINDSIGHT_API_DAEMON_LOG", str(Path.home() / ".hindsight" / "daemon.log")))
LOCKFILE_PATH = Path.home() / ".hindsight" / "daemon.lock"
DAEMON_LOG_PATH = Path.home() / ".hindsight" / "daemon.log"
class IdleTimeoutMiddleware:
@@ -52,10 +52,82 @@ class IdleTimeoutMiddleware:
logger.info(f"Idle timeout reached ({self.idle_timeout}s), shutting down daemon")
# Give a moment for any in-flight requests
await asyncio.sleep(1)
# Send SIGTERM to ourselves to trigger graceful shutdown
import signal
os._exit(0)
os.kill(os.getpid(), signal.SIGTERM)
class DaemonLock:
"""
File-based lock to prevent multiple daemon instances.
Uses fcntl.flock for atomic locking on Unix systems.
"""
def __init__(self, lockfile: Path = LOCKFILE_PATH):
self.lockfile = lockfile
self._fd = None
def acquire(self) -> bool:
"""
Try to acquire the daemon lock.
Returns True if lock acquired, False if another daemon is running.
"""
self.lockfile.parent.mkdir(parents=True, exist_ok=True)
try:
self._fd = open(self.lockfile, "w")
fcntl.flock(self._fd.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
# Write PID for debugging
self._fd.write(str(os.getpid()))
self._fd.flush()
return True
except (IOError, OSError):
# Lock is held by another process
if self._fd:
self._fd.close()
self._fd = None
return False
def release(self):
"""Release the daemon lock."""
if self._fd:
try:
fcntl.flock(self._fd.fileno(), fcntl.LOCK_UN)
self._fd.close()
except Exception:
pass
finally:
self._fd = None
# Remove lockfile
try:
self.lockfile.unlink()
except Exception:
pass
def is_locked(self) -> bool:
"""Check if the lock is held by another process."""
if not self.lockfile.exists():
return False
try:
fd = open(self.lockfile, "r")
fcntl.flock(fd.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
# We got the lock, so no one else has it
fcntl.flock(fd.fileno(), fcntl.LOCK_UN)
fd.close()
return False
except (IOError, OSError):
return True
def get_pid(self) -> int | None:
"""Get the PID of the daemon holding the lock."""
if not self.lockfile.exists():
return None
try:
with open(self.lockfile, "r") as f:
return int(f.read().strip())
except (ValueError, IOError):
return None
def daemonize():
@@ -64,21 +136,16 @@ def daemonize():
Uses double-fork technique to properly detach from terminal.
"""
# First fork - detach from parent
try:
pid = os.fork()
if pid > 0:
sys.exit(0)
except OSError as e:
sys.stderr.write(f"fork #1 failed: {e}\n")
sys.exit(1)
# First fork
pid = os.fork()
if pid > 0:
# Parent exits
sys.exit(0)
# Decouple from parent environment
os.chdir("/")
# Create new session
os.setsid()
os.umask(0)
# Second fork - prevent zombie
# Second fork to prevent zombie processes
pid = os.fork()
if pid > 0:
sys.exit(0)
@@ -111,3 +178,27 @@ def check_daemon_running(port: int = DEFAULT_DAEMON_PORT) -> bool:
return result == 0
except Exception:
return False
def stop_daemon(port: int = DEFAULT_DAEMON_PORT) -> bool:
"""Stop a running daemon by sending SIGTERM to the process."""
lock = DaemonLock()
pid = lock.get_pid()
if pid is None:
return False
try:
import signal
os.kill(pid, signal.SIGTERM)
# Wait for process to exit
for _ in range(50): # Wait up to 5 seconds
time.sleep(0.1)
try:
os.kill(pid, 0) # Check if process exists
except OSError:
return True # Process exited
return False
except OSError:
return False
@@ -1,5 +0,0 @@
"""Consolidation engine for automatic learning creation from memories."""
from .consolidator import run_consolidation_job
__all__ = ["run_consolidation_job"]
File diff suppressed because it is too large Load Diff
@@ -1,85 +0,0 @@
"""Prompts for the consolidation engine."""
CONSOLIDATION_SYSTEM_PROMPT = """You are a memory consolidation system. Your job is to convert facts into durable knowledge (observations) and merge with existing knowledge when appropriate.
You must output ONLY valid JSON with no markdown code blocks or additional text. However, the "text" field within each observation should use markdown formatting (headers, lists, bold, etc.) for clarity and readability.
## EXTRACT DURABLE KNOWLEDGE, NOT EPHEMERAL STATE
Facts often describe events or actions. Extract the DURABLE KNOWLEDGE implied by the fact, not the transient state.
Examples of extracting durable knowledge:
- "User moved to Room 203" -> "Room 203 exists" (location exists, not where user is now)
- "User visited Acme Corp at Room 105" -> "Acme Corp is located in Room 105"
- "User took the elevator to floor 3" -> "Floor 3 is accessible by elevator"
- "User met Sarah at the lobby" -> "Sarah can be found at the lobby"
DO NOT track current user position/state as knowledge - that changes constantly.
DO track permanent facts learned from the user's actions.
## PRESERVE SPECIFIC DETAILS
Keep names, locations, numbers, and other specifics. Do NOT:
- Abstract into general principles
- Generate business insights
- Make knowledge generic
GOOD examples:
- Fact: "John likes pizza" -> "John likes pizza"
- Fact: "Alice works at Google" -> "Alice works at Google"
BAD examples:
- "John likes pizza" -> "Understanding dietary preferences helps..." (TOO ABSTRACT)
- "User is at Room 203" -> "User is currently at Room 203" (EPHEMERAL STATE)
## MERGE RULES (when comparing to existing observations):
1. REDUNDANT: Same information worded differently → update existing
2. CONTRADICTION: Opposite information about same topic → update with temporal markers showing change
Example: "Alex used to love pizza but now hates it" OR "Alex's pizza preference changed from love to hate"
3. UPDATE: New state replacing old state → update showing the transition with "used to", "now", "changed from X to Y"
## CRITICAL RULES:
- NEVER merge facts about DIFFERENT people
- NEVER merge unrelated topics (food preferences vs work vs hobbies)
- When merging contradictions, the "text" field MUST capture BOTH states with temporal markers:
* Use "used to X, now Y" OR "changed from X to Y" OR "X but now Y"
* DO NOT just state the new fact - you MUST show the change
- Keep observations focused on ONE specific topic per person
- The "text" field MUST contain durable knowledge, not ephemeral state
- Do NOT include "tags" in output - tags are handled automatically"""
CONSOLIDATION_USER_PROMPT = """Analyze this new fact and consolidate into knowledge.
{mission_section}
NEW FACT: {fact_text}
EXISTING OBSERVATIONS (JSON array with source memories and dates):
{observations_text}
Each observation includes:
- id: unique identifier for updating
- text: the observation content
- proof_count: number of supporting memories
- tags: visibility scope (handled automatically)
- created_at/updated_at: when observation was created/modified
- occurred_start/occurred_end: temporal range of source facts
- source_memories: array of supporting facts with their text and dates
Instructions:
1. Extract DURABLE KNOWLEDGE from the new fact (not ephemeral state)
2. Review source_memories in existing observations to understand evidence
3. Check dates to detect contradictions or updates
4. Compare with observations:
- Same topic → UPDATE with learning_id
- New topic → CREATE new observation
- Purely ephemeral → return []
Output JSON array of actions (the "text" field should use markdown formatting for structure):
[
{{"action": "update", "learning_id": "uuid-from-observations", "text": "## Updated Knowledge\n\n**Key point**: details here\n\n- Supporting detail 1\n- Supporting detail 2", "reason": "..."}},
{{"action": "create", "text": "## New Durable Knowledge\n\nDescription with **emphasis** and proper structure", "reason": "..."}}
]
Return [] if fact contains no durable knowledge.
IMPORTANT: Format the "text" field with markdown for better readability:
- Use headers, lists, bold/italic, tables where appropriate
- CRITICAL: Add blank lines before and after block elements (tables, code blocks, lists)
- Ensure proper spacing for markdown to render correctly"""
File diff suppressed because it is too large Load Diff
@@ -1,284 +0,0 @@
"""
Database connection budget management.
Limits concurrent database connections per operation to prevent
a single operation (e.g., recall with parallel queries) from
exhausting the connection pool.
"""
import asyncio
import logging
import uuid
from contextlib import asynccontextmanager
from dataclasses import dataclass, field
from typing import TYPE_CHECKING, AsyncIterator
if TYPE_CHECKING:
import asyncpg
logger = logging.getLogger(__name__)
@dataclass
class OperationBudget:
"""
Tracks connection budget for a single operation.
Each operation gets a semaphore limiting its concurrent connections.
"""
operation_id: str
max_connections: int
semaphore: asyncio.Semaphore = field(init=False)
active_count: int = field(default=0, init=False)
def __post_init__(self):
self.semaphore = asyncio.Semaphore(self.max_connections)
class ConnectionBudgetManager:
"""
Manages per-operation connection budgets.
Usage:
manager = ConnectionBudgetManager(default_budget=4)
# Start an operation
async with manager.operation(max_connections=2) as op:
# Acquire connections within the budget
async with op.acquire(pool) as conn:
await conn.fetch(...)
# Multiple connections respect the budget
async with op.acquire(pool) as conn1, op.acquire(pool) as conn2:
# At most 2 concurrent connections for this operation
...
"""
def __init__(self, default_budget: int = 4):
"""
Initialize the budget manager.
Args:
default_budget: Default max connections per operation
"""
self.default_budget = default_budget
self._operations: dict[str, OperationBudget] = {}
self._lock = asyncio.Lock()
@asynccontextmanager
async def operation(
self,
max_connections: int | None = None,
operation_id: str | None = None,
) -> AsyncIterator["BudgetedOperation"]:
"""
Create a budgeted operation context.
Args:
max_connections: Max concurrent connections for this operation.
Defaults to manager's default_budget.
operation_id: Optional custom operation ID. Auto-generated if not provided.
Yields:
BudgetedOperation context for acquiring connections
"""
op_id = operation_id or f"op-{uuid.uuid4().hex[:12]}"
budget = max_connections or self.default_budget
async with self._lock:
if op_id in self._operations:
raise ValueError(f"Operation {op_id} already exists")
self._operations[op_id] = OperationBudget(op_id, budget)
try:
yield BudgetedOperation(self, op_id)
finally:
async with self._lock:
self._operations.pop(op_id, None)
def _get_budget(self, operation_id: str) -> OperationBudget:
"""Get budget for an operation (internal use)."""
budget = self._operations.get(operation_id)
if not budget:
raise ValueError(f"Operation {operation_id} not found")
return budget
class BudgetedOperation:
"""
A single operation with connection budget.
Provides methods to acquire connections within the budget.
"""
def __init__(self, manager: ConnectionBudgetManager, operation_id: str):
self._manager = manager
self.operation_id = operation_id
@property
def budget(self) -> OperationBudget:
"""Get the budget for this operation."""
return self._manager._get_budget(self.operation_id)
@asynccontextmanager
async def acquire(self, pool: "asyncpg.Pool") -> AsyncIterator["asyncpg.Connection"]:
"""
Acquire a connection within the operation's budget.
Blocks if the operation has reached its connection limit.
Args:
pool: asyncpg connection pool
Yields:
Database connection
"""
budget = self.budget
async with budget.semaphore:
budget.active_count += 1
conn = await pool.acquire()
try:
yield conn
finally:
budget.active_count -= 1
await pool.release(conn)
def wrap_pool(self, pool: "asyncpg.Pool") -> "BudgetedPool":
"""
Wrap a pool with this operation's budget.
The returned BudgetedPool can be passed to functions expecting a pool,
and all acquire() calls will be limited by this operation's budget.
Args:
pool: asyncpg connection pool to wrap
Returns:
BudgetedPool that limits connections to this operation's budget
"""
return BudgetedPool(pool, self)
async def acquire_many(
self,
pool: "asyncpg.Pool",
count: int,
) -> AsyncIterator[list["asyncpg.Connection"]]:
"""
Acquire multiple connections within the budget.
Note: This acquires connections sequentially to respect the budget.
For parallel acquisition, use multiple acquire() calls with asyncio.gather().
Args:
pool: asyncpg connection pool
count: Number of connections to acquire
Yields:
List of database connections
"""
connections = []
try:
for _ in range(count):
conn = await pool.acquire()
connections.append(conn)
yield connections
finally:
for conn in connections:
await pool.release(conn)
# Global default manager instance
_default_manager: ConnectionBudgetManager | None = None
def get_budget_manager(default_budget: int = 4) -> ConnectionBudgetManager:
"""
Get or create the global budget manager.
Args:
default_budget: Default max connections per operation
Returns:
Global ConnectionBudgetManager instance
"""
global _default_manager
if _default_manager is None:
_default_manager = ConnectionBudgetManager(default_budget=default_budget)
return _default_manager
@asynccontextmanager
async def budgeted_operation(
max_connections: int | None = None,
operation_id: str | None = None,
default_budget: int = 4,
) -> AsyncIterator[BudgetedOperation]:
"""
Convenience function to create a budgeted operation.
Args:
max_connections: Max concurrent connections for this operation
operation_id: Optional custom operation ID
default_budget: Default budget if manager not yet created
Yields:
BudgetedOperation context
Example:
async with budgeted_operation(max_connections=2) as op:
async with op.acquire(pool) as conn:
await conn.fetch(...)
"""
manager = get_budget_manager(default_budget)
async with manager.operation(max_connections, operation_id) as op:
yield op
class BudgetedPool:
"""
A pool wrapper that limits concurrent connection acquisitions.
This can be passed to functions expecting a pool, and acquire()
calls will be limited by the budget semaphore.
Usage:
async with budgeted_operation(max_connections=4) as op:
budgeted_pool = op.wrap_pool(pool)
# Pass budgeted_pool to functions that expect a pool
await some_function(budgeted_pool, ...)
"""
def __init__(self, pool: "asyncpg.Pool", operation: BudgetedOperation):
self._pool = pool
self._operation = operation
async def acquire(self) -> "asyncpg.Connection":
"""
Acquire a connection within the budget.
Note: Caller must release the connection when done.
Prefer using as context manager via acquire_with_retry or op.acquire().
"""
budget = self._operation.budget
await budget.semaphore.acquire()
budget.active_count += 1
try:
return await self._pool.acquire()
except Exception:
budget.active_count -= 1
budget.semaphore.release()
raise
async def release(self, conn: "asyncpg.Connection") -> None:
"""Release a connection back to the pool."""
budget = self._operation.budget
try:
await self._pool.release(conn)
finally:
budget.active_count -= 1
budget.semaphore.release()
def __getattr__(self, name):
"""Proxy other attributes to the underlying pool."""
return getattr(self._pool, name)
@@ -83,22 +83,11 @@ async def acquire_with_retry(pool: asyncpg.Pool, max_retries: int = DEFAULT_MAX_
Yields:
An asyncpg connection
"""
import time
start = time.time()
async def acquire():
return await pool.acquire()
conn = await retry_with_backoff(acquire, max_retries=max_retries)
acquire_time = time.time() - start
# Log slow connection acquisitions (indicates pool contention)
if acquire_time > 0.05: # 50ms threshold
pool_size = pool.get_size()
pool_free = pool.get_idle_size()
logger.warning(f"[DB POOL] Slow acquire: {acquire_time:.3f}s | size={pool_size}, idle={pool_free}")
try:
yield conn
finally:
@@ -1,5 +0,0 @@
"""Directives module for hard rules injected into prompts."""
from .models import Directive
__all__ = ["Directive"]
@@ -1,37 +0,0 @@
"""Pydantic models for directives."""
from datetime import datetime, timezone
from uuid import UUID
from pydantic import BaseModel, Field
class Directive(BaseModel):
"""A directive is a hard rule injected into prompts.
Directives are user-defined rules that guide agent behavior. Unlike mental models
which are automatically consolidated from memories, directives are explicit
instructions that are always included in relevant prompts.
Examples:
- "Always respond in formal English"
- "Never share personal data with third parties"
- "Prefer conservative investment recommendations"
"""
id: UUID = Field(description="Unique identifier")
bank_id: str = Field(description="Bank this directive belongs to")
name: str = Field(description="Human-readable name")
content: str = Field(description="The directive text to inject into prompts")
priority: int = Field(default=0, description="Higher priority directives are injected first")
is_active: bool = Field(default=True, description="Whether this directive is currently active")
tags: list[str] = Field(default_factory=list, description="Tags for filtering")
created_at: datetime = Field(
default_factory=lambda: datetime.now(timezone.utc), description="When this directive was created"
)
updated_at: datetime = Field(
default_factory=lambda: datetime.now(timezone.utc), description="When this directive was last updated"
)
class Config:
from_attributes = True
+27 -391
View File
@@ -11,28 +11,19 @@ Configuration via environment variables - see hindsight_api.config for all env v
import logging
import os
import warnings
from abc import ABC, abstractmethod
import httpx
from ..config import (
DEFAULT_EMBEDDINGS_COHERE_MODEL,
DEFAULT_EMBEDDINGS_LITELLM_MODEL,
DEFAULT_EMBEDDINGS_LITELLM_SDK_MODEL,
DEFAULT_EMBEDDINGS_LOCAL_FORCE_CPU,
DEFAULT_EMBEDDINGS_LOCAL_MODEL,
DEFAULT_EMBEDDINGS_LOCAL_TRUST_REMOTE_CODE,
DEFAULT_EMBEDDINGS_OPENAI_MODEL,
DEFAULT_EMBEDDINGS_PROVIDER,
DEFAULT_LITELLM_API_BASE,
ENV_EMBEDDINGS_COHERE_API_KEY,
ENV_EMBEDDINGS_LITELLM_SDK_API_KEY,
ENV_EMBEDDINGS_LOCAL_FORCE_CPU,
ENV_COHERE_API_KEY,
ENV_EMBEDDINGS_COHERE_MODEL,
ENV_EMBEDDINGS_LOCAL_MODEL,
ENV_EMBEDDINGS_LOCAL_TRUST_REMOTE_CODE,
ENV_EMBEDDINGS_OPENAI_API_KEY,
ENV_EMBEDDINGS_OPENAI_BASE_URL,
ENV_EMBEDDINGS_OPENAI_MODEL,
ENV_EMBEDDINGS_PROVIDER,
ENV_EMBEDDINGS_TEI_URL,
@@ -94,22 +85,15 @@ class LocalSTEmbeddings(Embeddings):
The embedding dimension is auto-detected from the model.
"""
def __init__(self, model_name: str | None = None, force_cpu: bool = False, trust_remote_code: bool = False):
def __init__(self, model_name: str | None = None):
"""
Initialize local SentenceTransformers embeddings.
Args:
model_name: Name of the SentenceTransformer model to use.
Default: BAAI/bge-small-en-v1.5
force_cpu: Force CPU mode (avoids MPS/XPC issues on macOS in daemon mode).
Default: False
trust_remote_code: Allow loading models with custom code (security risk).
Required for some models with custom architectures.
Default: False (disabled for security)
"""
self.model_name = model_name or DEFAULT_EMBEDDINGS_LOCAL_MODEL
self.force_cpu = force_cpu
self.trust_remote_code = trust_remote_code
self._model = None
self._dimension: int | None = None
@@ -137,53 +121,12 @@ class LocalSTEmbeddings(Embeddings):
)
logger.info(f"Embeddings: initializing local provider with model {self.model_name}")
# Determine device based on hardware availability.
# We always set low_cpu_mem_usage=False to prevent lazy loading (meta tensors)
# which can cause issues when accelerate is installed but no GPU is available.
import torch
# Force CPU mode if configured (used in daemon mode to avoid MPS/XPC issues on macOS)
if self.force_cpu:
device = "cpu"
logger.info("Embeddings: forcing CPU mode")
else:
# Check for GPU (CUDA) or Apple Silicon (MPS)
# Wrap in try-except to gracefully handle any device detection issues
# (e.g., in CI environments or when PyTorch is built without GPU support)
device = "cpu" # Default to CPU
try:
has_gpu = torch.cuda.is_available() or (
hasattr(torch.backends, "mps") and torch.backends.mps.is_available()
)
if has_gpu:
device = None # Let sentence-transformers auto-detect GPU/MPS
except Exception as e:
logger.warning(f"Failed to detect GPU/MPS, falling back to CPU: {e}")
# Suppress verbose transformers warnings during model loading
# This suppresses the "UNEXPECTED" warnings from BertModel which are harmless
# but look alarming to users (e.g., "embeddings.position_ids | UNEXPECTED")
with warnings.catch_warnings():
warnings.filterwarnings("ignore", category=UserWarning)
warnings.filterwarnings("ignore", message=".*was not found in model state dict.*")
warnings.filterwarnings("ignore", message=".*UNEXPECTED.*")
# Also suppress transformers library logging temporarily
transformers_logger = logging.getLogger("transformers")
original_level = transformers_logger.level
transformers_logger.setLevel(logging.ERROR)
try:
self._model = SentenceTransformer(
self.model_name,
device=device,
model_kwargs={"low_cpu_mem_usage": False},
trust_remote_code=self.trust_remote_code,
)
finally:
# Restore original logging level
transformers_logger.setLevel(original_level)
# Disable lazy loading (meta tensors) which causes issues with newer transformers/accelerate
# Setting low_cpu_mem_usage=False and device_map=None ensures tensors are fully materialized
self._model = SentenceTransformer(
self.model_name,
model_kwargs={"low_cpu_mem_usage": False, "device_map": None},
)
self._dimension = self._model.get_sentence_embedding_dimension()
logger.info(f"Embeddings: local provider initialized (dim: {self._dimension})")
@@ -200,7 +143,6 @@ class LocalSTEmbeddings(Embeddings):
"""
if self._model is None:
raise RuntimeError("Embeddings not initialized. Call initialize() first.")
embeddings = self._model.encode(texts, convert_to_numpy=True, show_progress_bar=False)
return [emb.tolist() for emb in embeddings]
@@ -380,7 +322,6 @@ class OpenAIEmbeddings(Embeddings):
self,
api_key: str,
model: str = DEFAULT_EMBEDDINGS_OPENAI_MODEL,
base_url: str | None = None,
batch_size: int = 100,
max_retries: int = 3,
):
@@ -390,13 +331,11 @@ class OpenAIEmbeddings(Embeddings):
Args:
api_key: OpenAI API key
model: OpenAI embedding model name (default: text-embedding-3-small)
base_url: Custom base URL for OpenAI-compatible API (e.g., Azure OpenAI endpoint)
batch_size: Maximum batch size for embedding requests (default: 100)
max_retries: Maximum number of retries for failed requests (default: 3)
"""
self.api_key = api_key
self.model = model
self.base_url = base_url
self.batch_size = batch_size
self.max_retries = max_retries
self._client = None
@@ -422,14 +361,8 @@ class OpenAIEmbeddings(Embeddings):
except ImportError:
raise ImportError("openai is required for OpenAIEmbeddings. Install it with: pip install openai")
base_url_msg = f" at {self.base_url}" if self.base_url else ""
logger.info(f"Embeddings: initializing OpenAI provider with model {self.model}{base_url_msg}")
# Build client kwargs, only including base_url if set (for Azure or custom endpoints)
client_kwargs = {"api_key": self.api_key, "max_retries": self.max_retries}
if self.base_url:
client_kwargs["base_url"] = self.base_url
self._client = OpenAI(**client_kwargs)
logger.info(f"Embeddings: initializing OpenAI provider with model {self.model}")
self._client = OpenAI(api_key=self.api_key, max_retries=self.max_retries)
# Try to get dimension from known models, otherwise do a test embedding
if self.model in self.MODEL_DIMENSIONS:
@@ -502,7 +435,6 @@ class CohereEmbeddings(Embeddings):
self,
api_key: str,
model: str = DEFAULT_EMBEDDINGS_COHERE_MODEL,
base_url: str | None = None,
batch_size: int = 96,
timeout: float = 60.0,
input_type: str = "search_document",
@@ -513,7 +445,6 @@ class CohereEmbeddings(Embeddings):
Args:
api_key: Cohere API key
model: Cohere embedding model name (default: embed-english-v3.0)
base_url: Custom base URL for Cohere-compatible API (e.g., Azure-hosted endpoint)
batch_size: Maximum batch size for embedding requests (default: 96, Cohere's limit)
timeout: Request timeout in seconds (default: 60.0)
input_type: Input type for embeddings (default: search_document).
@@ -521,7 +452,6 @@ class CohereEmbeddings(Embeddings):
"""
self.api_key = api_key
self.model = model
self.base_url = base_url
self.batch_size = batch_size
self.timeout = timeout
self.input_type = input_type
@@ -548,14 +478,8 @@ class CohereEmbeddings(Embeddings):
except ImportError:
raise ImportError("cohere is required for CohereEmbeddings. Install it with: pip install cohere")
base_url_msg = f" at {self.base_url}" if self.base_url else ""
logger.info(f"Embeddings: initializing Cohere provider with model {self.model}{base_url_msg}")
# Build client kwargs, only including base_url if set (for Azure or custom endpoints)
client_kwargs = {"api_key": self.api_key, "timeout": self.timeout}
if self.base_url:
client_kwargs["base_url"] = self.base_url
self._client = cohere.Client(**client_kwargs)
logger.info(f"Embeddings: initializing Cohere provider with model {self.model}")
self._client = cohere.Client(api_key=self.api_key, timeout=self.timeout)
# Try to get dimension from known models, otherwise do a test embedding
if self.model in self.MODEL_DIMENSIONS:
@@ -567,7 +491,7 @@ class CohereEmbeddings(Embeddings):
model=self.model,
input_type=self.input_type,
)
if response.embeddings and isinstance(response.embeddings, list):
if response.embeddings:
self._dimension = len(response.embeddings[0])
logger.info(f"Embeddings: Cohere provider initialized (model: {self.model}, dim: {self._dimension})")
@@ -605,290 +529,26 @@ class CohereEmbeddings(Embeddings):
return all_embeddings
class LiteLLMEmbeddings(Embeddings):
"""
LiteLLM embeddings implementation using LiteLLM proxy's /embeddings endpoint.
LiteLLM provides a unified interface for multiple embedding providers.
The proxy exposes an OpenAI-compatible /embeddings endpoint.
See: https://docs.litellm.ai/docs/embedding/supported_embedding
Supported providers via LiteLLM:
- OpenAI (text-embedding-3-small, text-embedding-ada-002, etc.)
- Cohere (embed-english-v3.0, etc.) - prefix with cohere/
- Vertex AI (textembedding-gecko, etc.) - prefix with vertex_ai/
- HuggingFace, Mistral, Voyage AI, etc.
The embedding dimension is auto-detected from the model at initialization.
"""
def __init__(
self,
api_base: str = DEFAULT_LITELLM_API_BASE,
api_key: str | None = None,
model: str = DEFAULT_EMBEDDINGS_LITELLM_MODEL,
batch_size: int = 100,
timeout: float = 60.0,
):
"""
Initialize LiteLLM embeddings client.
Args:
api_base: Base URL of the LiteLLM proxy (default: http://localhost:4000)
api_key: API key for the LiteLLM proxy (optional, depends on proxy config)
model: Embedding model name (default: text-embedding-3-small)
Use provider prefix for non-OpenAI models (e.g., cohere/embed-english-v3.0)
batch_size: Maximum batch size for embedding requests (default: 100)
timeout: Request timeout in seconds (default: 60.0)
"""
self.api_base = api_base.rstrip("/")
self.api_key = api_key
self.model = model
self.batch_size = batch_size
self.timeout = timeout
self._client: httpx.Client | None = None
self._dimension: int | None = None
@property
def provider_name(self) -> str:
return "litellm"
@property
def dimension(self) -> int:
if self._dimension is None:
raise RuntimeError("Embeddings not initialized. Call initialize() first.")
return self._dimension
async def initialize(self) -> None:
"""Initialize the HTTP client and detect embedding dimension."""
if self._client is not None:
return
logger.info(f"Embeddings: initializing LiteLLM provider at {self.api_base} with model {self.model}")
headers = {"Content-Type": "application/json"}
if self.api_key:
headers["Authorization"] = f"Bearer {self.api_key}"
self._client = httpx.Client(timeout=self.timeout, headers=headers)
# Do a test embedding to detect dimension
try:
response = self._client.post(
f"{self.api_base}/embeddings",
json={"model": self.model, "input": ["test"]},
)
response.raise_for_status()
result = response.json()
if result.get("data") and len(result["data"]) > 0:
self._dimension = len(result["data"][0]["embedding"])
logger.info(f"Embeddings: LiteLLM provider initialized (model: {self.model}, dim: {self._dimension})")
except httpx.HTTPError as e:
raise RuntimeError(f"Failed to connect to LiteLLM proxy at {self.api_base}: {e}")
def encode(self, texts: list[str]) -> list[list[float]]:
"""
Generate embeddings using the LiteLLM proxy.
Args:
texts: List of text strings to encode
Returns:
List of embedding vectors
"""
if self._client is None:
raise RuntimeError("Embeddings not initialized. Call initialize() first.")
if not texts:
return []
all_embeddings = []
# Process in batches
for i in range(0, len(texts), self.batch_size):
batch = texts[i : i + self.batch_size]
response = self._client.post(
f"{self.api_base}/embeddings",
json={"model": self.model, "input": batch},
)
response.raise_for_status()
result = response.json()
# Sort by index to ensure correct order
batch_embeddings = sorted(result["data"], key=lambda x: x["index"])
all_embeddings.extend([e["embedding"] for e in batch_embeddings])
return all_embeddings
class LiteLLMSDKEmbeddings(Embeddings):
"""
LiteLLM SDK embeddings for direct API integration.
Supports embeddings via LiteLLM SDK without requiring a proxy server.
Supported providers: Cohere, OpenAI, Azure OpenAI, HuggingFace, Voyage AI, Together AI, etc.
Example model names:
- cohere/embed-english-v3.0
- openai/text-embedding-3-small
- together_ai/togethercomputer/m2-bert-80M-8k-retrieval
- voyage/voyage-2
"""
def __init__(
self,
api_key: str,
model: str = DEFAULT_EMBEDDINGS_LITELLM_SDK_MODEL,
api_base: str | None = None,
batch_size: int = 100,
timeout: float = 60.0,
):
"""
Initialize LiteLLM SDK embeddings client.
Args:
api_key: API key for the embedding provider
model: Model name with provider prefix (e.g., "cohere/embed-english-v3.0")
api_base: Custom base URL for API (optional)
batch_size: Maximum batch size for embedding requests (default: 100)
timeout: Request timeout in seconds (default: 60.0)
"""
self.api_key = api_key
self.model = model
self.api_base = api_base
self.batch_size = batch_size
self.timeout = timeout
self._litellm = None # Will be set during initialization
self._dimension: int | None = None
@property
def provider_name(self) -> str:
return "litellm-sdk"
@property
def dimension(self) -> int:
if self._dimension is None:
raise RuntimeError("Embeddings not initialized. Call initialize() first.")
return self._dimension
async def initialize(self) -> None:
"""Initialize the LiteLLM SDK client and detect dimension."""
if self._litellm is not None:
return
try:
import litellm
self._litellm = litellm # Store reference
except ImportError:
raise ImportError("litellm is required for LiteLLMSDKEmbeddings. Install it with: pip install litellm")
api_base_msg = f" at {self.api_base}" if self.api_base else ""
logger.info(f"Embeddings: initializing LiteLLM SDK provider with model {self.model}{api_base_msg}")
# Do a test embedding to detect dimension
try:
# Build kwargs for embedding call
embed_kwargs = {
"model": self.model,
"input": ["test"],
"api_key": self.api_key,
}
if self.api_base:
embed_kwargs["api_base"] = self.api_base
# Use async embedding method (standard in litellm)
response = await self._litellm.aembedding(**embed_kwargs)
# Extract dimension from response
if response.data and len(response.data) > 0:
self._dimension = len(response.data[0]["embedding"])
else:
raise RuntimeError(f"Unable to detect embedding dimension for model {self.model}")
except Exception as e:
raise RuntimeError(f"Failed to initialize LiteLLM SDK embeddings: {e}")
logger.info(f"Embeddings: LiteLLM SDK provider initialized (model: {self.model}, dim: {self._dimension})")
def encode(self, texts: list[str]) -> list[list[float]]:
"""
Generate embeddings using the LiteLLM SDK.
Args:
texts: List of text strings to encode
Returns:
List of embedding vectors (one per input text)
"""
if self._litellm is None:
raise RuntimeError("Embeddings not initialized. Call initialize() first.")
if not texts:
return []
all_embeddings = []
# Process in batches
for i in range(0, len(texts), self.batch_size):
batch = texts[i : i + self.batch_size]
try:
# Build kwargs for embedding call
embed_kwargs = {
"model": self.model,
"input": batch,
"api_key": self.api_key,
}
if self.api_base:
embed_kwargs["api_base"] = self.api_base
# Use sync embedding (litellm doesn't have async in thread-safe way)
response = self._litellm.embedding(**embed_kwargs)
# Extract embeddings from response
# Sort by index to ensure correct order
batch_embeddings = sorted(response.data, key=lambda x: x.get("index", 0))
all_embeddings.extend([e["embedding"] for e in batch_embeddings])
except Exception as e:
import traceback
logger.error(
f"Error in LiteLLM embedding for batch starting at index {i}: {e}\n"
f"Traceback: {traceback.format_exc()}"
)
raise
return all_embeddings
def create_embeddings_from_env() -> Embeddings:
"""
Create an Embeddings instance based on configuration.
Create an Embeddings instance based on environment variables.
Reads configuration via get_config() to ensure consistency across the codebase.
See hindsight_api.config for environment variable names and defaults.
Returns:
Configured Embeddings instance
"""
from ..config import get_config
config = get_config()
provider = config.embeddings_provider.lower()
provider = os.environ.get(ENV_EMBEDDINGS_PROVIDER, DEFAULT_EMBEDDINGS_PROVIDER).lower()
if provider == "tei":
url = config.embeddings_tei_url
url = os.environ.get(ENV_EMBEDDINGS_TEI_URL)
if not url:
raise ValueError(f"{ENV_EMBEDDINGS_TEI_URL} is required when {ENV_EMBEDDINGS_PROVIDER} is 'tei'")
return RemoteTEIEmbeddings(base_url=url)
elif provider == "local":
return LocalSTEmbeddings(
model_name=config.embeddings_local_model,
force_cpu=config.embeddings_local_force_cpu,
trust_remote_code=config.embeddings_local_trust_remote_code,
)
model = os.environ.get(ENV_EMBEDDINGS_LOCAL_MODEL)
model_name = model or DEFAULT_EMBEDDINGS_LOCAL_MODEL
return LocalSTEmbeddings(model_name=model_name)
elif provider == "openai":
# Use dedicated embeddings API key, or fall back to LLM API key
api_key = os.environ.get(ENV_EMBEDDINGS_OPENAI_API_KEY) or os.environ.get(ENV_LLM_API_KEY)
@@ -898,36 +558,12 @@ def create_embeddings_from_env() -> Embeddings:
f"when {ENV_EMBEDDINGS_PROVIDER} is 'openai'"
)
model = os.environ.get(ENV_EMBEDDINGS_OPENAI_MODEL, DEFAULT_EMBEDDINGS_OPENAI_MODEL)
base_url = os.environ.get(ENV_EMBEDDINGS_OPENAI_BASE_URL) or None
return OpenAIEmbeddings(api_key=api_key, model=model, base_url=base_url)
return OpenAIEmbeddings(api_key=api_key, model=model)
elif provider == "cohere":
api_key = config.embeddings_cohere_api_key
api_key = os.environ.get(ENV_COHERE_API_KEY)
if not api_key:
raise ValueError(f"{ENV_EMBEDDINGS_COHERE_API_KEY} is required when {ENV_EMBEDDINGS_PROVIDER} is 'cohere'")
return CohereEmbeddings(
api_key=api_key,
model=config.embeddings_cohere_model,
base_url=config.embeddings_cohere_base_url,
)
elif provider == "litellm":
return LiteLLMEmbeddings(
api_base=config.embeddings_litellm_api_base,
api_key=config.embeddings_litellm_api_key,
model=config.embeddings_litellm_model,
)
elif provider == "litellm-sdk":
api_key = config.embeddings_litellm_sdk_api_key
if not api_key:
raise ValueError(
f"{ENV_EMBEDDINGS_LITELLM_SDK_API_KEY} is required when {ENV_EMBEDDINGS_PROVIDER} is 'litellm-sdk'"
)
return LiteLLMSDKEmbeddings(
api_key=api_key,
model=config.embeddings_litellm_sdk_model,
api_base=config.embeddings_litellm_sdk_api_base,
)
raise ValueError(f"{ENV_COHERE_API_KEY} is required when {ENV_EMBEDDINGS_PROVIDER} is 'cohere'")
model = os.environ.get(ENV_EMBEDDINGS_COHERE_MODEL, DEFAULT_EMBEDDINGS_COHERE_MODEL)
return CohereEmbeddings(api_key=api_key, model=model)
else:
raise ValueError(
f"Unknown embeddings provider: {provider}. "
f"Supported: 'local', 'tei', 'openai', 'cohere', 'litellm', 'litellm-sdk'"
)
raise ValueError(f"Unknown embeddings provider: {provider}. Supported: 'local', 'tei', 'openai', 'cohere'")
@@ -209,7 +209,7 @@ class EntityResolver:
# This handles duplicates via ON CONFLICT and returns all IDs
if entities_to_create:
# Group entities by canonical name (lowercase) to handle duplicates within batch
# For duplicates, we only insert once and reuse the ID, but track the count
# For duplicates, we only insert once and reuse the ID
unique_entities = {} # lowercase_name -> (entity_data, event_date, [indices])
for idx, entity_data, event_date in entities_to_create:
name_lower = entity_data["text"].lower()
@@ -223,32 +223,29 @@ class EntityResolver:
# Use a single query with unnest for speed
entity_names = []
entity_dates = []
entity_counts = [] # Track how many times each entity appears in this batch
indices_map = [] # Maps result index -> list of original indices
for name_lower, (entity_data, event_date, indices) in unique_entities.items():
entity_names.append(entity_data["text"])
entity_dates.append(event_date)
entity_counts.append(len(indices)) # Count of occurrences in this batch
indices_map.append(indices)
# Batch INSERT ... ON CONFLICT with RETURNING
# Uses the batch count for mention_count instead of always 1
# This is much faster than individual inserts
rows = await conn.fetch(
f"""
INSERT INTO {fq_table("entities")} (bank_id, canonical_name, first_seen, last_seen, mention_count)
SELECT $1, name, event_date, event_date, cnt
FROM unnest($2::text[], $3::timestamptz[], $4::int[]) AS t(name, event_date, cnt)
SELECT $1, name, event_date, event_date, 1
FROM unnest($2::text[], $3::timestamptz[]) AS t(name, event_date)
ON CONFLICT (bank_id, LOWER(canonical_name))
DO UPDATE SET
mention_count = {fq_table("entities")}.mention_count + EXCLUDED.mention_count,
mention_count = {fq_table("entities")}.mention_count + 1,
last_seen = EXCLUDED.last_seen
RETURNING id
""",
bank_id,
entity_names,
entity_dates,
entity_counts,
)
# Map returned IDs back to original indices
+61 -45
View File
@@ -48,7 +48,6 @@ class MemoryEngineInterface(ABC):
contents: list[dict[str, Any]],
*,
request_context: "RequestContext",
document_tags: list[str] | None = None,
) -> dict[str, Any]:
"""
Retain a batch of memory items.
@@ -56,9 +55,8 @@ class MemoryEngineInterface(ABC):
Args:
bank_id: The memory bank ID.
contents: List of content dicts with 'content', optional 'event_date',
'context', 'metadata', 'document_id', and per-item 'tags'.
'context', 'metadata', 'document_id'.
request_context: Request context for authentication.
document_tags: Optional tags applied to all items in the batch.
Returns:
Dict with processing results.
@@ -162,14 +160,14 @@ class MemoryEngineInterface(ABC):
request_context: "RequestContext",
) -> dict[str, Any]:
"""
Get bank profile including disposition and mission.
Get bank profile including disposition and background.
Args:
bank_id: The memory bank ID.
request_context: Request context for authentication.
Returns:
Bank profile dict with bank_id, name, disposition, and mission.
Bank profile dict.
"""
...
@@ -192,44 +190,25 @@ class MemoryEngineInterface(ABC):
...
@abstractmethod
async def merge_bank_mission(
async def merge_bank_background(
self,
bank_id: str,
new_info: str,
*,
update_disposition: bool = True,
request_context: "RequestContext",
) -> dict[str, Any]:
"""
Merge new mission information into bank profile.
Merge new background information into bank profile.
Args:
bank_id: The memory bank ID.
new_info: New mission information to merge.
new_info: New background information to merge.
update_disposition: Whether to infer disposition from background.
request_context: Request context for authentication.
Returns:
Updated mission info.
"""
...
@abstractmethod
async def set_bank_mission(
self,
bank_id: str,
mission: str,
*,
request_context: "RequestContext",
) -> dict[str, Any]:
"""
Set the bank's mission (replaces existing).
Args:
bank_id: The memory bank ID.
mission: The mission text.
request_context: Request context for authentication.
Returns:
Dict with bank_id and mission.
Updated background info.
"""
...
@@ -310,7 +289,6 @@ class MemoryEngineInterface(ABC):
bank_id: str,
*,
fact_type: str | None = None,
limit: int = 1000,
request_context: "RequestContext",
) -> dict[str, Any]:
"""
@@ -319,11 +297,10 @@ class MemoryEngineInterface(ABC):
Args:
bank_id: The memory bank ID.
fact_type: Filter by fact type.
limit: Maximum number of items to return (default: 1000).
request_context: Request context for authentication.
Returns:
Dict with nodes, edges, table_rows, total_units, limit.
Dict with nodes, edges, table_rows, total_units.
"""
...
@@ -427,20 +404,61 @@ class MemoryEngineInterface(ABC):
bank_id: str,
*,
limit: int = 100,
offset: int = 0,
request_context: "RequestContext",
) -> dict[str, Any]:
) -> list[dict[str, Any]]:
"""
List entities for a bank with pagination.
List entities for a bank.
Args:
bank_id: The memory bank ID.
limit: Maximum results.
offset: Offset for pagination.
request_context: Request context for authentication.
Returns:
Dict with items, total, limit, offset.
List of entity dicts.
"""
...
@abstractmethod
async def get_entity_observations(
self,
bank_id: str,
entity_id: str,
*,
limit: int = 10,
request_context: "RequestContext",
) -> list[Any]:
"""
Get observations for an entity.
Args:
bank_id: The memory bank ID.
entity_id: The entity ID.
limit: Maximum observations.
request_context: Request context for authentication.
Returns:
List of EntityObservation objects.
"""
...
@abstractmethod
async def regenerate_entity_observations(
self,
bank_id: str,
entity_id: str,
entity_name: str,
*,
request_context: "RequestContext",
) -> None:
"""
Regenerate observations for an entity.
Args:
bank_id: The memory bank ID.
entity_id: The entity ID.
entity_name: The entity's canonical name.
request_context: Request context for authentication.
"""
...
@@ -496,7 +514,7 @@ class MemoryEngineInterface(ABC):
bank_id: str,
*,
request_context: "RequestContext",
) -> dict[str, Any]:
) -> list[dict[str, Any]]:
"""
List async operations for a bank.
@@ -505,7 +523,7 @@ class MemoryEngineInterface(ABC):
request_context: Request context for authentication.
Returns:
Dict with 'total' (int) and 'operations' (list of operation dicts).
List of operation dicts with id, task_type, status, etc.
"""
...
@@ -539,16 +557,16 @@ class MemoryEngineInterface(ABC):
bank_id: str,
*,
name: str | None = None,
mission: str | None = None,
background: str | None = None,
request_context: "RequestContext",
) -> dict[str, Any]:
"""
Update bank name and/or mission.
Update bank name and/or background.
Args:
bank_id: The memory bank ID.
name: New bank name (optional).
mission: New mission text (optional, replaces existing).
background: New background text (optional, replaces existing).
request_context: Request context for authentication.
Returns:
@@ -563,7 +581,6 @@ class MemoryEngineInterface(ABC):
contents: list[dict[str, Any]],
*,
request_context: "RequestContext",
document_tags: list[str] | None = None,
) -> dict[str, Any]:
"""
Submit a batch retain operation to run asynchronously.
@@ -572,7 +589,6 @@ class MemoryEngineInterface(ABC):
bank_id: The memory bank ID.
contents: List of content dicts to retain.
request_context: Request context for authentication.
document_tags: Optional tags applied to all items in the async batch.
Returns:
Dict with operation_id and items_count.
@@ -1,207 +0,0 @@
"""
Abstract interface for LLM providers.
This module defines the interface that all LLM providers must implement,
enabling support for multiple LLM backends (OpenAI, Anthropic, Gemini, Codex, etc.)
"""
from abc import ABC, abstractmethod
from typing import Any
from .response_models import LLMToolCallResult, TokenUsage
class LLMInterface(ABC):
"""
Abstract interface for LLM providers.
All LLM provider implementations must inherit from this class and implement
the required methods.
"""
def __init__(
self,
provider: str,
api_key: str,
base_url: str,
model: str,
reasoning_effort: str = "low",
**kwargs: Any,
):
"""
Initialize LLM provider.
Args:
provider: Provider name (e.g., "openai", "codex", "anthropic", "gemini").
api_key: API key or authentication token.
base_url: Base URL for the API.
model: Model name.
reasoning_effort: Reasoning effort level for supported providers.
**kwargs: Additional provider-specific parameters.
"""
self.provider = provider.lower()
self.api_key = api_key
self.base_url = base_url
self.model = model
self.reasoning_effort = reasoning_effort
@abstractmethod
async def verify_connection(self) -> None:
"""
Verify that the LLM provider is configured correctly by making a simple test call.
Raises:
RuntimeError: If the connection test fails.
"""
pass
@abstractmethod
async def call(
self,
messages: list[dict[str, str]],
response_format: Any | None = None,
max_completion_tokens: int | None = None,
temperature: float | None = None,
scope: str = "memory",
max_retries: int = 10,
initial_backoff: float = 1.0,
max_backoff: float = 60.0,
skip_validation: bool = False,
strict_schema: bool = False,
return_usage: bool = False,
) -> Any:
"""
Make an LLM API call with retry logic.
Args:
messages: List of message dicts with 'role' and 'content'.
response_format: Optional Pydantic model for structured output.
max_completion_tokens: Maximum tokens in response.
temperature: Sampling temperature (0.0-2.0).
scope: Scope identifier for tracking.
max_retries: Maximum retry attempts.
initial_backoff: Initial backoff time in seconds.
max_backoff: Maximum backoff time in seconds.
skip_validation: Return raw JSON without Pydantic validation.
strict_schema: Use strict JSON schema enforcement (OpenAI only).
return_usage: If True, return tuple (result, TokenUsage) instead of just result.
Returns:
If return_usage=False: Parsed response if response_format is provided, otherwise text content.
If return_usage=True: Tuple of (result, TokenUsage) with token counts.
Raises:
OutputTooLongError: If output exceeds token limits.
Exception: Re-raises API errors after retries exhausted.
"""
pass
@abstractmethod
async def call_with_tools(
self,
messages: list[dict[str, Any]],
tools: list[dict[str, Any]],
max_completion_tokens: int | None = None,
temperature: float | None = None,
scope: str = "tools",
max_retries: int = 5,
initial_backoff: float = 1.0,
max_backoff: float = 30.0,
tool_choice: str | dict[str, Any] = "auto",
) -> LLMToolCallResult:
"""
Make an LLM API call with tool/function calling support.
Args:
messages: List of message dicts. Can include tool results with role='tool'.
tools: List of tool definitions in OpenAI format.
max_completion_tokens: Maximum tokens in response.
temperature: Sampling temperature (0.0-2.0).
scope: Scope identifier for tracking.
max_retries: Maximum retry attempts.
initial_backoff: Initial backoff time in seconds.
max_backoff: Maximum backoff time in seconds.
tool_choice: How to choose tools - "auto", "none", "required", or specific function.
Returns:
LLMToolCallResult with content and/or tool_calls.
"""
pass
async def supports_batch_api(self) -> bool:
"""
Check if this provider supports batch API operations.
Returns:
True if provider supports submit_batch/get_batch_status/retrieve_batch_results
"""
return False
async def submit_batch(
self,
requests: list[dict[str, Any]],
endpoint: str = "/v1/chat/completions",
completion_window: str = "24h",
) -> dict[str, Any]:
"""
Submit a batch of requests to the provider's batch API.
Args:
requests: List of request dicts in JSONL format (custom_id, method, url, body)
endpoint: API endpoint for the batch (e.g., "/v1/chat/completions")
completion_window: Completion window (e.g., "24h")
Returns:
Dict with batch metadata: {"batch_id": str, "status": str, ...}
Raises:
NotImplementedError: If provider doesn't support batch API
"""
raise NotImplementedError(f"Batch API not supported for provider: {self.provider}")
async def get_batch_status(self, batch_id: str) -> dict[str, Any]:
"""
Get the status of a batch job.
Args:
batch_id: Batch identifier returned from submit_batch
Returns:
Dict with status info: {"batch_id": str, "status": str, "completed_at": str, ...}
Raises:
NotImplementedError: If provider doesn't support batch API
"""
raise NotImplementedError(f"Batch API not supported for provider: {self.provider}")
async def retrieve_batch_results(self, batch_id: str) -> list[dict[str, Any]]:
"""
Retrieve completed batch results.
Args:
batch_id: Batch identifier returned from submit_batch
Returns:
List of result dicts (one per request, matched by custom_id)
Raises:
NotImplementedError: If provider doesn't support batch API
"""
raise NotImplementedError(f"Batch API not supported for provider: {self.provider}")
@abstractmethod
async def cleanup(self) -> None:
"""Clean up resources (close connections, etc.)."""
pass
class OutputTooLongError(Exception):
"""
Bridge exception raised when LLM output exceeds token limits.
This wraps provider-specific errors (e.g., OpenAI's LengthFinishReasonError)
to allow callers to handle output length issues without depending on
provider-specific implementations.
"""
pass
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -1,14 +0,0 @@
"""
Mental models module for Hindsight.
Mental models contain directives - hard rules that are injected into reflect prompts.
Directives are user-defined and their observations are user-provided (not LLM-generated).
Other types of consolidated knowledge are handled by:
- Learnings: Automatic bottom-up consolidation from facts
- Pinned Reflections: User-curated living documents
"""
from .models import MentalModel, MentalModelSubtype
__all__ = ["MentalModel", "MentalModelSubtype"]
@@ -1,53 +0,0 @@
"""
Pydantic models for mental models.
"""
from datetime import datetime, timezone
from enum import Enum
from pydantic import BaseModel, Field
class MentalModelSubtype(str, Enum):
"""Subtype of mental model.
Currently only DIRECTIVE is supported. Other types of consolidated knowledge
are handled by:
- Learnings: Automatic bottom-up consolidation from facts
- Pinned Reflections: User-curated living documents
"""
DIRECTIVE = "directive" # User-defined hard rules, observations user-provided
class MentalModel(BaseModel):
"""
A mental model representing synthesized understanding.
Mental models are the agent's consolidated knowledge. Unlike raw facts,
mental models provide:
- A one-liner description for quick scanning/retrieval
- A full summary for deep understanding
- Links to related mental models
"""
id: str = Field(description="Unique identifier within the bank")
bank_id: str = Field(description="Bank this mental model belongs to")
subtype: MentalModelSubtype = Field(description="How this model was created")
name: str = Field(description="Human-readable name")
description: str = Field(description="One-liner for quick scanning and retrieval matching")
summary: str | None = Field(default=None, description="Full synthesized understanding")
# References
entity_id: str | None = Field(default=None, description="Reference to entities table when type=entity")
source_facts: list[str] = Field(default_factory=list, description="Fact IDs used to generate summary")
links: list[str] = Field(default_factory=list, description="Related mental model IDs")
# Tags for scoped visibility (similar to document tags)
tags: list[str] = Field(default_factory=list, description="Tags for scoped visibility filtering")
# Timestamps
last_updated: datetime | None = Field(default=None, description="When summary was last regenerated")
created_at: datetime = Field(
default_factory=lambda: datetime.now(timezone.utc), description="When this model was created"
)
@@ -1,69 +0,0 @@
"""
Typed metadata models for async operations.
These dataclasses define the structure of result_metadata for different operation types.
The metadata is exposed in the API for debugging purposes and may change without notice.
"""
from dataclasses import asdict, dataclass
from typing import Any
@dataclass
class BatchRetainParentMetadata:
"""Metadata for parent batch_retain operations (when split into sub-batches)."""
items_count: int
total_tokens: int
num_sub_batches: int
is_parent: bool = True
def to_dict(self) -> dict[str, Any]:
"""Convert to dict for JSON serialization."""
return asdict(self)
@dataclass
class BatchRetainChildMetadata:
"""Metadata for child batch_retain operations (individual sub-batches)."""
items_count: int
parent_operation_id: str
sub_batch_index: int
total_sub_batches: int
def to_dict(self) -> dict[str, Any]:
"""Convert to dict for JSON serialization."""
return asdict(self)
@dataclass
class RetainMetadata:
"""Metadata for regular retain operations (non-batched, deprecated async path)."""
items_count: int
def to_dict(self) -> dict[str, Any]:
"""Convert to dict for JSON serialization."""
return asdict(self)
@dataclass
class ConsolidationMetadata:
"""Metadata for consolidation operations."""
# Currently empty, but structure for future fields
def to_dict(self) -> dict[str, Any]:
"""Convert to dict for JSON serialization."""
return asdict(self)
@dataclass
class RefreshMentalModelMetadata:
"""Metadata for mental model refresh operations."""
mental_model_id: str
def to_dict(self) -> dict[str, Any]:
"""Convert to dict for JSON serialization."""
return asdict(self)
@@ -1,60 +0,0 @@
"""File parser implementations."""
from .base import FileParser
from .markitdown import MarkitdownParser
__all__ = ["FileParser", "MarkitdownParser", "FileParserRegistry"]
class FileParserRegistry:
"""Registry for file parsers with auto-detection."""
def __init__(self):
"""Initialize empty parser registry."""
self._parsers: dict[str, FileParser] = {}
def register(self, parser: FileParser):
"""
Register a parser.
Args:
parser: FileParser instance
"""
self._parsers[parser.name()] = parser
def get_parser(
self,
name: str | None,
filename: str,
content_type: str | None = None,
) -> FileParser:
"""
Get parser by name or auto-detect.
Args:
name: Parser name (e.g., "markitdown") or None for auto-detect
filename: File name for auto-detection
content_type: MIME type (optional)
Returns:
FileParser instance
Raises:
ValueError: If no suitable parser found
"""
if name:
# Explicit parser requested
if name not in self._parsers:
raise ValueError(f"Parser '{name}' not found. Available: {list(self._parsers.keys())}")
return self._parsers[name]
# Auto-detect parser
for parser in self._parsers.values():
if parser.supports(filename, content_type):
return parser
raise ValueError(f"No parser found for {filename}. Available parsers: {list(self._parsers.keys())}")
def list_parsers(self) -> list[str]:
"""Get list of registered parser names."""
return list(self._parsers.keys())
@@ -1,49 +0,0 @@
"""Abstract base class for file parsers."""
from abc import ABC, abstractmethod
class FileParser(ABC):
"""Abstract base for file to markdown parsers."""
@abstractmethod
async def convert(self, file_data: bytes, filename: str) -> str:
"""
Parse file to markdown.
Args:
file_data: Raw file bytes
filename: Original filename (used for format detection)
Returns:
Markdown content as string
Raises:
ValueError: If file format is not supported
RuntimeError: If parsing fails
"""
pass
@abstractmethod
def supports(self, filename: str, content_type: str | None = None) -> bool:
"""
Check if parser supports this file type.
Args:
filename: File name (used for extension check)
content_type: MIME type (optional)
Returns:
True if this parser can handle the file
"""
pass
@abstractmethod
def name(self) -> str:
"""
Get parser name.
Returns:
Parser name (e.g., "markitdown")
"""
pass
@@ -1,109 +0,0 @@
"""Markitdown parser implementation."""
import asyncio
import logging
import tempfile
from pathlib import Path
from .base import FileParser
logger = logging.getLogger(__name__)
class MarkitdownParser(FileParser):
"""
Markitdown file parser.
Uses Microsoft's markitdown library to convert various file formats
to markdown including PDF, Office docs, images (via OCR), audio, HTML.
Supported formats:
- PDF (.pdf)
- Word (.docx, .doc)
- PowerPoint (.pptx, .ppt)
- Excel (.xlsx, .xls)
- Images (.jpg, .jpeg, .png) - with OCR
- HTML (.html, .htm)
- Text (.txt, .md)
- Audio (.mp3, .wav) - with transcription
"""
def __init__(self):
"""Initialize markitdown parser."""
# Lazy import to avoid requiring markitdown for all users
try:
from markitdown import MarkItDown
self._markitdown = MarkItDown()
except ImportError as e:
raise ImportError(
"markitdown package is required for file parsing. Install with: pip install markitdown"
) from e
async def convert(self, file_data: bytes, filename: str) -> str:
"""Parse file to markdown using markitdown."""
# markitdown is synchronous, so we run it in executor to avoid blocking
loop = asyncio.get_event_loop()
return await loop.run_in_executor(None, self._convert_sync, file_data, filename)
def _convert_sync(self, file_data: bytes, filename: str) -> str:
"""Synchronous parsing (runs in thread pool)."""
# Write to temp file (markitdown requires file path)
with tempfile.NamedTemporaryFile(suffix=Path(filename).suffix, delete=False) as tmp:
tmp.write(file_data)
tmp_path = tmp.name
try:
# Parse using markitdown
result = self._markitdown.convert(tmp_path)
if not result or not result.text_content:
raise RuntimeError(f"No content extracted from '{filename}'")
return result.text_content
except Exception as e:
logger.error(f"Markitdown parsing failed for {filename}: {e}")
raise RuntimeError(f"Failed to parse '{filename}': {e}") from e
finally:
# Clean up temp file
try:
Path(tmp_path).unlink()
except Exception:
pass
def supports(self, filename: str, content_type: str | None = None) -> bool:
"""Check if markitdown supports this file type."""
# Supported extensions (from markitdown docs)
supported_extensions = {
# Documents
".pdf",
".docx",
".doc",
".pptx",
".ppt",
".xlsx",
".xls",
# Images (with OCR)
".jpg",
".jpeg",
".png",
# Web
".html",
".htm",
# Text
".txt",
".md",
".csv",
# Audio (with transcription)
".mp3",
".wav",
}
ext = Path(filename).suffix.lower()
return ext in supported_extensions
def name(self) -> str:
"""Get parser name."""
return "markitdown"
@@ -1,14 +0,0 @@
"""
LLM provider implementations.
This package contains concrete implementations of the LLMInterface for various providers.
"""
from .anthropic_llm import AnthropicLLM
from .claude_code_llm import ClaudeCodeLLM
from .codex_llm import CodexLLM
from .gemini_llm import GeminiLLM
from .mock_llm import MockLLM
from .openai_compatible_llm import OpenAICompatibleLLM
__all__ = ["AnthropicLLM", "ClaudeCodeLLM", "CodexLLM", "GeminiLLM", "MockLLM", "OpenAICompatibleLLM"]
@@ -1,477 +0,0 @@
"""
Anthropic LLM provider using the Anthropic Python SDK.
This provider enables using Claude models from Anthropic with support for:
- Structured JSON output
- Tool/function calling with proper format conversion
- Extended thinking mode
- Retry logic with exponential backoff
"""
import asyncio
import json
import logging
import time
from typing import Any
from hindsight_api.engine.llm_interface import LLMInterface, OutputTooLongError
from hindsight_api.engine.response_models import LLMToolCall, LLMToolCallResult, TokenUsage
from hindsight_api.metrics import get_metrics_collector
logger = logging.getLogger(__name__)
class AnthropicLLM(LLMInterface):
"""
LLM provider using Anthropic's Claude models.
Supports structured output, tool calling, and extended thinking mode.
Handles format conversion between OpenAI-style messages and Anthropic's format.
"""
def __init__(
self,
provider: str,
api_key: str,
base_url: str,
model: str,
reasoning_effort: str = "low",
timeout: float = 300.0,
**kwargs: Any,
):
"""
Initialize Anthropic LLM provider.
Args:
provider: Provider name (should be "anthropic").
api_key: Anthropic API key.
base_url: Base URL for the API (optional, uses Anthropic default if empty).
model: Model name (e.g., "claude-sonnet-4-20250514").
reasoning_effort: Reasoning effort level (not used by Anthropic).
timeout: Request timeout in seconds.
**kwargs: Additional provider-specific parameters.
"""
super().__init__(provider, api_key, base_url, model, reasoning_effort, **kwargs)
if not self.api_key:
raise ValueError("API key is required for Anthropic provider")
# Import and initialize Anthropic client
try:
from anthropic import AsyncAnthropic
client_kwargs: dict[str, Any] = {"api_key": self.api_key}
if self.base_url:
client_kwargs["base_url"] = self.base_url
if timeout:
client_kwargs["timeout"] = timeout
self._client = AsyncAnthropic(**client_kwargs)
logger.info(f"Anthropic client initialized for model: {self.model}")
except ImportError as e:
raise RuntimeError("Anthropic SDK not installed. Run: uv add anthropic or pip install anthropic") from e
async def verify_connection(self) -> None:
"""
Verify that the Anthropic provider is configured correctly by making a simple test call.
Raises:
RuntimeError: If the connection test fails.
"""
try:
test_messages = [{"role": "user", "content": "test"}]
await self.call(
messages=test_messages,
max_completion_tokens=10,
temperature=0.0,
scope="verification",
max_retries=0,
)
logger.info("Anthropic connection verified successfully")
except Exception as e:
logger.error(f"Anthropic connection verification failed: {e}")
raise RuntimeError(f"Failed to verify Anthropic connection: {e}") from e
async def call(
self,
messages: list[dict[str, str]],
response_format: Any | None = None,
max_completion_tokens: int | None = None,
temperature: float | None = None,
scope: str = "memory",
max_retries: int = 10,
initial_backoff: float = 1.0,
max_backoff: float = 60.0,
skip_validation: bool = False,
strict_schema: bool = False,
return_usage: bool = False,
) -> Any:
"""
Make an LLM API call with retry logic.
Args:
messages: List of message dicts with 'role' and 'content'.
response_format: Optional Pydantic model for structured output.
max_completion_tokens: Maximum tokens in response.
temperature: Sampling temperature (0.0-2.0).
scope: Scope identifier for tracking.
max_retries: Maximum retry attempts.
initial_backoff: Initial backoff time in seconds.
max_backoff: Maximum backoff time in seconds.
skip_validation: Return raw JSON without Pydantic validation.
strict_schema: Use strict JSON schema enforcement (not supported by Anthropic).
return_usage: If True, return tuple (result, TokenUsage) instead of just result.
Returns:
If return_usage=False: Parsed response if response_format is provided, otherwise text content.
If return_usage=True: Tuple of (result, TokenUsage) with token counts.
Raises:
OutputTooLongError: If output exceeds token limits.
Exception: Re-raises API errors after retries exhausted.
"""
from anthropic import APIConnectionError, APIStatusError, RateLimitError
start_time = time.time()
# Convert OpenAI-style messages to Anthropic format
system_prompt = None
anthropic_messages = []
for msg in messages:
role = msg.get("role", "user")
content = msg.get("content", "")
if role == "system":
if system_prompt:
system_prompt += "\n\n" + content
else:
system_prompt = content
else:
anthropic_messages.append({"role": role, "content": content})
# Add JSON schema instruction if response_format is provided
if response_format is not None and hasattr(response_format, "model_json_schema"):
schema = response_format.model_json_schema()
schema_msg = f"\n\nYou must respond with valid JSON matching this schema:\n{json.dumps(schema, indent=2)}"
if system_prompt:
system_prompt += schema_msg
else:
system_prompt = schema_msg
# Prepare parameters
call_params: dict[str, Any] = {
"model": self.model,
"messages": anthropic_messages,
"max_tokens": max_completion_tokens if max_completion_tokens is not None else 4096,
}
if system_prompt:
call_params["system"] = system_prompt
if temperature is not None:
call_params["temperature"] = temperature
last_exception = None
for attempt in range(max_retries + 1):
try:
response = await self._client.messages.create(**call_params)
# Anthropic response content is a list of blocks
content = ""
for block in response.content:
if block.type == "text":
content += block.text
if response_format is not None:
# Models may wrap JSON in markdown code blocks
clean_content = content
if "```json" in content:
clean_content = content.split("```json")[1].split("```")[0].strip()
elif "```" in content:
clean_content = content.split("```")[1].split("```")[0].strip()
try:
json_data = json.loads(clean_content)
except json.JSONDecodeError:
# Fallback to parsing raw content if markdown stripping failed
json_data = json.loads(content)
if skip_validation:
result = json_data
else:
result = response_format.model_validate(json_data)
else:
result = content
# Record metrics and log slow calls
duration = time.time() - start_time
input_tokens = response.usage.input_tokens or 0 if response.usage else 0
output_tokens = response.usage.output_tokens or 0 if response.usage else 0
total_tokens = input_tokens + output_tokens
# Record LLM metrics
metrics = get_metrics_collector()
metrics.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
duration=duration,
input_tokens=input_tokens,
output_tokens=output_tokens,
success=True,
)
# Record trace span
from hindsight_api.tracing import _serialize_for_span, get_span_recorder
finish_reason = response.stop_reason if hasattr(response, "stop_reason") else None
span_recorder = get_span_recorder()
span_recorder.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
messages=messages,
response_content=_serialize_for_span(result),
input_tokens=input_tokens,
output_tokens=output_tokens,
duration=duration,
finish_reason=finish_reason,
error=None,
)
# Log slow calls
if duration > 10.0:
logger.info(
f"slow llm call: scope={scope}, model={self.provider}/{self.model}, "
f"input_tokens={input_tokens}, output_tokens={output_tokens}, "
f"time={duration:.3f}s"
)
if return_usage:
token_usage = TokenUsage(
input_tokens=input_tokens,
output_tokens=output_tokens,
total_tokens=total_tokens,
)
return result, token_usage
return result
except json.JSONDecodeError as e:
last_exception = e
if attempt < max_retries:
logger.warning("Anthropic returned invalid JSON, retrying...")
backoff = min(initial_backoff * (2**attempt), max_backoff)
await asyncio.sleep(backoff)
continue
else:
logger.error(f"Anthropic returned invalid JSON after {max_retries + 1} attempts")
raise
except (APIConnectionError, RateLimitError, APIStatusError) as e:
# Fast fail on 401/403
if isinstance(e, APIStatusError) and e.status_code in (401, 403):
logger.error(f"Anthropic auth error (HTTP {e.status_code}), not retrying: {str(e)}")
raise
last_exception = e
if attempt < max_retries:
# Check if it's a rate limit or server error
should_retry = isinstance(e, (APIConnectionError, RateLimitError)) or (
isinstance(e, APIStatusError) and e.status_code >= 500
)
if should_retry:
backoff = min(initial_backoff * (2**attempt), max_backoff)
jitter = backoff * 0.2 * (2 * (time.time() % 1) - 1)
await asyncio.sleep(backoff + jitter)
continue
logger.error(f"Anthropic API error after {max_retries + 1} attempts: {str(e)}")
raise
except Exception as e:
logger.error(f"Unexpected error during Anthropic call: {type(e).__name__}: {str(e)}")
raise
if last_exception:
raise last_exception
raise RuntimeError("Anthropic call failed after all retries")
async def call_with_tools(
self,
messages: list[dict[str, Any]],
tools: list[dict[str, Any]],
max_completion_tokens: int | None = None,
temperature: float | None = None,
scope: str = "tools",
max_retries: int = 5,
initial_backoff: float = 1.0,
max_backoff: float = 30.0,
tool_choice: str | dict[str, Any] = "auto",
) -> LLMToolCallResult:
"""
Make an LLM API call with tool/function calling support.
Args:
messages: List of message dicts. Can include tool results with role='tool'.
tools: List of tool definitions in OpenAI format.
max_completion_tokens: Maximum tokens in response.
temperature: Sampling temperature (0.0-2.0).
scope: Scope identifier for tracking.
max_retries: Maximum retry attempts.
initial_backoff: Initial backoff time in seconds.
max_backoff: Maximum backoff time in seconds.
tool_choice: How to choose tools - "auto", "none", "required", or specific function.
Returns:
LLMToolCallResult with content and/or tool_calls.
"""
from anthropic import APIConnectionError, APIStatusError
start_time = time.time()
# Convert OpenAI tool format to Anthropic format
anthropic_tools = []
for tool in tools:
func = tool.get("function", {})
anthropic_tools.append(
{
"name": func.get("name", ""),
"description": func.get("description", ""),
"input_schema": func.get("parameters", {"type": "object", "properties": {}}),
}
)
# Convert messages - handle tool results
system_prompt = None
anthropic_messages = []
for msg in messages:
role = msg.get("role", "user")
content = msg.get("content", "")
if role == "system":
system_prompt = (system_prompt + "\n\n" + content) if system_prompt else content
elif role == "tool":
# Anthropic uses tool_result blocks
anthropic_messages.append(
{
"role": "user",
"content": [
{"type": "tool_result", "tool_use_id": msg.get("tool_call_id", ""), "content": content}
],
}
)
elif role == "assistant" and msg.get("tool_calls"):
# Convert assistant tool calls
tool_use_blocks = []
for tc in msg["tool_calls"]:
tool_use_blocks.append(
{
"type": "tool_use",
"id": tc.get("id", ""),
"name": tc.get("function", {}).get("name", ""),
"input": json.loads(tc.get("function", {}).get("arguments", "{}")),
}
)
anthropic_messages.append({"role": "assistant", "content": tool_use_blocks})
else:
anthropic_messages.append({"role": role, "content": content})
call_params: dict[str, Any] = {
"model": self.model,
"messages": anthropic_messages,
"tools": anthropic_tools,
"max_tokens": max_completion_tokens or 4096,
}
if system_prompt:
call_params["system"] = system_prompt
if temperature is not None:
call_params["temperature"] = temperature
last_exception = None
for attempt in range(max_retries + 1):
try:
response = await self._client.messages.create(**call_params)
# Extract content and tool calls
content_parts = []
tool_calls: list[LLMToolCall] = []
for block in response.content:
if block.type == "text":
content_parts.append(block.text)
elif block.type == "tool_use":
tool_calls.append(LLMToolCall(id=block.id, name=block.name, arguments=block.input or {}))
content = "".join(content_parts) if content_parts else None
finish_reason = "tool_calls" if tool_calls else "stop"
# Extract token usage
input_tokens = response.usage.input_tokens or 0
output_tokens = response.usage.output_tokens or 0
# Record metrics
metrics = get_metrics_collector()
duration = time.time() - start_time
metrics.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
duration=duration,
input_tokens=input_tokens,
output_tokens=output_tokens,
success=True,
)
# Record OpenTelemetry span
from hindsight_api.tracing import get_span_recorder
span_recorder = get_span_recorder()
# Convert LLMToolCall objects to dicts for span recording
tool_calls_dict = (
[{"id": tc.id, "name": tc.name, "arguments": tc.arguments} for tc in tool_calls]
if tool_calls
else None
)
span_recorder.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
messages=messages,
response_content=content,
input_tokens=input_tokens,
output_tokens=output_tokens,
duration=duration,
finish_reason=finish_reason,
error=None,
tool_calls=tool_calls_dict,
)
return LLMToolCallResult(
content=content,
tool_calls=tool_calls,
finish_reason=finish_reason,
input_tokens=input_tokens,
output_tokens=output_tokens,
)
except (APIConnectionError, APIStatusError) as e:
if isinstance(e, APIStatusError) and e.status_code in (401, 403):
raise
last_exception = e
if attempt < max_retries:
await asyncio.sleep(min(initial_backoff * (2**attempt), max_backoff))
continue
raise
if last_exception:
raise last_exception
raise RuntimeError("Anthropic tool call failed")
async def cleanup(self) -> None:
"""Clean up resources (close Anthropic client connections)."""
if hasattr(self, "_client") and self._client:
await self._client.close()
@@ -1,510 +0,0 @@
"""
Claude Code LLM provider using Claude Agent SDK.
This provider enables using Claude Pro/Max subscriptions for API calls
via the Claude CLI authentication. It uses the Claude Agent SDK which
automatically handles authentication via `claude auth login` credentials.
"""
import asyncio
import json
import logging
import time
from typing import Any
from hindsight_api.engine.llm_interface import LLMInterface, OutputTooLongError
from hindsight_api.engine.response_models import LLMToolCall, LLMToolCallResult, TokenUsage
from hindsight_api.metrics import get_metrics_collector
logger = logging.getLogger(__name__)
class ClaudeCodeLLM(LLMInterface):
"""
LLM provider using Claude Code authentication.
Authenticates using Claude Pro/Max credentials via `claude auth login`
and makes API calls through the Claude Agent SDK.
"""
def __init__(
self,
provider: str,
api_key: str, # Will be ignored, uses CLI auth
base_url: str,
model: str,
reasoning_effort: str = "low",
**kwargs: Any,
):
"""Initialize Claude Code LLM provider."""
super().__init__(provider, api_key, base_url, model, reasoning_effort, **kwargs)
# Verify Claude Agent SDK is available
try:
self._verify_claude_code_available()
logger.info("Claude Code: Using Claude Agent SDK (authentication via claude auth login)")
except Exception as e:
raise RuntimeError(
f"Failed to initialize Claude Code provider: {e}\n\n"
"To set up Claude Code authentication:\n"
"1. Install Claude Code CLI: npm install -g @anthropics/claude-code\n"
"2. Login with your Pro/Max plan: claude auth login\n"
"3. Verify authentication: claude --version\n\n"
"Or use a different provider (anthropic, openai, gemini) with API keys."
) from e
# Metrics collector is imported at module level
def _verify_claude_code_available(self) -> None:
"""
Verify that Claude Agent SDK can be imported and is properly configured.
Raises:
ImportError: If Claude Agent SDK is not installed.
RuntimeError: If Claude Code is not authenticated.
"""
try:
# Import Claude Agent SDK
# Reduce Claude Agent SDK logging verbosity
import logging as sdk_logging
from claude_agent_sdk import query # noqa: F401
sdk_logging.getLogger("claude_agent_sdk").setLevel(sdk_logging.WARNING)
sdk_logging.getLogger("claude_agent_sdk._internal").setLevel(sdk_logging.WARNING)
logger.debug("Claude Agent SDK imported successfully")
except ImportError as e:
raise ImportError(
"Claude Agent SDK not installed. Run: uv add claude-agent-sdk or pip install claude-agent-sdk"
) from e
# SDK will automatically check for authentication when first used
# No need to verify here - let it fail gracefully on first call with helpful error
async def verify_connection(self) -> None:
"""
Verify that the Claude Code provider is configured correctly by making a simple test call.
Raises:
RuntimeError: If the connection test fails.
"""
try:
test_messages = [{"role": "user", "content": "test"}]
await self.call(
messages=test_messages,
max_completion_tokens=10,
temperature=0.0,
scope="verification",
max_retries=0,
)
logger.info("Claude Code connection verified successfully")
except Exception as e:
logger.error(f"Claude Code connection verification failed: {e}")
raise RuntimeError(f"Failed to verify Claude Code connection: {e}") from e
async def call(
self,
messages: list[dict[str, str]],
response_format: Any | None = None,
max_completion_tokens: int | None = None,
temperature: float | None = None,
scope: str = "memory",
max_retries: int = 10,
initial_backoff: float = 1.0,
max_backoff: float = 60.0,
skip_validation: bool = False,
strict_schema: bool = False,
return_usage: bool = False,
) -> Any:
"""
Make an LLM API call with retry logic.
Args:
messages: List of message dicts with 'role' and 'content'.
response_format: Optional Pydantic model for structured output.
max_completion_tokens: Maximum tokens in response (ignored by Claude Agent SDK).
temperature: Sampling temperature (ignored by Claude Agent SDK).
scope: Scope identifier for tracking.
max_retries: Maximum retry attempts.
initial_backoff: Initial backoff time in seconds.
max_backoff: Maximum backoff time in seconds.
skip_validation: Return raw JSON without Pydantic validation.
strict_schema: Use strict JSON schema enforcement (not supported).
return_usage: If True, return tuple (result, TokenUsage) instead of just result.
Returns:
If return_usage=False: Parsed response if response_format is provided, otherwise text content.
If return_usage=True: Tuple of (result, TokenUsage) with estimated token counts.
Raises:
OutputTooLongError: If output exceeds token limits (not supported by Claude Agent SDK).
Exception: Re-raises API errors after retries exhausted.
"""
from claude_agent_sdk import AssistantMessage, ClaudeAgentOptions, TextBlock, query
start_time = time.time()
# Build system prompt
system_prompt = ""
user_content = ""
for msg in messages:
role = msg.get("role", "user")
content = msg.get("content", "")
if role == "system":
system_prompt += ("\n\n" + content) if system_prompt else content
elif role == "user":
user_content += ("\n\n" + content) if user_content else content
elif role == "assistant":
# Claude Agent SDK doesn't support multi-turn easily in query()
# For now, prepend assistant messages to user content
user_content += f"\n\n[Previous assistant response: {content}]"
# Add JSON schema instruction if response_format is provided
if response_format is not None and hasattr(response_format, "model_json_schema"):
schema = response_format.model_json_schema()
schema_instruction = (
f"\n\nYou must respond with valid JSON matching this schema:\n{json.dumps(schema, indent=2)}\n\n"
"Respond with ONLY the JSON, no markdown formatting."
)
user_content += schema_instruction
# Configure SDK options
options = ClaudeAgentOptions(
system_prompt=system_prompt if system_prompt else None,
max_turns=1, # Single-turn for API-style interactions
allowed_tools=[], # Disable tools for standard LLM calls
)
# Call Claude Agent SDK
last_exception = None
for attempt in range(max_retries + 1):
try:
# Collect streaming response
full_text = ""
async for message in query(prompt=user_content, options=options):
if isinstance(message, AssistantMessage):
for block in message.content:
if isinstance(block, TextBlock):
full_text += block.text
# Handle structured output
if response_format is not None:
# Models may wrap JSON in markdown
clean_text = full_text
if "```json" in full_text:
clean_text = full_text.split("```json")[1].split("```")[0].strip()
elif "```" in full_text:
clean_text = full_text.split("```")[1].split("```")[0].strip()
try:
json_data = json.loads(clean_text)
except json.JSONDecodeError as e:
logger.warning(f"Claude Code JSON parse error (attempt {attempt + 1}/{max_retries + 1}): {e}")
if attempt < max_retries:
backoff = min(initial_backoff * (2**attempt), max_backoff)
await asyncio.sleep(backoff)
last_exception = e
continue
raise
if skip_validation:
result = json_data
else:
result = response_format.model_validate(json_data)
else:
result = full_text
# Record metrics
duration = time.time() - start_time
metrics = get_metrics_collector()
# Estimate token usage (Claude Agent SDK doesn't report exact counts)
# Use character count / 4 as rough estimate (1 token ≈ 4 characters)
estimated_input = sum(len(m.get("content", "")) for m in messages) // 4
estimated_output = len(full_text) // 4
metrics.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
duration=duration,
input_tokens=estimated_input,
output_tokens=estimated_output,
success=True,
)
# Record trace span
from hindsight_api.tracing import get_span_recorder
span_recorder = get_span_recorder()
span_recorder.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
messages=messages,
response_content=result if isinstance(result, str) else json.dumps(result),
input_tokens=estimated_input,
output_tokens=estimated_output,
duration=duration,
finish_reason=None,
error=None,
)
# Log slow calls
if duration > 10.0:
logger.info(
f"slow llm call: scope={scope}, model={self.provider}/{self.model}, time={duration:.3f}s"
)
if return_usage:
token_usage = TokenUsage(
input_tokens=estimated_input,
output_tokens=estimated_output,
total_tokens=estimated_input + estimated_output,
)
return result, token_usage
return result
except Exception as e:
last_exception = e
# Check for authentication errors
error_str = str(e).lower()
if "auth" in error_str or "login" in error_str or "credential" in error_str:
logger.error(f"Claude Code authentication error: {e}")
raise RuntimeError(
f"Claude Code authentication failed: {e}\n\n"
"Run 'claude auth login' to authenticate with Claude Pro/Max."
) from e
if attempt < max_retries:
backoff = min(initial_backoff * (2**attempt), max_backoff)
logger.warning(f"Claude Code error (attempt {attempt + 1}/{max_retries + 1}): {e}")
await asyncio.sleep(backoff)
continue
else:
logger.error(f"Claude Code error after {max_retries + 1} attempts: {e}")
raise
if last_exception:
raise last_exception
raise RuntimeError("Claude Code call failed after all retries")
async def call_with_tools(
self,
messages: list[dict[str, Any]],
tools: list[dict[str, Any]],
max_completion_tokens: int | None = None,
temperature: float | None = None,
scope: str = "tools",
max_retries: int = 5,
initial_backoff: float = 1.0,
max_backoff: float = 30.0,
tool_choice: str | dict[str, Any] = "auto",
) -> LLMToolCallResult:
"""
Make an LLM API call with tool/function calling support using Claude Agent SDK.
This implementation uses ClaudeSDKClient (not query()) because custom tools via
SDK MCP servers are only supported with the client. Tools are converted from OpenAI
format to SDK MCP tools, and tool names are formatted as mcp__hindsight_tools__{name}.
Args:
messages: List of message dicts. Can include tool results with role='tool'.
tools: List of tool definitions in OpenAI format.
max_completion_tokens: Maximum tokens in response (not used by Claude Agent SDK).
temperature: Sampling temperature (not used by Claude Agent SDK).
scope: Scope identifier for tracking.
max_retries: Maximum retry attempts.
initial_backoff: Initial backoff time in seconds.
max_backoff: Maximum backoff time in seconds.
tool_choice: How to choose tools (not used by Claude Agent SDK).
Returns:
LLMToolCallResult with content and/or tool_calls.
"""
from claude_agent_sdk import (
AssistantMessage,
ClaudeAgentOptions,
ClaudeSDKClient,
SdkMcpTool,
TextBlock,
ToolUseBlock,
create_sdk_mcp_server,
)
start_time = time.time()
# Convert OpenAI tool format to Claude Agent SDK SdkMcpTool format
sdk_tools: list[SdkMcpTool] = []
tool_names: list[str] = []
for tool in tools:
func = tool.get("function", {})
tool_name = func.get("name", "")
tool_description = func.get("description", "")
parameters = func.get("parameters", {})
# Create a handler with proper closure to avoid transport issues
def make_handler(name: str):
async def handler(args: dict[str, Any]) -> dict[str, Any]:
# Return immediately with success - tool execution happens externally
return {
"content": [
{
"type": "text",
"text": f"[Tool {name} called successfully]",
}
]
}
return handler
sdk_tools.append(
SdkMcpTool(
name=tool_name,
description=tool_description,
input_schema=parameters,
handler=make_handler(tool_name),
)
)
tool_names.append(tool_name)
# Create an MCP server with the tools
mcp_server = create_sdk_mcp_server(
name="hindsight_tools",
version="1.0.0",
tools=sdk_tools if sdk_tools else None,
)
# Build system prompt and user content from messages
system_prompt = ""
user_content = ""
for msg in messages:
role = msg.get("role", "user")
content = msg.get("content", "")
if role == "system":
system_prompt += ("\n\n" + content) if system_prompt else content
elif role == "user":
user_content += ("\n\n" + content) if user_content else content
elif role == "assistant":
# Include previous assistant messages as context
user_content += f"\n\n[Previous assistant response: {content}]"
elif role == "tool":
# Tool results are already in tool_results_map, append to user context
tool_call_id = msg.get("tool_call_id", "")
user_content += f"\n\n[Tool result for {tool_call_id}: {content}]"
# Format tool names for SDK MCP servers: mcp__{server_name}__{tool_name}
# This is required by the Claude Agent SDK for MCP server tools
allowed_tool_names = [f"mcp__hindsight_tools__{name}" for name in tool_names]
# Configure SDK options with MCP server
options = ClaudeAgentOptions(
system_prompt=system_prompt if system_prompt else None,
max_turns=1, # Single-turn for API-style interactions
mcp_servers={"hindsight_tools": mcp_server} if sdk_tools else {},
allowed_tools=allowed_tool_names if allowed_tool_names else [],
)
# Call Claude Agent SDK with retry logic
last_exception = None
for attempt in range(max_retries + 1):
try:
full_text = ""
tool_calls: list[LLMToolCall] = []
# Use ClaudeSDKClient for tool calling support
# Note: query() does NOT support custom tools, only ClaudeSDKClient does
async with ClaudeSDKClient(options=options) as client:
# Send the query
await client.query(user_content)
# Receive response
async for message in client.receive_response():
if isinstance(message, AssistantMessage):
for block in message.content:
if isinstance(block, TextBlock):
full_text += block.text
elif isinstance(block, ToolUseBlock):
# SDK returns tool names with MCP prefix (mcp__hindsight_tools__{name})
# Strip the prefix to return original tool name expected by caller
tool_name = block.name
if tool_name.startswith("mcp__hindsight_tools__"):
tool_name = tool_name.replace("mcp__hindsight_tools__", "", 1)
tool_calls.append(
LLMToolCall(
id=block.id,
name=tool_name,
arguments=block.input,
)
)
# Record metrics
duration = time.time() - start_time
metrics = get_metrics_collector()
# Estimate token usage (Claude Agent SDK doesn't report exact counts)
estimated_input = sum(len(m.get("content", "")) for m in messages) // 4
estimated_output = len(full_text) // 4
metrics.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
duration=duration,
input_tokens=estimated_input,
output_tokens=estimated_output,
success=True,
)
# Log slow calls
if duration > 10.0:
logger.info(
f"slow llm call: scope={scope}, model={self.provider}/{self.model}, time={duration:.3f}s"
)
return LLMToolCallResult(
content=full_text if full_text else None,
tool_calls=tool_calls,
finish_reason="tool_calls" if tool_calls else "stop",
input_tokens=estimated_input,
output_tokens=estimated_output,
)
except Exception as e:
last_exception = e
# Check for authentication errors
error_str = str(e).lower()
if "auth" in error_str or "login" in error_str or "credential" in error_str:
logger.error(f"Claude Code authentication error: {e}")
raise RuntimeError(
f"Claude Code authentication failed: {e}\n\n"
"Run 'claude auth login' to authenticate with Claude Pro/Max."
) from e
if attempt < max_retries:
backoff = min(initial_backoff * (2**attempt), max_backoff)
logger.warning(f"Claude Code tool call error (attempt {attempt + 1}/{max_retries + 1}): {e}")
await asyncio.sleep(backoff)
continue
else:
logger.error(f"Claude Code tool call error after {max_retries + 1} attempts: {e}")
raise
if last_exception:
raise last_exception
raise RuntimeError("Claude Code tool call failed after all retries")
async def cleanup(self) -> None:
"""Clean up resources (no HTTP client to close for Claude Agent SDK)."""
pass
@@ -1,621 +0,0 @@
"""
OpenAI Codex LLM provider using ChatGPT Plus/Pro OAuth authentication.
This provider enables using ChatGPT Plus/Pro subscriptions for API calls
without separate OpenAI Platform API credits. It uses OAuth tokens from
~/.codex/auth.json and communicates with the ChatGPT backend API.
"""
import asyncio
import json
import logging
import os
import time
import uuid
from pathlib import Path
from typing import Any
import httpx
from hindsight_api.engine.llm_interface import LLMInterface, OutputTooLongError
from hindsight_api.engine.response_models import LLMToolCall, LLMToolCallResult, TokenUsage
from hindsight_api.metrics import get_metrics_collector
logger = logging.getLogger(__name__)
class CodexLLM(LLMInterface):
"""
LLM provider using OpenAI Codex OAuth authentication.
Authenticates using ChatGPT Plus/Pro credentials stored in ~/.codex/auth.json
and makes API calls to chatgpt.com/backend-api/codex/responses.
"""
def __init__(
self,
provider: str,
api_key: str, # Will be ignored, reads from ~/.codex/auth.json
base_url: str,
model: str,
reasoning_effort: str = "low",
**kwargs: Any,
):
"""Initialize Codex LLM provider."""
super().__init__(provider, api_key, base_url, model, reasoning_effort, **kwargs)
# Load Codex OAuth credentials
try:
self.access_token, self.account_id = self._load_codex_auth()
logger.info(f"Loaded Codex OAuth credentials for account: {self.account_id}")
except Exception as e:
raise RuntimeError(
f"Failed to load Codex OAuth credentials from ~/.codex/auth.json: {e}\n\n"
"To set up Codex authentication:\n"
"1. Install Codex CLI: npm install -g @openai/codex\n"
"2. Login: codex auth login\n"
"3. Verify: ls ~/.codex/auth.json\n\n"
"Or use a different provider (openai, anthropic, gemini) with API keys."
) from e
# Use ChatGPT backend API endpoint
if not self.base_url:
self.base_url = "https://chatgpt.com/backend-api"
# Normalize model name (strip openai/ prefix if present)
if self.model.startswith("openai/"):
self.model = self.model[len("openai/") :]
# Map reasoning effort to Codex reasoning summary format
# Codex supports: "auto", "concise", "detailed"
self.reasoning_summary = self._map_reasoning_effort(reasoning_effort)
# HTTP client for SSE streaming
self._client = httpx.AsyncClient(timeout=120.0)
def _load_codex_auth(self) -> tuple[str, str]:
"""
Load OAuth credentials from ~/.codex/auth.json.
Returns:
Tuple of (access_token, account_id).
Raises:
FileNotFoundError: If auth file doesn't exist.
ValueError: If auth file is invalid.
"""
auth_file = Path.home() / ".codex" / "auth.json"
if not auth_file.exists():
raise FileNotFoundError(
f"Codex auth file not found: {auth_file}\nRun 'codex auth login' to authenticate with ChatGPT Plus/Pro."
)
with open(auth_file) as f:
data = json.load(f)
# Validate auth structure
auth_mode = data.get("auth_mode")
if auth_mode != "chatgpt":
raise ValueError(f"Expected auth_mode='chatgpt', got: {auth_mode}")
tokens = data.get("tokens", {})
access_token = tokens.get("access_token")
account_id = tokens.get("account_id")
if not access_token:
raise ValueError("No access_token found in Codex auth file. Run 'codex auth login' again.")
return access_token, account_id
def _map_reasoning_effort(self, effort: str) -> str:
"""
Map standard reasoning effort to Codex reasoning summary format.
Args:
effort: Standard effort level ("low", "medium", "high", "xhigh").
Returns:
Codex reasoning summary: "concise", "detailed", or "auto".
"""
mapping = {
"low": "concise",
"medium": "auto",
"high": "detailed",
"xhigh": "detailed",
}
return mapping.get(effort.lower(), "auto")
async def verify_connection(self) -> None:
"""Verify Codex connection by making a simple test call."""
try:
logger.info(f"Verifying Codex LLM: model={self.model}, account={self.account_id}...")
await self.call(
messages=[{"role": "user", "content": "Say 'ok'"}],
max_completion_tokens=10,
max_retries=2,
initial_backoff=0.5,
max_backoff=2.0,
scope="verification",
)
logger.info(f"Codex LLM verified: {self.model}")
except Exception as e:
raise RuntimeError(f"Codex LLM connection verification failed for {self.model}: {e}") from e
async def call(
self,
messages: list[dict[str, str]],
response_format: Any | None = None,
max_completion_tokens: int | None = None,
temperature: float | None = None,
scope: str = "memory",
max_retries: int = 10,
initial_backoff: float = 1.0,
max_backoff: float = 60.0,
skip_validation: bool = False,
strict_schema: bool = False,
return_usage: bool = False,
) -> Any:
"""Make API call to Codex backend with SSE streaming."""
start_time = time.time()
# Prepare system instructions
system_instruction = ""
user_messages = []
for msg in messages:
role = msg.get("role", "user")
content = msg.get("content", "")
if role == "system":
system_instruction += ("\n\n" + content) if system_instruction else content
else:
user_messages.append(msg)
# Add JSON schema instruction if response_format is provided
if response_format is not None and hasattr(response_format, "model_json_schema"):
schema = response_format.model_json_schema()
schema_msg = f"\n\nYou must respond with valid JSON matching this schema:\n{json.dumps(schema, indent=2)}"
system_instruction += schema_msg
# gpt-5.2-codex only supports "detailed" reasoning summary
reasoning_summary = "detailed" if "5.2" in self.model else self.reasoning_summary
# Build Codex request payload
payload = {
"model": self.model,
"instructions": system_instruction,
"input": [
{
"type": "message",
"role": msg.get("role", "user"),
"content": msg.get("content", ""),
}
for msg in user_messages
],
"tools": [],
"tool_choice": "auto",
"parallel_tool_calls": True,
"reasoning": {"summary": reasoning_summary},
"store": False, # Codex uses stateless mode
"stream": True, # SSE streaming
"include": ["reasoning.encrypted_content"],
"prompt_cache_key": str(uuid.uuid4()),
}
headers = {
"Authorization": f"Bearer {self.access_token}",
"Content-Type": "application/json",
"OpenAI-Account-ID": self.account_id,
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)",
"Origin": "https://chatgpt.com",
}
url = f"{self.base_url}/codex/responses"
last_exception = None
for attempt in range(max_retries + 1):
try:
response = await self._client.post(url, json=payload, headers=headers, timeout=120.0)
response.raise_for_status()
# Parse SSE stream
content = await self._parse_sse_stream(response)
# Handle structured output
if response_format is not None:
# Models may wrap JSON in markdown
clean_content = content
if "```json" in content:
clean_content = content.split("```json")[1].split("```")[0].strip()
elif "```" in content:
clean_content = content.split("```")[1].split("```")[0].strip()
try:
json_data = json.loads(clean_content)
except json.JSONDecodeError as e:
logger.warning(f"Codex JSON parse error (attempt {attempt + 1}/{max_retries + 1}): {e}")
if attempt < max_retries:
backoff = min(initial_backoff * (2**attempt), max_backoff)
await asyncio.sleep(backoff)
last_exception = e
continue
raise
if skip_validation:
result = json_data
else:
result = response_format.model_validate(json_data)
else:
result = content
# Record metrics
duration = time.time() - start_time
metrics = get_metrics_collector()
metrics.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
duration=duration,
input_tokens=0, # Codex doesn't report token counts in SSE
output_tokens=0,
success=True,
)
# Record trace span
from hindsight_api.tracing import get_span_recorder
# Estimate tokens for tracing
estimated_input = sum(len(m.get("content", "")) for m in messages) // 4
estimated_output = len(content) // 4
span_recorder = get_span_recorder()
span_recorder.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
messages=messages,
response_content=result if isinstance(result, str) else json.dumps(result),
input_tokens=estimated_input,
output_tokens=estimated_output,
duration=duration,
finish_reason=None,
error=None,
)
if return_usage:
# Codex doesn't provide token counts, estimate based on content
estimated_input = sum(len(m.get("content", "")) for m in messages) // 4
estimated_output = len(content) // 4
token_usage = TokenUsage(
input_tokens=estimated_input,
output_tokens=estimated_output,
total_tokens=estimated_input + estimated_output,
)
return result, token_usage
return result
except httpx.HTTPStatusError as e:
last_exception = e
status_code = e.response.status_code
# Fast fail on auth errors
if status_code in (401, 403):
logger.error(f"Codex auth error (HTTP {status_code}): {e.response.text[:200]}")
raise RuntimeError(
"Codex authentication failed. Your OAuth token may have expired.\n"
"Run 'codex auth login' to re-authenticate."
) from e
# Log the actual error message from the API
error_detail = e.response.text[:500] if hasattr(e.response, "text") else str(e)
if attempt < max_retries:
backoff = min(initial_backoff * (2**attempt), max_backoff)
logger.warning(
f"Codex HTTP error {status_code} (attempt {attempt + 1}/{max_retries + 1}): {error_detail}"
)
await asyncio.sleep(backoff)
continue
else:
logger.error(
f"Codex HTTP error after {max_retries + 1} attempts: Status {status_code}, Detail: {error_detail}"
)
raise
except httpx.RequestError as e:
last_exception = e
if attempt < max_retries:
backoff = min(initial_backoff * (2**attempt), max_backoff)
logger.warning(f"Codex connection error (attempt {attempt + 1}/{max_retries + 1}): {e}")
await asyncio.sleep(backoff)
continue
else:
logger.error(f"Codex connection error after {max_retries + 1} attempts: {e}")
raise
except Exception as e:
logger.error(f"Unexpected Codex error: {type(e).__name__}: {e}")
raise
if last_exception:
raise last_exception
raise RuntimeError("Codex call failed after all retries")
async def _parse_sse_stream(self, response: httpx.Response) -> str:
"""
Parse Server-Sent Events (SSE) stream from Codex API.
Args:
response: HTTP response with SSE stream.
Returns:
Extracted text content from stream.
"""
full_text = ""
event_type = None
async for line in response.aiter_lines():
if not line:
continue
# Track event type
if line.startswith("event: "):
event_type = line[7:]
# Parse data
elif line.startswith("data: "):
data_str = line[6:]
if data_str == "[DONE]":
break
try:
data = json.loads(data_str)
# Extract content based on event type
if event_type == "response.text.delta" and "delta" in data:
full_text += data["delta"]
elif event_type == "response.content_part.delta" and "delta" in data:
full_text += data["delta"]
# Check for item content
elif "item" in data:
item = data["item"]
if "content" in item:
content = item["content"]
if isinstance(content, list):
for part in content:
if isinstance(part, dict) and "text" in part:
full_text += part["text"]
elif isinstance(content, str):
full_text += content
except json.JSONDecodeError:
# Skip malformed JSON events
pass
return full_text
async def call_with_tools(
self,
messages: list[dict[str, Any]],
tools: list[dict[str, Any]],
max_completion_tokens: int | None = None,
temperature: float | None = None,
scope: str = "tools",
max_retries: int = 5,
initial_backoff: float = 1.0,
max_backoff: float = 30.0,
tool_choice: str | dict[str, Any] = "auto",
) -> LLMToolCallResult:
"""
Make API call with tool calling support.
Parses Codex SSE stream to extract tool calls from response.output_item.done events.
Tools are converted from OpenAI format to Codex format (flat structure at top level).
Args:
messages: List of message dicts. Can include tool results with role='tool'.
tools: List of tool definitions in OpenAI format.
max_completion_tokens: Maximum tokens in response.
temperature: Sampling temperature.
scope: Scope identifier for tracking.
max_retries: Maximum retry attempts.
initial_backoff: Initial backoff time in seconds.
max_backoff: Maximum backoff time in seconds.
tool_choice: How to choose tools - "auto", "none", "required", or specific function.
Returns:
LLMToolCallResult with content and/or tool_calls.
"""
start_time = time.time()
# Prepare system instructions
system_instruction = ""
user_messages = []
for msg in messages:
role = msg.get("role", "user")
content = msg.get("content", "")
if role == "system":
system_instruction += ("\n\n" + content) if system_instruction else content
elif role == "tool":
# Handle tool results
user_messages.append(
{
"type": "message",
"role": "user",
"content": f"Tool result: {content}",
}
)
else:
user_messages.append(
{
"type": "message",
"role": role,
"content": content,
}
)
# Convert tools to Codex format
# Codex expects tools with type and name/description/parameters at top level
codex_tools = []
for tool in tools:
func = tool.get("function", {})
codex_tools.append(
{
"type": "function",
"name": func.get("name", ""),
"description": func.get("description", ""),
"parameters": func.get("parameters", {}),
}
)
# gpt-5.2-codex only supports "detailed" reasoning summary
reasoning_summary = "detailed" if "5.2" in self.model else self.reasoning_summary
payload = {
"model": self.model,
"instructions": system_instruction,
"input": user_messages,
"tools": codex_tools,
"tool_choice": tool_choice,
"parallel_tool_calls": True,
"reasoning": {"summary": reasoning_summary},
"store": False,
"stream": True,
"include": ["reasoning.encrypted_content"],
"prompt_cache_key": str(uuid.uuid4()),
}
headers = {
"Authorization": f"Bearer {self.access_token}",
"Content-Type": "application/json",
"OpenAI-Account-ID": self.account_id,
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)",
"Origin": "https://chatgpt.com",
}
url = f"{self.base_url}/codex/responses"
# Debug logging for troubleshooting
logger.debug(f"Codex tool call request: url={url}, model={payload['model']}, tools={len(codex_tools)}")
try:
response = await self._client.post(url, json=payload, headers=headers, timeout=120.0)
# Log response details on error
if response.status_code != 200:
logger.error(f"Codex API error {response.status_code}: {response.text[:500]}")
response.raise_for_status()
# Parse SSE for tool calls and content
content, tool_calls = await self._parse_sse_tool_stream(response)
duration = time.time() - start_time
metrics = get_metrics_collector()
metrics.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
duration=duration,
input_tokens=0,
output_tokens=0,
success=True,
)
# Record OpenTelemetry span
from hindsight_api.tracing import get_span_recorder
span_recorder = get_span_recorder()
# Convert LLMToolCall objects to dicts for span recording
tool_calls_dict = (
[{"id": tc.id, "name": tc.name, "arguments": tc.arguments} for tc in tool_calls] if tool_calls else None
)
span_recorder.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
messages=messages,
response_content=content,
input_tokens=0, # Codex doesn't provide token counts
output_tokens=0,
duration=duration,
finish_reason="tool_calls" if tool_calls else "stop",
error=None,
tool_calls=tool_calls_dict,
)
return LLMToolCallResult(
content=content,
tool_calls=tool_calls,
finish_reason="tool_calls" if tool_calls else "stop",
input_tokens=0,
output_tokens=0,
)
except Exception as e:
logger.error(f"Codex tool call error: {e}")
raise
async def _parse_sse_tool_stream(self, response: httpx.Response) -> tuple[str | None, list[LLMToolCall]]:
"""
Parse SSE stream for tool calls and content.
Returns:
Tuple of (content, tool_calls).
"""
content = ""
tool_calls: list[LLMToolCall] = []
event_type = None
async for line in response.aiter_lines():
if not line:
continue
if line.startswith("event: "):
event_type = line[7:]
elif line.startswith("data: "):
data_str = line[6:]
if data_str == "[DONE]":
break
try:
data = json.loads(data_str)
# Extract text content
if event_type == "response.text.delta" and "delta" in data:
content += data["delta"]
# Extract completed tool calls from response.output_item.done
elif event_type == "response.output_item.done":
item = data.get("item", {})
if item.get("type") == "function_call" and item.get("status") == "completed":
tool_name = item.get("name", "")
arguments_str = item.get("arguments", "{}")
call_id = item.get("call_id", "")
try:
arguments = json.loads(arguments_str)
except json.JSONDecodeError:
logger.warning(f"Failed to parse tool arguments: {arguments_str}")
arguments = {}
tool_calls.append(
LLMToolCall(
id=call_id,
name=tool_name,
arguments=arguments,
)
)
except json.JSONDecodeError as e:
logger.warning(f"Failed to parse SSE data: {e}, data_str: {data_str[:200]}")
return content if content else None, tool_calls
async def cleanup(self) -> None:
"""Clean up HTTP client."""
await self._client.aclose()
@@ -1,550 +0,0 @@
"""
Google Gemini/VertexAI LLM provider.
This provider supports both:
1. Gemini API (api.generativeai.google.com) with API key authentication
2. Vertex AI with service account or Application Default Credentials (ADC)
"""
import asyncio
import json
import logging
import os
import time
from typing import Any
from google import genai
from google.genai import errors as genai_errors
from google.genai import types as genai_types
from hindsight_api.engine.llm_interface import LLMInterface, OutputTooLongError
from hindsight_api.engine.response_models import LLMToolCall, LLMToolCallResult, TokenUsage
from hindsight_api.metrics import get_metrics_collector
logger = logging.getLogger(__name__)
# Vertex AI imports (optional)
try:
import google.auth
from google.oauth2 import service_account
VERTEXAI_AVAILABLE = True
except ImportError:
VERTEXAI_AVAILABLE = False
class GeminiLLM(LLMInterface):
"""
LLM provider for Google Gemini and Vertex AI.
Supports:
- Gemini API: provider="gemini", requires api_key
- Vertex AI: provider="vertexai", requires project_id and region, uses ADC or service account
"""
def __init__(
self,
provider: str,
api_key: str,
base_url: str,
model: str,
reasoning_effort: str = "low",
**kwargs: Any,
):
"""Initialize Gemini/VertexAI LLM provider."""
super().__init__(provider, api_key, base_url, model, reasoning_effort, **kwargs)
self._client = None
self._is_vertexai = self.provider == "vertexai"
if self._is_vertexai:
self._init_vertexai(**kwargs)
else:
self._init_gemini()
def _init_gemini(self) -> None:
"""Initialize Gemini API client."""
if not self.api_key:
raise ValueError("Gemini provider requires api_key")
self._client = genai.Client(api_key=self.api_key)
logger.info(f"Gemini API: model={self.model}")
def _init_vertexai(self, **kwargs: Any) -> None:
"""Initialize Vertex AI client with project, region, and credentials."""
# Extract Vertex AI config from kwargs
project_id = kwargs.get("vertexai_project_id")
region = kwargs.get("vertexai_region", "us-central1")
service_account_key = kwargs.get("vertexai_service_account_key")
credentials = kwargs.get("vertexai_credentials") # Pre-loaded credentials object
if not project_id:
raise ValueError(
"HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID is required for Vertex AI provider. "
"Set it to your GCP project ID."
)
auth_method = "ADC"
# Use pre-loaded credentials if provided (passed from LLMProvider)
if credentials is not None:
auth_method = "service_account"
# Otherwise, load explicit service account credentials if path provided
elif service_account_key:
if not VERTEXAI_AVAILABLE:
raise ValueError(
"Vertex AI service account auth requires 'google-auth' package. "
"Install with: pip install google-auth"
)
credentials = service_account.Credentials.from_service_account_file(
service_account_key,
scopes=["https://www.googleapis.com/auth/cloud-platform"],
)
auth_method = "service_account"
logger.info(f"Vertex AI: Using service account key: {service_account_key}")
# Strip google/ prefix from model name — native SDK uses bare names
# e.g. "google/gemini-2.0-flash-lite-001" -> "gemini-2.0-flash-lite-001"
if self.model.startswith("google/"):
self.model = self.model[len("google/") :]
# Create Vertex AI client
client_kwargs: dict[str, Any] = {
"vertexai": True,
"project": project_id,
"location": region,
}
if credentials is not None:
client_kwargs["credentials"] = credentials
self._client = genai.Client(**client_kwargs)
logger.info(f"Vertex AI: project={project_id}, region={region}, model={self.model}, auth={auth_method}")
async def verify_connection(self) -> None:
"""
Verify that the Gemini/VertexAI provider is configured correctly.
Raises:
RuntimeError: If the connection test fails.
"""
try:
logger.info(f"Verifying {self.provider.upper()}: model={self.model}...")
await self.call(
messages=[{"role": "user", "content": "Say 'ok'"}],
max_completion_tokens=100,
max_retries=2,
initial_backoff=0.5,
max_backoff=2.0,
scope="verification",
)
logger.info(f"{self.provider.upper()} connection verified successfully")
except Exception as e:
raise RuntimeError(f"Failed to verify {self.provider.upper()} connection: {e}") from e
async def call(
self,
messages: list[dict[str, str]],
response_format: Any | None = None,
max_completion_tokens: int | None = None,
temperature: float | None = None,
scope: str = "memory",
max_retries: int = 10,
initial_backoff: float = 1.0,
max_backoff: float = 60.0,
skip_validation: bool = False,
strict_schema: bool = False,
return_usage: bool = False,
) -> Any:
"""
Make a Gemini/VertexAI API call with retry logic.
Args:
messages: List of message dicts with 'role' and 'content'.
response_format: Optional Pydantic model for structured output.
max_completion_tokens: Maximum tokens in response (not supported by Gemini).
temperature: Sampling temperature (0.0-2.0).
scope: Scope identifier for tracking.
max_retries: Maximum retry attempts.
initial_backoff: Initial backoff time in seconds.
max_backoff: Maximum backoff time in seconds.
skip_validation: Return raw JSON without Pydantic validation.
strict_schema: Use strict JSON schema enforcement (not supported by Gemini).
return_usage: If True, return tuple (result, TokenUsage).
Returns:
If return_usage=False: Parsed response if response_format provided, else text.
If return_usage=True: Tuple of (result, TokenUsage).
"""
start_time = time.time()
# Convert OpenAI-style messages to Gemini format
system_instruction = None
gemini_contents = []
for msg in messages:
role = msg.get("role", "user")
content = msg.get("content", "")
if role == "system":
if system_instruction:
system_instruction += "\n\n" + content
else:
system_instruction = content
elif role == "assistant":
gemini_contents.append(genai_types.Content(role="model", parts=[genai_types.Part(text=content)]))
else:
gemini_contents.append(genai_types.Content(role="user", parts=[genai_types.Part(text=content)]))
# Add JSON schema instruction if response_format is provided
if response_format is not None and hasattr(response_format, "model_json_schema"):
schema = response_format.model_json_schema()
schema_msg = f"\n\nYou must respond with valid JSON matching this schema:\n{json.dumps(schema, indent=2)}"
if system_instruction:
system_instruction += schema_msg
else:
system_instruction = schema_msg
# Build generation config
config_kwargs: dict[str, Any] = {}
if system_instruction:
config_kwargs["system_instruction"] = system_instruction
if response_format is not None:
config_kwargs["response_mime_type"] = "application/json"
config_kwargs["response_schema"] = response_format
if temperature is not None:
config_kwargs["temperature"] = temperature
generation_config = genai_types.GenerateContentConfig(**config_kwargs) if config_kwargs else None
last_exception = None
for attempt in range(max_retries + 1):
try:
response = await self._client.aio.models.generate_content(
model=self.model,
contents=gemini_contents,
config=generation_config,
)
content = response.text
# Handle empty response
if content is None:
block_reason = None
if hasattr(response, "candidates") and response.candidates:
candidate = response.candidates[0]
if hasattr(candidate, "finish_reason"):
block_reason = candidate.finish_reason
if attempt < max_retries:
logger.warning(f"Gemini returned empty response (reason: {block_reason}), retrying...")
backoff = min(initial_backoff * (2**attempt), max_backoff)
await asyncio.sleep(backoff)
continue
else:
raise RuntimeError(f"Gemini returned empty response after {max_retries + 1} attempts")
# Parse structured output if requested
if response_format is not None:
json_data = json.loads(content)
if skip_validation:
result = json_data
else:
result = response_format.model_validate(json_data)
else:
result = content
# Extract token usage
input_tokens = 0
output_tokens = 0
if hasattr(response, "usage_metadata") and response.usage_metadata:
usage = response.usage_metadata
input_tokens = usage.prompt_token_count or 0
output_tokens = usage.candidates_token_count or 0
# Record metrics
duration = time.time() - start_time
metrics = get_metrics_collector()
metrics.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
duration=duration,
input_tokens=input_tokens,
output_tokens=output_tokens,
success=True,
)
# Record trace span
from hindsight_api.tracing import get_span_recorder
finish_reason = None
if hasattr(response, "candidates") and response.candidates:
if hasattr(response.candidates[0], "finish_reason"):
finish_reason = str(response.candidates[0].finish_reason)
span_recorder = get_span_recorder()
from hindsight_api.tracing import _serialize_for_span
span_recorder.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
messages=messages,
response_content=_serialize_for_span(result),
input_tokens=input_tokens,
output_tokens=output_tokens,
duration=duration,
finish_reason=finish_reason,
error=None,
)
# Log slow calls
if duration > 10.0 and input_tokens > 0:
logger.info(
f"slow llm call: scope={scope}, model={self.provider}/{self.model}, "
f"input_tokens={input_tokens}, output_tokens={output_tokens}, "
f"time={duration:.3f}s"
)
if return_usage:
token_usage = TokenUsage(
input_tokens=input_tokens,
output_tokens=output_tokens,
total_tokens=input_tokens + output_tokens,
)
return result, token_usage
return result
except json.JSONDecodeError as e:
last_exception = e
if attempt < max_retries:
logger.warning("Gemini returned invalid JSON, retrying...")
backoff = min(initial_backoff * (2**attempt), max_backoff)
await asyncio.sleep(backoff)
continue
else:
logger.error(f"Gemini returned invalid JSON after {max_retries + 1} attempts")
raise
except genai_errors.APIError as e:
# Fast fail on auth errors - these won't recover with retries
if e.code in (401, 403):
logger.error(f"Gemini auth error (HTTP {e.code}), not retrying: {str(e)}")
raise
# Retry on retryable errors (rate limits, server errors, client errors)
if e.code in (400, 429, 500, 502, 503, 504) or (e.code and e.code >= 500):
last_exception = e
if attempt < max_retries:
backoff = min(initial_backoff * (2**attempt), max_backoff)
jitter = backoff * 0.2 * (2 * (time.time() % 1) - 1)
await asyncio.sleep(backoff + jitter)
else:
logger.error(f"Gemini API error after {max_retries + 1} attempts: {str(e)}")
raise
else:
logger.error(f"Gemini API error: {type(e).__name__}: {str(e)}")
raise
except Exception as e:
logger.error(f"Unexpected error during Gemini call: {type(e).__name__}: {str(e)}")
raise
if last_exception:
raise last_exception
raise RuntimeError("Gemini call failed after all retries")
async def call_with_tools(
self,
messages: list[dict[str, Any]],
tools: list[dict[str, Any]],
max_completion_tokens: int | None = None,
temperature: float | None = None,
scope: str = "tools",
max_retries: int = 5,
initial_backoff: float = 1.0,
max_backoff: float = 30.0,
tool_choice: str | dict[str, Any] = "auto",
) -> LLMToolCallResult:
"""
Make a Gemini/VertexAI API call with tool/function calling support.
Args:
messages: List of message dicts. Can include tool results with role='tool'.
tools: List of tool definitions in OpenAI format.
max_completion_tokens: Maximum tokens (not supported by Gemini).
temperature: Sampling temperature.
scope: Scope identifier for tracking.
max_retries: Maximum retry attempts.
initial_backoff: Initial backoff time in seconds.
max_backoff: Maximum backoff time in seconds.
tool_choice: How to choose tools (Gemini uses "auto" only).
Returns:
LLMToolCallResult with content and/or tool_calls.
"""
start_time = time.time()
# Convert tools to Gemini format
gemini_tools = []
for tool in tools:
func = tool.get("function", {})
gemini_tools.append(
genai_types.Tool(
function_declarations=[
genai_types.FunctionDeclaration(
name=func.get("name", ""),
description=func.get("description", ""),
parameters=func.get("parameters"),
)
]
)
)
# Convert messages
system_instruction = None
gemini_contents = []
for msg in messages:
role = msg.get("role", "user")
content = msg.get("content", "")
if role == "system":
system_instruction = (system_instruction + "\n\n" + content) if system_instruction else content
elif role == "tool":
# Gemini uses function_response
gemini_contents.append(
genai_types.Content(
role="user",
parts=[
genai_types.Part(
function_response=genai_types.FunctionResponse(
name=msg.get("name", ""),
response={"result": content},
)
)
],
)
)
elif role == "assistant":
gemini_contents.append(genai_types.Content(role="model", parts=[genai_types.Part(text=content)]))
else:
gemini_contents.append(genai_types.Content(role="user", parts=[genai_types.Part(text=content)]))
config_kwargs: dict[str, Any] = {"tools": gemini_tools}
if system_instruction:
config_kwargs["system_instruction"] = system_instruction
if temperature is not None:
config_kwargs["temperature"] = temperature
config = genai_types.GenerateContentConfig(**config_kwargs)
last_exception = None
for attempt in range(max_retries + 1):
try:
response = await self._client.aio.models.generate_content(
model=self.model,
contents=gemini_contents,
config=config,
)
# Extract content and tool calls
content = None
tool_calls: list[LLMToolCall] = []
if response.candidates and response.candidates[0].content:
parts = response.candidates[0].content.parts
if parts:
for part in parts:
if hasattr(part, "text") and part.text:
content = part.text
if hasattr(part, "function_call") and part.function_call:
fc = part.function_call
tool_calls.append(
LLMToolCall(
id=f"gemini_{len(tool_calls)}",
name=fc.name,
arguments=dict(fc.args) if fc.args else {},
)
)
finish_reason = "tool_calls" if tool_calls else "stop"
# Extract token usage
input_tokens = 0
output_tokens = 0
if response.usage_metadata:
input_tokens = response.usage_metadata.prompt_token_count or 0
output_tokens = response.usage_metadata.candidates_token_count or 0
# Record metrics
duration = time.time() - start_time
metrics = get_metrics_collector()
metrics.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
duration=duration,
input_tokens=input_tokens,
output_tokens=output_tokens,
success=True,
)
# Record OpenTelemetry span
from hindsight_api.tracing import get_span_recorder
span_recorder = get_span_recorder()
# Convert LLMToolCall objects to dicts for span recording
tool_calls_dict = (
[{"id": tc.id, "name": tc.name, "arguments": tc.arguments} for tc in tool_calls]
if tool_calls
else None
)
span_recorder.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
messages=messages,
response_content=content,
input_tokens=input_tokens,
output_tokens=output_tokens,
duration=duration,
finish_reason=finish_reason,
error=None,
tool_calls=tool_calls_dict,
)
return LLMToolCallResult(
content=content,
tool_calls=tool_calls,
finish_reason=finish_reason,
input_tokens=input_tokens,
output_tokens=output_tokens,
)
except genai_errors.APIError as e:
# Fast fail on auth errors
if e.code in (401, 403):
logger.error(f"Gemini auth error (HTTP {e.code}), not retrying: {str(e)}")
raise
# Retry on retryable errors
last_exception = e
if attempt < max_retries:
backoff = min(initial_backoff * (2**attempt), max_backoff)
await asyncio.sleep(backoff)
continue
raise
except Exception as e:
logger.error(f"Unexpected error during Gemini tool call: {type(e).__name__}: {str(e)}")
raise
if last_exception:
raise last_exception
raise RuntimeError("Gemini tool call failed")
async def cleanup(self) -> None:
"""Clean up resources (close connections, etc.)."""
# Gemini client doesn't require explicit cleanup
pass
@@ -1,301 +0,0 @@
"""
Mock LLM provider for testing.
This provider allows tests to record LLM calls and return configurable mock responses
without making actual API calls to external LLM services.
"""
import logging
from typing import Any
from ..llm_interface import LLMInterface
from ..response_models import LLMToolCall, LLMToolCallResult, TokenUsage
logger = logging.getLogger(__name__)
class MockLLM(LLMInterface):
"""
Mock LLM provider for testing.
This provider records all calls and returns configurable mock responses,
enabling tests to verify LLM interactions without making real API calls.
Example:
# Create mock provider
mock_llm = MockLLM(provider="mock", api_key="", base_url="", model="mock-model")
# Set mock response
mock_llm.set_mock_response({"answer": "test"})
# Make calls
result = await mock_llm.call(
messages=[{"role": "user", "content": "test"}],
response_format=MyResponseModel
)
# Verify calls
calls = mock_llm.get_mock_calls()
assert len(calls) == 1
assert calls[0]["scope"] == "memory"
"""
def __init__(
self,
provider: str,
api_key: str,
base_url: str,
model: str,
reasoning_effort: str = "low",
**kwargs: Any,
):
"""
Initialize mock LLM provider.
Args:
provider: Provider name (should be "mock").
api_key: Not used for mock provider.
base_url: Not used for mock provider.
model: Model name for tracking.
reasoning_effort: Not used for mock provider.
**kwargs: Additional parameters (not used).
"""
super().__init__(provider, api_key, base_url, model, reasoning_effort, **kwargs)
# Storage for test verification
self._mock_calls: list[dict] = []
self._mock_response: Any = None
self._mock_exception: Exception | None = None
async def verify_connection(self) -> None:
"""
Verify mock provider (always succeeds).
Mock provider doesn't need connection verification since it doesn't
make real API calls.
"""
logger.debug("Mock LLM: connection verification (always succeeds)")
async def call(
self,
messages: list[dict[str, str]],
response_format: Any | None = None,
max_completion_tokens: int | None = None,
temperature: float | None = None,
scope: str = "memory",
max_retries: int = 10,
initial_backoff: float = 1.0,
max_backoff: float = 60.0,
skip_validation: bool = False,
strict_schema: bool = False,
return_usage: bool = False,
) -> Any:
"""
Make a mock LLM API call.
Records the call for test verification and returns the configured mock response.
Args:
messages: List of message dicts with 'role' and 'content'.
response_format: Optional Pydantic model for structured output.
max_completion_tokens: Not used in mock.
temperature: Not used in mock.
scope: Scope identifier for tracking.
max_retries: Not used in mock.
initial_backoff: Not used in mock.
max_backoff: Not used in mock.
skip_validation: Return raw JSON without Pydantic validation.
strict_schema: Not used in mock.
return_usage: If True, return tuple (result, TokenUsage) instead of just result.
Returns:
If return_usage=False: Parsed response if response_format is provided, otherwise text content.
If return_usage=True: Tuple of (result, TokenUsage) with mock token counts.
"""
# Record the call for test verification
call_record = {
"provider": self.provider,
"model": self.model,
"messages": messages,
"response_format": response_format.__name__
if response_format and hasattr(response_format, "__name__")
else str(response_format),
"scope": scope,
}
self._mock_calls.append(call_record)
logger.debug(f"Mock LLM call recorded: scope={scope}, model={self.model}")
# Raise mock exception if configured
if self._mock_exception is not None:
raise self._mock_exception
# Record trace span (minimal for mock provider)
from hindsight_api.tracing import get_span_recorder
span_recorder = get_span_recorder()
span_recorder.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
messages=messages,
response_content="mock response",
input_tokens=10,
output_tokens=5,
duration=0.001, # Mock calls are instant
finish_reason="stop",
error=None,
)
# Return mock response
if self._mock_response is not None:
result = self._mock_response
elif response_format is not None:
# Try to create a minimal valid instance of the response format
try:
# For Pydantic models, try to create with minimal valid data
result = {"mock": True}
except Exception:
result = {"mock": True}
else:
result = "mock response"
if return_usage:
token_usage = TokenUsage(input_tokens=10, output_tokens=5, total_tokens=15)
return result, token_usage
return result
async def call_with_tools(
self,
messages: list[dict[str, Any]],
tools: list[dict[str, Any]],
max_completion_tokens: int | None = None,
temperature: float | None = None,
scope: str = "tools",
max_retries: int = 5,
initial_backoff: float = 1.0,
max_backoff: float = 30.0,
tool_choice: str | dict[str, Any] = "auto",
) -> LLMToolCallResult:
"""
Make a mock LLM API call with tool/function calling support.
Records the call for test verification and returns the configured mock response.
Args:
messages: List of message dicts. Can include tool results with role='tool'.
tools: List of tool definitions in OpenAI format.
max_completion_tokens: Not used in mock.
temperature: Not used in mock.
scope: Scope identifier for tracking.
max_retries: Not used in mock.
initial_backoff: Not used in mock.
max_backoff: Not used in mock.
tool_choice: Not used in mock.
Returns:
LLMToolCallResult with content and/or tool_calls.
"""
# Record the call for test verification
call_record = {
"provider": self.provider,
"model": self.model,
"messages": messages,
"tools": [t.get("function", {}).get("name") for t in tools],
"scope": scope,
}
self._mock_calls.append(call_record)
# Raise mock exception if configured
if self._mock_exception is not None:
raise self._mock_exception
# Record OpenTelemetry span
from hindsight_api.tracing import get_span_recorder
span_recorder = get_span_recorder()
if self._mock_response is not None:
if isinstance(self._mock_response, LLMToolCallResult):
result = self._mock_response
elif isinstance(self._mock_response, list):
# Allow setting just tool calls as a list
result = LLMToolCallResult(
tool_calls=[
LLMToolCall(id=f"mock_{i}", name=tc["name"], arguments=tc.get("arguments", {}))
for i, tc in enumerate(self._mock_response)
],
finish_reason="tool_calls",
)
else:
result = LLMToolCallResult(content="mock response", finish_reason="stop")
else:
result = LLMToolCallResult(content="mock response", finish_reason="stop")
# Record span with mock values
# Convert LLMToolCall objects to dicts for span recording
tool_calls_dict = (
[{"id": tc.id, "name": tc.name, "arguments": tc.arguments} for tc in result.tool_calls]
if result.tool_calls
else None
)
span_recorder.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
messages=messages,
response_content=result.content,
input_tokens=10, # Mock value
output_tokens=5, # Mock value
duration=0.1, # Mock value
finish_reason=result.finish_reason,
error=None,
tool_calls=tool_calls_dict,
)
return result
async def cleanup(self) -> None:
"""Clean up resources (no-op for mock provider)."""
pass
def set_mock_response(self, response: Any) -> None:
"""
Set the response to return from mock calls.
Args:
response: The response to return. Can be:
- A dict/Pydantic model for regular calls
- An LLMToolCallResult for tool calls
- A list of tool call dicts for tool calls
- Any other value to return as-is
"""
self._mock_response = response
def set_mock_exception(self, exception: Exception) -> None:
"""
Set an exception to raise from mock calls.
Args:
exception: The exception to raise on the next call.
After raising, the exception is cleared.
"""
self._mock_exception = exception
def get_mock_calls(self) -> list[dict]:
"""
Get the list of recorded mock calls.
Returns:
List of call records, each containing:
- provider: Provider name
- model: Model name
- messages: Messages sent
- response_format/tools: Format or tools used
- scope: Call scope
"""
return self._mock_calls
def clear_mock_calls(self) -> None:
"""Clear the recorded mock calls and any set exception."""
self._mock_calls = []
self._mock_exception = None
@@ -1,924 +0,0 @@
"""
OpenAI-compatible LLM provider supporting OpenAI, Groq, Ollama, and LMStudio.
This provider handles all OpenAI API-compatible models including:
- OpenAI: GPT-4, GPT-4o, GPT-5, o1, o3 (reasoning models)
- Groq: Fast inference with seed control and service tiers
- Ollama: Local models with native streaming API support
- LMStudio: Local models with OpenAI-compatible API
Features:
- Reasoning models with extended thinking (o1, o3, GPT-5 families)
- Strict JSON schema enforcement (OpenAI)
- Provider-specific parameters (Groq seed, service tier)
- Native Ollama streaming for better structured output
- Automatic token limit handling per model family
"""
import asyncio
import io
import json
import logging
import os
import re
import time
from typing import Any
import httpx
from openai import APIConnectionError, APIStatusError, AsyncOpenAI, LengthFinishReasonError
from hindsight_api.config import DEFAULT_LLM_TIMEOUT, ENV_LLM_TIMEOUT
from hindsight_api.engine.llm_interface import LLMInterface, OutputTooLongError
from hindsight_api.engine.response_models import LLMToolCall, LLMToolCallResult, TokenUsage
from hindsight_api.metrics import get_metrics_collector
logger = logging.getLogger(__name__)
# Seed applied to every Groq request for deterministic behavior
DEFAULT_LLM_SEED = 4242
class OpenAICompatibleLLM(LLMInterface):
"""
LLM provider for OpenAI-compatible APIs.
Supports:
- OpenAI: Standard models (GPT-4, GPT-4o) and reasoning models (o1, o3, GPT-5)
- Groq: Fast inference with seed control and service tiers
- Ollama: Local models with native streaming API for better structured output
- LMStudio: Local models with OpenAI-compatible API
"""
def __init__(
self,
provider: str,
api_key: str,
base_url: str,
model: str,
reasoning_effort: str = "low",
timeout: float | None = None,
groq_service_tier: str | None = None,
**kwargs: Any,
):
"""
Initialize OpenAI-compatible LLM provider.
Args:
provider: Provider name ("openai", "groq", "ollama", "lmstudio").
api_key: API key (optional for ollama/lmstudio).
base_url: Base URL for the API (uses defaults for groq/ollama/lmstudio if empty).
model: Model name.
reasoning_effort: Reasoning effort level for supported models ("low", "medium", "high").
timeout: Request timeout in seconds (uses env var or 300s default).
groq_service_tier: Groq service tier ("on_demand", "flex", "auto").
**kwargs: Additional provider-specific parameters.
"""
super().__init__(provider, api_key, base_url, model, reasoning_effort, **kwargs)
# Validate provider
valid_providers = ["openai", "groq", "ollama", "lmstudio"]
if self.provider not in valid_providers:
raise ValueError(f"OpenAICompatibleLLM only supports: {', '.join(valid_providers)}. Got: {self.provider}")
# Set default base URLs
if not self.base_url:
if self.provider == "groq":
self.base_url = "https://api.groq.com/openai/v1"
elif self.provider == "ollama":
self.base_url = "http://localhost:11434/v1"
elif self.provider == "lmstudio":
self.base_url = "http://localhost:1234/v1"
# For ollama/lmstudio, use dummy key if not provided
if self.provider in ("ollama", "lmstudio") and not self.api_key:
self.api_key = "local"
# Validate API key for cloud providers
if self.provider in ("openai", "groq") and not self.api_key:
raise ValueError(f"API key is required for {self.provider}")
# Service tier configuration (from config, not env vars)
self.groq_service_tier = groq_service_tier
self.openai_service_tier = kwargs.get("openai_service_tier")
# Get timeout config
self.timeout = timeout or float(os.getenv(ENV_LLM_TIMEOUT, str(DEFAULT_LLM_TIMEOUT)))
# Create OpenAI client
client_kwargs: dict[str, Any] = {"api_key": self.api_key, "max_retries": 0}
if self.base_url:
client_kwargs["base_url"] = self.base_url
if self.timeout:
client_kwargs["timeout"] = self.timeout
self._client = AsyncOpenAI(**client_kwargs)
logger.info(
f"OpenAI-compatible client initialized: provider={self.provider}, model={self.model}, "
f"base_url={self.base_url or 'default'}"
)
async def verify_connection(self) -> None:
"""
Verify that the provider is configured correctly by making a simple test call.
Raises:
RuntimeError: If the connection test fails.
"""
try:
logger.info(f"Verifying connection: {self.provider}/{self.model}")
await self.call(
messages=[{"role": "user", "content": "Say 'ok'"}],
max_completion_tokens=100,
max_retries=2,
initial_backoff=0.5,
max_backoff=2.0,
scope="verification",
)
logger.info(f"Connection verified: {self.provider}/{self.model}")
except Exception as e:
raise RuntimeError(f"Connection verification failed for {self.provider}/{self.model}: {e}") from e
def _supports_reasoning_model(self) -> bool:
"""Check if the current model is a reasoning model (o1, o3, GPT-5, DeepSeek)."""
model_lower = self.model.lower()
return any(x in model_lower for x in ["gpt-5", "o1", "o3", "deepseek"])
def _get_max_reasoning_tokens(self) -> int | None:
"""Get max reasoning tokens for reasoning models."""
model_lower = self.model.lower()
# GPT-4 and GPT-4.1 models have different caps
if any(x in model_lower for x in ["gpt-4.1", "gpt-4-"]):
return 32000
elif "gpt-4o" in model_lower:
return 16384
return None
async def call(
self,
messages: list[dict[str, str]],
response_format: Any | None = None,
max_completion_tokens: int | None = None,
temperature: float | None = None,
scope: str = "memory",
max_retries: int = 10,
initial_backoff: float = 1.0,
max_backoff: float = 60.0,
skip_validation: bool = False,
strict_schema: bool = False,
return_usage: bool = False,
) -> Any:
"""
Make an LLM API call with retry logic.
Args:
messages: List of message dicts with 'role' and 'content'.
response_format: Optional Pydantic model for structured output.
max_completion_tokens: Maximum tokens in response.
temperature: Sampling temperature (0.0-2.0).
scope: Scope identifier for tracking.
max_retries: Maximum retry attempts.
initial_backoff: Initial backoff time in seconds.
max_backoff: Maximum backoff time in seconds.
skip_validation: Return raw JSON without Pydantic validation.
strict_schema: Use strict JSON schema enforcement (OpenAI only).
return_usage: If True, return tuple (result, TokenUsage) instead of just result.
Returns:
If return_usage=False: Parsed response if response_format is provided, otherwise text content.
If return_usage=True: Tuple of (result, TokenUsage) with token counts.
Raises:
OutputTooLongError: If output exceeds token limits.
Exception: Re-raises API errors after retries exhausted.
"""
# Handle Ollama with native API for structured output (better schema enforcement)
if self.provider == "ollama" and response_format is not None:
return await self._call_ollama_native(
messages=messages,
response_format=response_format,
max_completion_tokens=max_completion_tokens,
temperature=temperature,
max_retries=max_retries,
initial_backoff=initial_backoff,
max_backoff=max_backoff,
skip_validation=skip_validation,
scope=scope,
return_usage=return_usage,
)
start_time = time.time()
# Build call parameters
call_params: dict[str, Any] = {
"model": self.model,
"messages": messages,
}
# Check if model supports reasoning parameter
is_reasoning_model = self._supports_reasoning_model()
# Apply model-specific token limits
if max_completion_tokens is not None:
max_tokens_cap = self._get_max_reasoning_tokens()
if max_tokens_cap and max_completion_tokens > max_tokens_cap:
max_completion_tokens = max_tokens_cap
# For reasoning models, enforce minimum to ensure space for reasoning + output
if is_reasoning_model and max_completion_tokens < 16000:
max_completion_tokens = 16000
call_params["max_completion_tokens"] = max_completion_tokens
# Temperature - reasoning models don't support custom temperature
if temperature is not None and not is_reasoning_model:
call_params["temperature"] = temperature
# Set reasoning_effort for reasoning models
if is_reasoning_model:
call_params["reasoning_effort"] = self.reasoning_effort
# Provider-specific parameters
if self.provider == "groq":
call_params["seed"] = DEFAULT_LLM_SEED
extra_body: dict[str, Any] = {}
# Add service_tier if configured
if self.groq_service_tier:
extra_body["service_tier"] = self.groq_service_tier
# Add reasoning parameters for reasoning models
if is_reasoning_model:
extra_body["include_reasoning"] = False
if extra_body:
call_params["extra_body"] = extra_body
# Prepare response format ONCE before retry loop
if response_format is not None:
schema = None
if hasattr(response_format, "model_json_schema"):
schema = response_format.model_json_schema()
if strict_schema and schema is not None:
# Use OpenAI's strict JSON schema enforcement
call_params["response_format"] = {
"type": "json_schema",
"json_schema": {
"name": "response",
"strict": True,
"schema": schema,
},
}
else:
# Soft enforcement: add schema to prompt and use json_object mode
if schema is not None:
schema_msg = (
f"\n\nYou must respond with valid JSON matching this schema:\n{json.dumps(schema, indent=2)}"
)
if call_params["messages"] and call_params["messages"][0].get("role") == "system":
first_msg = call_params["messages"][0]
if isinstance(first_msg, dict) and isinstance(first_msg.get("content"), str):
first_msg["content"] += schema_msg
elif call_params["messages"]:
first_msg = call_params["messages"][0]
if isinstance(first_msg, dict) and isinstance(first_msg.get("content"), str):
first_msg["content"] = schema_msg + "\n\n" + first_msg["content"]
if self.provider not in ("lmstudio", "ollama"):
# LM Studio and Ollama don't support json_object response format reliably
call_params["response_format"] = {"type": "json_object"}
last_exception = None
for attempt in range(max_retries + 1):
try:
if response_format is not None:
response = await self._client.chat.completions.create(**call_params)
content = response.choices[0].message.content
# Strip reasoning model thinking tags
# Supports: <think>, <thinking>, <reasoning>, |startthink|/|endthink|
if content:
original_len = len(content)
content = re.sub(r"<think>.*?</think>", "", content, flags=re.DOTALL)
content = re.sub(r"<thinking>.*?</thinking>", "", content, flags=re.DOTALL)
content = re.sub(r"<reasoning>.*?</reasoning>", "", content, flags=re.DOTALL)
content = re.sub(r"\|startthink\|.*?\|endthink\|", "", content, flags=re.DOTALL)
content = content.strip()
if len(content) < original_len:
logger.debug(f"Stripped {original_len - len(content)} chars of reasoning tokens")
# For local models, they may wrap JSON in markdown code blocks
if self.provider in ("lmstudio", "ollama"):
clean_content = content
if "```json" in content:
clean_content = content.split("```json")[1].split("```")[0].strip()
elif "```" in content:
clean_content = content.split("```")[1].split("```")[0].strip()
try:
json_data = json.loads(clean_content)
except json.JSONDecodeError:
# Fallback to parsing raw content
json_data = json.loads(content)
else:
# Log raw LLM response for debugging JSON parse issues
try:
json_data = json.loads(content)
except json.JSONDecodeError as json_err:
# Truncate content for logging
content_preview = content[:500] if content else "<empty>"
if content and len(content) > 700:
content_preview = f"{content[:500]}...TRUNCATED...{content[-200:]}"
logger.warning(
f"JSON parse error from LLM response (attempt {attempt + 1}/{max_retries + 1}): {json_err}\n"
f" Model: {self.provider}/{self.model}\n"
f" Content length: {len(content) if content else 0} chars\n"
f" Content preview: {content_preview!r}\n"
f" Finish reason: {response.choices[0].finish_reason if response.choices else 'unknown'}"
)
# Retry on JSON parse errors
if attempt < max_retries:
backoff = min(initial_backoff * (2**attempt), max_backoff)
await asyncio.sleep(backoff)
last_exception = json_err
continue
else:
logger.error(f"JSON parse error after {max_retries + 1} attempts, giving up")
raise
if skip_validation:
result = json_data
else:
result = response_format.model_validate(json_data)
else:
response = await self._client.chat.completions.create(**call_params)
result = response.choices[0].message.content
# Record token usage metrics
duration = time.time() - start_time
usage = response.usage
input_tokens = usage.prompt_tokens or 0 if usage else 0
output_tokens = usage.completion_tokens or 0 if usage else 0
total_tokens = usage.total_tokens or 0 if usage else 0
# Record LLM metrics
metrics = get_metrics_collector()
metrics.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
duration=duration,
input_tokens=input_tokens,
output_tokens=output_tokens,
success=True,
)
# Record trace span
from hindsight_api.tracing import _serialize_for_span, get_span_recorder
finish_reason = response.choices[0].finish_reason if response.choices else None
span_recorder = get_span_recorder()
span_recorder.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
messages=messages,
response_content=_serialize_for_span(result),
input_tokens=input_tokens,
output_tokens=output_tokens,
duration=duration,
finish_reason=finish_reason,
error=None,
)
# Log slow calls
if duration > 10.0 and usage:
ratio = max(1, output_tokens) / max(1, input_tokens)
cached_tokens = 0
if hasattr(usage, "prompt_tokens_details") and usage.prompt_tokens_details:
cached_tokens = getattr(usage.prompt_tokens_details, "cached_tokens", 0) or 0
cache_info = f", cached_tokens={cached_tokens}" if cached_tokens > 0 else ""
logger.info(
f"slow llm call: scope={scope}, model={self.provider}/{self.model}, "
f"input_tokens={input_tokens}, output_tokens={output_tokens}, "
f"total_tokens={total_tokens}{cache_info}, time={duration:.3f}s, ratio out/in={ratio:.2f}"
)
if return_usage:
token_usage = TokenUsage(
input_tokens=input_tokens,
output_tokens=output_tokens,
total_tokens=total_tokens,
)
return result, token_usage
return result
except LengthFinishReasonError as e:
logger.warning(f"LLM output exceeded token limits: {str(e)}")
raise OutputTooLongError(
"LLM output exceeded token limits. Input may need to be split into smaller chunks."
) from e
except APIConnectionError as e:
last_exception = e
status_code = getattr(e, "status_code", None) or getattr(
getattr(e, "response", None), "status_code", None
)
logger.warning(f"APIConnectionError (HTTP {status_code}), attempt {attempt + 1}: {str(e)[:200]}")
if attempt < max_retries:
backoff = min(initial_backoff * (2**attempt), max_backoff)
await asyncio.sleep(backoff)
continue
else:
logger.error(f"Connection error after {max_retries + 1} attempts: {str(e)}")
raise
except APIStatusError as e:
# Fast fail only on 401 (unauthorized) and 403 (forbidden)
if e.status_code in (401, 403):
logger.error(f"Auth error (HTTP {e.status_code}), not retrying: {str(e)}")
raise
# Handle tool_use_failed error - model outputted in tool call format
if e.status_code == 400 and response_format is not None:
try:
error_body = e.body if hasattr(e, "body") else {}
if isinstance(error_body, dict):
error_info: dict[str, Any] = error_body.get("error") or {}
if error_info.get("code") == "tool_use_failed":
failed_gen = error_info.get("failed_generation", "")
if failed_gen:
# Parse tool call format and convert to expected format
tool_call = json.loads(failed_gen)
tool_name = tool_call.get("name", "")
tool_args = tool_call.get("arguments", {})
converted = {"actions": [{"tool": tool_name, **tool_args}]}
if skip_validation:
result = converted
else:
result = response_format.model_validate(converted)
# Record metrics
duration = time.time() - start_time
metrics = get_metrics_collector()
metrics.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
duration=duration,
input_tokens=0,
output_tokens=0,
success=True,
)
if return_usage:
return result, TokenUsage(input_tokens=0, output_tokens=0, total_tokens=0)
return result
except (json.JSONDecodeError, KeyError, TypeError):
pass # Failed to parse tool_use_failed, continue with normal retry
last_exception = e
if attempt < max_retries:
backoff = min(initial_backoff * (2**attempt), max_backoff)
jitter = backoff * 0.2 * (2 * (time.time() % 1) - 1)
sleep_time = backoff + jitter
await asyncio.sleep(sleep_time)
else:
logger.error(f"API error after {max_retries + 1} attempts: {str(e)}")
raise
except Exception:
raise
if last_exception:
raise last_exception
raise RuntimeError("LLM call failed after all retries with no exception captured")
async def call_with_tools(
self,
messages: list[dict[str, Any]],
tools: list[dict[str, Any]],
max_completion_tokens: int | None = None,
temperature: float | None = None,
scope: str = "tools",
max_retries: int = 5,
initial_backoff: float = 1.0,
max_backoff: float = 30.0,
tool_choice: str | dict[str, Any] = "auto",
) -> LLMToolCallResult:
"""
Make an LLM API call with tool/function calling support.
Args:
messages: List of message dicts. Can include tool results with role='tool'.
tools: List of tool definitions in OpenAI format.
max_completion_tokens: Maximum tokens in response.
temperature: Sampling temperature (0.0-2.0).
scope: Scope identifier for tracking.
max_retries: Maximum retry attempts.
initial_backoff: Initial backoff time in seconds.
max_backoff: Maximum backoff time in seconds.
tool_choice: How to choose tools - "auto", "none", "required", or specific function.
Returns:
LLMToolCallResult with content and/or tool_calls.
"""
start_time = time.time()
# Build call parameters
call_params: dict[str, Any] = {
"model": self.model,
"messages": messages,
"tools": tools,
"tool_choice": tool_choice,
}
if max_completion_tokens is not None:
call_params["max_completion_tokens"] = max_completion_tokens
if temperature is not None:
call_params["temperature"] = temperature
# Provider-specific parameters
if self.provider == "groq":
call_params["seed"] = DEFAULT_LLM_SEED
last_exception = None
for attempt in range(max_retries + 1):
try:
response = await self._client.chat.completions.create(**call_params)
message = response.choices[0].message
finish_reason = response.choices[0].finish_reason
# Extract tool calls if present
tool_calls: list[LLMToolCall] = []
if message.tool_calls:
for tc in message.tool_calls:
try:
args = json.loads(tc.function.arguments) if tc.function.arguments else {}
except json.JSONDecodeError:
args = {"_raw": tc.function.arguments}
tool_calls.append(LLMToolCall(id=tc.id, name=tc.function.name, arguments=args))
content = message.content
# Record metrics
duration = time.time() - start_time
usage = response.usage
input_tokens = usage.prompt_tokens or 0 if usage else 0
output_tokens = usage.completion_tokens or 0 if usage else 0
metrics = get_metrics_collector()
metrics.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
duration=duration,
input_tokens=input_tokens,
output_tokens=output_tokens,
success=True,
)
# Record OpenTelemetry span
from hindsight_api.tracing import get_span_recorder
span_recorder = get_span_recorder()
# Convert LLMToolCall objects to dicts for span recording
tool_calls_dict = (
[{"id": tc.id, "name": tc.name, "arguments": tc.arguments} for tc in tool_calls]
if tool_calls
else None
)
span_recorder.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
messages=messages,
response_content=content,
input_tokens=input_tokens,
output_tokens=output_tokens,
duration=duration,
finish_reason=finish_reason,
error=None,
tool_calls=tool_calls_dict,
)
return LLMToolCallResult(
content=content,
tool_calls=tool_calls,
finish_reason=finish_reason,
input_tokens=input_tokens,
output_tokens=output_tokens,
)
except APIConnectionError as e:
last_exception = e
if attempt < max_retries:
await asyncio.sleep(min(initial_backoff * (2**attempt), max_backoff))
continue
raise
except APIStatusError as e:
if e.status_code in (401, 403):
raise
last_exception = e
if attempt < max_retries:
await asyncio.sleep(min(initial_backoff * (2**attempt), max_backoff))
continue
raise
except Exception:
raise
if last_exception:
raise last_exception
raise RuntimeError("Tool call failed after all retries")
async def _call_ollama_native(
self,
messages: list[dict[str, str]],
response_format: Any,
max_completion_tokens: int | None,
temperature: float | None,
max_retries: int,
initial_backoff: float,
max_backoff: float,
skip_validation: bool,
scope: str = "memory",
return_usage: bool = False,
) -> Any:
"""
Call Ollama using native API with JSON schema enforcement.
Ollama's native API supports passing a full JSON schema in the 'format' parameter,
which provides better structured output control than the OpenAI-compatible API.
"""
start_time = time.time()
# Get the JSON schema from the Pydantic model
schema = response_format.model_json_schema() if hasattr(response_format, "model_json_schema") else None
# Build the base URL for Ollama's native API
# Default OpenAI-compatible URL is http://localhost:11434/v1
# Native API is at http://localhost:11434/api/chat
base_url = self.base_url or "http://localhost:11434/v1"
if base_url.endswith("/v1"):
native_url = base_url[:-3] + "/api/chat"
else:
native_url = base_url.rstrip("/") + "/api/chat"
# Build request payload
payload: dict[str, Any] = {
"model": self.model,
"messages": messages,
"stream": False,
}
# Add schema as format parameter for structured output
if schema:
payload["format"] = schema
# Add optional parameters with optimized defaults for Ollama
options: dict[str, Any] = {
"num_ctx": 16384, # 16k context window for larger prompts
"num_batch": 512, # Optimal batch size for prompt processing
}
if max_completion_tokens:
options["num_predict"] = max_completion_tokens
if temperature is not None:
options["temperature"] = temperature
payload["options"] = options
last_exception = None
async with httpx.AsyncClient(timeout=300.0) as client:
for attempt in range(max_retries + 1):
try:
response = await client.post(native_url, json=payload)
response.raise_for_status()
result = response.json()
content = result.get("message", {}).get("content", "")
# Parse JSON response
try:
json_data = json.loads(content)
except json.JSONDecodeError as json_err:
content_preview = content[:500] if content else "<empty>"
if content and len(content) > 700:
content_preview = f"{content[:500]}...TRUNCATED...{content[-200:]}"
logger.warning(
f"Ollama JSON parse error (attempt {attempt + 1}/{max_retries + 1}): {json_err}\n"
f" Model: ollama/{self.model}\n"
f" Content length: {len(content) if content else 0} chars\n"
f" Content preview: {content_preview!r}"
)
if attempt < max_retries:
backoff = min(initial_backoff * (2**attempt), max_backoff)
await asyncio.sleep(backoff)
last_exception = json_err
continue
else:
raise
# Extract token usage from Ollama response
duration = time.time() - start_time
input_tokens = result.get("prompt_eval_count", 0) or 0
output_tokens = result.get("eval_count", 0) or 0
total_tokens = input_tokens + output_tokens
# Record LLM metrics
metrics = get_metrics_collector()
metrics.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
duration=duration,
input_tokens=input_tokens,
output_tokens=output_tokens,
success=True,
)
# Validate against Pydantic model or return raw JSON
if skip_validation:
validated_result = json_data
else:
validated_result = response_format.model_validate(json_data)
if return_usage:
token_usage = TokenUsage(
input_tokens=input_tokens,
output_tokens=output_tokens,
total_tokens=total_tokens,
)
return validated_result, token_usage
return validated_result
except httpx.HTTPStatusError as e:
last_exception = e
if attempt < max_retries:
logger.warning(
f"Ollama HTTP error (attempt {attempt + 1}/{max_retries + 1}): {e.response.status_code}"
)
backoff = min(initial_backoff * (2**attempt), max_backoff)
await asyncio.sleep(backoff)
continue
else:
logger.error(f"Ollama HTTP error after {max_retries + 1} attempts: {e}")
raise
except httpx.RequestError as e:
last_exception = e
if attempt < max_retries:
logger.warning(f"Ollama connection error (attempt {attempt + 1}/{max_retries + 1}): {e}")
backoff = min(initial_backoff * (2**attempt), max_backoff)
await asyncio.sleep(backoff)
continue
else:
logger.error(f"Ollama connection error after {max_retries + 1} attempts: {e}")
raise
except Exception as e:
logger.error(f"Unexpected error during Ollama call: {type(e).__name__}: {e}")
raise
if last_exception:
raise last_exception
raise RuntimeError("Ollama call failed after all retries")
async def supports_batch_api(self) -> bool:
"""Check if this provider supports batch API operations."""
# Only OpenAI and Groq support batch API
return self.provider in ("openai", "groq")
async def submit_batch(
self,
requests: list[dict[str, Any]],
endpoint: str = "/v1/chat/completions",
completion_window: str = "24h",
) -> dict[str, Any]:
"""
Submit a batch of requests to OpenAI/Groq Batch API.
Args:
requests: List of request dicts with custom_id, method, url, body
endpoint: API endpoint (e.g., "/v1/chat/completions")
completion_window: Completion window (e.g., "24h")
Returns:
Dict with batch metadata including batch_id
Raises:
NotImplementedError: If provider doesn't support batch API
"""
if not await self.supports_batch_api():
raise NotImplementedError(f"Batch API not supported for provider: {self.provider}")
logger.info(f"Submitting batch with {len(requests)} requests to {self.provider}")
# Format requests as JSONL
jsonl_content = "\n".join(json.dumps(req) for req in requests)
# Upload file to provider (wrap in BytesIO with filename)
file_bytes = io.BytesIO(jsonl_content.encode("utf-8"))
file_bytes.name = "batch_input.jsonl" # OpenAI SDK needs a filename
file_response = await self._client.files.create(
file=file_bytes,
purpose="batch",
)
logger.debug(f"Uploaded batch file: {file_response.id}")
# Create batch
batch_response = await self._client.batches.create(
input_file_id=file_response.id,
endpoint=endpoint,
completion_window=completion_window,
)
logger.info(f"Batch submitted: {batch_response.id}, status={batch_response.status}")
return {
"batch_id": batch_response.id,
"status": batch_response.status,
"input_file_id": file_response.id,
"created_at": batch_response.created_at,
"request_count": len(requests),
}
async def get_batch_status(self, batch_id: str) -> dict[str, Any]:
"""
Get the status of a batch job.
Args:
batch_id: Batch identifier
Returns:
Dict with status info (batch_id, status, completed_at, etc.)
"""
if not await self.supports_batch_api():
raise NotImplementedError(f"Batch API not supported for provider: {self.provider}")
batch = await self._client.batches.retrieve(batch_id)
result = {
"batch_id": batch.id,
"status": batch.status,
"created_at": batch.created_at,
"request_counts": {
"total": batch.request_counts.total if batch.request_counts else 0,
"completed": batch.request_counts.completed if batch.request_counts else 0,
"failed": batch.request_counts.failed if batch.request_counts else 0,
},
}
if batch.completed_at:
result["completed_at"] = batch.completed_at
if batch.output_file_id:
result["output_file_id"] = batch.output_file_id
if batch.error_file_id:
result["error_file_id"] = batch.error_file_id
if batch.errors:
result["errors"] = batch.errors
return result
async def retrieve_batch_results(self, batch_id: str) -> list[dict[str, Any]]:
"""
Retrieve completed batch results.
Args:
batch_id: Batch identifier
Returns:
List of result dicts (one per request, matched by custom_id)
"""
if not await self.supports_batch_api():
raise NotImplementedError(f"Batch API not supported for provider: {self.provider}")
# Get batch status
batch = await self._client.batches.retrieve(batch_id)
if batch.status != "completed":
raise ValueError(f"Batch {batch_id} is not completed yet (status: {batch.status})")
if not batch.output_file_id:
raise ValueError(f"Batch {batch_id} has no output file")
# Download results file
logger.debug(f"Downloading results for batch {batch_id} from file {batch.output_file_id}")
file_content = await self._client.files.content(batch.output_file_id)
# Parse JSONL results
results = []
for line in file_content.text.strip().split("\n"):
if line:
results.append(json.loads(line))
logger.info(f"Retrieved {len(results)} results for batch {batch_id}")
return results
async def cleanup(self) -> None:
"""Clean up resources (close OpenAI client connections)."""
if hasattr(self, "_client") and self._client:
await self._client.close()
@@ -84,7 +84,7 @@ class DateparserQueryAnalyzer(QueryAnalyzer):
Performance:
- ~10-50ms per query
- No model loading required (lazy import on first use)
- No model loading required
"""
def __init__(self):
@@ -112,6 +112,8 @@ class DateparserQueryAnalyzer(QueryAnalyzer):
Returns:
QueryAnalysis with temporal_constraint if found
"""
self.load()
if reference_date is None:
reference_date = datetime.now()
@@ -121,9 +123,6 @@ class DateparserQueryAnalyzer(QueryAnalyzer):
if period_result is not None:
return QueryAnalysis(temporal_constraint=period_result)
# Lazy load dateparser (only imports on first call, then cached)
self.load()
# Use dateparser's search_dates to find temporal expressions
settings = {
"RELATIVE_BASE": reference_date,
@@ -1,18 +0,0 @@
"""
Reflect agent module for agentic reflection with tools.
The reflect agent uses an iterative loop with tools to:
1. Lookup mental models (existing knowledge)
2. Recall facts (semantic + temporal search)
3. Expand memories (get chunk/document context)
"""
from .agent import ReflectAgentResult, run_reflect_agent
from .models import ReflectAction, ReflectActionBatch
__all__ = [
"run_reflect_agent",
"ReflectAgentResult",
"ReflectAction",
"ReflectActionBatch",
]
@@ -1,990 +0,0 @@
"""
Reflect agent - agentic loop for reflection with native tool calling.
Uses hierarchical retrieval:
1. search_mental_models - User-curated summaries (highest quality)
2. search_observations - Consolidated knowledge with freshness
3. recall - Raw facts as ground truth
"""
import asyncio
import json
import logging
import re
import time
from typing import TYPE_CHECKING, Any, Awaitable, Callable
from .models import DirectiveInfo, LLMCall, ReflectAgentResult, TokenUsageSummary, ToolCall
from .prompts import FINAL_SYSTEM_PROMPT, _extract_directive_rules, build_final_prompt, build_system_prompt_for_tools
from .tools_schema import get_reflect_tools
def _build_directives_applied(directives: list[dict[str, Any]] | None) -> list[DirectiveInfo]:
"""Build list of DirectiveInfo from directive mental models.
Handles multiple directive formats:
1. New format: directives have direct 'content' field
2. Fallback: directives have 'description' field
"""
if not directives:
return []
result = []
for directive in directives:
directive_id = directive.get("id", "")
directive_name = directive.get("name", "")
# Get content from 'content' field or fallback to 'description'
content = directive.get("content", "") or directive.get("description", "")
result.append(DirectiveInfo(id=directive_id, name=directive_name, content=content))
return result
if TYPE_CHECKING:
from ..llm_wrapper import LLMProvider
from ..response_models import LLMToolCall
logger = logging.getLogger(__name__)
DEFAULT_MAX_ITERATIONS = 10
def _normalize_tool_name(name: str) -> str:
"""Normalize tool name from various LLM output formats.
Some LLMs output tool names in non-standard formats:
- 'functions.done' (OpenAI-style prefix)
- 'call=functions.done' (some models)
- 'call=done' (some models)
- 'done<|channel|>commentary' (malformed special tokens appended)
Returns the normalized tool name (e.g., 'done', 'recall', etc.)
"""
# Handle 'call=functions.name' or 'call=name' format
if name.startswith("call="):
name = name[len("call=") :]
# Handle 'functions.name' format
if name.startswith("functions."):
name = name[len("functions.") :]
# Handle malformed special tokens appended to tool name
# e.g., 'done<|channel|>commentary' -> 'done'
if "<|" in name:
name = name.split("<|")[0]
return name
def _is_done_tool(name: str) -> bool:
"""Check if the tool name represents the 'done' tool."""
return _normalize_tool_name(name) == "done"
# Pattern to match done() call as text - handles done({...}) with nested JSON
_DONE_CALL_PATTERN = re.compile(r"done\s*\(\s*\{.*$", re.DOTALL)
# Patterns for leaked structured output in the answer field
_LEAKED_JSON_SUFFIX = re.compile(
r'\s*```(?:json)?\s*\{[^}]*(?:"(?:observation_ids|memory_ids|mental_model_ids)"|\})\s*```\s*$',
re.DOTALL | re.IGNORECASE,
)
_LEAKED_JSON_OBJECT = re.compile(
r'\s*\{[^{]*"(?:observation_ids|memory_ids|mental_model_ids|answer)"[^}]*\}\s*$', re.DOTALL
)
_TRAILING_IDS_PATTERN = re.compile(
r"\s*(?:observation_ids|memory_ids|mental_model_ids)\s*[=:]\s*\[.*?\]\s*$", re.DOTALL | re.IGNORECASE
)
def _clean_answer_text(text: str) -> str:
"""Clean up answer text by removing any done() tool call syntax.
Some LLMs output the done() call as text instead of a proper tool call.
This strips out patterns like: done({"answer": "...", ...})
"""
# Remove done() call pattern from the end of the text
cleaned = _DONE_CALL_PATTERN.sub("", text).strip()
return cleaned if cleaned else text
def _clean_done_answer(text: str) -> str:
"""Clean up the answer field from a done() tool call.
Some LLMs leak structured output patterns into the answer text, such as:
- JSON code blocks with observation_ids/memory_ids at the end
- Raw JSON objects with these fields
- Plain text like "observation_ids: [...]"
This cleans those patterns while preserving the actual answer content.
"""
if not text:
return text
cleaned = text
# Remove leaked JSON in code blocks at the end
cleaned = _LEAKED_JSON_SUFFIX.sub("", cleaned).strip()
# Remove leaked raw JSON objects at the end
cleaned = _LEAKED_JSON_OBJECT.sub("", cleaned).strip()
# Remove trailing ID patterns
cleaned = _TRAILING_IDS_PATTERN.sub("", cleaned).strip()
return cleaned if cleaned else text
async def _generate_structured_output(
answer: str,
response_schema: dict,
llm_config: "LLMProvider",
reflect_id: str,
) -> tuple[dict[str, Any] | None, int, int]:
"""Generate structured output from an answer using the provided JSON schema.
Args:
answer: The text answer to extract structured data from
response_schema: JSON Schema for the expected output structure
llm_config: LLM provider for making the extraction call
reflect_id: Reflect ID for logging
Returns:
Tuple of (structured_output, input_tokens, output_tokens).
structured_output is None if generation fails.
"""
try:
from typing import Any as TypingAny
from pydantic import create_model
def _json_schema_type_to_python(field_schema: dict) -> type:
"""Map JSON schema type to Python type for better LLM guidance."""
json_type = field_schema.get("type", "string")
if json_type == "array":
return list
elif json_type == "object":
return dict
elif json_type == "integer":
return int
elif json_type == "number":
return float
elif json_type == "boolean":
return bool
else:
return str
# Build fields from JSON schema properties
schema_props = response_schema.get("properties", {})
required_fields = set(response_schema.get("required", []))
fields: dict[str, TypingAny] = {}
for field_name, field_schema in schema_props.items():
field_type = _json_schema_type_to_python(field_schema)
default = ... if field_name in required_fields else None
fields[field_name] = (field_type, default)
if not fields:
logger.warning(f"[REFLECT {reflect_id}] No fields found in response_schema, skipping structured output")
return None, 0, 0
DynamicModel = create_model("StructuredResponse", **fields)
# Include the full schema in the prompt for better LLM guidance
schema_str = json.dumps(response_schema, indent=2)
# Build field descriptions for the prompt
field_descriptions = []
for field_name, field_schema in schema_props.items():
field_type = field_schema.get("type", "string")
field_desc = field_schema.get("description", "")
is_required = field_name in required_fields
req_marker = " (REQUIRED)" if is_required else " (optional)"
field_descriptions.append(f"- {field_name} ({field_type}){req_marker}: {field_desc}")
fields_text = "\n".join(field_descriptions)
# Call LLM with the answer to extract structured data
structured_prompt = f"""Your task is to extract specific information from the answer below and format it as JSON.
ANSWER TO EXTRACT FROM:
\"\"\"
{answer}
\"\"\"
REQUIRED OUTPUT FORMAT - Extract the following fields from the answer above:
{fields_text}
JSON Schema:
```json
{schema_str}
```
INSTRUCTIONS:
1. Read the answer carefully and identify the information that matches each field
2. Extract the ACTUAL content from the answer - do NOT leave fields empty if information is present
3. For string fields: use the exact text or a clear summary from the answer
4. For array fields: return a JSON array (e.g., ["item1", "item2"]), NOT a string
5. For required fields: you MUST provide a value extracted from the answer
6. Return ONLY the JSON object, no explanation
OUTPUT:"""
structured_result, usage = await llm_config.call(
messages=[
{
"role": "system",
"content": "You are a precise data extraction assistant. Extract information from text and return it as valid JSON matching the provided schema. Always extract actual content - never return empty strings for required fields if information is available.",
},
{"role": "user", "content": structured_prompt},
],
response_format=DynamicModel,
scope="reflect_structured",
skip_validation=True, # We'll handle the dict ourselves
return_usage=True,
)
# Convert to dict
if hasattr(structured_result, "model_dump"):
structured_output = structured_result.model_dump()
elif isinstance(structured_result, dict):
structured_output = structured_result
else:
# Try to parse as JSON
structured_output = json.loads(str(structured_result))
# Validate that required fields have non-empty values
for field_name in required_fields:
value = structured_output.get(field_name)
if value is None or value == "" or value == []:
logger.warning(f"[REFLECT {reflect_id}] Required field '{field_name}' is empty in structured output")
logger.info(f"[REFLECT {reflect_id}] Generated structured output with {len(structured_output)} fields")
return structured_output, usage.input_tokens, usage.output_tokens
except Exception as e:
logger.warning(f"[REFLECT {reflect_id}] Failed to generate structured output: {e}")
return None, 0, 0
async def run_reflect_agent(
llm_config: "LLMProvider",
bank_id: str,
query: str,
bank_profile: dict[str, Any],
search_mental_models_fn: Callable[[str, int], Awaitable[dict[str, Any]]],
search_observations_fn: Callable[[str, int], Awaitable[dict[str, Any]]],
recall_fn: Callable[[str, int], Awaitable[dict[str, Any]]],
expand_fn: Callable[[list[str], str], Awaitable[dict[str, Any]]],
context: str | None = None,
max_iterations: int = DEFAULT_MAX_ITERATIONS,
max_tokens: int | None = None,
response_schema: dict | None = None,
directives: list[dict[str, Any]] | None = None,
has_mental_models: bool = False,
budget: str | None = None,
) -> ReflectAgentResult:
"""
Execute the reflect agent loop using native tool calling.
The agent uses hierarchical retrieval:
1. search_mental_models - User-curated summaries (try first)
2. search_observations - Consolidated knowledge with freshness
3. recall - Raw facts as ground truth
Args:
llm_config: LLM provider for agent calls
bank_id: Bank identifier
query: Question to answer
bank_profile: Bank profile with name and mission
search_mental_models_fn: Tool callback for searching mental models (query, max_results) -> result
search_observations_fn: Tool callback for searching observations (query, max_results) -> result
recall_fn: Tool callback for recall (query, max_tokens) -> result
expand_fn: Tool callback for expand (memory_ids, depth) -> result
context: Optional additional context
max_iterations: Maximum number of iterations before forcing response
max_tokens: Maximum tokens for the final response
response_schema: Optional JSON Schema for structured output in final response
directives: Optional list of directive mental models to inject as hard rules
Returns:
ReflectAgentResult with final answer and metadata
"""
reflect_id = f"{bank_id[:8]}-{int(time.time() * 1000) % 100000}"
start_time = time.time()
# Build directives_applied for the trace
directives_applied = _build_directives_applied(directives)
# Extract directive rules for tool schema (if any)
directive_rules = _extract_directive_rules(directives) if directives else None
# Get tools for this agent (with directive compliance field if directives exist)
tools = get_reflect_tools(directive_rules=directive_rules)
# Build initial messages (directives are injected into system prompt at START and END)
system_prompt = build_system_prompt_for_tools(
bank_profile, context, directives=directives, has_mental_models=has_mental_models, budget=budget
)
messages: list[dict[str, Any]] = [
{"role": "system", "content": system_prompt},
{"role": "user", "content": query},
]
# Tracking
total_tools_called = 0
tool_trace: list[ToolCall] = []
tool_trace_summary: list[dict[str, Any]] = []
llm_trace: list[dict[str, Any]] = []
context_history: list[dict[str, Any]] = [] # For final prompt fallback
# Token usage tracking - accumulate across all LLM calls
total_input_tokens = 0
total_output_tokens = 0
# Track available IDs for validation (prevents hallucinated citations)
available_memory_ids: set[str] = set()
available_mental_model_ids: set[str] = set()
available_observation_ids: set[str] = set()
def _get_llm_trace() -> list[LLMCall]:
return [
LLMCall(
scope=c["scope"],
duration_ms=c["duration_ms"],
input_tokens=c.get("input_tokens", 0),
output_tokens=c.get("output_tokens", 0),
)
for c in llm_trace
]
def _get_usage() -> TokenUsageSummary:
return TokenUsageSummary(
input_tokens=total_input_tokens,
output_tokens=total_output_tokens,
total_tokens=total_input_tokens + total_output_tokens,
)
def _log_completion(answer: str, iterations: int, forced: bool = False):
elapsed_ms = int((time.time() - start_time) * 1000)
tools_summary = (
", ".join(
f"{t['tool']}({t['input_summary']})={t['duration_ms']}ms/{t.get('output_chars', 0)}c"
for t in tool_trace_summary
)
or "none"
)
llm_summary = ", ".join(f"{c['scope']}={c['duration_ms']}ms" for c in llm_trace) or "none"
total_llm_ms = sum(c["duration_ms"] for c in llm_trace)
total_tools_ms = sum(t["duration_ms"] for t in tool_trace_summary)
answer_preview = answer[:100] + "..." if len(answer) > 100 else answer
mode = "forced" if forced else "done"
logger.info(
f"[REFLECT {reflect_id}] {mode} | "
f"query='{query[:50]}...' | "
f"iterations={iterations} | "
f"llm=[{llm_summary}] ({total_llm_ms}ms) | "
f"tools=[{tools_summary}] ({total_tools_ms}ms) | "
f"answer='{answer_preview}' | "
f"total={elapsed_ms}ms"
)
for iteration in range(max_iterations):
is_last = iteration == max_iterations - 1
if is_last:
# Force text response on last iteration - no tools
prompt = build_final_prompt(query, context_history, bank_profile, context)
llm_start = time.time()
response, usage = await llm_config.call(
messages=[
{"role": "system", "content": FINAL_SYSTEM_PROMPT},
{"role": "user", "content": prompt},
],
scope="reflect",
max_completion_tokens=max_tokens,
return_usage=True,
)
llm_duration = int((time.time() - llm_start) * 1000)
total_input_tokens += usage.input_tokens
total_output_tokens += usage.output_tokens
llm_trace.append(
{
"scope": "final",
"duration_ms": llm_duration,
"input_tokens": usage.input_tokens,
"output_tokens": usage.output_tokens,
}
)
answer = _clean_answer_text(response.strip())
# Generate structured output if schema provided
structured_output = None
if response_schema and answer:
structured_output, struct_in, struct_out = await _generate_structured_output(
answer, response_schema, llm_config, reflect_id
)
total_input_tokens += struct_in
total_output_tokens += struct_out
_log_completion(answer, iteration + 1, forced=True)
return ReflectAgentResult(
text=answer,
structured_output=structured_output,
iterations=iteration + 1,
tools_called=total_tools_called,
tool_trace=tool_trace,
llm_trace=_get_llm_trace(),
usage=_get_usage(),
directives_applied=directives_applied,
)
# Call LLM with tools
llm_start = time.time()
try:
result = await llm_config.call_with_tools(
messages=messages,
tools=tools,
scope="reflect_tool_call",
tool_choice="required" if iteration == 0 else "auto", # Force tool use on first iteration
)
llm_duration = int((time.time() - llm_start) * 1000)
total_input_tokens += result.input_tokens
total_output_tokens += result.output_tokens
llm_trace.append(
{
"scope": f"agent_{iteration + 1}",
"duration_ms": llm_duration,
"input_tokens": result.input_tokens,
"output_tokens": result.output_tokens,
}
)
except Exception as e:
err_duration = int((time.time() - llm_start) * 1000)
logger.warning(f"[REFLECT {reflect_id}] LLM error on iteration {iteration + 1}: {e} ({err_duration}ms)")
llm_trace.append({"scope": f"agent_{iteration + 1}_err", "duration_ms": err_duration})
# Guardrail: If no evidence gathered yet, retry
has_gathered_evidence = (
bool(available_memory_ids) or bool(available_mental_model_ids) or bool(available_observation_ids)
)
if not has_gathered_evidence and iteration < max_iterations - 1:
continue
prompt = build_final_prompt(query, context_history, bank_profile, context)
llm_start = time.time()
response, usage = await llm_config.call(
messages=[
{"role": "system", "content": FINAL_SYSTEM_PROMPT},
{"role": "user", "content": prompt},
],
scope="reflect",
max_completion_tokens=max_tokens,
return_usage=True,
)
llm_duration = int((time.time() - llm_start) * 1000)
total_input_tokens += usage.input_tokens
total_output_tokens += usage.output_tokens
llm_trace.append(
{
"scope": "final",
"duration_ms": llm_duration,
"input_tokens": usage.input_tokens,
"output_tokens": usage.output_tokens,
}
)
answer = _clean_answer_text(response.strip())
# Generate structured output if schema provided
structured_output = None
if response_schema and answer:
structured_output, struct_in, struct_out = await _generate_structured_output(
answer, response_schema, llm_config, reflect_id
)
total_input_tokens += struct_in
total_output_tokens += struct_out
_log_completion(answer, iteration + 1, forced=True)
return ReflectAgentResult(
text=answer,
structured_output=structured_output,
iterations=iteration + 1,
tools_called=total_tools_called,
tool_trace=tool_trace,
llm_trace=_get_llm_trace(),
usage=_get_usage(),
directives_applied=directives_applied,
)
# No tool calls - LLM wants to respond with text
if not result.tool_calls:
if result.content:
answer = _clean_answer_text(result.content.strip())
# Generate structured output if schema provided
structured_output = None
if response_schema and answer:
structured_output, struct_in, struct_out = await _generate_structured_output(
answer, response_schema, llm_config, reflect_id
)
total_input_tokens += struct_in
total_output_tokens += struct_out
_log_completion(answer, iteration + 1)
return ReflectAgentResult(
text=answer,
structured_output=structured_output,
iterations=iteration + 1,
tools_called=total_tools_called,
tool_trace=tool_trace,
llm_trace=_get_llm_trace(),
usage=_get_usage(),
directives_applied=directives_applied,
)
# Empty response, force final
prompt = build_final_prompt(query, context_history, bank_profile, context)
llm_start = time.time()
response, usage = await llm_config.call(
messages=[
{"role": "system", "content": FINAL_SYSTEM_PROMPT},
{"role": "user", "content": prompt},
],
scope="reflect",
max_completion_tokens=max_tokens,
return_usage=True,
)
llm_duration = int((time.time() - llm_start) * 1000)
total_input_tokens += usage.input_tokens
total_output_tokens += usage.output_tokens
llm_trace.append(
{
"scope": "final",
"duration_ms": llm_duration,
"input_tokens": usage.input_tokens,
"output_tokens": usage.output_tokens,
}
)
answer = _clean_answer_text(response.strip())
# Generate structured output if schema provided
structured_output = None
if response_schema and answer:
structured_output, struct_in, struct_out = await _generate_structured_output(
answer, response_schema, llm_config, reflect_id
)
total_input_tokens += struct_in
total_output_tokens += struct_out
_log_completion(answer, iteration + 1, forced=True)
return ReflectAgentResult(
text=answer,
structured_output=structured_output,
iterations=iteration + 1,
tools_called=total_tools_called,
tool_trace=tool_trace,
llm_trace=_get_llm_trace(),
usage=_get_usage(),
directives_applied=directives_applied,
)
# Check for done tool call (handle various LLM output formats)
done_call = next((tc for tc in result.tool_calls if _is_done_tool(tc.name)), None)
if done_call:
# Guardrail: Require evidence before done
has_gathered_evidence = (
bool(available_memory_ids) or bool(available_mental_model_ids) or bool(available_observation_ids)
)
if not has_gathered_evidence and iteration < max_iterations - 1:
# Add assistant message and fake tool result asking for evidence
messages.append(
{
"role": "assistant",
"tool_calls": [_tool_call_to_dict(done_call)],
}
)
messages.append(
{
"role": "tool",
"tool_call_id": done_call.id,
"name": done_call.name, # Required by Gemini
"content": json.dumps(
{
"error": "You must search for information first. Use search_mental_models(), search_observations(), or recall() before providing your final answer."
}
),
}
)
continue
# Process done tool - wrap with tool call span
from hindsight_api.tracing import get_tracer
tracer = get_tracer()
span_name = "hindsight.reflect_tool_call"
with tracer.start_as_current_span(span_name) as span:
span.set_attribute("hindsight.scope", "reflect_tool_call")
span.set_attribute("hindsight.operation", "reflect_tool_call")
return await _process_done_tool(
done_call,
available_memory_ids,
available_mental_model_ids,
available_observation_ids,
iteration + 1,
total_tools_called,
tool_trace,
_get_llm_trace(),
_get_usage(),
_log_completion,
reflect_id,
directives_applied=directives_applied,
llm_config=llm_config,
response_schema=response_schema,
)
# Execute other tools in parallel (exclude done tool in all its format variants)
other_tools = [tc for tc in result.tool_calls if not _is_done_tool(tc.name)]
if other_tools:
# Add assistant message with tool calls
messages.append(
{
"role": "assistant",
"tool_calls": [_tool_call_to_dict(tc) for tc in other_tools],
}
)
# Execute tools in parallel
tool_tasks = [
_execute_tool_with_timing(
tc,
search_mental_models_fn,
search_observations_fn,
recall_fn,
expand_fn,
)
for tc in other_tools
]
tool_results = await asyncio.gather(*tool_tasks, return_exceptions=True)
total_tools_called += len(other_tools)
# Process results and add to messages
for tc, result_data in zip(other_tools, tool_results):
if isinstance(result_data, Exception):
# Tool execution failed - send error back to LLM so it can try again
logger.warning(f"[REFLECT {reflect_id}] Tool {tc.name} failed with exception: {result_data}")
output = {"error": f"Tool execution failed: {result_data}"}
duration_ms = 0
else:
output, duration_ms = result_data
# Normalize tool name for consistent tracking
normalized_tool_name = _normalize_tool_name(tc.name)
# Check if tool returned an error response - log but continue (LLM will see the error)
if isinstance(output, dict) and "error" in output:
logger.warning(
f"[REFLECT {reflect_id}] Tool {normalized_tool_name} returned error: {output['error']}"
)
# Track available IDs from tool results (only for successful responses)
if (
normalized_tool_name == "search_mental_models"
and isinstance(output, dict)
and "mental_models" in output
):
for mm in output["mental_models"]:
if "id" in mm:
available_mental_model_ids.add(mm["id"])
if (
normalized_tool_name == "search_observations"
and isinstance(output, dict)
and "observations" in output
):
for obs in output["observations"]:
if "id" in obs:
available_observation_ids.add(obs["id"])
if normalized_tool_name == "recall" and isinstance(output, dict) and "memories" in output:
for memory in output["memories"]:
if "id" in memory:
available_memory_ids.add(memory["id"])
# Add tool result message
messages.append(
{
"role": "tool",
"tool_call_id": tc.id,
"name": tc.name, # Required by Gemini
"content": json.dumps(output, default=str),
}
)
# Track for logging and context history
input_dict = {"tool": tc.name, **tc.arguments}
input_summary = _summarize_input(tc.name, tc.arguments)
# Extract reason from tool arguments (if provided)
tool_reason = tc.arguments.get("reason")
tool_trace.append(
ToolCall(
tool=tc.name,
reason=tool_reason,
input=input_dict,
output=output,
duration_ms=duration_ms,
iteration=iteration + 1,
)
)
try:
output_chars = len(json.dumps(output))
except (TypeError, ValueError):
output_chars = len(str(output))
tool_trace_summary.append(
{
"tool": tc.name,
"input_summary": input_summary,
"duration_ms": duration_ms,
"output_chars": output_chars,
}
)
# Keep context history for fallback final prompt
context_history.append({"tool": tc.name, "input": input_dict, "output": output})
# Should not reach here
answer = "I was unable to formulate a complete answer within the iteration limit."
_log_completion(answer, max_iterations, forced=True)
return ReflectAgentResult(
text=answer,
iterations=max_iterations,
tools_called=total_tools_called,
tool_trace=tool_trace,
llm_trace=_get_llm_trace(),
usage=_get_usage(),
directives_applied=directives_applied,
)
def _tool_call_to_dict(tc: "LLMToolCall") -> dict[str, Any]:
"""Convert LLMToolCall to OpenAI message format."""
return {
"id": tc.id,
"type": "function",
"function": {
"name": tc.name,
"arguments": json.dumps(tc.arguments),
},
}
async def _process_done_tool(
done_call: "LLMToolCall",
available_memory_ids: set[str],
available_mental_model_ids: set[str],
available_observation_ids: set[str],
iterations: int,
total_tools_called: int,
tool_trace: list[ToolCall],
llm_trace: list[LLMCall],
usage: TokenUsageSummary,
log_completion: Callable,
reflect_id: str,
directives_applied: list[DirectiveInfo],
llm_config: "LLMProvider | None" = None,
response_schema: dict | None = None,
) -> ReflectAgentResult:
"""Process the done tool call and return the result."""
args = done_call.arguments
# Extract and clean the answer - some LLMs leak structured output into the answer text
raw_answer = args.get("answer", "").strip()
answer = _clean_done_answer(raw_answer) if raw_answer else ""
if not answer:
answer = "No answer provided."
# Validate IDs (only include IDs that were actually retrieved)
used_memory_ids = [mid for mid in args.get("memory_ids", []) if mid in available_memory_ids]
used_mental_model_ids = [mid for mid in args.get("mental_model_ids", []) if mid in available_mental_model_ids]
used_observation_ids = [oid for oid in args.get("observation_ids", []) if oid in available_observation_ids]
# Generate structured output if schema provided
structured_output = None
final_usage = usage
if response_schema and llm_config and answer:
structured_output, struct_in, struct_out = await _generate_structured_output(
answer, response_schema, llm_config, reflect_id
)
# Add structured output tokens to usage
final_usage = TokenUsageSummary(
input_tokens=usage.input_tokens + struct_in,
output_tokens=usage.output_tokens + struct_out,
total_tokens=usage.total_tokens + struct_in + struct_out,
)
log_completion(answer, iterations)
return ReflectAgentResult(
text=answer,
structured_output=structured_output,
iterations=iterations,
tools_called=total_tools_called,
tool_trace=tool_trace,
llm_trace=llm_trace,
usage=final_usage,
used_memory_ids=used_memory_ids,
used_mental_model_ids=used_mental_model_ids,
used_observation_ids=used_observation_ids,
directives_applied=directives_applied,
)
async def _execute_tool_with_timing(
tc: "LLMToolCall",
search_mental_models_fn: Callable[[str, int], Awaitable[dict[str, Any]]],
search_observations_fn: Callable[[str, int], Awaitable[dict[str, Any]]],
recall_fn: Callable[[str, int], Awaitable[dict[str, Any]]],
expand_fn: Callable[[list[str], str], Awaitable[dict[str, Any]]],
) -> tuple[dict[str, Any], int]:
"""Execute a tool call and return result with timing."""
from hindsight_api.tracing import get_tracer
start_time = time.time()
# Create span for tool execution
tracer = get_tracer()
# Normalize tool name for span
normalized_name = _normalize_tool_name(tc.name)
span_name = f"hindsight.reflect_tool_exec.{normalized_name}"
# Calculate timestamps
start_time_ns = time.time_ns()
with tracer.start_as_current_span(
span_name,
start_time=start_time_ns,
end_on_exit=False,
) as span:
# Set attributes
span.set_attribute("hindsight.tool.name", normalized_name)
span.set_attribute("hindsight.tool.id", tc.id)
span.set_attribute("hindsight.tool.arguments", json.dumps(tc.arguments))
try:
result = await _execute_tool(
tc.name,
tc.arguments,
search_mental_models_fn,
search_observations_fn,
recall_fn,
expand_fn,
)
# Set success attributes
if isinstance(result, dict) and "error" in result:
from opentelemetry.trace import Status, StatusCode
span.set_status(Status(StatusCode.ERROR, result["error"]))
else:
from opentelemetry.trace import Status, StatusCode
span.set_status(Status(StatusCode.OK))
duration_ms = int((time.time() - start_time) * 1000)
span.set_attribute("hindsight.tool.duration_ms", duration_ms)
# End span with correct timestamp
end_time_ns = time.time_ns()
span.end(end_time=end_time_ns)
return result, duration_ms
except Exception as e:
from opentelemetry.trace import Status, StatusCode
span.set_status(Status(StatusCode.ERROR, str(e)))
span.record_exception(e)
duration_ms = int((time.time() - start_time) * 1000)
span.set_attribute("hindsight.tool.duration_ms", duration_ms)
end_time_ns = time.time_ns()
span.end(end_time=end_time_ns)
raise
async def _execute_tool(
tool_name: str,
args: dict[str, Any],
search_mental_models_fn: Callable[[str, int], Awaitable[dict[str, Any]]],
search_observations_fn: Callable[[str, int], Awaitable[dict[str, Any]]],
recall_fn: Callable[[str, int], Awaitable[dict[str, Any]]],
expand_fn: Callable[[list[str], str], Awaitable[dict[str, Any]]],
) -> dict[str, Any]:
"""Execute a single tool by name."""
# Normalize tool name for various LLM output formats
tool_name = _normalize_tool_name(tool_name)
if tool_name == "search_mental_models":
query = args.get("query")
if not query:
return {"error": "search_mental_models requires a query parameter"}
max_results = int(args.get("max_results") or 5)
return await search_mental_models_fn(query, max_results)
elif tool_name == "search_observations":
query = args.get("query")
if not query:
return {"error": "search_observations requires a query parameter"}
max_tokens = max(int(args.get("max_tokens") or 5000), 1000) # Default 5000, min 1000
return await search_observations_fn(query, max_tokens)
elif tool_name == "recall":
query = args.get("query")
if not query:
return {"error": "recall requires a query parameter"}
max_tokens = max(int(args.get("max_tokens") or 2048), 1000) # Default 2048, min 1000
return await recall_fn(query, max_tokens)
elif tool_name == "expand":
memory_ids = args.get("memory_ids", [])
if not memory_ids:
return {"error": "expand requires memory_ids"}
depth = args.get("depth", "chunk")
return await expand_fn(memory_ids, depth)
else:
return {"error": f"Unknown tool: {tool_name}"}
def _summarize_input(tool_name: str, args: dict[str, Any]) -> str:
"""Create a summary of tool input for logging, showing all params."""
if tool_name == "search_mental_models":
query = args.get("query", "")
query_preview = f"'{query[:30]}...'" if len(query) > 30 else f"'{query}'"
max_results = int(args.get("max_results") or 5)
return f"(query={query_preview}, max_results={max_results})"
elif tool_name == "search_observations":
query = args.get("query", "")
query_preview = f"'{query[:30]}...'" if len(query) > 30 else f"'{query}'"
max_tokens = max(int(args.get("max_tokens") or 5000), 1000)
return f"(query={query_preview}, max_tokens={max_tokens})"
elif tool_name == "recall":
query = args.get("query", "")
query_preview = f"'{query[:30]}...'" if len(query) > 30 else f"'{query}'"
# Show actual value used (default 2048, min 1000)
max_tokens = max(int(args.get("max_tokens") or 2048), 1000)
return f"(query={query_preview}, max_tokens={max_tokens})"
elif tool_name == "expand":
memory_ids = args.get("memory_ids", [])
depth = args.get("depth", "chunk")
return f"(memory_ids=[{len(memory_ids)} ids], depth={depth})"
elif tool_name == "done":
answer = args.get("answer", "")
answer_preview = f"'{answer[:30]}...'" if len(answer) > 30 else f"'{answer}'"
memory_ids = args.get("memory_ids", [])
mental_model_ids = args.get("mental_model_ids", [])
observation_ids = args.get("observation_ids", [])
return (
f"(answer={answer_preview}, mem={len(memory_ids)}, mm={len(mental_model_ids)}, obs={len(observation_ids)})"
)
return str(args)
@@ -1,109 +0,0 @@
"""
Pydantic models for the reflect agent.
"""
from typing import Any, Literal
from pydantic import BaseModel, Field
class ObservationSection(BaseModel):
"""A section within an observation with its supporting memories."""
title: str = Field(description="Section header (can be empty for intro)")
text: str = Field(description="Section content - no headers, use lists/tables/bold")
memory_ids: list[str] = Field(default_factory=list, description="Memory IDs supporting this section")
class ReflectAction(BaseModel):
"""Single action the reflect agent can take."""
tool: Literal["list_observations", "get_observation", "recall", "expand", "done"] = Field(
description="Tool to invoke: list_observations, get_observation, recall, expand, or done"
)
# Tool-specific parameters
observation_id: str | None = Field(default=None, description="Observation ID for get_observation")
query: str | None = Field(default=None, description="Search query for recall")
max_tokens: int | None = Field(default=None, description="Max tokens for recall results (default 2048)")
memory_ids: list[str] | None = Field(default=None, description="Memory unit IDs for expand (batched)")
depth: Literal["chunk", "document"] | None = Field(default=None, description="Expansion depth for expand")
observation_sections: list[ObservationSection] | None = Field(
default=None, description="Observation sections for done action (when output_mode=observations)"
)
# Plain text answer fields (for output_mode=answer)
answer: str | None = Field(default=None, description="Well-formatted markdown answer for done action")
answer_memory_ids: list[str] | None = Field(
default=None, description="Memory IDs supporting the answer", alias="memory_ids"
)
answer_model_ids: list[str] | None = Field(
default=None, description="Mental model IDs supporting the answer", alias="model_ids"
)
reasoning: str | None = Field(default=None, description="Brief reasoning for this action")
class ReflectActionBatch(BaseModel):
"""Batch of actions for parallel execution."""
actions: list[ReflectAction] = Field(description="List of actions to execute in parallel")
class ToolCall(BaseModel):
"""A single tool call made during reflect."""
tool: str = Field(description="Tool name: lookup, recall, expand")
reason: str | None = Field(default=None, description="Agent's reasoning for making this tool call")
input: dict = Field(description="Tool input parameters")
output: dict = Field(description="Tool output/result")
duration_ms: int = Field(description="Execution time in milliseconds")
iteration: int = Field(default=0, description="Iteration number (1-based) when this tool was called")
class LLMCall(BaseModel):
"""A single LLM call made during reflect."""
scope: str = Field(description="Call scope: agent_1, agent_2, final, etc.")
duration_ms: int = Field(description="Execution time in milliseconds")
input_tokens: int = Field(default=0, description="Input tokens used")
output_tokens: int = Field(default=0, description="Output tokens used")
class DirectiveInfo(BaseModel):
"""Information about a directive that was applied during reflect."""
id: str = Field(description="Directive mental model ID")
name: str = Field(description="Directive name")
content: str = Field(description="Directive content")
class TokenUsageSummary(BaseModel):
"""Total token usage across all LLM calls."""
input_tokens: int = Field(default=0, description="Total input tokens used")
output_tokens: int = Field(default=0, description="Total output tokens used")
total_tokens: int = Field(default=0, description="Total tokens (input + output)")
class ReflectAgentResult(BaseModel):
"""Result from the reflect agent."""
text: str = Field(description="Final answer text")
structured_output: dict[str, Any] | None = Field(
default=None, description="Structured output parsed according to provided response_schema"
)
iterations: int = Field(default=0, description="Number of iterations taken")
tools_called: int = Field(default=0, description="Total number of tool calls made")
tool_trace: list[ToolCall] = Field(default_factory=list, description="Trace of all tool calls made")
llm_trace: list[LLMCall] = Field(default_factory=list, description="Trace of all LLM calls made")
usage: TokenUsageSummary = Field(
default_factory=TokenUsageSummary, description="Total token usage across all LLM calls"
)
used_memory_ids: list[str] = Field(default_factory=list, description="Validated memory IDs actually used in answer")
used_mental_model_ids: list[str] = Field(
default_factory=list, description="Validated mental model IDs actually used in answer"
)
used_observation_ids: list[str] = Field(
default_factory=list, description="Validated observation IDs actually used in answer"
)
directives_applied: list[DirectiveInfo] = Field(
default_factory=list, description="Directive mental models that affected this reflection"
)
@@ -1,186 +0,0 @@
"""
Models and utilities for evidence-grounded observations with computed trends.
Observations are part of mental models and represent patterns/beliefs derived
from memories. Each observation must be grounded in specific evidence (quotes)
from memories, and trends are computed algorithmically from evidence timestamps.
"""
from datetime import datetime, timedelta, timezone
from enum import Enum
from pydantic import BaseModel, Field, computed_field, field_validator
class Trend(str, Enum):
"""Computed trend for an observation based on evidence timestamps.
Trends indicate how an observation's evidence is distributed over time:
- STABLE: Evidence spread across time, continues to present
- STRENGTHENING: More/denser evidence recently than before
- WEAKENING: Evidence mostly old, sparse recently
- NEW: All evidence within recent window
- STALE: No evidence in recent window (may no longer apply)
"""
STABLE = "stable"
STRENGTHENING = "strengthening"
WEAKENING = "weakening"
NEW = "new"
STALE = "stale"
class ObservationEvidence(BaseModel):
"""A single piece of evidence supporting an observation.
Each evidence item must include an exact quote from the source memory
to ensure observations are grounded and verifiable.
"""
memory_id: str = Field(description="ID of the memory unit this evidence comes from")
quote: str = Field(description="Exact quote from the memory supporting the observation")
relevance: str = Field(default="", description="Brief explanation of how this quote supports the observation")
timestamp: datetime = Field(description="When the source memory was created")
@field_validator("timestamp", mode="before")
@classmethod
def ensure_timezone_aware(cls, v: datetime | str | None) -> datetime:
"""Ensure timestamp is always timezone-aware UTC."""
if v is None:
return datetime.now(timezone.utc)
if isinstance(v, str):
# Parse ISO format string, handling 'Z' suffix
v = datetime.fromisoformat(v.replace("Z", "+00:00"))
if isinstance(v, datetime):
if v.tzinfo is None:
return v.replace(tzinfo=timezone.utc)
return v
raise ValueError(f"Invalid timestamp type: {type(v)}")
class Observation(BaseModel):
"""A single observation within a mental model.
Observations represent patterns, preferences, beliefs, or other insights
derived from memories. Each observation must be grounded in evidence
with exact quotes from source memories.
"""
title: str = Field(description="Short summary title for the observation (5-10 words)")
content: str = Field(description="The observation content - detailed explanation of what we believe to be true")
evidence: list[ObservationEvidence] = Field(default_factory=list, description="Supporting evidence with quotes")
created_at: datetime = Field(
default_factory=lambda: datetime.now(timezone.utc), description="When this observation was first created"
)
@field_validator("created_at", mode="before")
@classmethod
def ensure_created_at_timezone_aware(cls, v: datetime | str | None) -> datetime:
"""Ensure created_at is always timezone-aware UTC."""
if v is None:
return datetime.now(timezone.utc)
if isinstance(v, str):
v = datetime.fromisoformat(v.replace("Z", "+00:00"))
if isinstance(v, datetime):
if v.tzinfo is None:
return v.replace(tzinfo=timezone.utc)
return v
raise ValueError(f"Invalid created_at type: {type(v)}")
@computed_field
@property
def trend(self) -> Trend:
"""Compute trend from evidence timestamps."""
return compute_trend(self.evidence)
@computed_field
@property
def evidence_span(self) -> dict[str, str | None]:
"""Get the time span covered by evidence."""
if not self.evidence:
return {"from": None, "to": None}
timestamps = [e.timestamp for e in self.evidence]
return {
"from": min(timestamps).isoformat(),
"to": max(timestamps).isoformat(),
}
@computed_field
@property
def evidence_count(self) -> int:
"""Number of evidence items supporting this observation."""
return len(self.evidence)
def compute_trend(
evidence: list[ObservationEvidence],
now: datetime | None = None,
recent_days: int = 30,
old_days: int = 90,
) -> Trend:
"""Compute the trend for an observation based on evidence timestamps.
The trend indicates how the evidence is distributed over time:
- STABLE: Evidence spread across time, continues to present
- STRENGTHENING: More evidence recently than historically
- WEAKENING: Evidence mostly old, sparse recently
- NEW: All evidence is recent (within recent_days)
- STALE: No evidence in recent window
Args:
evidence: List of evidence items with timestamps
now: Reference time for calculations (defaults to current UTC time)
recent_days: Number of days to consider "recent" (default 30)
old_days: Number of days to consider "old" (default 90)
Returns:
Computed Trend enum value
"""
if now is None:
now = datetime.now(timezone.utc)
# Ensure now is timezone-aware
if now.tzinfo is None:
now = now.replace(tzinfo=timezone.utc)
if not evidence:
return Trend.STALE
recent_cutoff = now - timedelta(days=recent_days)
old_cutoff = now - timedelta(days=old_days)
# Normalize timestamps to UTC for comparison
def normalize_ts(ts: datetime) -> datetime:
if ts.tzinfo is None:
return ts.replace(tzinfo=timezone.utc)
return ts
recent = [e for e in evidence if normalize_ts(e.timestamp) > recent_cutoff]
old = [e for e in evidence if normalize_ts(e.timestamp) < old_cutoff]
middle = [e for e in evidence if old_cutoff <= normalize_ts(e.timestamp) <= recent_cutoff]
# No recent evidence = stale
if not recent:
return Trend.STALE
# All evidence is recent = new
if not old and not middle:
return Trend.NEW
# Compare density (evidence per day)
recent_density = len(recent) / recent_days if recent_days > 0 else 0
older_period = old_days - recent_days
older_density = (len(old) + len(middle)) / older_period if older_period > 0 else 0
# Avoid division by zero
if older_density == 0:
return Trend.NEW
ratio = recent_density / older_density
if ratio > 1.5:
return Trend.STRENGTHENING
elif ratio < 0.5:
return Trend.WEAKENING
else:
return Trend.STABLE
@@ -1,513 +0,0 @@
"""
System prompts for the reflect agent.
The reflect agent uses hierarchical retrieval:
1. search_mental_models - User-curated summaries (highest quality)
2. search_observations - Consolidated knowledge with freshness awareness
3. recall - Raw facts as ground truth fallback
"""
import json
from typing import Any
def _extract_directive_rules(directives: list[dict[str, Any]]) -> list[str]:
"""
Extract directive rules as a list of strings.
Args:
directives: List of directives with name and content
Returns:
List of directive rule strings
"""
rules = []
for directive in directives:
directive_name = directive.get("name", "")
# New format: directives have direct content field
content = directive.get("content", "")
if content:
if directive_name:
rules.append(f"**{directive_name}**: {content}")
else:
rules.append(content)
else:
# Legacy format: check for observations
observations = directive.get("observations", [])
if observations:
for obs in observations:
# Support both Pydantic Observation objects and dicts
if hasattr(obs, "title"):
title = obs.title
obs_content = obs.content
else:
title = obs.get("title", "")
obs_content = obs.get("content", "")
if title and obs_content:
rules.append(f"**{title}**: {obs_content}")
elif obs_content:
rules.append(obs_content)
elif directive_name:
# Fallback to description
desc = directive.get("description", "")
if desc:
rules.append(f"**{directive_name}**: {desc}")
return rules
def build_directives_section(directives: list[dict[str, Any]]) -> str:
"""
Build the directives section for the system prompt.
Directives are hard rules that MUST be followed in all responses.
Args:
directives: List of directive mental models with observations
"""
if not directives:
return ""
rules = _extract_directive_rules(directives)
if not rules:
return ""
parts = [
"## DIRECTIVES (MANDATORY)",
"These are hard rules you MUST follow in ALL responses:",
"",
]
for rule in rules:
parts.append(f"- {rule}")
parts.extend(
[
"",
"NEVER violate these directives, even if other context suggests otherwise.",
"IMPORTANT: Do NOT explain or justify how you handled directives in your answer. Just follow them silently.",
"",
]
)
return "\n".join(parts)
def build_directives_reminder(directives: list[dict[str, Any]]) -> str:
"""
Build a reminder section for directives to place at the end of the prompt.
Args:
directives: List of directive mental models with observations
"""
if not directives:
return ""
rules = _extract_directive_rules(directives)
if not rules:
return ""
parts = [
"",
"## REMINDER: MANDATORY DIRECTIVES",
"Before responding, ensure your answer complies with ALL of these directives:",
"",
]
for i, rule in enumerate(rules, 1):
parts.append(f"{i}. {rule}")
parts.append("")
parts.append("Your response will be REJECTED if it violates any directive above.")
parts.append("Do NOT include any commentary about how you handled directives - just follow them.")
return "\n".join(parts)
def build_system_prompt_for_tools(
bank_profile: dict[str, Any],
context: str | None = None,
directives: list[dict[str, Any]] | None = None,
has_mental_models: bool = False,
budget: str | None = None,
) -> str:
"""
Build the system prompt for tool-calling reflect agent.
The agent uses hierarchical retrieval:
1. search_mental_models - User-curated summaries (try first, if available)
2. search_observations - Consolidated knowledge with freshness
3. recall - Raw facts as ground truth
Args:
bank_profile: Bank profile with name and mission
context: Optional additional context
directives: Optional list of directive mental models to inject as hard rules
has_mental_models: Whether the bank has any mental models (skip if not)
budget: Search depth budget - "low", "mid", or "high". Controls exploration thoroughness.
"""
name = bank_profile.get("name", "Assistant")
mission = bank_profile.get("mission", "")
parts = []
# Anti-hallucination rule at the very top
parts.extend(
[
"CRITICAL: You MUST ONLY use information from retrieved tool results. NEVER make up names, people, events, or entities.",
"",
]
)
# Inject directives after anti-hallucination rule
if directives:
parts.append(build_directives_section(directives))
parts.extend(
[
"You are a reflection agent that answers questions by reasoning over retrieved memories.",
"",
]
)
parts.extend(
[
"## CRITICAL RULES",
"- ONLY use information from tool results - no external knowledge or guessing",
"- You SHOULD synthesize, infer, and reason from the retrieved memories",
"- You MUST search before saying you don't have information",
"",
"## How to Reason",
"- If memories mention someone did an activity, you can infer they likely enjoyed it",
"- Synthesize a coherent narrative from related memories",
"- Be a thoughtful interpreter, not just a literal repeater",
"- When the exact answer isn't stated, use what IS stated to give the best answer",
"",
"## HIERARCHICAL RETRIEVAL STRATEGY",
"",
]
)
# Build retrieval levels based on what's available
if has_mental_models:
parts.extend(
[
"You have access to THREE levels of knowledge. Use them in this order:",
"",
"### 1. MENTAL MODELS (search_mental_models) - Try First",
"- User-curated summaries about specific topics",
"- HIGHEST quality - manually created and maintained",
"- If a relevant mental model exists and is FRESH, it may fully answer the question",
"- Check `is_stale` field - if stale, also verify with lower levels",
"",
"### 2. OBSERVATIONS (search_observations) - Second Priority",
"- Auto-consolidated knowledge from memories",
"- Check `is_stale` field - if stale, ALSO use recall() to verify",
"- Good for understanding patterns and summaries",
"",
"### 3. RAW FACTS (recall) - Ground Truth",
"- Individual memories (world facts and experiences)",
"- Use when: no mental models/observations exist, they're stale, or you need specific details",
"- This is the source of truth that other levels are built from",
"",
]
)
else:
parts.extend(
[
"You have access to TWO levels of knowledge. Use them in this order:",
"",
"### 1. OBSERVATIONS (search_observations) - Try First",
"- Auto-consolidated knowledge from memories",
"- Check `is_stale` field - if stale, ALSO use recall() to verify",
"- Good for understanding patterns and summaries",
"",
"### 2. RAW FACTS (recall) - Ground Truth",
"- Individual memories (world facts and experiences)",
"- Use when: no observations exist, they're stale, or you need specific details",
"- This is the source of truth that observations are built from",
"",
]
)
parts.extend(
[
"## Query Strategy",
"recall() uses semantic search. NEVER just echo the user's question - decompose it into targeted searches:",
"",
"BAD: User asks 'recurring lesson themes between students' → recall('recurring lesson themes between students')",
"GOOD: Break it down into component searches:",
" 1. recall('lessons') - find all lesson-related memories",
" 2. recall('teaching sessions') - alternative phrasing",
" 3. recall('student progress') - find student-related memories",
"",
"Think: What ENTITIES and CONCEPTS does this question involve? Search for each separately.",
"",
]
)
# Add budget guidance
if budget:
budget_lower = budget.lower()
if budget_lower == "low":
parts.extend(
[
"## RESEARCH DEPTH: SHALLOW (Quick Response)",
"- Prioritize speed over completeness",
"- If mental models or observations provide a reasonable answer, stop there",
"- Only dig deeper if the initial results are clearly insufficient",
"- Prefer a quick overview rather than exhaustive details",
"- Answer promptly with available information",
"",
]
)
elif budget_lower == "mid":
parts.extend(
[
"## RESEARCH DEPTH: MODERATE (Balanced)",
"- Balance thoroughness with efficiency",
"- Check multiple sources when the question warrants it",
"- Verify stale data if it's central to the answer",
"- Don't over-explore, but ensure reasonable coverage",
"",
]
)
elif budget_lower == "high":
parts.extend(
[
"## RESEARCH DEPTH: DEEP (Thorough Exploration)",
"- Explore comprehensively before answering",
"- Search across all available knowledge levels",
"- Use multiple query variations to ensure coverage",
"- Verify information across different retrieval levels",
"- Use expand() to get full context on important memories",
"- Take time to synthesize a complete, well-researched answer",
"",
]
)
parts.append("## Workflow")
if has_mental_models:
parts.extend(
[
"1. First, try search_mental_models() - check if a curated summary exists",
"2. If no mental model or it's stale, try search_observations() for consolidated knowledge",
"3. If observations are stale OR you need specific details, use recall() for raw facts",
"4. Use expand() if you need more context on specific memories",
"5. When ready, call done() with your answer and supporting IDs",
]
)
else:
parts.extend(
[
"1. First, try search_observations() - check for consolidated knowledge",
"2. If observations are stale OR you need specific details, use recall() for raw facts",
"3. Use expand() if you need more context on specific memories",
"4. When ready, call done() with your answer and supporting IDs",
]
)
parts.extend(
[
"",
"## Output Format: Well-Formatted Markdown Answer",
"Call done() with a well-formatted markdown 'answer' field.",
"- USE markdown formatting for structure (headers, lists, bold, italic, code blocks, tables, etc.)",
"- CRITICAL: Add blank lines before and after block elements (tables, code blocks, lists)",
"- Format for clarity and readability with proper spacing and hierarchy",
"- NEVER include memory IDs, UUIDs, or 'Memory references' in the answer text",
"- Put IDs ONLY in the memory_ids/mental_model_ids/observation_ids arrays, not in the answer",
]
)
parts.append("")
parts.append(f"## Memory Bank: {name}")
if mission:
parts.append(f"Mission: {mission}")
# Disposition traits
disposition = bank_profile.get("disposition", {})
if disposition:
traits = []
if "skepticism" in disposition:
traits.append(f"skepticism={disposition['skepticism']}")
if "literalism" in disposition:
traits.append(f"literalism={disposition['literalism']}")
if "empathy" in disposition:
traits.append(f"empathy={disposition['empathy']}")
if traits:
parts.append(f"Disposition: {', '.join(traits)}")
if context:
parts.append(f"\n## Additional Context\n{context}")
# Add directive reminder at the END for recency effect
if directives:
parts.append(build_directives_reminder(directives))
return "\n".join(parts)
def build_agent_prompt(
query: str,
context_history: list[dict],
bank_profile: dict,
additional_context: str | None = None,
) -> str:
"""Build the user prompt for the reflect agent."""
parts = []
# Bank identity
name = bank_profile.get("name", "Assistant")
mission = bank_profile.get("mission", "")
parts.append(f"## Memory Bank Context\nName: {name}")
if mission:
parts.append(f"Mission: {mission}")
# Disposition traits if present
disposition = bank_profile.get("disposition", {})
if disposition:
traits = []
if "skepticism" in disposition:
traits.append(f"skepticism={disposition['skepticism']}")
if "literalism" in disposition:
traits.append(f"literalism={disposition['literalism']}")
if "empathy" in disposition:
traits.append(f"empathy={disposition['empathy']}")
if traits:
parts.append(f"Disposition: {', '.join(traits)}")
# Additional context from caller
if additional_context:
parts.append(f"\n## Additional Context\n{additional_context}")
# Tool call history
if context_history:
parts.append("\n## Tool Results (synthesize and reason from this data)")
for i, entry in enumerate(context_history, 1):
tool = entry["tool"]
output = entry["output"]
# Format as proper JSON for LLM readability
try:
output_str = json.dumps(output, indent=2, default=str)
except (TypeError, ValueError):
output_str = str(output)
parts.append(f"\n### Call {i}: {tool}\n```json\n{output_str}\n```")
# The question
parts.append(f"\n## Question\n{query}")
# Instructions
if context_history:
parts.append(
"\n## Instructions\n"
"Based on the tool results above, either call more tools or provide your final answer. "
"Synthesize and reason from the data - make reasonable inferences when helpful. "
"If you have related information, use it to give the best possible answer."
)
else:
parts.append(
"\n## Instructions\n"
"Start by searching for relevant information using the hierarchical retrieval strategy:\n"
"1. Try search_mental_models() first for curated summaries\n"
"2. Try search_observations() for consolidated knowledge\n"
"3. Use recall() for specific details or to verify stale data"
)
return "\n".join(parts)
def build_final_prompt(
query: str,
context_history: list[dict],
bank_profile: dict,
additional_context: str | None = None,
) -> str:
"""Build the final prompt when forcing a text response (no tools)."""
parts = []
# Bank identity
name = bank_profile.get("name", "Assistant")
mission = bank_profile.get("mission", "")
parts.append(f"## Memory Bank Context\nName: {name}")
if mission:
parts.append(f"Mission: {mission}")
# Disposition traits if present
disposition = bank_profile.get("disposition", {})
if disposition:
traits = []
if "skepticism" in disposition:
traits.append(f"skepticism={disposition['skepticism']}")
if "literalism" in disposition:
traits.append(f"literalism={disposition['literalism']}")
if "empathy" in disposition:
traits.append(f"empathy={disposition['empathy']}")
if traits:
parts.append(f"Disposition: {', '.join(traits)}")
# Additional context from caller
if additional_context:
parts.append(f"\n## Additional Context\n{additional_context}")
# Tool call history
if context_history:
parts.append("\n## Retrieved Data (synthesize and reason from this data)")
for entry in context_history:
tool = entry["tool"]
output = entry["output"]
# Format as proper JSON for LLM readability
try:
output_str = json.dumps(output, indent=2, default=str)
except (TypeError, ValueError):
output_str = str(output)
parts.append(f"\n### From {tool}:\n```json\n{output_str}\n```")
else:
parts.append("\n## Retrieved Data\nNo data was retrieved.")
# The question
parts.append(f"\n## Question\n{query}")
# Final instructions
parts.append(
"\n## Instructions\n"
"Provide a thoughtful answer by synthesizing and reasoning from the retrieved data above. "
"You can make reasonable inferences from the memories, but don't completely fabricate information. "
"If the exact answer isn't stated, use what IS stated to give the best possible answer. "
"Only say 'I don't have information' if the retrieved data is truly unrelated to the question.\n\n"
"IMPORTANT: Output ONLY the final answer. Do NOT include meta-commentary like "
'"I\'ll search..." or "Let me analyze...". Do NOT explain your reasoning process. '
"Just provide the direct synthesized answer."
)
return "\n".join(parts)
FINAL_SYSTEM_PROMPT = """CRITICAL: You MUST ONLY use information from retrieved tool results. NEVER make up names, people, events, or entities.
You are a thoughtful assistant that synthesizes answers from retrieved memories.
Your approach:
- Reason over the retrieved memories to answer the question
- Make reasonable inferences when the exact answer isn't explicitly stated
- Connect related memories to form a complete picture
- Be helpful - if you have related information, use it to give the best possible answer
- ONLY use information from tool results - no external knowledge or guessing
Only say "I don't have information" if the retrieved data is truly unrelated to the question.
FORMATTING: Use proper markdown formatting in your answer:
- Headers (##, ###) for sections
- Lists (bullet or numbered) for enumerations
- Bold/italic for emphasis
- Tables with proper syntax (ensure blank line before and after)
- Code blocks where appropriate
- CRITICAL: Always add blank lines before and after block elements (tables, code blocks, lists)
- Proper spacing between sections
CRITICAL: Output ONLY the final synthesized answer. Do NOT include:
- Meta-commentary about what you're doing ("I'll search...", "Let me analyze...")
- Explanations of your reasoning process
- Descriptions of your approach
Just provide the direct answer with proper markdown formatting."""

Some files were not shown because too many files have changed in this diff Show More