Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
7519ecac95 | ||
|
|
ab0f8cec33 | ||
|
|
19676d69a7 |
@@ -139,7 +139,7 @@ jobs:
|
||||
path: hindsight-clients/typescript/*.tgz
|
||||
retention-days: 1
|
||||
|
||||
release-openclaw-integration:
|
||||
release-moltbot-integration:
|
||||
runs-on: ubuntu-latest
|
||||
environment: npm
|
||||
|
||||
@@ -153,15 +153,15 @@ jobs:
|
||||
registry-url: 'https://registry.npmjs.org'
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: ./hindsight-integrations/openclaw
|
||||
working-directory: ./hindsight-integrations/moltbot
|
||||
run: npm ci
|
||||
|
||||
- name: Build
|
||||
working-directory: ./hindsight-integrations/openclaw
|
||||
working-directory: ./hindsight-integrations/moltbot
|
||||
run: npm run build
|
||||
|
||||
- name: Publish to npm
|
||||
working-directory: ./hindsight-integrations/openclaw
|
||||
working-directory: ./hindsight-integrations/moltbot
|
||||
run: |
|
||||
set +e
|
||||
OUTPUT=$(npm publish --access public 2>&1)
|
||||
@@ -178,14 +178,14 @@ jobs:
|
||||
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
|
||||
|
||||
- name: Pack for GitHub release
|
||||
working-directory: ./hindsight-integrations/openclaw
|
||||
working-directory: ./hindsight-integrations/moltbot
|
||||
run: npm pack
|
||||
|
||||
- name: Upload artifacts
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: openclaw-integration
|
||||
path: hindsight-integrations/openclaw/*.tgz
|
||||
name: moltbot-integration
|
||||
path: hindsight-integrations/moltbot/*.tgz
|
||||
retention-days: 1
|
||||
|
||||
release-control-plane:
|
||||
@@ -415,7 +415,7 @@ jobs:
|
||||
|
||||
create-github-release:
|
||||
runs-on: ubuntu-latest
|
||||
needs: [release-python-packages, release-typescript-client, release-openclaw-integration, release-control-plane, release-rust-cli, release-docker-images, release-helm-chart]
|
||||
needs: [release-python-packages, release-typescript-client, release-moltbot-integration, release-control-plane, release-rust-cli, release-docker-images, release-helm-chart]
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
@@ -438,11 +438,11 @@ jobs:
|
||||
name: typescript-client
|
||||
path: ./artifacts/typescript-client
|
||||
|
||||
- name: Download OpenClaw Integration
|
||||
- name: Download Moltbot Integration
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: openclaw-integration
|
||||
path: ./artifacts/openclaw-integration
|
||||
name: moltbot-integration
|
||||
path: ./artifacts/moltbot-integration
|
||||
|
||||
- name: Download Control Plane
|
||||
uses: actions/download-artifact@v4
|
||||
@@ -485,8 +485,8 @@ jobs:
|
||||
cp artifacts/python-packages/hindsight-embed/dist/* release-assets/ || true
|
||||
# TypeScript client
|
||||
cp artifacts/typescript-client/*.tgz release-assets/ || true
|
||||
# OpenClaw Integration
|
||||
cp artifacts/openclaw-integration/*.tgz release-assets/ || true
|
||||
# Moltbot Integration
|
||||
cp artifacts/moltbot-integration/*.tgz release-assets/ || true
|
||||
# Control Plane
|
||||
cp artifacts/control-plane/*.tgz release-assets/ || true
|
||||
# Rust CLI binaries
|
||||
|
||||
@@ -82,7 +82,7 @@ jobs:
|
||||
- name: Build TypeScript client
|
||||
run: npm run build --workspace=hindsight-clients/typescript
|
||||
|
||||
build-openclaw-integration:
|
||||
build-moltbot-integration:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
@@ -94,15 +94,15 @@ jobs:
|
||||
node-version: '22'
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: ./hindsight-integrations/openclaw
|
||||
working-directory: ./hindsight-integrations/moltbot
|
||||
run: npm ci
|
||||
|
||||
- name: Run tests
|
||||
working-directory: ./hindsight-integrations/openclaw
|
||||
working-directory: ./hindsight-integrations/moltbot
|
||||
run: npm test
|
||||
|
||||
- name: Build
|
||||
working-directory: ./hindsight-integrations/openclaw
|
||||
working-directory: ./hindsight-integrations/moltbot
|
||||
run: npm run build
|
||||
|
||||
build-control-plane:
|
||||
@@ -958,18 +958,6 @@ jobs:
|
||||
working-directory: ./hindsight-dev
|
||||
run: uv run pytest upgrade_tests/ -v --tb=short
|
||||
|
||||
- name: Show upgrade test logs
|
||||
if: always()
|
||||
run: |
|
||||
echo "=== Upgrade Test Server Logs ==="
|
||||
for log in /tmp/upgrade-test-*.log; do
|
||||
if [ -f "$log" ]; then
|
||||
echo ""
|
||||
echo "--- $log ---"
|
||||
tail -500 "$log"
|
||||
fi
|
||||
done
|
||||
|
||||
verify-generated-files:
|
||||
runs-on: ubuntu-latest
|
||||
env:
|
||||
|
||||
@@ -93,34 +93,6 @@ uv run ty check hindsight_api # Type check
|
||||
3. Run tests to ensure nothing breaks
|
||||
4. Submit a PR with a clear description of changes
|
||||
|
||||
## Release Process
|
||||
|
||||
The project uses `scripts/release.sh` for creating releases. This script automates the entire release workflow:
|
||||
|
||||
1. Bumps version in all components (API, clients, CLI, control plane, Helm)
|
||||
2. **Regenerates OpenAPI spec and client SDKs** (Python, TypeScript, Rust)
|
||||
3. Updates documentation versioning
|
||||
4. Creates a commit and git tag
|
||||
5. Pushes to GitHub (triggers CI/CD to publish packages)
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
./scripts/release.sh <version>
|
||||
```
|
||||
|
||||
**Example:**
|
||||
```bash
|
||||
./scripts/release.sh 0.5.0
|
||||
```
|
||||
|
||||
### Important for Developers
|
||||
|
||||
- During development, version bumps in `__init__.py` do NOT require client regeneration
|
||||
- Clients are only regenerated during releases
|
||||
- Do not manually run `./scripts/generate-clients.sh` unless testing generation changes
|
||||
- Client version comments will reflect the API version from the latest release
|
||||
|
||||
## Reporting Issues
|
||||
|
||||
Open an issue on GitHub with:
|
||||
|
||||
@@ -169,34 +169,16 @@ ENV PATH="/app/api/.venv/bin:${PATH}"
|
||||
|
||||
# Pre-download ML models to avoid runtime download (conditional)
|
||||
# Only runs if both PRELOAD_ML_MODELS=true AND INCLUDE_LOCAL_MODELS=true
|
||||
# Includes retry logic with exponential backoff for transient network failures
|
||||
ARG PRELOAD_ML_MODELS
|
||||
ARG INCLUDE_LOCAL_MODELS
|
||||
ENV HF_HUB_DOWNLOAD_TIMEOUT=600
|
||||
RUN if [ "$PRELOAD_ML_MODELS" = "true" ] && [ "$INCLUDE_LOCAL_MODELS" = "true" ]; then \
|
||||
MAX_RETRIES=3; \
|
||||
RETRY_DELAY=10; \
|
||||
for i in $(seq 1 $MAX_RETRIES); do \
|
||||
echo "Attempt $i/$MAX_RETRIES: Downloading ML models..."; \
|
||||
/app/api/.venv/bin/python -c "\
|
||||
import os; os.environ['HF_HUB_DOWNLOAD_TIMEOUT'] = '600'; \
|
||||
/app/api/.venv/bin/python -c "\
|
||||
from sentence_transformers import SentenceTransformer, CrossEncoder; \
|
||||
print('Downloading embedding model...'); \
|
||||
SentenceTransformer('BAAI/bge-small-en-v1.5'); \
|
||||
print('Downloading cross-encoder model...'); \
|
||||
CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2'); \
|
||||
print('Downloading tiktoken encoding...'); import tiktoken; tiktoken.get_encoding('cl100k_base'); \
|
||||
print('Models cached successfully')" && break; \
|
||||
if [ $i -lt $MAX_RETRIES ]; then \
|
||||
echo "Attempt $i failed, retrying in ${RETRY_DELAY}s..."; \
|
||||
sleep $RETRY_DELAY; \
|
||||
RETRY_DELAY=$((RETRY_DELAY * 2)); \
|
||||
fi; \
|
||||
done; \
|
||||
if [ $i -eq $MAX_RETRIES ] && ! /app/api/.venv/bin/python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('BAAI/bge-small-en-v1.5')" 2>/dev/null; then \
|
||||
echo "ERROR: Failed to download models after $MAX_RETRIES attempts"; \
|
||||
exit 1; \
|
||||
fi; \
|
||||
print('Models cached successfully')"; \
|
||||
elif [ "$INCLUDE_LOCAL_MODELS" != "true" ]; then echo "Skipping ML model preload (local-models not included)"; \
|
||||
else echo "Skipping ML model preload"; fi
|
||||
|
||||
@@ -295,34 +277,16 @@ ENV PATH="/app/api/.venv/bin:${PATH}"
|
||||
|
||||
# Pre-download ML models to avoid runtime download (conditional)
|
||||
# Only runs if both PRELOAD_ML_MODELS=true AND INCLUDE_LOCAL_MODELS=true
|
||||
# Includes retry logic with exponential backoff for transient network failures
|
||||
ARG PRELOAD_ML_MODELS
|
||||
ARG INCLUDE_LOCAL_MODELS
|
||||
ENV HF_HUB_DOWNLOAD_TIMEOUT=600
|
||||
RUN if [ "$PRELOAD_ML_MODELS" = "true" ] && [ "$INCLUDE_LOCAL_MODELS" = "true" ]; then \
|
||||
MAX_RETRIES=3; \
|
||||
RETRY_DELAY=10; \
|
||||
for i in $(seq 1 $MAX_RETRIES); do \
|
||||
echo "Attempt $i/$MAX_RETRIES: Downloading ML models..."; \
|
||||
/app/api/.venv/bin/python -c "\
|
||||
import os; os.environ['HF_HUB_DOWNLOAD_TIMEOUT'] = '600'; \
|
||||
/app/api/.venv/bin/python -c "\
|
||||
from sentence_transformers import SentenceTransformer, CrossEncoder; \
|
||||
print('Downloading embedding model...'); \
|
||||
SentenceTransformer('BAAI/bge-small-en-v1.5'); \
|
||||
print('Downloading cross-encoder model...'); \
|
||||
CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2'); \
|
||||
print('Downloading tiktoken encoding...'); import tiktoken; tiktoken.get_encoding('cl100k_base'); \
|
||||
print('Models cached successfully')" && break; \
|
||||
if [ $i -lt $MAX_RETRIES ]; then \
|
||||
echo "Attempt $i failed, retrying in ${RETRY_DELAY}s..."; \
|
||||
sleep $RETRY_DELAY; \
|
||||
RETRY_DELAY=$((RETRY_DELAY * 2)); \
|
||||
fi; \
|
||||
done; \
|
||||
if [ $i -eq $MAX_RETRIES ] && ! /app/api/.venv/bin/python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('BAAI/bge-small-en-v1.5')" 2>/dev/null; then \
|
||||
echo "ERROR: Failed to download models after $MAX_RETRIES attempts"; \
|
||||
exit 1; \
|
||||
fi; \
|
||||
print('Models cached successfully')"; \
|
||||
elif [ "$INCLUDE_LOCAL_MODELS" != "true" ]; then echo "Skipping ML model preload (local-models not included)"; \
|
||||
else echo "Skipping ML model preload"; fi
|
||||
|
||||
|
||||
@@ -2,8 +2,8 @@ apiVersion: v2
|
||||
name: hindsight
|
||||
description: Hindsight helm chart
|
||||
type: application
|
||||
version: 0.4.7
|
||||
appVersion: "0.4.7"
|
||||
version: 0.4.2
|
||||
appVersion: "0.4.2"
|
||||
keywords:
|
||||
- ai
|
||||
- memory
|
||||
|
||||
@@ -46,4 +46,4 @@ __all__ = [
|
||||
"RemoteTEICrossEncoder",
|
||||
"LLMConfig",
|
||||
]
|
||||
__version__ = "0.4.7"
|
||||
__version__ = "0.4.2"
|
||||
|
||||
@@ -1398,18 +1398,13 @@ def create_app(
|
||||
|
||||
# Start worker poller if enabled (standalone mode)
|
||||
if config.worker_enabled and memory._pool is not None:
|
||||
from ..config import DEFAULT_DATABASE_SCHEMA
|
||||
|
||||
worker_id = config.worker_id or socket.gethostname()
|
||||
# Convert default schema to None for SQL compatibility (no schema prefix)
|
||||
schema = None if config.database_schema == DEFAULT_DATABASE_SCHEMA else config.database_schema
|
||||
poller = WorkerPoller(
|
||||
pool=memory._pool,
|
||||
worker_id=worker_id,
|
||||
executor=memory.execute_task,
|
||||
poll_interval_ms=config.worker_poll_interval_ms,
|
||||
max_retries=config.worker_max_retries,
|
||||
schema=schema,
|
||||
tenant_extension=getattr(memory, "_tenant_extension", None),
|
||||
max_slots=config.worker_max_slots,
|
||||
consolidation_max_slots=config.worker_consolidation_max_slots,
|
||||
@@ -2290,23 +2285,6 @@ def _register_routes(app: FastAPI):
|
||||
):
|
||||
"""Get a mental model by ID."""
|
||||
try:
|
||||
# Pre-operation validation hook
|
||||
validator = app.state.memory._operation_validator
|
||||
if validator:
|
||||
from hindsight_api.extensions.operation_validator import MentalModelGetContext
|
||||
|
||||
ctx = MentalModelGetContext(
|
||||
bank_id=bank_id,
|
||||
mental_model_id=mental_model_id,
|
||||
request_context=request_context,
|
||||
)
|
||||
validation = await validator.validate_mental_model_get(ctx)
|
||||
if not validation.allowed:
|
||||
raise OperationValidationError(
|
||||
validation.reason or "Operation not allowed",
|
||||
status_code=validation.status_code,
|
||||
)
|
||||
|
||||
mental_model = await app.state.memory.get_mental_model(
|
||||
bank_id=bank_id,
|
||||
mental_model_id=mental_model_id,
|
||||
@@ -2314,31 +2292,9 @@ def _register_routes(app: FastAPI):
|
||||
)
|
||||
if mental_model is None:
|
||||
raise HTTPException(status_code=404, detail=f"Mental model '{mental_model_id}' not found")
|
||||
|
||||
# Post-operation hook
|
||||
if validator:
|
||||
from hindsight_api.extensions.operation_validator import MentalModelGetResult
|
||||
|
||||
content = mental_model.get("content", "")
|
||||
output_tokens = len(content) // 4 if content else 0
|
||||
|
||||
result_ctx = MentalModelGetResult(
|
||||
bank_id=bank_id,
|
||||
mental_model_id=mental_model_id,
|
||||
request_context=request_context,
|
||||
output_tokens=output_tokens,
|
||||
success=True,
|
||||
)
|
||||
try:
|
||||
await validator.on_mental_model_get_complete(result_ctx)
|
||||
except Exception as hook_err:
|
||||
logger.warning(f"Post-mental-model-get hook error (non-fatal): {hook_err}")
|
||||
|
||||
return MentalModelResponse(**mental_model)
|
||||
except (AuthenticationError, HTTPException):
|
||||
raise
|
||||
except OperationValidationError as e:
|
||||
raise HTTPException(status_code=e.status_code, detail=e.reason)
|
||||
except Exception as e:
|
||||
import traceback
|
||||
|
||||
@@ -2363,23 +2319,6 @@ def _register_routes(app: FastAPI):
|
||||
):
|
||||
"""Create a mental model (async - returns operation_id)."""
|
||||
try:
|
||||
# Pre-operation validation hook
|
||||
validator = app.state.memory._operation_validator
|
||||
if validator:
|
||||
from hindsight_api.extensions.operation_validator import MentalModelRefreshContext
|
||||
|
||||
ctx = MentalModelRefreshContext(
|
||||
bank_id=bank_id,
|
||||
mental_model_id=None, # Not yet created
|
||||
request_context=request_context,
|
||||
)
|
||||
validation = await validator.validate_mental_model_refresh(ctx)
|
||||
if not validation.allowed:
|
||||
raise OperationValidationError(
|
||||
validation.reason or "Operation not allowed",
|
||||
status_code=validation.status_code,
|
||||
)
|
||||
|
||||
# 1. Create the mental model with placeholder content
|
||||
mental_model = await app.state.memory.create_mental_model(
|
||||
bank_id=bank_id,
|
||||
@@ -2402,8 +2341,6 @@ def _register_routes(app: FastAPI):
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
except (AuthenticationError, HTTPException):
|
||||
raise
|
||||
except OperationValidationError as e:
|
||||
raise HTTPException(status_code=e.status_code, detail=e.reason)
|
||||
except Exception as e:
|
||||
import traceback
|
||||
|
||||
@@ -2426,23 +2363,6 @@ def _register_routes(app: FastAPI):
|
||||
):
|
||||
"""Refresh a mental model by re-running its source query (async)."""
|
||||
try:
|
||||
# Pre-operation validation hook
|
||||
validator = app.state.memory._operation_validator
|
||||
if validator:
|
||||
from hindsight_api.extensions.operation_validator import MentalModelRefreshContext
|
||||
|
||||
ctx = MentalModelRefreshContext(
|
||||
bank_id=bank_id,
|
||||
mental_model_id=mental_model_id,
|
||||
request_context=request_context,
|
||||
)
|
||||
validation = await validator.validate_mental_model_refresh(ctx)
|
||||
if not validation.allowed:
|
||||
raise OperationValidationError(
|
||||
validation.reason or "Operation not allowed",
|
||||
status_code=validation.status_code,
|
||||
)
|
||||
|
||||
result = await app.state.memory.submit_async_refresh_mental_model(
|
||||
bank_id=bank_id,
|
||||
mental_model_id=mental_model_id,
|
||||
@@ -2453,8 +2373,6 @@ def _register_routes(app: FastAPI):
|
||||
raise HTTPException(status_code=404, detail=str(e))
|
||||
except (AuthenticationError, HTTPException):
|
||||
raise
|
||||
except OperationValidationError as e:
|
||||
raise HTTPException(status_code=e.status_code, detail=e.reason)
|
||||
except Exception as e:
|
||||
import traceback
|
||||
|
||||
|
||||
@@ -83,12 +83,9 @@ def print_startup_info(
|
||||
embeddings_provider: str,
|
||||
reranker_provider: str,
|
||||
mcp_enabled: bool = False,
|
||||
version: str | None = None,
|
||||
):
|
||||
"""Print styled startup information."""
|
||||
print(color_start("Starting Hindsight API..."))
|
||||
if version:
|
||||
print(f" {dim('Version:')} {color(f'v{version}', 0.1)}")
|
||||
print(f" {dim('URL:')} {color(f'http://{host}:{port}', 0.2)}")
|
||||
print(f" {dim('Database:')} {color(database_url, 0.4)}")
|
||||
print(f" {dim('LLM:')} {color(f'{llm_provider} / {llm_model}', 0.6)}")
|
||||
|
||||
@@ -154,21 +154,7 @@ ENV_REFLECT_MAX_ITERATIONS = "HINDSIGHT_API_REFLECT_MAX_ITERATIONS"
|
||||
DEFAULT_DATABASE_URL = "pg0"
|
||||
DEFAULT_DATABASE_SCHEMA = "public"
|
||||
DEFAULT_LLM_PROVIDER = "openai"
|
||||
|
||||
# Provider-specific default models
|
||||
PROVIDER_DEFAULT_MODELS = {
|
||||
"openai": "o3-mini",
|
||||
"anthropic": "claude-haiku-4-5-20251001",
|
||||
"gemini": "gemini-2.5-flash",
|
||||
"groq": "openai/gpt-oss-120b",
|
||||
"ollama": "gemma3:12b",
|
||||
"lmstudio": "local-model",
|
||||
"vertexai": "gemini-2.0-flash-001",
|
||||
"openai-codex": "gpt-5.2-codex",
|
||||
"claude-code": "claude-sonnet-4-5-20250929",
|
||||
"mock": "mock-model",
|
||||
}
|
||||
DEFAULT_LLM_MODEL = "o3-mini" # Fallback if provider not in table
|
||||
DEFAULT_LLM_MODEL = "gpt-5-mini"
|
||||
DEFAULT_LLM_MAX_CONCURRENT = 32
|
||||
DEFAULT_LLM_MAX_RETRIES = 10 # Max retry attempts for LLM API calls
|
||||
DEFAULT_LLM_INITIAL_BACKOFF = 1.0 # Initial backoff in seconds for retry exponential backoff
|
||||
@@ -317,11 +303,6 @@ def _validate_extraction_mode(mode: str) -> str:
|
||||
return mode_lower
|
||||
|
||||
|
||||
def _get_default_model_for_provider(provider: str) -> str:
|
||||
"""Get the default model for a given provider."""
|
||||
return PROVIDER_DEFAULT_MODELS.get(provider.lower(), DEFAULT_LLM_MODEL)
|
||||
|
||||
|
||||
@dataclass
|
||||
class HindsightConfig:
|
||||
"""Configuration container for Hindsight API."""
|
||||
@@ -450,18 +431,14 @@ class HindsightConfig:
|
||||
@classmethod
|
||||
def from_env(cls) -> "HindsightConfig":
|
||||
"""Create configuration from environment variables."""
|
||||
# Get provider first to determine default model
|
||||
llm_provider = os.getenv(ENV_LLM_PROVIDER, DEFAULT_LLM_PROVIDER)
|
||||
llm_model = os.getenv(ENV_LLM_MODEL) or _get_default_model_for_provider(llm_provider)
|
||||
|
||||
return cls(
|
||||
# Database
|
||||
database_url=os.getenv(ENV_DATABASE_URL, DEFAULT_DATABASE_URL),
|
||||
database_schema=os.getenv(ENV_DATABASE_SCHEMA, DEFAULT_DATABASE_SCHEMA),
|
||||
# LLM
|
||||
llm_provider=llm_provider,
|
||||
llm_provider=os.getenv(ENV_LLM_PROVIDER, DEFAULT_LLM_PROVIDER),
|
||||
llm_api_key=os.getenv(ENV_LLM_API_KEY),
|
||||
llm_model=llm_model,
|
||||
llm_model=os.getenv(ENV_LLM_MODEL, DEFAULT_LLM_MODEL),
|
||||
llm_base_url=os.getenv(ENV_LLM_BASE_URL) or None,
|
||||
llm_max_concurrent=int(os.getenv(ENV_LLM_MAX_CONCURRENT, str(DEFAULT_LLM_MAX_CONCURRENT))),
|
||||
llm_max_retries=int(os.getenv(ENV_LLM_MAX_RETRIES, str(DEFAULT_LLM_MAX_RETRIES))),
|
||||
@@ -476,12 +453,7 @@ class HindsightConfig:
|
||||
# Per-operation LLM config (None = use default)
|
||||
retain_llm_provider=os.getenv(ENV_RETAIN_LLM_PROVIDER) or None,
|
||||
retain_llm_api_key=os.getenv(ENV_RETAIN_LLM_API_KEY) or None,
|
||||
retain_llm_model=os.getenv(ENV_RETAIN_LLM_MODEL)
|
||||
or (
|
||||
_get_default_model_for_provider(os.getenv(ENV_RETAIN_LLM_PROVIDER))
|
||||
if os.getenv(ENV_RETAIN_LLM_PROVIDER)
|
||||
else None
|
||||
),
|
||||
retain_llm_model=os.getenv(ENV_RETAIN_LLM_MODEL) or None,
|
||||
retain_llm_base_url=os.getenv(ENV_RETAIN_LLM_BASE_URL) or None,
|
||||
retain_llm_max_concurrent=int(os.getenv(ENV_RETAIN_LLM_MAX_CONCURRENT))
|
||||
if os.getenv(ENV_RETAIN_LLM_MAX_CONCURRENT)
|
||||
@@ -498,12 +470,7 @@ class HindsightConfig:
|
||||
retain_llm_timeout=float(os.getenv(ENV_RETAIN_LLM_TIMEOUT)) if os.getenv(ENV_RETAIN_LLM_TIMEOUT) else None,
|
||||
reflect_llm_provider=os.getenv(ENV_REFLECT_LLM_PROVIDER) or None,
|
||||
reflect_llm_api_key=os.getenv(ENV_REFLECT_LLM_API_KEY) or None,
|
||||
reflect_llm_model=os.getenv(ENV_REFLECT_LLM_MODEL)
|
||||
or (
|
||||
_get_default_model_for_provider(os.getenv(ENV_REFLECT_LLM_PROVIDER))
|
||||
if os.getenv(ENV_REFLECT_LLM_PROVIDER)
|
||||
else None
|
||||
),
|
||||
reflect_llm_model=os.getenv(ENV_REFLECT_LLM_MODEL) or None,
|
||||
reflect_llm_base_url=os.getenv(ENV_REFLECT_LLM_BASE_URL) or None,
|
||||
reflect_llm_max_concurrent=int(os.getenv(ENV_REFLECT_LLM_MAX_CONCURRENT))
|
||||
if os.getenv(ENV_REFLECT_LLM_MAX_CONCURRENT)
|
||||
@@ -522,12 +489,7 @@ class HindsightConfig:
|
||||
else None,
|
||||
consolidation_llm_provider=os.getenv(ENV_CONSOLIDATION_LLM_PROVIDER) or None,
|
||||
consolidation_llm_api_key=os.getenv(ENV_CONSOLIDATION_LLM_API_KEY) or None,
|
||||
consolidation_llm_model=os.getenv(ENV_CONSOLIDATION_LLM_MODEL)
|
||||
or (
|
||||
_get_default_model_for_provider(os.getenv(ENV_CONSOLIDATION_LLM_PROVIDER))
|
||||
if os.getenv(ENV_CONSOLIDATION_LLM_PROVIDER)
|
||||
else None
|
||||
),
|
||||
consolidation_llm_model=os.getenv(ENV_CONSOLIDATION_LLM_MODEL) or None,
|
||||
consolidation_llm_base_url=os.getenv(ENV_CONSOLIDATION_LLM_BASE_URL) or None,
|
||||
consolidation_llm_max_concurrent=int(os.getenv(ENV_CONSOLIDATION_LLM_MAX_CONCURRENT))
|
||||
if os.getenv(ENV_CONSOLIDATION_LLM_MAX_CONCURRENT)
|
||||
|
||||
@@ -15,7 +15,7 @@ from pathlib import Path
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Default daemon configuration
|
||||
DEFAULT_DAEMON_PORT = 8888
|
||||
DEFAULT_DAEMON_PORT = 8889
|
||||
DEFAULT_IDLE_TIMEOUT = 0 # 0 = no auto-exit (hindsight-embed passes its own timeout)
|
||||
LOCKFILE_PATH = Path.home() / ".hindsight" / "daemon.lock"
|
||||
DAEMON_LOG_PATH = Path.home() / ".hindsight" / "daemon.log"
|
||||
|
||||
@@ -1,146 +0,0 @@
|
||||
"""
|
||||
Abstract interface for LLM providers.
|
||||
|
||||
This module defines the interface that all LLM providers must implement,
|
||||
enabling support for multiple LLM backends (OpenAI, Anthropic, Gemini, Codex, etc.)
|
||||
"""
|
||||
|
||||
from abc import ABC, abstractmethod
|
||||
from typing import Any
|
||||
|
||||
from .response_models import LLMToolCallResult, TokenUsage
|
||||
|
||||
|
||||
class LLMInterface(ABC):
|
||||
"""
|
||||
Abstract interface for LLM providers.
|
||||
|
||||
All LLM provider implementations must inherit from this class and implement
|
||||
the required methods.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
provider: str,
|
||||
api_key: str,
|
||||
base_url: str,
|
||||
model: str,
|
||||
reasoning_effort: str = "low",
|
||||
**kwargs: Any,
|
||||
):
|
||||
"""
|
||||
Initialize LLM provider.
|
||||
|
||||
Args:
|
||||
provider: Provider name (e.g., "openai", "codex", "anthropic", "gemini").
|
||||
api_key: API key or authentication token.
|
||||
base_url: Base URL for the API.
|
||||
model: Model name.
|
||||
reasoning_effort: Reasoning effort level for supported providers.
|
||||
**kwargs: Additional provider-specific parameters.
|
||||
"""
|
||||
self.provider = provider.lower()
|
||||
self.api_key = api_key
|
||||
self.base_url = base_url
|
||||
self.model = model
|
||||
self.reasoning_effort = reasoning_effort
|
||||
|
||||
@abstractmethod
|
||||
async def verify_connection(self) -> None:
|
||||
"""
|
||||
Verify that the LLM provider is configured correctly by making a simple test call.
|
||||
|
||||
Raises:
|
||||
RuntimeError: If the connection test fails.
|
||||
"""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
async def call(
|
||||
self,
|
||||
messages: list[dict[str, str]],
|
||||
response_format: Any | None = None,
|
||||
max_completion_tokens: int | None = None,
|
||||
temperature: float | None = None,
|
||||
scope: str = "memory",
|
||||
max_retries: int = 10,
|
||||
initial_backoff: float = 1.0,
|
||||
max_backoff: float = 60.0,
|
||||
skip_validation: bool = False,
|
||||
strict_schema: bool = False,
|
||||
return_usage: bool = False,
|
||||
) -> Any:
|
||||
"""
|
||||
Make an LLM API call with retry logic.
|
||||
|
||||
Args:
|
||||
messages: List of message dicts with 'role' and 'content'.
|
||||
response_format: Optional Pydantic model for structured output.
|
||||
max_completion_tokens: Maximum tokens in response.
|
||||
temperature: Sampling temperature (0.0-2.0).
|
||||
scope: Scope identifier for tracking.
|
||||
max_retries: Maximum retry attempts.
|
||||
initial_backoff: Initial backoff time in seconds.
|
||||
max_backoff: Maximum backoff time in seconds.
|
||||
skip_validation: Return raw JSON without Pydantic validation.
|
||||
strict_schema: Use strict JSON schema enforcement (OpenAI only).
|
||||
return_usage: If True, return tuple (result, TokenUsage) instead of just result.
|
||||
|
||||
Returns:
|
||||
If return_usage=False: Parsed response if response_format is provided, otherwise text content.
|
||||
If return_usage=True: Tuple of (result, TokenUsage) with token counts.
|
||||
|
||||
Raises:
|
||||
OutputTooLongError: If output exceeds token limits.
|
||||
Exception: Re-raises API errors after retries exhausted.
|
||||
"""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
async def call_with_tools(
|
||||
self,
|
||||
messages: list[dict[str, Any]],
|
||||
tools: list[dict[str, Any]],
|
||||
max_completion_tokens: int | None = None,
|
||||
temperature: float | None = None,
|
||||
scope: str = "tools",
|
||||
max_retries: int = 5,
|
||||
initial_backoff: float = 1.0,
|
||||
max_backoff: float = 30.0,
|
||||
tool_choice: str | dict[str, Any] = "auto",
|
||||
) -> LLMToolCallResult:
|
||||
"""
|
||||
Make an LLM API call with tool/function calling support.
|
||||
|
||||
Args:
|
||||
messages: List of message dicts. Can include tool results with role='tool'.
|
||||
tools: List of tool definitions in OpenAI format.
|
||||
max_completion_tokens: Maximum tokens in response.
|
||||
temperature: Sampling temperature (0.0-2.0).
|
||||
scope: Scope identifier for tracking.
|
||||
max_retries: Maximum retry attempts.
|
||||
initial_backoff: Initial backoff time in seconds.
|
||||
max_backoff: Maximum backoff time in seconds.
|
||||
tool_choice: How to choose tools - "auto", "none", "required", or specific function.
|
||||
|
||||
Returns:
|
||||
LLMToolCallResult with content and/or tool_calls.
|
||||
"""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
async def cleanup(self) -> None:
|
||||
"""Clean up resources (close connections, etc.)."""
|
||||
pass
|
||||
|
||||
|
||||
class OutputTooLongError(Exception):
|
||||
"""
|
||||
Bridge exception raised when LLM output exceeds token limits.
|
||||
|
||||
This wraps provider-specific errors (e.g., OpenAI's LengthFinishReasonError)
|
||||
to allow callers to handle output length issues without depending on
|
||||
provider-specific implementations.
|
||||
"""
|
||||
|
||||
pass
|
||||
File diff suppressed because it is too large
Load Diff
@@ -303,10 +303,8 @@ class MemoryEngine(MemoryEngineInterface):
|
||||
db_url = db_url or config.database_url
|
||||
memory_llm_provider = memory_llm_provider or config.llm_provider
|
||||
memory_llm_api_key = memory_llm_api_key or config.llm_api_key
|
||||
# Ollama, openai-codex, claude-code, and mock don't require an API key
|
||||
# openai-codex uses OAuth tokens from ~/.codex/auth.json
|
||||
# claude-code uses OAuth tokens from macOS Keychain
|
||||
if not memory_llm_api_key and memory_llm_provider not in ("ollama", "openai-codex", "claude-code", "mock"):
|
||||
# Ollama and mock don't require an API key
|
||||
if not memory_llm_api_key and memory_llm_provider not in ("ollama", "mock"):
|
||||
raise ValueError("LLM API key is required. Set HINDSIGHT_API_LLM_API_KEY environment variable.")
|
||||
memory_llm_model = memory_llm_model or config.llm_model
|
||||
memory_llm_base_url = memory_llm_base_url or config.get_llm_base_url() or None
|
||||
@@ -538,15 +536,10 @@ class MemoryEngine(MemoryEngineInterface):
|
||||
f"[BATCH_RETAIN_TASK] Starting background batch retain for bank_id={bank_id}, {len(contents)} items"
|
||||
)
|
||||
|
||||
# Restore tenant_id/api_key_id from task payload so downstream operations
|
||||
# (e.g., consolidation and mental model refreshes) can attribute usage.
|
||||
# Use internal request context for background tasks (skips tenant auth when schema is pre-set)
|
||||
from hindsight_api.models import RequestContext
|
||||
|
||||
internal_context = RequestContext(
|
||||
internal=True,
|
||||
tenant_id=task_dict.get("_tenant_id"),
|
||||
api_key_id=task_dict.get("_api_key_id"),
|
||||
)
|
||||
internal_context = RequestContext(internal=True)
|
||||
await self.retain_batch_async(bank_id=bank_id, contents=contents, request_context=internal_context)
|
||||
|
||||
logger.info(f"[BATCH_RETAIN_TASK] Completed background batch retain for bank_id={bank_id}")
|
||||
@@ -572,13 +565,7 @@ class MemoryEngine(MemoryEngineInterface):
|
||||
|
||||
from .consolidation import run_consolidation_job
|
||||
|
||||
# Restore tenant_id/api_key_id from task payload so downstream operations
|
||||
# (e.g., mental model refreshes) can attribute usage to the correct org.
|
||||
internal_context = RequestContext(
|
||||
internal=True,
|
||||
tenant_id=task_dict.get("_tenant_id"),
|
||||
api_key_id=task_dict.get("_api_key_id"),
|
||||
)
|
||||
internal_context = RequestContext(internal=True)
|
||||
result = await run_consolidation_job(
|
||||
memory_engine=self,
|
||||
bank_id=bank_id,
|
||||
@@ -610,13 +597,7 @@ class MemoryEngine(MemoryEngineInterface):
|
||||
|
||||
from hindsight_api.models import RequestContext
|
||||
|
||||
# Restore tenant_id/api_key_id from task payload so extensions can
|
||||
# attribute the mental_model_refresh operation to the correct org.
|
||||
internal_context = RequestContext(
|
||||
internal=True,
|
||||
tenant_id=task_dict.get("_tenant_id"),
|
||||
api_key_id=task_dict.get("_api_key_id"),
|
||||
)
|
||||
internal_context = RequestContext(internal=True)
|
||||
|
||||
# Get the current mental model to get source_query
|
||||
mental_model = await self.get_mental_model(bank_id, mental_model_id, request_context=internal_context)
|
||||
@@ -660,42 +641,6 @@ class MemoryEngine(MemoryEngineInterface):
|
||||
request_context=internal_context,
|
||||
)
|
||||
|
||||
# Call post-operation hook if validator is configured
|
||||
if self._operation_validator:
|
||||
from hindsight_api.extensions.operation_validator import MentalModelRefreshResult
|
||||
|
||||
# Count facts and mental models from based_on
|
||||
facts_used = 0
|
||||
mental_models_used = 0
|
||||
if reflect_result.based_on:
|
||||
for fact_type, facts in reflect_result.based_on.items():
|
||||
if facts:
|
||||
if fact_type == "mental_models":
|
||||
mental_models_used += len(facts)
|
||||
else:
|
||||
facts_used += len(facts)
|
||||
|
||||
# Estimate tokens
|
||||
query_tokens = len(source_query) // 4 if source_query else 0
|
||||
output_tokens = len(generated_content) // 4 if generated_content else 0
|
||||
context_tokens = 0 # refresh doesn't use additional context
|
||||
|
||||
result_ctx = MentalModelRefreshResult(
|
||||
bank_id=bank_id,
|
||||
mental_model_id=mental_model_id,
|
||||
request_context=internal_context,
|
||||
query_tokens=query_tokens,
|
||||
output_tokens=output_tokens,
|
||||
context_tokens=context_tokens,
|
||||
facts_used=facts_used,
|
||||
mental_models_used=mental_models_used,
|
||||
success=True,
|
||||
)
|
||||
try:
|
||||
await self._operation_validator.on_mental_model_refresh_complete(result_ctx)
|
||||
except Exception as hook_err:
|
||||
logger.warning(f"Post-mental-model-refresh hook error (non-fatal): {hook_err}")
|
||||
|
||||
logger.info(f"[REFRESH_MENTAL_MODEL_TASK] Completed for bank_id={bank_id}, mental_model_id={mental_model_id}")
|
||||
|
||||
async def execute_task(self, task_dict: dict[str, Any]):
|
||||
@@ -5471,13 +5416,6 @@ class MemoryEngine(MemoryEngineInterface):
|
||||
task_payload: dict[str, Any] = {"contents": contents}
|
||||
if document_tags:
|
||||
task_payload["document_tags"] = document_tags
|
||||
# Pass tenant_id and api_key_id through task payload so the worker
|
||||
# can propagate request context to downstream operations (e.g.,
|
||||
# consolidation and mental model refreshes triggered after retain).
|
||||
if request_context.tenant_id:
|
||||
task_payload["_tenant_id"] = request_context.tenant_id
|
||||
if request_context.api_key_id:
|
||||
task_payload["_api_key_id"] = request_context.api_key_id
|
||||
|
||||
result = await self._submit_async_operation(
|
||||
bank_id=bank_id,
|
||||
@@ -5510,21 +5448,11 @@ class MemoryEngine(MemoryEngineInterface):
|
||||
Dict with operation_id
|
||||
"""
|
||||
await self._authenticate_tenant(request_context)
|
||||
|
||||
# Pass tenant_id and api_key_id through task payload so the worker
|
||||
# can provide request context to extension hooks (e.g., usage metering
|
||||
# for mental model refreshes triggered by consolidation).
|
||||
task_payload: dict[str, Any] = {}
|
||||
if request_context.tenant_id:
|
||||
task_payload["_tenant_id"] = request_context.tenant_id
|
||||
if request_context.api_key_id:
|
||||
task_payload["_api_key_id"] = request_context.api_key_id
|
||||
|
||||
return await self._submit_async_operation(
|
||||
bank_id=bank_id,
|
||||
operation_type="consolidation",
|
||||
task_type="consolidation",
|
||||
task_payload=task_payload,
|
||||
task_payload={},
|
||||
dedupe_by_bank=True,
|
||||
)
|
||||
|
||||
@@ -5554,21 +5482,13 @@ class MemoryEngine(MemoryEngineInterface):
|
||||
if not mental_model:
|
||||
raise ValueError(f"Mental model {mental_model_id} not found in bank {bank_id}")
|
||||
|
||||
# Pass tenant_id and api_key_id through task payload so the worker
|
||||
# can provide request context to extension hooks.
|
||||
task_payload: dict[str, Any] = {
|
||||
"mental_model_id": mental_model_id,
|
||||
}
|
||||
if request_context.tenant_id:
|
||||
task_payload["_tenant_id"] = request_context.tenant_id
|
||||
if request_context.api_key_id:
|
||||
task_payload["_api_key_id"] = request_context.api_key_id
|
||||
|
||||
return await self._submit_async_operation(
|
||||
bank_id=bank_id,
|
||||
operation_type="refresh_mental_model",
|
||||
task_type="refresh_mental_model",
|
||||
task_payload=task_payload,
|
||||
task_payload={
|
||||
"mental_model_id": mental_model_id,
|
||||
},
|
||||
result_metadata={"mental_model_id": mental_model_id, "name": mental_model["name"]},
|
||||
dedupe_by_bank=False,
|
||||
)
|
||||
|
||||
@@ -1,14 +0,0 @@
|
||||
"""
|
||||
LLM provider implementations.
|
||||
|
||||
This package contains concrete implementations of the LLMInterface for various providers.
|
||||
"""
|
||||
|
||||
from .anthropic_llm import AnthropicLLM
|
||||
from .claude_code_llm import ClaudeCodeLLM
|
||||
from .codex_llm import CodexLLM
|
||||
from .gemini_llm import GeminiLLM
|
||||
from .mock_llm import MockLLM
|
||||
from .openai_compatible_llm import OpenAICompatibleLLM
|
||||
|
||||
__all__ = ["AnthropicLLM", "ClaudeCodeLLM", "CodexLLM", "GeminiLLM", "MockLLM", "OpenAICompatibleLLM"]
|
||||
@@ -1,434 +0,0 @@
|
||||
"""
|
||||
Anthropic LLM provider using the Anthropic Python SDK.
|
||||
|
||||
This provider enables using Claude models from Anthropic with support for:
|
||||
- Structured JSON output
|
||||
- Tool/function calling with proper format conversion
|
||||
- Extended thinking mode
|
||||
- Retry logic with exponential backoff
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
from hindsight_api.engine.llm_interface import LLMInterface, OutputTooLongError
|
||||
from hindsight_api.engine.response_models import LLMToolCall, LLMToolCallResult, TokenUsage
|
||||
from hindsight_api.metrics import get_metrics_collector
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class AnthropicLLM(LLMInterface):
|
||||
"""
|
||||
LLM provider using Anthropic's Claude models.
|
||||
|
||||
Supports structured output, tool calling, and extended thinking mode.
|
||||
Handles format conversion between OpenAI-style messages and Anthropic's format.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
provider: str,
|
||||
api_key: str,
|
||||
base_url: str,
|
||||
model: str,
|
||||
reasoning_effort: str = "low",
|
||||
timeout: float = 300.0,
|
||||
**kwargs: Any,
|
||||
):
|
||||
"""
|
||||
Initialize Anthropic LLM provider.
|
||||
|
||||
Args:
|
||||
provider: Provider name (should be "anthropic").
|
||||
api_key: Anthropic API key.
|
||||
base_url: Base URL for the API (optional, uses Anthropic default if empty).
|
||||
model: Model name (e.g., "claude-sonnet-4-20250514").
|
||||
reasoning_effort: Reasoning effort level (not used by Anthropic).
|
||||
timeout: Request timeout in seconds.
|
||||
**kwargs: Additional provider-specific parameters.
|
||||
"""
|
||||
super().__init__(provider, api_key, base_url, model, reasoning_effort, **kwargs)
|
||||
|
||||
if not self.api_key:
|
||||
raise ValueError("API key is required for Anthropic provider")
|
||||
|
||||
# Import and initialize Anthropic client
|
||||
try:
|
||||
from anthropic import AsyncAnthropic
|
||||
|
||||
client_kwargs: dict[str, Any] = {"api_key": self.api_key}
|
||||
if self.base_url:
|
||||
client_kwargs["base_url"] = self.base_url
|
||||
if timeout:
|
||||
client_kwargs["timeout"] = timeout
|
||||
|
||||
self._client = AsyncAnthropic(**client_kwargs)
|
||||
logger.info(f"Anthropic client initialized for model: {self.model}")
|
||||
except ImportError as e:
|
||||
raise RuntimeError("Anthropic SDK not installed. Run: uv add anthropic or pip install anthropic") from e
|
||||
|
||||
async def verify_connection(self) -> None:
|
||||
"""
|
||||
Verify that the Anthropic provider is configured correctly by making a simple test call.
|
||||
|
||||
Raises:
|
||||
RuntimeError: If the connection test fails.
|
||||
"""
|
||||
try:
|
||||
test_messages = [{"role": "user", "content": "test"}]
|
||||
await self.call(
|
||||
messages=test_messages,
|
||||
max_completion_tokens=10,
|
||||
temperature=0.0,
|
||||
scope="test",
|
||||
max_retries=0,
|
||||
)
|
||||
logger.info("Anthropic connection verified successfully")
|
||||
except Exception as e:
|
||||
logger.error(f"Anthropic connection verification failed: {e}")
|
||||
raise RuntimeError(f"Failed to verify Anthropic connection: {e}") from e
|
||||
|
||||
async def call(
|
||||
self,
|
||||
messages: list[dict[str, str]],
|
||||
response_format: Any | None = None,
|
||||
max_completion_tokens: int | None = None,
|
||||
temperature: float | None = None,
|
||||
scope: str = "memory",
|
||||
max_retries: int = 10,
|
||||
initial_backoff: float = 1.0,
|
||||
max_backoff: float = 60.0,
|
||||
skip_validation: bool = False,
|
||||
strict_schema: bool = False,
|
||||
return_usage: bool = False,
|
||||
) -> Any:
|
||||
"""
|
||||
Make an LLM API call with retry logic.
|
||||
|
||||
Args:
|
||||
messages: List of message dicts with 'role' and 'content'.
|
||||
response_format: Optional Pydantic model for structured output.
|
||||
max_completion_tokens: Maximum tokens in response.
|
||||
temperature: Sampling temperature (0.0-2.0).
|
||||
scope: Scope identifier for tracking.
|
||||
max_retries: Maximum retry attempts.
|
||||
initial_backoff: Initial backoff time in seconds.
|
||||
max_backoff: Maximum backoff time in seconds.
|
||||
skip_validation: Return raw JSON without Pydantic validation.
|
||||
strict_schema: Use strict JSON schema enforcement (not supported by Anthropic).
|
||||
return_usage: If True, return tuple (result, TokenUsage) instead of just result.
|
||||
|
||||
Returns:
|
||||
If return_usage=False: Parsed response if response_format is provided, otherwise text content.
|
||||
If return_usage=True: Tuple of (result, TokenUsage) with token counts.
|
||||
|
||||
Raises:
|
||||
OutputTooLongError: If output exceeds token limits.
|
||||
Exception: Re-raises API errors after retries exhausted.
|
||||
"""
|
||||
from anthropic import APIConnectionError, APIStatusError, RateLimitError
|
||||
|
||||
start_time = time.time()
|
||||
|
||||
# Convert OpenAI-style messages to Anthropic format
|
||||
system_prompt = None
|
||||
anthropic_messages = []
|
||||
|
||||
for msg in messages:
|
||||
role = msg.get("role", "user")
|
||||
content = msg.get("content", "")
|
||||
|
||||
if role == "system":
|
||||
if system_prompt:
|
||||
system_prompt += "\n\n" + content
|
||||
else:
|
||||
system_prompt = content
|
||||
else:
|
||||
anthropic_messages.append({"role": role, "content": content})
|
||||
|
||||
# Add JSON schema instruction if response_format is provided
|
||||
if response_format is not None and hasattr(response_format, "model_json_schema"):
|
||||
schema = response_format.model_json_schema()
|
||||
schema_msg = f"\n\nYou must respond with valid JSON matching this schema:\n{json.dumps(schema, indent=2)}"
|
||||
if system_prompt:
|
||||
system_prompt += schema_msg
|
||||
else:
|
||||
system_prompt = schema_msg
|
||||
|
||||
# Prepare parameters
|
||||
call_params: dict[str, Any] = {
|
||||
"model": self.model,
|
||||
"messages": anthropic_messages,
|
||||
"max_tokens": max_completion_tokens if max_completion_tokens is not None else 4096,
|
||||
}
|
||||
|
||||
if system_prompt:
|
||||
call_params["system"] = system_prompt
|
||||
|
||||
if temperature is not None:
|
||||
call_params["temperature"] = temperature
|
||||
|
||||
last_exception = None
|
||||
|
||||
for attempt in range(max_retries + 1):
|
||||
try:
|
||||
response = await self._client.messages.create(**call_params)
|
||||
|
||||
# Anthropic response content is a list of blocks
|
||||
content = ""
|
||||
for block in response.content:
|
||||
if block.type == "text":
|
||||
content += block.text
|
||||
|
||||
if response_format is not None:
|
||||
# Models may wrap JSON in markdown code blocks
|
||||
clean_content = content
|
||||
if "```json" in content:
|
||||
clean_content = content.split("```json")[1].split("```")[0].strip()
|
||||
elif "```" in content:
|
||||
clean_content = content.split("```")[1].split("```")[0].strip()
|
||||
|
||||
try:
|
||||
json_data = json.loads(clean_content)
|
||||
except json.JSONDecodeError:
|
||||
# Fallback to parsing raw content if markdown stripping failed
|
||||
json_data = json.loads(content)
|
||||
|
||||
if skip_validation:
|
||||
result = json_data
|
||||
else:
|
||||
result = response_format.model_validate(json_data)
|
||||
else:
|
||||
result = content
|
||||
|
||||
# Record metrics and log slow calls
|
||||
duration = time.time() - start_time
|
||||
input_tokens = response.usage.input_tokens or 0 if response.usage else 0
|
||||
output_tokens = response.usage.output_tokens or 0 if response.usage else 0
|
||||
total_tokens = input_tokens + output_tokens
|
||||
|
||||
# Record LLM metrics
|
||||
metrics = get_metrics_collector()
|
||||
metrics.record_llm_call(
|
||||
provider=self.provider,
|
||||
model=self.model,
|
||||
scope=scope,
|
||||
duration=duration,
|
||||
input_tokens=input_tokens,
|
||||
output_tokens=output_tokens,
|
||||
success=True,
|
||||
)
|
||||
|
||||
# Log slow calls
|
||||
if duration > 10.0:
|
||||
logger.info(
|
||||
f"slow llm call: scope={scope}, model={self.provider}/{self.model}, "
|
||||
f"input_tokens={input_tokens}, output_tokens={output_tokens}, "
|
||||
f"time={duration:.3f}s"
|
||||
)
|
||||
|
||||
if return_usage:
|
||||
token_usage = TokenUsage(
|
||||
input_tokens=input_tokens,
|
||||
output_tokens=output_tokens,
|
||||
total_tokens=total_tokens,
|
||||
)
|
||||
return result, token_usage
|
||||
return result
|
||||
|
||||
except json.JSONDecodeError as e:
|
||||
last_exception = e
|
||||
if attempt < max_retries:
|
||||
logger.warning("Anthropic returned invalid JSON, retrying...")
|
||||
backoff = min(initial_backoff * (2**attempt), max_backoff)
|
||||
await asyncio.sleep(backoff)
|
||||
continue
|
||||
else:
|
||||
logger.error(f"Anthropic returned invalid JSON after {max_retries + 1} attempts")
|
||||
raise
|
||||
|
||||
except (APIConnectionError, RateLimitError, APIStatusError) as e:
|
||||
# Fast fail on 401/403
|
||||
if isinstance(e, APIStatusError) and e.status_code in (401, 403):
|
||||
logger.error(f"Anthropic auth error (HTTP {e.status_code}), not retrying: {str(e)}")
|
||||
raise
|
||||
|
||||
last_exception = e
|
||||
if attempt < max_retries:
|
||||
# Check if it's a rate limit or server error
|
||||
should_retry = isinstance(e, (APIConnectionError, RateLimitError)) or (
|
||||
isinstance(e, APIStatusError) and e.status_code >= 500
|
||||
)
|
||||
|
||||
if should_retry:
|
||||
backoff = min(initial_backoff * (2**attempt), max_backoff)
|
||||
jitter = backoff * 0.2 * (2 * (time.time() % 1) - 1)
|
||||
await asyncio.sleep(backoff + jitter)
|
||||
continue
|
||||
|
||||
logger.error(f"Anthropic API error after {max_retries + 1} attempts: {str(e)}")
|
||||
raise
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Unexpected error during Anthropic call: {type(e).__name__}: {str(e)}")
|
||||
raise
|
||||
|
||||
if last_exception:
|
||||
raise last_exception
|
||||
raise RuntimeError("Anthropic call failed after all retries")
|
||||
|
||||
async def call_with_tools(
|
||||
self,
|
||||
messages: list[dict[str, Any]],
|
||||
tools: list[dict[str, Any]],
|
||||
max_completion_tokens: int | None = None,
|
||||
temperature: float | None = None,
|
||||
scope: str = "tools",
|
||||
max_retries: int = 5,
|
||||
initial_backoff: float = 1.0,
|
||||
max_backoff: float = 30.0,
|
||||
tool_choice: str | dict[str, Any] = "auto",
|
||||
) -> LLMToolCallResult:
|
||||
"""
|
||||
Make an LLM API call with tool/function calling support.
|
||||
|
||||
Args:
|
||||
messages: List of message dicts. Can include tool results with role='tool'.
|
||||
tools: List of tool definitions in OpenAI format.
|
||||
max_completion_tokens: Maximum tokens in response.
|
||||
temperature: Sampling temperature (0.0-2.0).
|
||||
scope: Scope identifier for tracking.
|
||||
max_retries: Maximum retry attempts.
|
||||
initial_backoff: Initial backoff time in seconds.
|
||||
max_backoff: Maximum backoff time in seconds.
|
||||
tool_choice: How to choose tools - "auto", "none", "required", or specific function.
|
||||
|
||||
Returns:
|
||||
LLMToolCallResult with content and/or tool_calls.
|
||||
"""
|
||||
from anthropic import APIConnectionError, APIStatusError
|
||||
|
||||
start_time = time.time()
|
||||
|
||||
# Convert OpenAI tool format to Anthropic format
|
||||
anthropic_tools = []
|
||||
for tool in tools:
|
||||
func = tool.get("function", {})
|
||||
anthropic_tools.append(
|
||||
{
|
||||
"name": func.get("name", ""),
|
||||
"description": func.get("description", ""),
|
||||
"input_schema": func.get("parameters", {"type": "object", "properties": {}}),
|
||||
}
|
||||
)
|
||||
|
||||
# Convert messages - handle tool results
|
||||
system_prompt = None
|
||||
anthropic_messages = []
|
||||
for msg in messages:
|
||||
role = msg.get("role", "user")
|
||||
content = msg.get("content", "")
|
||||
|
||||
if role == "system":
|
||||
system_prompt = (system_prompt + "\n\n" + content) if system_prompt else content
|
||||
elif role == "tool":
|
||||
# Anthropic uses tool_result blocks
|
||||
anthropic_messages.append(
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "tool_result", "tool_use_id": msg.get("tool_call_id", ""), "content": content}
|
||||
],
|
||||
}
|
||||
)
|
||||
elif role == "assistant" and msg.get("tool_calls"):
|
||||
# Convert assistant tool calls
|
||||
tool_use_blocks = []
|
||||
for tc in msg["tool_calls"]:
|
||||
tool_use_blocks.append(
|
||||
{
|
||||
"type": "tool_use",
|
||||
"id": tc.get("id", ""),
|
||||
"name": tc.get("function", {}).get("name", ""),
|
||||
"input": json.loads(tc.get("function", {}).get("arguments", "{}")),
|
||||
}
|
||||
)
|
||||
anthropic_messages.append({"role": "assistant", "content": tool_use_blocks})
|
||||
else:
|
||||
anthropic_messages.append({"role": role, "content": content})
|
||||
|
||||
call_params: dict[str, Any] = {
|
||||
"model": self.model,
|
||||
"messages": anthropic_messages,
|
||||
"tools": anthropic_tools,
|
||||
"max_tokens": max_completion_tokens or 4096,
|
||||
}
|
||||
if system_prompt:
|
||||
call_params["system"] = system_prompt
|
||||
|
||||
if temperature is not None:
|
||||
call_params["temperature"] = temperature
|
||||
|
||||
last_exception = None
|
||||
for attempt in range(max_retries + 1):
|
||||
try:
|
||||
response = await self._client.messages.create(**call_params)
|
||||
|
||||
# Extract content and tool calls
|
||||
content_parts = []
|
||||
tool_calls: list[LLMToolCall] = []
|
||||
|
||||
for block in response.content:
|
||||
if block.type == "text":
|
||||
content_parts.append(block.text)
|
||||
elif block.type == "tool_use":
|
||||
tool_calls.append(LLMToolCall(id=block.id, name=block.name, arguments=block.input or {}))
|
||||
|
||||
content = "".join(content_parts) if content_parts else None
|
||||
finish_reason = "tool_calls" if tool_calls else "stop"
|
||||
|
||||
# Extract token usage
|
||||
input_tokens = response.usage.input_tokens or 0
|
||||
output_tokens = response.usage.output_tokens or 0
|
||||
|
||||
# Record metrics
|
||||
metrics = get_metrics_collector()
|
||||
metrics.record_llm_call(
|
||||
provider=self.provider,
|
||||
model=self.model,
|
||||
scope=scope,
|
||||
duration=time.time() - start_time,
|
||||
input_tokens=input_tokens,
|
||||
output_tokens=output_tokens,
|
||||
success=True,
|
||||
)
|
||||
|
||||
return LLMToolCallResult(
|
||||
content=content,
|
||||
tool_calls=tool_calls,
|
||||
finish_reason=finish_reason,
|
||||
input_tokens=input_tokens,
|
||||
output_tokens=output_tokens,
|
||||
)
|
||||
|
||||
except (APIConnectionError, APIStatusError) as e:
|
||||
if isinstance(e, APIStatusError) and e.status_code in (401, 403):
|
||||
raise
|
||||
last_exception = e
|
||||
if attempt < max_retries:
|
||||
await asyncio.sleep(min(initial_backoff * (2**attempt), max_backoff))
|
||||
continue
|
||||
raise
|
||||
|
||||
if last_exception:
|
||||
raise last_exception
|
||||
raise RuntimeError("Anthropic tool call failed")
|
||||
|
||||
async def cleanup(self) -> None:
|
||||
"""Clean up resources (close Anthropic client connections)."""
|
||||
if hasattr(self, "_client") and self._client:
|
||||
await self._client.close()
|
||||
@@ -1,352 +0,0 @@
|
||||
"""
|
||||
Claude Code LLM provider using Claude Agent SDK.
|
||||
|
||||
This provider enables using Claude Pro/Max subscriptions for API calls
|
||||
via the Claude CLI authentication. It uses the Claude Agent SDK which
|
||||
automatically handles authentication via `claude auth login` credentials.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
from hindsight_api.engine.llm_interface import LLMInterface, OutputTooLongError
|
||||
from hindsight_api.engine.response_models import LLMToolCall, LLMToolCallResult, TokenUsage
|
||||
from hindsight_api.metrics import get_metrics_collector
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ClaudeCodeLLM(LLMInterface):
|
||||
"""
|
||||
LLM provider using Claude Code authentication.
|
||||
|
||||
Authenticates using Claude Pro/Max credentials via `claude auth login`
|
||||
and makes API calls through the Claude Agent SDK.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
provider: str,
|
||||
api_key: str, # Will be ignored, uses CLI auth
|
||||
base_url: str,
|
||||
model: str,
|
||||
reasoning_effort: str = "low",
|
||||
**kwargs: Any,
|
||||
):
|
||||
"""Initialize Claude Code LLM provider."""
|
||||
super().__init__(provider, api_key, base_url, model, reasoning_effort, **kwargs)
|
||||
|
||||
# Verify Claude Agent SDK is available
|
||||
try:
|
||||
self._verify_claude_code_available()
|
||||
logger.info("Claude Code: Using Claude Agent SDK (authentication via claude auth login)")
|
||||
except Exception as e:
|
||||
raise RuntimeError(
|
||||
f"Failed to initialize Claude Code provider: {e}\n\n"
|
||||
"To set up Claude Code authentication:\n"
|
||||
"1. Install Claude Code CLI: npm install -g @anthropics/claude-code\n"
|
||||
"2. Login with your Pro/Max plan: claude auth login\n"
|
||||
"3. Verify authentication: claude --version\n\n"
|
||||
"Or use a different provider (anthropic, openai, gemini) with API keys."
|
||||
) from e
|
||||
|
||||
# Metrics collector is imported at module level
|
||||
|
||||
def _verify_claude_code_available(self) -> None:
|
||||
"""
|
||||
Verify that Claude Agent SDK can be imported and is properly configured.
|
||||
|
||||
Raises:
|
||||
ImportError: If Claude Agent SDK is not installed.
|
||||
RuntimeError: If Claude Code is not authenticated.
|
||||
"""
|
||||
try:
|
||||
# Import Claude Agent SDK
|
||||
# Reduce Claude Agent SDK logging verbosity
|
||||
import logging as sdk_logging
|
||||
|
||||
from claude_agent_sdk import query # noqa: F401
|
||||
|
||||
sdk_logging.getLogger("claude_agent_sdk").setLevel(sdk_logging.WARNING)
|
||||
sdk_logging.getLogger("claude_agent_sdk._internal").setLevel(sdk_logging.WARNING)
|
||||
|
||||
logger.debug("Claude Agent SDK imported successfully")
|
||||
except ImportError as e:
|
||||
raise ImportError(
|
||||
"Claude Agent SDK not installed. Run: uv add claude-agent-sdk or pip install claude-agent-sdk"
|
||||
) from e
|
||||
|
||||
# SDK will automatically check for authentication when first used
|
||||
# No need to verify here - let it fail gracefully on first call with helpful error
|
||||
|
||||
async def verify_connection(self) -> None:
|
||||
"""
|
||||
Verify that the Claude Code provider is configured correctly by making a simple test call.
|
||||
|
||||
Raises:
|
||||
RuntimeError: If the connection test fails.
|
||||
"""
|
||||
try:
|
||||
test_messages = [{"role": "user", "content": "test"}]
|
||||
await self.call(
|
||||
messages=test_messages,
|
||||
max_completion_tokens=10,
|
||||
temperature=0.0,
|
||||
scope="test",
|
||||
max_retries=0,
|
||||
)
|
||||
logger.info("Claude Code connection verified successfully")
|
||||
except Exception as e:
|
||||
logger.error(f"Claude Code connection verification failed: {e}")
|
||||
raise RuntimeError(f"Failed to verify Claude Code connection: {e}") from e
|
||||
|
||||
async def call(
|
||||
self,
|
||||
messages: list[dict[str, str]],
|
||||
response_format: Any | None = None,
|
||||
max_completion_tokens: int | None = None,
|
||||
temperature: float | None = None,
|
||||
scope: str = "memory",
|
||||
max_retries: int = 10,
|
||||
initial_backoff: float = 1.0,
|
||||
max_backoff: float = 60.0,
|
||||
skip_validation: bool = False,
|
||||
strict_schema: bool = False,
|
||||
return_usage: bool = False,
|
||||
) -> Any:
|
||||
"""
|
||||
Make an LLM API call with retry logic.
|
||||
|
||||
Args:
|
||||
messages: List of message dicts with 'role' and 'content'.
|
||||
response_format: Optional Pydantic model for structured output.
|
||||
max_completion_tokens: Maximum tokens in response (ignored by Claude Agent SDK).
|
||||
temperature: Sampling temperature (ignored by Claude Agent SDK).
|
||||
scope: Scope identifier for tracking.
|
||||
max_retries: Maximum retry attempts.
|
||||
initial_backoff: Initial backoff time in seconds.
|
||||
max_backoff: Maximum backoff time in seconds.
|
||||
skip_validation: Return raw JSON without Pydantic validation.
|
||||
strict_schema: Use strict JSON schema enforcement (not supported).
|
||||
return_usage: If True, return tuple (result, TokenUsage) instead of just result.
|
||||
|
||||
Returns:
|
||||
If return_usage=False: Parsed response if response_format is provided, otherwise text content.
|
||||
If return_usage=True: Tuple of (result, TokenUsage) with estimated token counts.
|
||||
|
||||
Raises:
|
||||
OutputTooLongError: If output exceeds token limits (not supported by Claude Agent SDK).
|
||||
Exception: Re-raises API errors after retries exhausted.
|
||||
"""
|
||||
from claude_agent_sdk import AssistantMessage, ClaudeAgentOptions, TextBlock, query
|
||||
|
||||
start_time = time.time()
|
||||
|
||||
# Build system prompt
|
||||
system_prompt = ""
|
||||
user_content = ""
|
||||
|
||||
for msg in messages:
|
||||
role = msg.get("role", "user")
|
||||
content = msg.get("content", "")
|
||||
|
||||
if role == "system":
|
||||
system_prompt += ("\n\n" + content) if system_prompt else content
|
||||
elif role == "user":
|
||||
user_content += ("\n\n" + content) if user_content else content
|
||||
elif role == "assistant":
|
||||
# Claude Agent SDK doesn't support multi-turn easily in query()
|
||||
# For now, prepend assistant messages to user content
|
||||
user_content += f"\n\n[Previous assistant response: {content}]"
|
||||
|
||||
# Add JSON schema instruction if response_format is provided
|
||||
if response_format is not None and hasattr(response_format, "model_json_schema"):
|
||||
schema = response_format.model_json_schema()
|
||||
schema_instruction = (
|
||||
f"\n\nYou must respond with valid JSON matching this schema:\n{json.dumps(schema, indent=2)}\n\n"
|
||||
"Respond with ONLY the JSON, no markdown formatting."
|
||||
)
|
||||
user_content += schema_instruction
|
||||
|
||||
# Configure SDK options
|
||||
options = ClaudeAgentOptions(
|
||||
system_prompt=system_prompt if system_prompt else None,
|
||||
max_turns=1, # Single-turn for API-style interactions
|
||||
allowed_tools=[], # Disable tools for standard LLM calls
|
||||
)
|
||||
|
||||
# Call Claude Agent SDK
|
||||
last_exception = None
|
||||
for attempt in range(max_retries + 1):
|
||||
try:
|
||||
# Collect streaming response
|
||||
full_text = ""
|
||||
|
||||
async for message in query(prompt=user_content, options=options):
|
||||
if isinstance(message, AssistantMessage):
|
||||
for block in message.content:
|
||||
if isinstance(block, TextBlock):
|
||||
full_text += block.text
|
||||
|
||||
# Handle structured output
|
||||
if response_format is not None:
|
||||
# Models may wrap JSON in markdown
|
||||
clean_text = full_text
|
||||
if "```json" in full_text:
|
||||
clean_text = full_text.split("```json")[1].split("```")[0].strip()
|
||||
elif "```" in full_text:
|
||||
clean_text = full_text.split("```")[1].split("```")[0].strip()
|
||||
|
||||
try:
|
||||
json_data = json.loads(clean_text)
|
||||
except json.JSONDecodeError as e:
|
||||
logger.warning(f"Claude Code JSON parse error (attempt {attempt + 1}/{max_retries + 1}): {e}")
|
||||
if attempt < max_retries:
|
||||
backoff = min(initial_backoff * (2**attempt), max_backoff)
|
||||
await asyncio.sleep(backoff)
|
||||
last_exception = e
|
||||
continue
|
||||
raise
|
||||
|
||||
if skip_validation:
|
||||
result = json_data
|
||||
else:
|
||||
result = response_format.model_validate(json_data)
|
||||
else:
|
||||
result = full_text
|
||||
|
||||
# Record metrics
|
||||
duration = time.time() - start_time
|
||||
metrics = get_metrics_collector()
|
||||
|
||||
# Estimate token usage (Claude Agent SDK doesn't report exact counts)
|
||||
# Use character count / 4 as rough estimate (1 token ≈ 4 characters)
|
||||
estimated_input = sum(len(m.get("content", "")) for m in messages) // 4
|
||||
estimated_output = len(full_text) // 4
|
||||
|
||||
metrics.record_llm_call(
|
||||
provider=self.provider,
|
||||
model=self.model,
|
||||
scope=scope,
|
||||
duration=duration,
|
||||
input_tokens=estimated_input,
|
||||
output_tokens=estimated_output,
|
||||
success=True,
|
||||
)
|
||||
|
||||
# Log slow calls
|
||||
if duration > 10.0:
|
||||
logger.info(
|
||||
f"slow llm call: scope={scope}, model={self.provider}/{self.model}, time={duration:.3f}s"
|
||||
)
|
||||
|
||||
if return_usage:
|
||||
token_usage = TokenUsage(
|
||||
input_tokens=estimated_input,
|
||||
output_tokens=estimated_output,
|
||||
total_tokens=estimated_input + estimated_output,
|
||||
)
|
||||
return result, token_usage
|
||||
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
last_exception = e
|
||||
|
||||
# Check for authentication errors
|
||||
error_str = str(e).lower()
|
||||
if "auth" in error_str or "login" in error_str or "credential" in error_str:
|
||||
logger.error(f"Claude Code authentication error: {e}")
|
||||
raise RuntimeError(
|
||||
f"Claude Code authentication failed: {e}\n\n"
|
||||
"Run 'claude auth login' to authenticate with Claude Pro/Max."
|
||||
) from e
|
||||
|
||||
if attempt < max_retries:
|
||||
backoff = min(initial_backoff * (2**attempt), max_backoff)
|
||||
logger.warning(f"Claude Code error (attempt {attempt + 1}/{max_retries + 1}): {e}")
|
||||
await asyncio.sleep(backoff)
|
||||
continue
|
||||
else:
|
||||
logger.error(f"Claude Code error after {max_retries + 1} attempts: {e}")
|
||||
raise
|
||||
|
||||
if last_exception:
|
||||
raise last_exception
|
||||
raise RuntimeError("Claude Code call failed after all retries")
|
||||
|
||||
async def call_with_tools(
|
||||
self,
|
||||
messages: list[dict[str, Any]],
|
||||
tools: list[dict[str, Any]],
|
||||
max_completion_tokens: int | None = None,
|
||||
temperature: float | None = None,
|
||||
scope: str = "tools",
|
||||
max_retries: int = 5,
|
||||
initial_backoff: float = 1.0,
|
||||
max_backoff: float = 30.0,
|
||||
tool_choice: str | dict[str, Any] = "auto",
|
||||
) -> LLMToolCallResult:
|
||||
"""
|
||||
Make an LLM API call with tool/function calling support.
|
||||
|
||||
Note: This is a simplified implementation. Full tool support would require
|
||||
integrating with Claude Agent SDK's tool system.
|
||||
|
||||
Args:
|
||||
messages: List of message dicts. Can include tool results with role='tool'.
|
||||
tools: List of tool definitions in OpenAI format.
|
||||
max_completion_tokens: Maximum tokens in response.
|
||||
temperature: Sampling temperature.
|
||||
scope: Scope identifier for tracking.
|
||||
max_retries: Maximum retry attempts.
|
||||
initial_backoff: Initial backoff time in seconds.
|
||||
max_backoff: Maximum backoff time in seconds.
|
||||
tool_choice: How to choose tools - "auto", "none", "required", or specific function.
|
||||
|
||||
Returns:
|
||||
LLMToolCallResult with content and/or tool_calls.
|
||||
"""
|
||||
# For now, use regular call without tools
|
||||
# Full implementation would require mapping OpenAI tool format to Claude Agent SDK tools
|
||||
logger.warning(
|
||||
"Claude Code provider does not fully support tool calling yet. Falling back to regular text completion."
|
||||
)
|
||||
|
||||
result = await self.call(
|
||||
messages=messages,
|
||||
response_format=None,
|
||||
max_completion_tokens=max_completion_tokens,
|
||||
temperature=temperature,
|
||||
scope=scope,
|
||||
max_retries=max_retries,
|
||||
initial_backoff=initial_backoff,
|
||||
max_backoff=max_backoff,
|
||||
return_usage=True,
|
||||
)
|
||||
|
||||
if isinstance(result, tuple):
|
||||
text, usage = result
|
||||
return LLMToolCallResult(
|
||||
content=text,
|
||||
tool_calls=[],
|
||||
finish_reason="stop",
|
||||
input_tokens=usage.input_tokens,
|
||||
output_tokens=usage.output_tokens,
|
||||
)
|
||||
else:
|
||||
# Fallback if return_usage didn't work as expected
|
||||
return LLMToolCallResult(
|
||||
content=str(result),
|
||||
tool_calls=[],
|
||||
finish_reason="stop",
|
||||
input_tokens=0,
|
||||
output_tokens=0,
|
||||
)
|
||||
|
||||
async def cleanup(self) -> None:
|
||||
"""Clean up resources (no HTTP client to close for Claude Agent SDK)."""
|
||||
pass
|
||||
@@ -1,527 +0,0 @@
|
||||
"""
|
||||
OpenAI Codex LLM provider using ChatGPT Plus/Pro OAuth authentication.
|
||||
|
||||
This provider enables using ChatGPT Plus/Pro subscriptions for API calls
|
||||
without separate OpenAI Platform API credits. It uses OAuth tokens from
|
||||
~/.codex/auth.json and communicates with the ChatGPT backend API.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import time
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
|
||||
from hindsight_api.engine.llm_interface import LLMInterface, OutputTooLongError
|
||||
from hindsight_api.engine.response_models import LLMToolCall, LLMToolCallResult, TokenUsage
|
||||
from hindsight_api.metrics import get_metrics_collector
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class CodexLLM(LLMInterface):
|
||||
"""
|
||||
LLM provider using OpenAI Codex OAuth authentication.
|
||||
|
||||
Authenticates using ChatGPT Plus/Pro credentials stored in ~/.codex/auth.json
|
||||
and makes API calls to chatgpt.com/backend-api/codex/responses.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
provider: str,
|
||||
api_key: str, # Will be ignored, reads from ~/.codex/auth.json
|
||||
base_url: str,
|
||||
model: str,
|
||||
reasoning_effort: str = "low",
|
||||
**kwargs: Any,
|
||||
):
|
||||
"""Initialize Codex LLM provider."""
|
||||
super().__init__(provider, api_key, base_url, model, reasoning_effort, **kwargs)
|
||||
|
||||
# Load Codex OAuth credentials
|
||||
try:
|
||||
self.access_token, self.account_id = self._load_codex_auth()
|
||||
logger.info(f"Loaded Codex OAuth credentials for account: {self.account_id}")
|
||||
except Exception as e:
|
||||
raise RuntimeError(
|
||||
f"Failed to load Codex OAuth credentials from ~/.codex/auth.json: {e}\n\n"
|
||||
"To set up Codex authentication:\n"
|
||||
"1. Install Codex CLI: npm install -g @openai/codex\n"
|
||||
"2. Login: codex auth login\n"
|
||||
"3. Verify: ls ~/.codex/auth.json\n\n"
|
||||
"Or use a different provider (openai, anthropic, gemini) with API keys."
|
||||
) from e
|
||||
|
||||
# Use ChatGPT backend API endpoint
|
||||
if not self.base_url:
|
||||
self.base_url = "https://chatgpt.com/backend-api"
|
||||
|
||||
# Normalize model name (strip openai/ prefix if present)
|
||||
if self.model.startswith("openai/"):
|
||||
self.model = self.model[len("openai/") :]
|
||||
|
||||
# Map reasoning effort to Codex reasoning summary format
|
||||
# Codex supports: "auto", "concise", "detailed"
|
||||
self.reasoning_summary = self._map_reasoning_effort(reasoning_effort)
|
||||
|
||||
# HTTP client for SSE streaming
|
||||
self._client = httpx.AsyncClient(timeout=120.0)
|
||||
|
||||
def _load_codex_auth(self) -> tuple[str, str]:
|
||||
"""
|
||||
Load OAuth credentials from ~/.codex/auth.json.
|
||||
|
||||
Returns:
|
||||
Tuple of (access_token, account_id).
|
||||
|
||||
Raises:
|
||||
FileNotFoundError: If auth file doesn't exist.
|
||||
ValueError: If auth file is invalid.
|
||||
"""
|
||||
auth_file = Path.home() / ".codex" / "auth.json"
|
||||
|
||||
if not auth_file.exists():
|
||||
raise FileNotFoundError(
|
||||
f"Codex auth file not found: {auth_file}\nRun 'codex auth login' to authenticate with ChatGPT Plus/Pro."
|
||||
)
|
||||
|
||||
with open(auth_file) as f:
|
||||
data = json.load(f)
|
||||
|
||||
# Validate auth structure
|
||||
auth_mode = data.get("auth_mode")
|
||||
if auth_mode != "chatgpt":
|
||||
raise ValueError(f"Expected auth_mode='chatgpt', got: {auth_mode}")
|
||||
|
||||
tokens = data.get("tokens", {})
|
||||
access_token = tokens.get("access_token")
|
||||
account_id = tokens.get("account_id")
|
||||
|
||||
if not access_token:
|
||||
raise ValueError("No access_token found in Codex auth file. Run 'codex auth login' again.")
|
||||
|
||||
return access_token, account_id
|
||||
|
||||
def _map_reasoning_effort(self, effort: str) -> str:
|
||||
"""
|
||||
Map standard reasoning effort to Codex reasoning summary format.
|
||||
|
||||
Args:
|
||||
effort: Standard effort level ("low", "medium", "high", "xhigh").
|
||||
|
||||
Returns:
|
||||
Codex reasoning summary: "concise", "detailed", or "auto".
|
||||
"""
|
||||
mapping = {
|
||||
"low": "concise",
|
||||
"medium": "auto",
|
||||
"high": "detailed",
|
||||
"xhigh": "detailed",
|
||||
}
|
||||
return mapping.get(effort.lower(), "auto")
|
||||
|
||||
async def verify_connection(self) -> None:
|
||||
"""Verify Codex connection by making a simple test call."""
|
||||
try:
|
||||
logger.info(f"Verifying Codex LLM: model={self.model}, account={self.account_id}...")
|
||||
await self.call(
|
||||
messages=[{"role": "user", "content": "Say 'ok'"}],
|
||||
max_completion_tokens=10,
|
||||
max_retries=2,
|
||||
initial_backoff=0.5,
|
||||
max_backoff=2.0,
|
||||
)
|
||||
logger.info(f"Codex LLM verified: {self.model}")
|
||||
except Exception as e:
|
||||
raise RuntimeError(f"Codex LLM connection verification failed for {self.model}: {e}") from e
|
||||
|
||||
async def call(
|
||||
self,
|
||||
messages: list[dict[str, str]],
|
||||
response_format: Any | None = None,
|
||||
max_completion_tokens: int | None = None,
|
||||
temperature: float | None = None,
|
||||
scope: str = "memory",
|
||||
max_retries: int = 10,
|
||||
initial_backoff: float = 1.0,
|
||||
max_backoff: float = 60.0,
|
||||
skip_validation: bool = False,
|
||||
strict_schema: bool = False,
|
||||
return_usage: bool = False,
|
||||
) -> Any:
|
||||
"""Make API call to Codex backend with SSE streaming."""
|
||||
start_time = time.time()
|
||||
|
||||
# Prepare system instructions
|
||||
system_instruction = ""
|
||||
user_messages = []
|
||||
|
||||
for msg in messages:
|
||||
role = msg.get("role", "user")
|
||||
content = msg.get("content", "")
|
||||
|
||||
if role == "system":
|
||||
system_instruction += ("\n\n" + content) if system_instruction else content
|
||||
else:
|
||||
user_messages.append(msg)
|
||||
|
||||
# Add JSON schema instruction if response_format is provided
|
||||
if response_format is not None and hasattr(response_format, "model_json_schema"):
|
||||
schema = response_format.model_json_schema()
|
||||
schema_msg = f"\n\nYou must respond with valid JSON matching this schema:\n{json.dumps(schema, indent=2)}"
|
||||
system_instruction += schema_msg
|
||||
|
||||
# Build Codex request payload
|
||||
payload = {
|
||||
"model": self.model,
|
||||
"instructions": system_instruction,
|
||||
"input": [
|
||||
{
|
||||
"type": "message",
|
||||
"role": msg.get("role", "user"),
|
||||
"content": msg.get("content", ""),
|
||||
}
|
||||
for msg in user_messages
|
||||
],
|
||||
"tools": [],
|
||||
"tool_choice": "auto",
|
||||
"parallel_tool_calls": True,
|
||||
"reasoning": {"summary": self.reasoning_summary},
|
||||
"store": False, # Codex uses stateless mode
|
||||
"stream": True, # SSE streaming
|
||||
"include": ["reasoning.encrypted_content"],
|
||||
"prompt_cache_key": str(uuid.uuid4()),
|
||||
}
|
||||
|
||||
headers = {
|
||||
"Authorization": f"Bearer {self.access_token}",
|
||||
"Content-Type": "application/json",
|
||||
"OpenAI-Account-ID": self.account_id,
|
||||
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)",
|
||||
"Origin": "https://chatgpt.com",
|
||||
}
|
||||
|
||||
url = f"{self.base_url}/codex/responses"
|
||||
last_exception = None
|
||||
|
||||
for attempt in range(max_retries + 1):
|
||||
try:
|
||||
response = await self._client.post(url, json=payload, headers=headers, timeout=120.0)
|
||||
response.raise_for_status()
|
||||
|
||||
# Parse SSE stream
|
||||
content = await self._parse_sse_stream(response)
|
||||
|
||||
# Handle structured output
|
||||
if response_format is not None:
|
||||
# Models may wrap JSON in markdown
|
||||
clean_content = content
|
||||
if "```json" in content:
|
||||
clean_content = content.split("```json")[1].split("```")[0].strip()
|
||||
elif "```" in content:
|
||||
clean_content = content.split("```")[1].split("```")[0].strip()
|
||||
|
||||
try:
|
||||
json_data = json.loads(clean_content)
|
||||
except json.JSONDecodeError as e:
|
||||
logger.warning(f"Codex JSON parse error (attempt {attempt + 1}/{max_retries + 1}): {e}")
|
||||
if attempt < max_retries:
|
||||
backoff = min(initial_backoff * (2**attempt), max_backoff)
|
||||
await asyncio.sleep(backoff)
|
||||
last_exception = e
|
||||
continue
|
||||
raise
|
||||
|
||||
if skip_validation:
|
||||
result = json_data
|
||||
else:
|
||||
result = response_format.model_validate(json_data)
|
||||
else:
|
||||
result = content
|
||||
|
||||
# Record metrics
|
||||
duration = time.time() - start_time
|
||||
metrics = get_metrics_collector()
|
||||
metrics.record_llm_call(
|
||||
provider=self.provider,
|
||||
model=self.model,
|
||||
scope=scope,
|
||||
duration=duration,
|
||||
input_tokens=0, # Codex doesn't report token counts in SSE
|
||||
output_tokens=0,
|
||||
success=True,
|
||||
)
|
||||
|
||||
if return_usage:
|
||||
# Codex doesn't provide token counts, estimate based on content
|
||||
estimated_input = sum(len(m.get("content", "")) for m in messages) // 4
|
||||
estimated_output = len(content) // 4
|
||||
token_usage = TokenUsage(
|
||||
input_tokens=estimated_input,
|
||||
output_tokens=estimated_output,
|
||||
total_tokens=estimated_input + estimated_output,
|
||||
)
|
||||
return result, token_usage
|
||||
|
||||
return result
|
||||
|
||||
except httpx.HTTPStatusError as e:
|
||||
last_exception = e
|
||||
status_code = e.response.status_code
|
||||
|
||||
# Fast fail on auth errors
|
||||
if status_code in (401, 403):
|
||||
logger.error(f"Codex auth error (HTTP {status_code}): {e.response.text[:200]}")
|
||||
raise RuntimeError(
|
||||
"Codex authentication failed. Your OAuth token may have expired.\n"
|
||||
"Run 'codex auth login' to re-authenticate."
|
||||
) from e
|
||||
|
||||
if attempt < max_retries:
|
||||
backoff = min(initial_backoff * (2**attempt), max_backoff)
|
||||
logger.warning(f"Codex HTTP error {status_code} (attempt {attempt + 1}/{max_retries + 1})")
|
||||
await asyncio.sleep(backoff)
|
||||
continue
|
||||
else:
|
||||
logger.error(f"Codex HTTP error after {max_retries + 1} attempts: {e}")
|
||||
raise
|
||||
|
||||
except httpx.RequestError as e:
|
||||
last_exception = e
|
||||
if attempt < max_retries:
|
||||
backoff = min(initial_backoff * (2**attempt), max_backoff)
|
||||
logger.warning(f"Codex connection error (attempt {attempt + 1}/{max_retries + 1}): {e}")
|
||||
await asyncio.sleep(backoff)
|
||||
continue
|
||||
else:
|
||||
logger.error(f"Codex connection error after {max_retries + 1} attempts: {e}")
|
||||
raise
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Unexpected Codex error: {type(e).__name__}: {e}")
|
||||
raise
|
||||
|
||||
if last_exception:
|
||||
raise last_exception
|
||||
raise RuntimeError("Codex call failed after all retries")
|
||||
|
||||
async def _parse_sse_stream(self, response: httpx.Response) -> str:
|
||||
"""
|
||||
Parse Server-Sent Events (SSE) stream from Codex API.
|
||||
|
||||
Args:
|
||||
response: HTTP response with SSE stream.
|
||||
|
||||
Returns:
|
||||
Extracted text content from stream.
|
||||
"""
|
||||
full_text = ""
|
||||
event_type = None
|
||||
|
||||
async for line in response.aiter_lines():
|
||||
if not line:
|
||||
continue
|
||||
|
||||
# Track event type
|
||||
if line.startswith("event: "):
|
||||
event_type = line[7:]
|
||||
|
||||
# Parse data
|
||||
elif line.startswith("data: "):
|
||||
data_str = line[6:]
|
||||
if data_str == "[DONE]":
|
||||
break
|
||||
|
||||
try:
|
||||
data = json.loads(data_str)
|
||||
|
||||
# Extract content based on event type
|
||||
if event_type == "response.text.delta" and "delta" in data:
|
||||
full_text += data["delta"]
|
||||
elif event_type == "response.content_part.delta" and "delta" in data:
|
||||
full_text += data["delta"]
|
||||
# Check for item content
|
||||
elif "item" in data:
|
||||
item = data["item"]
|
||||
if "content" in item:
|
||||
content = item["content"]
|
||||
if isinstance(content, list):
|
||||
for part in content:
|
||||
if isinstance(part, dict) and "text" in part:
|
||||
full_text += part["text"]
|
||||
elif isinstance(content, str):
|
||||
full_text += content
|
||||
|
||||
except json.JSONDecodeError:
|
||||
# Skip malformed JSON events
|
||||
pass
|
||||
|
||||
return full_text
|
||||
|
||||
async def call_with_tools(
|
||||
self,
|
||||
messages: list[dict[str, Any]],
|
||||
tools: list[dict[str, Any]],
|
||||
max_completion_tokens: int | None = None,
|
||||
temperature: float | None = None,
|
||||
scope: str = "tools",
|
||||
max_retries: int = 5,
|
||||
initial_backoff: float = 1.0,
|
||||
max_backoff: float = 30.0,
|
||||
tool_choice: str | dict[str, Any] = "auto",
|
||||
) -> LLMToolCallResult:
|
||||
"""
|
||||
Make API call with tool calling support.
|
||||
|
||||
Note: This is a basic implementation. Full tool calling support for Codex
|
||||
may require additional SSE event parsing.
|
||||
"""
|
||||
start_time = time.time()
|
||||
|
||||
# Prepare system instructions
|
||||
system_instruction = ""
|
||||
user_messages = []
|
||||
|
||||
for msg in messages:
|
||||
role = msg.get("role", "user")
|
||||
content = msg.get("content", "")
|
||||
|
||||
if role == "system":
|
||||
system_instruction += ("\n\n" + content) if system_instruction else content
|
||||
elif role == "tool":
|
||||
# Handle tool results
|
||||
user_messages.append(
|
||||
{
|
||||
"type": "message",
|
||||
"role": "user",
|
||||
"content": f"Tool result: {content}",
|
||||
}
|
||||
)
|
||||
else:
|
||||
user_messages.append(
|
||||
{
|
||||
"type": "message",
|
||||
"role": role,
|
||||
"content": content,
|
||||
}
|
||||
)
|
||||
|
||||
# Convert tools to Codex format
|
||||
codex_tools = []
|
||||
for tool in tools:
|
||||
func = tool.get("function", {})
|
||||
codex_tools.append(
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": func.get("name", ""),
|
||||
"description": func.get("description", ""),
|
||||
"parameters": func.get("parameters", {}),
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
payload = {
|
||||
"model": self.model,
|
||||
"instructions": system_instruction,
|
||||
"input": user_messages,
|
||||
"tools": codex_tools,
|
||||
"tool_choice": tool_choice,
|
||||
"parallel_tool_calls": True,
|
||||
"reasoning": {"summary": self.reasoning_summary},
|
||||
"store": False,
|
||||
"stream": True,
|
||||
"include": ["reasoning.encrypted_content"],
|
||||
"prompt_cache_key": str(uuid.uuid4()),
|
||||
}
|
||||
|
||||
headers = {
|
||||
"Authorization": f"Bearer {self.access_token}",
|
||||
"Content-Type": "application/json",
|
||||
"OpenAI-Account-ID": self.account_id,
|
||||
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)",
|
||||
"Origin": "https://chatgpt.com",
|
||||
}
|
||||
|
||||
url = f"{self.base_url}/codex/responses"
|
||||
|
||||
try:
|
||||
response = await self._client.post(url, json=payload, headers=headers, timeout=120.0)
|
||||
response.raise_for_status()
|
||||
|
||||
# Parse SSE for tool calls and content
|
||||
content, tool_calls = await self._parse_sse_tool_stream(response)
|
||||
|
||||
duration = time.time() - start_time
|
||||
metrics = get_metrics_collector()
|
||||
metrics.record_llm_call(
|
||||
provider=self.provider,
|
||||
model=self.model,
|
||||
scope=scope,
|
||||
duration=duration,
|
||||
input_tokens=0,
|
||||
output_tokens=0,
|
||||
success=True,
|
||||
)
|
||||
|
||||
return LLMToolCallResult(
|
||||
content=content,
|
||||
tool_calls=tool_calls,
|
||||
finish_reason="tool_calls" if tool_calls else "stop",
|
||||
input_tokens=0,
|
||||
output_tokens=0,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Codex tool call error: {e}")
|
||||
raise
|
||||
|
||||
async def _parse_sse_tool_stream(self, response: httpx.Response) -> tuple[str | None, list[LLMToolCall]]:
|
||||
"""
|
||||
Parse SSE stream for tool calls and content.
|
||||
|
||||
Returns:
|
||||
Tuple of (content, tool_calls).
|
||||
"""
|
||||
content = ""
|
||||
tool_calls: list[LLMToolCall] = []
|
||||
event_type = None
|
||||
|
||||
async for line in response.aiter_lines():
|
||||
if not line:
|
||||
continue
|
||||
|
||||
if line.startswith("event: "):
|
||||
event_type = line[7:]
|
||||
|
||||
elif line.startswith("data: "):
|
||||
data_str = line[6:]
|
||||
if data_str == "[DONE]":
|
||||
break
|
||||
|
||||
try:
|
||||
data = json.loads(data_str)
|
||||
|
||||
# Extract text content
|
||||
if event_type == "response.text.delta" and "delta" in data:
|
||||
content += data["delta"]
|
||||
|
||||
# Extract tool calls
|
||||
elif event_type == "response.function_call_arguments.delta":
|
||||
# Handle tool call events (implementation depends on actual Codex SSE format)
|
||||
pass
|
||||
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
|
||||
return content if content else None, tool_calls
|
||||
|
||||
async def cleanup(self) -> None:
|
||||
"""Clean up HTTP client."""
|
||||
await self._client.aclose()
|
||||
@@ -1,502 +0,0 @@
|
||||
"""
|
||||
Google Gemini/VertexAI LLM provider.
|
||||
|
||||
This provider supports both:
|
||||
1. Gemini API (api.generativeai.google.com) with API key authentication
|
||||
2. Vertex AI with service account or Application Default Credentials (ADC)
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
from google import genai
|
||||
from google.genai import errors as genai_errors
|
||||
from google.genai import types as genai_types
|
||||
|
||||
from hindsight_api.engine.llm_interface import LLMInterface, OutputTooLongError
|
||||
from hindsight_api.engine.response_models import LLMToolCall, LLMToolCallResult, TokenUsage
|
||||
from hindsight_api.metrics import get_metrics_collector
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Vertex AI imports (optional)
|
||||
try:
|
||||
import google.auth
|
||||
from google.oauth2 import service_account
|
||||
|
||||
VERTEXAI_AVAILABLE = True
|
||||
except ImportError:
|
||||
VERTEXAI_AVAILABLE = False
|
||||
|
||||
|
||||
class GeminiLLM(LLMInterface):
|
||||
"""
|
||||
LLM provider for Google Gemini and Vertex AI.
|
||||
|
||||
Supports:
|
||||
- Gemini API: provider="gemini", requires api_key
|
||||
- Vertex AI: provider="vertexai", requires project_id and region, uses ADC or service account
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
provider: str,
|
||||
api_key: str,
|
||||
base_url: str,
|
||||
model: str,
|
||||
reasoning_effort: str = "low",
|
||||
**kwargs: Any,
|
||||
):
|
||||
"""Initialize Gemini/VertexAI LLM provider."""
|
||||
super().__init__(provider, api_key, base_url, model, reasoning_effort, **kwargs)
|
||||
|
||||
self._client = None
|
||||
self._is_vertexai = self.provider == "vertexai"
|
||||
|
||||
if self._is_vertexai:
|
||||
self._init_vertexai(**kwargs)
|
||||
else:
|
||||
self._init_gemini()
|
||||
|
||||
def _init_gemini(self) -> None:
|
||||
"""Initialize Gemini API client."""
|
||||
if not self.api_key:
|
||||
raise ValueError("Gemini provider requires api_key")
|
||||
|
||||
self._client = genai.Client(api_key=self.api_key)
|
||||
logger.info(f"Gemini API: model={self.model}")
|
||||
|
||||
def _init_vertexai(self, **kwargs: Any) -> None:
|
||||
"""Initialize Vertex AI client with project, region, and credentials."""
|
||||
# Extract Vertex AI config from kwargs
|
||||
project_id = kwargs.get("vertexai_project_id")
|
||||
region = kwargs.get("vertexai_region", "us-central1")
|
||||
service_account_key = kwargs.get("vertexai_service_account_key")
|
||||
credentials = kwargs.get("vertexai_credentials") # Pre-loaded credentials object
|
||||
|
||||
if not project_id:
|
||||
raise ValueError(
|
||||
"HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID is required for Vertex AI provider. "
|
||||
"Set it to your GCP project ID."
|
||||
)
|
||||
|
||||
auth_method = "ADC"
|
||||
|
||||
# Use pre-loaded credentials if provided (passed from LLMProvider)
|
||||
if credentials is not None:
|
||||
auth_method = "service_account"
|
||||
# Otherwise, load explicit service account credentials if path provided
|
||||
elif service_account_key:
|
||||
if not VERTEXAI_AVAILABLE:
|
||||
raise ValueError(
|
||||
"Vertex AI service account auth requires 'google-auth' package. "
|
||||
"Install with: pip install google-auth"
|
||||
)
|
||||
credentials = service_account.Credentials.from_service_account_file(
|
||||
service_account_key,
|
||||
scopes=["https://www.googleapis.com/auth/cloud-platform"],
|
||||
)
|
||||
auth_method = "service_account"
|
||||
logger.info(f"Vertex AI: Using service account key: {service_account_key}")
|
||||
|
||||
# Strip google/ prefix from model name — native SDK uses bare names
|
||||
# e.g. "google/gemini-2.0-flash-lite-001" -> "gemini-2.0-flash-lite-001"
|
||||
if self.model.startswith("google/"):
|
||||
self.model = self.model[len("google/") :]
|
||||
|
||||
# Create Vertex AI client
|
||||
client_kwargs: dict[str, Any] = {
|
||||
"vertexai": True,
|
||||
"project": project_id,
|
||||
"location": region,
|
||||
}
|
||||
if credentials is not None:
|
||||
client_kwargs["credentials"] = credentials
|
||||
|
||||
self._client = genai.Client(**client_kwargs)
|
||||
|
||||
logger.info(f"Vertex AI: project={project_id}, region={region}, model={self.model}, auth={auth_method}")
|
||||
|
||||
async def verify_connection(self) -> None:
|
||||
"""
|
||||
Verify that the Gemini/VertexAI provider is configured correctly.
|
||||
|
||||
Raises:
|
||||
RuntimeError: If the connection test fails.
|
||||
"""
|
||||
try:
|
||||
logger.info(f"Verifying {self.provider.upper()}: model={self.model}...")
|
||||
await self.call(
|
||||
messages=[{"role": "user", "content": "Say 'ok'"}],
|
||||
max_completion_tokens=100,
|
||||
max_retries=2,
|
||||
initial_backoff=0.5,
|
||||
max_backoff=2.0,
|
||||
)
|
||||
logger.info(f"{self.provider.upper()} connection verified successfully")
|
||||
except Exception as e:
|
||||
raise RuntimeError(f"Failed to verify {self.provider.upper()} connection: {e}") from e
|
||||
|
||||
async def call(
|
||||
self,
|
||||
messages: list[dict[str, str]],
|
||||
response_format: Any | None = None,
|
||||
max_completion_tokens: int | None = None,
|
||||
temperature: float | None = None,
|
||||
scope: str = "memory",
|
||||
max_retries: int = 10,
|
||||
initial_backoff: float = 1.0,
|
||||
max_backoff: float = 60.0,
|
||||
skip_validation: bool = False,
|
||||
strict_schema: bool = False,
|
||||
return_usage: bool = False,
|
||||
) -> Any:
|
||||
"""
|
||||
Make a Gemini/VertexAI API call with retry logic.
|
||||
|
||||
Args:
|
||||
messages: List of message dicts with 'role' and 'content'.
|
||||
response_format: Optional Pydantic model for structured output.
|
||||
max_completion_tokens: Maximum tokens in response (not supported by Gemini).
|
||||
temperature: Sampling temperature (0.0-2.0).
|
||||
scope: Scope identifier for tracking.
|
||||
max_retries: Maximum retry attempts.
|
||||
initial_backoff: Initial backoff time in seconds.
|
||||
max_backoff: Maximum backoff time in seconds.
|
||||
skip_validation: Return raw JSON without Pydantic validation.
|
||||
strict_schema: Use strict JSON schema enforcement (not supported by Gemini).
|
||||
return_usage: If True, return tuple (result, TokenUsage).
|
||||
|
||||
Returns:
|
||||
If return_usage=False: Parsed response if response_format provided, else text.
|
||||
If return_usage=True: Tuple of (result, TokenUsage).
|
||||
"""
|
||||
start_time = time.time()
|
||||
|
||||
# Convert OpenAI-style messages to Gemini format
|
||||
system_instruction = None
|
||||
gemini_contents = []
|
||||
|
||||
for msg in messages:
|
||||
role = msg.get("role", "user")
|
||||
content = msg.get("content", "")
|
||||
|
||||
if role == "system":
|
||||
if system_instruction:
|
||||
system_instruction += "\n\n" + content
|
||||
else:
|
||||
system_instruction = content
|
||||
elif role == "assistant":
|
||||
gemini_contents.append(genai_types.Content(role="model", parts=[genai_types.Part(text=content)]))
|
||||
else:
|
||||
gemini_contents.append(genai_types.Content(role="user", parts=[genai_types.Part(text=content)]))
|
||||
|
||||
# Add JSON schema instruction if response_format is provided
|
||||
if response_format is not None and hasattr(response_format, "model_json_schema"):
|
||||
schema = response_format.model_json_schema()
|
||||
schema_msg = f"\n\nYou must respond with valid JSON matching this schema:\n{json.dumps(schema, indent=2)}"
|
||||
if system_instruction:
|
||||
system_instruction += schema_msg
|
||||
else:
|
||||
system_instruction = schema_msg
|
||||
|
||||
# Build generation config
|
||||
config_kwargs: dict[str, Any] = {}
|
||||
if system_instruction:
|
||||
config_kwargs["system_instruction"] = system_instruction
|
||||
if response_format is not None:
|
||||
config_kwargs["response_mime_type"] = "application/json"
|
||||
config_kwargs["response_schema"] = response_format
|
||||
if temperature is not None:
|
||||
config_kwargs["temperature"] = temperature
|
||||
|
||||
generation_config = genai_types.GenerateContentConfig(**config_kwargs) if config_kwargs else None
|
||||
|
||||
last_exception = None
|
||||
|
||||
for attempt in range(max_retries + 1):
|
||||
try:
|
||||
response = await self._client.aio.models.generate_content(
|
||||
model=self.model,
|
||||
contents=gemini_contents,
|
||||
config=generation_config,
|
||||
)
|
||||
|
||||
content = response.text
|
||||
|
||||
# Handle empty response
|
||||
if content is None:
|
||||
block_reason = None
|
||||
if hasattr(response, "candidates") and response.candidates:
|
||||
candidate = response.candidates[0]
|
||||
if hasattr(candidate, "finish_reason"):
|
||||
block_reason = candidate.finish_reason
|
||||
|
||||
if attempt < max_retries:
|
||||
logger.warning(f"Gemini returned empty response (reason: {block_reason}), retrying...")
|
||||
backoff = min(initial_backoff * (2**attempt), max_backoff)
|
||||
await asyncio.sleep(backoff)
|
||||
continue
|
||||
else:
|
||||
raise RuntimeError(f"Gemini returned empty response after {max_retries + 1} attempts")
|
||||
|
||||
# Parse structured output if requested
|
||||
if response_format is not None:
|
||||
json_data = json.loads(content)
|
||||
if skip_validation:
|
||||
result = json_data
|
||||
else:
|
||||
result = response_format.model_validate(json_data)
|
||||
else:
|
||||
result = content
|
||||
|
||||
# Extract token usage
|
||||
input_tokens = 0
|
||||
output_tokens = 0
|
||||
if hasattr(response, "usage_metadata") and response.usage_metadata:
|
||||
usage = response.usage_metadata
|
||||
input_tokens = usage.prompt_token_count or 0
|
||||
output_tokens = usage.candidates_token_count or 0
|
||||
|
||||
# Record metrics
|
||||
duration = time.time() - start_time
|
||||
metrics = get_metrics_collector()
|
||||
metrics.record_llm_call(
|
||||
provider=self.provider,
|
||||
model=self.model,
|
||||
scope=scope,
|
||||
duration=duration,
|
||||
input_tokens=input_tokens,
|
||||
output_tokens=output_tokens,
|
||||
success=True,
|
||||
)
|
||||
|
||||
# Log slow calls
|
||||
if duration > 10.0 and input_tokens > 0:
|
||||
logger.info(
|
||||
f"slow llm call: scope={scope}, model={self.provider}/{self.model}, "
|
||||
f"input_tokens={input_tokens}, output_tokens={output_tokens}, "
|
||||
f"time={duration:.3f}s"
|
||||
)
|
||||
|
||||
if return_usage:
|
||||
token_usage = TokenUsage(
|
||||
input_tokens=input_tokens,
|
||||
output_tokens=output_tokens,
|
||||
total_tokens=input_tokens + output_tokens,
|
||||
)
|
||||
return result, token_usage
|
||||
return result
|
||||
|
||||
except json.JSONDecodeError as e:
|
||||
last_exception = e
|
||||
if attempt < max_retries:
|
||||
logger.warning("Gemini returned invalid JSON, retrying...")
|
||||
backoff = min(initial_backoff * (2**attempt), max_backoff)
|
||||
await asyncio.sleep(backoff)
|
||||
continue
|
||||
else:
|
||||
logger.error(f"Gemini returned invalid JSON after {max_retries + 1} attempts")
|
||||
raise
|
||||
|
||||
except genai_errors.APIError as e:
|
||||
# Fast fail on auth errors - these won't recover with retries
|
||||
if e.code in (401, 403):
|
||||
logger.error(f"Gemini auth error (HTTP {e.code}), not retrying: {str(e)}")
|
||||
raise
|
||||
|
||||
# Retry on retryable errors (rate limits, server errors, client errors)
|
||||
if e.code in (400, 429, 500, 502, 503, 504) or (e.code and e.code >= 500):
|
||||
last_exception = e
|
||||
if attempt < max_retries:
|
||||
backoff = min(initial_backoff * (2**attempt), max_backoff)
|
||||
jitter = backoff * 0.2 * (2 * (time.time() % 1) - 1)
|
||||
await asyncio.sleep(backoff + jitter)
|
||||
else:
|
||||
logger.error(f"Gemini API error after {max_retries + 1} attempts: {str(e)}")
|
||||
raise
|
||||
else:
|
||||
logger.error(f"Gemini API error: {type(e).__name__}: {str(e)}")
|
||||
raise
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Unexpected error during Gemini call: {type(e).__name__}: {str(e)}")
|
||||
raise
|
||||
|
||||
if last_exception:
|
||||
raise last_exception
|
||||
raise RuntimeError("Gemini call failed after all retries")
|
||||
|
||||
async def call_with_tools(
|
||||
self,
|
||||
messages: list[dict[str, Any]],
|
||||
tools: list[dict[str, Any]],
|
||||
max_completion_tokens: int | None = None,
|
||||
temperature: float | None = None,
|
||||
scope: str = "tools",
|
||||
max_retries: int = 5,
|
||||
initial_backoff: float = 1.0,
|
||||
max_backoff: float = 30.0,
|
||||
tool_choice: str | dict[str, Any] = "auto",
|
||||
) -> LLMToolCallResult:
|
||||
"""
|
||||
Make a Gemini/VertexAI API call with tool/function calling support.
|
||||
|
||||
Args:
|
||||
messages: List of message dicts. Can include tool results with role='tool'.
|
||||
tools: List of tool definitions in OpenAI format.
|
||||
max_completion_tokens: Maximum tokens (not supported by Gemini).
|
||||
temperature: Sampling temperature.
|
||||
scope: Scope identifier for tracking.
|
||||
max_retries: Maximum retry attempts.
|
||||
initial_backoff: Initial backoff time in seconds.
|
||||
max_backoff: Maximum backoff time in seconds.
|
||||
tool_choice: How to choose tools (Gemini uses "auto" only).
|
||||
|
||||
Returns:
|
||||
LLMToolCallResult with content and/or tool_calls.
|
||||
"""
|
||||
start_time = time.time()
|
||||
|
||||
# Convert tools to Gemini format
|
||||
gemini_tools = []
|
||||
for tool in tools:
|
||||
func = tool.get("function", {})
|
||||
gemini_tools.append(
|
||||
genai_types.Tool(
|
||||
function_declarations=[
|
||||
genai_types.FunctionDeclaration(
|
||||
name=func.get("name", ""),
|
||||
description=func.get("description", ""),
|
||||
parameters=func.get("parameters"),
|
||||
)
|
||||
]
|
||||
)
|
||||
)
|
||||
|
||||
# Convert messages
|
||||
system_instruction = None
|
||||
gemini_contents = []
|
||||
for msg in messages:
|
||||
role = msg.get("role", "user")
|
||||
content = msg.get("content", "")
|
||||
|
||||
if role == "system":
|
||||
system_instruction = (system_instruction + "\n\n" + content) if system_instruction else content
|
||||
elif role == "tool":
|
||||
# Gemini uses function_response
|
||||
gemini_contents.append(
|
||||
genai_types.Content(
|
||||
role="user",
|
||||
parts=[
|
||||
genai_types.Part(
|
||||
function_response=genai_types.FunctionResponse(
|
||||
name=msg.get("name", ""),
|
||||
response={"result": content},
|
||||
)
|
||||
)
|
||||
],
|
||||
)
|
||||
)
|
||||
elif role == "assistant":
|
||||
gemini_contents.append(genai_types.Content(role="model", parts=[genai_types.Part(text=content)]))
|
||||
else:
|
||||
gemini_contents.append(genai_types.Content(role="user", parts=[genai_types.Part(text=content)]))
|
||||
|
||||
config_kwargs: dict[str, Any] = {"tools": gemini_tools}
|
||||
if system_instruction:
|
||||
config_kwargs["system_instruction"] = system_instruction
|
||||
if temperature is not None:
|
||||
config_kwargs["temperature"] = temperature
|
||||
|
||||
config = genai_types.GenerateContentConfig(**config_kwargs)
|
||||
|
||||
last_exception = None
|
||||
for attempt in range(max_retries + 1):
|
||||
try:
|
||||
response = await self._client.aio.models.generate_content(
|
||||
model=self.model,
|
||||
contents=gemini_contents,
|
||||
config=config,
|
||||
)
|
||||
|
||||
# Extract content and tool calls
|
||||
content = None
|
||||
tool_calls: list[LLMToolCall] = []
|
||||
|
||||
if response.candidates and response.candidates[0].content:
|
||||
parts = response.candidates[0].content.parts
|
||||
if parts:
|
||||
for part in parts:
|
||||
if hasattr(part, "text") and part.text:
|
||||
content = part.text
|
||||
if hasattr(part, "function_call") and part.function_call:
|
||||
fc = part.function_call
|
||||
tool_calls.append(
|
||||
LLMToolCall(
|
||||
id=f"gemini_{len(tool_calls)}",
|
||||
name=fc.name,
|
||||
arguments=dict(fc.args) if fc.args else {},
|
||||
)
|
||||
)
|
||||
|
||||
finish_reason = "tool_calls" if tool_calls else "stop"
|
||||
|
||||
# Extract token usage
|
||||
input_tokens = 0
|
||||
output_tokens = 0
|
||||
if response.usage_metadata:
|
||||
input_tokens = response.usage_metadata.prompt_token_count or 0
|
||||
output_tokens = response.usage_metadata.candidates_token_count or 0
|
||||
|
||||
# Record metrics
|
||||
duration = time.time() - start_time
|
||||
metrics = get_metrics_collector()
|
||||
metrics.record_llm_call(
|
||||
provider=self.provider,
|
||||
model=self.model,
|
||||
scope=scope,
|
||||
duration=duration,
|
||||
input_tokens=input_tokens,
|
||||
output_tokens=output_tokens,
|
||||
success=True,
|
||||
)
|
||||
|
||||
return LLMToolCallResult(
|
||||
content=content,
|
||||
tool_calls=tool_calls,
|
||||
finish_reason=finish_reason,
|
||||
input_tokens=input_tokens,
|
||||
output_tokens=output_tokens,
|
||||
)
|
||||
|
||||
except genai_errors.APIError as e:
|
||||
# Fast fail on auth errors
|
||||
if e.code in (401, 403):
|
||||
logger.error(f"Gemini auth error (HTTP {e.code}), not retrying: {str(e)}")
|
||||
raise
|
||||
|
||||
# Retry on retryable errors
|
||||
last_exception = e
|
||||
if attempt < max_retries:
|
||||
backoff = min(initial_backoff * (2**attempt), max_backoff)
|
||||
await asyncio.sleep(backoff)
|
||||
continue
|
||||
raise
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Unexpected error during Gemini tool call: {type(e).__name__}: {str(e)}")
|
||||
raise
|
||||
|
||||
if last_exception:
|
||||
raise last_exception
|
||||
raise RuntimeError("Gemini tool call failed")
|
||||
|
||||
async def cleanup(self) -> None:
|
||||
"""Clean up resources (close connections, etc.)."""
|
||||
# Gemini client doesn't require explicit cleanup
|
||||
pass
|
||||
@@ -1,234 +0,0 @@
|
||||
"""
|
||||
Mock LLM provider for testing.
|
||||
|
||||
This provider allows tests to record LLM calls and return configurable mock responses
|
||||
without making actual API calls to external LLM services.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from typing import Any
|
||||
|
||||
from ..llm_interface import LLMInterface
|
||||
from ..response_models import LLMToolCall, LLMToolCallResult, TokenUsage
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class MockLLM(LLMInterface):
|
||||
"""
|
||||
Mock LLM provider for testing.
|
||||
|
||||
This provider records all calls and returns configurable mock responses,
|
||||
enabling tests to verify LLM interactions without making real API calls.
|
||||
|
||||
Example:
|
||||
# Create mock provider
|
||||
mock_llm = MockLLM(provider="mock", api_key="", base_url="", model="mock-model")
|
||||
|
||||
# Set mock response
|
||||
mock_llm.set_mock_response({"answer": "test"})
|
||||
|
||||
# Make calls
|
||||
result = await mock_llm.call(
|
||||
messages=[{"role": "user", "content": "test"}],
|
||||
response_format=MyResponseModel
|
||||
)
|
||||
|
||||
# Verify calls
|
||||
calls = mock_llm.get_mock_calls()
|
||||
assert len(calls) == 1
|
||||
assert calls[0]["scope"] == "memory"
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
provider: str,
|
||||
api_key: str,
|
||||
base_url: str,
|
||||
model: str,
|
||||
reasoning_effort: str = "low",
|
||||
**kwargs: Any,
|
||||
):
|
||||
"""
|
||||
Initialize mock LLM provider.
|
||||
|
||||
Args:
|
||||
provider: Provider name (should be "mock").
|
||||
api_key: Not used for mock provider.
|
||||
base_url: Not used for mock provider.
|
||||
model: Model name for tracking.
|
||||
reasoning_effort: Not used for mock provider.
|
||||
**kwargs: Additional parameters (not used).
|
||||
"""
|
||||
super().__init__(provider, api_key, base_url, model, reasoning_effort, **kwargs)
|
||||
|
||||
# Storage for test verification
|
||||
self._mock_calls: list[dict] = []
|
||||
self._mock_response: Any = None
|
||||
|
||||
async def verify_connection(self) -> None:
|
||||
"""
|
||||
Verify mock provider (always succeeds).
|
||||
|
||||
Mock provider doesn't need connection verification since it doesn't
|
||||
make real API calls.
|
||||
"""
|
||||
logger.debug("Mock LLM: connection verification (always succeeds)")
|
||||
|
||||
async def call(
|
||||
self,
|
||||
messages: list[dict[str, str]],
|
||||
response_format: Any | None = None,
|
||||
max_completion_tokens: int | None = None,
|
||||
temperature: float | None = None,
|
||||
scope: str = "memory",
|
||||
max_retries: int = 10,
|
||||
initial_backoff: float = 1.0,
|
||||
max_backoff: float = 60.0,
|
||||
skip_validation: bool = False,
|
||||
strict_schema: bool = False,
|
||||
return_usage: bool = False,
|
||||
) -> Any:
|
||||
"""
|
||||
Make a mock LLM API call.
|
||||
|
||||
Records the call for test verification and returns the configured mock response.
|
||||
|
||||
Args:
|
||||
messages: List of message dicts with 'role' and 'content'.
|
||||
response_format: Optional Pydantic model for structured output.
|
||||
max_completion_tokens: Not used in mock.
|
||||
temperature: Not used in mock.
|
||||
scope: Scope identifier for tracking.
|
||||
max_retries: Not used in mock.
|
||||
initial_backoff: Not used in mock.
|
||||
max_backoff: Not used in mock.
|
||||
skip_validation: Return raw JSON without Pydantic validation.
|
||||
strict_schema: Not used in mock.
|
||||
return_usage: If True, return tuple (result, TokenUsage) instead of just result.
|
||||
|
||||
Returns:
|
||||
If return_usage=False: Parsed response if response_format is provided, otherwise text content.
|
||||
If return_usage=True: Tuple of (result, TokenUsage) with mock token counts.
|
||||
"""
|
||||
# Record the call for test verification
|
||||
call_record = {
|
||||
"provider": self.provider,
|
||||
"model": self.model,
|
||||
"messages": messages,
|
||||
"response_format": response_format.__name__
|
||||
if response_format and hasattr(response_format, "__name__")
|
||||
else str(response_format),
|
||||
"scope": scope,
|
||||
}
|
||||
self._mock_calls.append(call_record)
|
||||
logger.debug(f"Mock LLM call recorded: scope={scope}, model={self.model}")
|
||||
|
||||
# Return mock response
|
||||
if self._mock_response is not None:
|
||||
result = self._mock_response
|
||||
elif response_format is not None:
|
||||
# Try to create a minimal valid instance of the response format
|
||||
try:
|
||||
# For Pydantic models, try to create with minimal valid data
|
||||
result = {"mock": True}
|
||||
except Exception:
|
||||
result = {"mock": True}
|
||||
else:
|
||||
result = "mock response"
|
||||
|
||||
if return_usage:
|
||||
token_usage = TokenUsage(input_tokens=10, output_tokens=5, total_tokens=15)
|
||||
return result, token_usage
|
||||
return result
|
||||
|
||||
async def call_with_tools(
|
||||
self,
|
||||
messages: list[dict[str, Any]],
|
||||
tools: list[dict[str, Any]],
|
||||
max_completion_tokens: int | None = None,
|
||||
temperature: float | None = None,
|
||||
scope: str = "tools",
|
||||
max_retries: int = 5,
|
||||
initial_backoff: float = 1.0,
|
||||
max_backoff: float = 30.0,
|
||||
tool_choice: str | dict[str, Any] = "auto",
|
||||
) -> LLMToolCallResult:
|
||||
"""
|
||||
Make a mock LLM API call with tool/function calling support.
|
||||
|
||||
Records the call for test verification and returns the configured mock response.
|
||||
|
||||
Args:
|
||||
messages: List of message dicts. Can include tool results with role='tool'.
|
||||
tools: List of tool definitions in OpenAI format.
|
||||
max_completion_tokens: Not used in mock.
|
||||
temperature: Not used in mock.
|
||||
scope: Scope identifier for tracking.
|
||||
max_retries: Not used in mock.
|
||||
initial_backoff: Not used in mock.
|
||||
max_backoff: Not used in mock.
|
||||
tool_choice: Not used in mock.
|
||||
|
||||
Returns:
|
||||
LLMToolCallResult with content and/or tool_calls.
|
||||
"""
|
||||
# Record the call for test verification
|
||||
call_record = {
|
||||
"provider": self.provider,
|
||||
"model": self.model,
|
||||
"messages": messages,
|
||||
"tools": [t.get("function", {}).get("name") for t in tools],
|
||||
"scope": scope,
|
||||
}
|
||||
self._mock_calls.append(call_record)
|
||||
|
||||
if self._mock_response is not None:
|
||||
if isinstance(self._mock_response, LLMToolCallResult):
|
||||
return self._mock_response
|
||||
# Allow setting just tool calls as a list
|
||||
if isinstance(self._mock_response, list):
|
||||
return LLMToolCallResult(
|
||||
tool_calls=[
|
||||
LLMToolCall(id=f"mock_{i}", name=tc["name"], arguments=tc.get("arguments", {}))
|
||||
for i, tc in enumerate(self._mock_response)
|
||||
],
|
||||
finish_reason="tool_calls",
|
||||
)
|
||||
|
||||
return LLMToolCallResult(content="mock response", finish_reason="stop")
|
||||
|
||||
async def cleanup(self) -> None:
|
||||
"""Clean up resources (no-op for mock provider)."""
|
||||
pass
|
||||
|
||||
def set_mock_response(self, response: Any) -> None:
|
||||
"""
|
||||
Set the response to return from mock calls.
|
||||
|
||||
Args:
|
||||
response: The response to return. Can be:
|
||||
- A dict/Pydantic model for regular calls
|
||||
- An LLMToolCallResult for tool calls
|
||||
- A list of tool call dicts for tool calls
|
||||
- Any other value to return as-is
|
||||
"""
|
||||
self._mock_response = response
|
||||
|
||||
def get_mock_calls(self) -> list[dict]:
|
||||
"""
|
||||
Get the list of recorded mock calls.
|
||||
|
||||
Returns:
|
||||
List of call records, each containing:
|
||||
- provider: Provider name
|
||||
- model: Model name
|
||||
- messages: Messages sent
|
||||
- response_format/tools: Format or tools used
|
||||
- scope: Call scope
|
||||
"""
|
||||
return self._mock_calls
|
||||
|
||||
def clear_mock_calls(self) -> None:
|
||||
"""Clear the recorded mock calls."""
|
||||
self._mock_calls = []
|
||||
@@ -1,745 +0,0 @@
|
||||
"""
|
||||
OpenAI-compatible LLM provider supporting OpenAI, Groq, Ollama, and LMStudio.
|
||||
|
||||
This provider handles all OpenAI API-compatible models including:
|
||||
- OpenAI: GPT-4, GPT-4o, GPT-5, o1, o3 (reasoning models)
|
||||
- Groq: Fast inference with seed control and service tiers
|
||||
- Ollama: Local models with native streaming API support
|
||||
- LMStudio: Local models with OpenAI-compatible API
|
||||
|
||||
Features:
|
||||
- Reasoning models with extended thinking (o1, o3, GPT-5 families)
|
||||
- Strict JSON schema enforcement (OpenAI)
|
||||
- Provider-specific parameters (Groq seed, service tier)
|
||||
- Native Ollama streaming for better structured output
|
||||
- Automatic token limit handling per model family
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
from openai import APIConnectionError, APIStatusError, AsyncOpenAI, LengthFinishReasonError
|
||||
|
||||
from hindsight_api.config import DEFAULT_LLM_TIMEOUT, ENV_LLM_TIMEOUT
|
||||
from hindsight_api.engine.llm_interface import LLMInterface, OutputTooLongError
|
||||
from hindsight_api.engine.response_models import LLMToolCall, LLMToolCallResult, TokenUsage
|
||||
from hindsight_api.metrics import get_metrics_collector
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Seed applied to every Groq request for deterministic behavior
|
||||
DEFAULT_LLM_SEED = 4242
|
||||
|
||||
|
||||
class OpenAICompatibleLLM(LLMInterface):
|
||||
"""
|
||||
LLM provider for OpenAI-compatible APIs.
|
||||
|
||||
Supports:
|
||||
- OpenAI: Standard models (GPT-4, GPT-4o) and reasoning models (o1, o3, GPT-5)
|
||||
- Groq: Fast inference with seed control and service tiers
|
||||
- Ollama: Local models with native streaming API for better structured output
|
||||
- LMStudio: Local models with OpenAI-compatible API
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
provider: str,
|
||||
api_key: str,
|
||||
base_url: str,
|
||||
model: str,
|
||||
reasoning_effort: str = "low",
|
||||
timeout: float | None = None,
|
||||
groq_service_tier: str | None = None,
|
||||
**kwargs: Any,
|
||||
):
|
||||
"""
|
||||
Initialize OpenAI-compatible LLM provider.
|
||||
|
||||
Args:
|
||||
provider: Provider name ("openai", "groq", "ollama", "lmstudio").
|
||||
api_key: API key (optional for ollama/lmstudio).
|
||||
base_url: Base URL for the API (uses defaults for groq/ollama/lmstudio if empty).
|
||||
model: Model name.
|
||||
reasoning_effort: Reasoning effort level for supported models ("low", "medium", "high").
|
||||
timeout: Request timeout in seconds (uses env var or 300s default).
|
||||
groq_service_tier: Groq service tier ("on_demand", "flex", "auto").
|
||||
**kwargs: Additional provider-specific parameters.
|
||||
"""
|
||||
super().__init__(provider, api_key, base_url, model, reasoning_effort, **kwargs)
|
||||
|
||||
# Validate provider
|
||||
valid_providers = ["openai", "groq", "ollama", "lmstudio"]
|
||||
if self.provider not in valid_providers:
|
||||
raise ValueError(f"OpenAICompatibleLLM only supports: {', '.join(valid_providers)}. Got: {self.provider}")
|
||||
|
||||
# Set default base URLs
|
||||
if not self.base_url:
|
||||
if self.provider == "groq":
|
||||
self.base_url = "https://api.groq.com/openai/v1"
|
||||
elif self.provider == "ollama":
|
||||
self.base_url = "http://localhost:11434/v1"
|
||||
elif self.provider == "lmstudio":
|
||||
self.base_url = "http://localhost:1234/v1"
|
||||
|
||||
# For ollama/lmstudio, use dummy key if not provided
|
||||
if self.provider in ("ollama", "lmstudio") and not self.api_key:
|
||||
self.api_key = "local"
|
||||
|
||||
# Validate API key for cloud providers
|
||||
if self.provider in ("openai", "groq") and not self.api_key:
|
||||
raise ValueError(f"API key is required for {self.provider}")
|
||||
|
||||
# Groq service tier configuration
|
||||
self.groq_service_tier = groq_service_tier or os.getenv("HINDSIGHT_API_LLM_GROQ_SERVICE_TIER", "auto")
|
||||
|
||||
# Get timeout config
|
||||
self.timeout = timeout or float(os.getenv(ENV_LLM_TIMEOUT, str(DEFAULT_LLM_TIMEOUT)))
|
||||
|
||||
# Create OpenAI client
|
||||
client_kwargs: dict[str, Any] = {"api_key": self.api_key, "max_retries": 0}
|
||||
if self.base_url:
|
||||
client_kwargs["base_url"] = self.base_url
|
||||
if self.timeout:
|
||||
client_kwargs["timeout"] = self.timeout
|
||||
|
||||
self._client = AsyncOpenAI(**client_kwargs)
|
||||
logger.info(
|
||||
f"OpenAI-compatible client initialized: provider={self.provider}, model={self.model}, "
|
||||
f"base_url={self.base_url or 'default'}"
|
||||
)
|
||||
|
||||
async def verify_connection(self) -> None:
|
||||
"""
|
||||
Verify that the provider is configured correctly by making a simple test call.
|
||||
|
||||
Raises:
|
||||
RuntimeError: If the connection test fails.
|
||||
"""
|
||||
try:
|
||||
logger.info(f"Verifying connection: {self.provider}/{self.model}")
|
||||
await self.call(
|
||||
messages=[{"role": "user", "content": "Say 'ok'"}],
|
||||
max_completion_tokens=100,
|
||||
max_retries=2,
|
||||
initial_backoff=0.5,
|
||||
max_backoff=2.0,
|
||||
)
|
||||
logger.info(f"Connection verified: {self.provider}/{self.model}")
|
||||
except Exception as e:
|
||||
raise RuntimeError(f"Connection verification failed for {self.provider}/{self.model}: {e}") from e
|
||||
|
||||
def _supports_reasoning_model(self) -> bool:
|
||||
"""Check if the current model is a reasoning model (o1, o3, GPT-5, DeepSeek)."""
|
||||
model_lower = self.model.lower()
|
||||
return any(x in model_lower for x in ["gpt-5", "o1", "o3", "deepseek"])
|
||||
|
||||
def _get_max_reasoning_tokens(self) -> int | None:
|
||||
"""Get max reasoning tokens for reasoning models."""
|
||||
model_lower = self.model.lower()
|
||||
|
||||
# GPT-4 and GPT-4.1 models have different caps
|
||||
if any(x in model_lower for x in ["gpt-4.1", "gpt-4-"]):
|
||||
return 32000
|
||||
elif "gpt-4o" in model_lower:
|
||||
return 16384
|
||||
|
||||
return None
|
||||
|
||||
async def call(
|
||||
self,
|
||||
messages: list[dict[str, str]],
|
||||
response_format: Any | None = None,
|
||||
max_completion_tokens: int | None = None,
|
||||
temperature: float | None = None,
|
||||
scope: str = "memory",
|
||||
max_retries: int = 10,
|
||||
initial_backoff: float = 1.0,
|
||||
max_backoff: float = 60.0,
|
||||
skip_validation: bool = False,
|
||||
strict_schema: bool = False,
|
||||
return_usage: bool = False,
|
||||
) -> Any:
|
||||
"""
|
||||
Make an LLM API call with retry logic.
|
||||
|
||||
Args:
|
||||
messages: List of message dicts with 'role' and 'content'.
|
||||
response_format: Optional Pydantic model for structured output.
|
||||
max_completion_tokens: Maximum tokens in response.
|
||||
temperature: Sampling temperature (0.0-2.0).
|
||||
scope: Scope identifier for tracking.
|
||||
max_retries: Maximum retry attempts.
|
||||
initial_backoff: Initial backoff time in seconds.
|
||||
max_backoff: Maximum backoff time in seconds.
|
||||
skip_validation: Return raw JSON without Pydantic validation.
|
||||
strict_schema: Use strict JSON schema enforcement (OpenAI only).
|
||||
return_usage: If True, return tuple (result, TokenUsage) instead of just result.
|
||||
|
||||
Returns:
|
||||
If return_usage=False: Parsed response if response_format is provided, otherwise text content.
|
||||
If return_usage=True: Tuple of (result, TokenUsage) with token counts.
|
||||
|
||||
Raises:
|
||||
OutputTooLongError: If output exceeds token limits.
|
||||
Exception: Re-raises API errors after retries exhausted.
|
||||
"""
|
||||
# Handle Ollama with native API for structured output (better schema enforcement)
|
||||
if self.provider == "ollama" and response_format is not None:
|
||||
return await self._call_ollama_native(
|
||||
messages=messages,
|
||||
response_format=response_format,
|
||||
max_completion_tokens=max_completion_tokens,
|
||||
temperature=temperature,
|
||||
max_retries=max_retries,
|
||||
initial_backoff=initial_backoff,
|
||||
max_backoff=max_backoff,
|
||||
skip_validation=skip_validation,
|
||||
scope=scope,
|
||||
return_usage=return_usage,
|
||||
)
|
||||
|
||||
start_time = time.time()
|
||||
|
||||
# Build call parameters
|
||||
call_params: dict[str, Any] = {
|
||||
"model": self.model,
|
||||
"messages": messages,
|
||||
}
|
||||
|
||||
# Check if model supports reasoning parameter
|
||||
is_reasoning_model = self._supports_reasoning_model()
|
||||
|
||||
# Apply model-specific token limits
|
||||
if max_completion_tokens is not None:
|
||||
max_tokens_cap = self._get_max_reasoning_tokens()
|
||||
if max_tokens_cap and max_completion_tokens > max_tokens_cap:
|
||||
max_completion_tokens = max_tokens_cap
|
||||
# For reasoning models, enforce minimum to ensure space for reasoning + output
|
||||
if is_reasoning_model and max_completion_tokens < 16000:
|
||||
max_completion_tokens = 16000
|
||||
call_params["max_completion_tokens"] = max_completion_tokens
|
||||
|
||||
# Temperature - reasoning models don't support custom temperature
|
||||
if temperature is not None and not is_reasoning_model:
|
||||
call_params["temperature"] = temperature
|
||||
|
||||
# Set reasoning_effort for reasoning models
|
||||
if is_reasoning_model:
|
||||
call_params["reasoning_effort"] = self.reasoning_effort
|
||||
|
||||
# Provider-specific parameters
|
||||
if self.provider == "groq":
|
||||
call_params["seed"] = DEFAULT_LLM_SEED
|
||||
extra_body: dict[str, Any] = {}
|
||||
# Add service_tier if configured
|
||||
if self.groq_service_tier:
|
||||
extra_body["service_tier"] = self.groq_service_tier
|
||||
# Add reasoning parameters for reasoning models
|
||||
if is_reasoning_model:
|
||||
extra_body["include_reasoning"] = False
|
||||
if extra_body:
|
||||
call_params["extra_body"] = extra_body
|
||||
|
||||
# Prepare response format ONCE before retry loop
|
||||
if response_format is not None:
|
||||
schema = None
|
||||
if hasattr(response_format, "model_json_schema"):
|
||||
schema = response_format.model_json_schema()
|
||||
|
||||
if strict_schema and schema is not None:
|
||||
# Use OpenAI's strict JSON schema enforcement
|
||||
call_params["response_format"] = {
|
||||
"type": "json_schema",
|
||||
"json_schema": {
|
||||
"name": "response",
|
||||
"strict": True,
|
||||
"schema": schema,
|
||||
},
|
||||
}
|
||||
else:
|
||||
# Soft enforcement: add schema to prompt and use json_object mode
|
||||
if schema is not None:
|
||||
schema_msg = (
|
||||
f"\n\nYou must respond with valid JSON matching this schema:\n{json.dumps(schema, indent=2)}"
|
||||
)
|
||||
|
||||
if call_params["messages"] and call_params["messages"][0].get("role") == "system":
|
||||
first_msg = call_params["messages"][0]
|
||||
if isinstance(first_msg, dict) and isinstance(first_msg.get("content"), str):
|
||||
first_msg["content"] += schema_msg
|
||||
elif call_params["messages"]:
|
||||
first_msg = call_params["messages"][0]
|
||||
if isinstance(first_msg, dict) and isinstance(first_msg.get("content"), str):
|
||||
first_msg["content"] = schema_msg + "\n\n" + first_msg["content"]
|
||||
if self.provider not in ("lmstudio", "ollama"):
|
||||
# LM Studio and Ollama don't support json_object response format reliably
|
||||
call_params["response_format"] = {"type": "json_object"}
|
||||
|
||||
last_exception = None
|
||||
|
||||
for attempt in range(max_retries + 1):
|
||||
try:
|
||||
if response_format is not None:
|
||||
response = await self._client.chat.completions.create(**call_params)
|
||||
|
||||
content = response.choices[0].message.content
|
||||
|
||||
# Strip reasoning model thinking tags
|
||||
# Supports: <think>, <thinking>, <reasoning>, |startthink|/|endthink|
|
||||
if content:
|
||||
original_len = len(content)
|
||||
content = re.sub(r"<think>.*?</think>", "", content, flags=re.DOTALL)
|
||||
content = re.sub(r"<thinking>.*?</thinking>", "", content, flags=re.DOTALL)
|
||||
content = re.sub(r"<reasoning>.*?</reasoning>", "", content, flags=re.DOTALL)
|
||||
content = re.sub(r"\|startthink\|.*?\|endthink\|", "", content, flags=re.DOTALL)
|
||||
content = content.strip()
|
||||
if len(content) < original_len:
|
||||
logger.debug(f"Stripped {original_len - len(content)} chars of reasoning tokens")
|
||||
|
||||
# For local models, they may wrap JSON in markdown code blocks
|
||||
if self.provider in ("lmstudio", "ollama"):
|
||||
clean_content = content
|
||||
if "```json" in content:
|
||||
clean_content = content.split("```json")[1].split("```")[0].strip()
|
||||
elif "```" in content:
|
||||
clean_content = content.split("```")[1].split("```")[0].strip()
|
||||
try:
|
||||
json_data = json.loads(clean_content)
|
||||
except json.JSONDecodeError:
|
||||
# Fallback to parsing raw content
|
||||
json_data = json.loads(content)
|
||||
else:
|
||||
# Log raw LLM response for debugging JSON parse issues
|
||||
try:
|
||||
json_data = json.loads(content)
|
||||
except json.JSONDecodeError as json_err:
|
||||
# Truncate content for logging
|
||||
content_preview = content[:500] if content else "<empty>"
|
||||
if content and len(content) > 700:
|
||||
content_preview = f"{content[:500]}...TRUNCATED...{content[-200:]}"
|
||||
logger.warning(
|
||||
f"JSON parse error from LLM response (attempt {attempt + 1}/{max_retries + 1}): {json_err}\n"
|
||||
f" Model: {self.provider}/{self.model}\n"
|
||||
f" Content length: {len(content) if content else 0} chars\n"
|
||||
f" Content preview: {content_preview!r}\n"
|
||||
f" Finish reason: {response.choices[0].finish_reason if response.choices else 'unknown'}"
|
||||
)
|
||||
# Retry on JSON parse errors
|
||||
if attempt < max_retries:
|
||||
backoff = min(initial_backoff * (2**attempt), max_backoff)
|
||||
await asyncio.sleep(backoff)
|
||||
last_exception = json_err
|
||||
continue
|
||||
else:
|
||||
logger.error(f"JSON parse error after {max_retries + 1} attempts, giving up")
|
||||
raise
|
||||
|
||||
if skip_validation:
|
||||
result = json_data
|
||||
else:
|
||||
result = response_format.model_validate(json_data)
|
||||
else:
|
||||
response = await self._client.chat.completions.create(**call_params)
|
||||
result = response.choices[0].message.content
|
||||
|
||||
# Record token usage metrics
|
||||
duration = time.time() - start_time
|
||||
usage = response.usage
|
||||
input_tokens = usage.prompt_tokens or 0 if usage else 0
|
||||
output_tokens = usage.completion_tokens or 0 if usage else 0
|
||||
total_tokens = usage.total_tokens or 0 if usage else 0
|
||||
|
||||
# Record LLM metrics
|
||||
metrics = get_metrics_collector()
|
||||
metrics.record_llm_call(
|
||||
provider=self.provider,
|
||||
model=self.model,
|
||||
scope=scope,
|
||||
duration=duration,
|
||||
input_tokens=input_tokens,
|
||||
output_tokens=output_tokens,
|
||||
success=True,
|
||||
)
|
||||
|
||||
# Log slow calls
|
||||
if duration > 10.0 and usage:
|
||||
ratio = max(1, output_tokens) / max(1, input_tokens)
|
||||
cached_tokens = 0
|
||||
if hasattr(usage, "prompt_tokens_details") and usage.prompt_tokens_details:
|
||||
cached_tokens = getattr(usage.prompt_tokens_details, "cached_tokens", 0) or 0
|
||||
cache_info = f", cached_tokens={cached_tokens}" if cached_tokens > 0 else ""
|
||||
logger.info(
|
||||
f"slow llm call: scope={scope}, model={self.provider}/{self.model}, "
|
||||
f"input_tokens={input_tokens}, output_tokens={output_tokens}, "
|
||||
f"total_tokens={total_tokens}{cache_info}, time={duration:.3f}s, ratio out/in={ratio:.2f}"
|
||||
)
|
||||
|
||||
if return_usage:
|
||||
token_usage = TokenUsage(
|
||||
input_tokens=input_tokens,
|
||||
output_tokens=output_tokens,
|
||||
total_tokens=total_tokens,
|
||||
)
|
||||
return result, token_usage
|
||||
return result
|
||||
|
||||
except LengthFinishReasonError as e:
|
||||
logger.warning(f"LLM output exceeded token limits: {str(e)}")
|
||||
raise OutputTooLongError(
|
||||
"LLM output exceeded token limits. Input may need to be split into smaller chunks."
|
||||
) from e
|
||||
|
||||
except APIConnectionError as e:
|
||||
last_exception = e
|
||||
status_code = getattr(e, "status_code", None) or getattr(
|
||||
getattr(e, "response", None), "status_code", None
|
||||
)
|
||||
logger.warning(f"APIConnectionError (HTTP {status_code}), attempt {attempt + 1}: {str(e)[:200]}")
|
||||
if attempt < max_retries:
|
||||
backoff = min(initial_backoff * (2**attempt), max_backoff)
|
||||
await asyncio.sleep(backoff)
|
||||
continue
|
||||
else:
|
||||
logger.error(f"Connection error after {max_retries + 1} attempts: {str(e)}")
|
||||
raise
|
||||
|
||||
except APIStatusError as e:
|
||||
# Fast fail only on 401 (unauthorized) and 403 (forbidden)
|
||||
if e.status_code in (401, 403):
|
||||
logger.error(f"Auth error (HTTP {e.status_code}), not retrying: {str(e)}")
|
||||
raise
|
||||
|
||||
# Handle tool_use_failed error - model outputted in tool call format
|
||||
if e.status_code == 400 and response_format is not None:
|
||||
try:
|
||||
error_body = e.body if hasattr(e, "body") else {}
|
||||
if isinstance(error_body, dict):
|
||||
error_info: dict[str, Any] = error_body.get("error") or {}
|
||||
if error_info.get("code") == "tool_use_failed":
|
||||
failed_gen = error_info.get("failed_generation", "")
|
||||
if failed_gen:
|
||||
# Parse tool call format and convert to expected format
|
||||
tool_call = json.loads(failed_gen)
|
||||
tool_name = tool_call.get("name", "")
|
||||
tool_args = tool_call.get("arguments", {})
|
||||
converted = {"actions": [{"tool": tool_name, **tool_args}]}
|
||||
if skip_validation:
|
||||
result = converted
|
||||
else:
|
||||
result = response_format.model_validate(converted)
|
||||
|
||||
# Record metrics
|
||||
duration = time.time() - start_time
|
||||
metrics = get_metrics_collector()
|
||||
metrics.record_llm_call(
|
||||
provider=self.provider,
|
||||
model=self.model,
|
||||
scope=scope,
|
||||
duration=duration,
|
||||
input_tokens=0,
|
||||
output_tokens=0,
|
||||
success=True,
|
||||
)
|
||||
if return_usage:
|
||||
return result, TokenUsage(input_tokens=0, output_tokens=0, total_tokens=0)
|
||||
return result
|
||||
except (json.JSONDecodeError, KeyError, TypeError):
|
||||
pass # Failed to parse tool_use_failed, continue with normal retry
|
||||
|
||||
last_exception = e
|
||||
if attempt < max_retries:
|
||||
backoff = min(initial_backoff * (2**attempt), max_backoff)
|
||||
jitter = backoff * 0.2 * (2 * (time.time() % 1) - 1)
|
||||
sleep_time = backoff + jitter
|
||||
await asyncio.sleep(sleep_time)
|
||||
else:
|
||||
logger.error(f"API error after {max_retries + 1} attempts: {str(e)}")
|
||||
raise
|
||||
|
||||
except Exception:
|
||||
raise
|
||||
|
||||
if last_exception:
|
||||
raise last_exception
|
||||
raise RuntimeError("LLM call failed after all retries with no exception captured")
|
||||
|
||||
async def call_with_tools(
|
||||
self,
|
||||
messages: list[dict[str, Any]],
|
||||
tools: list[dict[str, Any]],
|
||||
max_completion_tokens: int | None = None,
|
||||
temperature: float | None = None,
|
||||
scope: str = "tools",
|
||||
max_retries: int = 5,
|
||||
initial_backoff: float = 1.0,
|
||||
max_backoff: float = 30.0,
|
||||
tool_choice: str | dict[str, Any] = "auto",
|
||||
) -> LLMToolCallResult:
|
||||
"""
|
||||
Make an LLM API call with tool/function calling support.
|
||||
|
||||
Args:
|
||||
messages: List of message dicts. Can include tool results with role='tool'.
|
||||
tools: List of tool definitions in OpenAI format.
|
||||
max_completion_tokens: Maximum tokens in response.
|
||||
temperature: Sampling temperature (0.0-2.0).
|
||||
scope: Scope identifier for tracking.
|
||||
max_retries: Maximum retry attempts.
|
||||
initial_backoff: Initial backoff time in seconds.
|
||||
max_backoff: Maximum backoff time in seconds.
|
||||
tool_choice: How to choose tools - "auto", "none", "required", or specific function.
|
||||
|
||||
Returns:
|
||||
LLMToolCallResult with content and/or tool_calls.
|
||||
"""
|
||||
start_time = time.time()
|
||||
|
||||
# Build call parameters
|
||||
call_params: dict[str, Any] = {
|
||||
"model": self.model,
|
||||
"messages": messages,
|
||||
"tools": tools,
|
||||
"tool_choice": tool_choice,
|
||||
}
|
||||
|
||||
if max_completion_tokens is not None:
|
||||
call_params["max_completion_tokens"] = max_completion_tokens
|
||||
if temperature is not None:
|
||||
call_params["temperature"] = temperature
|
||||
|
||||
# Provider-specific parameters
|
||||
if self.provider == "groq":
|
||||
call_params["seed"] = DEFAULT_LLM_SEED
|
||||
|
||||
last_exception = None
|
||||
|
||||
for attempt in range(max_retries + 1):
|
||||
try:
|
||||
response = await self._client.chat.completions.create(**call_params)
|
||||
|
||||
message = response.choices[0].message
|
||||
finish_reason = response.choices[0].finish_reason
|
||||
|
||||
# Extract tool calls if present
|
||||
tool_calls: list[LLMToolCall] = []
|
||||
if message.tool_calls:
|
||||
for tc in message.tool_calls:
|
||||
try:
|
||||
args = json.loads(tc.function.arguments) if tc.function.arguments else {}
|
||||
except json.JSONDecodeError:
|
||||
args = {"_raw": tc.function.arguments}
|
||||
tool_calls.append(LLMToolCall(id=tc.id, name=tc.function.name, arguments=args))
|
||||
|
||||
content = message.content
|
||||
|
||||
# Record metrics
|
||||
duration = time.time() - start_time
|
||||
usage = response.usage
|
||||
input_tokens = usage.prompt_tokens or 0 if usage else 0
|
||||
output_tokens = usage.completion_tokens or 0 if usage else 0
|
||||
|
||||
metrics = get_metrics_collector()
|
||||
metrics.record_llm_call(
|
||||
provider=self.provider,
|
||||
model=self.model,
|
||||
scope=scope,
|
||||
duration=duration,
|
||||
input_tokens=input_tokens,
|
||||
output_tokens=output_tokens,
|
||||
success=True,
|
||||
)
|
||||
|
||||
return LLMToolCallResult(
|
||||
content=content,
|
||||
tool_calls=tool_calls,
|
||||
finish_reason=finish_reason,
|
||||
input_tokens=input_tokens,
|
||||
output_tokens=output_tokens,
|
||||
)
|
||||
|
||||
except APIConnectionError as e:
|
||||
last_exception = e
|
||||
if attempt < max_retries:
|
||||
await asyncio.sleep(min(initial_backoff * (2**attempt), max_backoff))
|
||||
continue
|
||||
raise
|
||||
|
||||
except APIStatusError as e:
|
||||
if e.status_code in (401, 403):
|
||||
raise
|
||||
last_exception = e
|
||||
if attempt < max_retries:
|
||||
await asyncio.sleep(min(initial_backoff * (2**attempt), max_backoff))
|
||||
continue
|
||||
raise
|
||||
|
||||
except Exception:
|
||||
raise
|
||||
|
||||
if last_exception:
|
||||
raise last_exception
|
||||
raise RuntimeError("Tool call failed after all retries")
|
||||
|
||||
async def _call_ollama_native(
|
||||
self,
|
||||
messages: list[dict[str, str]],
|
||||
response_format: Any,
|
||||
max_completion_tokens: int | None,
|
||||
temperature: float | None,
|
||||
max_retries: int,
|
||||
initial_backoff: float,
|
||||
max_backoff: float,
|
||||
skip_validation: bool,
|
||||
scope: str = "memory",
|
||||
return_usage: bool = False,
|
||||
) -> Any:
|
||||
"""
|
||||
Call Ollama using native API with JSON schema enforcement.
|
||||
|
||||
Ollama's native API supports passing a full JSON schema in the 'format' parameter,
|
||||
which provides better structured output control than the OpenAI-compatible API.
|
||||
"""
|
||||
start_time = time.time()
|
||||
|
||||
# Get the JSON schema from the Pydantic model
|
||||
schema = response_format.model_json_schema() if hasattr(response_format, "model_json_schema") else None
|
||||
|
||||
# Build the base URL for Ollama's native API
|
||||
# Default OpenAI-compatible URL is http://localhost:11434/v1
|
||||
# Native API is at http://localhost:11434/api/chat
|
||||
base_url = self.base_url or "http://localhost:11434/v1"
|
||||
if base_url.endswith("/v1"):
|
||||
native_url = base_url[:-3] + "/api/chat"
|
||||
else:
|
||||
native_url = base_url.rstrip("/") + "/api/chat"
|
||||
|
||||
# Build request payload
|
||||
payload: dict[str, Any] = {
|
||||
"model": self.model,
|
||||
"messages": messages,
|
||||
"stream": False,
|
||||
}
|
||||
|
||||
# Add schema as format parameter for structured output
|
||||
if schema:
|
||||
payload["format"] = schema
|
||||
|
||||
# Add optional parameters with optimized defaults for Ollama
|
||||
options: dict[str, Any] = {
|
||||
"num_ctx": 16384, # 16k context window for larger prompts
|
||||
"num_batch": 512, # Optimal batch size for prompt processing
|
||||
}
|
||||
if max_completion_tokens:
|
||||
options["num_predict"] = max_completion_tokens
|
||||
if temperature is not None:
|
||||
options["temperature"] = temperature
|
||||
payload["options"] = options
|
||||
|
||||
last_exception = None
|
||||
|
||||
async with httpx.AsyncClient(timeout=300.0) as client:
|
||||
for attempt in range(max_retries + 1):
|
||||
try:
|
||||
response = await client.post(native_url, json=payload)
|
||||
response.raise_for_status()
|
||||
|
||||
result = response.json()
|
||||
content = result.get("message", {}).get("content", "")
|
||||
|
||||
# Parse JSON response
|
||||
try:
|
||||
json_data = json.loads(content)
|
||||
except json.JSONDecodeError as json_err:
|
||||
content_preview = content[:500] if content else "<empty>"
|
||||
if content and len(content) > 700:
|
||||
content_preview = f"{content[:500]}...TRUNCATED...{content[-200:]}"
|
||||
logger.warning(
|
||||
f"Ollama JSON parse error (attempt {attempt + 1}/{max_retries + 1}): {json_err}\n"
|
||||
f" Model: ollama/{self.model}\n"
|
||||
f" Content length: {len(content) if content else 0} chars\n"
|
||||
f" Content preview: {content_preview!r}"
|
||||
)
|
||||
if attempt < max_retries:
|
||||
backoff = min(initial_backoff * (2**attempt), max_backoff)
|
||||
await asyncio.sleep(backoff)
|
||||
last_exception = json_err
|
||||
continue
|
||||
else:
|
||||
raise
|
||||
|
||||
# Extract token usage from Ollama response
|
||||
duration = time.time() - start_time
|
||||
input_tokens = result.get("prompt_eval_count", 0) or 0
|
||||
output_tokens = result.get("eval_count", 0) or 0
|
||||
total_tokens = input_tokens + output_tokens
|
||||
|
||||
# Record LLM metrics
|
||||
metrics = get_metrics_collector()
|
||||
metrics.record_llm_call(
|
||||
provider=self.provider,
|
||||
model=self.model,
|
||||
scope=scope,
|
||||
duration=duration,
|
||||
input_tokens=input_tokens,
|
||||
output_tokens=output_tokens,
|
||||
success=True,
|
||||
)
|
||||
|
||||
# Validate against Pydantic model or return raw JSON
|
||||
if skip_validation:
|
||||
validated_result = json_data
|
||||
else:
|
||||
validated_result = response_format.model_validate(json_data)
|
||||
|
||||
if return_usage:
|
||||
token_usage = TokenUsage(
|
||||
input_tokens=input_tokens,
|
||||
output_tokens=output_tokens,
|
||||
total_tokens=total_tokens,
|
||||
)
|
||||
return validated_result, token_usage
|
||||
return validated_result
|
||||
|
||||
except httpx.HTTPStatusError as e:
|
||||
last_exception = e
|
||||
if attempt < max_retries:
|
||||
logger.warning(
|
||||
f"Ollama HTTP error (attempt {attempt + 1}/{max_retries + 1}): {e.response.status_code}"
|
||||
)
|
||||
backoff = min(initial_backoff * (2**attempt), max_backoff)
|
||||
await asyncio.sleep(backoff)
|
||||
continue
|
||||
else:
|
||||
logger.error(f"Ollama HTTP error after {max_retries + 1} attempts: {e}")
|
||||
raise
|
||||
|
||||
except httpx.RequestError as e:
|
||||
last_exception = e
|
||||
if attempt < max_retries:
|
||||
logger.warning(f"Ollama connection error (attempt {attempt + 1}/{max_retries + 1}): {e}")
|
||||
backoff = min(initial_backoff * (2**attempt), max_backoff)
|
||||
await asyncio.sleep(backoff)
|
||||
continue
|
||||
else:
|
||||
logger.error(f"Ollama connection error after {max_retries + 1} attempts: {e}")
|
||||
raise
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Unexpected error during Ollama call: {type(e).__name__}: {e}")
|
||||
raise
|
||||
|
||||
if last_exception:
|
||||
raise last_exception
|
||||
raise RuntimeError("Ollama call failed after all retries")
|
||||
|
||||
async def cleanup(self) -> None:
|
||||
"""Clean up resources (close OpenAI client connections)."""
|
||||
if hasattr(self, "_client") and self._client:
|
||||
await self._client.close()
|
||||
@@ -57,25 +57,21 @@ def _infer_temporal_date(fact_text: str, event_date: datetime) -> str | None:
|
||||
return None
|
||||
|
||||
|
||||
def _sanitize_text(text: str | None) -> str | None:
|
||||
def _sanitize_text(text: str) -> str:
|
||||
"""
|
||||
Sanitize text by removing characters that break downstream systems.
|
||||
Sanitize text by removing invalid Unicode surrogate characters.
|
||||
|
||||
Removes:
|
||||
- Null bytes (\\x00): Invalid in PostgreSQL UTF-8 encoding
|
||||
- Unicode surrogates (U+D800-U+DFFF): Invalid in UTF-8, break LLM APIs
|
||||
Surrogate characters (U+D800 to U+DFFF) are used in UTF-16 encoding
|
||||
but cannot be encoded in UTF-8. They can appear in Python strings
|
||||
from improperly decoded data (e.g., from JavaScript or broken files).
|
||||
|
||||
Surrogate characters are used in UTF-16 encoding but cannot be encoded
|
||||
in UTF-8. They can appear in Python strings from improperly decoded data
|
||||
(e.g., from JavaScript or broken files). Null bytes commonly appear in
|
||||
OCR output, PDF extraction, or copy-paste from binary sources.
|
||||
This function removes unpaired surrogates to prevent UnicodeEncodeError
|
||||
when the text is sent to the LLM API.
|
||||
"""
|
||||
if text is None:
|
||||
return None
|
||||
if not text:
|
||||
return text
|
||||
# Remove null bytes and surrogate characters
|
||||
text = text.replace("\x00", "")
|
||||
# Remove surrogate characters (U+D800 to U+DFFF) using regex
|
||||
# These are invalid in UTF-8 and cause encoding errors
|
||||
return re.sub(r"[\ud800-\udfff]", "", text)
|
||||
|
||||
|
||||
@@ -763,10 +759,6 @@ async def _extract_facts_from_chunk(
|
||||
|
||||
# Build user message with metadata and chunk content in a clear format
|
||||
# Format event_date with day of week for better temporal reasoning
|
||||
# Handle both datetime objects and ISO string formats (from deserialized async tasks)
|
||||
from .orchestrator import parse_datetime_flexible
|
||||
|
||||
event_date = parse_datetime_flexible(event_date)
|
||||
event_date_formatted = event_date.strftime("%A, %B %d, %Y") # e.g., "Monday, June 10, 2024"
|
||||
user_message = f"""Extract facts from the following text chunk.
|
||||
|
||||
@@ -1354,8 +1346,6 @@ def _add_temporal_offsets(facts: list[ExtractedFactType], contents: list[RetainC
|
||||
|
||||
Modifies facts in place.
|
||||
"""
|
||||
from .orchestrator import parse_datetime_flexible
|
||||
|
||||
# Group facts by content_index
|
||||
current_content_idx = 0
|
||||
content_fact_start = 0
|
||||
@@ -1370,10 +1360,10 @@ def _add_temporal_offsets(facts: list[ExtractedFactType], contents: list[RetainC
|
||||
fact_position = i - content_fact_start
|
||||
offset = timedelta(seconds=fact_position * SECONDS_PER_FACT)
|
||||
|
||||
# Apply offset to all temporal fields (handle both datetime objects and ISO strings)
|
||||
# Apply offset to all temporal fields
|
||||
if fact.occurred_start:
|
||||
fact.occurred_start = parse_datetime_flexible(fact.occurred_start) + offset
|
||||
fact.occurred_start = fact.occurred_start + offset
|
||||
if fact.occurred_end:
|
||||
fact.occurred_end = parse_datetime_flexible(fact.occurred_end) + offset
|
||||
fact.occurred_end = fact.occurred_end + offset
|
||||
if fact.mentioned_at:
|
||||
fact.mentioned_at = parse_datetime_flexible(fact.mentioned_at) + offset
|
||||
fact.mentioned_at = fact.mentioned_at + offset
|
||||
|
||||
@@ -8,7 +8,6 @@ import json
|
||||
import logging
|
||||
|
||||
from ..memory_engine import fq_table
|
||||
from .fact_extraction import _sanitize_text
|
||||
from .types import ProcessedFact
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -48,7 +47,7 @@ async def insert_facts_batch(
|
||||
tags_list = []
|
||||
|
||||
for fact in facts:
|
||||
fact_texts.append(_sanitize_text(fact.fact_text))
|
||||
fact_texts.append(fact.fact_text)
|
||||
# Convert embedding to string for asyncpg vector type
|
||||
embeddings.append(str(fact.embedding))
|
||||
# event_date: Use occurred_start if available, otherwise use mentioned_at
|
||||
@@ -57,7 +56,7 @@ async def insert_facts_batch(
|
||||
occurred_starts.append(fact.occurred_start)
|
||||
occurred_ends.append(fact.occurred_end)
|
||||
mentioned_ats.append(fact.mentioned_at)
|
||||
contexts.append(_sanitize_text(fact.context))
|
||||
contexts.append(fact.context)
|
||||
fact_types.append(fact.fact_type)
|
||||
# confidence_score is only for opinion facts
|
||||
confidence_scores.append(1.0 if fact.fact_type == "opinion" else None)
|
||||
@@ -158,8 +157,7 @@ async def handle_document_tracking(
|
||||
"""
|
||||
import hashlib
|
||||
|
||||
# Sanitize and calculate content hash
|
||||
combined_content = _sanitize_text(combined_content) or ""
|
||||
# Calculate content hash
|
||||
content_hash = hashlib.sha256(combined_content.encode()).hexdigest()
|
||||
|
||||
# Always delete old document first if it exists (cascades to units and links)
|
||||
|
||||
@@ -8,7 +8,6 @@ import logging
|
||||
import time
|
||||
import uuid
|
||||
from datetime import UTC, datetime
|
||||
from typing import Any
|
||||
|
||||
from ..db_utils import acquire_with_retry
|
||||
from . import bank_utils
|
||||
@@ -19,39 +18,6 @@ def utcnow():
|
||||
return datetime.now(UTC)
|
||||
|
||||
|
||||
def parse_datetime_flexible(value: Any) -> datetime:
|
||||
"""
|
||||
Parse a datetime value that could be either a datetime object or an ISO string.
|
||||
|
||||
This handles datetime values from both direct Python calls and deserialized JSON
|
||||
(where datetime objects are serialized as ISO strings).
|
||||
|
||||
Args:
|
||||
value: Either a datetime object or an ISO format string
|
||||
|
||||
Returns:
|
||||
datetime object (timezone-aware)
|
||||
|
||||
Raises:
|
||||
TypeError: If value is neither datetime nor string
|
||||
ValueError: If string is not a valid ISO datetime
|
||||
"""
|
||||
if isinstance(value, datetime):
|
||||
# Ensure timezone-aware
|
||||
if value.tzinfo is None:
|
||||
return value.replace(tzinfo=UTC)
|
||||
return value
|
||||
elif isinstance(value, str):
|
||||
# Parse ISO format string (handles both 'Z' and '+00:00' timezone formats)
|
||||
dt = datetime.fromisoformat(value.replace("Z", "+00:00"))
|
||||
# Ensure timezone-aware
|
||||
if dt.tzinfo is None:
|
||||
return dt.replace(tzinfo=UTC)
|
||||
return dt
|
||||
else:
|
||||
raise TypeError(f"Expected datetime or string, got {type(value).__name__}")
|
||||
|
||||
|
||||
from ..response_models import TokenUsage
|
||||
from . import (
|
||||
chunk_storage,
|
||||
@@ -123,18 +89,10 @@ async def retain_batch(
|
||||
# Merge item-level tags with document-level tags
|
||||
item_tags = item.get("tags", []) or []
|
||||
merged_tags = list(set(item_tags + (document_tags or [])))
|
||||
|
||||
# Handle event_date: parse flexibly (handles both datetime objects and ISO strings)
|
||||
event_date_value = item.get("event_date")
|
||||
if event_date_value:
|
||||
event_date_value = parse_datetime_flexible(event_date_value)
|
||||
else:
|
||||
event_date_value = utcnow()
|
||||
|
||||
content = RetainContent(
|
||||
content=item["content"],
|
||||
context=item.get("context", ""),
|
||||
event_date=event_date_value,
|
||||
event_date=item.get("event_date") or utcnow(),
|
||||
metadata=item.get("metadata", {}),
|
||||
entities=item.get("entities", []),
|
||||
tags=merged_tags,
|
||||
|
||||
@@ -182,16 +182,7 @@ class BrokerTaskBackend(TaskBackend):
|
||||
operation_id = task_dict.get("operation_id")
|
||||
task_type = task_dict.get("type", "unknown")
|
||||
bank_id = task_dict.get("bank_id")
|
||||
|
||||
# Custom encoder to handle datetime objects
|
||||
from datetime import datetime
|
||||
|
||||
def datetime_encoder(obj):
|
||||
if isinstance(obj, datetime):
|
||||
return obj.isoformat()
|
||||
raise TypeError(f"Object of type {type(obj).__name__} is not JSON serializable")
|
||||
|
||||
payload_json = json.dumps(task_dict, default=datetime_encoder)
|
||||
payload_json = json.dumps(task_dict)
|
||||
|
||||
schema = self._schema_getter() if self._schema_getter else self._schema
|
||||
table = fq_table("async_operations", schema)
|
||||
|
||||
@@ -24,11 +24,6 @@ from hindsight_api.extensions.operation_validator import (
|
||||
# Consolidation operation
|
||||
ConsolidateContext,
|
||||
ConsolidateResult,
|
||||
# Mental Model operations
|
||||
MentalModelGetContext,
|
||||
MentalModelGetResult,
|
||||
MentalModelRefreshContext,
|
||||
MentalModelRefreshResult,
|
||||
# Core operations
|
||||
OperationValidationError,
|
||||
OperationValidatorExtension,
|
||||
@@ -70,11 +65,6 @@ __all__ = [
|
||||
# Operation Validator - Consolidation
|
||||
"ConsolidateContext",
|
||||
"ConsolidateResult",
|
||||
# Operation Validator - Mental Model
|
||||
"MentalModelGetContext",
|
||||
"MentalModelGetResult",
|
||||
"MentalModelRefreshContext",
|
||||
"MentalModelRefreshResult",
|
||||
# Tenant/Auth
|
||||
"ApiKeyTenantExtension",
|
||||
"AuthenticationError",
|
||||
|
||||
@@ -196,57 +196,6 @@ class ConsolidateResult:
|
||||
error: str | None = None
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Mental Model Contexts
|
||||
# =============================================================================
|
||||
|
||||
|
||||
@dataclass
|
||||
class MentalModelGetContext:
|
||||
"""Context for a mental model GET operation validation (pre-operation)."""
|
||||
|
||||
bank_id: str
|
||||
mental_model_id: str
|
||||
request_context: "RequestContext"
|
||||
|
||||
|
||||
@dataclass
|
||||
class MentalModelRefreshContext:
|
||||
"""Context for a mental model refresh/create operation validation (pre-operation)."""
|
||||
|
||||
bank_id: str
|
||||
mental_model_id: str | None # None for create (not yet assigned)
|
||||
request_context: "RequestContext"
|
||||
|
||||
|
||||
@dataclass
|
||||
class MentalModelGetResult:
|
||||
"""Result context for post-mental-model-GET hook."""
|
||||
|
||||
bank_id: str
|
||||
mental_model_id: str
|
||||
request_context: "RequestContext"
|
||||
output_tokens: int # tokens in the returned content
|
||||
success: bool = True
|
||||
error: str | None = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class MentalModelRefreshResult:
|
||||
"""Result context for post-mental-model-refresh hook."""
|
||||
|
||||
bank_id: str
|
||||
mental_model_id: str
|
||||
request_context: "RequestContext"
|
||||
query_tokens: int # tokens in source_query
|
||||
output_tokens: int # tokens in generated content
|
||||
context_tokens: int # tokens in context (if any)
|
||||
facts_used: int # facts referenced in based_on
|
||||
mental_models_used: int # mental models referenced in based_on
|
||||
success: bool = True
|
||||
error: str | None = None
|
||||
|
||||
|
||||
class OperationValidatorExtension(Extension, ABC):
|
||||
"""
|
||||
Validates and hooks into retain/recall/reflect/consolidate operations.
|
||||
@@ -453,81 +402,3 @@ class OperationValidatorExtension(Extension, ABC):
|
||||
- error: Error message (if failed)
|
||||
"""
|
||||
pass
|
||||
|
||||
# =========================================================================
|
||||
# Mental Model - Pre-operation validation hook (optional - override to implement)
|
||||
# =========================================================================
|
||||
|
||||
async def validate_mental_model_get(self, ctx: MentalModelGetContext) -> ValidationResult:
|
||||
"""
|
||||
Validate a mental model GET operation before execution.
|
||||
|
||||
Override to implement custom validation logic for mental model retrieval.
|
||||
|
||||
Args:
|
||||
ctx: Context containing:
|
||||
- bank_id: Bank identifier
|
||||
- mental_model_id: Mental model identifier
|
||||
- request_context: Request context with auth info
|
||||
|
||||
Returns:
|
||||
ValidationResult indicating whether the operation is allowed.
|
||||
"""
|
||||
return ValidationResult.accept()
|
||||
|
||||
async def validate_mental_model_refresh(self, ctx: MentalModelRefreshContext) -> ValidationResult:
|
||||
"""
|
||||
Validate a mental model refresh/create operation before execution.
|
||||
|
||||
Override to implement custom validation logic for mental model refresh.
|
||||
|
||||
Args:
|
||||
ctx: Context containing:
|
||||
- bank_id: Bank identifier
|
||||
- mental_model_id: Mental model identifier (None for create)
|
||||
- request_context: Request context with auth info
|
||||
|
||||
Returns:
|
||||
ValidationResult indicating whether the operation is allowed.
|
||||
"""
|
||||
return ValidationResult.accept()
|
||||
|
||||
# =========================================================================
|
||||
# Mental Model - Post-operation hooks (optional - override to implement)
|
||||
# =========================================================================
|
||||
|
||||
async def on_mental_model_get_complete(self, result: MentalModelGetResult) -> None:
|
||||
"""
|
||||
Called after a mental model GET operation completes (success or failure).
|
||||
|
||||
Override to implement post-operation logic such as tracking or audit logging.
|
||||
|
||||
Args:
|
||||
result: Result context containing:
|
||||
- bank_id: Bank identifier
|
||||
- mental_model_id: Mental model identifier
|
||||
- output_tokens: Token count of the returned content
|
||||
- success: Whether the operation succeeded
|
||||
- error: Error message (if failed)
|
||||
"""
|
||||
pass
|
||||
|
||||
async def on_mental_model_refresh_complete(self, result: MentalModelRefreshResult) -> None:
|
||||
"""
|
||||
Called after a mental model refresh operation completes (success or failure).
|
||||
|
||||
Override to implement post-operation logic such as tracking or audit logging.
|
||||
|
||||
Args:
|
||||
result: Result context containing:
|
||||
- bank_id: Bank identifier
|
||||
- mental_model_id: Mental model identifier
|
||||
- query_tokens: Tokens in source_query
|
||||
- output_tokens: Tokens in generated content
|
||||
- context_tokens: Tokens in context
|
||||
- facts_used: Number of facts referenced
|
||||
- mental_models_used: Number of mental models referenced
|
||||
- success: Whether the operation succeeded
|
||||
- error: Error message (if failed)
|
||||
"""
|
||||
pass
|
||||
|
||||
@@ -20,7 +20,7 @@ import warnings
|
||||
|
||||
import uvicorn
|
||||
|
||||
from . import MemoryEngine, __version__
|
||||
from . import MemoryEngine
|
||||
from .api import create_app
|
||||
from .banner import print_banner
|
||||
from .config import DEFAULT_WORKERS, ENV_WORKERS, HindsightConfig, get_config
|
||||
@@ -362,7 +362,6 @@ def main():
|
||||
embeddings_provider=config.embeddings_provider,
|
||||
reranker_provider=config.reranker_provider,
|
||||
mcp_enabled=config.mcp_enabled,
|
||||
version=__version__,
|
||||
)
|
||||
|
||||
# Start idle checker in daemon mode
|
||||
|
||||
@@ -200,20 +200,15 @@ def main():
|
||||
if tenant_extension:
|
||||
print("Tenant extension loaded - schemas will be discovered dynamically on each poll")
|
||||
else:
|
||||
print(f"No tenant extension configured, using schema: {config.database_schema}")
|
||||
print("No tenant extension configured, using public schema only")
|
||||
|
||||
# Create a single poller that handles all schemas dynamically
|
||||
# Convert default schema to None for SQL compatibility (no schema prefix)
|
||||
from hindsight_api.config import DEFAULT_DATABASE_SCHEMA
|
||||
|
||||
schema = None if config.database_schema == DEFAULT_DATABASE_SCHEMA else config.database_schema
|
||||
poller = WorkerPoller(
|
||||
pool=memory._pool,
|
||||
worker_id=args.worker_id,
|
||||
executor=memory.execute_task,
|
||||
poll_interval_ms=args.poll_interval,
|
||||
max_retries=args.max_retries,
|
||||
schema=schema,
|
||||
tenant_extension=tenant_extension,
|
||||
max_slots=config.worker_max_slots,
|
||||
consolidation_max_slots=config.worker_consolidation_max_slots,
|
||||
|
||||
@@ -99,13 +99,11 @@ class WorkerPoller:
|
||||
self._in_flight_by_type: dict[str, int] = {}
|
||||
|
||||
async def _get_schemas(self) -> list[str | None]:
|
||||
"""Get list of schemas to poll. Returns [None] for default schema (no prefix)."""
|
||||
"""Get list of schemas to poll. Returns [None] for public schema."""
|
||||
if self._tenant_extension is not None:
|
||||
from ..config import DEFAULT_DATABASE_SCHEMA
|
||||
|
||||
tenants = await self._tenant_extension.list_tenants()
|
||||
# Convert default schema to None for SQL compatibility (no prefix), keep others as-is
|
||||
return [t.schema if t.schema != DEFAULT_DATABASE_SCHEMA else None for t in tenants]
|
||||
# Convert "public" to None for SQL compatibility, keep others as-is
|
||||
return [t.schema if t.schema != "public" else None for t in tenants]
|
||||
# Single schema mode
|
||||
return [self._schema]
|
||||
|
||||
@@ -196,9 +194,7 @@ class WorkerPoller:
|
||||
try:
|
||||
return await self._claim_batch_for_schema_inner(schema, limit, consolidation_limit)
|
||||
except Exception as e:
|
||||
# Format schema for logging: custom schemas in quotes, None as-is
|
||||
schema_display = f'"{schema}"' if schema else str(schema)
|
||||
logger.warning(f"Worker {self._worker_id} failed to claim tasks for schema {schema_display}: {e}")
|
||||
logger.warning(f"Worker {self._worker_id} failed to claim tasks for schema {schema or 'public'}: {e}")
|
||||
return []
|
||||
|
||||
async def _claim_batch_for_schema_inner(
|
||||
@@ -422,9 +418,7 @@ class WorkerPoller:
|
||||
count = int(result.split()[-1]) if result else 0
|
||||
total_count += count
|
||||
except Exception as e:
|
||||
# Format schema for logging: custom schemas in quotes, None as-is
|
||||
schema_display = f'"{schema}"' if schema else str(schema)
|
||||
logger.warning(f"Worker {self._worker_id} failed to recover tasks for schema {schema_display}: {e}")
|
||||
logger.warning(f"Worker {self._worker_id} failed to recover tasks for schema {schema or 'public'}: {e}")
|
||||
|
||||
if total_count > 0:
|
||||
logger.info(f"Worker {self._worker_id} recovered {total_count} stale tasks from previous run")
|
||||
@@ -463,8 +457,7 @@ class WorkerPoller:
|
||||
consolidation_count += 1
|
||||
|
||||
types_str = ", ".join(f"{k}:{v}" for k, v in task_types.items())
|
||||
# Display None as "default" in logs
|
||||
schemas_str = ", ".join(s if s else "default" for s in schemas_seen)
|
||||
schemas_str = ", ".join(s or "public" for s in schemas_seen)
|
||||
logger.info(
|
||||
f"Worker {self._worker_id} claimed {len(tasks)} tasks "
|
||||
f"({consolidation_count} consolidation): {types_str} (schemas: {schemas_str})"
|
||||
@@ -598,8 +591,7 @@ class WorkerPoller:
|
||||
other_workers.append(f"{wid}:{cnt}")
|
||||
others_str = ", ".join(other_workers) if other_workers else "none"
|
||||
|
||||
# Display None as "default" in logs
|
||||
schemas_str = ", ".join(s if s else "default" for s in schemas)
|
||||
schemas_str = ", ".join(s or "public" for s in schemas)
|
||||
logger.info(
|
||||
f"[WORKER_STATS] worker={self._worker_id} "
|
||||
f"slots={in_flight}/{self._max_slots} (consolidation={consolidation_count}/{self._consolidation_max_slots}) | "
|
||||
|
||||
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
||||
|
||||
[project]
|
||||
name = "hindsight-api"
|
||||
version = "0.4.7"
|
||||
version = "0.4.2"
|
||||
description = "Hindsight: Agent Memory That Works Like Human Memory"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.11"
|
||||
@@ -25,7 +25,7 @@ dependencies = [
|
||||
"psycopg2-binary>=2.9.11",
|
||||
"tiktoken>=0.12.0",
|
||||
"httpx>=0.27.0",
|
||||
"fastmcp>=2.14.0", # CVE-2025-66416
|
||||
"fastmcp>=2.14.0", # CVE-2025-66416
|
||||
"pg0-embedded>=0.11.0",
|
||||
"python-dateutil>=2.8.0",
|
||||
"opentelemetry-api>=1.20.0",
|
||||
@@ -41,17 +41,16 @@ dependencies = [
|
||||
"flashrank>=0.2.0",
|
||||
# Local ML models for embeddings/reranking - can be excluded in Docker with INCLUDE_LOCAL_MODELS=false
|
||||
"sentence-transformers>=3.3.0",
|
||||
"transformers>=4.53.0", # Security fixes for ReDoS vulnerabilities
|
||||
"torch>=2.6.0", # CVE fix for remote code execution
|
||||
"transformers>=4.53.0", # Security fixes for ReDoS vulnerabilities
|
||||
"torch>=2.6.0", # CVE fix for remote code execution
|
||||
"uvloop>=0.22.1",
|
||||
# Transitive dependency security fixes
|
||||
"pyasn1>=0.6.2", # DoS vulnerability fix
|
||||
"urllib3>=2.6.3", # Decompression-bomb safeguards bypass fix
|
||||
"langchain-core>=1.2.5", # Serialization injection vulnerability fix
|
||||
"filelock>=3.20.1", # TOCTOU race condition fix
|
||||
"authlib>=1.6.6", # Account takeover vulnerability fix
|
||||
"aiohttp>=3.13.3", # Multiple DoS vulnerabilities
|
||||
"claude-agent-sdk>=0.1.27",
|
||||
"pyasn1>=0.6.2", # DoS vulnerability fix
|
||||
"urllib3>=2.6.3", # Decompression-bomb safeguards bypass fix
|
||||
"langchain-core>=1.2.5", # Serialization injection vulnerability fix
|
||||
"filelock>=3.20.1", # TOCTOU race condition fix
|
||||
"authlib>=1.6.6", # Account takeover vulnerability fix
|
||||
"aiohttp>=3.13.3", # Multiple DoS vulnerabilities
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
|
||||
@@ -1098,134 +1098,3 @@ async def test_version_endpoint_returns_correct_version(api_client):
|
||||
assert isinstance(features["worker"], bool)
|
||||
|
||||
print(f"Version endpoint returned: api_version={result['api_version']}, features={features}")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_retain_with_timestamp_async(api_client, test_bank_id):
|
||||
"""Test that async retain accepts timestamp field and serializes correctly."""
|
||||
response = await api_client.post(
|
||||
f"/v1/default/banks/{test_bank_id}/memories",
|
||||
json={
|
||||
"items": [
|
||||
{
|
||||
"content": "Test memory with timestamp",
|
||||
"context": "test",
|
||||
"timestamp": "2026-01-30T11:45:00Z"
|
||||
}
|
||||
],
|
||||
"async": True
|
||||
}
|
||||
)
|
||||
|
||||
assert response.status_code == 200, f"Expected 200, got {response.status_code}: {response.text}"
|
||||
data = response.json()
|
||||
assert data["success"] is True
|
||||
assert data["async"] is True
|
||||
assert "operation_id" in data
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_retain_with_timestamp_sync(api_client, test_bank_id):
|
||||
"""Test that sync retain accepts timestamp field."""
|
||||
response = await api_client.post(
|
||||
f"/v1/default/banks/{test_bank_id}/memories",
|
||||
json={
|
||||
"items": [
|
||||
{
|
||||
"content": "Test memory with timestamp sync",
|
||||
"context": "test",
|
||||
"timestamp": "2026-01-30T11:45:00Z"
|
||||
}
|
||||
],
|
||||
"async": False
|
||||
}
|
||||
)
|
||||
|
||||
assert response.status_code == 200, f"Expected 200, got {response.status_code}: {response.text}"
|
||||
data = response.json()
|
||||
assert data["success"] is True
|
||||
assert data["async"] is False
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_retain_with_multiple_timestamps(api_client, test_bank_id):
|
||||
"""Test that multiple items with different timestamp formats work."""
|
||||
response = await api_client.post(
|
||||
f"/v1/default/banks/{test_bank_id}/memories",
|
||||
json={
|
||||
"items": [
|
||||
{
|
||||
"content": "Event 1",
|
||||
"timestamp": "2026-01-30T11:45:00Z" # With Z
|
||||
},
|
||||
{
|
||||
"content": "Event 2",
|
||||
"timestamp": "2026-01-30T12:00:00+00:00" # With timezone
|
||||
},
|
||||
{
|
||||
"content": "Event 3" # No timestamp
|
||||
}
|
||||
],
|
||||
"async": True
|
||||
}
|
||||
)
|
||||
|
||||
assert response.status_code == 200, f"Expected 200, got {response.status_code}: {response.text}"
|
||||
data = response.json()
|
||||
assert data["success"] is True
|
||||
assert data["items_count"] == 3
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_retain_with_timestamp_async_complete_processing(api_client, test_bank_id):
|
||||
"""Test that async retain with timestamp completes full processing including fact extraction."""
|
||||
# Submit async retain with timestamp
|
||||
response = await api_client.post(
|
||||
f"/v1/default/banks/{test_bank_id}/memories",
|
||||
json={
|
||||
"items": [
|
||||
{
|
||||
"content": "The quarterly meeting was held on January 30th 2026",
|
||||
"context": "meetings",
|
||||
"timestamp": "2026-01-30T11:45:00Z"
|
||||
}
|
||||
],
|
||||
"async": True
|
||||
}
|
||||
)
|
||||
|
||||
assert response.status_code == 200, f"Expected 200, got {response.status_code}: {response.text}"
|
||||
data = response.json()
|
||||
assert data["success"] is True
|
||||
assert data["async"] is True
|
||||
operation_id = data["operation_id"]
|
||||
|
||||
# Wait for async processing to complete (poll operation status)
|
||||
max_wait_seconds = 30
|
||||
poll_interval = 0.5
|
||||
elapsed = 0
|
||||
operation_completed = False
|
||||
|
||||
while elapsed < max_wait_seconds:
|
||||
response = await api_client.get(f"/v1/default/banks/{test_bank_id}/operations/{operation_id}")
|
||||
if response.status_code == 200:
|
||||
op_status = response.json()
|
||||
if op_status.get("status") == "completed":
|
||||
operation_completed = True
|
||||
break
|
||||
elif op_status.get("status") == "failed":
|
||||
raise AssertionError(f"Operation failed: {op_status.get('error_message')}")
|
||||
|
||||
await asyncio.sleep(poll_interval)
|
||||
elapsed += poll_interval
|
||||
|
||||
assert operation_completed, f"Async operation did not complete within {max_wait_seconds} seconds"
|
||||
|
||||
# Verify memories were actually stored
|
||||
response = await api_client.get(
|
||||
f"/v1/default/banks/{test_bank_id}/memories/list",
|
||||
params={"limit": 10}
|
||||
)
|
||||
assert response.status_code == 200
|
||||
items = response.json()["items"]
|
||||
assert len(items) > 0, "Should have stored memories after async processing"
|
||||
|
||||
@@ -19,10 +19,6 @@ MODEL_MATRIX = [
|
||||
("openai", "gpt-5-nano"),
|
||||
("openai", "gpt-5"),
|
||||
("openai", "gpt-5.2"),
|
||||
# Anthropic models
|
||||
("anthropic", "claude-sonnet-4-20250514"),
|
||||
("anthropic", "claude-opus-4-5-20251101"),
|
||||
("anthropic", "claude-haiku-4-20250514"),
|
||||
# Groq models
|
||||
("groq", "openai/gpt-oss-120b"),
|
||||
("groq", "openai/gpt-oss-20b"),
|
||||
@@ -40,7 +36,6 @@ def get_api_key_for_provider(provider: str) -> str | None:
|
||||
"""Get API key for provider from environment variables."""
|
||||
provider_key_map = {
|
||||
"openai": "OPENAI_API_KEY",
|
||||
"anthropic": "ANTHROPIC_API_KEY",
|
||||
"groq": "GROQ_API_KEY",
|
||||
"gemini": "GEMINI_API_KEY",
|
||||
}
|
||||
|
||||
@@ -30,8 +30,7 @@ async def test_llm_metrics_recorded_for_groq():
|
||||
# Create a mock metrics collector to track record_llm_call calls
|
||||
mock_collector = MagicMock(spec=MetricsCollector)
|
||||
|
||||
# Patch the provider module where get_metrics_collector is actually called
|
||||
with patch("hindsight_api.engine.providers.openai_compatible_llm.get_metrics_collector", return_value=mock_collector):
|
||||
with patch("hindsight_api.engine.llm_wrapper.get_metrics_collector", return_value=mock_collector):
|
||||
llm = LLMProvider(
|
||||
provider="groq",
|
||||
api_key=api_key,
|
||||
@@ -91,8 +90,7 @@ async def test_llm_metrics_recorded_for_structured_output():
|
||||
|
||||
mock_collector = MagicMock(spec=MetricsCollector)
|
||||
|
||||
# Patch the provider module where get_metrics_collector is actually called
|
||||
with patch("hindsight_api.engine.providers.openai_compatible_llm.get_metrics_collector", return_value=mock_collector):
|
||||
with patch("hindsight_api.engine.llm_wrapper.get_metrics_collector", return_value=mock_collector):
|
||||
llm = LLMProvider(
|
||||
provider="groq",
|
||||
api_key=api_key,
|
||||
|
||||
@@ -1,206 +0,0 @@
|
||||
"""Unit tests for mental model operation validator hooks.
|
||||
|
||||
Tests that the operation validator hooks are called correctly for
|
||||
mental model GET and refresh operations.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
|
||||
from hindsight_api.extensions.operation_validator import (
|
||||
MentalModelGetContext,
|
||||
MentalModelGetResult,
|
||||
MentalModelRefreshResult,
|
||||
OperationValidatorExtension,
|
||||
ValidationResult,
|
||||
)
|
||||
|
||||
|
||||
class TestMentalModelGetContextDataclass:
|
||||
"""Tests for MentalModelGetContext dataclass."""
|
||||
|
||||
def test_create_context(self):
|
||||
"""Test creating a MentalModelGetContext."""
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
request_context = MagicMock()
|
||||
ctx = MentalModelGetContext(
|
||||
bank_id="bank-1",
|
||||
mental_model_id="mm-1",
|
||||
request_context=request_context,
|
||||
)
|
||||
|
||||
assert ctx.bank_id == "bank-1"
|
||||
assert ctx.mental_model_id == "mm-1"
|
||||
assert ctx.request_context is request_context
|
||||
|
||||
|
||||
class TestMentalModelGetResultDataclass:
|
||||
"""Tests for MentalModelGetResult dataclass."""
|
||||
|
||||
def test_create_result_success(self):
|
||||
"""Test creating a successful MentalModelGetResult."""
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
request_context = MagicMock()
|
||||
result = MentalModelGetResult(
|
||||
bank_id="bank-1",
|
||||
mental_model_id="mm-1",
|
||||
request_context=request_context,
|
||||
output_tokens=250,
|
||||
)
|
||||
|
||||
assert result.bank_id == "bank-1"
|
||||
assert result.mental_model_id == "mm-1"
|
||||
assert result.output_tokens == 250
|
||||
assert result.success is True
|
||||
assert result.error is None
|
||||
|
||||
def test_create_result_failure(self):
|
||||
"""Test creating a failed MentalModelGetResult."""
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
result = MentalModelGetResult(
|
||||
bank_id="bank-1",
|
||||
mental_model_id="mm-1",
|
||||
request_context=MagicMock(),
|
||||
output_tokens=0,
|
||||
success=False,
|
||||
error="Not found",
|
||||
)
|
||||
|
||||
assert result.success is False
|
||||
assert result.error == "Not found"
|
||||
|
||||
|
||||
class TestMentalModelRefreshResultDataclass:
|
||||
"""Tests for MentalModelRefreshResult dataclass."""
|
||||
|
||||
def test_create_result_with_all_fields(self):
|
||||
"""Test creating a MentalModelRefreshResult with all fields."""
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
result = MentalModelRefreshResult(
|
||||
bank_id="bank-1",
|
||||
mental_model_id="mm-1",
|
||||
request_context=MagicMock(),
|
||||
query_tokens=50,
|
||||
output_tokens=500,
|
||||
context_tokens=0,
|
||||
facts_used=10,
|
||||
mental_models_used=2,
|
||||
)
|
||||
|
||||
assert result.query_tokens == 50
|
||||
assert result.output_tokens == 500
|
||||
assert result.context_tokens == 0
|
||||
assert result.facts_used == 10
|
||||
assert result.mental_models_used == 2
|
||||
assert result.success is True
|
||||
assert result.error is None
|
||||
|
||||
def test_create_result_failure(self):
|
||||
"""Test creating a failed MentalModelRefreshResult."""
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
result = MentalModelRefreshResult(
|
||||
bank_id="bank-1",
|
||||
mental_model_id="mm-1",
|
||||
request_context=MagicMock(),
|
||||
query_tokens=50,
|
||||
output_tokens=0,
|
||||
context_tokens=0,
|
||||
facts_used=0,
|
||||
mental_models_used=0,
|
||||
success=False,
|
||||
error="Reflect failed",
|
||||
)
|
||||
|
||||
assert result.success is False
|
||||
assert result.error == "Reflect failed"
|
||||
|
||||
|
||||
class TestDefaultHookBehavior:
|
||||
"""Tests for default (no-op) behavior of mental model hooks on OperationValidatorExtension."""
|
||||
|
||||
@pytest.fixture
|
||||
def validator(self):
|
||||
"""Create a concrete subclass for testing default behavior."""
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
# Create a concrete subclass that implements the abstract methods
|
||||
class TestValidator(OperationValidatorExtension):
|
||||
async def validate_retain(self, ctx):
|
||||
return ValidationResult.accept()
|
||||
|
||||
async def validate_recall(self, ctx):
|
||||
return ValidationResult.accept()
|
||||
|
||||
async def validate_reflect(self, ctx):
|
||||
return ValidationResult.accept()
|
||||
|
||||
return TestValidator(config={})
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_validate_mental_model_get_default_accepts(self, validator):
|
||||
"""Test that default validate_mental_model_get accepts."""
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
ctx = MentalModelGetContext(
|
||||
bank_id="bank-1",
|
||||
mental_model_id="mm-1",
|
||||
request_context=MagicMock(),
|
||||
)
|
||||
|
||||
result = await validator.validate_mental_model_get(ctx)
|
||||
|
||||
assert result.allowed is True
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_on_mental_model_get_complete_default_noop(self, validator):
|
||||
"""Test that default on_mental_model_get_complete is a no-op."""
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
result = MentalModelGetResult(
|
||||
bank_id="bank-1",
|
||||
mental_model_id="mm-1",
|
||||
request_context=MagicMock(),
|
||||
output_tokens=100,
|
||||
)
|
||||
|
||||
# Should not raise
|
||||
await validator.on_mental_model_get_complete(result)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_on_mental_model_refresh_complete_default_noop(self, validator):
|
||||
"""Test that default on_mental_model_refresh_complete is a no-op."""
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
result = MentalModelRefreshResult(
|
||||
bank_id="bank-1",
|
||||
mental_model_id="mm-1",
|
||||
request_context=MagicMock(),
|
||||
query_tokens=50,
|
||||
output_tokens=500,
|
||||
context_tokens=0,
|
||||
facts_used=5,
|
||||
mental_models_used=1,
|
||||
)
|
||||
|
||||
# Should not raise
|
||||
await validator.on_mental_model_refresh_complete(result)
|
||||
|
||||
|
||||
class TestExportsAvailable:
|
||||
"""Test that mental model hooks are properly exported."""
|
||||
|
||||
def test_imports_from_extensions_package(self):
|
||||
"""Test that all mental model types can be imported from hindsight_api.extensions."""
|
||||
from hindsight_api.extensions import (
|
||||
MentalModelGetContext,
|
||||
MentalModelGetResult,
|
||||
MentalModelRefreshResult,
|
||||
)
|
||||
|
||||
assert MentalModelGetContext is not None
|
||||
assert MentalModelGetResult is not None
|
||||
assert MentalModelRefreshResult is not None
|
||||
@@ -1,123 +0,0 @@
|
||||
"""Test provider-specific default models in config."""
|
||||
|
||||
import os
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
def test_provider_default_models():
|
||||
"""Test that each provider has a default model and it's used when model is not explicitly set."""
|
||||
from hindsight_api.config import PROVIDER_DEFAULT_MODELS, HindsightConfig, clear_config_cache
|
||||
|
||||
# Save original env vars
|
||||
original_provider = os.environ.get("HINDSIGHT_API_LLM_PROVIDER")
|
||||
original_model = os.environ.get("HINDSIGHT_API_LLM_MODEL")
|
||||
|
||||
try:
|
||||
# Test each provider has a default
|
||||
for provider, expected_model in PROVIDER_DEFAULT_MODELS.items():
|
||||
clear_config_cache()
|
||||
os.environ["HINDSIGHT_API_LLM_PROVIDER"] = provider
|
||||
# Remove explicit model setting to test default
|
||||
if "HINDSIGHT_API_LLM_MODEL" in os.environ:
|
||||
del os.environ["HINDSIGHT_API_LLM_MODEL"]
|
||||
|
||||
config = HindsightConfig.from_env()
|
||||
assert config.llm_provider == provider, f"Provider mismatch for {provider}"
|
||||
assert config.llm_model == expected_model, f"Expected {expected_model} for {provider}, got {config.llm_model}"
|
||||
|
||||
finally:
|
||||
# Restore original env vars
|
||||
clear_config_cache()
|
||||
if original_provider:
|
||||
os.environ["HINDSIGHT_API_LLM_PROVIDER"] = original_provider
|
||||
elif "HINDSIGHT_API_LLM_PROVIDER" in os.environ:
|
||||
del os.environ["HINDSIGHT_API_LLM_PROVIDER"]
|
||||
|
||||
if original_model:
|
||||
os.environ["HINDSIGHT_API_LLM_MODEL"] = original_model
|
||||
elif "HINDSIGHT_API_LLM_MODEL" in os.environ:
|
||||
del os.environ["HINDSIGHT_API_LLM_MODEL"]
|
||||
|
||||
|
||||
def test_explicit_model_overrides_provider_default():
|
||||
"""Test that explicit model setting overrides provider default."""
|
||||
from hindsight_api.config import HindsightConfig, clear_config_cache
|
||||
|
||||
original_provider = os.environ.get("HINDSIGHT_API_LLM_PROVIDER")
|
||||
original_model = os.environ.get("HINDSIGHT_API_LLM_MODEL")
|
||||
|
||||
try:
|
||||
clear_config_cache()
|
||||
os.environ["HINDSIGHT_API_LLM_PROVIDER"] = "anthropic"
|
||||
os.environ["HINDSIGHT_API_LLM_MODEL"] = "claude-sonnet-4-5-20250929"
|
||||
|
||||
config = HindsightConfig.from_env()
|
||||
assert config.llm_provider == "anthropic"
|
||||
assert config.llm_model == "claude-sonnet-4-5-20250929", "Explicit model should override default"
|
||||
|
||||
finally:
|
||||
clear_config_cache()
|
||||
if original_provider:
|
||||
os.environ["HINDSIGHT_API_LLM_PROVIDER"] = original_provider
|
||||
elif "HINDSIGHT_API_LLM_PROVIDER" in os.environ:
|
||||
del os.environ["HINDSIGHT_API_LLM_PROVIDER"]
|
||||
|
||||
if original_model:
|
||||
os.environ["HINDSIGHT_API_LLM_MODEL"] = original_model
|
||||
elif "HINDSIGHT_API_LLM_MODEL" in os.environ:
|
||||
del os.environ["HINDSIGHT_API_LLM_MODEL"]
|
||||
|
||||
|
||||
def test_per_operation_provider_default_model():
|
||||
"""Test that per-operation providers use their own default models."""
|
||||
from hindsight_api.config import HindsightConfig, clear_config_cache
|
||||
|
||||
original_provider = os.environ.get("HINDSIGHT_API_LLM_PROVIDER")
|
||||
original_model = os.environ.get("HINDSIGHT_API_LLM_MODEL")
|
||||
original_retain_provider = os.environ.get("HINDSIGHT_API_RETAIN_LLM_PROVIDER")
|
||||
original_retain_model = os.environ.get("HINDSIGHT_API_RETAIN_LLM_MODEL")
|
||||
|
||||
try:
|
||||
clear_config_cache()
|
||||
os.environ["HINDSIGHT_API_LLM_PROVIDER"] = "openai"
|
||||
# Remove explicit model to use provider default
|
||||
if "HINDSIGHT_API_LLM_MODEL" in os.environ:
|
||||
del os.environ["HINDSIGHT_API_LLM_MODEL"]
|
||||
|
||||
# Set retain-specific provider but not model
|
||||
os.environ["HINDSIGHT_API_RETAIN_LLM_PROVIDER"] = "anthropic"
|
||||
if "HINDSIGHT_API_RETAIN_LLM_MODEL" in os.environ:
|
||||
del os.environ["HINDSIGHT_API_RETAIN_LLM_MODEL"]
|
||||
|
||||
config = HindsightConfig.from_env()
|
||||
|
||||
# Global LLM should use OpenAI default
|
||||
assert config.llm_model == "o3-mini", f"Expected o3-mini, got {config.llm_model}"
|
||||
|
||||
# Retain should use Anthropic default
|
||||
assert (
|
||||
config.retain_llm_model == "claude-haiku-4-5-20251001"
|
||||
), f"Expected claude-haiku-4-5-20251001, got {config.retain_llm_model}"
|
||||
|
||||
finally:
|
||||
clear_config_cache()
|
||||
if original_provider:
|
||||
os.environ["HINDSIGHT_API_LLM_PROVIDER"] = original_provider
|
||||
elif "HINDSIGHT_API_LLM_PROVIDER" in os.environ:
|
||||
del os.environ["HINDSIGHT_API_LLM_PROVIDER"]
|
||||
|
||||
if original_model:
|
||||
os.environ["HINDSIGHT_API_LLM_MODEL"] = original_model
|
||||
elif "HINDSIGHT_API_LLM_MODEL" in os.environ:
|
||||
del os.environ["HINDSIGHT_API_LLM_MODEL"]
|
||||
|
||||
if original_retain_provider:
|
||||
os.environ["HINDSIGHT_API_RETAIN_LLM_PROVIDER"] = original_retain_provider
|
||||
elif "HINDSIGHT_API_RETAIN_LLM_PROVIDER" in os.environ:
|
||||
del os.environ["HINDSIGHT_API_RETAIN_LLM_PROVIDER"]
|
||||
|
||||
if original_retain_model:
|
||||
os.environ["HINDSIGHT_API_RETAIN_LLM_MODEL"] = original_retain_model
|
||||
elif "HINDSIGHT_API_RETAIN_LLM_MODEL" in os.environ:
|
||||
del os.environ["HINDSIGHT_API_RETAIN_LLM_MODEL"]
|
||||
@@ -1020,81 +1020,6 @@ class TestDynamicTenantDiscovery:
|
||||
for task in claimed:
|
||||
assert task.schema is None
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_poller_with_custom_schema(self, pool):
|
||||
"""Test that poller uses custom schema when schema parameter is provided."""
|
||||
from hindsight_api.worker import WorkerPoller
|
||||
|
||||
# Create a custom schema for testing
|
||||
test_schema = "test_custom_schema"
|
||||
|
||||
try:
|
||||
# Create schema and copy table structure
|
||||
await pool.execute(f'CREATE SCHEMA IF NOT EXISTS "{test_schema}"')
|
||||
await pool.execute(
|
||||
f"""
|
||||
CREATE TABLE "{test_schema}".async_operations (
|
||||
LIKE public.async_operations INCLUDING ALL
|
||||
)
|
||||
"""
|
||||
)
|
||||
|
||||
# Create pending tasks in the custom schema
|
||||
bank_id = f"test-worker-{uuid.uuid4().hex[:8]}"
|
||||
task_ids = []
|
||||
for i in range(3):
|
||||
op_id = uuid.uuid4()
|
||||
task_ids.append(str(op_id))
|
||||
payload = json.dumps({"type": "test_task", "index": i, "bank_id": bank_id})
|
||||
await pool.execute(
|
||||
f"""
|
||||
INSERT INTO "{test_schema}".async_operations (operation_id, bank_id, operation_type, status, task_payload)
|
||||
VALUES ($1, $2, 'test', 'pending', $3::jsonb)
|
||||
""",
|
||||
op_id,
|
||||
bank_id,
|
||||
payload,
|
||||
)
|
||||
|
||||
# Create poller with custom schema
|
||||
poller = WorkerPoller(
|
||||
pool=pool,
|
||||
worker_id="test-worker-custom-schema",
|
||||
executor=lambda x: None,
|
||||
schema=test_schema,
|
||||
)
|
||||
|
||||
# Claim tasks
|
||||
claimed = await poller.claim_batch()
|
||||
assert len(claimed) == 3, f"Expected 3 tasks, got {len(claimed)}"
|
||||
|
||||
# All tasks should have schema=test_schema
|
||||
claimed_ids = []
|
||||
for task in claimed:
|
||||
assert task.schema == test_schema, f"Expected schema '{test_schema}', got '{task.schema}'"
|
||||
claimed_ids.append(task.operation_id)
|
||||
|
||||
# Verify claimed tasks match what we inserted
|
||||
assert set(claimed_ids) == set(task_ids)
|
||||
|
||||
# Verify tasks are marked as processing in the custom schema
|
||||
rows = await pool.fetch(
|
||||
f"""
|
||||
SELECT operation_id, status, worker_id
|
||||
FROM "{test_schema}".async_operations
|
||||
WHERE operation_id = ANY($1)
|
||||
""",
|
||||
[uuid.UUID(tid) for tid in task_ids],
|
||||
)
|
||||
assert len(rows) == 3
|
||||
for row in rows:
|
||||
assert row["status"] == "processing"
|
||||
assert row["worker_id"] == "test-worker-custom-schema"
|
||||
|
||||
finally:
|
||||
# Clean up: drop the custom schema
|
||||
await pool.execute(f'DROP SCHEMA IF EXISTS "{test_schema}" CASCADE')
|
||||
|
||||
|
||||
async def test_worker_fire_and_forget_nonblocking(pool, clean_operations):
|
||||
"""
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "hindsight-cli"
|
||||
version = "0.4.7"
|
||||
version = "0.4.2"
|
||||
edition = "2021"
|
||||
authors = ["Hindsight Team"]
|
||||
description = "A beautiful CLI for Hindsight - semantic memory system"
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
@@ -489,7 +489,7 @@ class Configuration:
|
||||
return "Python SDK Debug Report:\n"\
|
||||
"OS: {env}\n"\
|
||||
"Python Version: {pyversion}\n"\
|
||||
"Version of the API: 0.4.7\n"\
|
||||
"Version of the API: 0.4.2\n"\
|
||||
"SDK Package Version: 0.0.7".\
|
||||
format(env=sys.platform, pyversion=sys.version)
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
+1
-1
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
+1
-1
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
HTTP API for Hindsight
|
||||
|
||||
The version of the OpenAPI document: 0.4.7
|
||||
The version of the OpenAPI document: 0.4.2
|
||||
Generated by OpenAPI Generator (https://openapi-generator.tech)
|
||||
|
||||
Do not edit the class manually.
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user