Compare commits
4
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
cc9c6103be | ||
|
|
94296b72e3 | ||
|
|
047400d6e4 | ||
|
|
7307fb6bcf |
@@ -918,6 +918,35 @@ jobs:
|
||||
echo "=== API Server Logs ==="
|
||||
cat /tmp/api-server.log || echo "No API server log found"
|
||||
|
||||
test-crewai-integration:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v5
|
||||
with:
|
||||
enable-cache: true
|
||||
prune-cache: false
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version-file: ".python-version"
|
||||
|
||||
- name: Build crewai integration
|
||||
working-directory: ./hindsight-integrations/crewai
|
||||
run: uv build
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: ./hindsight-integrations/crewai
|
||||
run: uv sync --frozen
|
||||
|
||||
- name: Run tests
|
||||
working-directory: ./hindsight-integrations/crewai
|
||||
run: uv run pytest tests -v
|
||||
|
||||
test-litellm-integration:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
|
||||
@@ -0,0 +1,241 @@
|
||||
---
|
||||
sidebar_position: 5
|
||||
---
|
||||
|
||||
# CrewAI
|
||||
|
||||
Persistent memory for AI agent crews via [CrewAI](https://github.com/crewAIInc/crewAI). Give your crews long-term memory with fact extraction, entity tracking, and temporal awareness.
|
||||
|
||||
## Features
|
||||
|
||||
- **Drop-in Storage Backend** - Implements CrewAI's `Storage` interface for `ExternalMemory`
|
||||
- **Automatic Memory Flow** - CrewAI automatically stores task outputs and retrieves relevant memories
|
||||
- **Per-Agent Banks** - Optionally give each agent its own isolated memory bank
|
||||
- **Reflect Tool** - Agents can explicitly reason over memories with disposition-aware synthesis
|
||||
- **Simple Configuration** - Configure once, use everywhere
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install hindsight-crewai
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
```python
|
||||
from hindsight_crewai import configure, HindsightStorage
|
||||
from crewai.memory.external.external_memory import ExternalMemory
|
||||
from crewai import Agent, Crew, Task
|
||||
|
||||
configure(hindsight_api_url="http://localhost:8888")
|
||||
|
||||
crew = Crew(
|
||||
agents=[Agent(role="Researcher", goal="Find information", backstory="...")],
|
||||
tasks=[Task(description="Research AI trends", expected_output="Report")],
|
||||
external_memory=ExternalMemory(
|
||||
storage=HindsightStorage(bank_id="my-crew")
|
||||
),
|
||||
)
|
||||
|
||||
crew.kickoff()
|
||||
```
|
||||
|
||||
That's it. CrewAI will automatically:
|
||||
- **Query memories** at the start of each task
|
||||
- **Store task outputs** to Hindsight after each task completes
|
||||
|
||||
Memories persist across crew runs, so your crew learns over time.
|
||||
|
||||
## How It Works
|
||||
|
||||
The integration maps CrewAI's 3-method `Storage` interface to Hindsight's API:
|
||||
|
||||
| CrewAI | Hindsight | What happens |
|
||||
|--------|-----------|--------------|
|
||||
| `save(value, metadata, agent)` | `retain(bank_id, content, ...)` | Task output is stored. Hindsight extracts facts, entities, and relationships from the raw text. |
|
||||
| `search(query, limit)` | `recall(bank_id, query, ...)` | CrewAI constructs a query from the task description. Hindsight runs semantic search, BM25, graph traversal, and reranking. |
|
||||
| `reset()` | `delete_bank(bank_id)` | Wipes the bank and optionally recreates it with its original mission. |
|
||||
|
||||
CrewAI calls `search()` automatically at the start of each task and `save()` after each task completes.
|
||||
|
||||
## Configuration Options
|
||||
|
||||
```python
|
||||
from hindsight_crewai import configure
|
||||
|
||||
configure(
|
||||
hindsight_api_url="http://localhost:8888", # Hindsight API URL
|
||||
api_key="your-api-key", # Or set HINDSIGHT_API_KEY env var
|
||||
budget="mid", # Recall budget: "low", "mid", "high"
|
||||
max_tokens=4096, # Max tokens for recall results
|
||||
tags=["env:prod"], # Tags for stored memories
|
||||
recall_tags=["scope:global"], # Tags to filter recall
|
||||
recall_tags_match="any", # Tag match: any/all/any_strict/all_strict
|
||||
verbose=True, # Enable logging
|
||||
)
|
||||
```
|
||||
|
||||
### Per-Storage Overrides
|
||||
|
||||
Constructor arguments override global configuration:
|
||||
|
||||
```python
|
||||
storage = HindsightStorage(
|
||||
bank_id="my-crew",
|
||||
budget="high",
|
||||
max_tokens=8192,
|
||||
tags=["team:alpha"],
|
||||
)
|
||||
```
|
||||
|
||||
## Bank Missions
|
||||
|
||||
Set a mission to guide how Hindsight processes and organizes memories:
|
||||
|
||||
```python
|
||||
storage = HindsightStorage(
|
||||
bank_id="my-crew",
|
||||
mission="Track software architecture decisions, technical debt, and team preferences.",
|
||||
)
|
||||
```
|
||||
|
||||
## Per-Agent Memory Banks
|
||||
|
||||
Give each agent its own isolated memory bank:
|
||||
|
||||
```python
|
||||
storage = HindsightStorage(
|
||||
bank_id="my-crew",
|
||||
per_agent_banks=True,
|
||||
# Researcher -> "my-crew-researcher"
|
||||
# Writer -> "my-crew-writer"
|
||||
)
|
||||
```
|
||||
|
||||
Or use a custom bank resolver for full control:
|
||||
|
||||
```python
|
||||
storage = HindsightStorage(
|
||||
bank_id="my-crew",
|
||||
bank_resolver=lambda base, agent: f"{base}-{agent.lower()}" if agent else base,
|
||||
)
|
||||
```
|
||||
|
||||
:::info
|
||||
When `per_agent_banks=True`, the automatic `search()` at task start queries the base bank (shared context), since CrewAI's `search()` method does not receive the agent parameter. For per-agent search isolation, create separate `HindsightStorage` instances per agent.
|
||||
:::
|
||||
|
||||
## Reflect Tool
|
||||
|
||||
CrewAI's storage interface only supports save/search/reset. To give agents access to Hindsight's `reflect` (disposition-aware memory synthesis), add it as a tool:
|
||||
|
||||
```python
|
||||
from hindsight_crewai import HindsightReflectTool
|
||||
|
||||
reflect_tool = HindsightReflectTool(
|
||||
bank_id="my-crew",
|
||||
budget="mid",
|
||||
reflect_context="You are helping a software team track decisions.",
|
||||
)
|
||||
|
||||
agent = Agent(
|
||||
role="Analyst",
|
||||
goal="Analyze project history",
|
||||
backstory="...",
|
||||
tools=[reflect_tool],
|
||||
)
|
||||
```
|
||||
|
||||
When the agent calls this tool, it gets a synthesized, contextual answer based on all relevant memories rather than raw fact snippets.
|
||||
|
||||
## Full Example
|
||||
|
||||
A research crew that remembers findings across runs:
|
||||
|
||||
```python
|
||||
from hindsight_crewai import configure, HindsightStorage, HindsightReflectTool
|
||||
from crewai.memory.external.external_memory import ExternalMemory
|
||||
from crewai import Agent, Crew, Task
|
||||
|
||||
configure(hindsight_api_url="http://localhost:8888")
|
||||
|
||||
storage = HindsightStorage(
|
||||
bank_id="research-crew",
|
||||
mission="Track technology research findings and comparisons.",
|
||||
)
|
||||
|
||||
reflect_tool = HindsightReflectTool(bank_id="research-crew", budget="mid")
|
||||
|
||||
researcher = Agent(
|
||||
role="Researcher",
|
||||
goal="Research topics, building on prior knowledge.",
|
||||
backstory="Before starting, use hindsight_reflect to check what you already know.",
|
||||
tools=[reflect_tool],
|
||||
)
|
||||
|
||||
writer = Agent(
|
||||
role="Writer",
|
||||
goal="Write summaries incorporating prior findings.",
|
||||
backstory="Use hindsight_reflect to recall prior research.",
|
||||
tools=[reflect_tool],
|
||||
)
|
||||
|
||||
crew = Crew(
|
||||
agents=[researcher, writer],
|
||||
tasks=[
|
||||
Task(description="Research the benefits of Rust", expected_output="Analysis", agent=researcher),
|
||||
Task(description="Write an executive summary", expected_output="Summary", agent=writer),
|
||||
],
|
||||
external_memory=ExternalMemory(storage=storage),
|
||||
)
|
||||
|
||||
# Run 1: researches Rust, stores findings
|
||||
crew.kickoff()
|
||||
|
||||
# Run 2: recalls Rust research when comparing with Go
|
||||
crew.tasks[0].description = "Compare Rust with Go"
|
||||
crew.kickoff()
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### Configuration
|
||||
|
||||
| Function | Description |
|
||||
|----------|-------------|
|
||||
| `configure(...)` | Set global connection and default settings |
|
||||
| `get_config()` | Get current configuration |
|
||||
| `reset_config()` | Reset configuration to None |
|
||||
|
||||
### Storage
|
||||
|
||||
| Parameter | Default | Description |
|
||||
|-----------|---------|-------------|
|
||||
| `bank_id` | required | Hindsight memory bank ID |
|
||||
| `hindsight_api_url` | from config | Override API URL |
|
||||
| `api_key` | from config | Override API key |
|
||||
| `budget` | `"mid"` | Recall budget (low/mid/high) |
|
||||
| `max_tokens` | `4096` | Max tokens for recall results |
|
||||
| `tags` | `None` | Tags applied when storing |
|
||||
| `recall_tags` | `None` | Tags to filter when searching |
|
||||
| `recall_tags_match` | `"any"` | Tag matching mode |
|
||||
| `per_agent_banks` | `False` | Give each agent its own bank |
|
||||
| `bank_resolver` | `None` | Custom `(bank_id, agent) -> bank_id` |
|
||||
| `mission` | `None` | Bank mission for memory organization |
|
||||
| `verbose` | `False` | Enable verbose logging |
|
||||
|
||||
### Reflect Tool
|
||||
|
||||
| Parameter | Default | Description |
|
||||
|-----------|---------|-------------|
|
||||
| `bank_id` | required | Hindsight memory bank ID |
|
||||
| `budget` | `"mid"` | Reflect budget (low/mid/high) |
|
||||
| `reflect_context` | `None` | Additional context for reasoning |
|
||||
| `hindsight_api_url` | from config | Override API URL |
|
||||
| `api_key` | from config | Override API key |
|
||||
|
||||
## Requirements
|
||||
|
||||
- Python >= 3.10
|
||||
- crewai >= 0.86.0
|
||||
- A running Hindsight API server
|
||||
@@ -210,6 +210,11 @@ const sidebars: SidebarsConfig = {
|
||||
id: 'sdks/integrations/ai-sdk',
|
||||
label: 'Vercel AI SDK',
|
||||
},
|
||||
{
|
||||
type: 'doc',
|
||||
id: 'sdks/integrations/crewai',
|
||||
label: 'CrewAI',
|
||||
},
|
||||
{
|
||||
type: 'doc',
|
||||
id: 'sdks/integrations/skills',
|
||||
|
||||
@@ -0,0 +1,156 @@
|
||||
# hindsight-crewai
|
||||
|
||||
Persistent memory for AI agent crews via Hindsight. Give your CrewAI crews long-term memory with fact extraction, entity tracking, and temporal awareness.
|
||||
|
||||
## Features
|
||||
|
||||
- **Drop-in Storage Backend** - Implements CrewAI's `Storage` interface for `ExternalMemory`
|
||||
- **Automatic Memory Flow** - CrewAI automatically stores task outputs and retrieves relevant memories
|
||||
- **Per-Agent Banks** - Optionally give each agent its own isolated memory bank
|
||||
- **Reflect Tool** - Agents can explicitly reason over memories with disposition-aware synthesis
|
||||
- **Simple Configuration** - Configure once, use everywhere
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install hindsight-crewai
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
```python
|
||||
from hindsight_crewai import configure, HindsightStorage
|
||||
from crewai.memory.external.external_memory import ExternalMemory
|
||||
from crewai import Agent, Crew, Task
|
||||
|
||||
# Step 1: Configure connection
|
||||
configure(hindsight_api_url="http://localhost:8888")
|
||||
|
||||
# Step 2: Create crew with Hindsight-backed memory
|
||||
crew = Crew(
|
||||
agents=[
|
||||
Agent(role="Researcher", goal="Find information", backstory="..."),
|
||||
Agent(role="Writer", goal="Write reports", backstory="..."),
|
||||
],
|
||||
tasks=[
|
||||
Task(description="Research AI trends", expected_output="Report"),
|
||||
],
|
||||
external_memory=ExternalMemory(
|
||||
storage=HindsightStorage(bank_id="my-crew")
|
||||
),
|
||||
)
|
||||
|
||||
crew.kickoff()
|
||||
```
|
||||
|
||||
That's it. CrewAI will automatically:
|
||||
- **Query memories** at the start of each task
|
||||
- **Store task outputs** to Hindsight after each task completes
|
||||
|
||||
Memories persist across crew runs, so your crew learns over time.
|
||||
|
||||
## Per-Agent Memory Banks
|
||||
|
||||
Give each agent its own isolated memory bank:
|
||||
|
||||
```python
|
||||
storage = HindsightStorage(
|
||||
bank_id="my-crew",
|
||||
per_agent_banks=True, # Researcher -> "my-crew-researcher", Writer -> "my-crew-writer"
|
||||
)
|
||||
```
|
||||
|
||||
Or use a custom bank resolver for full control:
|
||||
|
||||
```python
|
||||
storage = HindsightStorage(
|
||||
bank_id="my-crew",
|
||||
bank_resolver=lambda base, agent: f"{base}-{agent.lower()}" if agent else base,
|
||||
)
|
||||
```
|
||||
|
||||
## Reflect Tool
|
||||
|
||||
CrewAI's storage interface only supports save/search/reset. To give agents access to Hindsight's `reflect` (disposition-aware memory synthesis), add it as a tool:
|
||||
|
||||
```python
|
||||
from hindsight_crewai import HindsightReflectTool
|
||||
|
||||
reflect_tool = HindsightReflectTool(
|
||||
bank_id="my-crew",
|
||||
budget="mid",
|
||||
reflect_context="You are helping a software team track decisions.",
|
||||
)
|
||||
|
||||
agent = Agent(
|
||||
role="Analyst",
|
||||
goal="Analyze project history",
|
||||
backstory="...",
|
||||
tools=[reflect_tool],
|
||||
)
|
||||
```
|
||||
|
||||
When the agent calls this tool, it gets a synthesized, contextual answer based on all relevant memories — not just raw facts.
|
||||
|
||||
## Bank Missions
|
||||
|
||||
Set a mission to guide how Hindsight processes and organizes memories:
|
||||
|
||||
```python
|
||||
storage = HindsightStorage(
|
||||
bank_id="my-crew",
|
||||
mission="Track software architecture decisions, technical debt, and team preferences.",
|
||||
)
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### Global Configuration
|
||||
|
||||
```python
|
||||
from hindsight_crewai import configure
|
||||
|
||||
configure(
|
||||
hindsight_api_url="http://localhost:8888", # Default: production API
|
||||
api_key="your-api-key", # Or set HINDSIGHT_API_KEY env var
|
||||
budget="mid", # Recall budget: low/mid/high
|
||||
max_tokens=4096, # Max tokens for recall results
|
||||
tags=["env:prod"], # Tags for stored memories
|
||||
recall_tags=["scope:global"], # Tags to filter recall
|
||||
recall_tags_match="any", # Tag match mode: any/all/any_strict/all_strict
|
||||
verbose=True, # Enable logging
|
||||
)
|
||||
```
|
||||
|
||||
### Per-Storage Overrides
|
||||
|
||||
Constructor arguments override global configuration:
|
||||
|
||||
```python
|
||||
storage = HindsightStorage(
|
||||
bank_id="my-crew",
|
||||
budget="high", # Override global budget
|
||||
max_tokens=8192, # Override global max_tokens
|
||||
tags=["team:alpha"], # Override global tags
|
||||
)
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
See the [CrewAI memory example](https://github.com/vectorize-io/hindsight-cookbook/tree/main/applications/crewai-memory) in the Hindsight Cookbook for a complete working demo with a Researcher + Writer crew.
|
||||
|
||||
## Configuration Reference
|
||||
|
||||
| Parameter | Default | Description |
|
||||
|---|---|---|
|
||||
| `hindsight_api_url` | Production API | Hindsight API URL |
|
||||
| `api_key` | `HINDSIGHT_API_KEY` env | API key for authentication |
|
||||
| `budget` | `"mid"` | Recall budget level (low/mid/high) |
|
||||
| `max_tokens` | `4096` | Maximum tokens for recall results |
|
||||
| `tags` | `None` | Tags applied when storing memories |
|
||||
| `recall_tags` | `None` | Tags to filter when searching |
|
||||
| `recall_tags_match` | `"any"` | Tag matching mode |
|
||||
| `per_agent_banks` | `False` | Give each agent its own bank |
|
||||
| `bank_resolver` | `None` | Custom (bank_id, agent) -> bank_id function |
|
||||
| `mission` | `None` | Bank mission for memory organization |
|
||||
| `verbose` | `False` | Enable verbose logging |
|
||||
@@ -0,0 +1,50 @@
|
||||
"""Hindsight-CrewAI: Persistent memory for AI agent crews.
|
||||
|
||||
Provides a Hindsight-backed Storage implementation for CrewAI's
|
||||
ExternalMemory system, giving your crews long-term memory across runs.
|
||||
|
||||
Basic usage::
|
||||
|
||||
from hindsight_crewai import configure, HindsightStorage
|
||||
from crewai.memory.external.external_memory import ExternalMemory
|
||||
from crewai import Crew
|
||||
|
||||
configure(hindsight_api_url="http://localhost:8888")
|
||||
|
||||
crew = Crew(
|
||||
agents=[...],
|
||||
tasks=[...],
|
||||
external_memory=ExternalMemory(
|
||||
storage=HindsightStorage(bank_id="my-crew")
|
||||
),
|
||||
)
|
||||
|
||||
Per-agent banks::
|
||||
|
||||
storage = HindsightStorage(
|
||||
bank_id="crew-shared",
|
||||
per_agent_banks=True,
|
||||
)
|
||||
"""
|
||||
|
||||
from .config import (
|
||||
HindsightCrewAIConfig,
|
||||
configure,
|
||||
get_config,
|
||||
reset_config,
|
||||
)
|
||||
from .errors import HindsightError
|
||||
from .storage import HindsightStorage
|
||||
from .tools import HindsightReflectTool
|
||||
|
||||
__version__ = "0.1.0"
|
||||
|
||||
__all__ = [
|
||||
"configure",
|
||||
"get_config",
|
||||
"reset_config",
|
||||
"HindsightCrewAIConfig",
|
||||
"HindsightStorage",
|
||||
"HindsightReflectTool",
|
||||
"HindsightError",
|
||||
]
|
||||
@@ -0,0 +1,51 @@
|
||||
"""Async compatibility helpers.
|
||||
|
||||
CrewAI runs inside an async event loop. The Hindsight client's sync
|
||||
methods internally call ``loop.run_until_complete()``, which fails
|
||||
when a loop is already running or when ``asyncio.get_event_loop()``
|
||||
returns a foreign loop from another thread.
|
||||
|
||||
This module provides a dedicated worker thread with a persistent
|
||||
event loop for all Hindsight API calls, ensuring the aiohttp session
|
||||
stays bound to a single, stable loop.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import concurrent.futures
|
||||
import threading
|
||||
from typing import Any, Callable
|
||||
|
||||
_thread_pool = concurrent.futures.ThreadPoolExecutor(max_workers=2)
|
||||
_thread_init_lock = threading.Lock()
|
||||
_initialized_threads: set[int] = set()
|
||||
|
||||
|
||||
def _ensure_thread_loop() -> None:
|
||||
"""Ensure the current thread has a persistent event loop."""
|
||||
tid = threading.get_ident()
|
||||
if tid not in _initialized_threads:
|
||||
with _thread_init_lock:
|
||||
if tid not in _initialized_threads:
|
||||
loop = asyncio.new_event_loop()
|
||||
asyncio.set_event_loop(loop)
|
||||
_initialized_threads.add(tid)
|
||||
|
||||
|
||||
def call_sync(fn: Callable[..., Any], *args: Any, **kwargs: Any) -> Any:
|
||||
"""Call a sync Hindsight client method safely.
|
||||
|
||||
Runs the call in a dedicated thread pool where each thread has
|
||||
its own persistent event loop. This avoids:
|
||||
- Nested ``run_until_complete`` when CrewAI's loop is running
|
||||
- Cross-thread loop references that cause "Event loop is closed"
|
||||
- aiohttp session/loop binding issues
|
||||
"""
|
||||
|
||||
def _run() -> Any:
|
||||
_ensure_thread_loop()
|
||||
return fn(*args, **kwargs)
|
||||
|
||||
future = _thread_pool.submit(_run)
|
||||
return future.result(timeout=60)
|
||||
@@ -0,0 +1,92 @@
|
||||
"""Global configuration for Hindsight-CrewAI integration."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
|
||||
DEFAULT_HINDSIGHT_API_URL = "https://api.hindsight.vectorize.io"
|
||||
HINDSIGHT_API_KEY_ENV = "HINDSIGHT_API_KEY"
|
||||
|
||||
|
||||
@dataclass
|
||||
class HindsightCrewAIConfig:
|
||||
"""Connection and default settings for the CrewAI integration.
|
||||
|
||||
Attributes:
|
||||
hindsight_api_url: URL of the Hindsight API server.
|
||||
api_key: API key for Hindsight authentication.
|
||||
budget: Default recall budget level (low/mid/high).
|
||||
max_tokens: Default maximum tokens for recall results.
|
||||
tags: Default tags applied when storing memories.
|
||||
recall_tags: Default tags to filter when searching memories.
|
||||
recall_tags_match: Tag matching mode (any/all/any_strict/all_strict).
|
||||
verbose: Enable verbose logging.
|
||||
"""
|
||||
|
||||
hindsight_api_url: str = DEFAULT_HINDSIGHT_API_URL
|
||||
api_key: str | None = None
|
||||
budget: str = "mid"
|
||||
max_tokens: int = 4096
|
||||
tags: list[str] | None = None
|
||||
recall_tags: list[str] | None = None
|
||||
recall_tags_match: str = "any"
|
||||
verbose: bool = False
|
||||
|
||||
|
||||
_global_config: HindsightCrewAIConfig | None = None
|
||||
|
||||
|
||||
def configure(
|
||||
hindsight_api_url: str | None = None,
|
||||
api_key: str | None = None,
|
||||
budget: str = "mid",
|
||||
max_tokens: int = 4096,
|
||||
tags: list[str] | None = None,
|
||||
recall_tags: list[str] | None = None,
|
||||
recall_tags_match: str = "any",
|
||||
verbose: bool = False,
|
||||
) -> HindsightCrewAIConfig:
|
||||
"""Configure Hindsight connection and default settings.
|
||||
|
||||
Args:
|
||||
hindsight_api_url: Hindsight API URL (default: production).
|
||||
api_key: API key. Falls back to HINDSIGHT_API_KEY env var.
|
||||
budget: Default recall budget (low/mid/high).
|
||||
max_tokens: Default max tokens for recall.
|
||||
tags: Default tags for retain operations.
|
||||
recall_tags: Default tags to filter recall/search.
|
||||
recall_tags_match: Tag matching mode.
|
||||
verbose: Enable verbose logging.
|
||||
|
||||
Returns:
|
||||
The configured HindsightCrewAIConfig.
|
||||
"""
|
||||
global _global_config
|
||||
|
||||
resolved_url = hindsight_api_url or DEFAULT_HINDSIGHT_API_URL
|
||||
resolved_key = api_key or os.environ.get(HINDSIGHT_API_KEY_ENV)
|
||||
|
||||
_global_config = HindsightCrewAIConfig(
|
||||
hindsight_api_url=resolved_url,
|
||||
api_key=resolved_key,
|
||||
budget=budget,
|
||||
max_tokens=max_tokens,
|
||||
tags=tags,
|
||||
recall_tags=recall_tags,
|
||||
recall_tags_match=recall_tags_match,
|
||||
verbose=verbose,
|
||||
)
|
||||
|
||||
return _global_config
|
||||
|
||||
|
||||
def get_config() -> HindsightCrewAIConfig | None:
|
||||
"""Get the current global configuration."""
|
||||
return _global_config
|
||||
|
||||
|
||||
def reset_config() -> None:
|
||||
"""Reset global configuration to None."""
|
||||
global _global_config
|
||||
_global_config = None
|
||||
@@ -0,0 +1,7 @@
|
||||
"""Hindsight-CrewAI error types."""
|
||||
|
||||
|
||||
class HindsightError(Exception):
|
||||
"""Exception raised when a Hindsight memory operation fails."""
|
||||
|
||||
pass
|
||||
@@ -0,0 +1,313 @@
|
||||
"""CrewAI Storage backend powered by Hindsight.
|
||||
|
||||
Implements CrewAI's Storage interface (save/search/reset) using
|
||||
Hindsight's retain/recall APIs for persistent agent memory.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import threading
|
||||
from typing import Any, Callable
|
||||
|
||||
from crewai.memory.storage.interface import Storage
|
||||
|
||||
from ._compat import call_sync
|
||||
from .config import get_config
|
||||
from .errors import HindsightError
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class HindsightStorage(Storage):
|
||||
"""CrewAI Storage backend that persists memories to Hindsight.
|
||||
|
||||
Maps CrewAI's storage interface to Hindsight's memory API:
|
||||
- save(value, metadata, agent) -> client.retain(bank_id, content)
|
||||
- search(query, limit) -> client.recall(bank_id, query)
|
||||
- reset() -> client.delete_bank() + recreate
|
||||
|
||||
Args:
|
||||
bank_id: The Hindsight memory bank ID for this crew.
|
||||
hindsight_api_url: Override the configured API URL.
|
||||
api_key: Override the configured API key.
|
||||
budget: Recall budget level (low/mid/high). Overrides config.
|
||||
max_tokens: Max recall tokens. Overrides config.
|
||||
tags: Tags for retain operations. Overrides config.
|
||||
recall_tags: Tags to filter recall. Overrides config.
|
||||
recall_tags_match: Tag matching mode. Overrides config.
|
||||
per_agent_banks: If True, each agent gets its own bank
|
||||
(bank_id is suffixed with sanitized agent role). Default False.
|
||||
bank_resolver: Custom callable (bank_id, agent) -> resolved_bank_id.
|
||||
Overrides per_agent_banks if provided.
|
||||
mission: If provided, creates/updates the bank with this mission.
|
||||
verbose: Enable verbose logging. Overrides config.
|
||||
|
||||
Example::
|
||||
|
||||
from hindsight_crewai import configure, HindsightStorage
|
||||
from crewai.memory.external.external_memory import ExternalMemory
|
||||
|
||||
configure(hindsight_api_url="http://localhost:8888")
|
||||
storage = HindsightStorage(bank_id="my-crew")
|
||||
external_memory = ExternalMemory(storage=storage)
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
bank_id: str,
|
||||
hindsight_api_url: str | None = None,
|
||||
api_key: str | None = None,
|
||||
budget: str | None = None,
|
||||
max_tokens: int | None = None,
|
||||
tags: list[str] | None = None,
|
||||
recall_tags: list[str] | None = None,
|
||||
recall_tags_match: str | None = None,
|
||||
per_agent_banks: bool = False,
|
||||
bank_resolver: Callable[[str, str | None], str] | None = None,
|
||||
mission: str | None = None,
|
||||
verbose: bool | None = None,
|
||||
):
|
||||
self._bank_id = bank_id
|
||||
self._per_agent_banks = per_agent_banks
|
||||
self._bank_resolver = bank_resolver
|
||||
self._mission = mission
|
||||
self._local = threading.local()
|
||||
self._created_banks: set[str] = set()
|
||||
|
||||
# Resolve settings: constructor args override global config
|
||||
config = get_config()
|
||||
self._api_url = hindsight_api_url or (
|
||||
config.hindsight_api_url if config else "http://localhost:8888"
|
||||
)
|
||||
self._api_key = api_key or (config.api_key if config else None)
|
||||
self._budget = budget or (config.budget if config else "mid")
|
||||
self._max_tokens = max_tokens or (config.max_tokens if config else 4096)
|
||||
self._tags = tags or (config.tags if config else None)
|
||||
self._recall_tags = recall_tags or (config.recall_tags if config else None)
|
||||
self._recall_tags_match = recall_tags_match or (
|
||||
config.recall_tags_match if config else "any"
|
||||
)
|
||||
self._verbose = (
|
||||
verbose if verbose is not None else (config.verbose if config else False)
|
||||
)
|
||||
|
||||
# Eagerly create the default bank if mission is provided
|
||||
if mission:
|
||||
self._ensure_bank(self._bank_id)
|
||||
|
||||
def _get_client(self) -> Any:
|
||||
"""Get or create a thread-local Hindsight client.
|
||||
|
||||
Each thread gets its own client so the underlying aiohttp
|
||||
session stays bound to that thread's event loop.
|
||||
"""
|
||||
client = getattr(self._local, "client", None)
|
||||
if client is None:
|
||||
from hindsight_client import Hindsight
|
||||
|
||||
client = Hindsight(
|
||||
base_url=self._api_url,
|
||||
api_key=self._api_key,
|
||||
timeout=30.0,
|
||||
)
|
||||
self._local.client = client
|
||||
return client
|
||||
|
||||
def _resolve_bank_id(self, agent: str | None = None) -> str:
|
||||
"""Resolve the effective bank_id for this operation.
|
||||
|
||||
If bank_resolver is provided, delegates to it.
|
||||
If per_agent_banks=True and agent is provided, uses
|
||||
``f"{bank_id}-{sanitized_agent}"``.
|
||||
Otherwise returns the base bank_id.
|
||||
"""
|
||||
if self._bank_resolver:
|
||||
return self._bank_resolver(self._bank_id, agent)
|
||||
if self._per_agent_banks and agent:
|
||||
sanitized = agent.lower().replace(" ", "-")
|
||||
return f"{self._bank_id}-{sanitized}"
|
||||
return self._bank_id
|
||||
|
||||
def _ensure_bank(self, bank_id: str) -> None:
|
||||
"""Create bank if not already created in this session."""
|
||||
if bank_id in self._created_banks:
|
||||
return
|
||||
|
||||
def _create() -> None:
|
||||
client = self._get_client()
|
||||
client.create_bank(
|
||||
bank_id=bank_id,
|
||||
name=bank_id,
|
||||
mission=self._mission,
|
||||
)
|
||||
|
||||
try:
|
||||
call_sync(_create)
|
||||
self._created_banks.add(bank_id)
|
||||
if self._verbose:
|
||||
logger.info(f"Created/updated bank: {bank_id}")
|
||||
except Exception as e:
|
||||
# Bank may already exist — that's fine
|
||||
self._created_banks.add(bank_id)
|
||||
if self._verbose:
|
||||
logger.warning(f"Bank creation for {bank_id}: {e}")
|
||||
|
||||
def save(
|
||||
self,
|
||||
value: str,
|
||||
metadata: dict[str, Any] | None = None,
|
||||
agent: str | None = None,
|
||||
) -> None:
|
||||
"""Store a memory to Hindsight.
|
||||
|
||||
Called by CrewAI automatically after each task completes.
|
||||
|
||||
Args:
|
||||
value: The task output text to store.
|
||||
metadata: Optional metadata dict from CrewAI.
|
||||
agent: Optional agent role/name that produced this output.
|
||||
|
||||
Raises:
|
||||
HindsightError: If the retain operation fails.
|
||||
"""
|
||||
bank_id = self._resolve_bank_id(agent)
|
||||
self._ensure_bank(bank_id)
|
||||
|
||||
# Build retain metadata — Hindsight requires dict[str, str]
|
||||
retain_metadata: dict[str, str] = {"source": "crewai"}
|
||||
if agent:
|
||||
retain_metadata["agent"] = agent
|
||||
if metadata:
|
||||
for k, v in metadata.items():
|
||||
retain_metadata[k] = str(v)
|
||||
|
||||
def _retain() -> None:
|
||||
self._get_client().retain(
|
||||
bank_id=bank_id,
|
||||
content=value,
|
||||
context=f"crewai:task_output:{agent or 'unknown'}",
|
||||
metadata=retain_metadata,
|
||||
tags=self._tags,
|
||||
)
|
||||
|
||||
try:
|
||||
call_sync(_retain)
|
||||
if self._verbose:
|
||||
logger.info(
|
||||
f"Stored memory to bank {bank_id} (agent={agent}, len={len(value)})"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to store memory: {e}")
|
||||
raise HindsightError(f"Failed to store memory: {e}") from e
|
||||
|
||||
def search(
|
||||
self,
|
||||
query: str,
|
||||
limit: int = 10,
|
||||
score_threshold: float = 0.5,
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Search memories in Hindsight via recall.
|
||||
|
||||
Called by CrewAI automatically at the start of each task.
|
||||
|
||||
Args:
|
||||
query: The search query (constructed by CrewAI from task description).
|
||||
limit: Maximum results to return.
|
||||
score_threshold: Minimum relevance score (0-1).
|
||||
|
||||
Returns:
|
||||
List of dicts with keys: context, score, metadata.
|
||||
|
||||
Raises:
|
||||
HindsightError: If the recall operation fails.
|
||||
"""
|
||||
bank_id = self._resolve_bank_id(agent=None)
|
||||
|
||||
recall_kwargs: dict[str, Any] = {
|
||||
"bank_id": bank_id,
|
||||
"query": query,
|
||||
"budget": self._budget,
|
||||
"max_tokens": self._max_tokens,
|
||||
}
|
||||
if self._recall_tags:
|
||||
recall_kwargs["tags"] = self._recall_tags
|
||||
recall_kwargs["tags_match"] = self._recall_tags_match
|
||||
|
||||
def _recall() -> Any:
|
||||
return self._get_client().recall(**recall_kwargs)
|
||||
|
||||
try:
|
||||
response = call_sync(_recall)
|
||||
|
||||
# Convert RecallResponse to CrewAI's expected list[dict] format.
|
||||
results: list[dict[str, Any]] = []
|
||||
recall_results = response.results if hasattr(response, "results") else []
|
||||
total = max(len(recall_results), 1)
|
||||
|
||||
for i, r in enumerate(recall_results[:limit]):
|
||||
# Hindsight returns results ordered by relevance.
|
||||
# Assign descending synthetic scores.
|
||||
score = 1.0 - (i / total)
|
||||
|
||||
if score < score_threshold:
|
||||
break
|
||||
|
||||
result_metadata: dict[str, Any] = {}
|
||||
if r.type:
|
||||
result_metadata["type"] = r.type
|
||||
if r.context:
|
||||
result_metadata["source_context"] = r.context
|
||||
if r.occurred_start:
|
||||
result_metadata["occurred_start"] = r.occurred_start
|
||||
if r.document_id:
|
||||
result_metadata["document_id"] = r.document_id
|
||||
if r.metadata:
|
||||
result_metadata.update(r.metadata)
|
||||
if r.tags:
|
||||
result_metadata["tags"] = r.tags
|
||||
|
||||
results.append(
|
||||
{
|
||||
"context": r.text,
|
||||
"score": round(score, 4),
|
||||
"metadata": result_metadata,
|
||||
}
|
||||
)
|
||||
|
||||
if self._verbose:
|
||||
logger.info(
|
||||
f"Recalled {len(results)} memories from bank {bank_id} for query: {query[:80]}"
|
||||
)
|
||||
|
||||
return results
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to search memories: {e}")
|
||||
raise HindsightError(f"Failed to search memories: {e}") from e
|
||||
|
||||
def reset(self) -> None:
|
||||
"""Clear all memories by deleting and recreating the bank.
|
||||
|
||||
This removes all facts, entities, and mental models from the
|
||||
bank. The bank is recreated with its original mission if one
|
||||
was provided.
|
||||
"""
|
||||
|
||||
def _delete() -> None:
|
||||
self._get_client().delete_bank(self._bank_id)
|
||||
|
||||
try:
|
||||
call_sync(_delete)
|
||||
self._created_banks.discard(self._bank_id)
|
||||
|
||||
if self._verbose:
|
||||
logger.info(f"Reset bank: {self._bank_id}")
|
||||
|
||||
# Recreate the bank if mission was set
|
||||
if self._mission:
|
||||
self._ensure_bank(self._bank_id)
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to reset bank: {e}")
|
||||
# Don't raise — reset is best-effort
|
||||
@@ -0,0 +1,121 @@
|
||||
"""CrewAI Tool for Hindsight reflect operations.
|
||||
|
||||
Since CrewAI's Storage interface only has save/search/reset,
|
||||
reflect is exposed as a Tool that agents can call explicitly.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import threading
|
||||
from typing import Any
|
||||
|
||||
from crewai.tools import BaseTool
|
||||
from pydantic import Field, PrivateAttr
|
||||
|
||||
from ._compat import call_sync
|
||||
from .config import get_config
|
||||
from .errors import HindsightError
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class HindsightReflectTool(BaseTool):
|
||||
"""CrewAI tool that generates disposition-aware answers from memory.
|
||||
|
||||
Unlike recall (search), reflect synthesizes a coherent, reasoned
|
||||
response using the bank's personality/disposition and all relevant
|
||||
memories. Use this when agents need a thoughtful, contextual answer
|
||||
rather than raw memory facts.
|
||||
|
||||
Args:
|
||||
bank_id: The Hindsight memory bank to reflect against.
|
||||
hindsight_api_url: Override the configured API URL.
|
||||
api_key: Override the configured API key.
|
||||
budget: Reflect budget level (low/mid/high).
|
||||
reflect_context: Additional context for reflect reasoning.
|
||||
|
||||
Example::
|
||||
|
||||
from hindsight_crewai import HindsightReflectTool
|
||||
from crewai import Agent
|
||||
|
||||
reflect_tool = HindsightReflectTool(
|
||||
bank_id="my-crew",
|
||||
budget="mid",
|
||||
)
|
||||
agent = Agent(role="Analyst", tools=[reflect_tool], ...)
|
||||
"""
|
||||
|
||||
name: str = "hindsight_reflect"
|
||||
description: str = (
|
||||
"Generate a thoughtful, synthesized answer about a topic by reflecting "
|
||||
"on all relevant memories. Use this when you need a coherent summary "
|
||||
"of what you know, not just raw facts. Input: a question or topic."
|
||||
)
|
||||
|
||||
bank_id: str = Field(description="Hindsight memory bank ID")
|
||||
hindsight_api_url: str | None = Field(default=None, description="Override API URL")
|
||||
api_key: str | None = Field(default=None, description="Override API key")
|
||||
budget: str = Field(default="mid", description="Reflect budget (low/mid/high)")
|
||||
reflect_context: str | None = Field(
|
||||
default=None, description="Additional context for reflect reasoning"
|
||||
)
|
||||
|
||||
_local: Any = PrivateAttr(default_factory=threading.local)
|
||||
|
||||
def _get_client(self) -> Any:
|
||||
"""Get or create a thread-local Hindsight client."""
|
||||
client = getattr(self._local, "client", None)
|
||||
if client is None:
|
||||
from hindsight_client import Hindsight
|
||||
|
||||
config = get_config()
|
||||
api_url = self.hindsight_api_url or (
|
||||
config.hindsight_api_url if config else "http://localhost:8888"
|
||||
)
|
||||
api_key = self.api_key or (config.api_key if config else None)
|
||||
|
||||
client = Hindsight(
|
||||
base_url=api_url,
|
||||
api_key=api_key,
|
||||
timeout=30.0,
|
||||
)
|
||||
self._local.client = client
|
||||
return client
|
||||
|
||||
def _run(self, query: str) -> str:
|
||||
"""Execute the reflect tool.
|
||||
|
||||
Args:
|
||||
query: The question or topic to reflect on.
|
||||
|
||||
Returns:
|
||||
The synthesized reflect response text.
|
||||
|
||||
Raises:
|
||||
HindsightError: If the reflect operation fails.
|
||||
"""
|
||||
reflect_kwargs: dict[str, Any] = {
|
||||
"bank_id": self.bank_id,
|
||||
"query": query,
|
||||
"budget": self.budget,
|
||||
}
|
||||
if self.reflect_context:
|
||||
reflect_kwargs["context"] = self.reflect_context
|
||||
|
||||
def _reflect() -> Any:
|
||||
return self._get_client().reflect(**reflect_kwargs)
|
||||
|
||||
try:
|
||||
result = call_sync(_reflect)
|
||||
text = result.text if hasattr(result, "text") else str(result)
|
||||
|
||||
if not text:
|
||||
return "No relevant memories found to reflect on."
|
||||
|
||||
return text
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Reflect failed: {e}")
|
||||
raise HindsightError(f"Reflect failed: {e}") from e
|
||||
@@ -0,0 +1,58 @@
|
||||
[project]
|
||||
name = "hindsight-crewai"
|
||||
version = "0.1.0"
|
||||
description = "CrewAI memory integration via Hindsight - persistent memory for AI agent crews"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10"
|
||||
license = { text = "MIT" }
|
||||
authors = [
|
||||
{ name = "Vectorize", email = "[email protected]" }
|
||||
]
|
||||
keywords = [
|
||||
"ai",
|
||||
"memory",
|
||||
"crewai",
|
||||
"agents",
|
||||
"hindsight",
|
||||
]
|
||||
classifiers = [
|
||||
"Development Status :: 4 - Beta",
|
||||
"Intended Audience :: Developers",
|
||||
"License :: OSI Approved :: MIT License",
|
||||
"Programming Language :: Python :: 3",
|
||||
"Programming Language :: Python :: 3.10",
|
||||
"Programming Language :: Python :: 3.11",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
"Topic :: Scientific/Engineering :: Artificial Intelligence",
|
||||
]
|
||||
|
||||
dependencies = [
|
||||
"crewai>=0.86.0",
|
||||
"hindsight-client>=0.4.0",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
"pytest>=7.0.0",
|
||||
"pytest-mock>=3.10.0",
|
||||
]
|
||||
|
||||
[project.urls]
|
||||
Homepage = "https://github.com/vectorize-io/hindsight"
|
||||
Documentation = "https://github.com/vectorize-io/hindsight/tree/main/hindsight-integrations/crewai"
|
||||
Repository = "https://github.com/vectorize-io/hindsight"
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[tool.hatch.build.targets.wheel]
|
||||
packages = ["hindsight_crewai"]
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
|
||||
[dependency-groups]
|
||||
dev = [
|
||||
"pytest>=9.0.2",
|
||||
]
|
||||
@@ -0,0 +1,116 @@
|
||||
"""Manual integration test for hindsight-crewai.
|
||||
|
||||
Prerequisites:
|
||||
1. Hindsight API running on localhost:8888 (./scripts/dev/start-api.sh)
|
||||
2. OPENAI_API_KEY set (or configure CrewAI for another LLM provider)
|
||||
3. uv pip install -e . (from this directory)
|
||||
|
||||
Usage:
|
||||
uv run python test_manual.py
|
||||
"""
|
||||
|
||||
from hindsight_crewai import configure, HindsightStorage, HindsightReflectTool
|
||||
from crewai.memory.external.external_memory import ExternalMemory
|
||||
from crewai import Agent, Crew, Task
|
||||
|
||||
BANK_ID = "crewai-test"
|
||||
HINDSIGHT_URL = "http://localhost:8888"
|
||||
|
||||
# --- Configure ---
|
||||
|
||||
configure(hindsight_api_url=HINDSIGHT_URL, verbose=True)
|
||||
|
||||
storage = HindsightStorage(
|
||||
bank_id=BANK_ID,
|
||||
mission="Track research findings and summaries for a software team.",
|
||||
)
|
||||
|
||||
reflect_tool = HindsightReflectTool(bank_id=BANK_ID, budget="mid")
|
||||
|
||||
# --- Smoke test (no LLM needed) ---
|
||||
|
||||
print("=== SMOKE TEST: save/search/reset ===\n")
|
||||
|
||||
storage.save("Python is great for data science", metadata={"task": "research"}, agent="Tester")
|
||||
print("Saved memory.")
|
||||
|
||||
results = storage.search("What programming languages are useful?")
|
||||
print(f"Search returned {len(results)} result(s):")
|
||||
for r in results:
|
||||
print(f" - [{r['score']}] {r['context']}")
|
||||
|
||||
print()
|
||||
|
||||
# --- Full crew test ---
|
||||
|
||||
print("=== RUN 1: Initial research ===\n")
|
||||
|
||||
researcher = Agent(
|
||||
role="Researcher",
|
||||
goal="Research topics and remember findings",
|
||||
backstory="You are a diligent researcher who remembers everything.",
|
||||
tools=[reflect_tool],
|
||||
verbose=True,
|
||||
)
|
||||
|
||||
writer = Agent(
|
||||
role="Writer",
|
||||
goal="Write summaries based on research",
|
||||
backstory="You write clear, concise summaries.",
|
||||
tools=[reflect_tool],
|
||||
verbose=True,
|
||||
)
|
||||
|
||||
research_task = Task(
|
||||
description=(
|
||||
"Research the benefits of functional programming. "
|
||||
"List at least 3 key benefits with examples."
|
||||
),
|
||||
expected_output="A list of functional programming benefits with examples.",
|
||||
agent=researcher,
|
||||
)
|
||||
|
||||
summary_task = Task(
|
||||
description="Write a one-paragraph summary of the research findings.",
|
||||
expected_output="A concise summary paragraph.",
|
||||
agent=writer,
|
||||
)
|
||||
|
||||
crew = Crew(
|
||||
agents=[researcher, writer],
|
||||
tasks=[research_task, summary_task],
|
||||
external_memory=ExternalMemory(storage=storage),
|
||||
verbose=True,
|
||||
)
|
||||
|
||||
result = crew.kickoff()
|
||||
print(f"\nRun 1 result:\n{result}\n")
|
||||
|
||||
# --- Second run: recall from memory ---
|
||||
|
||||
print("=== RUN 2: Recall from memory ===\n")
|
||||
|
||||
recall_task = Task(
|
||||
description=(
|
||||
"What do you already know about functional programming from previous research? "
|
||||
"Use the hindsight_reflect tool to check your memories."
|
||||
),
|
||||
expected_output="A summary of what was previously learned.",
|
||||
agent=researcher,
|
||||
)
|
||||
|
||||
crew2 = Crew(
|
||||
agents=[researcher],
|
||||
tasks=[recall_task],
|
||||
external_memory=ExternalMemory(storage=storage),
|
||||
verbose=True,
|
||||
)
|
||||
|
||||
result2 = crew2.kickoff()
|
||||
print(f"\nRun 2 result:\n{result2}\n")
|
||||
|
||||
# --- Cleanup ---
|
||||
|
||||
print("=== CLEANUP ===\n")
|
||||
storage.reset()
|
||||
print("Bank reset. Done.")
|
||||
@@ -0,0 +1,76 @@
|
||||
"""Unit tests for hindsight_crewai configuration."""
|
||||
|
||||
import os
|
||||
from unittest.mock import patch
|
||||
|
||||
from hindsight_crewai import configure, get_config, reset_config
|
||||
from hindsight_crewai.config import (
|
||||
DEFAULT_HINDSIGHT_API_URL,
|
||||
HINDSIGHT_API_KEY_ENV,
|
||||
)
|
||||
|
||||
|
||||
class TestDefaults:
|
||||
def test_default_api_url(self):
|
||||
assert DEFAULT_HINDSIGHT_API_URL == "https://api.hindsight.vectorize.io"
|
||||
|
||||
def test_env_var_name(self):
|
||||
assert HINDSIGHT_API_KEY_ENV == "HINDSIGHT_API_KEY"
|
||||
|
||||
|
||||
class TestConfigure:
|
||||
def setup_method(self):
|
||||
reset_config()
|
||||
|
||||
def teardown_method(self):
|
||||
reset_config()
|
||||
|
||||
def test_configure_with_no_arguments(self):
|
||||
config = configure()
|
||||
assert config.hindsight_api_url == DEFAULT_HINDSIGHT_API_URL
|
||||
assert config.budget == "mid"
|
||||
assert config.max_tokens == 4096
|
||||
assert config.verbose is False
|
||||
|
||||
def test_configure_reads_api_key_from_env(self):
|
||||
with patch.dict(os.environ, {HINDSIGHT_API_KEY_ENV: "test-key"}):
|
||||
config = configure()
|
||||
assert config.api_key == "test-key"
|
||||
|
||||
def test_configure_explicit_overrides_env(self):
|
||||
with patch.dict(os.environ, {HINDSIGHT_API_KEY_ENV: "env-key"}):
|
||||
config = configure(api_key="explicit-key")
|
||||
assert config.api_key == "explicit-key"
|
||||
|
||||
def test_configure_all_options(self):
|
||||
config = configure(
|
||||
hindsight_api_url="http://custom:8888",
|
||||
api_key="my-key",
|
||||
budget="high",
|
||||
max_tokens=2048,
|
||||
tags=["env:test"],
|
||||
recall_tags=["scope:global"],
|
||||
recall_tags_match="all",
|
||||
verbose=True,
|
||||
)
|
||||
assert config.hindsight_api_url == "http://custom:8888"
|
||||
assert config.api_key == "my-key"
|
||||
assert config.budget == "high"
|
||||
assert config.max_tokens == 2048
|
||||
assert config.tags == ["env:test"]
|
||||
assert config.recall_tags == ["scope:global"]
|
||||
assert config.recall_tags_match == "all"
|
||||
assert config.verbose is True
|
||||
|
||||
def test_get_config_returns_none_without_configure(self):
|
||||
assert get_config() is None
|
||||
|
||||
def test_get_config_returns_config_after_configure(self):
|
||||
configure()
|
||||
assert get_config() is not None
|
||||
|
||||
def test_reset_config(self):
|
||||
configure()
|
||||
assert get_config() is not None
|
||||
reset_config()
|
||||
assert get_config() is None
|
||||
@@ -0,0 +1,259 @@
|
||||
"""Unit tests for HindsightStorage."""
|
||||
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from hindsight_crewai import HindsightStorage, configure, reset_config
|
||||
from hindsight_crewai.errors import HindsightError
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
def _passthrough(fn, *args, **kwargs):
|
||||
"""Replace call_sync with direct call for testing."""
|
||||
return fn(*args, **kwargs)
|
||||
|
||||
|
||||
class TestHindsightStorage:
|
||||
def setup_method(self):
|
||||
reset_config()
|
||||
configure(hindsight_api_url="http://localhost:8888")
|
||||
|
||||
def teardown_method(self):
|
||||
reset_config()
|
||||
|
||||
def _make_storage(self, **kwargs):
|
||||
"""Create a storage instance with a mocked client."""
|
||||
storage = HindsightStorage(bank_id="test-bank", **kwargs)
|
||||
mock_client = MagicMock()
|
||||
storage._local.client = mock_client
|
||||
storage._created_banks.add("test-bank")
|
||||
return storage, mock_client
|
||||
|
||||
def _make_recall_result(self, text="Memory text", type_="world", **kwargs):
|
||||
"""Create a mock RecallResult."""
|
||||
r = MagicMock()
|
||||
r.text = text
|
||||
r.type = type_
|
||||
r.context = kwargs.get("context")
|
||||
r.occurred_start = kwargs.get("occurred_start")
|
||||
r.document_id = kwargs.get("document_id")
|
||||
r.metadata = kwargs.get("metadata")
|
||||
r.tags = kwargs.get("tags")
|
||||
return r
|
||||
|
||||
# --- save() tests ---
|
||||
|
||||
@patch("hindsight_crewai.storage.call_sync", side_effect=_passthrough)
|
||||
def test_save_calls_retain(self, _mock_cs):
|
||||
storage, mock_client = self._make_storage()
|
||||
|
||||
storage.save("Task output text", metadata={"task": "research"}, agent="Researcher")
|
||||
|
||||
mock_client.retain.assert_called_once()
|
||||
call_kwargs = mock_client.retain.call_args[1]
|
||||
assert call_kwargs["bank_id"] == "test-bank"
|
||||
assert call_kwargs["content"] == "Task output text"
|
||||
assert call_kwargs["metadata"]["source"] == "crewai"
|
||||
assert call_kwargs["metadata"]["agent"] == "Researcher"
|
||||
assert call_kwargs["metadata"]["task"] == "research"
|
||||
|
||||
@patch("hindsight_crewai.storage.call_sync", side_effect=_passthrough)
|
||||
def test_save_stringifies_metadata_values(self, _mock_cs):
|
||||
storage, mock_client = self._make_storage()
|
||||
|
||||
storage.save("text", metadata={"count": 42, "active": True})
|
||||
|
||||
call_kwargs = mock_client.retain.call_args[1]
|
||||
assert call_kwargs["metadata"]["count"] == "42"
|
||||
assert call_kwargs["metadata"]["active"] == "True"
|
||||
|
||||
@patch("hindsight_crewai.storage.call_sync", side_effect=_passthrough)
|
||||
def test_save_without_metadata(self, _mock_cs):
|
||||
storage, mock_client = self._make_storage()
|
||||
|
||||
storage.save("text")
|
||||
|
||||
call_kwargs = mock_client.retain.call_args[1]
|
||||
assert call_kwargs["metadata"] == {"source": "crewai"}
|
||||
assert call_kwargs["context"] == "crewai:task_output:unknown"
|
||||
|
||||
@patch("hindsight_crewai.storage.call_sync", side_effect=_passthrough)
|
||||
def test_save_raises_hindsight_error_on_failure(self, _mock_cs):
|
||||
storage, mock_client = self._make_storage()
|
||||
mock_client.retain.side_effect = RuntimeError("connection refused")
|
||||
|
||||
with pytest.raises(HindsightError, match="Failed to store memory"):
|
||||
storage.save("text")
|
||||
|
||||
@patch("hindsight_crewai.storage.call_sync", side_effect=_passthrough)
|
||||
def test_save_passes_tags(self, _mock_cs):
|
||||
storage, mock_client = self._make_storage(tags=["env:prod"])
|
||||
|
||||
storage.save("text")
|
||||
|
||||
call_kwargs = mock_client.retain.call_args[1]
|
||||
assert call_kwargs["tags"] == ["env:prod"]
|
||||
|
||||
# --- search() tests ---
|
||||
|
||||
@patch("hindsight_crewai.storage.call_sync", side_effect=_passthrough)
|
||||
def test_search_calls_recall(self, _mock_cs):
|
||||
storage, mock_client = self._make_storage()
|
||||
|
||||
mock_response = MagicMock()
|
||||
mock_response.results = [self._make_recall_result()]
|
||||
mock_client.recall.return_value = mock_response
|
||||
|
||||
results = storage.search("programming preferences", limit=5)
|
||||
|
||||
mock_client.recall.assert_called_once()
|
||||
assert len(results) == 1
|
||||
assert results[0]["context"] == "Memory text"
|
||||
assert "score" in results[0]
|
||||
assert results[0]["metadata"]["type"] == "world"
|
||||
|
||||
@patch("hindsight_crewai.storage.call_sync", side_effect=_passthrough)
|
||||
def test_search_respects_limit(self, _mock_cs):
|
||||
storage, mock_client = self._make_storage()
|
||||
|
||||
mock_response = MagicMock()
|
||||
mock_response.results = [self._make_recall_result(text=f"Memory {i}") for i in range(10)]
|
||||
mock_client.recall.return_value = mock_response
|
||||
|
||||
results = storage.search("test", limit=3)
|
||||
assert len(results) == 3
|
||||
|
||||
@patch("hindsight_crewai.storage.call_sync", side_effect=_passthrough)
|
||||
def test_search_returns_empty_on_no_results(self, _mock_cs):
|
||||
storage, mock_client = self._make_storage()
|
||||
|
||||
mock_response = MagicMock()
|
||||
mock_response.results = []
|
||||
mock_client.recall.return_value = mock_response
|
||||
|
||||
results = storage.search("nonexistent topic")
|
||||
assert results == []
|
||||
|
||||
@patch("hindsight_crewai.storage.call_sync", side_effect=_passthrough)
|
||||
def test_search_includes_rich_metadata(self, _mock_cs):
|
||||
storage, mock_client = self._make_storage()
|
||||
|
||||
r = self._make_recall_result(
|
||||
context="conversation",
|
||||
occurred_start="2024-01-01",
|
||||
document_id="doc-1",
|
||||
metadata={"key": "value"},
|
||||
tags=["tag1"],
|
||||
)
|
||||
mock_response = MagicMock()
|
||||
mock_response.results = [r]
|
||||
mock_client.recall.return_value = mock_response
|
||||
|
||||
results = storage.search("test")
|
||||
meta = results[0]["metadata"]
|
||||
assert meta["source_context"] == "conversation"
|
||||
assert meta["occurred_start"] == "2024-01-01"
|
||||
assert meta["document_id"] == "doc-1"
|
||||
assert meta["key"] == "value"
|
||||
assert meta["tags"] == ["tag1"]
|
||||
|
||||
@patch("hindsight_crewai.storage.call_sync", side_effect=_passthrough)
|
||||
def test_search_passes_recall_tags(self, _mock_cs):
|
||||
storage, mock_client = self._make_storage(recall_tags=["scope:global"], recall_tags_match="all")
|
||||
|
||||
mock_response = MagicMock()
|
||||
mock_response.results = []
|
||||
mock_client.recall.return_value = mock_response
|
||||
|
||||
storage.search("test")
|
||||
|
||||
call_kwargs = mock_client.recall.call_args[1]
|
||||
assert call_kwargs["tags"] == ["scope:global"]
|
||||
assert call_kwargs["tags_match"] == "all"
|
||||
|
||||
@patch("hindsight_crewai.storage.call_sync", side_effect=_passthrough)
|
||||
def test_search_raises_hindsight_error_on_failure(self, _mock_cs):
|
||||
storage, mock_client = self._make_storage()
|
||||
mock_client.recall.side_effect = RuntimeError("timeout")
|
||||
|
||||
with pytest.raises(HindsightError, match="Failed to search memories"):
|
||||
storage.search("test")
|
||||
|
||||
@patch("hindsight_crewai.storage.call_sync", side_effect=_passthrough)
|
||||
def test_search_synthetic_scores_descend(self, _mock_cs):
|
||||
storage, mock_client = self._make_storage()
|
||||
|
||||
mock_response = MagicMock()
|
||||
mock_response.results = [self._make_recall_result(text=f"Memory {i}") for i in range(5)]
|
||||
mock_client.recall.return_value = mock_response
|
||||
|
||||
results = storage.search("test", limit=5, score_threshold=0.0)
|
||||
scores = [r["score"] for r in results]
|
||||
assert scores == sorted(scores, reverse=True)
|
||||
assert scores[0] == 1.0
|
||||
|
||||
# --- reset() tests ---
|
||||
|
||||
@patch("hindsight_crewai.storage.call_sync", side_effect=_passthrough)
|
||||
def test_reset_deletes_bank(self, _mock_cs):
|
||||
storage, mock_client = self._make_storage()
|
||||
|
||||
storage.reset()
|
||||
|
||||
mock_client.delete_bank.assert_called_once_with("test-bank")
|
||||
|
||||
@patch("hindsight_crewai.storage.call_sync", side_effect=_passthrough)
|
||||
def test_reset_is_best_effort(self, _mock_cs):
|
||||
storage, mock_client = self._make_storage()
|
||||
mock_client.delete_bank.side_effect = RuntimeError("not found")
|
||||
|
||||
# Should not raise
|
||||
storage.reset()
|
||||
|
||||
# --- per-agent banks ---
|
||||
|
||||
def test_per_agent_banks_resolves_bank_id(self):
|
||||
storage = HindsightStorage(bank_id="crew", per_agent_banks=True)
|
||||
assert storage._resolve_bank_id("Researcher") == "crew-researcher"
|
||||
assert storage._resolve_bank_id("Data Analyst") == "crew-data-analyst"
|
||||
assert storage._resolve_bank_id(None) == "crew"
|
||||
|
||||
def test_custom_bank_resolver(self):
|
||||
resolver = lambda base, agent: f"custom-{agent}" if agent else base
|
||||
storage = HindsightStorage(bank_id="crew", bank_resolver=resolver)
|
||||
assert storage._resolve_bank_id("Alice") == "custom-Alice"
|
||||
assert storage._resolve_bank_id(None) == "crew"
|
||||
|
||||
@patch("hindsight_crewai.storage.call_sync", side_effect=_passthrough)
|
||||
def test_save_uses_per_agent_bank(self, _mock_cs):
|
||||
storage = HindsightStorage(bank_id="crew", per_agent_banks=True)
|
||||
mock_client = MagicMock()
|
||||
storage._local.client = mock_client
|
||||
storage._created_banks.add("crew-researcher")
|
||||
|
||||
storage.save("output", agent="Researcher")
|
||||
|
||||
call_kwargs = mock_client.retain.call_args[1]
|
||||
assert call_kwargs["bank_id"] == "crew-researcher"
|
||||
|
||||
# --- config resolution ---
|
||||
|
||||
def test_constructor_overrides_config(self):
|
||||
configure(budget="low", max_tokens=1024)
|
||||
storage = HindsightStorage(bank_id="test", budget="high", max_tokens=8192)
|
||||
assert storage._budget == "high"
|
||||
assert storage._max_tokens == 8192
|
||||
|
||||
def test_falls_back_to_config(self):
|
||||
configure(budget="high", max_tokens=2048, verbose=True)
|
||||
storage = HindsightStorage(bank_id="test")
|
||||
assert storage._budget == "high"
|
||||
assert storage._max_tokens == 2048
|
||||
assert storage._verbose is True
|
||||
|
||||
def test_falls_back_to_defaults_without_config(self):
|
||||
reset_config()
|
||||
storage = HindsightStorage(bank_id="test")
|
||||
assert storage._api_url == "http://localhost:8888"
|
||||
assert storage._budget == "mid"
|
||||
assert storage._max_tokens == 4096
|
||||
@@ -0,0 +1,97 @@
|
||||
"""Unit tests for HindsightReflectTool."""
|
||||
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from hindsight_crewai import HindsightReflectTool, configure, reset_config
|
||||
from hindsight_crewai.errors import HindsightError
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
def _passthrough(fn, *args, **kwargs):
|
||||
"""Replace call_sync with direct call for testing."""
|
||||
return fn(*args, **kwargs)
|
||||
|
||||
|
||||
class TestReflectTool:
|
||||
def setup_method(self):
|
||||
reset_config()
|
||||
configure(hindsight_api_url="http://localhost:8888")
|
||||
|
||||
def teardown_method(self):
|
||||
reset_config()
|
||||
|
||||
@patch("hindsight_crewai.tools.call_sync", side_effect=_passthrough)
|
||||
def test_reflect_returns_text(self, _mock_cs):
|
||||
tool = HindsightReflectTool(bank_id="test-bank")
|
||||
mock_client = MagicMock()
|
||||
mock_result = MagicMock()
|
||||
mock_result.text = "The user is a Python developer who..."
|
||||
mock_client.reflect.return_value = mock_result
|
||||
tool._local.client = mock_client
|
||||
|
||||
result = tool._run("What do you know about the user?")
|
||||
|
||||
assert result == "The user is a Python developer who..."
|
||||
mock_client.reflect.assert_called_once()
|
||||
|
||||
@patch("hindsight_crewai.tools.call_sync", side_effect=_passthrough)
|
||||
def test_reflect_empty_returns_fallback(self, _mock_cs):
|
||||
tool = HindsightReflectTool(bank_id="test-bank")
|
||||
mock_client = MagicMock()
|
||||
mock_result = MagicMock()
|
||||
mock_result.text = ""
|
||||
mock_client.reflect.return_value = mock_result
|
||||
tool._local.client = mock_client
|
||||
|
||||
result = tool._run("anything")
|
||||
assert "No relevant memories" in result
|
||||
|
||||
@patch("hindsight_crewai.tools.call_sync", side_effect=_passthrough)
|
||||
def test_reflect_passes_context(self, _mock_cs):
|
||||
tool = HindsightReflectTool(
|
||||
bank_id="test-bank",
|
||||
reflect_context="Agent is a delivery robot.",
|
||||
)
|
||||
mock_client = MagicMock()
|
||||
mock_result = MagicMock()
|
||||
mock_result.text = "Synthesized answer"
|
||||
mock_client.reflect.return_value = mock_result
|
||||
tool._local.client = mock_client
|
||||
|
||||
tool._run("Where is Alice?")
|
||||
|
||||
call_kwargs = mock_client.reflect.call_args[1]
|
||||
assert call_kwargs["context"] == "Agent is a delivery robot."
|
||||
assert call_kwargs["bank_id"] == "test-bank"
|
||||
assert call_kwargs["budget"] == "mid"
|
||||
|
||||
@patch("hindsight_crewai.tools.call_sync", side_effect=_passthrough)
|
||||
def test_reflect_passes_budget(self, _mock_cs):
|
||||
tool = HindsightReflectTool(bank_id="test-bank", budget="high")
|
||||
mock_client = MagicMock()
|
||||
mock_result = MagicMock()
|
||||
mock_result.text = "answer"
|
||||
mock_client.reflect.return_value = mock_result
|
||||
tool._local.client = mock_client
|
||||
|
||||
tool._run("query")
|
||||
|
||||
call_kwargs = mock_client.reflect.call_args[1]
|
||||
assert call_kwargs["budget"] == "high"
|
||||
|
||||
@patch("hindsight_crewai.tools.call_sync", side_effect=_passthrough)
|
||||
def test_reflect_raises_hindsight_error_on_failure(self, _mock_cs):
|
||||
tool = HindsightReflectTool(bank_id="test-bank")
|
||||
mock_client = MagicMock()
|
||||
mock_client.reflect.side_effect = RuntimeError("timeout")
|
||||
tool._local.client = mock_client
|
||||
|
||||
with pytest.raises(HindsightError, match="Reflect failed"):
|
||||
tool._run("query")
|
||||
|
||||
def test_tool_metadata(self):
|
||||
tool = HindsightReflectTool(bank_id="test-bank")
|
||||
assert tool.name == "hindsight_reflect"
|
||||
assert "reflect" in tool.description.lower()
|
||||
assert "memories" in tool.description.lower()
|
||||
Generated
+3909
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user