Compare commits

..
4 Commits
Author SHA1 Message Date
Nicolò BoschiandClaude Sonnet 4.5 6f2fa9ae1d test: update consolidation test for source_memory_ids behavior
Updated test_consolidation_creates_memory_links to test_consolidation_uses_source_memory_ids
to reflect the new behavior where observations use source_memory_ids instead of memory_links
for traversal.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
2026-01-28 12:20:37 +01:00
Nicolò BoschiandClaude Sonnet 4.5 7039ab44a7 fix: observations rely on source_memory_ids, no link copying
Observations no longer copy any memory_links from their source facts.
Instead, retrieval uses source_memory_ids to traverse:
- Entity connections: observation → source_memory_ids → unit_entities
- Semantic similarity: observations have their own embeddings
- Temporal proximity: observations have their own temporal fields

This avoids data duplication and fixes bidirectionality issues with
entity links being copied to observations.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
2026-01-28 12:08:27 +01:00
Nicolò Boschi 1a2969724d fix tests 2026-01-28 11:12:27 +01:00
Nicolò Boschi 3ce8e557e4 chore: cleanup benchmarks runner with old flags 2026-01-28 09:54:20 +01:00
191 changed files with 523 additions and 10398 deletions
-1
View File
@@ -26,7 +26,6 @@ HINDSIGHT_API_LOG_LEVEL=info
# Database (Optional - uses embedded pg0 by default)
# HINDSIGHT_API_DATABASE_URL=postgresql://user:pass@host:5432/db
# HINDSIGHT_API_DATABASE_SCHEMA=public # PostgreSQL schema name (default: public)
# Embeddings Configuration (Optional - uses local by default)
# Provider: "local" (default) or "tei" (HuggingFace Text Embeddings Inference)
+1 -4
View File
@@ -45,12 +45,9 @@ hindsight-docs/static/llms-full.txt
hindsight-dev/benchmarks/locomo/results/
hindsight-dev/benchmarks/longmemeval/results/
hindsight-dev/benchmarks/consolidation/results/
benchmarks/results/
hindsight-cli/target
hindsight-clients/rust/target
.claude
whats-next.md
TASK.md
# Changelog is now tracked in hindsight-docs/src/pages/changelog.md
# CHANGELOG.md
CHANGELOG.md
+40 -53
View File
@@ -1,6 +1,6 @@
<div align="center">
![Hindsight Banner](./hindsight-docs/static/img/hindsight-github-banner.png)
![Hindsight Banner](./hindsight-docs/static/img/banner.svg)
[Documentation](https://hindsight.vectorize.io) • [Paper](https://arxiv.org/abs/2512.12818) • [Cookbook](https://hindsight.vectorize.io/cookbook) • [Hindsight Cloud](https://vectorize.io/hindsight/cloud)
@@ -17,31 +17,55 @@
## What is Hindsight?
Hindsight™ is an agent memory system built to create smarter agents that learn over time. Most agent memory systems focus on recalling conversation history. Hindsight is focused on making agents that learn, not just remember.
Hindsight™ is an agent memory system built to create smarter agents that learn over time. It eliminates the shortcomings of alternative techniques such as RAG and knowledge graph and delivers state-of-the-art performance on long term memory tasks.
Hindsight addresses common challenges that have frustrated AI engineers building agents to automate tasks and assist users with conversational interfaces. Many of these challenges stem directly from a lack of memory.
<video src="https://github.com/user-attachments/assets/923b798d-3581-4897-bb62-9cfa5a931682" controls></video>
- **Inconsistency:** Agents complete tasks successfully one time, then fail when asked to complete the same task again. Memory gives the agent a mechanism to remember what worked and what didn't and to use that information to reduce errors and improve consistency.
- **Hallucinations:** Long term memory can be seeded with external knowledge to ground agent behavior in reliable sources to augment training data.
- **Cognitive Overload:** As workflows get complex, retrievals, tool calls, user messages and agent responses can grow to fill the context window leading to context rot. Short term memory optimization allows agents to reduce tokens and focus context by removing irrelevant details.
It eliminates the shortcomings of alternative techniques such as RAG and knowledge graph and delivers state-of-the-art performance on long term memory tasks.
## How is Hindsight Different From Other Memory Systems?
![Overview](./hindsight-docs/static/img/hindsight-overview.webp)
Most agent memory implementation rely on basic vector search or sometimes use a knowledge graph. Hindsight uses biomimetic data structures to organize agent memories in a way that is more like how human memory works:
- **World:** Facts about the world ("The stove gets hot")
- **Experiences:** Agent's own experiences ("I touched the stove and it really hurt")
- **Opinion:** Beliefs with confidence scores ("I shouldn't touch the stove again" - .99 confidence)
- **Observation:** Complex mental models derived by reflecting on facts and experiences ("Curling irons, ovens, and fire are also hot. I shouldn't touch those either.")
Memories in Hindsight are stored in banks (i.e. memory banks). When memories are added to Hindsight, they are pushed into either the world facts or experiences memory pathway. They are then represented as a combination of entities, relationships, and time series with sparse/dense vector representations to aid in later recall.
Hindsight provides three simple methods to interact with the system:
- **Retain:** Provide information to Hindsight that you want it to remember
- **Recall:** Retrieve memories from Hindsight
- **Reflect:** Reflect on memories and experiences to generate new observations and insights from existing memories.
### Agent Memory That Learns
A key goal of Hindsight is to build agent memory that enables agents to learn and improve over time. This is the role of the `reflect` operation which provides the agent to form broader opinions and observations over time.
For example, imagine a product support agent that is helping a user troubleshoot a problem. It uses a `search-documentation` tool it found on an MCP server. Later in the conversation, the agent discovers that the documentation returned from the tool wasn't for the product the user was asking about. The agent now has an experience in its memory bank. And just like humans, we want that agent to learn from its experience.
As the agent gains more experiences, `reflect` allows the agent to form observations about what worked, what didn't, and what to do differently the next time it encounters a similar task.
---
## Memory Performance & Accuracy
Hindsight is the most accurate agent memory system ever tested according to benchmark performance. It has achieved state-of-the-art performance on the LongMemEval benchmark, widely used to assess memory system performance across a variety of conversational AI scenarios. The current reported performance of Hindsight and other agent memory solutions as of January 2026 is shown here:
Hindsight has achieved state-of-the-art performance on the LongMemEval benchmark, widely used to assess memory system performance across a variety of conversational
AI scenarios. The current reported performance of Hindsight and other agent memory solutions as of December 2025 is shown here:
![Overview](./hindsight-docs/static/img/hindsight-bench.jpg)
The benchmark performance data for Hindsight has been independently reproduced by research collaborators at the Virginia Tech [Sanghani Center for Artificial Intelligence and Data Analytics](https://sanghani.cs.vt.edu/) and The Washington Post. Other scores are self-reported by software vendors.
The benchmark performance data for Hindsight and GPT-4o (full context) have been reproduced by research collaborators at the Virginia Tech [Sanghani Center for Artificial Intelligence and Data Analytics](https://sanghani.cs.vt.edu/) and The Washington Post. Other scores are self-reported by software vendors.
Hindsight is being used in production at Fortune 500 enterprises and by a growing number of AI startups.
## Adding Hindsight to Your AI Agents
The easiest way use Hindsight with an existing agent is with the LLM Wrapper. You can add memory to your agent with 2 lines of code. That will swap your current LLM client out with the Hindsight wrapper. After that, memories will be stored and retrieved automatically as you make LLM calls.
If you need more control over how and when your agent stores and recalls memories, there's also a simple API you can integrate with using the SDKs or directly via HTTP.
![Hindsight Banner](./hindsight-docs/static/img/migration-code.png)
A thorough examination of the techniques implemented in Hindsight and detailed breakdowns of benchmark performance are [available on arXiv](https://arxiv.org/abs/2512.12818). This research is currently being prepared for conference submission and the wider peer review process.
The benchmark results from this research can be inspected in our [visual benchmark explorer](https://hindsight-benchmarks.vercel.app). As additional improvements are made to Hindsight, new benchmark data will be available for review using this same tool.
## Quick Start
@@ -124,45 +148,8 @@ await client.recall('my-bank', 'What does Alice like?');
---
## Use Cases
Hindsight is built to support conversational AI agents as well as agents that are intended to perform tasks autonomously. The ideal use case for Hindsight are agents that require a blend of these features such as AI employees that need to handle open-ended tasks, change behavior based on user feedback, and learn to perform complex tasks to automate work at a level that approximates a human work. Hindsight can be used with simple AI workflows like those built with n8n and other similar tools, but may be overkill for such applications.
### Per-User Memories and Chat History
One of the simpler use cases you can use Hindsight for is to personalize AI chatbots and other conversational agents by storing and recalling memories associated with individual users.
The requirements for this use case usually look something like this:
![Per-User Memories](./hindsight-docs/static/img/per-user-memory-requirements.png)
<video src="https://github.com/user-attachments/assets/4805e8e1-e7d1-47c6-a4f8-2344a5ec8906" controls></video>
Satisfying these requirements in Hindsight is straightforward. When new user inputs and tool calls are ingested into Hindsight using the retain operation, custom metadata can be used to enrich the new memories. Metadata provides a convenient way to isolate memories that need to be restricted to a given user. Once these are fed into the retain operation, any raw memories and mental models that get created can be filtered when retrieving relevant memories.
![Per-User Memories](./hindsight-docs/static/img/per-user-memory-howto.png)
---
## Architecture & Operations
![Overview](./hindsight-docs/static/img/hindsight-overview.webp)
Most agent memory implementation rely on basic vector search or sometimes use a knowledge graph. Hindsight uses biomimetic data structures to organize agent memories in a way that is more like how human memory works:
- **World:** Facts about the world ("The stove gets hot")
- **Experiences:** Agent's own experiences ("I touched the stove and it really hurt")
- **Mental Models:** Learned understanding of the agent's world formed by reflecting on raw memories and experiences.
Memories in Hindsight are stored in banks (i.e. memory banks). When memories are added to Hindsight, they are pushed into either the world facts or experiences memory pathway. They are then represented as a combination of entities, relationships, and time series with sparse/dense vector representations to aid in later recall.
Hindsight provides three simple methods to interact with the system:
- **Retain:** Provide information to Hindsight that you want it to remember
- **Recall:** Retrieve memories from Hindsight
- **Reflect:** Reflect on memories and experiences to generate new observations and insights from existing memories.
### Retain
The `retain` operation is used to push new memories into Hindsight. It tells Hindsight to _retain_ the information you pass in as an input.
@@ -221,7 +208,7 @@ The final output is trimmed as needed to fit within the token limit.
### Reflect
The reflect operation is used to perform a more thorough analysis of existing memories. This allows the agent to form new connections between memories and build a more thorough understanding of its world.
The reflect operation is used to perform a more thorough analysis of existing memories. This allows the agent to form new connections between memories which are then persisted as opinions and/or observations. When building agents, the reflect operation is a key capability to enable the agent to learn from its experiences.
For example, the `reflect` operation can be used to support use cases such as:
+2 -2
View File
@@ -2,8 +2,8 @@ apiVersion: v2
name: hindsight
description: Hindsight helm chart
type: application
version: 0.4.1
appVersion: "0.4.1"
version: 0.3.0
appVersion: "0.3.0"
keywords:
- ai
- memory
+1 -1
View File
@@ -46,4 +46,4 @@ __all__ = [
"RemoteTEICrossEncoder",
"LLMConfig",
]
__version__ = "0.4.1"
__version__ = "0.1.0"
+2 -3
View File
@@ -1323,7 +1323,7 @@ class VersionResponse(BaseModel):
model_config = ConfigDict(
json_schema_extra={
"example": {
"api_version": "0.4.0",
"api_version": "1.0.0",
"features": {
"observations": False,
"mcp": True,
@@ -1567,12 +1567,11 @@ def _register_routes(app: FastAPI):
Returns version info and feature flags that can be used by clients
to determine which capabilities are available.
"""
from hindsight_api import __version__
from hindsight_api.config import get_config
config = get_config()
return VersionResponse(
api_version=__version__,
api_version="1.0.0",
features=FeaturesInfo(
observations=config.enable_observations,
mcp=config.mcp_enabled,
+3 -35
View File
@@ -20,7 +20,6 @@ logger = logging.getLogger(__name__)
# Environment variable names
ENV_DATABASE_URL = "HINDSIGHT_API_DATABASE_URL"
ENV_DATABASE_SCHEMA = "HINDSIGHT_API_DATABASE_SCHEMA"
ENV_LLM_PROVIDER = "HINDSIGHT_API_LLM_PROVIDER"
ENV_LLM_API_KEY = "HINDSIGHT_API_LLM_API_KEY"
ENV_LLM_MODEL = "HINDSIGHT_API_LLM_MODEL"
@@ -47,7 +46,6 @@ ENV_CONSOLIDATION_LLM_BASE_URL = "HINDSIGHT_API_CONSOLIDATION_LLM_BASE_URL"
ENV_EMBEDDINGS_PROVIDER = "HINDSIGHT_API_EMBEDDINGS_PROVIDER"
ENV_EMBEDDINGS_LOCAL_MODEL = "HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL"
ENV_EMBEDDINGS_LOCAL_FORCE_CPU = "HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU"
ENV_EMBEDDINGS_TEI_URL = "HINDSIGHT_API_EMBEDDINGS_TEI_URL"
ENV_EMBEDDINGS_OPENAI_API_KEY = "HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY"
ENV_EMBEDDINGS_OPENAI_MODEL = "HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL"
@@ -67,7 +65,6 @@ ENV_RERANKER_LITELLM_MODEL = "HINDSIGHT_API_RERANKER_LITELLM_MODEL"
ENV_RERANKER_PROVIDER = "HINDSIGHT_API_RERANKER_PROVIDER"
ENV_RERANKER_LOCAL_MODEL = "HINDSIGHT_API_RERANKER_LOCAL_MODEL"
ENV_RERANKER_LOCAL_FORCE_CPU = "HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU"
ENV_RERANKER_LOCAL_MAX_CONCURRENT = "HINDSIGHT_API_RERANKER_LOCAL_MAX_CONCURRENT"
ENV_RERANKER_TEI_URL = "HINDSIGHT_API_RERANKER_TEI_URL"
ENV_RERANKER_TEI_BATCH_SIZE = "HINDSIGHT_API_RERANKER_TEI_BATCH_SIZE"
@@ -95,13 +92,11 @@ ENV_RETAIN_MAX_COMPLETION_TOKENS = "HINDSIGHT_API_RETAIN_MAX_COMPLETION_TOKENS"
ENV_RETAIN_CHUNK_SIZE = "HINDSIGHT_API_RETAIN_CHUNK_SIZE"
ENV_RETAIN_EXTRACT_CAUSAL_LINKS = "HINDSIGHT_API_RETAIN_EXTRACT_CAUSAL_LINKS"
ENV_RETAIN_EXTRACTION_MODE = "HINDSIGHT_API_RETAIN_EXTRACTION_MODE"
ENV_RETAIN_CUSTOM_INSTRUCTIONS = "HINDSIGHT_API_RETAIN_CUSTOM_INSTRUCTIONS"
ENV_RETAIN_OBSERVATIONS_ASYNC = "HINDSIGHT_API_RETAIN_OBSERVATIONS_ASYNC"
# Observations settings (consolidated knowledge from facts)
ENV_ENABLE_OBSERVATIONS = "HINDSIGHT_API_ENABLE_OBSERVATIONS"
ENV_CONSOLIDATION_BATCH_SIZE = "HINDSIGHT_API_CONSOLIDATION_BATCH_SIZE"
ENV_CONSOLIDATION_MAX_TOKENS = "HINDSIGHT_API_CONSOLIDATION_MAX_TOKENS"
# Optimization flags
ENV_SKIP_LLM_VERIFICATION = "HINDSIGHT_API_SKIP_LLM_VERIFICATION"
@@ -129,7 +124,6 @@ ENV_REFLECT_MAX_ITERATIONS = "HINDSIGHT_API_REFLECT_MAX_ITERATIONS"
# Default values
DEFAULT_DATABASE_URL = "pg0"
DEFAULT_DATABASE_SCHEMA = "public"
DEFAULT_LLM_PROVIDER = "openai"
DEFAULT_LLM_MODEL = "gpt-5-mini"
DEFAULT_LLM_MAX_CONCURRENT = 32
@@ -137,13 +131,11 @@ DEFAULT_LLM_TIMEOUT = 120.0 # seconds
DEFAULT_EMBEDDINGS_PROVIDER = "local"
DEFAULT_EMBEDDINGS_LOCAL_MODEL = "BAAI/bge-small-en-v1.5"
DEFAULT_EMBEDDINGS_LOCAL_FORCE_CPU = False # Force CPU mode for local embeddings (avoids MPS/XPC issues on macOS)
DEFAULT_EMBEDDINGS_OPENAI_MODEL = "text-embedding-3-small"
DEFAULT_EMBEDDING_DIMENSION = 384
DEFAULT_RERANKER_PROVIDER = "local"
DEFAULT_RERANKER_LOCAL_MODEL = "cross-encoder/ms-marco-MiniLM-L-6-v2"
DEFAULT_RERANKER_LOCAL_FORCE_CPU = False # Force CPU mode for local reranker (avoids MPS/XPC issues on macOS)
DEFAULT_RERANKER_LOCAL_MAX_CONCURRENT = 4 # Limit concurrent CPU-bound reranking to prevent thrashing
DEFAULT_RERANKER_TEI_BATCH_SIZE = 128
DEFAULT_RERANKER_TEI_MAX_CONCURRENT = 8
@@ -176,15 +168,13 @@ DEFAULT_MENTAL_MODEL_REFRESH_CONCURRENCY = 8 # Max concurrent mental model refr
DEFAULT_RETAIN_MAX_COMPLETION_TOKENS = 64000 # Max tokens for fact extraction LLM call
DEFAULT_RETAIN_CHUNK_SIZE = 3000 # Max chars per chunk for fact extraction
DEFAULT_RETAIN_EXTRACT_CAUSAL_LINKS = True # Extract causal links between facts
DEFAULT_RETAIN_EXTRACTION_MODE = "concise" # Extraction mode: "concise", "verbose", or "custom"
RETAIN_EXTRACTION_MODES = ("concise", "verbose", "custom") # Allowed extraction modes
DEFAULT_RETAIN_CUSTOM_INSTRUCTIONS = None # Custom extraction guidelines (only used when mode="custom")
DEFAULT_RETAIN_EXTRACTION_MODE = "concise" # Extraction mode: "concise" or "verbose"
RETAIN_EXTRACTION_MODES = ("concise", "verbose") # Allowed extraction modes
DEFAULT_RETAIN_OBSERVATIONS_ASYNC = False # Run observation generation async (after retain completes)
# Observations defaults (consolidated knowledge from facts)
DEFAULT_ENABLE_OBSERVATIONS = True # Observations enabled by default
DEFAULT_CONSOLIDATION_BATCH_SIZE = 50 # Memories to load per batch (internal memory optimization)
DEFAULT_CONSOLIDATION_MAX_TOKENS = 1024 # Max tokens for recall when finding related observations
# Database migrations
DEFAULT_RUN_MIGRATIONS_ON_STARTUP = True
@@ -278,7 +268,6 @@ class HindsightConfig:
# Database
database_url: str
database_schema: str
# LLM (default, used as fallback for per-operation config)
llm_provider: str
@@ -307,7 +296,6 @@ class HindsightConfig:
# Embeddings
embeddings_provider: str
embeddings_local_model: str
embeddings_local_force_cpu: bool
embeddings_tei_url: str | None
embeddings_openai_base_url: str | None
embeddings_cohere_base_url: str | None
@@ -315,8 +303,6 @@ class HindsightConfig:
# Reranker
reranker_provider: str
reranker_local_model: str
reranker_local_force_cpu: bool
reranker_local_max_concurrent: int
reranker_tei_url: str | None
reranker_tei_batch_size: int
reranker_tei_max_concurrent: int
@@ -342,13 +328,11 @@ class HindsightConfig:
retain_chunk_size: int
retain_extract_causal_links: bool
retain_extraction_mode: str
retain_custom_instructions: str | None
retain_observations_async: bool
# Observations settings (consolidated knowledge from facts)
enable_observations: bool
consolidation_batch_size: int
consolidation_max_tokens: int
# Optimization flags
skip_llm_verification: bool
@@ -380,7 +364,6 @@ class HindsightConfig:
return cls(
# Database
database_url=os.getenv(ENV_DATABASE_URL, DEFAULT_DATABASE_URL),
database_schema=os.getenv(ENV_DATABASE_SCHEMA, DEFAULT_DATABASE_SCHEMA),
# LLM
llm_provider=os.getenv(ENV_LLM_PROVIDER, DEFAULT_LLM_PROVIDER),
llm_api_key=os.getenv(ENV_LLM_API_KEY),
@@ -404,23 +387,12 @@ class HindsightConfig:
# Embeddings
embeddings_provider=os.getenv(ENV_EMBEDDINGS_PROVIDER, DEFAULT_EMBEDDINGS_PROVIDER),
embeddings_local_model=os.getenv(ENV_EMBEDDINGS_LOCAL_MODEL, DEFAULT_EMBEDDINGS_LOCAL_MODEL),
embeddings_local_force_cpu=os.getenv(
ENV_EMBEDDINGS_LOCAL_FORCE_CPU, str(DEFAULT_EMBEDDINGS_LOCAL_FORCE_CPU)
).lower()
in ("true", "1"),
embeddings_tei_url=os.getenv(ENV_EMBEDDINGS_TEI_URL),
embeddings_openai_base_url=os.getenv(ENV_EMBEDDINGS_OPENAI_BASE_URL) or None,
embeddings_cohere_base_url=os.getenv(ENV_EMBEDDINGS_COHERE_BASE_URL) or None,
# Reranker
reranker_provider=os.getenv(ENV_RERANKER_PROVIDER, DEFAULT_RERANKER_PROVIDER),
reranker_local_model=os.getenv(ENV_RERANKER_LOCAL_MODEL, DEFAULT_RERANKER_LOCAL_MODEL),
reranker_local_force_cpu=os.getenv(
ENV_RERANKER_LOCAL_FORCE_CPU, str(DEFAULT_RERANKER_LOCAL_FORCE_CPU)
).lower()
in ("true", "1"),
reranker_local_max_concurrent=int(
os.getenv(ENV_RERANKER_LOCAL_MAX_CONCURRENT, str(DEFAULT_RERANKER_LOCAL_MAX_CONCURRENT))
),
reranker_tei_url=os.getenv(ENV_RERANKER_TEI_URL),
reranker_tei_batch_size=int(os.getenv(ENV_RERANKER_TEI_BATCH_SIZE, str(DEFAULT_RERANKER_TEI_BATCH_SIZE))),
reranker_tei_max_concurrent=int(
@@ -459,7 +431,6 @@ class HindsightConfig:
retain_extraction_mode=_validate_extraction_mode(
os.getenv(ENV_RETAIN_EXTRACTION_MODE, DEFAULT_RETAIN_EXTRACTION_MODE)
),
retain_custom_instructions=os.getenv(ENV_RETAIN_CUSTOM_INSTRUCTIONS) or DEFAULT_RETAIN_CUSTOM_INSTRUCTIONS,
retain_observations_async=os.getenv(
ENV_RETAIN_OBSERVATIONS_ASYNC, str(DEFAULT_RETAIN_OBSERVATIONS_ASYNC)
).lower()
@@ -469,9 +440,6 @@ class HindsightConfig:
consolidation_batch_size=int(
os.getenv(ENV_CONSOLIDATION_BATCH_SIZE, str(DEFAULT_CONSOLIDATION_BATCH_SIZE))
),
consolidation_max_tokens=int(
os.getenv(ENV_CONSOLIDATION_MAX_TOKENS, str(DEFAULT_CONSOLIDATION_MAX_TOKENS))
),
# Database migrations
run_migrations_on_startup=os.getenv(ENV_RUN_MIGRATIONS_ON_STARTUP, "true").lower() == "true",
# Database connection pool
@@ -543,7 +511,7 @@ class HindsightConfig:
def log_config(self) -> None:
"""Log the current configuration (without sensitive values)."""
logger.info(f"Database: {self.database_url} (schema: {self.database_schema})")
logger.info(f"Database: {self.database_url}")
logger.info(f"LLM: provider={self.llm_provider}, model={self.llm_model}")
if self.retain_llm_provider or self.retain_llm_model:
retain_provider = self.retain_llm_provider or self.llm_provider
+1 -4
View File
@@ -52,10 +52,7 @@ class IdleTimeoutMiddleware:
logger.info(f"Idle timeout reached ({self.idle_timeout}s), shutting down daemon")
# Give a moment for any in-flight requests
await asyncio.sleep(1)
# Send SIGTERM to ourselves to trigger graceful shutdown
import signal
os.kill(os.getpid(), signal.SIGTERM)
os._exit(0)
class DaemonLock:
@@ -639,27 +639,28 @@ async def _find_related_observations(
request_context: "RequestContext",
) -> list[dict[str, Any]]:
"""
Find observations related to the given query using optimized recall.
Find observations related to the given query using the full recall system.
IMPORTANT: We do NOT filter by tags here. Consolidation needs to see ALL
potentially related observations regardless of scope, so the LLM can
decide on tag routing (same scope update vs cross-scope create).
Uses max_tokens to naturally limit observations (no artificial count limit).
Includes source memories with dates for LLM context.
This leverages:
- Semantic search (embedding similarity)
- BM25 text search (keyword matching)
- Entity-based retrieval (shared entities)
- Graph traversal (connected via entity links)
Returns:
List of related observations with their tags, source memories, and dates
List of related observations with their tags for LLM tag routing
"""
# Use recall to find related observations with token budget
# max_tokens naturally limits how many observations are returned
from ...config import get_config
config = get_config()
# Use recall to find related observations
# NO tags parameter - we want ALL observations regardless of scope
# Use low max_tokens since we only need observations, not memories
recall_result = await memory_engine.recall_async(
bank_id=bank_id,
query=query,
max_tokens=config.consolidation_max_tokens, # Token budget for observations (configurable)
max_tokens=5000, # Token budget for observations
fact_type=["observation"], # Only retrieve observations
request_context=request_context,
_quiet=True, # Suppress logging
@@ -667,82 +668,43 @@ async def _find_related_observations(
)
# If no observations returned, return empty list
# When fact_type=["observation"], results come back in `results` field
if not recall_result.results:
return []
# Batch fetch all observations in a single query (no artificial limit)
observation_ids = [uuid.UUID(obs.id) for obs in recall_result.results]
rows = await conn.fetch(
f"""
SELECT id, text, proof_count, history, tags, source_memory_ids, created_at, updated_at,
occurred_start, occurred_end, mentioned_at
FROM {fq_table("memory_units")}
WHERE id = ANY($1) AND bank_id = $2 AND fact_type = 'observation'
""",
observation_ids,
bank_id,
)
# Build results list preserving recall order
id_to_row = {row["id"]: row for row in rows}
# Trust recall's relevance filtering - fetch full data for each observation
results = []
for obs in recall_result.results:
obs_id = uuid.UUID(obs.id)
if obs_id not in id_to_row:
continue
row = id_to_row[obs_id]
history = row["history"]
if isinstance(history, str):
history = json.loads(history)
elif history is None:
history = []
# Fetch source memories to include their text and dates
source_memory_ids = row["source_memory_ids"] or []
source_memories = []
if source_memory_ids:
source_rows = await conn.fetch(
f"""
SELECT text, occurred_start, occurred_end, mentioned_at, event_date
FROM {fq_table("memory_units")}
WHERE id = ANY($1) AND bank_id = $2
ORDER BY created_at ASC
LIMIT 5
""",
source_memory_ids[:5], # Limit to first 5 source memories for token efficiency
bank_id,
)
for src_row in source_rows:
source_memories.append(
{
"text": src_row["text"],
"occurred_start": src_row["occurred_start"],
"occurred_end": src_row["occurred_end"],
"mentioned_at": src_row["mentioned_at"],
"event_date": src_row["event_date"],
}
)
results.append(
{
"id": row["id"],
"text": row["text"],
"proof_count": row["proof_count"] or 1,
"tags": row["tags"] or [],
"source_memories": source_memories,
"occurred_start": row["occurred_start"],
"occurred_end": row["occurred_end"],
"mentioned_at": row["mentioned_at"],
"created_at": row["created_at"],
"updated_at": row["updated_at"],
}
# Fetch full observation data from DB to get history, source_memory_ids, tags
row = await conn.fetchrow(
f"""
SELECT id, text, proof_count, history, tags, source_memory_ids, created_at, updated_at
FROM {fq_table("memory_units")}
WHERE id = $1 AND bank_id = $2 AND fact_type = 'observation'
""",
uuid.UUID(obs.id),
bank_id,
)
if row:
history = row["history"]
if isinstance(history, str):
history = json.loads(history)
elif history is None:
history = []
results.append(
{
"id": row["id"],
"text": row["text"],
"proof_count": row["proof_count"] or 1,
"history": history,
"tags": row["tags"] or [], # Include tags for LLM tag routing
"source_memory_ids": row["source_memory_ids"] or [],
"similarity": 1.0, # Retrieved via recall so assumed relevant
}
)
return results
@@ -770,43 +732,14 @@ async def _consolidate_with_llm(
- {"action": "create", "text": "...", "reason": "..."}
- [] if fact is purely ephemeral (no durable knowledge)
"""
# Format observations as JSON with source memories and dates
# Format observations WITH their tags (or "None" if empty)
if observations:
obs_list = []
for obs in observations:
obs_data = {
"id": str(obs["id"]),
"text": obs["text"],
"proof_count": obs["proof_count"],
"tags": obs["tags"],
"created_at": obs["created_at"].isoformat() if obs.get("created_at") else None,
"updated_at": obs["updated_at"].isoformat() if obs.get("updated_at") else None,
}
# Include temporal info if available
if obs.get("occurred_start"):
obs_data["occurred_start"] = obs["occurred_start"].isoformat()
if obs.get("occurred_end"):
obs_data["occurred_end"] = obs["occurred_end"].isoformat()
if obs.get("mentioned_at"):
obs_data["mentioned_at"] = obs["mentioned_at"].isoformat()
# Include source memories (up to 3 for brevity)
if obs.get("source_memories"):
obs_data["source_memories"] = [
{
"text": sm["text"],
"event_date": sm["event_date"].isoformat() if sm.get("event_date") else None,
"occurred_start": sm["occurred_start"].isoformat() if sm.get("occurred_start") else None,
}
for sm in obs["source_memories"][:3] # Limit to 3 for token efficiency
]
obs_list.append(obs_data)
observations_text = json.dumps(obs_list, indent=2)
observations_text = "\n".join(
f'- ID: {obs["id"]}, Tags: {json.dumps(obs["tags"])}, Text: "{obs["text"]}" (proof_count: {obs["proof_count"]})'
for obs in observations
)
else:
observations_text = "[]"
observations_text = "None (this is a new topic - create if fact contains durable knowledge)"
# Only include mission section if mission is set and not the default
mission_section = ""
@@ -47,31 +47,23 @@ CONSOLIDATION_USER_PROMPT = """Analyze this new fact and consolidate into knowle
{mission_section}
NEW FACT: {fact_text}
EXISTING OBSERVATIONS (JSON array with source memories and dates):
EXISTING OBSERVATIONS:
{observations_text}
Each observation includes:
- id: unique identifier for updating
- text: the observation content
- proof_count: number of supporting memories
- tags: visibility scope (handled automatically)
- created_at/updated_at: when observation was created/modified
- occurred_start/occurred_end: temporal range of source facts
- source_memories: array of supporting facts with their text and dates
Instructions:
1. Extract DURABLE KNOWLEDGE from the new fact (not ephemeral state)
2. Review source_memories in existing observations to understand evidence
3. Check dates to detect contradictions or updates
4. Compare with observations:
- Same topic → UPDATE with learning_id
- New topic → CREATE new observation
- Purely ephemeral → return []
1. First, extract the DURABLE KNOWLEDGE from the fact (not ephemeral state like "user is at X")
2. Then compare with existing observations:
- If an observation covers the same topic: UPDATE it with the new knowledge
- If no observation covers the topic: CREATE a new one
Output JSON array of actions:
Output JSON array of actions (ALWAYS an array, even for single action):
[
{{"action": "update", "learning_id": "uuid-from-observations", "text": "updated knowledge", "reason": "..."}},
{{"action": "update", "learning_id": "uuid", "text": "updated durable knowledge", "reason": "..."}},
{{"action": "create", "text": "new durable knowledge", "reason": "..."}}
]
Return [] if fact contains no durable knowledge."""
If NO consolidation is needed (fact is purely ephemeral with no durable knowledge):
[]
If no observations exist and fact contains durable knowledge:
[{{"action": "create", "text": "durable knowledge text", "reason": "new topic"}}]"""
@@ -20,7 +20,6 @@ from ..config import (
DEFAULT_RERANKER_FLASHRANK_CACHE_DIR,
DEFAULT_RERANKER_FLASHRANK_MODEL,
DEFAULT_RERANKER_LITELLM_MODEL,
DEFAULT_RERANKER_LOCAL_FORCE_CPU,
DEFAULT_RERANKER_LOCAL_MAX_CONCURRENT,
DEFAULT_RERANKER_LOCAL_MODEL,
DEFAULT_RERANKER_PROVIDER,
@@ -34,7 +33,6 @@ from ..config import (
ENV_RERANKER_FLASHRANK_CACHE_DIR,
ENV_RERANKER_FLASHRANK_MODEL,
ENV_RERANKER_LITELLM_MODEL,
ENV_RERANKER_LOCAL_FORCE_CPU,
ENV_RERANKER_LOCAL_MAX_CONCURRENT,
ENV_RERANKER_LOCAL_MODEL,
ENV_RERANKER_PROVIDER,
@@ -101,7 +99,7 @@ class LocalSTCrossEncoder(CrossEncoderModel):
_executor: ThreadPoolExecutor | None = None
_max_concurrent: int = 4 # Limit concurrent CPU-bound reranking calls
def __init__(self, model_name: str | None = None, max_concurrent: int = 4, force_cpu: bool = False):
def __init__(self, model_name: str | None = None, max_concurrent: int = 4):
"""
Initialize local SentenceTransformers cross-encoder.
@@ -110,11 +108,8 @@ class LocalSTCrossEncoder(CrossEncoderModel):
Default: cross-encoder/ms-marco-MiniLM-L-6-v2
max_concurrent: Maximum concurrent reranking calls (default: 2).
Higher values may cause CPU thrashing under load.
force_cpu: Force CPU mode (avoids MPS/XPC issues on macOS in daemon mode).
Default: False
"""
self.model_name = model_name or DEFAULT_RERANKER_LOCAL_MODEL
self.force_cpu = force_cpu
self._model = None
LocalSTCrossEncoder._max_concurrent = max_concurrent
@@ -144,23 +139,13 @@ class LocalSTCrossEncoder(CrossEncoderModel):
# after loading, which conflicts with accelerate's device_map handling.
import torch
# Force CPU mode if configured (used in daemon mode to avoid MPS/XPC issues on macOS)
if self.force_cpu:
device = "cpu"
logger.info("Reranker: forcing CPU mode (HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU=1)")
# Check for GPU (CUDA) or Apple Silicon (MPS)
has_gpu = torch.cuda.is_available() or (hasattr(torch.backends, "mps") and torch.backends.mps.is_available())
if has_gpu:
device = None # Let sentence-transformers auto-detect GPU/MPS
else:
# Check for GPU (CUDA) or Apple Silicon (MPS)
# Wrap in try-except to gracefully handle any device detection issues
# (e.g., in CI environments or when PyTorch is built without GPU support)
device = "cpu" # Default to CPU
try:
has_gpu = torch.cuda.is_available() or (
hasattr(torch.backends, "mps") and torch.backends.mps.is_available()
)
if has_gpu:
device = None # Let sentence-transformers auto-detect GPU/MPS
except Exception as e:
logger.warning(f"Failed to detect GPU/MPS, falling back to CPU: {e}")
device = "cpu"
self._model = CrossEncoder(
self.model_name,
@@ -178,11 +163,6 @@ class LocalSTCrossEncoder(CrossEncoderModel):
else:
logger.info("Reranker: local provider initialized (using existing executor)")
def _predict_sync(self, pairs: list[tuple[str, str]]) -> list[float]:
"""Synchronous prediction wrapper for thread pool execution."""
scores = self._model.predict(pairs, show_progress_bar=False)
return scores.tolist() if hasattr(scores, "tolist") else list(scores)
async def predict(self, pairs: list[tuple[str, str]]) -> list[float]:
"""
Score query-document pairs for relevance.
@@ -200,11 +180,11 @@ class LocalSTCrossEncoder(CrossEncoderModel):
# Use dedicated executor - limited workers naturally limits concurrency
loop = asyncio.get_event_loop()
return await loop.run_in_executor(
scores = await loop.run_in_executor(
LocalSTCrossEncoder._executor,
self._predict_sync,
pairs,
lambda: self._model.predict(pairs, show_progress_bar=False),
)
return scores.tolist() if hasattr(scores, "tolist") else list(scores)
class RemoteTEICrossEncoder(CrossEncoderModel):
@@ -803,33 +783,29 @@ class LiteLLMCrossEncoder(CrossEncoderModel):
def create_cross_encoder_from_env() -> CrossEncoderModel:
"""
Create a CrossEncoderModel instance based on configuration.
Create a CrossEncoderModel instance based on environment variables.
Reads configuration via get_config() to ensure consistency across the codebase.
See hindsight_api.config for environment variable names and defaults.
Returns:
Configured CrossEncoderModel instance
"""
from ..config import get_config
config = get_config()
provider = config.reranker_provider.lower()
provider = os.environ.get(ENV_RERANKER_PROVIDER, DEFAULT_RERANKER_PROVIDER).lower()
if provider == "tei":
url = config.reranker_tei_url
url = os.environ.get(ENV_RERANKER_TEI_URL)
if not url:
raise ValueError(f"{ENV_RERANKER_TEI_URL} is required when {ENV_RERANKER_PROVIDER} is 'tei'")
return RemoteTEICrossEncoder(
base_url=url,
batch_size=config.reranker_tei_batch_size,
max_concurrent=config.reranker_tei_max_concurrent,
)
batch_size = int(os.environ.get(ENV_RERANKER_TEI_BATCH_SIZE, str(DEFAULT_RERANKER_TEI_BATCH_SIZE)))
max_concurrent = int(os.environ.get(ENV_RERANKER_TEI_MAX_CONCURRENT, str(DEFAULT_RERANKER_TEI_MAX_CONCURRENT)))
return RemoteTEICrossEncoder(base_url=url, batch_size=batch_size, max_concurrent=max_concurrent)
elif provider == "local":
return LocalSTCrossEncoder(
model_name=config.reranker_local_model,
max_concurrent=config.reranker_local_max_concurrent,
force_cpu=config.reranker_local_force_cpu,
model = os.environ.get(ENV_RERANKER_LOCAL_MODEL)
model_name = model or DEFAULT_RERANKER_LOCAL_MODEL
max_concurrent = int(
os.environ.get(ENV_RERANKER_LOCAL_MAX_CONCURRENT, str(DEFAULT_RERANKER_LOCAL_MAX_CONCURRENT))
)
return LocalSTCrossEncoder(model_name=model_name, max_concurrent=max_concurrent)
elif provider == "cohere":
api_key = os.environ.get(ENV_COHERE_API_KEY)
if not api_key:
@@ -18,7 +18,6 @@ import httpx
from ..config import (
DEFAULT_EMBEDDINGS_COHERE_MODEL,
DEFAULT_EMBEDDINGS_LITELLM_MODEL,
DEFAULT_EMBEDDINGS_LOCAL_FORCE_CPU,
DEFAULT_EMBEDDINGS_LOCAL_MODEL,
DEFAULT_EMBEDDINGS_OPENAI_MODEL,
DEFAULT_EMBEDDINGS_PROVIDER,
@@ -27,7 +26,6 @@ from ..config import (
ENV_EMBEDDINGS_COHERE_BASE_URL,
ENV_EMBEDDINGS_COHERE_MODEL,
ENV_EMBEDDINGS_LITELLM_MODEL,
ENV_EMBEDDINGS_LOCAL_FORCE_CPU,
ENV_EMBEDDINGS_LOCAL_MODEL,
ENV_EMBEDDINGS_OPENAI_API_KEY,
ENV_EMBEDDINGS_OPENAI_BASE_URL,
@@ -94,18 +92,15 @@ class LocalSTEmbeddings(Embeddings):
The embedding dimension is auto-detected from the model.
"""
def __init__(self, model_name: str | None = None, force_cpu: bool = False):
def __init__(self, model_name: str | None = None):
"""
Initialize local SentenceTransformers embeddings.
Args:
model_name: Name of the SentenceTransformer model to use.
Default: BAAI/bge-small-en-v1.5
force_cpu: Force CPU mode (avoids MPS/XPC issues on macOS in daemon mode).
Default: False
"""
self.model_name = model_name or DEFAULT_EMBEDDINGS_LOCAL_MODEL
self.force_cpu = force_cpu
self._model = None
self._dimension: int | None = None
@@ -139,23 +134,13 @@ class LocalSTEmbeddings(Embeddings):
# which can cause issues when accelerate is installed but no GPU is available.
import torch
# Force CPU mode if configured (used in daemon mode to avoid MPS/XPC issues on macOS)
if self.force_cpu:
device = "cpu"
logger.info("Embeddings: forcing CPU mode")
# Check for GPU (CUDA) or Apple Silicon (MPS)
has_gpu = torch.cuda.is_available() or (hasattr(torch.backends, "mps") and torch.backends.mps.is_available())
if has_gpu:
device = None # Let sentence-transformers auto-detect GPU/MPS
else:
# Check for GPU (CUDA) or Apple Silicon (MPS)
# Wrap in try-except to gracefully handle any device detection issues
# (e.g., in CI environments or when PyTorch is built without GPU support)
device = "cpu" # Default to CPU
try:
has_gpu = torch.cuda.is_available() or (
hasattr(torch.backends, "mps") and torch.backends.mps.is_available()
)
if has_gpu:
device = None # Let sentence-transformers auto-detect GPU/MPS
except Exception as e:
logger.warning(f"Failed to detect GPU/MPS, falling back to CPU: {e}")
device = "cpu"
self._model = SentenceTransformer(
self.model_name,
@@ -178,7 +163,6 @@ class LocalSTEmbeddings(Embeddings):
"""
if self._model is None:
raise RuntimeError("Embeddings not initialized. Call initialize() first.")
embeddings = self._model.encode(texts, convert_to_numpy=True, show_progress_bar=False)
return [emb.tolist() for emb in embeddings]
@@ -702,28 +686,24 @@ class LiteLLMEmbeddings(Embeddings):
def create_embeddings_from_env() -> Embeddings:
"""
Create an Embeddings instance based on configuration.
Create an Embeddings instance based on environment variables.
Reads configuration via get_config() to ensure consistency across the codebase.
See hindsight_api.config for environment variable names and defaults.
Returns:
Configured Embeddings instance
"""
from ..config import get_config
config = get_config()
provider = config.embeddings_provider.lower()
provider = os.environ.get(ENV_EMBEDDINGS_PROVIDER, DEFAULT_EMBEDDINGS_PROVIDER).lower()
if provider == "tei":
url = config.embeddings_tei_url
url = os.environ.get(ENV_EMBEDDINGS_TEI_URL)
if not url:
raise ValueError(f"{ENV_EMBEDDINGS_TEI_URL} is required when {ENV_EMBEDDINGS_PROVIDER} is 'tei'")
return RemoteTEIEmbeddings(base_url=url)
elif provider == "local":
return LocalSTEmbeddings(
model_name=config.embeddings_local_model,
force_cpu=config.embeddings_local_force_cpu,
)
model = os.environ.get(ENV_EMBEDDINGS_LOCAL_MODEL)
model_name = model or DEFAULT_EMBEDDINGS_LOCAL_MODEL
return LocalSTEmbeddings(model_name=model_name)
elif provider == "openai":
# Use dedicated embeddings API key, or fall back to LLM API key
api_key = os.environ.get(ENV_EMBEDDINGS_OPENAI_API_KEY) or os.environ.get(ENV_LLM_API_KEY)
@@ -23,17 +23,12 @@ from ..metrics import get_metrics_collector
from .db_budget import budgeted_operation
# Context variable for current schema (async-safe, per-task isolation)
# Note: default is None, actual default comes from config via get_current_schema()
_current_schema: contextvars.ContextVar[str | None] = contextvars.ContextVar("current_schema", default=None)
_current_schema: contextvars.ContextVar[str] = contextvars.ContextVar("current_schema", default="public")
def get_current_schema() -> str:
"""Get the current schema from context (falls back to config default)."""
schema = _current_schema.get()
if schema is None:
# Fall back to configured default schema
return get_config().database_schema
return schema
"""Get the current schema from context (default: 'public')."""
return _current_schema.get()
def fq_table(table_name: str) -> str:
@@ -886,12 +881,11 @@ class MemoryEngine(MemoryEngineInterface):
if not self.db_url:
raise ValueError("Database URL is required for migrations")
logger.info("Running database migrations...")
# Use configured database schema for migrations (defaults to "public")
run_migrations(self.db_url, schema=get_config().database_schema)
run_migrations(self.db_url)
# Ensure embedding column dimension matches the model's dimension
# This is done after migrations and after embeddings.initialize()
ensure_embedding_dimension(self.db_url, self.embeddings.dimension, schema=get_config().database_schema)
ensure_embedding_dimension(self.db_url, self.embeddings.dimension)
logger.info(f"Connecting to PostgreSQL at {self.db_url}")
@@ -2770,7 +2764,7 @@ class MemoryEngine(MemoryEngineInterface):
param_count += 1
units = await conn.fetch(
f"""
SELECT id, text, event_date, context, occurred_start, occurred_end, mentioned_at, document_id, chunk_id, fact_type, tags, created_at, proof_count, source_memory_ids
SELECT id, text, event_date, context, occurred_start, occurred_end, mentioned_at, document_id, chunk_id, fact_type, tags, created_at, proof_count
FROM {fq_table("memory_units")}
{where_clause}
ORDER BY mentioned_at DESC NULLS LAST, event_date DESC
@@ -2783,18 +2777,7 @@ class MemoryEngine(MemoryEngineInterface):
# Get links, filtering to only include links between units of the selected agent
# Use DISTINCT ON with LEAST/GREATEST to deduplicate bidirectional links
unit_ids = [row["id"] for row in units]
unit_id_set = set(unit_ids)
# Collect source memory IDs from observations
source_memory_ids = []
for unit in units:
if unit["source_memory_ids"]:
source_memory_ids.extend(unit["source_memory_ids"])
source_memory_ids = list(set(source_memory_ids)) # Deduplicate
# Fetch links involving both visible units AND source memories
all_relevant_ids = unit_ids + source_memory_ids
if all_relevant_ids:
if unit_ids:
links = await conn.fetch(
f"""
SELECT DISTINCT ON (LEAST(ml.from_unit_id, ml.to_unit_id), GREATEST(ml.from_unit_id, ml.to_unit_id), ml.link_type, COALESCE(ml.entity_id, '00000000-0000-0000-0000-000000000000'::uuid))
@@ -2805,69 +2788,14 @@ class MemoryEngine(MemoryEngineInterface):
e.canonical_name as entity_name
FROM {fq_table("memory_links")} ml
LEFT JOIN {fq_table("entities")} e ON ml.entity_id = e.id
WHERE ml.from_unit_id = ANY($1::uuid[]) OR ml.to_unit_id = ANY($1::uuid[])
WHERE ml.from_unit_id = ANY($1::uuid[]) AND ml.to_unit_id = ANY($1::uuid[])
ORDER BY LEAST(ml.from_unit_id, ml.to_unit_id), GREATEST(ml.from_unit_id, ml.to_unit_id), ml.link_type, COALESCE(ml.entity_id, '00000000-0000-0000-0000-000000000000'::uuid), ml.weight DESC
""",
all_relevant_ids,
unit_ids,
)
else:
links = []
# Copy links from source memories to observations
# Observations inherit links from their source memories via source_memory_ids
# Build a map from source_id to observation_ids
source_to_observations = {}
for unit in units:
if unit["source_memory_ids"]:
for source_id in unit["source_memory_ids"]:
if source_id not in source_to_observations:
source_to_observations[source_id] = []
source_to_observations[source_id].append(unit["id"])
copied_links = []
for link in links:
from_id = link["from_unit_id"]
to_id = link["to_unit_id"]
# Get observations that should inherit this link
from_observations = source_to_observations.get(from_id, [])
to_observations = source_to_observations.get(to_id, [])
# If from_id is a source memory, copy links to its observations
if from_observations:
for obs_id in from_observations:
# Only include if the target is visible
if to_id in unit_id_set or to_observations:
target = to_observations[0] if to_observations and to_id not in unit_id_set else to_id
if target in unit_id_set:
copied_links.append(
{
"from_unit_id": obs_id,
"to_unit_id": target,
"link_type": link["link_type"],
"weight": link["weight"],
"entity_name": link["entity_name"],
}
)
# If to_id is a source memory, copy links to its observations
if to_observations and from_id in unit_id_set:
for obs_id in to_observations:
copied_links.append(
{
"from_unit_id": from_id,
"to_unit_id": obs_id,
"link_type": link["link_type"],
"weight": link["weight"],
"entity_name": link["entity_name"],
}
)
# Keep only direct links between visible nodes
direct_links = [
link for link in links if link["from_unit_id"] in unit_id_set and link["to_unit_id"] in unit_id_set
]
# Get entity information
unit_entities = await conn.fetch(f"""
SELECT ue.unit_id, e.canonical_name
@@ -2885,18 +2813,6 @@ class MemoryEngine(MemoryEngineInterface):
entity_map[unit_id] = []
entity_map[unit_id].append(entity_name)
# For observations, inherit entities from source memories
for unit in units:
if unit["source_memory_ids"] and unit["id"] not in entity_map:
# Collect entities from all source memories
source_entities = []
for source_id in unit["source_memory_ids"]:
if source_id in entity_map:
source_entities.extend(entity_map[source_id])
if source_entities:
# Deduplicate while preserving order
entity_map[unit["id"]] = list(dict.fromkeys(source_entities))
# Build nodes
nodes = []
for row in units:
@@ -2930,15 +2846,14 @@ class MemoryEngine(MemoryEngineInterface):
}
)
# Build edges (combine direct links and copied links from sources)
# Build edges
edges = []
all_links = direct_links + copied_links
for row in all_links:
for row in links:
from_id = str(row["from_unit_id"])
to_id = str(row["to_unit_id"])
link_type = row["link_type"]
weight = row["weight"]
entity_name = row.get("entity_name")
entity_name = row["entity_name"]
# Color by link type
if link_type == "temporal":
@@ -58,7 +58,6 @@ def _normalize_tool_name(name: str) -> str:
- 'functions.done' (OpenAI-style prefix)
- 'call=functions.done' (some models)
- 'call=done' (some models)
- 'done<|channel|>commentary' (malformed special tokens appended)
Returns the normalized tool name (e.g., 'done', 'recall', etc.)
"""
@@ -70,11 +69,6 @@ def _normalize_tool_name(name: str) -> str:
if name.startswith("functions."):
name = name[len("functions.") :]
# Handle malformed special tokens appended to tool name
# e.g., 'done<|channel|>commentary' -> 'done'
if "<|" in name:
name = name.split("<|")[0]
return name
@@ -69,7 +69,7 @@ async def tool_search_mental_models(
next_param += 1
if exclude_ids:
filters += f" AND id != ALL(${next_param}::text[])"
filters += f" AND id != ALL(${next_param}::uuid[])"
params.append(exclude_ids)
next_param += 1
@@ -432,15 +432,34 @@ def _chunk_conversation(turns: list[dict], max_chars: int) -> list[str]:
# FACT EXTRACTION PROMPTS
# =============================================================================
# Base prompt template (shared by concise and custom modes)
# Uses {extraction_guidelines} placeholder for mode-specific instructions
_BASE_FACT_EXTRACTION_PROMPT = """Extract SIGNIFICANT facts from text. Be SELECTIVE - only extract facts worth remembering long-term.
# Concise extraction prompt (default) - selective, high-quality facts
CONCISE_FACT_EXTRACTION_PROMPT = """Extract SIGNIFICANT facts from text. Be SELECTIVE - only extract facts worth remembering long-term.
LANGUAGE REQUIREMENT: Detect the language of the input text. All extracted facts, entity names, descriptions, and other output MUST be in the SAME language as the input. Do not translate to another language.
{fact_types_instruction}
{extraction_guidelines}
══════════════════════════════════════════════════════════════════════════
SELECTIVITY - CRITICAL (Reduces 90% of unnecessary output)
══════════════════════════════════════════════════════════════════════════
ONLY extract facts that are:
✅ Personal info: names, relationships, roles, background
✅ Preferences: likes, dislikes, habits, interests (e.g., "Alice likes coffee")
✅ Significant events: milestones, decisions, achievements, changes
✅ Plans/goals: future intentions, deadlines, commitments
✅ Expertise: skills, knowledge, certifications, experience
✅ Important context: projects, problems, constraints
✅ Sensory/emotional details: feelings, sensations, perceptions that provide context
✅ Observations: descriptions of people, places, things with specific details
DO NOT extract:
❌ Generic greetings: "how are you", "hello", pleasantries without substance
❌ Pure filler: "thanks", "sounds good", "ok", "got it", "sure"
❌ Process chatter: "let me check", "one moment", "I'll look into it"
❌ Repeated info: if already stated, don't extract again
CONSOLIDATE related statements into ONE fact when possible.
══════════════════════════════════════════════════════════════════════════
FACT FORMAT - BE CONCISE
@@ -488,33 +507,7 @@ ENTITIES
══════════════════════════════════════════════════════════════════════════
Include: people names, organizations, places, key objects, abstract concepts (career, friendship, etc.)
Always include "user" when fact is about the user.{examples}"""
# Concise mode guidelines
_CONCISE_GUIDELINES = """══════════════════════════════════════════════════════════════════════════
SELECTIVITY - CRITICAL (Reduces 90% of unnecessary output)
══════════════════════════════════════════════════════════════════════════
ONLY extract facts that are:
✅ Personal info: names, relationships, roles, background
✅ Preferences: likes, dislikes, habits, interests (e.g., "Alice likes coffee")
✅ Significant events: milestones, decisions, achievements, changes
✅ Plans/goals: future intentions, deadlines, commitments
✅ Expertise: skills, knowledge, certifications, experience
✅ Important context: projects, problems, constraints
✅ Sensory/emotional details: feelings, sensations, perceptions that provide context
✅ Observations: descriptions of people, places, things with specific details
DO NOT extract:
❌ Generic greetings: "how are you", "hello", pleasantries without substance
❌ Pure filler: "thanks", "sounds good", "ok", "got it", "sure"
❌ Process chatter: "let me check", "one moment", "I'll look into it"
❌ Repeated info: if already stated, don't extract again
CONSOLIDATE related statements into ONE fact when possible."""
# Concise mode examples
_CONCISE_EXAMPLES = """
Always include "user" when fact is about the user.
══════════════════════════════════════════════════════════════════════════
EXAMPLES
@@ -540,20 +533,6 @@ QUALITY OVER QUANTITY
Ask: "Would this be useful to recall in 6 months?" If no, skip it."""
# Assembled concise prompt (backward compatible - exact same output as before)
CONCISE_FACT_EXTRACTION_PROMPT = _BASE_FACT_EXTRACTION_PROMPT.format(
fact_types_instruction="{fact_types_instruction}",
extraction_guidelines=_CONCISE_GUIDELINES,
examples=_CONCISE_EXAMPLES,
)
# Custom prompt uses same base but without examples
CUSTOM_FACT_EXTRACTION_PROMPT = _BASE_FACT_EXTRACTION_PROMPT.format(
fact_types_instruction="{fact_types_instruction}",
extraction_guidelines="{custom_instructions}",
examples="", # No examples for custom mode
)
# Verbose extraction prompt - detailed, comprehensive facts (legacy mode)
VERBOSE_FACT_EXTRACTION_PROMPT = """Extract facts from text into structured format with FIVE required dimensions - BE EXTREMELY DETAILED.
@@ -701,12 +680,6 @@ async def _extract_facts_from_chunk(
Note: event_date parameter is kept for backward compatibility but not used in prompt.
The LLM extracts temporal information from the context string instead.
"""
import logging
from openai import BadRequestError
logger = logging.getLogger(__name__)
memory_bank_context = f"\n- Your name: {agent_name}" if agent_name and extract_opinions else ""
# Determine which fact types to extract based on the flag
@@ -725,27 +698,13 @@ async def _extract_facts_from_chunk(
extract_causal_links = config.retain_extract_causal_links
# Select base prompt based on extraction mode
if extraction_mode == "custom":
# Custom mode: inject user-provided guidelines
if not config.retain_custom_instructions:
logger.warning(
"extraction_mode='custom' but HINDSIGHT_API_RETAIN_CUSTOM_INSTRUCTIONS not set. "
"Falling back to 'concise' mode."
)
base_prompt = CONCISE_FACT_EXTRACTION_PROMPT
prompt = base_prompt.format(fact_types_instruction=fact_types_instruction)
else:
base_prompt = CUSTOM_FACT_EXTRACTION_PROMPT
prompt = base_prompt.format(
fact_types_instruction=fact_types_instruction,
custom_instructions=config.retain_custom_instructions,
)
elif extraction_mode == "verbose":
if extraction_mode == "verbose":
base_prompt = VERBOSE_FACT_EXTRACTION_PROMPT
prompt = base_prompt.format(fact_types_instruction=fact_types_instruction)
else:
base_prompt = CONCISE_FACT_EXTRACTION_PROMPT
prompt = base_prompt.format(fact_types_instruction=fact_types_instruction)
# Format the prompt with fact types instruction
prompt = base_prompt.format(fact_types_instruction=fact_types_instruction)
# Build the full prompt with or without causal relationships section
# Select appropriate response schema based on extraction mode and causal links
@@ -758,6 +717,12 @@ async def _extract_facts_from_chunk(
else:
response_schema = FactExtractionResponseNoCausal
import logging
from openai import BadRequestError
logger = logging.getLogger(__name__)
# Retry logic for JSON validation errors
max_retries = 2
last_error = None
@@ -155,6 +155,7 @@ class LinkExpansionRetriever(GraphRetriever):
all_seeds.extend(temporal_seeds)
if not all_seeds:
logger.info("[LinkExpansion] No seeds found, returning empty results")
return [], timings
seed_ids = list({s.id for s in all_seeds})
@@ -1,6 +1,5 @@
"""Built-in tenant extension implementations."""
from hindsight_api.config import get_config
from hindsight_api.extensions.tenant import AuthenticationError, Tenant, TenantContext, TenantExtension
from hindsight_api.models import RequestContext
@@ -11,13 +10,11 @@ class ApiKeyTenantExtension(TenantExtension):
This is a simple implementation that:
1. Validates the API key matches HINDSIGHT_API_TENANT_API_KEY
2. Returns the configured schema (HINDSIGHT_API_DATABASE_SCHEMA, default 'public')
for all authenticated requests
2. Returns 'public' as the schema for all authenticated requests
Configuration:
HINDSIGHT_API_TENANT_EXTENSION=hindsight_api.extensions.builtin.tenant:ApiKeyTenantExtension
HINDSIGHT_API_TENANT_API_KEY=your-secret-key
HINDSIGHT_API_DATABASE_SCHEMA=your-schema (optional, defaults to 'public')
For multi-tenant setups with separate schemas per tenant, implement a custom
TenantExtension that looks up the schema based on the API key or token claims.
@@ -30,11 +27,11 @@ class ApiKeyTenantExtension(TenantExtension):
raise ValueError("HINDSIGHT_API_TENANT_API_KEY is required when using ApiKeyTenantExtension")
async def authenticate(self, context: RequestContext) -> TenantContext:
"""Validate API key and return configured schema context."""
"""Validate API key and return public schema context."""
if context.api_key != self.expected_api_key:
raise AuthenticationError("Invalid API key")
return TenantContext(schema_name=get_config().database_schema)
return TenantContext(schema_name="public")
async def list_tenants(self) -> list[Tenant]:
"""Return configured schema for single-tenant setup."""
return [Tenant(schema=get_config().database_schema)]
"""Return public schema for single-tenant setup."""
return [Tenant(schema="public")]
+2 -9
View File
@@ -170,7 +170,6 @@ def main():
if args.log_level != config.log_level:
config = HindsightConfig(
database_url=config.database_url,
database_schema=config.database_schema,
llm_provider=config.llm_provider,
llm_api_key=config.llm_api_key,
llm_model=config.llm_model,
@@ -191,14 +190,11 @@ def main():
consolidation_llm_base_url=config.consolidation_llm_base_url,
embeddings_provider=config.embeddings_provider,
embeddings_local_model=config.embeddings_local_model,
embeddings_local_force_cpu=config.embeddings_local_force_cpu,
embeddings_tei_url=config.embeddings_tei_url,
embeddings_openai_base_url=config.embeddings_openai_base_url,
embeddings_cohere_base_url=config.embeddings_cohere_base_url,
reranker_provider=config.reranker_provider,
reranker_local_model=config.reranker_local_model,
reranker_local_force_cpu=config.reranker_local_force_cpu,
reranker_local_max_concurrent=config.reranker_local_max_concurrent,
reranker_tei_url=config.reranker_tei_url,
reranker_tei_batch_size=config.reranker_tei_batch_size,
reranker_tei_max_concurrent=config.reranker_tei_max_concurrent,
@@ -217,11 +213,9 @@ def main():
retain_chunk_size=config.retain_chunk_size,
retain_extract_causal_links=config.retain_extract_causal_links,
retain_extraction_mode=config.retain_extraction_mode,
retain_custom_instructions=config.retain_custom_instructions,
retain_observations_async=config.retain_observations_async,
enable_observations=config.enable_observations,
consolidation_batch_size=config.consolidation_batch_size,
consolidation_max_tokens=config.consolidation_max_tokens,
skip_llm_verification=config.skip_llm_verification,
lazy_reranker=config.lazy_reranker,
run_migrations_on_startup=config.run_migrations_on_startup,
@@ -346,7 +340,6 @@ def main():
# Start idle checker in daemon mode
if idle_middleware is not None:
# Start the idle checker in a background thread with its own event loop
import logging
import threading
def run_idle_checker():
@@ -357,8 +350,8 @@ def main():
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_until_complete(idle_middleware._check_idle())
except Exception as e:
logging.error(f"Idle checker error: {e}", exc_info=True)
except Exception:
pass
threading.Thread(target=run_idle_checker, daemon=True).start()
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "hindsight-api"
version = "0.4.1"
version = "0.3.0"
description = "Hindsight: Agent Memory That Works Like Human Memory"
readme = "README.md"
requires-python = ">=3.11"
-90
View File
@@ -1897,93 +1897,3 @@ class TestMentalModelRefreshAfterConsolidation:
# Cleanup
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.asyncio
async def test_graph_endpoint_observations_inherit_links_and_entities(
self, memory: MemoryEngine, request_context
):
"""Test that graph endpoint shows links and entities for observations filtered by type.
When filtering graph by type=observation:
- Observations should inherit links from their source memories
- Observations should show entities inherited from source memories
- Even when source memories are not visible, their links should be copied to observations
"""
bank_id = f"test-graph-obs-{uuid.uuid4().hex[:8]}"
# Create the bank
await memory.get_bank_profile(bank_id=bank_id, request_context=request_context)
# Retain content that will create world facts with shared entities
# This should create facts that are linked by shared entities
await memory.retain_async(
bank_id=bank_id,
content="Alice works at Google as a software engineer.",
request_context=request_context,
)
await memory.retain_async(
bank_id=bank_id,
content="Bob also works at Google in the sales department.",
request_context=request_context,
)
# Wait for consolidation to create observations
import asyncio
await asyncio.sleep(2)
# Get graph data filtered by observation type only
graph_data = await memory.get_graph_data(
bank_id=bank_id,
fact_type="observation",
limit=1000,
request_context=request_context,
)
# Should have observations
assert graph_data["total_units"] > 0, "Should have observations"
assert len(graph_data["nodes"]) > 0, "Should have observation nodes"
# Verify all nodes are observations
for row in graph_data["table_rows"]:
assert row["fact_type"] == "observation", f"All nodes should be observations, got {row['fact_type']}"
# Should have edges (inherited from source memories)
# Even though we're only showing observations, they should inherit links from their sources
assert len(graph_data["edges"]) > 0, (
"Observations should have edges inherited from source memories. "
f"Found {len(graph_data['edges'])} edges"
)
# Should have entities (inherited from source memories)
observations_with_entities = [
row for row in graph_data["table_rows"] if row["entities"] and row["entities"] != "None"
]
assert len(observations_with_entities) > 0, (
"Observations should inherit entities from source memories. "
f"Found {len(observations_with_entities)} observations with entities"
)
# Verify entities contain expected values
all_entities = " ".join([row["entities"] for row in graph_data["table_rows"]])
assert "Alice" in all_entities or "Bob" in all_entities or "Google" in all_entities, (
f"Expected to find Alice, Bob, or Google in entities, got: {all_entities}"
)
# Verify edge types are valid
valid_link_types = {"semantic", "temporal", "entity"}
for edge in graph_data["edges"]:
link_type = edge["data"]["linkType"]
assert link_type in valid_link_types, f"Invalid link type: {link_type}"
# Verify all edges connect visible observation nodes
visible_node_ids = {row["id"] for row in graph_data["table_rows"]}
for edge in graph_data["edges"]:
source_id = edge["data"]["source"]
target_id = edge["data"]["target"]
assert source_id in visible_node_ids, f"Edge source {source_id[:8]} not in visible nodes"
assert target_id in visible_node_ids, f"Edge target {target_id[:8]} not in visible nodes"
# Cleanup
await memory.delete_bank(bank_id, request_context=request_context)
@@ -1063,38 +1063,3 @@ async def test_retain_async_no_usage(api_client):
# Usage should be None for async operations
assert result.get("usage") is None, "Async retain should not include usage"
@pytest.mark.asyncio
async def test_version_endpoint_returns_correct_version(api_client):
"""Test that the /version endpoint returns the correct API version.
The version should match the __version__ defined in hindsight_api.__init__.py
and should not be a hardcoded string.
"""
from hindsight_api import __version__
# Call the /version endpoint
response = await api_client.get("/version")
assert response.status_code == 200
result = response.json()
# Verify response structure
assert "api_version" in result, "Response should include 'api_version' field"
assert "features" in result, "Response should include 'features' field"
# Verify the version matches the package version
assert result["api_version"] == __version__, (
f"API version should be {__version__}, got {result['api_version']}"
)
# Verify features field structure
features = result["features"]
assert "observations" in features
assert "mcp" in features
assert "worker" in features
assert isinstance(features["observations"], bool)
assert isinstance(features["mcp"], bool)
assert isinstance(features["worker"], bool)
print(f"Version endpoint returned: api_version={result['api_version']}, features={features}")
-11
View File
@@ -163,12 +163,6 @@ class TestToolNameNormalization:
assert _normalize_tool_name("call=functions.recall") == "recall"
assert _normalize_tool_name("call=functions.search_observations") == "search_observations"
def test_normalize_special_token_suffix(self):
"""Tool names with malformed special tokens should be normalized."""
assert _normalize_tool_name("done<|channel|>commentary") == "done"
assert _normalize_tool_name("recall<|endoftext|>") == "recall"
assert _normalize_tool_name("search_observations<|im_end|>extra") == "search_observations"
def test_is_done_tool(self):
"""Test _is_done_tool helper."""
# Standard
@@ -180,14 +174,9 @@ class TestToolNameNormalization:
assert _is_done_tool("call=done") is True
assert _is_done_tool("call=functions.done") is True
# With malformed special tokens
assert _is_done_tool("done<|channel|>commentary") is True
assert _is_done_tool("done<|endoftext|>") is True
# Not done
assert _is_done_tool("functions.recall") is False
assert _is_done_tool("call=functions.recall") is False
assert _is_done_tool("recall<|channel|>done") is False
class TestReflectAgentMocked:
-114
View File
@@ -2082,117 +2082,3 @@ def test_recall_result_model_empty_construction():
assert result.chunks == {}, "Should have empty chunks"
logger.info("✓ RecallResult empty construction works correctly")
@pytest.mark.asyncio
async def test_custom_extraction_mode():
"""
Test that custom extraction mode uses custom guidelines from env variable.
This test verifies that when HINDSIGHT_API_RETAIN_EXTRACTION_MODE=custom and
HINDSIGHT_API_RETAIN_CUSTOM_INSTRUCTIONS is set, the fact extraction uses the
custom guidelines while keeping structural parts intact.
"""
import os
from hindsight_api import LLMConfig
from hindsight_api.engine.retain.fact_extraction import extract_facts_from_text
from hindsight_api.config import clear_config_cache
# Save original env vars
original_mode = os.getenv("HINDSIGHT_API_RETAIN_EXTRACTION_MODE")
original_instructions = os.getenv("HINDSIGHT_API_RETAIN_CUSTOM_INSTRUCTIONS")
try:
# Set custom extraction mode with challenging language-specific guidelines
os.environ["HINDSIGHT_API_RETAIN_EXTRACTION_MODE"] = "custom"
os.environ["HINDSIGHT_API_RETAIN_CUSTOM_INSTRUCTIONS"] = """ONLY extract facts that are in ITALIAN language.
DO NOT extract:
❌ Facts in English
❌ Facts in any other language besides Italian
If the text contains both Italian and English content, extract ONLY the Italian facts."""
# Clear config cache to pick up new env vars
clear_config_cache()
# Test content with BOTH Italian (should extract) and English (should NOT extract) facts
# This is a much harder test than filtering greetings
text = """
The team discussed the new architecture. We will use microservices.
Il database PostgreSQL ha ridotto la latenza delle query del 60%.
Alice ha suggerito di usare il connection pooling per migliorare le prestazioni.
Bob mentioned that the API endpoint is ready for testing.
The deployment pipeline has been updated to use Kubernetes.
Marco ha completato la revisione del codice e ha approvato le modifiche.
Il sistema di autenticazione è stato migrato a OAuth 2.0.
"""
llm_config = LLMConfig.for_memory()
facts, _, _ = await extract_facts_from_text(
text=text,
event_date=datetime(2024, 1, 15, tzinfo=timezone.utc),
context="team meeting notes",
llm_config=llm_config,
agent_name="TestUser"
)
logger.info(f"\nExtracted {len(facts)} facts with custom mode (Italian only):")
for i, fact in enumerate(facts):
logger.info(f" {i+1}. {fact.fact}")
assert len(facts) > 0, "Should extract at least one Italian fact"
# All facts text
all_facts_text = " ".join([f.fact for f in facts])
# Should HAVE Italian content
italian_keywords = ["postgresql", "latenza", "query", "alice", "connection pooling", "prestazioni",
"marco", "revisione", "codice", "autenticazione", "oauth"]
has_italian = any(keyword in all_facts_text.lower() for keyword in italian_keywords)
assert has_italian, f"Should extract Italian facts. Got: {all_facts_text}"
# Should NOT have English-only content
# These are facts that appear ONLY in English sections
english_only_keywords = ["microservices", "bob", "api endpoint", "testing", "deployment pipeline", "kubernetes"]
# Check if facts contain English-only content (this would be wrong)
facts_lower = all_facts_text.lower()
found_english_only = [kw for kw in english_only_keywords if kw in facts_lower]
if found_english_only:
logger.warning(f"⚠ Found English-only keywords in facts: {found_english_only}")
logger.warning(f" Facts: {all_facts_text}")
logger.warning(f" This may indicate the LLM is not strictly following language-specific custom guidelines")
# Log but don't fail - LLM behavior can vary
else:
logger.info("✓ Successfully extracted only Italian facts, ignored English facts")
# At least verify we have some Italian indicators
italian_indicators = ["latenza", "prestazioni", "revisione", "codice", "autenticazione"]
italian_count = sum(1 for ind in italian_indicators if ind in facts_lower)
assert italian_count >= 1, \
f"Should extract facts with Italian words. Found {italian_count} Italian indicators in: {all_facts_text}"
logger.info("✓ Custom extraction mode works with language-specific guidelines")
logger.info(f"✓ Extracted {len(facts)} Italian facts, found {italian_count} Italian indicators")
finally:
# Restore original env vars
if original_mode is not None:
os.environ["HINDSIGHT_API_RETAIN_EXTRACTION_MODE"] = original_mode
else:
os.environ.pop("HINDSIGHT_API_RETAIN_EXTRACTION_MODE", None)
if original_instructions is not None:
os.environ["HINDSIGHT_API_RETAIN_CUSTOM_INSTRUCTIONS"] = original_instructions
else:
os.environ.pop("HINDSIGHT_API_RETAIN_CUSTOM_INSTRUCTIONS", None)
# Clear cache again to restore original config
clear_config_cache()
@@ -527,7 +527,6 @@ class TestRemoteTEICrossEncoderConfig:
"""Test creating encoder from environment variables."""
import os
from hindsight_api.config import clear_config_cache
from hindsight_api.engine.cross_encoder import create_cross_encoder_from_env
with patch.dict(
@@ -539,7 +538,6 @@ class TestRemoteTEICrossEncoderConfig:
"HINDSIGHT_API_RERANKER_TEI_MAX_CONCURRENT": "16",
},
):
clear_config_cache() # Clear cache to pick up patched env vars
encoder = create_cross_encoder_from_env()
assert isinstance(encoder, RemoteTEICrossEncoder)
@@ -547,8 +545,6 @@ class TestRemoteTEICrossEncoderConfig:
assert encoder.batch_size == 256
assert encoder.max_concurrent == 16
clear_config_cache() # Clear cache after test
# ============================================================================
# TEI Reranker Performance Benchmark Tests
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "hindsight-cli"
version = "0.4.1"
version = "0.3.0"
edition = "2021"
authors = ["Hindsight Team"]
description = "A beautiful CLI for Hindsight - semantic memory system"
-125
View File
@@ -1,125 +0,0 @@
use std::process::Command;
#[test]
fn test_cli_help() {
let output = Command::new("cargo")
.args(["run", "--", "--help"])
.output()
.expect("Failed to execute command");
assert!(output.status.success());
let stdout = String::from_utf8_lossy(&output.stdout);
assert!(stdout.contains("Hindsight CLI"));
}
#[test]
fn test_cli_version() {
let output = Command::new("cargo")
.args(["run", "--", "--version"])
.output()
.expect("Failed to execute command");
assert!(output.status.success());
let stdout = String::from_utf8_lossy(&output.stdout);
assert!(stdout.contains("hindsight"));
}
#[test]
fn test_ui_command_without_config() {
// Test that the ui command handles missing config gracefully
// Create a temp home directory with no config
let temp_dir = std::env::temp_dir().join(format!("hindsight-test-ui-{}", std::process::id()));
std::fs::create_dir_all(&temp_dir).expect("Failed to create temp dir");
let output = Command::new("cargo")
.args(["run", "--", "ui"])
.env_remove("HINDSIGHT_API_URL")
.env_remove("HINDSIGHT_API_KEY")
.env("HOME", &temp_dir)
.output()
.expect("Failed to execute command");
let stdout = String::from_utf8_lossy(&output.stdout);
let stderr = String::from_utf8_lossy(&output.stderr);
// Either it fails with a config error or it succeeds if there's a default config
// Just verify it doesn't crash unexpectedly
assert!(
!output.status.success()
|| stdout.contains("Launching Hindsight Control Plane UI")
|| stderr.contains("Configuration error")
|| stderr.contains("HINDSIGHT_API_URL"),
"Unexpected output - stdout: {}, stderr: {}",
stdout,
stderr
);
// Cleanup
std::fs::remove_dir_all(&temp_dir).ok();
}
#[test]
fn test_ui_command_with_config() {
// This test is skipped by default since it requires a running control plane
// and would block for a long time. The other tests cover the basic functionality.
// To run this test manually:
// 1. Build the control plane: cd hindsight-control-plane && npm run build
// 2. Run: cargo test test_ui_command_with_config -- --ignored
// Just verify that the ui command accepts the configuration
let temp_dir = std::env::temp_dir().join(format!("hindsight-test-ui-valid-{}", std::process::id()));
std::fs::create_dir_all(&temp_dir).expect("Failed to create temp dir");
// Write a minimal config
let config_dir = temp_dir.join(".config").join("hindsight");
std::fs::create_dir_all(&config_dir).expect("Failed to create config dir");
let config_file = config_dir.join("config");
std::fs::write(&config_file, "api_url=http://localhost:8888\napi_key=test-key\n")
.expect("Failed to write config");
let output = Command::new("cargo")
.args(["run", "--", "ui", "--help"])
.env("HOME", &temp_dir)
.output()
.expect("Failed to execute command");
// The --help should work regardless
let stdout = String::from_utf8_lossy(&output.stdout);
assert!(stdout.contains("Hindsight CLI") || output.status.success());
// Cleanup
std::fs::remove_dir_all(&temp_dir).ok();
}
#[test]
fn test_configure_command() {
// Test that configure command creates/updates config
let temp_dir = std::env::temp_dir().join(format!("hindsight-test-{}", std::process::id()));
std::fs::create_dir_all(&temp_dir).expect("Failed to create temp dir");
let output = Command::new("cargo")
.args([
"run",
"--",
"configure",
"--api-url",
"http://localhost:9999",
"--api-key",
"test-key-123"
])
.env("HOME", &temp_dir)
.output()
.expect("Failed to execute command");
assert!(
output.status.success(),
"Configure command failed: {}",
String::from_utf8_lossy(&output.stderr)
);
let stdout = String::from_utf8_lossy(&output.stdout);
assert!(stdout.contains("Configuration saved") || stdout.contains("success"));
// Cleanup
std::fs::remove_dir_all(&temp_dir).ok();
}
@@ -7,7 +7,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -489,7 +489,7 @@ class Configuration:
return "Python SDK Debug Report:\n"\
"OS: {env}\n"\
"Python Version: {pyversion}\n"\
"Version of the API: 0.4.1\n"\
"Version of the API: 0.1.0\n"\
"SDK Package Version: 0.0.7".\
format(env=sys.platform, pyversion=sys.version)
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -6,7 +6,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
@@ -5,7 +5,7 @@
HTTP API for Hindsight
The version of the OpenAPI document: 0.4.1
The version of the OpenAPI document: 0.1.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.

Some files were not shown because too many files have changed in this diff Show More