test: assert memory state via the engine read API, not raw SQL (#3591)

The suite asserted memory state by querying `memory_units` / `memory_links` /
`unit_entities` directly. That couples tests to the physical schema and makes
them unable to run against any store that keeps memory rows outside Postgres.
This ports what the read API can answer, extends it where it could not, and
marks the residue that is Postgres-shaped by nature.

**Ported to the engine API.** The "unconsolidated count" assertions spelled out
`consolidated_at IS NULL AND consolidation_failed_at IS NULL AND fact_type IN
('experience','world')` — character-for-character what
`list_memory_units(consolidation_state='pending')` already means, so seven sites
across five files became one call. Observation lineage, entity/tag checks and
chunk provenance moved to `list_memory_units` / `get_memory_unit` /
`list_document_chunks`; where the old query was an inner join, the port keeps
the same filtering and says why.

**Read model extended** so the rest could follow: `updated_at` and
`source_memory_ids` on list items, `entity_kind` on entity items, and a
list-valued `fact_type` that matches any of them. Every field is a column of the
row the query already fetched — the projection grows, the plan does not — so no
opt-in flag was needed and production paths pay nothing. Covered by a new test
module driving it all through the engine on retain-written units.

**260 of 6297 tests marked `memory_backend_incompatible`**, in two passes: those
that assert Postgres-internal state (raw `memory_links` counts, `embedding` /
`search_vector`), and those whose fixtures only exist in Postgres. The second set
was chosen on evidence — each both failed against a non-SQL store and touches
those tables in its body, a helper, or a fixture — never by grep alone. Postgres
runs are unchanged; the marker only takes effect behind
`-m 'not memory_backend_incompatible'`.

Also green-lights two checks that were red before this branch: the repo formatter
over five test files, and the coding-agents docs generator, which now rewrites our
own doc links to site-relative the way it already did for assets — the naive
regeneration would have degraded the docs-skill reference's file-relative links.
This commit is contained in:
Nicolò Boschi
2026-08-19 00:23:44 +02:00
committed by GitHub
parent edd0d0c5bb
commit 8b78b4ac04
72 changed files with 857 additions and 820 deletions
+22
View File
@@ -164,6 +164,28 @@ Flag any new logic that lacks test coverage.
See CLAUDE.md → Key Conventions → Testing for the full pattern.
### 6a. Check tests assert memory state via the engine API, not raw SQL
Tests must verify what a retain / recall / consolidation produced by calling the public
`MemoryEngine` read API — `list_memory_units` (units and their `metadata` / `tags`; counts via
`total`; `document_id` / `fact_type` / `entity_id` filters), `list_entities` (canonical names,
mention counts), `get_graph_data` (nodes/edges), `get_bank_stats`, `recall_async`**not** by
reaching into the memory tables (`memory_units`, `memory_links`, `unit_entities`) with raw SQL via
`pool.acquire()` / `conn.fetch*`. Asserting on those tables couples the test to a storage-layer
detail and checks a proxy instead of the observable property (see **General Principles** → tests
assert the property, and the handler rule in **7b**).
**Flag as should fix** any added or changed test whose assertion runs a `SELECT` / `COUNT` against
`memory_units` / `memory_links` / `unit_entities` where an engine read method returns the same
fact. Prime tell: `async with pool.acquire() as conn:` followed by `SELECT ... FROM memory_units`
inside a test body; a `fetchval("SELECT count(*) FROM memory_units ...")` that `list_memory_units`
`["total"]` would return; a `canonical_name` query that `list_entities` covers.
Direct SQL on those tables is legitimate **only** when it forces or inspects internal state the
public API cannot express — e.g. an `UPDATE documents SET updated_at` that forges a race, or a
raw `memory_links` row-count that the deduped `get_graph_data` edge list cannot reproduce. Those
must carry a comment saying why the direct access is necessary; flag any that do not.
### 7. Check API consistency
If any files in `hindsight-api-slim/hindsight_api/api/` were changed:
@@ -328,7 +328,7 @@ class MemoryEngineInterface(ABC):
self,
bank_id: str,
*,
fact_type: str | None = None,
fact_type: str | list[str] | None = None,
search_query: str | None = None,
entity_id: str | None = None,
created_before: datetime | None = None,
@@ -341,7 +341,8 @@ class MemoryEngineInterface(ABC):
Args:
bank_id: The memory bank ID.
fact_type: Filter by fact type.
fact_type: Filter by fact type. A list matches any of them; an empty
list is treated as no filter.
search_query: Full-text search query.
entity_id: Filter to memory units linked to this entity ID.
created_before: Keep units with ``created_at`` before this instant.
@@ -1043,7 +1043,7 @@ class MemoriesExtension(Extension, ABC):
ops,
fq_table,
bank_id: str,
fact_type: str | None = None,
fact_type: str | list[str] | None = None,
search_query: str | None = None,
consolidation_state: str | None = None,
state: str | None = None,
@@ -84,7 +84,7 @@ async def list_memory_units(
ops,
fq_table,
bank_id: str,
fact_type: str | None = None,
fact_type: str | list[str] | None = None,
search_query: str | None = None,
consolidation_state: str | None = None,
state: str | None = None,
@@ -104,7 +104,8 @@ async def list_memory_units(
ops: Dialect ops. Unused by this query; part of the interface signature.
fq_table: Table-name resolver.
bank_id: Filter by bank ID
fact_type: Filter by fact type (world, experience)
fact_type: Filter by fact type (world, experience). A list matches any of
them; an empty list is treated as no filter.
search_query: Full-text search query (searches text and context fields)
document_id: Optional filter to a single source document.
tags: Optional list of tag names to filter by. When omitted, no tag
@@ -154,8 +155,14 @@ async def list_memory_units(
if fact_type:
param_count += 1
query_conditions.append(f"fact_type = ${param_count}")
query_params.append(fact_type)
if isinstance(fact_type, str):
query_conditions.append(f"fact_type = ${param_count}")
query_params.append(fact_type)
else:
# A list is "any of these" — one array parameter rather than an IN list
# whose placeholder count varies with the caller's argument.
query_conditions.append(f"fact_type = ANY(${param_count}::text[])")
query_params.append(list(fact_type))
if document_id:
param_count += 1
@@ -240,7 +247,8 @@ async def list_memory_units(
f"""
SELECT id, text, event_date, context, fact_type, document_id,
mentioned_at, occurred_start, occurred_end, chunk_id, proof_count,
tags, metadata, consolidated_at, consolidation_failed_at, edited_at, {curation_cols}
tags, metadata, consolidated_at, consolidation_failed_at, edited_at,
updated_at, source_memory_ids, {curation_cols}
FROM {source_table}
{where_clause}
ORDER BY mentioned_at DESC NULLS LAST, created_at DESC
@@ -304,6 +312,12 @@ async def list_memory_units(
"invalidation_reason": row["invalidation_reason"],
"invalidated_at": row["invalidated_at"].isoformat() if row["invalidated_at"] else None,
"edited_at": row["edited_at"].isoformat() if row["edited_at"] else None,
# Both come off the row already selected above, so neither adds a
# query: updated_at is the write watermark curation and freshness
# checks compare against, and source_memory_ids is an observation's
# lineage (empty for a source fact).
"updated_at": row["updated_at"].isoformat() if row["updated_at"] else None,
"source_memory_ids": [str(sid) for sid in row["source_memory_ids"] or []],
}
)
@@ -463,7 +477,7 @@ async def list_entities(
# Get paginated entities
rows = await conn.fetch(
f"""
SELECT id, canonical_name, mention_count, first_seen, last_seen, metadata
SELECT id, canonical_name, entity_kind, mention_count, first_seen, last_seen, metadata
FROM {fq_table("entities")}
WHERE {where_clause}
ORDER BY mention_count DESC, last_seen DESC, id ASC
@@ -490,6 +504,9 @@ async def list_entities(
{
"id": str(row["id"]),
"canonical_name": row["canonical_name"],
# How the entity was classified (label vs free-form, etc.); same row,
# so listing it costs nothing extra.
"entity_kind": row["entity_kind"],
"mention_count": row["mention_count"],
"first_seen": row["first_seen"].isoformat() if row["first_seen"] else None,
"last_seen": row["last_seen"].isoformat() if row["last_seen"] else None,
@@ -459,7 +459,7 @@ class PostgresMemories(MemoriesExtension):
ops,
fq_table,
bank_id: str,
fact_type: str | None = None,
fact_type: str | list[str] | None = None,
search_query: str | None = None,
consolidation_state: str | None = None,
state: str | None = None,
@@ -9459,7 +9459,7 @@ class MemoryEngine(MemoryEngineInterface):
self,
bank_id: str,
*,
fact_type: str | None = None,
fact_type: str | list[str] | None = None,
search_query: str | None = None,
consolidation_state: str | None = None,
state: str | None = None,
@@ -9477,7 +9477,9 @@ class MemoryEngine(MemoryEngineInterface):
Args:
bank_id: Filter by bank ID
fact_type: Filter by fact type (world, experience)
fact_type: Filter by fact type (world, experience). A list matches any
of them (e.g. ``['world', 'experience']`` for source facts); an
empty list is treated as no filter.
search_query: Full-text search query (searches text and context fields)
document_id: Optional filter to a single source document.
entity_id: Optional filter to memory units linked to this entity ID
+1
View File
@@ -189,6 +189,7 @@ markers = [
"hs_llm_core: Core pipeline tests that need a real LLM but only one provider",
"integration: Live external-API integration tests (require provider credentials; skipped without)",
"slow: Slow tests (minutes); not run in fast CI",
"memory_backend_incompatible: asserts Postgres-internal state a non-SQL memories backend cannot reproduce — raw memory_links row counts (the graph read path dedupes bidirectional edges), or internal columns like embedding / search_vector that are not part of the public read model. Deselect when running against an alternative memories backend with -m 'not memory_backend_incompatible'.",
]
asyncio_mode = "auto"
asyncio_default_fixture_loop_scope = "function"
@@ -233,6 +233,7 @@ async def test_backup_restore_roundtrip(backup_test_schema):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_backup_restore_preserves_all_column_types(backup_test_schema):
"""Test that all column types are preserved: vectors, UUIDs, timestamps, JSONB."""
db_url, schema_name, _fq, embeddings = backup_test_schema
@@ -664,6 +664,7 @@ async def test_all_degenerate_facts_still_persist_document_chunks(memory, reques
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_streaming_offsets_chunk_local_causal_fact_indices(memory, request_context, monkeypatch):
"""Causal targets from independently extracted chunks must stay within their source chunk."""
from hindsight_api.engine.response_models import TokenUsage
@@ -765,17 +766,24 @@ async def test_degenerate_fact_preserves_later_chunk_provenance(memory, request_
request_context=request_context,
)
pool = await memory._get_pool()
rows = await pool.fetch(
"""
SELECT units.text AS fact_text, chunks.chunk_index AS chunk_index
FROM memory_units units
JOIN chunks ON chunks.chunk_id = units.chunk_id
WHERE units.bank_id = $1
""",
bank_id,
)
assert {(row["fact_text"], row["chunk_index"]) for row in rows} == {
# Each fact carries the chunk it came from, and the document's chunks carry
# their index, so the join the assertion needs is over two API reads.
chunk_index_by_id = {
chunk["chunk_id"]: chunk["chunk_index"]
for chunk in (
await memory.list_document_chunks(
bank_id, "degen-provenance-document", limit=500, request_context=request_context
)
)["items"]
}
units = (await memory.list_memory_units(bank_id, limit=500, request_context=request_context))["items"]
# Chunkless units (an observation, say) were dropped by the inner join before
# and are dropped here for the same reason: they have no provenance to check.
assert {
(unit["text"], chunk_index_by_id[unit["chunk_id"]])
for unit in units
if unit["chunk_id"] in chunk_index_by_id
} == {
("chunk zero real fact", 0),
("chunk one real fact", 1),
}
@@ -47,6 +47,7 @@ async def _insert_memory(memory, bank_id: str, text: str, *, failed: bool = Fals
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_bank_stats_exposes_memory_write_watermark(api_client, memory, test_bank_id):
"""/stats must carry the bank's newest memory write time.
@@ -217,6 +218,7 @@ async def test_memories_timeseries_reflects_retained_memories(api_client, test_b
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_bank_stats_reports_failed_consolidation(api_client, memory, test_bank_id):
"""/stats must surface the count of memories with consolidation_failed_at set."""
try:
@@ -237,6 +239,7 @@ async def test_bank_stats_reports_failed_consolidation(api_client, memory, test_
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_pending_consolidation_matches_pending_memory_list(api_client, memory, test_bank_id):
"""pending_consolidation must agree with ?consolidation_state=pending.
@@ -266,6 +269,7 @@ async def test_pending_consolidation_matches_pending_memory_list(api_client, mem
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_list_memories_filter_by_consolidation_state_failed(api_client, memory, test_bank_id):
"""?consolidation_state=failed returns only memories with consolidation_failed_at set."""
try:
@@ -332,6 +336,7 @@ async def test_bank_stats_link_counts_have_no_join(api_client, test_bank_id):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_get_bank_freshness_returns_only_consolidation_fields(memory, test_bank_id):
"""get_bank_freshness must return just the freshness keys, no link aggregation."""
from hindsight_api.extensions import RequestContext
@@ -55,6 +55,7 @@ class TestDistributedBankStatsCache:
assert isinstance(memory._bank_stats_cache, DistributedBankStatsCache)
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_result_is_written_and_served_from_table(self, memory: MemoryEngine, request_context: RequestContext):
if memory._database_backend_type != "postgresql":
pytest.skip("distributed cache is PostgreSQL-only")
@@ -93,6 +94,7 @@ class TestDistributedBankStatsCache:
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_force_refresh_bypasses_and_updates_cache(
self, memory: MemoryEngine, request_context: RequestContext
):
@@ -127,6 +129,7 @@ class TestDistributedBankStatsCache:
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_degrades_when_cache_table_unreachable(self, memory: MemoryEngine, request_context: RequestContext):
if memory._database_backend_type != "postgresql":
pytest.skip("distributed cache is PostgreSQL-only")
@@ -43,6 +43,7 @@ async def _seed_bank_and_document(conn, bank_id: str, document_id: str) -> None:
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_store_chunks_batch_is_idempotent_for_same_chunk_id(memory):
"""
Regression for #977.
+245 -361
View File
@@ -28,6 +28,20 @@ from hindsight_api.engine.reflect.tools import (
from tests.llm_judge import assert_meets_criteria
async def _unconsolidated(memory: MemoryEngine, bank_id: str, request_context) -> int:
"""How many source facts are still waiting to be consolidated.
consolidation_state='pending' is the read API's name for the predicate these
tests used to spell out in SQL: consolidated_at IS NULL and a source fact type.
It additionally excludes facts whose consolidation permanently failed — the
stricter reading, and the one the assertions here actually mean.
"""
page = await memory.list_memory_units(
bank_id=bank_id, consolidation_state="pending", limit=1, request_context=request_context
)
return page["total"]
@pytest.fixture(autouse=True)
def enable_observations():
"""Enable observations for all tests in this module."""
@@ -63,22 +77,18 @@ class TestConsolidationIntegration:
request_context=request_context,
)
# Verify observation exists in memory_units
# Verify observation exists via the list API
# (consolidation already ran as part of retain via SyncTaskBackend)
async with memory._pool.acquire() as conn:
observations = await conn.fetch(
"""
SELECT id, text, proof_count, fact_type
FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
""",
bank_id,
observations = (
await memory.list_memory_units(
bank_id, fact_type="observation", limit=1000, request_context=request_context
)
# With the deterministic mock, consolidation always produces observations
assert len(observations) >= 1, "Consolidation must create at least one observation"
obs = observations[0]
assert obs["proof_count"] >= 1
assert obs["fact_type"] == "observation"
)["items"]
# With the deterministic mock, consolidation always produces observations
assert len(observations) >= 1, "Consolidation must create at least one observation"
obs = observations[0]
assert obs["proof_count"] >= 1
assert obs["fact_type"] == "observation"
# Cleanup
await memory.delete_bank(bank_id, request_context=request_context)
@@ -106,21 +116,16 @@ class TestConsolidationIntegration:
)
# Check observations after both retains
async with memory._pool.acquire() as conn:
observations = await conn.fetch(
"""
SELECT id, text, proof_count
FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
ORDER BY proof_count DESC
""",
bank_id,
observations = (
await memory.list_memory_units(
bank_id, fact_type="observation", limit=1000, request_context=request_context
)
)["items"]
# Must have at least one observation from consolidation
assert len(observations) >= 1, "Consolidation must create observations from retained memories"
assert all(obs["text"] for obs in observations)
assert all(obs["proof_count"] >= 1 for obs in observations)
# Must have at least one observation from consolidation
assert len(observations) >= 1, "Consolidation must create observations from retained memories"
assert all(obs["text"] for obs in observations)
assert all(obs["proof_count"] >= 1 for obs in observations)
# Cleanup
await memory.delete_bank(bank_id, request_context=request_context)
@@ -208,31 +213,17 @@ class TestConsolidationIntegration:
)
# Check observation and its entity links
async with memory._pool.acquire() as conn:
observation = await conn.fetchrow(
"""
SELECT id
FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
LIMIT 1
""",
bank_id,
)
observations = await memory.list_memory_units(
bank_id, fact_type="observation", limit=1, request_context=request_context
)
# Consolidation must create an observation
assert observation is not None, "Consolidation must create an observation"
# Consolidation must create an observation
assert observations["items"], "Consolidation must create an observation"
# Check if entity links were copied
entity_links = await conn.fetch(
"""
SELECT entity_id
FROM unit_entities
WHERE unit_id = $1
""",
observation["id"],
)
# Observation should have inherited entity links from source memory
assert entity_links is not None
# An observation carries the entities of the facts it was drawn from, so the
# detail view is where the copied links show up.
observation = await memory.get_memory_unit(bank_id, observations["items"][0]["id"], request_context)
assert observation["entities"] is not None
# Cleanup
await memory.delete_bank(bank_id, request_context=request_context)
@@ -267,6 +258,7 @@ class TestConsolidationIntegration:
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_consolidation_uses_source_memory_ids(self, memory: MemoryEngine, request_context):
"""Test that observations use source_memory_ids (not memory_links) to track source facts.
@@ -366,31 +358,25 @@ class TestConsolidationIntegration:
)
# Check observations - should have separate observations for each person
async with memory._pool.acquire() as conn:
observations = await conn.fetch(
"""
SELECT id, text, source_memory_ids
FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
""",
bank_id,
observations = (
await memory.list_memory_units(
bank_id, fact_type="observation", limit=1000, request_context=request_context
)
)["items"]
# Should have multiple observations (one per person/fact)
# Not everything merged into one
assert len(observations) >= 2, (
f"Expected multiple observations for different people, got {len(observations)}"
)
# Should have multiple observations (one per person/fact)
# Not everything merged into one
assert len(observations) >= 2, f"Expected multiple observations for different people, got {len(observations)}"
# Fast structural check first: no single observation should name more
# than one of {John, Mary, Bob}. This catches the obvious failure mode
# cheaply without paying for a judge call per observation.
for obs in observations:
text = obs["text"].lower()
people_mentioned = sum(1 for name in ["john", "mary", "bob"] if name in text)
assert people_mentioned <= 1, f"Observation should not merge different people: {obs['text']}"
# Fast structural check first: no single observation should name more
# than one of {John, Mary, Bob}. This catches the obvious failure mode
# cheaply without paying for a judge call per observation.
for obs in observations:
text = obs["text"].lower()
people_mentioned = sum(1 for name in ["john", "mary", "bob"] if name in text)
assert people_mentioned <= 1, f"Observation should not merge different people: {obs['text']}"
obs_listing = "\n".join(f"Observation {i + 1}: {obs['text']}" for i, obs in enumerate(observations))
obs_listing = "\n".join(f"Observation {i + 1}: {obs['text']}" for i, obs in enumerate(observations))
# Semantic backup: catch the case where the LLM merges facts about different
# people using pronouns or referent shifts that bypass the proper-noun check
@@ -443,15 +429,12 @@ class TestConsolidationIntegration:
await memory.wait_for_background_tasks()
# Check we have one observation
async with memory._pool.acquire() as conn:
obs_before = await conn.fetch(
"""
SELECT id, text FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
""",
bank_id,
obs_before = (
await memory.list_memory_units(
bank_id, fact_type="observation", limit=1000, request_context=request_context
)
count_before = len(obs_before)
)["items"]
count_before = len(obs_before)
# Add contradicting fact (same person, same topic, opposite sentiment)
await memory.retain_async(
@@ -462,29 +445,23 @@ class TestConsolidationIntegration:
await memory.wait_for_background_tasks()
# Check observations after consolidation
async with memory._pool.acquire() as conn:
observations = await conn.fetch(
"""
SELECT id, text, source_memory_ids
FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
""",
bank_id,
)
observations = (
await memory.list_memory_units(bank_id, fact_type="observation", limit=500, request_context=request_context)
)["items"]
# The contradiction should be reflected in observations — either:
# 1. Merged into one observation with temporal context (e.g., "used to love, now hates")
# 2. The original observation updated to reflect the new state
# 3. Two separate observations capturing each state
# The key is that the contradiction is tracked, not ignored.
assert len(observations) >= 1, "Should have at least one observation after contradiction"
# The contradiction should be reflected in observations — either:
# 1. Merged into one observation with temporal context (e.g., "used to love, now hates")
# 2. The original observation updated to reflect the new state
# 3. Two separate observations capturing each state
# The key is that the contradiction is tracked, not ignored.
assert len(observations) >= 1, "Should have at least one observation after contradiction"
# Format as numbered list rather than pipe-separated — weaker judge
# models read pipe-joins as a single conflated statement.
obs_listing = "\n".join(f"Observation {i + 1}: {obs['text']}" for i, obs in enumerate(observations))
all_source_ids = []
for obs in observations:
all_source_ids.extend(obs["source_memory_ids"] or [])
# Format as numbered list rather than pipe-separated — weaker judge
# models read pipe-joins as a single conflated statement.
obs_listing = "\n".join(f"Observation {i + 1}: {obs['text']}" for i, obs in enumerate(observations))
all_source_ids = []
for obs in observations:
all_source_ids.extend(obs["source_memory_ids"])
# Either the observations reference both sentiments (via text content) or the
# consolidation linked both source memories together. The judge evaluates the
@@ -543,11 +520,11 @@ class TestConsolidationIntegration:
await memory.wait_for_background_tasks()
async with memory._pool.acquire() as conn:
observations = await conn.fetch(
"SELECT id, text FROM memory_units WHERE bank_id = $1 AND fact_type = 'observation' ORDER BY created_at",
bank_id,
observations = (
await memory.list_memory_units(
bank_id, fact_type="observation", limit=1000, request_context=request_context
)
)["items"]
obs_count = len(observations)
obs_texts = [o["text"] for o in observations]
@@ -760,19 +737,16 @@ class TestConsolidationTagRouting:
await self._retain_with_tags(memory, bank_id, "Alice likes coffee.", ["alice"], request_context)
# Check observation has correct tags
async with memory._pool.acquire() as conn:
obs_before = await conn.fetch(
"""
SELECT id, text, tags FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
""",
bank_id,
obs_before = (
await memory.list_memory_units(
bank_id, fact_type="observation", limit=1000, request_context=request_context
)
)["items"]
count_before = len(obs_before)
if obs_before:
assert "alice" in (obs_before[0]["tags"] or []), (
f"Expected observation to have 'alice' tag, got: {obs_before[0]['tags']}"
)
count_before = len(obs_before)
if obs_before:
assert "alice" in (obs_before[0]["tags"] or []), (
f"Expected observation to have 'alice' tag, got: {obs_before[0]['tags']}"
)
# Retain related memory with same tags
await self._retain_with_tags(
@@ -780,25 +754,22 @@ class TestConsolidationTagRouting:
)
# Check observations - should NOT have increased (same scope update)
async with memory._pool.acquire() as conn:
obs_after = await conn.fetch(
"""
SELECT id, text, tags, source_memory_ids FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
""",
bank_id,
obs_after = (
await memory.list_memory_units(
bank_id, fact_type="observation", limit=1000, request_context=request_context
)
)["items"]
# Count of observations should stay same or decrease (merge)
assert len(obs_after) <= count_before + 1, (
f"Same scope fact should update existing observation, not create new. "
f"Before: {count_before}, After: {len(obs_after)}"
)
# Count of observations should stay same or decrease (merge)
assert len(obs_after) <= count_before + 1, (
f"Same scope fact should update existing observation, not create new. "
f"Before: {count_before}, After: {len(obs_after)}"
)
# The observation(s) should still have alice tag
for obs in obs_after:
if "coffee" in obs["text"].lower() or "espresso" in obs["text"].lower():
assert "alice" in (obs["tags"] or []), f"Updated observation should keep 'alice' tag: {obs['text']}"
# The observation(s) should still have alice tag
for obs in obs_after:
if "coffee" in obs["text"].lower() or "espresso" in obs["text"].lower():
assert "alice" in (obs["tags"] or []), f"Updated observation should keep 'alice' tag: {obs['text']}"
# Cleanup
await memory.delete_bank(bank_id, request_context=request_context)
@@ -828,48 +799,41 @@ class TestConsolidationTagRouting:
await memory.wait_for_background_tasks()
# Check untagged observation exists
async with memory._pool.acquire() as conn:
obs_before = await conn.fetch(
"""
SELECT id, text, tags FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
""",
bank_id,
obs_before = (
await memory.list_memory_units(
bank_id, fact_type="observation", limit=1000, request_context=request_context
)
)["items"]
count_before = len(obs_before)
# Should be untagged or have empty tags
if obs_before:
assert not obs_before[0]["tags"] or len(obs_before[0]["tags"]) == 0, (
f"Expected untagged observation, got: {obs_before[0]['tags']}"
)
count_before = len(obs_before)
# Should be untagged or have empty tags
if obs_before:
assert not obs_before[0]["tags"] or len(obs_before[0]["tags"]) == 0, (
f"Expected untagged observation, got: {obs_before[0]['tags']}"
)
# Retain scoped memory that relates to the global topic
await self._retain_with_tags(memory, bank_id, "Pizza originated in Naples.", ["history"], request_context)
await memory.wait_for_background_tasks()
# Check - global observation should be updated OR new scoped observation created
async with memory._pool.acquire() as conn:
obs_after = await conn.fetch(
"""
SELECT id, text, tags, source_memory_ids FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
ORDER BY created_at
""",
bank_id,
obs_after = (
await memory.list_memory_units(
bank_id, fact_type="observation", limit=1000, request_context=request_context
)
)["items"]
# At least one observation should exist
assert len(obs_after) >= 1, "Expected at least one observation"
# At least one observation should exist
assert len(obs_after) >= 1, "Expected at least one observation"
# Check that global observation was updated (source_memory_ids increased)
# OR new observation was created with appropriate tags
global_observations = [o for o in obs_after if not o["tags"] or len(o["tags"]) == 0]
scoped_observations = [o for o in obs_after if o["tags"] and len(o["tags"]) > 0]
# Check that global observation was updated (source_memory_ids increased)
# OR new observation was created with appropriate tags
global_observations = [o for o in obs_after if not o["tags"] or len(o["tags"]) == 0]
scoped_observations = [o for o in obs_after if o["tags"] and len(o["tags"]) > 0]
# Either global was updated or scoped was created
assert len(global_observations) >= 1 or len(scoped_observations) >= 1, (
"Expected either global observation update or scoped observation creation"
)
# Either global was updated or scoped was created
assert len(global_observations) >= 1 or len(scoped_observations) >= 1, (
"Expected either global observation update or scoped observation creation"
)
# Cleanup
await memory.delete_bank(bank_id, request_context=request_context)
@@ -897,15 +861,12 @@ class TestConsolidationTagRouting:
await memory.wait_for_background_tasks()
# Check Alice's observation exists with correct tags
async with memory._pool.acquire() as conn:
obs_alice = await conn.fetch(
"""
SELECT id, text, tags FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
""",
bank_id,
obs_alice = (
await memory.list_memory_units(
bank_id, fact_type="observation", limit=1000, request_context=request_context
)
count_before = len(obs_alice)
)["items"]
count_before = len(obs_alice)
# Retain Bob's memory that relates to Alice's topic (cross-scope)
await self._retain_with_tags(
@@ -914,28 +875,22 @@ class TestConsolidationTagRouting:
await memory.wait_for_background_tasks()
# Check observations
async with memory._pool.acquire() as conn:
obs_after = await conn.fetch(
"""
SELECT id, text, tags, source_memory_ids FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
ORDER BY created_at
""",
bank_id,
obs_after = (
await memory.list_memory_units(
bank_id, fact_type="observation", limit=1000, request_context=request_context
)
)["items"]
# Note: some LLMs may or may not consolidate cross-scope facts.
# Just verify structural correctness of any observations that exist.
# Note: some LLMs may or may not consolidate cross-scope facts.
# Just verify structural correctness of any observations that exist.
# If observations were created, ensure alice and bob are not merged into same observation
# (cross-scope merging should not produce an observation with both tags)
if obs_after:
observations_with_both = [
o for o in obs_after if o["tags"] and "alice" in o["tags"] and "bob" in o["tags"]
]
assert len(observations_with_both) == 0, (
"Should not merge different scopes into one observation with both tags"
)
# If observations were created, ensure alice and bob are not merged into same observation
# (cross-scope merging should not produce an observation with both tags)
if obs_after:
observations_with_both = [o for o in obs_after if o["tags"] and "alice" in o["tags"] and "bob" in o["tags"]]
assert len(observations_with_both) == 0, (
"Should not merge different scopes into one observation with both tags"
)
# Cleanup
await memory.delete_bank(bank_id, request_context=request_context)
@@ -962,21 +917,18 @@ class TestConsolidationTagRouting:
)
# Check observation was created with correct tags
async with memory._pool.acquire() as conn:
observations = await conn.fetch(
"""
SELECT id, text, tags FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
""",
bank_id,
observations = (
await memory.list_memory_units(
bank_id, fact_type="observation", limit=1000, request_context=request_context
)
)["items"]
assert len(observations) >= 1, "Expected observation to be created"
assert len(observations) >= 1, "Expected observation to be created"
# The observation should have the fact's tags
obs = observations[0]
assert obs["tags"] is not None, "Observation should have tags"
assert "project_x" in obs["tags"], f"Observation should have 'project_x' tag, got: {obs['tags']}"
# The observation should have the fact's tags
obs = observations[0]
assert obs["tags"] is not None, "Observation should have tags"
assert "project_x" in obs["tags"], f"Observation should have 'project_x' tag, got: {obs['tags']}"
# Cleanup
await memory.delete_bank(bank_id, request_context=request_context)
@@ -1013,22 +965,18 @@ class TestConsolidationTagRouting:
await memory.wait_for_background_tasks()
# Check observations
async with memory._pool.acquire() as conn:
observations = await conn.fetch(
"""
SELECT id, text, tags, source_memory_ids FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
ORDER BY created_at
""",
bank_id,
observations = (
await memory.list_memory_units(
bank_id, fact_type="observation", limit=1000, request_context=request_context
)
)["items"]
# Either alice's observation was updated OR a global observation was created
# This is valid LLM behavior - just verify no errors and structure is correct.
# Note: with some LLMs, a single simple fact may not generate an observation,
# so we don't assert a minimum count - just verify structural correctness if any exist.
for obs in observations:
assert obs["text"], "Observation should have text"
# Either alice's observation was updated OR a global observation was created
# This is valid LLM behavior - just verify no errors and structure is correct.
# Note: with some LLMs, a single simple fact may not generate an observation,
# so we don't assert a minimum count - just verify structural correctness if any exist.
for obs in observations:
assert obs["text"], "Observation should have text"
# Cleanup
await memory.delete_bank(bank_id, request_context=request_context)
@@ -1099,15 +1047,12 @@ class TestConsolidationTagRouting:
await self._retain_with_tags(memory, bank_id, "Alice drinks coffee every morning.", ["alice"], request_context)
# Check observations before
async with memory._pool.acquire() as conn:
obs_before = await conn.fetch(
"""
SELECT id, text, tags, source_memory_ids FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
""",
bank_id,
obs_before = (
await memory.list_memory_units(
bank_id, fact_type="observation", limit=1000, request_context=request_context
)
count_before = len(obs_before)
)["items"]
count_before = len(obs_before)
# Add fact that could relate to both
await self._retain_with_tags(
@@ -1115,24 +1060,20 @@ class TestConsolidationTagRouting:
)
# Check observations after
async with memory._pool.acquire() as conn:
obs_after = await conn.fetch(
"""
SELECT id, text, tags, source_memory_ids, proof_count FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
ORDER BY created_at
""",
bank_id,
obs_after = (
await memory.list_memory_units(
bank_id, fact_type="observation", limit=1000, request_context=request_context
)
)["items"]
# Should have processed without errors
assert len(obs_after) >= 1, "Expected at least one observation"
# Should have processed without errors
assert len(obs_after) >= 1, "Expected at least one observation"
# Check that consolidation worked (either updates or maintains structure)
# The key is no errors and proper tag handling
for obs in obs_after:
assert obs["text"], "Observation should have text"
# Tags should be consistent (not mixing alice and bob, etc.)
# Check that consolidation worked (either updates or maintains structure)
# The key is no errors and proper tag handling
for obs in obs_after:
assert obs["text"], "Observation should have text"
# Tags should be consistent (not mixing alice and bob, etc.)
# Cleanup
await memory.delete_bank(bank_id, request_context=request_context)
@@ -1442,34 +1383,29 @@ class TestObservationDrillDown:
)
# Get the observation with source_memory_ids
async with memory._pool.acquire() as conn:
obs_rows = await conn.fetch(
"""
SELECT id, text, proof_count, source_memory_ids
FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
""",
bank_id,
)
obs_rows = (
await memory.list_memory_units(bank_id, fact_type="observation", limit=500, request_context=request_context)
)["items"]
assert obs_rows, "Consolidation must create observations"
# Collect all source_memory_ids across all observations
all_source_ids = []
for obs in obs_rows:
all_source_ids.extend(obs["source_memory_ids"] or [])
all_source_ids.extend(obs["source_memory_ids"])
assert all_source_ids, "Observations must have source_memory_ids"
# Verify source_memory_ids point to actual memories
async with memory._pool.acquire() as conn:
source_memories = await conn.fetch(
"""
SELECT id, text FROM memory_units
WHERE id = ANY($1) AND fact_type IN ('world', 'experience')
""",
all_source_ids,
# Verify source_memory_ids point to actual memories. The list arm takes both
# source fact types at once, so the lineage is checked against the same set
# the SQL IN clause used to name.
source_facts = (
await memory.list_memory_units(
bank_id, fact_type=["world", "experience"], limit=500, request_context=request_context
)
)["items"]
wanted = set(all_source_ids)
source_memories = [m for m in source_facts if m["id"] in wanted]
assert len(source_memories) >= 1, (
f"source_memory_ids should point to valid memories. IDs: {all_source_ids}, Found: {len(source_memories)}"
@@ -1519,14 +1455,11 @@ class TestHierarchicalRetrieval:
)
# Verify observation was created
async with memory._pool.acquire() as conn:
obs_count = await conn.fetchval(
"""
SELECT COUNT(*) FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
""",
bank_id,
obs_count = (
await memory.list_memory_units(
bank_id, fact_type="observation", limit=1000, request_context=request_context
)
)["total"]
assert obs_count >= 1, "Consolidation should have created an observation"
# Create a mental model about John (higher quality, user-curated)
@@ -1992,11 +1925,11 @@ async def test_consolidation_with_observations_mission(memory: "MemoryEngine", r
content="Alice uses Python for data analysis and loves its simplicity.",
request_context=request_context,
)
async with memory._pool.acquire() as conn:
observations = await conn.fetch(
"SELECT id, text, fact_type FROM memory_units WHERE bank_id = $1 AND fact_type = 'observation'",
bank_id,
observations = (
await memory.list_memory_units(
bank_id, fact_type="observation", limit=1000, request_context=request_context
)
)["items"]
assert isinstance(observations, list)
finally:
memory._config_resolver._global_config = original_global_config
@@ -2010,6 +1943,7 @@ async def test_consolidation_with_observations_mission(memory: "MemoryEngine", r
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_observation_scopes_explicit_multi_pass(memory: MemoryEngine, request_context):
"""Test that observation_scopes with an explicit list triggers separate consolidation passes.
@@ -2037,16 +1971,9 @@ async def test_observation_scopes_explicit_multi_pass(memory: MemoryEngine, requ
request_context=request_context,
)
async with memory._pool.acquire() as conn:
observations = await conn.fetch(
"""
SELECT id, text, tags
FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
ORDER BY created_at
""",
bank_id,
)
observations = (
await memory.list_memory_units(bank_id, fact_type="observation", limit=1000, request_context=request_context)
)["items"]
try:
# Must have at least 2 observations (one per tag scope)
@@ -2073,6 +2000,7 @@ async def test_observation_scopes_explicit_multi_pass(memory: MemoryEngine, requ
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_observation_scopes_per_tag(memory: MemoryEngine, request_context):
"""Test that observation_scopes='per_tag' derives one pass per individual tag.
@@ -2095,16 +2023,9 @@ async def test_observation_scopes_per_tag(memory: MemoryEngine, request_context)
request_context=request_context,
)
async with memory._pool.acquire() as conn:
observations = await conn.fetch(
"""
SELECT id, text, tags
FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
ORDER BY created_at
""",
bank_id,
)
observations = (
await memory.list_memory_units(bank_id, fact_type="observation", limit=1000, request_context=request_context)
)["items"]
try:
assert len(observations) >= 2, (
@@ -2147,16 +2068,9 @@ async def test_observation_scopes_combined(memory: MemoryEngine, request_context
request_context=request_context,
)
async with memory._pool.acquire() as conn:
observations = await conn.fetch(
"""
SELECT id, text, tags
FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
ORDER BY created_at
""",
bank_id,
)
observations = (
await memory.list_memory_units(bank_id, fact_type="observation", limit=1000, request_context=request_context)
)["items"]
try:
assert len(observations) >= 1, "Expected at least 1 observation, got 0"
@@ -2178,6 +2092,7 @@ async def test_observation_scopes_combined(memory: MemoryEngine, request_context
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_observation_scopes_all_combinations(memory: MemoryEngine, request_context):
"""Test that observation_scopes='all_combinations' generates passes for every tag subset.
@@ -2201,16 +2116,9 @@ async def test_observation_scopes_all_combinations(memory: MemoryEngine, request
request_context=request_context,
)
async with memory._pool.acquire() as conn:
observations = await conn.fetch(
"""
SELECT id, text, tags
FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
ORDER BY created_at
""",
bank_id,
)
observations = (
await memory.list_memory_units(bank_id, fact_type="observation", limit=1000, request_context=request_context)
)["items"]
try:
# With 2 tags there are 3 subsets: {alice}, {ben}, {alice, ben}
@@ -2671,6 +2579,7 @@ def test_max_observations_per_scope_default():
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_count_observations_for_scope(memory: MemoryEngine, request_context):
"""Test _count_observations_for_scope counts observations filtered by tags."""
bank_id = f"test-count-obs-scope-{uuid.uuid4().hex[:8]}"
@@ -2762,6 +2671,7 @@ def _make_mock_llm_one_obs_per_fact():
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_max_observations_per_scope_limits_creates(memory: MemoryEngine, request_context):
"""Mock LLM tries to create 1 obs per fact; with limit=2, only 2 should exist after 5 facts."""
bank_id = f"test-max-obs-limit-{uuid.uuid4().hex[:8]}"
@@ -2818,6 +2728,7 @@ async def test_max_observations_per_scope_limits_creates(memory: MemoryEngine, r
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_max_observations_per_scope_zero_forbids_all_creates(memory: MemoryEngine, request_context):
"""limit=0 means "no new observations": consolidation must create none.
@@ -2871,6 +2782,7 @@ async def test_max_observations_per_scope_zero_forbids_all_creates(memory: Memor
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_max_observations_per_scope_allows_updates_at_capacity(memory: MemoryEngine, request_context):
"""At capacity, the LLM can still update existing observations."""
from hindsight_api.engine.consolidation.consolidator import (
@@ -2971,6 +2883,7 @@ async def test_max_observations_per_scope_allows_updates_at_capacity(memory: Mem
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_max_observations_per_scope_no_tags_skips_limit(memory: MemoryEngine, request_context):
"""With limit=1, memories with no tags should bypass the limit and create freely."""
bank_id = f"test-max-obs-no-tags-{uuid.uuid4().hex[:8]}"
@@ -3006,12 +2919,12 @@ async def test_max_observations_per_scope_no_tags_skips_limit(memory: MemoryEngi
await run_consolidation_job(memory_engine=memory, bank_id=bank_id, request_context=request_context)
# No tag limit should apply — all 3 observations should be created
async with memory._pool.acquire() as conn:
obs = await conn.fetch(
"SELECT id FROM memory_units WHERE bank_id = $1 AND fact_type = 'observation'",
bank_id,
total = (
await memory.list_memory_units(
bank_id, fact_type="observation", limit=1000, request_context=request_context
)
assert len(obs) == 3, f"Expected 3 observations (no limit for no-tag), got {len(obs)}"
)["total"]
assert total == 3, f"Expected 3 observations (no limit for no-tag), got {total}"
finally:
memory._config_resolver._global_config = original_global_config
memory._consolidation_llm_config = original_llm
@@ -3020,6 +2933,7 @@ async def test_max_observations_per_scope_no_tags_skips_limit(memory: MemoryEngi
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_max_observations_unlimited_default(memory: MemoryEngine, request_context):
"""With default config (-1), all creates go through."""
bank_id = f"test-max-obs-unlimited-{uuid.uuid4().hex[:8]}"
@@ -3053,6 +2967,7 @@ async def test_max_observations_unlimited_default(memory: MemoryEngine, request_
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_targeted_consolidation_filters_by_scopes(memory: MemoryEngine, request_context):
"""Consolidation with observation_scopes only processes memories matching those scopes."""
bank_id = f"test-targeted-{uuid.uuid4().hex[:8]}"
@@ -3085,17 +3000,10 @@ async def test_targeted_consolidation_filters_by_scopes(memory: MemoryEngine, re
alice_obs = await _count_observations_for_scope(conn, bank_id, ["user:alice"])
assert alice_obs == 1
# Bob and Charlie should still be unconsolidated
unconsolidated = await conn.fetchval(
"""
SELECT COUNT(*) FROM memory_units
WHERE bank_id = $1
AND consolidated_at IS NULL
AND fact_type IN ('experience', 'world')
""",
bank_id,
)
assert unconsolidated == 2
# Bob and Charlie should still be unconsolidated. consolidation_state='pending'
# is the read API's name for this predicate; it also excludes facts whose
# consolidation failed, which is the stricter (and here equivalent) reading.
assert await _unconsolidated(memory, bank_id, request_context) == 2
# Now consolidate bob
result = await run_consolidation_job(
@@ -3107,23 +3015,14 @@ async def test_targeted_consolidation_filters_by_scopes(memory: MemoryEngine, re
assert result["memories_processed"] == 1
# Charlie still unconsolidated
async with memory._pool.acquire() as conn:
unconsolidated = await conn.fetchval(
"""
SELECT COUNT(*) FROM memory_units
WHERE bank_id = $1
AND consolidated_at IS NULL
AND fact_type IN ('experience', 'world')
""",
bank_id,
)
assert unconsolidated == 1
assert await _unconsolidated(memory, bank_id, request_context) == 1
finally:
memory._consolidation_llm_config = original_llm
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_targeted_consolidation_multiple_scopes(memory: MemoryEngine, request_context):
"""Consolidation with multiple observation_scopes matches memories in any scope."""
bank_id = f"test-targeted-multi-{uuid.uuid4().hex[:8]}"
@@ -3151,23 +3050,14 @@ async def test_targeted_consolidation_multiple_scopes(memory: MemoryEngine, requ
assert result["observations_created"] == 2
# Bob still unconsolidated
async with memory._pool.acquire() as conn:
unconsolidated = await conn.fetchval(
"""
SELECT COUNT(*) FROM memory_units
WHERE bank_id = $1
AND consolidated_at IS NULL
AND fact_type IN ('experience', 'world')
""",
bank_id,
)
assert unconsolidated == 1
assert await _unconsolidated(memory, bank_id, request_context) == 1
finally:
memory._consolidation_llm_config = original_llm
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_targeted_consolidation_no_scopes_processes_all(memory: MemoryEngine, request_context):
"""Consolidation without observation_scopes processes all unconsolidated memories (backward compat)."""
bank_id = f"test-targeted-all-{uuid.uuid4().hex[:8]}"
@@ -3197,6 +3087,7 @@ async def test_targeted_consolidation_no_scopes_processes_all(memory: MemoryEngi
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_targeted_consolidation_contains_semantics(memory: MemoryEngine, request_context):
"""Scope ["user:alice"] matches memories tagged ["user:alice", "team:eng"] (contains)."""
bank_id = f"test-targeted-contains-{uuid.uuid4().hex[:8]}"
@@ -3253,23 +3144,15 @@ async def test_enable_auto_consolidation_flag(memory: MemoryEngine, request_cont
)
# Check that memories are NOT consolidated
async with memory._pool.acquire() as conn:
unconsolidated = await conn.fetchval(
"""
SELECT COUNT(*) FROM memory_units
WHERE bank_id = $1
AND consolidated_at IS NULL
AND fact_type IN ('experience', 'world')
""",
bank_id,
)
assert unconsolidated > 0, "Memories should remain unconsolidated when auto consolidation is disabled"
unconsolidated = await _unconsolidated(memory, bank_id, request_context)
assert unconsolidated > 0, "Memories should remain unconsolidated when auto consolidation is disabled"
observations = await conn.fetchval(
"SELECT COUNT(*) FROM memory_units WHERE bank_id = $1 AND fact_type = 'observation'",
bank_id,
observations = (
await memory.list_memory_units(
bank_id, fact_type="observation", limit=1000, request_context=request_context
)
assert observations == 0, "No observations should be created when auto consolidation is disabled"
)["total"]
assert observations == 0, "No observations should be created when auto consolidation is disabled"
finally:
memory._config_resolver._global_config = original_global_config
await memory.delete_bank(bank_id, request_context=request_context)
@@ -3330,6 +3213,7 @@ def test_consolidation_prompt_split_is_cacheable_and_complete():
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_create_observation_populates_search_vector_native(memory, request_context):
"""Observations created via consolidation must have search_vector populated
when text_search_extension == 'native', so BM25 retrieval finds them."""
@@ -421,6 +421,7 @@ def _update_ctx(threshold: float = 0.97):
return kwargs, conn, llm
@pytest.mark.memory_backend_incompatible
async def test_dedup_update_merge_folds_into_twin_and_deletes_updated() -> None:
kwargs, conn, llm = _update_ctx()
llm.call.return_value = _DedupDecision(action="merge", text="Uzbek YouTube content is very rich and growing.")
@@ -167,12 +167,10 @@ async def _insert_memory(conn, bank_id: str, text: str, tags: list[str]) -> uuid
return mem_id
async def _count_observations(memory: MemoryEngine, bank_id: str) -> int:
async with memory._pool.acquire() as conn:
return await conn.fetchval(
"SELECT count(*) FROM memory_units WHERE bank_id = $1 AND fact_type = 'observation'",
bank_id,
)
async def _count_observations(memory: MemoryEngine, bank_id: str, request_context) -> int:
return (
await memory.list_memory_units(bank_id, fact_type="observation", limit=1000, request_context=request_context)
)["total"]
class _TxnSpy:
@@ -196,6 +194,7 @@ class _TxnSpy:
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_recall_failure_cancels_sibling_tag_groups(memory: MemoryEngine, request_context):
"""One group's recall times out → the other two groups are cancelled before
they write, and the job propagates the original error without waiting for
@@ -253,12 +252,13 @@ async def test_recall_failure_cancels_sibling_tag_groups(memory: MemoryEngine, r
# Nothing was written: the cancelled groups never reached their commit,
# and no orphan lands a write after the operation has already failed.
await asyncio.sleep(0.2)
assert await _count_observations(memory, bank_id) == 0
assert await _count_observations(memory, bank_id, request_context) == 0
finally:
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_failed_batch_aborts_its_write_group(memory: MemoryEngine, request_context):
"""A batch that raises before its witness commit decides its write-group
abort, rather than leaving it pending for the recovery sweep."""
@@ -292,6 +292,7 @@ async def test_failed_batch_aborts_its_write_group(memory: MemoryEngine, request
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_successful_batch_still_commits_its_write_group(memory: MemoryEngine, request_context):
"""Guard on the abort path: the happy path must still decide commit=True."""
bank_id = f"test-commit-{uuid.uuid4().hex[:8]}"
@@ -122,6 +122,7 @@ class TestAdaptiveBatchSplitting:
"""Verify that a failing batch is halved and retried until batch_size=1 succeeds."""
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_splitting_recovers_all_memories(self, memory_no_llm_verify: MemoryEngine, request_context):
"""When a batch of 2 fails, both are retried individually and succeed."""
bank_id = f"test-split-recovery-{uuid.uuid4().hex[:8]}"
@@ -152,15 +153,11 @@ class TestAdaptiveBatchSplitting:
assert result["memories_failed"] == 0
# Both memories must have consolidated_at set and consolidation_failed_at NULL
async with memory_no_llm_verify._pool.acquire() as conn:
rows = await conn.fetch(
"""
SELECT id, consolidated_at, consolidation_failed_at
FROM memory_units
WHERE bank_id = $1 AND fact_type = 'experience'
""",
bank_id,
rows = (
await memory_no_llm_verify.list_memory_units(
bank_id, fact_type="experience", limit=1000, request_context=request_context
)
)["items"]
assert len(rows) == 2
for row in rows:
assert row["consolidated_at"] is not None, f"Memory {row['id']} should have consolidated_at set"
@@ -175,6 +172,7 @@ class TestAdaptiveBatchSplitting:
await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_splitting_with_larger_batch(self, memory_no_llm_verify: MemoryEngine, request_context):
"""A batch of 4 that always fails at size>1 resolves to 4 individual calls."""
bank_id = f"test-split-large-{uuid.uuid4().hex[:8]}"
@@ -206,12 +204,11 @@ class TestAdaptiveBatchSplitting:
assert result["memories_processed"] == 4
assert result["memories_failed"] == 0
async with memory_no_llm_verify._pool.acquire() as conn:
rows = await conn.fetch(
"SELECT consolidated_at, consolidation_failed_at FROM memory_units "
"WHERE bank_id = $1 AND fact_type = 'experience'",
bank_id,
rows = (
await memory_no_llm_verify.list_memory_units(
bank_id, fact_type="experience", limit=1000, request_context=request_context
)
)["items"]
assert all(r["consolidated_at"] is not None for r in rows)
assert all(r["consolidation_failed_at"] is None for r in rows)
@@ -222,6 +219,7 @@ class TestConsolidationFailedAt:
"""Verify that consolidation_failed_at is set — and consolidated_at is NOT — when all retries fail."""
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_single_memory_permanent_failure(self, memory_no_llm_verify: MemoryEngine, request_context):
"""A single memory that exhausts all LLM retries gets consolidation_failed_at, not consolidated_at."""
bank_id = f"test-perm-fail-{uuid.uuid4().hex[:8]}"
@@ -243,11 +241,12 @@ class TestConsolidationFailedAt:
assert result["memories_failed"] == 1
assert result["memories_processed"] == 1
async with memory_no_llm_verify._pool.acquire() as conn:
row = await conn.fetchrow(
"SELECT consolidated_at, consolidation_failed_at FROM memory_units WHERE id = $1",
mem_id,
units = (
await memory_no_llm_verify.list_memory_units(
bank_id, fact_type="experience", limit=1000, request_context=request_context
)
)["items"]
row = next(u for u in units if str(u["id"]) == str(mem_id))
assert row["consolidated_at"] is None, "consolidated_at must NOT be set for a permanently failed memory"
assert row["consolidation_failed_at"] is not None, "consolidation_failed_at must be set"
@@ -255,6 +254,7 @@ class TestConsolidationFailedAt:
await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_failed_memory_excluded_from_next_run(self, memory_no_llm_verify: MemoryEngine, request_context):
"""A memory marked consolidation_failed_at is not re-processed on the next consolidation run."""
bank_id = f"test-excluded-{uuid.uuid4().hex[:8]}"
@@ -285,17 +285,19 @@ class TestConsolidationFailedAt:
assert result["memories_processed"] == 0
# Memory still has consolidation_failed_at set and consolidated_at NULL
async with memory_no_llm_verify._pool.acquire() as conn:
row = await conn.fetchrow(
"SELECT consolidated_at, consolidation_failed_at FROM memory_units WHERE id = $1",
mem_id,
units = (
await memory_no_llm_verify.list_memory_units(
bank_id, fact_type="experience", limit=1000, request_context=request_context
)
)["items"]
row = next(u for u in units if str(u["id"]) == str(mem_id))
assert row["consolidated_at"] is None
assert row["consolidation_failed_at"] is not None
await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_partial_batch_failure(self, memory_no_llm_verify: MemoryEngine, request_context):
"""In a batch of 2, if only the first individual retry fails, the second still succeeds."""
bank_id = f"test-partial-fail-{uuid.uuid4().hex[:8]}"
@@ -325,15 +327,12 @@ class TestConsolidationFailedAt:
assert result["memories_processed"] == 2
assert result["memories_failed"] == 1
async with memory_no_llm_verify._pool.acquire() as conn:
rows = {
str(r["id"]): r
for r in await conn.fetch(
"SELECT id, consolidated_at, consolidation_failed_at FROM memory_units "
"WHERE bank_id = $1 AND fact_type = 'experience'",
bank_id,
)
}
items = (
await memory_no_llm_verify.list_memory_units(
bank_id, fact_type="experience", limit=1000, request_context=request_context
)
)["items"]
rows = {str(r["id"]): r for r in items}
# One should have failed, one should have succeeded
failed = [r for r in rows.values() if r["consolidation_failed_at"] is not None]
@@ -350,6 +349,7 @@ class TestRecoverConsolidation:
"""Verify the retry_failed_consolidation() method and the /consolidation/recover endpoint."""
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_recover_resets_failed_memories(self, memory_no_llm_verify: MemoryEngine, request_context):
"""retry_failed_consolidation resets consolidation_failed_at and consolidated_at."""
bank_id = f"test-recover-reset-{uuid.uuid4().hex[:8]}"
@@ -375,12 +375,11 @@ class TestRecoverConsolidation:
assert result["retried_count"] == 2
async with memory_no_llm_verify._pool.acquire() as conn:
rows = await conn.fetch(
"SELECT consolidated_at, consolidation_failed_at FROM memory_units "
"WHERE bank_id = $1 AND fact_type = 'experience'",
bank_id,
rows = (
await memory_no_llm_verify.list_memory_units(
bank_id, fact_type="experience", limit=1000, request_context=request_context
)
)["items"]
assert all(r["consolidation_failed_at"] is None for r in rows), "consolidation_failed_at must be cleared"
assert all(r["consolidated_at"] is None for r in rows), "consolidated_at must also be cleared"
@@ -399,6 +398,7 @@ class TestRecoverConsolidation:
await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_recover_then_consolidate_succeeds(self, memory_no_llm_verify: MemoryEngine, request_context):
"""After recovery, the memory is picked up by the next consolidation run."""
bank_id = f"test-recover-consolidate-{uuid.uuid4().hex[:8]}"
@@ -425,17 +425,19 @@ class TestRecoverConsolidation:
assert run_result["memories_processed"] == 1
assert run_result["memories_failed"] == 0
async with memory_no_llm_verify._pool.acquire() as conn:
row = await conn.fetchrow(
"SELECT consolidated_at, consolidation_failed_at FROM memory_units WHERE id = $1",
mem_id,
units = (
await memory_no_llm_verify.list_memory_units(
bank_id, fact_type="experience", limit=1000, request_context=request_context
)
)["items"]
row = next(u for u in units if str(u["id"]) == str(mem_id))
assert row["consolidated_at"] is not None, "Memory should be consolidated after recovery"
assert row["consolidation_failed_at"] is None
await memory_no_llm_verify.delete_bank(bank_id, request_context=request_context)
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_recover_endpoint_via_http(self, memory_no_llm_verify: MemoryEngine, request_context):
"""The POST /consolidation/recover endpoint returns the correct retried_count."""
import httpx
@@ -79,16 +79,16 @@ async def _insert_entity_mm(conn, bank_id: str, tag: str) -> str:
return mm_id
async def _unconsolidated_count(memory, bank_id: str) -> int:
async with memory._pool.acquire() as conn:
return await conn.fetchval(
"""
SELECT COUNT(*) FROM memory_units
WHERE bank_id = $1 AND consolidated_at IS NULL
AND consolidation_failed_at IS NULL AND fact_type IN ('experience', 'world')
""",
bank_id,
)
async def _unconsolidated_count(memory, bank_id: str, request_context) -> int:
# consolidation_state='pending' is the read API's name for exactly this predicate:
# not consolidated, not failed, and a source fact type.
page = await memory.list_memory_units(
bank_id=bank_id,
consolidation_state="pending",
limit=1,
request_context=request_context,
)
return page["total"]
async def _pending_consolidations(memory, bank_id: str):
@@ -105,6 +105,7 @@ async def _pending_consolidations(memory, bank_id: str):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_multi_round_consolidation_refreshes_all_entity_models(
memory: MemoryEngine, request_context, monkeypatch
):
@@ -203,7 +204,7 @@ async def test_multi_round_consolidation_refreshes_all_entity_models(
assert consolidation_runs >= 2, (
f"expected a multi-round drain (round limit 3, backlog > 3), but only {consolidation_runs} consolidation(s) ran"
)
remaining = await _unconsolidated_count(memory, bank_id)
remaining = await _unconsolidated_count(memory, bank_id, request_context)
assert remaining == 0, f"backlog did not fully drain: {remaining} unconsolidated memories remain"
# 6. KEY ASSERTION — every refresh_after_consolidation model must be refreshed
@@ -55,16 +55,16 @@ def enable_observations():
config.enable_observations = original
async def _count_unconsolidated(memory, bank_id: str) -> int:
async with memory._pool.acquire() as conn:
return await conn.fetchval(
"""
SELECT COUNT(*) FROM memory_units
WHERE bank_id = $1 AND consolidated_at IS NULL
AND consolidation_failed_at IS NULL AND fact_type IN ('experience', 'world')
""",
bank_id,
)
async def _count_unconsolidated(memory, bank_id: str, request_context) -> int:
# consolidation_state='pending' is the read API's name for exactly this predicate:
# not consolidated, not failed, and a source fact type.
page = await memory.list_memory_units(
bank_id=bank_id,
consolidation_state="pending",
limit=1,
request_context=request_context,
)
return page["total"]
async def _pending_consolidation_ops(memory, bank_id: str) -> list[str]:
@@ -100,7 +100,7 @@ async def test_round_limited_consolidation_leaves_followup_pending_op(memory: Me
request_context=request_context,
)
unconsolidated_before = await _count_unconsolidated(memory, bank_id)
unconsolidated_before = await _count_unconsolidated(memory, bank_id, request_context)
assert unconsolidated_before >= backlog_size, (
f"Expected at least {backlog_size} unconsolidated memories, got {unconsolidated_before}"
)
@@ -148,7 +148,7 @@ async def test_round_limited_consolidation_leaves_followup_pending_op(memory: Me
assert row["status"] == "completed", f"first consolidation op should be marked completed, got {row['status']}"
# 4. Backlog must remain (round limit kept one round under the total)
unconsolidated_after = await _count_unconsolidated(memory, bank_id)
unconsolidated_after = await _count_unconsolidated(memory, bank_id, request_context)
assert 0 < unconsolidated_after < unconsolidated_before, (
f"expected backlog to shrink but still remain after one round; "
f"before={unconsolidated_before}, after={unconsolidated_after}"
@@ -48,15 +48,13 @@ async def test_round_limit_caps_processed_memories(memory: MemoryEngine, request
)
# Verify we have unconsolidated memories
async with memory._pool.acquire() as conn:
unconsolidated = await conn.fetchval(
"""
SELECT COUNT(*) FROM memory_units
WHERE bank_id = $1 AND consolidated_at IS NULL
AND consolidation_failed_at IS NULL AND fact_type IN ('experience', 'world')
""",
bank_id,
# consolidation_state='pending' is the read API's name for exactly this predicate:
# not consolidated, not failed, and a source fact type.
unconsolidated = (
await memory.list_memory_units(
bank_id=bank_id, consolidation_state="pending", limit=1, request_context=request_context
)
)["total"]
assert unconsolidated >= 6, f"Expected at least 6 unconsolidated memories, got {unconsolidated}"
# Run consolidation with a round limit of 3
@@ -91,15 +89,11 @@ async def test_round_limit_caps_processed_memories(memory: MemoryEngine, request
assert result.get("mental_models_refreshed", 0) == 0
# Verify some memories are still unconsolidated
async with memory._pool.acquire() as conn:
still_unconsolidated = await conn.fetchval(
"""
SELECT COUNT(*) FROM memory_units
WHERE bank_id = $1 AND consolidated_at IS NULL
AND consolidation_failed_at IS NULL AND fact_type IN ('experience', 'world')
""",
bank_id,
still_unconsolidated = (
await memory.list_memory_units(
bank_id=bank_id, consolidation_state="pending", limit=1, request_context=request_context
)
)["total"]
assert still_unconsolidated > 0, "Some memories should still be unconsolidated after hitting round limit"
await memory.delete_bank(bank_id, request_context=request_context)
@@ -140,15 +134,11 @@ async def test_unlimited_round_processes_all(memory: MemoryEngine, request_conte
mock_requeue.assert_not_called()
# All memories should be consolidated
async with memory._pool.acquire() as conn:
still_unconsolidated = await conn.fetchval(
"""
SELECT COUNT(*) FROM memory_units
WHERE bank_id = $1 AND consolidated_at IS NULL
AND consolidation_failed_at IS NULL AND fact_type IN ('experience', 'world')
""",
bank_id,
still_unconsolidated = (
await memory.list_memory_units(
bank_id=bank_id, consolidation_state="pending", limit=1, request_context=request_context
)
)["total"]
assert still_unconsolidated == 0
await memory.delete_bank(bank_id, request_context=request_context)
@@ -122,14 +122,12 @@ def _mock_llm_one_obs_per_fact():
return wrapper, mock_llm
async def _fetch_observation_tag_sets(memory: MemoryEngine, bank_id: str) -> list[frozenset[str]]:
async def _fetch_observation_tag_sets(memory: MemoryEngine, bank_id: str, request_context) -> list[frozenset[str]]:
"""Return the tag set (as a frozenset) of every observation in the bank."""
async with memory._pool.acquire() as conn:
rows = await conn.fetch(
"SELECT tags FROM memory_units WHERE bank_id = $1 AND fact_type = 'observation'",
bank_id,
)
return [frozenset(r["tags"] or []) for r in rows]
items = (
await memory.list_memory_units(bank_id, fact_type="observation", limit=1000, request_context=request_context)
)["items"]
return [frozenset(i["tags"] or []) for i in items]
# ---------------------------------------------------------------------------
@@ -139,6 +137,7 @@ async def _fetch_observation_tag_sets(memory: MemoryEngine, bank_id: str) -> lis
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_combined_mode_parallel_writes_to_memory_tag_set(memory: MemoryEngine, request_context):
"""combined (default) → each memory yields exactly one observation tagged
with the memory's full tag set. With three disjoint tag sets, dispatch
@@ -166,7 +165,7 @@ async def test_combined_mode_parallel_writes_to_memory_tag_set(memory: MemoryEng
memory._consolidation_llm_config = original_llm
assert result["status"] == "completed"
tag_sets = _ag_sorted(await _fetch_observation_tag_sets(memory, bank_id))
tag_sets = _ag_sorted(await _fetch_observation_tag_sets(memory, bank_id, request_context))
assert tag_sets == _ag_sorted(
[
frozenset({"user:alice"}),
@@ -179,6 +178,7 @@ async def test_combined_mode_parallel_writes_to_memory_tag_set(memory: MemoryEng
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_shared_mode_parallel_writes_only_untagged_scope(memory: MemoryEngine, request_context):
"""shared → every memory writes to the single untagged scope, ignoring its
own tags. Three memories with disjoint tags therefore all consolidate into
@@ -207,7 +207,7 @@ async def test_shared_mode_parallel_writes_only_untagged_scope(memory: MemoryEng
memory._consolidation_llm_config = original_llm
assert result["status"] == "completed"
tag_sets = await _fetch_observation_tag_sets(memory, bank_id)
tag_sets = await _fetch_observation_tag_sets(memory, bank_id, request_context)
# Every observation lands at the untagged scope — none carries a session tag.
assert tag_sets and all(t == frozenset() for t in tag_sets), tag_sets
finally:
@@ -215,6 +215,7 @@ async def test_shared_mode_parallel_writes_only_untagged_scope(memory: MemoryEng
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_per_tag_mode_parallel_writes_one_observation_per_tag(memory: MemoryEngine, request_context):
"""per_tag with tags [a, b] → two observations, tagged [a] and [b] respectively.
@@ -246,7 +247,7 @@ async def test_per_tag_mode_parallel_writes_one_observation_per_tag(memory: Memo
assert result["status"] == "completed"
tag_sets = await _fetch_observation_tag_sets(memory, bank_id)
tag_sets = await _fetch_observation_tag_sets(memory, bank_id, request_context)
# M1 writes to [alice]; M2 writes to [alice] and [session]. The mock LLM
# creates one observation per fact per pass, so we expect:
# - one [alice] obs from M1
@@ -263,6 +264,7 @@ async def test_per_tag_mode_parallel_writes_one_observation_per_tag(memory: Memo
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_all_combinations_mode_parallel_writes_every_subset(memory: MemoryEngine, request_context):
"""all_combinations with tags [a, b] → three observations at [a], [b], [a, b]."""
bank_id = f"test-allcombo-{uuid.uuid4().hex[:8]}"
@@ -286,7 +288,7 @@ async def test_all_combinations_mode_parallel_writes_every_subset(memory: Memory
memory._consolidation_llm_config = original_llm
assert result["status"] == "completed"
tag_sets = set(await _fetch_observation_tag_sets(memory, bank_id))
tag_sets = set(await _fetch_observation_tag_sets(memory, bank_id, request_context))
assert tag_sets == {
frozenset({"alice"}),
frozenset({"session"}),
@@ -297,6 +299,7 @@ async def test_all_combinations_mode_parallel_writes_every_subset(memory: Memory
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_explicit_scope_list_parallel_writes_declared_scopes(memory: MemoryEngine, request_context):
"""Explicit list[list[str]] → observations land at exactly those scopes,
regardless of the memory's own tag set."""
@@ -327,7 +330,7 @@ async def test_explicit_scope_list_parallel_writes_declared_scopes(memory: Memor
memory._consolidation_llm_config = original_llm
assert result["status"] == "completed"
tag_sets = set(await _fetch_observation_tag_sets(memory, bank_id))
tag_sets = set(await _fetch_observation_tag_sets(memory, bank_id, request_context))
assert tag_sets == {frozenset({"scope_a"}), frozenset({"scope_b", "scope_c"})}
# And NOT the memory's own tag.
assert frozenset({"tag_ignored"}) not in tag_sets
@@ -342,6 +345,7 @@ async def test_explicit_scope_list_parallel_writes_declared_scopes(memory: Memor
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_overlapping_scopes_serialise_under_parallelism(memory: MemoryEngine, request_context):
"""Two groups whose write-scope sets intersect on scope S must not have
overlapping in-flight LLM-recall windows for S.
@@ -424,6 +428,7 @@ async def test_overlapping_scopes_serialise_under_parallelism(memory: MemoryEngi
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_per_batch_log_line_attributes_only_own_work(memory: MemoryEngine, request_context, caplog):
"""Per-batch log timings / llm_calls / tokens / processed must reflect only
that batch's own work — not totals leaking in from other in-flight batches
@@ -502,6 +507,7 @@ async def test_per_batch_log_line_attributes_only_own_work(memory: MemoryEngine,
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_disjoint_scopes_run_concurrently(memory: MemoryEngine, request_context):
"""When write-scope sets are pairwise disjoint, the dispatcher must let
groups run in parallel — we should observe simultaneous in-flight recalls
@@ -58,16 +58,16 @@ def enable_observations():
config.enable_observations = original
async def _count_unconsolidated(memory, bank_id: str) -> int:
async with memory._pool.acquire() as conn:
return await conn.fetchval(
"""
SELECT COUNT(*) FROM memory_units
WHERE bank_id = $1 AND consolidated_at IS NULL
AND consolidation_failed_at IS NULL AND fact_type IN ('experience', 'world')
""",
bank_id,
)
async def _count_unconsolidated(memory, bank_id: str, request_context) -> int:
# consolidation_state='pending' is the read API's name for exactly this predicate:
# not consolidated, not failed, and a source fact type.
page = await memory.list_memory_units(
bank_id=bank_id,
consolidation_state="pending",
limit=1,
request_context=request_context,
)
return page["total"]
@pytest.mark.asyncio
@@ -95,7 +95,7 @@ async def test_requeue_failure_propagates_to_worker_retry(memory: MemoryEngine,
request_context=request_context,
)
unconsolidated_before = await _count_unconsolidated(memory, bank_id)
unconsolidated_before = await _count_unconsolidated(memory, bank_id, request_context)
assert unconsolidated_before >= backlog_size
op_id = uuid.uuid4()
@@ -147,7 +147,7 @@ async def test_requeue_failure_propagates_to_worker_retry(memory: MemoryEngine,
# consolidated stay consolidated; the consolidator's per-batch
# `UPDATE ... SET consolidated_at = NOW()` commits in its own
# transaction (consolidator.py:524-534), not inside the op-level state.
unconsolidated_after = await _count_unconsolidated(memory, bank_id)
unconsolidated_after = await _count_unconsolidated(memory, bank_id, request_context)
assert unconsolidated_after < unconsolidated_before, (
f"completed-round work must be durable across the re-queue failure; "
f"before={unconsolidated_before}, after={unconsolidated_after}"
@@ -138,6 +138,7 @@ async def _observations(memory: MemoryEngine, bank_id: str) -> list:
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_dedup_create_fold_widens_bounds_of_the_twin(memory: MemoryEngine, request_context, observations_enabled):
"""#3477: a CREATE folded into a near-twin must hand over its source facts' dates.
@@ -184,6 +185,7 @@ async def test_dedup_create_fold_widens_bounds_of_the_twin(memory: MemoryEngine,
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_update_widens_bounds_from_its_source_facts(memory: MemoryEngine, request_context, observations_enabled):
"""The ordinary UPDATE path inherits every temporal field from its sources, event_date included."""
bank_id = f"test-temporal-update-{uuid.uuid4().hex[:8]}"
@@ -235,6 +237,7 @@ async def test_update_widens_bounds_from_its_source_facts(memory: MemoryEngine,
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_dedup_update_fold_unions_the_bounds_of_both_rows(
memory: MemoryEngine, request_context, observations_enabled
):
@@ -32,6 +32,7 @@ async def _seed(conn, bank_id: str, *, tags: list[str], consolidated: bool = Fal
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_count_dedupes_across_overlapping_scopes(memory: MemoryEngine, request_context: RequestContext):
bank_id = f"test-count-{uuid.uuid4().hex[:8]}"
await memory.get_bank_profile(bank_id=bank_id, request_context=request_context)
@@ -48,6 +48,7 @@ async def _insert_fact(memory: MemoryEngine, bank_id: str, text: str) -> str:
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_patch_invalidate_and_revert_over_http(api_client, memory):
bank_id = f"curation-http-{uuid.uuid4().hex[:8]}"
mem_id = await _insert_fact(memory, bank_id, "srv-04 runs PostgreSQL 14.")
@@ -80,6 +81,7 @@ async def test_patch_invalidate_and_revert_over_http(api_client, memory):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_patch_clears_occurred_dates_with_explicit_null(api_client, memory):
bank_id = f"curation-http-clear-dates-{uuid.uuid4().hex[:8]}"
mem_id = await _insert_fact(memory, bank_id, "Release v1.2 happened on Monday.")
@@ -138,6 +140,7 @@ async def test_patch_empty_body_is_rejected(api_client, memory):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_patch_resolve_entities_reaches_the_engine(api_client, memory):
"""resolve_entities must survive the HTTP boundary, and default to True when omitted (#3479)."""
bank_id = f"curation-http-resolve-{uuid.uuid4().hex[:8]}"
@@ -27,6 +27,10 @@ from hindsight_api.engine.task_backend import SyncTaskBackend
from hindsight_api.extensions import TenantContext, TenantExtension
from hindsight_api.migrations import ensure_embedding_dimension, run_migrations
# The whole point of this module is the physical embedding column: it counts
# memory_units rows with a non-NULL embedding of a configured dimension.
pytestmark = pytest.mark.memory_backend_incompatible
# =============================================================================
# Shared Utilities
# =============================================================================
+29 -78
View File
@@ -2,7 +2,6 @@
Tests for delta retain — upsert optimization that only re-processes changed chunks.
"""
import json
import logging
from datetime import datetime, timezone
@@ -237,6 +236,7 @@ async def test_delta_retain_modified_chunk(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_delta_retain_entities_preserved_for_unchanged_chunks(memory, request_context):
"""
Entities linked to unchanged chunks should be preserved after delta retain.
@@ -256,13 +256,8 @@ async def test_delta_retain_entities_preserved_for_unchanged_chunks(memory, requ
assert len(v1_units) > 0
# Check entities exist
pool = await memory._get_pool()
async with pool.acquire() as conn:
v1_entities = await conn.fetch(
"SELECT canonical_name FROM entities WHERE bank_id = $1",
bank_id,
)
v1_entity_names = {e["canonical_name"].lower() for e in v1_entities}
v1_listing = await memory.list_entities(bank_id, request_context=request_context)
v1_entity_names = {e["canonical_name"].lower() for e in v1_listing["items"]}
assert len(v1_entity_names) > 0, "Should have entities after v1 retain"
# Upsert with same content — entities should persist
@@ -274,12 +269,8 @@ async def test_delta_retain_entities_preserved_for_unchanged_chunks(memory, requ
request_context=request_context,
)
async with pool.acquire() as conn:
v2_entities = await conn.fetch(
"SELECT canonical_name FROM entities WHERE bank_id = $1",
bank_id,
)
v2_entity_names = {e["canonical_name"].lower() for e in v2_entities}
v2_listing = await memory.list_entities(bank_id, request_context=request_context)
v2_entity_names = {e["canonical_name"].lower() for e in v2_listing["items"]}
# All v1 entities should still exist
assert v1_entity_names.issubset(v2_entity_names), (
@@ -308,13 +299,8 @@ async def test_delta_retain_new_entities_created_for_new_chunks(memory, request_
request_context=request_context,
)
pool = await memory._get_pool()
async with pool.acquire() as conn:
v1_entities = await conn.fetch(
"SELECT canonical_name FROM entities WHERE bank_id = $1",
bank_id,
)
v1_entity_names = {e["canonical_name"].lower() for e in v1_entities}
v1_listing = await memory.list_entities(bank_id, request_context=request_context)
v1_entity_names = {e["canonical_name"].lower() for e in v1_listing["items"]}
# Append content mentioning new entities
v2_content = v1_content + "\n\nBob joined Facebook. He works with Charlie on the Reality Labs project."
@@ -326,12 +312,8 @@ async def test_delta_retain_new_entities_created_for_new_chunks(memory, request_
request_context=request_context,
)
async with pool.acquire() as conn:
v2_entities = await conn.fetch(
"SELECT canonical_name FROM entities WHERE bank_id = $1",
bank_id,
)
v2_entity_names = {e["canonical_name"].lower() for e in v2_entities}
v2_listing = await memory.list_entities(bank_id, request_context=request_context)
v2_entity_names = {e["canonical_name"].lower() for e in v2_listing["items"]}
# Should have more entities after adding content with new people/orgs
assert len(v2_entity_names) > len(v1_entity_names), (
@@ -343,6 +325,7 @@ async def test_delta_retain_new_entities_created_for_new_chunks(memory, request_
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_delta_retain_links_preserved_for_unchanged_chunks(memory, request_context):
"""
Memory links (temporal, semantic, entity) for unchanged chunks should be preserved.
@@ -444,6 +427,7 @@ async def test_delta_retain_document_metadata_updated(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_delta_retain_metadata_consistent_for_unchanged_units(memory, request_context):
"""Metadata updates should reach facts preserved by metadata-only retain."""
bank_id = f"test_delta_unit_meta_{_ts()}"
@@ -479,25 +463,19 @@ async def test_delta_retain_metadata_consistent_for_unchanged_units(memory, requ
assert doc is not None
assert doc["document_metadata"] == {"source": "crm"}
pool = await memory._get_pool()
async with pool.acquire() as conn:
rows = await conn.fetch(
"SELECT metadata FROM memory_units WHERE bank_id = $1 AND document_id = $2",
bank_id,
document_id,
)
listing = await memory.list_memory_units(bank_id, document_id=document_id, request_context=request_context)
rows = listing["items"]
assert rows
for row in rows:
metadata = row["metadata"]
if isinstance(metadata, str):
metadata = json.loads(metadata)
assert metadata == {"source": "crm"}
# list_memory_units already parses the JSON metadata into a dict.
assert row["metadata"] == {"source": "crm"}
finally:
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_delta_retain_drops_null_metadata_values(memory, request_context):
"""A null-valued metadata key must never reach memory_units (issue #3209).
@@ -519,21 +497,12 @@ async def test_delta_retain_drops_null_metadata_values(memory, request_context):
async def _unit_metadata() -> dict[str, dict]:
"""The document's memory units, keyed by unit id, so a later call can
tell units that survived a delta from ones re-extracted from scratch."""
pool = await memory._get_pool()
async with pool.acquire() as conn:
rows = await conn.fetch(
"SELECT id, metadata FROM memory_units WHERE bank_id = $1 AND document_id = $2",
bank_id,
document_id,
)
listing = await memory.list_memory_units(bank_id, document_id=document_id, request_context=request_context)
rows = listing["items"]
assert rows, "expected memory units for the document"
units = {}
for row in rows:
metadata = row["metadata"]
if isinstance(metadata, str):
metadata = json.loads(metadata)
units[str(row["id"])] = metadata
return units
# list_memory_units already parses the JSON metadata into a dict and
# returns ids as strings.
return {row["id"]: row["metadata"] for row in rows}
async def _retain(content: str, source: str) -> None:
await memory.retain_batch_async(
@@ -598,13 +567,8 @@ async def test_delta_retain_tags_propagated_to_existing_units(memory, request_co
request_context=request_context,
)
pool = await memory._get_pool()
async with pool.acquire() as conn:
v1_tags = await conn.fetch(
"SELECT tags FROM memory_units WHERE bank_id = $1 AND document_id = $2",
bank_id,
document_id,
)
v1_listing = await memory.list_memory_units(bank_id, document_id=document_id, request_context=request_context)
v1_tags = v1_listing["items"]
assert all("team-a" in row["tags"] for row in v1_tags), "v1 units should have team-a tag"
# v2 with same content but different tags
@@ -620,12 +584,8 @@ async def test_delta_retain_tags_propagated_to_existing_units(memory, request_co
request_context=request_context,
)
async with pool.acquire() as conn:
v2_tags = await conn.fetch(
"SELECT tags FROM memory_units WHERE bank_id = $1 AND document_id = $2",
bank_id,
document_id,
)
v2_listing = await memory.list_memory_units(bank_id, document_id=document_id, request_context=request_context)
v2_tags = v2_listing["items"]
for row in v2_tags:
assert "team-b" in row["tags"], f"v2 units should have team-b tag, got {row['tags']}"
assert "important" in row["tags"], f"v2 units should have important tag, got {row['tags']}"
@@ -923,13 +883,8 @@ async def test_delta_retain_with_user_entities(memory, request_context):
request_context=request_context,
)
pool = await memory._get_pool()
async with pool.acquire() as conn:
v1_entities = await conn.fetch(
"SELECT canonical_name FROM entities WHERE bank_id = $1",
bank_id,
)
v1_names = {e["canonical_name"].lower() for e in v1_entities}
v1_listing = await memory.list_entities(bank_id, request_context=request_context)
v1_names = {e["canonical_name"].lower() for e in v1_listing["items"]}
# v2 with additional entity, same content
# Note: same content = delta path (no re-extraction)
@@ -951,12 +906,8 @@ async def test_delta_retain_with_user_entities(memory, request_context):
)
# Should have entities from both v1 and v2
async with pool.acquire() as conn:
v2_entities = await conn.fetch(
"SELECT canonical_name FROM entities WHERE bank_id = $1",
bank_id,
)
v2_names = {e["canonical_name"].lower() for e in v2_entities}
v2_listing = await memory.list_entities(bank_id, request_context=request_context)
v2_names = {e["canonical_name"].lower() for e in v2_listing["items"]}
# v1 entities should be preserved
assert v1_names.issubset(v2_names), f"v1 entities should be preserved: {v1_names} not in {v2_names}"
@@ -114,13 +114,11 @@ async def test_delta_detects_unchanged_after_first_retain(memory, request_contex
)
assert len(v1_units) > 0
pool = await memory._get_pool()
async with pool.acquire() as conn:
v1_count = await conn.fetchval(
"SELECT count(*) FROM memory_units WHERE bank_id = $1 AND document_id = $2",
bank_id,
document_id,
)
async def _unit_count() -> int:
listing = await memory.list_memory_units(bank_id, document_id=document_id, request_context=request_context)
return listing["total"]
v1_count = await _unit_count()
# Second retain — same content, should be detected as unchanged by delta
v2_units = await memory.retain_async(
@@ -135,12 +133,7 @@ async def test_delta_detects_unchanged_after_first_retain(memory, request_contex
assert v2_units == [], f"Delta with unchanged content should return empty, got {len(v2_units)} units"
# Memory unit count should not change
async with pool.acquire() as conn:
v2_count = await conn.fetchval(
"SELECT count(*) FROM memory_units WHERE bank_id = $1 AND document_id = $2",
bank_id,
document_id,
)
v2_count = await _unit_count()
assert v2_count == v1_count, f"Memory unit count changed on same-content upsert: {v1_count} -> {v2_count}"
# Third retain — verify stability
@@ -153,12 +146,7 @@ async def test_delta_detects_unchanged_after_first_retain(memory, request_contex
)
assert v3_units == [], "Third retain should also detect unchanged"
async with pool.acquire() as conn:
v3_count = await conn.fetchval(
"SELECT count(*) FROM memory_units WHERE bank_id = $1 AND document_id = $2",
bank_id,
document_id,
)
v3_count = await _unit_count()
assert v3_count == v1_count, f"Memory unit count changed on third upsert: {v1_count} -> {v3_count}"
finally:
@@ -189,18 +177,19 @@ async def test_stale_request_skipped_when_newer_retain_completed(memory, request
request_context=request_context,
)
pool = await memory._get_pool()
async with pool.acquire() as conn:
after_newer_count = await conn.fetchval(
"SELECT count(*) FROM memory_units WHERE bank_id = $1 AND document_id = $2",
bank_id,
document_id,
)
async def _unit_count() -> int:
listing = await memory.list_memory_units(bank_id, document_id=document_id, request_context=request_context)
return listing["total"]
after_newer_count = await _unit_count()
assert after_newer_count > 0, "Should have facts from newer content"
# Simulate the race condition by pushing the document's updated_at into
# the future. This makes any new retain appear "stale" (its start_time
# is before updated_at), as if another request already completed.
# is before updated_at), as if another request already completed. This
# forces internal store state (a document's updated_at) that the public
# API has no way to set, so it stays a direct write on purpose.
pool = await memory._get_pool()
async with pool.acquire() as conn:
await conn.execute(
"UPDATE documents SET updated_at = NOW() + INTERVAL '10 seconds' WHERE id = $1 AND bank_id = $2",
@@ -223,12 +212,7 @@ async def test_stale_request_skipped_when_newer_retain_completed(memory, request
assert result == [], f"Stale request should return empty, got {result}"
# Memory units should be unchanged (newer content preserved)
async with pool.acquire() as conn:
final_count = await conn.fetchval(
"SELECT count(*) FROM memory_units WHERE bank_id = $1 AND document_id = $2",
bank_id,
document_id,
)
final_count = await _unit_count()
assert final_count == after_newer_count, (
f"Stale request should not change memory units: {after_newer_count} -> {final_count}"
)
@@ -271,6 +255,7 @@ async def memory_no_llm(pg0_db_url, embeddings, cross_encoder, query_analyzer):
@pytest.mark.asyncio
@pytest.mark.flaky(reruns=2, reruns_delay=2)
@pytest.mark.memory_backend_incompatible
async def test_concurrent_upserts_no_duplicates(memory_no_llm, request_context):
"""
Stress test: N concurrent retains of the same document with different content.
@@ -350,12 +335,10 @@ async def test_concurrent_upserts_no_duplicates(memory_no_llm, request_context):
logger.info(f"Winning version: {winning_version} (out of {num_concurrent} concurrent retains)")
# 2. All memory units should belong to the winning version
async with pool.acquire() as conn:
units = await conn.fetch(
"SELECT text, chunk_id, id::text as unit_id FROM memory_units WHERE bank_id = $1 AND document_id = $2",
bank_id,
document_id,
)
listing = await memory_no_llm.list_memory_units(
bank_id, document_id=document_id, limit=1000, request_context=request_context
)
units = listing["items"]
unit_texts = [r["text"] for r in units]
assert len(unit_texts) > 0, "Should have at least 1 memory unit"
@@ -364,9 +347,7 @@ async def test_concurrent_upserts_no_duplicates(memory_no_llm, request_context):
# We check for "Person_N" rather than "VERSION_N" because the text
# splitter may cut mid-text, so later chunks might not start with the prefix.
winning_person = f"Person_{winning_version}"
wrong_version_units = [
(r["text"], r["chunk_id"], r["unit_id"]) for r in units if winning_person not in r["text"]
]
wrong_version_units = [(r["text"], r["chunk_id"], r["id"]) for r in units if winning_person not in r["text"]]
assert not wrong_version_units, (
f"Found {len(wrong_version_units)} memory units NOT from winning version "
f"{winning_version} (expected '{winning_person}' in every unit). "
@@ -115,6 +115,7 @@ async def _export_async(memory, bank_id, request_context, **kwargs):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_import_filters_degenerate_fact_without_shifting_archive_ordinals(memory, request_context):
"""A rejected archive fact must not shift chunks, causal links, or observation sources."""
dst = _unique_bank("transfer_degenerate_alignment")
@@ -393,6 +394,7 @@ async def test_export_bank_contents(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_export_tolerates_legacy_null_and_numeric_fact_metadata(memory, request_context):
"""A bank holding legacy metadata must still be exportable (issue #3209).
@@ -522,6 +524,7 @@ async def _observation_count(memory, bank_id):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_bank_import_preserves_consolidation_lifecycle(memory, request_context):
"""Whole-bank import restores each fact's consolidation lifecycle verbatim, so
previously-consolidated and previously-failed facts are never re-consolidated
@@ -622,6 +625,7 @@ async def test_bank_import_preserves_consolidation_lifecycle(memory, request_con
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_bank_export_import_exact_roundtrip(memory, request_context):
"""A whole-bank archive restores EXACT bank content (config, docs, facts,
observations, entities, links, webhooks, directives, mental models) with facts
@@ -793,6 +797,7 @@ async def test_bank_roundtrip_carries_mental_model_history(memory, request_conte
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_bank_roundtrip_carries_knowledge_pages(memory, request_context):
"""A whole-bank archive restores the Knowledge Pages tree — nested folders +
pages, parent_id / mental_model_id / managed / sort_order preserved — and
@@ -896,6 +901,7 @@ async def test_import_bank_refuses_existing_bank(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_export_import_roundtrip_without_llm(memory, request_context, monkeypatch):
"""Export from one bank and import into another without re-running the LLM."""
src = _unique_bank("transfer_src")
@@ -1019,6 +1025,7 @@ async def _bank_snapshot(memory, bank_id):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_full_roundtrip_integrity(memory, request_context):
"""Full export → import must reproduce every persisted artifact (counts + sizes)."""
src = _unique_bank("transfer_integ_src")
@@ -1072,6 +1079,7 @@ async def test_full_roundtrip_integrity(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_transfer_preserves_legacy_causal_links(memory, request_context):
"""Legacy causal edges survive export/import without becoming retain inputs."""
src = _unique_bank("transfer_legacy_causal_src")
@@ -1123,6 +1131,7 @@ async def test_transfer_preserves_legacy_causal_links(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_export_import_observations(memory, request_context):
"""With include_observations, observations transfer and their sources re-link."""
src = _unique_bank("transfer_obs_src")
@@ -1203,6 +1212,7 @@ async def test_export_import_observations(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_import_triggers_consolidation(memory, request_context):
"""Importing (without observations) triggers consolidation in the target bank,
so observations get generated there — same as a normal retain."""
@@ -1366,6 +1376,7 @@ async def test_import_on_conflict_modes(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_http_export_import_endpoints(api_client, memory, request_context):
"""Round trip through the async HTTP export (POST + poll + download) and import endpoints."""
src = _unique_bank("transfer_http_src")
@@ -1547,6 +1558,7 @@ async def test_import_rejects_invalid_on_conflict(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_bank_import_classifies_label_entities(memory, request_context):
"""An imported bank's label entities are stored with entity_kind='label'.
@@ -1624,6 +1636,7 @@ async def test_bank_import_classifies_label_entities(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_async_export_roundtrip(memory, request_context):
"""The async export operation stashes a real archive that re-imports cleanly.
@@ -1669,6 +1682,7 @@ async def test_async_export_include_observations_subset_rejected(memory, request
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_export_attach_batching_preserves_entities_and_causal_links(memory, request_context, monkeypatch):
"""Batched attach queries carry every fact's entities and cross-batch causal edges.
+16 -52
View File
@@ -1916,7 +1916,6 @@ async def test_retain_multivalue_tag_entities_all_stored(memory_real_llm, reques
The original bug: tags are added correctly, but unit_entities only stores
a subset (typically the first entity).
"""
from hindsight_api.engine.memory_engine import fq_table
bank_id = f"test-1558-multivalue-tag-{uuid.uuid4().hex[:8]}"
try:
@@ -1964,32 +1963,14 @@ async def test_retain_multivalue_tag_entities_all_stored(memory_real_llm, reques
assert len(unit_ids) > 0, "Should have extracted at least one fact"
async with memory_real_llm._pool.acquire() as conn:
# Check entities in unit_entities table
entity_rows = await conn.fetch(
f"""
SELECT e.canonical_name
FROM {fq_table("unit_entities")} ue
JOIN {fq_table("entities")} e ON e.id = ue.entity_id
WHERE ue.unit_id = ANY($1::uuid[])
""",
[u for u in unit_ids],
)
entity_names = {r["canonical_name"].lower() for r in entity_rows}
# Check tags on memory_units
tag_rows = await conn.fetch(
f"""
SELECT id, tags
FROM {fq_table("memory_units")}
WHERE id = ANY($1::uuid[])
""",
[u for u in unit_ids],
)
all_tags = set()
for row in tag_rows:
if row["tags"]:
all_tags.update(t.lower() for t in row["tags"])
# Entities and tags both come back on the unit itself, so one read per
# unit covers what the two joins used to.
entity_names: set[str] = set()
all_tags: set[str] = set()
for unit_id in unit_ids:
unit = await memory_real_llm.get_memory_unit(bank_id, str(unit_id), request_context)
entity_names.update(name.lower() for name in unit["entities"])
all_tags.update(tag.lower() for tag in unit["tags"])
# Filter to use:* entities/tags
use_entities = {n for n in entity_names if n.startswith("use:")}
@@ -2025,7 +2006,6 @@ async def test_retain_multivalue_tag_entities_second_retain(memory_real_llm, req
temporal proximity could exceed the 0.6 merge threshold, causing both to
resolve to the same entity ID.
"""
from hindsight_api.engine.memory_engine import fq_table
bank_id = f"test-1558-second-{uuid.uuid4().hex[:8]}"
try:
@@ -2075,30 +2055,14 @@ async def test_retain_multivalue_tag_entities_second_retain(memory_real_llm, req
assert len(unit_ids_2) > 0
async with memory_real_llm._pool.acquire() as conn:
entity_rows = await conn.fetch(
f"""
SELECT e.canonical_name
FROM {fq_table("unit_entities")} ue
JOIN {fq_table("entities")} e ON e.id = ue.entity_id
WHERE ue.unit_id = ANY($1::uuid[])
""",
[u for u in unit_ids_2],
)
entity_names = {r["canonical_name"].lower() for r in entity_rows}
tag_rows = await conn.fetch(
f"""
SELECT id, tags
FROM {fq_table("memory_units")}
WHERE id = ANY($1::uuid[])
""",
[u for u in unit_ids_2],
)
all_tags = set()
for row in tag_rows:
if row["tags"]:
all_tags.update(t.lower() for t in row["tags"])
# Entities and tags both come back on the unit itself, so one read per
# unit covers what the two joins used to.
entity_names: set[str] = set()
all_tags: set[str] = set()
for unit_id in unit_ids_2:
unit = await memory_real_llm.get_memory_unit(bank_id, str(unit_id), request_context)
entity_names.update(name.lower() for name in unit["entities"])
all_tags.update(tag.lower() for tag in unit["tags"])
use_entities = {n for n in entity_names if n.startswith("use:")}
use_tags = {t for t in all_tags if t.startswith("use:")}
@@ -685,6 +685,7 @@ async def test_converter_registry():
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_file_conversion_creates_separate_retain_operation(memory_no_llm_verify, sample_txt_content):
"""Test that file conversion and retain are two separate async operations.
@@ -150,6 +150,7 @@ async def test_graph_q_and_tags_filter_combined(api_client, test_bank_id):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_graph_document_filter_includes_observations_via_source_memories(
memory, api_client, test_bank_id, request_context
):
@@ -297,6 +298,7 @@ async def _seed_scoped_observations(memory, bank_id, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_observation_scopes_enumeration(memory, api_client, test_bank_id, request_context):
"""The scopes endpoint enumerates distinct tag sets (order-normalized) with counts."""
await _seed_scoped_observations(memory, test_bank_id, request_context)
@@ -313,6 +315,7 @@ async def test_observation_scopes_enumeration(memory, api_client, test_bank_id,
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_graph_exact_scope_filter(memory, api_client, test_bank_id, request_context):
"""tags_match=exact filters observations to exactly one scope, not supersets."""
await _seed_scoped_observations(memory, test_bank_id, request_context)
@@ -337,6 +340,7 @@ async def test_graph_exact_scope_filter(memory, api_client, test_bank_id, reques
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_graph_exact_global_scope_filter(memory, api_client, test_bank_id, request_context):
"""tags_match=exact with no tags is the global scope: untagged observations only."""
await _seed_scoped_observations(memory, test_bank_id, request_context)
@@ -27,6 +27,11 @@ from hindsight_api.engine.graph_maintenance import (
)
from hindsight_api.engine.memory_engine import MemoryEngine
# Every test here seeds memory_units / memory_links / entities with raw INSERTs and
# asserts raw link-row counts, as the module docstring says — none of it round-trips
# through the store, so a backend that keeps those rows outside SQL sees an empty graph.
pytestmark = pytest.mark.memory_backend_incompatible
async def _ensure_bank(memory: MemoryEngine, bank_id: str, request_context: RequestContext) -> None:
await memory.get_bank_profile(bank_id=bank_id, request_context=request_context)
@@ -196,6 +196,7 @@ async def test_unordered_concurrent_sweep_and_upsert_deadlocks(memory: MemoryEng
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_graph_maintenance_sweep_retries_on_deadlock(memory: MemoryEngine, request_context: RequestContext):
"""The entity-prune batch must survive a deadlock.
@@ -278,6 +278,7 @@ async def test_retrieve_semantic_bm25_grouped_by_fact_type(memory, request_conte
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_fetch_unit_dates_ignores_noncanonical_uuid_inputs(memory, request_context):
"""The indexed UUID lookup preserves the old text-comparison input behavior."""
from hindsight_api.engine.db.ops_postgresql import PostgreSQLOps
@@ -336,6 +337,11 @@ async def test_recall_reuses_semantic_pool_for_graph_seeds(memory, request_conte
@pytest.mark.asyncio
# Asserts *how* the graph arm seeds — that recall calls link_expansion_retrieval's
# _find_semantic_seeds — rather than what it returns. A store with its own graph
# retrieval never goes through that function, so the assertion is specific to the
# SQL retrieval path.
@pytest.mark.memory_backend_incompatible
async def test_recall_keeps_graph_seed_query_for_stricter_semantic_floor(memory, request_context, monkeypatch):
"""A semantic floor above the graph floor must retain the dedicated seed query."""
from hindsight_api.engine.response_models import MinScores
@@ -552,6 +552,7 @@ async def test_document_deletion_with_slashes_in_id(api_client):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_delete_bank(api_client):
"""Test delete bank endpoint.
@@ -227,6 +227,7 @@ class TestTree:
assert loose["trigger"]["refresh_after_consolidation"] is False
assert loose["trigger"]["mode"] == "delta" # untouched by the patch
@pytest.mark.memory_backend_incompatible
async def test_tree_staleness_follows_the_bank_watermark(self, api_client, memory, kb_bank):
"""The tree answers from one bank-wide watermark, not a scan per page.
@@ -151,6 +151,7 @@ async def test_repeated_large_same_id_replacement_is_idempotent(memory, request_
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_append_after_zero_fact_header_slice_skips_unchanged_history(
memory,
request_context,
@@ -53,6 +53,7 @@ def _ids(result: dict) -> set[str]:
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_created_before_filter(memory: MemoryEngine, request_context: RequestContext):
bank_id = f"test-lmu-created-{uuid.uuid4().hex[:8]}"
await _ensure_bank(memory, bank_id, request_context)
@@ -0,0 +1,105 @@
"""What ``list_memory_units`` / ``list_entities`` put in each item.
Tests that need a unit's write watermark, its lineage, or an entity's kind used to
read the columns straight out of ``memory_units`` / ``entities``. Those are part of
the read model, so they are on the item and asserted here through the engine, on
units written by retain rather than seeded with SQL, so the coverage holds for any
store behind the memories seam.
"""
import uuid
from datetime import datetime
import pytest
from hindsight_api import RequestContext
from hindsight_api.engine.memory_engine import MemoryEngine
async def _retain(memory: MemoryEngine, bank_id: str, content: str, request_context: RequestContext) -> list[str]:
return await memory.retain_async(bank_id=bank_id, content=content, request_context=request_context)
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_fact_type_accepts_a_list(memory: MemoryEngine, request_context: RequestContext):
"""A list of fact types matches any of them — the source-fact selection callers want."""
bank_id = f"test-lmu-facttypes-{uuid.uuid4().hex[:8]}"
await memory.get_bank_profile(bank_id=bank_id, request_context=request_context)
try:
await _retain(memory, bank_id, "Alice deployed the release on Friday.", request_context)
async def listed(fact_type):
page = await memory.list_memory_units(
bank_id, fact_type=fact_type, limit=500, request_context=request_context
)
return {item["id"] for item in page["items"]}, page["total"]
world_ids, world_total = await listed("world")
exp_ids, exp_total = await listed("experience")
both_ids, both_total = await listed(["world", "experience"])
# The list arm is exactly the union of the single-value arms, so the
# assertion holds however the LLM happened to classify the facts.
assert both_ids == world_ids | exp_ids
assert both_total == world_total + exp_total
# An empty list filters nothing, matching the "omitted" case.
_, empty_total = await listed([])
_, unfiltered_total = await listed(None)
assert empty_total == unfiltered_total
finally:
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_items_carry_updated_at_and_lineage(memory: MemoryEngine, request_context: RequestContext):
"""Each item carries its write watermark and (for observations) its sources."""
bank_id = f"test-lmu-readmodel-{uuid.uuid4().hex[:8]}"
await memory.get_bank_profile(bank_id=bank_id, request_context=request_context)
try:
await _retain(memory, bank_id, "Bob moved to Berlin in March.", request_context)
page = await memory.list_memory_units(bank_id, limit=500, request_context=request_context)
assert page["items"], "retain must have produced at least one fact"
source_ids = {item["id"] for item in page["items"] if item["fact_type"] != "observation"}
assert source_ids, "retain must have produced at least one source fact"
for item in page["items"]:
# Parseable rather than merely present: callers do date arithmetic on it.
assert isinstance(datetime.fromisoformat(item["updated_at"]), datetime)
if item["fact_type"] == "observation":
# An observation's lineage points at the facts it was drawn from,
# and those facts are in this same bank.
assert item["source_memory_ids"], "an observation must carry its sources"
assert set(item["source_memory_ids"]) <= source_ids
else:
# A source fact has no lineage; the field is always there, never absent.
assert item["source_memory_ids"] == []
# The list item and the detail view agree on the unit.
first = page["items"][0]
detail = await memory.get_memory_unit(bank_id, first["id"], request_context)
assert detail["text"] == first["text"]
if first["fact_type"] == "observation":
assert detail["source_memory_ids"] == first["source_memory_ids"]
finally:
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.asyncio
async def test_entities_carry_their_kind(memory: MemoryEngine, request_context: RequestContext):
"""list_entities reports how each entity was classified."""
bank_id = f"test-entities-kind-{uuid.uuid4().hex[:8]}"
await memory.get_bank_profile(bank_id=bank_id, request_context=request_context)
try:
await _retain(memory, bank_id, "Carol works with Dave at Acme.", request_context)
page = await memory.list_entities(bank_id, limit=500, request_context=request_context)
assert page["items"], "retain must have produced at least one entity"
for item in page["items"]:
assert "entity_kind" in item, "entity_kind is part of the entity read model"
finally:
await memory.delete_bank(bank_id, request_context=request_context)
@@ -21,6 +21,13 @@ from hindsight_api.engine.retain import embedding_processing
# ---------------------------------------------------------------------------
# Most of this module seeds its fixtures by INSERTing memory_units / memory_links /
# entities directly with the helpers below, then asserts on those rows (including the
# embedding and search_vector columns). Those classes and tests carry
# ``memory_backend_incompatible``; the handful that go through the engine end to end
# — test_not_found_returns_none, test_recall_excludes_invalidated — deliberately do not.
async def _insert_memory(
conn,
memory: MemoryEngine,
@@ -185,12 +192,11 @@ async def _entity_ids_for(conn, unit_id: uuid.UUID) -> list[uuid.UUID]:
return [r["entity_id"] for r in rows]
async def _obs_ids(conn, bank_id: str) -> list[str]:
rows = await conn.fetch(
"SELECT id FROM memory_units WHERE bank_id = $1 AND fact_type = 'observation'",
bank_id,
async def _obs_ids(memory: MemoryEngine, bank_id: str, request_context: RequestContext) -> list[str]:
listing = await memory.list_memory_units(
bank_id, fact_type="observation", limit=1000, request_context=request_context
)
return [str(r["id"]) for r in rows]
return [item["id"] for item in listing["items"]]
async def _consolidated_at(conn, mem_id: uuid.UUID):
@@ -207,6 +213,8 @@ async def _ensure_bank(memory: MemoryEngine, bank_id: str, request_context: Requ
class TestInvalidate:
pytestmark = pytest.mark.memory_backend_incompatible
@pytest.mark.asyncio
async def test_invalidate_moves_to_archive_and_prunes(self, memory: MemoryEngine, request_context: RequestContext):
bank_id = f"test-curation-inv-{uuid.uuid4().hex[:8]}"
@@ -250,7 +258,7 @@ class TestInvalidate:
"archive is cold storage with no index; the schema drops search_vector (#2503)"
)
assert await _link_count(conn, m1) == 0, "links cascade-pruned on move"
assert str(obs_id) not in await _obs_ids(conn, bank_id), "derived observation removed"
assert str(obs_id) not in await _obs_ids(memory, bank_id, request_context), "derived observation removed"
assert await _consolidated_at(conn, m2) is None, "surviving source reset for re-consolidation"
await memory.delete_bank(bank_id, request_context=request_context)
@@ -334,6 +342,8 @@ class TestInvalidate:
class TestEdit:
pytestmark = pytest.mark.memory_backend_incompatible
@pytest.mark.asyncio
async def test_edit_changes_text_and_rederives(self, memory: MemoryEngine, request_context: RequestContext):
bank_id = f"test-curation-edit-{uuid.uuid4().hex[:8]}"
@@ -385,7 +395,7 @@ class TestEdit:
assert row["consolidated_at"] is None, "edited memory re-consolidates"
assert "'assist'" not in row["search_vector"], "old text must not stay in native FTS search_vector"
assert "'user'" in row["search_vector"], "new text must refresh native FTS search_vector"
assert str(obs_id) not in await _obs_ids(conn, bank_id), "stale observation re-derived"
assert str(obs_id) not in await _obs_ids(memory, bank_id, request_context), "stale observation re-derived"
queued_ids = await conn.fetch("SELECT unit_id FROM graph_maintenance_queue WHERE bank_id = $1", bank_id)
assert {row["unit_id"] for row in queued_ids} == {m1, m2}, "edited memory and incoming victim both queued"
@@ -616,6 +626,8 @@ class TestCurationRelinking:
returns, the links are already rebuilt.
"""
pytestmark = pytest.mark.memory_backend_incompatible
@pytest.mark.asyncio
async def test_edit_with_only_outgoing_links_queues_itself(
self, memory: MemoryEngine, request_context: RequestContext
@@ -709,6 +721,7 @@ class TestCurationRelinking:
class TestGuardsAndListing:
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_cannot_curate_observation(self, memory: MemoryEngine, request_context: RequestContext):
bank_id = f"test-curation-obs-{uuid.uuid4().hex[:8]}"
await _ensure_bank(memory, bank_id, request_context)
@@ -734,6 +747,7 @@ class TestGuardsAndListing:
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_list_filters_by_state(self, memory: MemoryEngine, request_context: RequestContext):
bank_id = f"test-curation-list-{uuid.uuid4().hex[:8]}"
await _ensure_bank(memory, bank_id, request_context)
@@ -769,6 +783,7 @@ class TestGuardsAndListing:
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_list_and_get_memory_units_include_metadata(
self, memory: MemoryEngine, request_context: RequestContext
):
@@ -810,6 +825,7 @@ class TestGuardsAndListing:
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_list_filters_by_document(self, memory: MemoryEngine, request_context: RequestContext):
bank_id = f"test-curation-doc-{uuid.uuid4().hex[:8]}"
await _ensure_bank(memory, bank_id, request_context)
@@ -849,6 +865,7 @@ class TestGuardsAndListing:
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_list_filters_by_entity(self, memory: MemoryEngine, request_context: RequestContext):
bank_id = f"test-curation-entity-{uuid.uuid4().hex[:8]}"
await _ensure_bank(memory, bank_id, request_context)
@@ -952,6 +969,8 @@ class TestCausalLinkPreservation:
invalidate/revert round-trip must carry them through the archive.
"""
pytestmark = pytest.mark.memory_backend_incompatible
@pytest.mark.asyncio
async def test_edit_preserves_causal_links_and_drops_derived(
self, memory: MemoryEngine, request_context: RequestContext
@@ -569,7 +569,7 @@ async def test_retain_allows_clean_content(api_client) -> None:
@pytest.mark.asyncio
async def test_retain_stores_redacted_text(api_client, memory) -> None:
async def test_retain_stores_redacted_text(api_client, memory, request_context) -> None:
await api_client.put("/v1/default/banks/md-retain-2", json={})
await _set_policy(api_client, "md-retain-2", _REDACT_POLICY)
secret = "ghp_" + "A" * 36
@@ -578,8 +578,8 @@ async def test_retain_stores_redacted_text(api_client, memory) -> None:
json={"items": [{"content": f"my token is {secret}"}]},
)
assert r.status_code == 200, r.text
async with memory._pool.acquire() as conn:
texts = [row["text"] for row in await conn.fetch("SELECT text FROM memory_units WHERE bank_id = 'md-retain-2'")]
listing = await memory.list_memory_units("md-retain-2", limit=1000, request_context=request_context)
texts = [item["text"] for item in listing["items"]]
assert all(secret not in t for t in texts), texts
@@ -37,6 +37,10 @@ from hindsight_api.engine.db.ops_postgresql import PostgreSQLOps
from hindsight_api.engine.db.postgresql import PostgresConnection
from hindsight_api.engine.retain.link_utils import _bulk_insert_links
# Asserts a raw memory_links row count around a concurrent delete; the graph read
# path dedupes bidirectional edges, so the count is not reproducible through it.
pytestmark = pytest.mark.memory_backend_incompatible
async def _insert_unit(conn: asyncpg.Connection, bank_id: str) -> str:
"""Insert one committed memory_unit (autocommit) and return its id as text."""
@@ -84,6 +84,7 @@ async def _bank(memory: MemoryEngine, slug: str, request_context: RequestContext
class TestWritesThatMustStamp:
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_document_tag_propagation_stamps_updated_at(
self, memory: MemoryEngine, request_context: RequestContext
):
@@ -109,6 +110,7 @@ class TestWritesThatMustStamp:
assert await _updated_at(conn, mem_id) > _BASELINE
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_embedding_write_stamps_updated_at(self, memory: MemoryEngine, request_context: RequestContext):
"""The stored vector is part of the memory, so the store method stamps on its own.
@@ -214,6 +216,7 @@ class TestConsolidationBookkeepingIsExempt:
assert await _updated_at(conn, mem_id) == _BASELINE
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_requeue_after_observation_cleanup_leaves_updated_at_alone(
self, memory: MemoryEngine, request_context: RequestContext
):
@@ -97,6 +97,7 @@ async def test_tagged_strict_model_skipped_when_only_untagged_consolidated(
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_tagged_non_strict_model_refreshed_when_only_untagged_consolidated(
memory: MemoryEngine, request_context, monkeypatch
):
@@ -116,6 +117,7 @@ async def test_tagged_non_strict_model_refreshed_when_only_untagged_consolidated
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_tag_groups_model_refreshed_when_only_untagged_consolidated(
memory: MemoryEngine, request_context, monkeypatch
):
@@ -157,6 +159,7 @@ async def test_non_strict_model_skipped_when_nothing_changed(memory: MemoryEngin
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_tagged_model_refreshed_when_its_tag_was_consolidated(memory: MemoryEngine, request_context, monkeypatch):
"""The overlap path is unchanged: a strict tagged model is refreshed when a memory
carrying its tag was consolidated."""
@@ -174,6 +177,7 @@ async def test_tagged_model_refreshed_when_its_tag_was_consolidated(memory: Memo
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_non_strict_model_refreshed_on_mixed_run_with_foreign_tags(
memory: MemoryEngine, request_context, monkeypatch
):
@@ -275,6 +275,7 @@ class TestDeltaRefreshPlumbing:
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.memory_backend_incompatible
async def test_delta_no_new_facts_advances_watermark_to_newest_processed(
self,
memory: MemoryEngine,
@@ -403,6 +404,7 @@ class TestDeltaRefreshPlumbing:
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.memory_backend_incompatible
async def test_delta_refresh_watermark_survives_straddling_commit(
self,
memory: MemoryEngine,
@@ -193,6 +193,7 @@ async def test_routine_returns_cron_models_excludes_plain_and_in_flight(memory:
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_due_and_stale_model_is_refreshed(memory: MemoryEngine, request_context, monkeypatch):
"""A model whose cron is due and that has new memories in scope is refreshed."""
bank = await _make_bank(memory, request_context)
@@ -1173,6 +1173,7 @@ class TestMentalModelStaleness:
assert got["is_stale"] is False
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.memory_backend_incompatible
async def test_untagged_mm_stale_on_any_new_memory(self, memory: MemoryEngine, request_context):
bank_id = f"test-mm-stale-untagged-{uuid.uuid4().hex[:8]}"
await memory.get_bank_profile(bank_id, request_context=request_context)
@@ -1201,6 +1202,7 @@ class TestMentalModelStaleness:
assert got["is_stale"] is False
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.memory_backend_incompatible
async def test_tagged_mm_defaults_to_all_strict(self, memory: MemoryEngine, request_context):
bank_id = f"test-mm-stale-overlap-{uuid.uuid4().hex[:8]}"
await memory.get_bank_profile(bank_id, request_context=request_context)
@@ -1221,6 +1223,7 @@ class TestMentalModelStaleness:
assert got["is_stale"] is True
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.memory_backend_incompatible
async def test_tags_match_any_keeps_overlap_behavior(self, memory: MemoryEngine, request_context):
bank_id = f"test-mm-stale-any-{uuid.uuid4().hex[:8]}"
await memory.get_bank_profile(bank_id, request_context=request_context)
@@ -1238,6 +1241,7 @@ class TestMentalModelStaleness:
assert got["is_stale"] is True
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.memory_backend_incompatible
async def test_tag_groups_define_stale_scope(self, memory: MemoryEngine, request_context):
bank_id = f"test-mm-stale-groups-{uuid.uuid4().hex[:8]}"
await memory.get_bank_profile(bank_id, request_context=request_context)
@@ -1262,6 +1266,7 @@ class TestMentalModelStaleness:
assert got["is_stale"] is True
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.memory_backend_incompatible
async def test_flat_tags_and_fact_types_share_stale_scope(self, memory: MemoryEngine, request_context):
bank_id = f"test-mm-stale-flat-fact-{uuid.uuid4().hex[:8]}"
await memory.get_bank_profile(bank_id, request_context=request_context)
@@ -1283,6 +1288,7 @@ class TestMentalModelStaleness:
assert got["is_stale"] is True
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.memory_backend_incompatible
async def test_tag_groups_and_fact_types_share_stale_scope(self, memory: MemoryEngine, request_context):
bank_id = f"test-mm-stale-group-fact-{uuid.uuid4().hex[:8]}"
await memory.get_bank_profile(bank_id, request_context=request_context)
@@ -1328,6 +1334,7 @@ class TestMentalModelStaleness:
assert got["is_stale"] is True
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.memory_backend_incompatible
async def test_tags_match_all_strict_requires_all_tags(self, memory: MemoryEngine, request_context):
"""tags_match='all_strict' → memory must contain ALL MM tags (and be tagged)."""
bank_id = f"test-mm-stale-all-{uuid.uuid4().hex[:8]}"
@@ -1353,6 +1360,7 @@ class TestMentalModelStaleness:
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.memory_backend_incompatible
async def test_tags_match_any_strict_excludes_untagged(self, memory: MemoryEngine, request_context):
"""tags_match='any_strict' → untagged memory does NOT keep MM in scope."""
bank_id = f"test-mm-stale-anystrict-{uuid.uuid4().hex[:8]}"
@@ -1376,6 +1384,7 @@ class TestMentalModelStaleness:
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.memory_backend_incompatible
async def test_fact_type_filter_narrows_scope(self, memory: MemoryEngine, request_context):
bank_id = f"test-mm-stale-fact-{uuid.uuid4().hex[:8]}"
await memory.get_bank_profile(bank_id, request_context=request_context)
@@ -1399,6 +1408,7 @@ class TestMentalModelStaleness:
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.memory_backend_incompatible
async def test_tool_search_mental_models_returns_is_stale_per_mm(self, memory: MemoryEngine, request_context):
"""Regression: tool_search_mental_models must compute is_stale per-MM via scope,
not via a bank-wide pending_consolidation short-circuit."""
@@ -1435,6 +1445,7 @@ class TestMentalModelStaleness:
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.memory_backend_incompatible
async def test_tool_search_mental_models_skips_the_scan_below_the_watermark(
self, memory: MemoryEngine, request_context
):
@@ -1621,6 +1632,7 @@ class TestMentalModelRefreshTimestamps:
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.memory_backend_incompatible
async def test_staleness_keys_off_memories_seen_not_refresh_time(self, memory: MemoryEngine, request_context):
"""The inverse regression: making ``last_refreshed_at`` a wall clock must not let
a recent refresh mask a memory the document has never seen."""
@@ -2585,6 +2597,7 @@ class TestMentalModelRefreshFactTypeFilter:
memory._reflect_llm_config = wrapper
return mock_llm
@pytest.mark.memory_backend_incompatible
async def test_refresh_with_fact_types_experience_grounds_on_experience_facts(
self, memory: MemoryEngine, request_context
):
@@ -22,6 +22,10 @@ from alembic import command
from alembic.config import Config
from sqlalchemy import create_engine, text
# Drives alembic and asserts on the search_vector column itself — an internal FTS
# index column, not part of the public read model.
pytestmark = pytest.mark.memory_backend_incompatible
_SCRIPT_LOCATION = str(Path(__file__).parent.parent / "hindsight_api" / "alembic")
# Revision immediately before the backfill migration.
+11 -31
View File
@@ -167,17 +167,15 @@ async def test_observation_fact_type_in_database(memory, request_context, disabl
await memory.wait_for_background_tasks()
# Check that NO observations exist in memory_units
pool = await memory._get_pool()
async with pool.acquire() as conn:
observations = await conn.fetch(
"""
SELECT id, text, fact_type, context
FROM memory_units
WHERE bank_id = $1 AND fact_type = 'observation'
""",
# Check that NO observations exist as memory units
observations = (
await memory.list_memory_units(
bank_id,
fact_type="observation",
limit=1000,
request_context=request_context,
)
)["items"]
print(f"\n=== Observation Records in memory_units ===")
print(f"Found {len(observations)} observation records (should be 0)")
@@ -194,6 +192,7 @@ async def test_observation_fact_type_in_database(memory, request_context, disabl
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_entity_mention_counts(memory, request_context):
"""
Test that entity mention counts are tracked correctly.
@@ -235,17 +234,7 @@ async def test_entity_mention_counts(memory, request_context):
await memory.wait_for_background_tasks()
# Check entity mention counts
pool = await memory._get_pool()
async with pool.acquire() as conn:
entities = await conn.fetch(
"""
SELECT e.id, e.canonical_name, e.mention_count
FROM entities e
WHERE e.bank_id = $1
ORDER BY e.mention_count DESC
""",
bank_id,
)
entities = (await memory.list_entities(bank_id, limit=1000, request_context=request_context))["items"]
print(f"\n=== Entity Mention Counts Test ===")
print(f"Total entities: {len(entities)}")
@@ -283,6 +272,7 @@ async def test_entity_mention_counts(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.timeout(1200)
@pytest.mark.memory_backend_incompatible
async def test_entity_mention_ranking(memory, request_context):
"""
Test that entity mention counts correctly rank entities.
@@ -322,17 +312,7 @@ async def test_entity_mention_ranking(memory, request_context):
# Phase 3: Verify entities are ranked by mention count
print("\n=== Phase 3: Check entity ranking ===")
pool = await memory._get_pool()
async with pool.acquire() as conn:
all_entities = await conn.fetch(
"""
SELECT canonical_name, mention_count
FROM entities
WHERE bank_id = $1
ORDER BY mention_count DESC
""",
bank_id,
)
all_entities = (await memory.list_entities(bank_id, limit=1000, request_context=request_context))["items"]
print(f"\nAll entities by mention count:")
for e in all_entities:
@@ -207,6 +207,7 @@ class TestMarkOperationGracefulOnMissingRow:
class TestConsolidationCheckpoint:
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_consolidation_stops_early_when_op_cancelled(self, memory: MemoryEngine, request_context):
"""Consolidation returns 'cancelled' status after the first batch if _check_op_alive is False."""
from hindsight_api.config import _get_raw_config
@@ -161,6 +161,7 @@ async def test_progress_absent_returns_null(api_client, memory: MemoryEngine):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_consolidation_records_advancing_progress(memory: MemoryEngine, request_context, monkeypatch):
"""A real consolidation run emits scanning → processing_batch → refreshing_mental_models,
with processed advancing and a durable snapshot left on the operation row."""
@@ -37,6 +37,9 @@ except ImportError:
pytestmark = [
pytest.mark.skipif(not ORACLEDB_AVAILABLE, reason="oracledb not installed"),
pytest.mark.skipif(not os.getenv("ORACLE_TEST_DSN"), reason="ORACLE_TEST_DSN not set"),
# Talks to the backend's own tables in SQL, down to VECTOR_DISTANCE over the
# embedding column.
pytest.mark.memory_backend_incompatible,
]
@@ -94,6 +94,7 @@ async def _recall(engine, bank_id, *, query="animals and nature", **kwargs):
class TestRecallScores:
@pytest.mark.memory_backend_incompatible
async def test_every_result_has_scores(self, seeded_memory):
engine, bank_id = seeded_memory
result = await _recall(engine, bank_id)
@@ -119,6 +120,7 @@ class TestPostQueryFilters:
explicit = await _recall(engine, bank_id, min_scores=None)
assert _ids(baseline) == _ids(explicit)
@pytest.mark.memory_backend_incompatible
async def test_final_floor_filters_and_is_a_subset(self, seeded_memory):
engine, bank_id = seeded_memory
baseline = await _recall(engine, bank_id)
@@ -132,6 +134,7 @@ class TestPostQueryFilters:
for r in filtered.results:
assert r.scores.final >= threshold
@pytest.mark.memory_backend_incompatible
async def test_final_floor_above_all_returns_empty(self, seeded_memory):
engine, bank_id = seeded_memory
baseline = await _recall(engine, bank_id)
@@ -153,6 +156,7 @@ class TestPostQueryFilters:
class TestRetrievalLevelFilters:
@pytest.mark.memory_backend_incompatible
async def test_semantic_floor_prunes_in_retrieval(self, seeded_memory):
"""min_scores.semantic is a SQL-arm cutoff: every returned result has a
semantic score >= the floor, and a high floor returns nothing."""
@@ -154,6 +154,7 @@ async def seeded(memory_no_llm_verify: MemoryEngine):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_recall_includes_inherited_entities_for_observations(seeded):
"""Observation-only recall must surface entities inherited from source memories."""
engine, bank_id = seeded
@@ -187,6 +188,7 @@ async def test_recall_includes_inherited_entities_for_observations(seeded):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_get_memory_unit_inherits_observation_entities(seeded):
"""get_memory_unit shares the recall helper, so observation inheritance
must keep working through the per-memory endpoint as well.
@@ -96,6 +96,7 @@ async def seeded_combo(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_recall_all_enrichments_together_on_default_store(memory, request_context, seeded_combo):
"""All three enrichment flags at once on PostgresMemories, with prefer_observations."""
bank_id = seeded_combo["bank_id"]
@@ -105,6 +105,7 @@ async def seeded_obs_memory(memory_no_llm_verify: MemoryEngine):
class TestPreferObservations:
@pytest.mark.memory_backend_incompatible
async def test_disabled_returns_sources_and_observation(self, seeded_obs_memory):
"""Without the flag, the source facts AND the observation are all returned."""
engine, bank_id, ids = seeded_obs_memory
@@ -121,6 +122,7 @@ class TestPreferObservations:
assert ids["src2"] in found
assert ids["obs"] in found
@pytest.mark.memory_backend_incompatible
async def test_enabled_drops_source_facts_keeps_observation(self, seeded_obs_memory):
"""With the flag, the observation supersedes the facts it was consolidated from."""
engine, bank_id, ids = seeded_obs_memory
@@ -137,6 +139,7 @@ class TestPreferObservations:
assert ids["src1"] not in found, "source fact 1 is superseded by the observation"
assert ids["src2"] not in found, "source fact 2 is superseded by the observation"
@pytest.mark.memory_backend_incompatible
async def test_enabled_keeps_non_source_fact(self, seeded_obs_memory):
"""Dedup is provenance-based: a similar fact NOT in source_memory_ids survives."""
engine, bank_id, ids = seeded_obs_memory
@@ -151,6 +154,7 @@ class TestPreferObservations:
found = _result_ids(result)
assert ids["non_src"] in found, "a non-source fact must not be dropped, even if semantically similar"
@pytest.mark.memory_backend_incompatible
async def test_noop_without_observation_type(self, seeded_obs_memory):
"""The flag is a no-op when 'observation' is not among the requested types."""
engine, bank_id, ids = seeded_obs_memory
@@ -132,6 +132,7 @@ def _result_ids(result) -> set[str]:
class TestRecallTimeRange:
"""Verify created_after / created_before filtering at the recall level."""
@pytest.mark.memory_backend_incompatible
async def test_no_filter_returns_all(self, seeded_memory):
engine, bank_id = seeded_memory
result = await engine.recall_async(
@@ -145,6 +146,7 @@ class TestRecallTimeRange:
assert ID_MID in ids
assert ID_NEW in ids
@pytest.mark.memory_backend_incompatible
async def test_created_after_excludes_old(self, seeded_memory):
"""created_after=T1 excludes fact-old (updated_at == T1, not > T1)."""
engine, bank_id = seeded_memory
@@ -160,6 +162,7 @@ class TestRecallTimeRange:
assert ID_MID in ids
assert ID_NEW in ids
@pytest.mark.memory_backend_incompatible
async def test_created_after_excludes_old_and_mid(self, seeded_memory):
"""created_after=T2 returns only fact-new."""
engine, bank_id = seeded_memory
@@ -175,6 +178,7 @@ class TestRecallTimeRange:
assert ID_MID not in ids, "fact-mid (updated_at=T2) must be excluded by created_after=T2"
assert ID_NEW in ids
@pytest.mark.memory_backend_incompatible
async def test_created_before_excludes_new(self, seeded_memory):
"""created_before=T3 excludes fact-new (updated_at == T3, not < T3)."""
engine, bank_id = seeded_memory
@@ -190,6 +194,7 @@ class TestRecallTimeRange:
assert ID_MID in ids
assert ID_NEW not in ids, "fact-new (updated_at=T3) must be excluded by created_before=T3"
@pytest.mark.memory_backend_incompatible
async def test_created_before_excludes_mid_and_new(self, seeded_memory):
"""created_before=T2 returns only fact-old."""
engine, bank_id = seeded_memory
@@ -205,6 +210,7 @@ class TestRecallTimeRange:
assert ID_MID not in ids
assert ID_NEW not in ids
@pytest.mark.memory_backend_incompatible
async def test_range_both_bounds(self, seeded_memory):
"""created_after=T1, created_before=T3 returns only fact-mid."""
engine, bank_id = seeded_memory
@@ -233,6 +239,7 @@ class TestRecallTimeRange:
)
assert len(result.results) == 0, f"Expected no results after T3, got: {_result_ids(result)}"
@pytest.mark.memory_backend_incompatible
async def test_updated_at_catches_consolidation_updates(self, seeded_memory):
"""A fact created at T1 but updated at T3 appears with created_after=T2."""
engine, bank_id = seeded_memory
@@ -23,7 +23,12 @@ import pytest_asyncio
from hindsight_api import MemoryEngine, RequestContext
from hindsight_api.engine.retain import embedding_utils
pytestmark = pytest.mark.xdist_group("recall_time_range_graph")
# Each graph fixture seeds its out-of-window neighbour by INSERTing straight into
# memory_links, so the neighbourhood the assertions turn on only exists in SQL.
pytestmark = [
pytest.mark.xdist_group("recall_time_range_graph"),
pytest.mark.memory_backend_incompatible,
]
T_OLD = datetime(2026, 1, 1, 10, 0, 0, tzinfo=timezone.utc)
T_CUTOFF = datetime(2026, 1, 1, 11, 0, 0, tzinfo=timezone.utc)
@@ -500,6 +500,7 @@ class TestDefaultThresholdIsBackwardsCompatible:
class TestReconcileMechanics:
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_invalid_shape_index_is_rebuilt(
self, memory: MemoryEngine, request_context: RequestContext, low_threshold
):
+36 -55
View File
@@ -1447,6 +1447,7 @@ async def test_chunks_truncation_behavior(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_temporal_links_creation(memory, request_context):
"""
Test that temporal links are created between facts with nearby event dates.
@@ -1526,6 +1527,7 @@ async def test_temporal_links_creation(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_semantic_links_creation(memory, request_context):
"""
Test that semantic links are created between facts with similar content.
@@ -1600,6 +1602,7 @@ async def test_semantic_links_creation(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_entity_links_creation(memory, request_context):
"""
Test that entity edges surface in the /graph response between facts that
@@ -1672,6 +1675,7 @@ async def test_entity_links_creation(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_graph_entity_edges_cover_all_visible_units(memory, request_context):
"""
Regression test: when a hot entity is shared by more than the per-entity
@@ -1712,6 +1716,7 @@ async def test_graph_entity_edges_cover_all_visible_units(memory, request_contex
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_people_name_extraction(memory, request_context):
"""
Test that people names are correctly extracted as entities.
@@ -1739,16 +1744,8 @@ async def test_people_name_extraction(memory, request_context):
)
# Query entities to verify people names were extracted
async with memory._pool.acquire() as conn:
entities = await conn.fetch(
"""
SELECT canonical_name, mention_count
FROM entities
WHERE bank_id = $1
ORDER BY mention_count DESC, canonical_name
""",
bank_id,
)
listing = await memory.list_entities(bank_id, limit=1000, request_context=request_context)
entities = listing["items"]
logger.info(f"Extracted {len(entities)} entities")
for entity in entities:
@@ -1777,6 +1774,7 @@ async def test_people_name_extraction(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_mention_count_accuracy(memory, request_context):
"""
Test that mention_count is accurately tracked across retain calls.
@@ -1805,15 +1803,8 @@ async def test_mention_count_accuracy(memory, request_context):
)
# Check Alice's mention count
async with memory._pool.acquire() as conn:
alice_entity = await conn.fetchrow(
"""
SELECT canonical_name, mention_count
FROM entities
WHERE bank_id = $1 AND LOWER(canonical_name) LIKE '%alice%'
""",
bank_id,
)
listing = await memory.list_entities(bank_id, limit=1000, request_context=request_context)
alice_entity = next((e for e in listing["items"] if "alice" in e["canonical_name"].lower()), None)
assert alice_entity is not None, "Alice entity should exist"
logger.info(f"Alice mention_count after 5 separate retains: {alice_entity['mention_count']}")
@@ -1830,6 +1821,7 @@ async def test_mention_count_accuracy(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_mention_count_batch_retain(memory, request_context):
"""
Test that mention_count is accurate when using batch retain with multiple items.
@@ -1858,15 +1850,8 @@ async def test_mention_count_batch_retain(memory, request_context):
)
# Check Bob's mention count after batch retain
async with memory._pool.acquire() as conn:
bob_entity = await conn.fetchrow(
"""
SELECT canonical_name, mention_count
FROM entities
WHERE bank_id = $1 AND LOWER(canonical_name) LIKE '%bob%'
""",
bank_id,
)
listing = await memory.list_entities(bank_id, limit=1000, request_context=request_context)
bob_entity = next((e for e in listing["items"] if "bob" in e["canonical_name"].lower()), None)
assert bob_entity is not None, "Bob entity should exist after batch retain"
logger.info(f"Bob mention_count after batch retain of 6 items: {bob_entity['mention_count']}")
@@ -1889,15 +1874,8 @@ async def test_mention_count_batch_retain(memory, request_context):
)
# Check updated mention count
async with memory._pool.acquire() as conn:
bob_entity_updated = await conn.fetchrow(
"""
SELECT canonical_name, mention_count
FROM entities
WHERE bank_id = $1 AND LOWER(canonical_name) LIKE '%bob%'
""",
bank_id,
)
listing = await memory.list_entities(bank_id, limit=1000, request_context=request_context)
bob_entity_updated = next((e for e in listing["items"] if "bob" in e["canonical_name"].lower()), None)
logger.info(f"Bob mention_count after second batch: {bob_entity_updated['mention_count']}")
@@ -1919,6 +1897,7 @@ async def test_mention_count_batch_retain(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_causal_links_creation(memory, request_context):
"""
Test that causal links are created between facts with causal relationships.
@@ -1993,6 +1972,7 @@ async def test_causal_links_creation(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_all_link_types_together(memory, request_context):
"""
Integration test: Verify all link types can be created in a single retain operation.
@@ -2066,6 +2046,7 @@ async def test_all_link_types_together(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_semantic_links_within_same_batch(memory, request_context):
"""
Test that semantic links are created between facts retained in the SAME batch.
@@ -2127,6 +2108,7 @@ async def test_semantic_links_within_same_batch(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_semantic_links_phase1_ann_cross_batch(memory, request_context):
"""
Test that Phase 1 ANN search creates semantic links between facts from
@@ -2196,6 +2178,7 @@ async def test_semantic_links_phase1_ann_cross_batch(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_temporal_links_within_same_batch(memory, request_context):
"""
Test that temporal links are created between facts retained in the SAME batch.
@@ -2271,6 +2254,7 @@ async def test_temporal_links_within_same_batch(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_user_provided_entities_resolve_flag_is_scoped_to_them(memory, request_context):
"""resolve_entities=False keeps a caller's own entity names literal (#3479).
@@ -2322,6 +2306,7 @@ async def test_user_provided_entities_resolve_flag_is_scoped_to_them(memory, req
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_user_provided_entities(memory, request_context):
"""
Test that user-provided entities are merged with auto-extracted entities.
@@ -3083,6 +3068,7 @@ async def test_named_strategy_applied_end_to_end(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_semantic_ann_uses_hnsw_index(memory, request_context):
"""
Test that Phase 1 ANN semantic search creates links between similar world
@@ -3155,6 +3141,7 @@ async def test_semantic_ann_uses_hnsw_index(memory, request_context):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_temporal_links_scoped_by_fact_type(memory, request_context):
"""
Test that temporal links only connect facts of the SAME fact_type.
@@ -3409,15 +3396,13 @@ async def test_streaming_chunk_batching_produces_same_facts(memory_mock_llm, req
assert len(streaming_unit_ids) > 0, "Streaming should produce facts"
# Verify facts are in the DB
async with memory._pool.acquire() as conn:
fact_count = await conn.fetchval(
"SELECT COUNT(*) FROM memory_units WHERE bank_id = $1",
bank_id,
)
assert fact_count == len(streaming_unit_ids), (
f"DB has {fact_count} facts, but streaming returned {len(streaming_unit_ids)} unit_ids"
)
fact_count = (await memory.list_memory_units(bank_id, limit=1000, request_context=request_context))["total"]
assert fact_count == len(streaming_unit_ids), (
f"DB has {fact_count} facts, but streaming returned {len(streaming_unit_ids)} unit_ids"
)
# documents/chunks aren't exposed by the read API — check them directly.
async with memory._pool.acquire() as conn:
# Verify the document was tracked
doc = await conn.fetchrow(
"SELECT id FROM documents WHERE bank_id = $1 AND id = $2",
@@ -3474,11 +3459,9 @@ async def test_streaming_chunk_batching_recovery(memory_mock_llm, request_contex
first_unit_ids = result1[0] if result1 else []
assert len(first_unit_ids) > 0, "First retain should produce facts"
async with memory._pool.acquire() as conn:
first_fact_count = await conn.fetchval(
"SELECT COUNT(*) FROM memory_units WHERE bank_id = $1",
bank_id,
)
first_fact_count = (await memory.list_memory_units(bank_id, limit=1000, request_context=request_context))[
"total"
]
logger.info(f"First retain: {first_fact_count} facts")
@@ -3497,11 +3480,9 @@ async def test_streaming_chunk_batching_recovery(memory_mock_llm, request_contex
request_context=request_context,
)
async with memory._pool.acquire() as conn:
second_fact_count = await conn.fetchval(
"SELECT COUNT(*) FROM memory_units WHERE bank_id = $1",
bank_id,
)
second_fact_count = (await memory.list_memory_units(bank_id, limit=1000, request_context=request_context))[
"total"
]
logger.info(f"Second retain: {second_fact_count} facts")
@@ -18,6 +18,7 @@ def _ts():
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_append_mode_concatenates_content(memory, request_context):
"""
When update_mode='append', new content should be appended to the existing
@@ -121,20 +122,13 @@ async def test_append_mode_metadata_consistent_for_unchanged_and_new_units(memor
assert doc is not None
assert doc["document_metadata"] == {"source": "crm"}
pool = await memory._get_pool()
async with pool.acquire() as conn:
rows = await conn.fetch(
"SELECT metadata FROM memory_units WHERE bank_id = $1 AND document_id = $2",
bank_id,
document_id,
)
listing = await memory.list_memory_units(bank_id, document_id=document_id, request_context=request_context)
rows = listing["items"]
assert rows
for row in rows:
metadata = row["metadata"]
if isinstance(metadata, str):
metadata = json.loads(metadata)
assert metadata == {"source": "crm"}
# list_memory_units already parses the JSON metadata into a dict.
assert row["metadata"] == {"source": "crm"}
finally:
await memory.delete_bank(bank_id, request_context=request_context)
@@ -627,6 +627,7 @@ async def _append(memory, request_context, bank_id: str, document_id: str, body:
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_concurrent_appends_keep_every_turn(memory_stub_emb, request_context):
"""Regression: parallel appends used to drop all but one turn.
@@ -653,13 +654,10 @@ async def test_concurrent_appends_keep_every_turn(memory_stub_emb, request_conte
# The turns must also survive as memories, not just as document text: the
# losing writers used to cascade-delete the winner's units.
async with memory_stub_emb._pool.acquire() as conn:
rows = await conn.fetch(
"SELECT text FROM memory_units WHERE bank_id = $1 AND document_id = $2",
bank_id,
document_id,
)
stored = " ".join(r["text"] for r in rows)
listing = await memory_stub_emb.list_memory_units(
bank_id, document_id=document_id, limit=1000, request_context=request_context
)
stored = " ".join(item["text"] for item in listing["items"])
assert "TURN_ONE" in stored and "TURN_FOUR" in stored
@@ -130,6 +130,7 @@ async def test_reassert_locks_existing_parent_until_child_insert(pg0_db_url):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_phase2_reasserts_entity_pruned_after_resolution(pg0_db_url):
"""End-to-end: a prune between the two retain phases must not become data loss.
@@ -45,12 +45,9 @@ class GeminiLikeAPIError(Exception):
"""
async def _count_memory_units(memory, bank_id: str) -> int:
pool = await memory._get_pool()
return await pool.fetchval(
"SELECT COUNT(*) FROM memory_units WHERE bank_id = $1",
bank_id,
)
async def _count_memory_units(memory, bank_id: str, request_context) -> int:
listing = await memory.list_memory_units(bank_id, limit=1000, request_context=request_context)
return listing["total"]
async def _run_retain_through_worker(memory, extraction_error: Exception, *, retry_count: int = 0):
@@ -134,7 +131,7 @@ async def _run_retain_through_worker(memory, extraction_error: Exception, *, ret
ids=["rate_limit", "non_openai_provider_5xx", "value_error"],
)
@pytest.mark.asyncio
async def test_extraction_failure_is_retried_never_silently_completed(memory, extraction_error):
async def test_extraction_failure_is_retried_never_silently_completed(memory, extraction_error, request_context):
"""An extraction-LLM failure must NEVER complete the op with 0 facts.
Regardless of the failure type or provider, the error propagates into the
@@ -143,7 +140,7 @@ async def test_extraction_failure_is_retried_never_silently_completed(memory, ex
memory_units and ``retry_count`` 0, silently losing the document's memory.
"""
final, bank_id = await _run_retain_through_worker(memory, extraction_error)
unit_count = await _count_memory_units(memory, bank_id)
unit_count = await _count_memory_units(memory, bank_id, request_context)
assert final["status"] != "completed", (
"BUG (#1833): extraction failure was swallowed — operation marked 'completed' with "
@@ -155,7 +152,7 @@ async def test_extraction_failure_is_retried_never_silently_completed(memory, ex
@pytest.mark.asyncio
async def test_extraction_failure_at_retry_cap_fails_terminally(memory):
async def test_extraction_failure_at_retry_cap_fails_terminally(memory, request_context):
"""Once the worker retry cap is reached, extraction failures must fail terminally.
This guards the recovered-worker path seen in vectorize-io/hindsight#2413:
@@ -168,7 +165,7 @@ async def test_extraction_failure_at_retry_cap_fails_terminally(memory):
RuntimeError("structured JSON parse failed after all retain_extract_facts attempts"),
retry_count=3,
)
unit_count = await _count_memory_units(memory, bank_id)
unit_count = await _count_memory_units(memory, bank_id, request_context)
assert final["status"] == "failed", f"expected retry-capped task to fail, got {final['status']!r}"
assert final["retry_count"] == 3
@@ -226,6 +226,7 @@ class TestSchemaIsolation:
assert not errors, f"Schema context isolation errors: {errors}"
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_list_memories_respects_schema(self, memory, pg0_db_url):
"""
list_memory_units should only return memories from the current schema.
@@ -67,6 +67,7 @@ def store_document_text_disabled(memory):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_text_storage_disabled_nulls_text_but_keeps_memories(
memory, request_context, store_document_text_disabled
):
@@ -230,6 +231,7 @@ def test_store_document_text_is_bank_configurable():
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_store_document_text_per_bank_override(memory, request_context):
"""The flag is overridable per bank: one bank drops raw text while another,
on the same server default, keeps it."""
@@ -1415,6 +1415,7 @@ async def test_list_tags_ordered_by_count(api_client):
@pytest.mark.asyncio
@pytest.mark.memory_backend_incompatible
async def test_list_memories_includes_tags(api_client, test_bank_id):
"""Test that list memories endpoint returns tags for each memory unit.
@@ -45,18 +45,24 @@ async def test_temporal_ranges_are_written(memory_real_llm, request_context):
# Give it a moment for async processing
await asyncio.sleep(2)
# Retrieve facts from database directly
pool = await memory_real_llm._get_pool()
async with pool.acquire() as conn:
rows = await conn.fetch(
"""
SELECT id, text, event_date, occurred_start, occurred_end, mentioned_at
FROM memory_units
WHERE bank_id = $1
ORDER BY created_at
""",
bank_id,
)
# Retrieve the facts through the read API. It hands back the temporal fields as
# ISO strings, so parse them once here and the date arithmetic below is unchanged.
page = await memory_real_llm.list_memory_units(bank_id, limit=500, request_context=request_context)
def _dt(value: str | None) -> datetime | None:
return datetime.fromisoformat(value) if value else None
rows = [
{
"id": item["id"],
"text": item["text"],
"event_date": _dt(item["date"]),
"occurred_start": _dt(item["occurred_start"]),
"occurred_end": _dt(item["occurred_end"]),
"mentioned_at": _dt(item["mentioned_at"]),
}
for item in page["items"]
]
print(f"\n\n=== Retrieved {len(rows)} facts ===")
for i, row in enumerate(rows):
+4 -5
View File
@@ -1277,12 +1277,11 @@ class TestRetainCompletedWebhook:
outbox_callback=callback,
)
async with memory._pool.acquire() as conn:
stored_units = await conn.fetchval(
"SELECT count(*) FROM memory_units WHERE bank_id = $1 AND document_id = $2",
bank_id,
"doc-counted",
stored_units = (
await memory.list_memory_units(
bank_id, document_id="doc-counted", limit=1000, request_context=request_context
)
)["total"]
assert stored_units > 0, "fixture precondition: the mock LLM must extract facts here"
payloads = await self._retain_delivery_payloads(memory._pool, bank_id)
@@ -489,7 +489,8 @@ Every document this integration writes carries provenance tags — `source:chat`
`knowledge:<kind>`, plus anything from `retainTags`. Those tags say **who wrote** a memory; they are
what filters recall and draws each document's agent logo, and they stay on the facts.
They are not, however, a good boundary for [observations](/developer/observations). Consolidation's own
They are not, however, a good boundary for
[observations](/developer/observations). Consolidation's own
default (`combined`) builds one observation set per distinct tag set, so the same repository
worked on by two agents would grow two parallel sets of beliefs — one per harness — that never
merge, each blind to the other, at double the consolidation cost. Which agent happened to be typing
@@ -61,9 +61,12 @@ function build() {
.join('\n')
// Repo-relative links 404 on the docs site; keep the label, drop the link.
.replace(/\[([^\]]+)\]\((?!https?:|\/)[^)]+\)/g, '$1')
// Our own absolute asset URLs -> site-relative, so the page uses THIS build's static files
// rather than whatever is live in production.
.replace(/https:\/\/hindsight\.vectorize\.io\/(img\/[^\s"')]+)/g, '/$1')
// Our own absolute URLs -> site-relative. Assets so the page uses THIS build's static
// files rather than whatever is live in production; doc links so they resolve within the
// site (and so the docs-skill generator can turn them into file-relative paths, which it
// cannot do with an absolute URL). The README keeps them absolute because it also renders
// on GitHub, where site-relative would 404.
.replace(/https:\/\/hindsight\.vectorize\.io\/([^\s"')]+)/g, '/$1')
.replace(/\n{3,}/g, '\n\n')
.trim();
return `${FRONTMATTER}\n${body}\n`;
@@ -484,7 +484,8 @@ Every document this integration writes carries provenance tags — `source:chat`
`knowledge:<kind>`, plus anything from `retainTags`. Those tags say **who wrote** a memory; they are
what filters recall and draws each document's agent logo, and they stay on the facts.
They are not, however, a good boundary for [observations](../../developer/observations.md). Consolidation's own
They are not, however, a good boundary for
[observations](../../developer/observations.md). Consolidation's own
default (`combined`) builds one observation set per distinct tag set, so the same repository
worked on by two agents would grow two parallel sets of beliefs — one per harness — that never
merge, each blind to the other, at double the consolidation cost. Which agent happened to be typing