Duke/hindsight
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: report total on the mental-model and directive list endpoints (#3548)

Browse Source 
* fix: report total on the mental-model and directive list endpoints

Both list endpoints accepted limit/offset but returned a bare `items`
array, so a caller could not distinguish a full page from the end of the
collection and silently saw only the first 100 rows. They now return
`total` (every match, not just the page) with the applied `limit`/`offset`,
matching the documents/memories/tags/chunks/operations endpoints.

- engine: `list_mental_models` / `list_directives` return a typed page
  (`MentalModelPage` / `DirectivePage`) with items + total, counted in the
  same connection as the page query.
- engine: tie-break the ORDER BY on `id`. `last_refreshed_at` (models) and
  `(priority, created_at)` (directives) are not unique — a bank-template
  import stamps a whole batch at once — so ties could reorder between
  queries and a paging caller would see one row twice and miss another.
- engine: `limit=None` returns every match. Bank-template export and import
  now use it: under the default page size an export dropped everything past
  the first 100, and import's create/update decision was made against a
  partial view of the bank, so it could create duplicates.
- mcp: `list_mental_models` / `list_directives` gained limit/offset and
  report total — agents previously could not reach past the first 100.
- control plane: `listAllMentalModels` / `listAllDirectives` page to total;
  the stats freshness card, mental-models view, bank profile and think view
  use them. The directives proxy route forwards limit/offset.
- clients: both maintained wrappers gained limit/offset on the directive
  list, with mapping regression tests on each side.

* test(control-plane): cover the list-all paging helpers

The mental-model and directive paging loops read the new `total` to decide
whether to ask again, including the empty-page guard that stops them if rows
are deleted mid-page.

* fix: update the reflect LLM-config test mock and regenerate the Go client

Two CI misses from the paging change:

- `test_per_operation_llm_config.py` stubs `list_directives` on the engine and
  reflect now reads `.items` off it, so the stub has to return a DirectivePage.
- The Go client was not regenerated (the generator skips it when Go is absent),
  leaving `model_directive_list_response.go`, `model_mental_model_list_response.go`
  and `api/openapi.yaml` without the new total/limit/offset fields.
This commit is contained in:
Nicolò Boschi
2026-08-17 16:02:09 +02:00
committed by GitHub
parent 3770d62e34
commit 803a45171d
30 changed files with 1014 additions and 130 deletions
Download Patch File Download Diff File Expand all files Collapse all files
hindsight-api-slim/hindsight_api/api/http.py
+43 -19
View File
@@ -2110,6 +2110,9 @@ class DirectiveListResponse(BaseModel):
"""Response model for listing directives."""
items: list[DirectiveResponse]
total: int = Field(description="Total number of directives matching the filter (not just this page)")
limit: int = Field(description="Page size that was applied")
offset: int = Field(description="Offset that was applied")
class CreateDirectiveRequest(BaseModel):
@@ -2334,6 +2337,9 @@ class MentalModelListResponse(BaseModel):
"""Response model for listing mental models."""
items: list[MentalModelResponse]
total: int = Field(description="Total number of mental models matching the filter (not just this page)")
limit: int = Field(description="Page size that was applied")
offset: int = Field(description="Offset that was applied")
# =========================================================================
@@ -2940,17 +2946,21 @@ async def apply_bank_template_manifest(
projected_mental_model_ids = {item.id for item in default_mental_models} & imported_mental_model_ids
projected_directive_names = {item.name for item in default_directives} & imported_directive_names
# limit=None throughout the import path: a create/update decision per imported
# resource is only correct against the bank's *whole* set. Under the default
# page size a bank with more than 100 models would look like it lacked the
# ones past the first page, and the import would create duplicates.
existing_by_id: dict[str, dict[str, Any]] = {}
if bank_exists and manifest.mental_models:
existing = await memory.list_mental_models(bank_id=bank_id, request_context=request_context)
existing_by_id = {m["id"]: m for m in existing}
existing = await memory.list_mental_models(bank_id=bank_id, limit=None, request_context=request_context)
existing_by_id = {m["id"]: m for m in existing.items}
existing_by_name: dict[str, dict[str, Any]] = {}
if bank_exists and manifest.directives:
existing_directives = await memory.list_directives(
bank_id=bank_id, active_only=False, request_context=request_context
bank_id=bank_id, active_only=False, limit=None, request_context=request_context
)
existing_by_name = {d["name"]: d for d in existing_directives}
existing_by_name = {d["name"]: d for d in existing_directives.items}
bank_writes: list[BankTemplateImportWrite] = []
if config_updates:
@@ -2994,9 +3004,10 @@ async def apply_bank_template_manifest(
if projected_mental_model_ids:
provisioned = await memory.list_mental_models(
bank_id=bank_id,
limit=None,
request_context=request_context,
)
provisioned_by_id = {item["id"]: item for item in provisioned}
provisioned_by_id = {item["id"]: item for item in provisioned.items}
existing_by_id.update(
{
item_id: provisioned_by_id[item_id]
@@ -3008,9 +3019,10 @@ async def apply_bank_template_manifest(
provisioned = await memory.list_directives(
bank_id=bank_id,
active_only=False,
limit=None,
request_context=request_context,
)
provisioned_by_name = {item["name"]: item for item in provisioned}
provisioned_by_name = {item["name"]: item for item in provisioned.items}
existing_by_name.update(
{name: provisioned_by_name[name] for name in projected_directive_names & provisioned_by_name.keys()}
)
@@ -3038,17 +3050,18 @@ async def apply_default_bank_template_resources(
"""Apply only the resources from a server-owned default template."""
existing_by_id: dict[str, dict[str, Any]] = {}
if manifest.mental_models:
existing = await memory.list_mental_models(bank_id=bank_id, request_context=request_context)
existing_by_id = {model["id"]: model for model in existing}
existing = await memory.list_mental_models(bank_id=bank_id, limit=None, request_context=request_context)
existing_by_id = {model["id"]: model for model in existing.items}
existing_by_name: dict[str, dict[str, Any]] = {}
if manifest.directives:
existing_directives = await memory.list_directives(
bank_id=bank_id,
active_only=False,
limit=None,
request_context=request_context,
)
existing_by_name = {directive["name"]: directive for directive in existing_directives}
existing_by_name = {directive["name"]: directive for directive in existing_directives.items}
await _apply_bank_template_resources(
memory,
@@ -5217,7 +5230,7 @@ def _register_routes(app: FastAPI):
):
"""List mental models for a bank."""
try:
mental_models = await app.state.memory.list_mental_models(
page = await app.state.memory.list_mental_models(
bank_id=bank_id,
tags=tags_filter,
tags_match=tags_match,
@@ -5226,7 +5239,12 @@ def _register_routes(app: FastAPI):
offset=offset,
request_context=request_context,
)
return MentalModelListResponse(items=[MentalModelResponse(**m) for m in mental_models])
return MentalModelListResponse(
items=[MentalModelResponse(**m) for m in page.items],
total=page.total,
limit=limit,
offset=offset,
)
except OperationValidationError as e:
raise HTTPException(status_code=e.status_code, detail=e.reason)
except (AuthenticationError, HTTPException):
@@ -5956,7 +5974,7 @@ def _register_routes(app: FastAPI):
):
"""List directives for a bank."""
try:
directives = await app.state.memory.list_directives(
page = await app.state.memory.list_directives(
bank_id=bank_id,
tags=tags_filter,
tags_match=tags_match,
@@ -5965,7 +5983,12 @@ def _register_routes(app: FastAPI):
offset=offset,
request_context=request_context,
)
return DirectiveListResponse(items=[DirectiveResponse(**d) for d in directives])
return DirectiveListResponse(
items=[DirectiveResponse(**d) for d in page.items],
total=page.total,
limit=limit,
offset=offset,
)
except OperationValidationError as e:
raise HTTPException(status_code=e.status_code, detail=e.reason)
except (AuthenticationError, HTTPException):
@@ -7064,12 +7087,13 @@ def _register_routes(app: FastAPI):
filtered_overrides = {k: v for k, v in bank_overrides.items() if k in template_config_fields}
bank_config = BankTemplateConfig(**filtered_overrides) if filtered_overrides else None
# Get mental models
# Get mental models (limit=None — an export that stopped at the
# default page size would silently drop the rest of the bank)
mental_models_raw = await app.state.memory.list_mental_models(
bank_id=bank_id, request_context=request_context
bank_id=bank_id, limit=None, request_context=request_context
)
template_mental_models: list[BankTemplateMentalModel] = []
for mm in mental_models_raw:
for mm in mental_models_raw.items:
trigger_data = mm.get("trigger", {})
trigger = MentalModelTrigger(**trigger_data) if trigger_data else MentalModelTrigger()
template_mental_models.append(
@@ -7083,12 +7107,12 @@ def _register_routes(app: FastAPI):
)
)
# Get directives
# Get directives (limit=None for the same reason as the models above)
directives_raw = await app.state.memory.list_directives(
bank_id=bank_id, active_only=False, request_context=request_context
bank_id=bank_id, active_only=False, limit=None, request_context=request_context
)
template_directives: list[BankTemplateDirective] = []
for d in directives_raw:
for d in directives_raw.items:
template_directives.append(
BankTemplateDirective(
name=d["name"],
hindsight-api-slim/hindsight_api/engine/memory_engine.py
+95 -26
View File
@@ -311,6 +311,26 @@ class _LlmProbeOutcome:
latency_ms: float | None
@dataclass(frozen=True)
class MentalModelPage:
"""One page of mental models plus the number of models the filter matches.
``total`` counts every match, not the page — callers page until they have it
(the list endpoints for documents, memories and tags return the same shape).
"""
items: list[dict[str, Any]]
total: int
@dataclass(frozen=True)
class DirectivePage:
"""One page of directives plus the number of directives the filter matches."""
items: list[dict[str, Any]]
total: int
def _consolidation_retry_backoff_seconds(retry_count: int) -> int:
"""Capped exponential backoff: 5, 10, 20, 40, 80, 160, 320, 640, 1280, 1800, 1800, …"""
return min(
@@ -11372,7 +11392,7 @@ class MemoryEngine(MemoryEngineInterface):
request_context=request_context,
isolation_mode=True,
)
directives = directives_raw
directives = directives_raw.items
if directives:
logger.info(f"[REFLECT {reflect_id}] Loaded {len(directives)} directives")
@@ -11570,7 +11590,7 @@ class MemoryEngine(MemoryEngineInterface):
# Add directives to based_on["directives"]
# Store raw directive dicts (with id, name, content) for http.py to convert to ReflectDirective
for directive_raw in directives_raw:
for directive_raw in directives:
based_on["directives"].append(
{
"id": directive_raw["id"],
@@ -12395,10 +12415,10 @@ class MemoryEngine(MemoryEngineInterface):
tags: list[str] | None = None,
tags_match: str = "any",
detail: str = "full",
limit: int = 100,
limit: int | None = 100,
offset: int = 0,
request_context: "RequestContext",
) -> list[dict[str, Any]]:
) -> MentalModelPage:
"""List pinned mental models for a bank.
Args:
@@ -12406,12 +12426,15 @@ class MemoryEngine(MemoryEngineInterface):
tags: Optional tags to filter by
tags_match: How to match tags - 'any', 'all', or 'exact'
detail: Detail level - 'metadata', 'content', or 'full'
limit: Maximum number of results
limit: Maximum number of results, or None for every match. The HTTP
endpoint always caps it; None is for internal callers that must
see the whole set (bank-template export/import), which used to
silently take the first page and treat the rest as absent.
offset: Offset for pagination
request_context: Request context for authentication
Returns:
List of pinned mental model dicts
The requested page and the total number of matching models
"""
await self._authenticate_tenant(request_context)
if self._operation_validator:
@@ -12426,16 +12449,38 @@ class MemoryEngine(MemoryEngineInterface):
async with acquire_with_retry(backend) as conn:
# Build tag filter
tag_filter = ""
params: list[Any] = [bank_id, limit, offset]
filter_params: list[Any] = [bank_id]
if tags:
if tags_match == "all":
tag_filter = " AND tags @> $4::varchar[]"
tag_filter = " AND tags @> $2::varchar[]"
elif tags_match == "exact":
tag_filter = " AND tags = $4::varchar[]"
tag_filter = " AND tags = $2::varchar[]"
else: # any
tag_filter = " AND tags && $4::varchar[]"
params.append(tags)
tag_filter = " AND tags && $2::varchar[]"
filter_params.append(tags)
total = await conn.fetchval(
f"""
SELECT COUNT(*)
FROM {fq_table("mental_models")}
WHERE bank_id = $1 {tag_filter}
""",
*filter_params,
)
page_params = list(filter_params)
pagination = ""
if limit is not None:
pagination = f"LIMIT ${len(page_params) + 1} OFFSET ${len(page_params) + 2}"
page_params.extend([limit, offset])
elif offset:
pagination = f"OFFSET ${len(page_params) + 1}"
page_params.append(offset)
# Tie-break on id: last_refreshed_at is not unique (a bank-template
# import stamps a whole batch at once), and rows that tie can swap
# order between two queries, so a paging caller would see one model
# twice and never see another.
rows = await conn.fetch(
f"""
SELECT id, bank_id, name, source_query, content, tags,
@@ -12443,13 +12488,16 @@ class MemoryEngine(MemoryEngineInterface):
max_tokens, trigger, structured_content
FROM {fq_table("mental_models")}
WHERE bank_id = $1 {tag_filter}
ORDER BY last_refreshed_at DESC
LIMIT $2 OFFSET $3
ORDER BY last_refreshed_at DESC, id DESC
{pagination}
""",
*params,
*page_params,
)
return [self._row_to_mental_model(row, detail=detail) for row in rows]
return MentalModelPage(
items=[self._row_to_mental_model(row, detail=detail) for row in rows],
total=int(total or 0),
)
async def get_mental_model(
self,
@@ -14791,11 +14839,11 @@ class MemoryEngine(MemoryEngineInterface):
tags_match: TagsMatch = "any",
tag_groups: list[TagGroup] | None = None,
active_only: bool = True,
limit: int = 100,
limit: int | None = 100,
offset: int = 0,
request_context: "RequestContext",
isolation_mode: bool = False,
) -> list[dict[str, Any]]:
) -> DirectivePage:
"""List directives for a bank.
Args:
@@ -14806,7 +14854,8 @@ class MemoryEngine(MemoryEngineInterface):
if both are provided each applies its own OR-with-untagged wrapping
and the two are AND-ed together)
active_only: Only return active directives (default True)
limit: Maximum number of results
limit: Maximum number of results, or None for every match (used by
bank-template export/import, which must see the whole set)
offset: Offset for pagination
request_context: Request context for authentication
isolation_mode: When True and both tags and tag_groups are None, only
@@ -14815,7 +14864,7 @@ class MemoryEngine(MemoryEngineInterface):
behavior - returns all directives when no tag filter is supplied).
Returns:
List of directive dicts
The requested page and the total number of matching directives
"""
await self._authenticate_tenant(request_context)
if self._operation_validator:
@@ -14870,20 +14919,40 @@ class MemoryEngine(MemoryEngineInterface):
# This ensures tag-scoped directives don't apply to untagged operations
filters.append("(tags IS NULL OR tags = '{}')")
params.extend([limit, offset])
where_clause = " AND ".join(filters)
rows = await conn.fetch(
total = await conn.fetchval(
f"""
SELECT id, bank_id, name, content, priority, is_active, tags, created_at, updated_at
SELECT COUNT(*)
FROM {fq_table("directives")}
WHERE {" AND ".join(filters)}
ORDER BY priority DESC, created_at DESC
LIMIT ${param_idx} OFFSET ${param_idx + 1}
WHERE {where_clause}
""",
*params,
)
return [self._row_to_directive(row) for row in rows]
pagination = ""
if limit is not None:
pagination = f"LIMIT ${param_idx} OFFSET ${param_idx + 1}"
params.extend([limit, offset])
elif offset:
pagination = f"OFFSET ${param_idx}"
params.append(offset)
# Tie-break on id so ties on (priority, created_at) keep a stable
# order across pages — without it a paging caller can see one
# directive twice and miss another.
rows = await conn.fetch(
f"""
SELECT id, bank_id, name, content, priority, is_active, tags, created_at, updated_at
FROM {fq_table("directives")}
WHERE {where_clause}
ORDER BY priority DESC, created_at DESC, id DESC
{pagination}
""",
*params,
)
return DirectivePage(items=[self._row_to_directive(row) for row in rows], total=int(total or 0))
async def get_directive(
self,
hindsight-api-slim/hindsight_api/mcp_tools.py
+32 -8
View File
@@ -1307,6 +1307,8 @@ def _register_list_mental_models(mcp: FastMCP, memory: MemoryEngine, config: MCP
async def list_mental_models(
tags: list[str] | None = None,
detail: str = "full",
limit: int = 100,
offset: int = 0,
bank_id: str | None = None,
) -> str:
"""
@@ -1319,6 +1321,8 @@ def _register_list_mental_models(mcp: FastMCP, memory: MemoryEngine, config: MCP
Args:
tags: Optional tags to filter by (returns models matching any tag)
detail: Detail level - 'metadata' (names/tags only), 'content' (adds content/config), 'full' (includes reflect_response). Default: 'full'
limit: Maximum number of results (default: 100)
offset: Pagination offset (default: 0). Page until the returned items add up to 'total'.
bank_id: Optional bank to list from (defaults to session bank). Use for cross-bank operations.
"""
try:
@@ -1326,13 +1330,15 @@ def _register_list_mental_models(mcp: FastMCP, memory: MemoryEngine, config: MCP
if target_bank is None:
return '{"error": "No bank_id configured", "items": []}'
models = await memory.list_mental_models(
page = await memory.list_mental_models(
bank_id=target_bank,
tags=tags,
detail=detail,
limit=limit,
offset=offset,
request_context=_get_request_context(config),
)
return json.dumps({"items": models}, indent=2, default=str)
return json.dumps({"items": page.items, "total": page.total}, indent=2, default=str)
except OperationValidationError as e:
logger.warning(f"Operation rejected: {e}")
return json.dumps({"error": str(e)})
@@ -1346,6 +1352,8 @@ def _register_list_mental_models(mcp: FastMCP, memory: MemoryEngine, config: MCP
async def list_mental_models(
tags: list[str] | None = None,
detail: str = "full",
limit: int = 100,
offset: int = 0,
) -> dict:
"""
List mental models (pinned reflections) for this memory bank.
@@ -1357,19 +1365,23 @@ def _register_list_mental_models(mcp: FastMCP, memory: MemoryEngine, config: MCP
Args:
tags: Optional tags to filter by (returns models matching any tag)
detail: Detail level - 'metadata' (names/tags only), 'content' (adds content/config), 'full' (includes reflect_response). Default: 'full'
limit: Maximum number of results (default: 100)
offset: Pagination offset (default: 0). Page until the returned items add up to 'total'.
"""
try:
target_bank = config.bank_id_resolver()
if target_bank is None:
return {"error": "No bank_id configured", "items": []}
models = await memory.list_mental_models(
page = await memory.list_mental_models(
bank_id=target_bank,
tags=tags,
detail=detail,
limit=limit,
offset=offset,
request_context=_get_request_context(config),
)
return {"items": models}
return {"items": page.items, "total": page.total}
except OperationValidationError as e:
logger.warning(f"Operation rejected: {e}")
return {"error": str(e)}
@@ -2036,6 +2048,8 @@ def _register_list_directives(mcp: FastMCP, memory: MemoryEngine, config: MCPToo
async def list_directives(
tags: list[str] | None = None,
active_only: bool = True,
limit: int = 100,
offset: int = 0,
bank_id: str | None = None,
) -> str:
"""
@@ -2047,6 +2061,8 @@ def _register_list_directives(mcp: FastMCP, memory: MemoryEngine, config: MCPToo
Args:
tags: Optional tags to filter by
active_only: If True, only return active directives (default: True)
limit: Maximum number of results (default: 100)
offset: Pagination offset (default: 0). Page until the returned items add up to 'total'.
bank_id: Optional bank (defaults to session bank). Use for cross-bank operations.
"""
try:
@@ -2054,13 +2070,15 @@ def _register_list_directives(mcp: FastMCP, memory: MemoryEngine, config: MCPToo
if target_bank is None:
return '{"error": "No bank_id configured"}'
directives = await memory.list_directives(
page = await memory.list_directives(
target_bank,
tags=tags,
active_only=active_only,
limit=limit,
offset=offset,
request_context=_get_request_context(config),
)
return json.dumps({"items": directives}, indent=2, default=str)
return json.dumps({"items": page.items, "total": page.total}, indent=2, default=str)
except OperationValidationError as e:
logger.warning(f"Operation rejected: {e}")
return json.dumps({"error": str(e)})
@@ -2074,6 +2092,8 @@ def _register_list_directives(mcp: FastMCP, memory: MemoryEngine, config: MCPToo
async def list_directives(
tags: list[str] | None = None,
active_only: bool = True,
limit: int = 100,
offset: int = 0,
) -> dict:
"""
List directives for this memory bank.
@@ -2084,19 +2104,23 @@ def _register_list_directives(mcp: FastMCP, memory: MemoryEngine, config: MCPToo
Args:
tags: Optional tags to filter by
active_only: If True, only return active directives (default: True)
limit: Maximum number of results (default: 100)
offset: Pagination offset (default: 0). Page until the returned items add up to 'total'.
"""
try:
target_bank = config.bank_id_resolver()
if target_bank is None:
return {"error": "No bank_id configured", "items": []}
directives = await memory.list_directives(
page = await memory.list_directives(
target_bank,
tags=tags,
active_only=active_only,
limit=limit,
offset=offset,
request_context=_get_request_context(config),
)
return {"items": directives}
return {"items": page.items, "total": page.total}
except OperationValidationError as e:
logger.warning(f"Operation rejected: {e}")
return {"error": str(e)}
hindsight-api-slim/tests/e2e_oracle_smoke.py
+3 -3
View File
@@ -247,10 +247,10 @@ async def _run() -> None:
# ------------------------------------------------------------------
_log(7, total_steps, "Listing mental models ...")
models = await engine.list_mental_models(bank_id=bank_id, request_context=ctx)
assert len(models) > 0, "Should have at least one mental model"
found = any((m.get("mental_model_id") or m.get("id")) == mm_id for m in models)
assert models.total > 0, "Should have at least one mental model"
found = any((m.get("mental_model_id") or m.get("id")) == mm_id for m in models.items)
assert found, f"Mental model {mm_id} not found in list"
print(f" -> found {len(models)} mental model(s)")
print(f" -> found {models.total} mental model(s)")
# ------------------------------------------------------------------
# 8. Delete bank (cleanup)
hindsight-api-slim/tests/test_knowledge_base.py
+2 -2
View File
@@ -385,7 +385,7 @@ class TestCreate:
)
after = await memory.list_mental_models(bank_id, request_context=request_context)
assert {mm["id"] for mm in after} == {mm["id"] for mm in before}
assert {mm["id"] for mm in after.items} == {mm["id"] for mm in before.items}
await memory.delete_bank(bank_id, request_context=request_context)
async def test_create_page_under_page_rolls_back_mental_model(self, memory: MemoryEngine, request_context):
@@ -406,7 +406,7 @@ class TestCreate:
)
after = await memory.list_mental_models(bank_id, request_context=request_context)
assert {mm["id"] for mm in after} == {mm["id"] for mm in before}
assert {mm["id"] for mm in after.items} == {mm["id"] for mm in before.items}
await memory.delete_bank(bank_id, request_context=request_context)
async def test_duplicate_page_rolls_back_mental_model(self, memory: MemoryEngine, request_context):
hindsight-api-slim/tests/test_list_pagination.py
+170
View File
@@ -0,0 +1,170 @@
"""Pagination contract for the mental-model and directive list endpoints.
Both used to return a bare ``items`` array, so a caller could not tell a full
page from the end of the collection and silently saw only the first 100 rows.
They now report ``total`` (every match, not just the page) alongside the
applied ``limit``/``offset``, like the documents/memories/tags endpoints.
"""
import uuid
import httpx
import pytest
from hindsight_api.engine.memory_engine import MemoryEngine
pytestmark = pytest.mark.asyncio
async def _make_mental_models(memory: MemoryEngine, bank_id: str, count: int, request_context) -> None:
await memory.get_bank_profile(bank_id=bank_id, request_context=request_context)
for i in range(count):
await memory.create_mental_model(
bank_id=bank_id,
name=f"Model {i}",
source_query=f"Query {i}",
content=f"Content {i}",
tags=["even"] if i % 2 == 0 else ["odd"],
request_context=request_context,
)
async def _make_directives(memory: MemoryEngine, bank_id: str, count: int, request_context) -> None:
await memory.get_bank_profile(bank_id=bank_id, request_context=request_context)
for i in range(count):
await memory.create_directive(
bank_id=bank_id,
name=f"Directive {i}",
content=f"Rule {i}",
tags=["even"] if i % 2 == 0 else ["odd"],
request_context=request_context,
)
class TestMentalModelPagination:
async def test_total_counts_every_match_not_the_page(self, memory: MemoryEngine, request_context):
bank_id = f"test-mm-page-{uuid.uuid4().hex[:8]}"
await _make_mental_models(memory, bank_id, 5, request_context)
page = await memory.list_mental_models(bank_id=bank_id, limit=2, request_context=request_context)
assert len(page.items) == 2
assert page.total == 5
await memory.delete_bank(bank_id, request_context=request_context)
async def test_total_respects_the_tag_filter(self, memory: MemoryEngine, request_context):
bank_id = f"test-mm-page-{uuid.uuid4().hex[:8]}"
await _make_mental_models(memory, bank_id, 5, request_context)
page = await memory.list_mental_models(bank_id=bank_id, tags=["odd"], limit=1, request_context=request_context)
assert len(page.items) == 1
assert page.total == 2
await memory.delete_bank(bank_id, request_context=request_context)
async def test_paging_covers_every_model_exactly_once(self, memory: MemoryEngine, request_context):
"""The models are created in one burst, so last_refreshed_at ties — without the
id tie-break the pages would overlap and drop rows."""
bank_id = f"test-mm-page-{uuid.uuid4().hex[:8]}"
await _make_mental_models(memory, bank_id, 5, request_context)
seen: list[str] = []
offset = 0
while True:
page = await memory.list_mental_models(
bank_id=bank_id, limit=2, offset=offset, request_context=request_context
)
seen.extend(m["id"] for m in page.items)
offset += 2
if offset >= page.total:
break
assert len(set(seen)) == 5
assert len(seen) == 5
await memory.delete_bank(bank_id, request_context=request_context)
async def test_limit_none_returns_everything(self, memory: MemoryEngine, request_context):
"""The bank-template export/import path passes limit=None; it must see the
whole set, not the first page."""
bank_id = f"test-mm-page-{uuid.uuid4().hex[:8]}"
await _make_mental_models(memory, bank_id, 5, request_context)
page = await memory.list_mental_models(bank_id=bank_id, limit=None, request_context=request_context)
assert len(page.items) == 5
assert page.total == 5
await memory.delete_bank(bank_id, request_context=request_context)
async def test_http_response_reports_total_limit_offset(
self, memory: MemoryEngine, api_client: httpx.AsyncClient, request_context
):
bank_id = f"test-mm-page-{uuid.uuid4().hex[:8]}"
await _make_mental_models(memory, bank_id, 5, request_context)
resp = await api_client.get(
f"/v1/default/banks/{bank_id}/mental-models", params={"limit": 2, "offset": 1, "detail": "metadata"}
)
assert resp.status_code == 200
body = resp.json()
assert len(body["items"]) == 2
assert body["total"] == 5
assert body["limit"] == 2
assert body["offset"] == 1
await memory.delete_bank(bank_id, request_context=request_context)
class TestDirectivePagination:
async def test_total_counts_every_match_not_the_page(self, memory: MemoryEngine, request_context):
bank_id = f"test-dir-page-{uuid.uuid4().hex[:8]}"
await _make_directives(memory, bank_id, 5, request_context)
page = await memory.list_directives(bank_id=bank_id, limit=2, request_context=request_context)
assert len(page.items) == 2
assert page.total == 5
await memory.delete_bank(bank_id, request_context=request_context)
async def test_total_respects_active_only(self, memory: MemoryEngine, request_context):
bank_id = f"test-dir-page-{uuid.uuid4().hex[:8]}"
await memory.get_bank_profile(bank_id=bank_id, request_context=request_context)
await memory.create_directive(
bank_id=bank_id, name="Active", content="on", is_active=True, request_context=request_context
)
await memory.create_directive(
bank_id=bank_id, name="Inactive", content="off", is_active=False, request_context=request_context
)
active = await memory.list_directives(bank_id=bank_id, active_only=True, request_context=request_context)
assert active.total == 1
every = await memory.list_directives(bank_id=bank_id, active_only=False, request_context=request_context)
assert every.total == 2
await memory.delete_bank(bank_id, request_context=request_context)
async def test_limit_none_returns_everything(self, memory: MemoryEngine, request_context):
bank_id = f"test-dir-page-{uuid.uuid4().hex[:8]}"
await _make_directives(memory, bank_id, 5, request_context)
page = await memory.list_directives(bank_id=bank_id, limit=None, request_context=request_context)
assert len(page.items) == 5
assert page.total == 5
await memory.delete_bank(bank_id, request_context=request_context)
async def test_http_response_reports_total_limit_offset(
self, memory: MemoryEngine, api_client: httpx.AsyncClient, request_context
):
bank_id = f"test-dir-page-{uuid.uuid4().hex[:8]}"
await _make_directives(memory, bank_id, 5, request_context)
resp = await api_client.get(f"/v1/default/banks/{bank_id}/directives", params={"limit": 2, "offset": 1})
assert resp.status_code == 200
body = resp.json()
assert len(body["items"]) == 2
assert body["total"] == 5
assert body["limit"] == 2
assert body["offset"] == 1
await memory.delete_bank(bank_id, request_context=request_context)
hindsight-api-slim/tests/test_mcp_tools.py
+11 -7
View File
@@ -7,6 +7,7 @@ from unittest.mock import AsyncMock, MagicMock
import pytest
from hindsight_api.engine.memory_engine import DirectivePage, MentalModelPage
from hindsight_api.mcp_tools import (
MCPToolsConfig,
_validate_mental_model_inputs,
@@ -134,7 +135,8 @@ def mock_memory():
# Mental model methods — simulate engine detail filtering
async def _list_mental_models(**kwargs):
detail = kwargs.get("detail", "full")
return [_apply_detail(m, detail) for m in _FULL_MENTAL_MODELS]
items = [_apply_detail(m, detail) for m in _FULL_MENTAL_MODELS]
return MentalModelPage(items=items, total=len(items))
async def _get_mental_model(**kwargs):
detail = kwargs.get("detail", "full")
@@ -172,7 +174,9 @@ def mock_memory():
# Directive methods
memory.list_directives = AsyncMock(
return_value=[{"id": "dir-1", "name": "Be concise", "content": "Keep responses short"}]
return_value=DirectivePage(
items=[{"id": "dir-1", "name": "Be concise", "content": "Keep responses short"}], total=1
)
)
memory.create_directive = AsyncMock(return_value={"id": "dir-new", "name": "Test", "content": "Test content"})
memory.delete_directive = AsyncMock(return_value=True)
@@ -336,13 +340,13 @@ class TestMentalModelToolRegistration:
memory.list_banks = AsyncMock(return_value=[])
memory.get_bank_profile = AsyncMock(return_value={})
memory.update_bank = AsyncMock()
memory.list_mental_models = AsyncMock(return_value=[])
memory.list_mental_models = AsyncMock(return_value=MentalModelPage(items=[], total=0))
memory.get_mental_model = AsyncMock()
memory.create_mental_model = AsyncMock()
memory.submit_async_refresh_mental_model = AsyncMock()
memory.update_mental_model = AsyncMock()
memory.delete_mental_model = AsyncMock()
memory.list_directives = AsyncMock(return_value=[])
memory.list_directives = AsyncMock(return_value=DirectivePage(items=[], total=0))
memory.create_directive = AsyncMock()
memory.delete_directive = AsyncMock()
memory.list_memory_units = AsyncMock(return_value={})
@@ -406,13 +410,13 @@ class TestMentalModelToolRegistration:
memory.list_banks = AsyncMock(return_value=[])
memory.get_bank_profile = AsyncMock(return_value={})
memory.update_bank = AsyncMock()
memory.list_mental_models = AsyncMock(return_value=[])
memory.list_mental_models = AsyncMock(return_value=MentalModelPage(items=[], total=0))
memory.get_mental_model = AsyncMock()
memory.create_mental_model = AsyncMock()
memory.submit_async_refresh_mental_model = AsyncMock()
memory.update_mental_model = AsyncMock()
memory.delete_mental_model = AsyncMock()
memory.list_directives = AsyncMock(return_value=[])
memory.list_directives = AsyncMock(return_value=DirectivePage(items=[], total=0))
memory.create_directive = AsyncMock()
memory.delete_directive = AsyncMock()
memory.list_memory_units = AsyncMock(return_value={})
@@ -1876,7 +1880,7 @@ class TestEmptyListReturns:
assert '"items": []' in result or "[]" in result
async def test_list_directives_empty(self, mock_memory):
mock_memory.list_directives.return_value = []
mock_memory.list_directives.return_value = DirectivePage(items=[], total=0)
mcp = _make_mcp_server(mock_memory, {"list_directives"}, include_bank_id=True)
result = await _tools(mcp)["list_directives"].fn()
assert "[]" in result
hindsight-api-slim/tests/test_mental_models.py
+18 -17
View File
@@ -127,8 +127,9 @@ class TestDirectives:
bank_id=bank_id,
request_context=request_context,
)
assert len(directives) == 1
assert directives[0]["id"] == directive_id
assert directives.total == 1
assert len(directives.items) == 1
assert directives.items[0]["id"] == directive_id
# Update
updated = await memory.update_directive(
@@ -187,9 +188,9 @@ class TestDirectives:
bank_id=bank_id,
request_context=request_context,
)
assert len(directives) == 2
assert directives[0]["name"] == "High Priority"
assert directives[1]["name"] == "Low Priority"
assert directives.total == 2
assert directives.items[0]["name"] == "High Priority"
assert directives.items[1]["name"] == "Low Priority"
# Cleanup
await memory.delete_bank(bank_id, request_context=request_context)
@@ -224,8 +225,8 @@ class TestDirectives:
active_only=True,
request_context=request_context,
)
assert len(active_directives) == 1
assert active_directives[0]["name"] == "Active Rule"
assert active_directives.total == 1
assert active_directives.items[0]["name"] == "Active Rule"
# List all
all_directives = await memory.list_directives(
@@ -233,7 +234,7 @@ class TestDirectives:
active_only=False,
request_context=request_context,
)
assert len(all_directives) == 2
assert all_directives.total == 2
# Cleanup
await memory.delete_bank(bank_id, request_context=request_context)
@@ -300,7 +301,7 @@ class TestDirectiveTags:
bank_id=bank_id,
request_context=request_context,
)
assert len(all_directives) == 2
assert all_directives.total == 2
# Filter by project-a tag
filtered = await memory.list_directives(
@@ -308,8 +309,8 @@ class TestDirectiveTags:
tags=["project-a"],
request_context=request_context,
)
assert len(filtered) == 1
assert filtered[0]["name"] == "Rule A"
assert filtered.total == 1
assert filtered.items[0]["name"] == "Rule A"
# Cleanup
await memory.delete_bank(bank_id, request_context=request_context)
@@ -361,7 +362,7 @@ class TestDirectiveTags:
],
request_context=request_context,
)
names = {d["name"] for d in scoped}
names = {d["name"] for d in scoped.items}
assert names == {"Untagged Rule", "Hardware Rule"}, names
# Isolation mode with tag_groups still applies (only untagged + tag-matching).
@@ -371,7 +372,7 @@ class TestDirectiveTags:
request_context=request_context,
isolation_mode=True,
)
assert {d["name"] for d in isolated} == {"Untagged Rule", "Hardware Rule"}
assert {d["name"] for d in isolated.items} == {"Untagged Rule", "Hardware Rule"}
# Without any tag filter and isolation_mode=True, only untagged should come back —
# confirming this code path isn't accidentally short-circuited when tag_groups is empty.
@@ -380,7 +381,7 @@ class TestDirectiveTags:
request_context=request_context,
isolation_mode=True,
)
assert {d["name"] for d in untagged_only} == {"Untagged Rule"}
assert {d["name"] for d in untagged_only.items} == {"Untagged Rule"}
await memory.delete_bank(bank_id, request_context=request_context)
@@ -415,13 +416,13 @@ class TestDirectiveTags:
)
# Should return BOTH tagged and untagged directives
assert len(all_directives) == 2
directive_names = {d["name"] for d in all_directives}
assert all_directives.total == 2
directive_names = {d["name"] for d in all_directives.items}
assert "Untagged Directive" in directive_names
assert "Tagged Directive" in directive_names
# Verify the tagged directive has its tags
tagged = next(d for d in all_directives if d["name"] == "Tagged Directive")
tagged = next(d for d in all_directives.items if d["name"] == "Tagged Directive")
assert tagged["tags"] == ["project-x"]
# Cleanup
hindsight-api-slim/tests/test_oracle_integration.py
+4 -2
View File
@@ -838,7 +838,8 @@ class TestAdvancedFeatures:
# List
models = await oracle_memory.list_mental_models(bank_id=bank_id, request_context=request_context)
assert len(models) > 0
assert models.total > 0
assert len(models.items) > 0
# Get
fetched = await oracle_memory.get_mental_model(
@@ -990,7 +991,8 @@ class TestAdvancedFeatures:
assert directive is not None
directives = await oracle_memory.list_directives(bank_id=bank_id, request_context=request_context)
assert len(directives) > 0
assert directives.total > 0
assert len(directives.items) > 0
await oracle_memory.delete_directive(
bank_id=bank_id,
hindsight-api-slim/tests/test_per_operation_llm_config.py
+4 -1
View File
@@ -304,6 +304,7 @@ class TestReflectUsesReflectLLMConfig:
from unittest.mock import AsyncMock
from hindsight_api import MemoryEngine
from hindsight_api.engine.memory_engine import DirectivePage
from hindsight_api.engine.reflect.models import ReflectAgentResult
from hindsight_api.models import RequestContext
@@ -320,7 +321,9 @@ class TestReflectUsesReflectLLMConfig:
engine.get_bank_stats = AsyncMock(
return_value=SimpleNamespace(last_consolidated_at=None, pending_consolidation=0)
) # type: ignore[method-assign]
engine.list_directives = AsyncMock(return_value=[]) # type: ignore[method-assign]
engine.list_directives = AsyncMock( # type: ignore[method-assign]
return_value=DirectivePage(items=[], total=0)
)
engine._get_pool = AsyncMock(return_value=SimpleNamespace()) # type: ignore[method-assign]
engine._config_resolver = SimpleNamespace(
resolve_full_config=AsyncMock(return_value=SimpleNamespace(llm_gemini_safety_settings=None)),
hindsight-api-slim/tests/test_reflections.py
+4 -2
View File
@@ -149,7 +149,8 @@ class TestMentalModelsCRUD:
bank_id=bank_id,
request_context=request_context,
)
assert len(all_mental_models) == 2
assert len(all_mental_models.items) == 2
assert all_mental_models.total == 2
# List with tag filter
tag1_mental_models = await memory.list_mental_models(
@@ -157,7 +158,8 @@ class TestMentalModelsCRUD:
tags=["tag1"],
request_context=request_context,
)
assert len(tag1_mental_models) == 1
assert len(tag1_mental_models.items) == 1
assert tag1_mental_models.total == 1
# Cleanup
await memory.delete_bank(bank_id, request_context=request_context)
hindsight-clients/go/api/openapi.yaml
+38
View File
@@ -6042,6 +6042,9 @@ components:
DirectiveListResponse:
description: Response model for listing directives.
example:
total: 6
offset: 5
limit: 1
items:
- is_active: true
updated_at: updated_at
@@ -6070,8 +6073,24 @@ components:
items:
$ref: '#/components/schemas/DirectiveResponse'
type: array
total:
description: Total number of directives matching the filter (not just this
page)
title: Total
type: integer
limit:
description: Page size that was applied
title: Limit
type: integer
offset:
description: Offset that was applied
title: Offset
type: integer
required:
- items
- limit
- offset
- total
title: DirectiveListResponse
DirectiveResponse:
description: Response model for a directive.
@@ -8087,6 +8106,9 @@ components:
MentalModelListResponse:
description: Response model for listing mental models.
example:
total: 5
offset: 2
limit: 5
items:
- max_tokens: 0
created_at: created_at
@@ -8177,8 +8199,24 @@ components:
items:
$ref: '#/components/schemas/MentalModelResponse'
type: array
total:
description: Total number of mental models matching the filter (not just
this page)
title: Total
type: integer
limit:
description: Page size that was applied
title: Limit
type: integer
offset:
description: Offset that was applied
title: Offset
type: integer
required:
- items
- limit
- offset
- total
title: MentalModelListResponse
MentalModelRefreshScope:
description: |-
hindsight-clients/go/model_directive_list_response.go
+88 -1
View File
@@ -22,6 +22,12 @@ var _ MappedNullable = &DirectiveListResponse{}
// DirectiveListResponse Response model for listing directives.
type DirectiveListResponse struct {
Items []DirectiveResponse `json:"items"`
// Total number of directives matching the filter (not just this page)
Total int32 `json:"total"`
// Page size that was applied
Limit int32 `json:"limit"`
// Offset that was applied
Offset int32 `json:"offset"`
}
type _DirectiveListResponse DirectiveListResponse
@@ -30,9 +36,12 @@ type _DirectiveListResponse DirectiveListResponse
// This constructor will assign default values to properties that have it defined,
// and makes sure properties required by API are set, but the set of arguments
// will change when the set of required properties is changed
func NewDirectiveListResponse(items []DirectiveResponse) *DirectiveListResponse {
func NewDirectiveListResponse(items []DirectiveResponse, total int32, limit int32, offset int32) *DirectiveListResponse {
this := DirectiveListResponse{}
this.Items = items
this.Total = total
this.Limit = limit
this.Offset = offset
return &this
}
@@ -68,6 +77,78 @@ func (o *DirectiveListResponse) SetItems(v []DirectiveResponse) {
o.Items = v
}
// GetTotal returns the Total field value
func (o *DirectiveListResponse) GetTotal() int32 {
if o == nil {
var ret int32
return ret
}
return o.Total
}
// GetTotalOk returns a tuple with the Total field value
// and a boolean to check if the value has been set.
func (o *DirectiveListResponse) GetTotalOk() (*int32, bool) {
if o == nil {
return nil, false
}
return &o.Total, true
}
// SetTotal sets field value
func (o *DirectiveListResponse) SetTotal(v int32) {
o.Total = v
}
// GetLimit returns the Limit field value
func (o *DirectiveListResponse) GetLimit() int32 {
if o == nil {
var ret int32
return ret
}
return o.Limit
}
// GetLimitOk returns a tuple with the Limit field value
// and a boolean to check if the value has been set.
func (o *DirectiveListResponse) GetLimitOk() (*int32, bool) {
if o == nil {
return nil, false
}
return &o.Limit, true
}
// SetLimit sets field value
func (o *DirectiveListResponse) SetLimit(v int32) {
o.Limit = v
}
// GetOffset returns the Offset field value
func (o *DirectiveListResponse) GetOffset() int32 {
if o == nil {
var ret int32
return ret
}
return o.Offset
}
// GetOffsetOk returns a tuple with the Offset field value
// and a boolean to check if the value has been set.
func (o *DirectiveListResponse) GetOffsetOk() (*int32, bool) {
if o == nil {
return nil, false
}
return &o.Offset, true
}
// SetOffset sets field value
func (o *DirectiveListResponse) SetOffset(v int32) {
o.Offset = v
}
func (o DirectiveListResponse) MarshalJSON() ([]byte, error) {
toSerialize,err := o.ToMap()
if err != nil {
@@ -79,6 +160,9 @@ func (o DirectiveListResponse) MarshalJSON() ([]byte, error) {
func (o DirectiveListResponse) ToMap() (map[string]interface{}, error) {
toSerialize := map[string]interface{}{}
toSerialize["items"] = o.Items
toSerialize["total"] = o.Total
toSerialize["limit"] = o.Limit
toSerialize["offset"] = o.Offset
return toSerialize, nil
}
@@ -88,6 +172,9 @@ func (o *DirectiveListResponse) UnmarshalJSON(data []byte) (err error) {
// that every required field exists as a key in the generic map.
requiredProperties := []string{
"items",
"total",
"limit",
"offset",
}
allProperties := make(map[string]interface{})
hindsight-clients/go/model_mental_model_list_response.go
+88 -1
View File
@@ -22,6 +22,12 @@ var _ MappedNullable = &MentalModelListResponse{}
// MentalModelListResponse Response model for listing mental models.
type MentalModelListResponse struct {
Items []MentalModelResponse `json:"items"`
// Total number of mental models matching the filter (not just this page)
Total int32 `json:"total"`
// Page size that was applied
Limit int32 `json:"limit"`
// Offset that was applied
Offset int32 `json:"offset"`
}
type _MentalModelListResponse MentalModelListResponse
@@ -30,9 +36,12 @@ type _MentalModelListResponse MentalModelListResponse
// This constructor will assign default values to properties that have it defined,
// and makes sure properties required by API are set, but the set of arguments
// will change when the set of required properties is changed
func NewMentalModelListResponse(items []MentalModelResponse) *MentalModelListResponse {
func NewMentalModelListResponse(items []MentalModelResponse, total int32, limit int32, offset int32) *MentalModelListResponse {
this := MentalModelListResponse{}
this.Items = items
this.Total = total
this.Limit = limit
this.Offset = offset
return &this
}
@@ -68,6 +77,78 @@ func (o *MentalModelListResponse) SetItems(v []MentalModelResponse) {
o.Items = v
}
// GetTotal returns the Total field value
func (o *MentalModelListResponse) GetTotal() int32 {
if o == nil {
var ret int32
return ret
}
return o.Total
}
// GetTotalOk returns a tuple with the Total field value
// and a boolean to check if the value has been set.
func (o *MentalModelListResponse) GetTotalOk() (*int32, bool) {
if o == nil {
return nil, false
}
return &o.Total, true
}
// SetTotal sets field value
func (o *MentalModelListResponse) SetTotal(v int32) {
o.Total = v
}
// GetLimit returns the Limit field value
func (o *MentalModelListResponse) GetLimit() int32 {
if o == nil {
var ret int32
return ret
}
return o.Limit
}
// GetLimitOk returns a tuple with the Limit field value
// and a boolean to check if the value has been set.
func (o *MentalModelListResponse) GetLimitOk() (*int32, bool) {
if o == nil {
return nil, false
}
return &o.Limit, true
}
// SetLimit sets field value
func (o *MentalModelListResponse) SetLimit(v int32) {
o.Limit = v
}
// GetOffset returns the Offset field value
func (o *MentalModelListResponse) GetOffset() int32 {
if o == nil {
var ret int32
return ret
}
return o.Offset
}
// GetOffsetOk returns a tuple with the Offset field value
// and a boolean to check if the value has been set.
func (o *MentalModelListResponse) GetOffsetOk() (*int32, bool) {
if o == nil {
return nil, false
}
return &o.Offset, true
}
// SetOffset sets field value
func (o *MentalModelListResponse) SetOffset(v int32) {
o.Offset = v
}
func (o MentalModelListResponse) MarshalJSON() ([]byte, error) {
toSerialize,err := o.ToMap()
if err != nil {
@@ -79,6 +160,9 @@ func (o MentalModelListResponse) MarshalJSON() ([]byte, error) {
func (o MentalModelListResponse) ToMap() (map[string]interface{}, error) {
toSerialize := map[string]interface{}{}
toSerialize["items"] = o.Items
toSerialize["total"] = o.Total
toSerialize["limit"] = o.Limit
toSerialize["offset"] = o.Offset
return toSerialize, nil
}
@@ -88,6 +172,9 @@ func (o *MentalModelListResponse) UnmarshalJSON(data []byte) (err error) {
// that every required field exists as a key in the generic map.
requiredProperties := []string{
"items",
"total",
"limit",
"offset",
}
allProperties := make(map[string]interface{})
hindsight-clients/python/hindsight_client/hindsight_client.py
+19 -3
View File
@@ -1737,18 +1737,34 @@ class Hindsight:
return _run_async(self._directives_api.create_directive(bank_id, request_obj, _request_timeout=self._timeout))
def list_directives(self, bank_id: str, tags: list[str] | None = None):
def list_directives(
self,
bank_id: str,
tags: list[str] | None = None,
limit: int | None = None,
offset: int | None = None,
):
"""
List all directives in a bank (sync wrapper — use ``await client.directives.list_directives(...)`` in async code).
Args:
bank_id: The memory bank ID
tags: Optional tags to filter by
limit: Maximum number of directives to return
offset: Number of directives to skip (for pagination)
Returns:
ListDirectivesResponse with items
ListDirectivesResponse with items and the total matching the filter
"""
return _run_async(self._directives_api.list_directives(bank_id, tags=tags, _request_timeout=self._timeout))
return _run_async(
self._directives_api.list_directives(
bank_id,
tags=tags,
limit=limit,
offset=offset,
_request_timeout=self._timeout,
)
)
def get_directive(self, bank_id: str, directive_id: str):
"""
hindsight-clients/python/hindsight_client_api/models/directive_list_response.py
+9 -3
View File
@@ -17,7 +17,7 @@ import pprint
import re # noqa: F401
import json
from pydantic import BaseModel, ConfigDict
from pydantic import BaseModel, ConfigDict, Field, StrictInt
from typing import Any, ClassVar, Dict, List
from hindsight_client_api.models.directive_response import DirectiveResponse
from typing import Optional, Set
@@ -28,7 +28,10 @@ class DirectiveListResponse(BaseModel):
Response model for listing directives.
""" # noqa: E501
items: List[DirectiveResponse]
__properties: ClassVar[List[str]] = ["items"]
total: StrictInt = Field(description="Total number of directives matching the filter (not just this page)")
limit: StrictInt = Field(description="Page size that was applied")
offset: StrictInt = Field(description="Offset that was applied")
__properties: ClassVar[List[str]] = ["items", "total", "limit", "offset"]
model_config = ConfigDict(
populate_by_name=True,
@@ -88,7 +91,10 @@ class DirectiveListResponse(BaseModel):
return cls.model_validate(obj)
_obj = cls.model_validate({
"items": [DirectiveResponse.from_dict(_item) for _item in obj["items"]] if obj.get("items") is not None else None
"items": [DirectiveResponse.from_dict(_item) for _item in obj["items"]] if obj.get("items") is not None else None,
"total": obj.get("total"),
"limit": obj.get("limit"),
"offset": obj.get("offset")
})
return _obj
hindsight-clients/python/hindsight_client_api/models/mental_model_list_response.py
+9 -3
View File
@@ -17,7 +17,7 @@ import pprint
import re # noqa: F401
import json
from pydantic import BaseModel, ConfigDict
from pydantic import BaseModel, ConfigDict, Field, StrictInt
from typing import Any, ClassVar, Dict, List
from hindsight_client_api.models.mental_model_response import MentalModelResponse
from typing import Optional, Set
@@ -28,7 +28,10 @@ class MentalModelListResponse(BaseModel):
Response model for listing mental models.
""" # noqa: E501
items: List[MentalModelResponse]
__properties: ClassVar[List[str]] = ["items"]
total: StrictInt = Field(description="Total number of mental models matching the filter (not just this page)")
limit: StrictInt = Field(description="Page size that was applied")
offset: StrictInt = Field(description="Offset that was applied")
__properties: ClassVar[List[str]] = ["items", "total", "limit", "offset"]
model_config = ConfigDict(
populate_by_name=True,
@@ -88,7 +91,10 @@ class MentalModelListResponse(BaseModel):
return cls.model_validate(obj)
_obj = cls.model_validate({
"items": [MentalModelResponse.from_dict(_item) for _item in obj["items"]] if obj.get("items") is not None else None
"items": [MentalModelResponse.from_dict(_item) for _item in obj["items"]] if obj.get("items") is not None else None,
"total": obj.get("total"),
"limit": obj.get("limit"),
"offset": obj.get("offset")
})
return _obj
hindsight-clients/python/tests/test_directive_query_mapping.py
+47
View File
@@ -0,0 +1,47 @@
"""The maintained wrapper forwards directive list pagination to the SDK.
Mirrors the TypeScript wrapper's ``directive_query_mapping`` regression tests.
The directive list endpoint reports a ``total``, so dropping ``limit``/``offset``
in the wrapper would leave SDK callers stuck on the server's first page.
"""
from unittest.mock import MagicMock
from hindsight_client import Hindsight
def _capture_list(monkeypatch, client, captured):
async def fake_list(bank_id, **kwargs):
captured["bank_id"] = bank_id
captured["kwargs"] = kwargs
return MagicMock(items=[], total=0)
monkeypatch.setattr(client._directives_api, "list_directives", fake_list)
def test_list_forwards_every_supported_query_option(monkeypatch):
client = Hindsight(base_url="http://example.invalid")
captured: dict[str, object] = {}
_capture_list(monkeypatch, client, captured)
client.list_directives("bank-1", tags=["project"], limit=25, offset=50)
assert captured["bank_id"] == "bank-1"
kwargs = captured["kwargs"]
assert kwargs["tags"] == ["project"]
assert kwargs["limit"] == 25
assert kwargs["offset"] == 50
def test_list_defaults_leave_controls_unset(monkeypatch):
client = Hindsight(base_url="http://example.invalid")
captured: dict[str, object] = {}
_capture_list(monkeypatch, client, captured)
client.list_directives("bank-1")
kwargs = captured["kwargs"]
# Nothing forced on: the server keeps its own defaults for every control.
assert kwargs["tags"] is None
assert kwargs["limit"] is None
assert kwargs["offset"] is None
hindsight-clients/typescript/generated/types.gen.ts
+36
View File
@@ -1494,6 +1494,24 @@ export type DirectiveListResponse = {
* Items
*/
items: Array<DirectiveResponse>;
/**
* Total
*
* Total number of directives matching the filter (not just this page)
*/
total: number;
/**
* Limit
*
* Page size that was applied
*/
limit: number;
/**
* Offset
*
* Offset that was applied
*/
offset: number;
};
/**
@@ -3236,6 +3254,24 @@ export type MentalModelListResponse = {
* Items
*/
items: Array<MentalModelResponse>;
/**
* Total
*
* Total number of mental models matching the filter (not just this page)
*/
total: number;
/**
* Limit
*
* Page size that was applied
*/
limit: number;
/**
* Offset
*
* Offset that was applied
*/
offset: number;
};
/**
hindsight-clients/typescript/src/index.ts
+6 -2
View File
@@ -780,12 +780,16 @@ export class HindsightClient {
*/
async listDirectives(
bankId: string,
options?: { tags?: string[]; signal?: AbortSignal }
options?: { tags?: string[]; limit?: number; offset?: number; signal?: AbortSignal }
): Promise<DirectiveListResponse> {
const response = await sdk.listDirectives({
client: this.client,
path: { bank_id: bankId },
query: { tags: options?.tags },
query: {
tags: options?.tags,
...(options?.limit !== undefined ? { limit: options.limit } : {}),
...(options?.offset !== undefined ? { offset: options.offset } : {}),
},
signal: options?.signal,
});
hindsight-clients/typescript/tests/directive_query_mapping.test.ts
+45
View File
@@ -0,0 +1,45 @@
/**
* Unit tests for directive query forwarding in the hand-written wrapper.
*
* The directive list endpoint reports a `total`, so a regression that dropped
* `limit`/`offset` here would strand SDK callers on the server's first page.
*/
import { HindsightClient } from "../src";
import * as sdk from "../generated/sdk.gen";
jest.mock("../generated/sdk.gen");
const mockedList = sdk.listDirectives as jest.MockedFunction<typeof sdk.listDirectives>;
describe("directive query mapping", () => {
let client: HindsightClient;
beforeEach(() => {
client = new HindsightClient({ baseUrl: "http://localhost:8888" });
mockedList.mockReset();
mockedList.mockResolvedValue({ data: { items: [], total: 0 } } as any);
});
test("forwards every supported list query option", async () => {
const signal = new AbortController().signal;
await client.listDirectives("bank-1", { tags: ["project"], limit: 25, offset: 50, signal });
expect(mockedList).toHaveBeenCalledWith(
expect.objectContaining({
path: { bank_id: "bank-1" },
query: { tags: ["project"], limit: 25, offset: 50 },
signal,
})
);
});
test("preserves optionless and tags-only list request shapes", async () => {
await client.listDirectives("bank-1");
await client.listDirectives("bank-1", { tags: ["project"] });
expect(mockedList.mock.calls[0][0].query).toEqual({ tags: undefined });
expect(mockedList.mock.calls[1][0].query).toEqual({ tags: ["project"] });
});
});
hindsight-control-plane/src/app/api/banks/[bankId]/directives/route.ts
+8
View File
@@ -8,6 +8,8 @@ export async function GET(request: Request, { params }: { params: Promise<{ bank
const { searchParams } = new URL(request.url);
const tags = searchParams.getAll("tags");
const tagsMatch = searchParams.get("tags_match");
const limit = searchParams.get("limit");
const offset = searchParams.get("offset");
if (!bankId) {
return NextResponse.json(
@@ -26,6 +28,12 @@ export async function GET(request: Request, { params }: { params: Promise<{ bank
if (tagsMatch) {
queryParams.append("tags_match", tagsMatch);
}
if (limit) {
queryParams.append("limit", limit);
}
if (offset) {
queryParams.append("offset", offset);
}
const url = dataplaneBankUrl(
bankId,
hindsight-control-plane/src/components/bank-profile-view.tsx
+5 -4
View File
@@ -167,10 +167,11 @@ export function BankProfileView({ hideReflectFields = false }: { hideReflectFiel
try {
const [statsData, directivesData] = await Promise.all([
client.getBankStats(currentBank),
client.listDirectives(currentBank),
// The view lists every directive, so page past the endpoint's cap.
client.listAllDirectives(currentBank),
]);
setStats(statsData as BankStats);
setDirectives(directivesData.items || []);
setDirectives(directivesData);
} catch (error) {
console.error("Error refreshing stats:", error);
}
@@ -182,11 +183,11 @@ export function BankProfileView({ hideReflectFields = false }: { hideReflectFiel
const [profileData, statsData, directivesData] = await Promise.all([
client.getBankProfile(currentBank),
client.getBankStats(currentBank),
client.listDirectives(currentBank),
client.listAllDirectives(currentBank),
]);
setProfile(profileData);
setStats(statsData as BankStats);
setDirectives(directivesData.items || []);
setDirectives(directivesData);
} catch (error) {
// Error toast is shown automatically by the API client interceptor
} finally {
hindsight-control-plane/src/components/bank-stats-view.tsx
+5 -4
View File
@@ -1092,12 +1092,13 @@ export function BankStatsView() {
try {
const [statsData, mentalModelsData] = await Promise.all([
client.getBankStats(currentBank),
// The card needs names and refresh times only, and this reloads every few
// seconds — metadata keeps the stored reflect payloads off the wire.
client.listMentalModels(currentBank, { detail: "metadata" }),
// The card counts every model's freshness, so it needs the whole set, not
// the first page. It reloads every few seconds — metadata keeps the stored
// reflect payloads off the wire.
client.listAllMentalModels(currentBank, { detail: "metadata" }),
]);
setStats(statsData as BankStats);
setMentalModels(mentalModelsData.items || []);
setMentalModels(mentalModelsData);
} catch (error) {
console.error("Error loading bank stats:", error);
} finally {
hindsight-control-plane/src/components/mental-models-view.tsx
+5 -15
View File
@@ -158,21 +158,11 @@ export function MentalModelsView() {
setLoading(true);
try {
// The API caps each response at PAGE_SIZE, so page through until a short
// page is returned to load every mental model for this bank.
const PAGE_SIZE = 100;
const all: MentalModel[] = [];
for (let offset = 0; ; offset += PAGE_SIZE) {
const page = await client.listMentalModels(currentBank, {
tags: selectedTags.length > 0 ? selectedTags : undefined,
tagsMatch: selectedTags.length > 0 ? tagsMatch : undefined,
limit: PAGE_SIZE,
offset,
});
const items = page.items || [];
all.push(...items);
if (items.length < PAGE_SIZE) break;
}
// The API caps each response, so page through to the reported total.
const all = await client.listAllMentalModels(currentBank, {
tags: selectedTags.length > 0 ? selectedTags : undefined,
tagsMatch: selectedTags.length > 0 ? tagsMatch : undefined,
});
setMentalModels(all);
} catch (error) {
console.error("Error loading mental models:", error);
hindsight-control-plane/src/components/think-view.tsx
+2 -2
View File
@@ -96,8 +96,8 @@ export function ThinkView() {
setFeedbackSubmitting(true);
try {
// Find existing "General Feedback" directive
const directives = await client.listDirectives(currentBank);
const existingDirective = directives.items?.find((d) => d.name === FEEDBACK_DIRECTIVE_NAME);
const directives = await client.listAllDirectives(currentBank);
const existingDirective = directives.find((d) => d.name === FEEDBACK_DIRECTIVE_NAME);
if (existingDirective) {
// Append to existing directive content
hindsight-control-plane/src/lib/api.ts
+54 -1
View File
@@ -1155,7 +1155,12 @@ export class ControlPlaneClient {
/**
* List directives for a bank
*/
async listDirectives(bankId: string, tags?: string[], tagsMatch?: string) {
async listDirectives(
bankId: string,
tags?: string[],
tagsMatch?: string,
options: { limit?: number; offset?: number } = {}
) {
const params = new URLSearchParams();
if (tags && tags.length > 0) {
tags.forEach((t) => params.append("tags", t));
@@ -1163,6 +1168,12 @@ export class ControlPlaneClient {
if (tagsMatch) {
params.append("tags_match", tagsMatch);
}
if (options.limit !== undefined) {
params.append("limit", String(options.limit));
}
if (options.offset !== undefined) {
params.append("offset", String(options.offset));
}
const query = params.toString();
return this.fetchApi<{
items: Array<{
@@ -1176,9 +1187,27 @@ export class ControlPlaneClient {
created_at: string;
updated_at: string;
}>;
/** Every directive matching the filter, not just this page. */
total: number;
limit: number;
offset: number;
}>(bankApi(bankId, `/directives${query ? `?${query}` : ""}`));
}
/**
* List every directive for a bank, paging until `total` is reached.
*/
async listAllDirectives(bankId: string, tags?: string[], tagsMatch?: string) {
const PAGE_SIZE = 1000;
const items: Awaited<ReturnType<typeof this.listDirectives>>["items"] = [];
for (let offset = 0; ; offset += PAGE_SIZE) {
const page = await this.listDirectives(bankId, tags, tagsMatch, { limit: PAGE_SIZE, offset });
items.push(...(page.items || []));
if (items.length >= page.total || !page.items?.length) break;
}
return items;
}
/**
* Create a directive
*/
@@ -1446,9 +1475,33 @@ export class ControlPlaneClient {
based_on: Record<string, Array<{ id: string; text: string; type: string }>>;
};
}>;
/** Every mental model matching the filter, not just this page. */
total: number;
limit: number;
offset: number;
}>(bankApi(bankId, `/mental-models${query ? `?${query}` : ""}`));
}
/**
* List every mental model for a bank, paging until `total` is reached.
*
* The endpoint caps a response at 1000 models, so anything that needs the
* whole set (the list view, the freshness card) has to page.
*/
async listAllMentalModels(
bankId: string,
options: { tags?: string[]; tagsMatch?: string; detail?: "metadata" | "content" | "full" } = {}
) {
const PAGE_SIZE = 1000;
const items: Awaited<ReturnType<typeof this.listMentalModels>>["items"] = [];
for (let offset = 0; ; offset += PAGE_SIZE) {
const page = await this.listMentalModels(bankId, { ...options, limit: PAGE_SIZE, offset });
items.push(...(page.items || []));
if (items.length >= page.total || !page.items?.length) break;
}
return items;
}
/**
* Create a mental model (async - content auto-generated in background)
* Returns operation_id to track progress
hindsight-control-plane/tests/lib/list-all-paging.test.ts
+88
View File
@@ -0,0 +1,88 @@
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { ControlPlaneClient } from "@/lib/api";
vi.mock("sonner", () => ({
toast: {
error: vi.fn(),
warning: vi.fn(),
},
}));
/** A list response page: `total` counts every match, `items` only this page. */
function page(items: unknown[], total: number) {
return new Response(JSON.stringify({ items, total, limit: 1000, offset: 0 }), { status: 200 });
}
function itemsOf(count: number, offset = 0) {
return Array.from({ length: count }, (_, i) => ({ id: `item-${offset + i}` }));
}
describe("ControlPlaneClient paging helpers", () => {
let fetchSpy: ReturnType<typeof vi.spyOn>;
let client: ControlPlaneClient;
beforeEach(() => {
client = new ControlPlaneClient();
fetchSpy = vi.spyOn(globalThis, "fetch");
});
afterEach(() => {
fetchSpy.mockRestore();
});
it("stops after one request when the first page holds every mental model", async () => {
fetchSpy.mockResolvedValueOnce(page(itemsOf(3), 3));
const all = await client.listAllMentalModels("bank-a");
expect(all).toHaveLength(3);
expect(fetchSpy).toHaveBeenCalledTimes(1);
});
it("keeps requesting mental models until it has collected the reported total", async () => {
// A bank past the endpoint's 1000 cap: the total is what tells the client to
// ask again, which is exactly what a bare `items` response could not express.
fetchSpy
.mockResolvedValueOnce(page(itemsOf(1000), 1500))
.mockResolvedValueOnce(page(itemsOf(500, 1000), 1500));
const all = await client.listAllMentalModels("bank-a");
expect(all).toHaveLength(1500);
expect(fetchSpy).toHaveBeenCalledTimes(2);
const secondUrl = String(fetchSpy.mock.calls[1][0]);
expect(secondUrl).toContain("offset=1000");
});
it("forwards the filter options on every mental-model page", async () => {
fetchSpy.mockResolvedValueOnce(page(itemsOf(1), 1));
await client.listAllMentalModels("bank-a", { tags: ["work"], detail: "metadata" });
const url = String(fetchSpy.mock.calls[0][0]);
expect(url).toContain("tags=work");
expect(url).toContain("detail=metadata");
expect(url).toContain("limit=1000");
});
it("stops on an empty page even when the total disagrees", async () => {
// Rows deleted mid-page would otherwise leave the loop asking forever.
fetchSpy.mockResolvedValueOnce(page(itemsOf(2), 99)).mockResolvedValueOnce(page([], 99));
const all = await client.listAllMentalModels("bank-a");
expect(all).toHaveLength(2);
expect(fetchSpy).toHaveBeenCalledTimes(2);
});
it("pages directives to the reported total", async () => {
fetchSpy
.mockResolvedValueOnce(page(itemsOf(1000), 1200))
.mockResolvedValueOnce(page(itemsOf(200, 1000), 1200));
const all = await client.listAllDirectives("bank-a");
expect(all).toHaveLength(1200);
expect(fetchSpy).toHaveBeenCalledTimes(2);
});
});
hindsight-docs/static/openapi.json
+38 -2
View File
@@ -9221,11 +9221,29 @@
},
"type": "array",
"title": "Items"
},
"total": {
"type": "integer",
"title": "Total",
"description": "Total number of directives matching the filter (not just this page)"
},
"limit": {
"type": "integer",
"title": "Limit",
"description": "Page size that was applied"
},
"offset": {
"type": "integer",
"title": "Offset",
"description": "Offset that was applied"
}
},
"type": "object",
"required": [
"items"
"items",
"total",
"limit",
"offset"
],
"title": "DirectiveListResponse",
"description": "Response model for listing directives."
@@ -11998,11 +12016,29 @@
},
"type": "array",
"title": "Items"
},
"total": {
"type": "integer",
"title": "Total",
"description": "Total number of mental models matching the filter (not just this page)"
},
"limit": {
"type": "integer",
"title": "Limit",
"description": "Page size that was applied"
},
"offset": {
"type": "integer",
"title": "Offset",
"description": "Offset that was applied"
}
},
"type": "object",
"required": [
"items"
"items",
"total",
"limit",
"offset"
],
"title": "MentalModelListResponse",
"description": "Response model for listing mental models."
skills/hindsight-docs/references/openapi.json
+38 -2
View File
@@ -9221,11 +9221,29 @@
},
"type": "array",
"title": "Items"
},
"total": {
"type": "integer",
"title": "Total",
"description": "Total number of directives matching the filter (not just this page)"
},
"limit": {
"type": "integer",
"title": "Limit",
"description": "Page size that was applied"
},
"offset": {
"type": "integer",
"title": "Offset",
"description": "Offset that was applied"
}
},
"type": "object",
"required": [
"items"
"items",
"total",
"limit",
"offset"
],
"title": "DirectiveListResponse",
"description": "Response model for listing directives."
@@ -11998,11 +12016,29 @@
},
"type": "array",
"title": "Items"
},
"total": {
"type": "integer",
"title": "Total",
"description": "Total number of mental models matching the filter (not just this page)"
},
"limit": {
"type": "integer",
"title": "Limit",
"description": "Page size that was applied"
},
"offset": {
"type": "integer",
"title": "Offset",
"description": "Offset that was applied"
}
},
"type": "object",
"required": [
"items"
"items",
"total",
"limit",
"offset"
],
"title": "MentalModelListResponse",
"description": "Response model for listing mental models."