HindsightEmbedded forwarded every LLM/daemon setting on every construction, using
placeholder defaults for the ones the caller never mentioned. The embed manager
merges the caller's config over the profile .env and copies any non-None
HINDSIGHT_* entry into the daemon environment, so a client built without
credentials overwrote a key inherited from the profile or the parent shell --
and _register_profile then persisted the placeholders back into the profile's
.env file, leaving a profile configured for anthropic recorded as groq on disk.
llm_provider, llm_api_key, llm_model, log_level and idle_timeout now default to
None and are omitted when not passed, so the daemon resolves them from the
profile .env, then the parent environment, then its own defaults. An explicit
empty string remains an override, which is how a local LLM service with no
authentication clears an inherited key.
Knowledge pages and their backing mental models share one lifecycle, but
they were committed in separate transactions. A page insert failure could
therefore leave an orphaned mental model.
Extract mental-model embedding generation and insertion into typed helpers.
Create the bank, mental model, and page through one connection and transaction,
so any page failure rolls back all related writes. Preserve the duplicate-page
contract by returning None only for uq_kp_folder_pagename.
Add PostgreSQL and Oracle regression coverage for rollback after a real mental
model insert, plus parent-validation and duplicate-name coverage.
Keep bank configuration examples limited to fields that can actually be
overridden per bank, and replace the static LLM setting examples in the
endpoint descriptions.
Complete the operations and version response examples with their required
fields, and use the real refresh_mental_model operation type.
Regenerate the OpenAPI documentation, docs skill mirror, and generated client
descriptions.
Publish the Hindsight MIT license and generated third-party license notices
alongside the Rust CLI release assets. Generate the manifest from Cargo.lock
with cargo-about and preserve the complete dependency license text.
Run generation and verification in the Rust CLI PR test job so unsupported
license expressions fail before a release tag. Generate the platform-independent
manifest once in the Linux amd64 release job and fail if required assets are
missing before publishing.
Disable ClearlyDefined lookups to keep CI generation deterministic.
Fixes#3446
Explain how tags, tags_match, tag_groups, and directive isolation
interact across REST, MCP, versioned docs, and agent skills.
Clarify the different defaults used by reflect and directive listing,
then regenerate OpenAPI and supported client artifacts.
Show errors from document export, transfer import, and file uploads
that bypass the shared API error interceptor.
Format API errors safely so structured validation details remain
readable.
Raise a dedicated persistence conflict when a validated bank config
update can no longer find its bank row.
Map this race to HTTP 409 for template imports while preserving PATCH
404 behavior and 400 responses for invalid configuration. Add an
event-coordinated regression test for the deletion window.
Keep metadata and tags on unchanged memory units aligned with their
document during delta retain.
Cover metadata-only replace and append paths with regression tests.
Closes#3008
Default recalled memories to user context so dynamic recall content no
longer invalidates the stable system prompt prefix on every turn.
Keep explicit prepend and append settings unchanged. Align the manifest,
docs, and tests with the cache-friendly default.
Closes#3061
Stage the canonical repository license in each isolated Python build
context so wheels and source distributions include the MIT text.
Declare SPDX license metadata and verify every release artifact before
publishing to prevent repository firewalls from quarantining packages.
Closes#3054
Allow MarkItDown OCR clients to receive operator-defined default
headers for proxy routing and request tracing.
Wire the JSON environment setting through parser construction, document
the option, and cover configured and unset behavior.
Apply the exact tag filter even when the requested tag list is empty.
This keeps mental-model retrieval aligned with facts and observations.
Add a regression test that verifies the generated query selects only the
untagged global scope.
* chore(api): remove unused code
Remove confirmed unreferenced helpers from the API and engine.
Delete tests only where they cover superseded internal paths. Keep
active test helpers and public memory operations unchanged.
* chore(cli): remove unused code
Remove dead CLI configuration, client, and output helpers.
Drop the parser implementation and tests used only by the retired
output path.
* chore(control-plane): remove unused code
Remove unused ControlPlaneClient methods and unreachable directive
detail state from the think view.
* chore(dev): remove unused code
Remove unreferenced benchmark and repository maintenance helpers.
* chore(embed): remove unused code
Remove the unused daemon port lookup helper while preserving current
profile-based daemon discovery.
* chore(integrations): remove unused code
Remove unreferenced helpers across supported integrations.
Drop tests only for retired internal paths and retain active test and
lifecycle infrastructure.
* test(consolidation): port prompt regression tests to split builders
The dead-code cleanup removed build_batch_consolidation_prompt and its tests,
but those tests guarded behaviors that are still live in the current
build_consolidation_system_prompt / build_consolidation_input path:
- brace-safety of a mission / capacity note containing literal { } (a lone
brace would raise KeyError in the internal str.format() and crash
consolidation)
- output-language directive injection into the cached system prompt
- the built-in default mission when none is supplied
Re-add these as regression tests against the current builders instead of
dropping the coverage. Also fix a stale comment referencing the removed
utils.extract_facts module.
---------
Co-authored-by: Nicolò Boschi <[email protected]>
Require semantic-link thresholds to be passed explicitly to the
low-level ANN, within-batch, and batch-creation helpers.
Make the streaming final-ANN threshold keyword-only to prevent
positional argument mistakes, and rename the forwarding test to match
what it verifies.
* fix(graph): queue edited and restored memories for relinking
Graph maintenance rebuilds outgoing temporal and semantic links only for
units explicitly present in its queue. Edits and restores submitted the
worker without queuing the affected unit, so its outgoing links could
remain missing.
Queue edited units together with incoming-link victims in one sorted
insert to preserve the global lock order. Queue restored units after
their searchable fields have been rebuilt.
Cover outgoing-only restore and bidirectional edit cases, including a
single queue write for the edited unit and its victims.
Fixes#2889.
* test(graph): cover the outgoing-only relink case; tidy enqueue helper
The PR's tests only exercised mutually linked units, so the branch the bug
actually lived in — an edited/reverted unit with outgoing links but no
incoming ones, where the victim lookup is empty — was untested.
Tests:
- enqueue_relink_victims: include_affected_units with no victims (returns
the unit itself), with victims (one combined sorted insert), and the
default opt-out for delete callers.
- Curation: an outgoing-only edit queues itself, plus two end-to-end tests
that let the inline SyncTaskBackend drain the queue and assert the
temporal link is actually rebuilt after an edit and after a revert.
All five fail on the pre-fix engine.
Tidy:
- Rename deleted_unit_ids -> affected_unit_ids; with the new flag the
helper also takes units that stay live, so the old name/doc misled at
the edit call site. Same for the debug log wording.
- Spell out at both call sites why the edit combines self+victims in one
insert, why the invalidating edit opts out, and that revert rebuilds
only the reverted unit's outgoing links.
---------
Co-authored-by: Nicolò Boschi <[email protected]>
* fix(config): validate bank config updates before creating banks
Route external bank configuration writes through MemoryEngine so tenant
authentication and UPDATE_BANK_CONFIG authorization happen consistently.
Validate profile and configuration changes before creating a bank or
persisting either one. Rejected configuration updates through PUT,
PATCH, import, and MCP therefore leave no empty bank or partial profile
changes behind.
Keep memory-defense validation behavior unchanged, and cover the new
ordering and delegation paths with regression tests.
* fix(import): preflight template operations before creating banks
Preflight every template operation before creating a missing bank.
Reject duplicate mental models and directives before applying changes.
Reuse request-local authorization decisions while the import executes,
avoiding duplicate hook calls that may reserve quota or depend on time.
Precheck mental-model refresh availability so common failures do not
leave a newly created bank or a partially applied template behind.
Document that the authorization context creates the bank after all
checks pass.
* fix(mcp): create banks through public engine APIs
Delegate MCP bank creation to MemoryEngine's public profile and update
APIs instead of calling _ensure_bank_exists() directly.
Use get_bank_profile() for default creation and update_bank() when name
or mission fields are supplied. This keeps lifecycle validation and
authorization ordering inside the engine and avoids duplicate reads.
Add coverage for both public API paths and assert that MCP never invokes
the private creation helper.
* fix(config): fail loudly when persisting config for a missing bank
Bank creation moved out of ConfigResolver into MemoryEngine, but the
persist step still returned normally when the UPDATE matched zero rows.
A caller that skipped provisioning silently discarded its overrides
while reporting success — the failure mode #1940 originally fixed.
Raise instead, and translate the concurrent-delete case in update_bank's
update-only path into the same 404 its final profile read would produce.
* test(mcp): assert update_bank calls instead of a fixture's forwarding
The mock_memory fixture re-implemented _do_update_bank's routing by
forwarding config_updates to _config_resolver.update_bank_config, so the
existing assertions verified the fake rather than production code — they
would still pass if _do_update_bank stopped sending config entirely.
Assert on the update_bank mock, which is the call the tool now makes.
* test(api): cover the 404 mapping for a delete racing the config write
---------
Co-authored-by: Nicolò Boschi <[email protected]>
Expose graph seed, temporal semantic, and semantic-link similarity
thresholds through HindsightConfig while preserving existing defaults.
Wire the settings through retrieval, retain, streaming, and graph
maintenance paths. Add validation, environment examples, documentation,
and regression coverage.
Document how to calibrate all five embedding-dependent thresholds and
note that semantic-link changes do not rebuild existing graphs.
* fix(cli): pass tag filters to list memories
OpenAPI added tags and tags_match to list_memories in #2848, but the
CLI wrapper still passed the previous positional arguments. Generated
Rust client builds then failed with E0061.
Pass None for both filters to preserve existing CLI behavior and match
the generated method signature.
* feat(cli): expose terminal operation deletion
OpenAPI added delete_operation in #2777 without exposing it through
the Rust CLI or accounting for it in the coverage manifest. The CLI
coverage check therefore rejected branches rebased onto that change.
Add operation delete with confirmation and --yes support. Pass the
request through the generated client and cover command parsing. This
counts the endpoint as implemented without a coverage exception.
Run the existing build-docs job for every PR so the production docs
build remains an unconditional check.
Generate OpenAPI directly in verify-generated-files to avoid rebuilding
the Docusaurus site serially in that job.
Graph retrieval always selects its own bounded semantic seeds.
Remove the unused semantic_seeds and temporal_seeds inputs from
the graph retriever interface and link-expansion implementation.
The recall orchestrator no longer passes placeholder None values.
Document why graph seeds stay independent: the semantic and
temporal retrieval arms use different candidate limits and thresholds,
so reusing them would silently change graph recall behavior.
Add a regression assertion that the graph call contains no removed
seed inputs.
Add a precise operation-validator hook for bank creation, with a
no-op default so deployments without custom validators keep existing
behavior.
Route lazy bank creation through the hook from retain, imports, MCP
create_bank, and the default get_bank_profile auto-create path. This
keeps create-bank authorization separate from bank-scoped write
validation, which often assumes the target bank already exists.
Add regression coverage for rejected creation, existing-bank skips,
HTTP create/import paths, async retain, profile auto-create, and MCP
create_bank.
Treat the get_bank MCP tool as read-only by looking up bank profiles
without auto-creation in both single-bank and multi-bank modes.
Add regression coverage for missing banks so get_bank returns a
not-found error instead of creating the bank.
Link expansion ranks candidates by an additive entity, semantic,
and causal score, but returned the raw score from one signal as
activation. Cross-fact-type graph merging then re-ranked candidates
using that raw value.
Store the final additive score as activation and add a regression test
for cross-fact-type ordering.
Clarify that retain creates caused_by only. Storage and recall keep reading
historical causal link types, and transfer import alone restores them.
Correct stale code comments and tests, and preserve legacy edge types and
endpoints during transfer without widening the retain write contract.
Apply each entity fanout cap only after filtering candidates by fact type.
This prevents high-volume fact types from excluding valid target candidates.
Cover the PostgreSQL and Oracle CTE builders with a regression test.
Keep recall min_scores.semantic scoped to the semantic retrieval arm.
Temporal retrieval uses embeddings only to choose time-window entry
points. Reusing the request-level semantic floor there made temporal
recall unexpectedly narrower.
Callers that only wanted to prune weak semantic matches could also
narrow temporal recall. That made the min_scores contract surprising
and inconsistent with graph seed selection.
Use the temporal entry-point default instead. Semantic and BM25 request
floors remain unchanged.
The OpenCode plugin imports @opencode-ai/plugin/tool from its
built dist entrypoint, so the package must be present in
OpenCode's isolated plugin cache. Keeping it only as a peer
dependency lets npm skip it during plugin installation, which can
make the plugin fail to load with ERR_MODULE_NOT_FOUND.
Move @opencode-ai/plugin into dependencies and remove the
peer-only declaration. Keep @vectorize-io/hindsight-client as a
runtime dependency, and sync package-lock.json so npm installs the
cache tree needed by OpenCode.
Verified with npm run build, npm pack --json, and a local opencode
plugin install from the generated tarball. The generated cache
contained both runtime dependencies and direct import of the
plugin entrypoint succeeded.
The list and get memory-unit paths selected tags and timing fields
but skipped the memory_units metadata column, so metadata retained
on facts was invisible outside recall.
Select and serialize metadata for live and invalidated memory units,
add a curation regression test for both paths, and update docs plus
OpenAPI examples.
Update generated hindsight-docs skill references with Requesty provider
entries that are already present in the source documentation.
This keeps the generated skill bundle in sync with the docs generator so
pre-commit no longer rewrites these files.
Add PrecheckOperation, BankReadOperation, and BankWriteOperation
StrEnum types for operation validator hook contexts. Use them at
every precheck and validate_bank_read/write call site while
preserving string comparison compatibility for existing extensions.
Tests:
- uv run pytest tests/test_extensions.py -q
- ./scripts/hooks/lint.sh
Remove accidentally committed Playwright MCP logs, page snapshots, and
root-level screenshot artifacts.
Ignore future Playwright MCP output so local browser debugging does not
show up as repository changes.
Treat PATCH /v1/default/banks/{bank_id} as update-only by using a
non-creating bank profile lookup and returning 404 when the bank is
missing.
Add a regression test proving the endpoint does not create a bank as a
side effect.
Use the non-creating bank-profile lookup when dry-run extraction
resolves the optional narrator name. A preview endpoint promises no
persistence, so probing a missing bank must not insert a bank row.
Add a regression test that calls dry-run extraction against a missing
bank and verifies the bank still does not exist afterwards.
Run the pre-commit uv sync and workspace uv run commands with
--frozen so linting uses the checked-in lockfile without rewriting it
during ordinary code changes.
This avoids local uv resolver freshness checks producing unrelated
uv.lock diffs while preserving explicit dependency update workflows.
Restore HINDSIGHT_API_MCP_INSTRUCTIONS for HTTP MCP servers.
Append the extra guidance only to retain and recall tool
descriptions, matching the original local MCP behavior without
changing reflect or other management tools.
MarkItDown advertises image extensions, but without OCR config it can
fail screenshots or scanned images with low-level no-content errors.
Add server-level MarkItDown OCR config that is off by default and
independent from HINDSIGHT_API_LLM_*. When OCR is enabled, the OCR API
key, base URL, and model are required explicitly.
Wire those settings into MarkItDown's llm_client support with a built-in
OCR prompt. Image uploads now fail fast with actionable errors when OCR
is disabled or required settings are missing.
Docs and front-end copy explain that image OCR depends on server config
and requires an OpenAI-compatible OCR/vision endpoint.
Closes#927
Move explicit period extraction out of DateparserQueryAnalyzer. The analyzer
now delegates range parsing to temporal_periods, which keeps the public API and
non-Chinese rules while Chinese-specific rules and boundary handling live in
chinese_temporal_periods.
Handle simplified and traditional Chinese expressions for relative days,
weeks, months, years, weekends, half-year periods, Chinese month names,
quarters, and rolling past/future windows.
Separate precise point expressions from fuzzy range expressions so phrases such
as 两天前, 前两天, 几天前, and 一两周前 map to the intended constraint shape
instead of relying on dateparser fallback behavior.
Keep open future starts such as 明天起, 下周起, and 三天后开始 unconstrained
because the API only represents closed ranges.
Guard Chinese matching so non-CJK queries skip the Chinese regex path, while
Chinese substring checks avoid treating ordinary names and words as temporal
constraints.
Add retain_structured_chunk_size as an explicit retain chunking knob
for structured inputs. When unset, structured inputs follow the
effective retain_chunk_size instead of the hidden 1.5x overflow factor.
Thread the setting through retain extraction, append/prepend chunking,
bank config resolution, templates, MCP docs, maintained clients,
generated OpenAPI artifacts, the control-plane retain strategy UI, and
the Rust CLI set-config command.
Validate retain_chunk_size and retain_structured_chunk_size as positive
integers while allowing either value to be smaller. Keep the existing
retain_max_completion_tokens check scoped to retain_chunk_size.
Preserve upstream validation details for client errors through the
control-plane proxy so UI alerts and toasts can show concrete
configuration errors without exposing server-side failure details.
Update chunking, config, hierarchical config, template, MCP, client
payload, control-plane serialization, SDK-response, API-client, and
retain UI validation tests for the new behavior.
Remove unreferenced backend helpers, stale UI/docs components, and
unused imports across the API, control plane, clients, and integrations.
Drop obsolete consolidated-observation helpers and unused scoring code,
clean orphaned React/docs components, and remove stale Radix dependencies.
Align release scripts, Helm docs, lockfiles, generated clients, and
current API examples with the package and endpoint surface still in use.
Daemon mode previously inferred whether --host or --port was supplied by
comparing parsed values with the loaded config. If a CLI value matched an
env-derived default, such as HINDSIGHT_API_PORT=9555 with --port 9555,
the daemon treated the port as implicit and fell back to
DEFAULT_DAEMON_PORT.
Track explicit host/port through argparse itself using SUPPRESS defaults,
so argparse-accepted long-option abbreviations such as --po and --ho
follow the same path. Return a named dataclass from the resolver and cover
the daemon parsing edge cases in tests.
Fixes#1786.
Output of ./scripts/generate-docs-skill.sh - picks up the API
version bump (0.7.0 -> 0.7.1) in openapi.json. CI's
verify-generated-files gate flags this as out-of-sync on every new
branch off main; this commit clears the gate without affecting API
behaviour.
Also folds in the ./scripts/hooks/lint.sh formatter output for the
priority parser so the lint hook stays clean.
Use recall question_date/query_timestamp as the reference time for combined
scoring instead of always using server utcnow(). This keeps historical replay
and offline evaluations from penalizing memories that were recent at query
time.
Normalize naive query timestamps to UTC before scoring, update
API/client/OpenAPI/docs/MCP descriptions, and add recall-level coverage proving
combined scoring receives the query-time anchor.
Sanderhoff-alt