Nicolò Boschi
31c1aaf213
The semantic arm asked the index for `max(limit * 5, 100)` rows, but every pool connection runs with a fixed `hnsw.ef_search = 200`. In pgvector the candidate list is the result set — the ground-layer search runs once and the scan ends when that list drains — so a scan returned at most ~200 rows however large the LIMIT above it, and the recall budget moved the SQL and nothing else (low/mid/high all got ~200). Enable hnsw.iterative_scan (strict_order) on recall connections. The drained list is refilled in ef_search-sized rounds until the query's LIMIT is met, so depth follows each query's budget with no per-query statement — which matters behind a transaction-mode pooler, where a session GUC issued between statements can land on a different backend. Retain-side link probing pins it off; it is tuned for latency, not depth. Measured on a 40k-row bank, EXPLAIN confirming an ANN index scan: MID unfiltered 200 -> 300 rows, 5.0 -> 4.9ms HIGH unfiltered 200 -> 1000 rows, 4.9 -> 6.8ms HIGH + filter 200 -> 324 rows, 5.9 -> 14.2ms +8ms worst case against a ~2.6s recall; the perf suite puts it at +1.3% mean latency / -1.3% throughput end to end. Memory is not the constraint it looks like: pgvector caps a resumed scan at work_mem * hnsw.scan_mem_multiplier, but max_scan_tuples binds first — squeezing work_mem from 64MB to 256kB changes neither rows nor latency. Two controls, both static server-level config: - HINDSIGHT_API_ANN_ITERATIVE_SCAN (default true) — the kill switch. False drops the resume GUCs rather than sending iterative_scan=off, so a connection is left exactly as it was before this existed and the revert lands on the behaviour already in production. - HINDSIGHT_API_ANN_MAX_SCAN_TUPLES (default 4000, pgvector's own is 20000) — the dial that governs the cost. The initial scan is not counted, so even 1 leaves the pre-existing depth intact; it interpolates rather than switches. Separately, the row over-fetch is deleted rather than tuned. It never did anything: each arm's rows arrive already ordered by distance, so keeping the first N of 5N returns precisely what LIMIT N would have. Invisible on pgvector, real on backends with no such bound. The LIMIT is now max(limit, GRAPH_SEED_LIMIT), since the graph arm reads its entry points from the same rows. Also fixed: a GUC the server rejects as unknown is remembered and dropped from later batches, instead of costing a failed batch plus one statement per setting on every acquire — reachable via pg_trgm on a cluster without it, and via hnsw.iterative_scan on a pgvector older than 0.8, which reserves the "hnsw." prefix and rejects it outright. Retain's link probing skips such a GUC too: it applies these with SET LOCAL inside its own transaction, where an erroring statement would abort the link computation. Not measured: whether the extra candidates improve answers. Everything above is cost.
hindsight-embed
Hindsight embedded CLI - local memory operations with automatic daemon management.
This package provides a simple CLI for storing and recalling memories using Hindsight's memory engine. It automatically manages a background daemon for fast operations - no manual server setup required.
How It Works
hindsight-embed uses a background daemon architecture for optimal performance:
- First command: Automatically starts a local daemon (first run downloads dependencies and loads ML models - can take 1-3 minutes)
- Subsequent commands: Near-instant responses (~1-2s) since daemon is already running
- Auto-shutdown: Daemon automatically exits after 5 minutes of inactivity
The daemon runs on localhost:8888 and uses an embedded PostgreSQL database (pg0) - everything stays local on your machine.
Installation
pip install hindsight-embed
# or with uvx (no install needed)
uvx hindsight-embed --help
Quick Start
# Interactive setup (configures default profile)
hindsight-embed configure
# Or set your LLM API key manually
export OPENAI_API_KEY=sk-...
# Store a memory (bank_id = "default")
hindsight-embed memory retain default "User prefers dark mode"
# Recall memories
hindsight-embed memory recall default "What are user preferences?"
All commands use the "default" profile unless you specify a different one with --profile or HINDSIGHT_EMBED_PROFILE.
Commands
configure
Configure the default profile or create/update named profiles:
# Interactive setup for default profile
hindsight-embed configure
# Create/update named profile with single command
hindsight-embed configure --profile my-app \
--env HINDSIGHT_API_LLM_PROVIDER=openai \
--env HINDSIGHT_API_LLM_API_KEY=sk-xxx
# Create/update named profile interactively
hindsight-embed configure --profile staging
This will:
- Let you choose an LLM provider (OpenAI, Groq, Google, Ollama)
- Configure your API key
- Set the model
- Start the daemon with your configuration
memory retain
Store a memory:
hindsight-embed memory retain default "User prefers dark mode"
hindsight-embed memory retain default "Meeting on Monday" --context work
hindsight-embed memory retain myproject "API uses JWT authentication"
memory recall
Search memories:
hindsight-embed memory recall default "user preferences"
hindsight-embed memory recall default "upcoming events"
Use -o json for JSON output:
hindsight-embed memory recall default "user preferences" -o json
memory reflect
Get contextual answers that synthesize multiple memories:
hindsight-embed memory reflect default "How should I set up the dev environment?"
bank list
List all memory banks:
hindsight-embed bank list
profile
Manage configuration profiles:
# List all profiles with status
hindsight-embed profile list
# Show current active profile
hindsight-embed profile show
# Set active profile (persists across commands)
hindsight-embed profile set-active my-app
# Clear active profile (revert to default)
hindsight-embed profile set-active --none
# Delete a profile
hindsight-embed profile delete my-app
daemon
Manage the background daemon:
hindsight-embed daemon status # Check if daemon is running
hindsight-embed daemon start # Start the daemon
hindsight-embed daemon stop # Stop the daemon
hindsight-embed daemon logs # View last 50 lines of logs
hindsight-embed daemon logs -f # Follow logs in real-time
hindsight-embed daemon logs -n 100 # View last 100 lines
Configuration
Interactive Setup
Run hindsight-embed configure for a guided setup that saves to ~/.hindsight/embed.
Environment Variables
| Variable | Description | Default |
|---|---|---|
HINDSIGHT_EMBED_PROFILE |
Profile name to use (overrides active profile) | None (uses default profile) |
HINDSIGHT_API_LLM_API_KEY |
LLM API key (or use OPENAI_API_KEY); required only when the selected provider uses an API key |
Provider-dependent |
HINDSIGHT_API_LLM_PROVIDER |
LLM provider (openai, groq, google, ollama) |
openai |
HINDSIGHT_API_LLM_MODEL |
LLM model | gpt-4o-mini |
HINDSIGHT_EMBED_API_URL |
Use external API server instead of starting local daemon | None (starts local daemon) |
HINDSIGHT_EMBED_API_TOKEN |
Authentication token for external API (sent as Bearer token) | None |
HINDSIGHT_EMBED_API_DATABASE_URL |
Database URL for daemon | pg0://hindsight-embed |
HINDSIGHT_EMBED_DAEMON_IDLE_TIMEOUT |
Seconds before daemon auto-exits when idle | 300 |
HINDSIGHT_EMBED_DAEMON_LOG_MAX_BYTES |
Rotate the daemon log at startup at this size; 0 disables rotation |
10485760 (10 MiB) |
HINDSIGHT_EMBED_DAEMON_LOG_BACKUP_COUNT |
Retained backups; 0 truncates a full log at startup |
3 |
The size is checked only when a daemon starts, so a single uninterrupted run is never truncated and can
grow past MAX_BYTES — and at the next start that whole file is kept as the first backup. Retained size
is therefore around MAX_BYTES × (BACKUP_COUNT + 1) (40 MiB by default) only for daemons that restart
regularly; a daemon left running for weeks keeps whatever it wrote. Restart it, or lower
HINDSIGHT_EMBED_DAEMON_IDLE_TIMEOUT, to keep the bound meaningful.
Using an External API Server:
To connect to an existing Hindsight API server instead of starting the local daemon:
export HINDSIGHT_EMBED_API_URL=http://your-server:8000
export HINDSIGHT_EMBED_API_TOKEN=your-api-token # Optional, if API requires auth
hindsight-embed memory recall default "query"
Custom Database:
To use an external PostgreSQL database instead of the embedded pg0 database (useful when running as root or in containerized environments):
export HINDSIGHT_EMBED_API_DATABASE_URL=postgresql://user:password@localhost:5432/dbname
hindsight-embed daemon start
Note: All banks share a single database. Bank isolation happens within the database via the bank_id parameter passed to CLI commands.
Configuration Profiles
Profiles let you maintain multiple independent configurations (e.g., different API endpoints, LLM providers, or projects). Each profile runs its own daemon on a unique port (8889-9888).
The Default Profile:
When you run hindsight-embed configure without specifying a profile, it configures the "default" profile. This uses the backward-compatible configuration at ~/.hindsight/embed and runs on port 8888.
Creating Named Profiles:
# Create a profile with single command
hindsight-embed configure --profile my-app \
--env HINDSIGHT_API_LLM_PROVIDER=openai \
--env HINDSIGHT_API_LLM_API_KEY=sk-xxx \
--env HINDSIGHT_API_LLM_MODEL=gpt-4o-mini
# Create a profile interactively
hindsight-embed configure --profile staging
Using Profiles:
# Option 1: Environment variable (recommended for apps)
HINDSIGHT_EMBED_PROFILE=my-app hindsight-embed memory retain default "text"
# Option 2: CLI flag
hindsight-embed --profile my-app memory recall default "query"
# Option 3: Set as active (persists across commands)
hindsight-embed profile set-active my-app
hindsight-embed memory recall default "query" # Uses my-app profile
# Clear active profile (revert to default)
hindsight-embed profile set-active --none
Profile Management:
# List all profiles with status
hindsight-embed profile list
# Show active profile
hindsight-embed profile show
# Delete a profile
hindsight-embed profile delete my-app
Profile Resolution Priority:
HINDSIGHT_EMBED_PROFILEenvironment variable (highest)--profileCLI flag- Active profile from
~/.hindsight/active_profilefile - Default profile (lowest)
Note: If a profile is specified but doesn't exist, the command will fail with an error. Profiles must be explicitly created using hindsight-embed configure --profile <name>.
Files
Default Profile:
| Path | Description |
|---|---|
~/.hindsight/embed |
Configuration file for default profile |
~/.hindsight/daemon.log |
Daemon logs for default profile |
~/.hindsight/daemon.lock |
Daemon lock file (PID) for default profile |
Named Profiles:
| Path | Description |
|---|---|
~/.hindsight/profiles/<name>.env |
Configuration file for profile |
~/.hindsight/profiles/<name>.log |
Daemon logs for profile |
~/.hindsight/profiles/<name>.lock |
Daemon lock file (PID) for profile |
~/.hindsight/profiles/metadata.json |
Profile metadata (ports, timestamps) |
~/.hindsight/active_profile |
Active profile name (when set with profile set-active) |
Use with AI Coding Assistants
This CLI is designed to work with AI coding assistants like Claude Code, Cursor, and Windsurf. Install the Hindsight skill:
curl -fsSL https://hindsight.vectorize.io/get-skill | bash
This will configure the LLM provider and install the skill to your assistant's skills directory.
Troubleshooting
Daemon won't start:
# Check logs for errors
hindsight-embed daemon logs
# Stop any stuck daemon and restart
hindsight-embed daemon stop
hindsight-embed daemon start
Slow first command: This is expected - the first command needs to download dependencies, start the daemon, and load ML models. First run can take 1-3 minutes depending on network speed. Subsequent commands will be fast (~1-2s).
Change configuration:
# Re-run configure (automatically restarts daemon)
hindsight-embed configure
License
Apache 2.0