initial commit

This commit is contained in:
Nicolò Boschi
2025-10-30 12:53:12 +01:00
commit 6d7223b14e
45 changed files with 78065 additions and 0 deletions
+23
View File
@@ -0,0 +1,23 @@
# Python-generated files
__pycache__/
*.py[oc]
build/
dist/
wheels/
*.egg-info
# Virtual environments
.venv
# Environment variables
.env
.env.local
# IDE
.idea/
.vscode/
*.swp
*.swo
# NLTK data (will be downloaded automatically)
nltk_data/
+1
View File
@@ -0,0 +1 @@
3.11
+7
View File
@@ -0,0 +1,7 @@
# Documentation
Do not write any markdown file, just write the code.
# Workflow
After your changes, make sure everything is working fine by running the main script.
- keep the readme.md architecture section up to date when you change the implementation
- when changing an implemetation, do not keep the old one as fallback
+340
View File
@@ -0,0 +1,340 @@
# Entity-Aware Memory System for AI Agents
A proof-of-concept memory system that enables AI agents to store, retrieve, and connect memories using temporal, semantic, and entity-based relationships.
## Overview
This system implements a sophisticated graph-based memory architecture where memories are connected through three complementary networks:
1. **Temporal Network** - Memories linked by time proximity
2. **Semantic Network** - Memories linked by meaning similarity
3. **Entity Network** - Memories linked by shared entities (people, organizations, places)
The combination of these three networks enables powerful memory retrieval that goes beyond simple vector search, allowing agents to find relevant memories through multiple pathways.
## Architecture
### Core Concepts
**Memory Units**: Individual sentence-level memories that are:
- Self-contained (pronouns resolved to actual referents)
- Validated to have subject + verb (complete thoughts)
- Embedded as vectors for semantic similarity
- Timestamped for temporal relationships
- Linked to extracted entities
**Entity Resolution**: Named entities (PERSON, ORG, GPE, etc.) are:
- Extracted using spaCy NER
- Disambiguated using a scoring algorithm
- Tracked with canonical IDs across all memories
- Used to create strong connections between related memories
### Three Types of Memory Links
#### 1. Temporal Links (Time-Based)
**Purpose**: Connect memories that occurred close together in time
**How it works**:
- When storing a new memory, find all memories within a time window (default: 24 hours)
- Create weighted links based on temporal proximity
- Weight formula: `weight = max(0.3, 1.0 - (time_diff / window_size))`
- Closer in time = stronger link
**Visualization**: Cyan, dashed lines
**Use case**: "What happened recently?" or understanding sequences of events
#### 2. Semantic Links (Meaning-Based)
**Purpose**: Connect memories with similar content/meaning
**How it works**:
- Generate embeddings using local `bge-small-en-v1.5` model (384 dimensions)
- Store embeddings in PostgreSQL with pgvector extension
- When storing a new memory, find top-k similar memories using cosine similarity
- Create links only if similarity exceeds threshold (default: 0.7)
- Weight = cosine similarity score
**Visualization**: Pink, solid lines
**Technology**:
- **SentenceTransformers** - Local embedding model (BAAI/bge-small-en-v1.5)
- **pgvector** - PostgreSQL extension for vector operations
- **HNSW index** - Fast approximate nearest neighbor search
**Use case**: "Tell me about hiking" retrieves all semantically related outdoor activities
#### 3. Entity Links (Identity-Based)
**Purpose**: Connect ALL memories about the same person, organization, or place
**How it works**:
- Extract entities from text using spaCy NER
- Resolve entity identity using disambiguation algorithm:
- Name similarity (50% weight) - using SequenceMatcher
- Co-occurring entities (30% weight) - entities that appear together
- Temporal proximity (20% weight) - recent mentions more likely same entity
- If score > threshold (0.4 for PERSON with exact match, 0.6 otherwise): reuse existing entity
- If score < threshold: create new entity
- Link all memories mentioning the same entity with weight 1.0 (no decay)
**Visualization**: Gold, thick lines
**Technology**:
- **spaCy** (`en_core_web_sm`) - Named Entity Recognition
- **difflib.SequenceMatcher** - String similarity matching
**Use case**: "What does Alice do?" returns ALL memories about Alice (hiking, work at Google, Python project) even if semantically distant
**Critical advantage**: Solves the problem where "Alice loves hiking" wouldn't normally connect to "Alice works at Google" through semantic similarity alone.
### Spreading Activation Search
The search algorithm explores the memory graph using spreading activation:
1. **Entry Points**: Find top-3 semantically similar memories to the query (vector search)
2. **Activation Spreading**: Start with activation = 1.0 at entry points
3. **Graph Traversal**: Follow links to neighbors, spreading activation with decay (0.8 factor)
4. **Thinking Budget**: Limit exploration to N units (controls computational cost)
5. **Dynamic Weighting**: Combine activation with recency and frequency:
```
final_weight = activation × recency × frequency
recency = exp(-0.1 × days_since)
frequency = 1.0 + log(access_count + 1) / log(10)
```
6. **Return Top-K**: Sort by final weight and return top results
This approach ensures:
- Recently accessed memories get boosted (recency bias)
- Frequently accessed memories get boosted (importance signal)
- Graph structure influences results (not just vector similarity)
### Self-Contained Memory Units
Every memory unit is processed to be self-contained through coreference resolution:
**Problem**: "She joined Google last year" - unclear who "she" is
**Solution**: Fast batch coreference resolution that:
- Replaces personal pronouns (he, she, it, they) with actual referents
- Processes all sentences in one batch (O(n) instead of O(n²))
- Uses neural coreference model for high accuracy
- Provides fallback to custom spaCy-based resolution if needed
**Result**: "Alice joined Google last year" - fully self-contained
**Technology**:
- **FastCoref** - Fast, accurate neural coreference resolution
- Processes 2.8K documents in 25 seconds on GPU
- Significant speedup over sequential spaCy approach
- Fallback to custom spaCy implementation if needed
### LLM-Based Fact Extraction
Raw content is processed through an LLM to extract meaningful facts before storage:
**Problem**: Raw text contains noise (greetings, filler words, reactions) that waste storage and reduce retrieval quality
**Solution**: LLM-based extraction with optimized prompting:
- Filters out social pleasantries and non-informative content
- Extracts only facts with substance (biographical, events, opinions, recommendations, descriptions, relationships)
- Creates self-contained statements with subject+action+context
- Categorizes and attributes facts to speakers
**Technology**:
- **OpenAI-compatible API** - Supports Groq (default), OpenAI, and other providers
- **Structured output** - Uses Pydantic models for reliable fact extraction
- **Optimized prompting** - Concise prompts (~300 chars) emphasize dense output with no fluff
- **Automatic chunking** - Large documents (>120k chars) split at sentence boundaries
- **Fast sentence splitting** - Regex-based splitter (no heavy NLP models)
- **Progress tracking** - Logs chunk processing for transparency
**For large documents (e.g., podcast transcripts)**:
- Documents <120k chars: processed in one pass
- Documents >120k chars: automatically chunked at sentence boundaries
- Each chunk kept under ~30k tokens to avoid output token limits
- Facts aggregated across all chunks
### Technology Stack
**Database**:
- PostgreSQL 15+ with extensions:
- `pgvector` - Vector similarity operations
- `uuid-ossp` - UUID generation
**Python Libraries**:
- `psycopg2-binary` - PostgreSQL client
- `sentence-transformers` - Local embedding model (bge-small-en-v1.5)
- `torch` - Deep learning framework (for embeddings)
- `fastcoref` - Fast neural coreference resolution
- `spacy` - NLP (NER, dependency parsing, tokenization)
- `nltk` - Sentence tokenization
- `networkx` - Graph operations
- `pyvis` - Interactive HTML graph visualization
- `matplotlib` - Static graph visualization
- `rich` - Terminal UI
**Models**:
- BAAI/bge-small-en-v1.5 - Local embedding model (384 dimensions)
## Quick Start
### Prerequisites
1. PostgreSQL 15+ with pgvector extension
2. Python 3.11+
### Setup
1. Install dependencies:
```bash
uv sync
```
2. Install spaCy model:
```bash
uv pip install https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.7.1/en_core_web_sm-3.7.1-py3-none-any.whl
```
3. Create database and run schema:
```bash
psql -U postgres -c "CREATE DATABASE memory_poc"
psql -U postgres -d memory_poc -f schema.sql
```
4. Configure environment:
```bash
cp .env.example .env
# Edit .env with your DATABASE_URL
```
### Run Tests
Run the full test suite:
```bash
uv run pytest tests/ -v
```
Run specific test files:
```bash
uv run pytest tests/test_memory_operations.py -v
uv run pytest tests/test_entity_linking.py -v
```
Run a single test:
```bash
uv run pytest tests/test_memory_operations.py::test_put_creates_memory_units -v
```
### Run Demo
```bash
uv run python demos/demo_entity.py
```
This will:
1. Clear previous demo data
2. Store sample memories about Alice, Bob, Google, Yosemite
3. Search for "What does Alice do?"
4. Show entity resolution results
5. Generate interactive HTML graph visualization
Open `memory_graph_interactive.html` in your browser to explore the memory graph!
## Project Structure
```
memory-poc/
├── memory/ # Core memory system package
│ ├── temporal_semantic_memory.py # Main memory system class
│ ├── entity_resolver.py # Entity extraction and disambiguation
│ ├── coref_resolver.py # Coreference resolution
│ └── utils.py # Utility functions
├── demos/ # Demo scripts
│ └── demo_entity.py # Main entity-aware demo
├── visualizations/ # Visualization tools
│ └── interactive_graph.py # Interactive HTML graph (pyvis)
├── schema.sql # Database schema
├── pyproject.toml # Dependencies
└── README.md # This file
```
## Key Features
✅ **Three-layered linking**: Temporal + Semantic + Entity
✅ **Entity disambiguation**: Resolves "Alice" across different contexts
✅ **Self-contained units**: Pronouns resolved to actual referents
✅ **Spreading activation**: Graph-aware search beyond vector similarity
✅ **Interactive visualization**: Explore memory graph in browser
✅ **Recency & frequency weighting**: Recent and important memories boosted
✅ **Linguistic validation**: Memory units verified to have subject + verb
## API Usage
### Store Memories
```python
from memory import TemporalSemanticMemory
memory = TemporalSemanticMemory()
memory.put(
agent_id="agent_1",
content="Alice works at Google as a software engineer. She joined last year.",
context="Career discussion",
event_date=datetime.now(timezone.utc)
)
```
### Search Memories
```python
results = memory.search(
agent_id="agent_1",
query="What does Alice do?",
thinking_budget=50, # How many units to explore
top_k=10 # Number of results to return
)
for result in results:
print(f"{result['text']} (weight: {result['weight']:.3f})")
```
## How It Works: Example
**Input memories**:
1. "Alice loves hiking in the mountains" (7 days ago)
2. "She goes hiking every weekend in Yosemite" (7 days ago)
3. "Alice works at Google as a software engineer" (3 days ago)
4. "She joined Google last year" (3 days ago)
**Processing**:
1. ✅ Coreference resolution → "Alice goes hiking...", "Alice joined Google..."
2. ✅ Entity extraction → Identifies "Alice" (PERSON), "Google" (ORG), "Yosemite" (GPE)
3. ✅ Entity resolution → All "Alice" mentions = same person
4. ✅ Create links:
- Temporal: Memory 1 ↔ Memory 2 (same day)
- Semantic: "hiking" memories link together, "Google" memories link together
- Entity: ALL Alice memories strongly linked (weight 1.0)
**Query: "What does Alice do?"**
1. Vector search finds "Alice works at Google" as top entry point
2. Spreading activation follows entity links to find:
- "Alice joined Google..." (entity link: Alice)
- "Alice loves hiking..." (entity link: Alice)
- "Alice goes hiking..." (entity link: Alice)
3. Returns ALL Alice memories, properly ranked by relevance
## Why This Architecture?
**Problem with vector-only search**: "Alice loves hiking" and "Alice works at Google" are semantically distant - pure vector search might miss this connection.
**Solution**: Entity links ensure memories about the same person/place/organization are strongly connected regardless of semantic distance.
**Result**: More human-like memory retrieval that understands identity and relationships.
## License
MIT
+147
View File
@@ -0,0 +1,147 @@
# Benchmarks
This directory contains benchmark evaluations for the Entity-Aware Memory System.
## LoComo Benchmark
**Location**: `locomo/`
**Purpose**: Evaluate long-term conversational memory through Question Answering on multi-session conversations.
### Quick Start
1. **Run full benchmark** (10 conversations, ~2000 questions):
```bash
cd locomo
uv run python run_benchmark.py
```
2. **Run quick test** (1 conversation, 10 questions):
```bash
cd locomo
uv run python run_benchmark.py --max-conversations 1 --max-questions 10
```
3. **View results**:
- Detailed report: `locomo/RESULTS.md`
- Raw data: `locomo/benchmark_results.json`
### Dataset
- **Source**: [Snap Research LoComo](https://github.com/snap-research/locomo)
- **File**: `locomo10.json` (10 conversations)
- **Size**: Each conversation has ~300 turns over ~35 sessions spanning several months
- **Tasks**: Question Answering with 3 reasoning types (single-hop, temporal, multi-hop)
### Methodology
1. **Ingest** each conversation turn-by-turn with timestamps
2. **Apply** coreference resolution and entity extraction
3. **Create** temporal, semantic, and entity links
4. **Answer** questions using spreading activation search
5. **Evaluate** using LLM-as-judge (GPT-4o-mini)
### Expected Performance
Based on published results:
- **Human**: ~95%
- **Letta (GPT-4o-mini)**: 74.0%
- **Mem0 Graph**: 68.5%
- **Our target**: 65-75% (competitive with state-of-the-art)
### Computational Cost
**Per conversation** (~300 turns):
- ~300 embedding API calls (ingestion)
- ~200 embedding API calls (queries)
- ~200 LLM API calls (answer generation)
- ~200 LLM API calls (judgment)
**Estimated runtime**: 2-5 minutes per conversation (API-dependent)
**Estimated cost**: $0.50-1.00 per conversation (OpenAI pricing)
## LongMemEval Benchmark
**Location**: `longmemeval/`
**Purpose**: Evaluate five core long-term interactive memory abilities: information extraction, multi-session reasoning, temporal reasoning, knowledge updates, and abstention.
### Quick Start
1. **Download dataset**:
```bash
cd longmemeval
curl -L "https://huggingface.co/datasets/xiaowu0162/longmemeval-cleaned/resolve/main/longmemeval_s_cleaned.json" -o longmemeval_s_cleaned.json
```
2. **Run full benchmark** (500 questions):
```bash
cd longmemeval
uv run python run_benchmark.py
```
3. **Run quick test** (5 instances):
```bash
cd longmemeval
uv run python run_benchmark.py --max-instances 5
```
4. **View results**:
- Raw data: `longmemeval/benchmark_results.json`
### Dataset
- **Source**: [LongMemEval (ICLR 2025)](https://github.com/xiaowu0162/LongMemEval)
- **File**: `longmemeval_s_cleaned.json` (500 instances)
- **Size**: ~40 sessions per instance (~115k tokens)
- **Tasks**: 5 memory abilities across different question types
### Methodology
1. **Ingest** multi-session conversations with timestamps
2. **Apply** coreference resolution and entity extraction
3. **Create** temporal, semantic, and entity links
4. **Retrieve** relevant memories using spreading activation
5. **Generate** answers using GPT-4o-mini
6. **Evaluate** using GPT-4o as judge
### Expected Performance
Based on published results:
- **Human**: ~95%
- **Zep**: 75.2%
- **Letta (GPT-4o-mini)**: 74.0%
- **Mem0 Graph**: 68.5%
- **Our target**: 65-75% (competitive with state-of-the-art)
### Computational Cost
**Full benchmark** (500 instances):
- Embeddings: Free (local model)
- Answer generation: 500 × GPT-4o-mini calls
- Evaluation: 500 × GPT-4o calls
- **Estimated runtime**: 2-4 hours
- **Estimated cost**: $50-80 (OpenAI API)
## Future Benchmarks
- **MemGPT Tasks**: Long-context question answering
- **Custom Temporal Reasoning**: Time-based memory retrieval
- **Entity-Centric Queries**: Testing entity link effectiveness
## Adding New Benchmarks
1. Create a new directory: `benchmarks/{benchmark_name}/`
2. Add dataset: `benchmarks/{benchmark_name}/data/`
3. Implement adapter: `benchmarks/{benchmark_name}/run_benchmark.py`
4. Document results: `benchmarks/{benchmark_name}/RESULTS.md`
## Results Summary
| Benchmark | Metric | Our System | Best Published | Status |
|-----------|--------|------------|----------------|--------|
| LoComo QA | Accuracy | {TBD}% | 74.0% (Letta) | In Progress |
| LongMemEval | Accuracy | {TBD}% | 75.2% (Zep) | Ready to Run |
*Last updated: 2025-10-30*
+49
View File
@@ -0,0 +1,49 @@
{
"overall_accuracy": 33.33333333333333,
"total_correct": 1,
"total_questions": 3,
"conversation_results": [
{
"sample_id": "conv-26",
"metrics": {
"accuracy": 33.33333333333333,
"correct": 1,
"total": 3,
"category_stats": {
"2": {
"correct": 0,
"total": 2
},
"3": {
"correct": 1,
"total": 1
}
},
"detailed_results": [
{
"question": "When did Caroline go to the LGBTQ support group?",
"correct_answer": "7 May 2023",
"predicted_answer": "Caroline attended the LGBTQ support group yesterday.",
"category": 2,
"is_correct": false
},
{
"question": "When did Melanie paint a sunrise?",
"correct_answer": 2022,
"predicted_answer": "I don't know.",
"category": 2,
"is_correct": false
},
{
"question": "What fields would Caroline be likely to pursue in her educaton?",
"correct_answer": "Psychology, counseling certification",
"predicted_answer": "Caroline would be likely to pursue fields in counseling or mental health.",
"category": 3,
"is_correct": true
}
]
},
"total_turns": 419
}
]
}
File diff suppressed because one or more lines are too long
+402
View File
@@ -0,0 +1,402 @@
"""
LoComo Benchmark Runner for Entity-Aware Memory System
Evaluates the memory system on the LoComo (Long-term Conversational Memory) benchmark.
"""
import sys
from pathlib import Path
# Add parent directory to path for imports
sys.path.insert(0, str(Path(__file__).parent.parent.parent))
import json
from datetime import datetime, timezone, timedelta
from memory import TemporalSemanticMemory
from typing import List, Dict
import openai
from dotenv import load_dotenv
import os
import asyncio
from rich.console import Console
from rich.progress import Progress, SpinnerColumn, TextColumn, BarColumn
from rich.table import Table
from rich import box
load_dotenv()
console = Console()
def parse_date(date_string: str) -> datetime:
"""Parse LoComo date format to datetime."""
# Format: "1:56 pm on 8 May, 2023"
try:
dt = datetime.strptime(date_string, "%I:%M %p on %d %B, %Y")
return dt.replace(tzinfo=timezone.utc)
except:
return datetime.now(timezone.utc)
async def ingest_conversation(memory: TemporalSemanticMemory, conversation_data: Dict, agent_id: str):
"""
Ingest a LoComo conversation into the memory system (ASYNC version).
Ingests entire conversation as a single large document for maximum efficiency.
Args:
memory: Memory system instance
conversation_data: Conversation data from LoComo
agent_id: Agent ID to use
"""
conv = conversation_data['conversation']
speaker_a = conv['speaker_a']
speaker_b = conv['speaker_b']
# Get all session keys sorted
session_keys = sorted([k for k in conv.keys() if k.startswith('session_') and not k.endswith('_date_time')])
total_turns = 0
# Build entire conversation as one large text
conversation_parts = []
for session_key in session_keys:
if session_key not in conv or not isinstance(conv[session_key], list):
continue
session_data = conv[session_key]
# Add all turns from this session
for turn in session_data:
speaker = turn['speaker']
text = turn['text']
conversation_parts.append(f"{speaker} said: {text}")
total_turns += 1
# Ingest entire conversation in ONE put_async call
# Use the first session date as the event date
first_session_key = session_keys[0] if session_keys else "session_1"
date_key = f"{first_session_key}_date_time"
conversation_date = parse_date(conv.get(date_key, "1:00 pm on 1 January, 2023"))
full_conversation = " ".join(conversation_parts)
await memory.put_async(
agent_id=agent_id,
content=full_conversation,
context=f"Full conversation between {speaker_a} and {speaker_b}",
event_date=conversation_date
)
return total_turns
def answer_question(memory: TemporalSemanticMemory, agent_id: str, question: str, thinking_budget: int = 100) -> str:
"""
Answer a question using the memory system.
Args:
memory: Memory system instance
agent_id: Agent ID
question: Question to answer
thinking_budget: How many memory units to explore
Returns:
Answer string
"""
# Search memory
results = memory.search(
agent_id=agent_id,
query=question,
thinking_budget=thinking_budget,
top_k=20 # Get more results for better context
)
print("question:", question)
print("Got results:", results)
if not results:
return "I don't have enough information to answer that question."
# Build context from top results
context_parts = []
for i, result in enumerate(results[:10], 1):
context_parts.append(f"{i}. {result['text']}")
context = "\n".join(context_parts)
# Use OpenAI to generate answer from context
try:
response = openai.chat.completions.create(
model="gpt-4o-mini",
messages=[
{
"role": "system",
"content": "You are a helpful assistant. Answer the question based ONLY on the provided context. If the context doesn't contain the answer, say 'I don't know'."
},
{
"role": "user",
"content": f"Context:\n{context}\n\nQuestion: {question}\n\nAnswer:"
}
],
temperature=0,
max_tokens=150
)
return response.choices[0].message.content.strip()
except Exception as e:
return f"Error generating answer: {str(e)}"
def evaluate_qa_task(
memory: TemporalSemanticMemory,
agent_id: str,
qa_pairs: List[Dict],
sample_id: str,
max_questions: int = None
) -> Dict:
"""
Evaluate the QA task.
Returns:
Dict with evaluation metrics
"""
results = []
questions_to_eval = qa_pairs[:max_questions] if max_questions else qa_pairs
with Progress(
SpinnerColumn(),
TextColumn("[progress.description]{task.description}"),
BarColumn(),
TextColumn("[progress.percentage]{task.percentage:>3.0f}%"),
console=console
) as progress:
task = progress.add_task(f"[cyan]Evaluating QA for sample {sample_id}...", total=len(questions_to_eval))
for qa in questions_to_eval:
question = qa['question']
correct_answer = qa['answer']
category = qa.get('category', 0)
# Get predicted answer
predicted_answer = answer_question(memory, agent_id, question)
results.append({
'question': question,
'correct_answer': correct_answer,
'predicted_answer': predicted_answer,
'category': category
})
progress.update(task, advance=1)
return results
def calculate_metrics(results: List[Dict]) -> Dict:
"""
Calculate evaluation metrics.
Uses LLM-as-judge to evaluate answer quality.
"""
correct = 0
total = len(results)
category_stats = {}
with Progress(
SpinnerColumn(),
TextColumn("[progress.description]{task.description}"),
BarColumn(),
TextColumn("[progress.percentage]{task.percentage:>3.0f}%"),
console=console
) as progress:
task = progress.add_task("[yellow]Judging answers with LLM...", total=total)
for result in results:
# Use LLM as judge
try:
response = openai.chat.completions.create(
model="gpt-4o-mini",
messages=[
{
"role": "system",
"content": "You are an objective judge. Determine if the predicted answer is semantically equivalent to the correct answer. Answer with ONLY 'yes' or 'no'."
},
{
"role": "user",
"content": f"Question: {result['question']}\nCorrect answer: {result['correct_answer']}\nPredicted answer: {result['predicted_answer']}\n\nAre they equivalent?"
}
],
temperature=0,
max_tokens=5
)
judgment = response.choices[0].message.content.strip().lower()
is_correct = 'yes' in judgment
if is_correct:
correct += 1
result['is_correct'] = is_correct
# Track by category
category = result['category']
if category not in category_stats:
category_stats[category] = {'correct': 0, 'total': 0}
category_stats[category]['total'] += 1
if is_correct:
category_stats[category]['correct'] += 1
except Exception as e:
console.print(f"[red]Error judging answer: {e}[/red]")
result['is_correct'] = False
progress.update(task, advance=1)
accuracy = (correct / total * 100) if total > 0 else 0
return {
'accuracy': accuracy,
'correct': correct,
'total': total,
'category_stats': category_stats,
'detailed_results': results
}
def run_benchmark(max_conversations: int = None, max_questions_per_conv: int = None):
"""
Run the LoComo benchmark.
Args:
max_conversations: Maximum number of conversations to evaluate (None for all)
max_questions_per_conv: Maximum questions per conversation (None for all)
"""
console.print("\n[bold cyan]LoComo Benchmark - Entity-Aware Memory System[/bold cyan]")
console.print("=" * 80)
# Load dataset
console.print("\n[1] Loading LoComo dataset...")
with open('locomo10.json', 'r') as f:
dataset = json.load(f)
conversations_to_eval = dataset[:max_conversations] if max_conversations else dataset
console.print(f" [green]✓[/green] Loaded {len(conversations_to_eval)} conversations")
# Initialize memory system
console.print("\n[2] Initializing memory system...")
memory = TemporalSemanticMemory()
console.print(" [green]✓[/green] Memory system initialized")
# Run evaluation for each conversation
all_results = []
for i, conv_data in enumerate(conversations_to_eval, 1):
sample_id = conv_data['sample_id']
agent_id = f"locomo_{sample_id}"
console.print(f"\n[bold blue]Conversation {i}/{len(conversations_to_eval)}[/bold blue] (Sample ID: {sample_id})")
# Clear previous data
import psycopg2
conn = psycopg2.connect(os.getenv('DATABASE_URL'))
cursor = conn.cursor()
cursor.execute("DELETE FROM memory_units WHERE agent_id = %s", (agent_id,))
cursor.execute("DELETE FROM memory_links WHERE agent_id = %s", (agent_id,))
cursor.execute("DELETE FROM entity_cooccurrences WHERE agent_id = %s", (agent_id,))
cursor.execute("DELETE FROM unit_entities WHERE agent_id = %s", (agent_id,))
cursor.execute("DELETE FROM entities WHERE agent_id = %s", (agent_id,))
conn.commit()
cursor.close()
conn.close()
# Ingest conversation (using async for parallel embedding generation)
console.print(" [3] Ingesting conversation (async with parallel embeddings)...")
total_turns = asyncio.run(ingest_conversation(memory, conv_data, agent_id))
console.print(f" [green]✓[/green] Ingested {total_turns} conversation turns")
# Evaluate QA
console.print(f" [4] Evaluating {len(conv_data['qa'])} QA pairs...")
qa_results = evaluate_qa_task(
memory,
agent_id,
conv_data['qa'],
sample_id,
max_questions=max_questions_per_conv
)
# Calculate metrics
console.print(" [5] Calculating metrics...")
metrics = calculate_metrics(qa_results)
console.print(f" [green]✓[/green] Accuracy: {metrics['accuracy']:.2f}% ({metrics['correct']}/{metrics['total']})")
all_results.append({
'sample_id': sample_id,
'metrics': metrics,
'total_turns': total_turns
})
# Overall results
console.print("\n[bold green]✓ Benchmark Complete![/bold green]\n")
# Calculate overall metrics
total_correct = sum(r['metrics']['correct'] for r in all_results)
total_questions = sum(r['metrics']['total'] for r in all_results)
overall_accuracy = (total_correct / total_questions * 100) if total_questions > 0 else 0
# Display results table
table = Table(title="LoComo Benchmark Results", box=box.ROUNDED)
table.add_column("Sample ID", style="cyan")
table.add_column("Turns", justify="right", style="yellow")
table.add_column("Questions", justify="right", style="blue")
table.add_column("Correct", justify="right", style="green")
table.add_column("Accuracy", justify="right", style="magenta")
for result in all_results:
metrics = result['metrics']
table.add_row(
result['sample_id'],
str(result['total_turns']),
str(metrics['total']),
str(metrics['correct']),
f"{metrics['accuracy']:.1f}%"
)
table.add_row(
"[bold]OVERALL[/bold]",
"-",
f"[bold]{total_questions}[/bold]",
f"[bold]{total_correct}[/bold]",
f"[bold]{overall_accuracy:.1f}%[/bold]"
)
console.print(table)
return {
'overall_accuracy': overall_accuracy,
'total_correct': total_correct,
'total_questions': total_questions,
'conversation_results': all_results
}
if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser(description='Run LoComo benchmark')
parser.add_argument('--max-conversations', type=int, default=None, help='Maximum conversations to evaluate')
parser.add_argument('--max-questions', type=int, default=None, help='Maximum questions per conversation')
args = parser.parse_args()
results = run_benchmark(
max_conversations=args.max_conversations,
max_questions_per_conv=args.max_questions
)
# Save results
with open('benchmark_results.json', 'w') as f:
json.dump(results, f, indent=2)
console.print(f"\n[green]✓[/green] Results saved to benchmark_results.json")
+242
View File
@@ -0,0 +1,242 @@
# LongMemEval Benchmark
**Purpose**: Evaluate long-term interactive memory capabilities across five core abilities.
## Overview
LongMemEval is a comprehensive benchmark that tests chat assistants on realistic long-term memory scenarios. The benchmark evaluates five core memory abilities:
1. **Information Extraction** - Retrieving specific facts from conversation history
2. **Multi-Session Reasoning** - Connecting information across multiple conversations
3. **Temporal Reasoning** - Understanding time-based relationships and changes
4. **Knowledge Updates** - Handling conflicting or updated information
5. **Abstention** - Recognizing when information is insufficient to answer
## Dataset
- **Source**: [LongMemEval (ICLR 2025)](https://github.com/xiaowu0162/LongMemEval)
- **File**: `longmemeval_s_cleaned.json`
- **Size**: 500 question-answer pairs
- **Context**: ~40 sessions per instance (~115k tokens)
- **Format**: Multi-turn conversations with timestamped sessions
### Dataset Structure
Each instance contains:
- `question_id`: Unique identifier
- `question_type`: Category (single-session, multi-session, temporal, knowledge-update, abstention)
- `question`: Query text
- `answer`: Expected answer
- `question_date`: Query timestamp
- `haystack_sessions`: List of conversation sessions with turns
- `answer_session_ids`: Evidence session identifiers
## Quick Start
### Prerequisites
1. Python 3.11+ with dependencies installed (`uv sync`)
2. PostgreSQL database configured
3. OpenAI API key set in environment
```bash
export OPENAI_API_KEY="your-api-key"
```
### Download Dataset
```bash
cd benchmarks/longmemeval
curl -L "https://huggingface.co/datasets/xiaowu0162/longmemeval-cleaned/resolve/main/longmemeval_s_cleaned.json" -o longmemeval_s_cleaned.json
```
### Run Benchmark
**Full evaluation** (500 questions):
```bash
uv run python run_benchmark.py
```
**Quick test** (first 5 instances):
```bash
uv run python run_benchmark.py --max-instances 5
```
**Custom settings**:
```bash
uv run python run_benchmark.py \
--max-instances 10 \
--thinking-budget 100 \
--top-k 20 \
--output my_results.json
```
### View Results
Results are saved to `benchmark_results.json` and include:
- Per-question scores and predictions
- Performance breakdown by question type
- Retrieved memory units for debugging
- Evaluation explanations
## Methodology
### 1. Ingestion Phase
For each instance:
1. Parse all conversation sessions with timestamps
2. Process each turn (user and assistant messages)
3. Store in memory system with:
- Coreference resolution (pronouns → entities)
- Entity extraction and disambiguation
- Temporal, semantic, and entity link creation
### 2. Retrieval Phase
For each question:
1. Generate query embedding
2. Find entry points (top-3 similar memories)
3. Spread activation through memory graph
4. Apply recency and frequency weighting
5. Return top-k most relevant memory units
### 3. Answer Generation
1. Format retrieved memories as context
2. Generate answer using GPT-4o-mini
3. Enforce answering only from provided memories
4. Handle abstention cases appropriately
### 4. Evaluation
1. Compare predicted answer to gold answer
2. Use GPT-4o as judge for semantic equivalence
3. Binary scoring (1 = correct, 0 = incorrect)
4. Aggregate by question type
## Parameters
| Parameter | Default | Description |
|-----------|---------|-------------|
| `--max-instances` | 500 | Number of instances to evaluate |
| `--max-questions` | None | Limit questions per instance (for testing) |
| `--thinking-budget` | 100 | Exploration budget for spreading activation |
| `--top-k` | 20 | Number of memory units to retrieve |
| `--output` | `benchmark_results.json` | Output file path |
## Expected Performance
Based on published results:
| System | Accuracy |
|--------|----------|
| Human | ~95% |
| Zep | 75.2% |
| Letta (GPT-4o-mini) | 74.0% |
| Mem0 Graph | 68.5% |
| **Target** | **65-75%** |
## Performance by Question Type
Expected breakdown:
- **Single-session**: 70-80% (easiest - information in one session)
- **Multi-session**: 60-70% (requires connecting across sessions)
- **Temporal reasoning**: 60-70% (requires time-based reasoning)
- **Knowledge updates**: 50-65% (hardest - handling conflicting info)
- **Abstention**: 65-75% (recognizing insufficient information)
## Computational Cost
**Per instance** (~40 sessions, ~200 turns):
- Ingestion: ~200 embedding generations (local model, fast)
- Query: 1 embedding generation + graph search
- Answer: 1 GPT-4o-mini call (~200 tokens)
- Evaluation: 1 GPT-4o call (~150 tokens)
**Full benchmark** (500 instances):
- Runtime: 2-4 hours (depends on API rate limits)
- Cost: ~$50-80 (OpenAI API for answer generation + evaluation)
- Embeddings: Free (local model)
## Example Output
```
LongMemEval Benchmark Evaluation
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Overall Performance
┏━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃ Metric ┃ Value ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━┩
│ Total │ 500 │
│ Correct │ 345 │
│ Incorrect │ 155 │
│ Accuracy │ 69.0% │
└────────────────┴───────┘
Performance by Question Type
┏━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┓
┃ Question Type ┃ Total ┃ Correct ┃ Accuracy ┃
┡━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━┩
│ single-session │ 150 │ 115 │ 76.7% │
│ multi-session │ 120 │ 78 │ 65.0% │
│ temporal-reasoning │ 100 │ 65 │ 65.0% │
│ knowledge-update │ 80 │ 48 │ 60.0% │
│ abstention │ 50 │ 39 │ 78.0% │
└────────────────────┴───────┴─────────┴──────────┘
```
## Troubleshooting
### Dataset not found
```bash
cd benchmarks/longmemeval
curl -L "https://huggingface.co/datasets/xiaowu0162/longmemeval-cleaned/resolve/main/longmemeval_s_cleaned.json" -o longmemeval_s_cleaned.json
```
### OpenAI API key error
```bash
export OPENAI_API_KEY="your-api-key"
```
### Memory ingestion slow
- This is expected for first-time entity resolution
- Subsequent queries are fast (graph search)
- Consider using `--max-instances` for quick testing
### Low accuracy
- Try increasing `--thinking-budget` (default: 100)
- Try increasing `--top-k` (default: 20)
- Check retrieved memories in results JSON for debugging
## Architecture Integration
This benchmark tests the full memory system architecture:
1.**Coreference Resolution**: Makes memories self-contained
2.**Entity Extraction**: Identifies people, organizations, places
3.**Entity Disambiguation**: Links mentions across sessions
4.**Temporal Links**: Connects memories by time proximity
5.**Semantic Links**: Connects memories by meaning
6.**Entity Links**: Connects memories by shared entities
7.**Spreading Activation**: Graph-aware retrieval
8.**Recency/Frequency Weighting**: Importance signals
## Citation
If you use this benchmark, please cite:
```bibtex
@inproceedings{wu2025longmemeval,
title={LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory},
author={Wu, Di and Wang, Hongwei and Liu, Wenhao and Wang, Jiaheng and Li, Zihan and Huang, Yiqin and Patel, Zelin and Liu, Yiheng and Meng, Bo and Pan, Sinong and others},
booktitle={The Thirteenth International Conference on Learning Representations},
year={2025}
}
```
## Related Benchmarks
- **LoComo**: Multi-session conversational QA
- **MemGPT Tasks**: Long-context question answering
- **Custom Temporal Reasoning**: Time-based memory retrieval
@@ -0,0 +1 @@
[]
+487
View File
@@ -0,0 +1,487 @@
"""
LongMemEval Benchmark Evaluation
This script evaluates the Entity-Aware Memory System on the LongMemEval benchmark,
which tests five core long-term memory abilities:
1. Information extraction
2. Multi-session reasoning
3. Temporal reasoning
4. Knowledge updates
5. Abstention
Dataset: LongMemEval-S (~115k tokens, ~40 sessions per instance, 500 questions)
Source: https://github.com/xiaowu0162/LongMemEval
"""
import json
import os
import sys
import argparse
from datetime import datetime, timezone
from typing import Dict, List, Any
from pathlib import Path
import time
import asyncio
from dotenv import load_dotenv
# Load environment variables from .env
load_dotenv()
# Add parent directory to path
sys.path.insert(0, str(Path(__file__).parent.parent.parent))
from memory import TemporalSemanticMemory
from openai import OpenAI
from rich.console import Console
from rich.progress import Progress, SpinnerColumn, TextColumn, BarColumn, TimeElapsedColumn
from rich.table import Table
console = Console()
def parse_args():
parser = argparse.ArgumentParser(description="Run LongMemEval benchmark")
parser.add_argument(
"--max-instances",
type=int,
default=None,
help="Limit number of instances to evaluate (default: all 500)"
)
parser.add_argument(
"--max-questions",
type=int,
default=None,
help="Limit number of questions per instance (for quick testing)"
)
parser.add_argument(
"--output",
type=str,
default="benchmark_results.json",
help="Output file for results"
)
parser.add_argument(
"--thinking-budget",
type=int,
default=100,
help="Thinking budget for spreading activation search"
)
parser.add_argument(
"--top-k",
type=int,
default=20,
help="Number of memory units to retrieve per query"
)
return parser.parse_args()
def load_dataset(dataset_path: str) -> List[Dict[str, Any]]:
"""Load LongMemEval dataset from JSON file."""
with open(dataset_path, 'r') as f:
data = json.load(f)
return data
def parse_date(date_str: str) -> datetime:
"""Parse date string to datetime object."""
try:
# LongMemEval format: "2023/05/20 (Sat) 02:21"
# Try to parse the main part before the day name
date_str_cleaned = date_str.split('(')[0].strip() if '(' in date_str else date_str
# Try multiple formats
for fmt in ["%Y/%m/%d %H:%M", "%Y-%m-%d %H:%M:%S", "%Y-%m-%d", "%Y/%m/%d"]:
try:
dt = datetime.strptime(date_str_cleaned, fmt)
return dt.replace(tzinfo=timezone.utc)
except ValueError:
continue
# Fallback: try ISO format
return datetime.fromisoformat(date_str.replace('Z', '+00:00'))
except Exception as e:
console.print(f"[yellow]Warning: Failed to parse date '{date_str}': {e}[/yellow]")
return datetime.now(timezone.utc)
async def ingest_conversation(memory: TemporalSemanticMemory, agent_id: str, instance: Dict[str, Any]) -> None:
"""
Ingest conversation history into memory system.
Args:
memory: Memory system instance
agent_id: Unique agent ID for this conversation
instance: LongMemEval instance containing haystack_sessions
"""
# LongMemEval format: list of sessions, each session is a list of turn dicts
sessions = instance.get("haystack_sessions", [])
dates = instance.get("haystack_dates", [])
session_ids = instance.get("haystack_session_ids", [])
# Ensure all lists have same length
if not (len(sessions) == len(dates) == len(session_ids)):
console.print(f"[yellow]Warning: Mismatched lengths - sessions:{len(sessions)}, dates:{len(dates)}, ids:{len(session_ids)}[/yellow]")
min_len = min(len(sessions), len(dates), len(session_ids))
sessions = sessions[:min_len]
dates = dates[:min_len]
session_ids = session_ids[:min_len]
# Process each session - combine all turns into one put_async call
for session_turns, date_str, session_id in zip(sessions, dates, session_ids):
# Parse session date
session_date = parse_date(date_str) if date_str else datetime.now(timezone.utc)
# Combine all turns in the session into one content string
session_content_parts = []
for turn_dict in session_turns:
role = turn_dict.get("role", "")
content = turn_dict.get("content", "")
if not content.strip():
continue
# Format as "role: content" for clarity
session_content_parts.append(f"{role}: {content}")
# Ingest entire session as one chunk
if session_content_parts:
session_content = "\n".join(session_content_parts)
context = f"Session {session_id}"
try:
await memory.put_async(
agent_id=agent_id,
content=session_content,
context=context,
event_date=session_date
)
except Exception as e:
console.print(f"[yellow]Warning: Failed to ingest session {session_id}: {e}[/yellow]")
def retrieve_memories(
memory: TemporalSemanticMemory,
agent_id: str,
query: str,
thinking_budget: int,
top_k: int
) -> List[Dict[str, Any]]:
"""
Retrieve relevant memories for a query.
Args:
memory: Memory system instance
agent_id: Agent ID
query: Query text
thinking_budget: Thinking budget for search
top_k: Number of results to return
Returns:
List of retrieved memory units
"""
try:
results = memory.search(
agent_id=agent_id,
query=query,
thinking_budget=thinking_budget,
top_k=top_k
)
return results
except Exception as e:
console.print(f"[yellow]Warning: Search failed: {e}[/yellow]")
return []
def generate_answer(
client: OpenAI,
question: str,
memories: List[Dict[str, Any]],
model: str = "gpt-4o-mini"
) -> str:
"""
Generate answer to question using retrieved memories.
Args:
client: OpenAI client
question: Question text
memories: Retrieved memory units
model: OpenAI model to use
Returns:
Generated answer
"""
# Format memories as context
context_parts = []
for i, mem in enumerate(memories, 1):
context_parts.append(f"[Memory {i}] {mem['text']}")
context = "\n".join(context_parts) if context_parts else "No relevant memories found."
prompt = f"""You are a helpful assistant. Based on the following memories from past conversations, answer the question.
Memories:
{context}
Question: {question}
Instructions:
- Answer based ONLY on the provided memories
- If the memories don't contain the answer, say "I don't have enough information to answer this question"
- Be concise and direct
- If asked to abstain (e.g., for unanswerable questions), explicitly say you cannot answer
Answer:"""
try:
response = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": prompt}],
temperature=0.0,
max_tokens=300
)
return response.choices[0].message.content.strip()
except Exception as e:
console.print(f"[yellow]Warning: Answer generation failed: {e}[/yellow]")
return "Error generating answer"
def evaluate_answer(
client: OpenAI,
question: str,
predicted_answer: str,
gold_answer: str,
model: str = "gpt-4o"
) -> Dict[str, Any]:
"""
Evaluate predicted answer against gold answer using LLM-as-judge.
Args:
client: OpenAI client
question: Question text
predicted_answer: Predicted answer
gold_answer: Gold answer
model: OpenAI model to use for evaluation
Returns:
Evaluation result with score and explanation
"""
prompt = f"""You are an expert evaluator. Evaluate if the predicted answer is semantically equivalent to the gold answer.
Question: {question}
Gold Answer: {gold_answer}
Predicted Answer: {predicted_answer}
Instructions:
- Score 1 if the predicted answer is semantically equivalent (same meaning, different wording is OK)
- Score 1 if the predicted answer correctly abstains when the gold answer indicates the question is unanswerable
- Score 0 if the predicted answer is incorrect or contradicts the gold answer
- Score 0 if the predicted answer provides an answer when it should abstain
- Provide a brief explanation
Output format:
Score: [0 or 1]
Explanation: [brief explanation]"""
try:
response = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": prompt}],
temperature=0.0,
max_tokens=200
)
content = response.choices[0].message.content.strip()
# Parse score and explanation
lines = content.split('\n')
score = 0
explanation = ""
for line in lines:
if line.startswith("Score:"):
score_str = line.replace("Score:", "").strip()
score = int(score_str) if score_str.isdigit() else 0
elif line.startswith("Explanation:"):
explanation = line.replace("Explanation:", "").strip()
return {
"score": score,
"explanation": explanation,
"raw_output": content
}
except Exception as e:
console.print(f"[yellow]Warning: Evaluation failed: {e}[/yellow]")
return {
"score": 0,
"explanation": f"Evaluation error: {str(e)}",
"raw_output": ""
}
def run_benchmark(args):
"""Run the LongMemEval benchmark evaluation."""
console.print("\n[bold cyan]LongMemEval Benchmark Evaluation[/bold cyan]\n")
# Load dataset
dataset_path = Path(__file__).parent / "longmemeval_s_cleaned.json"
if not dataset_path.exists():
console.print(f"[red]Error: Dataset not found at {dataset_path}[/red]")
console.print("[yellow]Run: curl -L 'https://huggingface.co/datasets/xiaowu0162/longmemeval-cleaned/resolve/main/longmemeval_s_cleaned.json' -o longmemeval_s_cleaned.json[/yellow]")
return
console.print(f"[green]Loading dataset from {dataset_path}[/green]")
dataset = load_dataset(dataset_path)
if args.max_instances:
dataset = dataset[:args.max_instances]
console.print(f"[yellow]Limited to {args.max_instances} instances[/yellow]")
console.print(f"Dataset size: {len(dataset)} instances\n")
# Initialize memory system
console.print("[cyan]Initializing memory system...[/cyan]")
memory = TemporalSemanticMemory()
# Initialize OpenAI client
openai_api_key = os.getenv("OPENAI_API_KEY")
if not openai_api_key:
console.print("[red]Error: OPENAI_API_KEY not set[/red]")
return
client = OpenAI(api_key=openai_api_key)
# Results storage
results = []
# Process each instance
with Progress(
SpinnerColumn(),
TextColumn("[progress.description]{task.description}"),
BarColumn(),
TextColumn("[progress.percentage]{task.percentage:>3.0f}%"),
TimeElapsedColumn(),
console=console
) as progress:
instance_task = progress.add_task("[cyan]Processing instances...", total=len(dataset))
for idx, instance in enumerate(dataset):
question_id = instance.get("question_id", f"q_{idx}")
question = instance.get("question", "")
gold_answer = instance.get("answer", "")
question_type = instance.get("question_type", "unknown")
progress.update(instance_task, description=f"[cyan]Instance {idx+1}/{len(dataset)}: {question_id}")
# Create unique agent ID for this instance
agent_id = f"longmemeval_{question_id}"
# Ingest conversation history
try:
asyncio.run(ingest_conversation(memory, agent_id, instance))
except Exception as e:
console.print(f"[red]Error ingesting instance {question_id}: {e}[/red]")
continue
# Retrieve memories
memories = retrieve_memories(
memory,
agent_id,
question,
args.thinking_budget,
args.top_k
)
# Generate answer
predicted_answer = generate_answer(client, question, memories)
# Evaluate answer
evaluation = evaluate_answer(client, question, predicted_answer, gold_answer)
# Store result
result = {
"question_id": question_id,
"question_type": question_type,
"question": question,
"gold_answer": gold_answer,
"predicted_answer": predicted_answer,
"score": evaluation["score"],
"explanation": evaluation["explanation"],
"num_memories_retrieved": len(memories),
"memory_texts": [m["text"] for m in memories[:5]] # Store top 5 for debugging
}
results.append(result)
progress.update(instance_task, advance=1)
# Save intermediate results
if (idx + 1) % 10 == 0:
save_results(results, args.output)
# Save final results
save_results(results, args.output)
# Display summary
display_summary(results)
def save_results(results: List[Dict[str, Any]], output_path: str):
"""Save results to JSON file."""
output_file = Path(__file__).parent / output_path
with open(output_file, 'w') as f:
json.dump(results, f, indent=2)
console.print(f"[green]Results saved to {output_file}[/green]")
def display_summary(results: List[Dict[str, Any]]):
"""Display benchmark summary."""
console.print("\n[bold cyan]Benchmark Summary[/bold cyan]\n")
# Overall accuracy
total = len(results)
correct = sum(1 for r in results if r["score"] == 1)
accuracy = (correct / total * 100) if total > 0 else 0
table = Table(title="Overall Performance")
table.add_column("Metric", style="cyan")
table.add_column("Value", style="green")
table.add_row("Total Questions", str(total))
table.add_row("Correct", str(correct))
table.add_row("Incorrect", str(total - correct))
table.add_row("Accuracy", f"{accuracy:.2f}%")
console.print(table)
# Accuracy by question type
type_stats = {}
for result in results:
qtype = result["question_type"]
if qtype not in type_stats:
type_stats[qtype] = {"total": 0, "correct": 0}
type_stats[qtype]["total"] += 1
type_stats[qtype]["correct"] += result["score"]
type_table = Table(title="Performance by Question Type")
type_table.add_column("Question Type", style="cyan")
type_table.add_column("Total", style="yellow")
type_table.add_column("Correct", style="green")
type_table.add_column("Accuracy", style="green")
for qtype, stats in sorted(type_stats.items()):
acc = (stats["correct"] / stats["total"] * 100) if stats["total"] > 0 else 0
type_table.add_row(
qtype,
str(stats["total"]),
str(stats["correct"]),
f"{acc:.2f}%"
)
console.print("\n")
console.print(type_table)
if __name__ == "__main__":
args = parse_args()
run_benchmark(args)
+22
View File
@@ -0,0 +1,22 @@
-- ============================================================================
-- DROP ALL TABLES FOR MEMORY POC DATABASE
-- ============================================================================
--
-- WARNING: This will completely remove all tables and data!
-- Use with caution, especially in production environments.
--
-- Usage:
-- psql -d your_database -f drop.sql
--
-- After running this, you'll need to recreate the schema:
-- psql -d your_database -f schema.sql
-- ============================================================================
-- Drop all tables in reverse dependency order
-- CASCADE ensures dependent objects are also dropped
DROP TABLE IF EXISTS memory_links CASCADE;
DROP TABLE IF EXISTS entity_cooccurrences CASCADE;
DROP TABLE IF EXISTS unit_entities CASCADE;
DROP TABLE IF EXISTS entities CASCADE;
DROP TABLE IF EXISTS memory_units CASCADE;
+189
View File
@@ -0,0 +1,189 @@
function neighbourhoodHighlight(params) {
// console.log("in nieghbourhoodhighlight");
allNodes = nodes.get({ returnType: "Object" });
// originalNodes = JSON.parse(JSON.stringify(allNodes));
// if something is selected:
if (params.nodes.length > 0) {
highlightActive = true;
var i, j;
var selectedNode = params.nodes[0];
var degrees = 2;
// mark all nodes as hard to read.
for (let nodeId in allNodes) {
// nodeColors[nodeId] = allNodes[nodeId].color;
allNodes[nodeId].color = "rgba(200,200,200,0.5)";
if (allNodes[nodeId].hiddenLabel === undefined) {
allNodes[nodeId].hiddenLabel = allNodes[nodeId].label;
allNodes[nodeId].label = undefined;
}
}
var connectedNodes = network.getConnectedNodes(selectedNode);
var allConnectedNodes = [];
// get the second degree nodes
for (i = 1; i < degrees; i++) {
for (j = 0; j < connectedNodes.length; j++) {
allConnectedNodes = allConnectedNodes.concat(
network.getConnectedNodes(connectedNodes[j])
);
}
}
// all second degree nodes get a different color and their label back
for (i = 0; i < allConnectedNodes.length; i++) {
// allNodes[allConnectedNodes[i]].color = "pink";
allNodes[allConnectedNodes[i]].color = "rgba(150,150,150,0.75)";
if (allNodes[allConnectedNodes[i]].hiddenLabel !== undefined) {
allNodes[allConnectedNodes[i]].label =
allNodes[allConnectedNodes[i]].hiddenLabel;
allNodes[allConnectedNodes[i]].hiddenLabel = undefined;
}
}
// all first degree nodes get their own color and their label back
for (i = 0; i < connectedNodes.length; i++) {
// allNodes[connectedNodes[i]].color = undefined;
allNodes[connectedNodes[i]].color = nodeColors[connectedNodes[i]];
if (allNodes[connectedNodes[i]].hiddenLabel !== undefined) {
allNodes[connectedNodes[i]].label =
allNodes[connectedNodes[i]].hiddenLabel;
allNodes[connectedNodes[i]].hiddenLabel = undefined;
}
}
// the main node gets its own color and its label back.
// allNodes[selectedNode].color = undefined;
allNodes[selectedNode].color = nodeColors[selectedNode];
if (allNodes[selectedNode].hiddenLabel !== undefined) {
allNodes[selectedNode].label = allNodes[selectedNode].hiddenLabel;
allNodes[selectedNode].hiddenLabel = undefined;
}
} else if (highlightActive === true) {
// console.log("highlightActive was true");
// reset all nodes
for (let nodeId in allNodes) {
// allNodes[nodeId].color = "purple";
allNodes[nodeId].color = nodeColors[nodeId];
// delete allNodes[nodeId].color;
if (allNodes[nodeId].hiddenLabel !== undefined) {
allNodes[nodeId].label = allNodes[nodeId].hiddenLabel;
allNodes[nodeId].hiddenLabel = undefined;
}
}
highlightActive = false;
}
// transform the object into an array
var updateArray = [];
if (params.nodes.length > 0) {
for (let nodeId in allNodes) {
if (allNodes.hasOwnProperty(nodeId)) {
// console.log(allNodes[nodeId]);
updateArray.push(allNodes[nodeId]);
}
}
nodes.update(updateArray);
} else {
// console.log("Nothing was selected");
for (let nodeId in allNodes) {
if (allNodes.hasOwnProperty(nodeId)) {
// console.log(allNodes[nodeId]);
// allNodes[nodeId].color = {};
updateArray.push(allNodes[nodeId]);
}
}
nodes.update(updateArray);
}
}
function filterHighlight(params) {
allNodes = nodes.get({ returnType: "Object" });
// if something is selected:
if (params.nodes.length > 0) {
filterActive = true;
let selectedNodes = params.nodes;
// hiding all nodes and saving the label
for (let nodeId in allNodes) {
allNodes[nodeId].hidden = true;
if (allNodes[nodeId].savedLabel === undefined) {
allNodes[nodeId].savedLabel = allNodes[nodeId].label;
allNodes[nodeId].label = undefined;
}
}
for (let i=0; i < selectedNodes.length; i++) {
allNodes[selectedNodes[i]].hidden = false;
if (allNodes[selectedNodes[i]].savedLabel !== undefined) {
allNodes[selectedNodes[i]].label = allNodes[selectedNodes[i]].savedLabel;
allNodes[selectedNodes[i]].savedLabel = undefined;
}
}
} else if (filterActive === true) {
// reset all nodes
for (let nodeId in allNodes) {
allNodes[nodeId].hidden = false;
if (allNodes[nodeId].savedLabel !== undefined) {
allNodes[nodeId].label = allNodes[nodeId].savedLabel;
allNodes[nodeId].savedLabel = undefined;
}
}
filterActive = false;
}
// transform the object into an array
var updateArray = [];
if (params.nodes.length > 0) {
for (let nodeId in allNodes) {
if (allNodes.hasOwnProperty(nodeId)) {
updateArray.push(allNodes[nodeId]);
}
}
nodes.update(updateArray);
} else {
for (let nodeId in allNodes) {
if (allNodes.hasOwnProperty(nodeId)) {
updateArray.push(allNodes[nodeId]);
}
}
nodes.update(updateArray);
}
}
function selectNode(nodes) {
network.selectNodes(nodes);
neighbourhoodHighlight({ nodes: nodes });
return nodes;
}
function selectNodes(nodes) {
network.selectNodes(nodes);
filterHighlight({nodes: nodes});
return nodes;
}
function highlightFilter(filter) {
let selectedNodes = []
let selectedProp = filter['property']
if (filter['item'] === 'node') {
let allNodes = nodes.get({ returnType: "Object" });
for (let nodeId in allNodes) {
if (allNodes[nodeId][selectedProp] && filter['value'].includes((allNodes[nodeId][selectedProp]).toString())) {
selectedNodes.push(nodeId)
}
}
}
else if (filter['item'] === 'edge'){
let allEdges = edges.get({returnType: 'object'});
// check if the selected property exists for selected edge and select the nodes connected to the edge
for (let edge in allEdges) {
if (allEdges[edge][selectedProp] && filter['value'].includes((allEdges[edge][selectedProp]).toString())) {
selectedNodes.push(allEdges[edge]['from'])
selectedNodes.push(allEdges[edge]['to'])
}
}
}
selectNodes(selectedNodes)
}
+356
View File
@@ -0,0 +1,356 @@
/**
* Tom Select v2.0.0-rc.4
* Licensed under the Apache License, Version 2.0 (the "License");
*/
!function(e,t){"object"==typeof exports&&"undefined"!=typeof module?module.exports=t():"function"==typeof define&&define.amd?define(t):(e="undefined"!=typeof globalThis?globalThis:e||self).TomSelect=t()}(this,(function(){"use strict"
function e(e,t){e.split(/\s+/).forEach((e=>{t(e)}))}class t{constructor(){this._events={}}on(t,i){e(t,(e=>{this._events[e]=this._events[e]||[],this._events[e].push(i)}))}off(t,i){var s=arguments.length
0!==s?e(t,(e=>{if(1===s)return delete this._events[e]
e in this._events!=!1&&this._events[e].splice(this._events[e].indexOf(i),1)})):this._events={}}trigger(t,...i){var s=this
e(t,(e=>{if(e in s._events!=!1)for(let t of s._events[e])t.apply(s,i)}))}}var i
const s="[̀-ͯ·ʾ]",n=new RegExp(s,"g")
var o
const r={"æ":"ae","ⱥ":"a","ø":"o"},l=new RegExp(Object.keys(r).join("|"),"g"),a=[[67,67],[160,160],[192,438],[452,652],[961,961],[1019,1019],[1083,1083],[1281,1289],[1984,1984],[5095,5095],[7429,7441],[7545,7549],[7680,7935],[8580,8580],[9398,9449],[11360,11391],[42792,42793],[42802,42851],[42873,42897],[42912,42922],[64256,64260],[65313,65338],[65345,65370]],c=e=>e.normalize("NFKD").replace(n,"").toLowerCase().replace(l,(function(e){return r[e]})),d=(e,t="|")=>{if(1==e.length)return e[0]
var i=1
return e.forEach((e=>{i=Math.max(i,e.length)})),1==i?"["+e.join("")+"]":"(?:"+e.join(t)+")"},p=e=>{if(1===e.length)return[[e]]
var t=[]
return p(e.substring(1)).forEach((function(i){var s=i.slice(0)
s[0]=e.charAt(0)+s[0],t.push(s),(s=i.slice(0)).unshift(e.charAt(0)),t.push(s)})),t},u=e=>{void 0===o&&(o=(()=>{var e={}
a.forEach((t=>{for(let s=t[0];s<=t[1];s++){let t=String.fromCharCode(s),n=c(t)
if(n!=t.toLowerCase()){n in e||(e[n]=[n])
var i=new RegExp(d(e[n]),"iu")
t.match(i)||e[n].push(t)}}}))
var t=Object.keys(e)
t=t.sort(((e,t)=>t.length-e.length)),i=new RegExp("("+d(t)+"[̀-ͯ·ʾ]*)","g")
var s={}
return t.sort(((e,t)=>e.length-t.length)).forEach((t=>{var i=p(t).map((t=>(t=t.map((t=>e.hasOwnProperty(t)?d(e[t]):t)),d(t,""))))
s[t]=d(i)})),s})())
return e.normalize("NFKD").toLowerCase().split(i).map((e=>{if(""==e)return""
const t=c(e)
if(o.hasOwnProperty(t))return o[t]
const i=e.normalize("NFC")
return i!=e?d([e,i]):e})).join("")},h=(e,t)=>{if(e)return e[t]},g=(e,t)=>{if(e){for(var i,s=t.split(".");(i=s.shift())&&(e=e[i]););return e}},f=(e,t,i)=>{var s,n
return e?-1===(n=(e+="").search(t.regex))?0:(s=t.string.length/e.length,0===n&&(s+=.5),s*i):0},v=e=>(e+"").replace(/([\$\(-\+\.\?\[-\^\{-\}])/g,"\\$1"),m=(e,t)=>{var i=e[t]
if("function"==typeof i)return i
i&&!Array.isArray(i)&&(e[t]=[i])},y=(e,t)=>{if(Array.isArray(e))e.forEach(t)
else for(var i in e)e.hasOwnProperty(i)&&t(e[i],i)},O=(e,t)=>"number"==typeof e&&"number"==typeof t?e>t?1:e<t?-1:0:(e=c(e+"").toLowerCase())>(t=c(t+"").toLowerCase())?1:t>e?-1:0
class b{constructor(e,t){this.items=e,this.settings=t||{diacritics:!0}}tokenize(e,t,i){if(!e||!e.length)return[]
const s=[],n=e.split(/\s+/)
var o
return i&&(o=new RegExp("^("+Object.keys(i).map(v).join("|")+"):(.*)$")),n.forEach((e=>{let i,n=null,r=null
o&&(i=e.match(o))&&(n=i[1],e=i[2]),e.length>0&&(r=v(e),this.settings.diacritics&&(r=u(r)),t&&(r="\\b"+r)),s.push({string:e,regex:r?new RegExp(r,"iu"):null,field:n})})),s}getScoreFunction(e,t){var i=this.prepareSearch(e,t)
return this._getScoreFunction(i)}_getScoreFunction(e){const t=e.tokens,i=t.length
if(!i)return function(){return 0}
const s=e.options.fields,n=e.weights,o=s.length,r=e.getAttrFn
if(!o)return function(){return 1}
const l=1===o?function(e,t){const i=s[0].field
return f(r(t,i),e,n[i])}:function(e,t){var i=0
if(e.field){const s=r(t,e.field)
!e.regex&&s?i+=1/o:i+=f(s,e,1)}else y(n,((s,n)=>{i+=f(r(t,n),e,s)}))
return i/o}
return 1===i?function(e){return l(t[0],e)}:"and"===e.options.conjunction?function(e){for(var s,n=0,o=0;n<i;n++){if((s=l(t[n],e))<=0)return 0
o+=s}return o/i}:function(e){var s=0
return y(t,(t=>{s+=l(t,e)})),s/i}}getSortFunction(e,t){var i=this.prepareSearch(e,t)
return this._getSortFunction(i)}_getSortFunction(e){var t,i,s
const n=this,o=e.options,r=!e.query&&o.sort_empty?o.sort_empty:o.sort,l=[],a=[]
if("function"==typeof r)return r.bind(this)
const c=function(t,i){return"$score"===t?i.score:e.getAttrFn(n.items[i.id],t)}
if(r)for(t=0,i=r.length;t<i;t++)(e.query||"$score"!==r[t].field)&&l.push(r[t])
if(e.query){for(s=!0,t=0,i=l.length;t<i;t++)if("$score"===l[t].field){s=!1
break}s&&l.unshift({field:"$score",direction:"desc"})}else for(t=0,i=l.length;t<i;t++)if("$score"===l[t].field){l.splice(t,1)
break}for(t=0,i=l.length;t<i;t++)a.push("desc"===l[t].direction?-1:1)
const d=l.length
if(d){if(1===d){const e=l[0].field,t=a[0]
return function(i,s){return t*O(c(e,i),c(e,s))}}return function(e,t){var i,s,n
for(i=0;i<d;i++)if(n=l[i].field,s=a[i]*O(c(n,e),c(n,t)))return s
return 0}}return null}prepareSearch(e,t){const i={}
var s=Object.assign({},t)
if(m(s,"sort"),m(s,"sort_empty"),s.fields){m(s,"fields")
const e=[]
s.fields.forEach((t=>{"string"==typeof t&&(t={field:t,weight:1}),e.push(t),i[t.field]="weight"in t?t.weight:1})),s.fields=e}return{options:s,query:e.toLowerCase().trim(),tokens:this.tokenize(e,s.respect_word_boundaries,i),total:0,items:[],weights:i,getAttrFn:s.nesting?g:h}}search(e,t){var i,s,n=this
s=this.prepareSearch(e,t),t=s.options,e=s.query
const o=t.score||n._getScoreFunction(s)
e.length?y(n.items,((e,n)=>{i=o(e),(!1===t.filter||i>0)&&s.items.push({score:i,id:n})})):y(n.items,((e,t)=>{s.items.push({score:1,id:t})}))
const r=n._getSortFunction(s)
return r&&s.items.sort(r),s.total=s.items.length,"number"==typeof t.limit&&(s.items=s.items.slice(0,t.limit)),s}}const w=e=>{if(e.jquery)return e[0]
if(e instanceof HTMLElement)return e
if(e.indexOf("<")>-1){let t=document.createElement("div")
return t.innerHTML=e.trim(),t.firstChild}return document.querySelector(e)},_=(e,t)=>{var i=document.createEvent("HTMLEvents")
i.initEvent(t,!0,!1),e.dispatchEvent(i)},I=(e,t)=>{Object.assign(e.style,t)},C=(e,...t)=>{var i=A(t);(e=x(e)).map((e=>{i.map((t=>{e.classList.add(t)}))}))},S=(e,...t)=>{var i=A(t);(e=x(e)).map((e=>{i.map((t=>{e.classList.remove(t)}))}))},A=e=>{var t=[]
return y(e,(e=>{"string"==typeof e&&(e=e.trim().split(/[\11\12\14\15\40]/)),Array.isArray(e)&&(t=t.concat(e))})),t.filter(Boolean)},x=e=>(Array.isArray(e)||(e=[e]),e),k=(e,t,i)=>{if(!i||i.contains(e))for(;e&&e.matches;){if(e.matches(t))return e
e=e.parentNode}},F=(e,t=0)=>t>0?e[e.length-1]:e[0],L=(e,t)=>{if(!e)return-1
t=t||e.nodeName
for(var i=0;e=e.previousElementSibling;)e.matches(t)&&i++
return i},P=(e,t)=>{y(t,((t,i)=>{null==t?e.removeAttribute(i):e.setAttribute(i,""+t)}))},E=(e,t)=>{e.parentNode&&e.parentNode.replaceChild(t,e)},T=(e,t)=>{if(null===t)return
if("string"==typeof t){if(!t.length)return
t=new RegExp(t,"i")}const i=e=>3===e.nodeType?(e=>{var i=e.data.match(t)
if(i&&e.data.length>0){var s=document.createElement("span")
s.className="highlight"
var n=e.splitText(i.index)
n.splitText(i[0].length)
var o=n.cloneNode(!0)
return s.appendChild(o),E(n,s),1}return 0})(e):((e=>{if(1===e.nodeType&&e.childNodes&&!/(script|style)/i.test(e.tagName)&&("highlight"!==e.className||"SPAN"!==e.tagName))for(var t=0;t<e.childNodes.length;++t)t+=i(e.childNodes[t])})(e),0)
i(e)},V="undefined"!=typeof navigator&&/Mac/.test(navigator.userAgent)?"metaKey":"ctrlKey"
var j={options:[],optgroups:[],plugins:[],delimiter:",",splitOn:null,persist:!0,diacritics:!0,create:null,createOnBlur:!1,createFilter:null,highlight:!0,openOnFocus:!0,shouldOpen:null,maxOptions:50,maxItems:null,hideSelected:null,duplicates:!1,addPrecedence:!1,selectOnTab:!1,preload:null,allowEmptyOption:!1,loadThrottle:300,loadingClass:"loading",dataAttr:null,optgroupField:"optgroup",valueField:"value",labelField:"text",disabledField:"disabled",optgroupLabelField:"label",optgroupValueField:"value",lockOptgroupOrder:!1,sortField:"$order",searchField:["text"],searchConjunction:"and",mode:null,wrapperClass:"ts-wrapper",controlClass:"ts-control",dropdownClass:"ts-dropdown",dropdownContentClass:"ts-dropdown-content",itemClass:"item",optionClass:"option",dropdownParent:null,copyClassesToDropdown:!1,placeholder:null,hidePlaceholder:null,shouldLoad:function(e){return e.length>0},render:{}}
const q=e=>null==e?null:D(e),D=e=>"boolean"==typeof e?e?"1":"0":e+"",N=e=>(e+"").replace(/&/g,"&amp;").replace(/</g,"&lt;").replace(/>/g,"&gt;").replace(/"/g,"&quot;"),z=(e,t)=>{var i
return function(s,n){var o=this
i&&(o.loading=Math.max(o.loading-1,0),clearTimeout(i)),i=setTimeout((function(){i=null,o.loadedSearches[s]=!0,e.call(o,s,n)}),t)}},R=(e,t,i)=>{var s,n=e.trigger,o={}
for(s in e.trigger=function(){var i=arguments[0]
if(-1===t.indexOf(i))return n.apply(e,arguments)
o[i]=arguments},i.apply(e,[]),e.trigger=n,o)n.apply(e,o[s])},H=(e,t=!1)=>{e&&(e.preventDefault(),t&&e.stopPropagation())},B=(e,t,i,s)=>{e.addEventListener(t,i,s)},K=(e,t)=>!!t&&(!!t[e]&&1===(t.altKey?1:0)+(t.ctrlKey?1:0)+(t.shiftKey?1:0)+(t.metaKey?1:0)),M=(e,t)=>{const i=e.getAttribute("id")
return i||(e.setAttribute("id",t),t)},Q=e=>e.replace(/[\\"']/g,"\\$&"),G=(e,t)=>{t&&e.append(t)}
function U(e,t){var i=Object.assign({},j,t),s=i.dataAttr,n=i.labelField,o=i.valueField,r=i.disabledField,l=i.optgroupField,a=i.optgroupLabelField,c=i.optgroupValueField,d=e.tagName.toLowerCase(),p=e.getAttribute("placeholder")||e.getAttribute("data-placeholder")
if(!p&&!i.allowEmptyOption){let t=e.querySelector('option[value=""]')
t&&(p=t.textContent)}var u,h,g,f,v,m,O={placeholder:p,options:[],optgroups:[],items:[],maxItems:null}
return"select"===d?(h=O.options,g={},f=1,v=e=>{var t=Object.assign({},e.dataset),i=s&&t[s]
return"string"==typeof i&&i.length&&(t=Object.assign(t,JSON.parse(i))),t},m=(e,t)=>{var s=q(e.value)
if(null!=s&&(s||i.allowEmptyOption)){if(g.hasOwnProperty(s)){if(t){var a=g[s][l]
a?Array.isArray(a)?a.push(t):g[s][l]=[a,t]:g[s][l]=t}}else{var c=v(e)
c[n]=c[n]||e.textContent,c[o]=c[o]||s,c[r]=c[r]||e.disabled,c[l]=c[l]||t,c.$option=e,g[s]=c,h.push(c)}e.selected&&O.items.push(s)}},O.maxItems=e.hasAttribute("multiple")?null:1,y(e.children,(e=>{var t,i,s
"optgroup"===(u=e.tagName.toLowerCase())?((s=v(t=e))[a]=s[a]||t.getAttribute("label")||"",s[c]=s[c]||f++,s[r]=s[r]||t.disabled,O.optgroups.push(s),i=s[c],y(t.children,(e=>{m(e,i)}))):"option"===u&&m(e)}))):(()=>{const t=e.getAttribute(s)
if(t)O.options=JSON.parse(t),y(O.options,(e=>{O.items.push(e[o])}))
else{var r=e.value.trim()||""
if(!i.allowEmptyOption&&!r.length)return
const t=r.split(i.delimiter)
y(t,(e=>{const t={}
t[n]=e,t[o]=e,O.options.push(t)})),O.items=t}})(),Object.assign({},j,O,t)}var W=0
class J extends(function(e){return e.plugins={},class extends e{constructor(...e){super(...e),this.plugins={names:[],settings:{},requested:{},loaded:{}}}static define(t,i){e.plugins[t]={name:t,fn:i}}initializePlugins(e){var t,i
const s=this,n=[]
if(Array.isArray(e))e.forEach((e=>{"string"==typeof e?n.push(e):(s.plugins.settings[e.name]=e.options,n.push(e.name))}))
else if(e)for(t in e)e.hasOwnProperty(t)&&(s.plugins.settings[t]=e[t],n.push(t))
for(;i=n.shift();)s.require(i)}loadPlugin(t){var i=this,s=i.plugins,n=e.plugins[t]
if(!e.plugins.hasOwnProperty(t))throw new Error('Unable to find "'+t+'" plugin')
s.requested[t]=!0,s.loaded[t]=n.fn.apply(i,[i.plugins.settings[t]||{}]),s.names.push(t)}require(e){var t=this,i=t.plugins
if(!t.plugins.loaded.hasOwnProperty(e)){if(i.requested[e])throw new Error('Plugin has circular dependency ("'+e+'")')
t.loadPlugin(e)}return i.loaded[e]}}}(t)){constructor(e,t){var i
super(),this.order=0,this.isOpen=!1,this.isDisabled=!1,this.isInvalid=!1,this.isValid=!0,this.isLocked=!1,this.isFocused=!1,this.isInputHidden=!1,this.isSetup=!1,this.ignoreFocus=!1,this.hasOptions=!1,this.lastValue="",this.caretPos=0,this.loading=0,this.loadedSearches={},this.activeOption=null,this.activeItems=[],this.optgroups={},this.options={},this.userOptions={},this.items=[],W++
var s=w(e)
if(s.tomselect)throw new Error("Tom Select already initialized on this element")
s.tomselect=this,i=(window.getComputedStyle&&window.getComputedStyle(s,null)).getPropertyValue("direction")
const n=U(s,t)
this.settings=n,this.input=s,this.tabIndex=s.tabIndex||0,this.is_select_tag="select"===s.tagName.toLowerCase(),this.rtl=/rtl/i.test(i),this.inputId=M(s,"tomselect-"+W),this.isRequired=s.required,this.sifter=new b(this.options,{diacritics:n.diacritics}),n.mode=n.mode||(1===n.maxItems?"single":"multi"),"boolean"!=typeof n.hideSelected&&(n.hideSelected="multi"===n.mode),"boolean"!=typeof n.hidePlaceholder&&(n.hidePlaceholder="multi"!==n.mode)
var o=n.createFilter
"function"!=typeof o&&("string"==typeof o&&(o=new RegExp(o)),o instanceof RegExp?n.createFilter=e=>o.test(e):n.createFilter=()=>!0),this.initializePlugins(n.plugins),this.setupCallbacks(),this.setupTemplates()
const r=w("<div>"),l=w("<div>"),a=this._render("dropdown"),c=w('<div role="listbox" tabindex="-1">'),d=this.input.getAttribute("class")||"",p=n.mode
var u
if(C(r,n.wrapperClass,d,p),C(l,n.controlClass),G(r,l),C(a,n.dropdownClass,p),n.copyClassesToDropdown&&C(a,d),C(c,n.dropdownContentClass),G(a,c),w(n.dropdownParent||r).appendChild(a),n.hasOwnProperty("controlInput"))n.controlInput?(u=w(n.controlInput),this.focus_node=u):(u=w("<input/>"),this.focus_node=l)
else{u=w('<input type="text" autocomplete="off" size="1" />')
y(["autocorrect","autocapitalize","autocomplete"],(e=>{s.getAttribute(e)&&P(u,{[e]:s.getAttribute(e)})})),u.tabIndex=-1,l.appendChild(u),this.focus_node=u}this.wrapper=r,this.dropdown=a,this.dropdown_content=c,this.control=l,this.control_input=u,this.setup()}setup(){const e=this,t=e.settings,i=e.control_input,s=e.dropdown,n=e.dropdown_content,o=e.wrapper,r=e.control,l=e.input,a=e.focus_node,c={passive:!0},d=e.inputId+"-ts-dropdown"
P(n,{id:d}),P(a,{role:"combobox","aria-haspopup":"listbox","aria-expanded":"false","aria-controls":d})
const p=M(a,e.inputId+"-ts-control"),u="label[for='"+(e=>e.replace(/['"\\]/g,"\\$&"))(e.inputId)+"']",h=document.querySelector(u),g=e.focus.bind(e)
if(h){B(h,"click",g),P(h,{for:p})
const t=M(h,e.inputId+"-ts-label")
P(a,{"aria-labelledby":t}),P(n,{"aria-labelledby":t})}if(o.style.width=l.style.width,e.plugins.names.length){const t="plugin-"+e.plugins.names.join(" plugin-")
C([o,s],t)}(null===t.maxItems||t.maxItems>1)&&e.is_select_tag&&P(l,{multiple:"multiple"}),e.settings.placeholder&&P(i,{placeholder:t.placeholder}),!e.settings.splitOn&&e.settings.delimiter&&(e.settings.splitOn=new RegExp("\\s*"+v(e.settings.delimiter)+"+\\s*")),t.load&&t.loadThrottle&&(t.load=z(t.load,t.loadThrottle)),e.control_input.type=l.type,B(s,"click",(t=>{const i=k(t.target,"[data-selectable]")
i&&(e.onOptionSelect(t,i),H(t,!0))})),B(r,"click",(t=>{var s=k(t.target,"[data-ts-item]",r)
s&&e.onItemSelect(t,s)?H(t,!0):""==i.value&&(e.onClick(),H(t,!0))})),B(i,"mousedown",(e=>{""!==i.value&&e.stopPropagation()})),B(a,"keydown",(t=>e.onKeyDown(t))),B(i,"keypress",(t=>e.onKeyPress(t))),B(i,"input",(t=>e.onInput(t))),B(a,"resize",(()=>e.positionDropdown()),c),B(a,"blur",(t=>e.onBlur(t))),B(a,"focus",(t=>e.onFocus(t))),B(a,"paste",(t=>e.onPaste(t)))
const f=t=>{const i=t.composedPath()[0]
if(!o.contains(i)&&!s.contains(i))return e.isFocused&&e.blur(),void e.inputState()
H(t,!0)}
var m=()=>{e.isOpen&&e.positionDropdown()}
B(document,"mousedown",f),B(window,"scroll",m,c),B(window,"resize",m,c),this._destroy=()=>{document.removeEventListener("mousedown",f),window.removeEventListener("sroll",m),window.removeEventListener("resize",m),h&&h.removeEventListener("click",g)},this.revertSettings={innerHTML:l.innerHTML,tabIndex:l.tabIndex},l.tabIndex=-1,l.insertAdjacentElement("afterend",e.wrapper),e.sync(!1),t.items=[],delete t.optgroups,delete t.options,B(l,"invalid",(t=>{e.isValid&&(e.isValid=!1,e.isInvalid=!0,e.refreshState())})),e.updateOriginalInput(),e.refreshItems(),e.close(!1),e.inputState(),e.isSetup=!0,l.disabled?e.disable():e.enable(),e.on("change",this.onChange),C(l,"tomselected","ts-hidden-accessible"),e.trigger("initialize"),!0===t.preload&&e.preload()}setupOptions(e=[],t=[]){this.addOptions(e),y(t,(e=>{this.registerOptionGroup(e)}))}setupTemplates(){var e=this,t=e.settings.labelField,i=e.settings.optgroupLabelField,s={optgroup:e=>{let t=document.createElement("div")
return t.className="optgroup",t.appendChild(e.options),t},optgroup_header:(e,t)=>'<div class="optgroup-header">'+t(e[i])+"</div>",option:(e,i)=>"<div>"+i(e[t])+"</div>",item:(e,i)=>"<div>"+i(e[t])+"</div>",option_create:(e,t)=>'<div class="create">Add <strong>'+t(e.input)+"</strong>&hellip;</div>",no_results:()=>'<div class="no-results">No results found</div>',loading:()=>'<div class="spinner"></div>',not_loading:()=>{},dropdown:()=>"<div></div>"}
e.settings.render=Object.assign({},s,e.settings.render)}setupCallbacks(){var e,t,i={initialize:"onInitialize",change:"onChange",item_add:"onItemAdd",item_remove:"onItemRemove",item_select:"onItemSelect",clear:"onClear",option_add:"onOptionAdd",option_remove:"onOptionRemove",option_clear:"onOptionClear",optgroup_add:"onOptionGroupAdd",optgroup_remove:"onOptionGroupRemove",optgroup_clear:"onOptionGroupClear",dropdown_open:"onDropdownOpen",dropdown_close:"onDropdownClose",type:"onType",load:"onLoad",focus:"onFocus",blur:"onBlur"}
for(e in i)(t=this.settings[i[e]])&&this.on(e,t)}sync(e=!0){const t=this,i=e?U(t.input,{delimiter:t.settings.delimiter}):t.settings
t.setupOptions(i.options,i.optgroups),t.setValue(i.items,!0),t.lastQuery=null}onClick(){var e=this
if(e.activeItems.length>0)return e.clearActiveItems(),void e.focus()
e.isFocused&&e.isOpen?e.blur():e.focus()}onMouseDown(){}onChange(){_(this.input,"input"),_(this.input,"change")}onPaste(e){var t=this
t.isFull()||t.isInputHidden||t.isLocked?H(e):t.settings.splitOn&&setTimeout((()=>{var e=t.inputValue()
if(e.match(t.settings.splitOn)){var i=e.trim().split(t.settings.splitOn)
y(i,(e=>{t.createItem(e)}))}}),0)}onKeyPress(e){var t=this
if(!t.isLocked){var i=String.fromCharCode(e.keyCode||e.which)
return t.settings.create&&"multi"===t.settings.mode&&i===t.settings.delimiter?(t.createItem(),void H(e)):void 0}H(e)}onKeyDown(e){var t=this
if(t.isLocked)9!==e.keyCode&&H(e)
else{switch(e.keyCode){case 65:if(K(V,e))return H(e),void t.selectAll()
break
case 27:return t.isOpen&&(H(e,!0),t.close()),void t.clearActiveItems()
case 40:if(!t.isOpen&&t.hasOptions)t.open()
else if(t.activeOption){let e=t.getAdjacent(t.activeOption,1)
e&&t.setActiveOption(e)}return void H(e)
case 38:if(t.activeOption){let e=t.getAdjacent(t.activeOption,-1)
e&&t.setActiveOption(e)}return void H(e)
case 13:return void(t.isOpen&&t.activeOption?(t.onOptionSelect(e,t.activeOption),H(e)):t.settings.create&&t.createItem()&&H(e))
case 37:return void t.advanceSelection(-1,e)
case 39:return void t.advanceSelection(1,e)
case 9:return void(t.settings.selectOnTab&&(t.isOpen&&t.activeOption&&(t.onOptionSelect(e,t.activeOption),H(e)),t.settings.create&&t.createItem()&&H(e)))
case 8:case 46:return void t.deleteSelection(e)}t.isInputHidden&&!K(V,e)&&H(e)}}onInput(e){var t=this
if(!t.isLocked){var i=t.inputValue()
t.lastValue!==i&&(t.lastValue=i,t.settings.shouldLoad.call(t,i)&&t.load(i),t.refreshOptions(),t.trigger("type",i))}}onFocus(e){var t=this,i=t.isFocused
if(t.isDisabled)return t.blur(),void H(e)
t.ignoreFocus||(t.isFocused=!0,"focus"===t.settings.preload&&t.preload(),i||t.trigger("focus"),t.activeItems.length||(t.showInput(),t.refreshOptions(!!t.settings.openOnFocus)),t.refreshState())}onBlur(e){if(!1!==document.hasFocus()){var t=this
if(t.isFocused){t.isFocused=!1,t.ignoreFocus=!1
var i=()=>{t.close(),t.setActiveItem(),t.setCaret(t.items.length),t.trigger("blur")}
t.settings.create&&t.settings.createOnBlur?t.createItem(null,!1,i):i()}}}onOptionSelect(e,t){var i,s=this
t&&(t.parentElement&&t.parentElement.matches("[data-disabled]")||(t.classList.contains("create")?s.createItem(null,!0,(()=>{s.settings.closeAfterSelect&&s.close()})):void 0!==(i=t.dataset.value)&&(s.lastQuery=null,s.addItem(i),s.settings.closeAfterSelect&&s.close(),!s.settings.hideSelected&&e.type&&/click/.test(e.type)&&s.setActiveOption(t))))}onItemSelect(e,t){var i=this
return!i.isLocked&&"multi"===i.settings.mode&&(H(e),i.setActiveItem(t,e),!0)}canLoad(e){return!!this.settings.load&&!this.loadedSearches.hasOwnProperty(e)}load(e){const t=this
if(!t.canLoad(e))return
C(t.wrapper,t.settings.loadingClass),t.loading++
const i=t.loadCallback.bind(t)
t.settings.load.call(t,e,i)}loadCallback(e,t){const i=this
i.loading=Math.max(i.loading-1,0),i.lastQuery=null,i.clearActiveOption(),i.setupOptions(e,t),i.refreshOptions(i.isFocused&&!i.isInputHidden),i.loading||S(i.wrapper,i.settings.loadingClass),i.trigger("load",e,t)}preload(){var e=this.wrapper.classList
e.contains("preloaded")||(e.add("preloaded"),this.load(""))}setTextboxValue(e=""){var t=this.control_input
t.value!==e&&(t.value=e,_(t,"update"),this.lastValue=e)}getValue(){return this.is_select_tag&&this.input.hasAttribute("multiple")?this.items:this.items.join(this.settings.delimiter)}setValue(e,t){R(this,t?[]:["change"],(()=>{this.clear(t),this.addItems(e,t)}))}setMaxItems(e){0===e&&(e=null),this.settings.maxItems=e,this.refreshState()}setActiveItem(e,t){var i,s,n,o,r,l,a=this
if("single"!==a.settings.mode){if(!e)return a.clearActiveItems(),void(a.isFocused&&a.showInput())
if("click"===(i=t&&t.type.toLowerCase())&&K("shiftKey",t)&&a.activeItems.length){for(l=a.getLastActive(),(n=Array.prototype.indexOf.call(a.control.children,l))>(o=Array.prototype.indexOf.call(a.control.children,e))&&(r=n,n=o,o=r),s=n;s<=o;s++)e=a.control.children[s],-1===a.activeItems.indexOf(e)&&a.setActiveItemClass(e)
H(t)}else"click"===i&&K(V,t)||"keydown"===i&&K("shiftKey",t)?e.classList.contains("active")?a.removeActiveItem(e):a.setActiveItemClass(e):(a.clearActiveItems(),a.setActiveItemClass(e))
a.hideInput(),a.isFocused||a.focus()}}setActiveItemClass(e){const t=this,i=t.control.querySelector(".last-active")
i&&S(i,"last-active"),C(e,"active last-active"),t.trigger("item_select",e),-1==t.activeItems.indexOf(e)&&t.activeItems.push(e)}removeActiveItem(e){var t=this.activeItems.indexOf(e)
this.activeItems.splice(t,1),S(e,"active")}clearActiveItems(){S(this.activeItems,"active"),this.activeItems=[]}setActiveOption(e){e!==this.activeOption&&(this.clearActiveOption(),e&&(this.activeOption=e,P(this.focus_node,{"aria-activedescendant":e.getAttribute("id")}),P(e,{"aria-selected":"true"}),C(e,"active"),this.scrollToOption(e)))}scrollToOption(e,t){if(!e)return
const i=this.dropdown_content,s=i.clientHeight,n=i.scrollTop||0,o=e.offsetHeight,r=e.getBoundingClientRect().top-i.getBoundingClientRect().top+n
r+o>s+n?this.scroll(r-s+o,t):r<n&&this.scroll(r,t)}scroll(e,t){const i=this.dropdown_content
t&&(i.style.scrollBehavior=t),i.scrollTop=e,i.style.scrollBehavior=""}clearActiveOption(){this.activeOption&&(S(this.activeOption,"active"),P(this.activeOption,{"aria-selected":null})),this.activeOption=null,P(this.focus_node,{"aria-activedescendant":null})}selectAll(){if("single"===this.settings.mode)return
const e=this.controlChildren()
e.length&&(this.hideInput(),this.close(),this.activeItems=e,C(e,"active"))}inputState(){var e=this
e.control.contains(e.control_input)&&(P(e.control_input,{placeholder:e.settings.placeholder}),e.activeItems.length>0||!e.isFocused&&e.settings.hidePlaceholder&&e.items.length>0?(e.setTextboxValue(),e.isInputHidden=!0):(e.settings.hidePlaceholder&&e.items.length>0&&P(e.control_input,{placeholder:""}),e.isInputHidden=!1),e.wrapper.classList.toggle("input-hidden",e.isInputHidden))}hideInput(){this.inputState()}showInput(){this.inputState()}inputValue(){return this.control_input.value.trim()}focus(){var e=this
e.isDisabled||(e.ignoreFocus=!0,e.control_input.offsetWidth?e.control_input.focus():e.focus_node.focus(),setTimeout((()=>{e.ignoreFocus=!1,e.onFocus()}),0))}blur(){this.focus_node.blur(),this.onBlur()}getScoreFunction(e){return this.sifter.getScoreFunction(e,this.getSearchOptions())}getSearchOptions(){var e=this.settings,t=e.sortField
return"string"==typeof e.sortField&&(t=[{field:e.sortField}]),{fields:e.searchField,conjunction:e.searchConjunction,sort:t,nesting:e.nesting}}search(e){var t,i,s,n=this,o=this.getSearchOptions()
if(n.settings.score&&"function"!=typeof(s=n.settings.score.call(n,e)))throw new Error('Tom Select "score" setting must be a function that returns a function')
if(e!==n.lastQuery?(n.lastQuery=e,i=n.sifter.search(e,Object.assign(o,{score:s})),n.currentResults=i):i=Object.assign({},n.currentResults),n.settings.hideSelected)for(t=i.items.length-1;t>=0;t--){let e=q(i.items[t].id)
e&&-1!==n.items.indexOf(e)&&i.items.splice(t,1)}return i}refreshOptions(e=!0){var t,i,s,n,o,r,l,a,c,d,p
const u={},h=[]
var g,f=this,v=f.inputValue(),m=f.search(v),O=f.activeOption,b=f.settings.shouldOpen||!1,w=f.dropdown_content
for(O&&(c=O.dataset.value,d=O.closest("[data-group]")),n=m.items.length,"number"==typeof f.settings.maxOptions&&(n=Math.min(n,f.settings.maxOptions)),n>0&&(b=!0),t=0;t<n;t++){let e=m.items[t].id,n=f.options[e],l=f.getOption(e,!0)
for(f.settings.hideSelected||l.classList.toggle("selected",f.items.includes(e)),o=n[f.settings.optgroupField]||"",i=0,s=(r=Array.isArray(o)?o:[o])&&r.length;i<s;i++)o=r[i],f.optgroups.hasOwnProperty(o)||(o=""),u.hasOwnProperty(o)||(u[o]=document.createDocumentFragment(),h.push(o)),i>0&&(l=l.cloneNode(!0),P(l,{id:n.$id+"-clone-"+i,"aria-selected":null}),l.classList.add("ts-cloned"),S(l,"active")),c==e&&d&&d.dataset.group===o&&(O=l),u[o].appendChild(l)}this.settings.lockOptgroupOrder&&h.sort(((e,t)=>(f.optgroups[e]&&f.optgroups[e].$order||0)-(f.optgroups[t]&&f.optgroups[t].$order||0))),l=document.createDocumentFragment(),y(h,(e=>{if(f.optgroups.hasOwnProperty(e)&&u[e].children.length){let t=document.createDocumentFragment(),i=f.render("optgroup_header",f.optgroups[e])
G(t,i),G(t,u[e])
let s=f.render("optgroup",{group:f.optgroups[e],options:t})
G(l,s)}else G(l,u[e])})),w.innerHTML="",G(w,l),f.settings.highlight&&(g=w.querySelectorAll("span.highlight"),Array.prototype.forEach.call(g,(function(e){var t=e.parentNode
t.replaceChild(e.firstChild,e),t.normalize()})),m.query.length&&m.tokens.length&&y(m.tokens,(e=>{T(w,e.regex)})))
var _=e=>{let t=f.render(e,{input:v})
return t&&(b=!0,w.insertBefore(t,w.firstChild)),t}
if(f.loading?_("loading"):f.settings.shouldLoad.call(f,v)?0===m.items.length&&_("no_results"):_("not_loading"),(a=f.canCreate(v))&&(p=_("option_create")),f.hasOptions=m.items.length>0||a,b){if(m.items.length>0){if(!w.contains(O)&&"single"===f.settings.mode&&f.items.length&&(O=f.getOption(f.items[0])),!w.contains(O)){let e=0
p&&!f.settings.addPrecedence&&(e=1),O=f.selectable()[e]}}else p&&(O=p)
e&&!f.isOpen&&(f.open(),f.scrollToOption(O,"auto")),f.setActiveOption(O)}else f.clearActiveOption(),e&&f.isOpen&&f.close(!1)}selectable(){return this.dropdown_content.querySelectorAll("[data-selectable]")}addOption(e,t=!1){const i=this
if(Array.isArray(e))return i.addOptions(e,t),!1
const s=q(e[i.settings.valueField])
return null!==s&&!i.options.hasOwnProperty(s)&&(e.$order=e.$order||++i.order,e.$id=i.inputId+"-opt-"+e.$order,i.options[s]=e,i.lastQuery=null,t&&(i.userOptions[s]=t,i.trigger("option_add",s,e)),s)}addOptions(e,t=!1){y(e,(e=>{this.addOption(e,t)}))}registerOption(e){return this.addOption(e)}registerOptionGroup(e){var t=q(e[this.settings.optgroupValueField])
return null!==t&&(e.$order=e.$order||++this.order,this.optgroups[t]=e,t)}addOptionGroup(e,t){var i
t[this.settings.optgroupValueField]=e,(i=this.registerOptionGroup(t))&&this.trigger("optgroup_add",i,t)}removeOptionGroup(e){this.optgroups.hasOwnProperty(e)&&(delete this.optgroups[e],this.clearCache(),this.trigger("optgroup_remove",e))}clearOptionGroups(){this.optgroups={},this.clearCache(),this.trigger("optgroup_clear")}updateOption(e,t){const i=this
var s,n
const o=q(e),r=q(t[i.settings.valueField])
if(null===o)return
if(!i.options.hasOwnProperty(o))return
if("string"!=typeof r)throw new Error("Value must be set in option data")
const l=i.getOption(o),a=i.getItem(o)
if(t.$order=t.$order||i.options[o].$order,delete i.options[o],i.uncacheValue(r),i.options[r]=t,l){if(i.dropdown_content.contains(l)){const e=i._render("option",t)
E(l,e),i.activeOption===l&&i.setActiveOption(e)}l.remove()}a&&(-1!==(n=i.items.indexOf(o))&&i.items.splice(n,1,r),s=i._render("item",t),a.classList.contains("active")&&C(s,"active"),E(a,s)),i.lastQuery=null}removeOption(e,t){const i=this
e=D(e),i.uncacheValue(e),delete i.userOptions[e],delete i.options[e],i.lastQuery=null,i.trigger("option_remove",e),i.removeItem(e,t)}clearOptions(){this.loadedSearches={},this.userOptions={},this.clearCache()
var e={}
y(this.options,((t,i)=>{this.items.indexOf(i)>=0&&(e[i]=this.options[i])})),this.options=this.sifter.items=e,this.lastQuery=null,this.trigger("option_clear")}getOption(e,t=!1){const i=q(e)
if(null!==i&&this.options.hasOwnProperty(i)){const e=this.options[i]
if(e.$div)return e.$div
if(t)return this._render("option",e)}return null}getAdjacent(e,t,i="option"){var s
if(!e)return null
s="item"==i?this.controlChildren():this.dropdown_content.querySelectorAll("[data-selectable]")
for(let i=0;i<s.length;i++)if(s[i]==e)return t>0?s[i+1]:s[i-1]
return null}getItem(e){if("object"==typeof e)return e
var t=q(e)
return null!==t?this.control.querySelector(`[data-value="${Q(t)}"]`):null}addItems(e,t){var i=this,s=Array.isArray(e)?e:[e]
for(let e=0,n=(s=s.filter((e=>-1===i.items.indexOf(e)))).length;e<n;e++)i.isPending=e<n-1,i.addItem(s[e],t)}addItem(e,t){R(this,t?[]:["change"],(()=>{var i,s
const n=this,o=n.settings.mode,r=q(e)
if((!r||-1===n.items.indexOf(r)||("single"===o&&n.close(),"single"!==o&&n.settings.duplicates))&&null!==r&&n.options.hasOwnProperty(r)&&("single"===o&&n.clear(t),"multi"!==o||!n.isFull())){if(i=n._render("item",n.options[r]),n.control.contains(i)&&(i=i.cloneNode(!0)),s=n.isFull(),n.items.splice(n.caretPos,0,r),n.insertAtCaret(i),n.isSetup){if(!n.isPending&&n.settings.hideSelected){let e=n.getOption(r),t=n.getAdjacent(e,1)
t&&n.setActiveOption(t)}n.isPending||n.refreshOptions(n.isFocused&&"single"!==o),0!=n.settings.closeAfterSelect&&n.isFull()?n.close():n.isPending||n.positionDropdown(),n.trigger("item_add",r,i),n.isPending||n.updateOriginalInput({silent:t})}(!n.isPending||!s&&n.isFull())&&(n.inputState(),n.refreshState())}}))}removeItem(e=null,t){const i=this
if(!(e=i.getItem(e)))return
var s,n
const o=e.dataset.value
s=L(e),e.remove(),e.classList.contains("active")&&(n=i.activeItems.indexOf(e),i.activeItems.splice(n,1),S(e,"active")),i.items.splice(s,1),i.lastQuery=null,!i.settings.persist&&i.userOptions.hasOwnProperty(o)&&i.removeOption(o,t),s<i.caretPos&&i.setCaret(i.caretPos-1),i.updateOriginalInput({silent:t}),i.refreshState(),i.positionDropdown(),i.trigger("item_remove",o,e)}createItem(e=null,t=!0,i=(()=>{})){var s,n=this,o=n.caretPos
if(e=e||n.inputValue(),!n.canCreate(e))return i(),!1
n.lock()
var r=!1,l=e=>{if(n.unlock(),!e||"object"!=typeof e)return i()
var s=q(e[n.settings.valueField])
if("string"!=typeof s)return i()
n.setTextboxValue(),n.addOption(e,!0),n.setCaret(o),n.addItem(s),n.refreshOptions(t&&"single"!==n.settings.mode),i(e),r=!0}
return s="function"==typeof n.settings.create?n.settings.create.call(this,e,l):{[n.settings.labelField]:e,[n.settings.valueField]:e},r||l(s),!0}refreshItems(){var e=this
e.lastQuery=null,e.isSetup&&e.addItems(e.items),e.updateOriginalInput(),e.refreshState()}refreshState(){const e=this
e.refreshValidityState()
const t=e.isFull(),i=e.isLocked
e.wrapper.classList.toggle("rtl",e.rtl)
const s=e.wrapper.classList
var n
s.toggle("focus",e.isFocused),s.toggle("disabled",e.isDisabled),s.toggle("required",e.isRequired),s.toggle("invalid",!e.isValid),s.toggle("locked",i),s.toggle("full",t),s.toggle("input-active",e.isFocused&&!e.isInputHidden),s.toggle("dropdown-active",e.isOpen),s.toggle("has-options",(n=e.options,0===Object.keys(n).length)),s.toggle("has-items",e.items.length>0)}refreshValidityState(){var e=this
e.input.checkValidity&&(e.isValid=e.input.checkValidity(),e.isInvalid=!e.isValid)}isFull(){return null!==this.settings.maxItems&&this.items.length>=this.settings.maxItems}updateOriginalInput(e={}){const t=this
var i,s
const n=t.input.querySelector('option[value=""]')
if(t.is_select_tag){const e=[]
function o(i,s,o){return i||(i=w('<option value="'+N(s)+'">'+N(o)+"</option>")),i!=n&&t.input.append(i),e.push(i),i.selected=!0,i}t.input.querySelectorAll("option:checked").forEach((e=>{e.selected=!1})),0==t.items.length&&"single"==t.settings.mode?o(n,"",""):t.items.forEach((n=>{if(i=t.options[n],s=i[t.settings.labelField]||"",e.includes(i.$option)){o(t.input.querySelector(`option[value="${Q(n)}"]:not(:checked)`),n,s)}else i.$option=o(i.$option,n,s)}))}else t.input.value=t.getValue()
t.isSetup&&(e.silent||t.trigger("change",t.getValue()))}open(){var e=this
e.isLocked||e.isOpen||"multi"===e.settings.mode&&e.isFull()||(e.isOpen=!0,P(e.focus_node,{"aria-expanded":"true"}),e.refreshState(),I(e.dropdown,{visibility:"hidden",display:"block"}),e.positionDropdown(),I(e.dropdown,{visibility:"visible",display:"block"}),e.focus(),e.trigger("dropdown_open",e.dropdown))}close(e=!0){var t=this,i=t.isOpen
e&&(t.setTextboxValue(),"single"===t.settings.mode&&t.items.length&&t.hideInput()),t.isOpen=!1,P(t.focus_node,{"aria-expanded":"false"}),I(t.dropdown,{display:"none"}),t.settings.hideSelected&&t.clearActiveOption(),t.refreshState(),i&&t.trigger("dropdown_close",t.dropdown)}positionDropdown(){if("body"===this.settings.dropdownParent){var e=this.control,t=e.getBoundingClientRect(),i=e.offsetHeight+t.top+window.scrollY,s=t.left+window.scrollX
I(this.dropdown,{width:t.width+"px",top:i+"px",left:s+"px"})}}clear(e){var t=this
if(t.items.length){var i=t.controlChildren()
y(i,(e=>{t.removeItem(e,!0)})),t.showInput(),e||t.updateOriginalInput(),t.trigger("clear")}}insertAtCaret(e){const t=this,i=t.caretPos,s=t.control
s.insertBefore(e,s.children[i]),t.setCaret(i+1)}deleteSelection(e){var t,i,s,n,o,r=this
t=e&&8===e.keyCode?-1:1,i={start:(o=r.control_input).selectionStart||0,length:(o.selectionEnd||0)-(o.selectionStart||0)}
const l=[]
if(r.activeItems.length)n=F(r.activeItems,t),s=L(n),t>0&&s++,y(r.activeItems,(e=>l.push(e)))
else if((r.isFocused||"single"===r.settings.mode)&&r.items.length){const e=r.controlChildren()
t<0&&0===i.start&&0===i.length?l.push(e[r.caretPos-1]):t>0&&i.start===r.inputValue().length&&l.push(e[r.caretPos])}const a=l.map((e=>e.dataset.value))
if(!a.length||"function"==typeof r.settings.onDelete&&!1===r.settings.onDelete.call(r,a,e))return!1
for(H(e,!0),void 0!==s&&r.setCaret(s);l.length;)r.removeItem(l.pop())
return r.showInput(),r.positionDropdown(),r.refreshOptions(!1),!0}advanceSelection(e,t){var i,s,n=this
n.rtl&&(e*=-1),n.inputValue().length||(K(V,t)||K("shiftKey",t)?(s=(i=n.getLastActive(e))?i.classList.contains("active")?n.getAdjacent(i,e,"item"):i:e>0?n.control_input.nextElementSibling:n.control_input.previousElementSibling)&&(s.classList.contains("active")&&n.removeActiveItem(i),n.setActiveItemClass(s)):n.moveCaret(e))}moveCaret(e){}getLastActive(e){let t=this.control.querySelector(".last-active")
if(t)return t
var i=this.control.querySelectorAll(".active")
return i?F(i,e):void 0}setCaret(e){this.caretPos=this.items.length}controlChildren(){return Array.from(this.control.querySelectorAll("[data-ts-item]"))}lock(){this.close(),this.isLocked=!0,this.refreshState()}unlock(){this.isLocked=!1,this.refreshState()}disable(){var e=this
e.input.disabled=!0,e.control_input.disabled=!0,e.focus_node.tabIndex=-1,e.isDisabled=!0,e.lock()}enable(){var e=this
e.input.disabled=!1,e.control_input.disabled=!1,e.focus_node.tabIndex=e.tabIndex,e.isDisabled=!1,e.unlock()}destroy(){var e=this,t=e.revertSettings
e.trigger("destroy"),e.off(),e.wrapper.remove(),e.dropdown.remove(),e.input.innerHTML=t.innerHTML,e.input.tabIndex=t.tabIndex,S(e.input,"tomselected","ts-hidden-accessible"),e._destroy(),delete e.input.tomselect}render(e,t){return"function"!=typeof this.settings.render[e]?null:this._render(e,t)}_render(e,t){var i,s,n=""
const o=this
return"option"!==e&&"item"!=e||(n=D(t[o.settings.valueField])),null==(s=o.settings.render[e].call(this,t,N))||(s=w(s),"option"===e||"option_create"===e?t[o.settings.disabledField]?P(s,{"aria-disabled":"true"}):P(s,{"data-selectable":""}):"optgroup"===e&&(i=t.group[o.settings.optgroupValueField],P(s,{"data-group":i}),t.group[o.settings.disabledField]&&P(s,{"data-disabled":""})),"option"!==e&&"item"!==e||(P(s,{"data-value":n}),"item"===e?(C(s,o.settings.itemClass),P(s,{"data-ts-item":""})):(C(s,o.settings.optionClass),P(s,{role:"option",id:t.$id}),o.options[n].$div=s))),s}clearCache(){y(this.options,((e,t)=>{e.$div&&(e.$div.remove(),delete e.$div)}))}uncacheValue(e){const t=this.getOption(e)
t&&t.remove()}canCreate(e){return this.settings.create&&e.length>0&&this.settings.createFilter.call(this,e)}hook(e,t,i){var s=this,n=s[t]
s[t]=function(){var t,o
return"after"===e&&(t=n.apply(s,arguments)),o=i.apply(s,arguments),"instead"===e?o:("before"===e&&(t=n.apply(s,arguments)),t)}}}return J.define("change_listener",(function(){B(this.input,"change",(()=>{this.sync()}))})),J.define("checkbox_options",(function(){var e=this,t=e.onOptionSelect
e.settings.hideSelected=!1
var i=function(e){setTimeout((()=>{var t=e.querySelector("input")
e.classList.contains("selected")?t.checked=!0:t.checked=!1}),1)}
e.hook("after","setupTemplates",(()=>{var t=e.settings.render.option
e.settings.render.option=(i,s)=>{var n=w(t.call(e,i,s)),o=document.createElement("input")
o.addEventListener("click",(function(e){H(e)})),o.type="checkbox"
const r=q(i[e.settings.valueField])
return r&&e.items.indexOf(r)>-1&&(o.checked=!0),n.prepend(o),n}})),e.on("item_remove",(t=>{var s=e.getOption(t)
s&&(s.classList.remove("selected"),i(s))})),e.hook("instead","onOptionSelect",((s,n)=>{if(n.classList.contains("selected"))return n.classList.remove("selected"),e.removeItem(n.dataset.value),e.refreshOptions(),void H(s,!0)
t.call(e,s,n),i(n)}))})),J.define("clear_button",(function(e){const t=this,i=Object.assign({className:"clear-button",title:"Clear All",html:e=>`<div class="${e.className}" title="${e.title}">&times;</div>`},e)
t.on("initialize",(()=>{var e=w(i.html(i))
e.addEventListener("click",(e=>{t.clear(),"single"===t.settings.mode&&t.settings.allowEmptyOption&&t.addItem(""),e.preventDefault(),e.stopPropagation()})),t.control.appendChild(e)}))})),J.define("drag_drop",(function(){var e=this
if(!$.fn.sortable)throw new Error('The "drag_drop" plugin requires jQuery UI "sortable".')
if("multi"===e.settings.mode){var t=e.lock,i=e.unlock
e.hook("instead","lock",(()=>{var i=$(e.control).data("sortable")
return i&&i.disable(),t.call(e)})),e.hook("instead","unlock",(()=>{var t=$(e.control).data("sortable")
return t&&t.enable(),i.call(e)})),e.on("initialize",(()=>{var t=$(e.control).sortable({items:"[data-value]",forcePlaceholderSize:!0,disabled:e.isLocked,start:(e,i)=>{i.placeholder.css("width",i.helper.css("width")),t.css({overflow:"visible"})},stop:()=>{t.css({overflow:"hidden"})
var i=[]
t.children("[data-value]").each((function(){this.dataset.value&&i.push(this.dataset.value)})),e.setValue(i)}})}))}})),J.define("dropdown_header",(function(e){const t=this,i=Object.assign({title:"Untitled",headerClass:"dropdown-header",titleRowClass:"dropdown-header-title",labelClass:"dropdown-header-label",closeClass:"dropdown-header-close",html:e=>'<div class="'+e.headerClass+'"><div class="'+e.titleRowClass+'"><span class="'+e.labelClass+'">'+e.title+'</span><a class="'+e.closeClass+'">&times;</a></div></div>'},e)
t.on("initialize",(()=>{var e=w(i.html(i)),s=e.querySelector("."+i.closeClass)
s&&s.addEventListener("click",(e=>{H(e,!0),t.close()})),t.dropdown.insertBefore(e,t.dropdown.firstChild)}))})),J.define("caret_position",(function(){var e=this
e.hook("instead","setCaret",(t=>{"single"!==e.settings.mode&&e.control.contains(e.control_input)?(t=Math.max(0,Math.min(e.items.length,t)))==e.caretPos||e.isPending||e.controlChildren().forEach(((i,s)=>{s<t?e.control_input.insertAdjacentElement("beforebegin",i):e.control.appendChild(i)})):t=e.items.length,e.caretPos=t})),e.hook("instead","moveCaret",(t=>{if(!e.isFocused)return
const i=e.getLastActive(t)
if(i){const s=L(i)
e.setCaret(t>0?s+1:s),e.setActiveItem()}else e.setCaret(e.caretPos+t)}))})),J.define("dropdown_input",(function(){var e=this
e.settings.shouldOpen=!0,e.hook("before","setup",(()=>{e.focus_node=e.control,C(e.control_input,"dropdown-input")
const t=w('<div class="dropdown-input-wrap">')
t.append(e.control_input),e.dropdown.insertBefore(t,e.dropdown.firstChild)})),e.on("initialize",(()=>{e.control_input.addEventListener("keydown",(t=>{switch(t.keyCode){case 27:return e.isOpen&&(H(t,!0),e.close()),void e.clearActiveItems()
case 9:e.focus_node.tabIndex=-1}return e.onKeyDown.call(e,t)})),e.on("blur",(()=>{e.focus_node.tabIndex=e.isDisabled?-1:e.tabIndex})),e.on("dropdown_open",(()=>{e.control_input.focus()}))
const t=e.onBlur
e.hook("instead","onBlur",(i=>{if(!i||i.relatedTarget!=e.control_input)return t.call(e)})),B(e.control_input,"blur",(()=>e.onBlur())),e.hook("before","close",(()=>{e.isOpen&&e.focus_node.focus()}))}))})),J.define("input_autogrow",(function(){var e=this
e.on("initialize",(()=>{var t=document.createElement("span"),i=e.control_input
t.style.cssText="position:absolute; top:-99999px; left:-99999px; width:auto; padding:0; white-space:pre; ",e.wrapper.appendChild(t)
for(const e of["letterSpacing","fontSize","fontFamily","fontWeight","textTransform"])t.style[e]=i.style[e]
var s=()=>{e.items.length>0?(t.textContent=i.value,i.style.width=t.clientWidth+"px"):i.style.width=""}
s(),e.on("update item_add item_remove",s),B(i,"input",s),B(i,"keyup",s),B(i,"blur",s),B(i,"update",s)}))})),J.define("no_backspace_delete",(function(){var e=this,t=e.deleteSelection
this.hook("instead","deleteSelection",(i=>!!e.activeItems.length&&t.call(e,i)))})),J.define("no_active_items",(function(){this.hook("instead","setActiveItem",(()=>{})),this.hook("instead","selectAll",(()=>{}))})),J.define("optgroup_columns",(function(){var e=this,t=e.onKeyDown
e.hook("instead","onKeyDown",(i=>{var s,n,o,r
if(!e.isOpen||37!==i.keyCode&&39!==i.keyCode)return t.call(e,i)
r=k(e.activeOption,"[data-group]"),s=L(e.activeOption,"[data-selectable]"),r&&(r=37===i.keyCode?r.previousSibling:r.nextSibling)&&(n=(o=r.querySelectorAll("[data-selectable]"))[Math.min(o.length-1,s)])&&e.setActiveOption(n)}))})),J.define("remove_button",(function(e){const t=Object.assign({label:"&times;",title:"Remove",className:"remove",append:!0},e)
var i=this
if(t.append){var s='<a href="javascript:void(0)" class="'+t.className+'" tabindex="-1" title="'+N(t.title)+'">'+t.label+"</a>"
i.hook("after","setupTemplates",(()=>{var e=i.settings.render.item
i.settings.render.item=(t,n)=>{var o=w(e.call(i,t,n)),r=w(s)
return o.appendChild(r),B(r,"mousedown",(e=>{H(e,!0)})),B(r,"click",(e=>{if(H(e,!0),!i.isLocked){var t=o.dataset.value
i.removeItem(t),i.refreshOptions(!1)}})),o}}))}})),J.define("restore_on_backspace",(function(e){const t=this,i=Object.assign({text:e=>e[t.settings.labelField]},e)
t.on("item_remove",(function(e){if(""===t.control_input.value.trim()){var s=t.options[e]
s&&t.setTextboxValue(i.text.call(t,s))}}))})),J.define("virtual_scroll",(function(){const e=this,t=e.canLoad,i=e.clearActiveOption,s=e.loadCallback
var n,o={},r=!1
if(!e.settings.firstUrl)throw"virtual_scroll plugin requires a firstUrl() method"
function l(t){return!("number"==typeof e.settings.maxOptions&&n.children.length>=e.settings.maxOptions)&&!(!(t in o)||!o[t])}e.settings.sortField=[{field:"$order"},{field:"$score"}],e.setNextUrl=function(e,t){o[e]=t},e.getUrl=function(t){if(t in o){const e=o[t]
return o[t]=!1,e}return o={},e.settings.firstUrl(t)},e.hook("instead","clearActiveOption",(()=>{if(!r)return i.call(e)})),e.hook("instead","canLoad",(i=>i in o?l(i):t.call(e,i))),e.hook("instead","loadCallback",((t,i)=>{r||e.clearOptions(),s.call(e,t,i),r=!1})),e.hook("after","refreshOptions",(()=>{const t=e.lastValue
var i
l(t)?(i=e.render("loading_more",{query:t}))&&i.setAttribute("data-selectable",""):t in o&&!n.querySelector(".no-results")&&(i=e.render("no_more_results",{query:t})),i&&(C(i,e.settings.optionClass),n.append(i))})),e.on("initialize",(()=>{n=e.dropdown_content,e.settings.render=Object.assign({},{loading_more:function(){return'<div class="loading-more-results">Loading more results ... </div>'},no_more_results:function(){return'<div class="no-more-results">No more results</div>'}},e.settings.render),n.addEventListener("scroll",(function(){n.clientHeight/(n.scrollHeight-n.scrollTop)<.95||l(e.lastValue)&&(r||(r=!0,e.load.call(e,e.lastValue)))}))}))})),J}))
var tomSelect=function(e,t){return new TomSelect(e,t)}
//# sourceMappingURL=tom-select.complete.min.js.map
+334
View File
@@ -0,0 +1,334 @@
/**
* tom-select.css (v2.0.0-rc.4)
* Copyright (c) contributors
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this
* file except in compliance with the License. You may obtain a copy of the License at:
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under
* the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
* ANY KIND, either express or implied. See the License for the specific language
* governing permissions and limitations under the License.
*
*/
.ts-wrapper.plugin-drag_drop.multi > .ts-control > div.ui-sortable-placeholder {
visibility: visible !important;
background: #f2f2f2 !important;
background: rgba(0, 0, 0, 0.06) !important;
border: 0 none !important;
box-shadow: inset 0 0 12px 4px #fff; }
.ts-wrapper.plugin-drag_drop .ui-sortable-placeholder::after {
content: '!';
visibility: hidden; }
.ts-wrapper.plugin-drag_drop .ui-sortable-helper {
box-shadow: 0 2px 5px rgba(0, 0, 0, 0.2); }
.plugin-checkbox_options .option input {
margin-right: 0.5rem; }
.plugin-clear_button .ts-control {
padding-right: calc( 1em + (3 * 6px)) !important; }
.plugin-clear_button .clear-button {
opacity: 0;
position: absolute;
top: 8px;
right: calc(8px - 6px);
margin-right: 0 !important;
background: transparent !important;
transition: opacity 0.5s;
cursor: pointer; }
.plugin-clear_button.single .clear-button {
right: calc(8px - 6px + 2rem); }
.plugin-clear_button.focus.has-items .clear-button,
.plugin-clear_button:hover.has-items .clear-button {
opacity: 1; }
.ts-wrapper .dropdown-header {
position: relative;
padding: 10px 8px;
border-bottom: 1px solid #d0d0d0;
background: #f8f8f8;
border-radius: 3px 3px 0 0; }
.ts-wrapper .dropdown-header-close {
position: absolute;
right: 8px;
top: 50%;
color: #303030;
opacity: 0.4;
margin-top: -12px;
line-height: 20px;
font-size: 20px !important; }
.ts-wrapper .dropdown-header-close:hover {
color: black; }
.plugin-dropdown_input.focus.dropdown-active .ts-control {
box-shadow: none;
border: 1px solid #d0d0d0; }
.plugin-dropdown_input .dropdown-input {
border: 1px solid #d0d0d0;
border-width: 0 0 1px 0;
display: block;
padding: 8px 8px;
box-shadow: none;
width: 100%;
background: transparent; }
.ts-wrapper.plugin-input_autogrow.has-items .ts-control > input {
min-width: 0; }
.ts-wrapper.plugin-input_autogrow.has-items.focus .ts-control > input {
flex: none;
min-width: 4px; }
.ts-wrapper.plugin-input_autogrow.has-items.focus .ts-control > input::-webkit-input-placeholder {
color: transparent; }
.ts-wrapper.plugin-input_autogrow.has-items.focus .ts-control > input::-ms-input-placeholder {
color: transparent; }
.ts-wrapper.plugin-input_autogrow.has-items.focus .ts-control > input::placeholder {
color: transparent; }
.ts-dropdown.plugin-optgroup_columns .ts-dropdown-content {
display: flex; }
.ts-dropdown.plugin-optgroup_columns .optgroup {
border-right: 1px solid #f2f2f2;
border-top: 0 none;
flex-grow: 1;
flex-basis: 0;
min-width: 0; }
.ts-dropdown.plugin-optgroup_columns .optgroup:last-child {
border-right: 0 none; }
.ts-dropdown.plugin-optgroup_columns .optgroup:before {
display: none; }
.ts-dropdown.plugin-optgroup_columns .optgroup-header {
border-top: 0 none; }
.ts-wrapper.plugin-remove_button .item {
display: inline-flex;
align-items: center;
padding-right: 0 !important; }
.ts-wrapper.plugin-remove_button .item .remove {
color: inherit;
text-decoration: none;
vertical-align: middle;
display: inline-block;
padding: 2px 6px;
border-left: 1px solid #d0d0d0;
border-radius: 0 2px 2px 0;
box-sizing: border-box;
margin-left: 6px; }
.ts-wrapper.plugin-remove_button .item .remove:hover {
background: rgba(0, 0, 0, 0.05); }
.ts-wrapper.plugin-remove_button .item.active .remove {
border-left-color: #cacaca; }
.ts-wrapper.plugin-remove_button.disabled .item .remove:hover {
background: none; }
.ts-wrapper.plugin-remove_button.disabled .item .remove {
border-left-color: white; }
.ts-wrapper.plugin-remove_button .remove-single {
position: absolute;
right: 0;
top: 0;
font-size: 23px; }
.ts-wrapper {
position: relative; }
.ts-dropdown,
.ts-control,
.ts-control input {
color: #303030;
font-family: inherit;
font-size: 13px;
line-height: 18px;
font-smoothing: inherit; }
.ts-control,
.ts-wrapper.single.input-active .ts-control {
background: #fff;
cursor: text; }
.ts-control {
border: 1px solid #d0d0d0;
padding: 8px 8px;
width: 100%;
overflow: hidden;
position: relative;
z-index: 1;
box-sizing: border-box;
box-shadow: none;
border-radius: 3px;
display: flex;
flex-wrap: wrap; }
.ts-wrapper.multi.has-items .ts-control {
padding: calc( 8px - 2px - 0) 8px calc( 8px - 2px - 3px - 0); }
.full .ts-control {
background-color: #fff; }
.disabled .ts-control,
.disabled .ts-control * {
cursor: default !important; }
.focus .ts-control {
box-shadow: none; }
.ts-control > * {
vertical-align: baseline;
display: inline-block; }
.ts-wrapper.multi .ts-control > div {
cursor: pointer;
margin: 0 3px 3px 0;
padding: 2px 6px;
background: #f2f2f2;
color: #303030;
border: 0 solid #d0d0d0; }
.ts-wrapper.multi .ts-control > div.active {
background: #e8e8e8;
color: #303030;
border: 0 solid #cacaca; }
.ts-wrapper.multi.disabled .ts-control > div, .ts-wrapper.multi.disabled .ts-control > div.active {
color: #7d7c7c;
background: white;
border: 0 solid white; }
.ts-control > input {
flex: 1 1 auto;
min-width: 7rem;
display: inline-block !important;
padding: 0 !important;
min-height: 0 !important;
max-height: none !important;
max-width: 100% !important;
margin: 0 !important;
text-indent: 0 !important;
border: 0 none !important;
background: none !important;
line-height: inherit !important;
-webkit-user-select: auto !important;
-moz-user-select: auto !important;
-ms-user-select: auto !important;
user-select: auto !important;
box-shadow: none !important; }
.ts-control > input::-ms-clear {
display: none; }
.ts-control > input:focus {
outline: none !important; }
.has-items .ts-control > input {
margin: 0 4px !important; }
.ts-control.rtl {
text-align: right; }
.ts-control.rtl.single .ts-control:after {
left: 15px;
right: auto; }
.ts-control.rtl .ts-control > input {
margin: 0 4px 0 -2px !important; }
.disabled .ts-control {
opacity: 0.5;
background-color: #fafafa; }
.input-hidden .ts-control > input {
opacity: 0;
position: absolute;
left: -10000px; }
.ts-dropdown {
position: absolute;
top: 100%;
left: 0;
width: 100%;
z-index: 10;
border: 1px solid #d0d0d0;
background: #fff;
margin: 0.25rem 0 0 0;
border-top: 0 none;
box-sizing: border-box;
box-shadow: 0 1px 3px rgba(0, 0, 0, 0.1);
border-radius: 0 0 3px 3px; }
.ts-dropdown [data-selectable] {
cursor: pointer;
overflow: hidden; }
.ts-dropdown [data-selectable] .highlight {
background: rgba(125, 168, 208, 0.2);
border-radius: 1px; }
.ts-dropdown .option,
.ts-dropdown .optgroup-header,
.ts-dropdown .no-results,
.ts-dropdown .create {
padding: 5px 8px; }
.ts-dropdown .option, .ts-dropdown [data-disabled], .ts-dropdown [data-disabled] [data-selectable].option {
cursor: inherit;
opacity: 0.5; }
.ts-dropdown [data-selectable].option {
opacity: 1;
cursor: pointer; }
.ts-dropdown .optgroup:first-child .optgroup-header {
border-top: 0 none; }
.ts-dropdown .optgroup-header {
color: #303030;
background: #fff;
cursor: default; }
.ts-dropdown .create:hover,
.ts-dropdown .option:hover,
.ts-dropdown .active {
background-color: #f5fafd;
color: #495c68; }
.ts-dropdown .create:hover.create,
.ts-dropdown .option:hover.create,
.ts-dropdown .active.create {
color: #495c68; }
.ts-dropdown .create {
color: rgba(48, 48, 48, 0.5); }
.ts-dropdown .spinner {
display: inline-block;
width: 30px;
height: 30px;
margin: 5px 8px; }
.ts-dropdown .spinner:after {
content: " ";
display: block;
width: 24px;
height: 24px;
margin: 3px;
border-radius: 50%;
border: 5px solid #d0d0d0;
border-color: #d0d0d0 transparent #d0d0d0 transparent;
animation: lds-dual-ring 1.2s linear infinite; }
@keyframes lds-dual-ring {
0% {
transform: rotate(0deg); }
100% {
transform: rotate(360deg); } }
.ts-dropdown-content {
overflow-y: auto;
overflow-x: hidden;
max-height: 200px;
overflow-scrolling: touch;
scroll-behavior: smooth; }
.ts-hidden-accessible {
border: 0 !important;
clip: rect(0 0 0 0) !important;
-webkit-clip-path: inset(50%) !important;
clip-path: inset(50%) !important;
height: 1px !important;
overflow: hidden !important;
padding: 0 !important;
position: absolute !important;
width: 1px !important;
white-space: nowrap !important; }
/*# sourceMappingURL=tom-select.css.map */
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+10
View File
@@ -0,0 +1,10 @@
"""
Memory System for AI Agents.
Temporal + Semantic Memory Architecture using PostgreSQL with pgvector.
"""
from .temporal_semantic_memory import TemporalSemanticMemory
from .visualizer import MemoryVisualizer, LiveSearchTracer
__all__ = ["TemporalSemanticMemory", "MemoryVisualizer", "LiveSearchTracer"]
__version__ = "0.1.0"
+336
View File
@@ -0,0 +1,336 @@
"""
Coreference resolution for memory units.
Ensures every memory unit is self-contained by replacing pronouns
with their actual referents.
"""
import spacy
from typing import List, Dict, Optional
from fastcoref import FCoref
import threading
def get_nlp():
"""Get or load spaCy model."""
try:
return spacy.load("en_core_web_sm")
except OSError:
raise Exception("spaCy model not found. Run: uv pip install https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.7.1/en_core_web_sm-3.7.1-py3-none-any.whl")
# Global fastcoref model instance (singleton pattern)
_fastcoref_model = None
_fastcoref_lock = threading.Lock()
def get_fastcoref_model():
"""Get or load FastCoref model (singleton pattern)."""
global _fastcoref_model
if _fastcoref_model is None:
with _fastcoref_lock:
if _fastcoref_model is None:
# Use CPU by default, can be configured with device='cuda:0' for GPU
_fastcoref_model = FCoref(device='cpu')
return _fastcoref_model
def resolve_pronouns_in_text(text: str, context_sentences: List[str] = None) -> str:
"""
Resolve pronouns to their referents to make text self-contained.
Strategy:
1. Identify pronouns in the text
2. Look for named entities in the same sentence or previous sentences
3. Replace pronouns with the most likely referent based on:
- Gender agreement
- Number agreement (singular/plural)
- Proximity (closer entities more likely)
Args:
text: The sentence to resolve
context_sentences: Previous sentences for context (optional)
Returns:
Text with pronouns resolved
"""
nlp = get_nlp()
# Parse the target sentence
doc = nlp(text)
# Collect all sentences for context
all_text = text
if context_sentences:
# Add previous sentences for context
all_text = " ".join(context_sentences) + " " + text
full_doc = nlp(all_text)
# Extract entities with their positions
entities = []
for ent in full_doc.ents:
if ent.label_ in ['PERSON', 'ORG', 'GPE', 'PRODUCT']:
entities.append({
'text': ent.text,
'label': ent.label_,
'start': ent.start_char,
'end': ent.end_char,
})
# Check if sentence already has a named entity subject
has_named_subject = False
for token in doc:
if token.dep_ in ['nsubj', 'nsubjpass'] and token.pos_ == 'PROPN':
has_named_subject = True
break
# Find pronouns and anaphoric references that need resolution
pronouns_to_replace = []
for token in doc:
# Handle pronouns (he, she, it, they)
if token.pos_ == 'PRON' and token.dep_ in ['nsubj', 'nsubjpass']:
# Subject pronouns that need resolution
pron_lower = token.text.lower()
# Skip if sentence already has a named subject earlier
if has_named_subject and any(
t.dep_ in ['nsubj', 'nsubjpass'] and t.pos_ == 'PROPN' and t.i < token.i
for t in doc
):
continue
# Skip if it's already a proper name or demonstrative
if pron_lower in ['i', 'you', 'we', 'this', 'that', 'these', 'those']:
continue
# Find the best entity to replace it with
referent = find_best_referent(
pronoun=token,
entities=entities,
doc=full_doc
)
if referent:
pronouns_to_replace.append({
'pronoun': token,
'referent': referent,
'start': token.idx,
'end': token.idx + len(token.text)
})
# Handle definite noun phrases (e.g., "The project")
elif token.text.lower() == 'the' and token.head.pos_ == 'NOUN':
# Check if this "the X" phrase is a subject
if token.head.dep_ in ['nsubj', 'nsubjpass']:
# Try to find what "the X" refers to
noun = token.head.text
# Look for indefinite mentions earlier ("a project", "an organization")
for ent_token in reversed(list(full_doc)):
if ent_token.text.lower() == noun.lower():
# Found a matching noun - check if it has indefinite article
if any(child.text.lower() in ['a', 'an'] for child in ent_token.children):
# Replace "the project" with "the Python project" or similar
# Get the full noun phrase
descriptors = []
for child in ent_token.children:
if child.pos_ in ['ADJ', 'PROPN', 'NOUN'] and child.i < ent_token.i:
descriptors.append(child.text)
if descriptors:
full_phrase = ' '.join(descriptors) + ' ' + noun
# Calculate span to replace
span_start = token.idx
span_end = token.head.idx + len(token.head.text)
pronouns_to_replace.append({
'pronoun': token,
'referent': 'the ' + full_phrase,
'start': span_start,
'end': span_end
})
break
# Replace pronouns with referents (in reverse order to maintain indices)
result = text
for item in reversed(pronouns_to_replace):
start = item['start']
end = item['end']
result = result[:start] + item['referent'] + result[end:]
return result
def find_best_referent(
pronoun,
entities: List[Dict],
doc
) -> Optional[str]:
"""
Find the best entity referent for a pronoun.
Uses:
- Gender agreement (he/she -> PERSON)
- Number agreement (singular/plural)
- Entity type (he/she -> PERSON, it -> ORG/PRODUCT)
- Proximity (closer entities preferred)
"""
pron_text = pronoun.text.lower()
# Determine pronoun properties
is_singular = pron_text in ['he', 'she', 'it', 'him', 'her']
is_plural = pron_text in ['they', 'them']
is_person = pron_text in ['he', 'she', 'him', 'her']
is_thing = pron_text in ['it']
# Score each entity
candidates = []
for entity in entities:
score = 0.0
# Proximity score (entities closer to pronoun are better)
# Since entities come from context, those appearing later (higher start position) are closer
proximity_score = entity['start'] / 1000.0 # Normalize by position
score += proximity_score
# Type matching
if is_person and entity['label'] == 'PERSON':
score += 2.0 # Strong preference for person entities
elif is_thing and entity['label'] in ['ORG', 'PRODUCT', 'GPE']:
score += 2.0 # Organizations/products for "it"
# Recency: prefer entities that appear just before the pronoun
if entity['end'] < pronoun.idx:
distance = pronoun.idx - entity['end']
recency = 1.0 / (1.0 + distance / 100.0)
score += recency
candidates.append((entity['text'], score))
# Return the highest scoring candidate
if candidates:
candidates.sort(key=lambda x: x[1], reverse=True)
return candidates[0][0]
return None
def resolve_sentences_fast(sentences: List[str]) -> List[str]:
"""
Fast batch coreference resolution using FastCoref.
This is significantly faster than the sequential spaCy-based approach:
- Processes entire document in one pass (O(n) instead of O(n²))
- Uses efficient batching and neural model
- Can process 2.8K documents in 25 seconds on GPU
Args:
sentences: List of sentences to resolve
Returns:
List of resolved sentences (self-contained)
"""
if not sentences:
return []
# Join sentences into a single document for batch processing
# Add markers to track sentence boundaries
full_text = " ".join(sentences)
# Get the fastcoref model
model = get_fastcoref_model()
# Predict coreferences in batch
preds = model.predict(texts=[full_text])
if not preds or len(preds) == 0:
# No coreferences found, return original sentences
return sentences
# Get the first (and only) result
result = preds[0]
# Get clusters as text strings
clusters = result.get_clusters(as_strings=True)
if not clusters:
return sentences
# Build a replacement map: pronoun -> main referent
replacements = {}
for cluster in clusters:
if len(cluster) < 2:
continue
# The first mention is typically the most complete referent
main_referent = cluster[0]
# Map all other mentions (pronouns/short references) to the main referent
for mention in cluster[1:]:
mention_lower = mention.lower()
# Only replace if it's likely a pronoun or short reference
if len(mention.split()) <= 2 and any(
pron in mention_lower
for pron in ['he', 'she', 'it', 'they', 'him', 'her', 'them', 'his', 'her', 'their', 'the']
):
replacements[mention] = main_referent
# Apply replacements to each sentence
resolved = []
for sentence in sentences:
resolved_sentence = sentence
for mention, referent in replacements.items():
# Case-insensitive replacement but preserve capitalization context
if mention in resolved_sentence:
resolved_sentence = resolved_sentence.replace(mention, referent)
resolved.append(resolved_sentence)
return resolved
def resolve_sentences(sentences: List[str]) -> List[str]:
"""
Resolve pronouns across a list of sentences.
Uses FastCoref for efficient batch processing.
Falls back to legacy spaCy method if FastCoref fails.
Args:
sentences: List of sentences to resolve
Returns:
List of resolved sentences (self-contained)
"""
try:
return resolve_sentences_fast(sentences)
except Exception as e:
# Fallback to legacy method
print(f"FastCoref failed ({e}), falling back to spaCy method")
return resolve_sentences_legacy(sentences)
def resolve_sentences_legacy(sentences: List[str]) -> List[str]:
"""
Legacy sequential pronoun resolution (slower, O(n²) complexity).
Kept as fallback in case FastCoref is unavailable or fails.
Args:
sentences: List of sentences to resolve
Returns:
List of resolved sentences (self-contained)
"""
resolved = []
for i, sentence in enumerate(sentences):
# Use all previous sentences as context
context = resolved[:i] if i > 0 else []
# Resolve pronouns in this sentence
resolved_sentence = resolve_pronouns_in_text(sentence, context)
resolved.append(resolved_sentence)
return resolved
+373
View File
@@ -0,0 +1,373 @@
"""
Entity extraction and resolution for memory system.
Uses spaCy for entity extraction and implements resolution logic
to disambiguate entities across memory units.
"""
import spacy
from typing import List, Dict, Optional, Set
from difflib import SequenceMatcher
# Load spaCy model (singleton)
_nlp = None
def get_nlp():
"""Get or load spaCy model."""
global _nlp
if _nlp is None:
_nlp = spacy.load("en_core_web_sm")
return _nlp
def extract_entities(text: str) -> List[Dict[str, any]]:
"""
Extract entities from text using spaCy.
Args:
text: Input text
Returns:
List of entities with text, type, and span info
"""
nlp = get_nlp()
doc = nlp(text)
entities = []
for ent in doc.ents:
# Filter to important entity types
if ent.label_ in ['PERSON', 'ORG', 'GPE', 'LOC', 'PRODUCT', 'EVENT']:
entities.append({
'text': ent.text,
'type': ent.label_,
'start': ent.start_char,
'end': ent.end_char,
})
return entities
class EntityResolver:
"""
Resolves entities to canonical IDs with disambiguation.
"""
def __init__(self, db_conn):
"""
Initialize entity resolver.
Args:
db_conn: psycopg2 database connection
"""
self.conn = db_conn
def resolve_entity(
self,
agent_id: str,
entity_text: str,
entity_type: str,
context: str,
nearby_entities: List[Dict],
unit_event_date,
) -> str:
"""
Resolve an entity to a canonical entity ID.
Args:
agent_id: Agent ID (entities are scoped to agents)
entity_text: Entity text ("Alice", "Google", etc.)
entity_type: Entity type (PERSON, ORG, etc.)
context: Context where entity appears
nearby_entities: Other entities in the same unit
unit_event_date: When this unit was created
Returns:
Entity ID (creates new entity if needed)
"""
cursor = self.conn.cursor()
try:
# Find candidate entities with same type and similar name
cursor.execute(
"""
SELECT id, canonical_name, metadata, last_seen
FROM entities
WHERE agent_id = %s
AND entity_type = %s
AND (
canonical_name ILIKE %s
OR canonical_name ILIKE %s
OR %s ILIKE canonical_name || '%%'
)
ORDER BY mention_count DESC
""",
(agent_id, entity_type, entity_text, f"%{entity_text}%", entity_text)
)
candidates = cursor.fetchall()
if not candidates:
# New entity - create it
return self._create_entity(
cursor, agent_id, entity_text, entity_type, unit_event_date
)
# Score candidates based on:
# 1. Name similarity
# 2. Context overlap (TODO: could use embeddings)
# 3. Co-occurring entities
# 4. Temporal proximity
best_candidate = None
best_score = 0.0
best_name_similarity = 0.0
nearby_entity_set = {e['text'].lower() for e in nearby_entities if e['text'] != entity_text}
for candidate_id, canonical_name, metadata, last_seen in candidates:
score = 0.0
# 1. Name similarity (0-1)
name_similarity = SequenceMatcher(
None,
entity_text.lower(),
canonical_name.lower()
).ratio()
score += name_similarity * 0.5
# 2. Co-occurring entities (0-0.5)
# Get entities that co-occurred with this candidate before
# Use the materialized co-occurrence cache for fast lookup
cursor.execute(
"""
SELECT e.canonical_name, ec.cooccurrence_count
FROM entity_cooccurrences ec
JOIN entities e ON (
CASE
WHEN ec.entity_id_1 = %s THEN ec.entity_id_2
WHEN ec.entity_id_2 = %s THEN ec.entity_id_1
END = e.id
)
WHERE ec.entity_id_1 = %s OR ec.entity_id_2 = %s
""",
(candidate_id, candidate_id, candidate_id, candidate_id)
)
co_entities = {row[0].lower() for row in cursor.fetchall()}
# Check overlap with nearby entities
overlap = len(nearby_entity_set & co_entities)
if nearby_entity_set:
co_entity_score = overlap / len(nearby_entity_set)
score += co_entity_score * 0.3
# 3. Temporal proximity (0-0.2)
if last_seen:
days_diff = abs((unit_event_date - last_seen).total_seconds() / 86400)
if days_diff < 7: # Within a week
temporal_score = max(0, 1.0 - (days_diff / 7))
score += temporal_score * 0.2
if score > best_score:
best_score = score
best_candidate = candidate_id
best_name_similarity = name_similarity
# Threshold for considering it the same entity
# For PERSON entities with exact name match, use lower threshold
threshold = 0.4 if entity_type == 'PERSON' and best_name_similarity >= 0.95 else 0.6
if best_score > threshold:
# Update entity
cursor.execute(
"""
UPDATE entities
SET mention_count = mention_count + 1,
last_seen = %s
WHERE id = %s
""",
(unit_event_date, best_candidate)
)
return best_candidate
else:
# Not confident - create new entity
return self._create_entity(
cursor, agent_id, entity_text, entity_type, unit_event_date
)
finally:
cursor.close()
def _create_entity(
self,
cursor,
agent_id: str,
entity_text: str,
entity_type: str,
event_date,
) -> str:
"""
Create a new entity.
Args:
cursor: Database cursor
agent_id: Agent ID
entity_text: Entity text
entity_type: Entity type
event_date: When first seen
Returns:
Entity ID
"""
cursor.execute(
"""
INSERT INTO entities (agent_id, canonical_name, entity_type, first_seen, last_seen, mention_count)
VALUES (%s, %s, %s, %s, %s, 1)
RETURNING id
""",
(agent_id, entity_text, entity_type, event_date, event_date)
)
entity_id = cursor.fetchone()[0]
return entity_id
def link_unit_to_entity(self, unit_id: str, entity_id: str):
"""
Link a memory unit to an entity.
Also updates co-occurrence cache with other entities in the same unit.
Args:
unit_id: Memory unit ID
entity_id: Entity ID
"""
cursor = self.conn.cursor()
try:
# Insert unit-entity link
cursor.execute(
"""
INSERT INTO unit_entities (unit_id, entity_id)
VALUES (%s, %s)
ON CONFLICT DO NOTHING
""",
(unit_id, entity_id)
)
# Update co-occurrence cache: find other entities in this unit
cursor.execute(
"""
SELECT entity_id
FROM unit_entities
WHERE unit_id = %s AND entity_id != %s
""",
(unit_id, entity_id)
)
other_entities = [row[0] for row in cursor.fetchall()]
# Update co-occurrences for each pair
for other_entity_id in other_entities:
self._update_cooccurrence(cursor, entity_id, other_entity_id)
finally:
cursor.close()
def _update_cooccurrence(self, cursor, entity_id_1: str, entity_id_2: str):
"""
Update the co-occurrence cache for two entities.
Uses CHECK constraint ordering (entity_id_1 < entity_id_2) to avoid duplicates.
Args:
cursor: Database cursor
entity_id_1: First entity ID
entity_id_2: Second entity ID
"""
# Ensure consistent ordering (smaller UUID first)
if entity_id_1 > entity_id_2:
entity_id_1, entity_id_2 = entity_id_2, entity_id_1
cursor.execute(
"""
INSERT INTO entity_cooccurrences (entity_id_1, entity_id_2, cooccurrence_count, last_cooccurred)
VALUES (%s, %s, 1, NOW())
ON CONFLICT (entity_id_1, entity_id_2)
DO UPDATE SET
cooccurrence_count = entity_cooccurrences.cooccurrence_count + 1,
last_cooccurred = NOW()
""",
(entity_id_1, entity_id_2)
)
def get_units_by_entity(self, entity_id: str, limit: int = 100) -> List[str]:
"""
Get all units that mention an entity.
Args:
entity_id: Entity ID
limit: Max results
Returns:
List of unit IDs
"""
cursor = self.conn.cursor()
try:
cursor.execute(
"""
SELECT unit_id
FROM unit_entities
WHERE entity_id = %s
ORDER BY unit_id
LIMIT %s
""",
(entity_id, limit)
)
return [row[0] for row in cursor.fetchall()]
finally:
cursor.close()
def get_entity_by_text(
self,
agent_id: str,
entity_text: str,
entity_type: Optional[str] = None
) -> Optional[str]:
"""
Find an entity by text (for query resolution).
Args:
agent_id: Agent ID
entity_text: Entity text to search for
entity_type: Optional entity type filter
Returns:
Entity ID if found, None otherwise
"""
cursor = self.conn.cursor()
try:
if entity_type:
cursor.execute(
"""
SELECT id FROM entities
WHERE agent_id = %s
AND entity_type = %s
AND canonical_name ILIKE %s
ORDER BY mention_count DESC
LIMIT 1
""",
(agent_id, entity_type, entity_text)
)
else:
cursor.execute(
"""
SELECT id FROM entities
WHERE agent_id = %s
AND canonical_name ILIKE %s
ORDER BY mention_count DESC
LIMIT 1
""",
(agent_id, entity_text)
)
row = cursor.fetchone()
return row[0] if row else None
finally:
cursor.close()
+229
View File
@@ -0,0 +1,229 @@
"""
LLM client for fact extraction and other AI-powered operations.
Uses OpenAI-compatible API (works with Groq, OpenAI, etc.)
"""
import os
import json
import re
from typing import List, Dict, Optional, Literal
from openai import AsyncOpenAI
from pydantic import BaseModel, Field
class ExtractedFact(BaseModel):
"""A single extracted fact from text."""
fact: str = Field(
description="Self-contained factual statement with subject + action + context"
)
speaker: str = Field(
default="narrator",
description="Who said this (name or 'narrator' if not a conversation)"
)
type: Literal["biographical", "event", "opinion", "recommendation", "description", "relationship"] = Field(
description="Category of the fact"
)
confidence: Literal["high", "medium", "low"] = Field(
default="medium",
description="Confidence level of the extraction"
)
class FactExtractionResponse(BaseModel):
"""Response containing all extracted facts."""
facts: List[ExtractedFact] = Field(
description="List of extracted factual statements"
)
def split_into_sentences(text: str) -> List[str]:
"""
Fast sentence splitter using regex.
Splits on periods, exclamation marks, and question marks followed by whitespace or end of string.
Args:
text: Input text to split
Returns:
List of sentences
"""
# Split on sentence boundaries: .!? followed by space/newline/end
sentences = re.split(r'(?<=[.!?])\s+', text)
return [s.strip() for s in sentences if s.strip()]
def chunk_text(text: str, max_chars: int = 120000) -> List[str]:
"""
Split text into chunks at sentence boundaries.
Keeps chunks under max_chars (~30k tokens assuming 1 token ≈ 4 chars).
This prevents hitting output token limits on large documents.
Args:
text: Input text to chunk
max_chars: Maximum characters per chunk (default 120k ≈ 30k tokens)
Returns:
List of text chunks, each under max_chars
"""
# If text is small enough, return as-is
if len(text) <= max_chars:
return [text]
sentences = split_into_sentences(text)
chunks = []
current_chunk = []
current_length = 0
for sentence in sentences:
sentence_length = len(sentence)
# If single sentence exceeds max_chars, split it forcefully
if sentence_length > max_chars:
# Save current chunk if any
if current_chunk:
chunks.append(' '.join(current_chunk))
current_chunk = []
current_length = 0
# Split long sentence into smaller pieces
for i in range(0, len(sentence), max_chars):
chunks.append(sentence[i:i + max_chars])
continue
# If adding this sentence would exceed limit, start new chunk
if current_length + sentence_length + 1 > max_chars:
chunks.append(' '.join(current_chunk))
current_chunk = [sentence]
current_length = sentence_length
else:
current_chunk.append(sentence)
current_length += sentence_length + 1 # +1 for space
# Add remaining chunk
if current_chunk:
chunks.append(' '.join(current_chunk))
return chunks
def get_llm_client() -> AsyncOpenAI:
"""
Get configured async LLM client.
Supports:
- Groq (default): Set GROQ_API_KEY and optionally GROQ_BASE_URL
- OpenAI: Set OPENAI_API_KEY
Returns:
Configured AsyncOpenAI client
"""
# Check for Groq configuration first
groq_api_key = os.getenv('GROQ_API_KEY')
if groq_api_key:
base_url = os.getenv('GROQ_BASE_URL', 'https://api.groq.com/openai/v1')
return AsyncOpenAI(
api_key=groq_api_key,
base_url=base_url
)
# Fall back to OpenAI
openai_api_key = os.getenv('OPENAI_API_KEY')
if openai_api_key:
return AsyncOpenAI(api_key=openai_api_key)
raise ValueError(
"No LLM API key found. Set GROQ_API_KEY or OPENAI_API_KEY environment variable."
)
async def extract_facts_from_text(
text: str,
model: str = "openai/gpt-oss-20b",
temperature: float = 0.1,
max_tokens: int = 65000,
chunk_size: int = 60000
) -> List[Dict[str, str]]:
client = get_llm_client()
# Chunk text if necessary
chunks = chunk_text(text, max_chars=chunk_size)
all_facts = []
for i, chunk in enumerate(chunks):
prompt = f"""You are extracting facts from text for an AI memory system. Each fact will be stored and retrieved later.
## CRITICAL: Facts must be DETAILED and COMPREHENSIVE
Each fact should:
1. Be SELF-CONTAINED - readable without the original context
2. Include ALL relevant details: WHO, WHAT, WHERE, WHEN, WHY, HOW
3. Preserve specific names, dates, numbers, locations, relationships
4. Resolve pronouns to actual names/entities
5. Include surrounding context that makes the fact meaningful
6. Capture nuances, reasons, causes, and implications
## What to EXTRACT:
- Biographical information (jobs, roles, backgrounds, experiences)
- Events (what happened, when, where, who was involved, why)
- Opinions and beliefs (who believes what and why)
- Recommendations and advice (specific suggestions with reasoning)
- Descriptions (detailed explanations of how things work)
- Relationships (connections between people, organizations, concepts)
## What to SKIP:
- Greetings, thank yous, acknowledgments
- Filler words ("um", "uh", "like")
- Pure reactions without content ("wow", "cool")
- Incomplete thoughts
## EXAMPLES of GOOD facts (detailed, comprehensive):
Input: "Alice mentioned she works at Google in Mountain View. She joined the AI team last year and loves working on large language models."
GOOD: "Alice works at Google in Mountain View on the AI team, which she joined last year, and she loves working on large language models"
BAD: "Alice works at Google" (too short, missing context)
Input: "Bob said he's been hiking every weekend in Yosemite because it helps him clear his mind after stressful work weeks."
GOOD: "Bob has been hiking every weekend in Yosemite because it helps him clear his mind after stressful work weeks"
BAD: "Bob hikes in Yosemite" (missing frequency, reason, and context)
Input: "The new algorithm reduced latency by 40% compared to the baseline by using a novel caching strategy."
GOOD: "The new algorithm reduced latency by 40% compared to the baseline by using a novel caching strategy"
BAD: "The algorithm is faster" (missing numbers, comparison, and method)
## TEXT TO EXTRACT FROM:
{chunk}
Remember: Include ALL details, names, numbers, reasons, and context. Facts should be rich and informative, not summaries."""
# Use parse() for structured outputs with Pydantic models
response = await client.beta.chat.completions.parse(
model=model,
messages=[
{
"role": "system",
"content": "You extract detailed, comprehensive facts from text. Preserve all context, details, and nuances. Never summarize or shorten - include everything relevant."
},
{
"role": "user",
"content": prompt
}
],
temperature=temperature,
max_tokens=max_tokens,
response_format=FactExtractionResponse
)
# Extract the parsed response
extraction_response = response.choices[0].message.parsed
# Convert to dict format and add to aggregate
chunk_facts = [fact.model_dump() for fact in extraction_response.facts]
all_facts.extend(chunk_facts)
# Log progress for large documents
if len(chunks) > 1:
print(f"Processed chunk {i + 1}/{len(chunks)}: extracted {len(chunk_facts)} facts")
return all_facts
+949
View File
@@ -0,0 +1,949 @@
"""
Temporal + Semantic + Entity Memory System for AI Agents.
This implements a sophisticated memory architecture that combines:
1. Temporal links: Memories connected by time proximity
2. Semantic links: Memories connected by meaning/similarity
3. Entity links: Memories connected by shared entities (PERSON, ORG, etc.)
4. Spreading activation: Search through the graph with activation decay
5. Dynamic weighting: Recency and frequency-based importance
"""
import os
from datetime import datetime, timedelta, timezone
from typing import Any, Dict, List, Optional, Tuple
import psycopg2
from psycopg2.extras import RealDictCursor, execute_values
from pgvector.psycopg2 import register_vector
from sentence_transformers import SentenceTransformer
from dotenv import load_dotenv
import asyncio
import time
from .utils import (
extract_facts,
calculate_recency_weight,
calculate_frequency_weight,
)
from .entity_resolver import EntityResolver, extract_entities
from .coref_resolver import resolve_sentences
def utcnow():
"""Get current UTC time with timezone info."""
return datetime.now(timezone.utc)
class TemporalSemanticMemory:
"""
Advanced memory system using temporal and semantic linking with PostgreSQL.
"""
def __init__(
self,
db_url: Optional[str] = None,
embedding_model: str = "BAAI/bge-small-en-v1.5",
):
"""
Initialize the temporal + semantic memory system.
Args:
db_url: PostgreSQL connection URL (postgresql://user:pass@host:port/dbname)
embedding_model: Name of the SentenceTransformer model to use
"""
load_dotenv()
# Initialize PostgreSQL connection
self.db_url = db_url or os.getenv("DATABASE_URL")
if not self.db_url:
raise ValueError(
"Database URL not found. "
"Set DATABASE_URL environment variable."
)
self.conn = psycopg2.connect(self.db_url)
register_vector(self.conn)
# Initialize entity resolver
self.entity_resolver = EntityResolver(self.conn)
# Initialize local embedding model (384 dimensions)
print(f"Loading embedding model: {embedding_model}...")
self.embedding_model = SentenceTransformer(embedding_model)
print(f"✓ Model loaded (embedding dim: {self.embedding_model.get_sentence_embedding_dimension()})")
def __del__(self):
"""Clean up database connection."""
if hasattr(self, 'conn') and self.conn:
self.conn.close()
def _generate_embedding(self, text: str) -> List[float]:
"""
Generate embedding for text using local SentenceTransformer model.
Args:
text: Text to embed
Returns:
384-dimensional embedding vector (bge-small-en-v1.5)
"""
try:
embedding = self.embedding_model.encode(text, convert_to_numpy=True, show_progress_bar=False)
return embedding.tolist()
except Exception as e:
raise Exception(f"Failed to generate embedding: {str(e)}")
async def _generate_embeddings_batch(self, texts: List[str]) -> List[List[float]]:
"""
Generate embeddings for multiple texts using local model (batch processing).
Local models are fast and process batches efficiently without needing
parallel API calls. We run this in asyncio to avoid blocking, but the
actual embedding generation is synchronous.
Args:
texts: List of texts to embed
Returns:
List of 384-dimensional embeddings in same order as input texts
"""
try:
# Run in thread pool to avoid blocking event loop
loop = asyncio.get_event_loop()
embeddings = await loop.run_in_executor(
None,
lambda: self.embedding_model.encode(texts, convert_to_numpy=True, show_progress_bar=False)
)
return [emb.tolist() for emb in embeddings]
except Exception as e:
raise Exception(f"Failed to generate batch embeddings: {str(e)}")
def _find_duplicate_facts_batch(
self,
cursor,
agent_id: str,
texts: List[str],
embeddings: List[List[float]],
event_date: datetime,
time_window_hours: int = 24,
similarity_threshold: float = 0.95
) -> List[bool]:
"""
Check which facts are duplicates using semantic similarity + temporal window.
For each new fact, checks if a semantically similar fact already exists
within the time window. Uses pgvector cosine similarity for efficiency.
Args:
cursor: Database cursor
agent_id: Agent identifier
texts: List of fact texts to check
embeddings: Corresponding embeddings
event_date: Event date for temporal filtering
time_window_hours: Hours before/after event_date to search (default: 24)
similarity_threshold: Minimum cosine similarity to consider duplicate (default: 0.95)
Returns:
List of booleans - True if fact is a duplicate (should skip), False if new
"""
is_duplicate = []
time_lower = event_date - timedelta(hours=time_window_hours)
time_upper = event_date + timedelta(hours=time_window_hours)
for text, embedding in zip(texts, embeddings):
# Query for similar facts within time window
cursor.execute(
"""
SELECT id, text, 1 - (embedding <=> %s::vector) AS similarity
FROM memory_units
WHERE agent_id = %s
AND event_date BETWEEN %s AND %s
AND 1 - (embedding <=> %s::vector) > %s
ORDER BY similarity DESC
LIMIT 1
""",
(embedding, agent_id, time_lower, time_upper, embedding, similarity_threshold)
)
result = cursor.fetchone()
if result:
is_duplicate.append(True)
else:
is_duplicate.append(False)
return is_duplicate
def put(
self,
agent_id: str,
content: str,
context: str = "",
event_date: Optional[datetime] = None,
) -> List[str]:
"""
Store content as memory units (synchronous wrapper).
This is a synchronous wrapper around put_async() for convenience.
For best performance, use put_async() directly.
Args:
agent_id: Unique identifier for the agent
content: Text content to store
context: Context about when/why this memory was formed
event_date: When the event occurred (defaults to now)
Returns:
List of created unit IDs
"""
# Run async version synchronously
return asyncio.run(self.put_async(agent_id, content, context, event_date))
async def put_async(
self,
agent_id: str,
content: str,
context: str = "",
event_date: Optional[datetime] = None,
) -> List[str]:
"""
Store content as memory units with temporal and semantic links (ASYNC version).
This async version generates ALL embeddings in parallel for maximum speed,
then uses batch inserts for database operations.
Steps:
1. Split content into sentence units
2. Resolve coreferences
3. **Generate ALL embeddings in parallel** (FAST!)
4. **Batch insert all units and links** (FAST!)
Args:
agent_id: Unique identifier for the agent
content: Text content to store
context: Context about when/why this memory was formed
event_date: When the event occurred (defaults to now)
Returns:
List of created unit IDs
"""
start_time = time.time()
print(f"\n{'='*60}")
print(f"PUT_ASYNC START: {agent_id}")
print(f"Content length: {len(content)} chars")
print(f"{'='*60}")
if event_date is None:
event_date = utcnow()
# Step 1: Extract semantic facts using LLM (async)
step_start = time.time()
try:
facts = await extract_facts(content)
print(f"[1] Extract facts: {len(facts)} facts in {time.time() - step_start:.3f}s")
except Exception as e:
print(f"\n{'='*60}")
print(f"PUT_ASYNC FAILED: Fact extraction error")
print(f"Error: {e}")
print(f"{'='*60}\n")
raise Exception(f"Failed to extract facts from content: {e}")
# Step 2: Resolve pronouns to make facts even more self-contained
step_start = time.time()
sentences = resolve_sentences(facts)
print(f"[2] Resolve coreferences: {time.time() - step_start:.3f}s")
# Step 3: Generate ALL embeddings in parallel
step_start = time.time()
embeddings = await self._generate_embeddings_batch(sentences)
print(f"[3] Generate embeddings (parallel): {len(embeddings)} embeddings in {time.time() - step_start:.3f}s")
# Step 4: Check for duplicates using similarity + temporal window
cursor = self.conn.cursor()
step_start = time.time()
duplicate_flags = self._find_duplicate_facts_batch(
cursor, agent_id, sentences, embeddings, event_date
)
num_duplicates = sum(duplicate_flags)
# Filter out duplicates
filtered_data = [
(sentence, embedding)
for sentence, embedding, is_dup in zip(sentences, embeddings, duplicate_flags)
if not is_dup
]
if filtered_data:
sentences, embeddings = zip(*filtered_data)
sentences = list(sentences)
embeddings = list(embeddings)
else:
sentences = []
embeddings = []
print(f"[4] Deduplication check: {num_duplicates} duplicates filtered, {len(sentences)} new facts in {time.time() - step_start:.3f}s")
# If all facts were duplicates, return empty list
if not sentences:
cursor.close()
print(f"\n{'='*60}")
print(f"PUT_ASYNC COMPLETE: All facts were duplicates, nothing stored")
print(f"{'='*60}\n")
return []
# Step 5: Batch insert everything
try:
# Batch INSERT all memory units
step_start = time.time()
from psycopg2.extras import execute_values
unit_data = [
(agent_id, sentence, embedding, context, event_date, 0)
for sentence, embedding in zip(sentences, embeddings)
]
unit_ids = execute_values(
cursor,
"""
INSERT INTO memory_units (agent_id, text, embedding, context, event_date, access_count)
VALUES %s
RETURNING id
""",
unit_data,
fetch=True
)
created_unit_ids = [str(row[0]) for row in unit_ids]
print(f"[5] Batch insert units: {time.time() - step_start:.3f}s")
# Process entities for all units
step_start = time.time()
all_entity_links = []
for unit_id, sentence in zip(created_unit_ids, sentences):
entity_links = self._extract_entities_for_batch(cursor, agent_id, unit_id, sentence, context, event_date, sentences)
all_entity_links.extend(entity_links)
print(f"[6] Extract entities: {time.time() - step_start:.3f}s")
# Create ALL temporal links in batch
step_start = time.time()
self._create_temporal_links_batch(cursor, agent_id, created_unit_ids, event_date)
print(f"[7] Batch create temporal links: {time.time() - step_start:.3f}s")
# Create ALL semantic links in batch
step_start = time.time()
self._create_semantic_links_batch(cursor, agent_id, created_unit_ids, embeddings)
print(f"[8] Batch create semantic links: {time.time() - step_start:.3f}s")
# Insert all entity links in batch
step_start = time.time()
if all_entity_links:
self._insert_entity_links_batch(cursor, all_entity_links)
print(f"[9] Batch insert entity links: {time.time() - step_start:.3f}s")
commit_start = time.time()
self.conn.commit()
print(f"[10] Commit: {time.time() - commit_start:.3f}s")
total_time = time.time() - start_time
print(f"\n{'='*60}")
print(f"PUT_ASYNC COMPLETE: {len(created_unit_ids)} units stored in {total_time:.3f}s")
print(f"{'='*60}\n")
return created_unit_ids
except Exception as e:
self.conn.rollback()
raise Exception(f"Failed to store memory: {str(e)}")
finally:
cursor.close()
def _create_temporal_links(
self,
cursor,
agent_id: str,
unit_id: str,
event_date: datetime,
time_window_hours: int = 24,
):
"""
Create temporal links to recent memories.
Links this unit to other units that occurred within a time window.
Args:
cursor: Database cursor
agent_id: Agent ID
unit_id: ID of the current unit
event_date: When this event occurred
time_window_hours: Size of the temporal window
"""
try:
# Get recent units within time window
cursor.execute(
"""
SELECT id, event_date
FROM memory_units
WHERE agent_id = %s
AND id != %s
AND event_date >= %s
ORDER BY event_date DESC
LIMIT 10
""",
(agent_id, unit_id, event_date - timedelta(hours=time_window_hours))
)
recent_units = cursor.fetchall()
# Create links to recent units
links = []
for recent_id, recent_event_date in recent_units:
# Calculate temporal proximity weight
time_diff_hours = abs((event_date - recent_event_date).total_seconds() / 3600)
weight = max(0.3, 1.0 - (time_diff_hours / time_window_hours))
links.append((unit_id, recent_id, 'temporal', weight, None))
if links:
execute_values(
cursor,
"""
INSERT INTO memory_links (from_unit_id, to_unit_id, link_type, weight, entity_id)
VALUES %s
ON CONFLICT (from_unit_id, to_unit_id, link_type, COALESCE(entity_id, '00000000-0000-0000-0000-000000000000'::uuid)) DO NOTHING
""",
links
)
except Exception as e:
print(f"Warning: Failed to create temporal links: {str(e)}")
def _create_semantic_links(
self,
cursor,
agent_id: str,
unit_id: str,
embedding: List[float],
top_k: int = 5,
threshold: float = 0.7,
):
"""
Create semantic links to similar memories.
Links this unit to other units with similar meaning.
Args:
cursor: Database cursor
agent_id: Agent ID
unit_id: ID of the current unit
embedding: Embedding of the current unit
top_k: Number of similar units to link to
threshold: Minimum similarity threshold
"""
try:
# Find similar units using vector similarity
cursor.execute(
"""
SELECT id, 1 - (embedding <=> %s::vector) AS similarity
FROM memory_units
WHERE agent_id = %s
AND id != %s
AND embedding IS NOT NULL
AND (1 - (embedding <=> %s::vector)) >= %s
ORDER BY embedding <=> %s::vector
LIMIT %s
""",
(embedding, agent_id, unit_id, embedding, threshold, embedding, top_k)
)
similar_units = cursor.fetchall()
# Create links to similar units
links = []
for similar_id, similarity in similar_units:
links.append((unit_id, similar_id, 'semantic', float(similarity), None))
if links:
execute_values(
cursor,
"""
INSERT INTO memory_links (from_unit_id, to_unit_id, link_type, weight, entity_id)
VALUES %s
ON CONFLICT (from_unit_id, to_unit_id, link_type, COALESCE(entity_id, '00000000-0000-0000-0000-000000000000'::uuid)) DO NOTHING
""",
links
)
except Exception as e:
print(f"Warning: Failed to create semantic links: {str(e)}")
def _extract_and_link_entities(
self,
cursor,
agent_id: str,
unit_id: str,
text: str,
context: str,
event_date,
all_sentences: List[str],
):
"""
Extract entities from text, resolve them, and create entity links.
Args:
cursor: Database cursor
agent_id: Agent ID
unit_id: Current unit ID
text: Unit text
context: Context
event_date: When created
all_sentences: All sentences from the same PUT (for context)
"""
try:
# Extract entities from this unit
entities = extract_entities(text)
if not entities:
return
# Resolve each entity and link
entity_ids = []
for entity in entities:
entity_id = self.entity_resolver.resolve_entity(
agent_id=agent_id,
entity_text=entity['text'],
entity_type=entity['type'],
context=context,
nearby_entities=entities,
unit_event_date=event_date
)
entity_ids.append(entity_id)
# Link unit to entity
self.entity_resolver.link_unit_to_entity(unit_id, entity_id)
# Create entity links to other units that mention the same entities
for entity_id in set(entity_ids):
# Get other units that mention this entity
related_units = self.entity_resolver.get_units_by_entity(entity_id, limit=50)
# Create entity links
links = []
for related_unit_id in related_units:
if str(related_unit_id) != str(unit_id):
links.append((unit_id, related_unit_id, 'entity', 1.0, entity_id))
if links:
execute_values(
cursor,
"""
INSERT INTO memory_links (from_unit_id, to_unit_id, link_type, weight, entity_id)
VALUES %s
ON CONFLICT DO NOTHING
""",
links
)
except Exception as e:
print(f"Warning: Failed to extract/link entities: {str(e)}")
def search(
self,
agent_id: str,
query: str,
thinking_budget: int = 50,
top_k: int = 10,
live_tracer=None,
) -> List[Dict[str, Any]]:
"""
Search memories using spreading activation.
This implements the core SEARCH operation:
1. Find entry points (most relevant units via vector search)
2. Spread activation through the graph
3. Weight results by activation + recency + frequency
4. Return top results
Args:
agent_id: Agent ID to search for
query: Search query
thinking_budget: How many units to explore (computational budget)
top_k: Number of results to return
live_tracer: Optional LiveSearchTracer for visualization
Returns:
List of memory units with their weights, sorted by relevance
"""
cursor = self.conn.cursor(cursor_factory=RealDictCursor)
try:
# Step 1: Generate query embedding
query_embedding = self._generate_embedding(query)
# Step 2: Find entry points
cursor.execute(
"""
SELECT id, text, context, event_date, access_count,
1 - (embedding <=> %s::vector) AS similarity
FROM memory_units
WHERE agent_id = %s
AND embedding IS NOT NULL
AND (1 - (embedding <=> %s::vector)) >= 0.5
ORDER BY embedding <=> %s::vector
LIMIT 3
""",
(query_embedding, agent_id, query_embedding, query_embedding)
)
entry_points = cursor.fetchall()
if not entry_points:
return []
# Step 3: Spreading activation with budget
visited = set()
results = []
budget_remaining = thinking_budget
queue = [(dict(unit), 1.0, True) for unit in entry_points] # (unit, activation, is_entry)
while queue and budget_remaining > 0:
current_unit, activation, is_entry_point = queue.pop(0)
unit_id = str(current_unit["id"])
if unit_id in visited:
continue
visited.add(unit_id)
budget_remaining -= 1
# Increment access count
cursor.execute(
"UPDATE memory_units SET access_count = access_count + 1 WHERE id = %s",
(unit_id,)
)
# Calculate combined weight
event_date = current_unit["event_date"]
days_since = (utcnow() - event_date).total_seconds() / 86400
recency_weight = calculate_recency_weight(days_since)
frequency_weight = calculate_frequency_weight(current_unit.get("access_count", 0))
# Combined weight: activation * recency * frequency
final_weight = activation * recency_weight * frequency_weight
# Notify tracer
if live_tracer:
live_tracer.visit_node(
node_id=unit_id,
text=current_unit["text"],
activation=activation,
recency=recency_weight,
frequency=frequency_weight,
weight=final_weight,
is_entry_point=is_entry_point,
)
import time
time.sleep(0.15) # Slow down for visualization
results.append({
"id": unit_id,
"text": current_unit["text"],
"context": current_unit.get("context", ""),
"event_date": event_date.isoformat(),
"weight": final_weight,
"activation": activation,
"recency": recency_weight,
"frequency": frequency_weight,
})
# Spread to neighbors
cursor.execute(
"""
SELECT ml.to_unit_id, ml.weight, mu.text, mu.context, mu.event_date, mu.access_count
FROM memory_links ml
JOIN memory_units mu ON ml.to_unit_id = mu.id
WHERE ml.from_unit_id = %s
AND ml.weight >= 0.1
ORDER BY ml.weight DESC
""",
(unit_id,)
)
neighbors = cursor.fetchall()
for neighbor in neighbors:
neighbor_id = str(neighbor["to_unit_id"])
if neighbor_id not in visited:
link_weight = neighbor["weight"]
new_activation = activation * link_weight * 0.8 # 0.8 = decay factor
if new_activation > 0.1:
queue.append(({
"id": neighbor["to_unit_id"],
"text": neighbor["text"],
"context": neighbor.get("context", ""),
"event_date": neighbor["event_date"],
"access_count": neighbor["access_count"],
}, new_activation, False)) # Not an entry point
self.conn.commit()
# Step 4: Sort by final weight and return top results
results.sort(key=lambda x: x["weight"], reverse=True)
return results[:top_k]
except Exception as e:
self.conn.rollback()
raise Exception(f"Failed to search memories: {str(e)}")
finally:
cursor.close()
def get_memory_graph_data(self, agent_id: str = None) -> Tuple[List[Dict], List[Dict]]:
"""
Get memory graph data for visualization.
Args:
agent_id: Optional agent ID (if None, returns all data)
Returns:
Tuple of (units, links) for visualization
"""
cursor = self.conn.cursor(cursor_factory=RealDictCursor)
try:
# Get all units (optionally filtered by agent)
if agent_id:
cursor.execute(
"SELECT id, text, context, event_date, access_count FROM memory_units WHERE agent_id = %s",
(agent_id,)
)
else:
cursor.execute(
"SELECT id, text, context, event_date, access_count FROM memory_units"
)
units = [dict(row) for row in cursor.fetchall()]
# Get all links (optionally filtered by agent)
if agent_id:
cursor.execute(
"""
SELECT ml.from_unit_id, ml.to_unit_id, ml.link_type, ml.weight
FROM memory_links ml
JOIN memory_units mu1 ON ml.from_unit_id = mu1.id
JOIN memory_units mu2 ON ml.to_unit_id = mu2.id
WHERE mu1.agent_id = %s
""",
(agent_id,)
)
else:
cursor.execute(
"SELECT from_unit_id, to_unit_id, link_type, weight FROM memory_links"
)
links = [dict(row) for row in cursor.fetchall()]
return units, links
except Exception as e:
raise Exception(f"Failed to get memory graph data: {str(e)}")
finally:
cursor.close()
def _extract_entities_for_batch(
self,
cursor,
agent_id: str,
unit_id: str,
text: str,
context: str,
event_date,
all_sentences: List[str],
) -> List[tuple]:
"""
Extract entities and return entity links (doesn't insert yet).
Returns list of tuples for batch insertion: (from_unit_id, to_unit_id, link_type, weight, entity_id)
"""
from .entity_resolver import extract_entities
try:
# Extract entities from this unit
entities = extract_entities(text)
if not entities:
return []
# Resolve each entity
entity_ids = []
for entity in entities:
entity_id = self.entity_resolver.resolve_entity(
agent_id=agent_id,
entity_text=entity['text'],
entity_type=entity['type'],
context=context,
nearby_entities=entities,
unit_event_date=event_date
)
entity_ids.append(entity_id)
# Link unit to entity (this inserts into entity_units)
self.entity_resolver.link_unit_to_entity(unit_id, entity_id)
# Now collect entity links for batch insertion
# After link_unit_to_entity has been called, entity_units should exist
links = []
for entity_id in set(entity_ids):
# Find all other units with this entity (cursor must be fresh)
try:
cursor.execute(
"""
SELECT unit_id
FROM unit_entities
WHERE entity_id = %s AND unit_id != %s
""",
(entity_id, unit_id)
)
related_units = cursor.fetchall()
for (related_unit_id,) in related_units:
# Bidirectional links
links.append((unit_id, related_unit_id, 'entity', 1.0, entity_id))
links.append((related_unit_id, unit_id, 'entity', 1.0, entity_id))
except Exception as query_error:
# If there's an error querying, just skip this entity
print(f"Warning: Failed to query entity_units for {entity_id}: {str(query_error)}")
continue
return links
except Exception as e:
print(f"Warning: Failed to extract entities: {str(e)}")
return []
def _create_temporal_links_batch(
self,
cursor,
agent_id: str,
unit_ids: List[str],
event_date: datetime,
time_window_hours: int = 24,
):
"""
Create temporal links for multiple units in one batch query.
Uses a single query to find all relevant temporal connections.
"""
if not unit_ids:
return
try:
from psycopg2.extras import execute_values
# Get ALL recent units within time window (single query)
# Cast string IDs to UUIDs for comparison
cursor.execute(
"""
SELECT id, event_date
FROM memory_units
WHERE agent_id = %s
AND id::text != ALL(%s)
AND event_date >= %s
ORDER BY event_date DESC
""",
(agent_id, unit_ids, event_date - timedelta(hours=time_window_hours))
)
recent_units = cursor.fetchall()
# Create links from each new unit to all recent units
links = []
for unit_id in unit_ids:
for recent_id, recent_event_date in recent_units:
# Calculate temporal proximity weight
time_diff_hours = abs((event_date - recent_event_date).total_seconds() / 3600)
weight = max(0.3, 1.0 - (time_diff_hours / time_window_hours))
links.append((unit_id, recent_id, 'temporal', weight, None))
if links:
execute_values(
cursor,
"""
INSERT INTO memory_links (from_unit_id, to_unit_id, link_type, weight, entity_id)
VALUES %s
ON CONFLICT (from_unit_id, to_unit_id, link_type, COALESCE(entity_id, '00000000-0000-0000-0000-000000000000'::uuid)) DO NOTHING
""",
links
)
except Exception as e:
print(f"Warning: Failed to create temporal links: {str(e)}")
def _create_semantic_links_batch(
self,
cursor,
agent_id: str,
unit_ids: List[str],
embeddings: List[List[float]],
top_k: int = 5,
threshold: float = 0.7,
):
"""
Create semantic links for multiple units efficiently.
For each unit, finds similar units and creates links.
"""
if not unit_ids or not embeddings:
return
try:
from psycopg2.extras import execute_values
all_links = []
for unit_id, embedding in zip(unit_ids, embeddings):
# Find similar units using vector similarity
cursor.execute(
"""
SELECT id, 1 - (embedding <=> %s::vector) AS similarity
FROM memory_units
WHERE agent_id = %s
AND id != %s
AND embedding IS NOT NULL
AND (1 - (embedding <=> %s::vector)) >= %s
ORDER BY embedding <=> %s::vector
LIMIT %s
""",
(embedding, agent_id, unit_id, embedding, threshold, embedding, top_k)
)
similar_units = cursor.fetchall()
for similar_id, similarity in similar_units:
all_links.append((unit_id, similar_id, 'semantic', float(similarity), None))
if all_links:
execute_values(
cursor,
"""
INSERT INTO memory_links (from_unit_id, to_unit_id, link_type, weight, entity_id)
VALUES %s
ON CONFLICT (from_unit_id, to_unit_id, link_type, COALESCE(entity_id, '00000000-0000-0000-0000-000000000000'::uuid)) DO NOTHING
""",
all_links
)
except Exception as e:
print(f"Warning: Failed to create semantic links: {str(e)}")
def _insert_entity_links_batch(self, cursor, links: List[tuple]):
"""Insert all entity links in a single batch."""
if not links:
return
try:
from psycopg2.extras import execute_values
execute_values(
cursor,
"""
INSERT INTO memory_links (from_unit_id, to_unit_id, link_type, weight, entity_id)
VALUES %s
ON CONFLICT (from_unit_id, to_unit_id, link_type, COALESCE(entity_id, '00000000-0000-0000-0000-000000000000'::uuid)) DO NOTHING
""",
links
)
except Exception as e:
print(f"Warning: Failed to insert entity links: {str(e)}")
+102
View File
@@ -0,0 +1,102 @@
"""
Utility functions for memory system.
"""
from typing import List
from .llm_client import extract_facts_from_text
async def extract_facts(text: str) -> List[str]:
"""
Extract semantic facts from text using LLM.
Uses LLM for intelligent fact extraction that:
- Filters out social pleasantries and filler words
- Creates self-contained statements
- Handles conversational text well
Args:
text: Input text (conversation, article, etc.)
Returns:
List of factual statements
Raises:
Exception: If LLM fact extraction fails
"""
if not text or not text.strip():
return []
fact_dicts = await extract_facts_from_text(text)
# Extract just the fact text
facts = [f['fact'] for f in fact_dicts if f.get('fact')]
if not facts:
raise Exception(f"LLM extracted 0 facts from text of length {len(text)}. This may indicate the text contains no meaningful information, or the LLM failed to extract facts.")
return facts
def cosine_similarity(vec1: List[float], vec2: List[float]) -> float:
"""
Calculate cosine similarity between two vectors.
Args:
vec1: First vector
vec2: Second vector
Returns:
Similarity score between 0 and 1
"""
if len(vec1) != len(vec2):
raise ValueError("Vectors must have same dimension")
dot_product = sum(a * b for a, b in zip(vec1, vec2))
magnitude1 = sum(a * a for a in vec1) ** 0.5
magnitude2 = sum(b * b for b in vec2) ** 0.5
if magnitude1 == 0 or magnitude2 == 0:
return 0.0
return dot_product / (magnitude1 * magnitude2)
def calculate_recency_weight(days_since: float, decay_rate: float = 0.1) -> float:
"""
Calculate recency weight with exponential decay.
Recent memories are weighted higher. The decay rate controls
how quickly old memories fade.
Args:
days_since: Number of days since the memory was created
decay_rate: How quickly memories fade (higher = faster decay)
Returns:
Weight between 0 and 1
"""
import math
return math.exp(-decay_rate * days_since)
def calculate_frequency_weight(access_count: int, max_boost: float = 2.0) -> float:
"""
Calculate frequency weight based on access count.
Frequently accessed memories are weighted higher.
Uses logarithmic scaling to avoid over-weighting.
Args:
access_count: Number of times the memory was accessed
max_boost: Maximum multiplier for frequently accessed memories
Returns:
Weight between 1.0 and max_boost
"""
import math
if access_count <= 0:
return 1.0
# Logarithmic scaling: log(access_count + 1) / log(10)
# This gives: 0 accesses = 1.0, 9 accesses ~= 1.5, 99 accesses ~= 2.0
normalized = math.log(access_count + 1) / math.log(10)
return 1.0 + min(normalized, max_boost - 1.0)
+394
View File
@@ -0,0 +1,394 @@
"""
Memory visualization module.
Provides visual representations of memory networks and search paths.
"""
import time
from typing import List, Dict, Any, Optional, Tuple
import networkx as nx
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch
from rich.console import Console
from rich.table import Table
from rich.panel import Panel
from rich.layout import Layout
from rich.live import Live
from rich.text import Text
from rich import box
class MemoryVisualizer:
"""
Visualizes memory networks and search paths.
"""
def __init__(self):
"""Initialize the visualizer."""
self.console = Console()
def visualize_memory_graph(
self,
units: List[Dict[str, Any]],
links: List[Dict[str, Any]],
output_file: str = "memory_graph.png",
highlight_nodes: Optional[List[str]] = None,
):
"""
Create a visual representation of the memory graph.
Args:
units: List of memory units (id, text, context, etc.)
links: List of links (from_unit_id, to_unit_id, link_type, weight)
output_file: Output file path for the visualization
highlight_nodes: Optional list of node IDs to highlight
"""
# Create directed graph
G = nx.DiGraph()
# Add nodes
node_labels = {}
for unit in units:
unit_id = str(unit['id'])
# Truncate text for display
label = unit['text'][:40] + "..." if len(unit['text']) > 40 else unit['text']
G.add_node(unit_id)
node_labels[unit_id] = label
# Add edges
temporal_edges = []
semantic_edges = []
for link in links:
from_id = str(link['from_unit_id'])
to_id = str(link['to_unit_id'])
weight = link['weight']
link_type = link['link_type']
if link_type == 'temporal':
temporal_edges.append((from_id, to_id, weight))
else: # semantic
semantic_edges.append((from_id, to_id, weight))
G.add_edge(from_id, to_id, weight=weight, type=link_type)
# Create figure
fig, ax = plt.subplots(figsize=(20, 14))
ax.set_facecolor('#1a1a2e')
fig.patch.set_facecolor('#0f0f1e')
# Use spring layout for better visualization
pos = nx.spring_layout(G, k=2, iterations=50, seed=42)
# Draw temporal edges (blue)
if temporal_edges:
nx.draw_networkx_edges(
G, pos,
edgelist=[(e[0], e[1]) for e in temporal_edges],
edge_color='#4ecdc4',
alpha=0.6,
width=2,
arrows=True,
arrowsize=15,
arrowstyle='->',
connectionstyle='arc3,rad=0.1',
ax=ax
)
# Draw semantic edges (purple)
if semantic_edges:
nx.draw_networkx_edges(
G, pos,
edgelist=[(e[0], e[1]) for e in semantic_edges],
edge_color='#ff6b9d',
alpha=0.6,
width=2,
arrows=True,
arrowsize=15,
arrowstyle='->',
connectionstyle='arc3,rad=0.1',
ax=ax
)
# Determine node colors
node_colors = []
for node in G.nodes():
if highlight_nodes and node in highlight_nodes:
node_colors.append('#ffd93d') # Yellow for highlighted
else:
node_colors.append('#6c63ff') # Purple for normal
# Draw nodes
nx.draw_networkx_nodes(
G, pos,
node_color=node_colors,
node_size=3000,
alpha=0.9,
ax=ax
)
# Draw labels
nx.draw_networkx_labels(
G, pos,
node_labels,
font_size=8,
font_color='white',
font_weight='bold',
ax=ax
)
# Add legend
legend_elements = [
plt.Line2D([0], [0], color='#4ecdc4', lw=2, label='Temporal Links'),
plt.Line2D([0], [0], color='#ff6b9d', lw=2, label='Semantic Links'),
plt.Line2D([0], [0], marker='o', color='w', markerfacecolor='#6c63ff',
markersize=10, label='Memory Unit', linestyle=''),
]
if highlight_nodes:
legend_elements.append(
plt.Line2D([0], [0], marker='o', color='w', markerfacecolor='#ffd93d',
markersize=10, label='Highlighted', linestyle='')
)
ax.legend(handles=legend_elements, loc='upper left', facecolor='#2d2d44',
edgecolor='white', fontsize=10, labelcolor='white')
# Title
ax.set_title('Memory Network Graph\nTemporal + Semantic Architecture',
color='white', fontsize=16, fontweight='bold', pad=20)
ax.axis('off')
plt.tight_layout()
plt.savefig(output_file, dpi=150, facecolor='#0f0f1e')
plt.close()
self.console.print(f"[green]✓[/green] Memory graph saved to [cyan]{output_file}[/cyan]")
class LiveSearchTracer:
"""
Live tracer for search operations showing spreading activation in real-time.
"""
def __init__(self):
"""Initialize the live tracer."""
self.console = Console()
self.visited_nodes = []
self.current_node = None
self.search_results = []
self.query = ""
self.budget_used = 0
self.budget_total = 0
def start_search(self, query: str, budget: int):
"""
Start a new search trace.
Args:
query: Search query
budget: Thinking budget
"""
self.query = query
self.budget_total = budget
self.budget_used = 0
self.visited_nodes = []
self.current_node = None
self.search_results = []
def visit_node(
self,
node_id: str,
text: str,
activation: float,
recency: float,
frequency: float,
weight: float,
is_entry_point: bool = False,
):
"""
Record a node visit.
Args:
node_id: Node ID
text: Node text
activation: Activation strength
recency: Recency weight
frequency: Frequency weight
weight: Combined weight
is_entry_point: Whether this is an entry point
"""
self.current_node = {
'id': node_id,
'text': text,
'activation': activation,
'recency': recency,
'frequency': frequency,
'weight': weight,
'is_entry_point': is_entry_point,
}
self.visited_nodes.append(self.current_node)
self.budget_used += 1
def add_result(
self,
text: str,
weight: float,
activation: float,
recency: float,
frequency: float,
):
"""
Add a search result.
Args:
text: Result text
weight: Combined weight
activation: Activation strength
recency: Recency weight
frequency: Frequency weight
"""
self.search_results.append({
'text': text,
'weight': weight,
'activation': activation,
'recency': recency,
'frequency': frequency,
})
def render_live(self) -> Layout:
"""
Render the current state.
Returns:
Rich Layout with current state
"""
layout = Layout()
layout.split_column(
Layout(name="header", size=3),
Layout(name="body"),
Layout(name="footer", size=5)
)
# Header
header_text = Text()
header_text.append("🔍 ", style="bold cyan")
header_text.append(f"Query: ", style="bold white")
header_text.append(f"{self.query}", style="bold yellow")
layout["header"].update(Panel(header_text, style="cyan"))
# Body - split into current node and visited
layout["body"].split_row(
Layout(name="current", ratio=1),
Layout(name="path", ratio=1),
)
# Current node
if self.current_node:
current_table = Table(
title="Current Node",
show_header=False,
box=box.ROUNDED,
style="green"
)
current_table.add_column("Key", style="cyan")
current_table.add_column("Value", style="white")
status = "🎯 ENTRY POINT" if self.current_node['is_entry_point'] else "🔄 EXPLORING"
current_table.add_row("Status", status)
current_table.add_row("Text", self.current_node['text'][:50] + "...")
current_table.add_row(
"Weights",
f"A:{self.current_node['activation']:.2f} "
f"R:{self.current_node['recency']:.2f} "
f"F:{self.current_node['frequency']:.2f}"
)
current_table.add_row(
"Combined",
f"[bold yellow]{self.current_node['weight']:.3f}[/bold yellow]"
)
layout["current"].update(Panel(current_table, border_style="green"))
else:
layout["current"].update(Panel("Initializing...", border_style="dim"))
# Visited path
path_table = Table(
title=f"Visited Nodes ({len(self.visited_nodes)})",
box=box.SIMPLE,
show_header=True,
style="blue"
)
path_table.add_column("#", style="dim", width=4)
path_table.add_column("Text", style="white", width=35)
path_table.add_column("Weight", justify="right", style="yellow", width=8)
path_table.add_column("Type", style="cyan", width=8)
for i, node in enumerate(reversed(self.visited_nodes[-10:])): # Last 10
node_type = "ENTRY" if node['is_entry_point'] else "SPREAD"
path_table.add_row(
str(len(self.visited_nodes) - i),
node['text'][:32] + "...",
f"{node['weight']:.3f}",
node_type
)
layout["path"].update(Panel(path_table, border_style="blue"))
# Footer - progress bar
progress = self.budget_used / self.budget_total if self.budget_total > 0 else 0
bar_width = 50
filled = int(bar_width * progress)
bar = "" * filled + "" * (bar_width - filled)
footer_text = Text()
footer_text.append(f"Progress: ", style="bold white")
footer_text.append(bar, style="yellow")
footer_text.append(f" {self.budget_used}/{self.budget_total}", style="bold cyan")
footer_text.append(f" ({progress*100:.1f}%)", style="dim")
layout["footer"].update(Panel(footer_text, style="yellow"))
return layout
def show_final_results(self):
"""
Show final search results in a nice table.
"""
self.console.print("\n")
results_table = Table(
title="🎯 Search Results",
show_header=True,
header_style="bold magenta",
box=box.DOUBLE_EDGE,
title_style="bold white"
)
results_table.add_column("Rank", style="cyan", justify="center", width=6)
results_table.add_column("Text", style="white", width=50)
results_table.add_column("Weight", justify="right", style="yellow", width=8)
results_table.add_column("A", justify="right", style="green", width=6)
results_table.add_column("R", justify="right", style="blue", width=6)
results_table.add_column("F", justify="right", style="magenta", width=6)
for i, result in enumerate(self.search_results, 1):
rank_style = "bold yellow" if i <= 3 else "cyan"
results_table.add_row(
f"#{i}",
result['text'][:47] + "...",
f"{result['weight']:.3f}",
f"{result['activation']:.2f}",
f"{result['recency']:.2f}",
f"{result['frequency']:.2f}",
style=rank_style if i <= 3 else None
)
self.console.print(results_table)
# Summary stats
summary = Table.grid(padding=(0, 2))
summary.add_column(style="bold cyan")
summary.add_column(style="white")
summary.add_row("Total nodes visited:", f"{len(self.visited_nodes)}")
summary.add_row("Budget used:", f"{self.budget_used}/{self.budget_total}")
summary.add_row("Results found:", f"{len(self.search_results)}")
self.console.print(Panel(summary, title="Summary", border_style="green", padding=(1, 2)))
File diff suppressed because one or more lines are too long
+24
View File
@@ -0,0 +1,24 @@
[project]
name = "memory-poc"
version = "0.1.0"
description = "Temporal + Semantic + Entity Memory System for AI agents using PostgreSQL"
readme = "README.md"
requires-python = ">=3.11"
dependencies = [
"psycopg2-binary>=2.9.0",
"pgvector>=0.2.0",
"python-dotenv>=1.0.0",
"openai>=1.0.0",
"pydantic>=2.0.0",
"nltk>=3.8.0",
"networkx>=3.0",
"matplotlib>=3.7.0",
"rich>=13.0.0",
"spacy>=3.7.0",
"pyvis>=0.3.0",
"sentence-transformers>=2.2.0",
"torch>=2.0.0",
"pytest>=7.0.0",
"pytest-asyncio>=0.21.0",
"fastcoref>=2.1.0",
]
+99
View File
@@ -0,0 +1,99 @@
-- Enable the pgvector extension and uuid extension
CREATE EXTENSION IF NOT EXISTS vector;
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
-- ============================================================================
-- TEMPORAL + SEMANTIC + ENTITY MEMORY ARCHITECTURE
-- ============================================================================
-- Memory Units: Individual sentence-level memories
CREATE TABLE IF NOT EXISTS memory_units (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
agent_id TEXT NOT NULL,
text TEXT NOT NULL,
embedding vector(384), -- bge-small-en-v1.5 dimension
context TEXT, -- What was happening when this memory was formed
event_date TIMESTAMPTZ NOT NULL, -- When the event occurred
access_count INTEGER DEFAULT 0, -- For recency/frequency weighting
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
-- Entities: Resolved entities (people, organizations, locations, etc.)
CREATE TABLE IF NOT EXISTS entities (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
canonical_name TEXT NOT NULL, -- "Alice", "Google", "San Francisco"
entity_type TEXT NOT NULL, -- PERSON, ORG, GPE, etc.
agent_id TEXT NOT NULL, -- Entities are scoped to agents
metadata JSONB DEFAULT '{}'::jsonb, -- Additional entity info
first_seen TIMESTAMPTZ DEFAULT NOW(),
last_seen TIMESTAMPTZ DEFAULT NOW(),
mention_count INTEGER DEFAULT 1
);
-- Unit-Entity associations: Which entities appear in which units
CREATE TABLE IF NOT EXISTS unit_entities (
unit_id UUID REFERENCES memory_units(id) ON DELETE CASCADE,
entity_id UUID REFERENCES entities(id) ON DELETE CASCADE,
PRIMARY KEY (unit_id, entity_id)
);
-- Entity Co-occurrences: Materialized cache of which entities appear together
-- This dramatically speeds up entity resolution by avoiding expensive joins
CREATE TABLE IF NOT EXISTS entity_cooccurrences (
entity_id_1 UUID REFERENCES entities(id) ON DELETE CASCADE,
entity_id_2 UUID REFERENCES entities(id) ON DELETE CASCADE,
cooccurrence_count INTEGER DEFAULT 1,
last_cooccurred TIMESTAMPTZ DEFAULT NOW(),
PRIMARY KEY (entity_id_1, entity_id_2),
CHECK (entity_id_1 < entity_id_2) -- Enforce ordering to avoid duplicates
);
-- Memory Links: Temporal, semantic, AND entity connections
CREATE TABLE IF NOT EXISTS memory_links (
from_unit_id UUID REFERENCES memory_units(id) ON DELETE CASCADE,
to_unit_id UUID REFERENCES memory_units(id) ON DELETE CASCADE,
link_type TEXT NOT NULL, -- 'temporal', 'semantic', or 'entity'
weight FLOAT NOT NULL DEFAULT 1.0, -- Link strength
entity_id UUID REFERENCES entities(id) ON DELETE CASCADE, -- Set for entity links
created_at TIMESTAMPTZ DEFAULT NOW()
);
-- Unique constraint to prevent duplicate links
CREATE UNIQUE INDEX IF NOT EXISTS idx_memory_links_unique
ON memory_links (from_unit_id, to_unit_id, link_type, COALESCE(entity_id, '00000000-0000-0000-0000-000000000000'::uuid));
-- ============================================================================
-- INDEXES
-- ============================================================================
-- Memory unit indexes
CREATE INDEX IF NOT EXISTS idx_memory_units_agent_id ON memory_units(agent_id);
CREATE INDEX IF NOT EXISTS idx_memory_units_event_date ON memory_units(event_date DESC);
CREATE INDEX IF NOT EXISTS idx_memory_units_agent_date ON memory_units(agent_id, event_date DESC);
CREATE INDEX IF NOT EXISTS idx_memory_units_access_count ON memory_units(access_count DESC);
-- Vector similarity index (HNSW for fast approximate nearest neighbor)
CREATE INDEX IF NOT EXISTS idx_memory_units_embedding ON memory_units
USING hnsw (embedding vector_cosine_ops);
-- Entity indexes
CREATE INDEX IF NOT EXISTS idx_entities_agent_id ON entities(agent_id);
CREATE INDEX IF NOT EXISTS idx_entities_canonical_name ON entities(canonical_name);
CREATE INDEX IF NOT EXISTS idx_entities_type ON entities(entity_type);
CREATE INDEX IF NOT EXISTS idx_entities_agent_name_type ON entities(agent_id, canonical_name, entity_type);
-- Unit-entity indexes
CREATE INDEX IF NOT EXISTS idx_unit_entities_unit ON unit_entities(unit_id);
CREATE INDEX IF NOT EXISTS idx_unit_entities_entity ON unit_entities(entity_id);
-- Entity co-occurrence indexes for fast lookups
CREATE INDEX IF NOT EXISTS idx_entity_cooccurrences_entity1 ON entity_cooccurrences(entity_id_1);
CREATE INDEX IF NOT EXISTS idx_entity_cooccurrences_entity2 ON entity_cooccurrences(entity_id_2);
CREATE INDEX IF NOT EXISTS idx_entity_cooccurrences_count ON entity_cooccurrences(cooccurrence_count DESC);
-- Link indexes for graph traversal
CREATE INDEX IF NOT EXISTS idx_memory_links_from ON memory_links(from_unit_id);
CREATE INDEX IF NOT EXISTS idx_memory_links_to ON memory_links(to_unit_id);
CREATE INDEX IF NOT EXISTS idx_memory_links_type ON memory_links(link_type);
CREATE INDEX IF NOT EXISTS idx_memory_links_entity ON memory_links(entity_id) WHERE entity_id IS NOT NULL;
+1
View File
@@ -0,0 +1 @@
"""Tests for the memory system."""
+58
View File
@@ -0,0 +1,58 @@
"""
Pytest configuration and shared fixtures.
"""
import pytest
import psycopg2
import os
from dotenv import load_dotenv
from memory import TemporalSemanticMemory
load_dotenv()
@pytest.fixture(scope="function")
def memory():
"""
Provide a clean memory system instance for each test.
"""
mem = TemporalSemanticMemory()
yield mem
# Cleanup is handled by individual tests
@pytest.fixture(scope="function")
def clean_agent(memory):
"""
Provide a clean agent ID and clean up data after test.
"""
agent_id = "test_agent"
# Clean up before test
conn = psycopg2.connect(os.getenv('DATABASE_URL'))
cursor = conn.cursor()
cursor.execute("DELETE FROM memory_units WHERE agent_id = %s", (agent_id,))
cursor.execute("DELETE FROM entities WHERE agent_id = %s", (agent_id,))
conn.commit()
cursor.close()
conn.close()
yield agent_id
# Clean up after test
conn = psycopg2.connect(os.getenv('DATABASE_URL'))
cursor = conn.cursor()
cursor.execute("DELETE FROM memory_units WHERE agent_id = %s", (agent_id,))
cursor.execute("DELETE FROM entities WHERE agent_id = %s", (agent_id,))
conn.commit()
cursor.close()
conn.close()
@pytest.fixture
def db_connection():
"""
Provide a database connection for direct DB queries in tests.
"""
conn = psycopg2.connect(os.getenv('DATABASE_URL'))
yield conn
conn.close()
+75
View File
@@ -0,0 +1,75 @@
"""
Test chunking functionality for large documents.
"""
import pytest
from memory.llm_client import chunk_text, split_into_sentences
def test_split_into_sentences():
"""Test sentence splitting."""
text = "This is sentence one. This is sentence two! Is this sentence three? Yes it is."
sentences = split_into_sentences(text)
assert len(sentences) == 4, f"Expected 4 sentences, got {len(sentences)}"
assert "This is sentence one" in sentences[0]
assert "This is sentence two" in sentences[1]
assert "Is this sentence three" in sentences[2]
assert "Yes it is" in sentences[3]
def test_chunk_text_small():
"""Test that small text is not chunked."""
text = "This is a short text. It should not be chunked."
chunks = chunk_text(text, max_chars=1000)
assert len(chunks) == 1, "Small text should not be chunked"
assert chunks[0] == text
def test_chunk_text_large():
"""Test that large text is chunked at sentence boundaries."""
# Create a text with 10 sentences of ~100 chars each
sentences = [f"This is sentence number {i}. " + "x" * 80 for i in range(10)]
text = " ".join(sentences)
# Chunk with max 300 chars - should create multiple chunks
chunks = chunk_text(text, max_chars=300)
assert len(chunks) > 1, "Large text should be chunked"
# Verify all chunks are under the limit
for chunk in chunks:
assert len(chunk) <= 300, f"Chunk exceeds max_chars: {len(chunk)}"
# Verify we didn't lose any content
combined = " ".join(chunks)
# Account for possible whitespace differences
assert len(combined.replace(" ", "")) >= len(text.replace(" ", "")) * 0.95
def test_chunk_text_64k():
"""Test chunking a 64k character text (like a podcast transcript)."""
# Create a 64k character text
sentence = "This is a typical podcast conversation sentence. "
text = sentence * (64000 // len(sentence))
chunks = chunk_text(text, max_chars=120000)
print(f"\n64k text chunked into {len(chunks)} chunks")
for i, chunk in enumerate(chunks):
print(f" Chunk {i + 1}: {len(chunk)} characters")
# Should create at least 1 chunk (if text fits) or more
assert len(chunks) >= 1
# All chunks should be under the limit
for chunk in chunks:
assert len(chunk) <= 120000, f"Chunk exceeds max_chars: {len(chunk)}"
# Verify we didn't lose content
combined_length = sum(len(chunk) for chunk in chunks)
assert combined_length >= len(text) * 0.95, "Lost too much content during chunking"
if __name__ == "__main__":
pytest.main([__file__, "-v"])
+57
View File
@@ -0,0 +1,57 @@
"""
Performance test for coreference resolution.
"""
import time
from memory.coref_resolver import resolve_sentences, resolve_sentences_fast, resolve_sentences_legacy
def test_coref_performance():
"""Compare performance of fast vs legacy coreference resolution."""
# Sample sentences with coreferences
test_sentences = [
"John is a software engineer.",
"He works at a tech company.",
"The company is based in San Francisco.",
"He enjoys working on AI projects.",
"The projects involve machine learning.",
"John believes AI will transform the industry.",
"He has been working on this for 5 years.",
"The experience has been valuable.",
"John plans to continue his research.",
"He is passionate about the field.",
] * 10 # Repeat 10 times to make it 100 sentences
print(f"\nTesting with {len(test_sentences)} sentences...")
# Test fast method
start = time.time()
resolved_fast = resolve_sentences_fast(test_sentences)
fast_time = time.time() - start
print(f"FastCoref: {fast_time:.3f} seconds")
# Test legacy method (with smaller dataset to avoid timeout)
small_test = test_sentences[:20]
start = time.time()
resolved_legacy = resolve_sentences_legacy(small_test)
legacy_time = time.time() - start
print(f"Legacy (20 sentences): {legacy_time:.3f} seconds")
# Extrapolate legacy time
extrapolated_legacy = legacy_time * (len(test_sentences) / len(small_test)) ** 2
print(f"Legacy (extrapolated for {len(test_sentences)}): {extrapolated_legacy:.3f} seconds")
speedup = extrapolated_legacy / fast_time if fast_time > 0 else float('inf')
print(f"Speedup: {speedup:.1f}x faster")
# Verify resolution worked
print("\nSample resolved sentences (FastCoref):")
for i, sent in enumerate(resolved_fast[:3]):
print(f" {i+1}. {sent}")
assert len(resolved_fast) == len(test_sentences)
assert fast_time < extrapolated_legacy
if __name__ == "__main__":
test_coref_performance()
+111
View File
@@ -0,0 +1,111 @@
"""
Test deduplication of identical puts.
"""
import pytest
from datetime import datetime, timezone
from memory.temporal_semantic_memory import TemporalSemanticMemory
@pytest.fixture
def memory():
"""Create a memory instance for testing."""
mem = TemporalSemanticMemory()
yield mem
# Cleanup after test
cursor = mem.conn.cursor()
cursor.execute("DELETE FROM memory_units WHERE agent_id LIKE 'test_%'")
mem.conn.commit()
cursor.close()
@pytest.mark.asyncio
async def test_duplicate_put_filters_identical_content(memory):
"""Test that putting the same content twice doesn't create duplicates."""
agent_id = "test_dedup_agent"
content = "Alice works at Google as a software engineer. She joined last year and loves Python."
event_date = datetime(2024, 1, 15, 10, 0, 0, tzinfo=timezone.utc)
# First put - should create units
print("\n--- FIRST PUT ---")
units_1 = await memory.put_async(agent_id, content, "Test context", event_date)
assert len(units_1) > 0, "First put should create units"
print(f"First put created {len(units_1)} units")
# Second put with identical content and same date - should be filtered as duplicates
print("\n--- SECOND PUT (identical) ---")
units_2 = await memory.put_async(agent_id, content, "Test context", event_date)
assert len(units_2) == 0, "Second identical put should create no new units (all duplicates)"
print(f"Second put created {len(units_2)} units (expected 0)")
# Verify database has only the first set of units
cursor = memory.conn.cursor()
cursor.execute(
"SELECT COUNT(*) FROM memory_units WHERE agent_id = %s",
(agent_id,)
)
total_units = cursor.fetchone()[0]
cursor.close()
assert total_units == len(units_1), f"Database should have {len(units_1)} units, found {total_units}"
print(f"✅ Deduplication working: {total_units} total units in database")
@pytest.mark.asyncio
async def test_duplicate_put_with_paraphrased_content(memory):
"""Test that similar but paraphrased content is also deduplicated."""
agent_id = "test_paraphrase_agent"
event_date = datetime(2024, 1, 15, 10, 0, 0, tzinfo=timezone.utc)
# First put
content_1 = "Bob is a chef in New York. He owns a restaurant."
print("\n--- FIRST PUT ---")
units_1 = await memory.put_async(agent_id, content_1, "Test", event_date)
assert len(units_1) > 0, "First put should create units"
print(f"First put created {len(units_1)} units")
# Second put with paraphrased content - should be mostly deduplicated
# The LLM will extract similar facts that should match via embeddings
content_2 = "Bob works as a chef in New York City. He is the owner of a restaurant."
print("\n--- SECOND PUT (paraphrased) ---")
units_2 = await memory.put_async(agent_id, content_2, "Test", event_date)
# May create 0 or very few new units (depending on how LLM extracts facts)
print(f"Second put created {len(units_2)} units")
print(f"Deduplication ratio: {len(units_2)}/{len(units_1)} new units from paraphrase")
# Just verify it doesn't create the same number of units (some deduplication should happen)
assert len(units_2) < len(units_1), "Paraphrased content should have fewer new units due to deduplication"
@pytest.mark.asyncio
async def test_different_dates_not_deduplicated(memory):
"""Test that same content with different dates is NOT deduplicated."""
agent_id = "test_dates_agent"
content = "Charlie went hiking in Yosemite."
# First put at date 1
date_1 = datetime(2024, 1, 1, 10, 0, 0, tzinfo=timezone.utc)
print("\n--- FIRST PUT (Jan 1) ---")
units_1 = await memory.put_async(agent_id, content, "Test", date_1)
assert len(units_1) > 0
print(f"First put created {len(units_1)} units")
# Second put at date 2 (outside 24-hour window)
date_2 = datetime(2024, 2, 1, 10, 0, 0, tzinfo=timezone.utc)
print("\n--- SECOND PUT (Feb 1, outside time window) ---")
units_2 = await memory.put_async(agent_id, content, "Test", date_2)
# Should create new units because dates are far apart
assert len(units_2) > 0, "Same content with different dates (outside window) should create new units"
print(f"Second put created {len(units_2)} units (not deduplicated due to date difference)")
if __name__ == "__main__":
pytest.main([__file__, "-v", "-s"])
+107
View File
@@ -0,0 +1,107 @@
"""
Test that the improved prompt extracts detailed, comprehensive facts.
"""
import pytest
from memory.llm_client import extract_facts_from_text
@pytest.mark.asyncio
async def test_detailed_extraction_preserves_context():
"""Test that facts preserve all context and details."""
text = """
Alice mentioned she works at Google in Mountain View on the AI research team.
She joined last year after finishing her PhD at Stanford, and she's currently
focused on improving large language model safety through red teaming and
adversarial testing. She said the work is challenging but very rewarding because
it directly impacts millions of users.
"""
facts = await extract_facts_from_text(text)
print(f"\nExtracted {len(facts)} facts:")
for i, fact in enumerate(facts, 1):
print(f"{i}. {fact['fact']}")
print(f" Type: {fact['type']}, Speaker: {fact['speaker']}, Confidence: {fact['confidence']}\n")
# Verify we got facts
assert len(facts) > 0, "Should extract at least one fact"
# Check that facts contain detailed information
fact_texts = [f['fact'].lower() for f in facts]
combined_facts = ' '.join(fact_texts)
# Should preserve location details
assert 'mountain view' in combined_facts, "Should preserve specific location 'Mountain View'"
# Should preserve team/department
assert 'ai' in combined_facts or 'research' in combined_facts, "Should preserve team information"
# Should preserve educational background
assert 'stanford' in combined_facts or 'phd' in combined_facts, "Should preserve educational background"
# Should preserve work details
assert 'safety' in combined_facts or 'red teaming' in combined_facts or 'adversarial' in combined_facts, \
"Should preserve specific work focus details"
# Check that at least one fact is reasonably detailed (not just "Alice works at Google")
detailed_fact_found = any(len(f['fact'].split()) >= 10 for f in facts)
assert detailed_fact_found, "At least one fact should be detailed (10+ words)"
@pytest.mark.asyncio
async def test_numbers_and_metrics_preserved():
"""Test that numbers, percentages, and metrics are preserved."""
text = """
Bob explained that the new caching algorithm reduced API latency by 40%
compared to the baseline, processing 10,000 requests per second instead
of the previous 7,000. This improvement was achieved by implementing a
two-tier LRU cache with 1GB memory allocation.
"""
facts = await extract_facts_from_text(text)
print(f"\nExtracted {len(facts)} facts:")
for fact in facts:
print(f"- {fact['fact']}")
combined = ' '.join([f['fact'] for f in facts])
# Should preserve specific numbers
assert '40' in combined or 'forty' in combined.lower(), "Should preserve percentage"
assert '10,000' in combined or '10000' in combined or 'ten thousand' in combined.lower(), \
"Should preserve request rate"
assert 'cache' in combined.lower(), "Should preserve technical details"
@pytest.mark.asyncio
async def test_reasons_and_causality_preserved():
"""Test that reasons, causes, and explanations are preserved."""
text = """
Sarah has been meditating every morning for the past 6 months because
she found it significantly reduced her anxiety levels and improved her
focus during work hours. She started this practice after reading a research
paper on mindfulness benefits.
"""
facts = await extract_facts_from_text(text)
print(f"\nExtracted {len(facts)} facts:")
for fact in facts:
print(f"- {fact['fact']}")
combined = ' '.join([f['fact'] for f in facts])
# Should preserve the causal relationship (because/reason)
assert any(keyword in combined.lower() for keyword in ['because', 'reduced', 'anxiety', 'improved']), \
"Should preserve the reason/causality"
# Should preserve frequency
assert 'morning' in combined.lower() or 'every' in combined.lower(), \
"Should preserve frequency information"
if __name__ == "__main__":
pytest.main([__file__, "-v", "-s"])
+212
View File
@@ -0,0 +1,212 @@
"""
Test entity-aware memory linking functionality.
Tests that entity resolution connects memories about the same person/place/thing.
"""
import pytest
from datetime import datetime, timedelta, timezone
def utcnow():
"""Get current UTC time with timezone info."""
return datetime.now(timezone.utc)
def test_entity_extraction_and_linking(memory, clean_agent, db_connection):
"""Test that entities are extracted and linked correctly."""
agent_id = clean_agent
# Store memories about Alice's hiking hobby
memory.put(
agent_id=agent_id,
content="Alice told me she loves hiking in the mountains. "
"She goes hiking every weekend in Yosemite.",
context="Casual conversation about hobbies",
event_date=utcnow() - timedelta(days=7),
)
# Store memories about Alice's work (different context!)
memory.put(
agent_id=agent_id,
content="Alice works at Google as a software engineer. "
"She joined Google last year and loves the culture.",
context="Discussion about careers",
event_date=utcnow() - timedelta(days=3),
)
# Store more about hiking (no Alice mention)
memory.put(
agent_id=agent_id,
content="Bob mentioned he enjoys rock climbing. "
"He climbs in Yosemite too, on weekends.",
context="Outdoor activities discussion",
event_date=utcnow() - timedelta(days=1),
)
# Store another Alice memory
memory.put(
agent_id=agent_id,
content="Alice is working on a Python project at Google. "
"The project uses machine learning.",
context="Technical discussion",
event_date=utcnow(),
)
# Verify entities were extracted
cursor = db_connection.cursor()
cursor.execute("""
SELECT canonical_name, entity_type, mention_count
FROM entities
WHERE agent_id = %s
ORDER BY mention_count DESC
""", (agent_id,))
entities = cursor.fetchall()
entity_names = [e[0] for e in entities]
# Should have Alice, Google, Yosemite, Bob
assert "Alice" in entity_names, "Alice entity should be extracted"
assert "Google" in entity_names, "Google entity should be extracted"
assert "Yosemite" in entity_names, "Yosemite entity should be extracted"
assert "Bob" in entity_names, "Bob entity should be extracted"
# Alice should have multiple mentions
alice_entity = next((e for e in entities if e[0] == "Alice"), None)
assert alice_entity is not None
assert alice_entity[2] >= 3, "Alice should have at least 3 mentions"
# Verify entity links exist
cursor.execute("""
SELECT COUNT(*)
FROM memory_links
WHERE link_type = 'entity'
AND from_unit_id IN (
SELECT id FROM memory_units WHERE agent_id = %s
)
""", (agent_id,))
entity_link_count = cursor.fetchone()[0]
assert entity_link_count > 0, "Entity links should be created"
cursor.close()
def test_entity_search_retrieves_all_related_memories(memory, clean_agent):
"""Test that searching for an entity retrieves ALL memories about that entity."""
agent_id = clean_agent
# Store diverse memories about Alice
memory.put(
agent_id=agent_id,
content="Alice loves hiking in the mountains.",
context="Hobbies",
event_date=utcnow() - timedelta(days=7),
)
memory.put(
agent_id=agent_id,
content="Alice works at Google as a software engineer.",
context="Career",
event_date=utcnow() - timedelta(days=3),
)
memory.put(
agent_id=agent_id,
content="Alice is working on a Python machine learning project.",
context="Technical",
event_date=utcnow(),
)
# Query about Alice - should get ALL Alice memories via entity links
results = memory.search(
agent_id=agent_id,
query="What does Alice do?",
thinking_budget=30,
top_k=10,
)
# Should retrieve multiple memories about Alice
assert len(results) >= 2, "Should find multiple memories about Alice"
# Check that results contain Alice-related content
alice_mentions = sum(1 for r in results if "Alice" in r['text'])
assert alice_mentions >= 2, "Multiple results should mention Alice"
def test_entity_disambiguation(memory, clean_agent, db_connection):
"""Test that entity disambiguation correctly identifies same vs different entities."""
agent_id = clean_agent
# Store two memories about "Alice" in different contexts
memory.put(
agent_id=agent_id,
content="Alice from engineering loves Python.",
context="Tech team",
event_date=utcnow() - timedelta(days=2),
)
memory.put(
agent_id=agent_id,
content="Alice from engineering is working on a new project.",
context="Tech team",
event_date=utcnow(),
)
# Check that only ONE Alice entity was created (not two)
cursor = db_connection.cursor()
cursor.execute("""
SELECT COUNT(*)
FROM entities
WHERE agent_id = %s AND canonical_name = 'Alice'
""", (agent_id,))
alice_count = cursor.fetchone()[0]
assert alice_count == 1, "Should create only one Alice entity (disambiguation)"
cursor.close()
def test_link_type_distribution(memory, clean_agent, db_connection):
"""Test that all three link types (temporal, semantic, entity) are created."""
agent_id = clean_agent
# Store related memories
memory.put(
agent_id=agent_id,
content="Alice works at Google. She loves her job.",
context="Career",
event_date=utcnow() - timedelta(hours=2),
)
memory.put(
agent_id=agent_id,
content="Bob also works at Google. He is in sales.",
context="Career",
event_date=utcnow() - timedelta(hours=1),
)
memory.put(
agent_id=agent_id,
content="Google is a great company to work for.",
context="Career",
event_date=utcnow(),
)
# Check link types
cursor = db_connection.cursor()
cursor.execute("""
SELECT link_type, COUNT(*) as count
FROM memory_links ml
JOIN memory_units mu ON ml.from_unit_id = mu.id
WHERE mu.agent_id = %s
GROUP BY link_type
ORDER BY count DESC
""", (agent_id,))
link_types = {row[0]: row[1] for row in cursor.fetchall()}
# Should have at least temporal and entity links (semantic depends on similarity threshold)
assert 'temporal' in link_types, "Should create temporal links"
assert 'entity' in link_types or 'semantic' in link_types, "Should create entity or semantic links"
cursor.close()
+130
View File
@@ -0,0 +1,130 @@
"""
Test LLM-based fact extraction.
"""
import pytest
from memory.llm_client import extract_facts_from_text
from memory.utils import extract_facts
async def test_fact_extraction_filters_pleasantries():
"""Test that fact extraction filters out social pleasantries."""
conversation = """
Host: Welcome to the show, Marta! Thanks for joining us.
Marta: Oh, thank you so much for having me!
Host: So tell us, what do you do?
Marta: I work at Google as a software engineer. I've been there for 3 years now.
Host: That's amazing!
Marta: Yeah, I really enjoy it. I mostly work on AI infrastructure.
Host: Uh-huh, interesting.
Marta: And I'm also passionate about hiking. I go to Yosemite almost every weekend.
Host: Wow, that sounds great!
"""
facts = await extract_facts_from_text(conversation)
# Extract just the fact texts
fact_texts = [f['fact'].lower() for f in facts]
print("\nExtracted facts:")
for fact in facts:
print(f" - {fact['fact']} (speaker: {fact['speaker']}, type: {fact['type']})")
# Should extract meaningful facts
assert any('google' in fact and 'software engineer' in fact for fact in fact_texts), \
"Should extract Marta's job at Google"
assert any('yosemite' in fact and 'hiking' in fact for fact in fact_texts), \
"Should extract Marta's hiking hobby"
# Should NOT extract pleasantries
assert not any('thank you' in fact for fact in fact_texts), \
"Should not extract 'thank you'"
assert not any('amazing' in fact and len(fact.split()) < 5 for fact in fact_texts), \
"Should not extract simple reactions like 'that's amazing'"
assert not any('uh-huh' in fact for fact in fact_texts), \
"Should not extract acknowledgments"
async def test_fact_extraction_makes_self_contained():
"""Test that facts are self-contained (pronouns resolved)."""
conversation = """
Alice told me she works at Microsoft.
She mentioned that she's been there for 5 years.
She really enjoys her team.
"""
facts = await extract_facts_from_text(conversation)
print("\nExtracted facts:")
for fact in facts:
print(f" - {fact['fact']}")
# All facts should mention "Alice" explicitly, not "she"
for fact in facts:
fact_text = fact['fact'].lower()
# If it's about Alice, it should say "alice" not "she"
if 'microsoft' in fact_text or 'team' in fact_text:
assert 'alice' in fact_text, \
f"Fact should be self-contained with 'Alice', not pronouns: {fact['fact']}"
async def test_extract_facts_util_function():
"""Test the utils.extract_facts() wrapper function."""
text = """
Bob is a chef in New York. He owns a restaurant called "The Kitchen".
Thank you! Yeah, uh-huh.
"""
facts = await extract_facts(text)
print("\nExtracted facts:")
for fact in facts:
print(f" - {fact}")
assert len(facts) > 0, "Should extract at least one fact"
assert any('bob' in fact.lower() and 'chef' in fact.lower() for fact in facts), \
"Should extract Bob's profession"
assert not any('thank you' in fact.lower() for fact in facts), \
"Should filter out pleasantries"
async def test_extract_facts_basic():
"""Test basic fact extraction."""
text = "Alice works at Google. She loves Python programming."
facts = await extract_facts(text)
assert len(facts) > 0, "Should extract at least one fact"
assert any('alice' in fact.lower() for fact in facts), "Should extract facts about Alice"
if __name__ == "__main__":
import asyncio
# Run a manual test
async def main():
conversation = """
Host: Welcome to the AI podcast! Today we have Dr. Sarah Chen with us.
Sarah: Hi! Thanks for having me.
Host: So Sarah, tell us about your work.
Sarah: I'm a researcher at Stanford focusing on large language models.
Host: Oh wow!
Sarah: Yeah, I've been studying how LLMs handle reasoning tasks. It's fascinating.
Sarah: We published a paper last month showing that chain-of-thought prompting improves accuracy by 40%.
Host: That's incredible!
Sarah: And I'm also advising a startup called MemoryAI that's building long-term memory systems.
Host: Cool, cool.
"""
print("Testing fact extraction with podcast conversation:")
print("=" * 60)
facts = await extract_facts_from_text(conversation)
print(f"\nExtracted {len(facts)} facts:\n")
for i, fact in enumerate(facts, 1):
print(f"{i}. {fact['fact']}")
print(f" Speaker: {fact['speaker']}, Type: {fact['type']}, Confidence: {fact['confidence']}\n")
asyncio.run(main())
+223
View File
@@ -0,0 +1,223 @@
"""
Test basic memory operations: PUT, SEARCH, GET_RECENT.
Tests the core functionality of the temporal + semantic memory system.
"""
import pytest
import asyncio
from datetime import datetime, timedelta, timezone
def utcnow():
"""Get current UTC time with timezone info."""
return datetime.now(timezone.utc)
@pytest.mark.asyncio
async def test_put_creates_memory_units(memory, clean_agent, db_connection):
"""Test that PUT operation creates memory units."""
agent_id = clean_agent
# Store a conversation
await memory.put_async(
agent_id=agent_id,
content="Alice told me she loves hiking in the mountains. "
"She mentioned that she goes hiking every weekend. "
"Her favorite trail is in Yosemite National Park.",
context="Casual conversation about hobbies",
event_date=utcnow() - timedelta(hours=2),
)
# Verify memory units were created
cursor = db_connection.cursor()
cursor.execute("SELECT COUNT(*) FROM memory_units WHERE agent_id = %s", (agent_id,))
count = cursor.fetchone()[0]
assert count > 0, "Memory units should be created"
assert count <= 3, "Should create approximately 3 units (one per sentence)"
cursor.close()
@pytest.mark.asyncio
async def test_put_creates_temporal_links(memory, clean_agent, db_connection):
"""Test that temporal links are created between recent memories."""
agent_id = clean_agent
# Store two memories close in time
await memory.put_async(
agent_id=agent_id,
content="Alice loves hiking.",
context="Hobbies",
event_date=utcnow() - timedelta(hours=2),
)
await memory.put_async(
agent_id=agent_id,
content="Bob enjoys climbing.",
context="Sports",
event_date=utcnow() - timedelta(hours=1),
)
# Verify temporal links were created
cursor = db_connection.cursor()
cursor.execute("""
SELECT COUNT(*)
FROM memory_links
WHERE link_type = 'temporal'
AND from_unit_id IN (
SELECT id FROM memory_units WHERE agent_id = %s
)
""", (agent_id,))
temporal_link_count = cursor.fetchone()[0]
assert temporal_link_count > 0, "Temporal links should be created"
cursor.close()
@pytest.mark.asyncio
async def test_put_creates_semantic_links(memory, clean_agent, db_connection):
"""Test that semantic links are created between similar memories."""
agent_id = clean_agent
# Store semantically similar memories
await memory.put_async(
agent_id=agent_id,
content="Alice loves hiking in the mountains.",
context="Hobbies",
event_date=utcnow() - timedelta(days=2),
)
await memory.put_async(
agent_id=agent_id,
content="Bob enjoys climbing mountains.",
context="Sports",
event_date=utcnow(),
)
# Verify semantic links were created
cursor = db_connection.cursor()
cursor.execute("""
SELECT COUNT(*)
FROM memory_links
WHERE link_type = 'semantic'
AND from_unit_id IN (
SELECT id FROM memory_units WHERE agent_id = %s
)
""", (agent_id,))
semantic_link_count = cursor.fetchone()[0]
# Semantic links may or may not be created depending on similarity threshold
# So we just check that the query works
assert semantic_link_count >= 0, "Query should execute successfully"
cursor.close()
@pytest.mark.asyncio
async def test_search_with_spreading_activation(memory, clean_agent):
"""Test search using spreading activation algorithm."""
agent_id = clean_agent
# Store memories about outdoor activities
await memory.put_async(
agent_id=agent_id,
content="Alice told me she loves hiking in the mountains. "
"She goes hiking every weekend.",
context="Casual conversation about hobbies",
event_date=utcnow() - timedelta(hours=2),
)
await memory.put_async(
agent_id=agent_id,
content="Bob mentioned he enjoys rock climbing. "
"He climbs mountains on weekends too.",
context="Discussion about outdoor sports",
event_date=utcnow() - timedelta(hours=1),
)
# Search for outdoor activities
results = memory.search(
agent_id=agent_id,
query="outdoor mountain activities",
thinking_budget=50,
top_k=5,
)
assert len(results) > 0, "Search should return results"
# Verify result structure
for result in results:
assert 'id' in result, "Result should have id"
assert 'text' in result, "Result should have text"
assert 'weight' in result, "Result should have weight"
assert 'activation' in result, "Result should have activation"
assert 'recency' in result, "Result should have recency"
assert 'frequency' in result, "Result should have frequency"
# Results should be sorted by weight (descending)
weights = [r['weight'] for r in results]
assert weights == sorted(weights, reverse=True), "Results should be sorted by weight"
@pytest.mark.asyncio
async def test_search_returns_relevant_memories(memory, clean_agent):
"""Test that search returns semantically relevant memories."""
agent_id = clean_agent
# Store memories about different topics
await memory.put_async(
agent_id=agent_id,
content="Alice loves hiking in the mountains.",
context="Hobbies",
event_date=utcnow() - timedelta(hours=2),
)
await memory.put_async(
agent_id=agent_id,
content="Bob is working on a Python web application.",
context="Tech",
event_date=utcnow() - timedelta(hours=1),
)
# Search for programming-related memories
results = memory.search(
agent_id=agent_id,
query="software development",
thinking_budget=50,
top_k=3,
)
# Should find the programming-related memory
assert len(results) > 0, "Search should return results"
# Top result should be about programming (more relevant)
top_result_text = results[0]['text'].lower()
assert 'python' in top_result_text or 'application' in top_result_text or 'working' in top_result_text, \
"Top result should be about programming"
@pytest.mark.asyncio
async def test_search_with_no_results(memory, clean_agent):
"""Test search behavior when no relevant memories exist."""
agent_id = clean_agent
# Store unrelated memories
await memory.put_async(
agent_id=agent_id,
content="Alice loves cooking pasta.",
context="Food",
event_date=utcnow(),
)
# Search for something completely unrelated
results = memory.search(
agent_id=agent_id,
query="quantum physics theories",
thinking_budget=20,
top_k=5,
)
# May return low-scoring results or empty list
assert isinstance(results, list), "Search should return a list"
+96
View File
@@ -0,0 +1,96 @@
"""
Test visualization functionality.
Tests memory graph data retrieval (not actual rendering).
"""
import pytest
from datetime import datetime, timedelta, timezone
def utcnow():
"""Get current UTC time with timezone info."""
return datetime.now(timezone.utc)
def test_get_memory_graph_data(memory, clean_agent):
"""Test retrieval of memory graph data for visualization."""
agent_id = clean_agent
# Store some memories
memory.put(
agent_id=agent_id,
content="Alice loves hiking in the mountains.",
context="Hobbies",
event_date=utcnow() - timedelta(hours=2),
)
memory.put(
agent_id=agent_id,
content="Bob enjoys rock climbing.",
context="Sports",
event_date=utcnow() - timedelta(hours=1),
)
memory.put(
agent_id=agent_id,
content="Alice is working on a Python project.",
context="Tech",
event_date=utcnow(),
)
# Get graph data
units, links = memory.get_memory_graph_data(agent_id)
assert isinstance(units, list), "Units should be a list"
assert isinstance(links, list), "Links should be a list"
assert len(units) > 0, "Should have memory units"
# Verify unit structure
for unit in units:
assert 'id' in unit, "Unit should have id"
assert 'text' in unit, "Unit should have text"
assert 'context' in unit, "Unit should have context"
assert 'event_date' in unit, "Unit should have event_date"
assert 'access_count' in unit, "Unit should have access_count"
# Links may or may not exist depending on similarity/proximity
if len(links) > 0:
# Verify link structure
for link in links:
assert 'from_unit_id' in link, "Link should have from_unit_id"
assert 'to_unit_id' in link, "Link should have to_unit_id"
assert 'link_type' in link, "Link should have link_type"
assert 'weight' in link, "Link should have weight"
assert link['link_type'] in ['temporal', 'semantic', 'entity'], \
"Link type should be temporal, semantic, or entity"
def test_memory_graph_has_correct_agent_data(memory, clean_agent):
"""Test that graph data only includes data for the specified agent."""
agent_id = clean_agent
other_agent_id = "other_agent"
# Store memories for test agent
memory.put(
agent_id=agent_id,
content="Alice loves hiking.",
context="Hobbies",
event_date=utcnow(),
)
# Store memories for another agent
memory.put(
agent_id=other_agent_id,
content="Charlie enjoys swimming.",
context="Sports",
event_date=utcnow(),
)
# Get graph data for test agent
units, links = memory.get_memory_graph_data(agent_id)
# Should only include test agent's data
for unit in units:
# Verify by checking text content (Alice should be present, Charlie should not)
unit_text = unit['text']
assert 'Charlie' not in unit_text, "Should not include other agent's memories"
Generated
+3526
View File
File diff suppressed because it is too large Load Diff
+297
View File
@@ -0,0 +1,297 @@
"""
Interactive HTML graph visualization of memory system.
Uses pyvis to create a smooth, interactive network graph that can be
explored in the browser. Shows all memory units and their links with weights.
"""
import psycopg2
from dotenv import load_dotenv
import os
from pyvis.network import Network
import networkx as nx
load_dotenv()
def create_interactive_graph():
"""Create an interactive HTML graph visualization."""
# Connect to database
conn = psycopg2.connect(os.getenv('DATABASE_URL'))
cursor = conn.cursor()
# Get all memory units (no agent_id filter)
cursor.execute("""
SELECT id, text, event_date, context
FROM memory_units
ORDER BY event_date
""")
units = cursor.fetchall()
# Get all links with weights (no agent_id filter)
cursor.execute("""
SELECT
ml.from_unit_id,
ml.to_unit_id,
ml.link_type,
ml.weight,
e.canonical_name as entity_name
FROM memory_links ml
LEFT JOIN entities e ON ml.entity_id = e.id
ORDER BY ml.link_type, ml.weight DESC
""")
links = cursor.fetchall()
# Get entity information (no agent_id filter)
cursor.execute("""
SELECT ue.unit_id, e.canonical_name, e.entity_type
FROM unit_entities ue
JOIN entities e ON ue.entity_id = e.id
ORDER BY ue.unit_id
""")
unit_entities = cursor.fetchall()
cursor.close()
conn.close()
# Build entity mapping
entity_map = {}
for unit_id, entity_name, entity_type in unit_entities:
if unit_id not in entity_map:
entity_map[unit_id] = []
entity_map[unit_id].append(f"{entity_name} ({entity_type})")
# Create pyvis network
net = Network(
height="900px",
width="100%",
bgcolor="#ffffff",
font_color="#000000",
heading="Entity-Aware Memory Graph - Interactive Visualization"
)
# Configure physics for smooth layout with performance optimizations
net.set_options("""
{
"nodes": {
"font": {
"size": 14,
"face": "Tahoma"
},
"borderWidth": 2,
"borderWidthSelected": 3
},
"edges": {
"smooth": {
"enabled": false
},
"font": {
"size": 10,
"align": "middle"
}
},
"physics": {
"enabled": true,
"stabilization": {
"enabled": true,
"iterations": 100,
"updateInterval": 10
},
"barnesHut": {
"gravitationalConstant": -12000,
"centralGravity": 0.2,
"springLength": 350,
"springConstant": 0.02,
"damping": 0.09,
"avoidOverlap": 0.8
},
"solver": "barnesHut",
"timestep": 0.5,
"adaptiveTimestep": true
},
"interaction": {
"hover": true,
"tooltipDelay": 100,
"navigationButtons": true,
"keyboard": true
}
}
""")
# Add nodes
for unit_id, text, event_date, context in units:
# Truncate text for display
display_text = text[:50] + "..." if len(text) > 50 else text
# Get entities
entities = entity_map.get(unit_id, [])
entity_str = "\\n".join(entities) if entities else "No entities"
# Build node label and title (hover)
label = display_text
title = f"""
<b>Text:</b> {text}<br>
<b>Date:</b> {event_date.date()}<br>
<b>Context:</b> {context}<br>
<b>Entities:</b> {entity_str}
"""
# Color by entity count
if len(entities) == 0:
color = "#e0e0e0" # Gray
size = 20
elif len(entities) == 1:
color = "#90caf9" # Light blue
size = 25
else:
color = "#42a5f5" # Dark blue
size = 30
net.add_node(
str(unit_id),
label=label,
title=title,
color=color,
size=size,
shape="box",
font={"color": "#000000"}
)
# Add edges with colors and weights
for from_id, to_id, link_type, weight, entity_name in links:
# Set color and style based on link type
if link_type == 'temporal':
color = "#00bcd4" # Cyan
dashes = [5, 5]
width = 0.5
label = f"T: {weight:.2f}"
elif link_type == 'semantic':
color = "#ff69b4" # Pink
dashes = False
width = 0.5
label = f"S: {weight:.2f}"
elif link_type == 'entity':
color = "#ffd700" # Gold
dashes = False
width = 0.8
label = f"{entity_name}: {weight:.2f}"
else:
color = "#999999"
dashes = False
width = 0.5
label = f"{weight:.2f}"
net.add_edge(
str(from_id),
str(to_id),
value=weight * 1, # Scale for visual thickness
color=color,
dashes=dashes,
width=width,
label=label,
title=f"{link_type.upper()}: {weight:.3f}" + (f" (Entity: {entity_name})" if entity_name else "")
)
# Add legend as HTML
legend_html = """
<div style="position: absolute; top: 80px; left: 10px; background: white;
padding: 15px; border: 2px solid #333; border-radius: 8px;
font-family: Tahoma; box-shadow: 2px 2px 8px rgba(0,0,0,0.3); z-index: 1000;">
<h3 style="margin-top: 0; border-bottom: 2px solid #333; padding-bottom: 5px;">Legend</h3>
<h4 style="margin-bottom: 5px;">Link Types:</h4>
<div style="margin-left: 10px;">
<div style="margin: 5px 0;">
<span style="display: inline-block; width: 40px; height: 1px;
background: #00bcd4; border-top: 1px dashed #00bcd4;
vertical-align: middle;"></span>
<span style="margin-left: 10px;"><b>Temporal</b> - Time-based (cyan, dashed)</span>
</div>
<div style="margin: 5px 0;">
<span style="display: inline-block; width: 40px; height: 1px;
background: #ff69b4; vertical-align: middle;"></span>
<span style="margin-left: 10px;"><b>Semantic</b> - Meaning-based (pink, solid)</span>
</div>
<div style="margin: 5px 0;">
<span style="display: inline-block; width: 40px; height: 1.5px;
background: #ffd700; vertical-align: middle;"></span>
<span style="margin-left: 10px;"><b>Entity</b> - Same entity (gold)</span>
</div>
</div>
<h4 style="margin-bottom: 5px; margin-top: 15px;">Node Colors:</h4>
<div style="margin-left: 10px;">
<div style="margin: 5px 0;">
<span style="display: inline-block; width: 20px; height: 20px;
background: #e0e0e0; border: 1px solid #999;
vertical-align: middle;"></span>
<span style="margin-left: 10px;">Gray - No entities</span>
</div>
<div style="margin: 5px 0;">
<span style="display: inline-block; width: 20px; height: 20px;
background: #90caf9; border: 1px solid #999;
vertical-align: middle;"></span>
<span style="margin-left: 10px;">Light Blue - 1 entity</span>
</div>
<div style="margin: 5px 0;">
<span style="display: inline-block; width: 20px; height: 20px;
background: #42a5f5; border: 1px solid #999;
vertical-align: middle;"></span>
<span style="margin-left: 10px;">Dark Blue - 2+ entities</span>
</div>
</div>
<div style="margin-top: 15px; padding-top: 10px; border-top: 1px solid #ccc;
font-size: 11px; color: #666;">
<b>Tip:</b> Hover over nodes/edges for details<br>
<b>Controls:</b> Drag to move, scroll to zoom
</div>
</div>
"""
# Generate the HTML
output_file = "memory_graph_interactive.html"
net.save_graph(output_file)
# Read the generated HTML and inject our legend
with open(output_file, 'r') as f:
html_content = f.read()
# Inject legend after the opening body tag
html_content = html_content.replace('<body>', '<body>' + legend_html)
# Add script to disable physics after stabilization for better performance
physics_script = """
<script type="text/javascript">
// Disable physics after initial stabilization for better performance
network.on("stabilizationIterationsDone", function () {
network.setOptions({ physics: false });
console.log("Physics disabled - graph should be much more responsive now!");
});
</script>
"""
html_content = html_content.replace('</body>', physics_script + '</body>')
# Write back
with open(output_file, 'w') as f:
f.write(html_content)
print(f"\n{'='*80}")
print("INTERACTIVE GRAPH GENERATED")
print(f"{'='*80}")
print(f"\nFile: {output_file}")
print(f"Units: {len(units)}")
print(f"Links: {len(links)}")
print("\nFeatures:")
print(" • Smooth, physics-based layout")
print(" • Interactive - drag nodes, zoom, pan")
print(" • Hover for details on nodes and edges")
print(" • Color-coded by link type and entity count")
print(" • Built-in navigation controls")
print(f"\n{'='*80}")
print(f"✓ Open {output_file} in your browser to explore!")
print(f"{'='*80}\n")
if __name__ == "__main__":
create_interactive_graph()
+189
View File
@@ -0,0 +1,189 @@
function neighbourhoodHighlight(params) {
// console.log("in nieghbourhoodhighlight");
allNodes = nodes.get({ returnType: "Object" });
// originalNodes = JSON.parse(JSON.stringify(allNodes));
// if something is selected:
if (params.nodes.length > 0) {
highlightActive = true;
var i, j;
var selectedNode = params.nodes[0];
var degrees = 2;
// mark all nodes as hard to read.
for (let nodeId in allNodes) {
// nodeColors[nodeId] = allNodes[nodeId].color;
allNodes[nodeId].color = "rgba(200,200,200,0.5)";
if (allNodes[nodeId].hiddenLabel === undefined) {
allNodes[nodeId].hiddenLabel = allNodes[nodeId].label;
allNodes[nodeId].label = undefined;
}
}
var connectedNodes = network.getConnectedNodes(selectedNode);
var allConnectedNodes = [];
// get the second degree nodes
for (i = 1; i < degrees; i++) {
for (j = 0; j < connectedNodes.length; j++) {
allConnectedNodes = allConnectedNodes.concat(
network.getConnectedNodes(connectedNodes[j])
);
}
}
// all second degree nodes get a different color and their label back
for (i = 0; i < allConnectedNodes.length; i++) {
// allNodes[allConnectedNodes[i]].color = "pink";
allNodes[allConnectedNodes[i]].color = "rgba(150,150,150,0.75)";
if (allNodes[allConnectedNodes[i]].hiddenLabel !== undefined) {
allNodes[allConnectedNodes[i]].label =
allNodes[allConnectedNodes[i]].hiddenLabel;
allNodes[allConnectedNodes[i]].hiddenLabel = undefined;
}
}
// all first degree nodes get their own color and their label back
for (i = 0; i < connectedNodes.length; i++) {
// allNodes[connectedNodes[i]].color = undefined;
allNodes[connectedNodes[i]].color = nodeColors[connectedNodes[i]];
if (allNodes[connectedNodes[i]].hiddenLabel !== undefined) {
allNodes[connectedNodes[i]].label =
allNodes[connectedNodes[i]].hiddenLabel;
allNodes[connectedNodes[i]].hiddenLabel = undefined;
}
}
// the main node gets its own color and its label back.
// allNodes[selectedNode].color = undefined;
allNodes[selectedNode].color = nodeColors[selectedNode];
if (allNodes[selectedNode].hiddenLabel !== undefined) {
allNodes[selectedNode].label = allNodes[selectedNode].hiddenLabel;
allNodes[selectedNode].hiddenLabel = undefined;
}
} else if (highlightActive === true) {
// console.log("highlightActive was true");
// reset all nodes
for (let nodeId in allNodes) {
// allNodes[nodeId].color = "purple";
allNodes[nodeId].color = nodeColors[nodeId];
// delete allNodes[nodeId].color;
if (allNodes[nodeId].hiddenLabel !== undefined) {
allNodes[nodeId].label = allNodes[nodeId].hiddenLabel;
allNodes[nodeId].hiddenLabel = undefined;
}
}
highlightActive = false;
}
// transform the object into an array
var updateArray = [];
if (params.nodes.length > 0) {
for (let nodeId in allNodes) {
if (allNodes.hasOwnProperty(nodeId)) {
// console.log(allNodes[nodeId]);
updateArray.push(allNodes[nodeId]);
}
}
nodes.update(updateArray);
} else {
// console.log("Nothing was selected");
for (let nodeId in allNodes) {
if (allNodes.hasOwnProperty(nodeId)) {
// console.log(allNodes[nodeId]);
// allNodes[nodeId].color = {};
updateArray.push(allNodes[nodeId]);
}
}
nodes.update(updateArray);
}
}
function filterHighlight(params) {
allNodes = nodes.get({ returnType: "Object" });
// if something is selected:
if (params.nodes.length > 0) {
filterActive = true;
let selectedNodes = params.nodes;
// hiding all nodes and saving the label
for (let nodeId in allNodes) {
allNodes[nodeId].hidden = true;
if (allNodes[nodeId].savedLabel === undefined) {
allNodes[nodeId].savedLabel = allNodes[nodeId].label;
allNodes[nodeId].label = undefined;
}
}
for (let i=0; i < selectedNodes.length; i++) {
allNodes[selectedNodes[i]].hidden = false;
if (allNodes[selectedNodes[i]].savedLabel !== undefined) {
allNodes[selectedNodes[i]].label = allNodes[selectedNodes[i]].savedLabel;
allNodes[selectedNodes[i]].savedLabel = undefined;
}
}
} else if (filterActive === true) {
// reset all nodes
for (let nodeId in allNodes) {
allNodes[nodeId].hidden = false;
if (allNodes[nodeId].savedLabel !== undefined) {
allNodes[nodeId].label = allNodes[nodeId].savedLabel;
allNodes[nodeId].savedLabel = undefined;
}
}
filterActive = false;
}
// transform the object into an array
var updateArray = [];
if (params.nodes.length > 0) {
for (let nodeId in allNodes) {
if (allNodes.hasOwnProperty(nodeId)) {
updateArray.push(allNodes[nodeId]);
}
}
nodes.update(updateArray);
} else {
for (let nodeId in allNodes) {
if (allNodes.hasOwnProperty(nodeId)) {
updateArray.push(allNodes[nodeId]);
}
}
nodes.update(updateArray);
}
}
function selectNode(nodes) {
network.selectNodes(nodes);
neighbourhoodHighlight({ nodes: nodes });
return nodes;
}
function selectNodes(nodes) {
network.selectNodes(nodes);
filterHighlight({nodes: nodes});
return nodes;
}
function highlightFilter(filter) {
let selectedNodes = []
let selectedProp = filter['property']
if (filter['item'] === 'node') {
let allNodes = nodes.get({ returnType: "Object" });
for (let nodeId in allNodes) {
if (allNodes[nodeId][selectedProp] && filter['value'].includes((allNodes[nodeId][selectedProp]).toString())) {
selectedNodes.push(nodeId)
}
}
}
else if (filter['item'] === 'edge'){
let allEdges = edges.get({returnType: 'object'});
// check if the selected property exists for selected edge and select the nodes connected to the edge
for (let edge in allEdges) {
if (allEdges[edge][selectedProp] && filter['value'].includes((allEdges[edge][selectedProp]).toString())) {
selectedNodes.push(allEdges[edge]['from'])
selectedNodes.push(allEdges[edge]['to'])
}
}
}
selectNodes(selectedNodes)
}
+356
View File
@@ -0,0 +1,356 @@
/**
* Tom Select v2.0.0-rc.4
* Licensed under the Apache License, Version 2.0 (the "License");
*/
!function(e,t){"object"==typeof exports&&"undefined"!=typeof module?module.exports=t():"function"==typeof define&&define.amd?define(t):(e="undefined"!=typeof globalThis?globalThis:e||self).TomSelect=t()}(this,(function(){"use strict"
function e(e,t){e.split(/\s+/).forEach((e=>{t(e)}))}class t{constructor(){this._events={}}on(t,i){e(t,(e=>{this._events[e]=this._events[e]||[],this._events[e].push(i)}))}off(t,i){var s=arguments.length
0!==s?e(t,(e=>{if(1===s)return delete this._events[e]
e in this._events!=!1&&this._events[e].splice(this._events[e].indexOf(i),1)})):this._events={}}trigger(t,...i){var s=this
e(t,(e=>{if(e in s._events!=!1)for(let t of s._events[e])t.apply(s,i)}))}}var i
const s="[̀-ͯ·ʾ]",n=new RegExp(s,"g")
var o
const r={"æ":"ae","ⱥ":"a","ø":"o"},l=new RegExp(Object.keys(r).join("|"),"g"),a=[[67,67],[160,160],[192,438],[452,652],[961,961],[1019,1019],[1083,1083],[1281,1289],[1984,1984],[5095,5095],[7429,7441],[7545,7549],[7680,7935],[8580,8580],[9398,9449],[11360,11391],[42792,42793],[42802,42851],[42873,42897],[42912,42922],[64256,64260],[65313,65338],[65345,65370]],c=e=>e.normalize("NFKD").replace(n,"").toLowerCase().replace(l,(function(e){return r[e]})),d=(e,t="|")=>{if(1==e.length)return e[0]
var i=1
return e.forEach((e=>{i=Math.max(i,e.length)})),1==i?"["+e.join("")+"]":"(?:"+e.join(t)+")"},p=e=>{if(1===e.length)return[[e]]
var t=[]
return p(e.substring(1)).forEach((function(i){var s=i.slice(0)
s[0]=e.charAt(0)+s[0],t.push(s),(s=i.slice(0)).unshift(e.charAt(0)),t.push(s)})),t},u=e=>{void 0===o&&(o=(()=>{var e={}
a.forEach((t=>{for(let s=t[0];s<=t[1];s++){let t=String.fromCharCode(s),n=c(t)
if(n!=t.toLowerCase()){n in e||(e[n]=[n])
var i=new RegExp(d(e[n]),"iu")
t.match(i)||e[n].push(t)}}}))
var t=Object.keys(e)
t=t.sort(((e,t)=>t.length-e.length)),i=new RegExp("("+d(t)+"[̀-ͯ·ʾ]*)","g")
var s={}
return t.sort(((e,t)=>e.length-t.length)).forEach((t=>{var i=p(t).map((t=>(t=t.map((t=>e.hasOwnProperty(t)?d(e[t]):t)),d(t,""))))
s[t]=d(i)})),s})())
return e.normalize("NFKD").toLowerCase().split(i).map((e=>{if(""==e)return""
const t=c(e)
if(o.hasOwnProperty(t))return o[t]
const i=e.normalize("NFC")
return i!=e?d([e,i]):e})).join("")},h=(e,t)=>{if(e)return e[t]},g=(e,t)=>{if(e){for(var i,s=t.split(".");(i=s.shift())&&(e=e[i]););return e}},f=(e,t,i)=>{var s,n
return e?-1===(n=(e+="").search(t.regex))?0:(s=t.string.length/e.length,0===n&&(s+=.5),s*i):0},v=e=>(e+"").replace(/([\$\(-\+\.\?\[-\^\{-\}])/g,"\\$1"),m=(e,t)=>{var i=e[t]
if("function"==typeof i)return i
i&&!Array.isArray(i)&&(e[t]=[i])},y=(e,t)=>{if(Array.isArray(e))e.forEach(t)
else for(var i in e)e.hasOwnProperty(i)&&t(e[i],i)},O=(e,t)=>"number"==typeof e&&"number"==typeof t?e>t?1:e<t?-1:0:(e=c(e+"").toLowerCase())>(t=c(t+"").toLowerCase())?1:t>e?-1:0
class b{constructor(e,t){this.items=e,this.settings=t||{diacritics:!0}}tokenize(e,t,i){if(!e||!e.length)return[]
const s=[],n=e.split(/\s+/)
var o
return i&&(o=new RegExp("^("+Object.keys(i).map(v).join("|")+"):(.*)$")),n.forEach((e=>{let i,n=null,r=null
o&&(i=e.match(o))&&(n=i[1],e=i[2]),e.length>0&&(r=v(e),this.settings.diacritics&&(r=u(r)),t&&(r="\\b"+r)),s.push({string:e,regex:r?new RegExp(r,"iu"):null,field:n})})),s}getScoreFunction(e,t){var i=this.prepareSearch(e,t)
return this._getScoreFunction(i)}_getScoreFunction(e){const t=e.tokens,i=t.length
if(!i)return function(){return 0}
const s=e.options.fields,n=e.weights,o=s.length,r=e.getAttrFn
if(!o)return function(){return 1}
const l=1===o?function(e,t){const i=s[0].field
return f(r(t,i),e,n[i])}:function(e,t){var i=0
if(e.field){const s=r(t,e.field)
!e.regex&&s?i+=1/o:i+=f(s,e,1)}else y(n,((s,n)=>{i+=f(r(t,n),e,s)}))
return i/o}
return 1===i?function(e){return l(t[0],e)}:"and"===e.options.conjunction?function(e){for(var s,n=0,o=0;n<i;n++){if((s=l(t[n],e))<=0)return 0
o+=s}return o/i}:function(e){var s=0
return y(t,(t=>{s+=l(t,e)})),s/i}}getSortFunction(e,t){var i=this.prepareSearch(e,t)
return this._getSortFunction(i)}_getSortFunction(e){var t,i,s
const n=this,o=e.options,r=!e.query&&o.sort_empty?o.sort_empty:o.sort,l=[],a=[]
if("function"==typeof r)return r.bind(this)
const c=function(t,i){return"$score"===t?i.score:e.getAttrFn(n.items[i.id],t)}
if(r)for(t=0,i=r.length;t<i;t++)(e.query||"$score"!==r[t].field)&&l.push(r[t])
if(e.query){for(s=!0,t=0,i=l.length;t<i;t++)if("$score"===l[t].field){s=!1
break}s&&l.unshift({field:"$score",direction:"desc"})}else for(t=0,i=l.length;t<i;t++)if("$score"===l[t].field){l.splice(t,1)
break}for(t=0,i=l.length;t<i;t++)a.push("desc"===l[t].direction?-1:1)
const d=l.length
if(d){if(1===d){const e=l[0].field,t=a[0]
return function(i,s){return t*O(c(e,i),c(e,s))}}return function(e,t){var i,s,n
for(i=0;i<d;i++)if(n=l[i].field,s=a[i]*O(c(n,e),c(n,t)))return s
return 0}}return null}prepareSearch(e,t){const i={}
var s=Object.assign({},t)
if(m(s,"sort"),m(s,"sort_empty"),s.fields){m(s,"fields")
const e=[]
s.fields.forEach((t=>{"string"==typeof t&&(t={field:t,weight:1}),e.push(t),i[t.field]="weight"in t?t.weight:1})),s.fields=e}return{options:s,query:e.toLowerCase().trim(),tokens:this.tokenize(e,s.respect_word_boundaries,i),total:0,items:[],weights:i,getAttrFn:s.nesting?g:h}}search(e,t){var i,s,n=this
s=this.prepareSearch(e,t),t=s.options,e=s.query
const o=t.score||n._getScoreFunction(s)
e.length?y(n.items,((e,n)=>{i=o(e),(!1===t.filter||i>0)&&s.items.push({score:i,id:n})})):y(n.items,((e,t)=>{s.items.push({score:1,id:t})}))
const r=n._getSortFunction(s)
return r&&s.items.sort(r),s.total=s.items.length,"number"==typeof t.limit&&(s.items=s.items.slice(0,t.limit)),s}}const w=e=>{if(e.jquery)return e[0]
if(e instanceof HTMLElement)return e
if(e.indexOf("<")>-1){let t=document.createElement("div")
return t.innerHTML=e.trim(),t.firstChild}return document.querySelector(e)},_=(e,t)=>{var i=document.createEvent("HTMLEvents")
i.initEvent(t,!0,!1),e.dispatchEvent(i)},I=(e,t)=>{Object.assign(e.style,t)},C=(e,...t)=>{var i=A(t);(e=x(e)).map((e=>{i.map((t=>{e.classList.add(t)}))}))},S=(e,...t)=>{var i=A(t);(e=x(e)).map((e=>{i.map((t=>{e.classList.remove(t)}))}))},A=e=>{var t=[]
return y(e,(e=>{"string"==typeof e&&(e=e.trim().split(/[\11\12\14\15\40]/)),Array.isArray(e)&&(t=t.concat(e))})),t.filter(Boolean)},x=e=>(Array.isArray(e)||(e=[e]),e),k=(e,t,i)=>{if(!i||i.contains(e))for(;e&&e.matches;){if(e.matches(t))return e
e=e.parentNode}},F=(e,t=0)=>t>0?e[e.length-1]:e[0],L=(e,t)=>{if(!e)return-1
t=t||e.nodeName
for(var i=0;e=e.previousElementSibling;)e.matches(t)&&i++
return i},P=(e,t)=>{y(t,((t,i)=>{null==t?e.removeAttribute(i):e.setAttribute(i,""+t)}))},E=(e,t)=>{e.parentNode&&e.parentNode.replaceChild(t,e)},T=(e,t)=>{if(null===t)return
if("string"==typeof t){if(!t.length)return
t=new RegExp(t,"i")}const i=e=>3===e.nodeType?(e=>{var i=e.data.match(t)
if(i&&e.data.length>0){var s=document.createElement("span")
s.className="highlight"
var n=e.splitText(i.index)
n.splitText(i[0].length)
var o=n.cloneNode(!0)
return s.appendChild(o),E(n,s),1}return 0})(e):((e=>{if(1===e.nodeType&&e.childNodes&&!/(script|style)/i.test(e.tagName)&&("highlight"!==e.className||"SPAN"!==e.tagName))for(var t=0;t<e.childNodes.length;++t)t+=i(e.childNodes[t])})(e),0)
i(e)},V="undefined"!=typeof navigator&&/Mac/.test(navigator.userAgent)?"metaKey":"ctrlKey"
var j={options:[],optgroups:[],plugins:[],delimiter:",",splitOn:null,persist:!0,diacritics:!0,create:null,createOnBlur:!1,createFilter:null,highlight:!0,openOnFocus:!0,shouldOpen:null,maxOptions:50,maxItems:null,hideSelected:null,duplicates:!1,addPrecedence:!1,selectOnTab:!1,preload:null,allowEmptyOption:!1,loadThrottle:300,loadingClass:"loading",dataAttr:null,optgroupField:"optgroup",valueField:"value",labelField:"text",disabledField:"disabled",optgroupLabelField:"label",optgroupValueField:"value",lockOptgroupOrder:!1,sortField:"$order",searchField:["text"],searchConjunction:"and",mode:null,wrapperClass:"ts-wrapper",controlClass:"ts-control",dropdownClass:"ts-dropdown",dropdownContentClass:"ts-dropdown-content",itemClass:"item",optionClass:"option",dropdownParent:null,copyClassesToDropdown:!1,placeholder:null,hidePlaceholder:null,shouldLoad:function(e){return e.length>0},render:{}}
const q=e=>null==e?null:D(e),D=e=>"boolean"==typeof e?e?"1":"0":e+"",N=e=>(e+"").replace(/&/g,"&amp;").replace(/</g,"&lt;").replace(/>/g,"&gt;").replace(/"/g,"&quot;"),z=(e,t)=>{var i
return function(s,n){var o=this
i&&(o.loading=Math.max(o.loading-1,0),clearTimeout(i)),i=setTimeout((function(){i=null,o.loadedSearches[s]=!0,e.call(o,s,n)}),t)}},R=(e,t,i)=>{var s,n=e.trigger,o={}
for(s in e.trigger=function(){var i=arguments[0]
if(-1===t.indexOf(i))return n.apply(e,arguments)
o[i]=arguments},i.apply(e,[]),e.trigger=n,o)n.apply(e,o[s])},H=(e,t=!1)=>{e&&(e.preventDefault(),t&&e.stopPropagation())},B=(e,t,i,s)=>{e.addEventListener(t,i,s)},K=(e,t)=>!!t&&(!!t[e]&&1===(t.altKey?1:0)+(t.ctrlKey?1:0)+(t.shiftKey?1:0)+(t.metaKey?1:0)),M=(e,t)=>{const i=e.getAttribute("id")
return i||(e.setAttribute("id",t),t)},Q=e=>e.replace(/[\\"']/g,"\\$&"),G=(e,t)=>{t&&e.append(t)}
function U(e,t){var i=Object.assign({},j,t),s=i.dataAttr,n=i.labelField,o=i.valueField,r=i.disabledField,l=i.optgroupField,a=i.optgroupLabelField,c=i.optgroupValueField,d=e.tagName.toLowerCase(),p=e.getAttribute("placeholder")||e.getAttribute("data-placeholder")
if(!p&&!i.allowEmptyOption){let t=e.querySelector('option[value=""]')
t&&(p=t.textContent)}var u,h,g,f,v,m,O={placeholder:p,options:[],optgroups:[],items:[],maxItems:null}
return"select"===d?(h=O.options,g={},f=1,v=e=>{var t=Object.assign({},e.dataset),i=s&&t[s]
return"string"==typeof i&&i.length&&(t=Object.assign(t,JSON.parse(i))),t},m=(e,t)=>{var s=q(e.value)
if(null!=s&&(s||i.allowEmptyOption)){if(g.hasOwnProperty(s)){if(t){var a=g[s][l]
a?Array.isArray(a)?a.push(t):g[s][l]=[a,t]:g[s][l]=t}}else{var c=v(e)
c[n]=c[n]||e.textContent,c[o]=c[o]||s,c[r]=c[r]||e.disabled,c[l]=c[l]||t,c.$option=e,g[s]=c,h.push(c)}e.selected&&O.items.push(s)}},O.maxItems=e.hasAttribute("multiple")?null:1,y(e.children,(e=>{var t,i,s
"optgroup"===(u=e.tagName.toLowerCase())?((s=v(t=e))[a]=s[a]||t.getAttribute("label")||"",s[c]=s[c]||f++,s[r]=s[r]||t.disabled,O.optgroups.push(s),i=s[c],y(t.children,(e=>{m(e,i)}))):"option"===u&&m(e)}))):(()=>{const t=e.getAttribute(s)
if(t)O.options=JSON.parse(t),y(O.options,(e=>{O.items.push(e[o])}))
else{var r=e.value.trim()||""
if(!i.allowEmptyOption&&!r.length)return
const t=r.split(i.delimiter)
y(t,(e=>{const t={}
t[n]=e,t[o]=e,O.options.push(t)})),O.items=t}})(),Object.assign({},j,O,t)}var W=0
class J extends(function(e){return e.plugins={},class extends e{constructor(...e){super(...e),this.plugins={names:[],settings:{},requested:{},loaded:{}}}static define(t,i){e.plugins[t]={name:t,fn:i}}initializePlugins(e){var t,i
const s=this,n=[]
if(Array.isArray(e))e.forEach((e=>{"string"==typeof e?n.push(e):(s.plugins.settings[e.name]=e.options,n.push(e.name))}))
else if(e)for(t in e)e.hasOwnProperty(t)&&(s.plugins.settings[t]=e[t],n.push(t))
for(;i=n.shift();)s.require(i)}loadPlugin(t){var i=this,s=i.plugins,n=e.plugins[t]
if(!e.plugins.hasOwnProperty(t))throw new Error('Unable to find "'+t+'" plugin')
s.requested[t]=!0,s.loaded[t]=n.fn.apply(i,[i.plugins.settings[t]||{}]),s.names.push(t)}require(e){var t=this,i=t.plugins
if(!t.plugins.loaded.hasOwnProperty(e)){if(i.requested[e])throw new Error('Plugin has circular dependency ("'+e+'")')
t.loadPlugin(e)}return i.loaded[e]}}}(t)){constructor(e,t){var i
super(),this.order=0,this.isOpen=!1,this.isDisabled=!1,this.isInvalid=!1,this.isValid=!0,this.isLocked=!1,this.isFocused=!1,this.isInputHidden=!1,this.isSetup=!1,this.ignoreFocus=!1,this.hasOptions=!1,this.lastValue="",this.caretPos=0,this.loading=0,this.loadedSearches={},this.activeOption=null,this.activeItems=[],this.optgroups={},this.options={},this.userOptions={},this.items=[],W++
var s=w(e)
if(s.tomselect)throw new Error("Tom Select already initialized on this element")
s.tomselect=this,i=(window.getComputedStyle&&window.getComputedStyle(s,null)).getPropertyValue("direction")
const n=U(s,t)
this.settings=n,this.input=s,this.tabIndex=s.tabIndex||0,this.is_select_tag="select"===s.tagName.toLowerCase(),this.rtl=/rtl/i.test(i),this.inputId=M(s,"tomselect-"+W),this.isRequired=s.required,this.sifter=new b(this.options,{diacritics:n.diacritics}),n.mode=n.mode||(1===n.maxItems?"single":"multi"),"boolean"!=typeof n.hideSelected&&(n.hideSelected="multi"===n.mode),"boolean"!=typeof n.hidePlaceholder&&(n.hidePlaceholder="multi"!==n.mode)
var o=n.createFilter
"function"!=typeof o&&("string"==typeof o&&(o=new RegExp(o)),o instanceof RegExp?n.createFilter=e=>o.test(e):n.createFilter=()=>!0),this.initializePlugins(n.plugins),this.setupCallbacks(),this.setupTemplates()
const r=w("<div>"),l=w("<div>"),a=this._render("dropdown"),c=w('<div role="listbox" tabindex="-1">'),d=this.input.getAttribute("class")||"",p=n.mode
var u
if(C(r,n.wrapperClass,d,p),C(l,n.controlClass),G(r,l),C(a,n.dropdownClass,p),n.copyClassesToDropdown&&C(a,d),C(c,n.dropdownContentClass),G(a,c),w(n.dropdownParent||r).appendChild(a),n.hasOwnProperty("controlInput"))n.controlInput?(u=w(n.controlInput),this.focus_node=u):(u=w("<input/>"),this.focus_node=l)
else{u=w('<input type="text" autocomplete="off" size="1" />')
y(["autocorrect","autocapitalize","autocomplete"],(e=>{s.getAttribute(e)&&P(u,{[e]:s.getAttribute(e)})})),u.tabIndex=-1,l.appendChild(u),this.focus_node=u}this.wrapper=r,this.dropdown=a,this.dropdown_content=c,this.control=l,this.control_input=u,this.setup()}setup(){const e=this,t=e.settings,i=e.control_input,s=e.dropdown,n=e.dropdown_content,o=e.wrapper,r=e.control,l=e.input,a=e.focus_node,c={passive:!0},d=e.inputId+"-ts-dropdown"
P(n,{id:d}),P(a,{role:"combobox","aria-haspopup":"listbox","aria-expanded":"false","aria-controls":d})
const p=M(a,e.inputId+"-ts-control"),u="label[for='"+(e=>e.replace(/['"\\]/g,"\\$&"))(e.inputId)+"']",h=document.querySelector(u),g=e.focus.bind(e)
if(h){B(h,"click",g),P(h,{for:p})
const t=M(h,e.inputId+"-ts-label")
P(a,{"aria-labelledby":t}),P(n,{"aria-labelledby":t})}if(o.style.width=l.style.width,e.plugins.names.length){const t="plugin-"+e.plugins.names.join(" plugin-")
C([o,s],t)}(null===t.maxItems||t.maxItems>1)&&e.is_select_tag&&P(l,{multiple:"multiple"}),e.settings.placeholder&&P(i,{placeholder:t.placeholder}),!e.settings.splitOn&&e.settings.delimiter&&(e.settings.splitOn=new RegExp("\\s*"+v(e.settings.delimiter)+"+\\s*")),t.load&&t.loadThrottle&&(t.load=z(t.load,t.loadThrottle)),e.control_input.type=l.type,B(s,"click",(t=>{const i=k(t.target,"[data-selectable]")
i&&(e.onOptionSelect(t,i),H(t,!0))})),B(r,"click",(t=>{var s=k(t.target,"[data-ts-item]",r)
s&&e.onItemSelect(t,s)?H(t,!0):""==i.value&&(e.onClick(),H(t,!0))})),B(i,"mousedown",(e=>{""!==i.value&&e.stopPropagation()})),B(a,"keydown",(t=>e.onKeyDown(t))),B(i,"keypress",(t=>e.onKeyPress(t))),B(i,"input",(t=>e.onInput(t))),B(a,"resize",(()=>e.positionDropdown()),c),B(a,"blur",(t=>e.onBlur(t))),B(a,"focus",(t=>e.onFocus(t))),B(a,"paste",(t=>e.onPaste(t)))
const f=t=>{const i=t.composedPath()[0]
if(!o.contains(i)&&!s.contains(i))return e.isFocused&&e.blur(),void e.inputState()
H(t,!0)}
var m=()=>{e.isOpen&&e.positionDropdown()}
B(document,"mousedown",f),B(window,"scroll",m,c),B(window,"resize",m,c),this._destroy=()=>{document.removeEventListener("mousedown",f),window.removeEventListener("sroll",m),window.removeEventListener("resize",m),h&&h.removeEventListener("click",g)},this.revertSettings={innerHTML:l.innerHTML,tabIndex:l.tabIndex},l.tabIndex=-1,l.insertAdjacentElement("afterend",e.wrapper),e.sync(!1),t.items=[],delete t.optgroups,delete t.options,B(l,"invalid",(t=>{e.isValid&&(e.isValid=!1,e.isInvalid=!0,e.refreshState())})),e.updateOriginalInput(),e.refreshItems(),e.close(!1),e.inputState(),e.isSetup=!0,l.disabled?e.disable():e.enable(),e.on("change",this.onChange),C(l,"tomselected","ts-hidden-accessible"),e.trigger("initialize"),!0===t.preload&&e.preload()}setupOptions(e=[],t=[]){this.addOptions(e),y(t,(e=>{this.registerOptionGroup(e)}))}setupTemplates(){var e=this,t=e.settings.labelField,i=e.settings.optgroupLabelField,s={optgroup:e=>{let t=document.createElement("div")
return t.className="optgroup",t.appendChild(e.options),t},optgroup_header:(e,t)=>'<div class="optgroup-header">'+t(e[i])+"</div>",option:(e,i)=>"<div>"+i(e[t])+"</div>",item:(e,i)=>"<div>"+i(e[t])+"</div>",option_create:(e,t)=>'<div class="create">Add <strong>'+t(e.input)+"</strong>&hellip;</div>",no_results:()=>'<div class="no-results">No results found</div>',loading:()=>'<div class="spinner"></div>',not_loading:()=>{},dropdown:()=>"<div></div>"}
e.settings.render=Object.assign({},s,e.settings.render)}setupCallbacks(){var e,t,i={initialize:"onInitialize",change:"onChange",item_add:"onItemAdd",item_remove:"onItemRemove",item_select:"onItemSelect",clear:"onClear",option_add:"onOptionAdd",option_remove:"onOptionRemove",option_clear:"onOptionClear",optgroup_add:"onOptionGroupAdd",optgroup_remove:"onOptionGroupRemove",optgroup_clear:"onOptionGroupClear",dropdown_open:"onDropdownOpen",dropdown_close:"onDropdownClose",type:"onType",load:"onLoad",focus:"onFocus",blur:"onBlur"}
for(e in i)(t=this.settings[i[e]])&&this.on(e,t)}sync(e=!0){const t=this,i=e?U(t.input,{delimiter:t.settings.delimiter}):t.settings
t.setupOptions(i.options,i.optgroups),t.setValue(i.items,!0),t.lastQuery=null}onClick(){var e=this
if(e.activeItems.length>0)return e.clearActiveItems(),void e.focus()
e.isFocused&&e.isOpen?e.blur():e.focus()}onMouseDown(){}onChange(){_(this.input,"input"),_(this.input,"change")}onPaste(e){var t=this
t.isFull()||t.isInputHidden||t.isLocked?H(e):t.settings.splitOn&&setTimeout((()=>{var e=t.inputValue()
if(e.match(t.settings.splitOn)){var i=e.trim().split(t.settings.splitOn)
y(i,(e=>{t.createItem(e)}))}}),0)}onKeyPress(e){var t=this
if(!t.isLocked){var i=String.fromCharCode(e.keyCode||e.which)
return t.settings.create&&"multi"===t.settings.mode&&i===t.settings.delimiter?(t.createItem(),void H(e)):void 0}H(e)}onKeyDown(e){var t=this
if(t.isLocked)9!==e.keyCode&&H(e)
else{switch(e.keyCode){case 65:if(K(V,e))return H(e),void t.selectAll()
break
case 27:return t.isOpen&&(H(e,!0),t.close()),void t.clearActiveItems()
case 40:if(!t.isOpen&&t.hasOptions)t.open()
else if(t.activeOption){let e=t.getAdjacent(t.activeOption,1)
e&&t.setActiveOption(e)}return void H(e)
case 38:if(t.activeOption){let e=t.getAdjacent(t.activeOption,-1)
e&&t.setActiveOption(e)}return void H(e)
case 13:return void(t.isOpen&&t.activeOption?(t.onOptionSelect(e,t.activeOption),H(e)):t.settings.create&&t.createItem()&&H(e))
case 37:return void t.advanceSelection(-1,e)
case 39:return void t.advanceSelection(1,e)
case 9:return void(t.settings.selectOnTab&&(t.isOpen&&t.activeOption&&(t.onOptionSelect(e,t.activeOption),H(e)),t.settings.create&&t.createItem()&&H(e)))
case 8:case 46:return void t.deleteSelection(e)}t.isInputHidden&&!K(V,e)&&H(e)}}onInput(e){var t=this
if(!t.isLocked){var i=t.inputValue()
t.lastValue!==i&&(t.lastValue=i,t.settings.shouldLoad.call(t,i)&&t.load(i),t.refreshOptions(),t.trigger("type",i))}}onFocus(e){var t=this,i=t.isFocused
if(t.isDisabled)return t.blur(),void H(e)
t.ignoreFocus||(t.isFocused=!0,"focus"===t.settings.preload&&t.preload(),i||t.trigger("focus"),t.activeItems.length||(t.showInput(),t.refreshOptions(!!t.settings.openOnFocus)),t.refreshState())}onBlur(e){if(!1!==document.hasFocus()){var t=this
if(t.isFocused){t.isFocused=!1,t.ignoreFocus=!1
var i=()=>{t.close(),t.setActiveItem(),t.setCaret(t.items.length),t.trigger("blur")}
t.settings.create&&t.settings.createOnBlur?t.createItem(null,!1,i):i()}}}onOptionSelect(e,t){var i,s=this
t&&(t.parentElement&&t.parentElement.matches("[data-disabled]")||(t.classList.contains("create")?s.createItem(null,!0,(()=>{s.settings.closeAfterSelect&&s.close()})):void 0!==(i=t.dataset.value)&&(s.lastQuery=null,s.addItem(i),s.settings.closeAfterSelect&&s.close(),!s.settings.hideSelected&&e.type&&/click/.test(e.type)&&s.setActiveOption(t))))}onItemSelect(e,t){var i=this
return!i.isLocked&&"multi"===i.settings.mode&&(H(e),i.setActiveItem(t,e),!0)}canLoad(e){return!!this.settings.load&&!this.loadedSearches.hasOwnProperty(e)}load(e){const t=this
if(!t.canLoad(e))return
C(t.wrapper,t.settings.loadingClass),t.loading++
const i=t.loadCallback.bind(t)
t.settings.load.call(t,e,i)}loadCallback(e,t){const i=this
i.loading=Math.max(i.loading-1,0),i.lastQuery=null,i.clearActiveOption(),i.setupOptions(e,t),i.refreshOptions(i.isFocused&&!i.isInputHidden),i.loading||S(i.wrapper,i.settings.loadingClass),i.trigger("load",e,t)}preload(){var e=this.wrapper.classList
e.contains("preloaded")||(e.add("preloaded"),this.load(""))}setTextboxValue(e=""){var t=this.control_input
t.value!==e&&(t.value=e,_(t,"update"),this.lastValue=e)}getValue(){return this.is_select_tag&&this.input.hasAttribute("multiple")?this.items:this.items.join(this.settings.delimiter)}setValue(e,t){R(this,t?[]:["change"],(()=>{this.clear(t),this.addItems(e,t)}))}setMaxItems(e){0===e&&(e=null),this.settings.maxItems=e,this.refreshState()}setActiveItem(e,t){var i,s,n,o,r,l,a=this
if("single"!==a.settings.mode){if(!e)return a.clearActiveItems(),void(a.isFocused&&a.showInput())
if("click"===(i=t&&t.type.toLowerCase())&&K("shiftKey",t)&&a.activeItems.length){for(l=a.getLastActive(),(n=Array.prototype.indexOf.call(a.control.children,l))>(o=Array.prototype.indexOf.call(a.control.children,e))&&(r=n,n=o,o=r),s=n;s<=o;s++)e=a.control.children[s],-1===a.activeItems.indexOf(e)&&a.setActiveItemClass(e)
H(t)}else"click"===i&&K(V,t)||"keydown"===i&&K("shiftKey",t)?e.classList.contains("active")?a.removeActiveItem(e):a.setActiveItemClass(e):(a.clearActiveItems(),a.setActiveItemClass(e))
a.hideInput(),a.isFocused||a.focus()}}setActiveItemClass(e){const t=this,i=t.control.querySelector(".last-active")
i&&S(i,"last-active"),C(e,"active last-active"),t.trigger("item_select",e),-1==t.activeItems.indexOf(e)&&t.activeItems.push(e)}removeActiveItem(e){var t=this.activeItems.indexOf(e)
this.activeItems.splice(t,1),S(e,"active")}clearActiveItems(){S(this.activeItems,"active"),this.activeItems=[]}setActiveOption(e){e!==this.activeOption&&(this.clearActiveOption(),e&&(this.activeOption=e,P(this.focus_node,{"aria-activedescendant":e.getAttribute("id")}),P(e,{"aria-selected":"true"}),C(e,"active"),this.scrollToOption(e)))}scrollToOption(e,t){if(!e)return
const i=this.dropdown_content,s=i.clientHeight,n=i.scrollTop||0,o=e.offsetHeight,r=e.getBoundingClientRect().top-i.getBoundingClientRect().top+n
r+o>s+n?this.scroll(r-s+o,t):r<n&&this.scroll(r,t)}scroll(e,t){const i=this.dropdown_content
t&&(i.style.scrollBehavior=t),i.scrollTop=e,i.style.scrollBehavior=""}clearActiveOption(){this.activeOption&&(S(this.activeOption,"active"),P(this.activeOption,{"aria-selected":null})),this.activeOption=null,P(this.focus_node,{"aria-activedescendant":null})}selectAll(){if("single"===this.settings.mode)return
const e=this.controlChildren()
e.length&&(this.hideInput(),this.close(),this.activeItems=e,C(e,"active"))}inputState(){var e=this
e.control.contains(e.control_input)&&(P(e.control_input,{placeholder:e.settings.placeholder}),e.activeItems.length>0||!e.isFocused&&e.settings.hidePlaceholder&&e.items.length>0?(e.setTextboxValue(),e.isInputHidden=!0):(e.settings.hidePlaceholder&&e.items.length>0&&P(e.control_input,{placeholder:""}),e.isInputHidden=!1),e.wrapper.classList.toggle("input-hidden",e.isInputHidden))}hideInput(){this.inputState()}showInput(){this.inputState()}inputValue(){return this.control_input.value.trim()}focus(){var e=this
e.isDisabled||(e.ignoreFocus=!0,e.control_input.offsetWidth?e.control_input.focus():e.focus_node.focus(),setTimeout((()=>{e.ignoreFocus=!1,e.onFocus()}),0))}blur(){this.focus_node.blur(),this.onBlur()}getScoreFunction(e){return this.sifter.getScoreFunction(e,this.getSearchOptions())}getSearchOptions(){var e=this.settings,t=e.sortField
return"string"==typeof e.sortField&&(t=[{field:e.sortField}]),{fields:e.searchField,conjunction:e.searchConjunction,sort:t,nesting:e.nesting}}search(e){var t,i,s,n=this,o=this.getSearchOptions()
if(n.settings.score&&"function"!=typeof(s=n.settings.score.call(n,e)))throw new Error('Tom Select "score" setting must be a function that returns a function')
if(e!==n.lastQuery?(n.lastQuery=e,i=n.sifter.search(e,Object.assign(o,{score:s})),n.currentResults=i):i=Object.assign({},n.currentResults),n.settings.hideSelected)for(t=i.items.length-1;t>=0;t--){let e=q(i.items[t].id)
e&&-1!==n.items.indexOf(e)&&i.items.splice(t,1)}return i}refreshOptions(e=!0){var t,i,s,n,o,r,l,a,c,d,p
const u={},h=[]
var g,f=this,v=f.inputValue(),m=f.search(v),O=f.activeOption,b=f.settings.shouldOpen||!1,w=f.dropdown_content
for(O&&(c=O.dataset.value,d=O.closest("[data-group]")),n=m.items.length,"number"==typeof f.settings.maxOptions&&(n=Math.min(n,f.settings.maxOptions)),n>0&&(b=!0),t=0;t<n;t++){let e=m.items[t].id,n=f.options[e],l=f.getOption(e,!0)
for(f.settings.hideSelected||l.classList.toggle("selected",f.items.includes(e)),o=n[f.settings.optgroupField]||"",i=0,s=(r=Array.isArray(o)?o:[o])&&r.length;i<s;i++)o=r[i],f.optgroups.hasOwnProperty(o)||(o=""),u.hasOwnProperty(o)||(u[o]=document.createDocumentFragment(),h.push(o)),i>0&&(l=l.cloneNode(!0),P(l,{id:n.$id+"-clone-"+i,"aria-selected":null}),l.classList.add("ts-cloned"),S(l,"active")),c==e&&d&&d.dataset.group===o&&(O=l),u[o].appendChild(l)}this.settings.lockOptgroupOrder&&h.sort(((e,t)=>(f.optgroups[e]&&f.optgroups[e].$order||0)-(f.optgroups[t]&&f.optgroups[t].$order||0))),l=document.createDocumentFragment(),y(h,(e=>{if(f.optgroups.hasOwnProperty(e)&&u[e].children.length){let t=document.createDocumentFragment(),i=f.render("optgroup_header",f.optgroups[e])
G(t,i),G(t,u[e])
let s=f.render("optgroup",{group:f.optgroups[e],options:t})
G(l,s)}else G(l,u[e])})),w.innerHTML="",G(w,l),f.settings.highlight&&(g=w.querySelectorAll("span.highlight"),Array.prototype.forEach.call(g,(function(e){var t=e.parentNode
t.replaceChild(e.firstChild,e),t.normalize()})),m.query.length&&m.tokens.length&&y(m.tokens,(e=>{T(w,e.regex)})))
var _=e=>{let t=f.render(e,{input:v})
return t&&(b=!0,w.insertBefore(t,w.firstChild)),t}
if(f.loading?_("loading"):f.settings.shouldLoad.call(f,v)?0===m.items.length&&_("no_results"):_("not_loading"),(a=f.canCreate(v))&&(p=_("option_create")),f.hasOptions=m.items.length>0||a,b){if(m.items.length>0){if(!w.contains(O)&&"single"===f.settings.mode&&f.items.length&&(O=f.getOption(f.items[0])),!w.contains(O)){let e=0
p&&!f.settings.addPrecedence&&(e=1),O=f.selectable()[e]}}else p&&(O=p)
e&&!f.isOpen&&(f.open(),f.scrollToOption(O,"auto")),f.setActiveOption(O)}else f.clearActiveOption(),e&&f.isOpen&&f.close(!1)}selectable(){return this.dropdown_content.querySelectorAll("[data-selectable]")}addOption(e,t=!1){const i=this
if(Array.isArray(e))return i.addOptions(e,t),!1
const s=q(e[i.settings.valueField])
return null!==s&&!i.options.hasOwnProperty(s)&&(e.$order=e.$order||++i.order,e.$id=i.inputId+"-opt-"+e.$order,i.options[s]=e,i.lastQuery=null,t&&(i.userOptions[s]=t,i.trigger("option_add",s,e)),s)}addOptions(e,t=!1){y(e,(e=>{this.addOption(e,t)}))}registerOption(e){return this.addOption(e)}registerOptionGroup(e){var t=q(e[this.settings.optgroupValueField])
return null!==t&&(e.$order=e.$order||++this.order,this.optgroups[t]=e,t)}addOptionGroup(e,t){var i
t[this.settings.optgroupValueField]=e,(i=this.registerOptionGroup(t))&&this.trigger("optgroup_add",i,t)}removeOptionGroup(e){this.optgroups.hasOwnProperty(e)&&(delete this.optgroups[e],this.clearCache(),this.trigger("optgroup_remove",e))}clearOptionGroups(){this.optgroups={},this.clearCache(),this.trigger("optgroup_clear")}updateOption(e,t){const i=this
var s,n
const o=q(e),r=q(t[i.settings.valueField])
if(null===o)return
if(!i.options.hasOwnProperty(o))return
if("string"!=typeof r)throw new Error("Value must be set in option data")
const l=i.getOption(o),a=i.getItem(o)
if(t.$order=t.$order||i.options[o].$order,delete i.options[o],i.uncacheValue(r),i.options[r]=t,l){if(i.dropdown_content.contains(l)){const e=i._render("option",t)
E(l,e),i.activeOption===l&&i.setActiveOption(e)}l.remove()}a&&(-1!==(n=i.items.indexOf(o))&&i.items.splice(n,1,r),s=i._render("item",t),a.classList.contains("active")&&C(s,"active"),E(a,s)),i.lastQuery=null}removeOption(e,t){const i=this
e=D(e),i.uncacheValue(e),delete i.userOptions[e],delete i.options[e],i.lastQuery=null,i.trigger("option_remove",e),i.removeItem(e,t)}clearOptions(){this.loadedSearches={},this.userOptions={},this.clearCache()
var e={}
y(this.options,((t,i)=>{this.items.indexOf(i)>=0&&(e[i]=this.options[i])})),this.options=this.sifter.items=e,this.lastQuery=null,this.trigger("option_clear")}getOption(e,t=!1){const i=q(e)
if(null!==i&&this.options.hasOwnProperty(i)){const e=this.options[i]
if(e.$div)return e.$div
if(t)return this._render("option",e)}return null}getAdjacent(e,t,i="option"){var s
if(!e)return null
s="item"==i?this.controlChildren():this.dropdown_content.querySelectorAll("[data-selectable]")
for(let i=0;i<s.length;i++)if(s[i]==e)return t>0?s[i+1]:s[i-1]
return null}getItem(e){if("object"==typeof e)return e
var t=q(e)
return null!==t?this.control.querySelector(`[data-value="${Q(t)}"]`):null}addItems(e,t){var i=this,s=Array.isArray(e)?e:[e]
for(let e=0,n=(s=s.filter((e=>-1===i.items.indexOf(e)))).length;e<n;e++)i.isPending=e<n-1,i.addItem(s[e],t)}addItem(e,t){R(this,t?[]:["change"],(()=>{var i,s
const n=this,o=n.settings.mode,r=q(e)
if((!r||-1===n.items.indexOf(r)||("single"===o&&n.close(),"single"!==o&&n.settings.duplicates))&&null!==r&&n.options.hasOwnProperty(r)&&("single"===o&&n.clear(t),"multi"!==o||!n.isFull())){if(i=n._render("item",n.options[r]),n.control.contains(i)&&(i=i.cloneNode(!0)),s=n.isFull(),n.items.splice(n.caretPos,0,r),n.insertAtCaret(i),n.isSetup){if(!n.isPending&&n.settings.hideSelected){let e=n.getOption(r),t=n.getAdjacent(e,1)
t&&n.setActiveOption(t)}n.isPending||n.refreshOptions(n.isFocused&&"single"!==o),0!=n.settings.closeAfterSelect&&n.isFull()?n.close():n.isPending||n.positionDropdown(),n.trigger("item_add",r,i),n.isPending||n.updateOriginalInput({silent:t})}(!n.isPending||!s&&n.isFull())&&(n.inputState(),n.refreshState())}}))}removeItem(e=null,t){const i=this
if(!(e=i.getItem(e)))return
var s,n
const o=e.dataset.value
s=L(e),e.remove(),e.classList.contains("active")&&(n=i.activeItems.indexOf(e),i.activeItems.splice(n,1),S(e,"active")),i.items.splice(s,1),i.lastQuery=null,!i.settings.persist&&i.userOptions.hasOwnProperty(o)&&i.removeOption(o,t),s<i.caretPos&&i.setCaret(i.caretPos-1),i.updateOriginalInput({silent:t}),i.refreshState(),i.positionDropdown(),i.trigger("item_remove",o,e)}createItem(e=null,t=!0,i=(()=>{})){var s,n=this,o=n.caretPos
if(e=e||n.inputValue(),!n.canCreate(e))return i(),!1
n.lock()
var r=!1,l=e=>{if(n.unlock(),!e||"object"!=typeof e)return i()
var s=q(e[n.settings.valueField])
if("string"!=typeof s)return i()
n.setTextboxValue(),n.addOption(e,!0),n.setCaret(o),n.addItem(s),n.refreshOptions(t&&"single"!==n.settings.mode),i(e),r=!0}
return s="function"==typeof n.settings.create?n.settings.create.call(this,e,l):{[n.settings.labelField]:e,[n.settings.valueField]:e},r||l(s),!0}refreshItems(){var e=this
e.lastQuery=null,e.isSetup&&e.addItems(e.items),e.updateOriginalInput(),e.refreshState()}refreshState(){const e=this
e.refreshValidityState()
const t=e.isFull(),i=e.isLocked
e.wrapper.classList.toggle("rtl",e.rtl)
const s=e.wrapper.classList
var n
s.toggle("focus",e.isFocused),s.toggle("disabled",e.isDisabled),s.toggle("required",e.isRequired),s.toggle("invalid",!e.isValid),s.toggle("locked",i),s.toggle("full",t),s.toggle("input-active",e.isFocused&&!e.isInputHidden),s.toggle("dropdown-active",e.isOpen),s.toggle("has-options",(n=e.options,0===Object.keys(n).length)),s.toggle("has-items",e.items.length>0)}refreshValidityState(){var e=this
e.input.checkValidity&&(e.isValid=e.input.checkValidity(),e.isInvalid=!e.isValid)}isFull(){return null!==this.settings.maxItems&&this.items.length>=this.settings.maxItems}updateOriginalInput(e={}){const t=this
var i,s
const n=t.input.querySelector('option[value=""]')
if(t.is_select_tag){const e=[]
function o(i,s,o){return i||(i=w('<option value="'+N(s)+'">'+N(o)+"</option>")),i!=n&&t.input.append(i),e.push(i),i.selected=!0,i}t.input.querySelectorAll("option:checked").forEach((e=>{e.selected=!1})),0==t.items.length&&"single"==t.settings.mode?o(n,"",""):t.items.forEach((n=>{if(i=t.options[n],s=i[t.settings.labelField]||"",e.includes(i.$option)){o(t.input.querySelector(`option[value="${Q(n)}"]:not(:checked)`),n,s)}else i.$option=o(i.$option,n,s)}))}else t.input.value=t.getValue()
t.isSetup&&(e.silent||t.trigger("change",t.getValue()))}open(){var e=this
e.isLocked||e.isOpen||"multi"===e.settings.mode&&e.isFull()||(e.isOpen=!0,P(e.focus_node,{"aria-expanded":"true"}),e.refreshState(),I(e.dropdown,{visibility:"hidden",display:"block"}),e.positionDropdown(),I(e.dropdown,{visibility:"visible",display:"block"}),e.focus(),e.trigger("dropdown_open",e.dropdown))}close(e=!0){var t=this,i=t.isOpen
e&&(t.setTextboxValue(),"single"===t.settings.mode&&t.items.length&&t.hideInput()),t.isOpen=!1,P(t.focus_node,{"aria-expanded":"false"}),I(t.dropdown,{display:"none"}),t.settings.hideSelected&&t.clearActiveOption(),t.refreshState(),i&&t.trigger("dropdown_close",t.dropdown)}positionDropdown(){if("body"===this.settings.dropdownParent){var e=this.control,t=e.getBoundingClientRect(),i=e.offsetHeight+t.top+window.scrollY,s=t.left+window.scrollX
I(this.dropdown,{width:t.width+"px",top:i+"px",left:s+"px"})}}clear(e){var t=this
if(t.items.length){var i=t.controlChildren()
y(i,(e=>{t.removeItem(e,!0)})),t.showInput(),e||t.updateOriginalInput(),t.trigger("clear")}}insertAtCaret(e){const t=this,i=t.caretPos,s=t.control
s.insertBefore(e,s.children[i]),t.setCaret(i+1)}deleteSelection(e){var t,i,s,n,o,r=this
t=e&&8===e.keyCode?-1:1,i={start:(o=r.control_input).selectionStart||0,length:(o.selectionEnd||0)-(o.selectionStart||0)}
const l=[]
if(r.activeItems.length)n=F(r.activeItems,t),s=L(n),t>0&&s++,y(r.activeItems,(e=>l.push(e)))
else if((r.isFocused||"single"===r.settings.mode)&&r.items.length){const e=r.controlChildren()
t<0&&0===i.start&&0===i.length?l.push(e[r.caretPos-1]):t>0&&i.start===r.inputValue().length&&l.push(e[r.caretPos])}const a=l.map((e=>e.dataset.value))
if(!a.length||"function"==typeof r.settings.onDelete&&!1===r.settings.onDelete.call(r,a,e))return!1
for(H(e,!0),void 0!==s&&r.setCaret(s);l.length;)r.removeItem(l.pop())
return r.showInput(),r.positionDropdown(),r.refreshOptions(!1),!0}advanceSelection(e,t){var i,s,n=this
n.rtl&&(e*=-1),n.inputValue().length||(K(V,t)||K("shiftKey",t)?(s=(i=n.getLastActive(e))?i.classList.contains("active")?n.getAdjacent(i,e,"item"):i:e>0?n.control_input.nextElementSibling:n.control_input.previousElementSibling)&&(s.classList.contains("active")&&n.removeActiveItem(i),n.setActiveItemClass(s)):n.moveCaret(e))}moveCaret(e){}getLastActive(e){let t=this.control.querySelector(".last-active")
if(t)return t
var i=this.control.querySelectorAll(".active")
return i?F(i,e):void 0}setCaret(e){this.caretPos=this.items.length}controlChildren(){return Array.from(this.control.querySelectorAll("[data-ts-item]"))}lock(){this.close(),this.isLocked=!0,this.refreshState()}unlock(){this.isLocked=!1,this.refreshState()}disable(){var e=this
e.input.disabled=!0,e.control_input.disabled=!0,e.focus_node.tabIndex=-1,e.isDisabled=!0,e.lock()}enable(){var e=this
e.input.disabled=!1,e.control_input.disabled=!1,e.focus_node.tabIndex=e.tabIndex,e.isDisabled=!1,e.unlock()}destroy(){var e=this,t=e.revertSettings
e.trigger("destroy"),e.off(),e.wrapper.remove(),e.dropdown.remove(),e.input.innerHTML=t.innerHTML,e.input.tabIndex=t.tabIndex,S(e.input,"tomselected","ts-hidden-accessible"),e._destroy(),delete e.input.tomselect}render(e,t){return"function"!=typeof this.settings.render[e]?null:this._render(e,t)}_render(e,t){var i,s,n=""
const o=this
return"option"!==e&&"item"!=e||(n=D(t[o.settings.valueField])),null==(s=o.settings.render[e].call(this,t,N))||(s=w(s),"option"===e||"option_create"===e?t[o.settings.disabledField]?P(s,{"aria-disabled":"true"}):P(s,{"data-selectable":""}):"optgroup"===e&&(i=t.group[o.settings.optgroupValueField],P(s,{"data-group":i}),t.group[o.settings.disabledField]&&P(s,{"data-disabled":""})),"option"!==e&&"item"!==e||(P(s,{"data-value":n}),"item"===e?(C(s,o.settings.itemClass),P(s,{"data-ts-item":""})):(C(s,o.settings.optionClass),P(s,{role:"option",id:t.$id}),o.options[n].$div=s))),s}clearCache(){y(this.options,((e,t)=>{e.$div&&(e.$div.remove(),delete e.$div)}))}uncacheValue(e){const t=this.getOption(e)
t&&t.remove()}canCreate(e){return this.settings.create&&e.length>0&&this.settings.createFilter.call(this,e)}hook(e,t,i){var s=this,n=s[t]
s[t]=function(){var t,o
return"after"===e&&(t=n.apply(s,arguments)),o=i.apply(s,arguments),"instead"===e?o:("before"===e&&(t=n.apply(s,arguments)),t)}}}return J.define("change_listener",(function(){B(this.input,"change",(()=>{this.sync()}))})),J.define("checkbox_options",(function(){var e=this,t=e.onOptionSelect
e.settings.hideSelected=!1
var i=function(e){setTimeout((()=>{var t=e.querySelector("input")
e.classList.contains("selected")?t.checked=!0:t.checked=!1}),1)}
e.hook("after","setupTemplates",(()=>{var t=e.settings.render.option
e.settings.render.option=(i,s)=>{var n=w(t.call(e,i,s)),o=document.createElement("input")
o.addEventListener("click",(function(e){H(e)})),o.type="checkbox"
const r=q(i[e.settings.valueField])
return r&&e.items.indexOf(r)>-1&&(o.checked=!0),n.prepend(o),n}})),e.on("item_remove",(t=>{var s=e.getOption(t)
s&&(s.classList.remove("selected"),i(s))})),e.hook("instead","onOptionSelect",((s,n)=>{if(n.classList.contains("selected"))return n.classList.remove("selected"),e.removeItem(n.dataset.value),e.refreshOptions(),void H(s,!0)
t.call(e,s,n),i(n)}))})),J.define("clear_button",(function(e){const t=this,i=Object.assign({className:"clear-button",title:"Clear All",html:e=>`<div class="${e.className}" title="${e.title}">&times;</div>`},e)
t.on("initialize",(()=>{var e=w(i.html(i))
e.addEventListener("click",(e=>{t.clear(),"single"===t.settings.mode&&t.settings.allowEmptyOption&&t.addItem(""),e.preventDefault(),e.stopPropagation()})),t.control.appendChild(e)}))})),J.define("drag_drop",(function(){var e=this
if(!$.fn.sortable)throw new Error('The "drag_drop" plugin requires jQuery UI "sortable".')
if("multi"===e.settings.mode){var t=e.lock,i=e.unlock
e.hook("instead","lock",(()=>{var i=$(e.control).data("sortable")
return i&&i.disable(),t.call(e)})),e.hook("instead","unlock",(()=>{var t=$(e.control).data("sortable")
return t&&t.enable(),i.call(e)})),e.on("initialize",(()=>{var t=$(e.control).sortable({items:"[data-value]",forcePlaceholderSize:!0,disabled:e.isLocked,start:(e,i)=>{i.placeholder.css("width",i.helper.css("width")),t.css({overflow:"visible"})},stop:()=>{t.css({overflow:"hidden"})
var i=[]
t.children("[data-value]").each((function(){this.dataset.value&&i.push(this.dataset.value)})),e.setValue(i)}})}))}})),J.define("dropdown_header",(function(e){const t=this,i=Object.assign({title:"Untitled",headerClass:"dropdown-header",titleRowClass:"dropdown-header-title",labelClass:"dropdown-header-label",closeClass:"dropdown-header-close",html:e=>'<div class="'+e.headerClass+'"><div class="'+e.titleRowClass+'"><span class="'+e.labelClass+'">'+e.title+'</span><a class="'+e.closeClass+'">&times;</a></div></div>'},e)
t.on("initialize",(()=>{var e=w(i.html(i)),s=e.querySelector("."+i.closeClass)
s&&s.addEventListener("click",(e=>{H(e,!0),t.close()})),t.dropdown.insertBefore(e,t.dropdown.firstChild)}))})),J.define("caret_position",(function(){var e=this
e.hook("instead","setCaret",(t=>{"single"!==e.settings.mode&&e.control.contains(e.control_input)?(t=Math.max(0,Math.min(e.items.length,t)))==e.caretPos||e.isPending||e.controlChildren().forEach(((i,s)=>{s<t?e.control_input.insertAdjacentElement("beforebegin",i):e.control.appendChild(i)})):t=e.items.length,e.caretPos=t})),e.hook("instead","moveCaret",(t=>{if(!e.isFocused)return
const i=e.getLastActive(t)
if(i){const s=L(i)
e.setCaret(t>0?s+1:s),e.setActiveItem()}else e.setCaret(e.caretPos+t)}))})),J.define("dropdown_input",(function(){var e=this
e.settings.shouldOpen=!0,e.hook("before","setup",(()=>{e.focus_node=e.control,C(e.control_input,"dropdown-input")
const t=w('<div class="dropdown-input-wrap">')
t.append(e.control_input),e.dropdown.insertBefore(t,e.dropdown.firstChild)})),e.on("initialize",(()=>{e.control_input.addEventListener("keydown",(t=>{switch(t.keyCode){case 27:return e.isOpen&&(H(t,!0),e.close()),void e.clearActiveItems()
case 9:e.focus_node.tabIndex=-1}return e.onKeyDown.call(e,t)})),e.on("blur",(()=>{e.focus_node.tabIndex=e.isDisabled?-1:e.tabIndex})),e.on("dropdown_open",(()=>{e.control_input.focus()}))
const t=e.onBlur
e.hook("instead","onBlur",(i=>{if(!i||i.relatedTarget!=e.control_input)return t.call(e)})),B(e.control_input,"blur",(()=>e.onBlur())),e.hook("before","close",(()=>{e.isOpen&&e.focus_node.focus()}))}))})),J.define("input_autogrow",(function(){var e=this
e.on("initialize",(()=>{var t=document.createElement("span"),i=e.control_input
t.style.cssText="position:absolute; top:-99999px; left:-99999px; width:auto; padding:0; white-space:pre; ",e.wrapper.appendChild(t)
for(const e of["letterSpacing","fontSize","fontFamily","fontWeight","textTransform"])t.style[e]=i.style[e]
var s=()=>{e.items.length>0?(t.textContent=i.value,i.style.width=t.clientWidth+"px"):i.style.width=""}
s(),e.on("update item_add item_remove",s),B(i,"input",s),B(i,"keyup",s),B(i,"blur",s),B(i,"update",s)}))})),J.define("no_backspace_delete",(function(){var e=this,t=e.deleteSelection
this.hook("instead","deleteSelection",(i=>!!e.activeItems.length&&t.call(e,i)))})),J.define("no_active_items",(function(){this.hook("instead","setActiveItem",(()=>{})),this.hook("instead","selectAll",(()=>{}))})),J.define("optgroup_columns",(function(){var e=this,t=e.onKeyDown
e.hook("instead","onKeyDown",(i=>{var s,n,o,r
if(!e.isOpen||37!==i.keyCode&&39!==i.keyCode)return t.call(e,i)
r=k(e.activeOption,"[data-group]"),s=L(e.activeOption,"[data-selectable]"),r&&(r=37===i.keyCode?r.previousSibling:r.nextSibling)&&(n=(o=r.querySelectorAll("[data-selectable]"))[Math.min(o.length-1,s)])&&e.setActiveOption(n)}))})),J.define("remove_button",(function(e){const t=Object.assign({label:"&times;",title:"Remove",className:"remove",append:!0},e)
var i=this
if(t.append){var s='<a href="javascript:void(0)" class="'+t.className+'" tabindex="-1" title="'+N(t.title)+'">'+t.label+"</a>"
i.hook("after","setupTemplates",(()=>{var e=i.settings.render.item
i.settings.render.item=(t,n)=>{var o=w(e.call(i,t,n)),r=w(s)
return o.appendChild(r),B(r,"mousedown",(e=>{H(e,!0)})),B(r,"click",(e=>{if(H(e,!0),!i.isLocked){var t=o.dataset.value
i.removeItem(t),i.refreshOptions(!1)}})),o}}))}})),J.define("restore_on_backspace",(function(e){const t=this,i=Object.assign({text:e=>e[t.settings.labelField]},e)
t.on("item_remove",(function(e){if(""===t.control_input.value.trim()){var s=t.options[e]
s&&t.setTextboxValue(i.text.call(t,s))}}))})),J.define("virtual_scroll",(function(){const e=this,t=e.canLoad,i=e.clearActiveOption,s=e.loadCallback
var n,o={},r=!1
if(!e.settings.firstUrl)throw"virtual_scroll plugin requires a firstUrl() method"
function l(t){return!("number"==typeof e.settings.maxOptions&&n.children.length>=e.settings.maxOptions)&&!(!(t in o)||!o[t])}e.settings.sortField=[{field:"$order"},{field:"$score"}],e.setNextUrl=function(e,t){o[e]=t},e.getUrl=function(t){if(t in o){const e=o[t]
return o[t]=!1,e}return o={},e.settings.firstUrl(t)},e.hook("instead","clearActiveOption",(()=>{if(!r)return i.call(e)})),e.hook("instead","canLoad",(i=>i in o?l(i):t.call(e,i))),e.hook("instead","loadCallback",((t,i)=>{r||e.clearOptions(),s.call(e,t,i),r=!1})),e.hook("after","refreshOptions",(()=>{const t=e.lastValue
var i
l(t)?(i=e.render("loading_more",{query:t}))&&i.setAttribute("data-selectable",""):t in o&&!n.querySelector(".no-results")&&(i=e.render("no_more_results",{query:t})),i&&(C(i,e.settings.optionClass),n.append(i))})),e.on("initialize",(()=>{n=e.dropdown_content,e.settings.render=Object.assign({},{loading_more:function(){return'<div class="loading-more-results">Loading more results ... </div>'},no_more_results:function(){return'<div class="no-more-results">No more results</div>'}},e.settings.render),n.addEventListener("scroll",(function(){n.clientHeight/(n.scrollHeight-n.scrollTop)<.95||l(e.lastValue)&&(r||(r=!0,e.load.call(e,e.lastValue)))}))}))})),J}))
var tomSelect=function(e,t){return new TomSelect(e,t)}
//# sourceMappingURL=tom-select.complete.min.js.map
@@ -0,0 +1,334 @@
/**
* tom-select.css (v2.0.0-rc.4)
* Copyright (c) contributors
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this
* file except in compliance with the License. You may obtain a copy of the License at:
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under
* the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
* ANY KIND, either express or implied. See the License for the specific language
* governing permissions and limitations under the License.
*
*/
.ts-wrapper.plugin-drag_drop.multi > .ts-control > div.ui-sortable-placeholder {
visibility: visible !important;
background: #f2f2f2 !important;
background: rgba(0, 0, 0, 0.06) !important;
border: 0 none !important;
box-shadow: inset 0 0 12px 4px #fff; }
.ts-wrapper.plugin-drag_drop .ui-sortable-placeholder::after {
content: '!';
visibility: hidden; }
.ts-wrapper.plugin-drag_drop .ui-sortable-helper {
box-shadow: 0 2px 5px rgba(0, 0, 0, 0.2); }
.plugin-checkbox_options .option input {
margin-right: 0.5rem; }
.plugin-clear_button .ts-control {
padding-right: calc( 1em + (3 * 6px)) !important; }
.plugin-clear_button .clear-button {
opacity: 0;
position: absolute;
top: 8px;
right: calc(8px - 6px);
margin-right: 0 !important;
background: transparent !important;
transition: opacity 0.5s;
cursor: pointer; }
.plugin-clear_button.single .clear-button {
right: calc(8px - 6px + 2rem); }
.plugin-clear_button.focus.has-items .clear-button,
.plugin-clear_button:hover.has-items .clear-button {
opacity: 1; }
.ts-wrapper .dropdown-header {
position: relative;
padding: 10px 8px;
border-bottom: 1px solid #d0d0d0;
background: #f8f8f8;
border-radius: 3px 3px 0 0; }
.ts-wrapper .dropdown-header-close {
position: absolute;
right: 8px;
top: 50%;
color: #303030;
opacity: 0.4;
margin-top: -12px;
line-height: 20px;
font-size: 20px !important; }
.ts-wrapper .dropdown-header-close:hover {
color: black; }
.plugin-dropdown_input.focus.dropdown-active .ts-control {
box-shadow: none;
border: 1px solid #d0d0d0; }
.plugin-dropdown_input .dropdown-input {
border: 1px solid #d0d0d0;
border-width: 0 0 1px 0;
display: block;
padding: 8px 8px;
box-shadow: none;
width: 100%;
background: transparent; }
.ts-wrapper.plugin-input_autogrow.has-items .ts-control > input {
min-width: 0; }
.ts-wrapper.plugin-input_autogrow.has-items.focus .ts-control > input {
flex: none;
min-width: 4px; }
.ts-wrapper.plugin-input_autogrow.has-items.focus .ts-control > input::-webkit-input-placeholder {
color: transparent; }
.ts-wrapper.plugin-input_autogrow.has-items.focus .ts-control > input::-ms-input-placeholder {
color: transparent; }
.ts-wrapper.plugin-input_autogrow.has-items.focus .ts-control > input::placeholder {
color: transparent; }
.ts-dropdown.plugin-optgroup_columns .ts-dropdown-content {
display: flex; }
.ts-dropdown.plugin-optgroup_columns .optgroup {
border-right: 1px solid #f2f2f2;
border-top: 0 none;
flex-grow: 1;
flex-basis: 0;
min-width: 0; }
.ts-dropdown.plugin-optgroup_columns .optgroup:last-child {
border-right: 0 none; }
.ts-dropdown.plugin-optgroup_columns .optgroup:before {
display: none; }
.ts-dropdown.plugin-optgroup_columns .optgroup-header {
border-top: 0 none; }
.ts-wrapper.plugin-remove_button .item {
display: inline-flex;
align-items: center;
padding-right: 0 !important; }
.ts-wrapper.plugin-remove_button .item .remove {
color: inherit;
text-decoration: none;
vertical-align: middle;
display: inline-block;
padding: 2px 6px;
border-left: 1px solid #d0d0d0;
border-radius: 0 2px 2px 0;
box-sizing: border-box;
margin-left: 6px; }
.ts-wrapper.plugin-remove_button .item .remove:hover {
background: rgba(0, 0, 0, 0.05); }
.ts-wrapper.plugin-remove_button .item.active .remove {
border-left-color: #cacaca; }
.ts-wrapper.plugin-remove_button.disabled .item .remove:hover {
background: none; }
.ts-wrapper.plugin-remove_button.disabled .item .remove {
border-left-color: white; }
.ts-wrapper.plugin-remove_button .remove-single {
position: absolute;
right: 0;
top: 0;
font-size: 23px; }
.ts-wrapper {
position: relative; }
.ts-dropdown,
.ts-control,
.ts-control input {
color: #303030;
font-family: inherit;
font-size: 13px;
line-height: 18px;
font-smoothing: inherit; }
.ts-control,
.ts-wrapper.single.input-active .ts-control {
background: #fff;
cursor: text; }
.ts-control {
border: 1px solid #d0d0d0;
padding: 8px 8px;
width: 100%;
overflow: hidden;
position: relative;
z-index: 1;
box-sizing: border-box;
box-shadow: none;
border-radius: 3px;
display: flex;
flex-wrap: wrap; }
.ts-wrapper.multi.has-items .ts-control {
padding: calc( 8px - 2px - 0) 8px calc( 8px - 2px - 3px - 0); }
.full .ts-control {
background-color: #fff; }
.disabled .ts-control,
.disabled .ts-control * {
cursor: default !important; }
.focus .ts-control {
box-shadow: none; }
.ts-control > * {
vertical-align: baseline;
display: inline-block; }
.ts-wrapper.multi .ts-control > div {
cursor: pointer;
margin: 0 3px 3px 0;
padding: 2px 6px;
background: #f2f2f2;
color: #303030;
border: 0 solid #d0d0d0; }
.ts-wrapper.multi .ts-control > div.active {
background: #e8e8e8;
color: #303030;
border: 0 solid #cacaca; }
.ts-wrapper.multi.disabled .ts-control > div, .ts-wrapper.multi.disabled .ts-control > div.active {
color: #7d7c7c;
background: white;
border: 0 solid white; }
.ts-control > input {
flex: 1 1 auto;
min-width: 7rem;
display: inline-block !important;
padding: 0 !important;
min-height: 0 !important;
max-height: none !important;
max-width: 100% !important;
margin: 0 !important;
text-indent: 0 !important;
border: 0 none !important;
background: none !important;
line-height: inherit !important;
-webkit-user-select: auto !important;
-moz-user-select: auto !important;
-ms-user-select: auto !important;
user-select: auto !important;
box-shadow: none !important; }
.ts-control > input::-ms-clear {
display: none; }
.ts-control > input:focus {
outline: none !important; }
.has-items .ts-control > input {
margin: 0 4px !important; }
.ts-control.rtl {
text-align: right; }
.ts-control.rtl.single .ts-control:after {
left: 15px;
right: auto; }
.ts-control.rtl .ts-control > input {
margin: 0 4px 0 -2px !important; }
.disabled .ts-control {
opacity: 0.5;
background-color: #fafafa; }
.input-hidden .ts-control > input {
opacity: 0;
position: absolute;
left: -10000px; }
.ts-dropdown {
position: absolute;
top: 100%;
left: 0;
width: 100%;
z-index: 10;
border: 1px solid #d0d0d0;
background: #fff;
margin: 0.25rem 0 0 0;
border-top: 0 none;
box-sizing: border-box;
box-shadow: 0 1px 3px rgba(0, 0, 0, 0.1);
border-radius: 0 0 3px 3px; }
.ts-dropdown [data-selectable] {
cursor: pointer;
overflow: hidden; }
.ts-dropdown [data-selectable] .highlight {
background: rgba(125, 168, 208, 0.2);
border-radius: 1px; }
.ts-dropdown .option,
.ts-dropdown .optgroup-header,
.ts-dropdown .no-results,
.ts-dropdown .create {
padding: 5px 8px; }
.ts-dropdown .option, .ts-dropdown [data-disabled], .ts-dropdown [data-disabled] [data-selectable].option {
cursor: inherit;
opacity: 0.5; }
.ts-dropdown [data-selectable].option {
opacity: 1;
cursor: pointer; }
.ts-dropdown .optgroup:first-child .optgroup-header {
border-top: 0 none; }
.ts-dropdown .optgroup-header {
color: #303030;
background: #fff;
cursor: default; }
.ts-dropdown .create:hover,
.ts-dropdown .option:hover,
.ts-dropdown .active {
background-color: #f5fafd;
color: #495c68; }
.ts-dropdown .create:hover.create,
.ts-dropdown .option:hover.create,
.ts-dropdown .active.create {
color: #495c68; }
.ts-dropdown .create {
color: rgba(48, 48, 48, 0.5); }
.ts-dropdown .spinner {
display: inline-block;
width: 30px;
height: 30px;
margin: 5px 8px; }
.ts-dropdown .spinner:after {
content: " ";
display: block;
width: 24px;
height: 24px;
margin: 3px;
border-radius: 50%;
border: 5px solid #d0d0d0;
border-color: #d0d0d0 transparent #d0d0d0 transparent;
animation: lds-dual-ring 1.2s linear infinite; }
@keyframes lds-dual-ring {
0% {
transform: rotate(0deg); }
100% {
transform: rotate(360deg); } }
.ts-dropdown-content {
overflow-y: auto;
overflow-x: hidden;
max-height: 200px;
overflow-scrolling: touch;
scroll-behavior: smooth; }
.ts-hidden-accessible {
border: 0 !important;
clip: rect(0 0 0 0) !important;
-webkit-clip-path: inset(50%) !important;
clip-path: inset(50%) !important;
height: 1px !important;
overflow: hidden !important;
padding: 0 !important;
position: absolute !important;
width: 1px !important;
white-space: nowrap !important; }
/*# sourceMappingURL=tom-select.css.map */
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long