feat(memory): Add opt-in temporal decay for hybrid search scoring

Exponential decay (half-life configurable, default 30 days) applied
before MMR re-ranking. Dated daily files (memory/YYYY-MM-DD.md) use
filename date; evergreen files (MEMORY.md, topic files) are not
decayed; other sources fall back to file mtime.

Config: memorySearch.query.hybrid.temporalDecay.{enabled, halfLifeDays}
Default: disabled (backwards compatible, opt-in).
This commit is contained in:
Rodrigo Uroz
2026-02-10 08:42:22 -03:00
committed by Peter Steinberger
parent fa9420069a
commit 6b3e0710f4
13 changed files with 1372 additions and 35 deletions

View File

@@ -353,6 +353,7 @@ agents: {
```
Tools:
- `memory_search` — returns snippets with file + line ranges.
- `memory_get` — read memory file content by path.
@@ -428,23 +429,136 @@ This isn't "IR-theory perfect", but it's simple, fast, and tends to improve reca
If we want to get fancier later, common next steps are Reciprocal Rank Fusion (RRF) or score normalization
(min/max or z-score) before mixing.
#### Post-processing pipeline
After merging vector and keyword scores, two optional post-processing stages
refine the result list before it reaches the agent:
```
Vector + Keyword → Weighted Merge → Temporal Decay → Sort → MMR → Top-K Results
```
Both stages are **off by default** and can be enabled independently.
#### MMR re-ranking (diversity)
When hybrid search returns results, multiple chunks may contain similar or overlapping content.
For example, searching for "home network setup" might return five nearly identical snippets
from different daily notes that all mention the same router configuration.
**MMR (Maximal Marginal Relevance)** re-ranks the results to balance relevance with diversity,
ensuring the top results aren't all saying the same thing.
ensuring the top results cover different aspects of the query instead of repeating the same information.
How it works:
1. Results are scored by their original relevance (vector + BM25 weighted score).
2. MMR iteratively selects results that maximize: `λ × relevance (1λ) × similarity_to_selected`.
3. Already-selected results are penalized via Jaccard text similarity.
2. MMR iteratively selects results that maximize: `λ × relevance (1λ) × max_similarity_to_selected`.
3. Similarity between results is measured using Jaccard text similarity on tokenized content.
The `lambda` parameter controls the trade-off:
- `lambda = 1.0` → pure relevance (no diversity penalty)
- `lambda = 0.0` → maximum diversity (ignores relevance)
- Default: `0.7` (balanced, slight relevance bias)
Config:
**Example — query: "home network setup"**
Given these memory files:
```
memory/2026-02-10.md → "Configured Omada router, set VLAN 10 for IoT devices"
memory/2026-02-08.md → "Configured Omada router, moved IoT to VLAN 10"
memory/2026-02-05.md → "Set up AdGuard DNS on 192.168.10.2"
memory/network.md → "Router: Omada ER605, AdGuard: 192.168.10.2, VLAN 10: IoT"
```
Without MMR — top 3 results:
```
1. memory/2026-02-10.md (score: 0.92) ← router + VLAN
2. memory/2026-02-08.md (score: 0.89) ← router + VLAN (near-duplicate!)
3. memory/network.md (score: 0.85) ← reference doc
```
With MMR (λ=0.7) — top 3 results:
```
1. memory/2026-02-10.md (score: 0.92) ← router + VLAN
2. memory/network.md (score: 0.85) ← reference doc (diverse!)
3. memory/2026-02-05.md (score: 0.78) ← AdGuard DNS (diverse!)
```
The near-duplicate from Feb 8 drops out, and the agent gets three distinct pieces of information.
**When to enable:** If you notice `memory_search` returning redundant or near-duplicate snippets,
especially with daily notes that often repeat similar information across days.
#### Temporal decay (recency boost)
Agents with daily notes accumulate hundreds of dated files over time. Without decay,
a well-worded note from six months ago can outrank yesterday's update on the same topic.
**Temporal decay** applies an exponential multiplier to scores based on the age of each result,
so recent memories naturally rank higher while old ones fade:
```
decayedScore = score × e^(-λ × ageInDays)
```
where `λ = ln(2) / halfLifeDays`.
With the default half-life of 30 days:
- Today's notes: **100%** of original score
- 7 days ago: **~84%**
- 30 days ago: **50%**
- 90 days ago: **12.5%**
- 180 days ago: **~1.6%**
**Evergreen files are never decayed:**
- `MEMORY.md` (root memory file)
- Non-dated files in `memory/` (e.g., `memory/projects.md`, `memory/network.md`)
- These contain durable reference information that should always rank normally.
**Dated daily files** (`memory/YYYY-MM-DD.md`) use the date extracted from the filename.
Other sources (e.g., session transcripts) fall back to file modification time (`mtime`).
**Example — query: "what's Rod's work schedule?"**
Given these memory files (today is Feb 10):
```
memory/2025-09-15.md → "Rod works Mon-Fri, standup at 10am, pairing at 2pm" (148 days old)
memory/2026-02-10.md → "Rod has standup at 14:15, 1:1 with Zeb at 14:45" (today)
memory/2026-02-03.md → "Rod started new team, standup moved to 14:15" (7 days old)
```
Without decay:
```
1. memory/2025-09-15.md (score: 0.91) ← best semantic match, but stale!
2. memory/2026-02-10.md (score: 0.82)
3. memory/2026-02-03.md (score: 0.80)
```
With decay (halfLife=30):
```
1. memory/2026-02-10.md (score: 0.82 × 1.00 = 0.82) ← today, no decay
2. memory/2026-02-03.md (score: 0.80 × 0.85 = 0.68) ← 7 days, mild decay
3. memory/2025-09-15.md (score: 0.91 × 0.03 = 0.03) ← 148 days, nearly gone
```
The stale September note drops to the bottom despite having the best raw semantic match.
**When to enable:** If your agent has months of daily notes and you find that old,
stale information outranks recent context. A half-life of 30 days works well for
daily-note-heavy workflows; increase it (e.g., 90 days) if you reference older notes frequently.
#### Configuration
Both features are configured under `memorySearch.query.hybrid`:
```json5
agents: {
@@ -456,9 +570,15 @@ agents: {
vectorWeight: 0.7,
textWeight: 0.3,
candidateMultiplier: 4,
// Diversity: reduce redundant results
mmr: {
enabled: true,
lambda: 0.7
enabled: true, // default: false
lambda: 0.7 // 0 = max diversity, 1 = max relevance
},
// Recency: boost newer memories
temporalDecay: {
enabled: true, // default: false
halfLifeDays: 30 // score halves every 30 days
}
}
}
@@ -467,6 +587,12 @@ agents: {
}
```
You can enable either feature independently:
- **MMR only** — useful when you have many similar notes but age doesn't matter.
- **Temporal decay only** — useful when recency matters but your results are already diverse.
- **Both** — recommended for agents with large, long-running daily note histories.
### Embedding cache
OpenClaw can cache **chunk embeddings** in SQLite so reindexing and frequent updates (especially session transcripts) don't re-embed unchanged text.