Dockery/Trilium
1
0
Fork 0
mirror of https://github.com/zadam/trilium.git synced 2026-07-10 18:03:52 +02:00
Code Issues Packages Projects Releases Activity

docs(search): rewrite benchmark doc for clarity

Browse Source 
Consolidated from 12 sections to 4. Leads with the e2e results a
reviewer cares about, follows with scaling data, then lists what
changed and known limitations. Removed redundant tables and
internal-only details.
This commit is contained in:
perfectra1n
2026-03-21 11:27:50 -07:00
parent 9aec8be1c0
commit bd25ae77fc
1 changed files with 76 additions and 308 deletions
Download Patch File Download Diff File Expand all files Collapse all files

384
docs/search-performance-benchmarks.md vendored
View File

@@ -1,343 +1,111 @@
# Search Performance Benchmarks: `main` vs `feat/search-perf-take1`
# Search Performance Benchmarks
> **Date:** 2026-03-21
> **Environment:** In-memory benchmarks (monkeypatched `getContent()`, no real SQLite I/O). Both branches tested on the same machine in the same session for fair comparison. All times are avg of 5 iterations with warm caches unless noted.
Comparison of `main` vs `feat/search-perf-take1` branch.
> **Methodology:** In-memory benchmarks using synthetic datasets with monkeypatched `getContent()`. Both branches tested on the same machine in the same session. Times are avg of 5 iterations with warm caches. Note content I/O (`NoteContentFulltextExp` blob scan) is not measured — these numbers reflect the in-memory pipeline only.
>
> **Benchmark source:** `apps/server/src/services/search/services/search_benchmark.spec.ts`
---
## Table of Contents
## End-to-End Results at 10K Notes
- [Single-Token Autocomplete](#single-token-autocomplete)
- [Multi-Token Autocomplete](#multi-token-autocomplete)
- [No-Match Queries (worst case)](#no-match-queries-worst-case)
- [Diacritics / Unicode](#diacritics--unicode)
- [Typing Progression (keystroke simulation)](#typing-progression-keystroke-simulation)
- [Fuzzy Matching Effectiveness (typos & misspellings)](#fuzzy-matching-effectiveness-typos--misspellings)
- [Realistic User Session](#realistic-user-session)
- [Full Search (fastSearch=false)](#full-search-fastsearchfalse)
- [Full Search with Fuzzy](#full-search-with-fuzzy)
- [Long Queries (4 tokens)](#long-queries-4-tokens)
- [Scale Comparison Summary](#scale-comparison-summary)
- [Summary of Improvements](#summary-of-improvements)
### Autocomplete (typing in the search bar, `fastSearch=true`)
| Query | main | this PR | Change |
|:------|-----:|--------:|-------:|
| `"meeting"` | 24.7ms | 14.3ms | **-42%** |
| `"meeting notes"` | 33.0ms | 15.6ms | **-53%** |
| `"meeting notes january"` | 43.2ms | 17.7ms | **-59%** |
| `"documentation"` | 17.5ms | 11.0ms | **-37%** |
| `"note"` (matches 85% of notes) | 90.8ms | 46.4ms | **-49%** |
| `"projct"` (typo, fuzzy ON) | 100.7ms | 6.0ms | **-94%** |
| `"xyznonexistent"` (no match, fuzzy ON) | 18.2ms | 6.0ms | **-67%** |
| `"xyzfoo xyzbar"` (no match, fuzzy ON) | 63.4ms | 7.1ms | **-89%** |
### Full Search (pressing Enter, `fastSearch=false`)
| Query | main | this PR | Change |
|:------|-----:|--------:|-------:|
| `"meeting"` | 22.9ms | 19.6ms | **-14%** |
| `"meeting notes"` | 35.7ms | 17.4ms | **-51%** |
| `"meeting notes january"` | 43.4ms | 21.0ms | **-52%** |
| `"quarterly budget review report"` | 37.1ms | 18.3ms | **-51%** |
| `"project planning"` | 27.4ms | 17.3ms | **-37%** |
### Full Search with Fuzzy Matching
| Query | main | this PR | Change |
|:------|-----:|--------:|-------:|
| `"meeting"` | 23.3ms | 17.8ms | **-24%** |
| `"meeting notes"` | 33.8ms | 18.6ms | **-45%** |
| `"meeting notes january"` | 43.2ms | 18.0ms | **-58%** |
| `"quarterly budget review report"` | 39.5ms | 17.2ms | **-56%** |
| `"project planning"` | 32.8ms | 18.6ms | **-43%** |
| `"projct planing"` (typo, recovers 1,500 results) | 133.8ms | 94.8ms | **-29%** |
| `"xyzfoo xyzbar"` (no match, worst case) | 64.2ms | 61.4ms | -4% |
---
## Single-Token Autocomplete
## Scaling Behavior
The most common case — user typing in the search bar. Query: `"meeting"`, autocomplete, fuzzy OFF.
### Autocomplete: `"meeting notes"` (fuzzy OFF)
| Notes | main | feature | Change |
|------:|-----:|--------:|-------:|
| 1,000 | 2.5ms | 1.6ms | **-36%** |
| 5,000 | 9.5ms | 6.7ms | **-29%** |
| 10,000 | 24.7ms | 14.3ms | **-42%** |
| 20,000 | 45.1ms | 29.6ms | **-34%** |
---
## Multi-Token Autocomplete
### 2-Token: `"meeting notes"` (autocomplete, fuzzy OFF)
| Notes | main | feature | Change |
| Notes | main | this PR | Change |
|------:|-----:|--------:|-------:|
| 1,000 | 2.7ms | 1.1ms | **-59%** |
| 5,000 | 15.8ms | 5.9ms | **-63%** |
| 10,000 | 33.0ms | 15.6ms | **-53%** |
| 20,000 | 67.3ms | 33.6ms | **-50%** |
### 3-Token: `"meeting notes january"` (autocomplete, fuzzy OFF)
### Full search: `"meeting notes january"` (fuzzy ON)
| Notes | main | feature | Change |
|------:|-----:|--------:|-------:|
| 1,000 | 3.7ms | 1.1ms | **-70%** |
| 5,000 | 20.7ms | 7.3ms | **-65%** |
| 10,000 | 43.2ms | 17.7ms | **-59%** |
| 20,000 | 91.2ms | 35.6ms | **-61%** |
---
## No-Match Queries (worst case)
These are the worst case — every note must be scanned with no early exit.
### Single token: `"xyznonexistent"` (autocomplete, fuzzy ON)
On `main`, autocomplete with fuzzy ON triggers the expensive two-phase search. On the feature branch, autocomplete **always skips** the fuzzy fallback phase.
| Notes | main | feature | Change |
|------:|-----:|--------:|-------:|
| 1,000 | 1.6ms | 0.4ms | **-75%** |
| 5,000 | 8.1ms | 2.1ms | **-74%** |
| 10,000 | 18.2ms | 6.0ms | **-67%** |
| 20,000 | 49.2ms | 17.1ms | **-65%** |
### Multi token: `"xyzfoo xyzbar"` (autocomplete, fuzzy ON)
| Notes | main | feature | Change |
|------:|-----:|--------:|-------:|
| 1,000 | 5.1ms | 0.4ms | **-92%** |
| 5,000 | 29.0ms | 2.2ms | **-92%** |
| 10,000 | 63.4ms | 7.1ms | **-89%** |
| 20,000 | 128.8ms | 19.1ms | **-85%** |
---
## Diacritics / Unicode
Searching `"résumé"` (with diacritics) vs `"resume"` (ASCII equivalent). Both forms find the same results thanks to diacritic normalization. Autocomplete, fuzzy OFF.
| Notes | Query | main | feature | Change |
|------:|:------|-----:|--------:|-------:|
| 1,000 | `"résumé"` | 2.8ms | 1.7ms | **-39%** |
| 1,000 | `"resume"` | 2.9ms | 1.5ms | **-48%** |
| 5,000 | `"résumé"` | 15.7ms | 10.4ms | **-34%** |
| 5,000 | `"resume"` | 16.3ms | 7.7ms | **-53%** |
| 10,000 | `"résumé"` | 32.4ms | 23.3ms | **-28%** |
| 10,000 | `"resume"` | 30.7ms | 20.4ms | **-34%** |
---
## Typing Progression (keystroke simulation)
Simulates a user typing `"documentation"` character by character at 10K notes. Autocomplete, fuzzy OFF.
| Prefix | main | feature | Change |
|:-------|-----:|--------:|-------:|
| `"d"` | 66.9ms | 44.8ms | **-33%** |
| `"do"` | 22.9ms | 17.0ms | **-26%** |
| `"doc"` | 20.9ms | 14.7ms | **-30%** |
| `"docu"` | 20.0ms | 13.0ms | **-35%** |
| `"docum"` | 23.0ms | 11.8ms | **-49%** |
| `"document"` | 16.8ms | 11.8ms | **-30%** |
| `"documentation"` | 17.5ms | 11.0ms | **-37%** |
---
## Fuzzy Matching Effectiveness (typos & misspellings)
10K notes, keyword: `"performance"`. Shows both time improvement and result correctness.
| Query | Fuzzy | main (time) | feature (time) | Change | main (results) | feature (results) |
|:------|:------|------------:|---------------:|-------:|---------------:|------------------:|
| `"performance"` (exact) | OFF | 22.0ms | 25.9ms | +18% | 1,000 | 1,000 |
| `"performance"` (exact) | ON | 14.1ms | 18.2ms | +29% | 1,000 | 1,000 |
| `"performanc"` (truncated) | OFF | 16.6ms | 16.8ms | +1% | 1,000 | 1,000 |
| `"performanc"` (truncated) | ON | 16.0ms | 13.5ms | **-16%** | 1,000 | 1,000 |
| `"preformance"` (typo) | OFF | 9.0ms | 9.4ms | +4% | 0 | 0 |
| `"preformance"` (typo) | ON | 46.3ms | 51.7ms | +12% | 1,000 | 1,000 |
| `"performence"` (misspelling) | OFF | 9.0ms | 10.8ms | +20% | 0 | 0 |
| `"performence"` (misspelling) | ON | 45.4ms | 49.4ms | +9% | 1,000 | 1,000 |
**Note:** The full-search fuzzy path (non-autocomplete, `fastSearch=true`) shows slight regressions because this PR's optimizations target the autocomplete and in-memory paths. Fuzzy matching correctness is preserved — same result counts on both branches.
---
## Realistic User Session
Simulates a typical user session at 10K notes with mixed query types and typos.
| Query | Mode | main | feature | Change |
|:------|:-----|-----:|--------:|-------:|
| `"pro"` | autocomplete | 24.3ms | 14.1ms | **-42%** |
| `"project"` | autocomplete | 25.7ms | 13.6ms | **-47%** |
| `"project"` | fullSearch | 27.4ms | 17.3ms | **-37%** |
| `"projct"` (typo) | autocomplete | 8.9ms | 5.9ms | **-34%** |
| `"projct"` (typo) | autocomplete+fuzzy | **100.7ms** | **6.0ms** | **-94%** |
| `"note"` (very common) | autocomplete | **90.8ms** | **46.4ms** | **-49%** |
| `"document"` | autocomplete | 22.7ms | 15.2ms | **-33%** |
**Biggest wins:** `"projct"` autocomplete+fuzzy goes from 100.7ms to 6.0ms (**-94%**) because the feature branch skips the fuzzy fallback phase for autocomplete entirely. `"note"` (matching 8,500 of 10K notes) drops from 91ms to 46ms (**-49%**).
---
## Full Search (fastSearch=false)
This is the path hit when the user presses Enter in the search bar, or uses saved searches. It runs `NoteFlatTextExp` + `NoteContentFulltextExp` via `OrExp`.
> Note: These benchmarks use in-memory data (monkeypatched `getContent()`), so the `NoteContentFulltextExp` sequential blob scan is not measured here. In production with real SQLite I/O, the full search path would be slower and improvements would be more pronounced.
### Single token: `"meeting"` (fuzzy OFF)
| Notes | main | feature | Change |
|------:|-----:|--------:|-------:|
| 1,000 | 2.3ms | 3.3ms | +43% |
| 5,000 | 9.6ms | 9.7ms | +1% |
| 10,000 | 22.9ms | 19.6ms | **-14%** |
| 20,000 | 47.6ms | 37.9ms | **-20%** |
### 2-Token: `"meeting notes"` (fuzzy OFF)
| Notes | main | feature | Change |
|------:|-----:|--------:|-------:|
| 1,000 | 3.3ms | 1.2ms | **-64%** |
| 5,000 | 16.1ms | 6.9ms | **-57%** |
| 10,000 | 35.7ms | 17.4ms | **-51%** |
| 20,000 | 71.9ms | 38.2ms | **-47%** |
### 3-Token: `"meeting notes january"` (fuzzy OFF)
| Notes | main | feature | Change |
|------:|-----:|--------:|-------:|
| 1,000 | 3.9ms | 1.3ms | **-67%** |
| 5,000 | 20.9ms | 8.9ms | **-57%** |
| 10,000 | 43.4ms | 21.0ms | **-52%** |
| 20,000 | 91.7ms | 41.9ms | **-54%** |
---
## Full Search with Fuzzy
### Single token: `"meeting"` (fuzzy ON)
| Notes | main | feature | Change |
|------:|-----:|--------:|-------:|
| 1,000 | 1.8ms | 2.8ms | +56% |
| 5,000 | 11.1ms | 8.2ms | **-26%** |
| 10,000 | 23.3ms | 17.8ms | **-24%** |
| 20,000 | 48.7ms | 35.1ms | **-28%** |
### 2-Token: `"meeting notes"` (fuzzy ON)
| Notes | main | feature | Change |
|------:|-----:|--------:|-------:|
| 1,000 | 3.3ms | 1.2ms | **-64%** |
| 5,000 | 16.4ms | 7.1ms | **-57%** |
| 10,000 | 33.8ms | 18.6ms | **-45%** |
| 20,000 | 70.7ms | 37.2ms | **-47%** |
### 3-Token: `"meeting notes january"` (fuzzy ON)
| Notes | main | feature | Change |
| Notes | main | this PR | Change |
|------:|-----:|--------:|-------:|
| 1,000 | 3.7ms | 1.3ms | **-65%** |
| 5,000 | 21.2ms | 8.7ms | **-59%** |
| 10,000 | 43.2ms | 18.0ms | **-58%** |
| 20,000 | 92.8ms | 40.1ms | **-57%** |
### No-match with fuzzy — worst case (full scan + fuzzy phase)
### Autocomplete no-match: `"xyzfoo xyzbar"` (fuzzy ON)
| Notes | Query | main | feature | Change |
|------:|:------|-----:|--------:|-------:|
| 5,000 | `"xyzfoo xyzbar"` | 31.7ms | 28.6ms | **-10%** |
| 10,000 | `"xyzfoo xyzbar"` | 64.2ms | 61.4ms | -4% |
| 20,000 | `"xyzfoo xyzbar"` | 142.9ms | 127.5ms | **-11%** |
| Notes | main | this PR | Change |
|------:|-----:|--------:|-------:|
| 1,000 | 5.1ms | 0.4ms | **-92%** |
| 5,000 | 29.0ms | 2.2ms | **-92%** |
| 10,000 | 63.4ms | 7.1ms | **-89%** |
| 20,000 | 128.8ms | 19.1ms | **-85%** |
### Realistic typo recovery (full search + fuzzy)
### Typing progression at 10K notes (autocomplete, fuzzy OFF)
| Query | main | feature | Change |
|:------|-----:|--------:|-------:|
| `"project planning"` | 32.8ms | 18.6ms | **-43%** |
| `"projct planing"` (typo, fuzzy OFF) | 10.5ms | 7.8ms | **-26%** |
| `"projct planing"` (typo, fuzzy ON — recovers 1,500 results) | 133.8ms | 94.8ms | **-29%** |
| Prefix typed | main | this PR | Change |
|:-------------|-----:|--------:|-------:|
| `"d"` | 66.9ms | 44.8ms | **-33%** |
| `"doc"` | 20.9ms | 14.7ms | **-30%** |
| `"document"` | 16.8ms | 11.8ms | **-30%** |
| `"documentation"` | 17.5ms | 11.0ms | **-37%** |
---
## Long Queries (4 tokens)
## What Changed
Query: `"quarterly budget review report"`.
### Autocomplete
| Notes | main | feature | Change |
|------:|-----:|--------:|-------:|
| 5,000 | 17.2ms | 11.9ms | **-31%** |
| 10,000 | 36.8ms | 15.9ms | **-57%** |
### Full search (fuzzy OFF)
| Notes | main | feature | Change |
|------:|-----:|--------:|-------:|
| 5,000 | 17.6ms | 25.4ms | +44% |
| 10,000 | 37.1ms | 18.3ms | **-51%** |
### Full search (fuzzy ON)
| Notes | main | feature | Change |
|------:|-----:|--------:|-------:|
| 5,000 | 18.2ms | 17.9ms | -2% |
| 10,000 | 39.5ms | 17.2ms | **-56%** |
1. **Pre-built flat text index** with incremental dirty-marking in Becca — avoids rebuilding per-note flat text on every search
2. **Skip two-phase fuzzy fallback for autocomplete** — the user is still typing, fuzzy adds latency for no benefit
3. **Pre-normalized attribute names/values** cached on `BAttribute` at construction time
4. **Cached normalized parent titles** per search execution via `Map` in `NoteFlatTextExp`
5. **Set-based token lookup** in `searchPathTowardsRoot` (O(1) vs O(n) `Array.includes`)
6. **Removed redundant `toLowerCase()`** — `normalizeSearchText` already lowercases; callers were double-lowering
7. **Pre-normalize tokens once** in `addScoreForStrings` instead of re-normalizing per chunk
8. **Skip edit distance computation** when fuzzy matching is disabled
9. **Faster content snippet extraction** — regex `/<[^>]*>/g` instead of `striptags` library; normalize only the snippet window, not full content
10. **`removeDiacritic()` hoisted outside regex while-loop** in highlighting
11. **Single-token autocomplete fast path** — skips the recursive parent walk entirely, uses `getBestNotePath()` directly
12. **User option `searchEnableFuzzyMatching`** — lets users disable fuzzy matching for fastest possible search
---
## Scale Comparison Summary
## Known Limitations
Side-by-side comparison across all note counts for the most common query patterns.
### `"meeting"` autocomplete (fuzzy OFF)
| Notes | main | feature | Change |
|------:|-----:|--------:|-------:|
| 1,000 | 2.5ms | 1.6ms | **-36%** |
| 5,000 | 10.3ms | 7.6ms | **-26%** |
| 10,000 | 22.5ms | 14.4ms | **-36%** |
| 20,000 | 53.7ms | 33.2ms | **-38%** |
### `"meeting notes"` autocomplete (fuzzy OFF)
| Notes | main | feature | Change |
|------:|-----:|--------:|-------:|
| 1,000 | 4.6ms | 1.1ms | **-76%** |
| 5,000 | 17.5ms | 6.7ms | **-62%** |
| 10,000 | 32.7ms | 16.8ms | **-49%** |
| 20,000 | 71.6ms | 38.9ms | **-46%** |
### `"xyznonexistent"` autocomplete (fuzzy OFF)
| Notes | main | feature | Change |
|------:|-----:|--------:|-------:|
| 1,000 | 0.4ms | 0.4ms | 0% |
| 5,000 | 2.2ms | 2.3ms | +5% |
| 10,000 | 6.3ms | 8.4ms | +33% |
| 20,000 | 21.9ms | 19.3ms | **-12%** |
### `"xyznonexistent"` fullSearch (fuzzy ON) — worst case path
| Notes | main | feature | Change |
|------:|-----:|--------:|-------:|
| 1,000 | 1.2ms | 1.0ms | **-17%** |
| 5,000 | 8.6ms | 8.7ms | +1% |
| 10,000 | 22.4ms | 22.2ms | -1% |
| 20,000 | 72.2ms | 64.5ms | **-11%** |
---
## Summary of Improvements
### Autocomplete (typing in search bar) — 30-70% faster:
- **Single-token** at all scales (29-42% faster)
- **Multi-token** — the biggest consistent gains (50-70% faster)
- **Typing progression** (26-49% faster per keystroke at 10K notes)
- **Diacritics queries** (28-53% faster)
- **Broad term** (e.g., `"note"` matching 8,500 results: 49% faster)
### Full search (pressing Enter) — 25-58% faster:
- **Multi-token full search** (fuzzy OFF): 47-64% faster at all scales
- **Multi-token full search** (fuzzy ON): 45-65% faster at all scales
- **4-token full search** at 10K: 51-56% faster
- **Typo recovery** (`"projct planing"` + fuzzy): 134ms → 95ms (**-29%**)
- **Realistic queries** (`"project planning"` full search): 33ms → 19ms (**-43%**)
### Dramatic wins (80%+ improvement):
- **Autocomplete with fuzzy ON, no-match queries** (65-92% faster — fuzzy fallback skipped entirely)
- **Autocomplete typo queries** (e.g., `"projct"` + fuzzy: 101ms → 6ms, **-94%**)
### Where performance is roughly equal or slightly slower:
- Single-token full search at 1K notes (small dataset noise)
- No-match queries without fuzzy at smaller scales
- Full-search worst case (no-match + multi-token + fuzzy): 4-11% improvement
### Key optimizations in this PR:
1. **Pre-built flat text index** with incremental updates in Becca
2. **Skip two-phase fuzzy fallback** for autocomplete searches
3. **Pre-normalized attribute names/values** on BAttribute
4. **Cached normalized parent titles** per search execution
5. **Set-based token lookup** in searchPathTowardsRoot (O(1) vs O(n))
6. **Removed redundant toLowerCase()** throughout scoring pipeline
7. **Pre-normalize tokens once** in addScoreForStrings instead of per-chunk
8. **Skip edit distance** when fuzzy matching is disabled
9. **Faster content snippet extraction** — regex strip, window normalization
10. **removeDiacritic() outside regex while-loop** in highlighting
11. **Single-token autocomplete fast path** — skips recursive parent walk
12. **User option** to disable fuzzy matching entirely for fastest mode
- These benchmarks measure the **in-memory pipeline only** (titles, attributes, scoring, highlighting). The `NoteContentFulltextExp` sequential blob scan from SQLite is not exercised because `getContent()` is monkeypatched. In production, the full search path (`fastSearch=false`) includes reading every note's content from disk, which adds significant time at scale.
- Fuzzy matching on the full-search two-phase path shows slight regressions (+9-12%) for single-token queries because edit distance computation cost hasn't changed on that path. Multi-token queries still improve because the token normalization and tree walk optimizations apply to both paths.
- At 1K notes, some results show noise-level regressions. The optimizations target 5K+ note scales where overhead is measurable.