diff --git a/docs/DATABASE.md b/docs/DATABASE.md deleted file mode 100644 index 52df42e0b..000000000 --- a/docs/DATABASE.md +++ /dev/null @@ -1,736 +0,0 @@ -# Trilium Database Architecture - -> **Related:** [ARCHITECTURE.md](ARCHITECTURE.md) | [Database Schema](Developer%20Guide/Developer%20Guide/Development%20and%20architecture/Database/) - -## Overview - -Trilium uses **SQLite** as its embedded database engine, providing a reliable, file-based storage system that requires no separate database server. The database stores all notes, their relationships, metadata, and configuration. - -## Database File - -**Location:** -- Desktop: `~/.local/share/trilium-data/document.db` (Linux/Mac) or `%APPDATA%/trilium-data/document.db` (Windows) -- Server: Configured via `TRILIUM_DATA_DIR` environment variable -- Docker: Mounted volume at `/home/node/trilium-data/` - -**Characteristics:** -- Single-file database -- Embedded (no server required) -- ACID compliant -- Cross-platform -- Supports up to 281 TB database size -- Efficient for 100k+ notes - -## Database Driver - -**Library:** `better-sqlite3` - -**Why better-sqlite3:** -- Native performance (C++ bindings) -- Synchronous API (simpler code) -- Prepared statements -- Transaction support -- Type safety - -**Usage:** -```typescript -// apps/server/src/services/sql.ts -import Database from 'better-sqlite3' - -const db = new Database('document.db') -const stmt = db.prepare('SELECT * FROM notes WHERE noteId = ?') -const note = stmt.get(noteId) -``` - -## Schema Overview - -Schema location: `apps/server/src/assets/db/schema.sql` - -**Entity Tables:** -- `notes` - Core note data -- `branches` - Tree relationships -- `attributes` - Metadata (labels/relations) -- `revisions` - Version history -- `attachments` - File attachments -- `blobs` - Binary content storage - -**System Tables:** -- `options` - Application configuration -- `entity_changes` - Change tracking for sync -- `recent_notes` - Recently accessed notes -- `etapi_tokens` - API authentication tokens -- `user_data` - User credentials -- `sessions` - Web session storage - -## Entity Tables - -### Notes Table - -```sql -CREATE TABLE notes ( - noteId TEXT NOT NULL PRIMARY KEY, - title TEXT NOT NULL DEFAULT "note", - isProtected INT NOT NULL DEFAULT 0, - type TEXT NOT NULL DEFAULT 'text', - mime TEXT NOT NULL DEFAULT 'text/html', - blobId TEXT DEFAULT NULL, - isDeleted INT NOT NULL DEFAULT 0, - deleteId TEXT DEFAULT NULL, - dateCreated TEXT NOT NULL, - dateModified TEXT NOT NULL, - utcDateCreated TEXT NOT NULL, - utcDateModified TEXT NOT NULL -); - --- Indexes for performance -CREATE INDEX IDX_notes_title ON notes (title); -CREATE INDEX IDX_notes_type ON notes (type); -CREATE INDEX IDX_notes_dateCreated ON notes (dateCreated); -CREATE INDEX IDX_notes_dateModified ON notes (dateModified); -CREATE INDEX IDX_notes_utcDateModified ON notes (utcDateModified); -CREATE INDEX IDX_notes_blobId ON notes (blobId); -``` - -**Field Descriptions:** - -| Field | Type | Description | -|-------|------|-------------| -| `noteId` | TEXT | Unique identifier (UUID or custom) | -| `title` | TEXT | Note title (displayed in tree) | -| `isProtected` | INT | 1 if encrypted, 0 if not | -| `type` | TEXT | Note type: text, code, file, image, etc. | -| `mime` | TEXT | MIME type: text/html, application/json, etc. | -| `blobId` | TEXT | Reference to content in blobs table | -| `isDeleted` | INT | Soft delete flag | -| `deleteId` | TEXT | Unique delete operation ID | -| `dateCreated` | TEXT | Creation date (local timezone) | -| `dateModified` | TEXT | Last modified (local timezone) | -| `utcDateCreated` | TEXT | Creation date (UTC) | -| `utcDateModified` | TEXT | Last modified (UTC) | - -**Note Types:** -- `text` - Rich text with HTML -- `code` - Source code -- `file` - Binary file -- `image` - Image file -- `search` - Saved search -- `render` - Custom HTML rendering -- `relation-map` - Relationship diagram -- `canvas` - Excalidraw drawing -- `mermaid` - Mermaid diagram -- `book` - Container for documentation -- `web-view` - Embedded web page -- `mindmap` - Mind map -- `geomap` - Geographical map - -### Branches Table - -```sql -CREATE TABLE branches ( - branchId TEXT NOT NULL PRIMARY KEY, - noteId TEXT NOT NULL, - parentNoteId TEXT NOT NULL, - notePosition INTEGER NOT NULL, - prefix TEXT, - isExpanded INTEGER NOT NULL DEFAULT 0, - isDeleted INTEGER NOT NULL DEFAULT 0, - deleteId TEXT DEFAULT NULL, - utcDateModified TEXT NOT NULL -); - --- Indexes -CREATE INDEX IDX_branches_noteId_parentNoteId ON branches (noteId, parentNoteId); -CREATE INDEX IDX_branches_parentNoteId ON branches (parentNoteId); -``` - -**Field Descriptions:** - -| Field | Type | Description | -|-------|------|-------------| -| `branchId` | TEXT | Unique identifier for this branch | -| `noteId` | TEXT | Child note ID | -| `parentNoteId` | TEXT | Parent note ID | -| `notePosition` | INT | Sort order among siblings | -| `prefix` | TEXT | Optional prefix text (e.g., "Chapter 1:") | -| `isExpanded` | INT | Tree expansion state | -| `isDeleted` | INT | Soft delete flag | -| `deleteId` | TEXT | Delete operation ID | -| `utcDateModified` | TEXT | Last modified (UTC) | - -**Key Concepts:** -- **Cloning:** A note can have multiple branches (multiple parents) -- **Position:** Siblings ordered by `notePosition` -- **Prefix:** Display text before note title in tree -- **Soft Delete:** Allows sync before permanent deletion - -### Attributes Table - -```sql -CREATE TABLE attributes ( - attributeId TEXT NOT NULL PRIMARY KEY, - noteId TEXT NOT NULL, - type TEXT NOT NULL, - name TEXT NOT NULL, - value TEXT DEFAULT '' NOT NULL, - position INT DEFAULT 0 NOT NULL, - utcDateModified TEXT NOT NULL, - isDeleted INT NOT NULL, - deleteId TEXT DEFAULT NULL, - isInheritable INT DEFAULT 0 NULL -); - --- Indexes -CREATE INDEX IDX_attributes_name_value ON attributes (name, value); -CREATE INDEX IDX_attributes_noteId ON attributes (noteId); -CREATE INDEX IDX_attributes_value ON attributes (value); -``` - -**Field Descriptions:** - -| Field | Type | Description | -|-------|------|-------------| -| `attributeId` | TEXT | Unique identifier | -| `noteId` | TEXT | Note this attribute belongs to | -| `type` | TEXT | 'label' or 'relation' | -| `name` | TEXT | Attribute name | -| `value` | TEXT | Attribute value (text for labels, noteId for relations) | -| `position` | INT | Display order | -| `utcDateModified` | TEXT | Last modified (UTC) | -| `isDeleted` | INT | Soft delete flag | -| `deleteId` | TEXT | Delete operation ID | -| `isInheritable` | INT | Inherited by child notes | - -**Attribute Types:** - -**Labels** (key-value pairs): -```sql --- Example: #priority=high -INSERT INTO attributes (attributeId, noteId, type, name, value) -VALUES ('attr1', 'note123', 'label', 'priority', 'high') -``` - -**Relations** (links to other notes): -```sql --- Example: ~author=[[noteId]] -INSERT INTO attributes (attributeId, noteId, type, name, value) -VALUES ('attr2', 'note123', 'relation', 'author', 'author-note-id') -``` - -**Special Attributes:** -- `#run=frontendStartup` - Execute script on frontend load -- `#run=backendStartup` - Execute script on backend load -- `#customWidget` - Custom widget implementation -- `#iconClass` - Custom tree icon -- `#cssClass` - CSS class for note -- `#sorted` - Auto-sort children -- `#hideChildrenOverview` - Don't show child list - -### Revisions Table - -```sql -CREATE TABLE revisions ( - revisionId TEXT NOT NULL PRIMARY KEY, - noteId TEXT NOT NULL, - type TEXT DEFAULT '' NOT NULL, - mime TEXT DEFAULT '' NOT NULL, - title TEXT NOT NULL, - isProtected INT NOT NULL DEFAULT 0, - blobId TEXT DEFAULT NULL, - utcDateLastEdited TEXT NOT NULL, - utcDateCreated TEXT NOT NULL, - utcDateModified TEXT NOT NULL, - dateLastEdited TEXT NOT NULL, - dateCreated TEXT NOT NULL -); - --- Indexes -CREATE INDEX IDX_revisions_noteId ON revisions (noteId); -CREATE INDEX IDX_revisions_utcDateCreated ON revisions (utcDateCreated); -CREATE INDEX IDX_revisions_utcDateLastEdited ON revisions (utcDateLastEdited); -CREATE INDEX IDX_revisions_blobId ON revisions (blobId); -``` - -**Revision Strategy:** -- Automatic revision created on note modification -- Configurable interval (default: daily max) -- Stores complete note snapshot -- Allows reverting to previous versions -- Can be disabled with `#disableVersioning` - -### Attachments Table - -```sql -CREATE TABLE attachments ( - attachmentId TEXT NOT NULL PRIMARY KEY, - ownerId TEXT NOT NULL, - role TEXT NOT NULL, - mime TEXT NOT NULL, - title TEXT NOT NULL, - isProtected INT NOT NULL DEFAULT 0, - position INT DEFAULT 0 NOT NULL, - blobId TEXT DEFAULT NULL, - dateModified TEXT NOT NULL, - utcDateModified TEXT NOT NULL, - utcDateScheduledForErasureSince TEXT DEFAULT NULL, - isDeleted INT NOT NULL, - deleteId TEXT DEFAULT NULL -); - --- Indexes -CREATE INDEX IDX_attachments_ownerId_role ON attachments (ownerId, role); -CREATE INDEX IDX_attachments_blobId ON attachments (blobId); -``` - -**Attachment Roles:** -- `file` - Regular file attachment -- `image` - Image file -- `cover-image` - Note cover image -- Custom roles for specific purposes - -### Blobs Table - -```sql -CREATE TABLE blobs ( - blobId TEXT NOT NULL PRIMARY KEY, - content TEXT NULL DEFAULT NULL, - dateModified TEXT NOT NULL, - utcDateModified TEXT NOT NULL -); -``` - -**Blob Usage:** -- Stores actual content (text or binary) -- Referenced by notes, revisions, attachments -- Deduplication via hash-based blobId -- TEXT type stores both text and binary (base64) - -**Content Types:** -- **Text notes:** HTML content -- **Code notes:** Plain text source code -- **Binary notes:** Base64 encoded data -- **Protected notes:** Encrypted content - -## System Tables - -### Options Table - -```sql -CREATE TABLE options ( - name TEXT NOT NULL PRIMARY KEY, - value TEXT NOT NULL, - isSynced INTEGER DEFAULT 0 NOT NULL, - utcDateModified TEXT NOT NULL -); -``` - -**Key Options:** -- `documentId` - Unique installation ID -- `dbVersion` - Schema version -- `syncVersion` - Sync protocol version -- `passwordVerificationHash` - Password verification -- `encryptedDataKey` - Encryption key (encrypted) -- `theme` - UI theme -- Various feature flags and settings - -**Synced Options:** -- `isSynced = 1` - Synced across devices -- `isSynced = 0` - Local to this installation - -### Entity Changes Table - -```sql -CREATE TABLE entity_changes ( - id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, - entityName TEXT NOT NULL, - entityId TEXT NOT NULL, - hash TEXT NOT NULL, - isErased INT NOT NULL, - changeId TEXT NOT NULL, - componentId TEXT NOT NULL, - instanceId TEXT NOT NULL, - isSynced INTEGER NOT NULL, - utcDateChanged TEXT NOT NULL -); - --- Indexes -CREATE UNIQUE INDEX IDX_entityChanges_entityName_entityId - ON entity_changes (entityName, entityId); -CREATE INDEX IDX_entity_changes_changeId ON entity_changes (changeId); -``` - -**Purpose:** Track all entity modifications for synchronization - -**Entity Types:** -- `notes` -- `branches` -- `attributes` -- `revisions` -- `attachments` -- `options` -- `etapi_tokens` - -### Recent Notes Table - -```sql -CREATE TABLE recent_notes ( - noteId TEXT NOT NULL PRIMARY KEY, - notePath TEXT NOT NULL, - utcDateCreated TEXT NOT NULL -); -``` - -**Purpose:** Track recently accessed notes for quick access - -### Sessions Table - -```sql -CREATE TABLE sessions ( - sid TEXT PRIMARY KEY, - sess TEXT NOT NULL, - expired TEXT NOT NULL -); -``` - -**Purpose:** HTTP session storage for web interface - -### User Data Table - -```sql -CREATE TABLE user_data ( - tmpID INT PRIMARY KEY, - username TEXT, - email TEXT, - userIDEncryptedDataKey TEXT, - userIDVerificationHash TEXT, - salt TEXT, - derivedKey TEXT, - isSetup TEXT DEFAULT "false" -); -``` - -**Purpose:** Store user authentication credentials - -### ETAPI Tokens Table - -```sql -CREATE TABLE etapi_tokens ( - etapiTokenId TEXT PRIMARY KEY NOT NULL, - name TEXT NOT NULL, - tokenHash TEXT NOT NULL, - utcDateCreated TEXT NOT NULL, - utcDateModified TEXT NOT NULL, - isDeleted INT NOT NULL DEFAULT 0 -); -``` - -**Purpose:** API token authentication for external access - -## Data Relationships - -```mermaid -graph TB - Notes[Notes] - Branches[Branches] - Attributes[Attributes] - Attachments[Attachments] - Blobs[(Blobs)] - Revisions[Revisions] - - Notes --> Branches - Notes --> Attributes - Notes --> Attachments - Notes --> Blobs - Notes --> Revisions - - Branches --> Blobs - Attachments --> Blobs - Revisions --> Blobs - - style Notes fill:#e1f5ff - style Blobs fill:#ffe1e1 -``` - -**Relationships:** -- Notes ↔ Branches (many-to-many via noteId) -- Notes → Attributes (one-to-many) -- Notes → Blobs (one-to-one) -- Notes → Revisions (one-to-many) -- Notes → Attachments (one-to-many) -- Attachments → Blobs (one-to-one) -- Revisions → Blobs (one-to-one) - -## Database Access Patterns - -### Direct SQL Access - -**Location:** `apps/server/src/services/sql.ts` - -```typescript -// Execute query (returns rows) -const notes = sql.getRows('SELECT * FROM notes WHERE type = ?', ['text']) - -// Execute query (returns single row) -const note = sql.getRow('SELECT * FROM notes WHERE noteId = ?', [noteId]) - -// Execute statement (no return) -sql.execute('UPDATE notes SET title = ? WHERE noteId = ?', [title, noteId]) - -// Insert -sql.insert('notes', { - noteId: 'new-note-id', - title: 'New Note', - type: 'text', - // ... -}) - -// Transactions -sql.transactional(() => { - sql.execute('UPDATE ...') - sql.execute('INSERT ...') -}) -``` - -### Entity-Based Access (Recommended) - -**Via Becca Cache:** - -```typescript -// Get entity from cache -const note = becca.getNote(noteId) - -// Modify and save -note.title = 'Updated Title' -note.save() // Writes to database - -// Create new -const newNote = becca.createNote({ - parentNoteId: 'root', - title: 'New Note', - type: 'text', - content: 'Hello World' -}) - -// Delete -note.markAsDeleted() -``` - -## Database Migrations - -**Location:** `apps/server/src/migrations/` - -**Migration Files:** -- Format: `XXXX_migration_name.sql` or `XXXX_migration_name.js` -- Executed in numerical order -- Version tracked in `options.dbVersion` - -**SQL Migration Example:** -```sql --- 0280_add_new_column.sql -ALTER TABLE notes ADD COLUMN newField TEXT DEFAULT NULL; - -UPDATE options SET value = '280' WHERE name = 'dbVersion'; -``` - -**JavaScript Migration Example:** -```javascript -// 0285_complex_migration.js -module.exports = () => { - const notes = sql.getRows('SELECT * FROM notes WHERE type = ?', ['old-type']) - - for (const note of notes) { - sql.execute('UPDATE notes SET type = ? WHERE noteId = ?', - ['new-type', note.noteId]) - } -} -``` - -**Migration Process:** -1. Server checks `dbVersion` on startup -2. Compares with latest migration number -3. Executes pending migrations in order -4. Updates `dbVersion` after each -5. Restarts if migrations ran - -## Database Maintenance - -### Backup - -**Full Backup:** -```bash -# Copy database file -cp document.db document.db.backup - -# Or use Trilium's backup feature -# Settings → Backup -``` - -**Automatic Backups:** -- Daily backup (configurable) -- Stored in `backup/` directory -- Retention policy (keep last N backups) - -### Vacuum - -**Purpose:** Reclaim unused space, defragment - -```sql -VACUUM; -``` - -**When to vacuum:** -- After deleting many notes -- Database file size larger than expected -- Performance degradation - -### Integrity Check - -```sql -PRAGMA integrity_check; -``` - -**Result:** "ok" or list of errors - -### Consistency Checks - -**Built-in Consistency Checks:** - -Location: `apps/server/src/services/consistency_checks.ts` - -- Orphaned branches -- Missing parent notes -- Circular dependencies -- Invalid entity references -- Blob reference integrity - -**Run Checks:** -```typescript -// Via API -POST /api/consistency-check - -// Or from backend script -api.runConsistencyChecks() -``` - -## Performance Optimization - -### Indexes - -**Existing Indexes:** -- `notes.title` - Fast title searches -- `notes.type` - Filter by type -- `notes.dateCreated/Modified` - Time-based queries -- `branches.noteId_parentNoteId` - Tree navigation -- `attributes.name_value` - Attribute searches - -**Query Optimization:** -```sql --- Use indexed columns in WHERE clause -SELECT * FROM notes WHERE type = 'text' -- Uses index - --- Avoid functions on indexed columns -SELECT * FROM notes WHERE LOWER(title) = 'test' -- No index - --- Better -SELECT * FROM notes WHERE title = 'Test' -- Uses index -``` - -### Connection Settings - -```typescript -// apps/server/src/services/sql.ts -const db = new Database('document.db', { - // Enable WAL mode for better concurrency - verbose: console.log -}) - -db.pragma('journal_mode = WAL') -db.pragma('synchronous = NORMAL') -db.pragma('cache_size = -64000') // 64MB cache -db.pragma('temp_store = MEMORY') -``` - -**WAL Mode Benefits:** -- Better concurrency (readers don't block writers) -- Faster commits -- More robust - -### Query Performance - -**Use EXPLAIN QUERY PLAN:** -```sql -EXPLAIN QUERY PLAN -SELECT * FROM notes -WHERE type = 'text' - AND dateCreated > '2025-01-01' -``` - -**Analyze slow queries:** -- Check index usage -- Avoid SELECT * -- Use prepared statements -- Batch operations in transactions - -## Database Size Management - -**Typical Sizes:** -- 1,000 notes: ~5-10 MB -- 10,000 notes: ~50-100 MB -- 100,000 notes: ~500 MB - 1 GB - -**Size Reduction Strategies:** - -1. **Delete old revisions** -2. **Remove large attachments** -3. **Vacuum database** -4. **Compact blobs** -5. **Archive old notes** - -**Blob Deduplication:** -- Blobs identified by content hash -- Identical content shares one blob -- Automatic deduplication on insert - -## Security Considerations - -### Protected Notes Encryption - -**Encryption Process:** -```typescript -// Encrypt blob content -const encryptedContent = encrypt(content, dataKey) -blob.content = encryptedContent - -// Store encrypted -sql.insert('blobs', { blobId, content: encryptedContent }) -``` - -**Encryption Details:** -- Algorithm: AES-256-CBC -- Key derivation: PBKDF2 (10,000 iterations) -- Per-note encryption -- Master key encrypted with user password - -### SQL Injection Prevention - -**Always use parameterized queries:** -```typescript -// GOOD - Safe from SQL injection -sql.execute('SELECT * FROM notes WHERE title = ?', [userInput]) - -// BAD - Vulnerable to SQL injection -sql.execute(`SELECT * FROM notes WHERE title = '${userInput}'`) -``` - -### Database File Protection - -**File Permissions:** -- Owner read/write only -- No group/other access -- Located in user-specific directory - ---- - -**See Also:** -- [ARCHITECTURE.md](ARCHITECTURE.md) - Overall architecture -- [Database Schema Files](Developer%20Guide/Developer%20Guide/Development%20and%20architecture/Database/) -- [Migration Scripts](../apps/server/src/migrations/) diff --git a/docs/Developer Guide/!!!meta.json b/docs/Developer Guide/!!!meta.json index f19dbc16a..397bdcab4 100644 --- a/docs/Developer Guide/!!!meta.json +++ b/docs/Developer Guide/!!!meta.json @@ -172,64 +172,57 @@ "children": [ { "isClone": false, - "noteId": "2DJZgzpTJ078", + "noteId": "dsMq2EIOMOBU", "notePath": [ "jdjRLhLV3TtI", "MhwWMgxwDTZL", - "2DJZgzpTJ078" + "dsMq2EIOMOBU" ], - "title": "Client-server architecture", + "title": "Frontend", "notePosition": 10, "prefix": null, "isExpanded": false, "type": "text", "mime": "text/html", - "attributes": [], - "format": "markdown", - "attachments": [], - "dirFileName": "Client-server architecture", - "children": [ + "attributes": [ { - "isClone": false, - "noteId": "dsMq2EIOMOBU", - "notePath": [ - "jdjRLhLV3TtI", - "MhwWMgxwDTZL", - "2DJZgzpTJ078", - "dsMq2EIOMOBU" - ], - "title": "Frontend", - "notePosition": 10, - "prefix": null, - "isExpanded": false, - "type": "text", - "mime": "text/html", - "attributes": [], - "format": "markdown", - "dataFileName": "Frontend.md", - "attachments": [] - }, - { - "isClone": false, - "noteId": "tsswRlmHEnYW", - "notePath": [ - "jdjRLhLV3TtI", - "MhwWMgxwDTZL", - "2DJZgzpTJ078", - "tsswRlmHEnYW" - ], - "title": "Backend", - "notePosition": 20, - "prefix": null, - "isExpanded": false, - "type": "text", - "mime": "text/html", - "attributes": [], - "format": "markdown", - "dataFileName": "Backend.md", - "attachments": [] + "type": "label", + "name": "shareAlias", + "value": "frontend", + "isInheritable": false, + "position": 20 } - ] + ], + "format": "markdown", + "dataFileName": "Frontend.md", + "attachments": [] + }, + { + "isClone": false, + "noteId": "tsswRlmHEnYW", + "notePath": [ + "jdjRLhLV3TtI", + "MhwWMgxwDTZL", + "tsswRlmHEnYW" + ], + "title": "Backend", + "notePosition": 20, + "prefix": null, + "isExpanded": false, + "type": "text", + "mime": "text/html", + "attributes": [ + { + "type": "label", + "name": "shareAlias", + "value": "backend", + "isInheritable": false, + "position": 20 + } + ], + "format": "markdown", + "dataFileName": "Backend.md", + "attachments": [] }, { "isClone": false, @@ -240,7 +233,7 @@ "pRZhrVIGCbMu" ], "title": "Database", - "notePosition": 20, + "notePosition": 40, "prefix": null, "isExpanded": false, "type": "text", @@ -785,15 +778,23 @@ "MhwWMgxwDTZL", "Wxn82Em8B7U5" ], - "title": "API", - "notePosition": 30, + "title": "APIs", + "notePosition": 50, "prefix": null, "isExpanded": false, "type": "text", "mime": "text/html", - "attributes": [], + "attributes": [ + { + "type": "label", + "name": "shareAlias", + "value": "api", + "isInheritable": false, + "position": 20 + } + ], "format": "markdown", - "dataFileName": "API.md", + "dataFileName": "APIs.md", "attachments": [] }, { @@ -805,7 +806,7 @@ "Vk4zD1Iirarg" ], "title": "Arhitecture Decision Records", - "notePosition": 40, + "notePosition": 60, "prefix": null, "isExpanded": false, "type": "text", @@ -817,6 +818,13 @@ "value": "Jg7clqogFOyD", "isInheritable": false, "position": 20 + }, + { + "type": "label", + "name": "shareAlias", + "value": "adr", + "isInheritable": false, + "position": 30 } ], "format": "markdown", @@ -825,14 +833,14 @@ }, { "isClone": false, - "noteId": "QW1MB7RZB5Gf", + "noteId": "RHbKw3xiwk3S", "notePath": [ "jdjRLhLV3TtI", "MhwWMgxwDTZL", - "QW1MB7RZB5Gf" + "RHbKw3xiwk3S" ], - "title": "Security Architecture", - "notePosition": 50, + "title": "Security", + "notePosition": 80, "prefix": null, "isExpanded": false, "type": "text", @@ -841,13 +849,13 @@ { "type": "label", "name": "shareAlias", - "value": "security-architecture", + "value": "security", "isInheritable": false, "position": 20 } ], "format": "markdown", - "dataFileName": "Security Architecture.md", + "dataFileName": "Security.md", "attachments": [] } ] @@ -1153,6 +1161,13 @@ "value": "bx bx-rocket", "isInheritable": false, "position": 30 + }, + { + "type": "label", + "name": "shareAlias", + "value": "releasing", + "isInheritable": false, + "position": 40 } ], "format": "markdown", @@ -1181,6 +1196,13 @@ "value": "bx bxs-component", "isInheritable": false, "position": 20 + }, + { + "type": "label", + "name": "shareAlias", + "value": "dependencies", + "isInheritable": false, + "position": 30 } ], "format": "markdown", @@ -1527,6 +1549,13 @@ "value": "bx bx-microchip", "isInheritable": false, "position": 20 + }, + { + "type": "label", + "name": "shareAlias", + "value": "cache", + "isInheritable": false, + "position": 30 } ], "format": "markdown", @@ -2001,7 +2030,15 @@ "isExpanded": false, "type": "text", "mime": "text/html", - "attributes": [], + "attributes": [ + { + "type": "label", + "name": "shareAlias", + "value": "note-types", + "isInheritable": false, + "position": 20 + } + ], "format": "markdown", "attachments": [], "dirFileName": "Note Types", @@ -2547,6 +2584,7 @@ } ], "format": "markdown", + "dataFileName": "Synchronisation.md", "attachments": [], "dirFileName": "Synchronisation", "children": [ @@ -2794,7 +2832,15 @@ "isExpanded": false, "type": "text", "mime": "text/html", - "attributes": [], + "attributes": [ + { + "type": "label", + "name": "shareAlias", + "value": "unit-tests", + "isInheritable": false, + "position": 20 + } + ], "format": "markdown", "dataFileName": "Unit tests.md", "attachments": [] diff --git a/docs/Developer Guide/Developer Guide/Architecture.md b/docs/Developer Guide/Developer Guide/Architecture.md index 48a91fcde..66cb82eea 100644 --- a/docs/Developer Guide/Developer Guide/Architecture.md +++ b/docs/Developer Guide/Developer Guide/Architecture.md @@ -118,4 +118,85 @@ desktop → client → commons server → client → commons client → ckeditor5, codemirror, highlightjs ckeditor5 → ckeditor5-* plugins -``` \ No newline at end of file +``` + +## Security summary + +### Encryption System + +**Per-Note Encryption:** + +* Notes can be individually protected +* AES-128-CBC encryption for encrypted notes. +* Separate protected session management + +**Protected Session:** + +* Time-limited access to protected notes +* Automatic timeout +* Re-authentication required +* Frontend: `protected_session.ts` +* Backend: `protected_session.ts` + +### Authentication + +**Password Auth:** + +* PBKDF2 key derivation +* Salt per installation +* Hash verification + +**OpenID Connect:** + +* External identity provider support +* OAuth 2.0 flow +* Configurable providers + +**TOTP (2FA):** + +* Time-based one-time passwords +* QR code setup +* Backup codes + +### Authorization + +**Single-User Model:** + +* Desktop: single user (owner) +* Server: single user per installation + +**Share Notes:** + +* Public access without authentication +* Separate Shaca cache +* Read-only access + +### CSRF Protection + +**CSRF Tokens:** + +* Required for state-changing operations +* Token in header or cookie +* Validation middleware + +### Input Sanitization + +**XSS Prevention:** + +* DOMPurify for HTML sanitization +* CKEditor content filtering +* CSP headers + +**SQL Injection:** + +* Parameterized queries only +* Better-sqlite3 prepared statements +* No string concatenation in SQL + +### Dependency Security + +**Vulnerability Scanning:** + +* Renovate bot for updates +* npm audit integration +* Override vulnerable sub-dependencies \ No newline at end of file diff --git a/docs/Developer Guide/Developer Guide/Architecture/API.md b/docs/Developer Guide/Developer Guide/Architecture/APIs.md similarity index 99% rename from docs/Developer Guide/Developer Guide/Architecture/API.md rename to docs/Developer Guide/Developer Guide/Architecture/APIs.md index 92f6fb813..cdd954a61 100644 --- a/docs/Developer Guide/Developer Guide/Architecture/API.md +++ b/docs/Developer Guide/Developer Guide/Architecture/APIs.md @@ -1,4 +1,4 @@ -# API +# APIs ### Internal API **REST Endpoints** (`/api/*`) diff --git a/docs/Developer Guide/Developer Guide/Architecture/Client-server architecture/Backend.md b/docs/Developer Guide/Developer Guide/Architecture/Backend.md similarity index 100% rename from docs/Developer Guide/Developer Guide/Architecture/Client-server architecture/Backend.md rename to docs/Developer Guide/Developer Guide/Architecture/Backend.md diff --git a/docs/Developer Guide/Developer Guide/Architecture/Database.md b/docs/Developer Guide/Developer Guide/Architecture/Database.md index bb80ccdb1..b8836b56d 100644 --- a/docs/Developer Guide/Developer Guide/Architecture/Database.md +++ b/docs/Developer Guide/Developer Guide/Architecture/Database.md @@ -1,5 +1,5 @@ # Database -Trilium uses **SQLite** as its database engine, managed via `better-sqlite3`. +Trilium uses **SQLite** (via `better-sqlite3`) as its embedded database engine, providing a reliable, file-based storage system that requires no separate database server. The database stores all notes, their relationships, metadata, and configuration. Schema location: `apps/server/src/assets/db/schema.sql` diff --git a/docs/Developer Guide/Developer Guide/Architecture/Client-server architecture/Frontend.md b/docs/Developer Guide/Developer Guide/Architecture/Frontend.md similarity index 100% rename from docs/Developer Guide/Developer Guide/Architecture/Client-server architecture/Frontend.md rename to docs/Developer Guide/Developer Guide/Architecture/Frontend.md diff --git a/docs/Developer Guide/Developer Guide/Architecture/Security Architecture.md b/docs/Developer Guide/Developer Guide/Architecture/Security Architecture.md deleted file mode 100644 index ab0c88c06..000000000 --- a/docs/Developer Guide/Developer Guide/Architecture/Security Architecture.md +++ /dev/null @@ -1,79 +0,0 @@ -# Security Architecture -### Encryption System - -**Per-Note Encryption:** - -* Notes can be individually protected -* AES-128-CBC encryption for encrypted notes. -* Separate protected session management - -**Protected Session:** - -* Time-limited access to protected notes -* Automatic timeout -* Re-authentication required -* Frontend: `protected_session.ts` -* Backend: `protected_session.ts` - -### Authentication - -**Password Auth:** - -* PBKDF2 key derivation -* Salt per installation -* Hash verification - -**OpenID Connect:** - -* External identity provider support -* OAuth 2.0 flow -* Configurable providers - -**TOTP (2FA):** - -* Time-based one-time passwords -* QR code setup -* Backup codes - -### Authorization - -**Single-User Model:** - -* Desktop: single user (owner) -* Server: single user per installation - -**Share Notes:** - -* Public access without authentication -* Separate Shaca cache -* Read-only access - -### CSRF Protection - -**CSRF Tokens:** - -* Required for state-changing operations -* Token in header or cookie -* Validation middleware - -### Input Sanitization - -**XSS Prevention:** - -* DOMPurify for HTML sanitization -* CKEditor content filtering -* CSP headers - -**SQL Injection:** - -* Parameterized queries only -* Better-sqlite3 prepared statements -* No string concatenation in SQL - -### Dependency Security - -**Vulnerability Scanning:** - -* Renovate bot for updates -* npm audit integration -* Override vulnerable sub-dependencies \ No newline at end of file diff --git a/docs/Developer Guide/Developer Guide/Architecture/Security.md b/docs/Developer Guide/Developer Guide/Architecture/Security.md new file mode 100644 index 000000000..61fdc7377 --- /dev/null +++ b/docs/Developer Guide/Developer Guide/Architecture/Security.md @@ -0,0 +1,464 @@ +# Security +Trilium implements a **defense-in-depth security model** with multiple layers of protection for user data. The security architecture covers authentication, authorization, encryption, input sanitization, and secure communication. + +## Security Principles + +1. **Data Privacy**: User data is protected at rest and in transit +2. **Encryption**: Per-note encryption for sensitive content +3. **Authentication**: Multiple authentication methods supported +4. **Authorization**: Single-user model with granular protected sessions +5. **Input Validation**: All user input sanitized +6. **Secure Defaults**: Security features enabled by default +7. **Transparency**: Open source allows security audits + +## Threat Model + +### Threats Considered + +1. **Unauthorized Access** + * Physical access to device + * Network eavesdropping + * Stolen credentials + * Session hijacking +2. **Data Exfiltration** + * Malicious scripts + * XSS attacks + * SQL injection + * CSRF attacks +3. **Data Corruption** + * Malicious modifications + * Database tampering + * Sync conflicts +4. **Privacy Leaks** + * Unencrypted backups + * Search indexing + * Temporary files + * Memory dumps + +### Out of Scope + +* Nation-state attackers +* Zero-day vulnerabilities in dependencies +* Hardware vulnerabilities (Spectre, Meltdown) +* Physical access with unlimited time +* Quantum computing attacks + +## Authentication + +### Password Authentication + +**Implementation:** `apps/server/src/services/password.ts` + +### TOTP (Two-Factor Authentication) + +**Implementation:** `apps/server/src/routes/api/login.ts` + +### OpenID Connect + +**Implementation:** `apps/server/src/routes/api/login.ts` + +**Supported Providers:** + +* Any OpenID Connect compatible provider +* Google, GitHub, Auth0, etc. + +**Flow:** + +```typescript +// 1. Redirect to provider +GET /api/login/openid + +// 2. Provider redirects back with code +GET /api/login/openid/callback?code=... + +// 3. Exchange code for tokens +const tokens = await openidClient.callback(redirectUri, req.query) + +// 4. Verify ID token +const claims = tokens.claims() + +// 5. Create session +req.session.loggedIn = true +``` + +### Session Management + +**Session Storage:** SQLite database (sessions table) + +**Session Configuration:** + +```typescript +app.use(session({ + secret: sessionSecret, + resave: false, + saveUninitialized: false, + rolling: true, + cookie: { + maxAge: 7 * 24 * 60 * 60 * 1000, // 7 days + httpOnly: true, + secure: isHttps, + sameSite: 'lax' + }, + store: new SqliteStore({ + db: db, + table: 'sessions' + }) +})) +``` + +**Session Invalidation:** + +* Automatic timeout after inactivity +* Manual logout clears session +* Server restart invalidates all sessions (optional) + +## Authorization + +### Single-User Model + +**Desktop:** + +* Single user (owner of device) +* No multi-user support +* Full access to all notes + +**Server:** + +* Single user per installation +* Authentication required for all operations +* No user roles or permissions + +### Protected Sessions + +**Purpose:** Temporary access to encrypted (protected) notes + +**Implementation:** `apps/server/src/services/protected_session.ts` + +**Workflow:** + +```typescript +// 1. User enters password for protected notes +POST /api/protected-session/enter +Body: { password: "protected-password" } + +// 2. Derive encryption key +const protectedDataKey = deriveKey(password) + +// 3. Verify password (decrypt known encrypted value) +const decrypted = decrypt(testValue, protectedDataKey) +if (decrypted === expectedValue) { + // 4. Store in memory (not in session) + protectedSessionHolder.setProtectedDataKey(protectedDataKey) + + // 5. Set timeout + setTimeout(() => { + protectedSessionHolder.clearProtectedDataKey() + }, timeout) +} +``` + +**Protected Session Timeout:** + +* Default: 10 minutes (configurable) +* Extends on activity +* Cleared on browser close +* Separate from main session + +### API Authorization + +**Internal API:** + +* Requires authenticated session +* CSRF token validation +* Same-origin policy + +**ETAPI (External API):** + +* Token-based authentication +* No session required +* Rate limiting + +## Encryption + +### Note Encryption + +**Encryption Algorithm:** AES-256-CBC + +**Key Hierarchy:** + +``` +User Password + ↓ (scrypt) +Data Key (for protected notes) + ↓ (AES-128) +Protected Note Content +``` + +**Protected Note Metadata:** + +* Content IS encrypted +* Type and MIME are NOT encrypted +* Attributes are NOT encrypted + +### Data Key Management + +**Key Rotation:** + +* Not currently supported +* Requires re-encrypting all protected notes + +### Transport Encryption + +**HTTPS:** + +* Recommended for server installations +* TLS 1.2+ only +* Strong cipher suites preferred +* Certificate validation enabled + +**Desktop:** + +* Local communication (no network) +* No HTTPS required + +### Backup Encryption + +**Database Backups:** + +* Protected notes remain encrypted in backup +* Backup file should be protected separately +* Consider encrypting backup storage location + +## Input Sanitization + +### XSS Prevention + +* **HTML Sanitization** +* **CKEditor Configuration:** + + ``` + // apps/client/src/widgets/type_widgets/text_type_widget.ts + ClassicEditor.create(element, { + // Restrict allowed content + htmlSupport: { + allow: [ + { name: /./, attributes: true, classes: true, styles: true } + ], + disallow: [ + { name: 'script' }, + { name: 'iframe', attributes: /^(?!src$).*/ } + ] + } + }) + ``` +* Content Security Policy + +### SQL Injection Prevention + +**Parameterized Queries:** + +```typescript +const notes = sql.getRows( + 'SELECT * FROM notes WHERE title = ?', + [userInput] +) +``` + +**ORM Usage:** + +```typescript +// Entity-based access prevents SQL injection +const note = becca.getNote(noteId) +note.title = userInput // Sanitized by entity +note.save() // Parameterized query +``` + +### CSRF Prevention + +**CSRF Token Validation:** + +Location: `apps/server/src/routes/csrf_protection.ts` + +Stateless CSRF using [Double Submit Cookie Pattern](https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html#double-submit-cookie) via [`csrf-csrf`](https://github.com/Psifi-Solutions/csrf-csrf). + +### File Upload Validation + +**Validation:** + +```typescript +// Validate file size +const maxSize = 100 * 1024 * 1024 // 100 MB +if (file.size > maxSize) { + throw new Error('File too large') +} +``` + +## Network Security + +### HTTPS Configuration + +**Certificate Validation:** + +* Require valid certificates in production +* Self-signed certificates allowed for development +* Certificate pinning not implemented + +### Rate Limiting + +**Login Rate Limiting:** + +```typescript +const loginLimiter = rateLimit({ + windowMs: 15 * 60 * 1000, + max: 10, // 10 failed attempts + skipSuccessfulRequests: true +}) + +app.post('/api/login/password', loginLimiter, loginHandler) +``` + +## Data Security + +### Secure Data Deletion + +**Soft Delete:** + +```typescript +// Mark as deleted (sync first) +note.isDeleted = 1 +note.deleteId = generateUUID() +note.save() + +// Entity change tracked for sync +addEntityChange('notes', noteId, note) +``` + +**Hard Delete (Erase):** + +```typescript +// After sync completed +sql.execute('DELETE FROM notes WHERE noteId = ?', [noteId]) +sql.execute('DELETE FROM branches WHERE noteId = ?', [noteId]) +sql.execute('DELETE FROM attributes WHERE noteId = ?', [noteId]) + +// Mark entity change as erased +sql.execute('UPDATE entity_changes SET isErased = 1 WHERE entityId = ?', [noteId]) +``` + +**Blob Cleanup:** + +```typescript +// Find orphaned blobs (not referenced by any note/revision/attachment) +const orphanedBlobs = sql.getRows(` + SELECT blobId FROM blobs + WHERE blobId NOT IN (SELECT blobId FROM notes WHERE blobId IS NOT NULL) + AND blobId NOT IN (SELECT blobId FROM revisions WHERE blobId IS NOT NULL) + AND blobId NOT IN (SELECT blobId FROM attachments WHERE blobId IS NOT NULL) +`) + +// Delete orphaned blobs +for (const blob of orphanedBlobs) { + sql.execute('DELETE FROM blobs WHERE blobId = ?', [blob.blobId]) +} +``` + +### Memory Security + +**Protected Data in Memory:** + +* Protected data keys stored in memory only +* Cleared on timeout +* Not written to disk +* Not in session storage + +## Dependency Security + +### Vulnerability Scanning + +**Tools:** + +* Renovate bot - Automatic dependency updates +* `pnpm audit` - Check for known vulnerabilities +* GitHub Dependabot alerts + +**Process:** + +```sh +# Check for vulnerabilities +npm audit + +# Fix automatically +npm audit fix + +# Manual review for breaking changes +npm audit fix --force +``` + +### Dependency Pinning + +**package.json:** + +``` +{ + "dependencies": { + "express": "4.18.2", // Exact version + "better-sqlite3": "^9.2.2" // Compatible versions + } +} +``` + +**pnpm Overrides:** + +``` +{ + "pnpm": { + "overrides": { + "lodash@<4.17.21": ">=4.17.21", // Force minimum version + "axios@<0.21.2": ">=0.21.2" + } + } +} +``` + +### Patch Management + +**pnpm Patches:** + +```sh +# Create patch +pnpm patch @ckeditor/ckeditor5 + +# Edit files in temporary directory +# ... + +# Generate patch file +pnpm patch-commit /tmp/ckeditor5-patch + +# Patch applied automatically on install +``` + +## Security Auditing + +### Logs + +**Security Events Logged:** + +* Login attempts (success/failure) +* Protected session access +* Password changes +* ETAPI token usage +* Failed CSRF validations + +**Log Location:** + +* Desktop: Console output +* Server: Log files or stdout + +### Monitoring + +**Metrics to Monitor:** + +* Failed login attempts +* API error rates +* Unusual database changes +* Large exports/imports \ No newline at end of file diff --git a/docs/SYNCHRONIZATION.md b/docs/Developer Guide/Developer Guide/Concepts/Synchronisation.md similarity index 52% rename from docs/SYNCHRONIZATION.md rename to docs/Developer Guide/Developer Guide/Concepts/Synchronisation.md index 0945e533a..f97bd3c4e 100644 --- a/docs/SYNCHRONIZATION.md +++ b/docs/Developer Guide/Developer Guide/Concepts/Synchronisation.md @@ -1,16 +1,11 @@ -# Trilium Synchronization Architecture +# Synchronisation +Trilium implements a **bidirectional synchronization system** that allows users to sync their note databases across multiple devices (desktop clients and server instances). The sync protocol is designed to handle: -> **Related:** [ARCHITECTURE.md](ARCHITECTURE.md) | [User Guide: Synchronization](https://triliumnext.github.io/Docs/Wiki/synchronization) - -## Overview - -Trilium implements a sophisticated **bidirectional synchronization system** that allows users to sync their note databases across multiple devices (desktop clients and server instances). The sync protocol is designed to handle: - -- Concurrent modifications across devices -- Conflict resolution -- Partial sync (only changed entities) -- Protected note synchronization -- Efficient bandwidth usage +* Concurrent modifications across devices +* Simple conflict resolution (without “merge conflict” indication). +* Partial sync (only changed entities) +* Protected note synchronization +* Efficient bandwidth usage ## Sync Architecture @@ -35,7 +30,7 @@ graph TB Every modification to any entity (note, branch, attribute, etc.) creates an **entity change** record: -```sql +``` entity_changes ( id, -- Auto-increment ID entityName, -- 'notes', 'branches', 'attributes', etc. @@ -43,7 +38,7 @@ entity_changes ( hash, -- Content hash for integrity isErased, -- If entity was erased (deleted permanently) changeId, -- Unique change identifier - componentId, -- Installation identifier + componentId, -- Unique component/widget identifier instanceId, -- Process instance identifier isSynced, -- Whether synced to server utcDateChanged -- When change occurred @@ -51,17 +46,19 @@ entity_changes ( ``` **Key Properties:** -- **changeId**: Globally unique identifier (UUID) for the change -- **componentId**: Unique per Trilium installation (persists across restarts) -- **instanceId**: Unique per process (changes on restart) -- **hash**: SHA-256 hash of entity data for integrity verification + +* **changeId**: Globally unique identifier (UUID) for the change +* **componentId**: Unique identifier of the component/widget that generated to change (can be used to avoid refreshing the widget being edited). +* **instanceId**: Unique per process (changes on restart) +* **hash**: SHA-256 hash of entity data for integrity verification ### Sync Versions Each Trilium installation tracks: -- **Local sync version**: Highest change ID seen locally -- **Server sync version**: Highest change ID on server -- **Entity versions**: Last sync version for each entity type + +* **Local sync version**: Highest change ID seen locally +* **Server sync version**: Highest change ID on server +* **Entity versions**: Last sync version for each entity type ### Change Tracking @@ -87,11 +84,12 @@ function addEntityChange(entityName, entityId, entity) { ``` **Entity modification triggers:** -- Note content update -- Note metadata change -- Branch creation/deletion/reorder -- Attribute addition/removal -- Options modification + +* Note content update +* Note metadata change +* Branch creation/deletion/reorder +* Attribute addition/removal +* Options modification ## Sync Protocol @@ -122,9 +120,9 @@ Response: **Step 3: Decision** -- If `entityChanges > 0`: Pull changes from server -- If `outstandingPushCount > 0`: Push changes to server -- Both can happen in sequence +* If `entityChanges > 0`: Pull changes from server +* If `outstandingPushCount > 0`: Push changes to server +* Both can happen in sequence ### Pull Sync (Server → Client) @@ -159,10 +157,10 @@ Response: **Client Processing:** -1. Apply entity changes to local database -2. Update Froca cache -3. Update local sync version -4. Trigger UI refresh +1. Apply entity changes to local database +2. Update Froca cache +3. Update local sync version +4. Trigger UI refresh ### Push Sync (Client → Server) @@ -191,12 +189,12 @@ POST /api/sync/push **Server Processing:** -1. Validate changes -2. Check for conflicts -3. Apply changes to database -4. Update Becca cache -5. Mark as synced -6. Broadcast to other connected clients via WebSocket +1. Validate changes +2. Check for conflicts +3. Apply changes to database +4. Update Becca cache +5. Mark as synced +6. Broadcast to other connected clients via WebSocket **Conflict Detection:** @@ -215,21 +213,25 @@ if (serverLastModified > clientSyncVersion) { ### Conflict Types -**1. Content Conflict** -- Both client and server modified same note content -- **Resolution**: Last-write-wins based on `utcDateModified` +**1\. Content Conflict** -**2. Structure Conflict** -- Branch moved/deleted on one side, modified on other -- **Resolution**: Tombstone records, reconciliation +* Both client and server modified same note content +* **Resolution**: Last-write-wins based on `utcDateModified` -**3. Attribute Conflict** -- Same attribute modified differently -- **Resolution**: Last-write-wins +**2\. Structure Conflict** + +* Branch moved/deleted on one side, modified on other +* **Resolution**: Tombstone records, reconciliation + +**3\. Attribute Conflict** + +* Same attribute modified differently +* **Resolution**: Last-write-wins ### Conflict Resolution Strategy **Last-Write-Wins:** + ```typescript if (clientEntity.utcDateModified > serverEntity.utcDateModified) { // Client wins, apply client changes @@ -241,9 +243,10 @@ if (clientEntity.utcDateModified > serverEntity.utcDateModified) { ``` **Tombstone Records:** -- Deleted entities leave tombstone in `entity_changes` -- Prevents re-sync of deleted items -- `isErased = 1` for permanent deletions + +* Deleted entities leave tombstone in `entity_changes` +* Prevents re-sync of deleted items +* `isErased = 1` for permanent deletions ### Protected Notes Sync @@ -251,43 +254,26 @@ if (clientEntity.utcDateModified > serverEntity.utcDateModified) { **Solution:** -1. **Protected session required**: User must unlock protected notes -2. **Encrypted sync**: Content synced in encrypted form -3. **Hash verification**: Integrity checked without decryption -4. **Lazy decryption**: Only decrypt when accessed - -**Sync Flow:** - -```typescript -// Client side -if (note.isProtected && !protectedSessionHolder.isProtectedSessionAvailable()) { - // Skip protected notes if session not active - continue -} - -// Server side -if (note.isProtected) { - // Sync encrypted blob - // Don't decrypt for sync - syncEncryptedBlob(note.blobId) -} -``` +1. **Encrypted sync**: Content synced in encrypted form +2. **Hash verification**: Integrity checked without decryption +3. **Lazy decryption**: Only decrypt when accessed ## Sync States ### Connection States -- **Connected**: WebSocket connection active -- **Disconnected**: No connection to sync server -- **Syncing**: Actively transferring data -- **Conflict**: Sync paused due to conflict +* **Connected**: WebSocket connection active +* **Disconnected**: No connection to sync server +* **Syncing**: Actively transferring data +* **Conflict**: Sync paused due to conflict ### Entity Sync States Each entity can be in: -- **Synced**: In sync with server -- **Pending**: Local changes not yet pushed -- **Conflict**: Conflicting changes detected + +* **Synced**: In sync with server +* **Pending**: Local changes not yet pushed +* **Conflict**: Conflicting changes detected ### UI Indicators @@ -299,8 +285,6 @@ class SyncStatusWidget { showIcon('synced') } else if (isSyncing) { showIcon('syncing-spinner') - } else if (hasConflicts) { - showIcon('conflict-warning') } else { showIcon('not-synced') } @@ -314,7 +298,7 @@ class SyncStatusWidget { Only entities changed since last sync are transferred: -```sql +``` SELECT * FROM entity_changes WHERE id > :lastSyncedChangeId ORDER BY id ASC @@ -357,26 +341,12 @@ res.send(gzip(syncData)) ### Network Errors -**Retry Strategy:** -```typescript -const RETRY_DELAYS = [1000, 2000, 5000, 10000, 30000] - -async function syncWithRetry(attempt = 0) { - try { - await performSync() - } catch (error) { - if (attempt < RETRY_DELAYS.length) { - setTimeout(() => { - syncWithRetry(attempt + 1) - }, RETRY_DELAYS[attempt]) - } - } -} -``` +Reported to the user and the sync will be retried after the interval passes. ### Sync Integrity Checks **Hash Verification:** + ```typescript // Verify entity hash matches const calculatedHash = calculateHash(entity) @@ -388,16 +358,18 @@ if (calculatedHash !== receivedHash) { ``` **Consistency Checks:** -- Orphaned branches detection -- Missing parent notes -- Invalid entity references -- Circular dependencies + +* Orphaned branches detection +* Missing parent notes +* Invalid entity references +* Circular dependencies ## Sync Server Configuration ### Server Setup **Required Options:** + ```javascript { "syncServerHost": "https://sync.example.com", @@ -407,59 +379,26 @@ if (calculatedHash !== receivedHash) { ``` **Authentication:** -- Username/password or -- Sync token (generated on server) -### Client Setup - -**Desktop Client:** -```javascript -// Settings → Sync -{ - "syncServerHost": "https://sync.example.com", - "username": "user@example.com", - "password": "********" -} -``` - -**Test Connection:** -```typescript -POST /api/sync/test -Response: { "success": true } -``` +* Username/password or +* Sync token (generated on server) ## Sync API Endpoints Located at: `apps/server/src/routes/api/sync.ts` -**Endpoints:** - -- `POST /api/sync/check` - Check sync status -- `POST /api/sync/pull` - Pull changes from server -- `POST /api/sync/push` - Push changes to server -- `POST /api/sync/finished` - Mark sync complete -- `POST /api/sync/test` - Test connection -- `GET /api/sync/stats` - Sync statistics - ## WebSocket Sync Updates Real-time sync via WebSocket: ```typescript // Server broadcasts change to all connected clients -ws.broadcast('entity-change', { - entityName: 'notes', - entityId: 'abc123', - changeId: 'change-uuid', - sourceId: 'originating-component-id' +ws.broadcast('frontend-update', { + lastSyncedPush, + entityChanges }) -// Client receives and applies -ws.on('entity-change', (data) => { - if (data.sourceId !== myComponentId) { - froca.processEntityChange(data) - } -}) +// Client receives and processed the information. ``` ## Sync Scheduling @@ -467,107 +406,79 @@ ws.on('entity-change', (data) => { ### Automatic Sync **Desktop:** -- Sync on startup -- Periodic sync (configurable interval, default: 60s) -- Sync before shutdown + +* Sync on startup +* Periodic sync (configurable interval, default: 60s) **Server:** -- Sync on entity modification -- WebSocket push to connected clients + +* Sync on entity modification +* WebSocket push to connected clients ### Manual Sync User can trigger: -- Full sync -- Sync now -- Sync specific subtree + +* Full sync +* Sync now +* Sync specific subtree ## Troubleshooting ### Common Issues **Sync stuck:** -```sql + +``` -- Reset sync state UPDATE entity_changes SET isSynced = 0; DELETE FROM options WHERE name LIKE 'sync%'; ``` **Hash mismatch:** -- Data corruption detected -- Re-sync from backup -- Check database integrity + +* Data corruption detected +* Re-sync from backup +* Check database integrity **Conflict loop:** -- Manual intervention required -- Export conflicting notes -- Choose winning version -- Re-sync -### Sync Diagnostics - -**Check sync status:** -```typescript -GET /api/sync/stats -Response: { - "unsyncedChanges": 0, - "lastSyncDate": "2025-11-02T12:00:00Z", - "syncVersion": 12890 -} -``` - -**Entity change log:** -```sql -SELECT * FROM entity_changes -WHERE isSynced = 0 -ORDER BY id DESC; -``` +* Manual intervention required +* Export conflicting notes +* Choose winning version +* Re-sync ## Security Considerations ### Encrypted Sync -- Protected notes synced encrypted -- No plain text over network -- Server cannot read protected content +* Protected notes synced encrypted +* No plain text over network +* Server cannot read protected content ### Authentication -- Username/password over HTTPS only -- Sync tokens for token-based auth -- Session cookies with CSRF protection +* Username/password over HTTPS only +* Sync tokens for token-based auth +* Session cookies with CSRF protection ### Authorization -- Users can only sync their own data -- No cross-user sync support -- Sync server validates ownership +* Users can only sync their own data +* No cross-user sync support +* Sync server validates ownership ## Performance Metrics **Typical Sync Performance:** -- 1000 changes: ~2-5 seconds -- 10000 changes: ~20-50 seconds -- Initial full sync (100k notes): ~5-10 minutes + +* 1000 changes: ~2-5 seconds +* 10000 changes: ~20-50 seconds +* Initial full sync (100k notes): ~5-10 minutes **Factors:** -- Network latency -- Database size -- Number of protected notes -- Attachment sizes -## Future Improvements - -**Planned Enhancements:** -- Differential sync (binary diff) -- Peer-to-peer sync (no central server) -- Multi-server sync -- Partial sync (subtree only) -- Sync over Tor/I2P - ---- - -**See Also:** -- [ARCHITECTURE.md](ARCHITECTURE.md) - Overall architecture -- [Sync User Guide](https://triliumnext.github.io/Docs/Wiki/synchronization) -- [Sync API Source](../apps/server/src/routes/api/sync.ts) +* Network latency +* Database size +* Number of protected notes +* Attachment sizes \ No newline at end of file diff --git a/docs/QUICK_REFERENCE.md b/docs/QUICK_REFERENCE.md deleted file mode 100644 index 396cc0dd6..000000000 --- a/docs/QUICK_REFERENCE.md +++ /dev/null @@ -1,155 +0,0 @@ -# Trilium Technical Documentation - Quick Reference - -> **Start here:** [TECHNICAL_DOCUMENTATION.md](TECHNICAL_DOCUMENTATION.md) - Complete index of all documentation - -## 📖 Documentation Files - -| Document | Description | Size | Lines | -|----------|-------------|------|-------| -| [TECHNICAL_DOCUMENTATION.md](TECHNICAL_DOCUMENTATION.md) | Main index and navigation hub | 13KB | 423 | -| [ARCHITECTURE.md](ARCHITECTURE.md) | Complete system architecture | 30KB | 1,016 | -| [DATABASE.md](DATABASE.md) | Database schema and operations | 19KB | 736 | -| [SYNCHRONIZATION.md](SYNCHRONIZATION.md) | Sync protocol and implementation | 14KB | 583 | -| [SCRIPTING.md](SCRIPTING.md) | User scripting system guide | 17KB | 734 | -| [SECURITY_ARCHITECTURE.md](SECURITY_ARCHITECTURE.md) | Security implementation details | 19KB | 834 | - -**Total:** 112KB of comprehensive documentation across 4,326 lines! - -## 🎯 Quick Access by Role - -### 👤 End Users -- **Getting Started:** [User Guide](User%20Guide/User%20Guide/) -- **Scripting:** [SCRIPTING.md](SCRIPTING.md) -- **Sync Setup:** [SYNCHRONIZATION.md](SYNCHRONIZATION.md) - -### 💻 Developers -- **Architecture:** [ARCHITECTURE.md](ARCHITECTURE.md) -- **Development Setup:** [Developer Guide](Developer%20Guide/Developer%20Guide/Environment%20Setup.md) -- **Database:** [DATABASE.md](DATABASE.md) - -### 🔒 Security Auditors -- **Security:** [SECURITY_ARCHITECTURE.md](SECURITY_ARCHITECTURE.md) -- **Encryption:** [SECURITY_ARCHITECTURE.md#encryption](SECURITY_ARCHITECTURE.md#encryption) -- **Auth:** [SECURITY_ARCHITECTURE.md#authentication](SECURITY_ARCHITECTURE.md#authentication) - -### 🏗️ System Architects -- **Overall Design:** [ARCHITECTURE.md](ARCHITECTURE.md) -- **Cache System:** [ARCHITECTURE.md#three-layer-cache-system](ARCHITECTURE.md#three-layer-cache-system) -- **Entity Model:** [ARCHITECTURE.md#entity-system](ARCHITECTURE.md#entity-system) - -### 🔧 DevOps Engineers -- **Server Installation:** [User Guide - Server Installation](User%20Guide/User%20Guide/Installation%20&%20Setup/Server%20Installation.md) -- **Docker:** [Developer Guide - Docker](Developer%20Guide/Developer%20Guide/Development%20and%20architecture/Docker.md) -- **Sync Server:** [SYNCHRONIZATION.md#sync-server-configuration](SYNCHRONIZATION.md#sync-server-configuration) - -### 📊 Database Administrators -- **Schema:** [DATABASE.md#database-schema](DATABASE.md#database-schema) -- **Maintenance:** [DATABASE.md#database-maintenance](DATABASE.md#database-maintenance) -- **Performance:** [DATABASE.md#performance-optimization](DATABASE.md#performance-optimization) - -## 🔍 Quick Topic Finder - -### Core Concepts -- **Becca Cache:** [ARCHITECTURE.md#1-becca-backend-cache](ARCHITECTURE.md#1-becca-backend-cache) -- **Froca Cache:** [ARCHITECTURE.md#2-froca-frontend-cache](ARCHITECTURE.md#2-froca-frontend-cache) -- **Entity System:** [ARCHITECTURE.md#entity-system](ARCHITECTURE.md#entity-system) -- **Widget System:** [ARCHITECTURE.md#widget-based-ui](ARCHITECTURE.md#widget-based-ui) - -### Database -- **Schema Overview:** [DATABASE.md#schema-overview](DATABASE.md#schema-overview) -- **Notes Table:** [DATABASE.md#notes-table](DATABASE.md#notes-table) -- **Branches Table:** [DATABASE.md#branches-table](DATABASE.md#branches-table) -- **Migrations:** [DATABASE.md#database-migrations](DATABASE.md#database-migrations) - -### Synchronization -- **Sync Protocol:** [SYNCHRONIZATION.md#sync-protocol](SYNCHRONIZATION.md#sync-protocol) -- **Conflict Resolution:** [SYNCHRONIZATION.md#conflict-resolution](SYNCHRONIZATION.md#conflict-resolution) -- **Entity Changes:** [SYNCHRONIZATION.md#entity-changes](SYNCHRONIZATION.md#entity-changes) - -### Scripting -- **Frontend Scripts:** [SCRIPTING.md#frontend-scripts](SCRIPTING.md#frontend-scripts) -- **Backend Scripts:** [SCRIPTING.md#backend-scripts](SCRIPTING.md#backend-scripts) -- **Script Examples:** [SCRIPTING.md#script-examples](SCRIPTING.md#script-examples) -- **API Reference:** [SCRIPTING.md#script-api](SCRIPTING.md#script-api) - -### Security -- **Authentication:** [SECURITY_ARCHITECTURE.md#authentication](SECURITY_ARCHITECTURE.md#authentication) -- **Encryption:** [SECURITY_ARCHITECTURE.md#encryption](SECURITY_ARCHITECTURE.md#encryption) -- **Input Sanitization:** [SECURITY_ARCHITECTURE.md#input-sanitization](SECURITY_ARCHITECTURE.md#input-sanitization) -- **Best Practices:** [SECURITY_ARCHITECTURE.md#security-best-practices](SECURITY_ARCHITECTURE.md#security-best-practices) - -## 📚 Learning Paths - -### New to Trilium Development -1. Read [ARCHITECTURE.md](ARCHITECTURE.md) - System overview -2. Setup environment: [Environment Setup](Developer%20Guide/Developer%20Guide/Environment%20Setup.md) -3. Explore [DATABASE.md](DATABASE.md) - Understand data model -4. Check [Developer Guide](Developer%20Guide/Developer%20Guide/) - -### Want to Create Scripts -1. Read [SCRIPTING.md](SCRIPTING.md) - Complete guide -2. Check [Script API](Script%20API/) - API reference -3. Review examples: [SCRIPTING.md#script-examples](SCRIPTING.md#script-examples) -4. Explore [Advanced Showcases](https://triliumnext.github.io/Docs/Wiki/advanced-showcases) - -### Setting Up Sync -1. Understand protocol: [SYNCHRONIZATION.md](SYNCHRONIZATION.md) -2. Configure server: [SYNCHRONIZATION.md#sync-server-configuration](SYNCHRONIZATION.md#sync-server-configuration) -3. Setup clients: [SYNCHRONIZATION.md#client-setup](SYNCHRONIZATION.md#client-setup) -4. Troubleshoot: [SYNCHRONIZATION.md#troubleshooting](SYNCHRONIZATION.md#troubleshooting) - -### Security Review -1. Read threat model: [SECURITY_ARCHITECTURE.md#threat-model](SECURITY_ARCHITECTURE.md#threat-model) -2. Review authentication: [SECURITY_ARCHITECTURE.md#authentication](SECURITY_ARCHITECTURE.md#authentication) -3. Check encryption: [SECURITY_ARCHITECTURE.md#encryption](SECURITY_ARCHITECTURE.md#encryption) -4. Verify best practices: [SECURITY_ARCHITECTURE.md#security-best-practices](SECURITY_ARCHITECTURE.md#security-best-practices) - -## 🗺️ Documentation Map - -``` -docs/ -├── TECHNICAL_DOCUMENTATION.md ← START HERE (Index) -│ -├── Core Technical Docs -│ ├── ARCHITECTURE.md (System design) -│ ├── DATABASE.md (Data layer) -│ ├── SYNCHRONIZATION.md (Sync system) -│ ├── SCRIPTING.md (User scripting) -│ └── SECURITY_ARCHITECTURE.md (Security) -│ -├── Developer Guide/ -│ └── Developer Guide/ (Development setup) -│ -├── User Guide/ -│ └── User Guide/ (End-user docs) -│ -└── Script API/ (API reference) -``` - -## 💡 Tips for Reading Documentation - -1. **Start with the index:** [TECHNICAL_DOCUMENTATION.md](TECHNICAL_DOCUMENTATION.md) provides an overview -2. **Use search:** Press Ctrl+F / Cmd+F to find specific topics -3. **Follow links:** Documents are cross-referenced for easy navigation -4. **Code examples:** Most docs include practical code examples -5. **See Also sections:** Check bottom of each doc for related resources - -## 🔗 External Resources - -- **Website:** https://triliumnotes.org -- **Online Docs:** https://docs.triliumnotes.org -- **GitHub:** https://github.com/TriliumNext/Trilium -- **Discussions:** https://github.com/TriliumNext/Trilium/discussions -- **Matrix Chat:** https://matrix.to/#/#triliumnext:matrix.org - -## 🤝 Contributing to Documentation - -Found an error or want to improve the docs? See: -- [Contributing Guide](../README.md#-contribute) -- [Documentation Standards](TECHNICAL_DOCUMENTATION.md#documentation-conventions) - ---- - -**Version:** 0.99.3 -**Last Updated:** November 2025 -**Maintained by:** TriliumNext Team diff --git a/docs/SCRIPTING.md b/docs/SCRIPTING.md deleted file mode 100644 index 49e548c50..000000000 --- a/docs/SCRIPTING.md +++ /dev/null @@ -1,734 +0,0 @@ -# Trilium Scripting System - -> **Related:** [ARCHITECTURE.md](ARCHITECTURE.md) | [Script API Documentation](Script%20API/) - -## Overview - -Trilium features a **powerful scripting system** that allows users to extend and customize the application without modifying source code. Scripts are written in JavaScript and can execute both in the **frontend (browser)** and **backend (Node.js)** contexts. - -## Script Types - -### Frontend Scripts - -**Location:** Attached to notes with `#run=frontendStartup` attribute - -**Execution Context:** Browser environment - -**Access:** -- Trilium Frontend API -- Browser APIs (DOM, localStorage, etc.) -- Froca (frontend cache) -- UI widgets -- No direct file system access - -**Lifecycle:** -- `frontendStartup` - Run once when Trilium loads -- `frontendReload` - Run on every note context change - -**Example:** -```javascript -// Attach to note with #run=frontendStartup -const api = window.api - -// Add custom button to toolbar -api.addButtonToToolbar({ - title: 'My Button', - icon: 'star', - action: () => { - api.showMessage('Hello from frontend!') - } -}) -``` - -### Backend Scripts - -**Location:** Attached to notes with `#run=backendStartup` attribute - -**Execution Context:** Node.js server environment - -**Access:** -- Trilium Backend API -- Node.js APIs (fs, http, etc.) -- Becca (backend cache) -- Database (SQL) -- External libraries (via require) - -**Lifecycle:** -- `backendStartup` - Run once when server starts -- Event handlers (custom events) - -**Example:** -```javascript -// Attach to note with #run=backendStartup -const api = require('@triliumnext/api') - -// Listen for note creation -api.dayjs // Example: access dayjs library - -api.onNoteCreated((note) => { - if (note.title.includes('TODO')) { - note.setLabel('priority', 'high') - } -}) -``` - -### Render Scripts - -**Location:** Attached to notes with `#customWidget` or similar attributes - -**Purpose:** Custom note rendering/widgets - -**Example:** -```javascript -// Custom widget for a note -class MyWidget extends api.NoteContextAwareWidget { - doRender() { - this.$widget = $('
') - .text('Custom widget content') - return this.$widget - } -} - -module.exports = MyWidget -``` - -## Script API - -### Frontend API - -**Location:** `apps/client/src/services/frontend_script_api.ts` - -**Global Access:** `window.api` - -**Key Methods:** - -```typescript -// Note Operations -api.getNote(noteId) // Get note object -api.getBranch(branchId) // Get branch object -api.getActiveNote() // Currently displayed note -api.openNote(noteId, activateNote) // Open note in UI - -// UI Operations -api.showMessage(message) // Show toast notification -api.showDialog() // Show modal dialog -api.confirm(message) // Show confirmation dialog -api.prompt(message, defaultValue) // Show input prompt - -// Tree Operations -api.getTree() // Get note tree structure -api.expandTree(noteId) // Expand tree branch -api.collapseTree(noteId) // Collapse tree branch - -// Search -api.searchForNotes(searchQuery) // Search notes -api.searchForNote(searchQuery) // Get single note - -// Navigation -api.openTabWithNote(noteId) // Open note in new tab -api.closeActiveTab() // Close current tab -api.activateNote(noteId) // Switch to note - -// Attributes -api.getAttribute(noteId, type, name) // Get attribute -api.getAttributes(noteId, type, name) // Get all matching attributes - -// Custom Widgets -api.addButtonToToolbar(def) // Add toolbar button -api.addCustomWidget(def) // Add custom widget - -// Events -api.runOnNoteOpened(callback) // Note opened event -api.runOnNoteContentChange(callback) // Content changed event - -// Utilities -api.dayjs // Date/time library -api.formatDate(date) // Format date -api.log(message) // Console log -``` - -### Backend API - -**Location:** `apps/server/src/services/backend_script_api.ts` - -**Access:** `require('@triliumnext/api')` or global `api` - -**Key Methods:** - -```typescript -// Note Operations -api.getNote(noteId) // Get note from Becca -api.getNoteWithContent(noteId) // Get note with content -api.createNote(parentNoteId, title) // Create new note -api.deleteNote(noteId) // Delete note - -// Branch Operations -api.getBranch(branchId) // Get branch -api.createBranch(noteId, parentNoteId) // Create branch (clone) - -// Attribute Operations -api.getAttribute(noteId, type, name) // Get attribute -api.createAttribute(noteId, type, name, value) // Create attribute - -// Database Access -api.sql.getRow(query, params) // Execute SQL query (single row) -api.sql.getRows(query, params) // Execute SQL query (multiple rows) -api.sql.execute(query, params) // Execute SQL statement - -// Events -api.onNoteCreated(callback) // Note created event -api.onNoteUpdated(callback) // Note updated event -api.onNoteDeleted(callback) // Note deleted event -api.onAttributeCreated(callback) // Attribute created event - -// Search -api.searchForNotes(searchQuery) // Search notes - -// Date/Time -api.dayjs // Date/time library -api.now() // Current date/time - -// Logging -api.log(message) // Log message -api.error(message) // Log error - -// External Communication -api.axios // HTTP client library - -// Utilities -api.backup.backupNow() // Trigger backup -api.export.exportSubtree(noteId) // Export notes -``` - -## Script Attributes - -### Execute Attributes - -- `#run=frontendStartup` - Execute on frontend startup -- `#run=backendStartup` - Execute on backend startup -- `#run=hourly` - Execute every hour -- `#run=daily` - Execute daily - -### Widget Attributes - -- `#customWidget` - Custom note widget -- `#widget` - Standard widget integration - -### Other Attributes - -- `#disableVersioning` - Disable automatic versioning for this note -- `#hideChildrenOverview` - Hide children in overview -- `#iconClass` - Custom icon for note - -## Entity Classes - -### Frontend Entities - -**FNote** (`apps/client/src/entities/fnote.ts`) - -```typescript -class FNote { - noteId: string - title: string - type: string - mime: string - - // Relationships - getParentNotes(): FNote[] - getChildNotes(): FNote[] - getBranches(): FBranch[] - - // Attributes - getAttribute(type, name): FAttribute - getAttributes(type?, name?): FAttribute[] - hasLabel(name): boolean - getLabelValue(name): string - - // Content - getContent(): Promise - - // Navigation - open(): void -} -``` - -**FBranch** - -```typescript -class FBranch { - branchId: string - noteId: string - parentNoteId: string - prefix: string - notePosition: number - - getNote(): FNote - getParentNote(): FNote -} -``` - -**FAttribute** - -```typescript -class FAttribute { - attributeId: string - noteId: string - type: 'label' | 'relation' - name: string - value: string - - getNote(): FNote - getTargetNote(): FNote // For relations -} -``` - -### Backend Entities - -**BNote** (`apps/server/src/becca/entities/bnote.ts`) - -```typescript -class BNote { - noteId: string - title: string - type: string - mime: string - isProtected: boolean - - // Content - getContent(): string | Buffer - setContent(content: string | Buffer): void - - // Relationships - getParentNotes(): BNote[] - getChildNotes(): BNote[] - getBranches(): BBranch[] - - // Attributes - getAttribute(type, name): BAttribute - getAttributes(type?, name?): BAttribute[] - setLabel(name, value): BAttribute - setRelation(name, targetNoteId): BAttribute - hasLabel(name): boolean - getLabelValue(name): string - - // Operations - save(): void - markAsDeleted(): void -} -``` - -**BBranch** - -```typescript -class BBranch { - branchId: string - noteId: string - parentNoteId: string - prefix: string - notePosition: number - - getNote(): BNote - getParentNote(): BNote - save(): void -} -``` - -**BAttribute** - -```typescript -class BAttribute { - attributeId: string - noteId: string - type: 'label' | 'relation' - name: string - value: string - - getNote(): BNote - getTargetNote(): BNote // For relations - save(): void -} -``` - -## Script Examples - -### Frontend Examples - -**1. Custom Toolbar Button** - -```javascript -// #run=frontendStartup -api.addButtonToToolbar({ - title: 'Export to PDF', - icon: 'file-export', - action: async () => { - const note = api.getActiveNote() - if (note) { - await api.runOnBackend('exportToPdf', [note.noteId]) - api.showMessage('Export started') - } - } -}) -``` - -**2. Auto-Save Reminder** - -```javascript -// #run=frontendStartup -let saveTimer -api.runOnNoteContentChange(() => { - clearTimeout(saveTimer) - saveTimer = setTimeout(() => { - api.showMessage('Remember to save your work!') - }, 300000) // 5 minutes -}) -``` - -**3. Note Statistics Widget** - -```javascript -// #customWidget -class StatsWidget extends api.NoteContextAwareWidget { - doRender() { - this.$widget = $('
') - return this.$widget - } - - async refreshWithNote(note) { - const content = await note.getContent() - const words = content.split(/\s+/).length - const chars = content.length - - this.$widget.html(` -
Words: ${words}
-
Characters: ${chars}
- `) - } -} - -module.exports = StatsWidget -``` - -### Backend Examples - -**1. Auto-Tagging on Note Creation** - -```javascript -// #run=backendStartup -api.onNoteCreated((note) => { - // Auto-tag TODO notes - if (note.title.includes('TODO')) { - note.setLabel('type', 'todo') - note.setLabel('priority', 'normal') - } - - // Auto-tag meeting notes by date - if (note.title.match(/Meeting \d{4}-\d{2}-\d{2}/)) { - note.setLabel('type', 'meeting') - const dateMatch = note.title.match(/(\d{4}-\d{2}-\d{2})/) - if (dateMatch) { - note.setLabel('date', dateMatch[1]) - } - } -}) -``` - -**2. Daily Backup Reminder** - -```javascript -// #run=daily -const todayNote = api.getTodayNote() -todayNote.setLabel('backupDone', 'false') - -// Create reminder note -api.createNote(todayNote.noteId, '🔔 Backup Reminder', { - content: 'Remember to verify today\'s backup!', - type: 'text' -}) -``` - -**3. External API Integration** - -```javascript -// #run=backendStartup -api.onNoteCreated(async (note) => { - // Sync new notes to external service - if (note.hasLabel('sync-external')) { - try { - await api.axios.post('https://external-api.com/sync', { - noteId: note.noteId, - title: note.title, - content: note.getContent() - }) - note.setLabel('lastSync', api.dayjs().format()) - } catch (error) { - api.log('Sync failed: ' + error.message) - } - } -}) -``` - -**4. Database Cleanup** - -```javascript -// #run=weekly -// Clean up old revisions -const cutoffDate = api.dayjs().subtract(90, 'days').format() - -const oldRevisions = api.sql.getRows(` - SELECT revisionId FROM revisions - WHERE utcDateCreated < ? -`, [cutoffDate]) - -api.log(`Deleting ${oldRevisions.length} old revisions`) - -for (const row of oldRevisions) { - api.sql.execute('DELETE FROM revisions WHERE revisionId = ?', [row.revisionId]) -} -``` - -## Script Storage - -**Storage Location:** Scripts are stored as regular notes - -**Identifying Scripts:** -- Have `#run` attribute or `#customWidget` attribute -- Type is typically `code` with MIME `application/javascript` - -**Script Note Structure:** -``` -📁 Scripts (folder note) -├── 📜 Frontend Scripts -│ ├── Custom Toolbar Button (#run=frontendStartup) -│ └── Statistics Widget (#customWidget) -└── 📜 Backend Scripts - ├── Auto-Tagger (#run=backendStartup) - └── Daily Backup (#run=daily) -``` - -## Script Execution - -### Frontend Execution - -**Timing:** -1. Trilium frontend loads -2. Froca cache initializes -3. Script notes with `#run=frontendStartup` are found -4. Scripts execute in dependency order - -**Isolation:** -- Each script runs in separate context -- Shared `window.api` object -- Can access global window object - -### Backend Execution - -**Timing:** -1. Server starts -2. Becca cache loads -3. Script notes with `#run=backendStartup` are found -4. Scripts execute in dependency order - -**Isolation:** -- Each script is a separate module -- Can require Node.js modules -- Shared `api` global - -### Error Handling - -**Frontend:** -```javascript -try { - // Script code -} catch (error) { - api.showError('Script error: ' + error.message) - console.error(error) -} -``` - -**Backend:** -```javascript -try { - // Script code -} catch (error) { - api.log('Script error: ' + error.message) - console.error(error) -} -``` - -## Security Considerations - -### Frontend Scripts - -**Risks:** -- Can access all notes via Froca -- Can manipulate DOM -- Can make API calls -- Limited by browser security model - -**Mitigations:** -- User must trust scripts they add -- Scripts run with user privileges -- No access to file system - -### Backend Scripts - -**Risks:** -- Full Node.js access -- Can execute system commands -- Can access file system -- Can make network requests - -**Mitigations:** -- Scripts are user-created (trusted) -- Single-user model (no privilege escalation) -- Review scripts before adding `#run` attribute - -### Best Practices - -1. **Review script code** before adding execution attributes -2. **Use specific attributes** rather than wildcard searches -3. **Avoid eval()** and dynamic code execution -4. **Validate inputs** in scripts -5. **Handle errors** gracefully -6. **Log important actions** for audit trail - -## Performance Considerations - -### Optimization Tips - -**1. Cache Results:** -```javascript -// Bad: Re-query on every call -function getConfig() { - return api.getNote('config').getContent() -} - -// Good: Cache the result -let cachedConfig -function getConfig() { - if (!cachedConfig) { - cachedConfig = api.getNote('config').getContent() - } - return cachedConfig -} -``` - -**2. Use Efficient Queries:** -```javascript -// Bad: Load all notes and filter -const todos = api.searchForNotes('#type=todo') - -// Good: Use specific search -const todos = api.searchForNotes('#type=todo #status=pending') -``` - -**3. Batch Operations:** -```javascript -// Bad: Save after each change -notes.forEach(note => { - note.title = 'Updated' - note.save() -}) - -// Good: Batch changes -notes.forEach(note => { - note.title = 'Updated' -}) -// Save happens in batch -``` - -**4. Debounce Event Handlers:** -```javascript -let timeout -api.runOnNoteContentChange(() => { - clearTimeout(timeout) - timeout = setTimeout(() => { - // Process change - }, 500) -}) -``` - -## Debugging Scripts - -### Frontend Debugging - -**Browser DevTools:** -```javascript -console.log('Debug info:', data) -debugger // Breakpoint -``` - -**Trilium Log:** -```javascript -api.log('Script executed') -``` - -### Backend Debugging - -**Console Output:** -```javascript -console.log('Backend debug:', data) -api.log('Script log message') -``` - -**Inspect Becca:** -```javascript -api.log('Note count:', Object.keys(api.becca.notes).length) -``` - -## Advanced Topics - -### Custom Note Types - -Scripts can implement custom note type handlers: - -```javascript -// Register custom type -api.registerNoteType({ - type: 'mytype', - mime: 'application/x-mytype', - renderNote: (note) => { - // Custom rendering - } -}) -``` - -### External Libraries - -**Frontend:** -```javascript -// Load external library -const myLib = await import('https://cdn.example.com/lib.js') -``` - -**Backend:** -```javascript -// Use Node.js require -const fs = require('fs') -const axios = require('axios') -``` - -### State Persistence - -**Frontend:** -```javascript -// Use localStorage -localStorage.setItem('myScript:data', JSON.stringify(data)) -const data = JSON.parse(localStorage.getItem('myScript:data')) -``` - -**Backend:** -```javascript -// Store in special note -const stateNote = api.getNote('script-state-note') -stateNote.setContent(JSON.stringify(data)) - -const data = JSON.parse(stateNote.getContent()) -``` - ---- - -**See Also:** -- [Script API Documentation](Script%20API/) - Complete API reference -- [Advanced Showcases](https://triliumnext.github.io/Docs/Wiki/advanced-showcases) - Example scripts -- [ARCHITECTURE.md](ARCHITECTURE.md) - Overall architecture diff --git a/docs/SECURITY_ARCHITECTURE.md b/docs/SECURITY_ARCHITECTURE.md deleted file mode 100644 index 27993deac..000000000 --- a/docs/SECURITY_ARCHITECTURE.md +++ /dev/null @@ -1,834 +0,0 @@ -# Trilium Security Architecture - -> **Related:** [ARCHITECTURE.md](ARCHITECTURE.md) | [SECURITY.md](../SECURITY.md) - -## Overview - -Trilium implements a **defense-in-depth security model** with multiple layers of protection for user data. The security architecture covers authentication, authorization, encryption, input sanitization, and secure communication. - -## Security Principles - -1. **Data Privacy**: User data is protected at rest and in transit -2. **Encryption**: Per-note encryption for sensitive content -3. **Authentication**: Multiple authentication methods supported -4. **Authorization**: Single-user model with granular protected sessions -5. **Input Validation**: All user input sanitized -6. **Secure Defaults**: Security features enabled by default -7. **Transparency**: Open source allows security audits - -## Threat Model - -### Threats Considered - -1. **Unauthorized Access** - - Physical access to device - - Network eavesdropping - - Stolen credentials - - Session hijacking - -2. **Data Exfiltration** - - Malicious scripts - - XSS attacks - - SQL injection - - CSRF attacks - -3. **Data Corruption** - - Malicious modifications - - Database tampering - - Sync conflicts - -4. **Privacy Leaks** - - Unencrypted backups - - Search indexing - - Temporary files - - Memory dumps - -### Out of Scope - -- Nation-state attackers -- Zero-day vulnerabilities in dependencies -- Hardware vulnerabilities (Spectre, Meltdown) -- Physical access with unlimited time -- Quantum computing attacks - -## Authentication - -### Password Authentication - -**Implementation:** `apps/server/src/services/password.ts` - -**Password Storage:** -```typescript -// Password is never stored directly -const salt = crypto.randomBytes(32) -const derivedKey = crypto.pbkdf2Sync(password, salt, 10000, 32, 'sha256') -const verificationHash = crypto.createHash('sha256') - .update(derivedKey) - .digest('hex') - -// Store only salt and verification hash -sql.insert('user_data', { - salt: salt.toString('hex'), - derivedKey: derivedKey.toString('hex') // Used for encryption -}) - -sql.insert('options', { - name: 'passwordVerificationHash', - value: verificationHash -}) -``` - -**Password Requirements:** -- Minimum length: 4 characters (configurable) -- No maximum length -- All characters allowed -- Can be changed by user - -**Login Process:** -```typescript -// 1. User submits password -POST /api/login/password -Body: { password: "user-password" } - -// 2. Server derives key -const derivedKey = crypto.pbkdf2Sync(password, salt, 10000, 32, 'sha256') - -// 3. Verify against stored hash -const verificationHash = crypto.createHash('sha256') - .update(derivedKey) - .digest('hex') - -if (verificationHash === storedHash) { - // 4. Create session - req.session.loggedIn = true - req.session.regenerate() -} -``` - -### TOTP (Two-Factor Authentication) - -**Implementation:** `apps/server/src/routes/api/login.ts` - -**Setup Process:** -```typescript -// 1. Generate secret -const secret = speakeasy.generateSecret({ - name: `Trilium (${username})`, - length: 32 -}) - -// 2. Store encrypted secret -const encryptedSecret = encrypt(secret.base32, dataKey) -sql.insert('options', { - name: 'totpSecret', - value: encryptedSecret -}) - -// 3. Generate QR code -const qrCodeUrl = secret.otpauth_url -``` - -**Verification:** -```typescript -// User submits TOTP token -POST /api/login/totp -Body: { token: "123456" } - -// Verify token -const secret = decrypt(encryptedSecret, dataKey) -const verified = speakeasy.totp.verify({ - secret: secret, - encoding: 'base32', - token: token, - window: 1 // Allow 1 time step tolerance -}) -``` - -### OpenID Connect - -**Implementation:** `apps/server/src/routes/api/login.ts` - -**Supported Providers:** -- Any OpenID Connect compatible provider -- Google, GitHub, Auth0, etc. - -**Flow:** -```typescript -// 1. Redirect to provider -GET /api/login/openid - -// 2. Provider redirects back with code -GET /api/login/openid/callback?code=... - -// 3. Exchange code for tokens -const tokens = await openidClient.callback(redirectUri, req.query) - -// 4. Verify ID token -const claims = tokens.claims() - -// 5. Create session -req.session.loggedIn = true -``` - -### Session Management - -**Session Storage:** SQLite database (sessions table) - -**Session Configuration:** -```typescript -app.use(session({ - secret: sessionSecret, - resave: false, - saveUninitialized: false, - rolling: true, - cookie: { - maxAge: 7 * 24 * 60 * 60 * 1000, // 7 days - httpOnly: true, - secure: isHttps, - sameSite: 'lax' - }, - store: new SqliteStore({ - db: db, - table: 'sessions' - }) -})) -``` - -**Session Invalidation:** -- Automatic timeout after inactivity -- Manual logout clears session -- Server restart invalidates all sessions (optional) - -## Authorization - -### Single-User Model - -**Desktop:** -- Single user (owner of device) -- No multi-user support -- Full access to all notes - -**Server:** -- Single user per installation -- Authentication required for all operations -- No user roles or permissions - -### Protected Sessions - -**Purpose:** Temporary access to encrypted (protected) notes - -**Implementation:** `apps/server/src/services/protected_session.ts` - -**Workflow:** -```typescript -// 1. User enters password for protected notes -POST /api/protected-session/enter -Body: { password: "protected-password" } - -// 2. Derive encryption key -const protectedDataKey = deriveKey(password) - -// 3. Verify password (decrypt known encrypted value) -const decrypted = decrypt(testValue, protectedDataKey) -if (decrypted === expectedValue) { - // 4. Store in memory (not in session) - protectedSessionHolder.setProtectedDataKey(protectedDataKey) - - // 5. Set timeout - setTimeout(() => { - protectedSessionHolder.clearProtectedDataKey() - }, timeout) -} -``` - -**Protected Session Timeout:** -- Default: 10 minutes (configurable) -- Extends on activity -- Cleared on browser close -- Separate from main session - -### API Authorization - -**Internal API:** -- Requires authenticated session -- CSRF token validation -- Same-origin policy - -**ETAPI (External API):** -- Token-based authentication -- No session required -- Rate limiting - -## Encryption - -### Note Encryption - -**Encryption Algorithm:** AES-256-CBC - -**Key Hierarchy:** -``` -User Password - ↓ (PBKDF2) -Data Key (for protected notes) - ↓ (AES-256) -Protected Note Content -``` - -**Encryption Process:** -```typescript -// 1. Generate IV (initialization vector) -const iv = crypto.randomBytes(16) - -// 2. Encrypt content -const cipher = crypto.createCipheriv('aes-256-cbc', dataKey, iv) -let encrypted = cipher.update(content, 'utf8', 'base64') -encrypted += cipher.final('base64') - -// 3. Prepend IV to encrypted content -const encryptedBlob = iv.toString('base64') + ':' + encrypted - -// 4. Store in database -sql.insert('blobs', { - blobId: blobId, - content: encryptedBlob -}) -``` - -**Decryption Process:** -```typescript -// 1. Split IV and encrypted content -const [ivBase64, encryptedData] = encryptedBlob.split(':') -const iv = Buffer.from(ivBase64, 'base64') - -// 2. Decrypt -const decipher = crypto.createDecipheriv('aes-256-cbc', dataKey, iv) -let decrypted = decipher.update(encryptedData, 'base64', 'utf8') -decrypted += decipher.final('utf8') - -return decrypted -``` - -**Protected Note Metadata:** -- Title is NOT encrypted (for tree display) -- Type and MIME are NOT encrypted -- Content IS encrypted -- Attributes CAN be encrypted (optional) - -### Data Key Management - -**Master Data Key:** -```typescript -// Generated once during setup -const dataKey = crypto.randomBytes(32) // 256 bits - -// Encrypted with derived key from user password -const derivedKey = crypto.pbkdf2Sync(password, salt, 10000, 32, 'sha256') -const encryptedDataKey = encrypt(dataKey, derivedKey) - -// Stored in database -sql.insert('options', { - name: 'encryptedDataKey', - value: encryptedDataKey.toString('hex') -}) -``` - -**Key Rotation:** -- Not currently supported -- Requires re-encrypting all protected notes -- Planned for future version - -### Transport Encryption - -**HTTPS:** -- Required for server installations (recommended) -- TLS 1.2+ only -- Strong cipher suites preferred -- Certificate validation enabled - -**Desktop:** -- Local communication (no network) -- No HTTPS required - -### Backup Encryption - -**Database Backups:** -- Protected notes remain encrypted in backup -- Backup file should be protected separately -- Consider encrypting backup storage location - -## Input Sanitization - -### XSS Prevention - -**HTML Sanitization:** - -Location: `apps/client/src/services/dompurify.ts` - -```typescript -import DOMPurify from 'dompurify' - -// Configure DOMPurify -DOMPurify.setConfig({ - ALLOWED_TAGS: ['b', 'i', 'em', 'strong', 'a', 'p', 'br', 'div', ...], - ALLOWED_ATTR: ['href', 'title', 'class', 'id', ...], - ALLOW_DATA_ATTR: false -}) - -// Sanitize HTML before rendering -const cleanHtml = DOMPurify.sanitize(userHtml) -``` - -**CKEditor Configuration:** -```typescript -// apps/client/src/widgets/type_widgets/text_type_widget.ts -ClassicEditor.create(element, { - // Restrict allowed content - htmlSupport: { - allow: [ - { name: /./, attributes: true, classes: true, styles: true } - ], - disallow: [ - { name: 'script' }, - { name: 'iframe', attributes: /^(?!src$).*/ } - ] - } -}) -``` - -**Content Security Policy:** -```typescript -// apps/server/src/main.ts -app.use((req, res, next) => { - res.setHeader('Content-Security-Policy', - "default-src 'self'; " + - "script-src 'self' 'unsafe-inline' 'unsafe-eval'; " + - "style-src 'self' 'unsafe-inline'; " + - "img-src 'self' data: blob:;" - ) - next() -}) -``` - -### SQL Injection Prevention - -**Parameterized Queries:** -```typescript -// GOOD - Safe from SQL injection -const notes = sql.getRows( - 'SELECT * FROM notes WHERE title = ?', - [userInput] -) - -// BAD - Vulnerable to SQL injection -const notes = sql.getRows( - `SELECT * FROM notes WHERE title = '${userInput}'` -) -``` - -**ORM Usage:** -```typescript -// Entity-based access prevents SQL injection -const note = becca.getNote(noteId) -note.title = userInput // Sanitized by entity -note.save() // Parameterized query -``` - -### CSRF Prevention - -**CSRF Token Validation:** - -Location: `apps/server/src/routes/middleware/csrf.ts` - -```typescript -// Generate CSRF token -const csrfToken = crypto.randomBytes(32).toString('hex') -req.session.csrfToken = csrfToken - -// Validate on state-changing requests -app.use((req, res, next) => { - if (['POST', 'PUT', 'DELETE'].includes(req.method)) { - const token = req.headers['x-csrf-token'] - if (token !== req.session.csrfToken) { - return res.status(403).json({ error: 'CSRF token mismatch' }) - } - } - next() -}) -``` - -**Client-Side:** -```typescript -// apps/client/src/services/server.ts -const csrfToken = getCsrfToken() - -fetch('/api/notes', { - method: 'POST', - headers: { - 'X-CSRF-Token': csrfToken, - 'Content-Type': 'application/json' - }, - body: JSON.stringify(data) -}) -``` - -### File Upload Validation - -**Validation:** -```typescript -// apps/server/src/routes/api/attachments.ts -const allowedMimeTypes = [ - 'image/jpeg', - 'image/png', - 'application/pdf', - // ... -] - -if (!allowedMimeTypes.includes(file.mimetype)) { - throw new Error('File type not allowed') -} - -// Validate file size -const maxSize = 100 * 1024 * 1024 // 100 MB -if (file.size > maxSize) { - throw new Error('File too large') -} - -// Sanitize filename -const sanitizedFilename = path.basename(file.originalname) - .replace(/[^a-z0-9.-]/gi, '_') -``` - -## Network Security - -### HTTPS Configuration - -**Server Setup:** -```typescript -// apps/server/src/main.ts -const httpsOptions = { - key: fs.readFileSync('server.key'), - cert: fs.readFileSync('server.cert') -} - -https.createServer(httpsOptions, app).listen(443) -``` - -**Certificate Validation:** -- Require valid certificates in production -- Self-signed certificates allowed for development -- Certificate pinning not implemented - -### Secure Headers - -```typescript -// apps/server/src/main.ts -app.use((req, res, next) => { - // Prevent clickjacking - res.setHeader('X-Frame-Options', 'SAMEORIGIN') - - // Prevent MIME sniffing - res.setHeader('X-Content-Type-Options', 'nosniff') - - // XSS protection - res.setHeader('X-XSS-Protection', '1; mode=block') - - // Referrer policy - res.setHeader('Referrer-Policy', 'same-origin') - - // HTTPS upgrade - if (req.secure) { - res.setHeader('Strict-Transport-Security', 'max-age=31536000') - } - - next() -}) -``` - -### Rate Limiting - -**API Rate Limiting:** -```typescript -// apps/server/src/routes/middleware/rate_limit.ts -const rateLimit = require('express-rate-limit') - -const apiLimiter = rateLimit({ - windowMs: 15 * 60 * 1000, // 15 minutes - max: 1000, // Limit each IP to 1000 requests per window - message: 'Too many requests from this IP' -}) - -app.use('/api/', apiLimiter) -``` - -**Login Rate Limiting:** -```typescript -const loginLimiter = rateLimit({ - windowMs: 15 * 60 * 1000, - max: 5, // 5 failed attempts - skipSuccessfulRequests: true -}) - -app.post('/api/login/password', loginLimiter, loginHandler) -``` - -## Data Security - -### Secure Data Deletion - -**Soft Delete:** -```typescript -// Mark as deleted (sync first) -note.isDeleted = 1 -note.deleteId = generateUUID() -note.save() - -// Entity change tracked for sync -addEntityChange('notes', noteId, note) -``` - -**Hard Delete (Erase):** -```typescript -// After sync completed -sql.execute('DELETE FROM notes WHERE noteId = ?', [noteId]) -sql.execute('DELETE FROM branches WHERE noteId = ?', [noteId]) -sql.execute('DELETE FROM attributes WHERE noteId = ?', [noteId]) - -// Mark entity change as erased -sql.execute('UPDATE entity_changes SET isErased = 1 WHERE entityId = ?', [noteId]) -``` - -**Blob Cleanup:** -```typescript -// Find orphaned blobs (not referenced by any note/revision/attachment) -const orphanedBlobs = sql.getRows(` - SELECT blobId FROM blobs - WHERE blobId NOT IN (SELECT blobId FROM notes WHERE blobId IS NOT NULL) - AND blobId NOT IN (SELECT blobId FROM revisions WHERE blobId IS NOT NULL) - AND blobId NOT IN (SELECT blobId FROM attachments WHERE blobId IS NOT NULL) -`) - -// Delete orphaned blobs -for (const blob of orphanedBlobs) { - sql.execute('DELETE FROM blobs WHERE blobId = ?', [blob.blobId]) -} -``` - -### Memory Security - -**Protected Data in Memory:** -- Protected data keys stored in memory only -- Cleared on timeout -- Not written to disk -- Not in session storage - -**Memory Cleanup:** -```typescript -// Clear sensitive data -const clearSensitiveData = () => { - protectedDataKey = null - - // Force garbage collection if available - if (global.gc) { - global.gc() - } -} -``` - -### Temporary Files - -**Secure Temporary Files:** -```typescript -const tempDir = os.tmpdir() -const tempFile = path.join(tempDir, `trilium-${crypto.randomBytes(16).toString('hex')}`) - -// Write temp file -fs.writeFileSync(tempFile, data, { mode: 0o600 }) // Owner read/write only - -// Clean up after use -fs.unlinkSync(tempFile) -``` - -## Dependency Security - -### Vulnerability Scanning - -**Tools:** -- `npm audit` - Check for known vulnerabilities -- Renovate bot - Automatic dependency updates -- GitHub Dependabot alerts - -**Process:** -```bash -# Check for vulnerabilities -npm audit - -# Fix automatically -npm audit fix - -# Manual review for breaking changes -npm audit fix --force -``` - -### Dependency Pinning - -**package.json:** -```json -{ - "dependencies": { - "express": "4.18.2", // Exact version - "better-sqlite3": "^9.2.2" // Compatible versions - } -} -``` - -**pnpm Overrides:** -```json -{ - "pnpm": { - "overrides": { - "lodash@<4.17.21": ">=4.17.21", // Force minimum version - "axios@<0.21.2": ">=0.21.2" - } - } -} -``` - -### Patch Management - -**pnpm Patches:** -```bash -# Create patch -pnpm patch @ckeditor/ckeditor5 - -# Edit files in temporary directory -# ... - -# Generate patch file -pnpm patch-commit /tmp/ckeditor5-patch - -# Patch applied automatically on install -``` - -## Security Best Practices - -### For Users - -1. **Strong Passwords** - - Use unique password for Trilium - - Enable TOTP 2FA - - Protect password manager - -2. **Protected Notes** - - Use for sensitive information - - Set reasonable session timeout - - Don't leave sessions unattended - -3. **Backups** - - Regular backups to secure location - - Encrypt backup storage - - Test backup restoration - -4. **Server Setup** - - Use HTTPS only - - Keep software updated - - Firewall configuration - - Use reverse proxy (nginx, Caddy) - -5. **Scripts** - - Review scripts before using - - Be cautious with external scripts - - Understand script permissions - -### For Developers - -1. **Code Review** - - Review all security-related changes - - Test authentication/authorization changes - - Validate input sanitization - -2. **Testing** - - Write security tests - - Test edge cases - - Penetration testing - -3. **Dependencies** - - Regular updates - - Audit new dependencies - - Monitor security advisories - -4. **Secrets** - - No secrets in source code - - Use environment variables - - Secure key generation - -## Security Auditing - -### Logs - -**Security Events Logged:** -- Login attempts (success/failure) -- Protected session access -- Password changes -- ETAPI token usage -- Failed CSRF validations - -**Log Location:** -- Desktop: Console output -- Server: Log files or stdout - -### Monitoring - -**Metrics to Monitor:** -- Failed login attempts -- API error rates -- Unusual database changes -- Large exports/imports - -## Incident Response - -### Security Issue Reporting - -**Process:** -1. Email security@triliumnext.com -2. Include vulnerability details -3. Provide reproduction steps -4. Allow reasonable disclosure time - -**Response:** -1. Acknowledge within 48 hours -2. Investigate and validate -3. Develop fix -4. Coordinate disclosure -5. Release patch - -### Breach Response - -**If Compromised:** -1. Change password immediately -2. Review recent activity -3. Check for unauthorized changes -4. Restore from backup if needed -5. Update security settings - -## Future Security Enhancements - -**Planned:** -- Hardware security key support (U2F/WebAuthn) -- End-to-end encryption for sync -- Zero-knowledge architecture option -- Encryption key rotation -- Audit log enhancements -- Per-note access controls - -**Under Consideration:** -- Multi-user support with permissions -- Blockchain-based sync verification -- Homomorphic encryption for search -- Quantum-resistant encryption - ---- - -**See Also:** -- [SECURITY.md](../SECURITY.md) - Security policy -- [ARCHITECTURE.md](ARCHITECTURE.md) - Overall architecture -- [Protected Notes Guide](https://triliumnext.github.io/Docs/Wiki/protected-notes) diff --git a/docs/TECHNICAL_DOCUMENTATION.md b/docs/TECHNICAL_DOCUMENTATION.md deleted file mode 100644 index 445df0530..000000000 --- a/docs/TECHNICAL_DOCUMENTATION.md +++ /dev/null @@ -1,423 +0,0 @@ -# Trilium Notes - Technical Documentation Index - -Welcome to the comprehensive technical and architectural documentation for Trilium Notes. This index provides quick access to all technical documentation resources. - -## 📚 Core Architecture Documentation - -### [ARCHITECTURE.md](ARCHITECTURE.md) -**Main technical architecture document** covering the complete system design. - -**Topics Covered:** -- High-level architecture overview -- Monorepo structure and organization -- Core architecture patterns (Becca, Froca, Shaca) -- Entity system and data model -- Widget-based UI architecture -- Frontend and backend architecture -- API architecture (Internal, ETAPI, WebSocket) -- Build system and tooling -- Testing strategy -- Security overview - -**Audience:** Developers, architects, contributors - ---- - -### [DATABASE.md](DATABASE.md) -**Complete database architecture and schema documentation.** - -**Topics Covered:** -- SQLite database structure -- Entity tables (notes, branches, attributes, revisions, attachments, blobs) -- System tables (options, entity_changes, sessions) -- Data relationships and integrity -- Database access patterns -- Migrations and versioning -- Performance optimization -- Backup and maintenance -- Security considerations - -**Audience:** Backend developers, database administrators - ---- - -### [SYNCHRONIZATION.md](SYNCHRONIZATION.md) -**Detailed synchronization protocol and implementation.** - -**Topics Covered:** -- Sync architecture overview -- Entity change tracking -- Sync protocol (handshake, pull, push) -- Conflict resolution strategies -- Protected notes synchronization -- Performance optimizations -- Error handling and retry logic -- Sync server configuration -- WebSocket real-time updates -- Troubleshooting guide - -**Audience:** Advanced users, sync server administrators, contributors - ---- - -### [SCRIPTING.md](SCRIPTING.md) -**Comprehensive guide to the Trilium scripting system.** - -**Topics Covered:** -- Script types (frontend, backend, render) -- Frontend API reference -- Backend API reference -- Entity classes (FNote, BNote, etc.) -- Script examples and patterns -- Script storage and execution -- Security considerations -- Performance optimization -- Debugging techniques -- Advanced topics - -**Audience:** Power users, script developers, plugin creators - ---- - -### [SECURITY_ARCHITECTURE.md](SECURITY_ARCHITECTURE.md) -**In-depth security architecture and implementation.** - -**Topics Covered:** -- Security principles and threat model -- Authentication methods (password, TOTP, OpenID) -- Session management -- Authorization and protected sessions -- Encryption (notes, transport, backups) -- Input sanitization (XSS, SQL injection, CSRF) -- Network security (HTTPS, headers, rate limiting) -- Data security and secure deletion -- Dependency security -- Security best practices -- Incident response - -**Audience:** Security engineers, administrators, auditors - ---- - -## 🔧 Developer Documentation - -### [Developer Guide](Developer%20Guide/Developer%20Guide/) -Collection of developer-focused documentation for contributing to Trilium. - -**Key Documents:** -- [Environment Setup](Developer%20Guide/Developer%20Guide/Environment%20Setup.md) - Setting up development environment -- [Project Structure](Developer%20Guide/Developer%20Guide/Project%20Structure.md) - Monorepo organization -- [Development and Architecture](Developer%20Guide/Developer%20Guide/Development%20and%20architecture/) - Various development topics - -**Topics Include:** -- Local development setup -- Building and deployment -- Adding new note types -- Database schema details -- Internationalization -- Icons and UI customization -- Docker development -- Troubleshooting - -**Audience:** Contributors, developers - ---- - -## 📖 User Documentation - -### [User Guide](User%20Guide/User%20Guide/) -Comprehensive end-user documentation for using Trilium. - -**Key Sections:** -- Installation & Setup -- Basic Concepts and Features -- Note Types -- Advanced Usage -- Synchronization -- Import/Export - -**Audience:** End users, administrators - ---- - -### [Script API](Script%20API/) -Complete API reference for user scripting. - -**Coverage:** -- Frontend API methods -- Backend API methods -- Entity properties and methods -- Event handlers -- Utility functions - -**Audience:** Script developers, power users - ---- - -## 🚀 Quick Start Guides - -### For Users -1. [Installation Guide](User%20Guide/User%20Guide/Installation%20&%20Setup/) - Get Trilium running -2. [Basic Concepts](User%20Guide/User%20Guide/Basic%20Concepts%20and%20Features/) - Learn the fundamentals -3. [Scripting Guide](SCRIPTING.md) - Extend Trilium with scripts - -### For Developers -1. [Environment Setup](Developer%20Guide/Developer%20Guide/Environment%20Setup.md) - Setup development environment -2. [Architecture Overview](ARCHITECTURE.md) - Understand the system -3. [Contributing Guide](../README.md#-contribute) - Start contributing - -### For Administrators -1. [Server Installation](User%20Guide/User%20Guide/Installation%20&%20Setup/Server%20Installation.md) - Deploy Trilium server -2. [Synchronization Setup](SYNCHRONIZATION.md) - Configure sync -3. [Security Best Practices](SECURITY_ARCHITECTURE.md#security-best-practices) - Secure your installation - ---- - -## 🔍 Documentation by Topic - -### Architecture & Design -- [Overall Architecture](ARCHITECTURE.md) -- [Monorepo Structure](ARCHITECTURE.md#monorepo-structure) -- [Three-Layer Cache System](ARCHITECTURE.md#three-layer-cache-system) -- [Entity System](ARCHITECTURE.md#entity-system) -- [Widget-Based UI](ARCHITECTURE.md#widget-based-ui) - -### Data & Storage -- [Database Architecture](DATABASE.md) -- [Entity Tables](DATABASE.md#entity-tables) -- [Data Relationships](DATABASE.md#data-relationships) -- [Blob Storage](DATABASE.md#blobs-table) -- [Database Migrations](DATABASE.md#database-migrations) - -### Synchronization -- [Sync Architecture](SYNCHRONIZATION.md#sync-architecture) -- [Sync Protocol](SYNCHRONIZATION.md#sync-protocol) -- [Conflict Resolution](SYNCHRONIZATION.md#conflict-resolution) -- [Protected Notes Sync](SYNCHRONIZATION.md#protected-notes-sync) -- [WebSocket Sync](SYNCHRONIZATION.md#websocket-sync-updates) - -### Security -- [Authentication](SECURITY_ARCHITECTURE.md#authentication) -- [Encryption](SECURITY_ARCHITECTURE.md#encryption) -- [Input Sanitization](SECURITY_ARCHITECTURE.md#input-sanitization) -- [Network Security](SECURITY_ARCHITECTURE.md#network-security) -- [Security Best Practices](SECURITY_ARCHITECTURE.md#security-best-practices) - -### Scripting & Extensibility -- [Script Types](SCRIPTING.md#script-types) -- [Frontend API](SCRIPTING.md#frontend-api) -- [Backend API](SCRIPTING.md#backend-api) -- [Script Examples](SCRIPTING.md#script-examples) -- [Custom Widgets](SCRIPTING.md#render-scripts) - -### Frontend -- [Client Architecture](ARCHITECTURE.md#frontend-architecture) -- [Widget System](ARCHITECTURE.md#widget-based-ui) -- [Event System](ARCHITECTURE.md#event-system) -- [Froca Cache](ARCHITECTURE.md#2-froca-frontend-cache) -- [UI Components](ARCHITECTURE.md#ui-components) - -### Backend -- [Server Architecture](ARCHITECTURE.md#backend-architecture) -- [Service Layer](ARCHITECTURE.md#service-layer) -- [Route Structure](ARCHITECTURE.md#route-structure) -- [Becca Cache](ARCHITECTURE.md#1-becca-backend-cache) -- [Middleware](ARCHITECTURE.md#middleware) - -### Build & Deploy -- [Build System](ARCHITECTURE.md#build-system) -- [Package Manager](ARCHITECTURE.md#package-manager-pnpm) -- [Build Tools](ARCHITECTURE.md#build-tools) -- [Docker](Developer%20Guide/Developer%20Guide/Development%20and%20architecture/Docker.md) -- [Deployment](Developer%20Guide/Developer%20Guide/Building%20and%20deployment/) - -### Testing -- [Testing Strategy](ARCHITECTURE.md#testing-strategy) -- [Test Organization](ARCHITECTURE.md#test-organization) -- [E2E Testing](ARCHITECTURE.md#e2e-testing) - ---- - -## 📋 Reference Documentation - -### File Locations -``` -trilium/ -├── apps/ -│ ├── client/ # Frontend application -│ ├── server/ # Backend server -│ ├── desktop/ # Electron app -│ └── ... -├── packages/ -│ ├── commons/ # Shared code -│ ├── ckeditor5/ # Rich text editor -│ └── ... -├── docs/ -│ ├── ARCHITECTURE.md # Main architecture doc -│ ├── DATABASE.md # Database documentation -│ ├── SYNCHRONIZATION.md # Sync documentation -│ ├── SCRIPTING.md # Scripting guide -│ ├── SECURITY_ARCHITECTURE.md # Security documentation -│ ├── Developer Guide/ # Developer docs -│ ├── User Guide/ # User docs -│ └── Script API/ # API reference -└── ... -``` - -### Key Source Files -- **Backend Entry:** `apps/server/src/main.ts` -- **Frontend Entry:** `apps/client/src/desktop.ts` / `apps/client/src/index.ts` -- **Becca Cache:** `apps/server/src/becca/becca.ts` -- **Froca Cache:** `apps/client/src/services/froca.ts` -- **Database Schema:** `apps/server/src/assets/db/schema.sql` -- **Backend API:** `apps/server/src/services/backend_script_api.ts` -- **Frontend API:** `apps/client/src/services/frontend_script_api.ts` - -### Important Directories -- **Entities:** `apps/server/src/becca/entities/` -- **Widgets:** `apps/client/src/widgets/` -- **Services:** `apps/server/src/services/` -- **Routes:** `apps/server/src/routes/` -- **Migrations:** `apps/server/src/migrations/` -- **Tests:** Various `*.spec.ts` files throughout - ---- - -## 🎯 Common Tasks - -### Understanding the Codebase -1. Read [ARCHITECTURE.md](ARCHITECTURE.md) for overview -2. Explore [Monorepo Structure](ARCHITECTURE.md#monorepo-structure) -3. Review [Entity System](ARCHITECTURE.md#entity-system) -4. Check [Key Files](ARCHITECTURE.md#key-files-for-understanding-architecture) - -### Adding Features -1. Review relevant architecture documentation -2. Check [Developer Guide](Developer%20Guide/Developer%20Guide/) -3. Follow existing patterns in codebase -4. Write tests -5. Update documentation - -### Debugging Issues -1. Check [Troubleshooting](Developer%20Guide/Developer%20Guide/Troubleshooting/) -2. Review [Database](DATABASE.md) for data issues -3. Check [Synchronization](SYNCHRONIZATION.md) for sync issues -4. Review [Security](SECURITY_ARCHITECTURE.md) for auth issues - -### Performance Optimization -1. [Database Performance](DATABASE.md#performance-optimization) -2. [Cache Optimization](ARCHITECTURE.md#caching-system) -3. [Build Optimization](ARCHITECTURE.md#build-system) -4. [Script Performance](SCRIPTING.md#performance-considerations) - ---- - -## 🔗 External Resources - -### Official Links -- **Website:** https://triliumnotes.org -- **Documentation:** https://docs.triliumnotes.org -- **GitHub:** https://github.com/TriliumNext/Trilium -- **Discussions:** https://github.com/TriliumNext/Trilium/discussions -- **Matrix Chat:** https://matrix.to/#/#triliumnext:matrix.org - -### Community Resources -- **Awesome Trilium:** https://github.com/Nriver/awesome-trilium -- **TriliumRocks:** https://trilium.rocks/ -- **Wiki:** https://triliumnext.github.io/Docs/Wiki/ - -### Related Projects -- **TriliumDroid:** https://github.com/FliegendeWurst/TriliumDroid -- **Web Clipper:** Included in main repository - ---- - -## 📝 Documentation Conventions - -### Document Structure -- Overview section -- Table of contents -- Main content with headings -- Code examples where relevant -- "See Also" references - -### Code Examples -```typescript -// TypeScript examples with comments -const example = 'value' -``` - -```sql --- SQL examples with formatting -SELECT * FROM notes WHERE noteId = ? -``` - -### Cross-References -- Use relative links: `[text](path/to/file.md)` -- Reference sections: `[text](file.md#section)` -- External links: Full URLs - -### Maintenance -- Review on major releases -- Update for architectural changes -- Add examples for new features -- Keep API references current - ---- - -## 🤝 Contributing to Documentation - -### What to Document -- New features and APIs -- Architecture changes -- Migration guides -- Performance tips -- Security considerations - -### How to Contribute -1. Edit markdown files in `docs/` -2. Follow existing structure and style -3. Include code examples -4. Test links and formatting -5. Submit pull request - -### Documentation Standards -- Clear, concise language -- Complete code examples -- Proper markdown formatting -- Cross-references to related docs -- Updated version numbers - ---- - -## 📅 Version Information - -- **Documentation Version:** 0.99.3 -- **Last Updated:** November 2025 -- **Trilium Version:** 0.99.3+ -- **Next Review:** When major architectural changes occur - ---- - -## 💡 Getting Help - -### For Users -- [User Guide](User%20Guide/User%20Guide/) -- [GitHub Discussions](https://github.com/TriliumNext/Trilium/discussions) -- [Matrix Chat](https://matrix.to/#/#triliumnext:matrix.org) - -### For Developers -- [Developer Guide](Developer%20Guide/Developer%20Guide/) -- [Architecture Docs](ARCHITECTURE.md) -- [GitHub Issues](https://github.com/TriliumNext/Trilium/issues) - -### For Contributors -- [Contributing Guidelines](../README.md#-contribute) -- [Code of Conduct](../CODE_OF_CONDUCT) -- [Developer Setup](Developer%20Guide/Developer%20Guide/Environment%20Setup.md) - ---- - -**Maintained by:** TriliumNext Team -**License:** AGPL-3.0-only -**Repository:** https://github.com/TriliumNext/Trilium