mirror of
https://github.com/zadam/trilium.git
synced 2025-11-05 04:45:47 +01:00
docs(dev): integrate rest of the documentation
This commit is contained in:
736
docs/DATABASE.md
vendored
736
docs/DATABASE.md
vendored
@@ -1,736 +0,0 @@
|
|||||||
# Trilium Database Architecture
|
|
||||||
|
|
||||||
> **Related:** [ARCHITECTURE.md](ARCHITECTURE.md) | [Database Schema](Developer%20Guide/Developer%20Guide/Development%20and%20architecture/Database/)
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
Trilium uses **SQLite** as its embedded database engine, providing a reliable, file-based storage system that requires no separate database server. The database stores all notes, their relationships, metadata, and configuration.
|
|
||||||
|
|
||||||
## Database File
|
|
||||||
|
|
||||||
**Location:**
|
|
||||||
- Desktop: `~/.local/share/trilium-data/document.db` (Linux/Mac) or `%APPDATA%/trilium-data/document.db` (Windows)
|
|
||||||
- Server: Configured via `TRILIUM_DATA_DIR` environment variable
|
|
||||||
- Docker: Mounted volume at `/home/node/trilium-data/`
|
|
||||||
|
|
||||||
**Characteristics:**
|
|
||||||
- Single-file database
|
|
||||||
- Embedded (no server required)
|
|
||||||
- ACID compliant
|
|
||||||
- Cross-platform
|
|
||||||
- Supports up to 281 TB database size
|
|
||||||
- Efficient for 100k+ notes
|
|
||||||
|
|
||||||
## Database Driver
|
|
||||||
|
|
||||||
**Library:** `better-sqlite3`
|
|
||||||
|
|
||||||
**Why better-sqlite3:**
|
|
||||||
- Native performance (C++ bindings)
|
|
||||||
- Synchronous API (simpler code)
|
|
||||||
- Prepared statements
|
|
||||||
- Transaction support
|
|
||||||
- Type safety
|
|
||||||
|
|
||||||
**Usage:**
|
|
||||||
```typescript
|
|
||||||
// apps/server/src/services/sql.ts
|
|
||||||
import Database from 'better-sqlite3'
|
|
||||||
|
|
||||||
const db = new Database('document.db')
|
|
||||||
const stmt = db.prepare('SELECT * FROM notes WHERE noteId = ?')
|
|
||||||
const note = stmt.get(noteId)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Schema Overview
|
|
||||||
|
|
||||||
Schema location: `apps/server/src/assets/db/schema.sql`
|
|
||||||
|
|
||||||
**Entity Tables:**
|
|
||||||
- `notes` - Core note data
|
|
||||||
- `branches` - Tree relationships
|
|
||||||
- `attributes` - Metadata (labels/relations)
|
|
||||||
- `revisions` - Version history
|
|
||||||
- `attachments` - File attachments
|
|
||||||
- `blobs` - Binary content storage
|
|
||||||
|
|
||||||
**System Tables:**
|
|
||||||
- `options` - Application configuration
|
|
||||||
- `entity_changes` - Change tracking for sync
|
|
||||||
- `recent_notes` - Recently accessed notes
|
|
||||||
- `etapi_tokens` - API authentication tokens
|
|
||||||
- `user_data` - User credentials
|
|
||||||
- `sessions` - Web session storage
|
|
||||||
|
|
||||||
## Entity Tables
|
|
||||||
|
|
||||||
### Notes Table
|
|
||||||
|
|
||||||
```sql
|
|
||||||
CREATE TABLE notes (
|
|
||||||
noteId TEXT NOT NULL PRIMARY KEY,
|
|
||||||
title TEXT NOT NULL DEFAULT "note",
|
|
||||||
isProtected INT NOT NULL DEFAULT 0,
|
|
||||||
type TEXT NOT NULL DEFAULT 'text',
|
|
||||||
mime TEXT NOT NULL DEFAULT 'text/html',
|
|
||||||
blobId TEXT DEFAULT NULL,
|
|
||||||
isDeleted INT NOT NULL DEFAULT 0,
|
|
||||||
deleteId TEXT DEFAULT NULL,
|
|
||||||
dateCreated TEXT NOT NULL,
|
|
||||||
dateModified TEXT NOT NULL,
|
|
||||||
utcDateCreated TEXT NOT NULL,
|
|
||||||
utcDateModified TEXT NOT NULL
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Indexes for performance
|
|
||||||
CREATE INDEX IDX_notes_title ON notes (title);
|
|
||||||
CREATE INDEX IDX_notes_type ON notes (type);
|
|
||||||
CREATE INDEX IDX_notes_dateCreated ON notes (dateCreated);
|
|
||||||
CREATE INDEX IDX_notes_dateModified ON notes (dateModified);
|
|
||||||
CREATE INDEX IDX_notes_utcDateModified ON notes (utcDateModified);
|
|
||||||
CREATE INDEX IDX_notes_blobId ON notes (blobId);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Field Descriptions:**
|
|
||||||
|
|
||||||
| Field | Type | Description |
|
|
||||||
|-------|------|-------------|
|
|
||||||
| `noteId` | TEXT | Unique identifier (UUID or custom) |
|
|
||||||
| `title` | TEXT | Note title (displayed in tree) |
|
|
||||||
| `isProtected` | INT | 1 if encrypted, 0 if not |
|
|
||||||
| `type` | TEXT | Note type: text, code, file, image, etc. |
|
|
||||||
| `mime` | TEXT | MIME type: text/html, application/json, etc. |
|
|
||||||
| `blobId` | TEXT | Reference to content in blobs table |
|
|
||||||
| `isDeleted` | INT | Soft delete flag |
|
|
||||||
| `deleteId` | TEXT | Unique delete operation ID |
|
|
||||||
| `dateCreated` | TEXT | Creation date (local timezone) |
|
|
||||||
| `dateModified` | TEXT | Last modified (local timezone) |
|
|
||||||
| `utcDateCreated` | TEXT | Creation date (UTC) |
|
|
||||||
| `utcDateModified` | TEXT | Last modified (UTC) |
|
|
||||||
|
|
||||||
**Note Types:**
|
|
||||||
- `text` - Rich text with HTML
|
|
||||||
- `code` - Source code
|
|
||||||
- `file` - Binary file
|
|
||||||
- `image` - Image file
|
|
||||||
- `search` - Saved search
|
|
||||||
- `render` - Custom HTML rendering
|
|
||||||
- `relation-map` - Relationship diagram
|
|
||||||
- `canvas` - Excalidraw drawing
|
|
||||||
- `mermaid` - Mermaid diagram
|
|
||||||
- `book` - Container for documentation
|
|
||||||
- `web-view` - Embedded web page
|
|
||||||
- `mindmap` - Mind map
|
|
||||||
- `geomap` - Geographical map
|
|
||||||
|
|
||||||
### Branches Table
|
|
||||||
|
|
||||||
```sql
|
|
||||||
CREATE TABLE branches (
|
|
||||||
branchId TEXT NOT NULL PRIMARY KEY,
|
|
||||||
noteId TEXT NOT NULL,
|
|
||||||
parentNoteId TEXT NOT NULL,
|
|
||||||
notePosition INTEGER NOT NULL,
|
|
||||||
prefix TEXT,
|
|
||||||
isExpanded INTEGER NOT NULL DEFAULT 0,
|
|
||||||
isDeleted INTEGER NOT NULL DEFAULT 0,
|
|
||||||
deleteId TEXT DEFAULT NULL,
|
|
||||||
utcDateModified TEXT NOT NULL
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Indexes
|
|
||||||
CREATE INDEX IDX_branches_noteId_parentNoteId ON branches (noteId, parentNoteId);
|
|
||||||
CREATE INDEX IDX_branches_parentNoteId ON branches (parentNoteId);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Field Descriptions:**
|
|
||||||
|
|
||||||
| Field | Type | Description |
|
|
||||||
|-------|------|-------------|
|
|
||||||
| `branchId` | TEXT | Unique identifier for this branch |
|
|
||||||
| `noteId` | TEXT | Child note ID |
|
|
||||||
| `parentNoteId` | TEXT | Parent note ID |
|
|
||||||
| `notePosition` | INT | Sort order among siblings |
|
|
||||||
| `prefix` | TEXT | Optional prefix text (e.g., "Chapter 1:") |
|
|
||||||
| `isExpanded` | INT | Tree expansion state |
|
|
||||||
| `isDeleted` | INT | Soft delete flag |
|
|
||||||
| `deleteId` | TEXT | Delete operation ID |
|
|
||||||
| `utcDateModified` | TEXT | Last modified (UTC) |
|
|
||||||
|
|
||||||
**Key Concepts:**
|
|
||||||
- **Cloning:** A note can have multiple branches (multiple parents)
|
|
||||||
- **Position:** Siblings ordered by `notePosition`
|
|
||||||
- **Prefix:** Display text before note title in tree
|
|
||||||
- **Soft Delete:** Allows sync before permanent deletion
|
|
||||||
|
|
||||||
### Attributes Table
|
|
||||||
|
|
||||||
```sql
|
|
||||||
CREATE TABLE attributes (
|
|
||||||
attributeId TEXT NOT NULL PRIMARY KEY,
|
|
||||||
noteId TEXT NOT NULL,
|
|
||||||
type TEXT NOT NULL,
|
|
||||||
name TEXT NOT NULL,
|
|
||||||
value TEXT DEFAULT '' NOT NULL,
|
|
||||||
position INT DEFAULT 0 NOT NULL,
|
|
||||||
utcDateModified TEXT NOT NULL,
|
|
||||||
isDeleted INT NOT NULL,
|
|
||||||
deleteId TEXT DEFAULT NULL,
|
|
||||||
isInheritable INT DEFAULT 0 NULL
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Indexes
|
|
||||||
CREATE INDEX IDX_attributes_name_value ON attributes (name, value);
|
|
||||||
CREATE INDEX IDX_attributes_noteId ON attributes (noteId);
|
|
||||||
CREATE INDEX IDX_attributes_value ON attributes (value);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Field Descriptions:**
|
|
||||||
|
|
||||||
| Field | Type | Description |
|
|
||||||
|-------|------|-------------|
|
|
||||||
| `attributeId` | TEXT | Unique identifier |
|
|
||||||
| `noteId` | TEXT | Note this attribute belongs to |
|
|
||||||
| `type` | TEXT | 'label' or 'relation' |
|
|
||||||
| `name` | TEXT | Attribute name |
|
|
||||||
| `value` | TEXT | Attribute value (text for labels, noteId for relations) |
|
|
||||||
| `position` | INT | Display order |
|
|
||||||
| `utcDateModified` | TEXT | Last modified (UTC) |
|
|
||||||
| `isDeleted` | INT | Soft delete flag |
|
|
||||||
| `deleteId` | TEXT | Delete operation ID |
|
|
||||||
| `isInheritable` | INT | Inherited by child notes |
|
|
||||||
|
|
||||||
**Attribute Types:**
|
|
||||||
|
|
||||||
**Labels** (key-value pairs):
|
|
||||||
```sql
|
|
||||||
-- Example: #priority=high
|
|
||||||
INSERT INTO attributes (attributeId, noteId, type, name, value)
|
|
||||||
VALUES ('attr1', 'note123', 'label', 'priority', 'high')
|
|
||||||
```
|
|
||||||
|
|
||||||
**Relations** (links to other notes):
|
|
||||||
```sql
|
|
||||||
-- Example: ~author=[[noteId]]
|
|
||||||
INSERT INTO attributes (attributeId, noteId, type, name, value)
|
|
||||||
VALUES ('attr2', 'note123', 'relation', 'author', 'author-note-id')
|
|
||||||
```
|
|
||||||
|
|
||||||
**Special Attributes:**
|
|
||||||
- `#run=frontendStartup` - Execute script on frontend load
|
|
||||||
- `#run=backendStartup` - Execute script on backend load
|
|
||||||
- `#customWidget` - Custom widget implementation
|
|
||||||
- `#iconClass` - Custom tree icon
|
|
||||||
- `#cssClass` - CSS class for note
|
|
||||||
- `#sorted` - Auto-sort children
|
|
||||||
- `#hideChildrenOverview` - Don't show child list
|
|
||||||
|
|
||||||
### Revisions Table
|
|
||||||
|
|
||||||
```sql
|
|
||||||
CREATE TABLE revisions (
|
|
||||||
revisionId TEXT NOT NULL PRIMARY KEY,
|
|
||||||
noteId TEXT NOT NULL,
|
|
||||||
type TEXT DEFAULT '' NOT NULL,
|
|
||||||
mime TEXT DEFAULT '' NOT NULL,
|
|
||||||
title TEXT NOT NULL,
|
|
||||||
isProtected INT NOT NULL DEFAULT 0,
|
|
||||||
blobId TEXT DEFAULT NULL,
|
|
||||||
utcDateLastEdited TEXT NOT NULL,
|
|
||||||
utcDateCreated TEXT NOT NULL,
|
|
||||||
utcDateModified TEXT NOT NULL,
|
|
||||||
dateLastEdited TEXT NOT NULL,
|
|
||||||
dateCreated TEXT NOT NULL
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Indexes
|
|
||||||
CREATE INDEX IDX_revisions_noteId ON revisions (noteId);
|
|
||||||
CREATE INDEX IDX_revisions_utcDateCreated ON revisions (utcDateCreated);
|
|
||||||
CREATE INDEX IDX_revisions_utcDateLastEdited ON revisions (utcDateLastEdited);
|
|
||||||
CREATE INDEX IDX_revisions_blobId ON revisions (blobId);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Revision Strategy:**
|
|
||||||
- Automatic revision created on note modification
|
|
||||||
- Configurable interval (default: daily max)
|
|
||||||
- Stores complete note snapshot
|
|
||||||
- Allows reverting to previous versions
|
|
||||||
- Can be disabled with `#disableVersioning`
|
|
||||||
|
|
||||||
### Attachments Table
|
|
||||||
|
|
||||||
```sql
|
|
||||||
CREATE TABLE attachments (
|
|
||||||
attachmentId TEXT NOT NULL PRIMARY KEY,
|
|
||||||
ownerId TEXT NOT NULL,
|
|
||||||
role TEXT NOT NULL,
|
|
||||||
mime TEXT NOT NULL,
|
|
||||||
title TEXT NOT NULL,
|
|
||||||
isProtected INT NOT NULL DEFAULT 0,
|
|
||||||
position INT DEFAULT 0 NOT NULL,
|
|
||||||
blobId TEXT DEFAULT NULL,
|
|
||||||
dateModified TEXT NOT NULL,
|
|
||||||
utcDateModified TEXT NOT NULL,
|
|
||||||
utcDateScheduledForErasureSince TEXT DEFAULT NULL,
|
|
||||||
isDeleted INT NOT NULL,
|
|
||||||
deleteId TEXT DEFAULT NULL
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Indexes
|
|
||||||
CREATE INDEX IDX_attachments_ownerId_role ON attachments (ownerId, role);
|
|
||||||
CREATE INDEX IDX_attachments_blobId ON attachments (blobId);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Attachment Roles:**
|
|
||||||
- `file` - Regular file attachment
|
|
||||||
- `image` - Image file
|
|
||||||
- `cover-image` - Note cover image
|
|
||||||
- Custom roles for specific purposes
|
|
||||||
|
|
||||||
### Blobs Table
|
|
||||||
|
|
||||||
```sql
|
|
||||||
CREATE TABLE blobs (
|
|
||||||
blobId TEXT NOT NULL PRIMARY KEY,
|
|
||||||
content TEXT NULL DEFAULT NULL,
|
|
||||||
dateModified TEXT NOT NULL,
|
|
||||||
utcDateModified TEXT NOT NULL
|
|
||||||
);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Blob Usage:**
|
|
||||||
- Stores actual content (text or binary)
|
|
||||||
- Referenced by notes, revisions, attachments
|
|
||||||
- Deduplication via hash-based blobId
|
|
||||||
- TEXT type stores both text and binary (base64)
|
|
||||||
|
|
||||||
**Content Types:**
|
|
||||||
- **Text notes:** HTML content
|
|
||||||
- **Code notes:** Plain text source code
|
|
||||||
- **Binary notes:** Base64 encoded data
|
|
||||||
- **Protected notes:** Encrypted content
|
|
||||||
|
|
||||||
## System Tables
|
|
||||||
|
|
||||||
### Options Table
|
|
||||||
|
|
||||||
```sql
|
|
||||||
CREATE TABLE options (
|
|
||||||
name TEXT NOT NULL PRIMARY KEY,
|
|
||||||
value TEXT NOT NULL,
|
|
||||||
isSynced INTEGER DEFAULT 0 NOT NULL,
|
|
||||||
utcDateModified TEXT NOT NULL
|
|
||||||
);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Key Options:**
|
|
||||||
- `documentId` - Unique installation ID
|
|
||||||
- `dbVersion` - Schema version
|
|
||||||
- `syncVersion` - Sync protocol version
|
|
||||||
- `passwordVerificationHash` - Password verification
|
|
||||||
- `encryptedDataKey` - Encryption key (encrypted)
|
|
||||||
- `theme` - UI theme
|
|
||||||
- Various feature flags and settings
|
|
||||||
|
|
||||||
**Synced Options:**
|
|
||||||
- `isSynced = 1` - Synced across devices
|
|
||||||
- `isSynced = 0` - Local to this installation
|
|
||||||
|
|
||||||
### Entity Changes Table
|
|
||||||
|
|
||||||
```sql
|
|
||||||
CREATE TABLE entity_changes (
|
|
||||||
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
|
|
||||||
entityName TEXT NOT NULL,
|
|
||||||
entityId TEXT NOT NULL,
|
|
||||||
hash TEXT NOT NULL,
|
|
||||||
isErased INT NOT NULL,
|
|
||||||
changeId TEXT NOT NULL,
|
|
||||||
componentId TEXT NOT NULL,
|
|
||||||
instanceId TEXT NOT NULL,
|
|
||||||
isSynced INTEGER NOT NULL,
|
|
||||||
utcDateChanged TEXT NOT NULL
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Indexes
|
|
||||||
CREATE UNIQUE INDEX IDX_entityChanges_entityName_entityId
|
|
||||||
ON entity_changes (entityName, entityId);
|
|
||||||
CREATE INDEX IDX_entity_changes_changeId ON entity_changes (changeId);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Purpose:** Track all entity modifications for synchronization
|
|
||||||
|
|
||||||
**Entity Types:**
|
|
||||||
- `notes`
|
|
||||||
- `branches`
|
|
||||||
- `attributes`
|
|
||||||
- `revisions`
|
|
||||||
- `attachments`
|
|
||||||
- `options`
|
|
||||||
- `etapi_tokens`
|
|
||||||
|
|
||||||
### Recent Notes Table
|
|
||||||
|
|
||||||
```sql
|
|
||||||
CREATE TABLE recent_notes (
|
|
||||||
noteId TEXT NOT NULL PRIMARY KEY,
|
|
||||||
notePath TEXT NOT NULL,
|
|
||||||
utcDateCreated TEXT NOT NULL
|
|
||||||
);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Purpose:** Track recently accessed notes for quick access
|
|
||||||
|
|
||||||
### Sessions Table
|
|
||||||
|
|
||||||
```sql
|
|
||||||
CREATE TABLE sessions (
|
|
||||||
sid TEXT PRIMARY KEY,
|
|
||||||
sess TEXT NOT NULL,
|
|
||||||
expired TEXT NOT NULL
|
|
||||||
);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Purpose:** HTTP session storage for web interface
|
|
||||||
|
|
||||||
### User Data Table
|
|
||||||
|
|
||||||
```sql
|
|
||||||
CREATE TABLE user_data (
|
|
||||||
tmpID INT PRIMARY KEY,
|
|
||||||
username TEXT,
|
|
||||||
email TEXT,
|
|
||||||
userIDEncryptedDataKey TEXT,
|
|
||||||
userIDVerificationHash TEXT,
|
|
||||||
salt TEXT,
|
|
||||||
derivedKey TEXT,
|
|
||||||
isSetup TEXT DEFAULT "false"
|
|
||||||
);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Purpose:** Store user authentication credentials
|
|
||||||
|
|
||||||
### ETAPI Tokens Table
|
|
||||||
|
|
||||||
```sql
|
|
||||||
CREATE TABLE etapi_tokens (
|
|
||||||
etapiTokenId TEXT PRIMARY KEY NOT NULL,
|
|
||||||
name TEXT NOT NULL,
|
|
||||||
tokenHash TEXT NOT NULL,
|
|
||||||
utcDateCreated TEXT NOT NULL,
|
|
||||||
utcDateModified TEXT NOT NULL,
|
|
||||||
isDeleted INT NOT NULL DEFAULT 0
|
|
||||||
);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Purpose:** API token authentication for external access
|
|
||||||
|
|
||||||
## Data Relationships
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
graph TB
|
|
||||||
Notes[Notes]
|
|
||||||
Branches[Branches]
|
|
||||||
Attributes[Attributes]
|
|
||||||
Attachments[Attachments]
|
|
||||||
Blobs[(Blobs)]
|
|
||||||
Revisions[Revisions]
|
|
||||||
|
|
||||||
Notes --> Branches
|
|
||||||
Notes --> Attributes
|
|
||||||
Notes --> Attachments
|
|
||||||
Notes --> Blobs
|
|
||||||
Notes --> Revisions
|
|
||||||
|
|
||||||
Branches --> Blobs
|
|
||||||
Attachments --> Blobs
|
|
||||||
Revisions --> Blobs
|
|
||||||
|
|
||||||
style Notes fill:#e1f5ff
|
|
||||||
style Blobs fill:#ffe1e1
|
|
||||||
```
|
|
||||||
|
|
||||||
**Relationships:**
|
|
||||||
- Notes ↔ Branches (many-to-many via noteId)
|
|
||||||
- Notes → Attributes (one-to-many)
|
|
||||||
- Notes → Blobs (one-to-one)
|
|
||||||
- Notes → Revisions (one-to-many)
|
|
||||||
- Notes → Attachments (one-to-many)
|
|
||||||
- Attachments → Blobs (one-to-one)
|
|
||||||
- Revisions → Blobs (one-to-one)
|
|
||||||
|
|
||||||
## Database Access Patterns
|
|
||||||
|
|
||||||
### Direct SQL Access
|
|
||||||
|
|
||||||
**Location:** `apps/server/src/services/sql.ts`
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
// Execute query (returns rows)
|
|
||||||
const notes = sql.getRows('SELECT * FROM notes WHERE type = ?', ['text'])
|
|
||||||
|
|
||||||
// Execute query (returns single row)
|
|
||||||
const note = sql.getRow('SELECT * FROM notes WHERE noteId = ?', [noteId])
|
|
||||||
|
|
||||||
// Execute statement (no return)
|
|
||||||
sql.execute('UPDATE notes SET title = ? WHERE noteId = ?', [title, noteId])
|
|
||||||
|
|
||||||
// Insert
|
|
||||||
sql.insert('notes', {
|
|
||||||
noteId: 'new-note-id',
|
|
||||||
title: 'New Note',
|
|
||||||
type: 'text',
|
|
||||||
// ...
|
|
||||||
})
|
|
||||||
|
|
||||||
// Transactions
|
|
||||||
sql.transactional(() => {
|
|
||||||
sql.execute('UPDATE ...')
|
|
||||||
sql.execute('INSERT ...')
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
### Entity-Based Access (Recommended)
|
|
||||||
|
|
||||||
**Via Becca Cache:**
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
// Get entity from cache
|
|
||||||
const note = becca.getNote(noteId)
|
|
||||||
|
|
||||||
// Modify and save
|
|
||||||
note.title = 'Updated Title'
|
|
||||||
note.save() // Writes to database
|
|
||||||
|
|
||||||
// Create new
|
|
||||||
const newNote = becca.createNote({
|
|
||||||
parentNoteId: 'root',
|
|
||||||
title: 'New Note',
|
|
||||||
type: 'text',
|
|
||||||
content: 'Hello World'
|
|
||||||
})
|
|
||||||
|
|
||||||
// Delete
|
|
||||||
note.markAsDeleted()
|
|
||||||
```
|
|
||||||
|
|
||||||
## Database Migrations
|
|
||||||
|
|
||||||
**Location:** `apps/server/src/migrations/`
|
|
||||||
|
|
||||||
**Migration Files:**
|
|
||||||
- Format: `XXXX_migration_name.sql` or `XXXX_migration_name.js`
|
|
||||||
- Executed in numerical order
|
|
||||||
- Version tracked in `options.dbVersion`
|
|
||||||
|
|
||||||
**SQL Migration Example:**
|
|
||||||
```sql
|
|
||||||
-- 0280_add_new_column.sql
|
|
||||||
ALTER TABLE notes ADD COLUMN newField TEXT DEFAULT NULL;
|
|
||||||
|
|
||||||
UPDATE options SET value = '280' WHERE name = 'dbVersion';
|
|
||||||
```
|
|
||||||
|
|
||||||
**JavaScript Migration Example:**
|
|
||||||
```javascript
|
|
||||||
// 0285_complex_migration.js
|
|
||||||
module.exports = () => {
|
|
||||||
const notes = sql.getRows('SELECT * FROM notes WHERE type = ?', ['old-type'])
|
|
||||||
|
|
||||||
for (const note of notes) {
|
|
||||||
sql.execute('UPDATE notes SET type = ? WHERE noteId = ?',
|
|
||||||
['new-type', note.noteId])
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Migration Process:**
|
|
||||||
1. Server checks `dbVersion` on startup
|
|
||||||
2. Compares with latest migration number
|
|
||||||
3. Executes pending migrations in order
|
|
||||||
4. Updates `dbVersion` after each
|
|
||||||
5. Restarts if migrations ran
|
|
||||||
|
|
||||||
## Database Maintenance
|
|
||||||
|
|
||||||
### Backup
|
|
||||||
|
|
||||||
**Full Backup:**
|
|
||||||
```bash
|
|
||||||
# Copy database file
|
|
||||||
cp document.db document.db.backup
|
|
||||||
|
|
||||||
# Or use Trilium's backup feature
|
|
||||||
# Settings → Backup
|
|
||||||
```
|
|
||||||
|
|
||||||
**Automatic Backups:**
|
|
||||||
- Daily backup (configurable)
|
|
||||||
- Stored in `backup/` directory
|
|
||||||
- Retention policy (keep last N backups)
|
|
||||||
|
|
||||||
### Vacuum
|
|
||||||
|
|
||||||
**Purpose:** Reclaim unused space, defragment
|
|
||||||
|
|
||||||
```sql
|
|
||||||
VACUUM;
|
|
||||||
```
|
|
||||||
|
|
||||||
**When to vacuum:**
|
|
||||||
- After deleting many notes
|
|
||||||
- Database file size larger than expected
|
|
||||||
- Performance degradation
|
|
||||||
|
|
||||||
### Integrity Check
|
|
||||||
|
|
||||||
```sql
|
|
||||||
PRAGMA integrity_check;
|
|
||||||
```
|
|
||||||
|
|
||||||
**Result:** "ok" or list of errors
|
|
||||||
|
|
||||||
### Consistency Checks
|
|
||||||
|
|
||||||
**Built-in Consistency Checks:**
|
|
||||||
|
|
||||||
Location: `apps/server/src/services/consistency_checks.ts`
|
|
||||||
|
|
||||||
- Orphaned branches
|
|
||||||
- Missing parent notes
|
|
||||||
- Circular dependencies
|
|
||||||
- Invalid entity references
|
|
||||||
- Blob reference integrity
|
|
||||||
|
|
||||||
**Run Checks:**
|
|
||||||
```typescript
|
|
||||||
// Via API
|
|
||||||
POST /api/consistency-check
|
|
||||||
|
|
||||||
// Or from backend script
|
|
||||||
api.runConsistencyChecks()
|
|
||||||
```
|
|
||||||
|
|
||||||
## Performance Optimization
|
|
||||||
|
|
||||||
### Indexes
|
|
||||||
|
|
||||||
**Existing Indexes:**
|
|
||||||
- `notes.title` - Fast title searches
|
|
||||||
- `notes.type` - Filter by type
|
|
||||||
- `notes.dateCreated/Modified` - Time-based queries
|
|
||||||
- `branches.noteId_parentNoteId` - Tree navigation
|
|
||||||
- `attributes.name_value` - Attribute searches
|
|
||||||
|
|
||||||
**Query Optimization:**
|
|
||||||
```sql
|
|
||||||
-- Use indexed columns in WHERE clause
|
|
||||||
SELECT * FROM notes WHERE type = 'text' -- Uses index
|
|
||||||
|
|
||||||
-- Avoid functions on indexed columns
|
|
||||||
SELECT * FROM notes WHERE LOWER(title) = 'test' -- No index
|
|
||||||
|
|
||||||
-- Better
|
|
||||||
SELECT * FROM notes WHERE title = 'Test' -- Uses index
|
|
||||||
```
|
|
||||||
|
|
||||||
### Connection Settings
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
// apps/server/src/services/sql.ts
|
|
||||||
const db = new Database('document.db', {
|
|
||||||
// Enable WAL mode for better concurrency
|
|
||||||
verbose: console.log
|
|
||||||
})
|
|
||||||
|
|
||||||
db.pragma('journal_mode = WAL')
|
|
||||||
db.pragma('synchronous = NORMAL')
|
|
||||||
db.pragma('cache_size = -64000') // 64MB cache
|
|
||||||
db.pragma('temp_store = MEMORY')
|
|
||||||
```
|
|
||||||
|
|
||||||
**WAL Mode Benefits:**
|
|
||||||
- Better concurrency (readers don't block writers)
|
|
||||||
- Faster commits
|
|
||||||
- More robust
|
|
||||||
|
|
||||||
### Query Performance
|
|
||||||
|
|
||||||
**Use EXPLAIN QUERY PLAN:**
|
|
||||||
```sql
|
|
||||||
EXPLAIN QUERY PLAN
|
|
||||||
SELECT * FROM notes
|
|
||||||
WHERE type = 'text'
|
|
||||||
AND dateCreated > '2025-01-01'
|
|
||||||
```
|
|
||||||
|
|
||||||
**Analyze slow queries:**
|
|
||||||
- Check index usage
|
|
||||||
- Avoid SELECT *
|
|
||||||
- Use prepared statements
|
|
||||||
- Batch operations in transactions
|
|
||||||
|
|
||||||
## Database Size Management
|
|
||||||
|
|
||||||
**Typical Sizes:**
|
|
||||||
- 1,000 notes: ~5-10 MB
|
|
||||||
- 10,000 notes: ~50-100 MB
|
|
||||||
- 100,000 notes: ~500 MB - 1 GB
|
|
||||||
|
|
||||||
**Size Reduction Strategies:**
|
|
||||||
|
|
||||||
1. **Delete old revisions**
|
|
||||||
2. **Remove large attachments**
|
|
||||||
3. **Vacuum database**
|
|
||||||
4. **Compact blobs**
|
|
||||||
5. **Archive old notes**
|
|
||||||
|
|
||||||
**Blob Deduplication:**
|
|
||||||
- Blobs identified by content hash
|
|
||||||
- Identical content shares one blob
|
|
||||||
- Automatic deduplication on insert
|
|
||||||
|
|
||||||
## Security Considerations
|
|
||||||
|
|
||||||
### Protected Notes Encryption
|
|
||||||
|
|
||||||
**Encryption Process:**
|
|
||||||
```typescript
|
|
||||||
// Encrypt blob content
|
|
||||||
const encryptedContent = encrypt(content, dataKey)
|
|
||||||
blob.content = encryptedContent
|
|
||||||
|
|
||||||
// Store encrypted
|
|
||||||
sql.insert('blobs', { blobId, content: encryptedContent })
|
|
||||||
```
|
|
||||||
|
|
||||||
**Encryption Details:**
|
|
||||||
- Algorithm: AES-256-CBC
|
|
||||||
- Key derivation: PBKDF2 (10,000 iterations)
|
|
||||||
- Per-note encryption
|
|
||||||
- Master key encrypted with user password
|
|
||||||
|
|
||||||
### SQL Injection Prevention
|
|
||||||
|
|
||||||
**Always use parameterized queries:**
|
|
||||||
```typescript
|
|
||||||
// GOOD - Safe from SQL injection
|
|
||||||
sql.execute('SELECT * FROM notes WHERE title = ?', [userInput])
|
|
||||||
|
|
||||||
// BAD - Vulnerable to SQL injection
|
|
||||||
sql.execute(`SELECT * FROM notes WHERE title = '${userInput}'`)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Database File Protection
|
|
||||||
|
|
||||||
**File Permissions:**
|
|
||||||
- Owner read/write only
|
|
||||||
- No group/other access
|
|
||||||
- Located in user-specific directory
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**See Also:**
|
|
||||||
- [ARCHITECTURE.md](ARCHITECTURE.md) - Overall architecture
|
|
||||||
- [Database Schema Files](Developer%20Guide/Developer%20Guide/Development%20and%20architecture/Database/)
|
|
||||||
- [Migration Scripts](../apps/server/src/migrations/)
|
|
||||||
124
docs/Developer Guide/!!!meta.json
vendored
124
docs/Developer Guide/!!!meta.json
vendored
@@ -169,25 +169,6 @@
|
|||||||
"dataFileName": "Architecture.md",
|
"dataFileName": "Architecture.md",
|
||||||
"attachments": [],
|
"attachments": [],
|
||||||
"dirFileName": "Architecture",
|
"dirFileName": "Architecture",
|
||||||
"children": [
|
|
||||||
{
|
|
||||||
"isClone": false,
|
|
||||||
"noteId": "2DJZgzpTJ078",
|
|
||||||
"notePath": [
|
|
||||||
"jdjRLhLV3TtI",
|
|
||||||
"MhwWMgxwDTZL",
|
|
||||||
"2DJZgzpTJ078"
|
|
||||||
],
|
|
||||||
"title": "Client-server architecture",
|
|
||||||
"notePosition": 10,
|
|
||||||
"prefix": null,
|
|
||||||
"isExpanded": false,
|
|
||||||
"type": "text",
|
|
||||||
"mime": "text/html",
|
|
||||||
"attributes": [],
|
|
||||||
"format": "markdown",
|
|
||||||
"attachments": [],
|
|
||||||
"dirFileName": "Client-server architecture",
|
|
||||||
"children": [
|
"children": [
|
||||||
{
|
{
|
||||||
"isClone": false,
|
"isClone": false,
|
||||||
@@ -195,7 +176,6 @@
|
|||||||
"notePath": [
|
"notePath": [
|
||||||
"jdjRLhLV3TtI",
|
"jdjRLhLV3TtI",
|
||||||
"MhwWMgxwDTZL",
|
"MhwWMgxwDTZL",
|
||||||
"2DJZgzpTJ078",
|
|
||||||
"dsMq2EIOMOBU"
|
"dsMq2EIOMOBU"
|
||||||
],
|
],
|
||||||
"title": "Frontend",
|
"title": "Frontend",
|
||||||
@@ -204,7 +184,15 @@
|
|||||||
"isExpanded": false,
|
"isExpanded": false,
|
||||||
"type": "text",
|
"type": "text",
|
||||||
"mime": "text/html",
|
"mime": "text/html",
|
||||||
"attributes": [],
|
"attributes": [
|
||||||
|
{
|
||||||
|
"type": "label",
|
||||||
|
"name": "shareAlias",
|
||||||
|
"value": "frontend",
|
||||||
|
"isInheritable": false,
|
||||||
|
"position": 20
|
||||||
|
}
|
||||||
|
],
|
||||||
"format": "markdown",
|
"format": "markdown",
|
||||||
"dataFileName": "Frontend.md",
|
"dataFileName": "Frontend.md",
|
||||||
"attachments": []
|
"attachments": []
|
||||||
@@ -215,7 +203,6 @@
|
|||||||
"notePath": [
|
"notePath": [
|
||||||
"jdjRLhLV3TtI",
|
"jdjRLhLV3TtI",
|
||||||
"MhwWMgxwDTZL",
|
"MhwWMgxwDTZL",
|
||||||
"2DJZgzpTJ078",
|
|
||||||
"tsswRlmHEnYW"
|
"tsswRlmHEnYW"
|
||||||
],
|
],
|
||||||
"title": "Backend",
|
"title": "Backend",
|
||||||
@@ -224,12 +211,18 @@
|
|||||||
"isExpanded": false,
|
"isExpanded": false,
|
||||||
"type": "text",
|
"type": "text",
|
||||||
"mime": "text/html",
|
"mime": "text/html",
|
||||||
"attributes": [],
|
"attributes": [
|
||||||
|
{
|
||||||
|
"type": "label",
|
||||||
|
"name": "shareAlias",
|
||||||
|
"value": "backend",
|
||||||
|
"isInheritable": false,
|
||||||
|
"position": 20
|
||||||
|
}
|
||||||
|
],
|
||||||
"format": "markdown",
|
"format": "markdown",
|
||||||
"dataFileName": "Backend.md",
|
"dataFileName": "Backend.md",
|
||||||
"attachments": []
|
"attachments": []
|
||||||
}
|
|
||||||
]
|
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"isClone": false,
|
"isClone": false,
|
||||||
@@ -240,7 +233,7 @@
|
|||||||
"pRZhrVIGCbMu"
|
"pRZhrVIGCbMu"
|
||||||
],
|
],
|
||||||
"title": "Database",
|
"title": "Database",
|
||||||
"notePosition": 20,
|
"notePosition": 40,
|
||||||
"prefix": null,
|
"prefix": null,
|
||||||
"isExpanded": false,
|
"isExpanded": false,
|
||||||
"type": "text",
|
"type": "text",
|
||||||
@@ -785,15 +778,23 @@
|
|||||||
"MhwWMgxwDTZL",
|
"MhwWMgxwDTZL",
|
||||||
"Wxn82Em8B7U5"
|
"Wxn82Em8B7U5"
|
||||||
],
|
],
|
||||||
"title": "API",
|
"title": "APIs",
|
||||||
"notePosition": 30,
|
"notePosition": 50,
|
||||||
"prefix": null,
|
"prefix": null,
|
||||||
"isExpanded": false,
|
"isExpanded": false,
|
||||||
"type": "text",
|
"type": "text",
|
||||||
"mime": "text/html",
|
"mime": "text/html",
|
||||||
"attributes": [],
|
"attributes": [
|
||||||
|
{
|
||||||
|
"type": "label",
|
||||||
|
"name": "shareAlias",
|
||||||
|
"value": "api",
|
||||||
|
"isInheritable": false,
|
||||||
|
"position": 20
|
||||||
|
}
|
||||||
|
],
|
||||||
"format": "markdown",
|
"format": "markdown",
|
||||||
"dataFileName": "API.md",
|
"dataFileName": "APIs.md",
|
||||||
"attachments": []
|
"attachments": []
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -805,7 +806,7 @@
|
|||||||
"Vk4zD1Iirarg"
|
"Vk4zD1Iirarg"
|
||||||
],
|
],
|
||||||
"title": "Arhitecture Decision Records",
|
"title": "Arhitecture Decision Records",
|
||||||
"notePosition": 40,
|
"notePosition": 60,
|
||||||
"prefix": null,
|
"prefix": null,
|
||||||
"isExpanded": false,
|
"isExpanded": false,
|
||||||
"type": "text",
|
"type": "text",
|
||||||
@@ -817,6 +818,13 @@
|
|||||||
"value": "Jg7clqogFOyD",
|
"value": "Jg7clqogFOyD",
|
||||||
"isInheritable": false,
|
"isInheritable": false,
|
||||||
"position": 20
|
"position": 20
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "label",
|
||||||
|
"name": "shareAlias",
|
||||||
|
"value": "adr",
|
||||||
|
"isInheritable": false,
|
||||||
|
"position": 30
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"format": "markdown",
|
"format": "markdown",
|
||||||
@@ -825,14 +833,14 @@
|
|||||||
},
|
},
|
||||||
{
|
{
|
||||||
"isClone": false,
|
"isClone": false,
|
||||||
"noteId": "QW1MB7RZB5Gf",
|
"noteId": "RHbKw3xiwk3S",
|
||||||
"notePath": [
|
"notePath": [
|
||||||
"jdjRLhLV3TtI",
|
"jdjRLhLV3TtI",
|
||||||
"MhwWMgxwDTZL",
|
"MhwWMgxwDTZL",
|
||||||
"QW1MB7RZB5Gf"
|
"RHbKw3xiwk3S"
|
||||||
],
|
],
|
||||||
"title": "Security Architecture",
|
"title": "Security",
|
||||||
"notePosition": 50,
|
"notePosition": 80,
|
||||||
"prefix": null,
|
"prefix": null,
|
||||||
"isExpanded": false,
|
"isExpanded": false,
|
||||||
"type": "text",
|
"type": "text",
|
||||||
@@ -841,13 +849,13 @@
|
|||||||
{
|
{
|
||||||
"type": "label",
|
"type": "label",
|
||||||
"name": "shareAlias",
|
"name": "shareAlias",
|
||||||
"value": "security-architecture",
|
"value": "security",
|
||||||
"isInheritable": false,
|
"isInheritable": false,
|
||||||
"position": 20
|
"position": 20
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"format": "markdown",
|
"format": "markdown",
|
||||||
"dataFileName": "Security Architecture.md",
|
"dataFileName": "Security.md",
|
||||||
"attachments": []
|
"attachments": []
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
@@ -1153,6 +1161,13 @@
|
|||||||
"value": "bx bx-rocket",
|
"value": "bx bx-rocket",
|
||||||
"isInheritable": false,
|
"isInheritable": false,
|
||||||
"position": 30
|
"position": 30
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "label",
|
||||||
|
"name": "shareAlias",
|
||||||
|
"value": "releasing",
|
||||||
|
"isInheritable": false,
|
||||||
|
"position": 40
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"format": "markdown",
|
"format": "markdown",
|
||||||
@@ -1181,6 +1196,13 @@
|
|||||||
"value": "bx bxs-component",
|
"value": "bx bxs-component",
|
||||||
"isInheritable": false,
|
"isInheritable": false,
|
||||||
"position": 20
|
"position": 20
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "label",
|
||||||
|
"name": "shareAlias",
|
||||||
|
"value": "dependencies",
|
||||||
|
"isInheritable": false,
|
||||||
|
"position": 30
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"format": "markdown",
|
"format": "markdown",
|
||||||
@@ -1527,6 +1549,13 @@
|
|||||||
"value": "bx bx-microchip",
|
"value": "bx bx-microchip",
|
||||||
"isInheritable": false,
|
"isInheritable": false,
|
||||||
"position": 20
|
"position": 20
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "label",
|
||||||
|
"name": "shareAlias",
|
||||||
|
"value": "cache",
|
||||||
|
"isInheritable": false,
|
||||||
|
"position": 30
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"format": "markdown",
|
"format": "markdown",
|
||||||
@@ -2001,7 +2030,15 @@
|
|||||||
"isExpanded": false,
|
"isExpanded": false,
|
||||||
"type": "text",
|
"type": "text",
|
||||||
"mime": "text/html",
|
"mime": "text/html",
|
||||||
"attributes": [],
|
"attributes": [
|
||||||
|
{
|
||||||
|
"type": "label",
|
||||||
|
"name": "shareAlias",
|
||||||
|
"value": "note-types",
|
||||||
|
"isInheritable": false,
|
||||||
|
"position": 20
|
||||||
|
}
|
||||||
|
],
|
||||||
"format": "markdown",
|
"format": "markdown",
|
||||||
"attachments": [],
|
"attachments": [],
|
||||||
"dirFileName": "Note Types",
|
"dirFileName": "Note Types",
|
||||||
@@ -2547,6 +2584,7 @@
|
|||||||
}
|
}
|
||||||
],
|
],
|
||||||
"format": "markdown",
|
"format": "markdown",
|
||||||
|
"dataFileName": "Synchronisation.md",
|
||||||
"attachments": [],
|
"attachments": [],
|
||||||
"dirFileName": "Synchronisation",
|
"dirFileName": "Synchronisation",
|
||||||
"children": [
|
"children": [
|
||||||
@@ -2794,7 +2832,15 @@
|
|||||||
"isExpanded": false,
|
"isExpanded": false,
|
||||||
"type": "text",
|
"type": "text",
|
||||||
"mime": "text/html",
|
"mime": "text/html",
|
||||||
"attributes": [],
|
"attributes": [
|
||||||
|
{
|
||||||
|
"type": "label",
|
||||||
|
"name": "shareAlias",
|
||||||
|
"value": "unit-tests",
|
||||||
|
"isInheritable": false,
|
||||||
|
"position": 20
|
||||||
|
}
|
||||||
|
],
|
||||||
"format": "markdown",
|
"format": "markdown",
|
||||||
"dataFileName": "Unit tests.md",
|
"dataFileName": "Unit tests.md",
|
||||||
"attachments": []
|
"attachments": []
|
||||||
|
|||||||
@@ -119,3 +119,84 @@ server → client → commons
|
|||||||
client → ckeditor5, codemirror, highlightjs
|
client → ckeditor5, codemirror, highlightjs
|
||||||
ckeditor5 → ckeditor5-* plugins
|
ckeditor5 → ckeditor5-* plugins
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Security summary
|
||||||
|
|
||||||
|
### Encryption System
|
||||||
|
|
||||||
|
**Per-Note Encryption:**
|
||||||
|
|
||||||
|
* Notes can be individually protected
|
||||||
|
* AES-128-CBC encryption for encrypted notes.
|
||||||
|
* Separate protected session management
|
||||||
|
|
||||||
|
**Protected Session:**
|
||||||
|
|
||||||
|
* Time-limited access to protected notes
|
||||||
|
* Automatic timeout
|
||||||
|
* Re-authentication required
|
||||||
|
* Frontend: `protected_session.ts`
|
||||||
|
* Backend: `protected_session.ts`
|
||||||
|
|
||||||
|
### Authentication
|
||||||
|
|
||||||
|
**Password Auth:**
|
||||||
|
|
||||||
|
* PBKDF2 key derivation
|
||||||
|
* Salt per installation
|
||||||
|
* Hash verification
|
||||||
|
|
||||||
|
**OpenID Connect:**
|
||||||
|
|
||||||
|
* External identity provider support
|
||||||
|
* OAuth 2.0 flow
|
||||||
|
* Configurable providers
|
||||||
|
|
||||||
|
**TOTP (2FA):**
|
||||||
|
|
||||||
|
* Time-based one-time passwords
|
||||||
|
* QR code setup
|
||||||
|
* Backup codes
|
||||||
|
|
||||||
|
### Authorization
|
||||||
|
|
||||||
|
**Single-User Model:**
|
||||||
|
|
||||||
|
* Desktop: single user (owner)
|
||||||
|
* Server: single user per installation
|
||||||
|
|
||||||
|
**Share Notes:**
|
||||||
|
|
||||||
|
* Public access without authentication
|
||||||
|
* Separate Shaca cache
|
||||||
|
* Read-only access
|
||||||
|
|
||||||
|
### CSRF Protection
|
||||||
|
|
||||||
|
**CSRF Tokens:**
|
||||||
|
|
||||||
|
* Required for state-changing operations
|
||||||
|
* Token in header or cookie
|
||||||
|
* Validation middleware
|
||||||
|
|
||||||
|
### Input Sanitization
|
||||||
|
|
||||||
|
**XSS Prevention:**
|
||||||
|
|
||||||
|
* DOMPurify for HTML sanitization
|
||||||
|
* CKEditor content filtering
|
||||||
|
* CSP headers
|
||||||
|
|
||||||
|
**SQL Injection:**
|
||||||
|
|
||||||
|
* Parameterized queries only
|
||||||
|
* Better-sqlite3 prepared statements
|
||||||
|
* No string concatenation in SQL
|
||||||
|
|
||||||
|
### Dependency Security
|
||||||
|
|
||||||
|
**Vulnerability Scanning:**
|
||||||
|
|
||||||
|
* Renovate bot for updates
|
||||||
|
* npm audit integration
|
||||||
|
* Override vulnerable sub-dependencies
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
# API
|
# APIs
|
||||||
### Internal API
|
### Internal API
|
||||||
|
|
||||||
**REST Endpoints** (`/api/*`)
|
**REST Endpoints** (`/api/*`)
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
# Database
|
# Database
|
||||||
Trilium uses **SQLite** as its database engine, managed via `better-sqlite3`.
|
Trilium uses **SQLite** (via `better-sqlite3`) as its embedded database engine, providing a reliable, file-based storage system that requires no separate database server. The database stores all notes, their relationships, metadata, and configuration.
|
||||||
|
|
||||||
Schema location: `apps/server/src/assets/db/schema.sql`
|
Schema location: `apps/server/src/assets/db/schema.sql`
|
||||||
|
|
||||||
|
|||||||
@@ -1,79 +0,0 @@
|
|||||||
# Security Architecture
|
|
||||||
### Encryption System
|
|
||||||
|
|
||||||
**Per-Note Encryption:**
|
|
||||||
|
|
||||||
* Notes can be individually protected
|
|
||||||
* AES-128-CBC encryption for encrypted notes.
|
|
||||||
* Separate protected session management
|
|
||||||
|
|
||||||
**Protected Session:**
|
|
||||||
|
|
||||||
* Time-limited access to protected notes
|
|
||||||
* Automatic timeout
|
|
||||||
* Re-authentication required
|
|
||||||
* Frontend: `protected_session.ts`
|
|
||||||
* Backend: `protected_session.ts`
|
|
||||||
|
|
||||||
### Authentication
|
|
||||||
|
|
||||||
**Password Auth:**
|
|
||||||
|
|
||||||
* PBKDF2 key derivation
|
|
||||||
* Salt per installation
|
|
||||||
* Hash verification
|
|
||||||
|
|
||||||
**OpenID Connect:**
|
|
||||||
|
|
||||||
* External identity provider support
|
|
||||||
* OAuth 2.0 flow
|
|
||||||
* Configurable providers
|
|
||||||
|
|
||||||
**TOTP (2FA):**
|
|
||||||
|
|
||||||
* Time-based one-time passwords
|
|
||||||
* QR code setup
|
|
||||||
* Backup codes
|
|
||||||
|
|
||||||
### Authorization
|
|
||||||
|
|
||||||
**Single-User Model:**
|
|
||||||
|
|
||||||
* Desktop: single user (owner)
|
|
||||||
* Server: single user per installation
|
|
||||||
|
|
||||||
**Share Notes:**
|
|
||||||
|
|
||||||
* Public access without authentication
|
|
||||||
* Separate Shaca cache
|
|
||||||
* Read-only access
|
|
||||||
|
|
||||||
### CSRF Protection
|
|
||||||
|
|
||||||
**CSRF Tokens:**
|
|
||||||
|
|
||||||
* Required for state-changing operations
|
|
||||||
* Token in header or cookie
|
|
||||||
* Validation middleware
|
|
||||||
|
|
||||||
### Input Sanitization
|
|
||||||
|
|
||||||
**XSS Prevention:**
|
|
||||||
|
|
||||||
* DOMPurify for HTML sanitization
|
|
||||||
* CKEditor content filtering
|
|
||||||
* CSP headers
|
|
||||||
|
|
||||||
**SQL Injection:**
|
|
||||||
|
|
||||||
* Parameterized queries only
|
|
||||||
* Better-sqlite3 prepared statements
|
|
||||||
* No string concatenation in SQL
|
|
||||||
|
|
||||||
### Dependency Security
|
|
||||||
|
|
||||||
**Vulnerability Scanning:**
|
|
||||||
|
|
||||||
* Renovate bot for updates
|
|
||||||
* npm audit integration
|
|
||||||
* Override vulnerable sub-dependencies
|
|
||||||
464
docs/Developer Guide/Developer Guide/Architecture/Security.md
vendored
Normal file
464
docs/Developer Guide/Developer Guide/Architecture/Security.md
vendored
Normal file
@@ -0,0 +1,464 @@
|
|||||||
|
# Security
|
||||||
|
Trilium implements a **defense-in-depth security model** with multiple layers of protection for user data. The security architecture covers authentication, authorization, encryption, input sanitization, and secure communication.
|
||||||
|
|
||||||
|
## Security Principles
|
||||||
|
|
||||||
|
1. **Data Privacy**: User data is protected at rest and in transit
|
||||||
|
2. **Encryption**: Per-note encryption for sensitive content
|
||||||
|
3. **Authentication**: Multiple authentication methods supported
|
||||||
|
4. **Authorization**: Single-user model with granular protected sessions
|
||||||
|
5. **Input Validation**: All user input sanitized
|
||||||
|
6. **Secure Defaults**: Security features enabled by default
|
||||||
|
7. **Transparency**: Open source allows security audits
|
||||||
|
|
||||||
|
## Threat Model
|
||||||
|
|
||||||
|
### Threats Considered
|
||||||
|
|
||||||
|
1. **Unauthorized Access**
|
||||||
|
* Physical access to device
|
||||||
|
* Network eavesdropping
|
||||||
|
* Stolen credentials
|
||||||
|
* Session hijacking
|
||||||
|
2. **Data Exfiltration**
|
||||||
|
* Malicious scripts
|
||||||
|
* XSS attacks
|
||||||
|
* SQL injection
|
||||||
|
* CSRF attacks
|
||||||
|
3. **Data Corruption**
|
||||||
|
* Malicious modifications
|
||||||
|
* Database tampering
|
||||||
|
* Sync conflicts
|
||||||
|
4. **Privacy Leaks**
|
||||||
|
* Unencrypted backups
|
||||||
|
* Search indexing
|
||||||
|
* Temporary files
|
||||||
|
* Memory dumps
|
||||||
|
|
||||||
|
### Out of Scope
|
||||||
|
|
||||||
|
* Nation-state attackers
|
||||||
|
* Zero-day vulnerabilities in dependencies
|
||||||
|
* Hardware vulnerabilities (Spectre, Meltdown)
|
||||||
|
* Physical access with unlimited time
|
||||||
|
* Quantum computing attacks
|
||||||
|
|
||||||
|
## Authentication
|
||||||
|
|
||||||
|
### Password Authentication
|
||||||
|
|
||||||
|
**Implementation:** `apps/server/src/services/password.ts`
|
||||||
|
|
||||||
|
### TOTP (Two-Factor Authentication)
|
||||||
|
|
||||||
|
**Implementation:** `apps/server/src/routes/api/login.ts`
|
||||||
|
|
||||||
|
### OpenID Connect
|
||||||
|
|
||||||
|
**Implementation:** `apps/server/src/routes/api/login.ts`
|
||||||
|
|
||||||
|
**Supported Providers:**
|
||||||
|
|
||||||
|
* Any OpenID Connect compatible provider
|
||||||
|
* Google, GitHub, Auth0, etc.
|
||||||
|
|
||||||
|
**Flow:**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// 1. Redirect to provider
|
||||||
|
GET /api/login/openid
|
||||||
|
|
||||||
|
// 2. Provider redirects back with code
|
||||||
|
GET /api/login/openid/callback?code=...
|
||||||
|
|
||||||
|
// 3. Exchange code for tokens
|
||||||
|
const tokens = await openidClient.callback(redirectUri, req.query)
|
||||||
|
|
||||||
|
// 4. Verify ID token
|
||||||
|
const claims = tokens.claims()
|
||||||
|
|
||||||
|
// 5. Create session
|
||||||
|
req.session.loggedIn = true
|
||||||
|
```
|
||||||
|
|
||||||
|
### Session Management
|
||||||
|
|
||||||
|
**Session Storage:** SQLite database (sessions table)
|
||||||
|
|
||||||
|
**Session Configuration:**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
app.use(session({
|
||||||
|
secret: sessionSecret,
|
||||||
|
resave: false,
|
||||||
|
saveUninitialized: false,
|
||||||
|
rolling: true,
|
||||||
|
cookie: {
|
||||||
|
maxAge: 7 * 24 * 60 * 60 * 1000, // 7 days
|
||||||
|
httpOnly: true,
|
||||||
|
secure: isHttps,
|
||||||
|
sameSite: 'lax'
|
||||||
|
},
|
||||||
|
store: new SqliteStore({
|
||||||
|
db: db,
|
||||||
|
table: 'sessions'
|
||||||
|
})
|
||||||
|
}))
|
||||||
|
```
|
||||||
|
|
||||||
|
**Session Invalidation:**
|
||||||
|
|
||||||
|
* Automatic timeout after inactivity
|
||||||
|
* Manual logout clears session
|
||||||
|
* Server restart invalidates all sessions (optional)
|
||||||
|
|
||||||
|
## Authorization
|
||||||
|
|
||||||
|
### Single-User Model
|
||||||
|
|
||||||
|
**Desktop:**
|
||||||
|
|
||||||
|
* Single user (owner of device)
|
||||||
|
* No multi-user support
|
||||||
|
* Full access to all notes
|
||||||
|
|
||||||
|
**Server:**
|
||||||
|
|
||||||
|
* Single user per installation
|
||||||
|
* Authentication required for all operations
|
||||||
|
* No user roles or permissions
|
||||||
|
|
||||||
|
### Protected Sessions
|
||||||
|
|
||||||
|
**Purpose:** Temporary access to encrypted (protected) notes
|
||||||
|
|
||||||
|
**Implementation:** `apps/server/src/services/protected_session.ts`
|
||||||
|
|
||||||
|
**Workflow:**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// 1. User enters password for protected notes
|
||||||
|
POST /api/protected-session/enter
|
||||||
|
Body: { password: "protected-password" }
|
||||||
|
|
||||||
|
// 2. Derive encryption key
|
||||||
|
const protectedDataKey = deriveKey(password)
|
||||||
|
|
||||||
|
// 3. Verify password (decrypt known encrypted value)
|
||||||
|
const decrypted = decrypt(testValue, protectedDataKey)
|
||||||
|
if (decrypted === expectedValue) {
|
||||||
|
// 4. Store in memory (not in session)
|
||||||
|
protectedSessionHolder.setProtectedDataKey(protectedDataKey)
|
||||||
|
|
||||||
|
// 5. Set timeout
|
||||||
|
setTimeout(() => {
|
||||||
|
protectedSessionHolder.clearProtectedDataKey()
|
||||||
|
}, timeout)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Protected Session Timeout:**
|
||||||
|
|
||||||
|
* Default: 10 minutes (configurable)
|
||||||
|
* Extends on activity
|
||||||
|
* Cleared on browser close
|
||||||
|
* Separate from main session
|
||||||
|
|
||||||
|
### API Authorization
|
||||||
|
|
||||||
|
**Internal API:**
|
||||||
|
|
||||||
|
* Requires authenticated session
|
||||||
|
* CSRF token validation
|
||||||
|
* Same-origin policy
|
||||||
|
|
||||||
|
**ETAPI (External API):**
|
||||||
|
|
||||||
|
* Token-based authentication
|
||||||
|
* No session required
|
||||||
|
* Rate limiting
|
||||||
|
|
||||||
|
## Encryption
|
||||||
|
|
||||||
|
### Note Encryption
|
||||||
|
|
||||||
|
**Encryption Algorithm:** AES-256-CBC
|
||||||
|
|
||||||
|
**Key Hierarchy:**
|
||||||
|
|
||||||
|
```
|
||||||
|
User Password
|
||||||
|
↓ (scrypt)
|
||||||
|
Data Key (for protected notes)
|
||||||
|
↓ (AES-128)
|
||||||
|
Protected Note Content
|
||||||
|
```
|
||||||
|
|
||||||
|
**Protected Note Metadata:**
|
||||||
|
|
||||||
|
* Content IS encrypted
|
||||||
|
* Type and MIME are NOT encrypted
|
||||||
|
* Attributes are NOT encrypted
|
||||||
|
|
||||||
|
### Data Key Management
|
||||||
|
|
||||||
|
**Key Rotation:**
|
||||||
|
|
||||||
|
* Not currently supported
|
||||||
|
* Requires re-encrypting all protected notes
|
||||||
|
|
||||||
|
### Transport Encryption
|
||||||
|
|
||||||
|
**HTTPS:**
|
||||||
|
|
||||||
|
* Recommended for server installations
|
||||||
|
* TLS 1.2+ only
|
||||||
|
* Strong cipher suites preferred
|
||||||
|
* Certificate validation enabled
|
||||||
|
|
||||||
|
**Desktop:**
|
||||||
|
|
||||||
|
* Local communication (no network)
|
||||||
|
* No HTTPS required
|
||||||
|
|
||||||
|
### Backup Encryption
|
||||||
|
|
||||||
|
**Database Backups:**
|
||||||
|
|
||||||
|
* Protected notes remain encrypted in backup
|
||||||
|
* Backup file should be protected separately
|
||||||
|
* Consider encrypting backup storage location
|
||||||
|
|
||||||
|
## Input Sanitization
|
||||||
|
|
||||||
|
### XSS Prevention
|
||||||
|
|
||||||
|
* **HTML Sanitization**
|
||||||
|
* **CKEditor Configuration:**
|
||||||
|
|
||||||
|
```
|
||||||
|
// apps/client/src/widgets/type_widgets/text_type_widget.ts
|
||||||
|
ClassicEditor.create(element, {
|
||||||
|
// Restrict allowed content
|
||||||
|
htmlSupport: {
|
||||||
|
allow: [
|
||||||
|
{ name: /./, attributes: true, classes: true, styles: true }
|
||||||
|
],
|
||||||
|
disallow: [
|
||||||
|
{ name: 'script' },
|
||||||
|
{ name: 'iframe', attributes: /^(?!src$).*/ }
|
||||||
|
]
|
||||||
|
}
|
||||||
|
})
|
||||||
|
```
|
||||||
|
* Content Security Policy
|
||||||
|
|
||||||
|
### SQL Injection Prevention
|
||||||
|
|
||||||
|
**Parameterized Queries:**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
const notes = sql.getRows(
|
||||||
|
'SELECT * FROM notes WHERE title = ?',
|
||||||
|
[userInput]
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
**ORM Usage:**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// Entity-based access prevents SQL injection
|
||||||
|
const note = becca.getNote(noteId)
|
||||||
|
note.title = userInput // Sanitized by entity
|
||||||
|
note.save() // Parameterized query
|
||||||
|
```
|
||||||
|
|
||||||
|
### CSRF Prevention
|
||||||
|
|
||||||
|
**CSRF Token Validation:**
|
||||||
|
|
||||||
|
Location: `apps/server/src/routes/csrf_protection.ts`
|
||||||
|
|
||||||
|
Stateless CSRF using [Double Submit Cookie Pattern](https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html#double-submit-cookie) via [`csrf-csrf`](https://github.com/Psifi-Solutions/csrf-csrf).
|
||||||
|
|
||||||
|
### File Upload Validation
|
||||||
|
|
||||||
|
**Validation:**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// Validate file size
|
||||||
|
const maxSize = 100 * 1024 * 1024 // 100 MB
|
||||||
|
if (file.size > maxSize) {
|
||||||
|
throw new Error('File too large')
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Network Security
|
||||||
|
|
||||||
|
### HTTPS Configuration
|
||||||
|
|
||||||
|
**Certificate Validation:**
|
||||||
|
|
||||||
|
* Require valid certificates in production
|
||||||
|
* Self-signed certificates allowed for development
|
||||||
|
* Certificate pinning not implemented
|
||||||
|
|
||||||
|
### Rate Limiting
|
||||||
|
|
||||||
|
**Login Rate Limiting:**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
const loginLimiter = rateLimit({
|
||||||
|
windowMs: 15 * 60 * 1000,
|
||||||
|
max: 10, // 10 failed attempts
|
||||||
|
skipSuccessfulRequests: true
|
||||||
|
})
|
||||||
|
|
||||||
|
app.post('/api/login/password', loginLimiter, loginHandler)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Data Security
|
||||||
|
|
||||||
|
### Secure Data Deletion
|
||||||
|
|
||||||
|
**Soft Delete:**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// Mark as deleted (sync first)
|
||||||
|
note.isDeleted = 1
|
||||||
|
note.deleteId = generateUUID()
|
||||||
|
note.save()
|
||||||
|
|
||||||
|
// Entity change tracked for sync
|
||||||
|
addEntityChange('notes', noteId, note)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Hard Delete (Erase):**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// After sync completed
|
||||||
|
sql.execute('DELETE FROM notes WHERE noteId = ?', [noteId])
|
||||||
|
sql.execute('DELETE FROM branches WHERE noteId = ?', [noteId])
|
||||||
|
sql.execute('DELETE FROM attributes WHERE noteId = ?', [noteId])
|
||||||
|
|
||||||
|
// Mark entity change as erased
|
||||||
|
sql.execute('UPDATE entity_changes SET isErased = 1 WHERE entityId = ?', [noteId])
|
||||||
|
```
|
||||||
|
|
||||||
|
**Blob Cleanup:**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// Find orphaned blobs (not referenced by any note/revision/attachment)
|
||||||
|
const orphanedBlobs = sql.getRows(`
|
||||||
|
SELECT blobId FROM blobs
|
||||||
|
WHERE blobId NOT IN (SELECT blobId FROM notes WHERE blobId IS NOT NULL)
|
||||||
|
AND blobId NOT IN (SELECT blobId FROM revisions WHERE blobId IS NOT NULL)
|
||||||
|
AND blobId NOT IN (SELECT blobId FROM attachments WHERE blobId IS NOT NULL)
|
||||||
|
`)
|
||||||
|
|
||||||
|
// Delete orphaned blobs
|
||||||
|
for (const blob of orphanedBlobs) {
|
||||||
|
sql.execute('DELETE FROM blobs WHERE blobId = ?', [blob.blobId])
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Memory Security
|
||||||
|
|
||||||
|
**Protected Data in Memory:**
|
||||||
|
|
||||||
|
* Protected data keys stored in memory only
|
||||||
|
* Cleared on timeout
|
||||||
|
* Not written to disk
|
||||||
|
* Not in session storage
|
||||||
|
|
||||||
|
## Dependency Security
|
||||||
|
|
||||||
|
### Vulnerability Scanning
|
||||||
|
|
||||||
|
**Tools:**
|
||||||
|
|
||||||
|
* Renovate bot - Automatic dependency updates
|
||||||
|
* `pnpm audit` - Check for known vulnerabilities
|
||||||
|
* GitHub Dependabot alerts
|
||||||
|
|
||||||
|
**Process:**
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# Check for vulnerabilities
|
||||||
|
npm audit
|
||||||
|
|
||||||
|
# Fix automatically
|
||||||
|
npm audit fix
|
||||||
|
|
||||||
|
# Manual review for breaking changes
|
||||||
|
npm audit fix --force
|
||||||
|
```
|
||||||
|
|
||||||
|
### Dependency Pinning
|
||||||
|
|
||||||
|
**package.json:**
|
||||||
|
|
||||||
|
```
|
||||||
|
{
|
||||||
|
"dependencies": {
|
||||||
|
"express": "4.18.2", // Exact version
|
||||||
|
"better-sqlite3": "^9.2.2" // Compatible versions
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**pnpm Overrides:**
|
||||||
|
|
||||||
|
```
|
||||||
|
{
|
||||||
|
"pnpm": {
|
||||||
|
"overrides": {
|
||||||
|
"lodash@<4.17.21": ">=4.17.21", // Force minimum version
|
||||||
|
"axios@<0.21.2": ">=0.21.2"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Patch Management
|
||||||
|
|
||||||
|
**pnpm Patches:**
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# Create patch
|
||||||
|
pnpm patch @ckeditor/ckeditor5
|
||||||
|
|
||||||
|
# Edit files in temporary directory
|
||||||
|
# ...
|
||||||
|
|
||||||
|
# Generate patch file
|
||||||
|
pnpm patch-commit /tmp/ckeditor5-patch
|
||||||
|
|
||||||
|
# Patch applied automatically on install
|
||||||
|
```
|
||||||
|
|
||||||
|
## Security Auditing
|
||||||
|
|
||||||
|
### Logs
|
||||||
|
|
||||||
|
**Security Events Logged:**
|
||||||
|
|
||||||
|
* Login attempts (success/failure)
|
||||||
|
* Protected session access
|
||||||
|
* Password changes
|
||||||
|
* ETAPI token usage
|
||||||
|
* Failed CSRF validations
|
||||||
|
|
||||||
|
**Log Location:**
|
||||||
|
|
||||||
|
* Desktop: Console output
|
||||||
|
* Server: Log files or stdout
|
||||||
|
|
||||||
|
### Monitoring
|
||||||
|
|
||||||
|
**Metrics to Monitor:**
|
||||||
|
|
||||||
|
* Failed login attempts
|
||||||
|
* API error rates
|
||||||
|
* Unusual database changes
|
||||||
|
* Large exports/imports
|
||||||
@@ -1,16 +1,11 @@
|
|||||||
# Trilium Synchronization Architecture
|
# Synchronisation
|
||||||
|
Trilium implements a **bidirectional synchronization system** that allows users to sync their note databases across multiple devices (desktop clients and server instances). The sync protocol is designed to handle:
|
||||||
|
|
||||||
> **Related:** [ARCHITECTURE.md](ARCHITECTURE.md) | [User Guide: Synchronization](https://triliumnext.github.io/Docs/Wiki/synchronization)
|
* Concurrent modifications across devices
|
||||||
|
* Simple conflict resolution (without “merge conflict” indication).
|
||||||
## Overview
|
* Partial sync (only changed entities)
|
||||||
|
* Protected note synchronization
|
||||||
Trilium implements a sophisticated **bidirectional synchronization system** that allows users to sync their note databases across multiple devices (desktop clients and server instances). The sync protocol is designed to handle:
|
* Efficient bandwidth usage
|
||||||
|
|
||||||
- Concurrent modifications across devices
|
|
||||||
- Conflict resolution
|
|
||||||
- Partial sync (only changed entities)
|
|
||||||
- Protected note synchronization
|
|
||||||
- Efficient bandwidth usage
|
|
||||||
|
|
||||||
## Sync Architecture
|
## Sync Architecture
|
||||||
|
|
||||||
@@ -35,7 +30,7 @@ graph TB
|
|||||||
|
|
||||||
Every modification to any entity (note, branch, attribute, etc.) creates an **entity change** record:
|
Every modification to any entity (note, branch, attribute, etc.) creates an **entity change** record:
|
||||||
|
|
||||||
```sql
|
```
|
||||||
entity_changes (
|
entity_changes (
|
||||||
id, -- Auto-increment ID
|
id, -- Auto-increment ID
|
||||||
entityName, -- 'notes', 'branches', 'attributes', etc.
|
entityName, -- 'notes', 'branches', 'attributes', etc.
|
||||||
@@ -43,7 +38,7 @@ entity_changes (
|
|||||||
hash, -- Content hash for integrity
|
hash, -- Content hash for integrity
|
||||||
isErased, -- If entity was erased (deleted permanently)
|
isErased, -- If entity was erased (deleted permanently)
|
||||||
changeId, -- Unique change identifier
|
changeId, -- Unique change identifier
|
||||||
componentId, -- Installation identifier
|
componentId, -- Unique component/widget identifier
|
||||||
instanceId, -- Process instance identifier
|
instanceId, -- Process instance identifier
|
||||||
isSynced, -- Whether synced to server
|
isSynced, -- Whether synced to server
|
||||||
utcDateChanged -- When change occurred
|
utcDateChanged -- When change occurred
|
||||||
@@ -51,17 +46,19 @@ entity_changes (
|
|||||||
```
|
```
|
||||||
|
|
||||||
**Key Properties:**
|
**Key Properties:**
|
||||||
- **changeId**: Globally unique identifier (UUID) for the change
|
|
||||||
- **componentId**: Unique per Trilium installation (persists across restarts)
|
* **changeId**: Globally unique identifier (UUID) for the change
|
||||||
- **instanceId**: Unique per process (changes on restart)
|
* **componentId**: Unique identifier of the component/widget that generated to change (can be used to avoid refreshing the widget being edited).
|
||||||
- **hash**: SHA-256 hash of entity data for integrity verification
|
* **instanceId**: Unique per process (changes on restart)
|
||||||
|
* **hash**: SHA-256 hash of entity data for integrity verification
|
||||||
|
|
||||||
### Sync Versions
|
### Sync Versions
|
||||||
|
|
||||||
Each Trilium installation tracks:
|
Each Trilium installation tracks:
|
||||||
- **Local sync version**: Highest change ID seen locally
|
|
||||||
- **Server sync version**: Highest change ID on server
|
* **Local sync version**: Highest change ID seen locally
|
||||||
- **Entity versions**: Last sync version for each entity type
|
* **Server sync version**: Highest change ID on server
|
||||||
|
* **Entity versions**: Last sync version for each entity type
|
||||||
|
|
||||||
### Change Tracking
|
### Change Tracking
|
||||||
|
|
||||||
@@ -87,11 +84,12 @@ function addEntityChange(entityName, entityId, entity) {
|
|||||||
```
|
```
|
||||||
|
|
||||||
**Entity modification triggers:**
|
**Entity modification triggers:**
|
||||||
- Note content update
|
|
||||||
- Note metadata change
|
* Note content update
|
||||||
- Branch creation/deletion/reorder
|
* Note metadata change
|
||||||
- Attribute addition/removal
|
* Branch creation/deletion/reorder
|
||||||
- Options modification
|
* Attribute addition/removal
|
||||||
|
* Options modification
|
||||||
|
|
||||||
## Sync Protocol
|
## Sync Protocol
|
||||||
|
|
||||||
@@ -122,9 +120,9 @@ Response:
|
|||||||
|
|
||||||
**Step 3: Decision**
|
**Step 3: Decision**
|
||||||
|
|
||||||
- If `entityChanges > 0`: Pull changes from server
|
* If `entityChanges > 0`: Pull changes from server
|
||||||
- If `outstandingPushCount > 0`: Push changes to server
|
* If `outstandingPushCount > 0`: Push changes to server
|
||||||
- Both can happen in sequence
|
* Both can happen in sequence
|
||||||
|
|
||||||
### Pull Sync (Server → Client)
|
### Pull Sync (Server → Client)
|
||||||
|
|
||||||
@@ -215,21 +213,25 @@ if (serverLastModified > clientSyncVersion) {
|
|||||||
|
|
||||||
### Conflict Types
|
### Conflict Types
|
||||||
|
|
||||||
**1. Content Conflict**
|
**1\. Content Conflict**
|
||||||
- Both client and server modified same note content
|
|
||||||
- **Resolution**: Last-write-wins based on `utcDateModified`
|
|
||||||
|
|
||||||
**2. Structure Conflict**
|
* Both client and server modified same note content
|
||||||
- Branch moved/deleted on one side, modified on other
|
* **Resolution**: Last-write-wins based on `utcDateModified`
|
||||||
- **Resolution**: Tombstone records, reconciliation
|
|
||||||
|
|
||||||
**3. Attribute Conflict**
|
**2\. Structure Conflict**
|
||||||
- Same attribute modified differently
|
|
||||||
- **Resolution**: Last-write-wins
|
* Branch moved/deleted on one side, modified on other
|
||||||
|
* **Resolution**: Tombstone records, reconciliation
|
||||||
|
|
||||||
|
**3\. Attribute Conflict**
|
||||||
|
|
||||||
|
* Same attribute modified differently
|
||||||
|
* **Resolution**: Last-write-wins
|
||||||
|
|
||||||
### Conflict Resolution Strategy
|
### Conflict Resolution Strategy
|
||||||
|
|
||||||
**Last-Write-Wins:**
|
**Last-Write-Wins:**
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
if (clientEntity.utcDateModified > serverEntity.utcDateModified) {
|
if (clientEntity.utcDateModified > serverEntity.utcDateModified) {
|
||||||
// Client wins, apply client changes
|
// Client wins, apply client changes
|
||||||
@@ -241,9 +243,10 @@ if (clientEntity.utcDateModified > serverEntity.utcDateModified) {
|
|||||||
```
|
```
|
||||||
|
|
||||||
**Tombstone Records:**
|
**Tombstone Records:**
|
||||||
- Deleted entities leave tombstone in `entity_changes`
|
|
||||||
- Prevents re-sync of deleted items
|
* Deleted entities leave tombstone in `entity_changes`
|
||||||
- `isErased = 1` for permanent deletions
|
* Prevents re-sync of deleted items
|
||||||
|
* `isErased = 1` for permanent deletions
|
||||||
|
|
||||||
### Protected Notes Sync
|
### Protected Notes Sync
|
||||||
|
|
||||||
@@ -251,43 +254,26 @@ if (clientEntity.utcDateModified > serverEntity.utcDateModified) {
|
|||||||
|
|
||||||
**Solution:**
|
**Solution:**
|
||||||
|
|
||||||
1. **Protected session required**: User must unlock protected notes
|
1. **Encrypted sync**: Content synced in encrypted form
|
||||||
2. **Encrypted sync**: Content synced in encrypted form
|
2. **Hash verification**: Integrity checked without decryption
|
||||||
3. **Hash verification**: Integrity checked without decryption
|
3. **Lazy decryption**: Only decrypt when accessed
|
||||||
4. **Lazy decryption**: Only decrypt when accessed
|
|
||||||
|
|
||||||
**Sync Flow:**
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
// Client side
|
|
||||||
if (note.isProtected && !protectedSessionHolder.isProtectedSessionAvailable()) {
|
|
||||||
// Skip protected notes if session not active
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
// Server side
|
|
||||||
if (note.isProtected) {
|
|
||||||
// Sync encrypted blob
|
|
||||||
// Don't decrypt for sync
|
|
||||||
syncEncryptedBlob(note.blobId)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Sync States
|
## Sync States
|
||||||
|
|
||||||
### Connection States
|
### Connection States
|
||||||
|
|
||||||
- **Connected**: WebSocket connection active
|
* **Connected**: WebSocket connection active
|
||||||
- **Disconnected**: No connection to sync server
|
* **Disconnected**: No connection to sync server
|
||||||
- **Syncing**: Actively transferring data
|
* **Syncing**: Actively transferring data
|
||||||
- **Conflict**: Sync paused due to conflict
|
* **Conflict**: Sync paused due to conflict
|
||||||
|
|
||||||
### Entity Sync States
|
### Entity Sync States
|
||||||
|
|
||||||
Each entity can be in:
|
Each entity can be in:
|
||||||
- **Synced**: In sync with server
|
|
||||||
- **Pending**: Local changes not yet pushed
|
* **Synced**: In sync with server
|
||||||
- **Conflict**: Conflicting changes detected
|
* **Pending**: Local changes not yet pushed
|
||||||
|
* **Conflict**: Conflicting changes detected
|
||||||
|
|
||||||
### UI Indicators
|
### UI Indicators
|
||||||
|
|
||||||
@@ -299,8 +285,6 @@ class SyncStatusWidget {
|
|||||||
showIcon('synced')
|
showIcon('synced')
|
||||||
} else if (isSyncing) {
|
} else if (isSyncing) {
|
||||||
showIcon('syncing-spinner')
|
showIcon('syncing-spinner')
|
||||||
} else if (hasConflicts) {
|
|
||||||
showIcon('conflict-warning')
|
|
||||||
} else {
|
} else {
|
||||||
showIcon('not-synced')
|
showIcon('not-synced')
|
||||||
}
|
}
|
||||||
@@ -314,7 +298,7 @@ class SyncStatusWidget {
|
|||||||
|
|
||||||
Only entities changed since last sync are transferred:
|
Only entities changed since last sync are transferred:
|
||||||
|
|
||||||
```sql
|
```
|
||||||
SELECT * FROM entity_changes
|
SELECT * FROM entity_changes
|
||||||
WHERE id > :lastSyncedChangeId
|
WHERE id > :lastSyncedChangeId
|
||||||
ORDER BY id ASC
|
ORDER BY id ASC
|
||||||
@@ -357,26 +341,12 @@ res.send(gzip(syncData))
|
|||||||
|
|
||||||
### Network Errors
|
### Network Errors
|
||||||
|
|
||||||
**Retry Strategy:**
|
Reported to the user and the sync will be retried after the interval passes.
|
||||||
```typescript
|
|
||||||
const RETRY_DELAYS = [1000, 2000, 5000, 10000, 30000]
|
|
||||||
|
|
||||||
async function syncWithRetry(attempt = 0) {
|
|
||||||
try {
|
|
||||||
await performSync()
|
|
||||||
} catch (error) {
|
|
||||||
if (attempt < RETRY_DELAYS.length) {
|
|
||||||
setTimeout(() => {
|
|
||||||
syncWithRetry(attempt + 1)
|
|
||||||
}, RETRY_DELAYS[attempt])
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Sync Integrity Checks
|
### Sync Integrity Checks
|
||||||
|
|
||||||
**Hash Verification:**
|
**Hash Verification:**
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
// Verify entity hash matches
|
// Verify entity hash matches
|
||||||
const calculatedHash = calculateHash(entity)
|
const calculatedHash = calculateHash(entity)
|
||||||
@@ -388,16 +358,18 @@ if (calculatedHash !== receivedHash) {
|
|||||||
```
|
```
|
||||||
|
|
||||||
**Consistency Checks:**
|
**Consistency Checks:**
|
||||||
- Orphaned branches detection
|
|
||||||
- Missing parent notes
|
* Orphaned branches detection
|
||||||
- Invalid entity references
|
* Missing parent notes
|
||||||
- Circular dependencies
|
* Invalid entity references
|
||||||
|
* Circular dependencies
|
||||||
|
|
||||||
## Sync Server Configuration
|
## Sync Server Configuration
|
||||||
|
|
||||||
### Server Setup
|
### Server Setup
|
||||||
|
|
||||||
**Required Options:**
|
**Required Options:**
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
{
|
{
|
||||||
"syncServerHost": "https://sync.example.com",
|
"syncServerHost": "https://sync.example.com",
|
||||||
@@ -407,59 +379,26 @@ if (calculatedHash !== receivedHash) {
|
|||||||
```
|
```
|
||||||
|
|
||||||
**Authentication:**
|
**Authentication:**
|
||||||
- Username/password or
|
|
||||||
- Sync token (generated on server)
|
|
||||||
|
|
||||||
### Client Setup
|
* Username/password or
|
||||||
|
* Sync token (generated on server)
|
||||||
**Desktop Client:**
|
|
||||||
```javascript
|
|
||||||
// Settings → Sync
|
|
||||||
{
|
|
||||||
"syncServerHost": "https://sync.example.com",
|
|
||||||
"username": "user@example.com",
|
|
||||||
"password": "********"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Test Connection:**
|
|
||||||
```typescript
|
|
||||||
POST /api/sync/test
|
|
||||||
Response: { "success": true }
|
|
||||||
```
|
|
||||||
|
|
||||||
## Sync API Endpoints
|
## Sync API Endpoints
|
||||||
|
|
||||||
Located at: `apps/server/src/routes/api/sync.ts`
|
Located at: `apps/server/src/routes/api/sync.ts`
|
||||||
|
|
||||||
**Endpoints:**
|
|
||||||
|
|
||||||
- `POST /api/sync/check` - Check sync status
|
|
||||||
- `POST /api/sync/pull` - Pull changes from server
|
|
||||||
- `POST /api/sync/push` - Push changes to server
|
|
||||||
- `POST /api/sync/finished` - Mark sync complete
|
|
||||||
- `POST /api/sync/test` - Test connection
|
|
||||||
- `GET /api/sync/stats` - Sync statistics
|
|
||||||
|
|
||||||
## WebSocket Sync Updates
|
## WebSocket Sync Updates
|
||||||
|
|
||||||
Real-time sync via WebSocket:
|
Real-time sync via WebSocket:
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
// Server broadcasts change to all connected clients
|
// Server broadcasts change to all connected clients
|
||||||
ws.broadcast('entity-change', {
|
ws.broadcast('frontend-update', {
|
||||||
entityName: 'notes',
|
lastSyncedPush,
|
||||||
entityId: 'abc123',
|
entityChanges
|
||||||
changeId: 'change-uuid',
|
|
||||||
sourceId: 'originating-component-id'
|
|
||||||
})
|
})
|
||||||
|
|
||||||
// Client receives and applies
|
// Client receives and processed the information.
|
||||||
ws.on('entity-change', (data) => {
|
|
||||||
if (data.sourceId !== myComponentId) {
|
|
||||||
froca.processEntityChange(data)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Sync Scheduling
|
## Sync Scheduling
|
||||||
@@ -467,107 +406,79 @@ ws.on('entity-change', (data) => {
|
|||||||
### Automatic Sync
|
### Automatic Sync
|
||||||
|
|
||||||
**Desktop:**
|
**Desktop:**
|
||||||
- Sync on startup
|
|
||||||
- Periodic sync (configurable interval, default: 60s)
|
* Sync on startup
|
||||||
- Sync before shutdown
|
* Periodic sync (configurable interval, default: 60s)
|
||||||
|
|
||||||
**Server:**
|
**Server:**
|
||||||
- Sync on entity modification
|
|
||||||
- WebSocket push to connected clients
|
* Sync on entity modification
|
||||||
|
* WebSocket push to connected clients
|
||||||
|
|
||||||
### Manual Sync
|
### Manual Sync
|
||||||
|
|
||||||
User can trigger:
|
User can trigger:
|
||||||
- Full sync
|
|
||||||
- Sync now
|
* Full sync
|
||||||
- Sync specific subtree
|
* Sync now
|
||||||
|
* Sync specific subtree
|
||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
### Common Issues
|
### Common Issues
|
||||||
|
|
||||||
**Sync stuck:**
|
**Sync stuck:**
|
||||||
```sql
|
|
||||||
|
```
|
||||||
-- Reset sync state
|
-- Reset sync state
|
||||||
UPDATE entity_changes SET isSynced = 0;
|
UPDATE entity_changes SET isSynced = 0;
|
||||||
DELETE FROM options WHERE name LIKE 'sync%';
|
DELETE FROM options WHERE name LIKE 'sync%';
|
||||||
```
|
```
|
||||||
|
|
||||||
**Hash mismatch:**
|
**Hash mismatch:**
|
||||||
- Data corruption detected
|
|
||||||
- Re-sync from backup
|
* Data corruption detected
|
||||||
- Check database integrity
|
* Re-sync from backup
|
||||||
|
* Check database integrity
|
||||||
|
|
||||||
**Conflict loop:**
|
**Conflict loop:**
|
||||||
- Manual intervention required
|
|
||||||
- Export conflicting notes
|
|
||||||
- Choose winning version
|
|
||||||
- Re-sync
|
|
||||||
|
|
||||||
### Sync Diagnostics
|
* Manual intervention required
|
||||||
|
* Export conflicting notes
|
||||||
**Check sync status:**
|
* Choose winning version
|
||||||
```typescript
|
* Re-sync
|
||||||
GET /api/sync/stats
|
|
||||||
Response: {
|
|
||||||
"unsyncedChanges": 0,
|
|
||||||
"lastSyncDate": "2025-11-02T12:00:00Z",
|
|
||||||
"syncVersion": 12890
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Entity change log:**
|
|
||||||
```sql
|
|
||||||
SELECT * FROM entity_changes
|
|
||||||
WHERE isSynced = 0
|
|
||||||
ORDER BY id DESC;
|
|
||||||
```
|
|
||||||
|
|
||||||
## Security Considerations
|
## Security Considerations
|
||||||
|
|
||||||
### Encrypted Sync
|
### Encrypted Sync
|
||||||
|
|
||||||
- Protected notes synced encrypted
|
* Protected notes synced encrypted
|
||||||
- No plain text over network
|
* No plain text over network
|
||||||
- Server cannot read protected content
|
* Server cannot read protected content
|
||||||
|
|
||||||
### Authentication
|
### Authentication
|
||||||
|
|
||||||
- Username/password over HTTPS only
|
* Username/password over HTTPS only
|
||||||
- Sync tokens for token-based auth
|
* Sync tokens for token-based auth
|
||||||
- Session cookies with CSRF protection
|
* Session cookies with CSRF protection
|
||||||
|
|
||||||
### Authorization
|
### Authorization
|
||||||
|
|
||||||
- Users can only sync their own data
|
* Users can only sync their own data
|
||||||
- No cross-user sync support
|
* No cross-user sync support
|
||||||
- Sync server validates ownership
|
* Sync server validates ownership
|
||||||
|
|
||||||
## Performance Metrics
|
## Performance Metrics
|
||||||
|
|
||||||
**Typical Sync Performance:**
|
**Typical Sync Performance:**
|
||||||
- 1000 changes: ~2-5 seconds
|
|
||||||
- 10000 changes: ~20-50 seconds
|
* 1000 changes: ~2-5 seconds
|
||||||
- Initial full sync (100k notes): ~5-10 minutes
|
* 10000 changes: ~20-50 seconds
|
||||||
|
* Initial full sync (100k notes): ~5-10 minutes
|
||||||
|
|
||||||
**Factors:**
|
**Factors:**
|
||||||
- Network latency
|
|
||||||
- Database size
|
|
||||||
- Number of protected notes
|
|
||||||
- Attachment sizes
|
|
||||||
|
|
||||||
## Future Improvements
|
* Network latency
|
||||||
|
* Database size
|
||||||
**Planned Enhancements:**
|
* Number of protected notes
|
||||||
- Differential sync (binary diff)
|
* Attachment sizes
|
||||||
- Peer-to-peer sync (no central server)
|
|
||||||
- Multi-server sync
|
|
||||||
- Partial sync (subtree only)
|
|
||||||
- Sync over Tor/I2P
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**See Also:**
|
|
||||||
- [ARCHITECTURE.md](ARCHITECTURE.md) - Overall architecture
|
|
||||||
- [Sync User Guide](https://triliumnext.github.io/Docs/Wiki/synchronization)
|
|
||||||
- [Sync API Source](../apps/server/src/routes/api/sync.ts)
|
|
||||||
155
docs/QUICK_REFERENCE.md
vendored
155
docs/QUICK_REFERENCE.md
vendored
@@ -1,155 +0,0 @@
|
|||||||
# Trilium Technical Documentation - Quick Reference
|
|
||||||
|
|
||||||
> **Start here:** [TECHNICAL_DOCUMENTATION.md](TECHNICAL_DOCUMENTATION.md) - Complete index of all documentation
|
|
||||||
|
|
||||||
## 📖 Documentation Files
|
|
||||||
|
|
||||||
| Document | Description | Size | Lines |
|
|
||||||
|----------|-------------|------|-------|
|
|
||||||
| [TECHNICAL_DOCUMENTATION.md](TECHNICAL_DOCUMENTATION.md) | Main index and navigation hub | 13KB | 423 |
|
|
||||||
| [ARCHITECTURE.md](ARCHITECTURE.md) | Complete system architecture | 30KB | 1,016 |
|
|
||||||
| [DATABASE.md](DATABASE.md) | Database schema and operations | 19KB | 736 |
|
|
||||||
| [SYNCHRONIZATION.md](SYNCHRONIZATION.md) | Sync protocol and implementation | 14KB | 583 |
|
|
||||||
| [SCRIPTING.md](SCRIPTING.md) | User scripting system guide | 17KB | 734 |
|
|
||||||
| [SECURITY_ARCHITECTURE.md](SECURITY_ARCHITECTURE.md) | Security implementation details | 19KB | 834 |
|
|
||||||
|
|
||||||
**Total:** 112KB of comprehensive documentation across 4,326 lines!
|
|
||||||
|
|
||||||
## 🎯 Quick Access by Role
|
|
||||||
|
|
||||||
### 👤 End Users
|
|
||||||
- **Getting Started:** [User Guide](User%20Guide/User%20Guide/)
|
|
||||||
- **Scripting:** [SCRIPTING.md](SCRIPTING.md)
|
|
||||||
- **Sync Setup:** [SYNCHRONIZATION.md](SYNCHRONIZATION.md)
|
|
||||||
|
|
||||||
### 💻 Developers
|
|
||||||
- **Architecture:** [ARCHITECTURE.md](ARCHITECTURE.md)
|
|
||||||
- **Development Setup:** [Developer Guide](Developer%20Guide/Developer%20Guide/Environment%20Setup.md)
|
|
||||||
- **Database:** [DATABASE.md](DATABASE.md)
|
|
||||||
|
|
||||||
### 🔒 Security Auditors
|
|
||||||
- **Security:** [SECURITY_ARCHITECTURE.md](SECURITY_ARCHITECTURE.md)
|
|
||||||
- **Encryption:** [SECURITY_ARCHITECTURE.md#encryption](SECURITY_ARCHITECTURE.md#encryption)
|
|
||||||
- **Auth:** [SECURITY_ARCHITECTURE.md#authentication](SECURITY_ARCHITECTURE.md#authentication)
|
|
||||||
|
|
||||||
### 🏗️ System Architects
|
|
||||||
- **Overall Design:** [ARCHITECTURE.md](ARCHITECTURE.md)
|
|
||||||
- **Cache System:** [ARCHITECTURE.md#three-layer-cache-system](ARCHITECTURE.md#three-layer-cache-system)
|
|
||||||
- **Entity Model:** [ARCHITECTURE.md#entity-system](ARCHITECTURE.md#entity-system)
|
|
||||||
|
|
||||||
### 🔧 DevOps Engineers
|
|
||||||
- **Server Installation:** [User Guide - Server Installation](User%20Guide/User%20Guide/Installation%20&%20Setup/Server%20Installation.md)
|
|
||||||
- **Docker:** [Developer Guide - Docker](Developer%20Guide/Developer%20Guide/Development%20and%20architecture/Docker.md)
|
|
||||||
- **Sync Server:** [SYNCHRONIZATION.md#sync-server-configuration](SYNCHRONIZATION.md#sync-server-configuration)
|
|
||||||
|
|
||||||
### 📊 Database Administrators
|
|
||||||
- **Schema:** [DATABASE.md#database-schema](DATABASE.md#database-schema)
|
|
||||||
- **Maintenance:** [DATABASE.md#database-maintenance](DATABASE.md#database-maintenance)
|
|
||||||
- **Performance:** [DATABASE.md#performance-optimization](DATABASE.md#performance-optimization)
|
|
||||||
|
|
||||||
## 🔍 Quick Topic Finder
|
|
||||||
|
|
||||||
### Core Concepts
|
|
||||||
- **Becca Cache:** [ARCHITECTURE.md#1-becca-backend-cache](ARCHITECTURE.md#1-becca-backend-cache)
|
|
||||||
- **Froca Cache:** [ARCHITECTURE.md#2-froca-frontend-cache](ARCHITECTURE.md#2-froca-frontend-cache)
|
|
||||||
- **Entity System:** [ARCHITECTURE.md#entity-system](ARCHITECTURE.md#entity-system)
|
|
||||||
- **Widget System:** [ARCHITECTURE.md#widget-based-ui](ARCHITECTURE.md#widget-based-ui)
|
|
||||||
|
|
||||||
### Database
|
|
||||||
- **Schema Overview:** [DATABASE.md#schema-overview](DATABASE.md#schema-overview)
|
|
||||||
- **Notes Table:** [DATABASE.md#notes-table](DATABASE.md#notes-table)
|
|
||||||
- **Branches Table:** [DATABASE.md#branches-table](DATABASE.md#branches-table)
|
|
||||||
- **Migrations:** [DATABASE.md#database-migrations](DATABASE.md#database-migrations)
|
|
||||||
|
|
||||||
### Synchronization
|
|
||||||
- **Sync Protocol:** [SYNCHRONIZATION.md#sync-protocol](SYNCHRONIZATION.md#sync-protocol)
|
|
||||||
- **Conflict Resolution:** [SYNCHRONIZATION.md#conflict-resolution](SYNCHRONIZATION.md#conflict-resolution)
|
|
||||||
- **Entity Changes:** [SYNCHRONIZATION.md#entity-changes](SYNCHRONIZATION.md#entity-changes)
|
|
||||||
|
|
||||||
### Scripting
|
|
||||||
- **Frontend Scripts:** [SCRIPTING.md#frontend-scripts](SCRIPTING.md#frontend-scripts)
|
|
||||||
- **Backend Scripts:** [SCRIPTING.md#backend-scripts](SCRIPTING.md#backend-scripts)
|
|
||||||
- **Script Examples:** [SCRIPTING.md#script-examples](SCRIPTING.md#script-examples)
|
|
||||||
- **API Reference:** [SCRIPTING.md#script-api](SCRIPTING.md#script-api)
|
|
||||||
|
|
||||||
### Security
|
|
||||||
- **Authentication:** [SECURITY_ARCHITECTURE.md#authentication](SECURITY_ARCHITECTURE.md#authentication)
|
|
||||||
- **Encryption:** [SECURITY_ARCHITECTURE.md#encryption](SECURITY_ARCHITECTURE.md#encryption)
|
|
||||||
- **Input Sanitization:** [SECURITY_ARCHITECTURE.md#input-sanitization](SECURITY_ARCHITECTURE.md#input-sanitization)
|
|
||||||
- **Best Practices:** [SECURITY_ARCHITECTURE.md#security-best-practices](SECURITY_ARCHITECTURE.md#security-best-practices)
|
|
||||||
|
|
||||||
## 📚 Learning Paths
|
|
||||||
|
|
||||||
### New to Trilium Development
|
|
||||||
1. Read [ARCHITECTURE.md](ARCHITECTURE.md) - System overview
|
|
||||||
2. Setup environment: [Environment Setup](Developer%20Guide/Developer%20Guide/Environment%20Setup.md)
|
|
||||||
3. Explore [DATABASE.md](DATABASE.md) - Understand data model
|
|
||||||
4. Check [Developer Guide](Developer%20Guide/Developer%20Guide/)
|
|
||||||
|
|
||||||
### Want to Create Scripts
|
|
||||||
1. Read [SCRIPTING.md](SCRIPTING.md) - Complete guide
|
|
||||||
2. Check [Script API](Script%20API/) - API reference
|
|
||||||
3. Review examples: [SCRIPTING.md#script-examples](SCRIPTING.md#script-examples)
|
|
||||||
4. Explore [Advanced Showcases](https://triliumnext.github.io/Docs/Wiki/advanced-showcases)
|
|
||||||
|
|
||||||
### Setting Up Sync
|
|
||||||
1. Understand protocol: [SYNCHRONIZATION.md](SYNCHRONIZATION.md)
|
|
||||||
2. Configure server: [SYNCHRONIZATION.md#sync-server-configuration](SYNCHRONIZATION.md#sync-server-configuration)
|
|
||||||
3. Setup clients: [SYNCHRONIZATION.md#client-setup](SYNCHRONIZATION.md#client-setup)
|
|
||||||
4. Troubleshoot: [SYNCHRONIZATION.md#troubleshooting](SYNCHRONIZATION.md#troubleshooting)
|
|
||||||
|
|
||||||
### Security Review
|
|
||||||
1. Read threat model: [SECURITY_ARCHITECTURE.md#threat-model](SECURITY_ARCHITECTURE.md#threat-model)
|
|
||||||
2. Review authentication: [SECURITY_ARCHITECTURE.md#authentication](SECURITY_ARCHITECTURE.md#authentication)
|
|
||||||
3. Check encryption: [SECURITY_ARCHITECTURE.md#encryption](SECURITY_ARCHITECTURE.md#encryption)
|
|
||||||
4. Verify best practices: [SECURITY_ARCHITECTURE.md#security-best-practices](SECURITY_ARCHITECTURE.md#security-best-practices)
|
|
||||||
|
|
||||||
## 🗺️ Documentation Map
|
|
||||||
|
|
||||||
```
|
|
||||||
docs/
|
|
||||||
├── TECHNICAL_DOCUMENTATION.md ← START HERE (Index)
|
|
||||||
│
|
|
||||||
├── Core Technical Docs
|
|
||||||
│ ├── ARCHITECTURE.md (System design)
|
|
||||||
│ ├── DATABASE.md (Data layer)
|
|
||||||
│ ├── SYNCHRONIZATION.md (Sync system)
|
|
||||||
│ ├── SCRIPTING.md (User scripting)
|
|
||||||
│ └── SECURITY_ARCHITECTURE.md (Security)
|
|
||||||
│
|
|
||||||
├── Developer Guide/
|
|
||||||
│ └── Developer Guide/ (Development setup)
|
|
||||||
│
|
|
||||||
├── User Guide/
|
|
||||||
│ └── User Guide/ (End-user docs)
|
|
||||||
│
|
|
||||||
└── Script API/ (API reference)
|
|
||||||
```
|
|
||||||
|
|
||||||
## 💡 Tips for Reading Documentation
|
|
||||||
|
|
||||||
1. **Start with the index:** [TECHNICAL_DOCUMENTATION.md](TECHNICAL_DOCUMENTATION.md) provides an overview
|
|
||||||
2. **Use search:** Press Ctrl+F / Cmd+F to find specific topics
|
|
||||||
3. **Follow links:** Documents are cross-referenced for easy navigation
|
|
||||||
4. **Code examples:** Most docs include practical code examples
|
|
||||||
5. **See Also sections:** Check bottom of each doc for related resources
|
|
||||||
|
|
||||||
## 🔗 External Resources
|
|
||||||
|
|
||||||
- **Website:** https://triliumnotes.org
|
|
||||||
- **Online Docs:** https://docs.triliumnotes.org
|
|
||||||
- **GitHub:** https://github.com/TriliumNext/Trilium
|
|
||||||
- **Discussions:** https://github.com/TriliumNext/Trilium/discussions
|
|
||||||
- **Matrix Chat:** https://matrix.to/#/#triliumnext:matrix.org
|
|
||||||
|
|
||||||
## 🤝 Contributing to Documentation
|
|
||||||
|
|
||||||
Found an error or want to improve the docs? See:
|
|
||||||
- [Contributing Guide](../README.md#-contribute)
|
|
||||||
- [Documentation Standards](TECHNICAL_DOCUMENTATION.md#documentation-conventions)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Version:** 0.99.3
|
|
||||||
**Last Updated:** November 2025
|
|
||||||
**Maintained by:** TriliumNext Team
|
|
||||||
734
docs/SCRIPTING.md
vendored
734
docs/SCRIPTING.md
vendored
@@ -1,734 +0,0 @@
|
|||||||
# Trilium Scripting System
|
|
||||||
|
|
||||||
> **Related:** [ARCHITECTURE.md](ARCHITECTURE.md) | [Script API Documentation](Script%20API/)
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
Trilium features a **powerful scripting system** that allows users to extend and customize the application without modifying source code. Scripts are written in JavaScript and can execute both in the **frontend (browser)** and **backend (Node.js)** contexts.
|
|
||||||
|
|
||||||
## Script Types
|
|
||||||
|
|
||||||
### Frontend Scripts
|
|
||||||
|
|
||||||
**Location:** Attached to notes with `#run=frontendStartup` attribute
|
|
||||||
|
|
||||||
**Execution Context:** Browser environment
|
|
||||||
|
|
||||||
**Access:**
|
|
||||||
- Trilium Frontend API
|
|
||||||
- Browser APIs (DOM, localStorage, etc.)
|
|
||||||
- Froca (frontend cache)
|
|
||||||
- UI widgets
|
|
||||||
- No direct file system access
|
|
||||||
|
|
||||||
**Lifecycle:**
|
|
||||||
- `frontendStartup` - Run once when Trilium loads
|
|
||||||
- `frontendReload` - Run on every note context change
|
|
||||||
|
|
||||||
**Example:**
|
|
||||||
```javascript
|
|
||||||
// Attach to note with #run=frontendStartup
|
|
||||||
const api = window.api
|
|
||||||
|
|
||||||
// Add custom button to toolbar
|
|
||||||
api.addButtonToToolbar({
|
|
||||||
title: 'My Button',
|
|
||||||
icon: 'star',
|
|
||||||
action: () => {
|
|
||||||
api.showMessage('Hello from frontend!')
|
|
||||||
}
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
### Backend Scripts
|
|
||||||
|
|
||||||
**Location:** Attached to notes with `#run=backendStartup` attribute
|
|
||||||
|
|
||||||
**Execution Context:** Node.js server environment
|
|
||||||
|
|
||||||
**Access:**
|
|
||||||
- Trilium Backend API
|
|
||||||
- Node.js APIs (fs, http, etc.)
|
|
||||||
- Becca (backend cache)
|
|
||||||
- Database (SQL)
|
|
||||||
- External libraries (via require)
|
|
||||||
|
|
||||||
**Lifecycle:**
|
|
||||||
- `backendStartup` - Run once when server starts
|
|
||||||
- Event handlers (custom events)
|
|
||||||
|
|
||||||
**Example:**
|
|
||||||
```javascript
|
|
||||||
// Attach to note with #run=backendStartup
|
|
||||||
const api = require('@triliumnext/api')
|
|
||||||
|
|
||||||
// Listen for note creation
|
|
||||||
api.dayjs // Example: access dayjs library
|
|
||||||
|
|
||||||
api.onNoteCreated((note) => {
|
|
||||||
if (note.title.includes('TODO')) {
|
|
||||||
note.setLabel('priority', 'high')
|
|
||||||
}
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
### Render Scripts
|
|
||||||
|
|
||||||
**Location:** Attached to notes with `#customWidget` or similar attributes
|
|
||||||
|
|
||||||
**Purpose:** Custom note rendering/widgets
|
|
||||||
|
|
||||||
**Example:**
|
|
||||||
```javascript
|
|
||||||
// Custom widget for a note
|
|
||||||
class MyWidget extends api.NoteContextAwareWidget {
|
|
||||||
doRender() {
|
|
||||||
this.$widget = $('<div>')
|
|
||||||
.text('Custom widget content')
|
|
||||||
return this.$widget
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
module.exports = MyWidget
|
|
||||||
```
|
|
||||||
|
|
||||||
## Script API
|
|
||||||
|
|
||||||
### Frontend API
|
|
||||||
|
|
||||||
**Location:** `apps/client/src/services/frontend_script_api.ts`
|
|
||||||
|
|
||||||
**Global Access:** `window.api`
|
|
||||||
|
|
||||||
**Key Methods:**
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
// Note Operations
|
|
||||||
api.getNote(noteId) // Get note object
|
|
||||||
api.getBranch(branchId) // Get branch object
|
|
||||||
api.getActiveNote() // Currently displayed note
|
|
||||||
api.openNote(noteId, activateNote) // Open note in UI
|
|
||||||
|
|
||||||
// UI Operations
|
|
||||||
api.showMessage(message) // Show toast notification
|
|
||||||
api.showDialog() // Show modal dialog
|
|
||||||
api.confirm(message) // Show confirmation dialog
|
|
||||||
api.prompt(message, defaultValue) // Show input prompt
|
|
||||||
|
|
||||||
// Tree Operations
|
|
||||||
api.getTree() // Get note tree structure
|
|
||||||
api.expandTree(noteId) // Expand tree branch
|
|
||||||
api.collapseTree(noteId) // Collapse tree branch
|
|
||||||
|
|
||||||
// Search
|
|
||||||
api.searchForNotes(searchQuery) // Search notes
|
|
||||||
api.searchForNote(searchQuery) // Get single note
|
|
||||||
|
|
||||||
// Navigation
|
|
||||||
api.openTabWithNote(noteId) // Open note in new tab
|
|
||||||
api.closeActiveTab() // Close current tab
|
|
||||||
api.activateNote(noteId) // Switch to note
|
|
||||||
|
|
||||||
// Attributes
|
|
||||||
api.getAttribute(noteId, type, name) // Get attribute
|
|
||||||
api.getAttributes(noteId, type, name) // Get all matching attributes
|
|
||||||
|
|
||||||
// Custom Widgets
|
|
||||||
api.addButtonToToolbar(def) // Add toolbar button
|
|
||||||
api.addCustomWidget(def) // Add custom widget
|
|
||||||
|
|
||||||
// Events
|
|
||||||
api.runOnNoteOpened(callback) // Note opened event
|
|
||||||
api.runOnNoteContentChange(callback) // Content changed event
|
|
||||||
|
|
||||||
// Utilities
|
|
||||||
api.dayjs // Date/time library
|
|
||||||
api.formatDate(date) // Format date
|
|
||||||
api.log(message) // Console log
|
|
||||||
```
|
|
||||||
|
|
||||||
### Backend API
|
|
||||||
|
|
||||||
**Location:** `apps/server/src/services/backend_script_api.ts`
|
|
||||||
|
|
||||||
**Access:** `require('@triliumnext/api')` or global `api`
|
|
||||||
|
|
||||||
**Key Methods:**
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
// Note Operations
|
|
||||||
api.getNote(noteId) // Get note from Becca
|
|
||||||
api.getNoteWithContent(noteId) // Get note with content
|
|
||||||
api.createNote(parentNoteId, title) // Create new note
|
|
||||||
api.deleteNote(noteId) // Delete note
|
|
||||||
|
|
||||||
// Branch Operations
|
|
||||||
api.getBranch(branchId) // Get branch
|
|
||||||
api.createBranch(noteId, parentNoteId) // Create branch (clone)
|
|
||||||
|
|
||||||
// Attribute Operations
|
|
||||||
api.getAttribute(noteId, type, name) // Get attribute
|
|
||||||
api.createAttribute(noteId, type, name, value) // Create attribute
|
|
||||||
|
|
||||||
// Database Access
|
|
||||||
api.sql.getRow(query, params) // Execute SQL query (single row)
|
|
||||||
api.sql.getRows(query, params) // Execute SQL query (multiple rows)
|
|
||||||
api.sql.execute(query, params) // Execute SQL statement
|
|
||||||
|
|
||||||
// Events
|
|
||||||
api.onNoteCreated(callback) // Note created event
|
|
||||||
api.onNoteUpdated(callback) // Note updated event
|
|
||||||
api.onNoteDeleted(callback) // Note deleted event
|
|
||||||
api.onAttributeCreated(callback) // Attribute created event
|
|
||||||
|
|
||||||
// Search
|
|
||||||
api.searchForNotes(searchQuery) // Search notes
|
|
||||||
|
|
||||||
// Date/Time
|
|
||||||
api.dayjs // Date/time library
|
|
||||||
api.now() // Current date/time
|
|
||||||
|
|
||||||
// Logging
|
|
||||||
api.log(message) // Log message
|
|
||||||
api.error(message) // Log error
|
|
||||||
|
|
||||||
// External Communication
|
|
||||||
api.axios // HTTP client library
|
|
||||||
|
|
||||||
// Utilities
|
|
||||||
api.backup.backupNow() // Trigger backup
|
|
||||||
api.export.exportSubtree(noteId) // Export notes
|
|
||||||
```
|
|
||||||
|
|
||||||
## Script Attributes
|
|
||||||
|
|
||||||
### Execute Attributes
|
|
||||||
|
|
||||||
- `#run=frontendStartup` - Execute on frontend startup
|
|
||||||
- `#run=backendStartup` - Execute on backend startup
|
|
||||||
- `#run=hourly` - Execute every hour
|
|
||||||
- `#run=daily` - Execute daily
|
|
||||||
|
|
||||||
### Widget Attributes
|
|
||||||
|
|
||||||
- `#customWidget` - Custom note widget
|
|
||||||
- `#widget` - Standard widget integration
|
|
||||||
|
|
||||||
### Other Attributes
|
|
||||||
|
|
||||||
- `#disableVersioning` - Disable automatic versioning for this note
|
|
||||||
- `#hideChildrenOverview` - Hide children in overview
|
|
||||||
- `#iconClass` - Custom icon for note
|
|
||||||
|
|
||||||
## Entity Classes
|
|
||||||
|
|
||||||
### Frontend Entities
|
|
||||||
|
|
||||||
**FNote** (`apps/client/src/entities/fnote.ts`)
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
class FNote {
|
|
||||||
noteId: string
|
|
||||||
title: string
|
|
||||||
type: string
|
|
||||||
mime: string
|
|
||||||
|
|
||||||
// Relationships
|
|
||||||
getParentNotes(): FNote[]
|
|
||||||
getChildNotes(): FNote[]
|
|
||||||
getBranches(): FBranch[]
|
|
||||||
|
|
||||||
// Attributes
|
|
||||||
getAttribute(type, name): FAttribute
|
|
||||||
getAttributes(type?, name?): FAttribute[]
|
|
||||||
hasLabel(name): boolean
|
|
||||||
getLabelValue(name): string
|
|
||||||
|
|
||||||
// Content
|
|
||||||
getContent(): Promise<string>
|
|
||||||
|
|
||||||
// Navigation
|
|
||||||
open(): void
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**FBranch**
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
class FBranch {
|
|
||||||
branchId: string
|
|
||||||
noteId: string
|
|
||||||
parentNoteId: string
|
|
||||||
prefix: string
|
|
||||||
notePosition: number
|
|
||||||
|
|
||||||
getNote(): FNote
|
|
||||||
getParentNote(): FNote
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**FAttribute**
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
class FAttribute {
|
|
||||||
attributeId: string
|
|
||||||
noteId: string
|
|
||||||
type: 'label' | 'relation'
|
|
||||||
name: string
|
|
||||||
value: string
|
|
||||||
|
|
||||||
getNote(): FNote
|
|
||||||
getTargetNote(): FNote // For relations
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Backend Entities
|
|
||||||
|
|
||||||
**BNote** (`apps/server/src/becca/entities/bnote.ts`)
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
class BNote {
|
|
||||||
noteId: string
|
|
||||||
title: string
|
|
||||||
type: string
|
|
||||||
mime: string
|
|
||||||
isProtected: boolean
|
|
||||||
|
|
||||||
// Content
|
|
||||||
getContent(): string | Buffer
|
|
||||||
setContent(content: string | Buffer): void
|
|
||||||
|
|
||||||
// Relationships
|
|
||||||
getParentNotes(): BNote[]
|
|
||||||
getChildNotes(): BNote[]
|
|
||||||
getBranches(): BBranch[]
|
|
||||||
|
|
||||||
// Attributes
|
|
||||||
getAttribute(type, name): BAttribute
|
|
||||||
getAttributes(type?, name?): BAttribute[]
|
|
||||||
setLabel(name, value): BAttribute
|
|
||||||
setRelation(name, targetNoteId): BAttribute
|
|
||||||
hasLabel(name): boolean
|
|
||||||
getLabelValue(name): string
|
|
||||||
|
|
||||||
// Operations
|
|
||||||
save(): void
|
|
||||||
markAsDeleted(): void
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**BBranch**
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
class BBranch {
|
|
||||||
branchId: string
|
|
||||||
noteId: string
|
|
||||||
parentNoteId: string
|
|
||||||
prefix: string
|
|
||||||
notePosition: number
|
|
||||||
|
|
||||||
getNote(): BNote
|
|
||||||
getParentNote(): BNote
|
|
||||||
save(): void
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**BAttribute**
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
class BAttribute {
|
|
||||||
attributeId: string
|
|
||||||
noteId: string
|
|
||||||
type: 'label' | 'relation'
|
|
||||||
name: string
|
|
||||||
value: string
|
|
||||||
|
|
||||||
getNote(): BNote
|
|
||||||
getTargetNote(): BNote // For relations
|
|
||||||
save(): void
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Script Examples
|
|
||||||
|
|
||||||
### Frontend Examples
|
|
||||||
|
|
||||||
**1. Custom Toolbar Button**
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
// #run=frontendStartup
|
|
||||||
api.addButtonToToolbar({
|
|
||||||
title: 'Export to PDF',
|
|
||||||
icon: 'file-export',
|
|
||||||
action: async () => {
|
|
||||||
const note = api.getActiveNote()
|
|
||||||
if (note) {
|
|
||||||
await api.runOnBackend('exportToPdf', [note.noteId])
|
|
||||||
api.showMessage('Export started')
|
|
||||||
}
|
|
||||||
}
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
**2. Auto-Save Reminder**
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
// #run=frontendStartup
|
|
||||||
let saveTimer
|
|
||||||
api.runOnNoteContentChange(() => {
|
|
||||||
clearTimeout(saveTimer)
|
|
||||||
saveTimer = setTimeout(() => {
|
|
||||||
api.showMessage('Remember to save your work!')
|
|
||||||
}, 300000) // 5 minutes
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
**3. Note Statistics Widget**
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
// #customWidget
|
|
||||||
class StatsWidget extends api.NoteContextAwareWidget {
|
|
||||||
doRender() {
|
|
||||||
this.$widget = $('<div class="stats-widget">')
|
|
||||||
return this.$widget
|
|
||||||
}
|
|
||||||
|
|
||||||
async refreshWithNote(note) {
|
|
||||||
const content = await note.getContent()
|
|
||||||
const words = content.split(/\s+/).length
|
|
||||||
const chars = content.length
|
|
||||||
|
|
||||||
this.$widget.html(`
|
|
||||||
<div>Words: ${words}</div>
|
|
||||||
<div>Characters: ${chars}</div>
|
|
||||||
`)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
module.exports = StatsWidget
|
|
||||||
```
|
|
||||||
|
|
||||||
### Backend Examples
|
|
||||||
|
|
||||||
**1. Auto-Tagging on Note Creation**
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
// #run=backendStartup
|
|
||||||
api.onNoteCreated((note) => {
|
|
||||||
// Auto-tag TODO notes
|
|
||||||
if (note.title.includes('TODO')) {
|
|
||||||
note.setLabel('type', 'todo')
|
|
||||||
note.setLabel('priority', 'normal')
|
|
||||||
}
|
|
||||||
|
|
||||||
// Auto-tag meeting notes by date
|
|
||||||
if (note.title.match(/Meeting \d{4}-\d{2}-\d{2}/)) {
|
|
||||||
note.setLabel('type', 'meeting')
|
|
||||||
const dateMatch = note.title.match(/(\d{4}-\d{2}-\d{2})/)
|
|
||||||
if (dateMatch) {
|
|
||||||
note.setLabel('date', dateMatch[1])
|
|
||||||
}
|
|
||||||
}
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
**2. Daily Backup Reminder**
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
// #run=daily
|
|
||||||
const todayNote = api.getTodayNote()
|
|
||||||
todayNote.setLabel('backupDone', 'false')
|
|
||||||
|
|
||||||
// Create reminder note
|
|
||||||
api.createNote(todayNote.noteId, '🔔 Backup Reminder', {
|
|
||||||
content: 'Remember to verify today\'s backup!',
|
|
||||||
type: 'text'
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
**3. External API Integration**
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
// #run=backendStartup
|
|
||||||
api.onNoteCreated(async (note) => {
|
|
||||||
// Sync new notes to external service
|
|
||||||
if (note.hasLabel('sync-external')) {
|
|
||||||
try {
|
|
||||||
await api.axios.post('https://external-api.com/sync', {
|
|
||||||
noteId: note.noteId,
|
|
||||||
title: note.title,
|
|
||||||
content: note.getContent()
|
|
||||||
})
|
|
||||||
note.setLabel('lastSync', api.dayjs().format())
|
|
||||||
} catch (error) {
|
|
||||||
api.log('Sync failed: ' + error.message)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
**4. Database Cleanup**
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
// #run=weekly
|
|
||||||
// Clean up old revisions
|
|
||||||
const cutoffDate = api.dayjs().subtract(90, 'days').format()
|
|
||||||
|
|
||||||
const oldRevisions = api.sql.getRows(`
|
|
||||||
SELECT revisionId FROM revisions
|
|
||||||
WHERE utcDateCreated < ?
|
|
||||||
`, [cutoffDate])
|
|
||||||
|
|
||||||
api.log(`Deleting ${oldRevisions.length} old revisions`)
|
|
||||||
|
|
||||||
for (const row of oldRevisions) {
|
|
||||||
api.sql.execute('DELETE FROM revisions WHERE revisionId = ?', [row.revisionId])
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Script Storage
|
|
||||||
|
|
||||||
**Storage Location:** Scripts are stored as regular notes
|
|
||||||
|
|
||||||
**Identifying Scripts:**
|
|
||||||
- Have `#run` attribute or `#customWidget` attribute
|
|
||||||
- Type is typically `code` with MIME `application/javascript`
|
|
||||||
|
|
||||||
**Script Note Structure:**
|
|
||||||
```
|
|
||||||
📁 Scripts (folder note)
|
|
||||||
├── 📜 Frontend Scripts
|
|
||||||
│ ├── Custom Toolbar Button (#run=frontendStartup)
|
|
||||||
│ └── Statistics Widget (#customWidget)
|
|
||||||
└── 📜 Backend Scripts
|
|
||||||
├── Auto-Tagger (#run=backendStartup)
|
|
||||||
└── Daily Backup (#run=daily)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Script Execution
|
|
||||||
|
|
||||||
### Frontend Execution
|
|
||||||
|
|
||||||
**Timing:**
|
|
||||||
1. Trilium frontend loads
|
|
||||||
2. Froca cache initializes
|
|
||||||
3. Script notes with `#run=frontendStartup` are found
|
|
||||||
4. Scripts execute in dependency order
|
|
||||||
|
|
||||||
**Isolation:**
|
|
||||||
- Each script runs in separate context
|
|
||||||
- Shared `window.api` object
|
|
||||||
- Can access global window object
|
|
||||||
|
|
||||||
### Backend Execution
|
|
||||||
|
|
||||||
**Timing:**
|
|
||||||
1. Server starts
|
|
||||||
2. Becca cache loads
|
|
||||||
3. Script notes with `#run=backendStartup` are found
|
|
||||||
4. Scripts execute in dependency order
|
|
||||||
|
|
||||||
**Isolation:**
|
|
||||||
- Each script is a separate module
|
|
||||||
- Can require Node.js modules
|
|
||||||
- Shared `api` global
|
|
||||||
|
|
||||||
### Error Handling
|
|
||||||
|
|
||||||
**Frontend:**
|
|
||||||
```javascript
|
|
||||||
try {
|
|
||||||
// Script code
|
|
||||||
} catch (error) {
|
|
||||||
api.showError('Script error: ' + error.message)
|
|
||||||
console.error(error)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Backend:**
|
|
||||||
```javascript
|
|
||||||
try {
|
|
||||||
// Script code
|
|
||||||
} catch (error) {
|
|
||||||
api.log('Script error: ' + error.message)
|
|
||||||
console.error(error)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Security Considerations
|
|
||||||
|
|
||||||
### Frontend Scripts
|
|
||||||
|
|
||||||
**Risks:**
|
|
||||||
- Can access all notes via Froca
|
|
||||||
- Can manipulate DOM
|
|
||||||
- Can make API calls
|
|
||||||
- Limited by browser security model
|
|
||||||
|
|
||||||
**Mitigations:**
|
|
||||||
- User must trust scripts they add
|
|
||||||
- Scripts run with user privileges
|
|
||||||
- No access to file system
|
|
||||||
|
|
||||||
### Backend Scripts
|
|
||||||
|
|
||||||
**Risks:**
|
|
||||||
- Full Node.js access
|
|
||||||
- Can execute system commands
|
|
||||||
- Can access file system
|
|
||||||
- Can make network requests
|
|
||||||
|
|
||||||
**Mitigations:**
|
|
||||||
- Scripts are user-created (trusted)
|
|
||||||
- Single-user model (no privilege escalation)
|
|
||||||
- Review scripts before adding `#run` attribute
|
|
||||||
|
|
||||||
### Best Practices
|
|
||||||
|
|
||||||
1. **Review script code** before adding execution attributes
|
|
||||||
2. **Use specific attributes** rather than wildcard searches
|
|
||||||
3. **Avoid eval()** and dynamic code execution
|
|
||||||
4. **Validate inputs** in scripts
|
|
||||||
5. **Handle errors** gracefully
|
|
||||||
6. **Log important actions** for audit trail
|
|
||||||
|
|
||||||
## Performance Considerations
|
|
||||||
|
|
||||||
### Optimization Tips
|
|
||||||
|
|
||||||
**1. Cache Results:**
|
|
||||||
```javascript
|
|
||||||
// Bad: Re-query on every call
|
|
||||||
function getConfig() {
|
|
||||||
return api.getNote('config').getContent()
|
|
||||||
}
|
|
||||||
|
|
||||||
// Good: Cache the result
|
|
||||||
let cachedConfig
|
|
||||||
function getConfig() {
|
|
||||||
if (!cachedConfig) {
|
|
||||||
cachedConfig = api.getNote('config').getContent()
|
|
||||||
}
|
|
||||||
return cachedConfig
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**2. Use Efficient Queries:**
|
|
||||||
```javascript
|
|
||||||
// Bad: Load all notes and filter
|
|
||||||
const todos = api.searchForNotes('#type=todo')
|
|
||||||
|
|
||||||
// Good: Use specific search
|
|
||||||
const todos = api.searchForNotes('#type=todo #status=pending')
|
|
||||||
```
|
|
||||||
|
|
||||||
**3. Batch Operations:**
|
|
||||||
```javascript
|
|
||||||
// Bad: Save after each change
|
|
||||||
notes.forEach(note => {
|
|
||||||
note.title = 'Updated'
|
|
||||||
note.save()
|
|
||||||
})
|
|
||||||
|
|
||||||
// Good: Batch changes
|
|
||||||
notes.forEach(note => {
|
|
||||||
note.title = 'Updated'
|
|
||||||
})
|
|
||||||
// Save happens in batch
|
|
||||||
```
|
|
||||||
|
|
||||||
**4. Debounce Event Handlers:**
|
|
||||||
```javascript
|
|
||||||
let timeout
|
|
||||||
api.runOnNoteContentChange(() => {
|
|
||||||
clearTimeout(timeout)
|
|
||||||
timeout = setTimeout(() => {
|
|
||||||
// Process change
|
|
||||||
}, 500)
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
## Debugging Scripts
|
|
||||||
|
|
||||||
### Frontend Debugging
|
|
||||||
|
|
||||||
**Browser DevTools:**
|
|
||||||
```javascript
|
|
||||||
console.log('Debug info:', data)
|
|
||||||
debugger // Breakpoint
|
|
||||||
```
|
|
||||||
|
|
||||||
**Trilium Log:**
|
|
||||||
```javascript
|
|
||||||
api.log('Script executed')
|
|
||||||
```
|
|
||||||
|
|
||||||
### Backend Debugging
|
|
||||||
|
|
||||||
**Console Output:**
|
|
||||||
```javascript
|
|
||||||
console.log('Backend debug:', data)
|
|
||||||
api.log('Script log message')
|
|
||||||
```
|
|
||||||
|
|
||||||
**Inspect Becca:**
|
|
||||||
```javascript
|
|
||||||
api.log('Note count:', Object.keys(api.becca.notes).length)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Advanced Topics
|
|
||||||
|
|
||||||
### Custom Note Types
|
|
||||||
|
|
||||||
Scripts can implement custom note type handlers:
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
// Register custom type
|
|
||||||
api.registerNoteType({
|
|
||||||
type: 'mytype',
|
|
||||||
mime: 'application/x-mytype',
|
|
||||||
renderNote: (note) => {
|
|
||||||
// Custom rendering
|
|
||||||
}
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
### External Libraries
|
|
||||||
|
|
||||||
**Frontend:**
|
|
||||||
```javascript
|
|
||||||
// Load external library
|
|
||||||
const myLib = await import('https://cdn.example.com/lib.js')
|
|
||||||
```
|
|
||||||
|
|
||||||
**Backend:**
|
|
||||||
```javascript
|
|
||||||
// Use Node.js require
|
|
||||||
const fs = require('fs')
|
|
||||||
const axios = require('axios')
|
|
||||||
```
|
|
||||||
|
|
||||||
### State Persistence
|
|
||||||
|
|
||||||
**Frontend:**
|
|
||||||
```javascript
|
|
||||||
// Use localStorage
|
|
||||||
localStorage.setItem('myScript:data', JSON.stringify(data))
|
|
||||||
const data = JSON.parse(localStorage.getItem('myScript:data'))
|
|
||||||
```
|
|
||||||
|
|
||||||
**Backend:**
|
|
||||||
```javascript
|
|
||||||
// Store in special note
|
|
||||||
const stateNote = api.getNote('script-state-note')
|
|
||||||
stateNote.setContent(JSON.stringify(data))
|
|
||||||
|
|
||||||
const data = JSON.parse(stateNote.getContent())
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**See Also:**
|
|
||||||
- [Script API Documentation](Script%20API/) - Complete API reference
|
|
||||||
- [Advanced Showcases](https://triliumnext.github.io/Docs/Wiki/advanced-showcases) - Example scripts
|
|
||||||
- [ARCHITECTURE.md](ARCHITECTURE.md) - Overall architecture
|
|
||||||
834
docs/SECURITY_ARCHITECTURE.md
vendored
834
docs/SECURITY_ARCHITECTURE.md
vendored
@@ -1,834 +0,0 @@
|
|||||||
# Trilium Security Architecture
|
|
||||||
|
|
||||||
> **Related:** [ARCHITECTURE.md](ARCHITECTURE.md) | [SECURITY.md](../SECURITY.md)
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
Trilium implements a **defense-in-depth security model** with multiple layers of protection for user data. The security architecture covers authentication, authorization, encryption, input sanitization, and secure communication.
|
|
||||||
|
|
||||||
## Security Principles
|
|
||||||
|
|
||||||
1. **Data Privacy**: User data is protected at rest and in transit
|
|
||||||
2. **Encryption**: Per-note encryption for sensitive content
|
|
||||||
3. **Authentication**: Multiple authentication methods supported
|
|
||||||
4. **Authorization**: Single-user model with granular protected sessions
|
|
||||||
5. **Input Validation**: All user input sanitized
|
|
||||||
6. **Secure Defaults**: Security features enabled by default
|
|
||||||
7. **Transparency**: Open source allows security audits
|
|
||||||
|
|
||||||
## Threat Model
|
|
||||||
|
|
||||||
### Threats Considered
|
|
||||||
|
|
||||||
1. **Unauthorized Access**
|
|
||||||
- Physical access to device
|
|
||||||
- Network eavesdropping
|
|
||||||
- Stolen credentials
|
|
||||||
- Session hijacking
|
|
||||||
|
|
||||||
2. **Data Exfiltration**
|
|
||||||
- Malicious scripts
|
|
||||||
- XSS attacks
|
|
||||||
- SQL injection
|
|
||||||
- CSRF attacks
|
|
||||||
|
|
||||||
3. **Data Corruption**
|
|
||||||
- Malicious modifications
|
|
||||||
- Database tampering
|
|
||||||
- Sync conflicts
|
|
||||||
|
|
||||||
4. **Privacy Leaks**
|
|
||||||
- Unencrypted backups
|
|
||||||
- Search indexing
|
|
||||||
- Temporary files
|
|
||||||
- Memory dumps
|
|
||||||
|
|
||||||
### Out of Scope
|
|
||||||
|
|
||||||
- Nation-state attackers
|
|
||||||
- Zero-day vulnerabilities in dependencies
|
|
||||||
- Hardware vulnerabilities (Spectre, Meltdown)
|
|
||||||
- Physical access with unlimited time
|
|
||||||
- Quantum computing attacks
|
|
||||||
|
|
||||||
## Authentication
|
|
||||||
|
|
||||||
### Password Authentication
|
|
||||||
|
|
||||||
**Implementation:** `apps/server/src/services/password.ts`
|
|
||||||
|
|
||||||
**Password Storage:**
|
|
||||||
```typescript
|
|
||||||
// Password is never stored directly
|
|
||||||
const salt = crypto.randomBytes(32)
|
|
||||||
const derivedKey = crypto.pbkdf2Sync(password, salt, 10000, 32, 'sha256')
|
|
||||||
const verificationHash = crypto.createHash('sha256')
|
|
||||||
.update(derivedKey)
|
|
||||||
.digest('hex')
|
|
||||||
|
|
||||||
// Store only salt and verification hash
|
|
||||||
sql.insert('user_data', {
|
|
||||||
salt: salt.toString('hex'),
|
|
||||||
derivedKey: derivedKey.toString('hex') // Used for encryption
|
|
||||||
})
|
|
||||||
|
|
||||||
sql.insert('options', {
|
|
||||||
name: 'passwordVerificationHash',
|
|
||||||
value: verificationHash
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
**Password Requirements:**
|
|
||||||
- Minimum length: 4 characters (configurable)
|
|
||||||
- No maximum length
|
|
||||||
- All characters allowed
|
|
||||||
- Can be changed by user
|
|
||||||
|
|
||||||
**Login Process:**
|
|
||||||
```typescript
|
|
||||||
// 1. User submits password
|
|
||||||
POST /api/login/password
|
|
||||||
Body: { password: "user-password" }
|
|
||||||
|
|
||||||
// 2. Server derives key
|
|
||||||
const derivedKey = crypto.pbkdf2Sync(password, salt, 10000, 32, 'sha256')
|
|
||||||
|
|
||||||
// 3. Verify against stored hash
|
|
||||||
const verificationHash = crypto.createHash('sha256')
|
|
||||||
.update(derivedKey)
|
|
||||||
.digest('hex')
|
|
||||||
|
|
||||||
if (verificationHash === storedHash) {
|
|
||||||
// 4. Create session
|
|
||||||
req.session.loggedIn = true
|
|
||||||
req.session.regenerate()
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### TOTP (Two-Factor Authentication)
|
|
||||||
|
|
||||||
**Implementation:** `apps/server/src/routes/api/login.ts`
|
|
||||||
|
|
||||||
**Setup Process:**
|
|
||||||
```typescript
|
|
||||||
// 1. Generate secret
|
|
||||||
const secret = speakeasy.generateSecret({
|
|
||||||
name: `Trilium (${username})`,
|
|
||||||
length: 32
|
|
||||||
})
|
|
||||||
|
|
||||||
// 2. Store encrypted secret
|
|
||||||
const encryptedSecret = encrypt(secret.base32, dataKey)
|
|
||||||
sql.insert('options', {
|
|
||||||
name: 'totpSecret',
|
|
||||||
value: encryptedSecret
|
|
||||||
})
|
|
||||||
|
|
||||||
// 3. Generate QR code
|
|
||||||
const qrCodeUrl = secret.otpauth_url
|
|
||||||
```
|
|
||||||
|
|
||||||
**Verification:**
|
|
||||||
```typescript
|
|
||||||
// User submits TOTP token
|
|
||||||
POST /api/login/totp
|
|
||||||
Body: { token: "123456" }
|
|
||||||
|
|
||||||
// Verify token
|
|
||||||
const secret = decrypt(encryptedSecret, dataKey)
|
|
||||||
const verified = speakeasy.totp.verify({
|
|
||||||
secret: secret,
|
|
||||||
encoding: 'base32',
|
|
||||||
token: token,
|
|
||||||
window: 1 // Allow 1 time step tolerance
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
### OpenID Connect
|
|
||||||
|
|
||||||
**Implementation:** `apps/server/src/routes/api/login.ts`
|
|
||||||
|
|
||||||
**Supported Providers:**
|
|
||||||
- Any OpenID Connect compatible provider
|
|
||||||
- Google, GitHub, Auth0, etc.
|
|
||||||
|
|
||||||
**Flow:**
|
|
||||||
```typescript
|
|
||||||
// 1. Redirect to provider
|
|
||||||
GET /api/login/openid
|
|
||||||
|
|
||||||
// 2. Provider redirects back with code
|
|
||||||
GET /api/login/openid/callback?code=...
|
|
||||||
|
|
||||||
// 3. Exchange code for tokens
|
|
||||||
const tokens = await openidClient.callback(redirectUri, req.query)
|
|
||||||
|
|
||||||
// 4. Verify ID token
|
|
||||||
const claims = tokens.claims()
|
|
||||||
|
|
||||||
// 5. Create session
|
|
||||||
req.session.loggedIn = true
|
|
||||||
```
|
|
||||||
|
|
||||||
### Session Management
|
|
||||||
|
|
||||||
**Session Storage:** SQLite database (sessions table)
|
|
||||||
|
|
||||||
**Session Configuration:**
|
|
||||||
```typescript
|
|
||||||
app.use(session({
|
|
||||||
secret: sessionSecret,
|
|
||||||
resave: false,
|
|
||||||
saveUninitialized: false,
|
|
||||||
rolling: true,
|
|
||||||
cookie: {
|
|
||||||
maxAge: 7 * 24 * 60 * 60 * 1000, // 7 days
|
|
||||||
httpOnly: true,
|
|
||||||
secure: isHttps,
|
|
||||||
sameSite: 'lax'
|
|
||||||
},
|
|
||||||
store: new SqliteStore({
|
|
||||||
db: db,
|
|
||||||
table: 'sessions'
|
|
||||||
})
|
|
||||||
}))
|
|
||||||
```
|
|
||||||
|
|
||||||
**Session Invalidation:**
|
|
||||||
- Automatic timeout after inactivity
|
|
||||||
- Manual logout clears session
|
|
||||||
- Server restart invalidates all sessions (optional)
|
|
||||||
|
|
||||||
## Authorization
|
|
||||||
|
|
||||||
### Single-User Model
|
|
||||||
|
|
||||||
**Desktop:**
|
|
||||||
- Single user (owner of device)
|
|
||||||
- No multi-user support
|
|
||||||
- Full access to all notes
|
|
||||||
|
|
||||||
**Server:**
|
|
||||||
- Single user per installation
|
|
||||||
- Authentication required for all operations
|
|
||||||
- No user roles or permissions
|
|
||||||
|
|
||||||
### Protected Sessions
|
|
||||||
|
|
||||||
**Purpose:** Temporary access to encrypted (protected) notes
|
|
||||||
|
|
||||||
**Implementation:** `apps/server/src/services/protected_session.ts`
|
|
||||||
|
|
||||||
**Workflow:**
|
|
||||||
```typescript
|
|
||||||
// 1. User enters password for protected notes
|
|
||||||
POST /api/protected-session/enter
|
|
||||||
Body: { password: "protected-password" }
|
|
||||||
|
|
||||||
// 2. Derive encryption key
|
|
||||||
const protectedDataKey = deriveKey(password)
|
|
||||||
|
|
||||||
// 3. Verify password (decrypt known encrypted value)
|
|
||||||
const decrypted = decrypt(testValue, protectedDataKey)
|
|
||||||
if (decrypted === expectedValue) {
|
|
||||||
// 4. Store in memory (not in session)
|
|
||||||
protectedSessionHolder.setProtectedDataKey(protectedDataKey)
|
|
||||||
|
|
||||||
// 5. Set timeout
|
|
||||||
setTimeout(() => {
|
|
||||||
protectedSessionHolder.clearProtectedDataKey()
|
|
||||||
}, timeout)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Protected Session Timeout:**
|
|
||||||
- Default: 10 minutes (configurable)
|
|
||||||
- Extends on activity
|
|
||||||
- Cleared on browser close
|
|
||||||
- Separate from main session
|
|
||||||
|
|
||||||
### API Authorization
|
|
||||||
|
|
||||||
**Internal API:**
|
|
||||||
- Requires authenticated session
|
|
||||||
- CSRF token validation
|
|
||||||
- Same-origin policy
|
|
||||||
|
|
||||||
**ETAPI (External API):**
|
|
||||||
- Token-based authentication
|
|
||||||
- No session required
|
|
||||||
- Rate limiting
|
|
||||||
|
|
||||||
## Encryption
|
|
||||||
|
|
||||||
### Note Encryption
|
|
||||||
|
|
||||||
**Encryption Algorithm:** AES-256-CBC
|
|
||||||
|
|
||||||
**Key Hierarchy:**
|
|
||||||
```
|
|
||||||
User Password
|
|
||||||
↓ (PBKDF2)
|
|
||||||
Data Key (for protected notes)
|
|
||||||
↓ (AES-256)
|
|
||||||
Protected Note Content
|
|
||||||
```
|
|
||||||
|
|
||||||
**Encryption Process:**
|
|
||||||
```typescript
|
|
||||||
// 1. Generate IV (initialization vector)
|
|
||||||
const iv = crypto.randomBytes(16)
|
|
||||||
|
|
||||||
// 2. Encrypt content
|
|
||||||
const cipher = crypto.createCipheriv('aes-256-cbc', dataKey, iv)
|
|
||||||
let encrypted = cipher.update(content, 'utf8', 'base64')
|
|
||||||
encrypted += cipher.final('base64')
|
|
||||||
|
|
||||||
// 3. Prepend IV to encrypted content
|
|
||||||
const encryptedBlob = iv.toString('base64') + ':' + encrypted
|
|
||||||
|
|
||||||
// 4. Store in database
|
|
||||||
sql.insert('blobs', {
|
|
||||||
blobId: blobId,
|
|
||||||
content: encryptedBlob
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
**Decryption Process:**
|
|
||||||
```typescript
|
|
||||||
// 1. Split IV and encrypted content
|
|
||||||
const [ivBase64, encryptedData] = encryptedBlob.split(':')
|
|
||||||
const iv = Buffer.from(ivBase64, 'base64')
|
|
||||||
|
|
||||||
// 2. Decrypt
|
|
||||||
const decipher = crypto.createDecipheriv('aes-256-cbc', dataKey, iv)
|
|
||||||
let decrypted = decipher.update(encryptedData, 'base64', 'utf8')
|
|
||||||
decrypted += decipher.final('utf8')
|
|
||||||
|
|
||||||
return decrypted
|
|
||||||
```
|
|
||||||
|
|
||||||
**Protected Note Metadata:**
|
|
||||||
- Title is NOT encrypted (for tree display)
|
|
||||||
- Type and MIME are NOT encrypted
|
|
||||||
- Content IS encrypted
|
|
||||||
- Attributes CAN be encrypted (optional)
|
|
||||||
|
|
||||||
### Data Key Management
|
|
||||||
|
|
||||||
**Master Data Key:**
|
|
||||||
```typescript
|
|
||||||
// Generated once during setup
|
|
||||||
const dataKey = crypto.randomBytes(32) // 256 bits
|
|
||||||
|
|
||||||
// Encrypted with derived key from user password
|
|
||||||
const derivedKey = crypto.pbkdf2Sync(password, salt, 10000, 32, 'sha256')
|
|
||||||
const encryptedDataKey = encrypt(dataKey, derivedKey)
|
|
||||||
|
|
||||||
// Stored in database
|
|
||||||
sql.insert('options', {
|
|
||||||
name: 'encryptedDataKey',
|
|
||||||
value: encryptedDataKey.toString('hex')
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
**Key Rotation:**
|
|
||||||
- Not currently supported
|
|
||||||
- Requires re-encrypting all protected notes
|
|
||||||
- Planned for future version
|
|
||||||
|
|
||||||
### Transport Encryption
|
|
||||||
|
|
||||||
**HTTPS:**
|
|
||||||
- Required for server installations (recommended)
|
|
||||||
- TLS 1.2+ only
|
|
||||||
- Strong cipher suites preferred
|
|
||||||
- Certificate validation enabled
|
|
||||||
|
|
||||||
**Desktop:**
|
|
||||||
- Local communication (no network)
|
|
||||||
- No HTTPS required
|
|
||||||
|
|
||||||
### Backup Encryption
|
|
||||||
|
|
||||||
**Database Backups:**
|
|
||||||
- Protected notes remain encrypted in backup
|
|
||||||
- Backup file should be protected separately
|
|
||||||
- Consider encrypting backup storage location
|
|
||||||
|
|
||||||
## Input Sanitization
|
|
||||||
|
|
||||||
### XSS Prevention
|
|
||||||
|
|
||||||
**HTML Sanitization:**
|
|
||||||
|
|
||||||
Location: `apps/client/src/services/dompurify.ts`
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
import DOMPurify from 'dompurify'
|
|
||||||
|
|
||||||
// Configure DOMPurify
|
|
||||||
DOMPurify.setConfig({
|
|
||||||
ALLOWED_TAGS: ['b', 'i', 'em', 'strong', 'a', 'p', 'br', 'div', ...],
|
|
||||||
ALLOWED_ATTR: ['href', 'title', 'class', 'id', ...],
|
|
||||||
ALLOW_DATA_ATTR: false
|
|
||||||
})
|
|
||||||
|
|
||||||
// Sanitize HTML before rendering
|
|
||||||
const cleanHtml = DOMPurify.sanitize(userHtml)
|
|
||||||
```
|
|
||||||
|
|
||||||
**CKEditor Configuration:**
|
|
||||||
```typescript
|
|
||||||
// apps/client/src/widgets/type_widgets/text_type_widget.ts
|
|
||||||
ClassicEditor.create(element, {
|
|
||||||
// Restrict allowed content
|
|
||||||
htmlSupport: {
|
|
||||||
allow: [
|
|
||||||
{ name: /./, attributes: true, classes: true, styles: true }
|
|
||||||
],
|
|
||||||
disallow: [
|
|
||||||
{ name: 'script' },
|
|
||||||
{ name: 'iframe', attributes: /^(?!src$).*/ }
|
|
||||||
]
|
|
||||||
}
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
**Content Security Policy:**
|
|
||||||
```typescript
|
|
||||||
// apps/server/src/main.ts
|
|
||||||
app.use((req, res, next) => {
|
|
||||||
res.setHeader('Content-Security-Policy',
|
|
||||||
"default-src 'self'; " +
|
|
||||||
"script-src 'self' 'unsafe-inline' 'unsafe-eval'; " +
|
|
||||||
"style-src 'self' 'unsafe-inline'; " +
|
|
||||||
"img-src 'self' data: blob:;"
|
|
||||||
)
|
|
||||||
next()
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
### SQL Injection Prevention
|
|
||||||
|
|
||||||
**Parameterized Queries:**
|
|
||||||
```typescript
|
|
||||||
// GOOD - Safe from SQL injection
|
|
||||||
const notes = sql.getRows(
|
|
||||||
'SELECT * FROM notes WHERE title = ?',
|
|
||||||
[userInput]
|
|
||||||
)
|
|
||||||
|
|
||||||
// BAD - Vulnerable to SQL injection
|
|
||||||
const notes = sql.getRows(
|
|
||||||
`SELECT * FROM notes WHERE title = '${userInput}'`
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
**ORM Usage:**
|
|
||||||
```typescript
|
|
||||||
// Entity-based access prevents SQL injection
|
|
||||||
const note = becca.getNote(noteId)
|
|
||||||
note.title = userInput // Sanitized by entity
|
|
||||||
note.save() // Parameterized query
|
|
||||||
```
|
|
||||||
|
|
||||||
### CSRF Prevention
|
|
||||||
|
|
||||||
**CSRF Token Validation:**
|
|
||||||
|
|
||||||
Location: `apps/server/src/routes/middleware/csrf.ts`
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
// Generate CSRF token
|
|
||||||
const csrfToken = crypto.randomBytes(32).toString('hex')
|
|
||||||
req.session.csrfToken = csrfToken
|
|
||||||
|
|
||||||
// Validate on state-changing requests
|
|
||||||
app.use((req, res, next) => {
|
|
||||||
if (['POST', 'PUT', 'DELETE'].includes(req.method)) {
|
|
||||||
const token = req.headers['x-csrf-token']
|
|
||||||
if (token !== req.session.csrfToken) {
|
|
||||||
return res.status(403).json({ error: 'CSRF token mismatch' })
|
|
||||||
}
|
|
||||||
}
|
|
||||||
next()
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
**Client-Side:**
|
|
||||||
```typescript
|
|
||||||
// apps/client/src/services/server.ts
|
|
||||||
const csrfToken = getCsrfToken()
|
|
||||||
|
|
||||||
fetch('/api/notes', {
|
|
||||||
method: 'POST',
|
|
||||||
headers: {
|
|
||||||
'X-CSRF-Token': csrfToken,
|
|
||||||
'Content-Type': 'application/json'
|
|
||||||
},
|
|
||||||
body: JSON.stringify(data)
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
### File Upload Validation
|
|
||||||
|
|
||||||
**Validation:**
|
|
||||||
```typescript
|
|
||||||
// apps/server/src/routes/api/attachments.ts
|
|
||||||
const allowedMimeTypes = [
|
|
||||||
'image/jpeg',
|
|
||||||
'image/png',
|
|
||||||
'application/pdf',
|
|
||||||
// ...
|
|
||||||
]
|
|
||||||
|
|
||||||
if (!allowedMimeTypes.includes(file.mimetype)) {
|
|
||||||
throw new Error('File type not allowed')
|
|
||||||
}
|
|
||||||
|
|
||||||
// Validate file size
|
|
||||||
const maxSize = 100 * 1024 * 1024 // 100 MB
|
|
||||||
if (file.size > maxSize) {
|
|
||||||
throw new Error('File too large')
|
|
||||||
}
|
|
||||||
|
|
||||||
// Sanitize filename
|
|
||||||
const sanitizedFilename = path.basename(file.originalname)
|
|
||||||
.replace(/[^a-z0-9.-]/gi, '_')
|
|
||||||
```
|
|
||||||
|
|
||||||
## Network Security
|
|
||||||
|
|
||||||
### HTTPS Configuration
|
|
||||||
|
|
||||||
**Server Setup:**
|
|
||||||
```typescript
|
|
||||||
// apps/server/src/main.ts
|
|
||||||
const httpsOptions = {
|
|
||||||
key: fs.readFileSync('server.key'),
|
|
||||||
cert: fs.readFileSync('server.cert')
|
|
||||||
}
|
|
||||||
|
|
||||||
https.createServer(httpsOptions, app).listen(443)
|
|
||||||
```
|
|
||||||
|
|
||||||
**Certificate Validation:**
|
|
||||||
- Require valid certificates in production
|
|
||||||
- Self-signed certificates allowed for development
|
|
||||||
- Certificate pinning not implemented
|
|
||||||
|
|
||||||
### Secure Headers
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
// apps/server/src/main.ts
|
|
||||||
app.use((req, res, next) => {
|
|
||||||
// Prevent clickjacking
|
|
||||||
res.setHeader('X-Frame-Options', 'SAMEORIGIN')
|
|
||||||
|
|
||||||
// Prevent MIME sniffing
|
|
||||||
res.setHeader('X-Content-Type-Options', 'nosniff')
|
|
||||||
|
|
||||||
// XSS protection
|
|
||||||
res.setHeader('X-XSS-Protection', '1; mode=block')
|
|
||||||
|
|
||||||
// Referrer policy
|
|
||||||
res.setHeader('Referrer-Policy', 'same-origin')
|
|
||||||
|
|
||||||
// HTTPS upgrade
|
|
||||||
if (req.secure) {
|
|
||||||
res.setHeader('Strict-Transport-Security', 'max-age=31536000')
|
|
||||||
}
|
|
||||||
|
|
||||||
next()
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
### Rate Limiting
|
|
||||||
|
|
||||||
**API Rate Limiting:**
|
|
||||||
```typescript
|
|
||||||
// apps/server/src/routes/middleware/rate_limit.ts
|
|
||||||
const rateLimit = require('express-rate-limit')
|
|
||||||
|
|
||||||
const apiLimiter = rateLimit({
|
|
||||||
windowMs: 15 * 60 * 1000, // 15 minutes
|
|
||||||
max: 1000, // Limit each IP to 1000 requests per window
|
|
||||||
message: 'Too many requests from this IP'
|
|
||||||
})
|
|
||||||
|
|
||||||
app.use('/api/', apiLimiter)
|
|
||||||
```
|
|
||||||
|
|
||||||
**Login Rate Limiting:**
|
|
||||||
```typescript
|
|
||||||
const loginLimiter = rateLimit({
|
|
||||||
windowMs: 15 * 60 * 1000,
|
|
||||||
max: 5, // 5 failed attempts
|
|
||||||
skipSuccessfulRequests: true
|
|
||||||
})
|
|
||||||
|
|
||||||
app.post('/api/login/password', loginLimiter, loginHandler)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Data Security
|
|
||||||
|
|
||||||
### Secure Data Deletion
|
|
||||||
|
|
||||||
**Soft Delete:**
|
|
||||||
```typescript
|
|
||||||
// Mark as deleted (sync first)
|
|
||||||
note.isDeleted = 1
|
|
||||||
note.deleteId = generateUUID()
|
|
||||||
note.save()
|
|
||||||
|
|
||||||
// Entity change tracked for sync
|
|
||||||
addEntityChange('notes', noteId, note)
|
|
||||||
```
|
|
||||||
|
|
||||||
**Hard Delete (Erase):**
|
|
||||||
```typescript
|
|
||||||
// After sync completed
|
|
||||||
sql.execute('DELETE FROM notes WHERE noteId = ?', [noteId])
|
|
||||||
sql.execute('DELETE FROM branches WHERE noteId = ?', [noteId])
|
|
||||||
sql.execute('DELETE FROM attributes WHERE noteId = ?', [noteId])
|
|
||||||
|
|
||||||
// Mark entity change as erased
|
|
||||||
sql.execute('UPDATE entity_changes SET isErased = 1 WHERE entityId = ?', [noteId])
|
|
||||||
```
|
|
||||||
|
|
||||||
**Blob Cleanup:**
|
|
||||||
```typescript
|
|
||||||
// Find orphaned blobs (not referenced by any note/revision/attachment)
|
|
||||||
const orphanedBlobs = sql.getRows(`
|
|
||||||
SELECT blobId FROM blobs
|
|
||||||
WHERE blobId NOT IN (SELECT blobId FROM notes WHERE blobId IS NOT NULL)
|
|
||||||
AND blobId NOT IN (SELECT blobId FROM revisions WHERE blobId IS NOT NULL)
|
|
||||||
AND blobId NOT IN (SELECT blobId FROM attachments WHERE blobId IS NOT NULL)
|
|
||||||
`)
|
|
||||||
|
|
||||||
// Delete orphaned blobs
|
|
||||||
for (const blob of orphanedBlobs) {
|
|
||||||
sql.execute('DELETE FROM blobs WHERE blobId = ?', [blob.blobId])
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Memory Security
|
|
||||||
|
|
||||||
**Protected Data in Memory:**
|
|
||||||
- Protected data keys stored in memory only
|
|
||||||
- Cleared on timeout
|
|
||||||
- Not written to disk
|
|
||||||
- Not in session storage
|
|
||||||
|
|
||||||
**Memory Cleanup:**
|
|
||||||
```typescript
|
|
||||||
// Clear sensitive data
|
|
||||||
const clearSensitiveData = () => {
|
|
||||||
protectedDataKey = null
|
|
||||||
|
|
||||||
// Force garbage collection if available
|
|
||||||
if (global.gc) {
|
|
||||||
global.gc()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Temporary Files
|
|
||||||
|
|
||||||
**Secure Temporary Files:**
|
|
||||||
```typescript
|
|
||||||
const tempDir = os.tmpdir()
|
|
||||||
const tempFile = path.join(tempDir, `trilium-${crypto.randomBytes(16).toString('hex')}`)
|
|
||||||
|
|
||||||
// Write temp file
|
|
||||||
fs.writeFileSync(tempFile, data, { mode: 0o600 }) // Owner read/write only
|
|
||||||
|
|
||||||
// Clean up after use
|
|
||||||
fs.unlinkSync(tempFile)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Dependency Security
|
|
||||||
|
|
||||||
### Vulnerability Scanning
|
|
||||||
|
|
||||||
**Tools:**
|
|
||||||
- `npm audit` - Check for known vulnerabilities
|
|
||||||
- Renovate bot - Automatic dependency updates
|
|
||||||
- GitHub Dependabot alerts
|
|
||||||
|
|
||||||
**Process:**
|
|
||||||
```bash
|
|
||||||
# Check for vulnerabilities
|
|
||||||
npm audit
|
|
||||||
|
|
||||||
# Fix automatically
|
|
||||||
npm audit fix
|
|
||||||
|
|
||||||
# Manual review for breaking changes
|
|
||||||
npm audit fix --force
|
|
||||||
```
|
|
||||||
|
|
||||||
### Dependency Pinning
|
|
||||||
|
|
||||||
**package.json:**
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"dependencies": {
|
|
||||||
"express": "4.18.2", // Exact version
|
|
||||||
"better-sqlite3": "^9.2.2" // Compatible versions
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**pnpm Overrides:**
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"pnpm": {
|
|
||||||
"overrides": {
|
|
||||||
"lodash@<4.17.21": ">=4.17.21", // Force minimum version
|
|
||||||
"axios@<0.21.2": ">=0.21.2"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Patch Management
|
|
||||||
|
|
||||||
**pnpm Patches:**
|
|
||||||
```bash
|
|
||||||
# Create patch
|
|
||||||
pnpm patch @ckeditor/ckeditor5
|
|
||||||
|
|
||||||
# Edit files in temporary directory
|
|
||||||
# ...
|
|
||||||
|
|
||||||
# Generate patch file
|
|
||||||
pnpm patch-commit /tmp/ckeditor5-patch
|
|
||||||
|
|
||||||
# Patch applied automatically on install
|
|
||||||
```
|
|
||||||
|
|
||||||
## Security Best Practices
|
|
||||||
|
|
||||||
### For Users
|
|
||||||
|
|
||||||
1. **Strong Passwords**
|
|
||||||
- Use unique password for Trilium
|
|
||||||
- Enable TOTP 2FA
|
|
||||||
- Protect password manager
|
|
||||||
|
|
||||||
2. **Protected Notes**
|
|
||||||
- Use for sensitive information
|
|
||||||
- Set reasonable session timeout
|
|
||||||
- Don't leave sessions unattended
|
|
||||||
|
|
||||||
3. **Backups**
|
|
||||||
- Regular backups to secure location
|
|
||||||
- Encrypt backup storage
|
|
||||||
- Test backup restoration
|
|
||||||
|
|
||||||
4. **Server Setup**
|
|
||||||
- Use HTTPS only
|
|
||||||
- Keep software updated
|
|
||||||
- Firewall configuration
|
|
||||||
- Use reverse proxy (nginx, Caddy)
|
|
||||||
|
|
||||||
5. **Scripts**
|
|
||||||
- Review scripts before using
|
|
||||||
- Be cautious with external scripts
|
|
||||||
- Understand script permissions
|
|
||||||
|
|
||||||
### For Developers
|
|
||||||
|
|
||||||
1. **Code Review**
|
|
||||||
- Review all security-related changes
|
|
||||||
- Test authentication/authorization changes
|
|
||||||
- Validate input sanitization
|
|
||||||
|
|
||||||
2. **Testing**
|
|
||||||
- Write security tests
|
|
||||||
- Test edge cases
|
|
||||||
- Penetration testing
|
|
||||||
|
|
||||||
3. **Dependencies**
|
|
||||||
- Regular updates
|
|
||||||
- Audit new dependencies
|
|
||||||
- Monitor security advisories
|
|
||||||
|
|
||||||
4. **Secrets**
|
|
||||||
- No secrets in source code
|
|
||||||
- Use environment variables
|
|
||||||
- Secure key generation
|
|
||||||
|
|
||||||
## Security Auditing
|
|
||||||
|
|
||||||
### Logs
|
|
||||||
|
|
||||||
**Security Events Logged:**
|
|
||||||
- Login attempts (success/failure)
|
|
||||||
- Protected session access
|
|
||||||
- Password changes
|
|
||||||
- ETAPI token usage
|
|
||||||
- Failed CSRF validations
|
|
||||||
|
|
||||||
**Log Location:**
|
|
||||||
- Desktop: Console output
|
|
||||||
- Server: Log files or stdout
|
|
||||||
|
|
||||||
### Monitoring
|
|
||||||
|
|
||||||
**Metrics to Monitor:**
|
|
||||||
- Failed login attempts
|
|
||||||
- API error rates
|
|
||||||
- Unusual database changes
|
|
||||||
- Large exports/imports
|
|
||||||
|
|
||||||
## Incident Response
|
|
||||||
|
|
||||||
### Security Issue Reporting
|
|
||||||
|
|
||||||
**Process:**
|
|
||||||
1. Email security@triliumnext.com
|
|
||||||
2. Include vulnerability details
|
|
||||||
3. Provide reproduction steps
|
|
||||||
4. Allow reasonable disclosure time
|
|
||||||
|
|
||||||
**Response:**
|
|
||||||
1. Acknowledge within 48 hours
|
|
||||||
2. Investigate and validate
|
|
||||||
3. Develop fix
|
|
||||||
4. Coordinate disclosure
|
|
||||||
5. Release patch
|
|
||||||
|
|
||||||
### Breach Response
|
|
||||||
|
|
||||||
**If Compromised:**
|
|
||||||
1. Change password immediately
|
|
||||||
2. Review recent activity
|
|
||||||
3. Check for unauthorized changes
|
|
||||||
4. Restore from backup if needed
|
|
||||||
5. Update security settings
|
|
||||||
|
|
||||||
## Future Security Enhancements
|
|
||||||
|
|
||||||
**Planned:**
|
|
||||||
- Hardware security key support (U2F/WebAuthn)
|
|
||||||
- End-to-end encryption for sync
|
|
||||||
- Zero-knowledge architecture option
|
|
||||||
- Encryption key rotation
|
|
||||||
- Audit log enhancements
|
|
||||||
- Per-note access controls
|
|
||||||
|
|
||||||
**Under Consideration:**
|
|
||||||
- Multi-user support with permissions
|
|
||||||
- Blockchain-based sync verification
|
|
||||||
- Homomorphic encryption for search
|
|
||||||
- Quantum-resistant encryption
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**See Also:**
|
|
||||||
- [SECURITY.md](../SECURITY.md) - Security policy
|
|
||||||
- [ARCHITECTURE.md](ARCHITECTURE.md) - Overall architecture
|
|
||||||
- [Protected Notes Guide](https://triliumnext.github.io/Docs/Wiki/protected-notes)
|
|
||||||
423
docs/TECHNICAL_DOCUMENTATION.md
vendored
423
docs/TECHNICAL_DOCUMENTATION.md
vendored
@@ -1,423 +0,0 @@
|
|||||||
# Trilium Notes - Technical Documentation Index
|
|
||||||
|
|
||||||
Welcome to the comprehensive technical and architectural documentation for Trilium Notes. This index provides quick access to all technical documentation resources.
|
|
||||||
|
|
||||||
## 📚 Core Architecture Documentation
|
|
||||||
|
|
||||||
### [ARCHITECTURE.md](ARCHITECTURE.md)
|
|
||||||
**Main technical architecture document** covering the complete system design.
|
|
||||||
|
|
||||||
**Topics Covered:**
|
|
||||||
- High-level architecture overview
|
|
||||||
- Monorepo structure and organization
|
|
||||||
- Core architecture patterns (Becca, Froca, Shaca)
|
|
||||||
- Entity system and data model
|
|
||||||
- Widget-based UI architecture
|
|
||||||
- Frontend and backend architecture
|
|
||||||
- API architecture (Internal, ETAPI, WebSocket)
|
|
||||||
- Build system and tooling
|
|
||||||
- Testing strategy
|
|
||||||
- Security overview
|
|
||||||
|
|
||||||
**Audience:** Developers, architects, contributors
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### [DATABASE.md](DATABASE.md)
|
|
||||||
**Complete database architecture and schema documentation.**
|
|
||||||
|
|
||||||
**Topics Covered:**
|
|
||||||
- SQLite database structure
|
|
||||||
- Entity tables (notes, branches, attributes, revisions, attachments, blobs)
|
|
||||||
- System tables (options, entity_changes, sessions)
|
|
||||||
- Data relationships and integrity
|
|
||||||
- Database access patterns
|
|
||||||
- Migrations and versioning
|
|
||||||
- Performance optimization
|
|
||||||
- Backup and maintenance
|
|
||||||
- Security considerations
|
|
||||||
|
|
||||||
**Audience:** Backend developers, database administrators
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### [SYNCHRONIZATION.md](SYNCHRONIZATION.md)
|
|
||||||
**Detailed synchronization protocol and implementation.**
|
|
||||||
|
|
||||||
**Topics Covered:**
|
|
||||||
- Sync architecture overview
|
|
||||||
- Entity change tracking
|
|
||||||
- Sync protocol (handshake, pull, push)
|
|
||||||
- Conflict resolution strategies
|
|
||||||
- Protected notes synchronization
|
|
||||||
- Performance optimizations
|
|
||||||
- Error handling and retry logic
|
|
||||||
- Sync server configuration
|
|
||||||
- WebSocket real-time updates
|
|
||||||
- Troubleshooting guide
|
|
||||||
|
|
||||||
**Audience:** Advanced users, sync server administrators, contributors
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### [SCRIPTING.md](SCRIPTING.md)
|
|
||||||
**Comprehensive guide to the Trilium scripting system.**
|
|
||||||
|
|
||||||
**Topics Covered:**
|
|
||||||
- Script types (frontend, backend, render)
|
|
||||||
- Frontend API reference
|
|
||||||
- Backend API reference
|
|
||||||
- Entity classes (FNote, BNote, etc.)
|
|
||||||
- Script examples and patterns
|
|
||||||
- Script storage and execution
|
|
||||||
- Security considerations
|
|
||||||
- Performance optimization
|
|
||||||
- Debugging techniques
|
|
||||||
- Advanced topics
|
|
||||||
|
|
||||||
**Audience:** Power users, script developers, plugin creators
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### [SECURITY_ARCHITECTURE.md](SECURITY_ARCHITECTURE.md)
|
|
||||||
**In-depth security architecture and implementation.**
|
|
||||||
|
|
||||||
**Topics Covered:**
|
|
||||||
- Security principles and threat model
|
|
||||||
- Authentication methods (password, TOTP, OpenID)
|
|
||||||
- Session management
|
|
||||||
- Authorization and protected sessions
|
|
||||||
- Encryption (notes, transport, backups)
|
|
||||||
- Input sanitization (XSS, SQL injection, CSRF)
|
|
||||||
- Network security (HTTPS, headers, rate limiting)
|
|
||||||
- Data security and secure deletion
|
|
||||||
- Dependency security
|
|
||||||
- Security best practices
|
|
||||||
- Incident response
|
|
||||||
|
|
||||||
**Audience:** Security engineers, administrators, auditors
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔧 Developer Documentation
|
|
||||||
|
|
||||||
### [Developer Guide](Developer%20Guide/Developer%20Guide/)
|
|
||||||
Collection of developer-focused documentation for contributing to Trilium.
|
|
||||||
|
|
||||||
**Key Documents:**
|
|
||||||
- [Environment Setup](Developer%20Guide/Developer%20Guide/Environment%20Setup.md) - Setting up development environment
|
|
||||||
- [Project Structure](Developer%20Guide/Developer%20Guide/Project%20Structure.md) - Monorepo organization
|
|
||||||
- [Development and Architecture](Developer%20Guide/Developer%20Guide/Development%20and%20architecture/) - Various development topics
|
|
||||||
|
|
||||||
**Topics Include:**
|
|
||||||
- Local development setup
|
|
||||||
- Building and deployment
|
|
||||||
- Adding new note types
|
|
||||||
- Database schema details
|
|
||||||
- Internationalization
|
|
||||||
- Icons and UI customization
|
|
||||||
- Docker development
|
|
||||||
- Troubleshooting
|
|
||||||
|
|
||||||
**Audience:** Contributors, developers
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📖 User Documentation
|
|
||||||
|
|
||||||
### [User Guide](User%20Guide/User%20Guide/)
|
|
||||||
Comprehensive end-user documentation for using Trilium.
|
|
||||||
|
|
||||||
**Key Sections:**
|
|
||||||
- Installation & Setup
|
|
||||||
- Basic Concepts and Features
|
|
||||||
- Note Types
|
|
||||||
- Advanced Usage
|
|
||||||
- Synchronization
|
|
||||||
- Import/Export
|
|
||||||
|
|
||||||
**Audience:** End users, administrators
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### [Script API](Script%20API/)
|
|
||||||
Complete API reference for user scripting.
|
|
||||||
|
|
||||||
**Coverage:**
|
|
||||||
- Frontend API methods
|
|
||||||
- Backend API methods
|
|
||||||
- Entity properties and methods
|
|
||||||
- Event handlers
|
|
||||||
- Utility functions
|
|
||||||
|
|
||||||
**Audience:** Script developers, power users
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🚀 Quick Start Guides
|
|
||||||
|
|
||||||
### For Users
|
|
||||||
1. [Installation Guide](User%20Guide/User%20Guide/Installation%20&%20Setup/) - Get Trilium running
|
|
||||||
2. [Basic Concepts](User%20Guide/User%20Guide/Basic%20Concepts%20and%20Features/) - Learn the fundamentals
|
|
||||||
3. [Scripting Guide](SCRIPTING.md) - Extend Trilium with scripts
|
|
||||||
|
|
||||||
### For Developers
|
|
||||||
1. [Environment Setup](Developer%20Guide/Developer%20Guide/Environment%20Setup.md) - Setup development environment
|
|
||||||
2. [Architecture Overview](ARCHITECTURE.md) - Understand the system
|
|
||||||
3. [Contributing Guide](../README.md#-contribute) - Start contributing
|
|
||||||
|
|
||||||
### For Administrators
|
|
||||||
1. [Server Installation](User%20Guide/User%20Guide/Installation%20&%20Setup/Server%20Installation.md) - Deploy Trilium server
|
|
||||||
2. [Synchronization Setup](SYNCHRONIZATION.md) - Configure sync
|
|
||||||
3. [Security Best Practices](SECURITY_ARCHITECTURE.md#security-best-practices) - Secure your installation
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔍 Documentation by Topic
|
|
||||||
|
|
||||||
### Architecture & Design
|
|
||||||
- [Overall Architecture](ARCHITECTURE.md)
|
|
||||||
- [Monorepo Structure](ARCHITECTURE.md#monorepo-structure)
|
|
||||||
- [Three-Layer Cache System](ARCHITECTURE.md#three-layer-cache-system)
|
|
||||||
- [Entity System](ARCHITECTURE.md#entity-system)
|
|
||||||
- [Widget-Based UI](ARCHITECTURE.md#widget-based-ui)
|
|
||||||
|
|
||||||
### Data & Storage
|
|
||||||
- [Database Architecture](DATABASE.md)
|
|
||||||
- [Entity Tables](DATABASE.md#entity-tables)
|
|
||||||
- [Data Relationships](DATABASE.md#data-relationships)
|
|
||||||
- [Blob Storage](DATABASE.md#blobs-table)
|
|
||||||
- [Database Migrations](DATABASE.md#database-migrations)
|
|
||||||
|
|
||||||
### Synchronization
|
|
||||||
- [Sync Architecture](SYNCHRONIZATION.md#sync-architecture)
|
|
||||||
- [Sync Protocol](SYNCHRONIZATION.md#sync-protocol)
|
|
||||||
- [Conflict Resolution](SYNCHRONIZATION.md#conflict-resolution)
|
|
||||||
- [Protected Notes Sync](SYNCHRONIZATION.md#protected-notes-sync)
|
|
||||||
- [WebSocket Sync](SYNCHRONIZATION.md#websocket-sync-updates)
|
|
||||||
|
|
||||||
### Security
|
|
||||||
- [Authentication](SECURITY_ARCHITECTURE.md#authentication)
|
|
||||||
- [Encryption](SECURITY_ARCHITECTURE.md#encryption)
|
|
||||||
- [Input Sanitization](SECURITY_ARCHITECTURE.md#input-sanitization)
|
|
||||||
- [Network Security](SECURITY_ARCHITECTURE.md#network-security)
|
|
||||||
- [Security Best Practices](SECURITY_ARCHITECTURE.md#security-best-practices)
|
|
||||||
|
|
||||||
### Scripting & Extensibility
|
|
||||||
- [Script Types](SCRIPTING.md#script-types)
|
|
||||||
- [Frontend API](SCRIPTING.md#frontend-api)
|
|
||||||
- [Backend API](SCRIPTING.md#backend-api)
|
|
||||||
- [Script Examples](SCRIPTING.md#script-examples)
|
|
||||||
- [Custom Widgets](SCRIPTING.md#render-scripts)
|
|
||||||
|
|
||||||
### Frontend
|
|
||||||
- [Client Architecture](ARCHITECTURE.md#frontend-architecture)
|
|
||||||
- [Widget System](ARCHITECTURE.md#widget-based-ui)
|
|
||||||
- [Event System](ARCHITECTURE.md#event-system)
|
|
||||||
- [Froca Cache](ARCHITECTURE.md#2-froca-frontend-cache)
|
|
||||||
- [UI Components](ARCHITECTURE.md#ui-components)
|
|
||||||
|
|
||||||
### Backend
|
|
||||||
- [Server Architecture](ARCHITECTURE.md#backend-architecture)
|
|
||||||
- [Service Layer](ARCHITECTURE.md#service-layer)
|
|
||||||
- [Route Structure](ARCHITECTURE.md#route-structure)
|
|
||||||
- [Becca Cache](ARCHITECTURE.md#1-becca-backend-cache)
|
|
||||||
- [Middleware](ARCHITECTURE.md#middleware)
|
|
||||||
|
|
||||||
### Build & Deploy
|
|
||||||
- [Build System](ARCHITECTURE.md#build-system)
|
|
||||||
- [Package Manager](ARCHITECTURE.md#package-manager-pnpm)
|
|
||||||
- [Build Tools](ARCHITECTURE.md#build-tools)
|
|
||||||
- [Docker](Developer%20Guide/Developer%20Guide/Development%20and%20architecture/Docker.md)
|
|
||||||
- [Deployment](Developer%20Guide/Developer%20Guide/Building%20and%20deployment/)
|
|
||||||
|
|
||||||
### Testing
|
|
||||||
- [Testing Strategy](ARCHITECTURE.md#testing-strategy)
|
|
||||||
- [Test Organization](ARCHITECTURE.md#test-organization)
|
|
||||||
- [E2E Testing](ARCHITECTURE.md#e2e-testing)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📋 Reference Documentation
|
|
||||||
|
|
||||||
### File Locations
|
|
||||||
```
|
|
||||||
trilium/
|
|
||||||
├── apps/
|
|
||||||
│ ├── client/ # Frontend application
|
|
||||||
│ ├── server/ # Backend server
|
|
||||||
│ ├── desktop/ # Electron app
|
|
||||||
│ └── ...
|
|
||||||
├── packages/
|
|
||||||
│ ├── commons/ # Shared code
|
|
||||||
│ ├── ckeditor5/ # Rich text editor
|
|
||||||
│ └── ...
|
|
||||||
├── docs/
|
|
||||||
│ ├── ARCHITECTURE.md # Main architecture doc
|
|
||||||
│ ├── DATABASE.md # Database documentation
|
|
||||||
│ ├── SYNCHRONIZATION.md # Sync documentation
|
|
||||||
│ ├── SCRIPTING.md # Scripting guide
|
|
||||||
│ ├── SECURITY_ARCHITECTURE.md # Security documentation
|
|
||||||
│ ├── Developer Guide/ # Developer docs
|
|
||||||
│ ├── User Guide/ # User docs
|
|
||||||
│ └── Script API/ # API reference
|
|
||||||
└── ...
|
|
||||||
```
|
|
||||||
|
|
||||||
### Key Source Files
|
|
||||||
- **Backend Entry:** `apps/server/src/main.ts`
|
|
||||||
- **Frontend Entry:** `apps/client/src/desktop.ts` / `apps/client/src/index.ts`
|
|
||||||
- **Becca Cache:** `apps/server/src/becca/becca.ts`
|
|
||||||
- **Froca Cache:** `apps/client/src/services/froca.ts`
|
|
||||||
- **Database Schema:** `apps/server/src/assets/db/schema.sql`
|
|
||||||
- **Backend API:** `apps/server/src/services/backend_script_api.ts`
|
|
||||||
- **Frontend API:** `apps/client/src/services/frontend_script_api.ts`
|
|
||||||
|
|
||||||
### Important Directories
|
|
||||||
- **Entities:** `apps/server/src/becca/entities/`
|
|
||||||
- **Widgets:** `apps/client/src/widgets/`
|
|
||||||
- **Services:** `apps/server/src/services/`
|
|
||||||
- **Routes:** `apps/server/src/routes/`
|
|
||||||
- **Migrations:** `apps/server/src/migrations/`
|
|
||||||
- **Tests:** Various `*.spec.ts` files throughout
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🎯 Common Tasks
|
|
||||||
|
|
||||||
### Understanding the Codebase
|
|
||||||
1. Read [ARCHITECTURE.md](ARCHITECTURE.md) for overview
|
|
||||||
2. Explore [Monorepo Structure](ARCHITECTURE.md#monorepo-structure)
|
|
||||||
3. Review [Entity System](ARCHITECTURE.md#entity-system)
|
|
||||||
4. Check [Key Files](ARCHITECTURE.md#key-files-for-understanding-architecture)
|
|
||||||
|
|
||||||
### Adding Features
|
|
||||||
1. Review relevant architecture documentation
|
|
||||||
2. Check [Developer Guide](Developer%20Guide/Developer%20Guide/)
|
|
||||||
3. Follow existing patterns in codebase
|
|
||||||
4. Write tests
|
|
||||||
5. Update documentation
|
|
||||||
|
|
||||||
### Debugging Issues
|
|
||||||
1. Check [Troubleshooting](Developer%20Guide/Developer%20Guide/Troubleshooting/)
|
|
||||||
2. Review [Database](DATABASE.md) for data issues
|
|
||||||
3. Check [Synchronization](SYNCHRONIZATION.md) for sync issues
|
|
||||||
4. Review [Security](SECURITY_ARCHITECTURE.md) for auth issues
|
|
||||||
|
|
||||||
### Performance Optimization
|
|
||||||
1. [Database Performance](DATABASE.md#performance-optimization)
|
|
||||||
2. [Cache Optimization](ARCHITECTURE.md#caching-system)
|
|
||||||
3. [Build Optimization](ARCHITECTURE.md#build-system)
|
|
||||||
4. [Script Performance](SCRIPTING.md#performance-considerations)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔗 External Resources
|
|
||||||
|
|
||||||
### Official Links
|
|
||||||
- **Website:** https://triliumnotes.org
|
|
||||||
- **Documentation:** https://docs.triliumnotes.org
|
|
||||||
- **GitHub:** https://github.com/TriliumNext/Trilium
|
|
||||||
- **Discussions:** https://github.com/TriliumNext/Trilium/discussions
|
|
||||||
- **Matrix Chat:** https://matrix.to/#/#triliumnext:matrix.org
|
|
||||||
|
|
||||||
### Community Resources
|
|
||||||
- **Awesome Trilium:** https://github.com/Nriver/awesome-trilium
|
|
||||||
- **TriliumRocks:** https://trilium.rocks/
|
|
||||||
- **Wiki:** https://triliumnext.github.io/Docs/Wiki/
|
|
||||||
|
|
||||||
### Related Projects
|
|
||||||
- **TriliumDroid:** https://github.com/FliegendeWurst/TriliumDroid
|
|
||||||
- **Web Clipper:** Included in main repository
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📝 Documentation Conventions
|
|
||||||
|
|
||||||
### Document Structure
|
|
||||||
- Overview section
|
|
||||||
- Table of contents
|
|
||||||
- Main content with headings
|
|
||||||
- Code examples where relevant
|
|
||||||
- "See Also" references
|
|
||||||
|
|
||||||
### Code Examples
|
|
||||||
```typescript
|
|
||||||
// TypeScript examples with comments
|
|
||||||
const example = 'value'
|
|
||||||
```
|
|
||||||
|
|
||||||
```sql
|
|
||||||
-- SQL examples with formatting
|
|
||||||
SELECT * FROM notes WHERE noteId = ?
|
|
||||||
```
|
|
||||||
|
|
||||||
### Cross-References
|
|
||||||
- Use relative links: `[text](path/to/file.md)`
|
|
||||||
- Reference sections: `[text](file.md#section)`
|
|
||||||
- External links: Full URLs
|
|
||||||
|
|
||||||
### Maintenance
|
|
||||||
- Review on major releases
|
|
||||||
- Update for architectural changes
|
|
||||||
- Add examples for new features
|
|
||||||
- Keep API references current
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🤝 Contributing to Documentation
|
|
||||||
|
|
||||||
### What to Document
|
|
||||||
- New features and APIs
|
|
||||||
- Architecture changes
|
|
||||||
- Migration guides
|
|
||||||
- Performance tips
|
|
||||||
- Security considerations
|
|
||||||
|
|
||||||
### How to Contribute
|
|
||||||
1. Edit markdown files in `docs/`
|
|
||||||
2. Follow existing structure and style
|
|
||||||
3. Include code examples
|
|
||||||
4. Test links and formatting
|
|
||||||
5. Submit pull request
|
|
||||||
|
|
||||||
### Documentation Standards
|
|
||||||
- Clear, concise language
|
|
||||||
- Complete code examples
|
|
||||||
- Proper markdown formatting
|
|
||||||
- Cross-references to related docs
|
|
||||||
- Updated version numbers
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📅 Version Information
|
|
||||||
|
|
||||||
- **Documentation Version:** 0.99.3
|
|
||||||
- **Last Updated:** November 2025
|
|
||||||
- **Trilium Version:** 0.99.3+
|
|
||||||
- **Next Review:** When major architectural changes occur
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 💡 Getting Help
|
|
||||||
|
|
||||||
### For Users
|
|
||||||
- [User Guide](User%20Guide/User%20Guide/)
|
|
||||||
- [GitHub Discussions](https://github.com/TriliumNext/Trilium/discussions)
|
|
||||||
- [Matrix Chat](https://matrix.to/#/#triliumnext:matrix.org)
|
|
||||||
|
|
||||||
### For Developers
|
|
||||||
- [Developer Guide](Developer%20Guide/Developer%20Guide/)
|
|
||||||
- [Architecture Docs](ARCHITECTURE.md)
|
|
||||||
- [GitHub Issues](https://github.com/TriliumNext/Trilium/issues)
|
|
||||||
|
|
||||||
### For Contributors
|
|
||||||
- [Contributing Guidelines](../README.md#-contribute)
|
|
||||||
- [Code of Conduct](../CODE_OF_CONDUCT)
|
|
||||||
- [Developer Setup](Developer%20Guide/Developer%20Guide/Environment%20Setup.md)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Maintained by:** TriliumNext Team
|
|
||||||
**License:** AGPL-3.0-only
|
|
||||||
**Repository:** https://github.com/TriliumNext/Trilium
|
|
||||||
Reference in New Issue
Block a user