Implementation Status: 🔵 Proposed

Users System

This design document outlines a comprehensive multi-user system for Eidetica that provides user isolation, password-based authentication, and per-user key management.

Problem Statement

The current implementation has no concept of users:

No User Isolation: All keys and settings are stored at the Instance level, shared across all operations.
No Authentication: There's no way to protect access to private keys or restrict database operations to specific users.
No Multi-User Support: Only one implicit "user" can work with an Instance at a time.
Key Management Challenges: All private keys are accessible to anyone with Instance access, with no encryption or access control.
No User Preferences: Users cannot have personalized settings for which databases they care about, sync preferences, etc.

Goals

Unified Architecture: Single implementation that supports both embedded (single-user ergonomics) and server (multi-user) use cases.
Multi-User Support: Multiple users can have accounts on a single Instance, each with isolated keys and preferences.
Password-Based Authentication: Users authenticate with passwords to access their keys and perform operations.
User Isolation: Each user's private keys and preferences are encrypted and isolated from other users.
Root User: A special system user that the Instance uses for infrastructure operations.
User Preferences: Users can configure which databases they care about and how they want to sync them.
Database Tracking: Instance-wide visibility into which databases exist and which users access them.
Ergonomic APIs: Simple single-user API for embedded apps, explicit multi-user API for servers (both build on same foundation).

Non-Goals

Multi-Factor Authentication: Advanced auth methods deferred to future work.
Role-Based Access Control: Complex permission systems beyond user isolation are out of scope.
User Groups: Team/organization features are not included.
Federated Identity: External identity providers are not addressed.

Proposed Solution

Architecture Overview

The system separates infrastructure management (Instance) from contextual operations (User):

Instance (Infrastructure Layer)
├── Backend Storage (local only, not in databases)
│   └── _device_key (SigningKey for Instance identity)
│
├── System Databases (separate databases, authenticated with _device_key)
│   ├── _instance
│   │   └── Instance configuration and metadata
│   ├── _users (Table with UUID primary keys)
│   │   └── User directory: Maps UUID → UserInfo (username stored in UserInfo)
│   ├── _databases
│   │   └── Database tracking: Maps database_id → DatabaseTracking
│   └── _sync
│       └── Sync configuration and bootstrap requests
│
└── User Management
    ├── User creation (with or without password)
    └── User login (returns User session)

User (Operations Layer - returned from login)
├── User session with decrypted keys
├── Database operations (new, load, find)
├── Key management (add, list, get)
└── User preferences

Key Architectural Principle: Instance handles infrastructure (user accounts, backend, system databases). User handles all contextual operations (database creation, key management). All operations run in a User context after login.

Core Data Structures

1. UserInfo (stored in `_users` database)

Storage: Users are stored in a Table with auto-generated UUID primary keys. The username field is used for login lookups via search operations.

#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct UserInfo {
    /// Unique username (login identifier)
    /// Note: Stored with UUID primary key in Table, username used for search
    pub username: String,

    /// ID of the user's private database
    pub user_database_id: ID,

    /// Password hash (using Argon2id)
    /// None for passwordless users (single-user embedded mode)
    pub password_hash: Option<String>,

    /// Salt for password hashing (base64 encoded string)
    /// None for passwordless users (single-user embedded mode)
    pub password_salt: Option<String>,

    /// User account creation timestamp
    pub created_at: u64,

    /// Last login timestamp
    pub last_login: Option<u64>,

    /// Account status
    pub status: UserStatus,
}

#[derive(Clone, Debug, Serialize, Deserialize)]
pub enum UserStatus {
    Active,
    Disabled,
    Locked,
}

2. UserProfile (stored in user's private database `_settings` subtree)

#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct UserProfile {
    /// Username
    pub username: String,

    /// Display name
    pub display_name: Option<String>,

    /// Email or other contact info
    pub contact_info: Option<String>,

    /// User preferences
    pub preferences: UserPreferences,
}

#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct UserPreferences {
    /// Default sync behavior
    pub default_sync_enabled: bool,

    /// Other user-specific settings
    pub properties: HashMap<String, String>,
}

3. UserKey (stored in user's private database `keys` subtree)

#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct UserKey {
    /// Key identifier (typically the base64-encoded public key string)
    pub key_id: String,

    /// Private key bytes (encrypted or unencrypted based on encryption field)
    pub private_key_bytes: Vec<u8>,

    /// Encryption metadata
    pub encryption: KeyEncryption,

    /// Display name for this key
    pub display_name: Option<String>,

    /// When this key was created
    pub created_at: u64,

    /// Last time this key was used
    pub last_used: Option<u64>,

    /// Database-specific SigKey mappings
    /// Maps: Database ID → SigKey used in that database's auth settings
    pub database_sigkeys: HashMap<ID, String>,
}

#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(tag = "type", rename_all = "lowercase")]
pub enum KeyEncryption {
    /// Key is encrypted with password-derived key
    Encrypted {
        /// Encryption nonce/IV (12 bytes for AES-GCM)
        nonce: Vec<u8>,
    },
    /// Key is stored unencrypted (passwordless users only)
    Unencrypted,
}

4. UserDatabasePreferences (stored in user's private database `databases` Table)

Purpose: Tracks which databases a user cares about and their per-user sync preferences. The User tracks preferences (what the user wants), while the Sync module tracks status (what's happening). This separation allows multiple users with different sync preferences to sync the same database in a single Instance.

#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct UserDatabasePreferences {
    /// Database ID being tracked
    pub database_id: ID,

    /// Which user key to use for this database
    pub key_id: String,

    /// User's sync preferences for this database
    pub sync_settings: SyncSettings,

    /// When user added this database
    pub added_at: i64,
}

#[derive(Clone, Debug, Serialize, Deserialize, Default)]
pub struct SyncSettings {
    /// Whether user wants to sync this database
    pub sync_enabled: bool,

    /// Sync on commit
    pub sync_on_commit: bool,

    /// Sync interval (if periodic)
    pub interval_seconds: Option<u64>,

    /// Additional sync configuration
    pub properties: HashMap<String, String>,
}

#[derive(Clone, Debug)]
pub struct DatabasePreferences {
    /// Database ID to add/update
    pub database_id: ID,

    /// Which user key to use for this database
    pub key_id: String,

    /// Sync settings for this database
    pub sync_settings: SyncSettings,
}

Design Notes:

SigKey Discovery: When adding a database via add_database(), the system automatically discovers which SigKey the user can use via Database::find_sigkeys(), selecting the highest-permission SigKey available. The discovered SigKey is stored in UserKey.database_sigkeys HashMap.
Separation of Concerns: The key_id in UserDatabasePreferences references the user's key, while the actual SigKey mapping is stored in UserKey.database_sigkeys. This allows the same key to use different SigKeys in different databases.
Sync Settings vs Sync Status: User preferences indicate what the user wants (sync_enabled, sync_on_commit), while the Sync module tracks actual sync status (last_synced, connection state). Multiple users can have different preferences for the same database.

5. DatabaseTracking (stored in `_databases` table)

#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct DatabaseTracking {
    /// Database ID (this is the key in the table)
    pub database_id: ID,

    /// Cached database name (for quick lookup)
    pub name: Option<String>,

    /// Users who have this database in their preferences
    pub users: Vec<String>,

    /// Database creation time
    pub created_at: u64,

    /// Last modification time
    pub last_modified: u64,

    /// Additional metadata
    pub metadata: HashMap<String, String>,
}

System Databases

The Instance manages four separate system databases, all authenticated with _device_key:

`_instance` System Database

Type: Separate database
Purpose: Instance configuration and management
Structure: Configuration settings, metadata, system policies
Authentication: _device_key as Admin; admin users can be granted access
Access: Admin users have Admin permission, regular users have Read permission
Created: On Instance initialization

`_users` System Database

Type: Separate database
Purpose: User directory and authentication
Structure: Table with UUID primary keys, stores UserInfo (username field for login lookups)
Authentication: _device_key as Admin
Access: Admin users can manage users
Created: On Instance initialization
Note: Username uniqueness enforced at application layer via search; see Race Conditions section

`_databases` System Database

Type: Separate database
Purpose: Instance-wide database registry and optimization
Structure: Table mapping database_id → DatabaseTracking
Authentication: _device_key as Admin
Maintenance: Updated when users add/remove databases from preferences
Benefits: Fast discovery of databases, see which users care about each DB
Created: On Instance initialization

`_sync` System Database

Type: Separate database (existing)
Purpose: Synchronization configuration and bootstrap request management
Structure: Various subtrees for sync settings, peer info, bootstrap requests
Authentication: _device_key as Admin
Access: Managed by Instance and Sync module
Created: When sync is enabled via Instance::enable_sync()

Instance Identity vs User Management

The Instance identity is separate from user management:

Instance Identity

The Instance uses _device_key for its identity:

Storage: Stored in backend (local storage, not in any database)
Purpose: Instance sync identity and system database authentication
Access: Available to Instance on startup (no password required)
Usage: Used to authenticate to all system databases as Admin

User Management

Users are created by administrators or self-registration:

#![allow(unused)]
fn main() {
/// Users authenticate with passwords
/// Each has isolated key storage and preferences
/// Must login to perform operations
}

User Lifecycle:

Created via Instance::create_user() by an admin
User logs in via Instance::login_user()
User session provides access to keys and preferences
User logs out via User::logout()

Library Architecture Layers

The library separates infrastructure (Instance) from contextual operations (User):

Instance Layer: Infrastructure Management

Instance manages the multi-user infrastructure and system resources:

Initialization:

Load or generate _device_key from backend
Create system databases (_instance, _users, _databases) authenticated with _device_key
Initialize Instance with backend and system databases

Responsibilities:

User account management (create, login)
System database maintenance
Backend coordination
Database tracking

Key Points:

Instance is always multi-user underneath
No direct database or key operations
All operations require a User session

User Layer: Contextual Operations

User represents an authenticated session with decrypted keys:

Creation:

Returned from Instance::login_user(username, Option<password>)
Contains decrypted private keys in memory
Has access to user's preferences and database mappings

Responsibilities:

Database operations (create_database, open_database, find_database)
Key management (add_private_key, list_keys, get_signing_key)
Database preferences
Bootstrap approval

Key Points:

All database creation and key management happens through User
Keys are zeroized on logout or drop
Clean separation between users

Passwordless Users

For embedded/single-user scenarios, users can be created without passwords:

Creation:

// Create passwordless user
instance.create_user("alice", None)?;

// Login without password
let user = instance.login_user("alice", None)?;

// Use User API normally
let db = user.new_database(settings)?;

Characteristics:

No authentication overhead
Keys stored unencrypted in user database
Perfect for embedded apps, CLI tools, single-user deployments
Still uses full User API for operations

Password-Protected Users

For multi-user scenarios, users have password-based authentication:

Creation:

// Create password-protected user
instance.create_user("bob", Some("password123"))?;

// Login with password verification
let user = instance.login_user("bob", Some("password123"))?;

// Use User API normally
let db = user.new_database(settings)?;

Characteristics:

Argon2id password hashing
AES-256-GCM key encryption
Perfect for servers, multi-tenant applications
Clear separation between users

Instance API

Instance manages infrastructure and user accounts:

Initialization

impl Instance {
    /// Create instance
    /// - Loads/generates _device_key from backend
    /// - Creates system databases (_instance, _users, _databases)
    pub fn open(backend: Box<dyn BackendImpl>) -> Result<Self>;
}

User Management

impl Instance {
    /// Create a new user account
    /// Returns user_uuid (the generated primary key)
    pub fn create_user(
        &self,
        username: &str,
        password: Option<&str>,
    ) -> Result<String>;

    /// Login a user (returns User session object)
    /// Searches by username; errors if duplicate usernames detected
    pub fn login_user(
        &self,
        username: &str,
        password: Option<&str>,
    ) -> Result<User>;

    /// List all users (returns usernames)
    pub fn list_users(&self) -> Result<Vec<String>>;

    /// Disable a user account
    pub fn disable_user(&self, username: &str) -> Result<()>;
}

User API

/// User session object, returned after successful login
///
/// Represents an authenticated user with decrypted private keys loaded in memory.
/// All contextual operations (database creation, key management) happen through User.
pub struct User {
    user_uuid: String,   // Stable internal UUID (Table primary key)
    username: String,    // Username (login identifier)
    user_database: Database,
    instance: WeakInstance,  // Weak reference to Instance for storage access
    /// Decrypted user keys (in memory only during session)
    key_manager: UserKeyManager,
}

impl User {
    /// Get the internal user UUID (stable identifier)
    pub fn user_uuid(&self) -> &str;

    /// Get the username (login identifier)
    pub fn username(&self) -> &str;

    // === Database Operations ===

    /// Create a new database in this user's context
    pub fn create_database(&self, settings: Doc, signing_key: &str) -> Result<Database>;

    /// Load a database using this user's keys
    pub fn open_database(&self, database_id: &ID) -> Result<Database>;

    /// Find databases by name
    pub fn find_database(&self, name: impl AsRef<str>) -> Result<Vec<Database>>;

    /// Find the best key for accessing a database

    /// Get the SigKey mapping for a key in a specific database
    pub fn key_mapping(
        &self,
        key_id: &str,
        database_id: &ID,
    ) -> Result<Option<String>>;

    /// Add a SigKey mapping for a key in a specific database
    pub fn map_key(
        &mut self,
        key_id: &str,
        database_id: &ID,
        sigkey: &str,
    ) -> Result<()>;

    // === Database Tracking and Preferences ===

    /// Add a database to this user's tracked databases with auto-discovery of SigKeys.
    pub fn add_database(
        &mut self,
        prefs: DatabasePreferences,
    ) -> Result<()>;

    /// List all databases this user is tracking.
    pub fn list_database_prefs(&self) -> Result<Vec<UserDatabasePreferences>>;

    /// Get the preferences for a specific database.
    pub fn database_prefs(
        &self,
        database_id: &ID,
    ) -> Result<UserDatabasePreferences>;

    /// Set/update preferences for a database (upsert behavior).
    /// Alias for add_database.
    pub fn set_database(
        &mut self,
        prefs: DatabasePreferences,
    ) -> Result<()>;

    /// Remove a database from this user's tracked databases.
    pub fn remove_database(&mut self, database_id: &ID) -> Result<()>;

    // === Key Management ===

    /// Generate a new private key for this user
    pub fn add_private_key(
        &mut self,
        display_name: Option<&str>,
    ) -> Result<String>;

    /// List all key IDs owned by this user
    pub fn list_keys(&self) -> Result<Vec<String>>;

    /// Get a signing key by its ID
    pub fn get_signing_key(&self, key_id: &str) -> Result<SigningKey>;

    // === Session Management ===

    /// Logout (clears decrypted keys from memory)
    pub fn logout(self) -> Result<()>;
}

UserKeyManager (Internal)

/// Internal key manager that holds decrypted keys during user session
struct UserKeyManager {
    /// Decrypted keys (key_id → SigningKey)
    decrypted_keys: HashMap<String, SigningKey>,

    /// Key metadata (loaded from user database)
    key_metadata: HashMap<String, UserKey>,

    /// User's password-derived encryption key (for saving new keys)
    encryption_key: Vec<u8>,
}

See key_management.md for detailed implementation.

User Flows

User Creation Flow

Password-Protected User:

Admin calls instance.create_user(username, Some(password))
System searches _users Table for existing username (race condition possible)
System hashes password with Argon2id and random salt
Generates default Ed25519 keypair for the user (kept in memory only)
Retrieves instance _device_key public key from backend
Creates user database with authentication for both _device_key (Admin) and user's key (Admin)
Encrypts user's private key with password-derived key (AES-256-GCM)
Stores encrypted key in user database keys Table (using public key as identifier, signed with _device_key)
Creates UserInfo and inserts into _users Table (auto-generates UUID primary key)
Returns user_uuid

Passwordless User:

Admin calls instance.create_user(username, None)
System searches _users Table for existing username (race condition possible)
Generates default Ed25519 keypair for the user (kept in memory only)
Retrieves instance _device_key public key from backend
Creates user database with authentication for both _device_key (Admin) and user's key (Admin)
Stores unencrypted private key in user database keys Table (marked as Unencrypted)
Creates UserInfo with None for password fields and inserts into _users Table
Returns user_uuid

Note: For password-protected users, the keypair is never stored unencrypted in the backend. For passwordless users, keys are stored unencrypted for instant access. The user database is authenticated with both the instance _device_key (for admin operations) and the user's default key (for user ownership). Initial entries are signed with _device_key.

Password-Protected User:

User calls instance.login_user(username, Some(password))
System searches _users Table by username
If multiple users with same username found, returns DuplicateUsersDetected error
Verifies password against stored hash
Loads user's private database
Loads encrypted keys from user database
Derives encryption key from password
Decrypts all private keys
Creates UserKeyManager with decrypted keys
Updates last_login timestamp in _users Table (using UUID)
Returns User session object (contains both user_uuid and username)

Passwordless User:

User calls instance.login_user(username, None)
System searches _users Table by username
If multiple users with same username found, returns DuplicateUsersDetected error
Verifies UserInfo has no password (password_hash and password_salt are None)
Loads user's private database
Loads unencrypted keys from user database
Creates UserKeyManager with keys (no decryption needed)
Returns User session object (contains both user_uuid and username)

Database Creation Flow

User obtains User session via login
User creates database settings (Doc with name, etc.)
Calls user.new_database(settings)
System selects first available signing key from user's keyring
Creates database using Database::new() for root entry creation
Stores database_sigkeys mapping in UserKey for future loads
Returns Database object
User can now create transactions and perform operations on the database

Database Access Flow

The user accesses databases through the User.open_database() method, which handles all key management automatically:

User calls user.open_database(&database_id)
System finds appropriate key via find_key()
- Checks user's key metadata for SigKey mappings to this database
- Verifies keys are authorized in database's auth settings
- Selects key with highest permission level
System retrieves decrypted SigningKey from UserKeyManager
System gets SigKey mapping via key_mapping()
System loads Database with Database::open()
- Database stores KeySource::Provided with signing key and sigkey
User creates transactions normally: database.new_transaction()
- Transaction automatically receives provided key from Database
- No backend key lookup required
User performs operations and commits
- Transaction uses provided SigningKey directly during commit()

Key Insight: Once a Database is loaded via User.open_database(), all subsequent operations transparently use the user's keys. The user doesn't need to think about key management - it's handled at database load time.

Key Addition Flow

Password-Protected User:

User calls user.add_private_key(display_name)
System generates new Ed25519 keypair
Encrypts private key with user's password-derived key (AES-256-GCM)
Creates UserKey metadata with Encrypted variant
Stores encrypted key in user database
Adds to in-memory UserKeyManager
Returns key_id

Passwordless User:

User calls user.add_private_key(display_name)
System generates new Ed25519 keypair
Creates UserKey metadata with Unencrypted variant
Stores unencrypted key in user database
Adds to in-memory UserKeyManager
Returns key_id

Bootstrap Integration

The Users system integrates with the bootstrap protocol for access control:

User Authentication: Bootstrap requests approved by logged-in users
Permission Checking: Only users with a key that has Admin permission for the database can approve bootstrap requests
Key Discovery: User's key manager finds appropriate Admin key for database
Transaction Creation: Uses user's Admin key SigKey to add requesting key to database auth

See bootstrap.md for detailed bootstrap protocol and wildcard permissions.

Integration with Key Management

The key management design (see key_management.md) provides the technical implementation details for:

Password-Derived Encryption: How user passwords are used to derive encryption keys for private key storage
Key Encryption Format: Specific encryption algorithms and formats used
Database ID → SigKey Mapping: Technical structure and storage
Key Discovery Algorithms: How keys are matched to databases and permissions

The Users system provides the architectural context:

Who owns keys (users)
How keys are isolated (user databases)
When keys are decrypted (during user session)
How keys are managed (User API)

Security Considerations

Password Security

Password Hashing: Use Argon2id for password hashing with appropriate parameters
Random Salts: Each user has a unique random salt
No Password Storage: Only hashes stored, never plaintext
Rate Limiting: Login attempts should be rate-limited

Key Encryption

Password-Derived Keys: Use PBKDF2 or Argon2 to derive encryption keys from passwords
Authenticated Encryption: Use AES-GCM or ChaCha20-Poly1305
Unique Nonces: Each encrypted key has a unique nonce/IV
Memory Security: Clear decrypted keys from memory on logout

User Isolation

Database-Level Isolation: Each user's private database is separate
Access Control: Users cannot access other users' databases or keys
Authentication Required: All user operations require valid session
Session Timeouts: Consider implementing session expiration

Instance Identity Protection

Backend Security: _device_key stored in backend with appropriate file permissions
Limited Exposure: _device_key only used for system database authentication
Audit Logging: Log Instance-level operations on system databases
Key Rotation: Support rotating _device_key (requires updating all system databases)

Known Limitations

Username Uniqueness Race Condition

Issue: Username uniqueness is enforced at the application layer using search-then-insert operations, which creates a race condition in distributed/concurrent scenarios.

Current Behavior:

create_user() searches for existing username, then inserts if not found
Two concurrent creates with same username can both succeed
Results in multiple UserInfo records with same username but different UUIDs

Detection:

login_user() searches by username
If multiple matches found, returns UserError::DuplicateUsersDetected
Prevents login until conflict is resolved manually

Performance Implications

Login Cost: Password hashing and key decryption add latency to login (acceptable)
Memory Usage: Decrypted keys held in memory during session
Database Tracking: O(1) lookup for database metadata and user lists (via UUID primary key)
Username Lookup: O(n) search for username validation/login (where n = total users)
Key Discovery: O(n) where n = number of user's keys (typically small)

Implementation Strategy

Phase 1: Core User Infrastructure

Define data structures (UserInfo, UserProfile, UserKey, etc.)
Implement password hashing and verification
Implement key encryption/decryption
Create _instance system database
Create _users system database
Create _databases tracking table
Unit tests for crypto and data structures

Phase 2: User Management API

Implement Instance::create_user()
Implement Instance::login_user()
Implement User struct and basic methods
Implement UserKeyManager
Integration tests for user creation and login

Phase 3: Key Management Integration

Implement User::add_private_key()
Implement User::set_database_sigkey()
Implement key discovery methods
Update Transaction to work with User sessions
Tests for key operations

Phase 4: Database Preferences

Implement database preference storage
Implement database tracking updates
Implement preference query APIs
Tests for preference management

Phase 5: Migration and Integration

Update existing code to work with Users
Provide migration utilities for existing instances
Update documentation and examples
End-to-end integration tests

Future Work

Multi-Factor Authentication: Add support for TOTP, hardware keys
User Groups/Roles: Team collaboration features
Permission Delegation: Allow users to delegate access to specific databases
Key Recovery: Secure key recovery mechanisms
Session Management: Advanced session features (multiple devices, revocation)
Audit Logs: Comprehensive logging of user operations
User Quotas: Storage and database limits per user

Conclusion

The Users system provides a clean separation between infrastructure (Instance) and contextual operations (User):

Core Architecture:

Instance manages infrastructure: user accounts, backend, system databases
User handles all contextual operations: database creation, key management
Separate system databases (_instance, _users, _databases, _sync)
Instance identity (_device_key) stored in backend for system database authentication
Strong isolation between users

User Types:

Passwordless Users: Optional password support enables instant login without authentication overhead, perfect for embedded apps
Password-Protected Users: Argon2id password hashing and AES-256-GCM key encryption for multi-user scenarios

Key Benefits:

Clean separation: Instance = infrastructure, User = operations
All operations run in User context after login
Flexible authentication: users can have passwords or not
Instance restart just loads _device_key from backend

Keyboard shortcuts

Eidetica Documentation