Sync

Eidetica uses a Merkle-CRDT based sync protocol. Peers exchange tips (current DAG heads) and send only the entries the other is missing.

Sync Flow

sequenceDiagram
    participant A as Peer A
    participant B as Peer B

    A->>B: SyncTreeRequest (my tips)
    B->>A: Response (entries you're missing, my tips)
    A->>B: SendEntries (entries you're missing)

Peer A sends its current tips for a Tree
Peer B compares DAGs, returns entries A is missing plus B's tips
A sends entries B is missing based on the tip comparison

This is stateless and self-correcting - no tracking of previously synced entries.

Bootstrap vs Incremental

The same protocol handles both cases:

Empty tips (new database): Peer sends complete Tree from root
Has tips (existing database): Peer sends only missing entries

Transport Options

HTTP: REST API for server-based sync
Iroh P2P: QUIC-based with NAT traversal for peer-to-peer sync

Both transports implement the same SyncTransport trait. Multiple transports can be enabled simultaneously.

Architecture

graph LR
    App[Application] --> Sync[Sync Module]
    Sync --> BG[Background Thread]
    BG --> TM[TransportManager]
    TM --> HTTP[HTTP Transport]
    TM --> Iroh[Iroh Transport]

The Sync module queues operations for a background thread, which uses a TransportManager to route requests to the appropriate transport based on address type. Failed sends are retried with exponential backoff.

Multi-Transport Support

The TransportManager enables simultaneous use of multiple transports:

Address-based routing: Each transport declares which addresses it can handle via can_handle_address()
First-match routing: Requests are routed to the first transport that can handle the address
Independent servers: Each transport runs its own server on different ports/protocols
Unified API: accept_connections() starts all registered transports, get_all_server_addresses_async() returns all addresses

Current Limitations

The sync system is currently simple and 1:1. Each peer connection requires manual setup with explicit peer addresses. Planned improvements include:

Peer discovery
Address sharing and relay coordination
Multi-peer sync orchestration

See Bootstrap System for the key exchange flow when joining a database.

Keyboard shortcuts