CRDT-Based App Synchronization System Design

Overview

This system provides a robust framework for synchronizing data between a local SQLite database and a remote Firestore database. It is designed to handle various data types (represented as CloudSyncable entities), media file uploads, conflict resolution (using Last Write Wins), and potential network interruptions.

Core Components

CloudSyncable Class

• Purpose: Serves as the base class for any data model that needs to be synchronized.
• Functionality:
  • Tracks timestamps for creation, updates, deletion, and synchronization.
  • Stores local and remote file paths (if applicable).
  • Provides methods to check if an item needs syncing (data or file) and whether it has been deleted.
  • Includes convenience methods for creating copies of the object and for generating full local file paths.

LocalRepository Interface

• Purpose: Defines the contract for interacting with the local SQLite database.
• Methods:
  • Standard CRUD operations (create, read, update, delete).
  • Synchronization-specific methods for getting unsynced items, marking items as synced, etc.
  • Utility methods for clearing all data and marking all items as unsynced.

SqliteLocalRepository Abstract Class

• Purpose: Provides a base implementation for SQLite-based local repositories.
• Functionality:
  • Implements common SQLite operations and SQL generation logic.
  • Subclasses provide entity-specific details.

RemoteRepository Interface

• Purpose: Defines the contract for interacting with the remote Firestore database.
• Methods:
  • getLatestServerSyncTime(): Gets the latest server sync time.
  • pushUnsyncedItems(List<T> items): Pushes unsynced items to the server.
  • fetchItemsByServerWrittenTimeRange({DateTime? from, required DateTime to}): Fetches items within a server-written timestamp range.

FirestoreRemoteRepository Class

• Purpose: Concrete implementation of RemoteRepository for Firestore.
• Functionality:
  • Handles communication with Firestore and data conversion.

SyncService Abstract Class

• Purpose: An abstract class for synchronizing CloudSyncable entities between local and remote repositories.
• Methods:
  • sync(): Orchestrates the entire sync process:
    1. Uploads local files if not synced.
    2. Fetches data from Firestore within the specified time range.
    3. Filters fetched items based on the LWW conflict resolution strategy.
    4. Updates the local repository with fetched items.
    5. Fetches unsynced items after conflict resolution.
    6. Pushes unsynced items to Firestore.
    7. Manually marks the pushed items as synced by updating their serverTimeSyncedAt and localTimeSyncedAt fields.
  • _filterFetchedItems(List<T> fetchedItems): Filters fetched items based on LWW.

How to Use CloudSyncable for a Specific Data Model (e.g., Task)

1. Create the Task Class: Extend CloudSyncable and add task-specific fields.
2. Create Repositories:
   • Implement LocalTaskRepository (extending SqliteLocalRepository<Task>).
   • Implement RemoteTaskRepository (extending FirestoreRemoteRepository<Task>).
3. Create TaskSyncService:
   • Extend SyncService<Task> to implement task-specific synchronization logic.

Flexibility and Extensibility

The CloudSyncable system provides a flexible and extensible foundation for data synchronization in Flutter applications. By following the established patterns, you can easily adapt it to handle various data models and cloud storage providers.