delta_trace_db

(en)Japanese ver is here.
(ja) この解説の日本語版はここ にあります。

Overview

DeltaTraceDB is a lightweight and high-performance in-memory NoSQL database that stores and searches class structures as-is.
Although it is NoSQL, it also supports full-text search across nested child objects.

Queries in DeltaTraceDB are also represented as classes.
By serializing and storing these query objects, you can not only restore the database to any past state,
but also keep operation metadata such as who / when / what / why / from.
This allows you to build rich and highly detailed operation logs suitable for security audits and usage analysis.

Features

• Store and search classes directly (your model classes define the DB structure)
• Lightweight in-memory DB for Dart / Flutter
• High-speed search performance even with ~100,000 records
• Queries are classes, making it easy to preserve operation logs
• Python version available
  → https://pypi.org/project/delta-trace-db/
• GUI editor for DB content is under development
  → https://github.com/MasahideMori-SimpleAppli/delta_trace_studio

Quick Start

import 'package:delta_trace_db/delta_trace_db.dart';
import 'package:file_state_manager/file_state_manager.dart';

class User extends CloneableFile {
  final int id;
  final String name;
  final int age;
  final DateTime createdAt;
  final DateTime updatedAt;
  final Map<String, dynamic> nestedObj;

  User({
    required this.id,
    required this.name,
    required this.age,
    required this.createdAt,
    required this.updatedAt,
    required this.nestedObj,
  });

  static User fromDict(Map<String, dynamic> src) => User(
    id: src['id'],
    name: src['name'],
    age: src['age'],
    createdAt: DateTime.parse(src['createdAt']),
    updatedAt: DateTime.parse(src['updatedAt']),
    nestedObj: src['nestedObj'],
  );

  @override
  Map<String, dynamic> toDict() => {
    'id': id,
    'name': name,
    'age': age,
    'createdAt': createdAt.toUtc().toIso8601String(),
    'updatedAt': updatedAt.toUtc().toIso8601String(),
    'nestedObj': {...nestedObj},
  };

  @override
  User clone() {
    return User.fromDict(toDict());
  }
}

void main() {
  final db = DeltaTraceDatabase();
  final now = DateTime.now();
  List<User> users = [
    User(
      id: -1,
      name: 'Taro',
      age: 30,
      createdAt: now,
      updatedAt: now,
      nestedObj: {"a": "a"},
    ),
    User(
      id: -1,
      name: 'Jiro',
      age: 25,
      createdAt: now,
      updatedAt: now,
      nestedObj: {"a": "b"},
    ),
  ];
  // If you want the return value to be reflected immediately on the front end,
  // set returnData = true to get data that properly reflects the serial key.
  final query = QueryBuilder.add(
    target: 'users',
    addData: users,
    serialKey: "id",
    returnData: true,
  ).build();
  // Specifying the "User class" is only necessary if you want to easily revert to the original class.
  final r = db.executeQuery<User>(query);
  // If you want to check the return value, you can easily do so by using toDict, which serializes it.
  print(r.toDict());
  // You can easily convert from the Result object back to the original class.
  // The value of r.result is deserialized using the function specified by convert.
  List<User> results = r.convert(User.fromDict);
}

DB structure

In DeltaTraceDB, each collection corresponds to a list of class instances.
Since the data structure directly mirrors your class definitions,
it becomes easy to keep consistency between the frontend and backend while
focusing solely on retrieving the class objects you need.

📦 Database (DeltaTraceDB)
├── 🗂️ CollectionA (key: "collection_a")
│   ├── 📄 Item (ClassA)
│   │   ├── id: int
│   │   ├── name: String
│   │   └── timestamp: String
│   └── ...
├── 🗂️ CollectionB (key: "collection_b")
│   ├── 📄 Item (ClassB)
│   │   ├── uid: String
│   │   └── data: Map<String, dynamic>
└── ...

Basic Operations

For detailed usage, including how to write queries, see the documentation:

📘 Online Documentation

Performance

DeltaTraceDB is fast due to its in-memory design. Although it has no dedicated optimization mechanisms at the moment, its performance is roughly equivalent to a simple for loop over the data. Around 100,000 records can typically be handled without issues.

You can run performance tests using:

test/speed_test.dart

Below is an example result from a Ryzen 3600 machine:

speed test for 100000 records                                                                                                                                                                                                                                                       
start add
end add: 178 ms
start getAll (with object convert)
end getAll: 638 ms
returnsLength:100000
start save (with json string convert)
end save: 354 ms
start load (with json string convert)
end load: 259 ms
start search (with object convert)
end search: 780 ms
returnsLength:100000
start search paging, half limit pre search (with object convert)
end search paging: 440 ms
returnsLength:50000
start search paging by obj (with object convert)
end search paging by obj: 543 ms
returnsLength:50000
start search paging by offset (with object convert)
end search paging by offset: 438 ms
returnsLength:50000
start searchOne, the last index object search (with object convert)
end searchOne: 19 ms
returnsLength:1
start update at half index and last index object
end update: 22 ms
start updateOne of half index object
end updateOne: 6 ms
start conformToTemplate
end conformToTemplate: 65 ms
start delete half object (with object convert)
end delete: 450 ms
returnsLength:50000
start deleteOne for last object (with object convert)
end deleteOne: 6 ms
returnsLength:1
start add with serialKey
end add with serialKey: 54 ms
addedCount:100000

Future plans

Although further optimization is possible, performance improvements have lower priority.
The focus will instead be on improving usability and developing surrounding tools.

Notes

This package is designed for single-threaded environments.
When using parallel processing without shared memory, additional mechanisms such as message passing are required.

Support

There is no official support, but bugs are likely to be fixed actively.
Please open an issue on GitHub if you find any problems.

About version control

The C part will be changed at the time of version upgrade.
However, versions less than 1.0.0 may change the file structure regardless of the following rules.

• Changes such as adding variables, structure change that cause problems when reading previous files.
  • C.X.X
• Adding methods, etc.
  • X.C.X
• Minor changes and bug fixes.
  • X.X.C

License

This software is released under the Apache-2.0 License, see LICENSE file.

Copyright 2025 Masahide Mori

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Trademarks

• “Dart” and “Flutter” are trademarks of Google LLC.
  This package is not developed or endorsed by Google LLC.

• “Python” is a trademark of the Python Software Foundation.
  This package is not affiliated with the Python Software Foundation.

• GitHub and the GitHub logo are trademarks of GitHub, Inc.
  This package is not affiliated with GitHub, Inc.

Libraries

delta_trace_db