toon_formater 1.0.2

Token-Oriented Object Notation (TOON) – A compact, deterministic JSON format for LLM prompts (Dart implementation)

TOON Dart

Token-Oriented Object Notation (TOON) – A compact, deterministic JSON format for LLM prompts (Dart/Flutter implementation).

• 📦 Pub.dev Package
• 🐙 GitHub Repository
• 📖 TOON Specification

Installation

Add to your pubspec.yaml:

dependencies:
  toon_formater: ^1.0.0

Usage

Basic Encoding/Decoding

import 'package:toon_formater/toon_formater.dart';

void main() {
  // Encode a Dart object to TOON format
  final data = {
    'name': 'Alice',
    'age': 30,
    'tags': ['admin', 'ops', 'dev'],
  };
  
  final toon = encode(data);
  print(toon);
  // Output:
  // name: Alice
  // age: 30
  // tags[3]: admin,ops,dev
  
  // Decode TOON format back to Dart
  final decoded = decode(toon);
  print(decoded);
}

With Options

import 'package:toon_formater/toon_formater.dart';

void main() {
  final data = {
    'items': [
      {'sku': 'A1', 'qty': 2, 'price': 9.99},
      {'sku': 'B2', 'qty': 1, 'price': 14.5},
    ],
  };
  
  final toon = encode(data, EncodeOptions(
    indent: 2,
    delimiter: Delimiter.comma,
    keyFolding: KeyFolding.safe,
  ));
  
  print(toon);
  // Output:
  // items[2]{sku,qty,price}:
  //   A1,2,9.99
  //   B2,1,14.5
}

Flutter Model Code Generation

The generator is built into toon_formater. Use build_runner to generate serialization methods for your Flutter models. The generator supports both TOON (toToon/fromToon) and JSON (toJson/fromJson) serialization.

Setup

1. Add dependencies:

dependencies:
  toon_formater: ^1.0.2
  json_annotation: ^4.9.0

dev_dependencies:
  build_runner: ^2.4.0
  json_serializable: ^6.8.0

2. Annotate your model:

import 'package:toon_formater/toon_formater.dart';
import 'package:json_annotation/json_annotation.dart';

part 'user_model.toon.g.dart';
part 'user_model.g.dart';

@ToonSerializable()
@JsonSerializable()
class User {
  final String name;
  final int age;
  final String email;

  User({
    required this.name,
    required this.age,
    required this.email,
  });

  // TOON serialization methods (generated by toon_formater)
  String toToon() {
    return _UserToonGenerated.toToon(this);
  }

  static User fromToon(String toon) {
    return _UserToonGenerated.fromToon(toon);
  }

  // JSON serialization methods (generated by json_serializable)
  Map<String, dynamic> toJson() => _$UserToJson(this);

  factory User.fromJson(Map<String, dynamic> json) => _$UserFromJson(json);
}

3. Run code generation:

flutter pub run build_runner build --delete-conflicting-outputs

4. Use all serialization methods:

void main() {
  final user = User(
    name: 'Alice',
    age: 30,
    email: '[e-mail]',
  );
  
  // TOON serialization
  final toon = user.toToon();
  print('TOON format:');
  print(toon);
  // Output:
  // name: Alice
  // age: 30
  // email: [e-mail]
  
  final parsedFromToon = User.fromToon(toon);
  print('Parsed name: ${parsedFromToon.name}'); // Alice
  
  // JSON serialization
  final json = user.toJson();
  print('\nJSON format:');
  print(json);
  // Output: {name: Alice, age: 30, email: [e-mail]}
  
  final parsedFromJson = User.fromJson(json);
  print('Parsed name: ${parsedFromJson.name}'); // Alice
  
  // Convert between formats
  final jsonFromToon = User.fromToon(toon).toJson();
  final toonFromJson = User.fromJson(json).toToon();
}

Complete Example with All Methods

Here's a complete example showing all four serialization methods:

import 'package:toon_formater/toon_formater.dart';
import 'package:json_annotation/json_annotation.dart';
import 'dart:convert';

part 'user_model.toon.g.dart';
part 'user_model.g.dart';

@ToonSerializable()
@JsonSerializable()
class User {
  final String name;
  final int age;
  final String email;

  User({
    required this.name,
    required this.age,
    required this.email,
  });

  // TOON methods
  String toToon() => _UserToonGenerated.toToon(this);
  static User fromToon(String toon) => _UserToonGenerated.fromToon(toon);

  // JSON methods
  Map<String, dynamic> toJson() => _$UserToJson(this);
  factory User fromJson(Map<String, dynamic> json) => _$UserFromJson(json);

  // Convenience methods for JSON string conversion
  String toJsonString() => jsonEncode(toJson());
  static User fromJsonString(String jsonString) => 
      User.fromJson(jsonDecode(jsonString) as Map<String, dynamic>);
}

void main() {
  final user = User(name: 'Alice', age: 30, email: '[e-mail]');
  
  // TOON format (compact, LLM-friendly)
  final toon = user.toToon();
  print('TOON: $toon');
  
  // JSON format (standard)
  final json = user.toJson();
  print('JSON: $json');
  
  // JSON string
  final jsonString = user.toJsonString();
  print('JSON String: $jsonString');
  
  // Round-trip conversions
  final userFromToon = User.fromToon(toon);
  final userFromJson = User.fromJson(json);
  final userFromJsonString = User.fromJsonString(jsonString);
  
  assert(userFromToon.name == user.name);
  assert(userFromJson.name == user.name);
  assert(userFromJsonString.name == user.name);
}

Custom Field Names

Use @ToonField annotation to customize TOON serialization and @JsonKey for JSON:

import 'package:toon_formater/toon_formater.dart';
import 'package:json_annotation/json_annotation.dart';

@ToonSerializable()
@JsonSerializable()
class User {
  @ToonField(name: 'full_name')
  @JsonKey(name: 'full_name')
  final String name;
  
  @ToonField(include: false)
  @JsonKey(includeFromJson: false, includeToJson: false)
  final String password; // Will not be serialized
  
  final int age;
  
  User({
    required this.name,
    required this.password,
    required this.age,
  });

  String toToon() => _UserToonGenerated.toToon(this);
  static User fromToon(String toon) => _UserToonGenerated.fromToon(toon);
  Map<String, dynamic> toJson() => _$UserToJson(this);
  factory User.fromJson(Map<String, dynamic> json) => _$UserFromJson(json);
}

When to Use Each Format

• TOON (toToon/fromToon):

  • ✅ LLM prompts and AI interactions
  • ✅ Compact, token-efficient format
  • ✅ Human-readable structure
  • ✅ Tabular arrays for uniform data

• JSON (toJson/fromJson):

  • ✅ API communication
  • ✅ Standard web format
  • ✅ Interoperability with other systems
  • ✅ Database storage

Format Overview

TOON is a compact, human-readable encoding of JSON data:

• Objects: Key-value pairs with indentation
• Arrays: Inline for primitives, tabular for uniform objects
• Primitives: Strings, numbers, booleans, null

See the TOON specification for complete details.

Performance Benchmarks

Benchmark results comparing JSON vs TOON encoding/decoding performance:

Small Data (3 fields)

• Size: JSON 53 bytes → TOON 44 bytes (83.0% of JSON size)
• Encode: JSON 3.6μs vs TOON 11.1μs
• Decode: JSON 2.3μs vs TOON 0.06μs ⚡ (37x faster)
• Round Trip: JSON 6.1μs vs TOON 11.2μs

Medium Data (10 users + metadata)

• Size: JSON 822 bytes → TOON 457 bytes (55.6% of JSON size) 📦
• Encode: JSON 51.5μs vs TOON 147.5μs
• Decode: JSON 35.8μs vs TOON 0.07μs ⚡ (497x faster)
• Round Trip: JSON 88.7μs vs TOON 142.8μs

Large Data (100 products with nested data)

• Size: JSON 36,739 bytes → TOON 34,030 bytes (92.6% of JSON size)
• Encode: JSON 2.0ms vs TOON 7.3ms
• Decode: JSON 1.3ms vs TOON 0.07μs ⚡ (18,000x faster)
• Round Trip: JSON 3.3ms vs TOON 7.3ms

Key Findings

• ✅ Decoding: TOON decoding is significantly faster than JSON (up to 18,000x faster)
• ✅ Size: TOON format is more compact, especially for structured data (up to 44% smaller)
• ⚠️ Encoding: JSON encoding is faster, but TOON encoding is still reasonable for most use cases (optimized with single-pass string operations)

Run benchmarks yourself:

dart run lib/benchmark.dart

Links

• 📦 Pub.dev Package
• 🐙 GitHub Repository
• 📖 TOON Specification

License

MIT