coolson 0.0.1

Stream-based JSON to Dart object decoder/encoder with optional JSON Schema validation

coolson 🚀

coolson is a high-performance JSON processing library for Dart, specifically designed for environments where memory efficiency and low latency are critical—such as server-side microservices or mobile apps handling large datasets.

---

💡 Why coolson?

Dart's native jsonDecode() is simple, but it has a major drawback: it requires loading the entire JSON into memory as a String before it can even begin parsing. For a 100MB JSON, your app will consume at least 100MB of RAM just for the initial String.

coolson breaks this paradigm by processing data as a Byte Stream:

• ✅ Constant Memory: RAM usage stays low regardless of whether the JSON is 1KB or 1GB.
• ✅ Real-time Processing: Start creating Dart objects as soon as the first bytes arrive from the socket or disk.
• ✅ Integrated Validation: Validate against JSON Schema while decoding.

---

🔍 Decoding Strategies

1. SingleJsonDecoder<T>

What is it? Decodes a single JSON object from a byte stream.

• When to use it: Standard API responses or reading configuration files.
• Example:

final decoder = SingleJsonDecoder(User.fromJson);
// Decodes a single object from a stream (e.g., an HTTP request body)
final user = await decoder.convert(requestStream);
print(user.name);

2. ListJsonDecoder<T>

What is it? Processes a full JSON array and returns a List<T>.

• When to use it: When you need a collection of items (e.g., GET /users).
• Example:

final decoder = ListJsonDecoder(User.fromJson);
// Processes a JSON array like [{"id":1}, {"id":2}]
final List<User> users = await decoder.convert(requestStream);
print('Loaded ${users.length} users');

3. FirstJsonDecoder<T>

What is it? Extracts only the first element of a JSON array and closes the stream immediately.

• When to use it: "Search" operations where you only care about the first match, or existence checks.
• Advantage: Maximum performance. It stops reading from the disk/socket the moment the first object is found, saving bandwidth and time.
• Example:

final decoder = FirstJsonDecoder(User.fromJson);
// Stops reading as soon as the first User is parsed
final firstUser = await decoder.convert(requestStream);

4. NdJsonDecoder<T> (Newline Delimited JSON)

What is it? Processes JSON objects separated by newlines (\n).

• When to use it: Big Data, logs, massive data feeds. This is the standard for "True Streaming."
• Example:

final decoder = NdJsonDecoder(User.fromJson);
final lineStream = Stream.fromIterable(['{"name":"Alice"}\n', '{"name":"Bob"}\n']);

await for (final user in decoder.decodeNdJsonStream(lineStream)) {
  print('Processing ${user.name}...'); // Processes items one by one
}

5. JsonDtoConverter<T>

What is it? A high-performance Dart Converter that transforms raw JSON (Maps/Lists) into DTOs.

• When to use it: When you already have parsed JSON but need a fast, reusable way to map it to your data classes. It is significantly faster than manual mapping for large collections.
• Example:

final converter = JsonDtoConverter(fromJson: User.fromJson);

// Synchronous conversion
final List<User> users = converter.convert([
  {'name': 'Alice'}, 
  {'name': 'Bob'}
]).toList();

// Or use it to transform an existing stream of raw data
rawStream.transform(converter).listen((user) => print(user.name));

---

🏗️ Encoding Strategies

1. Map.asJsonStream

• What for: Converts a Map to a byte stream (Stream<List<int>>).
• Ideal use: Generating HTTP responses efficiently without creating huge intermediate strings.
• Example:

final data = {'status': 'ok', 'data': [...]};
// Stream bytes directly to the client
final stream = data.asJsonStream;

2. List.asJsonStream

• What for: Converts a List into a streaming JSON array.
• Ideal use: Serializing large data collections to an external client.
• Example:

final bigList = List.generate(1000, (i) => {'id': i});
final stream = bigList.asJsonStream; // Stream representation of "[{...},{...}]"

3. List.asNdJsonStream (Optimized)

• What for: Generates an NDJSON data flow (one object per line).
• Ideal use: Data pipelines, bulk exports, or microservice communication.
• Advantage: Internally uses Uint8List to avoid unnecessary memory copies, making it extremely fast.
• Example:

final records = [{'id': 1}, {'id': 2}];
final stream = records.asNdJsonStream;
// Output: {"id":1}\n{"id":2}\n

---

🛡️ JSON Schema Validation

Don't waste time processing invalid data. Integrate structural validation directly into the decoder:

final schema = JsonSchema.create({
  'type': 'object',
  'properties': {
    'id': {'type': 'integer'},
    'name': {'type': 'string', 'minLength': 3},
  },
  'required': ['id', 'name']
});

final decoder = SingleJsonDecoder(User.fromJson, schema: schema);

try {
  final user = await decoder.convert(requestStream);
} on SchemaValidationException catch (e) {
  print("Invalid data: ${e.errors}");
}

---

🚀 Decision Guide: When to use coolson?

Native vs. coolson: Performance & Memory

Use Case	Native jsonDecode	coolson Stream	Winner
Small Config (<100KB)	~0.5ms / low RAM	~0.8ms / low RAM	Native (Simplicity)
Large Array (10MB)	~120ms / 30MB RAM	~60ms / <1MB RAM	coolson (Efficiency)
Huge Data (100MB+)	Crash (Out of Memory)	~210ms / <1MB RAM	coolson (Scalability)

Check BENCHMARK.md for detailed metrics.

---

Strategy Selection

Decoding (Input)

Strategy	Input Type	Avg. Time (1k)	Memory Peak	Scalability	Use Case
SingleJsonDecoder	{} Map	~2.8ms	<1MB	High	Standard API objects
ListJsonDecoder	[] List	~2.8ms	<1MB	High	Full collections
FirstJsonDecoder	[] List	~0.1ms	<1MB	Extreme	Search/Validation
NdJsonDecoder	\n Lines	~2.1ms	<1MB	Extreme	Big Data / Logs
JsonDtoConverter	dynamic	~1.5ms	N/A	High	In-memory DTO mapping

Encoding (Output)

Strategy	Source	Avg. Time (1k)	Memory Peak	Scalability	Use Case
Map.asJsonStream	{} Map	~2.5ms	<1MB	High	API Responses
List.asJsonStream	[] List	~2.5ms	<1MB	High	Large Collections
List.asNdJsonStream	[] List	~1.8ms	<1MB	Extreme	High-speed Pipelines

---

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.