core/data/dataset library
PyTorch-style Dataset + DataLoader abstractions.
A Dataset is any indexable collection of examples of type T.
A DataLoader wraps a dataset and yields batches (List<T>) in
either sequential or shuffled order, optionally dropping the
last (short) batch.
Concrete datasets in this package:
- ImageFolderDataset — folder-per-class image classification / triplet sampling.
- TextTokenDataset — sliding-window language-modeling examples from a text file, using CharTokenizer or BpeTokenizer.
- CsvDataset — tabular
(features, label)regression / classification.
All three build tensors on a caller-specified Device.
Classes
-
DataLoader<
T> - Yields fixed-size batches of items from a Dataset.
-
Dataset<
T> -
Read-only, indexable collection of examples of type
T. -
ListDataset<
T> -
In-memory dataset built from a plain
List<T>.