рускій воєнний корабль, іди нах#й
You can support the Armed Forces of Ukraine, or contribute to the Humanitarian Assistance to Ukrainians affected by russia’s aggression. Thank you.
LoreDart NN
Simple library for creating and training Deep Neural Networks, written in pure Dart.
Conceptually, the library has educational and entertainment purposes. Hope you will find it fun to use DNN with loredart_nn.
Getting started
Just import library into the project.
import 'package:loredart_nn/loredart_nn.dart';
NeuralNetwork usage
In the loredart_nn
data inside the NeuralNetwork is as a (mini)batch Matrix, where each column representes one data record and each row representes one feature. So NeuralNetwork uses column-based data representation.
But user's inputs and model predictions are List<List<double>>
, where each row is a data record.
Here is a small example of creating MNIST classification Deep Neural Network:
Load the data
// List of 784 pixels for each digit
List<List<double>> xTrain = data[0].sublist(0,30000);
// One-Hot encoded digits' label
List<List<double>> yTrain = data[1].sublist(0,30000);
List<List<double>> xTest = data[0].sublist(30000);
List<List<double>> yTest = data[1].sublist(30000);
I used the 'flatten' version of MNIST, where the digit's image flattened into the record of 784 pixels.
Define the model
var model = NeuralNetwork(
784, // length of one input record = 784 pixels
[
Dense(128, activation: Activation.softplus()), // fully connected layer
LayerNormalization(), // normalization for each data record in the batch
Dense(10, activation: Activation.softmax()) // output layer with softmax for probabilities
],
loss: Loss.crossEntropy(), // crossEntropy loss for One-Hot encoded target values
optimizer: SGD(learningRate: 0.01, momentum: 0.9), // optionally customize SGD optimizer with momentum
useAccuracyMetric: true // for classification task you can use 'accuracy' metric
);
Train the model
var history = model.fit(xTrain, yTrain, epochs: 4, batchSize: 256, verbose: true);
With verbose == true
, you will see how the model is updating after each batch of data, and summary information for each epoch, like that:
epoch 1/4 |118/118| -> mean time per batch: 388.01ms, mean loss [cross_entropy]: 1.478636, mean accuracy: 68.05%
epoch 2/4 |118/118| -> mean time per batch: 400.87ms, mean loss [cross_entropy]: 0.868227, mean accuracy: 83.50%
epoch 3/4 |118/118| -> mean time per batch: 390.92ms, mean loss [cross_entropy]: 0.691678, mean accuracy: 85.72%
epoch 4/4 |118/118| -> mean time per batch: 386.92ms, mean loss [cross_entropy]: 0.606761, mean accuracy: 86.84%
Here you can control batch size and number of epochs. Model's fit
method returns history
- Map with buffered information about loss for each epoch.
print(history);
// {cross_entropy: [1.4786360357701471, 0.868226552200313, 0.6916779963409534, 0.6067611970768912],
// accuracy: [68.04819915254238, 83.49995586158192, 85.72342867231639, 86.83902718926552]}
Test the model
var metrics = model.evaluate(xTest, yTest, verbose: true)
print(metrics);
//{mean cross_entropy: 0.5742549607727949, mean accuracy: 0.8719166666666667}
Again, with verbose == true
, you will see a little bit more information:
// evaluating batch 100/100 -> mean time per batch: 68.52ms, mean loss [cross_entropy]: 0.574255, mean accuracy: 87.19%
Similarly, evaluate
method returns Map object with stored information about mean loss.
Use the model
Prediction is performed for many data rows at once, and output is a List of predictions for each input.
// data is some List of inputs
List<List<double>> data = ...;
List<List<double>> prediction = model.predict(data); // model.predict returns prediction for each row in the data
print(prediction[0]);
// prints something like [0.00, 0.00, 0.00, 0.99, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00]
Save and load weights of model
You can save weights and biases of trainable (aka Dense) layers into some directory.
// save parameters into the `mnist_classifier/model_weights.bin` file
model.saveWeights('mnist_classifier')
saveWeights
method saves only weights and biases of Dense layers
Then you can load weights and biases into the model, but be sure you use appropriate architecture, or the results of the loaded model won't meet expectations.
model.loadWeights('mnist_classifier')
If you want to load weights from Flutter assets, use the loadWeightsFromBytes
method.
var model = NeuralNetwork(...)
rootBundle.load('assets/model/model_weights.bin').then((value) {
model.loadWeightsFromBytes(value.buffer);
});
Flutter example with NeuralNetwork model
Simple example of using MNIST classifier within Flutter app.
Working Flutter project will be posted on GitHub.
Additional information
Supported layers:
Dense
- a regular fully connected layer with weights, biases and activation function.LayerNormalization
- the data record normalization layer; supports two normalizations:min-max
andz-score
.Input
- a special layer that is generated by the NeuralNetwork.
Supported activations:
Activation.sigmoid()
- Sigmoid activation functionActivation.swish()
- Swish activation functionActivation.softplus()
- Softplus activation functionActivation.softmax()
- Softmax activation functionActivation.relu()
- ReLU activation functionActivation.leakyReLU()
- Leaky ReLU activation functionActivation.elu()
- ELU activation function
Supported losses:
Loss.mae()
- Mean Absolute Error lossLoss.mse()
- Mean Square Error lossLoss.crossEntropy()
- Cross Entropy lossLoss.sparseCrossEntropy()
- Sparse Cross Entropy loss
Supported optimizers:
For now, loredart_nn
has only one optimizer:
SGD with momentum
.