рускій воєнний корабль, іди нах#й

You can support the Armed Forces of Ukraine, or contribute to the Humanitarian Assistance to Ukrainians affected by russia’s aggression. Thank you.

LoreDart NN

Simple library for creating and training Deep Neural Networks, written in pure Dart.

Conceptually, the library has educational and entertainment purposes. Hope you will find it fun to use DNN with loredart_nn.

Getting started

Just import library into the project.

import 'package:loredart_nn/loredart_nn.dart';

NeuralNetwork usage

In the loredart_nn data inside the NeuralNetwork is as a (mini)batch Matrix, where each column representes one data record and each row representes one feature. So NeuralNetwork uses column-based data representation.

But user's inputs and model predictions are List<List<double>>, where each row is a data record.

Here is a small example of creating MNIST classification Deep Neural Network:

Load the data

  // List of 784 pixels for each digit
  List<List<double>> xTrain = data[0].sublist(0,30000);
  // One-Hot encoded digits' label 
  List<List<double>> yTrain = data[1].sublist(0,30000);
  List<List<double>> xTest = data[0].sublist(30000);
  List<List<double>> yTest = data[1].sublist(30000);

I used the 'flatten' version of MNIST, where the digit's image flattened into the record of 784 pixels.

Define the model

var model = NeuralNetwork(
  784, // length of one input record = 784 pixels
  [
    Dense(128, activation: Activation.softplus()), // fully connected layer
    LayerNormalization(), // normalization for each data record in the batch
    Dense(10, activation: Activation.softmax()) // output layer with softmax for probabilities
  ],
  loss: Loss.crossEntropy(), // crossEntropy loss for One-Hot encoded target values
  optimizer: SGD(learningRate: 0.01, momentum: 0.9), // optionally customize SGD optimizer with momentum
  useAccuracyMetric: true // for classification task you can use 'accuracy' metric
);

Train the model

var history = model.fit(xTrain, yTrain, epochs: 4, batchSize: 256, verbose: true);

With verbose == true, you will see how the model is updating after each batch of data, and summary information for each epoch, like that:

epoch 1/4 |118/118| -> mean time per batch: 388.01ms, mean loss [cross_entropy]: 1.478636, mean accuracy: 68.05%
epoch 2/4 |118/118| -> mean time per batch: 400.87ms, mean loss [cross_entropy]: 0.868227, mean accuracy: 83.50%
epoch 3/4 |118/118| -> mean time per batch: 390.92ms, mean loss [cross_entropy]: 0.691678, mean accuracy: 85.72%
epoch 4/4 |118/118| -> mean time per batch: 386.92ms, mean loss [cross_entropy]: 0.606761, mean accuracy: 86.84%

Here you can control batch size and number of epochs. Model's fit method returns history - Map with buffered information about loss for each epoch.

print(history);
// {cross_entropy: [1.4786360357701471, 0.868226552200313, 0.6916779963409534, 0.6067611970768912],
//  accuracy: [68.04819915254238, 83.49995586158192, 85.72342867231639, 86.83902718926552]}

Test the model

  var metrics = model.evaluate(xTest, yTest, verbose: true)
  print(metrics);
  //{mean cross_entropy: 0.5742549607727949, mean accuracy: 0.8719166666666667}

Again, with verbose == true, you will see a little bit more information:

// evaluating batch 100/100 -> mean time per batch: 68.52ms, mean loss [cross_entropy]: 0.574255, mean accuracy: 87.19%

Similarly, evaluate method returns Map object with stored information about mean loss.

Use the model

Prediction is performed for many data rows at once, and output is a List of predictions for each input.

  // data is some List of inputs
  List<List<double>> data = ...;

  List<List<double>> prediction = model.predict(data); // model.predict returns prediction for each row in the data
  print(prediction[0]); 
  // prints something like [0.00, 0.00, 0.00, 0.99, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00]

Save and load weights of model

You can save weights and biases of trainable (aka Dense) layers into some directory.

  // save parameters into the `mnist_classifier/model_weights.bin` file
  model.saveWeights('mnist_classifier')

saveWeights method saves only weights and biases of Dense layers

Then you can load weights and biases into the model, but be sure you use appropriate architecture, or the results of the loaded model won't meet expectations.

 model.loadWeights('mnist_classifier')

If you want to load weights from Flutter assets, use the loadWeightsFromBytes method.

  var model = NeuralNetwork(...)
   rootBundle.load('assets/model/model_weights.bin').then((value) {
     model.loadWeightsFromBytes(value.buffer);
   });

Flutter example with NeuralNetwork model

Simple example of using MNIST classifier within Flutter app.

Working Flutter project will be posted on GitHub.

'1' exmaple '2' example '3' example '5' example

Additional information

Supported layers:

  • Dense - a regular fully connected layer with weights, biases and activation function.
  • LayerNormalization - the data record normalization layer; supports two normalizations: min-max and z-score.
  • Input - a special layer that is generated by the NeuralNetwork.

Supported activations:

  • Activation.sigmoid() - Sigmoid activation function
  • Activation.swish() - Swish activation function
  • Activation.softplus() - Softplus activation function
  • Activation.softmax() - Softmax activation function
  • Activation.relu() - ReLU activation function
  • Activation.leakyReLU() - Leaky ReLU activation function
  • Activation.elu() - ELU activation function

Supported losses:

  • Loss.mae() - Mean Absolute Error loss
  • Loss.mse() - Mean Square Error loss
  • Loss.crossEntropy() - Cross Entropy loss
  • Loss.sparseCrossEntropy() - Sparse Cross Entropy loss

Supported optimizers:

For now, loredart_nn has only one optimizer:

  • SGD with momentum.

Libraries

loredart_nn