batch

A lightweight and powerful Job Scheduling Framework.

pub package Dart SDK Version Test Analyzer codecov CodeFactor Issues Pull Requests Stars Code size Last Commits License Contributor Covenant FOSSA Status

Implementation examples are available at: Official Examples
Or more documentations are available at: Official Documents

1. Guide

1.1. Mission

The goal of this project is to provide a high-performance and intuitive job scheduling framework in the Dart language ecosystem that anyone can use in the world.

And the development concept of this framework is "DRY", "KISS" and "YAGNI", which has been said in software engineering circles for a long time.

1.2. Features

  • Easy and intuitive job scheduling.
  • No complicated configuration files.
  • Supports scheduling in Cron format as standard and customizable.
  • Supports powerful logging feature as standard and customizable.
  • Supports easily define parallel processes.
  • Supports conditional branching of jobs.
  • Supports extensive callback functions at each event.
  • Supports skipping and retrying according to user defined conditions.

1.3. Basic Usage

1.3.1. Install Library

 dart pub add batch

Note: In pub.dev, the automatic determination at the time of release of this library labels it as usable in Flutter, but it is not suitable by any stretch of the imagination.

1.3.2. Import

The following import will provide all the materials for developing job scheduling using Batch.dart.

import 'package:batch/batch.dart';

1.3.3. Basic Concept

Batch.dart represents the unit of scheduled processing as an Event. And Event is composed of the following elements.

  1. Job - The largest unit.
  2. Step - The intermediate unit.
  3. Task - The smallest unit.
  4. Parallel - It's kind of Task but represents parallel processes.

You can see more information about Event at here.

1.3.4. Configure Job Schedules

1.3.4.1. Sequential Process

When defining a simple sequential process, all that is required is to define a class that extends Task and implements the execute method.

Example

import 'package:batch/batch.dart';

void main() => BatchApplication()
      ..addJob(
        // Scheduled to start every minute in Cron format
        Job(name: 'Job', schedule: CronParser(value: '*/1 * * * *'))
          // Step phase
          ..nextStep(Step(name: 'Step')
            // Task phase
            ..registerTask(DoSomethingTask()
          ),
        ),
      )
      ..run();


class DoSomethingTask extends Task<DoSomethingTask> {
  @override
  void execute(ExecutionContext context) {
    // Write your code here.
  }
}

The above example is a very simple, and so you should refer to other documents also for more detailed specifications and implementation instructions.

You can see more details at Official Documents or Official Examples.

1.3.4.2. Parallel Process

Batch.dart supports powerful parallel processing and is easy to define.

When defining parallel processing, all you have to do is just inherit from ParallelTask and describe the process you want to parallelize in the execute method.

SharedParameters and JobParameters set in the main thread can be referenced through ExecutionContext. However, note that under the current specification, changes to the ExecutionContext value during parallel processing are not reflected in the main thread's ExecutionContext.

Example

import 'dart:async';

import 'package:batch/batch.dart';

void main() => BatchApplication()
      ..addJob(
        // Scheduled to start every minute in Cron format
        Job(name: 'Job', schedule: CronParser(value: '*/1 * * * *'))
          // Step phase
          ..nextStep(Step(name: 'Step')
            // Parallel task phase
            ..registerParallel(
              Parallel(
                name: 'Parallel Tasks',
                tasks: [
                  DoHeavyTask(),
                  DoHeavyTask(),
                  DoHeavyTask(),
                  DoHeavyTask(),
                ],
              ),
            )
          ),
        ),
      )
      ..run();


class DoHeavyTask extends ParallelTask<DoHeavyTask> {
  @override
  FutureOr<void> execute(ExecutionContext context) {
    int i = 0;
    while (i < 10000000000) {
      i++;
    }
  }
}

1.3.5. Logging

The Batch.dart provides the following well-known logging features as a standard. And the default log level is trace.

  • trace
  • debug
  • info
  • warn
  • error
  • fatal

The logging feature provided by Batch.dart has extensive customization options. For more information, you can refer to the Official Documents describing logging on Batch.dart.

1.3.5.1. On Sequential Process

It's very easy to use logging functions on sequential process.

The logging methods provided by the Batch.dart can be used from any class that imports batch.dart. So no need to instantiate any Loggers by yourself!

All you need to specify about logging in Batch.dart is the configuration of the log before run BatchApplication, and the Logger is provided safely under the lifecycle of the Batch.dart.

Example

import 'package:batch/batch.dart';

class TestLogTask extends Task<TestLogTask> {
  @override
  void execute() {
    log.trace('Test trace');
    log.debug('Test debug');
    log.info('Test info');
    log.warn('Test warning');
    log.error('Test error');
    log.fatal('Test fatal');
  }
}

For example, if you run example, you can get the following log output.

yyyy-MM-dd 19:25:10.575109 [info ] (_BatchApplication.run:129:11  ) - 🚀🚀🚀🚀🚀🚀🚀 The batch process has started! 🚀🚀🚀🚀🚀🚀🚀
yyyy-MM-dd 19:25:10.579318 [info ] (_BatchApplication.run:130:11  ) - Logger instance has completed loading
yyyy-MM-dd 19:25:10.580177 [info ] (_BootDiagnostics.run:32:9     ) - Batch application diagnostics have been started
yyyy-MM-dd 19:25:10.583234 [info ] (_BootDiagnostics.run:46:9     ) - Batch application diagnostics have been completed
yyyy-MM-dd 19:25:10.583344 [info ] (_BootDiagnostics.run:47:9     ) - Batch applications can be started securely
yyyy-MM-dd 19:25:10.585729 [info ] (JobScheduler.run:37:9         ) - Started Job scheduling on startup
yyyy-MM-dd 19:25:10.585921 [info ] (JobScheduler.run:38:9         ) - Detected 3 Jobs on the root
yyyy-MM-dd 19:25:10.586023 [info ] (JobScheduler.run:41:11        ) - Scheduling Job [name=Job1]
yyyy-MM-dd 19:25:10.595706 [info ] (JobScheduler.run:41:11        ) - Scheduling Job [name=Job2]
yyyy-MM-dd 19:25:10.597471 [info ] (JobScheduler.run:41:11        ) - Scheduling Job [name=Job4]
yyyy-MM-dd 19:25:10.597692 [info ] (JobScheduler.run:56:9         ) - Job scheduling has been completed and the batch application is now running

Note: The setup of the logger is done when executing the method run in BatchApplication. If you want to use the logging feature outside the life cycle of the batch library, be sure to do so after executing the run method of the BatchApplication.

1.3.5.2. On Parallel Process

Parallel processing cannot directly use the convenient logging features described above. This is because parallel processing in the Dart language does not share any instances.

Instead, use the following methods in classes that extend ParallelTask for parallel processing.

  • sendMessageAsTrace
  • sendMessageAsDebug
  • sendMessageAsInfo
  • sendMessageAsWarn
  • sendMessageAsError
  • sendMessageAsFatal

Example

class TestParallelTask extends ParallelTask<TestParallelTask> {
  @override
  FutureOr<void> invoke() {
    super.sendMessageAsTrace('Trace');
    super.sendMessageAsDebug('Debug');
    super.sendMessageAsInfo('Info');
    super.sendMessageAsWarn('Warn');
    super.sendMessageAsError('Error');
    super.sendMessageAsFatal('Fatal');
  }
}

It should be noted that log output does not occur at the moment the above sendMessageAsX method is used.

This is only a function that simulates log output in parallel processing, and all messages are output at once when all parallel processing included in Parallel is completed.

And you can get the following log output from parallel processes.

yyyy-MM-dd 10:05:06.662561 [trace] (solatedLogMessage.output:36:13) - Received from the isolated thread [message=Trace]
yyyy-MM-dd 10:05:06.662666 [debug] (solatedLogMessage.output:39:13) - Received from the isolated thread [message=Debug]
yyyy-MM-dd 10:05:06.662760 [info ] (solatedLogMessage.output:42:13) - Received from the isolated thread [message=Info]
yyyy-MM-dd 10:05:06.662856 [warn ] (solatedLogMessage.output:45:13) - Received from the isolated thread [message=Warn]
yyyy-MM-dd 10:05:06.662947 [error] (solatedLogMessage.output:48:13) - Received from the isolated thread [message=Error]
yyyy-MM-dd 10:05:06.663039 [fatal] (solatedLogMessage.output:51:13) - Received from the isolated thread [message=Fatal]

1.3.6. Branch

Batch.dart supports conditional branching for each scheduled event (it's just called "Branch" in Batch.dart).

Branch is designed to be derived from each event, such as Job and Step. There is no limit to the number of branches that can be set up, and a recursive nesting structure is also possible.

Creating a branch for each event is very easy.

To create branch

Step(name: 'Step')
  // Assume that this task will change the branch status.
  ..registerTask(ChangeBranchStatusTask())

  // Pass an event object to "to" argument that you want to execute when you enter this branch.
  ..createBranchOnSucceeded(to: Step(name: 'Step on succeeded')..registerTask(somethingTask))
  ..createBranchOnFailed(to: Step(name: 'Step on failed')..registerTask(somethingTask))

  // Branches that are "createBranchOnCompleted" are always executed regardless of branch status.
  ..createBranchOnCompleted(to: Step(name: 'Step on completed'))..registerTask(somethingTask);

And the conditional branching of Batch.dart is controlled by changing the BranchStatus of each Executions that can be referenced from the ExecutionContext. The default branch status is "completed".

To manage branch

class ChangeBranchStatusTask extends Task<ChangeBranchStatusTask> {
  @override
  void execute(ExecutionContext context) {
    // You can easily manage branch status through methods as below.
    context.jobExecution!.switchBranchToSucceeded();
    context.stepExecution!.switchBranchToFailed();
  }
}

Note: Branch creation is not supported for Task and Parallel.

1.4. More Examples

You can check more at Official Examples.

1.5. Contribution

If you would like to contribute to Batch.dart, please create an issue or create a Pull Request.

Owner will respond to issues and review pull requests as quickly as possible.

1.6. Support

The simplest way to show us your support is by giving the project a star at here.

And I'm always looking for sponsors to support this project. I'm not asking for royalties for use in providing this framework, but I do need support to continue ongoing open source development.

Sponsors can be individuals or corporations, and the amount is optional.

👇 Click on the button below to see more details! 👇

myconsciousness

1.7. License

All resources of Batch.dart is provided under the BSD-3 license.

FOSSA Status

Note: License notices in the source are strictly validated based on .github/header-checker-lint.yml. Please check header-checker-lint.yml for the permitted standards.

1.8. More Information

Batch.dart was designed and implemented by Kato Shinya.

Libraries

batch