A Lightweight and Powerful Job Scheduling Framework.
Do you need a scheduled and long-lived server-side processing?
If so, this is the framework you are looking for!
- Implementation examples are available at: Official Examples
- Or more documentations are available at: Official Documents
- 1. Guide
1. Guide
1.1. Mission
The goal of this project is to provide a high-performance and intuitive job scheduling in the Dart language. And to enable people around the world to automate their tasks more easily.
And the development concept of this framework is "DRY", "KISS" and "YAGNI", which has been said in software engineering circles for a long time.
1.2. Features
- Easy and intuitive job scheduling.
- No complicated configuration files.
- Supports scheduling in Cron format.
- Supports powerful logging feature and it's customizable.
- Supports easily define parallel processes.
- Supports conditional branching of jobs and steps.
- Supports convenient callback functions at each event.
- Supports skipping and retrying according to user defined conditions.
- and etc...
1.3. Getting Started
1.3.1. Install Library
dart pub add batch
Note: Pub.dev automatically labels this library as usable for Flutter, but the intended use of this library is long-lived server-side processing.
1.3.2. Import
The following import will provide all the materials for developing job scheduling using Batch.dart
.
import 'package:batch/batch.dart';
1.3.3. Basic Concept
Batch.dart
represents the unit of scheduled processing as an Event
.
And Event
is composed of the following elements.
Description | |
---|---|
Job | This Job event is a unit of batch processing in the broad sense. And Job has multiple Step events. |
ScheduledJob | It represents a scheduled job. And ScheduledJob has multiple Step events. |
Step | This event expresses the sequential processing. Each Step has a Task that defines one specific process. |
ParallelStep | This event expresses the parallel processing. Each ParallelStep has ParallelTask s that define specific processes. |
1.3.4. Schedule Jobs
1.3.4.1. Sequential Process
Scheduling jobs using this framework is very easy.
First, create a class that defines a class extends the Task
class and define processes in execute
method. The execute
method can define any process and supports both synchronous and asynchronous processing.
Second, you need to define a scheduled job. It is also easy to define a scheduled job by implementing the build
method in a class that implements ScheduledJobBuilder
, as in the following example. And Batch.dart
supports standard Cron
specifications.
Finally, let's execute runWorkflow
method on main method with scheduled jobs as arguments!
When the runWorkflow
method is executed, the scheduled batch process is started.
Example
import 'package:batch/batch.dart';
void main() => runWorkflow(
jobs: [SayHelloWorldJob()],
);
class SayHelloWorldJob implements ScheduledJobBuilder {
@override
ScheduledJob build() => ScheduledJob(
name: 'Test Job',
schedule: CronParser('*/2 * * * *'), // Execute every 2 minutes
steps: [
Step(
name: 'Test Step',
task: SayHelloWorldTask(),
)
],
);
}
class SayHelloWorldTask extends Task<SayHelloWorldTask> {
@override
void execute(ExecutionContext context) {
log.info('Hello, World!');
}
}
You can see more examples at Official Examples.
1.3.4.2. Parallel Process
Batch.dart
supports powerful parallel processing and is easy to define.
When defining parallel processing, all you have to do is just inherit from ParallelTask
and describe the process you want to parallelize in the execute
method.
SharedParameters
and JobParameters
set in the main thread can be referenced through ExecutionContext
. However, note that under the current specification, changes to the ExecutionContext
value during parallel processing are not reflected in the main thread's ExecutionContext
.
Example
import 'dart:async';
import 'package:batch/batch.dart';
void main() => runWorkflow(
jobs: [DoHeavyProcessJob()],
);
class DoHeavyProcessJob implements ScheduledJobBuilder {
@override
ScheduledJob build() => ScheduledJob(
name: 'Job',
schedule: CronParser('*/2 * * * *'), // Execute every 2 minutes
steps: [
ParallelStep(
name: 'Parallel Step',
tasks: [
DoHeavyTask(),
DoHeavyTask(),
DoHeavyTask(),
DoHeavyTask(),
],
)
],
);
}
class DoHeavyTask extends ParallelTask<DoHeavyTask> {
@override
FutureOr<void> execute(ExecutionContext context) {
int i = 0;
while (i < 10000000000) {
i++;
}
}
}
1.3.5. Logging
The Batch.dart
provides the following well-known logging features as a standard.
And the default log level is trace.
- trace
- debug
- info
- warn
- error
- fatal
The logging feature provided by Batch.dart
has extensive customization options. For more information, you can refer to the Official Documents describing logging on Batch.dart
.
1.3.5.1. On Sequential Process
It's very easy to use logging functions on sequential process.
The logging methods provided by the Batch.dart
can be used from any class that imports batch.dart
. So no need to instantiate any Loggers by yourself!
All you need to specify about logging in Batch.dart
is the configuration of the log before run BatchApplication
, and the Logger is provided safely under the lifecycle of the Batch.dart
.
Example
import 'package:batch/batch.dart';
class TestLogTask extends Task<TestLogTask> {
@override
void execute() {
log.trace('Test trace');
log.debug('Test debug');
log.info('Test info');
log.warn('Test warning');
log.error('Test error');
log.fatal('Test fatal');
}
}
For example, if you run example, you can get the following log output.
yyyy-MM-dd 19:25:10.575109 [info ] (_BatchApplication.run:129:11 ) - 🚀🚀🚀🚀🚀🚀🚀 The batch process has started! 🚀🚀🚀🚀🚀🚀🚀
yyyy-MM-dd 19:25:10.579318 [info ] (_BatchApplication.run:130:11 ) - Logger instance has completed loading
yyyy-MM-dd 19:25:10.580177 [info ] (_BootDiagnostics.run:32:9 ) - Batch application diagnostics have been started
yyyy-MM-dd 19:25:10.583234 [info ] (_BootDiagnostics.run:46:9 ) - Batch application diagnostics have been completed
yyyy-MM-dd 19:25:10.583344 [info ] (_BootDiagnostics.run:47:9 ) - Batch applications can be started securely
yyyy-MM-dd 19:25:10.585729 [info ] (JobScheduler.run:37:9 ) - Started Job scheduling on startup
yyyy-MM-dd 19:25:10.585921 [info ] (JobScheduler.run:38:9 ) - Detected 3 Jobs on the root
yyyy-MM-dd 19:25:10.586023 [info ] (JobScheduler.run:41:11 ) - Scheduling Job [name=Job1]
yyyy-MM-dd 19:25:10.595706 [info ] (JobScheduler.run:41:11 ) - Scheduling Job [name=Job2]
yyyy-MM-dd 19:25:10.597471 [info ] (JobScheduler.run:41:11 ) - Scheduling Job [name=Job4]
yyyy-MM-dd 19:25:10.597692 [info ] (JobScheduler.run:56:9 ) - Job scheduling has been completed and the batch application is now running
Note: The setup of the logger is done when executing the
runWorkflow
. If you want to use the logging feature outside the life cycle of this library, be sure to do so after executing therunWorkflow
.
1.3.5.2. On Parallel Process
Parallel processing cannot directly use the convenient logging features described above. This is because parallel processing in the Dart language does not share any instances.
Instead, use the following methods in classes that extend ParallelTask
for parallel processing.
- sendMessageAsTrace
- sendMessageAsDebug
- sendMessageAsInfo
- sendMessageAsWarn
- sendMessageAsError
- sendMessageAsFatal
Example
class TestParallelTask extends ParallelTask<TestParallelTask> {
@override
FutureOr<void> execute() {
super.sendMessageAsTrace('Trace');
super.sendMessageAsDebug('Debug');
super.sendMessageAsInfo('Info');
super.sendMessageAsWarn('Warn');
super.sendMessageAsError('Error');
super.sendMessageAsFatal('Fatal');
}
}
It should be noted that log output does not occur at the moment the above sendMessageAsX
method is used.
This is only a function that simulates log output in parallel processing, and all messages are output at once when all parallel processing included in Parallel
is completed.
And you can get the following log output from parallel processes.
yyyy-MM-dd 10:05:06.662561 [trace] (solatedLogMessage.output:36:13) - Received from the isolated thread [message=Trace]
yyyy-MM-dd 10:05:06.662666 [debug] (solatedLogMessage.output:39:13) - Received from the isolated thread [message=Debug]
yyyy-MM-dd 10:05:06.662760 [info ] (solatedLogMessage.output:42:13) - Received from the isolated thread [message=Info]
yyyy-MM-dd 10:05:06.662856 [warn ] (solatedLogMessage.output:45:13) - Received from the isolated thread [message=Warn]
yyyy-MM-dd 10:05:06.662947 [error] (solatedLogMessage.output:48:13) - Received from the isolated thread [message=Error]
yyyy-MM-dd 10:05:06.663039 [fatal] (solatedLogMessage.output:51:13) - Received from the isolated thread [message=Fatal]
1.3.6. Branch
Batch.dart
supports conditional branching for each scheduled event (it's just called "Branch" in Batch.dart
).
Branch
is designed to be derived from each event, such as Job
and Step
. There is no limit to the number of branches that can be set up, and a recursive nesting structure is also possible.
Creating a branch for each event is very easy.
To create branch
Step(
name: 'Step',
task: SwitchBranchStatusTask(),
// Each branch can be multiple and nested
branchesOnSucceeded: [Step(name: 'Step on succeeded', task: doSomethingTask)],
branchesOnFailed: [Step(name: 'Step on failed', task: doSomethingTask)],
branchesOnCompleted: [Step(name: 'Step on completed', task: doSomethingTask)],
);
And the conditional branching of Batch.dart
is controlled by switching the BranchStatus
of each Execution
s that can be referenced from the ExecutionContext
.
The default branch status is "completed".
To manage branch
class SwitchBranchStatusTask extends Task<SwitchBranchStatusTask> {
@override
void execute(ExecutionContext context) {
// You can easily manage branch status through methods as below.
context.jobExecution!.switchBranchToSucceeded();
context.stepExecution!.switchBranchToFailed();
}
}
1.4. More Examples
- Create a minimal and basic batch application
- Create a batch application consisting of multiple job nets
- Create a parallel processing tasks
- Pass command line arguments to batch application
- Create a branch and switch
- Use callback functions
- Define skippable exceptions
- Define retry processing
You can check more at Official Examples.
1.5. Contribution
If you would like to contribute to Batch.dart
, please create an issue or create a Pull Request.
Owner will respond to issues and review pull requests as quickly as possible.
1.6. Support
The simplest way to show us your support is by giving the project a star at here.
And I'm always looking for sponsors to support this project. I do need support to continue ongoing open source development.
Sponsors can be individuals or corporations, and the amount is optional.
1.7. License
All resources of Batch.dart
is provided under the BSD-3
license.
Note: License notices in the source are strictly validated based on
.github/header-checker-lint.yml
. Please check header-checker-lint.yml for the permitted standards.
1.8. More Information
Batch.dart
was designed and implemented by Kato Shinya.