Skip to main content

topspeed 0.1.0 copy "topspeed: ^0.1.0" to clipboard
topspeed: ^0.1.0 copied to clipboard

Pure Dart reader and writer for Clarion TopSpeed (.tps) database files: tables, keys, MEMO/BLOB, code pages and encrypted files. No native code.

topspeed #

Read and write Clarion TopSpeed (.tps) database files in pure Dart. It needs no native code, no ODBC driver and no Clarion runtime, and it runs on the Dart VM, in Flutter (mobile and desktop) and on the web.

import 'dart:io';

import 'package:topspeed/topspeed.dart';

void main() {
  final file = TpsFile.parse(File('CUSTOMER.TPS').readAsBytesSync());
  final customers = file.table();

  for (final row in customers.rows) {
    print('${row.recordNumber}: ${row['NAME']} ${row['BALANCE']}');
  }
}

Features #

Area Support
File structure header, B+tree pages, RLE page compression, several tables per file
Field types BYTE SHORT USHORT LONG ULONG SREAL REAL DECIMAL DATE TIME STRING CSTRING PSTRING PICTURE GROUP, DIM arrays
Unknown types returned as raw bytes; the rest of the file still loads
MEMO / BLOB read (row.memo, row.blob); kept on update, removed on delete
Text windows1252 (default), cp850, cp437, or any Encoding
Encryption files with an OWNER password: read, and written back encrypted
Keys DUP, OPT, NOCASE, descending parts; rebuilt on every change
Writing insert, update, delete, then toBytes()
Damaged files recovery mode scans leaf pages when the page tree is broken (read-only)

Installing #

dart pub add topspeed
# or, in a Flutter app
flutter pub add topspeed

Reading #

final file = TpsFile.parse(bytes);

// Most files hold one table; files with several need a name.
final table = file.tables.length == 1 ? file.table() : file.table('ORDERS');

// The layout.
for (final f in table.definition.valueFields) {
  print('${f.label}: ${f.type.name}, ${f.length} bytes');
}

// One row by record number, or all of them.
final row = table.row(42);
print(row?['NAME']);   // a field by label ('NAME') or full name ('CUS:NAME')
print(row?.values);    // every field: {NAME: ..., BALANCE: ...}

Values come back as these Dart types:

Clarion Dart
BYTE, SHORT, USHORT, LONG, ULONG int
SREAL, REAL double
DECIMAL String, such as "-12.50" (no precision loss)
STRING, PICTURE String, trailing spaces removed
CSTRING, PSTRING String
DATE DateTime? (null when blank)
TIME Duration? (null when blank)
DIM(n) List of the element type
unknown type Uint8List (raw bytes)

Dates stored in LONG fields #

Many Clarion programs keep dates and times in LONG fields as "standard" dates (days since 28 December 1800) and times (hundredths of a second since midnight, plus one). Convert them like this:

final shipped = clarionDateToDateTime(row['SHIPDATE'] as int);   // DateTime?
final at = clarionTimeToDuration(row['SHIPTIME'] as int);        // Duration?
table.update(row.recordNumber, {'SHIPDATE': dateTimeToClarionDate(DateTime.now())});

MEMO and BLOB #

final notes = row.memo('NOTES');   // String?
final photo = row.blob('PHOTO');   // Uint8List?
print(row.memos);                  // {NOTES: ..., PHOTO: ...}

Code pages #

Clarion stores text in the ANSI code page of the PC that wrote the file. The default is Windows-1252 (Western Europe and the Americas). Older DOS programs often used CP850 or CP437:

final file = TpsFile.parse(bytes, encoding: cp850);

Any Encoding from dart:convert works too (latin1, utf8, ...).

Encrypted files #

final file = TpsFile.parse(bytes, password: 'owner-password');

Without the password, parse throws a TpsException that says the file may be encrypted. toBytes() encrypts again with the same password.

Writing #

final file = TpsFile.parse(await File(path).readAsBytes());
final table = file.table();

// Only write when the library reproduces this file's keys exactly.
if (!table.indexesVerified) throw StateError('keys not verified');

final n = table.insert({'CODE': 10, 'NAME': 'New customer', 'BALANCE': '0.00'});
table.update(n, {'NAME': 'Renamed'});
table.delete(7);

await File(path).writeAsBytes(file.toBytes());
  • Changes stay in memory until you call toBytes(). That call returns the whole file with a freshly built page tree and a higher change counter.
  • Each change rebuilds the table's keys and checks unique keys. A duplicate throws TpsException.
  • indexesVerified compares the keys the library rebuilds with the keys stored in the file. If they differ, the library does not encode that file's keys the way TopSpeed does, and you should not write it.
  • MEMO and BLOB values cannot be written yet. update keeps them, and delete removes them with the row.

Safety checklist for production data #

  1. Never write while a Clarion program has the file open. This library does not take TopSpeed's locks.
  2. Keep a backup. Write the new bytes, read them back, and compare.
  3. Check indexesVerified before writing.
  4. Do not write a file opened in recovery mode (file.recovered). toBytes() refuses to do so.

Checking your own files #

The repository has a command-line checker. It parses every .tps file in a folder, decodes all values and memos, verifies the keys, and does a full rewrite in memory. It never changes the files.

dart run tool/tps_check.dart path/to/folder [maxBytes] [--password=x] [--encoding=cp850]
dart run tool/tps_dump.dart path/to/FILE.TPS [rows] [--password=x]

The result is a tally such as {OK: 120}. Any other status names the file and the problem: INDEX_MISMATCH, REWRITE_MISMATCH, RECOVERED, ERROR or SKIP(...). Please report such files (without private data) as issues.

Limitations #

  • MEMO/BLOB writing is not supported.
  • MEMOs, BLOBs and keys kept in external files (externalFile is not empty) are not read.
  • TpsFile.parse loads the whole file into memory.
  • The format is not publicly documented. This library follows the behaviour of real files and of tps-parse. Before writing, always check indexesVerified.

License #

MIT, see LICENSE. The encryption code is derived from tps-parse (Apache 2.0); see NOTICE.

Releasing (maintainers) #

Every push or merge to main runs the checks in .github/workflows/ci.yml. If pubspec.yaml holds a version that is not on pub.dev yet, release.yml creates the tag v<version>, and that tag starts publish.yml, which publishes to pub.dev.

To release a new version:

  1. Raise version in pubspec.yaml (plain X.Y.Z).
  2. Add a ## X.Y.Z section to CHANGELOG.md.
  3. Merge or push to main.

One-time setup:

  1. Publish the first version by hand: dart pub publish.
  2. On pub.dev, open the package's Admin tab and enable Automated publishing from GitHub Actions. Use repository Isaac80536/topspeed and tag pattern v{{version}}, and require the GitHub environment pub.dev.
  3. On GitHub, create the environment pub.dev (Settings > Environments).
  4. Create a fine-grained personal access token with Contents: Read and write on this repository only, and save it as the Actions secret RELEASE_TOKEN. Tags pushed with the default token do not start other workflows, so the tag must be pushed with this token.
2
likes
0
points
888
downloads

Publisher

unverified uploader

Weekly Downloads

Pure Dart reader and writer for Clarion TopSpeed (.tps) database files: tables, keys, MEMO/BLOB, code pages and encrypted files. No native code.

Repository (GitHub)
View/report issues

Topics

#clarion #topspeed #database #file-format #parser

License

unknown (license)

More

Packages that depend on topspeed