topspeed 0.4.0
topspeed: ^0.4.0 copied to clipboard
Pure Dart reader and writer for Clarion TopSpeed (.tps) database files: tables, keys, MEMO/BLOB, code pages and encrypted files. No native code.
0.4.0 #
TpsTable.applyBatch(deletes:, updates:, inserts:)deletes, changes and adds many rows with one rebuild of the indexes. The file ends up the same, byte for byte, as after the singledelete,updateandinsertcalls in that order, with the same new record numbers, but thousands of rows take seconds instead of minutes. It is all or nothing: when it throwsTpsException, nothing changes. Unique keys are checked against the final rows, so rows can swap unique keys in one batch. Thanks to the user who proposed it.insert,updateanddeleteare now one-row batches. They build each key once instead of twice, so a single write on a big table is faster.
0.3.0 #
- Read rows by key, like Clarion's
SET/NEXT/PREVIOUS/GET, from bothTpsFileandTpsReadertables:byKey(forward orreverse),seek,range,matchingandget. Key values may give only the leading parts of a key; NOCASE and descending parts work as in Clarion, and raw field bytes (Uint8List) can be passed for values no Dart value reproduces. WithTpsReader, a lookup reads only a few pages. TpsTableDefinition.index(name)finds a key by name or label.- New field types:
PDECIMAL(read asString, sign nibbles as the Clarion runtime writes them),BFLOAT4andBFLOAT8(Microsoft Binary Format, read asdouble). They can be written too. TpsRow.toJson()returns valuesjsonEncodeaccepts.- Dates and times as text:
formatClarionDateandformatClarionTime(patterns such asdd-MM-yyyyorh:mm a, with English or Spanish names) andformatClarionPicture(Clarion pictures@D1-@D16and@T1-@T6). They take Clarion standard dates and times fromLONGfields as well asDATEandTIMEvalues. package:topspeed/topspeed_io.dart:openTps(path)opens a reader, andreadTps(path, read)reads a file a Clarion program is writing to, reading again when the file changes meanwhile. No locks are taken.TpsFile.parserepairs trees that hold records out of order, keeping the copy in tree order of any record stored twice, and marks the filerecovered(read-only).- Tools:
tps_checkgains--recursiveand--passwords, names every skipped file, comparesTpsReaderwithTpsFile, checks key order and key lookups, counts stale key entries and stale row counts, and lists the field types and key kinds it saw. Newtps_exportwrites a table as CSV or JSON Lines.tps_dumpreports a wrong password cleanly.
0.2.0 #
TpsReaderreads large files a few pages at a time instead of loading them into memory. It finds rows through the page tree, so index entries are never read. It offersrowCount(from metadata),rows()(lazy, withfromRecord),row(n), MEMO and BLOB values, encrypted files andchangedOnDisk. It is read-only.TpsByteSourceis the bytes behind a reader.TpsMemorySourceworks on every platform. The new librarypackage:topspeed/topspeed_io.dartaddsTpsFileSource, which reads a file on disk.- On a 52 MB real file with 100,960 rows and 28 keys, a full scan takes
under a second with about 50 MB peak memory, against about 580 MB for
TpsFile.parse. One row lookup takes about 0.1 ms. - Breaking:
TpsRow.tableis now aTpsTableInfo(number, name and definition), so rows fromTpsFileandTpsReadershare one type. Code that calledrow.table.insert(...)must keep its ownTpsTable. - New tool:
tool/tps_stream_bench.dart.
0.1.0 #
First public release.
- Read Clarion TopSpeed (
.tps) files: header, B+tree pages, RLE page compression, several tables per file, multi-block table definitions. - Field types BYTE, SHORT, USHORT, LONG, ULONG, SREAL, REAL, DECIMAL, DATE,
TIME, STRING, CSTRING, PSTRING, PICTURE, GROUP, and
DIMarrays. Unknown type codes are returned as raw bytes and do not stop the file from loading. - MEMO and BLOB values, assembled from their blocks.
- Code pages:
windows1252(default),cp850,cp437, or anyEncoding. - Encrypted files (
OWNERpassword): read, and written back encrypted. - Write support:
insert,update,delete, with key rebuilding (DUP, OPT, NOCASE, descending parts), unique key checks and metadata counts.indexesVerifiedtells whether writing a given file is safe. - Recovery mode: a damaged page tree is read by scanning leaf pages (read-only).
- Helpers for Clarion standard dates and times kept in
LONGfields.