tom_secrets_service 0.8.0
tom_secrets_service: ^0.8.0 copied to clipboard
The secrets service of the Tom Secrets Manager: a background process that holds one user's unlocked vaults and answers local clients, the manager app and tsm.
tom_secrets_service #
The Tom Secrets Manager's secrets service: a background process that holds the
unlocked vaults of one OS user and answers that user's programs. It has no UI;
the manager app and tsm administer it. License: BSD-3-Clause.
What it does #
- Runtime directory (
TomSecretsRuntime): per OS user, as protocol.md 3.1 names it ($TSM_RUNTIME_DIR,$XDG_RUNTIME_DIR/tsm,$TMPDIR/tsm,%LOCALAPPDATA%\TomSecretsManager\run). It is made the user's alone (mode 0700; on Windows the inheritance cut and only the user and SYSTEM granted) and checked for owner and mode at every start.service.lockis the single-instance lock (the OS releases it when the process dies; a second start answersalready_running),service.jsonnames the PID, the start, the protocol and the endpoints. - Client endpoint:
client.sock(Linux, macOS), or a localhost TCP port inclient.portwith the per-start token inclient.token(Windows).helloandwelcome,getandgetManyresolved throughtom_secrets_core. While locked, a waiting request tells the subscribed admins (unlock_needed), sendsprogress, and is answered once the service is unlocked or the approval timeout passes (locked). Every resolved value is checked by aTomSecretsAccessPolicy; the default,TomSecretsDenyAllPolicy, refuses every value; without one the service's own approvals decide. - Peer identity (part-protocol, protocol.md 7.1;
TomSecretsSystemPeerMeasurer): each accepted connection's process is measured once, never taken from what it says. The peer PID comes fromSO_PEERCRED(Linux),LOCAL_PEERPIDandLOCAL_PEERCRED(macOS) or, for Windows' localhost TCP, the owner of the connection inGetExtendedTcpTable. Then the executable, arguments, working directory, start time and parent are read through/proc, libproc andsysctl(KERN_PROCARGS2), orkernel32,ntdlland the process snapshot. The start time is read before and after, so a reused PID is not identified. A peer of another OS user is refused (other_user), one that cannot be measured too (unidentifiable). Interpreted clients (python,node,bun,deno,dartand thedartvmit starts,java,ruby,perl, the shells, PowerShell;TomSecretsInterpreters, extendable) arescriptidentities with the script's real path and SHA-256, the package root ofdart runandpython -m. File hashes are cached by path, size and modification time (TomSecretsFileHashes);TomSecretsPinchecks a grant's pin (hash,off, relaxed until a date, recording the new hash meanwhile). The identity goes to the access policy (TomSecretsAccessRequest.identity) and intounlock_needed. - Approvals and grants (
TomSecretsApprovals, protocol.md 7, 9; the default policy). The allow list of this host is read from the primary vault (tsm.grants/<host>); a grant counts only when an approver key the device recorded at pairing (or its own) signed it with Ed25519 and ML-DSA-65. The most specific grant decides (exe, script,tsm-command; then folder; then interpreter; for one subject a reference beats a store and a store beatsall;neverbeatsallowat equal rank), and a pinned grant holds only while its file's hash is the one recorded (relaxed pins record the new hash until their date). A request no grant answers goes to the approvers:approval_neededto the attached app and over the approval link,progress {pending: approval}to the client;once,alwaysandfolderallow,neverrefuses, and a lasting decision is written as a grant signed with this device's approver key. With no approver attached the request gets an approval code; a single-use token from a trusted device (approval.issuethere,approval.redeemhere) or a signed approval record in the vault answers it; the code timeout ends itno_approver, the approval timeouttimeout. Prompt flooding is held back (one prompt per identity and reference, three per identity, three refusals in ten minutes hold the identity back).tsmanswers no prompt and pre-approves or issues tokens only where the app switched command-line approvals on (approvers.cliSwitch). - Client tokens (protocol.md 7.4, DP-20): for a request with
wantClientTokenthe approving device makes the token (the attached app inapprove, a remote approver with its signed grant over the link); the grant's subject isexe-token <path> <tokenId>, the host keeps only the token's hash and the identities it came with, the client gets the token in itsvalueand brings it in laterhellos; revoking the grant invalidates it; an approval code issues none. - The approval link (
TomSecretsLinkServer, protocol.md 9.7): while unlocked, on the device settingapprovalLink(a VPN or LAN address), the Noise KK handshake with ML-KEM-768 against the recorded approvers' link keys; a connected approver gets the prompts and sends answers, signed grants (applied at once as pending copies), revocations, key changes and new link keys. - The audit log (
TomSecretsAuditLog): one record per resolved item (time, host, identity, reference, outcome, grant or approver, entry UUID, never a value), each encrypted with XChaCha20-Poly1305 under the audit-log key, in a file per day under<data>/audit/, flushed in batches, deleted after 90 days;audit.queryreads it. - Admin endpoint:
admin.sock, oradmin.port. The admin's executable must betsmor the manager app beside the service, or one named inadmin_executables:ofbootstrap.yaml; any other is refused before its key is checked (wrong_executable). Nothing is answered before the mutual challenge (challenge,auth,authed: HMAC-SHA256 under the admin key, protocol.md 6.1) or aclaimwith the master password, which issues a new admin key and revokes the old one. Failed authentications delay the next one (1 s, 2 s, 4 s … 60 s). Thenunlock,unlockStore,lock,status,reload,subscribe(the eventsunlocked,locked,unlock_needed,store_changed,store_state,key_expiry,approval_needed,approval_closed,rate_limited),vaultKeys,save,approve,approval.redeem,approval.issue,grants.list,grants.add,grants.revoke,approvers.cliSwitch,audit.query,settings.get/settings.set(the device settings; a write only from the manager app, applied at once where it can be and the rest named for the next unlock),approvals.list(the requests waiting on this host, asapproval_neededreads them),keys(every key with its due date),compact(the history of agithub:vault repository cut now, with its own.tsm/compaction.yamlretention orretentionDaysinstead, 0 after a key change, pushed over git by this device),token.set(a vault repository's GitHub token renewed, manager app only: checked first — it sees the repository, a short-livedtsm-token-check-…branch is created and deleted to prove it may write, the expiry GitHub reports — then kept where the old one was, the keychain with its expiry or the primary's vault entries, and every store fetched again),keys.rotate(this device's approver and link key pairs rotated, manager app only: what the old approver key signed signed again, the key-change statement written into the vault for every host it approves for, both sent over the approval link at once, the previous link key kept for 14 days for the hosts not reached and the approvers that connect here) andshutdown. - The editing path (DP-13, protocol.md 6.4):
vaultKeyshands the app the keys of the open vaults (password,keyFile,outerKey) and their locations;savetakes the app's edited file, which must open with the store's keys. A file still atbaseRevisionis written as it is; a file that moved meanwhile is merged three-way when the app edited the service's copy, and refused (stale_base) otherwise. Every write goes through the opener'scommitover the copy the service holds:file:vaults on the local disk (merged when the file moved there);github:vaults as one commit on the branch by this device through the vault unit kept per store, merged three-way when another device pushed meanwhile, then kept in the offline cache (deniedwithhistory_rewrittenfor a branch rewritten without a compaction,branch_movingwhen it moved on every attempt). After a refusal for a rewritten branch the store reportsrewritten: {head}(status andstore_state), keeps the refused edit and is not refreshed until the manager app answers withhistory.resolve:take: remotefollows the rewritten branch,take: mergemerges the held copy over it and commits on top. A vault this machine does not serve (BP-15) goes the same way: the primary lists it with its keys,vaultKeySet.unservednames it,vaultKeysnaming it answers its keys (servedfalse), andvaultFilenaming it reads it from its location with those keys and the repository's token and holds the copy for editing until the lock;savewrites it as above. It is never resolved, served to clients, refreshed or listed instatus. A re-keyed file (part-key-rotation): the app writes a vault's new keys into the primary first and then saves the vault re-keyed. A further vault's file that opens only with the keys the primary lists for it now is taken as re-keyed; the primary's own carries its new keys in the save'skeys, which then replace its key file and repository key in the device keychain. A re-keyed file is written as it is through the opener'srekey(never merged; a file that moved since the held copy is refusedstale_base), and the store's keys are the new ones from then on. Until the re-keyed file arrives, the vault stays served with the keys held (and a vault not served keeps its copy while its file has not moved), so a failed re-keyed save can be made again.store_changednames the new revision. A store on its offline copy, or whose remote fails during the save, refuses withoffline/token_expired. The service's own writes of the primary (grants, their last use, the device key) go the same way; one that cannot be delivered stays in the held copy and the next refresh writes it before fetching (nothing of it survives a lock). github:vaults (TomSecretsGitHubOpener): opened throughtom_secrets_core'sTomSecretsVaultUnitoverTomGitHubStorage(the REST API, no clone), with the commit reader for compactions. Each repository's token is asked for on every request: the device keychain's (tsm.github-token.<owner>:<repository>), else one the primary's vault list keeps for a vault in that repository (never for the primary's own repository). The expiry GitHub reports for a token (GitHub-Authentication-Token-Expiration) is recorded for the reminders.- The offline cache (
bootstrap.yaml'scache_dir, default<data>/cache; the device settingofflineCache): every file fetched is kept there as stored. When GitHub cannot be reached or refuses the token, a vault opens from it (offlineortoken_expired,source: cache); without a copy it isunavailable. The primary is opened before the device settings are read, so it is always cached. - The refresh: at unlock, on
reload, and on a timer everyrefreshMinutes(the device setting; 0 switches the timer off). A file that did not move costs one conditional request and is not reopened; a moved file is reopened with the keys held and the index rebuilt (store_changed); a file whose key changed goes toneeds_unlock; a further vault the primary lists anew, or with new keys, is opened with them, and one it no longer lists is dropped. A failed fetch keeps the copy held and never locks the service: network failures back off 1, 2, 4 … 30 minutes, a refused token is not fetched again untilreloador the next unlock. Every change of a store's state or source is sent asstore_state;reloadanswersofflineortoken_expiredwith the stores still on their copy. - Key-expiry reminders (
TomSecretsExpiringKey): the GitHub tokens fromkeyExpiryWarningDays(default 14) before their expiry (the one GitHub reports, else the one recorded with the token: the keychain's, or the primary's vault list's); the vaults' key files and repository keys fromotherKeyWarningDays(default 30) days before their due date (a further vault's entry Expires, the primary'stsm.keys-due); this device's approver and approval-link key pairs and those of the peers it recorded fromotherKeyWarningDaysbefore their rotation age (keyPairAgeMonths, default 24, DP-85) is over (dated at creation, at pairing, at a key change or a new link key; trust material without dates is dated at the next unlock); then 7 days and 1 day before, and once a day once expired. Each step is sent once askey_expiryto the subscribed admins (an admin that subscribes gets those due at that moment), andstatuslists the keys due asexpiring. - Memory hygiene: the keys the service holds while unlocked are
TomProtectedValues oftom_crypto(masked, zeroed on lock), and a resolved value stays masked intom_secrets_core's field value. An answer carrying a secret (value,values,vaultKeySet) is written into the frame straight from bytes that are zeroed after use (tomSecretsSecretFrame), never through aString, and the frame itself is zeroed once the socket has flushed it. - Process hardening (
tomSecretsHardenProcess, at start, before the handover is read; best effort, reported instatus.hardening): no core files (setrlimit(RLIMIT_CORE, 0)on Linux and macOS, the error mode on Windows), not dumpable on Linux (prctl(PR_SET_DUMPABLE, 0)), and on Windows a process DACL that refusesPROCESS_VM_READandPROCESS_VM_WRITEto all but SYSTEM. The macOS hardened runtime is a signing flag of the build. - The operational log (
TomSecretsLog):service.login the data directory, rotated at 1 MiB with three older files kept. It never holds a value, password, key or token; an exception is logged by its type and code, never its message; references and client identities only atdebuglevel (log_level: debuginbootstrap.yaml). - Unlock and lock: the primary file first, then the device settings
container, then the further vaults the primary names (a vault whose key does
not open is
needs_unlock, the others are still served). Locking drops the vaults, the index and the device settings; the admin key survives.status.lockModenames the lock mode by the protocol's letter (anever,bon idle). - Lock modes (DP-14,
TomSecretsLockPolicyfromtom_secrets_core): (a) never relock, (b) relock after the user's input has been idle for a while, and a hard maximum unlocked time. The idle time comes from CoreGraphics on macOS,GetLastInputInfoon Windows and logind's idle hint on Linux; where it cannot be measured (no desktop session) the service never locks itself. - Device settings (DP-15,
TomSecretsDeviceContainerintom_secrets_core):bootstrap.yamlnames where the container's key is kept:keychain(the start is refused without one),vault(the overlay entry@host:<host>/.tom-secrets/deviceof the primary, which no reference reaches),keyfile(mode 0600 beside the container), orauto. The keychain isTomDesktopSecretStoreoftom_core_storage(servicetom-secrets-manager): the macOS login keychain, libsecret on Linux, the Windows Credential Manager. Where it does not answer (an SSH login without D-Bus or credential store)autotakes the vault. - Starting it (
tomSecretsStartService): the starter makes the admin key, starts the service detached, hands the key over on its standard input and reads one status line (ready,already_running,error). The starter,TomSecretsAdminClient, the runtime directory, the data directory and the keychain names aretom_secrets_client's admin library (package:tom_secrets_client/admin.dart), which this package re-exports: the app andtsmuse them without linking the service.
The service reaches KDBX only through tom_secrets_core (DP-29).
Running it #
dart build cli --target bin/tom_secrets_service.dart -o build/service
The bundle (build/service/bundle/) holds the executable in bin/ and
tom_pqc's native library in lib/; they ship together (dart compile exe
does not run tom_pqc's build hook).
It is started by the app or tsm start, not by hand: it expects the handover
line on its standard input (TomSecretsHandover).
Testing #
testkit :test
test/process_test.dart runs on macOS and Linux, test/windows_test.dart on
Windows, the others (the service, the editing path and the log, the secret
frames, the hardening seen from outside a hardened process, the peer identity
of a real client process, the github: vaults with the offline cache, the
refresh and the key-expiry reminders against tom_github_api's in-memory
GitHub) everywhere.