rpc_blob_minio 2.1.1
rpc_blob_minio: ^2.1.1 copied to clipboard
S3/MinIO storage adapter for rpc_blob, with presigned download URLs and streamed uploads.
2.1.1 #
- Requires
rpc_dart7.0.0.
2.1.0 #
Changed #
- Requires rpc_dart 6 and rpc_blob 2.3. Two core majors are spanned here: see
rpc_dart's 5.0.0 and 6.0.0 entries — flow control on by default, an expired
deadline as
RpcDeadlineExceededExceptionon every shape, a stream that ends without a trailer raisingUNAVAILABLE, and a handler's bareExceptiontext no longer reaching the caller.
2.0.1 #
RequestTimeTooSkewedis no longer retried. Nothing between attempts corrects the clock, so retrying only turned one error into four.- Documented that a retry after a timeout does not cancel the attempt that timed out — the client offers no cancellation — so both can be in flight. The wrapped operations are idempotent, which is what makes that safe.
2.0.0 #
Breaking — one bucket, a prefix per collection:
- Objects live at
<collection>/<id>in the single bucket named byS3BlobStorageOptions.bucket;bucketPrefixis gone. A bucket per collection does not survive a hosted S3, where accounts are capped on bucket count and creating one is heavyweight. Existing data must be copied under its prefixes before this version reads it. collectionSizereturns null: S3 cannot size a prefix, and the MinIO admin API that used to answer this reports per bucket, which no longer corresponds to anything.useAdminApiis gone with it, along with the SigV4 signing it needed and thehttp.Clientit leaked per call — the adapter now needs plain S3 permissions rather than admin ones.deleteCollectionwalks the prefix and deletes in batches instead of dropping a bucket;listCollectionsreads common prefixes.
Requests per operation:
- A write was six round trips (
bucketExists, thenheadBlobwhich is itselfstatObject+getObjectTags+getBucketPolicy, thenputObject, thengetBucketPolicyagain for the URL). It is one.ensureCollectionreplaces the per-write bucket check, a missing bucket is repaired from the error instead of polled for,immutableObjectsskips the read-before-write for content-addressed stores, andpublicReadstates what the bucket is instead of asking it per descriptor. listBlobsbuilt each descriptor with a HEAD, making a page of N cost N+1 requests. It reads size and mtime from the listing;includeMetadata— which already existed and promised exactly this — opts back into the HEADs.fetchObjectTags(default off) stops paying aGET ?taggingper head for tags this adapter never writes.
Hosted S3 readiness:
- Throttling and transient faults are retried with exponential backoff and
jitter (
maxRetries,retryBaseDelay), andrequestTimeoutbounds a call that never answers. Listings are deliberately not retried: their paging state lives in the client, so a restart would replay rather than resume. connecttakesregion, so a hosted bucket does not need a lookup on first use.writeBlobdocuments that it buffers the object in memory — the checksum is verified over what is stored and a retry has to be able to resend it, which caps object size atmemory / concurrent writesuntil multipart exists.
1.0.5 #
- Allow to pass raw sqlite db connection to adapter
1.0.4 #
- S3 adapter now auto-detects public buckets and returns plain download URLs when anonymous
s3:GetObjectis allowed; presigning is used only for private buckets, still honoringpresign*host overrides.
1.0.3 #
- Added presign-only endpoint overrides (
presignEndpoint/presignPort/presignUseSSL/presignPathStyle) for S3/MinIO to sign links on a public host behind a reverse proxy; removeddownloadUrlMapperto avoid generating invalid signatures.
1.0.2 #
- S3 adapter can rewrite presigned download URLs via
S3BlobStorageOptions.downloadUrlMapper(useful for serving through a proxy/CDN) and configure presign TTL viaS3BlobStorageOptions.presignTtlSeconds; constructor params consolidated intoS3BlobStorageOptions.
1.0.1 #
- S3 adapter now fetches object tags (
?tagging) and merges them intoBlobDescriptor.metadatawithout overwriting metadata provided on upload. - Added
xmldependency to parse S3 tag responses.
1.0.0 #
- Added
S3BlobStorageAdapter(S3/MinIO/Ceph-compatible) storing blobs under<prefix><collection>/<id>with metadata-based versioning and optimistic checks. - New
S3BlobStorageAdapter.connect(...)helper for quick setup; list/Head/read/write/delete/listCollections wired to S3 operations; descriptors include a presigned download URL. - README now documents S3/MinIO usage; pubspec updated with the MinIO client dependency.
- Initial version.