On the O3 side, we introduced a pluggable async I/O layer (AIO) across OSes, a console-based S3 management experience, AWS-aligned delete-marker & versioning behavior, and a refactored S3 object model with stronger bucket encryption, sync tooling, and IAM/S3 test coverage.

On the protocol side, we shipped Filecoin PDP storage, removed streaming mode, aligned metadata encryption paths with O3, batched transactions on both node and SDK, and patched a PDP edge case that could stall aggregation.

Here’s the full update.

About Akave O3

Akave O3 is our S3-compatible object storage API that sits on top of the Akave decentralized data layer. Everything we ship continues to center around three fundamentals:

“Security” – “Performance” – “Operability”

November pushed all three: async-by-default I/O, S3-correct semantics, and protocol-backed verifiability.

‍

Security You Can Trust

Bucket encryption & key handling

• Bucket-level encryption migration
  
  • Added migration logic to ensure buckets use AES256C encryption by default.
  • Upload paths now validate bucket encryption state before proceeding.
  • Expanded tests for:
    
    • Successful encryption removal.
    • “No documents” cases.
    • Database and repo error scenarios.
      
• S3 encryption flow refactor
  
  • S3 encryption key generation paths were simplified and documented for maintainability.
  • Encryption handling now has clearer separation between bucket policy, object metadata, and underlying IPC/PDP storage.

IAM & auth hardening

• IAM repository & service coverage massively expanded:
  
  • Signature verification, content SHA256 validation, and presigned request validation.
  • Repository tests cover:
    
    • Key creation/list/delete with user-not-found, key-limit, and DB failure paths.
    • Access key updates with missing params, invalid statuses, and DB errors.
    • Creation and retrieval of IAM keys, including inactive-key handling.
      
  • IAM context helpers (extractAccountId, extractUsername) now tested for both success and failure paths.
  • Key generation (generateAccessKey, generateSecretKey) tested for validity, uniqueness, and character constraints.
  • Policy document validation (validatePolicyDocument) now has broad test coverage for both valid and failing IAM policies.

Metadata & protocol encryption alignment

• At the protocol layer (Akave v0.4.0):
  
  • Metadata encryption key-derivation path changed, bringing it in line with the O3/SDK metadata encryption work.
  • CLI flags control metadata encryption, and view methods can now decrypt from encrypted state without relying on plaintext names.
  • Double-encryption paths during upload were eliminated.

Security is now enforced not just at the API surface, but end-to-end through IAM, bucket encryption, and the underlying PDP-backed storage.

‍

Performance & Stability

Pluggable async I/O across OSes (O3 v1.4.0 / v1.4.1)

We turned O3’s I/O stack into a configurable subsystem:

• AIO backends via aiomode:
  
  • std (default), posix, uring, dispatch and iocp.
    
• io_uring controls:
  
  • aioentries / RingEntities to tune ring size and concurrency.
  • Diagnostics helpers (urw_is_inited, urw_entries, urw_sq_space_left) for insight into ring usage.
    
• Network AIO:
  
  • AIO-backed listeners and netBackend plus an initial uring_net backend for async connections on Linux.
  • Shared TCP tuning helpers for more predictable connection behavior.

The result is a consistent async path across disk and network I/O, with Linux able to take advantage of io_uring where available.

Uploads, streaming, and IPC downloads

• MPU pipeline (v1.4.0)
  
  • Multipart uploads moved onto an OrderedPipe implementation with better locking, part ordering, and error handling.
  • Stabilized MPU shutdown paths (fixing “file already closed” / “bad FD” cases).
  • Extended checksum support: SHA1, SHA256, CRC32, CRC32C, CRC64, plus clearer invalid content-length errors.
    
    
• Chunked IPC downloads (v1.4.2)
  
  • New ipcChunkedFileDownload path with:
    
    • Ordered chunk processing.
    • Batching and retries per chunk group.
    • Improved logging around chunk ranges and IPC behavior.
      
  • This makes large IPC-backed downloads more robust and better suited for high-latency or high-error environments.
    
• Download mode control (v1.4.1)
  
  • AKAVE_DOWNLOAD_MODE allows environment-driven selection of download strategies (PDP-first, standard, etc.).
  • Centralized IpcDownloadMode with clean fallback logic and better observability.

Core services, file descriptors & jobs

• FileManager & FD metrics
  
  • NewFile / NewFileFromFd unify file descriptor handling.
  • GetRUsageNoFile collects FD stats on Linux and pushes them into logs for operational visibility.
    
• Job scheduling & cache cleanup
  
  • TTL and cache volume-limit jobs re-enabled with corrected scheduling on CacheInterval instead of BucketsInterval.
  • Sync cache walkers now combine lock-file deletion + file processing into a single pass for better I/O efficiency.
    
• Hot-path optimizations
  
  • Revision/version regexes moved to package-level compiled expressions, eliminating repeated regexp.Compile in hot paths.
  • AIO bindings built with stricter C flags (-Wall, -Wextra, -v) to catch inefficiencies and UB early.
  • New indexes on the objects collection (bucket, key, deletion status, latest) accelerate object lookup and delete flows.

‍

Enterprise-Ready Features

S3-correct delete markers, versioning & tagging

• Delete markers & soft-delete (v1.4.0 / v1.4.1)
  
  • DeleteObject now uses IsDeleted + IsLatest for soft deletes while preserving full version history.
  • GetObject without versionId properly excludes delete markers and returns 404 for deleted objects, aligning with AWS S3.
  • HeadObject, CopyObject, and SelectObjectContent propagate x-amz-delete-marker headers as expected.
  • DeleteMarkerEntry XML tagging fixed to match the S3 spec.
  • Sync operations skip pure delete markers that have no underlying data.
    
• Object existence & tagging (v1.4.2)
  
  • New ObjectExist method in httpContext centralizes existence checks.
  • PutObjectTagging, GetObjectTagging, and DeleteObjectTagging now:
    
    • Validate object existence before doing anything.
    • Return NoSuchKey for missing objects instead of vague errors.
      
  • Tagging tests now cover:
    
    • Non-existent objects.
    • Session/temporary objects.
    • Meta-service failure paths with the correct InternalError behavior.
      

Console-based S3 management (Prickly Pear continuation)

Work that started in October’s Prickly Pear GUI matured into a real console experience in November:

• Credentials are automatically created on first login via SSO (Google & GitHub), with the user’s S3 endpoint and access details published and ready to use.
• The console supports creating multiple independent credential sets so teams can silo departments or isolate specific buckets from each other directly through the GUI.

Now available: www.akave.com/free-trial 

‍

Config Auto-Upgrades & S3 Model Refactor

S3 object model refactor (v1.4.2)

• sdk.FileMeta has been replaced by entity.ObjectInfo / entity.ObjectInfoV2, aliasing sdk.IPCFileMeta types.
• Deprecated metadata (e.g., CommitedAt, RootCID) was removed from object info.
• Objects now separate fields into:
  
  • Compatibility (S3-facing surface).
  • Content metadata (size, checksums, encryption, etc.).
  • Internal tracking (IPC/PDP paths, internal IDs, etc.).
    
• A dedicated ObjectChunks type captures chunk metadata explicitly for better control over IPC downloads and repairs.
• S3 service imports and mocks moved from the akave package into the storage package.
• Helper DownloadWithMode was renamed to WithDownloadMode for clarity and consistency.
  

These changes are breaking for anyone pulling directly on internal types, but they make the object model cleaner and more maintainable going forward.

Config & tooling

• HTTP configuration standardized behind config.HttpConfig across services and tests.
• Various legacy S3/iam/kms/web apps and routes were removed to reduce dead code and focus on the new console surface.
• git-gen-next-version.sh was simplified for clearer version bump logic.

‍

‍

Docker / Compose Operational Improvements

• AIO & download mode surfaced in Compose
  
  • compose-rc.yml and setup.sh now expose:
    
    • AKAVE_DOWNLOAD_MODE
    • AIO defaults (aiomode, aioentries / ring entities)
      
• Liburing install & base image
  
  • Dockerfile and install-liburing.sh updated to build and install liburing automatically when aiomode=uring on Linux.
    
• Sane defaults and limits
  
  • File descriptor limits and job intervals aligned with real-world deployments.
  • Legacy debug logs and unused middleware/route definitions removed for a cleaner runtime.
    

This makes it easier to deploy O3 with async I/O enabled and debug behavior in production.

‍

Operator Tooling

• S3 sync improvements
  
  • New --bucket flag to sync a single bucket instead of the entire namespace.
  • Bucket context is propagated through command execution using a shared context key, ensuring correct behavior for filtered syncs.
    
• Bucket encryption tooling
  
  • Sync and encryption logic integrated so buckets are validated/migrated to the expected encryption state as part of normal operations.
    
  • DeleteBucketEncryption now:
    
    • Differentiates “no documents” from actual errors.
    • Behaves correctly in dry-run scenarios.
    • Surfaces meaningful errors from the backing datastore.
      
• S3 & IAM service test harnesses
  
  • Expanded tests for repository caching behavior, auth utilities, and handler binding errors.
  • Dedicated test files ensure handler error responses are stable and predictable.

Operators get a more predictable system with better tooling to understand where things fail and why.

‍

Protocol Layer Upgrades

Akave v0.4.0 – PDP, batching, and streaming removal

A major protocol release aligning Akave’s node/SDK with O3’s direction:

• Upgrade & compatibility
  
  • Not compatible with v0.3.0 or v0.3.1; node and SDK must move together.
    
• Breaking changes
  
  • Streaming mode removed across CLI, SDK, and node (cmd/akavecli, stress tools, API).
  • Metadata encryption key derivation changed, matching the SDK/O3 path for decryptable listings.
    
• Filecoin PDP support
  
  • Node now periodically aggregates data and sends it to a Filecoin PDP provider.
  • Data becomes downloadable directly from the PDP provider via the SDK.
  • PDP contracts, datasets, piece listing, EIP-712 signatures, and payments are wired into the node.
    
• Performance & batching
  
  • Node side: batched FillChunkBlock processing, randomized upload permutations, and IPC batch RPC paths.
  • SDK side: batched AddFileChunks and simplified EOF handling for more efficient uploads.
    
• Bug fixes & DX
  
  • CLI now returns non-zero exit codes for failures.
  • Fixed file re-creation issues after deletion.
  • Removed obsolete streaming and statesync paths.
  • CI pipeline runs PDP smoke tests before SDK tests to catch regressions early.
    

Akave v0.4.1 – PDP aggregation stability

A small but important patch on top of 0.4.0:

• Fixes a bug where a PDP piece file could be closed twice:
  
  • HTTP client auto-closing the request body plus explicit close in PDP code.
  • This could abort aggregation for a valid piece and stall the aggregation loop.
    
• Solution: wrap the file in a nop closer for PDP uploads, ensuring aggregation can proceed.

If you’re running PDP aggregation in production, v0.4.1 is the expected baseline.

‍

Release Recap (November)

O3 v1.4.0
A foundational async I/O and console release.
Key highlights:

• Pluggable AIO (std, posix, uring) with tunable ring entries.
• MPU overhaul on OrderedPipe.
• SSE AES-256 with per-bucket keys.
• New auth app and backend-driven console browsing.
• Expanded checksum support and stronger concurrency controls.

O3 v1.4.1
A console, AIO, and S3 semantics refinement.
Key highlights:

• New console-based S3 management experience.
• AIO extended to dispatch (macOS) and iocp (Windows); uring_net for network I/O.
• S3-accurate delete markers and soft-delete behavior.
• AKAVE_DOWNLOAD_MODE and standardized HttpConfig.
• TTL/cache jobs re-enabled with fixed scheduling.

O3 v1.4.2
An internal S3 model and reliability release.
Key highlights:

• sdk.FileMeta replaced by ObjectInfo / ObjectInfoV2 and ObjectChunks.
• Centralized object existence checking and tagging semantics with NoSuchKey behavior.
• Bucket encryption migration to AES256C and improved sync tooling.
• Large test expansion for IAM, S3, and tagging.
• WithDownloadMode helper and storage-package imports for S3 services.

Akave v0.4.0
Protocol layer aligned with PDP and O3 metadata.
Key highlights:

• Incompatible with v0.3.x; streaming removed.
• Updated metadata encryption derivation path.
• Filecoin PDP aggregation and retrieval.
• Batched block/chunk transactions on node and SDK.
• CLI and CI improvements.

Akave v0.4.1
A focused PDP bugfix release.
Key highlight:

• Resolved double-close of PDP piece files, unblocking aggregation loops.

‍

What’s Next

• Console as the primary control plane
  
  • IAM-aware credential management, project/tenant scoping, and richer S3 policy views.
  • Surfacing PDP datasets, pieces, and lifecycle status directly in the UI.
    
• PDP operator tooling
  
  • Repair workflows that connect PDP signals to object-level actions.
  • Dashboards for datasets, pieces, and verification state.
    
• Hosted O3 backed by Akave
  
  • A hosted O3 offering for teams that want S3 + proofs without running the protocol stack.
  • Lifecycle and tiering policies that move data across hot/warm/cold backed by cryptographic proofs.

FAQ

1. What is the biggest improvement in November’s O3 updates?

The introduction of async I/O (io_uring, POSIX AIO, dispatch, IOCP), enabling consistent high-performance paths across Linux, macOS, and Windows.

2. How does PDP integration improve durability?

Filecoin PDP adds cryptographic proof that stored data exists, is retrievable, and has not been altered — independent of Akave nodes.

3. Does O3 now match AWS S3 semantics?

Yes, delete markers, versioning, tagging, and NoSuchKey behaviors now align with AWS S3.

4. Is streaming mode still supported?

No, streaming mode has been fully removed in Akave v0.4.x for simplicity, security, and PDP alignment.

5. Do users need to upgrade?

Yes, Akave 0.4.0+ is not compatible with 0.3.x and requires coordinated node + SDK upgrades.

Connect with Us

Akave Cloud is an enterprise-grade, distributed and scalable object storage designed for large-scale datasets in AI, analytics, and enterprise pipelines. It offers S3 object compatibility, cryptographic verifiability, immutable audit trails, and SDKs for agentic agents; all with zero egress fees and no vendor lock-in saving up to 80% on storage costs vs. hyperscalers.

Akave Cloud works with a wide ecosystem of partners operating hundreds of petabytes of capacity, enabling deployments across multiple countries and powering sovereign data infrastructure. The stack is also pre-qualified with key enterprise apps such as Snowflake and others.

• Contact
• Join our Builder's Group
• Akave Network 
• Akave Cloud
• Docs
• Get Free trial