Skip to content

Add S3-compatible replay object storage#3184

Closed
xiaoyaoqilan wants to merge 9 commits into
dimensionalOS:mainfrom
xiaoyaoqilan:codex/replay-object-storage
Closed

Add S3-compatible replay object storage#3184
xiaoyaoqilan wants to merge 9 commits into
dimensionalOS:mainfrom
xiaoyaoqilan:codex/replay-object-storage

Conversation

@xiaoyaoqilan

Copy link
Copy Markdown
Contributor

Summary

  • add an S3-compatible backend for replay artifacts
  • support AWS S3, Cloudflare R2, MinIO, and compatible object stores through configurable endpoints and credentials
  • preserve content-addressed object IDs and manifest/hash validation
  • keep the filesystem backend available for local development and small deployments

Motivation

This is PR 2 of the #3158 implementation stack. It separates durable cloud object storage from the WebRTC live media path and provides a migration path beyond a single server disk.

Stack dependency

Validation

  • targeted object-storage and replay repository tests passed
  • ruff check and ruff format --check passed
  • targeted mypy checks passed

@autofix-ci

autofix-ci Bot commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Hi! I'm autofix logoautofix.ci, a bot that automatically fixes trivial issues such as code formatting in pull requests.

I would like to apply some automated changes to this pull request, but it looks like I don't have the necessary permissions to do so. To get this pull request into a mergeable state, please do one of the following two things:

  1. Allow edits by maintainers for your pull request, and then re-trigger CI (for example by pushing a new commit).
  2. Manually fix the issues identified for your pull request (see the GitHub Actions output for details on what I would like to change).

@codecov

codecov Bot commented Jul 25, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 73.51562% with 339 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
dimos/protocol/pubsub/impl/webrtc/mvp_cli.py 16.94% 195 Missing and 1 partial ⚠️
...s/protocol/pubsub/impl/webrtc/replay_repository.py 79.55% 59 Missing and 32 partials ⚠️
...rotocol/pubsub/impl/webrtc/replay_repository_s3.py 74.64% 23 Missing and 13 partials ⚠️
...otocol/pubsub/impl/webrtc/replay_repository_cli.py 87.03% 5 Missing and 2 partials ⚠️
dimos/protocol/pubsub/impl/webrtc/mvp_metrics.py 88.46% 3 Missing and 3 partials ⚠️
...os/protocol/pubsub/impl/webrtc/providers/broker.py 80.00% 1 Missing and 1 partial ⚠️
dimos/robot/cli/dimos.py 0.00% 1 Missing ⚠️
@@            Coverage Diff             @@
##             main    #3184      +/-   ##
==========================================
- Coverage   73.77%   71.17%   -2.61%     
==========================================
  Files        1109     1119      +10     
  Lines      104383   107201    +2818     
  Branches     9567    10075     +508     
==========================================
- Hits        77013    76301     -712     
- Misses      24695    28247    +3552     
+ Partials     2675     2653      -22     
Flag Coverage Δ
OS-ubuntu-24.04-arm 68.06% <73.51%> (+0.36%) ⬆️
OS-ubuntu-latest ?
Py-3.10 ?
Py-3.11 ?
Py-3.12 ?
Py-3.13 ?
Py-3.14 68.06% <73.51%> (-1.71%) ⬇️
Py-3.14t ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...otocol/pubsub/impl/webrtc/providers/video_track.py 24.39% <ø> (-2.28%) ⬇️
...os/protocol/pubsub/impl/webrtc/test_mvp_metrics.py 100.00% <100.00%> (ø)
...tocol/pubsub/impl/webrtc/test_replay_repository.py 100.00% <100.00%> (ø)
...l/pubsub/impl/webrtc/test_replay_repository_cli.py 100.00% <100.00%> (ø)
...ol/pubsub/impl/webrtc/test_replay_repository_s3.py 100.00% <100.00%> (ø)
dimos/robot/cli/dimos.py 60.24% <0.00%> (-8.16%) ⬇️
...os/protocol/pubsub/impl/webrtc/providers/broker.py 34.31% <80.00%> (-1.55%) ⬇️
dimos/protocol/pubsub/impl/webrtc/mvp_metrics.py 88.46% <88.46%> (ø)
...otocol/pubsub/impl/webrtc/replay_repository_cli.py 87.03% <87.03%> (ø)
...rotocol/pubsub/impl/webrtc/replay_repository_s3.py 74.64% <74.64%> (ø)
... and 2 more

... and 58 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@xiaoyaoqilan

Copy link
Copy Markdown
Contributor Author

Superseded by #3201. The implementation is consolidated into the dimos/hosted_data package and unified dimos data CLI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant