feat: add atomic manipulation obstacle updates

[TomCC7]

Contribution path

• Linked issue or discussion: DIM-1328

Current Flow

  ## Overall flow

  A. Manipulation client / RPC
     manipulation_client
          ↓
     ManipulationModule
          ↓
     WorldMonitor ─────→ WorldSpec backend ─────→ collision/planning state
          │                    │
          │                    └─ backend RLock provides atomicity
          ↓
     VisualizationSpec ───────→ Viser or embedded Meshcat


  B. Explicit collision-object messages
     CollisionObjectMessage
          ↓
     WorldMonitor.on_collision_object()
          ↓
     WorldObstacleMonitor
          ↓
     WorldMonitor ─────→ same WorldSpec + visualization path as A


  C. Perception
     Detection3D / ObjectDB objects
          ↓
     WorldObstacleMonitor
          ↓
     WorldMonitor ─────→ same add/remove/pose-update methods

  So WorldObstacleMonitor is present in B and C, but not A.

Problem

Obstacle replacement and native planning-scene queries did not share one
synchronization boundary, so collision checking could overlap a remove/add
replacement. Drake's pose update also moved only the visualization while its
collision geometry remained at the old pose. Complete replacement was not
available through the world, visualization, module RPC, or manipulation client
APIs.

Solution

• Add complete replacement keyed solely by Obstacle.name and retain a
  first-class pose-only update that preserves non-pose properties.
• Serialize obstacle mutations with collision, distance, state/context,
  scene-backed FK/Jacobian, and opaque RoboPlan-native planning operations.
• Replace Drake collision geometry for both complete and pose-only updates.
• Require world finalization for every obstacle operation, store defensive
  obstacle snapshots, and fail closed after unexpected native update failures.
• Update monitor routing and make visualization best-effort, with Viser logging
  and a persistent frontend warning on renderer failure.
• Expose both operations through ManipulationModule RPC and the interactive
  manipulation client (update_obstacle, shape-specific update helpers, and
  update_pose).

Complete updates never merge the prior obstacle. Generic planners may observe
different complete scene revisions between collision checks; opaque
RoboPlan-native planning holds the scene lock for its full call. Stored-plan
invalidation and execution-time obstacle guarantees remain out of scope.

Perception paths and known limitations

WorldObstacleMonitor has two distinct perception paths:

• Detection3D adds a complete box for a new detection, but an existing
  detection receives only a pose update. Bounding-box size changes are
  currently ignored, and tracking timestamps advance even if the world rejects
  the pose update.
• ObjectDB updates only the monitor cache. An explicit refresh_obstacles()
  prepares complete obstacles, then atomically removes and re-adds the full
  ObjectDB obstacle set under the shared world lock. This reflects geometry
  changes, but churns unchanged obstacles and has no rollback if an add fails.

The shared lock prevents planning and collision queries from observing either
perception batch midway through its world mutations. This PR provides the
complete-update primitive but does not migrate these perception paths to it.
A follow-up should cache accepted Detection3D geometry, use complete
replacement when dimensions change, advance tracking only after accepted
updates, and diff stable ObjectDB IDs into add, update, and remove operations.

How to Test

Start a manipulation blueprint:

dimos run xarm7-planner-coordinator

In another terminal:

python -i -m dimos.manipulation.planning.examples.manipulation_client

Then exercise both update paths:

add_box("box", 0.4, 0.0, 0.2, w=0.1, h=0.1, d=0.1)
update_box("box", 0.5, 0.0, 0.2, w=0.2, h=0.1, d=0.1)
update_pose("box", 0.6, 0.0, 0.2)

AI assistance

OpenAI Codex with GPT-5 was used extensively to explore the design, implement
the change, write tests and documentation, and draft the PR. The implementation
was validated with the manipulation test suite and Ruff; the remaining
environment-dependent checks are listed above.

Checklist

• I have read and approved the CLA.

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
dimensional	🟢 Ready	View Preview	Jul 24, 2026, 7:32 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
dimensional	🟡 Building	–	Jul 24, 2026, 7:31 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

[codecov]

Codecov Report

❌ Patch coverage is 72.26190% with 233 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
dimos/manipulation/planning/world/drake_world.py	37.25%	85 Missing and 11 partials ⚠️
...planning/world/test_drake_world_planning_groups.py	10.00%	45 Missing ⚠️
...imos/manipulation/planning/world/roboplan_world.py	83.62%	18 Missing and 10 partials ⚠️
...mos/manipulation/visualization/viser/visualizer.py	45.83%	8 Missing and 5 partials ⚠️
dimos/manipulation/visualization/viser/scene.py	64.70%	10 Missing and 2 partials ⚠️
dimos/manipulation/manipulation_module.py	54.54%	5 Missing and 5 partials ⚠️
...ulation/planning/monitor/world_obstacle_monitor.py	55.00%	5 Missing and 4 partials ⚠️
dimos/manipulation/test_roboplan_world.py	96.22%	5 Missing and 1 partial ⚠️
dimos/manipulation/visualization/test_factory.py	50.00%	5 Missing ⚠️
...ipulation/planning/examples/manipulation_client.py	73.33%	4 Missing ⚠️
... and 2 more

@@            Coverage Diff             @@
##             main    #3164      +/-   ##
==========================================
- Coverage   74.02%   74.00%   -0.02%     
==========================================
  Files        1105     1107       +2     
  Lines      103722   104485     +763     
  Branches     9500     9544      +44     
==========================================
+ Hits        76777    77326     +549     
- Misses      24225    24415     +190     
- Partials     2720     2744      +24     

Flag	Coverage Δ
OS-ubuntu-24.04-arm	67.45% <60.86%> (-0.03%)	⬇️
OS-ubuntu-latest	69.55% <67.02%> (-0.01%)	⬇️
Py-3.10	69.55% <67.02%> (-0.01%)	⬇️
Py-3.11	69.54% <67.02%> (-0.01%)	⬇️
Py-3.12	69.55% <67.02%> (+<0.01%)	⬆️
Py-3.13	69.54% <67.02%> (-0.01%)	⬇️
Py-3.14	69.56% <67.02%> (-0.01%)	⬇️
Py-3.14t	69.55% <67.02%> (+<0.01%)	⬆️
SelfHosted-Large	29.58% <11.95%> (-0.10%)	⬇️
SelfHosted-Linux	36.16% <18.53%> (-0.11%)	⬇️
SelfHosted-macOS	35.17% <18.53%> (-0.11%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...tion/planning/examples/test_manipulation_client.py	100.00% <100.00%> (ø)
...mos/manipulation/planning/monitor/world_monitor.py	71.81% <100.00%> (+0.63%)	⬆️
dimos/manipulation/planning/spec/models.py	98.78% <100.00%> (ø)
dimos/manipulation/planning/spec/protocols.py	100.00% <100.00%> (ø)
dimos/manipulation/test_manipulation_unit.py	100.00% <100.00%> (ø)
...anipulation/planning/monitor/test_world_monitor.py	93.70% <95.00%> (+0.10%)	⬆️
...on/visualization/viser/test_viser_visualization.py	96.89% <94.73%> (-0.16%)	⬇️
...ipulation/planning/examples/manipulation_client.py	42.60% <73.33%> (ø)
dimos/manipulation/visualization/test_factory.py	71.50% <50.00%> (-1.14%)	⬇️
dimos/manipulation/test_roboplan_world.py	97.74% <96.22%> (-0.11%)	⬇️
... and 7 more

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[greptile-apps]

Greptile Summary

Adds atomic obstacle replacement and pose updates throughout manipulation planning.

• Exposes complete and pose-only updates through RPC and the interactive client.
• Serializes world mutations and planning queries across Drake and RoboPlan backends.
• Replaces Drake collision geometry so updated obstacles affect planning queries.
• Adds best-effort Viser updates with persistent renderer warnings.
• Expands monitor, backend, client, and visualization test coverage.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failures remain.

Important Files Changed

Filename	Overview
dimos/manipulation/planning/world/drake_world.py	Adds finalized-world obstacle replacement, defensive snapshots, failure invalidation, shared locking, and collision-query coverage.
dimos/manipulation/planning/world/roboplan_world.py	Adds synchronized complete and pose-only obstacle updates around RoboPlan scene operations.
dimos/manipulation/planning/monitor/world_monitor.py	Routes accepted world updates to visualization while holding the authoritative mutation lock.
dimos/manipulation/planning/monitor/world_obstacle_monitor.py	Distinguishes complete collision-object replacements from pose-only updates.
dimos/manipulation/visualization/viser/visualizer.py	Makes Viser obstacle update failures non-fatal and exposes persistent frontend warnings.
dimos/manipulation/planning/world/test_drake_world_planning_groups.py	Verifies that post-finalization complete and pose-only updates change collision-query results.

Sequence Diagram

sequenceDiagram
    participant Client as Manipulation client / RPC
    participant Module as ManipulationModule
    participant Monitor as WorldMonitor
    participant World as WorldSpec backend
    participant Viz as VisualizationSpec
    Client->>Module: update obstacle or pose
    Module->>Monitor: authoritative mutation
    Monitor->>World: mutate under shared lock
    World-->>Monitor: accepted / rejected
    alt accepted
        Monitor->>Viz: best-effort visualization update
    end
    Monitor-->>Module: mutation result
    Module-->>Client: result

Loading

_{Reviews (3): Last reviewed commit: "test: verify Drake collision state after..." | Re-trigger Greptile}