Skip to content

Multirack rss config - #42

Draft
andrewjstone wants to merge 3 commits into
mainfrom
multirack-rss-config
Draft

Multirack rss config#42
andrewjstone wants to merge 3 commits into
mainfrom
multirack-rss-config

Conversation

@andrewjstone

Copy link
Copy Markdown
Contributor

This PR utilizes a new commission API from oxidecomputer/omicron#11143 that allows joining a second rack to another such that each can route DDM traffic to each other over front ports.

The interconnect_port code that utilized shell commands to configure interfaces and addresses on illumos was removed in favor of using the new commission API. This API results in sending a MultirackJoinRequest to the multirack join service. The join service then puts the RackNetworkConfig from the MultirackJoinRequest into the bootstore, which gets reconciled in the scrimlet reconcilers such that dpd creates the necessary ports and interfaces, and DDM FSMs are started for those newly created ports.

The interconnect ports qsfp2 and qsfp3 still have their configuration (PortConfig) automatically generated by voxel. It's just that that PortConfig is included in RackNetworkConfig and plumbed down via a MultirackJoinRequest now.

With this PR, and oxidecomputer/omicron#11143, and a proper 2 rack configuration voxel will end up running RSS on the first rack, and then automatically joining the second rack via the commission API.

The demo asciicast can be played in asciinema to show what occurs when both this PR and the omicron one are used in conjunction.

I've asked @nickelization to try to get this integrated, and would appreciate any help and guidance from @sion42x.

@sion42x

sion42x commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator

A couple things will be required here.

  1. Voxel Skaram/omicron pin c6b42fc #61 will bring the commission API and other omicron pins to Sled-agent: detect switch backend (tofino, softnpu, et al) at runtime omicron#11237 which detects switch backend at runtime, making it possible to voxel create image --from-tuf uninhibited.

For most things, and definitely for multirack, we will want to use TUF repos from official Omicron CI as testing proof. Building from --src is fine of course but the TUF repo + fully emulated SP/RoT + Commission API make the most sense and will likely become the default.

  1. To actually make this work for multirack, Multirack join service part 4 omicron#11143 also needs to land. That's where I'd focus my efforts for now @nickelization / @andrewjstone while we get voxel in good standing with in-flight Omicron PRs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants