Add baremetal ABI support for NODES_PLATFORM=baremetal#1922
Open
fonta-rh wants to merge 8 commits into
Open
Conversation
openshift-ci
Bot
added
the
do-not-merge/work-in-progress
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
openshift-ci
Bot
added
the
needs-ok-to-test
|
Hi @fonta-rh. Thanks for your PR. I'm waiting for a openshift-metal3 member to verify that this patch is reasonable to test. If it is, they should reply with Regular contributors should join the org to skip this step. Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
fonta-rh
force-pushed
the
ocpedge-2775-baremetal-deploy
branch
from
cbd7bec to
70dc4b3
Compare
5 tasks
openshift-ci
Bot
added
the
needs-rebase
fonta-rh
force-pushed
the
ocpedge-2775-baremetal-deploy
branch
from
b0a9a0b to
15e85c1
Compare
Enable the agent-based installation (make agent) pipeline to deploy on real baremetal nodes via a standalone provisioning host without libvirt. Three libvirt-dependent code paths are bypassed when NODES_PLATFORM=baremetal: - 02_configure_host.sh: early return after platform-agnostic setup (firewall, NTP, DNS, registry auth) — skips all VM/network creation - agent/05_agent_configure.sh: new get_baremetal_ips_and_macs() reads MACs from NODES_FILE and IPs from BAREMETAL_IPS instead of virsh - agent/06_agent_create_cluster.sh: new mount/eject_agent_iso_baremetal() delivers ISO via Redfish VirtualMedia instead of virt-xml Also adds a baremetal VIP branch in network.sh to configure dnsmasq DNS from BAREMETAL_API_VIP/BAREMETAL_INGRESS_VIP config vars, and documents all new BAREMETAL_* vars in config_example.sh. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The OS case statement only matched el9 variants, rejecting RHEL 10 with a hard exit. Add rhel10/centos10 as a supported DISTRO — same baseline as rhel9 (python3-pip, python symlink) without EPEL (unnecessary on el10). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
genisoimage is replaced by xorriso (which provides mkisofs) on RHEL 10, and python3-bcrypt is not available in base repos. Patch the metal3-dev-env package defaults to remove these entries on RHEL 10, following the same sed-patch pattern used for Go tarball arch and Ansible version fixes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Redfish address from ironic_nodes.json includes the full API path
(e.g., redfish://host/redfish/v1/Systems/1). After parsing, the 'system'
variable contains 'host/redfish/v1/Systems/1'. Using ${system%%/Systems/*}
strips /Systems/1 but leaves /redfish/v1 in the path, causing a doubled
/redfish/v1/redfish/v1/Managers URL.
Fix: extract just the BMC hostname (${system%%/*}) into bmc_base and build
all Redfish URLs from that. The @odata.id values from Redfish responses
are already absolute paths (/redfish/v1/Managers/1), so prepending
scheme+hostname produces correct URLs.
Fixes both mount_agent_iso_baremetal and eject_agent_iso_baremetal.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When NODES_PLATFORM=baremetal and 02_configure_host.sh is skipped (no libvirt on the provisioning host), switch_to_internal_dns() was never called. This left configure_dnsmasq() writing entries to /etc/NetworkManager/dnsmasq.d/ that NetworkManager ignored because the dns=dnsmasq plugin wasn't active. The installer then failed to resolve api.<cluster>.<domain> and timed out waiting for bootstrap. Call switch_to_internal_dns() in 05_agent_configure.sh when NODES_PLATFORM=baremetal, before the configure_dnsmasq calls. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two fixes for mount_agent_iso_baremetal: 1. Eject existing VirtualMedia before InsertMedia. iLO returns MaxVirtualMediaConnectionEstablished when a media is already mounted, silently failing the insert. The subsequent Inserted check returned true for the OLD image, masking the failure. 2. Symlink the agent ISO from OCP_DIR into WORKING_DIR/<cluster> so it's accessible via the HTTP server. The ISO is generated under ocp/<cluster>/ (relative to dev-scripts checkout) but BAREMETAL_ISO_SERVER points to a URL under the HTTP server root, which may be WORKING_DIR. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
On servers with multiple NICs, the PXE boot NIC (used in ironic_nodes.json ports) may differ from the data network NIC. When BAREMETAL_MACS is set, get_baremetal_ips_and_macs() uses those MACs for agent-config instead of ironic_nodes.json ports. Falls back to existing behavior when unset. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Drop BAREMETAL_GATEWAY from config_example.sh — nothing in dev-scripts reads it; nodes get gateway from DHCP. Remove registry /etc/hosts + login block from 02_configure_host.sh baremetal path — PROVISIONING_HOST_EXTERNAL_IP resolves to the wrong IP (computed from node subnet, not provisioning host address), and the baremetal flow uses CI_TOKEN for registry auth, not a local registry. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fonta-rh
force-pushed
the
ocpedge-2775-baremetal-deploy
branch
from
9e87a34 to
9a9a5d6
Compare
openshift-ci
Bot
removed
the
needs-rebase
openshift-ci
Bot
removed
the
do-not-merge/work-in-progress
Member
|
/ok-to-test |
openshift-ci
Bot
added
ok-to-test
|
@fonta-rh: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
NODES_PLATFORM=baremetalsupport to the agent-based installation (make agent) pipeline for deploying on real baremetal nodes via a standalone provisioning host (no libvirt)get_baremetal_ips_and_macs()reads from NODES_FILE + config vars), and ISO delivery (mount_agent_iso_baremetal()via Redfish VirtualMedia)BAREMETAL_API_VIP/BAREMETAL_INGRESS_VIPconfig vars, and document all newBAREMETAL_*variables inconfig_example.sh01_install_requirements.sh(package availability fixes, metal3-dev-env compatibility patches)BAREMETAL_MACSconfig var for multi-NIC servers where PXE boot MAC ≠ data network MACContext
This is part of OCPEDGE-2775 — enabling Two-Node with Fencing (TNF) deployment on real baremetal hardware. A companion tool (
make baremetal-adoptfrom OCPEDGE-2774) generates theconfig_baremetal_fencing.shthat exportsNODES_PLATFORM=baremetal,NODES_FILE, bridge overrides, and network vars.Target topology: standalone provisioning host without libvirt (e.g., RDU2 HPE e920t nodes with iLO BMC via Redfish).
Validated with three end-to-end deploys on RDU2 baremetal (OCP 4.22.0-rc.5 and 4.22.4), both nodes Ready, all operators Available, full TNF pipeline (auth → setup → fencing → after-setup → update-setup) completed.
Changes
01_install_requirements.sh02_configure_host.shexit 0(skips all libvirt VM/network/pool creation)network.shset_api_and_ingress_vip()— reads VIPs from config, calls existingconfigure_dnsmasq()agent/05_agent_configure.shget_baremetal_ips_and_macs()withBAREMETAL_MACSsupport for multi-NIC servers, conditional dispatch, guardenable_isolated_baremetal_networkandadd_dns_entryfor baremetalagent/06_agent_create_cluster.shmount_agent_iso_baremetal()/eject_agent_iso_baremetal()via Redfish VirtualMedia (eject-before-insert, ISO staging for HTTP server), guardget_node0_ip()agent/common.shgetRendezvousIP()— return firstBAREMETAL_IPSinstead of virshconfig_example.shBAREMETAL_IPS,BAREMETAL_API_VIP,BAREMETAL_INGRESS_VIP,BAREMETAL_ISO_SERVER,BAREMETAL_MACSDesign decisions
02_configure_host.sh— cherry-picks only platform-agnostic operations, zero changes to existing libvirt code pathsBAREMETAL_ISO_SERVER— operator stages the ISO on an HTTP server reachable from BMCs; dev-scripts does not auto-start an HTTP serverBAREMETAL_MACS— optional override for data NIC MACs when they differ from PXE boot MACs (common on multi-NIC servers like HPE e920t where PXE=5C:BA:2C:*, data=b4:96:91:*)genisoimageandpython3-bcryptwhich aren't available on RHEL 10; patched via conditional sed in the metal3-dev-env checkoutTest plan
shellcheckpasses on all modified files (pre-existing warnings only)make agentwithNODES_PLATFORM=baremetalconfig — verify early exit, dnsmasq, manifestsmake agentwith libvirt VMs still works (no changes to libvirt code paths)🤖 Generated with Claude Code