Conversation
TestConfigureSubset exercises the interactive tool-selection branch of `configure_workspace_command()` (it monkeypatches `prompt_for_tools`, `install_tool_binary`, and the config paths). But when the live `UCODE_TEST_WORKSPACE` has a managed CodingAgentConfig published, `configure` takes the managed auto-apply branch instead — which ignores `prompt_for_tools`, installs every managed tool, and (for codex) shells out to `codex debug models --bundled`, failing in the codex-less Claude CI shard. Force "no managed config" for these tests via the sanctioned `UCODE_MANAGED_CONFIG_STUB` hook (an explicit JSON `null`) in the shared `_redirect_config_paths` helper, so they deterministically run the interactive branch regardless of what the workspace publishes. No assertions weakened. Co-authored-by: Isaac <no-reply@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
tests/test_e2e.py::TestConfigureSubset(3 tests) started failing in the "Agent launch tests · Claude" shard with:Root cause (not a code regression)
These tests exercise the interactive tool-selection branch of
configure_workspace_command()— they monkeypatchprompt_for_tools,install_tool_binary, and the per-agent config paths ("testing selection plumbing, not the binaries").But the shared
UCODE_TEST_WORKSPACEnow has a managed CodingAgentConfig published on it. Sorefresh_managed_config()returns non-None andconfiguretakes the managed auto-apply branch instead, which:prompt_for_toolsand configures every managed tool (whytest_empty_picksawinstall_calls == ['claude','codex']), andprepare_codex_catalog→ realcodex debug models --bundled, which fails because the Claude shard installs nocodexbinary.The failing job's captured output confirms the managed branch ran (the "Managed MCP Servers" / "Coding Agents: Claude Code / Failed to configure: Codex" summary panel). This is a workspace-config/environment trigger, unrelated to any app code.
Fix
Make the tests hermetic: force "no managed config" via the sanctioned
UCODE_MANAGED_CONFIG_STUBhook (an explicit JSONnull) in the shared_redirect_config_pathshelper, so all 3 tests deterministically run the interactive branch they were written for — independent of what the live workspace publishes. With no managed config,codex_static_modelsis never set, soprepare_codex_catalogis never reached (no realcodexbinary needed). No assertions were weakened.Verified:
ruff check/formatclean, the 3 tests collect. Live run needsUCODE_TEST_WORKSPACE+DATABRICKS_BEARER(runs in CI).Note (infra, separate)
tests/README.mdsays managed-config coverage is meant to live on a separate workspace; the shared e2e workspace apparently now has a managed config. Worth checking whether that's intended or whether managed coverage should move off the shared workspace. This test fix makes CI robust either way.This pull request and its description were written by Isaac.