experimental/bundletest: resource handles for all DABs resource types - #6603
Draft
Sankalp-Mittal wants to merge 4 commits into
Draft
Sankalp-Mittal wants to merge 4 commits into
Sankalp-Mittal wants to merge 4 commits into
Conversation
Add a uniform resource-handle layer to the bundletest framework. A shared ResourceHandle base (config, KeyError-safe exists, permissions, grants) is read through the existing get_resource seam, so handles work unchanged on the local and cloud backends with no new backend primitive. JobHandle and VolumeHandle are refactored onto the base, keeping run/last_run and upload/file. Typed subclasses add resource-specific accessors only where a resource references tables/artifacts/other resources: pipelines (libraries, catalog, schema), dashboards and genie_spaces (source_tables, shared serialized-definition parser that handles inline dict or JSON string, matches qualified refs only, and raises LocalUnsupported for file_path-only definitions), quality_monitors (monitored_table), vector_search_indexes (source_table, endpoint_name), model_serving_endpoints (served_models), and apps (command, source_code_path). env.resource(kind, name) reaches every kind generically. The example bundle now declares one instance of every user-authored resource kind with an example test each: dedicated files for the typed handles, and test_config_resources.py covering the pure-config kinds (one labeled test per kind, each asserting a real declared field). The dashboard, genie space, and quality monitor all reference shop.gold.order_summary so they double as cross-resource wiring checks. Co-authored-by: Isaac <no-reply@databricks.com>
Sankalp-Mittal
added this pull request to stack #6604
September 10, 2026 13:58
Contributor
Waiting for approvalCould not determine reviewers from git history. Eligible reviewers: Suggestions based on git history. See OWNERS for ownership rules. |
Sankalp-Mittal
marked this pull request as draft
September 10, 2026 13:58
Collaborator
Integration test reportCommit: 6b503e8
Top 4 slowest tests (at least 2 minutes):
|
- GenieSpaceHandle.source_tables: a genie space's serialized_space uses a different schema from a Lakeview dashboard — it declares tables explicitly under data_sources.tables[].identifier, not in dataset SQL. The shared dashboard parser silently returned [] for genie spaces; give genie its own parser and split the shared dict/JSON-string preamble into _load_serialized. Update the example genie space to the real schema. - PipelineHandle.schema: make behavior match the doc — also read the legacy DLT `target` field, not just `schema`. - ResourceHandle.exists: document that it reports declared-in-config, not deployed/live workspace state. Co-authored-by: Isaac <no-reply@databricks.com>
… into resource-handles # Conflicts: # experimental/bundletest/src/bundletest/env.py
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a uniform resource-handle layer to the
experimental/bundletest/framework — the test-facing API for asserting on a deployed bundle resource. Stacks on the bundletest base (#6597).ResourceHandlebase —.config, KeyError-safe.exists(),.permissions(),.grants(), read through the existingget_resourceseam. Works unchanged on the local and (future) cloud backends; no new backend primitive.JobHandle/VolumeHandlerefactored onto the base (keep.run()/.last_run(),.upload()/.file()), so every handle is uniform.Typed subclasses with special accessors, only where a resource references tables/artifacts/other resources:
.catalog/.schema/.libraries().source_tables()(shared parser).monitored_table().source_table()/.endpoint_name.served_models().command()/.source_code_path.source_tables()parses the inline serialized definition (dict or JSON string), matches qualified refs only (drops CTE names/aliases), and raisesLocalUnsupported(loud skip) forfile_path-only definitions.env.resource(kind, name)reaches every kind generically; typedenv.<kind>()factories for the special ones.Examples
The example bundle declares one instance of every user-authored resource kind (~33), each with an example test — dedicated files for the typed handles, and
test_config_resources.pyfor the pure-config kinds (one labeled test per kind, each asserting a real declared field, no no-ops). The dashboard, genie space, and quality monitor all referenceshop.gold.order_summary(the gold table the jobs produce), so they double as cross-resource wiring checks.Testing
cd experimental/bundletest && uv run --extra dev pytest→ 51 passed, 1 skipped (the notebook-job loud skip), offline against the local DuckDB backend.ruff format+ruff checkclean.This pull request and its description were written by Isaac.