perf(sidebar): stop hierarchical schema search and refresh reading every schema - #3099
Merged
Merged
Conversation
Base automatically changed from
fix/schema-service-hierarchical-caches
to
main
September 23, 2026 19:24
…ma-query-storms # Conflicts: # CHANGELOG.md # TablePro/Core/Services/Query/SchemaRefreshService.swift # TablePro/Core/Services/Query/SchemaService.swift # TablePro/ViewModels/QuickSwitcherViewModel.swift # TableProTests/Services/SchemaServiceDatabaseSwitchTests.swift # TableProTests/Services/SchemaServiceRefreshTests.swift # TableProTests/ViewModels/QuickSwitcherCrossSchemaTests.swift
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found while investigating #3048 (#3060). Stacked on #3095, which keys these same lists by database; base is its branch, so this diff is only the change below.
Both defects are on the engines grouped by hierarchical schema: Oracle, Snowflake, BigQuery, Spanner, Trino, Dameng and Cloudflare R2 SQL. Their sidebar lists every schema of the database and reads each schema's objects on its own, two to four queries a schema (tables and routines, plus triggers on Oracle).
Defect 1: one keystroke in the sidebar filter read every schema
SidebarTreeView.scheduleSearchLoadloaded every schema not yet loaded, 300 ms after the filter text changed, so it could judge whether each one held a match. The outline also expanded every schema it could not judge yet, so each of those expansions would have read the schema even without that call. A database with 200 schemas cost 400 to 600 queries on the first keystroke.Fix. The hierarchical tree judges an unread schema from the all-schema listing #3060 added, the way #3060 made the Tree layout judge an unexpanded schema.
hierarchicalSchemaIsVisiblebecomeshierarchicalSchemaSearchVerdict, which returns the samematch/noMatch/unknownasschemaSearchVerdict:Only a match expands, so only a matching schema is read.
scheduleSearchLoadis gone, andSidebarViewModelasks for the browsed database's listing on a hierarchical connection too, through the sameAllSchemaTablesDemand, which asks again after a catalog change or reconnect.Procedures, functions, triggers and types. The listing holds tables and views only, and no engine here can list routines across schemas in one call. So a routine, trigger or type matches only in a schema whose objects have been read: one expanded since connecting, or one an earlier search matched and expanded. A schema read before the last catalog change still counts for these, since the match expands it and the expansion reads it again. This is the same reach #3060 gave the Tree layout, and
docs/features/connection-window.mdxnow says so. Keeping the old reach would mean reading every schema again, which is the defect.Defect 2: every COMMIT read every loaded schema again
Every COMMIT reports a catalog change (
.objectsand.schemas), andSchemaRefreshService.performRefreshthen calledrefreshLoadedSchemaObjects, which reloaded every schema in the loaded or loading state, one after another. After a search had loaded every schema, each COMMIT re-ran two to four queries per schema.Fix: stale-while-revalidate, as #3060 did for its listing. A catalog change marks every loaded schema stale and keeps what each one shows (
markLoadedSchemaObjectsStale), then reads at once only the schemas something is about to judge: the browsed schema, which is the browse catalog, and any schema holding a queued truncate or drop, which the change prunes against the refreshed lists when it finishes. Every other schema is read by its next reader: an expanded tree row (schemaObjectsNeedFetchreplaces the outline's.idleguard) or a caller ofloadSchemaObjects.CatalogFreshness, keyed by schema, with one addition: it records the revision a fetch started at, so a reader asks once per change. A failed read is not retried until the next change, since the failure publishes a change every reader observes. A read cut short by a cancel is asked for again.schemaLoadGenerations) is unchanged: the latest fetch to start is the one that commits.CatalogEditAdoption.loadedBrowseCatalogjudges queued operations against current lists only, and no longer counts a hierarchical engine's browsed schema as covered by the flat list, which is empty there.Measured
Mock driver counting every catalog read; in tests the Oracle plugin is not loaded, so a schema read is 2 queries (tables and routines); with the plugin it is 3.
After is measured (
HierarchicalSchemaSearchCostTests,SchemaRefreshCommitCostTests). Before is the code path:scheduleSearchLoadread every unloaded schema andrefreshLoadedSchemaObjectsreloaded every loaded one, 2 reads each here. On Oracle with its plugin loaded each is 3, so before is 600. A broad search still reads every schema it matches, since a matched schema expands to show its matches; that is the same cost #3060's Tree layout has.Tests
New:
SchemaServiceStaleSchemaTests(8): a change reads nothing and keeps rows;fetchingNowreads only the named loaded schema; a reader reads a stale schema once; a failed read keeps rows and waits for the next change; a fetch that began before a change shows its rows but stays stale; a read after a change does not join the earlier fetch; a cancelled read is asked for again; only current lists can prune a queued truncate or drop.SchemaRefreshCommitCostTests(1): the measured COMMIT case. It compiles against fix(sidebar): key per-schema object lists by database #3095's API; the edit that turns it red is putting back the reload of every loaded schema.HierarchicalSchemaSearchTests(16): the verdict's cases, routines in unread schemas included, anddatabasesListedForSearch.HierarchicalSchemaSearchCostTests(2): the measured search cases. The old verdict had no listing input, so these cannot run against it; the edit that turns them red is returning.unknownfor every unread schema, which is what made the outline expand and read them all.CatalogFreshnessTests(+5) for the fetch-started record.Updated:
DatabaseTreeFilterTests,DatabaseTreeFilterQualifiedSearchTests(new verdict API),QuickSwitcherCrossSchemaTests(staleSchemas, +2 cases),SchemaServiceRefreshTests,SchemaServiceDatabaseSwitchTests(new refresh signature).Verification
verify.sh testover 21 suites (the new ones plusSchemaServiceDatabaseSwitchTests SchemaServiceTests SchemaServiceRefreshTests SchemaServiceHierarchicalTests SchemaServiceSideObjectsTests SchemaServiceRoutinesTests SchemaRefreshServiceTests SchemaRefreshAfterWriteTests CatalogEditAdoptionTests SidebarViewModelTests DatabaseTreeAllSchemaTablesTests QuickSwitcherViewModelTests AllSchemaTablesDemandTests CatalogFreshnessTests DatabaseTreeFilterTests DatabaseTreeFilterQualifiedSearchTests QuickSwitcherCrossSchemaTests): PASS, 267 executed, 267 passed.--stricton the 19 changed Swift files: 0 violations.verify.sh docs: PASS.MainSplitViewController+RecentTabs.swiftlacks.versionHistory, fixed by fix(tabs): give version history tabs a symbol in the recent tabs list #3093;FileTabExternalChangeTestsandOracleColumnStatementsTestsbreak the test target, fixed by test: fix the two test files that stop the test target compiling on main #3098). I patched those three locally to run the suites; none of that is in this PR.Reviewed with a
feature-dev:code-revieweragent (Codex is out of quota until Sep 29). It found nothing at its confidence bar. Its one note, that a settled-but-stale schema's verdict is untested, is covered bystaleReadYieldsToTheListing,staleReadKeepsAProcedureMatchandstaleReadWithoutAMatchIsHiddeninHierarchicalSchemaSearchTests, which it did not open.No UI automation: the defects are query counts against a live server with hundreds of schemas, which no UI test here can stand up. The decisions are pure functions and service calls, tested directly.