Conversation
DatasetPage.resetVersionTabList() issued 2 service queries per file of every version (findVarMetByFileMetaId + findAllGroupsByFileMetadata). For 6 files x 2 versions the service-call section alone fired 86 SELECTs. - VariableServiceBean: add findVarMetsByFileMetaIds() and findAllGroupsByFileMetadatas() - single IN queries with to-one JOIN FETCH (fileMetadata, dataVariable, dataTable), so grouping the result costs no extra queries. Empty input yields empty list. - DatasetPage.resetVersionTabList(): collect all file-metadata ids across versions, run the 2 batched queries, group in memory. - VersionTabVariableLoadBudgetIT: QuickPerf/testcontainers repro that scopes measurement to the service calls: 86 -> 2 SELECTs, plus a behavioral check that every v1 file still gets its rows. Draft: for private perf review; not yet proposed upstream.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem (hotspot A, private repro)
DatasetPage.resetVersionTabList()(DatasetPage.java:4803+) runs 2 service queries per file of every version whenever a privileged user renders the Versions tab:Measured on a 6 files x 2 versions fixture, the service-call section alone fires 86 SELECTs (24 service queries + per-row eager hydration). A 1.5K-file x 10-version dataset would issue ~30K queries here (estimated).
Fix
VariableServiceBean.findVarMetsByFileMetaIds()+findAllGroupsByFileMetadatas(): singleIN (...)queries with to-oneJOIN FETCH(fileMetadata,dataVariable,dataTable) - no cartesian product, grouping costs zero extra queries. Empty input yields empty list.resetVersionTabList(): collect all file-metadata ids across versions, run the 2 batched queries, group in memory. Same lists per file as before.Verification
VersionTabVariableLoadBudgetIT(QuickPerf + testcontainers, follows the branch harness): 86 -> 2 SELECTs for the service-call section, plus a behavioral check that every v1 file still gets its 3 variable-metadata rows.expected 2 batched variable-service SELECTs, but got 86), GREEN post-fix.Notes / follow-ups
FileMetadata.dataFile(pre-existing, weaving=false territory covered by upstream Faster file and dataset pages for datasets with many files IQSS/dataverse#12740-style@BatchFetchwork) and per-versiongetContributorsNames.