feat(ome-tiff): opt-in per-timepoint split for offloadable OME-TIFF acquisitions - #650
Draft
hongquanli wants to merge 3 commits into
Draft
hongquanli wants to merge 3 commits into
hongquanli wants to merge 3 commits into
Conversation
…cquisitions Third of three stacked PRs for large acquisitions. OME-TIFF wrote one pre-allocated file per FOV holding every timepoint, so nothing could be offloaded before the last timepoint. With the new opt-in split each timepoint gets its own <t>/ome_tiff/<region>_<fov>.ome.tiff (SizeT=1, TheT=0, DeltaT relative to the acquisition start, TimeIncrement kept), which finalizes after Z x C planes and is then listed in the transfer manifest like every other format. Strictly opt-in: OME_TIFF_SPLIT_TIMEPOINTS (default off) or a per-run choice; with it off the writer, layout and OME-XML are byte-identical (verified against the previous module versions, UUIDs aside). - utils_ome_tiff_writer / SaveOMETiffJob: when AcquisitionInfo. split_timepoints, the output folder is the timepoint folder's ome_tiff/, the on-disk progress file is keyed per timepoint (an aborted timepoint never contaminates the next), shape and expected count are per timepoint, plane indices are local (t=0) while DeltaT stays global, and the OME Description names the global timepoint. - AcquisitionInfo gains split_timepoints and acquisition_start_time; the worker passes the run's effective flag and start time. - Setting + per-run flag: AcquisitionParameters.split_ome_timepoints, MultiPointController.set_split_ome_timepoints, folded with the global setting in build_params, recorded as acquisition.ome_tiff_split_timepoints in acquisition.yaml and read back (absent = unchanged), reset after the run. Preferences: "Split OME-TIFF by timepoint" in Large Acquisitions. - Pre-flight "Not Enough Disk Space" dialog: for a multi-timepoint OME-TIFF run offers "Split OME-TIFF by timepoint and continue", which also enables large acquisition mode for the run. Tests: writer round trips (split/unsplit, DeltaT continuity, per-timepoint progress files, abort isolation, simulation), plumbing and pre-flight matrix, Preferences, and simulated runs: unsplit layout unchanged and listed once at the end; split run writes one SizeT=1 file per timepoint, lists each before its timepoint_done, records the flag; the upload tool moves and verifies a split run. Browsing split runs in the NDViewer needs the companion ndviewer_light change (separate PR, submodule bump to follow). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F7SDD9fYF47FUoL8M1dfvF
…sitions-ome-split
…sitions-ome-split
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Third of three stacked PRs for large acquisitions (stacks on #649; design: AI-docs
Squid/to-do/2026-09-14-large-acquisitions-offload-and-pause-design.md). OME-TIFF acquisitions can now be split per timepoint, so finished timepoints become movable during the run like every other format.Strictly opt-in.
OME_TIFF_SPLIT_TIMEPOINTS(default off) or a per-run choice. With it off, the OME-TIFF writer, layout and metadata are byte-identical to today (oneome_tiff/<region>_<fov>.ome.tiffper FOV holding all timepoints, listed in the manifest only when the last timepoint finalizes it).When on
<t>/ome_tiff/<region>_<fov>.ome.tiffper timepoint (timepoint folder named withFILE_ID_PADDING, like the individual-images format), each a valid standalone OME-TIFF withSizeT=1, planeTheT=0,DeltaTrelative to the acquisition start (so times stay comparable across files),TimeIncrementretained.acquisition.ome_tiff_split_timepoints(absent = unchanged). The effective flag is recorded inacquisition.yaml.tools/upload_acquisition.py verifyunderstands both layouts.Tests
Writer round trips (split and unsplit, DeltaT continuity, per-timepoint progress files, abort of one timepoint not affecting the next, simulation mode), settings/YAML/controller plumbing, the pre-flight decision matrix and dialog wiring, Preferences persistence, and simulated runs (
test_ome_split_integration.py): unsplit layout unchanged and listed once at the end; split run writes oneSizeT=1file per timepoint, lists each before itstimepoint_done, records the flag inacquisition.yaml; the upload tool moves and verifies a split run.🤖 Generated with Claude Code
https://claude.ai/code/session_01F7SDD9fYF47FUoL8M1dfvF