feat(lammps): load selected dump frames efficiently

[njzjz]

Summary

• add f_idx support to lammps/dump so callers can load arbitrary non-negative frame indices directly
• stream trajectory frames one at a time, retain only requested frames, and stop after the largest requested index
• preserve requested order and duplicate indices for parity with System.sub_system
• validate empty, negative, out-of-range, and conflicting begin/step selections
• split selected trajectories at their actual ITEM: TIMESTEP boundaries instead of assuming a fixed block length

Example:

system = dpdata.System(
    "trajectory.dump",
    fmt="lammps/dump",
    type_map=["O", "H"],
    f_idx=[23, 56, 78],
)

Performance benchmark

The benchmark compares direct sparse loading against the existing workflow of loading the complete trajectory and then calling sub_system(indices).

• synthetic LAMMPS dump: 12.15 MiB, 5,000 frames, 100 atoms per frame
• selection: 10 evenly distributed frames, including the final frame
• measurement: one warm-up followed by 5 measured runs per method
• each measurement ran in a fresh Python process; table values are medians
• environment: Linux 6.8.0-110-generic, Python 3.13.9, NumPy 2.5.1

Method	Median wall time	Median peak RSS
System(..., f_idx=indices)	0.092 s	39.38 MiB
System(...).sub_system(indices)	1.328 s	110.59 MiB

Direct sparse loading was 14.38x faster and reduced peak RSS by 64.39% for this workload. Both methods returned the same 10 selected frames. Because the final frame was selected, both methods traversed the full file; the improvement comes from avoiding storage and parsing of unselected frames.

Validation

• python -m unittest test_lammps_dump_skipload.py test_lammps_dump_to_system.py test_lammps_dump_unfold.py test_lammps_dump_shift_origin.py test_lammps_dump_idx.py test_lammps_read_from_trajs.py test_lammps_spin.py — 70 tests passed
• ruff check dpdata/ tests/test_lammps_dump_skipload.py — passed
• ruff format --check dpdata/ tests/test_lammps_dump_skipload.py — passed
• dpdata --help and dpdata --version — passed
• full discovery ran 2,241 tests; 12 Amber mask tests errored because the optional parmed dependency is not installed in the environment, and 43 tests were skipped

Closes #367.

Coding agent: Codex
Codex version: codex-cli 0.144.6
Model: gpt-5.6-sol
Reasoning effort: xhigh

Summary by CodeRabbit

• New Features
  • Added precise frame selection when loading LAMMPS trajectories.
  • Select one or multiple frames using indices, while preserving requested order and duplicates.
  • Existing sequential frame sampling remains available.
• Bug Fixes
  • Added validation for empty, negative, or out-of-range frame selections.
  • Prevented incompatible combinations of direct frame selection with sequential sampling options.
  • Improved loading efficiency by stopping after the final requested frame.

[codspeed-hq]

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 2 untouched benchmarks

---

_{Comparing feat/lammps-selective-frames (376317d) with master (212ab49)}

[Copilot]

Pull request overview

This PR extends dpdata’s LAMMPS dump loader to support efficient sparse frame loading via a new f_idx selection option, enabling callers to load only specific non-negative frame indices without parsing and storing the entire trajectory.

Changes:

• Added f_idx support to dpdata.formats.lammps.dump.load_file() and wired it through the LAMMPS dump plugin (fmt="lammps/dump"), including validation for empty/negative/out-of-range indices and mutual exclusivity with non-default begin/step.
• Implemented streaming frame iteration and early-stop behavior (stop after the largest requested index) while preserving requested order and duplicate indices.
• Updated split_traj() to split frames based on actual ITEM: TIMESTEP boundaries rather than assuming a fixed block length, and added unit tests covering ordering/duplicates, scalar index selection, early stopping on file-like objects, and invalid inputs.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
dpdata/formats/lammps/dump.py	Adds streaming frame iteration, f_idx normalization/validation, sparse-loading logic, and robust frame splitting by timestep markers.
dpdata/plugins/lammps.py	Plumbs f_idx through the LAMMPS dump format plugin API and documents the new selection behavior/constraints.
tests/test_lammps_dump_skipload.py	Adds test coverage for f_idx semantics (order, duplicates, scalar index), validation, and early-stop behavior on file-like inputs.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[coderabbitai]

📝 Walkthrough

Walkthrough

LAMMPS dump loading now accepts specific frame indices, preserves their order and duplicates, validates selection errors, and stops reading after the final target. The plugin forwards f_idx, while tests cover selection, validation, and stream consumption.

Changes

LAMMPS frame selection

Layer / File(s)	Summary
Selective dump frame loading dpdata/formats/lammps/dump.py	load_file now validates and loads requested frame indices through streamed frame iteration, while retaining begin/step sampling and simplifying split_traj.
Plugin frame-selection wiring dpdata/plugins/lammps.py	LAMMPSDumpFormat.from_system exposes f_idx, documents its constraints, and forwards it to the dump loader.
Frame-selection validation coverage tests/test_lammps_dump_skipload.py	Tests verify integer and array selections, ordering, duplicates, early stopping, invalid indices, and incompatibility with begin.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant LAMMPSDumpFormat
  participant load_file
  participant split_traj
  LAMMPSDumpFormat->>load_file: request f_idx
  load_file->>split_traj: iterate dump frame lines
  split_traj-->>load_file: frame slices
  load_file-->>LAMMPSDumpFormat: selected frames

Loading

Suggested reviewers: copilot

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main change: efficient selective loading of LAMMPS dump frames.
Linked Issues check	✅ Passed	The PR adds f_idx-based sparse frame loading, preserves order/duplicates, and stops scanning after the last requested frame as requested in #367.
Out of Scope Changes check	✅ Passed	The changes stay focused on LAMMPS dump frame selection and its tests; no unrelated functionality was introduced.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (1)

tests/test_lammps_dump_skipload.py (1)

40-97: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Good coverage; consider adding a test for the TypeError path.

All validation paths (empty, negative, out-of-range, begin/step conflict) are covered, but the TypeError branch in _normalize_frame_indices (non-integer element, e.g. f_idx=[1.5] or f_idx="x") has no test.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/test_lammps_dump_skipload.py` around lines 40 - 97, Extend
test_invalid_frame_indices to cover the TypeError branch in
_normalize_frame_indices by passing a non-integer frame index, such as
f_idx=[1.5] or f_idx="x", and assert that TypeError is raised. Match the
expected validation message if one is defined.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/test_lammps_dump_skipload.py`:
- Around line 40-97: Extend test_invalid_frame_indices to cover the TypeError
branch in _normalize_frame_indices by passing a non-integer frame index, such as
f_idx=[1.5] or f_idx="x", and assert that TypeError is raised. Match the
expected validation message if one is defined.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 131e18d8-66e6-4c6a-ba86-c7e006b75047

📥 Commits

Reviewing files that changed from the base of the PR and between 212ab49 and 376317d.

📒 Files selected for processing (3)

• dpdata/formats/lammps/dump.py
• dpdata/plugins/lammps.py
• tests/test_lammps_dump_skipload.py

[codecov]

Codecov Report

❌ Patch coverage is 92.18750% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.62%. Comparing base (212ab49) to head (376317d).

Files with missing lines	Patch %	Lines
dpdata/formats/lammps/dump.py	92.06%	5 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1039      +/-   ##
==========================================
- Coverage   87.63%   87.62%   -0.01%     
==========================================
  Files          90       90              
  Lines        9209     9247      +38     
==========================================
+ Hits         8070     8103      +33     
- Misses       1139     1144       +5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.