fix(dpmodel): preserve virtual atom types in data utilities

[njzjz-bot]

Fixes #5633

Summary

• ignore negative virtual atom sentinels when collecting observed species
• count only configured real types in LMDB natoms vectors while retaining every coordinate slot in nloc
• share one atom-type remapping helper between training and test readers so negative sentinels survive type-map changes
• retain explicit NumPy bounds errors for malformed positive atom types

Why existing tests missed this

The observed-type tests covered valid types and a positive out-of-range index, but never a negative value. That omitted Python negative-index semantics, where type_map[-1] silently aliases the final real species.

The LMDB fixtures generated only type 0 and type 1 atoms. Existing reversed and superset type-map tests therefore exercised remapping without a virtual sentinel, and natoms-vector tests never passed a negative value to np.bincount. The dpmodel/PyTorch consistency coverage also uses the same LMDB reader on both sides, so it could only confirm a shared result rather than expose this shared bug. Existing virtual-atom model tests construct virtual atoms in memory instead of serializing them through these LMDB/stat utility paths.

Validation

• source/tests/common/dpmodel/test_observed_type.py
• source/tests/common/dpmodel/test_lmdb_data.py
• 79 tests passed
• ruff format .
• ruff check .

Coding agent: Codex
Codex version: codex-cli 0.144.4
Model: gpt-5.6-sol
Reasoning effort: xhigh

Summary by CodeRabbit

• Bug Fixes
  • Improved handling of virtual atom types during LMDB data loading and type remapping.
  • Prevented invalid or out-of-range atom types from being incorrectly interpreted as valid elements.
  • Corrected per-type atom counts and observed element detection when virtual or invalid types are present.
• Tests
  • Added coverage for virtual atom types and invalid index handling in data loading and observed-type detection.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: ec6e979a-c00e-48e3-b573-84ef00523865

📥 Commits

Reviewing files that changed from the base of the PR and between 6c3b985 and a653aef.

📒 Files selected for processing (4)

• deepmd/dpmodel/utils/lmdb_data.py
• deepmd/dpmodel/utils/stat.py
• source/tests/common/dpmodel/test_lmdb_data.py
• source/tests/common/dpmodel/test_observed_type.py

---

📝 Walkthrough

Walkthrough

LMDB atom-type remapping now preserves negative virtual sentinels and validates positive indices. Atom counts and observed-type collection ignore invalid types, with regression tests covering both behaviors.

Changes

Virtual atom type handling

Layer / File(s)	Summary
LMDB remapping and counting deepmd/dpmodel/utils/lmdb_data.py, source/tests/common/dpmodel/test_lmdb_data.py	LMDB readers preserve negative atom-type sentinels, exclude invalid types from per-type counts, and test remapping and out-of-range behavior.
Observed type filtering deepmd/dpmodel/utils/stat.py, source/tests/common/dpmodel/test_observed_type.py	Observed-type collection ignores negative and out-of-range indices, with regression coverage for virtual types.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• deepmodeling/deepmd-kit#5856: Addresses related handling of negative atype values in type-embedding and descriptor gather paths.

Suggested reviewers: iprozd, outisli

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main fix: preserving virtual atom types in dpmodel data utilities.
Linked Issues check	✅ Passed	The code and tests address the linked issue by filtering negative types, preserving sentinels, and covering the required regressions.
Out of Scope Changes check	✅ Passed	The changes stay focused on the virtual atom type bug fix and related regression tests, with no clear unrelated additions.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[njzjz]

Possible reviewers based on changed lines, exact file history, and exact-file review history:

• @wanghan-iapcm — 4 commits on changed files; 4 reviews on exact changed files (deepmd/dpmodel/utils/lmdb_data.py, deepmd/dpmodel/utils/stat.py, source/tests/common/dpmodel/test_lmdb_data.py).
• @iProzd — 6 commits on changed files (deepmd/dpmodel/utils/lmdb_data.py, deepmd/dpmodel/utils/stat.py, source/tests/common/dpmodel/test_lmdb_data.py, source/tests/common/dpmodel/test_observed_type.py).

No review request was made automatically.

Coding agent: Codex
Codex version: codex-cli 0.144.4
Model: gpt-5.6-sol
Reasoning effort: xhigh

[wanghan-iapcm]

LGTM. Correctly preserves negative virtual sentinels through the type remap (previously type_remap[-1] aliased them to the last real species) and avoids the np.bincount crash on negatives, with both LMDB readers now sharing _remap_atom_types. The new tests genuinely fail on pre-fix code (virtual type aliased / included), so they're real regression tests. One minor docstring nit inline.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.62%. Comparing base (6c3b985) to head (a653aef).
⚠️ Report is 23 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5837      +/-   ##
==========================================
+ Coverage   78.58%   78.62%   +0.04%     
==========================================
  Files        1050     1054       +4     
  Lines      120637   121762    +1125     
  Branches     4356     4404      +48     
==========================================
+ Hits        94801    95738     +937     
- Misses      24278    24456     +178     
- Partials     1558     1568      +10     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.