check: detect missing packs referenced by the index (#9898) - #10069
Open
mr-raj12 wants to merge 1 commit into
Open
check: detect missing packs referenced by the index (#9898)#10069mr-raj12 wants to merge 1 commit into
mr-raj12 wants to merge 1 commit into
Conversation
With an intact index, load the chunk index and report any referenced pack id that is absent from packs/. Such packs are counted as errors so the check fails. Skipped for a corrupt or invalid index, which is rebuilt from the packs on next use.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #10069 +/- ##
=======================================
Coverage 86.77% 86.78%
=======================================
Files 98 98
Lines 17109 17124 +15
Branches 2591 2595 +4
=======================================
+ Hits 14847 14861 +14
Misses 1570 1570
- Partials 692 693 +1 ☔ View full report in Codecov by Harness. |
ThomasWaldmann
requested changes
Aug 10, 2026
Comment on lines
+1045
to
1046
| if index_invalid: | ||
| logger.warning("chunk index is invalid (interrupted operation); it will be rebuilt on next use.") |
Member
There was a problem hiding this comment.
does it makes sense to continue after this warning?
| logger.info(summary) | ||
| if missing_pack_ids: | ||
| # one id per line (the list can be long). | ||
| logger.error(f"Found {len(missing_pack_ids)} missing pack(s) referenced by the index:") |
Member
There was a problem hiding this comment.
"Found 42 missing packs ..." can be misunderstood.
Use the same msg as above: "{len(missing_pack_ids)} pack(s) referenced by the index are missing."
Member
There was a problem hiding this comment.
Also think about the opposite case:
Could it be that we have packs that should be in the index, but aren't?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #9898.
A read-only check verified pack and index integrity separately but never compared them, so a pack that the chunk index still referenced while missing from packs/ passed clean. The loss only showed up later, on extract.
With an intact index, check now loads the chunk index, collects the pack ids it references, and reports any absent from packs/ as errors, failing the check. It is skipped for a corrupt or invalid index, which gets rebuilt from the packs on next use.
Tests cover the detection, the clean case, and the invalid-index skip.