Skip to content

Commit eaf0900

Browse files
fix: revert change to SubStats population in error reports
doesn't line up properly with other rejection stats
1 parent 7792d41 commit eaf0900

3 files changed

Lines changed: 8 additions & 12 deletions

File tree

src/dve/pipeline/pipeline.py

Lines changed: 5 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -894,18 +894,14 @@ def error_report(
894894
.agg(pl.col("Count").sum()) # type: ignore
895895
.iter_rows(named=True)
896896
}
897-
submission_rejections = err_types.get(
898-
ErrorReportCategories.FILE_REJECTION.reporting_name, 0
899-
)
900897
sub_stats = SubmissionStatisticsRecord(
901898
submission_id=submission_info.submission_id,
902899
record_count=submission_status.number_of_records,
903-
number_submission_rejections=submission_rejections,
904-
number_record_rejections=(
905-
submission_status.number_of_records
906-
if submission_rejections > 0 else err_types.get( # type: ignore
907-
ErrorReportCategories.RECORD_REJECTION.reporting_name, 0
908-
)
900+
number_submission_rejections=err_types.get(
901+
ErrorReportCategories.FILE_REJECTION.reporting_name, 0
902+
),
903+
number_record_rejections=err_types.get(
904+
ErrorReportCategories.RECORD_REJECTION.reporting_name, 0
909905
),
910906
number_warnings=err_types.get(ErrorReportCategories.WARNING.reporting_name, 0),
911907
)

tests/features/animals.feature

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,5 +55,5 @@ Feature: Pipeline tests using the animal dataset
5555
| parameter | value |
5656
| record_count | 7 |
5757
| number_submission_rejections | 1 |
58-
| number_record_rejections | 7 |
58+
| number_record_rejections | 2 |
5959
| number_warnings | 1 |

tests/features/movies.feature

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ Feature: Pipeline tests using the movies dataset
4242
| parameter | value |
4343
| record_count | 5 |
4444
| number_submission_rejections | 1 |
45-
| number_record_rejections | 5 |
45+
| number_record_rejections | 3 |
4646
| number_warnings | 2 |
4747
And the error aggregates are persisted
4848

@@ -81,7 +81,7 @@ Feature: Pipeline tests using the movies dataset
8181
| parameter | value |
8282
| record_count | 5 |
8383
| number_submission_rejections | 1 |
84-
| number_record_rejections | 5 |
84+
| number_record_rejections | 3 |
8585
| number_warnings | 2 |
8686
And the error aggregates are persisted
8787

0 commit comments

Comments
 (0)