[server] Handle duplicate tiering completion reports idempotently

[fhan688]

Purpose

Linked issue: close #4406

Tiering completion reports are delivered with at-least-once semantics and may overlap while an earlier heartbeat is still in flight. A repeated normal completion for the same table and tiering epoch currently passes epoch validation, overwrites the previously recorded tiering result, and then attempts two invalid state transitions.

This change makes duplicate normal completion reports idempotent and ensures that completion eligibility is validated before tiering statistics are modified.

Brief change log

• Treat a repeated normal completion for an already completed, non-zero tiering epoch as a successful no-op.
• Preserve table-existence and tiering-epoch validation before duplicate detection.
• Require non-duplicate completion reports to be in the Tiering state before updating tiering statistics.
• Preserve stale-epoch fencing after the next tiering round starts.
• Preserve the existing forced-completion behavior, including immediate transition to the next epoch.
• Add regression coverage for:
  • duplicate normal completion reports;
  • unchanged completion timestamp, duration, file size, and record count;
  • unchanged scheduling of the next tiering round;
  • rejection of stale completion reports after the next round starts;
  • rejection of completion reports from ineligible states;
  • fencing of repeated forced-completion reports.

Tests

• LakeTableTieringManagerTest
  • 11 tests passed with no failures or errors.
• Spotless check passed.
• Checkstyle passed.
• Apache RAT check passed.

Test command:

mvn -pl fluss-server -am \
  -DskipITs \
  -Dcheckstyle.skip=true \
  -DfailIfNoTests=false \
  -Dsurefire.failIfNoSpecifiedTests=false \
  -Dtest=LakeTableTieringManagerTest \
  test

API and Format

No public API, RPC protocol, or storage format changes.

Documentation

No documentation changes are required because this is an internal server-side correctness fix.

[beryllw]

LGTM!

[luoyuxia]

@fhan688 Thanks for the pr. I left minor comments

[luoyuxia]

Suggested change

	if (!isForceFinished
	if (tieringState == TieringState.Scheduled) {
	// Completion reports are delivered at least once. Overlapping heartbeats may copy the
	// same finishedTables entry before the first response removes it, and a lost response
	// may also cause the report to be retried. The first report has already moved the table
	// to Scheduled, so ignore subsequent reports to preserve the recorded statistics and
	// avoid invalid state transitions.
	LOG.debug(
	"Ignore the duplicate tiering completion report for table {} at epoch {} "
	+ "because the table is already in Scheduled state.",
	tableId,
	tieredEpoch);
	return;
	}

[luoyuxia]

I think we can remove this

[luoyuxia]

Could we simplify the regression coverage by extending the existing testFinishTableTieringReTriggerSchedule above instead of adding a separate full-lifecycle test? After the first successful finish, send the same completion again with different stats, then assert that the completion timestamp, duration, file size, record count, and original scheduling remain unchanged. The existing test can still advance to the next round and verify epoch 2.

With the proposed Scheduled no-op semantics, the initial-Scheduled half of testFinishTableTieringRequiresTieringState is no longer applicable; that test can be removed or reduced to only the remaining state behavior. Stale-epoch fencing is already covered by testTieringServiceTimeOutReTriggerPending, while the short repeated force-finish assertion can stay in the force-finish test. This should keep all important behavior covered while reducing the added test code substantially.

[fhan688]

Thanks for the review, @luoyuxia. I’ve addressed all the comments:

• Simplified duplicate detection to treat the Scheduled state as an idempotent no-op, with a debug log and an explanation of the at-least-once delivery behavior.
• Removed the additional Tiering state validation to keep this PR scoped to duplicate completion handling.
• Folded the duplicate completion assertions into the existing scheduling lifecycle test and removed the redundant standalone tests.
• Kept the repeated forced-completion fencing assertion in the existing force-finish test.

LakeTableTieringManagerTest passes with all 9 tests, and Spotless, Checkstyle, and RAT checks also pass.