Motivation
TieringSourceEnumerator.handleSplitRequest() synchronously waits for a heartbeat response when the pending split queue is empty. It then generates splits using blocking metadata calls. Periodic polling makes the heartbeat asynchronous, but split generation still runs in the coordinator callback.
The relevant code path is:
handleSplitRequest
-> requestTieringTableSplitsViaHeartBeat
-> waitHeartbeatResponse
-> generateAndAssignSplits
-> generateTieringSplits
-> getTableInfo(...).get()
-> TieringSplitGenerator.generateTableSplits(...)
This is a source-code call path, not a captured thread dump. Slow metadata requests can delay reader requests, source events, and split assignment on the coordinator thread. Partitioned tables also query offsets and snapshots partition by partition.
Solution
Run table claiming and split metadata generation in a bounded asynchronous workflow. Keep enumerator state changes and split assignment on the coordinator thread, coalesce concurrent requests, and reject stale results after failover or close.
Test with a delayed metadata response: coordinator events should remain responsive, and a result arriving after failover must not be assigned.
Anything else?
Related: #3261 reported a hanging offsets request. This proposal removes blocking metadata work from the coordinator thread, including requests that eventually complete.
Willingness to contribute
Motivation
TieringSourceEnumerator.handleSplitRequest()synchronously waits for a heartbeat response when the pending split queue is empty. It then generates splits using blocking metadata calls. Periodic polling makes the heartbeat asynchronous, but split generation still runs in the coordinator callback.The relevant code path is:
This is a source-code call path, not a captured thread dump. Slow metadata requests can delay reader requests, source events, and split assignment on the coordinator thread. Partitioned tables also query offsets and snapshots partition by partition.
Solution
Run table claiming and split metadata generation in a bounded asynchronous workflow. Keep enumerator state changes and split assignment on the coordinator thread, coalesce concurrent requests, and reject stale results after failover or close.
Test with a delayed metadata response: coordinator events should remain responsive, and a result arriving after failover must not be assigned.
Anything else?
Related: #3261 reported a hanging offsets request. This proposal removes blocking metadata work from the coordinator thread, including requests that eventually complete.
Willingness to contribute