Skip to content

fix(realtime): avoid false keepalive timeouts under load - #3533

Open
LauraGPT wants to merge 2 commits into
modelscope:mainfrom
LauraGPT:codex/realtime-ws-keepalive-3528
Open

fix(realtime): avoid false keepalive timeouts under load#3533
LauraGPT wants to merge 2 commits into
modelscope:mainfrom
LauraGPT:codex/realtime-ws-keepalive-3528

Conversation

@LauraGPT

Copy link
Copy Markdown
Collaborator

Summary

  • keep server WebSocket pings enabled every 20 seconds, but disable ping-timeout closure by default
  • make positive ping timeouts an explicit operator choice above measured worst-case decode/queue delay
  • add benchmark controls and JSONL fields for client ping interval/timeout
  • document the keepalive boundary in English and Chinese, including why ws-max-queue doesn't fix control-frame timeouts

Fixes #3528.

Root cause and controlled reproduction

Under concurrent long-audio load, synchronous VAD work and queued model decoding can delay WebSocket control-frame handling beyond 20 seconds even while a session is making valid progress. The previous ping_interval=20, ping_timeout=20 default then closed healthy connections with code 1011.

On one H100 80 GB service, using the same paced 47-second Chinese WAV and 20 concurrent clients:

Server setting Finals Errors Elapsed STOP p95
v1.4.4 default, timeout 20s 4/20 16 (keepalive ping timeout) 65.467s 18.429s among survivors
v1.4.4 explicit timeout off 20/20 0 81.879s 34.838s
this PR, default options 20/20 0 67.021s 19.970s

The timeout-off control proves that ws-max-queue isn't the failure boundary: it limits incoming data messages, while ping/pong are WebSocket control frames. The exact latency varies between runs, so the pass/fail and close-code result is the relevant comparison.

The reporter independently observed the same behavior on L20: 47-second/16-client sessions complete when server ping timeout and client pings are disabled. This PR's hardware result is H100 evidence, not a substitute for their L20 measurement.

Behavior and tradeoff

The default remains ping_interval=20, so healthy connections still exchange keepalive traffic. ping_timeout=None prevents compute backpressure from being mistaken for a dead peer. Deployments that require server-side pong expiry can set a positive --ws-ping-timeout after measuring their worst-case production queue/decode delay and coordinating with gateway idle timeouts.

Verification

  • 113 passed across realtime service, benchmark, install docs, vLLM source docs, and growth-plan tests
  • post-fix H100 default-options replay: 20 final messages, 0 client errors, no server traceback/error
  • git diff --check
  • Python bytecode compilation for changed Python modules and tests
  • signed commit with DCO trailer

Signed-off-by: LauraGPT <170200537+LauraGPT@users.noreply.github.com>
Signed-off-by: LauraGPT <170200537+LauraGPT@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1.4.3 的 funasr/bin/realtime_ws.py 的 run_session_work() 实现太简单粗暴,导致并发性能比1.3.9还差很多

2 participants