feat: gc.freeze hot processes and disable gc during cudagraph capture

[sufubao]

Python 的 gen-2 GC 会周期性全量扫描存活堆，在常驻服务进程里表现为解码时数百毫秒的卡顿。各常驻进程（model-infer worker、router、detokenization、httpserver）初始化完成后 gc.collect()+gc.freeze() 把存活对象移入 permanent generation（每进程约 50 万对象），cudagraph capture 期间临时关闭 gc，退出时恢复进入前的状态。

Qwen3.5-27B，TP2 H200，并发 32 压测 300 秒（p50 不变说明单步计算不受影响，收益全部来自消除 GC 卡顿）：

ITL	baseline	本 PR	变化
p50	10.79 ms	10.78 ms	持平
mean	12.39 ms	11.79 ms	-4.8%
p99	35.48 ms	28.92 ms	-18.5%
p99.9	111.2 ms	103.3 ms	-7.1%
max	330.9 ms	242.4 ms	-26.8%
完成请求数	8814	9152	+3.8%

Qwen3.5-4B 单卡同样条件下：max 255→110 ms，p99.9 51.3→47.2 ms，均值不变。模型越大、存活对象越多，收益越明显。

服务正常，/generate 和 /generate_stream 输出无异常。

[gemini-code-assist]

Code Review

This pull request introduces garbage collection optimization utilities to prevent periodic gen-2 GC stalls during steady-state inference and CUDA-graph capture. These optimizations are integrated across the model worker, HTTP server, detokenizer, and router processes. The feedback suggests improving the gc_frozen_and_disabled context manager to restore the pre-existing GC state instead of unconditionally enabling it, updating the corresponding unit tests, and conditionally applying the context manager during model initialization only when CUDA graph capture is actually enabled to avoid unnecessary overhead.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

The gc_frozen_and_disabled context manager unconditionally calls gc.enable() in its finally block. If automatic garbage collection was already disabled before entering this context manager, exiting it will unexpectedly enable GC. To ensure robust behavior and avoid altering the pre-existing global GC state, the context manager should check and restore the previous state of the garbage collector.

Suggested change

	def gc_frozen_and_disabled(tag: str = ""):
	"""Freeze current survivors, then disable GC for the duration — used around
	CUDA-graph capture so no collection fires mid-capture (SGLang #8577)."""
	freeze_gc(tag)
	gc.disable()
	try:
	yield
	finally:
	gc.enable()
	def gc_frozen_and_disabled(tag: str = ""):
	"""Freeze current survivors, then disable GC for the duration — used around
	CUDA-graph capture so no collection fires mid-capture (SGLang #8577)."""
	was_enabled = gc.isenabled()
	freeze_gc(tag)
	gc.disable()
	try:
	yield
	finally:
	if was_enabled:
	gc.enable()

[gemini-code-assist]

Update the test to verify that gc_frozen_and_disabled correctly restores the previous GC state when GC was already disabled before entering the context manager.

Suggested change

	def test_gc_frozen_and_disabled_toggles_and_restores():
	gc.unfreeze()
	gc.enable()
	with gc_frozen_and_disabled("test"):
	assert not gc.isenabled() # disabled during the block (e.g. cudagraph capture)
	assert gc.isenabled() # re-enabled on exit, even on error
	gc.unfreeze()
	def test_gc_frozen_and_disabled_toggles_and_restores():
	gc.unfreeze()
	# Test when GC was enabled
	gc.enable()
	with gc_frozen_and_disabled("test"):
	assert not gc.isenabled()
	assert gc.isenabled()
	# Test when GC was disabled
	gc.disable()
	with gc_frozen_and_disabled("test"):
	assert not gc.isenabled()
	assert not gc.isenabled()
	gc.enable()
	gc.unfreeze()

[sufubao]

已改为退出时恢复进入前的 gc 状态（之前是无条件 enable）。这个 helper 逻辑很简单，就不单独留测试文件了，两种状态（进入前 enabled/disabled）都手动验证过。

[gemini-code-assist]

If CUDA graph capture is disabled (both decode and prefill CUDA graphs are disabled), entering the gc_frozen_and_disabled context manager will still trigger a full garbage collection and freeze. Since gc.collect() on a large model worker process can be quite expensive, we should conditionally wrap the initialization in gc_frozen_and_disabled only when at least one CUDA graph capture is enabled.

Suggested change

	with gc_frozen_and_disabled("cudagraph-capture"):
	self._init_cudagraph()
	self._init_prefill_cuda_graph()
	if (not self.disable_cudagraph) or get_env_start_args().enable_prefill_cudagraph:
	with gc_frozen_and_disabled("cudagraph-capture"):
	self._init_cudagraph()
	self._init_prefill_cuda_graph()
	else:
	self._init_cudagraph()
	self._init_prefill_cuda_graph()