[LMCache] Add LMCache configs for dsv4 vllm b200/b300 agentix setups

[ApostaC]

Summary

• Adding LMCache configurations for dsv4-vllm launching script
• Adding 2 nvidia configs with lmcache agentix

[claude]

Additional findings (outside current diff — PR may have been updated during review):

• 🟡 perf-changelog.yaml:4720-4727 — PR title and description are English-only, but AGENTS.md requires bilingual PR metadata: title format <English title> / <中文标题> and a ## 中文说明 section in the body mirroring the English summary. Please update the title (e.g. append / [LMCache] 为 dsv4 vllm b200/b300 agentix 配置添加 LMCache) and add a ## 中文说明 section to the description before merge.

  Extended reasoning...

  AGENTS.md line 7 states: "PR and GitHub-issue titles & descriptions must be bilingual — include a Simplified Chinese version in addition to English. Title format: ' / <中文标题>'. In the PR/issue body, follow the English content with its Chinese translation (e.g. a '## 中文说明' section mirroring the summary)." This is a repo-wide convention that applies to every PR.

  The PR under review is missing both required pieces:

  • Title: [LMCache] Add LMCache configs for dsv4 vllm b200/b300 agentix setups — English only, no / <中文标题> suffix.
  • Body: two English bullets under # Summary ("Adding LMCache configurations for dsv4-vllm launching script" / "Adding 2 nvidia configs with lmcache agentix") with no ## 中文说明 section following them.

  Step-by-step proof of the violation:

  1. Open AGENTS.md at line 7 — the bilingual rule is explicit and unambiguous, with the exact title format and the ## 中文说明 section pattern spelled out.
  2. Read the PR title as rendered on GitHub — it contains only ASCII characters and no / separator introducing a Chinese portion.
  3. Read the PR body — the entire description is 3 lines of English (# Summary header plus two bullets); grep for 中文说明 in the body returns nothing.
  4. Therefore both required bilingual elements are absent, directly violating the rule at AGENTS.md:7.

  Impact: This is metadata-only — the benchmark scripts and configs themselves are unaffected, CI will still run, and no runtime behavior changes. But the repo convention exists so that non-English speakers on the team can review and search PR history, and every merged PR sets precedent. The rule is stated as a requirement, not a suggestion.

  How to fix (no code change needed):

  1. Edit the PR title to append a Chinese translation after /, e.g. [LMCache] Add LMCache configs for dsv4 vllm b200/b300 agentix setups / [LMCache] 为 dsv4 vllm b200/b300 agentix 配置添加 LMCache.
  2. Add a ## 中文说明 section to the PR body after the English summary, mirroring the two bullets — for example:

     ## 中文说明
     - 为 dsv4-vllm 启动脚本添加 LMCache 配置
     - 新增 2 个使用 LMCache agentix 的 NVIDIA 配置
     

  Severity is nit — this is a documentation/metadata policy issue, easy to fix in the PR description with no code impact, but should be addressed before merge since AGENTS.md declares it a requirement.

[claude]

🔴 The new --kv-transfer-config JSON for the lmcache arm omits the top-level "kv_connector_module_path":"lmcache.integration.vllm.lmcache_mp_connector" key. Both other in-tree LMCacheMPConnector call-sites include it (benchmarks/single_node/agentic/kimik2.5_fp4_b200.sh:150, benchmarks/single_node/agentic/dsv4_fp4_mi355x_vllm.sh:350); vLLMs KVConnectorFactory pre-registers built-in connectors like MooncakeStoreConnector by bare name but not the third-party LMCacheMPConnector, so vllm serve will fail to resolve the class at engine startup on every point of both the new B200 and B300 -lmcache configs. The same fix is needed at the equivalent OFFLOAD_ARGS block in dsv4_fp4_b300_vllm.sh.

Extended reasoning...

What the bug is

In the new lmcache branch of the KV_OFFLOAD_BACKEND case, the --kv-transfer-config JSON is built as:

{
  "kv_connector": "LMCacheMPConnector",
  "kv_role": "kv_both",
  "kv_connector_extra_config": {
    "lmcache.mp.host": "...",
    "lmcache.mp.port": ...,
    "lmcache.mp.mq_timeout": ...
  }
}

It is missing the top-level "kv_connector_module_path": "lmcache.integration.vllm.lmcache_mp_connector" key.

Why the existing in-tree pattern says this is required

Every other in-tree call-site of LMCacheMPConnector includes that module path explicitly:

• benchmarks/single_node/agentic/kimik2.5_fp4_b200.sh:150 (vLLM v0.22.0)
• benchmarks/single_node/agentic/dsv4_fp4_mi355x_vllm.sh:350 (AMD lmcache path)

Those two sites were authored by independent PRs and both pass the module path. The consistency of that pattern is strong evidence the key is load-bearing, not decorative.

By contrast, MooncakeStoreConnector is passed with a bare name across every in-tree site (including the sibling mooncake) branch in these same two scripts) with no module path — because Mooncake is pre-registered in vLLMs built-in KVConnectorFactory registry. LMCacheMPConnector is a third-party connector shipped by the lmcache PyPI package at lmcache.integration.vllm.lmcache_mp_connector; vLLM has no built-in entry for it, so KVConnectorFactory needs the module path to import the class.

Why the pre-existing import check does not save us

The script does run python3 -c "import lmcache.integration.vllm.lmcache_mp_connector" on the way up, but that runs in a separate subprocess and does not seed the vllm serve engine subprocess registry. Even if LMCache 0.5.1 ships entry-point auto-registration for vLLM 0.23.0, both other in-tree sites — one of them on the same LMCache generation — chose not to rely on it, so it is at best not guaranteed on this stack.

Impact and step-by-step failure

1. Sweep drives a point with kv-offload-backend: lmcache in either dsv4-fp4-b200-vllm-agentic-lmcache or dsv4-fp4-b300-vllm-agentic-lmcache.
2. The lmcache) branch runs lmcache server in the background and the healthcheck passes (this is the LMCache process, not vLLM).
3. vllm serve starts with --kv-transfer-config '{"kv_connector":"LMCacheMPConnector",...}' (no module path).
4. Inside the engine, KVConnectorFactory.create_connector(...) looks up the bare name "LMCacheMPConnector", does not find it in the built-in registry, has no kv_connector_module_path to import from, and raises — the engine dies at startup, SERVER_PID exits, wait_for_server_ready fails, and every point in the new -lmcache configs never produces results.

Because both dsv4-fp4-b200-vllm-agentic-lmcache and dsv4-fp4-b300-vllm-agentic-lmcache are 100% kv-offload-backend: lmcache, this blocks the PRs entire core purpose.

Fix

Add the module path in the JSON built at benchmarks/single_node/agentic/dsv4_fp4_b200_vllm.sh around line 251 and the equivalent block in benchmarks/single_node/agentic/dsv4_fp4_b300_vllm.sh around line 253:

OFFLOAD_ARGS=(
    --kv-transfer-config
    "{\"kv_connector\":\"LMCacheMPConnector\",\"kv_connector_module_path\":\"lmcache.integration.vllm.lmcache_mp_connector\",\"kv_role\":\"kv_both\",\"kv_connector_extra_config\":{\"lmcache.mp.host\":\"$LMCACHE_CONNECT_HOST\",\"lmcache.mp.port\":$LMCACHE_PORT,\"lmcache.mp.mq_timeout\":$LMCACHE_MQ_TIMEOUT}}"
)

This mirrors the pattern already established at kimik2.5_fp4_b200.sh:150 and dsv4_fp4_mi355x_vllm.sh:350.

[ApostaC]

No we don't need this in newer versions of vLLM

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29109297629
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=29109297629

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29109420492
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=29109420492

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29116938835
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=29116938835

[functionstackx]

generally we don't allow reinstalling kernel libraries

[functionstackx]

thanks for the contribution @ApostaC

can u please update recipes.vllm.ai with ur new configs? https://github.com/vllm-project/recipes

vllm recipes doesn't currently support offloading tho the inferact guys said they are open to contributions to support mooncake store, lmcache, vllm native offloading support in vllm recipes

as in sglang cookbook recipes , they also have support for cpu offloading recipes