[Presets] Add PD Disaggregation Support

[Bihan]

Preset Used :

type: preset
name: pd-preset-internode-l40s3

repo: meta-llama/Llama-3.2-3B-Instruct

trials: 5

min_context_length: 32768
max_ttft: 1000

concurrency: 16
input_tokens: 10000
output_tokens: 512

env:
  - HF_TOKEN

prompt: |
  Rules for every trial:
  - PD disaggregation only, and only INTER-NODE: prefill, decode, and
    router should be three different nodes.
  - The router always runs on a dedicated CPU instance node (no GPU). Never
    colocate it on a decode or prefill GPU node.
  - Use L40S GPUs for workers.Use all 4 L40S across prefill+decode. Do not idle a GPU.
  - Before choosing that P:D ratio, compute a spec-sheet roofline. Mention 
    in log why the ratio is chosen.

Summary of Trial Logs for above preset

1. The agent was asked to justify its P:D ratio before each trial. Its own calculation said the right ratio depends on how efficiently prefill runs — and that it had to measure this first. It then picked a ratio without measuring: it planned 2P + 2D, but ran 3P + 1D in trial 1, claiming only one machine had enough memory to act as a decode node. In fact three of the four did. Trial 2 went back to 2P:2D, and got 37% more throughput.
   Conclusion: We need to teach the agent how to plan a PD ratio from the workload and constraints.

2. Correctly identified that KV transfer between prefill and decode as the bottleneck and then tuned UCX over TCP. Untuned UCX-over-TCP sent small pieces of KV. After tuning it used bigger TCP chunks, a more direct copy (put_zcopy), and several sends at once. It changed the settings via ENV variables and it doubled the throughput.

3. To meet the TTFT cap, the agent switched the service router from cache_aware to round_robin because cache_aware tokenizes each prompt on the router. That extra work happens before the first token, so it inflates TTFT; round-robin skips it.
   Conclusion: The agent did not log cache_aware is anyway a poor fit here because shared_prefix_tokens=0. We need to teach the agent to justify changes in routing policy. The justification should be based on workload and constraints.