Hi Pytorch team,
The article PyTorch + vLLM =
– PyTorch shows 1.05x–1.9x speedups with torch.compile for models like Llama4/Qwen3/Gemma3. To better understand:
- What benchmark setup (hardware, input shape) was used?
- Was this measured by comparing vLLM with vs. without
torch.compileenabled? - Could you share a reference script/command for reproducing these tests?
Thanks for your insights! This will greatly help the community validate the optimizations.