Skip to content

vLLM 0.11.0 — PoC at vLLM 0.11.1 #628

Closed @tcharchian opened 2026-01-23 19:26 UTC 4 comments Updated 2026-03-21 01:59 UTC

Axel-t, together with @tamazgadaev is implementing an integrated PoC in vLLM using layer hooks, benchmarking it extensively from all angles, and investigating why H200 and B200 do not show meaningful performance differences. The current version is here (Ilya’s refinement tab): https://docs.google.com/document/d/1rLNPxsIvMN7fb8AmMaRIKkhakSlN35y3A_CS9quEK8Q/edit?tab=t.0#heading=h.ti9fqvdrp0cr. Overall, @tamazgadaev thinks it looks solid, but @tamazgadaev would make another iteration for hardware experimentation - Implementation (2–3 weeks) - Benchmarking (1–2 weeks) Total: 3–5 weeks.

💬 Comments (4)

@tcharchian commented 2026-01-23 19:26 UTC

work in progress by Axel-t

@tcharchian commented 2026-01-24 01:59 UTC

@tamazgadaev please review task description and edit as needed

@tamazgadaev commented 2026-01-26 17:48 UTC

Confirmed

@Red-Caesar commented 2026-03-18 08:31 UTC

Here is the report: https://github.com/gonka-ai/gonka/pull/912

🔄 Auto-synced from Issue #628 every hour.