Skip to content

Decode PoC: integration support #1688

Open @tcharchian opened 2026-08-31 20:15 UTC 6 comments Updated 2026-09-17 14:28 UTC

Integration of Decode PoC with the current chain and MLNode path. Design and seeding questions stay on this issue. Verification of the current MiniMax implementation on a full image is #1689. DeepSeek thresholds and the DeepSeek seeding scheme are #1690.

Integration — Current Decode PoC path is written up (what is in production shape, what is still experimental) — Integration points with MLNode and the epoch cycle are listed — Open questions on the Decode PoC design are listed in notes on this issue, not only off-thread

DeepSeek expert seeding DeepSeek seeding is more complex than MiniMax. This issue covers the integration discussion; the scheme fix itself is #1690. — Expert seeding for DeepSeek is described (how experts are chosen, what the current scheme does) — Weight distribution across seeded experts is measured — A closer-to-uniform distribution across seeded experts is proposed, with notes if uniform is not possible

Experiments (when the checklist on this issue is enough to run them) - [ ] Нardware used is written up - [ ] Models used are written up - [ ] What passed / what did not is written up, with links to notebooks or logs

💬 Comments (6)

@baychak commented 2026-08-31 20:26 UTC

TY!

@clanster commented 2026-08-31 20:26 UTC

ty

@baychak commented 2026-09-04 20:07 UTC

Status: in progress. The code lives on our forks as a residual/plugin pair; the design questions were still off-thread, and this comment moves them here.

Where the work is

branch on our fork PR lands in
residual kaitakuai/vllm : mixed-poc-vllm-0.25.1-dev gonka-ai/vllm#100 release/v0.25.1-decode-int
plugin kaitakuai/gonka-vllm-plugins : mixed-poc-vllm-0.25.1-dev gonka-ai/gonka-vllm-plugins#8 decode-poc-int

Development is easier on the forks, so both intake branches @vbgd0 created are still where they were on 2026-08-13.

Since last

  • @vbgd0 has been pushing directly into both PR branches: 6 commits on the residual, 14 on the plugin, latest 2026-09-02. Not only cleanup — the PoC artifact payload gained n_nan_steps and the max mismatch margin, and decode validation gained a nonce-level snap-margin stat test.
  • gonka-ai/vllm#100 points at exactly our working residual branch — no divergence. gonka-ai/gonka-vllm-plugins#8 is 41 commits behind our working plugin branch; nothing of his work is lost, ours is a strict superset. We will refresh it.
  • Zero review comments on either PR.

Open design questions

  1. Backward compatibility — keep the old prefill PoC untouched behind a flag, or accept that artifacts stop being cross-verifiable.
  2. How DeepSeek seeding should treat the hash-routed layers.
  3. Sign-off on the new consensus constants — @axeltec-gonka owns them; not given.

Next: refresh gonka-ai/gonka-vllm-plugins#8 onto the current plugin branch — update here by 2026-09-05.

@baychak commented 2026-09-07 18:54 UTC

Status: in progress.

Since last

  • gonka-ai/gonka-vllm-plugins#8 refreshed on 2026-09-06 to the current plugin branch (a346bb7, 108 commits); gonka-ai/vllm#100 unchanged. Promised for 09-05, done 09-06.
  • 2026-09-05, on @vbgd0's advice: the admission layer is removed rather than patched. The scheduler makes one PoC step call; nonce pacing moved to the client. Golden checks on B300 reproduce after the change.
  • PRs for that on our forks, sent to @vbgd0 for review: kaitakuai/gonka-vllm-plugins#4 and kaitakuai/vllm#22. They retarget onto the intake branches once #8 and #100 land.
  • Open design questions 1–3 unchanged.

Next: review of the two fork PRs by @vbgd0; merge order is his call — #8/#100 first and then #4/#22, or folded. Update here by 2026-09-14.

@baychak commented 2026-09-11 20:21 UTC

Status: in progress.

Since last

  • @vbgd0 opened #1743 — the chain picks the PoC scheme per model, built on the branches behind gonka-ai/gonka-vllm-plugins#8 and gonka-ai/vllm#100.
  • Both are still open and unreviewed. The admission-layer removal is merged into their heads (#8 at 140 commits, #100 at 14).
  • The removal is measured on a frozen revision across 11 configurations: R spread ×1.107 on MiniMax, ×1.17 on DeepSeek.
  • 2026-09-11: @vbgd0 asked us to add decode-PoC support for GLM-5.3-Flash — the simplest shim in the plugin and nonce/min for the model, targeting release/v0.28-decode-int. That opens a second 0.28 line beside the 0.25.1 one; branches started on our forks.

Next: merge order for #8 and #100 is @vbgd0's call; we build the 0.28 decode line meanwhile. Update here 2026-09-14.

@baychak commented 2026-09-17 14:28 UTC

Status: in progress. The 09-14 update was missed.

Since last

  • decode-PoC on GLM-5.3-Flash is done and measured on all four cards; the report went up on 09-16: kaitakuai/experiments/2026-09/decode-poc-0281-glm-freeze. Nonces per minute on 8 GPUs: 2×B300 5,050; 4×B200 4,202; 4×H200 1,702; 8×H100 921.
  • The GLM line moved to the vendor upstream base: gonka-ai/vllm#112 merged by @vbgd0 on 09-16; the engine half is gonka-ai/vllm#113, the plugin half gonka-ai/gonka-vllm-plugins#12 into decode-poc-glm53, both open. #113 changes the inference-validation path (trace replay instead of enforced_token_ids) — noted here for @vbgd0 ahead of review.
  • Release shape agreed with @vbgd0 on 09-16: DeepSeek and MiniMax on 0.25.1, GLM on its own 0.28 line; 0.29 after the release. Re-measured on 0.28: MiniMax unchanged, DeepSeek +17–20 % PoC everywhere except H200 (being re-measured); B200 not measured.
  • gonka-ai/gonka-vllm-plugins#8 (refreshed 09-16 with the ports of plugins#10/#11), gonka-ai/vllm#100 and #1743 are open, no review. The external review @vbgd0 requested on 09-11 has not answered yet.

Next: smoke of #113 on hardware — boot, acceptance check, one replay validation — needs a box; then review of #113/#12 by @vbgd0. Design questions: 2 is answered by the note on #1690; 1 and 3 unchanged. Update here 2026-09-21.

🔄 Auto-synced from Issue #1688 every hour.