Kimi-K2.6 non-stream on gateway v4: 502 nonce_finished=false (stream OK) #1680

Closed @paranjko opened 2026-08-30 19:49 UTC 3 comments Updated 2026-09-02 02:21 UTC

bug

We ran into a Kimi-K2.6 issue on gateway v4 (mainnet-v0.2.15-v4) and wanted to flag it.

What we see. stream=true is fine. stream=false with a tiny max_tokens (64) often returns JSON. Same prompt with max_tokens 256 / 1024 / 4096 comes back 502 after ~30–75s:

inference: winner inference incomplete (nonce_finished=false)

From the client it looks like a hang (no body until the 502). On the gateway the attempt already has a lot of upstream bytes (aggregate_bytes in the 100k+ range), then the request is failed with winner_failed_after_content / race_completed winner=true finished=false content_source=delta.reasoning.

v3. Same bodies, same night, gateway v3 (mainnet-v0.2.13-v3-post2, /devshard/v3): all those non-stream cases return 200 JSON. MiniMax-M2.7 non-stream on the same v4 is also 200. So this looks like a v4 regression on Kimi non-stream, not “Kimi hosts are down” and not “non-stream is dead on the gateway”.

How we tested. POST /v1/chat/completions straight at the gateway (not through a broker proxy). Unique marker= in the user text so the cache cannot hit. temperature=0. No host pin, no redundancy settings changed for the probes.

#	model	stream	max_tokens	prompt	v3	v4
1	Kimi-K2.6	true	64	“Reply with exactly OK”	200 SSE	200 SSE
2	Kimi-K2.6	false	64	same	200 JSON	200 JSON
3	Kimi-K2.6	true	1024	same	200 SSE	200 SSE
4	Kimi-K2.6	false	1024	same	200 JSON	502 nonce_finished=false
5	Kimi-K2.6	true	256	~3 sentences	200 SSE	200 SSE
6	Kimi-K2.6	false	256	same	200 JSON	502 same
7	Kimi-K2.6	false	4096	“Reply with exactly OK”	200 JSON	502 same
8	MiniMax-M2.7	false	64	short	—	200 JSON

Repro

curl -sS -m 120 "$GATEWAY/v1/chat/completions" \
  -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{
    "model": "moonshotai/Kimi-K2.6",
    "messages": [{"role": "user", "content": "Reply with exactly OK. marker='"$RANDOM"'"}],
    "max_tokens": 1024,
    "temperature": 0,
    "stream": false
  }'

Same payload with "stream": true succeeds. On the v4 log, grep request= from gateway_request_received — for the failing case you should see gateway_cache_miss stream=false → stream_forwarding_started → send_completed with many chunks → winner_failed_after_content / nonce_finished=false.

Gonka External TestLab

---

💬 Comments (3)

@qdanik commented 2026-08-30 20:18 UTC

Temporary fix is to disable force streaming:

curl -N http://127.0.0.1:18080/v1/admin/settings \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $DEVSHARD_ADMIN_API_KEY" \
  -d '{"redundancy":{"force_upstream_streaming":false}}'

@paranjko commented 2026-08-30 23:47 UTC

Retested on mainnet-v0.2.15-v4.0.1. Same Kimi non-stream max_tokens=1024 case now returns 200 JSON. Stream still OK. Great work @qdanik !

@qdanik commented 2026-09-02 02:21 UTC

fixed and merged into gateway-v4 branch

---

🔄 Auto-synced from Issue #1680 every hour.