logprobs, top_logprobs conditional stripping #1264
enhancement
The gateway forces logprobs upstream for validation, but clients who never asked for logprobs should not see them in the response (OpenAI-compatible default).
Clients who explicitly set logprobs: true or top_logprobs should get them back.
Now we always strip them even if client asked this fields in request
Recommendation: Adopt conditional stripping for logprobs and top_logprobs on the client-facing proxy path. Keep unconditional stripping for token_ids, prompt_token_ids, and prompt_logprobs.
💬 Comments (1)
🔄 Auto-synced from Issue #1264 every hour.
Merged to v4 https://github.com/gonka-ai/gonka/commit/4a8eb85af061c94e8f572d42bb5ee5f1c267fa2f