Ollama RC Update: Qwen3.5 and Streaming Fixes
Ollama v0.32.6-rc0 is out. It brings updates for Apple GPU users and fixes streaming issues.
New Features for Apple Users: The MLX engine now uses the MTP head for speculative decoding with Qwen3.5. This process predicts tokens ahead of time. The main model verifies them in batches. This reduces waiting time during decoding.
Ollama states this makes Qwen3.5 faster. They did not provide specific benchmarks or speed percentages in the notes. Do not assume a specific speed increase without your own tests.
Streaming Compatibility: The update changes how the /v1/chat/completions endpoint handles streaming. If your software relies on specific chunks or finish_reason data, test it first. Predictable behavior is as important as speed for production workflows.
Important Warnings:
- This is a release candidate. It is not a stable version.
- Experimental image generation is temporarily gone.
- Stay on version 0.32.5 if you need image generation.
How to proceed:
- Test this now if you use an Apple GPU and want better Qwen3.5 performance.
- Wait for a stable release if you need image generation or high stability.
- Measure prefill and decode speeds separately. MTP helps decoding but may not change total latency for every task.
Optional learning community: https://t.me/GyaanSetuAi