Ollama 0.32.14 drops GPU support for RTX 30-series cards by omitting the sm_86 CUDA kernels, so the runtime silently falls back to the CPU and model generation slows to a crawl.

What changed in 0.32.14

The new binary is built for compute architectures 7.5, 8.9, 10.0 and 12.0 only. Architecture 8.6 – the code name for NVIDIA’s RTX 30-series, the A40 and the A6000 – is missing. When the launcher looks for a matching kernel it finds none, reports a GPU “split” in ollama ps, and then proceeds without acceleration.

Why the old fallback no longer works

Earlier releases shipped a secondary path that, if the primary kernels failed, loaded the CUDA 12 library. That library still contained code for sm_86, letting the same hardware run the model. In 0.32.14 the fallback code was unintentionally removed, so the launcher skips the GPU entirely and runs on the host processor.

Who feels the impact

Anyone using an RTX 3080, 3080 Ti, 3090, 3090 Ti, the A40, the A6000 or any other card based on compute capability 8.6 will see the slowdown. The change is invisible – no error message, no crash – just a noticeable drop in throughput.

How to verify you’re on the CPU

  1. Start a generation and, in another terminal, run nvidia-smi. If the “Memory-Used” column stays at 0 MiB, the work is on the CPU.
  2. Check the Ollama logs for a line containing library=CUDA compute=8.6. Its absence confirms the fallback never fired.
  3. Compare token-per-second numbers against a known GPU baseline; a large model that takes minutes on earlier releases will now take tens of minutes.

Setting environment variables such as CUDA_VISIBLE_DEVICES does not repair the problem because the missing kernels are a compile-time omission, not a runtime flag.

Quick fix: pin to 0.32.13

Windows

  • Uninstall the current Ollama installation.
  • Download the 0.32.13 installer from the project’s GitHub releases page.
  • Run the installer and restart the Ollama service.

Linux

sudo systemctl stop ollama
# replace <package> with the 0.32.13 .deb or .rpm you downloaded
sudo dpkg -i <package>   # for Debian-based
# or sudo rpm -Uvh <package>   # for RPM-based
sudo systemctl start ollama

After the downgrade, repeat the nvidia-smi check; you should see non-zero VRAM usage and a jump in generation speed.

What to watch in future releases

The omission of sm_86 appears to be an oversight in the build script rather than a deliberate deprecation. Until the maintainers restore the missing kernels or re-enable the CUDA 12 fallback, any upgrade beyond 0.32.13 carries the same risk. Keep an eye on the project’s issue tracker for a patch that re-adds the 8.6 kernels, and test GPU usage immediately after each update.

Bottom line: the 0.32.14 release unintentionally disables RTX 30-series acceleration. Verify GPU activity with nvidia-smi, and if you rely on those cards, roll back to 0.32.13 until the missing kernels are restored.