Nemotron 3 Nano 30B A3B, the predecessor, was already positioned as a compact alternative to larger foundation models. By switching to a hybrid Mamba-Transformer architecture and activating only 3.6 billion parameters at any moment, Lightning pushes the efficiency frontier while still delivering reasoning power comparable to much bigger models.
Why speed matters now
AI agents are moving from batch-style inference to continuous interaction. A chatbot that pauses for several seconds feels sluggish; a code-completion tool that lags behind a developer’s typing disrupts workflow. In those settings, token-per-second throughput directly translates into perceived responsiveness. The 670-token-per-second figure is roughly double the 386 tokens per second reported for Google’s Gemini 3.5 Flash-Lite, and it cuts the time to finish a standard Intelligence Index task to about half a minute. By contrast, Qwen 3.6 35B A3B needs 3.5 minutes and Gemma 4 31B needs 5.8 minutes for the same task.
That gap matters for any service that must handle many simultaneous requests. A server that processes twice as many tokens on the same hardware can either lower costs or double capacity, a clear advantage for cloud providers and enterprises.
How the model achieves its numbers
Nemotron 3.5 Lightning’s hybrid architecture blends the long-range pattern-recognition strengths of Transformers with the state-space efficiency of Mamba blocks. The design keeps the total parameter count high—31.6 billion—to retain modeling capacity, but only 3.6 billion are active for any given token. Sparse activation reduces compute per token, raising throughput without a proportional loss in quality.
Artificial Analysis measured an Intelligence Index score of 24 for Lightning, a nine-point jump from the 15 scored by the earlier Nano model. That puts it on par with OpenAI’s gpt-oss-120b, even though Lightning uses roughly a quarter of the parameters. It still trails the top models—Meta’s Muse Glimmer (35) and Qwen 3.6 35B A3B (32)—but its intelligence-to-parameter ratio ranks among the best.
Agentic benchmarks tell a similar story
Agentic workloads—planning, tool use, multi-step reasoning—are where Lightning shines. On the GDPval-AA v2 benchmark, the model earned an Elo rating of 824, outpacing the 120-billion-parameter gpt-oss-120b (800) and Nvidia’s own larger Nemotron 3 Super (698). In the Terminal-Bench v2.1 test, Lightning’s success rate rose from 7 % to 24.3 %, closing in on the 26.2 % recorded by gpt-oss-120b.
Nvidia credited post-training collaborations with partners such as CodeRabbit and Harvey for tuning the model on domain-specific tasks. Those refinements keep the model fast while staying competitive on specialized workloads.
Availability and practical considerations
The model ships under the permissive OpenMDW-1.1 license, with weights in BF16 and NVFP4 formats. The NVFP4 variant packs the model more tightly with minimal quality loss, a useful option for edge deployments or cost-conscious cloud instances. A one-million-token context window lets the model handle very long prompts without truncation, valuable for document-level analysis or extensive codebases.
Serverless inference already runs on providers such as DeepInfra, Fireworks, FriendliAI, CoreWeave, GMI Cloud, Nebius and Crusoe. Developers can start experimenting without building custom infrastructure, though true cost-effectiveness will depend on each platform’s pricing.
Takeaway: Nemotron 3.5 Lightning proves a model can match the reasoning level of 120-billion-parameter systems while delivering almost twice the token throughput of leading competitors, making it a strong candidate for latency-sensitive AI agents.