NVIDIA Unveils Nemotron-3.5-Lightning: A High-Throughput Hybrid MoE Model

Nemotron-3.5-Lightning-30B-A3B-NVFP4 this week, and according to The Futurum Group, it's a 30-billion-parameter hybrid MoE that delivers roughly 4× the throughput at equivalent intelligence versus comparable models in its class — clearing agentic benchmark workloads up to 30% faster than open peers. The release ships with open weights, post-training datasets, and training recipes, and it's cleared for commercial use.

What's in the box

NVIDIA trained it with an NVFP4 recipe; the model is distilled from Nemotron 3 Ultra with contributions from the Nemotron Coalition. Multi-token prediction, a latent MoE design that doubles the effective expert count at inference, and speculative decoding round out the architecture.

On NVIDIA's cited agentic benchmark, Lightning completes 10,000 tasks in roughly 17 H100 GPU-hours. Qwen3.6-35B lands at ~24 hours and trails on accuracy; Gemma 4 26B comes in at ~25 hours and also trails. Lightning matches Qwen on accuracy while finishing faster.

The routing layer is the strategic play

NVIDIA paired the model with NeMo Switchyard — an open-source routing library that picks which model handles each step of an agentic workflow. Switchyard is embedded inside gateways developers already run; NVIDIA worked with OpenRouter, LiteLLM, and Kong on integration. The framing: most agentic calls don't need frontier reasoning, and the routing decision is where the real cost sits.

Intelligence benchmarks put Lightning at 24 on the Artificial Analysis composite — below Claude Opus 5 at 63 and slightly under Nemotron 3 Super at 26. The trade is throughput and footprint: Lightning runs locally on Jetson, GeForce RTX, DGX Spark, and DGX Station rather than through a cloud API call per task.

Sanity checklist before we wire it in

• Hardware fit. Confirm NVFP4 checkpoints match your stack. A 3B-active footprint means workstation-class inference is realistic — no cluster required for a smoke test.
• Routing policy. If you're already running OpenRouter, LiteLLM, or Kong, benchmark Switchyard against your current default. The routing layer shapes latency as much as the underlying model does.
• Distillation pipeline. NVIDIA reports roughly six weeks from Ultra to this Nano-scale footprint, including evaluation. If your team runs its own distillation recipes, compare quality-per-dollar rather than just benchmark scores.

Early-access numbers worth pressure-testing on your own traffic: CrowdStrike lifted benign recall by 45 points at roughly one-fifth the cost of a customized Nemotron 3 Super; CodeRabbit improved its coding router by 34 points and built the agent for $85 in two hours on Baseten via NeMo AutoModel.

For engineers who tend to watch live signals across too many dashboards at once, these free tracking tools are a lightweight layer worth bookmarking while long training jobs run.