Nvidia ships open 30B agent model and NeMo Switchyard router to cut inference cost

Nvidia added Nemotron 3.5 Lightning to its open Nemotron 3 lineup, positioning the 30-billion-parameter mixture-of-experts model as a fast, customizable worker for high-volume agent tasks like code review, tool calls, and security-alert triage. The pitch centers on cost and control: the company claims up to 4x faster output and 30% quicker task completion versus peers, and the weights can be fine-tuned on private data and run anywhere from RTX PCs and Jetson edge devices to DGX systems, on-prem servers, and the cloud. Nvidia is also publishing training data and techniques where licensing allows, plus an agentic reinforcement-learning dataset for coding agents. Early adopters cited include CrowdStrike, Harvey, and CodeRabbit.

The more strategically interesting piece is NeMo Switchyard, an open-source routing library that sits inside existing agent stacks and sends each step of a workflow to the cheapest model that can still handle it—mixing open, proprietary, and Nvidia models without app rewrites. Nvidia frames this as an answer to the two failure modes of single-model deployments: overpaying by defaulting to a frontier model, or losing quality by underpowering. Partner benchmarks are notably aggressive, including LangChain reporting 74% lower cost by routing just 7% of calls to a frontier model (at a 6% accuracy hit), Ramp cutting costs 58%, and Nvidia’s own figure of task cost dropping to roughly a third of running Opus 4.8 alone.

The underlying bet is that production agents are becoming ensembles of specialized models rather than one large model, and that whoever controls the routing layer captures the economics. By open-sourcing both a competitive small model and the router, and by integrating with Kong, LiteLLM, LangChain, and Cognition’s Devin, Nvidia is trying to make its hardware and models the default substrate for that ensemble architecture. Nemotron 3.5 Lightning is on Hugging Face, ModelScope, OpenRouter, and build.nvidia.com; Switchyard is on GitHub.