Llama-Nemotron: Efficient Reasoning Models
By: Akhiad Bercovich , Itay Levy , Izik Golan and more
Potential Business Impact:
Computers can now think and solve problems better.
We introduce the Llama-Nemotron series of models, an open family of heterogeneous reasoning models that deliver exceptional reasoning capabilities, inference efficiency, and an open license for enterprise use. The family comes in three sizes -- Nano (8B), Super (49B), and Ultra (253B) -- and performs competitively with state-of-the-art reasoning models such as DeepSeek-R1 while offering superior inference throughput and memory efficiency. In this report, we discuss the training procedure for these models, which entails using neural architecture search from Llama 3 models for accelerated inference, knowledge distillation, and continued pretraining, followed by a reasoning-focused post-training stage consisting of two main parts: supervised fine-tuning and large scale reinforcement learning. Llama-Nemotron models are the first open-source models to support a dynamic reasoning toggle, allowing users to switch between standard chat and reasoning modes during inference. To further support open research and facilitate model development, we provide the following resources: 1. We release the Llama-Nemotron reasoning models -- LN-Nano, LN-Super, and LN-Ultra -- under the commercially permissive NVIDIA Open Model License Agreement. 2. We release the complete post-training dataset: Llama-Nemotron-Post-Training-Dataset. 3. We also release our training codebases: NeMo, NeMo-Aligner, and Megatron-LM.
Similar Papers
Nemotron Elastic: Towards Efficient Many-in-One Reasoning LLMs
Computation and Language
Builds one smart AI that acts like many sizes.
Nemotron-Research-Tool-N1: Exploring Tool-Using Language Models with Reinforced Reasoning
Computation and Language
Teaches AI to use tools better than before.
Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoning
Machine Learning (CS)
Teaches computers to think better about everything.