NVIDIA Nemotron 3 Super:
120B Hybrid Mamba-Transformer MoE Redefining Agentic AI

Before diving into the architecture, let us understand the two constraints that Nemotron 3 Super was built to solve. As companies move beyond simple chatbots into multi-agent applications, they hit two walls:

Nemotron 3 Super addresses both: a 1-million-token context window lets agents retain full workflow state in memory — preventing goal drift — while the MoE design keeps only 12B params active, making inference dramatically cheaper than a dense 120B model.

Nemotron 3 Super combines three major innovations into a single architecture that delivers breakthrough efficiency and accuracy:

The training methodology behind Nemotron 3 Super is as impressive as the architecture. NVIDIA used a three-stage pipeline to produce the final model:

Nemotron 3 Super has claimed the top spot on Artificial Analysis for efficiency and openness with leading accuracy among models of the same size. Here are the numbers from the official model card:

Nemotron 3 Super is part of a family of three models, each targeting a different point on the efficiency-accuracy curve:

Here is what the 1M-token context and high-accuracy tool calling enable:

Enterprise leaders including Amdocs, Palantir, Cadence, Dassault Systèmes, and Siemens are deploying Nemotron 3 to automate workflows in telecom, cybersecurity, semiconductor design, and manufacturing.

Nemotron 3 Super supports configurable reasoning via the chat template — toggle deep thinking on or off per request, and even use a low-effort mode that generates significantly fewer reasoning tokens:

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
MODEL = "nvidia/nemotron-3-super"

# Full Reasoning ON (default)
response = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "Prove root 2 is irrational."}],
    max_tokens=16000,
    temperature=1.0, top_p=0.95,
    extra_body={"chat_template_kwargs": {"enable_thinking": True}}
)

# Reasoning OFF - fast direct responses
response = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "What is the capital of Japan?"}],
    max_tokens=16000,
    temperature=1.0, top_p=0.95,
    extra_body={"chat_template_kwargs": {"enable_thinking": False}}
)

# Low-effort reasoning - fewer thinking tokens
response = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "Summarize this document."}],
    max_tokens=16000,
    temperature=1.0, top_p=0.95,
    extra_body={"chat_template_kwargs": {
        "enable_thinking": True, "low_effort": True
    }}
)

Nemotron 3 Super supports vLLM, SGLang, and TensorRT-LLM. Here is the fastest way to get started:

pip install vllm==0.18.1

export MODEL_CKPT=nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8

vllm serve $MODEL_CKPT   --served-model-name nvidia/nemotron-3-super   --async-scheduling   --dtype auto   --kv-cache-dtype fp8   --tensor-parallel-size 4   --max-model-len 262144   --enable-expert-parallel   --trust-remote-code   --gpu-memory-utilization 0.9

Nemotron 3 Super is packaged as an NVIDIA NIM microservice for deployment anywhere:

NVIDIA Nemotron 3 Super is not just another large language model — it is a statement about where agentic AI is heading. The combination of Mamba-2 for efficient sequence processing, Latent MoE for radical parameter efficiency, and Multi-Token Prediction for generation speed creates an architecture purpose-built for the multi-agent future.

What excites me the most is the radical openness. NVIDIA did not just release weights — they released the entire training recipe: 25T+ tokens of pre-training data, post-training datasets, 15 RL environments, and evaluation tools. This sets a new standard for what “open” means in AI.

If you are building multi-agent systems, RAG pipelines, coding assistants, or any application that needs to reason across large contexts at scale — Nemotron 3 Super should be at the top of your evaluation list. The era of efficient, open agentic AI is here.