NVIDIA technical dossier 01
Vera Rubin NVL72
What NVIDIA’s rack-scale AI system is, why it exists and how its processors, memory and interconnects work together.01
What it is
NVIDIA Vera Rubin NVL72 is a complete liquid-cooled computing rack for building and running very large AI models. It combines 72 Rubin GPUs, 36 Vera CPUs, HBM4, NVLink switches and external networking in one integrated system.
Unlike a conventional server, NVL72 uses NVLink 6 switches to connect all 72 GPUs into one fast communication domain. Software can divide one model across the rack while the GPUs exchange model data without routing every transfer through ordinary data-centre networking. It is infrastructure for AI training, post-training and inference—not a desktop GPU or single server card.
Host processorVeraCPU control and data movement
Compute engineRubinGPU parallel AI computation
Scale-up fabricNVLNVLink-connected compute domain
System scale72Rubin GPUs operating as one rack
02
Why it exists
Large AI workloads do not depend on arithmetic alone. They repeatedly move model weights, activations and attention data between memory and processors. When a model is split across many GPUs, those GPUs must also exchange intermediate results and remain synchronised.
This creates four practical bottlenecks. NVIDIA designed Vera Rubin as a system-level response to them:
| Workload problem | Why it matters | Vera Rubin response |
|---|---|---|
| Model size | Weights and the key-value cache may not fit in one GPU’s memory. | 288 GB of HBM4 per GPU; 20.7 TB across the rack. |
| Memory movement | During token generation, processors repeatedly read weights and cached context. | Up to 22 TB/s HBM4 bandwidth per GPU. |
| GPU communication | Distributed models stall when GPUs wait for activations, routing or collective operations. | NVLink 6 provides 3.6 TB/s per GPU and an all-to-all 72-GPU domain. |
| CPU coordination | Data preparation, orchestration and agent environments can leave GPUs waiting. | 36 Vera CPUs connect coherently to Rubin GPUs through NVLink-C2C. |
03
How the system is built
| Component | Role in the system |
|---|---|
| Rubin GPU | Runs the highly parallel matrix, attention and scientific-computing operations. |
| Vera CPU | Runs host-side code, prepares work, manages control flow and supports CPU-heavy agent and reinforcement-learning environments. |
| HBM4 | Stores model weights, activations and key-value cache close to each GPU. |
| NVLink 6 switch | Moves data directly among the 72 GPUs inside the rack and accelerates collective operations. |
| ConnectX-9 SuperNIC | Connects the rack to other racks over InfiniBand or Ethernet for larger clusters. |
| BlueField-4 DPU | Offloads infrastructure work including networking, storage and security. |
04
How it works
The exact execution plan depends on the model and software, but a simplified inference cycle looks like this:
- Receive and prepare the request.CPU-side software tokenises input, schedules work and identifies which GPUs hold each part of the model.
- Read model state from HBM4.Each GPU loads the weights, activations and cached attention data required for its part of the calculation.
- Run transformer operations.Rubin Tensor Cores perform matrix multiplication while other GPU units handle attention, normalisation and data movement.
- Exchange intermediate results.When the model spans GPUs, NVLink 6 carries activations, expert-routing traffic and collective reductions across the rack.
- Select the next token.The model produces probabilities for the next output token. Serving software selects a token and updates the key-value cache.
- Repeat.Steps two to five repeat for each generated token. Long reasoning chains therefore magnify memory and communication efficiency.
Training differs from inference. Training also performs a backward pass to calculate gradients and update model weights. This requires more memory, communication and numerical precision. Inference normally keeps weights fixed and repeatedly generates output tokens.
05
What is new inside the Rubin GPU
| Technology | What it does | Why it matters |
|---|---|---|
| Dual compute dies and NV-HBI | Joins two reticle-limited compute dies inside one GPU package. | Allows a larger processor while presenting the dies as one GPU. |
| Third-generation Transformer Engine | Selects and executes supported low-precision formats, including NVFP4. | Uses less memory and performs more AI operations when the model tolerates lower precision. |
| HBM4 memory subsystem | Provides up to 288 GB capacity and 22 TB/s peak bandwidth per GPU. | Keeps more model state and longer-context cache close to compute. |
| Tensor Memory Accelerator updates | Lets mixture-of-experts tensors share layout descriptors while changing addresses and strides at runtime. | Reduces metadata and data-movement overhead as tokens route to different experts. |
| Larger K dimension per instruction | Processes twice as much of the reduction dimension in a Tensor Core instruction. | Reduces loop overhead in distributed matrix multiplication. |
| Activation sparsity and compression | Represents selected intermediate attention data in structured 2:4 sparse form. | Reduces work and data movement in later attention operations. |
| Counted NVLink writes | Allows a receiving GPU to track completion of device-initiated transfers with counters. | Reduces communication synchronisation overhead. |
06
Principal specifications
| Measure | One Rubin GPU | One Superchip | NVL72 rack |
|---|---|---|---|
| Configuration | 1 Rubin GPU | 2 Rubin GPUs; 1 Vera CPU | 72 Rubin GPUs; 36 Vera CPUs |
| NVFP4 inference | 50 PFLOPS | 100 PFLOPS | 3,600 PFLOPS |
| FP8 / FP6 training | 17.5 PFLOPS | 35 PFLOPS | 1,260 PFLOPS |
| FP16 / BF16 | 4 PFLOPS | 8 PFLOPS | 288 PFLOPS |
| Native FP64 | 33 TFLOPS | 67 TFLOPS | 2,400 TFLOPS |
| GPU memory | 288 GB HBM4 | 576 GB HBM4 | 20.7 TB HBM4 |
| GPU memory bandwidth | 22 TB/s | 44 TB/s | 1,580 TB/s |
| NVLink bandwidth | 3.6 TB/s | 7.2 TB/s | 260 TB/s switch bandwidth |
| NVLink-C2C | Not applicable | 1.8 TB/s | 65 TB/s aggregate |
| CPU cores | Not applicable | 88 Olympus cores | 3,168 Olympus cores |
| CPU memory | Not applicable | 1.5 TB LPDDR5X | 54 TB LPDDR5X |
| Scale-out networking | 0.4 TB/s | 0.8 TB/s | 28.8 TB/s aggregate |
How to read these figures
FLOPS measure theoretical floating-point operations, not the speed of a complete application. Lower-precision figures such as NVFP4 apply mainly to suitable AI operations; FP64 is relevant to scientific calculations requiring much higher numerical precision.
The figures are not interchangeable. A rack’s 3,600 PFLOPS of NVFP4 inference does not mean it produces a fixed number of tokens per second. Model architecture, batch size, context length, software, communication and achieved memory bandwidth all affect real performance.
07
The software layer
Hardware alone does not make 72 GPUs behave as one useful system. NVIDIA’s platform software provides the programming, communication and operational layers:
- CUDA and CUDA-X
- Programming model, compilers and accelerated libraries used to run computation on Rubin GPUs.
- Communication libraries
- Coordinate collective operations and data exchange across NVLink and the external cluster network.
- Training and inference frameworks
- Partition models, schedule work and manage distributed execution across processors.
- Mission Control and platform management
- Provision, monitor and operate rack and cluster infrastructure.
08
Terms used in this article
- GPU
- A processor designed for highly parallel computation.
- HBM
- High-bandwidth memory placed close to a GPU.
- Scale up
- Connect processors into one tightly coupled compute domain.
- Scale out
- Connect multiple servers or racks into a larger cluster.
- MoE
- Mixture of experts; a model that routes tokens to selected specialist sub-networks.
- KV cache
- Stored attention state reused while a model generates subsequent tokens.
- PFLOPS
- One quadrillion floating-point operations per second.
- Dense specification
- A peak figure calculated without assuming sparsity reduces the number of operations.
09
Limits of the available evidence
- NVIDIA marks the published specifications as preliminary, “up to” values that may change.
- The detailed architecture and performance figures currently come from NVIDIA and have not been independently measured for this dossier.
- Peak compute and bandwidth do not establish sustained application performance.
- NVIDIA reports that Vera Rubin is ramping into full production and that systems are shipping; availability varies by manufacturer and cloud provider.
- Price, total cost of ownership and vendor cost-per-token claims are deliberately excluded.
10
Sources
- NVIDIA Vera Rubin NVL72 product and specification pageConfiguration, compute, memory, interconnect and production status.
- Inside the NVIDIA Vera Rubin PlatformSystem architecture, component roles, software and scale-up design. Published 5 January 2026; updated 16 March 2026.
- Inside NVIDIA Rubin GPU ArchitectureCompute dies, HBM4, Tensor Memory Accelerator, attention and NVLink mechanisms. Published 21 July 2026.
- NVIDIA Vera Rubin platform announcementPlatform composition, production state and partner availability.
- Vera Rubin NVL72 on CoreWeave CloudCoreWeave video supplied for this article’s system overview.
- How Vera Rubin NVL72 works as one AI supercomputerVDO Review video supplied for the operational overview.