Arm technical article 009
Arm AGI CPU
What Arm’s first production silicon product is, how 136 Neoverse V3 cores feed accelerators and why memory, I/O and predictable per-thread capacity define the design.01
What it is
Arm AGI CPU is a 3 nm, 136-core data-centre processor built and sold as silicon by Arm. It is based on Neoverse V3 and is designed for CPU-side work around AI accelerators: orchestration, data preparation, retrieval, tool execution, API hosting, virtual machines and cloud services.
This is a material change in Arm’s role. Arm traditionally licenses instruction-set and processor IP, while partners design the finished chips. AGI CPU is Arm’s first production silicon product. It is still a general-purpose CPU—not a GPU, NPU or tensor accelerator—and runs one hardware thread per core rather than simultaneous multithreading.
CPU136 Neoverse V3 coresOne thread per core
Memory12-channel DDR5Up to 8,800 MT/s
I/OPCIe 6Accelerators, network and storage
Power300 W TDPPublished processor envelope
02
Why an AI system still needs CPUs
Accelerators execute dense tensor operations efficiently, but an AI service also performs irregular work: tokenisation, database retrieval, network handling, decompression, scheduling, security checks, tool calls and control-plane tasks. These operations branch frequently and touch memory and I/O in ways that do not map cleanly to large matrix engines.
When CPU threads cannot prepare data or coordinate distributed work quickly enough, expensive accelerators wait. AGI CPU addresses this host-side bottleneck with many independent cores, relatively high memory bandwidth per core and current server I/O.
| AI-system work | CPU role | Pressure addressed |
|---|---|---|
| Model serving | Request handling, tokenisation, batching and response assembly. | Many concurrent, latency-sensitive threads. |
| Retrieval and tools | Queries databases, APIs, sandboxes and enterprise software. | Branch-heavy code and irregular memory access. |
| Accelerator hosting | Feeds devices, manages queues and handles collectives software. | Memory and PCIe bandwidth around the accelerator. |
| Cloud isolation | Runs containers and virtual machines for separate tenants. | Predictable capacity and security boundaries. |
03
Balanced host architecture
04
How an agent request executes
- Accept the request.CPU cores terminate network connections, authenticate the caller and parse input.
- Assemble context.Threads retrieve documents, query stores, tokenise data and construct the accelerator input.
- Dispatch inference.The host places work in accelerator queues through PCIe and the device runtime.
- Overlap other work.Independent cores service other agents, prepare later batches or execute tools while inference runs.
- Process results.The CPU applies application logic, schedules another model call or invokes an external service.
- Return or continue.Responses are streamed to the caller; multi-step agents repeat the cycle until their workflow ends.
05
From processor to rack
| Level | Published configuration | Interpretation |
|---|---|---|
| Processor | 136 cores, dedicated thread per core, 300 W TDP. | General-purpose compute unit with its own memory and I/O controllers. |
| Dual-socket blade | Two CPUs; 272 physical cores with dedicated memory and I/O resources. | A server node that can operate independently or host accelerators. |
| Air-cooled rack | 30 blades; 8,160 cores; published 36 kW configuration. | Density figure derived from a particular reference design. |
| Liquid-cooled rack | Supermicro design with 336 CPUs and 45,696 cores at 200 kW. | High-power specialised configuration, not a universal rack standard. |
Deployment status. Arm announced AGI CPU on 24 March 2026. Early systems are available, with broader availability expected in the second half of 2026. Meta is the lead development partner; ASRock Rack, Lenovo, Quanta and Supermicro are named system partners.
06
Principal specifications
| Area | Published specification | Reading |
|---|---|---|
| Process | TSMC 3 nm | Manufacturing process for the finished Arm-designed chip. |
| CPU cores | Up to 136 Arm Neoverse V3 | Physical general-purpose cores per socket. |
| Threading | One hardware thread per core | No SMT sibling sharing a physical core. |
| Power | 300 W TDP | Thermal design point, not total server or rack power. |
| Memory | 12 DDR5 channels; up to 8,800 MT/s | Aggregate capacity depends on DIMM population and server design. |
| Per-core memory | 6 GB/s at sub-100 ns latency | Arm-published design figure; workload access patterns still matter. |
| Expansion | PCI Express 6 | Attaches accelerators, network interfaces and storage. |
| Reference blade | 1OU dual-node; two CPUs; 272 cores per blade | Open rack unit sizing differs from conventional 19-inch 1U. |
07
Mechanisms that affect utilisation
| Mechanism | Function | Why it matters |
|---|---|---|
| One thread per core | Assigns each scheduled hardware thread a complete physical core. | Reduces contention between SMT siblings under sustained load. |
| Twelve memory channels | Provides multiple parallel paths to DDR5. | Feeds many cores and reduces the chance that all wait on a narrow memory interface. |
| PCIe 6 | Connects the CPU to external devices at the current interface generation. | Host I/O can otherwise limit accelerator and network utilisation. |
| Neoverse V3 | Supplies the Arm server instruction set and general-purpose core design. | Runs ordinary cloud software rather than accelerator-only kernels. |
| Virtualisation stack | Supports virtual machines, containers and orchestration platforms. | Allows shared infrastructure to isolate many agent services. |
| OCP reference design | Defines reusable mechanical, management and interoperability elements. | Helps system builders deploy the processor without a proprietary rack format. |
08
Terms
- AGI CPU
- Arm’s product name for its agentic-AI data-centre processor.
- Neoverse
- Arm’s infrastructure CPU platform.
- SMT
- Simultaneous multithreading: multiple hardware threads sharing one core.
- TDP
- A processor thermal design target, not measured rack consumption.
- DDR5
- Server main memory connected through parallel memory channels.
- PCIe
- The peripheral interconnect used by accelerators, NICs and storage.
- 1OU
- One Open Rack unit in an OCP mechanical design.
09
Limits of the evidence
- AGI CPU is newly introduced; detailed cache sizes, complete lane counts and full SKU tables were not public at verification time.
- Arm’s per-core bandwidth and latency figures describe its published platform conditions, not every application.
- Rack core counts are reference-configuration arithmetic and do not establish workload throughput.
- Comparative performance, capital-cost and workload benchmark claims are excluded.
- Early systems are available, but broader deployment is still expected later in 2026.
- The product’s name does not mean it performs model training or inference without attached accelerators or software.
- The embedded videos are explanatory material, not sources for the specification tables.
- Price and promotional material are excluded.
10
Sources
- Arm AGI CPU launchProduct role, core count, TDP, process, partners and availability.
- Introducing Arm AGI CPUArchitecture rationale and Arm’s move into production silicon.
- Arm AGI CPU product pageCurrent platform specifications and system configurations.
- How Arm AGI CPU supports agentic workloadsCPU orchestration, memory and I/O workload context.
- Supermicro rack-scale Arm AGI CPU platformLiquid-cooled rack configuration and system integration.
- Arm AGI CPU and Open Compute ProjectOCP DC-MHS reference-server and open-rack context.
- Arm AGI CPU explainedCaleb Writes Code video supplied for product context.
- Agentic AI for finance — Powered by Arm AGI CPUOfficial Arm workload demonstration supplied for this article.