Solutions

Dual-port ZCube @ 16,384 GPUs

A scalable ZCube architecture designed for dual-port NICs such as CX7 and CX8.

[Dual-port NICs, 16,384 GPUs]

Intended audience

This solution is designed for:

Servers using dual-port NICs, or single-port NICs that support port split:

Dual-port NICs (recommended): CX8 900-9X81Q*, CX7 MCX7**106A*, BF3, and similar

Single-port NICs: CX8 900-9X81E*, MCX715105AS-WEAT, and similar

Dual-port NIC appearance and dimensions

Maximum scale

Maximum ZCube interconnect scale depends on NIC bandwidth and switch specification.

2x200G NICs, 51.2T switches (128x400G)

Maximum ZCube interconnect scale: 16,384 NICs

2x200G NICs, 102.4T switches (256x400G)

Maximum ZCube interconnect scale: 65,536 NICs

Solution details

ZCube networking designs at each scale.

Dual-port ZCube @ 4,096 GPUs

Example with 51.2T switches

Dual-port ZCube 4,096-GPU networking topology

Dual-port ZCube @ 16,384 GPUs

Example with 51.2T switches

Dual-port ZCube 16,384-GPU networking topology

Expected value

At every scale, ZCube uses fewer network devices than Clos.

Lower network hardware cost

33% fewer network switches

33% fewer 400G modules

25% fewer fiber cables

Construction shortened to 5–7 days

Solutions

Other solutions

See more scalable ZCube networking designs by cluster scale, NIC form, and switch specification.

[Single-port NICs, 8,192 GPUs]

Single-port ZCube @ 8,192 GPUs

A scalable ZCube architecture designed for single-port NICs that can form an equivalent dual-NIC pair.

Intended audience: PCIe topology that meets affinity requirements and can form an equivalent dual-NIC pair
Maximum scale: 51.2T switches can interconnect 8,192 GPUs
Solution details: See details
Expected value: Fewer network devices and stronger fault tolerance

View solution details

[B300, 16,384 GPUs]

B300 ZCube @ 16,384 GPUs

A scalable ZCube architecture designed for B300 clusters.

Intended audience: B300 GPUs with 2x400G NICs
Maximum scale: 102.4T switches can interconnect 16,384 GPUs
Solution details: See details
Expected value: Fewer network devices

View solution details

[Multi-port NICs, 100,000 GPUs]

Multi-plane ZCube @ 100,000 GPUs

Multi-port NICs (4 ports or more) can form a multi-plane ZCube architecture and raise scalability.

Intended audience: Multi-port NICs (4 ports or more)
Maximum scale: More than 16,384 GPUs
Solution details: See details
Expected value: Fewer network devices, with stronger ZCube architecture scalability

View solution details

Start with the network

Make the next AI data center more efficient, starting with the network

Whether you are building a new cluster, upgrading an existing network, tuning performance, or deploying intelligent operations, we can assess how the network can unlock more effective compute.