Solutions

B300 ZCube @ 16,384 GPUs

A scalable ZCube architecture designed for B300 clusters.

[B300, 16,384 GPUs]

Intended audience

This solution is designed for:

B300 accelerators using dual-port 2x400G NICs:

Maximum scale

Maximum ZCube interconnect scale depends on NIC bandwidth and switch specification.

2x400G NICs, 51.2T switches (128x400G)

Maximum ZCube interconnect scale: 4,096 NICs

2x400G NICs, 102.4T switches (256x400G)

Maximum ZCube interconnect scale: 16,384 NICs

Solution details

ZCube networking designs at each scale.

Dual-port ZCube with B300 @ 4,096 GPUs

Example with 51.2T switches

Dual-port ZCube B300 4,096-GPU networking topology

Dual-port ZCube with B300 @ 16,384 GPUs

Example with 102.4T switches

Dual-port ZCube B300 16,384-GPU networking topology

Expected value

At every scale, ZCube uses fewer network devices than Clos.

Lower network hardware cost

33% fewer network switches

33% fewer 400G modules

25% fewer fiber cables

Construction shortened to 5–7 days

Solutions

Other solutions

See more scalable ZCube networking designs by cluster scale, NIC form, and switch specification.

[Dual-port NICs, 16,384 GPUs]

Dual-port ZCube @ 16,384 GPUs

A scalable ZCube architecture designed for dual-port NICs such as CX7 and CX8.

Intended audience: Dual-port NICs, or single-port NICs that support port split
Maximum scale: 51.2T switches can interconnect 16,384 GPUs
Solution details: See details
Expected value: Fewer network devices

View solution details

[Single-port NICs, 8,192 GPUs]

Single-port ZCube @ 8,192 GPUs

A scalable ZCube architecture designed for single-port NICs that can form an equivalent dual-NIC pair.

Intended audience: PCIe topology that meets affinity requirements and can form an equivalent dual-NIC pair
Maximum scale: 51.2T switches can interconnect 8,192 GPUs
Solution details: See details
Expected value: Fewer network devices and stronger fault tolerance

View solution details

[Multi-port NICs, 100,000 GPUs]

Multi-plane ZCube @ 100,000 GPUs

Multi-port NICs (4 ports or more) can form a multi-plane ZCube architecture and raise scalability.

Intended audience: Multi-port NICs (4 ports or more)
Maximum scale: More than 16,384 GPUs
Solution details: See details
Expected value: Fewer network devices, with stronger ZCube architecture scalability

View solution details

Start with the network

Make the next AIDC more efficient, starting with the network

Whether you are building a new cluster, upgrading an existing network, tuning performance, or deploying intelligent NetOps, we can assess how the network can unlock more effective compute power.