Intended audience
This solution is designed for:
Servers with single-port NICs where the PCIe topology meets affinity requirements, so two NICs can be treated as an equivalent dual-port NIC:
Each accelerator is PCIe-affine to two equivalent NICs (NIC model is not restricted). This is common when a single PCIe Switch attaches two NICs and one or more accelerators
Maximum scale
Maximum ZCube interconnect scale depends on NIC bandwidth and switch specification.
1x400G NICs, 51.2T switches (128x400G)
Maximum ZCube interconnect scale: 8,192 NICs
1x400G NICs, 102.4T switches (256x400G)
Maximum ZCube interconnect scale: 32,768 NICs
Solution details
ZCube networking designs at each scale.
Single-port ZCube @ 4,096 GPUs
Example with 51.2T switches
Single-port ZCube @ 8,192 GPUs
Example with 51.2T switches
Expected value
At every scale, ZCube uses fewer network devices than Clos.
Lower network hardware cost
33% fewer network switches
50% fewer 400G modules
50% fewer fiber cables
Construction shortened to 5–7 days