Analysis | How ZCube Improves Network Performance Metrics

Large-model inference is redefining AI infrastructure. Networks are no longer merely supporting infrastructure; they have become a key variable determining inference-system throughput, tail latency, and the total cost of MaaS.

To address increasingly severe structural network congestion in P/D-disaggregated deployments, Harnets.AI, Zhipu AI, and Tsinghua University jointly deployed the ZCube network architecture in an online production environment this year. Using several months of production data, this article shows ZCube's end-to-end inference improvements and analyzes their causes through underlying network performance metrics.

1. Review: ZCube's End-to-End Inference Performance Gains

For large-model inference, cluster token throughput and time to first token (TTFT) are core measures of user experience and system efficiency.

Production monitoring data show that, in the same hardware environment, ZCube delivers significant advantages over conventional ROFT (Rail-Optimized Fat-Tree) architecture [1]:

Inference throughput: average GPU inference throughput increased by 15%;

TTFT latency: the P99 TTFT fell by more than 40%.

Alongside these inference gains, ZCube also reduced network-hardware costs, including switches and optical transceivers, by 33%, improving efficiency while lowering AI-cluster cost.

[1] https://z.ai/blog/zcube

2. Analysis: Network Performance Metrics Compared

To investigate the underlying causes of the inference gains, we collected and compared network performance metrics from the ZCube cluster and two conventionally designed clusters (CLOS#1 and CLOS#2). The environments were highly consistent: they used the same GPU-server model, with essentially identical cluster scale, running model (GLM 5.2), and workload traffic characteristics.

Network Congestion Frequency Falls Significantly

PFC (Priority Flow Control) pause frames are a key measure of network congestion. When congestion occurs and switch buffers are about to overflow, a switch port sends PAUSE frames upstream, as far as the NIC, stopping transmission until its queue depth falls sufficiently.

PFC RX is the number of PFC PAUSE frames received per second by an RDMA NIC and is directly related to the degree of network congestion.

Monitoring data (shown below at an hourly time scale) indicate that, because it has fewer traffic conflicts, the ZCube cluster substantially outperforms the other CLOS clusters on PFC RX:

PFC mean reduced by >79.5%: ZCube's mean PFC RX was 79.5% lower than CLOS#1 and 85.3% lower than CLOS#2.

PFC P99 reduced by >76.8%: ZCube's P99 PFC RX was 76.8% lower than CLOS#1 and 77.1% lower than CLOS#2.

This reduction indicates less congestion in the ZCube cluster network, allowing NICs to transmit traffic faster and improving network performance.

Theoretical analysis: ZCube breaks from the traditional Clos approach of hierarchically stacked switches, using a completely flat combination of single-rail and multi-rail interconnection for GPU servers. The ZCube routing policy uses the structural features of this flat fabric to achieve ideal traffic load balancing among switches across the network, theoretically reducing the probability of congestion.

KV Cache Transfer Latency Falls

As long-context and P/D-disaggregated inference become mainstream, data exchange between Prefill and Decode nodes continues to grow. KV Cache transfer performance significantly affects end-to-end inference performance. Under high load, ZCube substantially shortens this transfer process.

Transfer KV Cache Latency is the time from Prefill sending a KV Cache to Decode completing receipt.

Mean KV Cache transfer latency reduced by >27.2%: ZCube's mean Transfer KV Cache Latency was 27.2% lower than CLOS#1 and 40.6% lower than CLOS#2.

Maximum KV Cache transfer latency reduced by >39.1%: ZCube's maximum Transfer KV Cache Latency was 39.1% lower than CLOS#1 and 46.4% lower than CLOS#2.

This improvement shows that ZCube's congestion optimization directly improves KV Cache transfer performance in P/D-disaggregated inference.

3. Conclusion

Through deep optimization of underlying network load balancing, the ZCube cluster's network-transfer performance improved by 30%–50% over conventional clusters. This network-level advantage translates directly into end-to-end inference gains. In long-context and compute-intensive scenarios, ZCube can relieve network bottlenecks more effectively and further improve inference-cluster performance.