Without Spine Switches, What Supports ZCube's GPU-Cluster Interconnect?

00ZCube Has No Spine Layer, but It Still Forwards Traffic Between Switches

When network engineers hear “remove the Spine,” their first questions are often:

1. Without a Spine, how do Leaf switches communicate?
2. Are GPU servers directly connected to one another?
3. Are all switches fully meshed?
4. Does cross-server traffic traverse only one switch?
5. Where do ports come from as the cluster grows?

These are all natural questions.

In conventional Clos thinking, cross-Leaf communication almost inherently means:

GPU
↓
Leaf
↓
Spine
↓
Leaf
↓
GPU

The Spine is the transit layer between Leaf switches. Removing it appears to remove the backbone of the entire network.

But ZCube's answer is not: “eliminate transit.”

It is: do not create a dedicated Spine layer that only performs transit.

ZCube divides switches into two groups: an odd switch group and an even switch group.

The two groups form a complete bipartite interconnect: every odd-group switch connects to every even-group switch, and vice versa. At the same time, each GPU's dual-port NIC connects to the two switch groups separately.

ZCube therefore still has a network core. It changes the conventional: Leaf layer + dedicated Spine layer into: a flat network in which two switch groups both provide access and forwarding.

Official ZCube architecture material summarizes the design as removing the Spine layer, dividing switches equally into two groups, creating a complete bipartite interconnect between them, and using dual-port NICs to access Leaf switches through complementary single-rail and multi-rail modes.

ZCube removes not inter-switch forwarding, but the dedicated Spine role that only transits traffic and does not directly connect GPUs.

This is the first key to understanding the architecture.

01ZCube Does Not Replace NVLink or NVSwitch

The title refers to the underlying interconnect of a GPU cluster. The boundary must be clear: GPU interconnect has at least two layers. The first is Scale-Up interconnect inside a server.

This is high-speed communication within an NVLink domain.

The second is Scale-Out interconnect between servers.

ZCube addresses the second layer. It concerns how large numbers of GPUs/NICs connect across servers through a switched network.

Based on its official topology description, ZCube endpoints are dual-port NICs associated with GPUs, and each NIC connects to two switch groups. It is therefore fundamentally a GPU-cluster Scale-Out network topology, not an in-server Scale-Up interconnect. This is an engineering inference based on its published topology.

ZCube removes the Spine in the Scale-Out network; it does not remove NVSwitches inside servers.

These two kinds of “switch” must not be confused.

02What Does a Conventional Spine Layer Actually Solve?

Why have Leaf-Spine fabrics been popular? They are simple, regular, and scalable. Each Leaf connects servers. The Spine connects all Leaf switches.

With an appropriate number of Spine switches and uplink capacity, the fabric provides multiple equal-cost paths between different Leaf switches. A typical path from one server to a server under another Leaf is:

source server
↓
source Leaf
↓
a Spine switch
↓
destination Leaf
↓
destination server

But it has a hidden premise: multiple Spine paths must be used evenly.

If different large flows are hashed by ECMP to the same Spine, or some source-destination relationships naturally concentrate on a few Leaf switches, then: some Spine links are congested, while others are idle. Aggregate bandwidth appears sufficient, but the workload cannot use it fully.

The ZCube production report notes that conventional Clos performance depends heavily on ideal balancing among Spine switches, which real routing policy and workload traffic cannot reliably achieve. In P/D-disaggregated inference, asymmetric KV Cache transfers further create Leaf and link hotspots.

The Spine layer provides many possible paths, but “many paths” does not mean that large flows will necessarily be evenly separated.

This is the part ZCube seeks to redesign.

03If ROFT Is Already Rail-Optimized for GPUs, Why Is It Not Enough?

GPU clusters do not simply copy an ordinary Clos fabric. Engineering deployments often use ROFT: Rail-Optimized Fat-Tree. Its core idea is to place GPUs/NICs with the same index from different servers on the same rail.

For example:

NIC0 on every server → Rail0 Leaf
NIC1 on every server → Rail1 Leaf
NIC2 on every server → Rail2 Leaf
…

NCCL documentation also explains that in multi-NIC systems, NCCL considers rail-optimized topology and tries to use the same NIC across nodes to reduce cross-rail interference.

This design works well for training traffic because communications such as AllReduce, ReduceScatter, and AllGather are highly structured, allowing rank and NIC mappings to be planned in advance. NCCL collectives involve multiple ranks and create different many-point data-exchange patterns.

The ZCube production report starts from this issue: in P/D disaggregation, KV Cache traffic is strongly asymmetric in source, destination, and volume, so conventional ROFT rail mapping no longer naturally provides load balancing.

ROFT optimizes “regular rail communication”; ZCube addresses “dynamic and asymmetric GPU-to-GPU communication.”

04What Truly Replaces the Spine in ZCube Is the Complete Bipartite Graph

Now for the core idea. Assume ZCube has eight switches. Divide them into two groups:

odd group: S1, S3, S5, S7
even group: S2, S4, S6, S8

Connect them as follows:

S1 ↔ S2, S4, S6, S8
S3 ↔ S2, S4, S6, S8
S5 ↔ S2, S4, S6, S8
S7 ↔ S2, S4, S6, S8

Switches within the odd group do not need to be fully interconnected, nor do switches within the even group. This structure is called a: complete bipartite graph. Its key property is that every odd-group switch has a direct link to every even-group switch. Cross-switch traffic no longer needs to go through a separate Spine; a source switch can directly send it to the destination switch in the opposite group.

For example:

GPU-A
↓
S1
↓
S6
↓
GPU-B

Both S1 and S6 connect GPUs. For this traffic: S1 is the source access switch, S6 is the destination access switch, and they are directly interconnected.

The official ZCube definition centers on dividing switches into two equally sized groups and creating a complete bipartite connection between them.

ZCube does not rely on an “invisible Spine”; it gives every GPU-access switch direct reachability across the groups.

05It Is Neither Full GPU Mesh nor Full Switch Mesh

A “complete bipartite graph” can easily be mistaken for a full mesh, but they differ. Pairwise interconnecting every switch would require an enormous number of links. ZCube requires each odd-group switch to connect to all even-group switches, and vice versa. It does not require connections within either group. Assume four switches per group. The complete bipartite graph needs:

4 × 4 = 16 inter-group connections

A full mesh of eight switches requires:

8 × 7 ÷ 2 = 28 connections

ZCube is neither direct one-to-one GPU cabling nor full interconnection among all switches. It uses: structured full connectivity after grouping. This gives every odd-group/even-group switch pair one direct connection. Each GPU enters both groups through dual ports, creating short paths between GPU pairs.

ZCube's flatness is not unstructured; strict bipartite-graph rules control link count and path length.

06Dual-Port NICs Are Another Pillar of ZCube

A complete bipartite interconnect between switches is not enough. ZCube also depends on dual-port NICs associated with GPUs. Each GPU/NIC has two network ports: the first accesses the odd group and the second the even group. These are not connected arbitrarily to two switches. ZCube uses two complementary access modes: Single-Rail Access Multi-Rail Access Official material describes them as:

One switch group connects consecutive GPU-ID ranges, which can be understood as single-rail grouping; the other connects GPUs at the same relative position in different groups, forming crosswise multi-rail access.

Consider an example with 32 GPUs and eight GPUs connected to each switch.

Even-group switches may connect as follows:

S2: GPUs 1–8
S4: GPUs 9–16
S6: GPUs 17–24
S8: GPUs 25–32

This is contiguous grouping.

Odd-group switches may connect as follows:

S1: GPUs 1, 9, 17, 25 …
S3: GPUs 2, 10, 18, 26 …
S5: GPUs 3, 11, 19, 27 …
S7: GPUs 4, 12, 20, 28 …

The complete numbering extends according to the published formula, but the core idea is clear: one group is organized by consecutive ranges, and the other crosswise by the same relative index.

ZCube does not treat dual-port NICs as simple active/standby connections; its two ports carry two complementary GPU-grouping relationships.

This is why it can accommodate both regular training communication and more variable inference communication.

In summary, ZCube removes dedicated transit Spine equipment, not inter-switch forwarding. It combines a complete bipartite interconnect between two switch groups with two dual-port GPU-NIC access modes to create a flat network.

The next article examines how traffic actually travels between GPUs, why this can balance traffic more easily than ECMP hashing, and where its benefits and limits lie.