Shiftfly: Scaling the Accelerator Interconnect Past the Pod with a Shift-Routed Optical Tier

Original abstract

Google's TPU interconnect spent nine generations as a $k$-ary $n$-cube, whose diameter grows as $Θ(N^{1/n})$, before TPU 8i replaced it with Boardfly: a three-tier hierarchy in which every tier is a complete graph, giving pod diameter 7 over 1,152 chips. Boardfly suits a single inference pod but does not extend, because a complete global tier needs $G-1$ optical ports to reach $G$ groups. A 400,000-chip machine would need 12,499 per group against the 40 available, so another hierarchy level must be stacked, and each level costs four chip hops. We propose Shiftfly, which keeps Boardfly's building block and group verbatim and replaces only the global tier with a generalized Kautz digraph, installed as a fixed permutation on the optical circuit switch the fabric already owns. At an identical 40 ports per group, Shiftfly is flat, has guaranteed diameter $\lceil \log_d G \rceil$, routes without tables by a shift register, and addresses content natively. The evaluation is deliberately two-sided. Shiftfly loses at one-pod scale, where Boardfly achieves chip-level diameter 7 against Shiftfly's 11, and wins beyond it, cutting worst-case distance from 23 to 19 hops at 400,000 chips with roughly 2.7x better spectral expansion at equal cost. The redundancy argument that motivated the design does not survive its own evaluation: locality-aware placement supplies most of the achievable saving on shared content, leaving the shift algebra a 2.9% residual, and we report the metric inversion that conceals this. We also measure operability. Replacing a failed group costs the same 40 optical circuits in both designs; deriving the global wiring costs 28 bits of control-plane state against 550,000. Slice allocation is the one regression: an arbitrary induced subset of a shift graph is disconnected, so slices must be instantiated rather than carved.