CXL-ClusterSim: Modeling CXL-based Disaggregated Memory Cluster for Pooling and Sharing using gem5 and SST

Kaustav Goswami, Maryam Babaie, Hoa Nguyen, Venkatesh Akella, and Jason Lowe-Power at the University of California, Davis present CXL-ClusterSim, a full-system simulation framework for CXL-based disaggregated memory that combines gem5’s high-fidelity, OS-visible execution with SST’s MPI-parallel scaling. The novelty is a two-phase workflow: gem5 fast-forwards each host through Linux boot and application initialization, checkpoints the region of interest, then SST restores synchronized timing-accurate execution across multiple hosts and a remote memory blade. The framework models both CXL-style memory pooling, exposed as NUMA nodes, and memory sharing, exposed as DAX /dev/dax mappings for direct byte-addressable access. Validation shows the simulated remote link reports STREAM bandwidth within 0.99% of baseline measurements, and SST’s memory controller statistics are within 0.1%. In an 8-host STREAM study, pinning data to the remote pool reduced effective bandwidth to 6.45 GiB/s per node under interleaving, versus 11.4 GiB/s for local memory, while adding 170 ns and 250 ns CXL latency reduced bandwidth by 8.95% and 29%, respectively. For pooling, NAS Parallel Benchmarks class D showed that larger remote fractions sharply reduce IPC, with mg falling to 0.38 relative IPC when 52% of its 27 GiB working set spilled remote. For sharing, GAPBS kernels on a single shared graph achieved 31.8% of memory instructions from the remote blade, but pointer-chasing kernels suffered the largest slowdowns. The authors position CXL-ClusterSim as an open-source platform for exploring hardware/software co-design beyond NUMA emulation and note future work on hot-plugging, coherence, and switch modeling.