Architectural and System Implications of CXL-enabled Tiered Memory

Memory disaggregation is an emerging technology that decouples memory from traditional memory buses, enabling independent scaling of compute and memory. Compute Express Link (CXL), an open-standard interconnect technology, facilitates memory disaggregation by allowing processors to access remote memory through the PCIe bus while preserving the shared-memory programming model. This innovation creates a tiered memory architecture combining local DDR and remote CXL memory with distinct performance characteristics. In this paper, we investigate the architectural implications of CXL memory, focusing on its increased latency and performance heterogeneity, which can undermine the efficiency of existing processor designs optimized for (relatively) uniform memory latency. Using carefully designed micro-benchmarks, we identify bottlenecks such as limited hardware-level parallelism in CXL memory, unfair queuing in memory request handling, and its impact on DDR memory performance and inter-core synchronization. Our findings reveal that the disparity in memory tier parallelism can reduce DDR memory bandwidth by up to 81% under heavy loads. To address these challenges, we propose a Dynamic Memory Request Control mechanism, MIKU, that prioritizes DDR memory requests while serving CXL memory requests on a best-effort basis. By dynamically adjusting CXL request rates based on service time estimates, MIKU achieves near-peak DDR throughput while maintaining high performance for CXL memory. Our evaluation with micro-benchmarks and representative workloads demonstrates the potential of MIKU to enhance tiered memory system efficiency.