CXL Emulation Workgroup

We are pleased to announce the launch of the CXL Emulation Workgroup, an initiative designed to tackle challenges within the CXL ecosystem, such as cache coherency, scalability, and the constrained availability of CXL hardware. We warmly invite the community to engage, share expertise, and collaborate to propel CXL-enabled hyperscale computing.

Board of Advisors

Pacific Northwest National Laboratory (PNNL)

Dr. Andres Marquez

Dr. Andres Marquez

Chief Computer Scientist, PNNL

Principal investigator (PI) for the Department of Energy's Advanced Memory to Support AI for Science. Team Lead of the NextGen Architecture Design team within the Future Computing Technologies Group.

Dr. Roberto Gioiosa

Dr. Roberto Gioiosa

Chief Computer Scientist, PNNL

Principal investigator (PI) and director of the co-design Center for Artificial Intelligence-focused Architectures and Algorithms (ARIAA). Team Lead of the Innovative Emerging Computing team within the Future Computing Technologies Group.

Dr. Nathan R. Tallent

Dr. Nathan R. Tallent

Chief Computer Scientist, PNNL

Recipient of the DOE Early Career award. Team Lead of the Continuum Computing team within the Future Computing Technologies Group.

University of California, Merced

Prof. Dong Li

Prof. Dong Li

Associate Professor, UC Merced

Director of the Parallel Architecture, System, and Algorithm Lab (PASA); Director of the NSF IUCRC Center for Memory System Research (CEMSYS); and co-director of the High Performance Computing Systems and Architecture Group at UC Merced.

Prof. Hyeran Jeon

Prof. Hyeran Jeon

Associate Professor, UC Merced

Director of the Merced Computer Architecture Lab (MoCA). Research focuses on energy-efficient, reliable, and secure computer architecture and systems design.

University of California, Riverside

Prof. Daniel Wong

Prof. Daniel Wong

Associate Professor, UC Riverside

Director of Systems Optimization and Computer Architecture Lab (SoCal). Research spans Computer Architecture, High Performance Computing, and Warehouse-scale Computing.

Team Members

Mentors

Dr. Luanzheng Guo

Dr. Luanzheng "Lenny" Guo

Staff Computer Scientist, PNNL

Research focused on scientific computing, data management, large-scale systems, and machine learning. Serving as PI and co-PI of multiple Laboratory Directed Research and Development (LDRD) projects.

Students

C

Candice

Ph.D. Candidate, IIT Chicago

Supervised by Prof. Xian-He Sun. Research interests include parallel and distributed processing, memory and I/O systems.

R

Rafi

Ph.D. Candidate, UC Merced

Supervised by Prof. Hyeran Jeon. Research interests include architecture design and security.

J

Yiguang (Jack)

Third-year Undergraduate, UCSB

Advised by Prof. Dong Li

V

Vickie

Ph.D., UCSC

Supervised by Prof. Andi Quinn

Research Focus

CXL Simulation & Emulation

Summary of existing CXL EMU/SIM designs including POND, CXL-DMSIM, EmuCXL, and QEMU's CXL capabilities

Cache Coherency

Addressing cache coherency challenges within the CXL ecosystem for improved performance

Scalability

Developing solutions for scalability issues in CXL-enabled hyperscale computing environments

Hardware Availability

Tackling the constrained availability of CXL hardware through emulation and simulation

Pilot Project: OCEAN

OCEAN – Open-source CXL Emulation at Hyperscale Architecture and Networking

There is a project in progress that aligns with our CXL emulation efforts. The OCEAN project focuses on open-source CXL emulation for hyperscale architecture and networking applications.

Repository: https://github.com/mujahidalrafi/OCEAN

This project complements our workgroup's goals and provides a collaborative platform for advancing CXL emulation technologies.

OCEAN Achievements

  • Complete CXL 3.0 specification implementation
  • Memory Device emulation: device can expose up to 32 Host-managed Device Memory (HDM) decoders
  • Fabric Manager: distributed Fabric Manager implemented (host daemon + logic inside emulated devices)
  • CXL switch with arbitrary topology: distributor of cachelines configurable with up to 256 virtual ports
  • Kernel module for VMs providing a custom memory allocator for CXL memory regions in the guest
  • Guest communicates with emulator Fabric Manager to request CXL memory and returns memory to the pool on free
  • Maintains mapping between guest physical pages and global memory addresses
  • Efficient scalability: multi-host support tested
  • Consistency guarantees validated with litmus tests
  • Performance fidelity tested
  • Worked examples: MPI-based Gromacs tested on RDMA and MEMU-emulated CXL Pod
  • Tested Tigon (Distributed DB for a CXL Pod) — Huang et al., OSDI ’25

OCEAN Roadmap

  • Test more workloads suitable for CXL Pod (Llama.cpp with MPI, distributed Louvain, OSU benchmarks)
  • Discover additional workloads that realize CXL advantages
  • Perform performance/functionality assessment for each MEMU component:
    • Switch
    • Fabric Manager Daemon
    • Coherency protocols
  • Scale tests at hyperscale data centers and cloud services
  • Prepare full report for arXiv/OSTI with comprehensive evaluation

Open Source Licensing

NOTICE TO COLLABORATORS

Collaborator Acknowledgement CXL-EMU Open-Source Project Participation

Project Overview:

The AMAIS project, operated by Pacific Northwest National Laboratory (PNNL), sponsored by the Advanced Scientific Computing Research (ASCR) program within the Department of Energy (DOE), focuses on developing cutting-edge memory systems tailored for scientific computational research. Aimed at meeting the demands of large-scale AI-boosted simulations, the project addresses the need for enhanced memory capacity without compromising performance in terms of bandwidth and latency. PNNL is operated by Battelle Memorial Institute for the DOE under Contract DE-AC05-76RL01830.

Open Source Licensing of the Software:

  1. The CXL-EMU software will be licensed under the BSD 2-Clause license, with inclusion of contributions solely at the discretion of Pacific Northwest National Laboratory/Battelle Memorial Institute.
  2. Contributors should affix their copyright notice and the BSD-2 "Simplified" license to each contribution: Copyright 202X Contributor Name. Licensed under the BSD 2 Clause License https://opensource.org/license/bsd-2-clause

Contribution and Licensing:

  1. External collaborators and contributors are welcome to participate and enrich the development and enhancement process of the CXL-EMU project.
  2. Pacific Northwest National Laboratory/Battelle Memorial Institute aim to keep the software open source for anyone to use, safeguarding the project's integrity and innovation potential.

By engaging in this collaborative effort and contributing to the software, contributors acknowledge the terms outlined above for the advancement and success of AMAIS and the CXL-EMU initiative.

Get Involved

Join the CXL Emulation Community

We welcome researchers, developers, and enthusiasts to contribute to advancing CXL technology

Join Our Repository Contact Us