EGRAPHS Community

A Multi-Level Superoptimizer for Tensor Programs

2025-11-20

Abstract

We introduce Mirage, the first multi-level superoptimizer for tensor programs. A key idea in Mirage is µGraphs, a uniform representation of tensor programs at the kernel, thread block, and thread levels of the GPU compute hierarchy. µGraphs enable Mirage to discover novel optimizations that combine algebraic transformations, schedule transformations, and generation of new custom kernels. To navigate the large search space, Mirage introduces a pruning technique based on abstraction that significantly reduces the search space and provides a certain optimality guarantee. To ensure that the optimized µGraph is equivalent to the input program, Mirage introduces a probabilistic equivalence verification procedure with strong theoretical guarantees. Our evaluation shows that Mirage significantly outperforms existing approaches even for DNNs that are widely used and heavily optimized. Mirage is publicly available at https://github.com/mirage-project/mirage.

Speaker

Mengdi Wu is a third-year Ph.D. student in the Computer Science Department at Carnegie Mellon University, advised by Zhihao Jia and a member of the CMU Catalyst Group. Her research focuses on building efficient systems for machine learning workloads, with an emphasis on automating key system decisions for executing these workloads effectively.