With the end of Dennard scaling and the slowing of Moore’s Law, the performance of general-purpose processors has plateaued. At the same time, the energy cost of computing is following a grim exponential trajectory. For these reasons, computing systems are widely expected to transition toward highly heterogeneous accelerator-intensive platforms. As system designers prepare for this new era, the focus is largely on designing individual accelerator architectures. Unfortunately, much of the surrounding computing stack continues to rely on legacy processor-centric assumptions. This mismatch introduces new headaches when attempting to control, scale, and optimize accelerators in modern heterogeneous systems.

In this talk, I will argue that rethinking the rest of the hardware stack around accelerators is key to enable heterogeneity without these headaches. I will first present a processor–accelerator co-design paradigm that enables seamless interleaving between general-purpose and special-purpose computations. I will then introduce a design that specializes network hardware for distributed sparse applications, enabling sparse accelerators to scale across large clusters. Finally, I will highlight a lightweight machine-learning hardware agent, inspired by slot machines, that can substantially simplify performance optimization across heterogeneous hardware mechanisms. I will conclude by outlining future research directions toward a new generation of accelerator-centric systems capable of sustaining scientific and technological progress.