X-Cache: A modular architecture for domain-specific caches

With Dennard scaling ending, architects are turning to domain-specific accelerators (DSAs). Emerging DSAs work with sparse data and indirectly-indexed data structures. They introduce non-affine, and dynamic memory accesses, and require domain-specific caches. DSA caches need to support custom tags, data-structure walks, multiple refills, and preload for the datapath. Prior works include ad-hoc caches, but they're inseparable from the underlying DSA and do not implement the cache controller. This research proposes X-Cache, a reusable caching idiom for DSAs. There are three key ideas: i) DSA-specific Tags: The designer can use any combination of fields from the DSA-metadata for the tag. Meta-tag eliminates the overhead of walking and translating metadata to global addresses and improves load-to-use latency. ii) DSA-programmable walkers: It has been observed that a common set of microcode actions can be used to implement the DSA-specific walking, data block, and tag management. A programmable microcode engine has been developed to execute the data orchestration efficiently. iii) DSA-portable controller: We use a portable abstraction, coroutines, to let the designer express walking and orchestration. Coroutines capture the block-level parallelism, remain lightweight, and minimize controller occupancy. X-Cache outperforms address-based caches by 1.7 times and remains competitive with hardwired DSAs. Also, meta-tags save 26—79% energy compared to address-tags.