Design and Evaluation of a Processing-in-Memory Architecture for the Smart Memory Cube

3D integration of solid-state memories and logic, as demonstrated by the Hybrid Memory Cube HMC, offers major opportunities for revisiting near-memory computation and gives new hope to mitigate the power and performance losses caused by the "memory wall". Several publications in the past few years demonstrate this renewed interest. In this paper we present the first exploration steps towards design of the Smart Memory Cube SMC, a new Processor-in-Memory PIM architecture that enhances the capabilities of the logic-base LoB die in HMC. An accurate simulation environment called SMCSim has been developed, along with a full featured software stack. The key contribution of this work is full system analysis of near memory computation including high-level software to low-level firmware and hardware layers, considering offloading and dynamic overheads caused by the operating system OS, cache coherence, and memory management. A zero-copy pointer passing mechanism has been devised to allow low overhead data sharing between the host and the PIM. Benchmarking results demonstrate upi¾źto 2X performance improvement in comparison with the host System-on-Chip SoC, and around 1.5X against a similar host-side accelerator. Moreover, by scaling down the voltage and frequency of PIM's processor it is possible to reduce energy by around 70i¾ź% and 55i¾ź% in comparison with the host and the accelerator, respectively.