Samsung's LPDDR5X-PIM puts MAC units in every DRAM bank — clever silicon, hard software

At Hot Chips 2026, Samsung detailed LPDDR5X-PIM, a memory chip that embeds multiply-accumulate (MAC) units inside each of its 16 DRAM banks. The trick is bandwidth: normal DRAM access maxes out around 76.8 GB/s because it’s bottlenecked by the external bus, but compute placed at the banks can tap the chip’s internal bandwidth of roughly 614 GB/s. The MAC arrays handle low-precision formats (INT8, FP8, and 4-bit weights), yielding about 2.4 TOPS per package. That’s modest alone, but eight chips scale to ~9.6 INT8 TOPS — comparable to the NPU in Intel’s Meteor Lake, albeit at the cost of 128 GB of LPDDR5X, an expensive configuration.

The standout engineering feat is that PIM operates entirely within the standard LPDDR5X protocol, so it can sit behind an ordinary memory controller. Samsung repurposes reserved row addresses as MMIO-style control registers: activating specific rows flips the chip between single-bank (normal) and multi-bank mode, or redirects read/write commands to PIM registers instead of DRAM contents. In this scheme reads trigger computation rather than fetching data, writes flush accumulator results back to DRAM, and an ‘Address Align Mode’ lets the hardware tolerate the memory controller reordering accesses. Functionally it behaves like a heavily constrained SIMD engine, broadcasting one operation and operand across all 16 banks at once.

The catch is software. Because PIM modes change what ordinary DRAM commands mean, a chip can’t serve PIM compute and normal memory traffic at the same time — and the controller has no notion of which thread issued an access, so a stray read or write from a non-PIM thread can corrupt PIM results or write data to the wrong address. Isolating a dedicated PIM memory region is the obvious fix, but that means dropping channel interleaving and dedicating whole channels, which strands bandwidth for both PIM and non-PIM workloads. Layered on top are locking requirements for multithreaded use and unresolved coordination problems when multiple processes on a modern OS try to use PIM without knowing about each other.