PAGE: Processing-Using-DRAM Architecture-CircuitsCo-Optimization for Efficient Acceleration of General Matrix-Vector Multiplication
As large language models (LLMs) scale in size/complexity, DRAM access has become a performance bottleneck, especially during the memory-bound decoding phase dominated by general matrix-vector multiplication (GEMV) operations. Among near-/in-memory computing works that address such bottlenecks, Processing-Using-DRAM (PUD) offers a unique advantage as it performs logic operations entirely within DRAM peripherals with minimal circuit changes, enabling deployment on commercial off-the-shelf (COTS)DRAM. While prior work reported multi-input logic operations, accelerating GEMV operations in LLMs remains underexplored due to the limited compute capability of DRAM sense amplifiers (SAs).To overcome this, we propose PAGE, a PUD architecture that accelerates GEMV with four key optimizations: (1) mat-level parallelism that activates multiple consecutive mats; (2) inversion-enabled SA supporting NOT operations within a mat; (3) AP-based row copy mechanism that performs source–destination co–activation with only control-signal changes; and (4) adaptive adder-tree accumulation that reduces accumulation cycles. We demonstrate the baseline PUD architecture on a COTS DRAM–FPGA platform to verify the functionality/timing of PUD operations, implement the modified circuits in TSMC 16nm for circuit-level evaluation, and build an in-house simulator for system-level throughput/energy analysis. Over-all, PAGE achieves up to 14×/3.1× end-to-end throughput/energy improvement for GEMV workloads over SoTA PUD designs, demonstrating efficient acceleration of memory-bound LLM workloads.