HGPU groupA Novel Compiler Transformation for Fast Sparse Matrix Multiplication in GPUs
N-gated Hacker News🎉 Big news, folks! AMD's CDNA 4 is here to revolutionize... slightly. 📉 Chester Lam's riveting expose on how marginally improving matrix multiplication with low precision data types is the cutting edge of mediocrity. Who knew that less precision could be the future of tech? 🤔💡
https://chipsandcheese.com/p/amds-cdna-4-architecture-announcement #AMD #CDNA4 #CDNA4 #Revolution #MatrixMultiplication #LowPrecision #HackerNews #ngated
https://chipsandcheese.com/p/amds-cdna-4-architecture-announcement #AMD #CDNA4 #CDNA4 #Revolution #MatrixMultiplication #LowPrecision #HackerNews #ngated
Hacker NewsMatrix-vector multiplication implemented in off-the-shelf DRAM for Low-Bit LLMs
https://arxiv.org/abs/2503.23817
#HackerNews #MatrixMultiplication #LowBitLLMs #DRAM #Technology #ArtificialIntelligence #ResearchInnovation #MachineLearning
MVDRAM: Enabling GeMV Execution in Unmodified DRAM for Low-Bit LLM Acceleration
General matrix-vector multiplication (GeMV) remains a critical latency bottleneck in large language model (LLM) inference, even with quantized low-bit models. Processing-Using-DRAM (PUD), an analog in-DRAM computing technique, has the potential to repurpose on-device DRAM as a GeMV engine, offering additional high-throughput processing capabilities to widespread consumer devices without DRAM modifications. However, applying PUD to GeMV operations in the LLM inference pipeline incurs significant overheads $\textit{before}$ and $\textit{after}$ in-DRAM computation, diminishing the benefits of its high-throughput processing capabilities. This paper presents MVDRAM, the first practical system to accelerate GeMV operations for low-bit LLM inference using unmodified DRAM. By leveraging the data sharing patterns and mathematical linearity in GeMV operations, MVDRAM orchestrates the processor and DRAM to eliminate the costs associated with pre-arranging inputs and bit-transposition of outputs required in conventional PUD approaches. Our experimental evaluation with four DDR4 DRAM modules shows that MVDRAM achieves comparable or even better inference speed than the processor-based implementation for GeMV operations in low-bit (under 4-bit) LLM. In particular, MVDRAM achieves up to 7.29$\times$ speedup and 30.5$\times$ energy efficiency for low-bit GeMV operations. For end-to-end LLM inference, MVDRAM achieves 2.18$\times$ and 1.31$\times$ throughput improvements, along with 3.04$\times$ and 2.35$\times$ energy efficiency, for 2-bit and 4-bit quantized low-bit models, respectively. MVDRAM has the potential to redefine the AI hardware landscape by demonstrating the feasibility of standard DRAM as an LLM accelerator.
Stephan SchäperklausOptimizing Matrix Multiplication: The Core of Modern Computing 🚀 Investigating the lower bound of 6 operations for matrix multiplication & special cases like Google's PageRank. From AI breakthroughs to web search, efficient matrices drive innovation behind the scenes.
#MatrixMultiplication #AI #Optimization https://t.co/FIxJ497NRw
Optimizing Matrix Operations: The Backbone of Modern Computing
This paper explores the fundamental importance of efficient matrix operations in modern computing, with a focus on the minimum number of computational operations required for matrix multiplication. We discuss the mathematical proof that establishes a lower bound of six operations for general matrix multiplication, and examine exceptions to this rule, such as matrices containing only zeros and ones, as utilized in the Google PageRank algorithm. The paper also highlights the pervasive nature of these optimizations in programming languages and their critical role in various applications, including web search algorithms and artificial intelligence
Optimizing Matrix Multiplication on RDNA3
https://seb-v.github.io/optimization/update/2025/01/20/Fast-GPU-Matrix-multiplication.html
#HackerNews #Optimizing #Matrix #Multiplication #on #RDNA3 #GPU #Optimization #MatrixMultiplication #RDNA3 #Technology
रञ्जित (Ranjit Mathew)“Beating NumPy’s Matrix Multiplication In 150 Lines Of C Code”, Aman Salykov (https://salykova.github.io/matmul-cpu).
Via HN: https://news.ycombinator.com/item?id=40870345
On Lobsters: https://lobste.rs/s/6cktqx/beating_numpy_s_matrix_multiplication
#C #MatrixMultiplication #Math #Performance #BLAS #LinearAlgebra #MatMul #Speed #NumPy #Optimization
Advanced GEMM Optimization on Modern x86-64 Multi-Core Processors
This blog post explains how to optimize multi-threaded FP32 matrix multiplication for modern processors using FMA3 and AVX2 vector instructions. The optimized custom implementation resembles the BLIS design and outperforms existing BLAS libraries (including OpenBLAS and MKL) on a wide range of matrix sizes. Tags: High-performance GEMM on CPU. Fast SGEMM in C. High-performance matrix multiplication on CPU. SGEMM Optimization on CPU.
How much can we gain from Tensor Kernel Fusion on GPUs?
Fast and Practical Strassen’s Matrix Multiplication using FPGAs
#OpenCL #FPGA #MatrixMultiplication #BLAS #LinearAlgebra #GEMM #Package
Evaluation of computational and energy performance in matrix multiplication algorithms on CPU and GPU using MKL, cuBLAS and SYCL
#CUDA #SYCL #MKL #CUBLAS #MatrixMultiplication #LinearAlgebra #Performance #Package
Evaluation of computational and energy performance in matrix multiplication algorithms on CPU and GPU using MKL, cuBLAS and SYCL
Matrix multiplication is fundamental in the backpropagation algorithm used to train deep neural network models. Libraries like Intel’s MKL or NVIDIA’s cuBLAS implemented new and optimiz…
Jason YipNew Breakthrough Brings #MatrixMultiplication Closer to Ideal https://www.quantamagazine.org/new-breakthrough-brings-matrix-multiplication-closer-to-ideal-20240307/