The papers Optimal Brain Compression and GPTQ introduced a post-training quantization method that is one of the most popular quantization methods at the moment. The algorithm quantizes the rows of a…
Read article →This article explains the key idea in the paper A Flag Decomposition for Hierarchical Datasets by Mankovich et al. It allows one to decompose (or factorize) a matrix A into a product of matrices QR w…
Sinkhorn iteration involves normalization of rows of a matrix followed by normalization of columns of the same matrix. This process is repeated until the matrix converges to a matrix with all rows an…
We can write the function as:
Recently Scientific American reported that Fibonacci sequence appears as an answer to the following problem:
We look into the relationship between eigenvalues and eigenvectors of AAᵀ and AᵀA for a real-valued matrix A of shape (r x c). Without losing generality, we can assume that the matrix is wide i.e. r…
Continuous normalizing flow (CNF) is a type of normalizing flow that uses a continuous, differentiable function to transform a probability distribution. My first encounter with CNF was in the paper N…
We cover three main transformations the Jacobians of which are simply the product of the diagonal (possibly block) elements of the Jacobian.
The Kronecker product takes two matrices and produces a larger block matrix. This article visualizes the operation and explores some of its lesser-known properties.
This article explores three different interpretations of matrix products with the help of animations. Each of them provides a different perspective on the matrix product and can be useful in differen…
A permutation matrix simply permutes (or shuffles) the elements of the input vector it is multiplied with. When it is multiplied with a matrix, it shuffles the rows or columns of the matrix. This see…
Is using the matrix inverse faster than calculating the inverse of the smaller matrix?
A few weeks ago I discussed the problem of calculating the inverse of a matrix after removing a row and a column. At that time I completely failed to realize I was staring at a well known matrix asso…
The Khatri-Rao product takes two matrices with the same number of columns and pairs their columns up, replacing each pair by its Kronecker product. The result is a tall matrix with the same number of…