Blog

Notes and essays on linear algebra.

• Paper: A Flag Decomposition for Hierarchical Datasets

  This article explains the key idea in the paper A Flag Decomposition for Hierarchical Datasets by Mankovich et al. It allows one to decompose (or factorize) a matrix A into a product of matrices QR w…

• SINQ — Sinkhorn-like iterations for quantization

  Sinkhorn iteration involves normalization of rows of a matrix followed by normalization of columns of the same matrix. This process is repeated until the matrix converges to a matrix with all rows an…

• A new solution to GPTQ quantization algorithm

  The papers Optimal Brain Compression and GPTQ introduced a post-training quantization method that is one of the most popular quantization methods at the moment. The algorithm quantizes the rows of a…

• Minimization of squared norm of a vector sum

  We can write the function as:

• Another proof of Fibonacci sequence in classic probability puzzle

  Recently Scientific American reported that Fibonacci sequence appears as an answer to the following problem:

• Relationship between AAᵀ and AᵀA

  We look into the relationship between eigenvalues and eigenvectors of AAᵀ and AᵀA for a real-valued matrix A of shape (r x c). Without losing generality, we can assume that the matrix is wide i.e. r…

• A simpler derivation of continuous normalizing flows and its interpretation

  Continuous normalizing flow (CNF) is a type of normalizing flow that uses a continuous, differentiable function to transform a probability distribution. My first encounter with CNF was in the paper N…

• Nice Jacobians for normalizing flows

  We cover three main transformations the Jacobians of which are simply the product of the diagonal (possibly block) elements of the Jacobian.

• The Kronecker Product Visualized

  The Kronecker product takes two matrices and produces a larger block matrix. This article visualizes the operation and explores some of its lesser-known properties.

• Three interpretations of matrix products

  This article explores three different interpretations of matrix products with the help of animations. Each of them provides a different perspective on the matrix product and can be useful in differen…

• Permutation matrices - from shuffle to symmetry

  A permutation matrix simply permutes (or shuffles) the elements of the input vector it is multiplied with. When it is multiplied with a matrix, it shuffles the rows or columns of the matrix. This see…

• Inverse of a matrix after a row-and-column removal

  Is using the matrix inverse faster than calculating the inverse of the smaller matrix?

• Inverse of a block of a matrix and Schur Complement

  A few weeks ago I discussed the problem of calculating the inverse of a matrix after removing a row and a column. At that time I completely failed to realize I was staring at a well known matrix asso…