Tiny VoltMinimization of squared norm of a vector sum
Problem statement
Given a vector of the form , where is a matrix and is a vector, we want to find that minimizes the squared norm of the vector.
Find x for which is minimized. Here M is the basis vector and x is a scalar. The vector with minimum norm is shown in bold.
Solution
We can write the function as:
It then follows that . Setting it to zero, we get the solution:
Note that is the projection of onto the column space of .
Thus the optimal value of is the negative of the projection of onto the column space of . Ideally, if were equal to , the norm of the sum would be zero. But that may not be possible if is not in the column space of . Simply said, the optimal value of tries to get as close as possible to .