Review

Weighted sum of numbers is pervasive across scientific fields. One can generalize it to taking a weighted sum of arrays. An array is just an ordered list of numbers. One can get more insight out of it by giving it a geometry.

We will give a geometry to an array of numbers, and start calling it a vector. We will also give a geometric meaning to the two basic operations we saw earlier

• scaling an array
• adding two arrays

---

2.1 A vector is a point in space

Note that we interpret an array of length as a -dimensional vector. We represent an array as the point one can reach by:

• moving a distance of units along the x-axis, and then
• moving a distance of units along the y-axis.

Extending the logic to longer arrays

The same logic applies to an array of length

In this case one uses a -dimensional space with , and axes. The array can be seen as a point in d space one can reach by:

• moving a distance along the -axis, followed by
• moving a distance along the -axis, followed by
• moving a distance along the -axis.

---

2.2 A vector has a length and a direction

This visualization introduces two new properties which are intimately tied to vectors in a space:

1. How long the vector is
2. In what direction the vector points

These properties become more obvious only after visualizing an array as a vector. This is a benefit of ascribing a geometry to a vector.

---

2.2.1 length of a vector represented by the array [a,b,c,d]

A repeated application of the Pythagoras theorem tells us that the length of a vector [a,b,c,d,...] is

We will use the word norm (L2 norm to be precise) to denote how long the vector is. A function to calculate the norm of a vector would look something like

def norm(array):
    return sum(i**2 for i in array)**0.5

Eg This vector has norm of

---

2.2.2 direction of a vector represented by the array [a,b,c,d]

Specifying the direction of a vector in -dimensional space is straightforward. For a vector [x,y], we can use the angle of the vector from the x-axis, as a measure of direction. It can be shown as this infinite line.

But it gets complicated immediately as we move to a -dimensional space. The idea of direction is not tied to a single value like angle from the x-axis anymore.

However we can answer the question: how similar are the directions of two vectors represented by arrays [a,b,c,d,...] and [p,q,r,s,...]? For this question to make sense, the number of elements in these arrays should be the same. We will discuss this problem and the elegance of its solution later on.

---

A note on representing a vector:

An array when represented as a vector is shown with values stacked vertically like so:

---

2.3 Scaling and addition

2.3.1 Scaling a vector

Scaling a vector only changes its norm. It does not change its direction. E.g. given a vector one can modify the scale and see how it changes its magnitude.

• Increase scale
• Decrease scale
• Set scale = 0.5

Feel free to try out other values of scale and th vector.

2.3.2 Adding two or more vectors

One can only add vectors with the same dimension. Adding two vectors can be seen geometrically as placing the second vector at the point where the first vector ends. Then the vector which joins the origin to the head of the second vector is the sum. This can be extended to more than two vectors.

Say you are given four vectors va, vb, vc and vd.

• The sum va + vb can be visualized like this.

• The sum va + vb + vc can be visualized like this.

• Similarly the sum va + vb + vc + vd can be visualized like this.

2.3.3 Commutativity to symmetry

The order of vectors does not matter when adding them. In other words, addition is commutative:

We can visualize this commutativity by adding two vectors , in different orders - then or then . Note how the property of commutativity in algebra translates into the property of symmetry in geometry.

Given two vectors v1 and v2 , we can add them up in two different ways:

Note the symmetry about the output vector.

---

2.4 Combining scaling and addition

Now let us combine both operations we have seen so far - scaling and adding vectors. Below we take the three vectors , and , scale each one, and add the scaled vectors together.

You cannot change the vectors themselves, but you can change the scales , and that each vector is multiplied by:

Here is the scaled sum. Try tweaking the scales and watch how the scaled vectors are laid head-to-tail and their sum changes:

• : increase · decrease · reset to 1
• : increase · decrease · reset to 1
• : increase · decrease · reset to 1

---

2.4.1 Matrix-vector multiplication is just scaling and adding vectors

By now this heading should feel obvious: a matrix-vector multiplication is just a sum of scaled vectors. The vectors are the columns of the matrix and the scales are the elements of the vector. In other words, the scaled sum above is exactly

So whenever you see a matrix-vector product, you should be able to visualize it as scaling the columns of the matrix by the elements of the vector and adding the results.

2.4.2 A different point of view

In the next chapter, we will learn another way to look at matrix-vector multiplication.

---

← 1. Weighted means · 3. Matrix as a function →