Tiny Volt1. Weighted means
Let's start with calculating a weighted sum for an array of numbers where the weights are given by the array
def get_weighted_sum(values, weights):
return sum((v*w) for (v,w) in zip(values, weights))
It can be seen as a :
a) we scale each value by its corresponding weight
b) we add the scaled values up
def scale(value, weight):
return value * weight
def add(scaled_values):
return sum(scaled_values)
def get_weighted_sum(values, weights):
return add(scale(v, w) for (v, w) in zip(values, weights))
One thing we achieve by doing so is separation of logic for scaling the values and adding them. For instance, you can provide your own definitions of scaling and adding and pass them as parameters to get_weighted_sum:
def get_weighted_sum(values, weights, *, scale_function, add_function):
return add_function(
scale_function(v, w)
for (v, w) in zip(values, weights)
)
1.1. How to scale and add
Now we can simply focus on different ways to scale and add two values together. You already saw the scale and add functions above. But there are many other possible ways to scale and add.
One way is described below (this leads to the exciting area of Tropical Geometry). Here is an of it.
def tropical_scale(value, weight):
return value + weight
def tropical_add(scaled_values):
return max(scaled_values)
weighted_sum = get_weighted_sum(
[1,2,3], [6,1,3],
scale_function = tropical_scale,
add_function = tropical_add
)
However let us stick to the conventional notions of scaling and adding:
- we scale two numbers by multiplying them -
scale(x,y) = x*y - we add two numbers by summing them up -
add(x,y) = x+y.
But we are going to get creative with what to scale and add.
Instead of numbers, we will scale and add arrays of numbers.
How do we scale an array array_of_numbers with a given weight weight? Scale each value in the array with the given weight.
def scale_array(array_of_numbers, weight):
return [number * weight for number in array_of_numbers]
How do we add two or more arrays of numbers? Do an elementwise addition (all arrays need to have the same length).
def add_arrays(array_of_arrays):
result = [0] * len(array_of_arrays[0])
for array in array_of_arrays:
result = [(r+v) for (r,v) in zip(result, array)]
return result
Then we can pass these functions to get_weighted_sum along with the inputs to scale and add.
1.2. Weighted sums are everywhere
Here is an of the examples that follow.
1.2.1. Indexing
Given an array array = [x,y,z], its i-th element (counting from 0) is array[i]. This can be seen as a weighted sum where the weight corresponding to the i-th element is 1 and all other weights are zero.
array, weights = [2,3,5], [0,0,1]
index = 2
# this is the same as
# array_at_index = array[index]
array_at_index = get_weighted_sum(
array, weights,
scale_function = scale,
add_function = add
)
1.2.2. Expected value of a random variable
Given a six-sided die with values and probabilities , the expected value is:
expected_value = get_weighted_sum(
values, probs,
scale_function = scale,
add_function = add
)
Now let's look at some examples of weighted sums of arrays of numbers:
1.2.3. Rotation
A point on a unit circle with center at origin is given by . It can be seen as a weighted sum of two arrays and (what are the corresponding weights?):
import math
v1, v2 = [1,0], [0,1]
angle_in_radians = math.pi / 6
weights = [math.cos(angle_in_radians), math.sin(angle_in_radians)]
get_weighted_sum(
[v1, v2], weights,
scale_function = scale_array,
add_function = add_arrays
)
1.2.4. Polynomials
A polynomial itself is nothing but a weighted sum:
But a more interesting thing to note is that two or more polynomials can be scaled and added just like arrays of numbers. Given two polynomials
their sum is given by:
Similarly, a polynomial can be scaled by a weight like so:
1.3. A new format for weighted sums
A good format can greatly influence the ease of grasping an idea. Using an array of arrays seems a bit cumbersome. It obfuscates some obvious properties of scaling and adding arrays. Let's use a different format to represent an array. We will represent the array [x1,x2,x3] as:
The format for scaling [x1,x2,x3] with a weight w is:
The format for adding two (or more) arrays [x1, x2, x3] and [y1, y2, y3]
1.4. Some observations
First thing to note with this choice of scale and add - each item is itself a weighted mean
This observation allows us to implement get_weighted_sum for an array of arrays in yet another way which is hard to understand:
def get_weighted_sum_for_array_of_arrays(array_of_arrays, weights):
return [
get_weighted_sum(
[array[i] for array in array_of_arrays],
weights,
scale_function=scale,
add_function=add
)
for i in range(len(array_of_arrays[0]))
]
It is easy to see what we are trying to say using our newly concocted format:
We can simplify the format even more by stacking all the arrays as columns and removing the text since it is all what we have been talking about.
1.4.1. Who is scaling whom?
Note that an array of numbers values = [x,y,z] scaled by weights = [w1, w2, w3] and then summed up is the same as weights scaled by values and then summed up:
Using our new format, we can show this as:
This seemingly innocous equation has some really beautiful interpretations as we will see next.
1.5. What was this all about? What's next?
We began with taking weighted sums of numbers and then moved on to taking weighted sums of arrays. When we realized the code was getting messier, we came up with a different way to represent weighted sums of arrays. This new format made it easier to look at what was going on. It also enabled us to do some simple algebra on arrays.
1.5.1 Giving arrays a geometric setting
The arrays [1,1,1] and [2,2,2] seem quite similar in some way. In fact they are just scaled versions of each other.
Similarly [1,2,3] and [0.9, 2, 3.1] seem to be close to each other. This is because the values in the arrays are quite similar.
In order to draw some insights from these observations, a good way is to visualize them. In other words, we give these arrays some geometry. This geometry when combined with the algebra using the format we came up with gives rise to some beautiful things.