Orthogonal Sets

Orthogonal sets

Definition: orthogonal vectors mean they are perpendicular. If this is true then they are orthogonal. Intro:

The dot product of orthogonal vectors equals zero. All of a sudden it clicked when I remembered my conclusion as to what a dot product actually was, that is, "what amount of one vector goes in the direction of another." Basically, if vectors are orthogonal, then no amount of one will go in the direction of the other. Like how a tree casts no shadow at noon.

it's a row x a column.

We're going to show that the angle between the subspaces are orthogonal.

We know that x^2 + y^2 = (x+y)^2 thanks to pythagore.

Say we have x, we find an orthogonal vector y, and we know that is true because:

This works because: (the following works only when we have a right triangle).

everything cancels and so

Therefore the dot product of orthogonal vector = 0.

Definition Orthogonal Subspaces;

This means that if 2 vectors meet or intersect, for sure they are not orthogonal. The wall and the floor, if we think of them as subspaces are not orthogonal.

Now why is the row space orthogonal to the null space? (as per the screenshot above?)

Well, our nullspace is Ax=0, which means we have

And since each row x X = 0, they are orthogonal. Also we know that x is orthogonal to all the rowspace, meaning orthogonal to row1 x c1, row2 x c2, etc, which is true.

The same is true that the transpose of the nullspace of A is orthogonal to the columnspace of A.

Example: in 3 dimensions, if we have this line A:

Its dimension = 1, and threfore the dimension of the nullspace is 2. Therefore, the line A is orthogonal to the plane N(A). We couldn't have had just 2 lines in R^3 being orthogonal, because the dimensions don't add up to 3.

The main rule is that

Solving Ax=b when there are no solutions

In real life data, sometimes there will be noise in b and we have too many equations.

Pulse rate: you measure it multiple times.

We can't expect to solve Ax=b and have it exactly right, there's errors, mistakes in b. But there's a lot of info in x. Let's try to separate the noise from the X. What's the best solution? What do we do when elimination will tell you there are no solutions?

It's interesting to look at AT x A matrix. What do we know about it? it's square, symetric:

We also want to know is it invertable? If not what's its nullspace?

the central formula we'll want to solve is this one: with an X-hat that we hope will have a solution.

That's why we're interested in AT A and its invertability. So when is it invertable?

A A T is invertable exactly if A has independant columns, which means N(A) = 0.