# MATH 323 Lecture 19

« previous | Thursday, November 1, 2012 | next »

## Norm

${\displaystyle {\vec {x}}\in \mathbb {R} ^{n}}$

${\displaystyle \left\|{\vec {x}}\right\|={\sqrt {\sum _{i=1}^{n}{x_{i}}^{2}}}}$, where ${\displaystyle \left\|{\vec {x}}\right\|}$ is notation for the norm or length of ${\displaystyle {\vec {x}}}$.

${\displaystyle \left\|{\vec {x}}\right\|=\left\|-{\vec {x}}\right\|}$

Distance between two points/vectors: ${\displaystyle \left\|{\vec {x}}-{\vec {y}}\right\|}$

${\displaystyle \left\|{\vec {x}}\right\|^{2}={\vec {x}}\cdot {\vec {x}}=\sum _{i=1}^{n}{x_{i}}^{2}}$

#### Theorem 5.1.1

If ${\displaystyle {\vec {x}},{\vec {y}}\in \mathbb {R} ^{n}}$ and ${\displaystyle \theta }$ is the angle between them, then

${\displaystyle {\vec {x}}\cdot {\vec {y}}=\|{\vec {x}}\|\|{\vec {y}}\|\cos {\theta }}$

## Law of Cosines

Triangle defined by ${\displaystyle {\vec {x}}}$, ${\displaystyle {\vec {y}}}$, and ${\displaystyle {\vec {x}}-{\vec {y}}}$

${\displaystyle \left\|{\vec {y}}-{\vec {x}}\right\|^{2}=\left\|{\vec {x}}\right\|^{2}+\left\|{\vec {y}}\right\|^{2}-2\left\|{\vec {x}}\right\|\,\left\|{\vec {y}}\right\|\,\cos {\theta }}$

The above can be simplified to the form

${\displaystyle {\vec {x}}\cdot {\vec {y}}=\left\|{\vec {x}}\right\|\,\left\|{\vec {y}}\right\|\,\cos {\theta }}$

## Unit Vector

${\displaystyle {\vec {u}}}$ is a unit vector iff ${\displaystyle \left\|{\vec {u}}\right\|=1}$

Unit vector in direction of ${\displaystyle {\vec {x}}}$ is given by

${\displaystyle {\vec {u}}={\hat {x}}={\frac {\vec {x}}{\left\|{\vec {x}}\right\|}}}$

### Example

{\displaystyle {\begin{aligned}{\vec {x}}&=\left\langle 3,4\right\rangle &{\vec {y}}&=\left\langle -1,7\right\rangle \\{\vec {u}}&={\frac {1}{5}}\left\langle 3,4\right\rangle =\left\langle {\frac {3}{5}},{\frac {4}{5}}\right\rangle \\{\vec {v}}&={\frac {1}{5{\sqrt {2}}}}\left\langle -1,7\right\rangle =\left\langle -{\frac {1}{5{\sqrt {2}}}},{\frac {7}{5{\sqrt {2}}}}\right\rangle \\\cos {\theta }&={\vec {u}}\cdot {\vec {v}}=\dots ={\frac {1}{\sqrt {2}}}\\\theta &={\frac {pi}{4}}\end{aligned}}}

### Cauchy-Schwartz Inequality

${\displaystyle \left|{\vec {x}}\cdot {\vec {y}}\right|\leq \left\|x\right\|\,\left\|y\right\|\iff -1\leq {\frac {{\vec {x}}\cdot {\vec {y}}}{\left\|{\vec {x}}\right\|\,\left\|{\vec {y}}\right\|}}\leq 1}$

Equality iff one of the vectors is zero or one of the vectors is a multiple of another (${\displaystyle \theta =0,\pi }$)

## Orthogonality

${\displaystyle {\vec {x}}}$ and ${\displaystyle {\vec {y}}}$ are orthogonal (written ${\displaystyle {\vec {x}}\perp {\vec {y}}}$ if ${\displaystyle \theta ={\frac {\pi }{2}},{\frac {3\pi }{2}}\iff {\vec {x}}\cdot {\vec {y}}=0}$

## Scalar and Vector Projections

Given two noncolinear vectors ${\displaystyle {\vec {x}}}$ and ${\displaystyle {\vec {y}}}$

Let ${\displaystyle {\vec {u}}={\tfrac {\vec {y}}{\left\|{\vec {y}}\right\|}}}$ be a unit vector in the direction of ${\displaystyle {\vec {y}}}$ and ${\displaystyle {\vec {p}}=\alpha {\vec {u}}}$ be the vector projection of ${\displaystyle {\vec {x}}}$ in the direction of ${\displaystyle {\vec {y}}}$, where ${\displaystyle \alpha }$ is the scalar projection of ${\displaystyle {\vec {x}}}$ in the direction of ${\displaystyle {\vec {y}}}$

{\displaystyle {\begin{aligned}\left\|{\vec {p}}\right\|=\alpha &=\left\|{\vec {x}}\right\|\,\cos {\theta }\\&={\frac {\left\|{\vec {x}}\right\|\,\left\|{\vec {y}}\right\|\,\cos {\theta }}{\left\|{\vec {y}}\right\|}}\\&={\frac {{\vec {x}}\cdot {\vec {y}}}{\left\|{\vec {y}}\right\|}}\end{aligned}}}

### Exercise

Given a point ${\displaystyle P(1,4)}$ and a line ${\displaystyle y={\frac {1}{3}}\,x}$, find the point on the line closest to ${\displaystyle P}$.

Find vector ${\displaystyle {\vec {w}}}$ on the line: ${\displaystyle {\vec {w}}=\left\langle 3,1\right\rangle }$

Take vector projection of ${\displaystyle {\vec {v}}=P-O}$ onto ${\displaystyle {\vec {w}}}$:

${\displaystyle {\vec {p}}={\frac {{\vec {v}}\cdot {\vec {w}}}{\left\|{\vec {w}}\right\|}}\cdot {\vec {w}}=\left\langle 2.1,0.7\right\rangle }$

## Planes in 3D Space

(See MATH 251 Lecture 5#Planes→)

## Generalization of Pythagorean Theorem

Given two orthogonal vectors ${\displaystyle {\vec {x}}}$ and ${\displaystyle {\vec {y}}}$ in ${\displaystyle \mathbb {R} ^{n}}$,

{\displaystyle {\begin{aligned}\left\|{\vec {x}}\pm {\vec {y}}\right\|^{2}&=({\vec {x}}\pm {\vec {y}})\cdot ({\vec {x}}\pm {\vec {y}})\\&={\vec {x}}\cdot {\vec {x}}\pm 2{\vec {y}}\cdot {\vec {x}}+{\vec {y}}\cdot {\vec {y}}\\&=\left\|{\vec {x}}\right\|^{2}+\left\|{\vec {y}}\right\|^{2}\end{aligned}}}

## Orthogonal Subspaces

Two subspaces ${\displaystyle X}$ and ${\displaystyle Y}$ in ${\displaystyle \mathbb {R} ^{n}}$ are said to be orthogonal if ${\displaystyle {\vec {x}}\cdot {\vec {y}}=0}$ for all ${\displaystyle {\vec {x}}\in X}$ and ${\displaystyle {\vec {y}}\in Y}$.

${\displaystyle A}$ is ${\displaystyle m\times n}$ matrix

${\displaystyle N(A)}$ is subspace, ${\displaystyle {\vec {x}}\in \mathbb {R} ^{n}}$, ${\displaystyle A{\vec {x}}=0}$ iff ${\displaystyle a_{i1}x_{1}+a_{i2}x_{2}+\dots +a_{in}+x_{n}=0}$ for ${\displaystyle i=1,\ldots ,m}$.

This means that ${\displaystyle {\vec {x}}}$ is perpendicular to the ${\displaystyle i}$th row of ${\displaystyle A}$

...

### Orthogonal Complement

${\displaystyle Y\subset \mathbb {R} ^{n}}$

Orthogonal complement given by

${\displaystyle Y^{\perp }=\{{\vec {x}}\in \mathbb {R} ^{n}~\mid ~{\vec {x}}\cdot {\vec {y}}=0\quad \forall {\vec {y}}\in {\vec {y}}\}}$

#### Example

Let ${\displaystyle Y=\mathrm {Span} ({\vec {e}}_{1})}$ be the ${\displaystyle x}$-axis.

${\displaystyle Y^{\perp }=\mathrm {Span} ({\vec {e}}_{2},{\vec {e}}_{3})}$ is the ${\displaystyle yz}$-plane

### Theorem

1. If ${\displaystyle X\perp Y}$, then ${\displaystyle X\cap Y=\{{\vec {0}}\}}$
2. If ${\displaystyle Y}$ is a subspace of ${\displaystyle \mathbb {R} ^{n}}$, then ${\displaystyle Y^{\perp }}$ is also a subspace of ${\displaystyle \mathbb {R} ^{n}}$

#### Proof

1. Proof by contradiction. ${\displaystyle X\cap Y\neq \{{\vec {0}}\}\quad \longrightarrow \quad ({\vec {x}}\in X\cap Y)\cdot ({\vec {x}}\in X\cap Y)=\left\|x\right\|^{2}\neq 0}$ — CONTRADICTION!
2. ${\displaystyle {\vec {u}},{\vec {v}}\in Y^{\perp }}$, so ${\displaystyle {\vec {u}}\cdot {\vec {y}}={\vec {v}}\cdot {\vec {y}}=0\forall {\vec {y}}\in Y}$. ${\displaystyle (\alpha {\vec {u}}+\beta {\vec {v}})\cdot {\vec {y}}=0}$, so subspace defined by ${\displaystyle \alpha {\vec {u}}+\beta {\vec {v}}\in Y^{\perp }}$

## Fundamental Subspaces

${\displaystyle m\times n}$ matrix ${\displaystyle A}$

${\displaystyle A{\vec {x}}={\vec {b}}}$ for some ${\displaystyle {\vec {x}}\in \mathbb {R} ^{n}}$ iff ${\displaystyle {\vec {b}}}$ is in col space of ${\displaystyle A}$.

We can write ${\displaystyle A}$ as a linear transformation ${\displaystyle L_{A}:\mathbb {R} ^{n}\to \mathbb {R} ^{n}}$.

The column space of ${\displaystyle A}$ is also called the range of ${\displaystyle A}$:

${\displaystyle R(A)=\{{\vec {b}}\in \mathbb {R} ^{m}~\mid ~{\vec {b}}=A{\vec {x}}\exists {\vec {x}}\in \mathbb {R} ^{n}\}}$

### Range of transpose matrix

${\displaystyle R(A^{T})=\{{\vec {y}}\in \mathbb {R} ^{n}~\mid ~{\vec {y}}=A^{T}{\vec {x}}\exists {\vec {x}}\in \mathbb {R} ^{m}\}}$

Thus ${\displaystyle R(A)\subset \mathbb {R} ^{m}}$, and ${\displaystyle R(A^{T})\subset \mathbb {R} ^{n}}$, and

${\displaystyle R(A^{T})\perp N(A)}$

## Theorem 5.2.1

Fundamental Subspace Theorem

${\displaystyle N(A)=R(A^{T})^{\perp }}$

${\displaystyle N(A^{T})=R(A)^{\perp }}$