MATH 323 Lecture 19

« previous | Thursday, November 1, 2012 | next »

Norm

${\vec {x}}\in \mathbb {R} ^{n}$

$\left\|{\vec {x}}\right\|={\sqrt {\sum _{i=1}^{n}{x_{i}}^{2}}}$, where $\left\|{\vec {x}}\right\|$ is notation for the norm or length of ${\vec {x}}$.

$\left\|{\vec {x}}\right\|=\left\|-{\vec {x}}\right\|$

Distance between two points/vectors: $\left\|{\vec {x}}-{\vec {y}}\right\|$

$\left\|{\vec {x}}\right\|^{2}={\vec {x}}\cdot {\vec {x}}=\sum _{i=1}^{n}{x_{i}}^{2}$

Theorem 5.1.1

If ${\vec {x}},{\vec {y}}\in \mathbb {R} ^{n}$ and $\theta$ is the angle between them, then

${\vec {x}}\cdot {\vec {y}}=\|{\vec {x}}\|\|{\vec {y}}\|\cos {\theta }$

Law of Cosines

Triangle defined by ${\vec {x}}$, ${\vec {y}}$, and ${\vec {x}}-{\vec {y}}$

$\left\|{\vec {y}}-{\vec {x}}\right\|^{2}=\left\|{\vec {x}}\right\|^{2}+\left\|{\vec {y}}\right\|^{2}-2\left\|{\vec {x}}\right\|\,\left\|{\vec {y}}\right\|\,\cos {\theta }$

The above can be simplified to the form

${\vec {x}}\cdot {\vec {y}}=\left\|{\vec {x}}\right\|\,\left\|{\vec {y}}\right\|\,\cos {\theta }$

Unit Vector

${\vec {u}}$ is a unit vector iff $\left\|{\vec {u}}\right\|=1$

Unit vector in direction of ${\vec {x}}$ is given by

${\vec {u}}={\hat {x}}={\frac {\vec {x}}{\left\|{\vec {x}}\right\|}}$

Example

${\begin{aligned}{\vec {x}}&=\left\langle 3,4\right\rangle &{\vec {y}}&=\left\langle -1,7\right\rangle \\{\vec {u}}&={\frac {1}{5}}\left\langle 3,4\right\rangle =\left\langle {\frac {3}{5}},{\frac {4}{5}}\right\rangle \\{\vec {v}}&={\frac {1}{5{\sqrt {2}}}}\left\langle -1,7\right\rangle =\left\langle -{\frac {1}{5{\sqrt {2}}}},{\frac {7}{5{\sqrt {2}}}}\right\rangle \\\cos {\theta }&={\vec {u}}\cdot {\vec {v}}=\dots ={\frac {1}{\sqrt {2}}}\\\theta &={\frac {pi}{4}}\end{aligned}}$

Cauchy-Schwartz Inequality

$\left|{\vec {x}}\cdot {\vec {y}}\right|\leq \left\|x\right\|\,\left\|y\right\|\iff -1\leq {\frac {{\vec {x}}\cdot {\vec {y}}}{\left\|{\vec {x}}\right\|\,\left\|{\vec {y}}\right\|}}\leq 1$

Equality iff one of the vectors is zero or one of the vectors is a multiple of another ($\theta =0,\pi$)

Orthogonality

${\vec {x}}$ and ${\vec {y}}$ are orthogonal (written ${\vec {x}}\perp {\vec {y}}$ if $\theta ={\frac {\pi }{2}},{\frac {3\pi }{2}}\iff {\vec {x}}\cdot {\vec {y}}=0$

Scalar and Vector Projections

Given two noncolinear vectors ${\vec {x}}$ and ${\vec {y}}$

Let ${\vec {u}}={\tfrac {\vec {y}}{\left\|{\vec {y}}\right\|}}$ be a unit vector in the direction of ${\vec {y}}$ and ${\vec {p}}=\alpha {\vec {u}}$ be the vector projection of ${\vec {x}}$ in the direction of ${\vec {y}}$, where $\alpha$ is the scalar projection of ${\vec {x}}$ in the direction of ${\vec {y}}$

${\begin{aligned}\left\|{\vec {p}}\right\|=\alpha &=\left\|{\vec {x}}\right\|\,\cos {\theta }\\&={\frac {\left\|{\vec {x}}\right\|\,\left\|{\vec {y}}\right\|\,\cos {\theta }}{\left\|{\vec {y}}\right\|}}\\&={\frac {{\vec {x}}\cdot {\vec {y}}}{\left\|{\vec {y}}\right\|}}\end{aligned}}$

Exercise

Given a point $P(1,4)$ and a line $y={\frac {1}{3}}\,x$, find the point on the line closest to $P$.

Find vector ${\vec {w}}$ on the line: ${\vec {w}}=\left\langle 3,1\right\rangle$

Take vector projection of ${\vec {v}}=P-O$ onto ${\vec {w}}$:

${\vec {p}}={\frac {{\vec {v}}\cdot {\vec {w}}}{\left\|{\vec {w}}\right\|}}\cdot {\vec {w}}=\left\langle 2.1,0.7\right\rangle$

Planes in 3D Space

(See MATH 251 Lecture 5#Planes→)

Generalization of Pythagorean Theorem

Given two orthogonal vectors ${\vec {x}}$ and ${\vec {y}}$ in $\mathbb {R} ^{n}$,

${\begin{aligned}\left\|{\vec {x}}\pm {\vec {y}}\right\|^{2}&=({\vec {x}}\pm {\vec {y}})\cdot ({\vec {x}}\pm {\vec {y}})\\&={\vec {x}}\cdot {\vec {x}}\pm 2{\vec {y}}\cdot {\vec {x}}+{\vec {y}}\cdot {\vec {y}}\\&=\left\|{\vec {x}}\right\|^{2}+\left\|{\vec {y}}\right\|^{2}\end{aligned}}$

Orthogonal Subspaces

Two subspaces $X$ and $Y$ in $\mathbb {R} ^{n}$ are said to be orthogonal if ${\vec {x}}\cdot {\vec {y}}=0$ for all ${\vec {x}}\in X$ and ${\vec {y}}\in Y$.

$A$ is $m\times n$ matrix

$N(A)$ is subspace, ${\vec {x}}\in \mathbb {R} ^{n}$, $A{\vec {x}}=0$ iff $a_{i1}x_{1}+a_{i2}x_{2}+\dots +a_{in}+x_{n}=0$ for $i=1,\ldots ,m$.

This means that ${\vec {x}}$ is perpendicular to the $i$^th row of $A$

...

Orthogonal Complement

$Y\subset \mathbb {R} ^{n}$

Orthogonal complement given by

$Y^{\perp }=\{{\vec {x}}\in \mathbb {R} ^{n}~\mid ~{\vec {x}}\cdot {\vec {y}}=0\quad \forall {\vec {y}}\in {\vec {y}}\}$

Example

Let $Y=\mathrm {Span} ({\vec {e}}_{1})$ be the $x$-axis.

$Y^{\perp }=\mathrm {Span} ({\vec {e}}_{2},{\vec {e}}_{3})$ is the $yz$-plane

Theorem

1. If $X\perp Y$, then $X\cap Y=\{{\vec {0}}\}$
2. If $Y$ is a subspace of $\mathbb {R} ^{n}$, then $Y^{\perp }$ is also a subspace of $\mathbb {R} ^{n}$

Proof

1. Proof by contradiction. $X\cap Y\neq \{{\vec {0}}\}\quad \longrightarrow \quad ({\vec {x}}\in X\cap Y)\cdot ({\vec {x}}\in X\cap Y)=\left\|x\right\|^{2}\neq 0$ — CONTRADICTION!
2. ${\vec {u}},{\vec {v}}\in Y^{\perp }$, so ${\vec {u}}\cdot {\vec {y}}={\vec {v}}\cdot {\vec {y}}=0\forall {\vec {y}}\in Y$. $(\alpha {\vec {u}}+\beta {\vec {v}})\cdot {\vec {y}}=0$, so subspace defined by $\alpha {\vec {u}}+\beta {\vec {v}}\in Y^{\perp }$

Fundamental Subspaces

$m\times n$ matrix $A$

$A{\vec {x}}={\vec {b}}$ for some ${\vec {x}}\in \mathbb {R} ^{n}$ iff ${\vec {b}}$ is in col space of $A$.

We can write $A$ as a linear transformation $L_{A}:\mathbb {R} ^{n}\to \mathbb {R} ^{n}$.

The column space of $A$ is also called the range of $A$:

$R(A)=\{{\vec {b}}\in \mathbb {R} ^{m}~\mid ~{\vec {b}}=A{\vec {x}}\exists {\vec {x}}\in \mathbb {R} ^{n}\}$

Range of transpose matrix

$R(A^{T})=\{{\vec {y}}\in \mathbb {R} ^{n}~\mid ~{\vec {y}}=A^{T}{\vec {x}}\exists {\vec {x}}\in \mathbb {R} ^{m}\}$

Thus $R(A)\subset \mathbb {R} ^{m}$, and $R(A^{T})\subset \mathbb {R} ^{n}$, and

$R(A^{T})\perp N(A)$

Theorem 5.2.1

Fundamental Subspace Theorem

$N(A)=R(A^{T})^{\perp }$

$N(A^{T})=R(A)^{\perp }$