MATH 323 Lecture 19

« previous | Thursday, November 1, 2012 | next »

Norm

${\vec {x}}\in \mathbb {R} ^{n}$

$\left\|{\vec {x}}\right\|={\sqrt {\sum _{i=1}^{n}{x_{i}}^{2}}}$, where $\left\|{\vec {x}}\right\|$ is notation for the norm or length of ${\vec {x}}$.

$\left\|{\vec {x}}\right\|=\left\|-{\vec {x}}\right\|$

Distance between two points/vectors: $\left\|{\vec {x}}-{\vec {y}}\right\|$

$\left\|{\vec {x}}\right\|^{2}={\vec {x}}\cdot {\vec {x}}=\sum _{i=1}^{n}{x_{i}}^{2}$

Theorem 5.1.1

If ${\vec {x}},{\vec {y}}\in \mathbb {R} ^{n}$ and $\theta$ is the angle between them, then

${\vec {x}}\cdot {\vec {y}}=\|{\vec {x}}\|\|{\vec {y}}\|\cos {\theta }$

Law of Cosines

Triangle defined by ${\vec {x}}$, ${\vec {y}}$, and ${\vec {x}}-{\vec {y}}$

$\left\|{\vec {y}}-{\vec {x}}\right\|^{2}=\left\|{\vec {x}}\right\|^{2}+\left\|{\vec {y}}\right\|^{2}-2\left\|{\vec {x}}\right\|\,\left\|{\vec {y}}\right\|\,\cos {\theta }$

The above can be simplified to the form

${\vec {x}}\cdot {\vec {y}}=\left\|{\vec {x}}\right\|\,\left\|{\vec {y}}\right\|\,\cos {\theta }$

Unit Vector

${\vec {u}}$ is a unit vector iff $\left\|{\vec {u}}\right\|=1$

Unit vector in direction of ${\vec {x}}$ is given by

${\vec {u}}={\hat {x}}={\frac {\vec {x}}{\left\|{\vec {x}}\right\|}}$

Example

${\begin{aligned}{\vec {x}}&=\left\langle 3,4\right\rangle &{\vec {y}}&=\left\langle -1,7\right\rangle \\{\vec {u}}&={\frac {1}{5}}\left\langle 3,4\right\rangle =\left\langle {\frac {3}{5}},{\frac {4}{5}}\right\rangle \\{\vec {v}}&={\frac {1}{5{\sqrt {2}}}}\left\langle -1,7\right\rangle =\left\langle -{\frac {1}{5{\sqrt {2}}}},{\frac {7}{5{\sqrt {2}}}}\right\rangle \\\cos {\theta }&={\vec {u}}\cdot {\vec {v}}=\dots ={\frac {1}{\sqrt {2}}}\\\theta &={\frac {pi}{4}}\end{aligned}}$

Cauchy-Schwartz Inequality

$\left|{\vec {x}}\cdot {\vec {y}}\right|\leq \left\|x\right\|\,\left\|y\right\|\iff -1\leq {\frac {{\vec {x}}\cdot {\vec {y}}}{\left\|{\vec {x}}\right\|\,\left\|{\vec {y}}\right\|}}\leq 1$

Equality iff one of the vectors is zero or one of the vectors is a multiple of another ($\theta =0,\pi$)

Orthogonality

${\vec {x}}$ and ${\vec {y}}$ are orthogonal (written ${\vec {x}}\perp {\vec {y}}$ if $\theta ={\frac {\pi }{2}},{\frac {3\pi }{2}}\iff {\vec {x}}\cdot {\vec {y}}=0$

Scalar and Vector Projections

Given two noncolinear vectors ${\vec {x}}$ and ${\vec {y}}$

Let ${\vec {u}}={\tfrac {\vec {y}}{\left\|{\vec {y}}\right\|}}$ be a unit vector in the direction of ${\vec {y}}$ and ${\vec {p}}=\alpha {\vec {u}}$ be the vector projection of ${\vec {x}}$ in the direction of ${\vec {y}}$, where $\alpha$ is the scalar projection of ${\vec {x}}$ in the direction of ${\vec {y}}$

${\begin{aligned}\left\|{\vec {p}}\right\|=\alpha &=\left\|{\vec {x}}\right\|\,\cos {\theta }\\&={\frac {\left\|{\vec {x}}\right\|\,\left\|{\vec {y}}\right\|\,\cos {\theta }}{\left\|{\vec {y}}\right\|}}\\&={\frac {{\vec {x}}\cdot {\vec {y}}}{\left\|{\vec {y}}\right\|}}\end{aligned}}$

Exercise

Given a point $P(1,4)$ and a line $y={\frac {1}{3}}\,x$, find the point on the line closest to $P$.

Find vector ${\vec {w}}$ on the line: ${\vec {w}}=\left\langle 3,1\right\rangle$

Take vector projection of ${\vec {v}}=P-O$ onto ${\vec {w}}$:

${\vec {p}}={\frac {{\vec {v}}\cdot {\vec {w}}}{\left\|{\vec {w}}\right\|}}\cdot {\vec {w}}=\left\langle 2.1,0.7\right\rangle$

Planes in 3D Space

(See MATH 251 Lecture 5#Planes→)

Generalization of Pythagorean Theorem

Given two orthogonal vectors ${\vec {x}}$ and ${\vec {y}}$ in $\mathbb {R} ^{n}$,

${\begin{aligned}\left\|{\vec {x}}\pm {\vec {y}}\right\|^{2}&=({\vec {x}}\pm {\vec {y}})\cdot ({\vec {x}}\pm {\vec {y}})\\&={\vec {x}}\cdot {\vec {x}}\pm 2{\vec {y}}\cdot {\vec {x}}+{\vec {y}}\cdot {\vec {y}}\\&=\left\|{\vec {x}}\right\|^{2}+\left\|{\vec {y}}\right\|^{2}\end{aligned}}$

Orthogonal Subspaces

Two subspaces $X$ and $Y$ in Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbb{R}^n} are said to be orthogonal if ${\vec {x}}\cdot {\vec {y}}=0$ for all ${\vec {x}}\in X$ and ${\vec {y}}\in Y$.

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A} is Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle m \times n} matrix

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle N(A)} is subspace, Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \vec{x} \in \mathbb{R}^n} , Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A \vec{x} = 0} iff Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle a_{i1}x_1 + a_{i2}x_2 + \dots + a_{in} + x_n = 0} for Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle i=1,\ldots, m} .

This means that Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \vec{x}} is perpendicular to the Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle i} ^th row of Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A}

...

Orthogonal Complement

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle Y \subset \mathbb{R}^n}

Orthogonal complement given by

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle Y^\perp = \{ \vec{x} \in \mathbb{R}^n ~\mid~ \vec{x} \cdot \vec{y} = 0 \quad \forall \vec{y} \in \vec{y} \}}

Example

Let Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle Y = \mathrm{Span}(\vec{e}_1)} be the Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x} -axis.

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle Y^\perp = \mathrm{Span}(\vec{e}_2, \vec{e}_3)} is the Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle yz} -plane

Theorem

1. If Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle X \perp Y} , then Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle X \cap Y = \{ \vec{0} \}}
2. If Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle Y} is a subspace of Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbb{R}^n} , then Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle Y^\perp} is also a subspace of Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbb{R}^n}

Proof

1. Proof by contradiction. Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle X \cap Y \ne \{\vec{0}\} \quad \longrightarrow \quad (\vec{x} \in X \cap Y) \cdot (\vec{x} \in X \cap Y) = \left\| x \right\|^2 \ne 0} — CONTRADICTION!
2. Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \vec{u}, \vec{v} \in Y^\perp} , so Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \vec{u} \cdot \vec{y} = \vec{v} \cdot \vec{y} = 0 \forall \vec{y} \in Y} . Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle (\alpha \vec{u} + \beta \vec{v}) \cdot \vec{y} = 0} , so subspace defined by Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \alpha \vec{u} + \beta \vec{v} \in Y^\perp}

Fundamental Subspaces

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle m \times n} matrix Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A}

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A \vec{x} = \vec{b}} for some Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \vec{x} \in \mathbb{R}^n} iff Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \vec{b}} is in col space of Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A} .

We can write Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A} as a linear transformation Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle L_A : \mathbb{R}^n \to \mathbb{R}^n} .

The column space of Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A} is also called the range of Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A} :

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle R(A) = \{\vec{b} \in \mathbb{R}^m ~\mid~ \vec{b} = A\vec{x} \exists \vec{x} \in \mathbb{R}^n \}}

Range of transpose matrix

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle R(A^T) = \{\vec{y} \in \mathbb{R}^n ~\mid~ \vec{y} = A^T \vec{x} \exists \vec{x} \in \mathbb{R}^m \}}

Thus Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle R(A) \subset \mathbb{R}^m} , and Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle R(A^T) \subset \mathbb{R}^n} , and

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle R(A^T) \perp N(A)}

Theorem 5.2.1

Fundamental Subspace Theorem

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle N(A) = R(A^T)^\perp}

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle N(A^T) = R(A)^\perp}