MATH 414 Lecture 3

« previous | Friday, January 17, 2014 | next »

Distance

Difference in length in a vector space:

$\mathrm {dist} \left({\vec {u}},{\vec {v}}\right)=\left\|{\vec {u}}-{\vec {v}}\right\|$

In signal space $L^{2}$, recall

$\left\|f\right\|={\sqrt {\int _{a}^{b}\left|f(t)\right|^{2}\,\mathrm {d} t}}$

Hence $\mathrm {dist} \left(f,g\right)={\sqrt {\int _{a}^{b}\left|f(t)-g(t)\right|^{2}\,\mathrm {d} t}}$

Convergence

Approximation

If we have a sequence of functions $\left\{f_{n}(t)\right\}_{n=1}^{\infty }$

1. Naïve convergence: Fix $t$. then $\lim _{n\to \infty }f_{n}(t)=f(t)$
2. Uniform convergence: (Visual convergence): A sequence $f_{n}$ converges uniformly to $f$ on $\left[a,b\right]$ if and only if for all $\epsilon >0$ there exists a $N$ depending only on $\epsilon$ such that $\left|f_{n}(t)-f(t)\right|<\epsilon$ for all $n\geq N$ and $t\in \left[a,b\right]$.
3. Convergence in the mean or in $L^{2}$:

Example

$f_{n}(t)=t^{n}$ $0\leq t<1$.

Fix $t$. Let $n\to \infty$ and $f_{n}(t)$ converges to $0$.

Naïve convergence

observe $t^{n}=\mathrm {e} ^{-n\,\log {\left({\frac {1}{t}}\right)}}$. For $t^{N}=\epsilon$, we have $N={\frac {\log {\epsilon }}{\log {t}}}$.

for $n\geq N$, $t^{n}<t^{N}<\epsilon$

Uniform Convergence

$t^{n}$ does not converge uniformly since for any $\epsilon <1$ and $N$, $t^{n}\to 1$ as $t\to 1$

$L^{2}$

$\int _{a}^{b}\left|t^{n}\right|^{2}\,\mathrm {d} t=\int _{a}^{b}t^{2n}\,\mathrm {d} t=\left.{\frac {1}{2n+1}}t^{2n+1}\right|_{a}^{b}={\frac {1}{2n+1}}$, which converges to $0$ as $n\to \infty$. Therefore $f$ converges in $L^{2}$.

Note: Uniform convergence implies both Naïve convergence and $L^{2}$ convergence, but neither implies uniform, and they do not imply each other.

Orthogonality and Subspaces

We say that ${\vec {u}}$ and ${\vec {v}}$ are orthogonal if and only if $\left\langle {\vec {u}},{\vec {v}}\right\rangle =0$.

We say that $U\subset V$ is a subspace (not empty, but ${\vec {0}}\in U$), then ${\vec {v}}$ is orthogonal to $U$ when ${\vec {v}}\perp {\vec {u}}$ for all ${\vec {u}}\in U$.

We say that $\left\{u_{1},u_{2},\ldots ,u_{n}\right\}$ is an orthonormal set iff $\left\langle {\vec {u}}_{j},{\vec {u}}_{k}\right\rangle =\delta _{j,k}$, where $\delta _{j,k}$ is the $j,k$-th entry in the identity matrix; that is,

$\delta _{j,k}={\begin{cases}1&j=k\\0&{\mbox{otherwise}}\end{cases}}$

1. An orthonormal set is always linearly independent
2. If $U=\mathrm {Span} \left\{{\vec {u}}_{1},\ldots ,{\vec {u}}_{n}\right\}\subseteq V$, then ${\vec {u}}\in U$ can be written as ${\vec {u}}=\sum _{j=1}^{n}\left\langle {\vec {u}},{\vec {u}}_{j}\right\rangle {\vec {u}}_{j}$ (think ${\hat {\imath }}$, ${\hat {\jmath }}$, and Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \math{k}} in $\mathbb {R} ^{3}$)
3. Orthonormal projection onto $U$ of a vector ${\vec {v}}\in V$ is defined as ${\vec {p}}=\mathrm {Proj} _{U}\left({\vec {v}}\right)=\sum _{j=1}^{n}\left\langle {\vec {v}},{\vec {u}}_{j}\right\rangle \,{\vec {u}}_{j}$, where ${\vec {v}}-{\vec {p}}\perp U=\mathrm {Span} \left\{{\vec {u}}_{1},\ldots ,{\vec {u}}_{n}\right\}$.