MATH 414 Lecture 2

« previous | Wednesday, January 15, 2014 | next »

Inner Product

Most common example is the vector dot product:

$⟨\overrightarrow{u},\overrightarrow{v}⟩=\overrightarrow{u}\cdot \overrightarrow{v}=\sum _i{u}_i{v}_i=\left|\overrightarrow{u}\right|\left|\overrightarrow{v}\right|\cos \theta$

Generalized Algebraic Structure

We induce a geometry on the algebraic definition

Length: $\Vert \overrightarrow{v}\Vert =\sqrt{⟨\overrightarrow{v},\overrightarrow{v}⟩}$

Angle Between: $\theta ={\cos }^{-1}\left(\frac{⟨\overrightarrow{u},\overrightarrow{v}⟩}{\Vert \overrightarrow{u}\Vert \Vert \overrightarrow{v}\Vert }\right)$

The most important case is where $\theta =\frac{\pi }{2}$, in which case $\cos \theta =0$ and—more importantly—the vectors $\overrightarrow{u}$ and $\overrightarrow{v}$ are perperdicular.

Properties of the Inner Product

Positivity: $⟨\overrightarrow{u},\overrightarrow{u}⟩>0$ unless $\overrightarrow{u}=\overrightarrow{0}$, in which case $⟨\overrightarrow{0},\overrightarrow{0}=0⟩$

Conjugate Symmetry: $⟨\overrightarrow{u},\overrightarrow{v}⟩=\overline{⟨\overrightarrow{v},\overrightarrow{u}⟩}$. In the case of real numbers, the property becomes commutativity: $⟨\overrightarrow{u},\overrightarrow{v}⟩=⟨\overrightarrow{v},\overrightarrow{u}⟩$

Homogenuity: $⟨c\overrightarrow{u},\overrightarrow{v}⟩=c⟨\overrightarrow{u},\overrightarrow{v}⟩$

Linearity: $⟨\overrightarrow{u}+\overrightarrow{w},\overrightarrow{v}⟩=⟨\overrightarrow{u},\overrightarrow{v}⟩+⟨\overrightarrow{w},\overrightarrow{v}⟩$

Real Spaces

for ${\mathbb{ℝ}}^n$, where $\overrightarrow{x}=\left[\begin{array}{c}{x}_1\\ ⋮\\ x_n\end{array}\right]$ and $\overrightarrow{y}=\left[\begin{array}{c}{y}_1\\ ⋮\\ y_n\end{array}\right]$, we have

$⟨\overrightarrow{x},\overrightarrow{y}⟩=y^{T}x=\sum _{k=1}^n{x}_k{y}_k$

Complex Spaces

for ${\mathbb{ℂ}}^n$, $⟨\overrightarrow{x},\overrightarrow{y}⟩$ is given by ${\overrightarrow{y}}^{*}\overrightarrow{x}$ (also written ${\overrightarrow{y}}^H\overrightarrow{x}$), where ${\overrightarrow{y}}^{*}={\overrightarrow{y}}^H$ is the conjugate transpose of $\overrightarrow{y}$.

Example: $\overrightarrow{y}=\left[\begin{array}{c}i\\ 1\\ -1+2i\\ 3+i\end{array}\right]$, ${\overrightarrow{y}}^{*}=\left[\begin{array}{cccc}-i& 1& -1-2i& 3-i\end{array}\right]$

Some concrete implementations of inner products may not satisfy all properties for all members of the space.

Signal Spaces

Continuous and Discrete

A signal is just $f(t)$ measured on $[a,b]$.

Signals and Circuits

For current $I(t)$ (essentially our function $f(t)$)

We can measure voltage drop of a signal $V(t)=RI(t)$

Power at time $t$ is given by $P(t)=VI(t)=R{I}^2(t)$

Energy in time interval $[a,b]$ is given by $E=\underset{a}{\overset{b}{\int }}P(t)\mathrm{d}t=R\underset{a}{\overset{b}{\int }}{|I(t)|}^2\mathrm{d}t$.

Hence $E=\underset{a}{\overset{b}{\int }}{|f(t)|}^2\mathrm{d}t$

Finite Energy Signals

Complex valued $f(t)$ gives amplitude and phase.

Finite energy: $\underset{a}{\overset{b}{\int }}{|f(t)|}^2\mathrm{d}t<\mathrm{\infty }$. Moreover, $\underset{\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}{|f(t)|}^2\mathrm{d}t<\mathrm{\infty }$

Signals don't need to be continuous (digital circuit signals)

Continuous Signal Space

All signals with finite energy fit into a space:

$L^2=\{f:[a,b]\to \mathbb{ℂ}\mid \underset{a}{\overset{b}{\int }}{|f(t)|}^2\mathrm{d}t<\mathrm{\infty }\}$

The $L$ comes from mathematician Lebesgue (ca. 1900), and the "square" comes from the square in the integral. The reason for this name is because integral we compute is a Lebesgue integral, not a Riemann integral.

We define an inner product on $L^2$ as

${⟨f,g⟩}_{L^2[a,b]}=\underset{a}{\overset{b}{\int }}f(t)\overline{g(t)}\mathrm{d}t$

Discrete Signal Space

Notation: $l^2$

By sampling the function at fixed points, we obtain a sequence $X={\left\{x_n\right\}}_{n=-\mathrm{\infty }}^{\mathrm{\infty }}=(\dots ,x_{-1},x_0,x_1,\dots )$

Thus $l^2\subset L^2$.

We define our inner product by a discrete sum:

${⟨X,Y⟩}_{l^2}=\sum _{n=-\mathrm{\infty }}^{\mathrm{\infty }}{x}_n{\overline{y}}_n$

The energy in a discrete signal is similarly given by

$\sum _{n=-\mathrm{\infty }}^{\mathrm{\infty }}{\left|x_n\right|}^2$

Inner Product Spaces

Schwarz's Inequality

Using axioms from definition, we arrive at

$\left|⟨\overrightarrow{u},\overrightarrow{v}⟩\right|\le \Vert \overrightarrow{u}\Vert \Vert \overrightarrow{u}\Vert$, where $\Vert \overrightarrow{u}\Vert =\sqrt{⟨\overrightarrow{u},\overrightarrow{u}⟩}$ represents the norm of a vector $\overrightarrow{u}\in V$ ("LENGTH")

Triangle Inequality

Property of length:

$\Vert \overrightarrow{u}+\overrightarrow{v}\Vert \le \Vert \overrightarrow{u}\Vert +\Vert \overrightarrow{v}\Vert$.

"The sum of the lengths any two sides of a triangle must be at least the length of the third side."