MATH 414 Lecture 2

« previous | Wednesday, January 15, 2014 | next »

Inner Product

Most common example is the vector dot product:

$\left\langle {\vec {u}},{\vec {v}}\right\rangle ={\vec {u}}\cdot {\vec {v}}=\sum _{i}u_{i}\,v_{i}=\left|{\vec {u}}\right|\,\left|{\vec {v}}\right|\,\cos {\theta }$

Generalized Algebraic Structure

We induce a geometry on the algebraic definition

Length: Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \left\| \vec{v} \right\| = \sqrt{\left\langle \vec{v}, \vec{v} \right\rangle}}

Angle Between: Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \theta = \cos^{-1}{\left( \frac{\left\langle \vec{u}, \vec{v} \right\rangle}{\left\| \vec{u} \right\| \, \left\| \vec{v} \right\|} \right)}}

The most important case is where Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \theta = \frac{\pi}{2}} , in which case $\cos {\theta }=0$ and—more importantly—the vectors ${\vec {u}}$ and ${\vec {v}}$ are perperdicular.

Properties of the Inner Product

Positivity: $\left\langle {\vec {u}},{\vec {u}}\right\rangle >0$ unless ${\vec {u}}={\vec {0}}$, in which case $\left\langle {\vec {0}},{\vec {0}}=0\right\rangle$

Conjugate Symmetry: $\left\langle {\vec {u}},{\vec {v}}\right\rangle ={\overline {\left\langle {\vec {v}},{\vec {u}}\right\rangle }}$. In the case of real numbers, the property becomes commutativity: $\left\langle {\vec {u}},{\vec {v}}\right\rangle =\left\langle {\vec {v}},{\vec {u}}\right\rangle$

Homogenuity: $\left\langle c\,{\vec {u}},{\vec {v}}\right\rangle =c\,\left\langle {\vec {u}},{\vec {v}}\right\rangle$

Linearity: $\left\langle {\vec {u}}+{\vec {w}},{\vec {v}}\right\rangle =\left\langle {\vec {u}},{\vec {v}}\right\rangle +\left\langle {\vec {w}},{\vec {v}}\right\rangle$

Real Spaces

for $\mathbb {R} ^{n}$, where ${\vec {x}}={\begin{bmatrix}x_{1}\\\vdots \\x_{n}\end{bmatrix}}$ and ${\vec {y}}={\begin{bmatrix}y_{1}\\\vdots \\y_{n}\end{bmatrix}}$, we have

$\left\langle {\vec {x}},{\vec {y}}\right\rangle =y^{T}\,x=\sum _{k=1}^{n}x_{k}\,y_{k}$

Complex Spaces

for $\mathbb {C} ^{n}$, $\left\langle {\vec {x}},{\vec {y}}\right\rangle$ is given by ${\vec {y}}^{*}\,{\vec {x}}$ (also written ${\vec {y}}^{H}\,{\vec {x}}$), where ${\vec {y}}^{*}={\vec {y}}^{H}$ is the conjugate transpose of ${\vec {y}}$.

Example: ${\vec {y}}={\begin{bmatrix}i\\1\\-1+2i\\3+i\end{bmatrix}}$, ${\vec {y}}^{*}={\begin{bmatrix}-i&1&-1-2i&3-i\end{bmatrix}}$

Some concrete implementations of inner products may not satisfy all properties for all members of the space.

Signal Spaces

Continuous and Discrete

A signal is just $f(t)$ measured on $\left[a,b\right]$.

Signals and Circuits

For current $I(t)$ (essentially our function $f(t)$)

We can measure voltage drop of a signal $V(t)=R\,I(t)$

Power at time $t$ is given by $P(t)=V\,I(t)=R\,I^{2}(t)$

Energy in time interval $\left[a,b\right]$ is given by $E=\int _{a}^{b}P(t)\,\mathrm {d} t=R\,\int _{a}^{b}\left|I(t)\right|^{2}\,\mathrm {d} t$.

Hence $E=\int _{a}^{b}\left|f(t)\right|^{2}\,\mathrm {d} t$

Finite Energy Signals

Complex valued $f(t)$ gives amplitude and phase.

Finite energy: $\int _{a}^{b}\left|f(t)\right|^{2}\,\mathrm {d} t<\infty$. Moreover, $\int _{\infty }^{\infty }\left|f(t)\right|^{2}\,\mathrm {d} t<\infty$

Signals don't need to be continuous (digital circuit signals)

Continuous Signal Space

All signals with finite energy fit into a space:

$L^{2}=\left\{f:\left[a,b\right]\to \mathbb {C} \mid \int _{a}^{b}\left|f(t)\right|^{2}\,\mathrm {d} t<\infty \right\}$

The $L$ comes from mathematician Lebesgue (ca. 1900), and the "square" comes from the square in the integral. The reason for this name is because integral we compute is a Lebesgue integral, not a Riemann integral.

We define an inner product on $L^{2}$ as

$\left\langle f,g\right\rangle _{L^{2}[a,b]}=\int _{a}^{b}f(t)\,{\overline {g(t)}}\,\mathrm {d} t$

Discrete Signal Space

Notation: $l^{2}$

By sampling the function at fixed points, we obtain a sequence $X=\left\{x_{n}\right\}_{n=-\infty }^{\infty }=\left(\dots ,x_{-1},x_{0},x_{1},\dots \right)$

Thus $l^{2}\subset L^{2}$.

We define our inner product by a discrete sum:

$\left\langle X,Y\right\rangle _{l^{2}}=\sum _{n=-\infty }^{\infty }x_{n}\,{\bar {y}}_{n}$

The energy in a discrete signal is similarly given by

$\sum _{n=-\infty }^{\infty }\left|x_{n}\right|^{2}$

Inner Product Spaces

Schwarz's Inequality

Using axioms from definition, we arrive at

$\left|\left\langle {\vec {u}},{\vec {v}}\right\rangle \right|\leq \left\|{\vec {u}}\right\|\,\left\|{\vec {u}}\right\|$, where $\left\|{\vec {u}}\right\|={\sqrt {\left\langle {\vec {u}},{\vec {u}}\right\rangle }}$ represents the norm of a vector ${\vec {u}}\in V$ ("LENGTH")

Triangle Inequality

Property of length:

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \left\| \vec{u} + \vec{v} \right\| \le \left\| \vec{u} \right\| + \left\| \vec{v} \right\|} .

"The sum of the lengths any two sides of a triangle must be at least the length of the third side."