MATH 251 Lecture 28

« previous | Wednesday, April 4, 2012 | next »

Linear Regression

Given $x$, $y$, and you think $y=f(x)$, but know nothing (this is where statistics comes in).

Observe a finite collection of data $P_i=(x_i,y_i)$.

Data is assumed to be "noisy" (not 100% true). We want to reconstruct $f(x)$

In general, we need two things:

1. A class of functions $f(x)$
2. A measure $F(f(x),\mathrm{D}\mathrm{a}\mathrm{t}\mathrm{a})$ to be minimized

Linear Regression

Assume that our class is linear (i.e. $f(x)=mx+b$). We want the best choice for $m$ and $b$: Least-Squares linear regression

Measure the deviation from $f(x)$, so we need a function $F$ that depends on $f(x)$ and the data. Our goal is to minimize this function.

$F(m,b)=\sum _{i=1}^n{(y_i-f(x_i))}^2=\sum _{i=1}^n{(y_i-m{x}_i-b)}^2$

Calculate its gradient, set it to 0, and solve for $m$ and $b$.

"this is on Wikipedia"

Quadratic Regression

Least-Squares Quadratic Regression let $f(x)=a{x}^2+bx+c$, so $F(a,b,c)=\sum _{i=1}^n{(y_i-a{x}^2-bx-c)}^2$

Penalty Coefficient

Sometimes the data looks like a certain form, but could be something else. Introduce a penalty coefficient to check "fitness"

Line Integrals

Confused about Notation?

Given a curve $C$ and a vector field $\overrightarrow{F}(x,y)$, the line integral represents how much of the vector field $\overrightarrow{F}$ is in the direction of the curve $C$, like Work in physics.

Written: Calculate $\underset{C}{\int }\overrightarrow{F}\cdot \mathrm{d}\overrightarrow{r}$ (vector field notation) or $\underset{C}{\int }a\mathrm{d}x+b\mathrm{d}y+c\mathrm{d}z$ (differential form notation)

Example: $C$ represents the unit circle going counter-clockwise, and $\overrightarrow{F}=x\hat{ı}+y\hat{ȷ}$

1. Break into separate line regions (if necessary)
2. Parameterize each line $C$: $x(t)=\cos t$, $y(t)=\sin t$, $0\le t\le 2\pi$
3. Determine meaning of differential: $\mathrm{d}\overrightarrow{r}=\mathrm{d}x\hat{ı}+\mathrm{d}y\hat{ȷ}$
4. Plug in vectors into dot product: $\overrightarrow{F}\cdot \mathrm{d}\overrightarrow{r}=x\mathrm{d}x+y\mathrm{d}y$
5. Set up integral with internal components (use chain rule): $\underset{C}{\int }x\mathrm{d}x+y\mathrm{d}y=\underset{0}{\overset{2\pi }{\int }}x(t)(x^{\prime }(t)\mathrm{d}t)+y(t)(y^{\prime }(t)\mathrm{d}t)$
6. Evaluate: $\underset{0}{\overset{2\pi }{\int }}(-\cos t\sin t+\sin t\cos t)\mathrm{d}t=0$

Example

$C$ represents the triangle formed by (0,0), (1,0), and (1,1). Set up an integral for each line. (generally move counter-clockwise)

$\underset{C_1}{\int }+\underset{C_2}{\int }+\underset{C_3}{\int }$

Parameterization for each $C$:

1. $x(t)=t$, $y(t)=0$, $0\le t\le 1$
2. $x(t)=1$, $y(t)=t$, $0\le t\le 1$
3. $x(t)=1-t$, $y(t)=1-t$, $0\le t\le 1$

Example

Let $F$ represent a force field (like gravity). If the force is constant, then $W=Fd$. If the force changes for each point, use the integral to get a continuous sum of the Work at each point.