MATH 251 Lecture 28

« previous | Wednesday, April 4, 2012 | next »

Linear Regression

Given $x$, $y$, and you think $y=f(x)$, but know nothing (this is where statistics comes in).

Observe a finite collection of data $P_{i}=(x_{i},y_{i})$.

Data is assumed to be "noisy" (not 100% true). We want to reconstruct $f(x)$

In general, we need two things:

1. A class of functions $f(x)$
2. A measure $F(f(x),\mathrm {Data} )$ to be minimized

Linear Regression

Assume that our class is linear (i.e. $f(x)=mx+b$). We want the best choice for $m$ and $b$: Least-Squares linear regression

Measure the deviation from $f(x)$, so we need a function $F$ that depends on $f(x)$ and the data. Our goal is to minimize this function.

$F(m,b)=\sum _{i=1}^{n}\left(y_{i}-f(x_{i})\right)^{2}=\sum _{i=1}^{n}\left(y_{i}-mx_{i}-b\right)^{2}$

Calculate its gradient, set it to 0, and solve for $m$ and $b$.

"this is on Wikipedia"

Quadratic Regression

Least-Squares Quadratic Regression let $f(x)=ax^{2}+bx+c$, so $F(a,b,c)=\sum _{i=1}^{n}\left(y_{i}-ax^{2}-bx-c\right)^{2}$

Penalty Coefficient

Sometimes the data looks like a certain form, but could be something else. Introduce a penalty coefficient to check "fitness"

Line Integrals

Confused about Notation?

Given a curve $C$ and a vector field ${\vec {F}}(x,y)$, the line integral represents how much of the vector field ${\vec {F}}$ is in the direction of the curve $C$, like Work in physics.

Written: Calculate $\int _{C}{\vec {F}}\cdot \mathrm {d} {\vec {r}}$ (vector field notation) or $\int _{C}a\,\mathrm {d} x+b\,\mathrm {d} y+c\,\mathrm {d} z$ (differential form notation)

Example: $C$ represents the unit circle going counter-clockwise, and ${\vec {F}}=x{\hat {\imath }}+y{\hat {\jmath }}$

1. Break into separate line regions (if necessary)
2. Parameterize each line $C$: $x(t)=\cos {t}$, $y(t)=\sin {t}$, $0\leq t\leq 2\pi$
3. Determine meaning of differential: $\mathrm {d} {\vec {r}}=\mathrm {d} x{\hat {\imath }}+\mathrm {d} y{\hat {\jmath }}$
4. Plug in vectors into dot product: ${\vec {F}}\cdot \mathrm {d} {\vec {r}}=x\,\mathrm {d} x+y\,\mathrm {d} y$
5. Set up integral with internal components (use chain rule): $\int _{C}x\,\mathrm {d} x+y\,\mathrm {d} y=\int _{0}^{2\pi }x(t)(x'(t)\mathrm {d} t)+y(t)(y'(t)\mathrm {d} t)$
6. Evaluate: $\int _{0}^{2\pi }\left(-\cos {t}\sin {t}+\sin {t}\cos {t}\right)\,\mathrm {d} t=0$

Example

$C$ represents the triangle formed by (0,0), (1,0), and (1,1). Set up an integral for each line. (generally move counter-clockwise)

$\int _{C_{1}}+\int _{C_{2}}+\int _{C_{3}}$

Parameterization for each $C$:

1. $x(t)=t$, $y(t)=0$, $0\leq t\leq 1$
2. $x(t)=1$, $y(t)=t$, $0\leq t\leq 1$
3. $x(t)=1-t$, $y(t)=1-t$, $0\leq t\leq 1$

Example

Let $F$ represent a force field (like gravity). If the force is constant, then $W=Fd$. If the force changes for each point, use the integral to get a continuous sum of the Work at each point.