2.6 KiB
ECE 204: Numerical Methods
Linear regression
Given a regression \(y=mx+b\) and a data set \((x_{i..n}, y_{i..n})\), the residual is the difference between the actual and regressed data:
\[E_i=y_i-b-mx_i\]
Method of least squares
This method minimises the sum of the square of residuals.
\[\boxed{S_r=\sum^n_{i=1}E_i^2}\]
\(m\) and \(b\) can be found by taking the partial derivative and solving for them:
\[\frac{\partial S_r}{\partial m}=0, \frac{\partial S_r}{\partial b}=0\]
This returns, where \(\overline y\) is the mean of the actual \(y\)-values:
\[ \boxed{m=\frac{n\sum^n_{i=1}x_iy_i-\sum^n_{i=1}x_i\sum^n_{i=1}y_i}{n\sum^n_{i=1}x_i^2-\left(\sum^n_{i=1}x_i\right)^2}} \\ b=\overline y-m\overline x \]
The total sum of square around the mean is based off of the actual data:
\[\boxed{S_t=\sum(y_i-\overline y)^2}\]
Error is measured with the coefficient of determination \(r^2\) — the closer the value is to 1, the lower the error.
\[ r^2=\frac{S_t-S_r}{S_t} \]
If the intercept is the origin, \(m\) reduces down to a simpler form:
\[m=\frac{\sum^n_{i=1}x_iy_i}{\sum^n_{i=1}x_i^2}\]
Non-linear regression
Exponential regression
Solving for the same partial derivatives returns the same values, although bisection may be required for the exponent coefficient (\(e^{bx}\)) Instead, linearising may make things easier (by taking the natural logarithm of both sides. Afterward, solving as if it were in the form \(y=mx+b\) returns correct
!!! example \(y=ax^b\implies\ln y = \ln a + b\ln x\)
Polynomial regression
The residiual is the offset at the end of a polynomial.
\[y=a+bx+cx^2+E\]
Taking the relevant partial derivatives returns a system of equations which can be solved in a matrix.
Interpolation
Interpolation ensures that every point is crossed.
Direct method
To interpolate \(n+1\) data points, you need a polynomial of a degree up to \(n\), and points that enclose the desired value. Substituting the \(x\) and \(y\) values forms a system of equations for a polynomial of a degree equal to the number of points chosen - 1.
Newton’s divided difference method
This method guesses the slope to interpolate. Where \(x_0\) is an existing point:
\[\boxed{f(x)=b_0+b_1(x-x_0)}\]
The constant is an existing y-value and the slope is an average.
\[ \begin{align*} b_0&=f(x_0) \\ b_1&=\frac{f(x_1)-f(x_0)}{x_1-x_0} \end{align*} \]
This extends to a quadratic, where the second slope is the average of the first two slopes:
\[\boxed{f(x)=b_0+b_1(x-x_0)+b_2(x-x_0)(x-x_1)}\]
\[ b_2=\frac{\frac{f(x_2)-f(x_1)}{x_2-x_1}-\frac{f(x_1)-f(x_0)}{x_1-x_0}}{x_2-x_0} \]