ece204: add regression and interpolation

docs/2a/ece204.md

@@ -1 +1,86 @@
 # ECE 204: Numerical Methods
+
+## Linear regression
+
+Given a regression $y=mx+b$ and a data set $(x_{i..n}, y_{i..n})$, the **residual** is the difference between the actual and regressed data:
+
+$$E_i=y_i-b-mx_i$$
+
+### Method of least squares
+
+This method minimises the sum of the square of residuals.
+
+$$\boxed{S_r=\sum^n_{i=1}E_i^2}$$
+
+$m$ and $b$ can be found by taking the partial derivative and solving for them:
+
+$$\frac{\partial S_r}{\partial m}=0, \frac{\partial S_r}{\partial b}=0$$
+
+This returns, where $\overline y$ is the mean of the actual $y$-values:
+
+$$
+\boxed{m=\frac{n\sum^n_{i=1}x_iy_i-\sum^n_{i=1}x_i\sum^n_{i=1}y_i}{n\sum^n_{i=1}x_i^2-\left(\sum^n_{i=1}x_i\right)^2}} \\
+b=\overline y-m\overline x
+$$
+
+The total sum of square around the mean is based off of the actual data:
+
+$$\boxed{S_t=\sum(y_i-\overline y)^2}$$
+
+Error is measured with the **coefficient of determination** $r^2$ — the closer the value is to 1, the lower the error.
+
+$$
+r^2=\frac{S_t-S_r}{S_t}
+$$
+
+If the intercept is the **origin**, $m$ reduces down to a simpler form:
+
+$$m=\frac{\sum^n_{i=1}x_iy_i}{\sum^n_{i=1}x_i^2}$$
+
+## Non-linear regression
+
+### Exponential regression
+
+Solving for the same partial derivatives returns the same values, although bisection may be required for the exponent coefficient ($e^{bx}$) Instead, linearising may make things easier (by taking the natural logarithm of both sides. Afterward, solving as if it were in the form $y=mx+b$ returns correct 
+
+!!! example
+    $y=ax^b\implies\ln y = \ln a + b\ln x$
+
+### Polynomial regression
+
+The residiual is the offset at the end of a polynomial.
+
+$$y=a+bx+cx^2+E$$
+
+Taking the relevant partial derivatives returns a system of equations which can be solved in a matrix.
+
+## Interpolation
+
+Interpolation ensures that every point is crossed.
+
+### Direct method
+
+To interpolate $n+1$ data points, you need a polynomial of a degree **up to $n$**, and points that enclose the desired value. Substituting the $x$ and $y$ values forms a system of equations for a polynomial of a degree equal to the number of points chosen - 1.
+
+### Newton's divided difference method
+
+This method guesses the slope to interpolate. Where $x_0$ is an existing point:
+
+$$\boxed{f(x)=b_0+b_1(x-x_0)}$$
+
+The constant is an existing y-value and the slope is an average.
+
+$$
+\begin{align*}
+b_0&=f(x_0) \\
+b_1&=\frac{f(x_1)-f(x_0)}{x_1-x_0}
+\end{align*}
+$$
+
+This extends to a quadratic, where the second slope is the average of the first two slopes:
+
+$$\boxed{f(x)=b_0+b_1(x-x_0)+b_2(x-x_0)(x-x_1)}$$
+
+$$
+b_2=\frac{\frac{f(x_2)-f(x_1)}{x_2-x_1}-\frac{f(x_1)-f(x_0)}{x_1-x_0}}{x_2-x_0}
+$$