Least Means Square
Dr Robert Batzinger
Instructor Emeritus
Payap University
Chiang Mai, Thailand
21-Sep-2022
author: “R Batzinger” format: revealjs editor: visual —
\(i\) | 1 | 2 | 3 | 4 |
---|---|---|---|---|
\(x_i\) | 2 | 4 | 6 | 8 |
\(y_i\) | 3 | 6 | 5 | 7 |
\[y = mx + b\]
The error for each point of the line is the distance between \(y_i\) and the calculated \(mx_i + b\)
\[ \epsilon_i = y_i - (mx_i+b)\]
\[\sum_{i=1}^n \epsilon_i = \sum \epsilon_i = 0\]
\[\begin{eqnarray} \epsilon &=& \sum \left( y_i - (mx_i+b) \right)^2\\ &=& \sum \left(y_i^2 - 2mx_i y_i - 2b y_i + m^2 x_i^2 + 2mb x_i + b^2 \right)\\ &=& \sum y_i^2 - 2m \sum x_i y_i - 2b\sum y_i + m^2 \sum x_i^2 +\\ & & \quad \quad 2mb \sum x_i + n b^2\\ \end{eqnarray}\]
\[\begin{eqnarray}\frac{\partial}{\partial b} \sum y_i^2 - 2m \sum x_i y_i - 2b\sum y_i + m^2 \sum x_i^2 +\\ \quad \quad 2mb \sum x_i + n b^2 &=& 0\\ \\ - 2\sum y_i + 2m \sum x_i + 2n b &=&0 \\ - \sum y_i + m \sum x_i + n b &=& 0\\ \end{eqnarray}\]
\[\begin{eqnarray} -nb &=& - \sum y_i + m \sum x_i \\ b &=& \frac{\sum y_i}{n} - m \frac{\sum x_i}{n}\\ \\ b &=& \bar y - m \bar x\\ \end{eqnarray}\]
\[\begin{eqnarray}\frac{\partial}{\partial m} \sum y_i^2 - 2m \sum x_i y_i - 2b\sum y_i + \\ m^2 \sum x_i^2 + 2mb \sum x_i - n b^2 &=& 0\\ - 2 \sum x_i y_i + 2m \sum x_i^2 + 2b\sum x_i &=& 0\\ \sum x_i y_i - m \sum x_i^2 - b \sum x_i &=& 0\\ \sum x_i \left(y_i - m x_i - b \right) &=& 0\\ \end{eqnarray}\]
\[\begin{eqnarray} \sum x_i \left(y_i - m x_i - \left(\bar y - m \bar x\right) \right) &=& 0\\ \sum x_i \left(y_i - \bar y - m\left(x_i - \bar x\right) \right) &=& 0\\ \sum x_i \left(y_i - \bar y\right) - \sum x_i m\left(x_i - \bar x\right) &=& 0\\ \sum x_i \left(y_i - \bar y\right) &=& m \sum x_i \left(x_i - \bar x\right)\\ \frac{\sum x_i \left(y_i - \bar y\right)}{\sum x_i \left(x_i - \bar x\right)} &=& m\\ \end{eqnarray}\]
\[\begin{eqnarray} m &=& \frac{\sum x_i \left(y_i - \bar y\right)}{\sum x_i \left(x_i - \bar x\right)} \\ m &=& \frac{\sum \left(x_i -\bar x\right)\left(y_i - \bar y\right)}{\sum \left(x_i -\bar x\right) \left(x_i - \bar x\right)} \\ \\ m &=& \frac{\sum \left(x_i -\bar x\right)\left(y_i - \bar y\right)}{\sum \left(x_i -\bar x\right)^2}\\ \end{eqnarray}\]
\[\begin{eqnarray} m &=& \frac{\sum \left(x_i -\bar x\right)\left(y_i - \bar y\right)}{\sum \left(x_i -\bar x\right)^2}\\ b &=& \bar y - m \bar x\\ \end{eqnarray}\]
\(\tiny i\) | \(\tiny x_i\) | \(\tiny\bar x - x_i\) | \(\tiny(\bar x - x_i)^2\) | \(\tiny y_i\) | \(\tiny \bar y - y_i\) | \(\tiny (\bar y - y_i)(\bar x - x_i)\) | |
---|---|---|---|---|---|---|---|
\(\tiny 1\) | 2 | -3 | 9 | 3 | -2.25 | 6.75 | |
\(\tiny 2\) | 4 | -1 | 1 | 6 | 0.75 | -0.75 | |
\(\tiny 3\) | 6 | 1 | 1 | 5 | -0.25 | -0.25 | |
\(\tiny 4\) | 8 | 3 | 9 | 7 | 1.75 | 5.25 | |
\(\tiny \sum\) | 20 | 20 | 21 | 11 | |||
\(\tiny Ave\) | 5 | 5.25 |
\[m=\frac{11}{20} = 0.55\] \[b=5.25 - 0.55 \times 5.0 =2.5\]
Coefficients:
Estimate Std. Error t value Pr(>|t|)
Intercept 2.5000 1.4230 1.757 0.221
Slope 0.5500 0.2598 2.117 0.168
Residuals: 1 2 3 4
-0.6 1.3 -0.8 0.1
Residual standard error: 1.162 on 2 degrees of freedom
R-squared: 0.6914 (multiple), 0.5371 (Adjusted)
F-statistic: 4.481 on 1 and 2 DF, p-value: 0.1685
Formula: y ~ a * x * exp(-b * x)
Parameters:
Estimate Std. Error t value Pr(>|t|)
a 3.262e+02 1.226e+01 26.61 1.86e-07 ***
b 1.187e-01 2.447e-03 48.51 5.14e-09 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 19.04 on 6 degrees of freedom
Number of iterations to convergence: 8
Achieved convergence tolerance: 3.458e-06
List of 6
$ m :List of 16
..$ resid :function ()
..$ fitted :function ()
..$ formula :function ()
..$ deviance :function ()
..$ lhs :function ()
..$ gradient :function ()
..$ conv :function ()
..$ incr :function ()
..$ setVarying:function (vary = rep_len(TRUE, np))
..$ setPars :function (newPars)
..$ getPars :function ()
..$ getAllPars:function ()
..$ getEnv :function ()
..$ trace :function ()
..$ Rmat :function ()
..$ predict :function (newdata = list(), qr = FALSE)
..- attr(*, "class")= chr "nlsModel"
$ convInfo :List of 5
..$ isConv : logi TRUE
..$ finIter : int 8
..$ finTol : num 3.46e-06
..$ stopCode : int 0
..$ stopMessage: chr "converged"
$ data : symbol dta
$ call : language (function (formula, data = parent.frame(), start, control = nls.control(), algorithm = c("default", "plinear| __truncated__ ...
$ dataClasses: Named chr "numeric"
..- attr(*, "names")= chr "x"
$ control :List of 7
..$ maxiter : num 50
..$ tol : num 1e-05
..$ minFactor : num 0.000977
..$ printEval : logi FALSE
..$ warnOnly : logi FALSE
..$ scaleOffset: num 0
..$ nDcentral : logi FALSE
- attr(*, "class")= chr "nls"
\[\begin{eqnarray} C &=&\int axe^{(-bx)} dx\\ &=& - \frac{a(bx + 1)e^{(-bx)}}{b^2}\\ &=& 326 \left(\frac{100000000 - 1049600000 e^{-(1187/125)}}{1408969}\right)\\ \\ &=& 23041\\ \end{eqnarray}\]
\[\begin{eqnarray} 0 &=& f'(x) = \frac{d\left( axe^{(-bx)}\right)}{dx}\\ 0 &=& ae^{(-bx)} - abxe^{(-bx)} = −a\left(bx-1\right)e^{-bx}\\ 0 &=& bx -1\\ bx &=& 1\\ \\ x &=& \frac{1}{b} = \frac{1}{0.1187} =8.42\\ \end{eqnarray} \]
\[\begin{eqnarray} 0 &=& f''(x) = \frac{d(ae^{-bx} - abxe^{-bx})}{dx}\\ 0 &=& -abe^{(-bx)} + −ab\left(bx-1\right)e^{-bx}\\ 0 &=&ab\left(bx-1\right)e^{-bx}-abe^{-bx}\\ 0 &=&ab\left(bx-2)\right) e^{-bx}\\ 0 &=& bx - 2\\ \\ x &=& \frac{2}{b} = \frac{2}{0.1187}= 16.85 days\\ \end{eqnarray}\]
86 days
Formula: cases ~ a1 * cdays * exp(-b1 * cdays)
Parameters:
Estimate Std. Error t value Pr(>|t|)
a1 99.181963 5.538972 17.91 6.44e-13 ***
b1 0.265549 0.008625 30.79 < 2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 9.996 on 18 degrees of freedom
Number of iterations to convergence: 7
Achieved convergence tolerance: 1.959e-07
\[y = 0.445238*x + 0.3333333\]
\[y = \frac{L}{1 + C e^{-kx}}\]
\[y = mx + b\]
\[y = \frac{C}{1+ae^{kx}}\]
\[p = \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{(x-\mu)^2}{2\sigma^2}}\]
x | y |
---|---|
0 | 0.0000000 |
2 | 0.0000000 |
4 | 0.0000000 |
6 | 0.0000012 |
8 | 0.0012852 |
10 | 0.0828976 |
12 | 0.3324519 |
14 | 0.0828976 |
16 | 0.0012852 |
18 | 0.0000012 |
20 | 0.0000000 |
\[p(t) = -0.0375t^2 + 0.225t\]
t = c(0,5,12,19,26,33,40,47,54,61,68,75,81,87) p = c(95,222,470,800,1108,1358,1527,1621,1674, 1710,1724,1739,1750,1755)
Annual sales of CD ROM
Inflection = point of dimishing returns
\[p = \frac{a}{2}\]
x = c(0,1,2,3,4,5,6,7) p = c(0.5,2,8,33,95,258,403,496)
\[p = \frac{a}{1+ be^{-cx}}\]
x=c(1790,1800,1810,1820,1830,1840,1850,1860, 1870,1880,1890,1900,1910, 1920,1930,1940,1950,1960,1970,1980,1990, 2000)
y = c(3.9,5.3,7.2,9.6,12.9,17.1,23.1,31.4, 38.6,50.2,62.9,76.0,92.0, 105.7,122.8,131.7,150.7,179.3,203.3,248.7,281.4) )
\[Elasticity =E= \frac{\%\Delta demand}{\%\Delta price}=\frac{{\Delta q}/{q}}{{\Delta p}/{p}}=\frac{p}{q} \times \frac{\Delta q}{\Delta p}\] \[\left{\begin{matrix}if\ E<1 & inelastic\ and\ R\ \downarrow when \uparrow p\\ if\ E=1 & (critical\ point\ of\ revenue\ function)\\ if\ E >1 & elastic\ and\ R\uparrow when \uparrow p\\ \end{matrix}\right.\]
Product | E | Product | E | Product | E |
---|---|---|---|---|---|
Cabbage | 0.25 | Milk | 0.49 | Apples | 1.27 |
Potatoes | 0.27 | Butter | 0.62 | Peaches | 1.49 |
Wool | 0.33 | Oranges | 0.62 | Tomatoes | 2.22 |
Peanuts | 0.38 | Cream | 0.69 | Lettuce | 2.58 |
Eggs | 0.43 | Peas | 2.83 |
\[\eqalign{\frac{dR}{dp} &= \frac{d}{dp} (pq)\cr &= p\frac{dq}{dp}+\frac{dp}{dp}q\cr &= p\frac{dq}{dp}+ q\cr} \] \[p\frac{dq}{dp} + q = 0\] \[p\frac{dq}{dp}= - q\] \[1= \frac{p}{q} \frac{dq}{dp} = E\]
x= c(1977:2003) y = c(16.6,17.9,19.4,22.6,28.3,35.0,40.5, 43.7,46.2,48.1,50.5,53.8,57.1,59.0, 60.6,61.5,62.6,63.4,65.7,66.7,67.3, 67.4,68.0,67.8,69.2,68.9,68.0)
\[f(x) = ae^{bx}\]
\[\frac{dP}{dt} = k P(L-P)\]
\[f(x) = a \ln(bx+c)\]
$$ # Polynomial function
\[ f(x) = a_0 x^n + a_1 x^{n-1}+ ...+ a_{(n-1)} x + a_n\]
\[f(x) = a \sin(bx + c)\]
\[f(x) = \frac{C}{1 + ae^{-bx}}\]
\[y=axe^{-bx}\]
Formula: y ~ a * x * exp(-b * x)
Parameters:
Estimate Std. Error t value Pr(>|t|)
a 5.2236673 0.0228497 228.6 3.04e-11 ***
b 0.1984985 0.0006159 322.3 5.46e-12 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 0.03584 on 5 degrees of freedom
Number of iterations to convergence: 5
Achieved convergence tolerance: 6.009e-08
\[\eqalign{ \frac{dy}{dt} & =& ky\cr \int \frac{1}{y} dy &=& \int k\ dt\cr \ln(|y|) &=& kt + c\cr |y| &=& e^{kt + C} \cr |y| &=& e^{kt}e^{C}\cr y &=& \pm A e^{kt}\cr y &=& B e^{kt}\cr} \]
\[\eqalign{\frac{dy}{dt} & = - \frac{x}{y} \cr y dy &= - x dx\cr \int y dy &= - \int x dx\cr \frac{y^2}{2} &= -\frac{x^2}{2}\cr x^2 + y^2 &= C\cr}\]
\[\eqalign{\frac{dy}{dx} &= k(y-A)\cr \frac{1}{y-A} &= k dx\cr \int \frac{1}{y-A} &= \int k dx\cr \ln(|y - A|) &= kt + D\cr |y - A| &= e^{kt} e^{D}\cr y &= A + C e^{kt}\cr}\]
\[\eqalign{\frac{dP}{dt} &= 2P - 2Pt\cr \frac{1}{P}} &= (2-2t) dt\cr \int \frac{1}{P}} &= \int (2-2t) dt\cr \ln(|P|) &= 2t - t^2 + C\cr |P| &= e^{2t - t^2 + C}\cr P &= B e^{2t-t^2}\cr }\]
PYU IT204 2022/1: Calculus ….. [5]