IT204 - Applied calc

Least Means Square

Dr Robert Batzinger
Instructor Emeritus

Payap University
Chiang Mai, Thailand
21-Sep-2022

author: “R Batzinger” format: revealjs editor: visual —

1 Least Mean Square

1.1 Example with 4 points

\(i\) 1 2 3 4
\(x_i\) 2 4 6 8
\(y_i\) 3 6 5 7

1.2 The general formula for a line

\[y = mx + b\]

1.3 Determine the difference between observed and actual

The error for each point of the line is the distance between \(y_i\) and the calculated \(mx_i + b\)

\[ \epsilon_i = y_i - (mx_i+b)\]

1.4 The sum error of a middle line cancels all errors.

\[\sum_{i=1}^n \epsilon_i = \sum \epsilon_i = 0\]

1.5 Sum of all deviations

\[\begin{eqnarray} \epsilon &=& \sum \left( y_i - (mx_i+b) \right)^2\\ &=& \sum \left(y_i^2 - 2mx_i y_i - 2b y_i + m^2 x_i^2 + 2mb x_i + b^2 \right)\\ &=& \sum y_i^2 - 2m \sum x_i y_i - 2b\sum y_i + m^2 \sum x_i^2 +\\ & & \quad \quad 2mb \sum x_i + n b^2\\ \end{eqnarray}\]

1.6 Find the minimum of the partial in respect to \(b\)

\[\begin{eqnarray}\frac{\partial}{\partial b} \sum y_i^2 - 2m \sum x_i y_i - 2b\sum y_i + m^2 \sum x_i^2 +\\ \quad \quad 2mb \sum x_i + n b^2 &=& 0\\ \\ - 2\sum y_i + 2m \sum x_i + 2n b &=&0 \\ - \sum y_i + m \sum x_i + n b &=& 0\\ \end{eqnarray}\]

2 Solve for \(b\)

\[\begin{eqnarray} -nb &=& - \sum y_i + m \sum x_i \\ b &=& \frac{\sum y_i}{n} - m \frac{\sum x_i}{n}\\ \\ b &=& \bar y - m \bar x\\ \end{eqnarray}\]

2.1 Find the minimum of the partial in respect to \(m\)

\[\begin{eqnarray}\frac{\partial}{\partial m} \sum y_i^2 - 2m \sum x_i y_i - 2b\sum y_i + \\ m^2 \sum x_i^2 + 2mb \sum x_i - n b^2 &=& 0\\ - 2 \sum x_i y_i + 2m \sum x_i^2 + 2b\sum x_i &=& 0\\ \sum x_i y_i - m \sum x_i^2 - b \sum x_i &=& 0\\ \sum x_i \left(y_i - m x_i - b \right) &=& 0\\ \end{eqnarray}\]

2.2 Simplify

\[\begin{eqnarray} \sum x_i \left(y_i - m x_i - \left(\bar y - m \bar x\right) \right) &=& 0\\ \sum x_i \left(y_i - \bar y - m\left(x_i - \bar x\right) \right) &=& 0\\ \sum x_i \left(y_i - \bar y\right) - \sum x_i m\left(x_i - \bar x\right) &=& 0\\ \sum x_i \left(y_i - \bar y\right) &=& m \sum x_i \left(x_i - \bar x\right)\\ \frac{\sum x_i \left(y_i - \bar y\right)}{\sum x_i \left(x_i - \bar x\right)} &=& m\\ \end{eqnarray}\]

2.3 Solve for \(m\)

\[\begin{eqnarray} m &=& \frac{\sum x_i \left(y_i - \bar y\right)}{\sum x_i \left(x_i - \bar x\right)} \\ m &=& \frac{\sum \left(x_i -\bar x\right)\left(y_i - \bar y\right)}{\sum \left(x_i -\bar x\right) \left(x_i - \bar x\right)} \\ \\ m &=& \frac{\sum \left(x_i -\bar x\right)\left(y_i - \bar y\right)}{\sum \left(x_i -\bar x\right)^2}\\ \end{eqnarray}\]

2.4 Least means squares

\[\begin{eqnarray} m &=& \frac{\sum \left(x_i -\bar x\right)\left(y_i - \bar y\right)}{\sum \left(x_i -\bar x\right)^2}\\ b &=& \bar y - m \bar x\\ \end{eqnarray}\]

2.5 Using the formula

\(\tiny i\) \(\tiny x_i\) \(\tiny\bar x - x_i\) \(\tiny(\bar x - x_i)^2\) \(\tiny y_i\) \(\tiny \bar y - y_i\) \(\tiny (\bar y - y_i)(\bar x - x_i)\)
\(\tiny 1\) 2 -3 9 3 -2.25 6.75
\(\tiny 2\) 4 -1 1 6 0.75 -0.75
\(\tiny 3\) 6 1 1 5 -0.25 -0.25
\(\tiny 4\) 8 3 9 7 1.75 5.25
\(\tiny \sum\) 20 20 21 11
\(\tiny Ave\) 5 5.25

\[m=\frac{11}{20} = 0.55\] \[b=5.25 - 0.55 \times 5.0 =2.5\]

2.6 Linear Regression Model

 Coefficients:
                Estimate Std. Error t value Pr(>|t|)
    Intercept   2.5000     1.4230   1.757    0.221
    Slope       0.5500     0.2598   2.117    0.168
    
 Residuals:    1    2    3    4 
             -0.6  1.3 -0.8  0.1 
 
 Residual standard error: 1.162 on 2 degrees of freedom
 R-squared: 0.6914 (multiple),  0.5371 (Adjusted) 
 F-statistic: 4.481 on 1 and 2 DF,  p-value: 0.1685

2.7 Statistical interpretation

2.8 Experimental results

3 Surge


Formula: y ~ a * x * exp(-b * x)

Parameters:
   Estimate Std. Error t value Pr(>|t|)    
a 3.262e+02  1.226e+01   26.61 1.86e-07 ***
b 1.187e-01  2.447e-03   48.51 5.14e-09 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 19.04 on 6 degrees of freedom

Number of iterations to convergence: 8 
Achieved convergence tolerance: 3.458e-06
List of 6
 $ m          :List of 16
  ..$ resid     :function ()  
  ..$ fitted    :function ()  
  ..$ formula   :function ()  
  ..$ deviance  :function ()  
  ..$ lhs       :function ()  
  ..$ gradient  :function ()  
  ..$ conv      :function ()  
  ..$ incr      :function ()  
  ..$ setVarying:function (vary = rep_len(TRUE, np))  
  ..$ setPars   :function (newPars)  
  ..$ getPars   :function ()  
  ..$ getAllPars:function ()  
  ..$ getEnv    :function ()  
  ..$ trace     :function ()  
  ..$ Rmat      :function ()  
  ..$ predict   :function (newdata = list(), qr = FALSE)  
  ..- attr(*, "class")= chr "nlsModel"
 $ convInfo   :List of 5
  ..$ isConv     : logi TRUE
  ..$ finIter    : int 8
  ..$ finTol     : num 3.46e-06
  ..$ stopCode   : int 0
  ..$ stopMessage: chr "converged"
 $ data       : symbol dta
 $ call       : language (function (formula, data = parent.frame(), start, control = nls.control(),      algorithm = c("default", "plinear| __truncated__ ...
 $ dataClasses: Named chr "numeric"
  ..- attr(*, "names")= chr "x"
 $ control    :List of 7
  ..$ maxiter    : num 50
  ..$ tol        : num 1e-05
  ..$ minFactor  : num 0.000977
  ..$ printEval  : logi FALSE
  ..$ warnOnly   : logi FALSE
  ..$ scaleOffset: num 0
  ..$ nDcentral  : logi FALSE
 - attr(*, "class")= chr "nls"

3.1 Total number of cases

\[\begin{eqnarray} C &=&\int axe^{(-bx)} dx\\ &=& - \frac{a(bx + 1)e^{(-bx)}}{b^2}\\ &=& 326 \left(\frac{100000000 - 1049600000 e^{-(1187/125)}}{1408969}\right)\\ \\ &=& 23041\\ \end{eqnarray}\]

3.2 Peak day of peak infection

\[\begin{eqnarray} 0 &=& f'(x) = \frac{d\left( axe^{(-bx)}\right)}{dx}\\ 0 &=& ae^{(-bx)} - abxe^{(-bx)} = −a\left(bx-1\right)e^{-bx}\\ 0 &=& bx -1\\ bx &=& 1\\ \\ x &=& \frac{1}{b} = \frac{1}{0.1187} =8.42\\ \end{eqnarray} \]

3.3 Point of inflection

\[\begin{eqnarray} 0 &=& f''(x) = \frac{d(ae^{-bx} - abxe^{-bx})}{dx}\\ 0 &=& -abe^{(-bx)} + −ab\left(bx-1\right)e^{-bx}\\ 0 &=&ab\left(bx-1\right)e^{-bx}-abe^{-bx}\\ 0 &=&ab\left(bx-2)\right) e^{-bx}\\ 0 &=& bx - 2\\ \\ x &=& \frac{2}{b} = \frac{2}{0.1187}= 16.85 days\\ \end{eqnarray}\]

3.4 Duration of the infection

86 days

3.5 Statistical analysis

3.6 Covid 19 Chiang Mai


Formula: cases ~ a1 * cdays * exp(-b1 * cdays)

Parameters:
    Estimate Std. Error t value Pr(>|t|)    
a1 99.181963   5.538972   17.91 6.44e-13 ***
b1  0.265549   0.008625   30.79  < 2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 9.996 on 18 degrees of freedom

Number of iterations to convergence: 7 
Achieved convergence tolerance: 1.959e-07

3.7 Modelled data

3.8 Early warning

4 Linear

\[y = 0.445238*x + 0.3333333\]

5 Logistic Functions

\[y = \frac{L}{1 + C e^{-kx}}\]

\[y = mx + b\]

6 Sigmoid

\[y = \frac{C}{1+ae^{kx}}\]

7 Normal curve

\[p = \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{(x-\mu)^2}{2\sigma^2}}\]

x y
0 0.0000000
2 0.0000000
4 0.0000000
6 0.0000012
8 0.0012852
10 0.0828976
12 0.3324519
14 0.0828976
16 0.0012852
18 0.0000012
20 0.0000000

8 Shelf life of bananas

\[p(t) = -0.0375t^2 + 0.225t\]

8.1

9 SARS infections

t = c(0,5,12,19,26,33,40,47,54,61,68,75,81,87) p = c(95,222,470,800,1108,1358,1527,1621,1674, 1710,1724,1739,1750,1755)

10 CD Rom population

Annual sales of CD ROM

Inflection = point of dimishing returns

\[p = \frac{a}{2}\]

x = c(0,1,2,3,4,5,6,7) p = c(0.5,2,8,33,95,258,403,496)

\[p = \frac{a}{1+ be^{-cx}}\]

10.1 United States population

x=c(1790,1800,1810,1820,1830,1840,1850,1860, 1870,1880,1890,1900,1910, 1920,1930,1940,1950,1960,1970,1980,1990, 2000)

y = c(3.9,5.3,7.2,9.6,12.9,17.1,23.1,31.4, 38.6,50.2,62.9,76.0,92.0, 105.7,122.8,131.7,150.7,179.3,203.3,248.7,281.4) )

11 Elasticity of demand

\[Elasticity =E= \frac{\%\Delta demand}{\%\Delta price}=\frac{{\Delta q}/{q}}{{\Delta p}/{p}}=\frac{p}{q} \times \frac{\Delta q}{\Delta p}\] \[\left{\begin{matrix}if\ E<1 & inelastic\ and\ R\ \downarrow when \uparrow p\\ if\ E=1 & (critical\ point\ of\ revenue\ function)\\ if\ E >1 & elastic\ and\ R\uparrow when \uparrow p\\ \end{matrix}\right.\]

11.1 Elasticity of American demand of farm products

Product E Product E Product E
Cabbage 0.25 Milk 0.49 Apples 1.27
Potatoes 0.27 Butter 0.62 Peaches 1.49
Wool 0.33 Oranges 0.62 Tomatoes 2.22
Peanuts 0.38 Cream 0.69 Lettuce 2.58
Eggs 0.43 Peas 2.83

11.2 Revenue as a function of price

\[\eqalign{\frac{dR}{dp} &= \frac{d}{dp} (pq)\cr &= p\frac{dq}{dp}+\frac{dp}{dp}q\cr &= p\frac{dq}{dp}+ q\cr} \] \[p\frac{dq}{dp} + q = 0\] \[p\frac{dq}{dp}= - q\] \[1= \frac{p}{q} \frac{dq}{dp} = E\]

11.3 US Cable subscribers

x= c(1977:2003) y = c(16.6,17.9,19.4,22.6,28.3,35.0,40.5, 43.7,46.2,48.1,50.5,53.8,57.1,59.0, 60.6,61.5,62.6,63.4,65.7,66.7,67.3, 67.4,68.0,67.8,69.2,68.9,68.0)

11.4 Recorded vehicle speed in secs

12 Exponential function

\[f(x) = ae^{bx}\]

13 Logistic Model

\[\frac{dP}{dt} = k P(L-P)\]

14 Logarithmic function

\[f(x) = a \ln(bx+c)\]

$$ # Polynomial function

\[ f(x) = a_0 x^n + a_1 x^{n-1}+ ...+ a_{(n-1)} x + a_n\]

15 Periodic function

\[f(x) = a \sin(bx + c)\]

16 Logistic function

\[f(x) = \frac{C}{1 + ae^{-bx}}\]

17 Surge function

\[y=axe^{-bx}\]


Formula: y ~ a * x * exp(-b * x)

Parameters:
   Estimate Std. Error t value Pr(>|t|)    
a 5.2236673  0.0228497   228.6 3.04e-11 ***
b 0.1984985  0.0006159   322.3 5.46e-12 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.03584 on 5 degrees of freedom

Number of iterations to convergence: 5 
Achieved convergence tolerance: 6.009e-08

18 Separation of Variations

\[\eqalign{ \frac{dy}{dt} & =& ky\cr \int \frac{1}{y} dy &=& \int k\ dt\cr \ln(|y|) &=& kt + c\cr |y| &=& e^{kt + C} \cr |y| &=& e^{kt}e^{C}\cr y &=& \pm A e^{kt}\cr y &=& B e^{kt}\cr} \]

18.1 Separation of Variables: Ex 2

\[\eqalign{\frac{dy}{dt} & = - \frac{x}{y} \cr y dy &= - x dx\cr \int y dy &= - \int x dx\cr \frac{y^2}{2} &= -\frac{x^2}{2}\cr x^2 + y^2 &= C\cr}\]

18.2 Separation of Variables: Ex 3

\[\eqalign{\frac{dy}{dx} &= k(y-A)\cr \frac{1}{y-A} &= k dx\cr \int \frac{1}{y-A} &= \int k dx\cr \ln(|y - A|) &= kt + D\cr |y - A| &= e^{kt} e^{D}\cr y &= A + C e^{kt}\cr}\]

18.3 Separation of Variables: Ex 4

\[\eqalign{\frac{dP}{dt} &= 2P - 2Pt\cr \frac{1}{P}} &= (2-2t) dt\cr \int \frac{1}{P}} &= \int (2-2t) dt\cr \ln(|P|) &= 2t - t^2 + C\cr |P| &= e^{2t - t^2 + C}\cr P &= B e^{2t-t^2}\cr }\]