Population \(\rho\) Sample \(r\) \begin{equation}=\frac{Cov(X,Y)}{\sigma_{X}\sigma_{Y}},r=\frac{Cov(X,Y)}{s_{X}s_{Y}}\end{equation}
\(-\infty <Cov(X,Y) <+\infty\) represents the quantification and direction of the linear relationship between 2 continuous variables X and Y.
Hard to interpret beyond the sign because it will have units of X multiplied with units of Y.
Beyond the calculation for correlation it is useful in its own right in more complicated models as it has less restrictions compared to correlation.
\[Cov(X,Y)=E(XY)-E(X)E(Y)\] \[Cov(X,Y)=E[(X-E(X))E(Y-E(Y))] \]
\[\sigma^{2}=E(x-\mu_{X})^{2}\]
| X | Y | XY | \(x-E(X)\) | \(y-E(Y)\) | \((x-E(X))(y-E(Y))\) |
|---|---|---|---|---|---|
| 1 | 9 | 9 | 1-5=-4 | 9-5=4 | -4 \(\times\) 4=-16 |
| 5 | 5 | 25 | 5-5=0 | 5-5=0 | 0 \(\times\) 0=0 |
| 9 | 1 | 9 | 9-5=4 | 1-5=-4 | 4\(\times\) -4=-16 |
\[\frac{-16+0+-16}{3}=-10.67,\frac{-16+0+-16}{(3-1)}=-16\]
\[\begin{equation}\rho=\frac{Cov(X,Y)}{\sigma_{X}\sigma_{Y}},r=\frac{Cov(X,Y)}{s_{X}s_{Y}}\end{equation}\]
N=100000
# Generate independent variable
x <- rnorm(n=N, mean=0, sd=1)
# Generate the dependen variable
y <- 0.5*x +rnorm(n=N, mean = 0, sd = 1)
cor(x,y)[1] 0.448321
N=100000
# Generate independent variable
x <- rnorm(n=N, mean=0, sd=1)
# Generate the dependen variable
y <- -0.5*x +rnorm(n=N, mean = 0, sd = 1)
cor(x,y)[1] -0.4481473
N=100000
# Generate independent variable
x <- rnorm(n=N, mean=0, sd=1)
# Generate the dependen variable
y <- 0*x +rnorm(n=N, mean = 0, sd = 1)
cor(x,y)[1] 0.00105148How does \(Y=0.5*X+Z\) where z is the standard normal distribution lead to a correlation of 0.45.
No it is not an approximation of 0.5.
\[Y=0.5 \times X + Z\] \[Cor(X,Y)=\frac{Cov(X,Y)}{\sigma_{X}\sigma_{Y}} \] \[Cov(X,Y)=E(XY)-E(X)E(Y)\] \[E(XY)=?\]
\[\begin{align} E(XY)=E(X \times (0.5 \times X+Z)) \\ E(XY)=E(0.5X^{2}+X \times Z)\\ E(XY)=E(0.5X^{2})+E(X \times Z)\\ E(XY)=0.5E(X^{2})+E(X \times Z) \end{align}\]
Those of you who are practically minded might want to simulate a large number of \(X^{2}\) and compute the mean. This would provide an approximate solution and we can revert to this if we do not have an exact solution.
Can we do \(E(X)*E(X)\) to calculate \(E(X^{2})\)?
In that case X has a random variable whose variance is 1 as well.
Why is this helpful? Think about the variance calculation!
\[\begin{aligned} \sigma_{X}^{2}=\frac{(x-\mu)^{2}}{N} =E(X^{2})-[E(X)]^{2} \end{aligned}\]How does this help us? Try to work it out.
\[\begin{gather*} \sigma_{X}^{2}=E(X^{2})-[E(X)]^{2}\\ \sigma_{X}=1, \sigma_{X}^{2} = 1^{2}=1\\ 1=E(X^{2})-0^{2}\\ E(XY)=0.5 \times E(X^{2})+E(X \times Z)\\ E(XY)=0.5 \times 1^{2} + E(X) \times E(Z)\\ since \quad X \perp Z\\ E(XY)=0.5+0 \times 0 = 0.5\\ Cov(X,Y) = E(XY) - E(X) \times E(Y) = 0.5-0 \times 0 = 0.5\\ \end{gather*}\]
\[\begin{gather*} \rho=\frac{Cov(X,Y)}{\sigma_{X}\sigma_{Y}} \\ =\frac{0.5}{1 \times \sigma_{Y}} \\ \sigma_{Y}=\sqrt{\sigma_{Y}^{2}} \\ \sigma_{Y}^{2}= var(0.5\times X + Z)=\\ 0.5^{2}\times \sigma_{X}^{2}+\sigma_{Z}^{2}+2\times 0.5 \times Cov(X,Z)= \\ 1.25=0.25 \times 1 + 1 + 0 \\ \rho=\frac{0.5}{\sqrt{1.25}}=0.4472136 \end{gather*}\]
1 \(\rho\) is 0.4472136, \(r\) is not.
2 What does it mean for \(\rho_{X,Y}\) = 0.45
3 If you have 3 sets of correlations \(\rho_{X,Y}\) =0.45, \(\rho_{A,Y}\) = 0.75 and \(\rho_{A,X}\)=0.8 what are the conclusions you can draw?
A and X are positively correlated. The strength of \(\rho_{A,X}\) are stronger compared to the correlation of each variable compared to Y. This large value indicates if we try to build a model to explain how Y occurs, one of these variables (A,X) might be causing the other one.
None of these are given in the data but they are questions you need to think about when investigating the statistics from the data.
In the environment of automation it becomes more important to ask good questions.
- We have by default New Worksheet Ply Selected. This will create the table of correlations in a new worksheet.
Why does this table have missing rows?
Why?
\[r_{X,Y}=\frac{Cov(X,Y)}{\sigma_{X}\sigma_{Y}}=r_{Y,X}=\frac{Cov(Y,X)}{\sigma_{Y}\sigma_{X}} \] \(Cov(X,Y)=E[XY]-E[X]E[Y]=E[YX]-E[Y]E[X]\)
#Read the dataset which has header labels (X,Y,A),
#columns separated by tab delimited '\t'
data=read.table('C:/Users/rm84/Desktop/teaching/2333/2333_Linear_Regression_files/data.txt',header=TRUE,sep='\t')
cor(data) X Y A
X 1.0000000 0.4219863 0.6999944
Y 0.4219863 1.0000000 0.6101665
A 0.6999944 0.6101665 1.0000000read.table function reads the tab delimited values.
Full table instead of just the diagonal and lower triangle since the cor table can be used as an object in algorithms that require the whole table.