Outline
Goal: Learn how to use simulation to answer statistical questions!
In science, how do we support or refute a hypothesis?
Instead, we can run a simulation.
Simulation is an important (and big) topic for both statistics and for a variety of other areas where there is a need to introduce randomness.
Sometimes you want to implement a statistical procedure that requires random number generation or sampling (i.e. Markov chain Monte Carlo, the bootstrap, random forests, bagging) and sometimes you want to simulate a system and random number generators can be used to model random inputs.
R comes with a set of pseuodo-random number generators that allow you to simulate from well-known probability distributions like the Normal, Poisson, and binomial.
Some example functions for probability distributions in R.
For each probability distribution there are typically four functions available that start with a “r”, “d”, “p”, and “q”.
The “r” function is the one that actually simulates random numbers from that distribution.
The other functions are prefixed with a
If you’re only interested in simulating random numbers, then you will likely only need the “r” functions and not the others.
Probably the most common probability distribution to work with the is the Normal distribution (also known as the Gaussian). Working with the Normal distributions requires using these four functions.
## [1] -0.0410621 1.7128294 -1.1591722 -0.9578240 -0.3778651 0.1273887
## [7] 0.8001582 0.2627204 -0.7230348 -0.2266853
## [1] 0.4449222 -2.0122477 -0.1062258 0.7782035 0.4791456 1.2956687
## [7] 1.4761441 -0.8072108 0.2729874 0.6531121
## [1] -0.6264538 0.1836433 -0.8356286 1.5952808 0.3295078 -0.8204684
## [7] 0.4874291 0.7383247 0.5757814 -0.3053884
## [1] -0.6264538 0.1836433 -0.8356286 1.5952808 0.3295078 -0.8204684
## [7] 0.4874291 0.7383247 0.5757814 -0.3053884
## [1] 18.74709 20.36729 18.32874 23.19056 20.65902 18.35906 20.97486 21.47665
## [9] 21.15156 19.38922
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 18.33 18.91 20.51 20.26 21.11 23.19
It is possible to generate random numbers from other probability distributions like the Poisson. The Poisson distribution is commonly used to model data that come in the form of counts.
rpois(n, lambda)
## [1] 0 1 1 2 0 2 3 1 1 0
## [1] 16 18 19 30 23 16 17 18 18 21
## [1] 1 2 2 3 1 3 4 2 2 0
## [1] 2
## [1] 2.49
We can individually generate data and calculate sample means. But this doesn’t give us information about the variance of these means.
rexp(n, rate = 1)
## [1] 1.669803
## [1] 0.9870225
## [1] 1.026968
## [1] 1.014509
Simulating random numbers is useful but sometimes we want to simulate values that come from a specific model.
For that we need to specify the model and then simulate from it using the functions described above.
Suppose we want to simulate from the following linear model with intercept 0.5 and slope 2. The variable x might represent an important predictor of the outcome y. Here’s how we could do that in R.
set.seed(1)
x <- rnorm(100, 0, 1) ## Simulate predictor variable
e <- rnorm(100, 0, 2) ## Simulate the error term
y <- 0.5 + 2 * x + e ## Compute the outcome via the model
summary(y)## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -5.4816 -1.1493 0.7582 0.6422 2.3404 6.1534
##
## Call:
## lm(formula = y ~ x)
##
## Residuals:
## Min 1Q Median 3Q Max
## -3.754 -1.228 -0.279 1.079 4.692
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.4246 0.1940 2.189 0.031 *
## x 1.9979 0.2155 9.273 4.58e-15 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 1.926 on 98 degrees of freedom
## Multiple R-squared: 0.4674, Adjusted R-squared: 0.4619
## F-statistic: 85.99 on 1 and 98 DF, p-value: 4.583e-15
## [1] 1 1 1 1 1 1 0 0 1 1 0 1 0 0 0 0 0 1 1 0
##
## Shapiro-Wilk normality test
##
## data: scores
## W = 0.95327, p-value = 0.4195
##
## Two Sample t-test
##
## data: scores by groups
## t = -0.074459, df = 18, p-value = 0.9415
## alternative hypothesis: true difference in means between group 0 and group 1 is not equal to 0
## 95 percent confidence interval:
## -1.834543 1.708957
## sample estimates:
## mean in group 0 mean in group 1
## 10.34651 10.40930
##
## Welch Two Sample t-test
##
## data: scores by groups
## t = -0.071915, df = 14.06, p-value = 0.9437
## alternative hypothesis: true difference in means between group 0 and group 1 is not equal to 0
## 95 percent confidence interval:
## -1.934786 1.809200
## sample estimates:
## mean in group 0 mean in group 1
## 10.34651 10.40930
A matrix object is a rectangular \(n\) × \(m\) array of elements of same type: numerical, character, etc.
\(n\) is the number of rows, \(m\) is the number of columns.
Typically rows represent subjects, and columns represent different variables measured for each subject.
The rectangular data structure ensures same number of measurements per subject.
Having more than one variable per subject allows us to examine correlations between various measurements.
We could also view such data as a collection of equal length variable vectors, stacked next to each other.
## [,1] [,2] [,3] [,4]
## [1,] 1 4 7 10
## [2,] 2 5 8 11
## [3,] 3 6 9 12
## [,1] [,2] [,3] [,4]
## [1,] "a" "b" "c" "d"
## [2,] "e" "f" "g" "h"
## [3,] "i" "j" "k" "l"
## [,1] [,2] [,3] [,4]
## [1,] 1 4 7 10
## [2,] 2 5 8 11
## [3,] 3 6 9 12
## [,1] [,2] [,3] [,4]
## [1,] "a" "b" "c" "d"
## [2,] "e" "f" "g" "h"
## [3,] "i" "j" "k" "l"
## NULL
## [,1] [,2] [,3] [,4]
## row1 1 4 7 10
## row2 2 5 8 11
## row3 3 6 9 12
## col1 col2 col3 col4
## row1 1 4 7 10
## row2 2 5 8 11
## row3 3 6 9 12
## col1 col2 col3 col4
## row1 1 4 7 10
## row2 2 5 8 11
## row3 3 6 9 12
## [1] 3 4
## col3 col4
## row1 7 10
## row2 8 11
Matrices are added by adding corresponding elements.
Same for -, *, / .
Matrices must have same dimension (columns and rows).
## [,1] [,2] [,3] [,4]
## [1,] 12 9 6 3
## [2,] 11 8 5 2
## [3,] 10 7 4 1
## col1 col2 col3 col4
## row1 13 13 13 13
## row2 13 13 13 13
## row3 13 13 13 13
## col1 col2 col3 col4
## row1 1 4 7 10
## row2 2 5 8 11
## row3 3 6 9 12
## col1 col2 col3 col4
## row1 2 5 8 11
## row2 4 7 10 13
## row3 6 9 12 15
## col1 col2 col3 col4
## row1 2 5 8 11
## row2 4 7 10 13
## row3 6 9 12 15
## col1 col2 col3 col4
## row1 1 4 7 10
## row2 2 5 8 11
## row3 3 6 9 12
## col1 col2 col3 col4
## row1 2 8 10 12
## row2 4 6 12 14
## row3 6 8 10 16
## [,1] [,2]
## [1,] 1 3
## [2,] 2 4
## [,1] [,2] [,3]
## [1,] 6 4 2
## [2,] 5 3 1
## [,1] [,2] [,3]
## [1,] 21 13 5
## [2,] 32 20 8
## [,1] [,2]
## [1,] 1 3
## [2,] 2 4
## [1] 2 3
## [,1]
## [1,] 11
## [2,] 16
## [,1] [,2]
## [1,] 1 3
## [2,] 2 4
## [,1] [,2]
## [1,] -2 1.5
## [2,] 1 -0.5
## [,1] [,2]
## [1,] 1 0
## [2,] 0 1
## [,1] [,2] [,3]
## [1,] 1 4 7
## [2,] 2 5 8
## [3,] 3 6 9
## [,1] [,2] [,3]
## [1,] 1 4 7
## [2,] 2 10 8
## [3,] 3 6 9
## [,1] [,2] [,3]
## [1,] 1 4 7
## [2,] 2 10 8
## [3,] 3 24 9
## [,1] [,2] [,3]
## [1,] 1 4 7
## [2,] 2 5 8
## [3,] 3 6 9
## [,1] [,2] [,3]
## [1,] 0 0 7
## [2,] 0 5 8
## [3,] 0 6 9
## [,1] [,2] [,3]
## [1,] 1 4 7
## [2,] 2 5 8
## [3,] 3 6 9
## [,1] [,2] [,3]
## [1,] 1 2 3
## [2,] 4 5 6
## [3,] 7 8 9
We can fill in a matrix using indices.
In R, you should always start by initializing an empty matrix of the right size.
## [,1] [,2] [,3]
## [1,] NA NA NA
## [2,] NA NA NA
## [3,] NA NA NA
## [,1] [,2] [,3]
## [1,] NA NA NA
## [2,] 2 4 3
## [3,] NA NA NA
## [,1] [,2] [,3]
## [1,] 7 7 7
## [2,] 2 4 3
## [3,] 7 7 7
## [,1] [,2] [,3]
## [1,] 7 7 7
## [2,] 7 7 7
## [3,] 7 7 7
## [,1] [,2] [,3]
## [1,] 7 7 7
## [2,] 7 7 7
## [3,] 7 7 7
## [,1] [,2] [,3]
## [1,] "1" "2" "3"
## [2,] "a" "b" "c"
## [,1] [,2] [,3]
## [1,] "1" "2" "3"
## [2,] "a" "b" "c"
## [3,] "X" "Y" "Z"
## [,1] [,2] [,3]
## [1,] 0.5000000 0.8571429 0.9166667
## [2,] 0.6666667 0.8750000 0.9230769
## [3,] 0.7500000 0.8888889 0.9285714
## [4,] 0.8000000 0.9000000 0.9333333
## [5,] 0.8333333 0.9090909 0.9375000
## [,1] [,2] [,3]
## [1,] 1 6 11
## [2,] 2 7 12
## [3,] 3 8 13
## [4,] 4 9 14
## [5,] 5 10 15
## [1] 18 21 24 27 30
## [1] 15 40 65
## [,1] [,2] [,3]
## [1,] 1 6 11
## [2,] 2 7 12
## [3,] 3 8 13
## [4,] 4 9 14
## [5,] 5 10 15
## [1] 6 7 8 9 10
## [1] 3 8 13
##
## Call:
## lm(formula = y ~ x)
##
## Residuals:
## 1 2 3 4 5
## 5.672 -11.900 1.707 10.193 -5.672
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -65.636 11.715 -5.603 0.01123 *
## x 20.786 1.829 11.368 0.00146 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 10.21 on 3 degrees of freedom
## Multiple R-squared: 0.9773, Adjusted R-squared: 0.9697
## F-statistic: 129.2 on 1 and 3 DF, p-value: 0.001461
##
## Call:
## lm(formula = y ~ x)
##
## Residuals:
## 1 2 3 4 5
## 5.672 -11.900 1.707 10.193 -5.672
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -65.636 11.715 -5.603 0.01123 *
## x 20.786 1.829 11.368 0.00146 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 10.21 on 3 degrees of freedom
## Multiple R-squared: 0.9773, Adjusted R-squared: 0.9697
## F-statistic: 129.2 on 1 and 3 DF, p-value: 0.001461
## [1] 312.7489
## [1] 104.2496
## 2.5 % 97.5 %
## (Intercept) -102.9177 -28.35429
## x 14.9666 26.60492
## 5 % 95 %
## (Intercept) -93.20515 -38.06682
## x 16.48259 25.08893
## 1
## 104.8072
## fit lwr upr
## 1 104.8072 85.05122 124.5633
## [,1] [,2]
## [1,] 1 2.5
## [2,] 1 4.5
## [3,] 1 5.0
## [4,] 1 8.2
## [5,] 1 9.3
## [,1]
## [1,] -8
## [2,] 16
## [3,] 40
## [4,] 115
## [5,] 122
X <- matrix(c(rep(1, 5), 2.5, 4.5, 5, 8.2, 9.3), nrow = 5)
Y <- matrix(c(-8, 16, 40, 115, 122), nrow = 5)
betas <- solve(t(X) %*% X) %*% t(X) %*% Y
betas## [,1]
## [1,] -65.63598
## [2,] 20.78576
## [1] 312.7489
## [1] 13784
## [1] 0.9773107
## [,1]
## [1,] 0.9697476
## [,1]
## [1,] 312.7489
## [1] -65 -41 -17 58 65
## [,1]
## [1,] 13784
## [,1]
## [1,] 13471.25
## [,1]
## [1,] 13471.25
## [,1]
## [1,] 104.2496
## [,1]
## [1,] 129.2211
An auto part is manufactured by a company once a month in lots that vary in size as demand fluctuates.
The data below represent observations on lot size (y), and number of man-hours of labor (x) for 10 recent production runs.
y <- c(73,50,128,170,87,108,135,69,148,132)
x <- c(30,20,60,80,40,50,60,30,70,60)
model_1 <- lm(y ~ x)
summary(model_1) ##
## Call:
## lm(formula = y ~ x)
##
## Residuals:
## Min 1Q Median 3Q Max
## -3.0 -2.0 -0.5 1.5 5.0
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 10.00000 2.50294 3.995 0.00398 **
## x 2.00000 0.04697 42.583 1.02e-10 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.739 on 8 degrees of freedom
## Multiple R-squared: 0.9956, Adjusted R-squared: 0.9951
## F-statistic: 1813 on 1 and 8 DF, p-value: 1.02e-10
y <- matrix(c(73,50,128,170,87,108,135,69,148,132), nrow = 10)
x <- matrix(c(rep(1, 10), 30,20,60,80,40,50,60,30,70,60), nrow = 10)
betas <- solve(t(x) %*% x) %*% t(x) %*% y
betas## [,1]
## [1,] 10
## [2,] 2
## 2.5 % 97.5 %
## (Intercept) 4.228211 15.771789
## x 1.891694 2.108306
## [1] 60
## [1] 13784
## [1] 0.9956471
## [,1]
## [1,] 0.995103
## [1] 60
## [1] 7.5
## [1] 2.738613
## [,1]
## [1,] 60
## [,1]
## [1,] -37
## [2,] -60
## [3,] 18
## [4,] 60
## [5,] -23
## [6,] -2
## [7,] 25
## [8,] -41
## [9,] 38
## [10,] 22
## [,1]
## [1,] 13660
## [,1]
## [1,] 13600
## [,1]
## [1,] 13600
## [,1]
## [1,] 7.5
## [,1]
## [1,] 1813.333
## [,1] [,2]
## [1,] 0.83529412 -0.0147058824
## [2,] -0.01470588 0.0002941176
## [,1]
## [1,] 6.264706
## [,1]
## [1,] 0.002205882
## [1] 2.306004
## [,1]
## [1,] 4.228211
## [,1]
## [1,] 15.77179
## [,1]
## [1,] 1.891694
## [,1]
## [1,] 2.108306
## Dataset
df <- structure(list(x1 = c(68.5, 45.2, 91.3, 47.8, 46.9, 66.1, 49.5,
52, 48.9, 38.4, 87.9, 72.8, 88.4, 42.9, 52.5, 85.7, 41.3, 51.7,
89.6, 82.7, 52.3), x2 = c(16.7, 16.8, 18.2, 16.3, 17.3, 18.2,
15.9, 17.2, 16.6, 16, 18.3, 17.1, 17.4, 15.8, 17.8, 18.4, 16.5,
16.3, 18.1, 19.1, 16), y = c(174.4, 164.4, 244.2, 154.6, 181.6,
207.5, 152.8, 163.2, 145.4, 137.2, 241.9, 191.1, 232, 145.3,
161.1, 209.7, 146.4, 144, 232.6, 224.1, 166.5)), .Names = c("x1",
"x2", "y"), class = "data.frame", row.names = c(NA, -21L))
head(df)## x1 x2 y
## 1 68.5 16.7 174.4
## 2 45.2 16.8 164.4
## 3 91.3 18.2 244.2
## 4 47.8 16.3 154.6
## 5 46.9 17.3 181.6
## 6 66.1 18.2 207.5
##
## Call:
## lm(formula = y ~ x1 + x2, data = df)
##
## Residuals:
## Min 1Q Median 3Q Max
## -18.4239 -6.2161 0.7449 9.4356 20.2151
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -68.8571 60.0170 -1.147 0.2663
## x1 1.4546 0.2118 6.868 2e-06 ***
## x2 9.3655 4.0640 2.305 0.0333 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 11.01 on 18 degrees of freedom
## Multiple R-squared: 0.9167, Adjusted R-squared: 0.9075
## F-statistic: 99.1 on 2 and 18 DF, p-value: 1.921e-10
## 2.5 % 97.5 %
## (Intercept) -194.9480130 57.233867
## x1 1.0096226 1.899497
## x2 0.8274411 17.903560
## 1 2 3 4 5 6 7 8 9 10 11 12 13
## (Intercept) 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
## x1 68.5 45.2 91.3 47.8 46.9 66.1 49.5 52.0 48.9 38.4 87.9 72.8 88.4
## x2 16.7 16.8 18.2 16.3 17.3 18.2 15.9 17.2 16.6 16.0 18.3 17.1 17.4
## 14 15 16 17 18 19 20 21
## (Intercept) 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
## x1 42.9 52.5 85.7 41.3 51.7 89.6 82.7 52.3
## x2 15.8 17.8 18.4 16.5 16.3 18.1 19.1 16.0
## attr(,"assign")
## [1] 0 1 2
## 1 2 3 4 5 6 7 8 9 10 11 12 13
## y 174.4 164.4 244.2 154.6 181.6 207.5 152.8 163.2 145.4 137.2 241.9 191.1 232
## 14 15 16 17 18 19 20 21
## y 145.3 161.1 209.7 146.4 144 232.6 224.1 166.5
## [,1]
## [1,] -68.85707
## [2,] 1.45456
## [3,] 9.36550
## [,1] [,2]
## 1 187.1841 187.1841
## 2 154.2294 154.2294
## 3 234.3963 234.3963
## 4 153.3285 153.3285
## 5 161.3849 161.3849
## 6 197.7414 197.7414
## 7 152.0551 152.0551
## 8 167.8666 167.8666
## 9 157.7382 157.7382
## 10 136.8460 136.8460
## 11 230.3874 230.3874
## 12 197.1849 197.1849
## 13 222.6857 222.6857
## 14 141.5184 141.5184
## 15 174.2132 174.2132
## 16 228.1239 228.1239
## 17 145.7470 145.7470
## 18 159.0013 159.0013
## 19 230.9870 230.9870
## 20 230.3161 230.3161
## 21 157.0644 157.0644
## [,1] [,2]
## 1 187.1841 187.1841
## 2 154.2294 154.2294
## 3 234.3963 234.3963
## 4 153.3285 153.3285
## 5 161.3849 161.3849
## 6 197.7414 197.7414
## 7 152.0551 152.0551
## 8 167.8666 167.8666
## 9 157.7382 157.7382
## 10 136.8460 136.8460
## 11 230.3874 230.3874
## 12 197.1849 197.1849
## 13 222.6857 222.6857
## 14 141.5184 141.5184
## 15 174.2132 174.2132
## 16 228.1239 228.1239
## 17 145.7470 145.7470
## 18 159.0013 159.0013
## 19 230.9870 230.9870
## 20 230.3161 230.3161
## 21 157.0644 157.0644