vec1 <- 1:1000
vec2 <- sample(vec1)
Dat <- data.frame(vec1,vec2)
cor(Dat)
## vec1 vec2
## vec1 1.0000000 0.0482459
## vec2 0.0482459 1.0000000
I expected this correlation to be small because vec2 is a randomized sample of vec1. Since vec1 is ordered from 0 to 1000, its values increase steadily with its position. Vec2 draws those same values in random order and so there’s no real relationship between the value at a given index in vec1 and the value at the same index in vec2. This means that they are highly uncorrelated.
hdat <- read.csv("data_health_synth_small.csv")
nrow(hdat)
## [1] 48784
ncol(hdat)
## [1] 4
There are 48,784 rows and 4 columns. Each row corresponds to an individual person in the health synth health dataset, while the columns correspond to variables measured for that person (column 1 is cost, column 2 is race, column 3 is gender, and column 4 is bps_mean).
summary(hdat)
## cost race female bps_mean
## Min. : 0 Length :48784 Min. :0.0000 Min. : 0.0
## 1st Qu.: 1200 N.unique : 2 1st Qu.:0.0000 1st Qu.: 118.0
## Median : 2800 N.blank : 0 Median :1.0000 Median : 127.0
## Mean : 7660 Min.nchar: 5 Mean :0.6306 Mean : 127.3
## 3rd Qu.: 6600 Max.nchar: 5 3rd Qu.:1.0000 3rd Qu.: 136.0
## Max. :550500 Max. :1.0000 Max. :1323.0
## NAs :10668
The dataset has 48,784 observations with the cost ranging from 0 to 550,500 with a median of 2,800 and mean of 7,660. About 63% of the sample is female. bps_mean has a median of 127 and mean of 127.3, with 10,668 missing values.
The two curves show how sick patients are (number of chronic conditions on the y-axis) at each level of the algorithm’s risk score (x-axis), one for Black patients and one for White patients. They diverge because the purple curve is always higher; at the same risk score Black patients are sicker than White patients. Since the score decides who gets extra care, Black patients have to be sicker to qualify and so many miss out on the help they need. This happens because the algorithm uses how much money is spent on a patient to guess how sick they are and because less money is spent on Black patients it wrongly assumes they’re healthier.
worked with Jadyn Sabatino