Problem: to judge how well a model works, I need to compare its results with the truth, but with real data the truth is unknown. Goal: create a dataset where I decide the true relationship between x and y, so I can later check whether the models recover it.
How:
set.seed(123): fixes the starting point of the random
number generator1:20: generates the whole numbers from 1 to 20y = 20 + 10*x + 5*x^2 + noise, where 20 is the intercept,
10 the slope (linear term) and 5 the curvature (quadratic term)
rnorm(20, 0, 200): generates 20 random values from a
normal distribution with mean 0 and sd 200; this is the noise that makes
the relationship statistical instead of exactplot(): draws the points of y against xset.seed(123) # 1.1
x <- 1:20 # 1.2
y <- 20 + 10*x + 5*x^2 + rnorm(20, 0, 200) # 1.3
plot(x, y) # 1.4
Interpretation. The points show a curved, increasing pattern: y grows faster as x increases. This curvature comes from the quadratic term (5x²) built into the simulation. The points do not fall exactly on a smooth curve because of the random noise, which makes the relationship statistical rather than exact.
What this does not tell me: the plot shows the shape of the data, but not which model describes it best or whether a model can recover the true parameters. Step 2 answers this.
Problem: the data follow a curve with noise, but I do not know which model describes them best. Goal: fit three models of increasing complexity and check which one fits best and whether it recovers the true parameters (20, 10, 5).
How:
lm(): fits a linear model; the formula
y ~ 1 means “y explained only by a constant”lm(y ~ x): adds x as a predictorI(x^2): tells R to compute x² literally, because inside
a formula ^ means something elsesummary(): shows the estimated coefficients (Estimate),
their uncertainty (Std. Error), their p-values, the residual standard
error and the adjusted R²model_1 <- lm(y ~ 1) # 2.1
model_2 <- lm(y ~ x) # 2.2
model_3 <- lm(y ~ x + I(x^2)) # 2.3
summary(model_1) # 2.4
Call:
lm(formula = y ~ 1)
Residuals:
Min 1Q Median 3Q Max
-947.9 -532.4 -103.3 373.7 1284.5
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 870.8 156.6 5.56 2.31e-05 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 700.5 on 19 degrees of freedom
summary(model_2) # 2.4
Call:
lm(formula = y ~ x)
Residuals:
Min 1Q Median 3Q Max
-424.4 -127.4 -12.6 151.0 374.4
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -302.981 110.115 -2.752 0.0131 *
x 111.791 9.192 12.162 4.07e-10 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 237 on 18 degrees of freedom
Multiple R-squared: 0.8915, Adjusted R-squared: 0.8855
F-statistic: 147.9 on 1 and 18 DF, p-value: 4.068e-10
summary(model_3) # 2.4
Call:
lm(formula = y ~ x + I(x^2))
Residuals:
Min 1Q Median 3Q Max
-379.29 -128.02 -17.08 117.01 344.32
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 20.770 151.083 0.137 0.8923
x 23.495 33.135 0.709 0.4879
I(x^2) 4.205 1.533 2.743 0.0139 *
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 203.1 on 17 degrees of freedom
Multiple R-squared: 0.9248, Adjusted R-squared: 0.9159
F-statistic: 104.5 on 2 and 17 DF, p-value: 2.806e-10
Interpretation. Model 3
(y ~ x + I(x^2)) gives the best fit: it has the lowest
residual standard error (203.1) and the highest adjusted R² (0.916, it
explains 91.6% of the variation in y). Model 1 (y ~ 1)
explains nothing (R² = 0) because it always predicts the mean of y and
ignores x. Model 2 (y ~ x) fits much better (R² = 0.886),
but its error is larger because a straight line cannot capture the
curvature. The residual standard error of model 3 (203.1) is very close
to the sd of the noise I simulated (200): the model explains everything
that can be explained, and what remains is only random noise.
Model 3 also recovers the true parameters: the estimated intercept (20.8) and curvature (4.2) are close to the true values (20 and 5). The estimated slope (23.5) is further from the true value (10), but its standard error is large (33.1), so the plausible range (23.5 ± 2 × 33.1 = −42.7 to 89.7) includes the true value. With only 20 points and a large noise, the slope is estimated with low precision. The quadratic term is significant (p = 0.014), confirming that the curvature is real.
What this does not tell me: these numbers summarize each model as a whole, but they do not show where, at which values of x, each model succeeds or fails. Step 3 answers this.
Problem: the summaries give one global number per model, so I cannot see where each model fails. Goal: compare, point by point, the value each model predicts with the observed value.
How:
predict(): applies the fitted model to the x values and
returns the predicted ydata.frame(): builds a table from several vectors of
the same lengthhead(): prints the first 6 rowsfitted_1 <- predict(model_1) # 3.1
fitted_2 <- predict(model_2) # 3.1
fitted_3 <- predict(model_3) # 3.1
fit_table <- data.frame(x, observed = y, fitted_1, fitted_2, fitted_3) # 3.2
head(fit_table) # 3.3
x observed fitted_1 fitted_2 fitted_3
1 1 -77.09513 870.8248 -191.19043 48.47006
2 2 13.96450 870.8248 -79.39935 84.57888
3 3 406.74166 870.8248 32.39172 129.09683
4 4 154.10168 870.8248 144.18279 182.02392
5 5 220.85755 870.8248 255.97386 243.36015
6 6 603.01300 870.8248 367.76494 313.10553
Interpretation. Model 1 predicts the same value for every x (870.8, the mean of y), so it ignores the relationship between x and y. Model 2 follows the general increasing trend, but at low x values it predicts negative values (for example −191.2 for x = 1, while the observed value is −77.1), because a straight line cannot bend to follow the curvature. Model 3 gives fitted values closer to the observed ones across the range of x. None of the models matches the observed values exactly, because of the random noise.
What this does not tell me: comparing two columns by eye does not measure how much each model misses at each point, or in which direction. Step 4 answers this.
Problem: the fitted values never match the observed ones exactly, but comparing two columns by eye is not precise. Goal: measure the error of each model at each point, the residual (observed − fitted), which is what the model fails to explain.
How:
resid(): returns observed − fitted for each point; a
positive residual means the point is above the model’s prediction, a
negative one means it is belowdata.frame(): builds the table from scratch, so the
columns are not duplicated if the chunk runs twicehead(): prints the first 6 rowsresid_1 <- resid(model_1) # 4.1
resid_2 <- resid(model_2) # 4.1
resid_3 <- resid(model_3) # 4.1
fit_table <- data.frame(x, observed = y, fitted_1, fitted_2, fitted_3, resid_1, resid_2, resid_3) # 4.2
head(fit_table) # 4.3
x observed fitted_1 fitted_2 fitted_3 resid_1 resid_2 resid_3
1 1 -77.09513 870.8248 -191.19043 48.47006 -947.9199 114.095296 -125.56519
2 2 13.96450 870.8248 -79.39935 84.57888 -856.8603 93.363856 -70.61437
3 3 406.74166 870.8248 32.39172 129.09683 -464.0831 374.349944 277.64483
4 4 154.10168 870.8248 144.18279 182.02392 -716.7231 9.918887 -27.92224
5 5 220.85755 870.8248 255.97386 243.36015 -649.9672 -35.116316 -22.50261
6 6 603.01300 870.8248 367.76494 313.10553 -267.8118 235.248062 289.90747
Interpretation. Model 1 has by far the largest residuals (for example −947.9 at x = 1), and in the first rows they are all negative: the flat line (the mean of y) lies above every low-x point. A systematic sign pattern like this means the model is missing part of the relationship. The residuals of models 2 and 3 are much smaller and similar in size in these first rows.
What this does not tell me: six rows are not enough to tell models 2 and 3 apart, or to see whether the residuals follow a pattern across all values of x. Step 5 answers this.
Problem: the tables only show a few points, and models 2 and 3 look similar there. Goal: plot all 20 points to see which model follows the data best, and whether the residuals show a pattern or are just random noise.
How:
plot(): draws the observed points;
pch = 19 makes them solid circleslines(): draws a line through the fitted values;
col sets the color and lwd the line widthlegend(): adds a box that identifies each model by its
colorpar(mfrow = c(1, 3)): splits the plot window into 1 row
and 3 columnsabline(h = 0, lty = 2): draws a dashed horizontal line
at 0; in a good model the residuals are scattered randomly around it,
with no patternpar(mfrow = c(1, 1)): resets the window to one
plotplot(x, y, pch = 19, col = "grey50",
main = "Observed data and fitted models") # 5.1
lines(x, fitted_1, col = "blue", lwd = 2) # 5.1
lines(x, fitted_2, col = "red", lwd = 2) # 5.1
lines(x, fitted_3, col = "darkgreen", lwd = 2) # 5.1
legend("topleft", legend = c("y ~ 1", "y ~ x", "y ~ x + I(x^2)"),
col = c("blue", "red", "darkgreen"), lwd = 2) # 5.1
par(mfrow = c(1, 3)) # 5.2
plot(x, resid_1, pch = 19, main = "Residuals y ~ 1") # 5.2
abline(h = 0, lty = 2) # 5.2
plot(x, resid_2, pch = 19, main = "Residuals y ~ x") # 5.2
abline(h = 0, lty = 2) # 5.2
plot(x, resid_3, pch = 19, main = "Residuals y ~ x + I(x^2)") # 5.2
abline(h = 0, lty = 2) # 5.2
par(mfrow = c(1, 1)) # 5.2
Interpretation. In the first plot, model 1 (flat line) does not follow the data at all. Model 2 (straight line) follows the general trend, but it lies below the points at the beginning and at the end, and above them in the middle, because it cannot bend with the curvature. Model 3 (parabola) follows the curve along the whole range of x.
The residual plots confirm this. The residuals cannot be close to zero, because the noise I simulated has sd = 200, so even the correct model misses by about ±200. What matters is whether they show a pattern. Model 1 shows a clear increasing trend: it ignores x completely. Model 2 shows a U-shaped pattern (mostly positive at low and high x, mostly negative in the middle), the signature of the curvature it misses. Model 3 shows residuals scattered randomly around zero with no visible pattern, which means it captures the systematic part of the relationship and only random noise remains.
What this does not tell me: all these results come from one noise level (sd = 200), so I do not know how much of the remaining error, and of the precision of the estimates, depends on the noise. Step 6 answers this.
Problem: all the results so far depend on one noise level (sd = 200), so I do not know how much of what I saw is caused by the noise. Goal: repeat the simulation with a smaller noise (sd = 20) and a larger noise (sd = 1000), keeping everything else the same, and see how the fit of model 3 and its ability to recover the true parameters change.
How:
set.seed(123) before each rnorm(): gives
the same random pattern, only scaled to a different size, so the only
difference between datasets is the amount of noiselm(y_low ~ x + I(x^2)) and
lm(y_high ~ x + I(x^2)): the same correct model as
beforesummary(): shows these values for each modelpredict(): gives the fitted values of each new model,
drawn with lines()set.seed(123) # 6.1
y_low <- 20 + 10*x + 5*x^2 + rnorm(20, 0, 20) # 6.1
set.seed(123) # 6.1
y_high <- 20 + 10*x + 5*x^2 + rnorm(20, 0, 1000) # 6.1
model3_low <- lm(y_low ~ x + I(x^2)) # 6.2
model3_high <- lm(y_high ~ x + I(x^2)) # 6.2
summary(model3_low) # 6.3
Call:
lm(formula = y_low ~ x + I(x^2))
Residuals:
Min 1Q Median 3Q Max
-37.929 -12.802 -1.708 11.701 34.432
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 20.0770 15.1083 1.329 0.20145
x 11.3495 3.3135 3.425 0.00323 **
I(x^2) 4.9205 0.1533 32.104 < 2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 20.31 on 17 degrees of freedom
Multiple R-squared: 0.9992, Adjusted R-squared: 0.9991
F-statistic: 1.112e+04 on 2 and 17 DF, p-value: < 2.2e-16
summary(model3_high) # 6.3
Call:
lm(formula = y_high ~ x + I(x^2))
Residuals:
Min 1Q Median 3Q Max
-1896.43 -640.11 -85.41 585.04 1721.60
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 23.852 755.417 0.032 0.975
x 77.476 165.675 0.468 0.646
I(x^2) 1.023 7.663 0.133 0.895
Residual standard error: 1015 on 17 degrees of freedom
Multiple R-squared: 0.2714, Adjusted R-squared: 0.1857
F-statistic: 3.167 on 2 and 17 DF, p-value: 0.06775
par(mfrow = c(1, 3)) # 6.4
plot(x, y_low, pch = 19, col = "grey50", main = "Noise sd = 20") # 6.4
lines(x, predict(model3_low), col = "darkgreen", lwd = 2) # 6.4
plot(x, y, pch = 19, col = "grey50", main = "Noise sd = 200") # 6.4
lines(x, fitted_3, col = "darkgreen", lwd = 2) # 6.4
plot(x, y_high, pch = 19, col = "grey50",
main = "Noise sd = 1000") # 6.4
lines(x, predict(model3_high), col = "darkgreen", lwd = 2) # 6.4
par(mfrow = c(1, 1)) # 6.4
Interpretation. The amount of noise controls how well the model can fit the data and recover the truth. In all three cases the residual standard error of model 3 is close to the sd of the noise (20.3, 203.1 and about 1015), which confirms that the remaining error is only the random noise I built in. With small noise (sd = 20) the points follow an almost perfect parabola, R² is 0.999, and the estimates are very close to the true values (20.1, 11.3 and 4.92 vs 20, 10 and 5), with small standard errors. With the original noise (sd = 200) the estimates are less precise but the true values are still inside their plausible ranges. With large noise (sd = 1000) the curvature is no longer visible in the plot, the estimated quadratic term drops to about 1 and is no longer significant: even though model 3 is the correct model, 20 noisy points do not contain enough information to detect the curvature.
What this does not tell me: here I knew the true relationship because I built it. With real biological data the truth is unknown, the relationship may not be a simple curve, and the values may need normalization. Questions 2 to 5 apply the same tools to a real gene expression dataset.
y ~ x + I(x^2)) is the best
model: it has the lowest residual standard error (203.1) and
the highest adjusted R² (0.916), and its residuals show no pattern.y ~ 1) explains nothing: it
always predicts the mean of y (R² = 0), and its residuals show a clear
increasing trend.y ~ x) captures the trend but misses
the curvature: its residuals show a U-shaped pattern, the
signature of the quadratic term it leaves out.y ~ x + I(x^2) is still a linear model, because it is
linear in its coefficients, even if it is not linear in x.Problem: to fit a linear model for each gene, I need a continuous variable measured in the same samples (for example age or BMI), but I do not know yet which variables this dataset contains. Goal: download the sample information of GSE226151 and find a continuous variable to use as the dependent variable.
How:
library(GEOquery): loads the functions to download data
from GEOgetGEO("GSE226151", GSEMatrix = TRUE): downloads the
study; it returns a list, with one element per platformgse[[1]]: takes the first element of the listpData(): extracts the sample tablecolnames(): shows the column names; the sample
characteristics usually end in :ch1meta[, c(...)]: selects only the columns listed inside
c()head(): shows the first 6 rowsnrow(): counts the number of samples (rows)table(): counts how many samples there are in each
disease stateas.numeric(): converts the text values into
numberssummary(): shows the minimum, quartiles, mean and
maximumlibrary(GEOquery) # 1.1
gse <- getGEO("GSE226151", GSEMatrix = TRUE) # 1.2
meta <- pData(gse[[1]]) # 1.3
colnames(meta) # 1.4
[1] "title" "geo_accession"
[3] "status" "submission_date"
[5] "last_update_date" "type"
[7] "channel_count" "source_name_ch1"
[9] "organism_ch1" "characteristics_ch1"
[11] "characteristics_ch1.1" "characteristics_ch1.2"
[13] "characteristics_ch1.3" "characteristics_ch1.4"
[15] "growth_protocol_ch1" "molecule_ch1"
[17] "extract_protocol_ch1" "extract_protocol_ch1.1"
[19] "taxid_ch1" "data_processing"
[21] "data_processing.1" "data_processing.2"
[23] "data_processing.3" "data_processing.4"
[25] "data_processing.5" "platform_id"
[27] "contact_name" "contact_email"
[29] "contact_phone" "contact_laboratory"
[31] "contact_department" "contact_institute"
[33] "contact_address" "contact_city"
[35] "contact_zip/postal_code" "contact_country"
[37] "data_row_count" "instrument_model"
[39] "library_selection" "library_source"
[41] "library_strategy" "relation"
[43] "relation.1" "supplementary_file_1"
[45] "age:ch1" "bmi:ch1"
[47] "disease state:ch1" "Sex:ch1"
[49] "tissue:ch1"
head(meta[, c("age:ch1", "bmi:ch1", "disease state:ch1", "Sex:ch1", "tissue:ch1")]) # 1.5
age:ch1 bmi:ch1 disease state:ch1 Sex:ch1 tissue:ch1
GSM7065818 56 23.33 healthy aged female Skeletal muscle
GSM7065819 65 24.97 healthy aged female Skeletal muscle
GSM7065820 67 26.27 healthy aged female Skeletal muscle
GSM7065821 65 22.86 healthy aged female Skeletal muscle
GSM7065822 57 23.52 healthy aged female Skeletal muscle
GSM7065823 72 28.89 healthy aged female Skeletal muscle
nrow(meta) # 1.6
[1] 60
table(meta[, "disease state:ch1"]) # 1.6
healthy aged pre-sarcopenia sarcopenia
20 20 20
summary(as.numeric(meta[, "age:ch1"])) # 1.6
Min. 1st Qu. Median Mean 3rd Qu. Max.
53.00 65.00 69.50 69.13 74.00 88.00
summary(as.numeric(meta[, "bmi:ch1"])) # 1.6
Min. 1st Qu. Median Mean 3rd Qu. Max.
15.31 21.51 23.52 23.88 26.32 33.22
Interpretation.
<chr>), so they must be converted to numbers with
as.numeric() before modeling.What this does not tell me: I know which variable to use, but I still need the gene expression values for these 60 samples, matched to the same samples as the age values. Step 2 answers this.
Problem: I chose age as the continuous variable, but I still need the expression of every gene in the same 60 samples. Goal: download the gene expression table of GSE226151 and check that its samples match the samples of the metadata.
How:
library(data.table): loads fread(), a fast
function to read tables, also from the internetGEODataDownload() downloads internally)
url: the web address of the file; it contains the GSE
number of the datasetfread(url): downloads and reads the tableas.matrix(..., rownames = "GeneID"): converts it into a
matrix and uses the GeneID column as row names; each row is a gene, each
column is a sample, each value is the number of reads (counts) of that
gene in that sampledim(): returns the number of rows (genes) and columns
(samples)counts[1:5, 1:5]: shows the first 5 genes and the first
5 samplescolnames(counts): sample names of the count tablerownames(meta): sample names of the metadataall(... == ...): returns TRUE only if every name
matches in the same positionsetdiff(A, B): returns the names that are in A but not
in B (the samples without counts)intersect(A, B): returns the names that are in both A
and Bcounts[, common]: keeps only those columns (samples) of
the count tablemeta[common, ]: keeps only those rows (samples) of the
metadata, in the same orderall(... == ...): checks again that the names match one
by onelibrary(data.table) # 2.1
url <- paste0("https://www.ncbi.nlm.nih.gov/geo/download/?format=file&type=rnaseq_counts",
"&acc=GSE226151&file=GSE226151_raw_counts_GRCh38.p13_NCBI.tsv.gz") # 2.2
counts <- as.matrix(fread(url, header = TRUE), rownames = "GeneID") # 2.2
dim(counts) # 2.3
[1] 39376 58
counts[1:5, 1:5] # 2.4
GSM7065818 GSM7065819 GSM7065820 GSM7065821 GSM7065822
100287102 11 2 0 2 2
653635 1065 452 293 415 224
102466751 18 2 16 12 7
107985730 0 0 0 2 2
100302278 0 0 0 1 0
all(colnames(counts) == rownames(meta)) # 2.5
Warning in colnames(counts) == rownames(meta): longer object length is not a
multiple of shorter object length
[1] FALSE
setdiff(rownames(meta), colnames(counts)) # 2.6
[1] "GSM7065856" "GSM7065870"
common <- intersect(colnames(counts), rownames(meta)) # 2.6
counts <- counts[, common] # 2.6
meta <- meta[common, ] # 2.6
dim(counts) # 2.6
[1] 39376 58
all(colnames(counts) == rownames(meta)) # 2.6
[1] TRUE
Interpretation.
What this does not tell me: many of the 39,376 genes may have very few or zero reads, and age is still stored as text. Before fitting thousands of models, I need to prepare both. Step 3 answers this.
Problem: age is stored as text, and many of the 39,376 genes have almost no reads, so a model for them would carry no information. Goal: convert age into numbers and keep only the genes with enough reads to be analyzed.
How:
as.numeric(): converts text values like “56” into the
number 56summary(): checks the range and that there are no
missing valuesGEODataDownload())
counts >= 10: returns a TRUE/FALSE table, TRUE where
a gene has 10 or more reads in a samplerowSums(): counts, for each gene, how many samples are
TRUE>= 20: TRUE for the genes that reach the minimum in
20 or more samplessum(keep): counts how many genes pass the filtercounts[keep, ]: keeps the rows (genes) where
keep is TRUEdim(): checks the new size of the tableage <- as.numeric(meta[, "age:ch1"]) # 3.1
summary(age) # 3.1
Min. 1st Qu. Median Mean 3rd Qu. Max.
53.00 65.00 69.00 68.78 73.00 88.00
keep <- rowSums(counts >= 10) >= 20 # 3.2
sum(keep) # 3.2
[1] 16738
counts_f <- counts[keep, ] # 3.3
dim(counts_f) # 3.3
[1] 16738 58
Interpretation.
lm().What this does not tell me: the data are ready, but I still do not know which genes are associated with age. Step 4 fits one linear model per gene to answer this.
Problem: I have 16,738 genes and I do not know which ones are associated with age. Goal: fit a linear model for every gene, with age as the dependent variable and the gene’s expression as the independent variable, and keep the p-value of each model to measure how strong the association is.
How:
n_genes: the number of genes the loop will go
throughnumeric(n_genes): creates a vector of zeros with one
position per genenames(): labels each position with the gene ID, so I
know which p-value belongs to which genefor (i in 1:n_genes) { }: repeats the code inside the
braces once for each gene; i is the number of the current
genecounts_f[i, ]: the expression of gene i in
the 58 samples (row i of the table)lm(age ~ expression): fits the linear model, age
explained by the expression of the geneanova(fit)[1, "Pr(>F)"]: takes the p-value of the
gene effect from the ANOVA table (row 1, column “Pr(>F)”)head(): shows the first 6 valuessum(p_values < 0.05): counts how many genes are
significant at the 0.05 leveln_genes <- nrow(counts_f) # 4.1
p_values <- numeric(n_genes) # 4.2
names(p_values) <- rownames(counts_f)[1:n_genes] # 4.2
for (i in 1:n_genes) { # 4.3
expression <- counts_f[i, ] # 4.3
fit <- lm(age ~ expression) # 4.3
p_values[i] <- anova(fit)[1, "Pr(>F)"] # 4.3
}
head(p_values) # 4.4
653635 102466751 100996442 729737 102723897 112268260
0.5597498 0.8428116 0.7975149 0.4966472 0.8483896 0.7831087
sum(p_values < 0.05) # 4.4
[1] 691
Interpretation.
What this does not tell me: a single count of p-values below 0.05 does not show how the p-values are distributed across all genes. Step 5 plots this distribution.
Problem: the number of genes with p < 0.05 does not show the whole picture across 16,738 tests. Goal: plot the distribution of all the p-values and compare it with what is expected if no gene were associated with age (a flat distribution).
How:
hist(): counts how many p-values fall in each interval
and draws one bar per intervalbreaks = 20: splits the range 0–1 into 20 intervals of
width 0.05length(p_values) / 20: the number of genes divided by
the number of barsabline(h = ...): draws a horizontal line at that
height; col = "red" and lty = 2 make it a red
dashed linehist(p_values, breaks = 20, col = "grey80",
main = "Distribution of p-values (age ~ expression)",
xlab = "p-value") # 5.1
abline(h = length(p_values) / 20, col = "red", lty = 2) # 5.2
Interpretation.
What this does not tell me: the histogram describes all genes together, but not what the models of individual genes look like. Step 6 looks at the two extreme cases: the gene with the most and the gene with the least significant effect.
Problem: the histogram shows all genes together, but not what an individual model looks like. Goal: study in detail the two extreme cases, the gene with the most significant effect on age (smallest p-value) and the gene with the least significant effect (largest p-value), with their model summaries, ANOVA tables and diagnostic plots.
How:
which.min() / which.max(): return the
position of the smallest / largest p-valuenames(): returns the gene ID at that positionfread(): downloads the NCBI annotation table, which
links each GeneID with its symbol and descriptionannot[GeneID %in% ..., ]: keeps only the rows of the
two genescounts_f[gene, ]: the expression of that gene in the 58
sampleslm(age ~ expression): the same model used in the
loopsummary(): coefficients, p-values and R²anova(): the ANOVA table with the F test of the gene
effectplot(): draws the points; abline(fit):
draws the fitted straight linepar(mfrow = c(2, 2)): splits the window into 2 rows and
2 columnsplot(fit): draws the 4 standard diagnostic plots of a
linear modelbest_gene <- names(which.min(p_values)) # 6.1
worst_gene <- names(which.max(p_values)) # 6.1
p_values[c(best_gene, worst_gene)] # 6.1
7008 131474
0.0007595959 0.9999518246
library(org.Hs.eg.db) # 6.2
AnnotationDbi::select(org.Hs.eg.db, keys = c(best_gene, worst_gene),
columns = c("SYMBOL", "GENENAME"),
keytype = "ENTREZID") # 6.2
ENTREZID SYMBOL GENENAME
1 7008 TEF TEF transcription factor, PAR bZIP family member
2 131474 CHCHD4 coiled-coil-helix-coiled-coil-helix domain containing 4
best_expr <- counts_f[best_gene, ] # 6.3
worst_expr <- counts_f[worst_gene, ] # 6.3
fit_best <- lm(age ~ best_expr) # 6.3
fit_worst <- lm(age ~ worst_expr) # 6.3
summary(fit_best) # 6.4
Call:
lm(formula = age ~ best_expr)
Residuals:
Min 1Q Median 3Q Max
-15.2374 -5.0998 0.8196 4.8223 16.5361
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 76.293443 2.310805 33.016 < 2e-16 ***
best_expr -0.010146 0.002848 -3.562 0.00076 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 7.169 on 56 degrees of freedom
Multiple R-squared: 0.1847, Adjusted R-squared: 0.1702
F-statistic: 12.69 on 1 and 56 DF, p-value: 0.0007596
anova(fit_best) # 6.4
Analysis of Variance Table
Response: age
Df Sum Sq Mean Sq F value Pr(>F)
best_expr 1 652.12 652.12 12.689 0.0007596 ***
Residuals 56 2877.97 51.39
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
summary(fit_worst) # 6.4
Call:
lm(formula = age ~ worst_expr)
Residuals:
Min 1Q Median 3Q Max
-15.7760 -3.7759 0.2241 4.2241 19.2242
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 6.878e+01 4.769e+00 14.42 <2e-16 ***
worst_expr 1.121e-06 1.848e-02 0.00 1
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 7.94 on 56 degrees of freedom
Multiple R-squared: 6.568e-11, Adjusted R-squared: -0.01786
F-statistic: 3.678e-09 on 1 and 56 DF, p-value: 1
anova(fit_worst) # 6.4
Analysis of Variance Table
Response: age
Df Sum Sq Mean Sq F value Pr(>F)
worst_expr 1 0.0 0.000 0 1
Residuals 56 3530.1 63.037
par(mfrow = c(1, 2)) # 6.5
plot(best_expr, age, pch = 19, xlab = "Expression (raw counts)",
main = "Most significant gene") # 6.5
abline(fit_best, col = "red", lwd = 2) # 6.5
plot(worst_expr, age, pch = 19, xlab = "Expression (raw counts)",
main = "Least significant gene") # 6.5
abline(fit_worst, col = "red", lwd = 2) # 6.5
par(mfrow = c(2, 2)) # 6.6
plot(fit_best) # 6.6
plot(fit_worst) # 6.6
par(mfrow = c(1, 1)) # 6.6
Interpretation.
age ~ expression, so it describes age, not sarcopenia, and
it does not show that TEF protects against aging. It only shows that
older people in this dataset tend to express less TEF.Problem: in Question 2 I assumed a straight-line relationship between TEF expression and age, but the true relationship could be curved. Goal: fit two flexible, non-linear models (LOESS and GAM) to the same data and plot them together with the linear fit, to see whether they reveal a shape the straight line missed.
How:
loess(age ~ best_expr, span = 0.75): each local fit
uses the 75% of points closest to itlibrary(mgcv): loads the package for GAMs (it comes
with R)gam(age ~ s(best_expr), method = "REML"):
s() asks for a smooth function of expression; REML chooses
how much it can bendsummary(): shows the edf of the smooth term (edf ≈ 1
means a straight line; higher values mean curvature)order(best_expr): sorts the samples by expression, so
the curves are drawn from left to rightabline(fit_best): draws the linear fit from Question
2lines(): draws the LOESS and GAM fitted valueslegend(): identifies each fit by its colorfit_loess <- loess(age ~ best_expr, span = 0.75) # 1.1
library(mgcv) # 1.2
fit_gam <- gam(age ~ s(best_expr), method = "REML") # 1.2
summary(fit_gam) # 1.2
Family: gaussian
Link function: identity
Formula:
age ~ s(best_expr)
Parametric coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 68.7759 0.9413 73.06 <2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Approximate significance of smooth terms:
edf Ref.df F p-value
s(best_expr) 1 1 12.69 0.00076 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
R-sq.(adj) = 0.17 Deviance explained = 18.5%
-REML = 193.83 Scale est. = 51.392 n = 58
ord <- order(best_expr) # 1.3
plot(best_expr, age, pch = 19, col = "grey50",
xlab = "TEF expression (raw counts)", ylab = "Age",
main = "TEF: linear, LOESS and GAM fits") # 1.3
abline(fit_best, col = "red", lwd = 2, lty = 2) # 1.3
lines(best_expr[ord], fitted(fit_loess)[ord], col = "orange", lwd = 2) # 1.3
lines(best_expr[ord], fitted(fit_gam)[ord], col = "steelblue", lwd = 2) # 1.3
legend("topright", legend = c("Linear", "LOESS", "GAM"),
col = c("red", "orange", "steelblue"), lwd = 2, lty = c(2, 1, 1)) # 1.3
Interpretation.
What this does not tell me: the plot suggests the three fits describe the same trend, but I need a number that balances fit and complexity to decide which model is best. Step 2 answers this with AIC.
Problem: the plot suggests that the three fits are similar, but I need a number to decide which model is best. Goal: compare the linear model, LOESS and GAM with AIC, which rewards a good fit but penalizes complexity (lower AIC = better; a difference larger than 2 is usually considered meaningful).
How:
resid(): the residuals of each modelsum(... ^ 2): squares them and adds them upfit_loess$enp: the equivalent number of parameters of
LOESSsum(fit_gam$edf): the effective degrees of freedom of
the GAM (intercept + smooth term)AIC() does not work for LOESS
n * log(RSS / n) + 2 * (complexity + 1): the AIC for
models with normal errors; the +1 counts the residual variance. This
formula differs from R’s AIC() by a constant, so only the
differences between models matterdata.frame(): puts the results of the three models in
one tablen <- length(age) # 2.1
rss <- c(sum(resid(fit_best)^2),
sum(resid(fit_loess)^2),
sum(resid(fit_gam)^2)) # 2.1
complexity <- c(2, fit_loess$enp, sum(fit_gam$edf)) # 2.2
aic_table <- data.frame(model = c("Linear", "LOESS", "GAM"),
complexity = complexity,
RSS = rss,
AIC = n * log(rss / n) + 2 * (complexity + 1)) # 2.3
aic_table # 2.3
model complexity RSS AIC
1 Linear 2.000000 2877.965 232.4550
2 LOESS 4.838372 2725.068 234.9655
3 GAM 2.000023 2877.964 232.4550
Interpretation.
Problem: raw counts depend on the sequencing depth of each sample (how many reads the machine produced in total), which is a technical difference unrelated to age. A sample with twice as many reads shows about twice the counts for every gene. Goal: remove these technical differences with DESeq2 normalization, and put the data on a log scale, where every doubling counts as one unit.
How:
library(DESeq2): loads the packageDESeqDataSetFromMatrix(): combines the count matrix
(countData) and the sample information
(colData); row.names = colnames(counts_f)
links each age to its sample; design = ~ 1 because no group
comparison is needed hereestimateSizeFactors(): computes the size factorssizeFactors(): shows them (1 = typical depth; 2 = twice
as many reads)counts(dds, normalized = TRUE): returns the normalized
countslog2(... + 1): every doubling becomes +1; the +1 avoids
log2(0), which is −infinitylibrary(DESeq2) # 1.1
dds <- DESeqDataSetFromMatrix(countData = counts_f,
colData = data.frame(age = age, row.names = colnames(counts_f)),
design = ~ 1) # 1.1
dds <- estimateSizeFactors(dds) # 1.2
summary(sizeFactors(dds)) # 1.2
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.7409 0.9135 0.9804 1.0183 1.1164 1.6636
norm_counts <- counts(dds, normalized = TRUE) # 1.3
log_raw <- log2(counts_f + 1) # 1.4
log_norm <- log2(norm_counts + 1) # 1.4
Interpretation.
Problem: I need to see whether the samples really differed before normalization and whether normalization made them comparable. Goal: plot the distribution of the expression values of every sample before and after normalization.
How:
par(mfrow = c(1, 2)): two plots in one rowboxplot(): one box per column (sample); the thick line
inside each box is the medianoutline = FALSE: hides the extreme points, to see the
boxes betternames = FALSE: hides the 58 sample names, which would
not fitbarplot(): one bar per sample;
abline(h = 1) marks a typical depthpar(mfrow = c(1, 2)) # 2.1
boxplot(log_raw, outline = FALSE, names = FALSE, col = "grey80",
main = "Before normalization", ylab = "log2(counts + 1)") # 2.1
boxplot(log_norm, outline = FALSE, names = FALSE, col = "lightblue",
main = "After normalization", ylab = "log2(normalized counts + 1)") # 2.1
par(mfrow = c(1, 1)) # 2.1
barplot(sizeFactors(dds), names.arg = FALSE, col = "grey60",
main = "Size factor of each sample", ylab = "Size factor") # 2.2
abline(h = 1, col = "red", lty = 2) # 2.2
Interpretation.
Problem: the results of Question 2 were obtained with raw counts, which include technical noise. Goal: repeat exactly the same analysis (one linear model per gene, age ~ expression, p-value from ANOVA) on the normalized log2 data.
How:
numeric() and names(): as in Question
2log_norm[i, ]: the normalized expression of gene
i in the 58 sampleslm(age ~ expression) and
anova(fit)[1, "Pr(>F)"]: the same model and p-value as
in Question 2sum(p_values_norm < 0.05): number of genes with p
< 0.05 (raw data: 691; expected by chance: about 837)p_values_norm <- numeric(nrow(log_norm)) # 3.1
names(p_values_norm) <- rownames(log_norm) # 3.1
for (i in 1:nrow(log_norm)) { # 3.2
expression <- log_norm[i, ] # 3.2
fit <- lm(age ~ expression) # 3.2
p_values_norm[i] <- anova(fit)[1, "Pr(>F)"] # 3.2
}
sum(p_values_norm < 0.05) # 3.3
[1] 561
Interpretation.
Problem: a count of significant genes is not enough to decide which representation, raw or normalized, gives more trustworthy results. Goal: compare the distribution of p-values, the p-values of each gene, and the most significant gene in both analyses.
How:
hist() and abline(h = ...): as in Question
2-log10(): transforms p-values so that smaller p-values
become larger numbers (p = 0.01 → 2, p = 0.001 → 3)plot(): one point per gene; abline(0, 1)
is the line where both analyses agreewhich.min() and names(): the gene with the
smallest p-valueAnnotationDbi::select(): its symbol and namep_values_norm["7008"]: the p-value of TEF after
normalizationpar(mfrow = c(1, 2)) # 4.1
hist(p_values, breaks = 20, col = "grey80",
main = "Raw counts", xlab = "p-value") # 4.1
abline(h = length(p_values) / 20, col = "red", lty = 2) # 4.1
hist(p_values_norm, breaks = 20, col = "lightblue",
main = "Normalized log2", xlab = "p-value") # 4.1
abline(h = length(p_values_norm) / 20, col = "red", lty = 2) # 4.1
par(mfrow = c(1, 1)) # 4.1
plot(-log10(p_values), -log10(p_values_norm), pch = 20, col = "grey40",
xlab = "-log10(p) raw counts", ylab = "-log10(p) normalized",
main = "p-value of each gene: raw vs normalized") # 4.2
abline(0, 1, col = "red", lty = 2) # 4.2
best_gene_norm <- names(which.min(p_values_norm)) # 4.3
p_values_norm[best_gene_norm] # 4.3
100652739
0.001098593
AnnotationDbi::select(org.Hs.eg.db, keys = best_gene_norm,
columns = c("SYMBOL", "GENENAME"),
keytype = "ENTREZID") # 4.3
ENTREZID SYMBOL GENENAME
1 100652739 RAET1E-AS1 RAET1E antisense RNA 1
p_values_norm["7008"] # 4.3
7008
0.01284454
Interpretation.
Problem: the p-value of a linear model is only trustworthy if the model fits the data reasonably well. Goal: take the most significant gene from the analysis I judge most trustworthy (normalized data, Question 4), fit its model, and compare the ages it predicts with the real ages.
How:
log_norm[best_gene_norm, ]: the normalized expression
of the most significant gene of Question 4lm(age ~ final_expr): the linear modelpredict(): as in Question 1abline(0, 1): the line where predicted = observed; a
perfect model would put every point on itfinal_expr <- log_norm[best_gene_norm, ] # 1.1
fit_final <- lm(age ~ final_expr) # 1.1
summary(fit_final) # 1.1
Call:
lm(formula = age ~ final_expr)
Residuals:
Min 1Q Median 3Q Max
-15.036 -4.806 -1.331 5.021 17.206
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 50.517 5.388 9.375 4.49e-13 ***
final_expr 4.526 1.315 3.442 0.0011 **
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 7.213 on 56 degrees of freedom
Multiple R-squared: 0.1746, Adjusted R-squared: 0.1599
F-statistic: 11.85 on 1 and 56 DF, p-value: 0.001099
fitted_final <- predict(fit_final) # 1.2
plot(fitted_final, age, pch = 19, col = "grey40",
xlab = "Fitted age", ylab = "Observed age",
main = "Observed vs fitted age") # 1.3
abline(0, 1, col = "red", lty = 2) # 1.3
Interpretation.
Problem: the plot of observed vs fitted values shows the overall fit, but a linear model also assumes that its errors are random and normally distributed. Goal: obtain the residuals and look at their shape.
How:
resid(): as in Question 1plot(fitted, residuals) with
abline(h = 0): residuals should form a random cloud around
0hist(): the residuals should look like a bellqqnorm() and qqline(): if the residuals
are normal, the points follow the lineresid_final <- resid(fit_final) # 2.1
par(mfrow = c(1, 3)) # 2.2
plot(fitted_final, resid_final, pch = 19, col = "grey40",
xlab = "Fitted age", ylab = "Residual", main = "Residuals vs fitted") # 2.2
abline(h = 0, col = "red", lty = 2) # 2.2
hist(resid_final, breaks = 10, col = "grey80",
main = "Histogram of residuals", xlab = "Residual") # 2.2
qqnorm(resid_final, pch = 19, col = "grey40") # 2.2
qqline(resid_final, col = "red") # 2.2
par(mfrow = c(1, 1)) # 2.2
Interpretation.
Problem: judging normality by eye is subjective. Goal: test it formally with the Shapiro-Wilk test (H0: the residuals are normally distributed; p < 0.05 means they are not normal).
How:
shapiro.test(): returns the W statistic and the
p-valueshapiro.test(resid_final) # 3.1
Shapiro-Wilk normality test
data: resid_final
W = 0.9879, p-value = 0.8312
Interpretation.