Workshop 2, Financial Modeling and Programming

The market model at scale: alpha, beta and stock selection

Abstract

This is an INDIVIDUAL workshop. You already know the OLS regression and the market model from Financial Econometrics. Here we use them as a tool for stock selection: we decide which returns to use (simple excess returns), which market proxy and which risk-free rate; we program a function that estimates the model for one stock and a loop that applies it to many stocks; we test whether a stock is significantly riskier than the market; and we learn why finding ‘significant alphas’ in many stocks can be pure luck.

1 General directions

Same rules as Workshop 1: work in Google Colab with a notebook named W2-FinProg-YourFirstName-YourLastName, follow Predict → Run → Investigate → Modify → Make, write your pseudocode before using an LLM, verify your results, and submit the .ipynb file in Canvas. Grade: notebook 60% + Canvas Quiz W2 40%.

import os, io, requests
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import statsmodels.formula.api as smf
from scipy import stats

def import_csv(file, base_url="https://www.apradie.com/datos/", **kwargs):
    """Imports a csv file from base_url; downloads it only once and keeps a local copy."""
    if not os.path.exists(file):
        headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
                                 "(KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"}
        response = requests.get(base_url + file, headers=headers)
        response.raise_for_status()
        with open(file, "wb") as f:
            f.write(response.content)
    return pd.read_csv(file, **kwargs)

2 The market model and the CAPM: a quick review

In Financial Econometrics you estimated the market model, a simple regression of the stock return on the market return:

R_{i,t}=b_0+b_1R_{M,t}+\varepsilon_{i,t}

The CAPM says something more specific: the premium (excess) return of a stock over the risk-free rate is proportional to the premium return of the market:

E[R_i]-R_f=\beta_i\left(E[R_M]-R_f\right)

To test it, we run the regression with excess returns:

R_{i,t}-R_{f,t}=\alpha_i+\beta_i\left(R_{M,t}-R_{f,t}\right)+\varepsilon_{i,t}

• \beta_i measures the market (systematic) risk of the stock: by how much the stock’s excess return moves, on average, when the market’s excess return moves 1 percentage point.
• \alpha_i (Jensen’s alpha) is the average excess return that is not explained by the market. The CAPM says it should be 0. A positive and significant alpha means the stock paid more than what its market risk justifies.

Why excess returns? With raw returns, the intercept is b_0=\alpha_i+R_f(1-\beta_i): it mixes alpha with the risk-free rate. When interest rates are high (5% in 2023–2024), a stock with \beta<1 gets a “free” positive intercept that has nothing to do with skill or mispricing. With excess returns, the intercept is alpha.

2.1 Which returns: simple or continuously compounded?

In the Algorithms course we used this rule: cc returns add up over time (for holding-period returns, annualizing, volatility), and simple returns add up across assets (for portfolios). Where does the CAPM fit?

The CAPM is a statement about portfolios: the market is a weighted average of stocks, and the alpha and beta of a portfolio are the weighted averages of the alphas and betas of its stocks only with simple returns. Then, for the CAPM and the market model we use simple excess returns.

Does it matter? Let’s run an experiment. We simulate 120,000 months of returns (a long simulation, so that sampling noise does not hide the result) for two stocks whose true alpha is exactly zero: a very volatile stock (β = 2, idiosyncratic volatility of 15% per month, like Tesla) and a defensive stock (β = 0.5, 4%):

rng = np.random.default_rng(seed=42)
T = 120_000                                       # number of simulated months
mkt = rng.normal(0.008, 0.045, T)                 # simple market (excess) returns
results = []
for name, beta, sd_e in [("volatile (beta=2)", 2.0, 0.15), ("defensive (beta=0.5)", 0.5, 0.04)]:
    stock = 0.0 + beta * mkt + rng.normal(0, sd_e, T)        # TRUE alpha = 0
    sim = pd.DataFrame({"stock": stock, "mkt": mkt})
    simple = smf.ols("stock ~ mkt", data=sim).fit()
    logs = smf.ols("stock ~ mkt", data=np.log(1 + sim)).fit()
    results.append([name, 12 * simple.params["Intercept"], simple.params["mkt"],
                    12 * logs.params["Intercept"], logs.params["mkt"]])
pd.DataFrame(results, columns=["stock", "annual alpha (simple)", "beta (simple)",
                               "annual alpha (log)", "beta (log)"]).round(3)

	stock	annual alpha (simple)	beta (simple)	annual alpha (log)	beta (log)
0	volatile (beta=2)	0.007	1.999	-0.160	2.045
1	defensive (beta=0.5)	-0.000	0.501	-0.007	0.502

The betas are almost the same, but with log returns the alpha of the volatile stock is strongly negative even though its true alpha is zero. The reason: \ln(1+R)\approx R-\frac{R^2}{2}, so the log transformation subtracts approximately half of the variance, and volatile stocks lose more. Since in this course we rank stocks by alpha, using log returns would systematically punish volatile stocks.

2.2 Which market and which risk-free rate?

• Market: we use the adjusted price of SPY (the S&P 500 ETF). The adjusted price includes the dividends, like the adjusted prices of the stocks. The index ^GSPC is a price index without dividends: it understates the market return by about 1.5%–2% per year, and then all the alphas look higher than they are.
• Risk-free rate: the 3-month US Treasury Bill rate (FRED series TB3MS), published as an annual percentage. The monthly rate is R_f = \frac{TB3MS}{100\times12}.

3 The data

Two files prepared for this course from Yahoo Finance and FRED:

• fz2024_prices_monthly.csv: month-end adjusted prices of the S&P 500 firms of our catalog (same tickers as in W1; for example BRK_B), SPY and three ETFs (GLD: gold, TLT: long-term Treasury bonds, AGG: aggregate bonds).
• fz2024_rf_monthly.csv: tb3ms (annual %) and rf (monthly risk-free rate in decimals).

prices = import_csv("fz2024_prices_monthly.csv", index_col="date", parse_dates=True)
rf = import_csv("fz2024_rf_monthly.csv", index_col="date", parse_dates=True)
print(prices.shape, prices.index.min().date(), "to", prices.index.max().date())
rf.tail(3)

(142, 466) 2014-12-31 to 2026-09-30

	tb3ms	rf	source
date
2026-07-31	3.72	0.003100	Yahoo ^IRX
2026-08-31	3.71	0.003092	Yahoo ^IRX
2026-09-30	3.93	0.003275	Yahoo ^IRX

We calculate simple returns, and excess returns by subtracting the risk-free rate of the same month from every column. sub(..., axis=0) subtracts a column (the rf series) from every column of the data frame, aligning by date. We first reindex the rf series to the months of R: otherwise pandas aligns the union of both indexes and adds rows for the months that exist only in rf (a silent error!):

R = prices.pct_change(fill_method=None).iloc[1:]           # simple monthly returns
rf_m = rf["rf"].reindex(R.index)                           # same months as R
assert rf_m.notna().all(), "Missing risk-free rates"
R_ex = R.sub(rf_m, axis=0)                                 # excess returns
assert R_ex.index.equals(R.index)
R_ex[["AAPL", "SPY"]].tail(3)

	AAPL	SPY
date
2026-07-31	0.064463	-0.002752
2026-08-31	0.023496	0.023708
2026-09-30	0.047759	-0.006568

4 One stock: Apple

We estimate the model with the 36 months before our decision date of W1 (March 31, 2023): April 2020 to March 2023.

START, END = "2020-04-30", "2023-03-31"
win = R_ex.loc[START:END]
reg_data = pd.DataFrame({"stock": win["AAPL"], "mkt": win["SPY"]}).dropna()
aapl = smf.ols("stock ~ mkt", data=reg_data).fit()
print(aapl.summary().tables[1])

==============================================================================
                 coef    std err          t      P>|t|      [0.025      0.975]
------------------------------------------------------------------------------
Intercept      0.0109      0.010      1.058      0.298      -0.010       0.032
mkt            1.3417      0.181      7.419      0.000       0.974       1.709
==============================================================================

We extract the results by name (not by position), which is safer and works with any version of pandas:

alpha, beta = aapl.params["Intercept"], aapl.params["mkt"]
se_alpha, se_beta = aapl.bse["Intercept"], aapl.bse["mkt"]
ci = aapl.conf_int()                          # rows: Intercept, mkt; columns: 0 (lower), 1 (upper)
print(f"alpha = {alpha:.4f} per month ({12 * alpha:.2%} per year), p-value = {aapl.pvalues['Intercept']:.3f}")
print(f"beta  = {beta:.3f}, 95% CI = [{ci.loc['mkt', 0]:.3f}, {ci.loc['mkt', 1]:.3f}]")
print(f"R-squared = {aapl.rsquared:.3f}; n = {int(aapl.nobs)} months")

alpha = 0.0109 per month (13.04% per year), p-value = 0.298
beta  = 1.342, 95% CI = [0.974, 1.709]
R-squared = 0.618; n = 36 months

ImportantThree interpretations that students often get wrong

1. The p-value of beta in the output tests H_0:\beta=0, that is, “the stock is not related to the market”. Almost every stock rejects it. To know whether a stock is riskier than the market, we must test H_0:\beta=1: t=\frac{b_1-1}{SE(b_1)}
2. The 95% confidence interval is not “the values that beta will take in the future with 95% probability”. It is a range built with a method that, in 95% of the samples, contains the true (unknown) beta. In practice: the values inside the interval are values of beta that our data cannot reject at the 5% level.
3. Annualizing: a monthly alpha is annualized by multiplying by 12. A beta is not annualized: it is a ratio of returns of the same frequency.

The test of \beta=1 by hand and with statsmodels:

t_beta1 = (beta - 1) / se_beta
p_beta1 = 2 * stats.t.sf(abs(t_beta1), df=aapl.df_resid)       # two-tailed p-value
print(f"t = {t_beta1:.3f}, p-value = {p_beta1:.4f}")
print(aapl.t_test("mkt = 1"))

t = 1.889, p-value = 0.0674
                             Test for Constraints                             
==============================================================================
                 coef    std err          t      P>|t|      [0.025      0.975]
------------------------------------------------------------------------------
c0             1.3417      0.181      1.889      0.067       0.974       1.709
==============================================================================

4.1 Optional: OLS with matrix algebra

In the Algorithms course you used @ for portfolios and gradient descent for the logistic model. OLS has a closed-form solution:

\hat{b}=(X^{T}X)^{-1}X^{T}y

where X has a column of ones (for the intercept) and a column with the market excess returns:

X = np.column_stack([np.ones(len(reg_data)), reg_data["mkt"]])
y = reg_data["stock"].values
b_hat = np.linalg.inv(X.T @ X) @ X.T @ y
assert np.allclose(b_hat, aapl.params.values)
print(b_hat)

[0.01086929 1.3417063 ]

5 CHALLENGE 1: A function for the CAPM

Write a function capm(ticker, R_ex, start, end, market="SPY", min_obs=24) that:

1. selects the excess returns of ticker and of the market between start and end, and drops missing values;
2. returns None if there are fewer than min_obs months (a recent IPO, for example);
3. runs the OLS regression and returns a pandas Series with: alpha, se_alpha, p_alpha, beta, se_beta, beta_low, beta_high (95% CI), t_beta1, p_beta1 (test \beta=1), r2 and n.

Write the pseudocode first. Test your function with Apple and verify with an assert that the alpha and beta are equal to the ones we estimated above.

# Write your pseudocode in a text cell, then your code here

6 CHALLENGE 2: The market model for the 20 stocks of the W1 screen

These are the 20 S&P 500 firms with the highest ROA on March 31, 2023 (Workshop 1, Challenge 2):

w1_tickers = ["TXN", "VRSN", "MRNA", "NVDA", "KLAC", "QCOM", "STLD", "NVR", "NUE", "LRCX",
              "DVN", "ODFL", "AAPL", "IDXX", "MPWR", "CF", "MA", "MTD", "BLDR", "FAST"]

1. Write a loop that applies capm to each ticker with the window April 2020 – March 2023, and stores the results in a data frame capm_table with one row per ticker (hint: collect the Series in a dictionary and use pd.DataFrame(dict).T). Skip the tickers for which capm returns None and print their names.
2. Add a column alpha_annual = 12 × alpha. Sort the table by alpha_annual from highest to lowest.
3. Interpret with your own words the model of the stock with the highest alpha: alpha (value, annual value, significance), beta (value and the test of β = 1), and R-squared.
4. How many stocks have a positive and significant alpha at the 5% level? How many are significantly riskier than the market (β > 1 with p_beta1 < 0.05)? How many are significantly less risky (β < 1 with p_beta1 < 0.05)?
5. Selection (preview of the Evidence): select the 10 stocks with the highest alpha, and from them, the 5 with the lowest beta. Show the 5 tickers with their alpha and beta.

# Write your code here

7 CHALLENGE 3: Significant alphas by luck (multiple testing)

A significance level of 5% means that, when the true alpha is zero, we will wrongly find a significant alpha 5% of the time. If you test many stocks, some “significant” alphas will appear by pure chance.

1. Using rng = np.random.default_rng(seed=42), choose 100 random tickers of the S&P 500 firms in prices (exclude SPY, GLD, TLT and AGG): rng.choice(stock_cols, size=100, replace=False), where stock_cols is the sorted list of stock columns.
2. Run capm for each of them with the same window. How many have a significant alpha (positive or negative, p < 0.05)? How many would you expect if all the true alphas were zero?
3. Does alpha persist? For the 20 stocks of Challenge 2, estimate the model again with the next 12 months (April 2023 – March 2024, min_obs=12). Calculate the correlation between the alphas of both windows and make a scatter plot. Does a high alpha in the past predict a high alpha in the future?
4. Explain with your own words why selecting stocks by their past alpha can be dangerous, and why the Evidence uses alpha only as one filter among several.

# Write your code here

8 CHALLENGE 4: Simple vs log returns with real data

1. Estimate the CAPM for the 20 stocks of Challenge 2 using log excess returns: \ln(1+R_{i,t})-\ln(1+R_{f,t}) for the stock and the same for SPY.
2. Create a table with the annual alpha and the beta of both versions, and the difference of the alphas (log − simple).
3. Calculate the residual variance of each stock in the simple-return model, \sigma^2_\varepsilon (m.resid.var()), and make a scatter plot of the alpha difference (monthly) against \sigma^2_\varepsilon. Add the line y=-\frac{1}{2}x. What do you see? Would the ranking of Challenge 2 change?

# Write your code here

9 CHALLENGE 5: Bug hunt

Each piece of code runs, but the result is wrong. Find the bug, explain it and fix it.

5.1 A student wrote this pseudocode and then the code: “run a regression where Y is the market return and X is the stock return”:

m = smf.ols("mkt ~ stock", data=d).fit()
beta = m.params["stock"]

5.2 Excess returns:

R_ex = R.sub(rf["tb3ms"], axis=0)

5.3 A stock is “significantly riskier than the market”:

riskier = capm_table[(capm_table["beta"] > 1) & (capm_table["p_beta"] < 0.05)]   # p_beta from the summary

5.4 Annual alpha and annual beta:

capm_table["alpha_annual"] = capm_table["alpha"] * np.sqrt(12)
capm_table["beta_annual"] = capm_table["beta"] * 12

5.5 Market returns:

data = yf.download(["AAPL", "^GSPC"], interval="1mo", auto_adjust=True)["Close"]

Write your answers here

10 DataCamp

Take the chapter “Factor Investing” (Chapter 3 of the course Introduction to Portfolio Risk Management in Python), where you estimate the CAPM and multi-factor models with statsmodels.

11 W2 submission

Submit your .ipynb file in Canvas and answer the Canvas Quiz W2 before the first class of next week.