Backtesting
Rob Hayward
Introduction
Backtesting is the way that you test whether strategies work. You are giving an answer to the question: if we did this in the past, what would have happened? It is the evidence that you provide.
There are two big problems that need to be overcome:
Avoiding look-ahead bias: make sure that you do not use ay information that comes from the future. You cannot use a signal generated by the closing price to trading at the closing price. It would be useful to try to model in real time. What signal does it give me now?
Avoiding over-fitting: if the model is too complicated or has too many parameters it is likely that you fit the noise. In that case you will not forecast well out-of-sample. In that case, you need to make sure that the system works on new data. It is common to split the data into a training set and an test set so that the model can be created using the training data and then assessed with the test data.
Backtesting strategies
Here are a number of simple strategies performed on BAC data. You will see a common method that will help you translate these into other strategies. Ideally, you will think of new strategies for the assignment.
The steps to be taken are:
- calculate the technical indicator
- generate the signal
- turn the signal into a trading strategy
- assess the performance of the trading strategy
Moving average
library(quantmod)
library(PerformanceAnalytics)
# Getting stock prices of BAC
getSymbols('BAC')## [1] "BAC"sma20_BAC <- SMA(BAC$BAC.Close, n = 20)
sma50_BAC <- SMA(BAC$BAC.Close, n = 50)
barChart(BAC['2024'], theme = 'white')
addSMA(n = 20, col = 'blue')
addSMA(n = 50, col = 'red')
legend(inset = 0.02, 'bottomright', legend = c('BAC', 'MA20', 'MA50'),
col = c('black', 'blue', 'red'),
lty = c(1,1,1), cex = 0.7)
Exercise 1.0
Create 20 and 50 day moving averages for the security of your choice.
Identify the key levels that would suggest a change in current trend
What is your strategy?
Create the signal
This will take advantage of two key functions:
Lag()which will lag the variable by the number of periods that are identified.ifelse()which will test a logical outcome and return one value if it is true and another one if it is false.
For moving average cross-over we need to know whether the price is above the moving average AND was below the moving average in the last period.
ifelse(logical test, outcome if positive, outcome if negative)Here we combine the two functions and nest an ifelse
within an ifelse. If there is no movement in the price for
below to above the moving average, there is another ifelse
to test whether there is a movement above to below.
# SMA 20 Crossover Signal
sma20_BAC_ts <- Lag(
ifelse(Lag(Cl(BAC)) < Lag(sma20_BAC) & Cl(BAC) > sma20_BAC,1,
ifelse(Lag(Cl(BAC)) > Lag(sma20_BAC) & Cl(BAC) < sma20_BAC,-1,0)))
# Turn any nas into 0 as we don't have a signal in that case
sma20_BAC_ts[is.na(sma20_BAC_ts)] <- 0
# SMA 50 Crossover Signal
sma50_BAC_ts <- Lag(
ifelse(Lag(Cl(BAC)) < Lag(sma50_BAC) & Cl(BAC) > sma50_BAC,1,
ifelse(Lag(Cl(BAC)) > Lag(sma50_BAC) & Cl(BAC) < sma50_BAC,-1,0)))
sma50_BAC_ts[is.na(sma50_BAC_ts)] <- 0
# SMA 20 and SMA 50 Crossover Signal
sma_BAC_ts <- Lag(
ifelse(Lag(sma20_BAC) < Lag(sma50_BAC) & sma20_BAC > sma50_BAC,1,
ifelse(Lag(sma20_BAC) > Lag(sma50_BAC) & sma20_BAC < sma50_BAC,-1,0)))
sma_BAC_ts[is.na(sma_BAC_ts)] <- 0Create the trading strategy from the signal. In this case we use a standard, buy when there is a positive cross-over of the (short) 20-day moving average through the (long) 50-day moving average and sell when there is a negative cross-over.
The for statement
In this case we make use of the for() statement. The way
that this works is that an indicator (i in this case) counts over the
range that has been specified. In this case the range is 1 to the length
of BAC.Close. In other words, it is the length of the series that we
have. This is identified as
for(i in 1:length(Cl(BAC)) {
<action to be carried out while i sequences>
}the action to be carried out while i counts from 1 to the length of
Cl(BAC) is placed within the curly brackets. Work out what that is. It
is another ifelse statement. Then combine all the
components into a strategy object.
sma_BAC_strat <- ifelse(sma_BAC_ts > 1,0,1)
for (i in 1 : length(Cl(BAC))) {
sma_BAC_strat[i] <- ifelse(sma_BAC_ts[i] == 1,1,ifelse(sma_BAC_ts[i] == -1,0,sma_BAC_strat[i-1]))
}
sma_BAC_strat[is.na(sma_BAC_strat)] <- 1
sma_BAC_stratcomp <- cbind(sma20_BAC, sma50_BAC, sma_BAC_ts, sma_BAC_strat)
colnames(sma_BAC_stratcomp) <- c('SMA(20)','SMA(50)','SMA SIGNAL','SMA POSITION')
tail(sma_BAC_stratcomp, n = 20)## SMA(20) SMA(50) SMA SIGNAL SMA POSITION
## 2024-03-01 33.6395 33.4622 0 1
## 2024-03-04 33.7235 33.4950 0 1
## 2024-03-05 33.8435 33.5432 0 1
## 2024-03-06 33.9625 33.5876 0 1
## 2024-03-07 34.0850 33.6316 0 1
## 2024-03-08 34.2090 33.6664 0 1
## 2024-03-11 34.3500 33.7074 0 1
## 2024-03-12 34.4670 33.7490 0 1
## 2024-03-13 34.6335 33.7972 0 1
## 2024-03-14 34.7615 33.8330 0 1
## 2024-03-15 34.8285 33.8706 0 1
## 2024-03-18 34.9245 33.9148 0 1
## 2024-03-19 35.0280 33.9468 0 1
## 2024-03-20 35.1805 33.9986 0 1
## 2024-03-21 35.3755 34.0762 0 1
## 2024-03-22 35.5320 34.1452 0 1
## 2024-03-25 35.6945 34.2194 0 1
## 2024-03-26 35.8350 34.3052 0 1
## 2024-03-27 36.0100 34.4190 0 1
## 2024-03-28 36.1800 34.5414 0 1Rate of change
As we have seen before, rate of change will measure the rate at which prices are changing to try to determine the momentum and the risk that a trend is about to change. We look at the change over the last 25 days (or about one month).
roc_BAC <- ROC(BAC$BAC.Close, n = 25)
barChart(BAC['2024'], theme = 'white')
addROC(n = 25)
legend('left', col = 'red', legend = 'ROC(25)', lty = 1, bty = 'n',
text.col = 'white', cex = 0.8)
A Divergence between prices and momentum is an indicator that a bubble may be ready to burst.
Exercise 2.0
Calculate the NCO for your stock.
What signal does it give?
Signal
Now generate the signal that is based on the ROC moving above or below 0.05.
roc_BAC_ts <- Lag(
ifelse(Lag(roc_BAC) < (-0.05) & roc_BAC > (-0.05),1,
ifelse(Lag(roc_BAC) < (0.05) & roc_BAC > (0.05),-1,0)))
roc_BAC_ts[is.na(roc_BAC_ts)] <- 0Turn the signal into a trading strategy. When the price moves above 0.05 you will sell and when the price moves below -0.05 you buy.
roc_BAC_strat <- ifelse(roc_BAC_ts > 1,0,1)
for (i in 1 : length(Cl(BAC))) {
roc_BAC_strat[i] <- ifelse(roc_BAC_ts[i] == 1,1,ifelse(roc_BAC_ts[i] == -1,0,roc_BAC_strat[i-1]))
}
roc_BAC_strat[is.na(roc_BAC_strat)] <- 1
roc_BAC_stratcomp <- cbind(roc_BAC, roc_BAC_ts, roc_BAC_strat)
colnames(roc_BAC_stratcomp) <- c('ROC(25)','ROC SIGNAL','ROC POSITION')
tail(roc_BAC_stratcomp)## ROC(25) ROC SIGNAL ROC POSITION
## 2024-03-21 0.12416827 0 0
## 2024-03-22 0.08385111 0 0
## 2024-03-25 0.07812288 0 0
## 2024-03-26 0.08816406 0 0
## 2024-03-27 0.11507579 0 0
## 2024-03-28 0.12065497 0 0Take profit and stop loss
The easiest way to set up take-profit or stop-loss would be to identify the recent highs and lows. You would have to decide over which time period to calculate the stop.
Exercise 3.0
- Calculate the rolling stops and take-profit for your stock.
Parabolic stop and reverse strategy
Parabolic stop and reverse strategy is a dynamic system to find places to take-profit and top-loss. This is a moving point that is adjusted to current price movement by an Acceleration Factor (AF). The Extreme Point EP) is the highest since going long and the lowest point since going short. The SAR is based on the formula:
\[\text{Current SAR} = \text{Prior SAR} + AF * (\text{Prior EP} - \text{Prior SAR})\]
You can find out more about the parabolic stop and reverse strategy here
The following signal is based on the typical price and the default parameters for the accelerator (0.02 and 0.2)
sar_BAC <- SAR(cbind(Hi(BAC),Lo(BAC)), accel = c(0.02, 0.2))
barChart(BAC['2024'], theme = 'white')
addSAR(accel = c(0.02, 0.2))
Now generate the signal
sar_BAC_ts <- Lag(
ifelse(Lag(Cl(BAC)) < Lag(sar_BAC) & Cl(BAC) > sar_BAC,1,
ifelse(Lag(Cl(BAC)) > Lag(sar_BAC) & Cl(BAC) < sar_BAC,-1,0)))
sar_BAC_ts[is.na(sar_BAC_ts)] <- 0and generate trading strategy from signal
sar_BAC_strat <- ifelse(sar_BAC_ts > 1,0,1)
for (i in 1 : length(Cl(BAC))) {
sar_BAC_strat[i] <- ifelse(sar_BAC_ts[i] == 1,1,ifelse(sar_BAC_ts[i] == -1,0,sar_BAC_strat[i-1]))
}
sar_BAC_strat[is.na(sar_BAC_strat)] <- 1
sar_BAC_stratcomp <- cbind(Cl(BAC), sar_BAC, sar_BAC_ts, sar_BAC_strat)
colnames(sar_BAC_stratcomp) <- c('Close','SAR','SAR SIGNAL','SAR POSITION')
tail(sar_BAC_stratcomp)## Close SAR SAR SIGNAL SAR POSITION
## 2024-03-21 37.51 35.73000 0 1
## 2024-03-22 37.05 36.17880 0 1
## 2024-03-25 36.86 36.55091 0 1
## 2024-03-26 37.09 36.80000 0 1
## 2024-03-27 37.81 36.80000 0 1
## 2024-03-28 37.92 37.09400 0 1Commodity channel index
The Commodity Channel Index (CCI) seeks to identify trends by comparing the current price with an average of the typical price. It will usually range between 100 and -100 so that any movement outside this range will generate a signal. Here the we have 20 days as the time period and 0.015 as the constant value.
The formula is:
\[CCI = \frac{TP - ATP}{0.015 * MD}\]
where TP is the typical price, ATP is the average typical price over 20 days and MD is the mean deviation of the typical price. This is calculated as the absolute value of the difference between the typical price and the ATP in the last period divided by the average.
cci_BAC <- CCI(HLC(BAC), n = 20, c = 0.015)
barChart(BAC['2023/2024'], theme = 'white')
addCCI(n = 20, c = 0.015)
Now generate the signal.
cci_BAC_ts <- Lag(
ifelse(Lag(cci_BAC) < (-100) & cci_BAC > (-100),1,
ifelse(Lag(cci_BAC) < (100) & cci_BAC > (100),-1,0)))
cci_BAC_ts[is.na(cci_BAC_ts)] <- 0Now create the trading strategy from the signal. This is based on selling when the figure is above 100 and buying when it is below -100. This is based on the belief that this signals an extreme that is overdone and ready for a correction.
cci_BAC_strat <- ifelse(cci_BAC_ts > 1,0,1)
for (i in 1 : length(Cl(BAC))) {
cci_BAC_strat[i] <- ifelse(cci_BAC_ts[i] == 1,1,ifelse(cci_BAC_ts[i] == -1,0,cci_BAC_strat[i-1]))
}
cci_BAC_strat[is.na(cci_BAC_strat)] <- 1
cci_BAC_stratcomp <- cbind(cci_BAC, cci_BAC_ts, cci_BAC_strat)
colnames(cci_BAC_stratcomp) <- c('CCI','CCI SIGNAL','CCI POSITION')
tail(cci_BAC_stratcomp, n = 8)## CCI CCI SIGNAL CCI POSITION
## 2024-03-19 89.2084 0 0
## 2024-03-20 111.4310 0 0
## 2024-03-21 175.2633 -1 0
## 2024-03-22 155.6880 0 0
## 2024-03-25 127.5844 0 0
## 2024-03-26 121.2451 0 0
## 2024-03-27 155.4533 0 0
## 2024-03-28 146.2453 0 0Assess performance
To assess the performance it is necessary to calculate the returns and find a benchmark for comparison
ret_BAC <- diff(log(Cl(BAC)))
benchmark_BAC <- ret_BACMoving average
To assess the moving average strategy
sma_BAC_ret <- ret_BAC*sma_BAC_strat
sma_BAC_ret_commission_adj <- ifelse((sma_BAC_ts == 1|sma_BAC_ts == -1) & sma_BAC_strat != Lag(sma_BAC_ts), (ret_BAC-0.01)*sma_BAC_strat, ret_BAC*sma_BAC_strat)
sma_BAC_comp <- cbind(sma_BAC_ret, sma_BAC_ret_commission_adj, benchmark_BAC)
colnames(sma_BAC_comp) <- c('SMA','SMA Commission Adj','BAC Benchmark')
charts.PerformanceSummary(sma_BAC_comp, main = 'Bank of America Performance')
sma_BAC_comp_table <- table.AnnualizedReturns(sma_BAC_comp)
sma_BAC_comp_table## SMA SMA Commission Adj BAC Benchmark
## Annualized Return 0.0054 -0.0196 -0.1343
## Annualized Std Dev 0.2987 0.2994 0.4940
## Annualized Sharpe (Rf=0%) 0.0181 -0.0656 -0.2720Rate of Change
roc_BAC_ret <- ret_BAC*roc_BAC_strat
roc_BAC_ret_commission_adj <- ifelse((roc_BAC_ts == 1|roc_BAC_ts == -1) & roc_BAC_strat != Lag(roc_BAC_ts), (ret_BAC-0.01)*roc_BAC_strat, ret_BAC*roc_BAC_strat)
roc_BAC_comp <- cbind(roc_BAC_ret, roc_BAC_ret_commission_adj, benchmark_BAC)
colnames(roc_BAC_comp) <- c('ROC','ROC Commission Adj','BoA Benchmark')
charts.PerformanceSummary(roc_BAC_comp, main = 'BoA ROC Performance',
legend.loc = 'topright')
roc_BAC_comp_table <- table.AnnualizedReturns(roc_BAC_comp)
roc_BAC_comp_table## ROC ROC Commission Adj BoA Benchmark
## Annualized Return -0.1150 -0.1901 -0.1343
## Annualized Std Dev 0.3788 0.3795 0.4940
## Annualized Sharpe (Rf=0%) -0.3036 -0.5009 -0.2720Parabolic SAR
sar_BAC_ret <- ret_BAC*sar_BAC_strat
sar_BAC_ret_commission_adj <- ifelse((sar_BAC_ts == 1|sar_BAC_ts == -1) & sar_BAC_strat != Lag(sar_BAC_ts), (ret_BAC-0.01)*sar_BAC_strat, ret_BAC*sar_BAC_strat)
sar_BAC_comp <- cbind(sar_BAC_ret, sar_BAC_ret_commission_adj, benchmark_BAC)
colnames(sar_BAC_comp) <- c('SAR','SAR Commission Adj','BoA Benchmark')
charts.PerformanceSummary(sar_BAC_comp, main = 'BoA Parabolic SAR Performance', legend.loc = 'topright')
sar_BAC_comp_table <- table.AnnualizedReturns(sar_BAC_comp)
sar_BAC_comp_table## SAR SAR Commission Adj BoA Benchmark
## Annualized Return -0.0822 -0.1714 -0.1343
## Annualized Std Dev 0.3155 0.3167 0.4940
## Annualized Sharpe (Rf=0%) -0.2606 -0.5411 -0.2720Commodity Channel index
cci_BAC_ret <- ret_BAC*cci_BAC_strat
cci_BAC_ret_commission_adj <- ifelse((cci_BAC_ts == 1|cci_BAC_ts == -1) & cci_BAC_strat != Lag(cci_BAC_ts), (ret_BAC-0.01)*cci_BAC_strat, ret_BAC*cci_BAC_strat)
cci_BAC_comp <- cbind(cci_BAC_ret, cci_BAC_ret_commission_adj, benchmark_BAC)
colnames(cci_BAC_comp) <- c('CCI','CCI Commission Adj','BoA Benchmark')
charts.PerformanceSummary(cci_BAC_comp, main = 'BoA CCI Performance',
legend.loc = 'topright')
cci_BAC_comp_table <- table.AnnualizedReturns(cci_BAC_comp)
cci_BAC_comp_table## CCI CCI Commission Adj BoA Benchmark
## Annualized Return -0.1147 -0.2158 -0.1343
## Annualized Std Dev 0.3680 0.3686 0.4940
## Annualized Sharpe (Rf=0%) -0.3117 -0.5855 -0.2720Exercise 4.0
Apply one of these strategies to your stock
Assess the performance, can you adjust the strategy to improve?
Split the data into a training and test set. Adjust parameters of by using the training set of data and, when you are happy, try the new strategy on the testing set of data.
Comment on the performance