Backtesting
Rob Hayward
Introduction
Backtesting is the way that you test whether strategies work. It is the answer to the question: if we did this in the past, what would have happened? It is the evidence that you provide. However, it is important to acknowledge the limitation of back-testing. All the things that we have said about the use of samples and regime-change apply. Therefore it is usual to continually update the test for evidence that the strategy is changing, evolving or ending.
There are two additional problems to be overcome:
Avoiding look-ahead bias: make sure that you do not use any information that comes from the future. You cannot use a signal generated by the closing price to trading at the closing price. If you are implementing the module, what information is available at the time that you make the decision?
Avoiding over-fitting: if the model is too complicated or has too many parameters it is likely that you fit the noise. In that case you will not forecast well out-of-sample. In that case, you need to make sure that the system works on new data. In machine learningt is common to split the data into a training set and an test set so that the model can be created using the training data and then assessed with the test data.
Backtesting
Here are two simple technical strategies that are performed on BAC data as an example. The moving average strategy will try to capture the trends that are found in the evolution of secutities and asset prices; the rate-of-change strategy will try to capture the overshooting and reversals. There is a common method that can be adapted. You will think of new strategies for the assignment.
The steps to be taken are:
- calculate the technical indicator
- generate the signal
- turn the signal into a trading strategy
- assess the performance of the trading strategy
Moving average
This will try to determine trends by looking for times when the price passes through a moving average of the price. Alternatively, it could be a short moving average moves through a longer moving average.
First take a look at the BAC price and the indicator.
# install key packages
library(quantmod)
library(PerformanceAnalytics)
# Getting stock prices of BAC
getSymbols('BAC')## [1] "BAC"BAC <- BAC['2012/']
sma20_BAC <- SMA(BAC$BAC.Close, n = 20)
sma50_BAC <- SMA(BAC$BAC.Close, n = 50)
barChart(BAC['2024/'], theme = 'white')
addSMA(n = 20, col = 'blue')
addSMA(n = 50, col = 'red')
legend(inset = 0.02, 'bottomright', legend = c('BAC', 'MA20', 'MA50'),
col = c('black', 'blue', 'red'),
lty = c(1,1,1), cex = 0.7)
Create the signal
This will take advantage of two key functions:
Lag()which will lag the variable by the number of periods that are identified.ifelse()which will test a logical outcome and return one value if it is true and another one if it is false.
For moving average cross-over we need to know whether the price is above the moving average AND was below the moving average in the last period.
ifelse(logical test, outcome if positive, outcome if negative)Here we combine the two functions and nest an ifelse
within an ifelse. If there is no movement in the price from
below to above the moving average, there is another ifelse
to test whether there is a movement above to below.
# SMA 20 Crossover Signal
sma20_BAC_ts <- Lag(
ifelse(Cl(BAC) > sma20_BAC & Lag(Cl(BAC)) < Lag(sma20_BAC),1,
ifelse(Cl(BAC) < sma20_BAC & Lag(Cl(BAC)) > Lag(sma20_BAC),-1,0)))
# Turn any nas into 0 as we don't have a signal in that case
sma20_BAC_ts[is.na(sma20_BAC_ts)] <- 0Turn signal into position
Create the trading position from the signal. In this case we use a standard, buy when there is a positive cross-over of the closing price through the 20-day moving average and sell when there is a negative cross-over of the closing price through the 20-day moving average. Maintain current position if there is no cross-over.
In this case we make use of the for() statement. The way
that this works is that an indicator (i in this example) counts over the
range that has been specified. In this case the range is 1 to the length
of BAC.Close. In other words, it is the length of the series that we
have, it will run through every row. This is identified as
for(i in 1:length(Cl(BAC)) {
<action to be carried out while i sequences>
}the action to be carried out while i counts from 1 to the length of
Cl(BAC) is placed within the curly brackets. Work out what that is. It
is another ifelse statement.
# Create a vector to hold the position.
sma20_BAC_pos <- rep(1, length(Cl(BAC)))
for (i in 2 : length(Cl(BAC))) {
sma20_BAC_pos[i] <- ifelse(sma20_BAC_ts[i] == 1, 1,
ifelse(sma20_BAC_ts[i] == -1,0,sma20_BAC_pos[i-1]))
}
# Turn NAs into 1 to make the default position long
# This could be changed to be flat or short
sma20_BAC_pos[is.na(sma20_BAC_pos)] <- 1Then combine all the components into a strategy object.
sma20_BAC_strat <- cbind(Cl(BAC), sma20_BAC, sma20_BAC_ts, sma20_BAC_pos)
colnames(sma20_BAC_strat) <- c('Price', 'SMA(20)','SMA SIGNAL','SMA POSITION')
tail(sma20_BAC_strat, n = 20)## Price SMA(20) SMA SIGNAL SMA POSITION
## 2026-03-27 46.97 48.1160 -1 0
## 2026-03-30 47.23 47.9870 0 0
## 2026-03-31 48.75 47.9260 0 0
## 2026-04-01 49.27 47.8745 1 1
## 2026-04-02 49.38 47.8530 0 1
## 2026-04-06 50.06 47.9240 0 1
## 2026-04-07 50.28 48.0430 0 1
## 2026-04-08 51.88 48.2090 0 1
## 2026-04-09 52.71 48.4185 0 1
## 2026-04-10 52.54 48.6890 0 1
## 2026-04-13 53.35 49.0205 0 1
## 2026-04-14 53.35 49.3350 0 1
## 2026-04-15 54.32 49.6870 0 1
## 2026-04-16 53.51 50.0210 0 1
## 2026-04-17 53.91 50.3660 0 1
## 2026-04-20 53.95 50.7055 0 1
## 2026-04-21 53.48 51.0035 0 1
## 2026-04-22 53.12 51.2525 0 1
## 2026-04-23 52.47 51.4385 0 1
## 2026-04-24 52.05 51.6290 0 1Rate of change
Rate of change will measure the speed at which prices are changing to determine the momentum and the risk that a trend is about to change. In this case we look at the change over the last 25 days (or about one month).
First take a look at the BAC price and the indicator.
roc_BAC <- ROC(Cl(BAC), n = 25)
barChart(BAC['2024/'], theme = 'white')
addROC(n = 25)
addTA(xts(rep(-0.1, nrow(BAC)), index(BAC)),
col = 'black', lty = 2, lwd = 1, on = 3,
legend.loc = NULL)
addTA(xts(rep(0.1, nrow(BAC)), index(BAC)),
col = 'black', lty = 2, lwd = 1, on = 3,
legend.loc = NULL)
A Divergence between prices and momentum is an indicator that a bubble may be ready to burst. It shows that prices are moving higher but at a reduced pace. Are investors becoming less confident? Are they more cautious? Will it take less information now to cause a reversal?
What signal does it give?
Signal
In this case we look for an extreme movement and generate the signal based on the ROC moving above 0.05 or below -0.05. An alternative would be a case where there is a new price high but a lower rate of return indicator.
roc_BAC_ts <- Lag(
ifelse(roc_BAC < -0.1 & Lag(roc_BAC) > -0.1,1,
ifelse(roc_BAC > 0.1 & Lag(roc_BAC) < 0.1,-1,0)))
# In this case defaul is a flat position (turn NA into 0)
roc_BAC_ts[is.na(roc_BAC_ts)] <- 0Turn the signal into a trading strategy. When the price moves above 0.1 you will sell and when the price moves below -0.1 you buy. Otherwise, you maintain your position.
roc_BAC_pos <- rep(1, length(Cl(BAC)))
for (i in 2:length(Cl(BAC))) {
roc_BAC_pos[i] <- ifelse(roc_BAC_ts[i] == 1, 1,
ifelse(roc_BAC_ts[i] == -1,
0,roc_BAC_pos[i-1]))
}
roc_BAC_pos[is.na(roc_BAC_pos)] <- 1
roc_BAC_strat <- cbind(roc_BAC, roc_BAC_ts, roc_BAC_pos)
colnames(roc_BAC_strat) <- c('ROC(25)','ROC SIGNAL','ROC POSITION')
tail(roc_BAC_strat, 25)## ROC(25) ROC SIGNAL ROC POSITION
## 2026-03-20 -0.107647986 0 1
## 2026-03-23 -0.100614398 0 1
## 2026-03-24 -0.091260793 0 1
## 2026-03-25 -0.090356216 0 1
## 2026-03-26 -0.089754269 0 1
## 2026-03-27 -0.121914249 0 1
## 2026-03-30 -0.078167960 1 1
## 2026-03-31 -0.033484368 0 1
## 2026-04-01 -0.047948927 0 1
## 2026-04-02 -0.057450851 0 1
## 2026-04-06 0.004605064 0 1
## 2026-04-07 0.009391565 0 1
## 2026-04-08 0.037510530 0 1
## 2026-04-09 0.046800112 0 1
## 2026-04-10 0.053359007 0 1
## 2026-04-13 0.092427724 0 1
## 2026-04-14 0.107758413 0 1
## 2026-04-15 0.112092303 -1 0
## 2026-04-16 0.097892427 0 0
## 2026-04-17 0.134406223 0 0
## 2026-04-20 0.143885341 -1 0
## 2026-04-21 0.127884331 0 0
## 2026-04-22 0.116466138 0 0
## 2026-04-23 0.113717539 0 0
## 2026-04-24 0.101844471 0 0Assess performance
To assess the performance it is necessary to calculate the returns and find a benchmark for comparison. We will compare these two strategies relative to a buy and hold of the underlying stock (BAC). The strategy needs to perform better than the benchmark in order to make the strategy worthwhile.
Costs
As part of the comparison it is necessary to make an assumption for costs of trading. These costs will include the bid-ask spread, any commission and slippage. Slippage is the tendency of the price to slip away from larger orders: market-makers will make a spread for a specific size and you would anticipate that it will be wider for anything larger; market-orders may require limit orders away from the best-bid-offer. Both the bid-ask spread and slippage will therefore tend to increase with order size. Transaction costs will also be different in different markets and at different times: higher in more marginal and illiquid markets and at times of market stress.
You can see in the assessment below that there is an estimate of 0.001 percent for transaction costs. This can be adjusted according to type of fund, market or conditions. A more sophisticated approach to costs can be adopted.
ret_BAC <- diff(log(Cl(BAC)))
benchmark_BAC <- ret_BACMoving average
To assess the moving average strategy
# set the cost per trade
cost <- 0.001
sma_BAC_ret <- ret_BAC*sma20_BAC_pos
sma_BAC_ret_commission_adj <- ifelse((sma20_BAC_ts == 1|sma20_BAC_ts == -1)
& sma20_BAC_pos != Lag(sma20_BAC_pos),
(ret_BAC - cost) * sma20_BAC_pos,
ret_BAC * sma20_BAC_pos)
sma_BAC_comp <- cbind(sma_BAC_ret, sma_BAC_ret_commission_adj, benchmark_BAC)
colnames(sma_BAC_comp) <- c('SMA','SMA Commission Adj','BAC Benchmark')
charts.PerformanceSummary(sma_BAC_comp, main = 'Bank of America Performance')
sma_BAC_comp_table <- table.AnnualizedReturns(sma_BAC_comp)
sma_BAC_comp_table## SMA SMA Commission Adj BAC Benchmark
## Annualized Return 0.1233 0.1072 0.1138
## Annualized Std Dev 0.2000 0.2000 0.3027
## Annualized Sharpe (Rf=0%) 0.6164 0.5362 0.3761Rate of Change
# Cost has been set above
roc_BAC_ret <- ret_BAC*roc_BAC_pos
roc_BAC_ret_commission_adj <- ifelse((roc_BAC_ts == 1 | roc_BAC_ts == -1) &
roc_BAC_pos != Lag(roc_BAC_pos),
(ret_BAC - cost) * roc_BAC_pos,
ret_BAC * roc_BAC_pos)
roc_BAC_comp <- cbind(roc_BAC_ret, roc_BAC_ret_commission_adj, benchmark_BAC)
colnames(roc_BAC_comp) <- c('ROC','ROC Commission Adj','BoA Benchmark')
charts.PerformanceSummary(roc_BAC_comp, main = 'BoA ROC Performance',
legend.loc = 'topleft')
roc_BAC_comp_table <- table.AnnualizedReturns(roc_BAC_comp)
roc_BAC_comp_table## ROC ROC Commission Adj BoA Benchmark
## Annualized Return 0.0807 0.0790 0.1138
## Annualized Std Dev 0.2118 0.2119 0.3027
## Annualized Sharpe (Rf=0%) 0.3810 0.3729 0.3761Additional elements
The following additional elements could be contemplated:
A more complete model to assess costs with adjustments for fund size, market and market conditions
Consideration of stop-loss and take-profits. The moving average strategy will only go long or flat and has a natural take-profit and stop-loss action. However, this could be added as an extra signal. For example, in pseudo-code: if long and this loss happens..do…
Do you want to allow double positions and short positions? Some market signals may trigger more than once and some will suggest short-positions. Things to consider: larger positions may require additional capital; short-positions may incur higher transaction costs.