Equity Backtesting

Simple RSI Backtest

Project Description

The Relative Strength Index (RSI) is a momentum indicator that compares the magnitude of recent gains and losses over a specified time period to measure the speed and change of price movements of a security. It is primarily used in an attempt to identify overbought or oversold conditions in the trading of an asset.

Because the RSI provides a relative evaluation of the strength of an asset’s recent price performance, it can be used to directly compare different assets. The RSI value ranges from 0 to 100. Traditionally, an RSI value of 70 or above indicates that a stock is reaching an overbought condition and may see a trend reversal, or pullback, in price. Conversely, an RSI value of 30 or less indicates a stock is reaching oversold conditions and may see a trend reversal, or bounceback, towards a higher share price.

The default time period for comparing up periods and down periods is 14 days. However, different trading strategies will adjust the time period depending on whether the strategy is seeking short term gains (day traders) or long term gains (growth investors).

The RSI is commonly used in conjunction with trend lines to identify support or resistance of the RSI values.

The RSI can create false buy or sell signals when there are sudden large price movements. One method utilized to avoid false RSI signals is to use more extreme thresholds to indicate overbought and oversold conditions. So it’s not unusual to see RSI thresholds at 80 for overbought conditions and 20 for oversold conditions.

This project will be a simple test for oversold stocks with a threshold of 30 and a comparison time period of 14 days. A 3 month return on the stock will be calculated along with an optimal return which identifies the highest return possible within the 3 month period.

The source data will be the backtest environment created in the previous project, Building the Backtest Environment. Recall that the environment contains historical pricing data from 2007-2016. We also have a symbols table that contains all of the stocks that were successfully loaded in the environment.

As before, we will be using the quantmod package to retrieve data from the backtest environment and perform the RSI generation.

Libraries Required

library(quantmod)   # Quantitative financial strategies
library(dplyr)      # Data manipulation
library(lubridate)  # Date and time processing
library(knitr)      # Dynamic report generation

Working directory

setwd("U:/Equity Backtesting")

Prepare the Environment

We will require the two files previously created in order to perform the RSI backtest. To access the environment we created we simply need to load the environment file. The symbols table is in the form of an .rds.

Load Backtest Environment

load("Environments/bt_env_2007_2016.Rdata")

Read Backtest Symbols Table

bt_env_symbols <- readRDS("Symbols/bt_env_symbols.rds")

We can check to make sure that the data is loaded correctly and contains expected values. This time we can look for symbols that start with “TS”.

Verify Backtest Environment

ls(bt_env_2007_2016, pattern = "^TS")
##  [1] "TS"   "TSBK" "TSC"  "TSCO" "TSE"  "TSEM" "TSLA" "TSM"  "TSN"  "TSNU"
## [11] "TSO"  "TSQ"  "TSRI" "TSRO" "TSS"  "TST"  "TSU"

Verify Data in Backtest Environment

kable(head(bt_env_2007_2016$TSLA, 10))
TSLA.Open TSLA.High TSLA.Low TSLA.Close TSLA.Volume TSLA.Adjusted
19.00 25.00 17.54 23.89 18766300 23.89
25.79 30.42 23.30 23.83 17187100 23.83
25.00 25.92 20.27 21.96 8218800 21.96
23.00 23.10 18.71 19.20 5139800 19.20
20.00 20.00 15.83 16.11 6866900 16.11
16.40 16.63 14.98 15.80 6921700 15.80
16.14 17.52 15.57 17.46 7711400 17.46
17.58 17.90 16.55 17.40 4050600 17.40
17.95 18.07 17.00 17.05 2202500 17.05
17.39 18.64 16.90 18.14 2680100 18.14

Verify Backtest Symbols

kable(head(bt_env_symbols, 10))
Symbol Sector Industry Name Exchange
AAAP Health Care Major Pharmaceuticals Advanced Accelerator Applications S.A. NASDAQ
AAL Transportation Air Freight/Delivery Services American Airlines Group, Inc. NASDAQ
AAME Finance Life Insurance Atlantic American Corporation NASDAQ
AAOI Technology Semiconductors Applied Optoelectronics, Inc. NASDAQ
AAON Capital Goods Industrial Machinery/Components AAON, Inc. NASDAQ
AAPC Consumer Services Services-Misc. Amusement & Recreation Atlantic Alliance Partnership Corp. NASDAQ
AAPL Technology Computer Manufacturing Apple Inc. NASDAQ
AAWW Transportation Transportation Services Atlas Air Worldwide Holdings NASDAQ
ABAC Consumer Non-Durables Farming/Seeds/Milling Aoxin Tianli Group, Inc. NASDAQ
ABAX Capital Goods Industrial Machinery/Components ABAXIS, Inc. NASDAQ 
## [1] "Number of Backtest Symbols: 4667"

Process for RSI

Since this is a backtest project, we want to define the criteria for the data we are going to be retrieving. When using quantmod in a normal interactive mode, the symbol and historical data to be evaluated will be retrieved with the getSymbol() function. The symbol will be charted and the addRSI() function is called to add the RSI indicator to the chart. In this scenario, we want the RSI to be calculated and then stored for further use.

To accomplish this we will use the RSI() function call to perform the required calculations. The default parameters are a comparison length of 14 days and an Exponential Moving Average (EMA) type. We will be using the defaults for the test.

The next item to define is the value of the RSI that we are interested in. Since an RSI value of 30 is considered to be the signal for oversold conditions, we will be looking for RSI values that are equal to or less than 30. We will then compare returns at different levels of the RSI to see if there are differences in the ratio of postive to negative returns and the average returns realized.

The first step in the processing is to retrieve the desired symbol data from the backtest environment. This is accomplished using the following:

get(symbol_name, source_environment)

The second step is to approximate, or substitute, any NA values in the time series. This is needed when a time series has a missing set of values for a given date. If the data is not adjusted, the RSI call will fail and we lose that stock as a potential test object.

Next, we invoke the RSI command and specify which field to perform the test on. The suggested field to use is adjusted closing price.

We do need an error check for the RSI call for the situations in which there isn’t enough data to perform the calculation. So we will use the function try() to validate the call and test whether the call inherits an error.

If the call is successful the results will be placed in a data frame and then processed. The processing consists of the following:

The data is grouped and filtered to keep only one occurrence of the RSI threshold in order to test the RSI as a trigger for initiating a buy condition for the stock for a given month. The data frame created is then joined to a master data frame which will contain all occurrences of the RSI threshold being met for the first time in a given month for all of the stocks.

Create Symbol Array

bt_symbols <- bt_env_symbols$Symbol

RSI Gather Loop

j <- 1

for(i in 1:length(bt_symbols)) {
        
        sym_sum  <- get(bt_symbols[i], envir = bt_env_2007_2016)
        
        sym_sum  = na.approx(sym_sum)
        
        get_try  <- try(RSI(Ad(sym_sum), 14))
        
        if(inherits(get_try, "try-error")) {
                
        } 
        
        else {
         
              rsi_sum <- data.frame(RSI(Ad(sym_sum), 14))
              
              rsi_sum <- rsi_sum %>%
                         mutate(Date = rownames(.)) %>%
                         transform(EMA = round(EMA, 2)) %>%
                         filter(complete.cases(.),
                                EMA <= 30) %>%
                         mutate(Symbol = bt_symbols[i],
                                Year = year(Date),
                                Mnth = month(Date)) %>%
                         group_by(Year, Mnth) %>%
                         slice(1L) %>%
                         ungroup() %>%
                         rename(RSI = EMA) %>%
                         select(Symbol, Date, RSI)
  
              if(j == 1) { rsi_frame <- rsi_sum } else
                         { rsi_frame <- bind_rows(rsi_frame, rsi_sum) }
              
              j <- j + 1
              
              }
}
Symbol Date RSI
AAAP 2016-11-25 27.89
AAAP 2016-12-01 17.06
AAL 2007-03-05 29.84
AAL 2007-04-02 29.43
AAL 2007-05-01 23.05
AAL 2007-06-07 24.99
AAL 2007-08-15 27.58
AAL 2007-11-08 28.94
AAL 2007-12-12 28.15
AAL 2008-01-02 26.80 
## [1] "Symbols meeting threshold: 4508"

Process for Returns

The goal for the backtest is to determine if an RSI trigger, indicating a buy signal, is a potential predictor for positive returns based on the expectation of a bounceback in stock price after an oversold condition. For this example, we will look at a 3 month return from the date of the trigger.

Using the backtest environment again, we will retrieve the adjusted closing prices beginning with the date of the trigger and ending 3 months after. If the trigger date is such that the calculated end data is greater than 12/30/2016, the last trading day of 2016 and the last date in our backtest environment, the recorded returns will be NA.

During the 3 month testing window, it can be expected that the stock price will have volatility and that the starting and ending prices for the window may not reflect the low and high stock prices observed. To determine the potential for returns, we will also record the highest stock price during the window. It is possible that a 3 month return may show a negative value but there is an opportunity for a positive return within the window.

The rsi_frame will be updated with the following:

Add Fields to rsi_frame

rsi_frame <- rsi_frame %>%
             mutate(Trigger_Cl = 0,
                    Period_Cl  = 0,
                    Return_3mt = 0,
                    Period_Hi  = 0,
                    Return_Hi  = 0,
                    Volume_Min = 0,
                    Volume_Avg = 0)
Symbol Date RSI Trigger_Cl Period_Cl Return_3mt Period_Hi Return_Hi Volume_Min Volume_Avg
AAAP 2016-11-25 27.89 0 0 0 0 0 0 0
AAAP 2016-12-01 17.06 0 0 0 0 0 0 0
AAL 2007-03-05 29.84 0 0 0 0 0 0 0
AAL 2007-04-02 29.43 0 0 0 0 0 0 0
AAL 2007-05-01 23.05 0 0 0 0 0 0 0

Generating the returns requires a processing loop to retrieve the symbol and date of the RSI trigger. A target end date is generated for the 3 month window. Each month, on average, contains 4.33 weeks. So a 3 month testing window will consist of 13 weeks.

If the calculated end date falls outside the range of the backtest environment, we simply set a value of NA to the desired fields. If the date range is within the backtest environment range, we can process the record and retrieve/calculate the statistics. We will be using the adjusted close field for all of the data points.

Generate Returns

for(i in 1:nrow(rsi_frame)) {
        
        Symbol   <- rsi_frame$Symbol[i]
        Beg_Date <- as.Date(rsi_frame$Date[i])
        End_Date <- Beg_Date + dweeks(13)
        
        if(End_Date > "2016-12-31") {
                
                rsi_frame$Trigger_Cl[i]     <- NA
                rsi_frame$Period_Cl[i]      <- NA
                rsi_frame$Return_3mt[i]     <- NA
                rsi_frame$Period_Hi[i]      <- NA
                rsi_frame$Return_Hi[i]      <- NA
                rsi_frame$Volume_Min[i]     <- NA
                rsi_frame$Volume_Avg[i]     <- NA
                
        } 
        else {
                
                sym_sum    <- get(Symbol, envir = bt_env_2007_2016)
                
                date_range <- paste(as.character(Beg_Date),"/",as.character(End_Date), sep = "")
                
                sym_sum    <- data.frame(sym_sum[date_range])
                
                colnames(sym_sum) <- c("Open", "High", "Low", "Close", "Volume", "AdjClose")
                
                rsi_frame$Trigger_Cl[i] <- as.numeric(sym_sum$AdjClose[1])
                rsi_frame$Period_Cl[i]  <- as.numeric(sym_sum$AdjClose[nrow(sym_sum)])
                
                rsi_frame$Return_3mt[i] <- round((rsi_frame$Period_Cl[i]  - rsi_frame$Trigger_Cl[i])
                                                 / rsi_frame$Trigger_Cl[i], 3)
                
                rsi_frame$Period_Hi[i]  <- max(sym_sum$AdjClose)
                
                rsi_frame$Return_Hi[i]  <- round((rsi_frame$Period_Hi[i]  - rsi_frame$Trigger_Cl[i])
                                                 / rsi_frame$Trigger_Cl[i], 3)
                
                rsi_frame$Volume_Min[i] <- min(sym_sum$Volume)
                rsi_frame$Volume_Avg[i] <- round(mean(sym_sum$Volume),0)

        }
}
Symbol Date RSI Trigger_Cl Period_Cl Return_3mt Period_Hi Return_Hi Volume_Min Volume_Avg
AAAP 2016-11-25 27.89 NA NA NA NA NA NA NA
AAAP 2016-12-01 17.06 NA NA NA NA NA NA NA
AAL 2007-03-05 29.84 50.09 33.95 -0.322 50.23 0.003 1036200 2680567
AAL 2007-04-02 29.43 45.05 30.70 -0.319 47.87 0.063 1036200 3054338
AAL 2007-05-01 23.05 37.06 31.01 -0.163 37.06 0.000 1036200 3085808
AAL 2007-06-07 24.99 30.20 31.14 0.031 36.00 0.192 796400 2685427
AAL 2007-08-15 27.58 25.59 23.48 -0.082 32.56 0.272 796400 2521314
AAL 2007-11-08 28.94 22.04 15.40 -0.301 23.48 0.065 885000 3909548
AAL 2007-12-12 28.15 15.31 9.02 -0.411 15.98 0.044 885000 3542613
AAL 2008-01-02 26.80 13.32 9.17 -0.312 15.40 0.156 1235700 3650943

Once the data has been retrieved and the desired information stored in the data frame we can filter to remove records that do not meet our acceptance criteria. We will define the rejection criteria as the following:

The purpose of the acceptance criteria is to eliminate irregularly traded stocks which might otherwise skew the results.

Clean RSI frame

rsi_final <- rsi_frame %>%
             filter(complete.cases(.),
                    Trigger_Cl >= 5,
                    Trigger_Cl < 1000,
                    Volume_Min > 50000,
                    Volume_Avg >= 100000)
Symbol Date RSI Trigger_Cl Period_Cl Return_3mt Period_Hi Return_Hi Volume_Min Volume_Avg
AAL 2007-03-05 29.84 50.09 33.95 -0.322 50.23 0.003 1036200 2680567
AAL 2007-04-02 29.43 45.05 30.70 -0.319 47.87 0.063 1036200 3054338
AAL 2007-05-01 23.05 37.06 31.01 -0.163 37.06 0.000 1036200 3085808
AAL 2007-06-07 24.99 30.20 31.14 0.031 36.00 0.192 796400 2685427
AAL 2007-08-15 27.58 25.59 23.48 -0.082 32.56 0.272 796400 2521314
AAL 2007-11-08 28.94 22.04 15.40 -0.301 23.48 0.065 885000 3909548
AAL 2007-12-12 28.15 15.31 9.02 -0.411 15.98 0.044 885000 3542613
AAL 2008-01-02 26.80 13.32 9.17 -0.312 15.40 0.156 1235700 3650943
AAL 2008-03-10 29.20 10.29 3.92 -0.619 10.29 0.000 1386600 4533277
AAL 2008-04-22 29.09 6.84 4.27 -0.376 9.35 0.367 2646700 8324533 
## [1] "Total Remaining Records: 38905"

Review Results

The first thing we will review is the overall 3 month totals based on the adjusted closing prices from the beginning and end of the test window.

3 Month Summary Statistics

pos_ret_3mt <- nrow(rsi_final %>%
                    filter(Return_3mt > 0))

neg_ret_3mt <- nrow(rsi_final %>%
                    filter(Return_3mt <= 0))

pos_pct_3mt <- round(pos_ret_3mt / nrow(rsi_final), 3)

avg_ret_3mt <- round(mean(rsi_final$Return_3mt), 3)

avg_pos_ret <- round(mean(rsi_final$Return_3mt[rsi_final$Return_3mt > 0]), 3)
avg_neg_ret <- round(mean(rsi_final$Return_3mt[rsi_final$Return_3mt <= 0]), 3)
## [1] "           Total Records: 38905"
## [1] "  Total Positive Returns: 21345"
## [1] "  Total Negative Returns: 17560"
## [1] "Percent Positive Returns: 0.549"
## [1] "          Average Return: 0.023"
## [1] " Average Positive Return: 0.174"
## [1] " Average Negative Return: -0.161"

We see 54.9% of the stocks ended the 3 month testing window with a positive return. The average return, when factoring all stocks, is 2.3%. When looking at just the stocks with positive returns, the average is 17.4%.

Based on these statistics, using the RSI as a trigger for positive returns appears to be lacking on its own. However, the rate of return for stocks that do have a positive return associated with them leaves open the possibility of using the RSI in another fashion or in conjunction with a second trigger.

Now we can look at whether there was a potential for higher returns by looking at the period high stock price observed within the testing period instead of using the testing window closing price.

3 Month Period Statistics

pos_ret_per <- nrow(rsi_final %>%
                    filter(Return_Hi > 0))

neg_ret_per <- nrow(rsi_final %>%
                    filter(Return_Hi <= 0))

pos_pct_per <- round(pos_ret_per / nrow(rsi_final), 3)

avg_ret_per <- round(mean(rsi_final$Return_Hi[rsi_final$Return_Hi > 0]), 3)
## [1] "  Total Positive Returns: 36148"
## [1] "  Total Negative Returns: 2757"
## [1] "Percent Positive Returns: 0.929"
## [1] " Average Positive Return: 0.176"

The results indicate there is a larger opportunity for positive returns within the testing window than what is illustrated by looking at the closing price of the testing window. 92.9% of stocks had an opportunity for a positive return within the testing period. The average return is 17.6%, which is slightly better than the return for stocks with a positive return using the testing window closing price.

These statistics reveal that there is a bounceback in share price following an oversold condition for a large portion of the stocks. The bounceback may be short lived, however. This means that having an appropriate exit strategy, sell condition, is critical to maximizing investment returns.

Of course, using the average return potentially creates a false expectation of returns because large outliers can skew the statistical result. So we can look at the positive returns broken out by percentages of occurence. This helps identify a potential threshold for returns within the 3 month window that could be used as an exit point for the stock. That is, if we employ a strategy that sought a 5% return, or bounceback, from the entry point of the stock, triggered by the RSI, what would be the percentage of investments that would yield the desired return?

Generate Quantiles for Positive Returns

quantile(rsi_final$Return_Hi[rsi_final$Return_Hi > 0], prob = seq(0, 1, length = 11), type = 5)
##    0%   10%   20%   30%   40%   50%   60%   70%   80%   90%  100% 
## 0.001 0.026 0.049 0.073 0.098 0.126 0.159 0.199 0.261 0.374 3.786

Close to 80% of the observations in our testing window generated returns of 5% or more. And close to 60% generated returns of 10% or more.

Let’s take a look at the breakdown of potential returns by RSI. We can set three levels:

Potential Returns by RSI

rsi_ct_10 <- nrow(rsi_final %>%
                  filter(RSI <= 10))

rsi_ct_20 <- nrow(rsi_final %>%
                  filter(RSI > 10,
                         RSI <= 20))

rsi_ct_30 <- nrow(rsi_final %>%
                  filter(RSI > 20))

rsi_po_10 <- nrow(rsi_final %>%
                  filter(RSI <= 10,
                         Return_Hi > 0))

rsi_po_20 <- nrow(rsi_final %>%
                  filter(RSI > 10,
                         RSI <= 20,
                         Return_Hi > 0))

rsi_po_30 <- nrow(rsi_final %>%
                  filter(RSI > 20,
                         Return_Hi > 0))

rsi_pt_10 <- round(rsi_po_10 / rsi_ct_10, 3)
rsi_pt_20 <- round(rsi_po_20 / rsi_ct_20, 3)
rsi_pt_30 <- round(rsi_po_30 / rsi_ct_30, 3)

rsi_re_10 <- round(mean(rsi_final$Return_Hi[rsi_final$RSI <= 10 &
                                            rsi_final$Return_Hi > 0]), 3)

rsi_re_20 <- round(mean(rsi_final$Return_Hi[rsi_final$RSI  > 10 &
                                            rsi_final$RSI <= 20 &
                                            rsi_final$Return_Hi > 0]), 3)

rsi_re_30 <- round(mean(rsi_final$Return_Hi[rsi_final$RSI  > 20 &
                                            rsi_final$Return_Hi > 0]), 3)
## [1] "      Total Count - RSI 10: 38"
## [1] "      Total Count - RSI 20: 986"
## [1] "      Total Count - RSI 30: 37881"
## [1] "% Positive Return - RSI 10: 0.974"
## [1] "% Positive Return - RSI 20: 0.922"
## [1] "% Positive Return - RSI 30: 0.929"
## [1] "   Average Return - RSI 10: 0.202"
## [1] "   Average Return - RSI 20: 0.197"
## [1] "   Average Return - RSI 30: 0.176"

The majority of RSI triggers are in the 20-30 range and results in a potential return percentage of 17.6%, along with 92.9% of the stocks having positive earnings opportunities within the 3 month testing window. The RSI level between 20 and 30 has a success rate with 92.2% with a potential return of 19.7%. The RSI level of 10 and below has the highest success rate with 97.4% and a potential return of 20.2%.

The trade off is that there were only 38 stocks that met a trigger qualification with an RSI of 10 or below during the 10 year window.

Let’s look at the positive returns by RSI level broken out by percentages of occurence, as we did before.

Generate Quantiles for Positive Returns by RSI Level 10

quantile(rsi_final$Return_Hi[rsi_final$Return_Hi > 0 &
                             rsi_final$RSI <= 10], prob = seq(0, 1, length = 11), type = 5)
##     0%    10%    20%    30%    40%    50%    60%    70%    80%    90% 
## 0.0010 0.0260 0.0383 0.0590 0.0777 0.1260 0.1827 0.2516 0.3296 0.5498 
##   100% 
## 0.9470

Roughly 75% of the observations in our testing window generated returns of 5% or more. And around 55% generated returns of 10% or more.

Generate Quantiles for Positive Returns by RSI Level 20

quantile(rsi_final$Return_Hi[rsi_final$Return_Hi > 0 &
                             rsi_final$RSI >  10 &
                             rsi_final$RSI <= 20], prob = seq(0, 1, length = 11), type = 5)
##     0%    10%    20%    30%    40%    50%    60%    70%    80%    90% 
## 0.0010 0.0320 0.0520 0.0812 0.1070 0.1370 0.1770 0.2268 0.3007 0.4188 
##   100% 
## 1.7560

Over 80% of the observations in our testing window generated returns of 5% or more. And over 60% generated returns of 10% or more.

Generate Quantiles for Positive Returns by RSI Level 30

quantile(rsi_final$Return_Hi[rsi_final$Return_Hi > 0 &
                             rsi_final$RSI >  20], prob = seq(0, 1, length = 11), type = 5)
##    0%   10%   20%   30%   40%   50%   60%   70%   80%   90%  100% 
## 0.001 0.026 0.049 0.073 0.097 0.126 0.158 0.199 0.260 0.374 3.786

Nearly 80% of the observations in our testing window generated returns of 5% or more. And nearly 60% generated returns of 10% or more.

Conclusion

Many RSI backtesting strategies focus on the closing price of a stock at the end of the testing window. This seems to defeat the purpose of trying to capture gains during a bounceback period. In other backtesting projects, I found that the highest stock price within the testing period does not follow a predictable pattern and can occur anytime within the window. So the preferred strategy I employ is to determine an exit price based on a desired return within the 3 month period.

For example, if a broader index is providing an annual average return of 12%, it would make sense to have a strategy resulting in a higher annualized return while covering investment costs and risks.

Most of the time, the RSI is one statistic, or signal, that is used in an overall investment strategy. Just as important is to have signals for exit also. Keep in mind that investments in the backtesting project resulted in losses throughout the period 7.1% of the time.

As always, the goal is to maximize returns and minimize losses.

Using the data from this project, and the created RSI final data frame, additional information can be explored. That is, what companies, sectors, or industries have the highest return success rate, or highest potential return.

In the next projects, I will cover some other common backtesting strategies and signals and compare their results.

sessionInfo()
## R version 3.4.0 (2017-04-21)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## Running under: Windows 10 x64 (build 14393)
## 
## Matrix products: default
## 
## locale:
## [1] LC_COLLATE=English_United States.1252 
## [2] LC_CTYPE=English_United States.1252   
## [3] LC_MONETARY=English_United States.1252
## [4] LC_NUMERIC=C                          
## [5] LC_TIME=English_United States.1252    
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
## [1] knitr_1.16      lubridate_1.6.0 dplyr_0.5.0     quantmod_0.4-9 
## [5] TTR_0.23-1      xts_0.9-7       zoo_1.8-0      
## 
## loaded via a namespace (and not attached):
##  [1] Rcpp_0.12.11     magrittr_1.5     lattice_0.20-35  R6_2.2.1        
##  [5] rlang_0.1.1      stringr_1.2.0    highr_0.6        tools_3.4.0     
##  [9] grid_3.4.0       DBI_0.6-1        htmltools_0.3.6  lazyeval_0.2.0  
## [13] yaml_2.1.14      rprojroot_1.2    digest_0.6.12    assertthat_0.2.0
## [17] tibble_1.3.3     evaluate_0.10    rmarkdown_1.5    stringi_1.1.5   
## [21] compiler_3.4.0   backports_1.1.0