The TTR package is used to construct “Technical Trading Rules”. The TTR package can perform more sophisticated calculations and is worth exploring. The dygraphs library is a wrapper for a fast, open source JavaScript charting library. It is one of the htmlwidgets that makes R charting more dynamic and part of an html file instead of a static image. Lastly, the lubridate package is used for easy date manipulation
library(rvest)
library(pbapply)
library(TTR)
library(dygraphs)
library(lubridate)
library(tidyquant)
library(timetk)
pacman::p_load(dygraphs,DT,quantmod)
The quantmod package for R is designed to assist the quantitative trader in the development, testing, and deployment of statistically based trading models.
getSymbols("YHOO",src="google",from="2017-12-01", to="2018-01-01")%>%get() # from google finance
## YHOO.Open YHOO.High YHOO.Low YHOO.Close YHOO.Volume
## 2017-12-01 69.80 71.25 69.47 70.10 19712943
## 2017-12-04 71.06 71.34 68.16 68.31 15533597
## 2017-12-05 67.10 69.38 66.62 68.13 11963099
## 2017-12-06 67.78 69.62 67.36 69.41 11646582
## 2017-12-07 69.75 70.64 69.48 70.22 7084564
## 2017-12-08 71.00 71.63 70.82 71.25 7021125
## 2017-12-11 71.70 72.09 71.15 71.53 7359312
## 2017-12-12 71.07 71.15 69.55 69.77 11296188
## 2017-12-13 69.80 71.76 69.69 71.24 17321301
## 2017-12-14 70.60 70.62 68.84 69.23 17077065
## 2017-12-15 69.35 70.54 68.66 70.29 19517623
## 2017-12-18 70.54 70.74 69.95 70.14 6776982
## 2017-12-19 69.70 70.36 69.38 70.21 14654994
## 2017-12-20 70.60 71.05 70.17 70.90 12888149
## 2017-12-21 71.00 72.17 70.88 71.59 9682733
## 2017-12-22 71.42 71.87 71.22 71.58 10979165
## 2017-12-26 70.94 71.39 69.63 69.86 8542802
## 2017-12-27 69.77 70.49 69.69 70.06 6345124
## 2017-12-28 70.12 70.32 69.51 69.82 7556877
## 2017-12-29 69.79 70.13 69.43 69.85 6613070
getSymbols("GOOG",src="yahoo") # from yahoo finance
## [1] "GOOG"
getSymbols("DEXJPUS",src="FRED") # FX rates from FRED
## [1] "DEXJPUS"
# Specify lookup parameters, and save for future sessions.
setSymbolLookup(YHOO='google',GOOG='yahoo')
setSymbolLookup(DEXJPUS='FRED')
setSymbolLookup(XPTUSD=list(name="XPT/USD",src="oanda"))
saveSymbolLookup(file="mysymbols.rda")
# new sessions call loadSymbolLookup(file="mysymbols.rda")
getSymbols(c("YHOO","GOOG","DEXJPUS","XPTUSD"))
## [1] "YHOO" "GOOG" "DEXJPUS" "XPTUSD"
# Specify lookup parameters, and save for future sessions.
getSymbols("AAPL",src="yahoo", from="2017-01-01")
## [1] "AAPL"
barChart(to.monthly(AAPL),up.col='white',dn.col='blue')
Fig. 30
chartSeries(to.weekly(AAPL),up.col='white',dn.col='blue')
Fig. 30
# Add multi-coloring and change background to white
candleChart(AAPL,multi.col=TRUE,theme="white")
Fig. 30
#Non-OHLC and Volume series are handled automatically
getSymbols("XPT/USD",src="oanda")
## [1] "XPTUSD"
chartSeries(XPTUSD,name="Platinum (.oz) in $USD")
Fig. 30
Platinum, now weekly with custom color candles using the quantmod function to.weekly
chartSeries(to.weekly(XPTUSD),up.col='white',dn.col='blue')
Fig. 30
Technical analysis studies from package TTR to the above chart.
pacman::p_load(TTR)
getSymbols("AAPL")
## [1] "AAPL"
chartSeries(to.weekly(AAPL))
Fig. 30
addMACD()
Fig. 30
addBBands()
Fig. 30
# Create a quantmod object for use in
# in later model fitting. Note there is
# no need to load the data before hand.
setSymbolLookup(SPY='yahoo',VXN=list(name='^VIX',src='yahoo'))
mm <- specifyModel(Next(OpCl(SPY)) ~ OpCl(SPY) + Cl(VIX))
modelData(mm)%>%head()
## Next.OpCl.SPY OpCl.SPY Cl.VIX
## 2014-12-04 0.0006254149 0.0005782548 28447.7
## 2014-12-05 -0.0043851339 0.0006254149 26056.5
## 2014-12-08 0.0102755104 -0.0043851339 23582.8
## 2014-12-09 -0.0133553492 0.0102755104 21274.0
## 2014-12-10 0.0015204875 -0.0133553492 19295.0
## 2014-12-11 -0.0086360048 0.0015204875 17728.3
mm is now a quantmod object holding the model formula and data structure implying the next (Next) period’s open to close of the S&P 500 ETF (OpCl(SPY)) is modelled as a fucntion of the current period open to close and the current close of the VIX (Cl(VIX)).
The call to modelData extracts the relevant data set, with transforms magically applied. You can take the data and do with it as you’d like. A more direct function to accomplish the same end is buildData.
getSymbols("GS",from="2018-02-01") #Goldman OHLC from yahoo
## [1] "GS"
is.OHLC(GS) # does the data contain at least OHL and C?
## [1] TRUE
has.Vo(GS) # how about volume?
## [1] TRUE
Op(GS) # just the Open column please.
## GS.Open
## 2018-02-01 266.15
## 2018-02-02 270.00
## 2018-02-05 257.51
## 2018-02-06 244.21
## 2018-02-07 255.90
## 2018-02-08 257.93
## 2018-02-09 249.62
## 2018-02-12 251.15
## 2018-02-13 252.01
## 2018-02-14 255.00
## 2018-02-15 265.76
## 2018-02-16 265.99
## 2018-02-20 266.89
## 2018-02-21 263.92
## 2018-02-22 264.40
## 2018-02-23 263.00
seriesHi(GS) # where and what was the high point
## GS.Open GS.High GS.Low GS.Close GS.Volume GS.Adjusted
## 2018-02-01 266.15 272.59 266 272.23 3388200 272.23
OpCl(GS) #daily percent change open to close
## OpCl.GS
## 2018-02-01 0.022844325
## 2018-02-02 -0.036888856
## 2018-02-05 -0.032620126
## 2018-02-06 0.059334198
## 2018-02-07 0.004689379
## 2018-02-08 -0.044895853
## 2018-02-09 -0.001281917
## 2018-02-12 0.008003225
## 2018-02-13 0.013967716
## 2018-02-14 0.029725439
## 2018-02-15 0.007224499
## 2018-02-16 0.006128069
## 2018-02-20 -0.007493724
## 2018-02-21 -0.001970366
## 2018-02-22 -0.011232984
## 2018-02-23 0.014334559
OpOp(GS) #one period open to open change
## OpOp.GS
## 2018-02-01 NA
## 2018-02-02 0.0144655498
## 2018-02-05 -0.0462592222
## 2018-02-06 -0.0516484893
## 2018-02-07 0.0478685830
## 2018-02-08 0.0079327825
## 2018-02-09 -0.0322180368
## 2018-02-12 0.0061293127
## 2018-02-13 0.0034242525
## 2018-02-14 0.0118646286
## 2018-02-15 0.0421961176
## 2018-02-16 0.0008653672
## 2018-02-20 0.0033836800
## 2018-02-21 -0.0111281870
## 2018-02-22 0.0018186609
## 2018-02-23 -0.0052949850
HiCl(GS) #the percent change from high to close
## HiCl.GS
## 2018-02-01 -0.0013206097
## 2018-02-02 -0.0430910432
## 2018-02-05 -0.0538928509
## 2018-02-06 -0.0001931903
## 2018-02-07 -0.0185150058
## 2018-02-08 -0.0480330514
## 2018-02-09 -0.0094170740
## 2018-02-12 -0.0116728092
## 2018-02-13 -0.0033931163
## 2018-02-14 -0.0004948575
## 2018-02-15 -0.0011940560
## 2018-02-16 -0.0061646019
## 2018-02-20 -0.0147293692
## 2018-02-21 -0.0244444667
## 2018-02-22 -0.0183614219
## 2018-02-23 -0.0007866581
Lag(Cl(GS)) #One period lag of the close
## Lag.1
## 2018-02-01 NA
## 2018-02-02 272.23
## 2018-02-05 260.04
## 2018-02-06 249.11
## 2018-02-07 258.70
## 2018-02-08 257.10
## 2018-02-09 246.35
## 2018-02-12 249.30
## 2018-02-13 253.16
## 2018-02-14 255.53
## 2018-02-15 262.58
## 2018-02-16 267.68
## 2018-02-20 267.62
## 2018-02-21 264.89
## 2018-02-22 263.40
## 2018-02-23 261.43
Lag(Cl(GS),c(1,3,5)) #One, three, and five period lags
## Lag.1 Lag.3 Lag.5
## 2018-02-01 NA NA NA
## 2018-02-02 272.23 NA NA
## 2018-02-05 260.04 NA NA
## 2018-02-06 249.11 272.23 NA
## 2018-02-07 258.70 260.04 NA
## 2018-02-08 257.10 249.11 272.23
## 2018-02-09 246.35 258.70 260.04
## 2018-02-12 249.30 257.10 249.11
## 2018-02-13 253.16 246.35 258.70
## 2018-02-14 255.53 249.30 257.10
## 2018-02-15 262.58 253.16 246.35
## 2018-02-16 267.68 255.53 249.30
## 2018-02-20 267.62 262.58 253.16
## 2018-02-21 264.89 267.68 255.53
## 2018-02-22 263.40 267.62 262.58
## 2018-02-23 261.43 264.89 267.68
Next(OpCl(GS)) #The next periods open to close - today!
## Next
## 2018-02-01 -0.036888856
## 2018-02-02 -0.032620126
## 2018-02-05 0.059334198
## 2018-02-06 0.004689379
## 2018-02-07 -0.044895853
## 2018-02-08 -0.001281917
## 2018-02-09 0.008003225
## 2018-02-12 0.013967716
## 2018-02-13 0.029725439
## 2018-02-14 0.007224499
## 2018-02-15 0.006128069
## 2018-02-16 -0.007493724
## 2018-02-20 -0.001970366
## 2018-02-21 -0.011232984
## 2018-02-22 0.014334559
## 2018-02-23 NA
# Open to close one-day, two-day and three-day lags
Delt(Op(GS),Cl(GS),k=1:3)
## Delt.1.arithmetic Delt.2.arithmetic Delt.3.arithmetic
## 2018-02-01 NA NA NA
## 2018-02-02 -0.022956923 NA NA
## 2018-02-05 -0.077370367 -0.064024022 NA
## 2018-02-06 0.004621187 -0.041851807 -0.0279916670
## 2018-02-07 0.052782436 -0.001592187 -0.0477777556
## 2018-02-08 -0.037319219 0.008762946 -0.0433381366
## 2018-02-09 -0.033458652 -0.025791290 0.0208427004
## 2018-02-12 0.014181592 -0.018493348 -0.0107072687
## 2018-02-13 0.017439797 0.023676004 -0.0093048271
## 2018-02-14 0.041942749 0.045510624 0.0519188857
## 2018-02-15 0.049725463 0.062180066 0.0658172383
## 2018-02-16 0.006998739 0.049490176 0.0619419877
## 2018-02-20 -0.004135400 -0.003273611 0.0387843725
## 2018-02-21 -0.013076626 -0.009737193 -0.0088802525
## 2018-02-22 -0.009434752 -0.020457948 -0.0171434910
## 2018-02-23 0.008963673 0.010798635 -0.0004497208
GS['2018'] #returns all Goldman's 2018 OHLC
## GS.Open GS.High GS.Low GS.Close GS.Volume GS.Adjusted
## 2018-02-01 266.15 272.59 266.00 272.23 3388200 272.23
## 2018-02-02 270.00 271.75 259.35 260.04 5913900 260.04
## 2018-02-05 257.51 263.30 241.50 249.11 6528500 249.11
## 2018-02-06 244.21 258.75 242.11 258.70 6739900 258.70
## 2018-02-07 255.90 261.95 255.66 257.10 4125000 257.10
## 2018-02-08 257.93 258.78 246.05 246.35 4166200 246.35
## 2018-02-09 249.62 251.67 239.29 249.30 5978600 249.30
## 2018-02-12 251.15 256.15 248.82 253.16 3692500 253.16
## 2018-02-13 252.01 256.40 250.58 255.53 2700500 255.53
## 2018-02-14 255.00 262.71 254.48 262.58 3793800 262.58
## 2018-02-15 265.76 268.00 262.91 267.68 3523100 267.68
## 2018-02-16 265.99 269.28 265.90 267.62 2602500 267.62
## 2018-02-20 266.89 268.85 263.31 264.89 2618400 264.89
## 2018-02-21 263.92 270.00 263.13 263.40 3017900 263.40
## 2018-02-22 264.40 266.32 260.72 261.43 2678300 261.43
## 2018-02-23 263.00 266.98 262.40 266.77 2477900 266.77
GS['2018'] #now just 2008
## GS.Open GS.High GS.Low GS.Close GS.Volume GS.Adjusted
## 2018-02-01 266.15 272.59 266.00 272.23 3388200 272.23
## 2018-02-02 270.00 271.75 259.35 260.04 5913900 260.04
## 2018-02-05 257.51 263.30 241.50 249.11 6528500 249.11
## 2018-02-06 244.21 258.75 242.11 258.70 6739900 258.70
## 2018-02-07 255.90 261.95 255.66 257.10 4125000 257.10
## 2018-02-08 257.93 258.78 246.05 246.35 4166200 246.35
## 2018-02-09 249.62 251.67 239.29 249.30 5978600 249.30
## 2018-02-12 251.15 256.15 248.82 253.16 3692500 253.16
## 2018-02-13 252.01 256.40 250.58 255.53 2700500 255.53
## 2018-02-14 255.00 262.71 254.48 262.58 3793800 262.58
## 2018-02-15 265.76 268.00 262.91 267.68 3523100 267.68
## 2018-02-16 265.99 269.28 265.90 267.62 2602500 267.62
## 2018-02-20 266.89 268.85 263.31 264.89 2618400 264.89
## 2018-02-21 263.92 270.00 263.13 263.40 3017900 263.40
## 2018-02-22 264.40 266.32 260.72 261.43 2678300 261.43
## 2018-02-23 263.00 266.98 262.40 266.77 2477900 266.77
GS['2018-01'] #now just January of 2008
## GS.Open GS.High GS.Low GS.Close GS.Volume GS.Adjusted
GS['2017-06::2018-01-12'] #Jun of 17 through Jan 12 of 18
## GS.Open GS.High GS.Low GS.Close GS.Volume GS.Adjusted
GS['::'] # everything in GS
## GS.Open GS.High GS.Low GS.Close GS.Volume GS.Adjusted
## 2018-02-01 266.15 272.59 266.00 272.23 3388200 272.23
## 2018-02-02 270.00 271.75 259.35 260.04 5913900 260.04
## 2018-02-05 257.51 263.30 241.50 249.11 6528500 249.11
## 2018-02-06 244.21 258.75 242.11 258.70 6739900 258.70
## 2018-02-07 255.90 261.95 255.66 257.10 4125000 257.10
## 2018-02-08 257.93 258.78 246.05 246.35 4166200 246.35
## 2018-02-09 249.62 251.67 239.29 249.30 5978600 249.30
## 2018-02-12 251.15 256.15 248.82 253.16 3692500 253.16
## 2018-02-13 252.01 256.40 250.58 255.53 2700500 255.53
## 2018-02-14 255.00 262.71 254.48 262.58 3793800 262.58
## 2018-02-15 265.76 268.00 262.91 267.68 3523100 267.68
## 2018-02-16 265.99 269.28 265.90 267.62 2602500 267.62
## 2018-02-20 266.89 268.85 263.31 264.89 2618400 264.89
## 2018-02-21 263.92 270.00 263.13 263.40 3017900 263.40
## 2018-02-22 264.40 266.32 260.72 261.43 2678300 261.43
## 2018-02-23 263.00 266.98 262.40 266.77 2477900 266.77
GS['2018::'] # everything in GS, from 2018 onward
## GS.Open GS.High GS.Low GS.Close GS.Volume GS.Adjusted
## 2018-02-01 266.15 272.59 266.00 272.23 3388200 272.23
## 2018-02-02 270.00 271.75 259.35 260.04 5913900 260.04
## 2018-02-05 257.51 263.30 241.50 249.11 6528500 249.11
## 2018-02-06 244.21 258.75 242.11 258.70 6739900 258.70
## 2018-02-07 255.90 261.95 255.66 257.10 4125000 257.10
## 2018-02-08 257.93 258.78 246.05 246.35 4166200 246.35
## 2018-02-09 249.62 251.67 239.29 249.30 5978600 249.30
## 2018-02-12 251.15 256.15 248.82 253.16 3692500 253.16
## 2018-02-13 252.01 256.40 250.58 255.53 2700500 255.53
## 2018-02-14 255.00 262.71 254.48 262.58 3793800 262.58
## 2018-02-15 265.76 268.00 262.91 267.68 3523100 267.68
## 2018-02-16 265.99 269.28 265.90 267.62 2602500 267.62
## 2018-02-20 266.89 268.85 263.31 264.89 2618400 264.89
## 2018-02-21 263.92 270.00 263.13 263.40 3017900 263.40
## 2018-02-22 264.40 266.32 260.72 261.43 2678300 261.43
## 2018-02-23 263.00 266.98 262.40 266.77 2477900 266.77
non.contiguous <- c('2017-01','2007-02','2007-12')
GS[non.contiguous]
## GS.Open GS.High GS.Low GS.Close GS.Volume GS.Adjusted
Another common problem when trying to subset time series data often involves looking at a particular time period. Often the last n-periods may be desired when charting the price of a security, or when constructing a model for trading or analysis. [Note: these functions are now in the standalone package xts - which quantmod requires]
To facilitate this ‘time-based’ subsetting, one can use the functions first and last. Essentially extending the concept of head and tail, one can now use character strings to describe the part of the data to be returned. As is probably expected by now - an example may help to clarify.
last(GS) #returns the last obs.
## [1] 266.77
#last(GS, '3 months')
last(first(GS, '2 weeks'), '3 days')
## [1] 266.15
# let's try something a bit cooler.
#last(GS, '3 weeks')
#last(GS, '-2 weeks') # all except the last 2 weeks
Often, and especially with higher frequency data, it is necessary to aggregate data into lower frequency terms. For example, take daily data - OHLC or a standard time series - and convert it to weekly or monthly OHLC data.
With xts it is as simple as to.weekly or to.monthly. In fact, it is currently possible to take everything from minute data all the way up to quarterly data and convert it into something lower frequency. Minute data can become 5 or 10 minute data (to.minutes5 and to.minutes10, respectively), which can in turn be turned into hourly or daily data. Daily data can become weekly, monthly, or even yearly. All carried out in compiled code, and all blazingly fast, yes blazingly fast - convert 2 months of 1-minute bars into 3-minute bars in less than 0.1 seconds and anything lower in half that time. A full year of minute bars in less than a second on a moderately fast computer.
Is your data weekly, daily, or hourly? A call to periodicity will provide the answer; a call to nweeks will tell you the number of weeks as well.
periodicity(GS)
## Daily periodicity from 2018-02-01 to 2018-02-23
unclass(periodicity(GS))
## $difftime
## Time difference of 86400 days
##
## $frequency
## [1] 86400
##
## $start
## [1] "2018-02-01"
##
## $end
## [1] "2018-02-23"
##
## $units
## [1] "days"
##
## $scale
## [1] "daily"
##
## $label
## [1] "day"
to.weekly(GS)
## GS.Open GS.High GS.Low GS.Close GS.Volume GS.Adjusted
## 2018-02-02 266.15 272.59 259.35 260.04 9302100 260.04
## 2018-02-09 257.51 263.30 239.29 249.30 27538200 249.30
## 2018-02-16 251.15 269.28 248.82 267.62 16312400 267.62
## 2018-02-23 266.89 270.00 260.72 266.77 10792500 266.77
to.monthly(GS)
## GS.Open GS.High GS.Low GS.Close GS.Volume GS.Adjusted
## Feb 2018 266.15 272.59 239.29 266.77 63945200 266.77
#periodicity(to.monthly(GS))
ndays(GS); nweeks(GS); nyears(GS)
## [1] 16
## [1] 4
## [1] 1
# Let's try some non-OHLC to start
getFX("USD/EUR")
## [1] "USDEUR"
periodicity(USDEUR)
## Daily periodicity from 2017-08-31 to 2018-02-25
to.weekly(USDEUR)
## USDEUR.Open USDEUR.High USDEUR.Low USDEUR.Close
## 2017-09-03 0.841538 0.843034 0.841190 0.841720
## 2017-09-10 0.840512 0.840512 0.830135 0.831980
## 2017-09-17 0.834033 0.840856 0.834033 0.837547
## 2017-09-24 0.837044 0.839532 0.834508 0.839094
## 2017-10-01 0.841394 0.850608 0.841394 0.846481
## 2017-10-08 0.850892 0.853762 0.850040 0.852244
## 2017-10-15 0.851759 0.851759 0.843702 0.846069
## 2017-10-22 0.847410 0.849895 0.845750 0.848862
## 2017-10-29 0.850896 0.861444 0.848574 0.861444
## 2017-11-05 0.860084 0.861384 0.858084 0.860658
## 2017-11-12 0.861710 0.863012 0.857192 0.857226
## 2017-11-19 0.857868 0.857868 0.846988 0.847846
## 2017-11-26 0.850961 0.852056 0.838040 0.838040
## 2017-12-03 0.838817 0.843773 0.838817 0.840976
## 2017-12-10 0.843260 0.850480 0.843260 0.849527
## 2017-12-17 0.848514 0.851020 0.846602 0.851020
## 2017-12-24 0.848798 0.848798 0.842464 0.843826
## 2017-12-31 0.842984 0.842984 0.834758 0.834758
## 2018-01-07 0.833110 0.833110 0.829714 0.831254
## 2018-01-14 0.833770 0.837326 0.819653 0.819653
## 2018-01-21 0.816692 0.818451 0.816481 0.818240
## 2018-01-28 0.816724 0.816724 0.804242 0.804498
## 2018-02-04 0.807013 0.807013 0.801417 0.802910
## 2018-02-11 0.804272 0.816290 0.804272 0.816245
## 2018-02-18 0.814634 0.814634 0.801250 0.805784
## 2018-02-25 0.805853 0.813392 0.805853 0.813392
periodicity(to.weekly(USDEUR))
## Weekly periodicity from 2017-09-03 to 2018-02-25
It may be useful to identify endpoints in your data by date with the function endpoints. You can use those endpoints (or ones generated automatically) with the functions in the period.apply family. Quickly calculate periodic minimums, maximums, sums, and products - as well as general applys (with the periodic slant) with a few simple functions.
endpoints(GS,on="months")
## [1] 0 16
# find the maximum closing price each week
apply.weekly(GS,FUN=function(x) { max(Cl(x)) } )
## [,1]
## 2018-02-02 272.23
## 2018-02-09 258.70
## 2018-02-16 267.68
## 2018-02-23 266.77
# the same thing - only more general
period.apply(GS,endpoints(GS,on='weeks'),
FUN=function(x) { max(Cl(x)) } )
## [,1]
## 2018-02-02 272.23
## 2018-02-09 258.70
## 2018-02-16 267.68
## 2018-02-23 266.77
# same thing - only 50x faster!
as.numeric(period.max(Cl(GS),endpoints(GS,on='weeks')))
## [1] 272.23 258.70 267.68 266.77
Of course, additional wrappers exist to quickly apply arbitrary functions over daily, monthly, quarterly and annual time periods as well. There are also Fortran-based routines for period.min, period.sum, and period.prod, in addition to the period.max function.
The last set of functions simply provide a fast and reliable way to calculate returns over calendar periods - derived from the function periodReturn. Named for what they return. A note on starting/ending date convention, all periods could be named in a variety of ways - the first of the period, the first trading time of the period, the last trading time of the period, or even the last day of the period. xts has adopted the last observation of a given period as the date to record for the larger period. There may be a point in the future where this is settable as well. This is now user settable through the indexAt argument to the underlying to.period versions to.monthly and to.quarterly. The full details can be found in the related help pages, but a quick explanation is indexAt lets one set the resulting index to the first of each period (firstof), the last of each period (lastof), the starting observation of the period (startof), the ending observation of the period (endof), the month of the period (yearmon) or the quarter of the period (yearqtr). For most classes of time-series data this defaults to yearmon for monthly observations and yearqtr for quarterly requests.
# Quick returns - quantmod style
getSymbols("SBUX",from="2018-01-01")
## [1] "SBUX"
dailyReturn(SBUX) # returns by day
## daily.returns
## 2018-01-02 -0.005522002
## 2018-01-03 0.018740204
## 2018-01-04 0.003747249
## 2018-01-05 0.011539131
## 2018-01-08 -0.005032713
## 2018-01-09 -0.002191890
## 2018-01-10 0.010814464
## 2018-01-11 0.003009027
## 2018-01-12 0.006666700
## 2018-01-16 0.002648990
## 2018-01-17 0.001651238
## 2018-01-18 0.007088691
## 2018-01-19 0.002782747
## 2018-01-22 0.002448613
## 2018-01-23 0.004559502
## 2018-01-24 -0.013940623
## 2018-01-25 -0.004603041
## 2018-01-26 -0.042279059
## 2018-01-29 -0.016727056
## 2018-01-30 0.002981392
## 2018-01-31 -0.006644483
## 2018-02-01 -0.014258071
## 2018-02-02 -0.004107143
## 2018-02-05 -0.019365268
## 2018-02-06 0.016822125
## 2018-02-07 -0.020679769
## 2018-02-08 -0.012669831
## 2018-02-09 0.015064199
## 2018-02-12 0.014657365
## 2018-02-13 0.005236493
## 2018-02-14 0.005748231
## 2018-02-15 0.010537596
## 2018-02-16 -0.001767444
## 2018-02-20 -0.009029727
## 2018-02-21 0.002322619
## 2018-02-22 -0.012477648
## 2018-02-23 0.013357346
weeklyReturn(SBUX) # returns by week
## weekly.returns
## 2018-01-05 0.028645383
## 2018-01-12 0.013252826
## 2018-01-19 0.014238344
## 2018-01-26 -0.053378977
## 2018-02-02 -0.038282496
## 2018-02-09 -0.021337601
## 2018-02-16 0.034811248
## 2018-02-23 -0.006019848
monthlyReturn(SBUX) # returns by month, indexed by yearmon
## monthly.returns
## 2018-01-31 -0.01967213
## 2018-02-23 -0.01179373
# daily,weekly,monthly,quarterly, and yearly
allReturns(SBUX) # note the plural
## daily weekly monthly quarterly yearly
## 2018-01-02 NA NA NA NA NA
## 2018-01-03 0.018740204 NA NA NA NA
## 2018-01-04 0.003747249 NA NA NA NA
## 2018-01-05 0.011539131 0.028645383 NA NA NA
## 2018-01-08 -0.005032713 NA NA NA NA
## 2018-01-09 -0.002191890 NA NA NA NA
## 2018-01-10 0.010814464 NA NA NA NA
## 2018-01-11 0.003009027 NA NA NA NA
## 2018-01-12 0.006666700 0.013252826 NA NA NA
## 2018-01-16 0.002648990 NA NA NA NA
## 2018-01-17 0.001651238 NA NA NA NA
## 2018-01-18 0.007088691 NA NA NA NA
## 2018-01-19 0.002782747 0.014238344 NA NA NA
## 2018-01-22 0.002448613 NA NA NA NA
## 2018-01-23 0.004559502 NA NA NA NA
## 2018-01-24 -0.013940623 NA NA NA NA
## 2018-01-25 -0.004603041 NA NA NA NA
## 2018-01-26 -0.042279059 -0.053378977 NA NA NA
## 2018-01-29 -0.016727056 NA NA NA NA
## 2018-01-30 0.002981392 NA NA NA NA
## 2018-01-31 -0.006644483 NA -0.01967213 NA NA
## 2018-02-01 -0.014258071 NA NA NA NA
## 2018-02-02 -0.004107143 -0.038282496 NA NA NA
## 2018-02-05 -0.019365268 NA NA NA NA
## 2018-02-06 0.016822125 NA NA NA NA
## 2018-02-07 -0.020679769 NA NA NA NA
## 2018-02-08 -0.012669831 NA NA NA NA
## 2018-02-09 0.015064199 -0.021337601 NA NA NA
## 2018-02-12 0.014657365 NA NA NA NA
## 2018-02-13 0.005236493 NA NA NA NA
## 2018-02-14 0.005748231 NA NA NA NA
## 2018-02-15 0.010537596 NA NA NA NA
## 2018-02-16 -0.001767444 0.034811248 NA NA NA
## 2018-02-20 -0.009029727 NA NA NA NA
## 2018-02-21 0.002322619 NA NA NA NA
## 2018-02-22 -0.012477648 NA NA NA NA
## 2018-02-23 0.013357346 -0.006019848 -0.01179373 -0.03123386 -0.03123386
library(quantmod) # get stock prices; useful stock analysis functions
library(xts) # working with extensible time series
library(rvest) # web scraping
library(tidyverse) # ggplot2, purrr, dplyr, tidyr, readr, tibble
library(stringr) # working with strings
library(forcats) # working with factors
library(lubridate) # working with dates in tibbles / data frames
library(plotly) # Interactive plots
library(corrplot) # Visuazlize correlation plots
We’ll use MasterCard, ticker symbol “MA”, for the example.
getSymbols("MA", from = "2008-01-01", to = "2018-01-01")
## [1] "MA"
MA %>% class()
## [1] "xts" "zoo"
MA %>% str()
## An 'xts' object on 2008-01-02/2017-12-29 containing:
## Data: num [1:2518, 1:6] 21.8 21.5 20.5 20.2 20.1 ...
## - attr(*, "dimnames")=List of 2
## ..$ : NULL
## ..$ : chr [1:6] "MA.Open" "MA.High" "MA.Low" "MA.Close" ...
## Indexed by objects of class: [Date] TZ: UTC
## xts Attributes:
## List of 2
## $ src : chr "yahoo"
## $ updated: POSIXct[1:1], format: "2018-02-26 18:48:27"
MA %>% head()
## MA.Open MA.High MA.Low MA.Close MA.Volume MA.Adjusted
## 2008-01-02 21.790 22.029 21.260 21.468 20156000 17.31038
## 2008-01-03 21.456 21.505 20.513 21.212 28994000 17.10395
## 2008-01-04 20.482 20.920 19.844 20.119 38427000 16.22263
## 2008-01-07 20.171 20.199 18.744 20.051 43861000 16.16780
## 2008-01-08 20.091 20.770 19.599 19.800 33315000 15.96540
## 2008-01-09 19.700 20.021 18.965 19.658 38939000 15.97190
MA %>%
Ad() %>%
chartSeries()
Fig. 30
MA %>%
Cl() %>%
barChart()
Fig. 30
The chartSeries() function has a number of options help visualize the stock performance.
subset: Isolated the year to 2015. This takes dates and ranges in xts subset format. Here’s some tricks from StackOverflow. TA: Added Bollinger Bands, Volume, and Moving Average Convergence Divergence (MACD) plots. These can also be added after the fact with add___() functions (e.g addMACD()). theme: Changed to a white theme.
MA %>%
chartSeries(TA='addBBands();
addBBands(draw="p");
addVo();
addMACD()',
subset='2017',
theme="white"
)
Fig. 30
The quantmod package includes functions to get daily, weekly, monthly, quarterly, and annual returns. We’ll use dailyReturn() to get a xts object of daily returns (see ?periodReturn, which is the base function). The default is an arithmetic return. Specify type = “log” if a logarithmic return is needed
MA %>%
Ad() %>%
dailyReturn(type = 'log') %>%
head()
## daily.returns
## 2008-01-02 0.0000000000
## 2008-01-03 -0.0119966588
## 2008-01-04 -0.0529024317
## 2008-01-07 -0.0033856334
## 2008-01-08 -0.0125973558
## 2008-01-09 0.0004071101
The approach I use is similar to rfortraders.com’s Lecture 6 - Stochastic Processes and Monte Carlo. The fundamental idea is that stock returns, measured daily, weekly, monthly, …, are approximately normally distributed and uncorrelated. As a result, we can theoretically model the behavior of stock prices within a confidence interval based on the stock’s prior returns.
Applying the log-transformation, we can visually see that the daily returns are approximately normally distributed:
MA_log_returns <- MA %>%
Ad() %>%
dailyReturn(type = "log")
names(MA_log_returns) <- "MA.Log.Returns"
# Plot the log-returns
MA_log_returns %>%
ggplot(aes(x = MA.Log.Returns)) +
geom_histogram(bins = 100) +
geom_density() +
geom_rug(alpha = 0.5)
Fig. 30
We can examine the distribution of log returns by applying the quantile() function.
probs <- c(.005, .025, .25, .5, .75, .975, .995)
dist_log_returns <- MA_log_returns %>%
quantile(probs = probs, na.rm = TRUE)
dist_log_returns
## 0.5% 2.5% 25% 50% 75%
## -0.087845080 -0.041184360 -0.007923496 0.001225754 0.009356280
## 97.5% 99.5%
## 0.041506811 0.075800172
The median daily log return is 0.001226 and the 95% confidence interval is between -0.0412 and 0.0415.
The mean and standard deviation of the daily log returns are:
mean_log_returns <- mean(MA_log_returns, na.rm = TRUE)
sd_log_returns <- sd(MA_log_returns, na.rm = TRUE)
To get the actual returns, we need to re-transform the log returns. Pipe the mean of the log returns (mean_log_returns) to exp():
mean_log_returns %>% exp()
## [1] 1.000861
On average, the mean daily return is 0.086% more than the previous day’s price. Doesn’t sound like much, but it compounds daily at an exponential rate.
We can simulate random using a process called a random walk with the mean and standard deviation computed. We’ll simulate prices for 1000 trading days. Every year has approximately 252 trading days, so this simulation spans just under four years.
The script below does the following: we start by specifying the number of random walks (N), the mean (mu), and the standard deviation (sigma). The script then simulates prices by progressively calculating a new price using a random return from the normal distribution characterized by mu and sigma, and multiplying it to the previous day’s price. We then visualize the simulation.
# Parameters
N <- 1000
mu <- mean_log_returns
sigma <- sd_log_returns
day <- 1:N
price_init <- MA$MA.Adjusted[[nrow(MA$MA.Adjusted)]]
# Simulate prices
set.seed(386)
price <- c(price_init, rep(NA, N-1))
for(i in 2:N) {
price[i] <- price[i-1] * exp(rnorm(1, mu, sigma))
}
price_sim <- cbind(day, price) %>%
as_tibble()
# Visualize price simulation
price_sim %>%
ggplot(aes(day, price)) +
geom_line() +
ggtitle(str_c("MA: Simulated Prices for ", N," Trading Days"))
Fig. 30
Monte-Carlo simulation repeatedly perform the random walk simulation process hundreds or thousands of times. We’ll perform 250 Monte Carlo simulations (M = 250) for one year of trading days simulations (N = 252)
# Parameters
N <- 252 # Number of Stock Price Simulations
M <- 250 # Number of Monte Carlo Simulations
mu <- mean_log_returns
sigma <- sd_log_returns
day <- 1:N
price_init <- MA$MA.Adjusted[[nrow(MA$MA.Adjusted)]]
# Simulate prices
set.seed(123)
monte_carlo_mat <- matrix(nrow = N, ncol = M)
monte_carlo_mat[1, ] <- price_init
for (j in 1:M) {
for(i in 2:N) {
monte_carlo_mat[i, j] <- monte_carlo_mat[i - 1, j] * exp(rnorm(1, mu, sigma))
}
}
# Format and organize data frame
price_sim <- cbind(day, monte_carlo_mat) %>%
timetk::tk_tbl()
nm <- str_c("Sim.", seq(1, M))
#works similarly to paste0 below
#nm <- paste0("Sim.", seq(1, M))
nm <- c("Day", nm)
names(price_sim) <- nm
price_sim <- price_sim %>%
gather(key = "Simulation", value = "Stock.Price", -(Day))
# Visualize simulation
price_sim %>%
ggplot(aes(x = Day, y = Stock.Price, Group = Simulation)) +
geom_line(alpha = 0.1) +
ggtitle(str_c("MA: ", M,
" Monte Carlo Simulations for Prices Over ", N,
" Trading Days"))
Fig. 30
We can get confidence intervals for the stock price at the end of the simulation using the quantile() function.
end_stock_prices <- price_sim %>%
filter(Day == max(Day))
probs <- c(.005, .025, .25, .5, .75, .975, .995)
dist_end_stock_prices <- quantile(end_stock_prices$Stock.Price, probs = probs)
dist_end_stock_prices %>% round(2)
## 0.5% 2.5% 25% 50% 75% 97.5% 99.5%
## 97.29 104.27 151.93 185.72 230.46 310.91 422.70
The 95% confidence interval is between $104.27 and $310.91, with a median (“most likely”) estimated price of $185.72.
compound annual growth rate (CAGR):
# Inputs
N_hist <- nrow(MA) / 252
p_start_hist <- MA$MA.Adjusted[[1]]
p_end_hist <- MA$MA.Adjusted[[nrow(MA)]]
N_sim <- N / 252
p_start_sim <- p_end_hist
p_end_sim <- dist_end_stock_prices[[4]]
# CAGR calculations
CAGR_historical <- (p_end_hist / p_start_hist) ^ (1 / N_hist) - 1
CAGR_sim <- (p_end_sim / p_start_sim) ^ (1 / N_sim) - 1
summary(CAGR_sim)
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.229 0.229 0.229 0.229 0.229 0.229
library(rvest)
# Web-scrape SP500 stock list
sp_500 <- read_html("https://en.wikipedia.org/wiki/List_of_S%26P_500_companies") %>%
html_node("table.wikitable") %>%
html_table() %>%
select(`Ticker symbol`, Security, `GICS Sector`, `GICS Sub Industry`) %>%
as_tibble()
# Format names
names(sp_500) <- sp_500 %>%
names() %>%
str_to_lower() %>%
make.names()
# Show results
sp_500
## # A tibble: 505 x 4
## ticker.symbol security gics.sector gics.sub.industry
## <chr> <chr> <chr> <chr>
## 1 MMM 3M Company Industrials Industrial Conglomerat…
## 2 ABT Abbott Laborato… Health Care Health Care Equipment
## 3 ABBV AbbVie Inc. Health Care Pharmaceuticals
## 4 ACN Accenture plc Information Te… IT Consulting & Other …
## 5 ATVI Activision Bliz… Information Te… Home Entertainment Sof…
## 6 AYI Acuity Brands I… Industrials Electrical Components …
## 7 ADBE Adobe Systems I… Information Te… Application Software
## 8 AMD Advanced Micro … Information Te… Semiconductors
## 9 AAP Advance Auto Pa… Consumer Discr… Automotive Retail
## 10 AES AES Corp Utilities Independent Power Prod…
## 11 AET Aetna Inc Health Care Managed Health Care
## 12 AMG Affiliated Mana… Financials Asset Management & Cus…
## 13 AFL AFLAC Inc Financials Life & Health Insurance
## 14 A Agilent Technol… Health Care Health Care Equipment
## 15 APD Air Products & … Materials Industrial Gases
## 16 AKAM Akamai Technolo… Information Te… Internet Software & Se…
## 17 ALK Alaska Air Grou… Industrials Airlines
## 18 ALB Albemarle Corp Materials Specialty Chemicals
## 19 ARE Alexandria Real… Real Estate Office REITs
## 20 ALXN Alexion Pharmac… Health Care Biotechnology
## 21 ALGN Align Technology Health Care Health Care Supplies
## 22 ALLE Allegion Industrials Building Products
## 23 AGN Allergan, Plc Health Care Pharmaceuticals
## 24 ADS Alliance Data S… Information Te… Data Processing & Outs…
## 25 LNT Alliant Energy … Utilities Electric Utilities
## 26 ALL Allstate Corp Financials Property & Casualty In…
## 27 GOOGL Alphabet Inc Cl… Information Te… Internet Software & Se…
## 28 GOOG Alphabet Inc Cl… Information Te… Internet Software & Se…
## 29 MO Altria Group Inc Consumer Stapl… Tobacco
## 30 AMZN Amazon.com Inc Consumer Discr… Internet & Direct Mark…
## 31 AEE Ameren Corp Utilities Multi-Utilities
## 32 AAL American Airlin… Industrials Airlines
## 33 AEP American Electr… Utilities Electric Utilities
## 34 AXP American Expres… Financials Consumer Finance
## 35 AIG American Intern… Financials Property & Casualty In…
## 36 AMT American Tower … Real Estate Specialized REITs
## 37 AWK American Water … Utilities Water Utilities
## 38 AMP Ameriprise Fina… Financials Asset Management & Cus…
## 39 ABC AmerisourceBerg… Health Care Health Care Distributo…
## 40 AME AMETEK Inc Industrials Electrical Components …
## 41 AMGN Amgen Inc Health Care Biotechnology
## 42 APH Amphenol Corp Information Te… Electronic Components
## 43 APC Anadarko Petrol… Energy Oil & Gas Exploration …
## 44 ADI Analog Devices,… Information Te… Semiconductors
## 45 ANDV Andeavor Energy Oil & Gas Refining & M…
## 46 ANSS ANSYS Information Te… Application Software
## 47 ANTM Anthem Inc. Health Care Managed Health Care
## 48 AON Aon plc Financials Insurance Brokers
## 49 AOS A.O. Smith Corp Industrials Building Products
## 50 APA Apache Corporat… Energy Oil & Gas Exploration …
## 51 AIV Apartment Inves… Real Estate Residential REITs
## 52 AAPL Apple Inc. Information Te… Technology Hardware, S…
## 53 AMAT Applied Materia… Information Te… Semiconductor Equipment
## 54 APTV Aptiv Plc Consumer Discr… Auto Parts & Equipment
## 55 ADM Archer-Daniels-… Consumer Stapl… Agricultural Products
## 56 ARNC Arconic Inc Industrials Aerospace & Defense
## 57 AJG Arthur J. Galla… Financials Insurance Brokers
## 58 AIZ Assurant Inc Financials Multi-line Insurance
## 59 T AT&T Inc Telecommunicat… Integrated Telecommuni…
## 60 ADSK Autodesk Inc Information Te… Application Software
## 61 ADP Automatic Data … Information Te… Internet Software & Se…
## 62 AZO AutoZone Inc Consumer Discr… Specialty Stores
## 63 AVB AvalonBay Commu… Real Estate Residential REITs
## 64 AVY Avery Dennison … Materials Paper Packaging
## 65 BHGE Baker Hughes, a… Energy Oil & Gas Equipment & …
## 66 BLL Ball Corp Materials Metal & Glass Containe…
## 67 BAC Bank of America… Financials Diversified Banks
## 68 BK The Bank of New… Financials Asset Management & Cus…
## 69 BAX Baxter Internat… Health Care Health Care Equipment
## 70 BBT BB&T Corporation Financials Regional Banks
## 71 BDX Becton Dickinson Health Care Health Care Equipment
## 72 BRK.B Berkshire Hatha… Financials Multi-Sector Holdings
## 73 BBY Best Buy Co. In… Consumer Discr… Computer & Electronics…
## 74 BIIB Biogen Inc. Health Care Biotechnology
## 75 BLK BlackRock Financials Asset Management & Cus…
## 76 HRB Block H&R Financials Consumer Finance
## 77 BA Boeing Company Industrials Aerospace & Defense
## 78 BWA BorgWarner Consumer Discr… Auto Parts & Equipment
## 79 BXP Boston Properti… Real Estate Office REITs
## 80 BSX Boston Scientif… Health Care Health Care Equipment
## 81 BHF Brighthouse Fin… Financials Life & Health Insurance
## 82 BMY Bristol-Myers S… Health Care Health Care Distributo…
## 83 AVGO Broadcom Information Te… Semiconductors
## 84 BF.B Brown-Forman Co… Consumer Stapl… Distillers & Vintners
## 85 CHRW C. H. Robinson … Industrials Air Freight & Logistics
## 86 CA CA, Inc. Information Te… Systems Software
## 87 COG Cabot Oil & Gas Energy Oil & Gas Exploration …
## 88 CDNS Cadence Design … Information Te… Application Software
## 89 CPB Campbell Soup Consumer Stapl… Packaged Foods & Meats
## 90 COF Capital One Fin… Financials Consumer Finance
## 91 CAH Cardinal Health… Health Care Health Care Distributo…
## 92 CBOE CBOE Holdings Financials Financial Exchanges & …
## 93 KMX Carmax Inc Consumer Discr… Specialty Stores
## 94 CCL Carnival Corp. Consumer Discr… Hotels, Resorts & Crui…
## 95 CAT Caterpillar Inc. Industrials Construction Machinery…
## 96 CBG CBRE Group Real Estate Real Estate Services
## 97 CBS CBS Corp. Consumer Discr… Broadcasting
## 98 CELG Celgene Corp. Health Care Biotechnology
## 99 CNC Centene Corpora… Health Care Managed Health Care
## 100 CNP CenterPoint Ene… Utilities Multi-Utilities
## 101 CTL CenturyLink Inc Telecommunicat… Integrated Telecommuni…
## 102 CERN Cerner Health Care Health Care Technology
## 103 CF CF Industries H… Materials Fertilizers & Agricult…
## 104 SCHW Charles Schwab … Financials Investment Banking & B…
## 105 CHTR Charter Communi… Consumer Discr… Cable & Satellite
## 106 CHK Chesapeake Ener… Energy Oil & Gas Exploration …
## 107 CVX Chevron Corp. Energy Integrated Oil & Gas
## 108 CMG Chipotle Mexica… Consumer Discr… Restaurants
## 109 CB Chubb Limited Financials Property & Casualty In…
## 110 CHD Church & Dwight Consumer Stapl… Household Products
## 111 CI CIGNA Corp. Health Care Managed Health Care
## 112 XEC Cimarex Energy Energy Oil & Gas Exploration …
## 113 CINF Cincinnati Fina… Financials Property & Casualty In…
## 114 CTAS Cintas Corporat… Industrials Diversified Support Se…
## 115 CSCO Cisco Systems Information Te… Communications Equipme…
## 116 C Citigroup Inc. Financials Diversified Banks
## 117 CFG Citizens Financ… Financials Regional Banks
## 118 CTXS Citrix Systems Information Te… Internet Software & Se…
## 119 CLX The Clorox Comp… Consumer Stapl… Household Products
## 120 CME CME Group Inc. Financials Financial Exchanges & …
## 121 CMS CMS Energy Utilities Multi-Utilities
## 122 KO Coca-Cola Compa… Consumer Stapl… Soft Drinks
## 123 CTSH Cognizant Techn… Information Te… IT Consulting & Other …
## 124 CL Colgate-Palmoli… Consumer Stapl… Household Products
## 125 CMCSA Comcast Corp. Consumer Discr… Cable & Satellite
## 126 CMA Comerica Inc. Financials Diversified Banks
## 127 CAG Conagra Brands Consumer Stapl… Packaged Foods & Meats
## 128 CXO Concho Resources Energy Oil & Gas Exploration …
## 129 COP ConocoPhillips Energy Oil & Gas Exploration …
## 130 ED Consolidated Ed… Utilities Electric Utilities
## 131 STZ Constellation B… Consumer Stapl… Distillers & Vintners
## 132 COO The Cooper Comp… Health Care Health Care Supplies
## 133 GLW Corning Inc. Information Te… Electronic Components
## 134 COST Costco Wholesal… Consumer Stapl… Hypermarkets & Super C…
## 135 COTY Coty, Inc Consumer Stapl… Personal Products
## 136 CCI Crown Castle In… Real Estate Specialized REITs
## 137 CSRA CSRA Inc. Information Te… IT Consulting & Other …
## 138 CSX CSX Corp. Industrials Railroads
## 139 CMI Cummins Inc. Industrials Industrial Machinery
## 140 CVS CVS Health Consumer Stapl… Drug Retail
## 141 DHI D. R. Horton Consumer Discr… Homebuilding
## 142 DHR Danaher Corp. Health Care Health Care Equipment
## 143 DRI Darden Restaura… Consumer Discr… Restaurants
## 144 DVA DaVita Inc. Health Care Health Care Facilities
## 145 DE Deere & Co. Industrials Agricultural & Farm Ma…
## 146 DAL Delta Air Lines… Industrials Airlines
## 147 XRAY Dentsply Sirona Health Care Health Care Supplies
## 148 DVN Devon Energy Co… Energy Oil & Gas Exploration …
## 149 DLR Digital Realty … Real Estate Specialized REITs
## 150 DFS Discover Financ… Financials Consumer Finance
## 151 DISCA Discovery Commu… Consumer Discr… Cable & Satellite
## 152 DISCK Discovery Commu… Consumer Discr… Cable & Satellite
## 153 DISH Dish Network Consumer Discr… Cable & Satellite
## 154 DG Dollar General Consumer Discr… General Merchandise St…
## 155 DLTR Dollar Tree Consumer Discr… General Merchandise St…
## 156 D Dominion Energy Utilities Electric Utilities
## 157 DOV Dover Corp. Industrials Industrial Machinery
## 158 DWDP DowDuPont Materials Diversified Chemicals
## 159 DPS Dr Pepper Snapp… Consumer Stapl… Soft Drinks
## 160 DTE DTE Energy Co. Utilities Multi-Utilities
## 161 DRE Duke Realty Corp Real Estate Industrial REITs
## 162 DUK Duke Energy Utilities Electric Utilities
## 163 DXC DXC Technology Information Te… IT Consulting & Other …
## 164 ETFC E*Trade Financials Investment Banking & B…
## 165 EMN Eastman Chemical Materials Diversified Chemicals
## 166 ETN Eaton Corporati… Industrials Electrical Components …
## 167 EBAY eBay Inc. Information Te… Internet Software & Se…
## 168 ECL Ecolab Inc. Materials Specialty Chemicals
## 169 EIX Edison Int'l Utilities Electric Utilities
## 170 EW Edwards Lifesci… Health Care Health Care Equipment
## 171 EA Electronic Arts Information Te… Home Entertainment Sof…
## 172 EMR Emerson Electri… Industrials Electrical Components …
## 173 ETR Entergy Corp. Utilities Electric Utilities
## 174 EVHC Envision Health… Health Care Health Care Services
## 175 EOG EOG Resources Energy Oil & Gas Exploration …
## 176 EQT EQT Corporation Energy Oil & Gas Exploration …
## 177 EFX Equifax Inc. Industrials Research & Consulting …
## 178 EQIX Equinix Real Estate Specialized REITs
## 179 EQR Equity Resident… Real Estate Residential REITs
## 180 ESS Essex Property … Real Estate Residential REITs
## 181 EL Estee Lauder Co… Consumer Stapl… Personal Products
## 182 ES Eversource Ener… Utilities Multi-Utilities
## 183 RE Everest Re Grou… Financials Reinsurance
## 184 EXC Exelon Corp. Utilities Multi-Utilities
## 185 EXPE Expedia Inc. Consumer Discr… Internet & Direct Mark…
## 186 EXPD Expeditors Inte… Industrials Air Freight & Logistics
## 187 ESRX Express Scripts Health Care Health Care Distributo…
## 188 EXR Extra Space Sto… Real Estate Specialized REITs
## 189 XOM Exxon Mobil Cor… Energy Integrated Oil & Gas
## 190 FFIV F5 Networks Information Te… Communications Equipme…
## 191 FB Facebook, Inc. Information Te… Internet Software & Se…
## 192 FAST Fastenal Co Industrials Building Products
## 193 FRT Federal Realty … Real Estate Retail REITs
## 194 FDX FedEx Corporati… Industrials Air Freight & Logistics
## 195 FIS Fidelity Nation… Information Te… Internet Software & Se…
## 196 FITB Fifth Third Ban… Financials Regional Banks
## 197 FE FirstEnergy Corp Utilities Electric Utilities
## 198 FISV Fiserv Inc Information Te… Internet Software & Se…
## 199 FLIR FLIR Systems Information Te… Electronic Equipment &…
## 200 FLS Flowserve Corpo… Industrials Industrial Machinery
## 201 FLR Fluor Corp. Industrials Construction & Enginee…
## 202 FMC FMC Corporation Materials Fertilizers & Agricult…
## 203 FL Foot Locker Inc Consumer Discr… Apparel Retail
## 204 F Ford Motor Consumer Discr… Automobile Manufacture…
## 205 FTV Fortive Corp Industrials Industrial Machinery
## 206 FBHS Fortune Brands … Industrials Building Products
## 207 BEN Franklin Resour… Financials Asset Management & Cus…
## 208 FCX Freeport-McMoRa… Materials Copper
## 209 GPS Gap Inc. Consumer Discr… Apparel Retail
## 210 GRMN Garmin Ltd. Consumer Discr… Consumer Electronics
## 211 IT Gartner Inc Information Te… IT Consulting & Other …
## 212 GD General Dynamics Industrials Aerospace & Defense
## 213 GE General Electric Industrials Industrial Conglomerat…
## 214 GGP General Growth … Real Estate Retail REITs
## 215 GIS General Mills Consumer Stapl… Packaged Foods & Meats
## 216 GM General Motors Consumer Discr… Automobile Manufacture…
## 217 GPC Genuine Parts Consumer Discr… Specialty Stores
## 218 GILD Gilead Sciences Health Care Biotechnology
## 219 GPN Global Payments… Information Te… Data Processing & Outs…
## 220 GS Goldman Sachs G… Financials Investment Banking & B…
## 221 GT Goodyear Tire &… Consumer Discr… Tires & Rubber
## 222 GWW Grainger (W.W.)… Industrials Industrial Machinery
## 223 HAL Halliburton Co. Energy Oil & Gas Equipment & …
## 224 HBI Hanesbrands Inc Consumer Discr… Apparel, Accessories &…
## 225 HOG Harley-Davidson Consumer Discr… Motorcycle Manufacture…
## 226 HRS Harris Corporat… Information Te… Communications Equipme…
## 227 HIG Hartford Financ… Financials Property & Casualty In…
## 228 HAS Hasbro Inc. Consumer Discr… Leisure Products
## 229 HCA HCA Holdings Health Care Health Care Facilities
## 230 HCP HCP Inc. Real Estate Health Care REITs
## 231 HP Helmerich & Pay… Energy Oil & Gas Drilling
## 232 HSIC Henry Schein Health Care Health Care Distributo…
## 233 HSY The Hershey Com… Consumer Stapl… Packaged Foods & Meats
## 234 HES Hess Corporation Energy Integrated Oil & Gas
## 235 HPE Hewlett Packard… Information Te… Technology Hardware, S…
## 236 HLT Hilton Worldwid… Consumer Discr… Hotels, Resorts & Crui…
## 237 HOLX Hologic Health Care Health Care Equipment
## 238 HD Home Depot Consumer Discr… Home Improvement Retail
## 239 HON Honeywell Int'l… Industrials Industrial Conglomerat…
## 240 HRL Hormel Foods Co… Consumer Stapl… Packaged Foods & Meats
## 241 HST Host Hotels & R… Real Estate Hotel & Resort REITs
## 242 HPQ HP Inc. Information Te… Technology Hardware, S…
## 243 HUM Humana Inc. Health Care Managed Health Care
## 244 HBAN Huntington Banc… Financials Regional Banks
## 245 HII Huntington Inga… Industrials Aerospace & Defense
## 246 IDXX IDEXX Laborator… Health Care Health Care Equipment
## 247 INFO IHS Markit Ltd. Industrials Research & Consulting …
## 248 ITW Illinois Tool W… Industrials Industrial Machinery
## 249 ILMN Illumina Inc Health Care Life Sciences Tools & …
## 250 IR Ingersoll-Rand … Industrials Industrial Machinery
## 251 INTC Intel Corp. Information Te… Semiconductors
## 252 ICE Intercontinenta… Financials Financial Exchanges & …
## 253 IBM International B… Information Te… IT Consulting & Other …
## 254 INCY Incyte Health Care Biotechnology
## 255 IP International P… Materials Paper Packaging
## 256 IPG Interpublic Gro… Consumer Discr… Advertising
## 257 IFF Intl Flavors & … Materials Specialty Chemicals
## 258 INTU Intuit Inc. Information Te… Internet Software & Se…
## 259 ISRG Intuitive Surgi… Health Care Health Care Equipment
## 260 IVZ Invesco Ltd. Financials Asset Management & Cus…
## 261 IQV IQVIA Holdings … Health Care Life Sciences Tools & …
## 262 IRM Iron Mountain I… Real Estate Specialized REITs
## 263 JEC Jacobs Engineer… Industrials Construction & Enginee…
## 264 JBHT J. B. Hunt Tran… Industrials Trucking
## 265 SJM JM Smucker Consumer Stapl… Packaged Foods & Meats
## 266 JNJ Johnson & Johns… Health Care Health Care Equipment
## 267 JCI Johnson Control… Industrials Building Products
## 268 JPM JPMorgan Chase … Financials Diversified Banks
## 269 JNPR Juniper Networks Information Te… Communications Equipme…
## 270 KSU Kansas City Sou… Industrials Railroads
## 271 K Kellogg Co. Consumer Stapl… Packaged Foods & Meats
## 272 KEY KeyCorp Financials Regional Banks
## 273 KMB Kimberly-Clark Consumer Stapl… Household Products
## 274 KIM Kimco Realty Real Estate Retail REITs
## 275 KMI Kinder Morgan Energy Oil & Gas Storage & Tr…
## 276 KLAC KLA-Tencor Corp. Information Te… Semiconductor Equipment
## 277 KSS Kohl's Corp. Consumer Discr… General Merchandise St…
## 278 KHC Kraft Heinz Co Consumer Stapl… Packaged Foods & Meats
## 279 KR Kroger Co. Consumer Stapl… Food Retail
## 280 LB L Brands Inc. Consumer Discr… Apparel Retail
## 281 LLL L-3 Communicati… Industrials Aerospace & Defense
## 282 LH Laboratory Corp… Health Care Health Care Services
## 283 LRCX Lam Research Information Te… Semiconductor Equipment
## 284 LEG Leggett & Platt Consumer Discr… Home Furnishings
## 285 LEN Lennar Corp. Consumer Discr… Homebuilding
## 286 LUK Leucadia Nation… Financials Multi-Sector Holdings
## 287 LLY Lilly (Eli) & C… Health Care Pharmaceuticals
## 288 LNC Lincoln National Financials Multi-line Insurance
## 289 LKQ LKQ Corporation Consumer Discr… Distributors
## 290 LMT Lockheed Martin… Industrials Aerospace & Defense
## 291 L Loews Corp. Financials Multi-line Insurance
## 292 LOW Lowe's Cos. Consumer Discr… Home Improvement Retail
## 293 LYB LyondellBasell Materials Specialty Chemicals
## 294 MTB M&T Bank Corp. Financials Regional Banks
## 295 MAC Macerich Real Estate Retail REITs
## 296 M Macy's Inc. Consumer Discr… Department Stores
## 297 MRO Marathon Oil Co… Energy Oil & Gas Exploration …
## 298 MPC Marathon Petrol… Energy Oil & Gas Refining & M…
## 299 MAR Marriott Int'l. Consumer Discr… Hotels, Resorts & Crui…
## 300 MMC Marsh & McLennan Financials Insurance Brokers
## 301 MLM Martin Marietta… Materials Construction Materials
## 302 MAS Masco Corp. Industrials Building Products
## 303 MA Mastercard Inc. Information Te… Internet Software & Se…
## 304 MAT Mattel Inc. Consumer Discr… Leisure Products
## 305 MKC McCormick & Co. Consumer Stapl… Packaged Foods & Meats
## 306 MCD McDonald's Corp. Consumer Discr… Restaurants
## 307 MCK McKesson Corp. Health Care Health Care Distributo…
## 308 MDT Medtronic plc Health Care Health Care Equipment
## 309 MRK Merck & Co. Health Care Pharmaceuticals
## 310 MET MetLife Inc. Financials Life & Health Insurance
## 311 MTD Mettler Toledo Health Care Life Sciences Tools & …
## 312 MGM MGM Resorts Int… Consumer Discr… Casinos & Gaming
## 313 KORS Michael Kors Ho… Consumer Discr… Apparel, Accessories &…
## 314 MCHP Microchip Techn… Information Te… Semiconductors
## 315 MU Micron Technolo… Information Te… Semiconductors
## 316 MSFT Microsoft Corp. Information Te… Systems Software
## 317 MAA Mid-America Apa… Real Estate Residential REITs
## 318 MHK Mohawk Industri… Consumer Discr… Home Furnishings
## 319 TAP Molson Coors Br… Consumer Stapl… Brewers
## 320 MDLZ Mondelez Intern… Consumer Stapl… Packaged Foods & Meats
## 321 MON Monsanto Co. Materials Fertilizers & Agricult…
## 322 MNST Monster Beverage Consumer Stapl… Soft Drinks
## 323 MCO Moody's Corp Financials Financial Exchanges & …
## 324 MS Morgan Stanley Financials Investment Banking & B…
## 325 MOS The Mosaic Comp… Materials Fertilizers & Agricult…
## 326 MSI Motorola Soluti… Information Te… Communications Equipme…
## 327 MYL Mylan N.V. Health Care Pharmaceuticals
## 328 NDAQ Nasdaq, Inc. Financials Financial Exchanges & …
## 329 NOV National Oilwel… Energy Oil & Gas Equipment & …
## 330 NAVI Navient Financials Consumer Finance
## 331 NTAP NetApp Information Te… Internet Software & Se…
## 332 NFLX Netflix Inc. Information Te… Internet Software & Se…
## 333 NWL Newell Brands Consumer Discr… Housewares & Specialti…
## 334 NFX Newfield Explor… Energy Oil & Gas Exploration …
## 335 NEM Newmont Mining … Materials Gold
## 336 NWSA News Corp. Clas… Consumer Discr… Publishing
## 337 NWS News Corp. Clas… Consumer Discr… Publishing
## 338 NEE NextEra Energy Utilities Multi-Utilities
## 339 NLSN Nielsen Holdings Industrials Research & Consulting …
## 340 NKE Nike Consumer Discr… Apparel, Accessories &…
## 341 NI NiSource Inc. Utilities Multi-Utilities
## 342 NBL Noble Energy Inc Energy Oil & Gas Exploration …
## 343 JWN Nordstrom Consumer Discr… Department Stores
## 344 NSC Norfolk Souther… Industrials Railroads
## 345 NTRS Northern Trust … Financials Asset Management & Cus…
## 346 NOC Northrop Grumma… Industrials Aerospace & Defense
## 347 NCLH Norwegian Cruis… Consumer Discr… Hotels, Resorts & Crui…
## 348 NRG NRG Energy Utilities Independent Power Prod…
## 349 NUE Nucor Corp. Materials Steel
## 350 NVDA Nvidia Corporat… Information Te… Semiconductors
## 351 ORLY O'Reilly Automo… Consumer Discr… Specialty Stores
## 352 OXY Occidental Petr… Energy Oil & Gas Exploration …
## 353 OMC Omnicom Group Consumer Discr… Advertising
## 354 OKE ONEOK Energy Oil & Gas Storage & Tr…
## 355 ORCL Oracle Corp. Information Te… Application Software
## 356 PCAR PACCAR Inc. Industrials Construction Machinery…
## 357 PKG Packaging Corpo… Materials Paper Packaging
## 358 PH Parker-Hannifin Industrials Industrial Machinery
## 359 PDCO Patterson Compa… Health Care Health Care Supplies
## 360 PAYX Paychex Inc. Information Te… Internet Software & Se…
## 361 PYPL PayPal Information Te… Data Processing & Outs…
## 362 PNR Pentair Ltd. Industrials Industrial Machinery
## 363 PBCT People's United… Financials Thrifts & Mortgage Fin…
## 364 PEP PepsiCo Inc. Consumer Stapl… Soft Drinks
## 365 PKI PerkinElmer Health Care Health Care Equipment
## 366 PRGO Perrigo Health Care Pharmaceuticals
## 367 PFE Pfizer Inc. Health Care Pharmaceuticals
## 368 PCG PG&E Corp. Utilities Multi-Utilities
## 369 PM Philip Morris I… Consumer Stapl… Tobacco
## 370 PSX Phillips 66 Energy Oil & Gas Refining & M…
## 371 PNW Pinnacle West C… Utilities Multi-Utilities
## 372 PXD Pioneer Natural… Energy Oil & Gas Exploration …
## 373 PNC PNC Financial S… Financials Regional Banks
## 374 RL Polo Ralph Laur… Consumer Discr… Apparel, Accessories &…
## 375 PPG PPG Industries Materials Specialty Chemicals
## 376 PPL PPL Corp. Utilities Electric Utilities
## 377 PX Praxair Inc. Materials Industrial Gases
## 378 PCLN Priceline.com I… Consumer Discr… Internet & Direct Mark…
## 379 PFG Principal Finan… Financials Life & Health Insurance
## 380 PG Procter & Gamble Consumer Stapl… Personal Products
## 381 PGR Progressive Cor… Financials Property & Casualty In…
## 382 PLD Prologis Real Estate Industrial REITs
## 383 PRU Prudential Fina… Financials Life & Health Insurance
## 384 PEG Public Serv. En… Utilities Electric Utilities
## 385 PSA Public Storage Real Estate Specialized REITs
## 386 PHM Pulte Homes Inc. Consumer Discr… Homebuilding
## 387 PVH PVH Corp. Consumer Discr… Apparel, Accessories &…
## 388 QRVO Qorvo Information Te… Semiconductors
## 389 PWR Quanta Services… Industrials Construction & Enginee…
## 390 QCOM QUALCOMM Inc. Information Te… Semiconductors
## 391 DGX Quest Diagnosti… Health Care Health Care Services
## 392 RRC Range Resources… Energy Oil & Gas Exploration …
## 393 RJF Raymond James F… Financials Investment Banking & B…
## 394 RTN Raytheon Co. Industrials Aerospace & Defense
## 395 O Realty Income C… Real Estate Retail REITs
## 396 RHT Red Hat Inc. Information Te… Systems Software
## 397 REG Regency Centers… Real Estate Retail REITs
## 398 REGN Regeneron Health Care Biotechnology
## 399 RF Regions Financi… Financials Regional Banks
## 400 RSG Republic Servic… Industrials Environmental & Facili…
## 401 RMD ResMed Health Care Health Care Equipment
## 402 RHI Robert Half Int… Industrials Human Resource & Emplo…
## 403 ROK Rockwell Automa… Industrials Electrical Components …
## 404 COL Rockwell Collins Industrials Aerospace & Defense
## 405 ROP Roper Technolog… Industrials Industrial Conglomerat…
## 406 ROST Ross Stores Consumer Discr… Apparel Retail
## 407 RCL Royal Caribbean… Consumer Discr… Hotels, Resorts & Crui…
## 408 CRM Salesforce.com Information Te… Internet Software & Se…
## 409 SBAC SBA Communicati… Real Estate Specialized REITs
## 410 SCG SCANA Corp Utilities Multi-Utilities
## 411 SLB Schlumberger Lt… Energy Oil & Gas Equipment & …
## 412 SNI Scripps Network… Consumer Discr… Cable & Satellite
## 413 STX Seagate Technol… Information Te… Technology Hardware, S…
## 414 SEE Sealed Air Materials Paper Packaging
## 415 SRE Sempra Energy Utilities Multi-Utilities
## 416 SHW Sherwin-Williams Materials Specialty Chemicals
## 417 SIG Signet Jewelers Consumer Discr… Specialty Stores
## 418 SPG Simon Property … Real Estate Retail REITs
## 419 SWKS Skyworks Soluti… Information Te… Semiconductors
## 420 SLG SL Green Realty Real Estate Office REITs
## 421 SNA Snap-On Inc. Consumer Discr… Household Appliances
## 422 SO Southern Co. Utilities Electric Utilities
## 423 LUV Southwest Airli… Industrials Airlines
## 424 SPGI S&P Global, Inc. Financials Financial Exchanges & …
## 425 SWK Stanley Black &… Consumer Discr… Household Appliances
## 426 SBUX Starbucks Corp. Consumer Discr… Restaurants
## 427 STT State Street Co… Financials Asset Management & Cus…
## 428 SRCL Stericycle Inc Industrials Environmental & Facili…
## 429 SYK Stryker Corp. Health Care Health Care Equipment
## 430 STI SunTrust Banks Financials Regional Banks
## 431 SYMC Symantec Corp. Information Te… Application Software
## 432 SYF Synchrony Finan… Financials Consumer Finance
## 433 SNPS Synopsys Inc. Information Te… Application Software
## 434 SYY Sysco Corp. Consumer Stapl… Food Distributors
## 435 TROW T. Rowe Price G… Financials Asset Management & Cus…
## 436 TPR Tapestry, Inc. Consumer Discr… Apparel, Accessories &…
## 437 TGT Target Corp. Consumer Discr… General Merchandise St…
## 438 TEL TE Connectivity… Information Te… Electronic Manufacturi…
## 439 FTI TechnipFMC Energy Oil & Gas Equipment & …
## 440 TXN Texas Instrumen… Information Te… Semiconductors
## 441 TXT Textron Inc. Industrials Aerospace & Defense
## 442 TMO Thermo Fisher S… Health Care Health Care Equipment
## 443 TIF Tiffany & Co. Consumer Discr… Apparel, Accessories &…
## 444 TWX Time Warner Inc. Consumer Discr… Cable & Satellite
## 445 TJX TJX Companies I… Consumer Discr… Apparel Retail
## 446 TMK Torchmark Corp. Financials Life & Health Insurance
## 447 TSS Total System Se… Information Te… Internet Software & Se…
## 448 TSCO Tractor Supply … Consumer Discr… Specialty Stores
## 449 TDG TransDigm Group Industrials Aerospace & Defense
## 450 TRV The Travelers C… Financials Property & Casualty In…
## 451 TRIP TripAdvisor Consumer Discr… Internet & Direct Mark…
## 452 FOXA Twenty-First Ce… Consumer Discr… Publishing
## 453 FOX Twenty-First Ce… Consumer Discr… Publishing
## 454 TSN Tyson Foods Consumer Stapl… Packaged Foods & Meats
## 455 UDR UDR Inc Real Estate Residential REITs
## 456 ULTA Ulta Beauty Consumer Discr… Specialty Stores
## 457 USB U.S. Bancorp Financials Diversified Banks
## 458 UAA Under Armour Cl… Consumer Discr… Apparel, Accessories &…
## 459 UA Under Armour Cl… Consumer Discr… Apparel, Accessories &…
## 460 UNP Union Pacific Industrials Railroads
## 461 UAL United Continen… Industrials Airlines
## 462 UNH United Health G… Health Care Managed Health Care
## 463 UPS United Parcel S… Industrials Air Freight & Logistics
## 464 URI United Rentals,… Industrials Trading Companies & Di…
## 465 UTX United Technolo… Industrials Aerospace & Defense
## 466 UHS Universal Healt… Health Care Health Care Facilities
## 467 UNM Unum Group Financials Life & Health Insurance
## 468 VFC V.F. Corp. Consumer Discr… Apparel, Accessories &…
## 469 VLO Valero Energy Energy Oil & Gas Refining & M…
## 470 VAR Varian Medical … Health Care Health Care Equipment
## 471 VTR Ventas Inc Real Estate Health Care REITs
## 472 VRSN Verisign Inc. Information Te… Internet Software & Se…
## 473 VRSK Verisk Analytics Industrials Research & Consulting …
## 474 VZ Verizon Communi… Telecommunicat… Integrated Telecommuni…
## 475 VRTX Vertex Pharmace… Health Care Biotechnology
## 476 VIAB Viacom Inc. Consumer Discr… Cable & Satellite
## 477 V Visa Inc. Information Te… Internet Software & Se…
## 478 VNO Vornado Realty … Real Estate Office REITs
## 479 VMC Vulcan Materials Materials Construction Materials
## 480 WMT Wal-Mart Stores Consumer Stapl… Hypermarkets & Super C…
## 481 WBA Walgreens Boots… Consumer Stapl… Drug Retail
## 482 DIS The Walt Disney… Consumer Discr… Cable & Satellite
## 483 WM Waste Managemen… Industrials Environmental & Facili…
## 484 WAT Waters Corporat… Health Care Health Care Distributo…
## 485 WEC Wec Energy Grou… Utilities Electric Utilities
## 486 WFC Wells Fargo Financials Diversified Banks
## 487 HCN Welltower Inc. Real Estate Health Care REITs
## 488 WDC Western Digital Information Te… Technology Hardware, S…
## 489 WU Western Union Co Information Te… Internet Software & Se…
## 490 WRK WestRock Company Materials Paper Packaging
## 491 WY Weyerhaeuser Co… Real Estate Specialized REITs
## 492 WHR Whirlpool Corp. Consumer Discr… Household Appliances
## 493 WMB Williams Cos. Energy Oil & Gas Storage & Tr…
## 494 WLTW Willis Towers W… Financials Insurance Brokers
## 495 WYN Wyndham Worldwi… Consumer Discr… Hotels, Resorts & Crui…
## 496 WYNN Wynn Resorts Ltd Consumer Discr… Casinos & Gaming
## 497 XEL Xcel Energy Inc Utilities Multi-Utilities
## 498 XRX Xerox Corp. Information Te… Technology Hardware, S…
## 499 XLNX Xilinx Inc Information Te… Semiconductors
## 500 XL XL Capital Financials Property & Casualty In…
## 501 XYL Xylem Inc. Industrials Industrial Machinery
## 502 YUM Yum! Brands Inc Consumer Discr… Restaurants
## 503 ZBH Zimmer Biomet H… Health Care Health Care Equipment
## 504 ZION Zions Bancorp Financials Regional Banks
## 505 ZTS Zoetis Health Care Pharmaceuticals
The lapply() function below loops through each column of the data set counting the length of the unique items. The result is a count of distinct values for each column.
sp_500 %>%
lapply(function(x) x %>% unique() %>% length()) %>%
unlist() # show in condensed format
## ticker.symbol security gics.sector gics.sub.industry
## 505 505 11 122
We have 505 ticker symbols but only 505 securities. There are no duplicate entry in securities. Let’s inspect any securities that are duplicated. We’ll use the dplyr functions group_by(), summarize(), and filter() to find any security that appears more than once.
sp_500 %>%
group_by(security) %>%
summarize(count = n()) %>%
filter(count > 1)
## # A tibble: 0 x 2
## # ... with 2 variables: security <chr>, count <int>
We know from the lapply() function that there are 505 securities that are in 122 sub industries and in 11 sectors. However, we might want to understand the distribution of securities by sector. I again use group_by() and summarise() to get counts. In the visualization, we can use forcats::fct_reorder() to organize the gics.sector by greatest frequency (count). The forcats package simplifies working with caregories in R.
sp_500 %>%
# Summarise data by frequency
group_by(gics.sector) %>%
summarise(count = n()) %>%
# Visualize
ggplot(aes(x = gics.sector %>% fct_reorder(count),
y = count
)) +
geom_bar(stat = "identity") +
geom_text(aes(label = count), size = 3, nudge_y = 4, nudge_x = .1) +
scale_y_continuous(limits = c(0,100)) +
ggtitle(label = "Sector Frequency Among SP500 Stocks") +
xlab(label = "GICS Sector") +
theme(plot.title = element_text(size = 16)) +
coord_flip()
Fig. 30
get_stock_prices() is a wrapper for quantmod::getSymbols() that takes a ticker, a return format, and any other getSymbols() arguments and returns the prices in either xts or tibble format. In this post, we’ll use the tibble format for unnesting. We set auto.assign = FALSE to return the object to the variable rather than automatically creating the variable in the Global Environment.
get_stock_prices <- function(ticker, return_format = "tibble", ...) {
# Get stock prices
stock_prices_xts <- getSymbols(Symbols = ticker, auto.assign = FALSE, ...)
# Rename
names(stock_prices_xts) <- c("Open", "High", "Low", "Close", "Volume", "Adjusted")
# Return in xts format if tibble is not specified
if (return_format == "tibble") {
stock_prices <- stock_prices_xts %>%
as_tibble() %>%
rownames_to_column(var = "Date") %>%
mutate(Date = ymd(Date))
} else {
stock_prices <- stock_prices_xts
}
stock_prices
}
"MA" %>%
get_stock_prices(return_format = 'tibble')%>%head()
## # A tibble: 6 x 7
## Date Open High Low Close Volume Adjusted
## <date> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 2007-01-03 9.96 9.97 9.56 9.64 26289000 7.46
## 2 2007-01-04 9.69 10.2 9.53 10.1 27024000 7.83
## 3 2007-01-05 10.1 10.2 9.90 10.1 29632000 7.82
## 4 2007-01-08 9.90 10.2 9.90 10.1 16006000 7.82
## 5 2007-01-09 10.1 10.6 10.1 10.6 36952000 8.18
## 6 2007-01-10 10.6 10.6 10.3 10.5 35099000 8.17
get_log_returns() is a wrapper for quantmod::periodReturns() that takes a stock prices in xts or tibble format, a return format, and any other periodReturns() arguments and returns the log returns in either xts or tibble format.
get_log_returns <- function(x, return_format = "tibble", period = 'daily', ...) {
# Convert tibble to xts
if (!is.xts(x)) {
x <- xts(x[,-1], order.by = x$Date)
}
# Get log returns
log_returns_xts <- periodReturn(x = x$Adjusted, type = 'log', period = period, ...)
# Rename
names(log_returns_xts) <- "Log.Returns"
# Return in xts format if tibble is not specified
if (return_format == "tibble") {
log_returns <- log_returns_xts %>%
as_tibble() %>%
rownames_to_column(var = "Date") %>%
mutate(Date = ymd(Date))
} else {
log_returns <- log_returns_xts
}
log_returns
}
"MA" %>%
get_stock_prices(return_format = 'tibble')%>%
get_log_returns(return_format = "tibble")%>%head()
## # A tibble: 6 x 2
## Date Log.Returns
## <date> <dbl>
## 1 2007-01-03 0
## 2 2007-01-04 0.0479
## 3 2007-01-05 -0.000791
## 4 2007-01-08 0.000198
## 5 2007-01-09 0.0451
## 6 2007-01-10 -0.000954
sp_500 <- sp_500[1:50,] %>%
mutate(
stock.prices = map(ticker.symbol,
function(.x) get_stock_prices(.x,
return_format = "tibble",
from = "2015-01-01",
to = "2018-01-01")
),
log.returns = map(stock.prices,
function(.x) get_log_returns(.x, return_format = "tibble")),
mean.log.returns = map_dbl(log.returns, ~ mean(.$Log.Returns)),
sd.log.returns = map_dbl(log.returns, ~ sd(.$Log.Returns)),
n.trade.days = map_dbl(stock.prices, nrow)
)
sp_500[1:5,]
## # A tibble: 5 x 9
## ticker.symbol security gics.sector gics.sub.industry stock.prices
## <chr> <chr> <chr> <chr> <list>
## 1 MMM 3M Company Industrials Industrial Conglo… <tibble [755…
## 2 ABT Abbott Lab… Health Care Health Care Equip… <tibble [755…
## 3 ABBV AbbVie Inc. Health Care Pharmaceuticals <tibble [755…
## 4 ACN Accenture … Information … IT Consulting & O… <tibble [755…
## 5 ATVI Activision… Information … Home Entertainmen… <tibble [755…
## # ... with 4 more variables: log.returns <list>, mean.log.returns <dbl>,
## # sd.log.returns <dbl>, n.trade.days <dbl>
sp_500 %>%
select(ticker.symbol, stock.prices:log.returns)
## # A tibble: 50 x 3
## ticker.symbol stock.prices log.returns
## <chr> <list> <list>
## 1 MMM <tibble [755 × 7]> <tibble [755 × 2]>
## 2 ABT <tibble [755 × 7]> <tibble [755 × 2]>
## 3 ABBV <tibble [755 × 7]> <tibble [755 × 2]>
## 4 ACN <tibble [755 × 7]> <tibble [755 × 2]>
## 5 ATVI <tibble [755 × 7]> <tibble [755 × 2]>
## 6 AYI <tibble [755 × 7]> <tibble [755 × 2]>
## 7 ADBE <tibble [755 × 7]> <tibble [755 × 2]>
## 8 AMD <tibble [755 × 7]> <tibble [755 × 2]>
## 9 AAP <tibble [755 × 7]> <tibble [755 × 2]>
## 10 AES <tibble [755 × 7]> <tibble [755 × 2]>
## 11 AET <tibble [755 × 7]> <tibble [755 × 2]>
## 12 AMG <tibble [755 × 7]> <tibble [755 × 2]>
## 13 AFL <tibble [755 × 7]> <tibble [755 × 2]>
## 14 A <tibble [755 × 7]> <tibble [755 × 2]>
## 15 APD <tibble [755 × 7]> <tibble [755 × 2]>
## 16 AKAM <tibble [755 × 7]> <tibble [755 × 2]>
## 17 ALK <tibble [755 × 7]> <tibble [755 × 2]>
## 18 ALB <tibble [755 × 7]> <tibble [755 × 2]>
## 19 ARE <tibble [755 × 7]> <tibble [755 × 2]>
## 20 ALXN <tibble [755 × 7]> <tibble [755 × 2]>
## 21 ALGN <tibble [755 × 7]> <tibble [755 × 2]>
## 22 ALLE <tibble [755 × 7]> <tibble [755 × 2]>
## 23 AGN <tibble [755 × 7]> <tibble [755 × 2]>
## 24 ADS <tibble [755 × 7]> <tibble [755 × 2]>
## 25 LNT <tibble [755 × 7]> <tibble [755 × 2]>
## 26 ALL <tibble [755 × 7]> <tibble [755 × 2]>
## 27 GOOGL <tibble [755 × 7]> <tibble [755 × 2]>
## 28 GOOG <tibble [755 × 7]> <tibble [755 × 2]>
## 29 MO <tibble [755 × 7]> <tibble [755 × 2]>
## 30 AMZN <tibble [755 × 7]> <tibble [755 × 2]>
## 31 AEE <tibble [755 × 7]> <tibble [755 × 2]>
## 32 AAL <tibble [755 × 7]> <tibble [755 × 2]>
## 33 AEP <tibble [755 × 7]> <tibble [755 × 2]>
## 34 AXP <tibble [755 × 7]> <tibble [755 × 2]>
## 35 AIG <tibble [755 × 7]> <tibble [755 × 2]>
## 36 AMT <tibble [755 × 7]> <tibble [755 × 2]>
## 37 AWK <tibble [755 × 7]> <tibble [755 × 2]>
## 38 AMP <tibble [755 × 7]> <tibble [755 × 2]>
## 39 ABC <tibble [755 × 7]> <tibble [755 × 2]>
## 40 AME <tibble [755 × 7]> <tibble [755 × 2]>
## 41 AMGN <tibble [755 × 7]> <tibble [755 × 2]>
## 42 APH <tibble [755 × 7]> <tibble [755 × 2]>
## 43 APC <tibble [755 × 7]> <tibble [755 × 2]>
## 44 ADI <tibble [755 × 7]> <tibble [755 × 2]>
## 45 ANDV <tibble [755 × 7]> <tibble [755 × 2]>
## 46 ANSS <tibble [755 × 7]> <tibble [755 × 2]>
## 47 ANTM <tibble [755 × 7]> <tibble [755 × 2]>
## 48 AON <tibble [755 × 7]> <tibble [755 × 2]>
## 49 AOS <tibble [755 × 7]> <tibble [755 × 2]>
## 50 APA <tibble [755 × 7]> <tibble [755 × 2]>
sp_500$stock.prices[[1]]%>%head()
## # A tibble: 6 x 7
## Date Open High Low Close Volume Adjusted
## <date> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 2015-01-02 165 165 163 164 2116400 151
## 2 2015-01-05 163 164 160 160 3692900 148
## 3 2015-01-06 161 161 158 159 3532000 146
## 4 2015-01-07 160 160 159 160 3081300 147
## 5 2015-01-08 161 164 161 164 3142900 151
## 6 2015-01-09 164 164 161 162 2373800 149
library(plotly)
plot_ly(data = sp_500,
type = "scatter",
mode = "markers",
x = ~ sd.log.returns,
y = ~ mean.log.returns,
color = ~ n.trade.days,
colors = "Blues",
size = ~ n.trade.days,
text = ~ str_c("<em>", security, "</em><br>",
"Ticker: ", ticker.symbol, "<br>",
"Sector: ", gics.sector, "<br>",
"Sub Sector: ", gics.sub.industry, "<br>",
"No. of Trading Days: ", n.trade.days),
marker = list(opacity = 0.8,
symbol = 'circle',
sizemode = 'diameter',
sizeref = 4.0,
line = list(width = 2, color = '#FFFFFF'))
) %>%
layout(title = 'S&P500 Analysis: Stock Risk vs Reward',
xaxis = list(title = 'Risk/Variability (StDev Log Returns)',
gridcolor = 'rgb(255, 255, 255)',
zerolinewidth = 1,
ticklen = 5,
gridwidth = 2),
yaxis = list(title = 'Reward/Growth (Mean Log Returns)',
gridcolor = 'rgb(255, 255, 255)',
zerolinewidth = 1,
ticklen = 5,
gridwith = 2),
margin = list(l = 100,
t = 100,
b = 100),
font = list(color = '#FFFFFF'),
paper_bgcolor = 'rgb(0, 0, 0)',
plot_bgcolor = 'rgb(0, 0, 0)')
Fig. 30
sp_500 %>%
filter(mean.log.returns >= 0.001,
sd.log.returns < 0.0315) %>%
select(ticker.symbol, mean.log.returns:n.trade.days) %>%
arrange(mean.log.returns %>% desc())
## # A tibble: 6 x 4
## ticker.symbol mean.log.returns sd.log.returns n.trade.days
## <chr> <dbl> <dbl> <dbl>
## 1 ALGN 0.00182 0.0196 755
## 2 AMZN 0.00176 0.0177 755
## 3 ATVI 0.00155 0.0187 755
## 4 ADBE 0.00117 0.0147 755
## 5 AOS 0.00111 0.0142 755
## 6 ALB 0.00106 0.0190 755
"PCLN" %>%
getSymbols(auto.assign = FALSE,
from = "2007-01-01",
to = "2016-10-23") %>%
chartSeries(name = "PCLN")
Fig. 30
cut the list down to the top 30 stocks with more than two years of samples (approx 500 trading days). We’ll add a rank column using the min_rank() function on mean.log.returns, and filter the data on the top 30 ranked means.
limit <- 30
sp_500_hp <- sp_500 %>%
filter(n.trade.days > 500) %>%
filter(sd.log.returns < 0.2) %>%
mutate(rank = mean.log.returns %>% desc() %>% min_rank()) %>%
filter(rank <= limit) %>%
arrange(rank) %>%
select(ticker.symbol, rank, mean.log.returns, sd.log.returns, log.returns)
sp_500_hp
## # A tibble: 30 x 5
## ticker.symbol rank mean.log.returns sd.log.returns log.returns
## <chr> <int> <dbl> <dbl> <list>
## 1 ALGN 1 0.00182 0.0196 <tibble [755 × 2]>
## 2 AMD 2 0.00179 0.0412 <tibble [755 × 2]>
## 3 AMZN 3 0.00176 0.0177 <tibble [755 × 2]>
## 4 ATVI 4 0.00155 0.0187 <tibble [755 × 2]>
## 5 ADBE 5 0.00117 0.0147 <tibble [755 × 2]>
## 6 AOS 6 0.00111 0.0142 <tibble [755 × 2]>
## 7 ALB 7 0.00106 0.0190 <tibble [755 × 2]>
## 8 AET 8 0.000980 0.0155 <tibble [755 × 2]>
## 9 GOOG 9 0.000921 0.0139 <tibble [755 × 2]>
## 10 GOOGL 10 0.000911 0.0138 <tibble [755 × 2]>
## 11 ANTM 11 0.000841 0.0146 <tibble [755 × 2]>
## 12 ACN 12 0.000816 0.0117 <tibble [755 × 2]>
## 13 AWK 13 0.000786 0.0107 <tibble [755 × 2]>
## 14 ANSS 14 0.000784 0.0128 <tibble [755 × 2]>
## 15 ADI 15 0.000727 0.0162 <tibble [755 × 2]>
## 16 A 16 0.000705 0.0133 <tibble [755 × 2]>
## 17 APH 17 0.000690 0.0110 <tibble [755 × 2]>
## 18 MO 18 0.000649 0.0107 <tibble [755 × 2]>
## 19 ABBV 19 0.000649 0.0165 <tibble [755 × 2]>
## 20 ANDV 20 0.000643 0.0206 <tibble [755 × 2]>
## 21 ARE 21 0.000621 0.0118 <tibble [755 × 2]>
## 22 ALL 22 0.000604 0.0104 <tibble [755 × 2]>
## 23 MMM 23 0.000578 0.00977 <tibble [755 × 2]>
## 24 AFL 24 0.000576 0.00989 <tibble [755 × 2]>
## 25 AMT 25 0.000554 0.0120 <tibble [755 × 2]>
## 26 LNT 26 0.000519 0.0110 <tibble [755 × 2]>
## 27 ALLE 27 0.000517 0.0127 <tibble [755 × 2]>
## 28 AON 28 0.000512 0.0102 <tibble [755 × 2]>
## 29 AEE 29 0.000471 0.0107 <tibble [755 × 2]>
## 30 AME 30 0.000450 0.0118 <tibble [755 × 2]>
sp_500_hp_unnest <- sp_500_hp %>%
select(ticker.symbol, log.returns) %>%
unnest()
sp_500_hp_unnest%>%head()
## # A tibble: 6 x 3
## ticker.symbol Date Log.Returns
## <chr> <date> <dbl>
## 1 ALGN 2015-01-02 0
## 2 ALGN 2015-01-05 0.0140
## 3 ALGN 2015-01-06 0.00804
## 4 ALGN 2015-01-07 0.0362
## 5 ALGN 2015-01-08 0.0309
## 6 ALGN 2015-01-09 0.0184
sp_500_hp_spread <- sp_500_hp_unnest %>%
spread(key = ticker.symbol, value = Log.Returns) %>%
na.omit()
sp_500_hp_spread%>%head()
## # A tibble: 6 x 31
## Date A ABBV ACN ADBE ADI AEE
## <date> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 2015-01-02 0 0 0 0 0 0
## 2 2015-01-05 -0.0189 -0.0190 -0.0170 -0.00499 -0.0184 -0.0101
## 3 2015-01-06 -0.0157 -0.00496 -0.00724 -0.0204 -0.0238 -0.0117
## 4 2015-01-07 0.0132 0.0396 0.0208 0.00819 0.0105 0.00775
## 5 2015-01-08 0.0295 0.0104 0.0151 0.0251 0.0175 0.00352
## 6 2015-01-09 -0.00736 -0.0277 -0.00111 -0.0149 -0.000731 -0.0113
## # ... with 24 more variables: AET <dbl>, AFL <dbl>, ALB <dbl>, ALGN <dbl>,
## # ALL <dbl>, ALLE <dbl>, AMD <dbl>, AME <dbl>, AMT <dbl>, AMZN <dbl>,
## # ANDV <dbl>, ANSS <dbl>, ANTM <dbl>, AON <dbl>, AOS <dbl>, APH <dbl>,
## # ARE <dbl>, ATVI <dbl>, AWK <dbl>, GOOG <dbl>, GOOGL <dbl>, LNT <dbl>,
## # MMM <dbl>, MO <dbl>
sp_500_hp_cor <- sp_500_hp_spread %>%
select(-Date) %>%
cor()
sp_500_hp_cor[1:6, 1:6] # show first 6 columns and rows
## A ABBV ACN ADBE ADI AEE
## A 1.0000000 0.4375766 0.5364386 0.5100089 0.4855185 0.2252420
## ABBV 0.4375766 1.0000000 0.2506644 0.2966285 0.2492603 0.1730033
## ACN 0.5364386 0.2506644 1.0000000 0.4950324 0.4285719 0.2534351
## ADBE 0.5100089 0.2966285 0.4950324 1.0000000 0.4531565 0.2686607
## ADI 0.4855185 0.2492603 0.4285719 0.4531565 1.0000000 0.1658043
## AEE 0.2252420 0.1730033 0.2534351 0.2686607 0.1658043 1.0000000
The correlations can be visualized with corrplot(). It can be ordered using hclust, which groups the stocks by similarity. The addrect argument adds boxes around the highly correlated groups, and by inspecting the circle colors I settled on 11 boxes.
pacman::p_load(corrplot)
sp_500_hp_cor %>%
corrplot(order = "hclust",
addrect = 10)
Fig. 30