Scraping Yahoo Finance Data & Conducting Time Series Analysis with R (R Studio) - for the Meetup event in Feb 2023
Your name, taught by Dr. zhenning Jimmy xu, Twitter: https://twitter.com/MKTJimmyxu
02/16/2023
R Markdown
Description: This meetup is for anyone interested in learning and sharing knowledge about scraping data from Yahoo Finance using R. Yahoo Finance provides a wealth of financial data that can be used for research, analysis, and investment purposes. In this meetup, we will discuss the basics of web scraping, explore the structure of Yahoo Finance pages, and walk through the process of scraping data from Yahoo Finance and analyse the data using R and its libraries such as ggplot2, quantmod, and forecast.
Agenda:
Introduction to web scraping and its applications
Overview of R libraries such as ggplot2, quantmod, and forecast
Live coding session on scraping data from Yahoo Finance using R and its libraries
Tips and tricks for efficient web scraping and handling common issues
Perform very basic time series analysis
Discussion and Q&A session
Who should attend?
Anyone who is interested in learning about web scraping and its application to financial data, from beginners to experienced data analysts and investors. This meetup is open to all skill levels.
Requirements: Participants should bring their laptops to the online event. Basic knowledge of R programming is recommended, but not required. Internet access will be required to access Yahoo Finance pages during the live coding session.
Intro to Quantmod
Quantmod is an R package that provides a suite of tools for quantitative financial modeling and analysis. It enables users to access and manipulate financial data from various sources, including Yahoo Finance. In this tutorial, we will walk through the steps of using quantmod to retrieve and analyze Yahoo Finance data.
To start using quantmod and other free libraries, we need to load the package into R by running the following command:
##
## Attaching package: 'dplyr'## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, union## Loading required package: xts## Loading required package: zoo##
## Attaching package: 'zoo'## The following objects are masked from 'package:base':
##
## as.Date, as.Date.numeric##
## ######################### Warning from 'xts' package ##########################
## # #
## # The dplyr lag() function breaks how base R's lag() function is supposed to #
## # work, which breaks lag(my_xts). Calls to lag(my_xts) that you type or #
## # source() into this session won't work correctly. #
## # #
## # Use stats::lag() to make sure you're not using dplyr::lag(), or you can add #
## # conflictRules('dplyr', exclude = 'lag') to your .Rprofile to stop #
## # dplyr from breaking base R's lag() function. #
## # #
## # Code in packages is not affected. It's protected by R's namespace mechanism #
## # Set `options(xts.warn_dplyr_breaks_lag = FALSE)` to suppress this warning. #
## # #
## #################################################################################
## Attaching package: 'xts'## The following objects are masked from 'package:dplyr':
##
## first, last## Loading required package: TTR## Registered S3 method overwritten by 'quantmod':
## method from
## as.zoo.data.frame zoo##
## Attaching package: 'lubridate'## The following objects are masked from 'package:base':
##
## date, intersect, setdiff, unionRetrieving Yahoo Finance Data
The first step in using quantmod to retrieve Yahoo Finance data is to specify the ticker symbol for the stock you want to analyze. For example, if you want to retrieve data for Tyson Foods and the Froster Farms, the ticker symbol are SMSN.IL and NVDA, seprarately.
Once you have the ticker symbol, you can use the getSymbols() function to retrieve the data. This function downloads data from various sources, including Yahoo Finance, and returns it as an object that can be manipulated in R.
To retrieve data for SMSN.IL and NVDA, run the following command:
getSymbols('SMSN.IL', src = 'yahoo',
from = "2010-01-01", to = Sys.Date())## [1] "SMSN.IL"Stock1 <- data.frame(
SMSN.IL,
date = as.Date(rownames(data.frame(SMSN.IL)))
)
getSymbols('NVDA', src = 'yahoo',
from = "2010-01-01", to = Sys.Date())## [1] "NVDA"Stock2 <- data.frame(
NVDA,
date = as.Date(rownames(data.frame(NVDA)))
)
head(Stock2)## NVDA.Open NVDA.High NVDA.Low NVDA.Close NVDA.Volume NVDA.Adjusted
## 2010-01-04 4.6275 4.6550 4.5275 4.6225 80020400 4.240427
## 2010-01-05 4.6050 4.7400 4.6050 4.6900 72864800 4.302348
## 2010-01-06 4.6875 4.7300 4.6425 4.7200 64916800 4.329868
## 2010-01-07 4.6950 4.7150 4.5925 4.6275 54779200 4.245016
## 2010-01-08 4.5900 4.6700 4.5625 4.6375 47816800 4.254188
## 2010-01-11 4.6625 4.6825 4.5075 4.5725 55661200 4.194562
## date
## 2010-01-04 2010-01-04
## 2010-01-05 2010-01-05
## 2010-01-06 2010-01-06
## 2010-01-07 2010-01-07
## 2010-01-08 2010-01-08
## 2010-01-11 2010-01-11This will download the daily historical data for these two stocks.
Note that we specified the start date using the from argument and the end date using the to argument. We set the end date to Sys.Date(), which retrieves data up to the current date.
Exploring the Data
Once you have retrieved the data, you can use various functions to explore and manipulate it. Here are a few examples:
Summary Statistics
To get a summary of the data, run the summary() function. ### Summary Statistics To get the first six rows of the data, run the head() function.
Now let’s dive deeper!
## Warning in cbind(Stock1, Stock2): number of rows of result is not a multiple of
## vector length (arg 2)## Stock1 Stock2
## [1,] 250.8255 4.240427
## [2,] 250.8255 4.302348
## [3,] 250.8255 4.329868
## [4,] 250.8255 4.245016
## [5,] 250.8255 4.254188
## [6,] 250.8255 4.194562## Warning in Stock1/Stock2: longer object length is not a multiple of shorter
## object length## [1] 59.15101 58.29968 57.92914 59.08706 58.95966 59.79778
#Now let’s run an event analysis and a time series analysis (under development)
## [1] "SMSN.IL"## [1] "SPY"## SPY.Open SPY.High SPY.Low SPY.Close SPY.Volume SPY.Adjusted
## 2010-01-04 112.37 113.39 111.51 113.33 118944600 87.12997
## 2010-01-05 113.26 113.68 112.85 113.63 111579900 87.36059
## 2010-01-06 113.52 113.99 113.43 113.71 116074400 87.42208
## 2010-01-07 113.50 114.33 113.18 114.19 131091100 87.79111
## 2010-01-08 113.89 114.62 113.66 114.57 126402800 88.08326
## 2010-01-11 115.08 115.13 114.24 114.73 106375700 88.20629
## date
## 2010-01-04 2010-01-04
## 2010-01-05 2010-01-05
## 2010-01-06 2010-01-06
## 2010-01-07 2010-01-07
## 2010-01-08 2010-01-08
## 2010-01-11 2010-01-11## SMSN.IL.Open SMSN.IL.High SMSN.IL.Low SMSN.IL.Close SMSN.IL.Volume
## 1 332.7 332.7 332.7 332.7 0
## 2 332.7 332.7 332.7 332.7 0
## 3 332.7 332.7 332.7 332.7 0
## 4 332.7 332.7 332.7 332.7 0
## 5 332.7 332.7 332.7 332.7 0
## 6 332.7 332.7 332.7 332.7 0
## SMSN.IL.Adjusted date SPY.Open SPY.High SPY.Low SPY.Close SPY.Volume
## 1 250.8255 2010-01-04 112.37 113.39 111.51 113.33 118944600
## 2 250.8255 2010-01-05 113.26 113.68 112.85 113.63 111579900
## 3 250.8255 2010-01-06 113.52 113.99 113.43 113.71 116074400
## 4 250.8255 2010-01-07 113.50 114.33 113.18 114.19 131091100
## 5 250.8255 2010-01-08 113.89 114.62 113.66 114.57 126402800
## 6 250.8255 2010-01-11 115.08 115.13 114.24 114.73 106375700
## SPY.Adjusted
## 1 87.12997
## 2 87.36059
## 3 87.42208
## 4 87.79111
## 5 88.08326
## 6 88.20629##
## Call:
## lm(formula = SMSN.IL.Adjusted ~ SPY.Adjusted, data = time_series)
##
## Residuals:
## Min 1Q Median 3Q Max
## -350.11 -87.74 -18.73 64.49 727.15
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -7.91810 5.52475 -1.433 0.152
## SPY.Adjusted 3.39113 0.02127 159.445 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 144.2 on 3485 degrees of freedom
## Multiple R-squared: 0.8794, Adjusted R-squared: 0.8794
## F-statistic: 2.542e+04 on 1 and 3485 DF, p-value: < 2.2e-16
References: Intro to the quantmod package. https://www.quantmod.com/ Using R for Time Series Analysis https://a-little-book-of-r-for-time-series.readthedocs.io/en/latest/src/timeseries.html