Stock Market Price Evaluation and Forecast
David Montalvo
12/16/2019
Introduction
For the Data Analytics final project I decided to realize an evaluation and forecast of the stock market. I will be evaluating both the best and worst stocks through the last decade in order to understand the trends, behavior and the factors that affect the prices of these stocks. I selected a variety of datasets for the realization of my project. The first dataset I included is the “Huge Stock Market Dataset” found in Kaggle. This dataset provides detailed information of almost, if not all the stocks in the NYSE, NASDAQ, and NYSE MKT markets. This dataset includes information such as the opening and closing price of a stock, the highest and lowest prices that the stock reached throughout the day, volume, which is the amount of stocks that were traded on that day and of course, the date of the transactions that were made. Another Dataset that I used was the “S&P 500 Companies with Financial Information” found on DataHub. This dataset contains more specific information regarding the individual stocks, it does not show progression of prices throughout time. In this dataset, information such as the sector of each company, the market capital of the companies, earnings per share, amongst other variables can be observed. Finally, I used the Netflix and Freeport stock datasets from MacroTrends website in which historic progression of the stock prices can be observed for each stock which prove to be essential for the final part of the project. The questions I will be attempting to answer by the end of this project will be “Does the volume of the stocks sold affect the increase/decrease of its price in the opening and closing?”, “How do the most profitable stock compare to the least profitable?”, “Is the prediction model accurate?” and “Does sector play a role in the profitability of a stock?”. The main variables that will be used to answer these questions will be “Volume” (amount of stocks sold on each day), “Close” (the final price of a stock after a day) “AdjClose”(closing price of a stock after dividends have been paid), “Open” (the initial price of a stock when markets open), “High”(highest price reached), “Low” (lowest price reached), “Sector”, and “time”. Through the use of these datasets, variables along with the different evaluation methods that we have learned in this semester, I hope to accurately answer the research questions planted.
Dataset Links:
https://www.kaggle.com/borismarjanovic/price-volume-data-for-all-us-stocks-etfs, https://datahub.io/core/s-and-p-500-companies-financials#resource-constituents-financials, https://www.macrotrends.net/stocks/charts/NFLX/netflix/stock-price-history, https://www.macrotrends.net/stocks/charts/FCX/freeport-mcmoran/stock-price-history
Filtered Companies
head(WorstStocks)## # A tibble: 5 x 14
## Symbol Name Sector Price `Price/Earnings` `Dividend Yield`
## <chr> <chr> <chr> <dbl> <dbl> <dbl>
## 1 CTL Cent… Telec… 16.2 8.35 12.7
## 2 APA Apac… Energy 37.7 -252. 2.53
## 3 MOS The … Mater… 24.1 23.7 0.396
## 4 DVN Devo… Energy 34.9 23.9 0.653
## 5 FCX Free… Mater… 17.2 14.7 1.12
## # … with 8 more variables: `Earnings/Share` <dbl>, `52 Week Low` <dbl>,
## # `52 Week High` <dbl>, `Market Cap` <dbl>, EBITDA <dbl>,
## # `Price/Sales` <dbl>, `Price/Book` <dbl>, `SEC Filings` <chr>head(BestStocks)## # A tibble: 5 x 14
## Symbol Name Sector Price `Price/Earnings` `Dividend Yield`
## <chr> <chr> <chr> <dbl> <dbl> <dbl>
## 1 MNST Mons… Consu… 62.0 42.2 0
## 2 NFLX Netf… Infor… 250. 200. 0
## 3 TSCO Trac… Consu… 65.9 19.6 1.60
## 4 EQIX Equi… Real … 410. 23.1 1.88
## 5 ISRG Intu… Healt… 382. 48.6 0
## # … with 8 more variables: `Earnings/Share` <dbl>, `52 Week Low` <dbl>,
## # `52 Week High` <dbl>, `Market Cap` <dbl>, EBITDA <dbl>,
## # `Price/Sales` <dbl>, `Price/Book` <dbl>, `SEC Filings` <chr>Through the use of filter in order to select ten different stocks, they will be grouped by as “best” and “worst”. The information which allowed me to select these stocks came from Business Insider and CNBC, which provide detailed informatio on the performance of these stocks throughout the decade and have been ranked. The first five stocks that I filtered out of the “Huge Stock Market Dataset” (StocksListing) are: CenturyLink Inc (CTL), Apache Corporation (APA), The Mosaic Company (MOS), Devon Energy Corp. (DVN), Freeport-McMoRan Inc. (FCX). These stocks are the worst performing stocks of the decade with very low profitability. Subsequently, the next five stocks are the best performing ones of the decade which are: Monster Beverage (MNST), Netflix Inc. (NFLX), Tractor Supply Company (TSCO), Equinix (EQIX) and Intuitive Surgical Inc. (ISRG). These stocks have proven to be the most profitable and best performing stocks of the decade according to different sources, thus being the ones selected for my analysis.
Top 5 Stocks
B <- ggplot(BestStocksSL,aes(factor(symbol),close,color=symbol,frame=date)) +
geom_jitter(aes(size = close, colour=symbol, alpha=.02)) +
ylim(0,700)+
labs(title = "Top 5 Stocks Prices", x = "Stocks", y= "Close Price") +
theme(panel.border = element_blank(),
panel.grid.major = element_line(colour = "grey61", size = 0.5, linetype = "dotted"),
panel.grid.minor = element_blank(),
axis.line=element_line(colour="black"),
plot.title = element_text(hjust = 0.5,size=18,colour="indianred4"))
B
ggplot(BestStocks, aes(x=Name ,y=Price)) +geom_col() + scale_fill_manual() + labs(title = "Best Performing Stocks for the Last Decade")
In the graphs that are displayed above, the top 5 companies with most profitable shares are illustrated showing the average price of each company’s shares. Here, it can be observed how not necessarily being the most profitable stocks mean having expensive stocks such as Monster or, the Tractor Supply Company which have shares of less than one hundred dollars.
Bottom 5 Stocks
A <- ggplot(WorstStocksSL,aes(factor(symbol),close,color=symbol,frame=date)) +
geom_jitter(aes(size = close, colour=symbol, alpha=.02)) +
ylim(0,50)+
labs(title = "Bottom 5 Stocks Prices", x = "Stocks", y= "Close Price") +
theme(panel.border = element_blank(),
panel.grid.major = element_line(colour = "grey61", size = 0.5, linetype = "dotted"),
panel.grid.minor = element_blank(),
axis.line=element_line(colour="black"),
plot.title = element_text(hjust = 0.5,size=18,colour="indianred4"))
A
ggplot(WorstStocks, aes(x=Name ,y=Price)) +geom_col(stat="identity", colour="black") + scale_fill_manual()+ scale_fill_manual() + labs(title = "Worst Performing Stocks for the Last Decade")+ scale_colour_manual("", breaks = c("Apache Corporation", "CenturyLink Inc"))
The plots above illustrate the average price of the least profitable stocks, where there are similarities in the price of these 5 businesses shares, ranging in between about 15 and 40 dollars for one share.
Best Against Worst Stocks
Above can be demonstrated a difference in prices between the selected “best” and “worst” stocks. As it can be seen, the cheapest stock from the best ones is Monster with a price of 61.99, which is a high price for one stock, whereas the worst stocks have a highest price of 37.73 which is almos half of the cheapest best stock. Here it can be demosntrated how as companies tend to perform better their stock prices rise highly and on the other hand, when a company does not perform, their prices tend to be less costly.
Selection of Two Stocks for Analysis
head(Freeport)## # A tibble: 6 x 8
## date symbol volume open close high low adjclose
## <date> <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 2019-04-18 FCX 27426700 14.1 14 14.1 13.8 14
## 2 2019-04-17 FCX 31700500 14.4 14.2 14.7 14.1 14.2
## 3 2019-04-16 FCX 20299800 13.7 14.2 14.2 13.6 14.2
## 4 2019-04-15 FCX 15439300 13.6 13.7 13.7 13.4 13.7
## 5 2019-04-12 FCX 26915900 13.6 13.7 14.0 13.6 13.7
## 6 2019-04-11 FCX 16534500 13.3 13.4 13.5 13.1 13.4head(Netflix)## # A tibble: 6 x 8
## date symbol volume open close high low adjclose
## <date> <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 2019-04-18 NFLX 8353200 355 360. 360. 352. 360.
## 2 2019-04-17 NFLX 18054100 365. 355. 369. 351. 355.
## 3 2019-04-16 NFLX 18740200 355 359. 364. 353. 359.
## 4 2019-04-15 NFLX 8842300 351. 349. 352. 342. 349.
## 5 2019-04-12 NFLX 15646200 361. 351. 362. 349. 351.
## 6 2019-04-11 NFLX 6526900 365 368. 370. 361. 368.The both companies that I have picked to evaluate their performance have been Freeport and Netflix. I decided to pick Freeport, a mining company, due to it being highlighted as the least profitable stock. On the other hand, Netflix is amongst the most profitable ones and as it is in the technological industry, I assume that the stock prices would continuously increase. Thus, I will be comparing one of the most profitable companies to one of the least.
Animated Plots Netflix and Freeport Prices Through History
NetflixHist=gvisMotionChart(NetflixStockD,
idvar="close",
timevar="x")
plot(NetflixHist)FCXHist=gvisMotionChart(FCXStockD,
idvar="close",
timevar="x")
plot(FCXHist)Comparing both of the plots for Netflix and Freeport, there can be seen a difference of the progression of each stock as Netflix stock prices demonstrate to continuously grow compared to Freeport which constantly have prices that are maintained within a same range fluctuating constantly in highs and lows.
NetflixStockD %>%ggplot()+ aes(x=volume, y=x, col="Netflix") + geom_point() +labs(title = "Netflix Amount of Stocks Sold Throughout Time", x="Volume", y="Time")+xlim(0, 150000000)
With this plot, the variation of the amount of stocks sold throughout the years have changed for Netflix. There seems to not be much variation of the amount sold and can be said how Netflix has maintained the amount of shares they have sold since the creation of the company to stay between 0 and 50 million shares.
FCXStockD %>%ggplot()+ aes(x=volume, y=x, col="Netflix") + geom_point() +labs(title = "Netflix Amount of Stocks Sold Throughout Time", x="Volume", y="Time") 
On the other hand, in Freeport proves to be different than Netflix, as they have had a lot of variation in the amount of shares they have sold throughout time. As it can be seen, with the years, Freeport has increased the volume, additionally to how in 2010 and about 2017 the shares available for the public rose and almost reach 100 million shares sold. Compared to Netflix, Freeport sells less shares, yet the variation of the amount available demonstrates to be contrary to Netflix model which sells similar amount of shares each year.
Netflix %>%
ggplot(aes(x=volume, y=adjclose, color="Monster")) +
geom_line() + theme_tufte() + labs(x="Volume", y="Stock Price",
title="Netflix Stock Price Against Amount of Stocks Sold")
As it can be seen in the line graph shown above, Netflix’s amount of stocks sold is compared with the price of the shares. It can be seen how there is a relationship between both variables in which as there are more shares introduced to the market, price fluctuations happen frequently. There can also be seen how there are certain volumes that make the price of the stock rise or fall. For example, as there were 15 million shares available, price declined sharply, thus indicating a probable effect of certain amounts sold to an increase or decrease of its prices.
Freeport%>%ggplot(aes(x=volume, y=adjclose, color="Closing")) +
geom_line() + theme_tufte() + labs(x="Volume", y="Stock Price",
title="Freeport Stock Price Against Amount of Stocks Sold")
In this graph, a similar trend can be seen for Freeport were from 10 to 20 million shares, there is constant fluctuation of prices, yet as the 20 million shares is surpassed, there can be seen a peak as it approaches 30 million. Then, it can be said how volume actually does play a role in the price of a share, yet different factors also affect the profitability and progress of a stock, volume is not a very good indicator for the prediction of a stock.
cor(WorstStocksSL$volume, WorstStocksSL$adjclose)## [1] -0.7011382WorstStocksSL %>% ggplot(aes(x=volume, y=adjclose))+geom_point()+geom_smooth(method="lm") + labs(x="Volume", y="Adjusted Closing Price", title="Volume Compared to Closing Price for Least Profitable Stocks")
As the least profitable stocks are compared to the amount of stocks these companies sell a correlation amongst both variables of -0.7011382 can be seen, which is a strong negative correlation. This means that as these companies increase the amount of shares that are available in the market, prices will decrease almost linearly. This probabily relates to their profitability as prices can be controlled with the amount of shares sold, maintaining them in a similar value with little variation.
cor(BestStocksSL$volume, BestStocksSL$adjclose)## [1] -0.271193BestStocksSL %>% ggplot(aes(x=volume, y=adjclose))+geom_point()+geom_smooth(method="lm") + labs(x="Volume", y="Adjusted Closing Price", title="Volume Compared to Closing Price for Most Profitable Stocks")
On the other hand, for the most profitable stocks, prove to have a correlation of -0.271193 which is a low negative correlation. This demonstrates how volume does not affect greatly the price of these companies’ stocks and rather their profitability may be affected by other variables.
Linear Model Evaluation of Price Against Volume
BestModel<- lm(data=BestStocksSL, adjclose~volume)
summary(BestModel)##
## Call:
## lm(formula = adjclose ~ volume, data = BestStocksSL)
##
## Residuals:
## Min 1Q Median 3Q Max
## -263.88 -226.31 94.42 139.19 313.23
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 351.724076484 17.673258005 19.901 < 0.0000000000000002 ***
## volume -0.000016301 0.000004012 -4.063 0.0000685 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 189.9 on 208 degrees of freedom
## Multiple R-squared: 0.07355, Adjusted R-squared: 0.06909
## F-statistic: 16.51 on 1 and 208 DF, p-value: 0.00006852In the model shown above, comparing the adjusted closing value to the volume of the selected best stocks, the previous claims of volume not affecting the price of these companies shares is supported. As it can be seen, the adjusted R squared value of the model is of 0.06909, meaning that only 6.9% of the price is affected by the volume of the stocks available for sale. With the p-value, we reject the null hypothesis and even though it is proven how volume does make changes in the price of the shares, the value obtained does not reflect a very strong relationship.
WorstModel<- lm(data=WorstStocksSL, adjclose~volume)
summary(WorstModel)##
## Call:
## lm(formula = adjclose ~ volume, data = WorstStocksSL)
##
## Residuals:
## Min 1Q Median 3Q Max
## -15.013 -4.755 1.222 4.716 29.491
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 32.70797131810 0.78248386746 41.80 <0.0000000000000002 ***
## volume -0.00000092130 0.00000006496 -14.18 <0.0000000000000002 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 6.692 on 208 degrees of freedom
## Multiple R-squared: 0.4916, Adjusted R-squared: 0.4892
## F-statistic: 201.1 on 1 and 208 DF, p-value: < 0.00000000000000022On the other hand, with the linear model for the bottom 5 stocks a stronger relationship with an R squared value of 0.4892, meaning that 48.9% of the variation in price for these stocks is caused by an increase or decrease of their volume. Additionally, with the p-value being so low and close to 0, we reject the null hypothesis and accept the alternate hypothesis proving how volume variation does have a significant effect on the price of a stock.
cor(fsNFLX$volume, fsNFLX$adjclose)## [1] -0.1499235fsNFLX %>% ggplot(aes(x=volume, y=adjclose))+geom_point()+geom_smooth(method="lm") + labs(x="Volume", y="Adj usted Closing Price", title="Volume Compared to Closing Price")
The comparison between the adjusted closing price against the amount of stocks for Netflix further demonstrates how the most profitable stocks are not really affected by the volume of stocks available. The correlation of both variables is -0.1499235, which even though it is a negative relationship, it is a weak one due to its closure to 0. This reflect the low variation that a change in the amount of shares available causes in these stocks.
cor(fsFCX$volume, fsFCX$adjclose)## [1] 0.5304194fsFCX %>% ggplot(aes(x=volume, y=adjclose))+geom_point()+geom_smooth(method="lm") + labs(x="Volume", y="Adj usted Closing Price", title="Volume Compared to Closing Price")
As mentioned previously, the least profitable stocks seem to be the ones who tend to be the most affected by changes in volume in terms of price. As it can be seen above theres a correlation between the adjusted closing price and volume of 0.5304194. This proves to be a somewhat strong relationship between both variables. Thus, as volume is altered for the Freeport stock, price would be affected directly by it, causing probable significant fluctuations.
BestStocks%>% ggcorr()
FinancialStock %>% dplyr::filter(Sector== "Financials") -> FSf3
FinancialStock %>% dplyr::filter(Sector== "Real Estate") -> FSf2
FinancialStock %>% dplyr::filter(Sector== "Information Technology") -> FSf
rbind(FSf, FSf2, FSf3)->SectorFilter
BestStocksModC<-lm(data=BestStocks, EBITDA~`Dividend Yield`)
summary(BestStocksModC)##
## Call:
## lm(formula = EBITDA ~ `Dividend Yield`, data = BestStocks)
##
## Residuals:
## 1 2 3 4 5
## 186083776 -234366224 -424248760 362225432 110305776
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 1043394224 213755278 4.881 0.0164 *
## `Dividend Yield` 149944124 193601086 0.775 0.4951
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 371000000 on 3 degrees of freedom
## Multiple R-squared: 0.1666, Adjusted R-squared: -0.1112
## F-statistic: 0.5999 on 1 and 3 DF, p-value: 0.4951BestStocksModCa<-lm(data=SectorFilter, Price~Sector)
summary(BestStocksModCa)##
## Call:
## lm(formula = Price ~ Sector, data = SectorFilter)
##
## Residuals:
## Min 1Q Median 3Q Max
## -108.02 -53.74 -24.66 18.42 888.47
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 89.0560 14.4436 6.166 0.00000000505 ***
## SectorInformation Technology 30.1868 20.2799 1.489 0.138
## SectorReal Estate -0.3433 25.2684 -0.014 0.989
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 119.1 on 168 degrees of freedom
## Multiple R-squared: 0.01568, Adjusted R-squared: 0.003959
## F-statistic: 1.338 on 2 and 168 DF, p-value: 0.2652As it can be seen on the linear model above which compares the sector of different companies with the price of the stocks, it is demonstrated how whether a company operates in a different economic sector, the price of their stocks is not affected by this. With a p-value, higher than 0.05, of 0.2652 the alternate hypothesis is rejected, thus it can be said that sector is not a factor playing a role in the prices of stocks. Additionally, the R squared value obtained is of 0.003959 meaning that 0.3% of the price variation is due to sector.
NetflixStockD$volume <- as.numeric(as.factor(NetflixStockD$volume))
t.test(NetflixStockD$close, NetflixStockD$volume)##
## Welch Two Sample t-test
##
## data: NetflixStockD$close and NetflixStockD$volume
## t = -111.15, df = 4478.7, p-value < 0.00000000000000022
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -2128.089 -2054.317
## sample estimates:
## mean of x mean of y
## 66.34234 2157.54525With the observed p value which is very small, the null hypothesis is rejected and the alternate hypothesis is proven right. Thus, it can be concluded that the amount of stocks that are sold each day from the company do play a role in the closing price of the stock.
ggplot(NetflixStockD,aes(x,close)) +
geom_line(aes(color="Netflix")) +
geom_line(data=FCXStockD,aes(color="Freeport")) +
labs(color="Legend") +
scale_colour_manual("", breaks = c("Netflix", "Freeport"),
values = c("blue", "green")) +
labs(title = "Stock Price History: Netflix & Freeport", x="Time", y="Price" ) +
theme(plot.title = element_text(lineheight=.7, face="bold"))
ggplot(FCXStockD,aes(x,close)) +
geom_line(aes(color="Freeport")) +labs(x="Time", y="Close", title = "Freeport Stock History")
Netflix_xts <- xts(NetflixStockD$close,order.by=NetflixStockD$x,frequency=365)
Freeport_xts <- xts(FCXStockD$close,order.by=FCXStockD$x,frequency=365)
stocks <- cbind(Netflix_xts,Freeport_xts)
dygraph(stocks,ylab="Close",
main="Stock Price History: Netflix & Freeport") %>%
dySeries(label="Netflix") %>%
dySeries(label="Freeport") %>%
dyOptions(colors = c("blue","brown")) %>%
dyRangeSelector()Price Forecast
weeks<- 1:nrow(NetflixStockD)
pricelm <- lm(NetflixStockD$close~weeks)
plot(NetflixStockD$close, type="l", col="blue", lwd=1, ylab="Closing Prices", main="Price", xlab = "Weeks")
Future <- data.frame(weeks = 4421:4435)
price_pred <- predict(pricelm, Future)
plot(price_pred)
The prediction plot that I made is a linear prediction which only goes up. The prediction period for the Netflix stock is for the next 14 weeks after the end of the dataset period. This prediction is not accurate due to a variety of factors. At first, a stock price is not linear as there are always variations in at least a minimum amount, thus having such a straight prediction is almost impossible. Additionally, the prediction did not reflect the current values of the Netflix stock, which currently is at 298 dollars per share. Thus, the method employed for the prediction model is not accurate.
Conclusion
In conclusion, throughout this project the planted research questions were responded with the use of different plots and tests. The volume provided by companies to the public did prove to affect the price of the shares in some cases such as how for the least profitable stocks, the volume of stocks available does play a role in the price of each share, causing it to rise or lower based on different amounts. On the other hand, for the stocks with most profitability, volume did not play a role for affecting the price. This could be concluded through the use of a linear model, along with graphs portraying the relationship between both variables or the R squared value. As it was mentioned, the most profitable (Netflix) and the least profitable one (Freeport) differ on different aspects such as their price movements throughout time, the variation of these prices, the amount of stocks provided by each company, their market capital, amongst other variables. In this project, what was mainly evaluated when comparing both of the stocks was the variation of the prices through time, such as how Netflix continuously increased its share prices, whilst Freeport maintained their prices in neither a rise or a decline, but rather small variations. Likewise, both companies behave differently with the amount of shares available for the public, such as how the prices vary with different quantities. Additionally, by using a linear model to compare price against the sector of the different companies, it was proven how different industries do not reflect any sort of variance in profitability or the performance of the stocks. Finally, the developed prediction model for the Netflix shares, did not accurately predict future prices for a few weeks into the future. This model did not work because it was a linear model reflecting progressive continuous point only going upwards and additionally did not show a related price prediction regarding the previous prices. The future direction I would like my project to take direction in is the evaluation of a fixed amount of stocks provided by companies in order to maintained the stock prices of a company controlled, and also to evaluate whether there is a reason to do so. Additionally, as I attemped to realize the predition model, I realized I lacked the skills of coding to develop an effective predictive model reflecting fluctuations and accuracy. In the future I would like to obtain the proper skills to realize such a model and actually realizing an accurate plot that could reflect future prices close to what they would actually come to be.
Reference
This article, published on online journal “Money”, talks about how and why most of the stock forecasting models are not accurate and most of the time do not work as planned. At first, the author talks about how the predicions made by analysts are subjective due to how they are passionate and confident in a stock. This demonstrates how even in stock market prediction, people tend to have a bias towards certain companies and expect their picks to be profitable, possibly leading to big losses when these forecasts do not work. Additionally, the article highlights how even a slight error in the model would lead to a failure in the accuracy of it, thus making a wrong prediction. At the end of the day, it is impossible to predict the future, thus a stock market forecast is only as good as an educated guess.
Source: https://money.com/why-stock-forecasts-are-often-off-target/