Stock index comparsion
Stock index comparsion
jim lung
05-10-2017
In 2017, some financial companies uses advanced algorithms from data analystic to conduct 100% automatic finacial trading instead of human trading and analysis. I think this project is a good opportunity to help me figure out the relationship between different stock wave.
How the wave of Nasdaq index are affected by Amazon and Facebook?
To explore how the wave of Nasdaq index are affected by Amazon and Facebook, I want to break down to different topics about the financial markets.
The statistics of daily historical stock prices & volumes for one year under this category, I will compare from Nasdaq index, Amazon and Facebooks.
The mainly resource I will use is http://www.nasdaq.com
Both for short term and long term analysis, the related stock news will be used for analysis by sentiments in order to match for complexity.
Data scraping
Data sentiments
Scrapying method 1 - Using quantmod
## Loading required package: xts## Warning: package 'xts' was built under R version 3.3.3## Loading required package: zoo## Warning: package 'zoo' was built under R version 3.3.3##
## Attaching package: 'zoo'## The following objects are masked from 'package:base':
##
## as.Date, as.Date.numeric## Loading required package: TTR## Version 0.4-0 included new data defaults. See ?getSymbols.## As of 0.4-0, 'getSymbols' uses env=parent.frame() and
## auto.assign=TRUE by default.
##
## This behavior will be phased out in 0.5-0 when the call will
## default to use auto.assign=FALSE. getOption("getSymbols.env") and
## getOptions("getSymbols.auto.assign") are now checked for alternate defaults
##
## This message is shown once per session and may be disabled by setting
## options("getSymbols.warning4.0"=FALSE). See ?getSymbols for more details.## [1] "YHOO"Scraping method 2 - Loading CSV
## date close volume open high low
## 1 16:00 952.95 3,171,374 952.80 957.89 950.200
## 2 2017/05/09 952.82 3256357.0000 952.80 957.89 950.200
## 3 2017/05/08 949.04 3406954.0000 940.95 949.05 939.210
## 4 2017/05/05 934.15 2863978.0000 940.52 940.79 930.300
## 5 2017/05/04 937.53 2413486.0000 944.75 945.00 934.215
## 6 2017/05/03 941.03 3578108.0000 946.00 946.00 935.900Scraping method 3: To compare with the above method for scaping data, the method 3 can be retreive the most updated data for comparison, it may get the real time data, so we will use the method 3 for data analysis.
ddd <- function(i){
hkurl <- i
'ratings' <- function(hkurl)
{
require(XML)
hdoc <- getURLContent(hkurl)
# find all tables in webpage
hktables <- readHTMLTable(hdoc)
# find largest table and return as dataframe
columns <- unlist(lapply(hktables, function(t) dim(t)[2]))
df <- hktables[[max(columns)]]
return(df)
}
seriess <- ratings(hkurl)
colnames(seriess) <- c("Date","Open","High","Low","Close","Volume")
seriess
return(seriess)
}\[Range\quad of\quad daily\quad change\quad =\quad Close\quad -\quad Open\]
\[Rating\_ change\quad =\quad \frac { Close\quad -\quad Open }{ Open } \]
Dataset Example of Facebook:
Date Open High Low Close Volume change rating_change date_range
1 05/10/2017 151.49 152.59 150.21 150.48 16999575 -1.01 -0.006667107 2.38
2 05/09/2017 151.49 152.59 150.21 150.48 17381800 -1.01 -0.006667107 2.38
3 05/08/2017 150.71 151.08 149.74 151.06 15813350 0.35 0.002322341 1.34
4 05/05/2017 151.45 151.63 149.79 150.24 17104730 -1.21 -0.007989435 1.84
5 05/04/2017 150.17 151.52 148.72 150.85 36185180 0.68 0.004528201 2.80
6 05/03/2017 153.60 153.60 151.34 151.80 28301550 -1.80 -0.011718750 2.26
Stock index comparsion
Nasdaq and amazon
Nasdaq and facebook
Stock rating comparsion
range of stock change
The wave of facebook and nasdaq seem to be more similar than amazon.
Sentiment analysis from scraping most recent article about Amazon, Facebook and Nasdaq.
Amazon’s article
Each pages has 14 articles related to Amazon, that mean scraping this 14 articles equal to the most updated information about amazon.
web <- function(i){
weblinks <- read_html(paste('https://www.nasdaq.com/symbol/',i,'/news-headlines',sep=""))
doc <- htmlParse(weblinks)
links <- xpathSApply(doc, "//a/@href")
#http://www.nasdaq.com/article/
webs <-unlist(str_extract_all(links, 'http://www.nasdaq.com/article.+'))
webs
}[1] “http://www.nasdaq.com/article/4-stocks-im-never-selling-cm786315”
[2] “http://www.nasdaq.com/article/4-stocks-im-never-selling-cm786315”
[3] “http://www.nasdaq.com/article/the-first-thing-you-should-do-with-your-social-security-check-cm786333”
[4] “http://www.nasdaq.com/article/the-first-thing-you-should-do-with-your-social-security-check-cm786333”
[7] “http://www.nasdaq.com/article/key-takeaways-from-ices-q1-earnings-cm784490”
[8] “http://www.nasdaq.com/article/a-drop-in-trading-volume-curbs-nasdaqs-results-cm783594”
[9] “http://www.nasdaq.com/article/intercontinental-exchange-ice-beats-q1-earnings-estimates-cm783595”
[10] “http://www.nasdaq.com/article/ice-allots-10-mln-for-regulatory-probes-earnings-rise-20170503-01190”
[11] “http://www.nasdaq.com/article/key-takeaways-from-cmes-q1-earnings-cm781544”
[13] “http://www.nasdaq.com/article/key-takeaways-from-nasdaqs-q1-earnings-cm780749”
[14] “http://www.nasdaq.com/article/cme-group-cme-q1-earnings-beat-revenues-miss-estimates-cm780387"rr
A tibble: 252,936 ⊙ 3 company article word
1 amazon 1 doctype
2 amazon 1 html
3 amazon 1 html
4 amazon 1 lang
5 amazon 1 en
6 amazon 1 us
7 amazon 1 class
8 amazon 1 inner
9 amazon 1 news
10 amazon 1 story … with 252,926 more rows
A tibble: 251,425 ⊙ 3
company article word
<fctr> <int> <chr>1 facebook 1 doctype
2 facebook 1 html
3 facebook 1 html
A tibble: 245,628 ⊙ 3
company article word
1 nasdaq 1 doctype
2 nasdaq 1 html
3 nasdaq 1 html
Amazon:
A tibble: 10 ⊙ 2
sentiment n
<chr> <int>
1 positive 11885
2 trust 6574
3 negative 6104
4 anticipation 3104
5 sadness 2849
6 fear 2630
7 joy 2560
8 anger 2335
9 surprise 1870
10 disgust 1457
Facebook:
A tibble: 10 ⊙ 2
sentiment n
<chr> <int>
1 positive 11704
2 trust 6454
3 negative 5988
4 anticipation 3008
5 sadness 2798
6 fear 2597
7 joy 2511
8 anger 2309
9 surprise 1830
10 disgust 1476
Nasdaq:
A tibble: 10 ⊙ 2
sentiment n
<chr> <int>
1 positive 11691
2 trust 6403
3 negative 5941
4 anticipation 3007
5 sadness 2775
6 fear 2604
7 joy 2465
8 anger 2276
9 surprise 1821
10 disgust 1400
By the comparsion with index and sentiment for Amazon, Facebook and Nasdaq, the wave for facebook and Nasdaq are more similar and close relationship than comparing with Amazon. That mean the predication of Nasdaq may be more attend to Facebook, not amazon.