Giới thiệu
Principles of Economics là một cuốn sách kinh tế kinh điển của G. Mankiw và đây cũng là một cuốn giáo trình được sử dụng ở nhiều trường đại học kinh tế ở Việt Nam.
Bài viết này sẽ hướng dẫn cách đọc dữ liệu từ một file PDF đồng thời trả lời câu hỏi tác giả đã sử dụng bao nhiêu từ để viết nên cuốn sách cũng như liệt kê các từ xuất hiện với tần suất lớn hơn một ngưỡng nào đó trong sách.
Các thông tin có được có thể là hữu ích cho nhiều mục đích. Chẳng hạn, việc học từ vựng.
Trước hết các bạn load bản PDF của cuốn sách tại đây rồi để vào một folder có tên mankiw thuộc ổ E của máy tính.
Để R đọc được dữ liệu từ PDF các bạn cần có một số thao tác chuẩn bị. Trước hết các bạn download xpdfbin-win-3.04.zip tại đây.Giải nén ra và chọn hai file là pdftotext.exe và pdfinfo.exe rồi copy chúng để vào ổ E. Nhắc lại: copy và để vào ổ E.
Bước kế tiếp, kích chuột phải vào cửa sổ Start của window, chọn system, chọn tiếp Advanced system settings, chọn Advanced. Sau đó chọn Environment Variables, chọn Path, chọn Edit. Cuối cùng gõ chính xác đường dẫn của file zip vừa download về, trong tình huống máy tính của tôi, là C:10ls-win-3.0464 rồi kích OK.
Nếu máy của bạn là win32 thì thay bin64 thành bin32.
Các gói cần sử dụng là tm, SnowballC, wordcloud, RcolorBrewer, tidyverse.
# Load các gói:
pakg <- c("tm",
"SnowballC",
"wordcloud",
"RColorBrewer",
"tidyverse")
lapply(pakg, require, character.only = TRUE)## [[1]]
## [1] TRUE
##
## [[2]]
## [1] TRUE
##
## [[3]]
## [1] TRUE
##
## [[4]]
## [1] TRUE
##
## [[5]]
## [1] TRUE# Đọc dữ liệu từ file PDF:
setwd("E:/")
mk <- Corpus(DirSource("mankiw/", pattern = "pdf"),
readerControl = list(reader = readPDF))
# mk không phải là data frame quen thuộc:
inspect(mk)## <<VCorpus>>
## Metadata: corpus specific: 0, document level (indexed): 0
## Content: documents: 1
##
## [[1]]
## <<PlainTextDocument>>
## Metadata: 7
## Content: chars: 3027446#----------------------
# Xử lí data thô
#----------------------
# Chuyển hóa tất về chữ không in hoa:
mk <- mk %>%
tm_map(tolower)
# Xóa tất cả các loại dấu câu như chấm, phẩy:
mk <- mk %>% tm_map(removePunctuation)
# Bỏ các Stop Word trong tiếng Anh:
mk <- mk %>% tm_map(removeWords,stopwords("english"))
# Hoặc bỏ các từ như cengage, learning vì cengage learning là tên
# của nhà in. Sẽ không hợp lí nếu tính chúng vào phân tích:
mk <- mk %>% tm_map(removeWords, c("cengage", "learning"))
# Bỏ các con số:
mk <- mk %>% tm_map(removeNumbers)
# Một số xử lí khác:
mk <- mk %>% tm_map(stripWhitespace)
mk <- mk %>% tm_map(PlainTextDocument)
# Chuyển hóa về Document Matrix:
dtm <- TermDocumentMatrix(mk)
dtm ## <<TermDocumentMatrix (terms: 13498, documents: 1)>>
## Non-/sparse entries: 13498/0
## Sparsity : 0%
## Maximal term length: 257
## Weighting : term frequency (tf)m <- as.matrix(dtm)
v <- sort(rowSums(m), decreasing = TRUE)
d <- data.frame(word = names(v), freq = v)
# d là data frame quen thuộc:
str(d)## 'data.frame': 13498 obs. of 2 variables:
## $ word: Factor w/ 13498 levels "abajian","abandon",..: 9345 10366 2453 7442 3013 11789 11774 9678 7354 9775 ...
## $ freq: num 3217 2732 2677 2173 1954 ...# Số từ mà Mankiw sử dụng để viết sách:
nrow(d)## [1] 13498# 500 từ với tần suất xuất hiện lớn nhất:
head(d, 500)## word freq
## price price 3217
## rights rights 2732
## content content 2677
## may may 2173
## demand demand 1954
## suppressed suppressed 1772
## supply supply 1688
## quantity quantity 1670
## market market 1632
## rate rate 1522
## can can 1521
## curve curve 1508
## part part 1491
## tax tax 1469
## time time 1416
## cost cost 1307
## government government 1258
## money money 1251
## affect affect 1147
## economy economy 1137
## goods goods 1135
## income income 1120
## right right 1112
## reserves reserves 1050
## additional additional 1049
## inflation inflation 1016
## overall overall 1005
## third third 991
## review review 970
## whole whole 961
## experience experience 957
## firms firms 947
## one one 941
## require require 941
## due due 940
## restrictions restrictions 931
## party party 925
## copyright copyright 909
## reserved reserved 903
## subsequent subsequent 898
## deemed deemed 896
## electronic electronic 892
## remove remove 890
## ebook ebook 889
## editorial editorial 889
## andor andor 887
## echapters echapters 886
## materially materially 886
## copied copied 885
## duplicated duplicated 885
## scanned scanned 885
## good good 880
## total total 869
## interest interest 855
## economic economic 839
## chapter chapter 834
## will will 826
## percent percent 813
## policy policy 804
## people people 803
## level level 789
## marginal marginal 770
## prices prices 765
## two two 760
## trade trade 752
## increase increase 704
## production production 696
## many many 695
## labor labor 680
## firm firm 675
## gdp gdp 668
## workers workers 665
## unemployment unemployment 651
## figure figure 649
## real real 644
## equilibrium equilibrium 642
## capital capital 640
## less less 627
## services services 616
## markets markets 607
## revenue revenue 602
## consumer consumer 600
## change change 590
## costs costs 589
## much much 586
## higher higher 579
## surplus surplus 565
## value value 565
## aggregate aggregate 562
## new new 562
## output output 550
## economists economists 542
## run run 529
## average average 524
## example example 499
## also also 494
## shows shows 492
## per per 491
## taxes taxes 479
## pay pay 475
## use use 466
## might might 463
## changes changes 448
## case case 446
## make make 444
## buy buy 429
## point point 429
## amount amount 425
## investment investment 425
## theory theory 420
## economics economics 417
## saving saving 417
## long long 415
## year year 414
## now now 413
## growth growth 410
## study study 410
## profit profit 396
## effect effect 394
## world world 394
## spending spending 393
## explain explain 390
## number number 389
## public public 385
## rates rates 384
## first first 383
## wages wages 374
## competitive competitive 368
## shortrun shortrun 368
## funds funds 367
## states states 366
## shifts shifts 364
## rises rises 363
## buyers buyers 362
## net net 360
## consider consider 359
## exchange exchange 359
## high high 359
## increases increases 358
## consumers consumers 357
## united united 354
## bank bank 351
## wage wage 350
## countries countries 347
## produce produce 347
## sellers sellers 347
## thus thus 346
## monetary monetary 345
## demanded demanded 340
## system system 340
## person person 336
## lower lower 334
## budget budget 329
## fed fed 329
## result result 329
## work work 329
## suppose suppose 327
## longrun longrun 326
## resources resources 326
## nominal nominal 323
## years years 320
## monopoly monopoly 317
## large large 316
## consumption consumption 313
## product product 311
## way way 310
## called called 303
## effects effects 300
## supplied supplied 293
## even even 290
## financial financial 284
## better better 283
## see see 283
## important important 280
## stock stock 280
## exports exports 279
## natural natural 279
## like like 278
## another another 277
## dollar dollar 277
## different different 273
## federal federal 272
## reduce reduce 270
## table table 269
## falls falls 267
## domestic domestic 266
## elasticity elasticity 266
## curves curves 265
## must must 263
## country country 259
## given given 257
## panel panel 257
## fall fall 256
## shift shift 248
## national national 246
## often often 246
## whether whether 241
## well well 240
## analysis analysis 238
## three three 238
## yet yet 235
## policies policies 234
## producer producer 234
## banks banks 233
## policymakers policymakers 233
## low low 231
## society society 231
## university university 230
## problem problem 229
## rise rise 227
## greater greater 226
## just just 226
## short short 226
## among among 225
## however however 224
## loanable loanable 223
## college college 222
## raises raises 221
## raise raise 219
## benefits benefits 216
## take take 216
## without without 214
## equals equals 212
## measure measure 210
## want want 210
## dollars dollars 208
## data data 207
## behavior behavior 206
## private private 206
## used used 206
## sold sold 205
## produced produced 204
## show show 203
## aggregatesupply aggregatesupply 202
## living living 202
## makes makes 201
## reduces reduces 201
## give give 200
## quick quick 199
## small small 199
## nations nations 198
## households households 197
## therefore therefore 197
## index index 196
## competition competition 195
## variables variables 195
## job job 194
## choose choose 193
## second second 193
## deadweight deadweight 191
## laws laws 190
## worker worker 189
## according according 188
## free free 188
## using using 188
## answer answer 186
## future future 186
## percentage percentage 186
## economys economys 184
## find find 184
## following following 184
## foreign foreign 184
## sell sell 184
## choice choice 183
## paid paid 183
## producing producing 182
## need need 180
## problems problems 179
## best best 178
## oil oil 178
## measures measures 177
## social social 177
## decision decision 176
## get get 176
## international international 176
## risk risk 176
## variable variable 176
## welfare welfare 176
## fiscal fiscal 174
## population population 174
## decisions decisions 172
## efficient efficient 171
## loss loss 171
## benefit benefit 170
## business business 170
## phillips phillips 170
## fluctuations fluctuations 169
## lets lets 169
## question question 169
## equal equal 168
## larger larger 166
## reserve reserve 166
## advantage advantage 164
## industry industry 164
## key key 164
## million million 164
## central central 163
## opportunity opportunity 163
## zero zero 162
## economies economies 161
## every every 161
## jobs jobs 161
## model model 161
## various various 160
## deficit deficit 159
## outcome outcome 159
## poor poor 158
## rather rather 158
## size size 158
## sometimes sometimes 158
## fixed fixed 157
## likely likely 157
## common common 156
## times times 156
## information information 155
## productivity productivity 155
## left left 154
## company company 153
## cream cream 153
## indifference indifference 153
## insurance insurance 153
## american american 151
## aggregatedemand aggregatedemand 150
## area area 150
## causes causes 150
## efficiency efficiency 150
## power power 149
## always always 148
## depends depends 148
## fact fact 148
## ice ice 148
## life life 148
## outflow outflow 148
## questions questions 148
## cars cars 146
## making making 146
## currency currency 144
## balance balance 143
## governments governments 143
## keep keep 143
## contrast contrast 142
## help help 142
## negative negative 142
## discrimination discrimination 141
## unit unit 141
## pizza pizza 140
## pollution pollution 139
## today today 139
## billion billion 138
## instance instance 138
## minimum minimum 138
## open open 138
## reason reason 138
## bonds bonds 137
## decrease decrease 137
## poverty poverty 137
## possible possible 135
## next next 134
## influence influence 133
## turn turn 133
## ways ways 133
## expected expected 132
## gains gains 132
## imports imports 132
## political political 132
## source source 132
## producers producers 131
## economist economist 130
## willing willing 130
## available available 129
## burden burden 129
## account account 128
## book book 128
## employment employment 128
## factors factors 128
## state state 128
## affects affects 127
## assets assets 127
## earn earn 127
## words words 127
## measured measured 126
## news news 126
## buys buys 125
## cause cause 125
## incomes incomes 125
## law law 125
## made made 125
## spend spend 125
## graph graph 124
## relative relative 124
## ability ability 123
## produces produces 123
## purchases purchases 123
## rose rose 123
## around around 122
## based based 122
## discussed discussed 122
## elastic elastic 122
## ratio ratio 122
## hold hold 121
## icecream icecream 121
## means means 121
## quiz quiz 121
## similar similar 121
## wellbeing wellbeing 121
## although although 120
## happens happens 120
## health health 120
## positive positive 120
## potatoes potatoes 120
## reducing reducing 120
## water water 120
## car car 118
## education education 118
## slope slope 118
## takes takes 118
## tradeoff tradeoff 118
## pepsi pepsi 117
## principle principle 117
## products products 117
## recession recession 117
## four four 116
## particular particular 116
## the the 116
## together together 116
## diagram diagram 115
## president president 115
## purchase purchase 115
## standard standard 115
## understand understand 115
## losses losses 114
## distribution distribution 113
## gets gets 113
## single single 113
## decide decide 112
## substitution substitution 112
## true true 112
## end end 111
## hour hour 111
## impact impact 111
## increased increased 111
## deposits deposits 110
## respond respond 110
## textiles textiles 110
## become become 109
## constraint constraint 109
## customers customers 109
## externalities externalities 109
## human human 109
## inequality inequality 109
## abroad abroad 108
## food food 108
## incentive incentive 108
## return return 108
## said said 108
## things things 108
## role role 107
## think think 107
## conclusion conclusion 106
## families families 106
## debt debt 105
## goal goal 105
## individual individual 105
## macroeconomic macroeconomic 105
## macroeconomics macroeconomics 105
## pays pays 105
## shown shown 105
## smaller smaller 105
## determine determine 104
## force force 104
## day day 103
## land land 103
## last last 103
## process process 103
## profits profits 103
## receive receive 103
## exactly exactly 102
## group group 102
## period period 102
## principles principles 102
## relationship relationship 102
## typical typical 102
## view view 102
## control control 101
## hand hand 101
## hours hours 101
## increasing increasing 101
## outcomes outcomes 101
## perfect perfect 101
## provide provide 101
## related related 101
## companies companies 100
## everyone everyone 100
## examine examine 100
## set set 100
## technology technology 100
## buying buying 99
## difference difference 99
## gasoline gasoline 99
## instead instead 99# Các từ xuất hiện với tần suất hơn 400 lần:
u <- findFreqTerms(dtm, low = 400)
head(u)## [1] "additional" "affect" "aggregate" "also" "amount"
## [6] "andor"tail(u)## [1] "use" "value" "whole" "will" "workers" "year"# Có 117 từ như thế:
length(u)## [1] 117# Chúng là:
head(d, length(u))## word freq
## price price 3217
## rights rights 2732
## content content 2677
## may may 2173
## demand demand 1954
## suppressed suppressed 1772
## supply supply 1688
## quantity quantity 1670
## market market 1632
## rate rate 1522
## can can 1521
## curve curve 1508
## part part 1491
## tax tax 1469
## time time 1416
## cost cost 1307
## government government 1258
## money money 1251
## affect affect 1147
## economy economy 1137
## goods goods 1135
## income income 1120
## right right 1112
## reserves reserves 1050
## additional additional 1049
## inflation inflation 1016
## overall overall 1005
## third third 991
## review review 970
## whole whole 961
## experience experience 957
## firms firms 947
## one one 941
## require require 941
## due due 940
## restrictions restrictions 931
## party party 925
## copyright copyright 909
## reserved reserved 903
## subsequent subsequent 898
## deemed deemed 896
## electronic electronic 892
## remove remove 890
## ebook ebook 889
## editorial editorial 889
## andor andor 887
## echapters echapters 886
## materially materially 886
## copied copied 885
## duplicated duplicated 885
## scanned scanned 885
## good good 880
## total total 869
## interest interest 855
## economic economic 839
## chapter chapter 834
## will will 826
## percent percent 813
## policy policy 804
## people people 803
## level level 789
## marginal marginal 770
## prices prices 765
## two two 760
## trade trade 752
## increase increase 704
## production production 696
## many many 695
## labor labor 680
## firm firm 675
## gdp gdp 668
## workers workers 665
## unemployment unemployment 651
## figure figure 649
## real real 644
## equilibrium equilibrium 642
## capital capital 640
## less less 627
## services services 616
## markets markets 607
## revenue revenue 602
## consumer consumer 600
## change change 590
## costs costs 589
## much much 586
## higher higher 579
## surplus surplus 565
## value value 565
## aggregate aggregate 562
## new new 562
## output output 550
## economists economists 542
## run run 529
## average average 524
## example example 499
## also also 494
## shows shows 492
## per per 491
## taxes taxes 479
## pay pay 475
## use use 466
## might might 463
## changes changes 448
## case case 446
## make make 444
## buy buy 429
## point point 429
## amount amount 425
## investment investment 425
## theory theory 420
## economics economics 417
## saving saving 417
## long long 415
## year year 414
## now now 413
## growth growth 410
## study study 410# Vẽ đám mây từ:
par(bg = "black")
set.seed(1709)
wordcloud(words = d$word,
freq = d$freq,
# Chỉ hiện thị từ nào xuất hiện ít nhất 100 lần:
min.freq = 100,
# Số từ hiển thị trên wordcloud tối đa là 200:
max.words = 500,
random.order = FALSE,
# 35% số từ được hiển thị theo chiều thẳng đứng
rot.per = 0.35,
# Chọn kích cỡ chữ:
font = 2,
# Tô màu cho chữ:
colors = brewer.pal(8, "Dark2"))
# 30 từ xuất hiện nhiều nhất trong sách:
d %>% slice(1:30) %>%
ggplot(aes(reorder(word, freq), freq)) +
geom_bar(stat = "identity") +
coord_flip() +
labs(x = NULL,
y = "Frequency of words")
# Histogram về tần suất xuất hiện của các từ:
par(bg = "white")
hist(d$freq,
breaks = 100,
xlab = "",
ylab = "",
ylim = c(0, 14000))
# Nhận xét: phân phối này rất lệch. Nên có vẻ như chúng ta chỉ cần học
# 1000 từ với tần suất nhiều nhất là có thể đọc thông thạo cuốn sách này.
# Số từ xuất hiện nhiều hơn 10 lần:
nrow(d %>% filter(freq >= 10))## [1] 2948