DH_B: Lecture03 (Fall 2021)

Lecture3: 関数の作成

Warmup Practice

作業ディレクトリの設定

setwd("/cloud/project")

作業ディレクトリの確認

getwd()

[1] "/cloud/project"

準備：単語リスト

txt<-readLines("sample_texts/sample_en.txt")
wordLst<-strsplit(txt,"[[:space:]]|[[:punct:]]")
wordLst<-unlist(wordLst)
wordLst<-tolower(wordLst)
wordLst<- wordLst[wordLst != ""]

前回の補足

strsplit(txt," |[.,!?:...]")

[[1]]
 [1] "COVID-19"    "is"          "an"          "infectious"  "disease"     "caused"     
 [7] "by"          "a"           "coronavirus" "called"      "SARS-CoV-2"  ""           

[[2]]
 [1] "It"       "mainly"   "causes"   "symptoms" "such"     "as"       "fever"    "and/or"  
 [9] "cough"    ""        

[[3]]
 [1] "In"           "general"      ""             "it"           "is"           "spread"      
 [7] "through"      "droplet"      "and"          "contact"      "transmission" ""            

[[4]]
 [1] "It"       "has"      "been"     "pointed"  "out"      "that"     "it"       "may"     
 [9] "spread"   "before"   "symptoms" "appear"   ""        

[[5]]
 [1] "It"         "is"         "therefore"  "important"  "to"         "habitually" "follow"    
 [8] "the"        "general"    "strategies" "for"        "preventing" "infectious" "diseases"  
[15] ""           "such"       "as"         "social"     "distancing" "and"        "wearing"   
[22] "a"          "mask"       "when"       "in"         "public"    

結果部分出力

head(wordLst)

[1] "covid"      "19"         "is"         "an"         "infectious" "disease"   

単語のToken数

tokens <- length(wordLst)

単語のTypes数

• unique()関数は，リストの重複しない要素を返す

types <- length(unique(wordLst))

結果出力

print(paste("The number of tokens: ", tokens))

[1] "The number of tokens:  71"

print(paste("The number of types: ", types))

[1] "The number of types:  55"

関数の作成

TTR

返り（戻り）値なし関数：printTTR関数を作成

引数: 単語リスト

結果出力: TTRの結果

printTTR<- function(wLst) {
    num_tokens <- length(wLst)
    num_types <- length(unique(wLst))
    res_TTR = num_types/num_tokens * 100
    paste("TTR =", res_TTR)
}

printTTR関数の実行

printTTR(wordLst)

[1] "TTR = 77.4647887323944"

返り（戻り）値あり関数：calcTTR1関数を作成

引数: 単語リスト

戻り値: TTR計算値

calcTTR1<- function(wLst) {
    num_tokens <- length(wLst)
    num_types <- length(unique(wLst))
    res_TTR = num_types/num_tokens * 100
    return(res_TTR)
}

calcTTR1関数の実行

calcTTR1(wordLst)

[1] 77.46479

戻り値の利用

res <- calcTTR1(wordLst)
round(res,1)

[1] 77.5

返り（戻り）値あり関数：calcTTR2関数を作成

引数: 単語リスト

戻り値: TTR計算値

calcTTR2<- function(arg_tokens,arg_types) {
    res_TTR <- arg_types/arg_tokens * 100
    return(res_TTR)
}

calcTTR2関数の実行

calcTTR2(tokens,types)

[1] 77.46479

関数の読み込み

calcTTR3(wordLst)

[1] 77.46479

練習: RTTR(Root Type-Token Ratio) Giraudの値を計算する関数calcRTTRの作成

引数: 英文テキストファイル

戻り値: Giraud計算値

calcRTTR関数の実行

calcRTTR("sample_texts/sample_en.txt")

[1] 6.527299

Word Frequencies

(freq <- table(wordLst))

wordLst
          19            2            a           an          and       appear           as 
           1            1            2            1            3            1            2 
        been       before           by       called       caused       causes      contact 
           1            1            1            1            1            1            1 
 coronavirus        cough          cov        covid      disease     diseases   distancing 
           1            1            1            1            1            1            1 
     droplet        fever       follow          for      general   habitually          has 
           1            1            1            1            2            1            1 
   important           in   infectious           is           it       mainly         mask 
           1            2            2            3            5            1            1 
         may           or          out      pointed   preventing       public         sars 
           1            1            1            1            1            1            1 
      social       spread   strategies         such     symptoms         that          the 
           1            2            1            2            2            1            1 
   therefore      through           to transmission      wearing         when 
           1            1            1            1            1            1 

Sort

(freq_data<-sort(freq, decreasing=TRUE))

wordLst
          it          and           is            a           as      general           in 
           5            3            3            2            2            2            2 
  infectious       spread         such     symptoms           19            2           an 
           2            2            2            2            1            1            1 
      appear         been       before           by       called       caused       causes 
           1            1            1            1            1            1            1 
     contact  coronavirus        cough          cov        covid      disease     diseases 
           1            1            1            1            1            1            1 
  distancing      droplet        fever       follow          for   habitually          has 
           1            1            1            1            1            1            1 
   important       mainly         mask          may           or          out      pointed 
           1            1            1            1            1            1            1 
  preventing       public         sars       social   strategies         that          the 
           1            1            1            1            1            1            1 
   therefore      through           to transmission      wearing         when 
           1            1            1            1            1            1 

頻度テーブルをデータ型に変換

freqData <- data.frame(freq_data)
freqData

相対頻度テーブル

(relative<-sort(freq/tokens, decreasing=TRUE))

wordLst
          it          and           is            a           as      general           in 
  0.07042254   0.04225352   0.04225352   0.02816901   0.02816901   0.02816901   0.02816901 
  infectious       spread         such     symptoms           19            2           an 
  0.02816901   0.02816901   0.02816901   0.02816901   0.01408451   0.01408451   0.01408451 
      appear         been       before           by       called       caused       causes 
  0.01408451   0.01408451   0.01408451   0.01408451   0.01408451   0.01408451   0.01408451 
     contact  coronavirus        cough          cov        covid      disease     diseases 
  0.01408451   0.01408451   0.01408451   0.01408451   0.01408451   0.01408451   0.01408451 
  distancing      droplet        fever       follow          for   habitually          has 
  0.01408451   0.01408451   0.01408451   0.01408451   0.01408451   0.01408451   0.01408451 
   important       mainly         mask          may           or          out      pointed 
  0.01408451   0.01408451   0.01408451   0.01408451   0.01408451   0.01408451   0.01408451 
  preventing       public         sars       social   strategies         that          the 
  0.01408451   0.01408451   0.01408451   0.01408451   0.01408451   0.01408451   0.01408451 
   therefore      through           to transmission      wearing         when 
  0.01408451   0.01408451   0.01408451   0.01408451   0.01408451   0.01408451 

相対頻度テーブルをデータ型に変換

２つのデータ型変数を連結(merge)

freqMtx <- merge(freqData, relativeData, all=T, by="wordLst")

freqMtx[,1]

 [1] 19           2            a            an           and          appear       as          
 [8] been         before       by           called       caused       causes       contact     
[15] coronavirus  cough        cov          covid        disease      diseases     distancing  
[22] droplet      fever        follow       for          general      habitually   has         
[29] important    in           infectious   is           it           mainly       mask        
[36] may          or           out          pointed      preventing   public       sars        
[43] social       spread       strategies   such         symptoms     that         the         
[50] therefore    through      to           transmission wearing      when        
55 Levels: it and is a as general in infectious spread such symptoms 19 2 an appear been ... when

出現単語の情報を行ラベルにコピー

rownames(freqMtx)<-as.character(freqMtx[,1])

Warning: non-unique values when setting 'row.names': ‘1’, ‘2’, ‘3’
Error in `.rowNamesDF<-`(x, value = value) : 
  duplicate 'row.names' are not allowed

出現単語の情報(1列目)を削除

freqMtx<-freqMtx[-1]
colnames(freqMtx) <- c("raw", "relative")

粗頻度-相対頻度の行列