First of all, let’s find expected frequencies of words.
library(tidyverse)
## Warning: пакет 'tidyverse' был собран под R версии 4.1.3
## -- Attaching packages --------------------------------------- tidyverse 1.3.1 --
## v ggplot2 3.3.5 v purrr 0.3.4
## v tibble 3.2.1 v dplyr 1.1.2
## v tidyr 1.1.4 v stringr 1.4.0
## v readr 2.1.1 v forcats 0.5.1
## Warning: пакет 'tibble' был собран под R версии 4.1.3
## Warning: пакет 'dplyr' был собран под R версии 4.1.3
## -- Conflicts ------------------------------------------ tidyverse_conflicts() --
## x dplyr::filter() masks stats::filter()
## x dplyr::lag() masks stats::lag()
library(dplyr)
freq_rnc <- read_tsv("https://raw.githubusercontent.com/LingData2019/LingData/master/data/freq_rnc_ranked.csv")
## Rows: 52138 Columns: 7
## -- Column specification --------------------------------------------------------
## Delimiter: "\t"
## chr (2): Lemma, PoS
## dbl (5): Rank, Freq(ipm), R, D, Doc
##
## i Use `spec()` to retrieve the full column specification for this data.
## i Specify the column types or set `show_col_types = FALSE` to quiet this message.
freq_rnc
## # A tibble: 52,138 x 7
## Rank Lemma PoS `Freq(ipm)` R D Doc
## <dbl> <chr> <chr> <dbl> <dbl> <dbl> <dbl>
## 1 1 и conj 35802. 100 99 37704
## 2 2 в pr 31374. 100 98 37865
## 3 3 не part 18028 100 97 33999
## 4 4 на pr 15867. 100 98 36748
## 5 5 я spro 12684. 100 95 17116
## 6 6 быть v 12161. 100 98 34184
## 7 7 он spro 11791. 100 95 28132
## 8 8 с pr 11312. 100 99 35700
## 9 9 что conj 8354 100 98 32419
## 10 10 а conj 8198 100 97 32332
## # i 52,128 more rows
all_freqs <- freq_rnc$`Freq(ipm)`
all_freqs[1:10]
## [1] 35801.8 31374.2 18028.0 15867.3 12684.4 12160.7 11791.1 11311.9 8354.0
## [10] 8198.0
Now let’s take out the rows that contain verbs:
## # A tibble: 12,344 x 7
## Rank Lemma PoS `Freq(ipm)` R D Doc
## <dbl> <chr> <chr> <dbl> <dbl> <dbl> <dbl>
## 1 6 быть v 12161. 100 98 34184
## 2 37 мочь v 2912. 100 98 25413
## 3 42 сказать v 2397. 100 94 15426
## 4 58 говорить v 1755 100 96 17120
## 5 61 знать v 1714. 100 95 13494
## 6 62 стать v 1622. 100 97 21235
## 7 92 хотеть v 991. 100 95 12290
## 8 95 идти v 957. 100 96 13815
## 9 100 иметь v 907. 100 96 17493
## 10 114 видеть v 818. 100 96 9914
## # i 12,334 more rows
And now let’s summarize the frequencies of all verbs and divide them into the sum of frequencies of the whole dictionary:
## [1] 158043.6
## [1] 955648.2
## [1] 0.1653784
The null hypothesis will be the following = the probability of verbs in the corpus of verses from 1820s won’t be different from the dictionary. The alternative hypothesis will be the following = the probability of verbs in the corpus of verses from 1820s will be different from the dictionary.
Now let’s open the data set of verses and filter it by decade (we’re interested in 1820s)
## Rows: 364 Columns: 6
## -- Column specification --------------------------------------------------------
## Delimiter: "\t"
## chr (3): UPoS, LineText, Author
## dbl (3): Decade, RhymedNwords, RhymedNsyl
##
## i Use `spec()` to retrieve the full column specification for this data.
## i Specify the column types or set `show_col_types = FALSE` to quiet this message.
## # A tibble: 364 x 6
## Decade RhymedNwords RhymedNsyl UPoS LineText Author
## <dbl> <dbl> <dbl> <chr> <chr> <chr>
## 1 1820 1 1 NOUN «Промчалась туча грозных бед, В. Г.~
## 2 1820 1 1 NOUN Вечерний звон, вечерний звон! И. И.~
## 3 1820 1 1 NOUN Давно ли жизнию кипела в сердце ~ В. Н.~
## 4 1820 1 1 NOUN Наш славный Владимир, наш солныш~ П. А.~
## 5 1820 1 1 PRON Не привлекаю девы я, А. Ф.~
## 6 1820 1 1 PRON Сначала жизнь пленяет нас: Д. В.~
## 7 1820 1 1 PRON Ты в Петербурге, ты со мной, А. А.~
## 8 1820 1 2 ADJ Голов сто, мнений сто; год новый~ А. Г.~
## 9 1820 1 2 ADJ Девица юная подобна розе нежной, К. Н.~
## 10 1820 1 2 ADJ Отчего, мой ангел милый, И. П.~
## # i 354 more rows
## # A tibble: 87 x 6
## Decade RhymedNwords RhymedNsyl UPoS LineText Author
## <dbl> <dbl> <dbl> <chr> <chr> <chr>
## 1 1820 1 1 NOUN «Промчалась туча грозных бед, В. Г.~
## 2 1820 1 1 NOUN Вечерний звон, вечерний звон! И. И.~
## 3 1820 1 1 NOUN Давно ли жизнию кипела в сердце ~ В. Н.~
## 4 1820 1 1 NOUN Наш славный Владимир, наш солныш~ П. А.~
## 5 1820 1 1 PRON Не привлекаю девы я, А. Ф.~
## 6 1820 1 1 PRON Сначала жизнь пленяет нас: Д. В.~
## 7 1820 1 1 PRON Ты в Петербурге, ты со мной, А. А.~
## 8 1820 1 2 ADJ Голов сто, мнений сто; год новый~ А. Г.~
## 9 1820 1 2 ADJ Девица юная подобна розе нежной, К. Н.~
## 10 1820 1 2 ADJ Отчего, мой ангел милый, И. П.~
## # i 77 more rows
The number of rows for 1820 is the following:
## [1] 87
The number of rows for verbs in 1820s is the following:
## [1] 15
Now let’s count the p-value
##
## Exact binomial test
##
## data: number_1820s_verbs and number_1820s
## number of successes = 15, number of trials = 87, p-value = 0.885
## alternative hypothesis: true probability of success is not equal to 0.1653784
## 95 percent confidence interval:
## 0.09983452 0.26837791
## sample estimates:
## probability of success
## 0.1724138
The p-value is too big, so we cannot reject the null hypothesis.
First let’s determine the samples:
## [1] 1 5 4 3 5
## [1] 4 30 20 5 10 50
Here are the means for two samples:
## [1] 3.6
## [1] 19.83333
According to my intuition, the means are significantly different, so I assume that p-value will be noticeably low.
Now let’s run the T-test on two our samples:
##
## Welch Two Sample t-test
##
## data: sample1 and sample2
## t = -2.2245, df = 5.1061, p-value = 0.07558
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -34.875664 2.408998
## sample estimates:
## mean of x mean of y
## 3.60000 19.83333
The p-value is higher than 0.05. Here I would rather choose null hypothesis. I would say that the data in these samples is close to random.
##
## Welch Two Sample t-test
##
## data: sample1_mod_1 and sample2_mod_1
## t = -3.5948, df = 5.1061, p-value = 0.01508
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -44.875664 -7.591002
## sample estimates:
## mean of x mean of y
## 3.60000 29.83333
Increase of mean difference influences p-value - it is decreasing, which means that the sample is becoming less random. It makes sense, because t-test literally shows the difference between two samples (how random they are).
##
## Welch Two Sample t-test
##
## data: sample1_mod_2 and sample2_mod_2
## t = -2.3031, df = 5.3336, p-value = 0.06624
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -20.0467040 0.9133706
## sample estimates:
## mean of x mean of y
## 3.60000 13.16667
Decrease of mean difference influences p-value - it is increasing, which means that the sample is becoming more random. The data is becoming more similar and now it looks like it was taken randomly.
##
## Welch Two Sample t-test
##
## data: sample1_mod_3 and sample2_mod_3
## t = -1.7901, df = 3.2184, p-value = 0.1651
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -30.955494 8.122161
## sample estimates:
## mean of x mean of y
## 3.333333 14.750000
Decrease of sample size influences p-value - it is increasing, which means that the sample is becoming more random. The less data we have, the more similar it is, the less differences we can find among the values.
## [1] 3.6
## [1] 19.7
##
## Welch Two Sample t-test
##
## data: sample1_mod_4 and sample2_mod_4
## t = -3.5017, df = 9.2727, p-value = 0.006404
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -26.454289 -5.745711
## sample estimates:
## mean of x mean of y
## 3.6 19.7
Here I tried to extend samples while having the same mean. The p-value became noticeably decreased, which means that the sample is becoming less random. The more data we have, the less similar it is, the more differences we can find among the values.
## [1] 3.8
## [1] 19.66667
##
## Welch Two Sample t-test
##
## data: sample1_mod_5 and sample2_mod_5
## t = -31.663, df = 8.5569, p-value = 3.611e-10
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -17.00927 -14.72407
## sample estimates:
## mean of x mean of y
## 3.80000 19.66667
Here I tried to decrease sample diversity while having the same mean. The p-value drastically decreased, which means that the more similar data is inside of the samples, the more likely it’s not random.
## [1] 3.6
## [1] 19.66667
##
## Welch Two Sample t-test
##
## data: sample1_mod_6 and sample2_mod_6
## t = -1.4761, df = 5.4879, p-value = 0.1949
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -43.31412 11.18079
## sample estimates:
## mean of x mean of y
## 3.60000 19.66667
Here I tried to increase sample diversity while having the same mean. The p-value increased, which means that the less similar data is inside of the samples, the more likely it’s random.
Now let’s run the T-test on two our samples with alternative “less”:
##
## Welch Two Sample t-test
##
## data: sample1 and sample2
## t = -2.2245, df = 5.1061, p-value = 0.03779
## alternative hypothesis: true difference in means is less than 0
## 95 percent confidence interval:
## -Inf -1.595627
## sample estimates:
## mean of x mean of y
## 3.60000 19.83333
Here I would reject null hypothesis and choose alternative one. It makes sense because the mean of the first sample is less than the second one. It means that the drugs shorten the length of illness, which means that the drug seems to be effective. Though, if I’m not mistaken, in medicine way smaller p-values are used.
Now let’s run the T-test on two our samples with alternative “greater”:
##
## Welch Two Sample t-test
##
## data: sample1 and sample2
## t = -2.2245, df = 5.1061, p-value = 0.9622
## alternative hypothesis: true difference in means is greater than 0
## 95 percent confidence interval:
## -30.87104 Inf
## sample estimates:
## mean of x mean of y
## 3.60000 19.83333
Here the p-value is close to 1, which means that no effect is observed. It also makes sense, because the values of the first sample do not spread on the right side of the graph. The results do not mean that the illness is going to last longer for those who use drugs, it only means that it’s not going to last shorter.
## value sample_id
## 1 1 1
## 2 5 1
## 3 4 1
## 4 3 1
## 5 5 1
## 6 4 2
## 7 30 2
## 8 20 2
## 9 5 2
## 10 10 2
## 11 50 2
##
## Welch Two Sample t-test
##
## data: value by sample_id
## t = -2.2245, df = 5.1061, p-value = 0.07558
## alternative hypothesis: true difference in means between group 1 and group 2 is not equal to 0
## 95 percent confidence interval:
## -34.875664 2.408998
## sample estimates:
## mean in group 1 mean in group 2
## 3.60000 19.83333
Let’s open the data set.
## New names:
## Rows: 832 Columns: 8
## -- Column specification
## -------------------------------------------------------- Delimiter: "," chr
## (4): Lemma, Participant, Cond, Exponent dbl (4): ...1, LogDurationA,
## LogDurationW, Frequency
## i Use `spec()` to retrieve the full column specification for this data. i
## Specify the column types or set `show_col_types = FALSE` to quiet this message.
## * `` -> `...1`
## # A tibble: 832 x 8
## ...1 LogDurationA LogDurationW Lemma Participant Cond Exponent Frequency
## <dbl> <dbl> <dbl> <chr> <chr> <chr> <chr> <dbl>
## 1 11431 -1.32 -0.477 schab Art_02_S01 slow t -6.98
## 2 1 -1.36 -0.360 blas Art_02_S01 slow n -5.74
## 3 136 -1.50 -0.270 schlaf Art_02_S01 slow en -4.68
## 4 71 -1.28 -0.377 lahm Art_02_S01 slow en -6.26
## 5 11581 -1.33 -0.447 lahm Art_02_S01 slow t -6.47
## 6 33 -1.41 -0.261 fahnd Art_02_S01 slow en -6.15
## 7 100 -1.29 -0.336 mahn Art_02_S01 slow en -5.81
## 8 16957 -1.18 -0.409 bahn Art_02_S01 slow t -5.76
## 9 127 -1.29 -0.466 schad Art_02_S01 slow n -5.00
## 10 163 -1.47 -0.342 wat Art_02_S01 slow n -6.43
## # i 822 more rows
Here are the histograms:
As we know, negative logarithms stand for values less than 1, which means that the lower values are, the faster they’re pronounced. We can see on the graph that shorter segment duration are found for fast condition, while the longer one is found for slow condition. It is both seen on median value as well as on the value distribution.
Here we see the same situation: word duration is shorter in fast speech and longer in slow speech. Also, unsurprisingly, the lengths of words are bigger than segments. It should be noticed, that graphs show a more noticeable difference between different conditions for words, rather than we saw for segments.
Now let’s check our results using Students t-test.
##
## Welch Two Sample t-test
##
## data: dur_word_freq_slow$LogDurationW and dur_word_freq_fast$LogDurationW
## t = 11.562, df = 829.89, p-value < 2.2e-16
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## 0.1123477 0.1582922
## sample estimates:
## mean of x mean of y
## -0.4483053 -0.5836253
As we can see, for word duration p-value is extremely low, which means that the difference is significant and the distribution is not random.
##
## Welch Two Sample t-test
##
## data: dur_word_freq_slow$LogDurationA and dur_word_freq_fast$LogDurationA
## t = 7.5116, df = 822.67, p-value = 1.523e-13
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## 0.08075098 0.13788190
## sample estimates:
## mean of x mean of y
## -1.628756 -1.738072
For segment duration p-value is also extremely low, though a bit higher than for word duration, which means that the difference is significant and the distribution is not random, though a bit less significant than for word duration.