Fitting probability models to frequency data

• Goodness-of-fit tests
• \( \chi^2 \) GOF test

Definition: A goodness-of-fit test is a method for comparing an observed frequency distribution with the frequency distribution that would be expected under a simple probability model governing the occurrence of different outcomes.

Note: The binomial test is an example of a goodness-of-fit test.

Definition: The binomial test uses data to test whether a population proportion (\( p \)) matches a null expectation (\( p_{0} \)) for the proportion.

Definition: The null hypothesis \( H_{0} \) and alternative hypothesis \( H_{A} \) for a binomial test are given by:

    \( H_{0} \): Relative frequency of successes in population is \( p_{0} \).
    \( H_{A} \): Relative frequency of successes in population is not \( p_{0} \).

• The binomial test is limited to categorical variables with only two possible outcomes.

• We now introduce a more general goodness-of-fit test allowing to:

  • handle categorical and discrete numerical variables having more than two outcomes
  • assess the fit of more complex probability models

Assignment Problem #21

A recent study of Feline High-Rise Syndrom (FHRS) included data on the month in which each of 119 cats fell (Vnuk et al. 2004). The data are in the accompanying table. Can we infer that the rate of cat falling varies between months of the year?

Month	Number fallen	Month	Number fallen
January	4	July	19
February	6	August	13
March	8	September	12
April	10	October	12
May	9	November	7
June	14	December	5

A recent study of Feline High-Rise Syndrom (FHRS) included data on the month in which each of 119 cats fell (Vnuk et al. 2004). The data are in the accompanying table. Can we infer that the rate of cat falling varies between months of the year?

Question: What are the null and alternative hypotheses?

Answer:
     \( H_{0} \): The frequency of cats falling is the same in each month.
     \( H_{A} \): The frequency of cats falling is not the same in each month.

Observed and Expected Frequencies

d = read.csv("http://whitlockschluter.zoology.ubc.ca/wp-content/data/chapter08/chap08q21FallingCatsByMonth.csv") 
head(d)

     month
1  January
2  January
3  January
4  January
5 February
6 February

is.data.frame(d)

[1] TRUE

d$month[1:10]

 [1] January  January  January  January  February February February February
 [9] February February
12 Levels: April August December February January July June March ... September

let's use a loop to calculate the observed frequencies

dc = data.frame(matrix(rep(NA, 36), nr = 12))
colnames(dc) = c("month", "obs", "exp")
dc$month = c("January", "February", "March", "April", "May", "June", "July", "August", "September", "October", "November", "December")
for(i in 1 : 12){ 
  month.i = dc$month[i]
  d_month.i = subset( d, month == month.i )
  dc[i, "obs"] = nrow( d_month.i )# careful, `length` won't work, `d` is a dataframe...
  # Equivalent to: dc[i, "obs"] = length( d_month.i$month )
}             

Now let's compute the expected frequencies

( dc[, "exp"] = sum( dc[, "obs"] ) / 12 )

[1] 9.916667

dc

       month obs      exp
1    January   4 9.916667
2   February   6 9.916667
3      March   8 9.916667
4      April  10 9.916667
5        May   9 9.916667
6       June  14 9.916667
7       July  19 9.916667
8     August  13 9.916667
9  September  12 9.916667
10   October  12 9.916667
11  November   7 9.916667
12  December   5 9.916667

Definition: The \( \chi^2 \) statistic measures the discrepancy between observed frequencies from the data and expected frequencies from the null hypothesis and is given by

\[ \chi^2 = \sum_{i}\frac{(Observed_{i} - Expected_{i})^2}{Expected_{i}} \]

\[ \chi^2 = \frac{(4 -9.917)^2}{9.917}+\frac{(6 -9.917)^2}{9.917}+...+\frac{(5 -9.917)^2}{9.917} \]

Discuss: What would support the null hypothesis more: a small value or large value for \( \chi^{2} \)?

\[ \chi^2 = \frac{(4 -9.917)^2}{9.917}+\frac{(6 -9.917)^2}{9.917}+...+\frac{(5 -9.917)^2}{9.917} \]

Answer: Small value for \( \chi^{2} \)

\( \chi^2 \) test statistic - computed in R

chisq.test(x = dc[,"obs"], p = dc[,"exp"], rescale.p = TRUE)

    Chi-squared test for given probabilities

data:  dc[, "obs"]
X-squared = 20.664, df = 11, p-value = 0.03703

How is the P-value calculated?

Definition: The number of degrees of freedom of a \( \chi^2 \) statistic specifies which \( \chi^2 \) distribution to use as the null distribution and is given by

df = (Number of categories) - 1 - (Number of parameters estimated from data)

\( P \)-value = 0.037024

Discuss: Conclusion?

Conclusion: Reject \( H_{0} \), i.e. there is evidence that the frequency of cats falling is not the same in each month.

\[ Pr[\chi^{2} \geq \chi_{0.05,11}^2] = Pr[\chi^{2} \geq 19.675] = 0.05 \] \[ \mathrm{Observed} \ \chi^2 = 20.664 \]

The sampling distribution of the \( \chi^2 \) statistic follows a \( \chi^2 \) distribution only approximately. The approximation is excellent, as long as the following rules are obeyed:

• None of the categories should have an expected frequency less than one.
• No more than 20% of the categories should have expected frequencies less than five.