MT5762 Lecture 20

Most things examined thus far have fitted into the general framework of linear models

There are group of tests conducted on tables of counts/contingency tables which don't fit so well in this framwork that we cover now (you need to be au fait with these)

However, you could use Generalised Linear Models for similar analyses (as seen in MT5761)

We look at intimately related tests

• Chi-squared goodness-of-fit tests
• Chi-squared tests for homogeneity
• Chi-squared tests for independence

The testing approach is somewhat familiar

• We compare what we expect under some Null hypothesis, to what we actually observed
• The discrepency provides some measure of evidence against \( H_0 \)
• If the discrepency is so large \( H_0 \) seems implausible, we reject it
• We assess this by a test statistic, a probability distribution for it assuming \( H_0 \) is true and subsequent \( p \)-value

Consider the following data tabulating support for Democratic, Republican or Independent candidates. In total there are 2757 individuals in the sample (example from Agresti 2007).

Following our themes of drugs and/or disaster

Case-study The Scottish Schools Adolescent Lifestyle and Substance Use Survey SALSUS has been established by the Scottish Executive to provide a broad based approach to the monitoring of substance use among young people in Scotland.

• Cannabis was the most commonly reported drug used in the last month: 21% of 15 year olds. Few reported any other drug.
• 21% of 15 year olds who had used drugs in the month before the survey reported that they would like to stop using drugs now.
• Over two thirds (69%) of the pupils who used drugs most days, and over half (53%) of those who used drugs at least once a week reported that they 'would not like to give up'.
• Nearly half (47%) of 15 year old regular drug users reported spending 10 pounds or more each week on drugs.
• (Various other somewhat grim findings…)

• Chi-square tests on contingency tables look at the distributions of counts over the cells
• So, we might ask: does a particular row or column distribution differ significantly from some other distribution?

For example, does this SALSUS sample represent Scotland ethnically?

We will assess the similarly of SALSUS sample ethnicity to the census population.

• We have quantitative data (counts) which are categorised using one factor: ethnicity. We have a one-dimensional table.
• The census figures look quite similar to the SALSUS sample
• Some possible differences in the white and black categories.
• Are they explicable by sampling variability?

The usual strategy:

• we take the difference between our theory and data and express it in a standardised way e.g. in units of variability.
• This gives a test statistic, whose distribution we theoretically know under the Null hypothesis.
• How likely is the test statistic (or bigger ones) from our sample given this distribution (i.e. if \( H_0 \) is true)?
• If it is 'unlikely' then the data does not appear to support the \( H_0 \).

As for all tests, we have a null (\( H_0 \)) and alternative hypothesis (\( H_1 \)). In this case:

• \( H_0 \): the data come from the census population
• \( H_1 \): the data do not come from the census population

We obtain expected counts directly using our null hypothesis:

Expected count = total \( \times \) specified cell probability

eg. For the 'White' group: 95.23% of 22246 =

\[ 22246 \times \frac{97.26}{100}=22246 \times 0.9726=21693.67 \]

eg. For the 'Bangladeshi' group 0.10% of 22246:

\[ 22246 \times \frac{0.10}{100}=22246 \times 0.001=22.246 \]

The observed counts are obtained from our sample eg. 21249, 23,…,272. We calculate a measure of difference between the observed and expected counts using the Chi-square test statistic:

Chi-square test statistic

\[ \begin{align*} x^2_0 =& \sum_{\textrm{all cells}} \frac{\textrm{(observed count- expected count)}^2}{\textrm{expected count}}\\ =&\sum_{\textrm{all cells}} \frac{\textrm{(O - E)}^2}{\textrm{E}} \end{align*} \]

Note a key component to this term is a simple (squared) distance between what is predicted by our theory, and what we observed in our data (O-E)

• The Chi-square contributions are calculated for all ethnic groups in the table and these are added together to give the Chi-square test statistic.
• As with the \( t \)-test and the \( F \)-test, the larger the test statistic the stronger the evidence against \( H_0 \)

Note a key component to this term is a simple (squared) distance between what is predicted by our theory, and what we observed in our data (O-E)

• eg. The White group contributes 6.94 to the overall test statistic:

\[ \frac{(21249-21636.45)^2}{21636.45}=6.938 \]

• In this case, the Chi-square contributions from the 10 ethnic groups add to 1193.89.

• Is this test statistic (1193.9) peculiar value under \( H_0 \)?
• What \( \chi^2_0 \)) are likely/unlikely if \( H_0 \) is true?
• Theoretically under \( H_0 \), these follow a \( \chi^2 \) with df = number of categories-1.
• We have 12 ethnic groups and so \( df=10-1=9 \).

Looking at that distribution, you know we're effectively zero:

 pchisq(1193.9, 9, lower.tail = F)

[1] 2.515343e-251

• In our example, we got a very large Chi-square test statistic (1193.9).
• The chance of getting a test statistic this large, or larger, when the data come from the census is effectively zero
• We have very strong evidence against \( H_0 \), we have a highly significant result.

What is underlying this result? Why do we have such a huge test statistic?

Of course you don't do these things by hand…

• Black Other, and Other groups were particularly over-represented compared to census.
• We can also see that the White group was particularly under-represented compared to census.

So maybe a biased sample, or likely the demographics are changing

• Here we seek to test if attitudes towards those who use or sell drugs differ with drug use status.
• Specifically, do those in the population that use drugs feel differently about other drug users and drug suppliers than those that don't use drugs?
• The students were asked their opinion on the following 5 statements and their responses were classified according to their reported drug use (table follows).

(%-ages) 15 year olds pupils' attitudes to those involved with drugs, by drug use status: Scotland 2002.

• Most students who used drugs in the last month disagreed with statements 1–4 and agreed with statements 5.
• Conversely, the majority of students who reported they have never used drugs agreed with statements 1–3.
• The majority of students in both groups were in agreement for statements 4 & 5.

We are going to look at one particular statement 'People who take drugs are stupid'

'People who take drugs are stupid':

15 year olds pupils' attitudes to those involved with drugs, by drug use status: Scotland 2002

• The majority (67%) of those who had used drugs in the last month did not agree that people who take drugs are stupid.
• Conversely, the majority (65%) of those students who have never used drugs agreed that people who take drugs are stupid.
• Are any differences in opinion between these two groups real, or are these differences due to sampling variability?

We want to test if attitudes in the population towards those who take drugs differ with drug use.

• The null hypothesis (\( H_0 \)) The samples have the same underlying distribution i.e. Opinions are the same for those that have used drugs in the last month and those that have never used drugs.

• The alternative hypothesis (\( H_1 \)) The samples do not have the same underlying distribution i.e. Opinions are not the same for those that have used drugs in the last month and those that have never used drugs.

We obtain the expected counts, under the null hypothesis, using:

\[ E = \frac{\textrm{Row total} \times \textrm{Column total}}{\textrm{Grand Total}} \]

eg. 'Used drugs in last month' and 'Agree': \[ \left[\frac{2234 \times 4670}{8697}\right]=119.58 \]

Is our data consistent with the null hypothesis? Our Observed counts are obtained from our sample. eg. 469,1497,…,840.

Discrepency between observed and expected counts as before:

\( \chi^2 \) test statistic

\[ x^2_0=\sum_{\textrm{all cells}} \frac{\textrm{(O - E)}^2}{\textrm{E}} \]

• For example 'Used drugs in last month' and 'Agree' contributes 444.95 to the overall test statistic of 1602.03:

\[ \frac{[469-1199.58]^2}{1199.58}=444.95 \]

• All cell contributions are added together to give the Chi-square test statistic, \( x_0^2 \).
  
• The chi-square contributions from the 6 cells in the table add to 1602.03

\[ x^2_0 = 444.95+ 744.60+ 0.97+ 153.80+ 257.38+ 0.34=1602.3 \]

• Is this likely to occur just to chance if \( H_0 \) were true? We quantify the 'likelihood' of this result under the Null hypothesis

• How probable is our data result if \( H_0 \) were true?
• \( p \)-value from a chi-square distribution with df = number of columns-1 \( \times \) number of rows-1

\[ df=(3-1) \times (2-1) = 2 \times 1 = 2 \]

• Under the null hypothesis we expect to get Chi-square values 6 or less about 95% of the time when \( df=2 \).

  qchisq(0.95, 2)

[1] 5.991465

• In our example, we got a mammoth Chi-square value of 1602.3.

• In our example, we got a mammoth Chi-square value of 1602.3.

\[ Pr(\chi^2 \geq x^2_0) ~~\textrm{where} ~~ \chi^2 \sim \textrm{Chi-square}(df) \]

pchisq(1602, 2, lower.tail = F)

[1] 0

drugTest$p.value

[1] 0

• We have very strong evidence against \( H_0 \). ie. very strong evidence that the opinions are not the same for those that have used drugs in the last month and those that have never used drugs.

Let's return to voters and gender

• Is there a relationship between gender and voting intention?
• Is gender and voting intention independent of one another?

So we seek to test \( H_0 \): gender and voting intention are independent

So how do we calculate expected counts assuming \( H_0 \) is true?

• In this case, the probability of being in a particular cell, is the probability of being in the particular row, multiplied by the probability of being in the particular column
• \( P(A~and~B) = P(A)\times P(B) \) - as we saw waaay back with treating independent events
• This rationale results the same calculations for counts as before: (row total \( \times \) col total)/table total

As ever, there are assumptions

• Chi-square tests are only valid when the data are collected as a random sample or as a number of random samples.
• Chi-square tests are large sample tests that require the total count for the table to be sufficiently large.
• There isn't complete agreement about how large is large enough
• The following rules help ensure we don't use Chi-square tests for samples which are too small:
  • each expected count should be greater than 1
  • 80% of the expected counts should be at least 5