Topic 4B: Designing a Study Part II


These are the solutions for the Topic 4B Computer Lab.

1 Readings recap

1.1 Internal validity and experimental studies

No solutions required.

1.2 Internal validity and observational studies

No solutions required.

1.3 Identifying study limitations

No solutions required.

2 Internal validity and experimental studies: Exercises

Adapted from Dunn (2021).

2.1

Discuss your answer with others in your class.

2.2

The observer effect. The researcher is directly contacting the subjects, so may unintentionally influence their responses.

2.3

1. Randomly allocate the type of water to the subject (or the order in which the subjects taste-test each drink.) 2. The subjects do not know which type of water they are drinking. 3. The person providing the water and receiving the ratings does not know which type of water they are drinking. 4. Hard to find a control. 5. Any random sampling is good, if possible.

2.4

1. Response: The amount of sunscreen used; Explanatory: The time spent on sunscreen application. 2. They were looking at potential confounding variables. 3. If the mean of both the response and explanatory variables was different for females and males, then the sex of the participant would be a confounding variable, and this would need to be factored into the analysis of the data. 4. The participants are blinded to what is happening in the study.

3 Internal validity and observational studies: Exercises

Adapted from Dunn (2021).

3.1

1. Since this is an observational study, we cannot allocate stduents to receive bottled or tap water (because then the study would be an experimental study). In an experiment we could randomly allocate students to receive either bottled or tap water and have them rate the taste (or even randomly allocate students to receive bottled or tap water first, then swap to the other type of water, and each student would then provide two ratings). 2. The students would not be aware of which water they would be drinking. 3. Neither the students nor the researchers who give the students the water would know which type of water the students are drinking. 4. We can’t really set up a control here. 5. Any of the random sampling methods are possible, and are preferred. In practice, perhaps use a convenience sample, but try to get a sample as representative as possible (see Representative sampling).

3.2

Yes. Consider a study of the effect of smoking: non-smokers are the control. However, in an observational study, cases cannot be allocated to be controls.

3.3

No.  People can know they are being observed.

3.4

The descriptions indicates that patients probably knew they were involved, so the Hawthorne effect should be considered when interpreting the results.

4 Identifying study limitations: Exercises

Adapted from Dunn (2021).

4.1

Population: ‘USC students on-campus.’ External validity refers to whether the results apply to other members of this population, not to people outside this population (such as members of the general public).

4.2

1. P: Aircraft passengers aged 18 and over. O: Unclear; something about ‘composite of death or major traumatic injury.’ C: Between wearing a parachute and wearing a backpack. I: Yes: Having participants wear the parachute or backpack. 2. Experimental: The researchers decide if the participants use a parachute or backpack. 3. Explanatory: ‘whether or not a parachute is worn.’ Response: harder to understand; is it ‘whether or not the participant dies or sustains a major injury?’ 4. These results won’t apply in the real world; not ecologically valid. In the real world, parachutes are used at high altitude, for example. 5. The study is not very useful! 6. Speaking loosely: That jumping from a small plane that is on the ground, parachutes are equally effective as backpacks in keeping people safe.

4.3

Because the sample is not a random sample, the researchers are (rightly) noting that the results may not generalise to all hospitals. Because the data was only collected at night, perhaps the data is not ecologically valid.

References

Dunn, Peter K. 2021. Scientific Research and Methodology: An Introduction to Quantitative Research in Science and Health. https://bookdown.org/pkaldunn/Book.


These notes have been prepared by Amanda Shaker. The copyright for the material in these notes resides with the authors named above, with the Department of Mathematics and Statistics and with La Trobe University. Copyright in this work is vested in La Trobe University including all La Trobe University branding and naming. Unless otherwise stated, material within this work is licensed under a Creative Commons Attribution-Non Commercial-Non Derivatives License BY-NC-ND.