class: middle background-image: url(data:image/png;base64,#LTU_logo.jpg) background-position: top left background-size: 30% # STM1001 [Topic 6B](https://bookdown.org/a_shaker/STM1001_Topic_6B_Sci/) Lecture ## Designing a Study Part IV ### La Trobe University This presentation complements the [Topic 6B readings](https://bookdown.org/a_shaker/STM1001_Topic_6B_Sci/) --- # Topic 6B: Designing a Study Part IV ### In this topic, we will continue our discussion of internal validity, this time in the context of observational studies. We will then briefly consider the identification of study limitations, and procedures for collecting data. --- # Internal validity and observational studies In experimental studies, many aspects of the study design typically can be controlled by the researcher, so experimental studies are often easier to design to maximise internal validity. In contrast, *observational studies* have fewer design features that can be controlled by the researchers. -- For example, treatments are not imposed in observational studies, so random allocation of treatments is impossible, and hence *confounding* is always a potential threat to internal validity in observational studies. -- The best advice for observational studies is **to measure, observe, assess or record all the information that is likely to be important for understanding the data.** Observational studies can (and often do) have control groups. Indeed, one specific type of observational study is called a **case-control study.** --- # Internal validity and observational studies The **carry-over effect** is a possible compromise to internal validity in observational studies. However, since treatments are not allocated in observational studies, carry-over effects may be difficult to prevent. -- In observational studies, individuals may or may not know they are being observed. As with experimental studies, efforts should be made to ensure that individuals do not know that they are being observed to minimize the impact of the **Hawthorne effect** (that is, that the participants are **blinded**). -- The **observer effect** can be an issue in observational as well as experimental studies. E.g. if the researchers know whether or not the individual is a smoker when they record the blood pressure, then the observer effect could still come into play. The observer effect could be managed if the researchers first measured, and then asked if the individual was a smoker or not. -- The **placebo effect** is concerned with treatments, so are not directly relevant to observational studies. --- # Identifying study limitations The *type* of study and how that study is designed can determine how the results of the study should be interpreted. Ideally, a study would be perfectly externally and internally valid, but in practice this is very difficult to achieve. Practically *every* study has limitations. -- Limitations can often be discussed through three components: * **External validity** (the applicability of the study results outside the sample): The generalisability of the results to the *intended population*. * **Internal validity** (the effectiveness of the study in the sample): The steps taken to maximise the internal validity of the study, and the impacts of these on the interpretation of results. * **Ecological validity** (the practicality of the results to real life): The practicality of the results in the real world; how the study methods, materials and context approximate the real situation being studied. Almost every study has limitations. *Identifying* them, and *discussing* the impact that they have on the interpretation of the study results, is important and ethical. --- name: menti class: middle background-image: url(data:image/png;base64,#menti.jpg) background-size: 115% # Kahoot! ## Go to [kahoot.it](https://kahoot.it/) and use ## the code provided --- # Limitations: External validity External validity refers to the ability to *generalise* the results to other groups in the population apart from the sample studied (see section: External and internal validity). .content-box-blue[ Importantly, external validity refers to how well the sample is likely to represent the target population as given in the RQ.] -- External validity refers to the *applicability* or the *generalisability* of the *results* to the **target (or intended) population**, which depends on how the sample was obtained: *results from random samples are likely to generalise to the population and be externally valid when appropriately analysed*. Furthermore, results from approximately representative samples may generalise to the population and be externally valid if those in the study are not obviously different than those not in the study. --- # Limitations: Internal validity Internal validity refers to how reasonable and logical the results from the study are: the strength of the inferences that can be made from the sample. That is, an internally valid study is effective in demonstrating that the conclusions made from the sample cannot be explained any other way. -- **Internal validity** can be compromised by **confounding**, the **carryover effect**, the **Hawthorne effect**, the **observer effect**, and/or the **placebo effect**. Consequently, if any of these issues are likely to compromise internal validity, the implications on the interpretation of the results should be discussed. -- The internal validity of observational studies is often compromised because confounding can be less effectively managed than for experimental studies. --- # Limitations: Ecological validity The *practicality* of the study results in the real world should also be discussed. This is called *ecological validity*. .content-box-blue[ **Definition (Ecological validity)** A study is ecologically valid if the study methods, materials and context approximate the real situation being studied.] -- Studies don't need to be ecologically valid to be useful; much can be learnt under special conditions, as long as the potential limitations are understood when applying the results to the real world. Although ecological validity is not essential for a good study, ecological validity is useful if it is possible to achieve. --- background-image: url(data:image/png;base64,#computerlab.jpg) background-position: bottom background-size: 75% class: center # See you in the computer labs! --- class: middle <font color = "grey"> These notes have been prepared by Illia Donhauzer. They are based on material written by Peter K. Dunn. Unless otherwise stated, material within this work is licensed under a Creative Commons Attribution-Non Commercial-Share Alike License <a href="https://creativecommons.org/licenses/by-nc-sa/4.0/">CC BY-NC-SA </a> </font>