Evoking and measuring emotions using Virtual Reality

About Me

1. American (moved to NZ in 2014)

2. PhD from VUW earned in 2018

3. Post-doc in Affective Neuroscience with Gina Grimshaw

4. Founded the VR Lab at the School of Psychology in 2018

5. Current Research interests:

   1. Why do humans feel that we are “present” in reality?

   2. Can people actually down-regulate intense fear responses?

   3. Do different emotional states improve or impair mental faculties necessary for decision making?

Lecture Objectives - Lessons from Affective Neuroscience

1. Why VR is the perfect tool for experimental psychological science

2. Inducing Emotions with VR

3. Basics of Emotion Theory (their form and function)

4. The hard reality about the science of emotion (in 2024)

5. How to measure (and not measure) emotions

6. Debunking myths about emotions

7. Best Practices for creating (and validating) emotional VR

Why experimental psychological scientists want to use VR

• Question: What would need to happen for science to say “okay we’re all done now” ?

  • When we are able to use knowledge from previous studies (i.e., models) to make accurate predictions about all behaviours and outcomes that occur in the world.

  • Predictive Validity: The extent to which findings are able to inform accurate predictions.

• The goal of laboratory science is to gain insights about phenomena that occur in the real world (not just laboratory phenomena)

• VR provides limitless experimental control without trading ecological validity!

  • Ecological Validity: the extent to which results from a study generalize to real-life behaviour

  • Verisimilitude: The extent to which a study’s testing conditions matches the conditions of real-life

Myth #1: “Emotions are universally displayed on the face with expressions we recognize”

• It does NOT follow that a computer vision algorithm that accurately detects smiles on a face can accurately detect happiness or joy.

• Conflating an observation and an inference

  • Not all movements are expressions

• People differ from others in the way they express emotions using facial patterns (i.e., variability between people)

• People differ from themselves in the way they express emotions using facial patterns (variability within people)

  • Emotional expressions Context dependent!

Myth #2: “There are distinct bio-markers for different emotions”

• Physiological responses are non-specific

  • Increasing Heart Rate or skin conductance is not “Fear”

    • Could be excitement, sexual arousal, caffeine, circadian rhythm, interest/novelty, anticipation, or caused by any movement

    • Need to properly control for extraneous variables when making inferences from physiological measures

• Physiological activity differs from person to person!

  • 10% of the population are considered non-responders in skin conductance!

  • Therefore, important to collect a “resting baseline” for each participant and subtract this value from subsequent timepoints

• One physiological measure cannot tell you what emotion a person is feeling

  • Instead, need to use multiple measures (subjective, physio, behavioural) to show the emotion across all components

• There are no shortcuts:

  • if you want to use physiological measures correctly, you need to study some physiology to understand what you are measuring and what you would predict to observe!

Myth #3: “You should always use Questionnaires!”

• Questionnaires are (like most things) good and bad.

  • Upside: You can make questionnaires that can measure pretty much anything!

  • Downside: Questionnaires are retrospective and human memory SUCKS

• Sometimes there are alternatives that are way better than questionnaires

• Online (i.e., in-the-moment) verbal ratings

  • Upside: they tell you about how people felt (subjectively) at a specific moment

    • No need for people to rely on memory for emotions

  • Upside: by collecting ratings as multiple points you can assess emotional change across time

  • downside: they pull people’s attention from the simulation

Recommendation #3: Be cautious about how you interpret your changes in physiological measures

1. Measuring physio is fine, but you need to make sure you are interpreting your findings accurately/appropriately

   1. Heart rate increasing is not fear; it is arousal

   2. Skin conductance level increasing is not anxiety; it is arousal

   3. You need to also control for the other reasons physio could change

      1. Heat

      2. Movement

2. Behavioural measures (e.g., Motion Capture) are usually context specific!

Big Takeaways

1. We know more than nothing about emotions, but we definitely do not (yet) know everything about emotions. And we are DEFINITELY closer to knowing nothing than we are to knowing everything.

2. When it comes to how people express emotions, assume that variability is the norm!

3. AI/Machine Learning may one day be able to help us figure out how people are feeling (but facial expressions are not going to be the way)

4. Measuring emotions in physiological responses is extremely delicate and challenging (and requires multiple carefully chosen dependent measures and a well-thought out experimental design)

5. Emotions cannot be captured in a snapshot; they unfold across time

6. Emotions are embodied

7. In VR, people need to feel present in order to feel emotional