• Session 1: REF Familiarisation Workshop (today)
• Session 2: Understanding 4\(^*\) Research in Your Field
• Session 3: Recognising 3\(^*\) and 4\(^*\) Publications
• Session 4: Planning Research with 4\(^*\) Potential
• Session 5: Planning for REF-relevant Research Outputs
• Session 6: Writing Clinic: Shaping Manuscripts for REF
• Session 7: Ongoing Support

By the end, we should have:

• a shared understanding of why output quality matters for NTU Psychology,
• a clearer sense of what 3\(^*\) and 4\(^*\) mean in practice,
• a way of discussing output quality using originality, significance, and rigour,
• practice calibrating judgements using example abstracts.

• Universities submit evidence about research quality.
• Submissions are organised by Units of Assessment, such as UoA 4: Psychology, Psychiatry and Neuroscience.
• REF looks at research outputs (journal articles, books, software, exhibitions), impact beyond academia, and research environment (conditions that support research quality).
• The results affect reputation and future research funding.
• REF happens periodically: 2014, 2021, 2029.

A REF submission is not a count of everything we publish.

• What makes a piece of research strong enough to submit?
• How can we recognise 3\(^*\) and 4\(^*\) potential earlier?
• How can we improve outputs while still being planned, written, or revised?
• How can we discuss output quality across different areas of psychology?

NTU submitted 58.0 FTE to UoA 4: Psychology, Psychiatry and Neuroscience.

• 75% of eligible staff were submitted.
• 145 outputs were submitted.
• That is a broad research-active base.
• The issue is not simply whether research is happening.
• The strategic issue is the quality profile of the submitted outputs.

Most submitted outputs were 3\(^*\) or above. The main opportunity is increasing the proportion of outputs that can be defended as 4\(^*\).

REF matters because it shapes the conditions in which research happens.

• It affects reputation and external confidence in our research strength.
• It informs Quality-related Research funding allocated by Research England.
• That funding goes to the university, not directly to individuals.
• Stronger REF performance gives department a stronger case for investment (research time, PhD studentships, grant development, infrastructure)

Approximate annual quality-related research funding value of UoA4 outputs:

Source: calculated from Research England’s 2025/26 NTU grant data tables, UoA4 outputs. Exact estimates: 3\(^*\) = £2,987; 4\(^*\) = £11,947.

The key point is per year: a 4\(^*\) output is not worth about £11k once. Across an eight-year REF cycle, that is roughly £90k per 4\(^*\) output.

Under REF 2021 rules:

• a unit submitted 2.5 outputs per submitted FTE,
• each submitted staff normally needed at least 1 attributed output,
• no individual could normally have more than five outputs attributed to them.

So, for 10 submitted FTE, the unit needed about 25 outputs. Even if one person had many 4\(^*\) papers, only five could normally be attributed to them.

REF-focused development helps with work you already care about.

• Stronger papers are easier to position and defend.
• Clearer contribution claims improve introductions and discussions.
• Better planning can reduce late-stage rewriting.
• Shared peer feedback can make outputs stronger before submission.
• Calibration helps you decide where to invest time and ambition.

Go to menti.com

Think about your own area of psychology.

1. What criteria do you use to identify a strong piece of research?
2. What makes you trust it?
3. What makes you think it matters?
4. What makes it more than just another competent study?

REF outputs are judged against three criteria.

Originality

What is new, innovative, distinctive, or insight-generating?

Significance

Why does the contribution matter, and who could it influence?

Rigour

Is the work intellectually coherent, well executed, and appropriately evidenced?

How confident are you with REF output criteria?

Discuss on your tables:

Map the comments shown on menti.com to one or more REF output criteria:

• Originality
• Significance
• Rigour

Where does it fit easily? Where does it not fit neatly?

Used well, REF criteria can help us ask better research questions.

Common questions:

• Does this change how people in the field understand the problem?
• Is the conceptual contribution unusually strong?
• Is the evidence unusually persuasive, rich, complex, or cumulative?
• Does the work open up a new line of research, method, theory, policy, or practice?

4\(^*\) does not require:

• a particular journal,
• a fashionable topic,
• a huge sample in every case,
• a specific method,
• applied impact already achieved,
• perfection in every sentence.

It does require a contribution that an informed assessor can recognise as outstanding.

REF assessors judge the whole output, not just the abstract.

But a good abstract should make the REF-relevant case visible.

Abstracts are useful for first-pass calibration, but they are not enough for a final REF judgement, especially on rigour.

You will see four REF-submitted output examples.

Score all four abstracts individually on Menti for the visible evidence:

• Originality: 1-5
• Significance: 1-5
• Rigour: 1-5

Use 1 = unclear evidence, 3 = plausible evidence, 5 = very strong evidence.

We will use the results to identify a case for discussion.

Examples are based on outputs submitted to REF 2021 UoA4. Summaries are paraphrased and deliberately vary in how strongly they signal the REF criteria.

Title: Clinical and economic outcomes of remotely delivered cognitive behaviour therapy versus treatment as usual for repeat unscheduled care users with severe health anxiety: a multi-centre randomised controlled trial

This multi-centre randomised trial tested whether remotely delivered CBT could help people with severe health anxiety who repeatedly used unscheduled healthcare. The study compared remote CBT with treatment as usual and assessed clinical outcomes, healthcare use, quality of life, and cost-effectiveness over follow-up. The intervention reduced health anxiety and related symptoms, with evidence of lower costs. The authors argue that the approach may offer a scalable way to engage a patient group that services often struggle to support.

REF 2021 UoA4 submitted output. DOI: 10.1186/s12916-019-1253-5. Summary paraphrased.

Title: Advances in Facial Composite Technology, Utilizing Holistic Construction, Do Not Lead to an Increase in Eyewitness Misidentifications Compared to Older Feature-Based Systems

This experimental study examined whether constructing a facial composite affects later eyewitness identification. Participants viewed a staged crime and were assigned to construct a feature-based composite, construct a holistic composite, or complete no composite task. They later undertook an identification procedure after a delay intended to better resemble investigative conditions. The results suggested that neither composite method reduced later identification accuracy. The authors discuss implications for police use of facial-composite systems and eyewitness evidence.

REF 2021 UoA4 submitted output. DOI: 10.3389/fpsyg.2019.01962. Summary paraphrased.

Title: Cost-benefit trade-offs in decision-making and learning

This study asked whether the cognitive cost of managing conflict is part of value-based decision-making and learning. Participants completed a reversal-learning task with distracting flanker information, allowing comparison between free and instructed choices. Computational modelling tested how distractors, uncertainty, reward expectations, and conflict shaped decisions and learning rates. The results suggested that people sometimes follow irrelevant cues when cognitive control is costly, and that different forms of conflict can affect learning differently.

REF 2021 UoA4 submitted output. DOI: 10.1371/journal.pcbi.1007326. Summary paraphrased.

Title: “Having All of Your Internal Resources Exhausted Beyond Measure and Being Left with No Clean-Up Crew”: Defining Autistic Burnout

This qualitative study used a community-based participatory approach to define autistic burnout, a concept widely discussed by autistic adults but underdeveloped in academic and clinical literature. The researchers analysed interviews and public internet sources, identifying recurring features including chronic exhaustion, reduced tolerance to stimulus, and loss of skills. They also identified contributors such as life stress, masking, and barriers to support. The paper argues that autistic burnout is distinct from occupational burnout and depression, with implications for recognition, support, stigma reduction, and suicide prevention.

REF 2021 UoA4 submitted output. DOI: 10.1089/aut.2019.0079. Summary paraphrased.

We will discuss the abstract with the widest spread of scores.

• Which example made the strongest case on each criterion?
• Which criterion was easiest to judge from the abstract?
• Which criterion was hardest to judge?
• What wording made the contribution visible?
• What would you need from the full paper for a stronger judgement?

Use this after discussion, not before scoring. The numbers are a proposed calibration for the summaries, not known REF scores for the outputs.

• the research question,
• the design,
• the evidence,
• the collaboration,
• the writing,
• and the feedback process.