How to use this guide

This guide walks through the measurement work behind the OUtMATCH quality-of-life analyses. It assumes familiarity with clinical trial statistics (randomization, confidence intervals, p-values) but no background in psychometrics, the methods for judging and scoring questionnaires. Each step answers one measurement question, shows one or two figures, and states the finding and what it means for the manuscript. Results are calculated from the current study dataset when the guide is produced; they are preliminary and unvalidated, shared for discussion only.

The bottom line

  • The questionnaires are reliable on every version we analyze.
  • Because the FAQLQ version depends on age, some children change version during follow-up, and scores are not directly comparable across versions.
  • The parent version can be scored on its first 14 questions at every age, which makes it the most consistent long-term measure.
  • Child and teen versions can be bridged through shared questions, but the teen group is small, so the bridge should be a supporting analysis.
  • Parent and child reports agree only moderately and should be reported separately.
  • We propose leading with the total score and reporting two domains, emotional impact and avoidance and restrictions, alongside it.

How to read this guide

Three kinds of boxes appear throughout:

The idea. The measurement concept behind the step, with an everyday comparison where it helps.

What to look for. How to read the picture or table that follows.

What we found. The result, and what it means for the manuscript.

The map below shows the nine steps. The orange box marks where you are.

Step 1: Meet the questionnaire

The FAQLQ is scored from 1 (not at all) to 7 (extremely), with higher scores meaning more burden. The protocol assigns the version by age at each visit:

The idea. From a measurement point of view, the four versions are four different instruments on the same construct. They share some questions, but each has its own items, domains and precision. Children aged 8 to 12 are described twice at each visit, by a parent and by themselves.

Version Participants Questions analyzed
Parent version 147 26
Child version 73 24
Teen version 49 23

What we found. The parent version has the most participants and the teen version the fewest. The adult version was completed by only 13 people, too few to analyze in depth, so it is summarized descriptively and left out of the modelling.

Step 2: Who answered which version, and when

The study has several stages, and the questionnaire was repeated at set points in each. Because the version depends on age, a child can start the study on one version and finish on another.

The idea. A child who moves from the child version to the teen version is measured with a different instrument, much like switching from a ruler in inches to one in centimeters partway through a growth study. A change score across the switch mixes true change with the change of instrument, and the data cannot separate the two.

What to look for. Each cell is the number of participants who completed that version at that point in the study. The counts are not a fixed group followed over time: they change from one visit to the next for three reasons. Some participants miss a visit or leave the study; only some continue into each later stage, so later groups are self-selected; and a child who turns 13 moves out of the child column and into the teen column. The table after the figure shows this for the first two visits.

From the start of Stage 1 to the first food challenge visit (about 16 weeks). The last column equals the first, minus those who switched away or have no questionnaire at that visit, plus those who switched in or first appear at that visit. “No questionnaire” counts anyone without an analysis questionnaire at that visit, whether they missed it, withdrew, or had it excluded.

Version At Week 0 Switched to the other version No questionnaire at the first challenge visit Switched in from the other version New at the first challenge visit At first challenge visit
Parent version 146 0 11 0 1 136
Child version 58 5 2 0 1 52
Teen version 30 0 2 5 0 33
Path Participants Of these, switched during Stage 1
Stayed on the child version 54 -
Stayed on the teen version 30 -
Moved from the child version to the teen version 19 5

When the switches from the child to the teen version happened:

Timepoint Children who switched
First DBPCFC in Stage 1 5
First Omalizumab Injection in OLE 2
First DBPCFC in OLE 4
First DBPCFC in Stage 2 4
Six Months after starting Stage 3 3
Stage 3 Open OFC 1

What we found. 19 participants moved from the child version to the teen version during the study, 5 of them within the first stage. For these participants, change scores that span the switch are not interpretable. Most participants stay on one version throughout, and the first stage is largely, though not entirely, free of switches. Visit-to-visit counts also move because of missed visits and withdrawals, so a small change in a cell is not by itself a sign of switching.

Step 3: How the questions are answered

Before any modelling, we look at how each question was answered. An informative question spreads participants across the scale. A question that nearly everyone answers at the bottom (a floor effect) or the top (a ceiling effect) cannot distinguish participants at that end, or detect change there.

What to look for. Each bar is one question. The colours show the share of participants choosing each answer, from blue (“not”) to red (“extremely”). A bar that is almost all one colour means nearly everyone gave the same answer.

Answers by version

PF (n = 147)

CF (n = 73)

TF (n = 49)

What stands out

Version Questions Many answer “not at all” (over 15%) Many answer “extremely” (over 15%)
Child version 24 6 17
Parent version 26 17 3
Teen version 23 1 9

What we found. Answers are well spread on most questions. On the parent version, several questions about distress are mostly answered “not at all”. On the child and teen versions, a fair number of questions have many “extremely” answers. With the smaller child and teen groups, some answer choices are rarely used, so for the detailed modelling we combine neighbouring choices into four.

Step 4: Is each version dependable?

The idea. Reliability (here Cronbach’s alpha) measures how consistently a questionnaire’s items measure the same thing, much as a good scale gives the same reading twice. It runs from 0 to 1; above 0.70 is adequate for group comparisons and above 0.90 is excellent.

What to look for. One bar per version. Bars reaching past the right-hand dashed line (0.90) are excellent.

What we found. All three versions are excellent, between 0.92 and 0.96. Whatever limits our analysis, it is not the dependability of the questionnaires.

Step 5: Does each version measure one thing or several?

Each version produces a total score, the average of all its questions. A total score only makes sense if the questions are, broadly, measuring one thing.

The idea. This is called dimensionality. Think of a report card: if grades in every subject rise and fall together, one overall grade summarizes the student well; if they move independently, the overall grade hides something. Here the “subjects” are domains such as worry about reactions and restrictions on daily life. We assess it with a bifactor model, which estimates how much of the shared variation comes from one general factor.

What to look for. Each bar is the explained common variance (ECV): the share of what the questions have in common that comes from one general factor. Above about 0.70, one total score is a fair summary; below it, the domains carry information of their own.

What we found. The parent version is close to “one thing”, so its total score is a fair summary. The child and teen versions are partly one thing and partly several: questions about fear and emotions behave as their own group, separate from questions about restrictions. That is why we suggest reporting the total score first and two domains alongside it. These estimates come from small groups and moved by about 0.1 between analysis runs, so read them as approximate.

Step 6: The parent form’s age rule

Some parent forms looked incomplete. They are not: the form tells parents which questions to answer, depending on the child’s age.

The idea. The FAQLQ-PF instructs parents of children under 4 to answer only questions 1 to 14; questions 15 to 26 begin at age 4. These answers are missing by design, not lost. But a child who turns 4 during follow-up moves from a 14-item to a 26-item score, another change of instrument within the parent version. Questions 1 to 14 are the only set answered at every age.

What to look for. Left: the share of parents answering each question; blue questions are answered by everyone, orange ones only by parents of children aged 4 and over. Right: each dot is one child whose parent answered all 26 questions, comparing the score from the first 14 questions with the score from all 26. Dots on the dashed line mean the two scores agree.

What we found. 35 of 147 parents completed the short form at their first visit. Among children with the full form, the score from the first 14 questions agrees almost perfectly with the score from all 26 (agreement 0.96, where 1 is perfect). So we suggest following children over time with the 14 questions that every parent answers, and keeping the 26-question score as a back-up check for ages 4 and over.

Step 7: Can the child and teen versions be lined up?

For children who move from the child version to the teen version, it would help to put the two versions on one common scale. Some questions appear on both versions in nearly the same words, and these can act as a bridge.

The idea. Two maps of the same city drawn at different scales can be lined up using landmarks that appear on both; the shared questions are our landmarks. The method is item response theory (IRT), which scores people from how each question behaves. A landmark only helps if it sits in the same place on both maps, so we test each shared question: do a child and a teen with the same overall burden answer it the same way? A question that fails is said to show differential item functioning (DIF) and is not used for the bridge.

Because the groups are small and the tests have limited power, we ran the check three ways (with slightly different sets of children and teens) and kept only the questions that passed every time. We rely mainly on effect sizes, not p-values.

What to look for. Each dot is one shared question. The further right, the more differently children and teens answer it. Blue dots passed all three checks. Orange dots were flagged at least once, either because they sit past the dashed line or because a statistical test picked them out.

What we found. Of 17 shared questions, 11 passed every check, comfortably more than the 5 needed. So the child and teen versions can be lined up. The catch is that only 49 teens completed the teen version, which makes the bridge less precise, so we suggest it as a supporting analysis rather than the primary one.

The landmarks that passed: ‘May contain’ labels; Always alert to what you eat; Can eat fewer things; Frightened of eating something new; Frightened of eating the wrong food; Have to explain the allergy to others; Have to read labels; Hesitate when unsure a food is safe; Ingredients of a food change; Others eat your allergen; Try fewer foods when eating out.

Step 8: Do parents and children see it the same way?

For children aged 8 to 12, a parent and the child each completed a questionnaire at the same visit, which lets us compare proxy and self-report directly.

What to look for. Each dot is one child. Across is the child’s own score; up is the parent’s score for that child. If parents and children agreed perfectly, every dot would sit on the dashed line. Dots below the line are children whose parent reports less burden than the child does.

What we found. Across 73 parent and child pairs, the latent correlation is 0.56 (corrected for measurement error; 1 would be perfect agreement), and parents report 0.4 points less burden than their children on average. Parents and children are describing related but different experiences, consistent with earlier FAQLQ studies. The correlation is well below the roughly 0.9 that would justify putting parent and child reports on one scale, so we keep them separate and report the gap as a finding.

Step 9: What it all means

Total score or domains?

The idea. Back to the report card. The total score is the overall grade: the most reliable single number, involving the fewest comparisons, and the outcome the protocol specifies. The domains are the subject grades: less precise, but they show where change happens. We suggest leading with the total score for each version, then reporting the two domains that can be matched across versions: emotional impact, and avoidance and restrictions. For interpretation, a change of about 0.5 points on the 1 to 7 scale is the conventional minimal important difference for the FAQLQ.

An exploratory look at where treatment may act

The figure below is an exploratory, item-by-item comparison of omalizumab and placebo on the parent version at the end of Stage 1. It is not a planned analysis, is not adjusted for multiple comparisons, and should not be cited.

What to look for. Each row is one question. A dot left of the dashed line means less burden on omalizumab. The horizontal line is the unadjusted 95% confidence interval.

What we found. Most dots sit left of the line, and the ones furthest left are questions about restrictions: variety in the diet, restaurants, school and social events. Questions about worry and emotions sit close to the line. Every interval includes zero, so this is a pattern to examine in the planned domain analyses, not a result. It is the main reason to report domains alongside the total score.

The proposed plan

Decisions for the writing group

The accompanying decisions document gives the options, implications and a suggestion for each.

  1. How far the paper follows participants: Stage 1 only, through Stage 2, or through Stage 3?
  2. The headline outcome: total score, plus the two matched domains?
  3. Scoring the parent version on questions 1 to 14, with the 26-question score as a sensitivity analysis?
  4. Children who changed versions: within-version analyses only, or a child and teen bridge as a supporting analysis?
  5. Participants with a COVID hold or restart: include, exclude, or report both? (Excluding them does not change the conclusions.)
  6. The Stage 3 six-month window: keep 4 to 8 months, or widen to about 10 months?

Glossary

Term Plain meaning
Burden How much food allergy weighs on daily life; higher scores mean more burden
Total score The average of all the questions on a version
Domain A group of questions on one theme, such as emotional impact
Reliability How consistently a questionnaire measures (0 to 1; above 0.90 is excellent)
Dimensionality Whether the questions measure one thing or several
Shared question (anchor) A question asked in nearly the same words on two versions
DIF When the same question works differently for two groups, such as children and teens
IRT A method that scores a questionnaire by looking at how each question behaves, not just the average
Linking Putting two versions of a questionnaire on one common scale, using their shared questions