Can gut bacteria cause autism-like behaviours in mice?

Overview

In their 2019 Cell paper, Sharon et al claim that mice receiving gut microbiota via faecal transplants from autistic donors show increased “autism-like” behaviours. Several commentators (including myself) had concerns about the inferences drawn from the data, which I summarised in a Medium post.

However, a re-analysis by professor of biostatistics Thomas Lumley goes further, suggesting that there were errors in the SPSS syntax used to analyse the data. Specifically, mice receiving faecal transplants from the same donor were treated as if they were independent. In other words, the numerical effect sizes were evaluated as if there had been >200 donors when in fact there were between 8. Lumley has since summarised his blogpost on PubPeer.

Here, my aim is to extend Lumley’s critique to other analyses reported in the paper. I create two functions based on Lumley’s code.

The first (SharonAnalysis) attempts to replicate as closely as possible Sharon’s SPSS analysis in R. It treats each datapoint incorrectly as being independent. Across all the measures, there is qualitative agreement with the results reported by Sharon et al.

The second function (LumleyAnalysis) provides an alternative analysis that correctly treats Donor as a random factor. In other words, it takes into account the fact that multiple mice had the same donor. Now, all but one of the effects disappears. This is the case, whether we consider the whole available dataset or just the data from the 8 donors featured in Figure 1.

Datasets

Lumley’s re-analyses focus on Figure 1 of the paper which presents data from 8 donors (5 autistic children and 3 neurotypical controls) on three tasks. This dataset is available for upload from the Mendeley repository.

data_8<-read.csv("https://data.mendeley.com/datasets/ngzmj4zkms/1/files/fb7927df-a1ea-4257-bf2f-33fecc4090e9/Fig1EFGH_subset8.csv?dl=1")

However, these 8 donors were a subset of a larger sample of 16. There were also two further tasks completed by at least some of the mice. The full data was presented in Supplementary Figure 2 and is also available from Mendeley.

data_16<-read.csv("https://data.mendeley.com/datasets/ngzmj4zkms/1/files/1d419e86-f908-455e-b32b-9dc8672377b9/FigS2ABCDE_cyc1_3.csv?dl=1")

Libraries

We’ll make use of the following R libraries. I’m adding dplyr to those used by Lumley because itss filter command is far more intuitive than base R.

library(beeswarm)
library(lmerTest)
library(pbkrtest)
library(nlme)
library(dplyr)
library(knitr)

Functions

BeePlot <- function (DV, Data=data_8) {
  Data$DV <- Data[[DV]]
  beeswarm(DV~ASD_diagnosis,data=Data,method="center",pch=19,pwcol=Donor)
}

cell$ASD_sum<-C(cell$ASD_diagnosis,"contr.sum")
cell$Gender_sum<-C(cell$Gender,"contr.sum")
marbles_gls <- gls(MB_buried~ASD_sum*Gender_sum, weights=varIdent(form=~1|Donor),data=cell)

SharonAnalysis <- function (DV, Data=data_8, incGender=TRUE) {
  Data$DV <- Data[[DV]]
  Data <- filter(Data, is.na(DV)==FALSE)
  Data$ASD_sum<-C(Data$ASD_diagnosis,"contr.sum")
  Data$Gender_sum<-C(Data$Gender,"contr.sum")
  if (incGender==TRUE) {
    myGls <- gls(DV~ASD_sum*Gender_sum, weights=varIdent(form=~1|Donor),data=Data)
  } else {
    myGls <- gls(DV~ASD_sum, weights=varIdent(form=~1|Donor),data=Data)
  }
  myAnova <- anova(myGls, type="marginal")
  kable(myAnova, digits=3)
}

marbles<-lmer(MB_buried~ASD_sum*Gender_sum+(1|Round)+(1|Donor),data=cell)

LumleyAnalysis <- function (DV, Data=data_8, incGender=TRUE) {
  Data$DV <- Data[[DV]]
  if (incGender==TRUE) {
    myLmer <- lmer(DV~ASD_diagnosis*Gender+(1|Donor),data=Data)
  } else {
    myLmer <- lmer(DV~ASD_diagnosis+(1|Donor),data=Data)
  }
  myAnova <- anova(myLmer, ddf="Kenward-Roger")
  kable(myAnova, digits=3)  
}

Marble-burying

First of all, the marble-burying test. This provided the most compelling evidence for group differences. Here’s the data from 8 donors.

BeePlot ("MB_buried")

Sharon et al. reported a significant effect of donor diagnosis, which we can get close to replicating.

SharonAnalysis ("MB_buried")

Under Lumley’s analysis of the same data, including Donor as a random factor, there is still a significant effect of diagnostic group. But it’s only just significant.

LumleyAnalysis ("MB_buried")

Repeating this analysis on the larger dataset with 16 donors, the group effect is still significant. Note that here we are comparing across three groups of donors - ASD, Mild ASD, and Neurotypical.

BeePlot ("MB_buried", Data=data_16)

LumleyAnalysis ("MB_buried", Data=data_16)

Direct Social Interaction Test

Sharon et al reported that mice with autistic donors showed reduced sociability on the Direct Social Interaction test. Here are their data.

BeePlot ("DSI_Social_duration")

Using Lumley’s approximation of Sharon et al.’s analysis, we do achieve a significant effect of group.

SharonAnalysis ("DSI_Social_duration")

Here, however, is Lumley’s analysis replicated. With donor included in the model, there is no suggestion of a difference between mice with autistic and non-autistic donors.

LumleyAnalysis ("DSI_Social_duration")

Sharon et al. only tested mice from the 8 donors so there is no larger dataset to investigate here.

Open field test

Sharon et al reported that mice with autistic donors showed reduced locomotion (i.e., travelled less distance) during a 10 minute session in a large empty box. Here are the data for the 8 donors.

BeePlot("OFT_Distance")

Under Sharon et al.’s analysis, treating each mouse as an independent data point, there is a significant effect of ASD diagnosis

SharonAnalysis("OFT_Distance")

However, this disappears under Lumley’s re-analysis.

LumleyAnalysis("OFT_Distance")

And repeated with the full 16 participants. Again, no effect of group.

BeePlot("OFT_Distance", Data=data_16)

LumleyAnalysis("OFT_Distance", Data=data_16)

Sharon et al reported that there was a difference between the ASD group and the NT group, ignoring the Mild ASD cases. We can test that too. But again, we find no difference.

LumleyAnalysis("OFT_Distance", Data=filter(data_16,ASD_diagnosis!="Mild ASD"))

Ultrasonic vocalizations

Now moving onto the additional measures that Lumley did not look at. First, ultrasonic vocalizations. Sharon et al reported a significant group difference between mice with ASD and NT donors, excluding those whose donors had “Mild ASD”. But they didn’t include it in Figure 1, presumably because the donors were not the same as those in the other analyses.

BeePlot("USV_Duration", Data=data_16)

Here’s our approximation of Sharon et al.’s analysis, treating each mouse as independent, excluding mice with Mild ASD donors. Note that mouse gender is not included in the model because only male mice were tested.

SharonAnalysis("USV_Duration", Data=filter(data_16,ASD_diagnosis!="Mild ASD"), incGender=FALSE)

## Warning: contrasts dropped from factor ASD_sum due to missing levels

However, when we conduct the analysis with Donor as a random factor, we find no effect.

LumleyAnalysis("USV_Duration", Data=filter(data_16,ASD_diagnosis!="Mild ASD"), incGender=FALSE)

There’s also no effect if we make the three-way comparison between ASD, Mild ASD, and NT.

LumleyAnalysis("USV_Duration", Data=data_16, incGender=FALSE)

Three chambers test

Finally, we come to the Three Chambers test - a widely used assessment of sociability in mice. Here is the data for the 8 donors.

BeePlot("SI_index")

Sharon et al chose not to feature these data in Figure 1, presumably because even in their analysis, the group difference was not significant.

SharonAnalysis("SI_index")

It seems almost pointless at this stage. But for completeness, here is Lumley’s analysis applied to the three chambers test. With 8 donors, the group difference is non-significant.

LumleyAnalysis("SI_index", Data=data_8)

## boundary (singular) fit: see ?isSingular

We can also do this for the full dataset, again showing no group difference.

LumleyAnalysis("SI_index", Data=data_16)

## boundary (singular) fit: see ?isSingular