1 What is Deviance?

  • Deviance is a measure of error; lower deviance means better fit to data.
  • The greater the deviance, the worse the model fits compared to the best case (saturated).
  • Deviance is a quality-of-fit statistic for a model that is often used for statistical hypothesis testing.

\[D^\star(\boldsymbol{y};\hat{\boldsymbol{\mu}}) = 2 (L(\boldsymbol{y};\boldsymbol{y})-L(\hat{\boldsymbol{\mu}};\boldsymbol{y}))\]


2 Models

In glm (generalized linear models) there are 2 deviances between 3 models which are defined as follows:

  • The Saturated Model: is a model that assumes each data point has its own parameters, which means you have \(n\) parameters to estimate.
  • The Proposed Model: assumes you can explain your data points with \(p\) parameters plus an intercept term, so you have \(p+1\) parameters.
  • The Null Model: assumes the exact “opposite”, it assumes one parameter for all of the data points, which means you only estimate 1 parameter.

Next figure shows the two types of deviances for glm between the 3 models defined above.

3 How to obtain the deviance?

The Null deviance and the residual deviance are defined as:

\[Null Deviance = 2(ll(Saturated Model) - ll(Null Model))\] with \(df = df_{Sat} - df_{Null}\).

\[Residual Deviance = 2(ll(Saturated Model) - ll(Proposed Model))\] with \(df = df_{Sat} - df_{Res}\).

4 Example

Using a toy dataset we can fit a model to study the effect of body weight on cats gender (y=0 for female and y=1 for male). The dataset is as follows:

y <- c(1, 1, 0, 0, 1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 
       0, 0, 1, 1, 0, 1, 1, 0, 0, 1, 0, 1, 1, 1, 0, 0, 0, 
       1, 0, 1, 1, 1, 0, 1, 1, 0, 1, 0, 1, 1, 1, 0, 0)
weight <- c(2.1, 2.5, 1.2, 1, 3, 2.1, 1.5, 2.2, 1.9, 2.7, 1.1, 2.9, 1.2, 2.1, 
          2.2, 2.5, 1.9, 1.2, 2, 2.9, 2.2, 1.5, 3, 2.4, 1.2, 1.6, 2.3, 2.1, 
          2.6, 2.4, 2.5, 2, 1, 1.4, 2.9, 1.5, 3, 2.9, 2.9, 2.1, 2.8, 2.7, 1, 
          2.9, 1.1, 2.2, 1.3, 1.7, 1.5, 1.7)

To explore the data we can use the next code.

plot(x=weight, y, yaxt='n', pch=20)
axis(side=2, at=0:1, labels=0:1, las=1)

4.1 Saturated and null model

To fit the saturated and null model we can use the next code.

mod_sat <- glm(y ~ as.factor(1:length(y)), family=binomial)
mod_nul <- glm(y ~ 1, family=binomial)

4.2 Proposed model

Now we are going to fit the proposed model.

mod <- glm(y ~ weight, family=binomial)

4.3 Deviance manually

If we wanted to obtain the deviance manually we can use the expression given below and in R we

2*(logLik(mod_sat) - logLik(mod)) # Residual deviance
## 'log Lik.' 35.48677 (df=50)
2*(logLik(mod_sat) - logLik(mod_nul)) # Null deviance
## 'log Lik.' 67.30117 (df=50)

4.4 Deviance from summary

Another way to extract the deviance is using the next code.

summary(mod)
## 
## Call:
## glm(formula = y ~ weight, family = binomial)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -1.6090  -0.4077   0.1892   0.4775   2.2489  
## 
## Coefficients:
##             Estimate Std. Error z value Pr(>|z|)    
## (Intercept)  -7.0053     1.9512  -3.590 0.000330 ***
## weight        3.7997     0.9988   3.804 0.000142 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 67.301  on 49  degrees of freedom
## Residual deviance: 35.487  on 48  degrees of freedom
## AIC: 39.487
## 
## Number of Fisher Scoring iterations: 5

The two deviance for the model can be obtained as:

summary(mod)$null.deviance
## [1] 67.30117
summary(mod)$deviance
## [1] 35.48677

4.5 Deviance from deviance

The last form to obtain the deviance is using:

deviance(mod)
## [1] 35.48677

5 More about deviance

To know more about deviance you can consult this resource.