MLE for \((\mu,\sigma)\) in the Logistic Distribution
Let \(X_1,\dots, X_n\) be i.i.d. real random variables from a Logistic distribution with location parameter \(\mu\) and scale parameter \(\sigma\). Let \(x_1,\dots, x_n\) be an observed sample from this distribution (a realisation from such random variables). The corresponding pdf is \[ f(x;\mu ,\sigma) = {\frac {e^{-{\frac {x-\mu }{\sigma}}}}{\sigma\left(1+e^{-{\frac {x-\mu }{\sigma}}}\right)^{2}}}. \]
We will use the command optim() from the R programming language. This command implements several numerical optimisation methods, and we will use a method known as Nelder-Mead. This method requires an initial point to start the search for the maximiser. In this case, we will use the sample mean and the sample standard deviation as initial points (other points can be used as well). We use \(n=250\) simulated observations from a Logistic distribution with \(\mu=10\) and \(\sigma=1\). Notice how close the MLE is from the true values of the parameters.
library(knitr)
# Simulated (fake) data
# mu = 10, sigma = 1
set.seed(1234)
data <- rlogis(n = 250, location = 10, scale = 1)
print(data, digits = 3)## [1] 7.95 10.50 10.44 10.50 11.82 10.58 5.35 8.81 10.69 10.06 10.82
## [12] 10.18 9.07 12.49 9.12 11.64 9.09 8.99 8.53 8.80 9.23 9.17
## [23] 8.33 6.82 8.73 11.45 10.10 12.37 11.60 6.96 9.82 8.98 9.17
## [34] 10.03 8.49 11.15 8.62 8.95 14.84 11.43 10.21 10.60 9.21 10.50
## [45] 9.29 10.01 10.74 9.94 8.87 11.18 7.47 9.20 10.93 10.02 8.29
## [56] 10.02 9.98 11.11 8.45 11.72 11.86 6.87 9.23 5.73 8.84 10.88
## [67] 9.19 10.03 7.09 10.26 8.02 12.12 5.79 11.28 7.69 10.08 9.53
## [78] 7.41 9.25 10.70 12.53 9.89 8.21 10.18 8.59 12.18 9.55 9.20
## [89] 8.34 12.16 8.39 12.20 8.13 8.11 7.86 10.05 9.15 6.40 9.20
## [100] 11.06 6.70 10.26 9.06 8.64 8.13 9.27 8.30 8.10 9.74 6.79
## [111] 10.91 7.81 12.95 8.02 8.73 12.35 12.86 9.05 8.04 11.37 11.07
## [122] 12.39 15.22 12.79 9.94 9.07 8.91 10.01 9.99 9.24 13.24 10.55
## [133] 8.08 9.69 12.37 9.87 12.29 10.40 10.54 11.89 10.01 14.10 9.27
## [144] 9.93 9.41 10.52 11.05 10.27 13.93 10.31 9.75 8.78 7.59 11.74
## [155] 8.82 14.42 10.41 16.68 9.49 10.22 9.72 10.31 9.73 8.76 7.62
## [166] 10.56 9.72 7.45 11.40 9.27 11.14 10.34 10.89 9.71 9.35 11.15
## [177] 9.69 10.24 7.97 9.17 9.92 9.36 10.41 7.50 13.08 6.22 11.67
## [188] 10.54 9.20 11.06 10.57 14.89 8.08 12.02 11.45 11.53 11.62 11.01
## [199] 14.06 10.57 10.67 10.11 9.23 11.20 10.11 11.01 9.19 9.61 8.64
## [210] 14.23 10.27 9.06 8.52 11.14 10.27 12.62 10.57 10.85 9.92 11.74
## [221] 9.69 6.57 8.94 9.31 8.13 10.00 11.40 9.32 10.04 9.98 11.37
## [232] 10.27 7.88 11.43 10.27 8.69 11.10 9.19 9.96 14.57 9.69 8.87
## [243] 8.72 10.80 13.90 9.91 11.23 10.30 13.34 11.37# Summary of the data
summary(data)## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 5.353 8.956 9.993 10.032 11.056 16.676# Histogram of the data
hist(data)
box()
# Direct implementation of the log-likelihood function
log.lik <- function(par){
mu <- par[1]; sigma <- par[2]
if(sigma>0) return( sum(dlogis(data, par[1], par[2], log=TRUE)))
else return(-Inf)
}
# Optimisation step. "fnscale = -1" indicates that the function fn will be maximised
# requires an initial point, which we select as the mean and the standard deviation of the data
OPT <- optim(par = c(mean(data), sd(data)), fn = log.lik, control = list(fnscale = -1),
method = "Nelder-Mead")
# Extracting the MLE
print(OPT$par, digits = 3)## [1] 9.966 0.971