In Huber weighting, observations with small residuals get a weight of 1 and the larger the residual, the smaller the weight. This is defined by the weight function
\[\begin{eqnarray} w(e) = 1 \mbox{ } \mbox{for} |e| \leq k \\ w(e) = \frac{k}{|e|} \mbox{ } \mbox{for} |e| > k \end{eqnarray}\]
The value \(k\) for the Huber method is called a ; smaller values of k produce more resistance to outliers, but at the expense of lower efficiency when the errors are normally distributed.
The tuning constant is generally picked to give reasonably high efficiency in the normal case; in particular,$ k = 1.345$ for the Huber'’'s method, where \(\sigma\) is the standard deviation of the errors) produce 95-percent efficiency when the errors are normal, and still offer protection against outliers.
library(faraway)
data(cheddar)
names(cheddar)## [1] "taste" "Acetic" "H2S" "Lactic"library(MASS)
FitAll.rr = rlm(taste ~ Acetic + H2S + Lactic,data =cheddar)Regression Equation: \[ \hat{taste} = -20.75 -1.53 Acetic + 4.05 H2S + 20.14 Lactic\] From before, we noticed that observations 15 , 12 and 8 were influential in the determination of the coefficients. The following table indicates the weight given to each observation when using robust regression.
We can see that roughly, as the absolute residual goes down, the weight goes up. In other words, cases with a large residuals tend to be down-weighted.
%(You do not need to know how to implement the code used to generate this table, but we will be looking at how to construct data frames later in the course.)
hweights <- data.frame(taste = cheddar,
resid = FitAll.rr$resid,
weight = FitAll.rr$w)
hweights2 <- hweights[order(FitAll.rr$w), ]hweights2[1:15, ]## taste.taste taste.Acetic taste.H2S taste.Lactic resid weight
## 15 54.9 6.151 6.752 1.52 27.105636 0.4203556
## 12 57.2 6.446 7.908 1.90 17.518919 0.6504044
## 8 21.9 6.078 7.966 1.78 -16.162753 0.7049043
## 3 39.0 5.366 5.438 1.57 14.318512 0.7957592
## 18 6.4 5.412 4.700 1.49 -13.609277 0.8371707
## 28 0.7 5.328 3.912 1.25 -11.410452 0.9985018
## 1 12.3 4.543 3.135 0.86 9.990965 1.0000000
## 2 20.9 5.159 5.043 1.53 -1.692664 1.0000000
## 4 47.9 5.759 7.496 1.81 10.648009 1.0000000
## 5 5.6 4.663 3.807 0.99 -1.866642 1.0000000
## 6 25.9 5.697 7.601 1.09 2.632602 1.0000000
## 7 37.3 5.892 8.726 1.29 5.744433 1.0000000
## 9 18.1 4.898 3.850 1.29 4.775657 1.0000000
## 10 21.0 5.242 4.174 1.58 1.048052 1.0000000
## 11 34.9 5.740 6.142 1.68 5.723592 1.0000000