1 Introduction
Estimation of crop areas using pixel counting are subject to at least two sources of errors: mixed border’s pixels and misclassification of pure pixels. Considering the territory of interest is completely covered by satellite imagery, and assuming the effect of mixed border pixels can be neglected, the bias due to misclassification of pure pixel counting on crop areas estimate can be defined with respect to an error matrix, also called a confusion matrix.
Consider the classification of images of the whole territory of interest leading to the identification of \(M\) classes of land covering, so that \(𝑅 = (𝐴_{+1}, 𝐴_{+2}, \cdots, 𝐴_{+M})'\) is an \(𝑀\times 1\) column vector with the total area of pixels classified in each type of class. The actual areas, though, are represented by \(𝑇= (𝐴_{1+}, 𝐴_{2+},\cdots, 𝐴_{N+})'\) an \(N \times 1\) column vector with the truth total area of classes found on the ground.
The confusion matrix (or error matrix) is an \(N \times M\) matrix \(𝐴\) so that its elements are areas classified according to the remote sensing image and the ground truth:
\[A = \begin{pmatrix} A_{11} & A_{12} & \cdots & A_{1M}\\ A_{21} & A_{22} & \cdots & A_{2M} \\ \vdots & \vdots & \ddots & \vdots \\ A_{N1} & A_{N2} & \cdots & A_{NM} \end{pmatrix} \]
All the components can be arranged into a table with the same structure as the partitioned \({(N+1)\times(M+1)}\) matrix given by
\[ \begin{pmatrix} A & T\\ R' & A_{++} \end{pmatrix} \] Figure 1 illustrates an example of such a table with only three classes of land cover: wheat, corn and soy:
knitr::include_graphics('figures/matrix.png')Figure 1. Example of a simple bordered error matrix (confusion matrix).
Error matrix \(𝐴\)’s elements \(𝐴_{gc}\) are areas of class \(g\) (ground) classified as class \(c\) by pixel counting. Hence, in Figure 1, \(𝐴_{11}\), \(𝐴_{22}\), and \(𝐴_{33}\) represent area estimates of wheat, corn, and soy, respectively, using pixels correctly classified as the corresponding ground truth data. In addition, \(𝐴_{21}\) and \(𝐴_{31}\) represent area estimates of wheat based on pixel counting, that are indeed areas of corn and soy, respectively.
The summation \((𝐴_{21} + 𝐴_{31}) = \Phi_1\) is called the comission error related to the wheat area estimate. On the other hand, \(𝐴_{12}\) and \(𝐴_{13}\) are wheat ground area that were mistakenly estimated as corn and soy areas, respectively, using pixel counting. The summation \((𝐴_{12} + 𝐴_{13}) = \Psi_1\) is called the omission error related to wheat estimate.
Error matrix \(𝐴\) and vector \(𝑇\) are unknown. Estimating \(𝑇\) based only on \(𝑅\) is subject to a bias given by the difference between the commission and the omission errors:
\[ \begin{align} \Phi_c &= A_{+c} - A_{cc}\\ \Psi_c &= A_{c+} - A_{cc}\\ B_c &= \Phi_c - \Psi_c \\ &= A_{+c} - A_{c+} \end{align} \]
Denoting by \(B = (B_1, B_2, \cdots, B_M)'\) the column vector of biases for each of the classes’ estimates, one can write, if \(M = N\):
\[ B = R - T \]
Suppose ground information can be observed for a sample of test points, so that a sample \(𝑀 \times N\) error matrix \(a\) is available:
\[a = \begin{pmatrix} a_{11} & a_{12} & \cdots & a_{1M}\\ a_{21} & a_{22} & \cdots & a_{2M} \\ \vdots & \vdots & \ddots & \vdots \\ a_{N1} & a_{N2} & \cdots & a_{NM} \end{pmatrix}\]
Let \(r = (a_{+1}, a_{+2}, \cdots, a_{+M})'\) be an \(M \times 1\) column vector with the total area of sample test pixels classified in each type of class. Let \(t = (a_{+1}, a_{+2}, \cdots, a_{N+})'\) be an \(N \times 1\) column vector with the truth total area of classes found on the sample ground points.
Define the following diagonal matrices over exhaustive information from the whole territory:
\[\begin{eqnarray} D_R &= \text{diag}(A_{+1}, A_{+2}, \cdots, A_{+M})\\ D_T &= \text{diag}(A_{1+}, A_{2+}, \cdots, A_{N+}) \end{eqnarray} \]
1.0.1 Condicional probabilities
The following relative error matrices are defined over the population:
\[ E_{c|g} = A'\cdot D_{T}^{-1} = \begin{pmatrix} \frac{A_{11}}{A_{1+}} & \frac{A_{21}}{A_{2+}} & \cdots & \frac{A_{N1}}{A_{N+}}\\ \frac{A_{12}}{A_{1+}} & \frac{A_{22}}{A_{2+}} & \cdots & \frac{A_{N2}}{A_{N+}}\\ \vdots & \vdots & \ddots & \vdots \\ \frac{A_{1M}}{A_{1+}} & \frac{A_{2M}}{A_{2+}} & \cdots & \frac{A_{NM}}{A_{N+}} \end{pmatrix} \] Note that \(A_{i+} = \sum_j A_{ij}\), \(A_{+j} = \sum_i A_{ij}\) and
\[ 1\!\!1_{1\times M}\cdot E_{c|g} = \begin{pmatrix}\frac{\sum_i A_{1i}}{A_{1+}} & \frac{\sum_i A_{2i}}{A_{2+}} & \cdots & \frac{\sum_i A_{Ni}}{A_{N+}} \end{pmatrix} = 1\!\!1_{1\times N} % \text{ones(1, N)} = \]
Also \(E_c = \text{Pr}(c|g)\) is a \(M\times N\) matrix with the conditional probabilities that a pixel is classified as class \(c\) given the ground truth class is \(g\).
Likewise,
\[ E_{g|c} = A\cdot D_{R}^{-1} = \begin{pmatrix} \frac{A_{11}}{A_{+1}} & \frac{A_{12}}{A_{+2}} & \cdots & \frac{A_{1M}}{A_{+M}}\\ \frac{A_{21}}{A_{+1}} & \frac{A_{22}}{A_{+2}} & \cdots & \frac{A_{2M}}{A_{+M}} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{A_{N1}}{A_{+1}} & \frac{A_{N2}}{A_{+2}} & \cdots & \frac{A_{NM}}{A_{+M}} \end{pmatrix} \] and \(E_{g|c} = \text{Pr}(g|c)\) is a \(N\times M\) matrix with the conditional probability that a pixel is over the ground truth class \(g\) given the pixel is classified as \(c\).
Define the following diagonal matrices over a sample of test points:
\[\begin{eqnarray} D_r &= \text{diag}(a_{+1}, a_{+2}, \cdots, a_{+M})\\ D_t &= \text{diag}(a_{1+}, a_{2+}, \cdots, a_{N+}) \end{eqnarray} \]
Then, \(E_{c|g}\) and \(E_{g|c}\) can be approximately unbiasedly estimated by \(e_{c|g}\) and \(e_g\) given by:
\[ \begin{eqnarray} e_{c|g} &= a'\cdot D_t^{-1}\\ e_{g|c} &= a\cdot D_r^{-1}\\ \end{eqnarray} \]
1.0.2 Estimators
The following calibration estimators can be defined:
The direct calibration estimator:
\[ \hat{T}_{\text{direct}} = e_{g|c} \cdot R = \begin{pmatrix} \frac{a_{11}}{a_{+1}}A_{+1} + \frac{a_{12}}{a_{+2}}A_{+2} + \cdots + \frac{a_{1M}}{a_{1M}}A_{+M}\\ \frac{a_{21}}{a_{+1}}A_{+1} + \frac{a_{22}}{a_{+2}}A_{+2} + \cdots + \frac{a_{2M}}{a_{+M}}A_{+M} \\ \vdots \\ \frac{a_{N1}}{a_{+1}}A_{+1} + \frac{a_{N2}}{a_{+2}}A_{+2} + \cdots + \frac{a_{NM}}{a_{+M}}A_{+M} \end{pmatrix} \]
The inverse calibration estimator:
\[ \hat{T}_{\text{inverse}} = e_{c|g}^{+} \cdot R = \begin{pmatrix} \frac{a_{11}}{a_{1+}} & \frac{a_{21}}{a_{2+}} & \cdots & \frac{a_{N1}}{a_{N+}}\\ \frac{a_{12}}{a_{1+}} & \frac{a_{22}}{a_{2+}} & \cdots & \frac{a_{N2}}{a_{N+}}\\ \vdots & \vdots & \ddots & \vdots \\ \frac{a_{1M}}{a_{1+}} & \frac{a_{2M}}{a_{2+}} & \cdots & \frac{a_{NM}}{a_{N+}} \end{pmatrix}^{+}\cdot \begin{pmatrix} A_{+1} \\ A_{+2} \\ \vdots \\ A_{+M}\end{pmatrix} \]
where \(e_{c|g}^+\) is a left inverse \(M \times N\) matrix of \(e_{c|g}\), such that \(e_{c|g}^+ \cdot e_{c|g} = \text{I}\!\text{I}_N\). It can be found using:
\[ e_{c|g}^+ = (e'_{c|g} \cdot e_{c|g})^{-1}\cdot e'_{c|g} \]
2 Simulations
2.1 Packages
library(pracma)
library(tidyverse)
library(readxl)
library(glmnet)
library(knitr)
library(kableExtra)
library(plotly)2.2 Reading the data
FN1.tbl <- read_excel("data/CropType2018_PixelCounting.xlsx",
sheet = "Feuil1", range = "E2:F8", col_names = FALSE,
col_types = c("text", "numeric")) %>%
rename(crop = ...1, value = ...2) %>%
mutate(crop = recode(crop,
'Mill' = 'Millet',
'Groundnut' = 'Groundnuts',
'Coton' = 'Cotton'))
FN2.tbl <- read_excel("data/CropType2018_ConfusionMatrix.xlsx",
sheet = "Matrice_Regroupement 1",
range = "B2:I9") %>%
rename(crop = ...1) %>%
mutate(crop = recode(crop,
'Mill' = 'Millet',
'Groundnut' = 'Groundnuts',
'Coton' = 'Cotton')) %>%
rename(Cotton = Coton, Groundnuts = Groundnut)FN1.tbl %>%
mutate(value = format(value, nsmall = 0, big.mark = ",", digits = 10)) %>%
arrange(desc(value)) %>%
kable(caption = "Table FN1", align = "lr")%>%
kable_styling(bootstrap_options =
c("striped", "hover", "condensed", "responsive"),
full_width = FALSE, font_size = 12)| crop | value |
|---|---|
| Cotton | 102,803,268 |
| Maize | 99,454,021 |
| Millet | 68,759,277 |
| Sorghum | 10,911,297 |
| Rice | 9,654,264 |
| Other crops | 5,373,340 |
| Groundnuts | 651,230 |
FN2.tbl %>% kbl(caption = "Table FN2") %>%
kable_styling(bootstrap_options =
c("striped", "hover", "condensed", "responsive"),
full_width = FALSE, font_size = 12) %>%
column_spec(1, bold = TRUE)| crop | Maize | Other crops | Groundnuts | Cotton | Rice | Sorghum | Millet |
|---|---|---|---|---|---|---|---|
| Maize | 9713 | 954 | 29 | 2094 | 144 | 20 | 602 |
| Other crops | 2125 | 2448 | 95 | 2598 | 35 | 78 | 297 |
| Groundnuts | 441 | 85 | 419 | 464 | 19 | 37 | 41 |
| Cotton | 2370 | 541 | 114 | 12770 | 88 | 121 | 255 |
| Rice | 514 | 21 | 2 | 327 | 887 | 23 | 0 |
| Sorghum | 531 | 133 | 32 | 584 | 12 | 888 | 335 |
| Millet | 668 | 27 | 4 | 426 | 14 | 83 | 3510 |
2.3 Matrices
#Leg <- FN3.tbl
R_vector <-
FN1.tbl %>%
rename(R = value) %>%
column_to_rownames(var = "crop") %>%
as.matrix()
a_matrix <- FN2.tbl %>%
column_to_rownames(var = "crop") %>%
as.matrix() # %>% t()
R_vector %>%
as.data.frame() %>%
rownames_to_column(var = "crop") %>%
mutate(R = format(R, nsmall = 0, big.mark = ",", digits = 10)) %>%
kable(caption = "Table TC", align = "lr") %>%
kable_styling(bootstrap_options =
c("striped", "hover", "condensed", "responsive"),
full_width = FALSE, font_size = 12)| crop | R |
|---|---|
| Maize | 99,454,021 |
| Other crops | 5,373,340 |
| Groundnuts | 651,230 |
| Cotton | 102,803,268 |
| Rice | 9,654,264 |
| Sorghum | 10,911,297 |
| Millet | 68,759,277 |
a_matrix %>%
as.data.frame() %>%
rownames_to_column(var = "crop") %>%
kable(caption = "Table PCG") %>%
kable_styling(bootstrap_options =
c("striped", "hover", "condensed", "responsive"),
full_width = FALSE, font_size = 12) %>%
column_spec(1, bold = TRUE)| crop | Maize | Other crops | Groundnuts | Cotton | Rice | Sorghum | Millet |
|---|---|---|---|---|---|---|---|
| Maize | 9713 | 954 | 29 | 2094 | 144 | 20 | 602 |
| Other crops | 2125 | 2448 | 95 | 2598 | 35 | 78 | 297 |
| Groundnuts | 441 | 85 | 419 | 464 | 19 | 37 | 41 |
| Cotton | 2370 | 541 | 114 | 12770 | 88 | 121 | 255 |
| Rice | 514 | 21 | 2 | 327 | 887 | 23 | 0 |
| Sorghum | 531 | 133 | 32 | 584 | 12 | 888 | 335 |
| Millet | 668 | 27 | 4 | 426 | 14 | 83 | 3510 |
2.4 ConfVarEst function
ConfVarEst <- function(a_matrix, R_vector, replications = 100){
# a = confusion matrix
# R = classification vecotr
dim_a <- dim(a_matrix)
N = dim_a[1]
M = dim_a[2]
r_vec <- colSums(a_matrix)
t_vec <- rowSums(a_matrix)
D_r = diag(r_vec)
D_t = diag(t_vec)
iD_r = solve(D_r) # == diag(1/r_vec)
iD_t = solve(D_t) # == diag(1/t_vec)
e_c_dado_g = t(a_matrix) %*% iD_t
e_g_dado_c = a_matrix %*% iD_r
# Direct estimator
T_hat_direct = e_g_dado_c %*% R_vector
# Inverse estimator
T_hat_inverse = solve(e_c_dado_g) %*% R_vector
# T_hat_inverse = (solve((t(e_c_dado_g) %*% e_c_dado_g)) %*% t(e_c_dado_g)) %*% R_vector
rownames(T_hat_inverse) <- rownames(T_hat_direct)
# Bootstrapping
edges_g_dado_c <- rbind(matrix(rep(0, M), nrow = 1), apply(e_g_dado_c, 2, cumsum))
edges_c_dado_g <- rbind(matrix(rep(0, N), nrow = 1), apply(e_c_dado_g, 2, cumsum))
T_hat_direct_boot <- NULL
T_hat_inverse_boot <- NULL
e_g_dado_c_sampled <- array(dim = c(N, M))
e_c_dado_g_sampled <- array(dim = c(M, N))
for (i in 1:replications){
# Resampling on each ground class sepatately
for (j in 1:M) {
numbers <- matrix(runif(r_vec[j]), ncol = 1)
Eff <- pracma::histc(numbers, edges_g_dado_c[,j])$cnt
e_g_dado_c_sampled[,j] <- Eff[-(N + 1)]/r_vec[j]
}
for (j in 1:N) {
numbers <- matrix(runif(t_vec[j]), ncol = 1)
Eff <- pracma::histc(numbers, edges_c_dado_g[,j])$cnt
e_c_dado_g_sampled[,j] <- Eff[-(N + 1)]/t_vec[j]
}
# Direct estimator
direct = e_g_dado_c_sampled %*% R_vector
# Indirect estimator
inverse = solve(e_c_dado_g_sampled) %*% R_vector
T_hat_direct_boot <- cbind(T_hat_direct_boot, direct)
T_hat_inverse_boot <- cbind(T_hat_inverse_boot, inverse)
}
rownames(T_hat_direct_boot) <- rownames(T_hat_direct)
rownames(T_hat_inverse_boot) <- rownames(T_hat_inverse)
mean_direct <- rowMeans(T_hat_direct_boot)
names(mean_direct) <- rownames(T_hat_direct)
mean_inverse <- rowMeans(T_hat_inverse_boot)
names(mean_inverse) <- rownames(T_hat_inverse)
std_direct <- apply(T_hat_direct_boot, 1, function(x) sqrt(var(x)))
names(std_direct) <- rownames(T_hat_direct)
std_inverse <- apply(T_hat_inverse_boot, 1, function(x) sqrt(var(x)))
names(std_inverse) <- rownames(T_hat_inverse)
cv_direct <- apply(T_hat_direct_boot, 1, function(x) 100*sqrt(var(x))/mean(x))
names(cv_direct) <- rownames(T_hat_direct)
cv_inverse <- apply(T_hat_inverse_boot, 1, function(x) 100*sqrt(var(x))/mean(x))
names(cv_inverse) <- rownames(T_hat_inverse)
return(list(direct = T_hat_direct,
inverse = T_hat_inverse,
direct_boot = T_hat_direct_boot,
inverse_boot = T_hat_inverse_boot,
direct_mean = mean_direct,
inverse_mean = mean_inverse,
direct_std = std_direct,
inverse_std = std_inverse,
direct_cv = cv_direct,
inverse_cv = cv_inverse
)
)
}2.5 Results
result <- ConfVarEst(a_matrix, R_vector, replications = 100)
format_values <-
function(x, y)
format(x, nsmall = y, big.mark = ",", digits = 10)
result[c("direct", "direct_mean", "direct_std", "direct_cv")] %>%
as.data.frame() %>%
rownames_to_column(var = "Crop") %>%
rename(Estimate = R, Mean = direct_mean, 'Deviation' = direct_std, CV = direct_cv) %>%
mutate(Estimate = format_values(round(Estimate), 0),
Mean = format_values(round(Mean), 0),
Deviation = format_values(round(Deviation), 0),
CV = format_values(round(CV,2), 2)) %>%
kable(caption = "Direct Estimator Bootstrap Results (100 rounds)", align = "lrrr") %>%
kable_styling(bootstrap_options =
c("striped", "hover", "condensed", "responsive"),
full_width = FALSE, font_size = 12)| Crop | Estimate | Mean | Deviation | CV |
|---|---|---|---|---|
| Maize | 81,006,413 | 80,939,610 | 542,075 | 0.67 |
| Other crops | 35,010,352 | 34,991,148 | 422,639 | 1.21 |
| Groundnuts | 6,693,279 | 6,698,570 | 184,120 | 2.75 |
| Cotton | 88,598,112 | 88,655,214 | 484,162 | 0.55 |
| Rice | 12,240,929 | 12,267,055 | 214,055 | 1.74 |
| Sorghum | 18,962,406 | 18,990,938 | 350,274 | 1.84 |
| Millet | 55,095,207 | 55,064,162 | 499,337 | 0.91 |
result[c("inverse", "inverse_mean", "inverse_std", "inverse_cv")] %>%
as.data.frame() %>%
rownames_to_column(var = "Crop") %>%
rename(Estimate = R, Mean = inverse_mean, 'Deviation' = inverse_std, CV = inverse_cv) %>%
mutate(Estimate = format_values(round(Estimate), 0),
Mean = format_values(round(Mean), 0),
Deviation = format_values(round(Deviation), 0),
CV = format_values(round(CV,2), 2)) %>%
kable(caption = "Inverse Estimator Bootstrap Results (100 rounds)", align = "lrrr") %>%
kable_styling(bootstrap_options =
c("striped", "hover", "condensed", "responsive"),
full_width = FALSE, font_size = 12)| Crop | Estimate | Mean | Deviation | CV |
|---|---|---|---|---|
| Maize | 97,353,971 | 97,426,625 | 1,321,943 | 1.36 |
| Other crops | -21,021,744 | -20,962,776 | 1,183,052 | -5.64 |
| Groundnuts | -1,450,389 | -1,476,966 | 387,175 | -26.21 |
| Cotton | 101,128,468 | 100,964,505 | 1,001,341 | 0.99 |
| Rice | 15,658,343 | 15,735,261 | 459,216 | 2.92 |
| Sorghum | 24,447,448 | 24,455,953 | 785,448 | 3.21 |
| Millet | 81,490,601 | 81,464,095 | 848,480 | 1.04 |
df_direct <- result$direct_boot %>% t() %>%
as.matrix() %>%
as_tibble() %>%
pivot_longer(everything(), names_to = "crop", values_to = "value")
df_inverse <- result$inverse_boot %>% t() %>%
as.matrix() %>%
as_tibble() %>%
pivot_longer(everything(), names_to = "crop", values_to = "value")
p_direct <- df_direct %>% ggplot(aes(x = value)) + geom_histogram(alpha=0.25, position="identity", bins = 30, aes(color = crop, fill = crop)) + facet_wrap(. ~ crop,nrow = 4, scales = "free") + theme_bw() + guides(fill= 'none', color = 'none') + theme(strip.background = element_rect(fill = gray(.95))) + scale_x_continuous(labels = function(x) format(x, scientific = TRUE), n.breaks = 4) + ggtitle("Bootstrap replications for direct estimates") + theme(panel.spacing = unit(1, "lines"))
p_inverse <- df_inverse %>% ggplot(aes(x = value)) + geom_histogram(alpha=0.25, position="identity", bins = 30, aes(color = crop, fill = crop)) + facet_wrap(. ~ crop,nrow = 4, scales = "free") + theme_bw() + guides(fill= 'none', color = 'none') + theme(strip.background = element_rect(fill = gray(.95))) + scale_x_continuous(labels = function(x) format(x, scientific = TRUE), n.breaks = 4) + ggtitle("Bootstrap replications for inverse estimates") + theme(panel.spacing = unit(1, "lines"))
ggplotly(p_direct,tooltip = c("y")) %>%
layout(showlegend = FALSE) %>%
layout(plot_bgcolor='rgba(254, 254, 254, .25)') %>%
layout(paper_bgcolor='#eeeeee') %>%
config(displayModeBar = FALSE)ggplotly(p_inverse,tooltip = c("y")) %>%
layout(showlegend = FALSE) %>%
layout(plot_bgcolor='rgba(254, 254, 254, .25)') %>%
layout(paper_bgcolor='#eeeeee') %>%
config(displayModeBar = FALSE)3 Stay Tuned
Please visit the CastLAB page for latest updates and news. Comments, bug reports and pull requests are always welcome.