1 Introduction

Estimation of crop areas using pixel counting are subject to at least two sources of errors: mixed border’s pixels and misclassification of pure pixels. Considering the territory of interest is completely covered by satellite imagery, and assuming the effect of mixed border pixels can be neglected, the bias due to misclassification of pure pixel counting on crop areas estimate can be defined with respect to an error matrix, also called a confusion matrix.

Consider the classification of images of the whole territory of interest leading to the identification of \(M\) classes of land covering, so that \(𝑅 = (𝐴_{+1}, 𝐴_{+2}, \cdots, 𝐴_{+M})'\) is an \(𝑀\times 1\) column vector with the total area of pixels classified in each type of class. The actual areas, though, are represented by \(𝑇= (𝐴_{1+}, 𝐴_{2+},\cdots, 𝐴_{N+})'\) an \(N \times 1\) column vector with the truth total area of classes found on the ground.

The confusion matrix (or error matrix) is an \(N \times M\) matrix \(𝐴\) so that its elements are areas classified according to the remote sensing image and the ground truth:

\[A = \begin{pmatrix} A_{11} & A_{12} & \cdots & A_{1M}\\ A_{21} & A_{22} & \cdots & A_{2M} \\ \vdots & \vdots & \ddots & \vdots \\ A_{N1} & A_{N2} & \cdots & A_{NM} \end{pmatrix} \]

All the components can be arranged into a table with the same structure as the partitioned \({(N+1)\times(M+1)}\) matrix given by

\[ \begin{pmatrix} A & T\\ R' & A_{++} \end{pmatrix} \] Figure 1 illustrates an example of such a table with only three classes of land cover: wheat, corn and soy:

knitr::include_graphics('figures/matrix.png')
Figure 1. Example of a simple bordered error matrix (confusion matrix).

Figure 1. Example of a simple bordered error matrix (confusion matrix).

Error matrix \(𝐴\)’s elements \(𝐴_{gc}\) are areas of class \(g\) (ground) classified as class \(c\) by pixel counting. Hence, in Figure 1, \(𝐴_{11}\), \(𝐴_{22}\), and \(𝐴_{33}\) represent area estimates of wheat, corn, and soy, respectively, using pixels correctly classified as the corresponding ground truth data. In addition, \(𝐴_{21}\) and \(𝐴_{31}\) represent area estimates of wheat based on pixel counting, that are indeed areas of corn and soy, respectively.

The summation \((𝐴_{21} + 𝐴_{31}) = \Phi_1\) is called the comission error related to the wheat area estimate. On the other hand, \(𝐴_{12}\) and \(𝐴_{13}\) are wheat ground area that were mistakenly estimated as corn and soy areas, respectively, using pixel counting. The summation \((𝐴_{12} + 𝐴_{13}) = \Psi_1\) is called the omission error related to wheat estimate.

Error matrix \(𝐴\) and vector \(𝑇\) are unknown. Estimating \(𝑇\) based only on \(𝑅\) is subject to a bias given by the difference between the commission and the omission errors:

\[ \begin{align} \Phi_c &= A_{+c} - A_{cc}\\ \Psi_c &= A_{c+} - A_{cc}\\ B_c &= \Phi_c - \Psi_c \\ &= A_{+c} - A_{c+} \end{align} \]

Denoting by \(B = (B_1, B_2, \cdots, B_M)'\) the column vector of biases for each of the classes’ estimates, one can write, if \(M = N\):

\[ B = R - T \]

Suppose ground information can be observed for a sample of test points, so that a sample \(𝑀 \times N\) error matrix \(a\) is available:

\[a = \begin{pmatrix} a_{11} & a_{12} & \cdots & a_{1M}\\ a_{21} & a_{22} & \cdots & a_{2M} \\ \vdots & \vdots & \ddots & \vdots \\ a_{N1} & a_{N2} & \cdots & a_{NM} \end{pmatrix}\]

Let \(r = (a_{+1}, a_{+2}, \cdots, a_{+M})'\) be an \(M \times 1\) column vector with the total area of sample test pixels classified in each type of class. Let \(t = (a_{+1}, a_{+2}, \cdots, a_{N+})'\) be an \(N \times 1\) column vector with the truth total area of classes found on the sample ground points.

Define the following diagonal matrices over exhaustive information from the whole territory:

\[\begin{eqnarray} D_R &= \text{diag}(A_{+1}, A_{+2}, \cdots, A_{+M})\\ D_T &= \text{diag}(A_{1+}, A_{2+}, \cdots, A_{N+}) \end{eqnarray} \]

1.0.1 Condicional probabilities

The following relative error matrices are defined over the population:

\[ E_{c|g} = A'\cdot D_{T}^{-1} = \begin{pmatrix} \frac{A_{11}}{A_{1+}} & \frac{A_{21}}{A_{2+}} & \cdots & \frac{A_{N1}}{A_{N+}}\\ \frac{A_{12}}{A_{1+}} & \frac{A_{22}}{A_{2+}} & \cdots & \frac{A_{N2}}{A_{N+}}\\ \vdots & \vdots & \ddots & \vdots \\ \frac{A_{1M}}{A_{1+}} & \frac{A_{2M}}{A_{2+}} & \cdots & \frac{A_{NM}}{A_{N+}} \end{pmatrix} \] Note that \(A_{i+} = \sum_j A_{ij}\), \(A_{+j} = \sum_i A_{ij}\) and

\[ 1\!\!1_{1\times M}\cdot E_{c|g} = \begin{pmatrix}\frac{\sum_i A_{1i}}{A_{1+}} & \frac{\sum_i A_{2i}}{A_{2+}} & \cdots & \frac{\sum_i A_{Ni}}{A_{N+}} \end{pmatrix} = 1\!\!1_{1\times N} % \text{ones(1, N)} = \]

Also \(E_c = \text{Pr}(c|g)\) is a \(M\times N\) matrix with the conditional probabilities that a pixel is classified as class \(c\) given the ground truth class is \(g\).

Likewise,

\[ E_{g|c} = A\cdot D_{R}^{-1} = \begin{pmatrix} \frac{A_{11}}{A_{+1}} & \frac{A_{12}}{A_{+2}} & \cdots & \frac{A_{1M}}{A_{+M}}\\ \frac{A_{21}}{A_{+1}} & \frac{A_{22}}{A_{+2}} & \cdots & \frac{A_{2M}}{A_{+M}} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{A_{N1}}{A_{+1}} & \frac{A_{N2}}{A_{+2}} & \cdots & \frac{A_{NM}}{A_{+M}} \end{pmatrix} \] and \(E_{g|c} = \text{Pr}(g|c)\) is a \(N\times M\) matrix with the conditional probability that a pixel is over the ground truth class \(g\) given the pixel is classified as \(c\).

Define the following diagonal matrices over a sample of test points:

\[\begin{eqnarray} D_r &= \text{diag}(a_{+1}, a_{+2}, \cdots, a_{+M})\\ D_t &= \text{diag}(a_{1+}, a_{2+}, \cdots, a_{N+}) \end{eqnarray} \]

Then, \(E_{c|g}\) and \(E_{g|c}\) can be approximately unbiasedly estimated by \(e_{c|g}\) and \(e_g\) given by:

\[ \begin{eqnarray} e_{c|g} &= a'\cdot D_t^{-1}\\ e_{g|c} &= a\cdot D_r^{-1}\\ \end{eqnarray} \]

1.0.2 Estimators

The following calibration estimators can be defined:

The direct calibration estimator:

\[ \hat{T}_{\text{direct}} = e_{g|c} \cdot R = \begin{pmatrix} \frac{a_{11}}{a_{+1}}A_{+1} + \frac{a_{12}}{a_{+2}}A_{+2} + \cdots + \frac{a_{1M}}{a_{1M}}A_{+M}\\ \frac{a_{21}}{a_{+1}}A_{+1} + \frac{a_{22}}{a_{+2}}A_{+2} + \cdots + \frac{a_{2M}}{a_{+M}}A_{+M} \\ \vdots \\ \frac{a_{N1}}{a_{+1}}A_{+1} + \frac{a_{N2}}{a_{+2}}A_{+2} + \cdots + \frac{a_{NM}}{a_{+M}}A_{+M} \end{pmatrix} \]

The inverse calibration estimator:

\[ \hat{T}_{\text{inverse}} = e_{c|g}^{+} \cdot R = \begin{pmatrix} \frac{a_{11}}{a_{1+}} & \frac{a_{21}}{a_{2+}} & \cdots & \frac{a_{N1}}{a_{N+}}\\ \frac{a_{12}}{a_{1+}} & \frac{a_{22}}{a_{2+}} & \cdots & \frac{a_{N2}}{a_{N+}}\\ \vdots & \vdots & \ddots & \vdots \\ \frac{a_{1M}}{a_{1+}} & \frac{a_{2M}}{a_{2+}} & \cdots & \frac{a_{NM}}{a_{N+}} \end{pmatrix}^{+}\cdot \begin{pmatrix} A_{+1} \\ A_{+2} \\ \vdots \\ A_{+M}\end{pmatrix} \]

where \(e_{c|g}^+\) is a left inverse \(M \times N\) matrix of \(e_{c|g}\), such that \(e_{c|g}^+ \cdot e_{c|g} = \text{I}\!\text{I}_N\). It can be found using:

\[ e_{c|g}^+ = (e'_{c|g} \cdot e_{c|g})^{-1}\cdot e'_{c|g} \]

2 Simulations

2.1 Packages

library(pracma)
library(tidyverse)
library(readxl)
library(glmnet)
library(knitr)
library(kableExtra)
library(plotly)

2.2 Reading the data

FN1.tbl <- read_excel("data/CropType2018_PixelCounting.xlsx",
                      sheet = "Feuil1", range = "E2:F8", col_names = FALSE,
                      col_types = c("text", "numeric")) %>%
  rename(crop = ...1, value = ...2) %>% 
  mutate(crop = recode(crop, 
                       'Mill' = 'Millet', 
                       'Groundnut' = 'Groundnuts', 
                       'Coton' = 'Cotton'))

FN2.tbl <- read_excel("data/CropType2018_ConfusionMatrix.xlsx", 
                      sheet = "Matrice_Regroupement 1", 
                      range = "B2:I9") %>% 
  rename(crop = ...1) %>% 
  mutate(crop = recode(crop, 
                       'Mill' = 'Millet', 
                       'Groundnut' = 'Groundnuts', 
                       'Coton' = 'Cotton')) %>% 
  rename(Cotton = Coton, Groundnuts = Groundnut)
FN1.tbl %>% 
  mutate(value = format(value, nsmall = 0, big.mark = ",", digits = 10)) %>% 
  arrange(desc(value)) %>% 
  kable(caption = "Table FN1", align = "lr")%>%
  kable_styling(bootstrap_options = 
                  c("striped", "hover", "condensed", "responsive"), 
                full_width = FALSE, font_size = 12)
Table FN1
crop value
Cotton 102,803,268
Maize 99,454,021
Millet 68,759,277
Sorghum 10,911,297
Rice 9,654,264
Other crops 5,373,340
Groundnuts 651,230
FN2.tbl %>% kbl(caption = "Table FN2") %>%
    kable_styling(bootstrap_options = 
                      c("striped", "hover", "condensed", "responsive"), 
                  full_width = FALSE, font_size = 12) %>% 
  column_spec(1, bold = TRUE)
Table FN2
crop Maize Other crops Groundnuts Cotton Rice Sorghum Millet
Maize 9713 954 29 2094 144 20 602
Other crops 2125 2448 95 2598 35 78 297
Groundnuts 441 85 419 464 19 37 41
Cotton 2370 541 114 12770 88 121 255
Rice 514 21 2 327 887 23 0
Sorghum 531 133 32 584 12 888 335
Millet 668 27 4 426 14 83 3510

2.3 Matrices

#Leg <- FN3.tbl 

R_vector <- 
  FN1.tbl %>%
  rename(R = value) %>% 
  column_to_rownames(var = "crop") %>%
  as.matrix() 

a_matrix <- FN2.tbl %>% 
  column_to_rownames(var = "crop") %>%
  as.matrix() # %>% t()

R_vector %>% 
  as.data.frame() %>% 
  rownames_to_column(var = "crop") %>% 
  mutate(R = format(R, nsmall = 0, big.mark = ",", digits = 10)) %>% 
  kable(caption = "Table TC", align = "lr") %>%
  kable_styling(bootstrap_options =
                  c("striped", "hover", "condensed", "responsive"), 
                full_width = FALSE, font_size = 12)
Table TC
crop R
Maize 99,454,021
Other crops 5,373,340
Groundnuts 651,230
Cotton 102,803,268
Rice 9,654,264
Sorghum 10,911,297
Millet 68,759,277
a_matrix %>% 
  as.data.frame() %>% 
  rownames_to_column(var = "crop") %>% 
  kable(caption = "Table PCG") %>% 
  kable_styling(bootstrap_options = 
                  c("striped", "hover", "condensed", "responsive"), 
                full_width = FALSE, font_size = 12) %>% 
  column_spec(1, bold = TRUE)
Table PCG
crop Maize Other crops Groundnuts Cotton Rice Sorghum Millet
Maize 9713 954 29 2094 144 20 602
Other crops 2125 2448 95 2598 35 78 297
Groundnuts 441 85 419 464 19 37 41
Cotton 2370 541 114 12770 88 121 255
Rice 514 21 2 327 887 23 0
Sorghum 531 133 32 584 12 888 335
Millet 668 27 4 426 14 83 3510

2.4 ConfVarEst function

ConfVarEst <- function(a_matrix, R_vector, replications = 100){
  # a <- confusion matrix
  # R <- classification vecotr
  dim_a <- dim(a_matrix)
  N_g <- dim_a[1]
  N_c <- dim_a[2]
  N <- sum(a_matrix)
  
  
  r_vec <- colSums(a_matrix)
  t_vec <- rowSums(a_matrix)
  D_r <- diag(r_vec)
  D_t <- diag(t_vec)
  iD_r <- solve(D_r) # == diag(1/r_vec)
  iD_t <- solve(D_t) # == diag(1/t_vec)
  
  e_c_dado_g <- t(a_matrix) %*% iD_t
  e_g_dado_c <- a_matrix %*% iD_r
  
  # Direct estimator
  T_hat_direct = e_g_dado_c %*% R_vector
  
  # Inverse estimator
  T_hat_inverse <- solve(e_c_dado_g, R_vector) # solve(e_c_dado_g) %*% R_vector
  # T_hat_inverse = (solve((t(e_c_dado_g) %*% e_c_dado_g)) %*% t(e_c_dado_g))  %*% R_vector
  rownames(T_hat_inverse) <- rownames(T_hat_direct)
  
  # Bootstrapping 
  edges_c_dado_g <- rbind(matrix(rep(0, N_g), nrow = 1), apply(e_c_dado_g, 2, cumsum))
  edges_global <- c(0, cumsum(pracma::Reshape(a_matrix/N, N_g*N_c, 1)))
  
  T_hat_direct_boot <- NULL
  T_hat_inverse_boot <- NULL
  
  e_c_dado_g_sampled <- array(dim = c(N_c, N_g))
  
  for (i in 1:replications){
    # Resampling globally for direct estimator
    numbers <- runif(N)
    Eff <- pracma::Reshape(pracma::histc(numbers, edges_global)$cnt[-(N_g*N_c + 1)], N_g, N_c) 
    r_sampled <- colSums(Eff)
    e_g_dado_c_sampled <- Eff %*% diag(1/r_sampled) 
    # Resampling on each ground class separately for indirect estimator
    rm(numbers, r_sampled, Eff)
    for (j in 1:N_g) {
      numbers <- matrix(runif(t_vec[j]), ncol = 1)
      Eff <- pracma::histc(numbers, edges_c_dado_g[,j])$cnt[-(N_g + 1)]
      e_c_dado_g_sampled[j,] <- Eff/t_vec[j]
    }
    rm(numbers, Eff)
    # Direct estimator
    direct = e_g_dado_c_sampled %*% R_vector
    
    # Indirect estimator
    inverse =  solve(e_c_dado_g_sampled, R_vector) 
             # solve(e_c_dado_g_sampled) %*% R_vector
    
    T_hat_direct_boot <- cbind(T_hat_direct_boot, direct)
    T_hat_inverse_boot <- cbind(T_hat_inverse_boot, inverse)
  }
  
  rownames(T_hat_direct_boot) <- rownames(T_hat_direct)
  rownames(T_hat_inverse_boot) <- rownames(T_hat_inverse)
  
  
  mean_direct <- rowMeans(T_hat_direct_boot)
  names(mean_direct) <- rownames(T_hat_direct)
  
  mean_inverse <- rowMeans(T_hat_inverse_boot)
  names(mean_inverse) <- rownames(T_hat_inverse)
  
  std_direct <- apply(T_hat_direct_boot, 1, function(x) sqrt(var(x)))
  names(std_direct) <- rownames(T_hat_direct)
  
  std_inverse <- apply(T_hat_inverse_boot, 1, function(x) sqrt(var(x)))
  names(std_inverse) <- rownames(T_hat_inverse)
  
  
  cv_direct <- apply(T_hat_direct_boot, 1, function(x) 100*sqrt(var(x))/mean(x))
  names(cv_direct) <- rownames(T_hat_direct)
  
  cv_inverse <- apply(T_hat_inverse_boot, 1, function(x) 100*sqrt(var(x))/mean(x))
  names(cv_inverse) <- rownames(T_hat_inverse)
  
  return(list(direct = T_hat_direct, 
              inverse = T_hat_inverse, 
              direct_boot = T_hat_direct_boot, 
              inverse_boot = T_hat_inverse_boot,
              direct_mean = mean_direct,
              inverse_mean = mean_inverse,
              direct_std = std_direct,
              inverse_std = std_inverse,
              direct_cv = cv_direct,
              inverse_cv = cv_inverse
  )
  )
}

2.5 Results

result <- ConfVarEst(a_matrix, R_vector, replications = 100)

format_values <- 
  function(x, y) 
    format(x, nsmall = y, big.mark = ",", digits = 10)

result[c("direct", "direct_mean", "direct_std", "direct_cv")] %>% 
  as.data.frame() %>% 
  rownames_to_column(var = "Crop") %>% 
  rename(Estimate = R, Mean = direct_mean, 'Deviation' = direct_std, CV = direct_cv) %>% 
  mutate(Estimate = format_values(round(Estimate), 0),
         Mean = format_values(round(Mean), 0),
         Deviation = format_values(round(Deviation), 0),
         CV = format_values(round(CV,2), 2)) %>% 
  kable(caption = "Direct Estimator Bootstrap Results (100 rounds)", align = "lrrr") %>%
  kable_styling(bootstrap_options = 
                  c("striped", "hover", "condensed", "responsive"), 
                full_width = FALSE, font_size = 12)
Direct Estimator Bootstrap Results (100 rounds)
Crop Estimate Mean Deviation CV
Maize 81,006,413 81,029,167 587,794 0.73
Other crops 35,010,352 34,985,312 458,959 1.31
Groundnuts 6,693,279 6,726,816 196,998 2.93
Cotton 88,598,112 88,581,551 525,066 0.59
Rice 12,240,929 12,278,423 218,841 1.78
Sorghum 18,962,406 18,924,785 304,364 1.61
Millet 55,095,207 55,080,643 509,736 0.93
result[c("inverse", "inverse_mean", "inverse_std", "inverse_cv")] %>%
  as.data.frame() %>%
  rownames_to_column(var = "Crop") %>%
  rename(Estimate = R, Mean = inverse_mean, 'Deviation' = inverse_std, CV = inverse_cv) %>%
  mutate(Estimate = format_values(round(Estimate), 0),
         Mean = format_values(round(Mean), 0),
         Deviation = format_values(round(Deviation), 0),
         CV = format_values(round(CV,2), 2)) %>%
  kable(caption = "Inverse Estimator Bootstrap Results (100 rounds)", align = "lrrr") %>%
  column_spec(3, color = if_else(result$inverse_mean < 0, 'red', 'black')) %>%  
  column_spec(2, color = if_else(result$inverse < 0, 'red', 'black')) %>%  
  kable_styling(bootstrap_options =
                  c("striped", "hover", "condensed", "responsive"),
                full_width = FALSE, font_size = 12)
Inverse Estimator Bootstrap Results (100 rounds)
Crop Estimate Mean Deviation CV
Maize 97,353,971 133,497,433 1,371,336 1.03
Other crops -21,021,744 -217,775,044 6,199,153 -2.85
Groundnuts -1,450,389 -217,916,623 14,827,945 -6.80
Cotton 101,128,468 117,891,006 847,112 0.72
Rice 15,658,343 -93,391,758 5,462,659 -5.85
Sorghum 24,447,448 -106,832,648 7,053,496 -6.60
Millet 81,490,601 57,782,769 894,686 1.55
df_direct <- result$direct_boot %>% t() %>% 
  as.matrix() %>% 
  as_tibble() %>% 
  pivot_longer(everything(), names_to = "crop", values_to = "value") 

df_inverse <- result$inverse_boot %>% t() %>%
  as.matrix() %>%
  as_tibble() %>%
  pivot_longer(everything(), names_to = "crop", values_to = "value")

p_direct <-  df_direct %>% ggplot(aes(x = value)) + geom_histogram(alpha=0.25, position="identity", bins = 30, aes(color = crop, fill = crop)) + facet_wrap(. ~ crop,nrow = 4, scales = "free") + theme_bw() + guides(fill= 'none', color = 'none') + theme(strip.background = element_rect(fill = gray(.95)))  +  scale_x_continuous(labels = function(x) format(x, scientific = TRUE), n.breaks = 4) + ggtitle("Bootstrap replications for direct estimates") + theme(panel.spacing = unit(1, "lines"))


p_inverse <-  df_inverse %>% ggplot(aes(x = value)) + geom_histogram(alpha=0.25, position="identity", bins = 30, aes(color = crop, fill = crop)) + facet_wrap(. ~ crop,nrow = 4, scales = "free") + theme_bw() + guides(fill= 'none', color = 'none') + theme(strip.background = element_rect(fill = gray(.95)))  +  scale_x_continuous(labels = function(x) format(x, scientific = TRUE), n.breaks = 4) + ggtitle("Bootstrap replications for inverse estimates") + theme(panel.spacing = unit(1, "lines"))

ggplotly(p_direct,tooltip = c("y")) %>% 
  layout(showlegend = FALSE) %>% 
  layout(plot_bgcolor='rgba(254, 254, 254, .25)') %>% 
  layout(paper_bgcolor='#eeeeee') %>% 
  config(displayModeBar = FALSE)
ggplotly(p_inverse,tooltip = c("y")) %>%
  layout(showlegend = FALSE) %>%
  layout(plot_bgcolor='rgba(254, 254, 254, .25)') %>%
  layout(paper_bgcolor='#eeeeee') %>%
  config(displayModeBar = FALSE)

3 Stay Tuned

Please visit the CastLAB page for latest updates and news. Comments, bug reports and pull requests are always welcome.