1 Introduction

Estimation of crop areas using pixel counting are subject to at least two sources of errors: mixed border’s pixels and misclassification of pure pixels. Considering the territory of interest is completely covered by satellite imagery, and assuming the effect of mixed border pixels can be neglected, the bias due to misclassification of pure pixel counting on crop areas estimate can be defined with respect to an error matrix, also called a confusion matrix.

Consider the classification of images of the whole territory of interest leading to the identification of \(M\) classes of land covering, so that \(𝑅 = (𝐴_{+1}, 𝐴_{+2}, \cdots, 𝐴_{+M})'\) is an \(𝑀\times 1\) column vector with the total area of pixels classified in each type of class. The actual areas, though, are represented by \(𝑇= (𝐴_{1+}, 𝐴_{2+},\cdots, 𝐴_{N+})'\) an \(N \times 1\) column vector with the truth total area of classes found on the ground.

The confusion matrix (or error matrix) is an \(N \times M\) matrix \(𝐴\) so that its elements are areas classified according to the remote sensing image and the ground truth:

\[A = \begin{pmatrix} A_{11} & A_{12} & \cdots & A_{1M}\\ A_{21} & A_{22} & \cdots & A_{2M} \\ \vdots & \vdots & \ddots & \vdots \\ A_{N1} & A_{N2} & \cdots & A_{NM} \end{pmatrix} \]

All the components can be arranged into a table with the same structure as the partitioned \({(N+1)\times(M+1)}\) matrix given by

\[ \begin{pmatrix} A & T\\ R' & A_{++} \end{pmatrix} \] Figure 1 illustrates an example of such a table with only three classes of land cover: wheat, corn and soy:

knitr::include_graphics('figures/matrix.png')
Figure 1. Example of a simple bordered error matrix (confusion matrix).

Figure 1. Example of a simple bordered error matrix (confusion matrix).

Error matrix \(𝐴\)’s elements \(𝐴_{gc}\) are areas of class \(g\) (ground) classified as class \(c\) by pixel counting. Hence, in Figure 1, \(𝐴_{11}\), \(𝐴_{22}\), and \(𝐴_{33}\) represent area estimates of wheat, corn, and soy, respectively, using pixels correctly classified as the corresponding ground truth data. In addition, \(𝐴_{21}\) and \(𝐴_{31}\) represent area estimates of wheat based on pixel counting, that are indeed areas of corn and soy, respectively.

The summation \((𝐴_{21} + 𝐴_{31}) = \Phi_1\) is called the comission error related to the wheat area estimate. On the other hand, \(𝐴_{12}\) and \(𝐴_{13}\) are wheat ground area that were mistakenly estimated as corn and soy areas, respectively, using pixel counting. The summation \((𝐴_{12} + 𝐴_{13}) = \Psi_1\) is called the omission error related to wheat estimate.

Error matrix \(𝐴\) and vector \(𝑇\) are unknown. Estimating \(𝑇\) based only on \(𝑅\) is subject to a bias given by the difference between the commission and the omission errors:

\[ \begin{align} \Phi_c &= A_{+c} - A_{cc}\\ \Psi_c &= A_{c+} - A_{cc}\\ B_c &= \Phi_c - \Psi_c \\ &= A_{+c} - A_{c+} \end{align} \]

Denoting by \(B = (B_1, B_2, \cdots, B_M)'\) the column vector of biases for each of the classes’ estimates, one can write, if \(M = N\):

\[ B = R - T \]

Suppose ground information can be observed for a sample of test points, so that a sample \(𝑀 \times N\) error matrix \(a\) is available:

\[a = \begin{pmatrix} a_{11} & a_{12} & \cdots & a_{1M}\\ a_{21} & a_{22} & \cdots & a_{2M} \\ \vdots & \vdots & \ddots & \vdots \\ a_{N1} & a_{N2} & \cdots & a_{NM} \end{pmatrix}\]

Let \(r = (a_{+1}, a_{+2}, \cdots, a_{+M})'\) be an \(M \times 1\) column vector with the total area of sample test pixels classified in each type of class. Let \(t = (a_{+1}, a_{+2}, \cdots, a_{N+})'\) be an \(N \times 1\) column vector with the truth total area of classes found on the sample ground points.

Define the following diagonal matrices over exhaustive information from the whole territory:

\[\begin{eqnarray} D_R &= \text{diag}(A_{+1}, A_{+2}, \cdots, A_{+M})\\ D_T &= \text{diag}(A_{1+}, A_{2+}, \cdots, A_{N+}) \end{eqnarray} \]

1.0.1 Condicional probabilities

The following relative error matrices are defined over the population:

\[ E_{c|g} = A'\cdot D_{T}^{-1} = \begin{pmatrix} \frac{A_{11}}{A_{1+}} & \frac{A_{21}}{A_{2+}} & \cdots & \frac{A_{N1}}{A_{N+}}\\ \frac{A_{12}}{A_{1+}} & \frac{A_{22}}{A_{2+}} & \cdots & \frac{A_{N2}}{A_{N+}}\\ \vdots & \vdots & \ddots & \vdots \\ \frac{A_{1M}}{A_{1+}} & \frac{A_{2M}}{A_{2+}} & \cdots & \frac{A_{NM}}{A_{N+}} \end{pmatrix} \] Note that \(A_{i+} = \sum_j A_{ij}\), \(A_{+j} = \sum_i A_{ij}\) and

\[ 1\!\!1_{1\times M}\cdot E_{c|g} = \begin{pmatrix}\frac{\sum_i A_{1i}}{A_{1+}} & \frac{\sum_i A_{2i}}{A_{2+}} & \cdots & \frac{\sum_i A_{Ni}}{A_{N+}} \end{pmatrix} = 1\!\!1_{1\times N} % \text{ones(1, N)} = \]

Also \(E_c = \text{Pr}(c|g)\) is a \(M\times N\) matrix with the conditional probabilities that a pixel is classified as class \(c\) given the ground truth class is \(g\).

Likewise,

\[ E_{g|c} = A\cdot D_{R}^{-1} = \begin{pmatrix} \frac{A_{11}}{A_{+1}} & \frac{A_{12}}{A_{+2}} & \cdots & \frac{A_{1M}}{A_{+M}}\\ \frac{A_{21}}{A_{+1}} & \frac{A_{22}}{A_{+2}} & \cdots & \frac{A_{2M}}{A_{+M}} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{A_{N1}}{A_{+1}} & \frac{A_{N2}}{A_{+2}} & \cdots & \frac{A_{NM}}{A_{+M}} \end{pmatrix} \] and \(E_{g|c} = \text{Pr}(g|c)\) is a \(N\times M\) matrix with the conditional probability that a pixel is over the ground truth class \(g\) given the pixel is classified as \(c\).

Define the following diagonal matrices over a sample of test points:

\[\begin{eqnarray} D_r &= \text{diag}(a_{+1}, a_{+2}, \cdots, a_{+M})\\ D_t &= \text{diag}(a_{1+}, a_{2+}, \cdots, a_{N+}) \end{eqnarray} \]

Then, \(E_{c|g}\) and \(E_{g|c}\) can be approximately unbiasedly estimated by \(e_{c|g}\) and \(e_g\) given by:

\[ \begin{eqnarray} e_{c|g} &= a'\cdot D_t^{-1}\\ e_{g|c} &= a\cdot D_r^{-1}\\ \end{eqnarray} \]

1.0.2 Estimators

The following calibration estimators can be defined:

The direct calibration estimator:

\[ \hat{T}_{\text{direct}} = e_{g|c} \cdot R = \begin{pmatrix} \frac{a_{11}}{a_{+1}}A_{+1} + \frac{a_{12}}{a_{+2}}A_{+2} + \cdots + \frac{a_{1M}}{a_{1M}}A_{+M}\\ \frac{a_{21}}{a_{+1}}A_{+1} + \frac{a_{22}}{a_{+2}}A_{+2} + \cdots + \frac{a_{2M}}{a_{+M}}A_{+M} \\ \vdots \\ \frac{a_{N1}}{a_{+1}}A_{+1} + \frac{a_{N2}}{a_{+2}}A_{+2} + \cdots + \frac{a_{NM}}{a_{+M}}A_{+M} \end{pmatrix} \]

The inverse calibration estimator:

\[ \hat{T}_{\text{inverse}} = e_{c|g}^{+} \cdot R = \begin{pmatrix} \frac{a_{11}}{a_{1+}} & \frac{a_{21}}{a_{2+}} & \cdots & \frac{a_{N1}}{a_{N+}}\\ \frac{a_{12}}{a_{1+}} & \frac{a_{22}}{a_{2+}} & \cdots & \frac{a_{N2}}{a_{N+}}\\ \vdots & \vdots & \ddots & \vdots \\ \frac{a_{1M}}{a_{1+}} & \frac{a_{2M}}{a_{2+}} & \cdots & \frac{a_{NM}}{a_{N+}} \end{pmatrix}^{+}\cdot \begin{pmatrix} A_{+1} \\ A_{+2} \\ \vdots \\ A_{+M}\end{pmatrix} \]

where \(e_{c|g}^+\) is a left inverse \(M \times N\) matrix of \(e_{c|g}\), such that \(e_{c|g}^+ \cdot e_{c|g} = \text{I}\!\text{I}_N\). It can be found using:

\[ e_{c|g}^+ = (e'_{c|g} \cdot e_{c|g})^{-1}\cdot e'_{c|g} \]

2 Simulations

2.1 Packages

library(pracma)
library(tidyverse)
library(readxl)
library(glmnet)
library(knitr)
library(kableExtra)
library(plotly)

2.2 Reading the data

FN1.tbl <- read_excel("data/CropType2018_PixelCounting.xlsx",
                      sheet = "Feuil1", range = "E2:F8", col_names = FALSE,
                      col_types = c("text", "numeric")) %>%
  rename(crop = ...1, value = ...2) %>% 
  mutate(crop = recode(crop, 
                       'Mill' = 'Millet', 
                       'Groundnut' = 'Groundnuts', 
                       'Coton' = 'Cotton'))

FN2.tbl <- read_excel("data/CropType2018_ConfusionMatrix.xlsx", 
                      sheet = "Matrice_Regroupement 1", 
                      range = "B2:I9") %>% 
  rename(crop = ...1) %>% 
  mutate(crop = recode(crop, 
                       'Mill' = 'Millet', 
                       'Groundnut' = 'Groundnuts', 
                       'Coton' = 'Cotton')) %>% 
  rename(Cotton = Coton, Groundnuts = Groundnut)
FN1.tbl %>% 
  mutate(value = format(value, nsmall = 0, big.mark = ",", digits = 10)) %>% 
  arrange(desc(value)) %>% 
  kable(caption = "Table FN1", align = "lr")%>%
  kable_styling(bootstrap_options = 
                  c("striped", "hover", "condensed", "responsive"), 
                full_width = FALSE, font_size = 12)
Table FN1
crop value
Cotton 102,803,268
Maize 99,454,021
Millet 68,759,277
Sorghum 10,911,297
Rice 9,654,264
Other crops 5,373,340
Groundnuts 651,230
FN2.tbl %>% kbl(caption = "Table FN2") %>%
    kable_styling(bootstrap_options = 
                      c("striped", "hover", "condensed", "responsive"), 
                  full_width = FALSE, font_size = 12) %>% 
  column_spec(1, bold = TRUE)
Table FN2
crop Maize Other crops Groundnuts Cotton Rice Sorghum Millet
Maize 9713 954 29 2094 144 20 602
Other crops 2125 2448 95 2598 35 78 297
Groundnuts 441 85 419 464 19 37 41
Cotton 2370 541 114 12770 88 121 255
Rice 514 21 2 327 887 23 0
Sorghum 531 133 32 584 12 888 335
Millet 668 27 4 426 14 83 3510

2.3 Matrices

#Leg <- FN3.tbl 

R_vector <- 
  FN1.tbl %>%
  rename(R = value) %>% 
  column_to_rownames(var = "crop") %>%
  as.matrix() 

a_matrix <- FN2.tbl %>% 
  column_to_rownames(var = "crop") %>%
  as.matrix() # %>% t()

R_vector %>% 
  as.data.frame() %>% 
  rownames_to_column(var = "crop") %>% 
  mutate(R = format(R, nsmall = 0, big.mark = ",", digits = 10)) %>% 
  kable(caption = "Table TC", align = "lr") %>%
  kable_styling(bootstrap_options =
                  c("striped", "hover", "condensed", "responsive"), 
                full_width = FALSE, font_size = 12)
Table TC
crop R
Maize 99,454,021
Other crops 5,373,340
Groundnuts 651,230
Cotton 102,803,268
Rice 9,654,264
Sorghum 10,911,297
Millet 68,759,277
a_matrix %>% 
  as.data.frame() %>% 
  rownames_to_column(var = "crop") %>% 
  kable(caption = "Table PCG") %>% 
  kable_styling(bootstrap_options = 
                  c("striped", "hover", "condensed", "responsive"), 
                full_width = FALSE, font_size = 12) %>% 
  column_spec(1, bold = TRUE)
Table PCG
crop Maize Other crops Groundnuts Cotton Rice Sorghum Millet
Maize 9713 954 29 2094 144 20 602
Other crops 2125 2448 95 2598 35 78 297
Groundnuts 441 85 419 464 19 37 41
Cotton 2370 541 114 12770 88 121 255
Rice 514 21 2 327 887 23 0
Sorghum 531 133 32 584 12 888 335
Millet 668 27 4 426 14 83 3510

2.4 ConfVarEst function

ConfVarEst <- function(a_matrix, R_vector, replications = 100){
  # a = confusion matrix
  # R = classification vecotr
  dim_a <- dim(a_matrix)
  N = dim_a[1]
  M = dim_a[2]
  
  
  
  r_vec  <- colSums(a_matrix)
  t_vec <-  rowSums(a_matrix)
  D_r = diag(r_vec)
  D_t = diag(t_vec)
  iD_r = solve(D_r) # == diag(1/r_vec)
  iD_t = solve(D_t) # == diag(1/t_vec)
  
  e_c_dado_g = t(a_matrix) %*% iD_t
  e_g_dado_c = a_matrix %*% iD_r
  
  # Direct estimator
  T_hat_direct = e_g_dado_c %*% R_vector
  
  # Inverse estimator
  T_hat_inverse = solve(e_c_dado_g) %*% R_vector
  # T_hat_inverse = (solve((t(e_c_dado_g) %*% e_c_dado_g)) %*% t(e_c_dado_g))  %*% R_vector
  rownames(T_hat_inverse) <- rownames(T_hat_direct)

  # Bootstrapping 
  edges_g_dado_c <- rbind(matrix(rep(0, M), nrow = 1), apply(e_g_dado_c, 2, cumsum))
  edges_c_dado_g <- rbind(matrix(rep(0, N), nrow = 1), apply(e_c_dado_g, 2, cumsum))
  T_hat_direct_boot <- NULL
  T_hat_inverse_boot <- NULL
  
  e_g_dado_c_sampled <- array(dim = c(N, M))
  e_c_dado_g_sampled <- array(dim = c(M, N))
  
  for (i in 1:replications){
    # Resampling on each ground class sepatately  
    for (j in 1:M) {
      numbers <- matrix(runif(r_vec[j]), ncol = 1) 
      Eff <- pracma::histc(numbers, edges_g_dado_c[,j])$cnt 
      e_g_dado_c_sampled[,j] <- Eff[-(N + 1)]/r_vec[j]
    }
    
    for (j in 1:N) {
      numbers <- matrix(runif(t_vec[j]), ncol = 1)
      Eff <- pracma::histc(numbers, edges_c_dado_g[,j])$cnt
      e_c_dado_g_sampled[,j] <- Eff[-(N + 1)]/t_vec[j]
    }
    
    # Direct estimator
    direct = e_g_dado_c_sampled %*% R_vector
  
    # Indirect estimator
    inverse = solve(e_c_dado_g_sampled) %*% R_vector
    

    T_hat_direct_boot <- cbind(T_hat_direct_boot, direct)
    T_hat_inverse_boot <- cbind(T_hat_inverse_boot, inverse)
  }
  
  rownames(T_hat_direct_boot) <- rownames(T_hat_direct)
  rownames(T_hat_inverse_boot) <- rownames(T_hat_inverse)

  
  mean_direct <- rowMeans(T_hat_direct_boot)
  names(mean_direct) <- rownames(T_hat_direct)
  
  mean_inverse <- rowMeans(T_hat_inverse_boot)
  names(mean_inverse) <- rownames(T_hat_inverse)
  
  std_direct <- apply(T_hat_direct_boot, 1, function(x) sqrt(var(x)))
  names(std_direct) <- rownames(T_hat_direct)
  
  std_inverse <- apply(T_hat_inverse_boot, 1, function(x) sqrt(var(x)))
  names(std_inverse) <- rownames(T_hat_inverse)


  cv_direct <- apply(T_hat_direct_boot, 1, function(x) 100*sqrt(var(x))/mean(x))
  names(cv_direct) <- rownames(T_hat_direct)
  
  cv_inverse <- apply(T_hat_inverse_boot, 1, function(x) 100*sqrt(var(x))/mean(x))
  names(cv_inverse) <- rownames(T_hat_inverse)
  
  return(list(direct = T_hat_direct, 
              inverse = T_hat_inverse, 
              direct_boot = T_hat_direct_boot, 
              inverse_boot = T_hat_inverse_boot,
              direct_mean = mean_direct,
              inverse_mean = mean_inverse,
              direct_std = std_direct,
              inverse_std = std_inverse,
              direct_cv = cv_direct,
              inverse_cv = cv_inverse
              )
         )
}

2.5 Results

result <- ConfVarEst(a_matrix, R_vector, replications = 100)

format_values <- 
  function(x, y) 
    format(x, nsmall = y, big.mark = ",", digits = 10)

result[c("direct", "direct_mean", "direct_std", "direct_cv")] %>% 
  as.data.frame() %>% 
  rownames_to_column(var = "Crop") %>% 
  rename(Estimate = R, Mean = direct_mean, 'Deviation' = direct_std, CV = direct_cv) %>% 
  mutate(Estimate = format_values(round(Estimate), 0),
         Mean = format_values(round(Mean), 0),
         Deviation = format_values(round(Deviation), 0),
         CV = format_values(round(CV,2), 2)) %>% 
  kable(caption = "Direct Estimator Bootstrap Results (100 rounds)", align = "lrrr") %>%
  kable_styling(bootstrap_options = 
                  c("striped", "hover", "condensed", "responsive"), 
                full_width = FALSE, font_size = 12)
Direct Estimator Bootstrap Results (100 rounds)
Crop Estimate Mean Deviation CV
Maize 81,006,413 80,939,610 542,075 0.67
Other crops 35,010,352 34,991,148 422,639 1.21
Groundnuts 6,693,279 6,698,570 184,120 2.75
Cotton 88,598,112 88,655,214 484,162 0.55
Rice 12,240,929 12,267,055 214,055 1.74
Sorghum 18,962,406 18,990,938 350,274 1.84
Millet 55,095,207 55,064,162 499,337 0.91
result[c("inverse", "inverse_mean", "inverse_std", "inverse_cv")] %>%
  as.data.frame() %>%
  rownames_to_column(var = "Crop") %>%
  rename(Estimate = R, Mean = inverse_mean, 'Deviation' = inverse_std, CV = inverse_cv) %>%
  mutate(Estimate = format_values(round(Estimate), 0),
         Mean = format_values(round(Mean), 0),
         Deviation = format_values(round(Deviation), 0),
         CV = format_values(round(CV,2), 2)) %>%
  kable(caption = "Inverse Estimator Bootstrap Results (100 rounds)", align = "lrrr") %>%
  kable_styling(bootstrap_options =
                  c("striped", "hover", "condensed", "responsive"),
                full_width = FALSE, font_size = 12)
Inverse Estimator Bootstrap Results (100 rounds)
Crop Estimate Mean Deviation CV
Maize 97,353,971 97,426,625 1,321,943 1.36
Other crops -21,021,744 -20,962,776 1,183,052 -5.64
Groundnuts -1,450,389 -1,476,966 387,175 -26.21
Cotton 101,128,468 100,964,505 1,001,341 0.99
Rice 15,658,343 15,735,261 459,216 2.92
Sorghum 24,447,448 24,455,953 785,448 3.21
Millet 81,490,601 81,464,095 848,480 1.04
df_direct <- result$direct_boot %>% t() %>% 
  as.matrix() %>% 
  as_tibble() %>% 
  pivot_longer(everything(), names_to = "crop", values_to = "value") 

df_inverse <- result$inverse_boot %>% t() %>%
  as.matrix() %>%
  as_tibble() %>%
  pivot_longer(everything(), names_to = "crop", values_to = "value")

p_direct <-  df_direct %>% ggplot(aes(x = value)) + geom_histogram(alpha=0.25, position="identity", bins = 30, aes(color = crop, fill = crop)) + facet_wrap(. ~ crop,nrow = 4, scales = "free") + theme_bw() + guides(fill= 'none', color = 'none') + theme(strip.background = element_rect(fill = gray(.95)))  +  scale_x_continuous(labels = function(x) format(x, scientific = TRUE), n.breaks = 4) + ggtitle("Bootstrap replications for direct estimates") + theme(panel.spacing = unit(1, "lines"))


p_inverse <-  df_inverse %>% ggplot(aes(x = value)) + geom_histogram(alpha=0.25, position="identity", bins = 30, aes(color = crop, fill = crop)) + facet_wrap(. ~ crop,nrow = 4, scales = "free") + theme_bw() + guides(fill= 'none', color = 'none') + theme(strip.background = element_rect(fill = gray(.95)))  +  scale_x_continuous(labels = function(x) format(x, scientific = TRUE), n.breaks = 4) + ggtitle("Bootstrap replications for inverse estimates") + theme(panel.spacing = unit(1, "lines"))

ggplotly(p_direct,tooltip = c("y")) %>% 
  layout(showlegend = FALSE) %>% 
  layout(plot_bgcolor='rgba(254, 254, 254, .25)') %>% 
  layout(paper_bgcolor='#eeeeee') %>% 
  config(displayModeBar = FALSE)
ggplotly(p_inverse,tooltip = c("y")) %>%
  layout(showlegend = FALSE) %>%
  layout(plot_bgcolor='rgba(254, 254, 254, .25)') %>%
  layout(paper_bgcolor='#eeeeee') %>%
  config(displayModeBar = FALSE)

3 Stay Tuned

Please visit the CastLAB page for latest updates and news. Comments, bug reports and pull requests are always welcome.