1 Introduction

Estimation of crop areas using pixel counting are subject to at least two sources of errors: mixed border’s pixels and misclassification of pure pixels. Considering the territory of interest is completely covered by satellite imagery, and assuming the effect of mixed border pixels can be neglected, the bias due to misclassification of pure pixel counting on crop areas estimate can be defined with respect to an error matrix, also called a confusion matrix.

Consider the classification of images of the whole territory of interest leading to the identification of \(M\) classes of land covering, so that \(𝑅 = (𝐴_{+1}, 𝐴_{+2}, \cdots, 𝐴_{+M})'\) is an \(𝑀\times 1\) column vector with the total area of pixels classified in each type of class. The actual areas, though, are represented by \(𝑇= (𝐴_{1+}, 𝐴_{2+},\cdots, 𝐴_{N+})'\) an \(N \times 1\) column vector with the truth total area of classes found on the ground.

The confusion matrix (or error matrix) is an \(𝑀 \times N\) matrix \(𝐴\) so that its elements are areas classified according to the remote sensing image and the ground truth:

\[A = \begin{pmatrix} A_{11} & A_{12} & \cdots & A_{1N}\\ A_{21} & A_{22} & \cdots & A_{2N} \\ \vdots & \vdots & \ddots & \vdots \\ A_{M1} & A_{M2} & \cdots & A_{MN} \end{pmatrix} \]

All the components can be arranged into a table with the same structure as the partitioned \({(M+1)\times(N+1)}\) matrix given by

\[ \begin{pmatrix} A & R\\ T' & A_{++} \end{pmatrix} \] Figure 1 illustrates an example of such a table with only three classes of land cover: wheat, corn and soy:

knitr::include_graphics('figures/matrix.png')
Figure 1. Example of a simple bordered error matrix (confusion matrix).

Figure 1. Example of a simple bordered error matrix (confusion matrix).

Error matrix \(𝐴\)’s elements \(𝐴_{gc}\) are areas of class \(g\) (ground) classified as class \(c\) by pixel counting. Hence, in Figure 1, \(𝐴_{11}\), \(𝐴_{22}\), and \(𝐴_{33}\) represent area estimates of wheat, corn, and soy, respectively, using pixels correctly classified as the corresponding ground truth data. In addition, \(𝐴_{21}\) and \(𝐴_{31}\) represent area estimates of wheat based on pixel counting, that are indeed areas of corn and soy, respectively.

The summation \((𝐴_{21} + 𝐴_{31}) = \Phi_1\) is called the comission error related to the wheat area estimate. On the other hand, \(𝐴_{12}\) and \(𝐴_{13}\) are wheat ground area that were mistakenly estimated as corn and soy areas, respectively, using pixel counting. The summation \((𝐴_{12} + 𝐴_{13}) = \Psi_1\) is called the omission error related to wheat estimate.

Error matrix \(𝐴\) and vector \(𝑇\) are unknown. Estimating \(𝑇\) based only on \(𝑅\) is subject to a bias given by the difference between the commission and the omission errors:

\[ B_c = \Phi_c - \Psi_c \]

Suppose ground information can be observed for a sample of test points, so that a sample \(𝑀 \times N\) error matrix \(a\) is available:

\[a = \begin{pmatrix} a_{11} & a_{12} & \cdots & a_{1N}\\ a_{21} & a_{22} & \cdots & a_{2N} \\ \vdots & \vdots & \ddots & \vdots \\ a_{M1} & a_{M2} & \cdots & a_{MN} \end{pmatrix}\]

Let \(r = (a_{1+}, a_{2+}, \cdots, a_{M+})'\) be an \(M \times 1\) column vector with the total area of sample test pixels classified in each type of class. Let \(t = (a_{1+}, a_{2+}, \cdots, a_{+N})'\) be an \(N \times 1\) column vector with the truth total area of classes found on the sample ground points.

Define the following diagonal matrices over exhaustive information from the whole territory:

\[\begin{eqnarray} D_R &= \text{diag}(A_{1+}, A_{2+}, \cdots, A_{M+})\\ D_T &= \text{diag}(A_{+1}, A_{+2}, \cdots, A_{+N}) \end{eqnarray} \]

1.0.1 Condicional probabilities

The following relative error matrices are defined over the population:

\[ E_c = A\cdot D_{T}^{-1} = \begin{pmatrix} \frac{A_{11}}{A_{+1}} & \frac{A_{12}}{A_{+2}} & \cdots & \frac{A_{1N}}{A_{+N}}\\ \frac{A_{21}}{A_{+1}} & \frac{A_{22}}{A_{+2}} & \cdots & \frac{A_{2N}}{A_{+N}}\\ \vdots & \vdots & \ddots & \vdots \\ \frac{A_{M1}}{A_{+1}} & \frac{A_{M2}}{A_{+2}} & \cdots & \frac{A_{MN}}{A_{+N}} \end{pmatrix} \] Note that \(A_{i+} = \sum_j A_{ij}\), \(A_{+j} = \sum_i A_{ij}\) and

\[ 1\!\!1_{1\times M}\cdot E_c = \begin{pmatrix}\frac{\sum_i A_{i1}}{A_{+1}} & \frac{\sum_i A_{i2}}{A_{+2}} & \cdots & \frac{\sum_i A_{iN}}{A_{+N}} \end{pmatrix} = 1\!\!1_{1\times N} % \text{ones(1, N)} = \]

Also \(E_c = \text{Pr}(c|g)\) is a \(M\times N\) matrix with the conditional probabilities that a pixel is classified as class \(c\) given the ground truth class is \(g\).

Likewise,

\[ E_g = A'\cdot D_{R}^{-1} = \begin{pmatrix} \frac{A_{11}}{A_{1+}} & \frac{A_{21}}{A_{2+}} & \cdots & \frac{A_{M1}}{A_{M+}}\\ \frac{A_{12}}{A_{1+}} & \frac{A_{22}}{A_{2+}} & \cdots & \frac{A_{M2}}{A_{M+}} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{A_{1N}}{A_{1+}} & \frac{A_{2N}}{A_{2+}} & \cdots & \frac{A_{MN}}{A_{M+}} \end{pmatrix} \] and \(E_g = \text{Pr}(g|c)\) is a \(N\times M\) matrix with the conditional probability that a pixel is over the ground truth class \(g\) given the pixel is classified as \(c\).

Define the following diagonal matrices over a sample of test points:

\[\begin{eqnarray} D_r &= \text{diag}(a_{1+}, a_{2+}, \cdots, a_{M+})\\ D_t &= \text{diag}(a_{+1}, a_{+2}, \cdots, a_{+N}) \end{eqnarray} \]

Then, \(E_c\) and \(E_g\) can be approximately unbiasedly estimated by \(e_c\) and \(e_g\) given by:

\[ \begin{eqnarray} e_c &= a\cdot D_t^{-1}\\ e_g &= a' \cdot D_r^{-1}\\ \end{eqnarray} \]

1.0.2 Estimators

The following calibration estimators can be defined:

The direct calibration estimator:

\[ \hat{T}_{\text{direct}} = e_g \cdot R = \begin{pmatrix} \frac{a_{11}}{a_{1+}}A_{1+} + \frac{a_{21}}{a_{2+}}A_{2+} + \cdots + \frac{a_{M1}}{a_{M+}}A_{M+}\\ \frac{a_{12}}{a_{1+}}A_{1+} + \frac{a_{22}}{a_{2+}}A_{2+} + \cdots + \frac{a_{M2}}{a_{M+}}A_{M+} \\ \vdots \\ \frac{a_{1N}}{a_{1+}}A_{1+} + \frac{a_{2N}}{a_{2+}}A_{2+} + \cdots + \frac{a_{MN}}{a_{M+}}A_{M+} \end{pmatrix} \]

The inverse calibration estimator:

\[ \hat{T}_{\text{inverse}} = e_c^{+} \cdot R = \begin{pmatrix} \frac{a_{11}}{a_{+1}} & \frac{a_{12}}{a_{+2}} & \cdots & \frac{a_{1N}}{a_{+N}}\\ \frac{a_{21}}{a_{+1}} & \frac{a_{22}}{a_{+2}} & \cdots & \frac{a_{2N}}{a_{+N}}\\ \vdots & \vdots & \ddots & \vdots \\ \frac{a_{M1}}{a_{+1}} & \frac{a_{M2}}{a_{+2}} & \cdots & \frac{a_{MN}}{a_{+N}} \end{pmatrix}^{+}\cdot \begin{pmatrix} A_{1+} \\ A_{2+} \\ \vdots \\ A_{M+}\end{pmatrix} \]

where \(e_c^+\) is a left inverse \(N \times M\) matrix of \(e_c\), such that \(e_c^+ \cdot e_c = \text{I}\!\text{I}_N\). It can be found using:

\[ e_c^+ = (e'_c \cdot e_c)^{-1}\cdot e'_c \]

2 Simulations

2.1 Packages

library(pracma)
library(tidyverse)
library(readxl)
library(glmnet)
library(knitr)
library(kableExtra)
library(plotly)

2.2 Reading the data

FN1.tbl <- read_excel("data/NL_3035_countpixelEUcropmap.xlsx",
                      sheet = "Total", range = "I4:K15", col_names = FALSE,
                      col_types = c("text", "text", "numeric")) %>%
  rename(crop = ...1, code = ...2, value = ...3)

FN2.tbl <- read_excel("data/ConfMatrix_NL.xlsx", sheet = "ConfMatClean") %>% 
  rename(code = ...1)

FN3.tbl <- read_csv("data/EUCROPMAP_legend.csv",
                    col_types = cols(code = col_character()))
FN1.tbl %>% 
  mutate(value = format(value, nsmall = 0, big.mark = ",", digits = 10)) %>% 
  kable(caption = "Table FN1", align = "lcr")%>%
  kable_styling(bootstrap_options = 
                  c("striped", "hover", "condensed", "responsive"), 
                full_width = FALSE, font_size = 12)
Table FN1
crop code value
common wheat 211 20,287,640
barley 213 5,168,348
maize 216 36,809,904
potatoes 221 10,542,589
sugar beet 222 7,981,893
other roots crops 223 914,813
dry pulses, vegetables and flowers 240 1,807,891
fodder crops (cereals and leguminous) 250 443,079
bare arable land 290 472,773
woodland and shrubland vegetation (incl. permanent crops) 300 72,256,453
permanent and temporary grasslands 500 1,641,199
Masked 1 82,561,683
FN2.tbl %>% kbl(caption = "Table FN2") %>%
    kable_styling(bootstrap_options = 
                      c("striped", "hover", "condensed", "responsive"), 
                  full_width = FALSE, font_size = 12) %>% 
  column_spec(20, bold = TRUE) %>% 
  row_spec(13, bold = TRUE) %>% 
  scroll_box(width = "100%")
Table FN2
code 100 211 213 214 215 216 218 221 222 223 230 240 250 290 300 500 600 other classes TOTAL
211 7 57 9 2 1 1 1 0 0 1 0 2 3 5 0 44 0 2 135
213 0 5 5 1 1 0 0 0 0 0 1 1 0 2 0 6 0 0 22
216 3 1 2 0 1 102 0 20 10 5 2 4 2 1 3 22 1 3 182
221 0 0 0 0 0 0 0 59 6 0 0 1 0 0 0 0 0 0 66
222 1 0 0 0 0 0 0 14 22 1 0 0 0 0 0 0 0 0 38
223 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1
240 0 0 0 0 0 0 0 0 1 4 0 4 0 2 0 1 0 0 12
250 0 0 0 0 0 1 0 0 0 1 0 1 0 0 0 0 0 0 3
290 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 2
300 31 0 0 0 0 6 0 1 0 0 0 0 0 0 218 27 3 4 290
500 31 9 5 0 0 10 0 0 0 0 1 2 2 1 18 595 2 8 684
Masked 130 1 0 0 0 0 0 0 1 1 0 6 0 0 32 57 6 9 243
Total 203 73 21 3 3 120 1 94 40 14 4 21 7 13 271 752 12 26 1678
# FN2.tbl %>% kable(caption = "Table FN2") %>%
#   kable_styling(bootstrap_options = 
#                   c("striped", "hover", "condensed", "responsive"), 
#                 full_width = FALSE, font_size = 12) %>%
#   scroll_box(width = "100%")

FN3.tbl %>% kable(caption = "Table FN3") %>% 
  kable_styling(bootstrap_options = 
                  c("striped", "hover", "condensed", "responsive"), 
                full_width = FALSE, font_size = 12) 
Table FN3
code label LUCAS
100 artificial land A11/A12/A13/A21/A22
211 common wheat B11
212 durum wheat B12
213 barley B13
214 rye B14
215 oats B15
216 maize B16
217 rice B17
218 triticale B18
219 other cereals B19
221 potatoes B21
222 sugar beet B22
223 other roots crops B23
230 other non permanent industrial crops B34/B35/B36/B37
231 sunflower B31
232 rape and turnip rape B32
233 soya B33
240 dry pulses, vegetables and flowers B41/B42/B43/B44/B45
250 fodder crops (cereals and leguminous) B51/B52/B53/B54
290 bare arable land F40 U111/112/113
300 woodland and shrubland vegetation (incl. permanent crops) C10/C21/C22/C23/C31/C32/C33/D10/D20/B71-77/B81-84
500 permanent and temporary grasslands E10/E20/E30/B55
600 Bare land and lichens/moss F10/F20/F20/F40 not agricultural land use

2.3 Matrices

#Leg <- FN3.tbl 

class_codes <- setdiff(names(FN2.tbl), c("code", "other classes", "TOTAL"))
ground_codes <- FN2.tbl %>% pull(code) %>% setdiff(c("Masked", "Total"))

R_vector <- FN1.tbl %>%
  filter(crop != "Masked") %>% 
  select(code, R = value)  %>% 
  #bind_rows(tibble(code = 0, value = 0)) %>%
  mutate_all(as.numeric) %>%
  column_to_rownames(var = "code") %>%
  as.matrix() 

a_matrix <- FN2.tbl %>% 
  select(-one_of(c(setdiff(class_codes, ground_codes), "other classes"))) %>% 
  bind_cols(tibble(`other classes` = FN2.tbl %>% 
                     select_at(.vars = 
                                 c(setdiff(class_codes, ground_codes), 
                                   "other classes")) %>% 
                     rowSums())) %>% 
  #filter(code != "Total") %>%
  filter(code != "Total", code != "Masked") %>%
  select(-TOTAL) %>%
  #select(-TOTAL, -`other classes`) %>%
  column_to_rownames(var = "code") %>%
  mutate_all(as.numeric) %>%
  as.matrix() # %>% t()

R_vector %>% 
  as.data.frame() %>% 
  rownames_to_column(var = "code") %>% 
  mutate(R = format(R, nsmall = 0, big.mark = ",", digits = 10)) %>% 
  kable(caption = "Table TC", align = "lr") %>%
  kable_styling(bootstrap_options =
                  c("striped", "hover", "condensed", "responsive"), 
                full_width = FALSE, font_size = 12)
Table TC
code R
211 20,287,640
213 5,168,348
216 36,809,904
221 10,542,589
222 7,981,893
223 914,813
240 1,807,891
250 443,079
290 472,773
300 72,256,453
500 1,641,199
a_matrix %>% kable(caption = "Table PCG") %>% 
  kable_styling(bootstrap_options = 
                  c("striped", "hover", "condensed", "responsive"), 
                full_width = FALSE, font_size = 12)
Table PCG
211 213 216 221 222 223 240 250 290 300 500 other classes
211 57 9 1 0 0 1 2 3 5 0 44 13
213 5 5 0 0 0 0 1 0 2 0 6 3
216 1 2 102 20 10 5 4 2 1 3 22 10
221 0 0 0 59 6 0 1 0 0 0 0 0
222 0 0 0 14 22 1 0 0 0 0 0 1
223 0 0 0 0 0 1 0 0 0 0 0 0
240 0 0 0 0 1 4 4 0 2 0 1 0
250 0 0 1 0 0 1 1 0 0 0 0 0
290 0 0 0 0 0 0 0 0 2 0 0 0
300 0 0 6 1 0 0 0 0 0 218 27 38
500 9 5 10 0 0 0 2 2 1 18 595 42
#column_spec(1, bold = T, border_right = T)

2.4 ConfVarEst function

ConfVarEst <- function(a_matrix, R_vector, replications = 100, try = 1000){
  # a = confusion matrix
  # R = classification vecotr
  dim_a <- dim(a_matrix)
  M = dim_a[1]
  N = dim_a[2]
  
  
  
  r_vec  <- rowSums(a_matrix)
  t_vec <-  colSums(a_matrix)
  D_r = diag(r_vec)
  D_t = diag(t_vec)
  iD_r = solve(D_r)
  iD_t = solve(D_t)
  
  e_c = a_matrix %*% iD_t
  e_g = t(a_matrix) %*% iD_r
  
  # Direct estimator
  T_hat_direct = e_g %*% R_vector
  
  # # Indirect estimator
  # temp_matrix <- t(e_c) %*% e_c
  # det_e_c <- det(temp_matrix)
  # if (abs(det_e_c) > .Machine$double.eps) {
  #   e_c_plus = solve(temp_matrix) %*% t(e_c)
  # T_hat_indirect = e_c_plus %*% R_vector
  # } else {
  #   T_hat_indirect = NaN*T_hat_direct
  # }
  # 
  # rownames(T_hat_indirect) <- rownames(T_hat_direct)
  
  # Bootstrapping 
  edges_g <- rbind(matrix(rep(0, M), nrow = 1), apply(e_g, 2, cumsum))
  edges_c <- rbind(matrix(rep(0, N), nrow = 1), apply(e_c, 2, cumsum))
  T_hat_direct_boot <- NULL
  #T_hat_indirect_boot <- NULL
  
  e_g_sampled <- array(dim = c(N, M))
  e_c_sampled <- array(dim = c(M, N))
  
  for (i in 1:replications){
    # Resampling on each ground class sepatately  
    for (j in 1:M) {
      numbers <- matrix(runif(r_vec[j]), ncol = 1) 
      Eff <- pracma::histc(numbers, edges_g[,j])$cnt 
      e_g_sampled[,j] <- Eff[-(N + 1)]/r_vec[j]
    }
    
    # flag <- TRUE
    # while (flag) {
    #   for (j in 1:N) {
    #     numbers <- matrix(runif(t_vec[j]), ncol = 1) 
    #     Eff <- pracma::histc(numbers, edges_c[,j])$cnt 
    #     e_c_sampled[,j] <- Eff[-(M + 1)]/t_vec[j]
    #   }
    #   flag <- abs(det(t(e_c_sampled) %*% e_c_sampled)) < (.Machine$longdouble.eps*10)
    # }

    
    # Direct estimator
    direct = e_g_sampled %*% R_vector
  
  #   # Indirect estimator
  #   temp_matrix <- t(e_c_sampled) %*% e_c_sampled
  #   det_e_c_sampled <- det(temp_matrix)
  #   if (abs(det_e_c_sampled) > .Machine$longdouble.eps*10) {
  #     e_c_plus_sampled = solve(temp_matrix) %*% t(e_c_sampled)
  #     indirect = e_c_plus_sampled %*% R_vector
  #   } else {
  #   indirect = NaN
  # }
  
    T_hat_direct_boot <- cbind(T_hat_direct_boot, direct)
  # T_hat_indirect_boot <- cbind(T_hat_indirect_boot, indirect)
  }
  
  rownames(T_hat_direct_boot) <- rownames(T_hat_direct)
  #rownames(T_hat_indirect_boot) <- rownames(T_hat_indirect)
  
  mean_direct <- rowMeans(T_hat_direct_boot)
  #names(mean_direct) <- rownames(T_hat_direct)
  
  #mean_indirect <- rowMeans(T_hat_indirect_boot)
  #names(mean_indirect) <- rownames(T_hat_indirect)
  
  std_direct <- apply(T_hat_direct_boot, 1, function(x) sqrt(var(x)))
  names(std_direct) <- rownames(T_hat_direct)
  #std_indirect <- apply(T_hat_indirect_boot, 1, function(x) sqrt(var(x)))
  #names(std_indirect) <- rownames(T_hat_indirect)
  
  cv_direct <- apply(T_hat_direct_boot, 1, function(x) 100*sqrt(var(x))/mean(x))
  names(cv_direct) <- rownames(T_hat_direct)
  #cv_indirect <- apply(T_hat_indirect_boot, 1, function(x) 100*sqrt(var(x))/mean(x))
  #names(cv_indirect) <- rownames(T_hat_indirect)
  
  return(list(direct = T_hat_direct, 
              #indirect = T_hat_indirect, 
              direct_boot = T_hat_direct_boot, 
              #indirect_boot = T_hat_indirect_boot,
              direct_mean = mean_direct,
              #indirect_mean = mean_indirect,
              direct_std = std_direct,
              #indirect_std = std_indirect,
              direct_cv = cv_direct#,
              #indirect_cv = cv_indirect
              )
         )
}

2.5 Results

result <- ConfVarEst(a_matrix, R_vector, replications = 1000)

format_values <- 
  function(x, y) 
    format(x, nsmall = y, big.mark = ",", digits = 10)

result[c("direct", "direct_mean", "direct_std", "direct_cv")] %>% 
  as.data.frame() %>% 
  rownames_to_column(var = "Crop") %>% 
  rename(Estimate = R, Mean = direct_mean, 'Deviation' = direct_std, CV = direct_cv) %>% 
  mutate(Estimate = format_values(round(Estimate), 0),
         Mean = format_values(round(Mean), 0),
         Deviation = format_values(round(Deviation), 0),
         CV = format_values(round(CV,2), 2)) %>% 
  kable(caption = "Direct Estimator Bootstrap Results (100 rounds)", align = "lrrr") %>%
  kable_styling(bootstrap_options = 
                  c("striped", "hover", "condensed", "responsive"), 
                full_width = FALSE, font_size = 12)
Direct Estimator Bootstrap Results (100 rounds)
Crop Estimate Mean Deviation CV
211 9,964,364 9,968,656 998,315 10.01
213 2,943,635 2,897,157 671,466 23.18
216 22,446,653 22,528,500 1,501,960 6.67
221 16,659,338 16,682,005 1,198,827 7.19
222 7,752,693 7,732,907 997,988 12.91
223 3,036,726 3,033,411 601,802 19.84
240 2,259,350 2,222,741 619,397 27.87
250 860,140 847,790 378,374 44.63
290 2,199,984 2,247,125 550,072 24.48
300 54,966,866 54,935,776 1,906,788 3.47
500 20,777,000 20,784,912 1,799,801 8.66
other classes 14,459,834 14,445,602 1,635,037 11.32
# result[c("indirect", "indirect_mean", "indirect_std", "indirect_cv")] %>% 
#   as.data.frame() %>% 
#   rownames_to_column(var = "Crop") %>% 
#   rename(Estimate = R, Mean = indirect_mean, 'Deviation' = indirect_std, CV = indirect_cv) %>% 
#   mutate(Estimate = format_values(round(Estimate), 0),
#          Mean = format_values(round(Mean), 0),
#          Deviation = format_values(round(Deviation), 0),
#          CV = format_values(round(CV,2), 2)) %>% 
#   kable(caption = "Indirect Estimator Bootstrap Results (100 rounds)", align = "lrrr") %>%
#   kable_styling(bootstrap_options = 
#                   c("striped", "hover", "condensed", "responsive"), 
#                 full_width = FALSE, font_size = 12)

df_direct <- result$direct_boot %>% t() %>% 
  as.matrix() %>% 
  as_tibble() %>% 
  pivot_longer(everything(), names_to = "crop", values_to = "value") 

# df_indirect <- result$indirect_boot %>% t() %>% 
#   as.matrix() %>% 
#   as_tibble() %>% 
#   pivot_longer(everything(), names_to = "crop", values_to = "value")

p_direct <-  df_direct %>% ggplot(aes(x = value)) + geom_histogram(alpha=0.25, position="identity", bins = 30, aes(color = crop, fill = crop)) + facet_wrap(. ~ crop,nrow = 4, scales = "free") + theme_bw() + guides(fill= FALSE, color = FALSE) + theme(strip.background = element_rect(fill = gray(.95)))  +  scale_x_continuous(labels = function(x) format(x, scientific = TRUE), n.breaks = 4) + ggtitle("Bootstrap replications for direct estimates") + theme(panel.spacing = unit(1, "lines"))


# p_indirect <-  df_indirect %>% ggplot(aes(x = value)) + geom_histogram(alpha=0.25, position="identity", bins = 30, aes(color = crop, fill = crop)) + facet_wrap(. ~ crop,nrow = 4, scales = "free") + theme_bw() + guides(fill= FALSE, color = FALSE) + theme(strip.background = element_rect(fill = gray(.95)))  +  scale_x_continuous(labels = function(x) format(x, scientific = TRUE), n.breaks = 4) + ggtitle("Bootstrap replications for indirect estimates") + theme(panel.spacing = unit(1, "lines"))

ggplotly(p_direct,tooltip = c("y")) %>% 
  layout(showlegend = FALSE) %>% 
  layout(plot_bgcolor='rgba(254, 254, 254, .25)') %>% 
  layout(paper_bgcolor='#eeeeee') %>% 
  config(displayModeBar = FALSE)
# ggplotly(p_indirect,tooltip = c("y")) %>% 
#   layout(showlegend = FALSE) %>% 
#   layout(plot_bgcolor='rgba(254, 254, 254, .25)') %>% 
#   layout(paper_bgcolor='#eeeeee') %>% 
#   config(displayModeBar = FALSE)

3 Stay Tuned

Please visit the CastLAB page for latest updates and news. Comments, bug reports and pull requests are always welcome.