Why interpolating country’s data?

When comparisons among different countries are performed, one critical problem is that the classification criteria differ from one country to another. This prompted the need to reclassify the data according to a unique pattern.

What model is the “best” fit for this type of binned data?

What is the underlying distribution of agricultural holdings and their area by size? Is there any model which can fit properly to the observed data? In 1984, FAO adopted a statistical technique to analyse distribution of holdings and area of WCA 1970 data and it was discovered that in situation where we have left skewed data, log-normal distribution is a good candidate to model the underlying information. From the analysis of the data on number and area of holdings for about 70 countries participating in the 1970 WCA, it was concluded that the log-normal hypothesis appear to be satisfactory for the distribution of number of holdings and area by land size classes. Therefore, the log-normal distribution was adopted henceforth to model the data.

Case study: Jamaica Agricultural census 2007

Country’s land size classes Holdings Area
Total holdings with land 200613 325810
Less than 1 ha 151929 47712
1-5 ha 43731 86011
5-10 ha 2922 19721
10-25 ha 1283 19166
25-50 ha 338 11896
50-100 ha 170 11742
100-200 ha 100 13707
200 ha and over 140 115854

As you can see above, country’s distribution of holdings and area by land size classes is different from FAO’s recommended land size classes (see below). To reclassify this data, it is important to estimate how many holdings are in the classes 1-5 ha, 10-25 ha etc., according to FAO’s standard classes. That’s, we need to estimate the values for the land size classes 1-2 ha, 2-3 ha, 3-4 ha, 4-5 ha, 10-20 ha, and 20-50 ha using log-normal distribution model.

Defining the upper classes of the two bined data (FAO and country’s classes)

For the purpose of this report, FAO and country’s upper land size classes are defined as c_i and d_i respectively, and number of holdings or area is represented as n_i. It is possible to call these classes directly into the interpolation function that was created for this activity. For clarity sake, they are listed below.

c_i <- c(1,2,3,4,5,10,20,50,100,200,500)
d_i <- c(0,1,5,10,25,50,100,200,500)
n_i <- c(151929, 43731, 2922,1283,338,170, 100,140)

Graphical comparison of the original classes by fitting several distribution models

Using the R function fit_dist_addin() from the R package Actuar, you can make graphical comparison of the original country’s classes using various distribution models like normal, log-normal and exponential distributions etc,. The chart above is an example of what you can get using the fit_dist_addin() function.

R script for creating the Interpolation function

#create a function called Interpolation
Interpolation <- function(d_i,n_i,c_i){
#calculate the mean and standard deviation by using the meanlog and sdlog respectively.
mean=mean(log(d_i[-1],2.718)) 
stdev=sd(log(d_i[-1],2.718)*sqrt((length(d_i)-2)/(length(d_i)-1)))
#sort the unique classes in the country and fao's land size classes
d_i.c_i=sort(unique(c(d_i,c_i)))
n_int=rep(0,length(d_i.c_i)-1)
j=1
#loop through the sorted data by distributing the values appling lognormality using the plnorm function and the calculated stdev and mean 
for (int in 1:(length(d_i)-1)){
  lim_inf=d_i[int]
  lim_sup=d_i[int+1]
  number_in=sum(c_i<lim_sup & c_i>lim_inf)
  limits=c(lim_inf,c_i[which(c_i<lim_sup & c_i>lim_inf)],lim_sup)
  for (k in 1:(length(limits)-1)){
    n_int[j]=(plnorm(limits[k+1],mean,stdev)-plnorm(limits[k],mean,stdev))/
      (plnorm(lim_sup,mean,stdev)-plnorm(lim_inf,mean,stdev))*n_i[int]
    j=j+1
  }
}
cbind(d_i.c_i[-1],n_int)
interval=rep(0,length(d_i.c_i)-1)
for (new_int in 1:length(c_i)){
  temp=c_i[new_int]
  for (i in 1:length(interval)){
    if (interval[i]==0 & d_i.c_i[i+1]<=temp) interval[i]=new_int
  }
}
total=data.frame(d_i.c_i[-1],n_int,interval)

output <- data.frame(tapply(total$n_int,total$interval,sum))
#combine the output with Fao standard land size classes and mutate next column that would indicate if a land size class is interpolated or not
total2 <- cbind(output,c_i) %>% 
  mutate(status=if_else(tapply.total.n_int..total.interval..sum. %in% n_i, "", "Interpolated" ))
}

Using the “Interpolation” function that was created above, you can interpolate any country’s land size classes by using the log-normal distribution model. As for our case study, i have applied the function to these classes and the output is shown below. As you can see, the function has distributed the country’s number of holdings into FAO’s standard land size classes. A new column was mutated to show the classes where interpolation occurred and in cases where country’s classes correspond with FAO’s classes, interpolation was not performed. Please note that, before interpolating any country’s classes, you need to consider the minimum threshold applied by the country during data collection as this will determine the start point of the interpolation. In addition, when a country classes are not available, we do not interpolate the total number of holdings and area.

Output

Standard classes Value Status Unit
Holdings with land size 0-<1 151929 No
Holdings with land size 1-<2 13033 Interpolated No
Holdings with land size 2-<3 11485 Interpolated No
Holdings with land size 3-<4 10148 Interpolated No
Holdings with land size 4-<5 9065 Interpolated No
Holdings with land size 5-<10 2922 No
Holdings with land size 10-<20 954 Interpolated No
Holdings with land size 20-<50 667 Interpolated No
Holdings with land size 50-<100 170 No
Holdings with land size 100-<200 100 No
Holdings with land size 200-<500 140 No