A Comparative Analysis of old WCA data and current data on FAOSTAT from 1930 to 1980
This report presents a comprehensive comparison between the historical agricultural data collected from the old digitized World Census of Agriculture (WCA) and the current data available on FAOSTAT for the period spanning from 1930 to 1980. The primary indicators under scrutiny are the number and area of agricultural holdings, as these are the only metrics available for this time frame in both datasets. The analysis aims to elucidate the similarities and discrepancies in data availability, values, and country coverage between the two sources, as well as identify exclusive countries unique to each dataset.
The comparison involved extracting data on the total number and area of agricultural holdings from both the old digitized WCA and FAOSTAT datasets. This data was then meticulously processed and prepared for analysis. The core areas of examination encompass:
Same Countries with Different Values: This facet focuses on shared countries between the two datasets and scrutinizes instances where specific indicators exhibit disparities in their reported values.
Countries Available in the Old WCA Digitalized Dataset Only: Here, the spotlight is on countries present solely in the old digitized WCA dataset but absent in FAOSTAT. The reasons behind this discrepancy warrant further exploration.
Countries Available in FAOSTAT Only: This portion hones in on countries exclusively featured in the FAOSTAT dataset, which are not part of the old WCA digitalized dataset. Investigating the source of this unique data could provide insights into its origin and utility.
Certain countries within the WCA old data present unique challenges and require consultation with Amanda and the team. These specific cases are highlighted in the WCA 1950 and 1960 datasets. More details and access to this information can be found in the provided link.
| WCA 1950 | WCA 1960 |
|---|---|
| Northern Rhodesia (European holdings) | Virgin Islands (U.K.) |
| Ryukyu Islands | China (Taiwan) |
| Saar | Tanganyika (commercial holding only) |
| Southern Rhodesia(European holdings) | United Arab Republic |
The analysis of data from 1930 to 1980 reveals that both the WCA and FAOSTAT datasets encompass a significant number of countries. However, for some common countries, disparities exist between reported indicator values. These variations might be attributed to rounding in the FAOSTAT dataset and the conversion between different units of measurement.
For instance, the comparison between the total number of holdings and agricultural area for Australia reveals higher values in the WCA old data for the year 1930, while FAOSTAT records lower values for the same period. Addressing such discrepancies will necessitate team discussions for potential resolution.
To facilitate a thorough investigation of these differences, an interactive R Shiny app was developed. This tool visualizes the historical trends of all countries based on the percentage difference between the WCA old data and FAOSTAT data. Notably, countries falling within the percentage difference range of -1% to 1% can be attributed to rounding issues, rendering their discrepancies insignificant.
For countries with percentage differences ranging from 1% to 700%, in-depth analyses on a country-by-country basis are essential to determine the underlying issues. The default percentage difference range of the app is set to visualize this category. When we select Area, there are 13 countries in this category and 20 countries when we select holdings.
For countries that fall within the percentage difference range from -100% to -1%, we have to investigate on a country-by-country basis the underlying issues with these countries. As you can see, when we select Area, there are 8 countries in this category and 5 countries when we select holdings on the app.
You can visit the Shiny app here to visualize this historical trends for number of holdings and area.
Conversely, there are countries exclusively featured in the old WCA digitalized dataset that are conspicuously absent in FAOSTAT. While the rationale behind excluding this data from FAOSTAT remains unclear, it is recommended that these exclusions be addressed by incorporating the missing data into FAOSTAT for all WCA rounds. You can see these countries here in the second sheet for each round.
Lastly, attention was directed toward countries exclusively present in FAOSTAT but not in the old WCA digitalized dataset. A preliminary investigation yielded no insights into the source of this data. As a result, it is advised to remove this data from the WCA domain until its origin and reliability can be established. You can see these countries here in the third sheet for each round.
The detailed findings, explanations, and links offer a comprehensive view of the results of this comparison analysis, paving the way for informed decisions and potential actions.