Analysis of SILO gridded data, 2017 - 2020

Introduction

Streamflow modelling often involves taking inputs of climatic variables such as rainfall and evapotranspiration, and modelling streamflow using a rainfall-runoff model. Source model - a streamflow modelling software - is produced by eWater Ltd and is the primary water resource modelling tool used by XXXXXX.

Timeseries of rainfall and evapotranspiration data are key inputs to the Source model. A key process is the calibration of modelled streamflow with observed or gauged streamflow. The calibration of constants within the rainfall-runoff models enable climatic inputs of rainfall and evapotranspiration to reliably and accurately predict streamflow.

QLD Government produce SILO, which is a daily timeseries of gridded climatic data. SILO is updated daily and is produced through interpolation of rainfall gauge data across the whole of Australia. SILO is a commonly used input to streamflow models, including Source.

In 2017 XXXXXX acquired a Source model of the Upper Murrumbidgee River catchment. The model was calibrated over a period of 1980 - 2016 with SILO rainfall and evapotranspiration data.

In 2020 an updated SILO dataset was acquired from SILO, and was imported into the Source model to update the time series. A quick check revealed that modelled streamflow changed dramatically, indicating significant changes in the SILO dataset between 2017 and 2020. These changes are alluded to on the SILO website.

This analysis aims to explore the differences in the SILO gridded datasets for rainfall (mm/day) and Morton’s wet environment potential evapotranspiration (mwet, mm/day).

The following 4 questions are to be answered: 1 How has the SILO rainfall and evapotranspiration data changed between 2017 and 2020? 2 How as this change in SILO impacted upon modelled streamflow 3 Is the updated SILO dataset (2020) superior to the older (2017) version? 4 Is predicted streamflow more accurate with the updated SILO dataset?

Methods

Data collation

Daily rainfall (mm/day) and daily evapotranspiration (mm/day) were extracted for each of XX subcatchments within the Source model of the Upper Murrumbidgee River, covering the period 01/01/1980 to 31/12/2016. These data were extracted from the Source model containing the originally uploaded 2017 SILO dataset.

The 2020 SILO dataset for rainfall and evapotranspiration covering the same time period (1980 - 2016) was imported into the Source model using the ‘Climate Date Import Tool’ within Source. The climate data is a gridded 0.05 degree product, which is averaged to produce an area-weighted average for each subcatchment. After importation, subcatchment rainfall and evapotranspiration were extracted for analysis.

Modelled streamflow (ML/Day) from 1980 - 2016 were extracted from the model (current scenario) for 16 unregulated, gauged catchments throughout the Upper Murrumbidgee River catchment. Streamflow was calibrated using NSE, blah blah to minimise bias blah blah. Following output of streamflow data predicted using the 2017 SILO input, the 2020 SILO data was imported into Source, and the model run again to predict streamflow with the new SILO dataset.

Gauged streamflow data (ML/Day) was extracted from relevant databases (eg. HYDSTRA, BoM) for the 16 streamflow gauges examined in this study. Data was cleaned and processed for missing values, erroneous data etc.

Rainfall gauge data (mm/day) for XX rain gauges was extracted from the XXXXXX HYDSTRA database. Rainfall data was cleaned and processed for missing values, erroneous values etc.

Data analysis

Direct comparisons were made between the SILO datasets for both rainfall and evapotranspiration. Absolute and percentage changes in annual rainfall volume and evapotranspiration volume were calculated by comparing the 2020 SILO dataset to the 2017 SILO dataset. Temporal effects were examined by plotting annual rainfall and evapotranspiration by year. Spatial patterns were examined by mapping average percentage change in rainfall and evapotranspiration.

Changes in predicted streamflow were explored for 16 streamflow gauges. Flow exceedance curves were calculated and plotted for streamflow prediction derived from the calibrated Source model using the 2017 SILO data, and with the 2020 SILO input data, along with gauged streamflow data. Cumulative streamflow volume was also plotted.

To statistically validate differences in the predicted streamflow, annual volume percent bias, mean absolute error and root mean square error were also calculated. Box and whisker plots were produced to examine differences between the two predicted streamflow datasets, in relation to the gauged streamflow data.

To investigate whether the 2020 SILO dataset is superior to the 2017 dataset, an independent dataset is required. While empirical data for evapotranspiration is not available, an extensive network of automated raingauges (0.2mm tipping bucket) exist throughout the Australian Capital Territory and surrounds. Raingauges were spatially aligned with subcatchments within the Source model, and where multiple raingauges exist within a subcatchment, daily rainfall was averaged among the gauges. Finally where missing rainfall data occurred, datasets (SILO rainfall and gauged rainfall) were restricted to where rainfall data was available. For annual rainfall calculations, if more than 30 days of gauged rainfall data was missing, the entire year was excluded from analysis.

For annual rainfall analysis, average annual percent bias was calculated for each subcatchment for both the 2017 SILO dataset and the 2020 SILO dataset. Daily gauged rainfall and SILO daily rainfall were plotted and examined. In the instance of an accurate, unbiased SILO dataset, we would the relationship between gauged rainfall and SILO rainfall at any given gauge location to be a 1:1 relationship with an intercept of zero. To examine this, a simple linear model, SILO rainfall = b * gauged rainfall + c was fitted to each year of rainfall data for each gauge. Model co-efficients (intercept and slope terms) were extracted and plotted. Finally, a geometric smoother was applied to model coefficients (slope and intercept) over time, to examine temporal changes in bias between the two SILO datasets, relative to the gauged rainfall dataset.

Bias, mean absolute error and root mean square error were calculated for subcatchment. These were computed in two ways. Firstly, to explore spatial variation, average values were computed over all years for each subcatchment. Secondly to examine temporal variation, average values were computed over all subcatchments, for each each year. Box and whisker plots were produced for each measure of error, for both subcatchments, and years.

Finally, maps were produced of the relevant subcatchments to spatially represent average bias, mean absolute error and root mean square error between the two SILO datasets.

  • Updated SILO streamflow performance TBF

Results

Q1. How has SILO changed from 2017 to 2020?

Figure 1. Annual rainfall difference (mm/Year) for each subcatchment, through time (1980 - 2016). Dark blue line shows the average difference.

Figure 2. Annual percentage change in rainfall (%) for each subcatchment, through time (1980 - 2016). Dark blue line shows the average difference.

Figure 3. Frequency histogram of average annual percentage change in rainfall for subcatchments

Figure 4. Annual mwet difference (mm/Year) for each subcatchment, through time (1980 - 2016)

Figure 5. Annual mwet difference (mm/day) for each subcatchment through time (1980 - 2016). Dark-blue line shows the average difference.

Figure 6. Frequency histogram of average annual percentage difference in mwet for all subcatchments

Figure 7. Map of Source model subcatchments showing Average annual difference in rainfall (%) between new and old input datasets.

Figure 8. Map of Source model subcatchments showing average annual difference in evapotranspiration (%) between new and old input datasets.

Q2. How have changes to SILO impacted upon predicted streamflow?

A subset of streamflow gauges were selected for this analysis with the following traits:

  • unregulated
  • hydrologically independent (mostly)
  • spread across a range of catchments (spatially)
  • Good gauged data coverage

Table 1. List of 16 streamflow gauges included in this analysis

Gauge Number Site Name
410026 Yass River @ Yass
410033 Murrumbidgee River @ Mittagang
410062 Numeralla River @ School
410076 Strike-a-light River
410088 Goodradigbee River @ Wee Jasper
410141 Michelago creek @ Michelago
410705 Molonglo River @ Burbong
410713 Paddys River @ Riverview
410730 Cotter River @ Gingera
410731 Gudgeny River @ Mt. Tennant
410733 Condor Creek @ Threeways
410736 Orroral River @ Campsite
410745 Yarralumla Creek @ Curtin
410750 Ginninderra Creek @ Charnwood
410775 Sullivans Creek @ Barry Drive
410781 Queanbeyan River us Googong

Figure 9. Flow exceedance curves for modelled and observed (gauged) streamflow at 16 streamflow gauges around the Upper Murrumbidgee River catchment. Modelled represents the calibrated Source model outputs with the 2017 SILO inputs, while New_Modelled represents the Source model outputs with the 2020 SILO inputs (uncalibrated).

Figure 10. Cumulative flow plots for 16 streamflow gauges around the Upper Murrumbidgee River catchment. Modelled represents the calibrated Source model outputs with the 2017 SILO inputs, while New_Modelled represents the Source model outputs with the 2020 SILO inputs (uncalibrated).

Figure 11. Mean Error, Mean Absolute Error and Root Mean Square Error in annual streamflow volume from 16 streamflow gauges around the Upper Murrumbidgee River catchment. Old SILO represents the Source model outputs with the 2017 SILO inputs (calibrated) while New SILO represents the Source model outputs with the 2020 SILO inputs (uncalibrated). Note Needs work (formulas are suspect).

Q3. Is 2020 SILO better than 2017 SILO?

Figure 12. Average annual rainfall percent bias for the old SILO dataset (2017) and the new SILO dataset (2020). Figure shows that bias correction has occurred in the latter dataset, with the average percent bias (blue line) closer to 0 than the average percent bias from the old dataset (red line).

Figure 13. Relationship between gauged annual volume and old SILO annual volume (red points) and new SILO (blue points) in three random subcatchments. The dashed black line shows the expected 1:1 relationship. Deviation from this relationship (shown by the red and blue linear fits) indicate potential bias in SILO data.

Figure 14. Geometric smoother fitted to the linear model coefficients (intercept and slope parameter) of the relationship between gauged annual volume and old SILO annual volume (red lines) and new SILO volumes (blue lines), through time (1980 - 2016). This intercept values should be closer to 0, and slope parameters should be close to 1. Fits show that bias correction has been effective in the new SILO in terms of both intercept and slope characteristics. In particular, control of variation through time has been effectively achieved.

Figure 15. Box and whisker plots of Bias, Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) between old SILO (blue) and new SILO (red). Variation is presented in terms of variation between subcatchments (left panels), and years (right panels).

Figure 16. Chloropleths showing comparison between old SILO and new SILO Bias, Mean Absolute Error and Root Mean Square Error (RMSE) among subcatchments

Q4. Is streamflow prediction improved with 2020 SILO input data?

TBF

  • Comparison of predicted streamflow to gauged.

Discussion

It should be noted that the SILO website specifies that changes in their datasets occur continuously, as new data becomes available, errors are found and removed, and mathematical processes improve. This may explain the differences and a key assumption would be that the latter dataset would be superior.

There is an overall effect of reduced annual rainfall between the new and old datasets, and an overall average increase in evapotranspiration. This is consistent with the observed impacts on streamflow seen in the source model, with a general reduction in predicted streamflow due to the change in input data.

Rainfall tended to change up to ± 250mm per year, or ± 25%. This is a significant variation, but otherwise showed no obvious temporal patterns (Figure 2). Conversely, changes in evapotranspiration an average 30mm/Year increase in evapotranspiration, ranging from -50mm/year to + 125mm/Year. Furthermore, a few specific years (1997, 2008) exhibited systematic shifts, possibly suggesting specific input-data issues within the SILO data interpolation process.

There is no obvious gradient or spatial pattern to the changes in the input data. Some catchments are the centre of the domain (just south of the southern end of the Cotter catchment) exhibit a dramatic increase in evapotranspiration and reduction in rainfall. Conversely a tiny catchment on the northwestern boundary sees a -2.8% change in evapotranspiration.

The Alluvium report describes that the SILO data was found to be inferior to the gauge data that was present in the Molonglo catchment. Further investigation could examine to see if this new input dataset more accurately characterises what has been gauged. If so, this would imply the new dataset is superior.

Given the original dataset is not available from Alluvium at this stage, it seems unavoidable that the original data will have to be discarded. SILO has identified and rectified errors in their dataset in 2018, and as such, probably warrants changing the input data anyway.

Recalibrating the Source model would appear to be necessary. This process demonstrates that maintaining good oversight and storage of input data will be a critical component of managing the Source model long-term.