The WPD/Codalab/Catapult Challenge 1 was directed at estimating the range of value at a one minute resolution within each half hourly period given only half hourly measurements.

Looking at the 3x sites identified, there is some additional WPD open data that could assist here (the <external_dist_subs-3.csv> data from connecteddata.westernpower.co.uk)

Summary of total solarPV associated with each site:

It isn’t clear if these totals are the sum of solar installations or kW registered. However, either way, there appears to be more solar at Staplegrove.

The variability in the 1-min dataset at Staplegrove is reviewed further below. The data is divided into daylight and night-time series (suncalc provides sun altitude for any lat-long, i.e. night-time occurs for negative solar altitudes).

Boxplots for each of the 3x sites with each value separated into ‘daytime’ vs ‘night-time’ are included here.

All sites, all hours - 1 min dataset

ggplot(data=all1) +
  geom_boxplot(aes(x=dn, y=hl, fill=attrId)) +
  labs(
    title="All sites: (value-max - value-min)\n(1min data) log scale\nRed lines at 0.1, 0.05, i.e. 10%, 5%",
    x=" ",
    y="(Maxvalue - Minvalue)") +
  geom_hline(yintercept=c(0.1,0.05),
             color="red") +
  scale_y_log10(labels =trans_format('log10',
                                     math_format(10^.x)),
                breaks=breaks_log(n=4)) +
  theme_economist() +
  theme(legend.title=element_blank())

In comparison, all sites, 30 min dataset:

The range of max-min values at Staplegrove during daytime is significantly higher than for other sites.

There does appear to be a much larger variation in daytime recordings, particularly where there is a preponderance of solarPV.

(Additional graphs are included below which show the range of max-min values over each hour during daytime and night-time).

It may be useful to consider the variation in solarPV generation over very short timescales

There is a 1-second dataset available at https://uol.de/energiemeteorologie/forschung/solarenergie

And a graph of the 1-second (1Hz) variation is shown:

Perhaps the wider variation in sites associated with solarPV could be identified using a flickermeter?

Additional graphs of the range of values for Staplegrove

1-minute data:

Apart from a few outliers, the variability in night-time values is below 10%, and even 5% covers the variability of most values.

Graphing the same measurements at Staplegrove for daylight hours:

Repeat for daylight hours, 30-min dataset:

Recommendation:

Where solarPV is significant, check voltage flicker, and assume large variation in values for 30min data when the sun’s shining with intermittent cloud cover.

LS0tCnRpdGxlOiAiV1BEIE9wZW5EYXRhIENoYWxsZW5nZSAxIgpvdXRwdXQ6IAogIGh0bWxfbm90ZWJvb2s6IAogICAgY29kZV9mb2xkaW5nOiBoaWRlCiAgICBmaWdfd2lkdGg6IDQKICAgIGZpZ19oZWlnaHQ6IDQKLS0tCmBgYHtjc3MsIGVjaG89RkFMU0V9CnAgewogIGZvbnQtc2l6ZTogMThweDsKfQoubWFpbi1jb250YWluZXIgewogIG1heC13aWR0aDogNjAwcHg7CiAgbWFyZ2luLWxlZnQ6IGF1dG87CiAgbWFyZ2luLXJpZ2h0OiBhdXRvOwp9CmBgYAoKIyMjIyBUaGUgV1BEL0NvZGFsYWIvQ2F0YXB1bHQgQ2hhbGxlbmdlIDEgd2FzIGRpcmVjdGVkIGF0IGVzdGltYXRpbmcgdGhlIHJhbmdlIG9mIHZhbHVlIGF0IGEgb25lIG1pbnV0ZSByZXNvbHV0aW9uIHdpdGhpbiBlYWNoIGhhbGYgaG91cmx5IHBlcmlvZCBnaXZlbiBvbmx5IGhhbGYgaG91cmx5IG1lYXN1cmVtZW50cy4KCkxvb2tpbmcgYXQgdGhlIDN4IHNpdGVzIGlkZW50aWZpZWQsIHRoZXJlIGlzIHNvbWUgYWRkaXRpb25hbCBXUEQgb3BlbiBkYXRhIHRoYXQgY291bGQgYXNzaXN0IGhlcmUgKHRoZSBcPGV4dGVybmFsX2Rpc3Rfc3Vicy0zLmNzdlw+IGRhdGEgZnJvbSBjb25uZWN0ZWRkYXRhLndlc3Rlcm5wb3dlci5jby51aykKClN1bW1hcnkgb2YgdG90YWwgc29sYXJQViBhc3NvY2lhdGVkIHdpdGggZWFjaCBzaXRlOgoKYGBge3IgZWNobz1GQUxTRSwgd2FybmluZz1GQUxTRX0KI3dwcyA8LSBmcmVhZCgiaHR0cHM6Ly9jb25uZWN0ZWRkYXRhLndlc3Rlcm5wb3dlci5jby51ay9kYXRhc2V0LzI5ZDQzNWMyLTBjYmUtNDQyZC05NmZlLWEyMjlhMDMwN2ZiYS9yZXNvdXJjZS8xYWFjOGFmOS04MTNiLTQ1ZjgtOGQ3MC05NjIyZDY5NmQ3NmIvZG93bmxvYWQvZXh0ZXJuYWxfZGlzdF9zdWJzLTMuY3N2IikKd3BzcyA8LSB3cHMgJT4lCiAgc2VsZWN0KGBQcmltYXJ5IE5hbWVgLCBTb2xhcikgJT4lCiAgZmlsdGVyKGBQcmltYXJ5IE5hbWVgICVpbiUgYygiU3RhcGxlZ3JvdmUiLCAiTW91c2Vob2xlIiwgIkdlZXZvciIpKSAlPiUKICBncm91cF9ieShgUHJpbWFyeSBOYW1lYCkgJT4lCiAgc3VtbWFyaXNlKHNvbGFyID0gc3VtKFNvbGFyKSkKCmdncGxvdCh3cHNzLCBhZXMoeD1gUHJpbWFyeSBOYW1lYCwgeT1zb2xhcikpICsKICBnZW9tX2JhcihzdGF0PSJJZGVudGl0eSIpICsKICBsYWJzKHRpdGxlPSJTb2xhclBWIGF0IHRocmVlIHNpdGVzIiwKICAgICAgIHg9IiAiKSArCiAgdGhlbWVfZWNvbm9taXN0KCkKYGBgCgpJdCBpc24ndCBjbGVhciBpZiB0aGVzZSB0b3RhbHMgYXJlIHRoZSBzdW0gb2Ygc29sYXIgaW5zdGFsbGF0aW9ucyBvciBrVyByZWdpc3RlcmVkLiBIb3dldmVyLCBlaXRoZXIgd2F5LCB0aGVyZSBhcHBlYXJzIHRvIGJlIG1vcmUgc29sYXIgYXQgU3RhcGxlZ3JvdmUuCgpUaGUgdmFyaWFiaWxpdHkgaW4gdGhlIDEtbWluIGRhdGFzZXQgYXQgU3RhcGxlZ3JvdmUgaXMgcmV2aWV3ZWQgZnVydGhlciBiZWxvdy4gVGhlIGRhdGEgaXMgZGl2aWRlZCBpbnRvIGRheWxpZ2h0IGFuZCBuaWdodC10aW1lIHNlcmllcyAoc3VuY2FsYyBwcm92aWRlcyBzdW4gYWx0aXR1ZGUgZm9yIGFueSBsYXQtbG9uZywgaS5lLiBuaWdodC10aW1lIG9jY3VycyBmb3IgbmVnYXRpdmUgc29sYXIgYWx0aXR1ZGVzKS4KCkJveHBsb3RzIGZvciBlYWNoIG9mIHRoZSAzeCBzaXRlcyB3aXRoIGVhY2ggdmFsdWUgc2VwYXJhdGVkIGludG8gJ2RheXRpbWUnIHZzICduaWdodC10aW1lJyBhcmUgaW5jbHVkZWQgaGVyZS4KCgpBbGwgc2l0ZXMsIGFsbCBob3VycyAtIDEgbWluIGRhdGFzZXQKCmBgYHtyIHdhcm5pbmc9RkFMU0V9CmdncGxvdChkYXRhPWFsbDEpICsKICBnZW9tX2JveHBsb3QoYWVzKHg9ZG4sIHk9aGwsIGZpbGw9YXR0cklkKSkgKwogIGxhYnMoCiAgICB0aXRsZT0iQWxsIHNpdGVzOiAodmFsdWUtbWF4IC0gdmFsdWUtbWluKVxuKDFtaW4gZGF0YSkgbG9nIHNjYWxlXG5SZWQgbGluZXMgYXQgMC4xLCAwLjA1LCBpLmUuIDEwJSwgNSUiLAogICAgeD0iICIsCiAgICB5PSIoTWF4dmFsdWUgLSBNaW52YWx1ZSkiKSArCiAgZ2VvbV9obGluZSh5aW50ZXJjZXB0PWMoMC4xLDAuMDUpLAogICAgICAgICAgICAgY29sb3I9InJlZCIpICsKICBzY2FsZV95X2xvZzEwKGxhYmVscyA9dHJhbnNfZm9ybWF0KCdsb2cxMCcsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBtYXRoX2Zvcm1hdCgxMF4ueCkpLAogICAgICAgICAgICAgICAgYnJlYWtzPWJyZWFrc19sb2cobj00KSkgKwogIHRoZW1lX2Vjb25vbWlzdCgpICsKICB0aGVtZShsZWdlbmQudGl0bGU9ZWxlbWVudF9ibGFuaygpKQpgYGAKCkluIGNvbXBhcmlzb24sIGFsbCBzaXRlcywgMzAgbWluIGRhdGFzZXQ6CgpgYGB7ciBlY2hvPUZBTFNFLCB3YXJuaW5nPUZBTFNFfQpnZ3Bsb3QoZGF0YT1hMzApICsKICBnZW9tX2JveHBsb3QoYWVzKHg9ZG4sIHk9aGwsIGZpbGw9c2l0ZSkpICsKICBsYWJzKAogICAgdGl0bGU9IkFsbCBzaXRlczogKHZhbHVlLW1heCAtIHZhbHVlLW1pbilcbigzMG1pbiBkYXRhKVxuUmVkIGxpbmVzIGF0IDAuMSwgMC4wNSwgaS5lLiAxMCUsIDUlIiwKICAgIHg9IiAiLAogICAgeT0iKE1heHZhbHVlIC0gTWludmFsdWUpIikgKwogIGdlb21faGxpbmUoeWludGVyY2VwdD1jKDAuMSwwLjA1KSwKICAgICAgICAgICAgIGNvbG9yPSJyZWQiKSArCiAgc2NhbGVfeV9sb2cxMChsYWJlbHMgPXRyYW5zX2Zvcm1hdCgnbG9nMTAnLCBtYXRoX2Zvcm1hdCgxMF4ueCkpLAogICAgICAgICAgICAgICAgYnJlYWtzPWJyZWFrc19sb2cobj00KSkgKwogIHRoZW1lX2Vjb25vbWlzdCgpICsKICB0aGVtZShsZWdlbmQudGl0bGU9ZWxlbWVudF9ibGFuaygpKQpgYGAKClRoZSByYW5nZSBvZiBtYXgtbWluIHZhbHVlcyBhdCBTdGFwbGVncm92ZSBkdXJpbmcgZGF5dGltZSBpcyBzaWduaWZpY2FudGx5IGhpZ2hlciB0aGFuIGZvciBvdGhlciBzaXRlcy4KCiMjIyMgVGhlcmUgZG9lcyBhcHBlYXIgdG8gYmUgYSBtdWNoIGxhcmdlciB2YXJpYXRpb24gaW4gZGF5dGltZSByZWNvcmRpbmdzLCBwYXJ0aWN1bGFybHkgd2hlcmUgdGhlcmUgaXMgYSBwcmVwb25kZXJhbmNlIG9mIHNvbGFyUFYuCiMjIyMgKEFkZGl0aW9uYWwgZ3JhcGhzIGFyZSBpbmNsdWRlZCBiZWxvdyB3aGljaCBzaG93IHRoZSByYW5nZSBvZiBtYXgtbWluIHZhbHVlcyBvdmVyIGVhY2ggaG91ciBkdXJpbmcgZGF5dGltZSBhbmQgbmlnaHQtdGltZSkuCgojIyMjIEl0IG1heSBiZSB1c2VmdWwgdG8gY29uc2lkZXIgdGhlIHZhcmlhdGlvbiBpbiBzb2xhclBWIGdlbmVyYXRpb24gb3ZlciB2ZXJ5IHNob3J0IHRpbWVzY2FsZXMKClRoZXJlIGlzIGEgMS1zZWNvbmQgZGF0YXNldCBhdmFpbGFibGUgYXQKaHR0cHM6Ly91b2wuZGUvZW5lcmdpZW1ldGVvcm9sb2dpZS9mb3JzY2h1bmcvc29sYXJlbmVyZ2llCgpBbmQgYSBncmFwaCBvZiB0aGUgMS1zZWNvbmQgKDFIeikgdmFyaWF0aW9uIGlzIHNob3duOgoKYGBge3IgZWNobz1GQUxTRSwgd2FybmluZz1GQUxTRX0Ka25pdHI6OmluY2x1ZGVfZ3JhcGhpY3MocmVwKCJodHRwczovL3VvbC5kZS9mLzUvaW5zdC9waHlzaWsvYWcvZW5tZXQvMjAyMl9taWcvUlRFbWFnaWNDXzFhODc2MjRiMGQucG5nLnBuZyIsIDEpKQpgYGAKCiMjIyMgUGVyaGFwcyB0aGUgd2lkZXIgdmFyaWF0aW9uIGluIHNpdGVzIGFzc29jaWF0ZWQgd2l0aCBzb2xhclBWIGNvdWxkIGJlIGlkZW50aWZpZWQgdXNpbmcgYSBmbGlja2VybWV0ZXI/CgojIyMjIEFkZGl0aW9uYWwgZ3JhcGhzIG9mIHRoZSByYW5nZSBvZiB2YWx1ZXMgZm9yIFN0YXBsZWdyb3ZlCgoxLW1pbnV0ZSBkYXRhOgoKYGBge3IgZWNobz1GQUxTRSwgd2FybmluZz1GQUxTRX0KZ2dwbG90KGRhdGE9c20xbikgKwogIGdlb21fYm94cGxvdChhZXMoeD1oaCwgeT1obCwgZ3JvdXA9aGgpKSArCiAgbGFicygKICAgIHRpdGxlPSJTdGFwbGVncm92ZTogKHZhbHVlLW1heCAtIHZhbHVlLW1pbilcbk5pZ2h0LXRpbWUgICAoMW1pbiBkYXRhKVxuUmVkIGxpbmVzIGF0IDAuMSwgMC4wNSwgaS5lLiAxMCUsIDUlIiwKICAgIHg9IkhvdXIgb2YgZGF5IiwKICAgIHk9IihNYXh2YWx1ZSAtIE1pbnZhbHVlKSBMb2cgc2NhbGUiKSArCiAgZ2VvbV9obGluZSh5aW50ZXJjZXB0PWMoMC4xLDAuMDUpLAogICAgICAgICAgICAgY29sb3I9InJlZCIpICsKICBzY2FsZV95X2xvZzEwKGxhYmVscyA9dHJhbnNfZm9ybWF0KCdsb2cxMCcsIG1hdGhfZm9ybWF0KDEwXi54KSksCiAgICAgICAgICAgICAgICBicmVha3M9YnJlYWtzX2xvZyhuPTQpKSArCiAgc2NhbGVfeF9jb250aW51b3VzKGJyZWFrcyA9IGMoMCwzLCA2LDksMTIsMTUsMTgsMjEsMjQpKSArCiAgdGhlbWVfZWNvbm9taXN0KCkKYGBgCgpBcGFydCBmcm9tIGEgZmV3IG91dGxpZXJzLCB0aGUgdmFyaWFiaWxpdHkgaW4gbmlnaHQtdGltZSB2YWx1ZXMgaXMgYmVsb3cgMTAlLCBhbmQgZXZlbiA1JSBjb3ZlcnMgdGhlIHZhcmlhYmlsaXR5IG9mIG1vc3QgdmFsdWVzLgoKR3JhcGhpbmcgdGhlIHNhbWUgbWVhc3VyZW1lbnRzIGF0IFN0YXBsZWdyb3ZlIGZvciBkYXlsaWdodCBob3VyczoKCmBgYHtyIGVjaG89RkFMU0UsIHdhcm5pbmc9RkFMU0V9CmdncGxvdChkYXRhPXNtMWQpICsKICBnZW9tX2JveHBsb3QoYWVzKHg9aGgsIHk9aGwsIGdyb3VwPWhoKSkgKwogIGxhYnMoCiAgICB0aXRsZT0iU3RhcGxlZ3JvdmU6ICh2YWx1ZS1tYXggLSB2YWx1ZS1taW4pXG5EYXlsaWdodCAoMW1pbiBkYXRhKVxuUmVkIGxpbmVzIGF0IDAuMSwgMC4wNSwgaS5lLiAxMCUsIDUlIiwKICAgIHg9IkhvdXIgb2YgZGF5IiwKICAgIHk9Ik1heHZhbHVlIC0gTWludmFsdWUgTG9nIHNjYWxlIikgKwogIGdlb21faGxpbmUoeWludGVyY2VwdD1jKDAuMSwwLjA1KSwKICAgICAgICAgICAgIGNvbG9yPSJyZWQiKSArCiAgc2NhbGVfeV9sb2cxMChsYWJlbHMgPXRyYW5zX2Zvcm1hdCgnbG9nMTAnLCBtYXRoX2Zvcm1hdCgxMF4ueCkpLAogICAgICAgICAgICAgICAgYnJlYWtzPWJyZWFrc19sb2cobj00KSkgKwogIHNjYWxlX3hfY29udGludW91cyhicmVha3MgPSBjKDAsMywgNiw5LDEyLDE1LDE4LDIxLDI0KSkgKwogIHRoZW1lX2Vjb25vbWlzdCgpCmBgYAoKUmVwZWF0IGZvciBkYXlsaWdodCBob3VycywgMzAtbWluIGRhdGFzZXQ6CgpgYGB7ciBlY2hvPUZBTFNFfQoKc2FkJGhoIDwtIGhvdXIoc2FkJHRpbWUpCmdncGxvdChkYXRhPXNhZCkgKwogIGdlb21fYm94cGxvdChhZXMoeD1oaCwgeT1obCwgZ3JvdXA9aGgpKSArCiAgbGFicygKICAgIHRpdGxlPSJTdGFwbGVncm92ZTogKHZhbHVlLW1heCAtIHZhbHVlLW1pbilcbkRheWxpZ2h0IGhvdXJzIG9ubHkgKDMwbWluIGRhdGEpXG5SZWQgbGluZXMgYXQgMC4xLCAwLjA1LCBpLmUuIDEwJSwgNSUiLAogICAgeD0iSG91ciBvZiBkYXkiLAogICAgeT0iTWF4dmFsdWUgLSBNaW52YWx1ZSAgTG9nIHNjYWxlIikgKwogIGdlb21faGxpbmUoeWludGVyY2VwdD1jKDAuMSwwLjA1KSwKICAgICAgICAgICAgIGNvbG9yPSJyZWQiKSArCiAgc2NhbGVfeV9sb2cxMChicmVha3M9MTBeKC0xOjMpKSArCiAgc2NhbGVfeF9jb250aW51b3VzKGJyZWFrcyA9IGMoMCwzLCA2LDksMTIsMTUsMTgsMjEsMjQpKSArCiAgdGhlbWVfZWNvbm9taXN0KCkKYGBgCgojIyMjIFJlY29tbWVuZGF0aW9uOgojIyMjIFdoZXJlIHNvbGFyUFYgaXMgc2lnbmlmaWNhbnQsIGNoZWNrIHZvbHRhZ2UgZmxpY2tlciwgYW5kIGFzc3VtZSBsYXJnZSB2YXJpYXRpb24gaW4gdmFsdWVzIGZvciAzMG1pbiBkYXRhIHdoZW4gdGhlIHN1bidzIHNoaW5pbmcgd2l0aCBpbnRlcm1pdHRlbnQgY2xvdWQgY292ZXIuCgo8RU5EPgo=