A Quick Guide to National Grid’s Carbon Intensity API for R Users

R Markdown

National Grid ESO (UK) released an API to measure carbon intensity. This is calculated as the sum of products of different electricity generation types and their “Carbon Intensity”, given in grams of carbon dioxide per kilowatt-hour, divided by electricity demand at the time.

Hydro-, Nuclear-, Wind-, Solar-, and Pumped Storage generation have carbon intensity values set to zero, whereas the rest do not.

More information can be found here: https://carbonintensity.org.uk/

The API - Initial Steps

Information on how to use the API is available here: https://carbon-intensity.github.io/api-definitions/#carbon-intensity-api-v2-0-0

The first thing you may notice is that code samples for R are missing. So we need to think of our own.

The other caveat is we can only pull 1 month’s worth of data at a time. For now, let’s focus on a week.

If you are new to RCurl and it’s wonderful properties, you may wish to tinker with the below code.

library(RCurl); library(tidyverse)
the_url <- paste0("https://api.carbonintensity.org.uk/intensity/", as.Date("2019-04-01"), "T00:00Z/", as.Date("2019-04-07"), "T23:59Z")
df_json <- RCurl::basicTextGatherer()
df_json$reset()
RCurl::curlPerform(url = the_url,  writefunction = df_json$update)
data_response <- df_json$value() %>%  jsonlite::parse_json(simplifyVector = TRUE) %>% as.data.frame()
data_response$info <- NULL
data_response <- data.frame(data_response$data.from, data_response$data.to, data_response$data.intensity)
colnames(data_response)[1:2] <- c("data.from", "data.to")

head(data_response) %>% kable(caption = "Carbon Intensity")

Creating a Function for the API

Sticking with the dataset above, we can create our own function to pull data from a custom date range.

The dataset starts from 2017-09-12 onwards.

In order to parse the dates in an R-suitable format, you may wish to use an additional function, below defined as “date_fixer”. The default format is ISO8601.

library(magrittr)

date_fixer <- function(v){strptime(x = v, format = "%Y-%m-%dT%H:%MZ")}

carbon_api <- function(fromdate, todate){
fromdate <- as.Date(fromdate); todate <- as.Date(todate);

datelist <- seq.Date(from = fromdate, to = todate, by = "1 day")
carbon_intensity <- data.frame(matrix(nrow = 0, ncol = 5)) %>% set_colnames(c("data.from", "data.to", "data.intensity.forecast", "data.intensity.actual", "data.intensity.index"))

for (i in 1:length(datelist)){
if(fromdate < as.Date("2017-09-12")) {cat("Please select a from-date of 2017-09-12 or later", "\n\n"); break}
df_json <- RCurl::basicTextGatherer()
df_json$reset() # For caution
paste0("https://api.carbonintensity.org.uk/intensity/", datelist[i], "T00:00Z/", datelist[i], "T23:59Z") %>%
RCurl::curlPerform(url = .,  writefunction = df_json$update)
data_response <- df_json$value() %>%  jsonlite::parse_json(simplifyVector = TRUE) %>% as.data.frame()
data_response$info <- NULL
data_response <- data.frame(data_response$data.from, data_response$data.to, data_response$data.intensity)
colnames(data_response)[1:2] <- c("data.from", "data.to")
carbon_intensity  <- rbind(carbon_intensity, data_response)
} # close the loop
carbon_intensity$data.from %<>% date_fixer 
carbon_intensity$data.to %<>% date_fixer 
return(carbon_intensity)
} # close the function

To test whether our function works, you may wish to run some tests:

# First test - invalid date range
carbon_api(fromdate =  as.Date("2017-02-01"), todate = as.Date("2017-03-31")) 

## Please select a from-date of 2017-09-12 or later

## [1] data.from               data.to                 data.intensity.forecast
## [4] data.intensity.actual   data.intensity.index   
## <0 rows> (or 0-length row.names)

# Second test - valid date range
carbon_api(fromdate =  as.Date("2017-10-01"), todate = as.Date("2017-10-31")) %>% head() %>% kable()

Looks like it works!

The Generation Mix

The other dataset available is the generation mix, accessible by similar means. It is slightly more difficult to wrangle, although R users may wish to make use of the tibble::enframe function. Here is an example of a user-defined function:

gm_names <- c("biomass", "coal", "imports", "gas", "nuclear", "other", "hydro", "solar", "wind")

genmix_api <- function(fromdate, todate){
fromdate <- as.Date(fromdate); todate <- as.Date(todate);

datelist <- seq.Date(from = fromdate, to = todate, by = "1 day")
gen_mix <- data.frame(matrix(nrow = 0, ncol = 4))
colnames(gen_mix) <- c("data.from", "data.to", "data.generationmix.fuel", "data.generationmix.perc")

for (i in 1:length(datelist)){
if(fromdate < as.Date("2017-09-12")) {cat("\n\n", "Please select a from-date of 2017-09-12 or later", "\n\n"); break}
df_json <- RCurl::basicTextGatherer()
df_json$reset() # For caution
paste0("https://api.carbonintensity.org.uk/generation/", datelist[i], "T00:00Z/", datelist[i], "T23:59Z") %>%
RCurl::curlPerform(url = .,  writefunction = df_json$update)
data_response <- df_json$value() %>%  jsonlite::parse_json(simplifyVector = TRUE) %>% as.data.frame()
data_response$info <- NULL

gen_mix  <- rbind(gen_mix, data_response)
} # close the loop

data.from <- gen_mix$data.from %>% date_fixer 
data.to <- gen_mix$data.to %>% date_fixer 
gm_percentages <- gen_mix[[3]] %>% as.data.frame() %>% t() %>% set_colnames(gm_names)  %>% as.data.frame() %>% filter(biomass != "biomass")

gen_mix <- data.frame(data.from, data.to, gm_percentages) 

return(gen_mix)
} # close the function

. . . and here is the function in use. Note that the values represent percentages - you may wish to divide by 100!

 # Testing the function
gm_data <- genmix_api(fromdate =  as.Date("2018-10-01"), todate = as.Date("2018-10-07")) 
gm_data %>% head() %>% kable(caption = "Generation Mix")

Et Viola.