Manipulating Time Series Data in R with xts and zoo (DataCamp)

Ch. 1 - Introduction to eXtensible Time Series, using xts and zoo for time series

Introducing xts and zoo objects

[Video]

What is xts?

eXtensible Time Series
An extended zoo object
Matrix + Index
Observations + Times

An xts example

# XTS = MATRIX + INDEX

x <- matrix(1:4, ncol = 2, nrow = 2)
x

##      [,1] [,2]
## [1,]    1    3
## [2,]    2    4

idx <- as.Date(c("2015-01-01", "2015-02-01"))
idx

## [1] "2015-01-01" "2015-02-01"

xts allows you to use any timestamp, i.e., Date, POSIX times, timeDate, chron

# XTS = MATRIX + INDEX
X <- xts(x, order.by = idx)
X

##            [,1] [,2]
## 2015-01-01    1    3
## 2015-02-01    2    4

The xts constructor

?xts

Special xts behavior

xts is a matrix with associated times for each observation
Subsets preserve matrix form
Attributes are preserved ** i.e. a time-stamp that was acquired
xts is a subclass of zoo

Deconstructing xts

User interal components
coredata(x) is used to extract the data component

# coredata(x, fmt = FALSE)

index(x) to extract the index a.k.a times

# index(x)

What is an xts object?

There are many different types of objects in R. With a variety of different features, each has a unique purpose. Some classes inherit behavior from their parents, allowing for custom extensions to existing and well-understood R objects. This makes it easy to adapt existing code to new functionality.

It is for this reason that xts extends the popular zoo class. Knowing this, and what you have seen so far, which of these statements is true?

xts objects are matrix objects internally.
xts objects are indexed by a formal time object.
Most zoo methods work for xts.
[*] All of the above.

More than a matrix

# Load xts
library(xts)

# View the structure of ex_matrix
str(ex_matrix)

## An 'xts' object on 2016-06-01/2016-06-03 containing:
##   Data: num [1:3, 1:2] 1 1 1 2 2 2
##   Indexed by objects of class: [Date] TZ: UTC
##   xts Attributes:  
##  NULL

# Extract the 3rd observation of the 2nd column of ex_matrix
ex_matrix[3, 2]

##            [,1]
## 2016-06-03    2

# Extract the 3rd observation of the 2nd column of core 
core[3, 2]

## [1] 2

Your first xts object

# Create the object data using 5 random numbers
data <- rnorm(5)

# Create dates as a Date class object starting from 2016-01-01
dates <- seq(as.Date("2016-01-01"), length = 5, by = "days")

# Use xts() to create smith
smith <- xts(x = data, order.by = dates)

# Create bday (1899-05-08) using a POSIXct date class object
bday <- as.POSIXct("1899-05-08")

# Create hayek and add a new attribute called born
hayek <- xts(x = data, order.by = dates, born = bday)

Deconstructing xts

# Extract the core data of hayek
hayek_core <- coredata(hayek, fmt = FALSE)

# View the class of hayek_core
class(hayek_core)

## [1] "matrix"

# Extract the index of hayek
hayek_index <- index(hayek)

# View the class of hayek_index
class(hayek_index)

## [1] "Date"

Time based indices

# Create dates
dates <- as.Date("2016-01-01") + 0:4

# Create ts_a
ts_a <- xts(x = c(1:5), order.by = dates)

# Create ts_b
ts_b <- xts(x = c(1:5), order.by = as.POSIXct(dates))

# Extract the rows of ts_a using the index of ts_b
ts_a[index(ts_b)]

##            [,1]
## 2016-01-01    1
## 2016-01-02    2
## 2016-01-03    3
## 2016-01-04    4
## 2016-01-05    5

# Extract the rows of ts_b using the index of ts_a
ts_b[index(ts_a)]

##      [,1]

Importing, exporting and converting time series

[Video]

Reality check

Data usually already exists, and needs wrangling ** Often data isn’t in your preferred class
Data needs to be imported into R and converted to xts
You will need to convert, read and export xts objects

Converting using as.xts()

# Load data from R datasets
data(sunspots)
class(sunspots)

## [1] "ts"

sunspots_xts <- as.xts(sunspots)
class(sunspots_xts)

## [1] "xts" "zoo"

head(sunspots_xts)

##          [,1]
## Jan 1749 58.0
## Feb 1749 62.6
## Mar 1749 70.0
## Apr 1749 55.7
## May 1749 85.0
## Jun 1749 83.5

Importing external data to xts

Read data into R using built in (or external) functions ** i.e., read.table(), read.csv(), and read.zoo()
Coerce data to xts using

# as.xts(read.table("file"))
# as.xts(read.zoo("file"))

Exporting xts from R

Sometimes you will need your data outside of R
Use write.zoo() for external use (i.e. text files)

# write.zoo(x, "file")

Use saveRDS for R use

# saveRDS(x, "file")

Converting xts objects

# Convert austres to an xts object called au
au <- as.xts(austres)

# Then convert your xts object (au) into a matrix am
am <- as.matrix(au)

# Inspect the head of am
head(am)

##              au
## 1971 Q2 13067.3
## 1971 Q3 13130.5
## 1971 Q4 13198.4
## 1972 Q1 13254.2
## 1972 Q2 13303.7
## 1972 Q3 13353.9

# Convert the original austres into a matrix am2
am2 <- as.matrix(austres)

# Inspect the head of am2
head(am2)

##         [,1]
## [1,] 13067.3
## [2,] 13130.5
## [3,] 13198.4
## [4,] 13254.2
## [5,] 13303.7
## [6,] 13353.9

Importing data

# Create dat by reading tmp_file
dat <- read.csv(tmp_file)

# Convert dat into xts
xts(dat, order.by = as.Date(rownames(dat), "%m/%d/%Y"))

##            a b
## 2015-01-02 1 3
## 2015-02-03 2 4

# Read tmp_file using read.zoo
dat_zoo <- read.zoo(tmp_file, index.column = 0, sep = ",", format = "%m/%d/%Y")

# Convert dat_zoo to xts
dat_xts <- xts(dat_zoo)

Exporting xts objects

# Convert sunspots to xts using as.xts().
sunspots_xts <- as.xts(sunspots)

# Get the temporary file name
tmp <- tempfile()

# Write the xts object using zoo to tmp 
write.zoo(sunspots_xts, sep = ",", file = tmp)

# Read the tmp file. FUN = as.yearmon converts strings such as Jan 1749 into a proper time class
sun <- read.zoo(tmp, sep = ",", FUN = as.yearmon)

# Convert sun into xts. Save this as sun_xts
sun_xts <- as.xts(sun)

Manipulating Time Series Data in R with xts and zoo (DataCamp)

Ch. 1 - Introduction to eXtensible Time Series, using xts and zoo for time series

Introducing xts and zoo objects

What is an xts object?

More than a matrix

Your first xts object

Deconstructing xts

Time based indices

Importing, exporting and converting time series

Converting xts objects

Importing data

Exporting xts objects

Ch. 2 - First Order of Business - Basic Manipulations

Introducing time based queries

The ISO-8601 standard

Querying for dates

Extracting recurring intraday intervals

Alternative extraction techniques

Row selection with time objects

Update and replace elements

Methods to find periods in your data

Find the first or last period of time

Combining first and last

Math operations using xts

Matrix arithmetic - add, subtract, multiply, and divide in time!

Math with non-overlapping indexes

Ch. 3 - Merging and modifying time series

Merging time series

Combining xts by column with merge

Combining xts by row with rbind

What types of data can be combined using merge?

Handling missingness

Fill missing values using last or previous observation

NA interpolation using na.approx()

Lags and differences

Combine a leading and lagging time series

Calculate a difference of a series using diff()

What is the key difference in lag between xts and zoo

Ch. 4 - Apply and aggregate by time

Apply functions by time

Find intervals by time in xts

Apply a function by time period(s)

Using lapply() and split() to apply functions on intervals

Selection by endpoints vs. split-lapply-rbind

Converting periodicity

Convert univariate series to OHLC data

Convert a series to a lower frequency

Rolling functions

Calculate basic rolling value of series by month

Calculate the rolling standard deviation of a time series

Ch. 5 - Extra features of xts

Index, attributes, and time zones

Time via index()

Class attributes - tclass, tzone, and tformat

Time Zones (and why you should care!)

Periods, periodicity and timestamps

Determining periodicity

Find the number of periods in your data

Secret index tools

Modifying timestamps

Congratulations!

About Michael Mallari