HMC R Bootcamp 2018

HMC R Bootcamp 2018

DAY 1

Some (i.e., a lot of) materials are adapted from the following websites:

• R Bootcamp by Jared Knowles: http://jaredknowles.com/r-bootcamp/
  
• R Bootcamp at Berkely: http://statistics.berkeley.edu/computing/r-bootcamp/

(Do not click on the links–it will open a new )

• Slides at http: http://rpubs.com/jepark/hmc-r-bootcamp-2016-day1
• Files at http://bit.ly/hmc-r-bootcamp-file
• Data files at at http://bit.ly/hmc-r-bootcamp-file

• about R's programming aspects.
  • It's designed to help you start using and coding R.
• not about Statistics.
  • I assume that you already have a basic knowledge of Statistics
• for your summer research and beyond.
  • This bootcamp materials are used to help you get up to speed on using R as quickly as possible for your research activites

• Module 0: Introduction
  • About this bootcamp
  • Learning Objectives
  • About R
  • Getting ready (R environment setup)
• Module 1: The Basics of R
  • General Stuff
  • RStudio
  • R Data Structure (R Objects)
  • Basic graphics

• Module 2: Working with Data
  • (Almost) clean data (CSV files)
  • Data import and export (interactively)
  • Data cleaning
  • Subsetting
  • Matrix vs Data Frame vs Data Table
  • Attributes and missing values
  • Graphics (lattice and ggplot2)
  • Simple graph manipulations

• Goals and learning objectives
• What is R?
• What is not R?
• Then Why R?
• Getting ready for R Bootcamp

To prepare YOU to use R to handle and analyze sample data of your choice.

By the end of this two-day bootcamp, you will be able to:

• Import data from a simple CSV file in R
• Distinguish the differences between string variable and other R data types
• Convert different data types back and forth
• Create a loop using R’s for loop syntax
• Create an if-else statement in R
• Interpolate data points and use interpolating functions given
• Restate the existing R script and explain what each block of the script is for
• Create basic plots and scale x and y axis according to data points
• Use subsetting methods

• R is a statical programming language/environment.
• R is open source and free.
• R is widely used.
• R is cross-platform.
• R is hard to learn (really?).

• S: R's ancestor
• S-Plus: Commercial; modern implementation of S
• SAS: Commercial; widely used in the commercial analytics.
• STATA: Commercial; widely used by economists.
• SPSS: Commercial; easy to use; widely used in Social Science.
• MATLAB: Commercial; can do some Stats.
• Python: Also can do some Stats; one of the two mostly used programming languages in data science.

• R community is active and constantly growing
• R is one of the most popular stat programming lang
• R has tons of user generated libraries/packages
• R code is easily shared with others
• R is constantly improved

• R community is active and constantly growing
• R is one of the most popular stat programming lang
• R has tons of user generated libraries/packages
• R code is easily shared with others
• R is constantly improved

Further reading about R's popularity in science and engineering:

R moves up to 5th place in IEEE language rankings at https://www.r-bloggers.com/r-moves-up-to-5th-place-in-ieee-language-rankings/

https://spectrum.ieee.org/static/interactive-the-top-programming-languages-2017 (6th in 2017)

IEEE Spectrum: Popular Programming Languages in 2017

• R community is active and constantly growing
• R is one of the most popular stat programming lang)
• R has tons of user generated libraries/packages
• R code is easily shared with others
• R is constantly improved

On the Internet:

• Google! (Now Google knows “r” means the R programming language)
• Stack Overflow: http://stackoverflow.com/questions/tagged/r
• Cross-Validated: the statistics Q&A site http://stats.stackexchange.com/
• Rseek meta search engine: http://rseek.org/
• R-help listserv: https://www.r-project.org/mail.html

Local Resources:

• Contact CIS: helpdesk@hmc.edu or jepark@hmc.edu
• HMC R Users Group: r-users-l@g.hmc.edu

• R scripting environment
  • RStudio and its functionality
• General stuff
• Workspace of R and some calculations
• R Objects (Vectors, Matrices, Lists, Data frames, and Factors)
• Converting between different types of objects
• Getting help online and offline

• Integrated Development Environment for R
• Nice combination of GUI and CLI
• 4 main windows, tabs, etc
• Version control: Git and VPN
• R Markdown
• R Presentation

• Let's open RStudio

Take a look at RStudio panes and tabs

• RStudio project helps you to keep all the files associated with a project/work in one directory along with environment, history, version control (git) files.

• Let's create a new RStudio project by

  File >> New Project…

• Copy and paste the R command file and data files into the project directory. (RStudio limitation)

  • Download files from https://github.com/jehopark/r-workshop-for-beginners-sp2018

• Check R
  
• Check RStudio
• Check RStudio project
• Check slides
• Check R command script

demo() # display available demos

demo(graphics) # try graphics demo

library() # show available packages on the computer

search() # show loaded packages

?hist # search for the usage of hist function

??histogram # search for package documents containing the word "histogram"

R workspace stores objects like vecors, datasets and functions in memory (the available space for calculation is limited to the size of your computer's RAM).

a <- 5 # notice a in your Environment window

A <- "text" 

a
A
ls()
print(c(a,A))
print(a, A)

1+1

2+runif(1,0,1)

2+runif(1,min=0,max=1)

3^2

3*3

sqrt(3*3) # comments

# comments are preceded by hash sign
#ignorehashtag!

• R can take integrals and derivatives, for example:

Numerical Integral of

\( \displaystyle\int_0^{\infty} \frac{1}{\sqrt{x}(x+1)}dx \)

# define the integrated function
integrand <- function(x) {1/(sqrt(x)*(x+1))} 

# integrate the function from 0 to infinity
integrate(integrand, lower=0, upper=Inf) 

3.141593 with absolute error < 2.7e-05

The most basic form of an R object.
A scalar value is a vector of length one.
A vector is an array object of the same type (homogeneous) data elements.

class(a)
class(A)
B <- c(a,A) # concatenation function
B # see the values
class(B) # why?

a <- rnorm(10)
a[3:5] <- NA # NA is a missing value
a

R has five basic or “atomic” classes of objects:

• character
• numeric (real numbers)
• integer
• complex
• logical (True/False)

A vector contains a set of data in any one of the atomic classes.

A matrix is a two-dimensional rectangular object of the same type (homogeneous) data elements.

mat <- matrix(rnorm(6), nrow = 3, ncol = 2) 
mat # a matrix
dim(mat) # dimension
t(mat) 
summary(mat) 

A list is an object that can store different types of vectors.

list1 <- list(name=c("Joseph"), married=T, kids=2)
list1
list1$kids <- list1$kids+1
list1$kids
list2 <- list(numeric_data=a,character_data=A)
list2
all_list <- list(list1, list2)
all_list

A data frame is a list of vectors of equal length with possibly different types. It is used for storing rectangular data tables.

n <- c(2, 3, 5) # a vector 
s <- c("aa", "bb", "cc") # a vector
b <- c(TRUE, FALSE, TRUE) # a vector
df <- data.frame(n, s, b) # a data frame
df
class(df$s) # was a string vector but now a factor. 
mtcars # a built-in (attached) data frame
mtcars$mpg

my_df <- data.frame(y1=rnorm(100),y2=rnorm(100), y3=rnorm(100))
head(my_df) # display first few lines of data
names(my_df) # display column names
summary(my_df) # output depends on the data types
plot(my_df)

• Factors are a special compoud object used to represent categorical data such as gender, social class, etc.
• Factors have 'levels' attribute. They may be nominal or ordered.

v <- c("a","b","c","c","b")
x <- factor(v) # turn the character vector into a factor object
z <- factor(v, ordered = TRUE) # ordered factor
x
z
table(x)

Use of the as() family of functions. Type as. and wait to see the list of as() functions.

integers <- 1:10
as.character(integers)
as.numeric(c('3.7', '4.8'))

indices <- c(1.7, 2.3)
integers[indices] # sometimes R is too generous
integers[0.999999999] # close to 1 but...

df <- as.data.frame(mat)
df

• Rseek meta search engine: http://rseek.org/
• R-help listserv: https://www.r-project.org/mail.html
• Stack Overflow: http://stackoverflow.com/questions/tagged/r
• Cross-Validated: the statistics Q&A site http://stats.stackexchange.com/
• Contact CIS: helpdesk@hmc.edu or jepark@hmc.edu
• HMC R Users Group: r-users-l@g.hmc.edu

Basics
1) Using cars datasets, create a variable 'stop_mean' that contains mean stopping distance.
2) Create a matrix 'cars_mat' by converting the cars data frame.

Challenges
3) Install 'hflights' datasets.
4) Using hflights datasets, create a variable called 'x' that contains the mean flight arrival delay. (Hint: NA's)
5) Create a boolean (TRUE/FALSE) vector indicating whether the departure delay is shorter than the arrival delay.

Take a break!

1)

2)

3)

4)

5)

• Working with (almost) clean data
• Data import and export (interactive way)
• Subsetting
• Using data frames vs. matrices

• Use read.table() to read text files into R
• Try help document for read.table()

• read.csv() is a special case of read.table()
• Data import from your local folder

cpds <- read.csv(file.path('.', 'data', 'cpds.csv'))
head(cpds) # good to look at a few lines
class(cpds) # data.frame

• Data import from the Internet

data <- read.table(file="http://scicomp.hmc.edu/data/R/normtemp.txt", header=T)
tail(data)

rta <- read.table("./data/RTADataSub.csv", sep = ",", head = TRUE)
dim(rta)
rta[1:5, 1:5]
class(rta)
class(rta$time) # what? let's see ?read.table more carefully

rta2 <- read.table("./data/RTADataSub.csv", sep = ",", head = TRUE, stringsAsFactors = FALSE)
class(rta2$time)

• Use write.table() to write data to a CSV file

write.csv(data, file = "temp.csv", row.names = FALSE) 

• Writing out plots

pdf('myplot.pdf', width = 7, height = 7) # call pdf() before calling plot()
x <- rnorm(10); y <- rnorm(10)
plot(x, y)
dev.off()

Operators that can be used to extract subsets of R objects.

• '[' and ']' always returns an object of the same class as the original; can be used to select more than one element.
• '[[' and ']]' is used to extract elements of a list or a data frame; it can only be used to extract a single element.
• $ is used to extract elements of a list or data frame by name.

x <- c("a", "b", "c", "c", "d", "a")
x[1]
x[1:4]
x[x > "a"] 
u <- x > "a" # what's u here?
u
x[u] # subsetting using a boolean vector
y <- list(foo=x, bar=x[u]) 
y
y[[1]]
y$bar

five_gears <- subset(mtcars, gear == 5) # use of subset function for data frames

Consider the following:

• Same types or different types? Numeric or other type?
• Convenient using $ with col names?
• Data size too big? (memory efficiency and size)

m = matrix(1:400000, 2, 200000) # esp. for a large number of columns!
d = as.data.frame(m)
object.size(m) # 1600200 bytes
object.size(d) # 22400568 bytes

• Conversion between data frame and matrix
  • as.data.frame()
  • as.matrix() or data.matrix() # consider coercion

Basics
1) Try installing a new library (lmtest) from Console
2) Try installing another library (ggplot2) from the Packages tab

Challenges
3) From the dataset, hflights, create a subsample based on delay times greater than 10 hours (600 min) from ArrDelay and name it, hflightsSub.
4) Set all of the extreme delays in ArrDelay (more than 300 minutes) in hflightsSub to NA.
5) Convert the data frame, d, back to a matrix named m1 and compare the size of m and m1

1)

2)

3)

4)

5)

1. Install git on your computer (if it's not already installed)
2. Create a Github account (if you don't have it yet)
3. Create a test repository, hmc-r-bootcamp-2018, from GitHub
4. Create a R project with version control and use the hmc-r-bootcamp-2018 repository's URL to connect your R project with the GitHub repository
5. Create a new R markdown file (HMC-R-Bootcamp.Rmd) and try “push” to the repository
6. Change the “HMC-R-Bootcamp.Rmd” from GitHub
7. Try “pull” the Rmd file to your local repository
8. Digest what you've learned from Day 1