R语言进阶5: Utilities
Chris Qi
9/4/2018
Mathematical utilities
Have another look at some useful math functions that R features:
abs(): Calculate the absolute value. sum(): Calculate the sum of all the values in a data structure. mean(): Calculate the arithmetic mean. round(): Round the values to 0 decimal places by default. Try out ?round in the console for variations of round() and ways to change the number of digits to round to. As a data scientst in training, you’ve estimated a regression model on the sales data for the past six months. After evaluating your model, you see that the training error of your model is quite regular, showing both positive and negative values. The error values are already defined in the workspace on the right (errors).
INSTRUCTIONS
Calculate the sum of the absolute rounded values of the training errors. You can work in parts, or with a single one-liner. There’s no need to store the result in a variable, just have R print it.
HINT To know the order of operations, you should read the sentence in the instructions backwards: put round() inside abs(), and put the result of abs() in sum().Find the error
We went ahead and included some code on the right, but there’s still an error. Can you trace it and fix it?
In times of despair, help with functions such as sum() and rev() are a single command away; simply use ?sum and ?rev in the console.
INSTRUCTIONS
Fix the error by including code on the last line. Remember: you want to call mean() only once!
HINT mean() takes a vector of numerical values, so make sure that abs(vec1) and abs(vec2) are elements of a vector: c(abs(vec1), ___).
If you check out the documentation of mean(), you’ll see that only the first argument, x, should be a vector. If you also specify a second argument, R will match the arguments by position and expect a specification of the trim argument. Therefore, merging the two vectors is a must!
Data Utilities
R features a bunch of functions to juggle around with data structures::
seq(): Generate sequences, by specifying the from, to, and by arguments. rep(): Replicate elements of vectors and lists. sort(): Sort a vector in ascending order. Works on numerics, but also on character strings and logicals. rev(): Reverse the elements in a data structures for which reversal is defined. str(): Display the structure of any R object. append(): Merge vectors or lists. is.(): Check for the class of an R object. as.(): Convert an R object from one class to another. unlist(): Flatten (possibly embedded) lists to produce a vector. Remember the social media profile views data? Your LinkedIn and Facebook view counts for the last seven days are already defined as lists on the right.
INSTRUCTIONS
Convert both linkedin and facebook lists to a vector, and store them as li_vec and fb_vec respectively. Next, append fb_vec to the li_vec (Facebook data comes last). Save the result as social_vec. Finally, sort social_vec from high to low. Print the resulting vector.
HINT You can use unlist() to convert lists to vectors. For the last instruction, make sure to use the decreasing argument!These instructions required you to solve this challenge in a step-by-step approach. If you’re comfortable with the functions, you can combine some of these steps into powerful one-liners.
Find the error (2) Just as before, let’s switch roles. It’s up to you to see what unforgivable mistakes we’ve made. Go fix them!
INSTRUCTIONS
Correct the expression. Make sure that your fix still uses the functions rep() and seq().
HINT The use of seq() and rep() are mixed up.Debugging code is also a big part of the daily routine of a data scientist, and you seem to be great at it!
Beat Gauss using R
There is a popular story about young Gauss. As a pupil, he had a lazy teacher who wanted to keep the classroom busy by having them add up the numbers 1 to 100. Gauss came up with an answer almost instantaneously, 5050. On the spot, he had developed a formula for calculating the sum of an arithmetic series. There are more general formulas for calculating the sum of an arithmetic series with different starting values and increments. Instead of deriving such a formula, why not use R to calculate the sum of a sequence?
INSTRUCTIONS Using the function seq(), create a sequence that ranges from 1 to 500 in increments of 3. Assign the resulting vector to a variable seq1. Again with the function seq(), create a sequence that ranges from 1200 to 900 in increments of -7. Assign it to a variable seq2. Calculate the total sum of the sequences, either by using the sum() function twice and adding the two results, or by first concatenating the sequences and then using the sum() function once. Print the result to the console.
HINT For the first seq() call, set the from, to and by arguments to 1, 500 and 3 respectively. sum(seq1) + sum(seq2) will do the trick once you’ve correctly created seq1 and seq2.
grepl & grep
In their most basic form, regular expressions can be used to see whether a pattern exists inside a character string or a vector of character strings. For this purpose, you can use:
grepl(), which returns TRUE when a pattern is found in the corresponding character string. grep(), which returns a vector of indices of the character strings that contains the pattern. Both functions need a pattern and an x argument, where pattern is the regular expression you want to match for, and the x argument is the character vector from which matches should be sought.
In this and the following exercises, you’ll be querying and manipulating a character vector of email addresses! The vector emails has already been defined in the editor on the right so you can begin with the instructions straight away!
INSTRUCTIONS
Use grepl() to generate a vector of logicals that indicates whether these email addressess contain “edu”. Print the result to the output. Do the same thing with grep(), but this time save the resulting indexes in a variable hits. Use the variable hits to select from the emails vector only the emails that contain “edu”.
HINT The first argument you should pass to both grepl() and grep(), is the pattern: “edu”. The second argument is emails, in both cases. Once you’ve correctly defined hits, you can use it to subset emails: emails[hits].
You can probably guess what we’re trying to achieve here: select all the emails that end with “.edu”. However, the strings education@world.gov and invalid.edu were also matched. Let’s see in the next exercise what you can do to improve our pattern and remove these false positives.
grepl & grep (2) You can use the caret, ^, and the dollar sign, $ to match the content located in the start and end of a string, respectively. This could take us one step closer to a correct pattern for matching only the “.edu” email addresses from our list of emails. But there’s more that can be added to make the pattern more robust:
@, because a valid email must contain an at-sign. ., which matches any character (.) zero or more times (). Both the dot and the asterisk are metacharacters. You can use them to match any character between the at-sign and the “.edu” portion of an email address. \.edu$, to match the “.edu” part of the email at the end of the string. The \ part escapes the dot: it tells R that you want to use the . as an actual character. INSTRUCTIONS 70 XP Use grepl() with the more advanced regular expression to return a logical vector. Simply print the result. Do a similar thing with grep() to create a vector of indices. Store the result in the variable hits. Use emails[hits] again to subset the emails vector.
HINT Use the pattern “@.*\.edu$" inside grepl() and grep().
A careful construction of our regular expression leads to more meaningful matches. However, even our robust email selector will often match some incorrect email addresses (for instance kiara@@fakemail.edu). Let’s not worry about this too much and continue with sub() and gsub() to actually edit the email addresses!
sub & gsub
While grep() and grepl() were used to simply check whether a regular expression could be matched with a character vector, sub() and gsub() take it one step further: you can specify a replacement argument. If inside the character vector x, the regular expression pattern is found, the matching element(s) will be replaced with replacement.sub() only replaces the first match, whereas gsub() replaces all matches.
Suppose that emails vector you’ve been working with is an excerpt of DataCamp’s email database. Why not offer the owners of the .edu email addresses a new email address on the datacamp.edu domain? This could be quite a powerful marketing stunt: Online education is taking over traditional learning institutions! Convert your email and be a part of the new generation!
INSTRUCTIONS
With the advanced regular expression “@.*\.edu$“, use sub() to replace the match with”@datacamp.edu“. Since there will only be one match per character string, gsub() is not necessary here. Inspect the resulting output.
HINT The replacement argument should be equal to “@datacamp.edu” in your call of sub().
Notice how only the valid .edu addresses are changed while the other emails remain unchanged. To get a taste of other things you can accomplish with regex, head over to the next exercise.
sub & gsub (2) Regular expressions are a typical concept that you’ll learn by doing and by seeing other examples. Before you rack your brains over the regular expression in this exercise, have a look at the new things that will be used:
.*: A usual suspect! It can be read as “any character that is matched zero or more times”. \s: Match a space. The “s” is normally a character, escaping it (\) makes it a metacharacter. [0-9]+: Match the numbers 0 to 9, at least once (+). ([0-9]+): The parentheses are used to make parts of the matching string available to define the replacement. The \1 in the replacement argument of sub() gets set to the string that is captured by the regular expression [0-9]+. awards <- c(“Won 1 Oscar.”, “Won 1 Oscar. Another 9 wins & 24 nominations.”, “1 win and 2 nominations.”, “2 wins & 3 nominations.”, “Nominated for 2 Golden Globes. 1 more win & 2 nominations.”, “4 wins & 1 nomination.”)
sub(“.\s([0-9]+)\snomination.$”, “\1”, awards) What does this code chunk return? awards is already defined in the workspace so you can start playing in the console straight away.
A vector of character strings containing “Won 1 Oscar.”, “24”, “2”, “3”, “2”, “1”. press 4 HINT Try to see how the pattern responds to every string in the awards vector. If you are out of options, simply copy the sub() expression to the console and inspect the output.
Can you explain why all of this happened? The ([0-9]+) selects the entire number that comes before the word “nomination” in the string, and the entire match gets replaced by this number because of the \1 that reference to the content inside the parentheses. The next video will get you up to speed with times and dates in R!
Right here, right now
In R, dates are represented by Date objects, while times are represented by POSIXct objects. Under the hood, however, these dates and times are simple numerical values. Date objects store the number of days since the 1st of January in 1970. POSIXct objects on the other hand, store the number of seconds since the 1st of January in 1970.
The 1st of January in 1970 is the common origin for representing times and dates in a wide range of programming languages. There is no particular reason for this; it is a simple convention. Of course, it’s also possible to create dates and times before 1970; the corresponding numerical values are simply negative in this case.
INSTRUCTIONS
Ask R for the current date, and store the result in a variable today. To see what today looks like under the hood, call unclass() on it. Ask R for the current time, and store the result in a variable, now. To see the numerical value that corresponds to now, call unclass() on it.
HINT You can use Sys.Date() and Sys.time() to get the current date and time, respectively. Beware of the difference in capitalization!Using R to get the current date and time is nice, but you should also know how to create dates and times from character strings. Find out how in the next exercises!
Create and format dates To create a Date object from a simple character string in R, you can use the as.Date() function. The character string has to obey a format that can be defined using a set of symbols (the examples correspond to 13 January, 1982):
%Y: 4-digit year (1982) %y: 2-digit year (82) %m: 2-digit month (01) %d: 2-digit day of the month (13) %A: weekday (Wednesday) %a: abbreviated weekday (Wed) %B: month (January) %b: abbreviated month (Jan) The following R commands will all create the same Date object for the 13th day in January of 1982:
as.Date(“1982-01-13”) as.Date(“Jan-13-82”, format = “%b-%d-%y”) as.Date(“13 January, 1982”, format = “%d %B, %Y”) Notice that the first line here did not need a format argument, because by default R matches your character string to the formats “%Y-%m-%d” or “%Y/%m/%d”.
In addition to creating dates, you can also convert dates to character strings that use a different date notation. For this, you use the format() function. Try the following lines of code:
today <- Sys.Date() format(Sys.Date(), format = “%d %B, %Y”) format(Sys.Date(), format = “Today is a %A!”)
INSTRUCTIONS
In the editor on the right, three character strings representing dates have been created. Convert them to dates using as.Date(), and assign them to date1, date2, and date3 respectively. The code for date1 is already included. Extract useful information from the dates as character strings using format(). From the first date, select the weekday. From the second date, select the day of the month. From the third date, you should select the abbreviated month and the 4-digit year, separated by a space.
HINT For date2, you don’t have to specify the format argument, as the string has a standard form. To convert the third string to a date, use the “%d/%B/%Y” format. To convert the third date to the correct character string, use “%b %Y”You’re a date magician! You can use POSIXct objects, i.e. Time objects in R, in a similar fashion. Give it a try in the next exercise.
Create and format times
Similar to working with dates, you can use as.POSIXct() to convert from a character string to a POSIXct object, and format() to convert from a POSIXct object to a character string. Again, you have a wide variety of symbols:
%H: hours as a decimal number (00-23) %I: hours as a decimal number (01-12) %M: minutes as a decimal number %S: seconds as a decimal number %T: shorthand notation for the typical format %H:%M:%S %p: AM/PM indicator For a full list of conversion symbols, consult the strptime documentation in the console:
?strptime Again,as.POSIXct() uses a default format to match character strings. In this case, it’s %Y-%m-%d %H:%M:%S. In this exercise, abstraction is made of different time zones.
INSTRUCTIONS:
Convert two strings that represent timestamps, str1 and str2, to POSIXct objects called time1 and time2. Using format(), create a string from time1 containing only the minutes. From time2, extract the hours and minutes as “hours:minutes AM/PM”. Refer to the assignment text above to find the correct conversion symbols!
HINT You don’t need to specify the format to build time2, as str2 comes in a standard format. To convert time1 to the correct character string, you can use the format “%M”. To convert time2 to the correct character string, you can use the format “%I:%M %p”.Calculations with Dates
Both Date and POSIXct R objects are represented by simple numerical values under the hood. This makes calculation with time and date objects very straightforward: R performs the calculations using the underlying numerical values, and then converts the result back to human-readable time information again.
You can increment and decrement Date objects, or do actual calculations with them (try it out in the console!):
today <- Sys.Date() today + 1 today - 1
as.Date(“2015-03-12”) - as.Date(“2015-02-27”) To control your eating habits, you decided to write down the dates of the last five days that you ate pizza. In the workspace, these dates are defined as five Date objects, day1 to day5. The code on the right also contains a vector pizza with these 5 Date objects.
INSTRUCTIONS
Calculate the number of days that passed between the last and the first day you ate pizza. Print the result. Use the function diff() on pizza to calculate the differences between consecutive pizza days. Store the result in a new variable day_diff. Calculate the average period between two consecutive pizza days. Print the result.
HINT For more information on the diff() function, execute ?diff in the console. Basically, this functions shifts its input vector by 1 index and then calculates the difference between this shifted vector and the original vector. In this way, it computes the difference between consecutive vector elements.
Calculations with Times
Calculations using POSIXct objects are completely analogous to those using Date objects. Try to experiment with this code to increase or decrease POSIXct objects:
now <- Sys.time() now + 3600 # add an hour now - 3600 * 24 # subtract a day Adding or substracting time objects is also straightforward:
birth <- as.POSIXct(“1879-03-14 14:37:23”) death <- as.POSIXct(“1955-04-18 03:47:12”) einstein <- death - birth einstein You’re developing a website that requires users to log in and out. You want to know what is the total and average amount of time a particular user spends on your website. This user has logged in 5 times and logged out 5 times as well. These times are gathered in the vectors login and logout, which are already defined in the workspace.
INSTRUCTIONS
Calculate the difference between the two vectors logout and login, i.e. the time the user was online in each independent session. Store the result in a variable time_online. Inspect the variable time_online by printing it. Calculate the total time that the user was online. Print the result. Calculate the average time the user was online. Print the result. Show Answer (-70 XP) HINT For the first instruction, simply type logout - login. For the third and fourth instruction, you can use sum() and mean() out of the box.
Time is of the essence
The dates when a season begins and ends can vary depending on who you ask. People in Australia will tell you that spring starts on September 1st. The Irish people in the Northern hemisphere will swear that spring starts on February 1st, with the celebration of St. Brigid’s Day. Then there’s also the difference between astronomical and meteorological seasons: while astronomers are used to equinoxes and solstices, meteorologists divide the year into 4 fixed seasons that are each three months long. (source: www.timeanddate.com)
A vector astro, which contains character strings representing the dates on which the 4 astronomical seasons start, has been defined on your workspace. Similarly, a vector meteo has already been created for you, with the meteorological beginnings of a season.
INSTRUCTIONS
Use as.Date() to convert the astro vector to a vector containing Date objects. You will need the %d, %b and %Y symbols to specify the format. Store the resulting vector as astro_dates. Use as.Date() to convert the meteo vector to a vector with Date objects. This time, you will need the %B, %d and %y symbols for the format argument. Store the resulting vector as meteo_dates. With a combination of max(), abs() and -, calculate the maximum absolute difference between the astronomical and the meteorological beginnings of a season, i.e. astro_dates and meteo_dates. Simply print this maximum difference to the console output.
HINT To convert astro to a vector of Date objects, you’ll need the following format: “%d-%b-%Y”. To convert meteo to a vector of Date objects, you’ll want to use “%B %d, %y” in the format argument. For the final instruction, use max(abs(x - y)), where you set x and y appropriately