Week 3 R Character Manipulation and Date Processing

Introduction

The purpose of this assignment is to work with data manipulation involving strings and by using regex and testing the regular expressions by programming in R.

Import Libraries

library(tidyverse)
library(openintro)

Part 1

1. Using the 173 majors listed in fivethirtyeight.com’s College Majors dataset [https://fivethirtyeight.com/features/the-economic-guide-to-picking-a-college-major/], provide code that identifies the majors that contain either “DATA” or “STATISTICS”

url <- "https://raw.githubusercontent.com/pujaroy280/DATA607Week3/main/all-ages.csv"
college_majors_data <- read.csv(url) # Data has 173 observations (rows) of 11 variables (columns)

head(college_majors_data)

##   Major_code                                 Major
## 1       1100                   GENERAL AGRICULTURE
## 2       1101 AGRICULTURE PRODUCTION AND MANAGEMENT
## 3       1102                AGRICULTURAL ECONOMICS
## 4       1103                       ANIMAL SCIENCES
## 5       1104                          FOOD SCIENCE
## 6       1105            PLANT SCIENCE AND AGRONOMY
##                    Major_category  Total Employed Employed_full_time_year_round
## 1 Agriculture & Natural Resources 128148    90245                         74078
## 2 Agriculture & Natural Resources  95326    76865                         64240
## 3 Agriculture & Natural Resources  33955    26321                         22810
## 4 Agriculture & Natural Resources 103549    81177                         64937
## 5 Agriculture & Natural Resources  24280    17281                         12722
## 6 Agriculture & Natural Resources  79409    63043                         51077
##   Unemployed Unemployment_rate Median P25th P75th
## 1       2423        0.02614711  50000 34000 80000
## 2       2266        0.02863606  54000 36000 80000
## 3        821        0.03024832  63000 40000 98000
## 4       3619        0.04267890  46000 30000 72000
## 5        894        0.04918845  62000 38500 90000
## 6       2070        0.03179089  50000 35000 75000

# The dataset loaded or read into a variable is called 'college_majors_data'

# Filter the dataset to include majors containing either "DATA" or "STATISTICS"
majors_with_data_or_stats <- college_majors_data[grep(pattern = "DATA|STATISTICS", college_majors_data$Major, ignore.case = TRUE), ]

# grep() function is used to search for patterns "DATA" or "STATISTICS" in the Major column of the college_majors dataset. The ignore.case = TRUE argument ensures that the search is case-insensitive.

# Display the filtered majors
print(majors_with_data_or_stats)

##     Major_code                                         Major
## 20        2101      COMPUTER PROGRAMMING AND DATA PROCESSING
## 93        3702               STATISTICS AND DECISION SCIENCE
## 170       6212 MANAGEMENT INFORMATION SYSTEMS AND STATISTICS
##              Major_category  Total Employed Employed_full_time_year_round
## 20  Computers & Mathematics  29317    22828                         18747
## 93  Computers & Mathematics  24806    18808                         14468
## 170                Business 156673   134478                        118249
##     Unemployed Unemployment_rate Median P25th  P75th
## 20        2265        0.09026422  60000 40000  85000
## 93        1138        0.05705405  70000 43000 102000
## 170       6186        0.04397714  72000 50000 100000

majors_with_data_or_stats$Major

## [1] "COMPUTER PROGRAMMING AND DATA PROCESSING"     
## [2] "STATISTICS AND DECISION SCIENCE"              
## [3] "MANAGEMENT INFORMATION SYSTEMS AND STATISTICS"

Only 3 majors appeared and their major category included “Computers & Mathematics” and “Business”.

Part 2

2. Write code that transforms the data below: [1] “bell pepper” “bilberry” “blackberry” “blood orange” [5] “blueberry” “cantaloupe” “chili pepper” “cloudberry”
   [9] “elderberry” “lime” “lychee” “mulberry”
   [13] “olive” “salal berry” Into a format like this: c(“bell pepper”, “bilberry”, “blackberry”, “blood orange”, “blueberry”, “cantaloupe”, “chili pepper”, “cloudberry”, “elderberry”, “lime”, “lychee”, “mulberry”, “olive”, “salal berry”)

fruits <- c("bell pepper", "bilberry", "blackberry", "blood orange",
                   "blueberry", "cantaloupe", "chili pepper", "cloudberry",
                   "elderberry", "lime", "lychee", "mulberry", "olive",
                   "salal berry")
fruits

##  [1] "bell pepper"  "bilberry"     "blackberry"   "blood orange" "blueberry"   
##  [6] "cantaloupe"   "chili pepper" "cloudberry"   "elderberry"   "lime"        
## [11] "lychee"       "mulberry"     "olive"        "salal berry"

# Transform the data
# str_c is a function from the stringr package of R that concatenates strings.
fruits_transform <- str_c("c(", str_c('"', fruits, '"', collapse = ", "), ")")
str_view(fruits_transform)

## [1] │ c("bell pepper", "bilberry", "blackberry", "blood orange", "blueberry", "cantaloupe", "chili pepper", "cloudberry", "elderberry", "lime", "lychee", "mulberry", "olive", "salal berry")

Part 3

The two exercises below are taken from R for Data Science, 14.3.5.1 in the on-line version: 3) Describe, in words, what these expressions will match: (.)\1\1 “(.)(.)\2\1” (..)\1 “(.).\1.\1” “(.)(.)(.).*\3\2\1”

expression1 <- "(.)\\1\\1"
str_view(expression1)

## [1] │ (.)\1\1

str_view(c(words,"888", "$$$", "ppp"), expression1)

## [981] │ <888>
## [982] │ <$$$>
## [983] │ <ppp>

(.)\1\1 matches any sequence of three characters where the first character is the same as the second and third characters.

expression2 <- "(.)(.)\\2\\1"
str_view(expression2)

## [1] │ (.)(.)\2\1

str_view(c(words), expression2)

##  [19] │ after<noon>
##  [43] │ <appa>rent
##  [53] │ <arra>nge
## [107] │ b<otto>m
## [112] │ br<illi>ant
## [174] │ c<ommo>n
## [230] │ d<iffi>cult
## [259] │ <effe>ct
## [329] │ f<ollo>w
## [422] │ in<deed>
## [470] │ l<ette>r
## [521] │ m<illi>on
## [581] │ <oppo>rtunity
## [582] │ <oppo>se
## [877] │ tom<orro>w

The regular expression: “(.)(.)\2\1” matches a string length of four in which the first+fourth and second+third characters are the same.

expression3 <- "(..)\\1" # naming expression
str_view(expression3) # string view of expression

## [1] │ (..)\1

str_view(c(words, fruit), expression3) # testing matches

##  [696] │ r<emem>ber
##  [984] │ b<anan>a
## [1000] │ <coco>nut
## [1002] │ <cucu>mber
## [1021] │ <juju>be
## [1036] │ <papa>ya
## [1053] │ s<alal> berry

“(..)\1” is a regular expression that matches any string that has 2 identical characters.

expression4 <- "(.).\\1.\\1"
str_view(expression4) 

## [1] │ (.).\1.\1

str_view(c(words, fruit), expression4)

##  [265] │ <eleve>n
##  [984] │ b<anana>
## [1036] │ p<apaya>

The regular expression: “(.).\1.\1” matches a string length of five in which the 3 characters including the first, third and fifth are the same.

expression5 <- "(.)(.)(.).*\\3\\2\\1"
str_view(expression5) 

## [1] │ (.)(.)(.).*\3\2\1

str_view(c(words, fruit, "lalamimilala", "12345fedcba54321"), expression5)

##  [598] │ <paragrap>h
## [1061] │ <lalamimilal>a
## [1062] │ <12345fedcba54321>

The regular expression: “(.)(.)(.).*\3\2\1” matches any string length that starts with three characters that consist of any characters. This expression matches the last three characters of the string in reverse order of the original string.

Part 4

4. Construct regular expressions to match words that: Start and end with the same character. Contain a repeated pair of letters (e.g. “church” contains “ch” repeated twice.) Contain one letter repeated in at least three places (e.g. “eleven” contains three “e”s.)

expression6 <- "^(.).*\\1$" 
str_view(expression6)

## [1] │ ^(.).*\1$

str_view(c(words), expression6)

##  [36] │ <america>
##  [49] │ <area>
## [209] │ <dad>
## [213] │ <dead>
## [223] │ <depend>
## [258] │ <educate>
## [266] │ <else>
## [268] │ <encourage>
## [270] │ <engine>
## [278] │ <europe>
## [283] │ <evidence>
## [285] │ <example>
## [287] │ <excuse>
## [288] │ <exercise>
## [291] │ <expense>
## [292] │ <experience>
## [296] │ <eye>
## [386] │ <health>
## [394] │ <high>
## [450] │ <knock>
## ... and 16 more

expression7 <- "(..).*\\1"
str_view(expression7)

## [1] │ (..).*\1

str_view(c(words), expression7) 

##  [48] │ ap<propr>iate
## [152] │ <church>
## [181] │ c<ondition>
## [217] │ <decide>
## [275] │ <environmen>t
## [487] │ l<ondon>
## [598] │ pa<ragra>ph
## [603] │ p<articular>
## [617] │ <photograph>
## [638] │ p<repare>
## [641] │ p<ressure>
## [696] │ r<emem>ber
## [698] │ <repre>sent
## [699] │ <require>
## [739] │ <sense>
## [858] │ the<refore>
## [903] │ u<nderstand>
## [946] │ w<hethe>r

expression8 <- "(.).*\\1.*\\1" 
str_view(expression8) 

## [1] │ (.).*\1.*\1

str_view(c(words), expression8)

##  [48] │ a<pprop>riate
##  [62] │ <availa>ble
##  [86] │ b<elieve>
##  [90] │ b<etwee>n
## [119] │ bu<siness>
## [221] │ d<egree>
## [229] │ diff<erence>
## [233] │ di<scuss>
## [265] │ <eleve>n
## [275] │ e<nvironmen>t
## [283] │ <evidence>
## [288] │ <exercise>
## [291] │ <expense>
## [292] │ <experience>
## [423] │ <indivi>dual
## [598] │ p<aragra>ph
## [684] │ r<eceive>
## [696] │ r<emembe>r
## [698] │ r<eprese>nt
## [845] │ t<elephone>
## ... and 2 more

Conclusion

To sum it all up, I learned how to filter and work with regular expressions using Regex and R. It can often be confusing at the beginning due to the syntax, yet working on this assignment was rewarding since it provided me the knowledge of identifying patterns in strings.