Data Wrangling (Data Preprocessing)
Tuan Vu Nguyen - s3700604
2024-08-19
Setup
library(dplyr)##
## Attaching package: 'dplyr'## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionlibrary(magrittr)
library(kableExtra)##
## Attaching package: 'kableExtra'## The following object is masked from 'package:dplyr':
##
## group_rowsknitr::opts_chunk$set(echo = TRUE)Student names, numbers and percentage of contributions
| Student name | Student number | Percentage of contribution |
|---|---|---|
| Tuan Vu Nguyen | s3700604 | 100 |
Data Description
#the url for the online csv file:"https://vicroadsopendatastorehouse.vicroads.vic.gov.au/opendata/Road_Safety/ACCIDENT.csv"
# This data set has contained 171009 observations which is allocated in 23 variables. This data set provides an insight of how many vehicle accidents has been recorded in Victoria state, resourced by Victoria Police reports and Hospital injury information. Read/Import Data
url <- "https://vicroadsopendatastorehouse.vicroads.vic.gov.au/opendata/Road_Safety/ACCIDENT.csv"
mydata <- read.csv(url)
head(mydata)# explaination: because the data set has been save as .csv from url, so there is a simple function of reading csv file through url as read.csv(url), aftermath, checking head is a useful way to ensure all data imported in the right way as user expect. -->Inspect and Understand
dim(mydata)## [1] 171099 23colnames(mydata)## [1] "ACCIDENT_NO" "ACCIDENT_DATE" "ACCIDENT_TIME"
## [4] "ACCIDENT_TYPE" "ACCIDENT_TYPE_DESC" "DAY_OF_WEEK"
## [7] "DAY_WEEK_DESC" "DCA_CODE" "DCA_DESC"
## [10] "LIGHT_CONDITION" "NODE_ID" "NO_OF_VEHICLES"
## [13] "NO_PERSONS_KILLED" "NO_PERSONS_INJ_2" "NO_PERSONS_INJ_3"
## [16] "NO_PERSONS_NOT_INJ" "NO_PERSONS" "POLICE_ATTEND"
## [19] "ROAD_GEOMETRY" "ROAD_GEOMETRY_DESC" "SEVERITY"
## [22] "SPEED_ZONE" "RMA"glimpse(mydata)## Rows: 171,099
## Columns: 23
## $ ACCIDENT_NO <chr> "T20120000012", "T20120000013", "T20120000018", "T2…
## $ ACCIDENT_DATE <chr> "2012-01-01", "2012-01-01", "2012-01-01", "2012-01-…
## $ ACCIDENT_TIME <chr> "02:00:00", "03:35:00", "05:15:00", "07:30:00", "04…
## $ ACCIDENT_TYPE <int> 1, 1, 4, 4, 4, 2, 1, 1, 4, 2, 4, 1, 1, 1, 4, 8, 4, …
## $ ACCIDENT_TYPE_DESC <chr> "Collision with vehicle", "Collision with vehicle",…
## $ DAY_OF_WEEK <int> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, …
## $ DAY_WEEK_DESC <chr> "Sunday", "Sunday", "Sunday", "Sunday", "Sunday", "…
## $ DCA_CODE <int> 110, 160, 173, 171, 183, 108, 116, 120, 171, 102, 1…
## $ DCA_DESC <chr> "CROSS TRAFFIC(INTERSECTIONS ONLY)", "VEHICLE COLLI…
## $ LIGHT_CONDITION <int> 3, 3, 5, 1, 5, 3, 5, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, …
## $ NODE_ID <int> 41780, 69811, 22636, 248597, 248598, 53249, 43910, …
## $ NO_OF_VEHICLES <int> 2, 2, 1, 1, 1, 1, 2, 2, 1, 1, 1, 2, 2, 2, 1, 1, 1, …
## $ NO_PERSONS_KILLED <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, …
## $ NO_PERSONS_INJ_2 <int> 1, 1, 0, 0, 1, 0, 2, 1, 0, 0, 0, 1, 1, 0, 2, 1, 2, …
## $ NO_PERSONS_INJ_3 <int> 0, 0, 1, 2, 0, 1, 0, 0, 1, 1, 1, 2, 0, 1, 0, 0, 0, …
## $ NO_PERSONS_NOT_INJ <int> 2, 0, 0, 1, 0, 1, 1, 1, 1, 1, 0, 4, 1, 2, 0, 0, 0, …
## $ NO_PERSONS <int> 3, 1, 1, 3, 1, 2, 3, 2, 2, 2, 1, 7, 2, 3, 2, 1, 2, …
## $ POLICE_ATTEND <int> 1, 1, 1, 1, 1, 2, 1, 1, 2, 1, 1, 1, 1, 2, 1, 1, 1, …
## $ ROAD_GEOMETRY <int> 1, 2, 1, 5, 2, 2, 2, 2, 5, 2, 5, 2, 1, 1, 2, 1, 5, …
## $ ROAD_GEOMETRY_DESC <chr> "Cross intersection", "T intersection", "Cross inte…
## $ SEVERITY <int> 2, 2, 3, 3, 2, 3, 2, 2, 3, 3, 3, 2, 2, 3, 2, 2, 2, …
## $ SPEED_ZONE <int> 80, 60, 100, 50, 100, 50, 80, 60, 100, 60, 999, 80,…
## $ RMA <chr> "", "Arterial Other", "Arterial Highway", "Local Ro…#Explanation: to inspect and clearly understand the imported data, checking demension will be neccessary to apply through the function 'dim()' - this function allows to see briefly how many observations and variables of the imported data set contains. Next, to check the right name for each variable, colnames() function will be applicable. Finally, one of the most important things is needed to do for understanding data is that type of variables in the data set, so function 'glimpse()' will provide a detailed inforatiom to see through all type of data is contained in the dataSubsetting
df1 <- mydata[1:10, 1:23]
int1 <- df1 %>% select(where(is.integer))
as.matrix(int1)## ACCIDENT_TYPE DAY_OF_WEEK DCA_CODE LIGHT_CONDITION NODE_ID NO_OF_VEHICLES
## 1 1 1 110 3 41780 2
## 2 1 1 160 3 69811 2
## 3 4 1 173 5 22636 1
## 4 4 1 171 1 248597 1
## 5 4 1 183 5 248598 1
## 6 2 1 108 3 53249 1
## 7 1 1 116 5 43910 2
## 8 1 1 120 1 42677 2
## 9 4 1 171 1 250198 1
## 10 2 1 102 1 47545 1
## NO_PERSONS_KILLED NO_PERSONS_INJ_2 NO_PERSONS_INJ_3 NO_PERSONS_NOT_INJ
## 1 0 1 0 2
## 2 0 1 0 0
## 3 0 0 1 0
## 4 0 0 2 1
## 5 0 1 0 0
## 6 0 0 1 1
## 7 0 2 0 1
## 8 0 1 0 1
## 9 0 0 1 1
## 10 0 0 1 1
## NO_PERSONS POLICE_ATTEND ROAD_GEOMETRY SEVERITY SPEED_ZONE
## 1 3 1 1 2 80
## 2 1 1 2 2 60
## 3 1 1 1 3 100
## 4 3 1 5 3 50
## 5 1 1 2 2 100
## 6 2 2 2 3 50
## 7 3 1 2 2 80
## 8 2 1 2 2 60
## 9 2 2 5 3 100
## 10 2 1 2 3 60str(int1)## 'data.frame': 10 obs. of 15 variables:
## $ ACCIDENT_TYPE : int 1 1 4 4 4 2 1 1 4 2
## $ DAY_OF_WEEK : int 1 1 1 1 1 1 1 1 1 1
## $ DCA_CODE : int 110 160 173 171 183 108 116 120 171 102
## $ LIGHT_CONDITION : int 3 3 5 1 5 3 5 1 1 1
## $ NODE_ID : int 41780 69811 22636 248597 248598 53249 43910 42677 250198 47545
## $ NO_OF_VEHICLES : int 2 2 1 1 1 1 2 2 1 1
## $ NO_PERSONS_KILLED : int 0 0 0 0 0 0 0 0 0 0
## $ NO_PERSONS_INJ_2 : int 1 1 0 0 1 0 2 1 0 0
## $ NO_PERSONS_INJ_3 : int 0 0 1 2 0 1 0 0 1 1
## $ NO_PERSONS_NOT_INJ: int 2 0 0 1 0 1 1 1 1 1
## $ NO_PERSONS : int 3 1 1 3 1 2 3 2 2 2
## $ POLICE_ATTEND : int 1 1 1 1 1 2 1 1 2 1
## $ ROAD_GEOMETRY : int 1 2 1 5 2 2 2 2 5 2
## $ SEVERITY : int 2 2 3 3 2 3 2 2 3 3
## $ SPEED_ZONE : int 80 60 100 50 100 50 80 60 100 60chr1 <- df1 %>% select(where(is.character))
as.matrix(chr1)## ACCIDENT_NO ACCIDENT_DATE ACCIDENT_TIME ACCIDENT_TYPE_DESC
## 1 "T20120000012" "2012-01-01" "02:00:00" "Collision with vehicle"
## 2 "T20120000013" "2012-01-01" "03:35:00" "Collision with vehicle"
## 3 "T20120000018" "2012-01-01" "05:15:00" "Collision with a fixed object"
## 4 "T20120000021" "2012-01-01" "07:30:00" "Collision with a fixed object"
## 5 "T20120000028" "2012-01-01" "04:00:00" "Collision with a fixed object"
## 6 "T20120000032" "2012-01-01" "00:55:00" "Struck Pedestrian"
## 7 "T20120000043" "2012-01-01" "00:45:00" "Collision with vehicle"
## 8 "T20120000044" "2012-01-01" "16:25:00" "Collision with vehicle"
## 9 "T20120005091" "2012-01-01" "15:30:00" "Collision with a fixed object"
## 10 "T20120000046" "2012-01-01" "16:25:00" "Struck Pedestrian"
## DAY_WEEK_DESC DCA_DESC
## 1 "Sunday" "CROSS TRAFFIC(INTERSECTIONS ONLY)"
## 2 "Sunday" "VEHICLE COLLIDES WITH VEHICLE PARKED ON LEFT OF ROAD"
## 3 "Sunday" "RIGHT OFF CARRIAGEWAY INTO OBJECT/PARKED VEHICLE"
## 4 "Sunday" "LEFT OFF CARRIAGEWAY INTO OBJECT/PARKED VEHICLE"
## 5 "Sunday" "OFF LEFT BEND INTO OBJECT/PARKED VEHICLE"
## 6 "Sunday" "PED STRUCK WALKING TO/FROM OR BOARDING/ALIGHTING VEHICLE."
## 7 "Sunday" "LEFT NEAR (INTERSECTIONS ONLY)"
## 8 "Sunday" "HEAD ON (NOT OVERTAKING)"
## 9 "Sunday" "LEFT OFF CARRIAGEWAY INTO OBJECT/PARKED VEHICLE"
## 10 "Sunday" "FAR SIDE. PED HIT BY VEHICLE FROM THE LEFT"
## ROAD_GEOMETRY_DESC RMA
## 1 "Cross intersection" ""
## 2 "T intersection" "Arterial Other"
## 3 "Cross intersection" "Arterial Highway"
## 4 "Not at intersection" "Local Road"
## 5 "T intersection" ""
## 6 "T intersection" "Local Road"
## 7 "T intersection" "Arterial Highway"
## 8 "T intersection" "Arterial Other"
## 9 "Not at intersection" "Local Road"
## 10 "T intersection" "Arterial Other"str(chr1)## 'data.frame': 10 obs. of 8 variables:
## $ ACCIDENT_NO : chr "T20120000012" "T20120000013" "T20120000018" "T20120000021" ...
## $ ACCIDENT_DATE : chr "2012-01-01" "2012-01-01" "2012-01-01" "2012-01-01" ...
## $ ACCIDENT_TIME : chr "02:00:00" "03:35:00" "05:15:00" "07:30:00" ...
## $ ACCIDENT_TYPE_DESC: chr "Collision with vehicle" "Collision with vehicle" "Collision with a fixed object" "Collision with a fixed object" ...
## $ DAY_WEEK_DESC : chr "Sunday" "Sunday" "Sunday" "Sunday" ...
## $ DCA_DESC : chr "CROSS TRAFFIC(INTERSECTIONS ONLY)" "VEHICLE COLLIDES WITH VEHICLE PARKED ON LEFT OF ROAD" "RIGHT OFF CARRIAGEWAY INTO OBJECT/PARKED VEHICLE" "LEFT OFF CARRIAGEWAY INTO OBJECT/PARKED VEHICLE" ...
## $ ROAD_GEOMETRY_DESC: chr "Cross intersection" "T intersection" "Cross intersection" "Not at intersection" ...
## $ RMA : chr "" "Arterial Other" "Arterial Highway" "Local Road" ...#Explaination: To extracting the first 10 observations with all 23 variables out of the 'mydata' table, there is a new data table should be created and named as df1, technically, we can command 'df1 <- mydata[1:10, 1:23]', then will see the new one. To convert those data from df1 into the matrix, we need to follow the definition of a matrix is that the elements of a matrix must be of the same class. Therefore, to pick up the right variable with right type of data, a combination of function should be applied for this step as 'int1 <- df1 %>% select(where(is.integer))' - this facilitates to pick out 15 integer variables out of df1, then finally converting into a matrix by using as.matrix(int1). Also, to check the newly-created matrix, we can use str(int1). Same application on character variabless of df1, we can define as chr1 as a martrix of character varaibles. Creating a new data frame
new_df <- data.frame(score = c(51,48,69,72,35,56,78,47,82,90),
reslt = c("Pass","Fail","Credit","DI","Fail","Pass","DI","Fail","HD","HD"))
ordered_factor <- factor(c("Pass","Fail","Credit","DI","Fail","Pass","DI","Fail","HD","HD"), levels = c("Fail","Pass","Credit","DI","HD"), ordered = TRUE)
print(ordered_factor)## [1] Pass Fail Credit DI Fail Pass DI Fail HD HD
## Levels: Fail < Pass < Credit < DI < HDstr(new_df)## 'data.frame': 10 obs. of 2 variables:
## $ score: num 51 48 69 72 35 56 78 47 82 90
## $ reslt: chr "Pass" "Fail" "Credit" "DI" ...GPA <- c(1,0,2,3,0,1,3,0,4,4)
final_data <- cbind(new_df,GPA)
print(final_data)## score reslt GPA
## 1 51 Pass 1
## 2 48 Fail 0
## 3 69 Credit 2
## 4 72 DI 3
## 5 35 Fail 0
## 6 56 Pass 1
## 7 78 DI 3
## 8 47 Fail 0
## 9 82 HD 4
## 10 90 HD 4# Explanation: Creating a new data frame, there will be several steps to create some vectors. the data frame is all about the results of exam. Scores is set to display how many grades are recorded, and reslt as a result of the exam. new_df is set as the new one is required to create by applying function as new_df <- data.frame(score = c(51,48,69,72,35,56,78,47,82,90),reslt = c("Pass","Fail","Credit","DI","Fail","Pass","DI","Fail","HD","HD")).Then ordered_factor will be set to use the function factor() with levels() to display the order of the exam's result. Also, to add GPA as a new variable in the new_df, a new vector is assigned to GPA as: GPA <- c(1,0,2,3,0,1,3,0,4,4).Then using cbind() function to add them in together as: final_data <- cbind(new_df$score,new_df$reslt,GPA).R Markdown
This is an R Markdown document. Markdown is a simple formatting syntax for authoring HTML, PDF, and MS Word documents. For more details on using R Markdown see http://rmarkdown.rstudio.com.
When you click the Knit button a document will be generated that includes both content as well as the output of any embedded R code chunks within the document. You can embed an R code chunk like this:
summary(cars) ```
Including Plots
You can also embed plots, for example:
Note that the echo = FALSE parameter was added to the
code chunk to prevent printing of the R code that generated the
plot.