Data Management - An Introduction

Outline

• Creating, organizing, and sharing data
• Data cleaning
• Data manipulation
• Data analysis
• A simple case of data table

Data formats

• Textual data: XML, TXT, HTML, PDF
• Tabular data: XLSX, CSV
• Databases: XML, CSV
• Images: TIFF, PNG, SVG, JPEG
• Audio: FLAC, WAV, MP3
• Software output: SAV, SAS7BDAT, DTA, RDATA

Organizing data

• Project folder: Libs, Figs, Scripts, Data
• Name your files consistently and meaningfully
• Version control
• Readme.txt
• Codebook.txt

Sharing your data

• Storage media: CD, email, Hard drives, USB
• Backup
• Websites: Dropbox, Github, Google

Data cleaning

Data manipulation

• Aggregate
• Merge
• Reshape
• Select
• Sort
• Subset
• Transform
• Generate

Data analysis

Data management - Before

Data management - After

Data format I

Data format II

Data format III

Data format IV

Gender: Female = 1, Male = 0
Pregnant: No = 0, Yes = 1

Data format V

ID: Subject Index
Gender: Female = 1, Male = 0
Pregnant: No = 0, Yes = 1

Generate raw data with R

dta <- data.frame(Gender = rep(c(1, 0), c(4, 2)),   
                  Pregnant = c(1, rep(0, 5)))
dta

##   Gender Pregnant
## 1      1        1
## 2      1        0
## 3      1        0
## 4      1        0
## 5      0        0
## 6      0        0

Recode variables

dta <- within(dta, {     
    Gender <- ifelse(Gender == 0, "Male", "Female")
    Pregnant <- ifelse(Pregnant == 0, "No", "Yes") 
    } )
dta

##   Gender Pregnant
## 1 Female      Yes
## 2 Female       No
## 3 Female       No
## 4 Female       No
## 5   Male       No
## 6   Male       No

Cross-classification

table(dta)

##         Pregnant
## Gender   No Yes
##   Female  3   1
##   Male    2   0

with(dta, table(Pregnant, Gender))

##         Gender
## Pregnant Female Male
##      No       3    2
##      Yes      1    0

Order of transformation matters

t(table(dta))

##         Gender
## Pregnant Female Male
##      No       3    2
##      Yes      1    0

table(t(dta))

## 
## Female   Male     No    Yes 
##      4      2      5      1

Turn tables into data frames

data.frame(table(dta))

##   Gender Pregnant Freq
## 1 Female       No    3
## 2   Male       No    2
## 3 Female      Yes    1
## 4   Male      Yes    0

Mosaicplot as default for table object

plot(table(dta), main = "")

mosaicplot(table(dta), main = "")

Save a table object and name the cell elements

tdta <- table(dta)
as.vector(tdta)

## [1] 3 2 1 0

names(tdta) <- c("Female_No", "Male_No", "Female_Yes", "Male_Yes")

Pie charts

pie(tdta, radius = 1, col = gray(seq(1, .4, length = 4)))

Doughtnut plot as an improvement?

pie(tdta, radius = 1, col = gray(seq(.8, .2, length = 4)))
symbols(0, 0, circles = .5, bg = "white", add = T)

Cleveland’s dot plot - raw data

dotchart(table(dta))

Cleveland’s dot plot - proportions

dotchart(prop.table(table(dta), 1))

Version control

The End