Data Management - An Introduction

Outline

• Creating, organizing, and sharing data
• Data cleaning
• Data manipulation
• Data analysis
• A simple case of data table

Data formats

• Textual data: XML, TXT, HTML, PDF
• Tabular data: XLSX, CSV
• Databases: XML, CSV
• Images: TIFF, PNG, SVG, JPEG
• Audio: FLAC, WAV, MP3
• Software output: SAV, SAS7BDAT, DTA, RDATA

Organizing data

• Project folder: Libs, Figs, Scripts, Data
• Name your files consistently and meaningfully
• Version control
• Readme.txt
• Codebook.txt

Sharing your data

• Storage media: CD, email, Hard drives, USB
• Backup
• Websites: Dropbox, Github, Google

Data cleaning

Data manipulation

• Aggregate
• Merge
• Reshape
• Select
• Sort
• Subset
• Transform
• Generate

Data analysis

Data management - Before

Data management - After

Data format I

Data format II

Data format III

Data format IV

Gender: Female = 1, Male = 0
Pregnant: No = 0, Yes = 1

Data format V

ID: Subject Index
Gender: Female = 1, Male = 0
Pregnant: No = 0, Yes = 1

Generate raw data with R

dta <- data.frame(Gender = rep(c(1, 0), c(4, 2)),   
                  Pregnant = c(1, rep(0, 5)))
dta

  Gender Pregnant
1      1        1
2      1        0
3      1        0
4      1        0
5      0        0
6      0        0

Recode variables

dta <- within(dta, {     
    Gender <- ifelse(Gender == 0, "Male", "Female")
    Pregnant <- ifelse(Pregnant == 0, "No", "Yes") 
    } )
dta

  Gender Pregnant
1 Female      Yes
2 Female       No
3 Female       No
4 Female       No
5   Male       No
6   Male       No

Cross-classification

table(dta)

        Pregnant
Gender   No Yes
  Female  3   1
  Male    2   0

with(dta, table(Pregnant, Gender))

        Gender
Pregnant Female Male
     No       3    2
     Yes      1    0

Order of transformation matters

t(table(dta))

        Gender
Pregnant Female Male
     No       3    2
     Yes      1    0

table(t(dta))

Female   Male     No    Yes 
     4      2      5      1 

Turn tables into data frames

data.frame(table(dta))

  Gender Pregnant Freq
1 Female       No    3
2   Male       No    2
3 Female      Yes    1
4   Male      Yes    0

Mosaicplot as default for table object

plot(table(dta), main = "")

mosaicplot(table(dta), main = "")

Save a table object and name the cell elements

tdta <- table(dta)
as.vector(tdta)

[1] 3 2 1 0

names(tdta) <- c("Female_No", "Male_No", "Female_Yes", "Male_Yes")

Pie charts

pie(tdta, radius = 1, col = gray(seq(1, .4, length = 4)))

Doughtnut plot as an improvement?

pie(tdta, radius = 1, col = gray(seq(.8, .2, length = 4)))
symbols(0, 0, circles = .5, bg = "white", add = T)

Cleveland’s dot plot - raw data

dotchart(table(dta))

Cleveland’s dot plot - proportions

dotchart(prop.table(table(dta), 1))

Version control

The End