Data Wrangling in R

Alexander Matrunich
14 December 2017, R-Ladies Tbilisi

Agenda

  • Introduction
  • Basics of base R
    • Data types in R
    • Mysterious square brackets
  • Basics of tidyverse
    • Pipe %>%
    • dplyr
    • tydir

Intro

Alexander Matrunich: who am I?

  • Specialist degree in Sociology, 2003
  • Head of SocLab, Pskov Volny Institute (2005-2010)
  • Market Research expert, Danish Economical Support Programme (2008-2010)
  • Data Analyst, Institute of Regional Development (2011-2013)
  • R trainer, Statistian, Food and Agricultural Organization of the United Nations (2013-2017)

Questions to audience

Who does use anything from the following:

  • Excel
  • R
    • dplyr, tidyr, purrr
    • reshape, reshap2, plyr
    • ggplot2
    • lapply, vapply, sapply
  • Any other stat software?

Required software today

  • R + RStudio IDE
  • packages: dplyr, tidyr

Some basics of base R

Data types

Dimensions Homogeneous Heterogeneous
1D Atomic vector List
2D Matrix Data frame
nD Array Nested data frames (purrr)

Atomic (not recursive) data types

  • logical: TRUE \ FALSE
  • integer: 1L, 2L, 100L
  • double (numeric): 1, 1.1, 2.1, 3.1415
  • character: I am a string
  • null object: NULL

Extract or Replace Parts of an Object

  • [ returns candy with paper, [[ returns the candy only
  • [ can select more than one element
  • [[ and $ select only one element
  • Subsetting drops all attributes except names, dim and dimnames.
  • [[ supports computed indices

Your turn

  1. Create a list.
  2. Extract first element with [.
  3. Extract first element with [[.
  4. Compare.

Data processing in modern R

Pipe %>%

read_csv("/path/to/my.csv") %>% 
  select(age, sex) %>% 
  group_by(sex) %>% 
  summarize(age = mean(age))
summarize(
  group_by(
    select(
      read_csv("/path/to/my.csv"), age, sex),
    sex),
  age = mean(age))

A Grammar of Data Manipulation: dplyr

  • Similar tools to work with different data sources (R data frames, SQL data bases)
  • Many packages follow dplyr's philosophy
  • Easy to read code

Source code as a mean of communication

Only God knows

tibble - modern reimagining of the data.frame

  • does less (no changes of types or column names);
  • complains more (when a variable doesn't exist);
  • enhanced print method (also check dplyr::glimpse()).

Old-style data.frame

library(dplyr)
as.data.frame(starwars)
                    name height   mass
1         Luke Skywalker    172   77.0
2                  C-3PO    167   75.0
3                  R2-D2     96   32.0
4            Darth Vader    202  136.0
5            Leia Organa    150   49.0
6              Owen Lars    178  120.0
7     Beru Whitesun lars    165   75.0
8                  R5-D4     97   32.0
9      Biggs Darklighter    183   84.0
10        Obi-Wan Kenobi    182   77.0
11      Anakin Skywalker    188   84.0
12        Wilhuff Tarkin    180     NA
13             Chewbacca    228  112.0
14              Han Solo    180   80.0
15                Greedo    173   74.0
16 Jabba Desilijic Tiure    175 1358.0
17        Wedge Antilles    170   77.0
18      Jek Tono Porkins    180  110.0
19                  Yoda     66   17.0
20             Palpatine    170   75.0
21             Boba Fett    183   78.2
22                 IG-88    200  140.0
23                 Bossk    190  113.0
24      Lando Calrissian    177   79.0
25                 Lobot    175   79.0
26                Ackbar    180   83.0
27            Mon Mothma    150     NA
28          Arvel Crynyd     NA     NA
29 Wicket Systri Warrick     88   20.0
30             Nien Nunb    160   68.0
31          Qui-Gon Jinn    193   89.0
32           Nute Gunray    191   90.0
33         Finis Valorum    170     NA
34         Jar Jar Binks    196   66.0
35          Roos Tarpals    224   82.0
36            Rugor Nass    206     NA
37              Ric Olié    183     NA
38                 Watto    137     NA
39               Sebulba    112   40.0
40         Quarsh Panaka    183     NA
41        Shmi Skywalker    163     NA
42            Darth Maul    175   80.0
43           Bib Fortuna    180     NA
44           Ayla Secura    178   55.0
45              Dud Bolt     94   45.0
46               Gasgano    122     NA
47        Ben Quadinaros    163   65.0
48            Mace Windu    188   84.0
49          Ki-Adi-Mundi    198   82.0
50             Kit Fisto    196   87.0
51             Eeth Koth    171     NA
52            Adi Gallia    184   50.0
53           Saesee Tiin    188     NA
54           Yarael Poof    264     NA
55              Plo Koon    188   80.0
56            Mas Amedda    196     NA
57          Gregar Typho    185   85.0
58                 Cordé    157     NA
59           Cliegg Lars    183     NA
60     Poggle the Lesser    183   80.0
61       Luminara Unduli    170   56.2
62         Barriss Offee    166   50.0
63                 Dormé    165     NA
64                 Dooku    193   80.0
65   Bail Prestor Organa    191     NA
66            Jango Fett    183   79.0
67            Zam Wesell    168   55.0
68       Dexter Jettster    198  102.0
69               Lama Su    229   88.0
70               Taun We    213     NA
71            Jocasta Nu    167     NA
72         Ratts Tyerell     79   15.0
73                R4-P17     96     NA
74            Wat Tambor    193   48.0
75              San Hill    191     NA
76              Shaak Ti    178   57.0
77              Grievous    216  159.0
78               Tarfful    234  136.0
79       Raymus Antilles    188   79.0
80             Sly Moore    178   48.0
81            Tion Medon    206   80.0
82                  Finn     NA     NA
83                   Rey     NA     NA
84           Poe Dameron     NA     NA
85                   BB8     NA     NA
86        Captain Phasma     NA     NA
87         Padmé Amidala    165   45.0
      hair_color          skin_color
1          blond                fair
2           <NA>                gold
3           <NA>         white, blue
4           none               white
5          brown               light
6    brown, grey               light
7          brown               light
8           <NA>          white, red
9          black               light
10 auburn, white                fair
11         blond                fair
12  auburn, grey                fair
13         brown             unknown
14         brown                fair
15          <NA>               green
16          <NA>    green-tan, brown
17         brown                fair
18         brown                fair
19         white               green
20          grey                pale
21         black                fair
22          none               metal
23          none               green
24         black                dark
25          none               light
26          none        brown mottle
27        auburn                fair
28         brown                fair
29         brown               brown
30          none                grey
31         brown                fair
32          none       mottled green
33         blond                fair
34          none              orange
35          none                grey
36          none               green
37         brown                fair
38         black          blue, grey
39          none           grey, red
40         black                dark
41         black                fair
42          none                 red
43          none                pale
44          none                blue
45          none          blue, grey
46          none         white, blue
47          none grey, green, yellow
48          none                dark
49         white                pale
50          none               green
51         black               brown
52          none                dark
53          none                pale
54          none               white
55          none              orange
56          none                blue
57         black                dark
58         brown               light
59         brown                fair
60          none               green
61         black              yellow
62         black              yellow
63         brown               light
64         white                fair
65         black                 tan
66         black                 tan
67        blonde fair, green, yellow
68          none               brown
69          none                grey
70          none                grey
71         white                fair
72          none          grey, blue
73          none         silver, red
74          none         green, grey
75          none                grey
76          none    red, blue, white
77          none        brown, white
78         brown               brown
79         brown               light
80          none                pale
81          none                grey
82         black                dark
83         brown               light
84         brown               light
85          none                none
86       unknown             unknown
87         brown               light
       eye_color birth_year        gender
1           blue       19.0          male
2         yellow      112.0          <NA>
3            red       33.0          <NA>
4         yellow       41.9          male
5          brown       19.0        female
6           blue       52.0          male
7           blue       47.0        female
8            red         NA          <NA>
9          brown       24.0          male
10     blue-gray       57.0          male
11          blue       41.9          male
12          blue       64.0          male
13          blue      200.0          male
14         brown       29.0          male
15         black       44.0          male
16        orange      600.0 hermaphrodite
17         hazel       21.0          male
18          blue         NA          male
19         brown      896.0          male
20        yellow       82.0          male
21         brown       31.5          male
22           red       15.0          none
23           red       53.0          male
24         brown       31.0          male
25          blue       37.0          male
26        orange       41.0          male
27          blue       48.0        female
28         brown         NA          male
29         brown        8.0          male
30         black         NA          male
31          blue       92.0          male
32           red         NA          male
33          blue       91.0          male
34        orange       52.0          male
35        orange         NA          male
36        orange         NA          male
37          blue         NA          male
38        yellow         NA          male
39        orange         NA          male
40         brown       62.0          male
41         brown       72.0        female
42        yellow       54.0          male
43          pink         NA          male
44         hazel       48.0        female
45        yellow         NA          male
46         black         NA          male
47        orange         NA          male
48         brown       72.0          male
49        yellow       92.0          male
50         black         NA          male
51         brown         NA          male
52          blue         NA        female
53        orange         NA          male
54        yellow         NA          male
55         black       22.0          male
56          blue         NA          male
57         brown         NA          male
58         brown         NA        female
59          blue       82.0          male
60        yellow         NA          male
61          blue       58.0        female
62          blue       40.0        female
63         brown         NA        female
64         brown      102.0          male
65         brown       67.0          male
66         brown       66.0          male
67        yellow         NA        female
68        yellow         NA          male
69         black         NA          male
70         black         NA        female
71          blue         NA        female
72       unknown         NA          male
73     red, blue         NA        female
74       unknown         NA          male
75          gold         NA          male
76         black         NA        female
77 green, yellow         NA          male
78          blue         NA          male
79         brown         NA          male
80         white         NA        female
81         black         NA          male
82          dark         NA          male
83         hazel         NA        female
84         brown         NA          male
85         black         NA          none
86       unknown         NA        female
87         brown       46.0        female
        homeworld        species
1        Tatooine          Human
2        Tatooine          Droid
3           Naboo          Droid
4        Tatooine          Human
5        Alderaan          Human
6        Tatooine          Human
7        Tatooine          Human
8        Tatooine          Droid
9        Tatooine          Human
10        Stewjon          Human
11       Tatooine          Human
12         Eriadu          Human
13       Kashyyyk        Wookiee
14       Corellia          Human
15          Rodia         Rodian
16      Nal Hutta           Hutt
17       Corellia          Human
18     Bestine IV          Human
19           <NA> Yoda's species
20          Naboo          Human
21         Kamino          Human
22           <NA>          Droid
23      Trandosha     Trandoshan
24        Socorro          Human
25         Bespin          Human
26       Mon Cala   Mon Calamari
27      Chandrila          Human
28           <NA>          Human
29          Endor           Ewok
30        Sullust      Sullustan
31           <NA>          Human
32 Cato Neimoidia      Neimodian
33      Coruscant          Human
34          Naboo         Gungan
35          Naboo         Gungan
36          Naboo         Gungan
37          Naboo           <NA>
38       Toydaria      Toydarian
39      Malastare            Dug
40          Naboo           <NA>
41       Tatooine          Human
42       Dathomir         Zabrak
43         Ryloth        Twi'lek
44         Ryloth        Twi'lek
45        Vulpter     Vulptereen
46        Troiken          Xexto
47           Tund          Toong
48     Haruun Kal          Human
49          Cerea         Cerean
50    Glee Anselm       Nautolan
51       Iridonia         Zabrak
52      Coruscant     Tholothian
53        Iktotch       Iktotchi
54        Quermia       Quermian
55          Dorin        Kel Dor
56       Champala       Chagrian
57          Naboo          Human
58          Naboo          Human
59       Tatooine          Human
60       Geonosis      Geonosian
61         Mirial       Mirialan
62         Mirial       Mirialan
63          Naboo          Human
64        Serenno          Human
65       Alderaan          Human
66   Concord Dawn          Human
67          Zolan       Clawdite
68           Ojom       Besalisk
69         Kamino       Kaminoan
70         Kamino       Kaminoan
71      Coruscant          Human
72    Aleen Minor         Aleena
73           <NA>           <NA>
74          Skako        Skakoan
75     Muunilinst           Muun
76          Shili        Togruta
77          Kalee        Kaleesh
78       Kashyyyk        Wookiee
79       Alderaan          Human
80         Umbara           <NA>
81         Utapau         Pau'an
82           <NA>          Human
83           <NA>          Human
84           <NA>          Human
85           <NA>          Droid
86           <NA>           <NA>
87          Naboo          Human
                                                                                                                                       films
1                                            Revenge of the Sith, Return of the Jedi, The Empire Strikes Back, A New Hope, The Force Awakens
2                     Attack of the Clones, The Phantom Menace, Revenge of the Sith, Return of the Jedi, The Empire Strikes Back, A New Hope
3  Attack of the Clones, The Phantom Menace, Revenge of the Sith, Return of the Jedi, The Empire Strikes Back, A New Hope, The Force Awakens
4                                                               Revenge of the Sith, Return of the Jedi, The Empire Strikes Back, A New Hope
5                                            Revenge of the Sith, Return of the Jedi, The Empire Strikes Back, A New Hope, The Force Awakens
6                                                                                      Attack of the Clones, Revenge of the Sith, A New Hope
7                                                                                      Attack of the Clones, Revenge of the Sith, A New Hope
8                                                                                                                                 A New Hope
9                                                                                                                                 A New Hope
10                    Attack of the Clones, The Phantom Menace, Revenge of the Sith, Return of the Jedi, The Empire Strikes Back, A New Hope
11                                                                             Attack of the Clones, The Phantom Menace, Revenge of the Sith
12                                                                                                           Revenge of the Sith, A New Hope
13                                           Revenge of the Sith, Return of the Jedi, The Empire Strikes Back, A New Hope, The Force Awakens
14                                                                Return of the Jedi, The Empire Strikes Back, A New Hope, The Force Awakens
15                                                                                                                                A New Hope
16                                                                                        The Phantom Menace, Return of the Jedi, A New Hope
17                                                                                   Return of the Jedi, The Empire Strikes Back, A New Hope
18                                                                                                                                A New Hope
19                                Attack of the Clones, The Phantom Menace, Revenge of the Sith, Return of the Jedi, The Empire Strikes Back
20                                Attack of the Clones, The Phantom Menace, Revenge of the Sith, Return of the Jedi, The Empire Strikes Back
21                                                                         Attack of the Clones, Return of the Jedi, The Empire Strikes Back
22                                                                                                                   The Empire Strikes Back
23                                                                                                                   The Empire Strikes Back
24                                                                                               Return of the Jedi, The Empire Strikes Back
25                                                                                                                   The Empire Strikes Back
26                                                                                                     Return of the Jedi, The Force Awakens
27                                                                                                                        Return of the Jedi
28                                                                                                                        Return of the Jedi
29                                                                                                                        Return of the Jedi
30                                                                                                                        Return of the Jedi
31                                                                                                                        The Phantom Menace
32                                                                             Attack of the Clones, The Phantom Menace, Revenge of the Sith
33                                                                                                                        The Phantom Menace
34                                                                                                  Attack of the Clones, The Phantom Menace
35                                                                                                                        The Phantom Menace
36                                                                                                                        The Phantom Menace
37                                                                                                                        The Phantom Menace
38                                                                                                  Attack of the Clones, The Phantom Menace
39                                                                                                                        The Phantom Menace
40                                                                                                                        The Phantom Menace
41                                                                                                  Attack of the Clones, The Phantom Menace
42                                                                                                                        The Phantom Menace
43                                                                                                                        Return of the Jedi
44                                                                             Attack of the Clones, The Phantom Menace, Revenge of the Sith
45                                                                                                                        The Phantom Menace
46                                                                                                                        The Phantom Menace
47                                                                                                                        The Phantom Menace
48                                                                             Attack of the Clones, The Phantom Menace, Revenge of the Sith
49                                                                             Attack of the Clones, The Phantom Menace, Revenge of the Sith
50                                                                             Attack of the Clones, The Phantom Menace, Revenge of the Sith
51                                                                                                   The Phantom Menace, Revenge of the Sith
52                                                                                                   The Phantom Menace, Revenge of the Sith
53                                                                                                   The Phantom Menace, Revenge of the Sith
54                                                                                                                        The Phantom Menace
55                                                                             Attack of the Clones, The Phantom Menace, Revenge of the Sith
56                                                                                                  Attack of the Clones, The Phantom Menace
57                                                                                                                      Attack of the Clones
58                                                                                                                      Attack of the Clones
59                                                                                                                      Attack of the Clones
60                                                                                                 Attack of the Clones, Revenge of the Sith
61                                                                                                 Attack of the Clones, Revenge of the Sith
62                                                                                                                      Attack of the Clones
63                                                                                                                      Attack of the Clones
64                                                                                                 Attack of the Clones, Revenge of the Sith
65                                                                                                 Attack of the Clones, Revenge of the Sith
66                                                                                                                      Attack of the Clones
67                                                                                                                      Attack of the Clones
68                                                                                                                      Attack of the Clones
69                                                                                                                      Attack of the Clones
70                                                                                                                      Attack of the Clones
71                                                                                                                      Attack of the Clones
72                                                                                                                        The Phantom Menace
73                                                                                                 Attack of the Clones, Revenge of the Sith
74                                                                                                                      Attack of the Clones
75                                                                                                                      Attack of the Clones
76                                                                                                 Attack of the Clones, Revenge of the Sith
77                                                                                                                       Revenge of the Sith
78                                                                                                                       Revenge of the Sith
79                                                                                                           Revenge of the Sith, A New Hope
80                                                                                                 Attack of the Clones, Revenge of the Sith
81                                                                                                                       Revenge of the Sith
82                                                                                                                         The Force Awakens
83                                                                                                                         The Force Awakens
84                                                                                                                         The Force Awakens
85                                                                                                                         The Force Awakens
86                                                                                                                         The Force Awakens
87                                                                             Attack of the Clones, The Phantom Menace, Revenge of the Sith
                               vehicles
1    Snowspeeder, Imperial Speeder Bike
2                                      
3                                      
4                                      
5                 Imperial Speeder Bike
6                                      
7                                      
8                                      
9                                      
10                      Tribubble bongo
11 Zephyr-G swoop bike, XJ-6 airspeeder
12                                     
13                                AT-ST
14                                     
15                                     
16                                     
17                          Snowspeeder
18                                     
19                                     
20                                     
21                                     
22                                     
23                                     
24                                     
25                                     
26                                     
27                                     
28                                     
29                                     
30                                     
31                      Tribubble bongo
32                                     
33                                     
34                                     
35                                     
36                                     
37                                     
38                                     
39                                     
40                                     
41                                     
42                         Sith speeder
43                                     
44                                     
45                                     
46                                     
47                                     
48                                     
49                                     
50                                     
51                                     
52                                     
53                                     
54                                     
55                                     
56                                     
57                                     
58                                     
59                                     
60                                     
61                                     
62                                     
63                                     
64                     Flitknot speeder
65                                     
66                                     
67           Koro-2 Exodrive airspeeder
68                                     
69                                     
70                                     
71                                     
72                                     
73                                     
74                                     
75                                     
76                                     
77          Tsmeu-6 personal wheel bike
78                                     
79                                     
80                                     
81                                     
82                                     
83                                     
84                                     
85                                     
86                                     
87                                     
                                                                                                  starships
1                                                                                  X-wing, Imperial shuttle
2                                                                                                          
3                                                                                                          
4                                                                                           TIE Advanced x1
5                                                                                                          
6                                                                                                          
7                                                                                                          
8                                                                                                          
9                                                                                                    X-wing
10 Jedi starfighter, Trade Federation cruiser, Naboo star skiff, Jedi Interceptor, Belbullab-22 starfighter
11                                                Trade Federation cruiser, Jedi Interceptor, Naboo fighter
12                                                                                                         
13                                                                      Millennium Falcon, Imperial shuttle
14                                                                      Millennium Falcon, Imperial shuttle
15                                                                                                         
16                                                                                                         
17                                                                                                   X-wing
18                                                                                                   X-wing
19                                                                                                         
20                                                                                                         
21                                                                                                  Slave 1
22                                                                                                         
23                                                                                                         
24                                                                                        Millennium Falcon
25                                                                                                         
26                                                                                                         
27                                                                                                         
28                                                                                                   A-wing
29                                                                                                         
30                                                                                        Millennium Falcon
31                                                                                                         
32                                                                                                         
33                                                                                                         
34                                                                                                         
35                                                                                                         
36                                                                                                         
37                                                                                     Naboo Royal Starship
38                                                                                                         
39                                                                                                         
40                                                                                                         
41                                                                                                         
42                                                                                                 Scimitar
43                                                                                                         
44                                                                                                         
45                                                                                                         
46                                                                                                         
47                                                                                                         
48                                                                                                         
49                                                                                                         
50                                                                                                         
51                                                                                                         
52                                                                                                         
53                                                                                                         
54                                                                                                         
55                                                                                         Jedi starfighter
56                                                                                                         
57                                                                                            Naboo fighter
58                                                                                                         
59                                                                                                         
60                                                                                                         
61                                                                                                         
62                                                                                                         
63                                                                                                         
64                                                                                                         
65                                                                                                         
66                                                                                                         
67                                                                                                         
68                                                                                                         
69                                                                                                         
70                                                                                                         
71                                                                                                         
72                                                                                                         
73                                                                                                         
74                                                                                                         
75                                                                                                         
76                                                                                                         
77                                                                                 Belbullab-22 starfighter
78                                                                                                         
79                                                                                                         
80                                                                                                         
81                                                                                                         
82                                                                                                         
83                                                                                                         
84                                                                                      T-70 X-wing fighter
85                                                                                                         
86                                                                                                         
87                                                     H-type Nubian yacht, Naboo star skiff, Naboo fighter

tibble

starwars
# A tibble: 87 x 13
                 name height  mass    hair_color
                <chr>  <int> <dbl>         <chr>
 1     Luke Skywalker    172    77         blond
 2              C-3PO    167    75          <NA>
 3              R2-D2     96    32          <NA>
 4        Darth Vader    202   136          none
 5        Leia Organa    150    49         brown
 6          Owen Lars    178   120   brown, grey
 7 Beru Whitesun lars    165    75         brown
 8              R5-D4     97    32          <NA>
 9  Biggs Darklighter    183    84         black
10     Obi-Wan Kenobi    182    77 auburn, white
# ... with 77 more rows, and 9 more variables:
#   skin_color <chr>, eye_color <chr>,
#   birth_year <dbl>, gender <chr>,
#   homeworld <chr>, species <chr>, films <list>,
#   vehicles <list>, starships <list>

glimpse() from dplyr

glimpse(starwars)
Observations: 87
Variables: 13
$ name       <chr> "Luke Skywalker", "C-3PO",...
$ height     <int> 172, 167, 96, 202, 150, 17...
$ mass       <dbl> 77, 75, 32, 136, 49, 120, ...
$ hair_color <chr> "blond", NA, NA, "none", "...
$ skin_color <chr> "fair", "gold", "white, bl...
$ eye_color  <chr> "blue", "yellow", "red", "...
$ birth_year <dbl> 19.0, 112.0, 33.0, 41.9, 1...
$ gender     <chr> "male", NA, NA, "male", "f...
$ homeworld  <chr> "Tatooine", "Tatooine", "N...
$ species    <chr> "Human", "Droid", "Droid",...
$ films      <list> [<"Revenge of the Sith", ...
$ vehicles   <list> [<"Snowspeeder", "Imperia...
$ starships  <list> [<"X-wing", "Imperial shu...

dplyr's verbs (some)

  • filter(): filter rows based on conditions
  • select(): select columns
  • mutate(): create new variables
  • arrange(): reorder rows
  • group_by(): group rows by categories
  • summarize(): calculate [groups] totals
  • *_join(): family of functions to merge tables

filter()

starwars %>% 
  filter(species == "Droid")
# A tibble: 5 x 13
   name height  mass hair_color  skin_color
  <chr>  <int> <dbl>      <chr>       <chr>
1 C-3PO    167    75       <NA>        gold
2 R2-D2     96    32       <NA> white, blue
3 R5-D4     97    32       <NA>  white, red
4 IG-88    200   140       none       metal
5   BB8     NA    NA       none        none
# ... with 8 more variables: eye_color <chr>,
#   birth_year <dbl>, gender <chr>,
#   homeworld <chr>, species <chr>, films <list>,
#   vehicles <list>, starships <list>

filter()

starwars %>% 
  select(name, ends_with("color"))
# A tibble: 87 x 4
                 name    hair_color  skin_color
                <chr>         <chr>       <chr>
 1     Luke Skywalker         blond        fair
 2              C-3PO          <NA>        gold
 3              R2-D2          <NA> white, blue
 4        Darth Vader          none       white
 5        Leia Organa         brown       light
 6          Owen Lars   brown, grey       light
 7 Beru Whitesun lars         brown       light
 8              R5-D4          <NA>  white, red
 9  Biggs Darklighter         black       light
10     Obi-Wan Kenobi auburn, white        fair
# ... with 77 more rows, and 1 more variables:
#   eye_color <chr>

mutate() & select()

starwars %>% 
  mutate(name, bmi = mass / ((height / 100)  ^ 2)) %>%
  select(name:mass, bmi)
# A tibble: 87 x 4
                 name height  mass      bmi
                <chr>  <int> <dbl>    <dbl>
 1     Luke Skywalker    172    77 26.02758
 2              C-3PO    167    75 26.89232
 3              R2-D2     96    32 34.72222
 4        Darth Vader    202   136 33.33007
 5        Leia Organa    150    49 21.77778
 6          Owen Lars    178   120 37.87401
 7 Beru Whitesun lars    165    75 27.54821
 8              R5-D4     97    32 34.00999
 9  Biggs Darklighter    183    84 25.08286
10     Obi-Wan Kenobi    182    77 23.24598
# ... with 77 more rows

arrange()

starwars %>% 
  arrange(desc(mass))
# A tibble: 87 x 13
                    name height  mass  hair_color
                   <chr>  <int> <dbl>       <chr>
 1 Jabba Desilijic Tiure    175  1358        <NA>
 2              Grievous    216   159        none
 3                 IG-88    200   140        none
 4           Darth Vader    202   136        none
 5               Tarfful    234   136       brown
 6             Owen Lars    178   120 brown, grey
 7                 Bossk    190   113        none
 8             Chewbacca    228   112       brown
 9      Jek Tono Porkins    180   110       brown
10       Dexter Jettster    198   102        none
# ... with 77 more rows, and 9 more variables:
#   skin_color <chr>, eye_color <chr>,
#   birth_year <dbl>, gender <chr>,
#   homeworld <chr>, species <chr>, films <list>,
#   vehicles <list>, starships <list>

Let's rock!

starwars %>%
  group_by(species) %>%
  summarise(
    n = n(),
    mass = mean(mass, na.rm = TRUE)
  ) %>%
  filter(n > 1)
# A tibble: 9 x 3
   species     n      mass
     <chr> <int>     <dbl>
1    Droid     5  69.75000
2   Gungan     3  74.00000
3    Human    35  82.78182
4 Kaminoan     2  88.00000
5 Mirialan     2  53.10000
6  Twi'lek     2  55.00000
7  Wookiee     2 124.00000
8   Zabrak     2  80.00000
9     <NA>     5  48.00000

tidyr

What is tidy data?

  1. Each variable is in a column.
  2. Each observation is a row.
  3. Each value is a cell.

The patient

df1 <- tribble(
  ~student, ~biology, ~math,
    "Alex",       82,    70,
  "Sandro",       NA,    85,
   "Shura",       71,    68
)
df1
# A tibble: 3 x 3
  student biology  math
    <chr>   <dbl> <dbl>
1    Alex      82    70
2  Sandro      NA    85
3   Shura      71    68

A twin of the patient

(df2 <- tribble(
  ~Alex, ~Sandro, ~Shura,
     82,      NA,     71,
     70,      85,     68
))
# A tibble: 2 x 3
   Alex Sandro Shura
  <dbl>  <dbl> <dbl>
1    82     NA    71
2    70     85    68
df1
# A tibble: 3 x 3
  student biology  math
    <chr>   <dbl> <dbl>
1    Alex      82    70
2  Sandro      NA    85
3   Shura      71    68

Scalpel, please!

library(tidyr)
df1 %>% 
  gather("subject", "mark", -student)
# A tibble: 6 x 3
  student subject  mark
    <chr>   <chr> <dbl>
1    Alex biology    82
2  Sandro biology    NA
3   Shura biology    71
4    Alex    math    70
5  Sandro    math    85
6   Shura    math    68

What about the twin?

df2 %>% 
  mutate(subject = row_number()) %>% 
  glimpse() %>% 
  gather("student", "mark", -subject) %>% 
  select(2, 1, 3)
Observations: 2
Variables: 4
$ Alex    <dbl> 82, 70
$ Sandro  <dbl> NA, 85
$ Shura   <dbl> 71, 68
$ subject <int> 1, 2
# A tibble: 6 x 3
  student subject  mark
    <chr>   <int> <dbl>
1    Alex       1    82
2    Alex       2    70
3  Sandro       1    NA
4  Sandro       2    85
5   Shura       1    71
6   Shura       2    68

Sources

Contacts

Alexander Matrunich