Week 10 Challenge Problem

Preparation Instructions

• Be sure to change the author in the YAML to your name. Remember to keep it inside the quotes.

• Questions that require the use of R will have an R code chunk below it.

• Download the datasets about birds (titled OrdwayBirds.csv and OrdwaySpeciesNames.csv) from the Canvas page for this assignment and save these files to the folder where the RMD file is located.

• There is less hand-holding in this assignment.

  • The packages that need to be loaded will no longer be provided in the setup code chunk. You are responsible for loading the appropriate packages for the assignment.
  • Questions on examining the data will no longer be provided. You are responsible for getting a sense of the data prior to completing the analyses.

THE DATASETS

For this Challenge Problem assignment, you are going to be using dataset, titled OrdwayBirds, on Minnesota bird species. This data was collected as part of a historical record of birds captured and released at a nature preserve in Inver Grove Heights, Minnesota (i.e., the Katharine Ordway Natural History Study Area, which is owned and managed by Macalester College in St. Paul, Minnesota). Originally written by hand in a field notebook, the entries have been transcribed into electronic format under the supervision of Jerald Dosch, Dept. of Biology, Macalester College.¹

Due to mistakes in data entry, the SpeciesName variable in the OrdwayBirds dataset needs some fixing. SpeciesName is intended to identify the species of each of the birds, but the spelling often varies among birds of the same biological species. This leads to misclassification of birds.

Fortunately, this error is easy to fix. A different dataset, titled OrdwaySpeciesNames, was created to take into account all of the original variations in the spelling of the species names (SpeciesName) and translate them into a unified spelling (SpeciesNameCleaned). That is, this other dataset provides a cross-reference between the original spelling and a common or more appropriate one.

The information from the two datasets, OrdwayBirds and OrdwaySpeciesNames, can be merged using a join_function() to correct the original spellings and then carry out further explorations of the Minnesota birds dataset.

EXERCISES

Practice Problems

Now let’s practice joining tables and describing the result of the specific join_function().

1. Perform a left_join() between the OrdwayBirds dataset and the OrdwaySpeciesNames dataset, saving it to a new object (so all the rows aren’t knitted to your HTML), and examine the dimensions of the new merged dataset.

OrdwayBirds_merged <- OrdwayBirds %>%
  left_join(OrdwaySpeciesNames, by = "SpeciesName")

## Warning in left_join(., OrdwaySpeciesNames, by = "SpeciesName"): Detected an unexpected many-to-many relationship between `x` and `y`.
## ℹ Row 4 of `x` matches multiple rows in `y`.
## ℹ Row 211 of `y` matches multiple rows in `x`.
## ℹ If a many-to-many relationship is expected, set `relationship =
##   "many-to-many"` to silence this warning.

dim(OrdwayBirds)

## [1] 15829    26

dim(OrdwaySpeciesNames)

## [1] 265   2

dim(OrdwayBirds_merged)

## [1] 17120    27

2. Describe what modifications were made when the two datasets were joined together into one using this join function (e.g., row changes? column changes?). Also, did the left_join() do what you thought it would do? Explain.

• Before the datasets were merged, the OrdwayBirds dataset had 15,829 observations and 26 variables for the names of bird species. On the other hand, in the OrdwaySpeciesName dataset, there were only 265 observations and two variables. After merging the datasets using the join function, there were 17,120 observations and 27 variables. Because the datasets were joined, the ‘SpeciesNameCleaned’ variable was included in the merged dataset, thereby increasing the number of variables. Additionally, because the OrdwayBirds dataset contained some ‘SpeciesName’ observations that matched multiple rows in the OrdwaySpeciesName dataset, the merged dataset also had more observations. Therefore, the left_join() function performed as expected.

[Hint: Are the number of rows in the OrdwayBirds dataset the same as your new merged dataset?]

3. Modify the OrdwaySpeciesNames dataset so that only the unique/distinct rows are in the dataset. Save this new dataset in an object called sn.

sn <- OrdwaySpeciesNames %>%
  distinct()

4. Perform the left_join() again, but this time between the OrdwayBirds dataset and the sn dataset, saving it to a new object, and examine the dimensions of the new merged dataset. Is this new merged dataset what you would expect from doing a left_join()? Explain.

• After evaluating the new merged dataset, this is what is expected when using a left_join function. The OrdwayBirds dataset contained 15,829 observations and 26 variables. After merging the datasets using the left_join function, every row from the OrdwayBirds dataset was kept, and each bird species was matched to only one unique row in the ‘sn’ dataset. Therefore, the merged dataset had the same number of observations as the OrdwayBirds dataset. Furthermore, because the ‘SpeciesNameCleaned’ variable from the ‘sn’ dataset was included in the new merged dataset, the dataset now had 27 variables.

sn_OrdwayBirds <- OrdwayBirds %>%
  left_join(sn, by = "SpeciesName")


dim(sn)

## [1] 251   2

view(sn)
dim(OrdwayBirds)

## [1] 15829    26

dim(sn_OrdwayBirds)

## [1] 15829    27

5. Perform an inner_join() between the OrdwayBirds dataset and the sn dataset, saving it to a new object. Examine the dimensions of the new merged dataset. Describe what modifications were made when the two datasets were joined together into one (e.g., row changes? column changes?).

• When using the inner_join function, the ‘SpeciesName’ observations that match both in the OrdwayBirds and ‘sn’ datasets are kept. Therefore, after merging the two datasets, the number of observations (rows) decreased from 15,829 to 15,724. The inner_join function also adds any additional variable (column) from the ‘sn’ dataset. Therefore, the merged dataset, which has 27 variables, includes the 26 variables from the OrdwayBirds dataset and an additional ‘SpeciesNameCleaned’ variable from the ‘sn’ dataset.

OrdwayBirds_inner <- OrdwayBirds %>%
  inner_join(sn, by = "SpeciesName")

dim(OrdwayBirds)

## [1] 15829    26

dim(OrdwayBirds_inner)

## [1] 15724    27

6. Perform a full_join() between the OrdwayBirds dataset and the sn dataset, saving it to a new object. Examine the dimensions of the new merged dataset. Describe what modifications were made when the two datasets were joined together into one (e.g., row changes? column changes?).

• When using the full_join function, all ‘SpeciesName’ observations from both the OrdwayBirds and ‘sn’ datasets are kept. After merging the datasets, the number of observations (rows) increased slightly from 15,829 to 15,830 because the ‘sn’ dataset included an additional bird species that was not present in the OrdwayBirds dataset. The full_join function also adds any additional variable (column) from the ‘sn’ dataset. Therefore, the merged dataset, which has 27 variables, includes the 26 variables from the OrdwayBirds dataset and an additional ‘SpeciesNameCleaned’ variable from the ‘sn’ dataset.

OrdwayBirds_full <- OrdwayBirds %>% 
  full_join(sn, by = "SpeciesName")


dim(OrdwayBirds)

## [1] 15829    26

dim(OrdwayBirds_full)

## [1] 15830    27

7. Perform a semi_join() between the OrdwayBirds dataset and the sn dataset, saving it to a new object. Examine the dimensions of the new merged dataset. Describe what modifications were made when the two datasets were joined together into one (e.g., row changes? column changes?).

• When using the semi_join function, only the ‘SpeciesName’ observations in the OrdwayBirds dataset that also match in the ‘sn’ dataset are kept. After merging the two datasets, the number of observations (rows) decreased from 15,829 to 15,724 because some of the ‘SpeciesName’ observations in the OrdwayBirds dataset did not match with the ‘sn’ dataset. The semi_join function only keeps the variables (columns) from the OrdwayBirds dataset; therefore, the merged dataset has only 26 variables.

OrdwayBirds_semi <- OrdwayBirds %>%
  semi_join(sn, by = "SpeciesName")


dim(OrdwayBirds)

## [1] 15829    26

dim(OrdwayBirds_semi)

## [1] 15724    26

8. Perform an anti_join() between the OrdwayBirds dataset and the sn dataset, saving it to a new object. Examine the dimensions of the new merged dataset. Describe what modifications were made when the two datasets were joined together into one (e.g., row changes? column changes?).

• When using the anti_join function, only the ‘SpeciesName’ observations in the Ordway dataset that do not match any ‘SpeciesName’ observations in the ‘sn’ dataset are kept. Therefore, after merging the two datasets, the number of observations (rows) decreased from 15,829 to 105. The anti_join function only keeps the variables (columns) from the OrdwayBirds dataset; therefore, the merged dataset has only 26 variables.

OrdwayBirds_anti <- OrdwayBirds %>% 
  anti_join(sn, by = "SpeciesName")


dim(OrdwayBirds)

## [1] 15829    26

dim(OrdwayBirds_anti)

## [1] 105  26

Putting It All Together

Now let’s put the joins to use (in addition to other data verbs and data visualization) to answer a question.

What is month-to-month presence of the most common bird species in the Ordway nature preserve area?

Think of this assignment as creating a resource for birders on the ideal time of year to visit Ordway to see a particular species.

In addition to the errors in SpeciesName, there are also problems with the Month and Day variables. They are supposed to be numerical, but mistakes prevent them from being correctly identified as such. The following code will take care of this issue with Month and Day: [Note: Take out eval=FALSE in the options of the code chunk so that the code executes in your assignment.]

birds <- OrdwayBirds %>%
  mutate(Month = as.numeric(as.character(Month)), 
        Day = as.numeric(as.character(Day)))

The next set of questions are going to walk you through the process of how to explore the data to answer the question.

9. Including misspellings, how many different species are there in the (newly created) birds dataset?

birds %>% 
  distinct(SpeciesName) %>%
  count()

10. How many distinct species are there in the SpeciesNameCleaned variable in sn dataset?

sn %>% 
  distinct(SpeciesNameCleaned) %>%
  count()

11. Perform the appropriate join function to merge the birds dataset with the sn dataset, such that only the matching rows from sn and birds are included and all columns from birds and sn are included. Also use the na.omit() function to remove missing data from your dataset (Note: This will remove all rows with ANY NAs). Save this merged dataset to a new object called birds_sn.

birds_sn <- birds %>%
  inner_join(sn, by = "SpeciesName") %>%
  na.omit()

12. How many bird captures are reported for each of the (correct) species? Be sure to arrange the data in descending order from the species with the most birds to the least.

birds_sn %>%
  count(SpeciesNameCleaned) %>%
  arrange(desc(n))

13. Let’s keep only the the most common bird species captured. Create a new dataset of counts similar to the answer from the previous question in an object called top_species that contains only the top 6 most common bird species.

top_species <- birds_sn %>%
  count(SpeciesNameCleaned, sort = TRUE) %>%
  head(6)

top_species

14. Now let’s create the final dataset to help answer the question. Create a new dataset in an object called top_species_month that contains the top species and a month-by-month count of each of the most common species. (Hint: use a specific type of join to limit the birds_sn entries to only the birds included in the top_species dataset, then count the number of sightings by species and month).

top_species_month <- birds_sn %>%
  semi_join(top_species, by = "SpeciesNameCleaned") %>%
  group_by(SpeciesNameCleaned, Month) %>%
  count() %>%
  arrange(SpeciesNameCleaned, Month)

15. Create a data visualization that helps to answer the question. Be sure to take into account meaningful design elements as you create your plot.

top_species_month %>%
  ggplot(aes(x = factor(Month), y = n, group = SpeciesNameCleaned, color = SpeciesNameCleaned)) +
  geom_line() + 
  facet_wrap(~SpeciesNameCleaned) + 
  theme_bw() + 
  theme(legend.position = "none",
        strip.text.x = element_text(size = 6),
        axis.text = element_text(size = 6),
        axis.title = element_text(size = 12),
        plot.title = element_text(hjust = 0.5)) + 
  labs(x = "Month", y = "# of Captures", title = "Most Common Bird Species: Month by Month Count ") 

16. Use the data visualization to answer these questions for the birders:

    1. Which most common species are present year-round?
    • Tree Swallow
    2. Which species are migratory, that is, primarily present in one or two seasons?
    • The Field Sparrow species was only primarily present in one season while the American Goldfinch, Black-capped Chickadee, Robin, and State-Colored Junco species are primarily present in two seasons.
    3. Which is the peak month for each major species?
    • American Goldfinch: May and October
    • Back-capped Chickadee: February and November
    • Field Sparrow: May
    • Robin: April - May
    • State-colored Junco: April and October
    • Tree Swallow: March, June, and November

---

1. Source: Data Computing textbook Project: Bird Species.↩︎