STAT 545A Homework 3

Sep 23 2013

Contents

• Introduction
• Data Aggregation Tasks
• An Interesting Side Story
• Conclusions

Introduction

We will continue to look at the Gapminder data provided by Jenny Bryan for the STAT 545A class.

The data contains 6 variables (country, year, pop, continent, lifeExp, gdpPercap), with 1704 rows of data. A short summary of the data and the variables can be found in the following table:

A quick look into the structure of the data is provided by the str function in R:

## 'data.frame':    1704 obs. of  6 variables:
##  $ country  : Factor w/ 142 levels "Afghanistan",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ year     : int  1952 1957 1962 1967 1972 1977 1982 1987 1992 1997 ...
##  $ pop      : num  8425333 9240934 10267083 11537966 13079460 ...
##  $ continent: Factor w/ 5 levels "Africa","Americas",..: 3 3 3 3 3 3 3 3 3 3 ...
##  $ lifeExp  : num  28.8 30.3 32 34 36.1 ...
##  $ gdpPercap: num  779 821 853 836 740 ...
## NULL

Using the Gapminder data set, we will be focusing on data aggregation tasks using the plyr package written by Hadley Wickam. We will stress the importance of figure in data exploration tasks by trying to extract information from the the data set using only tables. As it will soon become painfully obvious, the table is not the best display to use when one is trying to find interesting relationships or trends, unique properties, or secrets that may lie within the data.

Data Aggregation Tasks

The Rich and the Poor (according to GDP per capita), by Continent

We begin by investigating the quantitative variable, “GDP per capita”. We are interested in finding the countries with the lowest and highest GDP per capita for each continent, as well as the difference between the two. This is displayed in the table below, sorted by ascending order of minimum GDP per capita.

There are several interesting points that can be found in this table. It seems that the continent, Africa, contains the nation with the smallest GDP per capita, as well as the smallest “rich” country out of all the continents. Asia has the greatest gap between minimum GDP per capita nation and maximum GDP per capita nation, and it is also the home to the highest GDP per capita nation in the world. Oceania has the richest “poorest” country, when compared to the other continents.

Let us try displaying the above table in “Tall” format to check if it is easier or harder to decipher the data:

It seems that the “wide” table format is easier to read in this first example. This may be because having the data in a long format does not organize the data in a way that is immediately useful for us to understand, forcing us to use our brains more. If I had to choose one table to display, I would keep the “wide” table.

The next immediate question that arises from looking at these tables is “Which country in these continents have the lowest/highest GDP per capita?”. It is a natural extension to our above tables, so we add the country variable for both the minimum and maximum GDP per capita variables. We will leave out the variable that depicted the difference in GDP per capita between the minimum and maximum in the next table.

We were initially a little confused when Australia was found to be the country with both the minimum GDP per capita and the maximum GDP per capita in Oceania. If we delve further into the data set, we find two interesting facts. First, the continent, Oceania, contains only two countries, Australia and New Zealand. Second, Australia had a smaller GDP per capita when compared to New Zealand in 1952. However, the growth of GDP per capita in Australia overtook that of New Zealand, and we find that the maximum of this variable is depicted by Australia in 2012, as we read in the above table. Due to the my ineptitude in geography, I cannot say much about the other countries in the table, except that the United States having the greatest GDP per capita in the Americas was not surprising.

Next, we take a look at the spread of GDP per capita in each continent, and also the number of countries in each continent. We will look at the standard deviation of the GDP per capita variable.

The table is sorted by smallest to largest in the standard deviation of GDP per capita variable. Africa seems to have the smallest spread of GDP per capita, and Asia seems to have the largest. If we take a look at our first table, we can deduce that this may partly be explained by the small and large spread between the minimum and maximum GDP per capita countries within those two continents. It seems that all the countries in Africa have similar GDP per capita, while Asia shows the most diverse set of countries in terms of GDP per capita.

Life Expectancy over Time and by Continent

Next, we aim our focus on Life Expectancy. It would be interesting to take a look at global life expectancy and how it changed over the years. For each year, we calculate the mean life expectancy, the trimmed mean life expectancy, and the median life expectancy. The trimmed mean is calculated with 10% of the observations trimmed from each end of the life expectancy data in each year.

Generally, it looks like life expectancy has been increasing over the years from 1952 to 2007; there is a clear upward trend in life expectancy. The three measures of center seem to generally agree with each other. The median shows the greatest change in life expectancy out of the three statistics.

How about we look at the mean and median life expectancy by continent and year? Since we are calculating some summary statistic, we also include the number of observations used to calculate the statistic.

We notice, again, that the “tall” data format is not the best way to view this data. It may be easy to follow the life expectancy for a single continent at a time, but it is not easy to make direct comparisons of life expectancy for different continents at the same year.

We apply some plyr kung-fu using the daply function to force our data to be aggregated as a “wide” table. (Normally, this task can be easily accomplished by using Hadley Wickham's reshape or reshape2 package, but Jenny challenged us with using only plyr). Since we are going to a “wide” table, let us just calculate the mean life expectancy for each year, just to keep things a simple and readable.

Again, we see the advantage of using a “wide” table format over a “tall” format when we want to manually interpret the data in a table. It looks like the secret of long life resides in Oceania. Africa seems to be the most dangerous continent to life, and although the average life expectancy increased quite a bit since 1952, it has only caught up to 1952 levels of life expectancy in 2007! Asia shows the greatest increase in mean life expectancy in the range of the data provided.

Let us define an arbitrary value for “low life expectancy”. Say, if a country has a life expectancy smaller than the overall median life expectancy (calculated using the entire data set), then it is labelled as having low life expectancy. We tabulate the proportion of countries within each continent that have life expectancies smaller than the overall median (which is equal to 60.7125 years).

We choose to skip the “tall” format of the table and skip directly to the “wide” format. Again, this is done using only plyr functions (namely, daply). This table tells a similar story to the table above it.

An Interesting Side Story

While we were investigating GDP per capita, we were interested in countries with the lowest and highest GDP per capita within each continent at each year.

Our investigations produced a rather obtuse, and hard to read table.

Nothing seems to immediately stand out in the table. Usually, countries with the smallest or largest GDP per capita remain so for at least several decades. Also, the minimum and maximum GDP per capita seem to be generally increasing over time, though this is not always the case.

Then we have Asia, the maximum GDP per capita winner according to our previous tables. However, while the rest of the world is slowly increasing in GDP per capita, the richest Asian country, Kuwuit, shows a declining trend for GDP per capita, even losing its crown as top GDP per capita country twice in the process (once to Saudi Arabia in 1982, and once to Singapore in 2002).

For the countries in Asia, we can perform a simple linear regression of GDP per capita versus time and find the slope and the intercept for each country. As we did in class, we correct the year variable by subtracting the smallest year in the data. The estimated intercept and slopes are the next table, which is sorted by lowest to highest slope estimates.

It seems that, according to our very simple linear regression model*, Kuwait starts with a very high intercept, but has a very large negative slope! On the other hand, Singapore, which we found overtook the GDP per capita of Kuwait in 2002, started off with a moderately negative intercept, but cause up due to a very large slope!

Note that the assumptions for the linear regression fits were not checked. We doubt that a linear regresison model is suitable for our data.

It would be interesting to investigate the GDP per capita of Asia at a future date with plots instead of just tables.

Conclusions

In the end, we found it quite challenging to explore the Gapminder dataset using only tables. On the other hand, this homework provided great practice for using the data aggregation package plyr. Using ddply made everything much easier, and I have grown an appreciation for the “Split-Apply-Combine” methodology preached by Hadley Wickham and recommended by Jenny. I believe that it will be a very useful and powerful tool for data analysis when combined with making plots.

For the code used to generate this report, click here