Principles of Data Visualization and Introduction to ggplot2

I have provided you with data about the 5,000 fastest growing companies in the US, as compiled by Inc. magazine. lets read this in:

library(dplyr)    # subseting, changing data layout
library(ggplot2)  # graphs generation
library(knitr)    # Report display, table format
library(kableExtra)

#Load data
inc <- read.csv("https://raw.githubusercontent.com/charleyferrari/CUNY_DATA_608/master/module1/Data/inc5000_data.csv", header= TRUE, stringsAsFactors = F)

And lets preview this data:

head(inc)
##   Rank                         Name Growth_Rate   Revenue
## 1    1                         Fuhu      421.48 1.179e+08
## 2    2        FederalConference.com      248.31 4.960e+07
## 3    3                The HCI Group      245.45 2.550e+07
## 4    4                      Bridger      233.08 1.900e+09
## 5    5                       DataXu      213.37 8.700e+07
## 6    6 MileStone Community Builders      179.38 4.570e+07
##                       Industry Employees         City State
## 1 Consumer Products & Services       104   El Segundo    CA
## 2          Government Services        51     Dumfries    VA
## 3                       Health       132 Jacksonville    FL
## 4                       Energy        50      Addison    TX
## 5      Advertising & Marketing       220       Boston    MA
## 6                  Real Estate        63       Austin    TX
summary(inc)
##       Rank          Name            Growth_Rate         Revenue         
##  Min.   :   1   Length:5001        Min.   :  0.340   Min.   :2.000e+06  
##  1st Qu.:1252   Class :character   1st Qu.:  0.770   1st Qu.:5.100e+06  
##  Median :2502   Mode  :character   Median :  1.420   Median :1.090e+07  
##  Mean   :2502                      Mean   :  4.612   Mean   :4.822e+07  
##  3rd Qu.:3751                      3rd Qu.:  3.290   3rd Qu.:2.860e+07  
##  Max.   :5000                      Max.   :421.480   Max.   :1.010e+10  
##                                                                         
##    Industry           Employees           City          
##  Length:5001        Min.   :    1.0   Length:5001       
##  Class :character   1st Qu.:   25.0   Class :character  
##  Mode  :character   Median :   53.0   Mode  :character  
##                     Mean   :  232.7                     
##                     3rd Qu.:  132.0                     
##                     Max.   :66803.0                     
##                     NA's   :12                          
##     State          
##  Length:5001       
##  Class :character  
##  Mode  :character  
##                    
##                    
##                    
##

Think a bit on what these summaries mean. Use the space below to add some more relevant non-visual exploratory information you think helps you understand this data:

Summary

Minimum employees per company is one, average is 233 and maximum is 66803. Values are for entire dataset.

Revenues also, summary is for entire dataset. Minmum revenues generated by a company is $200,000, average is $4,822,000 and maximum is $1,010,000,000

# Insert your code here, create more chunks as necessary

Question 1

Create a graph that shows the distribution of companies in the dataset by State (ie how many are in each state). There are a lot of States, so consider which axis you should use. This visualization is ultimately going to be consumed on a ‘portrait’ oriented screen (ie taller than wide), which should further guide your layout choices.

# Answer Question 1 here
# Calculate companies count
state.company <- inc %>%
  group_by(State) %>% 
  summarise(companiesCount = n()) %>%
  select (State,companiesCount)

#Display data
state.company %>% 
  kable(format="html", caption = "Companies Count by State") %>% 
  kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = F, position = "left")
Companies Count by State
State companiesCount
AK 2
AL 51
AR 9
AZ 100
CA 701
CO 134
CT 50
DC 43
DE 16
FL 282
GA 212
HI 7
IA 28
ID 17
IL 273
IN 69
KS 38
KY 40
LA 37
MA 182
MD 131
ME 13
MI 126
MN 88
MO 59
MS 12
MT 4
NC 137
ND 10
NE 27
NH 24
NJ 158
NM 5
NV 26
NY 311
OH 186
OK 46
OR 49
PA 164
PR 1
RI 16
SC 48
SD 3
TN 82
TX 387
UT 95
VA 283
VT 6
WA 130
WI 79
WV 2
WY 2 
#Generate the graph
ggplot(state.company, aes(x=State, y=companiesCount)) +
  geom_bar(width=.5,stat="identity",position = "dodge") +
  theme(text=element_text(size=8)) +
  ggtitle("Distribution of Companies By State") +
  xlab("State") + ylab("Companies Count") +
  coord_flip()

According to graph California has most companies. Followed by Texas and New York.

Quesiton 2

Lets dig in on the state with the 3rd most companies in the data set. Imagine you work for the state and are interested in how many people are employed by companies in different industries. Create a plot that shows the average and/or median employment by industry for companies in this state (only use cases with full data, use R’s complete.cases() function.) In addition to this, your graph should show how variable the ranges are, and you should deal with outliers.

I have used two methods to answer the question, the first method uses average employment per industry to remove outliers that fall outside \(\pm\ 2\ Standard deviations\) per industry. Second method calculates the average and standard deviation across all industries in the state and removes outliers. Both methods skew the results because outliers have a higher impact on output.

Example: Company Sutherland Global Services in Business Products & Services employs 32000 people and company TransPerfect employs 2218 people. Rest all companies combined in Business Products & Services provide employment to 4586 people. This distribution impacts average and standard deviation.

Average employment in Business Products & Services without companies Sutherland Global Services and TransPerfect is 191, in the presence of both companies number jumps to 1493. This explains outlier impact.

On the graph both methods display \(95\%\) of data.

Method 1

# Answer Question 2 here

#Remove rows with NA values using complete.cases()
inc <- inc[complete.cases(inc),]

#Get companies per state
state.company <- inc %>%
  group_by(State) %>% 
  summarise(companiesCount = n()) %>%
  select (State,companiesCount)

#Get third state with most companies
third.state <- state.company %>% 
  arrange(desc(companiesCount)) %>% 
  top_n(3) %>% 
  filter(companiesCount == min(companiesCount)) %>% 
  inner_join(inc, by = c("State" = "State")) %>%
  select (Name, Industry, Revenue, Employees, State)
## Selecting by companiesCount
#Get the state
state <- unique(third.state$State)

#Get mean and standard deviation
third.state.summary <- third.state %>% 
        group_by(Industry) %>% 
        summarise(Employees.avg = mean(Employees), Employees.sd = sd(Employees), Employees.sum = sum(Employees), Company.Count = n())

third.state.summary <- replace(third.state.summary, is.na(third.state.summary), 0)

#Calculate upper bound and lower bound
third.state.summary$lbound <- third.state.summary$Employees.avg - 2*third.state.summary$Employees.sd
third.state.summary$ubound <- third.state.summary$Employees.avg + 2*third.state.summary$Employees.sd

#Eliminate outliers that do not fall with in 2 standard deviations
third.state.data <- third.state %>% 
  inner_join(third.state.summary, by = c("Industry" = "Industry")) %>%
  filter(Employees >= lbound) %>%
  filter(Employees <= ubound) %>% 
  select (Name, Industry, Employees, Revenue, Employees.avg, Employees.sd)

#Display data
third.state.summary %>% 
  kable(format='pandoc', caption = "Employment Summary by Industry - With Outliers", digits=2)
Employment Summary by Industry - With Outliers
Industry Employees.avg Employees.sd Employees.sum Company.Count lbound ubound
Advertising & Marketing 58.44 62.23 3331 57 -66.02 182.90
Business Products & Services 1492.46 6240.71 38804 26 -10988.95 13973.87
Computer Hardware 44.00 0.00 44 1 44.00 44.00
Construction 61.00 79.95 366 6 -98.90 220.90
Consumer Products & Services 626.29 2415.74 10647 17 -4205.19 5457.78
Education 59.86 48.57 838 14 -37.29 157.01
Energy 129.20 105.20 646 5 -81.20 339.60
Engineering 53.50 39.79 214 4 -26.07 133.07
Environmental Services 155.00 134.35 310 2 -113.70 423.70
Financial Services 144.31 151.63 1876 13 -158.95 447.57
Food & Beverage 76.44 117.90 688 9 -159.35 312.24
Government Services 17.00 0.00 17 1 17.00 17.00
Health 81.85 86.63 1064 13 -91.42 255.11
Human Resources 437.55 680.79 4813 11 -924.04 1799.13
Insurance 32.50 24.75 65 2 -17.00 82.00
IT Services 204.09 473.47 8776 43 -742.85 1151.04
Logistics & Transportation 29.50 29.03 118 4 -28.57 87.57
Manufacturing 73.31 89.71 953 13 -106.12 252.73
Media 108.00 176.06 1188 11 -244.11 460.11
Real Estate 18.25 9.71 73 4 -1.17 37.67
Retail 24.79 25.26 347 14 -25.73 75.30
Security 135.00 210.12 540 4 -285.24 555.24
Software 245.92 374.82 3197 13 -503.72 995.56
Telecommunications 95.35 103.17 1621 17 -110.99 301.70
Travel & Hospitality 547.71 835.14 3834 7 -1122.58 2218.00 
#Generate the plot without outliers
ggplot(third.state.data, aes(x = Industry, y = Employees)) +
  geom_bar(stat = "identity") + coord_flip() + ggtitle(paste("Average Employees per Industry In",state,"State -")) + 
  labs(subtitle = "Using Average by Industry - Without Outliers")

According to the graph Business Products & Services industry employs most people. Followed by IT Services, when the graph is generated using employment average of each industry without outliers. The graph shows \(95\%\) of the data.

Method 2

#Get mean and standard deviation
state.avg = mean(third.state$Employees)
state.sd = sd(third.state$Employees)

lbound = state.avg - (2 * state.sd)
ubound = state.avg + (2 * state.sd)

#Eliminate outliers that do not fall with in 2 standard deviations
third.state.data <- third.state %>% 
  filter(Employees >= lbound) %>%
  filter(Employees <= ubound) %>% 
  select (Name, Industry, Employees, Revenue)

#Generate the plot without outliers
ggplot(third.state.data, aes(x = Industry, y = Employees)) +
  geom_bar(stat = "identity") + coord_flip() + ggtitle(paste("Average Employees per Industry In",state,"State"))  + labs(subtitle = "Using Average Employment of the State - Without Outliers")

According to the graph IT Services industry employs most people. Followed by Business Products & Services, when the graph is generated using employment average across the state without outliers. The graph shows \(95\%\) of the data.

Question 3

Now imagine you work for an investor and want to see which industries generate the most revenue per employee. Create a chart that makes this information clear. Once again, the distribution per industry should be shown.

# Answer Question 3 here
#Revenues generated per employee
third.state.revenue <- third.state.data %>% 
  group_by(Industry) %>% 
  summarise(Revenue = sum(Revenue)/sum(Employees))

#Generate the plot
ggplot(third.state.revenue, aes(x = Industry, y = Revenue)) +
  geom_bar(stat = "identity") + scale_y_continuous(labels = scales::comma) + coord_flip() + ggtitle(paste("Average Revenue per Employee per Industry In",state,"State"))

According to the graph Energy industry generates highest revenues per employee. Followed by Logistics & Transportation.