US Public & Private Universities
How expensive are US private schools really?
Joon Hyung Kim
7/20/2020
1) Overview
1-1) Motivation for this project
US colleges are well known for the huge difference in tuition fee between public and private universities. Through this project, it is our goal to visualize
the difference of the tuition fee between public and private universities.
whether there is a correlation between SAT score and tuition fee.
1-2) Challenges
Along the project, we are expecting to face some challenges including
Clean dataset - We will have to check if the dataset is ready to use. For example, if the datatype of tuition fee is in number.
What kind of charts to use - What kind of chart will most be suitable for our goal to show our desired goals.
1-3) Proposed Design
We will be creating three charts.
- Stacked bar chart
- To get the overview of the number of public and private schools in the dataset.
- To find out if the data is not skewed(eg. Much more private schools compared to public schools).
- If there is sufficient number of data for both public and private schools.
- Boxplot
- To figure out if our assumption is correct that public schools have lower tuition compared to that of private schools.
- If correct, do a further analysis whether there is a correlation of SAT score and tuition fee
- If incorrect, it would be a good example of how a thorough analysis on data can prove wrong to people’s common belief that private schools are more expensive than public schools.
- Scatterplot
- On top of the box_plot, to find out if SAT scores and tuition fee have correlations
- If so, how strongly correlated
- We will show a step by step method of creating this plot
2) Data Exploration
2-1) Importing necessary libraries
- Tidyverse
- For data wrangling
- Dplyr
- To avoid the error ‘cannot read rlang’
- Ggplot
- To make charts
library(tidyverse)
library(dplyr)
library(ggplot2)
2-2) Data wrangling
Read csv and assign the information into a varible called ‘college’
college <- read_csv('http://672258.youcanlearnit.net/college.csv')
2-2-1) head(college)
- First checking ‘head(college)’ to get the general picture
head(college)## # A tibble: 6 x 17
## id name city state region highest_degree control gender admission_rate
## <dbl> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <dbl>
## 1 102669 Alas~ Anch~ AK West Graduate Private CoEd 0.421
## 2 101648 Mari~ Mari~ AL South Associate Public CoEd 0.614
## 3 100830 Aubu~ Mont~ AL South Graduate Public CoEd 0.802
## 4 101879 Univ~ Flor~ AL South Graduate Public CoEd 0.679
## 5 100858 Aubu~ Aubu~ AL South Graduate Public CoEd 0.835
## 6 100663 Univ~ Birm~ AL South Graduate Public CoEd 0.857
## # ... with 8 more variables: sat_avg <dbl>, undergrads <dbl>, tuition <dbl>,
## # faculty_salary_avg <dbl>, loan_default_rate <chr>, median_debt <dbl>,
## # lon <dbl>, lat <dbl>
2-2-2) colnames(college)
- Then used ‘colnames(college)’ because there were too many columns(17 in total). After checking the column names, we found out that we need the information we need which is column names
- ‘control’ : whether the school is public or private,
- ‘tuition’ : tuition fee of the school,
- ‘sat_avg’ : SAT score of students entered
colnames(college)## [1] "id" "name" "city"
## [4] "state" "region" "highest_degree"
## [7] "control" "gender" "admission_rate"
## [10] "sat_avg" "undergrads" "tuition"
## [13] "faculty_salary_avg" "loan_default_rate" "median_debt"
## [16] "lon" "lat"
2-2-3) summary(college)
- Trying to figure out the datatype of each entry. For example, if tuition fee is a string, we will first have to change the datatype to float. In this case, we will have to change some ‘character’ in the into ‘factor’
summary(college)## id name city state
## Min. :100654 Length:1269 Length:1269 Length:1269
## 1st Qu.:153250 Class :character Class :character Class :character
## Median :186283 Mode :character Mode :character Mode :character
## Mean :186988
## 3rd Qu.:215284
## Max. :484905
## region highest_degree control gender
## Length:1269 Length:1269 Length:1269 Length:1269
## Class :character Class :character Class :character Class :character
## Mode :character Mode :character Mode :character Mode :character
##
##
##
## admission_rate sat_avg undergrads tuition
## Min. :0.0509 Min. : 720 Min. : 47 Min. : 2732
## 1st Qu.:0.5339 1st Qu.: 973 1st Qu.: 1296 1st Qu.: 8970
## Median :0.6687 Median :1040 Median : 2556 Median :20000
## Mean :0.6501 Mean :1060 Mean : 5629 Mean :21025
## 3rd Qu.:0.7859 3rd Qu.:1120 3rd Qu.: 6715 3rd Qu.:30364
## Max. :1.0000 Max. :1545 Max. :52280 Max. :51008
## faculty_salary_avg loan_default_rate median_debt lon
## Min. : 1451 Length:1269 Min. : 6056 Min. :-157.92
## 1st Qu.: 6191 Class :character 1st Qu.:21250 1st Qu.: -94.17
## Median : 7272 Mode :character Median :24589 Median : -84.89
## Mean : 7656 Mean :23483 Mean : -88.29
## 3rd Qu.: 8671 3rd Qu.:27000 3rd Qu.: -78.63
## Max. :20650 Max. :41000 Max. : -68.59
## lat
## Min. :19.71
## 1st Qu.:35.22
## Median :39.74
## Mean :38.61
## 3rd Qu.:41.81
## Max. :61.22
2-2-3-1) as.factor()
Changing character(string) into factor(float)
college <- college %>%
mutate(state=as.factor(state), region=as.factor(region),
highest_degree=as.factor(highest_degree),
control=as.factor(control), gender=as.factor(gender))
2-2-3-2) mutate()
We also see that loan_default rate is character(string) which is strange because it should be factor(number). Let’s check the dataset to see what the problem. Looks like R read one data is NAN which is a string. Let’s change to N/A.
unique(college$loan_default_rate)## [1] "0.077" "0.136" "0.106" "0.111" "0.045" "0.062" "0.096" "0.007" "0.103"
## [10] "0.063" "0.048" "0.061" "0.078" "0.075" "0.114" "0.102" "0.066" "0.044"
## [19] "0.125" "0.315" "0.156" "0.128" "0.172" "0.187" "0.169" "0.18" "0.171"
## [28] "0.089" "0.149" "0.109" "0.091" "0.086" "0.068" "0.175" "0.064" "0.031"
## [37] "0.247" "0.073" "0.059" "0.07" "0" "0.025" "0.008" "0.06" "0.028"
## [46] "0.029" "0.027" "0.035" "0.052" "0.038" "0.043" "0.036" "0.015" "0.055"
## [55] "0.058" "0.012" "0.03" "0.04" "0.023" "0.069" "0.057" "0.049" "0.032"
## [64] "0.016" "0.092" "0.034" "0.084" "0.022" "0.026" "0.08" "0.024" "0.033"
## [73] "0.019" "NULL" "0.039" "0.041" "0.056" "0.047" "0.046" "0.093" "0.143"
## [82] "0.014" "0.101" "0.006" "0.011" "0.132" "0.042" "0.088" "0.053" "0.021"
## [91] "0.018" "0.119" "0.148" "0.054" "0.129" "0.076" "0.05" "0.02" "0.147"
## [100] "0.065" "0.1" "0.159" "0.124" "0.051" "0.236" "0.182" "0.2" "0.144"
## [109] "0.081" "0.071" "0.017" "0.087" "0.127" "0.184" "0.083" "0.135" "0.157"
## [118] "0.202" "0.204" "0.126" "0.138" "0.037" "0.133" "0.01" "0.072" "0.108"
## [127] "0.134" "0.094" "0.105" "0.067" "0.104" "0.082" "0.123" "0.085" "0.164"
## [136] "0.079" "0.09" "0.095" "0.14" "0.154" "0.074" "0.117" "0.097" "0.131"
## [145] "0.22" "0.116" "0.121" "0.115" "0.158" "0.009" "0.099" "0.013" "0.19"
## [154] "0.179" "0.16" "0.176" "0.004" "0.002" "0.005" "0.152" "0.11" "0.217"
## [163] "0.155" "0.306" "0.142" "0.298" "0.218" "0.137" "0.113" "0.215" "0.197"
## [172] "0.233" "0.174" "0.13" "0.003" "0.107" "0.12" "0.193" "0.118" "0.311"
## [181] "0.167" "0.139" "0.153" "0.192" "0.196" "0.122" "0.259" "0.188" "0.334"
## [190] "0.186" "0.213" "0.098" "0.284" "0.151" "0.222" "0.237" "0.166" "0.23"college <- college %>%
mutate(loan_default_rate=as.numeric(loan_default_rate))## Warning in mask$eval_all_mutate(dots[[i]]): NAs introduced by coercion
3) Charts
3-1) Bar charts
3-1-1) How many schools are there?
Let’s see how many schools there are in the dataset. Looks like there is slightly more number of private universities but that’s okay because in the US there are more private universities than public ones. What’s important is that we have sufficient number of universities in both categories.
ggplot(data=college, aes(fill = control)) +
geom_bar(mapping=aes(x=control)) +
theme(panel.background=element_blank()) +
theme(plot.background=element_blank()) +
theme(panel.grid.major.y=element_line(color="grey")) +
scale_x_discrete(name="School Type") +
scale_y_continuous(name="Number of Schools") +
scale_fill_manual(values=c("pink","skyblue"), guide=guide_legend(title="Institution Type", label.position="bottom", nrow=1, keywidth=2.5))
3-1-2) Where are those schools?
We would want to check if the dataset is coming from every part of the US. If the data is coming from only part of the country, it would be difficult to say that it is an accurate representation of the whole country. Luckily, it seems like the data is more or less coming from each part of the US.
ggplot(data=college) +
geom_bar(mapping=aes(x=region, fill=control)) +
theme(panel.background=element_blank()) +
theme(plot.background=element_blank()) +
theme(panel.grid.major.y=element_line(color="grey")) +
scale_x_discrete(name="Region") +
scale_y_continuous(name="Number of Schools", limits=c(0,500)) +
scale_fill_manual(values=c("pink","skyblue"), guide=guide_legend(title="Institution Type", label.position="bottom", nrow=1, keywidth=2.5))
3-2) Box plot. Tuition fee difference between public & private schools
By looking at the box plots we can clearly see that private schools are more expensive than public schools. As a matter of fact, the difference is so huge that an outlier of a public school will still be lower than the low cost private schools.
ggplot(data=college) +
geom_boxplot(mapping=aes(x=control, y=tuition)) +
theme(panel.background=element_blank()) +
theme(plot.background=element_blank()) +
theme(panel.grid.major.y=element_line(color="grey")) +
scale_x_discrete(name="School Type") +
scale_y_continuous(name="Tuition Fee")
3-3) Histogram. Size of universities
Let’s also check how big the universities are. It seems like most universities are under the size of 20,000
ggplot(data=college) +
theme(panel.background=element_blank()) +
theme(plot.background=element_blank()) +
theme(panel.grid.major.y=element_line(color="grey")) +
geom_histogram(mapping=aes(x=undergrads), bins=15) +
scale_x_continuous(name="Undergraduate Student Number") +
scale_y_continuous(name="Number of Schools", limits = c(0, 600))
4) Final Representation
4-1) Put data into the scatter plot
ggplot(data=college) +
geom_point(mapping=aes(x=tuition, y=sat_avg))
4-2) Distinguish the private and public schools by color
ggplot(data=college) +
geom_point(mapping=aes(x=tuition, y=sat_avg, color=control))
4-3) Change the size of the circles differently in reference to schoool size(number of undergraduate students)
ggplot(data=college) +
geom_point(mapping=aes(x=tuition, y=sat_avg, color=control, size=undergrads))
4-4) Put some opacity in order to see the circles better
ggplot(data=college) +
geom_point(mapping=aes(x=tuition, y=sat_avg, color=control, size=undergrads), alpha=1/2)
4-5) Add annotation ‘mean SAT score’ and ‘mean tuition fee’
ggplot(data=college) +
geom_point(mapping=aes(x=tuition, y=sat_avg, color=control, size=undergrads), alpha=1/2) +
annotate("text", label="Elite Privates", x=45000,y=1500) +
geom_hline(yintercept=mean(college$sat_avg)) +
annotate("text", label="Mean SAT", x=50000, y=mean(college$sat_avg)-20) +
geom_vline(xintercept=mean(college$tuition)) +
annotate("text", label="Mean Tuition", x=mean(college$tuition)+5000, y=650)
4-6) Change legend and axis name
ggplot(data=college, xName = "Tuition Fee", yName = "SAT Average") +
geom_point(mapping=aes(x=tuition, y=sat_avg, color=control, size=undergrads), alpha=1/2) +
annotate("text", label="Elite Privates", x=45000,y=1500) +
geom_hline(yintercept=mean(college$sat_avg)) +
annotate("text", label="Mean SAT", x=50000, y=mean(college$sat_avg)-20) +
geom_vline(xintercept=mean(college$tuition)) +
annotate("text", label="Mean Tuition", x=mean(college$tuition)+5000, y=650) +
labs(x = "Tuition Fee", y = "SAT Average", color="School Type", size = "Number of Undergraduates")
4-7) Add title to the plot
ggplot(data=college) +
geom_point(mapping=aes(x=tuition, y=sat_avg, color=control, size=undergrads), alpha=1/2) +
annotate("text", label="Elite Privates", x=45000,y=1500) +
geom_hline(yintercept=mean(college$sat_avg)) +
annotate("text", label="Mean SAT", x=47500, y=mean(college$sat_avg)-15) +
geom_vline(xintercept=mean(college$tuition)) +
annotate("text", label="Mean Tuition", x=mean(college$tuition)+5500, y=700) +
labs(x = "Tuition Fee", y = "SAT Average", color="School Type", size = "Number of Undergraduates") +
ggtitle("More colleges are in the southern U.S. than any other region.", subtitle="Source: U.S. Department of Education")
4-8) Add regression line
We can see the upward trending line. So we could safely say that there is a positive correlation between SAT scores and tuition fee.
ggplot(data=college, mapping=aes(x=tuition, y=sat_avg, color=control)) +
geom_point(mapping=aes(x=tuition, y=sat_avg, color=control, size=undergrads), alpha=1/2) +
annotate("text", label="Elite Privates", x=45000,y=1500) +
geom_hline(yintercept=mean(college$sat_avg)) +
annotate("text", label="Mean SAT", x=47500, y=mean(college$sat_avg)-15) +
geom_vline(xintercept=mean(college$tuition)) +
annotate("text", label="Mean Tuition", x=mean(college$tuition)+5500, y=700) +
labs(x = "Tuition Fee", y = "SAT Average", color="School Type", size = "Number of Undergraduates") +
ggtitle("More colleges are in the southern U.S. than any other region.", subtitle="Source: U.S. Department of Education") +
geom_smooth()## `geom_smooth()` using method = 'loess' and formula 'y ~ x'
5) Last Comments
Major findings:
- Private Universities are much more expensive than public universities in the US.
- Expensive public schools will still be considered more inexpensive than the most affordable private schools.
- Private universities are expensive because it operates almost solely upon students' tuition compared to public schools that receive support from federal and local government.- A positive correlation between SAT score and Tuition fee.
- The higher the SAT score, the tuition fee also goes up.To answer to the question, ’Is the high priced private universities worth it?", one of the things we could do is to calculate the ROI, Return of Investment.
In the past, despite the high price, people believed the high cost of private universities were still worth it. However, it is increasing becoming doubtful these days as college degrees are becoming too common.