Abstract

Our dashboard is a tool that utilizes data from the US Department of Education’s College Scorecard. Its purpose is to provide a comprehensive view of higher education institutions across the United States. The dashboard presents key metrics such as graduation rates, median earnings, student demographics, and institutional characteristics in an integrated and user-friendly dashboard.

By consolidating this diverse set of data points, our dashboard enables users to uncover trends, patterns, and correlations. This information can be used for strategic planning, policy development, and program evaluation. In short, our dashboard offers a valuable resource for anyone interested in gaining insights into the higher education landscape in the United States.

Data Introduction

The data we are using is from the US Department of Education’s College Scorecard. We are using the Institution level data set. The data set includes information for each institution 1996-97 through 2021-22. It includes information on institutional characteristics, enrollment, student aid, costs, and student outcomes.The data comes from federal reporting from institutions, data on federal financial aid, and tax information. With most of the data coming from the Integrated Post-secondary Education Data System (IPEDS). The data is categorized under data area 4, Education.

The full dataset can be accessed through the following link

More Information about the data can be found here

The work was split up in the following ways:

Divya

• Multiple Regression
• Ridge Regression
• Naive Bayes

Grace

• Loess Regression
• Knn Classification
• Classification using Logistic Regression

Data Dictionary

Methodology

I will be using multiple linear regression to find the beset combinations of predictors and testing its accuracy. I will be using “Retention” as my response variable and the following predictor variables:

• Admission rate (ADM_RATE)
• Average SAT score (SAT_AVG)
• Completion rate (C100_4)
• Undergraduate population (UGDS)
• Undergraduate student to instructional faculty ratio (STUFACR)

Data Table

Stepwise varible selection

After performing stepwise regression analysis, it was found that SAT average, on-time completion rate, and undergraduate population have a statistically significant association with retention rate. However, Admission Rates and Student Faculty ratio were found to be less significant and were subsequently removed. The best model is RET_FT4 ~ SAT_AVG + C100_4 + UGDS.

Normality Assumption

Based on the plot, we can conclude that although a significant number of data points are aligned with the QQline, the tails of the plot exhibit some non-normal behavior. This suggests that the normality assumption is violated, which is further supported by the Shapiro-Wilk test. The test indicates a small p-value, providing evidence against the null hypothesis. Therefore, we can conclude that normality is violated.

QQ Plot

Shapiro test

Independence Assumption

According to the Durbin-Watson test results, the D-W statistic is less than 2, indicating a positive autocorrelation. Furthermore, the p-value is less than 0.05, providing evidence to reject the null hypothesis, and thereby violating the independence assumption.

Durbin Watson Test

Constant Variance Assumption

A Levene test was not conducted on the linear regression. Based on the results from the Breuch-Pagan test, we can conclude that the constant variance assumption is violated since the p-value is less than 0.05.

Breusch-Pagan Test

Linearity

The scatterplot matrix below shows the relationship between key variables in our analysis visually. Our dataset includes scatterplots for “RET_FT4,” “SAT_AVG,” “C100_4,” and “UGDS” variables.

Each cell in the upper triangle of the matrix presents a smoothed scatterplot between two variables. Moreover, the lower triangle of the matrix displays correlation coefficients. Positive correlations (values close to 1) suggest that if one variable increases, the other tends to increase as well. On the other hand, negative correlations (values close to -1) indicate an inverse relationship.

The second plot displays a cone shape, indicating heteroscedasticity in the model’s Standardized Residuals vs Fitted Values plot.

Scatterplot Matrix

Residuals vs Fitted Values

Outliers and influential points

According to the plot, we can observe that there are three substantial outliers, which are 458, 1857, and 5979. However, they were not eliminated from the dataset. Additionally, based on the table, we can identify six influential points that must be considered.

Outliers

Influential points

Summary Statistics

According to the low p-value for the F statistic, we can conclude that the model is statistically significant. Furthermore, all the predictors have a significant association with the response (retention rate).

The multiple \(R^2\) value of 0.66 indicates that 66% of the variability in retention rate of first years is explained by the predictors. However, there may be some additional predictors that were not considered during the initial model building process, which could account for some of the variability.

The adjusted \(R^2\) value of 0.665 adjusts the multiple \(R^2\) value to account for the number of predictors in the model. Since the two R^2 values are very close, we can conclude that the inclusion of the predictors has not substantially increased the multiple \(R^2\) value.

Summary Statistics

Scatter Plots of predictors against repsonse variable

Based on the plots provided, we can observe that the number of undergraduate students in most universities tends to be less than 20,000. However, some universities have significantly larger undergraduate populations, making the distribution of the undergraduate population right-skewed. Moving on to the SAT scores, the average score falls mostly between 1000 and 1200. This is not surprising since the majority of students score between this range. While there are fewer students who score less than 1000 or above 1500 (although there are more above 1500 than below 1000), not many students tend to get scores below 1000.

Correlation coefficients

There exists a significant positive relationship between the average SAT scores and student retention rates. This suggests that colleges and universities with higher SAT scores typically have higher retention rates for first-time, full-time students.

Similarly, retention rates also have a strong positive correlation with on-time completion rates.

Retention rates have a moderate positive correlation with the undergraduate population. Furthermore, there is a moderate negative correlation (-0.406) between retention rates and admission rates (ADM_RATE). This indicates that institutions with lower admission rates tend to have higher retention rates for first-time, full-time students.

The other correlations between variables also provide valuable insights into their relationships. For example, average SAT scores exhibit a negative correlation with admission rates, which implies that colleges and universities with higher average SAT scores typically have lower admission rates. This correlation makes sense.

Correlation coefficients matrix

VIF points

Since the VIF scores are all below 10, this suggests that multicollinearity is not a significant concern among the predictor variables in the model. For SAT_AVG and C100_4 they both have moderate correlation but not high enough to cause any concern.

VIF

Predictions

I attempted to predict the retention rate for a fictitious school. I assigned values to the predictor variables and used the predict function to obtain a retention rate for my multiple linear regression model. Based on this prediction, it appears that a school with an average SAT score of 1530, an undergraduate population of 15,000, and an on-time completion rate of 0.27 would have a 90% retention rate.

Variables for model

I will be using the completion rate as the response variable in my analysis. This variable indicates the percentage of first-time, full-time students who complete their degree within the expected time of completion (i.e., 100% completion rate). The predictor variables that I will be using in my analysis are as follows:

• In-state tuition/fees (TUITIONFEE_IN)
• Number of undergrad students (UGDS)
• Admission rate (ADM_RATE)
• Undergraduate student to instructional faculty ratio (STUFACR)
• Average SAT score (SAT_AVG)

Data Table

Fitting the model

Var1 - This column represents the names of variables or attributes

Var2 - This column represents the types or classes of the variables or attributes

Freq - This column represents the frequencies or counts associated with each variable or attribute

For example: a0 has a length of 100, belongs to the class ‘numeric’, and has a mode of ‘numeric’.

We can conclude from this that there are variables with different lengths, which means varying number of observations. The dataset contains variables of different classes, such as numeric, logical, or call. The variables have different storage modes, indicating how they are stored in memory. Finally, the frequencies of different attributes provide insights into the variable characteristic distribution in the dataset.

Model

cv_fit

By using the cv_glmnet() function, we can evaluate how well each model generalizes through cross-validation. The output plot displays a curve, where the lowest point indicates the optimal lambda value. In this case, the optimal lambda value is 0.01, which is the logarithmic value that minimizes the error in cross-validation. This value suggests that a relatively low amount of regularization is being applied in the model.

Lambda

Extracted fitted model

Making predictions

According to the analysis, the ridge regression model is able to explain approximately 68% of the variability in the completion rates for first-time students at 4-year institutions. This indicates that the model is fairly effective in predicting completion rates using the given predictors.

Model Evaluation

Methodology

I will be using LOESS fit to analyze this. I will be using the “Mean earnings of students working and not enrolled 10 years after entry” variable as my response and “Retention at 4-year Instiution” as my predictor.

In addition, I will splitting the data by region and will using regions 1-8 which eliminates service schools and the outlying areas.

• 1: New England (CT, ME, MA, NH, RI, VT)
• 2: Mid East (DE, DC, MD, NJ, NY, PA)
• 3: Great Lakes (IL, IN, MI, OH, WI)
• 4: Plains (IA, KS, MN, MO, NE, ND, SD)
• 5: Southeast (AL, AR, FL, GA, KY, LA, MS, NC, SC, TN, VA, WV)
• 6: Southwest (AZ, NM, OK, TX)
• 7: Rocky Mountains (CO, ID, MT, UT, WY)
• 8: Far West (AK, CA, HI, NV, OR, WA)

For each region I will perform LOESS for span values from 0.25 to 0.75 in steps of 0.05 and calculate the RSE values for each iteration. I will then choose the top three most accurate LOESS and graph them.

Residual Standard Error (RSE) Values for LOESS Regression

Span 0.25

Span 0.30

Span 0.35

Residual Standard Error (RSE) Values for LOESS Regression

Span 0.25

Span 0.30

Span 0.35

Residual Standard Error (RSE) Values for LOESS Regression

Span 0.25

Span 0.30

Span 0.35

Residual Standard Error (RSE) Values for LOESS Regression

Span 0.25

Span 0.30

Span 0.35

Residual Standard Error (RSE) Values for LOESS Regression

Span 0.25

Span 0.30

Span 0.35

Residual Standard Error (RSE) Values for LOESS Regression

Span 0.25

Span 0.30

Span 0.35

Residual Standard Error (RSE) Values for LOESS Regression

Span 0.25

Span 0.30

Span 0.35

Residual Standard Error (RSE) Values for LOESS Regression

Span 0.25

Span 0.30

Span 0.35

Conclusion

In general, the LOESS regression analysis indicates that most regions have a positive exponential growth with a positive y-intercept. This suggests that as retention rates increase in a school, the median earnings after graduation are expected to be higher. The only region that did not have a clear exponential curve was region 7, which could be due to the significantly lower number of schools in this region. Nonetheless, there is still a general positive trend between retention and median earnings after graduation.

For all regions, the most accurate span observed was 0.25, followed by 0.3 and 0.35. Based on the graphs for all regions, the LOESS regression lines with a span of 0.25 are the best fit, as the lines for the other two spans tend to over fit the data.

Which states have the most colleges?

Based on the bar graph, California has the most schools, with around 700. New York and Texas have the next most schools, with 450 and 415 schools, respectively. I will use the data from the top five states for my kNN classification, which will be California, New York, Texas, Florida, and Pennsylvania.

Number of Schools by State

Variables for my kNN model

I will be using the following variables for my kNN model:

• Number of undergraduate students (UGDS)
• Undergraduate student to instructional faculty ratio (STUFACR)
• Average faculty salary per month (AVGFACSAL)
• Retention (RET_FT4)
• Number of Branches (NUMBRANCH)
• Percent of all undergraduate students receiving a federal student loan (PCTFLOAN)

I will be using “Control of Institution” as my response variable. This variable indicates whether an institution is public (1), private non-profit (2), or private for-profit (3). For information about the other variables, please refer to the data dictionary located in the Project Overview tab.

Head of Dataset

Scatter Plots of predictors against repsonse variable

The plots presented below provide a comparison of each predictor with the response variable. While there is a significant overlap in the graphs, it is apparent that most private for-profit colleges tend to have a smaller number of undergraduate students, less than 10000, when compared to public and private non-profit colleges. Moreover, the faculty members at public schools receive an average salary that exceeds $5000 per month, whereas the faculty members at private schools can earn less than $5000 per month. Private for-profit faculty members earn between $2500 and $10000.

Splitting data into training and testing

I split the dataset into a 70% training and 30% testing split. Then, I plotted the training data and the testing points (represented by diamonds) on separate graphs. Overall, all graphs revealed that the two datasets exhibit similar patterns, indicating that the model predictions will align with the actual values.

Running kNN for several k values

The graph below displays the accuracy of the model for different k-values. According to the graph, the k-value of 5 resulted in the highest accuracy of 80%. As the k-values increase, the accuracy of the model decreases. To further analyze the performance of the kNN model, I will execute a confusion matrix for the k-value of 5.

kNN Accuracy by k Value

Confusion matrix for k = 5

This is the confusion matrix for k=5, which represents the most accurate kNN with an accuracy rate of approximately 80%. There were 35 correct predictions out of 44 actual instances for Public colleges, 104 correct predictions out of 129 for Private non-profit colleges, and 10 correct predictions out of 13 instances for Private for-profit colleges.

However, there were 7 instances where Public colleges were incorrectly classified as Private Nonprofit, and 13 instances where Private Nonprofit colleges were incorrectly classified as Public. These misclassifications suggest some overlap in the features or characteristics of these classes. The classifier also struggled with Private For-Profit, misclassifying 3 instances as Private Nonprofit, but none as Public.

It is essential to note that a smaller number of instances for Private For-Profit colleges may have contributed to the lower accuracy rate for this class.

Confusion Matrix

Geographic Distribution of Schools by Region

The plot shows the geographic distribution of Schools by the 10 regions defined by the College scorecard dataset. The regions are as follows:

• 0: U.S. Service Schools
• 1: New England (CT, ME, MA, NH, RI, VT)
• 2: Mid East (DE, DC, MD, NJ, NY, PA)
• 3: Great Lakes (IL, IN, MI, OH, WI)
• 4: Plains (IA, KS, MN, MO, NE, ND, SD)
• 5: Southeast (AL, AR, FL, GA, KY, LA, MS, NC, SC, TN, VA, WV)
• 6: Southwest (AZ, NM, OK, TX)
• 7: Rocky Mountains (CO, ID, MT, UT, WY)
• 8: Far West (AK, CA, HI, NV, OR, WA)
• 9: Outlying Areas (AS, FM, GU, MH, MP, PR, PW, VI)

Geographic Distribution of Schools

Variables in model

I will be using the following variables as predictors in my Naive Bayes:

• Admission rate (ADM_RATE)
• Average SAT score (SAT_AVG)
• Open admission (OPENADMP)
• Graduate population (GRADS)
• On-time Completion in a 4 year institute (C100_4)
• Undergraduate population (UGDS)
• State (STABBR)

I will be using the term ‘Region’ as my response variable. There are 10 regions, but I will only be using 8 of them. Specifically, I will be excluding U.S. Service Schools (0) and Outlying Areas (9). The remaining regions that I will be using are as follows:

• 1: New England (CT, ME, MA, NH, RI, VT)
• 2: Mid East (DE, DC, MD, NJ, NY, PA)
• 3: Great Lakes (IL, IN, MI, OH, WI)
• 4: Plains (IA, KS, MN, MO, NE, ND, SD)
• 5: Southeast (AL, AR, FL, GA, KY, LA, MS, NC, SC, TN, VA, WV)
• 6: Southwest (AZ, NM, OK, TX)
• 7: Rocky Mountains (CO, ID, MT, UT, WY)
• 8: Far West (AK, CA, HI, NV, OR, WA)

Head of data table

Naive Bayes

I split the dataset into 70% training and 30% testing for Naive Bayes. The model was 95% accurate and produced a confusion matrix. Notable correct predictions include 15 instances of class 1, 39 of region 2, 48 of region 3, 35 of region 4, 70 of region 5, 20 of region 6, 8 of region 7, and 16 of region 8. However, there were instances of misclassification, such as 4 instances of region 1 falsely predicted as region 2, 1 instance of region 5 as region 3, and 1 instance of region 8 as region 7. These suggest the need for further investigation and refinement of the model’s features or training approach.

Confusion matrix

What Precentage of Colleges offer Bachelor’s degree in engineering?

The following pie chart displays the percentage of colleges in the United States that provide a Bachelor’s degree in engineering. The chart shows nearly an equal number of colleges that provide and don’t provide this degree. Based on this observation, I am interested in knowing whether logistic regression can be used to predict with accuracy if a college offers this type of degree.

Distribution of engineering degrees

Varibles for the model

I will be using the following variables for my regression:

• Admission Rate (ADM_RATE)
• Average SAT Scores (SAT_AVG)
• Total share of enrollment of undergraduate students who are white (UGDS_WHITE)
• Total share of enrollment of undergraduate students who are Asian (UGDS_ASIAN)
• Total share of enrollment of undergraduate students who are Black (UGDS_BLACK)
• Total share of enrollment of undergraduate students who are Hispanic (UGDS_HISP)
• Share of full time faculty that are women (IRPS_WOMEN)
• Bachelor’s degree in Biological And Biomedical Sciences (CIP26BACHL)

I will be using the “Bachelor’s degree in Engineering” variable as my response. This variable indicates whether a college offers a Bachelor’s degree in Engineering, with 0 indicating no, 1 indicating yes, and 2 indicating that the program is offered through an exclusively distance-education program. I will be eliminating any schools with a value of 2 from the data set, so I will just focus on classifying whether a school simply offers the program or not.

The head of the data table that will be used for the regression is displayed below.

Head of data table

Finding the right model

I used stepwise variable selection to find the best model and the best model is CIP14BACHL ~ SAT_AVG + CIP26BACHL + UGDS_ASIAN + IRPS_WOMEN.

The coefficients of the best regression model is displayed below.

Model building

Plotting the predictors

The following plots plot the numerical predictors against the response varible catgroized by the catefgorial preictor of offering Bachelor’s degree in Biological And Biomedical Sciences.

Making predictions using testing and training data

We have evaluated the performance of our model on both training and testing data. On the training data, our model has an accuracy of 68% and a misclassification rate of approximately 31.79%. The confusion matrix reveals that the model made 285 correct predictions of “Yes” and 202 correct predictions of “No.” However, the model also misclassified 125 instances as “Yes” and 102 instances as “No.”

On the testing data, our model’s accuracy is slightly lower at 63%, with a misclassification rate of approximately 36.93%. The confusion matrix shows that there were 65 instances incorrectly predicted as “Yes” and 48 instances incorrectly predicted as “No.” Overall, the model’s performance on the testing data is not as good as the training data, with a higher rate of misclassification.

To further assess the model’s effectiveness, we performed 10-fold cross-validation. The average misclassification rate across the 10 folds was approximately 20%. These results suggest that the logistic regression model performs well and is effective with new data. Furthermore, these results indicate that the model’s performance is stable and consistent across different subsets of the dataset.

Confusion matrix for training data

Confusion matrix for testing data