The Graduate Programs Office has recently restructured and has developed tools, personnel, and strategies to increase retention, persistence, and graduation among students in the business school’s graduate programs. A significant part of this restructuring has been to professionalize advising services to its students by providing quality, personal advising to each graduate student. This level of focused, professional advising is critical in ensuring the best possible outcomes for students (Jones 2018). One of the ways to measure the effectiveness of these advising strategies is to survey students about their level of satisfaction with their advising experience. Such a survey was conducted of recent graduates from 2021-2022. The results of that survey are analysed herein to determine how effective the advising has been, to assess overall satisfaction with the advising experiences, and to understand student sentiment and suggestions for improving the process of advising going forward.
Project Setup
Installing package into '/cloud/lib/x86_64-pc-linux-gnu-library/4.3'
(as 'lib' is unspecified)
Installing package into '/cloud/lib/x86_64-pc-linux-gnu-library/4.3'
(as 'lib' is unspecified)
Installing package into '/cloud/lib/x86_64-pc-linux-gnu-library/4.3'
(as 'lib' is unspecified)
Installing package into '/cloud/lib/x86_64-pc-linux-gnu-library/4.3'
(as 'lib' is unspecified)
stm v1.3.7 successfully loaded. See ?stm for help.
Papers, resources, and other materials at structuraltopicmodel.com
Attaching package: 'dplyr'
The following objects are masked from 'package:stats':
filter, lag
The following objects are masked from 'package:base':
intersect, setdiff, setequal, union
── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ forcats 1.0.0 ✔ readr 2.1.5
✔ ggplot2 3.4.4 ✔ tibble 3.2.1
✔ lubridate 1.9.3 ✔ tidyr 1.3.1
✔ purrr 1.0.2
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag() masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
Loading required package: NLP
Attaching package: 'NLP'
The following object is masked from 'package:ggplot2':
annotate
Loading required package: viridisLite
Note: All identifiable student and advisor information contained in the original survey data was removed to protect individual privacy. The data uploaded for this analysis does not contain personal information such as student name or identification number.
Analysis Questions
Question 1: What type of interaction(s) did students have with their advisor?
Question 2: What was the overall level of satisfaction with the advising experience and how likely are students to recommend the program to others?
Question 3: Why are students likely to recommend this program based on their advising experience?
Analysis of Question 1
Question 1: What type of interaction(s) did students have with their advisor?
Students have various needs when it comes to advising, and one of the most important steps in building a relationship with students is to welcome them to the program and answer their initial questions. Other types of appointments indicate a level of engagement with students including making changes to their degree plan, academic advising, and engagement with campus resources. An additional component of a professional advisor’s day-to-day work is to help with recruiting new students and helping to remove barriers to during the admissions process.
In this survey, there were 225 responses. While this does not fully capture the day-to-day interactions with all students, it does provide a snapshot of those engagement opportunities.
Appt_type <- Advisor_Statisfaction_Survey$"Appointment Type"Appointment_list <-strsplit(Appt_type, ", ", fixed =TRUE)Appointment_count <-table(unlist(Appointment_list))Appointment_counts_df <-as.data.frame(Appointment_count)names(Appointment_counts_df) <-c("Appointment", "Count")library(ggplot2)ggplot(Appointment_counts_df, aes(x = Appointment, y = Count, fill = Appointment)) +geom_bar(stat ="identity") +labs(title ="Survey Results by Appointment Type", x ="Appointment", y ="Count") +theme(axis.text.x =element_text(angle =90, hjust = .5)) +scale_fill_manual(values =c("Welcome Call"="skyblue", "Advising"="orange", "Email"="green", "Prospective Student"="red", "Other"="blue"))
Analysis - From this chart, we can see that students most frequently engage with advisors for advising. This interaction could be as simple as asking a question about a class or asking to take a different course in a specific semester. The Welcome Call is significant as a proportion of appointments for students. This analysis is consistent with research that discusses the importance of forming relationships with students throughout their course of study and the impact on their participation in and response to alumni surveys [McDonald and Woodard (2021)](Drummond Hays 2000).
Analysis of Question 2
Question 2: What was the overall level of satisfaction with the advising experience and how likely are students to recommend the program to others?
This question was asked of students regarding their overall level of satisfaction with the advising experience and with their individual interactions with their assigned advisor. This satisfaction level was measured using the Likert Scale
extremely_satisfied <- Advisor_Statisfaction_Survey$'Satisfaction_Level'extremely_satisfied <-as.character(extremely_satisfied)satisfaction_list <-strsplit(extremely_satisfied, ", ", fixed =TRUE)satisfaction_levels <-c("Extremely satisfied", "Somewhat satisfied", "Neither satisfied nor dissatisfied", "Somewhat dissatisfied", "Extremely dissatisfied")satisfaction_counts <-table(unlist(satisfaction_list))satisfaction_counts_df <-as.data.frame(satisfaction_counts)names(satisfaction_counts_df) <-c("Satisfaction", "Count")library(ggplot2)ggplot(satisfaction_counts_df, aes(x = Satisfaction, y = Count, fill = Satisfaction)) +geom_bar(stat ="identity") +labs(title ="Survey Results by Level of Satisfaction", x ="Satisfaction", y ="Count") +theme(axis.text.x =element_text(angle =0, hjust =-5)) +scale_fill_manual(values =c("Extremely satisfied"="skyblue","Somewhat satisfied"="orange","Neither satisfied nor dissatisfied"="green","Somewhat dissatisfied"="red","Extremely dissatisfied"="purple"))
As clearly depicted in this chart, students are overwhelmingly “extremely satisfied” with the advising experience. However, one measure of satisfaction, is the student’s willingness, or eagerness, to recommend the program or service to others. In this case, the survey also asked that question as well.
Will_Recommend <- Advisor_Statisfaction_Survey$Will_Recommendlibrary(dplyr)recommend_counts <-table(Will_Recommend)recommend_counts_df <-as.data.frame(recommend_counts)names(recommend_counts_df) <-c("Rating", "Frequency")library(ggplot2)ggplot(recommend_counts_df, aes(x = Rating, y = Frequency)) +geom_bar(stat ="identity", fill ="skyblue") +labs(title ="Distribution of Recommendation Ratings", x ="Rating", y ="Frequency") +theme_minimal()
From this chart, we can see a distinct parallel between the overwhelming level of student satisfaction and the likelihood that the student will recommend the program to others. This finding is consistent with research on customer satisfaction ratings and their willingness to recommend a product or service (김현철 and Choi Byung Hoon 2014; Franky and Yanuar Rahmat Syah 2023; 서영수 and Seung Sin Lee 2014). In this case, we can see that students who are pleased with their experience in their learning environment are both extremely satisfied with that experience as well as being overwhelmingly willing to recommend the program to prospective students which is an important part of the recruiting process (Dennis 2020).
Analysis of Question 3
Question 3: Why are students likely to recommend this program based on their advising experience?
To analyze why students are likely to recommend our programs to others, we analyzed the open-form responses to the question that followed “How likely are you to recommend this program?” which was “Please explain why you provided this rating”. We used topic modeling to ascertain whether certain topics or points of concern were evident in the student’s responses in order to be able to identify areas where advising or program methods could be adjusted to better suit student needs.
# A tibble: 937 × 13
StartDate EndDate Status IPAddress Progress
<dttm> <dttm> <chr> <chr> <dbl>
1 2021-02-01 11:55:46 2021-02-01 11:56:56 IP Address 208.69.133.69 100
2 2021-02-03 18:32:32 2021-02-03 18:36:29 IP Address 134.102.11.231 20
3 2021-02-24 09:06:04 2021-02-24 09:06:52 IP Address 104.129.205.14 100
4 2021-03-08 20:24:34 2021-03-08 20:27:21 IP Address 45.37.57.111 100
5 2021-03-15 12:39:09 2021-03-15 12:45:27 IP Address 98.24.235.71 100
6 2021-03-15 12:39:09 2021-03-15 12:45:27 IP Address 98.24.235.71 100
7 2021-03-15 12:39:09 2021-03-15 12:45:27 IP Address 98.24.235.71 100
8 2021-03-15 12:39:09 2021-03-15 12:45:27 IP Address 98.24.235.71 100
9 2021-03-15 12:39:09 2021-03-15 12:45:27 IP Address 98.24.235.71 100
10 2021-03-15 12:39:09 2021-03-15 12:45:27 IP Address 98.24.235.71 100
# ℹ 927 more rows
# ℹ 8 more variables: `Duration (in seconds)` <dbl>, Finished <chr>,
# RecordedDate <dttm>, `Appointment Type` <chr>, Satisfaction_Level <chr>,
# Will_Recommend <dbl>, Why_Recommend <chr>, word <chr>
# A tibble: 488 × 2
word n
<chr> <int>
1 <NA> 175
2 advisor 18
3 lot 14
4 bangladesh 11
5 helpful 11
6 program 10
7 government 8
8 people 8
9 questions 7
10 ashley 6
# ℹ 478 more rows
Building corpus...
Converting to Lower Case...
Removing punctuation...
Removing stopwords...
Removing numbers...
Stemming...
Creating Output...
# A tibble: 1,196 × 14
StartDate EndDate Status IPAddress Progress
<dttm> <dttm> <chr> <chr> <dbl>
1 2021-02-01 11:55:46 2021-02-01 11:56:56 IP Address 208.69.133.69 100
2 2021-02-01 11:55:46 2021-02-01 11:56:56 IP Address 208.69.133.69 100
3 2021-02-01 11:55:46 2021-02-01 11:56:56 IP Address 208.69.133.69 100
4 2021-02-01 11:55:46 2021-02-01 11:56:56 IP Address 208.69.133.69 100
5 2021-02-03 18:32:32 2021-02-03 18:36:29 IP Address 134.102.11.231 20
6 2021-02-24 09:06:04 2021-02-24 09:06:52 IP Address 104.129.205.14 100
7 2021-03-08 20:24:34 2021-03-08 20:27:21 IP Address 45.37.57.111 100
8 2021-03-08 20:24:34 2021-03-08 20:27:21 IP Address 45.37.57.111 100
9 2021-03-08 20:24:34 2021-03-08 20:27:21 IP Address 45.37.57.111 100
10 2021-03-08 20:24:34 2021-03-08 20:27:21 IP Address 45.37.57.111 100
# ℹ 1,186 more rows
# ℹ 9 more variables: `Duration (in seconds)` <dbl>, Finished <chr>,
# RecordedDate <dttm>, `Appointment Type` <chr>, Satisfaction_Level <chr>,
# Will_Recommend <dbl>, Additional_Comments <chr>, word <chr>, stem <chr>
[1] 51
[1] 134
A LDA_VEM topic model with 15 topics.
A topic model with 20 topics, 133 documents and a 525 word dictionary.
# A tibble: 1,965 × 3
document topic gamma
<chr> <int> <dbl>
1 A very detailed session 1 0.0386
2 All of my interactions with advisor have been prompt and helpfu… 1 0.0386
3 All the advisors were great and really wanted to help! 1 0.0109
4 Always have a positive interaction with my advisor Tom. Respond… 1 0.0386
5 Always super communicative. Answers questions thoroughly and pr… 1 0.0386
6 Answered all questions with thorough information in a timely ma… 1 0.0386
7 As my advisor, Sarah Smith has always been responsive to my que… 1 0.0109
8 Ashley Ess is amazing and was so great to work with. She is wha… 1 0.0271
9 Ashley Ess is an amazing advisor. She is prompt, thorough, but … 1 0.0124
10 Ashley has been great. She's responsive, always connects me wit… 1 0.0386
# ℹ 1,955 more rows
Warning: `cols` is now required when using `unnest()`.
ℹ Please use `cols = c(terms)`.
Topic 2:
Always have a positive interaction with my advisor Tom. Responds to emails promptly and always initiates contact with each new class that begins to see how things are going. Always appreciate the updates I receive as well regarding my plan of study.
Prompt answers and information, had all information and instructions readily available even if the question was obscure.
The reason I selected the responses because it was the best to describe my interaction.
Always super communicative. Answers questions thoroughly and promptly.
Topic 16:
Their response gave made me feel proud to be accepted and excited to begin class. The congratulatory response seemed genuine.
Emma did an outstanding job explaining my plan of study and how registration will work.
When I talk about the MBA program, the first thing that I mention is how well coordinated it is.
My advisor has been extremely condecescending in a recent email when I have requested certain professors during my MBA program.
Dr. Deb Lucas is amazing and always willing to assist with any issue.
My advisor Ashley was incredibly helpful and knowledgeable during my appointment. I feel much more confident moving forward in my program.
Emma was informative and kind! A joy to talk to
Topic 3:
The level of service has taken a significant hit with staff changes. Previously, I felt informed, welcomed to ask questions, and was offered information that would generally be useful. It feels mostly like self service lately, with general updates and no comprehensive information about plan of work, changes made, what's coming next... I have no idea what happens next semester since I'm on target to graduate, etc. Very much miss my former advisor.
Joyce has been great with setting up my schedule and responding to my questions. Really alleviates a lot of the hassle with trying to properly plan out my degree.
I asked for specific info regarding a topic and received exactly that
Joyce is knowledgable about the program and friendly. I am not entirely sure of the range of advising services she might offer though.
Great Customer service
Conclusions
Overall, the analysis of student survey responses shows that students are extraordinarily satisfied with their experiences in our programs. Their experiences interacting with advisors has proven helpful in increasing student engagement, retention, persistence, and completion.
References
Dennis, Marguerite J. 2020. “The Role of Alumni in International Recruiting.”Recruiting & Retaining Adult Learners 23 (3): 1–4. https://doi.org/10.1002/nsr.30670.
Drummond Hays, Sarah. 2000. “Facilitating Master’s Student Success: A Quantitative Examination of Student Perspectives on Advising.”https://doi.org/10.15760/etd.1502.
Franky, Franky, and Tantri Yanuar Rahmat Syah. 2023. “The Effect of Customer Experience, Customer Satisfaction, and Customer Loyalty on Brand Power and Willingness to Pay a Price Premium.”Quantitative Economics and Management Studies 4 (3): 437–52. https://doi.org/10.35877/454ri.qems1639.
Jones, Kyle M. L. 2018. “Advising the Whole Student: eAdvising Analytics and the Contextual Suppression of Advisor Values.”Education and Information Technologies 24 (1): 437–58. https://doi.org/10.1007/s10639-018-9781-8.
McDonald, Courtney, and Tracey L Woodard. 2021. “Criminal Justice Student and Alumni Perspectives on Advising.”Journal of Criminal Justice Education 32 (2): 186–200. https://doi.org/10.1080/10511253.2021.1889629.
김현철, and Choi Byung Hoon. 2014. “Empirical Studies on How Service Quality of RIPC Affects Customer Satisfaction, Revisit Intention and Recommend Intention.”Public Policy Review 28 (3): 349–77. https://doi.org/10.17327/IPPA.2014.28.3.014.
서영수, and Seung Sin Lee. 2014. “A Study on Consumer Satisfaction and Willingness to Recommend by the Innovation Diffusion Theory: Comparison on Different Technology Adoption Stages of Smartphone.”Journal of Consumption Culture 17 (1): 89–111. https://doi.org/10.17053/JCC.2014.17.1.005.
