Examining Feedback Using Topic Modeling

1 Introduction

Text mining can allow for large amount of textual data to be sorted, assessed, and analyzed more quickly than ever before. This case study examines how topic modeling may be used to help sort through and quickly identify patterns in large amounts of qualitative text feedback.

This data used in this study has been collected from a feedback survey taken by university students in North India and contains no personal information about the students who responded or the instructors who were evaluated. To access the original data, click the following link: Institutional Academic Performance Evaluation through Student Feedback. The survey examined six categories: teaching, course content, examination, lab work, library facilities and extracurricular activities.

As this case study seeks to examine the qualitative feedback given by students using the unsupervised Latent Dirichlet Allocation (LDA) and Structural Topic Modeling (STM) algorithms to find topics within the feedback data, the data is wrangled to disregard the categories. The subset of data used in this study includes only the text comments pertaining to each of these categories and does not include the categories themselves. The response data is combined into one long textual string to see if the models can identify the original six categories, or if there were more topics discussed within the feedback. The two algorithms will be compared against each other. Each of these models will also compare k values. One will be a k-value pulled from the original number of categories (6) and the other will be a found k

These methods and alternative k-values are applied to produce models that can help answer the following questions:

• Can topic models produce accurate topics from qualitative textual student feedback?

• Are there more topics that students have feedback on that were not specified in the survey?

• Will the topics created by the models reveal patterns of what students liked and disliked?

• Will the inclusion of associated metadata improve the model results?

The models will be compared to the original data categories and to each other using many model evaluative and visualization techniques. This case study may serve as an introduction to text mining techniques and applications by comparing the model results to the originally categorized data.

2 Prepare the Environment

We will start by using the p_load() function in the pacman package to load the following packages needed for this project:

library(pacman)

# load packages
pacman::p_load(readxl,
               dplyr,
               stringr,
               tidyverse, 
               tidytext,
               SnowballC,
               topicmodels,
               stm,
               ldatuning,
               knitr,
               LDAvis
               )

3 Data Wrangling

Now we must prepare the data for modeling. This will include tokenizing, eliminating stop words, and creating a Document Term Matrix (DTM).

3.1 Import Data

To start, we must import the university feedback data file. To make the data suitable for our analysis, we will select only the columns with text strings that we will analyze, then rename the columns to be more concise:

# read data into environment
  
feedback_data <- read_excel("finalDataset0.2.xlsx") %>%
                  select(teaching...2,
                         coursecontent...4,
                         Examination,
                         labwork...8,
                         library_facilities...10,
                         extracurricular...12
                         ) %>%
                rename(
                  teaching = teaching...2,
                  content = coursecontent...4,
                  exam = Examination,
                  lab = labwork...8,
                  library = library_facilities...10,
                  extracurricular = extracurricular...12
                )

head(feedback_data)

# A tibble: 6 × 6
  teaching                           content exam  lab   library extracurricular
  <chr>                              <chr>   <chr> <chr> <chr>   <chr>          
1 teacher are punctual but they sho… conten… exam… not … librar… extracurricula…
2 Good                               Not go… Good  Good  Not go… Good           
3 Excellent lectures are delivered … All co… Exam… Lab … Librar… Extra curricul…
4 Good                               Conten… Agai… Good  Its th… Complete wasta…
5 teachers give us all the informat… conten… exam… prac… librar… extracurricula…
6 Yes                                Yes     Yes   Yes   Yes     Yes            

A look at the data shows that the written feedback strings were separated into categories. Now, because the text data is separated into categories and we are trying to use STM to identify topics, we will combine the categories and separate the feedback with a space and period:

# combine columns
feedback_strings <- feedback_data %>%
      mutate(combined_feedback = str_c(teaching, content, exam, lab, library, extracurricular, sep = ". "),
             feedback_id = row_number()) %>%
  select(feedback_id, combined_feedback)

head(feedback_strings)

# A tibble: 6 × 2
  feedback_id combined_feedback                                                 
        <int> <chr>                                                             
1           1 teacher are punctual but they should also give us the some practi…
2           2 Good. Not good. Good. Good. Not good. Good                        
3           3 Excellent lectures are delivered by teachers and all teachers are…
4           4 Good. Content of course is perfectly in line with the teaching ph…
5           5 teachers give us all the information required to improve the perf…
6           6 Yes. Yes. Yes. Yes. Yes. Yes                                      

3.2 Tokenize Data

Now we need to separate the text strings into individual words, or “tokens,” for analysis. This is considered tokenizing the data. At the same time, we can remove stop words of little value to the analysis.

  # tokenize text and remove stop words 
  feedback_tidy <- feedback_strings %>%
      unnest_tokens(output = word, input = combined_feedback) %>%
      anti_join(stop_words, by = "word")

Now, we will look at the most common words and remove any frequent words that we would not like to consider in our analysis. We will do this by creating a custom stop word list:

# find most common words
  feedback_tidy %>%
    count(word, sort = TRUE)

# A tibble: 850 × 2
   word           n
   <chr>      <int>
 1 excellent     77
 2 students      63
 3 university    62
 4 library       48
 5 books         47
 6 pattern       41
 7 lab           40
 8 activities    37
 9 knowledge     37
10 teachers      36
# ℹ 840 more rows

# create custom stop words
  custom_stopwords <- c("also", "upto", "lot", "much", "yes", "dont", "donot", "get", "every")

  # filter out custom stop words and numbers
  feedback_tidy <- feedback_strings %>%
      unnest_tokens(output = word, input = combined_feedback) %>%
      anti_join(stop_words, by = "word") %>%
      filter(!word %in% custom_stopwords) %>%
      filter(!str_detect(word, "^[0-9]+$"))
  
  head(feedback_tidy)

# A tibble: 6 × 2
  feedback_id word      
        <int> <chr>     
1           1 teacher   
2           1 punctual  
3           1 practical 
4           1 knowledge 
5           1 theortical
6           1 content   

A quick look at the data shows that all the words that occur frequently are related to education. There appear to be patterns, such as “library” and “books.” Another pattern may be “interaction” and “activities” or “lectures.”

If desired, it would be possible to stem the words at this point and reduce them down to their dictionary root. This means words such as “students” and “student” or “teachers” and “teaching” would be reduced and combined in the occurrence count to reduce the redundancy. However, because the efficacy of stemming is inconclusive, it was not included in the data preparation for this analysis.

3.3 Prepare for Modeling

Now we must create our document term matrix required by the Latent Dirichlet Allocation algorithm. We will treat each instance of feedback as a unique document. This matrix will create a row for each instance of feedback and columns for each word in the feedback, as well as the number of times the word occurs in each feedback.

 # count() how many times each word occurs in each post and create a matrix that contains one row per feedback with a column for each word and how many times that word occurs in each post.
  feedback_dtm <- feedback_tidy %>%
    count(feedback_id, word) %>% # count() how many times each word occurs in each feedback
    cast_dtm(feedback_id, word, n) # one row per feedback, a column for each word, and how many times that word occurs in each feedback

Next, to prepare for the Structural Topic Modeling, we will use the textProcessor() function.

# process combined_feedback strings in preparation for structural topic modeling
  feedback_temp <- textProcessor(feedback_strings$combined_feedback, # column in our data frame that contains the text to be processed
                        metadata = feedback_strings, # the data frame that contains the text of interest
                        lowercase = TRUE,
                        removestopwords = TRUE,
                        removenumbers = TRUE,
                        removepunctuation = TRUE,
                        wordLengths = c(3,Inf),
                        stem = FALSE,
                        onlycharacter = FALSE,
                        striphtml = TRUE,
                        customstopwords = custom_stopwords)

Using the stm package requires a very unique set of inputs that are specific to the package. The following code will pull elements from the temp list that was created that will be required later when we use the stm() function.

# pull elements from the temp list that was created that will be required for the stm() function
  meta <- feedback_temp$meta
  vocab <- feedback_temp$vocab
  docs <- feedback_temp$documents

4 Topic Modeling

Now we’re going analyze the data with topic modeling. We will use Latent Dirichlet Allocation and Structural Topic Modeling. These models will produce a specified number of topics based on the data and parameters they are given, which may allow us to find broader topics discussed in the feedback.

4.1 First Models: k = 6

To start, we will use the LDA() function for unsupervised classification of the feedback test to find groupings of words, or topics. LDA will allow documents to overlap each other in terms of content, meaning one document can be made up of many topics. Furthermore, for each topic there are many words, and each of these words can be shared between topics with a different probability of occurring in each topic.

Both models require a k value for the number of topics in our corpus, similar to k-means other other simple clustering approaches. We will start with a k-value of 6, as that was how many areas were covered in the feedback survey and therefore the number of columns in the pre-processing data. We must also set the seed so that when someone reproduces the results, the algorithm will be able to start in the same place.

  # run LDA topic model using 6 as k
  feedback_lda <- LDA(feedback_dtm, 
                      k = 6, 
                      control = list(seed = 588) # set seed for reproducibility
                      )

Then, to see if we can improve the assignment of words to these topics, we will use the stm() function. STM factors in meta data about the documents to improve the model results. Again, we will start with a k-value of 6, as that was how many categories the feedback survey data was separated into and therefore the number of columns in the pre-processing data.

 # fit STM topic model
  feedback_stm <- stm(documents = docs,
                      data = meta,
                      vocab = vocab,
                      prevalence =~ feedback_id,
                      K= 6,
                      max.em.its = 25,
                      verbose = FALSE)

4.2 Second Models: finding k

We started with a k value of 6 because of the original 6 categories. However, the value of k used can greatly impact the results. Let’s see if 6 was the optimal value of k using FindTopicsNumber() and the DTM created earlier. The following code will show the metrics for k values 0-20, plotted by 2:

# get metrics for a sequence of topics and plot the output
  k_metrics_feedback <- FindTopicsNumber(
                        feedback_dtm,
                        topics = seq(0, 20, by = 2),
                        metrics = "Griffiths2004",
                        method = "Gibbs",
                        control = list(),
                        mc.cores = NA,
                        return_models = FALSE,
                        verbose = FALSE,
                        libpath = NULL
                        )
                      
  FindTopicsNumber_plot(k_metrics_feedback)

It appears there is a peak at 10 for the value of k. Let’s use it to create new models.

# fit a model using different k value
  feedback_lda_2 <- LDA(feedback_dtm, 
                        k = 10, 
                        control = list(seed = 588) # set seed for reproducibility
                        )
  
  feedback_stm_2 <- stm(documents=docs,
                      data=meta,
                      vocab=vocab,
                      prevalence =~ feedback_id,
                      K = 10,
                      max.em.its=25,
                      verbose = FALSE)

5 Explore

Now we will explore and interpret the models we have created.

5.1 STM Visualization: LDAvis

Use the plot.STM() function to see the top 5 expected words assigned to each topic.

# plot the STM model
plot.STM(feedback_stm, n = 5)   

Plot the second STM model:

# plot the STM model
plot.STM(feedback_stm_2, n = 5)     

A quick glance does not reveal any obvious groups that are unique between the different STM models. Perahps the model with the k value of 10 may be more descriptive or specfic. We will further examine the differences between the k-value models in the next “Explore” section.

We can also visualizatize the STM models using toLDAvis(), which will show visualizations in the browser:

  # explore topic and word distributions
  toLDAvis(mod = feedback_stm, docs = docs)
  toLDAvis(mod = feedback_stm_2, docs = docs)

The first visualization dashboard shows some overlap between the half the topics, and the other half are completely isolated. Topic 1 has a very high frequency of the word “good,” but very low frequency for all other words.

The second visualization has high overlap between 8 of the 10 topics, with topics 1 and 10 being isolated. The frequency of words and even location on the intertopic distance map of topic 1 is very similar to topic 1 in the first visualization. Topic 10 appears to be full of descriptors such as “average” and “bad.” The overlapping topics appear to have most of the words relating to the university, courses, and descriptions.

Ideally, there would be distinct topics with almost no overlap, but both of the k values tested appear to produce overlap in the topics, although k = 6 performed much better in this regard.

5.2 LDA Visualization:

We can look at the LDA models by visualizing the beta and gamma values. First we will look at the beta values for each of our LDA models, which is how likely it is for a word to belong to a topic. First, convert the LDA model to a tidy dataframe to prepare for beta value visualizations:

  # convert LDA model to tidy dataframe
    tidy_feedback_lda <- tidy(feedback_lda)
    tidy_feedback_lda_2 <- tidy(feedback_lda_2)

Now visualize the top 5 terms for each of the LDA models:

 # visualize top 5 terms for each topic
  top_feedback_terms <- tidy_feedback_lda %>%
    group_by(topic) %>%
    slice_max(beta, n = 5, with_ties = FALSE) %>%
    ungroup() %>%
    arrange(topic, -beta)
  
  top_feedback_terms %>%
    mutate(term = reorder_within(term, beta, topic)) %>%
    group_by(topic, term) %>%    
    arrange(desc(beta)) %>%  
    ungroup() %>%
    ggplot(aes(beta, term, fill = as.factor(topic))) +
    geom_col(show.legend = FALSE) +
    scale_y_reordered() +
    labs(title = "Top 5 terms in each LDA topic",
         x = expression(beta), y = NULL) +
    facet_wrap(~ topic, ncol = 4, scales = "free")

The output shows the top words and their beta values in each of the 6 topics in our first LDA. A quick look shows that:

• Topic 5 appears to be about the lecture portion of the course or the lecture abilities of the instructor. This being a standalone topic is consistent with the original grouping of the data.

• All but one of the topics in the first LDA model have “university” as a top word. This is likely evident of the subject matter of the survey. It could be comments and ratings about the university as a whole or calls to action about how to act to fix areas of improvement.

Now, visualize the second LDA:

# visualize top 5 terms for each topic
  top_feedback_terms2 <- tidy_feedback_lda_2 %>%
    group_by(topic) %>%
    slice_max(beta, n = 5, with_ties = FALSE) %>%
    ungroup() %>%
    arrange(topic, -beta)
  
  top_feedback_terms2 %>%
    mutate(term = reorder_within(term, beta, topic)) %>%
    group_by(topic, term) %>%    
    arrange(desc(beta)) %>%  
    ungroup() %>%
    ggplot(aes(beta, term, fill = as.factor(topic))) +
    geom_col(show.legend = FALSE) +
    scale_y_reordered() +
    labs(title = "Top 5 terms in each LDA topic",
         x = expression(beta), y = NULL) +
    facet_wrap(~ topic, ncol = 4, scales = "free")

The output for the top words in our second LDA model with the k-value of 10 reveals other clear topics:

• It looks like Topic 3 is about the facilities, resources, and subsequent outcomes of the university.

• Topic 8 may be about the qualities of the professor.

• Similarly, Topic 2 appears to talk about the aspects of the courses or instructors.

• Topic 5 may be calling for improvements to the courses or instructors.

• Similarly, Topic 6 may be complaints about the university.

Now we want to find out how prevalent each word is in the overall feedback corpus, or all the feedback text. We can do this by combining the beta and gamma values. The gamma values are the probability that each document is generate from each topic.

 # create two data frames for beta and gamma values
  td_beta_feedback <- tidy(feedback_lda)
  
  td_gamma_feedback <- tidy(feedback_lda, matrix = "gamma")

 # create two data frames for beta and gamma values
  td_beta_feedback2 <- tidy(feedback_lda_2)
  
  td_gamma_feedback2 <- tidy(feedback_lda_2, matrix = "gamma")

Then we will create a filtered data frame of our top_terms, join this to a new data frame for gamma-terms, and create a nice clean table using the kabel() function knitr package:

 # create filtered dataframe of top_terms and join it with gamma-terms
  top_feedback_terms <- td_beta_feedback %>%
    arrange(beta) %>%
    group_by(topic) %>%
    top_n(5, beta) %>%
    arrange(-beta) %>%
    select(topic, term) %>%
    summarise(terms = list(term)) %>%
    mutate(terms = map(terms, paste, collapse = ", ")) %>% 
    unnest(cols = c(terms))
  
  gamma_terms_feedback <- td_gamma_feedback %>%
    group_by(topic) %>%
    summarise(gamma = mean(gamma)) %>%
    arrange(desc(gamma)) %>%
    left_join(top_feedback_terms, by = "topic") %>%
    mutate(topic = paste0("Topic ", topic),
           topic = reorder(topic, gamma))
  
  gamma_terms_feedback %>%
    select(topic, gamma, terms) %>%
    kable(digits = 3, 
          col.names = c("Topic", "Expected proportion", "Top 5 terms"))

Topic	Expected proportion	Top 5 terms
Topic 6	0.218	average, students, university, lectures, activities
Topic 2	0.217	excellent, lab, teaching, books, faculty
Topic 1	0.161	students, teachers, library, university, books
Topic 3	0.150	lab, university, library, material, pattern
Topic 4	0.143	books, time, university, depth, pattern
Topic 5	0.112	paper, university, teacher, checking, teaching

Do it again for the second model.

# create a second filtered dataframe of top_terms and join it with gamma-terms
  top_feedback_terms2 <- td_beta_feedback2 %>%
    arrange(beta) %>%
    group_by(topic) %>%
    top_n(5, beta) %>%
    arrange(-beta) %>%
    select(topic, term) %>%
    summarise(terms = list(term)) %>%
    mutate(terms = map(terms, paste, collapse = ", ")) %>% 
    unnest(cols = c(terms))
  
  gamma_terms_feedback2 <- td_gamma_feedback2 %>%
    group_by(topic) %>%
    summarise(gamma = mean(gamma)) %>%
    arrange(desc(gamma)) %>%
    left_join(top_feedback_terms2, by = "topic") %>%
    mutate(topic = paste0("Topic ", topic),
           topic = reorder(topic, gamma))
  
  gamma_terms_feedback2 %>%
    select(topic, gamma, terms) %>%
    kable(digits = 3, 
          col.names = c("Topic", "Expected proportion", "Top 5 terms"))

Topic	Expected proportion	Top 5 terms
Topic 6	0.182	average, university, bad, satisfactory, students
Topic 9	0.155	excellent, teaching, teachers, pattern, distribution
Topic 3	0.122	knowledge, lab, library, excellent, material
Topic 2	0.100	delivery, lecture, punctuality, checking, exam
Topic 4	0.094	books, time, university, depth, awesome
Topic 7	0.085	students, fine, coming, university, understand
Topic 10	0.078	excellent, books, library, lab, subjects
Topic 1	0.071	students, teachers, library, marks, books
Topic 5	0.064	university, students, improve, courses, teaching
Topic 8	0.048	teacher, content, subject, answers, student

Combining the LDA topic occurrence with the expected proportion suggests that there are some themes:

• University Topics: Most topics contain the terms “teacher”, “students”, “university”. These are, of course, an overarching theme due to the context of the feedback, but also residual of the prompts being asked in the feedback survey.

• The most prevalent topics, Topics 6 in both, seem to describe feelings of overall satisfaction towards the university and/or its components. It is worth noting that the term “average” may refer to their feelings towards the topics, or could be used as an adjective, such as “the average student” or “on average.” This will be examined later in the case study.

There are also many themes specific to the topics in the second k=10 LDA. This may indicate it was in fact a better k-value. The distinct topics appear to be:

• Teacher Rating: Topic 9 appears to include ratings of teachers. It is very high on the expected proportion list.

• University Facilities: Topic 3, containing words relating to facilities, resources, and subsequent outcomes of the university, has a very high expected proportion.

• Instructor Characteristics: Another clear topic, 2, presumed to be about the aspects of the course or instructors, is also very high. This relates to

• Course: Topic 8, while at first appearing to be clearly describing the course, has the lowest expected proportion.

• Topic 5 may be calling for improvements to the courses or instructors.

5.3 Comparing the Different Topic Models

We will compare the LDA visualization to the most prevalent topics and terms from our feedback_stm model using the plot() function.

5.3.1 Compare k = 6

# compared prevalent topics and terms 
  plot(feedback_stm, n = 5)

When comparing the LDA and STM models with k = 6, there are many terms that appear frequently across topics in both models, including excellent, books, university, and library.

Common themes surround library, books, and university, which is in line for the subject matter of the survey data.

• STM Topic 2 and LDA Topic 1 both emphasize students, library, and books.

• STM Topic 5 and LDA Topic 3 both mention university, lab, and material, pointing to discussions around academic resources and infrastructure.

• The STM topics appear to be more evaluative with words like good, excellent, and bad. There also appears to have more verbiage, like deliver, work, and provides.

Differences between the models also appear in the expected proportions. Most notably, the STM has a higher expected proportion of .41 for Topic 6 compared to the Topic 6 at .218 for the LDA. This could be a result of the STM’s use of metadata to identify a topic as more significant.

5.3.2 Compare k = 10

# compared prevalent topics and terms from feedback_stm_2
  plot(feedback_stm_2, n = 5)

The LDA and STM models with k = 10 also have many of the same frequent themes appearing across topics in both models, including university, students, teaching, library, and books. However, it seems there are more unique difference in the topics.

• The STM topics appear to be more evaluative with words like good, excellent, bad, and less.

• This time, the LDA appears to have more specific feedback areas, such as delivery, lecture, and punctuality.

• STM’s Topic 2 appears to describe the library facilities using words such as “nice” and “really.”

• LDA’s Topic 2 appears to describe aspects of the course or instructor, appearing to give clear feedback on the various characteristics.

Again, there are differences between expected proportions in the models. Again, the STM has a higher expected proportion of its highest topic. This further supports the theory that the STM’s use of metadata allows it to place higher significance on a topic.

5.4 STM: Qualitative Exploration

To examine some of these topics qualitatively, we can use the findThoughts() function to extract passages from documents within the corpus associate with topics that we specify. First we will look at some of the topics of note in the first STM:

feedback_reduced <-feedback_strings$combined_feedback[-feedback_temp$docs.removed]


# pass original datset through the findThoughts() function, and set argument to return n=10 to look at topic 
  findThoughts(feedback_stm,
               texts = feedback_reduced,
               topics = 2, 
               n = 10,
               thresh = 0.5)

 Topic 2: 
     Interaction is very good,accurate supplement material is provided but recommendation of books is very poor,the change of a particular teacher after 3 months or a semester of same subject causes problems. The books that are recommended do not contain appropriate content; leads the students to depend on slides given by teachers. Hence the knowledge remains superficial. Overall all these things are the plus point of the university. Average. Awesome, peaceful, relaxing atmosphere. Good but lacks audience.....as only participants can see these activities on duty leave basis
    they are showing partiality between north indians and south indians. they are giving good knowledge for us. its better. in lab also they are showing parilaty. i felt its better because there is no problem from library facilities. i felt happy because it encourages our talents
    average. average. the best pattern of examination are in lpu .. most of the teacher don't even bother about the practical performed by students.They give marks just by seeing face or something like that.
. Almost each and every books is there.But the time limit for keeping book is too much shorted.. Activities are not up to the mark as  advertisement.
    I WAS A STUDENT OF A BOARDING SCHOOL .....(jnv) I STUDIED FOR 7  YEARS THERE RESIDING IN HOSTEL ..NOW LIVING IN LPU IS SAME LIKE THAT..AND STILL MORE SAME AND PUNCTUALITY IS SHOWN HERE BY BOTH TEACHERS AND STUDENTS...AND ALSO BY THE STAFF..          SECURITY IS ABOVE 90%(IN CASE OF  STUDENT IS WELL MANURED)  ....CLASSES REGULAR ....SPORTS DAILY.....PARTICIPATION WELL/.......QUALITY BELOW AVERAGE. LESS TIME FOR STUDY ...LESS TIME FOR PAPER ATTEMTING...LESS TIME FOR EVERY THING ....LESS KNOWLEGE.....LESS DEPTH OF COURSE .....BUT PROPER COURSE MATERIAL ACCORDING TO EXAM. NICE TIMING... NOT GOOD .... VERY NICE....BUT BOOKS ARE NOT AVAILABE EVERY TIME. VERY HIGH......NEED TO TAKE DOWN SOME ACCITIVTIES ................................NOISE POLLITION IN THE UNIVERSITY  ALWAYS A BIG PROBLEM.
    some of our faculty member are not so experienced that's why most of the time we are facing problems subjects like CSE(c++). as per the courses all the sub are providing  worth knowledge. as per the exam pattern maths exam always have to be subjective rather than objective.. as per the lab evaluation we r satisfied with  it. duration of issuing   books from library should be changed from 1week to 2 week that would be great.. that's  great
    teachers give us all the information required to improve the performance.. content of courses improves my knowledge. examination pattern is good. practical work provides detail knowledge of theoretical work. library has huge collection of books from different authors. extracurricular activities increases mental and physical abilities
    It's good when compare with my Btech lectures and teaching activities.. Knowledge  and depth of the course  is good but while coming to the material it's not upto the mark.. This university is far better than other university when coming to the exam pattern,marks disturbation and while coming to paper checking no biased corrections equal to all.. Lab and practical are not upto the mark. The topic which are practiced are almost already done in Btech.. Good. While coming to extracurriculum activities it's better than any other university which I had come around.
    lecturer are punctual and can interact them easily.. content of the courses are update and mostly based on current technology.. exam pattern and marks distribution is very well procedure. well and good infrastructure and have trending software in lab. library is very well managed and provide needed facilities to us. extracurricular activities held in university are necessary it provides student a platform where they can improve and groom them.
    it is quite good , depends upon teacher and their experience. content is enough, we donot need more.. too many examination, we are frustrated. lab activities are good but not satisfactory in explaining point of view.. library is awesome and rich. good it is, but teachers do not consider the new ones.
    excellent university  the mode of teaching is completely different than any other university in india. lecturers are very friendly with every one they interact just like friends and they are very punctual. every thing is good subjects have a very good depth almost all the  books are available   in the library and more than that 4/5. exam pattern is completely different the entire exams are objective oriented resumbling  the education in abroad my review is 5/5. the lab evaluations are very strict. excellent. excellent  simply superb

Examining the first STM’s Topic 2 documents shows that activities are indeed discussed, and are mostly positive. There are also many reviews of the library and books. Interestingly, the reviews on the library appear to be positive, but the book policies are more negative. The overall impact of these factors and others on student knowledge is also discussed throughout the documents in this topic. The topic seems to have been accurately described.

# pass original datset through the findThoughts() function, and set argument to return n=10 to look at topic 
  findThoughts(feedback_stm,
               texts = feedback_reduced,
               topics = 5, 
               n = 10,
               thresh = 0.5)

 Topic 5: 
     Talk about punctuality, it's really good but the lecture delivery is somehow not appreciated because every teacher has different style of teaching, so lack of knowledge in terms of subject is there and interaction is also kind of boring. A teacher should be entertaining as well but they are really not. All they want us to be studios and disciplined which is really not the thinking of every student.. The materials or links provided for study purpose is good, clarity in depth of courses but again it differs from teacher to teacher.. Exam pattern and how it is conducted is really seems to be good but it's my first year and when I went to scrutiny for the first time it was not as per my expectations. Teachers have marked the answers as per their choice, they must look for the content rather than the length of answers. So poor paper checking is there.. The evaluation is again bad for our CA, as one of our faculty is really weird. He gives us remarks on the basis of how we present things in front of it, he never look for the content of our answer, which is really annoying. As he expects us to deliver answer in the way he wants us to do. But how can a person do that, as people do have different mentality and they do explain themselves in different manner, so a teacher must understand that quality of a child and should give him marks on the basis of what they say not what they should say.. Again some of the faculties don't know how to behave otherwise it is good.. This is really a good thing that events happen in our university regularly but they must be announced early so that every student can prepare themselves for participation because sometimes we get to know about most of the events when they have already been conducted. So announcement must be there.
    Good. Content of course is perfectly in line with the teaching philosophy that is being perpetuated here i.e cramming,rote learning.. Again the university tests students of their ability to memorize stuff. Questions should emphasize more on concepts rather than testing brain's cramming storage house.. Good. Its the best thing i have seen in this university,albeit the behaviour of library staff is unprofessional at times. They are whimsically rude.. Complete wastage of time. Again this opinion is strictly personal and may not coincide with others.
    The university is good but it also has a long way to go. More teachers should have their PhD and college fests related to academic curriculum must be improved to a great extent and must be managed properly.The interaction is good, lecture delivery needs more lucidity. The punctuality is top notch.. Content of the courses should be changed a bit. More of industry oriented feature like Linux and GitHub must be implemented. Proper detailing and advanced knowledge is required and is to be instilled in the students.The course material is good.. The examination pattern is good. More of logic based and subjective questions to be asked which is based upon conceptual learning. Paper checking must be done in a more lucid manner because many doubts arise with the checking of OMR sheet.. Lab evaluations are good.. Library facilities are the strength of this university. It is awesome.. Quite a lot of events are organized and students have a all round growth which is a nice thing.
    teaching is ok but the interaction with the students need to improve, i mean they have to focus on dull students more  than intelligents..... good. for some subjects like maths and english(passage writing) the time duration neds to increase. In labs,the lecturers has to interact with each student to improve their knowledge, in all the ways,before giving us the work they must explain about the experiment once so that we will come to know how to perform n take precautions while doing,........
. good but there is more need of keeping books of general topics about social,polity,gk,...etc other than related to subject so that we will get some knowledge about country related issues............... good
    teaching is good but some teachers are not able to interact properly and are not able to clear the concepts properly, teachers are punctual. 
. content is sufficient for us. exam pattern is very nice but paper checking is not at all good .. lab practical work help us to learn the things properly and evalution is also very necessary as it helps us to retain concepts. library facilty is good. university offers alkot of activities which is excellent on their part as students are able to explore themselves
    its to good. i dont know. its very bad thing not sowing answer of mcQ in exam we dont jiged what is rigth and rong where mak mistacvk. lab and practical work are litial good. i dont think about. hahahah
    Interaction between students and teachers is good ,The lectures delivery is fine, punctuality is good
 

. depth of the course are depended on the level of student intake, course material is delivered after the class in UMS 

. Exams are conducted upon the the basis of the revised chapters upto that point, marks distribution is appropriate, paper checking is good

. Lab is good the Prof explains everything before the lab begins and the Lab Asst helps us during the experiments. The library has insufficent number of books for all the students. good they are knowledgebale and helps us interacting with other students
    it is good depending on the faculty. the material provided is beneficial and provides much knowledge. it is neither too good nor too bad. lab evaluations are pretty good .they help us a lot.. library facilities provided are excellent. they are amazing and help to boost our stamina.
    Lectures helps in clear our concept. Easily available course material with sufficient knowledge gaining. Good marks distribution Students can easily check there performance throught the semester. Practical and easy learning. Good. Its give various opportunities and exposure
    Excellent lectures are delivered by teachers and all teachers are very punctual.. All courses material provide very good knowledge in depth .. Exam pattern is up to the mark and the Cgpa depends on the various marks distributions like ca, mte etc which is very nice as compare to other institutions. Paper checking does not depend on length of ques but on the material which is the best part of it.. Lab work is properly covered in the labs by the faculty and evaluations help to learn more practical knowledge in depth.. Library facilities are excellent in terms of good techniques that are used over there.. Extra curricular activities also help students to divert their mind from their study for some time. University plays an important role in this task.

The first STM’s Topic 5 is very broad and appears to have been accurately presented. Feedback appears to be given on course, lab, materials, and interactions, whereas the university is just the setting in which it takes place. The feedback on materials and activities appear to be positive. The feedback on the lab is mixed based whether on explanation, guidance, and support are given. Overall, course feedback given criticized simple lectures and memorization.

Now we will look at a topic of note in the second STM:

# pass original datset through the findThoughts() function, and set argument to return n=10 to look at topic 
  findThoughts(feedback_stm_2,
               texts = feedback_reduced,
               topics = 2, 
               n = 10,
               thresh = 0.5)

 Topic 2: 
     Talk about punctuality, it's really good but the lecture delivery is somehow not appreciated because every teacher has different style of teaching, so lack of knowledge in terms of subject is there and interaction is also kind of boring. A teacher should be entertaining as well but they are really not. All they want us to be studios and disciplined which is really not the thinking of every student.. The materials or links provided for study purpose is good, clarity in depth of courses but again it differs from teacher to teacher.. Exam pattern and how it is conducted is really seems to be good but it's my first year and when I went to scrutiny for the first time it was not as per my expectations. Teachers have marked the answers as per their choice, they must look for the content rather than the length of answers. So poor paper checking is there.. The evaluation is again bad for our CA, as one of our faculty is really weird. He gives us remarks on the basis of how we present things in front of it, he never look for the content of our answer, which is really annoying. As he expects us to deliver answer in the way he wants us to do. But how can a person do that, as people do have different mentality and they do explain themselves in different manner, so a teacher must understand that quality of a child and should give him marks on the basis of what they say not what they should say.. Again some of the faculties don't know how to behave otherwise it is good.. This is really a good thing that events happen in our university regularly but they must be announced early so that every student can prepare themselves for participation because sometimes we get to know about most of the events when they have already been conducted. So announcement must be there.
    some of our faculty member are not so experienced that's why most of the time we are facing problems subjects like CSE(c++). as per the courses all the sub are providing  worth knowledge. as per the exam pattern maths exam always have to be subjective rather than objective.. as per the lab evaluation we r satisfied with  it. duration of issuing   books from library should be changed from 1week to 2 week that would be great.. that's  great
    The university is good but it also has a long way to go. More teachers should have their PhD and college fests related to academic curriculum must be improved to a great extent and must be managed properly.The interaction is good, lecture delivery needs more lucidity. The punctuality is top notch.. Content of the courses should be changed a bit. More of industry oriented feature like Linux and GitHub must be implemented. Proper detailing and advanced knowledge is required and is to be instilled in the students.The course material is good.. The examination pattern is good. More of logic based and subjective questions to be asked which is based upon conceptual learning. Paper checking must be done in a more lucid manner because many doubts arise with the checking of OMR sheet.. Lab evaluations are good.. Library facilities are the strength of this university. It is awesome.. Quite a lot of events are organized and students have a all round growth which is a nice thing.
    Excellent lectures are delivered by teachers and all teachers are very punctual.. All courses material provide very good knowledge in depth .. Exam pattern is up to the mark and the Cgpa depends on the various marks distributions like ca, mte etc which is very nice as compare to other institutions. Paper checking does not depend on length of ques but on the material which is the best part of it.. Lab work is properly covered in the labs by the faculty and evaluations help to learn more practical knowledge in depth.. Library facilities are excellent in terms of good techniques that are used over there.. Extra curricular activities also help students to divert their mind from their study for some time. University plays an important role in this task.

The second STM’s Topic 2’s top terms describe the library. While the are positive descriptions of the library, the majority of the text appears to relate more to feedback on various aspects of the courses and instructors. The feedback is both positive and negative, with some topics overwhelmingly positive or negative or mixed. This is to be expected, as topics can overlap. Overall, it seems this topic was accurately predicted and represented, as it positively describes the library, but occurs in very few of the documents.

Finally, we will look at the second STM’s Topic 6:

# pass original datset through the findThoughts() function, and set argument to return n=10 to look at topic    
  findThoughts(feedback_stm_2,                
               texts = feedback_reduced,                
               topics = 6,                 
               n = 10,                
               thresh = 0.5)

 Topic 6: 
     good. good. good. it is difficult to understand experiment in 1 hour. good. good
    Good. Good. Good. Good. Good. Good
    good. good. good. good. good. good
    good. good. good. good. good. good
    good. good. good. good. good. good
    Good. Good. Good. Good. Good. Good
    good. good. good. good. good. good
    good. good. good. good. excellent. good
    Good. Good. Excellent. Good. Good. Good
    Good. Good. Good. Good. Good. Good

It is very obvious now how the predicted proportion for the second STM’s model 6 was so high. These strings of feedback do not appear to be very helpful to our analysis.

6 Communicate

6.1 Summary

• Can topic models produce accurate topics from qualitative textual student feedback?

Each of the models were able to identify many of the original areas of feedback in the survey, and include the common words given in the feedback on those topics, as shown with the findThoughts() function. It appears as though topic modeling is an effective way to find groupings of words. However, there was heavy overlap across topics with both models and both values of k. There was also a topic produced that did not have helpful and elaborative feedback.

• Are there more topics that students have feedback on that were not specified in the survey?

There appear to be a number of subtopics that arise from the central 6 or so topics. These are areas that are commonly referred to when giving feedback on the central 6 topics. None of the additional topics listed appear to have separate groups that do not include a central 6 topics. This could be due to the nature of the survey and feedback, especially as it is already categorized. It could also be that the topic models simply could not find any other topics that occur as frequently as the original 6.

• Will the topics created by the models reveal patterns of what students liked and disliked?

There were many topics where the words predicted to be in the topic did describe areas of the university in the topic. There were also cases were adjectives predicted in the topic described areas of the university in the topic, such as STM Topic 2 with the positive reviews on the library. It appears that the models may be more helpful in understanding what aspects of the survey areas students had feedback on.

• Will the inclusion of associated metadata improve the model results?

It appears that the all of the models, regardless of algorithm or k-value, returned mostly similar topics. The models with higher k-values appear to be more specific and form more distinct groups. However, this does not necessarily mean they are more accurate.

Overall, the models appear to be able to pull out common areas of feedback, as well as aspects and characteristics that were discussed in the feedback, both positively and negatively.

6.2 Applications and Future Use

Feedback may be important to a variety of audiences for a number of reasons. Students who attend the university may want to explore potential courses and teachers, and seeing the feedback of a professor may inform their choice. Teachers and administrators may want to understand where courses or teachers are falling short or exceeding.

The use of topic modeling on feedback data can be expanded and used to find areas that students are providing feedback on that are not included as categories in the original survey, such as practicality, interaction, and punctuality. Using the findThoughts() function helps quickly grab text pertaining to the topic, allowing evaluates to quickly see what the specific comments are for a generated topic of feedback.

This analysis is a start, and by building upon it, educators can better understand their learners’ satisfaction rates. By applying any necessary interventions to increase satisfaction rates, faculty can increase enrollment and course completion rates.

6.3 Limitations

This study focus on a higher education institution, but this case study could be applied to similar feedback surveys delivered in other educational settings. These visualization are intended to help understand patterns of feedback, stories of success, and areas of improvement for instructors and academic programs.

Because there were no other variables/features in the original data, they were not included in the analysis. However, including demographics, courses, instructors, and grades could add further depth to the analysis and reveal more patterns in the data. This would help faculty address areas of improvement and model after successes. The collected data includes improper grammar and unusual syntax, which could result from the participants being nonnative English speakers. In the data wrangling process, the data was also not stemmed, which, if done, may lead to different results. During the modeling process, only 2 k-values were tested on the models. Further testing may reveal that different values of k may prove to be better suited for the data.