# Text mining packages
# install.packages(c("tm", "wordcloud")) if you get errors here
library(tm)
library(wordcloud)Advanced Visualization Techniques
Learning Analytics — Week 4 (Required)
Learning objectives
By the end of this file, you will be able to:
- Create a word cloud from text data and interpret frequency patterns
- Build a heatmap to reveal patterns across two dimensions
- Visualize a social network of student interactions using SNA
- Reflect on when each visualization type is the most appropriate choice
When to use which advanced visualization
| Visualization | Best for | Typical data source |
|---|---|---|
| Word cloud | Frequency of words in text | Survey responses, discussion posts, feedback |
| Heatmap | Patterns across two categories | Scores by subject + week, engagement by module + cohort |
| SNA plot | Relationships between individuals | Forum reply data, collaboration records |
Before you run any code, read through the whole file once. Notice which dataset each part uses and make sure the file is in your data folder.
Part 1 · Word cloud
A word cloud visualizes how often words appear in a text — larger words appear more frequently. We will use the Handbook of Learning Analytics (2022), which has been converted to a plain text file.
Packages for this section
Load the text data
# read.delim() reads a plain text file
# header = FALSE and stringsAsFactors = FALSE read it as raw text
la_text <- read.delim("data/Handbook of LA.txt",
header = FALSE,
stringsAsFactors = FALSE)
# Create a text corpus — a collection of text documents the tm package can process
doc <- Corpus(VectorSource(la_text))
head(doc)<<SimpleCorpus>>
Metadata: corpus specific: 1, document level (indexed): 0
Content: documents: 1
Clean the text
Before generating the word cloud, we remove common words, punctuation, numbers, and other noise. This is called text preprocessing.
# Helper function: replace specific characters with a space
toSpace <- content_transformer(function(x, pattern) gsub(pattern, " ", x))
doc <- doc |>
(\(d) {
d <- tm_map(d, toSpace, "/")
d <- tm_map(d, toSpace, "@")
d <- tm_map(d, toSpace, "\\|")
d <- tm_map(d, content_transformer(tolower))
d <- tm_map(d, removeWords,
c(stopwords("english"), "https", "can", "doi", "also", "use"))
d <- tm_map(d, removeNumbers)
d <- tm_map(d, removePunctuation)
d <- tm_map(d, stripWhitespace)
d
})()You may see warning messages when running the preprocessing chunk. These are expected from the tm package — as long as you see no error messages, you can continue.
Generate the word cloud
# Build a term-document matrix — a table of word frequencies
dtm <- TermDocumentMatrix(doc)
m <- as.matrix(dtm)
v <- sort(rowSums(m), decreasing = TRUE)
word_freq <- data.frame(word = names(v), freq = v)
# Top 10 most frequent words
head(word_freq, 10)Word cloud from the Handbook of Learning Analytics (2022)
# Generate the word cloud
set.seed(1234)
wordcloud(
words = word_freq$word,
freq = word_freq$freq,
min.freq = 5,
max.words = 100,
random.order = FALSE,
rot.per = 0.35,
colors = brewer.pal(8, "Dark2")
)Task 1 — Parameter exploration: Try changing max.words to 50 or 150 and min.freq to 10. How does the word cloud change? Which version is more readable?
- [I did max words of 50 and min freq of 10. It made the circle smaller and I lost a lot of words. I liked the first version and found it more readable. It also cut off words from the bigger circle.]
Task 2 — Add your own stopwords: Look at the word cloud above. Find 2–3 words that appear large but feel too generic to be meaningful — words that are frequent only because they appear in every chapter, not because they signal a specific theme.
Add those words to the removeWords() line in the text-preprocessing chunk above, then re-run that chunk and the wordcloud-generate chunk. The line to edit looks like this:
d <- tm_map(d, removeWords,
c(stopwords("english"), "https", "can", "doi", "also", "use"))Add your words inside the c(...) vector, then re-run both chunks.
Reflection: Which words did you remove? Why did you choose them? Does the revised cloud reveal any themes the original missed?
- [I removed “international” and “network.” I chose them because they didn’t add anything to the word cloud. Learning is still the main word, but I can see others now like knowledge and analysis.]
Question: Looking at the most frequent words, what does this tell you about the themes the learning analytics field focuses on? Does anything surprise you?
- [It uses lots of words surrounding analytics and data which tells me this word cloud is focused on the analytics. I’m surprised because there are several words that repeat and several words that don’t make any sense as to why they are in here.]
Part 2 · Heatmap
A heatmap uses color intensity to show values across two categorical dimensions — useful for spotting patterns that would be invisible in a table or bar chart.
Package for this section
# reshape2 gives us melt() for converting wide to long format
# install.packages("reshape2") if needed
library(reshape2)Load the data
data_hm <- read.csv("data/student_assignment_scores.csv")
head(data_hm)glimpse(data_hm)Rows: 30
Columns: 11
$ Student_ID <chr> "Student_1", "Student_2", "Student_3", "Student_4", "Stu…
$ Assignment_1 <int> 98, 86, 50, 74, 59, 85, 67, 98, 96, 73, 56, 85, 74, 86, …
$ Assignment_2 <int> 77, 89, 52, 82, 91, 73, 79, 92, 64, 71, 57, 85, 53, 71, …
$ Assignment_3 <int> 52, 70, 51, 87, 59, 89, 54, 82, 98, 88, 85, 94, 91, 58, …
$ Assignment_4 <int> 98, 67, 54, 95, 89, 89, 70, 85, 85, 88, 91, 67, 76, 62, …
$ Assignment_5 <int> 54, 58, 65, 75, 92, 91, 91, 82, 78, 74, 100, 81, 66, 95,…
$ Assignment_6 <int> 87, 80, 55, 69, 64, 83, 91, 72, 61, 75, 90, 50, 51, 73, …
$ Assignment_7 <int> 54, 84, 63, 63, 77, 82, 86, 84, 75, 80, 94, 92, 81, 66, …
$ Assignment_8 <int> 80, 57, 65, 52, 86, 62, 61, 50, 79, 100, 80, 86, 96, 55,…
$ Assignment_9 <int> 82, 87, 92, 90, 71, 75, 69, 94, 86, 92, 64, 70, 83, 74, …
$ Assignment_10 <int> 79, 54, 71, 69, 99, 90, 60, 77, 87, 91, 58, 72, 75, 100,…
Reshape and plot
# melt() converts wide format (one column per assignment) to long format
# (one row per student-assignment combination) — required for ggplot heatmaps
data_hm_long <- melt(data_hm,
id.vars = "Student_ID",
variable.name = "Assignment",
value.name = "Score")
ggplot(data_hm_long,
aes(x = Assignment, y = Student_ID, fill = Score)) +
geom_tile(color = "white", linewidth = 0.3) +
scale_fill_gradient(low = "#F0FAF6", high = "#0F6E56") +
labs(
title = "Heatmap of Student Progress Across Assignments",
x = "Assignment",
y = "Student ID",
fill = "Score"
) +
theme_minimal() +
theme(
plot.title = element_text(size = 14, face = "bold", hjust = 0.5),
axis.text.x = element_text(angle = 45, hjust = 1),
axis.text.y = element_text(size = 8)
)Task: Copy the ggplot() call from the heatmap-plot chunk above into the blank chunk below. Make two changes:
- Change the
lowandhighcolors inscale_fill_gradient()— for example:low = "white", high = "#993C1D"(red gradient). - Update the
xandylabels inlabs()to something more descriptive than"Assignment"and"Student ID".
Copy the entire ggplot(data_hm_long, ...) block from above. You only need to change two things: scale_fill_gradient() and labs(). Everything else stays the same.
# YOUR CODE HERE — copy the ggplot() call above and apply your two changes
data_hm_long <- melt(data_hm,
id.vars = "Student_ID",
variable.name = "Assignment",
value.name = "Score")
ggplot(data_hm_long,
aes(x = Assignment, y = Student_ID, fill = Score)) +
geom_tile(color = "white", linewidth = 0.3) +
scale_fill_gradient(low = "white", high = "#993C1D") +
labs(
title = "Heatmap of Student Progress Across Assignments",
x = "Work",
y = "Student Number",
fill = "Score"
) +
theme_minimal() +
theme(
plot.title = element_text(size = 14, face = "bold", hjust = 0.5),
axis.text.x = element_text(angle = 45, hjust = 1),
axis.text.y = element_text(size = 8)
)Reflection: How does the color change affect how you interpret the data? Does the red gradient feel different from the green one — and why might that matter when presenting to a principal or department chair?
- [I think especially in education, green is a “good” color and red is a “needs improvement” color. If a student has a low score, you might expect it to be red and if they have a high score, you might expect it to be green or blue. The visuals matter to help everyone understand.]
Question: Looking at the heatmap, identify one student and one assignment that stand out. What would you do as an instructor based on what you see?
- [I am looking at Student 3. In the beginning, Student 3 is getting low scores. As we continue assignments, Student 3 does better. This tells me that Student 3 has growth in these assignments. I also look at assignment 3. There’s several students that got lower scores and some students that got higher scores. It makes me think that maybe we need to go back over Assignment 3 to ensure that all students understand and can apply the information.]
Part 4 · Visualization choice reflection
You have now used five visualization types: scatter plot, bar plot, line plot, histogram, word cloud, heatmap, and SNA plot (this week).
Question 1: For each of the three advanced techniques, describe one specific scenario from your track (K–12 or ID/higher ed) where it would be the most useful choice:
- Word cloud: [When I want to have people fill out a survey about a training I made and see the most used words in the survey.]
- Heatmap: [When I want to see who’s understanding the concepts and who’s maybe falling behind in their assigned trainings.]
- SNA plot: [When I want to see connections- maybe I might want to look at the different departments in a workplace and who talks to who and where there might be gaps because some departments don’t get a chance to talk to other departments throughout the day.]
Question 2: What challenges might you face implementing these visualizations in a real school or institutional setting? Think about data availability, privacy, and stakeholder interpretation.
- [Well especially with privacy, you don’t want to use real names of people and in some cases of projects too. Also, there may not be much data in a corporate setting, especially when it comes to trainings.]
Question 3: How could these techniques evolve in your field? What would be possible if these tools were integrated into an LMS dashboard that teachers or designers could access in real time?
- [If it were integrated into an LMS dashboard, multiple people could have access to the same data and that could help everyone work towards the same goals.]
Render & submit
Step 1 — Add your name
Change the author: field in the YAML header at the top to your name.
Step 2 — Render
Click Render in the toolbar. This file uses several packages (tm, wordcloud, igraph) that produce warnings during preprocessing — that is expected. As long as the final HTML page appears, the render was successful. If you see a true error (red text that stops the render), check that all packages are installed:
install.packages(c("tm", "wordcloud", "RColorBrewer", "reshape2", "igraph"))Step 3 — Publish
| Option | Best for | Link |
|---|---|---|
| Posit Cloud | Quickest — one click from your workspace | Guide |
| RPubs | Free, public, easy to share a link | rpubs.com |
| Quarto Pub | Clean public portfolio pages | Guide |
| GitHub Pages | Best for a professional portfolio | Guide |
This is the most visually impressive of the four files — word clouds, heatmaps, and network graphs are immediately recognizable as advanced data work to anyone reviewing your portfolio. If you are sharing one document from this course with a hiring committee or graduate school application, this is the one to lead with. Pair it with your capstone analysis for the full picture.
Share your published link with your instructor once you have rendered and published. Post in the course discussion board if you run into any technical issues.