🇷+🎨=🎉

About canvasR and this doc

According to the OED, a “canvasser” is “One who solicits custom, or goes about soliciting orders”.

Ultimately, I would like to write a package that helps people solicit information from the Canvas API, but for the time being, I will be using this to create scripts.

Please find attached a Rmarkdown document that will

create all the scripts you need to do some basic transactions with the Canvas API via R
demonstrate how to use these scripts in your Canvas Sandpit.

Note that I have deliberately not implemented any scripts to delete anything on Canvas.

I think the potential for wanton destruction too great.

Packages you will need to run these scripts

These scripts rely on the current development version of the httr2 package (httr2 version 0.2.3.9000) which you can install as follows

install.packages("remotes")
library(remotes)
install_dev("httr2")

Once you have installed the right version of httr2, you will need the following libraries:

# For tidy data goodness
suppressMessages( library(tidyverse)  )
# For API wrangling:
suppressMessages( library(urltools)   )
suppressMessages( library(httr2)      )
stopifnot(
  "httr2 version 0.2.3.9000 is required but not installed." =
    (packageVersion("httr2")=='0.2.3.9000')
)
# For json juggline
suppressMessages( library(jsonlite)   )
# For working with ...
suppressMessages( library(rlang)      )
# For nice presentation:
suppressMessages( library(kableExtra) )
suppressMessages( library(progress)   )

Function declarations

`my_CanvasAPI_key()`

For this function to work, you will need to have declared an environment variable called QUT_CANVAS_KEY whose value is your Canvas API key.

I have used the approach suggested in Package keys and secrets which is to run
- install.packages("usethis") if you need to
- usethis::edit_r_environ()
- then edit a line that looks like
  QUT_CANVAS_KEY = ")*GU%$DKUOP*(^gp87hP*&G076fgIYRFO&^G{)J754DOUGLIUH;hui;up*&GgR#sref"
- then restart R

# Retrieve your API key as per
# https://httr2.r-lib.org/articles/wrapping-apis.html#package-keys-and-secrets
my_CanvasAPI_key <- function() Sys.getenv("QUT_CANVAS_KEY")

`replace_path_vars(...)`

This helper function transforms paths

like /api/v1/courses/:course_id/users
into /api/v1/courses/<value>/users
when :course_id`=<value> appears in ...

# Transform paths like
#   /api/v1/courses/:course_id/users
# into
#   /api/v1/courses/<value>/users
# when
#   `:course_id`=<value>`
# appears in `...`
replace_path_vars <- function(path, ...){
  dots <- list2(...)
  # Detect where the path variables are in the "..." arguments
  path_vars_in_dots <-     dots |> names() |> str_detect(":[^:]+")
  # If there are no path variables...
  if(!any(path_vars_in_dots)) return(list(path=path, dots=dots))
  # Otherwise get ready to replave them with their values  
  replacements      <-     dots |> keep(path_vars_in_dots) |> map_chr(paste) 
  
  return(
    list(
      # See https://stackoverflow.com/a/60130520/1014385
      path = path |> str_replace_all(replacements),
      dots = dots |> discard(path_vars_in_dots)
    )
  )
}

`GET(path, ...)`

…a helper function to get information from the Canvas API using the REST protocol.

Uses the latest httr2 functions to handle paginated responses.

GET <- function(path, paginated=TRUE, perform=TRUE, progress=TRUE, ...){
  # Replace any path variables with their values
  replaced <- replace_path_vars(path=path, ...)

  req <-
    request("https://canvas.qut.edu.au") |>
    req_auth_bearer_token(my_CanvasAPI_key()) |>
    req_url_path_append(replaced$path) |>
    req_url_query(!!!replaced$dots) 

  if(paginated)
    req <-
      req |> req_paginate_next_url(              # How to get next response
      parse_resp = function(resp){               # How to parse each response
        list(                                    # What httr2 needs from each response
          next_url=resp_link_url(resp, "next"),  # Must have the next_url
          data=resp_body_json(resp)              # Must have data to paginate
        )
      }
    )

  if(!perform)
    return(req)
  if(paginated)
    return(req |> paginate_req_perform(progress=progress))
  else
    return(req |> req_perform() |> resp_body_json())
}

`List_your_courses()`

…returns a tibble containing useful course information from the Canvas API.

See List your courses for additional argnuments

# [List your courses](https://canvas.instructure.com/doc/api/courses.html#method.courses.index)
List_your_courses <- function(...){
  GET("/api/v1/courses/", per_page=100, `include[]`="total_students",...) |>
    (\(.json){
      tibble(
        course_id      = map_int(.json, "id",            .default=NA),
        name           = map_chr(.json, "name",          .default=NA),
        course_code    = map_chr(.json, "course_code",   .default=NA),
        total_students = map_int(.json, "total_students",.default=NA)
      )    
    } 
    )()   
}

`course_id_of(pattern)`

…is a helper function to pull out course_id values where the course name matches pattern.

course_id_of <- function(pattern){
  List_your_courses() |> 
    filter(str_detect(course_code, pattern))  |>
    pull(course_id)
}

`extract_first_name()`

…is a helper function to extract students’ first names from their Canvas short_name

# Add a new column called <name> to <df> by extracting the first word of <col>
extract_first_name <- function(df, col, name="first_name")
  separate_wider_delim(
      data=df,
      cols=enexpr(col),          # Pass col as an expression to be evaluated
      names=c(name, NA), 
      delim=" ", 
      too_few = "align_start", 
      too_many="merge",
      cols_remove = FALSE
    )

`List_users_in_course(course_id, ...)`

…lists all the users in the given course_id

see List users in course for additional arguments you can supply, e.g.,
- enrollment_type[]` = "student" will exclude users who are not students

List_users_in_course <- function(course_id, progress=TRUE, ...){
  GET(
    "/api/v1/courses/:course_id/users",     
    `:course_id`= course_id, 
    progress=progress,
    ...
  ) |>
    (\(.json){
      tibble(
        user_id        = map_int(.json, "id",            .default=NA),
        sortable_name  = map_chr(.json, "sortable_name", .default=NA),
        short_name     = map_chr(.json, "short_name",    .default=NA),
        sis_user_id    = map_chr(.json, "sis_user_id",   .default=NA),
        login_id       = map_chr(.json, "login_id",      .default=NA)
      )    
    } 
    )()
}

`List_assignments(course_id)`

See List assignments for additional arguments

List_assignments <- function(
    course_id, 
    progress=TRUE, tz="Australia/Brisbane", 
    ...
){
  GET(
    "/api/v1/courses/:course_id/assignments",     
    `:course_id`= course_id, 
    progress=progress,
    ...
  ) |>
    (\(.json){
      tibble(
        assignment_id = map_int(.json, "id",    .default=NA),
        name          = map_chr(.json, "name",  .default=NA),
        due_at        = map_chr(.json, "due_at",.default=NA) |> ymd_hms(tz=tz, quiet=TRUE)
      ) 
    } 
    )()
}

`List_assignment_submissions(course_id, assignment_id)`

See List assignment submissions for additional argnuments

List_assignment_submissions <- function(
    course_id, assignment_id,
    progress=TRUE, tz="Australia/Brisbane", 
    ...
){
  GET(
    "/api/v1/courses/:course_id/assignments/:assignment_id/submissions",     
    `:course_id`     = course_id, 
    `:assignment_id` = assignment_id, 
    progress=progress,
    ...
  ) |>
  (\(.json){
    tibble(
      submission_id = map_int(.json, "id", .default=NA),
      submitted_at  = map_chr(.json, "submitted_at", .default=NA) |> ymd_hms(tz=tz, quiet=TRUE),
      assignment_id = map_int(.json, "assignment_id", .default=NA),
      user_id = map_int(.json, "user_id", .default=NA),
      attempt = map_int(.json, "attempt", .default=NA),
      attachments = map(.json, "attachments", .default=NA)
    )
  }
  )() |>
  hoist(                          # Bring these variables up from $attachments[[1]]
    "attachments", 
    attachment_id=list(1, "id"),
    filename=list(1, "filename")
  ) |>
    select(-attachments)
}

`PUT(path, perform=TRUE, ...)`

…a helper function to put information from the Canvas API using the REST protocol:

PUT <- function(path, perform=TRUE, ...){
  # Replace any path variables with their values
  replaced <- replace_path_vars(path=path, ...)

  req <-
    request("https://canvas.qut.edu.au") |>
    req_method("PUT") |>
    req_auth_bearer_token(my_CanvasAPI_key()) |>
    req_url_path_append(replaced$path) |>
    req_url_query(!!!replaced$dots) 

  if(!perform)
    return(req)
  else
    return(req |> req_perform() |> resp_body_json())
}

`all_NULLs_to_NAs()`

…a helper function to convert NULL values to NA in responses from the Canvas API.

# Helper function to convert all NULLs to NAs in a list so that it can be more
# readily converted into a data frame
# See https://stackoverflow.com/a/65129784/1014385
all_NULLs_to_NAs <- function(.list){
  .list |> 
    modify_tree(
      leaf=function(x){
        ifelse(is.null(x), NA, x)
      }
    )
}

`Grade_or_comment_on_a_submission()`

See Grade or comment on a submission for additional arguments

Grade_or_comment_on_a_submission <- function(
    course_id, assignment_id, user_id, attempt, 
    tz="Australia/Brisbane", ...
){
  PUT(
    "/api/v1/courses/:course_id/assignments/:assignment_id/submissions/:user_id",     
    `:course_id`       = course_id, 
    `:assignment_id`   = assignment_id,
    `:user_id`         = user_id,
    `comment[attempt]` = attempt,
    ...
  )|>
  all_NULLs_to_NAs() |>
  (
    function(.json){
      with(
        .json,
        tibble(
          submission_id = id,
          submitted_at  = submitted_at |> ymd_hms(tz="Australia/Brisbane", quiet=TRUE),
          assignment_id = assignment_id,
          user_id       = user_id,
          grade         = grade,
          score         = score,
          attempt       = attempt,
          comment = tibble(
            id          =    submission_comments |> map_int("id")          |> list(),
            comment     =    submission_comments |> map_chr("comment")     |> list(),
            author_id   =    submission_comments |> map_int("author_id")   |> list(),
            author_name =    submission_comments |> map_chr("author_name") |> list()
          ),
        )
      )
    }
  )()
}

`POST(path, perform=TRUE, ...)`

…a helper function to get information from the Canvas API using the REST protocol:

POST <- function(path, perform=TRUE, ...){
  # Replace any path variables with their values
  replaced <- replace_path_vars(path=path, ...)

  req <-
    request("https://canvas.qut.edu.au") |>
    req_method("POST") |>
    req_auth_bearer_token(my_CanvasAPI_key()) |>
    req_url_path_append(replaced$path) |>
    req_url_query(!!!replaced$dots) 

  if(!perform)
    return(req)
  else
    return(req |> req_perform() |> resp_body_json())
}

`Upload_submission_comments_file()`

This function will

upload a file that can be attached to a submission comment
- See Submission Comments API: Upload a file
- Return a list whose $id is the file_id of the uploaded file so it can be attached to a comment using Grade_or_comment_on_a_submission()

Upload_submission_comments_file <- function(
    course_id, assignment_id, user_id, canvas_filename, local_filepath
){
  # [Uploading via POST](https://canvas.instructure.com/doc/api/file.file_uploads.html#method.file_uploads.post)
  # Step 1: Telling Canvas about the file upload and getting a token
  response1 <- 
    POST(
      path="/api/v1/courses/:course_id/assignments/:assignment_id/submissions/:user_id/comments/files",
      `:course_id`       = course_id, 
      `:assignment_id`   = assignment_id,
      `:user_id`         = user_id,
      name               = canvas_filename,
      size               = file.info(local_filepath)$size,
      on_duplicate       = "overwrite"
    )
  
  # Step 2: Upload the file data to the URL given in the previous response
  response2 <-
    request(response1$upload_url) |>
    req_method("POST") |>
    req_body_multipart(
      filename      =response1$upload_params$filename,
      content_type  =response1$upload_params$content_type,
      file          =curl::form_file(local_filepath)      
    ) |>
    req_perform()
  
  # Step 3: Confirm the upload's success
  request(response2$headers$Location) |>
    req_method("PUT") |>
    req_auth_bearer_token(my_CanvasAPI_key()) |>
    req_perform() |>
    resp_body_json() 
}

`List_submission_comments()`

This function will list every comment (and every file attached to it) on every submission to an assignment with assignment_id in a course with course_id

List_submission_comments <- function(
    course_id, assignment_id,
    progress=TRUE, tz="Australia/Brisbane", 
    ...
){
  GET(
    "/api/v1/courses/:course_id/assignments/:assignment_id/submissions",     
    `:course_id`     = course_id, 
    `:assignment_id` = assignment_id, 
    `include[]` = "submission_comments",
    progress=FALSE
  )  |>
    (\(.json){
      tibble(
        submission_id       = map_int(.json, "id",                  .default=NA),
        submitted_at        = map_chr(.json, "submitted_at",        .default=NA),
        assignment_id       = map_int(.json, "assignment_id",       .default=NA),
        user_id             = map_int(.json, "user_id",             .default=NA),
        submission_comments =     map(.json, "submission_comments", .default=NA)
      )
    }
    )() |>
    unnest(submission_comments) |>           # Expand this list column then...
    hoist(                                   # ...hoist the following columns up and out of... 
      submission_comments,                   # this list column:
      comment_id            = "id",
      attempt               = "attempt",
      comment               = "comment",
      comment_author_id     = "author_id",
      comment_author_name   = "author_name",
      comment_created_at    = "created_at",
      comment_edited_at     = "edited_at",
      attachments           = "attachments",
      .ptype=list(edited_at=character())     # ensuring this named column has this type
    ) |>  
    select(-submission_comments) |>
    unnest(attachments, keep_empty=TRUE) |>  # Expand the this list column then...
    hoist(                                   # ...hoist the following columns up and out of...
      attachments,                           # this list column:
      file_id               ="id",
      file_display_name     ="display_name",
      file_size             ="size",
      file_created_at       ="created_at",
      file_url              ="url") |>
    select(-attachments) |>
    mutate(
      submitted_at          = ymd_hms(submitted_at,       tz=tz, quiet=TRUE),
      comment_created_at    = ymd_hms(comment_created_at, tz=tz, quiet=TRUE),
      comment_edited_at     = ymd_hms(comment_edited_at,  tz=tz, quiet=TRUE),
      file_created_at       = ymd_hms(file_created_at,    tz=tz, quiet=TRUE)
    ) 
}

Function demonstrations

List your courses

List_your_courses() -> My.courses

My.courses |> head(6) |> kbl() |>   kable_minimal(full_width = F)

course_id	name	course_code	total_students
4089	DSB100_21se2 Fundamentals of Data Science	DSB100_21se2	0
12878	DSB100_22se2 Fundamentals of Data Science	DSB100_22se2	0
14567	DSB100_23se2 Fundamentals of Data Science	DSB100_23se2	116
2792	EGH400-1_23se1 Research Project 1	EGH400-1_23se1	418
14584	EGH400-1_23se2 Research Project 1	EGH400-1_23se2	145
2796	EGH400-2_23se1 Research Project 2	EGH400-2_23se1	149

Get course info

# Get the course_id of interest
course_id_of("Sandpit") -> Sandpit

Get assignment info

Note that for the following demo code to run you must

create and publish an assignment (called Test Assignment) in your Sandpit
switch to Student View then
add a submission to the Test Assignment.

# List the assignments from the course of interest
List_assignments(Sandpit) -> Sandpit.assignments

Sandpit.assignments |>  kbl() |>   kable_minimal(full_width = F)

assignment_id	name	due_at
152713	Test Assignment	NA

Get user info

# Get a dataframe of users (students, teachers, others) in that course
List_users_in_course(
  Sandpit,                                # The course of interest
  # Uncomment the following line to remove teaching team, test students, etc
  # `enrollment_type[]`="student",        # This excludes users who are not students
  progress=FALSE                          # Suppress progress bar
) |>
  extract_first_name(short_name)  ->      # Add first_name the the result
  Sandpit.students

Sandpit.students |>  kbl() |>   kable_minimal(full_width = F)

user_id	sortable_name	first_name	short_name	sis_user_id	login_id
1626	Lovell, David	David	David Lovell	1831953	lovelldr

Get assignment info

# Pull out the assignment_id of a named assignment
(
Test_Assessment <- 
  Sandpit.assignments |>
  filter(name=="Test Assignment") |>
  pull(assignment_id)
)

[1] 152713

Get assigment submissions info

# List the submissions made to the specified course assignment
List_assignment_submissions(Sandpit, Test_Assessment)  -> Submissions

Submissions |>  filter(!is.na(attempt)) |> kbl() |>   kable_minimal(full_width = F)

submission_id	submitted_at	assignment_id	user_id	attempt	attachment_id	filename
4627314	2023-09-30 08:13:31	152713	74948	2	NA	NA

# Pull out the user_id of a student who made a submission
(
  # User: just take the first submission's user_id
  Test_Student <- Submissions |> head(1) |> pull(user_id)
)

[1] 74948

# Pull out the number of attempts of a student who made a submission
(
  # User: just take the first student's final attempt at the submission
  Final_Attempt <- Submissions |> head(1) |> pull(attempt)
)

[1] 2

Upload submission comments file

To upload a submission comments file, we need to provide

the
- course_id (Sandpit = 10245),
- assignment_id (Test_Assessment = 152713) and
- user_id (Test_Student = 74948) of the student that the file will go to
a canvas_filename = test.2023.10.22.13.57.07.html for the file
the local_filepath = ./tmp/test.2023.10.22.13.57.07.html to the file we want to upload

and then we can…

Upload_submission_comments_file(
  course_id       = Sandpit, 
  assignment_id   = Test_Assessment, 
  user_id         = Test_Student, 
  canvas_filename = canvas_filename,
  local_filepath  = local_filepath) -> File.upload.response

File.upload.response$id = 3107093 is the Canvas file_id that we need to pass to Grade_or_comment_on_a_submission() so that the file we have just uploaded gets attached to a comment we make on the Test_Student’s submission.

Grade and comment on an attempt

To attach a grade, comment and file to a specific student’s attempt, we need to provide the

course_id (Sandpit = 10245),
assignment_id (Test_Assessment = 152713) and
user_id (Test_Student = 74948) of the student
attempt (Final_Attempt = 2) that we want to grade, etc
numeric score we want to give to their attempt (Score = 8.309751)
comment we want to add to their attempt (Comment)
Canvas file_id (File.upload.response$id = 3107093) of a file we have uploaded for the student in relation to the Test_Assessment
a canvas_filename = test.2023.10.22.13.57.07.html for the file
the local_filepath = ./tmp/test.2023.10.22.13.57.07.html` to the file we want to upload

and then we can

Grade_or_comment_on_a_submission(
  course_id                  = Sandpit, 
  assignment_id              = Test_Assessment, 
  user_id                    = Test_Student, 
  attempt                    = Final_Attempt, 
  `submission[posted_grade]` = Score,
  `comment[text_comment]`    = Comment,
  `comment[file_ids][]`      = File.upload.response$id) -> Grade.upload.response

To see the comments, grades and file attachment information, you will have to

Go to your SandPit
Open up the Gradebook (it may be called Markbook)
Open the Test Assignment submission you created as the Test Student
Open the submission in Speed Grader.

List comments on submissions

List_submission_comments(Sandpit, Test_Assessment) -> Submission_comments

Submission_comments |>
  filter(!is.na(attempt)) |>
  select(
    assignment_id, submission_id, attempt,comment_author_name, comment, file_id, file_display_name
  ) |>
  kbl() |>   kable_minimal(full_width = F)

assignment_id	submission_id	attempt	comment_author_name	comment	file_id	file_display_name
152713	4627314	1	Test student	A comment on my first attempt	NA	NA
152713	4627314	2	Test student	Comment #1 on my second attempt	NA	NA
152713	4627314	2	David Lovell	A comment made via canvasR at 2023.10.16.13.25.20	3070059	test.2023.10.16.13.25.20.html
152713	4627314	2	David Lovell	A comment made via canvasR at 2023.10.21.18.30.51	3103872	test.2023.10.21.18.30.51.html
152713	4627314	2	David Lovell	A comment made via canvasR at 2023.10.22.13.04.36	3106809	test.2023.10.22.13.04.36.html
152713	4627314	2	David Lovell	A comment made via canvasR at 2023.10.22.13.22.08	3106895	test.2023.10.22.13.22.08.html
152713	4627314	2	David Lovell	A comment made via canvasR at 2023.10.22.13.42.18	3107020	test.2023.10.22.13.42.18.html
152713	4627314	2	David Lovell	A comment made via canvasR at 2023.10.22.13.50.32	3107064	test.2023.10.22.13.50.32.html
152713	4627314	2	David Lovell	A comment made via canvasR at 2023.10.22.13.57.07	3107093	test.2023.10.22.13.57.07.html

canvasR-dev

Scripts for interacting with Canvas

David Lovell

2023-10-22

🇷+🎨=🎉

About canvasR and this doc

Packages you will need to run these scripts

Function declarations

`my_CanvasAPI_key()`

`replace_path_vars(...)`

`GET(path, ...)`

`List_your_courses()`

`course_id_of(pattern)`

`extract_first_name()`

`List_users_in_course(course_id, ...)`

`List_assignments(course_id)`

`List_assignment_submissions(course_id, assignment_id)`

`PUT(path, perform=TRUE, ...)`

`all_NULLs_to_NAs()`

`Grade_or_comment_on_a_submission()`

`POST(path, perform=TRUE, ...)`

`Upload_submission_comments_file()`

`List_submission_comments()`

Function demonstrations

List your courses

Get course info

Get assignment info

Get user info

Get assignment info

Get assigment submissions info

Upload submission comments file

Grade and comment on an attempt

List comments on submissions