Analyse Photo using two Face APIs - Batch extract emotion data from Selfies

Microsoft Azure API

For Azure API, click here to register for an account. In order to use this API, we also need to link the account to a credit card.

A free subscription includes 30,000 transactions per month with 20 calls per Minute.

Azure Face API Pricing

Then, setup the following request URL to get smile and emotion.

face_api_url <- "https://australiaeast.api.cognitive.microsoft.com/face/v1.0/detect?returnFaceAttributes=smile,emotion"

Assign API subscription key.

api_key <- "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"

Read the csv that contains all the photo names in the working directory.

photo_list <- read.csv("Photo List.csv", stringsAsFactors = FALSE)

Using the following code to batch extract emotion data.

# Create an empty dataframe
output <- data.frame()

for (i in 1:nrow(photo_list)){
  # Error handling
  tryCatch({

  # Get the photo with photo list
  body_image <- upload_file(photo_list[i, 1])
  
  # Use POST methord to get result from API
  result <- POST(face_api_url,
                body = body_image,
                add_headers(.headers = c("Content-Type" = "application/octet-stream",
                                         "Ocp-Apim-Subscription-Key" = api_key)))
  
  # Store result of the image to temp in data frame
  temp <- as.data.frame(content(result))
  temp$photo <- photo_list[i, 1]
  
  # Combine result with others
  output <- rbind(output, temp)
  
  # Pause 3 seconds per call, since the limit is 20 calls per mins
  # Otherwise will get errors after 20 calls
  Sys.sleep(3)
  
  },
  
  # Error handling
  error = function(e) {
    writeLines(sprintf("Error at index : %d, error : %s", i, e))
  }
  )
}

# Drop and reorder columns
output <- output %>% 
  select(-faceId, -starts_with("faceRectangle")) %>% 
  select(photo, everything())

# Rename columns
colnames(output) <- c("photo", "smile", "anger", "contempt", "disgust", "fear", "happiness", "neutral", "sadness", "surprise")

# write to a csv file
write.csv(output, "Selfie_Emotions_Azure_API.csv", row.names = FALSE)

Well, it seems I wasn’t very expressive when taking these selfies. Although the API works alright, but the result is not very useful for analysis.

F.A.C.E. API by Sightcorp

The API was mentioned by Yeonsoo in the DSI discussion forum. After testing with their Demo, I found this API performs much better than the Azure one. Click here to register on the website.

F.A.C.E. API by Sightcorp

The free trail only last 2 weeks, with include 5k montly usages with 1 call per second.

Same step, setup the request URL.

face_api_url <- "https://api-face.sightcorp.com/api/detect/"

Assign API subscription key for this API.

api_key <- "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"

Using the following code to batch extract emotion data.

# Create a new dataframe
photo_output <- data.frame()

for (i in 1:nrow(photo_list)){
  # Error handling
  tryCatch({
    
    # Get the photo with photo list
    body_image <- upload_file(photo_list[i, ])
    body_api = list('app_key' = face_api_key, 'img'= body_image)
    
    # Use POST method to get result from API & Use content to convert result into list
    json_pkg <- POST(face_api_url, body = body_api)
    json_content <- content(json_pkg)
    
    # Extract essential features (mood, gender, age, emotions, etc) in a new list
    temp <- c(mood = json_content$people[[1]]$mood, json_content$people[[1]]$emotions)
    # Convert list to data frame
    temp <- as.data.frame(temp) %>% 
      # Add photo name to the data frame
      mutate(photo_id = photo_list[i, ]) %>% 
      # Reorder columns
      select(photo_id, everything())
    
    # Combine result of this photo with others
    photo_output <- rbind(photo_output, temp)
    
  },
  
  # Error handling
  error = function(e) {
    writeLines(sprintf("Error at index : %d, error : %s", i, e))
  }
  )
}

write.csv(photo_output, "Selfie_Emotions_FACE_API.csv", row.names = FALSE)

The final output from F.A.C.E. API by Sightcorp:

Conclusion