Problem Description

• People are becoming more health conscious and interested in tracking their calorie intake.
• An image classifier that can provide nutritional information based on the picture of the food taken could be a unique selling point for calorie-tracking apps.
• We plan to scrape data using the FlickrAPI package to create an image classifier for food items and use it to output nutritional values of the respective food item.

Summary of Peers comments

• The questions raised pertain to the manner in which the model will effectively account for variations in serving sizes and cooking techniques, how it will accurately detect concealed ingredients and quantities of food items, and how it will aptly classify images according to predetermined categories.
• The comments are related to the project proposal for building an AI model to classify food images and estimate nutritional content.
• Suggestions included confining the project objective to image classification of a particular food type, using transfer learning techniques, and ensuring the accuracy and reliability of nutritional information.

Best Suggestion

As image classification model is time consuming to train on, why not consider the use of transfer learning techniques to leverage pre-trained models and improve the training efficiency and accuracy of the food image classification model. By using a pre-trained model, the model can leverage the features learned by the pre-trained model on a large dataset and retrain it on the food image dataset, which can help to reduce the amount of training required to achieve high accuracy. This can lead to faster development and better overall performance of the food image classification model.

Analytics Plan

Data Scraping

• Use getPhotoSearch method from the FlickrAPI package to find the most “interesting” pics under a certain tag and retrieve all metadata about the photos.
• Use download.file method to download the photos scraped URLs.

# Specify the food items to download
food_items <- c("burger", "banana", "apple", "pasta")

.....

# Make the API request for this page
photos <- getPhotoSearch(api_key = "a78a61870fb226f1aa6e348cd78c075e",
                             tags = food_item,
                             extras = "url_o",
                             img_size = "m",
                             per_page = per_page,
                             page = page,
                             sort = "interestingness-desc")

......
    
# Loop through and download 
download.file(url, filename)

Data Summary

Though the target was 22000, due to download issues we got to download 91% of the scraped data.

592 of these images were manually tagged for the purpose of testing.

60% of the data was used for training and the rest 40% for validation.

Data Exploration

• It is a common observation in data collection that as the sample size increases, the quality of the collected data may deteriorate due to the possibility of including more irrelevant or noisy data points.

• During the image scraping process, we conducted an analysis on a sample of 500 images and observed a relevance rate of at least 75%. This indicates that the image data set contained a high proportion of relevant images.

• However, as we scaled up our sample size to 5000 images, we noticed a lower proportion of relevant images. This suggests the possibility of higher levels of noise or irrelevant data in the collected data set, which may adversely impact the quality and validity of the analysis.

## Number of Training image files: 19526

## Number of Test image files: 592

## Number of Image files: 20118

Data Processing

• To preprocess the image data, we will rescale the pixel values of the images to be between 0 and 1.

• Additionally, we will specify a target size of 224 x 224 pixels for resizing the images. This will ensure that all images are of the same size, making it easier for the model to learn from them.

• We will also set the validation_split argument to 0.6, which means that 40% of the data will be used for validation. This will help us evaluate the performance of the model on unseen data and prevent overfitting.

# Get the list of labels for each class
label_list <- dir("train/")
output_n <- length(label_list)
# Save the list of labels to a file
save(label_list, file="label_list.R")

# Set the dimensions for the input images
width <- 224
height<- 224
target_size <- c(width, height)
rgb <- 3 #color channels

# Specify the path to the training data and create a data generator
path_train <- "train/"
train_data_gen <- image_data_generator(rescale = 1/255, 
                                       validation_split = .6)

Model Building

• To create an effective image classification model using transfer learning, we can leverage the power of pre-trained models like Xception which is a convolutional neural network architecture designed by Google researchers as part of the TensorFlow framework.
• By loading the Xception model with weights from the ImageNet dataset, we can take advantage of its powerful feature extraction capabilities.
• To prevent overfitting and improve model generalization, we can freeze the layers of the base model so that they are not updated during training. Finally, we can add some additional layers to fine-tune the model to our specific classification task.

Evaluation

Confusion Matrix

##            true_labels
## pred_labels   0   1   2   3
##      apple  108   1   0   0
##      banana  29 193   1   0
##      burger   0   0 101   0
##      pasta    0   0   1 158

Looking at the matrix, we can see that the model performs well in correctly identifying the apple and pasta classes, as there are no false predictions for these classes. However, it is less accurate in predicting the banana and burger classes, with 29 false predictions for banana and 1 false prediction for burger.

Class level Accuracy, Precision & Recall

## [1] "78.83%" "99.48%" "98.06%" "100%"

## [1] "78.83%" "99.48%" "98.06%" "100%"

## [1] "99.08%" "86.55%" "100%"   "99.37%"

The overall accuracy of the model is 94.6%.

The model has high precision for all classes, ranging from 86.55% for banana to 100% for burger and pasta.

And has high recall for all classes, ranging from 98.06% for banana to 100% for all other classes.

Predicting a random image

Determining the nutritional info

## Food Name:  PASTA

## Nutrients Data:

## It contains 375 calories.