Introduction

In the ever-evolving landscape of the automotive industry, Mitsubishi Motors Krama Yudha Indonesia (MMKI) has emerged as a pioneer in introducing transformative innovations to modern mobility. With an unwavering dedication to quality, MMKI has set trends in the realm of vehicles, providing high-quality solutions to customers worldwide. As a leading manufacturing company, MMKI has solidified its reputation through a commitment to cutting-edge technology, revolutionary design, and excellence in customer service. Through close collaboration with experts and skilled workforces, MMKI continues to achieve milestones, with the recent launch of the new Mitsubishi X-Force model being a prime example of enriching the product line. Within this context, this report will delve into how our “Stock Availability Detection in the Supply Sequence System with Faster R-CNN” project adapts to the challenges posed by the debut of this new model, optimizing the supply chain to meet the ever-changing demands of the market.

Company Profile

  • President Director: Mr. Saito Minoru
  • Total Employee: About 3300 employees
  • Established Year: 2016
  • Area: 510,000 m2
  • Production Capacity: 220,000 units/year

In April 2017, MMKI introduced the Pajero Sport model, followed by the launch of the Expander in September 2017. Shortly thereafter, in May 2018, MMKI commenced the production of the Colt L300 model. The journey continued in February 2019 with the unveiling of the Nissan New Livina within MMKI’s product lineup. Subsequently, in November 2019, MMKI introduced the bold addition of the Xpander Cross to their vehicle family, In November 2020, MMKI achieved a significant milestone by commencing in-house engine assembly / Powertrain. The most recent milestone is the unveiling of the X-Force on August 10, 2023, marking MMKI’s ongoing commitment to deliver innovation and high-quality vehicles to customers worldwide. With each launch, MMKI has consistently reinforced its position as a key player in the automotive industry, setting elevated standards for quality, technology, and customer satisfaction.

Plant Overview

  1. Stamping Stage:
    • Metal sheets are shaped with precision using advanced techniques.
  2. Welding Stage:
    • Components are meticulously integrated through welding processes, forming a strong and robust structure.
  3. Painting Stage:
    • The Painting Stage involves applying a high-quality paint finish to the vehicle’s body, ensuring both aesthetic appeal and protection against external elements.
  4. Assembly Stage:
    • Formed components are seamlessly brought together to create a complete vehicle.
    • Mechanical, electrical, and interior systems are meticulously installed with attention to detail.
  5. Inspection Stage:
    • Every aspect of the vehicle is thoroughly examined to meet established safety and performance standards.
  6. Shipping Stage:
    • The vehicle is carefully packaged and delivered to customers, aiming to provide an exceptional driving experience.

With dedication to each of these stages, MMKI produces vehicles that not only meet high standards but also fulfill customer expectations and needs.

Problem Scope

In addition, within the scope of the problem, we will be implementing a solution to effectively control the inventory of parts specifically in the welding phase. we aim to monitor the stock levels of these welding components with heightened accuracy.

The Workplace Scope of the project focuses on the welding/body area, where a Supply Sequence System empowered by Faster R-CNN will be established. Specifically, within this area, our goal is to develop a robust system that monitors and manages stock in alignment with the production sequence, utilizing the advanced capabilities of Faster R-CNN.

The welding process takes place in the vehicle body manufacturing stage, where previously produced metal components are integrated into a single cohesive structure. This process involves the use of high heat sources, such as electric arcs or gas, to melt the metal material at the joining points. Once the material melts and forms a liquid state, they merge together and create strong bonds as the material cools down and solidifies again.

Problem Statement

With the introduction of the new variant, X-Force, MMKI faces an impending challenge related to stock management. The addition of this variant is anticipated to lead to an increase in stock levels, necessitating dedicated storage space within the production line area.

  • “Pallet Packaging” is used as a designated platform for organizing and storing vehicle parts.
  • “RN”: Mitsubishi Expander
  • “RV”: Mitsubishi X-Force

The constrained space of the Pallet Packaging display area within the production line poses a significant challenge, as it cannot accommodate the presentation of all part variants at the line side. This limitation prompts the need for a solution to optimize the utilization of the available space.

Project Idea

The supply Sequence system holds the potential to minimize stock levels by ingeniously combining multiple variants within a single Pallet Packaging. By adopting the supply sequence system, the arrangement of stock within a single Pallet Packaging can be optimized to follow the production order. This innovative solution not only promises efficient space utilization but also facilitates the process of component retrieval in alignment with the production sequence. The successful implementation of this system will address the challenges arising from increased stock levels and contribute to MMKI’s ongoing efforts to achieve operational excellence and optimal resource utilization.

Pallet Packaging Development

The initial step in the “Pallet Packaging Development” process is to design a pallet capable of accommodating and storing components from various vehicle variants. This pallet is specifically engineered to optimize the arrangement of components, allowing for efficient and organized storage. Considering the differing characteristics among vehicle variants, this pallet design must ensure that each component can be securely placed and easily accessed when needed. This process involves in-depth analysis of the sizes, shapes, and quantities of components from each variant, as well as the application of ergonomic and spatial efficiency principles. By designing a pallet that can cater to the storage needs of different vehicle variants, this first step will establish a strong foundation for further development in creating an effective and organized supply sequence system.

In response to this challenge, it becomes essential to develop Pallet Packaging pallets that can accommodate both RN and RV variants. These specialized pallets are intended to enhance efficiency within the manufacturing line by efficiently utilizing the limited space and facilitating the smooth display of the required part variants.

Following the design phase, the next step involves the actual construction of the Pallet Packaging according to the designed specifications.

Using Faster R-CNN

The core concept of the project revolves around implementing Faster R-CNN to address the challenge of identifying empty stock within the supply sequence system. By harnessing the power of Faster R-CNN, we aim to revolutionize the way we detect and manage empty stock situations in our production process. The idea is to train the Faster R-CNN model to recognize specific visual patterns associated with empty stock, utilizing historical data and real-time visual input from our production line. This proactive approach will enable us to swiftly identify and rectify instances of empty stock, ensuring a continuous and uninterrupted supply sequence. Through the integration of Faster R-CNN-based empty stock detection, our project aspires to enhance operational efficiency, minimize disruptions, and reinforce the reliability of our supply chain management.

Output

The anticipated output of the project is a comprehensive Shiny dashboard that serves as a dynamic control center. This dashboard will offer a real-time display of the daily production sequence, providing insights into the progression of each step in the manufacturing process. Additionally, the dashboard will incorporate the innovative integration of Faster R-CNN and CCTV cameras to monitor stock levels with precision.

Through this Shiny dashboard, production managers and supervisors will have the ability to track the production order’s daily evolution. The real-time updates will allow them to identify any discrepancies or bottlenecks promptly, enabling quick adjustments to maintain optimal efficiency.

Furthermore, the Faster R-CNN-powered stock monitoring feature will provide a visual representation of stock levels, offering a clear indication of when stock is running low or has become depleted. This integration of advanced technology will significantly reduce the risk of stock shortages, ensuring that the production process remains seamless and uninterrupted.

In summary, the project’s output, the Shiny dashboard, will not only enhance daily production visibility but also elevate stock monitoring to a new level of accuracy and efficiency. This integration of data visualization and Faster R-CNN-powered stock tracking promises to transform our supply sequence system into a more responsive, adaptable, and ultimately, more successful operation.

Business Impact

The integration of Faster R-CNN into MMKI’s supply sequence system holds the potential for a significant positive impact on the business operations of MMKI. Through the effective identification and resolution of empty stock situations using advanced visual recognition, MMKI anticipates a substantial decrease in production disruptions and delays. This heightened accuracy in stock monitoring, made possible by Faster R-CNN, is poised to optimize inventory management and streamline the entirety of the production process. As a result, MMKI stands to achieve smoother operations, minimized downtime, and enhanced resource allocation, leading to increased operational efficiency and improved overall performance.

In addition to its immediate benefits, this project holds the potential to be a valuable model for application in other industries seeking effective stock control solutions. The utilization of Faster R-CNN for precise stock monitoring can be adapted across diverse sectors facing similar challenges in inventory management. The framework we develop here can serve as a blueprint for industries such as manufacturing, logistics, and retail, where maintaining optimal stock levels is critical.

By sharing our experience and findings, we can contribute to the advancement of stock control practices in various industries, helping them harness the power of modern technologies to optimize their operations and elevate their performance. Thus, beyond its immediate scope, this project holds the potential to drive positive change across multiple sectors by providing a scalable solution for effective stock control and management.

Explory Data Analysis

The key elements that form the basis of analysis are the images used as inputs for the model. These images serve as independent variables or features that will be processed by our model. They are the canvas carrying vital visual information.

On the other hand, the objects to be identified in these images, such as “cowltop” and “empty_pallet,” are the primary focus of our model. They are the targets we aim to detect and classify within these images.

“cowltop” is one of the components of a car typically located on the car’s dashboard. On the other hand, “empty_pallet” refers to a pallet or packaging frame used to pack a specific component or item. “Empty_pallet” signifies that the pallet is empty or does not contain any stock, meaning there are no components placed on it. In the context of this project, the use of “empty_pallet” is to identify packaging pallets that are empty or devoid of stock.

In this analysis, these images will be learned by our model to recognize and understand the characteristics of these objects. This is a crucial part of the object detection model development process, where we strive to enable the model to automatically identify and classify these objects in diverse images. Thus, these images serve as independent variables, and the target objects are interrelated elements in achieving the goals of this project.

We will utilize EDA techniques to examine the characteristics of the ‘cowltop’ and ‘empty_pallet’ images, explore any variations, and identify potential challenges in the dataset. EDA will enable us to make informed decisions regarding data preprocessing and model development, ensuring that our Faster R-CNN model performs optimally in detecting stock-related components within the Supply Sequence System at PT MMKI.

Image collection

The image collection is done directly at PT MMKI to support the development of the Faster R-CNN model. We collected a total of 120 images for each category, namely “cowltop” and “empty_pallet.” In the context of Faster R-CNN development, rich and representative data is crucial for effectively training the model. Each carefully captured image will be used to train the model to recognize and classify “cowltop” and “empty_pallet” components with high accuracy. With a strong and diverse dataset, we aim to enhance the capabilities of the Faster R-CNN model in efficiently detecting and monitoring stock within the context of the Supply Sequence System at PT MMKI.

Data labeling

“Data labeling” is a pivotal step in machine learning model development, particularly when utilizing technologies like Faster R-CNN or Faster R-CNN for object recognition and stock detection. During this phase, each meticulously collected image is assigned appropriate labels or annotations. For instance, images depicting “cowltop” and “empty_pallet” components are labeled to signify their respective categories or types.

Library

# Data wrangling
library(tidyverse)

# Image manipulation
library(imager)

# Deep learning
library(keras)

# Model Evaluation
library(caret)

# use conda env
#use_condaenv("r-tensorflow")

library(readr)

options(scipen = 999)

Data wrangling (library(tidyverse)): This library is used for data manipulation, data analysis, and data visualization. It includes several packages such as dplyr, ggplot2, tidyr, and others, which are useful for organizing and cleaning data.

Image manipulation (library(imager)): This library is used for image manipulation. It allows you to read, edit, and analyze images in various formats.

Deep learning (library(keras)): This library serves as an interface to the Keras deep learning framework. It is used to build, train, and evaluate deep learning models like neural networks.

Model Evaluation (library(caret)): This library is used for model evaluation. It provides various functions and metrics for measuring the performance of statistical and machine learning models.

Library(readr): This library is used to read data from various sources, such as text files or CSV files, into a data format that can be used for analysis.

Reading Data Image

“Reading Data Image” is used to read files in the training data folder, which contains the images used to train the model. This step involves loading the image data into the model for further processing and analysis.

folder_list <- list.files("data/train/")

folder_list
#> [1] "cowltop"      "empty_pallet"
folder_path <- paste0("data/train/", folder_list, "/")

folder_path
#> [1] "data/train/cowltop/"      "data/train/empty_pallet/"

Inside the ‘train’ directory, you’ll find distinct subfolders for ‘cowltop’ and ‘empty_pallet,’ each dedicated to a specific class or label. These individual subfolders correspond to the respective classes or labels and will be read by the model during the training process.

# Get file name
file_name <- map(folder_path, 
                 function(x) paste0(x, list.files(x))
                 ) %>% 
  unlist()

# first 6 file name
head(file_name)
#> [1] "data/train/cowltop/cowltop_1.JPG"             
#> [2] "data/train/cowltop/cowltop_10 - Copy.JPG"     
#> [3] "data/train/cowltop/cowltop_10.JPG"            
#> [4] "data/train/cowltop/cowltop_11.JPG"            
#> [5] "data/train/cowltop/cowltop_12.JPG"            
#> [6] "data/train/cowltop/cowltop_13 - Copy (10).JPG"

In this script, a function iterates through the provided folder_path variable, which contains the paths to designated folders. For each folder path, the function utilizes the list.files(x) function to retrieve the list of file names within that folder.

Subsequently, these file names are consolidated into a single vector called file_name using the map function and then further flattened into a single list. This list of file names represents the images contained within the specified folders.

Checking Total Images

Understanding the dataset’s size aids in evaluating whether it meets the requirements for machine learning tasks. Adequate data volume is often necessary for training robust and accurate machine learning models.

length(file_name)
#> [1] 200

Next, we will display samples of six images from the dataset for visualization and reference.

# Randomly select image
set.seed(99)
sample_image <- sample(file_name, 6)

# Load image into R
img <- map(sample_image, load.image)

# Plot image
par(mfrow = c(2, 3)) # Create 2 x 3 image grid
map(img, plot)

#> [[1]]
#> Image. Width: 5152 pix Height: 3864 pix Depth: 1 Colour channels: 3 
#> 
#> [[2]]
#> Image. Width: 5152 pix Height: 3864 pix Depth: 1 Colour channels: 3 
#> 
#> [[3]]
#> Image. Width: 5152 pix Height: 3864 pix Depth: 1 Colour channels: 3 
#> 
#> [[4]]
#> Image. Width: 5152 pix Height: 3864 pix Depth: 1 Colour channels: 3 
#> 
#> [[5]]
#> Image. Width: 5152 pix Height: 3864 pix Depth: 1 Colour channels: 3 
#> 
#> [[6]]
#> Image. Width: 5152 pix Height: 3864 pix Depth: 1 Colour channels: 3

Checking Image Dimension

Each image in the dataset comes with its own set of dimensions, including width, height, and color channels. These dimensions need to be consistent across all images for proper model training. Inconsistencies or variations in image dimensions can lead to issues during the training process, affecting the model’s ability to learn and generalize effectively. By examining and standardizing image dimensions, we ensure that the dataset is well-prepared and ready for the subsequent stages of model development. This attention to detail contributes to the overall success and reliability of the machine learning model.

# Full Image Description
img <- load.image(file_name[1])
img
#> Image. Width: 5152 pix Height: 3864 pix Depth: 1 Colour channels: 3
# Image Dimension
dim(img)
#> [1] 5152 3864    1    3

The information provided regarding the image dimensions reveals important insights about the dataset. Here’s an analysis of the details:

  • Width: 5152 pixels
  • Height: 3864 pixels
  • Depth: 1
  • Color Channels: 3

In summary, this image is a high-resolution grayscale image with three color channels, indicating that it may have been transformed into a color image format. Understanding these image characteristics is essential for further processing and analysis, especially in the context of object detection and machine learning tasks.

# Function for acquiring width and height of an image
get_dim <- function(x){
  img <- load.image(x) 
  
  df_img <- data.frame(height = height(img),
                       width = width(img),
                       filename = x
                       )
  
  return(df_img)
}

get_dim(file_name[1])

The Distribution of The Image Dimensions

By analyzing the dimensions (e.g., width and height) of the collected ‘cowltop’ and ‘empty_pallet’ images, we can gain insights into the variability in image sizes within our dataset. This understanding is critical for preparing the data and configuring our detection model, as it helps determine appropriate input dimensions, resizing strategies, and padding methods. A clear grasp of the distribution ensures that our Faster R-CNN model can effectively handle images of various sizes, contributing to the accurate detection of stock-related components within the Supply Sequence System at PT MMKI.

# Randomly get 100 sample images
set.seed(123)
sample_file <- sample(file_name, 1)

# Run the get_dim() function for each image
file_dim <- map_df(sample_file, get_dim)

head(file_dim, 1)
summary(file_dim)
#>      height         width        filename        
#>  Min.   :3864   Min.   :5152   Length:1          
#>  1st Qu.:3864   1st Qu.:5152   Class :character  
#>  Median :3864   Median :5152   Mode  :character  
#>  Mean   :3864   Mean   :5152                     
#>  3rd Qu.:3864   3rd Qu.:5152                     
#>  Max.   :3864   Max.   :5152

Data Augmentation

Data augmentation is a transformative technique that enriches the diversity and volume of a dataset by creating modified versions of existing data samples. This process involves applying a variety of operations, such as rotations, flips, translations, and changes in lighting, to generate new instances that are conceptually similar but exhibit slight variations. Data augmentation is particularly valuable in image analysis and machine learning, where a larger and more diverse dataset often leads to improved model generalization, performance, and robustness.

# Desired height and width of images
target_size <- c(224, 224)

# Batch size for training the model
batch_size <- 2
set.seed(100)
# Image Generator
train_data_gen <- image_data_generator(rescale = 1/255, # Scaling pixel value
                                       horizontal_flip = T, # Flip image horizontally
                                       vertical_flip = T, # Flip image vertically 
                                       rotation_range = 45, # Rotate image from 0 to 45 degrees
                                       zoom_range = 0.25, # Zoom in or zoom out range
                                       validation_split = 0.2 # 20% data as validation data
                                       )

Splitting Training and Validation Data for Model Development

Each image will be represented by 100 instances, partitioned into 80% for training and 20% for validation. This partitioning strategy enables the model to extensively learn from the larger training subset, while reserving a distinct portion for validation to evaluate its performance and generalization abilities.

set.seed(100)

# Training Dataset
train_image_array_gen <- flow_images_from_directory(directory = "data/train/", # Folder of the data
                                                    target_size = target_size,  
                                                    color_mode = "rgb", # use RGB color
                                                    batch_size = batch_size ,
                                                    class_mode = "categorical",
                                                    seed = 100,  # set random seed
                                                    subset = "training", # declare that this is for training data
                                                    generator = train_data_gen
                                                    )

# Validation Dataset
val_image_array_gen <- flow_images_from_directory(directory = "data/train/",
                                                  target_size = target_size, 
                                                  color_mode = "rgb", 
                                                  batch_size = batch_size ,
                                                  seed = 100,
                                                  subset = "validation", # declare that this is the validation data
                                                  generator = train_data_gen
                                                  )
# Load the ggplot2 library
library(ggplot2)

# Create a data frame to store the proportions
data_proportions <- data.frame(
  Dataset = c("Training", "Validation"),
  Proportion = c(length(train_image_array_gen$filenames), length(val_image_array_gen$filenames))
)

# Create a horizontal stacked bar chart
plot <- ggplot(data_proportions, aes(x = Dataset, y = Proportion, fill = Dataset)) +
  geom_bar(stat = "identity") +
  labs(title = "Proportion of Training and Validation Data") +
  theme_minimal() +
  theme(legend.position = "bottom") +
  scale_fill_manual(values = c("Training" = "black", "Validation" = "red")) +  # Set custom colors +
  coord_flip()  # Flip the coordinates to make it horizontal

# Add labels to the bars
plot + geom_text(aes(label = Proportion), hjust = -0.2, vjust = 0.5, size = 4)

Testing Dataset

In the testing dataset, there will be a total of 40 images.

set.seed(100)

folder_list <- list.files("data/test/")

folder_path <- paste0("data/test/", folder_list, "/")

# Get file name
file_name <- map(folder_path, 
                 function(x) paste0(x, list.files(x))
                 ) %>% 
  unlist()

length(file_name)
#> [1] 40
# Function to convert image to array
image_prep <- function(x) {
  arrays <- lapply(x, function(path) {
    img <- image_load(path, target_size = target_size, color_mode = "rgb")
    
    x <- image_to_array(img)
    x <- array_reshape(x, c(1, dim(x)))
    x <- x/255 # rescale image pixel
  })
  do.call(abind::abind, c(arrays, list(along = 1)))
}

data_test <- data.frame(ImageId = paste0(folder_path))
data_test$ImageId <- gsub("/$", "", data_test$ImageId)

test_x <- image_prep(data_test$ImageId)
dim(test_x)
#> [1]  40 224 224   3

After completing the Exploratory Data Analysis (EDA) phase, the next step in the project is to proceed with the creation of the machine learning model. EDA has provided valuable insights into the dataset, including its size, image dimensions, and other important characteristics. Armed with this understanding, the project can move forward to design, train, and evaluate the machine learning model, specifically the Faster R-CNN model. This model will utilize the dataset to learn and recognize objects, specifically “cowltop” and “empty_pallet,” which are vital components of the supply sequence system. The model’s training and validation will play a pivotal role in ensuring that it can accurately detect and classify these components within images, contributing to the success of the Supply Sequence System project at MMKI.