Resume Ranker with NLP

Introduction

Background

Proses rekrutmen dalam dunia kerja semakin kompleks dan kompetitif. Perusahaan-perusahaan menghadapi tuntutan untuk mengidentifikasi kandidat yang tepat dengan cepat dan efisien. Namun, pengelolaan resume secara manual sering kali memakan waktu dan tidak efektif. Respon terhadap berbagai lowongan pekerjaan menghasilkan tumpukan resume yang sulit dielola dan diseleksi.

Tantangan ini mendorong inovasi dalam pengelolaan resume. Penggunaan teknologi Natural Language Processing (NLP) menawarkan peluang untuk mengotomatisasi dan mengoptimalkan proses ini. Ide proyek ini adalah mengembangkan alat berbasis NLP yang memungkinkan perusahaan untuk mengelola dan mencari resume dengan lebih efektif. Dengan mengidentifikasi keterampilan, pengalaman, dan atribut kunci dalam resume, alat ini akan membantu perusahaan menemukan kandidat terbaik secara cepat dan cerdas.

Pemanfaatan teknologi NLP dalam manajemen resume adalah langkah progresif yang memiliki potensi untuk meningkatkan efisiensi, kualitas, dan keakuratan rekrutmen. Dengan solusi ini, perusahaan akan dapat lebih baik mengarahkan sumber daya mereka untuk fokus pada kandidat yang memiliki potensi tertinggi sesuai dengan kebutuhan organisasi.

Dengan demikian, proyek ini bertujuan untuk menghadirkan solusi inovatif dalam proses rekrutmen yang sesuai dengan perkembangan teknologi saat ini dan kebutuhan bisnis yang semakin dinamis.

Problem Statement

Dalam proses rekrutmen saat ini, pengelolaan dan analisis resume menjadi tantangan yang signifikan bagi perusahaan. Banyak resume dari berbagai sumber menghasilkan tumpukan data yang rumit dan memakan waktu. Identifikasi kandidat yang paling sesuai dengan kriteria pekerjaan seringkali melibatkan upaya manual yang rentan terhadap kehilangan informasi kunci maupun bias. Metode manual ini tidak hanya tidak efisien, tetapi juga dapat mengakibatkan kehilangan peluang untuk mengidentifikasi bakat yang sebenarnya.

Kondisi ini mengindikasikan adanya kebutuhan untuk pengelolaan resume yang lebih cerdas dan otomatis. Secara spesifik, perusahaan memerlukan solusi yang mampu melakukan analisis dan pencarian terstruktur berdasarkan keterampilan, pengalaman, dan atribut penting lainnya dalam resume. Dengan cara ini, perusahaan akan mampu mengidentifikasi calon karyawan yang paling sesuai dengan persyaratan pekerjaan dengan lebih cepat dan akurat.

Oleh karena itu, diperlukan solusi inovatif berbasis Natural Language Processing (NLP) yang dapat mengatasi masalah pengelolaan resume secara efisien dan efektif. Solusi ini diharapkan mampu mengubah tumpukan data resume menjadi informasi yang bermakna dan memberikan wawasan yang lebih dalam dalam proses rekrutmen. Dengan waktu yang dan tenaga yang tersimpan oleh inovasi ini, perusahaan dapat meningkatkan produktivitas tim rekrutmen dan fokus pada pengambilan keputusan strategis.

Dengan merumuskan permasalahan ini dengan jelas, proyek ini akan mengarah pada pengembangan alat yang memberikan solusi yang lebih cerdas dan efisien dalam pengelolaan resume, menghasilkan dampak positif dalam proses rekrutmen, serta mengoptimalkan upaya dan sumber daya perusahaan.

Project Idea

Optimasi Pencarian Calon Karyawan dengan Teknologi Natural Language Processing (NLP). Dengan itu, proyek ini bertujuan untuk mengembangkan sebuah sistem yang menggunakan Teknologi Natural Language Processing (NLP) untuk mengoptimalkan pencarian calon karyawan yang paling sesuai dengan kebutuhan perusahaan.

Work Flow

1. Data Preprocessing:

• Mengumpulkan banyak resume dalam bentuk folder yang berisi kumpulan PDF resume dari pelamar.
• Mengonversi semua berkas PDF menjadi teks mentah.
• Membuat dataframe berisi dua kolom yaitu file_name (nama file pdf) dan text (teks mentah resume).

2. Text Preprocessing:

• Mengubah teks masukan menjadi korpus teks.
• Mengubah semua teks menjadi huruf kecil.
• Menghapus stop-words.
• Menghapus tanda baca.
• Menerapkan stemming teks untuk mengurangi kata-kata ke bentuk dasarnya.
• Menghapus spasi ekstra.

4. Resume Ranking:

• Membuat document term matrix resume.
• Mengkonversi document term matrix resume menjadi tfidf matrix.
• Mengkonversi tfidf matrix menjadi cosine similarity matrix
• Mengurutkan resume berdasarkan kemiripan terhadap kriteria ideal
• Membuat dataframe rank resume yang berisi dua kolom yaitu Resume (nama file pdf) dan Similarity (Kesamaan terhadap kriteria ideal)

Output

Output dari project ini berupa API berbasis shiny apps yang dapat digunakan dengan cara meng-submit folder kumpulan resume dalam bentuk PDF, kemudian memasukkan input kriteria ideal yang diinginkan. Yang kemudian akan menghasilkan output berupa table rank resume

Business Impact:

• Meningkatkan efisiensi dalam seleksi calon karyawan dengan mengotomatisasi proses ranking.
• Menghemat waktu dan usaha tim HR dalam penyaringan berkas resume.
• Memberikan informasi lebih lanjut tentang tingkat kesesuaian keterampilan antara pelamar dan kriteria pekerjaan.
• Memungkinkan tim HR untuk fokus pada kandidat yang paling sesuai dengan kebutuhan perusahaan.

Code Preparation

Pacakge yang digunakan

library(dplyr)
library(tm)
library(NLP)
library(proxy)
library(NLP)
library(pdftools)
library(data.table)
library(tools)
library(tidytext)
library(textclean)
library(tibble)
library(stringr)
library(wordcloud)

Fungsi pembersih teks

Fungsi cleanText() ini akan digunakan untuk membersihkan data teks. Tahap pemrosesan yang dilakukan adalah:

1. Mengubah teks masukan menjadi korpus teks.
2. Mengubah semua teks menjadi huruf kecil.
3. Menghapus stop-words.
4. Menghapus tanda baca.
5. Menerapkan stemming teks untuk mengurangi kata-kata ke bentuk dasarnya.
6. Menghapus spasi ekstra.
7. Me-return korpus teks yang telah dibersihkan.

cleanText <- function(text, lang = "en", as.corpus = T){
  text_corpus <- text %>% VectorSource() %>% VCorpus()
  
  text_corpus <- tm_map(x = text_corpus,
                          FUN = content_transformer(tolower))

  text_corpus <- tm_map(x = text_corpus,
                        FUN = removeWords,
                        stopwords(kind = lang))

  text_corpus <- tm_map(x = text_corpus,
                        FUN = removePunctuation)

  text_corpus <- tm_map(x = text_corpus,
                        FUN = stemDocument)

  text_corpus <- tm_map(x = text_corpus,
                        FUN = stripWhitespace)
  
  if (as.corpus){
    return(text_corpus)
  }
  else(
    return(sapply(text_corpus, as.character))
  )
}

Fungsi convert folder ke table

Fungsi folder_to_table ini akan digunakan untuk mengubah folder menjadi table yang terdiri dari dua kolom, yaitu kolom file_name yang berisi nama file pdfnya, dan kolom text yang berisi teks mentah dari pdf tersebut yang sudah dibersikan. Fungsi ini menerima path folder sebagai input kemudian me-return table dalam bentuk data frame

folder_to_table <- function(folder_path) {
  # Function to convert PDF to text
  convert_pdf_to_text <- function(pdf_path) {
    pdf_text_content <- pdf_text(pdf_path)
    
    extracted_text <- list()
    
    for (page in seq_along(pdf_text_content)) {
      text <- pdf_text_content[[page]]
      extracted_text[[page]] <- text
    }
    
    all_text <- paste(extracted_text, collapse = "\n")
  }
  
  # Function to get file name without extension
  get_file_name <- function(file_path) {
    file_path_sans_ext(basename(file_path))
  }
  
  # Get PDF files from the specified folder
  pdf_files <- list.files(folder_path, pattern = ".pdf", full.names = TRUE)
  
  # Convert PDFs to text
  pdf_texts <- lapply(pdf_files, convert_pdf_to_text)
  
  # Create a data table with file names and extracted text
  table_data <- data.table(
    file_name = paste(sapply(pdf_files, get_file_name), ".pdf", sep = ""),
    text = unlist(pdf_texts)
  )
  
  return(table_data)
}

Fungsi resume ranker

Fungsi rank_resume() ini akan digunakan untuk me-rank resume berdasarkan tingkat kemiripannya dengan kriteria yang diinginkan. Fungsi ini menerima input kriteria ideal dan juga data frame resume, kemudian mengeluarkan output berupa rank resume berdasarkan tingkat kemiripannya dengan kriteria yang diinginkan.

Tahapan proses ranking yang dilakukan adalah:

1. Membuat document term matrix resume.
2. Mengkonversi document term matrix resume menjadi tfidf matrix.
3. Mengkonversi tfidf matrix menjadi cosine similarity matrix
4. Mengurutkan resume berdasarkan kemiripan terhadap kriteria ideal
5. Me-return rank resume dalam bentuk data frame

rank_resume <- function(ideal_criteria, resume_df) {
  # Merge ideal keywords with applicant resumes
  all_resumes <- c(ideal_criteria, resume_df$text)
  
  # Clean all resumes
  all_resumes <- cleanText(all_resumes,
                           lang = "en")
  
  # Create document term matrix
  dtm_matrix <- DocumentTermMatrix(all_resumes)
  
  # Convert to weight tfidf matrix
  tfidf_matrix <- dtm_matrix %>% 
    weightTfIdf() %>% 
    as.matrix()
  
  # Create similarity matrix
  cosine_sim_matrix <- simil(tfidf_matrix, method = "cosine") %>% 
    as.matrix()
  
  # Extract cosine similarity values for applicants (excluding HR resume)
  resume_similarities <- cosine_sim_matrix[2:nrow(cosine_sim_matrix), 1]
  
  # Rank applicants based on similarity scores (higher similarity, better rank)
  resume_ranking <- order(resume_similarities, decreasing = TRUE)
  
  # Create data frame of the rank
  rank_df <- data.frame(
    Rank = integer(0),
    Resume = character(0),
    Similarity = double(0)  # Changed to 'double' for floating-point values
  )
  
  # Print rank with file name
  for (count in seq_along(resume_ranking)) {
    resume <- resume_ranking[count]
    similarity <- round(resume_similarities[resume], 2)
    file_name <- resume_df$file_name[resume]
    
    rank <- data.frame(
      Rank = count,
      Resume = file_name,
      Similarity = similarity
    )
    
    rank_df <- rbind(rank_df, rank)
  }
  
  # Set the "Rank" column as row names
  rownames(rank_df) <- rank_df$Rank
  
  # Remove the "Rank" column (optional)
  rank_df <- rank_df[, -1]
  
  return(rank_df)
}

Fungsi wordcloud matching criteria

Fungsi matching_criteria_cloud() ini akan digunakana untuk mengeluarkan wordcloud, yang di mana menampung words yang matching antara sebuah resume dengan input kriteria ideal.

Fungsi ini menerima input kriteria ideal dan sebuah text resume

matching_criteria_cloud <- function(ideal_criteria, resume){
  
  unlist_ideal_criteria <- unlist(str_split(tolower(ideal_criteria), "\\W+"))
  unlist_resume<- unlist(str_split(tolower(resume), "\\W+"))
  
  intersection <- intersect(unlist_ideal_criteria, unlist_resume)
  
  words <- data.frame(word = intersection) %>% 
    count(word, sort = T)
  
  words %>%
    with(
      wordcloud(
        words = word,
        colors = "black", # Generate random colors for each unique word
        scale = c(2.5, 2.5), 
        random.order = F,
        max.words = nrow(words)
      )
    )
}

Function Testing

Testing fungsi folder_to_table()

designers <- folder_to_table("DESIGNER")

Bandingkan hasil ekstrak dengan resume asli

Resume asli

knitr::include_graphics("10751444.png")

Hasil ekstrak

designers[designers$file_name == "10751444.pdf", ]$text

## [1] "PRODUCT DESIGNER\nProfessional Summary\n4-5 years engineering experience and 1-2 years working experience. Able to work independently and under pressure, detail oriented, excellent\nproblem solver, Innovator. Efficient Mechanical Engineer leveraging a strong technical background in bringing products from the laboratory to\nmass-manufacturing. Mechanical Engineer with [Number] + years of training in varied industries, including manufacturing and high-tech\nenvironments. Creative manufacturing engineer. Lead team member on process redesign for [Describe product] . Design engineer who has worked\non [Number] new products, including the [Product name] recognized for industry excellence.\nSkills\n       CAD\n       Complex problem solving\n       Stress analysis training\n       Component functions and\n       testing requirements         Engine components, pumps, and fuel systems knowledgeFEA toolsAutoCAD proficientTeam\n       Technical direction and leadershipManufacturing systems integrationManufacturing systems integration\n       product strategies\n       Works well in diverse team\n       environment\n       Strong decision maker\n\nWork History\nProduct designer 10/2014 to Current\nCompany Name – City , State\n      The team wants to develop a portable, easily shipped, cost effective hardware that can send and receive digital content directly from\n      satellites.\n      Personally involve with prototype designing and 3D modeling.\n      Cooperating with a startup called Outernet (https://www.outernet.is/en/), a for-profit media company that already has two satellites covering\n      North America, Europe, and the Middle East and has recently started broadcasting free Internet content.\n      Assisting drafters in developing the structural design of products using drafting tools or computer-assisted design (CAD) or drafting\n      equipment and software.\n      Completing project mechanical design while providing technical solutions feedback.\nproduct design 09/2014 to Current\nCompany Name – City , State\n      Two engineers and designers to collaborate together to create new innovative wearable pieces for a fashion show competition.\n      Will access new Makerspce, which includes a 3D printer, will be given a $500 budget to create their wearable piece.\nRESEARCH EcoPRT Research Assistant 01/2014 to 05/2014\nCompany Name – City , State\n      The goal is to develop an economical, automated transit system.\n      It will focus on the hands on design and development of a small manned autonomous vehicle.\n      www.ecoprt.com).\n      The key in the design is to understand the impact weight has on the overall cost and performance, and the incorporation of automated\n      control.\n      Aspects of the development will possibly include\nproduct design 01/2014 to 05/2014\nCompany Name – City , State\n      VOLUNTEER The purpose of this project is to design and fabricate a cable management system for a public-access electric\n      EXPERIENCE vehicle charging station.\n      This system will dispense and retract 20 feet of cable for operation and provide secured storage for the cable when not in use.\n      The prototype will be subjected to the following constraints\nTeam member 10/2013 to 04/2014\nCompany Name – City , State\n      Attending scheduled control and mechanical teams' training classes.\n      EXPERIENCE · Learned shop safety, vehicle glider equations, drive cycle modeling, and Simulation.\n      Learned the powertrain architecture and components of the 2013 Chevrolet Malibu.\n      Learned vehicle dynamics.\n      And practiced model simulation by using MATLAB Simulink.\n      Mechanical Engineering Components design project (material design.\nmaterial design 10/2013 to 04/2014\n\nCompany Name – City , State\n      Designed fillet welds connections and bolts for the plate girder, which holds the pipe with horizontal and vertical force loads.\n      Calculated the related shear or bending stresses for the welds and bolts to determine the right materials and sizes of welds (thickness) and\n      bolts.\nEddy Current DYNO Research Assistant 09/2013 to 05/2014\nCompany Name – City , State\n      Built the engine stander for our engine and Eddy current dynamometer.\n      Currently installing the Eddy current dynamometer with graduate students.\n      Future possibility of experimenting with torque, horsepower, RPM, EGR (Exhaust Gas Recirculation) and temperature measurements of the\n      Kubota Diesel Engine after installation.\n      Possibility of learning the engine tuning.\nResearch Assistant 06/2013 to 08/2013\nCompany Name – City , State\n      Graphed sketches and figures for professor's Thermodynamics eBook.\n      Learned how to use Smartdraw.\n      Performed literature reviews on ongoing research topics and eBook materials.\n      Added video links and real-world images to the eBook.\nProgram Assistant 05/2013 to 06/2013\nCompany Name – City , State\n      Assisting Dr.\n      Eischen, the director of the Hangzhou Engineering Study Abroad Program at Zhejiang University, during his program this coming summer.\n      Helping with tasks such as translating, program activities, running errands, classes, transportation, and culture immersion.\n2323 04/2013 to 10/2013\nCompany Name – City , State\n      Designed Airplane Landing Gear by modeling with a mass-spring-damper SDOF system and designing the spring k and damper C that\n      limits the given amplitude.\n      Part 2\nwew 10/2012 to 04/2013\nCompany Name – City , State\n      Utilized MATLAB for statistical analysis of an elastic band rocket.\n      Learned how to make experimental designs, statistical processes, statistics simulations, and graphical displays of data on computer\n      workstations.\n      Used statistical methods including point and interval estimation of population parameters and curve and surface fitting (regression analysis).\n      Graphic Communications Project (3D design.\nrer 10/2012 to 04/2013\nCompany Name – City , State\n      Utilized SolidWorks to design a tape floss container.\n      Developed the ability to use SolidWorks within the context of a concurrent design process to understand how everyday objects are\n      designed and created.\n      Emphasis placed on decision-making processes involving creating geometry and the development of modeling strategies that incorporate the\n      intentions of the designer.\nre 02/2009 to 04/2009\nCompany Name – City , State\n      Visited construction sites with senior engineers.\n      Kept record of site investigations.\n      Dealt with paperwork with senior engineers and answered phone calls.\n      Helped install residential wiring in new construction sites.\n      Investigated electrical problems and developed the ability to read electrical diagrams and wire electrical panels.\nEducation\nMaster of science : Mechanical engineering Robotic & Manufacture Current Columbia University in the City of New York - City , State\n      Sep -2015 Dec Mechanical engineering Robotic & Manufacture\n\n       Coursework in Advanced Mechanical Engineering\n       Coursework in Drafting, Computer-Aided Design (CAD) and Computer-Aided Manufacturing (CAM)\nBachelor of science : Mechanical Engineering 1 2010 North Carolina State University, Raleigh (NCSU) - City , State\nGPA: Magna Cum Laude GPA: 3.5 GPA: 3.63/4.0 Mechanical Engineering Magna Cum Laude GPA: 3.5 GPA: 3.63/4.0\nNorth Carolina State University -\nGPA: Magna Cum Laude Magna Cum Laude\nAccomplishments\n       Listed in the dean's list for three semesters during Junior and Senior Year · Chosen to be on the cover of NC State freshman admissions\n       booklet · In the process of receiving the Professional Development Certificate · NCSU Chinese basketball team player.\n       Math and physics club member · Control and Mechanical Team member of NCSU EcoCAR2 · Took the global training class at NC\n       State University · CUSA member (Chinese undergraduate student association).\nSkills\n3D, 3D modeling, AutoCAD, broadcasting, budget, C, cable, Chinese, com, hardware, content, controller, data analysis, Dec, decision-making,\ndesigning, product design, English, fashion, focus, Fortran, frame, Graphic, Lathe, Linux, director, Maple, materials, MATLAB, mechanical,\nMechanical Engineering, access, Mill, modeling, navigation, printer, processes, profit, speaking, Python, Quantitative analysis, reading, read,\nresearch, safety, Simulation, sketching, SolidWorks, statistical analysis, Statistics, phone, translating, transportation, video, Welding, wiring, written\n"

Sesuai ya. Mantap!

Testing fungsi rank_resume()

Saya akan sengaja menggunakan skill yang dimiliki resume 10751444, jika resume 10751444 berada di rank 1, berarti fungsi saya bekerja dengan baik!

ideal_designer <- "3D modeling, AutoCAD, broadcasting, budget, C, cable, Chinese, com, hardware, content, controller, data analysis, Dec, decision-making,\ndesigning, product design, English, fashion, focus, Fortran, frame, Graphic, Lathe, Linux, director, Maple, materials, MATLAB, mechanical,\nMechanical Engineering, access, Mill, modeling, navigation, printer, processes, profit, speaking, Python, Quantitative analysis, reading, read,\nresearch, safety, Simulation, sketching, SolidWorks, statistical analysis, Statistics, phone, translating, transportation, video, Welding, wiring, written\n"

rank_resume(ideal_designer, designers)

Fungsi bekerja dengan baik!

Testing fungsi rank_resume() (2)

Kali ini, saya akan sengaja menggunakan kriteria yang dimiliki resume 10748989, jika resume 10748989 berada di rank 1, berarti fungsi saya bekerja dengan baik!

Sebelum itu, mari kita lihat terlebih dahulu resume 10748989.pdf!

knitr::include_graphics("10748989.png")

ideal_designer <- "Building codes knowledge Complex problem solving Strong analytical ability Excellent attention to detail Commercial interior design Working drawings and procedures Space planning methodology Sketching Rendering Digital drafting 3D rendering software Proficient in SketchUp"

rank_resume(ideal_designer, designers)

Fungsi bekerja dengan baik!

Testing fungsi matching_criteria_cloud

ideal_designer <- "design, rendering, autocad, cad, drawing, sketching, mapping, excel, sketchup, interior, planning, photoshop, illustrator, procreate"

clean_ideal_designer <- cleanText(ideal_designer, "en", F) %>% as.character()
clean_resume <- cleanText(designers[designers$file_name == "10748989.pdf", ]$text, "en", F) %>% as.character()

matching_criteria_cloud(clean_ideal_designer, clean_resume)

Fungsi bekerja dengan baik!

Web App Overview & User Workflow

Overview

Ini hanya gambaran sementara, web app-nya belum selesai dan belum bekerja sebagaimana mestinya.

knitr::include_graphics("overview.png")

Inputted Folder Example

knitr::include_graphics("inputted-folder-example.png")

User Work Flow

1. User mengupload folder kumpulan resume
2. User memasukkan kriteria yang diinginkan
3. Web App akan menghasilkan ranking dalam bentuk table
4. User dapat mengklik Resume pada table tersebut
5. Saat resume pada table diklik, Web App akan mengeluarkan word cloud yang merupakan kriteria yang matching dengan kriteria ideal.

Limitations

1. Input harus berupa folder dengan extension .zip
2. Isi folder harus berupa kumpulan file dengan extension .pdf