Team Shiny - Final Project Proposal
Overview
The task is to implement a Job Recommendation System for Data Scientists in major US cities.
The job seeker (candidate) will have to answer few questions related to skills and experience OR might upload the resume to a Shiny App UI interface. The App will combine NLP techniques such as topic modeling with classification-style learning to determine the best fit job based on the datasets referenced below. The App will provide with visualization to match by the candidate with the job listing. The candidate might be provided with a dropdown of different job types and show which keywords of the resume match or don’t match.
Engineering the Recommender System
The entities in the system will be Candidate, Employer, Job and Location. The relationship between the entities would be either based on user interactions or system generated. For example, the relations would be ‘visited’, ‘match’, ‘favorite’, ‘apply’, ‘posted’ etc. Based on these relationship a simplified graph will be generated using Neo4J or alternate graph DB.
We will initially build a basic recommender system and later try to enhance it to a hybrid recommender system by combining techniques like Content-based and Collaborative filtering, and make it more robust. Data mining techniques for similarity measure like Jaccard Index, Cosine distance and Pearson correlation might be used. Dimension reduction will be incorporated to improve accuracy of the recommendations.
Heatmaps of Candidates vs Skills and Job vs Skill Requirement will be created to visualize the match. Scatterplots of Employers/Industry and Salary/City will also be presented.
To begin with, the Shiny App will have to the job seeker register to the App using email which will be integrated with Firebase OAuth2 authentication process in the backend.
At the end, it would be interesting if we can apply the Recommender Model on a test dataset to verify the workings of the system.