Recap of Analytics Plan with Project Proposal

• Data extraction using Stack Exchange API

• Utilized 3 types of data: Posts, Users and Tags

• Current focus is on the Stack Overflow child website data for further analysis

Problem Description

• To predict whether a post will be answered or not
• To predict the view count of a particular post based on tags to maintain the community engagement
• To analyze the most popular topics on the stack overflow website

Summary of Peer Comments

• Most common problem, Duration to answer, recommendation to best site to use get answer
• Qualitatively visualize Prediction
• Will the timestamp data be collected for the topics discussed to predict the most engaged topics by mapping those topics over a time period?
• What type of regression modeling techniques are you planning to use?
• On what parameters are you planning on implementing the regression modeling techniques?
• What kind of visualization techniques are you planning to implement?

Data Summary and Results from Data Exploration (1/3)

• Tags data

• Users data

• Posts data

Data Summary and Results from Data Exploration (2/3)

• Here we can observe that 2017 and 2018 were the most engagement years. Also, 2020 is performing better as compared to 2019

• In the above pie-chart, we can observe that majority of the questions remain unanswered. This is a risk to the website being regularly visited by the users.

Data Summary and Results from Data Exploration (3/3)

In the below horizontal bar chart, we can observe the top 10 high frequency tags on the website.

• In the above line plot, we can infer that 2020 has the lowest average reputation which is not a good sign. Also, we checked the max reputation per year and found out the same for 2020.

ML/NLP Procedures and Results (1/3)

ML/NLP Procedures and Results (2/3)

ML/NLP Procedures and Results (3/3)

• Here we cover the Text mining on the title column of the posts dataset wherein we are creation a wordcloud of most common keywords used within titles. We can observe that “using” keyword within .NET framework is the most common used keyword in the discussion.

Key Findings

• We implemented 2 types of modeling techniques: Classification and Regression. Classification to predict if post will get answered or not (0/1) and regression to predict the view count of a particular post based on tags.

• After implementing above ML and NLP techniques, we conclude that highest accuracy is achieved by Decision Tree algorithm of 81.4% and we recommend using this techniques in order to find out whether a question will get answered or not.

• Sentiment analysis was not relevant in this project as posts do not have positive/negative sentiments associated with them unlike movie reviews.

• This project can also be applied to other Stack Exchange child websites to do similar analysis and prediction so that community engagement remains consistent and users are involved frequently.