Scrape Analysis

Author

Daniel Welch

Doctor Data

I chose to scrape doctor data off ratemds.com. I chose this because finding doctors for young adults, like myself, can be difficult and overwhelming. For the entire first part of your life your parents pick your doctor and schedule your appointments. But now that your grown, living in a new area, it’s time to find your own. I hope to provide an easy way to identify good doctors in your area to make the process much simpler.

How to Answer the Question

I intend on answering the question by identifying the different doctors a person might need, and showing them the best one for them. First I need to show all the different options of doctors in the area and how common they area. Next I need to show what types of doctors are more well liked than others to help them decide which they want. Once they identify what kind of doctor they want, its important to look at their rating and factor in how many people have reviewed them. Finally, they can identify the best doctors get a little sample review of them to help pick their favorite.

Cleaning

Break up the rating number and number of reviews because I will want to look at those individually. Change both of their data types to numeric so I can analyze them.

Doctor Types

Orthopedic Surgeons seem to be the most widely available doctors in the Cincinnati area, followed by Gynecologists and Obstetricians

Doctor Ratings

Most doctors are pretty highly rated overall except for Pain Management Specialists and Dermatologists.

Orthopedic Surgeons

There really appears to be no “bad” Orthopedic Surgeons, just some are better than others.

Review Number

It’s also important to factor in sample size because 5 stars on 50 ratings is a lot different than 5 stars on 1 rating. The doctors in the top right are the doctors we want.

# A tibble: 13 × 2
   doc_name                 doc_review                                          
   <chr>                    <chr>                                               
 1 Dr. Sam B. Koo           "\" I am sure Dr. Koo is a capable orthopedic surge…
 2 Dr. Matthew A. Johansen  "\" Great experience with Dr. Johansen. Worked on l…
 3 Dr. Andrew J. Razzano    "\" Dr. Razzano was exceptional and did a thorough …
 4 Dr. Ian P. Rodway        "\" Not what I expected when I signed up to the guy…
 5 Dr. Timothy B. McConnell "\" Dr McConnell is a great doctor. He repaired my …
 6 Dr. Mohab B. Foad        "\" He is a great Dr and surgeon. He done surgery o…
 7 Dr. Kevin J. Shaw        "\" Dr Shaw is a caring and compassionate surgeon. …
 8 Dr. Glen A. Mcclung      "\" I would recommend Dr. Glen A. Mcclung . He was …
 9 Dr. Robert H. Rolf       "\" Dr Rolf replaced my shoulder. Very skillful. Gr…
10 Dr. Stephen C. Hamilton  "\" Dr. Hamilton deserves a full encomium! He is co…
11 Dr. Rebecca Popham       "\" She was very helpful and friendly. We lucked ou…
12 Dr. Robert Burger        "\" should retire !! \""                            
13 Dr. Ronald T. Auer       "\" Awesome Ortho Doctor, takes the time to explain…

Findings

It was very simple and easy to find doctors for what I needed. I am able to look through all the different types of doctors and pick out the highest rated doctors that have the most reviews. With this method I am very confident I can find a good doctor for me.

--- title: "Scrape Analysis" author: "Daniel Welch" toc: true format: html: code-tools: TRUE self-contained: TRUE execute: warning: FALSE message: FALSE echo: FALSE --- ```{r} library(tidyverse) ``` # Doctor Data I chose to scrape doctor data off ratemds.com. I chose this because finding doctors for young adults, like myself, can be difficult and overwhelming. For the entire first part of your life your parents pick your doctor and schedule your appointments. But now that your grown, living in a new area, it's time to find your own. I hope to provide an easy way to identify good doctors in your area to make the process much simpler. # How to Answer the Question I intend on answering the question by identifying the different doctors a person might need, and showing them the best one for them. First I need to show all the different options of doctors in the area and how common they area. Next I need to show what types of doctors are more well liked than others to help them decide which they want. Once they identify what kind of doctor they want, its important to look at their rating and factor in how many people have reviewed them. Finally, they can identify the best doctors get a little sample review of them to help pick their favorite. ```{r} doctors_scrape <- read_csv("doctors.csv") #doctors_scrape <- read_csv("https://myxavier-my.sharepoint.com/:x:/g/personal/welchd3_xavier_edu/Ebg41WfObXZMtgde5xl4SgcBz-r1D5mF--z10xL9ToqcoQ?e=bUZd6l%3Fdownload%3D1") ``` # Cleaning Break up the rating number and number of reviews because I will want to look at those individually. Change both of their data types to numeric so I can analyze them. ```{r} doctors_scrape <- separate(doctors_scrape, doc_rating, into = c("Rating", "Reviews"), sep = " ") doctors_scrape$Reviews <- as.numeric(gsub("\\D", "", doctors_scrape$Reviews)) doctors_scrape$Rating <- as.numeric(doctors_scrape$Rating) ``` ```{r} doctors_scrape %>% ggplot(aes(doc_type))+ geom_bar()+ theme(axis.text.x = element_text(angle = 45, hjust = 1))+ labs(title = "Types of Doctors in Cincinnati", x = "Doctor Type", y = "Number of Doctors") ``` # Doctor Types Orthopedic Surgeons seem to be the most widely available doctors in the Cincinnati area, followed by Gynecologists and Obstetricians ```{r} doctors_scrape %>% group_by(doc_type) %>% summarise(avg_doc = mean(Rating, na.rm = TRUE)) %>% ggplot(aes(avg_doc, doc_type))+ geom_bar(stat = "identity")+ theme(axis.text.x = element_text(angle = 45, hjust = 1))+ labs(title = "Average Rating By Doctor Type", x = "Doctor Type", y = "Average Rating") ``` # Doctor Ratings Most doctors are pretty highly rated overall except for Pain Management Specialists and Dermatologists. ```{r} doctors_scrape %>% filter(doc_type == "Orthopedic Surgeon") %>% ggplot(aes(doc_name, Rating))+ geom_bar(stat = "identity")+ theme(axis.text.x = element_text(angle = 45, hjust = 1))+ labs(title = "Orthopedic Surgeon Ratings", x = "Doctor", y = "Rating") ``` # Orthopedic Surgeons There really appears to be no "bad" Orthopedic Surgeons, just some are better than others. ```{r} doctors_scrape %>% ggplot(aes(x=Rating,y=Reviews))+ geom_point(aes())+ labs(title = "Doctors by Rating and Review Number", x = "Rating", y = "Number of Reviews") ``` # Review Number It's also important to factor in sample size because 5 stars on 50 ratings is a lot different than 5 stars on 1 rating. The doctors in the top right are the doctors we want. ```{r} the_best <- doctors_scrape %>% filter(doc_type == "Orthopedic Surgeon") %>% filter(Rating > 4.85) %>% select(doc_name, doc_review) print(the_best) ``` # Findings It was very simple and easy to find doctors for what I needed. I am able to look through all the different types of doctors and pick out the highest rated doctors that have the most reviews. With this method I am very confident I can find a good doctor for me.