Automated EDA in R
Here we try automating exploratory data analysis with DataExplorer.
We start by loading hospital
dataset from Kaggle into Rstudio.
library(readr)
healthcare<-read_csv("C:/Users/email/Downloads/Healthcare_Investments_and_Hospital_Stay (1).csv")Rows: 518 Columns: 6── Column specification ─────────────────────────────────────────────────────────────
Delimiter: ","
chr (1): Location
dbl (5): Time, Hospital_Stay, MRI_Units, CT_Scanners, Hospital_Beds
ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.str(healthcare)spc_tbl_ [518 × 6] (S3: spec_tbl_df/tbl_df/tbl/data.frame)
$ Location : chr [1:518] "AUS" "AUS" "AUS" "AUS" ...
$ Time : num [1:518] 1992 1994 1995 1996 1997 ...
$ Hospital_Stay: num [1:518] 6.6 6.4 6.5 6.4 6.2 6.1 6.2 6.1 6.2 6.2 ...
$ MRI_Units : num [1:518] 1.43 2.36 2.89 2.96 3.53 4.51 6.01 3.52 3.79 3.74 ...
$ CT_Scanners : num [1:518] 16.7 18.5 20.6 21.9 23.3 ...
$ Hospital_Beds: num [1:518] 1.43 2.36 2.89 2.96 3.53 4.51 6.01 3.52 3.79 3.74 ...
- attr(*, "spec")=
.. cols(
.. Location = col_character(),
.. Time = col_double(),
.. Hospital_Stay = col_double(),
.. MRI_Units = col_double(),
.. CT_Scanners = col_double(),
.. Hospital_Beds = col_double()
.. )
- attr(*, "problems")=<externalptr> Now we load DataExplorer which will create some reports
some high-level summaries and graphs:
library(DataExplorer)
create_report(healthcare)
processing file: report.rmd
|
| | 0%
|
|. | 2%
|
|.. | 5% [global_options]
|
|... | 7%
|
|.... | 10% [introduce]
|
|..... | 12%
|
|...... | 14% [plot_intro]
|
|....... | 17%
|
|........ | 19% [data_structure]
|
|......... | 21%
|
|.......... | 24% [missing_profile]
|
|.......... | 26%
|
|........... | 29% [univariate_distribution_header]
|
|............ | 31%
|
|............. | 33% [plot_histogram]
|
|.............. | 36%
|
|............... | 38% [plot_density]
|
|................ | 40%
|
|................. | 43% [plot_frequency_bar]
|
|.................. | 45%
|
|................... | 48% [plot_response_bar]
|
|.................... | 50%
|
|..................... | 52% [plot_with_bar]
|
|...................... | 55%
|
|....................... | 57% [plot_normal_qq]
|
|........................ | 60%
|
|......................... | 62% [plot_response_qq]
|
|.......................... | 64%
|
|........................... | 67% [plot_by_qq]
|
|............................ | 69%
|
|............................. | 71% [correlation_analysis]
|
|.............................. | 74%
|
|.............................. | 76% [principal_component_analysis]
|
|............................... | 79%
|
|................................ | 81% [bivariate_distribution_header]
|
|................................. | 83%
|
|.................................. | 86% [plot_response_boxplot]
|
|................................... | 88%
|
|.................................... | 90% [plot_by_boxplot]
|
|..................................... | 93%
|
|...................................... | 95% [plot_response_scatterplot]
|
|....................................... | 98%
|
|........................................| 100% [plot_by_scatterplot]
"C:/Program Files/RStudio/resources/app/bin/quarto/bin/tools/pandoc" +RTS -K512m -RTS "C:\Users\email\OneDrive\Documents\report.knit.md" --to html4 --from markdown+autolink_bare_uris+tex_math_single_backslash --output pandocd601c233cc0.html --lua-filter "C:\Users\email\AppData\Local\R\win-library\4.2\rmarkdown\rmarkdown\lua\pagebreak.lua" --lua-filter "C:\Users\email\AppData\Local\R\win-library\4.2\rmarkdown\rmarkdown\lua\latex-div.lua" --embed-resources --standalone --variable bs3=TRUE --section-divs --table-of-contents --toc-depth 6 --template "C:\Users\email\AppData\Local\R\win-library\4.2\rmarkdown\rmd\h\default.html" --no-highlight --variable highlightjs=1 --variable theme=yeti --mathjax --variable "mathjax-url=https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" --include-in-header "C:\Users\email\AppData\Local\Temp\RtmpAPyRrA\rmarkdown-strd6013262276.html" This one function creates Basic Statistics, Raw Counts, Percentages, Data Structure, Missing Data Profile, Univariate Distribution, Histogram, Bar Chart (with frequency), QQ Plots, Correlation Analysis, and a Principal Component Analysis.
The missing data profile is nice. In less than 2 minutes, we get a high-level view of any missing values in the dataset.
DataExplorer has other helpful functions. For example,
to get introduced to your newly created dataset:
introduce(healthcare)To visualize the table above (with some light analysis):
plot_intro(healthcare)
To visualize frequency distributions for all discrete features:
plot_bar(healthcare)
To visualize distributions for all continuous features:
plot_histogram(healthcare)
To visualize correlation heatmap for all non-missing features:
plot_correlation(na.omit(healthcare), maxcat = 5L)Warning: Ignored all discrete features since `maxcat` set to 5 categories!
sadasdas
LS0tDQp0aXRsZTogIkF1dG9tYXRlZCBFREEgaW4gUiINCm91dHB1dDogDQogIGh0bWxfbm90ZWJvb2s6DQogIHRvYzogdHJ1ZQ0KLS0tDQoNCkhlcmUgd2UgdHJ5IGF1dG9tYXRpbmcgZXhwbG9yYXRvcnkgZGF0YSBhbmFseXNpcyB3aXRoIFtgRGF0YUV4cGxvcmVyYF0oaHR0cHM6Ly9jcmFuLnItcHJvamVjdC5vcmcvd2ViL3BhY2thZ2VzL0RhdGFFeHBsb3Jlci92aWduZXR0ZXMvZGF0YWV4cGxvcmVyLWludHJvLmh0bWwpLiBXZSBzdGFydCBieSBsb2FkaW5nIFtob3NwaXRhbCBkYXRhc2V0XShodHRwczovL3d3dy5rYWdnbGUuY29tL2RhdGFzZXRzL2JhYnlvZGEvaGVhbHRoY2FyZS1pbnZlc3RtZW50cy1hbmQtbGVuZ3RoLW9mLWhvc3BpdGFsLXN0YXkpIGZyb20gS2FnZ2xlIGludG8gUnN0dWRpby4NCg0KYGBge3J9DQpsaWJyYXJ5KHJlYWRyKQ0KaGVhbHRoY2FyZTwtcmVhZF9jc3YoIkM6L1VzZXJzL2VtYWlsL0Rvd25sb2Fkcy9IZWFsdGhjYXJlX0ludmVzdG1lbnRzX2FuZF9Ib3NwaXRhbF9TdGF5ICgxKS5jc3YiKQ0Kc3RyKGhlYWx0aGNhcmUpDQpgYGANCg0KTm93IHdlIGxvYWQgYERhdGFFeHBsb3JlcmAgd2hpY2ggd2lsbCBjcmVhdGUgc29tZSByZXBvcnRzIHNvbWUgaGlnaC1sZXZlbCBzdW1tYXJpZXMgYW5kIGdyYXBoczoNCg0KYGBge3J9DQpsaWJyYXJ5KERhdGFFeHBsb3JlcikNCmNyZWF0ZV9yZXBvcnQoaGVhbHRoY2FyZSkNCmBgYA0KDQpUaGlzIG9uZSBmdW5jdGlvbiBjcmVhdGVzIEJhc2ljIFN0YXRpc3RpY3MsIFJhdyBDb3VudHMsIFBlcmNlbnRhZ2VzLCBEYXRhIFN0cnVjdHVyZSwNCk1pc3NpbmcgRGF0YSBQcm9maWxlLCBVbml2YXJpYXRlIERpc3RyaWJ1dGlvbiwgSGlzdG9ncmFtLCBCYXIgQ2hhcnQgKHdpdGggZnJlcXVlbmN5KSwNClFRIFBsb3RzLCBDb3JyZWxhdGlvbiBBbmFseXNpcywgYW5kIGEgUHJpbmNpcGFsIENvbXBvbmVudCBBbmFseXNpcy4gDQoNClRoZSBtaXNzaW5nIGRhdGEgcHJvZmlsZSBpcyBuaWNlLiBJbiBsZXNzIHRoYW4gMiBtaW51dGVzLCB3ZSAgZ2V0IGEgaGlnaC1sZXZlbCB2aWV3IG9mIGFueSBtaXNzaW5nIHZhbHVlcyBpbiB0aGUgZGF0YXNldC4NCg0KYERhdGFFeHBsb3JlcmAgaGFzIG90aGVyIGhlbHBmdWwgZnVuY3Rpb25zLiBGb3IgZXhhbXBsZSwgdG8gZ2V0IGludHJvZHVjZWQgdG8geW91ciBuZXdseSBjcmVhdGVkIGRhdGFzZXQ6DQoNCmBgYHtyfQ0KaW50cm9kdWNlKGhlYWx0aGNhcmUpDQpgYGANCg0KVG8gdmlzdWFsaXplIHRoZSB0YWJsZSBhYm92ZSAod2l0aCBzb21lIGxpZ2h0IGFuYWx5c2lzKToNCg0KYGBge3J9DQpwbG90X2ludHJvKGhlYWx0aGNhcmUpDQpgYGANCg0KDQpUbyB2aXN1YWxpemUgZnJlcXVlbmN5IGRpc3RyaWJ1dGlvbnMgZm9yIGFsbCBkaXNjcmV0ZSBmZWF0dXJlczoNCg0KYGBge3J9DQpwbG90X2JhcihoZWFsdGhjYXJlKQ0KYGBgDQoNClRvIHZpc3VhbGl6ZSBkaXN0cmlidXRpb25zIGZvciBhbGwgY29udGludW91cyBmZWF0dXJlczoNCg0KYGBge3J9DQpwbG90X2hpc3RvZ3JhbShoZWFsdGhjYXJlKQ0KYGBgDQoNClRvIHZpc3VhbGl6ZSBjb3JyZWxhdGlvbiBoZWF0bWFwIGZvciBhbGwgbm9uLW1pc3NpbmcgZmVhdHVyZXM6DQoNCmBgYHtyfQ0KcGxvdF9jb3JyZWxhdGlvbihuYS5vbWl0KGhlYWx0aGNhcmUpLCBtYXhjYXQgPSA1TCkNCmBgYA0KDQpzYWRhc2Rhcw==