Week10 GLM
2024-03-25
data <-read.csv("C:\\Users\\Krishna\\Downloads\\productivity+prediction+of+garment+employees\\garments_worker_productivity.csv")# loading neccesary libraries
library(dplyr)##
## Attaching package: 'dplyr'## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionlibrary(broom)
library(ggplot2)# here i am choosing actual_productivity as a binary column and targeted_productivity as a explanatory variable from my dataset
# Check the distribution of the binary column
table(data$actual_productivity)##
## 0.233705476 0.235795455 0.238041667 0.24625 0.247316017 0.249416667
## 1 1 1 1 1 1
## 0.251399254 0.2565 0.258 0.259375 0.260978788 0.261174242
## 1 1 1 1 1 1
## 0.26369382 0.2640625 0.268214286 0.271875 0.272 0.280333333
## 1 1 1 3 1 1
## 0.283054487 0.283958333 0.285333333 0.286984568 0.287041667 0.295307738
## 1 1 1 1 1 1
## 0.302117347 0.30277037 0.303574468 0.307501459 0.309333333 0.311207464
## 1 1 1 1 1 1
## 0.313852814 0.314166667 0.325 0.327407407 0.328131579 0.329545455
## 1 1 1 1 1 1
## 0.329964879 0.330113636 0.332146465 0.332359307 0.337973485 0.342361111
## 1 1 1 1 1 1
## 0.345833333 0.349951389 0.34998951 0.35003125 0.350066986 0.35020649
## 1 1 1 1 1 1
## 0.35021836 0.350301724 0.350416667 0.35063299 0.350706422 0.353259649
## 1 3 1 1 1 1
## 0.354444444 0.35542803 0.355534483 0.356458333 0.361071429 0.362666667
## 1 1 1 1 1 1
## 0.365318713 0.366053523 0.36871875 0.37046657 0.3715625 0.376597222
## 1 1 1 1 2 2
## 0.378895152 0.385791667 0.388007813 0.388830357 0.393548851 0.394722222
## 1 1 1 1 2 1
## 0.397743056 0.39875 0.400332792 0.403242161 0.404144928 0.406354167
## 1 1 1 1 1 1
## 0.4078125 0.408960345 0.409545455 0.410833333 0.411553571 0.412119835
## 2 1 1 4 1 1
## 0.415172414 0.417916667 0.432122899 0.433263158 0.436326389 0.437995338
## 1 1 1 1 1 1
## 0.440375 0.441041667 0.441392 0.447083333 0.447916667 0.448722222
## 1 1 1 1 1 1
## 0.449964912 0.4509375 0.452012539 0.452979626 0.453125 0.456875
## 1 1 1 1 1 1
## 0.460578704 0.463194444 0.463403955 0.465757576 0.466821212 0.467693269
## 1 1 1 1 1 1
## 0.470769231 0.471108491 0.473134796 0.475718391 0.477291667 0.483333333
## 1 1 1 1 1 1
## 0.48792 0.4925 0.495416667 0.495617514 0.496549708 0.497885057
## 1 1 1 1 1 1
## 0.499980334 0.499998888 0.500025069 0.500033898 0.500035345 0.500061916
## 1 1 1 1 1 1
## 0.500117677 0.50012336 0.500241342 0.500258046 0.500290419 0.500380799
## 1 1 1 1 1 1
## 0.500528095 0.500547544 0.500567308 0.500610909 0.500720126 0.500801724
## 1 1 1 1 1 3
## 0.504596491 0.5046875 0.505128205 0.505888889 0.507903226 0.515606061
## 1 1 1 1 1 1
## 0.520237649 0.52118 0.522844828 0.526810345 0.528125 0.531666667
## 1 1 1 1 1 1
## 0.535677966 0.536901754 0.5375 0.537919444 0.538399621 0.5403125
## 1 1 1 1 1 1
## 0.540729167 0.541517857 0.54175 0.545657673 0.549791667 0.549969429
## 1 1 1 1 1 1
## 0.550349708 0.550403509 0.553333333 0.555430556 0.555500132 0.5565625
## 1 1 1 2 1 2
## 0.557252451 0.560625 0.561979167 0.562212644 0.565972222 0.567377778
## 1 2 1 1 1 1
## 0.568259587 0.576460393 0.578314394 0.579511494 0.58 0.581130952
## 1 1 2 1 2 1
## 0.5814 0.582045455 0.582301029 0.585 0.585315789 0.586041667
## 1 1 1 1 1 1
## 0.586465465 0.590435606 0.590617284 0.590740741 0.59114168 0.592083333
## 1 1 2 1 1 1
## 0.593055556 0.594871795 0.597348485 0.598627451 0.598792339 0.600028736
## 1 1 1 1 1 1
## 0.600040678 0.600062696 0.600070513 0.600099145 0.600125217 0.600143365
## 1 1 1 1 1 1
## 0.60017284 0.600224189 0.600229846 0.600239766 0.600273268 0.600291767
## 1 1 1 1 1 2
## 0.600369686 0.60041361 0.600436426 0.600447507 0.600528571 0.60059761
## 1 1 1 1 1 1
## 0.600710606 0.600982906 0.601037091 0.601278409 0.60128 0.601944444
## 1 1 1 1 1 1
## 0.602 0.603432184 0.604166667 0.605208333 0.606912879 0.607416667
## 1 1 1 1 1 1
## 0.607654321 0.609138258 0.609583333 0.610208333 0.611140537 0.612517157
## 1 1 1 1 1 1
## 0.61625 0.618181818 0.618361111 0.621971751 0.6225 0.622828125
## 1 1 1 1 1 1
## 0.6253125 0.625625 0.626577778 0.626822917 0.627011183 0.628333333
## 1 2 1 1 1 3
## 0.628882576 0.629416667 0.630402924 0.631354167 0.632361111 0.634666667
## 1 1 1 1 1 1
## 0.636049383 0.637711864 0.638614379 0.639867424 0.64025 0.640577652
## 1 1 1 2 1 1
## 0.646306818 0.648106061 0.649662222 0.64998056 0.649983281 0.650040783
## 1 1 2 1 1 1
## 0.650044068 0.650066445 0.650130952 0.650134 0.650148148 0.650198653
## 1 1 1 1 1 1
## 0.650223718 0.65024031 0.650243497 0.650299575 0.650307143 0.650407524
## 1 1 1 1 1 1
## 0.650416734 0.650421429 0.650596491 0.650834808 0.650962281 0.651007071
## 2 1 1 1 1 1
## 0.651515152 0.653431373 0.653598485 0.656666667 0.656763743 0.657083333
## 1 1 1 2 1 1
## 0.658541667 0.660683293 0.661837121 0.662255892 0.662270115 0.66237931
## 1 1 1 1 1 1
## 0.664583333 0.664875 0.666515152 0.667329545 0.667604167 0.668087121
## 2 3 1 1 1 1
## 0.670075758 0.670216049 0.671875 0.672135417 0.672140805 0.673083333
## 2 1 1 1 1 1
## 0.673245283 0.675568182 0.676666667 0.681060606 0.681598039 0.682433036
## 1 1 3 4 1 1
## 0.6825 0.682708333 0.683550607 0.683806818 0.684027778 0.684888889
## 1 1 1 1 1 1
## 0.687555556 0.688017677 0.688557555 0.689299242 0.6895 0.690182815
## 1 1 1 1 1 1
## 0.692045455 0.697708333 0.699965217 0.699984417 0.7 0.700019883
## 1 1 2 1 1 1
## 0.700029771 0.700051852 0.700058333 0.700060345 0.700063796 0.700069811
## 1 1 2 1 1 1
## 0.700079032 0.700094156 0.700095563 0.70009573 0.700106061 0.700134038
## 1 1 1 1 1 1
## 0.700135088 0.70013604 0.700164706 0.700170388 0.700184577 0.700206125
## 1 1 1 4 1 1
## 0.700211111 0.700236601 0.700246491 0.700250784 0.700251773 0.700256795
## 1 2 2 3 1 1
## 0.700278846 0.700354545 0.700362069 0.700386207 0.700398148 0.700422
## 1 1 2 1 1 4
## 0.700424138 0.70043672 0.700459889 0.700480831 0.700505263 0.700508929
## 2 2 1 1 2 1
## 0.700513566 0.700516224 0.700518519 0.700540441 0.7005417 0.700556897
## 1 1 1 1 3 1
## 0.700573099 0.700588406 0.700603448 0.700605263 0.700612069 0.700614035
## 1 1 3 1 1 3
## 0.70061442 0.700618234 0.700623 0.700632768 0.700659649 0.700710417
## 1 3 2 1 1 1
## 0.700888203 0.700903571 0.7018125 0.702666667 0.702777778 0.703770833
## 1 1 1 1 1 1
## 0.705576584 0.705916667 0.707045902 0.707111111 0.707446429 0.710125
## 1 1 1 1 1 1
## 0.712205247 0.712626263 0.714410494 0.715333333 0.715766667 0.721126957
## 1 1 1 1 1 1
## 0.722333333 0.722568627 0.722638889 0.725 0.725625 0.726933333
## 1 1 1 1 1 2
## 0.727349537 0.72830303 0.733277778 0.734645833 0.735984848 0.740444444
## 1 1 1 1 1 1
## 0.741 0.742901235 0.749166667 0.749188312 0.749987124 0.750027778
## 1 1 1 1 1 2
## 0.750031447 0.750031898 0.750037968 0.750041201 0.750050847 0.750057359
## 1 1 1 1 3 2
## 0.750057851 0.750062751 0.750068049 0.750079323 0.750098351 0.750140741
## 1 1 4 1 1 1
## 0.750162367 0.750176991 0.750206897 0.750212553 0.75024303 0.750254867
## 1 1 2 1 3 1
## 0.750283333 0.750293939 0.750344828 0.750347333 0.750348457 0.750356125
## 1 1 1 1 1 3
## 0.750371895 0.750392157 0.750395513 0.7504 0.750406233 0.750425926
## 2 2 8 2 1 1
## 0.750427826 0.750437269 0.750450658 0.750473684 0.750503571 0.750517565
## 1 1 1 4 1 1
## 0.750520115 0.75053268 0.750545455 0.750593103 0.750608 0.750621354
## 1 1 1 3 2 1
## 0.750646667 0.750648148 0.75065101 0.750651724 0.750716981 0.750727326
## 2 2 11 2 1 1
## 0.75075 0.750770115 0.750797009 0.750799435 0.753097531 0.753525
## 1 2 2 1 1 1
## 0.753683478 0.755166667 0.755208333 0.755486111 0.755555556 0.758173077
## 1 2 1 1 1 1
## 0.758229167 0.75885 0.759228395 0.759270833 0.760833333 0.763375
## 1 1 1 1 1 1
## 0.768847222 0.769292929 0.77011398 0.771583333 0.773333333 0.77815
## 2 1 1 2 3 1
## 0.778222222 0.779791667 0.782447917 0.78375 0.785864198 0.7866
## 1 2 1 1 1 1
## 0.786631944 0.787299691 0.788 0.790003236 0.791458333 0.792104167
## 1 1 1 1 1 1
## 0.793844697 0.794566667 0.7953875 0.795416667 0.796208333 0.796755556
## 1 1 1 1 1 1
## 0.7975 0.799963218 0.799975862 0.799982853 0.8 0.80000295
## 1 6 1 2 7 1
## 0.800015009 0.800020563 0.800023511 0.800024932 0.800030199 0.80003139
## 1 4 1 1 1 2
## 0.800034024 0.800034503 0.800055762 0.800071839 0.800076522 0.80007657
## 1 5 1 3 2 2
## 0.800094017 0.800107143 0.800115819 0.800117103 0.800125 0.800128721
## 1 1 3 7 3 8
## 0.800137255 0.800140969 0.800144144 0.800148649 0.800149813 0.800161172
## 3 1 1 1 1 1
## 0.800162602 0.800181818 0.800191989 0.800237288 0.800237838 0.800246011
## 1 1 1 1 1 2
## 0.800246753 0.800250962 0.800259023 0.800260821 0.800261486 0.800263218
## 6 1 1 1 1 1
## 0.800273829 0.800279693 0.800302791 0.800309211 0.800312375 0.800313433
## 4 1 1 1 1 1
## 0.800318644 0.800322936 0.800333333 0.800343766 0.800346445 0.80035194
## 4 1 2 5 1 2
## 0.800355208 0.800358775 0.80037492 0.800381944 0.800386364 0.800393224
## 1 1 1 1 2 1
## 0.8004 0.800401961 0.800415742 0.800434622 0.800436542 0.800470513
## 1 24 1 1 1 1
## 0.800473729 0.800489676 0.800497246 0.800511068 0.800513307 0.800516667
## 1 1 1 1 1 1
## 0.800534483 0.800534979 0.800537143 0.800566092 0.800570492 0.800578947
## 1 1 1 1 2 1
## 0.800579532 0.800594466 0.800598058 0.800612676 0.80062987 0.800643806
## 1 5 1 1 1 1
## 0.800684366 0.800701754 0.800711494 0.800725314 0.800746552 0.800779018
## 2 3 1 1 1 1
## 0.8008 0.800806306 0.800808642 0.800842424 0.800888889 0.800909609
## 1 1 2 1 1 1
## 0.800947475 0.800980392 0.801028213 0.802243319 0.803541667 0.804416667
## 1 3 1 1 1 1
## 0.804640152 0.804848485 0.805555556 0.80575 0.805909091 0.806058333
## 1 1 1 1 3 1
## 0.806879167 0.809236111 0.809564394 0.809640152 0.810111111 0.811388889
## 2 1 1 1 1 1
## 0.812625 0.813309028 0.813371212 0.813611111 0.81640625 0.817102273
## 3 1 1 1 1 1
## 0.817424242 0.819270833 0.820833333 0.8211125 0.821354167 0.821666667
## 2 1 3 1 2 1
## 0.823555556 0.825444444 0.82680303 0.827147436 0.827186544 0.828295455
## 1 1 1 1 1 1
## 0.83 0.8300625 0.8319375 0.83375 0.835757576 0.837594697
## 1 1 1 1 1 1
## 0.838383838 0.838666667 0.840533333 0.840888889 0.841 0.845069444
## 1 1 1 1 1 1
## 0.845458333 0.845833333 0.846950758 0.8471 0.849983766 0.850045
## 2 2 1 1 1 1
## 0.85007069 0.850084211 0.85011396 0.850136766 0.850170115 0.850181818
## 1 2 1 12 1 1
## 0.850223776 0.850252525 0.850312684 0.850345133 0.850362069 0.850364583
## 1 1 2 1 3 1
## 0.850410511 0.850415628 0.850426891 0.850436438 0.850446154 0.850502311
## 1 1 1 1 1 11
## 0.850520588 0.850522167 0.850532143 0.850569492 0.850610526 0.851174114
## 1 1 2 2 1 1
## 0.852793561 0.853666667 0.85695 0.857916667 0.858143939 0.858585859
## 1 2 2 1 7 1
## 0.86037037 0.860653409 0.861679012 0.861875 0.864342593 0.864583333
## 1 1 1 1 1 3
## 0.868888889 0.87 0.870083333 0.870580808 0.87115 0.8721
## 2 1 1 1 1 1
## 0.873068182 0.874027778 0.875390625 0.875555556 0.876444444 0.877552083
## 1 1 1 1 1 1
## 0.879714482 0.880530303 0.880754167 0.881575 0.884 0.884261364
## 1 2 1 1 1 2
## 0.885925926 0.8865 0.888125 0.888686869 0.890604167 0.891555556
## 1 2 1 1 2 2
## 0.891723485 0.892194444 0.893066667 0.893663194 0.894444444 0.895454545
## 2 1 1 1 1 1
## 0.896022727 0.899 0.899111111 0.899166667 0.899555556 0.899984058
## 1 1 1 1 1 1
## 0.900061017 0.900129762 0.900135693 0.90014152 0.900144811 0.900147246
## 1 5 2 1 1 1
## 0.900158405 0.900215716 0.900321106 0.90047076 0.900477825 0.900509044
## 1 2 2 2 1 1
## 0.900537069 0.900556277 0.90063244 0.90064806 0.9008 0.900833333
## 1 1 1 1 3 1
## 0.901262626 0.902222222 0.9025 0.902916667 0.902962963 0.905454545
## 1 2 2 2 3 1
## 0.906666667 0.908080808 0.909391667 0.910378788 0.910521886 0.911589744
## 1 1 1 2 1 1
## 0.912037037 0.912202112 0.912766667 0.91375 0.915229167 0.915766667
## 1 1 2 1 1 1
## 0.919125 0.919905405 0.919954545 0.92 0.920236905 0.921604938
## 1 1 1 1 1 2
## 0.921703704 0.922839506 0.925643939 0.926388889 0.927291667 0.927541667
## 2 1 1 1 1 1
## 0.928680556 0.92885 0.929074074 0.929183333 0.929277778 0.930340376
## 1 1 1 1 1 1
## 0.930416667 0.931645833 0.934607438 0.93532197 0.936355556 0.936496212
## 2 1 1 1 1 1
## 0.936861111 0.937242424 0.939166667 0.939513889 0.940625 0.940701058
## 1 1 1 1 1 1
## 0.940725424 0.942213805 0.9425 0.945277778 0.945555556 0.9456
## 1 1 1 1 1 1
## 0.947689394 0.949981609 0.950185965 0.950438596 0.950625 0.951420455
## 1 1 5 1 1 2
## 0.951944444 0.952020202 0.953110048 0.955151515 0.955791667 0.956270833
## 1 2 1 2 1 1
## 0.957638889 0.958901515 0.959191919 0.960433333 0.960625 0.961059028
## 1 1 1 1 1 1
## 0.961784512 0.962016667 0.963699495 0.964106061 0.966666667 0.966759259
## 1 1 1 1 1 2
## 0.966781346 0.970075758 0.970816667 0.971866667 0.973796791 0.974621212
## 1 2 1 12 1 1
## 0.976979167 0.977272727 0.977555556 0.978525641 0.979527778 0.980909091
## 1 1 1 1 1 1
## 0.980984848 0.985 0.98719697 0.987880435 0.988024691 0.988636364
## 1 1 1 1 1 1
## 0.989 0.991388889 0.9918 0.994270833 0.994375 0.99485
## 1 1 1 1 1 1
## 0.997792208 0.999533333 0.999924242 0.999995238 1.000018551 1.000065789
## 1 1 1 3 4 1
## 1.000230409 1.000344928 1.000402055 1.000446018 1.000457471 1.000602279
## 11 2 1 1 1 1
## 1.000671304 1.001416667 1.004888889 1.0115625 1.02 1.033155556
## 1 1 1 1 1 1
## 1.033570076 1.05028058 1.050666667 1.057962963 1.059621212 1.096633333
## 1 1 1 1 1 1
## 1.100483918 1.108125 1.1204375
## 1 1 1# since i have no binary columns in my dataset i am converting actual_productivity as a binary variable
data$actual_productivity_binary <- ifelse(data$actual_productivity > 0.8, 1, 0)# Creating a histogram for actual_productivity
ggplot(data, aes(x = actual_productivity)) +
geom_histogram(binwidth = 0.05, fill = "skyblue", color = "black") +
labs(title = "Histogram of Actual Productivity",
x = "Actual Productivity",
y = "Frequency")
# building a logistic regression model
model <- glm(actual_productivity_binary ~ targeted_productivity, data = data, family = binomial)
# Display model summary
summary(model)##
## Call:
## glm(formula = actual_productivity_binary ~ targeted_productivity,
## family = binomial, data = data)
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -9.2318 0.8143 -11.34 <2e-16 ***
## targeted_productivity 12.2626 1.0778 11.38 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 1654.2 on 1196 degrees of freedom
## Residual deviance: 1447.2 on 1195 degrees of freedom
## AIC: 1451.2
##
## Number of Fisher Scoring iterations: 5# Interpret the coefficients
# Coefficient for 'targeted_productivity'
coef_targeted_productivity <- coef(model)["targeted_productivity"]
coef_targeted_productivity## targeted_productivity
## 12.26255Here the coefficient for ‘targeted_productivity’ represents the change in the log odds of achieving high productivity (binary outcome) associated with a one-unit increase in the ‘targeted_productivity’. Specifically, if the coefficient for ‘targeted_productivity’ is positive, it indicates that an increase in the ‘targeted_productivity’ leads to higher log odds of achieving high productivity. Conversely, if the coefficient is negative, it suggests that an increase in ‘targeted_productivity’ decreases the log odds of achieving high productivity.
The coefficient for the targeted_productivity variable is 12.26255. Taking the exponent of this coefficient (e^12.26255), we get approximately 21330344.
This means that for every one-unit increase in targeted_productivity the odds of actual productivity being above the threshold increase by a factor of approximately 21330344.
# Construct confidence intervals for the coefficient of 'targeted_productivity'
conf_interval <- confint(model, "targeted_productivity")## Waiting for profiling to be done...conf_interval## 2.5 % 97.5 %
## 10.21403 14.43849# Translate the meaning of confidence interval
# Lower and upper bounds of confidence interval for 'targeted_productivity'
lower_bound <- conf_interval[1]
upper_bound <- conf_interval[2]The confidence interval for the coefficient of ‘targeted_productivity’ is (10.21403, 14.43849). The values 10.21403 and 14.43849 represent the lower and upper bounds of the confidence interval, respectively. Specifically, the interval (10.21403, 14.43849) indicates that we are 95% confident that the true coefficient of ‘targeted_productivity’ lies within this range.
#building a transformation for targeted_productivity
# Scatterplot before transformation
ggplot(data, aes(x = targeted_productivity, y = actual_productivity_binary)) +
geom_point() +
ggtitle("Scatterplot of targeted_productivity vs. actual_productivity_binary (Before Transformation)")
# Scatterplot after transformation
ggplot(data, aes(x = log(targeted_productivity), y = actual_productivity_binary)) +
geom_point() +
ggtitle("Scatterplot of log(targeted_productivity) vs. actual_productivity_binary (After Transformation)")
In the scatterplot before transformation, if the relationship between ‘targeted_productivity’ and ‘actual_productivity_binary’ is not linear or if the spread of points is not constant across the range of ‘targeted_productivity’, it indicates that the relationship might be nonlinear or heteroscedastic. In such cases, a transformation like taking the logarithm could help linearize the relationship or stabilize the variance, making it more suitable for modeling.
After applying the logarithmic transformation, if the scatterplot shows a more linear relationship or a more constant spread of points, it suggests that the transformation has addressed the nonlinearity or heteroscedasticity in the relationship between the explanatory variable and the outcome variable. Thus, demonstrating the need for the transformation.
INSIGHTS
1)Selecting a Binary Column: In this task, we identified the ‘actual_productivity’ column as the binary variable of interest. We converted it into a binary variable ‘actual_productivity_binary’ based on a threshold (in this case, 0.8), where values above the threshold indicate high productivity (1) and values below or equal indicate low productivity (0).
2)Building a Logistic Regression Model: We built a logistic regression model with ‘targeted_productivity’ as the explanatory variable and ‘actual_productivity_binary’ as the binary outcome variable. The logistic regression model helps us understand how changes in the targeted productivity influence the likelihood of achieving high productivity.
3)Interpreting Coefficients: The coefficient for ‘targeted_productivity’ in the logistic regression model represents the change in the log odds of achieving high productivity associated with a one-unit increase in the targeted productivity. A positive coefficient indicates that higher targeted productivity is associated with higher log odds of achieving high productivity, while a negative coefficient suggests the opposite.
4)Constructing Confidence Intervals: We constructed confidence intervals for the coefficient of ‘targeted_productivity’ to estimate the range within which the true coefficient is likely to lie. This provides a measure of uncertainty around the coefficient estimate and helps assess the statistical significance of the relationship between targeted productivity and actual productivity.
5)Considering Transformation for Explanatory Variable: We considered taking the log transformation of the explanatory variable ‘targeted_productivity’ to address potential nonlinearities or heteroscedasticity in the relationship with the binary outcome variable. By comparing scatterplots before and after transformation, we can evaluate whether the transformation improves the linearity or stability of the relationship, thus enhancing the model’s interpretability and predictive performance.