This is the data analysis RMarkdown file for the QCQ data in preparation for ISIRC 2022. Please read the subtitles and notes added as normal text in this document. Blocks with darker backgrounds are code chunks, mostly with their outputs. I tried to add #comments within the chunck just before the code line to explain the purpose of the code line. Meanwhile I explain the purpose of the section in the texts, so that is where to find information to give you fuller pictures.
# environment setup to run ordered logit properly
options(contrasts = rep("contr.treatment", 2))This chunk loads all the packages to use
library(tidyverse) # package for data cleaning and plotting
library(ordinal) # package for ordinal logit regression
library(MetBrewer) # package for nice color scale
library(modelsummary) # package for derive the model comparison tables
library(kableExtra) # model for saving outputs as tabkes
library(brant) # brant test for the parallel assumption for ordered logit
library(MASS) # models that work with the brant testFirst, import the sampled and coded data set
#import the raw data file
socialcare_raw <- read_csv("data/sample_new_cleaned.csv")Assign orders to the ordinal level variables and name the organizational form in a reader-friendly way.
#select relevant columns, rename and relabel
socialcare <- socialcare_raw %>%
# recode legal form types to be more readable / easier to present
mutate(location_id = factor(location_id),
form = case_when(form_num == 1 ~ "FPO",
form_num == 2 ~ "NPO",
form_num == 3 ~ "GOV",
form_num == 4 ~ "CIC",
form_num == 5 ~ "IND",
TRUE ~ NA_character_),
inherited = ifelse(inherited == "Y", TRUE, FALSE),
rating = recode(rating,
"Insufficient evidence to rate" = "NA",
"Requires improvement" = "Req improv")) %>%
# set the order of the values in the factors
mutate(form = ordered(form, levels = c("FPO", "NPO", "GOV", "CIC", "IND")),
# assume the order of the ratings as follows but need to double check with the source
rating = ordered(rating, levels = c("Inadequate","Req improv", "Good", "Outstanding")))
# show first several rows of the data set derived
head(socialcare)## # A tibble: 6 × 34
## ...1 locati…¹ Locat…² Locat…³ Care …⁴ prima…⁵ Locat…⁶ Locat…⁷ Locat…⁸ Locat…⁹
## <dbl> <fct> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr>
## 1 1 1-10000… VNH4P 69 Ten… N Commun… 69 Ten… Warmsw… Doncas… DN4 9PE
## 2 2 1-10000… VNH4P 69 Ten… N Commun… 69 Ten… Warmsw… Doncas… DN4 9PE
## 3 3 1-10000… VNH4P 69 Ten… N Commun… 69 Ten… Warmsw… Doncas… DN4 9PE
## 4 4 1-10000… VNH4P 69 Ten… N Commun… 69 Ten… Warmsw… Doncas… DN4 9PE
## 5 5 1-10000… VNH4P 69 Ten… N Commun… 69 Ten… Warmsw… Doncas… DN4 9PE
## 6 6 1-10000… VNH4P 69 Ten… N Commun… 69 Ten… Warmsw… Doncas… DN4 9PE
## # … with 24 more variables: `Location Local Authority` <chr>, region <chr>,
## # `Location NHS Region` <chr>, `Location ONSPD CCG Code` <chr>,
## # `Location ONSPD CCG` <chr>, `Location Commissioning CCG Code` <lgl>,
## # `Location Commissioning CCG Name` <lgl>,
## # `Service / Population Group` <chr>, domain <chr>, rating <ord>,
## # publication_date <chr>, `Report Type` <chr>, inherited <lgl>, URL <chr>,
## # `Provider ID` <chr>, location_type <chr>, form_num <dbl>, …The chunk below groups data by location, and check the distribution of different forms for all locations
location_count <-socialcare %>%
group_by(location_id) %>%
summarise(locatoion_id = first(location_id),
form_num = first(form_num),
form = first(form)) %>%
count(form_num, form)The chunk below groups data by organization and check the distribution of different forms for all organizations
provider_count <-socialcare %>%
group_by(provider_name) %>%
summarise(provider_name = first(provider_name),
form_num = first(form_num),
form = first(form)) %>%
count(form_num, form)The chunk below checks that the nomial variable for legal form
corresponds to the numerical form, and those coded with
form_num of “6” and “8” will be NA in the variable
form
socialcare %>%
count(form_num, form)## # A tibble: 8 × 3
## form_num form n
## <dbl> <ord> <int>
## 1 1 FPO 64187
## 2 2 NPO 12399
## 3 3 GOV 2431
## 4 4 CIC 504
## 5 5 IND 5153
## 6 6 <NA> 6
## 7 8 <NA> 1
## 8 NA <NA> 30I check here the region and primary_cat
variables, since the total number of categories are not too large, I
decide to first include region in the model. But the
primary_cat’s distribution is actually two main categories
– community based and residential (divided by 6 rating domains, there
are only 1 or 2 facilities in other categories).I will only keep the
data for these two categories.
The distribution of these variables will also be added to the slide for introducing control variables.
table(socialcare$region) %>%
kable()| Var1 | Freq |
|---|---|
| East Midlands | 8132 |
| East of England | 9378 |
| London | 9141 |
| North East | 3654 |
| North West | 10619 |
| South East | 15609 |
| South West | 10460 |
| West Midlands | 9660 |
| Yorkshire and The Humber | 8058 |
table(socialcare$primary_cat) %>%
kable()| Var1 | Freq |
|---|---|
| Community based adult social care services | 28201 |
| Community health - NHS & Independent | 6 |
| Community substance misuse | 12 |
| Mental health - community & hospital - independent | 12 |
| Residential social care | 56480 |
# creating a new dummy variable for facility category
socialcare <- socialcare %>%
mutate(category = case_when(primary_cat == "Community based adult social care services" ~ "community",
primary_cat == "Residential social care" ~ "residential",
TRUE ~ as.character(NA)))table(socialcare$category) %>%
kable()| Var1 | Freq |
|---|---|
| community | 28201 |
| residential | 56480 |
table(socialcare$inherited) %>%
kable()| Var1 | Freq |
|---|---|
| FALSE | 79873 |
| TRUE | 4838 |
The OLS models and the ordered logit models can be written as follows
\(rating_{numerical} = \beta_0 + \beta_1form + \beta_2category+ \beta_3region + \beta_4inherited + u\)
\(log-odds(rating_{ordinal} \leq j) = \beta_{j0} + \beta_1form + \beta_2category+ \beta_3region + \beta_4inherited + u\)
In this section, we first run the OLS models. In the OLS models, we kind of “cheat” R by treating the four rating levels with orders as if they are numbers 1-4. There are the flowing reasons that we report the results from OLS models, even though the more suitable methods should be ordered logit models, about which we will discuss in a while.
# converting the ordinal variable to numerical
socialcare_num <- socialcare %>%
mutate(rating_num = case_when(rating == "Inadequate" ~ 1,
rating == "Req improv" ~ 2,
rating == "Good" ~ 3,
rating == "Outstanding" ~ 4))# run the ols model with control on different domains
model_ols_overall <- lm(rating_num ~ form + category + region + inherited,
data = filter(socialcare_num, domain == "Overall"))
model_ols_safe <- lm(rating_num ~ form + category + region + inherited,
data = filter(socialcare_num, domain == "Safe"))
model_ols_effective <- lm(rating_num ~ form + category + region + inherited,
data = filter(socialcare_num, domain == "Effective"))
model_ols_caring <- lm(rating_num ~ form + category + region + inherited,
data = filter(socialcare_num, domain == "Caring"))
model_ols_well_led <- lm(rating_num ~ form + category + region + inherited,
data = filter(socialcare_num, domain == "Well-led"))
model_ols_responsive <- lm(rating_num ~ form + category + region + inherited,
data = filter(socialcare_num, domain == "Responsive"))models_ols <- modelsummary(list("overall" = model_ols_overall, "safe" = model_ols_safe,
"effective" = model_ols_effective, "caring"= model_ols_caring,
"well-led" = model_ols_well_led, "responsive" = model_ols_responsive),
statistic = "({p.value}) {stars}")
models_ols| overall | safe | effective | caring | well-led | responsive | |
|---|---|---|---|---|---|---|
| (Intercept) | 2.873*** | 2.789*** | 2.892*** | 3.016*** | 2.814*** | 2.961*** |
| (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | |
| formNPO | 0.082*** | 0.082*** | 0.040*** | 0.027*** | 0.100*** | 0.064*** |
| (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | |
| formGOV | 0.081*** | 0.097*** | 0.039* | 0.029+ | 0.097*** | 0.053** |
| (0.001) *** | (0.000) *** | (0.023) * | (0.061) + | (0.000) *** | (0.006) ** | |
| formCIC | 0.104* | 0.100* | 0.019 | 0.079* | 0.109+ | 0.039 |
| (0.043) * | (0.030) * | (0.604) | (0.014) * | (0.055) + | (0.339) | |
| formIND | −0.036* | −0.019 | −0.033** | −0.004 | −0.040* | −0.007 |
| (0.035) * | (0.214) | (0.007) ** | (0.712) | (0.033) * | (0.621) | |
| categoryresidential | −0.070*** | −0.064*** | −0.037*** | −0.047*** | −0.067*** | −0.030*** |
| (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | |
| regionEast of England | 0.043* | 0.047** | 0.041** | 0.015 | 0.030 | 0.035* |
| (0.017) * | (0.004) ** | (0.002) ** | (0.189) | (0.132) | (0.016) * | |
| regionLondon | 0.019 | 0.048** | 0.054*** | 0.003 | 0.025 | −0.003 |
| (0.308) | (0.004) ** | (0.000) *** | (0.807) | (0.206) | (0.820) | |
| regionNorth East | 0.133*** | 0.123*** | 0.095*** | 0.069*** | 0.155*** | 0.082*** |
| (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | |
| regionNorth West | 0.052** | 0.067*** | 0.064*** | 0.022* | 0.052** | 0.040** |
| (0.003) ** | (0.000) *** | (0.000) *** | (0.050) * | (0.007) ** | (0.004) ** | |
| regionSouth East | 0.062*** | 0.098*** | 0.049*** | 0.040*** | 0.057** | 0.051*** |
| (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.002) ** | (0.000) *** | |
| regionSouth West | 0.142*** | 0.124*** | 0.094*** | 0.085*** | 0.144*** | 0.091*** |
| (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | |
| regionWest Midlands | −0.030+ | 0.019 | 0.020 | −0.015 | −0.075*** | 0.000 |
| (0.092) + | (0.233) | (0.114) | (0.173) | (0.000) *** | (0.982) | |
| regionYorkshire and The Humber | −0.025 | −0.012 | 0.031* | 0.036** | −0.033 | 0.008 |
| (0.186) | (0.467) | (0.020) * | (0.002) ** | (0.115) | (0.576) | |
| inheritedTRUE | −0.020 | −0.018 | −0.032** | −0.032** | −0.018 | −0.032* |
| (0.260) | (0.248) | (0.008) ** | (0.003) ** | (0.346) | (0.020) * | |
| Num.Obs. | 12942 | 12966 | 12931 | 12927 | 12964 | 12953 |
| R2 | 0.023 | 0.023 | 0.013 | 0.017 | 0.025 | 0.013 |
| R2 Adj. | 0.022 | 0.021 | 0.012 | 0.016 | 0.024 | 0.012 |
| AIC | 16998.2 | 14402.5 | 8181.6 | 5099.3 | 19530.8 | 11037.1 |
| BIC | 17117.7 | 14522.0 | 8301.1 | 5218.8 | 19650.3 | 11156.6 |
| Log.Lik. | −8483.091 | −7185.242 | −4074.798 | −2533.650 | −9749.405 | −5502.563 |
| RMSE | 0.47 | 0.42 | 0.33 | 0.29 | 0.51 | 0.37 |
I find this reference regarding understanding ordered logit models useful. https://stats.oarc.ucla.edu/r/dae/ordinal-logistic-regression/
# derive output table
model_order_overall <- clm(rating ~ form + category + region + inherited,
data = filter(socialcare, domain == "Overall"),
link = "logit")
summary(model_order_overall)## formula: rating ~ form + category + region + inherited
## data: filter(socialcare, domain == "Overall")
##
## link threshold nobs logLik AIC niter max.grad cond.H
## logit flexible 12942 -8287.40 16608.79 7(0) 3.57e-08 2.6e+02
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## formNPO 0.45717 0.06438 7.101 1.24e-12 ***
## formGOV 0.42904 0.13436 3.193 0.001407 **
## formCIC 0.58629 0.29244 2.005 0.044981 *
## formIND -0.18331 0.08752 -2.095 0.036207 *
## categoryresidential -0.38747 0.04824 -8.032 9.58e-16 ***
## regionEast of England 0.24520 0.09601 2.554 0.010656 *
## regionLondon 0.10356 0.09573 1.082 0.279353
## regionNorth East 0.76090 0.13380 5.687 1.30e-08 ***
## regionNorth West 0.29293 0.09374 3.125 0.001779 **
## regionSouth East 0.32171 0.08625 3.730 0.000191 ***
## regionSouth West 0.83636 0.09756 8.573 < 2e-16 ***
## regionWest Midlands -0.15592 0.09189 -1.697 0.089747 .
## regionYorkshire and The Humber -0.12130 0.09631 -1.260 0.207843
## inheritedTRUE -0.10766 0.09132 -1.179 0.238431
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Threshold coefficients:
## Estimate Std. Error z value
## Inadequate|Req improv -4.44907 0.11032 -40.33
## Req improv|Good -1.70234 0.07845 -21.70
## Good|Outstanding 3.14414 0.08622 36.47
## (1176 observations deleted due to missingness)I will ran the models but only show the results in the comparing table
# run the ordered logit model with control
model_order_safe <- clm(rating ~ form + category + region + inherited,
data = filter(socialcare, domain == "Safe"),
link = "logit")
model_order_effective <- clm(rating ~ form + category + region + inherited,
data = filter(socialcare, domain == "Effective"),
link = "logit")
model_order_caring <- clm(rating ~ form + category + region + inherited,
data = filter(socialcare, domain == "Caring"),
link = "logit")
model_order_well_led <- clm(rating ~ form + category + region + inherited,
data = filter(socialcare, domain == "Well-led"),
link = "logit")
model_order_responsive <- clm(rating ~ form + category + region + inherited,
data = filter(socialcare, domain == "Responsive"),
link = "logit")The Brant test tests whether the parallel regression assumption (proportional odds assumption) holds. I searched resources explaining this like https://medium.com/evangelinelee/brant-test-for-proportional-odds-in-r-b0b373a93aa2. My own more natal language understanding is that, the models uses the same coefficients (log-odds) for IVs, e.g. NPO, CIC, to estimate their effects on the odds of above Outstanding vs. Good | Good vs. Require improvement | Require improvement vs. Inadequate. In other words, there are three comparison pairs here, that is three logit regression combined together, yet the methods force each logit have the same slope/coefs for each IV. The benefits of doing this is increasing power and have smaller standard errors.
The polr from the {MASS} package is used because it
works with the brant() function. But I used
clm() function from the {ordinal} package to run the
models, because that one gives you the standard errors and p-values. The
good thing is, as is shown, using results from the two models are the
same (expect for small rounding differences).
We gladly see that all the tests say the parallel regression assumption holds, I think the warning of ~400 missing combinations is negligiable because our number of observations is 84,711. The missing points should be due to some locations don’t have ratings for all domains.
Below I print out all the models and test results out just to share how these tests go. But they are quite repetitive, so feel free to scroll down to the model summary tables.
model_order_overall_mass <- polr(rating ~ form + category + region + inherited,
data = filter(socialcare, domain == "Overall"),
Hess = TRUE)
summary(model_order_overall_mass)## Call:
## polr(formula = rating ~ form + category + region + inherited,
## data = filter(socialcare, domain == "Overall"), Hess = TRUE)
##
## Coefficients:
## Value Std. Error t value
## formNPO 0.4571 0.06438 7.100
## formGOV 0.4288 0.13436 3.191
## formCIC 0.5862 0.29244 2.005
## formIND -0.1835 0.08751 -2.097
## categoryresidential -0.3875 0.04824 -8.032
## regionEast of England 0.2451 0.09602 2.553
## regionLondon 0.1035 0.09574 1.081
## regionNorth East 0.7608 0.13381 5.686
## regionNorth West 0.2929 0.09374 3.124
## regionSouth East 0.3217 0.08625 3.729
## regionSouth West 0.8363 0.09756 8.572
## regionWest Midlands -0.1560 0.09190 -1.697
## regionYorkshire and The Humber -0.1213 0.09631 -1.260
## inheritedTRUE -0.1078 0.09132 -1.181
##
## Intercepts:
## Value Std. Error t value
## Inadequate|Req improv -4.4492 0.1103 -40.3297
## Req improv|Good -1.7025 0.0785 -21.7009
## Good|Outstanding 3.1441 0.0862 36.4649
##
## Residual Deviance: 16574.79
## AIC: 16608.79
## (1176 observations deleted due to missingness)brant(model_order_overall_mass)## --------------------------------------------------------------------
## Test for X2 df probability
## --------------------------------------------------------------------
## Omnibus 77.75 28 0
## formNPO 16.57 2 0
## formGOV 2.66 2 0.26
## formCIC 0.27 2 0.87
## formIND 4.86 2 0.09
## categoryresidential 9.92 2 0.01
## regionEast of England 1.41 2 0.49
## regionLondon 22.44 2 0
## regionNorth East 5.03 2 0.08
## regionNorth West 12.75 2 0
## regionSouth East 16.44 2 0
## regionSouth West 8.61 2 0.01
## regionWest Midlands 9.56 2 0.01
## regionYorkshire and The Humber 6.78 2 0.03
## inheritedTRUE 1.79 2 0.41
## --------------------------------------------------------------------
##
## H0: Parallel Regression Assumption holds## Warning in brant(model_order_overall_mass): 364 combinations in table(dv,ivs) do
## not occur. Because of that, the test results might be invalid.model_order_safe_mass <- polr(rating ~ form + category + region + inherited,
data = filter(socialcare, domain == "Safe"),
Hess = TRUE)
summary(model_order_safe_mass)## Call:
## polr(formula = rating ~ form + category + region + inherited,
## data = filter(socialcare, domain == "Safe"), Hess = TRUE)
##
## Coefficients:
## Value Std. Error t value
## formNPO 0.57955 0.07460 7.7683
## formGOV 0.68506 0.16623 4.1212
## formCIC 0.79650 0.38820 2.0518
## formIND -0.11706 0.09062 -1.2917
## categoryresidential -0.42317 0.05241 -8.0737
## regionEast of England 0.27416 0.09604 2.8545
## regionLondon 0.25956 0.09760 2.6593
## regionNorth East 0.82783 0.14949 5.5377
## regionNorth West 0.42044 0.09581 4.3880
## regionSouth East 0.59711 0.08920 6.6943
## regionSouth West 0.84112 0.10341 8.1339
## regionWest Midlands 0.10125 0.09313 1.0872
## regionYorkshire and The Humber -0.06824 0.09532 -0.7159
## inheritedTRUE -0.14001 0.09634 -1.4533
##
## Intercepts:
## Value Std. Error t value
## Inadequate|Req improv -4.2534 0.1083 -39.2888
## Req improv|Good -1.4443 0.0785 -18.4062
## Good|Outstanding 5.8556 0.1627 35.9944
##
## Residual Deviance: 13406.48
## AIC: 13440.48
## (1153 observations deleted due to missingness)brant(model_order_safe_mass)## --------------------------------------------------------------------
## Test for X2 df probability
## --------------------------------------------------------------------
## Omnibus 35.6 28 0.15
## formNPO 2.31 2 0.32
## formGOV 0 2 1
## formCIC 1.86 2 0.39
## formIND 0.37 2 0.83
## categoryresidential 0.85 2 0.65
## regionEast of England 4.99 2 0.08
## regionLondon 7.99 2 0.02
## regionNorth East 5.64 2 0.06
## regionNorth West 8.07 2 0.02
## regionSouth East 8.32 2 0.02
## regionSouth West 9.21 2 0.01
## regionWest Midlands 7.01 2 0.03
## regionYorkshire and The Humber 4.04 2 0.13
## inheritedTRUE 1.38 2 0.5
## --------------------------------------------------------------------
##
## H0: Parallel Regression Assumption holds## Warning in brant(model_order_safe_mass): 418 combinations in table(dv,ivs) do
## not occur. Because of that, the test results might be invalid.model_order_effective_mass <- polr(rating ~ form + category + region + inherited,
data = filter(socialcare, domain == "Effective"),
Hess = TRUE)
summary(model_order_effective_mass)## Call:
## polr(formula = rating ~ form + category + region + inherited,
## data = filter(socialcare, domain == "Effective"), Hess = TRUE)
##
## Coefficients:
## Value Std. Error t value
## formNPO 0.4132 0.08547 4.8346
## formGOV 0.3966 0.18109 2.1898
## formCIC 0.1878 0.38125 0.4926
## formIND -0.2747 0.10739 -2.5575
## categoryresidential -0.3740 0.06289 -5.9464
## regionEast of England 0.3505 0.11561 3.0316
## regionLondon 0.4824 0.12028 4.0105
## regionNorth East 0.9481 0.17414 5.4446
## regionNorth West 0.5825 0.11664 4.9937
## regionSouth East 0.4160 0.10362 4.0147
## regionSouth West 0.9320 0.12227 7.6221
## regionWest Midlands 0.1683 0.11139 1.5107
## regionYorkshire and The Humber 0.2809 0.11955 2.3499
## inheritedTRUE -0.2884 0.11119 -2.5937
##
## Intercepts:
## Value Std. Error t value
## Inadequate|Req improv -5.9316 0.2061 -28.7806
## Req improv|Good -2.0790 0.0942 -22.0816
## Good|Outstanding 4.3955 0.1151 38.1915
##
## Residual Deviance: 10331.18
## AIC: 10365.18
## (1187 observations deleted due to missingness)brant(model_order_effective_mass)## --------------------------------------------------------------------
## Test for X2 df probability
## --------------------------------------------------------------------
## Omnibus 42.67 28 0.04
## formNPO 4.84 2 0.09
## formGOV 0.44 2 0.8
## formCIC 0.08 2 0.96
## formIND 4.44 2 0.11
## categoryresidential 8.61 2 0.01
## regionEast of England 3.61 2 0.16
## regionLondon 7.03 2 0.03
## regionNorth East 1.06 2 0.59
## regionNorth West 0 2 1
## regionSouth East 0.65 2 0.72
## regionSouth West 2.53 2 0.28
## regionWest Midlands 6.36 2 0.04
## regionYorkshire and The Humber 2.16 2 0.34
## inheritedTRUE 0.59 2 0.75
## --------------------------------------------------------------------
##
## H0: Parallel Regression Assumption holds## Warning in brant(model_order_effective_mass): 418 combinations in table(dv,ivs)
## do not occur. Because of that, the test results might be invalid.model_order_caring_mass <- polr(rating ~ form + category + region + inherited,
data = filter(socialcare, domain == "Caring"),
Hess = TRUE)
summary(model_order_caring_mass)## Call:
## polr(formula = rating ~ form + category + region + inherited,
## data = filter(socialcare, domain == "Caring"), Hess = TRUE)
##
## Coefficients:
## Value Std. Error t value
## formNPO 0.31621 0.08701 3.6343
## formGOV 0.32815 0.17877 1.8356
## formCIC 0.81934 0.32841 2.4949
## formIND -0.04976 0.13362 -0.3724
## categoryresidential -0.57419 0.06834 -8.4013
## regionEast of England 0.20752 0.14329 1.4482
## regionLondon 0.03723 0.14475 0.2572
## regionNorth East 0.86390 0.17957 4.8110
## regionNorth West 0.28595 0.13982 2.0451
## regionSouth East 0.50497 0.12898 3.9150
## regionSouth West 1.03022 0.13580 7.5865
## regionWest Midlands -0.20455 0.14079 -1.4529
## regionYorkshire and The Humber 0.45029 0.14794 3.0437
## inheritedTRUE -0.39689 0.13252 -2.9949
##
## Intercepts:
## Value Std. Error t value
## Inadequate|Req improv -7.0940 0.3098 -22.8992
## Req improv|Good -3.4307 0.1218 -28.1595
## Good|Outstanding 3.0235 0.1192 25.3586
##
## Residual Deviance: 8935.047
## AIC: 8969.047
## (1190 observations deleted due to missingness)brant(model_order_caring_mass)## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred## --------------------------------------------------------------------
## Test for X2 df probability
## --------------------------------------------------------------------
## Omnibus 63.07 28 0
## formNPO 11.8 2 0
## formGOV 3.4 2 0.18
## formCIC 1.03 2 0.6
## formIND 5.31 2 0.07
## categoryresidential 13.3 2 0
## regionEast of England 0.26 2 0.88
## regionLondon 12.8 2 0
## regionNorth East 2.11 2 0.35
## regionNorth West 4.55 2 0.1
## regionSouth East 5.37 2 0.07
## regionSouth West 0.4 2 0.82
## regionWest Midlands 0.99 2 0.61
## regionYorkshire and The Humber 8.44 2 0.01
## inheritedTRUE 0.07 2 0.97
## --------------------------------------------------------------------
##
## H0: Parallel Regression Assumption holds## Warning in brant(model_order_caring_mass): 424 combinations in table(dv,ivs) do
## not occur. Because of that, the test results might be invalid.model_order_well_led_mass <- polr(rating ~ form + category + region + inherited,
data = filter(socialcare, domain == "Well-led"),
Hess = TRUE)
summary(model_order_well_led_mass)## Call:
## polr(formula = rating ~ form + category + region + inherited,
## data = filter(socialcare, domain == "Well-led"), Hess = TRUE)
##
## Coefficients:
## Value Std. Error t value
## formNPO 0.47490 0.05958 7.971
## formGOV 0.44546 0.12536 3.553
## formCIC 0.47781 0.27763 1.721
## formIND -0.16189 0.08095 -2.000
## categoryresidential -0.30197 0.04414 -6.842
## regionEast of England 0.13262 0.08847 1.499
## regionLondon 0.13230 0.08910 1.485
## regionNorth East 0.74895 0.12462 6.010
## regionNorth West 0.25723 0.08681 2.963
## regionSouth East 0.23484 0.07968 2.947
## regionSouth West 0.71156 0.09011 7.896
## regionWest Midlands -0.33072 0.08471 -3.904
## regionYorkshire and The Humber -0.14441 0.08949 -1.614
## inheritedTRUE -0.07573 0.08480 -0.893
##
## Intercepts:
## Value Std. Error t value
## Inadequate|Req improv -4.1017 0.0973 -42.1342
## Req improv|Good -1.3337 0.0724 -18.4323
## Good|Outstanding 3.1785 0.0818 38.8350
##
## Residual Deviance: 18864.49
## AIC: 18898.49
## (1157 observations deleted due to missingness)brant(model_order_well_led_mass)## --------------------------------------------------------------------
## Test for X2 df probability
## --------------------------------------------------------------------
## Omnibus 98.49 28 0
## formNPO 11.01 2 0
## formGOV 1.95 2 0.38
## formCIC 6.57 2 0.04
## formIND 6.36 2 0.04
## categoryresidential 0.74 2 0.69
## regionEast of England 0.02 2 0.99
## regionLondon 23.1 2 0
## regionNorth East 5.38 2 0.07
## regionNorth West 14.18 2 0
## regionSouth East 19.61 2 0
## regionSouth West 13.03 2 0
## regionWest Midlands 5.16 2 0.08
## regionYorkshire and The Humber 7.64 2 0.02
## inheritedTRUE 3.93 2 0.14
## --------------------------------------------------------------------
##
## H0: Parallel Regression Assumption holds## Warning in brant(model_order_well_led_mass): 345 combinations in table(dv,ivs)
## do not occur. Because of that, the test results might be invalid.model_order_responsive_mass <- polr(rating ~ form + category + region + inherited,
data = filter(socialcare, domain == "Responsive"),
Hess = TRUE)
summary(model_order_responsive_mass)## Call:
## polr(formula = rating ~ form + category + region + inherited,
## data = filter(socialcare, domain == "Responsive"), Hess = TRUE)
##
## Coefficients:
## Value Std. Error t value
## formNPO 0.506885 0.07251 6.99082
## formGOV 0.414927 0.15029 2.76077
## formCIC 0.305849 0.32039 0.95460
## formIND -0.056738 0.10613 -0.53461
## categoryresidential -0.236602 0.05527 -4.28090
## regionEast of England 0.279675 0.11344 2.46532
## regionLondon -0.029464 0.11258 -0.26172
## regionNorth East 0.644790 0.14872 4.33547
## regionNorth West 0.323772 0.11060 2.92745
## regionSouth East 0.404234 0.10197 3.96420
## regionSouth West 0.721171 0.11073 6.51263
## regionWest Midlands -0.001995 0.11050 -0.01806
## regionYorkshire and The Humber 0.069080 0.11669 0.59201
## inheritedTRUE -0.251177 0.10529 -2.38548
##
## Intercepts:
## Value Std. Error t value
## Inadequate|Req improv -6.8557 0.3015 -22.7359
## Req improv|Good -2.3647 0.0933 -25.3561
## Good|Outstanding 2.9800 0.0968 30.7767
##
## Residual Deviance: 12692.39
## AIC: 12726.39
## (1165 observations deleted due to missingness)brant(model_order_responsive_mass)## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred## --------------------------------------------------------------------
## Test for X2 df probability
## --------------------------------------------------------------------
## Omnibus 88.61 28 0
## formNPO 8.07 2 0.02
## formGOV 7.13 2 0.03
## formCIC 0.05 2 0.98
## formIND 11.36 2 0
## categoryresidential 39.97 2 0
## regionEast of England 0.69 2 0.71
## regionLondon 6.27 2 0.04
## regionNorth East 2.76 2 0.25
## regionNorth West 1.18 2 0.56
## regionSouth East 1.71 2 0.42
## regionSouth West 2.07 2 0.35
## regionWest Midlands 8.96 2 0.01
## regionYorkshire and The Humber 1.74 2 0.42
## inheritedTRUE 0.62 2 0.73
## --------------------------------------------------------------------
##
## H0: Parallel Regression Assumption holds## Warning in brant(model_order_responsive_mass): 409 combinations in table(dv,ivs)
## do not occur. Because of that, the test results might be invalid.Showing the original coeffients from the ordered logit models.
ordinal_models <- modelsummary(list("overall" = model_order_overall, "safe" = model_order_safe,
"effective" = model_order_effective, "caring"= model_order_caring,
"well-led" = model_order_well_led, "responsive" = model_order_responsive),
statistic = "({p.value}) {stars}")
ordinal_models| overall | safe | effective | caring | well-led | responsive | |
|---|---|---|---|---|---|---|
| Inadequate|Req improv | −4.449*** | −4.253*** | −5.932*** | −7.094*** | −4.102*** | −6.856*** |
| (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | |
| Req improv|Good | −1.702*** | −1.444*** | −2.079*** | −3.431*** | −1.334*** | −2.365*** |
| (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | |
| Good|Outstanding | 3.144*** | 5.856*** | 4.395*** | 3.024*** | 3.178*** | 2.980*** |
| (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | |
| formNPO | 0.457*** | 0.580*** | 0.413*** | 0.316*** | 0.475*** | 0.507*** |
| (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | |
| formGOV | 0.429** | 0.685*** | 0.397* | 0.328+ | 0.445*** | 0.415** |
| (0.001) ** | (0.000) *** | (0.029) * | (0.066) + | (0.000) *** | (0.006) ** | |
| formCIC | 0.586* | 0.797* | 0.188 | 0.819* | 0.478+ | 0.306 |
| (0.045) * | (0.040) * | (0.622) | (0.013) * | (0.085) + | (0.340) | |
| formIND | −0.183* | −0.117 | −0.275* | −0.050 | −0.162* | −0.057 |
| (0.036) * | (0.197) | (0.011) * | (0.710) | (0.045) * | (0.592) | |
| categoryresidential | −0.387*** | −0.423*** | −0.374*** | −0.574*** | −0.302*** | −0.237*** |
| (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | |
| regionEast of England | 0.245* | 0.274** | 0.350** | 0.208 | 0.133 | 0.280* |
| (0.011) * | (0.004) ** | (0.002) ** | (0.147) | (0.134) | (0.014) * | |
| regionLondon | 0.104 | 0.260** | 0.482*** | 0.037 | 0.132 | −0.029 |
| (0.279) | (0.008) ** | (0.000) *** | (0.797) | (0.138) | (0.794) | |
| regionNorth East | 0.761*** | 0.828*** | 0.948*** | 0.864*** | 0.749*** | 0.645*** |
| (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | |
| regionNorth West | 0.293** | 0.420*** | 0.582*** | 0.286* | 0.257** | 0.324** |
| (0.002) ** | (0.000) *** | (0.000) *** | (0.041) * | (0.003) ** | (0.003) ** | |
| regionSouth East | 0.322*** | 0.597*** | 0.416*** | 0.505*** | 0.235** | 0.404*** |
| (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.003) ** | (0.000) *** | |
| regionSouth West | 0.836*** | 0.841*** | 0.932*** | 1.030*** | 0.712*** | 0.721*** |
| (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | |
| regionWest Midlands | −0.156+ | 0.101 | 0.168 | −0.204 | −0.331*** | −0.002 |
| (0.090) + | (0.277) | (0.131) | (0.146) | (0.000) *** | (0.985) | |
| regionYorkshire and The Humber | −0.121 | −0.068 | 0.281* | 0.450** | −0.144 | 0.069 |
| (0.208) | (0.474) | (0.019) * | (0.002) ** | (0.107) | (0.554) | |
| inheritedTRUE | −0.108 | −0.140 | −0.288** | −0.397** | −0.076 | −0.251* |
| (0.238) | (0.146) | (0.010) ** | (0.003) ** | (0.372) | (0.017) * | |
| Num.Obs. | 12942 | 12966 | 12931 | 12927 | 12964 | 12953 |
| AIC | 16608.8 | 13440.5 | 10365.2 | 8969.0 | 18898.5 | 12726.4 |
| BIC | 16735.8 | 13567.5 | 10492.1 | 9096.0 | 19025.5 | 12853.4 |
| RMSE | 2.26 | 2.12 | 2.13 | 2.21 | 2.26 | 2.27 |
Showing the exponent of the coefficients (log-odds)
ordinal_models_exp <- modelsummary(list("overall" = model_order_overall, "safe" = model_order_safe,
"effective" = model_order_effective, "caring"= model_order_caring,
"well-led" = model_order_well_led, "responsive" = model_order_responsive),
statistic = "({p.value}) {stars}", exponentiate = TRUE)
ordinal_models_exp| overall | safe | effective | caring | well-led | responsive | |
|---|---|---|---|---|---|---|
| Inadequate|Req improv | 0.012*** | 0.014*** | 0.003*** | 0.001*** | 0.017*** | 0.001*** |
| (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | |
| Req improv|Good | 0.182*** | 0.236*** | 0.125*** | 0.032*** | 0.263*** | 0.094*** |
| (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | |
| Good|Outstanding | 23.200*** | 349.197*** | 81.076*** | 20.564*** | 24.009*** | 19.686*** |
| (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | |
| formNPO | 1.580*** | 1.785*** | 1.512*** | 1.372*** | 1.608*** | 1.660*** |
| (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | |
| formGOV | 1.536** | 1.984*** | 1.487* | 1.389+ | 1.561*** | 1.515** |
| (0.001) ** | (0.000) *** | (0.029) * | (0.066) + | (0.000) *** | (0.006) ** | |
| formCIC | 1.797* | 2.218* | 1.207 | 2.269* | 1.612+ | 1.358 |
| (0.045) * | (0.040) * | (0.622) | (0.013) * | (0.085) + | (0.340) | |
| formIND | 0.833* | 0.890 | 0.760* | 0.951 | 0.851* | 0.945 |
| (0.036) * | (0.197) | (0.011) * | (0.710) | (0.045) * | (0.592) | |
| categoryresidential | 0.679*** | 0.655*** | 0.688*** | 0.563*** | 0.739*** | 0.789*** |
| (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | |
| regionEast of England | 1.278* | 1.315** | 1.420** | 1.231 | 1.142 | 1.323* |
| (0.011) * | (0.004) ** | (0.002) ** | (0.147) | (0.134) | (0.014) * | |
| regionLondon | 1.109 | 1.296** | 1.620*** | 1.038 | 1.141 | 0.971 |
| (0.279) | (0.008) ** | (0.000) *** | (0.797) | (0.138) | (0.794) | |
| regionNorth East | 2.140*** | 2.288*** | 2.580*** | 2.373*** | 2.115*** | 1.906*** |
| (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | |
| regionNorth West | 1.340** | 1.523*** | 1.790*** | 1.331* | 1.293** | 1.382** |
| (0.002) ** | (0.000) *** | (0.000) *** | (0.041) * | (0.003) ** | (0.003) ** | |
| regionSouth East | 1.379*** | 1.817*** | 1.516*** | 1.657*** | 1.265** | 1.498*** |
| (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.003) ** | (0.000) *** | |
| regionSouth West | 2.308*** | 2.319*** | 2.539*** | 2.802*** | 2.037*** | 2.057*** |
| (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | (0.000) *** | |
| regionWest Midlands | 0.856+ | 1.107 | 1.183 | 0.815 | 0.718*** | 0.998 |
| (0.090) + | (0.277) | (0.131) | (0.146) | (0.000) *** | (0.985) | |
| regionYorkshire and The Humber | 0.886 | 0.934 | 1.324* | 1.569** | 0.866 | 1.071 |
| (0.208) | (0.474) | (0.019) * | (0.002) ** | (0.107) | (0.554) | |
| inheritedTRUE | 0.898 | 0.869 | 0.750** | 0.672** | 0.927 | 0.778* |
| (0.238) | (0.146) | (0.010) ** | (0.003) ** | (0.372) | (0.017) * | |
| Num.Obs. | 12942 | 12966 | 12931 | 12927 | 12964 | 12953 |
| AIC | 16608.8 | 13440.5 | 10365.2 | 8969.0 | 18898.5 | 12726.4 |
| BIC | 16735.8 | 13567.5 | 10492.1 | 9096.0 | 19025.5 | 12853.4 |
| RMSE | 2.26 | 2.12 | 2.13 | 2.21 | 2.26 | 2.27 |
cic_num <- socialcare_num %>%
filter(form_num == 4)Now we know that for this dataset, OLS and ordered logit models are consistent and the latter is more robust. We will only run the ordered logit model to test if there any difference between the subtypes of CICs.
# run the ordered logit model with control
model_order_overall_cic <- clm(rating ~ category + region + inherited + cic_type,
data = filter(cic_num, domain == "Overall"),
link = "logit")
model_order_safe_cic <- clm(rating ~ category + region + inherited + cic_type,
data = filter(cic_num, domain == "Safe"),
link = "logit")
model_order_effective_cic <- clm(rating ~ category + region + inherited + cic_type,
data = filter(cic_num, domain == "Effective"),
link = "logit")
model_order_caring_cic <- clm(rating ~ category + region + inherited + cic_type,
data = filter(cic_num, domain == "Caring"),
link = "logit")
model_order_well_led_cic <- clm(rating ~ category + region + inherited + cic_type,
data = filter(cic_num, domain == "Well-led"),
link = "logit")
model_order_responsive_cic <- clm(rating ~ category + region + inherited + cic_type,
data = filter(cic_num, domain == "Responsive"),
link = "logit")model_order_overall_cicmass <- polr(rating ~ category + region + inherited + cic_type,
data = filter(socialcare, domain == "Overall"),
Hess = TRUE)## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurredsummary(model_order_overall_cicmass)## Call:
## polr(formula = rating ~ category + region + inherited + cic_type,
## data = filter(socialcare, domain == "Overall"), Hess = TRUE)
##
## Coefficients:
## Value Std. Error t value
## categoryresidential -0.8625 0.7850 -1.0987
## regionEast of England -1.0893 1.5295 -0.7122
## regionLondon -3.2656 1.5883 -2.0560
## regionNorth East -2.1399 2.3702 -0.9028
## regionNorth West 0.2104 1.2989 0.1620
## regionSouth East -3.2574 1.7198 -1.8941
## regionSouth West -0.5142 1.5356 -0.3349
## regionWest Midlands -3.4236 1.6419 -2.0852
## regionYorkshire and The Humber -2.3231 1.6905 -1.3742
## inheritedTRUE 0.4003 2.5239 0.1586
## cic_typeCIC_S -0.7389 0.6712 -1.1010
##
## Intercepts:
## Value Std. Error t value
## Inadequate|Req improv -15.0870 55.9461 -0.2697
## Req improv|Good -4.9253 1.4335 -3.4358
## Good|Outstanding 0.9100 1.1452 0.7946
##
## Residual Deviance: 87.05844
## AIC: 115.0584
## (14034 observations deleted due to missingness)brant(model_order_overall_cicmass)## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred## --------------------------------------------------------------------
## Test for X2 df probability
## --------------------------------------------------------------------
## Omnibus 1.44 22 1
## categoryresidential 0.17 2 0.92
## regionEast of England 0 2 1
## regionLondon 0 2 1
## regionNorth East 0 2 1
## regionNorth West 0 2 1
## regionSouth East 0 2 1
## regionSouth West 0 2 1
## regionWest Midlands 0 2 1
## regionYorkshire and The Humber 0 2 1
## inheritedTRUE 0 2 1
## cic_typeCIC_S 1.38 2 0.5
## --------------------------------------------------------------------
##
## H0: Parallel Regression Assumption holds## Warning in brant(model_order_overall_cicmass): 181 combinations in table(dv,ivs)
## do not occur. Because of that, the test results might be invalid.model_order_effective_cicmass <- polr(rating ~ category + region + inherited + cic_type,
data = filter(socialcare, domain == "Effective"),
Hess = TRUE)## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurredsummary(model_order_effective_cicmass)## Call:
## polr(formula = rating ~ category + region + inherited + cic_type,
## data = filter(socialcare, domain == "Effective"), Hess = TRUE)
##
## Coefficients:
## Value Std. Error t value
## categoryresidential -1.38870 0.9498 -1.462100
## regionEast of England -1.23843 2.4088 -0.514132
## regionLondon -0.02871 2.5667 -0.011185
## regionNorth East -0.47056 3.3106 -0.142139
## regionNorth West 0.52455 2.3816 0.220253
## regionSouth East 0.02433 2.7603 0.008815
## regionSouth West 0.02433 2.7603 0.008815
## regionWest Midlands -1.93175 2.3936 -0.807058
## regionYorkshire and The Humber 0.28596 2.6192 0.109177
## inheritedTRUE 1.49685 2.9268 0.511428
## cic_typeCIC_S 0.22315 0.9756 0.228723
##
## Intercepts:
## Value Std. Error t value
## Inadequate|Req improv -13.5410 50.9815 -0.2656
## Req improv|Good -3.4853 2.2104 -1.5768
## Good|Outstanding 3.6047 2.2084 1.6323
##
## Residual Deviance: 53.62115
## AIC: 81.62115
## (14034 observations deleted due to missingness)brant(model_order_effective_cicmass)## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred## --------------------------------------------------------------------
## Test for X2 df probability
## --------------------------------------------------------------------
## Omnibus 0.71 22 1
## categoryresidential 0 2 1
## regionEast of England 0 2 1
## regionLondon 0 2 1
## regionNorth East 0 2 1
## regionNorth West 0 2 1
## regionSouth East 0 2 1
## regionSouth West 0 2 1
## regionWest Midlands 0 2 1
## regionYorkshire and The Humber 0 2 1
## inheritedTRUE 0 2 1
## cic_typeCIC_S 0.01 2 1
## --------------------------------------------------------------------
##
## H0: Parallel Regression Assumption holds## Warning in brant(model_order_effective_cicmass): 185 combinations in
## table(dv,ivs) do not occur. Because of that, the test results might be invalid.model_order_caring_cicmass <- polr(rating ~ category + region + inherited + cic_type,
data = filter(socialcare, domain == "Caring"),
Hess = TRUE)## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurredsummary(model_order_caring_cicmass)## Call:
## polr(formula = rating ~ category + region + inherited + cic_type,
## data = filter(socialcare, domain == "Caring"), Hess = TRUE)
##
## Coefficients:
## Value Std. Error t value
## categoryresidential -0.67637 0.8441 -0.80134
## regionEast of England 2.53692 2.4541 1.03377
## regionLondon 0.12306 2.5916 0.04748
## regionNorth East -1.74068 2.9311 -0.59386
## regionNorth West 2.95078 2.3830 1.23825
## regionSouth East -0.06988 2.8403 -0.02460
## regionSouth West 1.95859 2.5283 0.77468
## regionWest Midlands -1.53848 2.5394 -0.60583
## regionYorkshire and The Humber 2.10617 2.5193 0.83602
## inheritedTRUE 3.80774 2.1662 1.75778
## cic_typeCIC_S -0.84790 0.7554 -1.12249
##
## Intercepts:
## Value Std. Error t value
## Inadequate|Req improv -19.6452 0.0102 -1920.0180
## Req improv|Good -3.8755 2.3133 -1.6754
## Good|Outstanding 3.4356 2.2966 1.4960
##
## Residual Deviance: 69.07379
## AIC: 97.07379
## (14033 observations deleted due to missingness)brant(model_order_caring_cicmass)## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred## --------------------------------------------------------------------
## Test for X2 df probability
## --------------------------------------------------------------------
## Omnibus 0.65 22 1
## categoryresidential 0 2 1
## regionEast of England 0 2 1
## regionLondon 0 2 1
## regionNorth East 0 2 1
## regionNorth West 0 2 1
## regionSouth East 0 2 1
## regionSouth West 0 2 1
## regionWest Midlands 0 2 1
## regionYorkshire and The Humber 0 2 1
## inheritedTRUE 0 2 1
## cic_typeCIC_S 0.52 2 0.77
## --------------------------------------------------------------------
##
## H0: Parallel Regression Assumption holds## Warning in brant(model_order_caring_cicmass): 184 combinations in table(dv,ivs)
## do not occur. Because of that, the test results might be invalid.model_order_well_led_cicmass <- polr(rating ~ category + region + inherited + cic_type,
data = filter(socialcare, domain == "Well-led"),
Hess = TRUE)
summary(model_order_well_led_cicmass)## Call:
## polr(formula = rating ~ category + region + inherited + cic_type,
## data = filter(socialcare, domain == "Well-led"), Hess = TRUE)
##
## Coefficients:
## Value Std. Error t value
## categoryresidential 0.05882 0.6357 0.09253
## regionEast of England -1.47101 1.3727 -1.07164
## regionLondon -2.11671 1.3792 -1.53476
## regionNorth East -3.40117 1.7926 -1.89735
## regionNorth West 0.43311 1.2312 0.35180
## regionSouth East -2.13729 1.4878 -1.43654
## regionSouth West -0.41668 1.4184 -0.29376
## regionWest Midlands -3.18905 1.4605 -2.18354
## regionYorkshire and The Humber -3.22033 1.4372 -2.24075
## inheritedTRUE 1.36124 1.8069 0.75337
## cic_typeCIC_S -0.67589 0.5851 -1.15516
##
## Intercepts:
## Value Std. Error t value
## Inadequate|Req improv -13.7273 30.9643 -0.4433
## Req improv|Good -3.6111 1.2386 -2.9156
## Good|Outstanding 0.8423 1.0868 0.7751
##
## Residual Deviance: 117.8561
## AIC: 145.8561
## (14037 observations deleted due to missingness)brant(model_order_well_led_cicmass)## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred## --------------------------------------------------------------------
## Test for X2 df probability
## --------------------------------------------------------------------
## Omnibus 6.14 22 1
## categoryresidential 0.75 2 0.69
## regionEast of England 0 2 1
## regionLondon 0 2 1
## regionNorth East 0 2 1
## regionNorth West 0 2 1
## regionSouth East 0 2 1
## regionSouth West 0 2 1
## regionWest Midlands 0 2 1
## regionYorkshire and The Humber 0 2 1
## inheritedTRUE 0 2 1
## cic_typeCIC_S 5.13 2 0.08
## --------------------------------------------------------------------
##
## H0: Parallel Regression Assumption holds## Warning in brant(model_order_well_led_cicmass): 177 combinations in
## table(dv,ivs) do not occur. Because of that, the test results might be invalid.model_order_responsive_cicmass <- polr(rating ~ category + region + inherited + cic_type,
data = filter(socialcare, domain == "Responsive"),
Hess = TRUE)## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurredsummary(model_order_responsive_cicmass)## Call:
## polr(formula = rating ~ category + region + inherited + cic_type,
## data = filter(socialcare, domain == "Responsive"), Hess = TRUE)
##
## Coefficients:
## Value Std. Error t value
## categoryresidential -1.55331 0.9781 -1.58806
## regionEast of England -3.09331 1.8866 -1.63963
## regionLondon -4.16348 1.9546 -2.13013
## regionNorth East -3.07223 3.1054 -0.98934
## regionNorth West 0.10702 1.3218 0.08096
## regionSouth East -2.61874 2.3034 -1.13689
## regionSouth West -2.61874 2.3034 -1.13689
## regionWest Midlands -3.69733 2.0133 -1.83643
## regionYorkshire and The Humber -2.14399 2.0087 -1.06737
## inheritedTRUE 1.27004 3.2721 0.38814
## cic_typeCIC_S -0.07933 0.7961 -0.09965
##
## Intercepts:
## Value Std. Error t value
## Inadequate|Req improv -14.3475 25.7418 -0.5574
## Req improv|Good -6.3436 1.7279 -3.6714
## Good|Outstanding 1.0800 1.1669 0.9255
##
## Residual Deviance: 59.93914
## AIC: 87.93914
## (14034 observations deleted due to missingness)brant(model_order_responsive_cicmass)## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred## --------------------------------------------------------------------
## Test for X2 df probability
## --------------------------------------------------------------------
## Omnibus 0.82 22 1
## categoryresidential 0 2 1
## regionEast of England 0 2 1
## regionLondon 0 2 1
## regionNorth East 0 2 1
## regionNorth West 0 2 1
## regionSouth East 0 2 1
## regionSouth West 0 2 1
## regionWest Midlands 0 2 1
## regionYorkshire and The Humber 0 2 1
## inheritedTRUE 0 2 1
## cic_typeCIC_S 0.82 2 0.66
## --------------------------------------------------------------------
##
## H0: Parallel Regression Assumption holds## Warning in brant(model_order_responsive_cicmass): 185 combinations in
## table(dv,ivs) do not occur. Because of that, the test results might be invalid.ordered_models_cic <- modelsummary(list("overall" = model_order_overall_cic, "safe" = model_order_safe_cic,
"effective" = model_order_effective_cic, "caring" = model_order_caring_cic,
"well-led" = model_order_well_led_cic, "responsive" = model_order_responsive_cic),
statistic = "({p.value}) {stars}")
ordered_models_cic| overall | safe | effective | caring | well-led | responsive | |
|---|---|---|---|---|---|---|
| Req improv|Good | −4.924*** | −4.165+ | −3.490 | −3.875+ | −3.606** | −6.357*** |
| (0.001) *** | (0.064) + | (0.115) | (0.094) + | (0.004) ** | (0.000) *** | |
| Good|Outstanding | 0.911 | 3.284 | 3.601 | 3.436 | 0.847 | 1.072 |
| (0.427) | (0.136) | (0.103) | (0.135) | (0.436) | (0.357) | |
| categoryresidential | −0.863 | 0.554 | −1.387 | −0.676 | 0.058 | −1.552 |
| (0.272) | (0.608) | (0.144) | (0.423) | (0.927) | (0.113) | |
| regionEast of England | −1.089 | −0.110 | −1.243 | 2.537 | −1.466 | −3.108+ |
| (0.477) | (0.963) | (0.606) | (0.301) | (0.286) | (0.100) + | |
| regionLondon | −3.264* | −2.579 | −0.031 | 0.123 | −2.112 | −4.177* |
| (0.040) * | (0.266) | (0.990) | (0.962) | (0.126) | (0.033) * | |
| regionNorth East | −2.144 | −0.581 | −0.472 | −1.740 | −3.395+ | −3.084 |
| (0.366) | (0.873) | (0.887) | (0.553) | (0.058) + | (0.321) | |
| regionNorth West | 0.211 | 1.059 | 0.519 | 2.951 | 0.439 | 0.098 |
| (0.871) | (0.651) | (0.827) | (0.216) | (0.722) | (0.941) | |
| regionSouth East | −3.256+ | −2.201 | 0.019 | −0.071 | −2.132 | −2.629 |
| (0.058) + | (0.373) | (0.995) | (0.980) | (0.152) | (0.254) | |
| regionSouth West | −0.514 | −0.130 | 0.019 | 1.959 | −0.412 | −2.629 |
| (0.738) | (0.962) | (0.995) | (0.438) | (0.772) | (0.254) | |
| regionWest Midlands | −3.422* | −1.911 | −1.937 | −1.538 | −3.183* | −3.713+ |
| (0.037) * | (0.440) | (0.418) | (0.545) | (0.029) * | (0.065) + | |
| regionYorkshire and The Humber | −2.322 | −1.242 | 0.287 | 2.107 | −3.215* | −2.160 |
| (0.170) | (0.615) | (0.913) | (0.403) | (0.025) * | (0.283) | |
| inheritedTRUE | 0.411 | 0.418 | 1.495 | 3.808+ | 1.360 | 1.269 |
| (0.871) | (0.909) | (0.609) | (0.079) + | (0.452) | (0.698) | |
| cic_typeCIC_S | −0.739 | −1.581+ | 0.223 | −0.848 | −0.676 | −0.078 |
| (0.271) | (0.063) + | (0.820) | (0.262) | (0.248) | (0.922) | |
| Num.Obs. | 84 | 84 | 84 | 84 | 84 | 84 |
| AIC | 113.1 | 84.2 | 79.6 | 95.1 | 143.9 | 85.9 |
| BIC | 144.7 | 115.8 | 111.2 | 126.7 | 175.5 | 117.5 |
| RMSE | 1.40 | 1.16 | 1.15 | 1.45 | 1.49 | 1.30 |
end of document updated on 8/31/2022