Civil job market research

Warning: package 'pROC' was built under R version 4.3.3
Warning: package 'zoo' was built under R version 4.3.2
Warning: package 'aod' was built under R version 4.3.2
Warning: package 'readxl' was built under R version 4.3.2
Warning: package 'ggplot2' was built under R version 4.3.3
Warning: package 'glmtoolbox' was built under R version 4.3.3
Warning: package 'logistf' was built under R version 4.3.3
Warning: package 'mfx' was built under R version 4.3.3
Warning: package 'sandwich' was built under R version 4.3.2
Warning: package 'lmtest' was built under R version 4.3.2
Warning: package 'betareg' was built under R version 4.3.3
Warning: package 'maxLik' was built under R version 4.3.2
Warning: package 'miscTools' was built under R version 4.3.2
Warning: package 'ResourceSelection' was built under R version 4.3.3
Warning: package 'questionr' was built under R version 4.3.3

Data set overview

The purpose of this research is to determine what factors influence whether an open job position in polish civil job market will find a candidate or not. Job postings in polish civil market are set for a finite amount of time. After closure of a job posting there are interviews and information about hiring or not hiring a candidate is released to public.

We have performed web scraping of job postings from the website: . It is a governmental website with all job postings in governmental institutions. We have gathered 2500 observations from November and December 2023. Some observations have been dropped due to posting cancellations. In the end 2107 job postings have been used. Below is a list of gathered variables:

  • y - result of a job posting. Binary variable with information whether a candidate was hired,

  • x1 - city class. The city of workplace have been put in one of five brackets based on the population of the city (0 - below 100 000 inhabitants, 1 - 100 001 - 250 000 inhabitants, 2 - 250 001 - 500 000 inhabitants, 3 - 500 001 - 1 000 000 inhabitants and 4 over 1 000 000 inhabitants. The last bracket accounts for Warsaw only as most of governmental institutions (and job postings) are in the capital city),

  • x2 - job group. In the polish civil market all job positions are in accordance to law regulation that describes all job positions and to what grade they belong to. The higher the grade the higher the qualifications and pay. There are 5 levels, but in our data set there were no job postings with highest grade,

  • x3 - number of positions. In some cases there were more than one position needed per job posting,

  • x4 - wage. As not all job postings had the wage displayed, we considered it a binary variable - whether the wage was given or not,

  • x5 - time. Proportion of full time of job posting,

  • x6 - education. Binary variable, where means 0 not requiring higher education, 1 otherwise,

  • x7 - Driving. Binary variable, where 0 means not requiring driving license, 1 otherwise,

  • x8 - English. Binary variable, where 0 means not requiring English language, 1 otherwise,

  • x9 - Hybrid. Binary variable, where 0 means no possibility of remote work, 1 otherwise,

  • x10 - Days_on. Amount of days between job posting introduction and date by which the application documents may be submitted,

  • x11 - Experience. Amount of experience required for the job position in years.

Data analysis

Some initial data analysis has been done beforehand, on the data acquisition level, here we are going to explore the data set further.

project<-read_xlsx("C:\\Users\\mateu\\Desktop\\Studies\\AdvancedEconometrics\\Project\\project.xlsx")
New names:
• `` -> `...1`
project %>%
  ggplot(aes(x=clean_result))+
  geom_bar()+
  labs(y="Count", x = "Job posting result")

selected_columns <- c("City_class", "Job_group", "education_clean",
                      "Number_of_positions", "Wage_clean", 
                      "driving_clean", "english_clean", "hybrid_clean")

plots <- list()

for (col in selected_columns) {
  # Create a bar plot for the current column
  plots[[col]] <- ggplot(project, aes_string(x = col)) +
                  geom_bar() +
                  labs(title = paste(col))
}
Warning: `aes_string()` was deprecated in ggplot2 3.0.0.
ℹ Please use tidy evaluation idioms with `aes()`.
ℹ See also `vignette("ggplot2-in-packages")` for more information.
combined_plot <- do.call(gridExtra::grid.arrange, c(plots, ncol = 3))

print(combined_plot)
TableGrob (3 x 3) "arrange": 8 grobs
                    z     cells    name           grob
City_class          1 (1-1,1-1) arrange gtable[layout]
Job_group           2 (1-1,2-2) arrange gtable[layout]
education_clean     3 (1-1,3-3) arrange gtable[layout]
Number_of_positions 4 (2-2,1-1) arrange gtable[layout]
Wage_clean          5 (2-2,2-2) arrange gtable[layout]
driving_clean       6 (2-2,3-3) arrange gtable[layout]
english_clean       7 (3-3,1-1) arrange gtable[layout]
hybrid_clean        8 (3-3,2-2) arrange gtable[layout]
selected_columns <- c("Days_on", "experience_clean", "time_clean")

plots <- list()

for (col in selected_columns) {
  # Create a bar plot for the current column
  plots[[col]] <- ggplot(project, aes_string(x = col)) +
                  geom_boxplot(fill = "skyblue", color = "black") +
                  labs(title = paste(col))
}

combined_plot <- do.call(gridExtra::grid.arrange, c(plots, ncol = 2))

print(combined_plot)
TableGrob (2 x 2) "arrange": 3 grobs
                 z     cells    name           grob
Days_on          1 (1-1,1-1) arrange gtable[layout]
experience_clean 2 (1-1,2-2) arrange gtable[layout]
time_clean       3 (2-2,1-1) arrange gtable[layout]

Binary Choice Model

1. Logit, probit model

Probit model with all variables gathered.

myprobit <- glm(clean_result~City_class+Job_group + Number_of_positions
                + Wage_clean + time_clean + education_clean + driving_clean
                + english_clean + hybrid_clean + Days_on + experience_clean, 
                data=project, family=binomial(link="probit"))
summary(myprobit)

Call:
glm(formula = clean_result ~ City_class + Job_group + Number_of_positions + 
    Wage_clean + time_clean + education_clean + driving_clean + 
    english_clean + hybrid_clean + Days_on + experience_clean, 
    family = binomial(link = "probit"), data = project)

Coefficients:
                     Estimate Std. Error z value Pr(>|z|)    
(Intercept)          0.081825   0.375621   0.218 0.827555    
City_class           0.022786   0.023021   0.990 0.322276    
Job_group            0.137713   0.059258   2.324 0.020128 *  
Number_of_positions  0.196542   0.075562   2.601 0.009293 ** 
Wage_clean           0.084322   0.061991   1.360 0.173754    
time_clean           0.617653   0.362639   1.703 0.088527 .  
education_clean     -1.007220   0.087080 -11.567  < 2e-16 ***
driving_clean       -0.405937   0.115060  -3.528 0.000419 ***
english_clean       -0.002332   0.084442  -0.028 0.977966    
hybrid_clean         0.225072   0.079237   2.841 0.004504 ** 
Days_on             -0.023062   0.005322  -4.334 1.47e-05 ***
experience_clean    -0.099783   0.028814  -3.463 0.000534 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 2891.9  on 2106  degrees of freedom
Residual deviance: 2523.8  on 2095  degrees of freedom
AIC: 2547.8

Number of Fisher Scoring iterations: 4
myprobit$coefficients
        (Intercept)          City_class           Job_group Number_of_positions 
        0.081824628         0.022786042         0.137712642         0.196542089 
         Wage_clean          time_clean     education_clean       driving_clean 
        0.084322148         0.617653059        -1.007219594        -0.405936778 
      english_clean        hybrid_clean             Days_on    experience_clean 
       -0.002332234         0.225072470        -0.023062001        -0.099783160 

Logit model with all gathered variables.

mylogit <- glm(clean_result~City_class+Job_group + Number_of_positions
                + Wage_clean + time_clean + education_clean + driving_clean
                + english_clean + hybrid_clean + Days_on + experience_clean, 
                data=project, family=binomial(link="logit"))
summary(mylogit)

Call:
glm(formula = clean_result ~ City_class + Job_group + Number_of_positions + 
    Wage_clean + time_clean + education_clean + driving_clean + 
    english_clean + hybrid_clean + Days_on + experience_clean, 
    family = binomial(link = "logit"), data = project)

Coefficients:
                     Estimate Std. Error z value Pr(>|z|)    
(Intercept)          0.106299   0.620491   0.171 0.863976    
City_class           0.040082   0.038087   1.052 0.292622    
Job_group            0.229806   0.098287   2.338 0.019382 *  
Number_of_positions  0.323053   0.127519   2.533 0.011297 *  
Wage_clean           0.137580   0.101776   1.352 0.176444    
time_clean           1.039710   0.598805   1.736 0.082509 .  
education_clean     -1.657599   0.145926 -11.359  < 2e-16 ***
driving_clean       -0.684984   0.192783  -3.553 0.000381 ***
english_clean       -0.006422   0.136991  -0.047 0.962610    
hybrid_clean         0.371233   0.130284   2.849 0.004380 ** 
Days_on             -0.037836   0.008974  -4.216 2.49e-05 ***
experience_clean    -0.163792   0.047429  -3.453 0.000554 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 2891.9  on 2106  degrees of freedom
Residual deviance: 2523.7  on 2095  degrees of freedom
AIC: 2547.7

Number of Fisher Scoring iterations: 4
mylogit$coefficients
        (Intercept)          City_class           Job_group Number_of_positions 
        0.106299376         0.040081994         0.229806162         0.323053176 
         Wage_clean          time_clean     education_clean       driving_clean 
        0.137579711         1.039709805        -1.657599143        -0.684984148 
      english_clean        hybrid_clean             Days_on    experience_clean 
       -0.006421884         0.371232641        -0.037835898        -0.163792147 

2. Choice between logit and probit models

The difference of AIC is slight. We will use the logit model.

3. General-to-specific method

mylogit1 = glm(clean_result~Job_group + Number_of_positions
                + education_clean + driving_clean
                + hybrid_clean + Days_on + experience_clean, 
                data=project, family=binomial(link="logit"))
general = glm(clean_result~City_class+Job_group + Number_of_positions
                + Wage_clean + time_clean + education_clean + driving_clean
                + english_clean + hybrid_clean + Days_on + experience_clean, 
                data=project, family=binomial(link="probit"))
anova(general, mylogit1)
Analysis of Deviance Table

Model 1: clean_result ~ City_class + Job_group + Number_of_positions + 
    Wage_clean + time_clean + education_clean + driving_clean + 
    english_clean + hybrid_clean + Days_on + experience_clean
Model 2: clean_result ~ Job_group + Number_of_positions + education_clean + 
    driving_clean + hybrid_clean + Days_on + experience_clean
  Resid. Df Resid. Dev Df Deviance
1      2095     2523.8            
2      2099     2530.5 -4  -6.7518

Analysis of variance tells us that all insignificant values are jointly insignificant. We can continue with significant values only.

4. Interaction between variables

We will check interaction between higher education requirement and possibility of remote work.

interaction_model <- glm(clean_result ~ Job_group + Number_of_positions
                + education_clean + driving_clean
                + hybrid_clean + Days_on + experience_clean + education_clean * hybrid_clean, data = project, family = binomial(link = "logit"))
summary(interaction_model)

Call:
glm(formula = clean_result ~ Job_group + Number_of_positions + 
    education_clean + driving_clean + hybrid_clean + Days_on + 
    experience_clean + education_clean * hybrid_clean, family = binomial(link = "logit"), 
    data = project)

Coefficients:
                              Estimate Std. Error z value Pr(>|z|)    
(Intercept)                   1.369843   0.191374   7.158 8.19e-13 ***
Job_group                     0.213826   0.098044   2.181 0.029189 *  
Number_of_positions           0.334661   0.127572   2.623 0.008708 ** 
education_clean              -1.693977   0.147708 -11.468  < 2e-16 ***
driving_clean                -0.769658   0.185197  -4.156 3.24e-05 ***
hybrid_clean                  0.177063   0.274987   0.644 0.519642    
Days_on                      -0.040820   0.008857  -4.609 4.05e-06 ***
experience_clean             -0.160406   0.047568  -3.372 0.000746 ***
education_clean:hybrid_clean  0.307193   0.307206   1.000 0.317332    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 2891.9  on 2106  degrees of freedom
Residual deviance: 2529.6  on 2098  degrees of freedom
AIC: 2547.6

Number of Fisher Scoring iterations: 4

P-value is greater than 0.05 so there is no evidence of interaction between these variables.

5. General and final model

stargazer(mylogit, mylogit1, interaction_model, myprobit, type = "text")

========================================================================
                                         Dependent variable:            
                             -------------------------------------------
                                            clean_result                
                                         logistic               probit  
                                (1)        (2)        (3)        (4)    
------------------------------------------------------------------------
City_class                     0.040                            0.023   
                              (0.038)                          (0.023)  
                                                                        
Job_group                     0.230**    0.209**    0.214**    0.138**  
                              (0.098)    (0.098)    (0.098)    (0.059)  
                                                                        
Number_of_positions           0.323**    0.334***   0.335***   0.197*** 
                              (0.128)    (0.128)    (0.128)    (0.076)  
                                                                        
Wage_clean                     0.138                            0.084   
                              (0.102)                          (0.062)  
                                                                        
time_clean                     1.040*                           0.618*  
                              (0.599)                          (0.363)  
                                                                        
education_clean              -1.658***  -1.654***  -1.694***  -1.007*** 
                              (0.146)    (0.142)    (0.148)    (0.087)  
                                                                        
driving_clean                -0.685***  -0.781***  -0.770***  -0.406*** 
                              (0.193)    (0.185)    (0.185)    (0.115)  
                                                                        
english_clean                  -0.006                           -0.002  
                              (0.137)                          (0.084)  
                                                                        
hybrid_clean                  0.371***   0.427***    0.177     0.225*** 
                              (0.130)    (0.122)    (0.275)    (0.079)  
                                                                        
Days_on                      -0.038***  -0.041***  -0.041***  -0.023*** 
                              (0.009)    (0.009)    (0.009)    (0.005)  
                                                                        
experience_clean             -0.164***  -0.154***  -0.160***  -0.100*** 
                              (0.047)    (0.047)    (0.048)    (0.029)  
                                                                        
education_clean:hybrid_clean                         0.307              
                                                    (0.307)             
                                                                        
Constant                       0.106     1.345***   1.370***    0.082   
                              (0.620)    (0.190)    (0.191)    (0.376)  
                                                                        
------------------------------------------------------------------------
Observations                   2,107      2,107      2,107      2,107   
Log Likelihood               -1,261.853 -1,265.263 -1,264.778 -1,261.887
Akaike Inf. Crit.            2,547.707  2,546.526  2,547.557  2,547.774 
========================================================================
Note:                                        *p<0.1; **p<0.05; ***p<0.01

Logit model with significant variables only has the lowest AIC.

Let’s explore our model a little further.

curve <- roc(project$clean_result, mylogit1$fitted.values)
Setting levels: control = 0, case = 1
Setting direction: controls < cases
plot(curve, main = "ROC curve", print.auc=T, grid=T)

Area under the curve is 0.725, which is quite high.

optimal_cutoff <- coords(curve, "best", ret = "threshold", 
                         best.method = "youden")
print(optimal_cutoff)
  threshold
1 0.6187321

6. Calculation and interpretation of marginal effects

logitmfx(clean_result~Job_group + Number_of_positions
                + education_clean + driving_clean
                + hybrid_clean + Days_on + experience_clean, 
                data=project)
Call:
logitmfx(formula = clean_result ~ Job_group + Number_of_positions + 
    education_clean + driving_clean + hybrid_clean + Days_on + 
    experience_clean, data = project)

Marginal Effects:
                         dF/dx  Std. Err.        z     P>|z|    
Job_group            0.0510905  0.0239620   2.1321 0.0329949 *  
Number_of_positions  0.0818408  0.0312414   2.6196 0.0088026 ** 
education_clean     -0.3755743  0.0280898 -13.3705 < 2.2e-16 ***
driving_clean       -0.1926632  0.0442601  -4.3530 1.343e-05 ***
hybrid_clean         0.1019234  0.0282931   3.6024 0.0003153 ***
Days_on             -0.0099551  0.0021701  -4.5875 4.486e-06 ***
experience_clean    -0.0378357  0.0115303  -3.2814 0.0010329 ** 
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

dF/dx is for discrete change for the following variables:

[1] "education_clean" "driving_clean"   "hybrid_clean"   
  1. Increasing the job grade by one increases the probability of hiring a candidate by 5.1 percentage points.

  2. Additional position for a job posting increases the probability of hiring a candidate by 8.1 percentage points.

  3. Requiring higher education decreases the probability of hiring a candidate by 37.6 percentage points.

  4. Requiring driving license decreases the probability of hiring a candidate by 19.3 percentage points.

  5. Possibility of remote work increases the probability of hiring a candidate by 10.2 percentage points.

  6. Additional day between opening job posting and the end of acquiring application documents decreases the probability of hiring a candidate by 1 percentage point.

  7. Additional year of experience required decreases the probability of hiring a candidate by 3.8 percentage points.

7. Calculation and interpretation of odds ratios

DescTools::OddsRatio(mylogit1)
Waiting for profiling to be done...

Call:
glm(formula = clean_result ~ Job_group + Number_of_positions + 
    education_clean + driving_clean + hybrid_clean + Days_on + 
    experience_clean, family = binomial(link = "logit"), data = project)

Odds Ratios:
                       or or.lci or.uci  Pr(>|z|)    
(Intercept)         3.838  2.643  5.563  1.27e-12 ***
Job_group           1.232  1.018  1.494    0.0330 *  
Number_of_positions 1.397  1.096  1.811    0.0088 ** 
education_clean     0.191  0.145  0.252 < 2.2e-16 ***
driving_clean       0.458  0.317  0.656  2.49e-05 ***
hybrid_clean        1.532  1.207  1.950  4.85e-04 ***
Days_on             0.960  0.943  0.977  4.36e-06 ***
experience_clean    0.857  0.781  0.939    0.0010 ** 
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 

Brier Score: 0.207     Nagelkerke R2: 0.211

Odds ratio for:

  1. job group is 1.23, meaning that for each increment of job grade, the odds of successful employment increases by a factor of 1.23, holding all other variables constant,

  2. number of positions is 1.4, meaning that for every additional job position, the odds of successful employment increases by a factor of 1.4, holding all other variables constant,

  3. education is 0.19, meaning that requirement of higher education decreases the odds of successful employment by a factor of 0.19, holding all other variables constant,

  4. driving is 0.46, meaning that requirement of driving license decreases the odds of successful employment by a factor of 0.46, holding all other variables constant,

  5. hybrid is 1.53, meaning that possibility of remote work increases the odds of successful employment by a factor of 0.19, holding all other variables constant,

  6. days on is 0.96, meaning that each additional day between the opening and closing job posting decreases the odds of successful employment by a factor of 0.96, holding all other variables constant,

  7. experience is 0.86, meaning that each year of required experience decreases the odds of successful employment by a factor of 0.86, holding all other variables constant.

9. R-squared statistics

predicted_probs <- predict(mylogit1, type = "response")

predicted_classes <- ifelse(predicted_probs >= 0.6187321, 1, 0)

count_r_squared <- 1 - sum(predicted_classes != project$clean_result) / length(project$clean_result)

classification_table <- table(Predicted = predicted_classes, Actual = project$clean_result)
most_frequent = (max(classification_table))

n <- nrow(project)
adj_count_r_squared <- (sum(predicted_classes == project$clean_result) - most_frequent) / (n - most_frequent)

# Print the results
print(paste("Count R-squared:", count_r_squared))
[1] "Count R-squared: 0.67062173706692"
print(paste("Adjusted Count R-squared:", adj_count_r_squared))
[1] "Adjusted Count R-squared: 0.488577745025792"
classification_table <- table(Predicted = predicted_classes, Actual = project$clean_result)
print(classification_table)
         Actual
Predicted   0   1
        0 750 514
        1 180 663

Count R-squared statistics equal to 0.66 means the model performs relatively well in terms of predicting the outcome.

PseudoR2(mylogit1, which="all")
       McFadden     McFaddenAdj        CoxSnell      Nagelkerke   AldrichNelson 
      0.1249607       0.1194281       0.1576091       0.2111216       0.1464016 
VeallZimmermann           Efron McKelveyZavoina            Tjur             AIC 
      0.2530679       0.1611981       0.2055019       0.1614977    2546.5261342 
            BIC          logLik         logLik0              G2 
   2591.7502975   -1265.2630671   -1445.9500725     361.3740108 

McKelvey-Zavoina statistic is equal to 0.21. It means that out logistic regression model explains a moderate proportion of the variability in the result variable.

10. Hypotheses verification

Primary

H0: Variables are jointly significant

H1: Variables are not jointly significant

null_logit = glm(clean_result~1, data=project, family=binomial(link="logit"))
lrtest(mylogit1, null_logit) # full_model, restricted_model
Likelihood ratio test

Model 1: clean_result ~ Job_group + Number_of_positions + education_clean + 
    driving_clean + hybrid_clean + Days_on + experience_clean
Model 2: clean_result ~ 1
  #Df  LogLik Df  Chisq Pr(>Chisq)    
1   8 -1265.3                         
2   1 -1446.0 -7 361.37  < 2.2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

The variables are jointly significant

Secondary:

H0: Higher education requirement has no affect on success of a job posting

H1: It has

no_edu_logit = glm(clean_result~Job_group + Number_of_positions
                + driving_clean
                + hybrid_clean + Days_on + experience_clean, 
                data=project, family=binomial(link="logit"))
lrtest(mylogit1, no_edu_logit) # full_model, restricted_model
Likelihood ratio test

Model 1: clean_result ~ Job_group + Number_of_positions + education_clean + 
    driving_clean + hybrid_clean + Days_on + experience_clean
Model 2: clean_result ~ Job_group + Number_of_positions + driving_clean + 
    hybrid_clean + Days_on + experience_clean
  #Df  LogLik Df  Chisq Pr(>Chisq)    
1   8 -1265.3                         
2   7 -1339.6 -1 148.67  < 2.2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Model with higher education performs significantly better.

11. Hosmer-Lemeshow test

predicted_probs <- predict(mylogit1, type = "response")
hoslem_test <- hoslem.test(project$clean_result, predicted_probs)
print(hoslem_test)

    Hosmer and Lemeshow goodness of fit (GOF) test

data:  project$clean_result, predicted_probs
X-squared = 13.902, df = 8, p-value = 0.08437

P-value is greater than 0.05 so we fail to reject the null hypothesis, meaning that the model fits the data well.

Summary

The model seems to perform quite well.