Evaluation /n see-it be-it
2023-02-20
1. Data cleaning decision
- “fego” seem different person since the different age (11 vs 14)
- “elha” seem duplicated in the pre-intervention survey
- total 103 pairs considering the name, school, age and gender
- causation might not be achieved because of absence of comparison group
2. Summary
2.1 overall missing rate change across all questions
Most of questions tend to decrease the missing rate between the pre-intervention and post-intervention
Missing rate are remarkably dropped in Q11.a, Q15.a, Q15 -> Q11.a. If yes, please describe what it is. (Q11. Do you know what a District Heating Network is?) -> 15.a. If yes, what can you remember about them? (Q15. Have you had lessons or talks about jobs and careers before?)
Missing rate are increased in Q6 -> Q6. Are there any jobs that you would like to know more about?
2.2 Jobs they`ve heard vs jobs they think to do
- Q10 and Q12 tend to show the different trend. Awareness on jobs on the list has been increased over two period, whereas intention of doing jobs on the list experienced the decrease. Students may want to know other jobs which are not on the list
2.3 Awareness improvement
- Awareness on the District Heating Network show the biggest difference between the pre and post survey (Q11)
- Exposure to see-it, be-it may contribute to the knowledge about career talks
2.4 Attitude change
- Both questions experience the positive change over two periods. Students who answered “the most important” increased by 5.8%
2.5 See-it, be-it related output
- Students answering both questions to “Yes” are 3.
3. Each output level effects
3.1 Students have an increased awareness of modern jobs (AWARENESS)
3.1.1 jobs that they can tell
- No significant change of the number of jobs they can write (p-value : 0.137) between the pre and post survey
3.1.2 Jobs they can tell
3.1.3 New jobs
- There are some changes between what they know and what did not exist in last ten years
4. data analysis
Q3. How many jobs can you name (up to 10)? Please list them below.
Q3_1. Missing rate
- no change on the missing rate
Q3_2. change of the number of jobs they can write over time
- no change on the number of jobs that they write
Q3_3. word frequencies
- Major changes do not appear in the most frequent words
Q4. Can you name 10 jobs that didn’t exist 10 years ago? List as many as you can (up to 10) below.
Q4_1. Missing rate (i don`t know, not sure etc)
- No change over reporting periods (0.47 in the pre vs 0.45 in the post)
Q4_2. Overall change in completion rate (anwering something) NA group (who respose “I don`t know in the pretest) vs non-NA group
- Those who did not response in the pre-intervention tend to answer something in the post intervention (48 -> 26)
Q4_3. What kind of difference can be captured between pre and post?
token approach (blank separation)
Q5. What jobs do you think you would be good at?
Q5_1. Missing rate
- No change of missing rate over time
Q5_2. what kind of difference between pre and post?
- There do not appear the significant change over time
Q5.a. Why do you think that?
wordcloud
- some personal preference word (like, love, interest)
- other ability associated word (good)
Q6. Are there any jobs that you would like to know more about?
Q6_1. Overall response rate
Q6.a. If yes, what jobs would you like to know more about?
Q6.a_1. Overall response rate
Q6.a_2.jobs they would like to know more
Q7. What is a ‘green job’?
Q7_1. Missing rate
- Missing rate tend to decrease in the post intervention
Q7_2.overall change in completion rate (anwering something) NA group (who respose “I don`t know in the pretest) vs non-NA group
- Missing rate tend to be decreased in the NA group
Q8. What skills do you think you need for a ‘green job’?
Q8_1. Missing rate
- Missing rate tend to increase in the post-intervention
Q8_2. Change on missing rate
Q8_3. What kind of difference can be captured between pre and post?
token approach (blank separation)
Q9. How would you get a ‘green job’?
Q9_1. Missing rate
- Missing rate in the post-intervention seem decreased
Q9_2. Change on missing rate
Q9_3. What kind of difference can be captured between pre and post?
token approach (blank separation)
Q10. Please tick which jobs you have heard of before: (list of green jobs)
Q10_1. Missing rate
- Missing rate in the post-intervention seem decreased
Q10_2.Change on missing rate
Q10_3.the number of jobs heard
- The overall jobs heard are increased in the post-intervention (4.8 -> 5.3)
Q10_4. Change between pre-intervention and post-intervention in each jobs
- “Heat Network Specialist” and “Energy Engineer” seem significantly increased in the post-intervention
Q11. Do you know what a District Heating Network is?
Q11_1. Awareness improvement
- Those who knows “District Heating Network” are increased in the post-intervention
11.a. If yes, please describe what it is.
Q11.a_1. Missing rate
Q11.a_2. key word (wordcloud)
Q12. Please tick any jobs you would consider doing from the following list (if any):
Q12_1. Missing rate
Q12_2. Change on the missing rate
Q12_3. the number of jobs doing
- The number of jobs they would like to do within the list is decreased in the post-intervention
Q12_4. Change between pre-intervention and post-intervention in each jobs
individual gain
Q12.a. Why would you like or not like to do these jobs? (data cleaning is tough)
Q12.a_1. Missing rate
- “I don’t know what they are” - NA
- “I am not interested” , “don`t appeal to me” coded as “No”
Q13. Describe what type of work you think you would like to be doing when you are 30.
Q13_1. Missing rate
- Missing rate is decreased in the post-intervention
Q14. Do you think that you need to go to university to do that kind of work?
Q14_1 Overall response rate
- Those who think university is needed to do the job they want is decreased in the post-intervention
Q15. Have you had lessons or talks about jobs and careers before?
Q15_1 Overall response rate
Q15.a. If yes, what can you remember about them?
Q15.a_1. Overall response rate
Q16. How important to you is fighting climate change?
Q16_1. Overall trend
- Those who response climate change is very important to fight are increased in the post-intervention
Q16_2. Quantitative approach
Q17. Do you know what the ‘climate crisis’ is?
Overall trend
## # A tibble: 2 × 3
## climate_crisis time n
## <chr> <fct> <int>
## 1 Yes pre 72
## 2 Yes post 73- there is no difference between the pre and post
Q18. What did you find interesting about the See It Be It films?
Q18_1. Wordcloud
Q19. Did the films make you think any differently about jobs you could do?
Q19_1. Overall response
Q19.a. If yes, in what ways?
Q19.a_1. All responses
## [1] "Don't Know"
## [2] "Found about jobs I would be good at."
## [3] "I felt more confident on findjnga good job"
## [4] "I thought that jobs all required the same studies but I was wrong"
## [5] "In what you need to study"
## [6] "It gives me a winder range of understanding about jobs"
## [7] "It made me think of how many jobs there are in the world."
## [8] "It opened my eyes to different opportunities"
## [9] "It was a different light too jobs"
## [10] "Made them seem more interesting."
## [11] "More of a variety"
## [12] "Showed me some other job opportunities."
## [13] "That thee are jobs which we can do to save the environment."
## [14] "it just influenced me a bit"
## [15] "like engineering"
## [16] "there is a lot more jobs out there than i thought"
## [17] "theres is more options than you think"Q20. Were the films different to other information you have had about jobs and careers?
Q20_1. change over time
- “yes” response is the lowest frequencies
Q20.a. If yes, in what ways were they different?
Q20.a_1. All responses
## [1] NA
## [2] "don't know"
## [3] "i understand more about people and their jobs"
## [4] "it was a video rather than someone coming in to talk"
## [5] "it was just green jobs"
## [6] "more in detail on the job"
## [7] "more specific"
## [8] "never heard of them"
## [9] "talked a lot about heat district networks."
## [10] "the whole way they do stuff to prevent damaging the enviroment."
## [11] "they brought new ideas to me"
## [12] "they explained in detail what there jobs do"
## [13] "they focused on one person in specific"
## [14] "they gave a subjective view of someone's work"
## [15] "they involved the environment"
## [16] "they looked like they were more sustainable and weren't ignoring the fact that what they do could end up damaging the earth and contributing to climate change and global warming."
## [17] "they require more skill."
## [18] "they showed lots of info on green and new types of jobs"
## [19] "they told me things i didnt get told in careers"
## [20] "they told more information"
## [21] "they were jobs that i had never heard of before"Q21. How confident do you feel about your job opportunities in the future?
Q21_1. overall trend
- Positive response including “very confident” and “confident” are the greatest shares