Ratings

Overall Score

The Final average final rating across all metrics was 4.38 / 5.

Final score by metric

Conclusion: Overall score not bad yay! Will continue to manage expectations carefully. Need to look into worst scores, distribution and time ->

All-time worst score by metric

The worst score given for any of the metrics on any part of the training

Score for all 5 metrics over time

Score Distribution

Conclusion: Both too easy and too hard; difficulty scores are also the most spread out. Will focus on managing varying skill levels: Will add more time for preliminary reading to get everyone to the same base; will add ‘bonus’ tasks; will push for multi-level training conducted in parallel (as originally planned). Overall intersting, so will keep the general topics covered.

Having ‘Too Hard’ outliers should have been anticipated. Need to identify them early on and provide extra support. ‘Mentoring’ plan to make sure everyone gets up to speed (Maybe assign mentors based on self-assessed skill levels).

Individual feedback and ideas Summary & Actions

Training structure

  • more structured

  • nice that feedback was implemented almost instantly Action: Will update the slides for more coherent structure.To keep the instant feedback implementation but have better structure, I will add “blank” timeslots throughout the training. That way we can stick to the structure and still add in the feedback. Any ‘digressing’ topics I will note down and cover in those timeslots rather than jumping around within a session.

  • wide range of prior knowledge means training is too difficult and too hard at the same time

Action: To accomodate for varying skill levels, will add ‘bonus’ tasks for more advanced users to all practical exercises. Will send out prerequisite reading earlier & push for reading time berforehand to make sure skill levels are harmonised

Exercises

  • good mix of theory and practice; should have been less separated
  • tiny bitesized practical example for each item
  • I liked the practical exercises particularly
  • more practical exercises
  • have a specific output as a desired result of a multiple-step process involving multiple tools and eventually leading to a brief analysis of a few hypotheses or indicators
  • have a few small tasks for people to try out with their own data
  • with a few lines of code everyone could do on their own)

Action: The next training will be organised around a real-world example project around which all exercises are built. Will try out having theory and training sessions in mini-intervals, so that each piece of theory can be tried out immediately (slowly building up the example project analysis)

Follow up from training

  • how to transfer the knowledge we aquire here into our missions?
  • At global workshop, much was discussed but not followed up on

Actions: As I am transferring the training also to Eliora, we will make all materials available, and put together a “trainer tool” on how to run the training. To the training plan, I will add a ‘follow up’ plan on how the shared ideas will be executed / implemented together with the participants. Will convert the skype group into an R-User group, to continue the discussion Have regular “micro events”; for example regular “show & tell” call of what people have been working on

More resources

  • Recommendations for books / tutorials
  • as an organization, we compiled useful training materials, tutorials, packages, etc.
  • upload the new version of your presentation

I’m making a single page linking all data unit resources. Adding the above to that list.

Requested Topics

  • Overview over existing tools will add

  • non-statistical sampling and how to correctly report findings this is a big topic, it probably needs to be a separate training track. It is not the priority because this is more manageable with ‘oldschool’ tools.

Brainstorming / Sharing

  • amazing opportunity to meet people from the other missions, learn from them, enrich own perspective
  • Would be interesting to have a bit of a group discussion on what people are doing in terms of development/automation/programming

Brainstorming / Sharing etc.: I will add this under the umbrella term ‘support community growth’ as a declared goal of the trainings and other data unit activities; exact changes not clear yet, but of course more time and more structure for brainstorming, discussion etc. during trainings.

Individual Ideas and Feedback

Listed below is the individual full text feedback sorted by agenda item it relates to. Please note that for the responses below, it was asked specifically for negative feedback, and to be as critical (but constructive) as possible.

Complete Individual Ideas and Feedback sorted by Agenda Item

Brainstorming/Hackathon

The idea is nice and there is nothing bad about exchanging ideas. However, the actual coding part of the hackathon will just put pressure on those who can code already and leave the beginners where they are.

Before doing something like this we should bring everybody to an acceptable level where they can effectively contributing to the designing and building of the tool.

Day 1

Overall fine

day 1 afternoon

All good but next time more exercises!

Today’s sessions were good, I am leaving with more knowledge than I came with

I feel like there is quite a big difference in the level of previous experience for this, which might be frustrating for some of the people in the training (i.e. those with a lot of experience). Considering the goal to have people with the skills to build tools in each mission, some people might need more training than others (in the future, after the RCM). However, as a complete novice, this applied approach is very useful for me.

A bit too slow

Discussion on analysis guidelines would be good

The first day was very informative regarding to we work with data/research in generally. Maybe (1) program outline on the srceen (what do we do now and what comes next) could help to stay on track for people with different speed of progress, as well as learning-friendly dataset

All the ideas presented were really good, hope we can move to actions not like what happened during Geneva workshop. (Brainstorming but then nothing happened :( )

day 1 morning

The first day overall was good but more examples/exercises with actual datasets would have been useful. I think it would also be good to get a better overview of what kind of tools currently exist, what they are doing and at what development stage they are.

day 2 afternoon

All good, shame we couldn’t play around with hypegrammaR! I enjoyed the brainstorming session.

day 2 morning

It is VERY intersting! It would be nice though to have a tiny bitesize practical example for each item, so that we can try things out as we learn them. Or maybe semihomework?

Very good! I am quite interested to see how Markdown can improve the way we structure our data analysis

Hey, do you suggest purchasing a book/guide of any sort like the “R for Data Science”, (or any other guide that helped you in getting where you are)?

day 3 morning

Being able to use hypegrammaR was great! I can’t wait to see it progress!

I found the hands-on exercises this morning very useful, and good to apply the basics I learnt during the last days. also, I think it’s a good foundation for using R in the future. However, I think we need to make sure that we’ll have enough time to also use the things we learnt, so the tools actually benefit our daily work. would be good if that’s taken into consideration of people’s TOR.

not session specific

Would be interesting to have a bit of a group discussion on what people are doing in terms of development/automation/programming

We should create cross-country project-specific teams (WASH / MSNA / Market Monitoring, Migration) and make them work together to find solution that can be generalizable.

I am wondering how we can transfer the knowledge we aquire here into our missions. I am sure people learn a lot and gain some insights on how to make their own work more efficient/analysis better/etc., but if it’s only one person per mission learning this, the gap between this group and the others will just increase. I think we should think of a way on how to bring other AOs/coordination-focused people to a certain level so that they can actually use the tools. At the moment, I can’t really see how we could do that (specifically in our mission, where the basic technical level in not bad, but there’s no real experts that could push to get everyone further). (Rating below is for the idea of incorporating a training for everyone in the mission)

Great training, good to see the tools developed at Impact. Very useful for people to share a common level. Very interesting brainstorming part.

Overall the training was super smooth and clear. It was not “technically” advanced (simply it could not be that way knowing that we are all coming from different background and not necessarily having the basic skills for that). Could you please upload the new version of your presentation. It is super helpful thanks :) :) :)

This is a very valuable training! Some things about it: -it was not too task- specific, so just allowed to conceptualize, explore guidelines, understand general principles of how to look for solution and how to debug the bug -it was an amazing opportunity to meet people from the other missions, learn from them, enrich own perspective -i think it is very hardcore for the trainer to be the only trainer during all three days, expecially taking into account that this is a pilot -it was nice that feedback was implemented almost instantly, so that finally we can do/try everything we wanted to -it was very inspirational! Bravo

I think overall this has been a good training. I liked the practical exercises particularly. Overall, it was a good introduction to existing R tools and how to generally load/run/troubleshoot R scripts etc. and should enable participants to use existing tools at least (which I guess was the intention of this, so, goal achieved). I think it could’ve been a bit more structured at times (the general structure was probably useful but we did jump a bit between different things on the first day). I also would have liked more practical exercises and personally felt it could have been a bit more challenging. Maybe it could have been good to have a specific output as a desired result of a multiple-step process involving multiple tools and eventually leading to a brief analysis of a few hypotheses or indicators. Maybe also could’ve been good to have a few small tasks for people to try out with their own data, e.g. aggregate data at a higher admin level, group data by population group or region, create a visualisation for one of these etc. (just with a few lines of code everyone could do on their own). I get that we had different skill levels and limited time so maybe not completely feasible but just wanted to bring it up as it could have helped people to have a few more experiences of success/progression and maybe a bit more insight into coding.

Overall

Overall the training was awesome! Thank you for putting this together. I feel much more comfortable now that I know where to find useful packages, read the documentation, and ask for help if needed. Getting to know GitHub was great and I will start implementing it in all my R projects now. Finally, for future reference, I’d like to see something regarding non-statistical sampling and how to correctly report findings.

Reading material

It would be beneficial if, as an organization, we compiled useful training materials, tutorials, packages, etc. (including everything from this training) that could be accessible for anyone looking to learn or improve their skills.

Annex

“Methodology”

Each day, this fully anonymous feedback form was filled out by the training participants ad-hoc between sessions, as well as after the workshop was finished. Feedback was then implemented in the subsequent sessions, which is why some of the feedback may be “contradictory” between sessions (i.e.: “X is missing from the training”, and later “it was good that we did X”.

At the end of the last day, the form was filled out by all participants. All ratings are based on 25 responses overall.

Raw Data

Zeitstempel I.have.. My.feedback.question.relates.to… My.feedback.question..s.. Rate..from.0…not.at.all..to.10…completely….This.part.of.the.training.was…..Too.easy. Rate..from.0…not.at.all..to.10…completely….This.part.of.the.training.was…..Too.hard. Rate..from.0…not.at.all..to.10…completely….This.part.of.the.training.was…..Too.detailed. Rate..from.0…not.at.all..to.10…completely….This.part.of.the.training.was…..Interesting. Rate..from.0…not.at.all..to.10…completely….This.part.of.the.training.was…..Boring. Rate..from.0…not.at.all..to.10…completely….This.part.of.the.training.was…..frustating.
11.02.2019 16:31:30 feedback not session specific Would be interesting to have a bit of a group discussion on what people are doing in terms of development/automation/programming 2 0 1 4 0 0
11.02.2019 17:24:08 feedback day 1 afternoon All good but next time more exercises! 4 0 1 5 1 1
11.02.2019 17:26:46 feedback day 1 afternoon Today’s sessions were good, I am leaving with more knowledge than I came with 1 1 1 4 1 2
11.02.2019 17:26:46 feedback day 1 afternoon I feel like there is quite a big difference in the level of previous experience for this, which might be frustrating for some of the people in the training (i.e. those with a lot of experience). Considering the goal to have people with the skills to build tools in each mission, some people might need more training than others (in the future, after the RCM). However, as a complete novice, this applied approach is very useful for me. 0 2 1 4 0 2
12.02.2019 08:33:33 feedback day 1 afternoon A bit too slow 4 0 2 5 0 0
12.02.2019 08:34:59 feedback Day 1 Overall fine 4 1 2 3 2 2
12.02.2019 08:35:09 feedback day 1 afternoon Discussion on analysis guidelines would be good 3 0 0 4 1 1
12.02.2019 08:35:18 feedback Reading material It would be beneficial if, as an organization, we compiled useful training materials, tutorials, packages, etc. (including everything from this training) that could be accessible for anyone looking to learn or improve their skills. 1 1 2 4 1 1
12.02.2019 08:36:24 feedback Brainstorming/Hackathon The idea is nice and there is nothing bad about exchanging ideas. However, the actual coding part of the hackathon will just put pressure on those who can code already and leave the beginners where they are. — — Before doing something like this we should bring everybody to an acceptable level where they can effectively contributing to the designing and building of the tool. 3 0 3 5 1 1
12.02.2019 08:36:57 feedback day 1 afternoon The first day was very informative regarding to we work with data/research in generally. Maybe (1) program outline on the srceen (what do we do now and what comes next) could help to stay on track for people with different speed of progress, as well as learning-friendly dataset 2 0 0 5 0 1
12.02.2019 08:37:24 feedback day 1 morning The first day overall was good but more examples/exercises with actual datasets would have been useful. I think it would also be good to get a better overview of what kind of tools currently exist, what they are doing and at what development stage they are. 3 0 2 3 1 2
12.02.2019 08:37:37 feedback day 1 afternoon All the ideas presented were really good, hope we can move to actions not like what happened during Geneva workshop. (Brainstorming but then nothing happened :( ) 4 1 3 4 0 0
12.02.2019 08:38:34 Idea not session specific We should create cross-country project-specific teams (WASH / MSNA / Market Monitoring, Migration) and make them work together to find solution that can be generalizable. 4 0 2 5 0 0
12.02.2019 08:39:15 a question not session specific I am wondering how we can transfer the knowledge we aquire here into our missions. I am sure people learn a lot and gain some insights on how to make their own work more efficient/analysis better/etc., but if it’s only one person per mission learning this, the gap between this group and the others will just increase. I think we should think of a way on how to bring other AOs/coordination-focused people to a certain level so that they can actually use the tools. At the moment, I can’t really see how we could do that (specifically in our mission, where the basic technical level in not bad, but there’s no real experts that could push to get everyone further). (Rating below is for the idea of incorporating a training for everyone in the mission) 1 4 3 4 0 3
12.02.2019 11:55:07 feedback day 2 morning It is VERY intersting! — It would be nice though to have a tiny bitesize practical example for each item, so that we can try things out as we learn them. Or maybe semihomework? 3 0 1 5 0 1
12.02.2019 13:04:45 feedback day 2 morning Very good! I am quite interested to see how Markdown can improve the way we structure our data analysis 1 3 2 5 0 0
12.02.2019 13:18:09 a question day 2 morning Hey, do you suggest purchasing a book/guide of any sort like the “R for Data Science”, (or any other guide that helped you in getting where you are)? 0 0 0 5 0 0
13.02.2019 08:05:02 feedback day 2 afternoon All good, shame we couldn’t play around with hypegrammaR! I enjoyed the brainstorming session. 0 0 0 5 0 0
13.02.2019 11:22:37 feedback day 3 morning Being able to use hypegrammaR was great! I can’t wait to see it progress! 0 0 0 5 0 1
13.02.2019 11:23:37 feedback not session specific Great training, good to see the tools developed at Impact. — Very useful for people to share a common level. — Very interesting brainstorming part. 4 1 1 5 0 0
13.02.2019 11:25:00 feedback day 3 morning I found the hands-on exercises this morning very useful, and good to apply the basics I learnt during the last days. also, I think it’s a good foundation for using R in the future. However, I think we need to make sure that we’ll have enough time to also use the things we learnt, so the tools actually benefit our daily work. would be good if that’s taken into consideration of people’s TOR. 0 0 0 4 0 0
13.02.2019 11:25:26 Overall Overall the training was awesome! Thank you for putting this together. I feel much more comfortable now that I know where to find useful packages, read the documentation, and ask for help if needed. — Getting to know GitHub was great and I will start implementing it in all my R projects now. — Finally, for future reference, I’d like to see something regarding non-statistical sampling and how to correctly report findings. 2 0 0 5 0 1
13.02.2019 11:29:33 feedback not session specific Overall the training was super smooth and clear. It was not “technically” advanced (simply it could not be that way knowing that we are all coming from different background and not necessarily having the basic skills for that). Could you please upload the new version of your presentation. It is super helpful thanks :) :) :) 3 1 3 4 0 0
13.02.2019 11:30:24 feedback not session specific This is a very valuable training! Some things about it: — -it was not too task- specific, so just allowed to conceptualize, explore guidelines, understand general principles of how to look for solution and how to debug the bug — -it was an amazing opportunity to meet people from the other missions, learn from them, enrich own perspective — -i think it is very hardcore for the trainer to be the only trainer during all three days, expecially taking into account that this is a pilot — -it was nice that feedback was implemented almost instantly, so that finally we can do/try everything we wanted to — -it was very inspirational! Bravo — 0 0 0 4 0 1
13.02.2019 11:34:17 feedback not session specific I think overall this has been a good training. I liked the practical exercises particularly. Overall, it was a good introduction to existing R tools and how to generally load/run/troubleshoot R scripts etc. and should enable participants to use existing tools at least (which I guess was the intention of this, so, goal achieved). I think it could’ve been a bit more structured at times (the general structure was probably useful but we did jump a bit between different things on the first day). I also would have liked more practical exercises and personally felt it could have been a bit more challenging. Maybe it could have been good to have a specific output as a desired result of a multiple-step process involving multiple tools and eventually leading to a brief analysis of a few hypotheses or indicators. Maybe also could’ve been good to have a few small tasks for people to try out with their own data, e.g. aggregate data at a higher admin level, group data by population group or region, create a visualisation for one of these etc. (just with a few lines of code everyone could do on their own). I get that we had different skill levels and limited time so maybe not completely feasible but just wanted to bring it up as it could have helped people to have a few more experiences of success/progression and maybe a bit more insight into coding. 3 0 0 4 2 0