class: center, middle, inverse, title-slide # Team Academy and Beyond ## Teaching Support System for Competences Development in an Era of Constant Disruption ### R Bonazzi*, A Perruchoud ### 13 Sept 2019 --- # This presentation in a nutshell  ###  ... Team Academy focuses on _relevance_ of acquired skills ###  ... Need for a theory to assess interdsciplinarity ###  ... Empirical assessment of the new artefact is undergoing --- class: inverted, center, middle #  Part 1: Team Academy ### (Concept - Principles - Tools) --- class: inverse, center, middle background-image: url(Entrenova_files/timiiakademia.jpg) --- # Old wine in a new bottle ? .pull-left[ ### Learning Tools 1. Learning contract 2. Portfolio 3. 24-hour birth-giving 4. Reflection paper & Motorola 5. Training notebook & sessions ] .pull-right[ ### Classic Theories -> Pedagogical contract(s); -> Mastery-based learning; -> Problem-based learning (PBL); -> Reflexive learning; -> Flipped classroom; ] --- class: center ### How to support educators collect and process data in each step of the experimental learning cycle for cognitive apprenticeship?  .pull-right[Source: [Kolb and Fry (1975)]] --- class: center ### How to support educators collect and process data in each step of the experimental learning cycle for cognitive apprenticeship?  .pull-right[Source: [David A. Kolb (2014)]] --- class: inverted, center, middle #  Part 2: Interdisciplinary theory for assessment ### (Experiental learning - Natural Language Processing - Decision Trees) --- ##  How to assess 21 skills in one hour? Source: Gijbels, Dochy, Van den Bossche, and Segers (2005) -- ### Best PBL assessment = linking concepts and practical procedures. -- ### Evaluators assess _arguments_ of students claiming to have _skills_. -- ### Each document is used as _evidence_ of an acquired skill. --- class: center # Theoretical framework from pedagogy  --- class: inverse, center, middle # Part 3: Empirical results ### Linking evidences and self-assessment to go beyond summative evaluations --- # Linking evidences and auto-evaluations ### Example of collected evidences done in one year by one student, who wrote in English ### R package quanteda (Benoit, Watanabe, Wang, Nulty, Obeng, Müller, and Matsuo, 2018) to analyze 2 texts from semester 01 and 2 texts from semester 02 ### Visualization of concepts in the text VS shift in the self-evaluation --- .pull-left[ <!-- --> ] -- .pull-right[ <!-- --> ] -- <div id="htmlwidget-c7610adc2f2ce1e35671" style="width:100%;height:auto;" class="datatables html-widget"></div> <script type="application/json" data-for="htmlwidget-c7610adc2f2ce1e35671">{"x":{"filter":"none","fillContainer":false,"data":[["1","2","3","4","5","6","7","8","9","10","11","12","13","14","15","16","17","18","19","20","21"],[37,35,2,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],["Marketing","Business Finance","Business strategy","Team leadership","Coaching and skills development","Courage and willingness to go beyond the borders","Courage to make choices and ambition","Creativity and innovation","Customer relationship, negotiation and sales","Information Processing and Research Methodologies","International opening","Interpersonal communication","Models of theories and conceptualization","Network creation","Personal methods and attitudes to learn to learn","Planning and organization","Project management","Self management","Self-initiative","Statistics, data management and business intelligence","Team learning"],[2,1,2,1,1,3,1,1,1,2,3,2,2,4,1,1,2,3,3,2,1],[2,2,3,3,2,4,3,2,3,3,4,4,2,5,2,3,4,4,4,2,2]],"container":"<table class=\"display\">\n <thead>\n <tr>\n <th> <\/th>\n <th>Evidences<\/th>\n <th>Skills<\/th>\n <th>Semester01<\/th>\n <th>Semester02<\/th>\n <\/tr>\n <\/thead>\n<\/table>","options":{"pageLength":2,"columnDefs":[{"className":"dt-right","targets":[1,3,4]},{"orderable":false,"targets":0}],"order":[],"autoWidth":false,"orderClasses":false,"lengthMenu":[2,10,25,50,100]}},"evals":[],"jsHooks":[]}</script> --- .pull-left[ <!-- --> ] .pull-right[ <!-- --> ] <div id="htmlwidget-7758aa482aafd9f2e9ef" style="width:100%;height:auto;" class="datatables html-widget"></div> <script type="application/json" data-for="htmlwidget-7758aa482aafd9f2e9ef">{"x":{"filter":"none","fillContainer":false,"data":[["1","2","3","4","5"],["Alpha","Marketing","Finance","Strategy","Leadership"],[0,3,0,0,2],[0,0,7,0,0],[0,0,0,2,0],[6,0,0,0,3]],"container":"<table class=\"display\">\n <thead>\n <tr>\n <th> <\/th>\n <th>document<\/th>\n <th>Marketing<\/th>\n <th>Business Finance<\/th>\n <th>Business Strategy<\/th>\n <th>Team Leadership<\/th>\n <\/tr>\n <\/thead>\n<\/table>","options":{"pageLength":1,"columnDefs":[{"className":"dt-right","targets":[2,3,4,5]},{"orderable":false,"targets":0}],"order":[],"autoWidth":false,"orderClasses":false,"lengthMenu":[1,10,25,50,100]}},"evals":[],"jsHooks":[]}</script> --- # Decision trees to go beyond mid-term exams .pull-left[] -- .pull-right[] Example of collected evidences from 100 students, who did 12 weekly peer-reviews, one mid-term exam and a final exam Source: Friedman, Hastie, and Tibshirani (2001) --- class: inverse, center background-image: url(Entrenova_files/conclusion.jpg) ??? --- # This presentation in a nutshell  ###  ... Team Academy focuses on _relevance_ of acquired skills ###  ... A rigorous evaluation at HES-SO by interdisciplinarity ###  ... Empirical assessment of the new artefact is undergoing --- class: inverse ### References ``` ## [Ben+18] K. Benoit, K. Watanabe, H. Wang, et al. "quanteda: An R ## package for the quantitative analysis of textual data." In: _J. ## Open Source Software_ 3.30 (2018), p. 774. ## ## [Dav14] David A. Kolb. _Experiential learning: Experience as the ## source of learning and development_. FT press, 2014. ## ## [FHT01] J. Friedman, T. Hastie, and R. Tibshirani. _The elements ## of statistical learning_. Vol. 1. 10. Springer series in ## statistics New York, 2001. ## ## [Gij+05] D. Gijbels, F. Dochy, P. Van den Bossche, et al. "Effects ## of problem-based learning: A meta-analysis from the angle of ## assessment". In: _Review of educational research_ 75.1 (2005), pp. ## 27-61. ## ## [KF75] D. Kolb and R. Fry. "Toward an Applied Theory of ## Experiential Learning". In: _Theories of group process_. Ed. by C. ## Cooper. New York: John Wiley & Sons, 1975. ``` --- class: center, middle  ### riccardo.bonazzi@hevs.ch -- _Slides created via the R package [**xaringan**](https://github.com/yihui/xaringan)_ --- class: center, middle # Timiiakatemia ### (Team learning + Teaching style inspired by Plato's Ἀκαδήμεια) --- class: center, middle background-image: url(Entrenova_files/Academy.jpg) ??? --- # Appendix: Learning Tools ### Once a semester - Learning journey - Portfolio of 21 skills - Learning contract (personal learning plan ”negotiated” with peers) -- ### Once a month - 24-hour birth giving (solving customer’s problem within 24 hours) - Pre- and postmotorola (reports before and after a project) - Reflection paper (self-reflection on issues learnt) - Literature (a guidebook with articles/books to be read) -- ### Once a week - Team training session (dialogue with a team, 4 hours) - Training notebook (learning diary) --- # Appendix: Extra tools <table> <thead> <tr> <th style="text-align:left;"> document </th> <th style="text-align:left;"> TTR </th> <th style="text-align:right;"> negative </th> <th style="text-align:right;"> positive </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> text1 </td> <td style="text-align:left;"> 0.564 </td> <td style="text-align:right;"> 25 </td> <td style="text-align:right;"> 96 </td> </tr> <tr> <td style="text-align:left;"> text2 </td> <td style="text-align:left;"> 0.561 </td> <td style="text-align:right;"> 35 </td> <td style="text-align:right;"> 64 </td> </tr> <tr> <td style="text-align:left;"> text3 </td> <td style="text-align:left;"> 0.570 </td> <td style="text-align:right;"> 22 </td> <td style="text-align:right;"> 75 </td> </tr> <tr> <td style="text-align:left;"> text4 </td> <td style="text-align:left;"> 0.575 </td> <td style="text-align:right;"> 25 </td> <td style="text-align:right;"> 43 </td> </tr> </tbody> </table> ### Monitoring the change in writing style. ### The shift in Type/Token Ratio (0.563 -> 0.573 ) indicates texts that are more dense. ### Assessing the critical stance of the student by comparing the ratio of postive/negative words used in semester 1 (2.67) and semester 2 (2.51)