Results and Discussion

## Warning: package 'dplyr' was built under R version 3.4.2
## Warning: package 'tidyr' was built under R version 3.4.2
## Warning: package 'Rcpp' was built under R version 3.4.2

When we looked at the metric of utility by race we saw a general trend that for each demographic, except for “Black,” the pluarlity was for a usefulness of experience score of 5 (Figure 2). The margin of difference between the number of people who scored the usefulness as a 5 versus 4 was significantly larger in participants who identified as “White” than any other ethnicity. This probably a product of the fact that vast majority of our participants identified as “White” (Figure 1). Taking a closer look at those who did not identify as “White,” we see that those who identified as “Hispanic” were the next highest in terms of percentage that scored the usefulness of experience as 5 (Figure 3). Of the 3 “Black” participants (Figure 4), there was 1 score of 3 and 2 scores of 4; it is the only demographic that did not have a single score of 5.

Figure 1: This figure shows the ethnic distribution of participants.

Figure 2: This figure shows the usefulness of experience score by ethnic groups.

Figure 3: This figure shows the usefulness of experience score by all the ethnic groups other than “White.”

ethnicity_race n
Asian / Pacific Islander 21
Black 3
Hispanic 23
Multi-racial 10

Figure 4: This table shows the total number of participants for each ethnic group for a better and more accurate understanding of Figure 3.

Curious about the trend seen in the previous figures about usefulness of experience scores, we were curious to see what correlation that had to education (Figure 5). Overall, a majority of participants had either a highest educational level of university or post graduate (Figure 5 and 15). It is interesting to note that for our participants that identified as “Black,” they all had post graduate education. For “Asian/Pacific Islander” and “Hispanic,” there was more of an even spread among high school, university, and post graduate.

Figure 5: This figure shows highest level of education of participants by ethnic group.

When we looked at the monetary value evaluations for the product, we saw that every ethnic group was represented except for those that identified as “Black” (Figure 6). The general trend for all ethnicities were the scores: average, above average, and excellent. For “White” participants, the majority was in these three aforementioned evaluations. This demographic was also the only one to have evaluations of poor. “Multi-racial” were the only other participants to those that identified as “White” to evaluate the monetary value of the product as below average. “Asian/Pacific Islander” participants scored as either excellent or above average. “Hispanic” participants evaluated it as either excellent or average.

Figure 6: This figure shows the money value evaluation of the product by ethnic group.

In order to take a more in-depth investigation into these trends, we looked at the free answer questions by ethnic group (Figures 7-10). For those that identified as “Black,” the only response written was to answer the question asking to describe their experience where they expressed enjoyment of the course but difficulty in applying it to every day.

Only Participants That Indicated Race of “Black”

gender age education ethnicity_race usefulness_of_experience money_value dislike change_for_improvement like_most describe_experience
male 19 - 29 post graduate Black 4 N/A N/A N/A N/A No answer
male 30 - 49 post graduate Black 3 N/A N/A N/A N/A No answer
male 50 - 64 post graduate Black 4 N/A N/A N/A N/A It is an excellent course and I thoroughly enjoyed it, unfortunately don’t often see micro expressions.

Figure 7: This table shows the answers for all the participants that identified as “Black.”

It is important to note that only 3 of the participants (total being 10) that identified as “Multi-racial” answered the questions regarding what they disliked about their experience, what they would change for an improvement, and what they liked most; conversely, all but 3 replied to the question asking to describe their overall experience. For the question asking what they disliked about their experience, “Multi-racial” partipants expressed that there wasn’t enough variety, reasoning behind the answers, and that it felt too brief. For the question asking what could be improved, they suggested more variety in scenarios as well as in the ways to test the knowledge, a more user-friendly interface, and more information behind the answers provided. For the question about what they enjoyed the most, they expressed overall enjoyment of the products we have as a company. For the question asking to describe their overall experience, all the answers were very positive and reflecting a learned skill that they enjoyed and were able to apply in everyday life.

Only Participants That Indicated Ethnicity of “Multi-racial”

gender age education ethnicity_race usefulness_of_experience money_value dislike change_for_improvement like_most describe_experience
female 19 - 29 university Multi-racial 3 Average N/A N/A N/A
male 50 - 64 post graduate Multi-racial 4 Above average Not enough variety of scenarios in Law enforcement, workplace and family modules Increase number of scenarios METT, SETT Good feedback on each expression
female 30 - 49 university Multi-racial 3 Below average It didn’t seem to work properly, I didn’t feel like I learned anything. I felt there wasn’t enough theory and information about the reasoning behind answers. more intuitive technology. More reasoning and theory needed. I really didn’t like this one. Though I have done other training from you previously and have learned a lot.
male 30 - 49 post graduate Multi-racial 3 Average Short for the money Video instead of pictures flashing. Maybe some diagrams pointing out which features are exactly being described when showcasing certain microexpressions. More differentiation of some of the similar ones (fear/surprise, for example) Good variety of faces
male 19 - 29 post graduate Multi-racial 5 N/A N/A N/A N/A It’s an effective way to get used to micro-expressions and to spot them effectively
female 19 - 29 university Multi-racial 5 N/A N/A N/A N/A Very interesting and easy to use.
male 19 - 29 post graduate Multi-racial 4 N/A N/A N/A N/A No answer
male 30 - 49 university Multi-racial 4 N/A N/A N/A N/A It is very interesting to understand what people really mean when they are trying to act differently. Now I have a better understanding but need a lot more practice.
male 30 - 49 post graduate Multi-racial 5 N/A N/A N/A N/A Been using for about 4 years.
male 30 - 49 post graduate Multi-racial 5 N/A N/A N/A N/A I always had this kind of natural observation to read people’s faces but without totally understand the meaning of them. In most cases, my mind would have noticed something was happening and tried to ge information from other channels. This has provide me with and profound background to finally associate expressions with specific emotions, leading me to new ways of interaction, despite confusion in other people when they are read.

Figure 8: This table shows the answers for all the participants that identified as “Multi-racial.”

Out of those that identified as “Hispanic,” only 2 of them answered the questions regarding what they disliked about their experience, what they would change for an improvement, and what they liked most. 6 of the 23 did not respond to the question asking to describe their overall experience. What they disliked was the length and digital platform. What both of them said in terms of changes for improvement would be a more mobile friendly platform. The things they liked most were being able to apply it to every day life and the content of the trainings. Their descriptions of the overall experience were very positive quoting interesting tools, usefulness and practicality, using it for teaching a college course, and being more in tune with others because of deeper understanding.

Only Participants That Indicated Ethnicity of “Hispanic”

gender age education ethnicity_race usefulness_of_experience money_value dislike change_for_improvement like_most describe_experience
male 19 - 29 high school Hispanic 5 Excellent The length, but honestly that can’t be helped. Making it more mobile friendly I would say. Sometimes I didn’t have the time to sit at my home computer, and while I could use it on my phone, it wasn’t very easy to do so. How I could almost immediately apply it in real life, and after going through the first training being able to recognise expressions from practice.
male 30 - 49 post graduate Hispanic 3 Average The digital plataform The same. The a better digital and mobile plataform The content and the trainings Interesting
male 30 - 49 post graduate Hispanic 5 N/A N/A N/A N/A Your is most definitely epic!!
male 19 - 29 high school Hispanic 3 N/A N/A N/A N/A The training tool was okay. I believe a more natural setting will do better, as well as, more faces and a neutral face
male 30 - 49 post graduate Hispanic 5 N/A N/A N/A N/A Fantastic tools
male 19 - 29 other Hispanic 5 N/A N/A N/A N/A They were easily understood and actually fun to do. This has helped greatly in my line of work.
male 50 - 64 post graduate Hispanic 4 N/A N/A N/A N/A Practical and interesting
male 0 -18 high school Hispanic 4 N/A N/A N/A N/A No answer
male 30 - 49 university Hispanic 5 N/A N/A N/A N/A No answer
female 30 - 49 post graduate Hispanic 5 N/A N/A N/A N/A Fue muy importante poder descubrir los movimientos musculares, estudiarlos una y otra vez para entender las expresiones y microecpresiones humanas. Considero que el Dr Ekman ha contribuido de una manera increíble no sólo en el tema de las expresiones faciales, sino en el conocimiento acabado y entendimiento de las emociones humanas. Mi más profundo agradecimiento para él y el grupo Ekman. I’m sorry i wrote in spanish my experience since my English si not that good.
male 30 - 49 post graduate Hispanic 5 N/A N/A N/A N/A I used it to teach in a college course. I hve trained first and then use the tools for teaching.
female 30 - 49 university Hispanic 4 N/A N/A N/A N/A i think the tool seems to be very easy but the fact it is required to invest to much time.
male 19 - 29 university Hispanic 4 N/A N/A N/A N/A Great
male 30 - 49 university Hispanic 5 N/A N/A N/A N/A No answer
male 19 - 29 university Hispanic 5 N/A N/A N/A N/A It was very helpful.
female 19 - 29 high school Hispanic 5 N/A N/A N/A N/A it took a while to understand it and see the expressions, but the voice in the background helped explained what to look for. I like how simple and easy the program is to use.
female 30 - 49 post graduate Hispanic 5 N/A N/A N/A N/A No answer
male 19 - 29 high school Hispanic 2 N/A N/A N/A N/A No answer
male 30 - 49 university Hispanic 5 N/A N/A N/A N/A It was a great experience, I could get in tune with my emotion and those of others. I’d humbly would like to recommend more training for de subtle expression that are the one we encounter the most in real life! A more thorough training with more pictures of subtle expression. And why not? Maybe a video training would be really nice. Doctor Ekman and Ekman group thank you for your amazing work!
female 50 - 64 post graduate Hispanic 4 N/A N/A N/A N/A The training itself was very helpful. My greatest concern is how difficult it is to contact you, Paul Ekman Group. My email went unanswered and there is no phone number to call.
female 50 - 64 post graduate Hispanic 5 N/A N/A N/A N/A Very helpful in learning to read micro expressions
male 30 - 49 post graduate Hispanic 5 N/A N/A N/A N/A Awesome tools, I was able to read people more efficiently
male 30 - 49 post graduate Hispanic 5 N/A N/A N/A N/A Excelente herramienta, para él área de recursos humanos y la familia ñ.

Figure 9: This table shows the answers for all the participants that identified as “Hispanic.”

Only Participants That Indicated Ethnicity of “White”

gender age education ethnicity_race usefulness_of_experience money_value dislike change_for_improvement like_most describe_experience
male 30 - 49 university White 4 Average Sometimes it is hard to find time for it. Maybe an app for the smartphone would be cool (to be able to use it everywhere). Smaller sessions. In general it is a cool thing understanding the non-verbal language of people.
male 30 - 49 university White 4 Average I still love the tool but would like to have the 99 dollar one time purchasing, not annual? 1. A life-time access for that money, until you do not change dramatically. 2. On the test please highlight the sections choosen (with another color) The general usability of spotting expressions can be helpful and once skilled also a nightmare in once. Nevertheless, it can be helpful, but it needs a very long training periode… (and to get it in flesh and bone more than a year) It would be helpful to have a version that runs smooth on both desktop and as web app on the smartphone. A version in German and Portuguese would be nice to see in the near future and for that reason face examples that meet the genders and races we face in that regions and/ or countries. To be honest, as we have less black/ asian people in Europe it would be more helpful to have faces that are more european - as those are the once we face in real life. Thanks
male 30 - 49 university White 3 Below average This kind of one year pay-serve… As explained, offline version, one time payment, more examples and a somehow a shift into real life situations, face plus postures plus situation….? It makes life in general more transparent, unfortunately you will have to live from that moments onwards with this new skill… sometimes you will wish to be naive again…
male 30 - 49 post graduate White 5 Excellent nothing the system does not run well over a wireless link to monitor/screen feedback The tools help deepen the experience and understanding of the FACS.
female 30 - 49 university White 5 Above average That it is only “flashes” and not talking sequences That it would be real conversations (like in the tests connected to the education) - maybe it could be a thing for people who have passed the causes? The ease of use, the “use-as-much-as-you-can” increases learning of the micro expressions, increases the concentration level
male 50 - 64 university White 5 Above average With most on-line training you get a lifetime access, which is here not the case see 8 The total set up and feedback Gives me a lot of insights about people behaviour very practical.
female 30 - 49 university White 4 Above average Nothing I little more detail in the explanations with other expressions that might be similar That it was all online I found it useful for the purposes of interviewing canidates for my company and working in HR
male 30 - 49 post graduate White 5 Above average The speed of some examples was pretty high. I think an even slower option would help. More faces as well would help Live courses in locations where I can attend. Or at least live courses I can take over high speed internet Very applicable in real life Loved them
male 50 - 64 university White 3 Above average sometimes difficult to notice subtle facial movements movements being in contact with something very different from my daily tasks
male 50 - 64 post graduate White 4 Average I think I tried to use it on my android tablet and it didn’t work out too well I don’t like the year expiration. The incremental cost to letting me continue to use it is negligible. You’ve gotten paid. I doubt few would pay twice for the same course. It just keeps me from visiting the site, where you might make another sale. Some diagrams isolating the detailed features of expressions Maybe what I’d like is some FACS training How about some detection exercises in flat affect videos? We’re being taught to always alarm, while in watching people in video, there often doesn’t seem that there is that much emotion being displayed. Also, simply some training on how to focus on a face would be good. For most of the course I focused on eyes, but naturally found myself missing a lot a the mouth. I tried to widen my focus to catch more of the mouth, and it helped, but it never felt that natural. The tests were fun The ability to flash the expression on and off was a function of the software Good, but I don’t feel the training translates to everyday use that well, as expressions are micro micro, and fewer and farther between
male 30 - 49 post graduate White 5 Excellent Nothing None. The ability to review and go over certain areas. It was great. Highly recommended. Thank you.
male 19 - 29 university White 5 Average They were very expensive. I understand the value lies in the research, not the programs themselves, but I would have liked some light weight tools (i.e. and app) that’s cheaper and could allow me to train more consistently and for less. See previous question. Very useful tools. I loved the program. Excellent. Worth the money, even though the tools are almost prohibitively expensive.
male 50 - 64 high school White 4 Above average NA It is well designed. I can do it at home. I found myself jumping to conclusions on some things, judgement will come with time.
male 30 - 49 post graduate White 4 Average Sometimes complicated N/A N/A Very good tool
male 30 - 49 post graduate White 4 Below average Nothing of note. It would be nice to slow down the expressions sometimes Fun to use No answer
male 30 - 49 university White 4 Below average N/A Using open standards and web standards giving people the freedom to use their browser on Mac, GNU/Linux or Microsoft Windows operating system. N/A N/A
female 50 - 64 university White 4 Above average The availability of only access it for a year. Longer time to train, as I have a busy schedule. I like that you can start off learning the micro expressions. Love them.
male 30 - 49 university White 4 Above average On rare occasions I would experience issues during the testing where the expression would be stuck instead of flashing at slow, normal or fast. Sometimes it would also not flash at all. This occurred on different computers and browsers. However, my main dislike is the training access only lasting a year. A phone app to make the training more mobile and easier to access would be great. I genuinely improved my ability to recognize different micro expressions. I am in law enforcement and with this training have been able to assist in investigations by watching subjects being interviewed. I am now observing expressions quicker and more accurately, several of which I would have missed prior to this training.
male 50 - 64 university White 4 Above average N/A N/A N/A N/A
male 19 - 29 university White 4 Average The pricing was a bit steep. Would be a okay price if it was lifetime. The option for a mobile app, for on the go training. To really feel that you starting to understand what you are learning. Think it was great was easy to understand descriptions and the video material made it easy to learn the basica.
female 50 - 64 university White 4 Below average on-line–too much money for the limited product delivered price that fits training . No answer
male 65-100 post graduate White 4 Above average Black man and women only expression is contempt. Black faces other emotions too. Good verbal and visual explanation. Used them for 15 years in teaching support of my Master Degree courses (Stanford MBA and Austria Masters).
female 30 - 49 university White 4 Average That the system lagged at times. More randomness and a mixture of the different trainings (face, side etc) That I feel the feelings I am trying to detect. I use Dr. Ekman’s training tools to prepare myself (the night before) for when I host different leadership programs and meet managers in bigger groups. Thanks to the refreshening of attunement to different expressions I can better read the group and pinpoint the person who has questions (reactions to what I or their peers are saying) that they need answering before moving on to other topics.
female 50 - 64 other White 4 Above average N/A We need more different programs Pic - repeat - solution function very good, but we i would need more different programs
male 30 - 49 post graduate White 3 Average N/A Personal / unlimited time license. Having to pay every year kind sucks. :( N/A N/A
female 50 - 64 trade / technical / vocational training White 4 Average It was a bit tricky being ready at the right moment for the emotion to flash- but that’s all part of the process I guess For me personally I think some reminders during the course of the year would have been good, ie to remind me I have x amout of time left with access to the materials Beginning to recognise things in life that I had learned on the course It was a good experience! Solidly useful, it will help me in my life and I’m glad I took the course
male 50 - 64 university White 5 Above average Small images, could be larger. See 8 :) Speed with which you can practise. Really enjoyed it! It has allowed me to really integrate the recognition of FA in my daily life, both personal and professional. Thank you!
male 30 - 49 post graduate White 4 Average n/a n/a Practical side No answer
male 19 - 29 post graduate White 4 Above average limited amount of photos add more photos of different people, otherwise after a few times one can memories the correct answers clear and concise Helps to understand people around
male 30 - 49 university White 4 Average Would like greater variety of profiles More variety The feedback Positive
female 30 - 49 post graduate White 5 Above average faces that are used are very much American/Spanish. Wider array of faces - so the level of difficult on recognizing ME first time would be higher. Also when training, there is only limited nr of shows, why not to show many more and than the program could choose the ones for re traiing to make it more diffcult. We not to connet it with FACS training - and make FACS training online? That you can go back to it and re train. very good.
male 19 - 29 university White 5 Above average not much really, maybe a bigger variety of examples. Even if you only train once a month, you start to memorize the faces. A bigger variety of example images. See above. The detailed explanations on how to regognize the micro expressions. I really liked the training I was getting with Dr. Ekman’s training tools. As a FACS-Coder, I can say, that even if you have no knowledge about facial muscles and facial expressions, these tools will teach you perfectly not only how to spot microexpressions, but also the know-how of recognising facial expressions in general. Now I am able to recognise microexpressions perfectly in realtime and I can combine the knowledge of facial coding and spotting microexpressions. Let’s just say, that this knowledge was adventageous for me more than one time :)
male 50 - 64 post graduate White 1 Poor Customer care Be able to download what I paid for The concept No answer
male 19 - 29 trade / technical / vocational training White 5 Above average I wish there was a wider variety of scenarios to watch. I also wish there was a mobile app that I could use on my iPad or iPhone when I’m not near a computer. Can’t think of anything. The variety of different faces to examine makes the program very diverse, applying to different people from all backgrounds. It’s a fascinating and innovative tool. The tools help a lot when interacting with people, whether at work or in my personal life.
male 30 - 49 post graduate White 4 Excellent On certain web-browsers the microexpression videos are very slow, others very fast, making the training either very easy or much more difficult to understand. Fixing microexpression video so the speed of the video is either the same across browsers, or modifiable to meet training needs. Flexibility to train whenever convenient. Thorough explanations in RETT modules. Overall very well done Overall very well done. My only complaint is the microexpression videos can be very easy or very difficult depending upon the web browser.
male 30 - 49 trade / technical / vocational training White 5 Average N/A More different pictures The experience to become faster and faster High quality and simplicity of use at a glance.
female 30 - 49 university White 1 Poor unable to say since I have not done it and did not realize my access would expire. unable to say since I have not done it and did not realize my access would expire. unable to say since I have not done it and did not realize my access would expire. unable to say since I have not done it and did not realize my access would expire.
male 65-100 post graduate White 3 Below average The web page is not easy enough to use. Too much clicks to do a test. Solving what I dislike. The test itself and how it is designed. Dr Ekman’s tools and work is excellent. The web page should improve to match that level.
female 50 - 64 post graduate White 4 Average that I would forget to log on and train! more variety of pictures easy to do at home helpful and applicable in many situations
male 30 - 49 university White 5 Above average N/A N/A Its almost like a quizz game, I like to play while learning new things. Easy to use
female 50 - 64 post graduate White 1 Excellent I didn’t always agree with the label for the facial expression but I learned them by rote We don’t guess emotions in a vacuum. I need to hear what people are saying as well as look at their face. Nothing. Frustrating.
male 30 - 49 post graduate White 3 Above average nothing More photos to practice Lo que mas me gusto es ver los resultados del entrenamiento de poder reconocer micro gestos El programa on-line es muy interesante para aplicar al trabajo de empatía que implica mi trabajo de clínica psicológica, pero me gustaría seguir avanzando
male 50 - 64 post graduate White 5 Excellent Nothing I cannot think of anything Ease of use These tools have been very effective as I conduct interviews.
male 50 - 64 post graduate White 5 N/A N/A N/A N/A No answer
male 30 - 49 university White 5 N/A N/A N/A N/A I love this training. I like to go back and test myself often. It has helped while talking to people, and noticing the little things many people may miss.
female 19 - 29 university White 5 N/A N/A N/A N/A No answer
male 30 - 49 post graduate White 3 N/A N/A N/A N/A It was great training, but I was unaware that it expired. Overall, the content was great, but sometimes the videos and clicking through did not work well.
male 30 - 49 university White 5 N/A N/A N/A N/A Dr Ekman’s training tools are extremely helpful for honing one’s skills of reading micro expressions. Once your sensory acuity improves it becomes so obvious while people speak to us people’s faces are talking to us all the time, even when they aren’t speaking.
male 30 - 49 university White 5 N/A N/A N/A N/A Good. I have not used them for a while but have found them very helpful as I sell and recruit for a living. They’re east to use and help improve my awareness.
female 30 - 49 other White 5 N/A N/A N/A N/A No answer
female 50 - 64 post graduate White 5 N/A N/A N/A N/A It was very helpful in explaining my instincts.
male 50 - 64 high school White 5 N/A N/A N/A N/A The material is awesome. I just need more time to complete it as I got sidetracked looking for job as my hours had been reduced where I was on. It has been on my mind so will do my best to look into it all over next month. Would love an extension of 90 days.my email is bicknellbob@gmail.com Thanks! Would love to facilitate others for you if there is an opportunity for that.
male 19 - 29 high school White 4 N/A N/A N/A N/A No answer
female 30 - 49 post graduate White 5 N/A N/A N/A N/A I found them easy to use and very helpful. I would love to have additional practice or assessment examples.
male 65-100 post graduate White 5 N/A N/A N/A N/A Very well done. This skill requires consistent practice (like most things). I recommend that PEG provide a practice site for all program graduates that requires no extra charge but is available for at least five years after the completion of the program.
male 30 - 49 university White 4 N/A N/A N/A N/A Good experience. Enjoyed the testing phases
male 19 - 29 university White 5 N/A N/A N/A N/A I’m really happy with this kind of opportunity to test yourself and learn from one of the best psychologists on planet. Proud that i achieved top scores on both tests, and hope that I will have opportunity to research and purchase other training tools and continue to practice in this subject.I’m psychology student very interested in emotions and non-verbal communication so this training tools are pretty amazing for me .
female 30 - 49 university White 5 N/A N/A N/A N/A I had become very isolated and insular and really had forgotten how to communicate with people, the tools helped me to identify how others might be feeling emotionally so that I could respond accordingly. I was a first but important step into re learning how to interact with people with increasing confidence. Now I feel this more intuitively and am considering a career in art therapy where the tools may once again be useful but for a different reason. Thank you for all the hard work that has gone into this. It has really helped me personally.
female 50 - 64 post graduate White 4 N/A N/A N/A N/A i found the tools very helpful, particularly the micro expression training, subtle expressions, profile. the one i did not find as useful was the responding effectively, perhaps my expectations were too high. i was expecting the training to be more interactive, have more alternatives, it is also a shame that the training only lasts a year. although i have practiced so often i sometimes feel i know the face of the person well, rather than the expression, i tend to leave the training for a while when i had 95-100% scores and then repeat it. i have shown the expressions to other people who have then gone onto buying the suite themselves. i find this work awesome and i wish i was part of it somehow. if you ever look for clinicians to use the tools to teach people to read facial expressions in daily life, for example to people who have autistic spectrum disorders, attention deficit disorder, trauma, then i am your guy. my details doctorana@mac.com www.doctoranasolutions.com www.childrensolutions.do.uk thank you all and paul Ekman for this good work
male 30 - 49 university White 4 N/A N/A N/A N/A Worked pretty well.
male 19 - 29 high school White 4 N/A N/A N/A N/A It is excellent! It could be further improved by showing moving faces (or how they would appear in conversation, as one does not always show micros from a neutral face)! Thank you!
female 19 - 29 university White 4 N/A N/A N/A N/A After having been trained in FACS, there was little left to learn, just good practice sets with better images
male 19 - 29 university White 5 N/A N/A N/A N/A It improves my social skills, I’m better to understand people now.
male 30 - 49 university White 3 N/A N/A N/A N/A No answer
male 19 - 29 university White 4 N/A N/A N/A N/A The added value of this training was understanding practically the rationale behind microexpressions and develop more accurate intuition to tell emotional signs thanks to immediate feedback. I ranked only 4stars because I was disappointed with RETT training. I thought it will teach catching micros in realtime situations. But the videos were the same for all the possibilities. Explanation of the possible replies was nice but the potential of this training was not met. If I haven´t bought RETT in a package with the rest of the training, I would consider wanting the money back. But since I enjoyed the rest and find it beneficial to my life and work I consider the money well spent. It would be great to have training not just with portraits but in real-time situations. It would get you closer to the Anatomy of lie. ;)
male 30 - 49 post graduate White 3 N/A N/A N/A N/A Very beautiful experience as a whole. But I could not take full advantage of the programs because of the language of the videos
female 50 - 64 university White 5 N/A N/A N/A N/A I was amazed at how fast I learned. I practice on HBO shows watching how the actors show emotion thru facial expressions besides dialog delivery.
female 19 - 29 university White 4 N/A N/A N/A N/A As a member of the Navy I wish that it was a longer prescription. I thought I would have the time to dedicate more to using these trainings but my commitments to the military superseded those desires.
male 30 - 49 university White 5 N/A N/A N/A N/A We use it in our organization to accompany our Interview Techniques course for officers.
female 19 - 29 university White 5 N/A N/A N/A N/A Very good programm, fast improvement. Thank you!
male 50 - 64 university White 5 N/A N/A N/A N/A Well they are easy to access, work 24/ 7 good pictures, good explenation
female 30 - 49 post graduate White 5 N/A N/A N/A N/A This tool is helpful for developing a recognition of micro expressions in others.
male 30 - 49 post graduate White 5 N/A N/A N/A N/A Awesome all around! :)
female 50 - 64 university White 4 N/A N/A N/A N/A I follow the work from Dr. Ekman for several years, I’ve read the books and was very glad for the possibility to train micro expressions online. Looking forward to get live-training possibilities in Germany.
male 19 - 29 university White 4 N/A N/A N/A N/A No answer
female 19 - 29 post graduate White 5 N/A N/A N/A N/A I use this tool for my personal education as well as through work as an occupational therapist to help kids on the autism spectrum understand how emotions manifest on faces and the possible motivations behind the emotion. Understanding emotions are one of the foundational components of social cognition and communication. Using the METT and SETT are useful for expanding these kids’ skill sets with best version of isolated facial expressions I’ve found!
male 30 - 49 other White 5 N/A N/A N/A N/A I can do things i didnt knew
male 30 - 49 post graduate White 4 N/A N/A N/A N/A I found them really educational and enjoyable. I did have some confusion regarding the tools which attained a certificate and those which didn’t. I found one or two mentioned in the My Trainings homepage that there was no certificate, but during the training itself made mention of a certificate. Clarification of this would be helpful
female 30 - 49 post graduate White 5 N/A N/A N/A N/A The RETT training was less useful. All other parts were great and extremely useful!
male 50 - 64 post graduate White 3 N/A N/A N/A N/A No answer
female 50 - 64 post graduate White 5 N/A N/A N/A N/A I am a trial consultant and used the tools for tune ups and new information.
female 19 - 29 post graduate White 4 N/A N/A N/A N/A No answer
male 50 - 64 post graduate White 5 N/A N/A N/A N/A Have studied them for some time, find that these skills have transformed my personal practice
male 30 - 49 university White 4 N/A N/A N/A N/A Good test… hard to show on a big screen… And a bit expensive…
female 19 - 29 university White 5 N/A N/A N/A N/A No answer
female 65-100 university White 5 N/A N/A N/A N/A Short Bio: Noemi Zarubova-Pfeffermannová studied at the Theatre Faculty of the Academy of Performing Arts, co-founded Duncan Centre Conservatory and taught art-therapy and ergotherapy at the First Faculty of Medicine of Charles University in Prague. In 1997 she has been appointed a vice-dean at the Music Faculty of the Academy of Performing Arts, where she leads courses in psychology of mime and gesture, stage speech and communication skills and supervises doctoral candidates. Dr. Ekman´s trainign tools she is using only at the Academy with students. It is a very big help.
male 30 - 49 university White 4 N/A N/A N/A N/A No answer
male 65-100 post graduate White 3 N/A N/A N/A N/A Did not put in enough time
male 30 - 49 trade / technical / vocational training White 4 N/A N/A N/A N/A Updated previous training. Nice to practice at own pace. Very happy with image quality and verbal instructions.
female 0 -18 other White 5 N/A N/A N/A N/A These training tools helped me to recognise deceit and to become more aware of other people’s feelings. Thank you very much Dr. Ekman for creating such useful tools and for sharing your knowledge with us!
male 50 - 64 trade / technical / vocational training White 4 N/A N/A N/A N/A unfortunately i didn’t spend much time with the program.
male 30 - 49 university White 3 N/A N/A N/A N/A Useful although slightly “old” design and UI
male 50 - 64 post graduate White 3 N/A N/A N/A N/A No answer
male 30 - 49 post graduate White 5 N/A N/A N/A N/A Very simple to use and to understand, a must for most people curious to understand better the world around.
female 19 - 29 post graduate White 5 N/A N/A N/A N/A I found it quite difficult in the beginning, but quickly started to gain skills and the ability to distinguish the expressions, even on the faster setting. Life got busy and I did not train with either tools or observation for quite a while - to my surprise, getting back to the training was a lot easier than expected and my skills had improved. The training makes for an excellent point of departure for real life use and skill improvement.
female 30 - 49 post graduate White 5 N/A N/A N/A N/A No answer
male 19 - 29 post graduate White 4 N/A N/A N/A N/A Really enjoyed the learning curve I experienced with the training - I find them oddly satisfying to complete :) (good thing)
male 19 - 29 university White 1 N/A N/A N/A N/A Not nearly enough samples to train on. I started recognizing the faces, and remembering what expression they were going to have instead of seeing the microexpressions. I want an order of magnitude more photos to train on.
male 30 - 49 university White 4 N/A N/A N/A N/A Easy to use, designed for quick learning and you are able to use what is inside for your everyday living
male 0 -18 other White 5 N/A N/A N/A N/A No answer
male 50 - 64 university White 3 N/A N/A N/A N/A I prefer METT, SETT than RETT. Let me explain. Dr. Eckman answers in RETT are time consuming and are not necessary. I would prefer, if all answers are written down. It would be more informative, if at least some cases would be presented for further two or three steps on. As it is now, it is not a training tool, but an explanation. Sorry. On the contrary METT, SETT are realy training tools. Sadly, they are technicaly not quite polished. I had suggested a year ago, to enlarge a video, and as I noticed, you did so to some degree. It is user friendly, if a video is almost as large as entire screen, and buttons neatly included at the bottom. So far I would recommend METT, SETT, but I would not recommend RETT.
male 65-100 post graduate White 5 N/A N/A N/A N/A The experience itself was excellent. I adminster it to students in an interpersonal communication class at the college level (Purdue University Fort Wayne campus). BUT way too expensive for only one year usage. I only teach one section of this class each semester. If it were unlimited or at least 3-4 years usage it would be more pragmatic. As a result, I will purchase no more tools to use in the classroom in spite of my high regard for Dr. Ekman and his work.
male 50 - 64 university White 4 N/A N/A N/A N/A I have nothing but good things to say about this program.
male 30 - 49 post graduate White 4 N/A N/A N/A N/A While this has nothing to do with the training itself, it was upsetting to get a coupon for a huge discount AFTER I purchased the entire package. I’m sure you’re familiar with all kinds of behavioral economics theories, so how do you think such a coupon might affect a customer’s satisfaction?
male 19 - 29 high school White 5 N/A N/A N/A N/A Since I started learning about microexpressions and body language my relationships had become easier to deal with, creating, keeping or leading them to an end. A dream also has born from these studies: work to create a world were people can properly understand each other, so I’m considering options from law enforcement to game development to work with. For those who gave me the chance to learn, specially Dr. Paul ekman, you have my deepest gratitude.
male 50 - 64 university White 5 N/A N/A N/A N/A Not only is the training valuable for recognizing micro expressions, it provides another level of awareness into human nature.
male 19 - 29 trade / technical / vocational training White 5 N/A N/A N/A N/A It’s simple! I use the technique everywhere now ! If you have promotion for other I can take it ! Just I has one problem with my password I can’t change it .. my account is blocked ..
male 30 - 49 post graduate White 5 N/A N/A N/A N/A Terrific
male 30 - 49 other White 5 N/A N/A N/A N/A Very good
male 30 - 49 university White 4 N/A N/A N/A N/A After the training results were much better. I’m not sure was it because of my development or because of remembering pictures.
male 30 - 49 university White 5 N/A N/A N/A N/A The tools I have used designed by Dr. Ekman have been invaluable in my work and in my life in general. I practiced with the tools daily for a month or two and then off and on again because it was fun and enjoyable for me. I learned a lot from the tools and have become so much more perceptive (correctly so I might add). I can now pick up on cues I would have never noticed without having gone through and used these fun and enjoyable tools. The feedback feature was amazingly helpful and helped me to get better and better each time I ran through the programs. Thank you Dr. Ekman!
female 50 - 64 trade / technical / vocational training White 4 N/A N/A N/A N/A Very helpful and fun to practice.
male 19 - 29 post graduate White 4 N/A N/A N/A N/A I liked a lot. Just think it could have more tests available with different photographs and even video.
male 50 - 64 university White 5 N/A N/A N/A N/A They have helped me in all walks of life and also to be more observent of people emotions I want to take the rest
female 30 - 49 other White 5 N/A N/A N/A N/A Very instructive, would like to buy a permanent licence to train any time i can :)
male 30 - 49 post graduate White 5 N/A N/A N/A N/A I believe these tools are core to improving communication. I would like to see this tools introduced to the American College of Healthcare Executives (ACHE) as a training tool for improving leadership communication as well as communication with our patients. My only recommendation would be to have the ability to periodically access the tools for me to refresh my skills. Thanks, Michael R. Feist PS: I like following Dr. Ekman on Linked In.
male 30 - 49 university White 5 N/A N/A N/A N/A I am able to see things that I was completely oblivious to before completing the training. I absolutely love it!
male 50 - 64 post graduate White 4 N/A N/A N/A N/A Useful Knowledge to know.
male 50 - 64 university White 5 N/A N/A N/A N/A Limited use, but what I did use was high quality and beneficial to my relationships.
female 50 - 64 post graduate White 5 N/A N/A N/A N/A I use this with students in our talk about empathy and really seeing a person and recognizing his/her feelings.
male 50 - 64 university White 4 N/A N/A N/A N/A very professional an helpfull for training
female 65-100 post graduate White 5 N/A N/A N/A N/A No answer
male 30 - 49 university White 5 N/A N/A N/A N/A An excellent tool in helping me to better understand those I work with.
female 30 - 49 university White 4 N/A N/A N/A N/A I use them for a teaching demonstration on the psychology of Emotion module on our degree
male 65-100 post graduate White 2 N/A N/A N/A N/A Interesting, but not really practical for my use.
female 50 - 64 university White 5 N/A N/A N/A N/A Wonderful, effective, profound . I have been planning to resume study. Thank you for the reminder. I get sad when I start to notice what might be deception in people I’ve trusted. So I’m going slowly and carefully.
male 0 -18 high school White 3 N/A N/A N/A N/A Good practice but sometimes would like more explanation
male 30 - 49 other White 5 N/A N/A N/A N/A Amazing understanding of facial expressions. I learned so much. I need to keep studying.
female 30 - 49 university White 5 N/A N/A N/A N/A No answer
female 19 - 29 university White 5 N/A N/A N/A N/A No answer
male 30 - 49 university White 5 N/A N/A N/A N/A Good/Nice
female 30 - 49 university White 1 N/A N/A N/A N/A I thought they were disappointing and overpriced.
female 50 - 64 post graduate White 4 N/A N/A N/A N/A HI I teach a Cognitive Psychology class (Psych of Sensation, Perception and Attention) at the School of the Art Institute Chicago. We have international students from 42 countries. This tool is important and it references our cross cultural traditions and practices, but I had to pay for it out of pocket on an Adjunct salary. It is only used for teaching this course. Will you extend my subscription for this term? I am teaching Jan 30- May 15. I would appreciate your help. Michelle A Downey Psy.D Lecturer School of the Art Institute 35 S Wabash Chicago IL 60605 312 656 2928 office
female 50 - 64 post graduate White 4 N/A N/A N/A N/A The opportunity to practice with different faces and to consider both subtle and profile expressions is invaluable.
female 50 - 64 post graduate White 4 N/A N/A N/A N/A Excellent tools but it takes a lot of time and practice to develop the skills.
male 30 - 49 post graduate White 3 N/A N/A N/A N/A It was the first time I was doing an online training and another language. It was interesting
male 50 - 64 post graduate White 5 N/A N/A N/A N/A Love them. Always have.
female 30 - 49 high school White 1 N/A N/A N/A N/A no support, no answer….
female 19 - 29 university White 5 N/A N/A N/A N/A I work with clients that have mental health symptoms and/or developmental disabilities and using the micro and subtle expressions training helped me recognize emotions that my clients weren’t comfortable expressing.
female 50 - 64 university White 4 N/A N/A N/A N/A I actually only managed to use it maybe 6-10 times. I was down for the count with vertigo for the year. I am just now recovered. From the limited use I’ve had, I was encouraged that it would help me in my business and my life.
male 50 - 64 post graduate White 3 N/A N/A N/A N/A Some freezing on videos and selections made a great course a bit frustrating at times. Some courses say it has a certificate but you never get to receive one despite scoring upper 90s…otherwise great content.
male 30 - 49 post graduate White 1 N/A N/A N/A N/A I contacted support a couple times and regardless of what they did and what computer I used. The simulation did not play correctly. Over time I forgot/didn’t have time to keep pestering them and was not able to use the training.
male 30 - 49 university White 5 N/A N/A N/A N/A No answer
female 19 - 29 other White 4 N/A N/A N/A N/A I liked the training tools very much in addition to my psychology studies. I would prefer an offer to buy the training tools permanently.
male 30 - 49 university White 4 N/A N/A N/A N/A I was a little disappointed by the tiny amount of emotions discussed, but it is definitely very interesting and innovative way of learning to respond to emotions. Thanks!
male 50 - 64 post graduate White 5 N/A N/A N/A N/A No answer
female 30 - 49 university White 4 N/A N/A N/A N/A Thank you so much for the free trail. It was incredible. If I end up teaching this course again, I will be sure to put the software in my budget.
male 19 - 29 university White 5 N/A N/A N/A N/A I regret I did not use these tools
male 30 - 49 post graduate White 2 N/A N/A N/A N/A Not much intstructions; not many viewing options; not much variety. Generally, disappointing. Feels like a waste of money.
male 30 - 49 university White 4 N/A N/A N/A N/A It’s a great way to learn and to improve our habilities!
male 30 - 49 university White 4 N/A N/A N/A N/A No answer
male 65-100 post graduate White 5 N/A N/A N/A N/A I used the tools in both Undergraduate and Graduate classes. After this training I also use some clips on how people respond to “microaggression” statements, and some case studies on deception from legal settings. It makes for a great educational package..
female 30 - 49 other White 5 N/A N/A N/A N/A Great tools! Thank you.
male 50 - 64 post graduate White 5 N/A N/A N/A N/A I am a high school forensic teacher. We set up a contest – $500 if a student could lie to the class. We modeled it after the experiment we saw on your TV show with the person looking at a picture and describing it. Round 1 – $5. Two girls won (the class voted). Prior to round 2 we watched the video of them, saw very clearly the tells. Round 2 – $50. No one won.
male 50 - 64 university White 5 N/A N/A N/A N/A I was unable to complete the training for the tools, but have seen them in use prior. Quality tools.
female 50 - 64 post graduate White 5 N/A N/A N/A N/A Our students have appreciated the tools and the professors will use these every semester for our process and skills classes.
male 30 - 49 trade / technical / vocational training White 5 N/A N/A N/A N/A It was a fantastic training tool. I have tried other training and this was the most realistic and Ekman is clearly the authority on the subject
female 30 - 49 post graduate White 5 N/A N/A N/A N/A Very useful introduction to facial expressions
female 30 - 49 post graduate White 3 N/A N/A N/A N/A If I repeat them, I can get a higher score, but I still can’t apply them in real life. I am not sure how to do so.
male 30 - 49 trade / technical / vocational training White 5 N/A N/A N/A N/A Very effective!!
male 30 - 49 university White 5 N/A N/A N/A N/A Very pleased and impressed.
male 65-100 university White 4 N/A N/A N/A N/A The training tool was verry succesfull for new police members.
male 30 - 49 post graduate White 4 N/A N/A N/A N/A No answer
male 30 - 49 university White 3 N/A N/A N/A N/A I only was able to do use the first tool that was really good. Due to circumstances I was unable to use the other tools of the package I rented and in that respect it was a poor deal for me. However the Staff was willing to give me some additional time what I appreciate a lot. I like the tools but I dislike the way it is offered. I just don’t like paying for something that expires and nothing left to review or use after that time. However I am probably more a person for a book or a DVD to learn at my own pace and be able to review later again whenever I need to refresh my memory. But that’s just me :-) Thank you for the material and the clear explanations
female 19 - 29 post graduate White 5 N/A N/A N/A N/A This training was just so helpful for my work as a conflict resolution professional. On a personal note, I found the trainings and my application of this knowledge an interesting psychological rollercoaster in itself.
male 65-100 university White 5 N/A N/A N/A N/A I enjoyed the ability to refresh the expressions and to repeat as necessary.
female 19 - 29 university White 4 N/A N/A N/A N/A An interesting insight into human expressions
female 30 - 49 high school White 5 N/A N/A N/A N/A I liked the micro expression training tools much better than the RETT tools, though RETT is probably helpful too.
male 50 - 64 post graduate White 5 N/A N/A N/A N/A I found Dr. Ekman’s tools opened the window to seeing Micro Expressions which I previously had been missing.
female 30 - 49 high school White 5 N/A N/A N/A N/A I prefer METT and SETT much more than RETT.
male 50 - 64 university White 5 N/A N/A N/A N/A I use them in class rooms training, i teach adults some can learn good some cant, some are made to go others want to go you have to approach each one differently almost to get the ideas across and see how it is received or the mood of the day :)
male 30 - 49 post graduate White 5 N/A N/A N/A N/A excellent tools . very helpful
female 30 - 49 trade / technical / vocational training White 5 N/A N/A N/A N/A No answer
male 50 - 64 post graduate White 5 N/A N/A N/A N/A Very humbling - I had to work and repeat the courses to have any chance of success.
male 30 - 49 university White 5 N/A N/A N/A N/A As a federal agent, this tool help me every single day at work. Very usefull.
male 50 - 64 university White 5 N/A N/A N/A N/A As a law enforcement officer and a crisis intervention coordinator the knowledge gained through the training has assisted me on numerous occasions. Whether I am helping a person in a psychiatric crisis or dealing with a criminal I am better enabled to have an insight into their mindset by reading their expressions. Thank you for your research.
male 50 - 64 trade / technical / vocational training White 3 N/A N/A N/A N/A Did not fit the page of my lap top and was always navigating around the page to see what’s going on. Apart from that seemed alright.

*Figure 10: This table shows the answers for all the participants that identified as “White.”

Lack of Variety and Racial Prejudice

The issue of lack of variety and racial prejudice could be addressed together by adding new photos to the set already in the product. The racial issue was that there was a number of comments on that certain demographics seemed to be only represented expressing negatively associated expressions (like anger, contempt, disgust). There could be the “prime” photos that Paul hand selected because he believed them to be best representative of the emotion in question but when it comes to just straight practicing, there can be a much larger pool of faces. A lot of people who brought up the issue of variety said they went through the photos a number of times and after such extensive practicing, they began doubting whether they were identifying the expression based on comprehension or recognition.

There’s also the issue of race. One thing that stands out to me is that race is a social construct and doesn’t in fact exist and yet we are using racial vernacular. Another thing is that, as mentioned in the first meeting, there would be value in looking at what the actual distribution is of ethnicity across the different emotions. Being a product based on and valued by its scientific integrity, I believe its also important that we show as little bias as possible. Depending on our vision and the scope of relavancy we want to encompass in terms of societal constructs and social issues, its also important to have (and ultimately teach with) an as inclusive of a tool as possible. Being able to read and understand the faces that are unlike ours (regardless of what ethnicity “ours” refers to but particularly when it comes to minorities) is so crucially integral particularly when it comes to the justice system and law enforcement - which happens to be a sector that a lot of our clients come from. MacArthur fellow and adjunct professor at Stanford University, Jennifer Eberhardt has done a lot of research on the importance of implicit racial biases and received the Genius award for her work. For us to not only be a commercial product but one whose core was founded in science, I think it is very important that we take this into consideration and really think about what message and messages we are sending and want to send.

Technical Issues

People did not find the website user-friendly or experienced technical issues. Some would mention that they would have difficulty loading a page or the website wouldn’t acknowledge that they had completed a section. The biggest themes were that our website is not mobile-friendly so it wasn’t as easily accessible and “on-the-go” as people would have liked and that there were times when the website said there was a certificate but they did not receive one despite scoring very high on the tests. I think this is a pretty easy fix in terms of ensuring that the website is compatible in terms of refresh rates, etc with all the major browsers and that we go through whatever channels in WordPress to ensure mobile-friendly access.

Money: Barrier to Entry

The barrier to entry issue of money could be solved in a variety of ways. People would say that they were so busy in their personal and professional lives that they didn’t have a chance to use their tools. I think a built-in aspect where if go to a help section, they will just be navigated to the option of a month extension that will be automatically granted if its their first time requesting (this will bypass the issues of customer service response time, etc.) If they go to cancel, make it simply worth it for them to keep the service – a steep discount or something like that. There could be a referral system with user-specific discount codes in order to track how people are finding out about us and why they are type buying. This would also be helpful in terms of giving us data to deal with without having to ask how they heard about us in a survey. If we don’t want to make it a cut in profit with discounts for renewing customers, there was also suggestions in the feedback sections that we provide them with something that they can continue to use after their year is up – the idea of paying so much for just one year and then no access to any of the materials after that was just not worth it for people.

Another metric we were interested in was usefulness of experience by gender (Figure 11). Here we can see that more of the participants were male. A majority for both men and women scored it as either 4 or 5 in terms of usefulness.

Figure 11: This figure shows usefulness of experience scores by gender.

When we looked at the usefulness of experience by age (Figure 12), we saw that a plurality of our participants fell within the age group of 30-49 (Figure 13). Almost all of our participants were between the ages of 19 and 64. For each age group except for 0-18, the highest count was for a usefulness score of 5. For 0-18 aged participants, it was a split between 3 and 5 for the majority. Usually the score of 4 was the next highest in terms of count but for the age group 65-100, scores of 3 and 4 were tied for second place.

Figure 12: This figure shows usefulness of experience scores by age.

Figure 13: This figure shows the age distribution of participants in order to have a more accurate understanding of Figure 12.

Finally, the last thing we looked at was usefulness of experience scores by education (Figure 14). We found that for every educaton group, the majority scored either 4 or 5. As was established earlier, most of our participants had university or post graduate education (Figure 15).

Figure 14: This figure shows usefulness of experience scores by education.

Figure 15: This figure shows the age distribution of participants in order to have a more accurate understanding of Figure 14.

Survey Issues/Collecting More Accurately Representative Information

Building some data gathering into the infrastructure of the website would make it much easier for us to discern trends that can be optimized upon in order to increase profit margins. Our target demographic is clear: (majority-based) male, 30-49, have university or post-graduate level education, white, and lives in the US, UK, or Canada. Getting a name, email address, zip code and country, and one or other two data points shouldn’t be that hard to do. Our product is not one of little investment - a sentiment that has been echoed by many in the Feedback Survey. If they have decided to buy the product, in order to set up their account and to use it, I am highly doubtful that they would object. Filling out those few boxes is omnipresent - our world is saturated with sign up pages for websites, products, etc. Also, there are many responses where people write their whole back story and why these tools were so valuable and useful to them - we cast a very specific net with these tools and as a result, in my opinion, catch a very dedicated group of people. I believe that detail should be noted and valued when it comes to customer participation and engagement. A referral system as previously stated would be helpful in terms of tracking what demographic our customer base lies in but would also provide incentive for current users and potentially bring us new customers that would otherwise not have bought our product. Considering we have a solid customer base in educational institutions and law enforcement, I believe the referral system would be particularly useful and helpful.