This week is course evaluation week. I’ve sent you all an email with the link for the evaluation form for this course. Before you start this weeks exercises I’d like you all to complete the evaluation (should take up to 20 minutes). Its really useful for me (and the course) to get your feedback so we can both see what is working weel, and see what we can improve for future courses. Thanks for doing this.
One of the advantages of Inquisit is that it allows you to post and run your Experiments online. To do this you need an Inquisit web licence. Fortunately, the university has one, Unfortunately we can only run one Experiment on it at a time. Therefore I can’t get you to practise doing this yourself. But before we start programming this weeks study, I will go through a demo of some of the features associated with using Inquisit on the web, by using one of our old Experiments (the face study).
This week the Experiment we program is based on the validation of the Team Role Test conducted by ( Mumford et al., 2008 ). For the part of this study we are interested in, the researchers presented participants with a scenario which requires the skills of someone with a certain role in the team (Critic, Calibrator, Coordinator, Creator etc.) to solve. To test the participants knowledge of the skills needed for this role, they were then presented with 4 possible solutions to the scenario, and asked to rate how effective each solution is. They did this by selecting one of 5 responses for each solution;“Very Ineffective”, “Somewhat Ineffective”, “Neutral”, “Somewhat Effective”, “Very Effective”. They did this for each of 10 different roles. we will do this for 2 roles. The details of the solutions and scenario for one of the roles (collaborator) can be found in the appendix of the paper.
As this study is essentially in a survey format, this week we will program this study, and in the process learn about different types of survey elements, including surveypages, radiobutton and survey elements.
Like every week we are going to start a new Inquisit file for this weeks exercises. Call this file surname_wpa9.iqx. This is the file you will send me at the end of the class. I strongly recommend that you copy the defaults element from last weeks script into the top of this file, so that you have an easy way to terminate the Experiment.
In previous weeks we have followed a very hierarchical structure when creating our Experiments. We first start by defining ht estimuli, how they are presented, how atrial works, how trials and linked in blocks, then how blocks are linked in the overall Experiment.
This week we are creating a survey, which will involve a different structure and elements, but will follow the same bottom up construction. First we are going to define the individual questions participants will be asked and how they can respond, then how these questions are presented together on a page, and finally how pages are linked together in the survey.
The first step is to create one a question that our participants will be asked. We do this by creating a survey item type element. There are various different types of survey items we can create depending upon the type of response your participant has to give. For instance, we used a slider element in wpa 6, which is a type of survey item.
This week we don’t want participants to respond on a slider, instead we want them to pick one of the 5 options (“Very Inefficient”, …, “Very Efficient”). There are two survey item types that look like they mighe be able to achive this. One is called radiobuttons and the other checkboxes. Have a look at the help file. Whats the difference between the two? Which would we want to use?`
The primary difference is that for radiobuttons the participant can only select one item for each question. For check boxes they can select as many as they want (i.e. they could select 1 or all 5, although you can also specify a number).
In our survey we want participants to only be able to select one response per solution/item so we should use the radiobuttons element.Now we’ll create our first survey item. Name this item cal1 as it will be our first calibration solution. There are three attributes that we will need to define for the element, the options attribute, the optionvalues attribute and the caption attribute.
The options attribute takes a list of labels for the choice options/radiobuttons. This also defacto defines how many choice options there are. Use the 5 labels from the Table in Appendix A of the paper.
<radiobuttons cal1>
/options = ("Very Ineffective", "Somewhat Ineffective", "Neutral", "Somewhat Effective", "Very Effective")
</radiobuttons>The captions attribute lets us specify text that should be displayed at the start of the Question. Use the text for the first Calibrator Question in the table
<radiobuttons cal1>
/caption = "Get the quiet new sales rep involved by asking if she has noticed that the older sales rep has taken some of her sales as well (role inconsistent)."
/options = ("Very Ineffective", "Somewhat Ineffective", "Neutral", "Somewhat Effective", "Very Effective")
</radiobuttons>In the paper each of the 5 responses was given a value, from 1 to 5, so that a total score could be calculated across all 4 items for each role. We can specify these values by using the optionvalues attribute.
<radiobuttons cal1>
/caption = "Get the quiet new sales rep involved by asking if she has noticed that the older sales rep has taken some of her sales as well (role inconsistent)."
/options = ("Very Ineffective", "Somewhat Ineffective", "Neutral", "Somewhat Effective", "Very Effective")
/optionvalues = ("1", "2", "3", "4", "5")
</radiobuttons>Before we go further and create more items you should have a look at what is displayed for this item. Unfortanely to do this we need to create at least a surveypage element (if you run the script without this it will just display nothing. We have covered surveypages before, but as a quick reminder we can use the questions attribute to define which survey items are displayed on the page and in what order, just like we use trials attributes at the block level (or blocks at the expt level).
Sice we only have 1 item, our questions attribute should have a single entry. Call the surveypage element.
<surveypage cal>
/ questions=[1=cal1]
</surveypage>If everything is working correclty you should see a single surveypage with a single question being asked, and 5 labelled radiobuttons listed under the question. However, these buttons are listed vertically. Since we are going to try and display 5 items on this page, it would make more sense if the repsonses are place next to each other. You can use the orientation attribute of the radiobuttons element to specify whether the buttons should be listed horizontally or vertically.
<radiobuttons cal1>
/caption = "Get the quiet new sales rep involved by asking if she has noticed that the older sales rep has taken some of her sales as well (role inconsistent)."
/options = ("Very Ineffective", "Somewhat Ineffective", "Neutral", "Somewhat Effective", "Very Effective")
/optionvalues = ("1", "2", "3", "4", "5")
/orientation=horizontal
</radiobuttons>For the current design we don’t want to display a single item per page. Instead we want to the participant to see all 4 solutions for a single scenario at the same time. So we need to create multiple survey items, place them all on the same page, and also add the scenario description.
To display multiple items on a page, we first need to have multiple survey items. Before we go further on your current surveypage, try to display your single radiobuttion survey item cal as both the first and second question. What happens?
<surveypage cal>
/ questions=[1=cal1; 2=cal1]
</surveypage>
On my system it displays the question only once, i.e. it does not display the duplicated question.This doesn’t really matter anyway because we wanted each item to have a different question caption. Create your 3 other items.
<radiobuttons cal2>
/caption = "Remind the two sales reps that personal attacks are not appropriate and that the team should focus on the future solutions."
/options = ("Very Ineffective", "Somewhat Ineffective", "Neutral", "Somewhat Effective", "Very Effective")
/optionvalues = ("1", "2", "3", "4", "5")
/orientation=horizontal
</radiobuttons>
<radiobuttons cal3>
/caption = "Support the new team members by taking their side to make sure they are not used as â scapegoats for the teams problems (role inconsistent)."
/options = ("Very Ineffective", "Somewhat Ineffective", "Neutral", "Somewhat Effective", "Very Effective")
/optionvalues = ("1", "2", "3", "4", "5")
/orientation=horizontal
</radiobuttons>
<radiobuttons cal4>
/caption = "Remind the team that making critical remarks about specific people makes people defensive and will prevent the members from accomplishing anything as a team."
/options = ("Very Ineffective", "Somewhat Ineffective", "Neutral", "Somewhat Effective", "Very Effective")
/optionvalues = ("1", "2", "3", "4", "5")
/orientation=horizontal
</radiobuttons>Now add them to the page so that all 4 items are displayed on a single survey page.
<surveypage cal>
/ questions=[1=cal1; 2=cal1; 3=cal3; 4=cal4]
</surveypage>Now we want to add the desctiption of the scenerio. In which element do you think we can add this description? Try it.
We want it to display at the top of the survey page once, not once for each survey item. So you just need to use the caption attribute of the survey page item.
<surveypage cal>
/caption = "You are a member of a sales team at a local bookstore, where recent sales have been decreasing substantially due to a shrinking number of customers. You are in a team meeting discussing solutions to the declining sales problem. The discussion becomes
a bit heated when the oldest team member suggests that the sales numbers for the new sales reps are quite low. One of the younger reps quickly counters that every time he asks for help with a customer, the older rep takes credit for the sale. The other new sales rep simply looks at the floor and says nothing.
Please rate the effectiveness of each of the following responses."
/ questions=[1=cal1; 2=cal1; 3=cal3; 4=cal4]
/fontstyle= ("Arial", 2%)
/ finishlabel="Next"
</surveypage>Most likely your questions and sceneraio don’t all fit on the page. So you’ll need to adjust the font size. There ar three places you can change this 1) On each Survey item, 2) on the surveypage, or 3) at the defaults. Whats the difference?
If you change it in the radiobutton element it will cahnge it only for that survey item (not the other 3 questions or the scenerio description.)
If you change it in the surveypage element, it will change it for all survey items and the the sceneraio description.
If you change it in the defaults section, it will also apply to future surveypageelements you create.post(
REMEMBER: If you change the fontstyle at a lower level (i.e. for a single survey item), it will ignore the fontstyle set at a higher level (i.e. surveypage or defaults), for that item.
)
<surveypage cal>
/caption = "You are a member of a sales team at a local bookstore, where recent sales have been decreasing substantially due to a shrinking number of customers. You are in a team meeting discussing solutions to the declining sales problem. The discussion becomes
a bit heated when the oldest team member suggests that the sales numbers for the new sales reps are quite low. One of the younger reps quickly counters that every time he asks for help with a customer, the older rep takes credit for the sale. The other new sales rep simply looks at the floor and says nothing.
Please rate the effectiveness of each of the following responses."
/ questions=[1=cal1; 2=cal1; 3=cal3; 4=cal4]
/fontstyle= ("Arial", 2%)
</surveypage>So far we haven’t created a survey with multiple linked pages. We’ll go through 2 ways to do this now. The first will use a block structure like you are familiar with. The second a survey structure.
First we need to have another surveypage to display. To do this we need to create another set of 4 survey items, and place them on another surveypage element, with a sceneraio description, Unfortunately the paper only has a single scenario and set of questions/solution, but you can go through and make up another sceneraio and for possible solutions for this scenario. or if you are lazy (like me) just add some placeholder text so you can check which Question number it should be.
for reference in my later code the surveypage element I created was called crit. I’m not going to paste the code here, as its the same as the code for your current surveypage and items, just with different captions.
One way to display both surveypages is to treat them each as trial. This means that if we want to display both, all we need to do is list them both in the trials attribute of a block, just like we’ve done in every experiment so far.
<block all>
/trials = [1=cal; 2=crit]
</block>The second way to combine them is to use a survey element instead. This has several advantages over combining them as blocks. I’ll point out two of them.
To combine them in a survey element is as easy as combining them in a block element. We just need to pass the 2 pages to the pages attribute of survey the same way we passed them to block.
<survey all>
/pages = [1=cal; 2=crit]
</survey>One advantage of combining them as a survey is that by default participants can move through the survey pages using the back and next keys. This allows them to return to a page they have already answered and change their response. A disadvantage of this is that Inquisit will only store the last response given on a page, so you won’t be able to track changes in their answers.
Of course you can also prevent participants from returning to previous pages if you ant, by using the showbackbutton attribute.
<survey all>
/pages = [1=cal; 2=crit]
/ showbackbutton = false
</survey>The second advantage is that the survey element has an array of options for definign exactly how the survey pages should look. For instance you can specify whethe the page number should be displayed or whether item number should be displayed.
Currently the pages are presented sequentially with Calibrator always shown first. You can instead randomise this (using the either survey or block version), just like you randomise trial order in normal blocks.
<survey all>
/pages = [1-2=noreplace(cal,crit)]
/ showbackbutton = false
</survey>You might also want to combine multiple surveys in a single experiment. One way you might be tempted to do this is just to treate each survey element like a trial, and place them all in a block. Try passing the survey element, which I’ve called all to a block twice, once as trial one of the block and once as trial 2. What happens? Can you think of another way to do this?
<block>
/trials = [1=all; 2=al]
</block>
Youll get an error. This is because Inquisit doesnt classify surveys as a type of trial (it will treat surveypage like a trial however). Instead it treats them more like a block. Therefore you should call repeat or multiple survey the same way you would multiple blocks.
<expt>
/blocks = [1=all; 2=all]
</expt>Sometimes within a single survey you might want to show the same page to a participant twice, for instance to see if there response changes after they see new information. try doing this in our design using both the block method and the survey method. Set it up so that the Calibrator page is shown, then the Critic page, then the Calibrator again.
<survey all>
/pages = [1=cal; 2=crit; 3=cal]
</survey>
<block all>
/trials = [1=cal; 2=crit; 3=cal]
</block>Do you notice any difference between the methods? Have a look at the datafile generated by each.
As I mentioned above if a participant returns to a surveypage using the back buttons, only the last response on that page is stored. Unfortunately it seems like the same thing happens if you force them to do the same page twice in a survey. If you look at the datafile for the survey version you should see that there is only a single response for questions cal1, cal2, cal3 and cal4 recorded.
The same issue doesnt occur in the version which treats them as trial (it also stores the individual question responses slightly differently).if you look at your two survey pages, they are basically the same with just the text for each question, and the scenario changing. This makes it seem like a lot of work to have everything duplicated. Below I’ve got some code for a version, using the block method (it doesn’t work for the survey method), which uses just a single surveypage element, and 4 radiobutton elements. See if you can understand how it works. I haven’t used the full scenerio and question texts, just placeholders. You can also look at file week10.2 in the Pres5 folder of the dropbox.
<defaults>
/quitcommand = (Ctrl+'e')
/fontstyle=("Arial", 3%)
</defaults>
<values>
/counter=1
</values>
<item category>
/1="Calibrator Scenario"
/2="Critic Scenario"
</item>
<item q1>
/1="Calibrator Question 1"
/2="Critic Question 1"
</item>
<item q2>
/1="Calibrator Question 2"
/2="Critic Question 2"
</item>
<item q3>
/1="Calibrator Question 3"
/2="Critic Question 3"
</item>
<item q4>
/1="Calibrator Question 4"
/2="Critic Question 4"
</item>
<radiobuttons cal1>
/caption = "<% getitem(item.q1, values.counter)%>"
/options = ("Very Ineffective", "Somewhat Ineffective", "Neutral", "Somewhat Effective", "Very Effective")
/optionvalues = ("1", "2", "3", "4", "5")
/orientation=horizontal
</radiobuttons>
<radiobuttons cal2>
/caption = "<% getitem(item.q2, values.counter)%>"
/options = ("Very Ineffective", "Somewhat Ineffective", "Neutral", "Somewhat Effective", "Very Effective")
/optionvalues = ("1", "2", "3", "4", "5")
/orientation=horizontal
</radiobuttons>
<radiobuttons cal3>
/caption = "<% getitem(item.q3, values.counter)%>"
/options = ("Very Ineffective", "Somewhat Ineffective", "Neutral", "Somewhat Effective", "Very Effective")
/optionvalues = ("1", "2", "3", "4", "5")
/orientation=horizontal
</radiobuttons>
<radiobuttons cal4>
/caption = "<% getitem(item.q4, values.counter)%>"
/options = ("Very Ineffective", "Somewhat Ineffective", "Neutral", "Somewhat Effective", "Very Effective")
/optionvalues = ("1", "2", "3", "4", "5")
/orientation=horizontal
</radiobuttons>
<surveypage cal>
/caption = "<% getitem(item.category, values.counter)%>"
/ questions=[1=cal1; 2=cal2; 3=cal3; 4=cal4]
/fontstyle= ("Arial", 2%)
/ finishlabel="Next"
/ontrialend = [values.counter = values.counter+1]
</surveypage>
<block>
/trials = [1=cal; 2=cal]
</block>