Summary conversation with EJ

After talking with EJ, here are my big “take home” points about where we ought to direct our efforts:

Human Factors

Under the direction of Polly Tremoulet (Human Factors specialist) and myself, we are submitting IRBs for four different projects:

  1. Flexplot: Graphics have the advantage of encoding a large quantity of information in an image, while also highlighting the degree of uncertainty associated with a hypothesis. Unfortunately, guidelines for mapping graphics onto procedures are lacking. In this project, we offer eight heuristics to use when producing graphics, and demonstrate their validity by showing graphics to participants that adhere to the heuristic and graphics that violate the heuristic and assessing confidence and accuracy. The results of this study will inform the default graphics shown by flexplot.

  2. Statistical Orientation: With Rink Hoekstra, Fiona Fidler, and Polly Tremoulet we are attempting to see how presenting statistical results to people from different theoretical orientations (Bayesian, NHST, estimation, and graphical) affects participants’ confidence and assessments of congruence between multiple studies. In this study, we present the same data to individuals using each of the different paradigms and assess the degree to which the presentation of data improves “statistical cognition.” I’m assuming this project will demonstrate that NHST-based statistics are detrimental and will make a stronger case for migrating to JASP.

  3. Results format. In this project, we seek to understand how the way statistical information is presented enhances or denigrates human perception. Specifically, we will vary amount of information (exhaustive versus minimal), the way summaries are reported (visual versus text-based), and the types of inferential statistics reported (p-values versus effect sizes/confidence intervals). We will start with focus groups and formative testing to identify an “optimal” way of presenting statistical information, then eventually test this format using a module created in JASP against SPSS (and probably against the existing JASP format). If individuals better understand data with the new presentation format, this could potentially inform how future updates of JASP choose to present data.

  4. Instructional method. This semester I am piloting a different approach to teaching statistics. My focus is on graphical interpretation, evaluating effect sizes, and interpreting confidence intervals. (In the future, I hope to instead teach credible intervals). At the end of this semester (sometime in April), I am going to assess how well my students are able to accurately interpret data, relative to a different class that was taught using cookbook-style NHST statistics. This will serve as pilot data for a quasi-experiment in the fall.

JASP development update

AI component

One of our students (Alek) has been parsing R packages for each function and trying to identify the types of inputs (e.g., vector, matrix, data frame, booleon) required for each function within a package. He recently learned of R CMD CHECK and will see if he can borrow code from that to make the task of identifying inputs easier.

Module development

We have looked into QML and the documentation is excellent. This week we will start development of a flexplot module. Once we get that working, we’ll start working on other modules (or elements of the same module), including a general linear model module that automatically chooses graphics to match the analysis, and probably a time-series module (or folder?).

Making graphics editable

This is currently put on hold until after we develop a module. We want to familiarize ourselves with integrating graphics with JASP first before trying to tackle that problem.

Using alpha JASP

I was finally able to download JASP and play around with it. I have a couple of questions/comments:

  1. Is there a way to do general linear models? Can I run a model with both numeric and categorical predictors? I’m not currently seeing that option.
  2. I like the addition of the collapsible sections for different analyses performed.
  3. It seems the text is cut off (after “Plot of coefficients”) and I don’t see any way to change it:
alt text

alt text

I’ll keep looking into it and offer more thoughts as they come.