I’m Brad Wakefield

• I’m a Statistical Consultant at UOW working in the Statistical Consulting Centre in NIASRA.
• I started working as a consultant in July 2021.
• I have been at UOW for a long time, first completing a Bachelors in Advanced Mathematics (Hons), and then my PhD in Statistical Disclosure Control.

Is a unit with NIASRA directed by Prof. Marijka Batterham, that provides (usually) free statistical advice to academics and HDR students.

Aim - To improve the statistical content of research carried out by members of the University.

We can help with…

We assist on many projects…

• Regardless of faculty, discipline, or entity, provided there is data and analysis, we can assist.

• It’s not a lecture or a lab.

• It’s not a test to see how much statistics my clients know.

• I won’t do my client’s work for them.

• I’m not a supervisor.

• I’m not an expert on everything.

• I do not make the data say anything I want.

• Clients can (and do) ignore my advice.

An initial consult tends to have the following outline

1. We introduce ourselves.
2. I ask the client to explain their problem to me.
3. I ask a few questions to make sure I understand them and their problem.
4. I then ask the client why they have come to see me.
5. I give some technical advice.
   • This may include explanations or demonstrations.
6. I encourage the client to ask me questions.
7. We decide on a plan forward.

Sometimes…

• people have very specific questions or calculations,

• people don’t know what questions to ask,

• people may be nervous or overwhelmed,

• people may be at very different stages of their project,

• multiple people may be in the meeting and have conflicting ideas / agendas.

As a consultant, being able to adjust your advice or approach is critical.

• When you enter the consult, you are the subject matter expert.

• I will ask you to explain your understanding of the problem, the research, the discipline more broadly. Be prepared for this.

• Provide a general / high level explanation - I may know as little about your area as you do about statistics.

I ask every one of my new clients to go easy on me when it comes to technical terms or jargon.

• In your brief, include:

  • The research problem / objectives.

  • A summary of the current stage of the project.

  • Key terminology / values we will be looking at (with accessible definitions).

  • Any analyses you would specifically like to discuss.

  • What you would ultimately like to have achieved from our consultation.

• Avoid:

  • Making references to literature.

  • Copying and pasting your research proposal or abstract or technical chunks.

  • Referring to terminology not defined.

• It is usually more efficient for me to ask you for the information directly or as needed.

• I usually prefer to chat before I look at any specific information.

• If I want material beforehand, I will specifically request it.

You don’t need to perform extensive analysis, but familiarise yourself with the data before coming (if you have it).

Be prepared for questions like…

• How many observations have you collected?
• What are the dependent / exposure / covariates we are looking at?
• Do you have repeated data? How is it recorded?
• Do you have missing data?
• What does the data look like and what form does it take?
• Do you have any small categories or corner cases?

If your data contains personal information, it should be treated as confidential.

• Do not send me your dataset before we meet if it contains information about people.

• Ensure you have a de-identified copy of your data.

• Familiarise yourself with the terms of your ethics approval and data management plan.

• IF I request to see your data, share safely.

Data privacy is an issue I care deeply about, please to do not put me in a compromising position.

Write down key goals for the consultation beforehand.

• What research objectives do you want to discuss?

• Are there any questions you would like to ask (technical or practical)?

• What aspects are you uncomfortable with or unsure about?

Sometimes we wont have time to address everything in one consultation, but raise everything so that we can properly prioritise and plan.

One of the biggest mistakes you can make during a consult is to tell me you understand when you don’t.

• Everyone understands things differently - being confused is natural, it does not make you slow or stupid.

• I will not judge you on your understanding of statistics (or lack thereof); I expect you not to judge me on my lack of understanding on your area of research.

• If you have an issue, it will not get addressed until it is acknowledged.

I am not the project manager, and I will not check in on your progress.

• If you have a deadline approaching, contact me early.

• Allow for time to work and review.

• You can’t expect I’ll be available when you need me.

• It is often best to book an appointment with me.

Simple steps to lighten the cognitive burden on me, will ensure we have the most time possible to discuss your problem.

• Highlight important sections you want advice on. - I’m not going to read an entire paper/thesis.
• Use graphs and figures to explain complex ideas.
• Provide context to explanations.
• Tell me what other people have done in similar papers / projects.

Sometimes you may hear feedback you don’t like.

Common Problems include:

• The analysis method performed has flaws.
• There is not enough data to conduct the analysis.
• The results are not significant and running alternative tests until you achieve significance is wrong.
• Complexity for the sake of complexity is problematic.

While as a client you choose what you do with the feedback. Failing to acknowledge it will surely lead to trouble.

When going into a consultation:

• Make it clear what role you will be providing.

• Make it clear what is outside the scope of your role.

• Set boundaries.

• Be honest about your potential commitment.

Clients wont know if they are overstepping the mark unless you tell them.

I want to know:

• Who is coming to see me and what field of research they come from.

• How familiar the client is with statistics and what stage of research are they in.

• What is their core research problem and why are they coming to see me.

• If there are any specific analyses / problems that I should brush up on.

What I don’t want:

• Their entire thesis or a stack of research papers related to the subject.
• All their data and statistical output.

Managing your time effectively is important.

• Trying to decipher stacks of information without context is very time consuming.
• Consultations may not exactly end up being as they appear on the brief.

It is usually better to speak to the client FIRST.

Getting to know the client helps to build rapport.

I ask the client to explain their research / problem to me first.

1. So I can get a better understanding of the field of research.
2. It gets them talking and about something they are very comfortable with.
3. I can get a sense of their level of understanding.
4. By asking questions, I can let the client know what terms, and concepts I am and aren’t familiar with.

When forming my an assessment consider.

• Is this feasible for the client?

• Is this feasible for their timeline?

• Is it statistically valid?

• What is the audience of this work?

Check in with your clients to make sure they are following.

Some people may feel uncomfortable saying they are confused.

Try and encourage the client to repeat your advice back to you.

Model this in your own actions by repeating your understanding of their problem back to them.

Although clients may only ask about the analysis think about:

• What form there data is in, and how to wrangle it?
• Will any pre-processing be necessary?
• What diagnostic checks will be needed?
• How will the results be presented?
• What role will data visualisation play.

Not all problems need to be addressed in the first consultation.

• Disclose when you are unfamiliar with a method or analysis.

“No one – however smart, however well-educated, however experienced – is the suppository [sic] of all wisdom” - Former PM Tony Abbott

• If you are unsure about some advice, qualify it.

• Do not take on problems, if you are unsuited to it.

Always end you consultation with a plan of action.

• What does the client need to do before our next meeting?

• What do I need to do?

• When should we meet next?

• What does the timeline look like?

Mistakes to look out for…

1. Analysing data types incorrectly.
2. Misunderstanding hypothesis testing, p-values, and significance.
3. Not checking assumptions.
4. Failing to acknowledge false positive bias.
5. \(p\)-Hacking
6. Over-fitting and under-fitting models.
7. Overstating results.
8. Correlation vs. causation confusion.
9. Failing to consider sample bias.

One of the most difficult to identify:

Obtaining conclusions from variables that do not measure what they think they measure.

We should also take steps to improve the professional practice of our clients:

• Encourage the use of open-source tools and software.
• Document everything.
• Use defensive coding.
• Use version control.
• Share data, code, and meta-data.
• Practice dependency management.
• Support literate programming.
• Encourage the ethical use of statistics.

For the vast majority of clients, their problems can be solved with “standard” statistical analysis.

Things like:

• Chi square tests

• T-Tests

• ANOVA

• Regression models - LMs GLMs LMMs GLMMs GEEs.

• Non-parametric tests (ranked-based).

• Questionnaire data

  • Dealing with Likert scales

  • Reliability analysis

  • Exploratory factor analysis

  • Confirmatory factor analysis

  • Structural equation modelling (still a bit shaky on).

Speaking in terminology more consistent with the clients knowledge is still tricky.

• Dependent / response / outcome variables.

• Independent / covariates / explanatory / factors / predictors variables.

• Modifiers / controlling variables.

• Mixed model / multilevel model / heirarchical linear model.

• Surveys / questionnaires / instruments.

• Measures / scales / constructs.

Educating the client about what they want is actually the biggest battle.

Some clients don’t know what they want. Or they have a general non-specific idea of what they want.

A client may come to me and say

“I want to know what factors affect Y”

My first questions will be … how did you measure Y? What factors are of interest? How were each of these factors measured? Do you think any other variables may modify the effect of each of the factors on Y? What does the data look like?

I often can’t answer these questions for the client.

Data often needs to be summarised or “wrangled” in order to be ready for analysis.

.

While the actual “analysis” may be easy …

Getting the data in the correct form - developing scores, extracting categories, dichotomising, converting to long form, linking data sets, can all be a huge battle.

Educating the client how best to wrangle the data is a BIG part of it.

Other topics I have been asked about:

• Data mining and linkage
• Power analysis
• Survival analysis
• Survey analysis
• Geo-spatial analysis
• Meta-analysis
• Machine Learning