Which Statistical Test?

Statistical Modeling: A Primer (by Kevin Gray)

November 7, 2022

Interesting article by Kevin Gray at Cannon Gray (http://cannongray.com) Model means different things to different people and different things at different times. As I briefly explain in A Model’s Many Faces, I often find it helpful to classify models as conceptual, operational or statistical. In this post we’ll have a closer look at the last of these, statistical models. First, it’s critical to understand that statistical models are simplified representations of reality and, to paraphrase the famous words of statistician George Box, they’re all wrong but some of them [READ MORE]

Which test: Compare TWO DEPENDENT groups (Paired, Matched, Same respondent groups)

August 3, 2022

When the research objective is to compare two dependent groups, which means they are paired, matched, and thus the same respondent groups in a pre- post-test, we have a choice among different statistical procedures, depending on the following variable characteristics: Number of variables: One dependent variable and one independent categorical variable (two levels or groups) Examples: Are the means / frequencies (on the dependent variable) of the same respondents over two different time periods significantly different such as in a pre- post test? Are [READ MORE]

Which test: Compare MORE THAN TWO DEPENDENT groups (Paired, Matched, Same respondent groups)

July 20, 2022

When the research objective is to compare more than two dependent groups, which means they are paired, matched, and thus the same respondent groups in a pre- post-test, we have a choice among different statistical procedures, depending on the following variable characteristics: Number of variables: [Unless where otherwise indicated] One dependent variable and one independent categorical variable (more than two levels or groups) Examples: Are the means / frequencies (on the dependent variable) of the same respondents over more than two different time periods significantly [READ MORE]

Which test: Compare MORE THAN TWO INDEPENDENT groups (Unpaired, Unmatched, Different respondent groups)

June 12, 2022

When the research objective is to compare more than two independent groups, which means they are unpaired, unmatched, and thus different respondent groups, we have a choice among different statistical procedures, depending on the following variable characteristics: Number of variables: [Unless where otherwise indicated] One dependent variable and one independent categorical variable (more than two levels) Examples: Are the means / frequencies of more than two independent groups of respondents significantly different? When the dependent variable is BINOMIAL / BINARY / [READ MORE]

Which Test: Logistic Regression or Discriminant Function Analysis

September 17, 2021

Discriminant Function Analysis (DFA) and Logistic Regression (LR) are so similar, yet so different. Which one when, or either at any time? Let’s see…. DISCRIMINANT FUNCTION ANALYSIS (DFA): Is used to model the value (exclusive group membership) of a either a dichotomous or a nominal dependent variable (outcome) based on its relationship with one or more continuous scaled independent variables (predictors). A predictive model consisting of one or more discriminant functions (based on the linear combinations of the predictor variables that provide the best discrimination between the [READ MORE]

So many regression procedures. Confused?

August 11, 2021

Regression is the work-horse of research analytics. It has been around for a long time and it probably will be around for a long time to come. Whether we always realise it or not, most of our analytical tools are in some way or another based on the concept of correlation and regression. Let us look at a few regression-based procedures in the researchers’ toolbox: 1. Simple and multiple linear regression: Applicable if both the single dependent variable (outcome or response variable) and one or many independent variables (predictors) are measured on an interval scale. If we [READ MORE]

Why ANOVA and not multiple t-tests? Why MANOVA and not multiple ANOVA’s, etc.

March 13, 2021

ANOVA reigns over the t-test and the MANOVA reigns over the ANOVA. Why? If we want to compare several predictors with a single outcome variable, we can either do a series of t-tests, or a single factorial ANOVA. Not only is a factorial ANOVA less work, but conducting several t-tests for each predictor separately will result in a higher probability of making Type I errors. In fact, with every single t-test, there is a chance of a Type I error. Conducting several t-tests compounds this probability. In contrast, a single factorial ANOVA controls for this error so that the probability of [READ MORE]

Which test: Compare a single group mean or frequency to a hypothetical / known value or proportion

February 20, 2021

When the research objective is to compare a single group mean or frequency to a hypothetical / known value or proportion (such as an action standard or a norm), we have a choice among different statistical procedures, depending on the following variable characteristics: Number of variables: One dependent variable Examples: Is our mean customer satisfaction score significantly different from the industry average (or action standard) of e.g. 4.6? Is the 54/46 gender proportion in our sample significantly different from the population’s age proportions of 51/49? When [READ MORE]

Repeated Measures ANOVA versus Linear Mixed Models.

January 9, 2021

You want to measure performance of the same individual measured over a period of time (repeated observations) on an interval scale dependant variable, but, which procedure to use? So we are looking for an equivalent of the paired samples t-test, but we want to allow for two or more levels of the categorical variable i.e. pre, during, post. The Repeated Measures ANOVA [SPSS: ANALYZE / GENERAL LINEAR MODEL / REPEATED MEASURES] is simpler to use but sadly its often not as accurate and flexible as using Linear Mixed Models (SPSS: ANALYZE / MIXED MODELS / LINEAR). Reminder that the Linear [READ MORE]

Which test: Quantify the association / relationship / correlation / dependence between two variables

October 12, 2020

When the research objective is to understand the association / relationship / correlation / dependence between two variables, we have a choice among different statistical procedures, depending on the following variable characteristics: Number of variables: One dependent variable and one independent variable (technically we don’t distinguish between a dependent and independent variable in these procedures). Examples: To what extent are two variables related to each other? Is one variable dependent on another variable? When the variables are BINOMIAL / BINARY / [READ MORE]

Which test: Compare TWO INDEPENDENT groups for differences (Unpaired, Unmatched, Different respondent group)

July 1, 2020

When the research objective is to compare two independent groups, which means they are unpaired, unmatched, and thus different respondent groups, we have a choice among different statistical procedures, depending on the following variable characteristics: Number of variables: One dependent variable and one independent categorical variable (two levels or groups) Examples: Are the means / frequencies of two independent groups of respondents (e.g. males vs. females) significantly different on the scores of the dependent variable? When the dependent variable is BINOMIAL / [READ MORE]

Revisiting the basics of data and measurement scales (Part 2)

January 28, 2020

The statistical procedures we choose depend on the type of data we collected with the different types of measurement scales we employed. We should be careful to understand the constructs we measure and the type of scales we employ as this will determine what statistical procedures are appropriate for analysis. This post (in following-up to part 1) is partly based on what S.S. Stevens told us in 1946 (see “Further Reading” below) about data and scales. Please note that this classification is no science as there is a lingering debate about the classification system. [READ MORE]

1 2 »