Describing Differences

Chi-square (χ²) Test of Independence

April 22, 2023

BRIEF DESCRIPTION: Whereas the One-sample Chi-square (χ²) goodness-of-fit test compares our sample distribution (observed frequencies) of a single variable with a known pre-defined distribution (expected frequencies) such as the population distribution, normal distribution, or poisson distribution, to test for the significance of deviation, the Chi-square (χ²) Test of Independence compares two categorical variables in a cross-tabulation fashion to determine group differences or degree of association (or non-association i.e. independence). Chi-square (χ²) is a [READ MORE]

One-Sample Chi-square (χ²) goodness-of-fit test

February 9, 2023

BRIEF DESCRIPTION: The Chi-square (χ²) goodness-of-fit test is a univariate measure for categorical scaled data, such as dichotomous, nominal, or ordinal data. It tests whether the variable’s observed frequencies differ significantly from a set of expected frequencies. For example, is our observed sample’s age distribution of 20%, 40%, 40% significantly different from what we expect (e.g. the population age distribution) of 30%, 30%, 40%. Chi-square (χ²) is a non-parametric procedure. SIMILAR STATISTICAL PROCEDURES: Binomial goodness-of-fit (for binary data) [READ MORE]

Which test: Compare TWO DEPENDENT groups (Paired, Matched, Same respondent groups)

August 3, 2022

When the research objective is to compare two dependent groups, which means they are paired, matched, and thus the same respondent groups in a pre- post-test, we have a choice among different statistical procedures, depending on the following variable characteristics: Number of variables: One dependent variable and one independent categorical variable (two levels or groups) Examples: Are the means / frequencies (on the dependent variable) of the same respondents over two different time periods significantly different such as in a pre- post test? Are [READ MORE]

Which test: Compare MORE THAN TWO DEPENDENT groups (Paired, Matched, Same respondent groups)

July 20, 2022

When the research objective is to compare more than two dependent groups, which means they are paired, matched, and thus the same respondent groups in a pre- post-test, we have a choice among different statistical procedures, depending on the following variable characteristics: Number of variables: [Unless where otherwise indicated] One dependent variable and one independent categorical variable (more than two levels or groups) Examples: Are the means / frequencies (on the dependent variable) of the same respondents over more than two different time periods significantly [READ MORE]

Which test: Compare MORE THAN TWO INDEPENDENT groups (Unpaired, Unmatched, Different respondent groups)

June 12, 2022

When the research objective is to compare more than two independent groups, which means they are unpaired, unmatched, and thus different respondent groups, we have a choice among different statistical procedures, depending on the following variable characteristics: Number of variables: [Unless where otherwise indicated] One dependent variable and one independent categorical variable (more than two levels) Examples: Are the means / frequencies of more than two independent groups of respondents significantly different? When the dependent variable is BINOMIAL / BINARY / [READ MORE]

Analysis of Covariance (ANCOVA)

May 3, 2022

BRIEF DESCRIPTION The Analysis of Covariance (ANCOVA) follows the same procedures as the ANOVA except for the addition of an exogenous variable (referred to as a covariate) as an independent variable. The ANCOVA procedure is quite straightforward: It uses regression to determine if the covariate can predict the dependent variable and then does a test of differences (ANOVA) of the residuals among the groups. If there remains a significant difference among the groups, it signifies a significant difference between the dependent variable and the predictors after the effect of the [READ MORE]

Why ANOVA and not multiple t-tests? Why MANOVA and not multiple ANOVA’s, etc.

March 13, 2021

ANOVA reigns over the t-test and the MANOVA reigns over the ANOVA. Why? If we want to compare several predictors with a single outcome variable, we can either do a series of t-tests, or a single factorial ANOVA. Not only is a factorial ANOVA less work, but conducting several t-tests for each predictor separately will result in a higher probability of making Type I errors. In fact, with every single t-test, there is a chance of a Type I error. Conducting several t-tests compounds this probability. In contrast, a single factorial ANOVA controls for this error so that the probability of [READ MORE]

Which test: Compare a single group mean or frequency to a hypothetical / known value or proportion

February 20, 2021

When the research objective is to compare a single group mean or frequency to a hypothetical / known value or proportion (such as an action standard or a norm), we have a choice among different statistical procedures, depending on the following variable characteristics: Number of variables: One dependent variable Examples: Is our mean customer satisfaction score significantly different from the industry average (or action standard) of e.g. 4.6? Is the 54/46 gender proportion in our sample significantly different from the population’s age proportions of 51/49? When [READ MORE]

Which test: Compare TWO INDEPENDENT groups for differences (Unpaired, Unmatched, Different respondent group)

July 1, 2020

When the research objective is to compare two independent groups, which means they are unpaired, unmatched, and thus different respondent groups, we have a choice among different statistical procedures, depending on the following variable characteristics: Number of variables: One dependent variable and one independent categorical variable (two levels or groups) Examples: Are the means / frequencies of two independent groups of respondents (e.g. males vs. females) significantly different on the scores of the dependent variable? When the dependent variable is BINOMIAL / [READ MORE]

Measuring effect size and statistical power analysis

March 3, 2020

Effect size measures are crucial to establish practical significance, in addition to statistical significance. Please read the post “Tests of Significant are dangerous and can be very misleading” to better appreciate the importance of practical significance. Normally we only consider differences and associations from a statistical significance point of view and report at what level e.g. p<.001 we reject the null hypothesis (H0) and accept that there is a difference or association (note that we can never “accept the alternative hypothesis (H1)” – see the [READ MORE]

One-way (Independent) ANOVA

January 13, 2020

BRIEF DESCRIPTION: The One-way ANOVA is an extension of the Two-independent sample t-test as it compares the observed mean on the dependent variable among more than two groups as defined by the independent variable. For example, is the mean customer satisfaction score (on the dependent variable) significantly different among three customer groups: adult men, adult women, and children (on the independent variable). In addition to expressing group differences on the dependent variable, we can also express the findings in terms of relationship or association, e.g. “Age [READ MORE]

One-Sample Kolmogorov-Smirnov goodness-of-fit test

October 2, 2018

BRIEF DESCRIPTION: The Kolmogorov-Smirnov (K-S) test is a goodness-of-fit measure for continuous scaled data. It tests whether the observations could reasonably have come from the specified distribution, such as the normal distribution (or poisson, uniform, or exponential distribution, etc.), so it most frequently is used to test for the assumption of univariate normality. The categorical data counterpart is the Chi-Square (χ²) goodness-of-fit test. The K-S test is a non-parametric procedure. SIMILAR STATISTICAL PROCEDURES: Adjusted Kolmogorov-Smirnov Lilliefors test (null [READ MORE]

1 2 »