Reporting statistics in client reports – A few thoughts

A few thoughts on reporting statistical findings (and no, this is not academical jargon – its is the smart way to report the findings to your clients):

1. As marketing researchers we generally follow the APA (American Psychological Association) style of reporting so when reporting e.g. significance (p‘<‘.05), never put a ‘0’ in front of the decimal (i.e. p‘<‘.05) if the number cannot be greater than 1.00. So p‘<‘.05 is correct, and p‘<‘0.05 is not the proper reporting style. Statistical output of significance testing, effect size (e.g. Cohen’s d), and statistical power all vary between 0 and 1, and as such all reported without a 0.

2. The value of p can never be zero. It is asymptotic, so even if the value in the output shows p=.000 (because the real value could be .00000024…), report it as p‘<‘.0001 because we know that the value is somewhere past that .000. We just don’t know if it is 1 more decimal or 1,000 more decimal places.

3. In hypothesis testing, note that if you fail to reject the null hypothesis that does NOT mean you can accept the null hypothesis as you can never prove the null hypothesis is true (so you can never accept the null hypothesis). If you fail to reject the null hypothesis by a small margin, do not report it as “almost significant”, and likewise, if you reject the null with a very small p-value (e.g. p‘<‘.001), do not report it as “highly significant” as in hypothesis testing, the findings are either significant or not significant, and you either reject or fail to reject the null. So make sure to speak the right language.

4. In several posts here on IntroSpective Mode, I have placed focus on the importance of practical significance in additional to the commonly used statistical significance (and its dangers) by calculating and reporting on effect size and statistical power. Here is an example on how to report the findings:

• “The difference is significant at p‘<‘.05, with an effect size of 0.69 which is a moderate (and typical effect), and the power is 0.90 (well above Cohen’s suggested 0.80) so the probability that I make any error type is remote. So I am very confident in saying there is a significant, moderate, and correctly identified difference between my two test groups.”

5. Reporting z-scores the APA-way:
Example: “Standardised z-scores were computed for the raw customer satisfaction scores. For the raw score 9.2,  z = 2.05. This z-score tells us that the specific score was well above the average customer satisfaction scores.”