One of the key things about how we use statistics in this course (and throughout the program) is that there are very little actual math calculations involved.
The statistical program SPSS can do all of the statistics calculation for you.
The one mathematical task you need to do is determine for some statistical tests if the result is significant or not significant.
In statistics we use the term p value to determine if a result is significant. SPSS, the stats program, will generate a p value when appropriate. Your task is then to determine if the p value is significant or not significant.
For most tests in the program, and all tests in this course, a significance is determined by the value 0.05.
If the p value is less than 0.05, it is significant. If it is more than 0.05 it is non-significant.
Here's a way to put this mathematically.
if p value ≤ 0.05 it is significant
if p value ≥ 0.05 it is non-significant
This is as hard as the math part of the course gets - simply taking a number and working out if it is bigger or smaller than .05. Let me give you some examples.
[Examples of significant and non-significant]
.89 > .05 = non-significant
.32 < .05 = significant
.02 < .05 = significant
.0001 < .05 = significant
.07 > .05 = non-significant
Determine which of these p values is significant or non-significant.
Answers:
| Significant: | Non-significant: | |
|---|---|---|
| .004729237423742398234 | .99 | .09 |
| .008 | .15 | .06 |
| .0005 | .08 | .05 |
| .04 | .100 | .10 |
| .003 | .009 | .95 |
Traditionally there were three p values of interest: .05, .01, and .001.
p value
And this is how it was traditionally written.
p value in traditional notation
p > .05 (bigger than .05 = not significant)
.01-.05 = p < .05
.001-.01 = p < .01
.00 -.001 = p < .001
The following p values belong to which of the four categories of traditional notation?
P Values
Answers:
Today in most disciplines (including APA style) you should write the exact p value to three decimal places. For example, if p = .032, write p = .032 rather than p < .05. And if p = .008, write p = .008 rather than p < .01.
p value in modern notation
p = .032
There is one exception.
In modern style, if the p value is very small (less than .001), you write p < .001 rather than the actual value.
For example, if p = .0000526, write p < .001.
p value in modern notation
p < .0000526
p < .001
So what does all this mean statistically? If p = .05, it means there is a 5% chance of being wrong and therefore a 95% chance of being right.
What does p value mean?
p = .05
5% wrong
95% right
If p = .03, it means there is a 3% chance of being wrong and a 97% chance of being right.
What does p value mean?
p = .03
3% wrong
97% right
If p = .01, there is a 1% chance of being wrong, therefore a 99% chance of being right.
What does p value mean?
p = .01
1% wrong
99% right
If p = .001, there is a .01% chance of being wrong, and a 99.9% chance of being right.
What does p value mean?
p = .001
.01% wrong
99.9% right
Frequently the more data you have, the more likely small differences may be significant.
The danger of significance
If I have data from 10 participants, males score 5.6, and females score 14.3, this difference may not be significant, even though here the difference in scores appears big.
Suppose I collect more data on males and females. After the scores of 10,000 participants have been collected, males now score 5.7 and females score 5.64.
This difference may be significant even though it appears small. With sufficient scores, observations, or data points, any difference, however small, can be significant. For this reason significance does not equal importance. Something can be significant but not important.
The danger of significance
significance ≠ importance
For example, let's say a machine produces cake mix with 1% too little flour.
The cake mix still works, so neither the consumer nor manufacturer care. The 1% difference was significant but not important.
The danger of significance
significant but not important
It can also be the other way around: something can be important but not significant. A simple test can improve an insurance company's ability to detect fraudulent claims by 1%. The 1% difference may not be significant but is important. It saves the company $4 million per year.
The danger of significance
important but not significant