Statistical Significance

What does "statistical significance" really mean?
In Social Behavioral Sciences it is customary to describe one's finding as statistically significant, when the obtained result is among those that (theoretically) would occur no more than 5 out of 100 (p < .05) times when the only factors operating are the chance variations that occur whenever random samples are drawn.

Procedure Used to Test for Significance
Whenever we perform a significance test, it involves comparing a test value that we have calculated to some critical value for the statistic. It does not matter what type of statistic we are calculating (e.g., a t-statistic, a chi-square statistic, an F-statistic, etc.), the procedure to test for significance is the same.

  1. Decide on the critical alpha level you will use (i.e., the error rate you are willing to accept such as .05 or .01).
  2. Conduct the research.
  3. Calculate the statistic.
  4. Compare the statistic to a critical value obtained from a table (click here for critical values tables for correlation, chi-square, t-test and ANOVA).

If your statistic is higher than the critical value from the table:

  • Your finding is significant.
  • You reject the null hypothesis.
  • The probability is small that the difference or relationship happened by chance, and p is less than the critical alpha level (p < alpha ).

If your statistic is lower than the critical value from the table:

  • Your finding is not significant.
  • You fail to reject the null hypothesis.
  • The probability is high that the difference or relationship happened by chance, and p is greater than the critical alpha level (p > alpha ).

Today we have the advantage of using computer software that can calculate exact probabilities for most test statistics. If you have an exact probability from computer software, simply compare it to your critical alpha level. If the exact probability is less than the critical alpha level, your finding is significant, and if the exact probability is greater than your critical alpha level, your finding is not significant. Using a table is not necessary when you have the exact probability for a statistic.