Tests of significance

Tests of significance (or significance testing) are the mathematical testing or checking of data to see how high or low the probability is that it would be different from the characteristics or views of the total population.

In other words, is the difference between one value or another true and meaningful, or a fluke or one-off? When doing such a test in research we are looking for a high confidence level (often set at 95%) which means there is a very low chance (in this example, 5% or less) that there were any variables or influencers that changed this outcome from being a typical response. Using a 95% confidence level is like saying if we completed this study 100 times, 95 of those studies would result in the same outcome.

Tests of significance can be calculated and displayed in different ways, but very often in research it is column comparison, with each column being attributed to a letter. For example, your columns might be children in a household and therefore labelled (a) 0, (b) 1, (c) 2, (d) 3, (e) 4 and (f) 5 or more. An example question might be ‘How much does your household spend on cordial each week?’ – if a significantly higher percentage of those with 0 children (column a) in their household say they spend £5 or less a week than all the other columns, the column would display the percentage who chose that option along with the letters of the columns it is significantly different to, for example, 85% (b, c, d, e, f).

It should be noted however that a figure won’t always be shown as significantly different to all other values or columns, in the example above those with 0 children (column a) might actually only be significantly different to those with 3, 4 and 5 or more children in the household and would instead be displayed as: 85% (d, e, f).

It is also important to understand that significance testing is intended as a guide, and although numbers that aren’t statistically significant should be treated with more caution, this does not mark them as wrong or inaccurate and they should not be disregarded.