Hypothesis test is an important tool in statistics and is commonly used in scientific research. I just come up with an idea that the hypothesis test is a generalization of proof by contradiction.

The basic setting for hypothesis test is that you have a null hypothesis and an alternative hypothesis. You construct a test statistics and pick a significance level. As the test statistics is above a threshold(note*) that is derived from the significance level and the distribution of test statistics under null hypothesis, you reject the null hypothesis.

A key idea is that the distribution of test statistics is calculated under null hypothesis. This shows a link to the proof by contradiction.

How come? Well, recall that as we perform prove by contradiction, we assume ‘something’ we want to prove that it contradicts to itself.

In the hypothesis test, we assume the ‘null hypothesis’ being true and we want to prove that it contradicts to itself. However, in probabilistic model, our data is randomly sampled. Even we’re under null hypothesis, everything is possible. We cannot use induction or a strong logical statement to show the null hypothesis contradicts to itself. However, we still want to use the way of reasoning in proof of contradiction. Hence, we use a ‘measure’ of contradiction based on data to carry out the similar reasoning.

This measure of contradiction is the test statistics compared to its behavior under null hypothesis (note**). If the test statistic shows common result as it should be in null hypothesis, then the measure of contradiction is small. That is, there is no ‘significant’ contradiction between data and the null hypothesis. In contrast, if the test statistics is very far from what it should be under null hypothesis, this shows that the measure of contradiction is high. Now as the measure of contradiction is larger than our tolerance(significance level), then we reject null hypothesis like what we conclude in proof of contradiction.

Since the hypothesis test is a generalization of proof by contradiction, a necessary condition for this reasoning to hold is that the null hypothesis and alternative hypothesis have to be compliment. Otherwise, it is possible that rejecting null hypothesis does not imply the alternative should be accepted.

In summary, we see that what we do in hypothesis test is in the similar way of proof by contradiction. The hypothesis test can be viewed as a generalization of proof by contradiction to the probabilistic model. The also explains why hypothesis test is so important in science: it allows us to ‘proof’ something based on data.

Note*: The large test statistics does not necessarily imply that we should reject null hypothesis. This really depends on the distribution of test statistics under null hypothesis. But usually in most test statistics, the larger test statistics, the more evidence against null hypothesis.

Note**: In fact, 1-(p value) is a better choice of measure of contradiction since in note*, we know that large test statistics may not imply stronger evidence against null hypothesis.

Great Post!