Many people asked me “Is statistics a science?”
To me, statistics belongs to the “formal science”, which is a discipline of science.
Simply put, the science is the knowledge built by a strict process, called the scientific approach and this knowledge is open to test. Usually, the science we refer to is the physical science or life science which concerns with the knowledge to physical objects or living creatures.
The formal science is the knowledge about the “sign systems” (formal system) like logic and mathematics. In the formal systems, everything is well-defined so that the inference can be made exact; there is no exception. We begin with a structure with some well-defined operations and we derive the implication/result to this structure based on the operation defined. For example, the number system is a formal system and an inference like 1+1 is exactly 2 with no exception.
The conclusion of our derivation gives us the knowledge to this formal system. In the system of probability, the law of large number tells us that the sample mean of independent, identical random variables will converges to the true population mean. Without this knowledge, we have no idea what will happen to the sample mean as we have infinite number of observations. The law of large number (in fact, it is a theorem) is a knowledge to the formal system derived by the axioms of probability theory.
The formal science plays a key role in other disciplines of science by providing well-defined models to describe the world. For instance, Einstein’s mass-energy equivalence states “E=mc^2” which gives a model that links three distinct physical quantities, the energy E, the mass m and the speed of light c. If we do not define the notation of multiplication or equality (imagine the life in thousands of years ago), we can never acquire this knowledge about mass and energy.
As a member of formal science, statistical theory provides many help in scientific research. Take the t-distribution as an example, without the knowledge of t-distribution, the commonly used t-test has no guarantee on its effectiveness. Thus, scientific studies that make conclusion by the t-test will be questioned about their discovery.
Thus, statistics is of course part of the science. But the system we contribute knowledge to is not the usual physical system but a formal system.