Recent days I was talking to some friends on the philosophy of analyzing data. I conclude that statistics is not the only way to study data but definitely a very useful way. In my opinion, statistics is the analysis of data using probability model. Such a probabilistic formalism has several merits. In this post, I briefly introduce two nice properties for the statistical model.
First, statistical model for data is always in a simple form. In most cases, the data is full of noises and it is often hard to explain the formation of the noises without using the probability. In statistics, we simply model the generating process by using the notion of probability so that we can easily explain the randomness or the uncertainty of the data. For instance, in regression, our data is pairs of (X,Y) and we want to find the relation between Y and X. Generally, it is hard to find a simple function such that Y=f(X) for all points. However, in statistical model, we use Y = f(X)+noise. This is so-called the signal+noise form. In this model, the function f(x) (also called the regression function) can be modeled as a simple function as the noise is very reasonably distributed (reasonably: in terms of probability). The well-known simple linear regression is to use f(X)=a+bX which is very simple and easy to interpret.
The second advantage is that the statistical model provides a population level analysis. The population level analysis means that we can perform lots of analysis given the true population. For instance, we know that given a specific sample size, the sample mean will differ from the population mean due to the randomness of sampling process. However, if we have full information on the true population (usually we assume what the population looks like), we know the strength of randomness in the sample mean. We can compare this to what we observe in the data. Hence, we can check the reasonableness of our assumptions. This leads to the hypothesis test and model checking (model diagnostic) in statistics and is also widely used in scientific studies.
Statistics also has other nice features such as the asymptotic analysis and uncertainty quantification. I will introduce these characteristics later.