Some Principles
1. After reading [Kass, Wasserman The Selection of Prior Distributions by Formal Rules], I have finally accepted the first principal rule, which I struggled to do so far:
There is an initial phase of data collection when the researcher is ignorant.
Sad but true. Imagine a researcher running in a computer simulation you started. He observes one normal draw in his world, when he is asked about the standard deviation based on one single sample. Clearly truth of his statements will be the result of luck.
2. Second principle is that inference should be non-informative, possibly with accurate or pessimistic frequentist coverage.
3. The third principle is that inference should be invariant over parametrization. (Bayesian inference on p parameter of Bernaulli trials is a good example. It does matter whether we use non-informative, i.e. uniform prior over p, or p square.)
Notes
Principle 1: Necessitates a subjectivist thinking in these situations, i.e. Bayesian or minimax.
Principle 3: Bernardo's reference prior or Jeffreys prior serve as invariant priors for Bayesian inference, but there are many others to choose from.
Note that there is no invariance on how to build a model. I.e. X, or X cube having normal distribution is in the discretion of the model designer. Even if choosing the model is not in our scope, subjectivity lurks in the process of inference one more time.
Conclusion
So far it appears that for doing inference in general, one must adhere to ignorance in some situations, subjectivity, and pessimism (1), aiming for accurate frequentist coverage (2) and the requirement for invariance over parametrization (3).
Frequentist thinking fulfills principles 2 and 3, but fails to handle 1, hence there is space for other inference methods, such as Bayesian.
1. After reading [Kass, Wasserman The Selection of Prior Distributions by Formal Rules], I have finally accepted the first principal rule, which I struggled to do so far:
There is an initial phase of data collection when the researcher is ignorant.
Sad but true. Imagine a researcher running in a computer simulation you started. He observes one normal draw in his world, when he is asked about the standard deviation based on one single sample. Clearly truth of his statements will be the result of luck.
2. Second principle is that inference should be non-informative, possibly with accurate or pessimistic frequentist coverage.
3. The third principle is that inference should be invariant over parametrization. (Bayesian inference on p parameter of Bernaulli trials is a good example. It does matter whether we use non-informative, i.e. uniform prior over p, or p square.)
Notes
Principle 1: Necessitates a subjectivist thinking in these situations, i.e. Bayesian or minimax.
In several cases is it wise to make probabilistic statements using minimax. I.e. assuming that the world has a distribution which is the worst outcome in response to your future actions based on your model. So one should choose a probabilistic model of the world, which minimizes your utility being maximized by your actions.
Principle 3: Bernardo's reference prior or Jeffreys prior serve as invariant priors for Bayesian inference, but there are many others to choose from.
Note that there is no invariance on how to build a model. I.e. X, or X cube having normal distribution is in the discretion of the model designer. Even if choosing the model is not in our scope, subjectivity lurks in the process of inference one more time.
Conclusion
So far it appears that for doing inference in general, one must adhere to ignorance in some situations, subjectivity, and pessimism (1), aiming for accurate frequentist coverage (2) and the requirement for invariance over parametrization (3).
Frequentist thinking fulfills principles 2 and 3, but fails to handle 1, hence there is space for other inference methods, such as Bayesian.
Nincsenek megjegyzések:
Megjegyzés küldése