Beyond any debate, what is certain is that Bayesian Inference allows us to optimally update our beliefs given a model and a data set. What more could you ask for?
Some initial bibliography
- Lee M.D. and Wagenmakers E.J. Bayesian Cognitive Modeling: A Practical Course
- Richard McElreath. Statistical Rethinking: A Bayesian Course with Examples in R and Stan
- Andrew Gelman et al. Bayesian Data Analysis
Evidence in favor of a scientific theory
In a 1935 paper and in his book Theory of Probability, Jeffreys developed a methodology for quantify the evidence in favor of a scientific theory. Bayesian model selection automatically encodes a preference for simpler, more constrained models. Many of the “machine learning” regularization techniques can be seen as an approximation to Bayesian inference.