When the posterior has a known distribution, as in Analytic Approach for Binomial Data, it can be relatively easy to make predictions, estimate an HDI and create a random sample. Even when this is not the case, we can often use the grid approach to accomplish our objectives (see Creating a Grid). Unfortunately, sometimes neither of these approaches is applicable. On this webpage, we demonstrate how to use a type of simulation, based on Markov chains, to achieve our objectives.
In a Markov chain process, there are a set of states and we progress from one state to another based on a fixed probability. Figure 1 displays a Markov chain with three states. E.g. the probability of transition from state C to state A is .3, from C to B is .2 and from C to C is .5, which sum up to 1 as expected.
Figure 1 – Markov Chain transition diagram
The important characteristic of a Markov chain is that at any stage the next state is only dependent on the current state and not on the previous states; in this sense it is memoryless.
Topics
References
Lee, P. M. (2012) Bayesian statistics an introduction. 4th Ed. Wiley
https://www.wiley.com/en-us/Bayesian+Statistics%3A+An+Introduction%2C+4th+Edition-p-9781118332573
Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., Rubin, D. B. (2014) Bayesian data analysis, 3rd Ed. CRC Press
https://statisticalsupportandresearch.files.wordpress.com/2017/11/bayesian_data_analysis.pdf
Marin, J-M and Robert, C. R. (2014) Bayesian essentials with R. 2nd Ed. Springer
https://www.springer.com/gp/book/9781461486862
Jordan, M. (2010) Bayesian modeling and inference. Course notes
https://people.eecs.berkeley.edu/~jordan/courses/260-spring10/lectures/index.html
Reich, B. J., Ghosh, S. K. (2019) Bayesian statistics methods. CRC Press