# Idea [[Monte Carlo]] [[Markov chain]] is a computer-driven sampling method ([[computational sampling]]) that allows us to directly sample sequences of values from the [[posterior|posterior distribution]], making it unnecessary to [[obtain analytical solutions for the posterior distribution in conjugate cases|compute analytic solutions]] when solving [[Bayes theorem|Bayes's rule]] in [[Bayesian statistics]]. Using MCMC, [[posterior|posterior distributions]] of any form can be approximated to any desired degree of accuracy (just use more MCMC samples for increased accuracy). The earliest algorithms were [[Gibbs sampling]] and [[Metropolis Hastings algorithm]]. More recent ones include [[Hamiltonian Monte Carlo]] (implemented via [[Stan]]), [[differential evolution|differential evolution MCMC]]. When we run MCMC algorithms, we obtain a sequence of MCMC samples from the [[posterior|posterior distribution]] called [[MCMC chain]]. From [[Lee 2014 - chapter 1 - basics of Bayesian analysis]]: > We recommend the relevant chapters from the following books, listed in order of increasing complexity: Kruschke (2010a), MacKay (2003), Gilks et al. (1996), Ntzoufras (2009), and Gamerman and Lopes (2006). An introductory overview is given in Andrieu, De Freitas, Doucet, and Jordan (2003). You can also browse the internet, and find resources such as http://www.learnbayes.org/. # References - https://m-clark.github.io/docs/ld_mcmc/#markov-chain-monte-carlo - https://learnbayes.org/ - [Twiecki's python tutorial for dummies](https://twiecki.io/blog/2015/11/10/mcmc-sampling/) - [metropolis hastings python from scratch](https://towardsdatascience.com/from-scratch-bayesian-inference-markov-chain-monte-carlo-and-metropolis-hastings-in-python-ef21a29e25a) - https://people.duke.edu/~ccc14/sta-663/MCMC.html - [[van Ravenzwaaij 2018 intro to MCMC sampling]] - https://towardsdatascience.com/monte-carlo-integration-in-python-a71a209d277e - https://www.lbreyer.com/classic.html - [[van Ravenzwaaij 2018 simple intro to MCMC sampling]] - https://chi-feng.github.io/mcmc-demo/app.html