To improve this convergence, the Metropolis–Hastings algorithm was proposed. Generally, Monte Carlo methods' idea is to use importance sampling to improve the convergence of the estimator Q ¯ ¯ N by sampling states according to an arbitrary distribution π π ( r ) , and use the appropriate estimator: