Probabilistic Precision and Recall Towards Reliable Evaluation of
  Generative Models
Dogyun Park Suhyun Kim
Assessing the fidelity and diversity of the generative model is a difficult
but important issue for technological advancement. So, recent papers have
introduced k-Nearest Neighbor ($k$NN) based precision-recall metrics to break
down the statistical distance into fidelity and diversity. While they provide
an intuitive method, we thoroughly analyze these metrics and identify
oversimplified assumptions and undesirable properties of kNN that result in
unreliable evaluation, such as susceptibility to outliers and insensitivity to
distributional changes. Thus, we propose novel metrics, P-precision and
P-recall (PP\&PR), based on a probabilistic approach that address the problems.
Through extensive investigations on toy experiments and state-of-the-art
generative models, we show that our PP\&PR provide more reliable estimates for
comparing fidelity and diversity than the existing metrics. The codes are
available at \url{https://github.com/kdst-team/Probablistic_precision_recall}.