Open-Domain Dialogue Quality Evaluation: Deriving Nugget-level Scores
  from Turn-level Scores
Rikiya Takehi Akihisa Watanabe Tetsuya Sakai
Existing dialogue quality evaluation systems can return a score for a given
system turn from a particular viewpoint, e.g., engagingness. However, to
improve dialogue systems by locating exactly where in a system turn potential
problems lie, a more fine-grained evaluation may be necessary. We therefore
propose an evaluation approach where a turn is decomposed into nuggets (i.e.,
expressions associated with a dialogue act), and nugget-level evaluation is
enabled by leveraging an existing turn-level evaluation system. We demonstrate
the potential effectiveness of our evaluation method through a case study.