Towards Interpretability in Audio and Visual Affective Machine Learning:
  A Review
David S. Johnson Olya Hakobyan Hanna Drimalla
Machine learning is frequently used in affective computing, but presents
challenges due the opacity of state-of-the-art machine learning methods.
Because of the impact affective machine learning systems may have on an
individual's life, it is important that models be made transparent to detect
and mitigate biased decision making. In this regard, affective machine learning
could benefit from the recent advancements in explainable artificial
intelligence (XAI) research. We perform a structured literature review to
examine the use of interpretability in the context of affective machine
learning. We focus on studies using audio, visual, or audiovisual data for
model training and identified 29 research articles. Our findings show an
emergence of the use of interpretability methods in the last five years.
However, their use is currently limited regarding the range of methods used,
the depth of evaluations, and the consideration of use-cases. We outline the
main gaps in the research and provide recommendations for researchers that aim
to implement interpretable methods for affective machine learning.