D3: Data Diversity Design for Systematic Generalization in Visual
  Question Answering
Amir Rahimi Vanessa D'Amario Moyuru Yamada Kentaro Takemoto Tomotake Sasaki Xavier Boix
Systematic generalization is a crucial aspect of intelligence, which refers
to the ability to generalize to novel tasks by combining known subtasks and
concepts. One critical factor that has been shown to influence systematic
generalization is the diversity of training data. However, diversity can be
defined in various ways, as data have many factors of variation. A more
granular understanding of how different aspects of data diversity affect
systematic generalization is lacking. We present new evidence in the problem of
Visual Question Answering (VQA) that reveals that the diversity of simple tasks
(i.e. tasks formed by a few subtasks and concepts) plays a key role in
achieving systematic generalization. This implies that it may not be essential
to gather a large and varied number of complex tasks, which could be costly to
obtain. We demonstrate that this result is independent of the similarity
between the training and testing data and applies to well-known families of
neural network architectures for VQA (i.e. monolithic architectures and neural
module networks). Additionally, we observe that neural module networks leverage
all forms of data diversity we evaluated, while monolithic architectures
require more extensive amounts of data to do so. These findings provide a first
step towards understanding the interactions between data diversity design,
neural network architectures, and systematic generalization capabilities.