DetermiNet: A Large-Scale Diagnostic Dataset for Complex
  Visually-Grounded Referencing using Determiners
Clarence Lee M Ganesh Kumar Cheston Tan
State-of-the-art visual grounding models can achieve high detection accuracy,
but they are not designed to distinguish between all objects versus only
certain objects of interest. In natural language, in order to specify a
particular object or set of objects of interest, humans use determiners such as
"my", "either" and "those". Determiners, as an important word class, are a type
of schema in natural language about the reference or quantity of the noun.
Existing grounded referencing datasets place much less emphasis on determiners,
compared to other word classes such as nouns, verbs and adjectives. This makes
it difficult to develop models that understand the full variety and complexity
of object referencing. Thus, we have developed and released the DetermiNet
dataset , which comprises 250,000 synthetically generated images and captions
based on 25 determiners. The task is to predict bounding boxes to identify
objects of interest, constrained by the semantics of the given determiner. We
find that current state-of-the-art visual grounding models do not perform well
on the dataset, highlighting the limitations of existing models on reference
and quantification tasks.