Categorizing the Visual Environment and Analyzing the Visual Attention
  of Dogs
Shreyas Sundara Raman Madeline H. Pelgrim Daphna Buchsbaum Thomas Serre
Dogs have a unique evolutionary relationship with humans and serve many
important roles e.g. search and rescue, blind assistance, emotional support.
However, few datasets exist to categorize visual features and objects available
to dogs, as well as how dogs direct their visual attention within their
environment. We collect and study a dataset with over 11,698 gazes to
categorize the objects available to be gazed at by 11 dogs in everyday outdoor
environments i.e. a walk around a college campus and urban area. We explore the
availability of these object categories and the visual attention of dogs over
these categories using a head mounted eye tracking apparatus. A small portion
(approx. 600 images or < 20% of total dataset) of the collected data is used to
fine tune a MaskRCNN for the novel image domain to segment objects present in
the scene, enabling further statistical analysis on the visual gaze tendencies
of dogs. The MaskRCNN, with eye tracking apparatus, serves as an end to end
model for automatically classifying the visual fixations of dogs. The fine
tuned MaskRCNN performs far better than chance. There are few individual
differences between the 11 dogs and we observe greater visual fixations on
buses, plants, pavement, and construction equipment. This work takes a step
towards understanding visual behavior of dogs and their interaction with the
physical world.