On Transferability of Driver Observation Models from Simulated to Real
  Environments in Autonomous Cars
Walter Morales-Alvarez Novel Certad Alina Roitberg Rainer Stiefelhagen Cristina Olaverri-Monreal
For driver observation frameworks, clean datasets collected in controlled
simulated environments often serve as the initial training ground. Yet, when
deployed under real driving conditions, such simulator-trained models quickly
face the problem of distributional shifts brought about by changing
illumination, car model, variations in subject appearances, sensor
discrepancies, and other environmental alterations.
  This paper investigates the viability of transferring video-based driver
observation models from simulation to real-world scenarios in autonomous
vehicles, given the frequent use of simulation data in this domain due to
safety issues. To achieve this, we record a dataset featuring actual autonomous
driving conditions and involving seven participants engaged in highly
distracting secondary activities. To enable direct SIM to REAL transfer, our
dataset was designed in accordance with an existing large-scale simulator
dataset used as the training source. We utilize the Inflated 3D ConvNet (I3D)
model, a popular choice for driver observation, with Gradient-weighted Class
Activation Mapping (Grad-CAM) for detailed analysis of model decision-making.
Though the simulator-based model clearly surpasses the random baseline, its
recognition quality diminishes, with average accuracy dropping from 85.7% to
46.6%. We also observe strong variations across different behavior classes.
This underscores the challenges of model transferability, facilitating our
research of more robust driver observation systems capable of dealing with real
driving conditions.